Are We Ready for AI-Generated Code?

Autocompleted code is convenient and quick, but it may expose your organization to security and compliance risks.

In recent months, we’ve marveled at the quality of computer-generated faces, cat pictures, videos, essays, and even art. Artificial intelligence (AI) and machine learning (ML) have also quietly slipped into software development, with tools like GitHub Copilot, Tabnine, Polycode, and others taking the logical next step of putting existing code autocomplete functionality on AI steroids. Unlike cat pics, though, the origin, quality, and security of application code can have wide-reaching implications — and at least for security, research shows that the risk is real.

Prior academic research has already shown that GitHub Copilot often generates code with security vulnerabilities. More recently, hands-on analysis from Invicti security engineer Kadir Arslan showed that insecure code suggestions are still the rule rather than the exception with Copilot. Arslan found that suggestions for many common tasks included only the absolute bare bones, often taking the most basic and least secure route, and that accepting them without modification could result in functional but vulnerable applications.

A tool like Copilot is (by design) autocompletion turned up a notch, trained on open source code to suggest snippets that could be relevant in a similar context. This makes the quality and security of suggestions closely tied to the quality and security of the training set. So the bigger questions are not about Copilot or any other specific tool but about AI-generated software code in general.

It’s reasonable to assume Copilot is only the tip of the spear and that similar generators will become commonplace in the years ahead. This means we, the technology industry, need to start asking how such code is being generated, how it’s used, and who will take responsibility when things go wrong.

Read more