GitHub Copilot Just Made Itself Even More Valuable to Developers
Improved code suggestion quality, personalization, and security issue identification
It is hard to overstate how much AI-based code generation tools are changing the software development process. Based on OpenAI Codex, Microsoft’s GitHub Copilot launched as a productivity tool for developers in a closed beta in 2021 and as general availability last year. Data from Microsoft suggests that developers are as much as 55% more productive when using Copilot.
Survey results released by Microsoft in September showed that developers generally had higher overall job satisfaction when using Copilot in addition to productivity gains. It seems clear that productivity gains, particularly for automating repetitive tasks, directly led to higher satisfaction. This is a delightful byproduct of the new productivity revolution in our midst.
Making Copilot Better
Synthedia has learned that GitHub is now updating the underlying code-generation model weekly. Last week, the company announced some significant upgrades. In a blog post by GitHub CEO Thomas Dohmke, he commented:
Back in June 2022, we reported that GitHub Copilot was already generating 27% of developers’ code. Today, we’re seeing this happen more and more with an average of 46% of code being built using GitHub Copilot across all programming languages, and 61% among developers using Java.
Three key updates to Copilot include:
Fill-in-the Middle (FIM) allows coders now to enter a code prefix and a common suffix, and Copilot will suggest or “fill-in” appropriate code in between. The earlier version of Copilot was based on the prefix only, and the new feature should improve the code acceptance rate given the added context.
A new Visual Studio feature enables customized code suggestions based on a developer’s usage behavior. GitHub says this feature has already reduced unwanted suggestions by 4.5%, and it will improve further in the recent model update.
Filtering security vulnerabilities is likely the most important upgrade. The company says the new model includes “an AI-based vulnerability prevention system that blocks insecure coding patterns in real-time to make GitHub Copilot suggestions more secure. Our model targets the most common vulnerable coding patterns, including hardcoded credentials, SQL injections, and path injections.”
Making Software Less Vulnerable
Security vulnerability filtering strikes me as the most important feature in this release. FIM and developer-level context customization can help with the productivity value proposition. However, we should also be wary of the “do no harm” objective when implementing new AI-driven systems. No one wants productivity increases to be offset by new security vulnerabilities.
There is another benefit here as well. Security vulnerability filtering is not just about reducing potential harm introduced by AI-based prediction models. Developers make these very same errors in code vulnerability today. Sometimes human developers make these errors due to carelessness, while in other instances, they are simply unaware of new vulnerability discoveries and how best practices have changed.
Discovering New Value
Code generators are not inventing this phenomenon, but they may help reduce it. You are more likely to create security vulnerability filters and enforce more secure code development through a centralized tool than by educating every developer. Maybe code review software would catch these issues later in the development pipeline, but it is always better to eliminate them as early as possible.
The result may be that as more developers employ Copilot, the incidence of security vulnerabilities in new software development will decrease. Now, that would be a delightful byproduct of new AI tools. It is also another example of how new technologies often introduce unexpected benefits.
