GitHub has published research on the growth and impact of AI on software development. Among their findings is that developers write code “55% faster” when using the GitHub Copilot code automation tool.
But this finding doesn’t tell the whole story. Is this code high-quality? Is this code necessary? Is the code contributing to the long-term value of our applications? Does the code contribute to the operation of the application in a clear and concise manner? In other words, is Copilot (and similar tools) helping or hurting our overall application complexity?
To help answer that question, I took a look at a newly released report by GitClear, which shows the impact of automated code generation on the overall quality of the code we are managing.
I found this report illuminating. It showed, among other things, that there are disturbing trends in the maintainability of code generated by automated code generation tools and that the increased use of AI code generation has contributed to an overall decrease in code quality over the last several years.
In this report, GitClear collected 153 million lines of changed code authored during the four years between January 2020 and December 2023. According to the report, this is the largest database of highly structured code change data known to exist. This data set showed some concerns.
The first concern observed in this data was an increase in code churn. In our context, code churn is defined as code that has been updated or reverted within two weeks of its original authorship. Code churn is a great indicator of code complexity since it indicates code instability in an application. According to this data, code churn is projected to double in 2024 compared to the pre-AI 2021 baseline. More code is being created and thrown away than ever before.
Additionally, there has been a large increase in the amount of copy/pasted code used in applications in the AI years than previously observed. DRY—Don’t Repeat Yourself—is a term used by software developers to describe the process of leveraging and reusing code to reduce the amount of unnecessarily repeated code. Unnecessarily repeated code is known to create support challenges, increase software defects, and decrease the understandability of an application. Non-DRY coding practices are a large contributor to overall application complexity. Software being developed in the AI generation years is less DRY than code created before AI generation was readily available.
In other words, code complexity and the support costs associated with complex code have increased in recent years in large part due to the proliferation of AI-generated code use.
When a developer is given access to quick and easy code suggestions, it becomes a strong temptation to simply add the code to the application, which discourages checking for reuse or refinement opportunities. Developers, who are typically under heavy schedule pressure, will often tend to use the easiest route to a solution rather than stepping back and looking for the best route. This is just basic human nature and is driven by the business and product pressures associated with most project schedules in development organizations.
Yet, this pattern often encourages software developers to use code that they understand less completely and may not fully comprehend the nuances and impact of that code on the application as a whole.
Does this mean that AI-generated code creation is all bad?
No.
However, using AI-generated code needs to be balanced with the expertise of humans who need to manage the code that’s created. Only when AI-generated code is used in proportion with other sources of coding inspiration can an appropriate balance be created that creates healthy, long-lived, quality application code bases.
Like many modern tools, they are best used in moderation and not in excess. AI-generated code is not a solution to our insatiable need for more software. Instead, it’s a tool that can assist human software developers in building better, higher-quality systems. But this goal will only occur if AI generation is used responsibly and with appropriate moderation.
Interested in learning more? I recommend reading the entire GitClear report.
