When AI writes the code, what happens to quality?

Generative AI is changing software engineering quickly. Developers can now generate scripts, components and substantial parts of automation flows in seconds, removing much of the manual work that previously took significant time.

This is clearly useful, but faster code generation doesn’t necessarily mean faster or better software delivery. In large enterprise environments, writing the code is only one part of the challenge. Much of the difficult work lies in understanding business requirements, anticipating edge cases, protecting revenue and knowing how a change will behave across a complex landscape of interconnected systems.

As more code is generated with AI, these questions become more important, not less. The real issue is whether the generated solution will reliably and securely deliver the intended business outcome once it becomes part of a much larger environment.

Faster cars need better brakes

There is a saying in motorsport that brakes don’t just exist to slow you down; they give you the confidence to drive faster. A Formula 1 driver can push a car to very high speeds because there is confidence in the engineering of the car as a whole, including the systems that allow it to stop safely.

I think the analogy works well for Quality Assurance. When AI allows development teams to produce code much faster, QA shouldn’t simply become a final checkpoint that determines whether something can be released. Its more valuable role is to create the conditions in which teams can move quickly while maintaining confidence in the outcome.

That still involves identifying defects, gaps and unintended behavior before they reach production, but it also means providing visibility into quality across interconnected systems and working with developers, business stakeholders and subject matter experts to understand scenarios that may not have been considered during development.

In that sense, QA combines several functions. It provides the safeguards that allow teams to move faster, gives us information about how our systems are behaving and helps us test what happens when conditions differ from the expected path. As the speed of development increases, all three become more important.

AI provides more horsepower. We need the engineering around it to keep pace.

When technically correct isn’t enough

At Arelion, our software landscape is complex because the business and network it supports are complex. Billing platforms, network provisioning systems, customer portals and other applications interact with each other, which means a change that is technically correct within one system may still have unintended consequences elsewhere.

AI doesn’t inherently understand the specific business processes, operational constraints, customer commitments and system dependencies within that environment. It can work with the context we provide, but knowing which context matters requires knowledge of the business and of how our systems behave in practice.

This has some important consequences. An AI-generated solution may meet the technical requirements it has been given while still missing an important business scenario. A problem identified during design or testing is also generally much easier and less costly to correct than an architectural flaw discovered after deployment. And a small change in one system can have an unexpected effect on another system, process or customer experience.

The requirements around security, data and compliance also remain exactly as important regardless of whether the underlying code was written manually or generated by an AI tool. Ultimately, responsibility remains with us. AI doesn’t sign our SLAs or take responsibility when a change has an unexpected effect on a customer.

Context matters as much as code

AI is increasingly good at generating code, suggesting solutions and identifying patterns from large amounts of information. What it doesn’t automatically have is the contextual knowledge accumulated by the people who work with our customers, systems and operational processes every day.

An LLM doesn’t inherently know how a particular global customer expects its services to behave. Nor does it understand the trade-offs an operational team may have to make during a critical network event at two in the morning, or why an apparently unusual exception is important to a particular business process. That knowledge is distributed across business experts, operational teams, architects, developers and others who work closely with our systems and customers.

QA teams hold part of that knowledge as well. Through testing and production experience, they build up an understanding of recurring defects, failed assumptions, business edge cases, integration risks and operational exceptions. Historically, much of this knowledge has been applied during validation. With AI-assisted development, there is an opportunity to use it considerably earlier.

The quality of AI-generated code depends heavily on the context provided to the model. If developers and AI coding agents can draw on what we have learned from previous defects, production issues, business exceptions and difficult integration scenarios, some problems can potentially be avoided before they are introduced into the code.

This changes where QA can contribute. Rather than only discovering issues during testing, QA can work with developers and business experts during solution design to explore real-world scenarios, identify dependencies that aren’t immediately obvious and understand how changes in one area could affect connected systems and processes.

There is also a more practical quality-enablement role. Regression suites, automated checks, monitoring and other quality guardrails can make rapid development more sustainable by giving teams continuous feedback on the impact of their changes. Increasingly, QA can also help determine what business, operational and quality context should be available to AI-powered development tools, allowing lessons from previous defects and production issues to influence future implementations.

To me, this is one of the more interesting consequences of bringing generative AI into software engineering: knowledge that was traditionally applied when testing a finished solution can increasingly inform how that solution is created in the first place.

Quality in AI-assisted development

At Arelion, we’re already seeing how AI can help teams automate repetitive tasks and reduce some of the effort involved in building and maintaining software. Those capabilities will continue to develop, and there is every reason to make use of them.

But our measure of success cannot simply be how quickly code is produced. A feature generated in seconds still has to work correctly across connected systems, support the relevant business processes and behave predictably when circumstances differ from the expected path. It also needs to meet the same requirements for security, reliability and compliance as any other change.

For that reason, I don’t see generative AI as reducing the importance of Quality Assurance. I see it changing where QA can add the most value. Quality is becoming less about a final validation step and more about bringing knowledge of business processes, operational realities, integration risks and previous failures into development earlier.

That knowledge can help developers make better decisions, but it can also improve the context we provide to AI itself. Used this way, QA doesn’t simply help us find problems faster. It can help prevent some of them from being introduced in the first place.

Arelion operates critical internet backbone infrastructure, where reliability is fundamental. The software and platforms supporting our network, customers, employees and operations must meet the same expectations. AI gives us an opportunity to build and change those systems faster, but making sure those changes work as intended remains a shared engineering responsibility.

 

David Brodziak, Quality Engineer