If AI coding is lowering your code quality, you’re not managing quality right

Iouri KhramtsovSubscribeSign inIf AI coding is lowering your code quality, you’re not managing quality rightIouri KhramtsovSep 14, 2026ShareOne common take on the coding agents that I see goes something like this: “Sure, AI helps you output more code, but won’t the quality suffer?”It certainly will if you just blindly merge the PRs and send them off to prod. But if you take a thoughtful, layered approach to managing quality, I find that it’s possible to not just keep the number of bugs stable but actually reduce it—while still increasing the output by 2-2x.Many of these defensive layers are pretty much the same as before Claude/Copilot/Codex/etc. (though they’re made easier now by AI), while others are new. Here’s a defensive setup that I’ve seen successfully used in practice, both on my team and elsewhere.Layer 1: Getting the requirements rightOne of the biggest surprises after I started using spec-driven development was the drop in bugs in the freshly written code. Before spec-driven development, when building, e.g., a new feature, the teams I was on often spent up to a third of the total effort on the post-development “polishing,” i.e., discovering and fixing various bugs. Many of these bugs occurred either because we didn’t foresee certain interactions and edge cases, or because the developer was tired that day and didn’t put in enough thought, or because the designer or PM didn’t think through certain scenarios. Some of these bugs were missed and ended up in production.After I started using spec-driven development, the number of these bugs in my code sharply dropped, and I’ve seen the same drop for some (but not all) of my teammates. As far as I can tell, the main cause of this drop is one specific step in the process: having the AI review the requirements or the tech design and find any gaps, edge cases, unexpected interactions with the existing code, or other similar problems.The AI doesn’t get tired and, when prompted right, is a lot less likely to give up hunting for potential issues. If anything, it can sometimes be overzealous, and I have to carefully review its proposed edits to the requirements to make sure that it doesn’t invent any issues that aren’t there.Layer 2: Unit tests at >95% coverageCoding agents now make test-driven development (TDD) trivial to the point where there’s no reason not to do it. However, it needs to be done right: you don’t want the agent to blindly write passing tests for any bugs it just added to the code. So the best planning and implementation skills I’ve seen usually follow this pattern:Instruct the agent to think through the test scenarios and test cases based on the requirements,Write the test cases,Write the implementation,Test the implementation against the test cases and fix any issues that come up,Maybe backfill any remaining coverage gaps—but again, keeping the requirements in mind.Also, with the agents writing the tests, there’s no excuse not to shoot for near-universal coverage or to wait on backfilling any missing unit tests.Layer 3: Manual testingThere’s still no substitute for a human (you, QA, PM, or someone else) actually trying out the feature, going through all the edge cases, and seeing whether everything works as expected or whether you need to make changes.These manual tests can take a while, especially if the test scenarios take some effort to set up. This is one of the steps that so far has seen only modest gains in productivity, and it’s the main reason that my output has increased only 2-3x instead of something like 10x. Though now that I think about it, there may be a few opportunities for automation here that I’ve missed.Layer 4: Extensive automated end-to-end testsEnd-to-end (E2E) tests are arguably the most important tests in the codebase because they verify that new changes haven’t broken any existing functionality as experienced by the end user. Ideally, they’d run on the PRs, in the test/stage environments, and in production after every deployment. Ideally, they’d also be maintained by the same developers who write regular code, but I understand that some organizations aren’t really set up for that.AI does make it easier to write E2E tests, but to do that effectively, it needs access to the tools or MCP servers that let it debug test failures—e.g., a browser tool or MCP access to the logs. However, it’s important to keep in mind that E2E tests aren’t a substitute for manual testing because they’re just a rough, incomplete check that nothing important broke.Layer 5: Code quality passes by AII find that coding agents aren’t great at following complex instructions in AGENTS.md or CLAUDE.md. But they do pretty well if you add a separate pass to find and fix specific issues. These can be:Security issues,Finding overcomplicated or duplicated code,Compliance with naming, file organization, or formatting rules,A general code review pass to find any issues with the logic,Overly long comments written in AI-ese instead of regular English,Any other specific things that you’d like to find and fix.If added to the planning or implementation skills, these can be pretty much “free” additions, adding maybe 5-15 min to the implementation time with no additional attention required.They can be also added to the PR reviews if you prefer to take a look at the comments before applying any fixes.Layer 6: PR reviews by humans and AII think I’m becoming convinced that for minor tweaks and simple bug fixes, human reviews can become optional. Provided that other defensive layers are still in place.But for complex changes, I find that it’s still necessary to review AI-written code. I still regularly find big-picture mistakes, missed adverse interactions with other features, overcomplicated or suboptimal implementations, and other problems. Not to mention weird word choices like “mint” instead of “generate” or “stamp” instead of “set.”AI code reviews have also been a really great addition. On my current team, we run both Claude and Cursor reviews on the PRs, and surprisingly, each of them finds different problems. You can also add other custom reviews from various angles, like security, efficiency, interactions with other repos, and so on, though be aware that AI can be overly nitpicky in its reviews, so it’s important to also have a pass where another agent prunes the proposed AI-generated PR comments that aren’t actually meaningful.Layer 7: Monitoring and alertingOnce the code is in production, at a minimum, it’s good to have someone periodically scroll through the logs or watch any user recordings in something like Fullstory, or review various dashboards that track error rates, latencies, and other issues.Even better would be an error tracking service like Sentry or GCP’s Error Reporting that detects and deduplicates errors.The best approach, however, would be to then have Claude/Cursor/whatever auto-diagnose these errors, figure out the root cause, and make PRs with the proposed fix.Like what you see? Subscribe for more:SubscribeConclusionsI’m sure I’ve missed other important components of maintaining high quality, but the main idea is that with the right set of defensive layers, the increased output doesn’t have to come at the cost of reliability. If anything, coding agents now make it cheaper to add more and deeper checks than before: more tests, more review passes, faster diagnosis of production issues.So if you’re sufficiently focused on quality, I think it’s entirely possible to double the delivery speed while keeping the bugs under control. Or maybe even reducing them.ShareTopLatestNo postsReady for more?Subscribe© 2026 Iouri Khramtsov · Privacy ∙ Terms ∙ Collection notice Start your SubstackGet the appSubstack is the home for great culture

This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts

The effectiveness of AI coding agents in generating code does not inherently lead to lower code quality; rather, maintaining high quality requires implementing a comprehensive, layered system of defensive coding practices. The author posits that by adopting a thoughtful approach to quality management, it is possible not only to stabilize the number of bugs but also to reduce them while simultaneously increasing output by two to four times. This framework involves establishing multiple defensive layers that leverage AI capabilities alongside traditional quality assurance steps.

The first layer focuses on ensuring correctness from the outset by perfecting the requirements. The author notes that adopting spec-driven development significantly reduces post-development debugging time because the AI can be prompted to review requirements and technical designs to proactively identify gaps, edge cases, and potential interactions with existing code. This proactive checking enables the AI to find issues that might otherwise be missed by developers burdened by time constraints.

The second layer emphasizes robust testing through unit tests, advocating for a target coverage of over ninety-five percent. When using coding agents, testing needs to be structured thoughtfully: instructing the agent to first conceptualize the necessary test scenarios based on requirements, then generating the test cases, followed by the implementation, and finally testing the implementation against those tests. This structured approach encourages near-universal coverage without relying on backfilling missing tests.

Thirdly, manual testing remains indispensable. Although it has seen only modest productivity gains compared to automated methods, human intervention is necessary to verify features, explore complex edge cases, and confirm overall functionality, a step that automation alone cannot fully replace.

Layer four addresses the verification of integrated functionality through extensive automated end-to-end tests. End-to-end tests are deemed crucial for ensuring that new code does not disrupt existing user experiences. While AI assists in writing these tests, effective implementation requires providing the agent with necessary access to debugging tools or system information to handle failures. However, the author cautions that end-to-end tests are a supplementary check and are not a substitute for thorough manual testing.

Layer five involves utilizing AI for specific code quality passes. Because agents struggle with complex instructions in documentation, a separate review pass is recommended to address specific quality metrics. These checks can involve identifying security issues, detecting overly complicated or duplicated code, ensuring compliance with naming conventions, enforcing file organization, and checking for logical flaws. Integrating these specialized checks can add only a few minutes to the implementation time without demanding additional focused attention.

Layer six outlines the necessity of human and AI collaboration during pull request reviews. While human reviewers are essential for catching big-picture mistakes, adverse interactions between features, and subtle semantic errors—such as incorrect word choices—AI code reviews are also valuable for minor fixes and spotting superficial issues. The author suggests running reviews from multiple agents, potentially applying custom checks for security, efficiency, and inter-repository interactions, while acknowledging the risk of over-nitpickiness in AI analysis.

Finally, the seventh layer concerns monitoring and alerting in the production environment. Once code is deployed, continuous vigilance is required through log review, dashboard monitoring for error rates and latency, or dedicated error tracking services. The most advanced application of this layer is leveraging AI to automatically diagnose these production errors, determine root causes, and generate pull requests containing proposed fixes, thereby streamlining the debugging cycle.

In conclusion, the core message is that increased output from coding agents must be balanced with rigorous quality management. By systematically implementing these seven defensive layers—from requirement clarification and rigorous testing to multiple review passes and post-deployment monitoring—organizations can effectively harness AI to achieve high delivery speeds without compromising code reliability.