Code Review in the Age of AI-Generated Code

Simon Willison projected that by the end of the year, half of all developers would generate over 90% of their code using AI agents. This rapid adoption is evident, with Victor Rentea noting that over half of his workshop attendees no longer write code manually. Industries with high stakes, such as fintech and high-frequency trading, are increasingly adopting AI-first workflows, indicating a broad industry transformation.

The concept of a 'Dark Factory' suggests a production environment where code is entirely generated and deployed without any human review, a vision many managers idealize. StrongDM, for example, is cited as a company implementing a 'no code review' policy for certain processes. However, this full automation is contrasted with the more pragmatic 'Gray Factory' concept, which acknowledges the need for responsible, progressive human oversight, especially in complex systems.

The 'Dark Factory' is considered an impractical utopia, with the 'Gray Factory' representing a pragmatic middle ground for integrating AI. Engineers are advised to 'dim the lights' by gradually reducing human oversight as AI agents consistently demonstrate reliability over months. The core responsibilities of human engineers remain crucial: handling on-call duties, providing production support, and ensuring the long-term evolution of the system.

The fundamental problem facing developers is the cognitive overload caused by massive volumes of AI-generated pull requests (PRs). Reviewers must shift their focus from line-by-line inspection to higher-level concerns, such as domain model changes and architectural integrity. Proposed focus areas include logical architecture, database schema evolution, and cyclomatic complexity to efficiently manage and validate AI contributions.

A more practical approach is the use of stacked PRs, which allow massive work to be broken down into focused, manageable sections. This method also enables cascading reviews, where feedback on one branch can ripple through dependent branches, streamlining the review process.

AI should be instructed to 'simplify' code rather than expanding it, preventing unnecessary bloat. Furthermore, implementing reinforcement learning with AI agents can help reduce boilerplate code and overall code bloat in subsequent PRs, leading to more concise and reviewable submissions.

Playwright can be used to record UI interactions, providing essential context for code reviews. The goal is to offer a visual 'warm-up' of changes, allowing reviewers to understand the impact before diving into the code. Automated environment provisioning further enables reviewers to interact with the merged PR state locally, facilitating a more comprehensive understanding of the modifications.

This approach focuses on developing minimal viable product (MVP)-style features. Breaking work into tangible milestones prevents the creation of massive, unreviewable pull requests, ensuring that AI-generated code remains manageable and integrated.

Text-based diagram tools like PlantUML often scramble their layouts when modifications are made, causing disruption to visual memory. For instance, adding a new line between two boxes can completely alter the diagram's structure. To prevent this drift, tests can be implemented to extract domain models via reflection, ensuring that diagrammatic representations remain consistent with the underlying code structure.

Draw.io PNG files can store actual XML data within their headers, a feature that can be exploited for maintaining diagram integrity. Tests can be developed to verify that this diagram metadata precisely matches the actual Java model, ensuring consistency between visual representations and code. AI can then be directed to automatically update diagrams based on failing integration tests, preventing manual discrepancies.

Adhering to Simon Brown's C4 model principles, manual layout remains important for visual memory and understanding of architectural diagrams. Full automation for diagram positioning is intentionally avoided to ensure that humans actively engage with and remember the system's structure. This human involvement is deemed critical for maintaining a clear and intuitive visual representation of the architecture.

Playwright offers a capability referred to as 'time-traveling' tests, allowing developers to inspect the state of a test at any given moment. This feature includes live network and console interaction during playback, providing deep insights into test execution. This advanced debugging replaces manual and often cumbersome Selenium or Cypress workflows, significantly streamlining the troubleshooting process.

Victor Rentea expresses disagreement with the views of Uncle Bob and Matt Pontin regarding mutation testing, advocating for a different focus in quality assurance. He prioritizes ensuring that tests comprehensively cover functional requirements rather than meticulously scrutinizing every line of implementation code. The emphasis is on addressing business corner cases and their impact, rather than solely achieving high code line coverage.

Imposing hard limits on the number of lines of code in pull requests (PRs) is strongly discouraged. Such constraints are considered brutal and are predicted to make AI agents produce lower-quality or 'shitty' code in an attempt to satisfy the arbitrary limits. Instead, it is suggested that features be sliced into smaller, more granular Jira tickets, promoting a more natural and effective development process.

Specialized tooling monitors the usage of standardized UI components to detect misuse of the design system. This system identifies instances where AI agents inadvertently reinvent components instead of utilizing existing, standardized design elements. Visual regression testing is then employed to verify the integrity of these components, ensuring that AI contributions adhere to established UI/UX guidelines and prevent design drift. The front-end is identified as a critical area where such inconsistencies commonly arise, requiring rigorous automated checks.

This code drift could negatively impact the agents themselves, as they perform better with consistent and readable code. Code review remains a critical tool to mitigate agent-driven technical debt and ensure that AI contributions maintain high quality and adherence to existing conventions.

The discussion revolves around determining if automated build failures for large methods are excessively brutal. Instead, using code metrics to raise alarms for human investigation is suggested as a more balanced approach. 'Code city' visualizations play a role in tracking the accumulation of complexity, providing visual cues that alert engineers to potential issues without immediately halting the development process.

This involves daily one-hour transcribed and recorded meetings where a human or synthetic agent asks the employee about their decades of work. The goal is to feed this extensive experience into a 'second brain,' effectively 'cloning' employees to prevent the loss of critical knowledge due to staff churn.

Every mistake identified in AI-generated code should lead to the question: Can the Markdown documentation be updated to prevent this issue from recurring? This approach treats AI errors as opportunities for continuous improvement in system intelligence. Pull requests should be reviewed statistically, meaning that if a repository consistently demonstrates impeccable AI performance, human oversight can be gradually scaled back. The overarching objective is to 'dim the lights,' progressively reducing manual code reading as the AI's reliability and performance prove consistent over time.

MCPs (Master Control Programs) are useful as Language Server Protocols (LSPs) for Java or in cloud-based connections. However, for simpler tasks like database connections, command-line interface (CLI) scripts can often suffice, rendering MCPs unnecessary. Local agent workflows frequently bypass the need for complex infrastructure, suggesting that MCPs are not universally required and their utility depends on the specific context and scale of the agentic operation.