AI Code: Speed, Instability, and the Human Element

CookUnity leverages AI tools like GitHub Copilot and CodeX to generate nearly 90% of its automation scripts, reflecting a significant reliance on AI for rapid development in a startup environment. This high adoption rate, however, redefines the engineering workflow, shifting the bulk of human effort towards intensive code review rather than initial code generation.

Code review now often takes longer than the actual AI code generation process, as AI frequently "over-engineers" solutions. This necessitates manual intervention to verify the relevance and necessity of assertions within the generated code, highlighting the critical "human in the loop" aspect for ensuring quality.

Motive, a company dealing with both software and firmware products, maintains a more conservative approach to AI adoption, capping AI-generated code contribution at 60-65%. This cautious stance is driven by the differing requirements of their products.

Firmware development, for instance, still demands significantly more human intervention due to its intricate nature. While AI-generated code is utilized for software testing, the company prioritizes organizational guardrails to balance innovation with maintaining stability and control.

Little Caesars integrates AI for 40-50% of its shipped code, but notes that day-to-day operations have not yet undergone fundamental changes. The company's quality engineering philosophy remains agnostic to how code is developed, focusing instead on requirements and risk management.

The primary responsibility of QE remains consistent regardless of code origin. However, Little Caesars anticipates future challenges in adapting to the escalating velocity and volume of AI-driven development.

Little Caesars confirms that they have not yet encountered the rising instability issues associated with AI, as they are still in the early stages of their AI adoption journey. The company's existing quality system and role as gatekeepers for outbound code contribute to this stability.

The core challenge for Little Caesars remains ensuring that validation processes are robust enough to handle increasing AI contributions while maintaining product quality.

The exponential speed of AI code generation naturally leads to increased instability if human oversight is not properly integrated. QE teams must evolve their focus from traditional coverage metrics to comprehensive risk mitigation strategies. This involves creating deterministic test harnesses to ensure system availability and stability.

Automated pull request (PR) reviews, while efficient, still require human verification to confirm that the AI's intent aligns with development goals. The challenge lies in capturing the benefits of AI speed without compromising system stability.

The effectiveness of AI output is entirely dependent on the subject matter expertise of the person writing the prompts and verifying the results. Without domain knowledge, a significant portion of AI-generated content—potentially 80%—can be nonsensical.

Human course correction is essential when AI fails to meet expectations, as AI lacks true intent and can only process information based on its training. Critical thinking is therefore indispensable for sifting through AI outputs and ensuring accuracy.

A startup aimed for bi-weekly mobile and web releases, facing the challenge of building over 200 test cases with zero additional resources. Initially, AI tools like Claude rapidly generated a mobile automation framework.

While the first-pass results seemed promising, deeper refinement was required to make the framework truly effective and reliable for the scale of testing needed.

Current AI-driven automation achieves approximately 70% accuracy, significantly surpassing human-level speed in code generation. However, achieving 100% accuracy remains a difficult challenge.

The trade-off for increased speed is a higher rate of flaky tests and persistent instability, necessitating human intervention to bridge the accuracy gap.

Before AI, failures typically indicated product or infrastructure bugs, but with AI, the first instinct is to audit the AI's generated code. This has made debugging more complex and messy, requiring a deeper understanding of AI's intent.

Development of AI-based 'failure analysis agents' is underway to help with triage. However, the true root causes of failures are often poor specifications or prompts given to the AI, rather than inherent product bugs.

The definition, specifications, and underlying processes behind code generation are now considered more important than the code itself. AI can produce syntactically correct code that still fails to meet business logic due to misinterpretation.

Human responsibility lies in defining ambiguous terms and clarifying business intent, as this foundational clarity is paramount for AI to generate truly effective solutions.

Nandini Srinivasan reports significant success in using AI for production fixes, with her team deploying over 25 Claude AI agents (both Claude and Glean) and generating more than 300 production fixes in the last three months alone. This success stems from augmenting, rather than replacing, human quality engineering expertise.

The strategy emphasizes leveraging pre-existing test automation expertise to enhance the capabilities of AI, ensuring that AI-driven solutions are integrated efficiently and effectively into the production environment.

The speed at which AI can generate and supply code often outpaces the ability of users to adopt new features. This discrepancy leads to product teams over-releasing features that users may not necessarily need or desire.

The constant push for updates, sometimes mimicking tech giants, fails to consider the unique adoption contexts and needs of individual industries and their user bases.

Ankur Agarwal challenges the notion that "code is not a moat" anymore, asserting that while AI makes code reproduction easier, fundamental coding skills remain vital for quality engineering. He argues that domain knowledge and coding expertise are complementary.

These skills are not mutually exclusive, and both continue to be important for developing and verifying software, even as AI streamlines the coding process.

As AI simplifies coding tasks, the core competency for Quality Engineers is shifting towards validating AI-generated output for correctness. This involves a greater emphasis on problem-solving skills and systemic thinking—understanding how an entire system behaves.

Fluency in specific programming languages like Python or Java becomes less critical than the ability to comprehend and verify the broader system implications of AI-generated code.

While technical frameworks can be taught in weeks, deep business and customer domain knowledge takes significantly longer to acquire and is the primary source of real QA value. AI can accelerate code generation, but without this critical domain understanding, it can lead to rapidly building the "wrong things."

This emphasizes that human insight into the business context and customer needs is irreplaceable in ensuring software quality.

This distinction highlights the ongoing importance of system reliability, trust, and quality guardrails.

Quality Engineers don't necessarily need entirely new skills; rather, they need to apply their existing "QA intuition" to interactions with AI, ensuring that the integrity and functionality of systems are maintained despite rapid AI-driven changes.