Agentic Engineering & Quality in Healthcare

Healthcare organizations have achieved substantial success by using AI to supplement existing knowledge areas, particularly by augmenting internal teams rather than relying on external consultancies. This strategy has proven cost-effective, replacing expenditures that previously amounted to hundreds of thousands of dollars. The focus is on leveraging AI as an enhancement tool, not a complete substitute for human expertise.

Test coverage, once a valuable indicator reflecting development effort, has lost its signal due to AI's ability to generate vast test suites quickly. While the coverage number remains, the underlying value has diminished because creating tests has become inexpensive. The primary challenge has shifted from generating tests to ensuring the accuracy and relevance of assertions within those tests, making this the new core of the testing process.

Large Language Models (LLMs) demonstrate significant capabilities but often underperform due to insufficient feedback mechanisms. For instance, self-improving test suites initially plateaued at a 53% mutation score when LLMs were only asked to improve the suite abstractly. When given specific failure feedback—showing the exact breaks that occurred—the same model and code achieved a 90% mutation score. This indicates that enhancing AI performance critically depends on closing the feedback loop with precise, actionable data.

Traditional artifact signing for validation is inadequate for adaptive agentic systems, necessitating a shift to protocol-based compliance. The FDA's 'predetermined change control plan' offers a viable framework for managing evolving AI. Testing now requires asserting over distributions and tolerances rather than exact matches, as pinning a model version is no longer a sustainable validation strategy with frequently updated vendor endpoints.

The advent of agentic solutions means testing is no longer a final maintenance phase but a continuous cycle. Continuous auditing of outputs is crucial to maintain trust in autonomous systems. While AI models show improved accuracy, traceability remains a primary concern, especially during failures. The focus has shifted from singular sign-offs to persistent auditability throughout the system's lifecycle.

AI significantly accelerates the comparison of complex data, such as identifying duplicate claims, allowing for near-real-time validation. Major healthcare entities, including UnitedHealthcare and Elevance, are deploying AI to expedite prior authorization processes. This adoption is partly driven by CMS mandates pushing the industry towards greater interoperability and stricter regulatory compliance. Consequently, testing is no longer a distinct phase but is distributed across the development pipeline, with shared responsibility among all team members.

Traditional testing gates are increasingly ineffective at catching bugs introduced by AI-generated code. This necessitates a strategic shift towards designing systems for rapid reversal of production releases. Engineering investments are now prioritizing reversibility, with operational gates weighted by the difficulty of rolling back changes. Mobile releases, for example, require more stringent gates due to their inherent lack of instant rollback capabilities.

Synthetic data often fails to replicate the intricate complexity of real-world edge cases. AI now enables teams to effectively query massive, deidentified production datasets to retrieve specific claim scenarios, overcoming this limitation. This advanced data mining capability replaces manual SQL queries previously used to reconstruct complex patient histories, offering a more robust approach than synthetic data generators.

The very mechanism that makes synthetic data private inadvertently eliminates critical outliers, which often represent complex comorbidities and long-term user histories. This structural problem means synthetic data frequently misses 'vintage' scenarios, failing to simulate multi-year state accumulation. Consequently, relying on synthetic data for testing can create a false sense of security, as crucial edge cases remain untested.

Enterprises are now leveraging production incident logs to create comprehensive test data. AI facilitates the automatic generation of negative scenarios that precisely mimic real-world production defects, moving testing beyond conventional 'happy path' approaches common in large organizations. This method also addresses the issue of stale test data through automated refreshing, ensuring test environments remain relevant and robust.

Human involvement in AI systems is strategically placed at three critical operational points: before, during, and after. Before content enters the knowledge base, clinical content review ensures accuracy. During operation, high-risk intents trigger human escalation. After interactions, humans label conversations to maintain the accuracy of evaluation suites, ensuring alignment with real-world complexities.

The traditional distinctions between developers and testers are blurring rapidly. The emerging 'superpowers' lie in providing context, articulating intent, and demonstrating a deep understanding of business processes. Future roles will demand a broad comprehension of the entire ecosystem, transforming the primary function from coding to effective communication.

The easy work of script authoring is being replaced by more challenging, senior-level tasks. Defining what constitutes success, known as Oracle design, is now a scarce and highly valued skill. Evaluation design has evolved from asserting equality to sophisticated statistical grading. Furthermore, deliberate test data generation, focusing on crafting cases that models would not produce independently, has become a key priority.

AI serves as a 'super suit' that enhances the capabilities of existing employees, augmenting human capital rather than replacing it. Human resources remain the primary drivers of value within an organization. Consequently, investing in people is as crucial as investing in technology to maximize the benefits of AI integration.

Thorough evaluations are essential to prevent AI agents from providing unauthorized medical advice. Additionally, contractual obligations, such as guild requirements, introduce further layers of complexity. For clients who distrust AI, fallback algorithmic alternatives must be available.

The rapid pace of AI-generated software introduces potential accountability gaps, necessitating a 'shift-left' approach where testing begins at the very start of the development lifecycle. Audit trails become critical test artifacts in this new paradigm. Continuing with outdated Software Development Life Cycle (SDLC) patterns alongside AI integration inevitably leads to bottlenecks, highlighting the need for early and continuous testing.