AI Shifts Testing from QA to Trust Engineering

The conference aims to unite industry leaders to address the challenges and opportunities of scaling quality in the AI era. This year's event highlights the critical shift towards integrating AI while maintaining trust and robust quality standards.

Kishor Wikhe brings over 33 years of experience in the BFSI (Banking, Financial Services, and Insurance) sector and enterprise technology. His extensive background includes leading global delivery center transformations at Capgemini, positioning him as a prominent voice in the evolving tech landscape.

Kishor Wikhe addressed a prevalent misconception among AI enthusiasts: that AI, particularly large language models (LLMs) writing code and automated pipelines, renders traditional testing obsolete. He acknowledged the narrative suggesting "the testing is more or less is the past of the thing." However, he challenged this perspective by posing critical questions about the real-world reliability and safety of fully automated AI-driven systems, emphasizing that the human element of assurance remains vital.

When confronted with these critical scenarios, a colleague admitted they lacked complete trust, underscoring the inherent risks that even advanced automation cannot fully mitigate.

AI hallucinations, once viewed primarily as model quality issues, now pose significant real-world liabilities for enterprises. Kishor Wikhe stated, "what it means is the hallucination is no longer a model quality issue. I mean, if you apply the generative AI business uh or a legal filing or your board decision, it really becomes a governance issue." A fabricated reference in a government report, for example, can lead to severe consequences, including loss of credibility, public apologies, and brand damage. The implementation of AI in sensitive sectors, such as legal filings or board-level decision-making, elevates these issues to a critical governance concern.

AI agents, when granted elevated authority, can inadvertently cause significant damage while merely performing their assigned tasks. Kishor Wikhe explained, "The agent absolutely was not malicious or was not trying to create any damage. Was just was trying to pursue the task which he's been asked to do it. And and you know, with the authority which it had uh been given by us. And the real risk is not intelligence, but not the intent, by the way, huh? So, what it means is agent failures are off often, you know, authority failures." He emphasized that the risks stem from 'authority failures' rather than malicious intent, as unintended consequences can arise when agents access sensitive resources with excessive credentials.

Organizations must now demonstrate and prove the actual behavior of AI systems after launch, rather than simply relying on initial trust or model predictions.

He explained, "It's basically going to be about the trust uh which has to be engineered across probably the data, the authority, or the identity, escalation monitoring, and also the evidences you know, and that is where the future of testing is bigger than just testing and doing the check box uh and and reporting the reports bugs actually, and the reporting the bugs. It is more about trust engineering." This holistic approach recognizes that model quality alone is insufficient for reliable AI systems, necessitating engineered trust across various dimensions, including data integrity, identity management, and continuous monitoring.

The primary question for Quality Assurance (QA) teams is changing from simply "does it work?" to the more profound "can it be trusted?" to reflect this new reality.

This implies that even if automated pipelines appear operational, business outcomes may remain unstable due to these emergent complexities.

Kishor Wikhe articulated, "I think we already talked about basically that the era of testing as a gatekeeper is kind of, you know, transition to more of the digital transformation area as actually as expanded scope." While traditional testing focused primarily on release readiness, the expanded scope now includes integrating AI systems and assuring the promises made to users, reflecting a move beyond digital transformation to AI-integrated paradigms.

This new paradigm demands ongoing validation and proof of trustworthiness.

It is crucial to differentiate between using AI as a tool for testing and testing AI systems themselves, as Kishor Wikhe highlighted. He explained, "Here the testing with AI is nothing but a productivity play, right? You test the testings, leverage that AI as part of your testing, provide the productivity gain. And then testing AI systems is basically you're you're building the trust, you're enabling the trust in the system." Testing with AI primarily focuses on productivity gains and automating existing tasks, whereas testing AI systems focuses on building and enabling trust and reliability within the AI-driven system itself. The latter represents the current and most significant evolution in the quality assurance field.

The new imperative is to provide empirical evidence that substantiates and justifies this trustworthiness across all aspects of AI integration and operation.

The host stated, "I was joking the other day with with someone that now is the time for chief quality officers because agents and AI is going to be embedded everywhere." With AI and autonomous agents becoming embedded into every system and process, quality engineering will need to extend its reach to every business function, making a dedicated C-suite role essential for overseeing comprehensive trust and assurance.