SDETs Transition to AI-Focused Roles

The advent of large language models (LLMs), coding harnesses, and agentic workflows is rapidly altering job functions across various engineering disciplines. This technological shift is fundamentally changing the nature of software testing, making it crucial for professionals to adapt.

Participants in the field are encouraged to proactively identify and delegate tasks they no longer wish to perform, focusing instead on higher-value activities that leverage human oversight and specialized judgment in the evolving AI landscape.

Professionals must consider which parts of their current job responsibilities are likely to diminish as AI automates routine tasks.

Simultaneously, it is essential to identify which existing skills and responsibilities will gain importance and become more valuable. Furthermore, new competencies must be acquired to effectively stretch into future demands and ensure continued relevance in the rapidly changing technological environment.

Despite advancements in AI, human oversight remains critical, particularly in identifying complex software failures that AI might miss. For example, a team successfully used AI agents to build and test a chatbot, but an engineering leader later discovered a load testing bottleneck that the AI agents had failed to detect.

The ability to catch critical failures typically stems from a deep understanding of how software operates and fails, not merely from expertise in AI models. As Adeel Mansoor noted, "The person who caught it wasn't an AI specialist. They knew software and they knew users."

An SDET functions as a software engineer primarily focused on quality, owning automation frameworks, shaping test strategy, and ensuring teams receive fast, trustworthy feedback upon shipping.

Adeel Mansoor emphasizes that the role will be promoted rather than eliminated if SDETs acquire a few specific skills. Successful SDETs will leverage their foundational knowledge to navigate and excel within the new AI-driven landscape, maintaining their critical function in software development.

This model is structured around four foundational columns representing current, essential skills that SDETs already possess.

Directly layered on top of these existing skills is a new set of capabilities specific to the AI era. This comprehensive structure aims to create a future-proof career path in software testing, ensuring SDETs remain relevant and effective amidst technological advancements.

Automation engineering's core principles are rooted in design, extending far beyond merely writing test scripts. The true value lies in creating repeatable workflows and robust frameworks that can adapt to future changes, rather than focusing solely on the individual scripts.

Adeel Mansoor clarified, "The skill was never writing test scripts. It was designing the thing that scripts live in." SDETs should concentrate on building extensible frameworks capable of integrating new test generators and accommodating evolving testing methodologies.

Deep expertise within a specific technology stack is currently more advantageous than a broad, superficial understanding across multiple areas, allowing for the development of more resilient and effective automation solutions.

In the agentic era, relying solely on mental risk models is insufficient; these models must be formalized for AI-driven workflows. Adeel Mansoor advises, "Most of the risk model lives in your head. Get it out. put it on paper, a checklist, a matrix, a page in the repo that says these are the high-risk areas."

Written documentation of risk areas, such as checklists, matrices, or dedicated repository pages, allows AI models to process and act upon human judgment. This codification serves as a crucial foundation for future AI tasks, ensuring that automated processes are guided by a clear understanding of potential risks.

Adeel Mansoor points out, "The skill isn't reading logs. A lot of people read logs. The skill is refusing to stop at a symptom."

While AI-generated explanations for flaky tests can be persuasive, they are frequently inaccurate, underscoring the need for human diagnostic skills. Implementing a systematic failure taxonomy helps identify underlying patterns and prevents reliance on ad-hoc, 'heroic' fixes.

Adeel Mansoor stated, "the actual job is delivering the truth about systems health to a human fast enough that they can still do something about it."

Pipelines and dashboards function as the implementation layer for this information delivery. Standardizing test tags and stages is essential for effectively surfacing critical quality signals, ensuring that human operators receive timely and actionable insights. A clean, structured pipeline is fundamental for seamlessly integrating future AI test suites.

Quality advocacy and collaboration are fundamental pillars for SDET work, particularly as AI integrates into development workflows. SDETs must hone their ability to explain testing decisions as compelling stories that transparently outline trade-offs involved.

Building trust with skeptical stakeholders is essential to advocate for findings generated by AI. Furthermore, risk-based sampling emerges as a key strategy for effectively reviewing the increasing volumes of code generated and modified in AI-driven development environments.

While routine manual tasks are diminishing, the demand for high-level judgment in the SDET role is expanding. Tasks such as manual regression passes and the maintenance of brittle selectors are increasingly becoming obsolete, automated by AI.

This shift away from repetitive typing tasks underscores and validates the growing necessity for higher-level judgment skills. As Adeel Mansoor puts it, "What doesn't shrink is everything on the map. The judgment, not the typing. And the distinction is the whole thing." These judgment-based skills form the essential foundation that prevents the SDET role from shrinking.

Adeel Mansoor defines it as "using prompts to generate test ideas, test data, and checks that are reliable, repeatable, and tied to actual risk."

This approach emphasizes that prompt libraries effectively function as test frameworks, requiring similar rigor in their development and maintenance. Versioning and parameterizing prompts are crucial for ensuring reliability and consistency, much like traditional test scripts.

SDETs can follow a three-step path: first, learn the behavior of AI models; second, generate effective test patterns; and third, maintain a robust repository of prompts.

Data curation, particularly in the context of designing AI data corpora, is an evolution of traditional equivalence partitioning in testing. Adeel Mansoor asserts, "You've been making that call your entire career. Equivalence partitioning is corpus design. You've been doing it by hand for years."

This perspective positions data curation as a core testing activity, rather than solely a data science task. It directly connects the design of data corpora to long-standing equivalence partitioning techniques, which testers have historically used to identify representative data subsets.

Existing resources like logs, bug trackers, and incident reports are recommended as valuable sources for curating relevant data. Datasets should be structured with four key columns: input, expected behavior, a risk tag, and severity, enabling comprehensive risk-based analysis.

Adeel Mansoor argues, "You do not need to be able to train a model. You need to be able to test one. That's a much lower bar and much more useful skill in today's at least for today."

Instead of viewing AI 'hallucinations' as mystical occurrences, SDETs should rebrand them as specific, debuggable system failures. These failures often have identifiable root causes, such as context window overflow, retrieval misses, or improper temperature settings.

SDETs are encouraged to approach AI model issues with the same analytical rigor applied to traditional UI bugs. This diagnostic mindset, focused on testing and root cause analysis, makes AI model understanding a highly accessible and valuable skill for the evolving SDET.

Adapting Continuous Integration/Continuous Delivery (CI/CD) for AI-generated code primarily impacts the speed and timing of execution rather than the fundamental structure. Adeel Mansoor notes, "I think in a way they don't change much but what changes is perhaps your speed and when you want to run things."

It is crucial to prioritize risk reduction by focusing on high-priority features and smoke tests, running critical tests as early as possible within the pipeline. While the core pipeline structure remains largely consistent, the increased feedback velocity demanded by AI-driven deployment speeds necessitates faster and more frequent execution of tests.

Adeel Mansoor advises, "Be constantly learning and evolving. And the second, like continue to align what you're doing with overall business goals that your team or company might have."

Instead of attempting to predict specific tools or technologies years in advance, SDETs should prioritize constant learning and adaptability. It is also critical to ensure that all technical efforts are directly aligned with the strategic goals of their team and company, allowing for greater flexibility in navigating market shifts and technological advancements.

Adeel Mansoor emphasizes, "Run it on your PRs before a human even has a chance to look at it."

Integrating security policies directly into the CI/CD pipeline and leveraging modern, cost-effective security tools are essential for this transition. This approach helps reduce noise in signal detection and ensures that security checks are executed automatically at the pull request (PR) level, even before human review.

This proficiency in evaluating AI model behavior provides a significant competitive advantage in the current job market.

Adeel Mansoor highlights the importance of this skill, stating, "If you can get really comfortable with evals and you're able to demonstrate that in your resume, your conversation with a recruiter or a hiring manager, you're going to give yourself a leg up."

SDETs should treat AI evaluations with the same rigor and importance that was previously given to mastering browser automation frameworks like Selenium or Playwright. Demonstrating expertise in this area can significantly enhance career prospects.