SDET 3

• Own test strategy and release quality criteria for the product area
• Build and maintain UI automation, API automation and end-to-end regression suites
• Design evaluation strategy for AI agents using LLM-as-judge, rubrics and other evaluation techniques
• Build eval datasets and automated evaluation suites in Python integrated with CI/CD
• Test AI agents for accuracy, consistency, tool usage, safety and data handling
• Debug agent failures using LangSmith and similar AI observability tools
• Analyse release-over-release quality and identify regressions
• Review product requirements and identify gaps in requirements and test coverage
• Review test code, set testing standards and mentor SDETs
• Communicate findings and recommendations to engineering, product and leadership
• Work closely with GenAI engineering, CX engineering and product teams

• 6–9 years in software testing, including hands-on SDET experience
• Good working knowledge of Python and Java
• Experience owning test strategy and improving automation and release quality
• Experience with UI automation using Selenium or Playwright
• Experience building API automation and end-to-end frameworks with CI/CD
• Ability to analyse product requirements and identify testing gaps
• Prior experience testing chatbots, IVRs and GenAI agents
• Experience designing evals, rubrics and LLM-as-judge evaluations
• Working knowledge of LangChain, LangGraph and LangSmith or similar tools
• Experience building eval harnesses or tooling for AI agents
• Strong analytical and communication skills

• Remote work arrangement
• Certified Great Place to Work engineering center