QA Engineer

About the role:

We are looking for an experienced and hands-on Senior QA Engineer with 7+ years of experience in software quality assurance. The ideal candidate must have strong expertise in manual testing, automation testing, Python, test automation frameworks, and AI-powered product testing.

The candidate should have experience working in a product-based technology company and be capable of owning the complete quality lifecycle—from requirement analysis and test planning to automation, release sign-off, AI evaluation, and production-quality monitoring.

Role: Senior QA Engineer

Requirement: 1

Location: Bangalore (Domlur | WFO 5 days)

Experience: 7+ years

Requirements

Responsibilities:

• Own the overall quality strategy for the assigned products and engineering teams.
• Lead manual and automation testing across web applications, mobile applications, APIs, backend services, AI features, and third-party integrations.
• Design, develop, and maintain scalable automation frameworks using Python.
• Create comprehensive test plans, test scenarios, test cases, and release-quality reports.
• Perform functional, regression, integration, API, database, exploratory, and performance testing.
• Define testing strategies for AI/ML and Generative AI features, including chatbots, recommendation systems, search, summarisation, classification, and content-generation workflows.
• Validate AI-generated responses for accuracy, relevance, consistency, completeness, safety, and business-rule compliance.
• Test AI systems for hallucinations, inappropriate responses, prompt injection, data leakage, bias, and edge cases.
• Build automated evaluation frameworks and datasets for testing LLM and AI-powered features.
• Test Retrieval-Augmented Generation (RAG) workflows, including document retrieval, context relevance, response grounding, and citation accuracy.
• Validate AI model and third-party LLM API integrations for reliability, latency, error handling, rate limits, token usage, and cost.
• Establish baseline quality metrics and regression suites for AI-generated outputs.
• Review product requirements, prompts, workflows, and technical designs to identify gaps and risks early in the development lifecycle.
• Define and track quality metrics such as defect leakage, automation coverage, regression effectiveness, release readiness, AI response accuracy, hallucination rate, and latency.