Senior QA Engineer - (NPW)
2 viewsMilestone Technologies, Inc.KochiHybrid
5+ years
Posted 4 days ago
- Devops
- Llm
- Python
Role Summary
Owns sprint-level functional and regression testing of product features, and response-accuracy validation against the golden dataset; owns platform-level test planning and execution, including regression suite operation and release validation support; and owns data validation testing across assigned deliverables, including dual-source attribution and fallback verification, golden dataset regression, and ingestion pipeline quality checks.
Key Responsibilities
- Perform sprint-level functional and regression testing of product features
- Validate response accuracy against the golden dataset baseline
- Support defect triage for issues surfaced during client-led UAT
- Plan and execute platform-level testing
- Operate the regression suite for platform components
- Support release validation and perform defect triage with root-cause analysis
- Validate dual-source attribution and fallback behavior
- Execute golden-dataset regression testing
- Perform ingestion pipeline quality checks
Required Skills and Experience
- 5+ years QA / test engineering experience
- Hands-on with pytest or an equivalent Python test framework, with strong defect-triage discipline
- Regression suite design and execution experience
- Experience validating AI/LLM response accuracy against a reference dataset
- Experience supporting UAT coordination with external / client stakeholders
- Regression suite design and maintenance at a multi-team scale
- Root-cause defect triage experience
- Experience testing data pipelines and multi-source attribution logic
- Regression testing experience against a governed reference dataset
Preferred Qualifications
- Experience testing RAG or LLM-based features specifically
- pytest or equivalent automation framework experience
- Experience testing orchestration/agent-routing systems specifically
- Experience validating fallback logic in dual-source retrieval systems specifically
- Experience testing knowledge-graph or SPARQL-backed pipelines
- Education: B.Tech/BCA in Computer Science or a related field, or equivalent practical experience; ISTQB certification preferred