Jobs by Fastorial
Back to all jobs

Senior QA Engineer - (NPW)

2 views
Milestone Technologies, Inc.KochiHybrid

5+ years

Posted 4 days ago

  • Devops
  • Llm
  • Python

Role Summary

Owns sprint-level functional and regression testing of product features, and response-accuracy validation against the golden dataset; owns platform-level test planning and execution, including regression suite operation and release validation support; and owns data validation testing across assigned deliverables, including dual-source attribution and fallback verification, golden dataset regression, and ingestion pipeline quality checks.

Key Responsibilities

  • Perform sprint-level functional and regression testing of product features
  • Validate response accuracy against the golden dataset baseline
  • Support defect triage for issues surfaced during client-led UAT
  • Plan and execute platform-level testing
  • Operate the regression suite for platform components
  • Support release validation and perform defect triage with root-cause analysis
  • Validate dual-source attribution and fallback behavior
  • Execute golden-dataset regression testing
  • Perform ingestion pipeline quality checks

Required Skills and Experience

  • 5+ years QA / test engineering experience
  • Hands-on with pytest or an equivalent Python test framework, with strong defect-triage discipline
  • Regression suite design and execution experience
  • Experience validating AI/LLM response accuracy against a reference dataset
  • Experience supporting UAT coordination with external / client stakeholders
  • Regression suite design and maintenance at a multi-team scale
  • Root-cause defect triage experience
  • Experience testing data pipelines and multi-source attribution logic
  • Regression testing experience against a governed reference dataset

Preferred Qualifications

  • Experience testing RAG or LLM-based features specifically
  • pytest or equivalent automation framework experience
  • Experience testing orchestration/agent-routing systems specifically
  • Experience validating fallback logic in dual-source retrieval systems specifically
  • Experience testing knowledge-graph or SPARQL-backed pipelines
  • Education: B.Tech/BCA in Computer Science or a related field, or equivalent practical experience; ISTQB certification preferred