Jobs by Fastorial
Back to all jobs

Senior Data Engineer

1 view
ShimentoX TechnologiesBengaluruHybrid

6+ years

Posted 1 day ago

  • Airflow
  • Git
  • MySQL
  • PostgreSQL
  • Python
  • Spark

🚀 We're Hiring: Senior Data Engineer

📍 Location: Bangalore

💼 Experience: 6+ Years

About the Role

We're looking for a Senior Data Engineer with strong data architecture and system design skills to design, build, and own end-to-end production pipelines powering analytics and decision-making across the organization. The role centers on deep Python and SQL expertise, robust transaction-level fact tables, and rigorous data quality and auditability, ideally within a finance, risk, or compliance data domain.

What You'll Do

🔹 Own data architecture and system design decisions for pipelines and data models, including grain, partitioning, schema evolution, and scalability trade-offs.

🔹 Take end-to-end ownership of production pipelines: design, build, deploy, monitor, and operate.

🔹 Design and build transaction tables (append-only, immutable event/transaction-grain fact tables) alongside standard fact/dimension models using medallion (bronze/silver/gold) architecture.

🔹 Build and support pipelines for finance, risk, or compliance use cases where accuracy, auditability, and data lineage are critical.

🔹 Implement data quality and auditability controls, including validation checks, reconciliation logic, anomaly detection, and audit trails for every pipeline.

🔹 Write complex, cross-dialect SQL (MySQL + PostgreSQL), including window functions, multi-layered CTEs, CASE-driven logic, COALESCE/NULLIF, type casting, and date/timestamp handling.

🔹 Build idempotent reload patterns (DELETE + INSERT), UNION ALL/set operations, and templated (Jinja-style) SQL, including handling late-arriving/corrected transaction records without double-counting.

🔹 Orchestrate Airflow-style DAGs with cross-pipeline dependencies and secure credential handling.

🔹 Build Python ETL tooling using pandas (CSV ↔ DB), SQLAlchemy + raw drivers (MySQLdb, psycopg2), type hints/dataclasses, and class-based pipeline design.

🔹 Integrate with Google Drive/Sheets APIs, build YAML-driven pipeline configurations, and handle pipeline staging requirements.

🔹 Own CI/CD for pipelines through standard Git workflows.

🔹 Document data models and pipeline logic for maintainability, keeping compliance and audit requirements in mind.

What We're Looking For

Must-Have Skills:

✅ 6+ years of experience in data engineering.

✅ Strong data architecture and system design skills to make sound trade-offs on modeling, partitioning, and scalability - not just write queries.

✅ Demonstrated end-to-end production ownership of data pipelines from design through deployment, monitoring, and operational support.

✅ Expert-level Python and SQL as the primary toolset. Hive/Hadoop is a nice-to-have only.

✅ Robust data quality and auditability practices, including validation, reconciliation, anomaly detection, and audit trails.

✅ Experience working with finance, risk, or compliance data, with an understanding of accuracy, auditability, and data-lineage requirements.

✅ Proven experience designing and building transaction/fact tables.

✅ Expert SQL skills, including window functions, CTEs, CASE logic, and cross-dialect MySQL/PostgreSQL.

✅ Strong Python skills with pandas, SQLAlchemy, type hints/dataclasses, and class-based pipelines.

✅ Airflow-like orchestration experience.

✅ Solid data modeling fundamentals, including fact/dimension models, normalization, and medallion architecture.

✅ YAML-based configuration experience, along with standard CI/CD and Git workflows.

Nice-to-Have Skills

🔹 Hive/Hadoop or Spark experience.

📩 Interested candidates: Share your updated resume

#Hiring #SeniorDataEngineer #DataEngineering #Python #SQL #DataArchitecture #ETL #DataPipelines #BangaloreJobs #TechHiring