Sr Data Engineer
1 viewNarwalPuneOnsite
6-12 years
Posted 3 days ago
- Airflow
- AWS
- GCP
- Git
- Kafka
About the Role
We are looking for an experienced Data Engineer with 6+ years of hands-on experience in designing, developing, and operationalizing enterprise data ingestion pipelines and trusted data products within a Data Lake ecosystem.
Key Responsibilities
- Lead the design, development, and implementation of enterprise data ingestion initiatives using Fivetran, Kafka, and Apache Iceberg.
- Design and build scalable, reliable batch and near-real-time ingestion pipelines to integrate data from diverse source systems.
- Develop and maintain ingestion frameworks and data pipelines, ensuring performance, scalability, and operational reliability.
- Build and optimize data transformation pipelines using dbt, following modularity, reusability, testing, and documentation best practices.
- Build, schedule, monitor, and troubleshoot workflows using Apache Airflow and Astronomer.
- Publish curated datasets to Snowflake and support efficient data models for analytics and downstream use cases.
- Implement data quality tests, validation rules, and reconciliation mechanisms to ensure data accuracy, consistency, and reliability.
- Implement appropriate logging, alerting, error handling, and recovery mechanisms for production-grade data pipelines.
Requirements
- 6 to 12 years of overall experience in Data Engineering with production-grade data pipelines.
- Strong hands-on experience with Fivetran, Apache Kafka, Apache Iceberg, Snowflake, and dbt.
- Strong experience with Apache Airflow for workflow orchestration and DAG development.
- Strong SQL skills and understanding of ETL/ELT and data modeling concepts.
- Experience with Git, code reviews, and software engineering practices.
- Exposure to cloud platforms such as AWS or GCP is preferred.