Data Engineering Lead
1 viewImpelsysDubaiHybrid
8+ years
Posted 5 days ago
- Airflow
- AWS
- Git
- Python
- Terraform
We are looking for an experienced Data Engineering Lead to provide technical leadership across multiple data engineering projects and squads.
The ideal candidate will have strong hands-on experience with Apache Airflow, DBT, Python, SQL, and AWS, along with proven experience designing scalable data pipelines and leading data engineering teams.
You will be responsible for developing and optimizing Airflow workflows, managing DBT infrastructure, improving data engineering architecture, and ensuring reliable, scalable, and cost-effective data solutions.
Key Responsibilities
- Provide technical leadership across data acquisition, transformation, and distribution.
- Lead, mentor, and coach Data Engineers, Data Architects, and Database Administrators.
- Design, implement, and manage scalable data pipelines using Apache Airflow.
- Develop, test, and maintain complex Airflow DAGs and custom operators.
- Monitor, troubleshoot, and optimize Airflow workflows for performance and reliability.
- Develop, test, and maintain DBT infrastructure and models executed through Airflow.
- Manage DBT infrastructure upgrades and optimize DBT code for performance and efficiency.
- Integrate DBT workflows with Apache Airflow.
- Ensure data quality, integrity, reliability, and consistency across data pipelines.
- Collaborate with Data Engineers, Data Scientists, Product Owners, and other stakeholders to translate requirements into scalable solutions.
- Drive adoption of modern ETL/ELT, workflow orchestration, and data management practices.
Required Skills & Experience
- Proven hands-on experience with Apache Airflow and complex DAGs.
- Strong knowledge of DBT (Data Build Tool).
- Strong SQL and relational database experience.
- Strong understanding of ETL/ELT concepts and data pipeline architecture.
- Hands-on experience with AWS data and infrastructure services (ECS, Fargate, EC2, EKS).
- Experience with Git and version control.
- Proven experience mentoring and leading technical teams.
AWS Infrastructure Responsibilities
- Manage and optimize AWS infrastructure supporting Apache Airflow.
- Ensure Airflow infrastructure is scalable, reliable, secure, and cost-effective.
- Troubleshoot infrastructure, deployment, and performance issues.