AWS Glue Data Engineer - Lakehouse Ingestion
0 viewsVenquisAbu DhabiHybrid
2+ years
Posted 3 days ago
- AWS
- Git
- Spark
- Terraform
About the Role
Venquis is seeking an AWS Glue Data Engineer to join the Data Factory Squad in Abu Dhabi/Dubai. You will build scalable ingestion pipelines, using AWS Glue and PySpark, to migrate source systems into the Lakehouse ingestion zone.
Responsibilities
- Implement CDC, full/delta loads, and historical loading strategies for data migration
- Build and optimize AWS Glue ingestion jobs and Lakehouse pipelines
- Optimize pipeline performance and resource allocations
- Automate infrastructure deployments via Git and Terraform
- Collaborate closely with Data Architects and Data Assurance leads
Requirements
- Hands-on experience building scalable ingestion pipelines using AWS Glue and PySpark
- Experience with CDC, delta loads, and historical batch migration
- Infrastructure as code experience with Terraform and Git
- Familiarity with AWS data services including S3, Athena, and Lambda