Senior Data Engineer - Databricks Data Migration
0 viewsEPAM Systems IncAbu DhabiOnsite
5+ years
Posted 4 days ago
- Devops
- Platform Engineering
- Spark
EPAM Systems in Abu Dhabi, UAE, is seeking a Senior Data Engineer to join onsite. You will lead production-grade data pipelines on Databricks using PySpark and SQL, migrating transformation logic from a legacy platform, and ensuring data quality across the data platform.
You will optimize performance through partitioning and Delta Lake, design validation processes, review code, mentor junior engineers, and collaborate in CI/CD and Agile delivery.
Requirements:
- Lead production-grade data pipelines on Databricks using PySpark and SQL
- Migrate transformation logic from a legacy platform
- Ensure data quality across the data platform
- Optimize performance through partitioning and Delta Lake
- Design data validation processes
- Review code
- Mentor junior engineers
- Collaborate in CI/CD and Agile delivery
- Work onsite in Abu Dhabi, UAE