Jobs by Fastorial
Back to all jobs

Senior Data Engineer - Databricks Data Migration

0 views
EPAM Systems IncAbu DhabiOnsite

5+ years

Posted 4 days ago

  • Devops
  • Platform Engineering
  • Spark

EPAM Systems in Abu Dhabi, UAE, is seeking a Senior Data Engineer to join onsite. You will lead production-grade data pipelines on Databricks using PySpark and SQL, migrating transformation logic from a legacy platform, and ensuring data quality across the data platform.

You will optimize performance through partitioning and Delta Lake, design validation processes, review code, mentor junior engineers, and collaborate in CI/CD and Agile delivery.

Requirements:

  • Lead production-grade data pipelines on Databricks using PySpark and SQL
  • Migrate transformation logic from a legacy platform
  • Ensure data quality across the data platform
  • Optimize performance through partitioning and Delta Lake
  • Design data validation processes
  • Review code
  • Mentor junior engineers
  • Collaborate in CI/CD and Agile delivery
  • Work onsite in Abu Dhabi, UAE