Jobs by Fastorial
Back to all jobs

Senior Data Engineer - Hadoop, PySpark & Airflow Architect

0 views
Go Ahead ConsultingDubaiHybrid

10+ years

Posted 3 days ago

  • Airflow
  • Git
  • Go
  • Spark

About the Role

Go Ahead Consulting in Dubai, UAE, seeks a Senior Data Engineer to own the design, building, scaling, and maintenance of high-performance big data pipelines, Hadoop clusters, PySpark transformations, and Apache Airflow workflows integrated with GitHub.

Responsibilities

  • Architect, build, and optimize big data pipelines and distributed data processing systems
  • Manage Hadoop clusters, PySpark distributed jobs, and Apache Airflow DAGs
  • Collaborate with data scientists, software engineers, and technical leadership across GCC client engagements
  • Implement data governance, pipeline monitoring, and robust security standards
  • Track and manage code versioning, reviews, and CI/CD via GitHub

Requirements

  • Extensive experience in big data engineering and distributed data platforms
  • Solid proficiency with Hadoop ecosystem tools and PySpark transformations
  • Practical experience orchestrating complex workflows with Apache Airflow
  • Strong version control practices with GitHub and CI/CD pipelines
  • Proven ability to design low-latency, scalable data analytics platforms