Jobs by Fastorial
Back to all jobs

Data Engineer

0 views
VaporVMDubaiHybrid

2+ years

Posted 5 days ago

  • AWS
  • Docker
  • Git
  • Kafka
  • Kubernetes
  • Python
  • Spark
  • Terraform

We are seeking an experienced Data Engineer with strong AWS and banking sector experience to join its engineering team in the UAE. The candidate will be responsible for designing, developing, and optimising scalable Lakehouse data architectures and pipelines that support enterprise analytics and machine learning applications. This role requires strong expertise in AWS data services, Databricks, distributed data processing, and modern data engineering practices, alongside an understanding of financial datasets, banking regulations, and enterprise data governance.

Key Responsibilities

  • Data Architecture: Design, develop, and maintain scalable Lakehouse architectures using Databricks and Delta Lake on AWS.
  • Data Pipeline Development: Build and optimise batch and real-time ETL/ELT pipelines using Python, PySpark, SQL, and Apache Kafka to process financial and transactional data.
  • AWS Infrastructure: Develop and manage data solutions using AWS services, including S3, Glue, EMR, Redshift, and Athena.
  • Performance Optimisation: Improve data processing efficiency, query performance, and storage utilisation through partitioning, indexing, and distributed processing techniques.
  • Machine Learning Integration: Collaborate with data science and MLOps teams to develop feature engineering pipelines, manage experiments using MLflow, and support model deployment through AWS SageMaker.
  • Data Governance and Security: Implement data access controls, encryption, monitoring, and audit mechanisms in line with banking security, privacy, and regulatory requirements.
  • Infrastructure Automation: Apply Infrastructure as Code practices using Terraform and manage CI/CD pipelines through GitHub Actions or AWS CodePipeline.
  • Software Engineering Practices: Develop clean, maintainable, and well-tested Python code while following version control and development best practices.

Required Qualifications and Experience

  • Proven experience in data engineering within banking, financial services, or fintech environments.
  • Strong understanding of financial datasets, transactional systems, and banking data governance and regulatory requirements.
  • Extensive hands-on experience with AWS data services, particularly S3, Glue, EMR, and Redshift.
  • Strong experience with Apache Spark, PySpark, and Apache Kafka for large-scale data processing.
  • Demonstrated experience designing and implementing Lakehouse architectures using Databricks and Delta Lake.
  • Solid understanding of data modelling, schema evolution, ACID transactions, and data warehousing principles.
  • Proficiency in Python and SQL. Experience with Scala is an advantage.
  • Practical experience with Docker, Kubernetes (EKS), Terraform, and Git-based workflows.