Jobs by Fastorial
Back to all jobs

Data Engineer (Azure)

0 views
RapidBrainsBengaluruHybrid

5-8 years

Posted 1 day ago

  • Azure
  • Devops
  • FastAPI
  • Git
  • Jenkins
  • Oracle
  • Python
  • Spark

Job Description

We are looking for an experienced Data Engineer (Azure) with strong expertise in Python, PySpark, Azure Databricks, and cloud-based data engineering solutions. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data pipelines, data processing workflows, and data infrastructure using Azure cloud services.

Experience: 5+ Years

Location : Bangalore/Pune/Gurgaon

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Azure Databricks, Python, PySpark, and Spark SQL.
  • Implement data engineering solutions using Azure Data Factory, Azure Data Lake Storage Gen2, Azure Functions, Microsoft Fabric, and Azure services.
  • Develop, optimize, and maintain ETL/ELT workflows and data processing pipelines.
  • Build and maintain APIs using Python frameworks such as FastAPI and REST APIs.
  • Develop, configure, and maintain data models, data quality rules, and data warehouse solutions.
  • Work with SQL, Microsoft SQL Server, Oracle, RDBMS, and NoSQL databases.
  • Analyze and optimize existing PySpark, Python, and SQL code to improve performance and reduce execution time.
  • Implement CI/CD pipelines and follow DevOps practices using Git, Jenkins, and related tools.
  • Troubleshoot data-related issues and support stakeholders with data infrastructure requirements.
  • Maintain technical documentation covering data pipelines, architecture, development processes, and data governance.
  • Perform unit testing, integration testing, quality checks, and ensure compliance with SLA/TAT requirements.
  • Collaborate with business and technical stakeholders to understand requirements and deliver effective data solutions.

Required Skills & Qualifications

  • 5 - 8 years of experience in Data Engineering or software development, with a strong focus on data projects.
  • Strong hands-on experience with Azure Databricks, Python, PySpark, and Spark SQL.
  • Experience with Azure Data Factory, Azure Data Lake Storage Gen2, Azure Functions, Azure Databases, and Microsoft Fabric.
  • Strong SQL skills and experience working with SQL Server, Oracle, RDBMS, and NoSQL databases.
  • Experience developing APIs using FastAPI or similar Python frameworks.
  • Good understanding of ETL/ELT processes, data warehousing, data modeling, and data quality.
  • Familiarity with Git, Jenkins, CI/CD pipelines, and DevOps practices.
  • Experience in data pipeline development, performance optimization, and testing.
  • Strong communication, problem-solving, and stakeholder management skills.