Data Engineer
0 views6+ years
Posted today
- Airflow
- GCP
- Git
- Jenkins
- Spark
- Terraform
Locations - Bangalore, Chennai, Hyderabad, Pune, Vadodara
Experience - 6+ yrs
Build and maintain batch and real-time streaming ETL/ELT pipelines using Dataflow (Apache Beam), Pub/Sub , Cloud Storage , and Dataproc (Spark/Hadoop).
Design schemas, partition/cluster tables, and write high-performance SQL in BigQuery (and relational/NoSQL stores like Cloud SQL , Bigtable , or Firestore )
Schedule and monitor workflows using Cloud Composer (Apache Airflow) or Cloud Workflows.
Provision and manage cloud resources as code (e.g., Terraform ) and automate deployments via CI/CD pipelines (e.g., Jenkins, GitHub Actions, Cloud Build)
Core GCP Stack: BigQuery, Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, and Dataproc.
Posted by - Pranali Gajbhiye