Senior Data Engineer
1 view7+ years
Posted 4 days ago
- Airflow
- GCP
- Git
- Kafka
- Python
- Spark
- Terraform
About the Client
A FTSE 250 global fintech company headquartered in London with a presence in 18 countries and five continents. Their award-winning products and platforms empower go-getters worldwide, giving them access to over 19,000 financial markets.
About the Role
Your task will be to run a project generating risk reports important for the entire company. The tool is currently under consideration for a complete renovation. The main scope of responsibilities is the implementation of new functionalities, adding new reports, writing tests, pipeline analysis and active participation in the team's scrum life. Technologies used: Python, pandas, numpy, Kafka, Airflow, Rest, GCP, bash. Candidates do not need to have big data knowledge or a lot of cloud experience - it is important that you are willing to expand your skills!
What You Will Do
You will be responsible to ensure a high quality deliverable within Data Engineering Team.
- Adding new reports
- Writing tests
- Building deployment pipeline
- Analyze and design technical solutions
- Actively participate in the team's scrum life
Requirements
You will be someone who has 7+ years of experience on the below areas:
- Strong experience in Python
- Data processing (pandas/pyspark)
- Good experience of SQL and GCP
- Understanding development life-cycle (CI/CD, Gitlab)
- Unit testing
- DBT
- Cloud
Nice to Have
- Apache Kafka
- Apache Spark
- Message formats AVRO, Parquet, ORC, JSON
- Terraform
Perks & Benefits
- Flexible working hours and work-from-home opportunities
- Performance-related bonuses
- Insurance and medical plans
- Technical and leadership training (online and in-class)
- Contribution to gym memberships and more
- Extra paid time off (birthday leave, volunteering leave)