Senior Specialist - Data Engineering
1 view7-11 years
Posted 8 days ago
- Airflow
- AWS
- Azure
- Java
- Kafka
- Python
- Scala
- Spark
About the Role
We are looking for an experienced Senior Specialist - Data Engineering to design and establish scalable Data Lake/Lakehouse architectures for manufacturing and industrial data environments. The role involves building reliable data pipelines, implementing data governance and quality practices, enabling analytics and AI use cases, and assessing the security and compliance of ITAR-controlled data across the data landscape.
Job Description
The position focuses on defining and implementing data lake architecture to support manufacturing analytics, reporting, and applicable AI/ML use cases. The candidate will work with machine, equipment, MES, test, and probe data, ensuring accurate lot-, wafer-, and die-level traceability.
The role also involves evaluating existing data environments, strengthening data quality, lineage, governance, and access controls, and ensuring appropriate handling of ITAR-controlled information. The candidate will collaborate with technical teams and senior stakeholders to document findings, recommend improvements, and deliver scalable data solutions.
Key Responsibilities
- Design and implement scalable Data Lake/Lakehouse architectures across cloud or on-premises environments.
- Build and optimize data pipelines for manufacturing, machine, equipment, MES, test, and probe data.
- Establish data quality, governance, lineage, cataloguing, and traceability practices.
- Develop data processing solutions using SQL, Python, Scala/Java, and Spark.
- Implement data orchestration, streaming, CI/CD, and infrastructure automation.
- Assess ITAR-controlled data classification, segregation, access, and security controls.
- Enable analytics, reporting, and ML/AI use cases within manufacturing data environments.
- Engage with stakeholders, document assessments, and communicate technical recommendations.
Required Skills
- Data Engineering: Strong experience in enterprise data engineering, particularly in manufacturing, semiconductor, or industrial environments.
- Data Lake/Lakehouse: Azure Data Lake, Delta Lake, Iceberg, Parquet-based storage, or equivalent architectures.
- Programming: SQL and Python or Scala/Java.
- Distributed Processing: Apache Spark.
- Cloud Platforms: Azure, AWS, GCP, or on-premises data platforms.
- Orchestration and Streaming: Airflow, Azure Data Factory, Kafka, or Event Hubs.
- DevOps: CI/CD and Infrastructure as Code.
- Data Governance: Data cataloguing, lineage, and data quality practices.
- Analytics and BI: Power BI, Tableau, or Spotfire.
- AI/ML: Understanding of ML/AI data pipelines and manufacturing analytics use cases.
- Security and Compliance: ITAR/EAR export controls, IAM, RBAC/ABAC, encryption, network segmentation, and audit logging.
Mandatory Skills
- Azure Data Lake
- Azure DevOps
Benefits Package
- Competitive salary and benefits package.
- Opportunities for professional growth and development.
- A collaborative and inclusive work environment.
- The opportunity to work on exciting and challenging projects with leading clients.