Jobs by Fastorial
Back to all jobs

Senior Data Engineer - (NPW)

1 view
Milestone Technologies, Inc.KochiHybrid

5+ years

Posted 4 days ago

  • Llm
  • Platform Engineering
  • Python
  • Spark

Role Summary

Owns the Enterprise Knowledge Ingestion Framework: pipeline hardening, chunking, metadata extraction, indexing, versioning, and synchronization extension to new sources; owns integration with the client's operational data platform API and the Planisware field expansion for an upcoming release, and the dual-source retrieval architecture with source attribution and fallback protocol; and owns the enterprise ingestion RAG pipeline and Databricks Enterprise Vector Store integration, supporting the Knowledge Discovery and Tribal Intelligence Suite.

Key Responsibilities

  • Harden the Enterprise Knowledge Ingestion Framework pipeline
  • Implement chunking, metadata extraction, indexing, and versioning
  • Extend synchronization to Teams, Smartsheet, and the client's action-tracking system
  • Build and maintain integration with the client's operational data platform API
  • Deliver Planisware field expansion and verify API completeness
  • Build the dual-source retrieval architecture with source attribution and defined fallback protocols
  • Build and maintain the enterprise ingestion RAG pipeline
  • Own Databricks Enterprise Vector Store integration
  • Build knowledge-base pipelines supporting Knowledge Discovery and Tribal Intelligence Suite

Required Skills and Experience

  • 5+ years data engineering experience, including ingestion pipeline design
  • Experience with chunking and metadata-extraction strategies for RAG pipelines
  • Vector store indexing, versioning, and synchronization experience
  • Strong Python, REST API integration, and ETL pipeline design skills
  • Experience integrating enterprise planning or operational systems
  • Experience designing fallback / source-attribution protocols for dual-source retrieval
  • Hands-on Databricks experience
  • Embeddings and vector-store pipeline management experience
  • RAG integration experience
  • Education: B.Tech/M.Tech in Computer Science, Data Engineering, or a related field, or equivalent practical experience

Preferred Qualifications

  • Prior Smartsheet or Teams API integration experience
  • Direct Planisware integration experience
  • Databricks Enterprise Vector Store production experience specifically