LEAD DATA SCIENCE (ML Ops)

Job description

Job Overview

Solves complex problems and help stakeholders make data- driven decisions by leveraging quantitative methods, such as machine learning. It often involves synthesizing large volume of information and extracting signals from data in a programmatic way.

Job Responsibilities

• Architect and evolve the ML platform on Databricks (Unity Catalog, MLflow, Feature Store, Model Serving, Workflows) integrated with AWS services
• Define and enforce standards for CI/CD, testing, model promotion, and reproducibility across teams
• Build automated pipelines for training, validation, deployment, and retraining
• Implement monitoring for model performance, data drift, latency, and cost (Lakehouse Monitoring, CloudWatch, Prometheus/Grafana, or similar)

Own governance

• model lineage, access control, audit trails, and compliance with data and regulatory requirements
• Manage compute and cost optimization (cluster policies, GPU usage, serving autoscaling)
• Mentor engineers, run design and code reviews, and drive incident response and postmortems
• Evaluate and introduce new tooling where it earns its place

Required Skills

Databricks

• Production experience with MLflow (tracking, registry, model serving), Unity Catalog, Delta Lake, and Databricks Workflows/Jobs
• Databricks Asset Bundles or Terraform provider for workspace and job deployment
• Spark/PySpark for scalable data and feature pipelines
• Cluster configuration, policies, and performance tuning AWS

Core services

• S3, IAM, VPC, EC2, ECR, Lambda, CloudWatch, Secrets Manager, KMS
• Databricks-on-AWS networking and security (PrivateLink, cross-account roles, workspace setup)
• Familiarity with SageMaker, EKS, or API Gateway as complements to Databricks serving

Engineering and DevOps

• Strong Python; clean, tested, production-grade code
• Infrastructure as Code (Terraform strongly preferred)
• CI/CD (GitHub Actions, GitLab CI, Jenkins, or Azure DevOps)
• Docker; Kubernetes is a must MLOps practices
• Batch and real-time inference patterns
• Drift detection, A/B and shadow deployment, rollback strategies
• Experiment tracking, model versioning, and reproducibility

LLMOps on Databricks

• Mosaic AI, Vector Search, RAG pipelines, evaluation and guardrails
• Feature store design and streaming (Structured Streaming, Must have: Kafka/Kinesis)
• Databricks or AWS certifications (ML Professional, Solutions Architect)
• FinOps experience and regulated-industry or data-residency exposure