Data Migration Specialist

Job description

Role

Data Migration Specialist.

Experience

• 6+ years experience in large data platform and largescale data migration in Cloudera ADF & Databricks Key Responsibilities (R&R)
• Own end-to-end historical data migration from Cloudera (Hive/HDFS) and legacy systems
• Design and execute one-off bulk history migration
• Design daily incremental loads via ADF, fully metadata driven
• Implement migration using customer ingestion framework or custom framework
• Design restart able, idempotent migration pipelines
• Build reconciliation reports (counts, checksums, balances)
• Develop automated testing packs for HDM (history data migration)
• Handle schema drift, data quality issues, and legacy anomalies
• Support parallel run, cutover, and business signoff
• Act as migration SPOC across platform, ingestion, and business teams Mandatory Skills (Must Have) Azure Data Factory (ADF) – metadata driven pipelines Basic Cloudera working knowledge Cloudera → Cloud migration (mandatory) Large volume historical data migration Azure Databricks (Spark / SQL) Data reconciliation and audit signoff Framework based data ingestion / migration Good to Have Azure Data Box–based migration experience Bulk history extraction using Data Box Landing data into ADLS Gen2 Processing history in Databricks (Delta) Orchestrating incremental via ADF