Junior Data Engineer – Databricks & Data Modeling

Job description

Description and Requirements

Key Responsibilities

• Develop and maintain assigned data pipelines using Azure Databricks, PySpark, Spark SQL, SQL, and Delta Lake, building Bronze/Silver/Gold objects to the program’s medallion standards.
• Assist with converting legacy Informatica mappings, SQL, and stored procedures into Databricks processing, and support migration of inbound/outbound HR integrations (including Oracle HCM feeds).
• Implement source-to-target mappings and transformation specs, and help build logical/physical, dimensional, and analytics-ready models for HR data domains under senior direction.
• Perform data profiling, unit testing, and defect resolution; build data-quality and reconciliation checks and compare legacy vs. target-state outputs during parallel runs.
• Produce clear technical documentation – mappings, data dictionaries, test cases, runbooks, and deployment instructions – following established naming, repository, and version-control standards.
• Follow MetLife requirements for handling sensitive HR data, applying approved access-control, classification, and governance practices via Unity Catalog and related tooling.
• Participate in agile ceremonies, incorporate code-review feedback, escalate unclear requirements and risks to the Senior Data Engineer, and provide overlap with U.S. Eastern Time.

Required Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or related discipline (or equivalent practical experience), with 2+ years in data engineering, ETL, or data-platform delivery.

Azure Databricks & Spark

• 1+ year of practical exposure to Spark, PySpark, Databricks, or a comparable cloud environment, plus working experience with Python and SQL.

Data fundamentals

• Foundational understanding of ETL/ELT, Delta Lake, medallion architecture, data modeling (keys, relationships, normalization, dimensions), and source-to-target mappings.

AI-assisted engineering

• Willingness to use AI/Copilot tooling to accelerate development, testing, and documentation.
• Curious, detail-oriented, and coachable – able to follow technical direction, apply code-review feedback, and produce accurate, well-tested, well-documented work.
• Strong written and verbal communication, comfortable raising questions and blockers clearly within a distributed agile delivery team.

Preferred Qualifications

• Suggested 1–2 years exposure to data migration or ETL modernization (e.g., Informatica-to-Databricks) and building pipelines within a team environment.

Azure ecosystem

• Data Lake Storage, Data Factory, DevOps, Azure SQL/Synapse, Unity Catalog (or Microsoft Purview), and Git-based CI/CD practices.

HR domain data

• Oracle HCM Cloud extracts/PVOs/HDL, or workforce, employee, organization, position, talent, and related HR data.
• Microsoft Azure or Databricks coursework/certification, and experience supporting U.S.-based enterprise clients from an offshore delivery model.
• Experience in insurance, financial services, HR technology, or another regulated, highly governed data environment.