Azure Databricks

Job description

Azure Data Lake, Databricks, ADF, PySpark, SQL, Kafka7+ years of experience in data engineering with strong expertise in Azure Data Lake, Databricks, and ADF.

• Hands-on expertise in SQL and PySpark or Scala for building complex ETL/ELT solutions.
• Proven track record with batch & real-time ingestion, Kafka, and large-scale big data processing.

Experience with relational and NoSQL engines

• SQL Server, Oracle, Azure SQL, and Cosmos DB.
• Experience with Informatica PowerCenter, Informatica IDMC, or equivalent cloud data integration tools.

Good-to-Have

• Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data).
• Experience with Azure DevOps (AzDO) and GitHub-based CI/CD pipelines.
• Practical usage of Copilot for data engineering and development productivity.
• Exposure to Generative AI / GenAI use cases in data platforms and analytics.
• Familiarity with Maestro or similar enterprise schedulers, and data

Qualifications

BACHELOR OF ENGINEERING