Lead Data Platform Engineer

Job description

Experian Credit Services & Verifications are looking for a Lead Data Platform Engineer to join our Ascend Cloud Foundation Platform team. Reporting into our Engineering Director, you will join an established agile team, with a passion and lead a technical team that supports innovation, learning and collaboration. Ensure that the team follows the highest technical standards. Support Agile methods to deliver high value outcomes.

This is a Hybrid, Nottingham based role (40% in the office).

Main responsibilities

• Lead an outstanding Agile technical team Build data pipelines and data lake
• Deliver quality software
• Collaborate with the risk, security and compliance teams to ensure adherence to regulatory requirements (e.g., GDPR, PCI DSS) and industry standards related to data privacy and security
• Understand where there are overlapping technical requirements in your team and other teams and help build out technology roadmaps Codebase ownership Technical Application ownership

Qualifications

• Strong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins , Harness
• Strong experience of developing on Apache Spark/Hadoop
• Strong experience with AWS cloud services including AWS Glue, S3, Step
• Strong experience building data pipelines and data lake Experience of the full development lifecycle/application Lifecycle Management Tools - BitBucket & Jira Significant experience of programming using Scala or Python.

Configuration as code principles and API integration Write clean, scalable and re-usable code that implements SOLID principles, common design patterns where applicable and adheres to published coding standards Experience analyse, investigate and compare large data sets when required Functions, Lambdas, EventBridge, SQS and Lake formation Experience with automated testing methodologies