Lead Data Engineer - ETL Java AI

Job description

We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible. As a Lead Software Engineer at JPMorganChase within the Consumer & Community Banking Marketing Technology team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives.

Job responsibilities

• Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or breakdown technical problems
• Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.
• Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation
• Lead architecture and hands-on delivery of large-scale data pipelines (ETL/ELT) covering ingestion, transformation, validation, reconciliation, and publishing across curated layers
• Design and operationalize data lake patterns, including partitioning strategies, data quality controls, lineage, governance, and reusable datasets/data products
• Build and optimize distributed processing workloads using Apache Spark and modern storage/file formats (e.g., Parquet, Avro)
• Drive performance tuning across compute and storage (e.g., Spark tuning: shuffle, joins, caching, skew handling; and warehouse tuning where applicable)

Implement engineering best practices

code quality, automated testing, CI/CD, observability (metrics/logs/traces), security-by-design, and operational readiness

• Build and maintain Java-based services and components that support data workflows (e.g., ingestion services, orchestration helpers, APIs, data access layers) Develop REST APIs and integration components to enable downstream consumption and platform interoperability
• Apply modern application engineering practices (clean architecture, SOLID principles, test automation, and secure coding practices) Required qualifications, capabilities, and skills Formal training or certification on software engineering concepts and 5+ years applied experience
• Demonstrated experience leading effective use of approved AI-assisted software development tools (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security.
• Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching engineers on safe, compliant adoption within delivery practices
• 10+ years of software engineering experience, with strong depth in data engineering/big data platforms
• Strong hands-on development experience in Java plus proficiency in at least one data-focused language such as Python
• Proven experience designing and building robust ETL/ELT pipelines and data integration frameworks
• Strong experience with Apache Spark and distributed processing concepts (fault tolerance, partitioning, performance tuning)
• Strong understanding of data storage serialization and formats such as Parquet and Avro
• Solid knowledge of data lake/lakehouse concepts and patterns (e.g., Medallion architecture), including data quality, metadata, and governance considerations Experience working with cloud data warehouses such as Snowflake (loading/unloading patterns, performance basics) or equivalent
• Ability to lead technical discussions, communicate clearly to varied stakeholders, and drive delivery in cross-functional environments Preferred qualifications, capabilities, and skills Experience with orchestration and workflow scheduling tools (e.g., Airflow, Dagster, Control-M, etc.) Exposure to streaming/event-driven processing (e.g., Kafka, Spark Structured Streaming) and incremental processing patterns (CDC, upserts) Experience with table formats / ACID layers (e.g., Delta Lake, Apache Iceberg, Hudi).

Strong production operations mindset (on-call practices, SLAs/SLOs, observability, incident response). Experience in regulated environments (risk, audit, compliance, privacy, PII handling). Experience with cost optimization / FinOps practices for data platforms. Experience with working with Agentic AI / Gen AI solutions and capabilities will be an added advantage.