Quality Engineering Lead – Data & Reporting Platforms

Job description

Role Overview

We are seeking a highly skilled and experienced VP, Quality Engineering Lead to define, build, and drive our automated data and report testing strategy. In this role, you will lead the Quality Engineering (QE) initiatives for our next-generation, AI-powered data and reporting ecosystem. As a hands-on leader, you will design robust automated test suites to validate complex data architectures—specifically focusing on data virtualization, massive data federation, data contract testing, and the verification of emerging natural language/conversational AI query interfaces.

You will manage a talented team of quality engineers, establish testing standards, and collaborate closely with engineering, and product teams to ensure high-quality, secure, and performant data and report delivery.

Key Responsibilities

• Test Strategy & Quality Leadership Data & Reporting Test Strategy: Architect and execute a comprehensive, end-to-end automated testing strategy covering data virtualization, federated queries, BI/reporting, and AI-enabled analytical interfaces.

Team Leadership

Lead, mentor, and functionally manage a specialized team of Data & Report Quality Engineers, fostering a culture of modern Quality Engineering (QE) and continuous improvement.

Governance & Compliance

Define operating standards, automated quality gates, and data verification protocols across the analytics and reporting delivery lifecycle.

Stakeholder Management

Own the reporting of quality metrics, pipeline coverage, and test automation maturity to senior global technology and engineering leaders.

• Data Virtualization & Federation Testing Federated Query & Virtualization Validation: Develop automated testing frameworks to validate query execution, latency, and data integrity across massive federated query engines and data virtualization platforms (e.g., Starburst, Trino, Presto, Denodo, Dremio, AWS Athena, or Apache Drill ) connecting dozens of heterogeneous catalogs without physical data movement.

Data Contract & Schema Validation

Implement automated schema validation and data contract testing to ensure curated, virtualized data products strictly adhere to published business definitions and system requirements.

Access Control & Security Testing

Design data-driven security tests to verify that centralized data access governance (e.g., Apache Ranger, role/attribute-based access controls) and data masking are flawlessly applied.

• AI & Conversational Intelligence Testing Natural Language Query Testing: Establish frameworks to test conversational AI interfaces that allow users to query data using natural language.

Validate natural language processing (NLP) models, intent recognition, NLP-to-SQL translation logic, and the accuracy of the underlying datasets returned.

Autonomous Agent Verification

Design testing patterns for non-deterministic AI agents (e.g., automated alerting systems and contextual research assistants), validating logical outputs, threshold actions, and boundary limits.

• Big Data & Reporting Platform Testing Report & Dashboard Verification: Devise automated strategies to test visual correctness, performance, and backend data reconciliation for BI platforms (e.g., Tableau, custom web-based dashboards) during large-scale migration phases of legacy systems (comprising hundreds of reports).

Data Lakehouse & Pipeline Testing

Lead automation efforts validating complex data pipelines across hybrid databases (Oracle, SQL Server) and modern analytical lakehouses.

Data Reconciliation

Design and automate source-to-target data reconciliation, schema drift detection, and data lineage validation to ensure reports match underlying source systems perfectly.

CI/CD & Test Automation Engineering Continuous Quality Pipelines

• Seamlessly integrate data and report automation suites into enterprise CI/CD pipelines (Jenkins, Tekton, GitLab, etc.) to trigger continuous verification with each deployment code path.

Triage & Defect Management

Champion structured defect triage, prioritizations, and root cause analysis across complex, multi-tiered data and reporting infrastructure environments. Technology Skills Required Technical Skillsets Data Virtualization & Federation: Hands-on experience with enterprise data virtualization or query federation platforms, such as Starburst, Trino, Presto, Denodo, Dremio, AWS Athena, or Apache Drill .

BI & Reporting Platforms

Deep expertise in testing BI and reporting platforms (e.g., Tableau, custom web-based dashboards, Aspose, or similar reporting engines).

Database Querying & Testing

Advanced SQL expertise with hands-on experience testing relational databases (Oracle, SQL Server) and NoSQL databases.

Programming Languages

Proficiency in Python or Java to build, maintain, and scale custom test automation frameworks.

API Testing

Strong experience with API testing (REST/SOAP) and data contract validation using tools like Postman, RestAssured, or custom scripts.

CI/CD Integration

Experience integrating automated data test suites into enterprise CI/CD pipelines (e.g., Jenkins, Tekton, GitLab CI) to enable continuous testing.

Test Methodologies

Deep understanding of Agile/Scrum methodologies, functional, integration, regression, and parallel-run testing for large-scale migrations. Preferred / Nice-to-Have Skillsets Modern Lakehouse & Warehouse Platforms: Familiarity with cloud-native data platforms, such as Databricks or Snowflake (experience with Google BigQuery is also valued).

Distributed Data Processing Engines

Familiarity with distributed compute engines, specifically Apache Spark (PySpark, Spark SQL) or Apache Flink for large-scale data processing.

Data Quality Automation

Experience implementing automated data quality frameworks using industry-standard tools such as Great Expectations , dbt test , Soda / SodaCL , or Deequ / PyDeequ .

AI/ML & NLP Testing

Experience testing LLM-backed applications, validating Natural Language-to-SQL engines (e.g., conversational query interfaces), prompt validation, and autonomous agent testing.

Leadership & Methodology Agile QE Leadership

Strong experience running QA cycles within Scrum/Kanban frameworks, managing sprint closures, and collaborating with cross-functional Dev/Product leads.

Test Strategy Design

Proven track record of designing multi-layered testing strategies (unit, integration, regression, system, and regression parallel runs for migrations).

Experience & Qualifications

Total Testing Experience: Minimum 10-12 years of relevant experience in software testing, quality engineering, or data engineering.

Data Automation Experience

Minimum 5-8 years of hands-on experience in automated data testing, ETL testing, or data pipeline quality engineering.

BI & Analytics Verification

Minimum 5-8 years of experience in report/BI testing, data reconciliation, and source-to-target data validation.

Education

Bachelor’s degree in Computer Science, Information Systems, or equivalent engineering field. ------------------------------------------------------ Job Family Group: Technology ------------------------------------------------------ Job Family: Applications Development ------------------------------------------------------ Time Type: Full time ------------------------------------------------------ Most Relevant Skills

Please see the requirements listed above. ------------------------------------------------------ Other Relevant Skills For complementary skills, please see above and/or contact the recruiter. ------------------------------------------------------ Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi . View Citi’s EEO Policy Statement and the Know Your Rights poster.