[Skip to main content](#main-content) [OpenNash Jobs](https://jobs.opennash.com/) [Hiring report](/reports/hiring/) · [All Royal Bank of Canada jobs](https://jobs.opennash.com/companies/royal-bank-of-canada/) · [Search jobs](/search/) Royal Bank of Canada # Staff Data Scientist, AI Evaluations Platform Senior individual contributor Data & Analytics Listed on Royal Bank of Canada’s official careers site. Posted October 10, 2026. Check that it is still open and apply there. [Apply on Royal Bank of Canada’s careers site →](https://rbc.wd3.myworkdayjobs.com/RBCGLOBAL1/job/6666-RUE-ST-URBAIN-FLOOR-3MONTRAL/Staff-Data-Scientist--AI-Evaluations-Platform_R-0000188822) [OpenNash Jobs](https://jobs.opennash.com/) / [Royal Bank of Canada](https://jobs.opennash.com/companies/royal-bank-of-canada/) / Staff Data Scientist, AI Evaluations Platform ## Job description ### Job Description What is the opportunity? RBC’s AI Group is building trusted AI capabilities for the enterprise, and evaluation is one of the core controls that makes that possible. As Staff Data Scientist, AI Evaluations, you will be a senior technical leader in the data science function responsible for how RBC measures model and agent quality, safety, risk, and performance. You will design, build, and continuously improve evaluation datasets, sourcing methods, LLM judge design, deterministic scorers, human evaluation protocols, quality benchmarks, and measurement frameworks that help AI systems move from experimentation to production with evidence and control, while providing technical guidance and mentorship to other data scientists on the team. What will you do? Design and drive advanced model and agent evaluation methodologies, including evaluation datasets, rubrics, LLM-as-judge methods, deterministic scorers, human evaluation, and measurement frameworks, providing technical guidance to junior team members. Define evaluation science standards that translate model risk, responsible AI, product quality, safety, and business expectations into measurable criteria, repeatable methods, and clear evidence. Own the end-to-end lifecycle for evaluation datasets and scorecards, including sourcing, curation, validation, quality checks, versioning, lineage, reuse, and ongoing improvement. Design scalable evaluation approaches for generative AI and agentic systems, including task-level, workflow-level, trajectory-level, and runtime evaluation methods. Partner with AI research, platform engineering, product, risk, governance, and business teams to embed evaluations into build, release, certification, monitoring, and recertification workflows. Establish human evaluation and review protocols that produce reliable labels, reviewer guidance, adjudication processes, quality controls, and audit-ready evidence. Measure and improve scorer accuracy, calibration, robustness, failure-mode coverage, and explainability across automated and human evaluation approaches. Provide clear technical leadership, executive-ready communication, and mentorship to junior data scientists to help RBC scale trusted AI with speed, rigor, and control. In this role, you will communicate and interact frequently with RBC partners and/or employees located across Canada and/or worldwide What do you need to succeed? Must Have 8+ years of experience in data science, applied machine learning, AI evaluation, ML quality, or a related technical field, including experience providing technical leadership and mentorship within a team. Strong experience designing evaluation frameworks for ML, generative AI, or agentic AI systems, including metrics, datasets, benchmarks, rubrics, and quality measurement. Practical experience with LLM evaluation methods such as LLM-as-judge, deterministic scoring, human evaluation, hallucination assessment, factuality assessment, safety evaluation, or model quality benchmarking. Strong technical foundation in data science, statistics, machine learning, experimentation, data curation, Python, SQL, and modern AI/ML development practices. Proven ability to translate governance, model risk, responsible AI, and business requirements into measurable controls, repeatable evaluation processes, and decision-ready evidence. Strong communication and stakeholder management skills, with the ability to influence senior leaders across research, engineering, product, governance, risk, and business teams. ### Nice to Have Experience evaluating agentic AI systems, tool-calling workflows, multi-step reasoning, runtime traces, trajectory scoring, or workflow-level performance. Experience in financial services, regulated AI, model risk management, responsible AI, enterprise governance, or audit-ready evidence processes. Familiarity with tools and platforms such as MLflow, Langfuse, LangSmith, OpenTelemetry, Grafana, CI/CD pipelines, or comparable evaluation and observability tooling. Publications, patents, open-source contributions, or industry work related to AI evaluation, ML quality, AI safety, applied research, or responsible AI. Ph.D. or Master’s degree in a quantitative or technical field, such as Applied Mathematics, Engineering, Computer Science or a related computational discipline. What’s in it for you? We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual. A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable Leaders who support your development through coaching and managing opportunities - Ability to make a difference and lasting impact - Work in a dynamic, collaborative, progressive, and high-performing team A world-class training program in financial services Opportunities to do challenging work. #LI-Post #TECHPJ ### Job Skills Big Data Analytics, Critical Thinking, Decision Making, Industry Knowledge, Machine Learning (ML), Results-Oriented, Software Engineering, Software Product Design Additional Job Details Address: 6666 RUE ST URBAIN, FLOOR 3:MONTRÉAL City: Montréal Country: Canada Work hours/week: 37.5 Employment Type: Full time Platform: TECHNOLOGY AND OPERATIONS Job Type: Regular Pay Type: Salaried Posted Date: 2026-10-09 Application Deadline: 2026-10-31 Note: Applications will be accepted until 11:59 PM on the day prior to the application deadline date above Our Employment Opportunities At RBC, we are guided by living shared values of Client First, Integrity, Collaboration, Respect and Excellence and winning together as One RBC. We believe an inclusive workplace that has diverse perspectives is core to our continued growth as one of the largest and most successful banks in the world. Maintaining a workplace where our employees feel supported to perform at their best, effectively collaborate, drive innovation, and grow professionally helps to bring our Purpose to life and create value for our clients and communities. RBC strives to deliver this through policies and programs intended to foster a workplace based on respect, belonging and opportunity for all. Join our Talent Community Stay in-the-know about great career opportunities at RBC. Sign up and get customized info on our latest jobs, career tips and Recruitment events that matter to you. Expand your limits and create a new future together at RBC. Find out how we use our passion and drive to enhance the well-being of our clients and communities at jobs.rbc.com. RBC is presently inviting candidates to apply for this existing vacancy. Applying to this posting allows you to express your interest in this current career opportunity at RBC. Qualified applicants may be contacted to review their resume in more detail. ## At a glance - Posted by the employer: Oct 10, 2026 - First seen by OpenNash: Oct 12, 2026 - Last checked: Oct 19, 2026 - Listed for: 9 days[Apply on Royal Bank of Canada’s careers site →](https://rbc.wd3.myworkdayjobs.com/RBCGLOBAL1/job/6666-RUE-ST-URBAIN-FLOOR-3MONTRAL/Staff-Data-Scientist--AI-Evaluations-Platform_R-0000188822) ## More jobs at Royal Bank of Canada ### [Associate Director, Counterparty Credit Risk Models, Global Risk Analytics](https://jobs.opennash.com/companies/royal-bank-of-canada/jobs/d3a1bfc1b196ca27a079/) TORONTO, Ontario, Canada ### [Associate Director, Finance & Risk Data Management Office, Chief Data Office](https://jobs.opennash.com/companies/royal-bank-of-canada/jobs/7d0d3ade6fef36a1c570/) TORONTO, Ontario, Canada ### [Associate Director, GRA - Market Risk Analytics](https://jobs.opennash.com/companies/royal-bank-of-canada/jobs/e6e9609a8e9eba6dc333/) TORONTO, Ontario, Canada ### [Associate, Market Data Governance - COO Group](https://jobs.opennash.com/companies/royal-bank-of-canada/jobs/04a67593cf6c4f10bc32/) Jersey City, New Jersey, United States of America ### [Client Intelligence Data & Analytics](https://jobs.opennash.com/companies/royal-bank-of-canada/jobs/75f2a18872a6817b7113/) Ioi Resort City Ioi City Tower One:Putrajaya ## Similar jobs at other companies ### [Staff Data Scientist – AI/ML, Generative AI & Agentic Systems](https://jobs.opennash.com/companies/sandisk/jobs/4289e65697b140f8232f/) Sandisk ### [Staff Product Data Scientist - Developer Productivity](https://jobs.opennash.com/companies/shopify/jobs/72aa5daa94f5b365e2f9/) Shopify ### [Staff Scientist - Pharma](https://jobs.opennash.com/companies/thermo-fisher-scientific/jobs/ad49cd098162ecfa8ee2/) Thermo Fisher Scientific · Athlone, Ireland ### [Staff Scientist, Molecular Biology](https://jobs.opennash.com/companies/thermo-fisher-scientific/jobs/7ed0afb04f57f264b711/) Thermo Fisher Scientific · Singapore OpenNash lists jobs from employers’ own careers sites and does not handle applications. Dates come from the employer where stated; “first seen” and “last checked” are our own observations. OpenNash Jobs · Official listings with dates and links to the employer. [Weekly hiring report](/reports/hiring/) · [Methodology](/methodology/)