[Skip to main content](#main-content) [OpenNash Jobs](https://jobs.opennash.com/) [Hiring report](/reports/hiring/) · [All Amazon jobs](https://jobs.opennash.com/companies/amazon/) · [Search jobs](/search/) Amazon · IN, TS, Hyderabad # AI Benchmarking Lead, Performance Benchmarking Evaluation IN, TS, Hyderabad Manager or lead Retail & Store Listed on Amazon’s official careers site. Posted September 29, 2026. Check that it is still open and apply there. [Apply on Amazon’s careers site →](https://www.amazon.jobs/en/jobs/10563414/ai-benchmarking-lead-performance-benchmarking-evaluation) [OpenNash Jobs](https://jobs.opennash.com/) / [Amazon](https://jobs.opennash.com/companies/amazon/) / AI Benchmarking Lead, Performance Benchmarking Evaluation ## Job description Join our mission-critical team supporting Seller Assistant, Amazon's Gen-AI powered copilot that helps sellers navigate Amazon's complex ecosystem and grow their businesses. As a AI Benchmarking Lead, you'll play a pivotal role in ensuring the reliability and accuracy of AI model evaluations as we scale from 61% to 90%+ active seller coverage worldwide. ### About Seller Assistant Seller Assistant is a conversational AI copilot that understands the full context of a seller's business. It intelligently orchestrates back end tools to deliver actionable, drilled-down responses and can independently complete complex tasks on behalf of sellers with their permission. ### Our Scale and Impact - Expanded to 2.44MM sellers (45x growth vs. Dec 2024) - Currently serving 61% of active sellers worldwide across 9 international stores (CN2XX, IN, UK, DE, JP, BR, MX, AE, SA) - Supporting four languages: English, Chinese, German, and Japanese - 2026 Goal: Scale to 90%+ active sellers WW with 5 new store launches (France, Italy, Spain, Canada, Australia) As a AI Benchmarking Lead, you will benchmark Seller Assistant AI models for relevancy, correctness, and completeness. Your primary responsibilities include: 1) - Evaluate audits performed by the core auditing team to increase confidence in evaluation metrics, 2) - Improve audit reliability and consistency through systematic measurement of auditor accuracy,3) - Conduct targeted calibration to ensure quality standards across the auditing function, 4) - Enforce quality standards by quality-checking audits and providing actionable feedback to team members, 5) - Drive continuous improvement in audit processes and methodologies. - You conduct quality checks on audits performed by the core auditing team. - You identify rubric gaps and evaluation ambiguities that lead to inconsistent audit outcomes. - You surface high-confidence product issues earlier by validating and categorizing model failures. - You serve as point of contact for annotation tasks across ML data process areas, ensuring quality execution and delivery - You understand dependencies across ML data workflows and articulate customer impact effectively - You modify existing annotation methods and update SOPs. - You document SOP changes, secure approval, share knowledge with the team, and audit adoption and execution - You test new SOPs and tools, providing feedback on quality and improvement recommendations to support onboarding ### Key job responsibilities - You structure data collection, analyse results and share inputs for SOP changes. - You collate, track, and report progress on key metrics agreed to with respective stakeholders (e.g., Program managers, Applied Scientist) specific to your functional area. - You identify operational issues related to process and tooling and recommend suggestions to improve key project metrics such as productivity and quality. ## At a glance - Posted by the employer: Sep 29, 2026 - First seen by OpenNash: Oct 11, 2026 - Last checked: Oct 18, 2026 - Listed for: 19 days - Location: IN, TS, Hyderabad[Apply on Amazon’s careers site →](https://www.amazon.jobs/en/jobs/10563414/ai-benchmarking-lead-performance-benchmarking-evaluation) ## More jobs at Amazon ### [AHS Specialist - English & German, Account Health Support](https://jobs.opennash.com/companies/amazon/jobs/93d59a604529cb8e8bd6/) IN, TS, Hyderabad ### [AHS Specialist - Italian & English, Account Health Support](https://jobs.opennash.com/companies/amazon/jobs/e8288c053090b82a9415/) IN, TS, Hyderabad ### [Program Manager I, GCOI](https://jobs.opennash.com/companies/amazon/jobs/b6c399f9dea9e826f010/) IN, TS, Hyderabad ## Similar jobs at other companies ### [Experienced Engineer – Power, Performance Benchmarking and Competitive Analysis](https://jobs.opennash.com/companies/qualcomm/jobs/53038a248c722ed3134f/) Qualcomm · Guangdong, China ### [Staff Engineer, Product Validation Engineering ( CPU / GPU / NPU) Performance & Benchmarking, 9-13 years, Bangalore](https://jobs.opennash.com/companies/sandisk/jobs/3809ff86925a71e98567/) Sandisk ### [Intern - GenAI Benchmarking (MLE) / On-Device Model Deployment (SWE)](https://jobs.opennash.com/companies/qualcomm/jobs/2753c263b449995386e2/) Qualcomm · Seoul, Korea, Republic of ### [Thesis Work on Benchmarking of grid-forming converter control methods](https://jobs.opennash.com/companies/abb/jobs/d4aee5b1ce93ab5c6bc5/) ABB · Vaesteras, Vastmanland County, Sweden ### [Group Strategy & Benchmarking – Internship Opportunity 2026](https://jobs.opennash.com/companies/unicredit/jobs/a8363f422959723536c5/) UniCredit [Retail & Store jobs](/functions/retail-store/) OpenNash lists jobs from employers’ own careers sites and does not handle applications. Dates come from the employer where stated; “first seen” and “last checked” are our own observations. OpenNash Jobs · Official listings with dates and links to the employer. [Weekly hiring report](/reports/hiring/) · [Methodology](/methodology/)