[Skip to main content](#main-content) [OpenNash Jobs](https://jobs.opennash.com/) [Hiring report](/reports/hiring/) · [All Microsoft jobs](https://jobs.opennash.com/companies/microsoft/) · [Search jobs](/search/) Microsoft # Principal Software Engineer United States, Washington, Redmond, New York, California, Mountain View Senior individual contributor Software Engineering Listed on Microsoft’s official careers site. Posted September 21, 2026. Check that it is still open and apply there. [Apply on Microsoft’s careers site →](https://apply.careers.microsoft.com/careers/job/1970393556955579) [OpenNash Jobs](https://jobs.opennash.com/) / [Microsoft](https://jobs.opennash.com/companies/microsoft/) / Principal Software Engineer ## Job description Sets the technical direction for the Copilot offline evaluation platform and partners with teams across the Copilot organization to turn ambiguous quality questions into rigorous, reproducible evaluations—the scenarios, metrics, and scorecards that gate what ships. Leads the architecture of an AI-first, agentic-first evaluation platform in which autonomous agents are first-class operators, designing the services, pipelines, and tooling that agents can operate, extend, and reason about, with the observability and guardrails required to make agent-driven operation trustworthy. ### Owns the platform's hardest challenge delivering trustworthy results across a deep chain of dependencies pushed far beyond their normal operating envelope, with designs for graceful degradation, intelligent retries, dependency-aware gating, and reliability under sustained and growing load. Defines how Copilot's agentic experiences are evaluated end to end, from multi-turn trajectories and tool selection to task outcomes and modalities such as UX and voice, building the simulation, scraping, and scoring capabilities needed to make agent behavior measurable. Leads by example and mentors engineers across teams to build extensible, maintainable systems, driving modernization of the evaluation runtime and platform so it can scale with rapidly growing demand while improving cost efficiency and latency. Stays at the forefront of agentic systems, large-scale AI evaluation, and autonomous operations, bringing emerging patterns and practices into the organization and sharing that knowledge to raise the bar for how Copilot measures quality. Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience. These requirements include but are not limited to the following specialized security screenings: Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience. Deep experience with large-scale distributed systems and reliability engineering, especially delivering dependable outcomes across many interacting service dependencies operating under heavy or unusual load. Experience building AI-first or agentic systems, and designing systems, APIs, and tooling intended to be operated by autonomous agents, including the observability, guardrails, and safety mechanisms that make agent-driven operation trustworthy. Experience with AI/ML evaluation, measurement, or benchmarking systems, such as LLM-based grading, metric design, or large-scale data and quality pipelines. ## At a glance - Posted by the employer: Sep 21, 2026 - First seen by OpenNash: Aug 21, 2026 - Last checked: Oct 18, 2026 - Listed for: 58 days - Location: United States, Washington, Redmond, New York, California, Mountain View[Apply on Microsoft’s careers site →](https://apply.careers.microsoft.com/careers/job/1970393556955579) ## More jobs at Microsoft ### [Art Director](https://jobs.opennash.com/companies/microsoft/jobs/3d323cdcc6813f949926/) United States, Washington, Redmond, New York, California, Mountain View ### [Sr. Creative Director](https://jobs.opennash.com/companies/microsoft/jobs/fc6fe97a9958d50501cb/) United States, Washington, Redmond, New York, California, Mountain View ### [Backend Software Engineer](https://jobs.opennash.com/companies/microsoft/jobs/8d87de29b261313ed137/) Czech Republic, Multiple Locations ### [Business Program Manager - Capacity Automation & Backend Tools](https://jobs.opennash.com/companies/microsoft/jobs/e7256a1bba75b9768669/) United States, Washington, Redmond, Multiple Locations ### [Compiler Software Engineer](https://jobs.opennash.com/companies/microsoft/jobs/15ff463d0aab951e2430/) Czech Republic, Multiple Locations ## Similar jobs at other companies ### [Principal Software Engineer – Mobile Software Architecture](https://jobs.opennash.com/companies/qualcomm/jobs/914589d5d4db917886f4/) Qualcomm · San Diego, California, United States of America ### [Principal Software Engineer](https://jobs.opennash.com/companies/jpmorganchase/jobs/43ddee4b84a577509be8/) JPMorganChase · Seattle, WA, United States ### [Principal Software Engineer](https://jobs.opennash.com/companies/citigroup/jobs/57f0d1514e469a39bd0b/) Citigroup · Jacksonville Florida United States ### [Principal Software Engineer](https://jobs.opennash.com/companies/aramark/jobs/ab582b5e1a57f5b633f8/) Aramark · Philadelphia, PA, US ### [Principal Software Engineer - Apache Cassandra](https://jobs.opennash.com/companies/jpmorganchase/jobs/a7c527beea02339f82da/) JPMorganChase · Columbus, OH, United States OpenNash lists jobs from employers’ own careers sites and does not handle applications. Dates come from the employer where stated; “first seen” and “last checked” are our own observations. OpenNash Jobs · Official listings with dates and links to the employer. [Weekly hiring report](/reports/hiring/) · [Methodology](/methodology/)