Member of Technical Staff - Applied AI Lead, Health

Job description

Design and oversee evaluation systems that test LLM capabilities in the health domain, including internal benchmarking and regression testing that capture model accuracy, safety and utility - and make sure results are interpreted and clearly communicated to stakeholders. Guide the design of agentic, multi-step systems that combine prompt / context engineering, tool use and retrieval, and champion best practices for building and deploying them reliably at scale.

Improve the internal tooling used to implement, run and analyse evaluations, and the data pipelines - dataset sourcing, curation and synthesis - that feed them. Required Bachelor's or higher degree in Computer Science or a related technical discipline, AND significant Python programming experience / machine learning research. Very strong proficiency with LLM evaluations - demonstrated experience designing, building and running eval pipelines, curating and synthesising datasets, designing automated analyses, and explaining results to internal stakeholders.

Very strong proficiency with LLM orchestration - deep, hands-on experience building with and around LLMs, including prompt / context engineering, tool use, harness engineering, retrieval and agentic, multi-step systems, and building tools to analyse and understand their performance. Proven engineering leadership - 8+ years of software engineering experience, including at least 3 years leading technical teams or projects as a tech lead and/or people manager - mentoring and developing engineers, and guiding a group to deliver high-quality results.

Formal people-management experience is preferred; a strong technical lead ready to step into a TLM role will be considered. 0-to-1 experience with a bias towards shipping and learning while balancing a high-quality bar. Experience collaborating in cross-functional teams, working through ambiguity to deliver high-quality results, and a proven ability to contribute to a positive, inclusive work environment that fosters knowledge sharing and growth.

Experience in healthcare technology, or experience in the health domain. Experience with data engineering - handling text dataset sourcing, curation and processing tasks at scale. Experience translating cutting-edge research into shipped products in a fast-paced, startup-like environment. Passionate about conversational AI and its deployment. Demonstrated written and verbal communication skills, with the ability to work closely with cross-functional teams including product managers, designers and other engineers.

Passion for learning new technologies and staying up to date with industry trends, best practices and emerging patterns in AI.