Senior Staff ML Engineer, AI Safeguards - Moveworks

Job description

• Design, build and ship safeguards in the agent harness for prompt injection, single and multi-turn jailbreaks, toxicity and sensitive topics, and sensitive data exposure
• Build evals that measure safeguard efficacy on both false and true positives, using datasets from customer reports, adversarial testing and production traffic

Work closely with our Security team

• turn their adversarial findings into eval cases and fixes, and build mitigations together that hold when detection misses
• Select, integrate and fine-tune detection models within a stack shared by our commercial and government environments
• Balance quality, latency and cost for controls that run in the request path
• Build configurable safeguard policies so customers can decide how sensitive topics are handled
• Monitor safeguard behavior in production and fix regressions

Qualifications

• Typically 8 or more years building production software, including 3 or more years of applied ML (training, evaluating and deploying models in production systems)
• Hands-on experience developing safeguards for LLMs, such as guardrail models, content classifiers, prompt injection or jailbreak detection, or safety evals
• Strong model evaluation fundamentals for agent trajectories and text classification, and attention to the quality of training and evaluation data
• Working knowledge of how LLM safeguards get bypassed, including indirect prompt injection through tools and documents, and multi-turn jailbreaks
• Excellent Python Experience