Sr Software Dev Engineer, SageMaker Training

Job description

Interested in building the distributed systems that let customers customize foundation models at scale? The SageMaker Training and Model Customization team builds the services customers use to fine-tune and post-train models against their own accuracy, reliability, and cost targets. We are looking for a Sr Software Development Engineer to build these capabilities.

Model customization is the fastest-moving layer of the machine learning stack. Post-training has gone from supervised fine-tuning to reinforcement learning against verifiable rewards, and from single-turn tasks to agentic training where a model learns by acting in an environment over long trajectories.

Each shift changes the shape of the workload

reinforcement learning puts an inference engine inside the training loop, and agentic training adds environments and tool calls that move the bottleneck from run to run. You turn each new technique into a capability customers can use, without rebuilding the platform every time the research moves. Underneath, it stays a hard distributed systems problem across large accelerator fleets where a single node failure can halt progression.

As a Senior SDE you own your team's architectural direction, including influencing decisions in systems you depend on but don't control. You lead the team's software development alongside peers on related teams, and you are responsible for the quality of what it ships. You take on problems where the customer case is defined but the technology strategy is not, balancing speed of delivery against the foundation for the future and advocating for the right solution.

You are a key influencer in the team's strategy and goals, you drive adoption of engineering best practices, and you coach and mentor other engineers.

Key job responsibilities

- Design, develop, and operate the distributed services that run large-scale training and reinforcement learning workloads

- Build the scheduling, capacity, and fleet-health systems that place customer jobs onto large GPU clusters and keep them running through hardware failure

- Build the infrastructure that hosts and scales the engines, including multi-tenant GPU sharing across customers

- Partner with Science, Product, and partner service teams to translate evolving model and customer requirements into scalable technical solutions

- Maintain a high operational excellence bar for services in the critical path of customer training workloads

About the team

We build the managed services customers use to customize foundation models. A customer brings a task, a dataset, and a definition of what a good answer is worth, and our services run the post-training loop on their behalf. We own that entire path, from the customer-facing API down to the training-session infrastructure.