Software Engineer II

Job description

Design, develop, and maintain AI infrastructure services in Go, Rust, Python, C++, and C#, deployed on large-scale Kubernetes clusters to support inference, pre-training, and post-training workloads for state-of-the-art AI models. Collaborate with engineers, researchers, and external partners to troubleshoot issues, improve reliability, and optimize the performance of large-scale AI training and inference systems. Build and enhance distributed systems that deliver high reliability, low latency, operational efficiency, and strong security across Azure and partner cloud environments.

Develop automation and tooling to improve GPU capacity utilization, streamline fleet operations, and enable efficient scaling of AI infrastructure. Provide operational support, technical leadership, and vision while contributing to the deployment, monitoring, and continuous improvement of engineering systems and practices. Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or

Python OR equivalent experience. 2+ years designing, developing, and shipping high quality software. 2+ years of experience with distributed systems and cloud-based infrastructure. 1+ year of experience with DevOps practices (CI/CD, automated testing, deployment, etc.). 2+ years of software development experience in C#, C++, Python, or similar languages. 2+ years of experience with containerization tools (e.g., Docker, Kubernetes).

Knowledge and hands on experience with production ML systems, large-scale training infrastructure, NCCL, CUDA libraries and tools AIINFRA Software Engineering IC3 - The typical base pay range for this role across the U.S. is USD $102,100 - $202,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $133,800 - $219,200 per year.

Certain roles may be eligible for benefits and other compensation.