Research Scientist Intern, On-Device AI Model Optimization (PhD)

Job description

Reality Labs advances Meta's vision for virtual reality (VR) and augmented reality (AR). Delivering compelling experiences on AR/VR devices requires innovation and co-design across the full technology stack, from algorithms to custom silicon. Our teams push the state of the art in computer vision, speech, virtual assistants, machine learning, mixed reality, graphics, displays, sensors, and human body tracking. We are seeking interns with a background in developing efficient machine learning models for AR/VR applications, including computer vision, speech, sensor fusion, natural language processing (NLP), and multimodal AI.

Our team develops high-performing models within tight compute, memory, and power constraints, using optimization techniques such as sparsity, quantization, compression, neural architecture search, and tensor decomposition. Ideal candidates will have practical experience building real-time models that maintain high accuracy under on-device deployment constraints. In this role, you will gain experience across the full stack, from user experiences and algorithms to hardware execution.

You will collaborate with domain experts to understand application challenges and develop state-of-the-art neural networks to address them, then partner with software and hardware teams to deploy these solutions on-device. Our internships range from twelve (12) to twenty-four (24) weeks, with start dates throughout the year.

Responsibilities

Define, plan, and execute cutting-edge deep learning research to advance AR/VR experiences. Develop novel deep learning techniques to achieve state-of-the-art accuracy for large language models (LLMs), multimodal models, and reasoning models within on-device resource and real-time execution constraints. Collaborate with research scientists and software engineers to develop model optimization techniques, such as quantization, compression, and pruning, for vision, speech, user interfaces, and other applications.

Partner with software and hardware engineers to develop trade-off curves between model accuracy and runtime requirements, including latency and energy consumption. Clearly communicate experimental results and recommendations to the team and cross-functional partners.

Qualifications

Currently holds or is pursuing a PhD in Computer Science, Electrical Engineering, or a related field

• Must obtain work authorization in the country of employment at the time of hire and maintain ongoing work authorization during employment Experience developing machine learning solutions Programming experience and experience using deep learning frameworks to prototype ideas A track record of first-author publications at top-tier machine learning or AI conferences, such as NeurIPS, ICML, ICLR, ACL, AAAI, or similar venues Experience training and running inference with LLMs, multimodal models, or reasoning models Experience with one or more model optimization techniques, such as quantization, pruning, distillation, parameter-efficient fine-tuning (PEFT), or speculative decoding, for cloud or on-device AI deployment
• Demonstrated research and software engineering experience through internships, coding competitions, or contributions to open-source repositories, such as those hosted on GitHub