Senior Research Scientist, Multimodal Foundation Models and Robotics

3 Weeks ago • 5 Years + • Artificial Intelligence • Research & Development • $184,000 PA - $356,500 PA

Job Summary

Job Description

NVIDIA seeks a Senior Research Scientist to contribute to the development of humanoid robot foundation models and systems within their Generalist Embodied Agent Research (GEAR) group. Responsibilities include designing and implementing AI algorithms and models for general-purpose robots and embodied agents; developing large-scale AI training and inference methods for foundation models; optimizing and deploying AI models in simulation and on robot hardware; and collaborating with research and engineering teams. The ideal candidate will possess a Ph.D. in a relevant field and extensive experience in multimodal foundation models and robotics, including hands-on experience with robot learning, reinforcement learning, and various programming languages and frameworks.
Must have:
  • Ph.D. in relevant field
  • 5+ years experience
  • Multimodal Foundation Models expertise
  • Robotics experience (RL, imitation learning)
  • Programming skills (Python, C++, ROS)
  • Large-scale AI system experience
Good to have:
  • C++ and CUDA proficiency
  • Experience with MuJoCo and Isaac Sim
  • Robot hardware design experience
Perks:
  • Equity
  • Benefits

Job Details

We are now looking for a Senior Research Scientist focused on Multimodal Foundation Models and Robotics! NVIDIA is searching for an outstanding research scientist to build humanoid robot foundation models and systems in the Generalist Embodied Agent Research (GEAR) group. Everything that moves will eventually be autonomous. Our mission is to build general-purpose embodied agents that learn to explore and master complex skills across the virtual and the physical world.

You will work with an amazing and collaborative research team that consistently produces influential works on multimodal foundation models, large-scale robot learning, game AI, and physical simulation. Our past projects include Eureka, VIMA, Voyager, MineDojo, MimicPlay, Prismer, and more. One of our team’s most recent milestones includes Project GR00T, a foundation model for humanoid robots. Your contributions will have a significant impact on our moonshot research projects and product roadmaps.

What you will be doing:

  • Design and implement novel AI algorithms and models for general-purpose humanoid robots and embodied agents;

  • Develop large-scale AI training and inference methods for foundation models;

  • Optimize and deploy AI models in physical simulation and on robot hardware;

  • Collaborate with research and engineering teams across all of NVIDIA to transfer research to products and services.

What we need to see:

  • A Ph.D. in Computer Science/Engineering, Electrical Engineering, etc., or equivalent research experience.

  • 5 years of relevant work/research experience across one or both of these fields:

    • Multimodal Foundation Models

      • Hands-on training experience and publications in at least one of the following topics: LLMs; Large vision-language models; Video generative models and diffusion algorithms; or Action-based transformers.

      • Outstanding engineering skills in rapid prototyping and model training frameworks (PyTorch, Jax, Tensorflow, etc.). Python is required; C++ and CUDA proficiencies are a big plus;

      • Excellent skills in working with large-scale machine learning/AI systems and compute infrastructure.

    • Robotics:

      • Hands-on training experience and publications in robot learning, such as reinforcement learning, imitation learning, classical control methods, etc. 

      • Strong programming skills in Python, C++,  ROS, and machine learning frameworks like PyTorch.

      • Deep understanding of robot kinematics, dynamics, and sensors;

      • Ability to safely operate robot hardware, lab equipment, and tools;

      • Knowledge of control methods, including PID, model predictive control, and whole-body control;

      • Familiarity with physics simulation frameworks such as MuJoCo and Isaac Sim;

      • Robot hardware design and hands-on building experience.

NVIDIA is widely considered to be one of the technology world's most desirable employers. We have some of the most forward-thinking and productive people in the world. Please join us and be part of the forefront of developing general-purpose robots and embodied agents!

The base salary range is 184,000 USD - 356,500 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.

You will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Similar Jobs

YallaPlay - Production Lead

YallaPlay

(Remote)
1 Week ago
Cirrus Logic - Device Characterization and Spice Modeling Engineer

Cirrus Logic

Edinburgh, Scotland, United Kingdom (Hybrid)
5 Months ago
Life church - Support Operations Specialist

Life church

Edmond, Oklahoma, United States (On-Site)
6 Months ago
Unity - Senior Technical Artist, Tools & UI

Unity

Singapore, Singapore (On-Site)
1 Month ago
Haptic - Lead VFX Artist

Haptic

Paris, Île-de-France, France (Remote)
4 Months ago
Meta - Research Scientist Intern, Language and Multimodal Research for MetaAI (PhD)

Meta

Seattle, Washington, United States (On-Site)
5 Months ago
Hedra - Research Scientist

Hedra

San Francisco, California, United States (On-Site)
1 Month ago
Google - Image Processing Engineer, Display Metrology

Google

Fremont, California, United States (On-Site)
1 Week ago
Inworld AI - Senior Product Manager, AI Engine - USA

Inworld AI

Mountain View, California, United States (On-Site)
6 Months ago
NVIDIA - Senior Developer Relations Manager - Robotics

NVIDIA

Tokyo, Japan (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Garena - Game Developer

Garena

Hanoi, Hanoi, Vietnam (On-Site)
3 Months ago
Ubisoft - The Division Resurgence – Lead Level & Quest Designer (W/M/NB)

Ubisoft

Saint-Mandé, Île-de-France, France (On-Site)
3 Months ago
Appirits - 2D Illustrator

Appirits

Shibuya, Tokyo, Japan (On-Site)
1 Month ago
Ubisoft - The Division Resurgence – Generalist Developer

Ubisoft

Saint-Mandé, Île-de-France, France (Hybrid)
3 Months ago
Google - Hardware Test Engineer, Platforms Infrastructure Engineering

Google

Sunnyvale, California, United States (On-Site)
2 Weeks ago
Viziverse - Game Designer

Viziverse

(Remote)
7 Hours ago
Playrix - Lead Unity Software Engineer (Gameplay)

Playrix

Ukraine (Remote)
6 Months ago
Rivos - Analog Mixed Signal Design

Rivos

Hsinchu, Hsinchu City, Taiwan (Hybrid)
6 Months ago
TT Games - Senior Game Programmer

TT Games

Knutsford, England, United Kingdom (Hybrid)
3 Months ago
Riot Games - Software Engineering Intern

Riot Games

Sydney, New South Wales, Australia (Hybrid)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Santa Clara, California, United States

Flow - Senior/Staff Backend Software Engineer

Flow

Miami, Florida, United States (Hybrid)
6 Months ago
Anavation - Atlassian Subject Matter Expert

Anavation

Colorado Springs, Colorado, United States (Remote)
1 Week ago
Nintendo - Brand Ambassador

Nintendo

Dallas, Texas, United States (On-Site)
7 Months ago
Epic Games - Senior Music Designer

Epic Games

Cary, North Carolina, United States (On-Site)
7 Months ago
ByteDance - Finance Business Partner - Global Monetization

ByteDance

New York, New York, United States (On-Site)
2 Days ago
Biofire DX - Associate Customer Care Specialist

Biofire DX

Morrisville, North Carolina, United States (On-Site)
1 Week ago
Riot Games - Principal Researcher - Central User Research

Riot Games

Los Angeles, California, United States (On-Site)
2 Weeks ago
Meta - Research Scientist, Computer Vision for Generative AI (PhD)

Meta

Menlo Park, California, United States (On-Site)
5 Months ago
ByteDance - Backend Software Engineer - Global E-Commerce Supply Chain

ByteDance

Seattle, Washington, United States (On-Site)
6 Months ago
Google - Structural Simulation Engineer

Google

Mountain View, California, United States (On-Site)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Artificial Intelligence Jobs

Google - Technical Program Manager III, NPI System Software, Cloud AI Systems

Google

Sunnyvale, California, United States (On-Site)
2 Weeks ago
Hitachi - Artificial Intelligence - JBU

Hitachi

Chennai, Tamil Nadu, India (On-Site)
6 Months ago
Luxoft - Regular Data Engineer

Luxoft

(Remote)
5 Months ago
Google - Conversational AI Consultant

Google

Bengaluru, Karnataka, India (On-Site)
2 Days ago
ION - Senior AI Engineer, Italy

ION

Pisa, Tuscany, Italy (On-Site)
6 Months ago
Google - Staff Software Engineer

Google

Warsaw, Masovian Voivodeship, Poland (On-Site)
2 Weeks ago
Tencent - Senior Researcher: Artificial General Intelligence (Natural Language Processing)

Tencent

Washington, United States (On-Site)
2 Months ago
Scale AI - AI Product Manager, Generative AI

Scale AI

San Francisco, California, United States (On-Site)
6 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling the creation of the metaverse. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry.

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

Massachusetts, United States (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

Texas, United States (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (Hybrid)

Santa Clara, California, United States (Hybrid)

View All Jobs

Get notified when new jobs are added by NVIDIA

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug