Senior Software Engineer - Automated Parallel Programming

1 Month ago • 4 Years + • Research & Development • Artificial Intelligence • $184,000 PA - $287,500 PA

Job Summary

Job Description

NVIDIA's PyTorch team seeks a Senior Software Engineer to design and build tools for AI practitioners. Responsibilities include crafting code generation systems to accelerate machine learning frameworks, optimizing GPU performance in PyTorch, building production AI solutions, and collaborating with internal and external teams. The role involves working with NVIDIA's nvFuser (parallel programming tool), influencing the software stack (CUDA compiler, Lightning-Thunder Graph Compiler), and advising on future hardware designs. The ideal candidate will have experience with parallel programming (CUDA or similar), C++, large software projects, and strong communication skills. Experience with deep learning compilers, distributed parallelism, and Large Language Models is a plus.
Must have:
  • MS/PhD in CS/related field or equiv. exp.
  • 4+ years C++ experience
  • Parallel programming (CUDA)
  • Large software project experience
  • Excellent communication skills
Good to have:
  • CPU/GPU architecture knowledge
  • Deep learning compiler expertise
  • Optimized distributed parallelism
  • LLM parallelization
  • Heuristic generation (cost models, ML, auto-tuning)
  • Contributions to PyTorch, NumPy, etc.
Perks:
  • Equity
  • Benefits

Job Details

The PyTorch Team @ NVIDIA is hiring passionate parallel programmers. Join us to design and build the tools used by millions of AI practitioners deploying AI applications scalable to thousands of GPUs. Our team is responsible for the continual delivery of best in class experience on NVIDIA's hardware with PyTorch. Join our team and collaborate with many multi-disciplinary engineering teams within NVIDIA and internationally in the PyTorch open source community to deliver our customers the best of NVIDIA software.

In this position you will learn innovative techniques from NVIDIA's domain experts for efficiently programming the world's most sophisticated computer systems. Build these techniques into NVIDIA/Fuser (commonly known as "nvFuser") applying our groundbreaking Parallel Programming Theory, allowing these optimization techniques to be applied to algorithms broadly, automatically, and safely to algorithms written in Numpy and PyTorch. Beyond building nvFuser influence and improve the entire software stack all the way from users to the CUDA compiler, to the Lightning-Thunder Graph Compiler, as well as influence the future design of NVIDIA's hardware platform. Join our ambitious and diverse team who strive to lead the best in AI programming.

What you will be doing:

  • Crafting a code generation system to accelerate portions of a graph collected from a machine learning framework.

  • Partnering with NVIDIA’s hardware and software teams to improve GPU performance in PyTorch.

  • Design, build and support production AI solutions used by enterprise customers and partners.

  • Optimize the performance of influential, modern Deep Learning models coming out of academic and industry research, for NVIDIA GPUs and systems.

  • Collaborating with internal applied researchers to improve their AI tools.

  • Advise design of new hardware generations.

What we need to see:

  • MS or PhD Computer Science, Computer Engineering, Electrical Engineering or a related field (or equivalent experience).

  • Parallel programming experience with writing optimized kernels in the NVIDIA CUDA Programming Language or similar parallel languages

  • 4+ years of experience with C++ programming.

  • Demonstrated experience developing large software projects.

  • We require excellent verbal and written communication skills.

Ways to stand out from the crowd:

  • Proven technical foundation in CPU and GPU architectures, numeric libraries, modular software design.

  • A background in deep learning compilers or compiler infrastructure

  • Expertise with optimized distributed parallelism techniques and it's a bonus if that includes parallelizing Large Language Models!

  • Knowledge of heuristic generation that employs cost models, machine learning, or auto-tuning.

  • Contributions to PyTorch, Numpy, JAX, TensorFlow, OpenAI-Triton, Lightning Thunder, TVM, Halide or similar system.

The base salary range is 184,000 USD - 287,500 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.

You will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Similar Jobs

ByteDance - Engineering Manager Machine Learning Infrastructure

ByteDance

Seattle, Washington, United States (On-Site)
6 Months ago
Eightfold - Staff Engineer-Backend

Eightfold

Bengaluru, Karnataka, India (Hybrid)
7 Months ago
ByteDance - Video Analysis and Quality Algorithm Intern 2023 Summer/Fall (PHD)

ByteDance

Seattle, Washington, United States (On-Site)
6 Months ago
Meta - Research Scientist, Machine Learning (PhD)

Meta

Bellevue, Washington, United States (On-Site)
2 Weeks ago
Jane Street - Machine Learning Engineer

Jane Street

London, England, United Kingdom (On-Site)
6 Hours ago
Riot Games - Senior Researcher, China Studio

Riot Games

Shanghai, Shanghai, China (On-Site)
2 Weeks ago
Cadence - Senior DFT Engineer

Cadence

Bengaluru, Karnataka, India (On-Site)
7 Months ago
NVIDIA - DFT Engineer

NVIDIA

Bengaluru, Karnataka, India (Hybrid)
2 Months ago
Easygo - Software Development Engineer, Frontend - Payments & Fraud

Easygo

Melbourne, Victoria, Australia (On-Site)
1 Month ago
Ubisoft - Lead R&D Programmer - La Forge

Ubisoft

Montreal, Quebec, Canada (Hybrid)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

NVIDIA - Senior Software Engineer - Automated Parallel Programming

NVIDIA

Santa Clara, California, United States (Remote)
3 Months ago
prizepicks - Data Science Engineering Manager

prizepicks

(Remote)
8 Hours ago
Google - Software Engineer III, Distributed Machine Learning, YouTube

Google

Mountain View, California, United States (On-Site)
1 Week ago
Ubisoft - Lead R&D Scientist

Ubisoft

Shanghai, Shanghai, China (On-Site)
2 Weeks ago
Scale AI - Machine Learning Engineer, GenAI Quality

Scale AI

San Francisco, California, United States (On-Site)
1 Day ago
ByteDance - Software Engineer in Large Model System Graduate (Machine Learning Sys-US) - 2024 Start (BS/MS)

ByteDance

San Jose, California, United States (On-Site)
6 Months ago
Attentive - Senior Machine Learning Engineer

Attentive

San Francisco, California, United States (Hybrid)
6 Months ago
Kwalee - Machine Learning Engineer

Kwalee

Royal Leamington Spa, England, United Kingdom (On-Site)
1 Month ago
Google - Staff Software Engineer, Machine Learning Performance, TPU

Google

Mountain View, California, United States (On-Site)
2 Days ago
Genies - Lead Machine Learning Engineer, 3D Gen AI & Graphics

Genies

San Mateo, California, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Jobs in North Carolina, United States

Life church - Product Manager

Life church

Edmond, Oklahoma, United States (On-Site)
6 Months ago
ByteDance - Software Engineer, Cloud Infrastructure

ByteDance

San Jose, California, United States (On-Site)
6 Months ago
Redpoints  - Sales Development Representative (SDR)

Redpoints

New York, New York, United States (Hybrid)
2 Months ago
everi - Technical Artist IV

everi

Austin, Texas, United States (Hybrid)
2 Weeks ago
Google - Senior Silicon Product Engineer

Google

Mountain View, California, United States (On-Site)
1 Week ago
The Walt Disney Company - Principal Software Engineer

The Walt Disney Company

Glendale, California, United States (On-Site)
2 Weeks ago
Guardian - Business Intelligence Analyst IV

Guardian

Bethlehem, Pennsylvania, United States (Hybrid)
4 Hours ago
IGT gaming - End of Game Auditor

IGT gaming

Lakeland, Florida, United States (On-Site)
2 Days ago
Next Level Business Services - Site reliability engineer -SMTP Service Management (Full) Time

Next Level Business Services

Redmond, Washington, United States (On-Site)
6 Months ago
Next Level Business Services - Solution Consultant

Next Level Business Services

Pasadena, California, United States (On-Site)
6 Months ago

Get notifed when new similar jobs are uploaded

Research & Development Jobs

Riot Games - Staff Software Engineer (UI) - League of Legends

Riot Games

Los Angeles, California, United States (On-Site)
3 Months ago
Microsoft - Senior Medical Device Specialist

Microsoft

London, England, United Kingdom (On-Site)
3 Days ago
Meta - Software Engineer, Machine Learning

Meta

Pittsburgh, Pennsylvania, United States (On-Site)
5 Months ago
Assystems - Ingénieur coordination technique H/F

Assystems

Cherbourg-en-Cotentin, Normandy, France (On-Site)
6 Months ago
NVIDIA - Senior Photonic Layout Design Engineer

NVIDIA

Santa Clara, California, United States (On-Site)
3 Months ago
Netflix - Technical Program Manager (L5), Content Promotion & Distribution

Netflix

Los Angeles, California, United States (On-Site)
1 Week ago
Riot Games - Game Designer III - VALORANT, Competitive Systems

Riot Games

Los Angeles, California, United States (On-Site)
7 Months ago
NVIDIA - HPC Operations Manager – Hardware Engineering

NVIDIA

Westford, Massachusetts, United States (On-Site)
2 Months ago
NVIDIA - Senior Technical Instructor - AI and Data Center Infrastructure

NVIDIA

Ra'anana, Center District, Israel (On-Site)
1 Month ago
Google - Staff Engineer Tech Lead, Opinion Rewards, Google Ads

Google

Bengaluru, Karnataka, India (On-Site)
1 Week ago

Get notifed when new similar jobs are uploaded

About The Company

Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling the creation of the metaverse. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry.

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

Massachusetts, United States (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

Texas, United States (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (Hybrid)

Santa Clara, California, United States (Hybrid)

View All Jobs

Get notified when new jobs are added by NVIDIA

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug