Senior Software Engineer - Automated Parallel Programming

2 Months ago • 4 Years + • Research & Development • Artificial Intelligence • $184,000 PA - $287,500 PA

Job Summary

Job Description

NVIDIA's PyTorch team seeks a Senior Software Engineer specializing in automated parallel programming. The role involves designing and building tools for AI practitioners, ensuring optimal performance on NVIDIA hardware. Responsibilities include crafting a code generation system to accelerate machine learning frameworks, collaborating with hardware and software teams to improve GPU performance in PyTorch, optimizing deep learning models, and collaborating with researchers. The ideal candidate possesses strong parallel programming experience (CUDA), C++ expertise, and experience with large software projects. Influence extends across the software stack, from users to the CUDA compiler and hardware design.
Must have:
  • MS/PhD in relevant field or equivalent experience
  • 4+ years C++ programming experience
  • Parallel programming (CUDA)
  • Large software project experience
  • Excellent communication skills
Good to have:
  • CPU/GPU architecture knowledge
  • Deep learning compilers expertise
  • Optimized distributed parallelism
  • LLM parallelization
  • Heuristic generation (cost models, ML, auto-tuning)
  • Contributions to PyTorch, NumPy, etc.
Perks:
  • Equity
  • Benefits

Job Details

The PyTorch Team @ NVIDIA is hiring passionate parallel programmers. Join us to design and build the tools used by millions of AI practitioners deploying AI applications scalable to thousands of GPUs. Our team is responsible for the continual delivery of best in class experience on NVIDIA's hardware with PyTorch. Join our team and collaborate with many multi-disciplinary engineering teams within NVIDIA and internationally in the PyTorch open source community to deliver our customers the best of NVIDIA software.

In this position you will learn innovative techniques from NVIDIA's domain experts for efficiently programming the world's most sophisticated computer systems. Build these techniques into NVIDIA/Fuser (commonly known as "nvFuser") applying our groundbreaking Parallel Programming Theory, allowing these optimization techniques to be applied to algorithms broadly, automatically, and safely to algorithms written in Numpy and PyTorch. Beyond building nvFuser influence and improve the entire software stack all the way from users to the CUDA compiler, to the Lightning-Thunder Graph Compiler, as well as influence the future design of NVIDIA's hardware platform. Join our ambitious and diverse team who strive to lead the best in AI programming.

What you will be doing:

  • Crafting a code generation system to accelerate portions of a graph collected from a machine learning framework.

  • Partnering with NVIDIA’s hardware and software teams to improve GPU performance in PyTorch.

  • Design, build and support production AI solutions used by enterprise customers and partners.

  • Optimize the performance of influential, modern Deep Learning models coming out of academic and industry research, for NVIDIA GPUs and systems.

  • Collaborating with internal applied researchers to improve their AI tools.

  • Advise design of new hardware generations.

What we need to see:

  • MS or PhD Computer Science, Computer Engineering, Electrical Engineering or a related field (or equivalent experience).

  • Parallel programming experience with writing optimized kernels in the NVIDIA CUDA Programming Language or similar parallel languages

  • 4+ years of experience with C++ programming.

  • Demonstrated experience developing large software projects.

  • We require excellent verbal and written communication skills.

Ways to stand out from the crowd:

  • Proven technical foundation in CPU and GPU architectures, numeric libraries, modular software design.

  • A background in deep learning compilers or compiler infrastructure

  • Expertise with optimized distributed parallelism techniques and it's a bonus if that includes parallelizing Large Language Models!

  • Knowledge of heuristic generation that employs cost models, machine learning, or auto-tuning.

  • Contributions to PyTorch, Numpy, JAX, TensorFlow, OpenAI-Triton, Lightning Thunder, TVM, Halide or similar system.

The base salary range is 184,000 USD - 287,500 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.

You will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Similar Jobs

Seedify - AI Product Manager (Web3 Games)

Seedify

Philippines (Remote)
1 Week ago
SparkCognition - Data Scientist

SparkCognition

Bengaluru, Karnataka, India (On-Site)
6 Months ago
ByteDance - Video Analysis and Quality Algorithm Intern 2023 Summer/Fall (MS)

ByteDance

San Jose, California, United States (On-Site)
5 Months ago
ByteDance - Senior Machine Learning Engineer - AML Algorithm

ByteDance

San Jose, California, United States (On-Site)
5 Months ago
NVIDIA - AI Computing Software Development Engineer, TensorRT

NVIDIA

Hsinchu, Hsinchu City, Taiwan (On-Site)
1 Month ago
NVIDIA - Senior System Software Engineer - Tegra

NVIDIA

Canada (On-Site)
1 Month ago
Samsung Semiconductor - Staff Engineer, Formal Verification

Samsung Semiconductor

San Jose, California, United States (Hybrid)
2 Weeks ago
NVIDIA - SOC Clock Distribution Engineer

NVIDIA

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
4 Weeks ago
Google - Staff Software Engineer, Google Cloud Compute

Google

(On-Site)
4 Months ago
Rivos - Silicon DFT - Full time

Rivos

Hsinchu, Hsinchu City, Taiwan (Hybrid)
5 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Netflix - Research Engineer L4/L5 -LLMs for Search, Recommendations, and Personalization

Netflix

Los Gatos, California, United States (On-Site)
5 Months ago
ByteDance - Research Scientist, Multimodality

ByteDance

San Jose, California, United States (On-Site)
5 Months ago
Zazz - Artificial Intelligence Engineer

Zazz

(Remote)
1 Month ago
Canva - Senior Computer Vision Engineer - Photo AI

Canva

Vienna, Vienna, Austria (Remote)
2 Months ago
Stonewall Collision & Auto Painting - Senior Data Scientist

Stonewall Collision & Auto Painting

Vijayawada, Andhra Pradesh, India (On-Site)
6 Months ago
The Walt Disney Company - Lead Software Engineer, Machine Learning - Ad Platforms

The Walt Disney Company

San Francisco, California, United States (On-Site)
4 Months ago
ByteDance - Research Scientist in Foundation Model, Speech Understanding - 2024 Start (PhD)

ByteDance

San Jose, California, United States (On-Site)
5 Months ago
NVIDIA - Senior Software Engineer, AI Resiliency

NVIDIA

Santa Clara, California, United States (On-Site)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Santa Clara, California, United States

Riot Games - Manager, Software Engineering - VALORANT, Live Design

Riot Games

Los Angeles, California, United States (On-Site)
5 Months ago
ByteDance - Software Engineer, Machine Learning Platform

ByteDance

Seattle, Washington, United States (On-Site)
5 Months ago
Next Level Business Services - UI Angular Developer

Next Level Business Services

New York, New York, United States (On-Site)
5 Months ago
ByteDance - Backend Software Engineer - Global E-Commerce Supply Chain Operation Platform

ByteDance

San Jose, California, United States (On-Site)
5 Months ago
ByteDance - Researcher - Interdisciplinary

ByteDance

New York, New York, United States (On-Site)
6 Days ago
Rockstar Games - Senior Software Engineer (C#)

Rockstar Games

Carlsbad, California, United States (On-Site)
6 Months ago
Daybreak Game Company LLC - Associate Character Artist

Daybreak Game Company LLC

San Diego, California, United States (Hybrid)
1 Month ago
Onward Search - Financial Marketing Copywriter

Onward Search

Westwood, Massachusetts, United States (Hybrid)
1 Month ago
Thumbtack - Staff Software Engineer,  Machine Learning Infrastructure

Thumbtack

United States (Remote)
4 Months ago
Samsung Semiconductor - Staff Engineer, DRAM

Samsung Semiconductor

San Jose, California, United States (Hybrid)
6 Days ago

Get notifed when new similar jobs are uploaded

Research & Development Jobs

Meta - Software Engineer, Machine Learning

Meta

Singapore (On-Site)
4 Months ago
NVIDIA - Senior Platform Software Engineer, PCIe

NVIDIA

Canada (On-Site)
1 Week ago
HP - Staff Video Embedded Engineer- Remote (Ireland)

HP

Galway, County Galway, Ireland (Remote)
6 Months ago
NVIDIA - STA Engineer

NVIDIA

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
1 Month ago
Riot Games - Senior Product Inclusion Program Manager

Riot Games

Los Angeles, California, United States (On-Site)
1 Month ago
Riot Games - Senior Software Engineer - Data Foundation

Riot Games

Los Angeles, California, United States (On-Site)
1 Week ago
RoofStack - Senior Software Developer (Payment Systems)

RoofStack

İstanbul, İstanbul, Türkiye (Hybrid)
3 Months ago
Google - Senior Staff Software Engineer, Google Cloud

Google

Hyderabad, Telangana, India (On-Site)
4 Months ago
Meta - Research Scientist Intern, Machine Perception for Input and Interaction (PhD)

Meta

Sausalito, California, United States (On-Site)
4 Months ago
NVIDIA - Performance Engineering Intern - Summer 2025

NVIDIA

Toronto, Ontario, Canada (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling the creation of the metaverse. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry.


Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (Hybrid)

Santa Clara, California, United States (Hybrid)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

Ra'anana, Center District, Israel (On-Site)

Ra'anana, Center District, Israel (On-Site)

Yokne'am Illit, North District, Israel (On-Site)

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)

View All Jobs

Get notified when new jobs are added by NVIDIA

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug