Senior Software Engineer - Distributed Inference

2 Months ago • 8 Years + • Artificial Intelligence • $184,000 PA - $356,500 PA

Job Summary

Job Description

NVIDIA seeks a Senior Software Engineer to develop and enhance tools like GenAI-Perf and Triton Model Analyzer for deep learning performance analysis. Responsibilities include collaborating with researchers and engineers to understand performance needs, developing efficient algorithms for measuring throughput and latency, integrating tools for a user-friendly experience, automating testing, and contributing to documentation. The role involves working with LLMs, generative AI, and deep learning models, leveraging frameworks like PyTorch, TensorFlow, and TensorRT. Experience with distributed systems programming, Python, and debugging is crucial.
Must have:
  • 8+ years experience in relevant field
  • Distributed systems programming knowledge
  • Excellent Python programming skills
  • Deep learning algorithm & framework knowledge
  • Performance analysis and test design skills
Good to have:
  • Experience with LLMs (Large Language Models)
  • Experience with PyTorch, TensorFlow, TensorRT, ONNX Runtime
  • Experience with cloud platforms (AWS, Azure, GCP)
  • Contribution to large open source projects
  • Experience with NVIDIA GPUs and deep learning inference frameworks
Perks:
  • Equity
  • Benefits

Job Details

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.

We are now looking for a Senior System Software Engineer to work on user facing tools for Triton Inference Server! NVIDIA is hiring software engineers for its GPU-accelerated deep learning software team, and we are a remote friendly work environment. Academic and commercial groups around the world are using GPUs to power a revolution in deep learning, enabling breakthroughs in problems from LLM, image classification to speech recognition to natural language processing. We are a fast-paced team building tools and software to make the design and deployment of new deep learning models easier and accessible to more data scientists.

What you'll be doing:

  • Develop and enhance functionalities within the GenAI-Perf, Triton Performance Analyzer and Triton Model Analyzer tools.

  • Collaborate with researchers and engineers to understand their performance analysis needs and translate them into actionable features.

  • Collaborate closely with cross-functional teams including software engineers, system architects, and product managers to drive performance improvements throughout the development lifecycle.

  • Responsible for setting up, executing, and analyzing the performance of LLM, Generative AI and deep learning models.

  • Develop and implement efficient algorithms for measuring deep learning throughput and latency, benchmarking large language models, and deploying models.

  • Integrate various tools to create a unified and user-friendly experience for deep learning performance analysis.

  • Automate testing processes to ensure the quality and stability of the tools.

  • Contribute to technical documentation and user guides. Stay up-to-date on the latest advancements in deep learning performance analysis and LLM optimization techniques.

What we need to see:

  • Bachelor's, Masters or PhD or equivalent experience

  • 8+ years in Computer Science, computer architecture, or related field

  • Knowledge of distributed systems programming.

  • Ability to work in a fast-paced, agile team environment

  • Excellent Python programming and software design skills, including debugging, performance analysis, and test design.

Ways to stand out from the crowd:

  • Experience with deep learning algorithms and frameworks. Especially experience with Large Language Models and frameworks such as PyTorch, TensorFlow, TensorRT, and ONNX Runtime.

  • Excellent troubleshooting abilities spanning multiple software (storage systems, kernels and containers).

  • Experience contributing to a large open source project - use of GitHub, bug tracking, branching and merging code, OSS licensing issues handling patches, etc.

  • Familiarity with cloud computing platforms (e.g., AWS, Azure, GCP) and Experience building and deploying cloud services using HTTP REST, gRPC, protobuf, JSON and related technologies.

  • Experience working with NVIDIA GPUs and deep learning inference frameworks is a plus.

NVIDIA has continuously reinvented itself over three decades. Our invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined modern computer graphics, and revolutionized parallel computing. More recently, GPU deep learning ignited modern AI — the next era of computing. We are widely considered to be the leader of AI computing, and one of the technology world’s most desirable employers. We have some of the most forward-thinking and committed people in the world working for us. If you're creative and autonomous, we want to hear from you!

The base salary range is 184,000 USD - 356,500 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.

You will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Similar Jobs

ByteDance - Software Engineer Intern (Doubao (Seed) - Machine Learning System) - 2025 Summer (PhD)

ByteDance

Seattle, Washington, United States (On-Site)
7 Months ago
Agara labs - Senior Product Manager

Agara labs

(Remote)
1 Month ago
NVIDIA - Senior Technical Program Manager, AI Datacenter

NVIDIA

Shenzhen, Guangdong Province, China (On-Site)
3 Months ago
prizepicks - Data Science Engineering Manager

prizepicks

(Remote)
1 Month ago
ByteDance - Video Algorithm Engineer - Multimedia Lab

ByteDance

San Jose, California, United States (On-Site)
7 Months ago
Google - Technical Solutions Engineer, AI/ML

Google

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Immersion Labs - Junior Prompt Engineer

Immersion Labs

Warsaw, Masovian Voivodeship, Poland (Hybrid)
3 Months ago
Krafton  - Technical Project Manager

Krafton

Seoul, South Korea (On-Site)
2 Months ago
ByteDance - Machine Learning Scientist Graduate (Scaling AI for Biology (AI-for-Science))

ByteDance

Seattle, Washington, United States (On-Site)
2 Months ago
Lionbridge Games - Language AI (Games) Program Manager

Lionbridge Games

Masovian Voivodeship, Poland (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Microsoft - Member of Technical Staff, AI Pretraining

Microsoft

London, England, United Kingdom (On-Site)
2 Months ago
Ubisoft - Lead R&D Scientist

Ubisoft

Shanghai, Shanghai, China (On-Site)
1 Month ago
Google - Senior Software Engineer, Core Machine Learning, Google Cloud

Google

New York, New York, United States (On-Site)
6 Months ago
sony global (Games) - Software Design Engineer

sony global (Games)

Wuxi, Jiangsu, China (On-Site)
1 Month ago
Inkittt - Senior Machine Learning Engineer, Recommendations

Inkittt

San Francisco, California, United States (Hybrid)
4 Months ago
NVIDIA - Engineering Farm Engineer

NVIDIA

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Canva - Design AI - ML Engineering Manager

Canva

Sydney, New South Wales, Australia (Remote)
2 Months ago
Extreme Network - Edge AI Staff Engineer

Extreme Network

(Remote)
1 Month ago
ByteDance - Research Scientist, Multimodality

ByteDance

Seattle, Washington, United States (On-Site)
7 Months ago
NVIDIA - GPU Kernel Software Engineering Intern - 2025

NVIDIA

Shanghai, Shanghai, China (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

Jobs in California, United States

Google - Open Career Opportunities, Verily Life Sciences

Google

Mountain View, California, United States (On-Site)
6 Months ago
Whoop - Senior Information Security Engineer

Whoop

Boston, Massachusetts, United States (On-Site)
2 Months ago
WebFX - Jr. Social Media Ads and Analytics Specialist

WebFX

Harrisburg, Pennsylvania, United States (On-Site)
7 Months ago
Meta - Research Scientist Intern, Photorealistic Telepresence (PhD)

Meta

Redmond, Washington, United States (On-Site)
6 Months ago
Ember Lab - Senior Accountant

Ember Lab

Orange, California, United States (On-Site)
1 Month ago
WebFX - Jr. UX Designer

WebFX

Ann Arbor, Michigan, United States (On-Site)
6 Months ago
Nintendo - Intern – Machine Learning Software Engineer (NTD)

Nintendo

Redmond, Washington, United States (On-Site)
6 Months ago
Oportun - Bilingual (Spanish/English) Member Loyalty Representative

Oportun

Los Angeles, California, United States (On-Site)
1 Month ago
Glean - Software Engineer, Frontend

Glean

Palo Alto, California, United States (On-Site)
1 Month ago
IManage - Principal Software Engineer

IManage

Chicago, Illinois, United States (Hybrid)
2 Months ago

Get notifed when new similar jobs are uploaded

Artificial Intelligence Jobs

Arrise Solutions (India)   - Senior ML Engineer

Arrise Solutions (India)

Hyderabad, Telangana, India (On-Site)
8 Months ago
NVIDIA - Senior Solutions Architect, Generative AI

NVIDIA

Mumbai, Maharashtra, India (On-Site)
2 Months ago
ByteDance - Research Engineer Graduate (Vision AI Platform)

ByteDance

San Jose, California, United States (On-Site)
1 Month ago
Google - CPU AI Workloads and Performance Architect

Google

Mountain View, California, United States (On-Site)
1 Month ago
Keywords Studios - AI - Technical Research Associate (Prompts)

Keywords Studios

Silesian Voivodeship, Poland (On-Site)
2 Months ago
Canva - Machine Learning Research Engineering Manager - Image Generation

Canva

Vienna, Vienna, Austria (Remote)
3 Months ago
Krafton  - Head of Deep Learning PM & Ops

Krafton

Seoul, South Korea (On-Site)
2 Months ago
Hedra - Machine Learning Engineer (CUDA)

Hedra

San Francisco, California, United States (On-Site)
2 Months ago
Google - Staff Software Engineer, GPU Performance, Google Scale

Google

Kirkland, Washington, United States (On-Site)
1 Month ago
NVIDIA - NIM Solution Architect

NVIDIA

Shanghai, Shanghai, China (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling the creation of the metaverse. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry.

Santa Clara, California, United States (On-Site)

Massachusetts, United States (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

Texas, United States (On-Site)

Santa Clara, California, United States (Hybrid)

Santa Clara, California, United States (Hybrid)

Pune, Maharashtra, India (On-Site)

Taipei City, Taiwan (On-Site)

View All Jobs

Get notified when new jobs are added by NVIDIA

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug