Outscal Logooutscal logo

Senior Software Engineer - Distributed Inference

1 Week ago • 8 Years + • Artificial Intelligence • $184,000 PA - $356,500 PA

Job Summary

Job Description

NVIDIA seeks a Senior Software Engineer to develop and enhance tools like GenAI-Perf and Triton Model Analyzer for deep learning performance analysis. Responsibilities include collaborating with researchers and engineers to understand performance needs, developing efficient algorithms for measuring throughput and latency, integrating tools for a user-friendly experience, automating testing, and contributing to documentation. The role involves working with LLMs, generative AI, and deep learning models, leveraging frameworks like PyTorch, TensorFlow, and TensorRT. Experience with distributed systems programming, Python, and debugging is crucial.
Must have:
  • 8+ years experience in relevant field
  • Distributed systems programming knowledge
  • Excellent Python programming skills
  • Deep learning algorithm & framework knowledge
  • Performance analysis and test design skills
Good to have:
  • Experience with LLMs (Large Language Models)
  • Experience with PyTorch, TensorFlow, TensorRT, ONNX Runtime
  • Experience with cloud platforms (AWS, Azure, GCP)
  • Contribution to large open source projects
  • Experience with NVIDIA GPUs and deep learning inference frameworks
Perks:
  • Equity
  • Benefits

Job Details

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.

We are now looking for a Senior System Software Engineer to work on user facing tools for Triton Inference Server! NVIDIA is hiring software engineers for its GPU-accelerated deep learning software team, and we are a remote friendly work environment. Academic and commercial groups around the world are using GPUs to power a revolution in deep learning, enabling breakthroughs in problems from LLM, image classification to speech recognition to natural language processing. We are a fast-paced team building tools and software to make the design and deployment of new deep learning models easier and accessible to more data scientists.

What you'll be doing:

  • Develop and enhance functionalities within the GenAI-Perf, Triton Performance Analyzer and Triton Model Analyzer tools.

  • Collaborate with researchers and engineers to understand their performance analysis needs and translate them into actionable features.

  • Collaborate closely with cross-functional teams including software engineers, system architects, and product managers to drive performance improvements throughout the development lifecycle.

  • Responsible for setting up, executing, and analyzing the performance of LLM, Generative AI and deep learning models.

  • Develop and implement efficient algorithms for measuring deep learning throughput and latency, benchmarking large language models, and deploying models.

  • Integrate various tools to create a unified and user-friendly experience for deep learning performance analysis.

  • Automate testing processes to ensure the quality and stability of the tools.

  • Contribute to technical documentation and user guides. Stay up-to-date on the latest advancements in deep learning performance analysis and LLM optimization techniques.

What we need to see:

  • Bachelor's, Masters or PhD or equivalent experience

  • 8+ years in Computer Science, computer architecture, or related field

  • Knowledge of distributed systems programming.

  • Ability to work in a fast-paced, agile team environment

  • Excellent Python programming and software design skills, including debugging, performance analysis, and test design.

Ways to stand out from the crowd:

  • Experience with deep learning algorithms and frameworks. Especially experience with Large Language Models and frameworks such as PyTorch, TensorFlow, TensorRT, and ONNX Runtime.

  • Excellent troubleshooting abilities spanning multiple software (storage systems, kernels and containers).

  • Experience contributing to a large open source project - use of GitHub, bug tracking, branching and merging code, OSS licensing issues handling patches, etc.

  • Familiarity with cloud computing platforms (e.g., AWS, Azure, GCP) and Experience building and deploying cloud services using HTTP REST, gRPC, protobuf, JSON and related technologies.

  • Experience working with NVIDIA GPUs and deep learning inference frameworks is a plus.

NVIDIA has continuously reinvented itself over three decades. Our invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined modern computer graphics, and revolutionized parallel computing. More recently, GPU deep learning ignited modern AI — the next era of computing. We are widely considered to be the leader of AI computing, and one of the technology world’s most desirable employers. We have some of the most forward-thinking and committed people in the world working for us. If you're creative and autonomous, we want to hear from you!

The base salary range is 184,000 USD - 356,500 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.

You will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Similar Jobs

Kaedim - Machine Learning Engineer

Kaedim

London, England, United Kingdom (On-Site)
7 Months ago
NVIDIA - Senior Signal and Power Integrity Engineer

NVIDIA

Canada (On-Site)
2 Weeks ago
NVIDIA - Solution Architect - CSP Cloud

NVIDIA

Beijing, Beijing, China (On-Site)
2 Months ago
ByteDance - Student Researcher (Doubao (Seed) - Foundation Model - MultiModal Generative Model)

ByteDance

San Jose, California, United States (On-Site)
19 Hours ago
Krafton  - Deep Learning Research Scientist - Core

Krafton

Seoul, South Korea (On-Site)
1 Month ago
Hedra - Applied Research Scientist

Hedra

San Francisco, California, United States (On-Site)
12 Hours ago
NVIDIA - Solution Architect - Auto

NVIDIA

Beijing, Beijing, China (On-Site)
2 Months ago
Trend Micro - Sr. AI Engineer

Trend Micro

Taipei City, Taiwan (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

NVIDIA - Senior ASIC Verification Engineer - HSIO

NVIDIA

Austin, Texas, United States (On-Site)
1 Month ago
NVIDIA - HPC Lab Manager

NVIDIA

Yokne'am Illit, North District, Israel (On-Site)
2 Months ago
Onward Search - API Developer

Onward Search

North Arlington, New Jersey, United States (Remote)
1 Month ago
Google - Software Engineer III, Machine Learning, Search

Google

Mountain View, California, United States (On-Site)
4 Months ago
ByteDance - Research Scientist, Foundation Model, Vision

ByteDance

Singapore (On-Site)
4 Months ago
NVIDIA - Machine Learning Engineer Intern - 2025

NVIDIA

Shanghai, Shanghai, China (On-Site)
2 Months ago
Genies - Lead Machine Learning Engineer, 3D Gen AI & Graphics

Genies

Los Angeles, California, United States (On-Site)
3 Weeks ago
ByteDance - Research Scientist Graduate (Foundation Model, Video Generation) - 2025 Start (PhD)

ByteDance

Seattle, Washington, United States (On-Site)
4 Months ago
ByteDance - Software Engineer (Applied Machine Learning - Enterprise)

ByteDance

San Jose, California, United States (On-Site)
20 Hours ago
ByteDance - Software Engineer in ML Engineering Platform

ByteDance

Seattle, Washington, United States (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

Jobs in California, United States

Zoox - Director, Process & Tools, PMO

Zoox

Foster City, California, United States (On-Site)
5 Months ago
NVIDIA - Research Scientist, ASIC and VLSI - New College Grad 2025

NVIDIA

Santa Clara, California, United States (On-Site)
2 Months ago
ByteDance - Student Researcher (Doubao (Seed) - Foundation Model - Video Generative Model)

ByteDance

San Jose, California, United States (On-Site)
1 Day ago
Life church - APIs Staff Software Engineer

Life church

Edmond, Oklahoma, United States (On-Site)
5 Months ago
Electronic Arts - VP, Advertising and Sponsorships

Electronic Arts

Redwood City, California, United States (On-Site)
1 Week ago
Nintendo - Technical Localization Specialist (NTD)

Nintendo

Redmond, Washington, United States (On-Site)
9 Months ago
Sandbox VR - Retail Associate

Sandbox VR

Oak Brook, Illinois, United States (On-Site)
5 Months ago
Next Level Business Services - Apigee Technical Architect

Next Level Business Services

San Francisco, California, United States (On-Site)
4 Months ago
On Location - Marketing Manager, B2C Marketing – FIFA World Cup 26™

On Location

New York, New York, United States (On-Site)
1 Month ago
Life church - Associate Host Team Pastor

Life church

United States (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

Artificial Intelligence Jobs

Zoox - Senior Machine Learning Engineer - Collision Avoidance System

Zoox

Foster City, California, United States (Hybrid)
5 Months ago
Meta - AI Research Scientist, Language - Generative AI

Meta

Bellevue, Washington, United States (On-Site)
4 Months ago
Meta - Research Scientist, Machine Learning (PhD)

Meta

Sunnyvale, California, United States (On-Site)
4 Months ago
ByteDance - Research Scientist Intern (Doubao (Seed) - Machine Learning System) - 2025 Summer (PhD)

ByteDance

Seattle, Washington, United States (On-Site)
4 Months ago
NVIDIA - Principal Software Engineer - Enterprise AI Platform

NVIDIA

Canada (Hybrid)
1 Month ago
The Walt Disney Company - Lead Machine Learning Engineer

The Walt Disney Company

Seattle, Washington, United States (On-Site)
1 Day ago
NVIDIA - Principal Engineer - DL and AI Software

NVIDIA

Canada (On-Site)
1 Month ago
Meta - Software Engineer, Systems ML - SW/HW Co-design

Meta

Fremont, California, United States (Remote)
4 Months ago
Thumbtack - Staff Software Engineer,  Machine Learning Infrastructure

Thumbtack

United States (Remote)
4 Months ago
ByteDance - Software Engineer Intern (Doubao (Seed) - Machine Learning System) - 2025 Summer (MS)

ByteDance

Seattle, Washington, United States (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling the creation of the metaverse. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry.


Hsinchu, Hsinchu City, Taiwan (On-Site)

Yokne'am Illit, North District, Israel (On-Site)

Seoul, South Korea (Hybrid)

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)

Ra'anana, Center District, Israel (On-Site)

Shanghai, Shanghai, China (On-Site)

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)

Be'er Sheva, South District, Israel (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by NVIDIA

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug