NVIDIA's PyTorch team seeks a Senior Software Engineer specializing in automated parallel programming.  The role involves designing and building tools for AI practitioners, ensuring optimal performance on NVIDIA hardware.  Responsibilities include crafting a code generation system to accelerate machine learning frameworks, collaborating with hardware and software teams to improve GPU performance in PyTorch, optimizing deep learning models, and collaborating with researchers.  The ideal candidate possesses strong parallel programming experience (CUDA), C++ expertise, and experience with large software projects.  Influence extends across the software stack, from users to the CUDA compiler and hardware design.

Explore gaming industy jobs in one of the leading Game Studios.

NVIDIA

System Software Engineer - Android Platform Security

System Software Engineering Manager

Senior Silicon Solution Engineer - Hardware

Senior System Validation Engineer

Senior System Performance and Power Engineer

Circuit Design Engineer

Senior System Integration Engineer

Senior System Software Engineer - AI Performance and Efficiency Tools

Staff Systems Software Engineer - Server

M&A and Integration Lead

<div data-automation-id="jobPostingDescription" class="css-4r17ng">The PyTorch Team @ NVIDIA is hiring passionate parallel programmers. Join us to design and build the tools used by millions of AI practitioners deploying AI applications scalable to thousands of GPUs. Our team is responsible for the continual delivery of best in class experience on NVIDIA's hardware with PyTorch. Join our team and collaborate with many multi-disciplinary engineering teams within NVIDIA and internationally in the PyTorch open source community to deliver our customers the best of NVIDIA software.In this position you will learn innovative techniques from NVIDIA's domain experts for efficiently programming the world's most sophisticated computer systems. Build these techniques into <a rel="noopener noreferrer" target="_blank" href="https://github.com/nvidia/fuser">NVIDIA/Fuser (commonly known as "nvFuser")</a> applying our groundbreaking Parallel Programming Theory, allowing these optimization techniques to be applied to algorithms broadly, automatically, and safely to algorithms written in Numpy and PyTorch. Beyond building nvFuser influence and improve the entire software stack all the way from users to the CUDA compiler, to the <a rel="noopener noreferrer" target="_blank" href="https://github.com/Lightning-AI/lightning-thunder">Lightning-Thunder Graph Compiler</a>, as well as influence the future design of NVIDIA's hardware platform. Join our ambitious and diverse team who strive to lead the best in AI programming.What you will be doing:<ul><li>Crafting a code generation system to accelerate portions of a graph collected from a machine learning framework.</li><li>Partnering with NVIDIA’s hardware and software teams to improve GPU performance in PyTorch.</li><li>Design, build and support production AI solutions used by enterprise customers and partners.</li><li>Optimize the performance of influential, modern Deep Learning models coming out of academic and industry research, for NVIDIA GPUs and systems.</li><li>Collaborating with internal applied researchers to improve their AI tools.</li><li>Advise design of new hardware generations.</li></ul>What we need to see:<ul><li>MS or PhD Computer Science, Computer Engineering, Electrical Engineering or a related field (or equivalent experience).</li><li>Parallel programming experience with writing optimized kernels in the NVIDIA CUDA Programming Language or similar parallel languages</li><li>4+ years of experience with C++ programming.</li><li>Demonstrated experience developing large software projects.</li><li>We require excellent verbal and written communication skills.</li></ul>Ways to stand out from the crowd:<ul><li>Proven technical foundation in CPU and GPU architectures, numeric libraries, modular software design.</li><li>A background in deep learning compilers or compiler infrastructure</li><li>Expertise with optimized distributed parallelism techniques and it's a bonus if that includes parallelizing Large Language Models!</li><li>Knowledge of heuristic generation that employs cost models, machine learning, or auto-tuning.</li><li>Contributions to PyTorch, Numpy, JAX, TensorFlow, OpenAI-Triton, Lightning Thunder, TVM, Halide or similar system.</li></ul>The base salary range is 184,000 USD - 287,500 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.<div><div><div><div><div><div><div>You will also be eligible for equity and <a rel="noopener noreferrer" target="_blank" href="https://www.nvidia.com/en-us/benefits/">benefits</a>. NVIDIA accepts applications on an ongoing basis. </div></div></div></div></div></div></div>NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.</div>

Senior Software Engineer - Automated Parallel Programming