Deep Learning Engineer, LLM Accuracy Evaluation

NVIDIA is seeking a Senior Deep Learning Engineer to pioneer new methodologies for accurately assessing the performance of cutting-edge deep learning models, including LLMs, RAG, agents, and vision models. This role involves collaborating with partners and the open-source community to optimize flagship models as NVIDIA Inference Microservices (NIM). Responsibilities include researching and developing innovative deep learning evaluation methodologies, analyzing and enhancing AI/DL libraries, and building robust tools and infrastructure pipelines to support AI initiatives. The position offers a chance to shape the future of AI, working with powerful GPU clusters and unreleased hardware.

Must Have

• Hands-on experience in AI for natural language processing (NLP) and large language models (LLMs)
• Strong problem-solving skills
• Debugging skills
• Performance analysis skills
• Test design skills
• Documentation skills
• Solid mathematical foundations
• Expertise in AI/DL algorithms
• Excellent written communication skills
• Excellent verbal communication skills
• Ability to work independently and collaboratively

Good to Have

• Experience in accuracy evaluation of LLMs (OpenLLM Leaderboard or HELM)
• Hands-on experience with inference and deployment environments like TensorRT, ONNX, or Triton
• Passion for DevOps/MLOps practices in deep learning product development
• Experience running large-scale workloads in high-performance computing (HPC) clusters
• Strong understanding of Linux environments
• Understanding of containerization technologies like Docker

Job Requisition ID

JR2001661

Job Category

Engineering

Time Type

Full time

We are seeking senior engineers to pioneer new methodologies for accurately assessing the performance of ground-breaking deep learning models, including LLMs, RAG, agents, and vision models. You will collaborate across the organization to bring the latest flagship models from our community and partners—such as Gemma and Llama-3—to life as optimized NVIDIA Inference Microservices (NIM). This role offers an outstanding opportunity to craft the future of AI at a fast-growing company at the forefront of the AI revolution. Join our team of world-class software engineers and partners to deliver the most advanced models with lightning-fast inference. You'll work on the most powerful, enterprise-grade GPU clusters capable of hundreds of PetaFLOPS and gain early access to unreleased hardware, making a direct impact on NVIDIA's roadmap and the broader AI landscape!

What you’ll be doing:

• Collaborate closely with our partners and the open-source community to deliver their flagship models as highly optimized NVIDIA Inference Microservices (NIM).
• Research and develop innovative deep learning methodologies to accurately evaluate new model families across diverse domains.
• Analyze, influence, and enhance AI/DL libraries, frameworks, and APIs, ensuring consistency with the best engineering practices.
• Research, prototype, and build robust tools and infrastructure pipelines to support our ground-breaking AI initiatives.

What we need to see:

• BS, MS, or PhD in Computer Science, AI, Applied Math, or a related field, or equivalent experience.
• 10+ years of hands-on experience in AI for natural language processing (NLP) and large language models (LLMs).
• Strong problem-solving, debugging, performance analysis, test design, and documentation skills.
• Solid mathematical foundations and expertise in AI/DL algorithms.
• Excellent written and verbal communication skills, with the ability to work both independently and collaboratively in a fast-paced environment.

Ways to stand out from the crowd:

• Experience in accuracy evaluation of LLMs (OpenLLM Leaderboard or HELM).
• Hands-on experience with inference and deployment environments like TensorRT, ONNX, or Triton.
• Passion for DevOps/MLOps practices in deep learning product development.
• Experience running large-scale workloads in high-performance computing (HPC) clusters.
• Strong understanding of Linux environments and containerization technologies like Docker.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. For Poland: The base salary range is 292,500 PLN - 507,000 PLN.

Insights from previous hires

Top skills

Artificial Intelligence

API

Previously worked as

1. Software Engineer

2. Software Development Engineer

3. Senior Software Engineer

4. Lead Software Engineer

5. Senior Data Scientist

Similar jobs

• [Senior Performance Software Engineer, Deep Learning Libraries

JR2006435

US, CA, Santa Clara + 4 more](https://nvidia.eightfold.ai/careers/job/893391764682)

• [Senior Deep Learning Software Engineer, Inference

JR1997930

Remote - Netherlands + 2 more

Remote](https://nvidia.eightfold.ai/careers/job/893382583022)

• [Senior Deep Learning Software Engineer, LLM Performance

JR1997464

US, CA, Santa Clara + 1 more

Remote](https://nvidia.eightfold.ai/careers/job/893382409379)

• [Senior Software Engineer - Deep Learning

JR2006872

US, CA, Santa Clara](https://nvidia.eightfold.ai/careers/job/893391844077)

• [Senior Software Engineer, Deep Learning - MLIR TRT

JR2009329

US, CA, Santa Clara](https://nvidia.eightfold.ai/careers/job/893392493845)

• [Senior Deep Learning Software Engineer, FlashInfer

JR2001340

US, CA, Santa Clara](https://nvidia.eightfold.ai/careers/job/893383762583)

• [System Software Engineer, Python and C/C++ - Deep Learning

JR1980641

Poland, Warsaw + 1 more

Remote](https://nvidia.eightfold.ai/careers/job/893375056564)

• [Senior Deep Learning Performance Engineer - Training at Scale

JR1980206

Remote - Switzerland + 3 more

Remote](https://nvidia.eightfold.ai/careers/job/893375056871)

• [Senior Software Engineer, Deep Learning - Torch-TRT

JR2009341

US, CA, Santa Clara](https://nvidia.eightfold.ai/careers/job/893392478768)

• [Senior Deep Learning Algorithm Engineer

JR2006567

US, CA, Santa Clara + 1 more

Remote](https://nvidia.eightfold.ai/careers/job/893391817282)