ML Model Serving Engineer

5 Months ago • All levels • Research Development • $175,000 PA - $280,000 PA

Job Summary

Job Description

Sesame is building a future where computers are lifelike, focusing on voice companions. The ML Model Serving Engineer will enhance their serving layer, which handles LLM, speech, and vision models. This role involves partnering with ML infrastructure and training teams to create a fast, cost-effective, accurate, and reliable serving layer for a new consumer product. Responsibilities include modifying LLM serving frameworks like VLLM and SGLang, experimenting with new compilers for various hardware, optimizing models for faster inference without quality sacrifice, and reducing initialization times. The ideal candidate will be an expert in PyTorch, model optimization for serving, and systems programming.
Must have:
  • Expert in PyTorch or similar differentiable array computing framework
  • Expert in optimizing ML models for high throughput, low latency serving
  • Significant systems programming experience (e.g., VLLM internals)
  • Significant performance engineering experience (e.g., bottleneck analysis)
  • Up-to-date on model serving optimization techniques
Good to have:
  • Familiarity with high-performance LLM serving (VLLM, SGlang)
  • Experience with public cloud platforms (GCP, AWS, Azure)
  • Experience scaling inference workloads with Kubernetes, Ray
  • Track record of leading complex multi-month projects
  • Eagerness to learn new things and work in multiple roles
Perks:
  • 401k matching
  • 100% employer-paid health, vision, and dental benefits
  • Unlimited PTO and sick time
  • Flexible spending account matching

Job Details

About Sesame

Sesame believes in a future where computers are lifelike - with the ability to see, hear, and collaborate with us in ways that feel natural and human. With this vision, we're designing a new kind of computer, focused on making voice companions part of our daily lives. Our team brings together founders from Oculus and Ubiquity6, alongside proven leaders from Meta, Google, and Apple, with deep expertise spanning hardware and software. Join us in shaping a future where computers truly come alive.

Responsibilities:

  • Turbocharge our serving layer, consisting of a variety of LLM, speech, and vision models.

  • Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer to power a new consumer product category.

  • Modify and extend LLM serving frameworks like VLLM and SGLang to take advantage of the latest techniques in high-performance model serving.

  • Experiment with new compilers to support running models on a variety of hardware compute platforms.

  • Work with the training team to identify opportunities to produce faster models without sacrificing quality.

  • Use techniques like in-flight batching, caching, and custom kernels to speed up inference.

  • Find ways to reduce model initialization times without sacrificing quality.

Required Qualifications:

  • Expert in some differentiable array computing framework, preferably PyTorch.

  • Expert in optimizing machine learning models for serving reliably at high throughput, with low latency.

  • Significant systems programming experience; ex. Experience working on high-performance server systems—you’d be just as comfortable with the internals of VLLM as you would with a complex PyTorch codebase.

  • Significant performance engineering experience; ex. Bottleneck analysis in high-scale server systems or profiling low-level systems code.

  • Always up to date on the latest techniques for model serving optimization.

Preferred Qualifications:

  • Familiarity with high-performance LLM serving; ex. experience with VLLM, SGlang deployment, and internals.

  • Experience with a public cloud platform such as GCP, AWS, or Azure.

  • Experience deploying and scaling inference workloads in the cloud using Kubernetes, Ray, etc.

  • You like to ship and have a track record of leading complex multi-month projects without assistance.

  • You’re excited to learn new things and work in a multitude of roles.

Sesame is committed to a workplace where everyone feels valued, respected, and empowered. We welcome all qualified applicants, embracing diversity in race, gender, identity, orientation, ability, and more. We provide reasonable accommodations for applicants with disabilities—contact careers@sesame.com for assistance.

Full-time Employee Benefits: 

  • 401k matching

  • 100% employer-paid health, vision, and dental benefits 

  • Unlimited PTO and sick time 

  • Flexible spending account matching (medical FSA) 

Benefits do not apply to contingent/contract workers

Similar Jobs

Trend Micro - Regional Account Manager

Trend Micro

Indiana, United States (Remote)
2 Months ago
52 entertainment - Cloud Engineer M/F

52 entertainment

Villeneuve-d'Ascq, Hauts-de-France, France (Hybrid)
3 Weeks ago
Bede Gaming - Principal Application Architect

Bede Gaming

Sofia, Sofia City Province, Bulgaria (On-Site)
3 Months ago
C3 IoT - Site Reliability Engineer - Field Operations

C3 IoT

Redwood City, California, United States (On-Site)
1 Month ago
Ajmera Infotech - Sr. Backend Engineer - Node Expert

Ajmera Infotech

Bengaluru, Karnataka, India (On-Site)
1 Year ago
Glean - Software Engineer, Machine Learning

Glean

Palo Alto, California, United States (On-Site)
3 Months ago
Sprinkler - Director of Machine Learning Research

Sprinkler

Gurugram, Haryana, India (On-Site)
1 Year ago
Qualcomm - Senior Engineer, AI Orchestration

Qualcomm

Bengaluru, Karnataka, India (On-Site)
3 Months ago
Game freak - New Graduate Programmer (Entry Sheet/Research Achievements Selection)

Game freak

Chiyoda City, Tokyo, Japan (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

JDA - Staff Software Engineer (Python)

JDA

Scottsdale, Arizona, United States (On-Site)
2 Months ago
Arketa - Senior Software Engineer - Automation Team (Node.js, Tech Lead)

Arketa

United States (Remote)
1 Month ago
Granicus - Systems Engineer II - Windows

Granicus

Bengaluru, Karnataka, India (Remote)
3 Months ago
pentair - Sr Specialist Firmware

pentair

Noida, Uttar Pradesh, India (On-Site)
3 Months ago
Scopely - Senior Software Engineer - Server - Unannounced Project

Scopely

Barcelona, Catalonia, Spain (Hybrid)
6 Months ago
Siemens  - Container Solution Architect (Containers PO)

Siemens

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Riot Games - Principal Software Engineer, Services

Riot Games

Los Angeles, California, United States (On-Site)
2 Months ago
Rackner - AWS Cloud Architect

Rackner

Honolulu, Hawaii, United States (On-Site)
3 Months ago
Forcepoint - Software Engineer III (Android/iOS Developer)

Forcepoint

Thane, Maharashtra, India (On-Site)
1 Month ago
King - Staff Data Engineer

King

London, England, United Kingdom (On-Site)
4 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in San Francisco, California, United States

22squared - Copywriter

22squared

Atlanta, Georgia, United States (Hybrid)
2 Months ago
Intelerad - System Support Analyst I - Medical Imaging Systems / Linux

Intelerad

United States (Remote)
1 Month ago
Aptive - Sales Account Manager

Aptive

Denver, Colorado, United States (On-Site)
3 Months ago
Hawkeye Innovations - College Sports Systems Technician

Hawkeye Innovations

Pennsylvania, United States (On-Site)
4 Months ago
Figma - Product Manager, AI

Figma

San Francisco, California, United States (Remote)
4 Weeks ago
HCL Tech - Java Senior Technical Lead with DevOps, AWS

HCL Tech

New Jersey, United States (On-Site)
2 Months ago
Dialpad AI - Account Executive, Mid-Market

Dialpad AI

Austin, Texas, United States (On-Site)
2 Months ago
Toast - Bilingual Spanish Emerging Markets Manager

Toast

San Diego, California, United States (On-Site)
2 Months ago
Zuora - Senior Product Manager, Payments

Zuora

Redwood City, California, United States (Remote)
1 Month ago
Lionsgate - Manager, IT Security GRC

Lionsgate

Santa Monica, California, United States (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

Research Development Jobs

Apple - AIML - Staff Machine Learning Engineer - Reinforcement Learning

Apple

Santa Clara, California, United States (On-Site)
2 Months ago
Apple - Staff Machine Learning Engineer

Apple

Cupertino, California, United States (On-Site)
2 Months ago
Synechron - Java Enterprise Architect (Greenfield Design & Gen AI)

Synechron

Phoenix, Arizona, United States (On-Site)
1 Month ago
Sika Group - Lead R&D - Building Finishing System

Sika Group

Navi Mumbai, Maharashtra, India (On-Site)
1 Month ago
Sailpoint - Manager, Engineering Operations - AI

Sailpoint

Austin, Texas, United States (On-Site)
1 Month ago
Ubisoft - Programmeur senior ML _ Groupe Technologique Création de Contenu

Ubisoft

Montreal, Quebec, Canada (On-Site)
7 Months ago
Capgemini - Machine Learning Developer

Capgemini

Bengaluru, Karnataka, India (On-Site)
3 Months ago
Foster and partners  - Design Systems Analyst (Applied R&D)

Foster and partners

London, England, United Kingdom (On-Site)
1 Month ago
Keywords Studios - AI - Senior Research Associate (Prompts)

Keywords Studios

Silesian Voivodeship, Poland (On-Site)
5 Months ago
AiDash - Staff Machine Learning / Perception Engineer

AiDash

Palo Alto, California, United States (Hybrid)
3 Months ago

Get notifed when new similar jobs are uploaded

About The Company

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by Sesame

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug