Engineering Manager - Inference Backend

3 Months ago • 10 Years + • Backend Development • $300,000 PA - $450,000 PA

Job Summary

Job Description

Lambda is seeking an Engineering Manager for its Inference Backend team. This role involves leading the development of a rapidly growing inference-as-a-service product, focusing on new model types, serving modes, and scaling across thousands of GPUs. The manager will prioritize customer needs, balancing feature velocity, reliability, and security with reducing operational overhead. Responsibilities include collaborating with infrastructure teams, hiring and developing top-tier engineers in distributed systems and machine learning, and fostering a culture of sustainable, empathetic, and high-velocity engineering with an emphasis on cross-team collaboration, documentation, and data-driven decisions.
Must have:
  • 6+ years in a full-time management role
  • 10+ years in software engineering, focused on distributed systems
  • Proven record of leading high-performance systems teams
  • Exceptional leadership and empathy skills
  • Customer-facing skills (pre-sales, support, incident management)
  • 1+ year of experience running ML model inference
  • Knowledge of ML model lifecycle
  • Experience managing long-term and short-term projects
  • Experience collaborating with product and sales teams
  • Ability to review Python and Go applications
Good to have:
  • Experience running ML workloads on GPUs at scale
  • Academic or scientific ML experience
  • Experience with vLLM, sglang, PyTorch
  • Experience managing a remote team
  • Strong Python/Go experience
  • Sales, customer service, or support experience
Perks:
  • Generous cash & equity compensation
  • Health, dental, and vision coverage
  • Wellness and Commuter stipends
  • 401k Plan with 2% company match
  • Flexible Paid Time Off

Job Details

Lambda is the #1 GPU Cloud for ML/AI teams training, fine-tuning and inferencing AI models, where engineers can easily, securely and affordably build, test and deploy AI products at scale. Lambda’s product portfolio includes on-prem GPU systems, hosted GPUs across public & private clouds and managed inference services – servicing government, researchers, startups and Enterprises world-wide.


If you'd like to build the world's best deep learning cloud, join us. 


*Note: This position requires presence in our San Francisco office location 4 days per week; Lambda’s designated work from home day is currently Tuesday.

Engineering at Lambda is responsible for building and scaling our cloud offering. Our scope includes the Lambda website, cloud APIs and systems as well as internal tooling for system deployment, management and maintenance.

What you’ll do

  • Lead our Inference team, developing our rapidly growing inference-as-a-service product as it tackles new types of models, new serving modes, and an ever increasing customer base, across thousands to tens of thousands of GPUs.

  • Be product-focused in your leadership and execution, always placing the needs of the customer first, with a particular focus on feature velocity, reliability and security.

  • Balance development work for cutting-edge features and models against reducing operational overhead and scaling costs.

  • Proactively engage with other teams who run the lower-level infrastructure, datacenter builds, and frontends and work to develop a holistic product and engineering plan.

  • Hire, grow and retain top-tier engineers, in the fields of both distributed systems engineering and machine learning.

  • Shape a culture of sustainable, empathetic, and high-velocity engineering, with a deep focus on cross-team collaboration, documentation, and data-driven decision-making.

You

  • 6+ years in a  full-time management role at a high-growth technology company 

  • 10+ years of industry experience in software engineering, with a focus on large-scale distributed systems and backend systems.

  • Proven record of leading and building engineering teams that work on mission-critical, high performance systems.

  • Exceptional leadership skills that encompass leading by trust, building empathy with your reports and other teams, and maintaining a sustainable but rapid velocity.

  • customer-facing skills, including pre-sales, general support, and incident management.

  • At least one year of experience running inference for machine learning models

  • Knowledge of the machine learning model lifecycle and experience with each of its steps (including pre-training, fine-tuning, and inference)

  • Demonstrated expertise in managing long-term projects alongside urgent, short-term priorities and incident resolution.

  • Extensive experience collaborating with product, sales, and other engineering teams to build cohesive products with a focus on user experience and reliability.

  • Ability to understand, review and structure Python and Go applications.

Nice to Have

  • Significant experience running machine learning workloads on GPUs at large scale

  • Academic or scientific experience working with machine learning

  • Direct experience working with vLLM, sglang, PyTorch, or other ML libraries

  • Experience managing a remote, distributed team

  • Strong experience in Python and/or Go

  • Significant sales, customer service or support experience

Salary Range Information 

Based on market data and other factors, the annual salary range for this position is $300,000-$450,000. However, a salary higher or lower than this range may be appropriate for a candidate whose qualifications differ meaningfully from those listed in the job description. 

About Lambda

  • Founded in 2012, ~350 employees (2024) and growing fast

  • We offer generous cash & equity compensation

  • Our investors include Andra Capital, SGW, Andrej Karpathy, ARK Invest, Fincadia Advisors, G Squared, In-Q-Tel (IQT), KHK & Partners, NVIDIA, Pegatron, Supermicro, Wistron, Wiwynn, US Innovative Technology, Gradient Ventures, Mercato Partners, SVB, 1517, Crescent Cove.

  • We are experiencing extremely high demand for our systems, with quarter over quarter, year over year profitability

  • Our research papers have been accepted into top machine learning and graphics conferences, including NeurIPS, ICCV, SIGGRAPH, and TOG

  • Health, dental, and vision coverage for you and your dependents

  • Wellness and Commuter stipends for select roles

  • 401k Plan with 2% company match (USA employees)

  • Flexible Paid Time Off Plan that we all actually use

A Final Note:

You do not need to match all of the listed expectations to apply for this position. We are committed to building a team with a variety of backgrounds, experiences, and skills.

Equal Opportunity Employer

Lambda is an Equal Opportunity employer. Applicants are considered without regard to race, color, religion, creed, national origin, age, sex, gender, marital status, sexual orientation and identity, genetic information, veteran status, citizenship, or any other factors prohibited by local, state, or federal law.

Similar Jobs

NetEase Games - International Tax Manager (SG)

NetEase Games

(On-Site)
5 Months ago
Lilith games - Overseas Advertising Manager

Lilith games

Shanghai, China (On-Site)
3 Weeks ago
appier - Campaign Executive

appier

Taipei City, Taiwan (On-Site)
7 Months ago
Tesla - Automotive Mechatronics Technician

Tesla

Hanau, Hessen, Germany (On-Site)
6 Months ago
PwC - Legal Associate

PwC

Bangkok, Bangkok, Thailand (On-Site)
10 Months ago
Enphase Energy - EVSE - Tech Lead / Senior Staff Backend Developer

Enphase Energy

Bengaluru, Karnataka, India (On-Site)
4 Months ago
krea.ai - Backend Engineer

krea.ai

San Francisco, California, United States (On-Site)
3 Weeks ago
GoTo Group - Senior Software Engineer (Backend) - Consumer Lending

GoTo Group

Jakarta, Indonesia (On-Site)
1 Month ago
LeoVegas - Backend Engineer - Payments

LeoVegas

Växjö, Kronoberg County, Sweden (Hybrid)
3 Months ago
ShyftLabs - Senior Backend Developer

ShyftLabs

Noida, Uttar Pradesh, India (On-Site)
9 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

upwork - Director of Payments & Financial Services Partnerships

upwork

United States (Remote)
1 Month ago
Workato - Engagement Manager

Workato

New York, United States (On-Site)
2 Months ago
Ajmera Infotech - Senior QA Engineer – Lead Test Strategy for Life-Critical Software

Ajmera Infotech

Ahmedabad, Gujarat, India (On-Site)
1 Month ago
EveryMatrix - Junior Corporate Legal Counsel

EveryMatrix

Sliema, Malta (Hybrid)
2 Months ago
Lionsgate - Director, Acquisition Strategy

Lionsgate

Santa Monica, California, United States (On-Site)
2 Months ago
Zuru - Influencer Research Executive

Zuru

Ahmedabad, Gujarat, India (On-Site)
1 Year ago
Granicus - Account Executive - New Business

Granicus

United States (Remote)
3 Months ago
Inveniolsi - SAP Ui5/Fiori Senior Consultant

Inveniolsi

Delhi, India (On-Site)
7 Months ago
Signal Space Lab - Lead Programmer

Signal Space Lab

Montreal, Quebec, Canada (On-Site)
4 Months ago
Hawkeye Innovations - Match Operations Assistant - Almaty

Hawkeye Innovations

Almaty, Almaty Region, Kazakhstan (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

Jobs in San Francisco, California, United States

ElevenLabs - Sales Enablement Lead

ElevenLabs

San Francisco, California, United States (Remote)
4 Months ago
2K - Senior Product Manager, Data Science

2K

Los Angeles, California, United States (On-Site)
3 Weeks ago
HP - Import / Export Compliance Specialist

HP

Houston, Texas, United States (On-Site)
2 Weeks ago
Zelis  - FP&A Manager, Cashflow and Capex

Zelis

St. Louis, Missouri, United States (Remote)
3 Weeks ago
The New York Times - Director, Measurement

The New York Times

New York, United States (Hybrid)
1 Month ago
Warm n fuzzy  - CG Generalist

Warm n fuzzy

Los Angeles, California, United States (On-Site)
1 Month ago
Apple - Engineering Project Manager, Apps Analytics

Apple

Cupertino, California, United States (On-Site)
3 Months ago
InnoPhase IoT - Principal PHY/MAC RTL Design Engineers

InnoPhase IoT

San Jose, California, United States (On-Site)
3 Months ago
frames store - FREELANCE: NUKE - LOS ANGELES

frames store

Los Angeles, California, United States (On-Site)
1 Year ago
hogarth - Sr. Client Finance Analyst

hogarth

New York, United States (Hybrid)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Backend Development Jobs

Capgemini - Java Backend

Capgemini

Pune, Maharashtra, India (On-Site)
3 Months ago
MURKA - Java Backend Developer

MURKA

(Remote)
4 Months ago
Backbone - Engineering Manager, Backend

Backbone

Seattle, Washington, United States (On-Site)
1 Year ago
bytedance - Backend Software Engineer - CapCut - Seattle (SEA)

bytedance

Seattle, Washington, United States (On-Site)
8 Months ago
PostHog - Backend Engineer

PostHog

United States (Remote)
4 Weeks ago
Playtika - Youda-PHP Developer

Playtika

Netherlands (Hybrid)
4 Months ago
Tellius - Software Engineer 1 - Backend

Tellius

Bengaluru, Karnataka, India (On-Site)
3 Months ago
Glean - Software Engineer, Backend

Glean

Palo Alto, California, United States (Hybrid)
1 Month ago
Moon Active - Backend Developer

Moon Active

Tel Aviv-Yafo, Tel Aviv District, Israel (Hybrid)
3 Weeks ago
Match Group - Senior Software Engineer, Backend

Match Group

Vancouver, British Columbia, Canada (Hybrid)
3 Weeks ago

Get notifed when new similar jobs are uploaded

About The Company

San Francisco, California, United States (Hybrid)

San Jose, California, United States (Hybrid)

San Francisco, California, United States (Hybrid)

San Francisco, California, United States (Hybrid)

San Jose, California, United States (Hybrid)

San Jose, California, United States (Hybrid)

San Francisco, California, United States (Hybrid)

San Francisco, California, United States (Hybrid)

San Jose, California, United States (Hybrid)

San Francisco, California, United States (Hybrid)

View All Jobs

Get notified when new jobs are added by Lambda

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug