Software Engineer, ML Infrastructure - Training Platform

3 Minutes ago • 4 Years + • Devops • $160,000 PA - $225,600 PA

Job Summary

Job Description

Scale is looking for an AI/ML Infrastructure Engineer to join our Machine Learning Infrastructure team to build out our Training Platform. You will partner closely with Machine Learning researchers to understand their requirements and apply your own domain expertise and our compute resources to accelerate experimentation throughput. The ideal candidate has strong fundamentals in machine learning, backend system design, and prior ML Infrastructure experience, comfortable with infrastructure and large scale system design, and diagnosing both model performance and system failures.
Must have:
  • Build highly available, observable, performant, and cost-effective APIs for model training.
  • Participate in our team’s on call process to ensure the availability of our services.
  • Own projects end-to-end, from requirements, scoping, design, to implementation, in a highly collaborative and cross-functional environment.
  • Exercise good taste in building systems and tools and know when to make build vs. buy tradeoffs, with an eye for cost efficiency.
Good to have:
  • LLM inference latency optimization techniques
  • kernel fusion
  • quantization
  • dynamic batching
  • cloud technology stack
  • AWS
  • GCP
Perks:
  • Comprehensive health, dental and vision coverage
  • retirement benefits
  • a learning and development stipend
  • generous PTO
  • commuter stipend

Job Details

Scale is looking for an AI/ML Infrastructure Engineer to join our Machine Learning Infrastructure team to build out our Training Platform. You will partner closely with Machine Learning researchers to understand their requirements and apply your own domain expertise and our compute resources to accelerate experimentation throughput.

The ideal candidate is someone who has strong fundamentals in machine learning, backend system design, and has prior ML Infrastructure experience. You should also be comfortable with infrastructure and large scale system design, as well as diagnosing both model performance and system failures.

You will:

  • Build highly available, observable, performant, and cost-effective APIs for model training.
  • Participate in our team’s on call process to ensure the availability of our services.
  • Own projects end-to-end, from requirements, scoping, design, to implementation, in a highly collaborative and cross-functional environment.
  • Exercise good taste in building systems and tools and know when to make build vs. buy tradeoffs, with an eye for cost efficiency.

Ideally you'd have:

  • 4+ years of experience building machine learning training pipelines or inference services in a production setting.
  • Experience with distributed training techniques such as DeepSpeed, FSDP, etc.
  • Experience building, deploying, and monitoring complex microservice architectures.
  • Experience with Python, Docker, Kubernetes, and Infrastructure as code (e.g. terraform).

Nice to haves:

  • Experience with LLM inference latency optimization techniques, e.g. kernel fusion, quantization, dynamic batching, etc.
  • Experience working with a cloud technology stack (eg. AWS or GCP).

Compensation packages at Scale for eligible roles include base salary, equity, and benefits. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position, determined by work location and additional factors, including job-related skills, experience, interview performance, and relevant education or training. Scale employees in eligible roles are also granted equity based compensation, subject to Board of Director approval. Your recruiter can share more about the specific salary range for your preferred location during the hiring process, and confirm whether the hired role will be eligible for equity grant. You’ll also receive benefits including, but not limited to: Comprehensive health, dental and vision coverage, retirement benefits, a learning and development stipend, and generous PTO. Additionally, this role may be eligible for additional benefits such as a commuter stipend.

Please reference the job posting's subtitle for where this position will be located. For pay transparency purposes, the base salary range for this full-time position in the locations of San Francisco, New York, Seattle is:

$160,000 - $225,600 USD

PLEASE NOTE: Our policy requires a 90-day waiting period before reconsidering candidates for the same role. This allows us to ensure a fair and thorough evaluation of all applicants.

About Us:

At Scale, we believe that the transition from traditional software to AI is one of the most important shifts of our time. Our mission is to make that happen faster across every industry, and our team is transforming how organizations build and deploy AI. Our products power the world's most advanced LLMs, generative models, and computer vision models. We are trusted by generative AI companies such as OpenAI, Meta, and Microsoft, government agencies like the U.S. Army and U.S. Air Force, and enterprises including GM and Accenture. We are expanding our team to accelerate the development of AI applications.

We believe that everyone should be able to bring their whole selves to work, which is why we are proud to be an inclusive and equal opportunity workplace. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability status, gender identity or Veteran status.

We are committed to working with and providing reasonable accommodations to applicants with physical and mental disabilities. If you need assistance and/or a reasonable accommodation in the application or recruiting process due to a disability, please contact us at accommodations@scale.com. Please see the United States Department of Labor's Know Your Rights poster for additional information.

We comply with the United States Department of Labor's Pay Transparency provision.

PLEASE NOTE: We collect, retain and use personal data for our professional business purposes, including notifying you of job opportunities that may be of interest and sharing with our affiliates. We limit the personal data we collect to that which we believe is appropriate and necessary to manage applicants’ needs, provide our services, and comply with applicable laws. Any information we collect in connection with your application will be treated in accordance with our internal policies and programs designed to protect personal data. Please see our privacy policy for additional information.

Similar Jobs

Google - Staff Software Engineer, Infrastructure, Core

Google

Sunnyvale, California, United States (On-Site)
3 Months ago
Riot Games - Senior Manager, Game Production - League of Legends

Riot Games

Singapore (On-Site)
6 Months ago
aspyr - Head of Creative

aspyr

Austin, Texas, United States (On-Site)
2 Months ago
Rippling - Revenue Transformation Lead

Rippling

San Francisco, California, United States (On-Site)
3 Weeks ago
Cadence - STA/EMIR IC Solutions Architect

Cadence

Austin, Texas, United States (On-Site)
1 Year ago
bytedance - Site Reliability Engineer Graduate (Technical Infrastructure) - 2025 Start (BS/MS)

bytedance

Seattle, Washington, United States (On-Site)
8 Months ago
Interactive Brokers - Senior Cloud Platform Engineer

Interactive Brokers

Greenwich, Connecticut, United States (Hybrid)
2 Months ago
Jam City - DevOps Engineer

Jam City

Montevideo, Montevideo Department, Uruguay (Hybrid)
1 Month ago
Animoca Brands - Senior DevOps Engineer

Animoca Brands

Hong Kong (On-Site)
10 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Cred - Operations Excellence

Cred

Bengaluru, Karnataka, India (On-Site)
11 Months ago
Marsh McLennan - Data Product Owner

Marsh McLennan

Bethesda, Maryland, United States (Hybrid)
2 Months ago
Zones - Regional Manager West

Zones

Mumbai, Maharashtra, India (On-Site)
4 Months ago
PwC - Senior Associate, Business Analyst, Data and Analytics, Advisory

PwC

Mumbai, Maharashtra, India (On-Site)
4 Days ago
Ziff Davis - CNET Content Writer I - Commerce

Ziff Davis

Pune, Maharashtra, India (On-Site)
3 Weeks ago
Canva - Staff Backend Software Engineer - Developer Experience

Canva

Auckland, Auckland, New Zealand (Remote)
3 Weeks ago
luxsoft - Senior React Developer

luxsoft

New Delhi, Delhi, India (Remote)
1 Month ago
Palo Alto Networks - Finance Business Manager - Strategic Finance

Palo Alto Networks

Santa Clara, California, United States (On-Site)
2 Weeks ago
SSC Technologies - Head of Internal Audit

SSC Technologies

London, England, United Kingdom (Hybrid)
1 Month ago
Single Store - Technical Account Manager

Single Store

Pune, Maharashtra, India (Remote)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in San Francisco, California, United States

Lorikeet - Solutions Engineer

Lorikeet

United States (Remote)
1 Month ago
The E.W. Scripps Company - Vice President, Total Rewards

The E.W. Scripps Company

Cincinnati, Ohio, United States (Hybrid)
2 Months ago
Yodlee - Director, Information Security Risk Manager

Yodlee

Berwyn, Pennsylvania, United States (Hybrid)
1 Month ago
Zscaler - Senior Staff Software Engineer - API Tooling and Frameworks

Zscaler

San Jose, California, United States (On-Site)
3 Weeks ago
GOAT Group - Junior Authentication Training Specialist

GOAT Group

Atlanta, Georgia, United States (On-Site)
6 Days ago
Qualcomm - Senior Analog Mixed Signal Design Engineer

Qualcomm

San Diego, California, United States (On-Site)
2 Months ago
HCL Tech - Support Lead - ITSM, Telecom

HCL Tech

California, United States (On-Site)
2 Months ago
Lambda - Senior Software Engineer - Inference Apps

Lambda

San Francisco, California, United States (Hybrid)
1 Month ago
Temporal Technologies - Staff Software Engineer - Cloud Data Storage

Temporal Technologies

United States (Remote)
3 Weeks ago
Intel  - Business Development Manager - Intel Foundry (Aerospace Defense and Government Business Group)

Intel

Santa Clara, California, United States (On-Site)
1 Year ago

Get notifed when new similar jobs are uploaded

Devops Jobs

Thales - DevOps Manager

Thales

Rehovot, Center District, Israel (Hybrid)
2 Months ago
endava - Senior Cloud Operations Engineer - AWS

endava

Iași, Iași County, Romania (On-Site)
1 Month ago
Hitachi - D365 F&O Senior Solution Architect

Hitachi

Chicago, Illinois, United States (Remote)
9 Months ago
Flexera - Senior Site Reliability Engineer

Flexera

Bengaluru, Karnataka, India (Hybrid)
10 Months ago
London stock Exchange - Cloud Site Reliability Engineer

London stock Exchange

Bengaluru, Karnataka, India (On-Site)
6 Days ago
CData Software - Platform Engineer

CData Software

Bengaluru, Karnataka, India (On-Site)
10 Months ago
Globalization Partners - Principal Solution Architect

Globalization Partners

United States (Remote)
5 Months ago
Easybrain - Build Engineer

Easybrain

(Remote)
2 Months ago
Zones - Inside Solutions Architect - UC & Collaboration

Zones

Islamabad, Islamabad Capital Territory, Pakistan (On-Site)
1 Month ago
Capgemini - Java Solution Architect

Capgemini

Pune, Maharashtra, India (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

About The Company

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

London, England, United Kingdom (On-Site)

London, England, United Kingdom (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (Hybrid)

Riyadh, Riyadh Province, Saudi Arabia (On-Site)

San Francisco, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by Scale AI

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug