Software Engineer, GenAI Model Quality

1 Day ago • 3 Years + • $160,000 PA - $192,000 PA

Job Summary

Job Description

As a Software Engineer on the Quality team at Scale AI, you will be responsible for building web-based applications to measure the quality of contributors' work. You will develop systems to monitor and flag quality issues in large-scale data collections, utilizing Benchmark testing, ML models, and various evaluation strategies. The role involves owning significant areas of the product, building features end-to-end, and delivering high-quality experiments quickly. You will work collaboratively with cross-functional teams to design and launch new features, while also mentoring less experienced engineers and contributing to the engineering team's culture and processes.
Must have:
  • At least 3 years of relevant experience is preferred.
  • Track record of shipping high-quality products at scale.
  • Ability to work in a fast-paced environment.
  • Ability to turn business ideas into engineering solutions.
  • Excellent problem-solving skills, working independently or in a team.

Job Details

About Scale

At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we’re accelerating the abundance of frontier data to pave the road to Artificial General Intelligence (AGI), and building upon our prior model evaluation work with enterprise customers and governments, to deepen our capabilities and offerings for both public and private evaluations.

About Data Engine

Our Generative AI Data Engine powers the world’s most advanced LLMs and generative models through world-class RLHF (Reinforcement Learning with Human Feedback), human data generation, model evaluation, safety, and alignment. The data we are producing is some of the most important work for how humanity will interact with AI.

About our Quality team

Generating high-quality data is the core problem our business solves. Our Quality team focuses on building web-based applications that help measure our contributors' quality. As a Software Engineer on the team, you’ll focus on building systems that monitor and flag quality issues with large-scale data collections. You will invest in Benchmark testing, ML models, heuristics, and other evaluation strategies to measure contributors' quality on a variety of Gen AI tasks. 

Responsibilities:

  • Own large areas within our product
  • Build features end-to-end: front-end, back-end, system design, debugging and testing
  • Deliver experiments at a high velocity and level of quality to engage our customers
  • Work across the entire product lifecycle from conceptualization through production
  • Influence the culture, values, and processes of a growing engineering team
  • Inspire and mentor less experienced engineers
  • Collaborating with cross-functional teams to define, design, and ship new product features and experiences.

Requirements:

  • At least 3 years of relevant experience is preferred
  • Track record of shipping high-quality products and features at scale
  • Desire to work in a very fast-paced environment
  • Ability to turn business and product ideas into engineering solutions
  • Excellent problem-solving skills, and be able to work independently or as part of a team.
  • Excited to join a dynamic, hybrid team in either San Francisco or New York City



Similar Jobs

Converse.AI - Back-End Developer

Converse.AI

(Remote)
2 Years ago
Life church - Staff Quality Engineer

Life church

Edmond, Oklahoma, United States (On-Site)
6 Months ago
Toku - Full-Stack Engineer

Toku

(Remote)
1 Day ago
Scale AI - Engineering Manager, Contributor Quality

Scale AI

San Francisco, California, United States (Hybrid)
1 Day ago
Codeway - Fullstack Developer

Codeway

İstanbul, Türkiye (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

ByteDance - Software Development Engineer, Network Automation - San Jose

ByteDance

San Jose, California, United States (On-Site)
6 Months ago
ByteDance - (EUI) Solutions Architect - Byteplus

ByteDance

London, England, United Kingdom (On-Site)
6 Months ago
Mozilla - Credential Management Student Worker

Mozilla

(Remote)
7 Hours ago
Google - Software Engineer III, Full Stack, Corporate Engineering

Google

Hyderabad, Telangana, India (On-Site)
2 Days ago
The Walt Disney Company - Senior Product Manager II

The Walt Disney Company

Santa Monica, California, United States (On-Site)
3 Months ago
The Walt Disney Company - Sr. Technical Program Manager, Live Stream Tooling

The Walt Disney Company

Morrisville, North Carolina, United States (On-Site)
1 Month ago
Google - Software Engineer III, Full Stack, Google Cloud Platforms

Google

New York, New York, United States (On-Site)
2 Days ago
DMarket - Senior Back-end Developer

DMarket

Ukraine (Remote)
1 Month ago
ION - Senior C#/.NET Developer, Budapest

ION

Budapest, Hungary (Hybrid)
6 Months ago
ByteDance - Backend Engineer Intern, Video-On-Demand

ByteDance

Singapore (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Jobs in San Francisco, California, United States

Nintendo - Senior Software Engineer (NTD)

Nintendo

Redmond, Washington, United States (On-Site)
8 Months ago
Google - Student Researcher, BS/MS, Winter/Summer 2025

Google

Mountain View, California, United States (On-Site)
5 Months ago
Linden Lab - Manager, Engineering - Mobile

Linden Lab

San Francisco, California, United States (Remote)
9 Months ago
Mattel  Inc  - American Girl Retail Associate (Seasonal)

Mattel Inc

Chicago, Illinois, United States (On-Site)
2 Weeks ago
InfoStretch Corporation - AI Developer

InfoStretch Corporation

Lansing, Michigan, United States (On-Site)
1 Month ago
Info Stretch - Business Analyst 5

Info Stretch

Lansing, Michigan, United States (Remote)
1 Month ago
Tencent - Principal Cinematic Artist

Tencent

Washington, United States (On-Site)
2 Months ago
The Walt Disney Company - Crowds Artist

The Walt Disney Company

Burbank, California, United States (On-Site)
1 Month ago
IGT - Lottery Field Service Technician II- (Vacaville/Fairfield , CA)

IGT

California, United States (On-Site)
4 Months ago
buildstsaff - Java Developer

buildstsaff

Alexandria, Virginia, United States (On-Site)
6 Years ago

Get notifed when new similar jobs are uploaded

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

About The Company

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

Doha, Doha Municipality, Qatar (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by Scale AI

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug