Software Engineer, GenAI Model Quality

3 Months ago • 3 Years + • Quality Assurance • $160,000 PA - $192,000 PA

Job Summary

Job Description

As a Software Engineer on the Quality team at Scale AI, you will be responsible for building web-based applications to measure the quality of contributors' work. You will develop systems to monitor and flag quality issues in large-scale data collections, utilizing Benchmark testing, ML models, and various evaluation strategies. The role involves owning significant areas of the product, building features end-to-end, and delivering high-quality experiments quickly. You will work collaboratively with cross-functional teams to design and launch new features, while also mentoring less experienced engineers and contributing to the engineering team's culture and processes.
Must have:
  • At least 3 years of relevant experience is preferred.
  • Track record of shipping high-quality products at scale.
  • Ability to work in a fast-paced environment.
  • Ability to turn business ideas into engineering solutions.
  • Excellent problem-solving skills, working independently or in a team.

Job Details

About Scale

At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we’re accelerating the abundance of frontier data to pave the road to Artificial General Intelligence (AGI), and building upon our prior model evaluation work with enterprise customers and governments, to deepen our capabilities and offerings for both public and private evaluations.

About Data Engine

Our Generative AI Data Engine powers the world’s most advanced LLMs and generative models through world-class RLHF (Reinforcement Learning with Human Feedback), human data generation, model evaluation, safety, and alignment. The data we are producing is some of the most important work for how humanity will interact with AI.

About our Quality team

Generating high-quality data is the core problem our business solves. Our Quality team focuses on building web-based applications that help measure our contributors' quality. As a Software Engineer on the team, you’ll focus on building systems that monitor and flag quality issues with large-scale data collections. You will invest in Benchmark testing, ML models, heuristics, and other evaluation strategies to measure contributors' quality on a variety of Gen AI tasks. 

Responsibilities:

  • Own large areas within our product
  • Build features end-to-end: front-end, back-end, system design, debugging and testing
  • Deliver experiments at a high velocity and level of quality to engage our customers
  • Work across the entire product lifecycle from conceptualization through production
  • Influence the culture, values, and processes of a growing engineering team
  • Inspire and mentor less experienced engineers
  • Collaborating with cross-functional teams to define, design, and ship new product features and experiences.

Requirements:

  • At least 3 years of relevant experience is preferred
  • Track record of shipping high-quality products and features at scale
  • Desire to work in a very fast-paced environment
  • Ability to turn business and product ideas into engineering solutions
  • Excellent problem-solving skills, and be able to work independently or as part of a team.
  • Excited to join a dynamic, hybrid team in either San Francisco or New York City



Similar Jobs

bytedance - Backend Software Engineer - Customer Service Platform - Seattle

bytedance

Seattle, Washington, United States (On-Site)
9 Months ago
CD PROJEKT RED - Data Scientist

CD PROJEKT RED

Warsaw, Masovian Voivodeship, Poland (On-Site)
3 Months ago
Apple - Strategic EPM

Apple

Cupertino, California, United States (On-Site)
4 Weeks ago
Capgemini - Java Backend

Capgemini

Pune, Maharashtra, India (On-Site)
2 Months ago
ISS Stoxx - Senior Web Developer (WordPress)

ISS Stoxx

Makati City, Metro Manila, Philippines (Hybrid)
1 Month ago
Beta Craft - QA Engineer

Beta Craft

Pune, Maharashtra, India (On-Site)
2 Months ago
HCL Tech - Senior Test Lead with Selenium, Java

HCL Tech

California, United States (On-Site)
2 Months ago
original games - Senior QA Engineer

original games

(Remote)
2 Months ago
The Walt Disney Company - Mechanical Parts Quality Inspector

The Walt Disney Company

Île-de-France, France (On-Site)
5 Months ago
UXBERT Labs - Senior QA Engineer

UXBERT Labs

Riyadh, Riyadh Province, Saudi Arabia (Hybrid)
7 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Canva - Staff Backend Engineer (Java)

Canva

Auckland, Auckland, New Zealand (Remote)
4 Weeks ago
Axon - Order Management Analyst

Axon

Boston, Massachusetts, United States (On-Site)
4 Weeks ago
Visa - Director, Visa Direct Global Strategy & Operations

Visa

Atlanta, Georgia, United States (Hybrid)
4 Weeks ago
Kavalirio - SME Application Developer

Kavalirio

Herndon, Virginia, United States (On-Site)
2 Months ago
TransUnion - Advisor, Data Science and Analytics

TransUnion

(Remote)
2 Months ago
Maliyo Games - Graphics Designer

Maliyo Games

Nigeria (On-Site)
8 Months ago
OKX - Senior Audit Manager, FinCrime

OKX

New York, United States (On-Site)
2 Months ago
Demandbase - Staff DSP Software Engineer

Demandbase

San Francisco, California, United States (Remote)
1 Month ago
Qualcomm - Machine Learning & AI ADAS Applications Senior Staff Engineer

Qualcomm

Shanghai, China (On-Site)
2 Months ago
Tesla - Senior Manufacturing Engineer Process Equipment Paint

Tesla

Brandenburg, Germany (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

Jobs in San Francisco, California, United States

Techstar group - Project Manager

Techstar group

Irving, Texas, United States (On-Site)
2 Years ago
Playstation - Producer, Systems Design

Playstation

Los Angeles, California, United States (On-Site)
1 Month ago
zeta - Senior Product Head, HSA and Employee Benefits

zeta

United States (Remote)
2 Months ago
Whatnot - Recruiting Manager, Early Talent

Whatnot

Los Angeles, California, United States (On-Site)
2 Months ago
Toast - Territory Account Executive

Toast

Port St. Lucie, Florida, United States (On-Site)
2 Months ago
Sprinkler - Vice President, GTM Systems

Sprinkler

United States (Remote)
1 Year ago
Rippling - Staff Backend Engineer - Device Products

Rippling

San Francisco, California, United States (On-Site)
4 Weeks ago
Hudl - Senior Quality Assurance Engineer

Hudl

Dallas, Texas, United States (Remote)
2 Months ago
Glean - Enterprise Account Executive

Glean

Detroit, Michigan, United States (Remote)
4 Weeks ago
Bazaar Voice - Sales Development Representative

Bazaar Voice

Austin, Texas, United States (Hybrid)
1 Month ago

Get notifed when new similar jobs are uploaded

Quality Assurance Jobs

BetterMe - Manual QA Engineer (Android)

BetterMe

Kyiv, Kyiv City, Ukraine (Remote)
2 Months ago
Fanatee - Quality Assurance Engineer

Fanatee

Spain (Hybrid)
4 Months ago
Krafton India  - Sr Manager - QA

Krafton India

Bengaluru, Karnataka, India (On-Site)
2 Months ago
Coherent corp. - Manufacturing Quality Inspector II

Coherent corp.

Philadelphia, Pennsylvania, United States (On-Site)
3 Months ago
Contentstack - Senior Engineer II - QA (Playwright)

Contentstack

Pune, Maharashtra, India (Hybrid)
1 Month ago
Drive mode - QA Engineer

Drive mode

Tokyo, Japan (Hybrid)
4 Months ago
Coherent corp. - QA Metrology Technician

Coherent corp.

Saxonburg, Pennsylvania, United States (On-Site)
1 Month ago
Mendix - Quality Engineer

Mendix

Rotterdam, South Holland, Netherlands (Hybrid)
3 Months ago
Zones - QA Engineer, Insights & Analytics

Zones

Islamabad, Islamabad Capital Territory, Pakistan (On-Site)
1 Month ago
Scale AI - Senior Fullstack Software Engineer, GenAI Model Quality

Scale AI

San Francisco, California, United States (Hybrid)
3 Months ago

Get notifed when new similar jobs are uploaded

About The Company

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

London, England, United Kingdom (On-Site)

London, England, United Kingdom (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (Hybrid)

View All Jobs

Get notified when new jobs are added by Scale AI

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug