Senior Software Engineer, GenAI Model Quality

3 Months ago • 5 Years + • Quality Assurance • $188,000 PA - $225,600 PA

Job Summary

Job Description

As a Senior Software Engineer on the Quality team at Scale AI, you will be responsible for building web-based applications that monitor and flag quality issues within large-scale data collections for generative AI models. Your role involves developing systems for benchmark testing, utilizing ML models, and implementing heuristics to evaluate contributors' quality across various Gen AI tasks. This position requires owning significant areas within the product, building features end-to-end, and delivering experiments quickly. You'll collaborate with cross-functional teams, mentor other engineers, and influence the engineering culture.
Must have:
  • 5+ years relevant experience in software engineering.
  • Proven track record of shipping high-quality products.

Job Details

About Scale

At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we’re accelerating the abundance of frontier data to pave the road to Artificial General Intelligence (AGI), and building upon our prior model evaluation work with enterprise customers and governments, to deepen our capabilities and offerings for both public and private evaluations.

About Data Engine

Our Generative AI Data Engine powers the world’s most advanced LLMs and generative models through world-class RLHF (Reinforcement Learning with Human Feedback), human data generation, model evaluation, safety, and alignment. The data we are producing is some of the most important work for how humanity will interact with AI.

About our Quality team

Generating high-quality data is the core problem our business solves. Our Quality team focuses on building web-based applications that help measure our contributors' quality. As a Software Engineer on the team, you’ll focus on building systems that monitor and flag quality issues with large-scale data collections. You will invest in Benchmark testing, ML models, heuristics, and other evaluation strategies to measure contributors' quality on a variety of Gen AI tasks. 

Responsibilities:

  • Own large areas within our product
  • Build features end-to-end: front-end, back-end, system design, debugging and testing
  • Deliver experiments at a high velocity and level of quality to engage our customers
  • Work across the entire product lifecycle from conceptualization through production
  • Influence the culture, values, and processes of a growing engineering team
  • Inspire and mentor less experienced engineers
  • Collaborating with cross-functional teams to define, design, and ship new product features and experiences.

Requirements:

  • At least 5 years of relevant experience is preferred
  • Track record of shipping high-quality products and features at scale
  • Desire to work in a very fast-paced environment
  • Ability to turn business and product ideas into engineering solutions
  • Excellent problem-solving skills, and be able to work independently or as part of a team.
  • Excited to join a dynamic, hybrid team in either San Francisco or New York City



Similar Jobs

NXP - Sr. Event Program Manager

NXP

Austin, Texas, United States (On-Site)
2 Months ago
Super.com - Senior Product Manager, Cash Advance

Super.com

United States (Remote)
1 Month ago
Remote - Senior RevOps Processes Specialist

Remote

(Remote)
2 Months ago
Niantic - Senior Game Designer

Niantic

San Francisco, California, United States (Hybrid)
1 Month ago
Tesla - EHS Technical Engineer R&D

Tesla

Baden-Württemberg, Germany (On-Site)
5 Months ago
Capgemini - Test Automation Lead/Manager

Capgemini

Bengaluru, Karnataka, India (On-Site)
2 Months ago
Epic Games - Quality Assurance Programmer

Epic Games

Montreal, Quebec, Canada (On-Site)
3 Months ago
Marvell - Staff engineer (Baseband PHY & SoC Validation)

Marvell

Bengaluru, Karnataka, India (On-Site)
2 Months ago
Apple - WiFi Connectivity Validation Engineer

Apple

Sunnyvale, California, United States (On-Site)
2 Months ago
The Walt Disney Company - Quality Inspector of Mechanical Parts

The Walt Disney Company

Île-de-France, France (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

NVIDIA - Production System Engineer

NVIDIA

Pune, Maharashtra, India (On-Site)
4 Months ago
Bungie - Contract Associate Creator Marketing Manager

Bungie

(Hybrid)
7 Months ago
Barracuda - Senior Product Marketing Manager

Barracuda

Chelmsford, Massachusetts, United States (On-Site)
5 Months ago
Jane Street - Regulatory Risk Management

Jane Street

New York, United States (On-Site)
4 Weeks ago
Netflix - Product Manager, Ad Insertion (Ads)

Netflix

Los Angeles, California, United States (On-Site)
6 Months ago
Outscal - Sales Lead

Outscal

Delhi, India (On-Site)
9 Months ago
Qualcomm - Director, Business Development - AT&T Account

Qualcomm

Austin, Texas, United States (On-Site)
2 Months ago
Wrike - Marketing Operations Program Coordinator

Wrike

Prague, Prague, Czechia (Hybrid)
2 Months ago
Zazz - Senior UI/UX Designer

Zazz

(Remote)
6 Months ago
Grab - Manager, Corporate Strategy

Grab

Pasig, Metro Manila, Philippines (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in San Francisco, California, United States

Apple - CAD Engineer - RTL Construction

Apple

Austin, Texas, United States (On-Site)
2 Months ago
Kavalirio - Embedded Software Engineer

Kavalirio

Colorado Springs, Colorado, United States (On-Site)
2 Months ago
Thumbtack - Senior Data Engineer, Marketing Technology

Thumbtack

United States (Remote)
2 Months ago
Rockstar Games - Associate Systems Engineer

Rockstar Games

Andover, Massachusetts, United States (On-Site)
2 Months ago
Postman - Account Development Representative

Postman

San Francisco, California, United States (Hybrid)
3 Months ago
The New York Times - Editor, Cooking Product Strategy

The New York Times

New York, United States (Hybrid)
2 Months ago
caliogo - Account Executive

caliogo

United States (On-Site)
4 Weeks ago
Palo Alto Networks - Finance Manager, Revenue Forecasting

Palo Alto Networks

Santa Clara, California, United States (On-Site)
1 Month ago
Nintendo - Senior UX Writer

Nintendo

Redmond, Washington, United States (Hybrid)
7 Months ago
HCL Tech - Technical Specialist - Siemens PLM

HCL Tech

Michigan, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Quality Assurance Jobs

luxsoft - Senior QA Automation Engineer

luxsoft

Bengaluru, Karnataka, India (On-Site)
2 Months ago
Zeeco, Inc. - QC Manager

Zeeco, Inc.

Lissone, Lombardy, Italy (On-Site)
8 Months ago
Playtika - Manual QA Engineer

Playtika

București, București, Romania (Hybrid)
3 Weeks ago
CD PROJEKT RED - Publishing QA Manager

CD PROJEKT RED

Warsaw, Masovian Voivodeship, Poland (Hybrid)
2 Months ago
Veeam Software - Python Developer in QA

Veeam Software

Prague, Czechia (Hybrid)
4 Weeks ago
binance - Senior QA Engineer (Trading Tech)

binance

Abu Dhabi, Abu Dhabi, United Arab Emirates (Hybrid)
1 Month ago
zoox - Senior Systems Engineer, Fleet Validation Lead

zoox

Foster City, California, United States (Hybrid)
2 Months ago
Apple - Software Quality Engineer - Applied Networking

Apple

Cupertino, California, United States (On-Site)
3 Weeks ago
Playtika - QA Engineer

Playtika

Netherlands (Hybrid)
6 Months ago
bounteous - QA Architect

bounteous

India (Hybrid)
4 Weeks ago

Get notifed when new similar jobs are uploaded

About The Company

San Francisco, California, United States (Hybrid)

San Francisco, California, United States (Hybrid)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

San Francisco, California, United States (On-Site)

London, England, United Kingdom (On-Site)

View All Jobs

Get notified when new jobs are added by Scale AI

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug