LLM Global Data - LLM Performance Operation Analyst (Non-safety)

1 Month ago • All levels • Monetization

Job Summary

Job Description

As an LLM Global Data - LLM Performance Operation Analyst at ByteDance, you'll play a crucial role in measuring and analyzing the performance of Large Language Models (LLMs). Responsibilities include creating metrics, designing and conducting tests to understand LLM and product performance, delivering performance analysis to identify issues in LLM training and product design, collaborating with product managers and engineers to improve testing workflows and feedback loops, and working with the annotation team to assess training data quality. This role requires strong analytical, project management, and communication skills, along with a deep interest in LLMs. Experience in consulting, market research, or NLP model evaluation is preferred.
Must have:
  • Professional proficiency in English
  • Critical thinking & analytical skills
  • Strong project management capabilities
  • Comfortable with deadlines and creative solutions
  • Experience in LLM performance measurement
Good to have:
  • Consulting/market research experience
  • NLP model & search product evaluation experience
  • Deep interest in LLMs

Job Details

Responsibilities
About ByteDance Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content. Why Join Us Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible. Together, we inspire creativity and enrich life - a mission we aim towards achieving every day. To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always. At ByteDance, we create together and grow together. That's how we drive impact-for ourselves, our company, and the users we serve. Join us. As a core member of our LLM Global Data Team, you'll be at the heart of our operations. Gain first-hand experience in understanding how to measure and analyze performance of LLM on various dimensions. Your Role Will Involve: 1. Gaining direct experience in LLM and LLM-relevant product performance measurement, including creating metrics & designing and conducting tests, to help tech and product teams to understand LLM and product performance. 2. Delivering performance and feedback analysis to identify issues in LLM training and product design. 3. Collaborating with product managers and engineers to build testing workflows, create feedback loops, and enhance the effectiveness of testing processes and standards. 4. Working closely with the annotation team to gauge the quality of training data sets.
Qualifications
Minimum Qualifications: 1. Professional proficiency in English with excellent writing and critique skills. 2. Critical thinking, numerate and analytical skills are required 3. Strong project management capabilities with proven ability to work effectively with both internal and external stakeholders; comfortable handling ambiguous requirements and adapting to changes while maintaining project momentum 4. Comfortable with meeting deadlines and generating creative solutions, including the use of technology and tools, to enhance the quality of both individual and team outputs Preferred Qualifications: 1. Experience in consulting, market research, auditing, operational process design and improvement is valued. Experience as NLP model and search product performance evaluation is strongly preferred. 2. A deep interest in LLM is essential. The ideal candidate is an enthusiastic learner who finds engagement with diverse case studies stimulating. ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Similar Jobs

WebFX - Jr. Social Media Ads and Analytics Specialist

WebFX

Harrisburg, Pennsylvania, United States (On-Site)
7 Months ago
Starkflow - Business Development Executive

Starkflow

Gujarat, India (On-Site)
4 Months ago
VVater - Sales Representative

VVater

Austin, Texas, United States (On-Site)
1 Month ago
Google - Data and Measurement Account Manager

Google

Dublin, County Dublin, Ireland (On-Site)
1 Month ago
Google - Account Manager, LCS, Retail Marketplaces and Groceries

Google

Dublin, County Dublin, Ireland (On-Site)
1 Month ago
ByteDance - Software Engineer Intern (Privacy & Security - Global Payment)

ByteDance

San Jose, California, United States (On-Site)
2 Months ago
ByteDance - Student Researcher (Doubao (Seed) - Foundation Model - Speech Understanding) - 2025 Start (PhD)

ByteDance

San Jose, California, United States (On-Site)
7 Months ago
ByteDance - Research Scientist Graduates, Large Language Model (Doubao) - 2025 Start

ByteDance

San Jose, California, United States (On-Site)
7 Months ago
ByteDance - Compliance Officer, Philippines - Global Payment

ByteDance

Pasig, Metro Manila, Philippines (On-Site)
5 Months ago
ByteDance - Payment Strategy Intern (LATAM)

ByteDance

Singapore (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

YouGov - Graduate Database Administrator

YouGov

London, England, United Kingdom (Hybrid)
1 Month ago
YouGov - Research Manager

YouGov

Sydney, New South Wales, Australia (On-Site)
1 Month ago
IGG - Marketing Analyst

IGG

São Paulo, State Of São Paulo, Brazil (On-Site)
7 Months ago
Relax Gaming  - Game Design Director/Head of Games

Relax Gaming

Malmö, Skåne County, Sweden (Hybrid)
2 Months ago
Google - Account Product Strategist, LCS

Google

Buenos Aires, Buenos Aires, Argentina (On-Site)
1 Month ago
Zuru - Territory Manager

Zuru

Auckland, Auckland, New Zealand (Hybrid)
1 Month ago
Blazesoft - Senior Manager, Performance Marketing (iGaming)

Blazesoft

Vaughan, Ontario, Canada (On-Site)
6 Months ago
Daybreak Game Company LLC - Director, Brand Marketing

Daybreak Game Company LLC

San Diego, California, United States (Hybrid)
8 Months ago
Social Discovery Group - Head of Product (Premium products)

Social Discovery Group

Indonesia (Remote)
6 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

NinjaVan - Delivery Driver (Part-time/Flexi)

NinjaVan

Singapore, Singapore (On-Site)
7 Months ago
ByteDance - Security Expert, LLM-based Vulnerability Mining

ByteDance

Singapore (On-Site)
1 Month ago
NinjaVan - Fleet Supervisor

NinjaVan

Singapore, Singapore (On-Site)
7 Months ago
Microsoft - Senior Researcher

Microsoft

Singapore (On-Site)
1 Month ago
Google - Technical Program Manager, Global Network Delivery

Google

Singapore (On-Site)
1 Month ago
Google - Senior Solutions Acceleration Architect, Application

Google

Singapore (On-Site)
1 Month ago
IGG - HR & Admin Intern

IGG

Singapore (On-Site)
7 Months ago
Google - New Business Account Strategist

Google

Singapore (On-Site)
1 Month ago
SPH Media - The Straits Times Specialist Internship – Interactive Graphics (4 to 6 months)

SPH Media

Singapore, Singapore (On-Site)
10 Months ago
ByteDance - Compliance Legal Counsel, Global Sanctions

ByteDance

Singapore (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Monetization Jobs

ByteDance - Search Operations QA - Global E-commerce

ByteDance

Tokyo, Japan (On-Site)
1 Month ago
Playtika - Live Ops/Monetization Specialist

Playtika

Netherlands (Hybrid)
4 Months ago
InMobiInMobi - Senior Manager - Central Strategy & Operations

InMobiInMobi

Bengaluru, Karnataka, India (On-Site)
1 Month ago
InMobiInMobi - Manager, Programmatic Trading Desk Development, NA

InMobiInMobi

New York, New York, United States (On-Site)
5 Months ago
ByteDance - Software Engineer, Model Inference

ByteDance

Seattle, Washington, United States (On-Site)
1 Month ago
Netflix - Manager, FS&A Ads (Platform)

Netflix

Los Angeles, California, United States (On-Site)
7 Months ago
ByteDance - Video Codec Algorithm Engineer - Multimedia Lab

ByteDance

Seattle, Washington, United States (On-Site)
7 Months ago
Netflix - Software Engineer (L6), Ads Demand Management

Netflix

United States (Remote)
7 Months ago
Supercell - Monetization Manager, Supercell Store

Supercell

Helsinki, Uusimaa, Finland (On-Site)
6 Months ago
ByteDance - Software Engineer (User Product) Intern - Global Payment

ByteDance

Singapore (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

San Jose, California, United States (On-Site)

Tokyo, Japan (On-Site)

Taguig, Metro Manila, Philippines (On-Site)

San Jose, California, United States (On-Site)

Ho Chi Minh City, Vietnam (On-Site)

View All Jobs

Get notified when new jobs are added by bytedance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug