LLM Global Data - LLM Performance Operation Analyst

1 Month ago • All levels • Monetization

Job Summary

Job Description

As an LLM Global Data - LLM Performance Operation Analyst at ByteDance, you'll measure and analyze the performance of Large Language Models (LLMs) across various dimensions. Responsibilities include creating metrics, designing and conducting tests to understand LLM and product performance, delivering performance analysis to identify issues in LLM training and product design, collaborating with product teams to build testing workflows and feedback loops, and working with the annotation team to assess training data quality. The role requires strong analytical, project management, and communication skills, with experience in NLP model and search product performance evaluation being highly preferred. A deep interest in LLMs and a passion for creative problem-solving are essential.
Must have:
  • Professional English proficiency
  • Critical thinking & analytical skills
  • Strong project management
  • LLM performance measurement expertise
  • Collaboration with tech & product teams
Good to have:
  • Consulting/market research experience
  • NLP model & search product evaluation experience
  • Deep interest in LLMs

Job Details

Responsibilities
About ByteDance Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content. Why Join Us Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible. Together, we inspire creativity and enrich life - a mission we aim towards achieving every day. To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always. At ByteDance, we create together and grow together. That's how we drive impact-for ourselves, our company, and the users we serve. Join us. As a core member of our LLM Global Data Team, you'll be at the heart of our operations. Gain first-hand experience in understanding how to measure and analyze performance of LLM on various dimensions. Your Role Will Involve: 1. Gaining direct experience in LLM and LLM-relevant product performance measurement, including creating metrics & designing and conducting tests, to help tech and product teams to understand LLM and product performance. 2. Delivering performance and feedback analysis to identify issues in LLM training and product design. 3. Collaborating with product managers and engineers to build testing workflows, create feedback loops, and enhance the effectiveness of testing processes and standards. 4. Working closely with the annotation team to gauge the quality of training data sets.
Qualifications
Minimum Qualifications: 1. Professional proficiency in English with excellent writing and critique skills. 2. Critical thinking, numerate and analytical skills are required 3. Strong project management capabilities with proven ability to work effectively with both internal and external stakeholders; comfortable handling ambiguous requirements and adapting to changes while maintaining project momentum 4. Comfortable with meeting deadlines and generating creative solutions, including the use of technology and tools, to enhance the quality of both individual and team outputs Preferred Qualifications: 1. Experience in consulting, market research, auditing, operational process design and improvement is valued. Experience as NLP model and search product performance evaluation is strongly preferred. 2. A deep interest in LLM is essential. The ideal candidate is an enthusiastic learner who finds engagement with diverse case studies stimulating. ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Similar Jobs

Google - Account Manager, Black Community Inclusion

Google

São Paulo, State Of São Paulo, Brazil (On-Site)
2 Days ago
G5 Games - Game Product Manager

G5 Games

(Remote)
2 Weeks ago
Putnam Associates - Apprentice, HTA & Market Access

Putnam Associates

Paris, Île-de-France, France (Hybrid)
8 Hours ago
In mobi - Director of Marketing, APAC

In mobi

Bengaluru, Karnataka, India (On-Site)
1 Day ago
Google - Account Associate, Scaled, Google Customer Solutions

Google

Beijing, Beijing, China (On-Site)
2 Days ago
ByteDance - Shop Tab Strategist

ByteDance

Taguig, Metro Manila, Philippines (On-Site)
1 Month ago
ByteDance - Account Management, Electronics (Philippines E-commerce)

ByteDance

Taguig, Metro Manila, Philippines (On-Site)
1 Month ago
ByteDance - Payment Strategy Intern - Global Payment

ByteDance

Singapore (On-Site)
1 Month ago
ByteDance - Software Engineer (Payment Network) Intern

ByteDance

Singapore (On-Site)
2 Months ago
ByteDance - TikTok Shop Fashion Category Manager SMB

ByteDance

New York, New York, United States (On-Site)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

The Pokemon Company International - Business Intelligence Analyst - Market Research

The Pokemon Company International

Bellevue, Washington, United States (Hybrid)
2 Weeks ago
Interface AI - Lead Product Manager - Conversational AI

Interface AI

United States (Remote)
2 Months ago
Plarium - Survey Researcher

Plarium

Herzliya, Tel Aviv District, Israel (On-Site)
2 Months ago
Google - Senior Account Strategist, Mid-Market Sales

Google

São Paulo, State Of São Paulo, Brazil (On-Site)
2 Days ago
SmartBear - Senior Director of Product Marketing - API

SmartBear

Somerville, Massachusetts, United States (Hybrid)
1 Day ago
NewGlobe - Product Manager

NewGlobe

Lagos, Lagos, Nigeria (On-Site)
8 Hours ago
Kolibri Games - Creative Marketing Lead

Kolibri Games

Berlin, Berlin, Germany (Hybrid)
1 Month ago
Visa - Business Associate - Visa Leadership Accelerator Program (For Fresh Graduates)

Visa

Abidjan, Abidjan Autonomous District, Côte D'Ivoire (On-Site)
2 Months ago
TVH - Indirect Procurement Operations Category Buyer

TVH

Waregem, Flanders, Belgium (On-Site)
17 Hours ago
Booming games - Business Development Manager

Booming games

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

ByteDance - Strategy Manager - BytePlus

ByteDance

Singapore (On-Site)
6 Months ago
ByteDance - Data Engineer Intern - Global Payment - 2025 Start

ByteDance

Singapore (On-Site)
1 Month ago
ByteDance - Android/iOS Engineer, Flow - 2025 Start

ByteDance

Singapore (On-Site)
6 Months ago
ByteDance - Global Legal Compliance Counsel

ByteDance

Singapore (Hybrid)
5 Months ago
Google - Data Center Technician

Google

Singapore (On-Site)
2 Weeks ago
PwC - Risk Services - Change Management Specialist

PwC

Singapore (On-Site)
7 Months ago
Eleven Labs - Customer Success Manager - APAC

Eleven Labs

Singapore (Remote)
1 Month ago
ByteDance - Product Leader - Product Strategy - Singapore

ByteDance

Singapore (On-Site)
6 Months ago

Get notifed when new similar jobs are uploaded

Monetization Jobs

ByteDance - Software Engineer — Data Security

ByteDance

San Jose, California, United States (On-Site)
6 Months ago
ByteDance - Regional Head of Solution Architect, Cloud Security

ByteDance

Singapore (On-Site)
2 Months ago
ByteDance - Backend Software Engineer Intern (Product RD and Infrastructure-Global E-Commerce)- 2025 Summer (BS/MS)

ByteDance

San Jose, California, United States (On-Site)
6 Months ago
ByteDance - Business Analysis & Operation Director

ByteDance

Singapore (On-Site)
6 Months ago
Playtika - Monetization Manager

Playtika

Israel (On-Site)
6 Months ago
Playtika - Director of Monetization

Playtika

Israel (On-Site)
2 Months ago
Netflix - Product Manager, Ad Insertion (Ads)

Netflix

Seattle, Washington, United States (On-Site)
2 Weeks ago
ByteDance - Senior Natural Language Processing Algorithm Engineer

ByteDance

Seattle, Washington, United States (On-Site)
1 Month ago
Tencent - Live Operation Manager 游戏商业化策划

Tencent

Shenzhen, Guangdong Province, China (On-Site)
9 Months ago
Plarium - Monetization Product Manager

Plarium

Helsinki, Uusimaa, Finland (Hybrid)
2 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Where imagination meets innovation, delivering limitless gaming experiences.

San Diego, California, United States (On-Site)

San Jose, California, United States (On-Site)

Dubai, Dubai, United Arab Emirates (On-Site)

New York, New York, United States (On-Site)

San Jose, California, United States (On-Site)

San Jose, California, United States (On-Site)

Seattle, Washington, United States (On-Site)

View All Jobs

Get notified when new jobs are added by ByteDance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug