LLM Performance Operation-Analysts

6 Months ago • All levels • Monetization

Job Summary

Job Description

As a core member of our LLM Global Data Team, you'll be at the heart of our operations. Gain first-hand experience in understanding how to measure and analyze performance of LLM on various dimensions. Your role will involve: 1. Gaining direct experience in LLM and LLM-relevant product performance measurement, including creating metrics, designing and conducting tests, to help tech and product teams to understand LLM and product performance. 2. Delivering performance and feedback analysis to identify issues in LLM training and product design. 3. Collaborating with product managers and engineers to build testing workflows, create feedback loops, and enhance the effectiveness of testing processes and standards. 4. Working closely with the annotation team to gauge the quality of training data sets.
Must have:
  • Professional proficiency in English with excellent writing and critique skills.
  • Critical thinking and analytical skills.
  • Comfort with meeting deadlines and generating creative solutions.
Good to have:
  • Experience in consulting, market research, auditing, operational process design and improvement.
  • Experience as NLP model and search product performance evaluators.
  • A deep interest in LLM.

Job Details

Responsibilities
About ByteDance Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok, Helo, and Resso, as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content. Why Join Us Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible. Together, we inspire creativity and enrich life - a mission we aim towards achieving every day. To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always. At ByteDance, we create together and grow together. That's how we drive impact - for ourselves, our company, and the users we serve. Join us. As a core member of our LLM Global Data Team, you'll be at the heart of our operations. Gain first-hand experience in understanding how to measure and analyze performance of LLM on various dimensions. Your Role Will Involve: 1. Gaining direct experience in LLM and LLM-relevant product performance measurement, including creating metrics, designing and conducting tests, to help tech and product teams to understand LLM and product performance. 2. Delivering performance and feedback analysis to identify issues in LLM training and product design. 3. Collaborating with product managers and engineers to build testing workflows, create feedback loops, and enhance the effectiveness of testing processes and standards. 4. Working closely with the annotation team to gauge the quality of training data sets.
Qualifications
Minimum Qualifications: 1. Professional proficiency in English with excellent writing and critique skills. 2. Critical thinking and analytical skills are required. 3. Comfort with meeting deadlines and generating creative solutions, including the use of technology and tools, to enhance the quality of both individual and team outputs. Preferred Qualifications: 1. Experience in consulting, market research, auditing, operational process design and improvement is valued. Experience as NLP model and search product performance evaluators is strongly preferred. 2. A deep interest in LLM is essential. The ideal candidate is an enthusiastic learner who finds engagement with diverse case studies stimulating. Note:This role requires a paper test prior to interviews. ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Similar Jobs

Pubg Emea - Investment Analyst

Pubg Emea

Amsterdam, North Holland, Netherlands (On-Site)
3 Months ago
Super - Senior Product Designer, Core App Experiences

Super

(Remote)
2 Months ago
PwC - IN-Director_Oil & Gas_Fuels & Resources_Advisory_Mumbai

PwC

Mumbai, Maharashtra, India (On-Site)
6 Months ago
ByteDance - Technology Internal Audit Lead

ByteDance

San Jose, California, United States (Hybrid)
4 Months ago
ByteDance - Automotive Industry Product Sales - Hong Kong

ByteDance

(On-Site)
1 Month ago
ByteDance - Privacy Security Operation - Information System - Singapore

ByteDance

Singapore (On-Site)
4 Months ago
ByteDance - Senior Machine Learning Engineer - AML Algorithm

ByteDance

Seattle, Washington, United States (On-Site)
6 Months ago
ByteDance - Senior Natural Language Processing Algorithm Engineer

ByteDance

Seattle, Washington, United States (On-Site)
1 Month ago
Paytm - Ads Sales - Manager - Noida

Paytm

Noida, Uttar Pradesh, India (On-Site)
6 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Warner Bros Games - Lead UX Researcher

Warner Bros Games

Bellevue, Washington, United States (Hybrid)
1 Month ago
Playtech - Customer Success Manager

Playtech

Las Vegas, Nevada, United States (On-Site)
1 Month ago
NVIDIA - Senior Manager, Internal Audit and SOX

NVIDIA

Santa Clara, California, United States (On-Site)
1 Month ago
PwC - IN-Senior Associate_ JAVA_Utility Transformation _Advisory_Kolkata

PwC

Kolkata, West Bengal, India (On-Site)
4 Months ago
PwC - Executive - Data Analyst

PwC

Colombo, Western Province, Sri Lanka (On-Site)
7 Months ago
PwC - Senior de Auditoria Interna (Caracas)

PwC

Caracas, Capital District, Venezuela (On-Site)
7 Months ago
Fortis Games - Senior Cloud Security Engineer

Fortis Games

Hungary (On-Site)
2 Months ago
Relax Gaming  - Partnership Delivery Coordinator/Manager

Relax Gaming

Sliema, Malta (Hybrid)
1 Month ago
PwC - Deals - Consultant, Forensic Services - Subcontractor Opportunity

PwC

Amman, Amman Governorate, Jordan (On-Site)
6 Months ago
PwC - Senior Associate Forensic BCN

PwC

Barcelona, Catalonia, Spain (On-Site)
7 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

Nagarro - Principal Engineer, Java Fullstack

Nagarro

Singapore, Singapore (On-Site)
6 Months ago
ByteDance - Site Reliability Engineer Lead, Security Engineering

ByteDance

Singapore (On-Site)
6 Months ago
Riot Games - Product Coordinator, VALORANT SEA (Contract)

Riot Games

Singapore (On-Site)
3 Months ago
ByteDance - BNPL Operations Manager - Global Payment

ByteDance

Singapore (On-Site)
2 Months ago
ByteDance - Asset Manager, Datacenter Infrastructure and Services

ByteDance

Singapore (On-Site)
1 Month ago
Netflix - Senior Manager, AV Production Lead - APAC

Netflix

Singapore, Singapore (On-Site)
4 Months ago
ByteDance - Corporate Level Risk Control Business Partner

ByteDance

Singapore (On-Site)
3 Months ago
ByteDance - Senior Site Reliability Engineer, CDN

ByteDance

Singapore (On-Site)
6 Months ago
Alpha Sense - Product Specialist

Alpha Sense

Singapore, Singapore (On-Site)
5 Months ago
The Walt Disney Company - Senior Manager, Travel Operations & Business System Management

The Walt Disney Company

Singapore, Singapore (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Monetization Jobs

ByteDance - Financial Risk Strategy Expert - Global Payment

ByteDance

Singapore (On-Site)
6 Months ago
ByteDance - AI Research Scientist Graduate, Large Language Model (Applied Machine Learning) - 2024 Start (PhD)

ByteDance

San Jose, California, United States (On-Site)
6 Months ago
ByteDance - Graduate Development Program (Philippines E-commerce) - 2025 Start

ByteDance

Taguig, Metro Manila, Philippines (On-Site)
6 Months ago
Amanotes - Ads Monetization Specialist

Amanotes

Ho Chi Minh City, Ho Chi Minh City, Vietnam (On-Site)
1 Month ago
Garena - Growth & Monetization Strategy

Garena

Casablanca, Casablanca-Settat, Morocco (On-Site)
6 Months ago
ByteDance - Research Scientist, Reinforcement Learning

ByteDance

San Jose, California, United States (On-Site)
6 Months ago
Goodgame Studios - Monetization & Sales Strategist - Empire franchise

Goodgame Studios

Hamburg, Hamburg, Germany (Hybrid)
1 Month ago
CrazyLabs - Senior Growth Analyst

CrazyLabs

Berlin, Berlin, Germany (On-Site)
2 Months ago
Netflix - Associate, Ads Revenue & Yield Analytics

Netflix

New York, New York, United States (On-Site)
2 Months ago
Voodoo - Product Lead - Paper.io 2

Voodoo

Paris, Île-de-France, France (Remote)
1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Where imagination meets innovation, delivering limitless gaming experiences.

San Diego, California, United States (On-Site)

San Jose, California, United States (On-Site)

Dubai, Dubai, United Arab Emirates (On-Site)

New York, New York, United States (On-Site)

San Jose, California, United States (On-Site)

San Jose, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by ByteDance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug