Software Engineer - Machine Learning Training

3 Months ago • 3 Years + • Research Development

Job Summary

Job Description

ByteDance's Large Model Team seeks a Software Engineer specializing in Machine Learning training in Singapore. This role focuses on building and optimizing the next generation of SFT/RL training frameworks for LLMs and AIGCs. Responsibilities include enhancing training efficiency (memory usage reduction, model parallelism tuning), ensuring framework ease-of-use, and collaborating with product teams. The ideal candidate possesses strong C/C++ skills, deep learning knowledge (PyTorch/TensorFlow), and experience with GPU high-performance computing. The team works on a large scale heterogeneous system and requires experience with distributed systems and parallel computing.
Must have:
  • 3+ years C/C++ experience
  • Deep learning fundamentals
  • PyTorch/TensorFlow knowledge
  • Build SFT/RL training framework
  • Optimize LLM/AIGC training efficiency
Good to have:
  • CUDA GPU optimization
  • Experience with FSDP, Deepspeed, Megatron
  • Strong LLM model knowledge

Job Details

Responsibilities
ByteDance will be prioritizing applicants who have a current right to work in Singapore, and do not require ByteDance sponsorship of a visa. About the team The ByteDance Large Model Team is committed to developing the most advanced AI large model technology in the industry, becoming a world-class research team, and contributing to technological and social development. The Large Model Team has a long-term vision and determination in the field of AI, with research directions covering NLP, CV, speech, and other areas. Relying on the abundant data and computing resources of the platform, the team has continued to invest in relevant fields and has launched its own general large model, providing multi-modal capabilities. The Machine Learning (ML) System sub-team combines system engineering and the art of machine learning to develop and maintain massively distributed ML training and inference system/services around the world, providing high-performance, highly reliable, scalable systems for LLM/AIGC/AGI. In our team, you'll have the opportunity to build the large scale heterogeneous system integrating with GPU/NPU/RDMA/Storage and keep it running steadily and reliably, enrich your expertise in coding, performance analysis and distributed system, and be involved in the decision-making process. You'll also be part of a global team with members from the United States, China and Singapore working collaboratively towards unified project direction. Responsibilities: - Responsible for building the future generation of SFT/RL training framework - Responsible for optimizing e2e LLM/AIGC training efficiency, including reducing memory usage, tuning model parallelism strategies, etc. - Responsible for making the training framework easy-to-adopt with good out-of-box performance - Collaborate with critical product teams to enable their LLM/AIGC products landing
Qualifications
Minimum Qualifications: - Bachelor's degree or above, major in computer/electronics/automation/software or related; - At least 3 years of working experiences in C/C++, proficient in algorithms and data structures, familiar with Python - Understand the basic principles of deep learning algorithms, be familiar with the basic architecture of neural networks and understand deep learning training frameworks such as PyTorch and TensorFlow - Have a strong sense of responsibility, good learning ability, communication ability and self-drive, good team spirit Preferred Qualifications: - Proficient in GPU high-performance computing optimization technology on CUDA, in-depth understanding of computer architecture, familiar with parallel computing optimization, memory access optimization, low-bit computing, etc. - Familiar with FSDP, Deepspeed, Megatron, etc. - Strong Knowledge of LLM models, experience in accelerating LLM model optimization
Job Information
About Us

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok, Lemon8, CapCut and Pico as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

Why Join ByteDance

Inspiring creativity is at the core of ByteDance's mission. Our innovative products are built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and enrich life - a mission we work towards every day.

As ByteDancers, we strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our Company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Similar Jobs

dbt Labs - Staff Software Engineer

dbt Labs

Ireland (Remote)
2 Weeks ago
Lilt - Gulf Arabic-based Medical Translators needed

Lilt

Abu Dhabi, Abu Dhabi, United Arab Emirates (Remote)
1 Month ago
Pattern - Senior Ecommerce Manager

Pattern

Lehi, Utah, United States (Hybrid)
10 Months ago
Resolver - Automation Test Engineer

Resolver

Hyderabad, Telangana, India (On-Site)
4 Years ago
Riot Games - 3D Character Art Lead - Unannounced R&D Product

Riot Games

Los Angeles, California, United States (On-Site)
3 Months ago
Apple - Machine Learning Engineer, Siri Attention & Invocation

Apple

Cupertino, California, United States (On-Site)
1 Month ago
Sailpoint - Manager, Engineering Operations - AI

Sailpoint

Austin, Texas, United States (On-Site)
4 Weeks ago
Thales - Innovation and Applied-AI Engineer

Thales

Singapore (Hybrid)
2 Weeks ago
Game freak - New Graduate Programmer (Entry Sheet/Research Achievements Selection)

Game freak

Chiyoda City, Tokyo, Japan (On-Site)
4 Weeks ago
Samsung Semiconductor - Staff Engineer, AI/ML Software Compiler

Samsung Semiconductor

San Jose, California, United States (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

EveryMatrix - Mid/Senior Data Analyst

EveryMatrix

Yerevan, Yerevan, Armenia (On-Site)
4 Months ago
Nice - Senior Specialist Program Manager

Nice

Pune, Maharashtra, India (Hybrid)
1 Month ago
Morning Star - Senior Software Engineer

Morning Star

Toronto, Ontario, Canada (Hybrid)
2 Weeks ago
Tencent - Payroll Associate

Tencent

Kuala Lumpur, Federal Territory Of Kuala Lumpur, Malaysia (On-Site)
7 Months ago
Wolters Kluwer - Production Specialist (Copy Editing)

Wolters Kluwer

Pune, Maharashtra, India (Hybrid)
1 Month ago
Pinterest - Senior Client Partner, Full Funnel

Pinterest

New York, United States (Hybrid)
2 Weeks ago
Capgemini - Transaction Monitoring

Capgemini

Noida, Uttar Pradesh, India (On-Site)
2 Months ago
GoTo Group - Logistics Strategic Insight Manager

GoTo Group

Jakarta, Indonesia (On-Site)
6 Months ago
Riot Games - Senior Manager, Content Production - Wild Rift Publishing

Riot Games

Shanghai, Shanghai, China (On-Site)
3 Months ago
Apple - CapEx Global Supply Manager (GSM) - Mechanical Enclosures

Apple

Cupertino, California, United States (On-Site)
1 Week ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

Illumina - Staff Software Engineer - Full Stack

Illumina

Singapore (On-Site)
1 Year ago
Moloco - Senior/Data Scientist II, Growth Analytics

Moloco

Singapore (On-Site)
4 Weeks ago
HoYoverse - Intellectual Property Counsel

HoYoverse

Singapore (On-Site)
8 Months ago
Cubic corporation - Coordinator Program Support SGP

Cubic corporation

Singapore (On-Site)
9 Months ago
Jane Street - Cybersecurity Detection and Response Analyst

Jane Street

Singapore (On-Site)
2 Months ago
bytedance - Software Engineer - Data Engineering (Video Arch)

bytedance

Singapore (On-Site)
9 Months ago
bytedance - Network Implementation Engineer, Physical Network Infrastructure

bytedance

Singapore (On-Site)
4 Months ago
Bushiroad - Events Executive

Bushiroad

Singapore, Singapore (On-Site)
3 Months ago
bytedance - Strategy Intern, BytePlus

bytedance

Singapore (On-Site)
3 Months ago
Marsh McLennan - Fiduciary Operations Lead - Asia Placement Hub

Marsh McLennan

Singapore (Hybrid)
1 Year ago

Get notifed when new similar jobs are uploaded

Research Development Jobs

Gusto - Tax Research Analyst, Symmetry

Gusto

Scottsdale, Arizona, United States (Remote)
1 Week ago
C3 IoT - Software Engineer - Generative AI

C3 IoT

Guadalajara, Jalisco, Mexico (On-Site)
3 Weeks ago
Single Store - AI & Automation Analyst

Single Store

Costa Rica (On-Site)
2 Weeks ago
Apple - Machine Learning Manager - Apple Ads

Apple

Cupertino, California, United States (On-Site)
1 Month ago
ARHS - Machine Learning Engineer (Dataiku)

ARHS

Luxembourg (On-Site)
1 Month ago
rivos - Deep Learning Libraries Engineer

rivos

United Kingdom (Hybrid)
1 Year ago
DevRev - Applied AI Engineer - Internship

DevRev

Buenos Aires, Buenos Aires, Argentina (On-Site)
2 Weeks ago
Agara labs - Lead / Staff ML Scientist - NLP

Agara labs

Bengaluru, Karnataka, India (Remote)
3 Months ago
Springer Group - AI Engineering Manager

Springer Group

Groningen, Groningen, Netherlands (On-Site)
2 Months ago
Reddit - Staff Machine Learning Engineer, Ads Ranking

Reddit

Ontario, Canada (Remote)
2 Weeks ago

Get notifed when new similar jobs are uploaded

About The Company

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

San Jose, California, United States (On-Site)

Pasig, Metro Manila, Philippines (On-Site)

Taguig, Metro Manila, Philippines (On-Site)

San Jose, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by bytedance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug