Software Engineer, LLM Storage System Intern

1 Month ago • All levels

Job Summary

Job Description

This Software Engineer internship focuses on designing and developing components for large model storage systems. Responsibilities include optimizing KVCache, improving data IO, and managing a multi-level storage system using various media like HBM and remote storage. The role involves optimizing KV Cache hit rate, designing data access interfaces, and ensuring system stability in Kubernetes. The engineer will also handle system setup, disaster recovery, and data placement across clusters for multi-datacenter scenarios. This internship offers an opportunity to contribute to cutting-edge AI technology and work with a world-class research team.
Must have:
  • Design and develop components for machine learning systems.
  • Optimize KVCache for large model inference.
  • Implement multi-level storage systems.
  • Design efficient and user-friendly data access interfaces.
  • Manage and maintain storage systems in Kubernetes.
  • Handle system setup and disaster recovery in multi-cloud scenarios.

Job Details

About the Team ByteDance Doubao Large Model Team was established in 2023, dedicated to developing the most advanced AI large model technology in the industry, becoming a world-class research team, and contributing to the development of technology and society. The Doubao large model team has a long-term vision and determination in the field of AI, with research directions covering NLP, CV, speech, etc. They have laboratories and research positions in China, Singapore, the US and other places. The team relies on sufficient data, computing and other resources on the platform, continuously invests in related fields, and has launched self-developed general large models, providing MultiModal Machine Learning capabilities. Downstream support includes 50 + businesses such as Doubao, Coze, Dreamina, and is open to enterprise customers through Volcengine. Currently, Doubao APP has become the largest AIGC application in the Chinese market. 1. Assume responsibility for the design and development of components associated with the storage of machine learning systems, catering to diverse business scenarios of large model inference (LLM/S2S/VLM/multimodal, etc.). This includes model distribution and loading, KVCache optimization, enhancement of data IO performance, and improvement of TTFT and TBT in LLM serving 2. Take charge of designing and implementing a multi-level storage system for large model inference. Comprehensively utilize various media, including HBM, host memory, distributed disk, and remote large-capacity storage systems (HDFS/object storage) for data storage and migration management. Realize an integrated hierarchical system of "near-compute cache + remote large-capacity storage". 3. Be accountable for optimizing the hit rate of large model KV Cache. Formulate customized optimization strategies from multiple system dimensions, such as the inference framework, traffic scheduling, and multi-level cache. Optimize data IO performance by fully leveraging NVLink, RDMA high-speed network, and GPU Direct technologies on the near-compute side to achieve efficient data transmission. Optimize the storage strategy of data replicas to achieve a reasonable distribution of load traffic and stored data. 4. Undertake the design and implementation of efficient and user-friendly data access interfaces. Realize seamless docking with the inference framework, and manage the lifecycle of KV Cache. 5. Be responsible for the access, management, operation and maintenance, and monitoring of the multi-level storage system in the Kubernetes scenario to ensure stability. 6. Assume the task of system setup and disaster recovery in multi-datacenter, multi-region, and multi-cloud scenarios, and optimize data placement across clusters.

Similar Jobs

Crowd Strick - Software Development Engineer in Test III, Detections Platform

Crowd Strick

United Kingdom (Remote)
2 Weeks ago
Devoteam - Multi-Cloud Engineer (m/w/d)

Devoteam

Frankfurt Am Main, Hessen, Germany (Remote)
7 Months ago
The Walt Disney Company - Lead Software Engineer - Front End

The Walt Disney Company

Glendale, California, United States (On-Site)
1 Month ago
Gravity CO  - Integrated Platform Developer

Gravity CO

Seoul, South Korea (On-Site)
1 Month ago
Ion - Cloud Engineer/Architect (DevOps)

Ion

Italy (On-Site)
8 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Alpha Sense - Lead AI Platform Engineer

Alpha Sense

New York, New York, United States (On-Site)
6 Months ago
Scale AI - AI Infrastructure Engineer, ML Data Platform

Scale AI

San Francisco, California, United States (On-Site)
1 Month ago
Demonware - Expert Engineer, Accounts and Authentication

Demonware

Dublin, County Dublin, Ireland (On-Site)
1 Month ago
Flexera Software - Partner Business Manager (FinOps)

Flexera Software

United States (On-Site)
3 Days ago
Razer - AI Software Engineer

Razer

Singapore (On-Site)
1 Week ago
HCL Tech - Golang Senior Technical Lead

HCL Tech

California, United States (On-Site)
4 Days ago
skillz - Lead Application Security Engineer

skillz

Las Vegas, Nevada, United States (On-Site)
2 Months ago
JMA - Advanced Engineer - Baseband Unit QA

JMA

Syracuse, New York, United States (On-Site)
3 Months ago
Tide - Principal Cloud Engineer

Tide

Hyderabad, Telangana, India (Remote)
2 Weeks ago
Better ME - Backend Engineer (Hardware)

Better ME

Ukraine (Remote)
4 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

OKX - Senior Product Manager, Trading Risk Management

OKX

Singapore, Singapore (On-Site)
7 Months ago
Zscaler - Senior Sales Engineer

Zscaler

Singapore (Hybrid)
2 Weeks ago
bytedance - Legal Counsel, Global AI Products

bytedance

Singapore (On-Site)
1 Month ago
HoYoverse - Quest Designer

HoYoverse

Singapore (On-Site)
2 Months ago
HoYoverse - Data Analyst - Honkai: Star Rail - Fresh Grad

HoYoverse

Singapore, Singapore (On-Site)
1 Month ago
bytedance - Principal Algorithm Engineer, Trust and Safety

bytedance

Singapore (On-Site)
7 Months ago
HoYoverse - Intellectual Property Counsel

HoYoverse

Singapore, Singapore (On-Site)
1 Month ago
Alation - Senior Account Executive

Alation

Singapore (Remote)
4 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

About The Company

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

Seattle, Washington, United States (On-Site)

San Jose, California, United States (On-Site)

Tokyo, Japan (On-Site)

San Jose, California, United States (On-Site)

San Jose, California, United States (On-Site)

San Jose, California, United States (On-Site)

San Jose, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by bytedance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug