Software Engineer, LLM Storage System Intern

2 Hours ago • All levels

Job Summary

Job Description

This Software Engineer internship focuses on designing and developing components for large model storage systems. Responsibilities include optimizing KVCache, improving data IO, and managing a multi-level storage system using various media like HBM and remote storage. The role involves optimizing KV Cache hit rate, designing data access interfaces, and ensuring system stability in Kubernetes. The engineer will also handle system setup, disaster recovery, and data placement across clusters for multi-datacenter scenarios. This internship offers an opportunity to contribute to cutting-edge AI technology and work with a world-class research team.
Must have:
  • Design and develop components for machine learning systems.
  • Optimize KVCache for large model inference.
  • Implement multi-level storage systems.
  • Design efficient and user-friendly data access interfaces.
  • Manage and maintain storage systems in Kubernetes.
  • Handle system setup and disaster recovery in multi-cloud scenarios.

Job Details

About the Team ByteDance Doubao Large Model Team was established in 2023, dedicated to developing the most advanced AI large model technology in the industry, becoming a world-class research team, and contributing to the development of technology and society. The Doubao large model team has a long-term vision and determination in the field of AI, with research directions covering NLP, CV, speech, etc. They have laboratories and research positions in China, Singapore, the US and other places. The team relies on sufficient data, computing and other resources on the platform, continuously invests in related fields, and has launched self-developed general large models, providing MultiModal Machine Learning capabilities. Downstream support includes 50 + businesses such as Doubao, Coze, Dreamina, and is open to enterprise customers through Volcengine. Currently, Doubao APP has become the largest AIGC application in the Chinese market. 1. Assume responsibility for the design and development of components associated with the storage of machine learning systems, catering to diverse business scenarios of large model inference (LLM/S2S/VLM/multimodal, etc.). This includes model distribution and loading, KVCache optimization, enhancement of data IO performance, and improvement of TTFT and TBT in LLM serving 2. Take charge of designing and implementing a multi-level storage system for large model inference. Comprehensively utilize various media, including HBM, host memory, distributed disk, and remote large-capacity storage systems (HDFS/object storage) for data storage and migration management. Realize an integrated hierarchical system of "near-compute cache + remote large-capacity storage". 3. Be accountable for optimizing the hit rate of large model KV Cache. Formulate customized optimization strategies from multiple system dimensions, such as the inference framework, traffic scheduling, and multi-level cache. Optimize data IO performance by fully leveraging NVLink, RDMA high-speed network, and GPU Direct technologies on the near-compute side to achieve efficient data transmission. Optimize the storage strategy of data replicas to achieve a reasonable distribution of load traffic and stored data. 4. Undertake the design and implementation of efficient and user-friendly data access interfaces. Realize seamless docking with the inference framework, and manage the lifecycle of KV Cache. 5. Be responsible for the access, management, operation and maintenance, and monitoring of the multi-level storage system in the Kubernetes scenario to ensure stability. 6. Assume the task of system setup and disaster recovery in multi-datacenter, multi-region, and multi-cloud scenarios, and optimize data placement across clusters.

Similar Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Skill Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Jobs in Singapore

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

About The Company

Where imagination meets innovation, delivering limitless gaming experiences.

San Jose, California, United States (On-Site)

Singapore (On-Site)

Taguig, Metro Manila, Philippines (On-Site)

Seattle, Washington, United States (On-Site)

Taguig, Metro Manila, Philippines (On-Site)

Seattle, Washington, United States (On-Site)

London, England, United Kingdom (On-Site)

View All Jobs

Get notified when new jobs are added by ByteDance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug