Research Scientist, Multimodal Interaction & World Model

1 Month ago • All levels • Artificial Intelligence • Research & Development

Job Summary

Job Description

The Research Scientist, Multimodal Interaction & World Model will contribute to cutting-edge research on multimodal intelligence and virtual reality world interaction within ByteDance's Doubao team. Responsibilities include exploring multimodal understanding, generation, and machine learning techniques; optimizing large-scale multimodal models; developing multimodal agents for virtual worlds; and building world models using pre-training and simulation. The ideal candidate will possess in-depth research experience in computer vision, AIGC, or machine learning, and strong analytical and problem-solving skills.
Must have:
  • Multimodal understanding & generation expertise
  • Large-scale model optimization skills
  • Strong analytical & problem-solving abilities
  • Experience with AIGC, ML, or CV
  • Excellent communication & collaboration
Good to have:
  • Publications in top conferences (CVPR, NeurIPS, etc.)
  • Proficiency in C/C++ or Python
  • Experience leading influential projects in relevant fields
  • ACM/ICPC or similar competition experience

Job Details

Responsibilities
Established in 2023, the ByteDance Doubao (Seed) Team is dedicated to building industry-leading AI foundation models. We aim to do world-leading research and foster both technological and social progress. With a long-term vision and a strong commitment to the AI field, the Team conducts research in a range of areas including natural language processing (NLP), computer vision (CV), and speech recognition and generation. It has labs and researcher roles in China, Singapore, and the US. Leveraging substantial data and computing resources and through continued investment in these domains, our team has built a proprietary general-purpose model with multimodal capabilities. In the market, Doubao models power over 50 ByteDance apps and business lines, including Doubao, Coze, and Dreamina, and was launched to external enterprise clients through Volcano Engine. The Doubao app is the most used AIGC app in China. Why Join Us Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible. Together, we inspire creativity and enrich life - a mission we aim towards achieving every day. To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always. At ByteDance, we create together and grow together. That's how we drive impact - for ourselves, our company, and the users we serve. Join us. About the team Welcome to the Multimodal Interaction & World Model team. Our mission is to solve the challenge of multimodal intelligence、virtual reality world interaction in AI. We conduct cutting-edge research on areas such as Foundations and applications of multimodal understanding models, Multimodal agent and inference, Unified models for generation and understanding, World Model. Our team is comprising experienced research scientists and engineers who are dedicated to developing models that boast human-level multimodal understanding and interaction capabilities. The team also aspires to advance the exploration and development of multimodal assistant products. We foster a feedback-driven environment to continuously enhance our foundation technologies. Come join us in shaping the future of AI and transforming the product experience for users worldwide. Responsibilities - Explore and research multi-modal understanding, generative, machine learning, reinforcement learning, AIGC, computer vision, artificial intelligence and other cutting-edge technologies. - Explore the basic model of large-scale/ultra-large-scale multi-modal understanding and generation interweaving, and carry out extreme system optimization; Data construction, instruction fine-tuning, preference alignment, model optimization; Improve the ability of data synthesis, scalable oversight, model reasoning and planning, build a comprehensive, objective and accurate evaluation system, and explore and improve the ability of large models. - Explore and break through the advanced capabilities of multi-modal models and world models, including but not limited to multi-modal RAG, visual COT and Agent, and build a universal multi-modal Agent for GUI/ games and other virtual worlds. - Use pre-training, simulation and other technologies to model various environments in the virtual/real world, provide the basic ability of multi-modal interactive exploration, promote application landing, and develop new technologies and new products with artificial intelligence technology as the core.
Qualifications
Minimum Qualifications: - Bachelor degree or above, computer, electronics, mathematics and other related majors. - Have in-depth research in one or more fields such as computer vision, multimodal, AIGC, machine learning, rendering generation, etc.. - Excellent analytical and problem solving skills; Ability to solve large model training and application problems; Ability to explore solutions independently. - With good communication and collaboration skills, proactive work, able to harmoniously cooperate with the team to explore new technologies and promote technological progress. Preferred Qualifications: - With excellent basic algorithms, solid foundation of machine learning, familiar with CV, AIGC, NLP, RL, ML and other fields of technology, CVPR, ECCV, ICCV, NeurIPS, ICLR, SIGGRAPH or SIGGRAPH Asia and other top conferences/journals published papers are preferred. - With excellent coding ability, proficient in C/C++ or Python programming language, ACM/ICPC, NOI/IOl, Top Coder, Kaggle and other competition winners preferred. - In the fields of multimodal, large model, basic model, world model, RL, rendering generation, leading projects with too much influence is preferred. ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Similar Jobs

Google - Imaging System Architect

Google

Mountain View, California, United States (On-Site)
1 Month ago
IO Interactive - Multiplayer Programmer

IO Interactive

Brighton And Hove, England, United Kingdom (Hybrid)
2 Months ago
NVIDIA - System Software Architect, Programmable Vision Accelerator

NVIDIA

Bengaluru, Karnataka, India (On-Site)
4 Months ago
Google - Software Engineer, gReach Program for People with Disabilities

Google

Shanghai, Shanghai, China (On-Site)
1 Month ago
SideFX Software - Senior Compositing Software Developer/Technical Director

SideFX Software

Ontario, Canada (Hybrid)
2 Months ago
Google - Software Engineer II, AI/ML, Technical Infrastructure

Google

Taipei City, Taiwan (On-Site)
1 Month ago
C3 AI - Solution Engineer

C3 AI

Bengaluru, Karnataka, India (On-Site)
7 Months ago
Genies - Research Scientist Intern - LLM/Vision/Speech

Genies

San Mateo, California, United States (Hybrid)
2 Months ago
Meta - Research Scientist Intern, Smart Glasses in Wearables AI (PhD)

Meta

Menlo Park, California, United States (On-Site)
6 Months ago
Google - Accelerator Architect and Performance Engineer, Generative AI

Google

San Diego, California, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Qualcomm - Senior ML Compiler Engineer

Qualcomm

Hyderabad, Telangana, India (On-Site)
6 Days ago
Intel  - Software Application Development Engineer

Intel

Bengaluru, Karnataka, India (On-Site)
1 Year ago
Rackspace Technology - Sr Big Data Engineer - Oozie and Pig (GCP)

Rackspace Technology

United States (Remote)
1 Month ago
The Walt Disney Company - Sr Machine Learning Engineer

The Walt Disney Company

San Francisco, California, United States (On-Site)
6 Months ago
Mindbotix - ASO Executive

Mindbotix

Lahore, Punjab, Pakistan (On-Site)
2 Weeks ago
Inworld AI - Senior Software Engineer (C++ Focus)

Inworld AI

Mountain View, California, United States (Hybrid)
2 Months ago
NVIDIA - System Software Architect, Programmable Vision Accelerator

NVIDIA

Hyderabad, Telangana, India (On-Site)
3 Months ago
RoofStack - Software Developer

RoofStack

Istanbul, İstanbul, Türkiye (On-Site)
2 Months ago
NVIDIA - Senior Software Engineer - Automated Parallel Programming

NVIDIA

North Carolina, United States (Remote)
3 Months ago
CynLr - Software Engineer - GPU performance

CynLr

Bengaluru, Karnataka, India (On-Site)
7 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

hogarth - Studio Lead (Content)

hogarth

Singapore (On-Site)
2 Weeks ago
bytedance - Machine Learning Engineer Intern - Global Payment - 2025 Start

bytedance

Singapore (On-Site)
3 Months ago
bytedance - AML Compliance Advisory - Global Payment

bytedance

Singapore (On-Site)
1 Month ago
bytedance - Backend Engineer, Machine Learning Systems - Singapore

bytedance

Singapore (On-Site)
7 Months ago
OKX - (Senior/Principal) Product Manager, Blockchain Explorer

OKX

Singapore, Singapore (On-Site)
7 Months ago
Workato - Commercial Sales Manager

Workato

Singapore (On-Site)
1 Week ago
Veeam Software - Senior Director Professional Services

Veeam Software

Singapore, Singapore (On-Site)
1 Month ago
Adyen - Senior Regulatory Counsel

Adyen

Singapore (On-Site)
2 Weeks ago
bytedance - Ediscovery Expert - Information System

bytedance

Singapore (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Artificial Intelligence Jobs

Meta - Research Scientist Intern, Smart Glasses in Wearables AI (PhD)

Meta

Redmond, Washington, United States (On-Site)
6 Months ago
Google - ML System Engineer, AICore, Platforms and Devices

Google

Taipei City, Taiwan (On-Site)
1 Month ago
Token Metrics - Crypto Data Scientist / Machine Learning Engineer  (Remote)

Token Metrics

Tiranë, Tirana County, Albania (Remote)
7 Months ago
Meta - Software Engineer, Machine Learning

Meta

New York, New York, United States (On-Site)
6 Months ago
Canva - Senior Machine Learning Engineer - Photo AI

Canva

Vienna, Vienna, Austria (Remote)
4 Months ago
FTF Studios - FTF Mid-Level Programmer

FTF Studios

(Remote)
1 Year ago
Google - Staff Software Engineer, ML Performance, GPUs

Google

Sunnyvale, California, United States (On-Site)
1 Month ago
bytedance - Research Engineer (Foundation Model) - Machine Learning Systems

bytedance

Singapore (On-Site)
7 Months ago
bytedance - Researcher Graduate (Applied Machine Learning - Enterprise)

bytedance

San Jose, California, United States (On-Site)
2 Months ago
bytedance - Student Researcher (Doubao (Seed) - Foundation Model - Video Generation) - 2025 Start (PhD)

bytedance

San Jose, California, United States (On-Site)
7 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

San Jose, California, United States (On-Site)

Tokyo, Japan (On-Site)

Taguig, Metro Manila, Philippines (On-Site)

San Jose, California, United States (On-Site)

Ho Chi Minh City, Vietnam (On-Site)

San Diego, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by bytedance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug