Research Scientist - Multimodal Language Models

6 Months ago • All levels • Research Development • $200,000 PA - $300,000 PA

Job Summary

Job Description

Luma is seeking experienced researchers to build multimodal AI, focusing on integrating vision and audio with language models. The role involves end-to-end work on cutting-edge multimodal language models, with a strong emphasis on audio and visual data. Contributions will significantly influence research projects and product roadmaps. Responsibilities include designing and implementing novel AI algorithms, building evaluation tools, developing large-scale training and inference methods, ensuring efficient model implementation, processing multimodal data, collaborating with teams, and implementing prototypes for multimodal generative AI.
Must have:
  • Expertise in Python & Pytorch
  • Experience with full AI development pipeline
  • Experience with large-scale text data
  • Hands-on experience with LLMs or VLM or ALM or generative video models
Good to have:
  • Experience with interleaved audio, video, image, and/or text data
Perks:
  • Competitive equity packages in the form of stock options
  • Comprehensive benefits plan

Job Details

Luma’s mission is to build multimodal AI to expand human imagination and capabilities. We believe that multimodality is critical for intelligence. To go beyond language models and build more aware, capable and useful systems, the next step function change will come from vision and audio. So, we are working on training and scaling up multimodal foundation models for systems that can see, hear and understand, show and explain, and eventually interact with our world to effect change.

We are looking for researchers with significant experience solving hard problems in multimodal language models. You will work end–to–end on cutting edge multimodal language models with strong emphasis on audio and visual data. Your contributions will be pivotal in shaping various research projects and product roadmaps.

Responsibilities

  • Design and implement novel AI algorithms and architectures for multimodal language models.

  • Build tools to evaluate and benchmark multimodal language models.

  • Develop large-scale AI training and inference methods.

  • Ensure efficient implementation of models & systems for data processing and training.

  • Build tools to analyze and process multimodal data.

  • Collaborate with research and engineering teams across Luma to transfer research to products and services.

  • Implement cutting-edge product prototypes based on multimodal generative AI.

Experience

  • Expertise in Python & Pytorch, including practical experience working with the full development pipeline from data processing & data loading to training, inference, and optimization.

  • Experience working with large-scale text data, or (bonus) interleaved data spanning audio, video, image, and/or text.

  • Hands-on experience in developing or benchmarking at least one of the following topics: LLMs, Vision Language Models, Audio Language Models, generative video models .

Compensation

  • The pay range for this position in California is $200,000 - $300,000yr; however, base pay offered may vary depending on job-related knowledge, skills, candidate location, and experience. We also offer competitive equity packages in the form of stock options and a comprehensive benefits plan. 

Your application is reviewed by real people.

Similar Jobs

Figma - Content Strategist, Product

Figma

United States (Hybrid)
3 Weeks ago
LTI Mindtree - Senior Engineer - Industrial IoT

LTI Mindtree

Monterrey, Nuevo Leon, Mexico (On-Site)
1 Month ago
Vertx Inc. - Senior Technical Product Manager

Vertx Inc.

United States (Remote)
1 Year ago
frozenbyte games - Programming

frozenbyte games

Helsinki, Uusimaa, Finland (On-Site)
1 Month ago
Ramp - Senior Design-Engineer | Design Systems Tooling

Ramp

United States (Remote)
3 Weeks ago
appier - Senior Machine Learning Scientist

appier

Taipei City, Taiwan (On-Site)
1 Month ago
Apple - Senior AI Application Engineer

Apple

Cupertino, California, United States (On-Site)
2 Months ago
Mistral AI - Applied AI Engineer

Mistral AI

Luxembourg (On-Site)
1 Month ago
Supabase - AI Tooling Engineer

Supabase

(Remote)
1 Month ago
Apple - Physical Design Engineer, Machine Learning

Apple

Sunnyvale, California, United States (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Marsh McLennan - Underwriting Assistant

Marsh McLennan

Ottawa, Ontario, Canada (Hybrid)
1 Month ago
Hitachi - Data Scientist

Hitachi

Pune, Maharashtra, India (Remote)
10 Months ago
Shield AI - Contracts Administrator, Commercial (R3190)

Shield AI

San Diego, California, United States (On-Site)
3 Weeks ago
HYCU - Software Engineer – Cloud Data Protection

HYCU

Bengaluru, Karnataka, India (Hybrid)
1 Year ago
bytedance - Senior Site Reliability Engineer, ML System

bytedance

Seattle, Washington, United States (On-Site)
9 Months ago
Lilith games - AIGC Designer

Lilith games

Shanghai, China (On-Site)
3 Weeks ago
Stord - Area Manager I

Stord

Blaine, Washington, United States (On-Site)
2 Years ago
Stibo Systems - Accounting Assistant

Stibo Systems

Aarhus, Denmark (On-Site)
2 Months ago
Iron Mountain - Global FP&A Senior Specialist

Iron Mountain

Bogotá, Bogota, Colombia (Remote)
3 Weeks ago
Sprinkler - Platinum Support Account Manager

Sprinkler

Seoul, South Korea (On-Site)
4 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Palo Alto, California, United States

Apple - Software Engineer - Build Engineering

Apple

Sunnyvale, California, United States (On-Site)
2 Months ago
Apple - Senior Controls Engineer

Apple

Cupertino, California, United States (On-Site)
2 Months ago
TransUnion - Advisor, Partner Enablement Solution Architect

TransUnion

New York, United States (Hybrid)
1 Year ago
WebMD - Medical Science Director, Oncology

WebMD

United States (On-Site)
5 Months ago
Nintendo - Supervisor. Packaging & Distribution

Nintendo

North Bend, Washington, United States (On-Site)
8 Months ago
Open Systems Technologies - Oracle Business Analyst

Open Systems Technologies

St. Louis, Missouri, United States (On-Site)
1 Month ago
Perplexity - AI Software Engineer - Data Platform

Perplexity

New York, New York, United States (On-Site)
8 Months ago
Open Systems Technologies - Exercise Planner Subject Matter Expert

Open Systems Technologies

Tacoma, Washington, United States (On-Site)
3 Weeks ago
Riot Games - Principal Game Designer - Unpublished R&D Product

Riot Games

Los Angeles, California, United States (On-Site)
4 Months ago
Sonar Source - Staff Cloud Engineer (AWS)

Sonar Source

Austin, Texas, United States (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

Research Development Jobs

Haven Studios  Inc  - Senior User Experience Researcher

Haven Studios Inc

Montreal, Quebec, Canada (On-Site)
1 Month ago
Autodesk - Principal Experience Design Researcher

Autodesk

British Columbia, Canada (Remote)
2 Months ago
bytedance - Research Scientist, Reinforcement Learning

bytedance

Seattle, Washington, United States (On-Site)
9 Months ago
Krafton - [Global Strategy & BD Div.] Strategy Manager(AI Ethics) (4년 ~ 7년)

Krafton

Seoul, South Korea (On-Site)
8 Months ago
Azerion - Marketing Intern – Creative Video & AI Enthusiast

Azerion

Schiphol-Rijk, North Holland, Netherlands (Hybrid)
1 Month ago
Microsoft - Senior Applied Scientist

Microsoft

Redmond, Washington, United States (On-Site)
4 Months ago
Telnyx - Software Engineer, Python (AI)

Telnyx

Dublin, County Dublin, Ireland (On-Site)
2 Months ago
bytedance - Student Researcher (Doubao (Seed) - Foundation Model AI Platform) - 2025 Start (PhD)

bytedance

Seattle, Washington, United States (On-Site)
9 Months ago
London stock Exchange - Senior Manager Developer AI Solutions

London stock Exchange

London, England, United Kingdom (On-Site)
2 Months ago
Capgemini - AI & Analytics

Capgemini

Noida, Uttar Pradesh, India (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

View All Jobs

Get notified when new jobs are added by Luma

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug