Research Engineer - Multimodal Language Models

5 Months ago • All levels • Research Development • $200,000 PA - $300,000 PA

Job Summary

Job Description

Luma is building multimodal AI to expand human imagination and capabilities. This role involves training and scaling multimodal foundation models that can see, understand, show, explain, and interact with the world. The engineer will work end-to-end on cutting-edge multimodal language models with a focus on audio and visual data, significantly contributing to research projects and product roadmaps. Responsibilities include designing large-scale annotation efforts, building evaluation tools for multimodal models, developing AI training and inference methods, and ensuring efficient implementation of models and systems.
Must have:
  • Expertise in Python and PyTorch
  • Experience with full development pipeline
  • Experience processing large-scale text data
  • Experience with LLMs, Vision Language Models, or Audio Language Models
Good to have:
  • Experience with interleaved data (audio, video, image, text)
  • Experience developing or benchmarking generative video models
  • Experience designing annotation tools
  • Experience with synthetic data
Perks:
  • Competitive equity packages (stock options)
  • Comprehensive benefits plan

Job Details

Luma’s mission is to build multimodal AI to expand human imagination and capabilities.

We believe that multimodality is critical for intelligence. To go beyond language models and build more aware, capable and useful systems, the next step function change will come from vision. So, we are working on training and scaling up multimodal foundation models for systems that can see and understand, show and explain, and eventually interact with our world to effect change.


We are looking for engineers with significant experience solving hard problems in PyTorch, multimodal data, and distributed systems. You will work as a team to end-to-end build cutting edge multimodal language models with strong emphasis on audio and visual data. Your contributions will be pivotal in shaping various research projects and product roadmaps.

Responsibilities

  • Design and develop large-scale annotation efforts for model post-training.

  • Build tools to evaluate and benchmark multimodal language models.

  • Develop large-scale AI training and inference methods.

  • Ensure efficient implementation of models & systems for data processing and training.

  • Build tools to visualize, evaluate and filter datasets.

  • Collaborate with research and engineering teams across Luma to transfer research to products and services.

  • Implement cutting-edge product prototypes based on multimodal generative AI.

Experience

  • Expertise in Python & Pytorch, including practical experience working with the full development pipeline from data processing, preparation & data loading to training and inference.

  • Experience processing large-scale text data, or (bonus) interleaved data spanning audio, video, image, and/or text.

  • Hands-on experience in developing or benchmarking at least one of the following topics: LLMs, Vision Language Models, Audio Language Models, generative video models.


  • Good to have

    • Experience in design and development of annotation tools

    • Experience in synthetic data

Compensation

  • The pay range for this position in California is $200,000 - $300,000/yr; however, base pay offered may vary depending on job-related knowledge, skills, candidate location, and experience. We also offer competitive equity packages in the form of stock options and a comprehensive benefits plan. 

Your application is reviewed by real people.

Similar Jobs

Motorola solutions - Site Reliability Engineer

Motorola solutions

Gatineau, Quebec, Canada (On-Site)
1 Month ago
Aptive - Test Engineer

Aptive

Gdańsk, Pomeranian Voivodeship, Poland (On-Site)
3 Months ago
Cred - Business Development and Strategic Partnerships - CRED Pay

Cred

Bengaluru, Karnataka, India (On-Site)
3 Weeks ago
Haleon - Sales & Use Tax Manager

Haleon

Warren, New Jersey, United States (Hybrid)
1 Month ago
Unity - Graphic Designer - Temporary

Unity

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
1 Month ago
C3 IoT - Senior Director, Clinical AI Solutions

C3 IoT

Redwood City, California, United States (On-Site)
1 Month ago
NXP - Internship - AI/ML Engineer

NXP

Nijmegen, Gelderland, Netherlands (On-Site)
1 Year ago
Decagon - Machine Learning Research Engineer

Decagon

San Francisco, California, United States (On-Site)
1 Month ago
bytedance - Research Scientist, Applied GAI-Vision

bytedance

San Jose, California, United States (On-Site)
9 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Electronic Arts - Consumer Insights Senior Manager

Electronic Arts

Redwood City, California, United States (Hybrid)
1 Month ago
Activision - Principal Online Programmer

Activision

Santa Monica, California, United States (On-Site)
2 Months ago
Infosys - Java Full Stack and Angular Developer [M/F/D] - Toulouse Technology Analyst

Infosys

Toulouse, Occitanie, France (Hybrid)
3 Weeks ago
Carbon Health - Primary Care Physician

Carbon Health

Lawrence Township, New Jersey, United States (On-Site)
1 Year ago
PayPal - Sr Staff Software Engineer - Android

PayPal

San Jose, California, United States (Hybrid)
2 Months ago
Shield AI - Senior Propulsion Design Engineer

Shield AI

Dallas, Texas, United States (On-Site)
3 Weeks ago
NVIDIA - Solution Architect - AI and ML

NVIDIA

(Remote)
7 Months ago
Marvell - Senior University Recruiter

Marvell

Bengaluru, Karnataka, India (On-Site)
1 Year ago
DraftKings - Senior Software Engineer, iOS

DraftKings

Canada (Remote)
1 Year ago
Ness - GIS Data Analyst L2 Support

Ness

Prague, Prague, Czechia (Hybrid)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Palo Alto, California, United States

PayPal - Sr. Staff Software Engineer - Mini App

PayPal

San Jose, California, United States (Hybrid)
2 Months ago
Sony pictures animation  - Visual Development Artist

Sony pictures animation

Los Angeles, California, United States (On-Site)
3 Months ago
Take-Two Interactive - Senior Data Scientist - AI

Take-Two Interactive

New York, New York, United States (Remote)
1 Month ago
Reddit - Principal Machine Learning Engineer, Ads Measurement

Reddit

United States (Remote)
3 Months ago
Discord - Staff Software Engineer - Desktop Platform

Discord

San Francisco, California, United States (On-Site)
2 Months ago
Apple - Manufacturing Design Engineer (MDE) - iPhone

Apple

Cupertino, California, United States (On-Site)
1 Month ago
Progress - Senior Inside Account Manager

Progress

Burlington, Massachusetts, United States (Hybrid)
1 Month ago
Sailpoint - Senior Director, Regional Field Marketing - Americas Region

Sailpoint

United States (On-Site)
1 Month ago
Figma - Software Engineer, Data Infrastructure

Figma

San Francisco, California, United States (Remote)
3 Weeks ago
Roblox - Senior Director of Engineering, User Connection

Roblox

San Mateo, California, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Research Development Jobs

Valeo - R&D Department Internship

Valeo

Skawina, Lesser Poland Voivodeship, Poland (On-Site)
3 Weeks ago
Riot Games - Senior Researcher, WR

Riot Games

Shanghai, China (On-Site)
3 Months ago
flying wild hog - Lead User Researcher

flying wild hog

Poland (Remote)
5 Months ago
bytedance - Research Scientist, Vision Foundation Model

bytedance

San Jose, California, United States (On-Site)
9 Months ago
Playtika - Application Security Researcher

Playtika

Israel (On-Site)
9 Months ago
Silicon Labs - Senior Engineer - AI/ML

Silicon Labs

Hyderabad, Telangana, India (On-Site)
3 Weeks ago
Single Store - AI & Automation Analyst

Single Store

Bengaluru, Karnataka, India (On-Site)
1 Month ago
dun bradstreet - Market Research Intern

dun bradstreet

Navi Mumbai, Maharashtra, India (On-Site)
5 Months ago
Match Group - Machine Learning Engineer (Contents Understanding)

Match Group

Seoul, South Korea (Hybrid)
1 Month ago
Lilt - Talent Recruitment Manager - AI Data Solutions

Lilt

Buenos Aires, Buenos Aires, Argentina (Remote)
1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

Palo Alto, California, United States (Hybrid)

View All Jobs

Get notified when new jobs are added by Luma

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug