Member of Technical Staff, Large Generative Models

1 Month ago • All levels • Software Development & Engineering • $175,000 PA - $275,000 PA

Job Summary

Job Description

Captions is seeking an exceptional Research Engineer (Member of Technical Staff) to advance the state-of-the-art in large-scale multimodal video diffusion models. The role involves conducting novel research on generative modeling architectures, developing new training techniques, and scaling models to billions of parameters. You will work at the cutting edge of multimodal generation, building systems for natural, controllable video creation, with a focus on audio-video generation, realistic human behavior, and natural storytelling. Responsibilities include designing and implementing novel architectures, developing multimodal fusion and temporal modeling approaches, researching temporal video editing, validating scaling laws, creating new loss functions, driving experimentation, and validating research through product deployment. The role also involves training and optimizing models at massive scale, developing distributed training approaches, designing model surgery techniques, creating memory optimization methods, researching training stability, and conducting systematic empirical studies.
Must have:
  • Master's or PhD in Computer Science, Machine Learning, or related field
  • Track record of research contributions at top ML conferences
  • Deep expertise in generative modeling approaches (diffusion, autoregressive, VAEs)
  • Strong proficiency in modern deep learning tooling (PyTorch, CUDA, Triton, FSDP)
  • Experience training diffusion models with 10B+ parameters
  • Ability to write clean, modular research code that scales
  • Strong software engineering practices including testing and code review
Good to have:
  • Experience with very large language models (200B+ parameters)
  • Deep understanding of attention, transformers, and modern multimodal architectures
  • Expertise in distributed training systems and model parallelism
  • Proven ability to implement and improve complex model architectures
  • Track record of systematic empirical research and rigorous evaluation
  • Experience with rapid prototyping and experimental design
  • Strong analytical skills for debugging model behavior and training dynamics
  • Facility with profiling and optimization tools
  • Track record of bringing research ideas to production
  • Experience maintaining high code quality in a research environment
Perks:
  • Comprehensive medical, dental, and vision plans
  • 401K with employer match
  • Commuter Benefits
  • Catered lunch multiple days per week
  • Dinner stipend every night if working late
  • Doordash DashPass subscription
  • Health & Wellness Perks (Talkspace, Kindbody, One Medical subscription, HealthAdvocate, Teladoc)
  • Multiple team offsites per year with team events every month
  • Generous PTO policy

Job Details

Captions is the leading AI video company—our mission is to empower anyone, anywhere to tell their stories through video. Over 10 million creators and businesses have used Captions to simplify video creation with truly novel and groundbreaking AI capabilities.

We are a rapidly growing team of ambitious, experienced, and devoted engineers, researchers, designers, marketers, and operators based in NYC. As an early member of our team, you’ll have an opportunity to have an outsized impact on our products and our company's culture.

Our Technology

Mirage Announcement our proprietary omni-modal foundation model

Seeing Voices (technical paper) generating A-roll video from audio with Mirage

Mirage Studio for generating expressive videos at scale

"Captions: For Talking Videos” available in the iOS app store

Press Coverage

Lenny’s Podcast: Interview with Gaurav Misra (CEO)

Latest Fundraise: Series C Announcement

The Information: 50 Most Promising Startups

Fast Company: Next Big Things in Tech

Business Insider: 34 most promising AI startups

TIME: The Best Inventions of 2024

Our Investors

We’re very fortunate to have some the best investors and entrepreneurs backing us, including Index Ventures, Kleiner Perkins, Sequoia Capital, Andreessen Horowitz, Uncommon Projects, Kevin Systrom, Mike Krieger, Lenny Rachitsky, Antoine Martin, Julie Zhuo, Ben Rubin, Jaren Glover, SVAngel, 20VC, Ludlow Ventures, Chapter One, and more.

** Please note that all of our roles will require you to be in-person at our NYC HQ (located in Union Square)

We do not work with third-party recruiting agencies, please do not contact us**

About the role:

Captions is seeking an exceptional Research Engineer (MOTS) to advance the state-of-the-art in large-scale multimodal video diffusion models. You'll conduct novel research on generative modeling architectures, develop new training techniques, and scale models to billions of parameters. As a key member of our ML Research team, you'll work at the cutting edge of multimodal generation while building systems that enable natural, controllable video creation. We're already training large-scale models with demonstrated product impact, and we're excited to continue expanding the scope and capabilities of our research.

We're especially excited about pushing the boundaries of audio-video generation, with a focus on realistic and charismatic human behavior that enables natural storytelling and creative iteration. Our models power creative tools used by millions of creators, and we're tackling fundamental challenges in how to generate compelling human motion, expression, and speech. 

Key Responsibilities:

Research & Architecture Development:

  • Design and implement novel architectures for large-scale video and multimodal diffusion models

  • Develop new approaches to multimodal fusion, temporal modeling, and video control

  • Research temporal video editing techniques and controllable generation

  • Research and validate scaling laws for video generation models

  • Create new loss functions and training objectives for improved generation quality

  • Drive rapid experimentation with model architectures and training strategies

  • Validate research directly through product deployment and user feedback

Model Training & Optimization:

  • Train and optimize models at massive scale (10s-100s of billions of parameters)

  • Develop sophisticated distributed training approaches using FSDP, DeepSpeed, Megatron-LM

  • Design and implement model surgery techniques (pruning, distillation, quantization)

  • Create new approaches to memory optimization and training efficiency

  • Research techniques for improving training stability at scale

  • Conduct systematic empirical studies of architecture and optimization choices

Technical Innovation:

  • Advance state-of-the-art in video model architecture design and optimization 

  • Develop new approaches to temporal modeling for video generation

  • Create novel solutions for multimodal learning and cross-modal alignment

  • Research and implement new optimization techniques for generative modeling and sampling

  • Design and validate new evaluation metrics for generation quality

  • Systematically analyze and improve model behavior across different regimes

Requirements:

Research Experience:

  • Master's or PhD in Computer Science, Machine Learning, or related field

  • Track record of research contributions at top ML conferences (NeurIPS, ICML, ICLR)

  • Demonstrated experience implementing and improving upon state-of-the-art architectures

  • Deep expertise in generative modeling approaches (diffusion, autoregressive, VAEs, etc.)

  • Strong background in optimization techniques and loss function design

  • Experience with empirical scaling studies and systematic architecture research

Technical Expertise:

  • Strong proficiency in modern deep learning tooling (PyTorch, CUDA, Triton, FSDP, etc.)

  • Experience training diffusion models with 10B+ parameters

  • Experience with very large language models (200B+ parameters) is a plus

  • Deep understanding of attention, transformers, and modern multimodal architectures

  • Expertise in distributed training systems and model parallelism

  • Proven ability to implement and improve complex model architectures

  • Track record of systematic empirical research and rigorous evaluation

Engineering Capabilities:

  • Ability to write clean, modular research code that scales

  • Strong software engineering practices including testing and code review

  • Experience with rapid prototyping and experimental design

  • Strong analytical skills for debugging model behavior and training dynamics

  • Facility with profiling and optimization tools

  • Track record of bringing research ideas to production

  • Experience maintaining high code quality in a research environment

About the Team:

You'll work directly alongside our research and engineering teams in our NYC office. We've intentionally built a culture where technical innovation and research excellence are highly valued - your success will be measured by your contributions to improving our models and advancing the field, not by your ability to navigate politics. We're a team that loves diving deep into complex technical problems and emerging with practical breakthroughs.

  • Our team values:

    • Open technical discussions and collaboration

    • Rapid iteration and practical solutions

    • Deep technical expertise and continuous learning

    • Direct impact on research and product outcomes

  • What sets us apart:

    • Opportunity to advance the state-of-the-art in video generation

    • Direct impact on products used by millions of creators

    • Access to massive compute resources and diverse, large-scale datasets

    • Environment that values both research excellence and practical impact

    • Ability to validate research through direct product feedback

Benefits:

  • Comprehensive medical, dental, and vision plans

  • 401K with employer match

  • Commuter Benefits

  • Catered lunch multiple days per week

  • Dinner stipend every night if you're working late and want a bite!

  • Doordash DashPass subscription

  • Health & Wellness Perks (Talkspace, Kindbody, One Medical subscription, HealthAdvocate, Teladoc)

  • Multiple team offsites per year with team events every month

  • Generous PTO policy

Captions provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

Please note benefits apply to full time employees only.

Similar Jobs

AI Fund - Curriculum Engineer

AI Fund

(Remote)
6 Months ago
Garena - Associate/Senior Associate, Revenue Operations

Garena

Bengaluru, Karnataka, India (On-Site)
3 Months ago
Alphawave Semi - Senior Staff Engineer - IP Design

Alphawave Semi

Bengaluru, Karnataka, India (On-Site)
1 Week ago
Self - Principal Growth Analyst

Self

Austin, Texas, United States (Remote)
2 Weeks ago
Saviynt - Technical Lead, Professional Services - NA

Saviynt

Bengaluru, Karnataka, India (Hybrid)
9 Months ago
Gigamon - Senior Sales Engineer

Gigamon

Seattle, Washington, United States (On-Site)
1 Month ago
Assystems - Site Mechanical Engineer

Assystems

Bagalkote, Karnataka, India (On-Site)
9 Months ago
lucas films - Senior IS Software Engineer

lucas films

Vancouver, British Columbia, Canada (Hybrid)
1 Month ago
Qualcomm - Staff Embedded Software Development Engineer

Qualcomm

Santa Clara, California, United States (On-Site)
1 Month ago
Mighty Jaxx - Software Engineer Intern

Mighty Jaxx

Singapore (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Ion - Senior Business Consultant - Aspect

Ion

Houston, Texas, United States (On-Site)
9 Months ago
HCL Tech - Full stack java tech. lead (react.js)

HCL Tech

Texas, United States (On-Site)
2 Months ago
Canonical - Graduate Software Quality Assurance Engineer

Canonical

Taipei City, Taiwan (On-Site)
2 Months ago
ansira - Senior Data Pipeline Developer

ansira

Colombia (Remote)
1 Year ago
Adobe - Software Development Engineer, Site Reliability Engineering

Adobe

Bucharest, Bucharest, Romania (On-Site)
3 Months ago
Survay Monkey - Senior Systems Engineer - DevPlatform Team

Survay Monkey

Bengaluru, Karnataka, India (Hybrid)
3 Months ago
CAE - Senior Market Analyst

CAE

Montreal, Quebec, Canada (On-Site)
2 Months ago
JDA - Senior Support Engineer 2 (BY SCPO/ BY Demand, Fulfillment)

JDA

Bengaluru, Karnataka, India (On-Site)
1 Week ago
hogarth - Content Manager - Contract

hogarth

Hyderabad, Telangana, India (Hybrid)
1 Week ago

Get notifed when new similar jobs are uploaded

Jobs in New York, United States

Glean - Software Engineer, Machine Learning

Glean

Palo Alto, California, United States (On-Site)
2 Months ago
Nice - Senior Professional Services Engineer, Actimize

Nice

Hoboken, New Jersey, United States (Hybrid)
1 Month ago
Gusto - Sales Manager, GustoPro (Accountant Channel)

Gusto

Denver, Colorado, United States (Hybrid)
1 Week ago
PlayGig - Lead Engineer

PlayGig

El Segundo, California, United States (On-Site)
4 Months ago
Alpha Sense - Senior Product Manager

Alpha Sense

United States (Remote)
2 Months ago
Axon - Revenue Accountant

Axon

Boston, Massachusetts, United States (Hybrid)
2 Months ago
C3 IoT - Solution Engineer

C3 IoT

Redwood City, California, United States (On-Site)
3 Weeks ago
Findhelp - Senior Staff Technical Solutions Account Manager

Findhelp

Madison, Wisconsin, United States (On-Site)
2 Months ago
Bungie - Destiny Senior Activity Designer

Bungie

Bellevue, Washington, United States (Hybrid)
2 Weeks ago
Epic Games - Lead Engineer

Epic Games

Cary, North Carolina, United States (On-Site)
6 Months ago

Get notifed when new similar jobs are uploaded

Software Development & Engineering Jobs

truecaller - Director of Engineering

truecaller

Stockholm, Stockholm County, Sweden (On-Site)
2 Months ago
Rocket - Manager, Professional Services Engineering

Rocket

United States (Remote)
3 Weeks ago
Apple - SoC DRAM Memory Subsystem Validation Engineering Program Manager

Apple

Cupertino, California, United States (On-Site)
2 Months ago
Alphawave Semi - Staff Engineer I - DFT

Alphawave Semi

Bengaluru, Karnataka, India (On-Site)
2 Months ago
luxsoft - L2 Support Engineer

luxsoft

Bucharest, Romania (On-Site)
3 Weeks ago
Toast - Engineering Manager - Infrastructure

Toast

United States (Remote)
1 Month ago
Toast - Engineering Manager II - Ecommerce

Toast

United States (Remote)
5 Months ago
bytedance - Software Engineer, Global Payment Privacy

bytedance

San Jose, California, United States (On-Site)
3 Months ago
Patreon - Engineering Manager

Patreon

New York, New York, United States (Hybrid)
4 Months ago
AECOM - Senior Traffic Engineer / Supervising Traffic Engineer

AECOM

Orange, California, United States (Hybrid)
1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Captions is the leading video AI company, building the future of video creation. Over 10 million creators and businesses have used Captions to create videos for social media, marketing, sales, and more. We're on a mission to serve the next billion. We are a rapidly growing team of ambitious, experienced, and devoted engineers, researchers, designers, marketers, and operators based in NYC. You'll join an early team and have an outsized impact on the product and the company's culture.

New York, New York, United States (On-Site)

New York, New York, United States (On-Site)

New York, New York, United States (On-Site)

New York, United States (On-Site)

New York, United States (On-Site)

New York, New York, United States (On-Site)

New York, New York, United States (On-Site)

New York, United States (On-Site)

New York, United States (On-Site)

View All Jobs

Get notified when new jobs are added by Captions

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug