Machine Learning Engineer - Perception Semantics

2 Months ago • All levels • Research Development • $160,000 PA - $245,000 PA

Job Summary

Job Description

The Perception team at Zoox is crucial for their autonomous vehicle technology, responsible for creating the understanding of the world for self-driving robots. They enable safe and efficient navigation in complex environments through sophisticated detection, classification, and tracking systems. As an engineer in the Semantics and Foundation Model team, you will develop advanced multimodal large language models to enhance the vehicles' environmental understanding. You will develop and fine-tune these models for off-vehicle analysis and optimize them for real-time performance on the robotaxi platform, ensuring efficient identification of hazards and interpretation of driving restrictions with minimal latency. Working with world-class engineers and researchers, you will leverage premium sensor data and cutting-edge infrastructure to validate algorithms in real-world conditions, directly impacting the safety and capability of Zoox's autonomous system.
Must have:
  • Lead development of multimodal large language models
  • Design efficient model architectures
  • Drive end-to-end ML solutions
  • Collaborate with perception, planning, and systems teams
  • Validate and optimize solutions using real-world scenarios
  • MS or PhD in Computer Science, ML, or related field
  • Experience training and deploying perception/computer vision models
  • Experience building ML training pipelines
  • Proficiency in Python and ML libraries (PyTorch, NumPy)
  • Experience with model optimization techniques
Good to have:
  • Proficiency in C++
  • Publications in top-tier conferences
  • Experience with autonomous robotics systems

Job Details

The Perception team at Zoox is fundamental to our autonomous vehicle technology, creating the understanding of the world for our self-driving robots. We enable safe and efficient navigation in complex environments through sophisticated detection, classification, and tracking systems.
As an engineer in the Semantics and Foundation Model team, you will develop advanced multimodal large language models that enhance our vehicles' environmental understanding. You'll develop and fine-tune these models for off-vehicle analysis while also optimizing them for real-time performance on our robotaxi platform, ensuring they can efficiently identify hazards and interpret driving restrictions with minimal latency. Working alongside world-class engineers and researchers, you'll leverage premium sensor data and cutting-edge infrastructure to validate your algorithms in real-world conditions, directly impacting the safety and capability of Zoox's autonomous system.

In this role, you will:

    • Lead the development of multimodal large language models that enhance our robotaxis' understanding of complex urban environments
    • Design and implement efficient model architectures, balancing performance and computational constraints through fine-tuning and distillation techniques
    • Drive end-to-end ML solutions from research to production, utilizing Zoox's extensive data pipelines and infrastructure to improve autonomous driving capabilities
    • Collaborate with perception, planning, and systems teams to integrate your models into the vehicle's decision-making pipeline
    • Validate and optimize your solutions using real-world driving scenarios, directly contributing to the safety and reliability of Zoox's autonomous system

Qualifications

    • MS or PhD in Computer Science, Machine Learning, or related technical field
    • Demonstrated experience training and deploying perception and computer vision based models
    • Experience building and maintaining ML training pipelines, including data preprocessing, model training, and evaluationProficiency in Python and ML libraries (PyTorch, NumPy) demonstrated through professional or research projects
    • Experience with model optimization techniques such as quantization, pruning, or knowledge distillation

Bonus Qualifications

    • Proficiency in C++
    • Publications in top-tier conferences (CVPR, ICCV, RSS, ICRA)
    • Experience with autonomous robotics systems

Similar Jobs

Critical mass - Project Manager

Critical mass

San Jose, California, United States (Hybrid)
3 Weeks ago
Philips - Solutions Program Manager, SRC

Philips

Murrysville, Pennsylvania, United States (On-Site)
2 Months ago
London stock Exchange - Principal Platform Engineer

London stock Exchange

London, England, United Kingdom (On-Site)
2 Months ago
Ramboll3 - Design Engineer - Structural

Ramboll3

Hyderabad, Telangana, India (On-Site)
1 Month ago
The Globel Talent Co - Sales Specialist (German-speaker)

The Globel Talent Co

Johannesburg, Gauteng, South Africa (Remote)
6 Months ago
NVIDIA - Deep Learning Intern - Fall 2025

NVIDIA

Shanghai, Shanghai, China (On-Site)
3 Months ago
Genies - Engineering Manager, Machine Learning

Genies

Los Angeles, California, United States (On-Site)
4 Months ago
Dream Sports - Senior ML Scientist

Dream Sports

Mumbai, Maharashtra, India (On-Site)
8 Months ago
DevRev - Member of Applied AI Engineering Team ( ETL)

DevRev

Ljubljana, Ljubljana, Slovenia (On-Site)
2 Months ago
bytedance - Backend Software Engineer, AI Applications

bytedance

San Jose, California, United States (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

PwC - BPS Associate (Proby)

PwC

Pasig, Metro Manila, Philippines (On-Site)
9 Months ago
Ion - Software Developer/Engineer - Graduate Development Program

Ion

Pisa, Tuscany, Italy (On-Site)
9 Months ago
Hawkeye Innovations - Match Day Technician

Hawkeye Innovations

Tripoli, Greece (On-Site)
1 Month ago
MiQ - Account Manager

MiQ

Manila, Metro Manila, Philippines (On-Site)
2 Months ago
Toppan MErril - Second Shift Senior Fulfillment Specialist

Toppan MErril

Sartell, Minnesota, United States (On-Site)
1 Year ago
Hudl - Senior Quality Assurance Manager

Hudl

Lincoln, Nebraska, United States (On-Site)
1 Month ago
IGT - Director of Brand Licensing

IGT

Providence, Rhode Island, United States (On-Site)
2 Months ago
Diligent Corporation - Head of Human Resources Business Partnering (EMEA)

Diligent Corporation

London, England, United Kingdom (Hybrid)
1 Month ago
Britive - ENGINEERING MANAGER

Britive

Bengaluru, Karnataka, India (Remote)
8 Months ago
Tesla - Training Coordinator - Parts Operations

Tesla

Lisbon, Lisbon, Portugal (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Foster City, California, United States

Nintendo - CONTRACT - Software Engineer - Multimedia Technology (NTD)

Nintendo

Redmond, Washington, United States (On-Site)
8 Months ago
Sawhorse Productions - Data Analyst

Sawhorse Productions

California, United States (Remote)
4 Months ago
Light Speed Studios - Senior Technical Artist

Light Speed Studios

Irvine, California, United States (On-Site)
7 Months ago
Christ Fellowship - Project Associate

Christ Fellowship

Palm Beach Gardens, Florida, United States (On-Site)
3 Weeks ago
Interface AI - Technical Program Manager

Interface AI

San Francisco, California, United States (On-Site)
1 Month ago
bytedance - Research Scientist, Foundation Model, Speech & Audio

bytedance

San Jose, California, United States (On-Site)
8 Months ago
Kavalirio - Cloud Engineer

Kavalirio

Denver, Colorado, United States (On-Site)
1 Month ago
Luxoft - Solution Architect

Luxoft

Poland, Ohio, United States (Remote)
7 Months ago
Apple - Sr. Infrastructure Software Engineer (ASE Infra Hardware)

Apple

Cupertino, California, United States (On-Site)
1 Month ago
Luxoft - Data Engineer for Market Data Projects (with Streamlit Expertise)

Luxoft

Brazil, Indiana, United States (Remote)
8 Months ago

Get notifed when new similar jobs are uploaded

Research Development Jobs

Apple - AIML - Machine Learning Engineer, Foundation Models

Apple

Cupertino, California, United States (On-Site)
2 Months ago
PayPal - Sr Staff Software Engineer – Machine Learning

PayPal

San Jose, California, United States (Hybrid)
1 Month ago
bytedance - Software Engineer Intern (Applied Machine Learning-Enterprise) - 2025 Summer/Fall (PhD)

bytedance

San Jose, California, United States (On-Site)
9 Months ago
Inwave - R&D Specialist

Inwave

(On-Site)
5 Months ago
Well - Senior Machine Learning Engineer

Well

New York, United States (On-Site)
1 Month ago
upwork - Senior Lead Machine Learning Engineer

upwork

(Remote)
2 Months ago
Nice - AI Architect

Nice

Sandy, Utah, United States (On-Site)
3 Weeks ago
Token Metrics - AI Prompt Engineer

Token Metrics

Medellín, Antioquia, Colombia (Remote)
8 Months ago
brown bag films - GLAAD Media Awards Nominations for Ada Twist, Scientist and Ridley Jones

brown bag films

Dublin, County Dublin, Ireland (On-Site)
1 Year ago
bytedance - Research Scientist in Large Language Model

bytedance

San Jose, California, United States (On-Site)
8 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Zoox is transforming mobility-as-a-service by developing a fully autonomous, purpose-built fleet designed for AI to drive and humans to enjoy.

Foster City, California, United States (On-Site)

Fremont, California, United States (On-Site)

Foster City, California, United States (Hybrid)

Foster City, California, United States (Hybrid)

Foster City, California, United States (Hybrid)

Foster City, California, United States (Hybrid)

Foster City, California, United States (On-Site)

Foster City, California, United States (Hybrid)

Foster City, California, United States (Hybrid)

Foster City, California, United States (Hybrid)

View All Jobs

Get notified when new jobs are added by zoox

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug