Machine Learning Engineer - Perception Semantics

1 Month ago • All levels • Research Development • $160,000 PA - $245,000 PA

Job Summary

Job Description

The Perception team at Zoox is crucial for their autonomous vehicle technology, responsible for creating the understanding of the world for self-driving robots. They enable safe and efficient navigation in complex environments through sophisticated detection, classification, and tracking systems. As an engineer in the Semantics and Foundation Model team, you will develop advanced multimodal large language models to enhance the vehicles' environmental understanding. You will develop and fine-tune these models for off-vehicle analysis and optimize them for real-time performance on the robotaxi platform, ensuring efficient identification of hazards and interpretation of driving restrictions with minimal latency. Working with world-class engineers and researchers, you will leverage premium sensor data and cutting-edge infrastructure to validate algorithms in real-world conditions, directly impacting the safety and capability of Zoox's autonomous system.
Must have:
  • Lead development of multimodal large language models
  • Design efficient model architectures
  • Drive end-to-end ML solutions
  • Collaborate with perception, planning, and systems teams
  • Validate and optimize solutions using real-world scenarios
  • MS or PhD in Computer Science, ML, or related field
  • Experience training and deploying perception/computer vision models
  • Experience building ML training pipelines
  • Proficiency in Python and ML libraries (PyTorch, NumPy)
  • Experience with model optimization techniques
Good to have:
  • Proficiency in C++
  • Publications in top-tier conferences
  • Experience with autonomous robotics systems

Job Details

The Perception team at Zoox is fundamental to our autonomous vehicle technology, creating the understanding of the world for our self-driving robots. We enable safe and efficient navigation in complex environments through sophisticated detection, classification, and tracking systems.
As an engineer in the Semantics and Foundation Model team, you will develop advanced multimodal large language models that enhance our vehicles' environmental understanding. You'll develop and fine-tune these models for off-vehicle analysis while also optimizing them for real-time performance on our robotaxi platform, ensuring they can efficiently identify hazards and interpret driving restrictions with minimal latency. Working alongside world-class engineers and researchers, you'll leverage premium sensor data and cutting-edge infrastructure to validate your algorithms in real-world conditions, directly impacting the safety and capability of Zoox's autonomous system.

In this role, you will:

    • Lead the development of multimodal large language models that enhance our robotaxis' understanding of complex urban environments
    • Design and implement efficient model architectures, balancing performance and computational constraints through fine-tuning and distillation techniques
    • Drive end-to-end ML solutions from research to production, utilizing Zoox's extensive data pipelines and infrastructure to improve autonomous driving capabilities
    • Collaborate with perception, planning, and systems teams to integrate your models into the vehicle's decision-making pipeline
    • Validate and optimize your solutions using real-world driving scenarios, directly contributing to the safety and reliability of Zoox's autonomous system

Qualifications

    • MS or PhD in Computer Science, Machine Learning, or related technical field
    • Demonstrated experience training and deploying perception and computer vision based models
    • Experience building and maintaining ML training pipelines, including data preprocessing, model training, and evaluationProficiency in Python and ML libraries (PyTorch, NumPy) demonstrated through professional or research projects
    • Experience with model optimization techniques such as quantization, pruning, or knowledge distillation

Bonus Qualifications

    • Proficiency in C++
    • Publications in top-tier conferences (CVPR, ICCV, RSS, ICRA)
    • Experience with autonomous robotics systems

Similar Jobs

Hudl - Senior Quality Assurance Engineer

Hudl

Austin, Texas, United States (Remote)
1 Month ago
Adobe - Manager, Public Policy

Adobe

Washington, District Of Columbia, United States (On-Site)
1 Month ago
PwC - Associate

PwC

Chennai, Tamil Nadu, India (On-Site)
9 Months ago
Adyen - Data Engineering Manager - Reporting & Reconciliation

Adyen

Amsterdam, North Holland, Netherlands (On-Site)
1 Month ago
Netflix - Sales Operations Manager, Revenue Analytics & Insights

Netflix

New York, New York, United States (On-Site)
6 Months ago
Mozilla - Staff Machine Learning Engineer, Gen AI

Mozilla

Germany (Remote)
8 Months ago
eBay - Applied Researcher

eBay

Bengaluru, Karnataka, India (Hybrid)
3 Weeks ago
Autodesk - Distinguished Research Scientist, Physics Informed AI

Autodesk

Toronto, Ontario, Canada (On-Site)
2 Weeks ago
Apple - AIML - Machine Learning Engineer, Measurement

Apple

Seattle, Washington, United States (On-Site)
3 Weeks ago
Apple - Machine Learning Applied Research Scientist

Apple

Cupertino, California, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Riot Games - Producer III - League Studios, Creative eXpressions

Riot Games

Los Angeles, California, United States (On-Site)
1 Month ago
Liquid nitro games - Recruiter

Liquid nitro games

Hyderabad, Telangana, India (On-Site)
2 Months ago
Nice - Senior Revenue Accountant

Nice

Pune, Maharashtra, India (Hybrid)
1 Week ago
Western Digital - Senior Engineer, Engineering

Western Digital

Bengaluru, Karnataka, India (On-Site)
2 Weeks ago
Sporty - Marketing & Business Development Manager

Sporty

São Paulo, Brazil (Hybrid)
8 Months ago
Progress - Director, Product Marketing - Digital Experience BU

Progress

United States (Remote)
1 Month ago
ARHS - Solution Architect (Data Migration)

ARHS

Stockholm, Stockholm County, Sweden (Remote)
8 Months ago
Rocket Science - Software Engineer - UI

Rocket Science

Cardiff, Wales, United Kingdom (Hybrid)
3 Months ago
Ubisoft - Team Lead - Animation

Ubisoft

Montreal, Quebec, Canada (Hybrid)
1 Month ago
upwork - Director of Product Operations

upwork

United States (Remote)
5 Days ago

Get notifed when new similar jobs are uploaded

Jobs in Foster City, California, United States

Cloud Imperium Games - Community Manager

Cloud Imperium Games

Austin, Texas, United States (On-Site)
4 Weeks ago
LLNL - Waste Characterization Analyst

LLNL

Livermore, California, United States (Hybrid)
2 Weeks ago
Apple - US Carrier Sales Finance Manager

Apple

Cupertino, California, United States (On-Site)
3 Weeks ago
Nintendo - Senior Software Engineer (Python/React)

Nintendo

Redmond, Washington, United States (Hybrid)
10 Months ago
Google - Customer Engineer II, Cloud AI

Google

San Francisco, California, United States (On-Site)
2 Months ago
Yahoo - Business Development Lead

Yahoo

United States (Hybrid)
3 Weeks ago
bytedance - Software Development Engineer (SDN Traffic Intelligence & Control)

bytedance

San Jose, California, United States (On-Site)
3 Months ago
Scout - Senior Specialist, Retail Architecture

Scout

Fremont, California, United States (On-Site)
1 Month ago
Axon - Senior Enterprise Account Executive

Axon

Seattle, Washington, United States (Remote)
4 Months ago
Google - Senior Technical Program Manager I, Infrastructure, Google Cloud

Google

Mountain View, California, United States (On-Site)
6 Months ago

Get notifed when new similar jobs are uploaded

Research Development Jobs

Tesla - Associate Application Engineer R&D

Tesla

Prüm, Rhineland-Palatinate, Germany (On-Site)
4 Months ago
bytedance - Research Engineer / Scientist - AI for Databases

bytedance

Seattle, Washington, United States (On-Site)
1 Month ago
Ubisoft - Junior R&D Engineer

Ubisoft

Pune, Maharashtra, India (Hybrid)
2 Weeks ago
Apple - Machine Learning Applied Research Scientist

Apple

Cupertino, California, United States (On-Site)
1 Month ago
Apple - Red Team Platform and Hardware Security Researcher

Apple

Cupertino, California, United States (On-Site)
1 Week ago
Ansys - R&D Verification Engineer II (Structural)

Ansys

Waterloo, Ontario, Canada (On-Site)
1 Month ago
JDA - Director of Internal AI / Gen-AI Technology

JDA

Dallas, Texas, United States (Remote)
3 Weeks ago
London stock Exchange - Lead Research Analyst (Arabic)

London stock Exchange

Penang, Malaysia (Hybrid)
2 Weeks ago
Apple - Senior Machine Learning Research Engineer (Computer Vision)

Apple

Seattle, Washington, United States (On-Site)
1 Month ago
DevRev - Software Engineer - Applied AI Support

DevRev

Buenos Aires, Buenos Aires, Argentina (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Zoox is transforming mobility-as-a-service by developing a fully autonomous, purpose-built fleet designed for AI to drive and humans to enjoy.

Foster City, California, United States (On-Site)

Foster City, California, United States (On-Site)

Foster City, California, United States (Hybrid)

Foster City, California, United States (On-Site)

Foster City, California, United States (On-Site)

Foster City, California, United States (Hybrid)

Foster City, California, United States (Hybrid)

Foster City, California, United States (Hybrid)

Foster City, California, United States (Hybrid)

View All Jobs

Get notified when new jobs are added by zoox

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug