AIML - Machine Learning Engineer, Model Evaluations
4 Months ago • All levels • $175,800 PA - $312,200 PA
Research Development
Job Description
This role focuses on evaluating and mitigating safety risks in generative AI features within Apple Intelligence. The responsibilities include developing metrics for safety and fairness evaluations, designing datasets, collaborating with cross-functional teams, translating regional safety requirements into evaluation criteria, building expertise in machine translation and data synthesis, and developing ML-based enhancements to improve product quality. The role involves working with highly sensitive and potentially offensive content. The candidate will work at the intersection of applied data science, empirical analysis, cultural and linguistic expertise, and stakeholder communication.
Must Have:
- Develop metrics for evaluation of safety and fairness risks.
- Design datasets, identify data needs and creative solutions.
- Collaborate with cross-functional partners.
- Translate regional safety and inclusivity requirements.
Apple Intelligence is driven by intentional data design—spanning careful sampling, creation, and curation of high-quality datasets, enriched with precise annotations. Our data powers our ability to evaluate and mitigate safety risks in new generative AI features. This role sits at the intersection of applied data science, empirical analysis, cultural and linguistic expertise, and stakeholder communication. It requires strong scientific judgment, cross-functional collaboration, and the ability to translate evaluation findings into actionable insights. - Develop metrics for evaluation of safety and fairness risks inherent to generative models and Gen-AI features - Design datasets, identify data needs, and work on creative solutions, scaling and expanding data coverage through human and synthetic generation methods - Collaborate with cross-functional partners—including engineering, product, and research teams—to ensure evaluations align with feature goals and deployment plans - Partner with policy teams to translate regional safety and inclusivity requirements into measurable evaluation criteria - Build expertise in machine translation and data synthesis techniques to generate localized and culturally aligned evaluation datasets at scale - Develop ML-based enhancements to red teaming, model evaluation, and other processes to improve the quality of Apple Intelligence’s user-facing products - Work with highly-sensitive content with exposure to offensive and controversial content
Set alerts for more jobs like AIML - Machine Learning Engineer, Model Evaluations
Set alerts for new jobs by Apple
Set alerts for new Research Development jobs in United States
Set alerts for new jobs in United States
Set alerts for Research Development (Remote) jobs
 Electric Powertrain Application/Performance Development Engineer (General Layer, General Manager/Staff Position)](/_next/image?url=https%3A%2F%2Foutscal-assets.s3.ap-south-1.amazonaws.com%2Fproduction%2Fcompany%2F10_26_2024__07_13_21.png&w=96&q=75){kind=small}
