AI Scientist, Safety
Job Summary
Job Description
Mistral AI is seeking an AI Scientist, Safety to evaluate, enhance, and build safety mechanisms for their large language models (LLMs). This role involves identifying and addressing potential risks, biases, and misuses of LLMs, ensuring that AI systems are ethical, fair, and beneficial to society. The scientist will monitor models, prevent misuse, and ensure user well-being. Responsibilities include designing and executing adversarial attacks, assessing LLMs for biases, developing monitoring systems, building multi-layered defenses, investigating incidents of LLM misuse, and contributing to AI ethics policies. The role also involves safety fine-tuning to improve model robustness and collaborating with the AI development team.
Must have:
- Degree in Computer Science, AI, or ML
- Proficient in Python and another programming language
- Experience with AI frameworks (TensorFlow, PyTorch, Jax)
- High technical engineering competence
- High scientific track record
- Self-starter, autonomous, low-ego
- Team player mindset
Good to have:
- Proven experience in AI safety/responsible AI
- Familiarity with LLMs and their risks
- Hands-on experience with Generative AI
- Knowledge of transformer-based models
- Experience with MLOps technical stack
Perks:
- Competitive cash salary and equity
- Daily lunch vouchers (France)
- Monthly contribution to Gympass subscription (France)
- Monthly contribution to a mobility pass (France)
- Full health insurance for you and your family (France)
- Generous parental leave policy (France)
- Visa sponsorship
- Insurance (UK)
- Reimbursement for office parking charges or public transport (UK)
- Monthly reimbursement for gym membership (UK)
- Monthly meal allowance (UK)
- Pension plan (UK)
7 skills required
Job Details
About Mistral
At Mistral AI, we believe in the power of AI to simplify tasks, save time, and enhance learning and creativity. Our technology is designed to integrate seamlessly into daily working life.
We democratize AI through high-performance, optimized, open-source and cutting-edge models, products and solutions. Our comprehensive AI platform is designed to meet enterprise needs, whether on-premises or in cloud environments. Our offerings include le Chat, the AI assistant for life and work.
We are a dynamic, collaborative team passionate about AI and its potential to transform society.
Our diverse workforce thrives in competitive environments and is committed to driving innovation. Our teams are distributed between France, USA, UK, Germany and Singapore. We are creative, low-ego and team-spirited.
Join us to be part of a pioneering company shaping the future of AI. Together, we can make a meaningful impact. See more about our culture on https://mistral.ai/careers.
Role Summary
We are seeking an AI Scientist, Safety to evaluate, enhance, and build safety mechanisms for our large language models (LLMs). This role involves identifying and addressing potential risks, biases, and misuses of LLMs, ensuring that our AI systems are ethical, fair, and beneficial to society. You will work to monitor models, prevent misuse, and ensure user well-being, applying your technical skills to uphold principles of safety, transparency, and oversight.
Location : Paris or London
What you will do
Adversarial & Fairness Testing
• Design and execute adversarial attacks to uncover vulnerabilities in LLMs.
• Evaluate potential risks and harms associated with LLM outputs.
• Assess LLMs for biases and unfairness in their responses, and develop strategies to mitigate these issues.
Tools & Monitoring
• Develop monitoring systems (eg. moderation tools) to detect unwanted behaviors in Mistral’s products.
• Build robust and reliable multi-layered defenses for real-time improvement of safety mechanisms that work at scale.
• Investigate and respond to incidents involving LLM misuse or harmful outputs, and develop post-incident recommendations.
• Analyze user reports of inappropriate content or accounts.
• Contribute to the development of AI ethics policies and guidelines that govern the responsible use of LLMs.
Safety Fine Tuning
• Work on safety tuning to improve robustness of models.
• Collaborate with the AI development team to create and implement safety measures, such as content filters, moderation tools, and model fine-tuning techniques.
• Keep up-to-date with the latest research and trends in AI safety, LLMs, and responsible AI, and continuously improve our safety practices.
• Build robust and reliable multi-layered defenses for real-time improvement of safety mechanisms that work at scale
About you
• You have a degree in Computer Science, AI, Machine Learning, or a related field. Advanced degrees (MSc, PhD) are preferred.
• You are familiar with Python and you are a highly proficient software engineer in a least one programming language (e.g. Python, Rust, Go, Java)You have, hands-on experience with AI frameworks and tools (e.g., TensorFlow, PyTorch, Jax)
• You have high technical engineering competence. This means being able to design complex software and make them usable in production
• You have a high scientific track record in a field of science.
• You are self-starter, autonomous and low-ego.
• Collaborative and have a real team player mindset.
Note that this is not an exhaustive or necessary list of requirements, please consider applying if you believe you have the skills to contribute to Mistral's mission.
Now, it would be ideal if
• You have proven experience in AI safety, responsible AI, or a related field. Familiarity with LLMs and their potential risks is essential.
• You have hands-on experience with Generative AI e.g. experience with transformer based models and a broad knowledge of the field of AI, and specific knowledge or interest in fine-tuning and using language models for applications.
• You are able to navigate the full MLOps technical stack, with a focus on architecture development and model evaluation and usage
Benefits
France
💰 Competitive cash salary and equity
🥕 Food : Daily lunch vouchers
🥎 Sport : Monthly contribution to a Gympass subscription
🚴 Transportation : Monthly contribution to a mobility pass
🧑⚕️ Health : Full health insurance for you and your family
🍼 Parental : Generous parental leave policy
🌎 Visa sponsorship
UK
💰 Competitive cash salary and equity
🚑 Insurance
🚴 Transportation: Reimburse office parking charges, or 90GBP/month for public transport
🥎 Sport: 90GBP/month reimbursement for gym membership
🥕 Meal voucher: £200 monthly allowance for its meals
💰 Pension plan: SmartPension (percentages are 5% Employee & 3% Employer)
Similar Jobs
bytedance
Seattle, Washington, United States (On-Site)
• 4 Months ago
Discord
San Francisco, California, United States (Remote)
• 3 Weeks ago
Get notifed when new similar jobs are uploaded
Similar Skill Jobs
Naughty Dog
Los Angeles, California, United States (Hybrid)
• 3 Months ago
PwC
Dublin, County Dublin, Ireland (On-Site)
• 10 Months ago
bytedance
Singapore (On-Site)
• 9 Months ago
Get notifed when new similar jobs are uploaded
Jobs in Paris, Île-de-France, France
Square
Brest, Brittany, France (On-Site)
• 3 Weeks ago
Square
Montreuil, Île-de-France, France (On-Site)
• 3 Weeks ago
Ubisoft
Bordeaux, Nouvelle-Aquitaine, France (Hybrid)
• 6 Months ago
Square
Limoges, Nouvelle-Aquitaine, France (On-Site)
• 1 Month ago
We are social
Paris, Île-de-France, France (Hybrid)
• 2 Months ago
Get notifed when new similar jobs are uploaded
Research Development Jobs
world resource institute
Jakarta, Indonesia (On-Site)
• 2 Weeks ago
New York, New York, United States (On-Site)
• 3 Months ago
Apple
Sunnyvale, California, United States (On-Site)
• 2 Months ago
Get notifed when new similar jobs are uploaded
