Research Scientist/ML Engineer - Answer Quality
4 Months ago • 4 Years + • $200,000 PA - $280,000 PA
Job Description
As a Machine Learning Engineer / Research Scientist in the Answer Quality group, you will develop and refine evaluation methodologies to measure the quality, safety, and impact of AI models. Responsibilities include designing and building evaluation systems, collaborating with product and data teams, conducting research, and driving answer quality. You will design and analyze experiments, integrate eval results, and contribute to a scalable evaluation platform. The role involves building tools and establishing norms to improve answer quality across new products, ensuring high-quality answers that drive user engagement. This role is pivotal in ensuring the search engine delivers great answers.
Good To Have:
- Prior experience in evaluating large language models.
- Exposure to research methodologies.
- Experience in a startup environment.
Must Have:
- 4+ years of ML Engineer or Research Scientist experience.
- Experience in designing and running experiments.
- Solid understanding of Bayesian statistics.
- Deep knowledge in machine learning.
- Proficiency in Python and ML frameworks.
Perks:
- Comprehensive health, dental, and vision insurance.
- 401(k) plan.
- Equity may be part of total compensation.
We are hiring for a Machine Learning Engineer / Research Scientist – Evals to join our AI team. As an early member of our Answer Quality group, you will play a pivotal role in developing and refining the evaluation methodologies that measure the quality, safety, and impact of our AI models. Your work will be central to ensuring that our search engine delivers high-quality answers that drive user engagement and reduce churn.
Responsibilities
- Design and Build Evaluation Systems, ensuring our evaluation metrics are reliable, scalable, and actionable. Design, implement, and analyze experiments (such as A/B tests) to validate hypotheses regarding answer quality.
- Integration with Product and Data Teams: Collaborate closely with our PM and AI teams to define common evaluation standards and integrate eval results into our product feedback loop.
- Research and Innovation: Conduct research into state-of-the-art evaluation methods and contribute to the long-term vision for a centralized, scalable evaluation platform.
- Drive Answer Quality via Culture and Tooling: As Perplexity ships new products, build the tools and establish organizational norms to drive answer quality up for each of the new products (Deep Research, Pro Search, Vertical Search, Enterprise, etc.)
Qualifications
- Have 4+ years of experience as an ML Engineer or Research Scientist, with a strong background in designing and running experiments in a fast-growing technology environment
- Solid understanding of Bayesian statistics, and various tools and methodologies for evaluating uncertainty in a way that is specific to the given product being shipped
- Possess deep knowledge in machine learning, including evaluation methods, statistical testing, and algorithm development
- Proficiency in Python and have hands-on experience with ML frameworks (e.g., TensorFlow, PyTorch) and data processing tools
Bonuses:
- Prior experience in evaluating large language models (LLMs) or building evaluation systems for consumer search products
- Exposure to research methodologies in ranking, retrieval, or natural language processing
- Experience in a startup or early-stage company environment
The cash compensation range for this role is $200,000 - $280,000.
At Perplexity, we've experienced tremendous growth and adoption since publicly launching the world's first fully functional conversational answer engine in 2022. We've grown from answering 2.5 million questions per day at the start of 2024 to around 20 million daily queries in December 2024. We also offer Perplexity Enterprise Pro, which counts leading companies like Nvidia, the Cleveland Cavaliers, Bridgewater, and Zoom as customers.
To support our rapid expansion, we've raised significant funding from some of the most respected technology investors. Our investor base includes IVP, NEA, Jeff Bezos, NVIDIA, Databricks, Bessemer Venture Partners, Elad Gil, Nat Friedman, Daniel Gross, Naval Ravikant, Tobi Lutke, and many other visionary individuals. In 2024, our employee base grew nearly 300%, and we're just getting started.Final offer amounts are determined by multiple factors, including, experience and expertise, and may vary from the amounts listed above.
Equity: In addition to the base salary, equity may be part of the total compensation package.
Benefits: Comprehensive health, dental, and vision insurance for you and your dependents. Includes a 401(k) plan.
Set alerts for new jobs by Perplexity
Set alerts for new jobs in United States
