Product Manager - LLM Data Acquisition and Production
Job Description
- Candidates with global education or work experience are preferred
- A strong interest in both LLMs and human behavior/experience/wellbeing
- Enthusiastic learner, not finding reading papers and books tedious
- Collaborative team player who knows how to assertively advocate for their ideas without being overbearing
- Skilled in maintaining attention to detail while staying focused on prioritized goals and the bigger picture
- Responsible for requirement analysis and strategy optimization of Big Language Model
- Responsible for data quality and model effect
- Collaborate with technical teams to precisely define LLM data requirements
- Develop methodologies for data acquisition and production, monitoring costs and assessing effectiveness
- Ensure data quality is maintained at high standards and continually enhance processes
- Evaluate the impact of data production tools and work with technical teams to enhance them
- Proactively identify and mitigate potential biases and vulnerabilities in data production
- Solid understanding of alignment and related approaches (SFT, RLHF) or solid experience in moderation, labeling, search quality, or data synthesis
- Minimum of four years experience as a product manager or product strategist, including process design experience
- Proficiency in English (spoken and written)
Responsibilities
Responsible for requirement analysis and strategy optimization of Big Language Model, responsible for data quality and model effect, and working with multiple teams such as evaluation and R & D to improve model effect Responsibilities:
1. Collaborate with technical teams to precisely define the data requirements necessary for aligning Large Language Models (LLMs) across global markets.
2. Develop methodologies for data acquisition and production, while also monitoring costs and assessing the effectiveness and efficiency of these processes. Ensure data quality is maintained at high standards and continually enhance processes based on both quantitative and qualitative feedback.
3. Evaluate the impact of data production tools on the effectiveness and quality of data production. Work closely with technical teams to enhance both the quality and effectiveness of these tools.
4. Proactively identify and mitigate potential biases and vulnerabilities in the data production process. This includes, but is not limited to, addressing human and system biases through the implementation of technical, product, and operational solutions.
Qualifications
Minimum Qualifications:
1. Solid understanding of alignment and related approaches, such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), and others. Or solid experience in moderation, labeling, search quality, or data synthesis.
2. Minimum of four years experience as a product manager or product strategist, including process design experience.
3. Proficiency in English (spoken and written).
Preferred Qualifications:
1. Candidates with global education or work experience are preferred.
2. A strong interest in both LLMs and human behavior/experience/wellbeing. Must be an enthusiastic learner. This position is not suitable for individuals who find reading papers and books to be tedious.
3. Collaborative team player who knows how to assertively advocate for their ideas without being overbearing. Skilled in maintaining attention to detail while staying focused on prioritized goals and the bigger picture.
