Nousresearch

Machine Learning Engineer (Data & Evaluation Infrastructure)

Head of Finance

Machine Learning Engineer (Training Infrastructure)

Research Scientist

General Counsel

We are seeking a Machine Learning Engineer (MLE) to manage our post-training evaluation pipeline. The role involves building and scaling evaluation processes to assess model capabilities across various tasks, pinpointing areas of failure, and driving improvements. Key responsibilities include identifying tasks for evaluation, creating or curating test cases and measurement methods, implementing evaluations through objective verification, LLM judging, reward modeling, or human evaluation. You will also be responsible for expanding coverage, deeply analyzing failure cases, identifying solutions, and developing scalable and accessible internal evaluation presentation methods, such as GUIs or Slurm scripts.

<div class="job-description"> We’re looking for an MLE to own our post-training evaluation pipeline. You’ll build and scale evals depth and breadth that measure model capabilities across diverse tasks, identify failure modes, and drive model improvements. <h2><b>Responsibilities:</b></h2> <ul>   <li aria-level="1">Identifying tasks for evaluation coverage</li>   <li aria-level="1">Creating, curating, or generating test cases and ways to measure these tasks</li>   <li aria-level="1">Implementing evaluation through objective output verification, LLM judge/reward modeling, human evaluation, or any tricks of the trade you may bring to the table</li>   <li aria-level="1">Adding coverage and diving deep into analyzing what’s really gone wrong in failure cases</li>   <li aria-level="1">Identifying ways to remedy failure cases</li>   <li aria-level="1">Developing ways to present and make the evals scalable and accessible internally (e.g. light GUIs, scalable Slurm scripts, etc for running the evals)</li> </ul> <h2><b>Qualifications:</b></h2> <ul>   <li aria-level="1">Strong experience with evaluation frameworks</li>   <li aria-level="1">Experience with both automated and human evaluation methodologies</li>   <li aria-level="1">Ability to build evaluation infrastructure from scratch and scale existing systems</li> </ul> <h2><b>Preferred:</b></h2> <ul>   <li aria-level="1">History of OSS contributions</li> </ul> </div>

thumbnail_Rockstar.webp

Rockstar.webp

Rockstar Jobs

thumbnail_EA.webp

EA.webp

EA Jobs

thumbnail_Playstation.webp

Playstation.webp

Playstation Jobs

small_Ubisoft.webp

thumbnail_Ubisoft.webp

Ubisoft.webp

Ubisoft Jobs

thumbnail_Epic Games.webp

Epic Games.webp

Epic Game Jobs

thumbnail_Riot Games.webp

Riot Games.webp

Riot Jobs

08cafc09dac48d655284704f063dc64e.webp

CDPR Jobs

80ad888d-4837-48d9-8b65-0ffdee910f35.webp

Krafton Jobs

thumbnail_a14feb4b393500244d0b8864bda07b2c.webp

a14feb4b393500244d0b8864bda07b2c.webp

Twitch Jobs

thumbnail_865c7fd0-1ffd-4000-80b6-1e69d6665257.webp

865c7fd0-1ffd-4000-80b6-1e69d6665257.webp

Larian Jobs

thumbnail_02_21_2024__11_00_53.webp

02_21_2024__11_00_53.webp

Wargaming Jobs

medium_naughty dog.webp

small_naughty dog.webp

thumbnail_naughty dog.webp

naughty dog.webp

Naughty Dog Jobs

large_netflix.webp

medium_netflix.webp

small_netflix.webp

thumbnail_netflix.webp

netflix.webp

Netflix Jobs

large_supercell.webp

medium_supercell.webp

small_supercell.webp

thumbnail_supercell.webp

supercell.webp

Supercell Jobs

zynga_logo.jpg

Zynga Jobs

Test your Unity Animations knowledge with this quiz! Explore concepts like Animator Controller, Blend Trees, and many more.

Test your Unity UI knowledge with this quiz! Explore essential concepts like Canvas, EventSystem and many more. 

Challenge yourself with this Unity Input System quiz! Test your skills in input mapping, device handling, and action events.

Take your Unity skills to the next level with Part 3 of this quiz, covering scripting, physics, rendering, and performance optimization!

Test your Unity expertise with this quiz, covering initialization to advanced rendering and physics!

Test your Unity knowledge with this comprehensive quiz, covering everything from MonoBehaviour basics to advanced physics, rendering, and optimization techniques across multiple difficulty levels!

Assess your knowledge of Unity's Rigidbody component and its advanced physics interactions with this in-depth quiz!

Test your understanding of Unity’s Physics, Movement, and MonoBehaviour with this insightful beginner-friendly quiz!

Dive into the fundamentals of open-world game development in Unity with this beginner-friendly quiz.

Challenge yourself with this beginner-friendly quiz on building an endless runner in Unity!

Test your beginner-level knowledge of Unity platformer development with this engaging quiz.

Test your basic knowledge of Unity and Unreal Engine with this beginner-friendly quiz on game development fundamentals.

This assessment evaluates your understanding of Unity's component-based architecture, how GameObjects and their components interact, the hierarchy system with nested GameObjects, and how to reference them in scripts.

This assessment evaluates your understanding of Unity's NavMesh system, including navigation baking, NavMeshAgents, pathfinding, obstacles, and scripting interactions with NavMesh to create AI-driven movement.

An assessment to test the correct usage of Unity's events and lifecycle functions in the context of the game "Clash Royale"

<p>- This assessment gives us an idea of your C++ skills<br>- Companies that evaluate this skillset in interviews: <strong>Rockstar Games, Ubisoft, Mayhem Studios, etc.</strong><br>- 10 Multiple Choice questions<br>- Average time to complete: <strong>7 to 10 minutes</strong></p>

<p>Let's assess your Data Structures! 📋<br>&nbsp;- This assessment gives us an idea of your problem solving skills using DSA<br>- Companies that evaluate this skillset in interviews: MPL, LILA Games, GSN Games, etc.<br>- 10 Multiple Choice questions<br>- Average time to complete: 7 to 10 minutes</p>

<p>Let's assess your Programming Patterns! 📋</p><p>- This assessment gives us an idea of your code architecture skills<br>- Companies that evaluate this skillset in interviews: Dirtcube, PTW, Playshifu, etc.<br>- 10 Multiple Choice<strong> </strong>questions<br>- Average time to complete: 7<strong> </strong>to 10 minutes</p>

<p>Let's assess your OOPs skills! 📋</p><p>- This assessment gives us an idea of your fundamental OOPS concepts and Solid principles skills<br>- Companies that evaluate this skillset in interviews: Hitwicket, GSN Games, Tomo Club, etc.<br>- 10 Multiple Choice questions<br>- Average time to complete:<strong> </strong>7 to 10 minutes</p>

<p>Let's assess your Unity Skills 📋<br><br>- This assessment gives us an idea of your Unity fundamentals<br>- Companies you can apply for after taking this assessment: Godspeed games, Salvation Games, nCore Games, etc.<br>- 9 questions: 9 Multiple Choice<br>- Minimum score needed to qualify: 80%<br>- Average time to complete: 5 to 10 minutes</p>

Side

Character Concept Artist - Talent Pool

GoDaddy

Senior Business Analyst - Sales Enablement

pipa studios

Senior Product Designer

SoftSwiss

Senior Accountant - Senior

Automation QA Engineer - Senior

CRM Platform Manager – Senior

Manual QA Engineer - Middle

Moonton 

Asset Director

【RPG-D】Lead Story Planner (Hangzhou)

This listing is for Side's Talent Pool for Co-Development Roles, not an immediate open position. By applying, you express interest in being considered for future project-based opportunities contingent on client needs. Side maintains strong relationships with its talent pool, providing updates and information on upcoming opportunities, and carefully vets candidates for project needs.

Side is a global video game development and services provider, offering technical and creative solutions to many of the largest developers and studios around the world. Founded in Japan in 1994, Side has grown to become a global force in the video game industry, with over 40 studios in 15 countries worldwide and offices across North America, Europe, South America, and Asia. Our industry-leading services include game development, art production, localization, audio production, quality assurance, localization QA, player support, community management, and datasets. Help us bring stories to the world. Join Side's global team of passionate gamers and contribute to top-notch game development.

3D Character Artist - Talent Pool (LATAM)

Tech & VFX Artist - Talent Pool

Functionality Quality Assurance Tester (FQA) - Gaming

Software QA Engineer

Japonais/Japanese - Localisation de jeux | Localization QA Video Game Tester

English to Russian Translator

Japanese Operations Support Agent

Environment/Props Concept Artist - Talent Pool

Audio Designer - Talent Pool

<h3 class="text-xl md:text-2xl font-bold mt-8 mb-3 text-gray-800">Description</h3><p class="my-4 text-gray-700 leading-relaxed">Side is a global video game development and services provider, offering technical and creative solutions to many of the largest developers and studios around the world. Founded in Japan in 1994, Side has grown to become a global force in the video game industry, with over 40 studios in 15 countries worldwide and offices across North America, Europe, South America, and Asia.</p><p class="my-4 text-gray-700 leading-relaxed">Our industry-leading services include codev, art production, localization, audio production, quality assurance, localization QA, player support, community management, and datasets.</p><p class="my-4 text-gray-700 leading-relaxed">Help us bring stories to the world. Join Side's global team of passionate gamers and contribute to top-notch game development. Discover opportunities in Asia, North America, South America, Europe, and beyond.</p><p class="my-4 text-gray-700 leading-relaxed">Experience our side of life! For more information, visit <a href="https://www.side.inc/" target="_blank" rel="noopener noreferrer" class="text-blue-600 hover:text-blue-800 hover:underline transition-colors duration-200">www.side.inc</a></p><h3 class="text-xl md:text-2xl font-bold mt-8 mb-3 text-gray-800">About This Opportunity</h3><p class="my-4 text-gray-700 leading-relaxed"><em>Please note, this listing is not for an immediate open position but for our Talent Pool for Co-Development Roles.</em> By applying, you express interest in being considered for future project-based opportunities that align with your skills and experience.</p><p class="my-4 text-gray-700 leading-relaxed">These opportunities are contingent on client needs and project availability. As projects are initiated or client proposals are developed, we will review candidates from this pool and contact those who most closely align with the specific requirements of the co-development opportunity.</p><h3 class="text-xl md:text-2xl font-bold mt-8 mb-3 text-gray-800">How We Keep You Engaged</h3><p class="my-4 text-gray-700 leading-relaxed">We maintain strong relationships with our talent pool by providing regular updates, industry insights, and information on upcoming opportunities. We will check in to confirm your availability and interest.</p><p class="my-4 text-gray-700 leading-relaxed">Our hiring teams carefully vet and select top candidates who best meet project needs, ensuring that you are matched with the right opportunity when it becomes available.</p><h3 class="text-xl md:text-2xl font-bold mt-8 mb-3 text-gray-800">Next Steps</h3><p class="my-4 text-gray-700 leading-relaxed">If you are passionate about contributing to cutting-edge projects and want to be considered for future co-development opportunities, we encourage you to apply to this listing. While there may not be an immediate opening, your application ensures that you will be part of our ongoing talent review for upcoming roles.</p><h3 class="text-xl md:text-2xl font-bold mt-8 mb-3 text-gray-800">Key Responsibilities</h3><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Create detailed and visually striking character concept art, including clothing, accessories, and other character elements.</li><li class="leading-relaxed">Collaborate closely with art leads, 3D artists, and game designers to ensure character designs align with the project’s creative vision and gameplay requirements.</li><li class="leading-relaxed">Generate character designs across a variety of styles, from realistic to stylized, depending on the project needs.</li><li class="leading-relaxed">Develop iterations of character concepts, providing a range of options and exploring unique visual ideas.</li><li class="leading-relaxed">Maintain consistency and quality in design, ensuring characters fit cohesively into the game’s world.</li><li class="leading-relaxed">Provide detailed callouts and reference materials for 3D artists and animators to guide asset production.</li><li class="leading-relaxed">Participate in regular feedback sessions, iterating on designs based on team and client input.</li></ul><h3 class="text-xl md:text-2xl font-bold mt-8 mb-3 text-gray-800">Requirements</h3><p class="my-4 text-gray-700 leading-relaxed"><strong>Experience</strong>:</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">3+ years of professional experience as a Character Concept Artist in the video game industry.</li><li class="leading-relaxed">Demonstrated contribution to at least one shipped title on PC, console, mobile, or VR/AR.</li></ul><p class="my-4 text-gray-700 leading-relaxed"><strong>Technical Skills</strong>:</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Proficiency with industry-standard tools such as Photoshop, Procreate, or equivalent software.</li><li class="leading-relaxed">Strong understanding of anatomy, proportion, and character design principles.</li><li class="leading-relaxed">Ability to design characters with attention to personality, storytelling, and gameplay functionality.</li><li class="leading-relaxed">Knowledge of rendering techniques to create high-quality, polished concept art.</li></ul><p class="my-4 text-gray-700 leading-relaxed"><strong>Creative and Collaborative Skills</strong>:</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Ability to generate original and imaginative character concepts that push creative boundaries.</li><li class="leading-relaxed">Strong communication and teamwork abilities, with experience incorporating feedback into iterative designs.</li><li class="leading-relaxed">Attention to detail and a commitment to delivering exceptional character designs.</li></ul><p class="my-4 text-gray-700 leading-relaxed"><strong>Passion for Gaming</strong>: Genuine enthusiasm for character design in video games and an understanding of how design choices impact gameplay and storytelling.</p><h3 class="text-xl md:text-2xl font-bold mt-8 mb-3 text-gray-800">Preferred Skills</h3><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Familiarity with 3D modeling tools for block-out or rough sculpting.</li><li class="leading-relaxed">Experience designing characters for a range of genres, including fantasy, sci-fi, and realistic settings.</li><li class="leading-relaxed">Background in visual storytelling or narrative illustration.</li><li class="leading-relaxed">Knowledge of current trends and techniques in concept art and game development.</li></ul><p class="my-4 text-gray-700 leading-relaxed"><em>Note: Specific requirements may vary depending on project needs.</em></p>

Follow Mayank - career advice

Follow Mayank Grover for game development resources

Free | C# | Midnight Pizza Fight

Free | C++ | Lethos

Free | C++ | Pong

Follow Abhishek - tech content

Follow Abhishek Smith for game programming content

This assessment gives us an idea of your C++ skills- Companies that evaluate this skillset in interviews: Rockstar Games, Ubisoft, Mayhem Studios, etc.- 10 Multiple Choice questions- Average time to complete: 7 to 10 minutes

Let's assess your Data Structures! - This assessment gives us an idea of your problem solving skills using DSA- Companies that evaluate this skillset in interviews: MPL, LILA Games, GSN Games, etc.- 10 Multiple Choice questions

Data Structures and Algorithms

Let's assess your Programming Patterns! ????- This assessment gives us an idea of your code architecture skills- Companies that evaluate this skillset in interviews: Dirtcube, PTW, Playshifu, etc.

Programming Patterns

Let's assess your OOPs skills! ????- This assessment gives us an idea of your fundamental OOPS concepts and Solid principles skills- Companies that evaluate this skillset in interviews: Hitwicket, GSN Games, Tomo Club, etc.

Object Oriented Programming

Let's assess your Unity Skills ????- This assessment gives us an idea of your Unity fundamentals- Companies you can apply for after taking this assessment: Godspeed games, Salvation Games, nCore Games, etc.