Nousresearch

Machine Learning Engineer (Data & Evaluation Infrastructure)

Machine Learning Engineer (Training Infrastructure)

Research Scientist

General Counsel

We are seeking a Machine Learning Engineer (MLE) to manage our post-training evaluation pipeline. The role involves building and scaling evaluation processes to assess model capabilities across various tasks, pinpointing areas of failure, and driving improvements. Key responsibilities include identifying tasks for evaluation, creating or curating test cases and measurement methods, implementing evaluations through objective verification, LLM judging, reward modeling, or human evaluation. You will also be responsible for expanding coverage, deeply analyzing failure cases, identifying solutions, and developing scalable and accessible internal evaluation presentation methods, such as GUIs or Slurm scripts.

<div class="job-description"> We’re looking for an MLE to own our post-training evaluation pipeline. You’ll build and scale evals depth and breadth that measure model capabilities across diverse tasks, identify failure modes, and drive model improvements. <h2><b>Responsibilities:</b></h2> <ul>   <li aria-level="1">Identifying tasks for evaluation coverage</li>   <li aria-level="1">Creating, curating, or generating test cases and ways to measure these tasks</li>   <li aria-level="1">Implementing evaluation through objective output verification, LLM judge/reward modeling, human evaluation, or any tricks of the trade you may bring to the table</li>   <li aria-level="1">Adding coverage and diving deep into analyzing what’s really gone wrong in failure cases</li>   <li aria-level="1">Identifying ways to remedy failure cases</li>   <li aria-level="1">Developing ways to present and make the evals scalable and accessible internally (e.g. light GUIs, scalable Slurm scripts, etc for running the evals)</li> </ul> <h2><b>Qualifications:</b></h2> <ul>   <li aria-level="1">Strong experience with evaluation frameworks</li>   <li aria-level="1">Experience with both automated and human evaluation methodologies</li>   <li aria-level="1">Ability to build evaluation infrastructure from scratch and scale existing systems</li> </ul> <h2><b>Preferred:</b></h2> <ul>   <li aria-level="1">History of OSS contributions</li> </ul> </div>

thumbnail_Rockstar.webp

Rockstar.webp

Rockstar Jobs

thumbnail_EA.webp

EA.webp

EA Jobs

thumbnail_Playstation.webp

Playstation.webp

Playstation Jobs

small_Ubisoft.webp

thumbnail_Ubisoft.webp

Ubisoft.webp

Ubisoft Jobs

thumbnail_Epic Games.webp

Epic Games.webp

Epic Game Jobs

thumbnail_Riot Games.webp

Riot Games.webp

Riot Jobs

08cafc09dac48d655284704f063dc64e.webp

CDPR Jobs

80ad888d-4837-48d9-8b65-0ffdee910f35.webp

Krafton Jobs

thumbnail_a14feb4b393500244d0b8864bda07b2c.webp

a14feb4b393500244d0b8864bda07b2c.webp

Twitch Jobs

thumbnail_865c7fd0-1ffd-4000-80b6-1e69d6665257.webp

865c7fd0-1ffd-4000-80b6-1e69d6665257.webp

Larian Jobs

thumbnail_02_21_2024__11_00_53.webp

02_21_2024__11_00_53.webp

Wargaming Jobs

medium_naughty dog.webp

small_naughty dog.webp

thumbnail_naughty dog.webp

naughty dog.webp

Naughty Dog Jobs

large_netflix.webp

medium_netflix.webp

small_netflix.webp

thumbnail_netflix.webp

netflix.webp

Netflix Jobs

large_supercell.webp

medium_supercell.webp

small_supercell.webp

thumbnail_supercell.webp

supercell.webp

Supercell Jobs

zynga_logo.jpg

Zynga Jobs

Test your Unity Animations knowledge with this quiz! Explore concepts like Animator Controller, Blend Trees, and many more.

Test your Unity UI knowledge with this quiz! Explore essential concepts like Canvas, EventSystem and many more. 

Challenge yourself with this Unity Input System quiz! Test your skills in input mapping, device handling, and action events.

Take your Unity skills to the next level with Part 3 of this quiz, covering scripting, physics, rendering, and performance optimization!

Test your Unity expertise with this quiz, covering initialization to advanced rendering and physics!

Test your Unity knowledge with this comprehensive quiz, covering everything from MonoBehaviour basics to advanced physics, rendering, and optimization techniques across multiple difficulty levels!

Assess your knowledge of Unity's Rigidbody component and its advanced physics interactions with this in-depth quiz!

Test your understanding of Unity’s Physics, Movement, and MonoBehaviour with this insightful beginner-friendly quiz!

Dive into the fundamentals of open-world game development in Unity with this beginner-friendly quiz.

Challenge yourself with this beginner-friendly quiz on building an endless runner in Unity!

Test your beginner-level knowledge of Unity platformer development with this engaging quiz.

Test your basic knowledge of Unity and Unreal Engine with this beginner-friendly quiz on game development fundamentals.

This assessment evaluates your understanding of Unity's component-based architecture, how GameObjects and their components interact, the hierarchy system with nested GameObjects, and how to reference them in scripts.

This assessment evaluates your understanding of Unity's NavMesh system, including navigation baking, NavMeshAgents, pathfinding, obstacles, and scripting interactions with NavMesh to create AI-driven movement.

An assessment to test the correct usage of Unity's events and lifecycle functions in the context of the game "Clash Royale"

<p>- This assessment gives us an idea of your C++ skills<br>- Companies that evaluate this skillset in interviews: <strong>Rockstar Games, Ubisoft, Mayhem Studios, etc.</strong><br>- 10 Multiple Choice questions<br>- Average time to complete: <strong>7 to 10 minutes</strong></p>

<p>Let's assess your Data Structures! 📋<br>&nbsp;- This assessment gives us an idea of your problem solving skills using DSA<br>- Companies that evaluate this skillset in interviews: MPL, LILA Games, GSN Games, etc.<br>- 10 Multiple Choice questions<br>- Average time to complete: 7 to 10 minutes</p>

<p>Let's assess your Programming Patterns! 📋</p><p>- This assessment gives us an idea of your code architecture skills<br>- Companies that evaluate this skillset in interviews: Dirtcube, PTW, Playshifu, etc.<br>- 10 Multiple Choice<strong> </strong>questions<br>- Average time to complete: 7<strong> </strong>to 10 minutes</p>

<p>Let's assess your OOPs skills! 📋</p><p>- This assessment gives us an idea of your fundamental OOPS concepts and Solid principles skills<br>- Companies that evaluate this skillset in interviews: Hitwicket, GSN Games, Tomo Club, etc.<br>- 10 Multiple Choice questions<br>- Average time to complete:<strong> </strong>7 to 10 minutes</p>

<p>Let's assess your Unity Skills 📋<br><br>- This assessment gives us an idea of your Unity fundamentals<br>- Companies you can apply for after taking this assessment: Godspeed games, Salvation Games, nCore Games, etc.<br>- 9 questions: 9 Multiple Choice<br>- Minimum score needed to qualify: 80%<br>- Average time to complete: 5 to 10 minutes</p>

ness digital 

Data Quality Analyst - Data Roadmap Delivery (MDM Impl

HR Generalist

Lilt

Staff Backend Engineer

Highspot

Sr. Software Development Engineer

Nine

Sales Executive

Producer

Moloco

Machine Learning Engineer (전문연구요원)

Yahoo

Sr Software Engineer, Full Stack

Autodesk

Senior Software Developer

An Informatica MDM Data Quality Analyst at Ness Romania SRL focuses on ensuring data accuracy, consistency, and reliability within an organization's Master Data Management (MDM) system, leveraging Informatica tools. This role involves data profiling, cleansing, transformation, and monitoring data quality through the implementation and maintenance of data quality rules and scorecards. Collaboration with various teams, including data stewards, business analysts, and developers, is crucial for defining and enforcing data quality standards. You will work on products and platforms for innovative software companies, gaining knowledge and accelerating your career.

Product lead

Informatica MDM Developer - Data Roadmap Delivery (MDM Impl

Senior Full-Stack Software Engineer

Full stack (FE React as a primary)

UAT Lead Support Engineer plus Business Analyst - FISERV Database Migration

Snowflake Engineer - FISERV Database Migration

UAT Support Engineer - FISERV Database Migration

Senior Data Engineer

<h3 class="text-xl md:text-2xl font-bold mt-8 mb-3 text-gray-800">Description</h3><p class="my-4 text-gray-700 leading-relaxed"><em>Position at Ness Romania SRL</em></p><p class="my-4 text-gray-700 leading-relaxed">Job ID 6481</p><h4 class="text-lg md:text-xl font-semibold mt-6 mb-2 text-gray-800">Why Ness</h4><p class="my-4 text-gray-700 leading-relaxed">We know that people are our greatest asset. Our staff’s professionalism, innovation, teamwork, and dedication to excellence have helped us become one of the world’s leading technology companies. It is these qualities that are vital to our continued success. As a Ness employee, you will be working on products and platforms for some of the most innovative software companies in the world.</p><p class="my-4 text-gray-700 leading-relaxed">You’ll gain knowledge working alongside other highly skilled professionals that will help accelerate your career progression.</p><p class="my-4 text-gray-700 leading-relaxed">You’ll also benefit from an array of advantages like access to trainings and certifications, bonuses, and aids, socializing activities and attractive compensation.</p><h4 class="text-lg md:text-xl font-semibold mt-6 mb-2 text-gray-800">Requirements and responsibilities</h4><p class="my-4 text-gray-700 leading-relaxed">An Informatica MDM Data Quality Analyst focuses on ensuring data accuracy, consistency, and reliability within an organization's Master Data Management (MDM) system, leveraging Informatica tools. This role involves data profiling, cleansing, transformation, and monitoring data quality through the implementation and maintenance of data quality rules and scorecards. Collaboration with various teams, including data stewards, business analysts, and developers, is crucial for defining and enforcing data quality standards.</p><h4 class="text-lg md:text-xl font-semibold mt-6 mb-2 text-gray-800">What you’ll do</h4><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Data Profiling and Analysis:</li></ul><p class="my-4 text-gray-700 leading-relaxed">Analyzing data patterns, identifying anomalies, and assessing data quality using tools like Informatica Data Quality (IDQ).</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Data Cleansing and Transformation:</li></ul><p class="my-4 text-gray-700 leading-relaxed">Implementing processes to cleanse, standardize, and transform data within the MDM system, ensuring data integrity and consistency.</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Data Quality Rule Development:</li></ul><p class="my-4 text-gray-700 leading-relaxed">Designing, developing, and implementing data quality rules and workflows within IDQ to monitor and improve data quality.</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">MDM System Configuration:</li></ul><p class="my-4 text-gray-700 leading-relaxed">Configuring and maintaining the Informatica MDM SaaS platform, including data models, match and merge rules, and trust frameworks.</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Collaboration and Communication:</li></ul><p class="my-4 text-gray-700 leading-relaxed">Working closely with data stewards, business analysts, and other stakeholders to define data quality requirements, resolve data issues, and ensure alignment with business needs.</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Data Governance Support:</li></ul><p class="my-4 text-gray-700 leading-relaxed">Contributing to the development and enforcement of data governance policies and standards within the organization, ensuring data quality and compliance.</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Performance Monitoring and Reporting:</li></ul><p class="my-4 text-gray-700 leading-relaxed">Developing and maintaining data quality scorecards, dashboards, and reports to track key metrics and identify areas for improvement.</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Root Cause Analysis:</li></ul><p class="my-4 text-gray-700 leading-relaxed">Investigating data quality issues, identifying root causes, and implementing solutions to prevent recurrence.</p><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Data Migration and Enrichment:</li></ul><p class="my-4 text-gray-700 leading-relaxed">Supporting data migration and enrichment activities by ensuring the quality and consistency of data during these processes.</p><h4 class="text-lg md:text-xl font-semibold mt-6 mb-2 text-gray-800">What you’ll bring</h4><ul class="list-disc list-inside my-5 pl-5 space-y-2 text-gray-700"><li class="leading-relaxed">Strong understanding and experience with Informatica MDM and IDQ, including data profiling, cleansing, transformation, and rule development.</li><li class="leading-relaxed">Solid understanding of data quality principles, including accuracy, completeness, consistency, and validity.</li><li class="leading-relaxed">Experience in profiling and analyzing data to identify quality issues and trends.</li><li class="leading-relaxed">Proficiency in data cleansing and transformation techniques using tools like IDQ.</li><li class="leading-relaxed">Knowledge of SQL and data modeling concepts to support data analysis and MDM system configuration..</li></ul><h4 class="text-lg md:text-xl font-semibold mt-6 mb-2 text-gray-800">Not checking every single requirement?</h4><p class="my-4 text-gray-700 leading-relaxed">If this role sounds good to you, even if you don’t meet every single bullet point in the job description, we encourage you to apply anyway. For most of the candidates that applied, we found a role that was a very good fit with their skills.</p><p class="my-4 text-gray-700 leading-relaxed">Let’s meet and you may just be the right candidate for one of our roles.</p><p class="my-4 text-gray-700 leading-relaxed">At Ness Digital Engineering we are willing to build a work culture that is based on diversification, inclusion, and authenticity.</p>

Follow Mayank - career advice

Follow Mayank Grover for game development resources

Free | C# | Midnight Pizza Fight

Free | C++ | Lethos

Free | C++ | Pong

Follow Abhishek - tech content

Follow Abhishek Smith for game programming content

This assessment gives us an idea of your C++ skills- Companies that evaluate this skillset in interviews: Rockstar Games, Ubisoft, Mayhem Studios, etc.- 10 Multiple Choice questions- Average time to complete: 7 to 10 minutes

Let's assess your Data Structures! - This assessment gives us an idea of your problem solving skills using DSA- Companies that evaluate this skillset in interviews: MPL, LILA Games, GSN Games, etc.- 10 Multiple Choice questions

Data Structures and Algorithms

Let's assess your Programming Patterns! ????- This assessment gives us an idea of your code architecture skills- Companies that evaluate this skillset in interviews: Dirtcube, PTW, Playshifu, etc.

Programming Patterns

Let's assess your OOPs skills! ????- This assessment gives us an idea of your fundamental OOPS concepts and Solid principles skills- Companies that evaluate this skillset in interviews: Hitwicket, GSN Games, Tomo Club, etc.

Object Oriented Programming

Let's assess your Unity Skills ????- This assessment gives us an idea of your Unity fundamentals- Companies you can apply for after taking this assessment: Godspeed games, Salvation Games, nCore Games, etc.