Senior Software Quality Engineer

Senior Software Engineer - II, QE - ML/AI

We are a cloud-native SaaS data analytics platform that solves complex observability and security problems. Customers choose our product because it allows them to easily monitor, optimize, and secure their applications, systems, and infrastructures. Our microservices architecture hosted on AWS ingests petabytes of data daily across many geographic regions. Millions of queries a day analyze hundreds of petabytes of data.

What can you expect to do?

We are looking for a dedicated QA Engineer(AI/ML) to ensure the integrity and performance of our ML models through rigorous testing and validation. The ML-QE will be pivotal in utilizing datasets curated by our Data Engineers, incorporating them into a comprehensive testing and automation framework to ensure the precision of data models. This vital role requires a deep understanding of data quality, consistency, and the ability to anticipate data needs for robust testing of model solutions.

Responsibilities

• Design comprehensive test plans that rigorously assess the accuracy, reliability, and robustness of machine learning algorithms.
• Develop, expand, and maintain an automated testing framework tailored for evaluating the performance and accuracy of machine learning models
• Collaborate with ML Engineers and Product Managers to deeply understand the user experience and product context of ML product features under test.
• Work closely with ML Engineers to understand the structure, characteristics, and intended use of the datasets created for machine learning tasks.
• Analyze test results meticulously, identifying discrepancies, diagnosing potential causes, and recommending corrective measures.
• Establish benchmarks and metrics for model performance and ensure that machine learning outputs meet pre-defined quality standards.
• Integrate quality assurance methodologies into the continuous integration and deployment pipeline (CI/CD), ensuring consistent delivery of high-quality machine learning applications.
• Continuously monitor model performance post-deployment to detect and resolve issues through systematic re-testing and persistent optimization.
• Collaborate with ML Engineers to develop in-product (“live”) quality monitoring via mechanisms such as explicit feedback, user behavior tracking, or post-hoc annotation.

Requirements

• Having an experience of 6+ years in testing around ML models
• Bachelor’s or master’s degree in Data Science, Computer Science, Statistics, Mathematics, or a related field.
• Good understanding of machine learning concepts and model training processes
• Experience with data visualization tools and techniques.
• Demonstrable skills in SQL and working knowledge of database technologies.
• Solid understanding of statistical methods and their application in validating and testing ML models.
• Familiarity with version control systems like Git, and CI/CD tools such as Jenkins or Travis CI.
• Strong analytical and critical thinking skills with a problem-solving mindset.

Preferred Skills

• Previous experience collaborating with data science teams and leveraging their datasets in a testing environment.
• Experience with Docker, Kubernetes, or similar container and orchestration platforms is advantageous.
• Exceptional attention to detail and a passion for achieving data accuracy through rigorous testing.
• Experience with modern DevOps monitoring and deployment/rollout technologies.

If interested, share your resume at Shreya@hirojet.com