QA Engineer - AI/ ML

1 Day ago • 4-6 Years

Job Summary

Job Description

As a QA Engineer, you will ensure the quality and reliability of software applications, focusing on those that incorporate AI and ML components. You will design, implement, and execute automated tests, particularly utilizing Playwright. Responsibilities include developing test plans, test cases, and test scripts for chatbot interactions and report generation. You'll also define test strategies for LLM-based chatbot validation, ensuring comprehensive functional, regression, performance, and security testing. Moreover, you'll write automation scripts for conversational flows, user interactions, and report generation using Playwright, plus implement API testing for LLM interactions, backend workflows, and data retrieval. You'll also be involved in maintaining an automated regression suite to validate updates, working with developers and DevOps teams to integrate tests into CI/CD pipelines, ensuring the accuracy of generated reports, and contributing to QA documentation.
Must have:
  • Develop test plans, test cases, and test scripts for chatbot interactions.
  • Define test strategies for LLM-based chatbot validation.
  • Write automation scripts using Playwright.
  • Implement API testing (Postman, RestAssured, or similar).
  • Maintain automated regression suite for updates and retraining.
Good to have:
  • Conduct load and stress testing to measure chatbot's response time.
  • Identify and optimize latency issues in chatbot responses.
  • Use tools like JMeter, Locust, or k6 for performance analysis.
  • Validate chatbot UI/UX across web and mobile platforms.
  • Conduct usability testing to ensure smooth conversations.
Perks:
  • A dynamic and creative work environment.
  • Opportunities for professional growth and development.
  • A collaborative culture that values innovation and quality.
  • Competitive salary and benefits.
  • Travel sponsorship for international technical conferences.
  • Craftsmanship Culture: Learning is integral part of our culture.

Job Details

Company Description

BetaCraft is a technology-driven software company that provides solutions and outsourcing services to companies around the globe. We are experts in web and mobile product development with over 10+ years of experience. Our mission is to instill great technology into upcoming businesses with the help of world-class processes and practices.

We have built our own Tech startups like Twist and AppSurfer. Doing so, we acquired a deep understanding of building a product from scratch. It helps us understand the product maker’s mindset and serve them with valuable insights into their product-building decisions.

We have also worked with industry leaders like Intel to deliver excellent solutions with cutting-edge technology.

We value passion and craftsmanship while writing software. Here are some perks of working at BetaCraft - 

  • A dynamic and creative work environment.
  • Opportunities for professional growth and development.
  • A collaborative culture that values innovation and quality.
  • Competitive salary and benefits.
  • Travel sponsorship for international technical conferences if your talk gets selected.
  • Craftsmanship Culture: We value and love learning, and it's an integral part of our culture. You will have the opportunity to learn from senior craftsmen and benefit from sponsored paid content to enhance your skills in writing better software.

Job Description

As a QA Engineer, you will play a crucial role in ensuring the quality and reliability of our software applications. You will be responsible for designing, implementing, and executing automated tests, focusing on applications that incorporate AI and ML components. Your expertise in Playwright will help enhance our testing framework, and you will work closely with developers and data scientists to deliver high-quality software solutions.

Responsibilities:

Test Strategy & Planning:

  • Develop test plans, test cases, and test scripts tailored for chatbot interactions and report generation.

  • Define test strategies for LLM-based chatbot validation and output verification.

  • Ensure comprehensive functional, regression, performance, and security testing.

Chatbot-Specific Testing:

  • Validate chatbot responses for accuracy, relevance, consistency, and hallucination detection.

  • Test prompt variations and evaluate LLM-generated responses across different user intents.

Automation & API Testing:

  • Write automation scripts for conversational flows, user interactions, and report generation using Playwright

  • Implement API testing (Postman, RestAssured, or similar) for LLM interactions, backend workflows, and data retrieval.

  • Automate test cases for prompt injection security, rate limits, and bot performance.

Performance & Load Testing (Good to have):

  • Conduct load and stress testing to measure the chatbot’s response time under concurrent usage.

  • Identify and optimize latency issues in chatbot responses.

  • Use tools like JMeter, Locust, or k6 for performance analysis.

User Experience (UX) Testing:

  • Validate chatbot UI/UX across web and mobile platforms.

  • Conduct usability testing to ensure smooth and natural conversations.

  • Identify edge cases and error-handling improvements.

Regression & Continuous Testing:

  • Maintain an automated regression suite to validate updates and model retraining.

  • Work closely with developers and DevOps teams to integrate tests into CI/CD pipelines (Jenkins, GitHub Actions, GitLab CI/CD).

Data Integrity & Report Validation:

  • Ensure accuracy of generated reports (format, structure, and consistency).

Bug Reporting & Documentation:

  • Log and track bugs in Jira, Trello, or similar tools with detailed repro steps.

  • Provide test reports and work with developers to prioritize fixes.

  • Contribute to QA documentation for test cases, automation setups, and best practices.

Qualifications

  • Bachelor’s degree in Computer Science, Software Engineering, or a related field.

  • 4-6 years of experience in software quality assurance with a strong focus on automation testing.

  • Proven experience with Playwright or similar testing frameworks (e.g., Selenium, Cypress).

  • Solid understanding of AI and ML principles and their application in software.

  • Experience developing and implementing automated tests for web applications.

  • Strong programming skills in languages such as JavaScript, Python, or Java.

  • Familiarity with API testing and tools such as Postman or REST Assured.

  • Proficient in version control systems such as Git.

  • Strong analytical and troubleshooting skills.

  • Excellent communication and collaboration abilities.

Additional Information

What We Offer:

  • Opportunity to work on cutting-edge projects in a fast-paced and innovative environment.
  • Collaborative and inclusive work culture.
  • Competitive compensation package and benefits.
  • Professional growth opportunities and skill development.

Similar Jobs

Voodoo - Senior QA Engineer

Voodoo

Paris, Île-de-France, France (On-Site)
1 Week ago
Ubisoft - Level Artist

Ubisoft

Chengdu, Sichuan, China (On-Site)
4 Weeks ago
Image Engine - CFX TD - Lead

Image Engine

Vancouver, British Columbia, Canada (Hybrid)
1 Week ago
Coupa - Sr. Software Engineer in Test

Coupa

Pune, Maharashtra, India (Hybrid)
6 Months ago
Aristocrat Gaming - Director, Quality Assurance

Aristocrat Gaming

Sofia, Sofia City Province, Bulgaria (Hybrid)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Pentair - Quality Supervisor

Pentair

Dover, New Hampshire, United States (On-Site)
21 Hours ago
Thumbtack - Growth Marketing Associate

Thumbtack

Philippines (Remote)
2 Months ago
PlatinumGames - Visual Effects Artist

PlatinumGames

Chiyoda City, Tokyo, Japan (On-Site)
2 Weeks ago
Veeam Software - Devops Engineer

Veeam Software

Prague, Czechia (Hybrid)
8 Hours ago
Riot Games - Manager, Software Engineering - Teamfight Tactics, Gameplay

Riot Games

Los Angeles, California, United States (On-Site)
20 Hours ago
CD PROJEKT RED - Senior QA Engineer

CD PROJEKT RED

Boston, Massachusetts, United States (Hybrid)
1 Day ago
YouGov - Associate Research Manager

YouGov

Boulder, Colorado, United States (On-Site)
10 Hours ago
Sitetracker - Small Business Account Executive (SMB)

Sitetracker

Washington, District Of Columbia, United States (Remote)
6 Months ago
Google - Software Engineer, Engineering Productivity, Google Cast

Google

New Taipei City, Taiwan (On-Site)
2 Weeks ago
Aryaka - QA - Security (Threat Analyst)

Aryaka

Bengaluru, Karnataka, India (On-Site)
1 Year ago

Get notifed when new similar jobs are uploaded

Jobs in Pune, Maharashtra, India

Google - Staff UX Designer, Google Cloud

Google

Bengaluru, Karnataka, India (On-Site)
2 Weeks ago
PHINIA - IT Security Risk and Compliance Analyst

PHINIA

Bengaluru, Karnataka, India (On-Site)
1 Day ago
Likewise - Solarwinds Tools Expert

Likewise

Chennai, Tamil Nadu, India (On-Site)
1 Day ago
Zinnia - Senior Data Analyst (Global Ops Analytics)

Zinnia

Gurugram, Haryana, India (On-Site)
1 Day ago
Haleon - Expert Sales Executive

Haleon

Kota, Rajasthan, India (On-Site)
1 Day ago
Assystems - Manager / Senior Manager - Finance

Assystems

Chennai, Tamil Nadu, India (On-Site)
6 Months ago
KloudPortal  - Senior Engineer (Backend)

KloudPortal

Bengaluru, Karnataka, India (On-Site)
7 Months ago
Kaarlo Training & HR Solutions   - Video Editor - Trichy

Kaarlo Training & HR Solutions

Tiruchirappalli, Tamil Nadu, India (On-Site)
7 Months ago
Tesla - Delivery Operations Specialist

Tesla

Mumbai, Maharashtra, India (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!