QA Engineer - AI/ ML

6 Hours ago • 4-6 Years

Job Summary

Job Description

As a QA Engineer, you will ensure the quality and reliability of software applications, focusing on those that incorporate AI and ML components. You will design, implement, and execute automated tests, particularly utilizing Playwright. Responsibilities include developing test plans, test cases, and test scripts for chatbot interactions and report generation. You'll also define test strategies for LLM-based chatbot validation, ensuring comprehensive functional, regression, performance, and security testing. Moreover, you'll write automation scripts for conversational flows, user interactions, and report generation using Playwright, plus implement API testing for LLM interactions, backend workflows, and data retrieval. You'll also be involved in maintaining an automated regression suite to validate updates, working with developers and DevOps teams to integrate tests into CI/CD pipelines, ensuring the accuracy of generated reports, and contributing to QA documentation.
Must have:
  • Develop test plans, test cases, and test scripts for chatbot interactions.
  • Define test strategies for LLM-based chatbot validation.
  • Write automation scripts using Playwright.
  • Implement API testing (Postman, RestAssured, or similar).
  • Maintain automated regression suite for updates and retraining.
Good to have:
  • Conduct load and stress testing to measure chatbot's response time.
  • Identify and optimize latency issues in chatbot responses.
  • Use tools like JMeter, Locust, or k6 for performance analysis.
  • Validate chatbot UI/UX across web and mobile platforms.
  • Conduct usability testing to ensure smooth conversations.
Perks:
  • A dynamic and creative work environment.
  • Opportunities for professional growth and development.
  • A collaborative culture that values innovation and quality.
  • Competitive salary and benefits.
  • Travel sponsorship for international technical conferences.
  • Craftsmanship Culture: Learning is integral part of our culture.

Job Details

Company Description

BetaCraft is a technology-driven software company that provides solutions and outsourcing services to companies around the globe. We are experts in web and mobile product development with over 10+ years of experience. Our mission is to instill great technology into upcoming businesses with the help of world-class processes and practices.

We have built our own Tech startups like Twist and AppSurfer. Doing so, we acquired a deep understanding of building a product from scratch. It helps us understand the product maker’s mindset and serve them with valuable insights into their product-building decisions.

We have also worked with industry leaders like Intel to deliver excellent solutions with cutting-edge technology.

We value passion and craftsmanship while writing software. Here are some perks of working at BetaCraft - 

  • A dynamic and creative work environment.
  • Opportunities for professional growth and development.
  • A collaborative culture that values innovation and quality.
  • Competitive salary and benefits.
  • Travel sponsorship for international technical conferences if your talk gets selected.
  • Craftsmanship Culture: We value and love learning, and it's an integral part of our culture. You will have the opportunity to learn from senior craftsmen and benefit from sponsored paid content to enhance your skills in writing better software.

Job Description

As a QA Engineer, you will play a crucial role in ensuring the quality and reliability of our software applications. You will be responsible for designing, implementing, and executing automated tests, focusing on applications that incorporate AI and ML components. Your expertise in Playwright will help enhance our testing framework, and you will work closely with developers and data scientists to deliver high-quality software solutions.

Responsibilities:

Test Strategy & Planning:

  • Develop test plans, test cases, and test scripts tailored for chatbot interactions and report generation.

  • Define test strategies for LLM-based chatbot validation and output verification.

  • Ensure comprehensive functional, regression, performance, and security testing.

Chatbot-Specific Testing:

  • Validate chatbot responses for accuracy, relevance, consistency, and hallucination detection.

  • Test prompt variations and evaluate LLM-generated responses across different user intents.

Automation & API Testing:

  • Write automation scripts for conversational flows, user interactions, and report generation using Playwright

  • Implement API testing (Postman, RestAssured, or similar) for LLM interactions, backend workflows, and data retrieval.

  • Automate test cases for prompt injection security, rate limits, and bot performance.

Performance & Load Testing (Good to have):

  • Conduct load and stress testing to measure the chatbot’s response time under concurrent usage.

  • Identify and optimize latency issues in chatbot responses.

  • Use tools like JMeter, Locust, or k6 for performance analysis.

User Experience (UX) Testing:

  • Validate chatbot UI/UX across web and mobile platforms.

  • Conduct usability testing to ensure smooth and natural conversations.

  • Identify edge cases and error-handling improvements.

Regression & Continuous Testing:

  • Maintain an automated regression suite to validate updates and model retraining.

  • Work closely with developers and DevOps teams to integrate tests into CI/CD pipelines (Jenkins, GitHub Actions, GitLab CI/CD).

Data Integrity & Report Validation:

  • Ensure accuracy of generated reports (format, structure, and consistency).

Bug Reporting & Documentation:

  • Log and track bugs in Jira, Trello, or similar tools with detailed repro steps.

  • Provide test reports and work with developers to prioritize fixes.

  • Contribute to QA documentation for test cases, automation setups, and best practices.

Qualifications

  • Bachelor’s degree in Computer Science, Software Engineering, or a related field.

  • 4-6 years of experience in software quality assurance with a strong focus on automation testing.

  • Proven experience with Playwright or similar testing frameworks (e.g., Selenium, Cypress).

  • Solid understanding of AI and ML principles and their application in software.

  • Experience developing and implementing automated tests for web applications.

  • Strong programming skills in languages such as JavaScript, Python, or Java.

  • Familiarity with API testing and tools such as Postman or REST Assured.

  • Proficient in version control systems such as Git.

  • Strong analytical and troubleshooting skills.

  • Excellent communication and collaboration abilities.

Additional Information

What We Offer:

  • Opportunity to work on cutting-edge projects in a fast-paced and innovative environment.
  • Collaborative and inclusive work culture.
  • Competitive compensation package and benefits.
  • Professional growth opportunities and skill development.

Similar Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Skill Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Jobs in Pune, Maharashtra, India

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!