QA Engineer - AI/ ML

BetaCraft is a technology-driven software company that provides solutions and outsourcing services to companies around the globe. We are experts in web and mobile product development with over 10+ years of experience. Our mission is to instill great technology into upcoming businesses with the help of world-class processes and practices.

We have built our own Tech startups like Twist and AppSurfer. Doing so, we acquired a deep understanding of building a product from scratch. It helps us understand the product maker’s mindset and serve them with valuable insights into their product-building decisions.

We have also worked with industry leaders like Intel to deliver excellent solutions with cutting-edge technology.

We value passion and craftsmanship while writing software. Here are some perks of working at BetaCraft - 

• A dynamic and creative work environment.
• Opportunities for professional growth and development.
• A collaborative culture that values innovation and quality.
• Competitive salary and benefits.
• Travel sponsorship for international technical conferences if your talk gets selected.
• Craftsmanship Culture: We value and love learning, and it's an integral part of our culture. You will have the opportunity to learn from senior craftsmen and benefit from sponsored paid content to enhance your skills in writing better software.

As a QA Engineer, you will play a crucial role in ensuring the quality and reliability of our software applications. You will be responsible for designing, implementing, and executing automated tests, focusing on applications that incorporate AI and ML components. Your expertise in Playwright will help enhance our testing framework, and you will work closely with developers and data scientists to deliver high-quality software solutions.

Responsibilities:

Test Strategy & Planning:

• Develop test plans, test cases, and test scripts tailored for chatbot interactions and report generation.

• Define test strategies for LLM-based chatbot validation and output verification.

• Ensure comprehensive functional, regression, performance, and security testing.

Chatbot-Specific Testing:

• Validate chatbot responses for accuracy, relevance, consistency, and hallucination detection.

• Test prompt variations and evaluate LLM-generated responses across different user intents.

Automation & API Testing:

• Write automation scripts for conversational flows, user interactions, and report generation using Playwright

• Implement API testing (Postman, RestAssured, or similar) for LLM interactions, backend workflows, and data retrieval.

• Automate test cases for prompt injection security, rate limits, and bot performance.

Performance & Load Testing (Good to have):

• Conduct load and stress testing to measure the chatbot’s response time under concurrent usage.

• Identify and optimize latency issues in chatbot responses.

• Use tools like JMeter, Locust, or k6 for performance analysis.

User Experience (UX) Testing:

• Validate chatbot UI/UX across web and mobile platforms.

• Conduct usability testing to ensure smooth and natural conversations.

• Identify edge cases and error-handling improvements.

Regression & Continuous Testing:

• Maintain an automated regression suite to validate updates and model retraining.

• Work closely with developers and DevOps teams to integrate tests into CI/CD pipelines (Jenkins, GitHub Actions, GitLab CI/CD).

Data Integrity & Report Validation:

• Ensure accuracy of generated reports (format, structure, and consistency).

Bug Reporting & Documentation:

• Log and track bugs in Jira, Trello, or similar tools with detailed repro steps.

• Provide test reports and work with developers to prioritize fixes.

• Contribute to QA documentation for test cases, automation setups, and best practices.

• Bachelor’s degree in Computer Science, Software Engineering, or a related field.

• 4-6 years of experience in software quality assurance with a strong focus on automation testing.

• Proven experience with Playwright or similar testing frameworks (e.g., Selenium, Cypress).

• Solid understanding of AI and ML principles and their application in software.

• Experience developing and implementing automated tests for web applications.

• Strong programming skills in languages such as JavaScript, Python, or Java.

• Familiarity with API testing and tools such as Postman or REST Assured.

• Proficient in version control systems such as Git.

• Strong analytical and troubleshooting skills.

• Excellent communication and collaboration abilities.

What We Offer:

• Opportunity to work on cutting-edge projects in a fast-paced and innovative environment.
• Collaborative and inclusive work culture.
• Competitive compensation package and benefits.
• Professional growth opportunities and skill development.