Test Automation Senior Specialist (AI Testing)

Your role and responsibilities:

We are seeking a highly skilled Test Automation Senior Specialist to join our AI product development team. The ideal candidate will have expertise in designing and implementing automated testing frameworks for complex AI and machine learning systems. 

 Furthermore, you will: 

• Develop and maintain comprehensive automated test suites for generative AI models, including language models, and multimodal systems.
• Create innovative testing strategies to validate the quality, coherence, and safety of AI-generated content.
• Design and implement automation frameworks to test generative AI products across multiple platforms and use cases.
• Collaborate closely with AI researchers, data scientists, and product developers to understand product requirements and unique testing challenges of generative systems.
•  Develop tools and scripts to generate diverse test data sets and simulate real-world generative AI scenarios.
• Implement automated tests for prompt engineering, fine-tuning processes, and model versioning.
• Continuously improve testing methodologies to keep pace with rapidly evolving generative AI technologies.
• Analyze test results and provide detailed reports on model performance, potential biases, and areas for improvement.

We expect you to have :

• Bachelor's or Master's degree in Computer Science, AI/ML, or related technical field.
• Minimum 5+ years of experience in software test automation, with at least 2 years focused on AI/ML systems, preferably generative AI.
• Strong programming skills in Python.

• Expertise in test automation tools and frameworks suitable for AI testing tools/frameworks like RAGAS, Deep eval, TrueLens, AWS Bedrock Evaluator, pytest, Robot Framework.
• Hands-on experience/knowledge of AI frameworks like Langchain or LlamaIndex.

• In-depth understanding of natural language processing, computer vision, and generative AI concepts.
•  Experience with large language models (e.g., GPT family, BERT).
• Familiarity with prompt engineering and techniques for evaluating generative AI outputs.
• Knowledge of ethical AI principles and experience in testing for bias and fairness in AI systems.

Come as You Are

Nasdaq is an equal opportunity employer. We positively encourage applications from suitably qualified and eligible candidates regardless of age, color, disability, national origin, ancestry, race, religion, gender, sexual orientation, gender identity and/or expression, veteran status, genetic information, or any other status protected by applicable law.

We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request an accommodation.