Generative AI Testing & Prompt Engineer

1 Month ago • 3-5 Years • Testing

Job Summary

Job Description

Synechron is looking for a Generative AI Testing & Prompt Engineer to join their technology team. The role involves designing and executing prompt strategies, developing automation workflows, and evaluating AI outputs for quality, bias, and reliability. Responsibilities include refining prompts for Generative AI models (NLP and multimodal), designing testing strategies to assess performance and ethical considerations, and automating testing workflows. The engineer will analyze AI outputs, collaborate with AI data scientists and developers for model tuning, document processes, and stay updated on AI advancements. This position requires expertise in Python, shell scripting, Git, and AI evaluation tools, with preferred experience in AI/ML frameworks and CI/CD tools. The goal is to enhance AI performance, safety, and ethical standards.
Must have:
  • Develop and refine prompts for Generative AI models
  • Design comprehensive testing strategies
  • Automate testing workflows
  • Analyze and interpret AI outputs
  • Collaborate with AI data scientists and developers
  • Document prompt scripts, testing procedures, and workflows
  • Python (advanced scripting and automation)
  • Version control (Git)
  • Generative AI evaluation tools
  • Scripting for automation
  • Reporting tools
Good to have:
  • Experience with AI/ML frameworks (TensorFlow, PyTorch)
  • CI/CD automation tools
  • Data analysis tools and platforms
  • Bash or shell scripting
  • Familiarity with cloud platforms (AWS, Azure, GCP)
  • Data labeling and quality control

Job Details

Job Summary
Synechron is seeking a proactive and skilled Generative AI Testing & Prompt Engineer to join our innovative technology team. In this role, you will design and execute prompt strategies, develop automation workflows, and evaluate AI outputs to ensure high-quality, unbiased, and reliable generative models. Your expertise will help improve AI performance, safety, and ethical standards, enabling the organization to leverage cutting-edge AI capabilities effectively and responsibly.

Software Requirements

  • Required:
    • Python (including libraries such as NumPy, Pandas, and testing frameworks)
    • Bash or shell scripting for automation workflows
    • Version control tools (Git)
    • AI evaluation metrics tools and frameworks for measuring quality, bias, and safety
  • Preferred:
    • Experience with AI/ML frameworks such as TensorFlow or PyTorch
    • CI/CD automation tools for seamless testing and deployment workflows
    • Data analysis tools and platforms for metrics development

Overall Responsibilities

  • Develop and refine prompts to evaluate the capabilities, accuracy, creativity, and safety of Generative AI models, including NLP and multimodal models.
  • Design comprehensive testing strategies across diverse scenarios using prompt variations to assess performance, bias, and ethical considerations.
  • Automate testing workflows to streamline data collection, prompt execution, output analysis, and reporting.
  • Analyze and interpret AI outputs, providing insights for improvements and bias mitigation.
  • Collaborate closely with AI data scientists and developers to tune prompt techniques and enhance model behavior through automation and feedback.
  • Document prompt scripts, testing procedures, automation workflows, and test results for transparency and reproducibility.
  • Continuously refine prompt engineering and testing methodologies based on latest AI advancements and industry best practices.
  • Stay informed about new trends, tools, and ethical considerations related to generative AI, prompt engineering, and automation.

Technical Skills (By Category)

  • Programming Languages:
    • Required: Python (advanced scripting and automation capabilities)
    • Preferred: Bash, shell scripting, or other automation languages
  • Databases/Data Management:
    • Basic understanding of data storage and querying for managing testing datasets
    • Experience with data labeling and quality control processes
  • Cloud Technologies:
    • Preferred: Basic familiarity with cloud platforms (AWS, Azure, GCP) for AI deployment and testing
  • Frameworks and Libraries:
    • Required: Generative AI evaluation tools, scripting frameworks for automation, prompt management tools
    • Preferred: TensorFlow, PyTorch, or similar AI/ML libraries
  • Development Tools and Methodologies:
    • Required: Version control (Git), scripting for automation, reporting tools
    • Preferred: CI/CD pipelines for automated testing and deployment in cloud or on-prem environments
  • Security Protocols:
    • Awareness of data privacy, security standards, and bias mitigation techniques

Experience Requirements

  • 3-5 years of professional experience in testing generative AI models, including prompt writing and optimization.
  • Proven experience designing prompt strategies for NLP and multimodal models.
  • Hands-on experience automating evaluation workflows for AI outputs, bias detection, and safety assessment.
  • Familiarity with model evaluation metrics, including relevance, coherence, bias, and safety.
  • Industry experience in AI development, testing, or related research preferred; alternative experience with related automation and data analysis roles may be considered.

Day-to-Day Activities

  • Develop, test, and optimize user prompts for AI model evaluations across multiple scenarios.
  • Automate processes for prompt execution, data collection, and output assessment.
  • Monitor and analyze AI model responses for quality, relevance, bias, and ethical compliance.
  • Collaborate with data scientists, AI developers, and product teams to implement prompt tuning and automation improvements.
  • Document testing strategies, automation scripts, and test outcomes thoroughly.
  • Provide actionable feedback to improve model behavior and address bias or safety issues.
  • Review industry trends and incorporate new testing tools or methodologies to advance testing capabilities.

Qualifications

  • Bachelor’s degree in Computer Science, Data Science, AI, or related field.
  • Relevant certifications in AI, data analysis, or automation practices are a plus.
  • Demonstrated commitment to continuous learning of AI ethics, bias mitigation, and prompt engineering.

Professional Competencies

  • Critical thinking and analytical skills to interpret complex AI outputs and identify issues.
  • Problem-solving abilities to develop effective prompts, troubleshoot automation workflows, and improve testing strategies.
  • Strong communication skills for documenting processes and collaborating with multidisciplinary teams.
  • Ability to adapt rapidly to evolving AI technologies and industry standards.
  • Interpersonal skills for stakeholder engagement and feedback provision.
  • Effective time management to handle multiple test scenarios and documentation tasks efficiently.

S​YNECHRON’S DIVERSITY & INCLUSION STATEMENT
 

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.


All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.

Candidate Application Notice

Similar Jobs

Ion - Product Management Analyst

Ion

Pune, Maharashtra, India (On-Site)
10 Months ago
Tencent - Senior Technical Artist

Tencent

(On-Site)
4 Months ago
Aledade - Senior Analyst, Office of the President, HQ

Aledade

Bethesda, Maryland, United States (Hybrid)
3 Weeks ago
Varonis  - Corporate Sales Engineer

Varonis

United States (On-Site)
3 Weeks ago
Reltio - Intern - Software Engineering

Reltio

Lisbon, Lisbon, Portugal (On-Site)
2 Months ago
luxsoft - Senior Mobile Tester

luxsoft

India (Remote)
1 Month ago
HCL Tech - Sr tester - embedded c, c++

HCL Tech

California, United States (On-Site)
2 Months ago
kaizen gaming  - Senior Software Engineer in Test

kaizen gaming

Sofia, Sofia City Province, Bulgaria (Hybrid)
1 Month ago
endava - Senior Tester

endava

Buenos Aires, Buenos Aires, Argentina (On-Site)
2 Months ago
Gunzilla - Manual QA Tester

Gunzilla

Kyiv, Kyiv City, Ukraine (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Nasdaq - Java Developer

Nasdaq

Mexico City, Mexico (On-Site)
1 Year ago
Barracuda - Manager, Technical Support

Barracuda

Bengaluru, Karnataka, India (On-Site)
4 Months ago
NXP - 2026 Intern - Product Master Data Maintenance

NXP

Tianjin, Tianjin, China (On-Site)
3 Weeks ago
endava - Senior Project Manager

endava

Osijek, Osijek-Baranja County, Croatia (On-Site)
1 Month ago
Plug power - Field Service Technician

Plug power

Montgomery, New York, United States (On-Site)
1 Year ago
Accenture - Candidate Source & Screen Senior Analyst

Accenture

Mumbai, Maharashtra, India (On-Site)
2 Months ago
hogarth - Packaging Manager

hogarth

Sunnyvale, California, United States (Hybrid)
2 Months ago
USE Insider - Commercial Director-Korea

USE Insider

Seoul, South Korea (Hybrid)
2 Weeks ago
Glean - Quality Assurance Manager

Glean

Bengaluru, Karnataka, India (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Bengaluru, Karnataka, India

FICO - Customer Support Engineer I

FICO

Bengaluru, Karnataka, India (On-Site)
1 Month ago
hogarth - Senior Copywriter

hogarth

Gurugram, India (On-Site)
1 Month ago
Tekion Corp - Software Engineering Architect

Tekion Corp

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Precisly - Digital Archiving Solutions

Precisly

India (On-Site)
1 Month ago
Toast - Senior Backend Software Engineer

Toast

Chennai, Tamil Nadu, India (Hybrid)
1 Month ago
Brillio - Senior Architect, Cloud Engineering

Brillio

Bengaluru, Karnataka, India (Hybrid)
6 Months ago
zeta - Vice President, Product & Engineering

zeta

Bengaluru, Karnataka, India (On-Site)
4 Months ago
AiDash - Senior Product Designer

AiDash

Bengaluru, Karnataka, India (Hybrid)
1 Month ago
Interactive Brokers - Senior Software Development Engineer in Test

Interactive Brokers

Mumbai, Maharashtra, India (Hybrid)
2 Months ago
Enphase Energy - Solargraf Sr. QA Engineer

Enphase Energy

Bengaluru, Karnataka, India (On-Site)
8 Months ago

Get notifed when new similar jobs are uploaded

Testing Jobs

Magna International - Sr. Penetration Test Engineer

Magna International

Bengaluru, Karnataka, India (On-Site)
10 Months ago
Haven Studios  Inc  - Generalist Programmer (Automated Testing)

Haven Studios Inc

Montreal, Quebec, Canada (Hybrid)
1 Month ago
Scout - Engineer, Safety DVP & Public Domain Testing

Scout

Novi, Michigan, United States (On-Site)
1 Month ago
Power Integrations - Test Development Engineer

Power Integrations

Penang, Malaysia (On-Site)
7 Months ago
Lionbridge Games - Senior Video Game Tester

Lionbridge Games

Brossard, Quebec, Canada (Remote)
2 Weeks ago
Unada labs - Manual Tester

Unada labs

Ahmedabad, Gujarat, India (On-Site)
7 Months ago
Universally Speaking - Traditional Chinese Games Tester

Universally Speaking

Liverpool, England, United Kingdom (On-Site)
4 Months ago
HCL Tech - Senior Automation Tester with selenium, Java

HCL Tech

Illinois, United States (On-Site)
2 Months ago
Universally Speaking - Simplified Chinese Games Tester

Universally Speaking

England, United Kingdom (On-Site)
4 Months ago
Capgemini - Data Testing

Capgemini

Mumbai, Maharashtra, India (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

About The Company

At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and optimization strategies spanend-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy financial services and technology firms. Through research and development initiatives in ourFinLabswe develop solutions for modernization, from Artificial Intelligence and Blockchain to Data Science models, Digital Underwriting, mobile-first applications and more.Over the last20+ years, our company has been honored with multiple employer awards, recognizing our commitment to our talented teams. With top clients to boast about, Synechron has a global workforce of14,500+,and has58offices in 21 countrieswithin key global markets.For more information on the company, please visit ourwebsiteorLinkedIncommunity.

Bengaluru, Karnataka, India (On-Site)

Pune, Maharashtra, India (On-Site)

Jersey City, New Jersey, United States (On-Site)

New York, United States (On-Site)

Pittsburgh, Pennsylvania, United States (On-Site)

View All Jobs

Get notified when new jobs are added by Synechron

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug