Software Engineering SMTS (Salesforce AI Quality Engineer)

We are seeking an experienced Quality Engineer for Salesforce Agentforce to join our Digital Success Engineering team. The ideal candidate should have a strong background in AI, retrieval-augmented generation (RAG), and Agentic technologies testing, along with expertise in Salesforce and programming languages like APEX, Python or Java. A passion for building robust automated testing solutions for Agentic user experience and application programming interfaces (APIs) is essential.

In this role, you will develop automated quality assurance practices for scalable customer-facing and internal Agentforce implementations. Proficiency in testing framework, JavaScript, modern UI frameworks, WebDriver, Selenium, and API testing is required. You will create functional and automation tests for Agent UI and APIs while collaborating with team members to enhance existing code and frameworks. Experience with Jenkins or other CI/CD tools is a plus.

This role is crucial in redefining how we enable innovative experiences driven by Agentforce. You will key to the evolution of Customer Success in the realm of Agentforce.

The Team

Our Digital Success Engineering team is an interdisciplinary mix of distributed software engineers, architects and engineering managers working collaboratively to build Unified Experiences for our Trailblazers. As customer zero, we drive innovation by harnessing Salesforce technology to create and easy and expert self-service experience that fuels Trailblazer Success.

The Role

You will develop and implement a quality engineering, testing strategy, testing framework, managing manual and automated test cases to ensure seamless human-agent experiences with Salesforce Agentforce and DataCloud. Your responsibilities include defining and executing a comprehensive testing plan and collaborating with Product Managers, Developers, and cross-functional teams to deliver exceptional Customer Success experiences.

PRIMARY RESPONSIBILITIES

• Quality Engineering Strategy & Leadership: Define, develop, and execute a comprehensive quality engineering and testing strategy specifically for Salesforce Agentforce, Data Cloud, and other relevant Salesforce platform capabilities.
• Test Automation Expertise: Design, develop, and maintain robust automated test scripts for Agent UI, Experience Site UI, and APIs using industry-standard tools and frameworks such as Selenium, WebDriver, JavaScript frameworks, and API testing tools.
• AI & Agentforce Testing: Develop specialized testing strategies and test cases for AI-powered features, including Retrieval-Augmented Generation (RAG), Agentic workflows, single turn / multi Turn conversation, Natural Language Processing (NLP), and Large Language Models (LLMs). Use AI and RAG testing metrics to define and measure testing outcome.
• Data-Driven Quality: Build and maintain datasets for training, fine-tuning, and evaluating the performance and accuracy of LLMs and other AI models. Implement data-driven approaches to measure and improve the quality of AI features.
• Functional & Salesforce Testing: Design and execute functional test plans and test cases tailored to Salesforce applications, including customizations using Flows, Triggers, Apex Jobs, and more. Possess a strong understanding of Salesforce best practices and governor limits.
• Continuous Integration & Continuous Delivery (CI/CD): Build and maintain automated quality control pipelines using Jenkins or other CI/CD tools to ensure seamless integration and deployment of high-quality software.
• Defect Management: Effectively identify, document, track, and manage defects to ensure timely resolution and contribute to root cause analysis.
• Collaboration & Communication: Partner effectively with Product Managers, Developers, UX Designers, and other stakeholders to understand requirements, provide timely feedback, communicate testing progress, and collaborate on solutions.
• Best Practices & Standards: Establish and champion quality standards, methodologies, and best practices within the team and across the organization.
• Performance Optimization: Continuously assess the performance of Agentforce implementations , identify potential bottlenecks, and recommend and implement proactive measures for optimization.
• Technical Expertise: Stay up-to-date with the latest Salesforce releases, AI advancements, and emerging testing trends.
• Stakeholder Management: Communicate testing progress, risks, and mitigation strategies effectively to various stakeholders, including both technical and non-technical audiences.
• Agile Participation: Actively participate in Agile development methodologies, contributing to sprint planning, daily stand-ups, and retrospectives.
• Problem-Solving & Analytical Skills: Demonstrate strong analytical and problem-solving skills to identify and resolve complex quality issues effectively.

Professional Experience/Skills Required:

• Overall 7+ years of experience in Software Quality Engineering.
• 2+ years of hands-on experience with the Salesforce platform, including configuration and development across various Salesforce Customer 360 products (e.g., Sales Cloud, Service Cloud, Data Cloud, Einstein ).
• 3+ years of proven experience in testing Machine Learning (ML), Artificial Intelligence (AI) systems, and/or services.
• Strong proficiency in at least one programming language such as Python or Java, particularly relevant for test automation and AI/ML testing.
• Experience with Large Language Models (LLMs) and prompt engineering testing frameworks.
• Solid understanding of Retrieval-Augmented Generation (RAG) and Agentic technologies.
• Demonstrated experience in building and maintaining robust automated testing frameworks for UI and APIs.
• Proficiency in JavaScript and modern UI frameworks (e.g., React, Angular, Vue.js) for UI test automation.
• Experience with WebDriver and Selenium for browser automation.
• Strong analytical mindset with experience in using a data-driven approach to measure and improve the quality of complex ML/AI products.
• Proven experience with Salesforce DevOps, CI/CD processes, and automation tools.
• Deep understanding of Salesforce architecture, functionalities, and various clouds.
• Strong understanding of AI, machine learning, and natural language processing principles.
• Experience with data management principles and Customer Data Platforms (CDPs) or Data Lakes.
• Familiarity with development tools like SLDS, XML, HTML, SQL, JavaScript, JSON, and CSS.
• Proven ability to work effectively within cross-functional teams to achieve common goals and business objectives.
• Experience with Agile development or Scrum project management methodologies and tools.
• Solid understanding of software engineering principles, key design patterns, and best practices.
• Strategic thinker with the ability to see the big picture, innovate, and adapt to constant change.
• Ability to work under pressure, highly adaptable, and well-organized.
• Highly effective written and verbal communication skills with audiences across all levels of the organization.
• Strong organizational skills with the ability to establish and manage priorities in a complex and fast-paced environment.
• Strong knowledge of data management concepts and architectures.
• Bachelor's or Master's degree in Computer Science, Software Engineering, or equivalent experience.