Software Engineer (Testing)

8 Minutes ago • Upto 3 Years • Testing

Job Summary

Job Description

At ObviouslyAI, our vision is to turn every company into an AI company by providing access to on-demand data science talent. We empower data scientists with groundbreaking tools to deliver results quickly. We are a small, agile team focused on rapid iteration and work-life balance, with flexible hours. Backed by top US venture capital firms, this is an opportunity to join a fast-growing company with a significant mission. We seek a creative Backend & AI Agent Testing Engineer to ensure our AI systems and backend services are reliable, even in unpredictable scenarios.
Must have:
  • Write, debug, and maintain backend code in Python or JavaScript.
  • Implement APIs and ensure authentication workflows work.
  • Design and execute creative test strategies for AI agent behavior.
  • Evaluate AI agent outputs and prompts, contributing to LLM evaluation.
  • Write scripts and automation to test AI agents and backend workflows.
  • Build lightweight automation frameworks and develop test infrastructure.
  • Dive into new, unfamiliar apps and services quickly.
  • Get into the “user’s shoes” to anticipate edge cases.
  • Work hands-on with integrations and modern collaboration tools.
  • Test and validate backend workflows connecting user data across systems.
  • Collaborate closely with cross-functional teams in a startup environment.
  • Continuously learn new frameworks, tools, and approaches.
  • Able to write backend code, debug, and write test cases in Python or JavaScript.
  • Exposure to or experience testing and prompting AI agents, especially GenAI/LLM-based systems.
  • Comfortable writing backend scripts, automating tests, and creating testing frameworks.
  • Exposure to integrations, understanding of API interactions and authentication.
Good to have:
  • Familiarity with Selenium
  • HubSpot integration experience
  • Salesforce integration experience
  • Familiarity with Cursor
  • Familiarity with Windsurf
  • LLM evaluation frameworks
  • LLM metrics
  • MCPs
Perks:
  • Work at the intersection of backend engineering, AI, and creative testing.
  • A fast-moving, supportive startup culture that values experimentation and creativity.
  • Opportunity to work with cutting-edge AI systems, tools, and frameworks.
  • Learn and grow rapidly alongside a talented and collaborative team.

Job Details

Description

Data Science problems are everywhere, but the talent is not. At ObviouslyAI, our vision is to turn every company into an AI company. We do this by providing businesses with access to world-class, on-demand data science talent that helps them solve real business problems. On the back end, we empower data scientists with a set of internal groundbreaking tools to help them deliver results in minutes, not months.

We’re a small, scrappy group of people with a strong bent toward failing fast, a bias for action, and attention to detail. We’re focused on doing the best work of our lives and believe in having a healthy separation of work and life. We keep working hours flexible and are building a team with business teams located in San Francisco, CA and engineering teams located in Bangalore, India.

Obviously AI is backed by some of the top venture capital firms in the US, and you’ll be on the ground floor of a fast-growing company with a big mission.

About You

We’re looking for a creative and curious Backend & AI Agent Testing Engineer to join our team. You’ll work hands-on with our AI agents and backend services, writing code, debugging, scripting tests, and evaluating LLM prompts to ensure our systems behave reliably — even in unpredictable scenarios.

This is not a traditional backend or QA role. You’ll quickly learn new tools, explore unfamiliar apps, and design tests from a user’s perspective to help tame and improve our cutting-edge AI systems.

Responsibilities

  • Build and Maintain
  • Write, debug, and maintain backend code in Python or JavaScript, building test cases and backend scripts to support robust and experimental systems.
  • Implement APIs and ensure authentication workflows work as expected, with exposure to integrations.
  • AI Agent Testing & Prompt Evaluation
  • Design and execute creative test strategies for AI agent behavior, particularly around LLM-based (GenAI) agents, ensuring systems behave reliably despite their unpredictable nature.
  • Evaluate AI agent outputs and prompts, contributing to LLM evaluation and metrics using tools like Deepchecks.
  • Scripting & Automation
  • Write scripts and automation to test AI agents and backend workflows.
  • Build lightweight automation frameworks and develop or extend test infrastructure; familiarity with Selenium or similar is a plus.
  • Experimentation & Exploration
  • Dive into new, unfamiliar apps and services quickly — learning them and building tests as if you were an end-user.
  • Get into the “user’s shoes” to anticipate edge cases and potential failure modes.
  • Tame the “beast” — creatively managing and testing AI systems that may behave inconsistently.
  • Tools & Integrations
  • Work hands-on with integrations (e.g., HubSpot, Salesforce — bonus points for experience here) and modern collaboration tools like Cursor or Windsurf.
  • Test and validate backend workflows that connect names, emails, and other critical user data across systems.
  • Collaboration & Continuous Learning
  • Collaborate closely with cross-functional teams in a startup environment, embracing rapid experimentation and iteration.
  • Continuously learn new frameworks, tools, and approaches without handholding, demonstrating a strong growth mindset.

Requirements

  • Experience Level: ~0–3 years in backend development or testing, ideally in a startup or experimental role.
  • Backend Engineering Basics: Able to write backend code, debug, and write test cases in Python or JavaScript.
  • Testing Creativity: Not a traditional tester — you think creatively, experiment boldly, and approach testing like teaching a child.
  • AI Agent Experience: Exposure to or experience testing and prompting AI agents, especially GenAI/LLM-based systems.
  • Automation & Scripting: Comfortable writing backend scripts, automating tests, and creating testing frameworks.
  • Integrations & APIs: Exposure to integrations (bonus if HubSpot, Salesforce), understanding of API interactions and authentication.
  • Tools: Familiarity with tools like Cursor or Windsurf preferred.
  • Knowledge of modern automation frameworks (e.g., Selenium).
  • Experience in B2B product environments.
  • LLM Evaluation & Metrics: Bonus if you’ve worked with LLM evaluation frameworks, metrics, or MCPs.
  • Mindset:
  • Curious and fast learner — you’re comfortable diving into completely unknown tools or apps and figuring them out.
  • Super creative in designing tests beyond “clicking around” — you understand that testing AI systems means “taming the beast.”
  • Willing to experiment, work on ambiguous problems, and wear multiple hats.

Benefits

  • Work at the intersection of backend engineering, AI, and creative testing.
  • A fast-moving, supportive startup culture that values experimentation and creativity.
  • Opportunity to work with cutting-edge AI systems, tools, and frameworks.
  • Learn and grow rapidly alongside a talented and collaborative team.

Similar Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Skill Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Jobs in Brazil

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Testing Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

About The Company

Zams is an enterprise platform to build AI agents that automate back-office work. Backed by top Silicon Valley investors like B Capital Group, UTEC, TMV, Sequoia Scouts and Facebook. Trusted by hundreds of businesses worldwide.

San Francisco, California, United States (On-Site)

San Francisco, California, United States (Hybrid)

San Francisco, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by Obviously A

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug