Evaluation Engineer (AI Assistant)

Evaluation is becoming a crucial part of modern AI development. The ability to measure, refine, and validate AI models is what separates successful AI-powered products from those that fail.At JetBrains, we are committed to building robust evaluation systems that enable the development of high-quality AI features. Many companies advocate for evaluation-driven development, and industry experts recognize evaluation as a key differentiator in AI. Evaluation is also essential for integrating CI/CD principles into LLM development.Despite its importance, evaluation is still an evolving field with many uncertainties and areas for optimization. We are looking for a talented and passionate software developer to join our team and help us build cutting-edge evaluation tooling for the IntelliJ Platform and beyond. The ideal candidate thrives in an environment where they can take initiative, work autonomously, and collaborate effectively with others### As part of our team, you will:* Design and develop evaluation tools to assess AI-powered features in JetBrains products.* Empower teams across the company by providing best-in-class evaluation tooling to improve AI models and workflows.* Explore and implement modern evaluation techniques, adapting them to the JetBrains ecosystem.* Build APIs that are intuitive, flexible, and scalable.* Integrate internal tools with third-party systems to track, analyze, and visualize experiments effectively### Requirements:* 3+ years of experience in Java/Kotlin and a solid understanding of software development best practices.* A strong understanding of development best practices and hands-on experience with CI/CD pipelines.* Excellent communication skills, with the ability to articulate ideas clearly and work effectively with different teams.* A strong problem-solving mindset, with a passion for creating tools that empower developers and enhance their workflow### Preferable:* Experience developing AI-driven functionality or working with AI-powered tools