Machine Learning Operations Engineer

1 Month ago • 2 Years + • Research Development • $50,000 PA - $80,000 PA

Job Summary

Job Description

DraftWise is seeking a Machine Learning Operations (ML Ops) Engineer to enhance their legal evaluation platform for AI solutions in legal contract analysis. Key responsibilities include improving and maintaining the platform, developing continuous auto-evaluation frameworks, designing load balancing for LLM API calls, and deploying/maintaining proprietary LLM models with training and fine-tuning support. The role involves collaborating with ML, NLP, and Backend engineers to drive AI solutions for legal contract analysis. The ideal candidate will have proven Python expertise, experience with Flask, and a strong background in deploying, maintaining, and fine-tuning ML models.
Must have:
  • Python expertise with Flask
  • Deploying, maintaining, fine-tuning ML models
  • Developing auto-evaluation and load balancing frameworks
  • Experience with LLMs and continuous auto-evaluation pipelines
  • 2+ years of relevant experience
  • Collaborate with ML, NLP, backend engineers
  • Customer-centric approach
  • Strong analytical skills
Good to have:
  • Basic knowledge of HTML/JS
Perks:
  • Remote-first work style in Europe
  • Meaningful equity plan
  • Competitive salary
  • Private medical care
  • New laptop
  • Work-from-home stipend
  • Unlimited PTO and sick leave

Job Details

About DraftWise

DraftWise is revolutionizing the legal industry with an innovative knowledge platform that transforms how lawyers and legal professionals work. Our products unlock, for the first time, years of historic legal contracts negotiated by firms for their clients and seamlessly integrate generative AI into the workflows of transactional attorneys. By leveraging cutting-edge data management, machine learning, generative AI, and language analytics, we empower legal professionals to draft better, safer contracts with unprecedented efficiency.

Our founding team brings together former Palantir and Google engineers, alongside an experienced lawyer from a top global law firm. This diverse combination of expertise in engineering, product development, and customer success enables us to deliver industry-leading solutions. While DraftWise operates as a fully remote company, we also provide coworking opportunities in hubs across New York City and London, with plans to expand to additional cities as we grow density in new regions. We are committed to hiring the best talent globally, fostering a culture of respect, collaboration, and excellence.

Backed by Y Combinator and prominent venture capital funds, including a Series A funding round led by Index Ventures with participation from Earlybird Ventures (now Bek Ventures), DraftWise is already generating revenue from some of the most prestigious law firms in the world. Our clients include members of the Vault 10, Magic Circle, AMLAW 100, and other top-tier legal rankings. Our rapid growth spans multiple countries, with further expansions underway.

Following our recent fundraise, we are investing in critical functions and scaling our team to meet rising demand. Joining DraftWise means working alongside a highly talented team to deliver outstanding results for our customers and making a tangible impact on the world’s most influential law firms.

What we value

  • Strong communication skills in an open environment.
  • The ability to work independently and make informed decisions with minimal supervision.
  • Interest in working in a dynamic environment with dynamic objectives.
  • A commitment to autonomy, ownership, and delivering high-quality solutions.
  • Openness to giving and receiving constructive feedback.

About this role

We are seeking a Machine Learning Operations (ML Ops) Engineer to contribute to the development of cutting-edge AI solutions for legal contract analysis. You will be part of our growing ML team, working closely with our ML engineer, NLP engineer and Backend engineers.

Key Responsibilities:

  • Improving and maintaining our legal evaluation platform:
    • Drive continuous enhancements to our state-of-the-art legal evaluation platform by developing innovative evaluation frameworks and ensuring its robust, scalable, and precise operation.
    • Collaborate closely with cross-functional teams to create new methodologies that empower our legal experts to rigorously assess LLM output quality while ensuring the platform remains adaptive to evolving challenges and technological advances.
  • Establishing a Continuous Auto-Evaluation Framework:
    • Develop and deploy an automated system that continuously monitors and validates our LLM outputs against stringent quality thresholds.
    • Incorporate real-time performance metrics, anomaly detection, and proactive remediation triggers—ensuring that any dip in response quality is promptly identified and corrected.
    • Integrate feedback loops that sustain our commitment to excellence, thereby ensuring our platform maintains consistently high standards.
  • Designing a Load Balancing Framework for LLM API Calls:
    • Architect and deploy a dynamic load balancing system that intelligently distributes API calls across multiple LLM endpoints.
    • Implement robust fallback mechanisms to automatically reroute traffic when an LLM instance is down or reaching usage quotas.
    • Integrate real-time health monitoring and performance analytics to assess endpoint availability and adjust routing in real time.
  • Deploying and Maintaining Proprietary LLM Models with Training and Fine-Tuning Support:
    • Deploy and scale in-house LLM models to ensure reliable and high-performance operations tailored to our unique legal evaluation needs.
    • Establish automated pipelines that streamline the training, fine-tuning, and deployment processes, ensuring continuous model improvement and adaptation.
    • Collaborate closely with ML and NLP teams to integrate the latest techniques, rigorous evaluation metrics, and industry best practices into the training workflow.
    • Implement proactive monitoring and maintenance protocols to quickly identify and address performance bottlenecks or operational issues.

About you

We're looking for teammates who possess:

  • Proven expertise in Python with hands-on experience using Flask, along with a willingness to explore and adapt to new frameworks when necessary.
  • A robust background in deploying, maintaining, and fine-tuning ML models, with a focus on developing automated evaluation and load balancing frameworks.
  • Demonstrated experience in working with large language models (LLMs) and establishing continuous auto-evaluation pipelines. For example, you may have built systems that monitor key performance metrics (e.g., latency, quality scores) in real time, or used an open-source framework such as DeepEval or Phoenix.
  • At least 2 years of relevant experience, ideally within startup environments, showcasing a resourceful and proactive approach to addressing evolving challenges.
  • The ability to work closely with ML, NLP, and backend engineers to drive cutting-edge AI solutions tailored for legal contract analysis.
  • While not a necessity, basic knowledge of HTML/JS is a plus, enhancing your contributions within a full-stack development context.
  • A customer-centric approach coupled with strong analytical skills to ensure the platform's robust, scalable, and precise operation.

What we offer:

  • Remote-first work style: Work anywhere in Europe.
  • Meaningful equity plan, giving you a stake in our growth.
  • Competitive salary to reward your expertise.
  • Private medical care to support your well-being.
  • A new laptop and a work-from-home stipend for necessary accessories.
  • Unlimited PTO and sick leave to prioritize work-life balance.
  • The opportunity to shape our company and disrupt the legal world.

Similar Jobs

IGN - Director of Subscriptions

IGN

Los Angeles, California, United States (Hybrid)
3 Months ago
Inkittt - Senior Marketing Manager

Inkittt

San Francisco, California, United States (Hybrid)
4 Months ago
Sonar Source - AI Engineer

Sonar Source

Singapore (On-Site)
4 Months ago
Thumbtack - Senior Engineering Manager, Data

Thumbtack

Ontario, Canada (Remote)
1 Month ago
Google - Technical Program Manager III, AI/ML, Cloud AI Systems

Google

Austin, Texas, United States (On-Site)
3 Months ago
bytedance - Student Researcher (Doubao (Seed) - Foundation Model - Vision Generative AI)

bytedance

San Jose, California, United States (On-Site)
4 Months ago
Tencent - Senior Staff Researcher

Tencent

Palo Alto, California, United States (On-Site)
9 Months ago
Morning Star - Senior Software Development Engineer, ML Operations

Morning Star

Mumbai, Maharashtra, India (Hybrid)
2 Months ago
Apple - AIML - Staff Machine Learning Engineer - Reinforcement Learning

Apple

Santa Clara, California, United States (On-Site)
1 Month ago
WebFX - AI Digital Marketing Specialist

WebFX

Cape Town, Western Cape, South Africa (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

kuda  - Product Manager

kuda

Dar Es Salaam, Dar Es Salam, Tanzania (Hybrid)
2 Months ago
Zuora - Software Engineer III

Zuora

Chennai, Tamil Nadu, India (On-Site)
3 Months ago
EvenUp - Account Executive

EvenUp

Detroit, Michigan, United States (Remote)
1 Month ago
CrazyLabs - QA Engineer

CrazyLabs

Skopje, Greater Skopje, North Macedonia (On-Site)
3 Weeks ago
Toast - Manager, Customer Care - International

Toast

Chennai, Tamil Nadu, India (Hybrid)
2 Months ago
Microsoft - Member of Technical Staff, AI Post-Training

Microsoft

London, England, United Kingdom (On-Site)
4 Months ago
Kyruus Health - Senior Data Architect

Kyruus Health

United States (Remote)
4 Months ago
Rippling - Senior Software Engineer - Data Bridge

Rippling

Seattle, Washington, United States (On-Site)
2 Months ago
AiDash - Software Development Engineer in Test II

AiDash

Bengaluru, Karnataka, India (Hybrid)
1 Month ago
Wind River - Industry Marketing Manager – Automotive

Wind River

Troy, Michigan, United States (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in London, England, United Kingdom

Likewize - Content Workflow Administrator

Likewize

England, United Kingdom (On-Site)
1 Month ago
Cloud Imperium Games - VFX Artist

Cloud Imperium Games

Manchester, England, United Kingdom (On-Site)
5 Months ago
flix interactive - Senior UI Artist

flix interactive

Birmingham, England, United Kingdom (Remote)
3 Months ago
Yahoo - DSP Client Services Senior Account Specialist

Yahoo

United Kingdom (Hybrid)
1 Year ago
Microsoft - Member of Technical Staff, AI Multimodal

Microsoft

London, England, United Kingdom (On-Site)
4 Months ago
Synthesia - Business Talent Acquisition Partner - Contract

Synthesia

London, England, United Kingdom (On-Site)
1 Month ago
Corsair gaming - DIY Marketing Campaign Manager

Corsair gaming

United Kingdom (Remote)
2 Months ago
Square - Purchasing Manager

Square

Hounslow, England, United Kingdom (On-Site)
4 Weeks ago
Resolver - Generalist Analyst (Russian Speaker)

Resolver

Leeds, England, United Kingdom (On-Site)
3 Months ago
quience - Head of Planning & Inventory

quience

London, England, United Kingdom (On-Site)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Research Development Jobs

jetbrains - Research Engineer

jetbrains

Yerevan, Yerevan, Armenia (On-Site)
3 Months ago
Apple - MacOS Machine Learning Engineer

Apple

Seattle, Washington, United States (On-Site)
1 Month ago
Microsoft - Member of Technical Staff, AI Pretraining Platform

Microsoft

Zürich, Zurich, Switzerland (On-Site)
3 Months ago
Cursor - Research Engineer

Cursor

San Francisco, California, United States (On-Site)
1 Month ago
DevRev - Applied AI - Forward Deployed Architect

DevRev

United States (On-Site)
1 Month ago
Voki games - Machine Learning Engineer

Voki games

Kyiv, Kyiv City, Ukraine (Hybrid)
1 Month ago
Luma - Research engineer/Scientist- Post Training

Luma

Palo Alto, California, United States (Hybrid)
3 Weeks ago
PlayStation Global - Staff Machine Learning Engineer, Anomaly Detection

PlayStation Global

London, England, United Kingdom (Hybrid)
9 Months ago
bytedance - Research Scientist (Multimedia Streaming)

bytedance

San Diego, California, United States (On-Site)
2 Months ago
bytedance - Student Researcher Intern (Edge Research Project for General Intelligence)

bytedance

San Jose, California, United States (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

About The Company

New York, United States (Remote)

New York, United States (Remote)

New York, United States (Remote)

London, England, United Kingdom (Remote)

New York, United States (Remote)

New York, United States (Hybrid)

New York, United States (Hybrid)

New York, United States (Remote)

New York, United States (Remote)

View All Jobs

Get notified when new jobs are added by Draftwise

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug