Senior Software Engineer

3 Months ago • 4 Years + • Artificial Intelligence • $117,200 PA - $250,200 PA

Job Summary

Job Description

The Senior Software Engineer will contribute to Microsoft's AI Frameworks team, focusing on optimizing the performance of Large Language Models (LLMs). Responsibilities include benchmarking LLMs on various hardware (GPUs, Microsoft silicon), performance debugging and optimization, building software tools for performance monitoring and insights, and enabling rapid deployment of LLMs at scale. The role involves working across multiple levels of the AI software stack (abstractions, compilers, runtimes, APIs) and collaborating with internal and external partners. The goal is to improve inference performance, reduce hardware footprint, and accelerate time-to-market for LLMs used in major Microsoft products like Office, Windows, and Bing.
Must have:
  • 4+ years experience
  • C/C++/Python coding
  • High-performance application experience
  • GPU performance optimization
  • LLM inference experience
Good to have:
  • Experience with PyTorch, TensorFlow, ONNX Runtime
  • CUDA/ROCm/Triton familiarity
  • Cross-team collaboration skills
Perks:
  • Industry-leading healthcare
  • Educational resources
  • Product and service discounts
  • Savings and investment programs
  • Maternity/paternity leave
  • Generous time off
  • Giving programs
  • Networking opportunities

Job Details

Overview

The Artificial Intelligence (AI) Frameworks team at Microsoft develops AI software that enables running AI models everywhere, from world’s fastest AI supercomputers, to servers, desktops, mobile phones, IoT devices and internet browsers. We collaborate with our hardware teams and partners, both internal and external, and operate at the intersection of AI algorithmic innovation, purpose-built AI hardware, systems, and software. We are a team of highly capable and motivated people that pride themselves on a collaborative and inclusive culture.   

 

We own inference performance of OpenAI and other state of the art Large Language Models (LLMs) and work directly with OpenAI on the models hosted on the Azure OpenAI service serving some of the largest workloads on the planet with trillions of inferences per day in major Microsoft products, including Office, Windows, Bing, SQL Server, and Dynamics.  

 

As a Senior Software Engineer on the team, you will have the opportunity to work on multiple levels of the AI software stack, including the fundamental abstractions, programming models, compilers, runtimes, libraries and APIs to enable large scale training and inferencing of models. You will benchmark OpenAI and other LLMs for performance on GPUs and Microsoft HW, debug and optimize performance, monitor performance and enable these models to be deployed in the shortest amount of time and the least amount of HW possible helping achieve Microsoft Azure's capex goals. 

    

This is a hands-on technical role requiring software design and development skills. We’re looking for someone who has a demonstrated history of solving technical problems and is motivated to tackle the hardest problems in building a full end-to-end AI stack. An entrepreneurial approach and ability to take initiative and move fast are essential. 

 

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. 

 

By applying to this position, location is flexible and remote work is possible.

 

Relocation assistance is unavailable for this role.

Qualifications

Required Qualifications 

  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to C, C++, or Python  
    • OR equivalent experience. 
  • 2+ years’ practical experience working on high performance applications and performance debug and optimization on CPU's/GPU's.

Other Requirements 

  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings:
    • Microsoft Cloud Background Check:This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter. 

Preferred Qualifications 

  • Technical background and solid foundation in software engineering principles, computer architecture, GPU architecture, HW neural net acceleration.
  • Experience in end-to-end performance analysis and optimization of state of the art LLMs, HPC applications including proficiency using GPU profiling tools.
  • Experience in DNN/LLM inference and experience in one or more DL frameworks such as PyTorchTensorflow, or ONNX Runtime and familiarity with CUDA, ROCm, Triton.
  • Cross-team collaboration skills and the desire to collaborate in a team of researchers and developers.
  • Software Engineering IC4 - The typical base pay range for this role across the U.S. is USD $117,200 - $229,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $153,600 - $250,200 per year.
      
    Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:   
      
    Microsoft will accept applications and processes offers for these roles on an ongoing basis.
  •  

 

#aiplatform #opensource #SWE24 #SHPE24MSFT

Responsibilities

As a Senior Software Engineer on the team the common tasks of the job would include, but not be limited to: 

  • Identify and drive improvements to end-to-end inference performance of OpenAI and other state of the art LLMs 
  • Measure, benchmark performance on Nvidia/AMD GPU's and first party Microsoft silicon 
  • Optimize and monitor performance of LLMs and build SW tooling to enable insights into performance opportunities ranging from the model level to the systems and silicon level, help reduce the footprint of the computing fleet and achieve Azure AI capex goals 
  • Enable fast time to market of LLMs/models and their deployments at scale by building SW tools that afford velocity in porting models on new Nvidia, AMD GPUs and Maia silicon 
  • Design, implement, and test functions or components for our AI/DNN/LLM frameworks and tools 
  • Speeding up/reducing complexity of key components/pipelines to improve performance and/or efficiency of our systems 
  • Communicate and collaborate with our partners both internal and external  
  • Embody our and  
Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.
Industry leading healthcare
Educational resources
Discounts on products and services
Savings and investments
Maternity and paternity leave
Generous time away
Giving programs
Opportunities to network and connect

Similar Jobs

Tencent - Senior Staff Researcher

Tencent

Palo Alto, California, United States (On-Site)
5 Months ago
NVIDIA - Senior HPC and AI Networking Performance Research and Analysis Engineer

NVIDIA

Santa Clara, California, United States (Hybrid)
3 Months ago
Coursera - Senior Machine Learning Scientist

Coursera

India (Remote)
4 Months ago
Altagram Group - Data Science Internship/Workstudent

Altagram Group

Germany (On-Site)
2 Months ago
Google - Practice Specialist, Generative AI, Google Cloud

Google

(On-Site)
4 Months ago
Tencent - Senior Researcher: Artificial General Intelligence (Natural Language Processing)

Tencent

Bellevue, Washington, United States (On-Site)
8 Months ago
Warner Bros Games - Game Development & Machine Learning Intern

Warner Bros Games

San Francisco, California, United States (On-Site)
3 Months ago
ByteDance - Software Researcher/Engineer - Applied Research Center (Infrastructure+AI)

ByteDance

San Jose, California, United States (On-Site)
5 Months ago
SSC Technologies - Senior Contact Center Application Engineer

SSC Technologies

Illinois, United States (Remote)
3 Months ago
CharacterAI - Research Engineer, ML Systems (All Industry Levels)

CharacterAI

New York, New York, United States (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

ByteDance - Senior Machine Learning Engineer - AML Algorithm

ByteDance

Seattle, Washington, United States (On-Site)
5 Months ago
Inkittt - Senior Machine Learning Engineer, Recommendations

Inkittt

San Francisco, California, United States (Hybrid)
3 Months ago
Ubisoft - Senior ML Data Scientist

Ubisoft

Montreal, Quebec, Canada (On-Site)
3 Months ago
Tencent - Senior Researcher: Artificial General Intelligence (Natural Language Processing) 104531

Tencent

Bellevue, Washington, United States (On-Site)
4 Months ago
ByteDance - Student Researcher (Doubao (Seed) - Foundation Model - Speech Understanding) - 2025 Start (PhD)

ByteDance

Seattle, Washington, United States (On-Site)
5 Months ago
PlayStation Global - Machine Learning Engineer for Game Technology

PlayStation Global

Aliso Viejo, California, United States (On-Site)
9 Months ago
The Walt Disney Company - Lead Data Scientist

The Walt Disney Company

Santa Monica, California, United States (On-Site)
5 Months ago
ByteDance - Machine Learning Engineer

ByteDance

Seattle, Washington, United States (On-Site)
3 Months ago
Zazz - Machine Learning Engineer

Zazz

(Remote)
2 Months ago
Match Group - 機械学習エンジニア(Machine Learning Engineer)

Match Group

Tokyo, Japan (Hybrid)
6 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Mountain View, California, United States

Kokotree - Marketing Interns

Kokotree

Wilmington, North Carolina, United States (On-Site)
5 Months ago
The Walt Disney Company - Disney Signature Experiences Communications Intern

The Walt Disney Company

Celebration, Florida, United States (On-Site)
2 Months ago
Warner Bros Games - Golf Digest Editorial Intern

Warner Bros Games

New York, New York, United States (Hybrid)
2 Months ago
Warner Bros Games - DC Comics Creative Affairs and Production Internships

Warner Bros Games

Burbank, California, United States (Hybrid)
3 Months ago
Twitch - Product Manager - Content Moderation

Twitch

Seattle, Washington, United States (Remote)
7 Months ago
Overwolf - Planning Lead

Overwolf

New York, New York, United States (On-Site)
1 Month ago
Cirrus Logic - Sr. Staff Digital Design Engineer (CP-50021089)

Cirrus Logic

Austin, Texas, United States (Hybrid)
6 Months ago
Intrepid Studios,  Inc  - Senior Network Engineer

Intrepid Studios, Inc

San Diego, California, United States (On-Site)
2 Months ago
Xsolla - Growth Marketing Manager

Xsolla

Los Angeles, California, United States (Remote)
5 Months ago
Patreon - Staff Product Designer, Podcasting

Patreon

San Francisco, California, United States (Hybrid)
5 Months ago

Get notifed when new similar jobs are uploaded

Artificial Intelligence Jobs

PwC - Conversational AI Architect-Senior Associate

PwC

Bengaluru, Karnataka, India (On-Site)
6 Months ago
Evolution - Data Scientist

Evolution

Warsaw, Masovian Voivodeship, Poland (Hybrid)
8 Months ago
Google - Senior Software Engineer, Machine Learning, Google Ads

Google

Los Angeles, California, United States (On-Site)
4 Months ago
RoofStack - Senior Data Scientist

RoofStack

İstanbul, İstanbul, Türkiye (Hybrid)
3 Months ago
Outlier - Software Engineer (Python)

Outlier

Faridabad, Haryana, India (Hybrid)
6 Months ago
Wargaming - Gen AI Business Development Manager

Wargaming

Warsaw, Masovian Voivodeship, Poland (On-Site)
2 Months ago
Zoox - Senior/Staff Software Engineer - Machine Learning

Zoox

Boston, Massachusetts, United States (Hybrid)
6 Months ago
NVIDIA - Senior Technical Instructor - AI and Data Center Infrastructure

NVIDIA

United Kingdom (Remote)
1 Month ago
ByteDance - Research Scientist- Foundation Model, Video Generation

ByteDance

Seattle, Washington, United States (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Microsoft is a tech giant that develops, licenses, and supports a range of software products, services, and devices.

Bengaluru, Karnataka, India (Hybrid)

Redmond, Washington, United States (On-Site)

Santa Clara, California, United States (On-Site)

Vancouver, British Columbia, Canada (On-Site)

Santa Clara, California, United States (On-Site)

California, United States (On-Site)

Redmond, Washington, United States (On-Site)

Bengaluru, Karnataka, India (On-Site)

View All Jobs

Get notified when new jobs are added by Microsoft

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug