Member of Technical Staff – Data Engineer

1 Month ago • 6-10 Years • Backend Development • $161,600 PA - $314,400 PA

Job Summary

Job Description

Microsoft AI seeks an experienced Data Engineer to architect, design, and implement a highly scalable data platform processing 50TB/day of AI system data. Responsibilities include owning the data platform architecture, superior coding in Spark (Scala, Python, Java), collaborating with engineers and researchers, delivering high-quality code, overcoming roadblocks, and thriving in a fast-paced environment. The ideal candidate has experience building large-scale data processing platforms using technologies like Apache Hadoop, Kafka, columnar databases, and cloud infrastructure (Azure, AWS, GCP). The role requires building distributed systems and a deep understanding of systems architecture.
Must have:
  • 8+ years experience in software development or data engineering
  • 5+ years building large-scale data processing platforms
  • 5+ years building scalable services on public cloud
  • Superior coding skills in Spark (Scala, Python, Java)
  • Experience with Apache Hadoop ecosystem and related technologies
Good to have:
  • 10+ years experience building data platforms handling large data volumes
  • Experience with C#, SQL
  • Proficiency in identifying and resolving complex technical issues

Job Details

Job Description

Overview: 
As Microsoft continues to push the boundaries of AI, we are on the lookout for passionate individuals to work with us on the most interesting and challenging AI questions of our time. Our vision is bold and broad — to build systems that have true artificial intelligence across agents, applications, services, and infrastructure. It’s also inclusive: we aim to make AI accessible to all — consumers, businesses, developers — so that everyone can realize its benefits.

Microsoft AI (MS AI) is seeking an experienced Data Engineer to help architect, design and implement a highly scalable and optimized data platform for processing 50TB/day of data from AI systems. We’re looking for someone who possesses technical prowess, a methodical approach to problem-solving, proficiency in data platform and pipeline technologies, and a mastery of templating to architect solutions that stand the test of time and who will bring an abundance of positive energy, empathy, and kindness to the team every day, in addition to being highly effective. The right candidate enjoys building world-class data platforms that are scalable, maintainable and kept up to date with current tech stacks and design patterns.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

By applying to this U.S. New York, New York position, you are required to be local to the New York area and in office 3 days a week. 
 
Responsibilities: 
  • Thought leader and owner of the architecture blueprint of data platform
  • Superior coding skills in SPARK using Scala, Python and Java
  • Work collaboratively with other Platform, infrastructure, application engineers as well as AI Researchers to build next generation data platform services and capabilities.
  • Ship high-quality, well-tested, secure, and maintainable code.
  • Find a path to get things done despite roadblocks to get your work into the hands of users quickly and iteratively.
  • Enjoy working in a fast-paced, design-driven, product development cycle.
  • Embody our and . 
 
Qualifications: 
Required Qualifications: 
  • Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 8 years experience in software development, data modeling or data engineering work 
  • OR Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6 year(s) experience in software development, or data engineering work 
  • OR equivalent experience. 
  • 5 years of hands experience in building from ground up large scale data processing platforms with more than 100 data pipelines. Use of Apache Hadoop eco system, Kafka Stream/Flink, columnar databases, vector databases, semantic search technologies, RDBMS databases, key value stores, etc. 
  • 5 years' experience building scalable services on top of public cloud infrastructure like Azure, AWS, or GCP. 
  • 2 years' experience building distributed systems at scale and extensive systems knowledge that spans bare-metal hosts to containers to networking. 

Preferred Qualifications: 
  • 10 years technical engineering experience building data platforms that handle large volumes of data in both batch and real time. Coding in languages including, but not limited to, Python, C#, Java, Spark, SQL 
  • Ability to identify, analyze, and resolve complex technical issues, ensuring optimal performance, scalability, and user experience. 
  • Dedication to writing clean, maintainable, and well-documented code with a focus on application quality, performance, and security. 
  • Demonstrated interpersonal skills and ability to work closely with cross-functional teams, including product managers, designers, and other engineers. 
  • Ability to clearly communicate complex technical concepts to both technical and non-technical stakeholders. 
  • Passion for learning new technologies and staying up to date with industry trends, best practices, and emerging technologies in web development and AI. 
  • Ability to work in a fast-paced environment, manage multiple priorities, and adapt to changing requirements and deadlines. 
  • Proven ability to collaborate and contribute to a positive, inclusive work environment, fostering knowledge sharing and growth within the team. 

Data Engineering IC6 - The typical base pay range for this role across the U.S. is USD $161,600 - $286,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $209,600 - $314,400 per year. 
 
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:  
Microsoft will accept applications and processes offers for these roles on an ongoing basis. 

Microsoft will accept applications and processes offers for these roles on an ongoing basis.

#copilot #microsoftai 


Similar Jobs

ION - Senior Java Developer - Italy

ION

Collecchio, Emilia-Romagna, Italy (On-Site)
7 Months ago
CyberArk - Staff Site Reliability Engineer

CyberArk

United States (Remote)
3 Weeks ago
Epic Games - Senior Gameplay Systems Programmer

Epic Games

Porto Alegre, State Of Rio Grande Do Sul, Brazil (On-Site)
1 Month ago
Supercell - Senior Server Engineer, Clash of Clans

Supercell

Helsinki, Uusimaa, Finland (On-Site)
7 Months ago
ByteDance - Backend Software Engineer

ByteDance

San Jose, California, United States (On-Site)
1 Month ago
Amanotes - Unity Developer (LiveOps Team)

Amanotes

Ho Chi Minh City, Ho Chi Minh City, Vietnam (On-Site)
1 Month ago
People Can Fly - Senior Backend Programmer

People Can Fly

Gateshead, England, United Kingdom (Remote)
1 Month ago
ByteDance - Senior Backend Software Engineer

ByteDance

San Jose, California, United States (On-Site)
1 Month ago
Google - Software Engineering Manager II, Namespaces Site Reliability Engineering

Google

Dublin, County Dublin, Ireland (On-Site)
1 Month ago
Google - Software Engineering Manager II, Site Reliability Engineering

Google

Sunnyvale, California, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Coda - Senior/Staff Software Engineer

Coda

Manila, Metro Manila, Philippines (Hybrid)
3 Years ago
Qualcomm - Sr Lead Engineer- Display

Qualcomm

Hyderabad, Telangana, India (On-Site)
3 Weeks ago
anavatio  - Software Engineer

anavatio

Chantilly, Virginia, United States (On-Site)
4 Months ago
Scale AI - Robotics Field Engineer

Scale AI

San Francisco, California, United States (On-Site)
1 Month ago
Axon - Manager, Site Reliability Engineering

Axon

Seattle, Washington, United States (Remote)
3 Months ago
Sumo Logic - Senior Software Engineer I - DevOps

Sumo Logic

India (Remote)
2 Weeks ago
Google - Software Engineer III, Full Stack, Google Cloud

Google

Warsaw, Masovian Voivodeship, Poland (On-Site)
6 Months ago
Next Level Business Services - JAVA DEVELOPER

Next Level Business Services

Cincinnati, Ohio, United States (On-Site)
7 Months ago
Nagarro - Staff Engineer, Java Fullstack

Nagarro

Mexico (Remote)
7 Months ago
Capgemini - Automation Testing

Capgemini

Bengaluru, Karnataka, India (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Jobs in New York, New York, United States

Apple - US-Store Leader

Apple

Houston, Texas, United States (On-Site)
2 Weeks ago
Axon - Sr. Business Systems Analyst - Salesforce

Axon

San Francisco, California, United States (On-Site)
2 Weeks ago
Varonis Internal - Director of Sales Development (North America)

Varonis Internal

Morrisville, North Carolina, United States (On-Site)
1 Month ago
Cadence - Principal Software Engineer

Cadence

San Jose, California, United States (On-Site)
1 Month ago
Forescout - Technical Support Engineer

Forescout

Dallas, Texas, United States (On-Site)
1 Month ago
Riot Games - Game Designer III - VALORANT, Competitive Systems

Riot Games

Los Angeles, California, United States (On-Site)
6 Months ago
Scale AI - Instructional Designer

Scale AI

San Francisco, California, United States (On-Site)
6 Days ago
The Walt Disney Company - Senior Technical Program Manager

The Walt Disney Company

New York, New York, United States (On-Site)
1 Month ago
AGBO - Senior Director, Financial Planning & Analysis

AGBO

Los Angeles, California, United States (On-Site)
8 Months ago
UPF Industries  - General Labor

UPF Industries

Prattville, Alabama, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Backend Development Jobs

Tesla - Software Developer, IT Application

Tesla

Barcelona, Catalonia, Spain (On-Site)
3 Months ago
Hacksaw Studios - Gameplay developer / programmer (backend)

Hacksaw Studios

Stockholm, Stockholm County, Sweden (On-Site)
10 Months ago
Playrix - Engineering Manager (Golang)

Playrix

Ireland (Remote)
1 Month ago
Gunzilla - Blockchain Go Engineer

Gunzilla

Kyiv, Kyiv City, Ukraine (On-Site)
2 Months ago
ByteDance - Software Engineer - Data Engineering (Video Arch)

ByteDance

Singapore (On-Site)
7 Months ago
Canva - Staff Backend Engineer - Product Insights Enablement - Java

Canva

Melbourne, Victoria, Australia (Remote)
3 Months ago
Playtika - PHP Developer

Playtika

Poland (Hybrid)
7 Months ago
Eneba Games - Backend Engineer, PHP

Eneba Games

(Remote)
2 Months ago
Sovrun - Senior Rust Engineer

Sovrun

Makati, Metro Manila, Philippines (Remote)
2 Months ago
Sony Interactive Entertainment - System Development Engineer (PlayStation Platform Game Content Authorship and Delivery System)

Sony Interactive Entertainment

Tokyo, Japan (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Vancouver, British Columbia, Canada (On-Site)

Mountain View, California, United States (Hybrid)

Shenzhen, Guangdong Province, China (On-Site)

Noida, Uttar Pradesh, India (On-Site)

Redmond, Washington, United States (On-Site)

Paris, Île-de-France, France (On-Site)

View All Jobs

Get notified when new jobs are added by Microsoft

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug