Senior Data Engineer

We are looking for a Senior Data Engineer to join our Insider Risk Program team. In this role, you will work at the intersection of data engineering and data science, developing and managing data pipelines, joining and filtering data sets, and building advanced insider risk detections to proactively identify and address potential threats. Security represents the most critical priorities for our customers in a world awash in digital threats, regulatory scrutiny, and estate complexity. Microsoft Security aspires to make the world a safer place for all. We want to reshape security and empower every user, customer, and developer with a security cloud that protects them with end to end, simplified solutions.

The Microsoft Security organization accelerates Microsoft’s mission and bold ambitions to ensure that our company and industry is securing digital technology platforms, devices, and clouds in our customers’ heterogeneous environments, as well as ensuring the security of our own internal estate. Our culture is centered on embracing a growth mindset, a theme of inspiring excellence, and encouraging teams and leaders to bring their best each day. In doing so, we create life-changing innovations that impact billions of lives around the world. 

The Microsoft Insider Risk Program is dedicated to safeguarding our people, data, and intellectual property from internal threats that could compromise the security and integrity of the company. Built on a foundation of innovation and collaboration, the program combines proactive detection, advanced data analytics, and effective response mechanisms to identify and mitigate insider risks in real-time. Our team is focused on developing advanced detection and response capabilities, enabling Microsoft to remain resilient against internal risks in an ever-evolving threat landscape.

As a Senior Data Engineer within the Insider Risk Program, you’ll play a critical role in building and optimizing the data infrastructure that powers our risk detection and response efforts. You will work closely with data scientists, software engineers, and insider risk analysts to develop data pipelines, integrate and filter large datasets, and design scalable systems that support complex insider risk detections. In addition to creating reliable data solutions, this role involves maintaining and ensuring compliance with Microsoft’s security and engineering standards, enabling the team to respond to risks swiftly and proactively.

This position offers a unique opportunity to shape the insider risk landscape at one of the world’s leading technology companies. You will be instrumental in architecting and refining data processes that enhance Microsoft’s ability to detect, analyze, and respond to insider risks at scale, directly contributing to the company’s overall security.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Required/Minimum Qualifications

• Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling or data engineering work
• OR Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ year(s) experience in business analytics, data science, software development, or data engineering work
• OR equivalent experience.
• 2+ years experience in query languages (e.g., SQL, KQL)
• 2+ years experience with object-oriented programming languages (e.g., Python, C#, Java, or C++)
• 1+ years experience with big data systems and tools, such as PySpark, Databricks, or Azure Synapse

Other Requirements: 

• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.  

Additional or Preferred Qualifications

• Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering , or related field AND 8+ years experience in business analytics, data science, software development, data modeling or data engineering work
• OR Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years of business analytics, data science, software development, data modeling or data engineering work experience
• OR equivalent experience.
• Experience with machine learning and statistical modeling for detection development.
• Demonstrated ability to build and manage data systems in the cloud
• Understanding of engineering and security compliance standards, with experience in regulated environments.
• Problem-solving skills and attention to detail, with a proactive approach to identifying and mitigating risks.

Data Engineering IC4 - The typical base pay range for this role across the U.S. is USD $117,200 - $229,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $153,600 - $250,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:

Microsoft will accept applications for the role until January 24, 2025   

#MSFTSECURITY 

• Design, build, and optimize data pipelines to ingest, process, and prepare data for use in insider risk detection models.
• Work with large datasets, applying advanced data transformation techniques to ensure data quality and accessibility for risk detection.
• Join, filter, and integrate diverse data sources to create comprehensive datasets that enable effective and accurate insider risk detections.
• Develop, test, and deploy insider risk detection models based on data-driven insights to proactively identify anomalous or risky behavior patterns.
• Collaborate with the insider risk team to define and refine detection use cases, ensuring they are accurate, scalable, and aligned with business needs.
• Conduct exploratory data analysis to uncover insights, validate model assumptions, and optimize detection accuracy.
• Maintain and monitor insider risk engineering systems to ensure reliable operation, security, and compliance with internal engineering standards and policies.
• Document data processes, detection workflows, and system configurations to support future development and maintenance.