Senior Data and Applied Scientist

Microsoft is a company where passionate innovators come to collaborate, envision what can be and take their careers further. This is a world of more possibilities, more innovation, more openness, and the sky is the limit thinking in a cloud-enabled world.

The AI Platform organization at Microsoft builds the end-to-end Azure AI stack/PaaS and is core to Azure’s innovation and differentiation, as well as all of Microsoft’s flagship products, from Office to Teams, to Xbox. We are the team building Azure OpenAI, Azure ML, Cognitive Services, and the global Azure AI infrastructure for running the largest AI workloads on the planet. 

Within the AI Platform, our Evaluation AI team specializes in building evaluation frameworks for cutting-edge deep learning models, including Large Language Models, Small Language Models, RAG, fine-tuned, and distilled models across NLP, vision, multimodal, Co-pilot and agentic frameworks. We build the next-generation model evaluation platform for generative applications, leveraging state-of-the-art (SOTA) OSS and OAI models.

We are looking for a passionate, creative, analytical Data Scientist who loves NLP, deep learning and wants to ship products quickly at a massive scale. We will provide a lot of opportunities for you to learn, grow and contribute.

• Design and build sophisticated evaluation frameworks for advanced deep learning models, including LLMs, SLMs, RAG, fine-tuned, distilled models, and Generative AI applications like Co-pilots and agentic frameworks
• Research, implement, and refine AI evaluation frameworks to assess model performance across diverse metrics, such as quality, robustness, fairness, and efficiency
• Conduct extensive experiments to evaluate model performance, robustness, and generalization capabilities, applying advanced prompt engineering techniques to enhance outcomes
• Collaborate with cross-functional teams, including researchers, data scientists, software engineers, and product managers
• Work with large-scale datasets, ensuring proper preprocessing, feature selection, and data representation
• Develop, evaluate, and fine-tune large language models (LLMs) using transformer-based architectures and state-of-the-art Generative AI techniques
• Design and build end-to-end ML pipelines covering model training, data analysis, model serving and model evaluation
• Drive innovation by implementing novel techniques from published literature and industry best practices to enhance evaluation capabilities for Azure Evaluation platform
• Embody our  and