Data Architect (Azure and Databricks)

3 Weeks ago • 6-8 Years • Data Analyst

Job Summary

Job Description

This role involves designing and implementing scalable, secure, and efficient data architectures on the Databricks platform. Responsibilities include leading the technical design of data warehouse/data lake migrations from legacy systems, developing data engineering frameworks, establishing CI/CD pipelines, implementing data catalog solutions and governance frameworks, creating technical specifications, providing technical leadership, collaborating with cross-functional teams, and ensuring data architectures meet security and compliance requirements. The ideal candidate will have extensive experience in data architecture design and implementation, strong software engineering skills (Python or Scala), experience with Databricks, and knowledge of healthcare data requirements and regulations.
Must have:
  • Extensive data architecture experience
  • Strong software engineering (Python/Scala)
  • Databricks Lakehouse expertise
  • CI/CD pipeline implementation
  • Data catalog & governance
  • Healthcare data knowledge
Good to have:
  • AWS, Azure, GCP experience
  • Terraform, CloudFormation
  • Collibra, Alation
  • HL7, FHIR knowledge

Job Details

Overview 
We are seeking an experienced Data Architect with extensive expertise in designing and implementing modern data architectures. This role requires strong software engineering principles, hands-on coding abilities, and experience building data engineering frameworks. The ideal candidate will have a proven track record of implementing Databricks-based solutions in the healthcare industry, with expertise in data catalog implementation and governance frameworks. 
 
About the Role 
As a Data Architect, you will be responsible for designing and implementing scalable, secure, and efficient data architectures on the Databricks platform. You will lead the technical design of data migration initiatives from legacy systems to modern Lakehouse architecture, ensuring alignment with business requirements, industry best practices, and regulatory compliance. 
 
Key Responsibilities 

Design and implement modern data architectures using Databricks Lakehouse platform 
Lead the technical design of Data Warehouse/Data Lake migration initiatives from legacy systems 
Develop data engineering frameworks and reusable components to accelerate delivery 
Establish CI/CD pipelines and infrastructure-as-code practices for data solutions 
Implement data catalog solutions and governance frameworks 
Create technical specifications and architecture documentation 
Provide technical leadership to data engineering teams 
Collaborate with cross-functional teams to ensure alignment of data solutions 
Evaluate and recommend technologies, tools, and approaches for data initiatives 
Ensure data architectures meet security, compliance, and performance requirements 
Mentor junior team members on data architecture best practices 
Stay current with emerging technologies and industry trends 

Qualifications 
Extensive experience in data architecture design and implementation 
Strong software engineering background with expertise in Python or Scala 
Proven experience building data engineering frameworks and reusable components 
Experience implementing CI/CD pipelines for data solutions 
Expertise in infrastructure-as-code and automation 
Experience implementing data catalog solutions and governance frameworks 
Deep understanding of Databricks platform and Lakehouse architecture 
Experience migrating workloads from legacy systems to modern data platforms 
Strong knowledge of healthcare data requirements and regulations 
Experience with cloud platforms (AWS, Azure, GCP) and their data services 
Bachelor's degree in Computer Science, Information Systems, or related field; advanced degree preferred 

Technical Skills 
Programming languages: Python and/or Scala (required) 
Data processing frameworks: Apache Spark, Delta Lake 
CI/CD tools: Jenkins, GitHub Actions, Azure DevOps 
Infrastructure-as-code (optional): Terraform, CloudFormation, Pulumi 
Data catalog tools: Databricks Unity Catalog, Collibra, Alation 
Data governance frameworks and methodologies 
Data modeling and design patterns 
API design and development 
Cloud platforms: AWS, Azure, GCP 
Container technologies: Docker, Kubernetes 
Version control systems: Git 
SQL and NoSQL databases 
Data quality and testing frameworks 

Optional - Healthcare Industry Knowledge 
Healthcare data standards (HL7, FHIR, etc.) 
Clinical and operational data models 
Healthcare interoperability requirements 
Healthcare analytics use cases 
undefinedundefinedundefined

Similar Jobs

PwC - Senior Associate_Databricks_Data & Analytics_Advisory_PAN  India

PwC

Kolkata, West Bengal, India (On-Site)
6 Months ago
Playtika - Youda-PHP Developer

Playtika

Netherlands (Hybrid)
1 Week ago
Sonar Source - Support Engineer

Sonar Source

Geneva, Geneva, Switzerland (On-Site)
5 Months ago
Microsoft - Principal Software Engineer

Microsoft

Noida, Uttar Pradesh, India (On-Site)
6 Days ago
Tencent - DevOps Engineer Intern

Tencent

(On-Site)
1 Month ago
The Walt Disney Company - Manager, Content Analytics

The Walt Disney Company

Buenos Aires, Buenos Aires, Argentina (On-Site)
1 Week ago
NinjaVan - Senior Data Engineer

NinjaVan

Hyderabad, Telangana, India (On-Site)
6 Months ago
Playrix - Senior Data Analyst (Attribution)

Playrix

Georgia (Remote)
6 Months ago
WebFX - Digital Marketing Data Analyst/Strategist

WebFX

Philippines (Remote)
4 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Lockwood - Senior Data Analyst

Lockwood

United Kingdom (Remote)
2 Days ago
Epic Games - Lead Online Programmer

Epic Games

Montreal, Quebec, Canada (On-Site)
1 Week ago
PwC - IN-Manager – D365 Scm -Ms Dynamics– Advisory  - Mumbai

PwC

Mumbai, Maharashtra, India (On-Site)
6 Months ago
Oriserve - Lead DevOps Engineer (5+ Yrs Exp)

Oriserve

Noida, Uttar Pradesh, India (On-Site)
5 Months ago
UXBERT Labs - Senior Technical Lead

UXBERT Labs

Riyadh, Riyadh Province, Saudi Arabia (On-Site)
3 Months ago
Ness Digital - QA Engineer with Tosca

Ness Digital

Timișoara, Timiș, Romania (Remote)
4 Weeks ago
Hitachi - Project Manager

Hitachi

Paris, Île-de-France, France (Hybrid)
5 Months ago
Rackspace Technology - Lead Cloud Engineer

Rackspace Technology

United States (Remote)
1 Month ago
Nagarro - Senior Staff Engineer

Nagarro

Philippines (Remote)
6 Months ago
Omnissa - Staff Engineer (C++,MacOS Internals)

Omnissa

Bengaluru, Karnataka, India (Hybrid)
6 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Gurugram, Haryana, India

Zeta - Lead Site Reliability Engineer

Zeta

Bengaluru, Karnataka, India (On-Site)
6 Months ago
Kwalee - Senior Game Programmer (Creative Marketing)

Kwalee

Bengaluru, Karnataka, India (On-Site)
2 Weeks ago
PwC - IN-Senior Associate_Azure DevOps Architect_OneCloud_Advisory _Bangalore

PwC

Bengaluru, Karnataka, India (On-Site)
4 Months ago
DNEG - FX Lead (DNEG Animation)

DNEG

India (On-Site)
8 Months ago
Hitachi - MSD365 F&O Technical-Feb 2024

Hitachi

Pune, Maharashtra, India (Remote)
6 Months ago
InMobiInMobi - Intern – Business Analytics & Insights

InMobiInMobi

Bengaluru, Karnataka, India (On-Site)
1 Day ago
Google - Software Engineer, Kernel, ChromeOS

Google

Bengaluru, Karnataka, India (On-Site)
1 Week ago
Zeta - Manager - Software Development

Zeta

Bengaluru, Karnataka, India (On-Site)
5 Months ago
STOXX - Data Engineer

STOXX

Maharashtra, India (Hybrid)
6 Months ago
DNEG - Animator

DNEG

Karnataka, India (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

Google - Analytical Lead, Apps and Fintech, Large Customer Sales

Google

Guangdong Province, China (On-Site)
1 Week ago
Hawk Eye Innovations - Senior Data Test Automation Engineer

Hawk Eye Innovations

Budapest, Hungary (Hybrid)
1 Day ago
Glean - Data Science Lead, Product

Glean

Palo Alto, California, United States (On-Site)
5 Months ago
Xsolla - Anti-Fraud Analyst

Xsolla

France (Hybrid)
4 Weeks ago
Meta - Global Sales Analytics Lead

Meta

San Francisco, California, United States (Remote)
5 Months ago
Google - Data Transformation Lead, Media and Entertainment

Google

New York, New York, United States (On-Site)
1 Week ago
Nagarro - Staff Engineer

Nagarro

Sri Lanka (Remote)
6 Months ago
MURKA - Data Scientist

MURKA

Poland (On-Site)
3 Months ago
InMobiInMobi - Senior Product Analyst

InMobiInMobi

Bengaluru, Karnataka, India (On-Site)
2 Months ago
ION - Data Operations ( Markets - Shared Services)

ION

Woking, England, United Kingdom (On-Site)
6 Months ago

Get notifed when new similar jobs are uploaded