Databricks Architect

About the job

Position Description

A Candidate who is having minimal 10 + yrs experience in the relevant field

Job Description: Databricks Architect

Responsibilities:

• Architecture Design: Design scalable and reliable data analytics and machine learning architectures leveraging the Databricks platform to meet business requirements and objectives.
• Platform Implementation: Implement and configure Databricks clusters, workspaces, notebooks, and other components to support data processing, analytics, and model development workflows.
• Data Engineering: Develop and optimize data pipelines for ingesting, processing, and transforming large volumes of data from diverse sources using Databricks and related technologies.
• Data Modeling: Design and implement data models, schemas, and storage structures within Databricks for efficient data organization and querying.
• Machine Learning: Build and deploy machine learning models using Databricks ML frameworks and libraries, and integrate them into production workflows for predictive analytics and decision support.
• Performance Optimization: Optimize Databricks jobs, queries, and workflows for performance, scalability, and cost-efficiency, leveraging best practices and optimization techniques.
• Security and Governance: Implement security controls, access management policies, and data governance frameworks within Databricks to ensure data privacy, compliance, and regulatory requirements.
• Collaboration: Collaborate with data engineers, data scientists, business analysts, and stakeholders to understand requirements, design solutions, and deliver actionable insights using Databricks.
• Documentation and Training: Document Databricks architecture, design decisions, configurations, and best practices, and provide guidance and training to team members on Databricks usage and methodologies.
• Continuous Improvement: Stay abreast of the latest developments in Databricks and related technologies, evaluate new features and capabilities, and recommend enhancements to optimize data analytics workflows.
  
  

Qualifications:

• Experience in Oil & Gas domain preferable.
• Proven experience designing, implementing, and managing data analytics and machine learning solutions using Databricks.
• Strong proficiency in Apache Spark, SQL, Python, and Scala, with hands-on experience developing data pipelines and machine learning models.
• Expertise in cloud platforms such as AWS, Azure, or Google Cloud Platform, with a focus on data services and analytics offerings.
• Experience with data engineering tools and frameworks such as Apache Kafka, Apache Airflow, and Apache Hadoop.
• Knowledge of containerization technologies (e.g., Docker, Kubernetes) and infrastructure-as-code tools (e.g., Terraform) is a plus.
• Excellent problem-solving and analytical skills, with the ability to troubleshoot complex issues and optimize performance in distributed systems.
• Effective communication and collaboration skills, with the ability to work with cross-functional teams in a dynamic environment.
• Databricks certification (e.g., Databricks Certified Associate Developer, Databricks Certified Associate Data Scientist) is desirable but not mandatory.
  
  

Your future duties and responsibilities

Required Qualifications To Be Successful In This Role

Insights you can act on

While technology is at the heart of our clients’ digital transformation, we understand that people are at the heart of business success.

When you join CGI, you become a trusted advisor, collaborating with colleagues and clients to bring forward actionable insights that deliver meaningful and sustainable outcomes. We call our employees “members” because they are CGI shareholders and owners, and, as owners, we enjoy working and growing together to build a company we are proud of. This has been our Dream since 1976, and it has brought us to where we are today—one of the world’s largest independent providers of IT and business consulting services.

At CGI, we recognize the richness that diversity brings. We strive to create a work culture where everyone belongs, and we collaborate with clients in building more inclusive communities. As an equal opportunity employer, we empower all our members to succeed and grow. If you require an accommodation at any point during the recruitment process, please let us know. We will be happy to assist.

Ready to become part of our success story? Join CGI—where your ideas and actions make a difference.