Senior Data Engineer

2 Months ago • 5 Years + • Data Analysis

Job Summary

Job Description

As a Senior Data Engineer specializing in Databricks, you will be responsible for setting up and maintaining the Databricks platform, building robust data pipelines, and collaborating with solution architects and data scientists. You will design, build, and maintain data pipelines within Databricks, debug complex issues, and enhance system performance. You will also be involved in setting up Databricks environments on cloud platforms, automating processes, creating documentation, and developing integration tests. The role requires a deep understanding of data governance and management strategies. The team is looking for someone who can collaborate with international teams to develop scalable data solutions, ensuring data quality and reliability.
Must have:
  • 5+ years of experience as a Data Engineer.
  • Advanced proficiency in Python for data pipelines.
  • Hands-on experience with Databricks platform.
  • Strong knowledge of Apache Spark.
  • Familiarity with Azure or AWS cloud environments.
  • Proficiency with SQL and relational databases.
  • Experience with Airflow or similar tools.
  • Understanding of CI/CD pipelines.
  • Solid skills in debugging data pipeline issues.
  • Proficiency in structured documentation practices.
  • B2 level or higher proficiency in English.
  • Strong collaboration skills.
Good to have:
  • Experience with Docker and Kubernetes.
  • Familiarity with Elasticsearch or other vector databases.
  • Understanding of DBT (data build tool).
  • Ability to travel abroad twice a year.
Perks:
  • Flexible working format (remote, office-based, or flexible).
  • Competitive salary and good compensation package.
  • Personalized career growth.
  • Professional development tools.
  • Active tech communities with regular knowledge sharing.
  • Education reimbursement.
  • Memorable anniversary presents.
  • Corporate events and team buildings.
  • Other location-specific benefits.

Job Details

We are seeking a Senior Data Engineer specializing in Databricks to join our global team. You will be instrumental in setting up and maintaining our Databricks platform, building robust data pipelines, and collaborating closely with our solution architects and data scientists. Your expertise will directly support our mission to leverage data and AI effectively within a cutting-edge automotive claims management environment.

Key Responsibilities:

  • Design, build, and maintain robust data pipelines within Databricks.
  • Collaborate closely with international teams, including data scientists and architects, to develop scalable data solutions.
  • Debug complex issues in data pipelines and proactively enhance system performance and reliability.
  • Set up Databricks environments on cloud platforms (Azure/AWS).
  • Automate processes using CI/CD practices and infrastructure tools such as Terraform.
  • Create and maintain detailed documentation, including workflows and operational checklists.
  • Develop integration and unit tests to ensure data quality and reliability.
  • Migrate legacy data systems to Databricks, ensuring minimal disruption.
  • Participate actively in defining data governance and management strategies.

What We Expect from You (Requirements):

  • 5+ years of proven experience as a Data Engineer.
  • Advanced proficiency in Python for developing production-grade data pipelines.
  • Extensive hands-on experience with Databricks platform.
  • Strong knowledge of Apache Spark for big data processing.
  • Familiarity with cloud environments, specifically Azure or AWS.
  • Proficiency with SQL and experience managing relational databases (MS SQL preferred).
  • Practical experience with Airflow or similar data orchestration tools.
  • Strong understanding of CI/CD pipelines and experience with tools like GitLab.
  • Solid skills in debugging complex data pipeline issues.
  • Proficiency in structured documentation practices.
  • B2 level or higher proficiency in English.
  • Strong collaboration skills, ability to adapt, and eagerness to learn in an international team environment.

Nice to have:

  • Experience with Docker and Kubernetes.
  • Familiarity with Elasticsearch or other vector databases.
  • Understanding of DBT (data build tool).
  • Ability to travel abroad twice a year for on-site workshops.

Why Join Us

  • Work on impactful projects with cross-functional teams.
  • Opportunity to grow your BI and analytics career in a data-driven organization.
  • Flexible working hours and remote work options.
  • Competitive compensation and benefits.
  • Opportunity to work on presales

We offer*:

  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits

*not applicable for freelancers

Similar Jobs

Rackspace Technology - Cloud Business Consultant

Rackspace Technology

Aguascalientes, Aguascalientes, Mexico (Remote)
4 Months ago
Sierra - Technical Program Manager, Agent Development

Sierra

San Francisco, California, United States (On-Site)
2 Weeks ago
JDA - Director Solutions Marketing – Government

JDA

Boston, Massachusetts, United States (Remote)
1 Year ago
Univision - Director, Data Science - Personalization & Product Analytics

Univision

Miami, Florida, United States (On-Site)
1 Year ago
Capgemini - SHAREPOINT ADMIN

Capgemini

Bengaluru, Karnataka, India (On-Site)
2 Months ago
London stock Exchange - Business Analyst – Data Product (Cloud)

London stock Exchange

Bengaluru, Karnataka, India (On-Site)
2 Months ago
bytedance - Data Analyst - Corporate Information System

bytedance

Singapore (On-Site)
5 Months ago
Canonical - Software Engineer - Data Infrastructure

Canonical

(Remote)
2 Months ago
ARHS - Senior Data Architect

ARHS

Luxembourg (On-Site)
2 Weeks ago
Epic Games - Senior Data Scientist

Epic Games

Cary, North Carolina, United States (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Philips - Senior PTP Specialist

Philips

Suzhou, Jiangsu, China (On-Site)
2 Months ago
Flow - Engineering Manager

Flow

Palo Alto, California, United States (Hybrid)
9 Months ago
OKX - Associate General Counsel, U.S. Licensing Maintenance

OKX

New York, United States (Hybrid)
2 Weeks ago
ShyftLabs - Technical Project Manager

ShyftLabs

Toronto, Ontario, Canada (Hybrid)
2 Months ago
Scopely - Senior SQL Developer (Oracle)

Scopely

Mexico City, Mexico (On-Site)
1 Month ago
bounteous - Front-End Architect (Angular)

bounteous

Chennai, Tamil Nadu, India (Hybrid)
5 Months ago
oportun - Senior Data Engineer

oportun

India (Remote)
2 Months ago
Unity - Senior Client Partner, Ad Monetization

Unity

San Francisco, California, United States (On-Site)
1 Month ago
Granicus - Implementation Project Manager

Granicus

Costa Rica (Remote)
1 Month ago
Canva - Engineering Manager - Development Collaboration

Canva

Adelaide, South Australia, Australia (Remote)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Poland

Techland - Art Director (Dying Light Univere)

Techland

Warsaw, Masovian Voivodeship, Poland (On-Site)
12 Months ago
Huuuge Games - Monetization Specialist

Huuuge Games

Warsaw, Masovian Voivodeship, Poland (Hybrid)
2 Weeks ago
Techland - Animation Director

Techland

Poland (On-Site)
9 Months ago
Techland - Compensation and Benefits Manager

Techland

Warsaw, Masovian Voivodeship, Poland (Hybrid)
2 Months ago
Whatnot - Software Engineer, Payments

Whatnot

Kraków, Lesser Poland Voivodeship, Poland (On-Site)
2 Months ago
CD PROJEKT RED - Engineer, Locomotion & Animation

CD PROJEKT RED

Warsaw, Masovian Voivodeship, Poland (On-Site)
2 Months ago
moonmana - Hire Game Developers

moonmana

Gdańsk, Pomeranian Voivodeship, Poland (On-Site)
1 Month ago
Axel springer - Business Growth Manager Fakt.pl

Axel springer

Warsaw, Masovian Voivodeship, Poland (On-Site)
2 Months ago
flying wild hog - Junior IT Support

flying wild hog

Warsaw, Masovian Voivodeship, Poland (On-Site)
3 Months ago
Papaya Gaming - Backend Engineer (Player Journey)

Papaya Gaming

Warsaw, Masovian Voivodeship, Poland (Hybrid)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Powerplay studio - Data Analyst

Powerplay studio

Bratislava, Bratislava Region, Slovakia (Hybrid)
2 Months ago
Dentsu - Senior Data Scientist

Dentsu

Maryland, United States (Remote)
2 Months ago
Winzo - Data Engineer

Winzo

New Delhi, Delhi, India (On-Site)
3 Months ago
CommerceIQ - Data Scientist II

CommerceIQ

Bengaluru, Karnataka, India (On-Site)
3 Months ago
Adyen - Software Engineer - Distributed Data Stores

Adyen

Amsterdam, North Holland, Netherlands (On-Site)
1 Month ago
Capgemini - IAM Business Analyst

Capgemini

Bengaluru, Karnataka, India (On-Site)
3 Months ago
GoTo Group - Business Intelligence Analyst

GoTo Group

Jakarta, Jakarta, Indonesia (On-Site)
1 Month ago
bytedance - Data Analyst - Corporate Information System

bytedance

Singapore (On-Site)
3 Months ago
Go Fund Me - Staff Data Analyst

Go Fund Me

San Francisco, California, United States (Hybrid)
1 Month ago
quience - Staff Data Scientist, ML - Growth

quience

United States (Remote)
3 Months ago

Get notifed when new similar jobs are uploaded