Data Engineer / R
23 Minutes ago • All levels
Data Analysis
Job Description
Join a cutting-edge initiative focused on predictive maintenance for jet engines. You’ll be part of a dynamic team developing scalable data solutions that power real-time analytics and decision-making in aviation technology. Responsibilities include designing, developing, and maintaining scalable data pipelines using Databricks and Apache Spark, implementing complex ETL workflows, optimizing data transformations and Spark jobs, collaborating with data scientists, ensuring data security, and troubleshooting production issues.
Good To Have:
- Experience with Spark for data processing.
- Knowledge of Azure storage services.
- Familiarity with tools like Jupyter, Pandas, or Dask.
Must Have:
- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
- Implement complex ETL workflows and optimize data transformations for performance and efficiency.
- Tune Spark jobs and optimize resource utilization to improve pipeline performance.
- Collaborate with data scientists, analysts, and stakeholders to deliver high-quality datasets.
- Ensure data security, governance, and compliance with industry standards.
- Troubleshoot production issues and implement robust monitoring solutions.
- Strong proficiency in SQL.
- Hands-on experience with Python development.
- Proven background in Data Engineering.
- Familiarity with Big Data processing standards and tools.
- Experience working with Databricks (preferably on Azure).
Perks:
- Flexible working hours.
- Hybrid work model, allowing employees to divide their time between home and modern offices in key Polish cities.
- A cafeteria system that allows employees to personalize benefits by choosing from a variety of options.
- Generous referral bonuses, offering up to PLN6,000 for referring specialists.
- Additional revenue sharing opportunities for initiating partnerships with new clients.
- Ongoing guidance from a dedicated Team Manager for each employee.
- Tailored technical mentoring from an assigned technical leader, depending on individual expertise and project needs.
- Dedicated team-building budget for online and on-site team events.
- Opportunities to participate in charitable initiatives and local sports programs.
- A supportive and inclusive work culture with an emphasis on diversity and mutual respect.
Job Description
Join a cutting-edge initiative focused on predictive maintenance for jet engines. You’ll be part of a dynamic team developing scalable data solutions that power real-time analytics and decision-making in aviation technology.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
- Implement complex ETL workflows and optimize data transformations for performance and efficiency.
- Tune Spark jobs and optimize resource utilization to improve pipeline performance.
- Collaborate with data scientists, analysts, and stakeholders to deliver high-quality datasets.
- Ensure data security, governance, and compliance with industry standards.
- Troubleshoot production issues and implement robust monitoring solutions.
Tech Stack:
- Azure Cloud, Databricks, Python, .NET, SQL, Terraform, Azure DevOps
Qualifications
- Strong proficiency in SQL.
- Hands-on experience with Python development.
- Proven background in Data Engineering.
- Familiarity with Big Data processing standards and tools.
- Experience working with Databricks (preferably on Azure).
Nice-to-Have Skills:
- Experience with Spark for data processing.
- Knowledge of Azure storage services.
- Familiarity with tools like Jupyter, Pandas, or Dask.
Additional Information
Hybrid work – 2 days per week at the office in Warsaw, Katowice, Poznań, Lublin, Rzeszow or Lodz.
Set alerts for more jobs like Data Engineer / R
Set alerts for new jobs by Square
Set alerts for new Data Analysis jobs in Poland
Set alerts for new jobs in Poland
Set alerts for Data Analysis (Remote) jobs
