Senior Data Engineer

GT was founded in 2019 by a former Apple, Nest, and Google executive. GT’s mission is to connect the world’s best talent with product careers offered by high-growth companies in the UK, USA, Canada, Germany, and the Netherlands.

On behalf of our client Asahi Europe & International, GT is looking for a Senior Data Engineer interested in the retail domain.

About the Client

Our client is one of the leading beverage companies in Australia and New Zealand, offering a diverse portfolio of alcoholic and non-alcoholic beverages. Its well-known brands include Schweppes, Cool Ridge, and Pepsi in the non-alcoholic segment, while the alcoholic portfolio features Asahi Super Dry, Peroni, and various craft beers, ciders, and spirits.

The company was formed in 2014 through the integration of Schweppes Australia, Asahi Premium Beverages, and Independent Liquor in New Zealand. With a strong focus on innovation and category growth, Asahi Beverages continues to evolve in response to changing consumer demands.

About the Role

We are looking for a highly skilled Data Engineer with strong experience in Databricks to play a key role in overseeing and optimizing the work of System Integrators (SIs). In this role, you will ensure best practices are followed, hold SIs accountable for their deliverables, and prevent costly inefficiencies caused by poor data management. You will also bring recent hands-on expertise, combined with the ability to influence and guide teams effectively, fostering a culture of excellence and collaboration.

Your primary responsibility will be to ensure clean, scalable, and efficient data pipelines, cluster management, and adherence to industry-leading best practices. You will also play a vital role in maintaining optimal data storage practices to prevent unnecessary costs and technical debt.

Responsibilities:

• Oversee the work of System Integrators (SIs), ensuring deliverables are clean, efficient, and adhere to best practices.

• Proactively hold SIs accountable for maintaining high standards, addressing issues, and ensuring data management is efficient and cost-effective.

• Provide expert guidance on cluster management and data pipeline best practices, ensuring performance and scalability.

• Identify and address inefficiencies in data storage to avoid excessive costs and long-term technical debt.

• Act as an influencer within the team, promoting a culture of accountability, excellence, and collaboration.

• Design, implement, and optimize data pipelines and workflows, leveraging Databricks and the Microsoft Azure stack.

• Ensure robust data governance, including cleanup processes to avoid build-up of redundant or costly data.

• Stay hands-on with data engineering to bring practical insights, while stepping into a management role to guide the team and SIs.

• Collaborate with data architects, analysts, and business stakeholders to align data engineering solutions with business needs.

Essential knowledge, skills & experience:

• Proven expertise in Databricks, including hands-on experience with best practices for cluster management, performance optimization, and scalable data solutions.

• Recent hands-on experience in data engineering, coupled with readiness to step into a management and oversight role.

• Strong knowledge of CI/CD pipelines and DataOps best practices.

• Solid understanding of Azure Data Factory, Synapse, Databricks, and PySpark.

Nice-to-have

• Experience in the CPG or Retail industries, with knowledge of their data requirements and KPIs.

• Experience with Azure Fabric or Dataiku is a plus.

• Experience as a solution architect on DataBricks is a big plus.

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field is an asset.

Soft Skills

• Ability to identify and address inefficiencies in data storage and management practices.

• Strong communication and collaboration skills to:

  • Influence and manage stakeholders constructively.

  • Articulate technical solutions in business terms.

  • Provide coaching and mentorship to team members and SIs.

• A self-starter mindset with a strong sense of ownership and accountability.

• Strong problem-solving skills and the ability to communicate technical solutions in business-friendly language

• Ability to work in fast-paced, cross-functional teams, collaborating with both onshore and offshore resources

Interview Steps

1. GT interview with Recruiter

2. Technical interview

3. Final interview

4. Reference check