Senior Data Engineer

What you’ll do:

Technical Responsibilities:

• Design, build, and manage robust data pipelines using Airflow, ensuring seamless integration, transformation, and accessibility of data.
• Contribute to the development of our data platform, building custom plugins and operators to enhance pipeline functionality and reusability.
• Manage and optimize cloud-based infrastructure, primarily using AWS services.
• Implement and manage infrastructure as code using Terraform.

Data Responsibilities:

• Translate complex business needs into effective data architecture and workflows.
• Design and maintain data models that align with business requirements, ensuring scalability and accuracy.
• Use SQL to build and optimize data models and serving layers that enable analytics and other teams to access ready-to-use, high-quality data.
• Advocate for and implement data governance best practices, focusing on visibility, quality, and compliance with industry standards.
• Develop and enforce data quality checks, contracts, and validation frameworks to ensure the integrity of data assets.

Communication & Collaboration:

• Collaborate closely with cross-functional teams to understand and fulfill their data needs, ensuring alignment between technical and business objectives.
• Clearly communicate technical concepts to non-technical stakeholders, fostering transparency and understanding.
• Manage and prioritize your workload effectively, demonstrating strong project management skills.
• For senior-level professionals: Lead data engineering projects and initiatives, mentoring team members and driving cross-team collaboration.

What we are looking for:

Technical Skills:

• 5+ years of experience in data engineering roles.
• Proficiency in Python with a proven ability to write and maintain production-grade code.
• Hands-on experience with AWS cloud services and orchestration tools (e.g., Airflow), as well as infrastructure as code tools like Terraform.
• Expertise in SQL and analytical data warehouses (preferably Snowflake), with knowledge of ETL/ELT best practices.
• Experience with Retrieval-Augmented Generation (RAG) AI systems is a plus.

Data Expertise:

• Strong data modeling skills and experience in translating business requirements into scalable data solutions.
• Deep understanding of data governance practices, including data quality frameworks and tools such as Soda or similar platforms.
• Familiarity with defining and tracking metrics to assess system performance and cost-efficiency.

Communication & Leadership:

• Excellent communication skills with the ability to bridge technical and business perspectives.
• Experience working closely with business teams to identify requirements and deliver impactful data solutions.
• Proven ability to manage and prioritize tasks, estimate effort, and lead projects or initiatives.
• For senior-level professionals: Demonstrated leadership in mentoring and guiding team members and leading collaborative efforts across departments.