Manager, Data Engineering

Responsibilities

• Strategic Leadership & Architecture
• Define and drive the data engineering vision, strategy, and roadmap aligned with business goals.
• Design and oversee the implementation of modern, scalable data architecture on AWS cloud (e.g., data lakes, data mesh, data warehouses).
• Lead data platform modernization and migration initiatives. Stay abreast of emerging technologies and trends in the data engineering space. Evaluate and recommend tools and technologies to enhance the data engineering infrastructure.
• Team Management & Execution
• Build, lead, and mentor a high-performing team of data engineers and platform specialists.
• Establish development best practices, code standards, and CI/CD workflows for data pipelines and infrastructure.
• Data Infrastructure & Operations
• Oversee the ingestion, transformation, and integration of large-scale, complex data sets from internal systems (e.g., transaction processing, customer service) and external partners (e.g., credit bureaus, open banking APIs).
• Ensure high availability, reliability, and performance of the data platform in a highly regulated financial environment.
• Monitor and optimize the performance of data systems and pipelines. Identify and address bottlenecks and inefficiencies in data processes.
• Collaborate with Data Science, Analytics, Product, and Compliance teams to ensure secure and governed access to data assets.
• Governance, Security, and Compliance
• Partner with InfoSec, Compliance, and Risk teams to implement robust data security, privacy, and governance controls (e.g., PII management, SOC 2, GDPR, PCIDSS).
• Support data lineage, cataloging, and metadata management initiatives.
• Project Management
• Plan, prioritize, and manage multiple data engineering projects simultaneously. Track project progress and resource utilization.
• Documentation and Training
• Ensure thorough documentation of data engineering processes and systems. Provide training to team members and other stakeholders on data engineering best practices.

Skills

• Technical Skills
• Expertise in data modeling, database design, and data warehousing. Proficient in programming languages such as Python, Java, or Scala.
• Cloud-native architecture expertise (AWS, GCP, or Azure), including containerization (Docker, Kubernetes) and infrastructure-as-code (Terraform, CloudFormation).
• Familiarity with streaming data architectures and tools like Debezium for change data capture
• Experience with distributed query engines like Trino (formerly Presto) for high performance querying of big data.
• Leadership and Communication: Excellent leadership and people management skills. Strong communication skills to effectively convey technical concepts to non-technical stakeholders. Ability to provide technical guidance and mentorship to team members.
• Problem Solving and Analytical Thinking: Strong problem-solving skills and the ability to analyze complex data issues. Experience in troubleshooting and resolving data-related issues.
• Project Management: Proven experience in project management and delivery of data engineering projects.
• Data Governance and Security: Deep understanding of data governance principles and practices. Knowledge of data security best practices and regulatory requirements.

Experience and Qualifications

• Bachelor’s/Master’s degree in engineering (computer science, information systems) with 10+ years of experience in data engineering, with a strong focus on data architecture, ETL, and data modeling using tools such as Apache Spark, Flink, Trino, Airflow, DBT (good to have), and Python.
• 4+ years of experience in managing a team of data engineers, with a track record of successful delivery of complex data projects and strong leadership skills.