Data Engineer with GCP, Databricks & PySpark Expertise

Software Requirements

Required Skills:

• Hands-on experience with GCP services, specifically BigQuery, Cloud Storage, and Composer for data pipeline orchestration
• Proficiency in Databricks platform with PySpark for building and optimizing large-scale ETL/ELT processes
• Expertise in writing and tuning complex SQL queries for data transformation, aggregation, and reporting on large datasets
• Experience integrating data from multiple sources such as APIs, cloud storage, and databases into a central data warehouse
• Familiarity with workflow orchestration tools like Apache Airflow or Cloud Composer for scheduling, monitoring, and managing data jobs
• Knowledge of version control systems (Git), CI/CD practices, and Agile development methodologies

Preferred Skills:

• Experience with other cloud platforms (AWS, Azure) or additional GCP services (Dataflow, Pub/Sub)
• Knowledge of data modeling and data governance best practices
• Familiarity with containerization tools like Docker or Kubernetes

Overall Responsibilities

• Design, develop, and maintain scalable data pipelines using GCP, Databricks, and associated tools
• Write efficient, well-documented SQL queries to support data transformation, data quality, and reporting needs
• Integrate data from diverse sources, including APIs, cloud storage, and databases, to create a reliable central data repository
• Develop automated workflows and schedules for data processing tasks utilizing Composer or Airflow
• Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and deliver solutions
• Monitor, troubleshoot, and optimize data pipelines for performance, scalability, and reliability
• Maintain data security, privacy standards, and documentation compliance
• Stay informed about emerging data engineering technologies and apply them effectively to improve workflows

Technical Skills (By Category)

• Programming Languages:
• Required: PySpark (Python in Databricks), SQL
• Preferred: Python, Java, or Scala for custom data processing
• Databases/Data Management:
• Required: BigQuery, relational databases, large-scale data transformation and querying
• Preferred: Data cataloging and governance tools
• Cloud Technologies:
• Required: GCP services including BigQuery, Cloud Storage, Composer
• Preferred: Experience with other cloud services (AWS, Azure)
• Frameworks and Libraries:
• Required: Databricks with PySpark, Airflow or Cloud Composer
• Preferred: Data processing frameworks such as Apache Beam, Dataflow
• Development Tools and Methodologies:
• Version control using Git
• CI/CD pipelines for automated deployment and testing
• Agile development practices
• Security & Compliance:
• Knowledge of data security best practices, access controls, and data privacy regulations

Experience Requirements

• Minimum of 3 years of professional experience in data engineering or a related role
• Proven expertise in designing and implementing large-scale data pipelines using GCP and Databricks
• Hands-on experience with complex SQL query development and optimization
• Working knowledge of workflow orchestration tools such as Airflow or Cloud Composer
• Experience processing data from multiple sources, including APIs and cloud storage solutions
• Experience in an Agile environment preferred

Alternative pathways:

Candidates with strong data pipeline experience on other cloud platforms who are willing to adapt and learn GCP services may be considered.

Day-to-Day Activities

• Develop, test, and deploy data pipelines that facilitate analytics, reporting, and data science initiatives
• Collaborate with cross-functional teams during sprint planning, stand-ups, and code reviews
• Monitor scheduled jobs for successful execution, troubleshoot failures, and optimize performance
• Document processes, workflows, and data sources in compliance with organizational standards
• Continuously review pipeline performance, implement improvements, and ensure robustness
• Participate in scalable architecture design discussions and recommend best practices

Qualifications

• Bachelor’s degree in Computer Science, Data Science, Information Technology, or equivalent field
• At least 3 years of experience in data engineering, data architecture, or related roles
• Demonstrated expertise with GCP, Databricks, SQL, and workflow orchestration tools

Certifications (preferred):

• GCP certifications such as Professional Data Engineer or equivalent
• Databricks Data Engineer certification

Professional Competencies

• Critical thinking and effective problem-solving skills related to large-scale data processing
• Strong collaboration abilities across multidisciplinary teams and stakeholders
• Excellent communication skills with the ability to translate technical details into clear insights
• Adaptability to evolving technologies and project requirements
• Ability to prioritize tasks, manage time efficiently, and deliver on deadlines
• Innovative mindset with a focus on continuous learning and process improvement

S​YNECHRON’S DIVERSITY & INCLUSION STATEMENT

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.

All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.

Candidate Application Notice

About Us

At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy financial services and technology firms. Through research and development initiatives in our FinLabs we develop solutions for modernization, from Artificial Intelligence and Blockchain to Data Science models, Digital Underwriting, mobile-first applications and more.

Over the last 20+ years, our company has been honored with multiple employer awards, recognizing our commitment to our talented teams. With top clients to boast about, Synechron has a global workforce of 14,500+, and has 58 offices in 21 countries within key global markets.

For more information on the company, please visit our website

or LinkedIn

community.

Read More

Sustainability and Health Safety Commitment

At Synechron, we are committed to integrating sustainability into our business strategy, ensuring responsible growth while minimizing environmental impact. Employees play a key role in driving our sustainability initiatives, from reducing our carbon footprint to fostering ethical and sustainable business practices across global operations. All positions are required to adhere to our Sustainability and Health Safety standards, demonstrating a commitment to environmental stewardship, workplace safety, and sustainable practices.

Read More

Let's Talk

Not finding the right fit?  Let us know you're interested in a future opportunity by clicking Get Started below or create an account by clicking 'Sign In' at the top of the page to set up email alerts as new job postings become available that meet your interest!

Get Started