Senior Data Engineer

2 Months ago • All levels • Data Analysis

Job Summary

Job Description

The Senior Data Engineer will be responsible for delivering end-to-end data and analytics capabilities, including data ingestion, transformation, data science, and data visualization. This involves designing and deploying databases and data pipelines to support analytics projects. The engineer will develop scalable and fault-tolerant workflows, clearly document issues and solutions, and apply various tools and technologies proficiently, such as Python, PySpark, SQL, Spark, Snowflake, Airflow, AWS, and others. They will also be expected to optimize performance, develop briefings, analyze client data, and provide support to other data engineers and analysts. The candidate should be able to lead the team, communicate with business, gather and interpret business requirements.
Must have:
  • Expert experience in SQL, Python, and PySpark.
  • Experience with data and analytics technologies like SQL/NoSQL databases.
  • Knowledge of CI/CD tools (Gitlab, AWS CodeCommit).
  • Experience with AWS services (EMR, Glue, Athena, etc.).
  • Solid scripting skills (bash/shell scripts, Python).
  • Experience with data streaming technologies.
  • Experience with Big Data technologies like Hadoop, Spark, Hive, etc.
Good to have:
  • Exposure to Snowflake and Airflow.
  • Other programming languages (R, Scala, SAS, Java, etc.)
  • AWS Solutions Architect / Developer / Data Analytics certifications.

Job Details

Data Engineer Responsibilities :
  • Deliver end-to-end data and analytics capabilities, including data ingest, data transformation, data science, and data visualization in collaboration with Data and Analytics stakeholder groups 
  • Design and deploy databases and data pipelines to support analytics projects 
  • Develop scalable and fault-tolerant workflows 
  • Clearly document issues, solutions, findings and recommendations to be shared internally & externally 
  • Learn and apply tools and technologies proficiently, including: 
    • Languages: Python, PySpark, ANSI SQL, Python ML libraries
    • Frameworks/Platform: Spark, Snowflake, Airflow, Hadoop , Kafka 
    • Cloud Computing: AWS
    • Tools/Products: PyCharm, Jupyter, Tableau, PowerBI 
  • Performance optimization for queries and dashboards
  • Develop and deliver clear, compelling briefings to internal and external stakeholders on findings, recommendations, and solutions 
  • Analyze client data & systems to determine whether requirements can be met 
  • Test and validate data pipelines, transformations, datasets, reports, and dashboards built by team
  • Develop and communicate solutions architectures and present solutions to both business and technical stakeholders
  • Provide end user support to other data engineers and analysts
Candidate Requirements :
  • Expert experience in the following[Should have/Good to have]:
    • SQL, Python, PySpark, Python ML libraries. Other programming languages (R, Scala, SAS, Java, etc.) are a plus
    • Data and analytics technologies including SQL/NoSQL/Graph databases, ETL, and BI
    • Knowledge of CI/CD and related tools such as Gitlab, AWS CodeCommit etc. 
    • AWS services including EMR, Glue, Athena, Batch, Lambda CloudWatch, DynamoDB, EC2, CloudFormation, IAM and EDS
    • Exposure to Snowflake and Airflow.
  • Solid scripting skills (e.g., bash/shell scripts, Python) 
  • Proven work experience in the following:
    • Data streaming technologies
    • Big Data technologies including, Hadoop, Spark, Hive, Teradata, etc.
    • Linux command-line operations
    • Networking knowledge (OSI network layers, TCP/IP, virtualization) 
  • Candidate should be able to lead the team, communicate with business, gather and interpret business requirements
  • Experience with agile delivery methodologies using Jira or similar tools 
  • Experience working with remote teams 
  • AWS Solutions Architect / Developer / Data Analytics Specialty certifications, Professional certification is a plus 
  • Bachelor Degree in Computer Science relevant field, Masters Degree is a plus 

Similar Jobs

Electronic Arts - Development Director - EA SPORTS FC

Electronic Arts

Vancouver, British Columbia, Canada (On-Site)
3 Weeks ago
Mcdonalds - Software Engineer III - Android

Mcdonalds

Mexico City, Mexico (On-Site)
1 Month ago
Marsh McLennan - Senior Leader - Fiduciary

Marsh McLennan

Norwich, England, United Kingdom (Hybrid)
2 Months ago
Wargaming - Lead Level Artist

Wargaming

Warsaw, Masovian Voivodeship, Poland (Remote)
4 Weeks ago
USE Insider - Senior Learning and Development Specialist

USE Insider

Istanbul, İstanbul, Türkiye (Hybrid)
8 Months ago
C3 IoT - Senior Software Engineer, Platform - Data + AI (Back-End)

C3 IoT

Redwood City, California, United States (On-Site)
6 Days ago
Codeway Studios - Jr. Data Analyst

Codeway Studios

Istanbul, İstanbul, Türkiye (On-Site)
2 Months ago
Apple - Data Visualization / BI Engineer

Apple

Cupertino, California, United States (On-Site)
3 Weeks ago
Nine - Principal Data Engineer

Nine

North Sydney, New South Wales, Australia (On-Site)
1 Month ago
Fliff - Data Scientist

Fliff

Austin, Texas, United States (On-Site)
1 Year ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

PwC - Manager - IFS - Entrepreneurial Private and Business

PwC

Jakarta, Jakarta, Indonesia (On-Site)
9 Months ago
PwC - Associate - Consulting - Workforce Transformation (Talent Pool)

PwC

Jakarta, Jakarta, Indonesia (On-Site)
9 Months ago
Barracuda - Manager, Cloud Site Reliability Engineering

Barracuda

Reading, England, United Kingdom (Hybrid)
3 Months ago
Kavalirio - AVP, Training & Development Facilitator

Kavalirio

Santa Rosa, California, United States (On-Site)
1 Month ago
PwC - Legal Senior Associate

PwC

Bangkok, Bangkok, Thailand (On-Site)
9 Months ago
CD PROJEKT RED - Lead VFX Artist

CD PROJEKT RED

Warsaw, Masovian Voivodeship, Poland (On-Site)
2 Months ago
Electronic Arts - VP, GM Live Service Franchise

Electronic Arts

Vancouver, British Columbia, Canada (Hybrid)
2 Months ago
Wargaming - Lead Level Artist (World of Tanks)

Wargaming

Belgrade, Serbia (Hybrid)
3 Months ago
Dave Ramsey - Sr. Director of Marketing

Dave Ramsey

Franklin, Tennessee, United States (On-Site)
1 Month ago
Monzo - Credit Risk Director, Flex Originations (Credit Cards)

Monzo

London, England, United Kingdom (Remote)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Thiruvananthapuram, Kerala, India

Pivotroots - Account Director – Social Media

Pivotroots

Mumbai, Maharashtra, India (On-Site)
2 Months ago
Toku - Senior Manager - Finance

Toku

Mumbai, Maharashtra, India (Remote)
6 Months ago
Capgemini - Lead Software Engineer

Capgemini

Pune, Maharashtra, India (On-Site)
1 Month ago
Accenture - Software Development Lead

Accenture

Jaipur, Rajasthan, India (On-Site)
1 Month ago
Pay2 - QA Automation Engineer

Pay2

Gurugram, India (On-Site)
2 Months ago
Capgemini - DC ACI Tac Engineer

Capgemini

Gurugram, Haryana, India (On-Site)
1 Month ago
bounteous - Lead Angular Ionic Developer

bounteous

Chennai, Tamil Nadu, India (Hybrid)
7 Months ago
NCR Voyix - Middleware Administrator II - SOA

NCR Voyix

Gurugram, Haryana, India (On-Site)
2 Months ago
WebMobril Technologies - Jr. Unity Developer

WebMobril Technologies

Noida, Uttar Pradesh, India (Hybrid)
1 Year ago
Capgemini - Salesforce Vlocity Developer

Capgemini

India (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Axi - Data Engineer

Axi

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Valve corporation - Statistician / Data Scientist

Valve corporation

Bellevue, Washington, United States (On-Site)
8 Months ago
ShyftLabs - Staff Data Engineer

ShyftLabs

Toronto, Ontario, Canada (Hybrid)
2 Months ago
Buckman - Senior Digital Innovation Engineer - Data Science

Buckman

Chennai, Tamil Nadu, India (On-Site)
2 Weeks ago
Rackspace Technology - Presales Data Science Architect – AWS Cloud

Rackspace Technology

San Antonio, Texas, United States (Remote)
3 Weeks ago
beghou consulting - Data Engineer

beghou consulting

Hyderabad, Telangana, India (Hybrid)
2 Months ago
Euromonitor - Data Platform Engineer II

Euromonitor

Bengaluru, Karnataka, India (Hybrid)
3 Months ago
binance - Senior Data Analyst - Compliance

binance

Taipei City, Taiwan (Remote)
1 Year ago
Penumbrainc - Business Process Analyst, CFSO

Penumbrainc

Alameda, California, United States (Hybrid)
6 Months ago
Tide - Senior Data Analytics Engineer - Marketing

Tide

Lithuania (Remote)
1 Week ago

Get notifed when new similar jobs are uploaded

About The Company

Boston, Massachusetts, United States (Hybrid)

Berwyn, Pennsylvania, United States (Hybrid)

Berwyn, Pennsylvania, United States (Hybrid)

Raleigh, North Carolina, United States (Hybrid)

Thiruvananthapuram, Kerala, India (On-Site)

Thiruvananthapuram, Kerala, India (On-Site)

Denver, Colorado, United States (Hybrid)

Thiruvananthapuram, Kerala, India (On-Site)

Berwyn, Pennsylvania, United States (Hybrid)

Raleigh, North Carolina, United States (Remote)

View All Jobs

Get notified when new jobs are added by Yodlee

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug