Senior Data Engineer

3 Months ago • All levels • Data Analysis

Job Summary

Job Description

The Senior Data Engineer will be responsible for delivering end-to-end data and analytics capabilities, including data ingestion, transformation, data science, and data visualization. This involves designing and deploying databases and data pipelines to support analytics projects. The engineer will develop scalable and fault-tolerant workflows, clearly document issues and solutions, and apply various tools and technologies proficiently, such as Python, PySpark, SQL, Spark, Snowflake, Airflow, AWS, and others. They will also be expected to optimize performance, develop briefings, analyze client data, and provide support to other data engineers and analysts. The candidate should be able to lead the team, communicate with business, gather and interpret business requirements.
Must have:
  • Expert experience in SQL, Python, and PySpark.
  • Experience with data and analytics technologies like SQL/NoSQL databases.
  • Knowledge of CI/CD tools (Gitlab, AWS CodeCommit).
  • Experience with AWS services (EMR, Glue, Athena, etc.).
  • Solid scripting skills (bash/shell scripts, Python).
  • Experience with data streaming technologies.
  • Experience with Big Data technologies like Hadoop, Spark, Hive, etc.
Good to have:
  • Exposure to Snowflake and Airflow.
  • Other programming languages (R, Scala, SAS, Java, etc.)
  • AWS Solutions Architect / Developer / Data Analytics certifications.

Job Details

Data Engineer Responsibilities :
  • Deliver end-to-end data and analytics capabilities, including data ingest, data transformation, data science, and data visualization in collaboration with Data and Analytics stakeholder groups 
  • Design and deploy databases and data pipelines to support analytics projects 
  • Develop scalable and fault-tolerant workflows 
  • Clearly document issues, solutions, findings and recommendations to be shared internally & externally 
  • Learn and apply tools and technologies proficiently, including: 
    • Languages: Python, PySpark, ANSI SQL, Python ML libraries
    • Frameworks/Platform: Spark, Snowflake, Airflow, Hadoop , Kafka 
    • Cloud Computing: AWS
    • Tools/Products: PyCharm, Jupyter, Tableau, PowerBI 
  • Performance optimization for queries and dashboards
  • Develop and deliver clear, compelling briefings to internal and external stakeholders on findings, recommendations, and solutions 
  • Analyze client data & systems to determine whether requirements can be met 
  • Test and validate data pipelines, transformations, datasets, reports, and dashboards built by team
  • Develop and communicate solutions architectures and present solutions to both business and technical stakeholders
  • Provide end user support to other data engineers and analysts
Candidate Requirements :
  • Expert experience in the following[Should have/Good to have]:
    • SQL, Python, PySpark, Python ML libraries. Other programming languages (R, Scala, SAS, Java, etc.) are a plus
    • Data and analytics technologies including SQL/NoSQL/Graph databases, ETL, and BI
    • Knowledge of CI/CD and related tools such as Gitlab, AWS CodeCommit etc. 
    • AWS services including EMR, Glue, Athena, Batch, Lambda CloudWatch, DynamoDB, EC2, CloudFormation, IAM and EDS
    • Exposure to Snowflake and Airflow.
  • Solid scripting skills (e.g., bash/shell scripts, Python) 
  • Proven work experience in the following:
    • Data streaming technologies
    • Big Data technologies including, Hadoop, Spark, Hive, Teradata, etc.
    • Linux command-line operations
    • Networking knowledge (OSI network layers, TCP/IP, virtualization) 
  • Candidate should be able to lead the team, communicate with business, gather and interpret business requirements
  • Experience with agile delivery methodologies using Jira or similar tools 
  • Experience working with remote teams 
  • AWS Solutions Architect / Developer / Data Analytics Specialty certifications, Professional certification is a plus 
  • Bachelor Degree in Computer Science relevant field, Masters Degree is a plus 

Similar Jobs

Bonfire Studios - Lead Environment Artist

Bonfire Studios

California, United States (Hybrid)
6 Months ago
Salesforce - KSA Account Executive - Public Sector - General Business Accounts

Salesforce

Riyadh, Riyadh Province, Saudi Arabia (On-Site)
2 Months ago
AI Fund - Chief Executive Officer (Mental Health)

AI Fund

San Francisco, California, United States (Remote)
9 Months ago
PwC - Senior Associate-SAP ABAP-RDC-Kolkata

PwC

Kolkata, West Bengal, India (On-Site)
10 Months ago
Zelis  - Senior Atlassian Administrator

Zelis

Hyderabad, Telangana, India (On-Site)
3 Months ago
Syniverse - Messaging Trust Data Analyst

Syniverse

San José Province, Costa Rica (Hybrid)
1 Year ago
Madison Logic - Senior Data Engineer

Madison Logic

Pune, Maharashtra, India (On-Site)
1 Month ago
Visa - Lead Data Science

Visa

Almaty, Almaty Region, Kazakhstan (Hybrid)
6 Months ago
Epic Games - Data Engineer

Epic Games

London, England, United Kingdom (On-Site)
7 Months ago
Western Digital - Intern - Business Systems Analyst

Western Digital

Prachin Buri, Thailand (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

PwC - Deals | Senior Associate Financial Due Diligence

PwC

Madrid, Community Of Madrid, Spain (On-Site)
10 Months ago
PwC - Senior Associate - Risk Assurance Services (Business Solution)

PwC

Ho Chi Minh City, Ho Chi Minh City, Vietnam (On-Site)
9 Months ago
Dave Ramsey - Software Engineer 3

Dave Ramsey

Franklin, Tennessee, United States (On-Site)
2 Months ago
Shield AI - Sr. Engineering Manager- Autonomy, Hivemind Pilot (R2424)

Shield AI

San Diego, California, United States (On-Site)
3 Weeks ago
Banyan Software - Sales and Marketing Manager

Banyan Software

Augusta, Georgia, United States (Remote)
2 Months ago
Zelis  - Claims Settlement Coordinator

Zelis

Hyderabad, Telangana, India (On-Site)
2 Months ago
Site Core - Senior Director, Solutions Engineering - America

Site Core

United States (On-Site)
3 Weeks ago
Mcdonalds - Franchising Recruitment Manager

Mcdonalds

London, England, United Kingdom (Hybrid)
1 Month ago
Guardian - Regional Sales Leader - Atlanta, GA

Guardian

Atlanta, Georgia, United States (Hybrid)
3 Weeks ago
Jumio - Head of Sales - Americas

Jumio

United States (Remote)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Thiruvananthapuram, Kerala, India

pentair - Sr Specialist Firmware

pentair

Noida, Uttar Pradesh, India (On-Site)
3 Months ago
rivos - CPU Physical Design - Full time

rivos

Bengaluru, Karnataka, India (On-Site)
10 Months ago
Wind River - Senior Engineer - Network Stack

Wind River

Bengaluru, Karnataka, India (Hybrid)
3 Months ago
Scopely - Senior Animator (2D)

Scopely

Bengaluru, Karnataka, India (Hybrid)
6 Months ago
Saama Technologies,  Inc  - Senior Site Reliability Engineer

Saama Technologies, Inc

Chennai, Tamil Nadu, India (On-Site)
10 Months ago
PhonePe - Senior Business Operations Analyst, Payments Growth

PhonePe

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Ciklum - Expert Android Developer

Ciklum

Chennai, Tamil Nadu, India (Hybrid)
10 Months ago
Blenheim Chalcot India - Senior Data Engineer

Blenheim Chalcot India

Mumbai, Maharashtra, India (On-Site)
3 Weeks ago
Unity - Global Indirect Tax Analyst

Unity

Bengaluru, Karnataka, India (On-Site)
2 Months ago
Anthology  Inc  - Amazon Connect Technical Engineer

Anthology Inc

Chennai, Tamil Nadu, India (Hybrid)
3 Months ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

OKX - Senior Business Intelligence Analyst, Growth

OKX

Singapore (On-Site)
3 Months ago
Meta - Data Scientist, Product Analytics

Meta

United States (Remote)
3 Months ago
Visa - Lead Data Science

Visa

Almaty, Almaty Region, Kazakhstan (Hybrid)
6 Months ago
Unity - Staff Data AI Engineer

Unity

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
1 Month ago
Rolls-Royce - Technical Data Scientist

Rolls-Royce

Derby, England, United Kingdom (Hybrid)
1 Month ago
Ubisoft - Data Analytics Manager

Ubisoft

Newcastle Upon Tyne, England, United Kingdom (Hybrid)
2 Months ago
Match Group - Senior Data Engineer, Trust & Safety

Match Group

New York, United States (Hybrid)
10 Months ago
endava - Lead Agile Business Analyst

endava

Brisbane, Queensland, Australia (On-Site)
2 Months ago
luxsoft - Data Engineer - PostgreSQL

luxsoft

Warsaw, Masovian Voivodeship, Poland (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Thiruvananthapuram, Kerala, India (On-Site)

Chicago, Illinois, United States (On-Site)

Berwyn, Pennsylvania, United States (Hybrid)

Berwyn, Pennsylvania, United States (Hybrid)

Berwyn, Pennsylvania, United States (Hybrid)

Thiruvananthapuram, Kerala, India (On-Site)

United States (Remote)

Berwyn, Pennsylvania, United States (Hybrid)

Thiruvananthapuram, Kerala, India (On-Site)

Thiruvananthapuram, Kerala, India (On-Site)

View All Jobs

Get notified when new jobs are added by Yodlee

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug