Lead Data Engineer

6 Months ago • 10 Years + • DevOps

Job Summary

Job Description

As a Data Engineer with ETL/ELT background, the candidate needs to design and develop reusable data ingestion processes from variety of sources and build data pipelines for Synapse Azure cloud data warehouse platform and reporting processes. Responsibilities include designing, developing & implementing ETL processes on Pyspark, Azure Cloud using Databricks and Azure Synapse, advanced SQL knowledge, handling large volume data, and working with onshore teams, business analysts and other data engineering teams to ensure alignment of requirements, methodologies and best practices.
Must have:
  • Bachelor's Degree or master’s degree in Computer Science.
  • 10+ years of hands-on software engineering experience.
  • Proven work experience in Spark , Python ,SQL , Any RDBMS.
  • Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS
  • Strong database fundamentals including SQL, performance and schema design.
  • Ability to interpret/write custom shell scripts.
  • Experience with any cloud is desired (Azure platform and Synapse/ AWS).
Good to have:
  • Understanding of CI/CD framework
  • Python scripting

Job Details

Description

Position at Apexon

Company Description: Apexon is a digital-first technology services firm backed by Goldman Sachs Asset Management and Everstone Capital. We specialize in accelerating business transformation and delivering human-centric digital experiences. For over 17 years, Apexon has been meeting customers wherever they are in the digital lifecycle and helping them outperform their competition through speed and innovation.

As a Data Engineer with ETL/ELT background, the candidate needs to design and develop reusable data ingestion processes from variety of sources and build data pipelines for Synapse Azure cloud data warehouse platform and reporting processes.


 
Responsibilities

·         Design, develop & implement ETL processes on Pyspark, Azure Cloud using Databricks and Azure Synapse

·         Advanced SQL knowledge, capable to write optimized queries for faster data work flows.

·         Must be extremely well versed with handling large volume data and work using different tools to derive the required solution.

·         Work with onshore team, business analysts and other data engineering teams to ensure alignment of requirements, methodologies and best practices

Requirements

·         Bachelor's Degree or master’s degree in Computer Science.

·         10+ years of hands-on software engineering experience.

·         Proven work experience in Spark , Python ,SQL , Any RDBMS.

·         Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS

·         Strong database fundamentals including SQL, performance and schema design.

·         Understanding of CI/CD framework is an added advantage.

·         Ability to interpret/write custom shell scripts. Python scripting is a plus.

·         Experience with any cloud is desired (Azure platform and Synapse/ AWS).

·         Experience with Git / Azure DevOps

  • To be able to work in a fast-paced agile development environment.

If this role sounds like you, we invite you to apply. Our recruiting team will be in touch shortly with qualified candidates regarding next steps.

Disclaimer: If you feel that this is a good match for your skillsets, please submit a current word version of your resume along with a cover letter describing your skills, experience and salary expectations. We are an Equal Opportunity Employer (EOE). You can read our job applicant privacy policy here.

Similar Jobs

Litmus - Senior Software Engineer - Golang

Litmus

Pune, Maharashtra, India (On-Site)
18 Hours ago
Hitachi - Windchill PLM Consultant

Hitachi

Chennai, Tamil Nadu, India (On-Site)
6 Months ago
Ness Digital - Cloud Systems Administrator

Ness Digital

Timișoara, Timiș, Romania (Hybrid)
2 Months ago
Offworld - DevOps Engineer

Offworld

New Westminster, British Columbia, Canada (On-Site)
1 Month ago
Transaction Network Services - Quality Assurance Engineer - Test Automation

Transaction Network Services

Coimbatore, Tamil Nadu, India (On-Site)
9 Months ago
ByteDance - Backend Software Engineer - Foundational Technology

ByteDance

Singapore (On-Site)
1 Month ago
Rackspace Technology - Service Delivery Manager

Rackspace Technology

India (Remote)
2 Months ago
Google - Software Developer II, Site Reliability Development

Google

Waterloo, Ontario, Canada (On-Site)
2 Days ago
Netflix - Engineering Manager - Edge Gateway & Services

Netflix

United States (Remote)
1 Week ago
Microsoft - Azure Infrastructure Specialist

Microsoft

Dublin, County Dublin, Ireland (On-Site)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

ByteDance - Site Reliability Engineer, Edge Services

ByteDance

San Jose, California, United States (On-Site)
2 Weeks ago
ByteDance - Network Reliability Engineer - Physical Network Infrastructure

ByteDance

Singapore (On-Site)
3 Months ago
HEAL Software  Inc  - Engineer – QA (Manual)

HEAL Software Inc

Bengaluru, Karnataka, India (On-Site)
6 Months ago
jetbrains - On-duty DevOps engineer

jetbrains

Yerevan, Yerevan, Armenia (On-Site)
1 Day ago
Tesla - Cellular Software Development Engineer

Tesla

Brandenburg, Germany (On-Site)
2 Months ago
Epic Games - Build Engineer, Fortnite

Epic Games

(On-Site)
2 Months ago
Google - Network Operations Residency Program

Google

Bengaluru, Karnataka, India (On-Site)
2 Days ago
Microsoft - Principal Software Engineering Manager

Microsoft

Vancouver, British Columbia, Canada (On-Site)
1 Week ago
ByteDance - Research Scientist, Foundation Model, Speech & Audio

ByteDance

Seattle, Washington, United States (On-Site)
6 Months ago
Google - Senior Systems Engineer, Cloud Filestore, Site Reliability Engineering

Google

Dublin, County Dublin, Ireland (On-Site)
2 Days ago

Get notifed when new similar jobs are uploaded

Jobs in Bengaluru, Karnataka, India

Google - Senior Software Engineer, Full Stack, Geo

Google

Hyderabad, Telangana, India (On-Site)
1 Week ago
Warner Bros Games - Staff Software Engineer, C++ Player

Warner Bros Games

Bengaluru, Karnataka, India (Hybrid)
4 Weeks ago
PwC - IN_Senior Associate _ERP_Utility Transformation_Advisory_Jaipur

PwC

Jaipur, Rajasthan, India (On-Site)
4 Months ago
PwC - IN-Manager_Industrial Decarbonization_Decarbonization_Advisory_Ahmedabad

PwC

Ahmedabad, Gujarat, India (On-Site)
7 Months ago
Head Digital Works - Data Scientist

Head Digital Works

Hyderabad, Telangana, India (On-Site)
10 Months ago
Assystems - Sr. Structural Design Engineer - Buildings

Assystems

Hyderabad, Telangana, India (On-Site)
6 Months ago
Shyft Labs - Data Engineer

Shyft Labs

Noida, Uttar Pradesh, India (Hybrid)
3 Weeks ago
Orion Innovation - Data Engineer-AI,ML

Orion Innovation

Chennai, Tamil Nadu, India (On-Site)
6 Months ago
Boomi - Software Quality Senior Engineer

Boomi

Bengaluru, Karnataka, India (On-Site)
18 Hours ago
InvenioLSI - Sales Executive

InvenioLSI

New Delhi, Delhi, India (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

DevOps Jobs

Google - Customer Engineer, Platform, Google Cloud

Google

Portland, Oregon, United States (On-Site)
1 Week ago
The Walt Disney Company - Build & Release Engineer

The Walt Disney Company

California, United States (On-Site)
1 Month ago
Google - Cloud Technical Solutions Engineering Manager

Google

Tokyo, Japan (On-Site)
2 Days ago
Info Stretch - Lead Data Engineer

Info Stretch

Pune, Maharashtra, India (On-Site)
6 Months ago
Rackspace Technology - Cloud Database Engineer I/II

Rackspace Technology

Gurugram, Haryana, India (Remote)
2 Weeks ago
Interactive Brokers - Senior DevOps/Software Engineer

Interactive Brokers

Greenwich, Connecticut, United States (Hybrid)
6 Months ago
N-iX - Senior Python Engineer

N-iX

Ukraine (Remote)
1 Week ago
ByteDance - Backend Software Engineer (Business Infra), ByteCloud - 2025 Start

ByteDance

Singapore (On-Site)
6 Months ago
Google - Workspace Engineer

Google

Bengaluru, Karnataka, India (On-Site)
2 Weeks ago

Get notifed when new similar jobs are uploaded

About The Company

United States (Remote)

Lansing, Michigan, United States (Remote)

Indianapolis, Indiana, United States (On-Site)

Indianapolis, Indiana, United States (On-Site)

Lansing, Michigan, United States (Hybrid)

Mechanicsburg, Pennsylvania, United States (On-Site)

Sunderland, England, United Kingdom (Hybrid)

Warsaw, Masovian Voivodeship, Poland (On-Site)

Dublin, County Dublin, Ireland (On-Site)

Poland (On-Site)

View All Jobs

Get notified when new jobs are added by Info Stretch

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug