Lead Data Engineer

5 Months ago • 10 Years + • DevOps

Job Summary

Job Description

As a Data Engineer with ETL/ELT background, the candidate needs to design and develop reusable data ingestion processes from variety of sources and build data pipelines for Synapse Azure cloud data warehouse platform and reporting processes. Responsibilities include designing, developing & implementing ETL processes on Pyspark, Azure Cloud using Databricks and Azure Synapse, advanced SQL knowledge, working with large volume data, and collaborating with onshore teams and other data engineering teams to ensure alignment of requirements, methodologies and best practices.
Must have:
  • Bachelor's Degree or master’s degree in Computer Science.
  • 10+ years of hands-on software engineering experience.
  • Proven work experience in Spark , Python ,SQL , Any RDBMS.
  • Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS
  • Strong database fundamentals including SQL, performance and schema design.
  • Understanding of CI/CD framework
  • Ability to interpret/write custom shell scripts. Python scripting is a plus.
Good to have:
  • Experience with any cloud is desired (Azure platform and Synapse/ AWS).
  • Experience with Git / Azure DevOps

Job Details

Description

Position at Apexon

Company Description: Apexon is a digital-first technology services firm backed by Goldman Sachs Asset Management and Everstone Capital. We specialize in accelerating business transformation and delivering human-centric digital experiences. For over 17 years, Apexon has been meeting customers wherever they are in the digital lifecycle and helping them outperform their competition through speed and innovation.

As a Data Engineer with ETL/ELT background, the candidate needs to design and develop reusable data ingestion processes from variety of sources and build data pipelines for Synapse Azure cloud data warehouse platform and reporting processes.


 
Responsibilities

·         Design, develop & implement ETL processes on Pyspark, Azure Cloud using Databricks and Azure Synapse

·         Advanced SQL knowledge, capable to write optimized queries for faster data work flows.

·         Must be extremely well versed with handling large volume data and work using different tools to derive the required solution.

·         Work with onshore team, business analysts and other data engineering teams to ensure alignment of requirements, methodologies and best practices

Requirements

·         Bachelor's Degree or master’s degree in Computer Science.

·         10+ years of hands-on software engineering experience.

·         Proven work experience in Spark , Python ,SQL , Any RDBMS.

·         Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS

·         Strong database fundamentals including SQL, performance and schema design.

·         Understanding of CI/CD framework is an added advantage.

·         Ability to interpret/write custom shell scripts. Python scripting is a plus.

·         Experience with any cloud is desired (Azure platform and Synapse/ AWS).

·         Experience with Git / Azure DevOps

  • To be able to work in a fast-paced agile development environment.

If this role sounds like you, we invite you to apply. Our recruiting team will be in touch shortly with qualified candidates regarding next steps.

Disclaimer: If you feel that this is a good match for your skillsets, please submit a current word version of your resume along with a cover letter describing your skills, experience and salary expectations. We are an Equal Opportunity Employer (EOE). You can read our job applicant privacy policy here.

Similar Jobs

PwC - Senior Associate _ Automation Tester_ Emerging  Technologies_ Advisory_ Bengaluru

PwC

Bengaluru, Karnataka, India (On-Site)
6 Months ago
Visa - Staff Systems Engineer - DevEx

Visa

Singapore, Singapore (On-Site)
5 Months ago
PwC - Manager– Oracle – Oracle DBA – Advisory  - Bangalore

PwC

Bengaluru, Karnataka, India (On-Site)
6 Months ago
Luxoft - Murex Front Office Developer

Luxoft

Hyderabad, Telangana, India (On-Site)
4 Months ago
ByteDance - Linux System Engineer

ByteDance

London, England, United Kingdom (On-Site)
6 Days ago
Sonar Source - Site Reliability Engineer

Sonar Source

Bochum, North Rhine-Westphalia, Germany (On-Site)
5 Months ago
Epic Games - Senior DevOps Programmer

Epic Games

Cary, North Carolina, United States (On-Site)
1 Month ago
Argus Labs - Site Reliability Engineer (LATAM)

Argus Labs

(Remote)
1 Week ago
Velotio Technologies - Lead Engineer (DevOps OpenShift)

Velotio Technologies

Maharashtra, India (Remote)
2 Weeks ago
Onward Search - Senior Cloud Engineer

Onward Search

Santa Monica, California, United States (Hybrid)
1 Week ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

ByteDance - Research Scientist, Multimodality

ByteDance

San Jose, California, United States (On-Site)
5 Months ago
ION - Technical Support Analyst, Jersey City - 9781

ION

Jersey City, New Jersey, United States (On-Site)
5 Months ago
Nagarro - Staff Engineer (Cloud Infrastructure)

Nagarro

Gurugram, Haryana, India (On-Site)
5 Months ago
Tencent - Site Reliability Engineer

Tencent

(On-Site)
3 Months ago
CloudLinux - Senior Go Developer

CloudLinux

Masovian Voivodeship, Poland (Remote)
1 Week ago
Nielsen Holdings - Software Engineer - Bigdata (Java/Scala and SQL)

Nielsen Holdings

Bengaluru, Karnataka, India (Hybrid)
5 Months ago
Luxoft - Murex Datamart Production Support

Luxoft

Hyderabad, Telangana, India (On-Site)
4 Months ago
The Walt Disney Company - Senior Systems Engineer, Data Services [Database Administration]

The Walt Disney Company

Vancouver, British Columbia, Canada (On-Site)
4 Months ago
NVIDIA - Design Verification Engineer (RDSS Intern)

NVIDIA

Hsinchu, Hsinchu City, Taiwan (On-Site)
2 Months ago
ION - Senior Technical Consultant – IT2

ION

Central Sulawesi, Indonesia (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Pune, Maharashtra, India

Tesla - Order Operations Specialist

Tesla

Mumbai, Maharashtra, India (On-Site)
1 Month ago
Warner Bros Games - Staff Software Engineer - Golang - QoE Platform

Warner Bros Games

Bengaluru, Karnataka, India (Hybrid)
3 Weeks ago
Omnissa - Staff Engineer (Data Science)

Omnissa

Bengaluru, Karnataka, India (Hybrid)
4 Months ago
Luxoft - Lead Python & C++ Engineer

Luxoft

Chennai, Tamil Nadu, India (On-Site)
3 Months ago
Tenstorrent - Staff Design Verification Engineer - CPU Subsystem

Tenstorrent

Bengaluru, Karnataka, India (On-Site)
6 Months ago
N-iX - Senior UKG Integrator (Consultant)

N-iX

India (Remote)
1 Week ago
Rackspace Technology - Sales Operations Analyst

Rackspace Technology

India (Remote)
4 Months ago
CloudHire - Senior Backend Engineer - TypeScript

CloudHire

Hyderabad, Telangana, India (Remote)
6 Days ago
Paytm - Sales Team Lead- Bangalore

Paytm

Bengaluru, Karnataka, India (On-Site)
3 Months ago
PhonePe - Software Engineer (5-7 Yrs) - Core

PhonePe

Bengaluru, Karnataka, India (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

DevOps Jobs

ION - Cloud Engineer Kubernetes

ION

Collecchio, Emilia-Romagna, Italy (Hybrid)
5 Months ago
Axon - Senior Privacy Engineer

Axon

Scottsdale, Arizona, United States (Hybrid)
4 Months ago
N-iX - Senior Engineer with AWS Greengrass Expertise

N-iX

Ukraine (Remote)
1 Month ago
SmileGate - Build Manager [LOST ARK Mobile]

SmileGate

Seongnam-si, Gyeonggi-do, South Korea (On-Site)
2 Months ago
GoReel - DevOps Lead

GoReel

Romania (Remote)
5 Days ago
Innoactive - Software Engineer

Innoactive

(Remote)
3 Months ago
ByteDance - Site Reliability Engineer Lead, Security Engineering

ByteDance

Singapore (On-Site)
5 Months ago
Jagex - IT Infrastructure Lead

Jagex

Cambridge, England, United Kingdom (Hybrid)
6 Days ago
Scopely - Director, Cloud FinOps

Scopely

Canada (Remote)
1 Month ago
Bethesda - Senior DevOps Programmer

Bethesda

Montreal, Quebec, Canada (On-Site)
3 Days ago

Get notifed when new similar jobs are uploaded

About The Company

Lansing, Michigan, United States (Remote)

United States (Remote)

Indianapolis, Indiana, United States (On-Site)

Indianapolis, Indiana, United States (On-Site)

Lansing, Michigan, United States (Hybrid)

Mechanicsburg, Pennsylvania, United States (On-Site)

Sunderland, England, United Kingdom (Hybrid)

Warsaw, Masovian Voivodeship, Poland (On-Site)

Dublin, County Dublin, Ireland (On-Site)

View All Jobs

Get notified when new jobs are added by Info Stretch

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug