Lead Data Engineer

6 Months ago • 10 Years + • DevOps

Job Summary

Job Description

As a Data Engineer with ETL/ELT background, the candidate needs to design and develop reusable data ingestion processes from variety of sources and build data pipelines for Synapse Azure cloud data warehouse platform and reporting processes. Responsibilities include designing, developing, and implementing ETL processes on Pyspark, Azure Cloud using Databricks and Azure Synapse, demonstrating advanced SQL knowledge, writing optimized queries for faster data work flows, handling large volume data and working using different tools to derive the required solution, collaborating with onshore team, business analysts and other data engineering teams to ensure alignment of requirements, methodologies and best practices.
Must have:
  • Bachelor's Degree or master’s degree in Computer Science.
  • 10+ years of hands-on software engineering experience.
  • Proven work experience in Spark, Python, SQL, Any RDBMS.
  • Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS
  • Strong database fundamentals including SQL, performance and schema design.
  • Understanding of CI/CD framework
  • Ability to interpret/write custom shell scripts.
Good to have:
  • Python scripting
  • Experience with any cloud (Azure platform and Synapse/ AWS).
  • Experience with Git / Azure DevOps

Job Details

Description

Position at Apexon

Company Description: Apexon is a digital-first technology services firm backed by Goldman Sachs Asset Management and Everstone Capital. We specialize in accelerating business transformation and delivering human-centric digital experiences. For over 17 years, Apexon has been meeting customers wherever they are in the digital lifecycle and helping them outperform their competition through speed and innovation.

As a Data Engineer with ETL/ELT background, the candidate needs to design and develop reusable data ingestion processes from variety of sources and build data pipelines for Synapse Azure cloud data warehouse platform and reporting processes.


 
Responsibilities

·         Design, develop & implement ETL processes on Pyspark, Azure Cloud using Databricks and Azure Synapse

·         Advanced SQL knowledge, capable to write optimized queries for faster data work flows.

·         Must be extremely well versed with handling large volume data and work using different tools to derive the required solution.

·         Work with onshore team, business analysts and other data engineering teams to ensure alignment of requirements, methodologies and best practices

Requirements

·         Bachelor's Degree or master’s degree in Computer Science.

·         10+ years of hands-on software engineering experience.

·         Proven work experience in Spark , Python ,SQL , Any RDBMS.

·         Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS

·         Strong database fundamentals including SQL, performance and schema design.

·         Understanding of CI/CD framework is an added advantage.

·         Ability to interpret/write custom shell scripts. Python scripting is a plus.

·         Experience with any cloud is desired (Azure platform and Synapse/ AWS).

·         Experience with Git / Azure DevOps

  • To be able to work in a fast-paced agile development environment.

If this role sounds like you, we invite you to apply. Our recruiting team will be in touch shortly with qualified candidates regarding next steps.

Disclaimer: If you feel that this is a good match for your skillsets, please submit a current word version of your resume along with a cover letter describing your skills, experience and salary expectations. We are an Equal Opportunity Employer (EOE). You can read our job applicant privacy policy here.

Similar Jobs

HEAL Software  Inc  - Engineer – QA (Manual)

HEAL Software Inc

Bengaluru, Karnataka, India (On-Site)
6 Months ago
Google - Software Engineer II, Platform Storage, Google Cloud Platform

Google

Taipei City, Taiwan (On-Site)
2 Days ago
ByteDance - Senior Machine Learning Ops Engineer, ML System

ByteDance

Seattle, Washington, United States (On-Site)
6 Months ago
Saviynt - Technical Lead, Professional Services - NA

Saviynt

Bengaluru, Karnataka, India (Hybrid)
6 Months ago
Netflix - Broadcast Engineer, Live Broadcast Technology

Netflix

United States (Remote)
2 Months ago
Info Stretch - Senior Engineer

Info Stretch

Pune, Maharashtra, India (On-Site)
6 Months ago
ByteDance - Linux System Engineer

ByteDance

London, England, United Kingdom (On-Site)
2 Weeks ago
The Walt Disney Company - Senior Software Engineer

The Walt Disney Company

England, United Kingdom (On-Site)
2 Weeks ago
Bethesda - Senior DevOps Programmer

Bethesda

Austin, Texas, United States (On-Site)
1 Month ago
Zazz - Data Engineer

Zazz

(Remote)
3 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

BrightEdge - Devops Engineer

BrightEdge

Hyderabad, Telangana, India (On-Site)
6 Months ago
Google - Technical Solutions Engineer, AI/ML

Google

Bengaluru, Karnataka, India (On-Site)
1 Week ago
Cirrus Logic - Staff PDK Engineer

Cirrus Logic

Austin, Texas, United States (Hybrid)
5 Months ago
Tencent - Senior Backend Engineer for Global Realistic 3A Action Game

Tencent

Shenzhen, Guangdong Province, China (On-Site)
4 Months ago
ByteDance - Network Resource Management Specialist

ByteDance

Singapore (On-Site)
6 Months ago
Corsair - Automation Engineer

Corsair

Vietnam (On-Site)
1 Month ago
NVIDIA - Senior Functional Test Engineer

NVIDIA

Yokne'am Illit, North District, Israel (On-Site)
3 Months ago
Progres - Senior DBA Consultant - OpenEdge

Progres

(Remote)
2 Weeks ago
Google - Software Engineer, Site Reliability Engineering

Google

Dublin, County Dublin, Ireland (On-Site)
2 Days ago
Google - Database Engineer

Google

Fremont, California, United States (On-Site)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Hyderabad, Telangana, India

BOT VFX - Junior Business Analyst

BOT VFX

Chennai, Tamil Nadu, India (On-Site)
2 Days ago
Sprinkler - Senior Manager, Outbound Product Marketing — MEA & APAC (Geo-focused)

Sprinkler

Bengaluru, Karnataka, India (On-Site)
1 Day ago
Google - Strategic Partner Development Manager, gTech Sustainability

Google

Bengaluru, Karnataka, India (On-Site)
1 Week ago
Lakshya Digital - Senior Executive - People & Culture

Lakshya Digital

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Educational Initiatives - Associate Manager- Design

Educational Initiatives

Karnataka, India (Hybrid)
7 Months ago
Hitachi - Power platform Senior developer

Hitachi

Bengaluru, Karnataka, India (On-Site)
6 Months ago
Paytm - KAM - Enterprise Mid Market (Jaipur)

Paytm

Jaipur, Rajasthan, India (On-Site)
4 Months ago
Omnissa - C++ Engineering Manager

Omnissa

Bengaluru, Karnataka, India (Hybrid)
7 Months ago
MRI Software - Software Engineer - .Net, Blazor, ReactJS

MRI Software

Vadodara, Gujarat, India (Hybrid)
19 Hours ago
Capgemini - Business Solution Designer

Capgemini

Kolkata, West Bengal, India (On-Site)
1 Day ago

Get notifed when new similar jobs are uploaded

DevOps Jobs

Hitachi Digital Services - Container Security - Expert (Hyderabad, Bangalore, Pune)

Hitachi Digital Services

Bengaluru, Karnataka, India (Hybrid)
7 Months ago
Velotio Technologies - Cloud Security Engineer

Velotio Technologies

Maharashtra, India (Remote)
1 Month ago
One of Us - Tools Developer

One of Us

London, England, United Kingdom (Hybrid)
1 Month ago
Dream Sports - SDE - 1 - DevOps

Dream Sports

Mumbai, Maharashtra, India (On-Site)
6 Months ago
Google - Software Engineer, PhD

Google

Bengaluru, Karnataka, India (On-Site)
2 Weeks ago
PwC - Power Platform Developer Associate

PwC

Rome, Lazio, Italy (On-Site)
1 Month ago
Google - OTT Live Video Engineer

Google

New York, New York, United States (On-Site)
1 Week ago
Google - Technical Solutions Engineer, Data, Google Cloud

Google

Seoul, South Korea (On-Site)
2 Days ago
Omnissa - C++ Windows Internals Dev_MTS2/3 (2-7 Yrs)_Horizon Team

Omnissa

Bengaluru, Karnataka, India (Hybrid)
7 Months ago
Crunchyroll - DevOps Engineer, Core Infrastructure Engineering

Crunchyroll

San Francisco, California, United States (Hybrid)
2 Months ago

Get notifed when new similar jobs are uploaded

About The Company

United States (Remote)

Lansing, Michigan, United States (Remote)

Indianapolis, Indiana, United States (On-Site)

Indianapolis, Indiana, United States (On-Site)

Lansing, Michigan, United States (Hybrid)

Mechanicsburg, Pennsylvania, United States (On-Site)

Sunderland, England, United Kingdom (Hybrid)

Warsaw, Masovian Voivodeship, Poland (On-Site)

Dublin, County Dublin, Ireland (On-Site)

Poland (On-Site)

View All Jobs

Get notified when new jobs are added by Info Stretch

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug