Senior Data Engineer

3 Months ago • 5-5 Years • Data Analyst

Job Summary

Job Description

Senior Data Engineer needed to design, implement and maintain high-performance big data infrastructure. 5+ years experience in distributed computing, Python, SQL, Apache Airflow and Google Cloud Platform is required.
Must have:
  • Big Data Systems
  • Python Programming
  • Google Cloud
  • Apache Airflow
Good to have:
  • Kubernetes
  • REST APIs
  • Object Oriented
  • Network Programming

Job Details

About the job

Equal Opportunity Employer

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, protected veteran status, or disability status. EEO/AA/M/F/Disabled/Vets

Job Description :

Senior Data Engineer

One of the most innovative and high-profile teams in News Corp is looking for a seasoned data engineer to help accelerate its vision. The role will involve combining all of News Corp’s data across categories, countries and organizations into a singular view of a customer enabling engagement, measurement and targeting. The HQData Senior Data Engineer will partner closely with product, data & engineering teams to help make this data available and accessible to our businesses, teams and partners in order to drive revenue and improve the customer experience.

Responsibilities

  • Design, implement and maintain high performance big data infrastructure/systems & big data processing pipelines scaling to billions of structured and unstructured events daily
  • Design, implement and maintain deep integration with up-stream systems
  • Monitor performance of the data platform and optimize as needed
  • Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)
  • Support product with the overall roadmap and ensure updates to senior leadership are 100% technically correct.
  • Data analysis, understanding of business requirements and translation into logical pipelines & processes
  • Conduct timely and effective research in response to specific requests (e.g. data collection, summarization, analysis, and synthesis of relevant data and information)
  • Evaluate and prototype new technologies in the area of data processing
  • Think quickly, communicate clearly and work collaboratively with product, data, engineering, QA and operations teams
  • High energy level, strong team player and good work ethic

Technologies we use

  • Google Cloud Platform (Storage, Dataflow, BigQuery, DataStudio, DataTransfer, Composer)
  • Python, R, Spark (PySpark)
  • Kubernetes
  • Airflow
  • Git for source code management
  • Jira

Qualifications

  • BS in Computer Science or other technical discipline
  • 5+ years of experience in designing and developing big data processing systems using distributed computing
  • 5+ years in Python Programming
  • 5+ years experience in SQL
  • 4+ years experience in Apache Airflow
  • Working with REST APIs
  • 5+ years of Hands on experience in Google Cloud Platform
  • Expert knowledge in optimizing complex SQL queries
  • Experience with job orchestration tools
  • An affinity for automation
  • Familiarity with networking and network application programming, including HTTP/HTTPS, JSON, and REST APIs
  • Experience with at least one object oriented language (ex: Java)
  • Strong OO design, data structure, and algorithm design skills

This job is posted with NTS Technology Services Pvt. Ltd.

Job Category:

Similar Jobs

GoTo Group - Software Engineer - Comms Platform

GoTo Group

Bengaluru, Karnataka, India (On-Site)
4 Months ago
Dream Sports - Quality Assurance- Engineer

Dream Sports

Mumbai, Maharashtra, India (On-Site)
1 Month ago
ASSIST Software - Java Developer

ASSIST Software

Suceava, Suceava County, Romania (Remote)
3 Months ago
Truecaller - Senior Android Engineer

Truecaller

Stockholm, Stockholm County, Sweden (On-Site)
2 Months ago
Jagex - Senior Engine Engineer

Jagex

Cambridge, England, United Kingdom (On-Site)
2 Months ago
Scopely - Lead Product Analyst - Monopoly Go!

Scopely

Tel Aviv-Yafo, Tel Aviv District, Israel (Hybrid)
3 Months ago
SmileGate - Game Data Engineer [LOST ARK]

SmileGate

Seongnam-si, Gyeonggi-do, South Korea (On-Site)
1 Month ago
DraftKings - Analyst I, Sportsbook

DraftKings

Boston, Massachusetts, United States (On-Site)
4 Weeks ago
Microsoft - Supply Chain Data Analyst

Microsoft

Reston, Virginia, United States (Hybrid)
1 Month ago
Thumbtack - Lead Data Scientist, Business Analytics

Thumbtack

United States (Remote)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

The Walt Disney Company - Sr Software Engineer

The Walt Disney Company

San Francisco, California, United States (On-Site)
1 Month ago
Luxoft - Senior OpenEdge Engineer

Luxoft

Wrocław, Lower Silesian Voivodeship, Poland (On-Site)
2 Months ago
Microsoft - Software Engineer II - Mesh Experiences

Microsoft

Redmond, Washington, United States (On-Site)
1 Month ago
Mashgin - Senior Software Engineer, Infrastructure

Mashgin

Palo Alto, California, United States (Hybrid)
4 Months ago
Yodo1 - Unity3D Mobile SDK Team Lead

Yodo1

(Remote)
7 Months ago
Nagarro - Staff Engineer, Java Fullstack

Nagarro

Mumbai, Maharashtra, India (On-Site)
4 Months ago
Thatgamecompany - Game Engine Engineer (Optimization) - China

Thatgamecompany

Shanghai, Shanghai, China (On-Site)
5 Months ago
Sabre India - Data Scientist

Sabre India

Bengaluru, Karnataka, India (On-Site)
3 Months ago
Meta - Software Engineer (Leadership) - Machine Learning

Meta

Menlo Park, California, United States (Remote)
3 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Bengaluru, Karnataka, India

Nagarro - Principal Engineer, Java Fullstack

Nagarro

Mumbai, Maharashtra, India (On-Site)
4 Months ago
Egnyte - Sr. Customer Success Manager

Egnyte

India (Remote)
1 Month ago
DNEG - Pipeline TD

DNEG

Mumbai, Maharashtra, India (On-Site)
7 Months ago
PwC - IN_Manager_Credit Cards_FS Tech_Advisory_Mumbai

PwC

Mumbai, Maharashtra, India (On-Site)
4 Months ago
Highspot - Engineering Manager - Analytics & Data Engineering

Highspot

Hyderabad, Telangana, India (Hybrid)
4 Months ago
ByteDance - Innovation Tech Solution Sales

ByteDance

Gurugram, Haryana, India (On-Site)
4 Weeks ago
Nomiso - Data Scientist

Nomiso

Bengaluru, Karnataka, India (On-Site)
7 Months ago
bosh group india - PreSales/ Solution Architect- Enterprise

bosh group india

Karnataka, India (On-Site)
2 Months ago
Nagarro - Staff Engineer, Frontend AngularJS

Nagarro

India (Remote)
4 Months ago
Extreme Network - SR PROGRAMMER - Oracle Fusion Cloud- VBCS/ BI Reports/ OTBI/FRS & SmartView

Extreme Network

Chennai, Tamil Nadu, India (Hybrid)
4 Months ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

HypeHype  Inc  - Lead Data Analyst

HypeHype Inc

Helsinki, Uusimaa, Finland (On-Site)
8 Months ago
Trend Micro - (Sr.) Data Engineer/AI Trainer

Trend Micro

Taipei City, Taiwan (On-Site)
4 Months ago
Tripledot Studios - Product Analyst

Tripledot Studios

Jakarta, Jakarta, Indonesia (Hybrid)
4 Weeks ago
PwC - IN-Senior Associate _.Net Developer _Data & Analytics _Advisory _PAN India

PwC

Kolkata, West Bengal, India (On-Site)
4 Months ago
ION - Data Operations

ION

Noida, Uttar Pradesh, India (On-Site)
3 Months ago
Luxoft - Scala Tooling Visualization Developer

Luxoft

(Remote)
3 Months ago
Tripledot Studios - Senior Product Analyst

Tripledot Studios

London, England, United Kingdom (Hybrid)
2 Months ago
Playrix - Senior Data Analyst (Attribution)

Playrix

Ukraine (Remote)
4 Months ago
Steer Studios - Data Engineer

Steer Studios

Riyadh, Riyadh Province, Saudi Arabia (On-Site)
3 Months ago
PwC - Consulting - Technology - Data Manager - Amman

PwC

Amman, Amman Governorate, Jordan (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded