Senior Big Data Engineer

2 Months ago • 8 Years + • Data Analysis

Job Summary

Job Description

This role involves designing and building ingestion pipelines using ELT and schema on read in Databricks Delta lake. The candidate will develop "Transformations" aspects using ELT framework to modernize the Ingestion pipelines and build data transformations at scale. They will also provide technical expertise in the areas of design and implementation of Ratings Data Ingestion pipelines with modern AWS cloud and other technologies, ensure data governance principles are adopted, participate in the development of Ingestion pipelines, and stay updated with emerging trends in Big data and cloud technologies.
Must have:
  • 8+ years of application development experience.
  • Experience with Ratings Data Ingestion pipelines on AWS cloud.
  • Exposure to software development lifecycle methodologies like Agile.
  • Experience working with high-volume data and computationally intensive systems.
  • Garbage collection friendly programming experience - tuning Java garbage collection & performance is a must.
  • Proficiency in the development environment, including IDE, web & application server, GIT, Continuous Integration, unit-testing tools, and defect management tools
  • Excellent communication skills are essential, with strong verbal and writing proficiencies.
  • Mentor teams, innovate and experiment, give a face to business ideas, and present to key stakeholders.
Good to have:
  • Domain knowledge in the Financial Industry and Capital Markets is a plus.
Perks:
  • Access to trainings and certifications
  • Bonuses
  • Aids
  • Socializing activities
  • Attractive compensation

Job Details

Job ID 5627

Why Ness

We know that people are our greatest asset. Our staff’s professionalism, innovation, teamwork, and dedication to excellence have helped us become one of the world’s leading technology companies. It is these qualities that are vital to our continued success. As a Ness employee, you will be working on products and platforms for some of the most innovative software companies in the world.

You’ll gain knowledge working alongside other highly skilled professionals that will help accelerate your career progression.

You’ll also benefit from an array of advantages like access to trainings and certifications, bonuses, and aids, socializing activities, and attractive compensation.

Requirements and responsibilities

We are looking for a self-motivated, enthusiastic, and passionate software engineer to develop technology solutions for the Ness marketplace product.

The ideal candidate thrives in a highly technical role and will design and develop software using cutting-edge web applications, data pipelines, big data, machine learning, and multi-cloud technologies.

The development is already underway, so the candidate would be expected to get up to speed quickly and start contributing.

You will enable Ness business to showcase our proprietary Ness data, combine it with “curated” alternative data, further enrich it with value-add services from Kenshoo and others, and deliver it via the clients’ channel of choice to help them make better investment and business decisions, with confidence.

What you can expect: An unmatched experience in handling huge volumes of data, analytics, visualization, and services over cloud technologies along with appreciation in product development life cycle to convert an idea into revenue generating stream.

What you'll do

  • Design & Build ingestion pipelines using ELT and schema on read in Databricks Delta lake;
  • Design & Develop “Transformations” aspects using ELT framework to modernize the Ingestion pipelines and build data transformations at scale;
  • Provide technical expertise in the areas of design and implementation of Ratings Data Ingestion pipelines with modern AWS cloud and other technologies such as S3, Hive, Databricks, Scala, Python, and Spark;
  • Build and maintain a data environment for speed, accuracy, consistency, and \"up\" time;
  • Work closely with other data teams & Data Science team and participate in the development of Ingestion pipelines;
  • Ensure data governance principles are adopted, data quality checks and data lineage are implemented in each hop of the data;
  • Be in tune with emerging trends Big data and cloud technologies and participate in the evaluation of new technologies;
  • Ensure compliance through adopting enterprise standards and promoting best practices/guiding principles aligned with organization standards.

What you'll bring

  • A minimum of 8+ years of significant experience in application development;
  • Previous experience in the areas of design and implementation of Ratings Data Ingestion pipelines with modern AWS cloud and other technologies such as S3, Hive, Databricks, Scala, Python, and Spark;
  • Development, design, and architecture exposure & the ability to ensure quality across various technology components that are developed by geographically diversified software engineers with superior knowledge of system architecture, object-oriented design, and design patterns;
  • Proficient with software development lifecycle (SDLC) methodologies like Agile and test-driven development;
  • Experience working with high-volume data and computationally intensive systems;
  • Garbage collection friendly programming experience - tuning Java garbage collection & performance is a must;
  • Proficiency in the development environment, including IDE, web & application server, GIT, Continuous Integration, unit-testing tools, and defect management tools Domain knowledge in the Financial Industry and Capital Markets is a plus;
  • Excellent communication skills are essential, with strong verbal and writing proficiencies;
  • Mentor teams, innovate and experiment, give a face to business ideas, and present to key stakeholders.

Not checking every single requirement?

If this role sounds good to you, even if you don’t meet every single bullet point in the job description, we encourage you to apply anyway. For most of the candidates that applied, we found a role that was a very good fit with their skills.

Let’s meet and you may just be the right candidate for one of our roles.

At Ness Digital Engineering we are willing to build a work culture that is based on diversification, inclusion, and authenticity.

Similar Jobs

mighty koi - Senior Technical Artist

mighty koi

Warsaw, Masovian Voivodeship, Poland (On-Site)
2 Months ago
Whatnot - Director, Logistics Engineering

Whatnot

San Francisco, California, United States (On-Site)
1 Month ago
Yodlee - Senior Talent Acquisition Specialist

Yodlee

Berwyn, Pennsylvania, United States (On-Site)
4 Weeks ago
Reddit - Staff Frontend Engineer (Internal Applications)

Reddit

United States (Remote)
1 Month ago
PwC - Senior Associate - Power BI - Data and Analytics - Advisory

PwC

Mumbai, Maharashtra, India (On-Site)
4 Weeks ago
Capco - SMS Analyst - Data Intelligence

Capco

Brazil (Remote)
5 Months ago
PayPal - Sr. Engineer -Data Infra - Venmo

PayPal

San Jose, California, United States (Hybrid)
4 Weeks ago
Apple - Senior / Staff Data Infrastructure Engineer for Lakehouse, Apple Data Platform

Apple

Cupertino, California, United States (On-Site)
3 Weeks ago
Opendoor - Senior Data Engineer

Opendoor

India (Remote)
1 Month ago
Canva - Staff Data Scientist – Analytics Prototyping and Development, Remote across ANZ

Canva

Sydney, New South Wales, Australia (Remote)
7 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Valeo - eRO Counter

Valeo

Batam, Riau Islands, Indonesia (On-Site)
1 Month ago
OKX - Finance Business Partner Senior Manager

OKX

Singapore (On-Site)
1 Month ago
Tesla - Automotive Mechatronics/Auto Mechanic

Tesla

Frankfurt Am Main, Hessen, Germany (On-Site)
5 Months ago
Lurkit - Platform Execution Specialist (Project-Based)

Lurkit

Stockholm, Stockholm County, Sweden (Remote)
2 Months ago
Ziff Davis - Growth Marketing Associate

Ziff Davis

Boston, Massachusetts, United States (Remote)
3 Weeks ago
HHA Exchange - Events & Association Marketing Manager

HHA Exchange

New York, United States (Hybrid)
3 Weeks ago
The Walt Disney Company - Senior QA Analyst

The Walt Disney Company

Glendale, California, United States (On-Site)
5 Months ago
Coherent corp. - Account Manager, Directed Energy, Laser Components

Coherent corp.

Santa Clara, California, United States (Remote)
2 Months ago
Adyen - Senior Regulatory Counsel

Adyen

Singapore (On-Site)
1 Month ago
Capgemini - Industrial Operations Engineer

Capgemini

Noida, Uttar Pradesh, India (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Timișoara, Timiș, Romania

Crowd Strick - Sr. Software Engineer

Crowd Strick

Romania (Remote)
1 Month ago
Marvell - Senior Engineer, Physical Design

Marvell

Bucharest, Bucharest, Romania (On-Site)
1 Month ago
Google - Software Engineer III, Android

Google

Bucharest, Bucharest, Romania (On-Site)
2 Months ago
endava - Senior Python Automation Tester

endava

Cluj-Napoca, Cluj County, Romania (On-Site)
4 Weeks ago
Threat connect - .NET Software Engineer (maternity backfill)

Threat connect

Cluj-Napoca, Cluj County, Romania (Remote)
1 Month ago
Adobe - Senior Software Engineer, Site Reliability Engineering

Adobe

Bucharest, Bucharest, Romania (On-Site)
1 Year ago
luxsoft - Senior DevOps Engineer (with Python experience)

luxsoft

Romania (Remote)
1 Month ago
Ubisoft - Application Specialist

Ubisoft

Bucharest, Bucharest, Romania (Hybrid)
3 Months ago
Electronic Arts - Senior Analyst - EA Sports FC

Electronic Arts

Bucharest, Bucharest, Romania (Hybrid)
3 Weeks ago
London stock Exchange - Automation Senior Engineer, Quality Engineering

London stock Exchange

Bucharest, Bucharest, Romania (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Yodo1 - Data Engineer

Yodo1

(Remote)
10 Months ago
binance - Data Scientist (LLM)

binance

Taipei City, Taiwan (On-Site)
7 Months ago
Yodlee - Senior Business Analyst

Yodlee

Thiruvananthapuram, Kerala, India (On-Site)
3 Months ago
Flow - Senior / Staff Data Engineer

Flow

New York, New York, United States (On-Site)
2 Months ago
Diligent Corporation - Product Data Scientist

Diligent Corporation

Budapest, Hungary (On-Site)
2 Months ago
Nagarro - Staff Consultant ,SAP Analytics Data Manageme

Nagarro

India (Remote)
8 Months ago
The Walt Disney Company - Business Systems Analyst

The Walt Disney Company

Hong Kong (On-Site)
3 Months ago
ISS Stoxx - Junior Data Collections Analyst

ISS Stoxx

Makati, Metro Manila, Philippines (Hybrid)
1 Month ago
Rockstar Games - Senior Data Engineer

Rockstar Games

Andover, Massachusetts, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Ontario, Canada (On-Site)

Ontario, Canada (On-Site)

Timișoara, Timiș, Romania (On-Site)

Košice, Košice Region, Slovakia (Remote)

New York, United States (Hybrid)

Iași, Iași County, Romania (Remote)

Prague, Prague, Czechia (On-Site)

Iași, Iași County, Romania (On-Site)

Timișoara, Timiș, Romania (On-Site)

View All Jobs

Get notified when new jobs are added by Ness

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug