Senior Big Data Engineer

1 Month ago • 8 Years +

Job Summary

Job Description

This role involves designing and building ingestion pipelines using ELT and schema on read in Databricks Delta lake. The candidate will develop "Transformations" aspects using ELT framework to modernize the Ingestion pipelines and build data transformations at scale. They will also provide technical expertise in the areas of design and implementation of Ratings Data Ingestion pipelines with modern AWS cloud and other technologies, ensure data governance principles are adopted, participate in the development of Ingestion pipelines, and stay updated with emerging trends in Big data and cloud technologies.
Must have:
  • 8+ years of application development experience.
  • Experience with Ratings Data Ingestion pipelines on AWS cloud.
  • Exposure to software development lifecycle methodologies like Agile.
  • Experience working with high-volume data and computationally intensive systems.
  • Garbage collection friendly programming experience - tuning Java garbage collection & performance is a must.
  • Proficiency in the development environment, including IDE, web & application server, GIT, Continuous Integration, unit-testing tools, and defect management tools
  • Excellent communication skills are essential, with strong verbal and writing proficiencies.
  • Mentor teams, innovate and experiment, give a face to business ideas, and present to key stakeholders.
Good to have:
  • Domain knowledge in the Financial Industry and Capital Markets is a plus.
Perks:
  • Access to trainings and certifications
  • Bonuses
  • Aids
  • Socializing activities
  • Attractive compensation

Job Details

Job ID 5627

Why Ness

We know that people are our greatest asset. Our staff’s professionalism, innovation, teamwork, and dedication to excellence have helped us become one of the world’s leading technology companies. It is these qualities that are vital to our continued success. As a Ness employee, you will be working on products and platforms for some of the most innovative software companies in the world.

You’ll gain knowledge working alongside other highly skilled professionals that will help accelerate your career progression.

You’ll also benefit from an array of advantages like access to trainings and certifications, bonuses, and aids, socializing activities, and attractive compensation.

Requirements and responsibilities

We are looking for a self-motivated, enthusiastic, and passionate software engineer to develop technology solutions for the Ness marketplace product.

The ideal candidate thrives in a highly technical role and will design and develop software using cutting-edge web applications, data pipelines, big data, machine learning, and multi-cloud technologies.

The development is already underway, so the candidate would be expected to get up to speed quickly and start contributing.

You will enable Ness business to showcase our proprietary Ness data, combine it with “curated” alternative data, further enrich it with value-add services from Kenshoo and others, and deliver it via the clients’ channel of choice to help them make better investment and business decisions, with confidence.

What you can expect: An unmatched experience in handling huge volumes of data, analytics, visualization, and services over cloud technologies along with appreciation in product development life cycle to convert an idea into revenue generating stream.

What you'll do

  • Design & Build ingestion pipelines using ELT and schema on read in Databricks Delta lake;
  • Design & Develop “Transformations” aspects using ELT framework to modernize the Ingestion pipelines and build data transformations at scale;
  • Provide technical expertise in the areas of design and implementation of Ratings Data Ingestion pipelines with modern AWS cloud and other technologies such as S3, Hive, Databricks, Scala, Python, and Spark;
  • Build and maintain a data environment for speed, accuracy, consistency, and \"up\" time;
  • Work closely with other data teams & Data Science team and participate in the development of Ingestion pipelines;
  • Ensure data governance principles are adopted, data quality checks and data lineage are implemented in each hop of the data;
  • Be in tune with emerging trends Big data and cloud technologies and participate in the evaluation of new technologies;
  • Ensure compliance through adopting enterprise standards and promoting best practices/guiding principles aligned with organization standards.

What you'll bring

  • A minimum of 8+ years of significant experience in application development;
  • Previous experience in the areas of design and implementation of Ratings Data Ingestion pipelines with modern AWS cloud and other technologies such as S3, Hive, Databricks, Scala, Python, and Spark;
  • Development, design, and architecture exposure & the ability to ensure quality across various technology components that are developed by geographically diversified software engineers with superior knowledge of system architecture, object-oriented design, and design patterns;
  • Proficient with software development lifecycle (SDLC) methodologies like Agile and test-driven development;
  • Experience working with high-volume data and computationally intensive systems;
  • Garbage collection friendly programming experience - tuning Java garbage collection & performance is a must;
  • Proficiency in the development environment, including IDE, web & application server, GIT, Continuous Integration, unit-testing tools, and defect management tools Domain knowledge in the Financial Industry and Capital Markets is a plus;
  • Excellent communication skills are essential, with strong verbal and writing proficiencies;
  • Mentor teams, innovate and experiment, give a face to business ideas, and present to key stakeholders.

Not checking every single requirement?

If this role sounds good to you, even if you don’t meet every single bullet point in the job description, we encourage you to apply anyway. For most of the candidates that applied, we found a role that was a very good fit with their skills.

Let’s meet and you may just be the right candidate for one of our roles.

At Ness Digital Engineering we are willing to build a work culture that is based on diversification, inclusion, and authenticity.

Similar Jobs

Nagarro - Associate Staff Engineer, Java

Nagarro

India (Remote)
7 Months ago
warner bros games - Senior Software Engineer - Java

warner bros games

Bengaluru, Karnataka, India (Hybrid)
3 Months ago
Pingle studios - Android Developer

Pingle studios

(Remote)
2 Weeks ago
Capgemini - Android Infotainment Testing

Capgemini

Bengaluru, Karnataka, India (On-Site)
1 Week ago
The Walt Disney Company - Lead Software Engineer in Test, iOS/Android

The Walt Disney Company

Glendale, California, United States (On-Site)
7 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Smilegate - SRE Platform Development Lead

Smilegate

Seongnam-si, Gyeonggi-do, South Korea (On-Site)
4 Months ago
LeoVegas - Senior Backend Engineer - Retention/Sports/iGaming

LeoVegas

Málaga, Andalusia, Spain (Hybrid)
7 Months ago
Meta - Software Engineer, Android

Meta

Bellevue, Washington, United States (On-Site)
7 Months ago
Riot Games - Manager, Software Engineering

Riot Games

Shanghai, Shanghai, China (On-Site)
2 Months ago
Demandbase - Software Engineer 2 (Frontend)

Demandbase

Hyderabad, Telangana, India (On-Site)
3 Weeks ago
Diligent - Senior Software Engineer (Windows Mobile)

Diligent

Bengaluru, Karnataka, India (On-Site)
1 Month ago
playrix  - Director of Engineering

playrix

Montenegro (Remote)
7 Months ago
TransUnion - Advisor, Data Science and Analytics

TransUnion

(Remote)
1 Month ago
Google - Application Engineer

Google

Hyderabad, Telangana, India (On-Site)
1 Month ago
IBKR External - Software Engineer

IBKR External

Mumbai, Maharashtra, India (Hybrid)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Timișoara, Timiș, Romania

endava - Senior Data Engineer with Python and Databricks

endava

Timișoara, Timiș, Romania (On-Site)
1 Week ago
Veeam Software - Customer Success Systems Engineer with Spanish

Veeam Software

Bucharest, Bucharest, Romania (Remote)
2 Weeks ago
Funcom - Senior Game Tester

Funcom

Bucharest, Bucharest, Romania (Hybrid)
3 Days ago
PwC - Senior Accountant

PwC

Bucharest, Bucharest, Romania (On-Site)
8 Months ago
Amazon games  - Lead Artist, Amazon Games

Amazon games

Bucharest, Bucharest, Romania (Hybrid)
10 Months ago
Assist software  - AI Engineer

Assist software

Suceava, Suceava County, Romania (Remote)
7 Months ago
endava - Java Automation Tester

endava

Brașov, Brașov, Romania (On-Site)
2 Weeks ago
endava - Procurement Administrator

endava

Timișoara, Timiș, Romania (On-Site)
1 Week ago
Amazon games  - Quality Assurance Engineer

Amazon games

Bucharest, Bucharest, Romania (On-Site)
1 Month ago
Ubisoft - Senior Programmer

Ubisoft

Bucharest, Bucharest, Romania (Hybrid)
7 Months ago

Get notifed when new similar jobs are uploaded

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

About The Company

Berlin, Berlin, Germany (Hybrid)

Iași, Iași County, Romania (On-Site)

Košice, Košice Region, Slovakia (Hybrid)

Prague, Czechia (Remote)

London, England, United Kingdom (On-Site)

New Jersey, United States (Hybrid)

Chicago, Illinois, United States (Hybrid)

Timișoara, Timiș, Romania (Remote)

Prague, Czechia (On-Site)

View All Jobs

Get notified when new jobs are added by Ness

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug