Senior Data Engineer

1 Year ago • 3 Years + • Data Analysis

Job Summary

Job Description

This role involves collaborating with customers to gather requirements and understand their business processes. The Senior Data Engineer will create and maintain optimal data pipeline architecture, assemble complex data sets, and identify process improvements like automation and infrastructure redesign. Responsibilities include building data pipelines using Spark, SQL, and Azure or AWS data technologies, and developing analytics tools for insights into customer acquisition and operational efficiency. The role also requires deploying and monitoring system performance in production. A strong background in Big Data technologies, advanced SQL, and experience with data pipeline tools and scripting languages like Python or Java is essential.
Must have:
  • 3+ years of experience in Data Engineering
  • Big data tools: Hadoop, Spark, Kafka
  • SQL databases: Oracle, SQL Server, MySQL, Postgres
  • Data pipeline tools: Azkaban, Luigi, Airflow
  • Stream-processing systems: Storm, Spark-Streaming
  • Scripting languages: Python, Java, C++, Scala
  • Project management and organizational skills

Job Details

JOB DESCRIPTION
• Collaborate with customer to gather requirements and to understand their business processes.
• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes,
optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a
wide variety of data sources using Spark, SQL and Azure or AWS data technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer
acquisition, operational efficiency and other key business performance metrics
.• Use tools to deploy and monitor the performance of the systems in production

REQUIREMENTS
• Demonstrate knowledge and real-world experience on Big data technologies. A successful history
of manipulating, processing and extracting value from large datasets.
• Advanced working SQL knowledge and experience working with relational databases, query
authoring (SQL) as well as working familiarity with a variety of databases.
• Strong analytic skills related to working with unstructured datasets.
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

• We are looking for a candidate with > 3+ years of experience in a Data Engineer role. They should also have experience using the following software/tools:o Experience with big data tools: Hadoop, Spark, Kafka, etc.o Experience with relational SQL databases such as Oracle, SQL Server, MySQL or Postgres
o Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
o Experience with stream-processing systems: Storm, Spark-Streaming, etc.
o Experience with object-oriented/object function scripting languages: Python, Java, C++,
Scala, etc
.• For senior data engineers also should have demonstrable experience ino Project management and organizational skills.


About Rackspace Technology

We are the multicloud solutions experts. We combine our expertise with the world’s leading technologies — across applications, data and security — to deliver end-to-end solutions. We have a proven record of advising customers based on their business challenges, designing solutions that scale, building and managing those solutions, and optimizing returns into the future. Named a best place to work, year after year according to Fortune, Forbes and Glassdoor, we attract and develop world-class talent. Join us on our mission to embrace technology, empower customers and deliver the future.

 

 

More on Rackspace Technology

Though we’re all different, Rackers thrive through our connection to a central goal: to be a valued member of a winning team on an inspiring mission. We bring our whole selves to work every day. And we embrace the notion that unique perspectives fuel innovation and enable us to best serve our customers and communities around the globe. We welcome you to apply today and want you to know that we are committed to offering equal employment opportunity without regard to age, color, disability, gender reassignment or identity or expression, genetic information, marital or civil partner status, pregnancy or maternity status, military or veteran status, nationality, ethnic or national origin, race, religion or belief, sexual orientation, or any legally protected characteristic. If you have a disability or special need that requires accommodation, please let us know.

Similar Jobs

Social Discovery Ventures - IT Procurement Specialist

Social Discovery Ventures

Belgrade, Serbia (Remote)
1 Month ago
Fandom  - Sr Fullstack Software Engineer, Ad Tech

Fandom

New York, New York, United States (Remote)
1 Month ago
Qualcomm - Lead RF Systems Engineer

Qualcomm

San Diego, California, United States (On-Site)
1 Month ago
Hitachi - Quality Analyst

Hitachi

Pune, Maharashtra, India (On-Site)
9 Months ago
Tesla - Sales Advisor

Tesla

Lincoln, England, United Kingdom (On-Site)
5 Months ago
Cognite - Data Engineer

Cognite

Bengaluru, Karnataka, India (Hybrid)
1 Year ago
dun bradstreet - Principal, Data Scientist

dun bradstreet

Oslo, Oslo, Norway (Hybrid)
1 Month ago
Peak Games - Big Data Platform Engineer - Java/Go

Peak Games

(On-Site)
4 Months ago
Ion - Internship - Data Science

Ion

London, England, United Kingdom (On-Site)
9 Months ago
Monzo - Senior Data Scientist, Growth Marketing

Monzo

London, England, United Kingdom (Remote)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

bounteous - Manager Quality Automation - BOT

bounteous

India (Remote)
9 Months ago
Applike - Senior Machine Learning Engineer

Applike

Hamburg, Hamburg, Germany (Hybrid)
1 Month ago
Help Scout - iOS Engineer

Help Scout

Boston, Massachusetts, United States (Remote)
4 Weeks ago
Fictiv - Strategic Sourcing Analyst

Fictiv

Bengaluru, Karnataka, India (On-Site)
2 Days ago
Capgemini - Finance Controller

Capgemini

Mumbai, Maharashtra, India (On-Site)
2 Months ago
Qualcomm - SoC Systems Performance Engineer

Qualcomm

San Diego, California, United States (On-Site)
1 Month ago
Qualcomm - Engineer, Senior - WDD

Qualcomm

Bengaluru, Karnataka, India (On-Site)
1 Month ago
My Fitness Pal - Senior Client Partner, Advertising - CPG/QSR

My Fitness Pal

Chicago, Illinois, United States (On-Site)
2 Weeks ago
WebFX - Remote Copywriter

WebFX

United States (Remote)
7 Months ago
Univision - Social Media Producer

Univision

Miami, Florida, United States (On-Site)
1 Year ago

Get notifed when new similar jobs are uploaded

Jobs in Vietnam

Amanotes - Creative Motion Design Freelancer - Mobile Music Game

Amanotes

Ho Chi Minh City, Ho Chi Minh City, Vietnam (Remote)
10 Months ago
Amanotes - Game Product Lead

Amanotes

Ho Chi Minh City, Vietnam (On-Site)
2 Months ago
Thunder cloud studio - UE4 GAME DEVELOPER

Thunder cloud studio

Hanoi, Hanoi, Vietnam (On-Site)
1 Year ago
Kgroup - Unity Developer

Kgroup

Thành Phố Hồ Chí Minh, Vietnam (On-Site)
1 Year ago
Bosch Group - SAP Fiori (Frontend) Developer Intern

Bosch Group

Ho Chi Minh City, Vietnam (On-Site)
3 Weeks ago
Amanotes - People Manager

Amanotes

Ho Chi Minh City, Vietnam (On-Site)
1 Month ago
antada studio - 2D Background Artist

antada studio

Vietnam (On-Site)
17 Hours ago
Zitga gaming studio - Video Editor

Zitga gaming studio

Hanoi, Vietnam (On-Site)
1 Week ago
Axon - Mid/Senior Mobile Android Engineer

Axon

Ho Chi Minh City, Vietnam (On-Site)
2 Weeks ago
bytedance - CapCut VN Country Operation Manager

bytedance

Ho Chi Minh City, Vietnam (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Rippling - Senior Staff Software Engineer - Data Products

Rippling

Seattle, Washington, United States (On-Site)
5 Months ago
Progress - Data Scientist

Progress

Sofia, Sofia City Province, Bulgaria (Hybrid)
1 Month ago
NetEase Games - Data Scientist

NetEase Games

Singapore (On-Site)
2 Days ago
Global Business Travel - Data Advanced Insight Analyst

Global Business Travel

Chicago, Illinois, United States (Hybrid)
1 Month ago
Crunchyroll - Senior Data Analyst

Crunchyroll

Hyderabad, Telangana, India (On-Site)
4 Months ago
beghou consulting - Senior Data Engineer

beghou consulting

Hyderabad, Telangana, India (Hybrid)
2 Months ago
Apple - Lab and Data Automation Engineer

Apple

Cupertino, California, United States (On-Site)
1 Month ago
bytedance - Backend Software Engineer, Data Engineering

bytedance

Singapore (On-Site)
8 Months ago
Nagarro - Associate Principal Consultant, Business Analyst

Nagarro

(On-Site)
8 Months ago
GoTo Group - Data Scientist Intern

GoTo Group

Singapore (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Gurugram, Haryana, India (Hybrid)

Columbus, Ohio, United States (Remote)

San Antonio, Texas, United States (Hybrid)

Gurugram, Haryana, India (Hybrid)

Giza, Giza Governorate, Egypt (Hybrid)

Riyadh, Riyadh Province, Saudi Arabia (On-Site)

Toronto, Ontario, Canada (Remote)

Giza, Giza Governorate, Egypt (Hybrid)

Gurugram, Haryana, India (Remote)

View All Jobs

Get notified when new jobs are added by Rackspace Technology

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug