Regular Data Engineer

4 Months ago • All levels • Data Analyst • Artificial Intelligence

Job Summary

Job Description

As a Data Engineer, you'll develop data & analytics solutions using vast datasets from various consumer-focused companies. You'll design high-performance algorithms, cutting-edge analytical techniques (machine learning, AI), and intuitive workflows for data insights. Responsibilities include collaborating with cross-functional teams, designing and developing robust machine learning models, cleaning/pre-processing data, implementing end-to-end ML pipelines, selecting appropriate ML techniques, performing exploratory data analysis, evaluating/fine-tuning models, deploying models into production, integrating solutions into applications, staying updated on ML advancements, and documenting solutions. You will work with big data technologies like Hive/Impala, Python, Java, Kafka, Spark, and R.
Must have:
  • Bachelor's degree in relevant field
  • Proven ML engineering experience
  • Strong programming (Python, R)
  • Understanding of ML algorithms
  • Experience with ML libraries
  • Data preprocessing & visualization
  • Cloud platform experience (AWS, Azure, GCP)
  • Version control (Git)
  • Problem-solving & communication skills
  • Teamwork & project completion
Good to have:
  • Python/Scala, Spark, SQL, Hadoop
  • Java, Spring Boot, JUnit
  • Software testing approaches
  • RESTful APIs & microservices
  • CI/CD experience
  • SQL databases (Postgres, Oracle)
  • Hadoop tools (Hive, Impala, Spark)
  • Data pipeline tools (NIFI, Airflow)
  • Shell scripting
  • Database/ETL performance tuning
  • Cloud APIs (Azure, AWS)
  • Agile experience

Job Details

Project description

As a DataEngineer in the Data Engineering & Analytics team, you will develop data & analytics solutions that sit atop vast datasets gathered by retail stores, restaurants, banks, and other consumer-focused companies. The challenge will be to create high-performance algorithms, cutting-edge analytical techniques including machine learning and artificial intelligence, and intuitive workflows that allow our users to derive insights from big data that in turn drive their businesses. You will have the opportunity to create high-performance analytic solutions based on data sets measured in the billions of transactions and front-end visualizations to unleash the value of big data. You will have the opportunity to develop data-driven innovative analytical solutions and identify opportunities to support business and client needs in a quantitative manner and facilitate informed recommendations/decisions through activities like building ML models, automated data pipelines, designing data architecture/schema, performing jobs in big data cluster by using different execution engines and program languages such as Hive/Impala, Python, Java, Kafka, Spark, R, etc.

Responsibilities

Collaborate with cross-functional teams to understand business requirements and translate them into machine learning solutions

Design and develop robust machine learning models and algorithms that solve complex business problems

Design and develop data and analytics solutions that sit atop vast datasets

Clean, preprocess, and analyze data to ensure its suitability for machine learning applications

Implement end-to-end machine learning pipelines, from data collection and feature engineering to model training and deployment

Select appropriate machine learning techniques and algorithms based on the problem's requirements and constraints

Perform exploratory data analysis and generate insights to guide model development

Evaluate and fine-tune machine learning models for performance, accuracy, and reliability

Deploy machine learning models into production environments, ensuring scalability and maintainability

Collaborate with technical team to integrate machine learning solutions into applications

Stay up-to-date with the latest advancements in machine learning and recommend innovative approaches to enhance our capabilities

Document and communicate machine learning solutions, findings, and insights to technical and non-technical stakeholders

Additional tasks as required

Skills

Must have

Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field

Proven experience as a machine learning engineer, working on complex machine learning projects

Strong programming skills in languages like Python, R, or similar

Solid understanding of machine learning algorithms, deep learning frameworks, and statistical modeling techniques

Hands-on experience with machine learning libraries such as TensorFlow, PyTorch, or scikit-learn

Proficiency in data preprocessing, feature engineering, and data visualization

Experience with cloud platforms such as AWS, Azure, or GCP for deploying machine learning models

Familiarity with version control systems (e.g., Git) and collaborative development practices

Strong problem-solving skills and ability to troubleshoot and optimize machine learning models

Excellent communication skills to convey technical concepts to both technical and non-technical stakeholders

Proven ability to work in a collaborative team environment and drive projects to completion Ideal Candidate Qualifications:

Working proficiency in using Python/Scala, Spark (tuning jobs), SQL, Hadoop platforms to build Big Data products & platforms.

Good programming skills in Java and spring boot and Junit.

Knowledge in software development test approaches & frameworks

Familiarity with RESTful APIs and micro-services architectures

Experience in working with CI/CD

Experience in working with SQL database like Postgres, Oracle

Preferably with hands-on experience with Hadoop big data tools (Hive, Impala, Spark)

Experience with data pipeline and workflow management tools: NIFI, Airflow.

Comfortable in developing shell scripts for automation.

Good troubleshooting and debugging skills.

Proficient in standard software development, such as version control, testing, and deployment

Demonstrated basic knowledge of statistical analytical techniques, coding, and data engineering

Ability to quickly learn and implement new technologies

Ability to Solve complex problems with multi-layered data sets

Ability to innovate and determine new approaches & technologies to solve business problems and generate business insights & recommendations.

Ability to multi-task and strong attention to detail

Flexibility to work as a member of a matrix based diverse and geographically distributed project teams

Good communication skills

both verbal and written

and strong relationship, collaboration skills, and organizational skills

Nice to have

Experience with performance Tuning of Database Schemas, Databases, SQL, ETL Jobs, and related scripts

Experience in working with Cloud APIs (e.g., Azure, AWS)

Experience participating in complex engineering projects in an Agile setting e.g. Scrum

Other

Languages

English: C1 Advanced

Seniority

Regular

Similar Jobs

Inkittt - VP of Operations

Inkittt

San Francisco, California, United States (Hybrid)
3 Months ago
IGG - Unity Programmer Intern

IGG

Singapore (On-Site)
5 Months ago
Google - Software Engineer III, Infrastructure, Google Cloud Platforms

Google

Sunnyvale, California, United States (On-Site)
4 Months ago
Sleeper - Performance Creative Associate (TikTok Ads)

Sleeper

Las Vegas, Nevada, United States (On-Site)
5 Days ago
Onward Search - Social Media Specialist

Onward Search

Addison, Texas, United States (Hybrid)
1 Month ago
PwC - IN_Manager_ Data Governance, D&A_Advisory_Manager

PwC

Mumbai, Maharashtra, India (On-Site)
6 Months ago
ION - UK Holdings - India - 859

ION

Mumbai, Maharashtra, India (On-Site)
5 Months ago
Next Level Business Services - Big data Architect with Azure Experience

Next Level Business Services

Columbus, Indiana, United States (On-Site)
5 Months ago
EXUSIA - Azure Databricks / Python / PySpark Senior Data Engineer (Developer)

EXUSIA

Hyderabad, Telangana, India (Remote)
2 Months ago
Aristocrat Gaming - Director of Advanced Insights

Aristocrat Gaming

Las Vegas, Nevada, United States (Hybrid)
2 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Trend Micro - Sr. Data Scientist (AI Lab)

Trend Micro

Taipei City, Taiwan (On-Site)
6 Months ago
Owlcat Games - Network Programmer

Owlcat Games

Nicosia, Nicosia, Cyprus (Hybrid)
1 Month ago
Blind Squirrel Games - Sr. Generalist Engineer, Austin

Blind Squirrel Games

Austin, Texas, United States (Hybrid)
1 Week ago
HP - Machine Learning Intern

HP

Austin, Texas, United States (On-Site)
6 Months ago
Inkittt - Lead Front-End Engineer: Mobile Team (m/f/d)

Inkittt

Krakow Am See, Mecklenburg-Vorpommern, Germany (On-Site)
8 Months ago
Google - Software Engineer II, Infrastructure, Google Cloud

Google

Bengaluru, Karnataka, India (On-Site)
4 Months ago
The Walt Disney Company - Sr Machine Learning Engineer

The Walt Disney Company

San Francisco, California, United States (On-Site)
4 Months ago
ByteDance - Research Scientist Graduate (Foundation Model - Vision and Language)

ByteDance

Seattle, Washington, United States (On-Site)
2 Months ago
OYELA - Fashion Model

OYELA

Gurugram, Haryana, India (On-Site)
5 Months ago
Fandom - Senior Product Manager

Fandom

Rugeley, England, United Kingdom (Remote)
1 Week ago

Get notifed when new similar jobs are uploaded

Jobs in undefined

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Data Analyst Jobs

Playrix - Senior Data Analyst (Game)

Playrix

Almaty, Almaty Region, Kazakhstan (Remote)
5 Months ago
PwC - Senior Associate_GCP Data Engineer_Data and  Analytics_Advisory_Bengaluru

PwC

Bengaluru, Karnataka, India (On-Site)
6 Months ago
Arrise Solutions (India)   - Data Scientist - Recommender S/m's

Arrise Solutions (India)

Hyderabad, Telangana, India (On-Site)
6 Months ago
EXUSIA - Lead Data Engineers – Azure/Databricks/Snowflake

EXUSIA

United States (Remote)
2 Months ago
CloudHire - Data Analyst

CloudHire

Bengaluru, Karnataka, India (Remote)
6 Days ago
Metacore - Analytics Engineer

Metacore

Helsinki, Uusimaa, Finland (Hybrid)
1 Month ago
Onward Search - Data Analyst II

Onward Search

San Jose, California, United States (On-Site)
2 Months ago
Ubisoft - Data Scientist Assistant Internship

Ubisoft

Paris, Île-de-France, France (On-Site)
6 Days ago
PwC - PowerBI Lead

PwC

Toronto, Ontario, Canada (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Luxoft, a DXC Technology Company (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. Acquired by U.S. company DXC Technology in 2019, Luxoft is a global operation in 44 cities and 21 countries with an international, agile workforce of nearly 18,000 people. It combines a unique blend of engineering excellence and deep industry expertise, helping over 425 global clients innovate in the areas of automotive, financial services, travel and hospitality, healthcare, life sciences, media and telecommunications.

DXC Technology is a leading Fortune 500 IT services company which helps global companies run their mission critical systems. Together, DXC and Luxoft offer a differentiated customer-value proposition for digital transformation by combining Luxoft’s front-end digital capabilities with DXC’s expertise in IT modernization and integration. Follow our profile for regular updates and insights into technology and business needs.

Poland, Ohio, United States (Remote)

Kraków, Lesser Poland Voivodeship, Poland (On-Site)

Wrocław, Lower Silesian Voivodeship, Poland (On-Site)

Ukrainka, Kyiv Oblast, Ukraine (Remote)

Bengaluru, Karnataka, India (On-Site)

Mississauga, Ontario, Canada (On-Site)

Toronto, Ontario, Canada (On-Site)

Poland, Ohio, United States (Remote)

Pune, Maharashtra, India (On-Site)

View All Jobs

Get notified when new jobs are added by Luxoft

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug