Data Engineer

1 Month ago • 2-5 Years • Data Analyst

About the job

Job Description

We are looking for a Senior Data Engineer to join our Data team. In this role, you will be responsible for design, build, and maintain the infrastructure required to analyze large data sets. As an expert in data management, ETL (extract, transform, load) processes, and data warehousing, you will work with various big data technologies such as Spark, Hadoop, and NoSQL databases. You will work closely with data and analytics team to identify and prioritize data engineering projects, ensuring the data infrastructure aligns with overall business goals and objectives.
Must have:
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field
  • 2+ years of proven experience developing large-scale software using an object-oriented or a functional language.
  • 3+ years of professional experience in data engineering
  • Strong experience with Spark, Scala, and Python
  • Experience with AWS services
  • Familiarity with data warehousing concepts and technologies
  • Experience with data pipeline orchestration and scheduling using tools such as Airflow
  • Strong problem-solving skills
Good to have:
  • Expertise with Vertica or Redshift
  • Experience with machine learning and/or data science projects
  • Knowledge of data governance and security best practices
  • Knowledge of Spark internals (tuning, query optimization)
Perks:
  • Flexible, balanced environment
  • Remote work options
  • Ongoing learning and career advancement opportunities
Not hearing back from companies?
Unlock the secrets to a successful job application and accelerate your journey to your next opportunity.

Description

Join a Company That Invests in You

Seeking Alpha is the world's leading community of engaged investors. We’re the go-to destination for investors looking for actionable stock market opinions, real-time market analysis, and unique financial insights. At the same time, we’re also dedicated to creating a workplace where our team thrives. We’re passionate about fostering a flexible, balanced environment with remote work options and an array of perks that make a real difference.

Here, your growth matters. We prioritize your development through ongoing learning and career advancement opportunities, helping you reach new milestones. Join Seeking Alpha to be part of a company that values your unique journey, supports your success, and champions both your personal well-being and professional goals.

What We're Looking For

We are looking for a Senior Data Engineer to join our Data team. In this role, you will be responsible for design, build, and maintain the infrastructure required to analyze large data sets. As an expert in data management, ETL (extract, transform, load) processes, and data warehousing, you will work with various big data technologies such as Spark, Hadoop, and NoSQL databases. You will work closely with data and analytics team to identify and prioritize data engineering projects, ensuring the data infrastructure aligns with overall business goals and objectives.

What You'll Do

  • Work closely with data scientists/analytics and other stakeholders to identify and prioritize data engineering projects and to ensure that the data infrastructure is aligned with business goals and objectives
  • Design, build and maintain optimal data pipeline architecture for extraction, transformation, and loading of data from a wide variety of data sources, including external APIs, data streams, and data stores. 
  • Continuously monitor and optimize the performance and reliability of the data infrastructure, and identify and implement solutions to improve scalability, efficiency, and security
  • Stay up-to-date with the latest trends and developments in the field of data engineering, and leverage this knowledge to identify opportunities for improvement and innovation within the organization
  • Solve challenging problems in a fast-paced and evolving environment while maintaining uncompromising quality.
  • Implement data privacy and security requirements to ensure solutions comply with security standards and frameworks.
  • Enhance the team's dev-ops capabilities.

Requirements

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field
  • 2+ years of proven experience developing large-scale software using an object-oriented or a functional language.
  • 3+ years of professional experience in data engineering, focusing on building and maintaining data pipelines and data warehouses
  • Strong experience with Spark, Scala, and Python, including the ability to write high-performance, maintainable code
  • Experience with AWS services, including EC2, S3, Athena, Lambda and EMR
  • Familiarity with data warehousing concepts and technologies, such as columnar storage, data lakes, and SQL
  • Experience with data pipeline orchestration and scheduling using tools such as Airflow
  • Strong problem-solving skills and the ability to work independently as well as part of a team
  • High-level English - a must
  • A team player with excellent collaboration skills.

         Nice to Have:

  • Expertise with Vertica or Redshift, including experience with query optimization and performance tuning
  • Experience with machine learning and/or data science projects
  • Knowledge of data governance and security best practices, including data privacy regulations such as GDPR and CCPA.
  • Knowledge of Spark internals (tuning, query optimization)
View Full Job Description

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug