Sr. Data Engineer
Accurate
Job Summary
The Senior Data Engineer will implement and execute the data strategy by building scalable and optimized ETL/Data Pipelines. They will deliver high-quality data products by implementing data quality and transformations across storage layers. This role involves collaborating with business teams to translate product requirements into an efficient data model, providing secure data access to BI and reporting teams. The role requires building scalable data pipelines, automating workflows, defining best practices, understanding cloud platforms (preferably AWS), creating design and mapping documents, and working on medium to large data warehouses. This position involves monitoring audits, collaborating with data architects, and communicating with stakeholders.
Must Have
- Build scalable data/ETL pipelines for BI/Reporting.
- Create automation workflows/orchestrations for data pipelines.
- Implement data engineering best practices, coding standards.
- Experience with cloud platforms, preferably AWS.
- Develop and optimize medium to large data warehouses.
Good to Have
- dBT tool.
- Python/Spark.
- Data Governance Processes.
- SSIS Integrations.
- Knowledge on Tableau/Power BI tools.
Perks & Benefits
- Fast-paced environment
- Room for growth
- Diverse and inclusive culture
Job Description
Responsibilities:
- Ability to build Scalable Data / ETL pipelines in Batch & Realtime patterns for BI / Reporting Platforms
- Should be able to create automation workflows / Orchestrations for the Data Pipelines
- Define and enforce best practices for data engineering, including coding standards, source control, and CI/CD processes to minimize technical debt and ensure high-quality code deployments.
- Should be able to understand different cloud Platforms, preferably on AWS Platform Services for Data & Analytics
- Should be able to create Design Documents/Mapping documents
- Should have hands on experience in developing & optimizing Medium to Large Data Warehouses (on any of Cloud Data Warehouses), preferably Snowflake, SQL Server, MongoDB, real time DB's
- Should be aware of Data Architecture/ Design Patterns in the areas of Data Ingestion / Curation / Data Consumption / Reporting Semantic Models
- Aware of Data Ingestion Tools, Data Quality Management, Metadata Management, Data Lineage, Data Security
- Should be able to monitor Audit the logging to troubleshoot issues and provide a fix to reduce business impacts
- Should work collaboratively with Data Architect in Designing & Developing Reusable utilities, understand & align to Architecture Design principles
- Should be able to communicate & Collaborate with all stakeholders (Director level, Business Units, other Architects, Product Managers, Scrum Masters)
Qualifications:
- Should hold a Bachelor’s degree of engineering/MCA with good Academic CGPA
- Should have at least 6-10 years of total IT experience in Software Development, with 2 years exclusively in ETL Design in Data Warehousing projects
- Snowflake (Stored procedures, Tasks, Streams, Snow pipes, Dynamic Tables)
- Open Table formats (Iceberg tables) on Snowflake/AWS
- SQL Server, MongoDB, Data Analysis, SQL Query proficiency
- Realtime Databases (like Cassandra, DynamoDB)
- Replication tools (DMS, Qlik Replica)
Nice To Have
- dBT tool
- Python/Spark
- Data Governance Processes
- SSIS Integrations
- Knowledge on Tableau/Power BI tools
- Solid understanding of Agile development process and software release processes.
- Must be a self-starter who is highly organized, hands-on, and a team player.
- Should be able to independently do Unit testing
- Should be able to do peer-peer Code reviews, Code walk-through sessions
- Detail-oriented, with strong written and verbal communication.
Working Conditions:
- The company environment is dynamic and reflective of rapid growth
- Friendly, helpful, open, and all inclusive
- Fast paced with frequent changes
- Sits and works at desk computer-keyboard for extended periods of time
- Works with others
- Our company uses E-Verify to confirm the employment eligibility of all newly hired employees. To learn more about e-Verify, including your rights and responsibilities, please visit www.dhs.gov/E-Verify.
- All qualified applicants will receive consideration of employment without regard to race, color, religion, gender, national origin, or other status protected by applicable law