Crawler Analyst

1 Month ago • All levels

Job Summary

Job Description

As a Crawler Analyst, you will be responsible for automating information extraction from websites using an internal tool. This involves collaborating with the Bots (Python) team to address website-specific challenges, updating templates (YAML files) to ensure continued extraction, validating existing templates, and providing feedback for tool improvement. You will also need to analyze system logs to identify and troubleshoot errors. The role requires strong analytical and problem-solving skills, as well as the ability to work both independently and as part of a team. A good understanding of HTML, XPATH, regex and Linux environments is required, along with basic knowledge of Docker and Agile methodologies. Upper-Intermediate level of English is required.
Must have:
  • Experience in scraping, scripting, programming, or coding, HTML, XPATH, regex required.
  • Experience working with Linux environments (grep, awk etc.)
  • Basic knowledge of using Docker or similar tools.
  • Familiarity with Agile methodologies (Scrum, kanban, Jira Suite...)
  • Knowledge of Git as a version control system.
  • Upper-Intermediate level of English proficiency.
Good to have:
  • More experience in Python will help to solve and detect problems easily.
  • Some experience working with Proxies.
  • Ability to find solutions by observing logs.
Perks:
  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits

Job Details

As our Crawler Analyst, you will work using our internal tool to automate information extraction from websites. Every site has its own challenges so you’ll have to collaborate with the Bots (Python) team (and occasionally other teams and departments) to solve them as you learn, as well as with the developers of this tool to propose improvements. You also have to check system logs to detect possible errors in production and deduce if something is not well defined in terms of development.

Responsibilities

  • Automate information extraction for hundreds of websites using custom templates.
  • Keep extraction procedures working as websites change by updating templates (YAML files)
  • Validate current templates and escalate the more complex ones to the Python development team.
  • Thinking about elegant, reliable, and long term solutions.
  • Provide feedback about the tool so that we can improve it and make the job easier over time.


Your qualifications:

  • Have strong analytical and technical problem-solving skills.
  • Have strong communication and organizational skills.
  • Be able to work well in a team and autonomously.
  • Be flexible and able to adapt easily to changes and learn quickly.
  • Be able to learn new skills and tools using documentation and examples.
  • Be proactive and propose solutions to detected problems, and think about how to improve the process.


Requirements:

  • Experience in scraping, scripting, programming or coding is required, especially good knowledge of HTML, XPATH and regex.
  • Experience working with Linux environments (grep, awk…)
  • Basic knowledge of using Docker or similar tools.
  • Familiarity with Agile methodologies (Scrum, kanban, Jira Suite...)
  • Some familiarity with Python, although you will not use it frequently.
  • Knowledge of Git as a version control system.
  • Upper-Intermediate level of English (we are an international team and the official language in the office is English) 

 

Nice to have:

  • More experience in Python will help you to solve and detect problems easily with our tool.
  • Some experience working with Proxies.
  • Ability to find solutions by observing logs.

We offer*:

  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits

*not applicable for freelancers

Similar Jobs

Addepar - Web Enablement Engineering Manager

Addepar

Pune, Maharashtra, India (On-Site)
1 Day ago
Nagarro - Associate Staff Engineer, Java Fullstack

Nagarro

Sri Lanka (Remote)
7 Months ago
Tonic Inq. - Sr. Frontend Engineer - Web3 games & NFT Startup

Tonic Inq.

(Remote)
3 Years ago
EveryMatrix - Trainee QA & Configuration Manager

EveryMatrix

L'viv, Dnipropetrovsk Oblast, Ukraine (Hybrid)
4 Weeks ago
Qualcomm - Senior technical writer

Qualcomm

Bengaluru, Karnataka, India (On-Site)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

luxsoft - Regular Full Stack Developer (Angular+Node.js)

luxsoft

Ukraine (Remote)
1 Week ago
PayPal - Senior SDET

PayPal

Chennai, Tamil Nadu, India (Hybrid)
1 Week ago
Apple - Machine Learning Engineer

Apple

Cupertino, California, United States (On-Site)
1 Week ago
Canva - Senior Frontend Engineer - Apps API Platform

Canva

Brisbane, Queensland, Australia (Remote)
2 Weeks ago
AnyDesk - Software Developer Generalist

AnyDesk

Stuttgart, Baden-Württemberg, Germany (Hybrid)
3 Weeks ago
WebFX - Digital Marketing - Account Management Internship

WebFX

Harrisburg, Pennsylvania, United States (On-Site)
7 Months ago
Infosys - .NET Full Stack Developer

Infosys

Guadalajara, Jalisco, Mexico (On-Site)
2 Weeks ago
Riot Games - Staff Software Engineer (Services) - League of Legends, Motivations

Riot Games

Los Angeles, California, United States (On-Site)
8 Months ago
Fictiv - Wordpress Developer

Fictiv

Pune, Maharashtra, India (On-Site)
3 Weeks ago
Aristocrat Gaming - Front End Developer

Aristocrat Gaming

Warsaw, Masovian Voivodeship, Poland (Hybrid)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Ukraine

playrix  - Lead SDET

playrix

Ukraine (Remote)
7 Months ago
Every matrix - Manual QA Tester

Every matrix

Lviv, Lviv Oblast, Ukraine (Hybrid)
2 Months ago
Motorola solutions - Channel Account Manager Enterprise/Corporate - Video Security & Access Control

Motorola solutions

Kyiv, Kyiv City, Ukraine (On-Site)
3 Weeks ago
Ubisoft - Junior UX Requirements Specialist

Ubisoft

Kyiv, Kyiv City, Ukraine (On-Site)
2 Weeks ago
Magic Media - 3D Generalist Artist

Magic Media

Kyiv, Kyiv City, Ukraine (Remote)
1 Month ago
Better ME - Global Supply Chain Manager

Better ME

Ukraine (Remote)
3 Weeks ago
EveryMatrix - Middle BI/DB Developer

EveryMatrix

Lviv, Lviv Oblast, Ukraine (Hybrid)
3 Weeks ago
Virtuos - Lead 3D Artist

Virtuos

Ukraine (Remote)
1 Month ago
plarium - Unity Developer (Battle team)

plarium

Lviv, Lviv Oblast, Ukraine (On-Site)
2 Weeks ago
Better ME - Android Engineer

Better ME

Ukraine (Remote)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!