Data Engineer, Tool Scraper

Questah Staffing

Agency
Questah Staffing
Work location
Remote
Employment type
Contract
Published

Description

Maintain the page scraper and the screenshot analysis that collect the evidence behind the TallyCounter.org comparison of online tally counter tools. This role is with TallyCounter.org. Nice to have: Experience calling large language model APIs from application code. Experience with image analysis or optical character recognition.

Responsibilities

Maintain the Node scraper that opens a tool's page in a headless browser, saves the page text and takes a screenshot. Maintain the Laravel command that stores each scrape and the features it detects. Improve the screenshot analysis so that it finds the features in the catalog and records evidence for each one. Track scrape failures and fix the pages that no longer load or no longer match. Keep the list of feature gaps current so that analysts see which features a tool offers that the catalog lacks.

Requirements

Experience writing scrapers or page automation with Node and a headless browser. Experience with PHP and Laravel console commands. Comfortable with SQL and schema migrations.

Apply to this job

Applications to this job go to Questah Staffing. Create a contractor account with Questah Staffing, or sign in, and you come back to this page to apply.