Senior Python Data Scraping Engineer – Freelance

About the role

Mindrift is seeking a Senior Python Data Scraping Engineer to join the Tendem project on a part‑time freelance basis. You will act as an AI Pilot, collaborating with Tendem Agents to design, execute, and validate complex web‑scraping workflows that feed our hybrid AI‑human system.

Key responsibilities

• Own end‑to‑end data extraction pipelines for complex, dynamic websites.
• Utilize internal tools such as Apify and OpenRouter, as well as custom Python scripts, to collect and validate data.
• Handle JavaScript‑rendered content, AJAX calls, infinite scroll, and other interactive elements.
• Enforce data quality through validation checks, cross‑source consistency, and formatting standards.
• Scale scraping operations using batching or parallel processing while monitoring failures and site changes.

Required profile

• Minimum 5 years of experience in data engineering, web scraping, or software development.
• Strong problem‑solving skills and ability to work independently.
• Experience with AI‑assisted workflows and large language models is a plus.
• Bachelor’s or Master’s degree in a technical field is advantageous but not mandatory.

Required skills

• Python programming
• Web‑scraping libraries (BeautifulSoup, Selenium or equivalents)
• Handling dynamic content (JS, AJAX, infinite scroll)
• API integration and proxy management
• Data cleaning, normalization, and validation
• Output formats: CSV, JSON, Google Sheets
• Tools: Apify, OpenRouter

What we offer

• Flexible remote work schedule
• Opportunity to contribute to cutting‑edge AI‑human collaboration projects
• Freelance contract with competitive compensation