Python Data Scraper
Private
About this project
Built For: Work and Travel Guide GmbH (WTG)
A set of Python automation tools that scrape data from various websites and turn it into clean, structured records.
The scrapers run headless browser automation to collect listings, then enrich each entry with things like validated emails, phone numbers, and business details before pushing everything into a shared database.
Key features:
- Headless browser automation to scrape and verify listings at scale
- Email discovery and validation, with duplicate/invalid contact cleanup
- Concurrent/async workers for faster large-batch processing
- Automatic retries, rate limiting, and progress logging to handle long-running jobs
- Data consistency checks (name/address matching) to filter out bad matches
- Syncs results directly into a shared database for downstream use
Tech Stack
Python
Playwright
Pandas
aiohttp
Airtable API