#data-engineering
- From a scraper to a monitored data pipeline
· data-engineering, web-scraping, monitoring
- What web scraping really costs per 1,000 pages
· web-scraping, proxies, data-engineering
- Scheduling Scrapy with Airflow: retries, backfills, alerts
· data-engineering, scrapy, airflow, python
- Scrapy, part 4: cleaning and storing the data
· web-scraping, scrapy, data-engineering