How to install tmux in Synology NAS Server New Way
A cleaner way to get tmux onto a Synology NAS than my first attempt. Fewer steps, survives updates.
Read full article →Things I figured out while building scrapers, pipelines, and SaaS apps, written down so I don't figure them out twice.
Docker, Kubernetes, CI/CD, and monitoring. What it takes to keep a scraper fleet running while you sleep.
Scraped data is messy; delivered data can't be. Validation pipelines and storage across MongoDB, PostgreSQL, and Elasticsearch.
Proxies, fingerprints, and distributed crawls: how I run scrapers across millions of pages without getting blocked, and where the legal lines sit.
Modern sites don't hand you clean HTML. Rendering JavaScript, submitting forms, and chasing the AJAX endpoints that actually hold the data.
Scrapy's architecture, selectors, and items, ending with a first spider that extracts real data without falling over.
Subscriptions, payment processing, and deployment: the last 20% that takes 80% of the time.
Monitoring first, then profiling, then fixes. And knowing what to leave alone.
Authentication, authorization, and the security work you can't bolt on later.
Query patterns, indexes, and cache layers that keep pages fast once real data shows up.
One email when something new goes up. That's it.