reference youtube caption ratelimit vpn ineffective 20260715T140128
name: YouTube caption-fetch rate limiting: residential IP has real quota, VPN exits don't help description: yt-dlp caption scraping: home IP earns hundreds of fetches before throttling and recovers on a quiet window; shared VPN/proxy exits (e.g. gluetun/Proton) are already saturated and get 429 immediately. type: reference
When bulk-scraping YouTube auto-captions with yt-dlp (e.g. for a large video corpus), the per-IP rate limit behaves differently than expected:
- Residential/home IP is the privileged lane. It earns a real burst quota (100+ caption fetches) before throttling kicks in, and recovers after a sustained quiet window (observed ~90-120 min of zero requests was needed; shorter windows under-recover). Repeated offenses within a short span appear to escalate the penalty (yields dropped run-over-run even after a 60-min quiet window on a heavily-hit day).
- VPN/proxy exits are NOT a workaround, and are worse. Tested using the existing gluetun Proton VPN container (media-server CT102) as an HTTP proxy for yt-dlp: page metadata fetched fine (no bot-wall), but the caption download itself got HTTP 429 on the very first request. Shared VPN exit IPs are hit by hundreds of other users hitting the same YouTube endpoints, so they're permanently "hot" from YouTube's limiter's perspective, worse than a clean residential IP.
Why: discovered while re-scraping ~550 CreatorTrack transcripts after a dev-slot wipe; considered routing through the Proton VPN to parallelize/bypass the home IP's cooldown, tested empirically instead of assuming, confirmed it doesn't help.
How to apply: for any future large yt-dlp caption/metadata scrape, don't reach for a VPN/proxy to dodge YouTube rate limits, it won't help and may perform worse. Instead: pace fetches, back off with long silent windows (90+ min) when throttled, and expect escalating penalties if you keep probing during the cooldown (probing itself can reset the clock). Related: [[reference_watchyoutube]], [[reference_gluetun_qbit_portforward_bug]] (same gluetun container, different bug).
[auto-memory session 60d132de-4659-485c-8044-427af0ccce7f, confidence 0.75, mode staged]