reference youtube captions vpn rate limit 20260715T135112
name: yt-dlp/YouTube caption scraping: residential IP outperforms VPN exits description: Home IP earns hundreds of yt-dlp caption fetches before YouTube throttles; shared VPN exits (tested via gluetun/Proton) get HTTP 429 on the first request. type: reference
Tested empirically during the CreatorTrack transcript corpus scrape (2026-07-15): routing yt-dlp caption downloads through the Proton VPN exit already wired into media-server's gluetun container was tried as a way to dodge YouTube's per-IP caption rate limit. Result: YouTube served page metadata fine through the VPN (no bot-wall), but the caption download itself returned HTTP 429 on the very first request. Root cause: popular VPN exit IPs are shared across hundreds of users hitting the same YouTube endpoints, so they read as permanently 'hot' to YouTube's limiter. By contrast, Console's residential home IP earns a real burst quota (hundreds of fetches) before throttling, and recovers with a long quiet window (roughly 60-120 min, penalty appears to escalate with repeated hammering).
Why: Saves re-testing this workaround on any future bulk YouTube scraping job (captions, metadata, or otherwise) — the VPN route is a dead end, not just untried.
How to apply: For any future large-scale yt-dlp/YouTube pull that hits rate limits, do not reach for gluetun/Proton as a bypass. Instead pace requests with long silent windows off the residential IP, and treat a shrinking per-attempt yield as a signal to lengthen the next wait rather than retry sooner.
[auto-memory session 60d132de-4659-485c-8044-427af0ccce7f, confidence 0.75, mode staged]