E-commerce giants like Amazon invest millions in bot detection, and your small IP pool might be the giveaway.
If you're hitting blocks, your IPs are likely already in a public database. Small pools get "burned" fast. I started looking for providers with massive networks, and the one i'm using has over 20 million real devices worldwide. Because these are real home connections, they have natural history and pass anti-bot checks much more easily. You can distribute the load so each IP only makes a human-level number of requests.
What’s the toughest website you’ve ever tried to scrape?
u/CanZealousideal3170 — 10 days ago