Fraud Scores and Scraping Blocks: Which Ones Are Really About the IP

Scrapers get blocked, and the first thing most people check is the proxy. They put the IP into a fraud score page, see a high number, and decide the address is burnt. Most of the time that is the wrong diagnosis. A block can be caused by the address, by the shape of your requests, or by the target's own rules, and each of those has a different fix. This article separates the three. It explains why a US carrier address scores high on public checkers before you have sent a single request, which blocks are really about your headers and rate, and which signals, mainly Cloudflare and Google, tell you the address itself is the problem.

Three reasons a request gets refused

When a target returns a 403, a captcha page or an empty response, one of three things has happened. The address was refused on reputation. The request was refused on how it looked or how often it arrived. Or the target refuses everything that is not a logged-in human, in which case no proxy on earth helps and the job needs a different approach. Sorting your failures into those three piles is the most useful thing you can do before changing anything.

Fraud score checkers only speak to the first pile, and even there they speak loosely. They estimate how risky an address looks from outside, based on the history of everything that has passed through it. They know nothing about your crawler, your headers, or the site you are pulling from.

Why a mobile address looks risky before you use it

US carriers run their networks behind carrier-grade NAT, or CGNAT, which means a single public IPv4 address is shared by hundreds or thousands of subscribers at once. Your proxy is a dedicated modem with a real AT&T, T-Mobile or Verizon SIM, and nobody else uses your port, but the public address it presents is the carrier's shared one.

From the checker's side, that address carries a crowd's worth of traffic: dozens of user agents, sessions that start and stop constantly, activity around the clock, addresses that reshuffle as devices roam. Those are exactly the signals a scoring model treats as suspicious, so the number rises. The address is not on a blocklist. It is simply doing what a mobile address does. Scores in the eighties or nineties are the usual result for a carrier range and tell you almost nothing about how a target will treat you.

Blocks that are about the request, not the address

Most scraping blocks are caused by the request. Modern bot protection looks at far more than the source address, and the address is often the least interesting part. Before you blame the proxy, look for these patterns in your own traffic.

The telltale sign of a request-shaped block is that it follows your crawler and not your address. Rotate to a fresh IP and the block returns within minutes; slow the same crawler down or send it through a real browser and the block disappears on the same IP. When you see that, the fix is in your code, and switching proxies only delays the next failure.

• A user agent that does not match the rest of the headers, such as a Chrome UA with no Accept-Language, or a header order that no real Chrome sends
• A TLS handshake that identifies your HTTP library rather than a browser; many protection layers fingerprint the handshake before reading a single header
• Hundreds of requests per minute from one address to one site, which no phone user produces, so it is flagged wherever the address sits
• No cookies at all, or cookies carried over from a session that was already challenged
• Missing referrers, or the same deep URL hit repeatedly with nothing loaded before it
• Concurrency that opens many connections at once against a site that expects one page load at a time

Blocks that are really about the address

A small share of blocks genuinely are the address, and they have a distinct feel. They arrive before you have done anything. They do not care how polite your crawler is. And they show up on unrelated sites at the same time, because the address's reputation travels with it.

A single Cloudflare challenge on a single site is not proof; some sites challenge every visitor. A challenge everywhere is a real signal. The same logic applies to Google: an occasional captcha during heavy searching is rate control, but a captcha on the first query, in a fresh browser, is reputation. When you see that pattern, the address needs to change, and that is where we come in.

• A Cloudflare challenge on nearly every protected site you open, including ones you have never scraped
• A Google captcha on the very first search, and again after solving it
• A 403 from a major site's front page in a plain browser with a clean profile
• The address listed on a major abuse database for spam or attacks in the past few days

A ten-minute triage

Here is a quick way to tell the piles apart without guessing. Open a normal browser, point it at the proxy, and visit the target and two or three unrelated protected sites by hand. If the browser sails through and only your crawler is blocked, the block is request-shaped. If the browser is challenged everywhere, the block is address-shaped.

Then rotate, which you can do per request or on demand, and repeat the browser check on the new address. Two addresses in a row that are challenged everywhere on the same carrier and city are rare, and worth telling us about. One address that is challenged and a second that is clean tells you the first one had a bad history from a previous subscriber, and you have already solved it.

When the address is the problem, what we do

If the triage points at the address, message support in the live chat on the site and say which sites challenged you and what you saw. We rotate the address for you, move you to a different carrier in the same city, or move the proxy to another metro free of charge. We run the modems ourselves in New York, Los Angeles, Chicago, Houston, Phoenix, Miami, North Carolina and Boston, so a move is a configuration change, not a wait.

One honest caveat. If every address we give you gets challenged on the same target, the target is responding to your traffic pattern and not to any address, and no move will change that. At that point the useful conversation is about request rate, headers and session handling, and we are happy to have it.

Frequently asked