Web Scraping Mobile Proxies: USA Carrier IPs for Data Jobs
Most scraping problems are not bandwidth problems. They are trust problems: the target has decided your address is a machine and is serving you a challenge, a soft block or a degraded page. A dedicated WeProx mobile line changes the category your requests fall into, because a US carrier address is shared by many real phones and the target has to treat it as a crowd. This page is for data teams collecting public information within a site's terms and rate limits. It covers how to scope sessions to rotations, why the carrier IP matters, how much concurrency a line should carry, how to pick a metro, and how to interpret the responses a site sends when it is unhappy.
Scope a session to an IP, then rotate
A scraper that changes address on every request looks like nothing a human does. A scraper that keeps one address for a realistic session, browsing a listing, opening a few detail pages, then leaving, looks like a phone. So build around sessions. Hold the IP sticky for one logical unit of work, with cookies and headers consistent inside it, then call the rotation link, wait for the new address to come up, and start the next session with a fresh cookie jar. The modem re-attaches to the carrier gateway and takes another address from the carrier's regional pool. That pool is finite and repeats happen, so key your state on session ID rather than on IP.
Rotation is unlimited and on demand, and it is triggered by a simple request to the line's rotation URL, which is easy to call from a job runner. Verify the exit IP after each rotation before resuming, because a request that lands during the re-attach will fail and can be mistaken for a target-side block. A short pause here saves a great deal of confused log reading later.
Why a carrier address changes the target's response
Carrier-grade NAT places many real subscribers behind one public address. A site that blocks that address blocks all of them, so anti-bot systems weight the IP lightly and lean on other signals, such as headers, TLS fingerprint and behaviour. That is the whole advantage, and it is why a mobile IP scores as high risk on fraud checkers while being handled gently by real sites: both are reading the same crowd. A datacenter IP has no crowd and is the easiest thing to block. A dedicated line adds exclusivity: your modem's history is yours, not the residue of another customer's careless job.
Many targets also expose mobile API endpoints for their apps that return cleaner data than the desktop HTML. Those endpoints expect carrier traffic, so requesting them from a carrier address with app-like headers is the most ordinary traffic they see. The endpoints are still the site's property and its terms still apply; the point is that when you are permitted to collect the data, the mobile path often gives you a structured response instead of markup you have to parse and re-parse every time the layout shifts.
Concurrency and jobs per line
A line gives you 20-45 Mbps on 4G or 50+ Mbps on 5G and unlimited data, so throughput is rarely the ceiling. Realism is. One public address that opens forty connections to one site at once is a fingerprint, however clean the IP. Keep a small number of concurrent sessions per line against any single target, spread lines across targets, and add lines for more parallelism rather than multiplying threads on one. Respect robots directives, published rate limits and the site's terms; a WeProx line changes the network path, not what you are permitted to collect. Group jobs so a target that challenges one line does not take a different client's job down with it.
Choosing the metro
For many data jobs any US carrier address will do, and you should take whichever city has stock. When the target localises content by region, such as pricing, inventory, delivery options or local search results, the metro is part of the dataset. Carrier IPs geolocate approximately to the metro or state, which is the granularity most sites use for localisation. Run the job during the metro's local daytime if the target adjusts behaviour by time. WeProx runs lines in New York, Los Angeles, Chicago, Houston, Phoenix, Miami, North Carolina and Boston, so regional sampling across the country is a matter of ordering one line per city.
Reading what the target sends back
A status code that says too many requests is the polite version: slow down, hold sticky, lengthen the pause between sessions. A challenge page served with a successful status is the quiet version, so check the body, not only the code, and treat a shrinking page size as a signal. Pages missing the data you expect often mean the site served a lite variant to what it thinks is a bot; compare against a manual load through the same line. A request that fails right after a rotation is the re-attach window, not a block. If an IP-echo request through the line fails, confirm credentials and port, then contact support with the line ID.
Keep logs with exit IP, session ID, status and body size for every request; patterns show up quickly. A body size that drops on one session and not its neighbours is a challenge aimed at that session. A drop across every session on a line is the target reacting to the address, and the answer is pacing, not a faster rotation. To size your fleet, look at plans on the WeProx homepage and choose metros on the USA locations page.
Setting up a Web scraping proxy on WeProx
- Create a line in your WeProx dashboard, choosing the metro your data needs or any city with stock.
- Copy the host, port, username, password and rotation URL from the line card.
- Put them in your scraper's proxy configuration, a browser automation profile, or a test phone's Wi-Fi proxy settings.
- Hit an IP-echo endpoint to verify the exit IP and metro, then hold sticky for each session and call the rotation URL between sessions.
Web scraping proxy questions
Is web scraping through a mobile proxy allowed?
That is decided by the target site's terms, its robots directives, applicable law and what data you collect. Public data gathered at a respectful rate within those limits is ordinary research and monitoring work. A WeProx line changes the network path only; it does not grant permission to collect anything a site forbids or to bypass access controls.
Rotating or sticky for scraping?
Sticky for one realistic session, then rotate. Hold the address while a session browses a set of pages with consistent cookies, call the rotation URL, verify the new exit IP, and begin the next session fresh. Per-request rotation looks mechanical and wastes the trust a carrier address carries. Rotation is unlimited and on demand.
How many concurrent jobs per line?
Keep a small number of concurrent sessions per line against any single target and add lines for more parallelism. Bandwidth is not the constraint with unlimited data at 20-45 Mbps on 4G or 50+ Mbps on 5G; realism is. Dozens of simultaneous connections from one address is a fingerprint regardless of how clean the IP is.
Can I test the target from my phone through the line?
Yes. Enter the host, port, username and password in the phone's Wi-Fi proxy settings and browse the target as a subscriber would. It is a useful way to compare what a real mobile user sees against what your scraper receives, especially when a site serves a different layout or a challenge to automated clients.
Do you have US IPs in specific metros for localised data?
Lines are US carrier SIMs on AT&T, T-Mobile and Verizon in New York, Los Angeles, Chicago, Houston, Phoenix, Miami, North Carolina and Boston. Geolocation is approximate at the metro or state level, which is the resolution most sites use when they localise prices, inventory or results, so one line per city covers regional sampling.
Will a mobile line slow my scraper down?
Per-request latency is higher than a datacenter proxy because traffic crosses the carrier network, and throughput is 20-45 Mbps on 4G or 50+ Mbps on 5G. For session-based collection that mirrors human pacing, this is not the bottleneck, and unlimited data means long-running jobs never stop for an allowance.