Redfin Mobile Proxies for Housing Research

Redfin is a brokerage as well as a listings site, and that shapes the data it exposes: listing details, sold history, the Redfin Estimate, agent notes, and a public market-data program that many researchers never look at before they start writing a crawler. A good Redfin pipeline starts with what the company already publishes, then fills the gaps with careful, low-rate collection of public pages. This page walks through that order of operations, the quirks of Redfin's pages and internal endpoints, and how a WeProx dedicated US mobile line keeps the collection part steady and honest.

Start with what Redfin already publishes

Before any crawler, check the Redfin Data Center. It publishes downloadable housing market data at national, metro, city, zip and neighborhood levels, including median sale price, inventory, days on market and sale-to-list measures, updated on a regular schedule. For trend work that is cleaner than anything you can reconstruct from listing pages, and it is offered for download. Search results pages also offer a download link for the homes in the current view, up to a limit. Between those two sources, a surprising number of research questions are answered with zero crawling.

What those sources do not give you is property-level change over time for a specific watchlist: when a price cut landed, when status moved to pending, when the Redfin Estimate shifted. That is where a small, scheduled collection job earns its place.

The JSON behind the pages

Redfin's web app fetches much of its data from internal endpoints that return JSON, and those responses begin with a short guard prefix before the actual object. A parser that feeds the raw body to a JSON decoder fails every time, and a surprising number of crawler bug reports come down to that prefix. Strip it, then parse. These endpoints are undocumented and change without notice, so pin your parser to fields you actually need and fail loudly when one disappears instead of writing nulls into your dataset for a week.

Redfin's robots file disallows a number of paths, and its terms of use limit automated access. Read both before you start, keep to public listing pages you have a legitimate reason to monitor, and keep volume proportional to that reason.

Pacing a watchlist through one line

A watchlist job is naturally gentle: a fixed set of property URLs checked once or twice a day. Run it as a series of short sticky sessions on a WeProx line, each covering a handful of properties in one neighborhood, with pauses between pages that vary rather than repeat. Rotate between sessions. Because each line is one real carrier device dedicated to you, the address has no history from another customer's aggressive job, and because carrier-grade NAT puts many real phones behind the same public IP, Redfin has to judge the traffic on behavior rather than blocking the address outright. That only helps if the behavior is reasonable.

Unlimited data means the job can load full pages with images and scripts in a real browser when you need rendered values, without budgeting gigabytes.

• Check each property no more often than its data actually changes.
• Keep one or two concurrent sessions per line against Redfin.
• Back off for hours, not seconds, after repeated 429 or 403 responses.

Where location changes the answer

Redfin operates as a brokerage in specific markets, and some features, including agent availability, tour booking and, in places, estimates, depend on whether an area is a Redfin service market. Those differences follow the property's location, not yours. The visitor's IP mostly affects default views and suggested searches. If you are studying how the homepage and featured content look to shoppers in Los Angeles versus Houston, a line in each metro gives you that; for watchlists, any available WeProx city is fine.

When the numbers look wrong

If an estimate is missing, check whether the property is in an area Redfin estimates at all before assuming a block. If sold prices are blank, the state may be a non-disclosure state where sale prices are not public record. If the page body shrinks suddenly across one line only, that line's recent rate was too high; pause it. If every request fails right after a rotation, wait for the modem to finish re-attaching and confirm the new exit IP.

Setting up a Redfin proxy on WeProx

1. Pull the Redfin Data Center files that cover your metros before writing any collection code.
2. Build a watchlist of specific property URLs that the downloads cannot answer.
3. Create a WeProx line and copy host, port, credentials and rotation URL.
4. Strip the guard prefix from JSON responses before parsing.
5. Check properties in short sticky sessions grouped by neighborhood, rotating between them.
6. Alert on missing fields and shrinking bodies rather than only on status codes.

Redfin proxy questions