Proxies in Rust: reqwest with HTTP, SOCKS5 and rotation

reqwest is the HTTP client most Rust projects reach for, and its proxy support covers everything you need: authenticated HTTP proxies, SOCKS5 behind a feature flag, and per-client settings that decide whether your IPs rotate. This guide walks through each, ending with a concurrent fetcher.

Your username and password are in the generator. The examples use the Residential gateway geo.crawlproxies.com.

Setup

[dependencies]
reqwest = { version = "0.12", features = ["socks"] }
tokio = { version = "1", features = ["full"] }
futures = "0.3"

The socks feature is only needed for SOCKS5; leave it out if you use the HTTP port.

HTTP proxy

use std::time::Duration;

#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
    let proxy = reqwest::Proxy::all("http://geo.crawlproxies.com:8080")?
        .basic_auth("USERNAME", "PASSWORD");

    let client = reqwest::Client::builder()
        .proxy(proxy)
        .timeout(Duration::from_secs(30))
        .build()?;

    let res = client.get("https://ipinfo.io/json").send().await?;
    let status = res.status();
    let body = res.text().await?;
    println!("{status} {body}");
    Ok(())
}

Proxy::all sends both http:// and https:// URLs through the proxy; HTTPS goes through a CONNECT tunnel, so the connection to the website stays encrypted. basic_auth keeps the password out of the URL, so special characters need no escaping.

SOCKS5

With the socks feature enabled:

let proxy = reqwest::Proxy::all("socks5h://USERNAME:PASSWORD@geo.crawlproxies.com:1080")?;

socks5h lets the proxy resolve hostnames; with socks5 the DNS lookups happen on your machine.

Making IPs rotate

With a rotating username, every new connection gets a new IP. reqwest keeps idle connections in a pool and reuses them, so a loop of requests through one client can share an IP. Turn the pool off when you want a new IP per request:

let client = reqwest::Client::builder()
    .proxy(proxy)
    .pool_max_idle_per_host(0)   // no reused connections, so a new IP every request
    .build()?;

Each request then pays for a fresh TCP and TLS handshake, so keep the pool on when rotation doesn't matter.

Sticky sessions per task

For the opposite (one IP across a login or a multi-step flow), put a session id and a duration in the username and give each task its own client:

fn sticky_client(session: &str) -> reqwest::Result<reqwest::Client> {
    let user = format!("USERNAME-country-us-session-{session}-time-1800");
    reqwest::Client::builder()
        .proxy(reqwest::Proxy::all("http://geo.crawlproxies.com:8080")?.basic_auth(&user, "PASSWORD"))
        .build()
}

Build one client per session and reuse it for every request in that flow. For the shared rotating client, clone it into each task instead: clones are cheap and share one pool. Sticky vs rotating sessions covers when to use which, and the geo-targeting guide lists every username option.

Concurrent requests with a cap

buffer_unordered from the futures crate runs up to N requests at once:

use futures::stream::{self, StreamExt};

let urls = vec!["https://example.com/a", "https://example.com/b", "https://example.com/c"];

let results: Vec<_> = stream::iter(urls)
    .map(|url| {
        let client = client.clone();
        async move {
            let status = client.get(url).send().await.map(|r| r.status());
            (url, status)
        }
    })
    .buffer_unordered(10)
    .collect()
    .await;

for (url, status) in results {
    match status {
        Ok(s) => println!("{s} {url}"),
        Err(e) => println!("error {url}: {e}"),
    }
}

Start around 10 and raise it gradually while watching for 429 responses.

Errors you might see

reqwest wraps the root cause, so print errors with {:?} or walk std::error::Error::source() to see the full chain.