A data team I spoke with last year ran an eight-figure product catalogue crawl entirely through Puppeteer. Every single URL, including plain server-rendered category pages that returned complete HTML on the first byte, went through a full Chromium instance. Their residential bandwidth bill was roughly forty times what it needed to be, and their block rate was higher than a simpler setup would have produced, because the browser was firing hundreds of third-party requests per page through the same exit IP.
That is the practical cost of treating "web scraping" as one problem with one tool. Cheerio and Puppeteer solve different halves of it, and the proxy infrastructure each one needs differs in almost every dimension that matters: bandwidth consumed per page, how credentials are passed, how sessions bind to IPs, and how much fingerprint surface you expose to anti-bot systems.
This article breaks down those differences in detail, shows how to configure proxies correctly in both, and explains how to build a hybrid pipeline that only pays the browser tax when the target actually requires it.
What Cheerio Actually Is (and What It Is Not)
Cheerio is not a scraper. It is a server-side implementation of a jQuery-like API for traversing and manipulating an HTML string. It does not make network requests, it does not execute JavaScript, it does not maintain cookies, and it has no concept of a proxy.
That last point is the one that trips people up. When someone asks "how do I set a proxy in Cheerio?", the answer is that you do not. You set the proxy on whatever HTTP client fetches the markup: axios, got, node-fetch, undici, or the native fetch in modern Node. Cheerio receives a string and parses it.
import axios from 'axios';import * as cheerio from 'cheerio';import { HttpsProxyAgent } from 'https-proxy-agent';const agent = new HttpsProxyAgent( 'http://USER:[email protected]:8000');const { data } = await axios.get('https://target.example/catalog?p=3', { httpAgent: agent, httpsAgent: agent, headers: { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) ...', 'Accept-Language': 'en-GB,en;q=0.9' }, timeout: 15000});const $ = cheerio.load(data);const items = $('[data-product-card]').map((_, el) => ({ sku: $(el).attr('data-sku'), price: $(el).find('.price').text().trim()})).get();
The entire network footprint of that request is one TCP connection, one TLS handshake, one HTTP GET, and one HTML document. Typically 30KB to 300KB depending on the page. Nothing else is fetched: no images, no fonts, no analytics beacons, no ad-tech pixels.
What Puppeteer Actually Does
Puppeteer drives a real Chromium build over the DevTools Protocol. When you call page.goto(), the browser does everything a browser does. It resolves the document, then parses it, then fetches every subresource referenced in it, then executes the JavaScript, which in turn triggers XHR and fetch calls, which trigger more subresources.
A single modern e-commerce product page routinely generates 80 to 250 requests and pulls 2MB to 6MB of data. All of that traffic goes through your proxy, because the proxy is set at the browser process level.
This is not a flaw. It is the entire reason to use Puppeteer: you get the fully hydrated DOM, client-side rendered content, and behavioural signals (real TLS fingerprint, real HTTP/2 frame ordering, real JavaScript execution) that a bare HTTP client cannot produce. But you pay for it in bandwidth, CPU, and latency.
The Five Ways Proxy Requirements Diverge
1. Bandwidth economics dominate the decision
If you are on a metered residential plan, the cost difference between the two approaches is not marginal. Fetching 100,000 static pages at an average of 120KB each consumes roughly 12GB. Rendering those same 100,000 pages in Puppeteer at an average of 3MB each consumes roughly 300GB.
That ratio is what makes pool selection a budget question rather than a technical preference. Static Cheerio pipelines can often afford premium residential IPs precisely because they use so little data per request. Headless pipelines frequently have to lean on datacenter or ISP pools for the bulk of traffic and reserve residential for the pages that genuinely need it.
You can claw back a large share of headless bandwidth with request interception, and you should:
await page.setRequestInterception(true);page.on('request', (req) => { const type = req.resourceType(); if (['image', 'media', 'font', 'stylesheet'].includes(type)) { return req.abort(); } req.continue();});
Blocking images, fonts, media and stylesheets commonly cuts per-page transfer by 60 to 85 percent. Be careful though: some anti-bot vendors check whether their own script and its dependencies were loaded, and aggressive blocking of CSS can break layout-dependent selectors or trip "headless likely" heuristics. Block by domain and resource type deliberately, not with a blunt allowlist.
2. Authentication is handled completely differently
With an HTTP client, proxy credentials go straight into the agent URL or a Proxy-Authorization header. It is one line and it works per request, which means you can rotate credentials (and therefore sticky-session identifiers) on every call with zero overhead.
Chromium does not accept inline credentials in --proxy-server. Passing http://user:pass@host:port on the command line is silently ignored or rejected, which is one of the most common reasons a Puppeteer proxy setup "just doesn't work". You have two supported routes:
const browser = await puppeteer.launch({ args: ['--proxy-server=http://gateway.example.net:8000']});const page = await browser.newPage();await page.authenticate({ username: 'USER', password: 'PASS' });
The page.authenticate() call responds to the proxy's 407 challenge. It must be issued before the first navigation, and it applies per page, so every new tab needs it again.
The alternative is IP whitelist authentication, where the proxy provider trusts your server's outbound IP and no credentials are exchanged at all. For headless fleets running on fixed infrastructure, whitelisting removes an entire class of bug. For containers on ephemeral IPs it is usually impractical.
3. Session binding granularity
This is the deepest architectural difference.
With Cheerio plus an HTTP client, the proxy is bound to a request. You can hit ten different countries in ten consecutive lines of code from the same process, and nothing carries over except the cookies you explicitly manage. Rotation is free.
With Puppeteer, the proxy is traditionally bound to the browser process, because --proxy-server is a Chromium launch flag. Changing the exit IP means launching a new browser, which costs 300ms to 2s and a chunk of RAM. Running 50 concurrent geo-targeted sessions historically meant 50 Chromium processes.
Modern Puppeteer softens this with per-context proxies:
const context = await browser.createBrowserContext({ proxyServer: 'http://gateway.example.net:8000', proxyBypassList: ['<-loopback>']});const page = await context.newPage();await page.authenticate({ username: 'USER', password: 'PASS' });// ... work ...await context.close();
Browser contexts are isolated for cookies and storage, and each can use its own exit node. That makes it viable to run a handful of contexts per browser instance rather than one browser per identity. It is still heavier than a per-request swap, but it is a large improvement over spawning processes.
The consequence for pool choice: headless work strongly favours sticky sessions measured in minutes, because an IP change mid-session invalidates whatever the site associated with that connection. Static Cheerio work can use fast rotation, because each request is atomic.
4. Detection surface
An axios or got request from Node has a TLS fingerprint that looks nothing like Chrome. The cipher suite ordering, the extension list, the ALPN negotiation, and the HTTP/2 SETTINGS frame all identify the client as a Node runtime within the first packets of the handshake. No amount of User-Agent spoofing changes that.
Against a site with no meaningful bot defence, this does not matter. Against anything running behavioural or TLS-level fingerprinting, it matters enormously: the request can be flagged before your headers are even parsed. Libraries that emulate browser TLS profiles exist and help, but they add operational complexity.
Puppeteer, by contrast, produces genuine Chrome network fingerprints, because it is Chrome. Its exposure is elsewhere: navigator.webdriver, missing plugin arrays, headless-specific rendering differences in WebGL and canvas, and automation-shaped timing patterns. Stealth plugins patch many of these, though the patches themselves have detectable signatures.
The proxy implication is that the two tools want different IP quality for different reasons. Static clients often need higher-trust IPs to compensate for a weak client fingerprint. Headless browsers have a strong client fingerprint but generate so much traffic per page that an IP with borderline reputation burns out quickly.
5. Concurrency and pool sizing
A Node process running Cheerio can comfortably hold hundreds of concurrent HTTP requests in flight on modest hardware. Your ceiling is the target's rate limits and your proxy pool's diversity, not your own CPU.
Puppeteer's ceiling is memory. Each Chromium instance typically wants 150MB to 500MB of RAM depending on the page. A 16GB worker realistically supports 20 to 40 concurrent pages with careful tuning.
So for the same throughput target, a static pipeline needs a wider IP pool (more parallel requests means more simultaneous distinct IPs) while a headless pipeline needs fewer but longer-lived, higher-stability IPs. That single distinction should drive your plan sizing more than any marketing comparison of pool sizes.
Choosing Between Them: The Criteria That Actually Matter
Rather than picking a default, test the target and decide per domain.
Does the data exist in the initial HTML? Fetch the page with curl and search the raw response for a known value such as a price or a product title. If it is there, Cheerio is sufficient and Puppeteer is pure waste. If the HTML is a shell with a JSON blob hydrating it, you may still avoid a browser by parsing the embedded JSON directly.
Is there an underlying API? Open the network tab, filter to XHR, and look for the endpoint the front end calls. Hitting that JSON endpoint through an HTTP client is almost always faster, cheaper, and more stable than rendering the page. Many teams skip this step and go straight to headless.
Does the site gate content behind a JavaScript challenge? If the first response is an interstitial that runs a proof-of-work or fingerprint script before issuing a clearance cookie, a plain HTTP client will not get past it unaided. This is the clearest case for a browser.
Do you need interaction? Infinite scroll, "load more" buttons, multi-step forms, authenticated dashboards with client-side routing. Browsers only.
What is the volume? At 500 pages a day, the efficiency difference is academic. At 5 million pages a month, the bandwidth delta between the two approaches can exceed the cost of the engineering time needed to build the static path properly.
The Hybrid Pattern Worth Building
The architecture that holds up in production is a two-tier pipeline with escalation.
Tier one fetches every URL with an HTTP client through a cheap pool and hands the body to Cheerio. A validator checks whether the expected selectors matched and whether the response looks like a challenge page, an empty shell, or a soft block. If validation passes, you are done at a fraction of a penny per page.
If validation fails, the URL is pushed to a second queue handled by a Puppeteer worker pool on higher-trust IPs, with request interception enabled. Successful browser runs write back useful artefacts: the clearance cookie, the discovered XHR endpoint, the correct sticky session duration. Tier one then reuses those artefacts so that subsequent pages on the same domain drop back to the cheap path.
In most catalogues I have seen, this pattern routes 80 to 95 percent of URLs through the static tier while retaining full coverage. The proxy configuration follows the same split: broad, fast-rotating IPs for tier one, fewer sticky sessions with longer lifetimes for tier two.
Common Mistakes
Rotating the IP inside a live browser session. If you swap exit nodes while a Puppeteer context is mid-flow, the site sees a session whose IP changed between the document request and its own XHR calls. That is a strong automation signal. Bind the context to one IP for its lifetime.
Forgetting that subresources use the proxy too. Every analytics beacon and ad pixel in a headless page hits your proxy and consumes billable bandwidth, and every one of them is another observation of your exit IP by a third-party tracking network.
Setting a proxy on the HTTP agent but leaking DNS. Some client configurations resolve hostnames locally before connecting. Prefer proxy configurations that pass the hostname to the proxy for remote resolution.
Mismatching geography and locale in headless runs. A Chromium instance reporting a Berlin timezone and German locale behind a Texas exit node is trivially inconsistent. Set --lang, the emulated timezone, and the Accept-Language header to match the proxy's country.
Assuming Cheerio failures mean you need a browser. Half the time the real cause is a missing header, a cookie consent redirect, or a 403 from an IP with poor reputation. Diagnose before escalating.
Where Proxies Fit In
Both tools depend on the same thing: an exit IP the target is willing to serve. The difference is what each tool demands of that IP.
Static pipelines need breadth. Hundreds of concurrent requests should not repeatedly land on the same handful of addresses or the same ASN, because subnet-level clustering is one of the first things rate limiters look for. Fast rotation across rotating residential proxy pools with wide geographic spread is what keeps a Cheerio crawler under per-IP thresholds while maintaining throughput.
Headless pipelines need depth. Fewer IPs, held longer, with consistent geolocation and stable routing for the duration of a browser context. Session control matters more than raw pool size here, and so does pool type: ISP and datacenter IPs handle high-bandwidth rendering economically, while residential and mobile IPs are reserved for the targets that inspect IP class. Access to multiple pool types under one account is what makes the hybrid architecture practical, since EnigmaProxy exposes residential, ISP, datacenter and mobile options with pricing you can model against a known bandwidth profile rather than guess at.
Ethical sourcing is the part nobody sees in a benchmark but everyone feels in production. Pools built on consented peer networks behave predictably, keep reputable ASN neighbourhoods, and do not vanish when an enforcement action lands. Before you commit a pipeline to a pool, it is worth checking what a given exit node actually reports for country, ASN, and leak exposure, then repeating that check on a sample of the pool rather than a single lucky IP.
Strategic Insights: Where This Is Heading
Rendering is moving back to the server. Frameworks pushing server components and streamed HTML mean more content is present in the initial response again. Static parsing is becoming more viable for a slice of the web, not less, which rewards teams who kept a working HTTP path instead of standardising on a browser.
Detection is shifting further from the IP. TLS fingerprints, HTTP/2 frame ordering, and behavioural telemetry now carry more weight than reputation scores alone. That narrows the gap between tools in one sense and widens it in another: a clean IP cannot rescue a Node client against a TLS-aware defence, and a perfect browser fingerprint cannot rescue a burned subnet.
Browser automation is getting cheaper per page. Better interception APIs, lighter Chromium builds, and per-context proxy support are steadily reducing the overhead penalty. The gap will not close, but it is narrowing enough that the escalation threshold in hybrid pipelines will keep moving.
Agentic scraping will blur the categories. LLM-driven agents that navigate and extract in one loop sit firmly in the browser camp and hold sessions for minutes at a time. Infrastructure built around sticky, geographically consistent sessions is better positioned for that than infrastructure optimised purely for request throughput.
Conclusion
Cheerio and Puppeteer are not competitors. Cheerio parses markup that something else fetched, and its proxy behaviour is determined entirely by your HTTP client: cheap, atomic, per-request, and limited by a client fingerprint that sophisticated targets can read. Puppeteer runs a real browser with a genuine Chrome fingerprint and pays for it in bandwidth, memory, and coarser session binding.
The practical advice is unchanged by any framework release: check whether the data is in the initial HTML, look for the underlying API, use the static path wherever it works, and escalate to headless only for the pages that genuinely need rendering or interaction. Then match your proxy configuration to each tier rather than buying one pool and hoping it suits both.
Get that split right and the infrastructure stops being the bottleneck. A provider like EnigmaProxy, with multiple pool types and transparent sourcing, gives you the room to run both tiers on the terms each one actually requires.