We measured it: AI vs. code for scraping 1,000 products

The short version

• Accuracy was close. Our extraction code got all 1,000 books right. On the pages we sampled, the larger AI model got 800 of 800 fields right. The smaller one got 798 of 800, and one of its two mistakes was a web link it made up.
• Everything else was not. Code used no AI tokens and took 17 milliseconds per page to extract the data. The models used about 9,400 to 10,000 tokens and took 3.6 to 7.0 seconds per page. On the free tier, raw pages were refused outright.
• At 50,000 pages, letting the model read every page means about 353 million input tokens per run. That's roughly $60 to $2,200 per run depending on the model, paid again on every run. Real product pages are bigger than these practice pages, so treat that as a floor.

What we tested, and why these sites

We used two websites built for people learning to scrape: books.toscrape.com, a fake bookshop with 1,000 books across 50 catalogue pages, and quotes.toscrape.com, with 100 quotes across 10 pages. They exist to be scraped, they don't change, and anyone can repeat what we did.

They are also easy. The markup is clean, nothing depends on JavaScript, and there is no bot protection. That flatters both approaches, and we come back to it under limitations.

How we ran it

Approach A: code

This is how a scheduled Siftyr harvest works. Each page is downloaded once, one request at a time, at least 1.1 seconds apart. Fields are pulled out with CSS selectors (rules like "the title is in the title attribute of the link inside each product card"), then cleaned and typed: prices become numbers, star ratings become 1 to 5.

To make sure the code itself was right, we compared its output with a second, independent parse of the same pages using different selectors, and with the totals each site publishes. All 1,100 rows agreed, and the bookshop's own counter says 1,000 books, which is what we found.

Approach B: the model reads every page

For a sample of pages (8 of the 50 book pages, spread evenly, and 3 of the 10 quote pages), we sent each model the exact HTML the code had downloaded and asked for the same fields as JSON: title, price, rating, in stock and link for books; text, author and tags for quotes.

• Models: gpt-oss-20b and gpt-oss-120b, OpenAI's open-weight models, served by Groq on its free tier. Every sampled page went to both.
• Inputs: the raw HTML as downloaded, and a compacted version with scripts, styles and page headers removed, which is about 30% smaller.
• One change from the defaults: we raised the output limit to 8,192 tokens. In a trial run the provider's default limit of 2,048 cut the smaller model off halfway through a 20-book answer, and we wanted to measure the model, not the limit.

Scoring

Every field the model returned was compared with the code's value for the same row, after normalising things that don't change meaning: whitespace, HTML entities, a price compared as a number to the cent, a relative link resolved to a full one. A wrong value that appears nowhere on the page counts as invented.

Results

Book pages, with the models reading compacted HTML (raw HTML was refused, see below):

Tokens per page are prompt plus output: 7,072 in + 2,372 out (20b) and 7,072 in + 2,908 out (120b). Time is extraction time for code and the model's average response time for AI. Both approaches also spend the same time downloading each page.

Neither model missed a row, returned a duplicate, or added a book or quote that isn't on the page.

On the quotes site, both models got 90 of 90 fields right from compacted HTML. From raw HTML they got every word right but sometimes changed the typography, turning straight apostrophes into curly ones or three dots into an ellipsis: 6 of 90 fields for the smaller model and 2 of 90 for the larger. A reader wouldn't notice. A database matching on the exact text would.

What went wrong for the AI

One invented link

On one page, gpt-oss-20b put a space inside a product link (…save-the- earth_751/…). That link doesn't exist anywhere on the site. It looks right at a glance, which is the problem. The same model also returned one title with a stray backslash in it.

One error in 800 fields is far too few to estimate a rate from. But for scale: 50,000 pages of 20 products with 5 fields each is 5 million fields per run. If the rate held, that would be about 6,000 wrong values a run, each one plausible-looking.

Raw HTML was refused

A raw book page is about 9,900 prompt tokens. Groq's free tier allows 8,000 tokens per minute per model, and it rejects any single request bigger than that with "Request too large". All 16 raw book-page requests were refused. That's a limit of the free tier, not of the models, and a paid tier would accept them. They would also cost about 43% more input tokens than compacted pages.

Rate limits set the pace

Each book page used 9,400 to 10,500 tokens, more than a whole minute's allowance. The AI part of the run took 23.3 minutes for 28 accepted calls. The models were actually working for 119 seconds of that. The rest was waiting for the token allowance to refill, which works out to about one catalogue page every 65 to 75 seconds per model.

Slower even without limits

Ignoring rate limits entirely, the models took 3.6 and 7.0 seconds to answer for each page. The code took 17 milliseconds. That's roughly 200 to 400 times longer for the extraction step.

Half the output was the model thinking

The gpt-oss models reason before they answer, and that reasoning is billed as output. It was about half of every answer: 1,218 of 2,372 output tokens per page for the smaller model, and 1,665 of 2,908 for the larger.

What 50,000 pages would cost

This part is an extrapolation, not a measurement. We treat 50,000 pages as our 50-page catalogue repeated 1,000 times, and use the token counts we measured.^a The model bill depends only on the price per token, so we can apply any model's list price to the same counts.

^a Cost per run = (input tokens in millions × input price) + (output tokens in millions × output price). Input is 352.8 million tokens: the measured size of each of the 50 compacted pages plus 261 tokens of instructions per request, times 1,000. Output is 118.6 million tokens (gpt-oss-20b's measured 2,372 per page) or 145.4 million (gpt-oss-120b's 2,908). We didn't run Claude, so its rows show both output sizes as a range; a model that doesn't reason first would write roughly half as much. Example: 352.8 × $2 + 118.6 × $10 = $1,892. List prices read on October 20, 2026, without caching or batch discounts. Other models' tokenizers count text slightly differently.

How long it would take

Downloading in parallel shortens the first row for both approaches equally. The AI rows come on top of it.

Real product pages are bigger

Our practice pages are small: about 9,800 tokens of raw HTML and 6,800 after compacting. That's close to a typical web page, whose HTML has a median size of 33 to 35 KB, or roughly 8,000 tokens.³ Retail product pages tend to be heavier. Published studies measured an average of 44,706 raw tokens per page,⁶ 114,319 raw tokens per page,⁴ and 477 KB of HTML per product page (around 120,000 tokens).⁵

Cleaning helps a lot. The same studies cut pages down to 5,735⁶ and 16,985⁴ tokens. But cleaning has to be done well for each site, and the format you hand the model matters: one study found the same model's accuracy went from very poor to very good depending only on how the page was simplified.⁷

Input cost grows in step with page size. Here is the input bill alone, for 50,000 pages, before any output:

Input tokens × input price, no output. * gpt-oss reads at most 131,072 tokens at once, so many raw product pages wouldn't fit at all. † Haiku 5.5 charges $0.50 per million input tokens on prompts over 100,000 tokens, five times its normal rate.²

What this means

To be fair to the models: on clean pages, a current AI model reads a page about as well as code does. If you need a few hundred pages once, having a model read them is a reasonable choice, and you'll probably get good data.

A data feed is a different job. It's the same kinds of pages, every day. Code costs almost nothing per page once it's written. A model charges for every page on every run, takes seconds per page, runs into rate limits, and its rare mistakes look exactly like real data.

That's why Siftyr uses AI once per site, not once per page. You describe the fields you want in plain English. The AI looks at a sample page and proposes selectors, someone checks them, and from then on code reads every page. Every harvest is then checked against the last one (row count, fill rate, schema), and a harvest that looks wrong is held back instead of delivered. A planning call reads one compacted page and writes a short plan, around 6,000 tokens in and 500 out. At the list prices above, that's about a tenth of a cent per site.^b

Independent research points the same way. A 2025 study of 3,000 product pages found that having the model write extraction code, instead of reading every page, cost 1.6 points of accuracy (96.48% vs. 98.09%) and needed 95.82% fewer model calls.⁸ Another found that model-written scripts pay off once a site has at least 4 pages with the same layout.⁴ Pinterest reported a per-page cost of $0.0000079 for its non-AI extractor against $0.002027 for GPT-4o-mini, at 2025 prices.⁵

Code has a cost of its own: someone has to write the selectors, and fix them when a site changes its layout. We didn't measure that here. It's the reason we check every harvest, and the reason fixing broken extractors is our ops team's job rather than yours.

^b 6,000 × $0.15 per million + 500 × $0.60 per million = $0.0012 on gpt-oss-120b at Groq's price; 6,000 × $0.10 + 500 × $0.50 per million = $0.00085 on Haiku 5.5.

Limitations

• Small sample. The AI results rest on 8 book pages (160 rows) and 3 quote pages (30 rows) per model. A perfect score on 160 rows does not mean a perfect score on 50,000 pages.
• One run. Model output varies from run to run. Output per book page ranged from 1,735 to 3,553 tokens within this one run.
• Easy, static sites. Real sites have JavaScript, bot protection, logins and layouts that change. That's harder for both approaches: selectors break, and models face longer, messier pages.
• The free tier shaped the test. We couldn't test raw HTML on book pages at all, we kept the sample small to stay inside daily limits, and response times may differ on paid tiers.
• Reasoning models. About half of each answer was reasoning. A model that doesn't reason first, or a lower reasoning setting, would use fewer output tokens. We didn't test that.
• Only listing pages. We didn't test product detail pages, finding the next page, or the "AI writes selectors once" approach itself in this run.
• Other models are priced, not tested. The Claude rows apply list prices to gpt-oss token counts. A different model could be more or less verbose.
• We have a stake in the answer. Siftyr is built on the code approach. That's why we've published the method, every error the models made, and the parts that went in their favour.

Reproduce it

The benchmark is a script in Siftyr's codebase, and the method above is complete enough to repeat with any HTML parser and any AI provider with an OpenAI-compatible API:

1. Download all 50 catalogue pages of books.toscrape.com, one request at a time.
2. Extract the five fields with CSS selectors, and check the result against a second parse and the site's count of 1,000.
3. Send a spread-out sample of the same HTML to a model, raw and with scripts, styles and the page head removed, asking for the same fields as JSON.
4. Score each field against the code's output, and record prompt tokens, output tokens and response time from the API.
5. Multiply the measured tokens by the prices you pay.

Sources

Measured figures come from our own run on October 20, 2026, described under How we ran it. Everything else is below. All pages were read on October 20, 2026.

1. Groq, model cards for openai/gpt-oss-20b and openai/gpt-oss-120b (prices, context length, developer-tier rate limits). console.groq.com/docs/models
2. Anthropic, Claude pricing (Sonnet 5.5, Haiku 5.5 and Haiku 5.5's long-prompt rate). claude.com/pricing
3. HTTP Archive, Web Almanac 2025, Page Weight chapter, January 15, 2026 (median HTML document 33 KB mobile, 35 KB desktop). almanac.httparchive.org/en/2025/page-weight
4. SCRIBES, arXiv 2510.01832, October 2, 2025, revised August 17, 2026 (114,318.6 raw tokens per page on average; 16,985.1 after de-duplication; scripts pay off at 4+ similar pages). arxiv.org/html/2510.01832v2
5. Pinterest, product attribute extraction, arXiv 2508.01096, August 1, 2025 (mean HTML 477 KB per product page; cost per page by method). The token estimate of about 120,000 is ours, at about 4 characters per token. arxiv.org/abs/2508.01096
6. AICC / MinerU-HTML, arXiv 2511.16397, November 20, 2025 (44,706 raw input tokens, 5,735 after preprocessing). arxiv.org/html/2511.16397v2
7. NEXT-EVAL, arXiv 2505.17125, May 21, 2025 (same model scored F1 0.10 on slimmed HTML and 0.96 on flat JSON). arxiv.org/html/2505.17125
8. LLM-generated extraction code vs. direct LLM extraction on 3,000 food product pages, arXiv 2506.21585, June 17, 2025 (96.48% vs. 98.09% accuracy; 95.82% fewer LLM calls). arxiv.org/abs/2506.21585

Related: The carbon cost of scraping the web with AI uses these measurements to estimate energy use, and Can't I just scrape it with ChatGPT? covers everything else that a data feed needs.

Send us a site. We'll send you a sample.

Tell us the sites, the fields and how often you need them. You'll get a checked sample before you commit to anything.

Request a sample

books.toscrape.com and quotes.toscrape.com are public sandboxes for scraping practice. gpt-oss is a model family released by OpenAI; Groq and Claude are trademarks of their owners. Siftyr isn't affiliated with any of them. Model prices change often; the prices here are list prices read on October 20, 2026.