Charging AI agents: pay per crawl, x402 and paid MCP tools
AI crawlers and agents read a great deal of the web, and until recently there was no simple way to charge them. That is changing through an old idea: HTTP status 402, Payment Required. The current HTTP standard still describes it as reserved for future use (RFC 9110). The future has arrived in three forms.
Three tollbooths
Pay per crawl is Cloudflare's. A site owner sets a price, a crawler that has agreed to pay gets the page, and one that has not gets a 402 with the price. Cloudflare is the merchant of record and handles billing (Cloudflare docs). Cloudflare is replacing pay per crawl with Pay Per Use. Pay Per Use has been in beta since 30 September 2026, and it pays publishers when their content is used in an answer, not each time a page is fetched (Cloudflare blog).
x402 is an open standard for paying over HTTP (x402.org). A request without payment gets a 402 with the terms in a PAYMENT-REQUIRED header. The caller signs a payment in USDC, a stablecoin that Circle says is always redeemable 1:1 for US dollars (Circle), and sends the request again with a PAYMENT-SIGNATURE header. The answer comes back with a receipt in a PAYMENT-RESPONSE header. There is no account to create and no API key to manage.
Cloudflare also has a Monetization Gateway that applies x402 payment rules to matching traffic. It is a closed beta, and buyers and sellers must be based in the United States (Cloudflare docs). An x402 endpoint you run yourself on a Cloudflare Worker has no such limit: it works from any country.
Paid MCP tools apply the same idea to the Model Context Protocol, the standard way AI agents call tools (modelcontextprotocol.io). In Cloudflare's Agents SDK, a server registers a tool with paidTool and a price in US dollars, such as 0.01 per call, and a caller without payment gets a 402 (Cloudflare docs). On the other side, an agent built with withX402Client can pay, either automatically or after asking a person to confirm (Cloudflare docs).
What we did on this site
We did not charge for pages. This site sells GEO work, so being read and cited by AI assistants is the point, and our robots.txt welcomes every major AI crawler. Charging crawlers here would cost us the visibility we sell.
Instead we put a price on one tool: the GEO audit. People still run it free on the GEO audit page. For agents, we built the same audit as a pay-per-call API at /api/agent/geo-audit, paid per audit over x402 on a Cloudflare Worker, which works from any country. It stays switched off until we set the wallet that receives the payments, and the free price list at /api/agent always shows whether it is live.
Three rules keep it fair to the agent paying. Rate limits and address checks run before payment, so a refused request is never charged. The payment is only settled after the audit succeeds, so a site that cannot be reached costs nothing. And a missing or malformed wallet setting keeps the paid route off, so nothing can be paid to the wrong place.
From testing on 13 November 2026: a request without payment gets a 402 asking for 50,000 base units of USDC, the on-chain form of the $0.05 price. A correctly signed payment from an empty test wallet was refused by Coinbase's test-network facilitator with the reason insufficient balance, which is exactly what should happen. The whole Worker, payment library included, is 181 KB compressed.
Prepaid credits: the other way to sell to agents
Our WebDesk MCP server takes a different approach. Agents use an API key and spend prepaid credits, one per website created. When the credits run out, the agent calls buy_credits and gets a Stripe Checkout link for a person, or an agent that can pay by card, to complete. Credits suit larger purchases, and the money arrives in ordinary currency. Per-call payments suit small, frequent ones.
Who should charge
Most small sites will earn little from crawlers, and charging the wrong bots costs AI visibility (see which AI bots do what). A price makes sense when you have something an agent needs and cannot easily get elsewhere: original data, a working tool, fresh prices or listings. If that is you, our AI Tollbooth Setup covers the crawler policy, Cloudflare's paid crawler access where your account has it, and paid x402 endpoints that work from any country, tested on a free test network before real money moves.
Sources
- Our test: the paid GEO audit on a local Worker against the x402.org test-network facilitator, 13 Nov 2026
- RFC 9110: HTTP Semantics, 402 Payment Required, IETF
- What is pay per crawl?, Cloudflare
- Pay Per Use: when AI uses your work, you should get paid, Cloudflare, 30 Sept 2026
- Monetization Gateway, Cloudflare
- x402: the payment protocol for the internet, x402.org
- Charge for MCP tools, Cloudflare
- Pay from the Agents SDK, Cloudflare
- What is USDC?, Circle
- Model Context Protocol, modelcontextprotocol.io