AI-crawler traffic report
As of 2026-09-05, ChangeGamer (changegamer.ai) received 1,743 AI-crawler requests in the trailing 30 days, 1,090 of them user-initiated retrievals (ChatGPT-User + Claude-User) — 63% of AI-crawler traffic and 3% of all 59,321 worker-served requests recorded in that window.
Key facts
- On 2026-09-05, AI crawlers made 1,743 requests to ChangeGamer in the trailing 30 days, +44 vs the 2026-08-29 snapshot, out of 59,321 total worker-served requests (3%).
- GPTBot (OpenAI's crawl-and-index bot) made 148 requests as of 2026-09-05, −5 vs 2026-08-29.
- ChatGPT-User (live, user-triggered ChatGPT browsing) made 1,065 requests as of 2026-09-05, −32 vs 2026-08-29 — the single largest source of AI-crawler traffic to the site.
- Anthropic's crawlers combined (ClaudeBot + Claude-User) made 137 requests as of 2026-09-05, −3 vs 2026-08-29.
- PerplexityBot made 356 requests as of 2026-09-05, +87 vs 2026-08-29.
- The most-fetched page in the 2026-09-05 snapshot was `/resources/synthetic-data-generation` with 157 AI-crawler hits in 30 days — see the full top-20 table below.
AI crawlers by bot (latest vs previous snapshot)
| Bot | 2026-09-05 | Previous | Delta |
|---|---|---|---|
| AI total | 1,743 | 1,699 | +44 |
| GPTBot | 148 | 153 | −5 |
| OAI-SearchBot | 37 | 40 | −3 |
| ChatGPT-User | 1,065 | 1,097 | −32 |
| ClaudeBot | 112 | 115 | −3 |
| Claude-User | 25 | 25 | no change |
| PerplexityBot | 356 | 269 | +87 |
| Google-Extended | 0 | 0 | no change |
| Bytespider | 0 | 0 | no change |
| All worker hits | 59,321 | 56,146 | +3,175 |
Top 20 pages AI crawlers fetch (rolling 30 days, snapshot 2026-09-05)
| Rank | Page | AI-crawler hits (30d) |
|---|---|---|
| 1 | synthetic-data-generation | 157 |
| 2 | evaluating-ai-agents | 106 |
| 3 | durable-execution-for-agents | 86 |
| 4 | web-data-for-agents | 77 |
| 5 | generative-ui-for-agents | 66 |
| 6 | open-weight-models-for-agents | 62 |
| 7 | computer-use-browser-automation | 56 |
| 8 | essential-agent-publications | 56 |
| 9 | reliable-tool-calling | 49 |
| 10 | streaming-for-agents | 35 |
| 11 | multi-agent-orchestration-patterns | 31 |
| 12 | document-extraction-for-agents | 31 |
| 13 | mcp-2026-spec-revision | 30 |
| 14 | llms-txt-explained | 29 |
| 15 | agent-delegation-chains | 28 |
| 16 | agent-observability | 27 |
| 17 | agent-memory-context | 25 |
| 18 | content-signals-explained | 25 |
| 19 | code-execution-sandboxing | 25 |
| 20 | agent-spend-controls | 24 |
Method
The worker that serves every ChangeGamer page logs each request to Cloudflare Analytics Engine, tagged by user-agent. A scheduled GitHub Actions job (.github/workflows/seo-crawler-stats.yml) queries that log weekly and commits the snapshot this report renders. Two limits matter: asset-served paths — /llms.txt, robots.txt, the sitemap — bypass the worker entirely and are deliberately NOT counted, so every number here is a floor on crawler interest, never a total; and user-agent strings are self-declared by the crawler, not independently verified, so a bot that spoofs or omits its user-agent will not appear correctly (or at all). Read this as retrieval demand, not citation — a crawl means a page was fetched, not that it was used or cited in an answer.
Reuse
This dataset is free to quote, republish, or build on — with attribution to ChangeGamer (https://changegamer.ai/reports/ai-crawler-traffic) — no key or license required. Machine-readable twin: https://changegamer.ai/api/reports/ai-crawler-traffic.json.
Frequently asked questions
- What counts as an "AI-crawler request" in this report?
- A request whose user-agent matches one of eight declared AI bots — GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, Bytespider — as logged by the ChangeGamer worker to Cloudflare Analytics Engine. It is a fetch, not a citation: see the Method section above.
- Why do the totals exclude /llms.txt and robots.txt?
- Those paths are served directly by Cloudflare as static assets and never pass through the logging worker, so no per-request user-agent is recorded for them. Their traffic is real but invisible to this dataset, which is why every number here is a floor, not a total.
- How often is this report updated?
- The underlying snapshot in agents/SEO-AEO-CRAWLERS.md is committed weekly by a scheduled GitHub Actions job; this page and its .md/.json twins are regenerated from that same file on every site build, so they always show the latest committed snapshot.