# AI-crawler traffic report

> Monthly first-party dataset · latest snapshot 2026-09-05 · JSON twin: https://changegamer.ai/api/reports/ai-crawler-traffic.json

As of 2026-09-05, ChangeGamer (changegamer.ai) received 1,743 AI-crawler requests in the trailing 30 days, 1,090 of them user-initiated retrievals (ChatGPT-User + Claude-User) — 63% of AI-crawler traffic and 3% of all 59,321 worker-served requests recorded in that window.

## Key facts

- On 2026-09-05, AI crawlers made 1,743 requests to ChangeGamer in the trailing 30 days, +44 vs the 2026-08-29 snapshot, out of 59,321 total worker-served requests (3%).
- GPTBot (OpenAI's crawl-and-index bot) made 148 requests as of 2026-09-05, −5 vs 2026-08-29.
- ChatGPT-User (live, user-triggered ChatGPT browsing) made 1,065 requests as of 2026-09-05, −32 vs 2026-08-29 — the single largest source of AI-crawler traffic to the site.
- Anthropic's crawlers combined (ClaudeBot + Claude-User) made 137 requests as of 2026-09-05, −3 vs 2026-08-29.
- PerplexityBot made 356 requests as of 2026-09-05, +87 vs 2026-08-29.
- The most-fetched page in the 2026-09-05 snapshot was `/resources/synthetic-data-generation` with 157 AI-crawler hits in 30 days — see the full top-20 table below.

## AI crawlers by bot (latest vs previous snapshot)

| Bot | 2026-09-05 | Previous | Delta |
|---|---|---|---|
| AI total | 1,743 | 1,699 | +44 |
| GPTBot | 148 | 153 | −5 |
| OAI-SearchBot | 37 | 40 | −3 |
| ChatGPT-User | 1,065 | 1,097 | −32 |
| ClaudeBot | 112 | 115 | −3 |
| Claude-User | 25 | 25 | no change |
| PerplexityBot | 356 | 269 | +87 |
| Google-Extended | 0 | 0 | no change |
| Bytespider | 0 | 0 | no change |
| All worker hits | 59,321 | 56,146 | +3,175 |

## Top 20 pages AI crawlers fetch (rolling 30 days, snapshot 2026-09-05)

| Rank | Page | AI-crawler hits (30d) |
|---|---|---|
| 1 | [synthetic-data-generation](https://changegamer.ai/resources/synthetic-data-generation) | 157 |
| 2 | [evaluating-ai-agents](https://changegamer.ai/resources/evaluating-ai-agents) | 106 |
| 3 | [durable-execution-for-agents](https://changegamer.ai/resources/durable-execution-for-agents) | 86 |
| 4 | [web-data-for-agents](https://changegamer.ai/resources/web-data-for-agents) | 77 |
| 5 | [generative-ui-for-agents](https://changegamer.ai/resources/generative-ui-for-agents) | 66 |
| 6 | [open-weight-models-for-agents](https://changegamer.ai/resources/open-weight-models-for-agents) | 62 |
| 7 | [computer-use-browser-automation](https://changegamer.ai/resources/computer-use-browser-automation) | 56 |
| 8 | [essential-agent-publications](https://changegamer.ai/resources/essential-agent-publications) | 56 |
| 9 | [reliable-tool-calling](https://changegamer.ai/resources/reliable-tool-calling) | 49 |
| 10 | [streaming-for-agents](https://changegamer.ai/resources/streaming-for-agents) | 35 |
| 11 | [multi-agent-orchestration-patterns](https://changegamer.ai/resources/multi-agent-orchestration-patterns) | 31 |
| 12 | [document-extraction-for-agents](https://changegamer.ai/resources/document-extraction-for-agents) | 31 |
| 13 | [mcp-2026-spec-revision](https://changegamer.ai/resources/mcp-2026-spec-revision) | 30 |
| 14 | [llms-txt-explained](https://changegamer.ai/resources/llms-txt-explained) | 29 |
| 15 | [agent-delegation-chains](https://changegamer.ai/resources/agent-delegation-chains) | 28 |
| 16 | [agent-observability](https://changegamer.ai/resources/agent-observability) | 27 |
| 17 | [agent-memory-context](https://changegamer.ai/resources/agent-memory-context) | 25 |
| 18 | [content-signals-explained](https://changegamer.ai/resources/content-signals-explained) | 25 |
| 19 | [code-execution-sandboxing](https://changegamer.ai/resources/code-execution-sandboxing) | 25 |
| 20 | [agent-spend-controls](https://changegamer.ai/resources/agent-spend-controls) | 24 |

## Method

The worker that serves every ChangeGamer page logs each request to Cloudflare Analytics Engine, tagged by user-agent. A scheduled GitHub Actions job (.github/workflows/seo-crawler-stats.yml) queries that log weekly and commits the snapshot this report renders. Two limits matter: asset-served paths — /llms.txt, robots.txt, the sitemap — bypass the worker entirely and are deliberately NOT counted, so every number here is a floor on crawler interest, never a total; and user-agent strings are self-declared by the crawler, not independently verified, so a bot that spoofs or omits its user-agent will not appear correctly (or at all). Read this as retrieval demand, not citation — a crawl means a page was fetched, not that it was used or cited in an answer.

## Reuse

This dataset is free to quote, republish, or build on — with attribution to ChangeGamer (https://changegamer.ai/reports/ai-crawler-traffic) — no key or license required. Machine-readable twin: https://changegamer.ai/api/reports/ai-crawler-traffic.json.

## Frequently asked questions

**What counts as an "AI-crawler request" in this report?**

A request whose user-agent matches one of eight declared AI bots — GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, Bytespider — as logged by the ChangeGamer worker to Cloudflare Analytics Engine. It is a fetch, not a citation: see the Method section above.

**Why do the totals exclude /llms.txt and robots.txt?**

Those paths are served directly by Cloudflare as static assets and never pass through the logging worker, so no per-request user-agent is recorded for them. Their traffic is real but invisible to this dataset, which is why every number here is a floor, not a total.

**How often is this report updated?**

The underlying snapshot in agents/SEO-AEO-CRAWLERS.md is committed weekly by a scheduled GitHub Actions job; this page and its .md/.json twins are regenerated from that same file on every site build, so they always show the latest committed snapshot.

---

Source: https://github.com/beneb89/changegamer/blob/main/agents/SEO-AEO-CRAWLERS.md
Full history and corpus counts: https://changegamer.ai/stats · License: https://changegamer.ai/license.xml · HTML: https://changegamer.ai/reports/ai-crawler-traffic
Previous snapshot compared above: 2026-08-29.