u/UptownOnion

I did a super detailed audit of our new landing page to see how AI agent ready it was. Scored 9/100 (it's two weeks old website so no feelings hurt).

So I spent an afternoon fixing all the issues on the audit report. The next day we got this spike in AI traffic and it continued for the days after.

I'm still monitoring whether these agent visits can bring us actual human users. But seeing the chart move that fast off a few hours of work is really promising so I wanted to share out checklist.

Here's exactly what I fixed, in order:

1. AI Accessibility
   • Publish a robots.txt that explicitly allowlists major AI bots (GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended, CCBot)
   • Add a Content-Signal directive and a sitemap reference inside robots.txt
   • Serve a sitemap.xml with lastmod on every URL
   • Publish llms.txt (short) and llms-full.txt (comprehensive), per llmstxt.org
   • Add an AGENTS.md to guide coding agents through your product
   • Render pages server-side, not as a client-only SPA (SPAs return blank HTML; agents read HTML, not JavaScript)
2. Brand Identity
   • Embed sitewide JSON-LD Organization + WebSite schema (name, url, logo, description)
   • Match the brand string exactly across <title>, og:title, and Organization name (inconsistencies split your brand authority across duplicates)
   • Ship full Open Graph + Twitter Card metadata
3. Content Readability
   • Write real semantic HTML sections with proper headings and body text (agents can only quote what they can parse)
   • Load web fonts with display: swap
4. Quotability
   • Embed FAQPage JSON-LD wherever you have Q&A content
   • Chunk content into structured blocks (easier for agents to lift quotable snippets)
5. Platform Fit
   • Combine your AI bot allowlist, Content-Signal, llms.txt, and FAQ schema to cover per-platform signals
6. Site Hygiene
   • Set <link rel="canonical"> on every page
   • Include meta description, keywords, and authors
   • Ship a full favicon set (favicon.ico, apple-icon, 192px, 512px)
   • Configure robots directives (index, follow, max-image-preview: large)

https://preview.redd.it/fkuxufjp152h1.png?width=1080&format=png&auto=webp&s=99568ca12f84379791e4c982345e4a6b168108b0

I did a super detailed audit of our new landing page to see how AI agent ready it was. Scored 9/100 (it's two weeks old website so no feelings hurt).

So I spent an afternoon fixing all the issues on the audit report. The next day we got this spike in AI traffic and it continued for the days after.

I'm still monitoring whether these agent visits can bring us actual human users. But seeing the chart move that fast off a few hours of work is really promising so I wanted to share out checklist.

Here's exactly what I fixed, in order:

1. AI Accessibility
   • Publish a robots.txt that explicitly allowlists major AI bots (GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended, CCBot)
   • Add a Content-Signal directive and a sitemap reference inside robots.txt
   • Serve a sitemap.xml with lastmod on every URL
   • Publish llms.txt (short) and llms-full.txt (comprehensive), per llmstxt.org
   • Add an AGENTS.md to guide coding agents through your product
   • Render pages server-side, not as a client-only SPA (SPAs return blank HTML; agents read HTML, not JavaScript)
2. Brand Identity
   • Embed sitewide JSON-LD Organization + WebSite schema (name, url, logo, description)
   • Match the brand string exactly across <title>, og:title, and Organization name (inconsistencies split your brand authority across duplicates)
   • Ship full Open Graph + Twitter Card metadata
3. Content Readability
   • Write real semantic HTML sections with proper headings and body text (agents can only quote what they can parse)
   • Load web fonts with display: swap
4. Quotability
   • Embed FAQPage JSON-LD wherever you have Q&A content
   • Chunk content into structured blocks (easier for agents to lift quotable snippets)
5. Platform Fit
   • Combine your AI bot allowlist, Content-Signal, llms.txt, and FAQ schema to cover per-platform signals
6. Site Hygiene
   • Set <link rel="canonical"> on every page
   • Include meta description, keywords, and authors
   • Ship a full favicon set (favicon.ico, apple-icon, 192px, 512px)
   • Configure robots directives (index, follow, max-image-preview: large)

If you want to check your website's AI readiness, try our audit tool at agentraffic.com

Lately this sub has had a lot of posts about web traffic dropping noticeably. The diagnosis usually go to algo updates, lost rankings or competitors. But AI traffic barely comes up as a possible cause and I wanted to understand why.

What seems off to me is that SEO professionals already know customers are shifting from search engines to LLMs. They also know GA4 can't track agents server-side, so traffic from LLMs is basically invisible. But tracking that traffic still doesn't seem to be something people do. Why is that?

I built a server-side tool to measure it: arrivl.ai. Below is a screenshot of the dashboard showing AI agent traffic on our own site.

https://preview.redd.it/lg2nx17o081h1.png?width=1513&format=png&auto=webp&s=edc011d1f9b7a7d384ebda0dd371042242d4a16b

Does this matter to anyone here? or is it still too early to care?

I see a lot of tools like Profound, Peec AI, and others building AI visibility to help brands get recommended by LLMs. What I don't see is tools focused on tracking AI traffic to your website.

Wouldn't that also be a critical part of measuring how effective your site is for AI visitors?

Since GA4 has no server-side logic to track AI agents reliably and yet people still use GA4 referral data to figure out whether AI is visiting their site. Why?

I thought this was the gap, so I built a tracking analytics for AI traffic. People sign up, get curious, then don't install. (I admit it's prob partly because we are still a small startup) but a bigger question I want to ask:

is AI traffic just not relevant to AEO work the way I'm assuming it is?

https://preview.redd.it/wdthc9bpir0h1.png?width=1518&format=png&auto=webp&s=356ac65a6a9cc93b3dffe45c4cc82896d791b0f8

AI agents are crawling websites constantly now, but barely any sites are built for them to read properly, and I mean barely any. I ran an audit on Nike and it scored only 45%. You can see the report yourself here.

Here's the thing, most builders have no idea this is happening. ChatGPT, Claude, Google AI Overviews, etc, they're all sending traffic, picking sites to recommend, citing sources. If your site isn't readable, you're basically invisible.

Drop your link below and I'll send back a report that checks 26 things across 6 areas for free:

• can AI bots actually access your site
• does your brand identity show up in machine-readable form
• is your content structured the way AI models prefer to quote
• readiness for ChatGPT, Claude, Perplexity, Google AI Overviews specifically
• site hygiene

It looks technical but it's really not. Most can be fixed in an afternoon and I'll tell you how in the report.

Built a tool to log AI crawler/agent visits and break them down by intent: training crawlers (GPTBot, ClaudeBot), search/indexing (OAI-SearchBot, Googlebot), agentic (agents acting on a user's behalf), and social previews. Also surfacing top crawled pages, agent type breakdown, and a live hit feed.

The design question I'm stuck on: should human traffic live in the same dashboard?

1. Separate: keeps the AI lens clean, but you lose the comparison
2. Together: you get AI:human ratio per page, which is probably the more honest measure of AI's share of attention

For people optimizing for AEO, what would you actually want to see? Is intent breakdown useful for prioritizing what to optimize, or is raw crawl frequency per page enough?

Also curious if agentic as its own category resonates, or if you'd lump it in with search.

https://preview.redd.it/dq9d3rzgetzg1.png?width=1610&format=png&auto=webp&s=85d723daf85012ea977776ab0fa01f560b7fba1a

Since GA4 doesn't catch AI crawler visits (GPTBot, ClaudeBot, PerplexityBot, etc.), I built a tool to track them. Currently surfacing: visits per agent, top pages crawled, agent intent breakdown and a live feed of hits.

The question I'm stuck on, should human traffic live in the same dashboard?

Since AI traffic is a different decision making points, mixing in human numbers can pollute the data. Curious what people here think, especially anyone who's built internal dashboards and had to make this kind of scope call before.

Also open to feedback on:

1. Anything that's confusing or missing?
2. Is agent intent a useful categorization?

https://preview.redd.it/382ixfbvctzg1.png?width=1610&format=png&auto=webp&s=13c192002011008938a26ed280c0f4ce4a8fac2f

I've been seeing a lot of posts here asking how to track traffic from AI tools since most analytics platforms like GA4 can't catch them. My team and I have been building a analytics to capture this. It logs visits from GPTBot, ClaudeBot, PerplexityBot, and others, then shows you which pages they're reading, how often, and which agents are showing up most.

Posting here since AI traffic measurement is still early and this sub has the sharpest takes on AEO. I would really value feedbacks on:

• Is the data shown actually useful, or missing the metric you'd act on?
• Anything that feels confusing?

Screenshot of the dashboard below. Try it at arrivl.ai (It's completely free)

https://preview.redd.it/j2tj8m0y1szg1.png?width=1610&format=png&auto=webp&s=ea8887cbcc4460b615f36192b38b130f4a9f6f2b

I've been seeing a lot of posts here asking how to track traffic from AI tools since most analytics platforms like GA4 can't catch them. My team and I have been building Arrivl to capture this. It logs visits from GPTBot, ClaudeBot, PerplexityBot, and others, then shows you which pages they're reading, how often, and which agents are showing up most.

Posting here since AI traffic measurement is still early and this sub has the sharpest takes on AEO. I would really value feedbacks on:

• Is the data shown actually useful, or missing the metric you'd act on?
• Anything that feels confusing?

Screenshot of the dashboard below. Try it at arrivl.ai (It's completely free)

https://preview.redd.it/aectj9e2a6zg1.png?width=1080&format=png&auto=webp&s=f1cbf87d8e6084295af15e31110299fe189b2323

I've been seeing a lot of posts here asking how to track traffic from AI tools since most analytics platforms like GA4 can't catch them. My team and I have been building Arrivl to capture this. It logs visits from GPTBot, ClaudeBot, PerplexityBot, and others, then shows you which pages they're reading, how often, and which agents are showing up most.

Posting here since AI traffic measurement is still early and this sub has the sharpest takes on AEO. I would really value feedbacks on:

• Is the data shown actually useful, or missing the metric you'd act on?
• Anything that feels confusing?

Screenshot of the dashboard below. (It's completely free)

https://preview.redd.it/t3mc75zp96zg1.png?width=1080&format=png&auto=webp&s=cffb7a9262ce413d50c9af89b8a2d3407ed19cd8

I've been seeing a lot of posts here asking how to track traffic from AI tools since most analytics platforms like GA4 can't catch them. My team and I have been building Arrivl to capture this. It logs visits from GPTBot, ClaudeBot, PerplexityBot, and others, then shows you which pages they're reading, how often, and which agents are showing up most.

Posting here since AI traffic measurement is still early and this sub has the sharpest takes on AEO. I would really value feedbacks on:

• Is the data shown actually useful, or missing the metric you'd act on?
• Anything that feels confusing?

Screenshot of the dashboard below. Try it at arrivl.ai (It's completely free)

https://preview.redd.it/21v55kor86zg1.png?width=1080&format=png&auto=webp&s=60ecaac445f7383525c809c05487f737d3a7b1de

I've been seeing a lot of posts here asking how to track traffic from AI tools since most analytics platforms like GA4 can't catch them. My team and I have been building Arrivl to capture this. It logs visits from GPTBot, ClaudeBot, PerplexityBot, and others, then shows you which pages they're reading, how often, and which agents are showing up most.

Posting here since AI traffic measurement is still early and this sub has the sharpest takes on AEO. I would really value feedbacks on:

• Is the data shown actually useful, or missing the metric you'd act on?
• Anything that feels confusing?

Screenshot of the dashboard below. Try it at arrivl.ai (It's completely free)

https://preview.redd.it/je5q61fz66zg1.png?width=1878&format=png&auto=webp&s=260bf81bf932cce296fafc574a87c52d451c8a2c