u/Good_Flight6250

Free WordPress plugin: Suspicious Traffic Viewer for masked and hard-to-spot traffic

Following up on my earlier post about AI crawlers and masked traffic, I turned the idea into a free WordPress plugin.

https://www.reddit.com/r/Wordpress/comments/1sxvtqz/wordpress_sites_why_allow_ai_crawlers_if_they/

LiteCache Suspicious Traffic Viewer (STV) is not a generic traffic viewer and not a realtime logger. It focuses on suspicious traffic that often looks normal, blends in, or only becomes questionable when observed over time.

This is also not some speculative reaction to the current AI crawler hype. The logic behind STV is based on about 10 years of practical validation. AI crawlers did not create the problem. They mainly made it harder to ignore.

Another reason behind STV is that conventional traffic analyzers like GA4 or Matomo may see a lot, but still not everything. Especially not the traffic that deliberately avoids the very mechanisms those analyzers rely on.

Masked requests are only part of the problem. A much bigger issue is simple direct HTTP request traffic that bypasses typical analytics tracking completely, while still generating a large number of requests and unnecessary server load. That is not theory for me. It is part of my daily work when 1 human-like request can be accompanied by 500 clearly not-human-like requests.

STV is db-less while logging, but it has a clear limitation: requests fully served by page cache or CDN cache remain invisible.

The plugin will be available in the WordPress Plugin Repository soon. Until then, I can share the readme.txt with anyone interested.

If you want to read it or give feedback before release, let me know.

STV Plugin readme.txt: https://www.litecache.dev/readme.txt

LiteCache Suspicious Traffic Viewer Makes The Invisible Visible!

First Preview:

https://preview.redd.it/dsxlmhgh150h1.png?width=1333&format=png&auto=webp&s=270dad46822818f27319f84a59fe16f97b105986

reddit.com
u/Good_Flight6250 — 5 days ago
▲ 83 r/SEOandBacklinks+2 crossposts

Search engines used to crawl websites and send traffic back. That was the deal.

On WordPress sites, almost every request can trigger PHP, plugins, and database queries - so even AI crawlers create real server load.

AI crawlers seem to break that deal: they consume content, generate answers from it, but send little to no referral traffic back.

For website owners, this means:

- server load still exists

- content is still used

- attribution is weak

- traffic may decline

So my question is simple:

If AI systems take content but don’t send meaningful traffic back, why should website owners allow them to crawl at all?

Is “AI visibility” actually worth anything without referrals?

Sources:
https://blog.cloudflare.com/crawlers-click-ai-bots-training/
https://radar.cloudflare.com/ai-insights

[Update]

FYI: Google is currently transforming from a search engine into an answer engine, and ads will soon be positioned within the Chatbot answers. This means Google is training its AI for free with website content and making money from it.

[Update]

According to CloudFlare, there are now website operators who are also blocking Google because Google does not provide information about which Google User-Agent is used to train Google AI.

https://blog.cloudflare.com/uk-google-ai-crawler-policy/

[Update]

For users who want to block ChatGPT crawlers. For several months now, OpenAI has no longer exclusively used a unique user agent, but instead the "signature-agent" header. Filter rules based on the user agent no longer work as a result. Filter rules may need to be adjusted.

https://help.openai.com/en/articles/11845367-chatgpt-agent-allowlisting

[Update]

Reddit itself is an interesting example here, because Reddit has licensing/API agreements with companies like Google and OpenAI for AI-related use of its content. So even this discussion exists on a platform that has already turned user-generated content into licensed AI input. Means, the more you comment the more Reddit benefits.

https://openai.com/index/openai-and-reddit-partnership/
https://blog.google/company-news/inside-google/company-announcements/expanded-reddit-partnership/

[Update]

If you don't like the topic of this post because you fear it could jeopardize your business, please don't punish me by downvoting. I'm just the messenger of bad news, and you don't punish messengers. Thank you!

[Update]

One angle that hasn’t really been discussed yet is the actual impact on server load.

Not in theory, but in how WordPress handles requests in practice.

AI crawlers don’t behave like normal users, but WordPress still processes every request the same way.

I’ll dig a bit deeper into this in a follow-up, because I think this part is often overlooked. So stay tuned what is coming next. :)

u/Good_Flight6250 — 11 days ago