For this post, I am stealing some knowledge from my incredibly smart colleague Ofir Haim. This feels like another one of those posts that exposes my own ignorance, but in the spirit of being vulnerable and hoping it helps someone else out – here we go. There’s a lot of stuff I can’t put into words until someone spoon-feeds it to me, and he recently passed over a document that broke out AI in a way that just makes sense. Here’s how he breaks it down:
- Actors – The Agent. An entity that runs an agentic loop. They reason and act. This is what we mean by Agent.
- Hosts – AI-native applications that embed an Actor. The Host does not act – the Actor inside it does. Very popular as closed-source AI solutions.
- Non-Agents – Crawlers and AI-related traffic with no autonomous behavior. No goal pursuit on the site. It may have AI layers inside, but it operates as a machine, not an entity.
Now he didn’t invent these terms, but we use “AI” in a lot of different contexts. Even more specifically, the words “AI agent” can mean something different depending on the context. If you’re already saying “duh” – I promise I have content that’s relevant to you later in this post, but I want to talk about this distinction for a minute. In this post, I am talking about a specific type of Actor.
Table of Contents
Agent Distinction
The first important distinction is that an AI crawler or retrieval fetch is not an agent (or an actor). The crawler is what indexes and the retrieval fetcher is what buzzes between sources to determine whether orange juice pairs well with toothpaste (it doesn’t). Where this stuff gets complicated is when we talk about “AI agent measurement.” Technically an AI agent DEPLOYS a crawler/fetcher when the actor is answering one of your questions. When you are looking at server-side data, you are probably measuring a fetcher INITIATED by an agent… but it’s not an agent itself. At the same time, you’re inferring an AI agent’s intent – but that agent lives somewhere else.
Now when we talk about physically measuring an agent, we break that up into 2 types of agents:
- Programmatic Agents
- GUI AI Agents
Programmatic agents do back-end MCP or API stuff in the background. They never physically browse a human-readable surface like a website… but they have some kind of goal (which makes them an actor/agent). What we’re going to talk about in the next section is GUI agents. These are the agents that browse the site like a person. They physically load a website (JavaScript and all) and try to achieve some kind of goal. This is the traffic you can measure in your traditional site analytics tool.
GUI Agent Measurement
Let’s talk about GUI Agents… because I don’t have enough time to even start with server-side stuff. At CHEQ, we’re still seeing a pretty low percentage of traffic leveraging these agents… like less than 1% of all traffic for most clients. If I had to guess, that number is going to steadily increase over the next few years (duh). There are a lot of reasons an AI vendor (like Google, Open AI, Anthropic, etc.) might want to encourage users to use GUI agents to complete tasks. Adoption of GUI agents drive dependency on the tool, increases data stored, and… most importantly… meets customers where they are today (their browser).
How to measure GUI agents
Currently, there isn’t an effective way to measure this traffic out-of-the-box with analytics tools. Atlas, Comet, Claude, Manus, and others use Chrome extensions to operate – so it’s not like we can depend on user agent (though Claude does have its own baked-in browser in the desktop app). It also means detection services that operate on the edge can’t pick them up. You have to supplement the data with something, which means you need a tool like CHEQ to classify that data. I wish this didn’t make me sound like a marketing shill, but what do you want me to say here? Do you have a better solution? Please leave a comment if you have a free method of comprehensively detecting these agents (NOT just using navigator.webdriver or UA). I am drawing a blank.
What we’re currently doing at CHEQ is passing the agent classification into the data layer so you can attach it to your server calls. Think of it like a Boolean – either it’s an agent or it’s not. What’s more interesting is the classification of this traffic is asynchronous since users can open the Claude extension and tell it to do something at any point. The way we’re solving for that is sending a message to the data layer when that happens. In Workspace (for Adobe users), it would look something like this:
We have a nice, neat, trending count of agents hitting our site over time. This lets us know how it’s growing (or not) and gives us a nice, raw count of this unique traffic.
Making decisions with this data
So you’re counting GUI agent traffic… now what? Print and frame it? Keep an eye on it? To what end? What decisions are we really making here? Maybe we want to know what they’re doing. We can look at an aggregate Path Exploration. If I’m being honest, looking at that pathing is less useful than looking at a list of pageviews. I guess I want to understand what the user or the agent is trying to do. Like at what point is someone deciding “I want this agent to do a thing on my behalf”? Similarly, at what point is the agent saying “I need a human to do something on MY behalf”?
Introducing Agent Delegation and Escalation Events
My goal as an analyst has always been to bring clients closer to a decision. That’s why, with the help of the talented folks at CHEQ, we built some new metrics: agent delegation and escalation. Similar to what I mentioned above, we want to be able to say whether a page is conducive to agent navigation, whether humans don’t want to bother with taking a task, or whether there’s friction along the way. I can figure this out better when I am able to measure the handoff event between agents and humans.
To answer that question, we built a mechanism to show exactly when that event happens. This involves asynchronous measurement, passing data into the data layer and transmitting a descriptive event back to CHEQ. That means you’ll understand which pages have to be escalated to humans and which pages humans prefer to delegate back to agents. While the volume is relatively low, as a business I want to get ahead of the curve and understand where I can streamline the human-agent workflow. The goal of this is to take us beyond simple counting while leveraging the unique detection from the CHEQ platform.
Final Thoughts
The future of what surface will be predominantly used is still unclear. There are so many unanswered questions:
- Will people want to watch agents browse a website and do stuff for them?
- Will they prefer to do everything via MCP?
- At what point are users comfortable relinquishing the feeling of control?
It’s like grocery delivery – do you trust someone picking out your meat and produce or would you prefer to get them yourself? It’s more complicated than that.
- Are brands even comfortable relinquishing ownership of the shopping surface?
Sure, I just want people to buy more of my stuff! However, when the prioritization, marketing, and decisioning lives exclusively within the conversational LLM surface – will brands be okay with that blind trust? Will they have a choice? The concept of brands abandoning their owned presence seems pretty silly. It could change shape, but there has to be some original content out there to mine and prioritize inside of LLMs. The most probable outcome is some kind of hybrid like WebMCP. Maybe that’s a lazy answer… or the easy one. We can’t optimize toward every outcome, but in the interim we’re seeing momentum toward GUI agent integration. It feels like the safest adoptable surface. Like I can walk away from it and let it do its thing or I can sit and watch it because I am a control freak. Checks both boxes.
If the direction is something like WebMCP, which makes the decisioning from the handoff measurement easy, action is easy. Agents have to escalate? Configure WebMCP so we aren’t optimizing for the agent experience at the expense of the human experience. Assuming this is the future, the best time to start collecting the data is yesterday. The second best is today.