Agent Traffic

Agent traffic dashboard

The Agent Traffic dashboard provides a complete view of AI agent visits to your site. It shows which agents are visiting, what pages they access, how often they return, and what type of response they receive (optimized or passthrough).

This data is collected automatically through your server-side integration. No additional configuration is required beyond the initial setup.

Metrics

The dashboard tracks the following core metrics:

Total agent visits

The total number of requests from identified AI agents within the selected time period. Each individual HTTP request counts as one visit. Duplicate requests from the same agent to the same URL within a 1-second window are deduplicated.

Visits by platform

Agent visits broken down by source platform:

• ChatGPT / GPTBot — Requests from OpenAI’s crawler.
• Claude / ClaudeBot — Requests from Anthropic’s crawler.
• Perplexity / PerplexityBot — Requests from Perplexity’s crawler.
• Gemini / Google-Extended — Requests from Google’s AI crawler.
• Copilot / BingBot — Requests from Microsoft’s AI-enabled crawler.
• Rufus / AmazonBot — Requests from Amazon’s shopping agent crawler.
• Other — Agents identified by behavioral patterns but not matching a known User-Agent.

Visits by page

A ranked list of your most-visited pages by agent traffic. Shows total agent visits per URL, with platform breakdown for each page.

Response types served

For each agent visit, the dashboard indicates what response was delivered:

• Optimized — The request was intercepted by Inception Agents and served enhanced content (enriched structured data, optimized variants, llms.txt).
• Passthrough — The request was identified as an agent but served your original unmodified content. This occurs for pages not yet optimized or when the integration is in monitor-only mode.

Agent identification

Agents are classified through multiple detection methods:

User-Agent string matching

The primary identification method. Known agent User-Agent strings are matched against an updated registry:

Behavioral patterns

Some agents do not identify themselves via User-Agent. The platform uses behavioral signals to detect likely agent traffic:

• Request frequency and timing patterns.
• Navigation patterns (systematic vs. human-like).
• Header fingerprinting (accept headers, encoding preferences).
• IP range correlation with known agent infrastructure.

Behaviorally identified agents are labeled as “Likely Agent” in the dashboard with a confidence score.

Request patterns

Certain request patterns are strong agent indicators:

• Requests for /llms.txt or /llms-full.txt.
• Requests for /.well-known/agent.json.
• Requests with agent-specific accept headers.
• Systematic crawling of product pages or structured data endpoints.

Traffic sources

Agent traffic is categorized by purpose:

• Crawlers — Agents indexing your content for their knowledge base (GPTBot, ClaudeBot, PerplexityBot). These are the most common agent visitors.
• Shopping agents — Agents evaluating products for purchase recommendations (Rufus, Copilot Shopping). These typically access product pages and pricing.
• Research agents — Agents gathering information for user queries in real-time. These often access specific pages based on current user questions.
• Code agents — Agents accessing documentation, API references, and technical content. These typically focus on developer-facing pages.

Real-time vs. historical

Real-time view

The live agent traffic view shows agent requests as they arrive:

• Streaming request log with agent identity, URL, response type, and timestamp.
• Auto-updating counters for active agent sessions.
• Highlighted entries for new or unusual agent activity.

Access the real-time view from Analytics > Agent Traffic > Live.

Historical view

Time-series charts showing agent traffic over selectable time periods:

• Hourly, daily, weekly, and monthly aggregations.
• Platform-level trend lines.
• Page-level traffic history.
• Response type distribution over time.

Filtering

Filter agent traffic data by:

• Platform — Show traffic from a specific agent platform only.
• Page — Show traffic to a specific URL or URL pattern.
• Response type — Show only optimized or only passthrough requests.
• Date range — Custom date range selection with preset options (today, last 7 days, last 30 days, last 90 days).

Filters can be combined. All dashboard views respect the active filter set.

Export

Download agent traffic data for custom analysis:

• CSV export — Full traffic log with all fields: timestamp, agent, URL, response type, status code, response time.
• Date range selection — Export data for any available time period.
• Filtered export — Active filters are applied to the export.

Export is available from Analytics > Agent Traffic > Export.

Browser beacon data

In addition to server-side agent detection, the client-side browser beacon provides referral attribution data:

• document.referrer matching — When a human user arrives on your site from an AI platform (e.g., clicking a link in a ChatGPT response), the beacon captures the referral source.
• Correlation — Beacon data is correlated with server-side agent traffic to build a complete picture: agent crawl leads to recommendation, recommendation leads to human visit.

The beacon is included automatically with server-side integrations. See Referrer Attribution for full details.