How to Build an Agent-Friendly Website

Vadim Kravcenko
Vadim Kravcenko
May 04, 2026 · 18 min read

TL;DR: AI agents read your site through three different lenses (screenshots, raw DOM, and the accessibility tree), and most sites are accidentally hostile to all three. The fixes overlap with accessibility and Core Web Vitals work you should already be doing. This is a priority order, a 5-minute audit, and a measurement loop you can run today.

The moment GPT couldn't see our hover states

I was building our agent-ready tool a few weeks ago. The whole point was to feed real customer sites to a Playwright-based agent and watch how it tries to use them. So I pointed it at our own homepage, the one I'd shipped a thousand times, and asked it to find pricing.

It clicked the wrong CTA. Then it tried to click a button that wasn't a button (a styled <div> we'd put together in a hurry). Then it gave up and asked me where pricing was.

The thing is, I knew our site had hover states that revealed the secondary nav. A human moves their cursor, the menu appears, problem solved. The agent was working from a single screenshot of the resting state. There was no cursor to move. The hover-only nav literally did not exist for it.

That was the day I stopped thinking about AI agents as smarter crawlers and started thinking of them as a new kind of visitor with strange limitations. (Side note: I'd assumed for months that they worked like Googlebot with extra steps. I was wrong about that for the first six months of last year.) What follows is what I've learned since, mostly from breaking things and watching the agents react.

The new visitor on your site

An AI agent landing on your homepage isn't crawling for a search index. It's trying to do a job a human asked it to do. Book the flight. Compare the plans. Add the thing to the cart. It has minutes, not hours. And every wrong click costs the user money in inference tokens.

That changes the design problem. A search bot can fail silently and you'll never know. An agent fails out loud, in front of a paying user, and the next time that user wants the same thing they'll ask the agent to skip your site. AI search engines are starting to behave the same way. If their crawler can't extract a confident answer from your page, the answer goes to whoever can be parsed cleanly. The clicks you used to get from a SERP get replaced by citations from an AI summary, and citations are won by readability, not by ad budget.

For a longer take on the AI-as-discovery-channel side of this, the silo pillar at ask engine optimization covers the search side directly. This article is about the other half: what happens when an agent actually tries to use your site.

The three ways an AI agent reads your site

This is the part I wish someone had explained to me a year ago. There isn't one way agents see web pages. There are three, and the major vendors mix them differently.

"Agents can view your website in 3 primary ways: screenshots, raw HTML, and the accessibility tree. The accessibility tree is a browser-native API [that] distills the DOM into what's most important: roles, names, and states of interactive elements. For an AI agent, it functions as a high-fidelity map that ignores the visual 'noise' of CSS to focus on pure utility."

— Kasper Kulikowski & Omkar More, web.dev: Build agent-friendly websites

Here's how that breaks down by vendor, as of early 2026. Anthropic's computer-use is screenshot-only. Claude sees a 1024×768 image, reasons about pixel coordinates, and tells the harness to move the mouse. There is no DOM input on Anthropic's side. OpenAI's Operator (CUA) layers screenshots with DOM and accessibility-tree data, though OpenAI's own launch copy emphasizes the screenshot loop and the DOM layering shows up mostly in third-party reverse-engineering. Google's Project Mariner reads both pixels and the underlying DOM. Playwright-based frameworks like browser-use prefer the accessibility tree first and fall back to screenshots when that fails.

Three-pane diagram showing the same product page rendered as a screenshot, a raw DOM tree, and an accessibility tree, with the accessibility tree highlighting only roles, names, and states of interactive elements
The same page, rendered three ways. Most sites optimize for the first pane and ignore the third.

The economics here matter more than the architecture. An accessibility-tree snapshot of a typical page weighs 2 to 5 KB. A screenshot of the same page is 100 KB or more. That's roughly a 20× to 50× per-step cost difference in inference tokens. Agents that can read the accessibility tree have strong incentive to do so. Sites that produce a clean accessibility tree get used. Sites that don't get expensive screenshots, more failed actions, and eventually skipped.

You can see your own site's accessibility tree right now. Open Chrome DevTools, go to the Accessibility pane, check "Enable full accessibility tree." That's roughly what an agent like browser-use will pull. Chrome hides ignored nodes and unnamed generic nodes by default, which is useful, because those are exactly the elements agents struggle with.

Chrome DevTools Accessibility pane showing a real e-commerce page's accessibility tree, with role names and accessible names visible and unlabeled generic nodes hidden
Chrome DevTools' accessibility tree view. This is the closest a developer can get to seeing what an a11y-tree-first agent sees.

Why this became an SEO problem

Citations are the new clicks, at least for some queries. Per Profound's cross-platform breakdown, Wikipedia accounts for 47.9% of ChatGPT's top-10 source share, Reddit takes 21% of Google AI Overviews and 46.7% of Perplexity. The patterns shift constantly (per Leapd's tracking, ChatGPT's Reddit share collapsed from about 60% to 10% in six weeks last winter after a single Google parameter change), but the direction holds: AI engines reward content they can confidently parse and discard the rest.

"Some LLMs, like ChatGPT and Claude, do not execute JavaScript (unlike Gemini), making Server-Side Rendering (SSR) critical for important content to appear in their answers."

— Aleyda Solis, AI Search: Where Are We & What Can We Do About It?

That single sentence is more concrete than most of the AI-SEO advice I've read. If your important content is rendered client-side, ChatGPT and Claude don't see it. Gemini does. Whatever you think about that, it's a falsifiable engineering requirement, not vibes. Our piece on the AI crawler playbook goes deeper on which crawlers run JavaScript and which don't.

I should be honest about what we don't know. There is no public study that directly correlates accessibility scores with AI-citation rates. The mechanism is defensible (the accessibility tree is the same data structure agents and screen readers consume), but the empirical link hasn't been proven. The strongest adjacent data is from accessibility.works' summary of a 10,000-site study finding WCAG-compliant sites pulled 23% more organic traffic and ranked for 27% more keywords than non-compliant peers. The methodology is under-documented (no specified WCAG conformance level, no peer review), so I treat that as suggestive, not load-bearing.

The 8 signals that make a site agent-readable

You'll see lots of "8 things" listicles in this space. The list is genuinely useful as an organizing structure; it stops being useful the moment you treat all eight as equally important. They are not. Here's the comparison.

Signal What an agent sees if you get it right What breaks if you get it wrong Effort
Native semantic elements <button>, <a>, <h1> appear in the a11y tree with implicit roles Styled <div> buttons disappear from the a11y tree entirely Low
Accessible names on inputs "input, type=email, label='Email address'" "input, type=text" (placeholder text alone does not create a name) Low
Visible interactive targets Buttons larger than ~8 sq px (web.dev's empirical floor) and ideally 24×24 CSS px (WCAG 2.5.8 AA) Vision models filter out small targets; mouse-coordinate agents miss them Low
Stable layout (low CLS) Click coordinates land on the intended element Lazy-loaded images shift the page; agent clicks the wrong button Medium
Server-rendered content ChatGPT and Claude see the full HTML on first fetch Hydrated-only content is invisible to non-JS-executing crawlers Medium-High
Heading hierarchy One <h1>, nested <h2>/<h3> with implicit heading roles Visual headings styled as <p> = no document outline for the agent Low
No ghost overlays / focus traps Click events reach the visible element Transparent layers swallow clicks; agent can't recover Medium
Predictable URLs and forms Agent can resume after a navigation; form fields self-describe Single-page-app routes that swallow history confuse multi-step tasks Medium
Side-by-side code comparison showing a div-soup checkout button on the left versus a semantic HTML button element on the right, with annotations showing what each looks like in the accessibility tree
Same visual button, two implementations. The one on the right is the only one an agent can find.

Two of these dominate. Native semantic elements and accessible names are the foundation. Get those wrong and the rest doesn't matter. Across the few hundred sites we've put through our agent-ready check since launching the tool, the most common single failure is exactly this: a CTA built as a styled <div>, which then disappears from the accessibility tree entirely. Roughly half the homepages we audit have at least one. The 8-square-pixel target floor I mentioned (from web.dev) is empirical, not a hard model-architecture limit; vision encoders for CLIP-style models actually use 14×14 or 16×16 pixel patches, which is a much larger floor. Treat it as a sane minimum rather than a magic number.

Audit your site in 5 minutes

This is the part I never see in other agent-readiness pieces. Most of them give you a checklist of 8 to 30 things and leave you to it. Here's what I actually do when I have 5 minutes between meetings.

Minute 1. Run our free agent-ready check. It feeds your homepage to a Playwright-based agent and reports back things like a styled <div role="button"> with no tabindex, a hidden input that traps focus, or a CTA the agent could see in the screenshot but couldn't reach in the accessibility tree. This is the tool Lida and I shipped after the GPT-can't-see-our-hover-states moment (we disagreed on timing — she wanted to wait for cleaner data; I shipped the rough version). It's free, it doesn't ask for an email.

Minute 2. Open your site in Chrome, hit DevTools, switch to the Accessibility pane, enable the full tree. Look for nodes with role "generic" and no name. Each one of those is a place where an agent has to guess.

Minute 3. Run the vibe-coded scanner against the same URL. It's a code-quality scan we built to catch the patterns that Cursor, Copilot, and Lovable tend to ship by default: <div onClick> buttons, missing form labels, headings styled as paragraphs. (I built this one because half the sites we audit are now AI-generated, and the AI generators have not yet learned semantic HTML.)

Minute 4. Run a Lighthouse audit. Note the CLS score. Anything above 0.1 is borderline; above 0.25 is breaking agent clicks regularly. Note the accessibility score too, since the same audit catches missing labels and contrast issues that affect both screen readers and agents.

Minute 5. Pick the worst single problem from those four reports and write it down. Tomorrow, fix that one thing. Don't try to fix the list.

Screenshot of the SEOJuice agent-ready tool showing a results page with parseability score, list of detected interactive elements, and flagged issues including hover-only navigation and styled div buttons
Output from our agent-ready tool on a real customer site. The "hover-only navigation" flag is the one that ate me on our own homepage.

Five minutes is the actual budget most marketing leads have. If you have more, great, dig deeper. If you don't, this loop is enough to surface the single biggest problem on your site, which is usually all you need to know.

Anti-patterns that break agents

Three patterns produce most of the failures I see. The first is the styled <div> as a button. It looks like a button, it has a hover state, it triggers on click, but it's not a button to the accessibility tree.

"Don't use a <div>, <span>, or some other non-interactive element. If you cannot tab to it with a keyboard, it's probably a terrible idea to use."

— Adrian Roselli, Links, Buttons, Submits, and Divs, Oh Hell

Roselli wrote that in 2016, a decade before anyone said the words "AI agent" in this context. The keyboard-tab heuristic he describes maps perfectly onto agent-readability now: if a keyboard can't reach it, the accessibility tree won't expose it, and a Playwright-based agent can't see it. The bill on a decade of styled-div buttons is coming due.

The second pattern is layout instability. We saw this badly during our .io to .com migration. Some new lazy-loaded hero images on the .com side were shifting the "Get started" button down 40 pixels about 800ms after the page rendered. Human users barely noticed. Playwright-based agents identified the button at the original position, waited a beat, and clicked the empty space where the button used to be. This isn't a hypothetical: Playwright issue #6238 documents lazy-load-induced layout shift causing the wrong element to be clicked, and it's still open. Lighthouse's CLS metric is the proxy I trust here. Anything above 0.1 is suspect.

Three frames showing the Add to Cart button in different positions on the same page as lazy-loaded images shift the layout, with the agent's predicted click position drifting away from the actual button location
Layout shift kills agent clicks. The agent picks coordinates from frame one and clicks them in frame three.

The third pattern is ghost overlays. Cookie banners that don't actually dismiss, GDPR popups with transparent backdrops left over after the modal closes, intercom widgets that hover at z-index 9999 over your CTA. Each of these intercepts clicks the agent meant for something visible underneath. Web.dev calls these out explicitly, and our vibe-coded scanner flags them.

Screenshot of the SEOJuice vibe-coded scanner showing a YES verdict with confidence 81 and category breakdown for builder fingerprints, code hygiene, content patterns, and SEO basics
The vibe-coded scanner result for one of the AI-generated sites we audit. Hits builder fingerprints + code hygiene the hardest.

Where accessibility, SEO, and AI agents converge

"These devices rely on the Accessibility Tree—the mechanism computers use to allow assistive technology to programmatically read and interpret user interface content—to function. By relying on well-supported, time-tested markup to describe our content, we are able to make our work versatile and robust."

— Eric Bailey, Truths about digital accessibility

Bailey wrote that in 2019, talking about screen readers and assistive tech generally. The bit I keep coming back to is "the mechanism computers use." He defined the accessibility tree as a programmatic interface for machines, not specifically for humans with disabilities. The arrival of AI agents collapses what used to be three jobs (accessibility, SEO crawlability, agent readability) into one underlying job: produce a clean machine-readable description of what your interface does.

You'll find a longer treatment of the SEO half of this convergence in our accessibility-for-SEO pillar. The short version is that the same fixes (semantic HTML, accessible names, stable layout) move three needles at once.

Priority order — what to fix first

If you do nothing else from this article, do these in this order.

  1. Replace <div> and <span> buttons with <button>. Highest leverage, lowest cost. This single fix moves you from "invisible" to "visible" in a11y-tree-first agents like browser-use, Mariner, and Playwright-driven crawlers.
  2. Add <label for> to every form input. Placeholder text is not an accessible name. If your contact form's email field has no label, agents reading the a11y tree see an unnamed text input, full stop.
  3. Server-render content that needs to be read by ChatGPT and Claude. If your pricing page hydrates client-side, the JavaScript-free crawlers literally cannot see your prices. Either SSR or accept that ChatGPT and Claude won't cite that page.
  4. Get CLS under 0.1. Reserve space for lazy-loaded images. Set explicit dimensions. This is the same fix Core Web Vitals already wants from you.
  5. Bump touch targets to 24×24 CSS pixels minimum (WCAG 2.5.8 AA). 44×44 if you want to clear Apple HIG. The 8-square-pixel floor from web.dev is the absolute minimum below which screenshot-based agents lose targets entirely.
  6. Audit ghost overlays. Cookie banners, intercom widgets, modals that don't actually close.
  7. Heading hierarchy. One <h1> per page. <h2> for sections. Don't style a <p> to look like a heading.
  8. llms.txt and MCP tools. These are speculative bets on standards that haven't shipped broadly yet. Do them if 1 through 7 are clean.
Pyramid diagram showing the eight agent-readability fixes from highest leverage at the base (semantic elements) to most speculative at the top (WebMCP and llms.txt), with the foundation labeled 'fix this first' and the apex labeled 'fix this last'
Priority pyramid. Base fixes move three signals at once; the top fixes are speculative bets.

I've watched teams spend two weeks publishing an llms.txt file before fixing their styled-div checkout button. That ordering is backwards.

Measuring whether your fixes worked

You shipped the fixes. Now what.

For layout stability, the Lighthouse audit is the cheapest reliable measurement. Run it against the same URL before and after, watch CLS, accessibility score, and INP move together. (When we cleaned up the <div> buttons on our own marketing pages, our Lighthouse accessibility score jumped 14 points and CLS dropped from 0.18 to 0.04. That's not a controlled experiment, but it's the kind of change you can produce in an afternoon.)

For agent readability specifically, re-run the agent-ready check. The tool reports a parseability score plus a list of CTAs the agent could and couldn't reach. The metric I watch is "CTAs successfully identified": if it's not 100%, the agent is still guessing somewhere.

For AI citations (the SEO outcome you actually care about), use our backlink checker to track mentions in AI summaries. Citations from ChatGPT, Perplexity, and AI Overviews look like backlinks in shape, just from new sources. If your fixes are working, you'll see those citations appear over weeks, not days. Our AI visibility audit methodology walks through the slow-loop measurement side.

Screenshot of a Lighthouse report with the Cumulative Layout Shift metric highlighted, showing a before-and-after comparison of 0.18 dropping to 0.04 after fixing lazy-loaded image dimensions
CLS went from 0.18 to 0.04 after we set explicit image dimensions. The same fix raised the accessibility score by 14 points.

I want to flag the Wil Reynolds stat that gets passed around via Orbit Media's writeup, "0.8% of traffic, 10% of leads," because everyone in this conversation eventually hears it. It's a real, citable, useful data point. It's also one company's number from a specific window, and Reynolds himself has been clear it's not a universal multiplier. I treat it as evidence that AI-driven traffic punches above its weight, not as a multiplier to forecast against. If your AI traffic is converting at 5x your organic average, you have something. If it isn't, the multiplier doesn't apply to you.

What's next: WebMCP and the standards layer

WebMCP is the spec to watch. It lets a website register JavaScript tools that AI agents can call directly, turning the page itself into a Model Context Protocol server. Chrome 146 Canary shipped a flag-gated preview. Edge 147 added support in March 2026. The W3C Community Group draft was published 2026-04-23.

I like it. I'm also realistic about timing. The spec itself disclaims: "It is not a W3C Standard nor is it on the W3C Standards Track." No major LLM vendor has documented consuming WebMCP tools yet. Cloudflare's own crawl data found fewer than 15 sites globally implementing MCP Server Cards. This is a 2-to-5-year story at best.

"The web browser is designed for human users and developers, not for agentic AI systems. Most web content is structured for visual consumption rather than programmatic access, with dynamic elements, complex layouts, and interactive components that resist simple parsing."

— Cobus Greyling, The Future of Web Browsing AI Agents

Greyling's argument is that the agent-friendly checklist (this article, basically) is a workaround for a deeper architectural mismatch that proper standards like WebMCP are trying to resolve. He might be right. I don't know if WebMCP will be the standard that wins, or if llms.txt and Microsoft's NLWeb will, or if something we haven't seen yet absorbs all three. What I do know is that the priority pyramid above is the work to do now, while the standards layer settles. Our take on where this is heading is in agentic SEO workflows.

Frequently asked questions

What is an agent-friendly website?

A website that can be read and used reliably by AI agents (Claude computer-use, OpenAI Operator, Project Mariner, Playwright-based crawlers) in addition to human visitors. The core requirement is that interactive elements expose themselves clearly through the browser's accessibility tree, the layout doesn't shift while the agent is working, and important content is server-rendered so non-JavaScript-executing crawlers can read it.

How do AI agents read a website differently from humans?

Agents use one or more of three input modalities: screenshots, raw HTML, or the accessibility tree. Screenshot-only agents (Anthropic's computer-use) reason about pixel coordinates and miss anything below an empirical 8-square-pixel target. DOM-aware agents (Project Mariner, Operator) layer in element data. Accessibility-tree-first agents (browser-use, most Playwright-driven crawlers) get a clean structural map but miss anything not exposed semantically.

Do AI agents use schema markup?

Bing has confirmed schema helps Copilot understand content. Google has acknowledged structured data gives an advantage in search results. Direct evidence that schema lifts AI-citation rates is mixed: one study found no correlation, another found attribute-rich schema produced a 61.7% citation rate. Treat it as helpful for the AI-search side, not as a guaranteed multiplier.

What is llms.txt and do I need one?

It's a proposed file format (similar in spirit to robots.txt) that tells AI crawlers where to find your important content in a clean format. Over 844,000 sites had published one as of October 2025. Adoption is high but consumption is low: Google's John Mueller and Gary Illyes both said publicly that no major AI system uses it as an operational signal. Worth publishing if you have 10 spare minutes; not worth prioritizing over fixing your styled-div buttons.

How do I check if my website is agent-ready?

The fastest path is a 5-minute audit: run our agent-ready check, open Chrome DevTools' Accessibility pane, run a Lighthouse report, and run a code-quality scan with our vibe-coded tool. Together those four surface the largest issue on your site. Fix the worst one before adding the next.

TL;DR and next steps

AI agents read your site through screenshots, the DOM, and the accessibility tree, in different mixes by vendor. Most of the fixes are accessibility fixes you should already be doing: native semantic elements, accessible names, stable layout, server-rendered content. The standards layer (WebMCP, llms.txt, NLWeb) is real but slow; the unsexy fixes are urgent.

Run our free agent-readiness check to see how readable your site is to ChatGPT, Claude, and Perplexity. It'll tell you which CTAs the agent missed, which buttons disappeared from the accessibility tree, and which layout shifts are eating clicks. It's the tool I wish I'd had the day GPT couldn't see our hover states.

SEOJuice
Stay visible everywhere
Get discovered across Google and AI platforms with research-based optimizations.
Works with any CMS
Automated Internal Links
On-Page SEO Optimizations
Get Started Free

no credit card required