Your documentation is bottom-of-funnel content, and most companies still fund it like a cost center they'd close tomorrow if the support queue allowed it. It gets a contractor, a backlog, and apologies. Meanwhile the buyer three weeks into evaluating you is reading it at eleven at night, working out whether the product does the specific thing they need, and they'll never fill in a form to ask.
That was true long before generative engines existed, and what changed is that a second reader showed up, reading the same pages for the same reason in roughly the same way. Retrieval doesn't read a website the way a person reads a brochure, front to back and in the order you chose. It works more like someone pulling one part off a shelf in a workshop, where the labelled shelf wins and the unsorted pile loses, however good the parts in that pile happen to be. Documentation is the labelled shelf. Most of the rest of a B2B website is the pile in the corner.
So what's that traffic actually worth, and which pages do we fix first?
Is small traffic bad traffic?
Not when it converts. Patrick Stox published Ahrefs' own analytics for a 30-day window in June 2025, and AI search sent 0.5% of their total traffic while producing 12.1% of their signups, a conversion rate 23 times higher than traditional organic search. Twelve percent of signups from half a percent of visits!
Should we rebuild a website around half a percent of its traffic? Well, of course not, and Stox himself states two caveats that are worth reading before anything gets moved. Visitors from AI search click links about 75% less than they do in organic search, and he isn't sure the pattern scales, quoting a colleague who expects these click rates are the highest they'll ever be. So the multiple will come down. The direction holds anyway, and it matches practice: this traffic is small in volume, late in the cycle, and it lands after the comparison work has happened somewhere we can't watch.
Which is why the number to manage is qualified traffic rather than raw traffic, tagged with UTMs and judged on the leads it converts. This reader turns up at the end of their research rather than at the start of it. So the pages worth optimizing are the end-of-funnel ones: pricing, comparisons, integration limits, security posture, and the documentation that answers what a sales call would have answered.
What makes a document trustworthy is what makes it retrievable
An engineer scanning a reference page wants one section, wants to confirm it's current, and wants to leave. They skim headings. They trust a definition that defines, they distrust a page that buries the specific inside the general, and they abandon anything that reads like it was written to fill a content calendar. Retrieval behaves in a recognizably similar way, because it works with pieces of your content rather than the whole polished narrative, and the pieces it can use are the ones that stand on their own.
So why do the two disciplines keep issuing the same instructions? Because they're solving the same problem: a reader who needs a precise answer and has no patience for the paragraph it's buried in. No fluff, high accuracy, strong structure, lean content, clear hierarchy.
| What a technical reader needs | What retrieval needs | The move for both |
|---|---|---|
| To land on one section | Passages that stand alone | H2s phrased as the question |
| Terms defined the same way everywhere | One canonical passage per concept | A definitions page everything links to |
| Proof the page is current | Freshness it can check | Last-updated dates, versioned pages |
| Values, not prose | Structure it can lift | Tables for limits, tiers, specs, versions |
| The next answer one click away | Links between related answers | Deliberate internal linking |
I've built that shape several times, and none of it was built for retrieval, because retrieval wasn't a consideration in most of those years. At Lightrun, a 250-page knowledge base from a blank slate in eight weeks. At Firebolt, 100+ pages standing before public launch, and at Lytx and Surfsight, a 300+ page developer portal. At Coro, a compliance hub covering more than 20 global regulatory frameworks, built as an always-on resource rather than a campaign. At Visual Layer I documented the entire product codebase, and it came back from the engineers who wrote that code with close to zero corrections. That's the only quality test for technical content that's ever meant much to me.
None of this is a new argument either. A deck I wrote in March 2021 had a slide titled "Doc use cases: the bottom line", and its first line read "leverage docs for additional SEO + POCs and decision-making". So were docs funded as a discovery channel back then? They weren't, and mostly they still aren't, even though they were already the layer that buyers evaluated us on.
So is llms.txt the fix?
Not on its own. A mapping file or an llms.txt is an internal planning tool that forces us to decide which page owns which question. That's genuinely useful, and it's mostly useful to us rather than to the machine, because the work that changes outcomes is duller.
- FAQs written the way a person asks. Somebody types "does it work behind a proxy", not "network configuration considerations".
- The follow-up question, answered underneath it. Every answer produces a second one.
- Target terms inside the H2s. Structure carries as much weight as keyword presence.
- Internal links between the answers. Cheap, unglamorous, and usually skipped.
- Natural writing instead of stuffing. A relevant subject contains the right terms anyway.
One warning belongs in bold here: don't damage your Google rankings while you optimize for answer engines. SEO isn't over. It has been declared dead several times before, and every time the answer came back the same, which is that it needs continuous adjustment rather than a funeral.
How do we know any of it is working?
Partly through tooling, carefully: on a competitive engagement for a B2B SaaS client we mapped their visibility against five competitors and evaluated five AI visibility and brand-monitoring tools along the way. The tools disagree with each other, sometimes badly, which is what you'd expect from a category this young. So we treated their numbers as a baseline to move rather than a score to win, and put the weight on GA4 and UTM-tagged traffic, where conversions are countable and arguments are shorter.
The limit is worth stating plainly, because restructuring your documentation won't rescue positioning that was never clear in the first place. It won't make a model recommend you over a genuinely better product, and it can't repair a claim your buyers already doubt. What it does is make you findable, quotable, and easy for a skeptical reader to verify. Whether the reader likes what they verify is a separate piece of work, and it happens well upstream of anything in this post.
Key takeaways
- Docs answer evaluation questions, which makes them bottom-of-funnel content.
- Qualified traffic is the number that matters, so tag it with UTMs and count leads.
- Technical writing and answer-engine optimization ask for the same things: accuracy, hierarchy, definitions, tables, links.
- llms.txt is planning. FAQs, follow-up questions, headings, and internal linking are the work.
The Snyk post on PCI and open source security requirements was built this way in 2019, long before anyone said GEO, and it holds up because the structure did the work rather than the adjectives. If your documentation is accurate and nobody can find anything in it, that's fixable, and it's cheaper than the campaign you were about to run instead. Bring the sitemap to a free 30-minute call, and we'll pick the ten pages your buyers evaluate you on and decide which question each one owns.
