Access

Bots and llms.txt are hygiene


Abstract still: an L-path, an accent line, and three outlined bars.

robots.txt, llms.txt, sitemaps, and headers are hygiene. They tell crawlers what they may read. They are not a ticket into ChatGPT, Gemini, or anyone’s allocator.

If a bot cannot reach the page, no amount of FAQ spam will make the model cite you. File access first. Then extractable facts. Then measure again.

llms.txt is a public map — a courtesy document. Some retrievers look. Some ignore it. Selling it as a ranking lever is how the category lies. We inventory what you ship and label discrepancies: a sitemap that disagrees with robots, a header that blocks the bot you invited, HTML that is not the HTML a person sees.

Same public HTML for people and crawlers. No cloaking. No user-agent fork. That is product law, not a mood. Discover — submitting a URL, opening a path — is not a citation guarantee. It is the chance to be read.

Agent analytics, when we show it, comes from logs you already have. Training traffic, retrieval fetches, and indexing hits are different jobs. Mixing them into “AI traffic” is another fog word.

A token-light payload for agents, with human HTML unchanged, is a later idea in this stack. It is not cloaking. It is also not in the v1 offer. Do not buy a story about an invisible second site.

The honest sequence is boring, which is why it works: can the bot fetch, can a person quote the span, did the next monthly slice move. Hygiene, not folklore.

All ideas