llms.txt and the AI crawlers
Ask a model about a service or a person and it answers from whatever content it has read. When nothing clear turns up, it guesses, or it leaves you out entirely.
What llms.txt is
A plain text file at the root of your site that explains what the site is, written in direct language. It carries no HTML or navigation menus, and no advertising either.
The idea is the same one that turned robots.txt into a standard: a simple file that tells the machine what it needs to know.
What I actually put in it
One paragraph that identifies you in concrete terms, with the marketing language left out: who you are, where you are based, what you do, and what sets you apart.
A list of your pages, each described in a single line.
A numbered list of your work. Of everything in the file, this piece taught me the most: models read lists far more reliably than they read prose, since a numbered item is structured and extracts cleanly.
Contact details, spelled out in full.
The rule that makes it work
Specific numbers and facts get cited. Vague phrasing does not.
"Extensive experience in systems" earns no citations; it could describe anyone. "33 tables through 18 migrations, and 36 tests on every push" gets picked up because it is specific and unique.
The robots file too
Name the AI crawlers you allow, one by one: the crawlers run by the known model providers, and the ones behind model-backed search. Leave nothing to assumption.
And block what should be blocked: private client pages, signing pages, and any path shared by direct link.
What it does not do
It does not guarantee you a mention. What it does is make you understood by whoever comes reading; fame is a separate problem.
That distinction matters. When a model describes you with wrong or outdated information, the usual cause is that it found no clear source to draw on. A good llms.txt does more to correct what gets said about you than to amplify it.
A quick test
Open the file and ask one question: could a stranger who read it explain what you do in two sentences? If they could not, neither can the model.
Related reading
Tell me what you want to build. Your first 15 minutes of consulting are free.
Book a consultation