Skip to content

feat(seo): add llms-full.txt with the full text of every document - #127

Merged
ahnafnafee merged 1 commit into
mainfrom
feat/llms-full-txt
Jul 28, 2026
Merged

feat(seo): add llms-full.txt with the full text of every document#127
ahnafnafee merged 1 commit into
mainfrom
feat/llms-full-txt

Conversation

@ahnafnafee

Copy link
Copy Markdown
Owner

Summary

llms.txt is only an index, so an assistant that wants the actual content has to crawl each URL or hit the per-post markdown API. This adds a companion llms-full.txt following the llmstxt.org convention: every blog post, research entry, and portfolio project inlined verbatim under an H1 and a URL: line.

  • 26 documents, ~171 KB, generated by the same scripts/generate-llms-txt.js that already writes llms.txt.
  • MDX bodies carry no H1 of their own (the page template renders the frontmatter title), so one H1 per document keeps the concatenated heading hierarchy intact.
  • llms.txt now points at it from both ## Discovery and # How to Use This Site.
  • Both files are tracked and regenerated by the build, so the committed copies are the fallback and the build output is what ships.

CI

Adds a git diff --exit-code check on the pair after the build, so a content edit committed without a regenerate fails the run. This is the same class of silent drift that left llms.txt six posts stale.

Caveat on impact

llms-full.txt is an unratified community convention. No major provider has confirmed using it as a retrieval or ranking input, and Google has said publicly it does not use llms.txt. This is cheap insurance if adoption happens, not a measurable ranking change today.

Test plan

  • yarn generate:llms writes both files; 26 documents, 171 KB
  • yarn build regenerates both as part of the chained build
  • git diff --exit-code on the pair is clean after a build (the CI gate passes)
  • yarn lint, yarn validate:sitemap, yarn validate:json-ld
  • CI green
  • After deploy, /llms-full.txt returns 200 text/plain

llms.txt is only an index, so an assistant that wants the actual content has to
crawl each URL or hit the per-post markdown API. Emit a companion llms-full.txt
alongside it: every blog post, research entry, and portfolio project inlined
verbatim under an H1 and a URL line, following the llmstxt.org convention.

MDX bodies carry no H1 of their own (the page template renders the frontmatter
title), so one H1 per document keeps the concatenated heading hierarchy intact.
llms.txt now points at it from both Discovery and How to Use This Site.

Both files are tracked and regenerated by the build, so the committed copies are
the fallback and the build output is what ships. CI runs `git diff --exit-code`
on the pair after the build, which fails a content edit that was committed
without a regenerate.
@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
ahnafnafee-dev Ready Ready Preview, Comment Jul 28, 2026 3:41pm

@ahnafnafee
ahnafnafee merged commit 88ea833 into main Jul 28, 2026
5 of 6 checks passed
@ahnafnafee
ahnafnafee deleted the feat/llms-full-txt branch July 28, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant