From ca8f8f9c4412ad44cf581d6e592f98b5d0ab5f78 Mon Sep 17 00:00:00 2001 From: Ahnaf An Nafee Date: Tue, 28 Jul 2026 11:33:35 -0400 Subject: [PATCH] feat(seo): add llms-full.txt with the full text of every document llms.txt is only an index, so an assistant that wants the actual content has to crawl each URL or hit the per-post markdown API. Emit a companion llms-full.txt alongside it: every blog post, research entry, and portfolio project inlined verbatim under an H1 and a URL line, following the llmstxt.org convention. MDX bodies carry no H1 of their own (the page template renders the frontmatter title), so one H1 per document keeps the concatenated heading hierarchy intact. llms.txt now points at it from both Discovery and How to Use This Site. Both files are tracked and regenerated by the build, so the committed copies are the fallback and the build output is what ships. CI runs `git diff --exit-code` on the pair after the build, which fails a content edit that was committed without a regenerate. --- .github/workflows/ci.yml | 6 + CLAUDE.md | 11 +- public/llms-full.txt | 2886 ++++++++++++++++++++++++++++++++++ public/llms.txt | 2 + scripts/generate-llms-txt.js | 52 +- 5 files changed, 2951 insertions(+), 6 deletions(-) create mode 100644 public/llms-full.txt diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7fc884a..c8f06c5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -67,6 +67,12 @@ jobs: - name: Validate sitemap run: yarn validate:sitemap + # llms.txt and llms-full.txt are regenerated by the build but are also + # tracked, so a stale committed copy is what ships if generation is ever + # skipped. Fail when the build output differs from what is in the tree. + - name: Check generated llms files are current + run: git diff --exit-code -- public/llms.txt public/llms-full.txt + - name: Validate JSON-LD on built output run: yarn validate:json-ld diff --git a/CLAUDE.md b/CLAUDE.md index 3b6031b..514bca3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,7 +28,7 @@ yarn validate:json-ld # walk built HTML, verify every