fix(sitemap): generate sitemap.xml under Yarn and gate it in CI - #126
Merged
Conversation
next-sitemap ran from the `postbuild` npm lifecycle hook, but CI and Vercel both build with Yarn 4, which does not run pre/post scripts for user-defined commands. The sitemap was therefore never generated: /sitemap.xml fell through to the Next.js HTML 404 page, which every XML client reports as a parse error. public/sitemap.xml is gitignored, so there was no committed fallback, while the committed robots.txt kept advertising the missing file. `prebuild` was dead for the same reason, leaving llms.txt frozen six posts behind. Chain both generators explicitly at the ends of `build` so they run under any package manager, and add `yarn validate:sitemap` as a CI gate immediately after the build step: the sitemap must exist, must not be an HTML error page, must parse as XML with no unescaped entities, and must carry absolute canonical URLs whose origin matches the Sitemap: line in robots.txt. Also resolve site-relative frontmatter images against siteUrl, so the four portfolio entries using /images/*.png stop being dropped from <image:loc>.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
https://www.ahnafnafee.dev/sitemap.xmlreturns HTTP 404. Browsers report it as an XML parsing error because they are parsing the Next.js HTML 404 page.next-sitemapwas wired to thepostbuildnpm lifecycle hook. CI and Vercel both build with Yarn 4 (packageManager: yarn@4.14.1, corepack invercel.json), and Yarn does not runpre/postscripts for user-defined commands:So the sitemap was never generated.
public/sitemap.xmlis gitignored (**/public/*.xml), so there was no committed fallback, and the committedrobots.txtkept pointing at the missing file.prebuildwas dead for the same reason, leavingpublic/llms.txtsix blog posts behind.Summary
generate:llmsandgenerate:sitemapexplicitly insidebuildinstead of relying on lifecycle hooks, so they run under any package manager.scripts/validate-sitemap.jsand aValidate sitemapCI step after the build. It runs on the same package manager Vercel uses, so it fails on exactly this regression.siteUrl, recovering 4 portfolio entries that were being dropped from<image:loc>(27 -> 31 images).public/llms.txt.What the gate checks
File exists, is not an HTML error page, has no unescaped
&, parses as well-formed XML, root is<urlset>with at least one<url>, every<loc>and<image:loc>is an absolute https URL, no duplicate locs, no localhost, and theSitemap:origin inrobots.txtmatches.Test plan
yarn buildfrom a deletedpublic/sitemap.xmlregenerates ityarn validate:sitemappasses on real output (36 URLs, 31 images)&, malformed XML, localhost/non-https loc, duplicate loc, empty urlset, robots.txt origin mismatchyarn type-check,yarn lint,yarn test(66 passing)/sitemap.xmlreturns 200 XML (the current 404 is cached with a 24hCache-Control)