Skip to content

docs: refresh the Environment Variables screenshots and correct the control names they contradicted - #795

Open
promptless[bot] wants to merge 2 commits into
mainfrom
promptless/refresh-env-vars-screenshots
Open

docs: refresh the Environment Variables screenshots and correct the control names they contradicted#795
promptless[bot] wants to merge 2 commits into
mainfrom
promptless/refresh-env-vars-screenshots

Conversation

@promptless

@promptless promptless Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Open this suggestion in Promptless to view citations and reasoning process

Requested by: Matt Link

Both dashboard screenshots on Manage environment variables had drifted away from the live product. They showed a Settings tab row that no longer exists (it began with "Agent Settings") and sidebar entries that have since been removed, so a reader comparing the page to their own dashboard saw two different products.

Both images are replaced with fresh captures from the live dashboard. Refreshing them exposed three places where the surrounding prose was wrong too, so those are corrected here rather than left to contradict the new images:

  • The add-a-variable flow ended on "Click Save," but there is no Save button in that flow — the submit control is Add Variable. Save appears only after you click Update on an existing row, so the update section now describes that flow instead. This was the most consequential error: it sent readers hunting for a button that isn't there.
  • The Secret control is a toggle, not a checkbox.
  • Settings now opens directly on Environment Variables, so the old "find the Environment Variables section" step is folded into step 1 and the step list drops from five items to four.

The update-and-delete section now names the Update and Delete buttons in the Actions column, matching what the refreshed screenshot shows, instead of referring to a "delete icon."

A Doc Detective spec is added alongside the page, asserting that the documented controls still exist. It is non-destructive — it fills fields and checks visibility and masking, and never submits the form or deletes a variable — so the next time this UI drifts, a test catches it instead of a reader.


Update after review — two changes, both in the spec only

@InlinePizza approved this at f38db6a. Since then the diff has grown by one commit, d890eee, touching only .doc-detective/tests/environment-variables-settings.spec.json. Nothing about the screenshot refresh or the control-name corrections changed — environment-variables.mdx is byte-identical to what was approved.

1. The spec description is reframed around the journey it protects

Matt's follow-up: the description was episodic — it narrated how the test came to be written rather than what it protects. It ran ~4,700 characters of authoring decisions, CI-history commit hashes, Doc Detective framework semantics, and a justification of every matcher choice. An engineer could not tell from it which product behavior was under test.

It now opens with the journey, in the vocabulary this repo already uses (docs/content_strategy/journeys/): cuj-screenshots, step "Authenticate to the app being captured" — this page is where a user stores the test-account credentials Promptless Capture logs in with. Then what breaks if the test fails, then what is and is not covered. The operational facts an engineer needs to actually run it (auth, data prerequisite, invocation, CI status) are kept, demoted to labeled bullets at the end. The same episodic voice is removed from the per-test and step descriptions so the file isn't internally inconsistent. Net: ~4,700 → ~1,980 characters.

2. Two assertions were wrong and are corrected — this is why the diff grew

While verifying the description against the live dashboard rather than against the PR text, two of the spec's six probes turned out to fail against a healthy product:

Probe Asserted Actual
Add Variable form shape secretToggle=present secretToggle=missing
Post-click state secretChecked=true secretChecked=missing

Cause: the Secret control is a CSS-styled <span> toggle. Its <label> contains no input[type=checkbox], no [role=switch] or [role=checkbox], and aria-checked is null — the only checkbox on the page belongs to an unrelated sidebar switch. So the spec's claim that Secret is "a checkbox inside the label" was wrong, and it was the sentence that generated both broken assertions. It also contradicted this PR's own prose, which correctly calls Secret a toggle.

Both now assert the masking the docs actually promise — the Value input flipping from text to password when Secret is on. That is non-vacuous: with the toggle click removed it returns valueType=text and fails.

The toggle's visual on-state is deliberately not asserted. Its only machine-readable signal is computed CSS (track colour, knob offset), which the docs never promise and which any theme change would break. The Secret label text remains asserted, so the rename detection this spec exists for is intact.

Verification

Run against the live dashboard, signed in with the test account:

  • 6 of 6 probes pass, up from 4 of 6 before this commit.
  • The four previously passing probes are byte-identical to the approved commit — scripts and expected output alike. Exactly four non-description changes in the diff, all four listed in the table above.
  • The spec is still non-destructive: no step submits the form, clicks Add Variable / Save / Delete, or types $ENTER$. After all verification runs the table still held its original three variables and the inert sentinel never reached it.

One thing unchanged on purpose: the spec is still not wired into .doc-detective.ci.json. Clerk-dependent flows were pulled from CI earlier as unstable, so that remains your call.

Trigger Events

…ames

Replace both dashboard screenshots on the Manage environment variables page
with fresh captures from the live product, and correct the prose the stale
images had drifted away from:

- Settings now opens on Environment Variables, so the old "find the section"
  step folds into step 1.
- The add-a-variable flow submits with Add Variable, not Save. Save only
  appears in the per-row Update flow, which the update section now describes.
- The Secret control is a toggle, not a checkbox.
- Name the Update and Delete buttons in the Actions column.

Add a Doc Detective spec asserting the documented controls still exist, so the
next UI drift is caught rather than discovered by a reader.
@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
promptless-docs Ready Ready Preview, Comment Jul 28, 2026 5:39pm

Request Review

@promptless

promptless Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Just a reminder: I review PR comments by default. If you want me to ignore a specific comment, start it with aside or /aside.

… it protects

The spec's description narrated how the test was written -- authoring
decisions, CI-history commit hashes, framework semantics, and a
justification of each matcher choice. An engineer could not tell from it
which product behavior was under test or what breaks when it fails.

It now leads with the journey it protects (cuj-screenshots, step
"Authenticate to the app being captured"), what breaks for users if the
test fails, and what is and is not covered. The operational facts an
engineer needs to run it -- auth, data prerequisite, invocation, CI
status -- are kept as subordinate notes. The same episodic voice is
removed from the per-test and step descriptions.

Two assertions are corrected while here, because they described a DOM
that does not exist. The Secret control is a CSS-styled toggle: its
label contains no checkbox and no switch role, and aria-checked is null.
The spec asserted secretToggle=present and secretChecked=true, so two
steps failed against a healthy dashboard, and the claim contradicted the
page shipping beside it, whose prose correctly calls Secret a toggle.
Both now assert the masking the docs actually promise -- the Value input
flipping to password -- which fails if the toggle stops working. The
toggle's on-state has no machine-readable signal beyond computed CSS,
which the docs never promise, so it is not asserted.

The spec stays non-destructive: nothing was added that submits the form
or deletes a variable. All six probes now pass against the live
dashboard, up from four.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants