Background Job to Run a Command in Headless Chrome by tintinthong · Pull Request #4024 · cardstack/boxel

tintinthong · 2026-02-19T13:16:54Z

Demo

https://www.loom.com/share/2fe9a3e58a7c459ba574b2c0f747a667

Testing Locally

Start bot-runner with pnpm start:development.
Start Realm server with pnpm start:all
Inside packages/matrix, pnpm setup-submission-bot
Open http://localhost:4200/experiments/BotRequestDemo/bot-request-demo.
Click Send Show Card Bot Request on the demo card.

Flow Diagram

User issuing task

Have made an example card in experiments to simulate but commands can be called from anywhere

flowchart TD

    B["Trigger 'Send Show Card Bot Request'" from UI]
    B --> D["CreateShowCardRequestCommand.execute()"]
    D --> E["SendBotTriggerEventCommand.execute()"]
    E --> F["sendEvent('app.boxel.bot-trigger')"]

    F --> G["bot-runner receives timeline event"]
    G --> H["enqueueRunCommandJob() to DB"]

Command runner architecture

flowchart TD

    A["Bot Runner DB access"] -->|enqueue job| B["Queue jobs table"]
    B --> C["Worker runtime common"]
    C -->|run command task| D["Prerenderer remote"]
    D -->|run command request| E["Prerender Manager"]
    E -->|proxy to prerender server| F["Prerender App"]
    F -->|run command in tab| G["Headless Chrome tab"]
    G -->|execute command| H["Host app command-runner route"]
    H -->|result in DOM| G
    G -->|capture result| F
    F --> D
    D --> C
    C --> B

Matrix Event Payload (app.boxel.bot-trigger)

const event = {
  type: 'app.boxel.bot-trigger',
  content: {
    type: 'show-card',
    realm: 'http://localhost:4201/experiments/',
    input: {
      cardId: 'http://localhost:4201/experiments/Author/jane-doe',
      format: 'isolated',
    },
  } 
};

Bot Trigger Data Structures (with realm)

type SendBotTriggerEventInput = {
  roomId: string;
  type: string;
  realm: string;
  input: Record<string, unknown>;
};

type BotTriggerContent = {
  type: string;
  realm: string;
  input: unknown;
};

Bot-runner filter

Commands registered for submission-bot

The bot runner checks if the matrix events comes thru via the filter blob

[
  {
    "name": "create-listing-pr",
    "command": "http://localhost:4201/commands/create-listing-pr/default",
    "filter": {
      "type": "matrix-event",
      "event_type": "app.boxel.bot-trigger",
      "content_type": "create-listing-pr"
    }
  },
  {
    "name": "show-card",
    "command": "http://localhost:4201/boxel-host/commands/show-card/default",
    "filter": {
      "type": "matrix-event",
      "event_type": "app.boxel.bot-trigger",
      "content_type": "show-card"
    }
  }
]

Run Command Task

RunCommandArgs.realmURL is derived from event.content.realm.

{
  "realmURL": "http://localhost:4201/experiments/",
  "realmUsername": "@alice:localhost",
  "runAs": "@alice:localhost",
  "command": {
    "module": "@cardstack/boxel-host/commands/show-card",
    "name": "default"
  },
  "commandInput": {
    "cardId": "http://localhost:4201/experiments/Author/jane-doe",
    "format": "isolated"
  }
}

Command Runner Route on Host

route: /command-runner/:nonce

type CommandRunnerRouteParams = {
  nonce: string;
};

type CommandRunnerQueryParams = {
  command?: string;
  input?: string;
};

Example (URL + queryParams)

const url = 'http://localhost:4200/command-runner/2';

// human-readable command id
const commandId = '@cardstack/boxel-host/commands/show-card/default';

// command-runner query param expects an encoded ResolvedCodeRef JSON object
const commandCodeRef = {
  module: '@cardstack/boxel-host/commands/show-card',
  name: 'default',
};

const queryParams: { command: string; input: string } = {
  command: encodeURIComponent(JSON.stringify(commandCodeRef)),
  input:
    encodeURIComponent(
      JSON.stringify({
        cardId: 'http://localhost:4201/experiments/Author/jane-doe',
        format: 'isolated',
      }),
    ),
};

http://localhost:4200/command-runner/2?command=%7B%22module%22%3A%22%40cardstack%2Fboxel-host%2Fcommands%2Fshow-card%22%2C%22name%22%3A%22default%22%7D&input=%7B%22cardId%22%3A%22http%3A%2F%2Flocalhost%3A4201%2Fexperiments%2FAuthor%2Fjane-doe%22%2C%22format%22%3A%22isolated%22%7D

github-actions · 2026-02-19T13:24:31Z

Preview deployments

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5d7b24826

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-19T13:24:59Z

packages/host/app/routes/command-runner.ts

+      return model;
+    }
+
+    void this.#runCommand(model, command, commandInput);


Restrict command-runner execution to trusted callers

This route executes whatever resolved command is supplied in the URL query params (void this.#runCommand(...)) without validating caller trust or command allowlisting. Since /command-runner/:nonce is a normal GET route, a crafted link can make an authenticated user’s browser execute write-capable commands with that user’s stored session, which creates a CSRF-style arbitrary command execution path.

Useful? React with 👍 / 👎.

packages/realm-server/prerender/render-runner.ts

chatgpt-codex-connector · 2026-02-19T13:24:59Z

packages/realm-server/prerender/render-runner.ts

+        let resultElement = container?.querySelector(
+          '[data-command-result]',
+        ) as HTMLElement | null;


Return the serialized command output payload

The response extractor reads result from [data-command-result], but the new command-runner template never renders that attribute, so payload.result is always null even when the route computes model.result. This drops command output from successful run-command responses and makes the result field effectively unusable.

Useful? React with 👍 / 👎.

Copilot

Pull request overview

This PR implements a background job system to run commands in headless Chrome via the bot-runner. It introduces a new run-command task that enables bots to execute commands in a headless browser and capture results, supporting use cases like showing cards or patching card instances through Matrix events.

Changes:

Added run-command task and job infrastructure in runtime-common for executing commands via prerenderer
Implemented command-runner route in host app to render command results in headless Chrome
Extended bot-runner timeline handler to enqueue run-command jobs from bot-trigger events
Removed hardcoded bot trigger command type restrictions to support custom command types
Added submission bot to AI assistant rooms and updated setup script to register multiple bot commands

Reviewed changes

Copilot reviewed 40 out of 40 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/runtime-common/worker.ts	Registers the new run-command task with the worker queue
packages/runtime-common/tasks/run-command.ts	New task implementation for running commands with permission checks
packages/runtime-common/tasks/index.ts	Exports run-command task
packages/runtime-common/jobs/run-command.ts	Job enqueueing logic for run-command jobs
packages/runtime-common/index.ts	Type definitions for RunCommandArgs and RunCommandResponse, adds runCommand to Prerenderer interface
packages/runtime-common/code-ref.ts	New assertIsResolvedCodeRef utility function
packages/runtime-common/bot-trigger.ts	Removes hardcoded command type validation, adds realm field requirement
packages/runtime-common/bot-command.ts	Removes hardcoded command type restrictions to allow any string content_type
packages/runtime-common/matrix-constants.ts	Removes BOT_TRIGGER_COMMAND_TYPES constant
packages/realm-server/prerender/prerender-app.ts	New /run-command endpoint in prerender server
packages/realm-server/prerender/prerenderer.ts	Implements runCommand method with browser restart fallback
packages/realm-server/prerender/render-runner.ts	Implements runCommandAttempt to navigate to command-runner route and extract results
packages/realm-server/prerender/remote-prerenderer.ts	Adds runCommand method to remote prerenderer client
packages/realm-server/prerender/manager-app.ts	Proxies /run-command requests to prerender servers
packages/realm-server/prerender/utils.ts	New buildCommandRunnerURL utility and enhanced transitionTo with query params support
packages/realm-server/tests/prerender-server-test.ts	Test coverage for run-command endpoint
packages/realm-server/tests/prerender-proxy-test.ts	Mock prerenderer updated to support command kind
packages/realm-server/tests/prerender-manager-test.ts	Test coverage for command proxying in manager
packages/realm-server/tests/definition-lookup-test.ts	Mock prerenderer updated with runCommand stub
packages/host/app/router.ts	New command-runner route definition
packages/host/app/routes/command-runner.ts	Route implementation that executes commands and tracks state
packages/host/app/templates/command-runner.gts	Template that renders command results with prerender data attributes
packages/host/app/commands/show-card.ts	Updated to return CardDef for bot-runner compatibility
packages/host/app/commands/index.ts	Registers new bot-request commands and shimming
packages/host/app/commands/create-ai-assistant-room.ts	Invites submission bot to new AI assistant rooms
packages/host/app/commands/bot-requests/send-bot-trigger-event.ts	Moved to bot-requests folder, adds realm to required fields
packages/host/app/commands/bot-requests/create-show-card-request.ts	New command to request showing a card via bot runner
packages/host/app/commands/bot-requests/create-listing-pr-request.ts	Moved to bot-requests folder, adds realm parameter
packages/host/tests/acceptance/commands-test.gts	Test coverage for command-runner route
packages/host/tests/integration/commands/send-bot-trigger-event-test.gts	Updated tests to include realm field and allow custom types
packages/bot-runner/main.ts	Passes queuePublisher to timeline handler
packages/bot-runner/lib/timeline-handler.ts	Implements command enqueueing from bot-trigger events with URL-to-CodeRef conversion
packages/bot-runner/tests/bot-runner-test.ts	Updated test fixture with realm field
packages/matrix/scripts/setup-submission-bot.ts	Registers multiple bot commands (show-card, patch-card-instance, create-listing-pr)
packages/experiments-realm/commands/create-show-card-request.ts	Experiments realm version of create-show-card-request command
packages/experiments-realm/commands/create-patch-card-instance-request.ts	New command to request patching card instances
packages/experiments-realm/bot-request-demo.gts	Demo card with tabbed UI for testing bot requests
packages/experiments-realm/BotRequestDemo/bot-request-demo.json	Card instance data for demo
packages/base/matrix-event.gts	Removes hardcoded bot trigger command types, adds realm to BotTriggerContent
packages/base/command.gts	New CreateShowCardRequestInput type and realm field on SendBotTriggerEventInput

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

packages/host/app/templates/command-runner.gts

packages/realm-server/prerender/render-runner.ts

packages/experiments-realm/commands/create-patch-card-instance-request.ts

github-actions · 2026-02-19T13:37:00Z

Host Test Results

1 files ±0 1 suites ±0 1h 39m 15s ⏱️ - 4m 58s
1 857 tests +1 1 842 ✅ ±0 14 💤 ±0 0 ❌ ±0 1 🔥 +1
1 872 runs +1 1 856 ✅ - 1 14 💤 ±0 1 ❌ +1 1 🔥 +1

For more details on these errors, see this check.

Results for commit fced14f. ± Comparison against base commit 82e0e25.

This pull request removes 1 and adds 2 tests. Note that renamed tests count towards both.

Chrome ‑ Integration | commands | send-bot-trigger-event: rejects unknown trigger types

Chrome ‑ Acceptance | Commands tests > command-runner: route captures command runtime error
Chrome ‑ Acceptance | Commands tests > command-runner: route renders command result card

♻️ This comment has been updated with latest results.

tintinthong · 2026-02-19T15:15:10Z

packages/matrix/scripts/setup-submission-bot.ts

+  },
+  {
+    name: 'show-card',
+    commandURL: `${realmServerURL}/boxel-host/commands/show-card/default`,


perhaps this should be @boxel-host/commands which is the import specifier but in url form?

tintinthong · 2026-02-19T15:15:45Z

packages/bot-runner/lib/timeline-handler.ts

+    // TODO: boxel-host commands are not exposed internally as HTTP URLs; they
+    // are only available via module specifiers, so we map those URLs to code refs.


Perhaps, this can be removed if we treat the url as an import specifier

packages/host/app/commands/bot-requests/create-show-card-request.ts

tintinthong · 2026-02-19T15:31:52Z

packages/base/command.gts

  @field roomId = contains(StringField);
  @field type = contains(StringField);
  @field input = contains(JsonField);
+  @field realm = contains(StringField);


It seems that the worker task requires a realm to be known to check for permissions. So we just constantly send a realm from UI

tintinthong · 2026-02-19T17:46:29Z

packages/realm-server/prerender/render-runner.ts

+      let response: RunCommandResponse = {
+        status: payload.status,
+        result: payload.result ?? undefined,
+        error: payload.error ?? undefined,
+      };


I realise payload.result is the serialized json. There is also .value which is the actual card instance. Currently, there is no meaning to this as we are not using the rendering of the command anywhere other than the command-runner route.

So I'm not sure what is best to return here

it's up to you what you want to return here. it looks like a command can return either a card or a string. you should have optional properties for these 2 different types of results.

habdelra · 2026-02-19T19:49:39Z

packages/host/app/routes/command-runner.ts

+    let queryParams = transition?.to?.queryParams ?? {};
+    let command = parseResolvedCodeRef(getQueryParam(queryParams, 'command'));
+    let commandInput = parseJson(getQueryParam(queryParams, 'input'));


this is not a good pattern: you should add the command and the input as route segments. let's not use query params for our routes. so your route should look like:

this.route('command-runner', { path: '/command-runner/:command/:input/:nonce' });

habdelra · 2026-02-19T19:52:55Z

packages/host/app/routes/command-runner.ts

+    if (command) {
+      model.commandRef = JSON.stringify(command);
+    }
+    if (commandInput) {
+      model.commandInput = JSON.stringify(commandInput);
+    }
+
+    if (!command) {
+      model.status = 'error';
+      model.error = new Error('Missing or invalid command');
+      return model;
+    }


this all goes away if you use route segments for the command and command input, they are guaranteed to be there

habdelra · 2026-02-19T19:58:10Z

~~There are tests missing that i'm expecting to see:~~

you should have a host test module that tests your command-runner route directly. similar to prerender-html-test.gts, prerender-meta-test.gts, and prerender-module-test.gts. This way you can have a smaller easier place to test just the stuff that runs inside the prerender server instead of having to boot up puppeteer. That way your server tests can focus on the stuff outside of what is happening inside the command-runner route.

MY BAD! I just noticed acceptance-test/commands/test-.gts. this is exactly what I wanted to see 👍

habdelra · 2026-02-19T21:47:15Z

packages/host/tests/acceptance/commands-test.gts

+  test('command-runner route renders command result card', async function (assert) {
+    let commandRef = {
+      module: `${testRealmURL}command-runner-hello-command`,
+      name: 'default',
+    };
+    let commandParam = encodeURIComponent(JSON.stringify(commandRef));
+    await visit(`/command-runner/1?command=${commandParam}`);
+
+    await waitFor('[data-prerender][data-prerender-status="ready"]');
+    assert
+      .dom('[data-test-command-runner-greeting]')
+      .includesText('Hello from command runner');
+  });
+


add a test for when the command throws an error and ensure that the DOM is what you would expect for an error

habdelra · 2026-02-19T21:48:39Z

packages/realm-server/tests/prerender-server-test.ts

      assert.ok(res.body.meta?.pool?.pageId, 'has pool.pageId');
    });

+    test('it handles run-command request', async function (assert) {


Add a test for when there is an error generated by the command and assert that the prerenderer is able to capture the error from the DOM correctly and return it in the properly serialized format for a command error

habdelra · 2026-02-19T21:50:22Z

packages/host/app/templates/command-runner.gts

+        <CardContainer class='command-runner-result'>
+          <CardRenderer @card={{@model.value}} @format='isolated' />
+        </CardContainer>


does a command always return a card?

habdelra · 2026-02-19T21:53:53Z

packages/host/app/routes/command-runner.ts

+
+class CommandRunState implements CommandInvocation<CardDefConstructor> {
+  @tracked status: CommandInvocation<CardDefConstructor>['status'] = 'pending';
+  @tracked value: CardDef | null = null;


this.value and this.result are confusing for me. i think what you really mean is resultCard, resultString. can we rename this to make more sense for what they actually are. its kind of ambiguous now.

habdelra · 2026-02-19T21:56:08Z

packages/realm-server/tests/prerender-server-test.ts

+      assert.notOk(res.body.data.attributes.error, 'no command error');
+      assert.ok(res.body.meta?.timing?.totalMs >= 0, 'has timing');
+      assert.ok(res.body.meta?.pool?.pageId, 'has pool.pageId');
+    });


a command can return a card or it can return a string result. you need to make sure that you test the ability for the prerenderer to capture these 2 different types of responses from a command

habdelra · 2026-02-19T21:57:17Z

packages/host/tests/acceptance/commands-test.gts

+    assert
+      .dom('[data-test-command-runner-greeting]')
+      .includesText('Hello from command runner');


a command can return either a card or it can return a string. which one of these is this? and where is your test for the other case?

habdelra · 2026-02-19T22:01:12Z

packages/realm-server/prerender/prerender-app.ts

    },
  });

+  router.post('/run-command', async (ctxt: Koa.Context) => {


can we try to reuse the registerPrerenderRoute function here? you have a lot of dupe code that is covered by that function.

habdelra · 2026-02-19T22:02:12Z

packages/realm-server/prerender/utils.ts

+  } catch {
+    // best effort; fall back to raw page url
+  }
+  let url = `${origin}/command-runner/${encodeURIComponent(nonce)}?command=${encodedCommand}`;


like i mentioned before, let's not use query params for the command. make it a dynamic segment of the route--this is idiomatic ember...

habdelra · 2026-02-19T22:02:40Z

packages/realm-server/prerender/utils.ts

+  }
+  let url = `${origin}/command-runner/${encodeURIComponent(nonce)}?command=${encodedCommand}`;
+  if (encodedInput) {
+    url += `&input=${encodedInput}`;


ditto for the input

tintinthong requested a review from Copilot February 19, 2026 13:21

Copilot started reviewing on behalf of tintinthong February 19, 2026 13:21 View session

tintinthong changed the title ~~Run command in headless chrome~~ Background Job to Run a Command in Headless Chrome Feb 19, 2026

chatgpt-codex-connector bot reviewed Feb 19, 2026

View reviewed changes

Copilot AI reviewed Feb 19, 2026

View reviewed changes

packages/host/app/templates/command-runner.gts Show resolved Hide resolved

packages/realm-server/prerender/render-runner.ts Outdated Show resolved Hide resolved

packages/experiments-realm/commands/create-patch-card-instance-request.ts Outdated Show resolved Hide resolved

tintinthong requested review from a team and habdelra February 19, 2026 14:43

tintinthong commented Feb 19, 2026

View reviewed changes

packages/host/app/commands/bot-requests/create-show-card-request.ts Outdated Show resolved Hide resolved

tintinthong commented Feb 19, 2026

View reviewed changes

habdelra reviewed Feb 19, 2026

View reviewed changes

tintinthong added 4 commits February 20, 2026 11:13

temporarily add submission bot to newly created rooms

ac64abb

build architecture to run in headless chrome

f55043a

update bot runner

a284ca5

update setup-submissions

55753c5

tintinthong added 17 commits February 20, 2026 11:13

add test

c2e1202

add command requests in experiments

0197023

introduce bot-request-demo in experiments harness

16cc21a

fix lint

4c10b2f

update log naming

00585a4

return a serialized result no matter what

bc38e7e

refactor bot-request-utils

f8eb802

fix nonce scope

ecd9392

fix lint

cb54b80

add submission bot to every room

2d78dd9

clean up create-show-card-requests

f5499de

since there is no more filter we remove this test

36d7498

remove test that checks for filter types

b2d4eb1

race condition for submission bot is fixed

5382690

refactor CommandInvocation to explicitly call .value as .cardResult

59fa926

Rename .result to .resultString

a605a05

add command parsing separately

9613119

tintinthong force-pushed the get-bot-runner-to-run-headless-command branch from f3b3dd0 to 9613119 Compare February 20, 2026 06:33

tintinthong added 3 commits February 20, 2026 14:34

add cardResultString

b40ed28

add error handling for prerender server

e1175ce

refactor queryParam to pathParam. Change codeRef json to url encoded url

fced14f

		// TODO: boxel-host commands are not exposed internally as HTTP URLs; they
		// are only available via module specifiers, so we map those URLs to code refs.

Comments

Conversation

tintinthong commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Demo

Testing Locally

Flow Diagram

User issuing task

Command runner architecture

Matrix Event Payload (app.boxel.bot-trigger)

Bot Trigger Data Structures (with realm)

Bot-runner filter

Commands registered for submission-bot

Run Command Task

Command Runner Route on Host

Example (URL + queryParams)

Uh oh!

github-actions bot commented Feb 19, 2026

Preview deployments

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chatgpt-codex-connector bot Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Host Test Results

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

habdelra Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

habdelra commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

tintinthong commented Feb 19, 2026 •

edited

Loading

github-actions bot commented Feb 19, 2026 •

edited

Loading

habdelra Feb 19, 2026 •

edited

Loading

habdelra commented Feb 19, 2026 •

edited

Loading