feat: configurable protocolTimeout to avoid 180s suite aborts

[kevinccbsg]

Problem

twd-cli runs the entire suite inside a single page.evaluate, which Puppeteer maps to one Runtime.callFunctionOn CDP command bound by protocolTimeout (implicit default 180000 ms). Slow-but-passing suites on CI runners hit that ceiling and abort with no per-test output:

ProtocolError: Runtime.callFunctionOn timed out.

The existing timeout config only feeds page.waitForSelector, so there was no way to raise the protocol ceiling.

Changes (Phase 1)

• Add protocolTimeout to DEFAULT_CONFIG (default 300000 = 5 min, above Puppeteer's implicit 180s). 0 means no timeout.
• Pass it through to puppeteer.launch(...).
• When a run aborts on a protocol timeout, print a hint explaining the cause and how to raise the limit.
• Document the field in twd.config.example.json, README.md, and CLAUDE.md.

To unblock a slow pipeline, set in twd.config.json:

{ "protocolTimeout": 600000 }

Out of scope

Phase 2 from the spec (running tests incrementally instead of one page.evaluate, with streamed per-test output) is intentionally deferred to a follow-up — it requires a coordinated twd-js change to expose the test list.

Tests

npm run test:ci — 200 passing, including config-merge coverage for the new field and a runTests test for the timeout hint.

🤖 Generated with Claude Code

[github-actions]

TWD Contract Validation

Spec	Passed	Failed	Warnings	Mode
./contracts/users-3.0.json	2	3	1	warn
./contracts/posts-3.1.json	2	2	0	warn
./contracts/products-3.0.json	13	23	2	warn
./contracts/events-3.1.json	6	13	0	warn

23 passed · 41 failed · 3 warnings · 1 skipped

Failed validations

./contracts/users-3.0.json

• GET /users/{userId} (200) — mock getUserNoAddress — in "Contract Validation - Mismatches > should fail: missing nested address field"
  • response.address: missing required property "address"
• GET /users/{userId} (200) — mock getUserBadAddress — in "Contract Validation - Mismatches > should fail: nested address missing required city"
  • response.address.city: missing required property "city"
  • response.address.country: missing required property "country"
• GET /users/{userId} (200) — mock getUserBadRole — in "Contract Validation - Mismatches > should fail: oneOf role with invalid variant"
  • response.role: oneOf best match (branch 2 of 2) failed: must be one of: "viewer"

./contracts/posts-3.1.json

• GET /posts/{postId} (200) — mock getPostNoAuthor — in "Contract Validation - Mismatches > should fail: post missing nested author object"
  • response.author: missing required property "author"
• GET /posts/{postId} (200) — mock getPostBadMeta — in "Contract Validation - Mismatches > should fail: post oneOf metadata matches neither variant"
  • response.metadata: oneOf best match (branch 1 of 2) failed: missing required property "category", unexpected property "duration", must be one of: "article"

./contracts/products-3.0.json

• GET /products (200) — mock getProductEmptyName — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: empty name violates minLength"
  • response[0].name: must NOT have fewer than 1 characters
• GET /products (200) — mock getProductBadSku — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid SKU pattern"
  • response[0].sku: must match pattern "^[A-Z]{2,4}-\d{4,8}$"
• GET /products (200) — mock getProductBadUuid — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid uuid format for id"
  • response[0].id: must match format "uuid"
• GET /products (200) — mock getProductBadDateTime — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid date-time format"
  • response[0].createdAt: must match format "date-time"
• GET /products (200) — mock getProductBadDate — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid date format"
  • response[0].releaseDate: must match format "date"
• GET /products (200) — mock getProductBadEmail — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid email format"
  • response[0].contactEmail: must match format "email"
• GET /products (200) — mock getProductBadUri — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid uri format"
  • response[0].website: must match format "uri"
• GET /products (200) — mock getProductBadIp — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid ipv4 format"
  • response[0].serverIp: must match format "ipv4"
• GET /products (200) — mock getProductBadIpV6 — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid ipv6 format"
  • response[0].serverIpV6: must match format "ipv6"
• GET /products (200) — mock getProductZeroPrice — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: price of 0 violates exclusiveMinimum"
  • response[0].price: must be > 0
• GET /products (200) — mock getProductNegQty — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: negative quantity violates minimum"
  • response[0].quantity: must be >= 0
• GET /products (200) — mock getProductOverQty — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: quantity exceeds maximum"
  • response[0].quantity: must be <= 999999
• GET /products (200) — mock getProductBadWeight — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: weight not multipleOf 0.01"
  • response[0].weight: must be multiple of 0.01
• GET /products (200) — mock getProductBadRating — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: rating above maximum (5)"
  • response[0].rating: must be <= 5
• GET /products (200) — mock getProductBadCurrency — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid enum value for currency"
  • response[0].currency: must be one of: "USD", "EUR", "GBP", "JPY"
• GET /products (200) — mock getProductBadCategory — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid enum value for category"
  • response[0].category: must be one of: "electronics", "clothing", "food", "books", "toys"
• GET /products (200) — mock getProductBadBool — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: string value for boolean inStock"
  • response[0].inStock: expected boolean, got string
• GET /products (200) — mock getProductDupTags — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: duplicate tags violates uniqueItems"
  • response[0].tags: must NOT have duplicate items (items ## 1 and 0 are identical)
• GET /products (200) — mock getProductTooManyTags — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: tags exceeds maxItems (10)"
  • response[0].tags: must NOT have more than 10 items
• GET /products (200) — mock getProductBadMeta — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: non-string value in metadata additionalProperties"
  • response[0].metadata.count: expected string, got number
• GET /settings (200) — mock getSettingsBadExtra — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: extra property on Settings (additionalProperties: false)"
  • response.extraField: unexpected property "extraField"
• GET /settings (200) — mock getSettingsBadLang — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid language pattern in Settings"
  • response.language: must match pattern "^[a-z]{2}(-[A-Z]{2})?$"
• GET /products (200) — mock getProductBadNullable — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: wrong type for nullable description (number instead of string|null)"
  • response[0].description: expected string,null, got number

./contracts/events-3.1.json

• GET /events (200) — mock getEventsEmpty — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: empty events array violates minItems (1)"
  • response: must NOT have fewer than 1 items
• GET /events (200) — mock getEventShortName — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: event name too short (minLength: 3)"
  • response[0].name: must NOT have fewer than 3 characters
• GET /events (200) — mock getEventBadDate — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid date-time format for startDate"
  • response[0].startDate: must match format "date-time"
• GET /events (200) — mock getEventFloatId — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: float value for integer id"
  • response[0].id: expected integer, got number
  • response[0].id: must match format "int64"
• GET /events (200) — mock getEventBadBool — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: number value for boolean active"
  • response[0].active: expected boolean, got number
• GET /events (200) — mock getEventBadStatus — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid enum value for status"
  • response[0].status: must be one of: "draft", "published", "archived"
• GET /events (200) — mock getEventScoreMax — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: score at exclusiveMaximum boundary (100)"
  • response[0].score: must be < 100
• GET /events (200) — mock getEventLowPriority — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: priority below minimum (1)"
  • response[0].priority: must be >= 1
• GET /events (200) — mock getEventHighPriority — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: priority above maximum (5)"
  • response[0].priority: must be <= 5
• GET /events (200) — mock getEventDupAttendees — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: duplicate attendees violates uniqueItems"
  • response[0].attendees: must NOT have duplicate items (items ## 1 and 0 are identical)
• GET /events (200) — mock getEventNoAttendees — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: empty attendees array violates minItems (1)"
  • response[0].attendees: must NOT have fewer than 1 items
• GET /events (200) — mock getEventBadAttendee — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid email format in attendees"
  • response[0].attendees[0]: must match format "email"
• GET /events/{eventId} (200) — mock getEventBadNullable — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: wrong type for nullable description (number instead of string|null)"
  • response.description: expected string,null, got number

View full report →