Watched document handling and misc fixes

[peachbits]

CHANGELOG

Does this branch warrant an entry to the CHANGELOG?

• Yes
• No

Dependencies

none

Description

none

---

Note

Medium Risk
Touches v3 synced-document initialization and introduces a DB watcher, which can affect runtime config loading and update propagation. Also changes rate-provider behavior (coinrank filtering and historical cutoff), which may alter returned datasets for some requests.

Overview
Centralizes v3 rates_settings CouchDB synced-document handling by introducing a one-time bootstrapV3SyncedDocs() that performs an initial sync() across all v3 docs, starts 30-minute fallback re-sync intervals, and begins a watchDatabase changes-feed watcher; both src/v3/index.ts and src/v3/indexEngines.ts now call this before serving/starting engines.

Adds shared synced-document options (cleanFailStrategy: 'preserve' + Slack reporting on cleaner failures) and refactors providers/router to export their synced-doc lists and use these options when constructing syncedDocument(...) instances.

Tightens data handling: Coinrank now treats Coingecko market_cap_rank as optional and filters unranked assets from the cached market list, and CoinMarketCap historical lookups now short-circuit for dates older than a configurable maxHistoricalMonths (new env/config setting). Also bumps edge-server-tools to ^0.2.24 and makes a few error typings explicit (unknown).

^{Reviewed by Cursor Bugbot for commit dd35df3. Bugbot is set up for automated code reviews on this repo. Configure here.}

---

• To see the specific tasks where the Asana app for GitHub is being used, see below:
  • https://app.asana.com/0/0/1213106592724666

[cursor]

Filtering rank reduces cached market coverage

Medium Severity

Filtering marketsPage by m.rank != null while still stopping at a fixed NUM_PAGES can store fewer markets than before and omit ranked assets that appear on later pages. This can make the cached dataset incomplete relative to the expected “top N” coverage and lead to unexpectedly short result sets downstream.

 

[peachbits]

This is fine. There is no expectation that there will be exactly 2000 ranked assets

[paullinator]

Clean, correct change that handles newly listed/unranked assets gracefully. One suggestion to tighten the TypeScript type after filtering — see inline comment.

[paullinator]

After this filter, all entries in markets are guaranteed to have a numeric rank. However, the TypeScript type still reflects rank: number | undefined (since asOptional(asNumber) returns number | undefined). Any downstream consumer of CoinrankMarkets will see rank as potentially undefined even though it never is after filtering.

Suggestion: Use a type predicate in the filter to narrow the type:

markets = [
  ...markets,
  ...marketsPage.filter((m): m is typeof m & { rank: number } => m.rank != null)
]

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Engine process never bootstraps synced documents
  • The engines entrypoint now awaits bootstrapV3SyncedDocs before starting loops so synced docs are initially loaded and watched in the ratesEngines process too.

Or push these changes by commenting:

@cursor push 03341bc33b

Preview (03341bc33b)

diff --git a/src/v3/indexEngines.ts b/src/v3/indexEngines.ts
--- a/src/v3/indexEngines.ts
+++ b/src/v3/indexEngines.ts
@@ -5,6 +5,7 @@
   dbProviders,
   memoryProviders
 } from './providers/allProviders'
+import { bootstrapV3SyncedDocs } from './syncedDocs'
 import type { Frequency, FrequencySeconds, RateEngine } from './types'
 
 const frequencyToMs: Record<Frequency, number> = {
@@ -56,4 +57,13 @@
     }
   }
 }
-startEngines()
+
+async function main(): Promise<void> {
+  await bootstrapV3SyncedDocs()
+  startEngines()
+}
+
+main().catch((error: unknown) => {
+  console.error(error)
+  process.exit(1)
+})

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

[j0ntz]

watchDatabase() already starts the Couch changes feed before doing its initial sync() calls. Doing a separate Promise.allSettled(...sync()) first creates a race window where a rates_settings update can land after this preload but before the watcher starts with since: "now", so this process misses that change until the 30-minute interval fires. I would let watchDatabase own the initial sync (or start watching before the preload) to avoid serving stale mappings during startup.

[peachbits]

It took me a minute to understand the comment. The issue is the onChange wouldn't fire if the doc was edited between the initial sync and when watchDatabase is called. I don't think this actually matters because this bootstrap function blocks the engine from starting and therefore those docs aren't being written to. There are human-managed docs in the settings db that could technically be written to during those milliseconds but the likelihood is small (it's rare for humans to touch the docs) and trivial (only a 30 minute delay in the doc updating)

[j0ntz]

#122 (comment) is still unaddressed?

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit fc4cb62. Configure here.}