chore: sync upstream 2026-03-27#1
Open
hanzo-dev wants to merge 17 commits into
Open
Conversation
…lowing shell caching (formancehq#1291)
formancehq#1295) * fix(ledger): push address filter into LATERAL join for GIN index usage (formancehq#1293) * fix(ledger): push address filter into LATERAL join for GIN index usage When filtering by partial address (e.g. $match[system:cashier:xxx]), the query builder was placing the address_array jsonpath filter only in the outer WHERE clause, after GROUP BY. This forced Postgres to seq scan the entire moves/accounts_volumes table, join all accounts, aggregate, then discard non-matching rows (~6.6s on large datasets). By duplicating the address filter inside the LATERAL join on accounts, Postgres can leverage the GIN index on accounts_address_array to resolve matching accounts first (~3ms, x2200 improvement). Key implementation details: - The UseFilter callback always returns false to avoid short-circuit evaluation, ensuring ALL address values are captured (critical for $or queries mixing exact and partial addresses) - All address values (exact + partial) are pushed as OR conditions into the lateral, so exact addresses are not excluded - The outer WHERE filter (via ResolveFilter) is preserved as-is * fix(ledger): handle $not queries in lateral join address filter The address filter pushed into LATERAL joins is a positive match, which is incompatible with $not (negation). When $not is present, the lateral would keep only matching rows, but the outer WHERE would negate them → 0 results. Add queryHasNegation() detection that builds the SQL and checks for "not (" presence. When detected, the lateral optimization is skipped and the query falls back to the original (correct) behavior. * fix(ledger): use JSON inspection for $not detection and fix non-PIT guard Address review feedback: - Replace fragile SQL string matching ("not (") with JSON marshaling to detect "$not" operator in query builder AST - Add missing !queryHasNegation guard in non-PIT path of aggregated_balances (was present in PIT path and both volumes paths) * fix(ledger): skip lateral filter for $or queries and rename guard The lateral join address filter optimization is also unsafe with $or when mixing address filters with non-address conditions (e.g. balance). The lateral excludes rows not matching the address, but those rows might match another $or branch → missing results. Rename queryHasNegation to canPushAddressFilterToLateral and extend it to also detect $or via JSON AST inspection. The optimization now only applies to pure $and queries (the common single-filter case). * refactor(ledger): extract helpers to reduce duplication in lateral filter Extract collectAddressFilters() and applyLateralAddressFilter() to eliminate duplicated code: - 10-line address collection block (was in 2 files) → 1-line call - 3-line push-to-lateral guard (was in 4 locations) → 1-line call No behavioral change. * fix: handle \$in operator in collectAddressFilters and remove unused function * style: fix import grouping and formatting for CI Fix import grouping (stdlib, external, internal) and reformat the collectAddressFilters interface parameter onto multiple lines to satisfy the goimports/gofmt formatter used by the CI Dirty check.
…to charm.land (formancehq#1305) The security update PR formancehq#1302 bumped pulumi-aws/sdk from v6 to v7, which pulled in charmbracelet v2 modules that have migrated their module paths from github.com/charmbracelet/* to charm.land/*. This broke go mod tidy.
…ancehq#1306) This reverts commit d8ba1b5.
…ring matching (formancehq#1297) * fix(ledger): use AST analysis for lateral filter safety instead of string matching (formancehq#1296) The previous implementation detected $not/$or by searching for literal strings in the JSON output, which was too conservative: it disabled the LATERAL join optimization even when $not/$or didn't affect address filters (e.g. $not on metadata, single-item $or wrappers). Replace with a proper JSON AST walker that tracks structural context: - isNodeSafeForLateral: walks the tree tracking $not ancestry - nodeContainsAddressFilter: checks if a subtree has address filters - Only disables optimization when $not/$or actually impacts addresses This fixes the production case where $and(address, $not(metadata)) was incorrectly skipping the GIN index optimization. Also adds unit tests for canPushAddressFilterToLateral (17 cases) and caches the safety check result per BuildDataset call. * fix(ledger): reorder imports in utils_test.go to satisfy linter Group stdlib imports separately from third-party, with testify before formancehq per goimports ordering rules.
* chore: upgrade Go to 1.26 from pkgs-unstable Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update go.mod to Go 1.26.0 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate Speakeasy client v1.758.0 Upgrade Speakeasy SDK generator to v1.758.0 and regenerate the Go client. Fix OpenAPI spec invalid $ref to parameter in schema context (sort field). Update test files and provisioner for V2ExporterConfiguration -> V2CreateExporterRequest rename, and V2RevertTransactionResponse field access in revert tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(openapi): fix invalid $ref in v2.yaml source for sort field The sort field in V2QueryParams used $ref to a parameter object in a schema context. Fix the source file (v2.yaml) to match the already fixed merged openapi.yaml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: tidy tools/provisioner go.mod Remove unused ericlagergren/decimal dependency. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate API docs and tidy tools/generator - Regenerate docs/api/README.md from updated OpenAPI spec - Remove unused ericlagergren/decimal from tools/generator Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump go-libs/v4 to 56bca73d45aa (Go 1.26 fixes) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(test): buffer export body before import to fix Go 1.26 timeout Go 1.26 changed HTTP response body streaming behavior, causing the ImportLogs request to hang when directly reusing an export response body as a request body. Buffer with io.ReadAll first. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump go-libs/v4 to v4.1.3 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: fix import endpoint data race and SDK DrainBody issue - controllers_logs_import.go: fix data race between goroutine reading stream channel and main goroutine setting stream=nil. Use separate select after close(stream) instead of nil-channel trick. - pkg/client/v2.go: remove DrainBody call on ExportLogs 200 response. The response body IS the streaming payload; draining it discards the export data before callers can read it. - Revert test buffering (body is now preserved by SDK). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(import): block on errChan after stream close instead of select After EOF closes the stream, the import goroutine's result is authoritative. A two-way select could pick r.Context().Done() even when errChan is ready, losing the import result. Block directly on errChan instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(openapi): specify ExportLogs response body as application/octet-stream The export endpoint was missing content type on 200 response, causing Speakeasy to generate DrainBody() which discards the streaming payload. Also fix error response to use application/json with V2ErrorResponse. Regenerate SDK client and update test to use response.Bytes field. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate API docs after ExportLogs spec fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…rmance (formancehq#1311) Without this index, listing transactions without filters causes a full table sequential scan and on-disk sort of all rows before applying LIMIT. On large ledgers (12M+ rows), this results in ~200s response times. The descending index allows PostgreSQL to use an Index Scan Backward for the default ORDER BY id DESC pagination, reducing response time to milliseconds.
…ters responses (formancehq#1312) V2ListPipelinesResponse and V2ListExportersResponse had a nested cursor structure (cursor.cursor.data) instead of the expected flat one (cursor.data). The responses wrapped the CursorResponse schema in an extra cursor property with a redundant allOf, while the referenced schemas already contained the cursor structure. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
# Conflicts: # go.mod
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated upstream sync