Skip to content

Support mixed depth container merkle proofs#508

Merged
ed255 merged 7 commits into
mainfrom
feat/small-mtps
May 6, 2026
Merged

Support mixed depth container merkle proofs#508
ed255 merged 7 commits into
mainfrom
feat/small-mtps

Conversation

@ed255
Copy link
Copy Markdown
Collaborator

@ed255 ed255 commented May 5, 2026

Add support for mixed depth container merkle proofs (both for state proofs and transition proofs)
I've refactored the fields related to container merkle proofs in params for better organization.
In his PR only "small" and "medium" depths are implemented, but the code organization already lays out the foundation for adding a future "big" depth.

I've also removed the enabled flag from all merkle proofs to save a few constraints (not many) and to simplify the circuits because our architecture doesn't really need disabling those circuits.

For the small container state proofs, the circuit only verifies inclusion (which requires less constraints than inclusion+non-inclusion).
We don't have a circuit that only verifies transition updates (instead of insert+update+delete). That can be future work to remove more constraints for the small container updates. Nevertheless the implementation in this PR already assumes that the small container transitions are only updates, so that in the future we can just swap the circuit and benefit without any further change.

I didn't touch the frontend: the MultiPod builder only considers the medium proofs capacity for now, so it doesn't generate optimal scheduling. I propose adding support after #502 is merged

There are 2 TODOS in the code about a potential off-by-one error. When I consider max_depth I assume we get 2^max_depth leafs in the tree, and max_depth siblings. But I'm not sure that's actually the case. This is a pending task for me: to validate that this is the case, or else change the code.

@ed255
Copy link
Copy Markdown
Collaborator Author

ed255 commented May 5, 2026

Here's some numbers!
The params in this PR give:

  • small depth: 8
  • medium depth: 32
  • max container state
    • small: 22
    • medium: 8
  • max container transition
    • small: 12
    • medium: 6
degree_bits = 16, free_gates = 8507
  • A small container state proof is 29.0 gates VS 108.5 gates for medium
  • A small container update proof is 90.3 gates VS 324.5 gates for medium
    • The small version can be smaller if we only support updates. The current circuit supports insert/update/delete.
  • The AuxTable is now larger, which increases a bit the cost of each operation.
Gate count:
- RecCircuit: 1 x 43799.0 = 43799
- RecCircuit/MainPodVerify: 1 x 32718.0 = 32718
- RecCircuit/MainPodVerify/OpVerifyPub: 8 x 35.4 = 283
- RecCircuit/MainPodVerify/OpVerifyPub/OpCopy: 8 x 13.6 = 109
- RecCircuit/MainPodVerify/OpVerifyPub/OpNone: 8 x 11.8 = 94
- RecCircuit/MainPodVerify/OpVerifyPub/ResolveOpArgs: 8 x 10.0 = 80
- RecCircuit/MainPodVerify/OpVerifyPriv: 40 x 562.5 = 22498
- RecCircuit/MainPodVerify/OpVerifyPriv/OpCustom: 40 x 94.7 = 3788
- RecCircuit/MainPodVerify/OpVerifyPriv/MerkleDeleteOp: 40 x 15.4 = 616
- RecCircuit/MainPodVerify/OpVerifyPriv/MerkleUpdateOp: 40 x 16.1 = 644
- RecCircuit/MainPodVerify/OpVerifyPriv/MerkleInsertOp: 40 x 17.3 = 692
- RecCircuit/MainPodVerify/OpVerifyPriv/OpSignedBy: 40 x 14.3 = 572
- RecCircuit/MainPodVerify/OpVerifyPriv/OpPublicKeyOf: 40 x 13.8 = 552
- RecCircuit/MainPodVerify/OpVerifyPriv/OpNotContainsFromEntries: 40 x 14.3 = 572
- RecCircuit/MainPodVerify/OpVerifyPriv/OpContainsFromEntries: 40 x 15.8 = 632
- RecCircuit/MainPodVerify/OpVerifyPriv/OpReplaceValueWithEntry: 40 x 27.2 = 1090
- RecCircuit/MainPodVerify/OpVerifyPriv/OpMaxOf: 40 x 19.4 = 774
- RecCircuit/MainPodVerify/OpVerifyPriv/OpProductOf: 40 x 41.0 = 1638
- RecCircuit/MainPodVerify/OpVerifyPriv/OpSumOf: 40 x 20.7 = 828
- RecCircuit/MainPodVerify/OpVerifyPriv/OpHashOf: 40 x 15.3 = 612
- RecCircuit/MainPodVerify/OpVerifyPriv/OpLtToNeq: 40 x 12.7 = 508
- RecCircuit/MainPodVerify/OpVerifyPriv/OpTransitiveEq: 40 x 16.9 = 676
- RecCircuit/MainPodVerify/OpVerifyPriv/OpLtEqFromEntries: 40 x 24.3 = 972
- RecCircuit/MainPodVerify/OpVerifyPriv/OpEqNeqFromEntries: 40 x 17.1 = 686
- RecCircuit/MainPodVerify/OpVerifyPriv/OpCopy: 40 x 13.3 = 532
- RecCircuit/MainPodVerify/OpVerifyPriv/OpNone: 40 x 11.8 = 472
- RecCircuit/MainPodVerify/OpVerifyPriv/GetTaggedTblEntry: 40 x 46.0 = 1840
- RecCircuit/MainPodVerify/OpVerifyPriv/ResolveOpArgs: 40 x 93.2 = 3730
- RecCircuit/MainPodVerify/CalculateStsHash: 1 x 44.0 = 44
- RecCircuit/MainPodVerify/BuildOpAuxTbl: 1 x 7814.0 = 7814
- RecCircuit/MainPodVerify/BuildOpAuxTbl/CustomPredVerify: 8 x 258.6 = 2069
- RecCircuit/MainPodVerify/BuildOpAuxTbl/CustomPredVerify/HashTaggedTblEntry: 8 x 41.0 = 328
- RecCircuit/MainPodVerify/BuildOpAuxTbl/CustomPredVerify/CustomOpVerify: 8 x 184.6 = 1477
- RecCircuit/MainPodVerify/BuildOpAuxTbl/CustomPredVerify/CustomOpVerify/PredFromTmpl: 40 x 1.3 = 53
- RecCircuit/MainPodVerify/BuildOpAuxTbl/CustomPredVerify/CustomOpVerify/StArgFromTmpl: 40 x 21.8 = 870
- RecCircuit/MainPodVerify/BuildOpAuxTbl/SignedBy: 4 x 187.2 = 749
- RecCircuit/MainPodVerify/BuildOpAuxTbl/SignedBy/HashTaggedTblEntry: 4 x 2.0 = 8
- RecCircuit/MainPodVerify/BuildOpAuxTbl/SignedBy/SignatureVerify: 4 x 183.2 = 733
- RecCircuit/MainPodVerify/BuildOpAuxTbl/PublicKeyOf: 2 x 146.5 = 293
- RecCircuit/MainPodVerify/BuildOpAuxTbl/PublicKeyOf/HashTaggedTblEntry: 2 x 2.0 = 4
- RecCircuit/MainPodVerify/BuildOpAuxTbl/MerkleProofStateTransition_32: 6 x 324.5 = 1947
- RecCircuit/MainPodVerify/BuildOpAuxTbl/MerkleProofStateTransition_32/MerkleProofExist_32: 6 x 107.2 = 643
- RecCircuit/MainPodVerify/BuildOpAuxTbl/MerkleProofStateTransition_32/MerkleProof_32: 6 x 108.5 = 651
- RecCircuit/MainPodVerify/BuildOpAuxTbl/MerkleProofStateTransition_8: 12 x 90.3 = 1084
- RecCircuit/MainPodVerify/BuildOpAuxTbl/MerkleProofStateTransition_8/MerkleProofExist_8: 12 x 29.2 = 351
- RecCircuit/MainPodVerify/BuildOpAuxTbl/MerkleProofStateTransition_8/MerkleProof_8: 12 x 30.5 = 366
- RecCircuit/MainPodVerify/BuildOpAuxTbl/MerkleProof_32: 8 x 108.5 = 868
- RecCircuit/MainPodVerify/BuildOpAuxTbl/MerkleProofExist_8: 22 x 29.0 = 639
- RecCircuit/MainPodVerify/BuildOpAuxTbl/HashTaggedTblEntry: 49 x 3.3 = 163
- RecCircuit/MainPodVerify/BuildCustomPredTbl: 1 x 1478.0 = 1478
- RecCircuit/MainPodVerify/BuildCustomPredTbl/CustomPred: 8 x 184.8 = 1478
- RecCircuit/MainPodVerify/BuildCustomPredTbl/CustomPred/MerkleProof_16: 8 x 56.4 = 451
- RecCircuit/MainPodVerify/VerifyInPod: 2 x 105.5 = 211
- RecCircuit/MainPodVerify/VerifyInPod/MerkleProof_6: 2 x 24.5 = 49
- RecCircuit/MainPodVerify/VerifyInPod/CalculateStsHash: 2 x 44.0 = 88
- RecCircuit/VerifyProof: 2 x 5442.5 = 10885
- CalculateStsHash: 1 x 6.0 = 6

@ed255 ed255 requested a review from robknight May 5, 2026 14:44
Copy link
Copy Markdown
Collaborator

@robknight robknight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. The one bug I raised is a pre-existing issue.

OperationAux::MerkleTreeStateTransitionProofIndex(i) => {
write!(f, " merkle_tree_state_transition_proof_{:02}", i)?
OperationAux::MerkleTransitionProofIndex(size, i) => {
write!(f, " {:?}merkle_tree_state_transition_proof_{:02}", size, i)?
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing underscore after {:?}?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right, I'll add a Display impl so that it formats nicely as well.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed here b6bb65c

verified_proof.verifier_data_hash.elements[i],
vd_mt_proof.value.elements[i],
)
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we are not ensuring that existence is true here, so a non-existence proof would be accepted in place of an existence proof. Not a new issue, seems like it was always that way!

Copy link
Copy Markdown
Collaborator Author

@ed255 ed255 May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh no... you're totally right! This was a very serious bug! Great catch!
I'll switch to the circuit that only verifies existence.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed here b6bb65c

MerkleTransitionProof = 2,
CustomPredVerify = 3,
PublicKeyOf = 4,
SignedBy = 5,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, is there any particular reason for re-numbering these?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I took this chance to change it for two reasons:

  1. It felt strange to have the merkle state proofs and transition proofs in non-consecutive order
  2. I wanted to have an order that reflects the importance of each subtable. I see merkle trees and custom predicates as very core to pod2, whereas PublicKeyOf and SignedBy seem more like nice to have.
  3. The renumbering here is not very important, but I also reordered the aux table (so that state and transition merkle proofs are consecutive and custom predicate verifications come next), and I wanted to be consistent with the same order everywhere.

@ed255 ed255 merged commit 5e3ac9a into main May 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants