Skip to content

Update server manager for output unification #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
diegonehab wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update server manager for output unification #27

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 41 additions & 67 deletions server-manager.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ package CartesiServerManager;
// Each Session is divided into Epochs.
// The list of past Epochs (and the current Epoch) for a Session can be obtained with a call to GetSessionStatus.
// Within each Epoch, the state of the associated machine is advanced by individual inputs (see AdvanceState).
// The processing of each input, when successful, may produce several Vouchers (i.e., collateral effects
// actionable in the blockchain) and a variety of Notices (which describe any relevant changes to the internal
// state of applications running inside the Cartesi Machine).
// The processing of each input, when successful, may produce several Outputs (Vouchers are collateral effects
// actionable in the blockchain, Notices describe any relevant changes to the internal state of applications
// running inside the Cartesi Machine).
// The current Epoch can be closed (and the next Epoch started) with a call to FinishEpoch.
// This is a synchronous call that can only be issued after all inputs in the active epoch have been processed.
// The entire state of the machine can be stored on disk (and later recovered) when the Epoch is finished.
Expand All @@ -45,8 +45,7 @@ package CartesiServerManager;
// (Note that machines stored on disk are always retained.)
// When an input is skipped, for whatever reason, the state of the Cartesi Machine is reverted to what it was
// before the input was given to the machine: i.e., from the perspective of the machine, it never happened.
// Certain error conditions may cause a Session to become unusable.
// An unusable Session is said to be "tainted".
// Certain error conditions may cause a Session to become "tainted" (i.e., unusable).
// All further operations in a tainted Session (other than EndSession) will fail.
// The underlying reason can be obtained with a call to GetSessionStatus.
// Between inputs, the state of the machine can be inspected with a query (see InspectState).
Expand All @@ -56,24 +55,22 @@ package CartesiServerManager;
// to the machine: i.e., from the perspective of the machine, it never happened.
// State inspection is synchronous: the query is processed as soon as the current input is done (if any) and all
// produced Reports are immediately returned.
// A Cartesi Machine used with the Cartesi Server Manager must define 5 memory ranges, in addition to however
// A Cartesi Machine used with the Cartesi Server Manager must define 2 memory ranges, in addition to however
// many flash drives the application it runs might require.
// Each memory range must have a power-of-two length and must start at a multiple of its size.
// The first two are the rx buffer and tx buffer memory ranges.
// The rx buffer is used by the Cartesi Server Manager to send the input and query payloads into the Cartesi
// Machine.
// The tx buffer is used by the Cartesi Machine to send Vouchers, Notices, and Reports to the Cartesi Server
// Manager.
// Each input to AdvanceState, in addition to the payload, requires some metadata.
// This is sent into the Cartesi Machine through the input metadata memory range.
// The last two remaining memory ranges are the voucher hashes and notice hashes memory ranges.
// These contain an array of hashes, respectively of each voucher and notice produced while processing a given input.
// Whenever an input is processed, the Cartesi Server Manager collects from the Cartesi Machine a Merkle proof
// that the voucher hashes and notice hashes memory ranges are part of the Cartesi Machine State.
// Likewise, each voucher and notice accompanies a proof that its hash is part of, respectively, the voucher hashes
// memory range and notice hashes memory range.
// Finally, the Cartesi Server Manager also maintains two additional Merkle trees, respectively containing as
// leaves the Merkle tree root hash of the voucher hashes memory range and the notice hashes memory range.
// These are the rx buffer and tx buffer memory ranges.
// The rx buffer is used by the Cartesi Server Manager to send the input and query data into the Cartesi Machine.
// The tx buffer is used by the Cartesi Machine to send Vouchers, Notices, and Reports to the Cartesi Server Manager.
// Each input data to AdvanceState includes the old payload and metadata, combined.
// As each input is processed, it may generate many outputs (which can be vouchers or notices) that are collected
// by the Cartesi Server Manager.
// Each output is backed by a proof that its hash is part of an "output Merkle tree" that collects all such hashes
// This root hash is maintained by the Cartesi Machine itself, and is returned when the machine is done
// processing the input
// The Cartesi Server Manager recreates this Merkle tree to check against the root hash returned by the Cartesi Machine,
// and so it can extract the proofs for each output
// Finally, the Cartesi Server Manager also maintains another Merkle tree, containing as leaves the Merkle tree root
// hash of output Merkle trees of all inputs in the epoch.

service ServerManager {
rpc GetVersion(CartesiMachine.Void) returns (Versioning.GetVersionResponse) {}
Expand Down Expand Up @@ -120,11 +117,10 @@ message CyclesConfig {
message StartSessionRequest {
string session_id = 1; // Id of session to start
string machine_directory = 2; // Machine to instantiate for session
uint64 active_epoch_index = 3; // Active epoch for the newly instantiated machine
uint64 processed_input_count = 4; // Number of processed inputs since genesis
CyclesConfig server_cycles = 5; // Cycle limit for server tasks
DeadlineConfig server_deadline = 6; // Time limit for server tasks
CartesiMachine.MachineRuntimeConfig runtime = 7; // Machine runtime parameters
uint64 processed_input_count = 3; // Number of processed inputs since genesis
CyclesConfig server_cycles = 4; // Cycle limit for server tasks
DeadlineConfig server_deadline = 5; // Time limit for server tasks
CartesiMachine.MachineRuntimeConfig runtime = 6; // Machine runtime parameters
}

// Information about why the session became invalid
Expand Down Expand Up @@ -152,20 +148,11 @@ message Address {
bytes data = 1; // 20-byte address
}

message InputMetadata {
Address msg_sender = 1; // 20-byte address of sender
uint64 block_number = 2; // Block number when input was posted
uint64 timestamp = 3; // Time stamp of the block (Unix?)
uint64 epoch_index = 4; // Deprecated. Always receives 0
uint64 input_index = 5; // Input index starting from genesis
}

message AdvanceStateRequest {
string session_id = 1;
uint64 active_epoch_index = 2; // To double-check whether the desired epoch is the active one
uint64 current_input_index = 3; // To double-check whether the current input is the expected one
InputMetadata input_metadata = 4; // Information sent via the input metadata memory range
bytes input_payload = 5; // Payload sent via the rx buffer memory range
bytes input_data = 4; // Input data sent via the rx buffer memory range (combines old input_payload and input_metadata)
}

message GetEpochStatusRequest {
Expand All @@ -184,9 +171,12 @@ enum CompletionStatus {
PAYLOAD_LENGTH_LIMIT_EXCEEDED = 6;
}

message Output {
bytes data = 1; // Contents of output
}

message AcceptedData {
repeated Voucher vouchers = 1; // List of vouchers produced when processing the input
repeated Notice notices = 2; // List of notices produced when processing the input
repeated Output output = 1;
}

message ProcessedInput {
Expand All @@ -196,7 +186,7 @@ message ProcessedInput {
AcceptedData accepted_data = 3; // Result of processed input when completed with success (ACCEPTED)
bytes exception_data = 4; // Exception payload when there was an EXCEPTION
}
repeated Report reports = 5; // Reports produced during input or query processing
repeated Output reports = 5; // Reports produced during input or query processing
Copy link
Copy Markdown

@mpolitzer mpolitzer Feb 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't these still reports?
Did we completely remove them?

}

enum EpochState {
Expand All @@ -213,19 +203,6 @@ message GetEpochStatusResponse {
TaintStatus taint_status = 6; // If the session is tainted, an error code and message giving the cause
}

message Notice {
bytes payload = 1; // Notice payload
}

message Voucher {
Address destination = 1; // 20-byte address
bytes payload = 2; // Voucher payload
}

message Report {
bytes payload = 1; // Report payload
}

message InspectStateRequest {
string session_id = 1; // Session to inspect
bytes query_payload = 2; // Query payload
Expand All @@ -237,33 +214,31 @@ message InspectStateResponse {
uint64 processed_input_count = 3; // Number of processed inputs since genesis
CompletionStatus status = 4; // Whether inspection completed or not (and why not)
optional bytes exception_data = 5; // Exception payload when finished with EXCEPTION
repeated Report reports = 6; // Reports produced while processing the query
repeated Output reports = 6; // Reports produced while processing the query
}

// Validity proof for an output
message OutputValidityProof {
uint64 input_index_within_epoch = 1; // Local input index within the context of the related epoch
uint64 output_index_within_input = 2; // Output index within the context of the input that produced it
CartesiMachine.Hash output_hashes_root_hash = 3; // Merkle root of all output hashes of the related input
CartesiMachine.Hash vouchers_epoch_root_hash = 4; // Merkle root of all voucher hashes of the related epoch
CartesiMachine.Hash notices_epoch_root_hash = 5; // Merkle root of all notice hashes of the related epoch
CartesiMachine.Hash machine_state_hash = 6; // Hash of the machine state claimed for the related epoch
repeated CartesiMachine.Hash output_hash_in_output_hashes_siblings = 7; // Proof that this output hash is in the output-hashes merkle tree. This array of siblings is bottom-up ordered (from the leaf to the root).
repeated CartesiMachine.Hash output_hashes_in_epoch_siblings = 8; // Proof that this output-hashes root hash is in epoch's output merkle tree. This array of siblings is bottom-up ordered (from the leaf to the root).
CartesiMachine.Hash outputs_epoch_root_hash = 4; // Merkle root of all outputs_epoch_root_hash for inputs in epoch
CartesiMachine.Hash machine_state_hash = 5; // Hash of the machine state claimed for the related epoch
repeated CartesiMachine.Hash output_hash_in_output_hashes_siblings = 6; // Proof that this output hash is in the output-hashes merkle tree. This array of siblings is bottom-up ordered (from the leaf to the root).
repeated CartesiMachine.Hash output_hashes_in_epoch_siblings = 7; // Proof that this output-hashes root hash is in epoch's output merkle tree. This array of siblings is bottom-up ordered (from the leaf to the root).
}

enum OutputEnum {
VOUCHER = 0;
NOTICE = 1;
message InputRange {
uint64 first_input = 1; // index of first input in range
uint64 last_input = 2; // index of last input in range
}

// Data that can be used as proof to validate notices and execute vouchers on the base layer blockchain
message Proof {
uint64 input_index = 1; // Index of input starting from genesis
uint64 output_index = 2; // Index of output (voucher or notice) in the context of the input
OutputEnum output_enum = 3; // Type of the output
OutputValidityProof validity = 4; // Validity proof for an output
bytes context = 5; // Data that allows the validity proof to be contextualized within submitted claims. Currently, the context is the epoch number as a ABI-encoded uint256.
OutputValidityProof validity = 3; // Validity proof for an output
Copy link
Copy Markdown

@guidanoli guidanoli Jan 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a proof for a specific output that has been mentioned above, so "the" output.

InputRange input_range = 4; // Range of processed in the epoch
Copy link
Copy Markdown

@guidanoli guidanoli Jan 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a noun is missing after "processed".

}

message FinishEpochRequest {
Expand All @@ -275,9 +250,8 @@ message FinishEpochRequest {

message FinishEpochResponse {
CartesiMachine.Hash machine_hash = 1; // Machine hash in epoch
CartesiMachine.Hash vouchers_epoch_root_hash = 2; // Root hash for Merkle tree of voucher hashes memory ranges
CartesiMachine.Hash notices_epoch_root_hash = 3; // Root hash for Merkle tree of notice hashes memory ranges
repeated Proof proofs = 4; // Proofs for the outputs
CartesiMachine.Hash outputs_epoch_root_hash = 2; // Merkle root of all outputs_epoch_root_hash for inputs in epoch
Comment thread
mpolitzer marked this conversation as resolved.
repeated Proof proofs = 3; // Proofs for the outputs
}

message DeleteEpochRequest {
Expand Down