iuna

iuna - experimental devnet protocol
git clone https://iuna.jhx.app/git/iuna.git
Log | Files | Refs | README | LICENSE

commit 0b7dfff14e34762c9eb852e27536d013283711d8
parent 033972c1f61b7bd629de7a2e76fe19075dc30641
Author: Joris Hartog <jorishartog@hotmail.com>
Date:   Wed,  5 Aug 2026 11:03:23 +0200

Add reveal bundle committee

Diffstat:
Mdocs/protocol.md | 28++++++++++++++++++++++------
Msrc/adapters/chain_store.rs | 130++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------
Msrc/adapters/http.rs | 198++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----------------
Msrc/adapters/p2p.rs | 65+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
Msrc/app.rs | 151++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----------
Msrc/domain.rs | 721+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------
Mtests/iuna.rs | 4++--
7 files changed, 1097 insertions(+), 200 deletions(-)

diff --git a/docs/protocol.md b/docs/protocol.md @@ -50,7 +50,7 @@ Every normal block must include at least one plaintext burn. A blinded transacti This mandatory burn is a liveness rule for the ticket pool, not a fairness rule for ticket distribution. It guarantees that normal block production keeps creating future tickets. Fairness against self-serving finalizers comes from blinded third-party burns. -Plaintext transaction fees go to the block finalizer immediately. Blinded transaction fees are paid when the payload is revealed and executed: half goes to the finalizer that originally committed the envelope, and the rest goes to the finalizer that includes the reveal. For odd micro-unit fees, the extra micro-unit goes to the reveal executor. +Plaintext transaction fees go to the block finalizer immediately. Blinded transaction fees are paid when the payload is revealed and executed. Half goes to the finalizer that originally committed the envelope. The other half is the executor share and depends on reveal-bundle participation: with `0`, `1`, `2`, or `3` included reveal bundles, the reveal-block finalizer receives `0/3`, `1/3`, `2/3`, or `3/3` of that executor share. Any missing executor share is burned. ## VDF Timing @@ -129,9 +129,23 @@ Normal mempool traffic uses blinded transaction content. A wallet encrypts a nor The finalizer can rank the envelope by fee per visible envelope byte, but cannot see whether the encrypted payload is a transfer or a burn before committing it to a block. -Reveal is a later step. A `BlindedReveal` carries only the commitment and decryption key. When a valid reveal is included, nodes decrypt the earlier payload, check the commitment and payload hash, decode the normal transaction, validate it against the current UTXO set, and execute it. If the decrypted transaction is a burn, it creates burn tickets at the reveal height, not the earlier envelope-commit height. +Reveal is a later step. A `BlindedReveal` carries only the commitment and decryption key. Reveals are not included as loose block items. They are carried in signed reveal bundles. -Fees are paid without inflating the reveal block reward. The decrypted transaction must pay the same fee declared by the blinded envelope. When it executes, the fee is split using deterministic fee outputs tied to the commitment: `floor(fee / 2)` to the envelope committer and `ceil(fee / 2)` to the reveal executor. If both roles are held by the same finalizer, that finalizer receives the full fee through the two deterministic outputs. +For each next block height, nodes compute a reveal committee from the burn leader ranking. The last three ranked eligible tickets form the three reveal-bundle slots. A committee member can sign one bundle for its slot, height, and parent hash. A bundle is at most `10,000` bytes and lists valid pending reveals ordered by visible fee rate. Empty bundles are not gossiped. + +A block has an envelope section and up to three reveal-bundle sections in fixed slot order. The envelope section contains the finalizer's plaintext burn, other plaintext block items, and blinded transaction envelopes. The bundle sections contain reveals selected by the committee members. + +A block may contain at most one bundle per slot. If a node sees two different signed bundles for the same height and slot before block assembly, it treats that slot as locally equivocated and does not use either bundle for that round. + +The block VDF seed is bound to the reveal bundle hashes: + +`seed = hash(parent hash || height || bundle_hash[0] || bundle_hash[1] || bundle_hash[2])` + +If a slot has no included bundle, it contributes a fixed default hash for that slot. This means the finalizer must choose the reveal-bundle set before doing the VDF work. A finalizer can still claim that a bundle arrived too late, but it cannot secretly swap or remove a timely bundle after computing the VDF without changing the seed. + +When a valid bundled reveal executes, nodes decrypt the earlier payload, check the commitment and payload hash, decode the normal transaction, validate it against the current UTXO set, and execute it once. If multiple committee bundles contain the same reveal, the reveal is still executed only once. If the decrypted transaction is a burn, it creates burn tickets at the reveal height, not the earlier envelope-commit height. + +Fees are paid without inflating the reveal block reward. The decrypted transaction must pay the same fee declared by the blinded envelope. `floor(fee / 2)` goes to the envelope committer. The reveal-block finalizer can receive up to the remaining executor share, scaled by the number of included reveal bundles. Missing executor share is burned instead of redistributed. Expiry is exclusive: a blinded envelope with expiry height `H` can be included only in blocks below height `H`, and revealed only while the current chain height is below `H`. The expiry height must be within `20` blocks of the node's current chain height when the envelope is accepted or selected. Expired envelopes and reveals are dropped from local selection. @@ -143,6 +157,7 @@ The P2P mempool gossips only: - blinded transaction envelopes; - blinded reveal keys; +- signed reveal bundles; - block inventory and blocks. It does not gossip plaintext transfers, burns, or mine actions. Wallet-created transfers, burns, and mine actions enter the network as blinded envelopes first, and are only decoded after a reveal. The one plaintext burn required for every normal block is produced locally by the finalizer and appears in the block itself. @@ -151,10 +166,11 @@ It does not gossip plaintext transfers, burns, or mine actions. Wallet-created t When a node builds a block, it selects transactions in this order: -1. Include valid blinded reveals first, so already committed encrypted payloads can execute. -2. Ensure the block has at least one plaintext burn from local block construction. +1. Collect valid signed reveal bundles for the next height. +2. Ensure the envelope has at least one plaintext burn from local block construction. 3. For recovery blocks, ensure at least one plaintext burn is from the recovery finalizer. -4. Fill remaining space with valid blinded transaction envelopes ordered by fee rate. +4. Fill remaining envelope space with valid blinded transaction envelopes ordered by fee rate. +5. Bind the VDF seed to the three reveal-bundle slot hashes, using default hashes for missing slots. Blocks are bounded by transaction count and serialized byte size. The devnet maximum block size is `100,000` bytes. diff --git a/src/adapters/chain_store.rs b/src/adapters/chain_store.rs @@ -11,7 +11,7 @@ use serde::Serialize; use crate::domain::{ Amount, BlindedReveal, BlindedTransaction, Block, ChainSnapshot, FinalizerMode, LaunchProfile, - LeaderProof, Ledger, MINE_REWARD, OutPoint, Transaction, TxInput, TxOutput, + LeaderProof, Ledger, MINE_REWARD, OutPoint, RevealBundle, Transaction, TxInput, TxOutput, revealed_blinded_transactions, }; @@ -299,7 +299,7 @@ fn clear_metrics_in_transaction(transaction: &rusqlite::Transaction<'_>) -> Resu } const COMPACT_SNAPSHOT_MAGIC: &[u8] = b"IUNA-SNAPSHOT"; -const COMPACT_SNAPSHOT_VERSION: u8 = 1; +const COMPACT_SNAPSHOT_VERSION: u8 = 2; fn encode_compact_snapshot(snapshot: &ChainSnapshot) -> Result<Vec<u8>> { let mut writer = CompactWriter::default(); @@ -410,9 +410,9 @@ fn encode_block_body(writer: &mut CompactWriter, block: &Block) -> Result<()> { for transaction in &block.blinded_transactions { encode_blinded_transaction(writer, transaction)?; } - writer.varint(block.blinded_reveals.len() as u64); - for reveal in &block.blinded_reveals { - encode_blinded_reveal(writer, reveal)?; + writer.varint(block.reveal_bundles.len() as u64); + for bundle in &block.reveal_bundles { + encode_reveal_bundle(writer, bundle)?; } writer.varint(block.transactions.len() as u64); for transaction in &block.transactions { @@ -448,7 +448,7 @@ fn decode_block_body( None }; let blinded_transactions = decode_vec(reader, decode_blinded_transaction)?; - let blinded_reveals = decode_vec(reader, decode_blinded_reveal)?; + let reveal_bundles = decode_vec(reader, decode_reveal_bundle)?; let transactions = decode_vec(reader, decode_transaction)?; let hash = reader.hex()?; Ok(Block { @@ -463,7 +463,7 @@ fn decode_block_body( vdf_output, leader_proof, blinded_transactions, - blinded_reveals, + reveal_bundles, transactions, hash, }) @@ -508,6 +508,30 @@ fn decode_blinded_reveal(reader: &mut CompactReader<'_>) -> Result<BlindedReveal }) } +fn encode_reveal_bundle(writer: &mut CompactWriter, bundle: &RevealBundle) -> Result<()> { + writer.varint(bundle.height); + writer.hex(&bundle.prev_hash)?; + writer.varint(u64::from(bundle.slot)); + writer.hex(&bundle.member)?; + writer.varint(bundle.reveals.len() as u64); + for reveal in &bundle.reveals { + encode_blinded_reveal(writer, reveal)?; + } + writer.hex(&bundle.signature)?; + Ok(()) +} + +fn decode_reveal_bundle(reader: &mut CompactReader<'_>) -> Result<RevealBundle> { + Ok(RevealBundle { + height: reader.varint()?, + prev_hash: reader.hex()?, + slot: u8::try_from(reader.varint()?).context("reveal bundle slot does not fit u8")?, + member: reader.hex()?, + reveals: decode_vec(reader, decode_blinded_reveal)?, + signature: reader.hex()?, + }) +} + fn encode_transaction(writer: &mut CompactWriter, transaction: &Transaction) -> Result<()> { match transaction { Transaction::Transfer { @@ -844,12 +868,9 @@ fn metrics_from_snapshot(snapshot: &ChainSnapshot) -> Result<Vec<BlockMetricRow> let ledger = Ledger::from_persisted_snapshot(snapshot.clone()) .context("failed to rebuild ledger for metrics")?; let revealed = revealed_blinded_transactions(snapshot)?.into_iter().fold( - std::collections::BTreeMap::<u64, Vec<Transaction>>::new(), + std::collections::BTreeMap::<u64, Vec<crate::domain::RevealedBlindedTransaction>>::new(), |mut by_height, revealed| { - by_height - .entry(revealed.height) - .or_default() - .push(revealed.transaction); + by_height.entry(revealed.height).or_default().push(revealed); by_height }, ); @@ -872,14 +893,11 @@ fn metrics_from_snapshot(snapshot: &ChainSnapshot) -> Result<Vec<BlockMetricRow> let mut burn_count = 0_u64; let mut mine_count = 0_u64; let mut burned_amount = 0_u64; + let mut burned_fee_amount = 0_u64; let mut mine_issued_amount = 0_u64; let mut fees_amount = 0_u64; - for transaction in block - .transactions - .iter() - .chain(revealed_transactions.iter()) - { + for transaction in &block.transactions { fees_amount = fees_amount .checked_add(transaction.fee()) .context("block metric fees overflow")?; @@ -900,8 +918,39 @@ fn metrics_from_snapshot(snapshot: &ChainSnapshot) -> Result<Vec<BlockMetricRow> } } } + for revealed in &revealed_transactions { + let transaction = &revealed.transaction; + fees_amount = fees_amount + .checked_add(transaction.fee()) + .context("block metric fees overflow")?; + let committer_fee = transaction.fee() / 2; + let executor_full_fee = transaction.fee() - committer_fee; + let executor_fee = executor_full_fee + .saturating_mul(block.included_reveal_bundle_count() as u64) + / crate::domain::REVEAL_COMMITTEE_SIZE as u64; + burned_fee_amount = burned_fee_amount + .checked_add(executor_full_fee.saturating_sub(executor_fee)) + .context("block metric burned fees overflow")?; + match transaction { + Transaction::Transfer { .. } => transfer_count += 1, + Transaction::Burn { amount, .. } => { + burn_count += 1; + burned_amount = burned_amount + .checked_add(*amount) + .context("block metric burns overflow")?; + } + Transaction::Mine { .. } => { + mine_count += 1; + mine_issued_amount = mine_issued_amount + .checked_add(MINE_REWARD) + .and_then(|amount| amount.checked_add(transaction.fee())) + .context("block metric mine issuance overflow")?; + } + } + } total_burned_amount = total_burned_amount .checked_add(burned_amount) + .and_then(|amount| amount.checked_add(burned_fee_amount)) .context("total burned metric overflows")?; let issued_amount = block_issued_amount(block.height, block.reward, mine_issued_amount) @@ -918,6 +967,7 @@ fn metrics_from_snapshot(snapshot: &ChainSnapshot) -> Result<Vec<BlockMetricRow> })?; circulating_supply = circulating_supply .checked_sub(burned_amount) + .and_then(|amount| amount.checked_sub(burned_fee_amount)) .with_context(|| { format!( "circulating supply metric underflows while subtracting burns at block {}", @@ -978,7 +1028,7 @@ mod tests { use tempfile::tempdir; - use crate::domain::{BLOCK_REWARD, GenesisBurn, Ledger, Wallet}; + use crate::domain::{BLOCK_REWARD, GenesisBurn, Ledger, Wallet, run_vdf}; use super::{ BlockMetricRow, SqliteChainStore, decode_compact_snapshot, encode_compact_snapshot, @@ -1035,7 +1085,7 @@ mod tests { .build_blinded_burn(&carol, 3, 7, ledger.height() + 4) .unwrap(); ledger - .submit_blinded_transaction(blinded.transaction) + .submit_blinded_transaction(blinded.transaction.clone()) .unwrap(); let leader = ledger.expected_leader_for_next_block().unwrap(); let wallet = wallets @@ -1045,6 +1095,10 @@ mod tests { let burn = ledger.build_burn(wallet, 1, 0).unwrap(); ledger.submit_transaction(burn).unwrap(); let block = ledger.mine_next_block(wallet, 1).unwrap(); + assert_eq!( + block.blinded_transactions, + vec![blinded.transaction.clone()] + ); ledger.apply_locally_mined_block(block).unwrap(); ledger.submit_blinded_reveal(blinded.reveal).unwrap(); let leader = ledger.expected_leader_for_next_block().unwrap(); @@ -1054,7 +1108,24 @@ mod tests { .unwrap(); let burn = ledger.build_burn(wallet, 1, 0).unwrap(); ledger.submit_transaction(burn).unwrap(); - let block = ledger.mine_next_block(wallet, 2).unwrap(); + let bundles = ledger + .reveal_committee_for_next_block() + .into_iter() + .filter_map(|member| { + let wallet = wallets + .iter() + .find(|wallet| wallet.address() == member.owner) + .unwrap(); + ledger.build_reveal_bundle(wallet).unwrap() + }) + .collect::<Vec<_>>(); + assert!(!bundles.is_empty()); + let prepared = ledger + .prepare_next_block_with_reveal_bundles(wallet.address(), 2, bundles) + .unwrap(); + let vdf_output = run_vdf(prepared.vdf_seed(), prepared.vdf_rounds()); + let block = prepared.finish(wallet, vdf_output); + assert_eq!(block.all_blinded_reveals().len(), 1); ledger.apply_locally_mined_block(block).unwrap(); let snapshot = ledger.snapshot(); @@ -1165,7 +1236,24 @@ mod tests { .unwrap(); let burn = ledger.build_burn(wallet, 1, 0).unwrap(); ledger.submit_transaction(burn).unwrap(); - let block = ledger.mine_next_block(wallet, 2).unwrap(); + let bundles = ledger + .reveal_committee_for_next_block() + .into_iter() + .filter_map(|member| { + let wallet = wallets + .iter() + .find(|wallet| wallet.address() == member.owner) + .unwrap(); + ledger.build_reveal_bundle(wallet).unwrap() + }) + .collect::<Vec<_>>(); + assert!(!bundles.is_empty()); + let prepared = ledger + .prepare_next_block_with_reveal_bundles(wallet.address(), 2, bundles) + .unwrap(); + let vdf_output = run_vdf(prepared.vdf_seed(), prepared.vdf_rounds()); + let block = prepared.finish(wallet, vdf_output); + assert_eq!(block.all_blinded_reveals().len(), 1); ledger.apply_locally_mined_block(block).unwrap(); store.save_with_metrics(&ledger.snapshot(), true).unwrap(); diff --git a/src/adapters/http.rs b/src/adapters/http.rs @@ -34,8 +34,8 @@ use crate::{ }, domain::{ Amount, BlindedReveal, BlindedTransaction, Block, BurnLeaderRank, ChainSnapshot, Ledger, - MINE_FINALIZER_FEE, MINE_REWARD, OutPoint, Transaction, TxInput, TxOutput, Wallet, - hex_hash, revealed_blinded_transactions, + MINE_FINALIZER_FEE, MINE_REWARD, OutPoint, RevealedBlindedTransaction, Transaction, + TxInput, TxOutput, Wallet, hex_hash, revealed_blinded_transactions, }, }; @@ -370,10 +370,21 @@ struct UiBlock { burn_leader_ranks: Vec<BurnLeaderRank>, transactions: Vec<UiTransaction>, revealed_transactions: Vec<UiTransaction>, + reveal_bundles: Vec<UiRevealBundle>, hash: String, } #[derive(Clone, Debug, Eq, PartialEq, Serialize)] +#[serde(rename_all = "camelCase")] +struct UiRevealBundle { + slot: u8, + member: String, + hash: String, + byte_size: usize, + reveals: Vec<UiTransaction>, +} + +#[derive(Clone, Debug, Eq, PartialEq, Serialize)] struct UiTransaction { kind: &'static str, from: String, @@ -1595,7 +1606,7 @@ fn wallet_transaction_rows( pending: Vec<Transaction>, owned_blinded: Vec<Transaction>, chain: &[Block], - revealed_by_height: &BTreeMap<u64, Vec<Transaction>>, + revealed_by_height: &BTreeMap<u64, Vec<RevealedBlindedTransaction>>, outputs: &BTreeMap<OutPoint, TxOutput>, filters: WalletTransactionFilters, ) -> Vec<WalletTransactionRow> { @@ -1639,7 +1650,8 @@ fn wallet_transaction_rows( } } if let Some(revealed_transactions) = revealed_by_height.get(&block.height) { - for (index, tx) in revealed_transactions.iter().rev().enumerate() { + for (index, revealed) in revealed_transactions.iter().rev().enumerate() { + let tx = &revealed.transaction; if !filters.allows(tx) { continue; } @@ -1756,17 +1768,16 @@ fn wallet_transaction_row( } } -fn revealed_transactions_by_height(snapshot: &ChainSnapshot) -> BTreeMap<u64, Vec<Transaction>> { +fn revealed_transactions_by_height( + snapshot: &ChainSnapshot, +) -> BTreeMap<u64, Vec<RevealedBlindedTransaction>> { revealed_blinded_transactions(snapshot) .unwrap_or_default() .into_iter() .fold( - BTreeMap::<u64, Vec<Transaction>>::new(), + BTreeMap::<u64, Vec<RevealedBlindedTransaction>>::new(), |mut by_height, revealed| { - by_height - .entry(revealed.height) - .or_default() - .push(revealed.transaction); + by_height.entry(revealed.height).or_default().push(revealed); by_height }, ) @@ -1793,15 +1804,15 @@ fn ui_block( block: Block, outputs: &BTreeMap<OutPoint, TxOutput>, burn_leader_ranks: &BTreeMap<String, Vec<BurnLeaderRank>>, - revealed_transactions: &[Transaction], + revealed_transactions: &[RevealedBlindedTransaction], ) -> UiBlock { let ranks = burn_leader_ranks .get(&block.hash) .cloned() .unwrap_or_default(); - let revealed_fees = revealed_transactions - .iter() - .fold(0_u64, |total, tx| total.saturating_add(tx.fee())); + let revealed_fees = revealed_transactions.iter().fold(0_u64, |total, revealed| { + total.saturating_add(revealed.transaction.fee()) + }); let mut transactions = block .transactions .iter() @@ -1816,8 +1827,32 @@ fn ui_block( transactions.extend( revealed_transactions .iter() - .map(|tx| ui_revealed_transaction(tx, outputs)), + .map(|revealed| ui_revealed_transaction(&revealed.transaction, outputs)), ); + let revealed_by_commitment = revealed_transactions + .iter() + .map(|revealed| (revealed.commitment.clone(), revealed.transaction.clone())) + .collect::<BTreeMap<_, _>>(); + let reveal_bundles = block + .reveal_bundles + .iter() + .map(|bundle| UiRevealBundle { + slot: bundle.slot, + member: bundle.member.clone(), + hash: bundle.bundle_hash(), + byte_size: bundle.serialized_size_bytes().unwrap_or_default(), + reveals: bundle + .reveals + .iter() + .map(|reveal| { + revealed_by_commitment + .get(&reveal.commitment) + .map(|tx| ui_revealed_transaction(tx, outputs)) + .unwrap_or_else(|| ui_blinded_reveal(reveal)) + }) + .collect(), + }) + .collect(); UiBlock { height: block.height, prev_hash: block.prev_hash, @@ -1834,8 +1869,9 @@ fn ui_block( transactions, revealed_transactions: revealed_transactions .iter() - .map(|tx| ui_revealed_transaction(tx, outputs)) + .map(|revealed| ui_revealed_transaction(&revealed.transaction, outputs)) .collect(), + reveal_bundles, hash: block.hash, } } @@ -2034,6 +2070,11 @@ fn known_output_index( ); } let revealed = revealed_blinded_transactions(snapshot).unwrap_or_default(); + let blocks_by_height = snapshot + .blocks + .iter() + .map(|block| (block.height, block)) + .collect::<BTreeMap<_, _>>(); for block in &snapshot.blocks { for transaction in &block.transactions { index_transaction_outputs(&mut outputs, transaction); @@ -2050,14 +2091,33 @@ fn known_output_index( } for revealed in revealed { index_transaction_outputs(&mut outputs, &revealed.transaction); - if revealed.transaction.fee() > 0 { - outputs.insert( - blinded_fee_outpoint(&revealed.commitment), - TxOutput { - address: revealed.included_by, - amount: revealed.transaction.fee(), - }, - ); + let fee = revealed.transaction.fee(); + if fee > 0 { + let committer_fee = fee / 2; + if committer_fee > 0 { + outputs.insert( + blinded_committer_fee_outpoint(&revealed.commitment), + TxOutput { + address: revealed.included_by, + amount: committer_fee, + }, + ); + } + if let Some(block) = blocks_by_height.get(&revealed.height) { + let executor_full_fee = fee - committer_fee; + let executor_fee = executor_full_fee + .saturating_mul(block.included_reveal_bundle_count() as u64) + / crate::domain::REVEAL_COMMITTEE_SIZE as u64; + if executor_fee > 0 { + outputs.insert( + blinded_executor_fee_outpoint(&revealed.commitment), + TxOutput { + address: block.miner.clone(), + amount: executor_fee, + }, + ); + } + } } } for transaction in pending { @@ -2103,13 +2163,20 @@ fn reward_outpoint(block_hash: &str) -> OutPoint { } } -fn blinded_fee_outpoint(commitment: &str) -> OutPoint { +fn blinded_committer_fee_outpoint(commitment: &str) -> OutPoint { OutPoint { txid: commitment.to_string(), index: u32::MAX - 1, } } +fn blinded_executor_fee_outpoint(commitment: &str) -> OutPoint { + OutPoint { + txid: commitment.to_string(), + index: u32::MAX - 2, + } +} + async fn replace_setup_wallet_with_generated_seed( state: &HttpState, headers: &HeaderMap, @@ -3023,6 +3090,10 @@ const INDEX_HTML: &str = r#"<!doctype html> .rank-number { color: #d7f2ff; font-weight: 700; } .rank-details { display: grid; gap: 6px; min-width: 0; } .tx-list { display: grid; gap: 8px; } + .tx-section { display: grid; gap: 8px; } + .tx-section + .tx-section { margin-top: 10px; padding-top: 10px; border-top: 1px solid #2f363c; } + .tx-section-title { display: flex; align-items: center; justify-content: space-between; gap: 8px; color: #f4f7f8; font-size: 13px; font-weight: 800; } + .tx-section-meta { color: #8e979e; font-size: 12px; font-weight: 600; } .tx-card, .mempool-item { position: relative; display: grid; gap: 6px; border: 1px solid #2f363c; border-radius: 8px; padding: 12px; background: #111316; cursor: pointer; text-align: left; } .tx-card .pill, .mempool-item .pill { position: absolute; top: 10px; right: 10px; } .pill { display: inline-flex; align-items: center; border-radius: 999px; padding: 2px 8px; font-size: 12px; font-weight: 800; background: #2b3136; color: #d6dee2; } @@ -3496,22 +3567,43 @@ const INDEX_HTML: &str = r#"<!doctype html> </div> <div class="tx-list"> <h3>Transactions</h3> - <template x-for="tx in selectedBlock.transactions" :key="tx.signature"> - <div class="tx-card" role="button" tabindex="0" @click="openTransactionModal(tx, { source: 'Block', blockHeight: selectedBlock.height, blockFinalizer: selectedBlock.miner })" @keydown.enter.prevent="openTransactionModal(tx, { source: 'Block', blockHeight: selectedBlock.height, blockFinalizer: selectedBlock.miner })" @keydown.space.prevent="openTransactionModal(tx, { source: 'Block', blockHeight: selectedBlock.height, blockFinalizer: selectedBlock.miner })"> - <span class="pill" :class="txPillClass(tx)" x-text="txPillLabel(tx)"></span> - <div class="tx-field" x-show="!isBlindedMempoolItem(tx)"><span class="tx-label">Amount</span><span class="tx-value money">IUNA <span x-text="amountLabel(txAmount(tx))"></span></span></div> - <div class="tx-field"><span class="tx-label">Fee</span><span class="tx-value money">IUNA <span x-text="amountLabel(tx.fee ?? 0)"></span></span></div> - <div class="tx-field" x-show="!isBlindedMempoolItem(tx)"><span class="tx-label">From</span><code class="tx-value hash" x-text="short(txFrom(tx))"></code></div> - <div class="tx-field" x-show="txTo(tx)"><span class="tx-label">To</span><code class="tx-value hash" x-text="short(txTo(tx))"></code></div> - <div class="tx-field" x-show="isBlindedMempoolItem(tx)"><span class="tx-label">Commitment</span><code class="tx-value hash" x-text="short(tx.commitment || tx.signature)"></code></div> - <div class="tx-field" x-show="tx.encrypted_size || tx.encryptedSize"><span class="tx-label">Bytes</span><span class="tx-value number" x-text="tx.encrypted_size || tx.encryptedSize"></span></div> - <div class="tx-field" x-show="tx.expires_at_height || tx.expiresAtHeight"><span class="tx-label">Expires</span><span class="tx-value number" x-text="tx.expires_at_height || tx.expiresAtHeight"></span></div> - <div class="tx-field" x-show="isMineTx(tx)"><span class="tx-label">Proof Bits</span><span class="tx-value number"><span x-text="txProofBits(tx) ?? '-'"></span> / <span x-text="txDifficultyBits(tx) ?? '-'"></span></span></div> - <div class="tx-field" x-show="isMineTx(tx)"><span class="tx-label">Proof Hash</span><code class="tx-value hash" x-text="short(txProofHash(tx))"></code></div> - <div class="tx-field" x-show="!isBlindedMempoolItem(tx)"><span class="tx-label">Signature</span><code class="tx-value hash" x-text="short(tx.signature)"></code></div> + <div class="tx-section"> + <div class="tx-section-title"><span>Envelope</span><span class="tx-section-meta" x-text="short(selectedBlock.miner)"></span></div> + <template x-for="tx in selectedBlock.transactions" :key="tx.signature"> + <div class="tx-card" role="button" tabindex="0" @click="openTransactionModal(tx, { source: 'Envelope', blockHeight: selectedBlock.height, blockFinalizer: selectedBlock.miner })" @keydown.enter.prevent="openTransactionModal(tx, { source: 'Envelope', blockHeight: selectedBlock.height, blockFinalizer: selectedBlock.miner })" @keydown.space.prevent="openTransactionModal(tx, { source: 'Envelope', blockHeight: selectedBlock.height, blockFinalizer: selectedBlock.miner })"> + <span class="pill" :class="txPillClass(tx)" x-text="txPillLabel(tx)"></span> + <div class="tx-field" x-show="!isBlindedMempoolItem(tx)"><span class="tx-label">Amount</span><span class="tx-value money">IUNA <span x-text="amountLabel(txAmount(tx))"></span></span></div> + <div class="tx-field"><span class="tx-label">Fee</span><span class="tx-value money">IUNA <span x-text="amountLabel(tx.fee ?? 0)"></span></span></div> + <div class="tx-field" x-show="!isBlindedMempoolItem(tx)"><span class="tx-label">From</span><code class="tx-value hash" x-text="short(txFrom(tx))"></code></div> + <div class="tx-field" x-show="txTo(tx)"><span class="tx-label">To</span><code class="tx-value hash" x-text="short(txTo(tx))"></code></div> + <div class="tx-field" x-show="isBlindedMempoolItem(tx)"><span class="tx-label">Commitment</span><code class="tx-value hash" x-text="short(tx.commitment || tx.signature)"></code></div> + <div class="tx-field" x-show="tx.encrypted_size || tx.encryptedSize"><span class="tx-label">Bytes</span><span class="tx-value number" x-text="tx.encrypted_size || tx.encryptedSize"></span></div> + <div class="tx-field" x-show="tx.expires_at_height || tx.expiresAtHeight"><span class="tx-label">Expires</span><span class="tx-value number" x-text="tx.expires_at_height || tx.expiresAtHeight"></span></div> + <div class="tx-field" x-show="isMineTx(tx)"><span class="tx-label">Proof Bits</span><span class="tx-value number"><span x-text="txProofBits(tx) ?? '-'"></span> / <span x-text="txDifficultyBits(tx) ?? '-'"></span></span></div> + <div class="tx-field" x-show="isMineTx(tx)"><span class="tx-label">Proof Hash</span><code class="tx-value hash" x-text="short(txProofHash(tx))"></code></div> + <div class="tx-field" x-show="!isBlindedMempoolItem(tx)"><span class="tx-label">Signature</span><code class="tx-value hash" x-text="short(tx.signature)"></code></div> + </div> + </template> + <div class="muted" x-show="selectedBlock.transactions.length === 0">No envelope transactions</div> + </div> + <template x-for="bundle in selectedBlock.reveal_bundles || selectedBlock.revealBundles || []" :key="bundle.hash"> + <div class="tx-section"> + <div class="tx-section-title"><span x-text="`Reveal bundle ${bundle.slot}`"></span><span class="tx-section-meta"><span x-text="short(bundle.member)"></span> ยท <span x-text="bundle.byte_size || bundle.byteSize || 0"></span>B</span></div> + <template x-for="tx in bundle.reveals" :key="tx.signature"> + <div class="tx-card" role="button" tabindex="0" @click="openTransactionModal(tx, { source: 'Reveal bundle', blockHeight: selectedBlock.height, blockFinalizer: bundle.member })" @keydown.enter.prevent="openTransactionModal(tx, { source: 'Reveal bundle', blockHeight: selectedBlock.height, blockFinalizer: bundle.member })" @keydown.space.prevent="openTransactionModal(tx, { source: 'Reveal bundle', blockHeight: selectedBlock.height, blockFinalizer: bundle.member })"> + <span class="pill" :class="txPillClass(tx)" x-text="txPillLabel(tx)"></span> + <div class="tx-field" x-show="!isBlindedMempoolItem(tx)"><span class="tx-label">Amount</span><span class="tx-value money">IUNA <span x-text="amountLabel(txAmount(tx))"></span></span></div> + <div class="tx-field"><span class="tx-label">Fee</span><span class="tx-value money">IUNA <span x-text="amountLabel(tx.fee ?? 0)"></span></span></div> + <div class="tx-field" x-show="!isBlindedMempoolItem(tx)"><span class="tx-label">From</span><code class="tx-value hash" x-text="short(txFrom(tx))"></code></div> + <div class="tx-field" x-show="txTo(tx)"><span class="tx-label">To</span><code class="tx-value hash" x-text="short(txTo(tx))"></code></div> + <div class="tx-field" x-show="isBlindedMempoolItem(tx)"><span class="tx-label">Commitment</span><code class="tx-value hash" x-text="short(tx.commitment || tx.signature)"></code></div> + <div class="tx-field" x-show="!isBlindedMempoolItem(tx)"><span class="tx-label">Signature</span><code class="tx-value hash" x-text="short(tx.signature)"></code></div> + </div> + </template> + <div class="muted" x-show="bundle.reveals.length === 0">No reveals in bundle</div> </div> </template> - <div class="muted" x-show="selectedBlock.transactions.length === 0">No transactions</div> + <div class="muted" x-show="selectedBlock.transactions.length === 0 && !(selectedBlock.reveal_bundles || selectedBlock.revealBundles || []).length">No transactions</div> </div> </div> </template> @@ -4017,7 +4109,13 @@ mod tests { let mut block = fake_block(7, Vec::new()); block.blinded_transactions = vec![built.transaction.clone()]; - let ui_block = super::ui_block(block, &BTreeMap::new(), &BTreeMap::new(), &[transfer]); + let revealed = crate::domain::RevealedBlindedTransaction { + height: 7, + commitment: built.transaction.commitment.clone(), + included_by: "miner".to_string(), + transaction: transfer, + }; + let ui_block = super::ui_block(block, &BTreeMap::new(), &BTreeMap::new(), &[revealed]); assert_eq!(ui_block.transactions.len(), 2); assert_eq!(ui_block.transactions[0].kind, "blinded"); @@ -4047,7 +4145,14 @@ mod tests { let mut commit_block = fake_block(7, Vec::new()); commit_block.blinded_transactions = vec![built.transaction.clone()]; let mut reveal_block = fake_block(8, Vec::new()); - reveal_block.blinded_reveals = vec![built.reveal.clone()]; + reveal_block.reveal_bundles = vec![crate::domain::RevealBundle { + height: reveal_block.height, + prev_hash: reveal_block.prev_hash.clone(), + slot: 0, + member: reveal_block.miner.clone(), + reveals: vec![built.reveal.clone()], + signature: "11".repeat(64), + }]; let snapshot = fake_snapshot( allocations, vec![commit_block.clone(), reveal_block.clone()], @@ -4904,7 +5009,14 @@ mod tests { let mut commit_block = fake_block(7, Vec::new()); commit_block.blinded_transactions = vec![built.transaction]; let mut reveal_block = fake_block(8, Vec::new()); - reveal_block.blinded_reveals = vec![built.reveal]; + reveal_block.reveal_bundles = vec![crate::domain::RevealBundle { + height: reveal_block.height, + prev_hash: reveal_block.prev_hash.clone(), + slot: 0, + member: reveal_block.miner.clone(), + reveals: vec![built.reveal], + signature: "11".repeat(64), + }]; let chain = vec![commit_block.clone(), reveal_block.clone()]; let snapshot = fake_snapshot(BTreeMap::new(), chain.clone()); let revealed_by_height = super::revealed_transactions_by_height(&snapshot); @@ -5058,7 +5170,7 @@ mod tests { vdf_output: "vdf".to_string(), leader_proof: None, blinded_transactions: Vec::new(), - blinded_reveals: Vec::new(), + reveal_bundles: Vec::new(), transactions, hash: format!("hash-{height}"), } diff --git a/src/adapters/p2p.rs b/src/adapters/p2p.rs @@ -1043,6 +1043,12 @@ async fn process_envelope( GossipEnvelope::BlindedReveals { reveals } => { process_blinded_reveals(network, remote_addr, known_peer, reveals).await; } + GossipEnvelope::RevealBundle(bundle) => { + process_reveal_bundles(network, remote_addr, known_peer, vec![bundle]).await; + } + GossipEnvelope::RevealBundles { bundles } => { + process_reveal_bundles(network, remote_addr, known_peer, bundles).await; + } GossipEnvelope::Block(block) => { let adjusted_time_ms = network_adjusted_time_ms(network).await; let needs_vdf = { @@ -1168,6 +1174,34 @@ async fn process_blinded_reveals( network.forward_outbox().await; } +async fn process_reveal_bundles( + network: &GossipNetwork, + remote_addr: SocketAddr, + known_peer: &Option<String>, + bundles: Vec<crate::domain::RevealBundle>, +) { + let first_error = { + let mut node = network.inner.node.lock().await; + let mut first_error = None; + for bundle in bundles { + if let Err(error) = node.receive_reveal_bundle(bundle) { + first_error.get_or_insert(error); + } + } + first_error + }; + record_inbound_result( + network, + known_peer, + remote_addr, + first_error + .map(|error| Err(anyhow!(format!("{error:#}")))) + .unwrap_or(Ok(())), + ) + .await; + network.forward_outbox().await; +} + async fn maybe_request_catchup( network: &GossipNetwork, writer: &mut OwnedWriteHalf, @@ -1223,7 +1257,7 @@ async fn catchup_payload_for_peer( node: &SharedNode, peer_status: &PeerStatus, ) -> Vec<GossipEnvelope> { - let node = node.lock().await; + let mut node = node.lock().await; let local_status = node.ledger().status(); if node.ledger().is_setup_placeholder() { return Vec::new(); @@ -1420,6 +1454,25 @@ fn record_received_envelope_kind(metrics: &P2pMetricsCounters, envelope: &Gossip P2pMetricsCounters::inc(&metrics.blinded_reveal_envelopes_received); P2pMetricsCounters::add(&metrics.blinded_reveals_received, reveals.len() as u64); } + GossipEnvelope::RevealBundle(bundle) => { + P2pMetricsCounters::inc(&metrics.data_envelopes_received); + P2pMetricsCounters::inc(&metrics.blinded_reveal_envelopes_received); + P2pMetricsCounters::add( + &metrics.blinded_reveals_received, + bundle.reveals.len() as u64, + ); + } + GossipEnvelope::RevealBundles { bundles } => { + P2pMetricsCounters::inc(&metrics.data_envelopes_received); + P2pMetricsCounters::inc(&metrics.blinded_reveal_envelopes_received); + P2pMetricsCounters::add( + &metrics.blinded_reveals_received, + bundles + .iter() + .map(|bundle| bundle.reveals.len() as u64) + .sum::<u64>(), + ); + } GossipEnvelope::Block(_) | GossipEnvelope::Blocks { .. } | GossipEnvelope::ChainSnapshot(_) => { @@ -1471,6 +1524,13 @@ fn validate_envelope_limits(envelope: &GossipEnvelope) -> Result<()> { TRANSACTION_BATCH_LIMIT, )?; } + GossipEnvelope::RevealBundles { bundles } => { + ensure_len( + "reveal bundle batch", + bundles.len(), + TRANSACTION_BATCH_LIMIT, + )?; + } GossipEnvelope::Blocks { blocks } => { ensure_len("block batch", blocks.len(), MAX_BLOCK_BATCH)?; } @@ -1485,6 +1545,7 @@ fn validate_envelope_limits(envelope: &GossipEnvelope) -> Result<()> { | GossipEnvelope::PeerStatus { .. } | GossipEnvelope::BlindedTransaction(_) | GossipEnvelope::BlindedReveal(_) + | GossipEnvelope::RevealBundle(_) | GossipEnvelope::Block(_) | GossipEnvelope::PeerAnnouncement { .. } | GossipEnvelope::PeerVerificationChallenge { .. } @@ -2693,7 +2754,7 @@ mod tests { vdf_output: "vdf".to_string(), leader_proof: None, blinded_transactions: Vec::new(), - blinded_reveals: Vec::new(), + reveal_bundles: Vec::new(), transactions: Vec::new(), hash: "hash".to_string(), }; diff --git a/src/app.rs b/src/app.rs @@ -1,5 +1,5 @@ use std::{ - collections::BTreeMap, + collections::{BTreeMap, BTreeSet}, sync::{ Arc, atomic::{AtomicBool, Ordering}, @@ -16,8 +16,8 @@ use crate::domain::{ Amount, BlindedReveal, BlindedTransaction, Block, BuiltBlindedTransaction, BurnLeaderRank, ChainSnapshot, ChainStatus, DEFAULT_FEE_PER_BYTE, DEFAULT_TRANSACTION_FEE, Ledger, MAX_BLINDED_TRANSACTION_EXPIRY_HEIGHTS, MINE_FINALIZER_FEE, OutPoint, OwnedBlindedTransaction, - PreparedBlock, StratumMineShare, StratumMineTemplate, Transaction, TransactionSubmitOutcome, - VDF_TARGET_BLOCK_MS, Wallet, run_vdf, + PreparedBlock, RevealBundle, StratumMineShare, StratumMineTemplate, Transaction, + TransactionSubmitOutcome, VDF_TARGET_BLOCK_MS, Wallet, run_vdf, }; pub type SharedNode = Arc<Mutex<NodeCore>>; @@ -26,7 +26,7 @@ pub type SharedPeerBook = Arc<Mutex<PeerBook>>; pub const DEFAULT_BURN_PER_BLOCK: Amount = 0; pub const DEFAULT_VDF_ROUNDS: u32 = 67_000_000; pub const PROTOCOL_VERSION: u32 = 1; -pub const NETWORK_ID: &str = "iuna-devnet-v2"; +pub const NETWORK_ID: &str = "iuna-devnet-v3"; pub const BLOCK_REQUEST_LIMIT: usize = 128; pub const TRANSACTION_BATCH_LIMIT: usize = 128; const IMPORT_REBROADCAST_LIMIT: usize = 128; @@ -121,6 +121,10 @@ pub enum GossipEnvelope { BlindedReveals { reveals: Vec<BlindedReveal>, }, + RevealBundle(RevealBundle), + RevealBundles { + bundles: Vec<RevealBundle>, + }, Block(Block), Blocks { blocks: Vec<Block>, @@ -249,6 +253,8 @@ pub struct NodeCore { owned_blinded_reveals: BTreeMap<String, BlindedReveal>, owned_blinded_payloads: BTreeMap<String, Transaction>, owned_blinded_outbox_version: u64, + reveal_bundles: BTreeMap<(u64, u8), RevealBundle>, + equivocated_reveal_bundle_slots: BTreeSet<(u64, u8)>, local_block_anchor_burn: Option<(u64, Transaction)>, outbox: Vec<GossipEnvelope>, } @@ -339,6 +345,8 @@ impl NodeCore { owned_blinded_reveals: BTreeMap::new(), owned_blinded_payloads: BTreeMap::new(), owned_blinded_outbox_version: 0, + reveal_bundles: BTreeMap::new(), + equivocated_reveal_bundle_slots: BTreeSet::new(), local_block_anchor_burn: None, outbox: Vec::new(), } @@ -362,6 +370,8 @@ impl NodeCore { self.owned_blinded_reveals.clear(); self.owned_blinded_payloads.clear(); self.bump_owned_blinded_outbox_version(); + self.reveal_bundles.clear(); + self.equivocated_reveal_bundle_slots.clear(); self.local_block_anchor_burn = None; } @@ -484,7 +494,8 @@ impl NodeCore { Ok(()) } - pub fn mempool_gossip(&self) -> Vec<GossipEnvelope> { + pub fn mempool_gossip(&mut self) -> Vec<GossipEnvelope> { + let _ = self.publish_reveal_bundle_for_next_block(); let mut gossip = Vec::new(); gossip.extend( self.ledger @@ -502,6 +513,13 @@ impl NodeCore { reveals: chunk.to_vec(), }), ); + gossip.extend( + self.usable_reveal_bundles() + .chunks(TRANSACTION_BATCH_LIMIT) + .map(|chunk| GossipEnvelope::RevealBundles { + bundles: chunk.to_vec(), + }), + ); gossip } @@ -839,6 +857,78 @@ impl NodeCore { Ok(()) } + pub fn receive_reveal_bundle(&mut self, bundle: RevealBundle) -> Result<()> { + let next_height = self.ledger.height().saturating_add(1); + if bundle.height <= self.ledger.height() { + return Ok(()); + } + if bundle.height > next_height { + return Ok(()); + } + let key = (bundle.height, bundle.slot); + if self.equivocated_reveal_bundle_slots.contains(&key) { + return Ok(()); + } + if let Some(existing) = self.reveal_bundles.get(&key) { + if existing.canonical() != bundle.canonical() { + self.reveal_bundles.remove(&key); + self.equivocated_reveal_bundle_slots.insert(key); + } + return Ok(()); + } + self.ledger + .validate_next_block_reveal_bundles(vec![bundle.clone()])?; + self.reveal_bundles.insert(key, bundle.clone()); + self.outbox.push(GossipEnvelope::RevealBundle(bundle)); + Ok(()) + } + + fn usable_reveal_bundles(&self) -> Vec<RevealBundle> { + let next_height = self.ledger.height().saturating_add(1); + let mut bundles = self + .reveal_bundles + .iter() + .filter(|((height, slot), _)| { + *height == next_height + && !self + .equivocated_reveal_bundle_slots + .contains(&(*height, *slot)) + }) + .map(|(_, bundle)| bundle.clone()) + .collect::<Vec<_>>(); + bundles.sort_by_key(|bundle| bundle.slot); + bundles + } + + fn prune_reveal_bundles(&mut self) { + let height = self.ledger.height(); + self.reveal_bundles + .retain(|(bundle_height, _), _| *bundle_height > height); + self.equivocated_reveal_bundle_slots + .retain(|(bundle_height, _)| *bundle_height > height); + } + + fn publish_reveal_bundle_for_next_block(&mut self) -> Result<()> { + let wallet = match &self.wallet { + NodeWallet::Unlocked(wallet) => wallet, + NodeWallet::Locked { .. } => return Ok(()), + }; + let Some(bundle) = self.ledger.build_reveal_bundle(wallet)? else { + return Ok(()); + }; + let key = (bundle.height, bundle.slot); + if self.equivocated_reveal_bundle_slots.contains(&key) + || self.reveal_bundles.contains_key(&key) + { + return Ok(()); + } + self.ledger + .validate_next_block_reveal_bundles(vec![bundle.clone()])?; + self.reveal_bundles.insert(key, bundle.clone()); + self.outbox.push(GossipEnvelope::RevealBundle(bundle)); + Ok(()) + } + fn submit_owned_blinded_transaction( &mut self, built: BuiltBlindedTransaction, @@ -908,7 +998,7 @@ impl NodeCore { let before = self.owned_blinded_transactions.len() + self.owned_blinded_reveals.len() + self.owned_blinded_payloads.len(); - for reveal in &block.blinded_reveals { + for reveal in block.all_blinded_reveals() { self.owned_blinded_transactions.remove(&reveal.commitment); self.owned_blinded_reveals.remove(&reveal.commitment); self.owned_blinded_payloads.remove(&reveal.commitment); @@ -1101,15 +1191,17 @@ impl NodeCore { } } + if let Err(error) = self.publish_reveal_bundle_for_next_block() { + plan.skipped_reason = Some(format!("{error:#}")); + return plan; + } + let wallet_rank = self .ledger .finalizer_rank_for_next_block(self.wallet.address()); if wallet_rank.is_none() { if self.ledger.recovery_block_available_at(timestamp_ms) { - match self - .ledger - .prepare_recovery_block(self.wallet.address(), timestamp_ms) - { + match self.prepare_recovery_block_with_local_anchor(timestamp_ms) { Ok(work) => { plan.work = Some(work); } @@ -1342,12 +1434,20 @@ impl NodeCore { fn prepare_next_block_with_local_anchor(&self, timestamp_ms: u64) -> Result<PreparedBlock> { self.ledger_with_local_block_anchor()? - .prepare_next_block(self.wallet.address(), timestamp_ms) + .prepare_next_block_with_reveal_bundles( + self.wallet.address(), + timestamp_ms, + self.usable_reveal_bundles(), + ) } fn prepare_recovery_block_with_local_anchor(&self, timestamp_ms: u64) -> Result<PreparedBlock> { self.ledger_with_local_block_anchor()? - .prepare_recovery_block(self.wallet.address(), timestamp_ms) + .prepare_recovery_block_with_reveal_bundles( + self.wallet.address(), + timestamp_ms, + self.usable_reveal_bundles(), + ) } fn ledger_with_local_block_anchor(&self) -> Result<Ledger> { @@ -1372,15 +1472,10 @@ impl NodeCore { } pub fn mine_one_at(&mut self, timestamp_ms: u64) -> Result<Block> { - let block = self - .ledger - .mine_next_block(self.wallet.unlocked()?, timestamp_ms)?; - self.ledger.apply_locally_mined_block(block.clone())?; - self.clear_stale_local_block_anchor(); - self.prune_owned_blinded_payloads_for_block(&block); - self.outbox.push(GossipEnvelope::Block(block.clone())); - self.publish_owned_reveals_for_block(&block)?; - Ok(block) + self.publish_reveal_bundle_for_next_block()?; + let work = self.prepare_next_block_with_local_anchor(timestamp_ms)?; + let vdf_output = run_vdf(work.vdf_seed(), work.vdf_rounds()); + self.complete_prepared_block_at(work, vdf_output, timestamp_ms) } pub fn complete_prepared_block( @@ -1400,6 +1495,7 @@ impl NodeCore { let block = work.finish_at(self.wallet.unlocked()?, vdf_output, timestamp_ms); self.ledger.apply_locally_mined_block(block.clone())?; self.clear_stale_local_block_anchor(); + self.prune_reveal_bundles(); self.prune_owned_blinded_payloads_for_block(&block); self.outbox.push(GossipEnvelope::Block(block.clone())); self.publish_owned_reveals_for_block(&block)?; @@ -1428,11 +1524,19 @@ impl NodeCore { } Ok(()) } + GossipEnvelope::RevealBundle(bundle) => self.receive_reveal_bundle(bundle), + GossipEnvelope::RevealBundles { bundles } => { + for bundle in bundles { + self.receive_reveal_bundle(bundle)?; + } + Ok(()) + } GossipEnvelope::Block(block) => { let previous_height = self.ledger.height(); self.ledger.apply_block(block.clone())?; if self.ledger.height() > previous_height { self.clear_stale_local_block_anchor(); + self.prune_reveal_bundles(); self.prune_owned_blinded_payloads_for_block(&block); self.publish_owned_reveals_for_block(&block)?; self.outbox.push(GossipEnvelope::Block(block)); @@ -1446,6 +1550,7 @@ impl NodeCore { self.ledger.apply_block(block.clone())?; if self.ledger.height() > previous_height { self.clear_stale_local_block_anchor(); + self.prune_reveal_bundles(); self.prune_owned_blinded_payloads_for_block(&block); self.publish_owned_reveals_for_block(&block)?; imported.push(block); @@ -1470,6 +1575,7 @@ impl NodeCore { .apply_preverified_block_at(block.clone(), now_ms)?; if self.ledger.height() > previous_height { self.clear_stale_local_block_anchor(); + self.prune_reveal_bundles(); self.prune_owned_blinded_payloads_for_block(&block); self.publish_owned_reveals_for_block(&block)?; self.outbox.push(GossipEnvelope::Block(block)); @@ -1495,6 +1601,7 @@ impl NodeCore { self.last_auto_pow_mine_status = None; self.auto_pow_mine_cursor = None; self.clear_stale_local_block_anchor(); + self.prune_reveal_bundles(); self.enqueue_imported_blocks(previous_height)?; } Ok(()) @@ -1517,6 +1624,7 @@ impl NodeCore { self.last_auto_pow_mine_status = None; self.auto_pow_mine_cursor = None; self.clear_stale_local_block_anchor(); + self.prune_reveal_bundles(); self.enqueue_imported_blocks(previous_height)?; Ok(true) } @@ -1533,6 +1641,7 @@ impl NodeCore { .ledger .blocks_from(previous_height + 1, IMPORT_REBROADCAST_LIMIT); for block in &blocks { + self.prune_reveal_bundles(); self.prune_owned_blinded_payloads_for_block(block); self.publish_owned_reveals_for_block(block)?; } diff --git a/src/domain.rs b/src/domain.rs @@ -27,6 +27,8 @@ pub const RECOVERY_BLOCK_DELAY_MS: u64 = VDF_TARGET_BLOCK_MS * 6; pub const MAX_VDF_ROUNDS: u64 = i64::MAX as u64; pub const MINE_DIFFICULTY_BITS: u32 = 12; pub const MAX_BLINDED_TRANSACTION_EXPIRY_HEIGHTS: u64 = 20; +pub const REVEAL_COMMITTEE_SIZE: usize = 3; +pub const MAX_REVEAL_BUNDLE_BYTES: usize = 10_000; const MINE_RETARGET_WINDOW_BLOCKS: u64 = 10; const MINE_TARGET_ACTIONS_PER_BLOCK: u64 = 1; const MINE_MAX_RETARGET_STEP_BITS: u32 = 2; @@ -94,6 +96,18 @@ impl Wallet { signature, } } + + fn reveal_bundle(&self, payload: RevealBundlePayload) -> RevealBundle { + let signature = self.sign_payload(&payload.canonical()); + RevealBundle { + height: payload.height, + prev_hash: payload.prev_hash, + slot: payload.slot, + member: self.address.clone(), + reveals: payload.reveals, + signature, + } + } } #[derive(Clone, Debug, Deserialize, Eq, Ord, PartialEq, PartialOrd, Serialize)] @@ -470,8 +484,80 @@ impl BlindedReveal { } } -fn canonical_blinded_block_items(blinded: &str, reveals: &str) -> String { - format!("blinded:{blinded}:reveals:{reveals}") +#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct RevealBundle { + pub height: u64, + pub prev_hash: String, + pub slot: u8, + pub member: String, + pub reveals: Vec<BlindedReveal>, + pub signature: String, +} + +impl RevealBundle { + pub fn canonical_payload(&self) -> String { + RevealBundlePayload { + height: self.height, + prev_hash: self.prev_hash.clone(), + slot: self.slot, + member: self.member.clone(), + reveals: self.reveals.clone(), + } + .canonical() + } + + pub fn canonical(&self) -> String { + format!("{}:{}", self.canonical_payload(), self.signature) + } + + pub fn bundle_hash(&self) -> String { + hex_hash(self.canonical()) + } + + pub fn serialized_size_bytes(&self) -> Result<usize> { + serde_json::to_vec(self) + .map(|bytes| bytes.len()) + .context("failed to serialize reveal bundle for size check") + } +} + +#[derive(Clone, Debug, Eq, PartialEq)] +struct RevealBundlePayload { + height: u64, + prev_hash: String, + slot: u8, + member: String, + reveals: Vec<BlindedReveal>, +} + +impl RevealBundlePayload { + fn canonical(&self) -> String { + let reveals = self + .reveals + .iter() + .map(BlindedReveal::canonical) + .collect::<Vec<_>>() + .join("|"); + format!( + "iuna-reveal-bundle-v1:{}:{}:{}:{}:{}", + self.height, self.prev_hash, self.slot, self.member, reveals + ) + } +} + +#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct RevealCommitteeMember { + pub slot: u8, + pub rank: u32, + pub ticket_id: String, + pub owner: String, + pub amount: Amount, +} + +fn canonical_blinded_block_items(blinded: &str, reveals: &str, bundles: &str) -> String { + format!("blinded-v2:{blinded}:reveals:{reveals}:bundles:{bundles}") } impl TxInput { @@ -821,7 +907,7 @@ pub struct Block { #[serde(default)] pub blinded_transactions: Vec<BlindedTransaction>, #[serde(default)] - pub blinded_reveals: Vec<BlindedReveal>, + pub reveal_bundles: Vec<RevealBundle>, pub transactions: Vec<Transaction>, pub hash: String, } @@ -848,7 +934,7 @@ impl Block { vdf_output: draft.vdf_output, leader_proof: draft.leader_proof, blinded_transactions: draft.blinded_transactions, - blinded_reveals: draft.blinded_reveals, + reveal_bundles: draft.reveal_bundles, transactions: draft.transactions, hash: String::new(), }; @@ -866,11 +952,17 @@ impl Block { } pub fn vdf_seed(&self) -> String { + let bundle_hashes = self.reveal_bundle_hashes(); match self.finalizer_mode { - FinalizerMode::Ticket => vdf_seed_for_child(&self.prev_hash, self.height), - FinalizerMode::Recovery => { - recovery_vdf_seed_for_child(&self.prev_hash, self.height, self.timestamp_ms) + FinalizerMode::Ticket => { + vdf_seed_for_child(&self.prev_hash, self.height, &bundle_hashes) } + FinalizerMode::Recovery => recovery_vdf_seed_for_child( + &self.prev_hash, + self.height, + self.timestamp_ms, + &bundle_hashes, + ), } } @@ -888,9 +980,15 @@ impl Block { .collect::<Vec<_>>() .join("|"); let reveals = self - .blinded_reveals + .all_blinded_reveals() .iter() - .map(BlindedReveal::canonical) + .map(|reveal| reveal.canonical()) + .collect::<Vec<_>>() + .join("|"); + let bundles = self + .reveal_bundles + .iter() + .map(RevealBundle::canonical) .collect::<Vec<_>>() .join("|"); let leader_proof = self @@ -903,7 +1001,7 @@ impl Block { ) }) .unwrap_or_default(); - if !self.blinded_transactions.is_empty() || !self.blinded_reveals.is_empty() { + if !self.blinded_transactions.is_empty() || !self.reveal_bundles.is_empty() { return hex_hash(format!( "block-content-v3:{}:{}:{}:{}:{}:{}:{}:{}:{}:{}", self.height, @@ -915,7 +1013,7 @@ impl Block { self.vdf_rounds, leader_proof, txs, - canonical_blinded_block_items(&blinded, &reveals) + canonical_blinded_block_items(&blinded, &reveals, &bundles) )); } hex_hash(format!( @@ -980,6 +1078,23 @@ impl Block { .map(|bytes| bytes.len()) .context("failed to serialize block for size check") } + + pub fn all_blinded_reveals(&self) -> Vec<&BlindedReveal> { + let mut seen = BTreeSet::new(); + self.reveal_bundles + .iter() + .flat_map(|bundle| bundle.reveals.iter()) + .filter(|reveal| seen.insert(reveal.commitment.clone())) + .collect() + } + + pub fn reveal_bundle_hashes(&self) -> [String; REVEAL_COMMITTEE_SIZE] { + reveal_bundle_hashes(&self.reveal_bundles) + } + + pub fn included_reveal_bundle_count(&self) -> usize { + self.reveal_bundles.len() + } } impl FinalizerMode { @@ -1078,7 +1193,7 @@ pub struct PreparedBlock { vdf_seed: String, leader_ticket: Option<BurnTicket>, blinded_transactions: Vec<BlindedTransaction>, - blinded_reveals: Vec<BlindedReveal>, + reveal_bundles: Vec<RevealBundle>, transactions: Vec<Transaction>, } @@ -1142,7 +1257,7 @@ impl PreparedBlock { vdf_output, leader_proof, blinded_transactions: self.blinded_transactions, - blinded_reveals: self.blinded_reveals, + reveal_bundles: self.reveal_bundles, transactions: self.transactions, }) } @@ -1161,7 +1276,7 @@ struct BlockDraft { vdf_output: String, leader_proof: Option<LeaderProof>, blinded_transactions: Vec<BlindedTransaction>, - blinded_reveals: Vec<BlindedReveal>, + reveal_bundles: Vec<RevealBundle>, transactions: Vec<Transaction>, } @@ -1320,7 +1435,6 @@ enum TransactionKind { struct BlockSelection { transactions: Vec<Transaction>, blinded_transactions: Vec<BlindedTransaction>, - blinded_reveals: Vec<BlindedReveal>, } #[derive(Clone, Copy, Debug, Eq, PartialEq)] @@ -1634,7 +1748,7 @@ impl Ledger { { carry_forward.extend(block.transactions.clone()); carry_forward_blinded.extend(block.blinded_transactions.clone()); - carry_forward_reveals.extend(block.blinded_reveals.clone()); + carry_forward_reveals.extend(block.all_blinded_reveals().into_iter().cloned()); } let mined_signatures = candidate @@ -1652,7 +1766,7 @@ impl Ledger { let mined_reveal_commitments = candidate .chain .iter() - .flat_map(|block| block.blinded_reveals.iter()) + .flat_map(|block| block.all_blinded_reveals()) .map(|reveal| reveal.commitment.clone()) .collect::<BTreeSet<_>>(); @@ -1727,7 +1841,7 @@ impl Ledger { apply_finalizer_ticket_effects(block, &mut tickets)?; tickets.extend(tickets_created_by_block(block, &self.launch_profile)?); let mut revealed_transactions = Vec::new(); - for reveal in &block.blinded_reveals { + for reveal in block.all_blinded_reveals() { let active = active_blinded.get(&reveal.commitment).with_context(|| { format!( "block {} reveals unknown blinded transaction {}", @@ -1775,6 +1889,30 @@ impl Ledger { .collect()) } + pub fn reveal_committee_for_next_block(&self) -> Vec<RevealCommitteeMember> { + self.reveal_committee_for_height(self.tip().height + 1) + } + + pub fn reveal_committee_for_height(&self, height: u64) -> Vec<RevealCommitteeMember> { + let ranked = ranked_tickets_for_height(self.tip(), height, &self.tickets); + let committee_start = ranked.len().saturating_sub(REVEAL_COMMITTEE_SIZE); + ranked + .into_iter() + .enumerate() + .skip(committee_start) + .enumerate() + .filter_map(|(slot, (rank, ticket))| { + Some(RevealCommitteeMember { + slot: u8::try_from(slot).ok()?, + rank: u32::try_from(rank).ok()?, + ticket_id: ticket.id, + owner: ticket.owner, + amount: ticket.amount, + }) + }) + .collect() + } + pub fn genesis_hash(&self) -> &str { &self.chain[0].hash } @@ -1836,6 +1974,131 @@ impl Ledger { &self.pending_reveals } + pub fn build_reveal_bundle(&self, wallet: &Wallet) -> Result<Option<RevealBundle>> { + let height = self.tip().height + 1; + let prev_hash = self.tip().hash.clone(); + let Some(member) = self + .reveal_committee_for_next_block() + .into_iter() + .find(|member| member.owner == wallet.address()) + else { + return Ok(None); + }; + let mut reveals = self.valid_pending_blinded_reveals(); + reveals.sort_by(|left, right| { + self.reveal_fee_order_key(right) + .cmp(&self.reveal_fee_order_key(left)) + .then_with(|| left.commitment.cmp(&right.commitment)) + }); + + let mut selected = Vec::new(); + for reveal in reveals { + let mut candidate = selected.clone(); + candidate.push(reveal); + let bundle = wallet.reveal_bundle(RevealBundlePayload { + height, + prev_hash: prev_hash.clone(), + slot: member.slot, + member: wallet.address().to_string(), + reveals: candidate.clone(), + }); + if bundle.serialized_size_bytes()? <= MAX_REVEAL_BUNDLE_BYTES { + selected = candidate; + } + } + if selected.is_empty() { + return Ok(None); + } + Ok(Some(wallet.reveal_bundle(RevealBundlePayload { + height, + prev_hash, + slot: member.slot, + member: wallet.address().to_string(), + reveals: selected, + }))) + } + + pub fn validate_next_block_reveal_bundles( + &self, + bundles: Vec<RevealBundle>, + ) -> Result<Vec<RevealBundle>> { + let expected_height = self.tip().height + 1; + let expected_prev_hash = self.tip().hash.clone(); + self.validate_reveal_bundles_for_block(expected_height, &expected_prev_hash, bundles) + } + + fn validate_reveal_bundles_for_block( + &self, + expected_height: u64, + expected_prev_hash: &str, + mut bundles: Vec<RevealBundle>, + ) -> Result<Vec<RevealBundle>> { + if bundles.len() > REVEAL_COMMITTEE_SIZE { + bail!("block has too many reveal bundles"); + } + if bundles.windows(2).any(|pair| pair[0].slot >= pair[1].slot) { + bail!("reveal bundles are not in slot order"); + } + bundles.sort_by_key(|bundle| bundle.slot); + let committee = self + .reveal_committee_for_height(expected_height) + .into_iter() + .map(|member| (member.slot, member)) + .collect::<BTreeMap<_, _>>(); + let mut seen_slots = BTreeSet::new(); + let mut seen_members = BTreeSet::new(); + for bundle in &bundles { + if bundle.height != expected_height { + bail!("reveal bundle height is invalid"); + } + if bundle.prev_hash != expected_prev_hash { + bail!("reveal bundle parent hash is invalid"); + } + if usize::from(bundle.slot) >= REVEAL_COMMITTEE_SIZE { + bail!("reveal bundle slot is invalid"); + } + if !seen_slots.insert(bundle.slot) { + bail!("duplicate reveal bundle slot"); + } + if !seen_members.insert(bundle.member.clone()) { + bail!("duplicate reveal bundle member"); + } + let member = committee + .get(&bundle.slot) + .context("reveal bundle slot is not assigned")?; + if bundle.member != member.owner { + bail!("reveal bundle member is not assigned to slot"); + } + if bundle.serialized_size_bytes()? > MAX_REVEAL_BUNDLE_BYTES { + bail!("reveal bundle exceeds max size"); + } + verify_address_signature( + &bundle.member, + &bundle.canonical_payload(), + &bundle.signature, + "reveal bundle", + )?; + let mut seen_bundle_reveals = BTreeSet::new(); + let mut previous_key: Option<((u128, Amount), String)> = None; + for reveal in &bundle.reveals { + if !seen_bundle_reveals.insert(reveal.commitment.clone()) { + bail!("duplicate blinded reveal in reveal bundle"); + } + self.pending_reveal_transaction(reveal)?; + let key = (self.reveal_fee_order_key(reveal), reveal.commitment.clone()); + if let Some((previous_fee_key, previous_commitment)) = &previous_key { + if key.0 > *previous_fee_key + || key.0 == *previous_fee_key && key.1 < *previous_commitment + { + bail!("reveal bundle is not fee ordered"); + } + } + previous_key = Some(key); + } + } + Ok(bundles) + } + pub fn orphan_transactions(&self) -> &[Transaction] { &self.orphans } @@ -1887,7 +2150,7 @@ impl Ledger { .any(|reveal| reveal.commitment == commitment) || self.chain.iter().any(|block| { block - .blinded_reveals + .all_blinded_reveals() .iter() .any(|reveal| reveal.commitment == commitment) }) @@ -2323,6 +2586,15 @@ impl Ledger { } pub fn prepare_next_block(&self, miner: &str, timestamp_ms: u64) -> Result<PreparedBlock> { + self.prepare_next_block_with_reveal_bundles(miner, timestamp_ms, Vec::new()) + } + + pub fn prepare_next_block_with_reveal_bundles( + &self, + miner: &str, + timestamp_ms: u64, + reveal_bundles: Vec<RevealBundle>, + ) -> Result<PreparedBlock> { let height = self.tip().height + 1; let Some((finalizer_rank, leader_ticket)) = self.finalizer_ticket_for_miner(height, miner) else { @@ -2332,13 +2604,15 @@ impl Ledger { bail!("no selected leader for block {height}"); } + let reveal_bundles = self.validate_next_block_reveal_bundles(reveal_bundles)?; let selection = self.select_block_transactions()?; ensure_block_has_burn(&selection.transactions)?; let tip = self.tip(); let prev_hash = tip.hash.clone(); let timestamp_ms = timestamp_ms.max(ticket_block_min_timestamp(tip, finalizer_rank)?); - let vdf_seed = vdf_seed_for_child(&prev_hash, height); + let bundle_hashes = reveal_bundle_hashes(&reveal_bundles); + let vdf_seed = vdf_seed_for_child(&prev_hash, height, &bundle_hashes); Ok(PreparedBlock { height, prev_hash, @@ -2351,7 +2625,7 @@ impl Ledger { finalizer_rank, leader_ticket: Some(leader_ticket), blinded_transactions: selection.blinded_transactions, - blinded_reveals: selection.blinded_reveals, + reveal_bundles, transactions: selection.transactions, }) } @@ -2367,12 +2641,22 @@ impl Ledger { } pub fn prepare_recovery_block(&self, miner: &str, timestamp_ms: u64) -> Result<PreparedBlock> { + self.prepare_recovery_block_with_reveal_bundles(miner, timestamp_ms, Vec::new()) + } + + pub fn prepare_recovery_block_with_reveal_bundles( + &self, + miner: &str, + timestamp_ms: u64, + reveal_bundles: Vec<RevealBundle>, + ) -> Result<PreparedBlock> { let height = self.tip().height + 1; let min_timestamp = self.recovery_block_min_timestamp(); if timestamp_ms < min_timestamp { bail!("recovery block is not available before timestamp {min_timestamp}"); } + let reveal_bundles = self.validate_next_block_reveal_bundles(reveal_bundles)?; let selection = self.select_recovery_block_transactions(miner)?; ensure_block_has_burn(&selection.transactions)?; ensure_block_has_burn_from(&selection.transactions, miner)?; @@ -2380,7 +2664,9 @@ impl Ledger { let tip = self.tip(); let prev_hash = tip.hash.clone(); let timestamp_ms = timestamp_ms.max(tip.timestamp_ms + 1); - let vdf_seed = recovery_vdf_seed_for_child(&prev_hash, height, timestamp_ms); + let bundle_hashes = reveal_bundle_hashes(&reveal_bundles); + let vdf_seed = + recovery_vdf_seed_for_child(&prev_hash, height, timestamp_ms, &bundle_hashes); Ok(PreparedBlock { height, prev_hash, @@ -2393,7 +2679,7 @@ impl Ledger { vdf_seed, leader_ticket: None, blinded_transactions: selection.blinded_transactions, - blinded_reveals: selection.blinded_reveals, + reveal_bundles, transactions: selection.transactions, }) } @@ -2452,7 +2738,8 @@ impl Ledger { apply_transaction(tx, &mut utxos)?; } let mut revealed_commitments = BTreeSet::new(); - for reveal in &block.blinded_reveals { + let included_reveal_bundle_count = block.included_reveal_bundle_count(); + for reveal in block.all_blinded_reveals() { if !revealed_commitments.insert(reveal.commitment.clone()) { bail!("duplicate blinded reveal in block"); } @@ -2462,7 +2749,13 @@ impl Ledger { .context("blinded reveal does not reference an active blinded transaction")?; let tx = self.decrypt_active_blinded(active, reveal)?; apply_transaction(&tx, &mut utxos)?; - credit_blinded_fee_outputs(&mut utxos, active, &block.miner, &tx)?; + credit_blinded_fee_outputs( + &mut utxos, + active, + &block.miner, + &tx, + included_reveal_bundle_count, + )?; revealed_transactions.push(tx); } if block.reward != fee_reward(&block.transactions)? { @@ -2489,8 +2782,8 @@ impl Ledger { .map(|transaction| transaction.commitment.clone()) .collect::<BTreeSet<_>>(); let revealed_blinded = block - .blinded_reveals - .iter() + .all_blinded_reveals() + .into_iter() .map(|reveal| reveal.commitment.clone()) .collect::<BTreeSet<_>>(); self.utxos = utxos; @@ -2613,7 +2906,7 @@ impl Ledger { } let block_item_count = block.transactions.len() + block.blinded_transactions.len() - + block.blinded_reveals.len(); + + block.all_blinded_reveals().len(); if block_item_count > self.launch_profile.max_block_transactions { bail!("block has too many transaction items"); } @@ -2621,6 +2914,11 @@ impl Ledger { bail!("block exceeds max block size"); } ensure_block_has_burn(&block.transactions)?; + self.validate_reveal_bundles_for_block( + block.height, + &block.prev_hash, + block.reveal_bundles.clone(), + )?; validate_block_blinded_items(block, self)?; match block.finalizer_mode { FinalizerMode::Ticket => { @@ -2750,10 +3048,8 @@ impl Ledger { let mut utxos = self.utxos.clone(); let mut remaining = self.valid_pending_transactions(); let mut remaining_blinded = self.valid_pending_blinded_transactions(); - let mut remaining_reveals = self.valid_pending_blinded_reveals(); let mut selected = Vec::new(); let mut selected_blinded = Vec::new(); - let mut selected_reveals = Vec::new(); let needs_first_burn = !selected.iter().any(Transaction::is_burn); let needs_owner_burn = required_burn_owner.is_some_and(|owner| { @@ -2772,7 +3068,6 @@ impl Ledger { let mut candidate = BlockSelection { transactions: selected.clone(), blinded_transactions: selected_blinded.clone(), - blinded_reveals: selected_reveals.clone(), }; candidate.transactions.push(tx.clone()); if estimated_block_selection_size_bytes(&candidate, required_burn_owner.is_some())? @@ -2785,7 +3080,7 @@ impl Ledger { } while selected.len() < self.launch_profile.max_block_transactions { - let selected_count = selected.len() + selected_blinded.len() + selected_reveals.len(); + let selected_count = selected.len() + selected_blinded.len(); if selected_count >= self.launch_profile.max_block_transactions { break; } @@ -2795,9 +3090,7 @@ impl Ledger { let best_blinded = best_selectable_blinded_index(&remaining_blinded).map(|index| { SelectableItem::Blinded(index, blinded_fee_rate_key(&remaining_blinded[index])) }); - let best_reveal = - best_selectable_reveal_index(&remaining_reveals).map(SelectableItem::Reveal); - let Some(item) = best_selectable_item(best_plain, best_blinded, best_reveal) else { + let Some(item) = best_selectable_item(best_plain, best_blinded) else { break; }; @@ -2807,7 +3100,6 @@ impl Ledger { let mut candidate = BlockSelection { transactions: selected.clone(), blinded_transactions: selected_blinded.clone(), - blinded_reveals: selected_reveals.clone(), }; candidate.transactions.push(tx.clone()); if estimated_block_selection_size_bytes( @@ -2824,7 +3116,6 @@ impl Ledger { let mut candidate = BlockSelection { transactions: selected.clone(), blinded_transactions: selected_blinded.clone(), - blinded_reveals: selected_reveals.clone(), }; candidate.blinded_transactions.push(transaction.clone()); if estimated_block_selection_size_bytes( @@ -2835,28 +3126,11 @@ impl Ledger { selected_blinded.push(transaction); } } - SelectableItem::Reveal(index) => { - let reveal = remaining_reveals.remove(index); - let mut candidate = BlockSelection { - transactions: selected.clone(), - blinded_transactions: selected_blinded.clone(), - blinded_reveals: selected_reveals.clone(), - }; - candidate.blinded_reveals.push(reveal.clone()); - if estimated_block_selection_size_bytes( - &candidate, - required_burn_owner.is_some(), - )? <= self.launch_profile.max_block_bytes - { - selected_reveals.push(reveal); - } - } } } Ok(BlockSelection { transactions: selected, blinded_transactions: selected_blinded, - blinded_reveals: selected_reveals, }) } @@ -3072,6 +3346,19 @@ impl Ledger { .collect() } + fn reveal_fee_order_key(&self, reveal: &BlindedReveal) -> (u128, Amount) { + let Some(active) = self.active_blinded.get(&reveal.commitment) else { + return (0, 0); + }; + let size = active.transaction.fee_rate_size_bytes(); + let rate = if size == 0 { + 0 + } else { + u128::from(active.transaction.fee) * 1_000_000 / size as u128 + }; + (rate, active.transaction.fee) + } + fn pending_reveal_transaction(&self, reveal: &BlindedReveal) -> Result<Transaction> { self.validate_blinded_reveal_terms(reveal)?; let active = self @@ -3533,17 +3820,12 @@ fn blinded_fee_rate_key(transaction: &BlindedTransaction) -> u128 { enum SelectableItem { Plain(usize, u128), Blinded(usize, u128), - Reveal(usize), } fn best_selectable_item( plain: Option<SelectableItem>, blinded: Option<SelectableItem>, - reveal: Option<SelectableItem>, ) -> Option<SelectableItem> { - if reveal.is_some() { - return reveal; - } match (plain, blinded) { ( Some(SelectableItem::Plain(_, plain_rate)), @@ -3574,14 +3856,6 @@ fn best_selectable_blinded_index(transactions: &[BlindedTransaction]) -> Option< .map(|(index, _)| index) } -fn best_selectable_reveal_index(reveals: &[BlindedReveal]) -> Option<usize> { - reveals - .iter() - .enumerate() - .min_by(|(_, left), (_, right)| left.commitment.cmp(&right.commitment)) - .map(|(index, _)| index) -} - fn best_selectable_transaction_index( transactions: &[Transaction], utxos: &BTreeMap<OutPoint, TxOutput>, @@ -3843,7 +4117,7 @@ fn estimated_block_selection_size_bytes( signature: "f".repeat(128), }), blinded_transactions: selection.blinded_transactions.clone(), - blinded_reveals: selection.blinded_reveals.clone(), + reveal_bundles: Vec::new(), transactions: selection.transactions.clone(), hash: "f".repeat(64), }; @@ -3884,25 +4158,70 @@ fn verify_leader_proof(block: &Block, tickets: &[BurnTicket]) -> Result<()> { } fn verify_leader_signature(proof: &LeaderProof, payload: &LeaderProofPayload) -> Result<()> { - let public_key = decode_hex_array::<PUBLIC_KEY_BYTES>(&proof.public_key) - .with_context(|| format!("invalid leader public key {}", proof.public_key))?; - let signature = decode_hex_array::<SIGNATURE_BYTES>(&proof.signature) - .context("invalid leader signature hex")?; - let verifying_key = - VerifyingKey::from_bytes(&public_key).context("invalid leader public key")?; + verify_address_signature( + &proof.public_key, + &payload.canonical(), + &proof.signature, + "leader", + ) +} + +fn verify_address_signature( + address: &str, + payload: &str, + signature: &str, + label: &str, +) -> Result<()> { + let public_key = decode_hex_array::<PUBLIC_KEY_BYTES>(address) + .with_context(|| format!("invalid {label} public key {address}"))?; + let signature = decode_hex_array::<SIGNATURE_BYTES>(signature) + .with_context(|| format!("invalid {label} signature hex"))?; + let verifying_key = VerifyingKey::from_bytes(&public_key) + .with_context(|| format!("invalid {label} public key"))?; let signature = Signature::from_bytes(&signature); verifying_key - .verify(payload.canonical().as_bytes(), &signature) - .context("leader signature is invalid") + .verify(payload.as_bytes(), &signature) + .with_context(|| format!("{label} signature is invalid")) } -fn vdf_seed_for_child(prev_hash: &str, height: u64) -> String { - hex_hash(format!("iuna-vdf-child:{prev_hash}:{height}")) +pub fn default_reveal_bundle_hash(slot: usize) -> String { + hex_hash(format!("iuna-default-reveal-bundle-v1:{slot}")) +} + +fn reveal_bundle_hashes(bundles: &[RevealBundle]) -> [String; REVEAL_COMMITTEE_SIZE] { + std::array::from_fn(|slot| { + bundles + .iter() + .find(|bundle| usize::from(bundle.slot) == slot) + .map(RevealBundle::bundle_hash) + .unwrap_or_else(|| default_reveal_bundle_hash(slot)) + }) +} + +fn canonical_reveal_bundle_hashes(bundle_hashes: &[String; REVEAL_COMMITTEE_SIZE]) -> String { + bundle_hashes.join("|") +} + +fn vdf_seed_for_child( + prev_hash: &str, + height: u64, + bundle_hashes: &[String; REVEAL_COMMITTEE_SIZE], +) -> String { + hex_hash(format!( + "iuna-vdf-child:{prev_hash}:{height}:{}", + canonical_reveal_bundle_hashes(bundle_hashes) + )) } -fn recovery_vdf_seed_for_child(prev_hash: &str, height: u64, timestamp_ms: u64) -> String { +fn recovery_vdf_seed_for_child( + prev_hash: &str, + height: u64, + timestamp_ms: u64, + bundle_hashes: &[String; REVEAL_COMMITTEE_SIZE], +) -> String { hex_hash(format!( - "iuna-recovery-vdf-child:{prev_hash}:{height}:{timestamp_ms}" + "iuna-recovery-vdf-child:{prev_hash}:{height}:{timestamp_ms}:{}", + canonical_reveal_bundle_hashes(bundle_hashes) )) } @@ -3985,13 +4304,13 @@ fn validate_block_blinded_items(block: &Block, ledger: &Ledger) -> Result<()> { } let mut reveals = BTreeSet::new(); - for reveal in &block.blinded_reveals { + for reveal in block.all_blinded_reveals() { if !reveals.insert(reveal.commitment.clone()) { bail!("duplicate blinded reveal in block"); } if ledger.chain.iter().any(|block| { block - .blinded_reveals + .all_blinded_reveals() .iter() .any(|existing| existing.commitment == reveal.commitment) }) { @@ -4081,13 +4400,16 @@ fn credit_blinded_fee_outputs( active: &ActiveBlindedTransaction, reveal_executor: &str, transaction: &Transaction, + included_reveal_bundle_count: usize, ) -> Result<()> { let fee = transaction.fee(); if fee == 0 { return Ok(()); } let committer_fee = fee / 2; - let executor_fee = fee - committer_fee; + let executor_full_fee = fee - committer_fee; + let executor_fee = executor_full_fee.saturating_mul(included_reveal_bundle_count as u64) + / REVEAL_COMMITTEE_SIZE as u64; let mut outputs = Vec::new(); if committer_fee > 0 { outputs.push(( @@ -4231,7 +4553,7 @@ fn build_genesis_block( vdf_output, leader_proof: None, blinded_transactions: Vec::new(), - blinded_reveals: Vec::new(), + reveal_bundles: Vec::new(), transactions, hash: String::new(), }; @@ -4336,7 +4658,7 @@ fn validate_genesis_block(block: &Block) -> Result<()> { if block.leader_proof.is_some() { bail!("genesis block must not carry a leader proof"); } - if !block.blinded_transactions.is_empty() || !block.blinded_reveals.is_empty() { + if !block.blinded_transactions.is_empty() || !block.reveal_bundles.is_empty() { bail!("genesis block must not carry blinded transactions"); } if block.compute_hash() != block.hash { @@ -4405,7 +4727,7 @@ pub fn revealed_blinded_transactions( let mut active = BTreeMap::<String, ActiveBlindedTransaction>::new(); let mut revealed = Vec::new(); for block in &snapshot.blocks { - for reveal in &block.blinded_reveals { + for reveal in block.all_blinded_reveals() { let active_transaction = active.get(&reveal.commitment).with_context(|| { format!( "block {} reveals unknown blinded transaction {}", @@ -4729,7 +5051,7 @@ mod tests { vdf_output: String::new(), leader_proof: None, blinded_transactions: Vec::new(), - blinded_reveals: Vec::new(), + reveal_bundles: Vec::new(), transactions: Vec::new(), hash: String::new(), } @@ -4804,6 +5126,31 @@ mod tests { block } + fn mine_preverified_as_next_leader_with_reveal_bundles( + ledger: &mut Ledger, + wallets: &[Wallet], + timestamp_ms: u64, + ) -> Block { + let bundles = ledger + .reveal_committee_for_next_block() + .into_iter() + .filter_map(|member| { + let wallet = wallet_for_address(wallets, &member.owner); + ledger.build_reveal_bundle(wallet).unwrap() + }) + .collect::<Vec<_>>(); + let leader = ledger.expected_leader_for_next_block().unwrap(); + let wallet = wallet_for_address(wallets, &leader); + let prepared = ledger + .prepare_next_block_with_reveal_bundles(wallet.address(), timestamp_ms, bundles) + .unwrap(); + let block = prepared.finish(wallet, "preverified-vdf".to_string()); + ledger + .apply_preverified_block_at(block.clone(), u64::MAX) + .unwrap(); + block + } + fn queue_next_leader_burn(ledger: &mut Ledger, wallets: &[Wallet]) { let leader = ledger.expected_leader_for_next_block().unwrap(); let wallet = wallet_for_address(wallets, &leader); @@ -5008,7 +5355,14 @@ mod tests { }; let mut utxos = BTreeMap::new(); - credit_blinded_fee_outputs(&mut utxos, &active, executor.address(), &transaction).unwrap(); + credit_blinded_fee_outputs( + &mut utxos, + &active, + executor.address(), + &transaction, + REVEAL_COMMITTEE_SIZE, + ) + .unwrap(); assert!(!utxos.contains_key(&blinded_committer_fee_outpoint(&commitment))); assert_eq!( @@ -5021,6 +5375,51 @@ mod tests { } #[test] + fn blinded_executor_fee_scales_with_included_reveal_bundles() { + let committer = Wallet::from_seed("blinded-scale-committer"); + let executor = Wallet::from_seed("blinded-scale-executor"); + let commitment = "05".repeat(32); + let active = ActiveBlindedTransaction { + transaction: BlindedTransaction { + commitment: commitment.clone(), + fee: 7, + encrypted_size: 1, + expires_at_height: 2, + nonce: "02".repeat(BLINDED_NONCE_BYTES), + ciphertext: "03".to_string(), + payload_hash: "04".repeat(32), + }, + included_height: 1, + included_by: committer.address().to_string(), + }; + let transaction = Transaction::Transfer { + inputs: Vec::new(), + outputs: Vec::new(), + fee: 7, + signature: String::new(), + }; + let mut utxos = BTreeMap::new(); + + credit_blinded_fee_outputs(&mut utxos, &active, executor.address(), &transaction, 1) + .unwrap(); + + assert_eq!( + utxos.get(&blinded_committer_fee_outpoint(&commitment)), + Some(&TxOutput { + address: committer.address().to_string(), + amount: 3, + }) + ); + assert_eq!( + utxos.get(&blinded_executor_fee_outpoint(&commitment)), + Some(&TxOutput { + address: executor.address().to_string(), + amount: 1, + }) + ); + } + + #[test] fn transfer_rejects_invalid_recipient_address() { let alice = Wallet::from_seed("invalid-transfer-recipient-alice"); let ledger = ledger_with_wallet_utxos(&alice, &[10]); @@ -5654,7 +6053,7 @@ mod tests { signature: "signature".to_string(), }), blinded_transactions: Vec::new(), - blinded_reveals: Vec::new(), + reveal_bundles: Vec::new(), transactions: Vec::new(), }); @@ -5776,10 +6175,11 @@ mod tests { ledger.submit_blinded_reveal(blinded.reveal).unwrap(); queue_next_leader_burn(&mut ledger, &finalizers); - let reveal_block = mine_preverified_as_next_leader(&mut ledger, &finalizers, 2); + let reveal_block = + mine_preverified_as_next_leader_with_reveal_bundles(&mut ledger, &finalizers, 2); let reveal_executor = reveal_block.miner.clone(); - assert_eq!(reveal_block.blinded_reveals.len(), 1); + assert_eq!(reveal_block.all_blinded_reveals().len(), 1); assert_eq!( ledger.balance_of(carol.address()), before_carol - burn_amount - fee @@ -5800,7 +6200,9 @@ mod tests { total + transaction.amount() + transaction.fee() }); let committer_fee = fee / 2; - let executor_fee = fee - committer_fee; + let executor_fee = (fee - committer_fee) + * reveal_block.included_reveal_bundle_count() as u64 + / REVEAL_COMMITTEE_SIZE as u64; assert_eq!( ledger .utxos @@ -5826,7 +6228,7 @@ mod tests { } ); let inclusion_finalizer_fee = if inclusion_finalizer == reveal_executor { - fee + committer_fee + executor_fee } else { committer_fee }; @@ -5838,6 +6240,94 @@ mod tests { } #[test] + fn reveal_bundle_hashes_are_bound_to_next_block_vdf_seed() { + let alice = Wallet::from_seed("bundle-seed-alice"); + let bob = Wallet::from_seed("bundle-seed-bob"); + let carol = Wallet::from_seed("bundle-seed-carol"); + let finalizers = [alice.clone(), bob.clone()]; + let mut ledger = ledger_with_finalizers(&finalizers, &[(&carol, 10 * MICRO_IUNA)]); + let blinded = ledger + .build_blinded_burn(&carol, 3, 7, ledger.height() + 4) + .unwrap(); + ledger + .submit_blinded_transaction(blinded.transaction.clone()) + .unwrap(); + queue_next_leader_burn(&mut ledger, &finalizers); + mine_preverified_as_next_leader(&mut ledger, &finalizers, 1); + ledger.submit_blinded_reveal(blinded.reveal).unwrap(); + queue_next_leader_burn(&mut ledger, &finalizers); + let leader = ledger.expected_leader_for_next_block().unwrap(); + let leader_wallet = wallet_for_address(&finalizers, &leader); + let bundles = ledger + .reveal_committee_for_next_block() + .into_iter() + .filter_map(|member| { + let wallet = wallet_for_address(&finalizers, &member.owner); + ledger.build_reveal_bundle(wallet).unwrap() + }) + .collect::<Vec<_>>(); + if bundles.len() > 1 { + let mut reversed = bundles.clone(); + reversed.reverse(); + let error = ledger + .validate_next_block_reveal_bundles(reversed) + .unwrap_err(); + assert!(format!("{error:#}").contains("reveal bundles are not in slot order")); + } + + let without_bundles = ledger + .prepare_next_block(leader_wallet.address(), ledger.tip().timestamp_ms + 1) + .unwrap(); + let with_bundles = ledger + .prepare_next_block_with_reveal_bundles( + leader_wallet.address(), + ledger.tip().timestamp_ms + 1, + bundles, + ) + .unwrap(); + + assert_ne!(without_bundles.vdf_seed(), with_bundles.vdf_seed()); + } + + #[test] + fn reveal_bundle_validation_rejects_wrong_signature_and_slot() { + let alice = Wallet::from_seed("bundle-invalid-alice"); + let bob = Wallet::from_seed("bundle-invalid-bob"); + let carol = Wallet::from_seed("bundle-invalid-carol"); + let finalizers = [alice.clone(), bob.clone()]; + let mut ledger = ledger_with_finalizers(&finalizers, &[(&carol, 10 * MICRO_IUNA)]); + let blinded = ledger + .build_blinded_burn(&carol, 3, 7, ledger.height() + 4) + .unwrap(); + ledger + .submit_blinded_transaction(blinded.transaction.clone()) + .unwrap(); + queue_next_leader_burn(&mut ledger, &finalizers); + mine_preverified_as_next_leader(&mut ledger, &finalizers, 1); + ledger.submit_blinded_reveal(blinded.reveal).unwrap(); + let member = ledger.reveal_committee_for_next_block()[0].clone(); + let wallet = wallet_for_address(&finalizers, &member.owner); + let bundle = ledger.build_reveal_bundle(wallet).unwrap().unwrap(); + + let mut wrong_signature = bundle.clone(); + wrong_signature.signature = "00".repeat(SIGNATURE_BYTES); + let error = ledger + .validate_next_block_reveal_bundles(vec![wrong_signature]) + .unwrap_err(); + assert!(format!("{error:#}").contains("reveal bundle signature is invalid")); + + let mut wrong_slot = bundle; + wrong_slot.slot = REVEAL_COMMITTEE_SIZE as u8 - 1; + let error = ledger + .validate_next_block_reveal_bundles(vec![wrong_slot]) + .unwrap_err(); + assert!( + format!("{error:#}").contains("reveal bundle slot is not assigned") + || format!("{error:#}").contains("reveal bundle member is not assigned to slot") + ); + } + + #[test] fn blinded_reveal_with_wrong_key_is_rejected_in_block() { let alice = Wallet::from_seed("blinded-wrong-key-finalizer-alice"); let bob = Wallet::from_seed("blinded-wrong-key-finalizer-bob"); @@ -5860,10 +6350,21 @@ mod tests { let mut prepared = ledger .prepare_next_block(wallet.address(), ledger.tip().timestamp_ms + 1) .unwrap(); - prepared.blinded_reveals.push(BlindedReveal { + let committee_member = ledger.reveal_committee_for_next_block()[0].clone(); + let committee_wallet = wallet_for_address(&finalizers, &committee_member.owner); + let wrong_reveal = BlindedReveal { commitment: blinded.transaction.commitment, key: "00".repeat(BLINDED_KEY_BYTES), - }); + }; + prepared + .reveal_bundles + .push(committee_wallet.reveal_bundle(RevealBundlePayload { + height: prepared.height, + prev_hash: prepared.prev_hash.clone(), + slot: committee_member.slot, + member: committee_wallet.address().to_string(), + reveals: vec![wrong_reveal], + })); let block = prepared.finish(wallet, "preverified-vdf".to_string()); let error = ledger @@ -5949,7 +6450,6 @@ mod tests { let recovery_selection = BlockSelection { transactions: vec![recovery_burn], blinded_transactions: vec![blinded.transaction.clone()], - blinded_reveals: Vec::new(), }; let recovery_estimate = estimated_block_selection_size_bytes(&recovery_selection, true).unwrap(); @@ -5975,11 +6475,8 @@ mod tests { let alice = Wallet::from_seed("recovery-blinded-reveal-alice"); let bob = Wallet::from_seed("recovery-blinded-reveal-bob"); let carol = Wallet::from_seed("recovery-blinded-reveal-carol"); - let finalizers = [alice.clone()]; - let mut ledger = ledger_with_finalizers( - &finalizers, - &[(&bob, 10 * MICRO_IUNA), (&carol, 10 * MICRO_IUNA)], - ); + let finalizers = [alice.clone(), bob.clone()]; + let mut ledger = ledger_with_finalizers(&finalizers, &[(&carol, 10 * MICRO_IUNA)]); let blinded = ledger .build_blinded_burn(&carol, MICRO_IUNA, 7, ledger.height() + 4) .unwrap(); @@ -5994,13 +6491,27 @@ mod tests { let recovery_burn = ledger.build_burn(&bob, MICRO_IUNA, 0).unwrap(); ledger.submit_transaction(recovery_burn).unwrap(); - let block = ledger - .mine_recovery_block(&bob, ledger.recovery_block_min_timestamp()) + let bundles = ledger + .reveal_committee_for_next_block() + .into_iter() + .filter_map(|member| { + let wallet = wallet_for_address(&finalizers, &member.owner); + ledger.build_reveal_bundle(wallet).unwrap() + }) + .collect::<Vec<_>>(); + let prepared = ledger + .prepare_recovery_block_with_reveal_bundles( + bob.address(), + ledger.recovery_block_min_timestamp(), + bundles, + ) .unwrap(); + let vdf_output = run_vdf(prepared.vdf_seed(), prepared.vdf_rounds()); + let block = prepared.finish(&bob, vdf_output); assert!( block - .blinded_reveals + .all_blinded_reveals() .iter() .any(|reveal| reveal.commitment == blinded.transaction.commitment) ); @@ -6095,7 +6606,7 @@ mod tests { .submit_blinded_reveal(blinded.reveal.clone()) .unwrap(); queue_next_leader_burn(&mut ledger, &finalizers); - mine_preverified_as_next_leader(&mut ledger, &finalizers, 2); + mine_preverified_as_next_leader_with_reveal_bundles(&mut ledger, &finalizers, 2); let leader = ledger.expected_leader_for_next_block().unwrap(); let wallet = wallet_for_address(&finalizers, &leader); diff --git a/tests/iuna.rs b/tests/iuna.rs @@ -1583,7 +1583,7 @@ fn joined_nodes_import_transfer_block_and_blinded_burn_reveal() { queue_plaintext_burn(network.node_mut("a").unwrap(), &alice, 1); let reveal_block = network.node_mut("a").unwrap().mine_one_at(7).unwrap(); - assert_eq!(reveal_block.blinded_reveals.len(), 1); + assert_eq!(reveal_block.all_blinded_reveals().len(), 1); mined_by.push(reveal_block.miner.clone()); network.deliver_until_idle().unwrap(); let revealed = revealed_blinded_transactions(&network.node("a").unwrap().chain_snapshot()) @@ -1955,7 +1955,7 @@ fn multiple_peers_can_contribute_blinded_burns_to_lottery_ranks() { .unwrap() .mine_one_at(2) .unwrap(); - assert_eq!(reveal_block.blinded_reveals.len(), 4); + assert_eq!(reveal_block.all_blinded_reveals().len(), 4); network.deliver_until_idle().unwrap(); for height in 3..=4 {