solana-docs 0.0.0

package/src/implemented-proposals/installer.md

@@ -0,0 +1,216 @@
+---
+title: Cluster Software Installation and Updates
+---
+
+Currently users are required to build the solana cluster software themselves from the git repository and manually update it, which is error prone and inconvenient.
+
+This document proposes an easy to use software install and updater that can be used to deploy pre-built binaries for supported platforms. Users may elect to use binaries supplied by Solana or any other party provider. Deployment of updates is managed using an on-chain update manifest program.
+
+## Motivating Examples
+
+### Fetch and run a pre-built installer using a bootstrap curl/shell script
+
+The easiest install method for supported platforms:
+
+```bash
+$ curl -sSf https://raw.githubusercontent.com/solana-labs/solana/v1.0.0/install/solana-install-init.sh | sh
+```
+
+This script will check github for the latest tagged release and download and run the `solana-install-init` binary from there.
+
+If additional arguments need to be specified during the installation, the following shell syntax is used:
+
+```bash
+$ init_args=.... # arguments for `solana-install-init ...`
+$ curl -sSf https://raw.githubusercontent.com/solana-labs/solana/v1.0.0/install/solana-install-init.sh | sh -s - ${init_args}
+```
+
+### Fetch and run a pre-built installer from a Github release
+
+With a well-known release URL, a pre-built binary can be obtained for supported platforms:
+
+```bash
+$ curl -o solana-install-init https://github.com/solana-labs/solana/releases/download/v1.0.0/solana-install-init-x86_64-apple-darwin
+$ chmod +x ./solana-install-init
+$ ./solana-install-init --help
+```
+
+### Build and run the installer from source
+
+If a pre-built binary is not available for a given platform, building the installer from source is always an option:
+
+```bash
+$ git clone https://github.com/solana-labs/solana.git
+$ cd solana/install
+$ cargo run -- --help
+```
+
+### Deploy a new update to a cluster
+
+Given a solana release tarball \(as created by `ci/publish-tarball.sh`\) that has already been uploaded to a publicly accessible URL, the following commands will deploy the update:
+
+```bash
+$ solana-keygen new -o update-manifest.json  # <-- only generated once, the public key is shared with users
+$ solana-install deploy http://example.com/path/to/solana-release.tar.bz2 update-manifest.json
+```
+
+### Run a validator node that auto updates itself
+
+```bash
+$ solana-install init --pubkey 92DMonmBYXwEMHJ99c9ceRSpAmk9v6i3RdvDdXaVcrfj  # <-- pubkey is obtained from whoever is deploying the updates
+$ export PATH=~/.local/share/solana-install/bin:$PATH
+$ solana-keygen ...  # <-- runs the latest solana-keygen
+$ solana-install run solana-validator ...  # <-- runs a validator, restarting it as necesary when an update is applied
+```
+
+## On-chain Update Manifest
+
+An update manifest is used to advertise the deployment of new release tarballs on a solana cluster. The update manifest is stored using the `config` program, and each update manifest account describes a logical update channel for a given target triple \(eg, `x86_64-apple-darwin`\). The account public key is well-known between the entity deploying new updates and users consuming those updates.
+
+The update tarball itself is hosted elsewhere, off-chain and can be fetched from the specified `download_url`.
+
+```text
+use solana_sdk::signature::Signature;
+
+/// Information required to download and apply a given update
+pub struct UpdateManifest {
+    pub timestamp_secs: u64, // When the release was deployed in seconds since UNIX EPOCH
+    pub download_url: String, // Download URL to the release tar.bz2
+    pub download_sha256: String, // SHA256 digest of the release tar.bz2 file
+}
+
+/// Data of an Update Manifest program Account.
+#[derive(Serialize, Deserialize, Default, Debug, PartialEq)]
+pub struct SignedUpdateManifest {
+    pub manifest: UpdateManifest,
+    pub manifest_signature: Signature,
+}
+```
+
+Note that the `manifest` field itself contains a corresponding signature \(`manifest_signature`\) to guard against man-in-the-middle attacks between the `solana-install` tool and the solana cluster RPC API.
+
+To guard against rollback attacks, `solana-install` will refuse to install an update with an older `timestamp_secs` than what is currently installed.
+
+## Release Archive Contents
+
+A release archive is expected to be a tar file compressed with bzip2 with the following internal structure:
+
+- `/version.yml` - a simple YAML file containing the field `"target"` - the
+
+  target tuple. Any additional fields are ignored.
+
+- `/bin/` -- directory containing available programs in the release.
+
+  `solana-install` will symlink this directory to
+
+  `~/.local/share/solana-install/bin` for use by the `PATH` environment
+
+  variable.
+
+- `...` -- any additional files and directories are permitted
+
+## solana-install Tool
+
+The `solana-install` tool is used by the user to install and update their cluster software.
+
+It manages the following files and directories in the user's home directory:
+
+- `~/.config/solana/install/config.yml` - user configuration and information about currently installed software version
+- `~/.local/share/solana/install/bin` - a symlink to the current release. eg, `~/.local/share/solana-update/<update-pubkey>-<manifest_signature>/bin`
+- `~/.local/share/solana/install/releases/<download_sha256>/` - contents of a release
+
+### Command-line Interface
+
+```text
+solana-install 0.16.0
+The solana cluster software installer
+
+USAGE:
+    solana-install [OPTIONS] <SUBCOMMAND>
+
+FLAGS:
+    -h, --help       Prints help information
+    -V, --version    Prints version information
+
+OPTIONS:
+    -c, --config <PATH>    Configuration file to use [default: .../Library/Preferences/solana/install.yml]
+
+SUBCOMMANDS:
+    deploy    deploys a new update
+    help      Prints this message or the help of the given subcommand(s)
+    info      displays information about the current installation
+    init      initializes a new installation
+    run       Runs a program while periodically checking and applying software updates
+    update    checks for an update, and if available downloads and applies it
+```
+
+```text
+solana-install-init
+initializes a new installation
+
+USAGE:
+    solana-install init [OPTIONS]
+
+FLAGS:
+    -h, --help    Prints help information
+
+OPTIONS:
+    -d, --data_dir <PATH>    Directory to store install data [default: .../Library/Application Support/solana]
+    -u, --url <URL>          JSON RPC URL for the solana cluster [default: http://api.devnet.solana.com]
+    -p, --pubkey <PUBKEY>    Public key of the update manifest [default: 9XX329sPuskWhH4DQh6k16c87dHKhXLBZTL3Gxmve8Gp]
+```
+
+```text
+solana-install info
+displays information about the current installation
+
+USAGE:
+    solana-install info [FLAGS]
+
+FLAGS:
+    -h, --help     Prints help information
+    -l, --local    only display local information, don't check the cluster for new updates
+```
+
+```text
+solana-install deploy
+deploys a new update
+
+USAGE:
+    solana-install deploy <download_url> <update_manifest_keypair>
+
+FLAGS:
+    -h, --help    Prints help information
+
+ARGS:
+    <download_url>               URL to the solana release archive
+    <update_manifest_keypair>    Keypair file for the update manifest (/path/to/keypair.json)
+```
+
+```text
+solana-install update
+checks for an update, and if available downloads and applies it
+
+USAGE:
+    solana-install update
+
+FLAGS:
+    -h, --help    Prints help information
+```
+
+```text
+solana-install run
+Runs a program while periodically checking and applying software updates
+
+USAGE:
+    solana-install run <program_name> [program_arguments]...
+
+FLAGS:
+    -h, --help    Prints help information
+
+ARGS:
+    <program_name>            program to run
+    <program_arguments>...    arguments to supply to the program
+
+The program will be restarted upon a successful software update
+```

package/src/implemented-proposals/instruction_introspection.md

@@ -0,0 +1,28 @@
+---
+title: instruction introspection
+---
+
+## Problem
+
+Some smart contract programs may want to verify that another Instruction is present in a
+given Message since that Instruction could be be performing a verification of certain data,
+in a precompiled function. (See secp256k1_instruction for an example).
+
+## Solution
+
+Add a new sysvar Sysvar1nstructions1111111111111111111111111 that a program can reference
+and received the Message's instruction data inside, and also the index of the current instruction.
+
+Two helper functions to extract this data can be used:
+
+```
+fn load_current_index(instruction_data: &[u8]) -> u16;
+fn load_instruction_at(instruction_index: usize, instruction_data: &[u8]) -> Result<Instruction>;
+```
+
+The runtime will recognize this special instruction, serialize the Message instruction data
+for it and also write the current instruction index and then the bpf program can extract the
+necessary information from there.
+
+Note: custom serialization of instructions is used because bincode is about 10x slower
+in native code and exceeds current BPF instruction limits.

package/src/implemented-proposals/leader-leader-transition.md

@@ -0,0 +1,55 @@
+---
+title: Leader-to-Leader Transition
+---
+
+This design describes how leaders transition production of the PoH ledger between each other as each leader generates its own slot.
+
+## Challenges
+
+Current leader and the next leader are both racing to generate the final tick for the current slot. The next leader may arrive at that slot while still processing the current leader's entries.
+
+The ideal scenario would be that the next leader generated its own slot right after it was able to vote for the current leader. It is very likely that the next leader will arrive at their PoH slot height before the current leader finishes broadcasting the entire block.
+
+The next leader has to make the decision of attaching its own block to the last completed block, or wait to finalize the pending block. It is possible that the next leader will produce a block that proposes that the current leader failed, even though the rest of the network observes that block succeeding.
+
+The current leader has incentives to start its slot as early as possible to capture economic rewards. Those incentives need to be balanced by the leader's need to attach its block to a block that has the most commitment from the rest of the network.
+
+## Leader timeout
+
+While a leader is actively receiving entries for the previous slot, the leader can delay broadcasting the start of its block in real time. The delay is locally configurable by each leader, and can be dynamically based on the previous leader's behavior. If the previous leader's block is confirmed by the leader's TVU before the timeout, the PoH is reset to the start of the slot and this leader produces its block immediately.
+
+The downsides:
+
+- Leader delays its own slot, potentially allowing the next leader more time to
+
+  catch up.
+
+The upsides compared to guards:
+
+- All the space in a block is used for entries.
+- The timeout is not fixed.
+- The timeout is local to the leader, and therefore can be clever. The leader's heuristic can take into account turbine performance.
+- This design doesn't require a ledger hard fork to update.
+- The previous leader can redundantly transmit the last entry in the block to the next leader, and the next leader can speculatively decide to trust it to generate its block without verification of the previous block.
+- The leader can speculatively generate the last tick from the last received entry.
+- The leader can speculatively process transactions and guess which ones are not going to be encoded by the previous leader. This is also a censorship attack vector. The current leader may withhold transactions that it receives from the clients so it can encode them into its own slot. Once processed, entries can be replayed into PoH quickly.
+
+## Alternative design options
+
+### Guard tick at the end of the slot
+
+A leader does not produce entries in its block after the _penultimate tick_, which is the last tick before the first tick of the next slot. The network votes on the _last tick_, so the time difference between the _penultimate tick_ and the _last tick_ is the forced delay for the entire network, as well as the next leader before a new slot can be generated. The network can produce the _last tick_ from the _penultimate tick_.
+
+If the next leader receives the _penultimate tick_ before it produces its own _first tick_, it will reset its PoH and produce the _first tick_ from the previous leader's _penultimate tick_. The rest of the network will also reset its PoH to produce the _last tick_ as the id to vote on.
+
+The downsides:
+
+- Every vote, and therefore confirmation, is delayed by a fixed timeout. 1 tick, or around 100ms.
+- Average case confirmation time for a transaction would be at least 50ms worse.
+- It is part of the ledger definition, so to change this behavior would require a hard fork.
+- Not all the available space is used for entries.
+
+The upsides compared to leader timeout:
+
+- The next leader has received all the previous entries, so it can start processing transactions without recording them into PoH.
+- The previous leader can redundantly transmit the last entry containing the _penultimate tick_ to the next leader. The next leader can speculatively generate the _last tick_ as soon as it receives the _penultimate tick_, even before verifying it.

package/src/implemented-proposals/leader-validator-transition.md

@@ -0,0 +1,52 @@
+---
+title: Leader-to-Validator Transition
+---
+
+A validator typically spends its time validating blocks. If, however, a staker delegates its stake to a validator, it will occasionally be selected as a _slot leader_. As a slot leader, the validator is responsible for producing blocks during an assigned _slot_. A slot has a duration of some number of preconfigured _ticks_. The duration of those ticks are estimated with a _PoH Recorder_ described later in this document.
+
+## BankFork
+
+BankFork tracks changes to the bank state over a specific slot. Once the final tick has been registered the state is frozen. Any attempts to write to are rejected.
+
+## Validator
+
+A validator operates on many different concurrent forks of the bank state until it generates a PoH hash with a height within its leader slot.
+
+## Slot Leader
+
+A slot leader builds blocks on top of only one fork, the one it last voted on.
+
+## PoH Recorder
+
+Slot leaders and validators use a PoH Recorder for both estimating slot height and for recording transactions.
+
+### PoH Recorder when Validating
+
+The PoH Recorder acts as a simple VDF when validating. It tells the validator when it needs to switch to the slot leader role. Every time the validator votes on a fork, it should use the fork's latest [blockhash](../terminology.md#blockhash) to re-seed the VDF. Re-seeding solves two problems. First, it synchronizes its VDF to the leader's, allowing it to more accurately determine when its leader slot begins. Second, if the previous leader goes down, all wallclock time is accounted for in the next leader's PoH stream. For example, if one block is missing when the leader starts, the block it produces should have a PoH duration of two blocks. The longer duration ensures the following leader isn't attempting to snip all the transactions from the previous leader's slot.
+
+### PoH Recorder when Leading
+
+A slot leader use the PoH Recorder to record transactions, locking their positions in time. The PoH hash must be derived from a previous leader's last block. If it isn't, its block will fail PoH verification and be rejected by the cluster.
+
+The PoH Recorder also serves to inform the slot leader when its slot is over. The leader needs to take care not to modify its bank if recording the transaction would generate a PoH height outside its designated slot. The leader, therefore, should not commit account changes until after it generates the entry's PoH hash. When the PoH height falls outside its slot any transactions in its pipeline may be dropped or forwarded to the next leader. Forwarding is preferred, as it would minimize network congestion, allowing the cluster to advertise higher TPS capacity.
+
+## Validator Loop
+
+The PoH Recorder manages the transition between modes. Once a ledger is replayed, the validator can run until the recorder indicates it should be the slot leader. As a slot leader, the node can then execute and record transactions.
+
+The loop is synchronized to PoH and does a synchronous start and stop of the slot leader functionality. After stopping, the validator's TVU should find itself in the same state as if a different leader had sent it the same block. The following is pseudocode for the loop:
+
+1. Query the LeaderScheduler for the next assigned slot.
+2. Run the TVU over all the forks. 1. TVU will send votes to what it believes is the "best" fork. 2. After each vote, restart the PoH Recorder to run until the next assigned
+
+   slot.
+
+3. When time to be a slot leader, start the TPU. Point it to the last fork the
+
+   TVU voted on.
+
+4. Produce entries until the end of the slot. 1. For the duration of the slot, the TVU must not vote on other forks. 2. After the slot ends, the TPU freezes its BankFork. After freezing,
+
+   the TVU may resume voting.
+
+5. Goto 1.

package/src/implemented-proposals/persistent-account-storage.md

@@ -0,0 +1,83 @@
+---
+title: Persistent Account Storage
+---
+
+## Persistent Account Storage
+
+The set of accounts represent the current computed state of all the transactions that have been processed by a validator. Each validator needs to maintain this entire set. Each block that is proposed by the network represents a change to this set, and since each block is a potential rollback point, the changes need to be reversible.
+
+Persistent storage like NVMEs are 20 to 40 times cheaper than DDR. The problem with persistent storage is that write and read performance is much slower than DDR. Care must be taken in how data is read or written to. Both reads and writes can be split between multiple storage drives and accessed in parallel. This design proposes a data structure that allows for concurrent reads and concurrent writes of storage. Writes are optimized by using an AppendVec data structure, which allows a single writer to append while allowing access to many concurrent readers. The accounts index maintains a pointer to a spot where the account was appended to every fork, thus removing the need for explicit checkpointing of state.
+
+## AppendVec
+
+AppendVec is a data structure that allows for random reads concurrent with a single append-only writer. Growing or resizing the capacity of the AppendVec requires exclusive access. This is implemented with an atomic `offset`, which is updated at the end of a completed append.
+
+The underlying memory for an AppendVec is a memory-mapped file. Memory-mapped files allow for fast random access and paging is handled by the OS.
+
+## Account Index
+
+The account index is designed to support a single index for all the currently forked Accounts.
+
+```text
+type AppendVecId = usize;
+
+type Fork = u64;
+
+struct AccountMap(Hashmap<Fork, (AppendVecId, u64)>);
+
+type AccountIndex = HashMap<Pubkey, AccountMap>;
+```
+
+The index is a map of account Pubkeys to a map of Forks and the location of the Account data in an AppendVec. To get the version of an account for a specific Fork:
+
+```text
+/// Load the account for the pubkey.
+/// This function will load the account from the specified fork, falling back to the fork's parents
+/// * fork - a virtual Accounts instance, keyed by Fork.  Accounts keep track of their parents with Forks,
+///       the persistent store
+/// * pubkey - The Account's public key.
+pub fn load_slow(&self, id: Fork, pubkey: &Pubkey) -> Option<&Account>
+```
+
+The read is satisfied by pointing to a memory-mapped location in the `AppendVecId` at the stored offset. A reference can be returned without a copy.
+
+### Root Forks
+
+[Tower BFT](tower-bft.md) eventually selects a fork as a root fork and the fork is squashed. A squashed/root fork cannot be rolled back.
+
+When a fork is squashed, all accounts in its parents not already present in the fork are pulled up into the fork by updating the indexes. Accounts with zero balance in the squashed fork are removed from fork by updating the indexes.
+
+An account can be _garbage-collected_ when squashing makes it unreachable.
+
+Three possible options exist:
+
+- Maintain a HashSet of root forks. One is expected to be created every second. The entire tree can be garbage-collected later. Alternatively, if every fork keeps a reference count of accounts, garbage collection could occur any time an index location is updated.
+- Remove any pruned forks from the index. Any remaining forks lower in number than the root are can be considered root.
+- Scan the index, migrate any old roots into the new one. Any remaining forks lower than the new root can be deleted later.
+
+## Garbage collection
+
+As accounts get updated, they move to the end of the AppendVec. Once capacity has run out, a new AppendVec can be created and updates can be stored there. Eventually references to an older AppendVec will disappear because all the accounts have been updated, and the old AppendVec can be deleted.
+
+To speed up this process, it's possible to move Accounts that have not been recently updated to the front of a new AppendVec. This form of garbage collection can be done without requiring exclusive locks to any of the data structures except for the index update.
+
+The initial implementation for garbage collection is that once all the accounts in an AppendVec become stale versions, it gets reused. The accounts are not updated or moved around once appended.
+
+## Index Recovery
+
+Each bank thread has exclusive access to the accounts during append, since the accounts locks cannot be released until the data is committed. But there is no explicit order of writes between the separate AppendVec files. To create an ordering, the index maintains an atomic write version counter. Each append to the AppendVec records the index write version number for that append in the entry for the Account in the AppendVec.
+
+To recover the index, all the AppendVec files can be read in any order, and the latest write version for every fork should be stored in the index.
+
+## Snapshots
+
+To snapshot, the underlying memory-mapped files in the AppendVec need to be flushed to disk. The index can be written out to disk as well.
+
+## Performance
+
+- Append-only writes are fast. SSDs and NVMEs, as well as all the OS level kernel data structures, allow for appends to run as fast as PCI or NVMe bandwidth will allow \(2,700 MB/s\).
+- Each replay and banking thread writes concurrently to its own AppendVec.
+- Each AppendVec could potentially be hosted on a separate NVMe.
+- Each replay and banking thread has concurrent read access to all the AppendVecs without blocking writes.
+- Index requires an exclusive write lock for writes. Single-thread performance for HashMap updates is on the order of 10m per second.
+- Banking and Replay stages should use 32 threads per NVMe. NVMes have optimal performance with 32 concurrent readers or writers.

package/src/implemented-proposals/readonly-accounts.md

@@ -0,0 +1,25 @@
+---
+title: Read-Only Accounts
+---
+
+This design covers the handling of readonly and writable accounts in the [runtime](../validator/runtime.md). Multiple transactions that modify the same account must be processed serially so that they are always replayed in the same order. Otherwise, this could introduce non-determinism to the ledger. Some transactions, however, only need to read, and not modify, the data in particular accounts. Multiple transactions that only read the same account can be processed in parallel, since replay order does not matter, providing a performance benefit.
+
+In order to identify readonly accounts, the transaction MessageHeader structure contains `num_readonly_signed_accounts` and `num_readonly_unsigned_accounts`. Instruction `program_ids` are included in the account vector as readonly, unsigned accounts, since executable accounts likewise cannot be modified during instruction processing.
+
+## Runtime handling
+
+Runtime transaction processing rules need to be updated slightly. Programs still can't write or spend accounts that they do not own. But new runtime rules ensure that readonly accounts cannot be modified, even by the programs that own them.
+
+Readonly accounts have the following property:
+
+- Read-only access to all account fields, including lamports (cannot be credited or debited), and account data
+
+Instructions that credit, debit, or modify the readonly account will fail.
+
+## Account Lock Optimizations
+
+The Accounts module keeps track of current locked accounts in the runtime, which separates readonly accounts from the writable accounts. The default account lock gives an account the "writable" designation, and can only be accessed by one processing thread at one time. Readonly accounts are locked by a separate mechanism, allowing for parallel reads.
+
+Although not yet implemented, readonly accounts could be cached in memory and shared between all the threads executing transactions. An ideal design would hold this cache while a readonly account is referenced by any transaction moving through the runtime, and release the cache when the last transaction exits the runtime.
+
+Readonly accounts could also be passed into the processor as references, saving an extra copy.

package/src/implemented-proposals/reliable-vote-transmission.md

@@ -0,0 +1,60 @@
+---
+title: Reliable Vote Transmission
+---
+
+Validator votes are messages that have a critical function for consensus and continuous operation of the network. Therefore it is critical that they are reliably delivered and encoded into the ledger.
+
+## Challenges
+
+1. Leader rotation is triggered by PoH, which is clock with high drift. So many nodes are likely to have an incorrect view if the next leader is active in realtime or not.
+2. The next leader may be easily be flooded. Thus a DDOS would not only prevent delivery of regular transactions, but also consensus messages.
+3. UDP is unreliable, and our asynchronous protocol requires any message that is transmitted to be retransmitted until it is observed in the ledger. Retransmittion could potentially cause an unintentional _thundering herd_ against the leader with a large number of validators. Worst case flood would be `(num_nodes * num_retransmits)`.
+4. Tracking if the vote has been transmitted or not via the ledger does not guarantee it will appear in a confirmed block. The current observed block may be unrolled. Validators would need to maintain state for each vote and fork.
+
+## Design
+
+1. Send votes as a push message through gossip. This ensures delivery of the vote to all the next leaders, not just the next future one.
+2. Leaders will read the Crds table for new votes and encode any new received votes into the blocks they propose. This allows for validator votes to be included in rollback forks by all the future leaders.
+3. Validators that receive votes in the ledger will add them to their local crds table, not as a push request, but simply add them to the table. This shortcuts the push message protocol, so the validation messages do not need to be retransmitted twice around the network.
+4. CrdsValue for vote should look like this `Votes(Vec<Transaction>)`
+
+Each vote transaction should maintain a `wallclock` in its data. The merge strategy for Votes will keep the last N set of votes as configured by the local client. For push/pull the vector is traversed recursively and each Transaction is treated as an individual CrdsValue with its own local wallclock and signature.
+
+Gossip is designed for efficient propagation of state. Messages that are sent through gossip-push are batched and propagated with a minimum spanning tree to the rest of the network. Any partial failures in the tree are actively repaired with the gossip-pull protocol while minimizing the amount of data transfered between any nodes.
+
+## How this design solves the Challenges
+
+1. Because there is no easy way for validators to be in sync with leaders on the leader's "active" state, gossip allows for eventual delivery regardless of that state.
+2. Gossip will deliver the messages to all the subsequent leaders, so if the current leader is flooded the next leader would have already received these votes and is able to encode them.
+3. Gossip minimizes the number of requests through the network by maintaining an efficient spanning tree, and using bloom filters to repair state. So retransmit back-off is not necessary and messages are batched.
+4. Leaders that read the crds table for votes will encode all the new valid votes that appear in the table. Even if this leader's block is unrolled, the next leader will try to add the same votes without any additional work done by the validator. Thus ensuring not only eventual delivery, but eventual encoding into the ledger.
+
+## Performance
+
+1. Worst case propagation time to the next leader is Log\(N\) hops with a base depending on the fanout. With our current default fanout of 6, it is about 6 hops to 20k nodes.
+2. The leader should receive 20k validation votes aggregated by gossip-push into MTU-sized shreds. Which would reduce the number of packets for 20k network to 80 shreds.
+3. Each validators votes is replicated across the entire network. To maintain a queue of 5 previous votes the Crds table would grow by 25 megabytes. `(20,000 nodes * 256 bytes * 5)`.
+
+## Two step implementation rollout
+
+Initially the network can perform reliably with just 1 vote transmitted and maintained through the network with the current Vote implementation. For small networks a fanout of 6 is sufficient. With small network the memory and push overhead is minor.
+
+### Sub 1k validator network
+
+1. Crds just maintains the validators latest vote.
+2. Votes are pushed and retransmitted regardless if they are appearing in the ledger.
+3. Fanout of 6.
+4. Worst case 256kb memory overhead per node.
+5. Worst case 4 hops to propagate to every node.
+6. Leader should receive the entire validator vote set in 4 push message shreds.
+
+### Sub 20k network
+
+Everything above plus the following:
+
+1. CRDS table maintains a vector of 5 latest validator votes.
+2. Votes encode a wallclock. CrdsValue::Votes is a type that recurses into the transaction vector for all the gossip protocols.
+3. Increase fanout to 20.
+4. Worst case 25mb memory overhead per node.
+5. Sub 4 hops worst case to deliver to the entire network.
+6. 80 shreds received by the leader for all the validator messages.

package/src/implemented-proposals/rent.md

@@ -0,0 +1,69 @@
+---
+title: Rent
+---
+
+Accounts on Solana may have owner-controlled state \(`Account::data`\) that's separate from the account's balance \(`Account::lamports`\). Since validators on the network need to maintain a working copy of this state in memory, the network charges a time-and-space based fee for this resource consumption, also known as Rent.
+
+## Two-tiered rent regime
+
+Accounts which maintain a minimum balance equivalent to 2 years of rent payments are exempt. The _2 years_ is drawn from the fact hardware cost drops by 50% in price every 2 years and the resulting convergence due to being a geometric series. Accounts whose balance falls below this threshold are charged rent at a rate specified in genesis, in lamports per byte-year. The network charges rent on a per-epoch basis, in credit for the next epoch, and `Account::rent_epoch` keeps track of the next time rent should be collected from the account.
+
+Currently, the rent cost is fixed at the genesis. However, it's anticipated to be dynamic, reflecting the underlying hardware storage cost at the time. So the price is generally expected to decrease as the hardware cost declines as the technology advances.
+
+## Timings of collecting rent
+
+There are two timings of collecting rent from accounts: \(1\) when referenced by a transaction, \(2\) periodically once an epoch. \(1\) includes the transaction to create the new account itself, and it happens during the normal transaction processing by the bank as part of the load phase. \(2\) exists to ensure to collect rents from stale accounts, which aren't referenced in recent epochs at all. \(2\) requires the whole scan of accounts and is spread over an epoch based on account address prefix to avoid load spikes due to this rent collection.
+
+On the contrary, rent collection isn't applied to accounts that are directly manipulated by any of protocol-level bookkeeping processes including:
+
+- The distribution of rent collection itself (Otherwise, it may cause recursive rent collection handling)
+- The distribution of staking rewards at the start of every epoch (To reduce as much as processing spike at the start of new epoch)
+- The distribution of transaction fee at the end of every slot
+
+Even if those processes are out of scope of rent collection, all of manipulated accounts will eventually be handled by the \(2\) mechanism.
+
+## Actual processing of collecting rent
+
+Rent is due for one epoch's worth of time, and accounts have `Account::rent_epoch` of `current_epoch` or `current_epoch + 1` depending on the rent regime.
+
+If the account is in the exempt regime, `Account::rent_epoch` is simply updated to `current_epoch`.
+
+If the account is non-exempt, the difference between the next epoch and `Account::rent_epoch` is used to calculate the amount of rent owed by this account \(via `Rent::due()`\). Any fractional lamports of the calculation are truncated. Rent due is deducted from `Account::lamports` and `Account::rent_epoch` is updated to `current_epoch + 1` (= next epoch). If the amount of rent due is less than one lamport, no changes are made to the account.
+
+Accounts whose balance is insufficient to satisfy the rent that would be due simply fail to load.
+
+A percentage of the rent collected is destroyed. The rest is distributed to validator accounts by stake weight, a la transaction fees, at the end of every slot.
+
+Finally, rent collection happens according to the protocol-level account updates like the rent distribution to validators, meaning there is no corresponding transaction for rent deductions. So, rent collection is rather invisible, only implicitly observable by a recent transaction or predetermined timing given its account address prefix.
+
+## Design considerations
+
+### Current design rationale
+
+Under the preceding design, it is NOT possible to have accounts that linger, never get touched, and never have to pay rent. Accounts always pay rent exactly once for each epoch, except rent-exempt, sysvar and executable accounts.
+
+This is an intended design choice. Otherwise, it would be possible to trigger unauthorized rent collection with `Noop` instruction by anyone who may unfairly profit from the rent (a leader at the moment) or save the rent given anticipated fluctuating rent cost.
+
+As another side-effect of this choice, also note that this periodic rent collection effectively forces validators not to store stale accounts into a cold storage optimistically and save the storage cost, which is unfavorable for account owners and may cause transactions on them to stall longer than others. On the flip side, this prevents malicious users from creating significant numbers of garbage accounts, burdening validators.
+
+As the overall consequence of this design, all accounts are stored equally as a validator's working set with the same performance characteristics, reflecting the uniform rent pricing structure.
+
+### Ad-hoc collection
+
+Collecting rent on an as-needed basis \(i.e. whenever accounts were loaded/accessed\) was considered. The issues with such an approach are:
+
+- accounts loaded as "credit only" for a transaction could very reasonably be expected to have rent due,
+
+  but would not be writable during any such transaction
+
+- a mechanism to "beat the bushes" \(i.e. go find accounts that need to pay rent\) is desirable,
+
+  lest accounts that are loaded infrequently get a free ride
+
+### System instruction for collecting rent
+
+Collecting rent via a system instruction was considered, as it would naturally have distributed rent to active and stake-weighted nodes and could have been done incrementally. However:
+
+- it would have adversely affected network throughput
+- it would require special-casing by the runtime, as accounts with non-SystemProgram owners may be debited by this instruction
+- someone would have to issue the transactions