solana-docs 0.0.0

package/src/developing/programming-model/accounts.md

@@ -0,0 +1,231 @@
+---
+title: "Accounts"
+---
+
+## Storing State between Transactions
+
+If the program needs to store state between transactions, it does so using
+_accounts_. Accounts are similar to files in operating systems such as Linux in
+that they may hold arbitrary data that persists beyond
+the lifetime of a program. Also like a file, an account includes metadata that
+tells the runtime who is allowed to access the data and how.
+
+Unlike a file, the account includes metadata for the lifetime of the file. That
+lifetime is expressed by a number of fractional native
+tokens called _lamports_. Accounts are held in validator memory and pay
+["rent"](#rent) to stay there. Each validator periodically scans all accounts
+and collects rent. Any account that drops to zero lamports is purged. Accounts
+can also be marked [rent-exempt](#rent-exemption) if they contain a sufficient
+number of lamports.
+
+In the same way that a Linux user uses a path to look up a file, a Solana client
+uses an _address_ to look up an account. The address is a 256-bit public key.
+
+## Signers
+
+Transactions may include digital [signatures](terminology.md#signature)
+corresponding to the accounts' public keys referenced by the transaction. Such
+signatures signify that the holder of the
+account's private key signed and thus "authorized" the transaction.  In this case,
+the account is referred to as a _signer_. Whether an account is a signer or not
+is communicated to the program as part of the account's metadata. Programs can
+then use that information to make authority decisions.
+
+## Read-only
+
+Transactions can [indicate](transactions.md#message-header-format) that some of
+the accounts it references be treated as _read-only accounts_ in order to enable
+parallel account processing between transactions. The runtime permits read-only
+accounts to be read concurrently by multiple programs. If a program attempts to
+modify a read-only account, the transaction is rejected by the runtime.
+
+## Executable
+
+If an account is marked "executable" in its metadata, then it is considered a
+program which can be executed by including the account's public key in an
+instruction's [program id](transactions.md#program-id). Accounts are marked as
+executable during a successful program deployment process by the loader that
+owns the account. When a program is deployed to the execution engine (BPF deployment),
+the loader determines that the bytecode in the account's data is valid.
+If so, the loader permanently marks the program account as executable.
+
+If a program is marked as final (non-upgradeable), the runtime enforces that the
+account's data (the program) is immutable. Through the upgradeable loader, it is
+possible to upload a totally new program to an existing program address.
+
+## Creating
+
+To create an account, a client generates a _keypair_ and registers its public key
+using the `SystemProgram::CreateAccount` instruction with a fixed
+storage size in bytes preallocated.
+The current maximum size of an account's data is 10 megabytes.
+
+An account address can be any arbitrary 256 bit value, and there are mechanisms
+for advanced users to create derived addresses
+(`SystemProgram::CreateAccountWithSeed`,
+[`Pubkey::CreateProgramAddress`](calling-between-programs.md#program-derived-addresses)).
+
+Accounts that have never been created via the system program can also be passed
+to programs. When an instruction references an account that hasn't been
+previously created, the program will be passed an account with no data and zero lamports
+that is owned by the system program.
+
+Such newly created accounts reflect
+whether they sign the transaction and therefore can be used as an
+authority. Authorities in this context convey to the program that the holder of
+the private key associated with the account's public key signed the transaction.
+The account's public key may be known to the program or recorded in another
+account, signifying some kind of ownership or authority over an asset or
+operation the program controls or performs.
+
+## Ownership and Assignment to Programs
+
+A created account is initialized to be _owned_ by a built-in program called the
+System program and is called a _system account_ aptly. An account includes
+"owner" metadata. The owner is a program id. The runtime grants the program
+write access to the account if its id matches the owner. For the case of the
+System program, the runtime allows clients to transfer lamports and importantly
+_assign_ account ownership, meaning changing owner to different program id. If
+an account is not owned by a program, the program is only permitted to read its
+data and credit the account.
+
+## Verifying validity of unmodified, reference-only accounts
+
+For security purposes, it is recommended that programs check the validity of any
+account it reads but does not modify.
+
+This is because a malicious user
+could create accounts with arbitrary data and then pass these accounts to the
+program in place of valid accounts. The arbitrary data could be crafted in
+a way that leads to unexpected or harmful program behavior.
+
+The security model enforces that an account's data can only be modified by the
+account's `Owner` program. This allows the program to trust that the data
+passed to them via accounts they own. The
+runtime enforces this by rejecting any transaction containing a program that
+attempts to write to an account it does not own.
+
+If a program were to not check account validity, it might read an account
+it thinks it owns but doesn't.  Anyone can
+issue instructions to a program, and the runtime does not know that those
+accounts are expected to be owned by the program.
+
+To check an account's validity, the program should either check the account's
+address against a known value or check that the account is indeed owned
+correctly (usually owned by the program itself).
+
+One example is when programs use a sysvar account. Unless the program checks the
+account's address or owner, it's impossible to be sure whether it's a real and
+valid sysvar account merely by successful deserialization of the account's data.
+
+Accordingly, the Solana SDK [checks the sysvar account's validity during
+deserialization](https://github.com/solana-labs/solana/blob/a95675a7ce1651f7b59443eb146b356bc4b3f374/sdk/program/src/sysvar/mod.rs#L65).
+A alternative and safer way to read a sysvar is via the sysvar's [`get()`
+function](https://github.com/solana-labs/solana/blob/64bfc14a75671e4ec3fe969ded01a599645080eb/sdk/program/src/sysvar/mod.rs#L73)
+which doesn't require these checks.
+
+If the program always modifies the account in question, the address/owner check
+isn't required because modifying an unowned account will be rejected by the runtime,
+and the containing transaction will be thrown out.
+
+## Rent
+
+Keeping accounts alive on Solana incurs a storage cost called _rent_ because the
+blockchain cluster must actively maintain the data to process any future transactions.
+This is different from Bitcoin and Ethereum, where storing accounts doesn't
+incur any costs.
+
+The rent is debited from an account's balance by the runtime upon the first
+access (including the initial account creation) in the current epoch by
+transactions or once per an epoch if there are no transactions. The fee is
+currently a fixed rate, measured in bytes-times-epochs. The fee may change in
+the future.
+
+For the sake of simple rent calculation, rent is always collected for a single,
+full epoch. Rent is not pro-rated, meaning there are neither fees nor refunds
+for partial epochs. This means that, on account creation, the first rent
+collected isn't for the current partial epoch, but collected up front for the
+next full epoch. Subsequent rent collections are for further future epochs. On
+the other end, if the balance of an already-rent-collected account drops below
+another rent fee mid-epoch, the account will continue to exist through the
+current epoch and be purged immediately at the start of the upcoming epoch.
+
+Accounts can be exempt from paying rent if they maintain a minimum balance. This
+rent-exemption is described below.
+
+### Calculation of rent
+
+Note: The rent rate can change in the future.
+
+As of writing, the fixed rent fee is 19.055441478439427 lamports per byte-epoch
+on the testnet and mainnet-beta clusters. An [epoch](terminology.md#epoch) is
+targeted to be 2 days (For devnet, the rent fee is 0.3608183131797095 lamports
+per byte-epoch with its 54m36s-long epoch).
+
+This value is calculated to target 0.01 SOL per mebibyte-day (exactly matching
+to 3.56 SOL per mebibyte-year):
+
+```text
+Rent fee: 19.055441478439427 = 10_000_000 (0.01 SOL) * 365(approx. day in a year) / (1024 * 1024)(1 MiB) / (365.25/2)(epochs in 1 year)
+```
+
+And rent calculation is done with the `f64` precision and the final result is
+truncated to `u64` in lamports.
+
+The rent calculation includes account metadata (address, owner, lamports, etc)
+in the size of an account. Therefore the smallest an account can be for rent
+calculations is 128 bytes.
+
+For example, an account is created with the initial transfer of 10,000 lamports
+and no additional data. Rent is immediately debited from it on creation,
+resulting in a balance of 7,561 lamports:
+
+```text
+Rent: 2,439 = 19.055441478439427 (rent rate) * 128 bytes (minimum account size) * 1 (epoch)
+Account Balance: 7,561 = 10,000 (transfered lamports) - 2,439 (this account's rent fee for an epoch)
+```
+
+The account balance will be reduced to 5,122 lamports at the next epoch even if
+there is no activity:
+
+```text
+Account Balance: 5,122 = 7,561 (current balance) - 2,439 (this account's rent fee for an epoch)
+```
+
+Accordingly, a minimum-size account will be immediately removed after creation
+if the transferred lamports are less than or equal to 2,439.
+
+### Rent exemption
+
+Alternatively, an account can be made entirely exempt from rent collection by
+depositing at least 2 years worth of rent. This is checked every time an
+account's balance is reduced, and rent is immediately debited once the balance
+goes below the minimum amount.
+
+Program executable accounts are required by the runtime to be rent-exempt to
+avoid being purged.
+
+Note: Use the [`getMinimumBalanceForRentExemption` RPC
+endpoint](developing/clients/jsonrpc-api.md#getminimumbalanceforrentexemption) to calculate the
+minimum balance for a particular account size. The following calculation is
+illustrative only.
+
+For example, a program executable with the size of 15,000 bytes requires a
+balance of 105,290,880 lamports (=~ 0.105 SOL) to be rent-exempt:
+
+```text
+105,290,880 = 19.055441478439427 (fee rate) * (128 + 15_000)(account size including metadata) * ((365.25/2) * 2)(epochs in 2 years)
+```
+
+Rent can also be estimated via the [`solana rent` CLI subcommand](cli/usage.md#solana-rent)
+
+```text
+$ solana rent 15000
+Rent per byte-year: 0.00000348 SOL
+Rent per epoch: 0.000288276 SOL
+Rent-exempt minimum: 0.10529088 SOL
+```
+
+Note: Rest assured that, should the storage rent rate need to be increased at some
+point in the future, steps will be taken to ensure that accounts that are rent-exempt
+before the increase will remain rent-exempt afterwards

package/src/developing/programming-model/calling-between-programs.md

@@ -0,0 +1,351 @@
+---
+title: Calling Between Programs
+---
+
+## Cross-Program Invocations
+
+The Solana runtime allows programs to call each other via a mechanism called
+cross-program invocation. Calling between programs is achieved by one program
+invoking an instruction of the other. The invoking program is halted until the
+invoked program finishes processing the instruction.
+
+For example, a client could create a transaction that modifies two accounts,
+each owned by separate on-chain programs:
+
+```rust,ignore
+let message = Message::new(vec![
+    token_instruction::pay(&alice_pubkey),
+    acme_instruction::launch_missiles(&bob_pubkey),
+]);
+client.send_and_confirm_message(&[&alice_keypair, &bob_keypair], &message);
+```
+
+A client may instead allow the `acme` program to conveniently invoke `token`
+instructions on the client's behalf:
+
+```rust,ignore
+let message = Message::new(vec![
+    acme_instruction::pay_and_launch_missiles(&alice_pubkey, &bob_pubkey),
+]);
+client.send_and_confirm_message(&[&alice_keypair, &bob_keypair], &message);
+```
+
+Given two on-chain programs `token` and `acme`, each implementing instructions
+`pay()` and `launch_missiles()` respectively, acme can be implemented with a
+call to a function defined in the `token` module by issuing a cross-program
+invocation:
+
+```rust,ignore
+mod acme {
+    use token_instruction;
+
+    fn launch_missiles(accounts: &[AccountInfo]) -> Result<()> {
+        ...
+    }
+
+    fn pay_and_launch_missiles(accounts: &[AccountInfo]) -> Result<()> {
+        let alice_pubkey = accounts[1].key;
+        let instruction = token_instruction::pay(&alice_pubkey);
+        invoke(&instruction, accounts)?;
+
+        launch_missiles(accounts)?;
+    }
+```
+
+`invoke()` is built into Solana's runtime and is responsible for routing the
+given instruction to the `token` program via the instruction's `program_id`
+field.
+
+Note that `invoke` requires the caller to pass all the accounts required by the
+instruction being invoked. This means that both the executable account (the
+ones that matches the instruction's program id) and the accounts passed to the
+instruction processor.
+
+Before invoking `pay()`, the runtime must ensure that `acme` didn't modify any
+accounts owned by `token`. It does this by applying the runtime's policy to the
+current state of the accounts at the time `acme` calls `invoke` vs. the initial
+state of the accounts at the beginning of the `acme`'s instruction. After
+`pay()` completes, the runtime must again ensure that `token` didn't modify any
+accounts owned by `acme` by again applying the runtime's policy, but this time
+with the `token` program ID. Lastly, after `pay_and_launch_missiles()`
+completes, the runtime must apply the runtime policy one more time, where it
+normally would, but using all updated `pre_*` variables. If executing
+`pay_and_launch_missiles()` up to `pay()` made no invalid account changes,
+`pay()` made no invalid changes, and executing from `pay()` until
+`pay_and_launch_missiles()` returns made no invalid changes, then the runtime
+can transitively assume `pay_and_launch_missiles()` as whole made no invalid
+account changes, and therefore commit all these account modifications.
+
+### Instructions that require privileges
+
+The runtime uses the privileges granted to the caller program to determine what
+privileges can be extended to the callee. Privileges in this context refer to
+signers and writable accounts. For example, if the instruction the caller is
+processing contains a signer or writable account, then the caller can invoke an
+instruction that also contains that signer and/or writable account.
+
+This privilege extension relies on the fact that programs are immutable, except
+during the special case of program upgrades.
+
+In the case of the `acme` program, the runtime can safely treat the transaction's
+signature as a signature of a `token` instruction. When the runtime sees the
+`token` instruction references `alice_pubkey`, it looks up the key in the `acme`
+instruction to see if that key corresponds to a signed account. In this case, it
+does and thereby authorizes the `token` program to modify Alice's account.
+
+### Program signed accounts
+
+Programs can issue instructions that contain signed accounts that were not
+signed in the original transaction by using [Program derived
+addresses](#program-derived-addresses).
+
+To sign an account with program derived addresses, a program may
+`invoke_signed()`.
+
+```rust,ignore
+        invoke_signed(
+            &instruction,
+            accounts,
+            &[&["First addresses seed"],
+              &["Second addresses first seed", "Second addresses second seed"]],
+        )?;
+```
+
+### Call Depth
+
+Cross-program invocations allow programs to invoke other programs directly but
+the depth is constrained currently to 4.
+
+### Reentrancy
+
+Reentrancy is currently limited to direct self recursion capped at a fixed
+depth. This restriction prevents situations where a program might invoke another
+from an intermediary state without the knowledge that it might later be called
+back into. Direct recursion gives the program full control of its state at the
+point that it gets called back.
+
+## Program Derived Addresses
+
+Program derived addresses allow programmatically generated signatures to be used
+when [calling between programs](#cross-program-invocations).
+
+Using a program derived address, a program may be given the authority over an
+account and later transfer that authority to another. This is possible because
+the program can act as the signer in the transaction that gives authority.
+
+For example, if two users want to make a wager on the outcome of a game in
+Solana, they must each transfer their wager's assets to some intermediary that
+will honor their agreement. Currently, there is no way to implement this
+intermediary as a program in Solana because the intermediary program cannot
+transfer the assets to the winner.
+
+This capability is necessary for many DeFi applications since they require
+assets to be transferred to an escrow agent until some event occurs that
+determines the new owner.
+
+- Decentralized Exchanges that transfer assets between matching bid and ask
+  orders.
+
+- Auctions that transfer assets to the winner.
+
+- Games or prediction markets that collect and redistribute prizes to the
+  winners.
+
+Program derived address:
+
+1. Allow programs to control specific addresses, called program addresses, in
+   such a way that no external user can generate valid transactions with
+   signatures for those addresses.
+
+2. Allow programs to programmatically sign for program addresses that are
+   present in instructions invoked via [Cross-Program Invocations](#cross-program-invocations).
+
+Given the two conditions, users can securely transfer or assign the authority of
+on-chain assets to program addresses and the program can then assign that
+authority elsewhere at its discretion.
+
+### Private keys for program addresses
+
+A Program address does not lie on the ed25519 curve and therefore has no valid
+private key associated with it, and thus generating a signature for it is
+impossible. While it has no private key of its own, it can be used by a program
+to issue an instruction that includes the Program address as a signer.
+
+### Hash-based generated program addresses
+
+Program addresses are deterministically derived from a collection of seeds and a
+program id using a 256-bit pre-image resistant hash function. Program address
+must not lie on the ed25519 curve to ensure there is no associated private key.
+During generation an error will be returned if the address is found to lie on
+the curve. There is about a 50/50 chance of this happening for a given
+collection of seeds and program id. If this occurs a different set of seeds or
+a seed bump (additional 8 bit seed) can be used to find a valid program address
+off the curve.
+
+Deterministic program addresses for programs follow a similar derivation path as
+Accounts created with `SystemInstruction::CreateAccountWithSeed` which is
+implemented with `Pubkey::create_with_seed`.
+
+For reference that implementation is as follows:
+
+```rust,ignore
+pub fn create_with_seed(
+    base: &Pubkey,
+    seed: &str,
+    program_id: &Pubkey,
+) -> Result<Pubkey, SystemError> {
+    if seed.len() > MAX_ADDRESS_SEED_LEN {
+        return Err(SystemError::MaxSeedLengthExceeded);
+    }
+
+    Ok(Pubkey::new(
+        hashv(&[base.as_ref(), seed.as_ref(), program_id.as_ref()]).as_ref(),
+    ))
+}
+```
+
+Programs can deterministically derive any number of addresses by using seeds.
+These seeds can symbolically identify how the addresses are used.
+
+From `Pubkey`::
+
+```rust,ignore
+/// Generate a derived program address
+///     * seeds, symbolic keywords used to derive the key
+///     * program_id, program that the address is derived for
+pub fn create_program_address(
+    seeds: &[&[u8]],
+    program_id: &Pubkey,
+) -> Result<Pubkey, PubkeyError>
+
+/// Find a valid off-curve derived program address and its bump seed
+///     * seeds, symbolic keywords used to derive the key
+///     * program_id, program that the address is derived for
+pub fn find_program_address(
+    seeds: &[&[u8]],
+    program_id: &Pubkey,
+) -> Option<(Pubkey, u8)> {
+    let mut bump_seed = [std::u8::MAX];
+    for _ in 0..std::u8::MAX {
+        let mut seeds_with_bump = seeds.to_vec();
+        seeds_with_bump.push(&bump_seed);
+        if let Ok(address) = create_program_address(&seeds_with_bump, program_id) {
+            return Some((address, bump_seed[0]));
+        }
+        bump_seed[0] -= 1;
+    }
+    None
+}
+```
+
+### Using program addresses
+
+Clients can use the `create_program_address` function to generate a destination
+address. In this example, we assume that
+`create_program_address(&[&["escrow"]], &escrow_program_id)` generates a valid
+program address that is off the curve.
+
+```rust,ignore
+// deterministically derive the escrow key
+let escrow_pubkey = create_program_address(&[&["escrow"]], &escrow_program_id);
+
+// construct a transfer message using that key
+let message = Message::new(vec![
+    token_instruction::transfer(&alice_pubkey, &escrow_pubkey, 1),
+]);
+
+// process the message which transfer one 1 token to the escrow
+client.send_and_confirm_message(&[&alice_keypair], &message);
+```
+
+Programs can use the same function to generate the same address. In the function
+below the program issues a `token_instruction::transfer` from a program address
+as if it had the private key to sign the transaction.
+
+```rust,ignore
+fn transfer_one_token_from_escrow(
+    program_id: &Pubkey,
+    accounts: &[AccountInfo],
+) -> ProgramResult {
+    // User supplies the destination
+    let alice_pubkey = keyed_accounts[1].unsigned_key();
+
+    // Deterministically derive the escrow pubkey.
+    let escrow_pubkey = create_program_address(&[&["escrow"]], program_id);
+
+    // Create the transfer instruction
+    let instruction = token_instruction::transfer(&escrow_pubkey, &alice_pubkey, 1);
+
+    // The runtime deterministically derives the key from the currently
+    // executing program ID and the supplied keywords.
+    // If the derived address matches a key marked as signed in the instruction
+    // then that key is accepted as signed.
+    invoke_signed(&instruction, accounts, &[&["escrow"]])
+}
+```
+
+Note that the address generated using `create_program_address` is not guaranteed
+to be a valid program address off the curve. For example, let's assume that the
+seed `"escrow2"` does not generate a valid program address.
+
+To generate a valid program address using `"escrow2` as a seed, use
+`find_program_address`, iterating through possible bump seeds until a valid
+combination is found. The preceding example becomes:
+
+```rust,ignore
+// find the escrow key and valid bump seed
+let (escrow_pubkey2, escrow_bump_seed) = find_program_address(&[&["escrow2"]], &escrow_program_id);
+
+// construct a transfer message using that key
+let message = Message::new(vec![
+    token_instruction::transfer(&alice_pubkey, &escrow_pubkey2, 1),
+]);
+
+// process the message which transfer one 1 token to the escrow
+client.send_and_confirm_message(&[&alice_keypair], &message);
+```
+
+Within the program, this becomes:
+
+```rust,ignore
+fn transfer_one_token_from_escrow2(
+    program_id: &Pubkey,
+    accounts: &[AccountInfo],
+) -> ProgramResult {
+    // User supplies the destination
+    let alice_pubkey = keyed_accounts[1].unsigned_key();
+
+    // Iteratively derive the escrow pubkey
+    let (escrow_pubkey2, bump_seed) = find_program_address(&[&["escrow2"]], program_id);
+
+    // Create the transfer instruction
+    let instruction = token_instruction::transfer(&escrow_pubkey2, &alice_pubkey, 1);
+
+    // Include the generated bump seed to the list of all seeds
+    invoke_signed(&instruction, accounts, &[&["escrow2", &[bump_seed]]])
+}
+```
+
+Since `find_program_address` requires iterating over a number of calls to
+`create_program_address`, it may use more
+[compute budget](developing/programming-model/runtime.md#compute-budget) when
+used on-chain. To reduce the compute cost, use `find_program_address` off-chain
+and pass the resulting bump seed to the program.
+
+### Instructions that require signers
+
+The addresses generated with `create_program_address` and `find_program_address`
+are indistinguishable from any other public key. The only way for the runtime to
+verify that the address belongs to a program is for the program to supply the
+seeds used to generate the address.
+
+The runtime will internally call `create_program_address`, and compare the
+result against the addresses supplied in the instruction.
+
+## Examples
+
+Refer to [Developing with
+Rust](developing/on-chain-programs/../../../on-chain-programs/developing-rust.md#examples)
+and [Developing with
+C](developing/on-chain-programs/../../../on-chain-programs/developing-c.md#examples)
+for examples of how to use cross-program invocation.

package/src/developing/programming-model/overview.md

@@ -0,0 +1,17 @@
+---
+title: "Overview"
+---
+
+An [app](terminology.md#app) interacts with a Solana cluster by sending it
+[transactions](transactions.md) with one or more
+[instructions](transactions.md#instructions). The Solana [runtime](runtime.md)
+passes those instructions to [programs](terminology.md#program) deployed by app
+developers beforehand. An instruction might, for example, tell a program to
+transfer [lamports](terminology.md#lamport) from one [account](accounts.md) to
+another or create an interactive contract that governs how lamports are
+transferred. Instructions are executed sequentially and atomically for each
+transaction. If any instruction is invalid, all account changes in the
+transaction are discarded.
+
+To start developing immediately you can build, deploy, and run one of the
+[examples](developing/on-chain-programs/examples.md).