solana-docs 0.0.0

package/src/developing/clients/rust-api.md

@@ -0,0 +1,36 @@
+---
+title: Rust API
+---
+
+Solana's Rust crates are [published to crates.io][crates.io] and can be found
+[on docs.rs with the "solana-" prefix][docs.rs].
+
+[crates.io]: https://crates.io/search?q=solana-
+[docs.rs]: https://docs.rs/releases/search?query=solana-
+
+Some important crates:
+
+- [`solana-program`] — Imported by programs running on Solana, compiled
+  to BPF. This crate contains many fundamental data types and is re-exported from
+  [`solana-sdk`], which cannot be imported from a Solana program.
+
+- [`solana-sdk`] — The basic off-chain SDK, it re-exports
+  [`solana-program`] and adds more APIs on top of that. Most Solana programs
+  that do not run on-chain will import this.
+
+- [`solana-client`] — For interacting with a Solana node via the
+  [JSON RPC API](jsonrpc-api).
+
+- [`solana-cli-config`] — Loading and saving the Solana CLI configuration
+  file.
+
+- [`solana-clap-utils`] — Routines for setting up a CLI, using [`clap`],
+  as used by the main Solana CLI. Includes functions for loading all types of
+  signers supported by the CLI.
+
+[`solana-program`]: https://docs.rs/solana-program
+[`solana-sdk`]: https://docs.rs/solana-sdk
+[`solana-client`]: https://docs.rs/solana-client
+[`solana-cli-config`]: https://docs.rs/solana-cli-config
+[`solana-clap-utils`]: https://docs.rs/solana-clap-utils
+[`clap`]: https://docs.rs/clap

package/src/developing/on-chain-programs/debugging.md

@@ -0,0 +1,110 @@
+---
+title: "Debugging"
+---
+
+Solana programs run on-chain, so debugging them in the wild can be challenging.
+To make debugging programs easier, developers can write unit tests that directly
+test their program's execution via the Solana runtime, or run a local cluster
+that will allow RPC clients to interact with their program.
+
+## Running unit tests
+
+- [Testing with Rust](developing-rust.md#how-to-test)
+- [Testing with C](developing-c.md#how-to-test)
+
+## Logging
+
+During program execution both the runtime and the program log status and error
+messages.
+
+For information about how to log from a program see the language specific
+documentation:
+
+- [Logging from a Rust program](developing-rust.md#logging)
+- [Logging from a C program](developing-c.md#logging)
+
+When running a local cluster the logs are written to stdout as long as they are
+enabled via the `RUST_LOG` log mask. From the perspective of program
+development it is helpful to focus on just the runtime and program logs and not
+the rest of the cluster logs. To focus in on program specific information the
+following log mask is recommended:
+
+`export RUST_LOG=solana_runtime::system_instruction_processor=trace,solana_runtime::message_processor=info,solana_bpf_loader=debug,solana_rbpf=debug`
+
+Log messages coming directly from the program (not the runtime) will be
+displayed in the form:
+
+`Program log: <user defined message>`
+
+## Error Handling
+
+The amount of information that can be communicated via a transaction error is
+limited but there are many points of possible failures. The following are
+possible failure points and information about what errors to expect and where to
+get more information:
+
+- The BPF loader may fail to parse the program, this should not happen since the
+  loader has already _finalized_ the program's account data.
+  - `InstructionError::InvalidAccountData` will be returned as part of the
+    transaction error.
+- The BPF loader may fail to setup the program's execution environment
+  - `InstructionError::Custom(0x0b9f_0001)` will be returned as part of the
+    transaction error. "0x0b9f_0001" is the hexadecimal representation of
+    [`VirtualMachineCreationFailed`](https://github.com/solana-labs/solana/blob/bc7133d7526a041d1aaee807b80922baa89b6f90/programs/bpf_loader/src/lib.rs#L44).
+- The BPF loader may have detected a fatal error during program executions
+  (things like panics, memory violations, system call errors, etc...)
+  - `InstructionError::Custom(0x0b9f_0002)` will be returned as part of the
+    transaction error. "0x0b9f_0002" is the hexadecimal representation of
+    [`VirtualMachineFailedToRunProgram`](https://github.com/solana-labs/solana/blob/bc7133d7526a041d1aaee807b80922baa89b6f90/programs/bpf_loader/src/lib.rs#L46).
+- The program itself may return an error
+  - `InstructionError::Custom(<user defined value>)` will be returned. The
+    "user defined value" must not conflict with any of the [builtin runtime
+    program
+    errors](https://github.com/solana-labs/solana/blob/bc7133d7526a041d1aaee807b80922baa89b6f90/sdk/program/src/program_error.rs#L87).
+    Programs typically use enumeration types to define error codes starting at
+    zero so they won't conflict.
+
+In the case of `VirtualMachineFailedToRunProgram` errors, more information about
+the specifics of what failed are written to the [program's execution
+logs](debugging.md#logging).
+
+For example, an access violation involving the stack will look something like
+this:
+
+`BPF program 4uQeVj5tqViQh7yWWGStvkEG1Zmhx6uasJtWCJziofM failed: out of bounds memory store (insn #615), addr 0x200001e38/8`
+
+## Monitoring Compute Budget Consumption
+
+The program can log the remaining number of compute units it will be allowed
+before program execution is halted. Programs can use these logs to wrap
+operations they wish to profile.
+
+- [Log the remaining compute units from a Rust
+  program](developing-rust.md#compute-budget)
+- [Log the remaining compute units from a C
+  program](developing-c.md#compute-budget)
+
+See [compute budget](developing/programming-model/runtime.md#compute-budget)
+for more information.
+
+## ELF Dump
+
+The BPF shared object internals can be dumped to a text file to gain more
+insight into a program's composition and what it may be doing at runtime.
+
+- [Create a dump file of a Rust program](developing-rust.md#elf-dump)
+- [Create a dump file of a C program](developing-c.md#elf-dump)
+
+## Instruction Tracing
+
+During execution the runtime BPF interpreter can be configured to log a trace
+message for each BPF instruction executed. This can be very helpful for things
+like pin-pointing the runtime context leading up to a memory access violation.
+
+The trace logs together with the [ELF dump](#elf-dump) can provide a lot of
+insight (though the traces produce a lot of information).
+
+To turn on BPF interpreter trace messages in a local cluster configure the
+`solana_rbpf` level in `RUST_LOG` to `trace`. For example:
+
+`export RUST_LOG=solana_rbpf=trace`

package/src/developing/on-chain-programs/deploying.md

@@ -0,0 +1,24 @@
+---
+title: "Deploying"
+---
+
+![SDK tools](/img/sdk-tools.svg)
+
+As shown in the diagram above, a program author creates a program, compiles it
+to an ELF shared object containing BPF bytecode, and uploads it to the Solana
+cluster with a special _deploy_ transaction. The cluster makes it available to
+clients via a _program ID_. The program ID is an _address_ specified when
+deploying and is used to reference the program in subsequent transactions.
+
+Upon a successful deployment the account that holds the program is marked
+executable.  If the program is marked "final", its account data become permanently
+immutable. If any changes are required to the finalized program (features, patches,
+etc...) the new program must be deployed to a new program ID.
+
+If a program is upgradeable, the account that holds the program is marked
+executable, but it is possible to redeploy a new shared object to the same
+program ID, provided that the program's upgrade authority signs the transaction.
+
+The Solana command line interface supports deploying programs, for more
+information see the [`deploy`](cli/usage.md#deploy-program) command line usage
+documentation.

package/src/developing/on-chain-programs/developing-c.md

@@ -0,0 +1,202 @@
+---
+title: "Developing with C"
+---
+
+Solana supports writing on-chain programs using the C and C++ programming
+languages.
+
+## Project Layout
+
+C projects are laid out as follows:
+
+```
+/src/<program name>
+/makefile
+```
+
+The `makefile` should contain the following:
+
+```bash
+OUT_DIR := <path to place to resulting shared object>
+include ~/.local/share/solana/install/active_release/bin/sdk/bpf/c/bpf.mk
+```
+
+The bpf-sdk may not be in the exact place specified above but if you setup your
+environment per [How to Build](#how-to-build) then it should be.
+
+Take a look at
+[helloworld](https://github.com/solana-labs/example-helloworld/tree/master/src/program-c)
+for an example of a C program.
+
+## How to Build
+
+First setup the environment:
+
+- Install the latest Rust stable from https://rustup.rs
+- Install the latest Solana command-line tools from
+  https://docs.solana.com/cli/install-solana-cli-tools
+
+Then build using make:
+
+```bash
+make -C <program directory>
+```
+
+## How to Test
+
+Solana uses the [Criterion](https://github.com/Snaipe/Criterion) test framework
+and tests are executed each time the program is built [How to
+Build](#how-to-build)].
+
+To add tests, create a new file next to your source file named `test_<program name>.c` and populate it with criterion test cases. For an example see the
+[helloworld C
+tests](https://github.com/solana-labs/example-helloworld/blob/master/src/program-c/src/helloworld/test_helloworld.c)
+or the [Criterion docs](https://criterion.readthedocs.io/en/master) for
+information on how to write a test case.
+
+## Program Entrypoint
+
+Programs export a known entrypoint symbol which the Solana runtime looks up and
+calls when invoking a program. Solana supports multiple [versions of the BPF
+loader](overview.md#versions) and the entrypoints may vary between them.
+Programs must be written for and deployed to the same loader. For more details
+see the [overview](overview#loaders).
+
+Currently there are two supported loaders [BPF
+Loader](https://github.com/solana-labs/solana/blob/7ddf10e602d2ed87a9e3737aa8c32f1db9f909d8/sdk/program/src/bpf_loader.rs#L17)
+and [BPF loader
+deprecated](https://github.com/solana-labs/solana/blob/7ddf10e602d2ed87a9e3737aa8c32f1db9f909d8/sdk/program/src/bpf_loader_deprecated.rs#L14)
+
+They both have the same raw entrypoint definition, the following is the raw
+symbol that the runtime looks up and calls:
+
+```c
+extern uint64_t entrypoint(const uint8_t *input)
+```
+
+This entrypoint takes a generic byte array which contains the serialized program
+parameters (program id, accounts, instruction data, etc...). To deserialize the
+parameters each loader contains its own [helper function](#Serialization).
+
+Refer to [helloworld's use of the
+entrypoint](https://github.com/solana-labs/example-helloworld/blob/bc0b25c0ccebeff44df9760ddb97011558b7d234/src/program-c/src/helloworld/helloworld.c#L37)
+as an example of how things fit together.
+
+### Serialization
+
+Refer to [helloworld's use of the deserialization
+function](https://github.com/solana-labs/example-helloworld/blob/bc0b25c0ccebeff44df9760ddb97011558b7d234/src/program-c/src/helloworld/helloworld.c#L43).
+
+Each loader provides a helper function that deserializes the program's input
+parameters into C types:
+
+- [BPF Loader
+  deserialization](https://github.com/solana-labs/solana/blob/d2ee9db2143859fa5dc26b15ee6da9c25cc0429c/sdk/bpf/c/inc/solana_sdk.h#L304)
+- [BPF Loader deprecated
+  deserialization](https://github.com/solana-labs/solana/blob/8415c22b593f164020adc7afe782e8041d756ddf/sdk/bpf/c/inc/deserialize_deprecated.h#L25)
+
+Some programs may want to perform deserialzaiton themselves and they can by
+providing their own implementation of the [raw entrypoint](#program-entrypoint).
+Take note that the provided deserialization functions retain references back to
+the serialized byte array for variables that the program is allowed to modify
+(lamports, account data). The reason for this is that upon return the loader
+will read those modifications so they may be committed. If a program implements
+their own deserialization function they need to ensure that any modifications
+the program wishes to commit must be written back into the input byte array.
+
+Details on how the loader serializes the program inputs can be found in the
+[Input Parameter Serialization](overview.md#input-parameter-serialization) docs.
+
+## Data Types
+
+The loader's deserialization helper function populates the
+[SolParameters](https://github.com/solana-labs/solana/blob/8415c22b593f164020adc7afe782e8041d756ddf/sdk/bpf/c/inc/solana_sdk.h#L276)
+structure:
+
+```c
+/**
+ * Structure that the program's entrypoint input data is deserialized into.
+ */
+typedef struct {
+  SolAccountInfo* ka; /** Pointer to an array of SolAccountInfo, must already
+                          point to an array of SolAccountInfos */
+  uint64_t ka_num; /** Number of SolAccountInfo entries in `ka` */
+  const uint8_t *data; /** pointer to the instruction data */
+  uint64_t data_len; /** Length in bytes of the instruction data */
+  const SolPubkey *program_id; /** program_id of the currently executing program */
+} SolParameters;
+```
+
+'ka' is an ordered array of the accounts referenced by the instruction and
+represented as a
+[SolAccountInfo](https://github.com/solana-labs/solana/blob/8415c22b593f164020adc7afe782e8041d756ddf/sdk/bpf/c/inc/solana_sdk.h#L173)
+structures. An account's place in the array signifies its meaning, for example,
+when transferring lamports an instruction may define the first account as the
+source and the second as the destination.
+
+The members of the `SolAccountInfo` structure are read-only except for
+`lamports` and `data`. Both may be modified by the program in accordance with
+the [runtime enforcement
+policy](developing/programming-model/accounts.md#policy). When an instruction
+reference the same account multiple times there may be duplicate
+`SolAccountInfo` entries in the array but they both point back to the original
+input byte array. A program should handle these case delicately to avoid
+overlapping read/writes to the same buffer. If a program implements their own
+deserialization function care should be taken to handle duplicate accounts
+appropriately.
+
+`data` is the general purpose byte array from the [instruction's instruction
+data](developing/programming-model/transactions.md#instruction-data) being
+processed.
+
+`program_id` is the public key of the currently executing program.
+
+## Heap
+
+C programs can allocate memory via the system call
+[`calloc`](https://github.com/solana-labs/solana/blob/c3d2d2134c93001566e1e56f691582f379b5ae55/sdk/bpf/c/inc/solana_sdk.h#L245)
+or implement their own heap on top of the 32KB heap region starting at virtual
+address x300000000. The heap region is also used by `calloc` so if a program
+implements their own heap it should not also call `calloc`.
+
+## Logging
+
+The runtime provides two system calls that take data and log it to the program
+logs.
+
+- [`sol_log(const char*)`](https://github.com/solana-labs/solana/blob/d2ee9db2143859fa5dc26b15ee6da9c25cc0429c/sdk/bpf/c/inc/solana_sdk.h#L128)
+- [`sol_log_64(uint64_t, uint64_t, uint64_t, uint64_t, uint64_t)`](https://github.com/solana-labs/solana/blob/d2ee9db2143859fa5dc26b15ee6da9c25cc0429c/sdk/bpf/c/inc/solana_sdk.h#L134)
+
+The [debugging](debugging.md#logging) section has more information about working
+with program logs.
+
+## Compute Budget
+
+Use the system call
+[`sol_log_compute_units()`](https://github.com/solana-labs/solana/blob/d3a3a7548c857f26ec2cb10e270da72d373020ec/sdk/bpf/c/inc/solana_sdk.h#L140)
+to log a message containing the remaining number of compute units the program
+may consume before execution is halted
+
+See [compute budget](developing/programming-model/runtime.md#compute-budget)
+for more information.
+
+## ELF Dump
+
+The BPF shared object internals can be dumped to a text file to gain more
+insight into a program's composition and what it may be doing at runtime. The
+dump will contain both the ELF information as well as a list of all the symbols
+and the instructions that implement them. Some of the BPF loader's error log
+messages will reference specific instruction numbers where the error occurred.
+These references can be looked up in the ELF dump to identify the offending
+instruction and its context.
+
+To create a dump file:
+
+```bash
+$ cd <program directory>
+$ make dump_<program name>
+```
+
+## Examples
+
+The [Solana Program Library github](https://github.com/solana-labs/solana-program-library/tree/master/examples/c) repo contains a collection of C examples