@xgent-ai/skills 0.1.0 → 0.3.0

package/skills/rust/references/chapter_03.md

@@ -0,0 +1,209 @@
+# Chapter 3 - Performance Mindset
+
+The **golden rule** of performance work:
+
+> Don't guess, measure.
+
+Rust code is often already pretty fast - don't "optimize" without evidence. Optimize only after finding bottlenecks.
+
+### A good first steps
+* Use `--release` flag on you builds (might sound dummy, but it is quite common to hear people complaining that their Rust code is slower than their X language code, and 99% of the time is because they didn't use the `--release` flag).
+* `$ cargo clippy -- -D clippy::perf` gives you important tips on best practices for performance.
+* [`cargo bench`](https://doc.rust-lang.org/cargo/commands/cargo-bench.html) is a cargo tool to create micro-benchmarks and test different code solutions. Write a test scenario and bench you solution against the original code, if your improvement is larger than 5%, might be a good performance improvement.
+* [`cargo flamegraph`](https://github.com/flamegraph-rs/flamegraph) a powerful profiler for Rust code. For MacOS, [samply](https://github.com/mstange/samply) might be a better DX option.
+
+> #### Further reading on Benchmarking:
+> - [How to build a Custom Benchmarking Harness in Rust](https://bencher.dev/learn/benchmarking/rust/custom-harness/)
+
+
+## 3.1 Flamegraph
+
+Flamegraph helps you visualize how much time CPU spent on each task.
+
+```shell
+# Installing flamegraph
+cargo install flamegraph
+
+# cargo support provided through the cargo-flamegraph binary!
+# defaults to profiling cargo run --release
+cargo flamegraph
+
+# by default, `--release` profile is used,
+# but you can override this:
+cargo flamegraph --dev
+
+# if you'd like to profile a specific binary:
+cargo flamegraph --bin=stress2
+
+# Profile unit tests.
+# Note that a separating `--` is necessary if `--unit-test` is the last flag.
+cargo flamegraph --unit-test -- test::in::package::with::single::crate
+cargo flamegraph --unit-test crate_name -- test::in::package::with::multiple:crate
+
+# Profile integration tests.
+cargo flamegraph --test test_name
+
+# Run criterion benchmark
+# Note that the last --bench is required for `criterion 0.3` to run in benchmark mode, instead of test mode.
+cargo flamegraph --bench some_benchmark --features some_features -- --bench
+
+# Run workspace example
+cargo flamegraph --example some_example --features some_features
+```
+
+> ❗ Always run your profiles with `--release` enabled, the `--dev` flag isn't realistic as it doesn't have optimizations enabled.
+
+The result will look like a flame graph where:
+
+* The `y-axis` shows the **stack depth number**. When looking at a flamegraph, the main function of your program will be closer to the bottom, and the called functions will be stacked on top, with the functions that they call stacked on top of them.
+
+* The `width of each box` shows the **total time that that function** is on the CPU or is part of the call stack. If a function's box is wider than others, that means that it consumes more CPU per execution than other functions, or that it is called more than other functions.
+
+> ❗ The **color of each box** isn't significant, and **is chosen at random**.
+
+### 🚨 Remember
+* Thick stacks: heavy CPU usage
+* Thin stacks: low intensity (cheap)
+
+## 3.2 Avoid Redundant Cloning
+
+> Cloning is cheap... **until it isn't**
+
+In sections [Borrowing over Cloning](./chapter_01.md#11-borrowing-over-cloning) and [Important Clippy lints to respect](./chapter_02.md#23-important-clippy-lints-to-respect) we mentioned the impacts of cloning and the relevant clippy lint [`redundant_clone`](https://rust-lang.github.io/rust-clippy/master/#redundant_clone), so in this section we will explore a bit "when to pass ownership".
+
+* 🚨 If you really need to clone, leave it to the last moment.
+
+### When to pass ownership?
+
+* Only `.clone()` if you truly need a new owned copy. A few examples:
+    * Crate API Design requires owned data.
+    * Have overloaded `std::ops` but still need ownership to the old data:
+    ```rust
+    use std::ops::Add;
+
+    #[derive(Debug, Copy, Clone, PartialEq)]
+    struct Point {
+        x: i32,
+        y: i32,
+    }
+
+    impl Add for Point {
+        type Output = Self;
+
+        fn add(self, other: Self) -> Self {
+            Self {
+                x: self.x + other.x,
+                y: self.y + other.y,
+            }
+        }
+    }
+
+    assert_eq!(Point { x: 1, y: 0 } + Point { x: 2, y: 3 },
+               Point { x: 3, y: 3 });
+    ```
+    * Need to do comparison snapshots or due to API you need multiple owned instances of the data.
+    ```rust
+    fn snapshot(a: &MyValue, b:&MyValue) -> MyValueDiff {
+        a - b
+    }
+
+    impl Sub for MyValue {
+        type Output = MyValueDiff;
+
+        fn sub(self, other: Self) -> MyValue {
+            ...
+        }
+    }
+
+    fn main() {
+        let mut a = MyValue::default();
+        let b = a.clone();
+
+        a.magical_update();
+        println!("{:?}", snapshot(&a, &b));
+    }
+    ```
+* You have reference counted pointers (`Arc, Rc`).
+* You have small structs that are to big to `Copy` but as costly as `std::collections`. An example is HTTP client like `hyper_util::client::legacy::Client` that cloning allows you to share the connection pool.
+* You have a chained struct modifier that needs owned mutation, some **builders** require owned mutation, but most custom builders can be done with `pub fn with_xyz(&mut self, value: Xyz) -> &mut Self`.
+```rust
+// Inline `HashMap` insertion extension
+
+fn insert_owned(mut self, key: K, value: V) -> Self {
+    self.insert(key, value);
+    self
+}
+```
+* Ownership can also be a good way to model business logic / state. For example:
+```rust
+let not_validated: String = ...;// some user source
+let validated = Validate::try_from(not_validated)?;
+// Technically that `try_from` maybe didn't need ownership, but taking it lets us model intent
+```
+
+### When **NOT** to pass ownership?
+
+* Prefer API designs that take reference (`fn process(values: &[T])`), instead of ownership (`fn process(values: Vec<T>)`).
+* If you only need read access to elements, prefer `.iter` or slices:
+```rust
+for item in &some_vec {
+    ...
+}
+```
+* You need to mutate data that is owned by another thread, use `&mut MyStruct`.
+
+### Use `Cow` for `Maybe Owned` data
+
+Sometimes you don't actually need owned data, but that is not clear from the API perspective, so using [`std::borrow::Cow`](https://doc.rust-lang.org/std/borrow/enum.Cow.html) is a way to efficiently address this case:
+
+```rust
+use std::borrow::Cow;
+
+fn hello_greet(name: Cow<'_, str>) {
+    println!("Hello {name}");
+}
+
+hello_greet(Cow::Borrowed("Julia"));
+hello_greet(Cow::Owned("Naomi".to_string()));
+```
+
+## 3.3 Stack vs Heap: Be size-smart!
+
+### ✅ Good Practices 
+
+* Keep small types (`impl Copy`, `usize`, `bool`, etc) **on the stack**.
+* Avoid passing huge types (`> 512 bytes`) by value or transferring ownership. Prefer pass by reference (e.g. `&T` and `&mut T`).
+* Heap allocate recursive data structures:
+```rust
+enum OctreeNode<T> {
+    Leaf(T),
+    Children(Box<[OctreeNode<T>; 8]>),
+}
+```
+* Return small types by value, types that implement `Copy` or a cheaply Cloned are efficient to return by value (e.g. `struct Vector2 {x: f32, y: f32}`).
+
+### ❗ Be Mindful
+
+* Only use `#[inline]` when benchmark proves beneficial, Rust is already pretty good at inlining **without** hints.
+* Avoid massive stack allocations, box them. Example `let buffer: Box<[u8; 65536]> = Box::new(..)` would first allocate `[u8; 65536]` on the stack then box it, a non-const solution to this would be `let buffer: Box<[u8]> = vec![0; 65536].into_boxed_slice()`.
+* [`smallvec`](https://docs.rs/smallvec/latest/smallvec/) is for the *inverse* case: a `Vec`-like type that keeps **small** payloads inline on the stack and spills to the heap only once it outgrows the inline capacity. Reach for it when most values are small and a few are not - not as a way to heap-allocate large arrays.
+
+## 3.4 Iterators and Zero-Cost Abstractions
+
+Rust iterators are lazy, but eventually compiled away into very efficient tight loops that are only called when consumed. Chaining `.filter()`, `.map()`, `.rev()`, `.skip()`, `.take()`, `.collect()` usually doesn't cost extra and the compiler can reason well enough how to optimize them.
+* Prefer `iterators` over manual `for` loops when working with collections, the compiler can optimize them better than manually doing it.
+* Calling `.iter()` only creates a **reference** to the original collection, this allows you to hold multiple iterators of the same collection.
+
+#### ❗ Avoid creating intermediate collections unless it is really needed:
+
+* Consider that `process` accepts an `iterator`.
+* ❌ BAD - useless intermediate collection:
+```rust
+let doubled: Vec<_> = items.iter().map(|x| x * 2).collect();
+process(doubled);
+```
+* ✅ GOOD - pass the iterator (`fn process(arg: impl Iterator<Item = T>)`):
+```rust
+let doubled_iter = items.iter().map(|x| x * 2);
+process(doubled_iter);
+```

package/skills/rust/references/chapter_04.md

@@ -0,0 +1,183 @@
+# Chapter 4 - Errors Handling
+
+Rust enforces a strict error handling approach, but *how* you handle them defines where your code feels ergonomic, consistent and safe - as opposing cryptic and painful. This chapter dives into best practices for modeling and managing fallible operations across libraries and binaries.
+
+> Even if you decide to crash you application with `unwrap` or `expect`, Rust forces you to declare that intentionally.
+
+## 4.1 Prefer `Result`, avoid panic 🫨
+
+Rust has a powerful type that wraps fallible data, [`Result<T, E>`](https://doc.rust-lang.org/std/result/), this allows us to handle Error cases according to our needs and manage the state of the application based on that.
+
+* If your function can fail, prefer to return a `Result`:
+```rust
+fn divide(x: f64, y: f64) -> Result<f64, DivisionError> {
+    if y == 0.0 {
+        Err(DivisionError::DividedByZero)
+    } else {
+        Ok(x / y)
+    }
+}
+```
+
+* Use `panic!` only in unrecoverable conditions - typically tests, assertions, bugs or a need to crash the application for some explicit reason.
+* There are 3 relevant macros that can replace `panic!` in appropriate conditions:
+    * `todo!`, similar to panic, but alerts the compiler that you are aware that there is code missing.
+    * `unreachable!`, you have reasoned about the code block and are sure that condition `xyz` is not possible and if ever becomes possible you want to be alerted.
+    * `unimplemented!`, specially useful for alerting that a block is not yet implement with a reason.
+
+## 4.2 Avoid `unwrap`/`expect` in Production
+
+Although `expect` is preferred to `unwrap`, as it can have context, they should be avoided in production code as there are smarter alternatives to them. Considering that, they should be used in the following scenarios:
+- In tests, assertions or test helper functions.
+- When failure is impossible.
+- When the smarter options can't handle the specific case.
+
+### 🚨 Alternative ways of handling `unwrap`/`expect`:
+
+* If your `Result` (or `Option`) can have a predefined early return value in case of `Result::Err`, that doesn't need to know the `Err` value, use `let Ok(..) = else { return ... }` pattern, as it helps with flatten functions:
+```rust
+let Ok(json) = serde_json::from_str(&input) else {
+    return Err(MyError::InvalidJson);
+}; // `let ... else` is a statement: the `;` is required
+```
+* If your `Result` (or `Option`) needs error recovery in case of `Result::Err`, and the recovery path produces a value, use plain `if let ... else`:
+```rust
+// ❗ Note the shape: `if let PATTERN = EXPR { .. } else { .. }` - there is no
+// `else` block attached to the pattern itself, that form only exists in `let ... else`.
+if let Ok(json) = serde_json::from_str(&input) {
+    handle(json)
+} else {
+    recover_from(&input)
+}
+```
+* Functions that can have to handle `Option::None` values are recommended to return `Result<T, E>`, where `E` is a crate or module level error, like the examples above.
+* Lastly `unwrap_or`, `unwrap_or_else` or `unwrap_or_default`, these functions help you create alternative exits to unwrap that manage the uninitialized values.
+
+## 4.3 `thiserror` for Crate level errors
+
+Deriving Error manually is verbose and error prone, the rust ecosystem has a really good crate to help with this, `thiserror`. It allows you to create error types that easily implement `From` trait as well as easy error message (`Display`), improving developer experience while working seamlessly with `?` and integrating with `std::error::Error`:
+
+```rust
+#[derive(Debug, thiserror::Error)]
+pub enum MyError {
+    #[error("Network Timeout")]
+    Timeout,
+    #[error("Invalid data: {0}")]
+    InvalidData(String),
+    #[error(transparent)]
+    Serialization(#[from] serde_json::Error),
+    #[error("Invalid request information. Header: {headers}, Metadata: {metadata}")]
+    InvalidRequest {
+        headers: Headers,
+        metadata: Metadata
+    }
+}
+```
+
+### Error Hierarchies and Wrapping
+
+For layered systems the best practice is to use nested `enum/struct` errors with `#[from]`:
+
+```rust
+use crate::database::DbError;
+use crate::external_services::ExternalHttpError;
+
+#[derive(Debug, thiserror::Error)]
+pub enum ServiceError {
+    #[error("Database handler error: {0}")]
+    Db(#[from] DbError),
+    #[error("External services error: {0}")]
+    ExternalServices(#[from] ExternalHttpError)
+}
+```
+
+## 4.4 Reserve `anyhow` for Binaries
+
+`anyhow` is an amazing crate, and quite useful for projects that are beginning and need accelerated speed. However, there is a turning point where it just painfully propagates through your code, considering this, `anyhow` is recommended only for **binaries**, where ergonomic error handling is needed and there is no need for precise error types:
+
+```rust
+use anyhow::{Context, Result, anyhow};
+
+fn main() -> Result<()> {
+    let content = std::fs::read_to_string("config.json")
+        .context("Failed to read config file")?;
+    Config::from_str(&content)
+        .map_err(|err| anyhow!("Config parsing error: {err}"))
+}
+```
+
+### 🚨 `Anyhow` Gotchas
+
+* Keeping the `context` and `anyhow` strings up-to-date in all code base is harder than keeping `thiserror` messages as you don't have a single point of entry.
+* `anyhow::Result` erases context that a caller might need, so avoid using it in a library.
+* test helper functions can use `anyhow` with little to no issues.
+
+## 4.5 Use `?` to Bubble Errors
+
+Prefer using `?` over verbose alternatives like `match` chains:
+```rust
+fn handle_request(req: &Request) -> Result<ValidatedRequest, MyError> {
+    validate_headers(req)?;
+    validate_body_format(req)?;
+    validate_credentials(req)?;
+    let body = Body::try_from(req)?;
+
+    // Return the `Result` directly - `Ok(expr?)` would trip `clippy::needless_question_mark`
+    ValidatedRequest::try_from((req, body))
+}
+```
+
+> In case error recovery is needed, use `or_else`, `map_err`, `if let Ok(..) else`. To **inspect or log your error**, use `inspect_err`.
+
+## 4.6 Unit Test should exercise errors
+
+While many errors don't implement PartialEq and Eq, making it hard to do direct assertions between them, it is possible to check the error messages with `format!` or `to_string()`, making the errors meaningful and test validated:
+
+```rust
+#[test]
+fn error_does_not_implement_partial_eq() {
+    let err = divide(10., 0.0).unwrap_err();
+    assert_eq!(err.to_string(), "division by zero");
+}
+
+#[test]
+fn error_implements_partial_eq() {
+    let err = process(my_value).unwrap_err();
+
+    assert_eq!(
+        err,
+        MyError {
+            ..
+        }
+    )
+}
+```
+
+## 4.7 Important Topics
+
+### Custom Error Structs
+
+Sometimes you don't need an enum to handle your errors, as there is only one type of error that your module can have. This can be solved with `struct Errors`:
+
+```rust
+#[derive(Debug, thiserror::Error, PartialEq)]
+#[error("Request failed with code `{code}`: {message}")]
+struct HttpError {
+    code: u16,
+    message: String
+}
+```
+
+### Async Errors
+
+When using async runtimes, like Tokio, make sure that your errors implement `Send + Sync + 'static` where needed, specially in tasks or across `.await` boundaries:
+
+```rust
+#[tokio::main]
+async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
+    ...
+    Ok(())
+}
+```
+
+> Avoid `Box<dyn std::error::Error>` in libraries unless it is really needed