@lloyal-labs/sdk 3.0.0 → 3.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +2 -2
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -1,8 +1,8 @@
1
1
  # @lloyal-labs/sdk
2
2
 
3
- Backend-agnostic inference primitives for the [lloyal HDK](https://github.com/lloyal-ai/hdk).
3
+ **Fork a language model's train of thought.**
4
4
 
5
- Composable inference primitives for forkable decode state, shared-prefix KV branching, and continuous tree batching. Branches share a KV prefix while keeping independent machinery — sampler chain, grammar, logits snapshot, perplexity tracker — for controlled divergence at decode time. `BranchStore` packs tokens from N branches (each at a different position, different seq_id, each needing independent logits captured) into a single `llama_batch` and dispatches once.
5
+ `Branch` is decode state you can fork: each branch inherits the full KV prefix (free, via the attention cache) and diverges with its own sampler, grammar, and logits — explore N continuations of one context without re-decoding it. `BranchStore` then packs every branch's next token into a single `llama_batch`, so N branches advance in one GPU dispatch. These are the primitives the [lloyal HDK](https://github.com/lloyal-ai/hdk)'s agent runtime is built on — backend-agnostic, usable standalone.
6
6
 
7
7
  ```bash
8
8
  npm i @lloyal-labs/sdk
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@lloyal-labs/sdk",
3
- "version": "3.0.0",
3
+ "version": "3.0.1",
4
4
  "description": "Backend-agnostic TypeScript SDK for the lloyal inference platform",
5
5
  "main": "dist/index.js",
6
6
  "types": "dist/index.d.ts",