create-sia-app 0.1.16 → 0.1.18

package/package.json

@@ -1,6 +1,16 @@
 {
   "name": "create-sia-app",
-  "version": "0.1.16",
+  "version": "0.1.18",
+  "description": "Scaffold a private, encrypted storage app on the Sia network.",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/SiaFoundation/create-sia-app.git"
+  },
+  "bugs": {
+    "url": "https://github.com/SiaFoundation/create-sia-app/issues"
+  },
+  "homepage": "https://github.com/SiaFoundation/create-sia-app#readme",
   "type": "module",
   "bin": {
     "create-sia-app": "./dist/index.js"

package/template/CLAUDE.md

@@ -1,157 +1,249 @@
 # Sia Starter — AI Assistant Guide
 
-## Overview
+This is a starter for apps backed by the [Sia](https://sia.tech) storage network. Read this before writing code: it contains the mental model, the canonical patterns, and the footguns. If you're about to invent a pattern that isn't here, the odds are you shouldn't.
 
-A starter template for building decentralized storage apps on the Sia network. Uses [`@siafoundation/sia-storage`](https://www.npmjs.com/package/@siafoundation/sia-storage) — a TypeScript SDK that ships a pre-compiled WASM binary for encryption, uploads, downloads, and key management. WASM runs on the main thread (Rust async — no workers required).
+## Stack
 
-**Tech stack:** React 19, TypeScript, Vite, Tailwind CSS 4, Zustand, `@siafoundation/sia-storage`
+React 19, TypeScript, Vite, Tailwind CSS 4, Zustand, [`@siafoundation/sia-storage`](https://www.npmjs.com/package/@siafoundation/sia-storage).
 
-## Architecture
+## Types are the source of truth
 
-### Auth Flow State Machine
+The SDK's shape lives in its `.d.ts` files — more current and precise than any prose. Before calling an unfamiliar method, read:
 
-The app uses a step-based auth flow managed by Zustand (`src/stores/auth.ts`):
+- `node_modules/@siafoundation/sia-storage/dist/index.d.ts` — top-level exports.
+- `node_modules/@siafoundation/sia-storage/wasm/sia_storage_wasm.d.ts` — WASM-bound classes with full method signatures.
 
-```
-loading → connect → approve → recovery → connected
-```
+Don't hallucinate methods. If a method isn't in those files, it doesn't exist.
+
+## Core concepts
+
+**Indexer** — A service that coordinates storage: it tracks which hosts hold which encrypted shards, handles payments, and repairs slabs when hosts disappear. **It sees only ciphertext.** Trusted for availability and correctness of the repair/payment flow, *not* for data privacy. The indexer URL lives in `src/lib/constants.ts`.
+
+**Hosts** — The actual storage providers. The browser talks to them directly over WebTransport for uploads and downloads. Erasure coding means any sufficient subset of hosts is enough to reconstruct a file.
+
+**App** — Identified to the indexer by `APP_KEY` (32-byte hex) + `APP_META` in `src/lib/constants.ts`. Apps are namespaces: objects stored under one `APP_KEY` aren't visible to another.
+
+**User key** — An `AppKey` instance derived from the user's 12-word BIP-39 recovery phrase. It's the encryption key and the indexer-auth identity for that user *within* the app. Persisted as hex in `localStorage` so users don't re-enter the phrase every session.
+
+> **`APP_KEY` vs `AppKey`** — `APP_KEY` (constant, screaming snake) is the *app's* identity in `APP_META.appId`. `AppKey` (class, PascalCase) is the *user's* ed25519 key. They are not the same thing.
+
+**Object** — A file or blob you upload. Represented at rest by a `PinnedObject` handle. Has an ID, a size, one or more slabs, and encrypted metadata.
+
+**Slab / shard** — A file is split into slabs (default ~40 MB each), and each slab is erasure-coded into shards (10 data + 20 parity by default — see `DATA_SHARDS` / `PARITY_SHARDS` in `src/lib/constants.ts`). Shards are what actually ship to hosts.
+
+**Pin** — "This object should persist." An unpinned object is transient. Always `await sdk.pinObject(obj)` after a successful upload, or the indexer will garbage-collect it.
 
-- **loading**: WASM initializes, checks for stored app key
-- **connect**: User enters indexer URL, app constructs a `Builder` and calls `requestConnection()`
-- **approve**: User visits approval URL in another tab; app polls `builder.waitForApproval()`
-- **recovery**: User generates or enters a 12-word BIP-39 recovery phrase; `builder.register(phrase)` returns the `Sdk`
-- **connected**: `Sdk` is ready, main app UI renders
+**Metadata** — Encrypted app-defined bytes attached to an object (filename, MIME type, tags, etc.). Call `event.object.metadata()` to read, `pinnedObject.updateMetadata(bytes)` + `sdk.updateObjectMetadata(pinnedObject)` to write. Keep it under a few KB — it's a descriptor, not a payload.
 
-### Returning users
+## Auth flow
 
-Returning users skip `requestConnection`/`waitForApproval`/`register` entirely. `AuthFlow` constructs a `Builder` and calls `builder.connected(appKey)` with the persisted key — it returns an `Sdk` if the key is still valid, or `undefined` to fall back to the `connect` step.
+Step-based, managed by Zustand (`src/stores/auth.ts`):
 
-### SDK
+```
+loading → connect → approve → recovery → connected
+```
 
-`@siafoundation/sia-storage` handles:
-- Encrypted file uploads/downloads (erasure coding + encryption)
-- Key derivation from recovery phrases (BIP-39)
-- Object pinning and metadata management
-- Connection auth with indexers
-- Direct streaming to/from Sia hosts (no worker pool needed)
+- **loading** — `initSia()` loads WASM; `AuthFlow` checks for a stored user key.
+- **connect** — User enters indexer URL. App constructs `new Builder(url, APP_META)` and calls `requestConnection()`.
+- **approve** — User visits `builder.responseUrl()` in another tab; the app polls `builder.waitForApproval()`.
+- **recovery** — User generates or enters a BIP-39 phrase; `builder.register(phrase)` returns the `Sdk`.
+- **connected** — `Sdk` is ready; main UI renders.
 
-### Zustand Persistence
+**Returning users** skip connect/approve/recovery entirely: `AuthFlow` constructs a `Builder` and calls `builder.connected(appKey)` with the persisted key. Returns an `Sdk` if valid, `undefined` to fall back to `connect`.
 
-Auth state persists to localStorage via Zustand's `persist` middleware. The storage key is `{app-name}-auth`. Persisted fields: `storedKeyHex`, `indexerUrl`. The app key is stored as hex via the TC39 `Uint8Array.prototype.toHex` method.
+**Persistence**: Zustand `persist` middleware writes to `localStorage` under `sia-auth-<first-16-of-APP_KEY>` (keyed by app so two scaffolds on `localhost:5173` don't collide). Persisted: `storedKeyHex`, `indexerUrl`. The live `Sdk` is **not** persisted — it's rehydrated by calling `builder.connected(appKey)` on mount.
 
-## Key Files
+## Key files
 
-| File | Description |
-|------|-------------|
-| `src/lib/constants.ts` | App key, app name, indexer URL, app metadata (typed `AppMetadata`) |
-| `src/stores/auth.ts` | Auth state machine (Zustand + persist), holds the `Sdk` |
-| `src/stores/toast.ts` | Toast notification store (auto-dismiss) |
-| `src/components/Navbar.tsx` | App navbar with title, public key, sign out |
-| `src/components/Toast.tsx` | Toast overlay component |
-| `src/components/CopyButton.tsx` | Copy-to-clipboard button with toast |
-| `src/components/auth/AuthFlow.tsx` | Auth orchestrator — `initSia()`, returning-user reconnect via `Builder.connected` |
-| `src/components/auth/ConnectScreen.tsx` | Indexer URL form; constructs `new Builder(url, APP_META)` and calls `requestConnection()` |
-| `src/components/auth/ApproveScreen.tsx` | Approval polling (auto-polls on mount) |
-| `src/components/auth/RecoveryScreen.tsx` | Recovery phrase generation/import; `builder.register(phrase)` → `Sdk` |
-| `src/components/upload/UploadZone.tsx` | File upload/download dropzone + file list |
-| `src/components/DevNote.tsx` | Developer callout component |
-| `src/types/uint8array-hex.d.ts` | Ambient types for TC39 `Uint8Array.toHex`/`fromHex` (drop once TS lib ships them) |
+| File | Role |
+|---|---|
+| `src/lib/constants.ts` | `APP_KEY`, `APP_NAME`, `APP_META` (`AppMetadata`), indexer default, erasure-coding constants |
+| `src/stores/auth.ts` | Zustand store: holds the `Sdk`, persists `storedKeyHex` + `indexerUrl` |
+| `src/stores/toast.ts` | Toast notifications (auto-dismiss) |
+| `src/components/auth/AuthFlow.tsx` | Orchestrator: `initSia()`, returning-user reconnect |
+| `src/components/auth/ConnectScreen.tsx` | `new Builder(url, APP_META).requestConnection()` |
+| `src/components/auth/ApproveScreen.tsx` | Polls `builder.waitForApproval()` |
+| `src/components/auth/RecoveryScreen.tsx` | Generate / validate phrase → `builder.register()` → `Sdk` |
+| `src/components/upload/UploadZone.tsx` | **Reference implementation.** Full cycle: dropzone → upload → pin → metadata → list → download. Read this first when building new features. |
+| `src/components/Navbar.tsx` | Public key + sign out |
+| `src/components/DevNote.tsx` | Amber callout — remove or replace for production |
+| `src/types/uint8array-hex.d.ts` | Ambient types for TC39 `Uint8Array.{toHex,fromHex}` (drop once TS lib ships them) |
 
-## SDK Usage Patterns
+## SDK usage patterns
 
-### Upload a file
+### Upload → pin → metadata
 
 ```ts
 import { PinnedObject } from '@siafoundation/sia-storage'
+import { DATA_SHARDS, PARITY_SHARDS } from '../../lib/constants'
 
 const object = new PinnedObject()
-const pinnedObject = await sdk.upload(object, file.stream(), {
+const pinned = await sdk.upload(object, file.stream(), {
   maxInflight: 10,
-  onShardUploaded: (progress) => {
-    // progress: { hostKey, shardSize, shardIndex, slabIndex, elapsedMs }
-    console.log(`shard ${progress.shardIndex} uploaded (${progress.shardSize}b)`)
+  dataShards: DATA_SHARDS,
+  parityShards: PARITY_SHARDS,
+  onShardUploaded: (p) => {
+    // p: { hostKey, shardSize, shardIndex, slabIndex, elapsedMs }
+    // shardSize is post-erasure-coding bytes, not source bytes.
   },
 })
 
-pinnedObject.updateMetadata(new TextEncoder().encode(JSON.stringify({
-  name: 'file.txt',
-  type: 'text/plain',
-  size: file.size,
-  hash: '...',
-})))
-await sdk.pinObject(pinnedObject)
-await sdk.updateObjectMetadata(pinnedObject)
+pinned.updateMetadata(
+  new TextEncoder().encode(JSON.stringify({ name: file.name, type: file.type, size: file.size })),
+)
+await sdk.pinObject(pinned)
+await sdk.updateObjectMetadata(pinned)
 ```
 
-### Download a file
+All three calls matter:
+1. `upload` writes the encrypted shards to hosts.
+2. `pinObject` tells the indexer to keep it (without this, it's eventually GC'd).
+3. `updateObjectMetadata` persists the descriptor so other sessions can find it.
 
-`sdk.download` returns a `ReadableStream` of `Uint8Array` chunks. Buffer it, or stream it directly into a `Response`/`Blob`:
+### Byte progress (source units, not on-wire units)
+
+`onShardUploaded.shardSize` is on-wire bytes (encoded). To show a progress bar in source-file units, use `encodedSize` as the denominator:
 
 ```ts
-const stream = sdk.download(pinnedObject, {
-  maxInflight: 10,
-  onShardDownloaded: (progress) => {
-    console.log(`shard ${progress.shardIndex} downloaded`)
-  },
-})
-const blob = await new Response(stream).blob()
+import { encodedSize } from '@siafoundation/sia-storage'
+const encodedTotal = encodedSize(file.size, DATA_SHARDS, PARITY_SHARDS)
+const sourceProgress = (bytesUploaded / encodedTotal) * file.size
 ```
 
-### List files
+### Download
+
+Returns a `ReadableStream<Uint8Array>`. Buffer or pipe:
 
 ```ts
-const events = await sdk.objectEvents(undefined, 100)
-for (const event of events) {
-  if (!event.deleted && event.object) {
-    const meta = JSON.parse(new TextDecoder().decode(event.object.metadata()))
-    console.log(meta.name, event.object.size())
-  }
-}
+const stream = sdk.download(pinnedObject, { maxInflight: 10 })
+const blob = await new Response(stream).blob()
 ```
 
-### Delete a file
+### Delete
 
 ```ts
 await sdk.deleteObject(objectId)
 ```
 
-### Share a file
+### Share / consume a share URL
 
 ```ts
-const validUntil = new Date(Date.now() + 7 * 24 * 60 * 60 * 1000) // 7 days
-const shareUrl = sdk.shareObject(pinnedObject, validUntil)
+const validUntil = new Date(Date.now() + 7 * 24 * 60 * 60 * 1000)
+const url = sdk.shareObject(pinnedObject, validUntil)
+// On the recipient side (can be a different app / no auth needed):
+const obj = await sdk.sharedObject(url)
+const stream = sdk.download(obj)
 ```
 
-### Download a shared file
+Share URLs embed the decryption key in the fragment (`#...`) — never sent to the indexer.
+
+### Pack many small files
+
+`sdk.uploadPacked()` batches small files into shared slabs to avoid wasting storage:
 
 ```ts
-const object = await sdk.sharedObject(shareUrl)
-const stream = sdk.download(object)
+const packed = sdk.uploadPacked({ maxInflight: 10 })
+await packed.add(fileA.stream())
+await packed.add(fileB.stream())
+for (const obj of await packed.finalize()) await sdk.pinObject(obj)
 ```
 
-## Customization
+## Syncing with the indexer
 
-### Change app key
+`sdk.objectEvents(cursor, limit)` is the one sync primitive. Each `ObjectEvent` has `id`, `updatedAt: Date`, `deleted: boolean`, `object: PinnedObject | null`.
 
-Edit `src/lib/constants.ts`. The app key is a 32-byte hex string that identifies your app to the indexer. Generate one with:
+Cursor is `{ id: string, after: Date }`. Passing it returns events strictly after that point.
+
+### `updatedAt` bumps
+
+- **User actions from any device using this app key**: `updateObjectMetadata`, `deleteObject`, re-pin.
+- **Indexer repairs**: when a host goes offline the indexer migrates shards to healthy hosts and bumps `updatedAt`. Your next poll picks up the repaired state automatically.
+
+Both surface through the same stream, which is why polling `objectEvents` is the pattern every Sia app should implement.
+
+### Cross-device sync
 
 ```ts
-crypto.getRandomValues(new Uint8Array(32)).toHex()
+// Fresh session: no cursor, pull latest N.
+const events = await sdk.objectEvents(undefined, 500)
+events.forEach(apply)
+persistCursor(latest(events))
+
+// Periodic tick: only what changed.
+const cursor = loadCursor() // { id, after: Date } or undefined
+const events = await sdk.objectEvents(cursor, 200)
+events.forEach(apply)
+if (events.length) persistCursor(latest(events))
 ```
 
-### Replace the upload UI
+Operational tips:
+- Persist the cursor in `localStorage` keyed by app key.
+- Poll only while the tab is visible (`document.visibilityState === 'visible'`).
+- Advance the cursor only after local merge succeeds.
+- `event.deleted === true` → evict from local store.
+
+If a specific SDK build rejects the cursor shape (edge cases with `Date` serialization happen), fall back to `undefined` + client-side filter on `event.updatedAt.getTime() > watermarkMs`.
+
+## Gotchas
+
+Things that look right but aren't:
+
+- **Don't persist `Sdk` to storage.** It's a live WASM handle; rehydrate it via `Builder.connected(appKey)` on mount.
+- **Don't forget `pinObject`.** A successful `upload` that isn't pinned is a transient object — the indexer will eventually drop it.
+- **Don't conflate `APP_KEY` and `AppKey`.** `APP_KEY` is the app identity constant; `AppKey` is the user's key class.
+- **Don't stuff large payloads into metadata.** It's a descriptor. Put file bytes in the object, not in metadata.
+- **Don't re-bundle or wrap the WASM.** Vite dev needs `optimizeDeps: { exclude: ['@siafoundation/sia-storage'] }` (already set in `vite.config.ts`) because the SDK's `import.meta.url`-relative WASM path breaks under pre-bundling. If you add another bundler (Webpack, Rollup), check the SDK README for the equivalent.
+- **Don't call `initSia()` more than once per mount.** It's already called in `AuthFlow`; additional call sites create race conditions.
+- **Don't call `builder.waitForApproval()` twice.** React strict mode will remount — `ApproveScreen` guards this with a `pollStarted` ref. Follow that pattern.
+- **`onShardUploaded.shardSize` is encoded bytes, not source bytes.** If you sum it, you're measuring on-wire traffic. Use `encodedSize()` for the matching denominator, or scale to source via `(bytes / encodedTotal) * file.size`.
+- **Numeric types differ on Node vs browser.** Browser uses `number` (~9 PB safe); Node uses `bigint`. Template is browser-only, so `number` is correct here.
+- **Sign-out should clear localStorage.** `useAuthStore.getState().reset()` + `window.location.reload()` is the pattern in `Navbar.tsx`.
+
+## Extending the starter
+
+### Swap out `UploadZone`
+
+`src/App.tsx` renders `<UploadZone />` after auth. Replace it with your own post-auth component. Read `UploadZone.tsx` first — it shows the full upload → pin → metadata → list cycle that most apps will want to reuse in some form.
+
+Access the SDK:
 
-The post-auth UI is rendered in `src/App.tsx`. Replace `<UploadZone />` with your own component. The `Sdk` is available via `useAuthStore((s) => s.sdk)`. All SDK methods live directly on `Sdk` — `sdk.upload()`, `sdk.download()`, `sdk.objectEvents()`, etc.
+```tsx
+const sdk = useAuthStore((s) => s.sdk)
+if (!sdk) return null
+```
 
 ### Add routes
 
-Install `react-router-dom` and wrap your app. The auth flow should gate all routes.
+Install `react-router-dom`. Gate routes on `step === 'connected'`; render `<AuthFlow />` otherwise.
+
+### Add fields to file metadata
+
+Extend the `FileMetadata` type in `UploadZone.tsx`, write the extra fields in the upload handler, read them back in `loadFiles`. Schema is app-owned — do whatever makes sense. Just keep it small.
+
+### Search / filter
+
+Metadata is encrypted at rest but decrypted client-side, so once you've hydrated from `objectEvents` you can filter/sort/search it in memory like any local list. The indexer can't do this for you (it sees ciphertext) — search is always client-side.
+
+### Multi-device updates
 
-## Common Commands
+Implement the polling pattern from **Syncing with the indexer**. That's how uploads on one device appear on another.
+
+### Change erasure-coding parameters
+
+Edit `DATA_SHARDS` / `PARITY_SHARDS` in `src/lib/constants.ts`. More parity = survives more host failures at the cost of more on-wire bytes. Keep `UploadZone`'s `encodedSize()` call in sync (it already reads from the same constants).
+
+### Change the app key
+
+`APP_KEY` in `src/lib/constants.ts`. Generate with `crypto.getRandomValues(new Uint8Array(32)).toHex()`. **Changing it makes all previously uploaded data invisible to the app** — app key is the namespace.
+
+## Commands
 
 ```bash
-bun install     # Install dependencies
-bun dev         # Start dev server
-bun run build   # Production build
-bun run check   # Lint with Biome
+bun install     # Install deps
+bun dev         # Vite dev server (WASM loads lazily, ~100ms)
+bun run build   # tsc + Vite production build
+bun run check   # Biome lint + format check (use --write to fix)
+bun x playwright test e2e/smoke.spec.ts   # App-loads-without-errors smoke
 ```
+
+After any substantive change, run `bun run check` (auto-fix with `bun x biome check . --write`) and `bun run build` before committing.

package/template/package.json

@@ -12,7 +12,7 @@
   "dependencies": {
     "react": "^19.2.0",
     "react-dom": "^19.2.0",
-    "@siafoundation/sia-storage": "^0.0.8",
+    "@siafoundation/sia-storage": "^0.0.9",
     "zustand": "^5.0.11"
   },
   "devDependencies": {

package/template/src/components/auth/ApproveScreen.tsx

@@ -1,5 +1,5 @@
-import { useEffect, useRef, useState } from 'react'
 import type { Builder } from '@siafoundation/sia-storage'
+import { useEffect, useRef, useState } from 'react'
 import { useAuthStore } from '../../stores/auth'
 import { CopyButton } from '../CopyButton'
 import { DevNote } from '../DevNote'

package/template/src/components/auth/AuthFlow.tsx

@@ -1,5 +1,5 @@
-import { useEffect, useRef } from 'react'
 import { AppKey, Builder, initSia } from '@siafoundation/sia-storage'
+import { useEffect, useRef } from 'react'
 import { APP_META } from '../../lib/constants'
 import { useAuthStore } from '../../stores/auth'
 import { ApproveScreen } from './ApproveScreen'

package/template/src/components/auth/ConnectScreen.tsx

@@ -1,5 +1,5 @@
-import { useState } from 'react'
 import { Builder } from '@siafoundation/sia-storage'
+import { useState } from 'react'
 import { APP_META, DEFAULT_INDEXER_URL } from '../../lib/constants'
 import { useAuthStore } from '../../stores/auth'
 import { DevNote } from '../DevNote'

package/template/src/components/auth/RecoveryScreen.tsx

@@ -1,9 +1,9 @@
-import { useState } from 'react'
 import {
   type Builder,
   generateRecoveryPhrase,
   validateRecoveryPhrase,
 } from '@siafoundation/sia-storage'
+import { useState } from 'react'
 import { useAuthStore } from '../../stores/auth'
 import { CopyButton } from '../CopyButton'
 import { DevNote } from '../DevNote'

package/template/src/components/upload/UploadZone.tsx

@@ -1,5 +1,9 @@
+import {
+  encodedSize,
+  PinnedObject,
+  type ShardProgress,
+} from '@siafoundation/sia-storage'
 import { useCallback, useEffect, useRef, useState } from 'react'
-import { encodedSize, PinnedObject, type ShardProgress } from '@siafoundation/sia-storage'
 import { APP_KEY, DATA_SHARDS, PARITY_SHARDS } from '../../lib/constants'
 import { useAuthStore } from '../../stores/auth'
 import { DevNote } from '../DevNote'