@game_ryo/lsji 0.1.0 → 0.1.1

package/docs/docs/api/qlearning.md

@@ -1,138 +0,0 @@
----
-title: QLearning API
-description: Tabular Q-Learning engine with TD updates
----
-
-# QLearning API
-
-The `QLearning` class implements tabular Q-Learning with epsilon-greedy exploration.
-
-## Import
-
-```typescript
-import { QLearning } from 'lsji';
-```
-
-## Constructor
-
-```typescript
-const qlearning = new QLearning({
-  alpha: 0.1,     // Learning rate [0, 1] (default: 0.1)
-  gamma: 0.9,     // Discount factor [0, 1] (default: 0.9)
-  epsilon: 0.1,   // Exploration rate [0, 1] (default: 0.1)
-  storage: Storage // Required: Storage backend
-});
-```
-
-Values are automatically clamped to [0, 1] range.
-
-## Methods
-
-### `initialize()`
-Load Q-table from storage. Called automatically by other methods.
-
-```typescript
-await qlearning.initialize();
-```
-
-### `getQValue(state, action)`
-Get Q-value for a state-action pair.
-
-```typescript
-const value = qlearning.getQValue('state1', 0);
-// Returns: number (0 if unseen)
-```
-
-### `setQValue(state, action, value)`
-Set Q-value and persist to storage.
-
-```typescript
-await qlearning.setQValue('state1', 0, 0.5);
-```
-
-### `act(state, actionSize)`
-Select action using epsilon-greedy policy.
-
-```typescript
-const action = await qlearning.act('state1', 3);
-// Returns: number (0 to actionSize-1)
-```
-
-**Behavior:**
-- With probability `epsilon`: random action
-- With probability `1-epsilon`: best known action (ties broken randomly)
-
-### `learn(state, action, reward, nextState, nextActionSize)`
-Full TD update: Q(s,a) ← Q(s,a) + α[r + γ·maxₐ' Q(s',a') - Q(s,a)]
-
-```typescript
-const newQ = await qlearning.learn('state1', 0, 1, 'state2', 3);
-// Returns: updated Q-value
-```
-
-### `learnSimple(state, action, reward)`
-Simplified update for terminal states: Q(s,a) ← Q(s,a) + α[r - Q(s,a)]
-
-```typescript
-const newQ = await qlearning.learnSimple('state1', 0, 1);
-// Returns: updated Q-value
-```
-
-### `reset()`
-Clear in-memory Q-table cache.
-
-```typescript
-await qlearning.reset();
-// Note: Does not clear storage
-```
-
-### `getStateValues(state, actionSize)`
-Get all Q-values for a state.
-
-```typescript
-const values = qlearning.getStateValues('state1', 3);
-// Returns: { 0: 0.5, 1: 0.2, 2: -0.1 }
-```
-
-### `getFullQTable()`
-Get entire Q-table for inspection.
-
-```typescript
-const table = await qlearning.getFullQTable();
-// Returns:
-[
-  { state: 'state1', action: 0, q_value: 0.5 },
-  { state: 'state1', action: 1, q_value: 0.2 },
-  ...
-]
-```
-
-## Hyperparameters
-
-| Parameter | Default | Range | Description |
-|-----------|---------|-------|-------------|
-| `alpha` | 0.1 | [0, 1] | Learning rate — how much new info overrides old |
-| `gamma` | 0.9 | [0, 1] | Discount factor — future reward importance |
-| `epsilon` | 0.1 | [0, 1] | Exploration rate — random action probability |
-
-## Example
-
-```typescript
-import { QLearning, createStorage } from 'lsji';
-
-const storage = await createStorage('memory');
-const ql = new QLearning({ alpha: 0.1, gamma: 0.9, epsilon: 0.1, storage });
-
-await ql.initialize();
-
-// Train on simple state-action pairs
-await ql.learnSimple('state1', 0, 1);  // Win
-await ql.learnSimple('state1', 0, 1);  // Win again
-await ql.learnSimple('state1', 1, -1); // Lose
-
-// Check learned values
-console.log(ql.getQValue('state1', 0)); // ~0.19
-console.log(ql.getQValue('state1', 1)); // ~-0.1
-
-await storage.close();
-```

package/docs/docs/api/storage.md

@@ -1,168 +0,0 @@
----
-title: Storage API
-description: Pluggable storage backends
----
-
-# Storage API
-
-LSJI provides a pluggable storage abstraction with three built-in implementations.
-
-## Import
-
-```typescript
-import { createStorage, Storage, SqliteStorage, BetterSqliteStorage, MemoryStorage } from 'lsji';
-```
-
-## Factory Function
-
-```typescript
-const storage = await createStorage(type, options);
-```
-
-**Types:**
-- `'sqlite'` — Node.js built-in `node:sqlite` (recommended)
-- `'better-sqlite'` — `better-sqlite3` package
-- `'memory'` — In-memory (testing only)
-
-**Options:**
-```typescript
-// SQLite
-{ path: './my-agent.db' }
-
-// Better-SQLite
-{ path: './my-agent.db' }
-
-// Memory
-{}  // No options needed
-```
-
-## Storage Interface
-
-All backends implement this interface:
-
-```typescript
-interface Storage {
-  initialize(): Promise<void>;
-  close(): Promise<void>;
-  getSetting(key: string): Promise<{key: string, value: string} | null>;
-  setSetting(key: string, value: string | number): Promise<void>;
-  getQTable(): Promise<Array<{state: string, action: number, q_value: number}>>;
-  updateQ(state: string, action: number, qValue: number): Promise<void>;
-  addBattle(record: BattleRecord): Promise<void>;
-  getTodayBattleCount(): Promise<number>;
-  getPerformanceStats(): Promise<Array<{mode: string, total: number, win_rate: number}>>;
-}
-
-interface BattleRecord {
-  mode: 'train' | 'test';
-  handA: number;
-  handB: number;
-  reward: number;
-  createdAt: string;  // ISO timestamp
-}
-```
-
-## Backends
-
-### SqliteStorage (Recommended)
-
-Uses Node.js 22+ built-in `node:sqlite` — **zero dependencies**.
-
-```typescript
-import { SqliteStorage } from 'lsji';
-
-const storage = new SqliteStorage('./agent.db');
-await storage.initialize();
-// ... use storage
-await storage.close();
-```
-
-**Features:**
-- Synchronous API (fast)
-- Automatic schema creation
-- Indexes on battle_history for performance
-- Persistent across process restarts
-
-### BetterSqliteStorage
-
-Uses `better-sqlite3` for high-performance synchronous access.
-
-```typescript
-import { BetterSqliteStorage } from 'lsji';
-
-const storage = new BetterSqliteStorage('./agent.db');
-await storage.initialize();
-```
-
-**Requires:** `npm install better-sqlite3`
-
-**Use when:** You need faster writes or advanced SQLite features.
-
-### MemoryStorage
-
-Pure in-memory implementation.
-
-```typescript
-import { MemoryStorage } from 'lsji';
-
-const storage = new MemoryStorage();
-await storage.initialize();
-```
-
-**Features:**
-- No persistence (data lost on exit)
-- Fastest for testing/CI
-- Implements `clear()` for test isolation
-
-## Database Schema
-
-All SQL backends create these tables:
-
-```sql
--- Settings (key-value)
-CREATE TABLE settings (
-  key TEXT PRIMARY KEY,
-  value TEXT NOT NULL
-);
-
--- Battle history
-CREATE TABLE battle_history (
-  id INTEGER PRIMARY KEY AUTOINCREMENT,
-  mode TEXT NOT NULL CHECK (mode IN ('train', 'test')),
-  hand_a INTEGER NOT NULL,
-  hand_b INTEGER NOT NULL,
-  reward INTEGER NOT NULL,
-  created_at TEXT NOT NULL
-);
-
--- Q-table
-CREATE TABLE q_table (
-  state TEXT NOT NULL,
-  action INTEGER NOT NULL,
-  q_value REAL NOT NULL DEFAULT 0,
-  PRIMARY KEY (state, action)
-);
-
--- Indexes
-CREATE INDEX idx_battle_history_date ON battle_history(created_at);
-CREATE INDEX idx_battle_history_mode ON battle_history(mode);
-```
-
-## Example
-
-```typescript
-import { createStorage, QLearning, Agent, RockPaperScissorsEnv } from 'lsji';
-
-// Production: persistent SQLite
-const storage = await createStorage('sqlite', { path: './production.db' });
-
-// Testing: in-memory
-const testStorage = await createStorage('memory');
-
-const qlearning = new QLearning({ alpha: 0.1, gamma: 0.9, epsilon: 0.1, storage });
-const env = new RockPaperScissorsEnv({ opponent: 'random' });
-const agent = new Agent({ qlearning, storage, env });
-
-await agent.train({ episodes: 1000 });
-await storage.close();
-```

package/docs/docs/architecture.md

@@ -1,155 +0,0 @@
----
-title: Architecture
-description: Design decisions and internals
----
-
-# Architecture
-
-## Design Principles
-
-1. **Single Package** — All core functionality in one npm package
-2. **Zero Dependencies** — Core runs on Node.js 22+ built-ins only
-3. **Storage Abstraction** — Easy to swap backends
-4. **ESM Only** — Modern JavaScript modules
-5. **Apache 2.0** — License compatible with Apache Incubator
-
-## Project Structure
-
-```
-src/
-├── core/
-│   ├── env.ts          # Environment interface
-│   ├── qlearning.ts    # Q-Learning engine
-│   └── agent.ts        # Agent orchestration
-├── storage/
-│   ├── index.ts        # Storage interface + factory
-│   ├── sqlite.ts       # node:sqlite implementation
-│   ├── better-sqlite.ts # better-sqlite3 implementation
-│   └── memory.ts       # In-memory implementation
-├── envs/
-│   └── rps.ts          # Rock-Paper-Scissors environment
-├── cli.ts              # Command-line interface
-└── index.ts            # Public API exports
-```
-
-## Core Components
-
-### Q-Learning Engine
-
-**Algorithm:** Tabular Q-Learning with Temporal Difference updates
-
-**Update Rules:**
-- Full TD: `Q(s,a) ← Q(s,a) + α[r + γ·maxₐ' Q(s',a') - Q(s,a)]`
-- Simple: `Q(s,a) ← Q(s,a) + α[r - Q(s,a)]` (terminal states)
-
-**Exploration:** Epsilon-greedy with configurable ε
-
-**State Representation:** String keys for Q-table indexing
-
-### Storage Layer
-
-**Interface:** Abstract `Storage` class with 9 required methods
-
-**Implementations:**
-1. `SqliteStorage` — Node.js built-in `node:sqlite` (sync API)
-2. `BetterSqliteStorage` — `better-sqlite3` (faster, more features)
-3. `MemoryStorage` — Pure JS Maps/Arrays (testing)
-
-**Schema:**
-- `settings` — Key-value configuration
-- `battle_history` — Training/play records with timestamps
-- `q_table` — State-action values with composite primary key
-
-**Indexes:** Date and mode indexes on battle_history for fast queries
-
-### Agent Orchestration
-
-**Responsibilities:**
-- System lifecycle (start/stop)
-- Training loop with batching
-- Play/evaluation loop
-- Statistics aggregation
-
-**Training Loop:**
-```
-for episode in episodes:
-  state = env.getState()
-  action = qlearning.act(state, actionSize)  // ε-greedy
-  result = env.step(action)
-  qlearning.learnSimple(state, action, result.reward)
-  storage.addBattle(record)
-  batch.flush()
-```
-
-### Environment Interface
-
-**Contract:**
-```typescript
-abstract class Env {
-  abstract getState(): string;
-  abstract step(action: number): Promise<StepResult>;
-  abstract actionSize(): number;
-  abstract reset(): Promise<string>;
-  render(): string;  // Optional
-}
-```
-
-**Built-in:** `RockPaperScissorsEnv` with 4 opponent strategies
-
-## Data Flow
-
-```
-┌─────────┐     ┌──────────────┐     ┌─────────┐
-│  Env    │────▶│   Agent      │────▶│ Storage │
-│ (State) │     │  Orchestrates│     │(Persist)│
-└─────────┘     └──────┬───────┘     └─────────┘
-                       │
-                       ▼
-                ┌──────────────┐
-                │  QLearning   │
-                │  (Updates)   │
-                └──────────────┘
-```
-
-## Concurrency Model
-
-- **Single-threaded** Node.js event loop
-- **Synchronous SQLite** — No async/await for DB operations
-- **Batch writes** — Reduce I/O overhead
-- **In-memory Q-table cache** — Fast reads, periodic persistence
-
-## Migration from Cloudflare Workers
-
-**Original:** Worker.js with D1 database, cron triggers, HTTP endpoints
-
-**Changes:**
-| Before | After |
-|--------|-------|
-| `env.DB` (D1) | `Storage` abstraction |
-| Cron triggers | Manual/CLI training |
-| HTTP endpoints | Library API + CLI |
-| Global state | Instance-based |
-| `Request/Response` | Function parameters |
-
-**Preserved:**
-- Q-Learning algorithm (α=0.1, γ=0.9, ε=0.1)
-- Reward calculation: `(ai - user + 3) % 3`
-- Training patterns (0-3)
-- Battle history schema
-
-## Performance Characteristics
-
-| Operation | Complexity | Notes |
-|-----------|------------|-------|
-| `act()` | O(actions) | Scans all actions for max Q |
-| `learnSimple()` | O(1) | Single Q-value update |
-| `learn()` | O(actions) | Finds max next Q |
-| `getFullQTable()` | O(states×actions) | Full table scan |
-| Batch insert | O(batch) | Single transaction |
-
-## Future Extensibility
-
-- **New algorithms:** Extend `QLearning` or add `SARSA`, `DQN` classes
-- **New environments:** Implement `Env` interface
-- **New storage:** Implement `Storage` interface
-- **Function approximation:** Replace tabular Q-table with neural networks

package/docs/docs/cli.md

@@ -1,210 +0,0 @@
----
-title: CLI Reference
-description: Command-line interface for training and playing
----
-
-# CLI Reference
-
-LSJI includes a command-line interface for training and playing without writing code.
-
-## Installation
-
-```bash
-# Global install
-npm install -g lsji
-
-# Or use npx
-npx lsji --help
-```
-
-## Commands
-
-### `lsji train`
-
-Train the agent.
-
-```bash
-lsji train [options]
-```
-
-**Options:**
-
-| Option | Description | Default |
-|--------|-------------|---------|
-| `--episodes <n>` | Number of training episodes | 200 |
-| `--pattern <0-3>` | Training pattern | 0 |
-| `--batch-size <n>` | Database batch size | 200 |
-| `--opponent <type>` | Opponent strategy | random |
-| `--storage <type>` | Storage backend | sqlite |
-| `--db-path <path>` | Database file path | ./lsji.db |
-| `--alpha <n>` | Learning rate | 0.1 |
-| `--gamma <n>` | Discount factor | 0.9 |
-| `--epsilon <n>` | Exploration rate | 0.1 |
-| `--json` | Output as JSON | false |
-
-**Training Patterns:**
-- `0` — Random actions
-- `1` — Always Rock
-- `2` — Counter previous action
-- `3` — Sequential (0,1,2,0,1,2...)
-
-**Opponent Strategies:**
-- `random` — Random actions
-- `always_rock` — Always plays Rock
-- `counter` — Counters agent's previous action
-- `sequential` — Cycles through actions
-
-**Examples:**
-```bash
-# Default training
-lsji train --episodes 500
-
-# Train against always-rock opponent
-lsji train --episodes 100 --pattern 1 --opponent always_rock
-
-# Train with custom hyperparameters
-lsji train --episodes 1000 --alpha 0.05 --gamma 0.95 --epsilon 0.2
-
-# Use memory storage (ephemeral)
-lsji train --episodes 100 --storage memory
-```
-
-### `lsji play`
-
-Play a single game against the agent.
-
-```bash
-lsji play --hand <0|1|2> [options]
-```
-
-**Options:**
-
-| Option | Description |
-|--------|-------------|
-| `--hand <0\|1\|2>` | Your hand: 0=Rock, 1=Scissors, 2=Paper |
-| `--opponent <type>` | Opponent strategy |
-| `--storage <type>` | Storage backend |
-| `--db-path <path>` | Database file path |
-| `--json` | Output as JSON |
-
-**Examples:**
-```bash
-lsji play --hand 0        # Play Rock
-lsji play --hand 1        # Play Scissors
-lsji play --hand 2 --json # Play Paper, JSON output
-```
-
-### `lsji status`
-
-Show system status and statistics.
-
-```bash
-lsji status [options]
-```
-
-**Options:**
-
-| Option | Description |
-|--------|-------------|
-| `--storage <type>` | Storage backend |
-| `--db-path <path>` | Database file path |
-| `--json` | Output as JSON |
-
-**Example:**
-```bash
-lsji status --json
-```
-
-Output:
-```json
-{
-  "status": "running",
-  "todayTotal": 42,
-  "limit": 90000,
-  "performance": [
-    { "mode": "train", "total": 1000, "win_rate": 65.5 },
-    { "mode": "test", "total": 50, "win_rate": 72.0 }
-  ],
-  "aiBrain": [
-    { "state": "0", "action": 0, "q_value": 0.45 },
-    { "state": "0", "action": 1, "q_value": 0.12 }
-  ]
-}
-```
-
-### `lsji start`
-
-Enable training and play.
-
-```bash
-lsji start [options]
-```
-
-### `lsji stop`
-
-Disable training and play (system paused).
-
-```bash
-lsji stop [options]
-```
-
-### `lsji help`
-
-Show help message.
-
-```bash
-lsji help
-lsji --help
-lsji -h
-```
-
-## Environment Variables
-
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `LSJI_STORAGE` | Default storage backend | sqlite |
-| `LSJI_DB_PATH` | Default database path | ./lsji.db |
-
-## Examples
-
-### Full Training Session
-
-```bash
-# Start fresh
-rm -f lsji.db
-
-# Train against random opponent
-lsji train --episodes 500 --opponent random
-
-# Train against counter opponent
-lsji train --episodes 500 --opponent counter
-
-# Check progress
-lsji status --json
-
-# Play a few games
-lsji play --hand 0
-lsji play --hand 1
-lsji play --hand 2
-```
-
-### Using Memory Storage (CI/Testing)
-
-```bash
-lsji train --episodes 100 --storage memory
-lsji play --hand 0 --storage memory
-lsji status --storage memory
-```
-
-### Custom Hyperparameters
-
-```bash
-lsji train \
-  --episodes 2000 \
-  --alpha 0.05 \
-  --gamma 0.95 \
-  --epsilon 0.2 \
-  --opponent random \
-  --storage sqlite \
-  --db-path ./custom.db
-```