@game_ryo/lsji 0.1.0 → 0.3.0

package/docs/docs/api/env.md

@@ -1,133 +0,0 @@
----
-title: Env Interface
-description: Base environment interface for RL problems
----
-
-# Env Interface
-
-All reinforcement learning environments must extend the abstract `Env` class.
-
-## Import
-
-```typescript
-import { Env } from 'lsji';
-```
-
-## Abstract Methods
-
-### `getState()`
-Return current state as a string key.
-
-```typescript
-getState(): string;
-```
-
-### `step(action)`
-Execute an action and return the result.
-
-```typescript
-async step(action: number): Promise<StepResult>;
-```
-
-**StepResult:**
-```typescript
-interface StepResult {
-  state: string;      // New state
-  reward: number;     // Reward for this step
-  done: boolean;      // Episode ended
-  info?: object;      // Additional info
-}
-```
-
-### `actionSize()`
-Return number of possible actions.
-
-```typescript
-actionSize(): number;
-```
-
-### `reset()`
-Reset environment to initial state.
-
-```typescript
-async reset(): Promise<string>;  // Returns initial state
-```
-
-## Optional Methods
-
-### `render()`
-Human-readable representation for debugging.
-
-```typescript
-render(): string;
-```
-
-## StateEncoder Utility
-
-Helper for encoding/decoding complex states:
-
-```typescript
-import { StateEncoder } from 'lsji';
-
-// Encode object to string
-const key = StateEncoder.encode({ position: 5, inventory: ['sword'] });
-// Returns: '{"position":5,"inventory":["sword"]}'
-
-// Decode string to object
-const state = StateEncoder.decode(key);
-// Returns: { position: 5, inventory: ['sword'] }
-```
-
-## Creating a Custom Environment
-
-```typescript
-import { Env } from 'lsji';
-
-class GridWorldEnv extends Env {
-  constructor() {
-    super();
-    this.position = 0;
-    this.gridSize = 10;
-  }
-
-  getState() {
-    return String(this.position);
-  }
-
-  async step(action) {
-    // Actions: 0=left, 1=right
-    if (action === 0) this.position = Math.max(0, this.position - 1);
-    if (action === 1) this.position = Math.min(this.gridSize - 1, this.position + 1);
-
-    const done = this.position === this.gridSize - 1;
-    const reward = done ? 1 : -0.01;
-
-    return {
-      state: String(this.position),
-      reward,
-      done,
-      info: { position: this.position }
-    };
-  }
-
-  actionSize() {
-    return 2;
-  }
-
-  async reset() {
-    this.position = 0;
-    return '0';
-  }
-
-  render() {
-    return `GridWorld: position ${this.position}/${this.gridSize - 1}`;
-  }
-}
-```
-
-## Best Practices
-
-1. **State as string** — Use simple string keys for Q-table indexing
-2. **Deterministic rewards** — Same state-action should give consistent rewards
-3. **Action space** — Keep action space small for tabular Q-learning
-4. **Reset** — Always implement proper reset for episode boundaries

package/docs/docs/api/environments.md

@@ -1,102 +0,0 @@
----
-title: Built-in Environments
-description: Pre-built environments for quick start
----
-
-# Built-in Environments
-
-LSJI includes a Rock-Paper-Scissors environment for demonstration and testing.
-
-## Import
-
-```typescript
-import { 
-  RockPaperScissorsEnv, 
-  TrainingPattern, 
-  getTrainingAction 
-} from 'lsji';
-```
-
-## RockPaperScissorsEnv
-
-Classic Rock-Paper-Scissors game environment.
-
-### Constructor
-
-```typescript
-const env = new RockPaperScissorsEnv({
-  opponent: 'random'  // 'random' | 'always_rock' | 'counter' | 'sequential'
-});
-```
-
-### Opponent Strategies
-
-| Strategy | Description |
-|----------|-------------|
-| `'random'` | Uniform random actions (default) |
-| `'always_rock'` | Always plays Rock (0) |
-| `'counter'` | Plays counter to agent's previous action |
-| `'sequential'` | Cycles through Rock→Scissors→Paper |
-
-### Methods
-
-All standard `Env` methods plus:
-
-```typescript
-// Static helpers
-RockPaperScissorsEnv.getHandName(0);     // 'Rock'
-RockPaperScissorsEnv.getHandName(1);     // 'Scissors'
-RockPaperScissorsEnv.getHandName(2);     // 'Paper'
-
-const { judge, reward, outcome } = RockPaperScissorsEnv.calculateOutcome(0, 2);
-// judge: 2, reward: 1, outcome: 'WIN'
-```
-
-### Training Patterns
-
-```typescript
-import { TrainingPattern, getTrainingAction } from 'lsji';
-
-// Pattern IDs
-TrainingPattern.RANDOM;        // 0
-TrainingPattern.ALWAYS_ROCK;   // 1
-TrainingPattern.COUNTER;       // 2
-TrainingPattern.SEQUENTIAL;    // 3
-
-// Get action for pattern
-const action = getTrainingAction(TrainingPattern.COUNTER, episode, lastAction);
-```
-
-### Example
-
-```typescript
-import { 
-  Agent, QLearning, createStorage, 
-  RockPaperScissorsEnv, TrainingPattern, getTrainingAction 
-} from 'lsji';
-
-const storage = await createStorage('sqlite', { path: './rps.db' });
-const qlearning = new QLearning({ alpha: 0.1, gamma: 0.9, epsilon: 0.1, storage });
-
-// Train against counter opponent
-const env = new RockPaperScissorsEnv({ opponent: 'counter' });
-const agent = new Agent({ qlearning, storage, env });
-
-await agent.train({ 
-  episodes: 1000,
-  actionSelector: (ep, last) => getTrainingAction(TrainingPattern.RANDOM, ep, last)
-});
-
-// Play against random opponent
-const playEnv = new RockPaperScissorsEnv({ opponent: 'random' });
-agent.setEnvironment(playEnv);
-
-const result = await agent.play(0); // You play Rock
-console.log(`AI: ${RockPaperScissorsEnv.getHandName(result.action)} | ${result.reward > 0 ? 'WIN' : 'LOSE'}`);
-
-await storage.close();
-```
-
-## Creating Custom Environments
-
-See [Custom Environment Example](/docs/examples/custom-environment) for a complete guide.

package/docs/docs/api/qlearning.md

@@ -1,138 +0,0 @@
----
-title: QLearning API
-description: Tabular Q-Learning engine with TD updates
----
-
-# QLearning API
-
-The `QLearning` class implements tabular Q-Learning with epsilon-greedy exploration.
-
-## Import
-
-```typescript
-import { QLearning } from 'lsji';
-```
-
-## Constructor
-
-```typescript
-const qlearning = new QLearning({
-  alpha: 0.1,     // Learning rate [0, 1] (default: 0.1)
-  gamma: 0.9,     // Discount factor [0, 1] (default: 0.9)
-  epsilon: 0.1,   // Exploration rate [0, 1] (default: 0.1)
-  storage: Storage // Required: Storage backend
-});
-```
-
-Values are automatically clamped to [0, 1] range.
-
-## Methods
-
-### `initialize()`
-Load Q-table from storage. Called automatically by other methods.
-
-```typescript
-await qlearning.initialize();
-```
-
-### `getQValue(state, action)`
-Get Q-value for a state-action pair.
-
-```typescript
-const value = qlearning.getQValue('state1', 0);
-// Returns: number (0 if unseen)
-```
-
-### `setQValue(state, action, value)`
-Set Q-value and persist to storage.
-
-```typescript
-await qlearning.setQValue('state1', 0, 0.5);
-```
-
-### `act(state, actionSize)`
-Select action using epsilon-greedy policy.
-
-```typescript
-const action = await qlearning.act('state1', 3);
-// Returns: number (0 to actionSize-1)
-```
-
-**Behavior:**
-- With probability `epsilon`: random action
-- With probability `1-epsilon`: best known action (ties broken randomly)
-
-### `learn(state, action, reward, nextState, nextActionSize)`
-Full TD update: Q(s,a) ← Q(s,a) + α[r + γ·maxₐ' Q(s',a') - Q(s,a)]
-
-```typescript
-const newQ = await qlearning.learn('state1', 0, 1, 'state2', 3);
-// Returns: updated Q-value
-```
-
-### `learnSimple(state, action, reward)`
-Simplified update for terminal states: Q(s,a) ← Q(s,a) + α[r - Q(s,a)]
-
-```typescript
-const newQ = await qlearning.learnSimple('state1', 0, 1);
-// Returns: updated Q-value
-```
-
-### `reset()`
-Clear in-memory Q-table cache.
-
-```typescript
-await qlearning.reset();
-// Note: Does not clear storage
-```
-
-### `getStateValues(state, actionSize)`
-Get all Q-values for a state.
-
-```typescript
-const values = qlearning.getStateValues('state1', 3);
-// Returns: { 0: 0.5, 1: 0.2, 2: -0.1 }
-```
-
-### `getFullQTable()`
-Get entire Q-table for inspection.
-
-```typescript
-const table = await qlearning.getFullQTable();
-// Returns:
-[
-  { state: 'state1', action: 0, q_value: 0.5 },
-  { state: 'state1', action: 1, q_value: 0.2 },
-  ...
-]
-```
-
-## Hyperparameters
-
-| Parameter | Default | Range | Description |
-|-----------|---------|-------|-------------|
-| `alpha` | 0.1 | [0, 1] | Learning rate — how much new info overrides old |
-| `gamma` | 0.9 | [0, 1] | Discount factor — future reward importance |
-| `epsilon` | 0.1 | [0, 1] | Exploration rate — random action probability |
-
-## Example
-
-```typescript
-import { QLearning, createStorage } from 'lsji';
-
-const storage = await createStorage('memory');
-const ql = new QLearning({ alpha: 0.1, gamma: 0.9, epsilon: 0.1, storage });
-
-await ql.initialize();
-
-// Train on simple state-action pairs
-await ql.learnSimple('state1', 0, 1);  // Win
-await ql.learnSimple('state1', 0, 1);  // Win again
-await ql.learnSimple('state1', 1, -1); // Lose
-
-// Check learned values
-console.log(ql.getQValue('state1', 0)); // ~0.19
-console.log(ql.getQValue('state1', 1)); // ~-0.1
-
-await storage.close();
-```

package/docs/docs/api/storage.md

@@ -1,168 +0,0 @@
----
-title: Storage API
-description: Pluggable storage backends
----
-
-# Storage API
-
-LSJI provides a pluggable storage abstraction with three built-in implementations.
-
-## Import
-
-```typescript
-import { createStorage, Storage, SqliteStorage, BetterSqliteStorage, MemoryStorage } from 'lsji';
-```
-
-## Factory Function
-
-```typescript
-const storage = await createStorage(type, options);
-```
-
-**Types:**
-- `'sqlite'` — Node.js built-in `node:sqlite` (recommended)
-- `'better-sqlite'` — `better-sqlite3` package
-- `'memory'` — In-memory (testing only)
-
-**Options:**
-```typescript
-// SQLite
-{ path: './my-agent.db' }
-
-// Better-SQLite
-{ path: './my-agent.db' }
-
-// Memory
-{}  // No options needed
-```
-
-## Storage Interface
-
-All backends implement this interface:
-
-```typescript
-interface Storage {
-  initialize(): Promise<void>;
-  close(): Promise<void>;
-  getSetting(key: string): Promise<{key: string, value: string} | null>;
-  setSetting(key: string, value: string | number): Promise<void>;
-  getQTable(): Promise<Array<{state: string, action: number, q_value: number}>>;
-  updateQ(state: string, action: number, qValue: number): Promise<void>;
-  addBattle(record: BattleRecord): Promise<void>;
-  getTodayBattleCount(): Promise<number>;
-  getPerformanceStats(): Promise<Array<{mode: string, total: number, win_rate: number}>>;
-}
-
-interface BattleRecord {
-  mode: 'train' | 'test';
-  handA: number;
-  handB: number;
-  reward: number;
-  createdAt: string;  // ISO timestamp
-}
-```
-
-## Backends
-
-### SqliteStorage (Recommended)
-
-Uses Node.js 22+ built-in `node:sqlite` — **zero dependencies**.
-
-```typescript
-import { SqliteStorage } from 'lsji';
-
-const storage = new SqliteStorage('./agent.db');
-await storage.initialize();
-// ... use storage
-await storage.close();
-```
-
-**Features:**
-- Synchronous API (fast)
-- Automatic schema creation
-- Indexes on battle_history for performance
-- Persistent across process restarts
-
-### BetterSqliteStorage
-
-Uses `better-sqlite3` for high-performance synchronous access.
-
-```typescript
-import { BetterSqliteStorage } from 'lsji';
-
-const storage = new BetterSqliteStorage('./agent.db');
-await storage.initialize();
-```
-
-**Requires:** `npm install better-sqlite3`
-
-**Use when:** You need faster writes or advanced SQLite features.
-
-### MemoryStorage
-
-Pure in-memory implementation.
-
-```typescript
-import { MemoryStorage } from 'lsji';
-
-const storage = new MemoryStorage();
-await storage.initialize();
-```
-
-**Features:**
-- No persistence (data lost on exit)
-- Fastest for testing/CI
-- Implements `clear()` for test isolation
-
-## Database Schema
-
-All SQL backends create these tables:
-
-```sql
--- Settings (key-value)
-CREATE TABLE settings (
-  key TEXT PRIMARY KEY,
-  value TEXT NOT NULL
-);
-
--- Battle history
-CREATE TABLE battle_history (
-  id INTEGER PRIMARY KEY AUTOINCREMENT,
-  mode TEXT NOT NULL CHECK (mode IN ('train', 'test')),
-  hand_a INTEGER NOT NULL,
-  hand_b INTEGER NOT NULL,
-  reward INTEGER NOT NULL,
-  created_at TEXT NOT NULL
-);
-
--- Q-table
-CREATE TABLE q_table (
-  state TEXT NOT NULL,
-  action INTEGER NOT NULL,
-  q_value REAL NOT NULL DEFAULT 0,
-  PRIMARY KEY (state, action)
-);
-
--- Indexes
-CREATE INDEX idx_battle_history_date ON battle_history(created_at);
-CREATE INDEX idx_battle_history_mode ON battle_history(mode);
-```
-
-## Example
-
-```typescript
-import { createStorage, QLearning, Agent, RockPaperScissorsEnv } from 'lsji';
-
-// Production: persistent SQLite
-const storage = await createStorage('sqlite', { path: './production.db' });
-
-// Testing: in-memory
-const testStorage = await createStorage('memory');
-
-const qlearning = new QLearning({ alpha: 0.1, gamma: 0.9, epsilon: 0.1, storage });
-const env = new RockPaperScissorsEnv({ opponent: 'random' });
-const agent = new Agent({ qlearning, storage, env });
-
-await agent.train({ episodes: 1000 });
-await storage.close();
-```

package/docs/docs/architecture.md

@@ -1,155 +0,0 @@
----
-title: Architecture
-description: Design decisions and internals
----
-
-# Architecture
-
-## Design Principles
-
-1. **Single Package** — All core functionality in one npm package
-2. **Zero Dependencies** — Core runs on Node.js 22+ built-ins only
-3. **Storage Abstraction** — Easy to swap backends
-4. **ESM Only** — Modern JavaScript modules
-5. **Apache 2.0** — License compatible with Apache Incubator
-
-## Project Structure
-
-```
-src/
-├── core/
-│   ├── env.ts          # Environment interface
-│   ├── qlearning.ts    # Q-Learning engine
-│   └── agent.ts        # Agent orchestration
-├── storage/
-│   ├── index.ts        # Storage interface + factory
-│   ├── sqlite.ts       # node:sqlite implementation
-│   ├── better-sqlite.ts # better-sqlite3 implementation
-│   └── memory.ts       # In-memory implementation
-├── envs/
-│   └── rps.ts          # Rock-Paper-Scissors environment
-├── cli.ts              # Command-line interface
-└── index.ts            # Public API exports
-```
-
-## Core Components
-
-### Q-Learning Engine
-
-**Algorithm:** Tabular Q-Learning with Temporal Difference updates
-
-**Update Rules:**
-- Full TD: `Q(s,a) ← Q(s,a) + α[r + γ·maxₐ' Q(s',a') - Q(s,a)]`
-- Simple: `Q(s,a) ← Q(s,a) + α[r - Q(s,a)]` (terminal states)
-
-**Exploration:** Epsilon-greedy with configurable ε
-
-**State Representation:** String keys for Q-table indexing
-
-### Storage Layer
-
-**Interface:** Abstract `Storage` class with 9 required methods
-
-**Implementations:**
-1. `SqliteStorage` — Node.js built-in `node:sqlite` (sync API)
-2. `BetterSqliteStorage` — `better-sqlite3` (faster, more features)
-3. `MemoryStorage` — Pure JS Maps/Arrays (testing)
-
-**Schema:**
-- `settings` — Key-value configuration
-- `battle_history` — Training/play records with timestamps
-- `q_table` — State-action values with composite primary key
-
-**Indexes:** Date and mode indexes on battle_history for fast queries
-
-### Agent Orchestration
-
-**Responsibilities:**
-- System lifecycle (start/stop)
-- Training loop with batching
-- Play/evaluation loop
-- Statistics aggregation
-
-**Training Loop:**
-```
-for episode in episodes:
-  state = env.getState()
-  action = qlearning.act(state, actionSize)  // ε-greedy
-  result = env.step(action)
-  qlearning.learnSimple(state, action, result.reward)
-  storage.addBattle(record)
-  batch.flush()
-```
-
-### Environment Interface
-
-**Contract:**
-```typescript
-abstract class Env {
-  abstract getState(): string;
-  abstract step(action: number): Promise<StepResult>;
-  abstract actionSize(): number;
-  abstract reset(): Promise<string>;
-  render(): string;  // Optional
-}
-```
-
-**Built-in:** `RockPaperScissorsEnv` with 4 opponent strategies
-
-## Data Flow
-
-```
-┌─────────┐     ┌──────────────┐     ┌─────────┐
-│  Env    │────▶│   Agent      │────▶│ Storage │
-│ (State) │     │  Orchestrates│     │(Persist)│
-└─────────┘     └──────┬───────┘     └─────────┘
-                       │
-                       ▼
-                ┌──────────────┐
-                │  QLearning   │
-                │  (Updates)   │
-                └──────────────┘
-```
-
-## Concurrency Model
-
-- **Single-threaded** Node.js event loop
-- **Synchronous SQLite** — No async/await for DB operations
-- **Batch writes** — Reduce I/O overhead
-- **In-memory Q-table cache** — Fast reads, periodic persistence
-
-## Migration from Cloudflare Workers
-
-**Original:** Worker.js with D1 database, cron triggers, HTTP endpoints
-
-**Changes:**
-| Before | After |
-|--------|-------|
-| `env.DB` (D1) | `Storage` abstraction |
-| Cron triggers | Manual/CLI training |
-| HTTP endpoints | Library API + CLI |
-| Global state | Instance-based |
-| `Request/Response` | Function parameters |
-
-**Preserved:**
-- Q-Learning algorithm (α=0.1, γ=0.9, ε=0.1)
-- Reward calculation: `(ai - user + 3) % 3`
-- Training patterns (0-3)
-- Battle history schema
-
-## Performance Characteristics
-
-| Operation | Complexity | Notes |
-|-----------|------------|-------|
-| `act()` | O(actions) | Scans all actions for max Q |
-| `learnSimple()` | O(1) | Single Q-value update |
-| `learn()` | O(actions) | Finds max next Q |
-| `getFullQTable()` | O(states×actions) | Full table scan |
-| Batch insert | O(batch) | Single transaction |
-
-## Future Extensibility
-
-- **New algorithms:** Extend `QLearning` or add `SARSA`, `DQN` classes
-- **New environments:** Implement `Env` interface
-- **New storage:** Implement `Storage` interface
-- **Function approximation:** Replace tabular Q-table with neural networks