@game_ryo/lsji 0.1.0 → 0.1.1

package/docs/docs/contributing.md

@@ -1,162 +0,0 @@
----
-title: Contributing
-description: How to contribute to LSJI
----
-
-# Contributing
-
-Thank you for your interest in contributing to LSJI!
-
-## Development Setup
-
-```bash
-# Clone the repository
-git clone https://github.com/ryotagtagtag-wq/LSJI.git
-cd LSJI
-
-# Install dependencies
-npm install
-
-# Run tests
-npm test
-
-# Build documentation
-cd docs && npm run build
-```
-
-## Project Structure
-
-```
-LSJI/
-├── src/                 # Core library
-│   ├── core/           # QLearning, Agent, Env
-│   ├── storage/        # Storage backends
-│   ├── envs/           # Built-in environments
-│   ├── cli.ts          # CLI
-│   └── index.ts        # Public exports
-├── test/               # Vitest tests
-├── docs/               # Docusaurus documentation
-└── bin/                # CLI entry point
-```
-
-## Making Changes
-
-### 1. Create a Branch
-
-```bash
-git checkout -b feature/my-feature
-```
-
-### 2. Make Changes
-
-Follow the existing code style:
-- TypeScript with JSDoc comments
-- ESM imports/exports
-- No external dependencies in core
-
-### 3. Run Tests
-
-```bash
-npm test
-```
-
-### 4. Update Documentation
-
-If you add new features, update relevant docs in `docs/docs/`.
-
-### 5. Commit
-
-```bash
-git add .
-git commit -m "feat: add my feature"
-```
-
-**Commit Message Format:**
-- `feat:` — New feature
-- `fix:` — Bug fix
-- `docs:` — Documentation
-- `refactor:` — Code refactoring
-- `test:` — Tests
-- `chore:` — Maintenance
-
-### 6. Push and Create PR
-
-```bash
-git push origin feature/my-feature
-```
-
-## Adding a New Environment
-
-1. Create `src/envs/my-env.ts` extending `Env`
-2. Implement all abstract methods
-3. Export from `src/index.ts`
-4. Add documentation in `docs/docs/api/environments.md`
-5. Add example in `docs/docs/examples/`
-
-## Adding a New Storage Backend
-
-1. Create `src/storage/my-backend.ts` extending `Storage`
-2. Implement all abstract methods
-3. Add to `createStorage` factory in `src/storage/index.ts`
-3. Export from `src/index.ts`
-4. Add tests in `test/storage/`
-
-## Modifying Learning Algorithm
-
-1. Extend `QLearning` class or create new class in `src/core/`
-2. Maintain compatibility with `Agent` interface
-3. Add tests for new algorithm
-4. Document in `docs/docs/api/`
-
-## Code Style
-
-- **TypeScript** with strict mode
-- **ESM** modules (`import`/`export`)
-- **JSDoc** for all public APIs
-- **No `any`** unless absolutely necessary
-- **Async/await** for async operations
-
-## Testing Guidelines
-
-- Use `MemoryStorage` for unit tests
-- Test both success and error cases
-- Test edge cases (empty Q-table, terminal states)
-- Keep tests fast and isolated
-
-```typescript
-// Example test structure
-import { describe, it, expect, beforeEach } from 'vitest';
-import { MyFeature } from '../src/core/my-feature';
-import { MemoryStorage } from '../src/storage/memory';
-
-describe('MyFeature', () => {
-  let storage;
-  let feature;
-
-  beforeEach(async () => {
-    storage = new MemoryStorage();
-    await storage.initialize();
-    feature = new MyFeature({ storage });
-  });
-
-  it('should do something', async () => {
-    const result = await feature.doSomething();
-    expect(result).toBe(expected);
-  });
-});
-```
-
-## Documentation
-
-- Update relevant `.md` files in `docs/docs/`
-- Add JSDoc comments for new public APIs
-- Include code examples
-
-## License
-
-By contributing, you agree that your contributions will be licensed under the Apache 2.0 License.
-
-## Questions?
-
-- Open a [GitHub Issue](https://github.com/ryotagtagtag-wq/LSJI/issues)
-- Start a [Discussion](https://github.com/ryotagtagtag-wq/LSJI/discussions)

package/docs/docs/core-concepts.md

@@ -1,152 +0,0 @@
----
-title: Core Concepts
-description: Understand the core architecture of LSJI
----
-
-# Core Concepts
-
-LSJI is built around four core abstractions that work together to create a flexible reinforcement learning framework.
-
-## Architecture Overview
-
-```
-┌─────────────────────────────────────────────────────────────┐
-│                        Agent                                 │
-│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
-│  │ QLearning   │  │  Storage    │  │      Env            │  │
-│  │ (Engine)    │◄─┤ (Backend)   │  │ (Environment)       │  │
-│  └─────────────┘  └─────────────┘  └─────────────────────┘  │
-└─────────────────────────────────────────────────────────────┘
-```
-
-### 1. Environment (`Env`)
-
-The `Env` interface defines the problem domain. Any RL environment must implement:
-
-```typescript
-abstract class Env {
-  getState(): string;                    // Current state representation
-  step(action: number): Promise<StepResult>;  // Execute action
-  actionSize(): number;                  // Number of possible actions
-  reset(): Promise<string>;              // Reset to initial state
-}
-```
-
-**StepResult** contains:
-- `state` — New state after action
-- `reward` — Reward received (-1, 0, 1)
-- `done` — Whether episode ended
-- `info` — Additional diagnostic info
-
-### 2. Q-Learning Engine (`QLearning`)
-
-Tabular Q-Learning with Temporal Difference (TD) updates:
-
-```typescript
-class QLearning {
-  constructor({ alpha, gamma, epsilon, storage });
-  
-  // Epsilon-greedy action selection
-  async act(state: string, actionSize: number): Promise<number>;
-  
-  // Full TD update: Q(s,a) ← Q(s,a) + α[r + γ·max Q(s',a') - Q(s,a)]
-  async learn(state, action, reward, nextState, nextActionSize);
-  
-  // Simplified update (terminal states): Q(s,a) ← Q(s,a) + α[r - Q(s,a)]
-  async learnSimple(state, action, reward);
-  
-  // Get all Q-values for inspection
-  async getFullQTable(): Promise<QTableRecord[]>;
-}
-```
-
-**Hyperparameters:**
-- `alpha` (0.1) — Learning rate
-- `gamma` (0.9) — Discount factor  
-- `epsilon` (0.1) — Exploration rate
-
-### 3. Storage Backend (`Storage`)
-
-Pluggable persistence layer with three implementations:
-
-| Backend | Package | Use Case |
-|---------|---------|----------|
-| `SqliteStorage` | `node:sqlite` (built-in) | **Recommended** — Zero dependencies |
-| `BetterSqliteStorage` | `better-sqlite3` | High-performance synchronous access |
-| `MemoryStorage` | Built-in | Testing, CI, ephemeral workloads |
-
-All implement the same interface:
-```typescript
-interface Storage {
-  initialize(): Promise<void>;
-  close(): Promise<void>;
-  getSetting(key): Promise<Setting>;
-  setSetting(key, value): Promise<void>;
-  getQTable(): Promise<QTableRecord[]>;
-  updateQ(state, action, qValue): Promise<void>;
-  addBattle(record): Promise<void>;
-  getTodayBattleCount(): Promise<number>;
-  getPerformanceStats(): Promise<PerformanceStat[]>;
-}
-```
-
-### 4. Agent (`Agent`)
-
-High-level orchestration combining all components:
-
-```typescript
-class Agent {
-  constructor({ qlearning, storage, env });
-  
-  async train({ episodes, actionSelector, batchSize });
-  async play(options?): Promise<PlayResult>;
-  async status(): Promise<StatusInfo>;
-  async start(): Promise<{status, message}>;
-  async stop(): Promise<{status, message}>;
-  setEnvironment(env): void;
-}
-```
-
-## Data Flow
-
-### Training Loop
-```
-for each episode:
-  1. Get current state from Env
-  2. Select action via QLearning.act() (ε-greedy)
-  3. Execute action in Env → StepResult
-  4. Update Q-table via QLearning.learnSimple()
-  5. Persist battle record to Storage
-  6. Batch DB writes for performance
-```
-
-### Play Loop
-```
-1. Get current state from Env
-2. Select best action via QLearning.act() (ε=0 for exploitation)
-3. Execute action in Env
-4. Update Q-table with result
-5. Record battle to Storage
-6. Return result
-```
-
-## Reward System (RPS Example)
-
-| Outcome | Judge Formula | Reward |
-|---------|---------------|--------|
-| Win     | (ai - user + 3) % 3 = 2 | +1 |
-| Lose    | (ai - user + 3) % 3 = 1 | -1 |
-| Draw    | (ai - user + 3) % 3 = 0 | 0 |
-
-## Training Patterns
-
-Built-in patterns for the RPS environment:
-
-| Pattern | ID | Description |
-|---------|-----|-------------|
-| Random | 0 | Uniform random actions |
-| Always Rock | 1 | Always play action 0 |
-| Counter | 2 | Play counter to previous action |
-| Sequential | 3 | Cycle through 0,1,2,0,1,2... |
-
-Custom patterns can be implemented via `actionSelector` function.

package/docs/docs/examples/advanced-training.md

@@ -1,244 +0,0 @@
----
-title: Advanced Training
-description: Custom training patterns and techniques
----
-
-# Advanced Training
-
-Learn advanced training techniques for better agent performance.
-
-## Custom Action Selectors
-
-The `train()` method accepts an `actionSelector` function for custom training patterns.
-
-```typescript
-const result = await agent.train({
-  episodes: 1000,
-  actionSelector: (episode, lastAction) => {
-    // Your custom logic here
-    return action;
-  }
-});
-```
-
-### Epsilon-Greedy with Decay
-
-```typescript
-let epsilon = 1.0;
-const minEpsilon = 0.01;
-const decayRate = 0.9995;
-
-const result = await agent.train({
-  episodes: 10000,
-  actionSelector: (episode, lastAction) => {
-    epsilon = Math.max(minEpsilon, epsilon * decayRate);
-    
-    if (Math.random() < epsilon) {
-      return Math.floor(Math.random() * 3); // Explore
-    }
-    
-    // Exploit: use agent's Q-learning
-    const state = await agent.env.getState();
-    return agent.qlearning.act(state, 3);
-  }
-});
-```
-
-### Curriculum Learning
-
-Start with easy opponents, progress to harder ones.
-
-```typescript
-const opponents = ['always_rock', 'sequential', 'counter', 'random'];
-const episodesPerStage = 250;
-
-for (const opponent of opponents) {
-  const env = new RockPaperScissorsEnv({ opponent });
-  agent.setEnvironment(env);
-  
-  console.log(`Training against ${opponent}...`);
-  await agent.train({ episodes: episodesPerStage });
-  
-  const status = await agent.status();
-  console.log(`Win rate: ${status.performance.find(p => p.mode === 'train')?.win_rate}%`);
-}
-```
-
-### Self-Play Training
-
-Train agent against itself.
-
-```typescript
-// Create two agents sharing the same Q-table
-const storage = await createStorage('sqlite', { path: './selfplay.db' });
-const qlearning = new QLearning({ alpha: 0.1, gamma: 0.9, epsilon: 0.1, storage });
-
-const env1 = new RockPaperScissorsEnv({ opponent: 'random' });
-const env2 = new RockPaperScissorsEnv({ opponent: 'random' });
-
-const agent1 = new Agent({ qlearning, storage, env: env1 });
-const agent2 = new Agent({ qlearning, storage, env: env2 });
-
-// Alternate training
-for (let i = 0; i < 100; i++) {
-  await agent1.train({ episodes: 50 });
-  await agent2.train({ episodes: 50 });
-  
-  if (i % 10 === 0) {
-    const status = await agent1.status();
-    console.log(`Iteration ${i}: ${status.performance[0].win_rate}% win rate`);
-  }
-}
-```
-
-## Hyperparameter Tuning
-
-### Grid Search
-
-```typescript
-const configs = [
-  { alpha: 0.05, gamma: 0.9, epsilon: 0.1 },
-  { alpha: 0.1, gamma: 0.9, epsilon: 0.1 },
-  { alpha: 0.2, gamma: 0.9, epsilon: 0.1 },
-  { alpha: 0.1, gamma: 0.95, epsilon: 0.1 },
-  { alpha: 0.1, gamma: 0.9, epsilon: 0.2 },
-];
-
-for (const config of configs) {
-  const storage = await createStorage('memory');
-  const qlearning = new QLearning({ ...config, storage });
-  const env = new RockPaperScissorsEnv({ opponent: 'random' });
-  const agent = new Agent({ qlearning, storage, env });
-  
-  await agent.train({ episodes: 2000 });
-  const status = await agent.status();
-  const winRate = status.performance.find(p => p.mode === 'train')?.win_rate || 0;
-  
-  console.log(`${JSON.stringify(config)} => ${winRate}%`);
-  await storage.close();
-}
-```
-
-### Bayesian Optimization
-
-Use libraries like `bayes-opt` for efficient hyperparameter search.
-
-## Evaluation Techniques
-
-### Fixed Opponent Evaluation
-
-```typescript
-async function evaluate(agent, opponent, games = 100) {
-  const env = new RockPaperScissorsEnv({ opponent });
-  agent.setEnvironment(env);
-  
-  let wins = 0, losses = 0, draws = 0;
-  
-  for (let i = 0; i < games; i++) {
-    const result = await agent.play(Math.floor(Math.random() * 3));
-    if (result.reward > 0) wins++;
-    else if (result.reward < 0) losses++;
-    else draws++;
-  }
-  
-  return { wins, losses, draws, winRate: wins / games };
-}
-
-const agents = {
-  random: await evaluate(agent, 'random'),
-  alwaysRock: await evaluate(agent, 'always_rock'),
-  counter: await evaluate(agent, 'counter'),
-  sequential: await evaluate(agent, 'sequential'),
-};
-
-console.table(agents);
-```
-
-### Cross-Validation
-
-```typescript
-async function crossValidate(config, folds = 5, episodesPerFold = 1000) {
-  const results = [];
-  
-  for (let fold = 0; fold < folds; fold++) {
-    const storage = await createStorage('memory');
-    const qlearning = new QLearning({ ...config, storage });
-    const env = new RockPaperScissorsEnv({ opponent: 'random' });
-    const agent = new Agent({ qlearning, storage, env });
-    
-    await agent.train({ episodes: episodesPerFold });
-    const evalResult = await evaluate(agent, 'random', 200);
-    results.push(evalResult.winRate);
-    
-    await storage.close();
-  }
-  
-  const mean = results.reduce((a, b) => a + b, 0) / results.length;
-  const std = Math.sqrt(results.reduce((a, b) => a + (b - mean) ** 2, 0) / results.length);
-  
-  return { mean, std, results };
-}
-```
-
-## Checkpointing and Resuming
-
-```typescript
-// Save Q-table periodically
-async function trainWithCheckpoints(agent, episodes, checkpointEvery = 100) {
-  for (let i = 0; i < episodes; i += checkpointEvery) {
-    const batch = Math.min(checkpointEvery, episodes - i);
-    await agent.train({ episodes: batch });
-    
-    // Q-table automatically persisted to storage
-    const status = await agent.status();
-    console.log(`Checkpoint ${i + batch}: ${status.aiBrain.length} Q-values`);
-  }
-}
-
-// Resume from existing Q-table
-const storage = await createStorage('sqlite', { path: './existing.db' });
-const qlearning = new QLearning({ alpha: 0.1, gamma: 0.9, epsilon: 0.1, storage });
-// Q-table loads automatically on first use
-```
-
-## Distributed Training
-
-Run multiple training processes with shared storage.
-
-```bash
-# Terminal 1
-lsji train --episodes 500 --db-path ./shared.db --storage sqlite
-
-# Terminal 2 (same database)
-lsji train --episodes 500 --db-path ./shared.db --storage sqlite
-
-# Terminal 3
-lsji train --episodes 500 --db-path ./shared.db --storage sqlite
-```
-
-All processes read/write to the same SQLite database, enabling parallel training.
-
-## Monitoring Training Progress
-
-```typescript
-async function trainWithLogging(agent, episodes) {
-  const history = [];
-  
-  for (let i = 0; i < episodes; i += 100) {
-    await agent.train({ episodes: 100 });
-    
-    const status = await agent.status();
-    const trainStat = status.performance.find(p => p.mode === 'train');
-    
-    history.push({
-      episode: i + 100,
-      winRate: trainStat?.win_rate || 0,
-      qTableSize: status.aiBrain.length
-    });
-    
-    console.log(`Episode ${i + 100}: ${history[history.length - 1].winRate}% win rate`);
-  }
-  
-  return history;
-}
-```