@game_ryo/lsji 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (75) hide show
  1. package/package.json +15 -7
  2. package/src/cli.js +395 -62
  3. package/src/execution/budget/circuit-breaker.js +245 -0
  4. package/src/execution/budget/cost-tracker.js +387 -0
  5. package/src/execution/budget/index.js +63 -0
  6. package/src/execution/budget/token-counter.js +159 -0
  7. package/src/execution/engine.js +428 -0
  8. package/src/execution/hitl/approval-gate.js +210 -0
  9. package/src/execution/hitl/index.js +12 -0
  10. package/src/execution/hitl/notifier.js +151 -0
  11. package/src/execution/hitl/store.js +311 -0
  12. package/src/execution/idempotency.js +312 -0
  13. package/src/execution/index.js +14 -0
  14. package/src/index.js +80 -4
  15. package/src/llm/index.js +21 -0
  16. package/src/llm/llm-agent.js +357 -0
  17. package/src/llm/memory/conversation.js +271 -0
  18. package/src/llm/memory/episodic.js +312 -0
  19. package/src/llm/memory/index.js +12 -0
  20. package/src/llm/memory/semantic.js +324 -0
  21. package/src/llm/plugins/index.js +202 -0
  22. package/src/llm/prompt-manager.js +332 -0
  23. package/src/llm/providers/anthropic.js +250 -0
  24. package/src/llm/providers/base.js +116 -0
  25. package/src/llm/providers/local.js +163 -0
  26. package/src/llm/providers/openai.js +212 -0
  27. package/src/llm/tools/registry.js +342 -0
  28. package/src/server/index.js +416 -0
  29. package/src/server/ui/index.html +16 -0
  30. package/src/server/ui/package.json +19 -0
  31. package/src/server/ui/src/main.jsx +10 -0
  32. package/src/server/ui/src/styles.css +260 -0
  33. package/src/server/ui/vite.config.js +27 -0
  34. package/docs/README.md +0 -43
  35. package/docs/blog/2019-05-28-first-blog-post.mdx +0 -12
  36. package/docs/blog/2019-05-29-long-blog-post.mdx +0 -44
  37. package/docs/blog/2021-08-01-mdx-blog-post.mdx +0 -24
  38. package/docs/blog/2021-08-26-welcome/docusaurus-plushie-banner.jpeg +0 -0
  39. package/docs/blog/2021-08-26-welcome/index.mdx +0 -29
  40. package/docs/blog/authors.yml +0 -25
  41. package/docs/blog/tags.yml +0 -19
  42. package/docs/docs/api/agent.md +0 -151
  43. package/docs/docs/api/env.md +0 -133
  44. package/docs/docs/api/environments.md +0 -102
  45. package/docs/docs/api/qlearning.md +0 -138
  46. package/docs/docs/api/storage.md +0 -168
  47. package/docs/docs/architecture.md +0 -155
  48. package/docs/docs/cli.md +0 -210
  49. package/docs/docs/contributing.md +0 -162
  50. package/docs/docs/core-concepts.md +0 -152
  51. package/docs/docs/examples/advanced-training.md +0 -244
  52. package/docs/docs/examples/custom-environment.md +0 -198
  53. package/docs/docs/examples/custom-storage.md +0 -251
  54. package/docs/docs/getting-started.md +0 -91
  55. package/docs/docusaurus.config.ts +0 -149
  56. package/docs/package-lock.json +0 -19522
  57. package/docs/package.json +0 -49
  58. package/docs/sidebars.ts +0 -33
  59. package/docs/src/components/HomepageFeatures/index.tsx +0 -71
  60. package/docs/src/components/HomepageFeatures/styles.module.css +0 -11
  61. package/docs/src/css/custom.css +0 -79
  62. package/docs/src/pages/index.module.css +0 -23
  63. package/docs/src/pages/index.tsx +0 -44
  64. package/docs/src/pages/markdown-page.mdx +0 -7
  65. package/docs/static/.nojekyll +0 -0
  66. package/docs/static/img/docusaurus-social-card.jpg +0 -0
  67. package/docs/static/img/docusaurus.png +0 -0
  68. package/docs/static/img/favicon.ico +0 -0
  69. package/docs/static/img/logo.png +0 -0
  70. package/docs/static/img/undraw_docusaurus_mountain.svg +0 -171
  71. package/docs/static/img/undraw_docusaurus_react.svg +0 -170
  72. package/docs/static/img/undraw_docusaurus_tree.svg +0 -40
  73. package/docs/tsconfig.json +0 -12
  74. package/legacy/worker.js +0 -166
  75. package/legacy/wrangler.toml +0 -11
package/docs/docs/cli.md DELETED
@@ -1,210 +0,0 @@
1
- ---
2
- title: CLI Reference
3
- description: Command-line interface for training and playing
4
- ---
5
-
6
- # CLI Reference
7
-
8
- LSJI includes a command-line interface for training and playing without writing code.
9
-
10
- ## Installation
11
-
12
- ```bash
13
- # Global install
14
- npm install -g lsji
15
-
16
- # Or use npx
17
- npx lsji --help
18
- ```
19
-
20
- ## Commands
21
-
22
- ### `lsji train`
23
-
24
- Train the agent.
25
-
26
- ```bash
27
- lsji train [options]
28
- ```
29
-
30
- **Options:**
31
-
32
- | Option | Description | Default |
33
- |--------|-------------|---------|
34
- | `--episodes <n>` | Number of training episodes | 200 |
35
- | `--pattern <0-3>` | Training pattern | 0 |
36
- | `--batch-size <n>` | Database batch size | 200 |
37
- | `--opponent <type>` | Opponent strategy | random |
38
- | `--storage <type>` | Storage backend | sqlite |
39
- | `--db-path <path>` | Database file path | ./lsji.db |
40
- | `--alpha <n>` | Learning rate | 0.1 |
41
- | `--gamma <n>` | Discount factor | 0.9 |
42
- | `--epsilon <n>` | Exploration rate | 0.1 |
43
- | `--json` | Output as JSON | false |
44
-
45
- **Training Patterns:**
46
- - `0` — Random actions
47
- - `1` — Always Rock
48
- - `2` — Counter previous action
49
- - `3` — Sequential (0,1,2,0,1,2...)
50
-
51
- **Opponent Strategies:**
52
- - `random` — Random actions
53
- - `always_rock` — Always plays Rock
54
- - `counter` — Counters agent's previous action
55
- - `sequential` — Cycles through actions
56
-
57
- **Examples:**
58
- ```bash
59
- # Default training
60
- lsji train --episodes 500
61
-
62
- # Train against always-rock opponent
63
- lsji train --episodes 100 --pattern 1 --opponent always_rock
64
-
65
- # Train with custom hyperparameters
66
- lsji train --episodes 1000 --alpha 0.05 --gamma 0.95 --epsilon 0.2
67
-
68
- # Use memory storage (ephemeral)
69
- lsji train --episodes 100 --storage memory
70
- ```
71
-
72
- ### `lsji play`
73
-
74
- Play a single game against the agent.
75
-
76
- ```bash
77
- lsji play --hand <0|1|2> [options]
78
- ```
79
-
80
- **Options:**
81
-
82
- | Option | Description |
83
- |--------|-------------|
84
- | `--hand <0\|1\|2>` | Your hand: 0=Rock, 1=Scissors, 2=Paper |
85
- | `--opponent <type>` | Opponent strategy |
86
- | `--storage <type>` | Storage backend |
87
- | `--db-path <path>` | Database file path |
88
- | `--json` | Output as JSON |
89
-
90
- **Examples:**
91
- ```bash
92
- lsji play --hand 0 # Play Rock
93
- lsji play --hand 1 # Play Scissors
94
- lsji play --hand 2 --json # Play Paper, JSON output
95
- ```
96
-
97
- ### `lsji status`
98
-
99
- Show system status and statistics.
100
-
101
- ```bash
102
- lsji status [options]
103
- ```
104
-
105
- **Options:**
106
-
107
- | Option | Description |
108
- |--------|-------------|
109
- | `--storage <type>` | Storage backend |
110
- | `--db-path <path>` | Database file path |
111
- | `--json` | Output as JSON |
112
-
113
- **Example:**
114
- ```bash
115
- lsji status --json
116
- ```
117
-
118
- Output:
119
- ```json
120
- {
121
- "status": "running",
122
- "todayTotal": 42,
123
- "limit": 90000,
124
- "performance": [
125
- { "mode": "train", "total": 1000, "win_rate": 65.5 },
126
- { "mode": "test", "total": 50, "win_rate": 72.0 }
127
- ],
128
- "aiBrain": [
129
- { "state": "0", "action": 0, "q_value": 0.45 },
130
- { "state": "0", "action": 1, "q_value": 0.12 }
131
- ]
132
- }
133
- ```
134
-
135
- ### `lsji start`
136
-
137
- Enable training and play.
138
-
139
- ```bash
140
- lsji start [options]
141
- ```
142
-
143
- ### `lsji stop`
144
-
145
- Disable training and play (system paused).
146
-
147
- ```bash
148
- lsji stop [options]
149
- ```
150
-
151
- ### `lsji help`
152
-
153
- Show help message.
154
-
155
- ```bash
156
- lsji help
157
- lsji --help
158
- lsji -h
159
- ```
160
-
161
- ## Environment Variables
162
-
163
- | Variable | Description | Default |
164
- |----------|-------------|---------|
165
- | `LSJI_STORAGE` | Default storage backend | sqlite |
166
- | `LSJI_DB_PATH` | Default database path | ./lsji.db |
167
-
168
- ## Examples
169
-
170
- ### Full Training Session
171
-
172
- ```bash
173
- # Start fresh
174
- rm -f lsji.db
175
-
176
- # Train against random opponent
177
- lsji train --episodes 500 --opponent random
178
-
179
- # Train against counter opponent
180
- lsji train --episodes 500 --opponent counter
181
-
182
- # Check progress
183
- lsji status --json
184
-
185
- # Play a few games
186
- lsji play --hand 0
187
- lsji play --hand 1
188
- lsji play --hand 2
189
- ```
190
-
191
- ### Using Memory Storage (CI/Testing)
192
-
193
- ```bash
194
- lsji train --episodes 100 --storage memory
195
- lsji play --hand 0 --storage memory
196
- lsji status --storage memory
197
- ```
198
-
199
- ### Custom Hyperparameters
200
-
201
- ```bash
202
- lsji train \
203
- --episodes 2000 \
204
- --alpha 0.05 \
205
- --gamma 0.95 \
206
- --epsilon 0.2 \
207
- --opponent random \
208
- --storage sqlite \
209
- --db-path ./custom.db
210
- ```
@@ -1,162 +0,0 @@
1
- ---
2
- title: Contributing
3
- description: How to contribute to LSJI
4
- ---
5
-
6
- # Contributing
7
-
8
- Thank you for your interest in contributing to LSJI!
9
-
10
- ## Development Setup
11
-
12
- ```bash
13
- # Clone the repository
14
- git clone https://github.com/ryotagtagtag-wq/LSJI.git
15
- cd LSJI
16
-
17
- # Install dependencies
18
- npm install
19
-
20
- # Run tests
21
- npm test
22
-
23
- # Build documentation
24
- cd docs && npm run build
25
- ```
26
-
27
- ## Project Structure
28
-
29
- ```
30
- LSJI/
31
- ├── src/ # Core library
32
- │ ├── core/ # QLearning, Agent, Env
33
- │ ├── storage/ # Storage backends
34
- │ ├── envs/ # Built-in environments
35
- │ ├── cli.ts # CLI
36
- │ └── index.ts # Public exports
37
- ├── test/ # Vitest tests
38
- ├── docs/ # Docusaurus documentation
39
- └── bin/ # CLI entry point
40
- ```
41
-
42
- ## Making Changes
43
-
44
- ### 1. Create a Branch
45
-
46
- ```bash
47
- git checkout -b feature/my-feature
48
- ```
49
-
50
- ### 2. Make Changes
51
-
52
- Follow the existing code style:
53
- - TypeScript with JSDoc comments
54
- - ESM imports/exports
55
- - No external dependencies in core
56
-
57
- ### 3. Run Tests
58
-
59
- ```bash
60
- npm test
61
- ```
62
-
63
- ### 4. Update Documentation
64
-
65
- If you add new features, update relevant docs in `docs/docs/`.
66
-
67
- ### 5. Commit
68
-
69
- ```bash
70
- git add .
71
- git commit -m "feat: add my feature"
72
- ```
73
-
74
- **Commit Message Format:**
75
- - `feat:` — New feature
76
- - `fix:` — Bug fix
77
- - `docs:` — Documentation
78
- - `refactor:` — Code refactoring
79
- - `test:` — Tests
80
- - `chore:` — Maintenance
81
-
82
- ### 6. Push and Create PR
83
-
84
- ```bash
85
- git push origin feature/my-feature
86
- ```
87
-
88
- ## Adding a New Environment
89
-
90
- 1. Create `src/envs/my-env.ts` extending `Env`
91
- 2. Implement all abstract methods
92
- 3. Export from `src/index.ts`
93
- 4. Add documentation in `docs/docs/api/environments.md`
94
- 5. Add example in `docs/docs/examples/`
95
-
96
- ## Adding a New Storage Backend
97
-
98
- 1. Create `src/storage/my-backend.ts` extending `Storage`
99
- 2. Implement all abstract methods
100
- 3. Add to `createStorage` factory in `src/storage/index.ts`
101
- 3. Export from `src/index.ts`
102
- 4. Add tests in `test/storage/`
103
-
104
- ## Modifying Learning Algorithm
105
-
106
- 1. Extend `QLearning` class or create new class in `src/core/`
107
- 2. Maintain compatibility with `Agent` interface
108
- 3. Add tests for new algorithm
109
- 4. Document in `docs/docs/api/`
110
-
111
- ## Code Style
112
-
113
- - **TypeScript** with strict mode
114
- - **ESM** modules (`import`/`export`)
115
- - **JSDoc** for all public APIs
116
- - **No `any`** unless absolutely necessary
117
- - **Async/await** for async operations
118
-
119
- ## Testing Guidelines
120
-
121
- - Use `MemoryStorage` for unit tests
122
- - Test both success and error cases
123
- - Test edge cases (empty Q-table, terminal states)
124
- - Keep tests fast and isolated
125
-
126
- ```typescript
127
- // Example test structure
128
- import { describe, it, expect, beforeEach } from 'vitest';
129
- import { MyFeature } from '../src/core/my-feature';
130
- import { MemoryStorage } from '../src/storage/memory';
131
-
132
- describe('MyFeature', () => {
133
- let storage;
134
- let feature;
135
-
136
- beforeEach(async () => {
137
- storage = new MemoryStorage();
138
- await storage.initialize();
139
- feature = new MyFeature({ storage });
140
- });
141
-
142
- it('should do something', async () => {
143
- const result = await feature.doSomething();
144
- expect(result).toBe(expected);
145
- });
146
- });
147
- ```
148
-
149
- ## Documentation
150
-
151
- - Update relevant `.md` files in `docs/docs/`
152
- - Add JSDoc comments for new public APIs
153
- - Include code examples
154
-
155
- ## License
156
-
157
- By contributing, you agree that your contributions will be licensed under the Apache 2.0 License.
158
-
159
- ## Questions?
160
-
161
- - Open a [GitHub Issue](https://github.com/ryotagtagtag-wq/LSJI/issues)
162
- - Start a [Discussion](https://github.com/ryotagtagtag-wq/LSJI/discussions)
@@ -1,152 +0,0 @@
1
- ---
2
- title: Core Concepts
3
- description: Understand the core architecture of LSJI
4
- ---
5
-
6
- # Core Concepts
7
-
8
- LSJI is built around four core abstractions that work together to create a flexible reinforcement learning framework.
9
-
10
- ## Architecture Overview
11
-
12
- ```
13
- ┌─────────────────────────────────────────────────────────────┐
14
- │ Agent │
15
- │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
16
- │ │ QLearning │ │ Storage │ │ Env │ │
17
- │ │ (Engine) │◄─┤ (Backend) │ │ (Environment) │ │
18
- │ └─────────────┘ └─────────────┘ └─────────────────────┘ │
19
- └─────────────────────────────────────────────────────────────┘
20
- ```
21
-
22
- ### 1. Environment (`Env`)
23
-
24
- The `Env` interface defines the problem domain. Any RL environment must implement:
25
-
26
- ```typescript
27
- abstract class Env {
28
- getState(): string; // Current state representation
29
- step(action: number): Promise<StepResult>; // Execute action
30
- actionSize(): number; // Number of possible actions
31
- reset(): Promise<string>; // Reset to initial state
32
- }
33
- ```
34
-
35
- **StepResult** contains:
36
- - `state` — New state after action
37
- - `reward` — Reward received (-1, 0, 1)
38
- - `done` — Whether episode ended
39
- - `info` — Additional diagnostic info
40
-
41
- ### 2. Q-Learning Engine (`QLearning`)
42
-
43
- Tabular Q-Learning with Temporal Difference (TD) updates:
44
-
45
- ```typescript
46
- class QLearning {
47
- constructor({ alpha, gamma, epsilon, storage });
48
-
49
- // Epsilon-greedy action selection
50
- async act(state: string, actionSize: number): Promise<number>;
51
-
52
- // Full TD update: Q(s,a) ← Q(s,a) + α[r + γ·max Q(s',a') - Q(s,a)]
53
- async learn(state, action, reward, nextState, nextActionSize);
54
-
55
- // Simplified update (terminal states): Q(s,a) ← Q(s,a) + α[r - Q(s,a)]
56
- async learnSimple(state, action, reward);
57
-
58
- // Get all Q-values for inspection
59
- async getFullQTable(): Promise<QTableRecord[]>;
60
- }
61
- ```
62
-
63
- **Hyperparameters:**
64
- - `alpha` (0.1) — Learning rate
65
- - `gamma` (0.9) — Discount factor
66
- - `epsilon` (0.1) — Exploration rate
67
-
68
- ### 3. Storage Backend (`Storage`)
69
-
70
- Pluggable persistence layer with three implementations:
71
-
72
- | Backend | Package | Use Case |
73
- |---------|---------|----------|
74
- | `SqliteStorage` | `node:sqlite` (built-in) | **Recommended** — Zero dependencies |
75
- | `BetterSqliteStorage` | `better-sqlite3` | High-performance synchronous access |
76
- | `MemoryStorage` | Built-in | Testing, CI, ephemeral workloads |
77
-
78
- All implement the same interface:
79
- ```typescript
80
- interface Storage {
81
- initialize(): Promise<void>;
82
- close(): Promise<void>;
83
- getSetting(key): Promise<Setting>;
84
- setSetting(key, value): Promise<void>;
85
- getQTable(): Promise<QTableRecord[]>;
86
- updateQ(state, action, qValue): Promise<void>;
87
- addBattle(record): Promise<void>;
88
- getTodayBattleCount(): Promise<number>;
89
- getPerformanceStats(): Promise<PerformanceStat[]>;
90
- }
91
- ```
92
-
93
- ### 4. Agent (`Agent`)
94
-
95
- High-level orchestration combining all components:
96
-
97
- ```typescript
98
- class Agent {
99
- constructor({ qlearning, storage, env });
100
-
101
- async train({ episodes, actionSelector, batchSize });
102
- async play(options?): Promise<PlayResult>;
103
- async status(): Promise<StatusInfo>;
104
- async start(): Promise<{status, message}>;
105
- async stop(): Promise<{status, message}>;
106
- setEnvironment(env): void;
107
- }
108
- ```
109
-
110
- ## Data Flow
111
-
112
- ### Training Loop
113
- ```
114
- for each episode:
115
- 1. Get current state from Env
116
- 2. Select action via QLearning.act() (ε-greedy)
117
- 3. Execute action in Env → StepResult
118
- 4. Update Q-table via QLearning.learnSimple()
119
- 5. Persist battle record to Storage
120
- 6. Batch DB writes for performance
121
- ```
122
-
123
- ### Play Loop
124
- ```
125
- 1. Get current state from Env
126
- 2. Select best action via QLearning.act() (ε=0 for exploitation)
127
- 3. Execute action in Env
128
- 4. Update Q-table with result
129
- 5. Record battle to Storage
130
- 6. Return result
131
- ```
132
-
133
- ## Reward System (RPS Example)
134
-
135
- | Outcome | Judge Formula | Reward |
136
- |---------|---------------|--------|
137
- | Win | (ai - user + 3) % 3 = 2 | +1 |
138
- | Lose | (ai - user + 3) % 3 = 1 | -1 |
139
- | Draw | (ai - user + 3) % 3 = 0 | 0 |
140
-
141
- ## Training Patterns
142
-
143
- Built-in patterns for the RPS environment:
144
-
145
- | Pattern | ID | Description |
146
- |---------|-----|-------------|
147
- | Random | 0 | Uniform random actions |
148
- | Always Rock | 1 | Always play action 0 |
149
- | Counter | 2 | Play counter to previous action |
150
- | Sequential | 3 | Cycle through 0,1,2,0,1,2... |
151
-
152
- Custom patterns can be implemented via `actionSelector` function.