logicstamp-context 0.2.1 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +35 -89
- package/dist/cli/commands/context/statsCalculator.js +1 -1
- package/dist/cli/commands/context/statsCalculator.js.map +1 -1
- package/dist/cli/commands/context/tokenEstimator.js +1 -1
- package/dist/cli/commands/context/tokenEstimator.js.map +1 -1
- package/dist/utils/tokens.d.ts +4 -1
- package/dist/utils/tokens.d.ts.map +1 -1
- package/dist/utils/tokens.js +4 -1
- package/dist/utils/tokens.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
<img src="assets/logicstamp-fox.svg" alt="LogicStamp Fox Mascot" width="120" height="120">
|
|
5
5
|
</div>
|
|
6
6
|
|
|
7
|
-
|
|
8
8
|
|
|
9
9
|
|
|
10
10
|
|
|
@@ -25,7 +25,7 @@ That's it! LogicStamp Context will scan your project and generate `context.json`
|
|
|
25
25
|
|
|
26
26
|
*Terminal output showing `stamp context` execution and generated context.json structure*
|
|
27
27
|
|
|
28
|
-
> **Note:** This is a beta release (v0.2.
|
|
28
|
+
> **Note:** This is a beta release (v0.2.2). We're actively improving the tool based on user feedback. If you encounter any issues or have suggestions, please [open an issue on GitHub](https://github.com/LogicStamp/logicstamp-context/issues).
|
|
29
29
|
|
|
30
30
|
## What is this?
|
|
31
31
|
|
|
@@ -45,6 +45,20 @@ After installation, the `stamp` command will be available globally.
|
|
|
45
45
|
- **Local install**: `npm install logicstamp-context` → only available in that project
|
|
46
46
|
- **Global install**: `npm install -g logicstamp-context` → available everywhere via `stamp` command
|
|
47
47
|
|
|
48
|
+
## What's New in v0.2.2
|
|
49
|
+
|
|
50
|
+
🔧 **Documentation Fixes**
|
|
51
|
+
- **Fixed optional dependencies documentation** - Corrected all documentation to accurately reflect that `@dqbd/tiktoken` and `@anthropic-ai/tokenizer` are included as optional dependencies in package.json and are automatically installed by npm when installing `logicstamp-context`
|
|
52
|
+
- **Updated user-facing messages** - Console output and source code comments now correctly explain optional dependency installation behavior
|
|
53
|
+
|
|
54
|
+
## What's New in v0.2.1
|
|
55
|
+
|
|
56
|
+
🔧 **Bug Fixes**
|
|
57
|
+
- **Dynamic version loading** - Fixed hardcoded version string in generated context files to dynamically load from `package.json`, ensuring version consistency across all generated context files
|
|
58
|
+
|
|
59
|
+
### Changed
|
|
60
|
+
- Updated all version references in documentation to reflect 0.2.1 release
|
|
61
|
+
|
|
48
62
|
## What's New in v0.2.0
|
|
49
63
|
|
|
50
64
|
🎨 **Style Metadata Extraction**
|
|
@@ -60,7 +74,7 @@ After installation, the `stamp` command will be available globally.
|
|
|
60
74
|
- **Four-mode comparison** - `--compare-modes` now shows `none`, `header`, `header+style`, and `full` modes
|
|
61
75
|
- **Dual comparison tables** - Shows savings vs raw source and vs full context for better decision-making
|
|
62
76
|
- **Accurate style impact** - Automatically regenerates contracts with/without style metadata for precise token counts
|
|
63
|
-
- **Optional tokenizer support** -
|
|
77
|
+
- **Optional tokenizer support** - Includes `@dqbd/tiktoken` (GPT-4) and `@anthropic-ai/tokenizer` (Claude) as optional dependencies. npm automatically attempts to install them when installing `logicstamp-context`. If installation succeeds, the tool uses them for accurate token counts. If installation fails or is skipped, gracefully falls back to character-based estimation
|
|
64
78
|
|
|
65
79
|
🏗️ **Architectural Improvements**
|
|
66
80
|
- **Modular CLI structure** - Refactored into dedicated handlers for better maintainability and testability
|
|
@@ -68,40 +82,35 @@ After installation, the `stamp` command will be available globally.
|
|
|
68
82
|
- **Modularized style extraction** - Organized style extraction into dedicated modules (tailwind, scss, motion, layout, etc.)
|
|
69
83
|
- **Improved code organization** - Better separation of concerns and easier contribution
|
|
70
84
|
|
|
71
|
-
**
|
|
85
|
+
**Optional Tokenizers for Accurate Token Counts:**
|
|
72
86
|
|
|
73
|
-
|
|
87
|
+
LogicStamp Context includes `@dqbd/tiktoken` and `@anthropic-ai/tokenizer` as optional dependencies. npm will automatically attempt to install them when you install `logicstamp-context`, but installation may be skipped if there are build issues (this is normal for optional dependencies).
|
|
74
88
|
|
|
75
|
-
**If
|
|
89
|
+
**If tokenizers are installed (automatic):**
|
|
90
|
+
- Token counts will be model-accurate for GPT-4 and Claude
|
|
91
|
+
- No additional setup required
|
|
76
92
|
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
93
|
+
**If tokenizers are not installed (installation failed or skipped):**
|
|
94
|
+
- LogicStamp Context automatically falls back to character-based estimation
|
|
95
|
+
- Estimates are typically within 10–15% of real token counts
|
|
96
|
+
- This is fine for most use cases
|
|
80
97
|
|
|
81
|
-
|
|
82
|
-
npm install @dqbd/tiktoken @anthropic-ai/tokenizer
|
|
98
|
+
**To manually install tokenizers (optional):**
|
|
83
99
|
|
|
84
|
-
|
|
85
|
-
npx stamp context
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
**If you installed LogicStamp Context globally:**
|
|
100
|
+
If the automatic installation failed and you want accurate token counts, you can install them manually:
|
|
89
101
|
|
|
90
102
|
```bash
|
|
91
|
-
#
|
|
92
|
-
npm install -
|
|
103
|
+
# For local installs
|
|
104
|
+
npm install -D @dqbd/tiktoken @anthropic-ai/tokenizer
|
|
93
105
|
|
|
94
|
-
#
|
|
106
|
+
# For global installs
|
|
95
107
|
npm install -g @dqbd/tiktoken @anthropic-ai/tokenizer
|
|
96
|
-
|
|
97
|
-
# Run from anywhere
|
|
98
|
-
stamp context
|
|
99
108
|
```
|
|
100
109
|
|
|
101
110
|
**Important:**
|
|
102
|
-
-
|
|
103
|
-
-
|
|
104
|
-
-
|
|
111
|
+
- Tokenizers are installed as **optional dependencies** of `logicstamp-context` and will be installed automatically in most cases
|
|
112
|
+
- If automatic installation fails, LogicStamp Context gracefully falls back to character-based estimation
|
|
113
|
+
- You do **not** need to manually install tokenizers unless you specifically want accurate token counts and the automatic installation failed
|
|
105
114
|
|
|
106
115
|
## What's New in v0.1.1
|
|
107
116
|
|
|
@@ -149,57 +158,6 @@ stamp context
|
|
|
149
158
|
- Enhanced `--stats` output with mode estimates
|
|
150
159
|
- JSON output optimized for CI parsing
|
|
151
160
|
|
|
152
|
-
## Quick Start
|
|
153
|
-
|
|
154
|
-
```bash
|
|
155
|
-
# Install globally
|
|
156
|
-
npm i -g logicstamp-context
|
|
157
|
-
|
|
158
|
-
# Optional: Initialize project (set up .gitignore and LLM_CONTEXT.md)
|
|
159
|
-
stamp init
|
|
160
|
-
|
|
161
|
-
# Generate context.json (llm-chat profile)
|
|
162
|
-
stamp context
|
|
163
|
-
|
|
164
|
-
# Generate context with style metadata (Tailwind, SCSS, animations, layout)
|
|
165
|
-
stamp context style
|
|
166
|
-
|
|
167
|
-
# Or use the flag (equivalent)
|
|
168
|
-
stamp context --include-style
|
|
169
|
-
|
|
170
|
-
# Preview stats without writing files
|
|
171
|
-
stamp context --dry-run --stats
|
|
172
|
-
|
|
173
|
-
# Compare token costs across modes
|
|
174
|
-
stamp context --compare-modes
|
|
175
|
-
|
|
176
|
-
# Generate minimal API documentation
|
|
177
|
-
stamp context --include-code none --format pretty --out docs/api.json
|
|
178
|
-
|
|
179
|
-
# Compare all context files for drift (multi-file mode)
|
|
180
|
-
stamp context compare
|
|
181
|
-
|
|
182
|
-
# Auto-approve and update all drifted files (like jest -u)
|
|
183
|
-
stamp context compare --approve
|
|
184
|
-
|
|
185
|
-
# Compare with per-folder token stats
|
|
186
|
-
stamp context compare --stats
|
|
187
|
-
|
|
188
|
-
# Compare two specific context files
|
|
189
|
-
stamp context compare old.json new.json --stats
|
|
190
|
-
|
|
191
|
-
# Validate generated context
|
|
192
|
-
stamp context validate context.json
|
|
193
|
-
|
|
194
|
-
# Clean all context artifacts (dry run)
|
|
195
|
-
stamp context clean
|
|
196
|
-
|
|
197
|
-
# Actually delete all context files
|
|
198
|
-
stamp context clean --all --yes
|
|
199
|
-
```
|
|
200
|
-
|
|
201
|
-
**Why?** Generate AI-ready context from your React/TS codebase in seconds with built-in token cost optimization.
|
|
202
|
-
|
|
203
161
|
## What does it generate?
|
|
204
162
|
|
|
205
163
|
LogicStamp Context analyzes your React components and outputs a structured JSON file containing:
|
|
@@ -451,17 +409,7 @@ Output:
|
|
|
451
409
|
|
|
452
410
|
**Note:** The `--compare-modes` flag automatically regenerates contracts with and without style metadata to provide accurate token counts for all modes. This ensures you see the true impact of including style information.
|
|
453
411
|
|
|
454
|
-
**Optional tokenizers for accurate counts:**
|
|
455
|
-
|
|
456
|
-
- `@dqbd/tiktoken` for GPT-4 token counts
|
|
457
|
-
- `@anthropic-ai/tokenizer` for Claude token counts
|
|
458
|
-
|
|
459
|
-
**Installation location:**
|
|
460
|
-
- **Local install** (recommended): Install tokenizers in your project root when using `npm install -D logicstamp-context`
|
|
461
|
-
- **Global install**: Install tokenizers globally when using `npm install -g logicstamp-context`
|
|
462
|
-
- **Do not** install tokenizers inside the `logicstamp-context` package folder
|
|
463
|
-
|
|
464
|
-
LogicStamp Context will automatically detect and use tokenizers if available, or fall back to character-based estimation (typically within 10-15% accuracy) if not installed.
|
|
412
|
+
**Optional tokenizers for accurate counts:** LogicStamp Context includes `@dqbd/tiktoken` (GPT-4) and `@anthropic-ai/tokenizer` (Claude) as optional dependencies. npm will automatically attempt to install them when you install `logicstamp-context`. If installation succeeds, you get model-accurate token counts. If installation fails or is skipped (normal for optional dependencies), LogicStamp Context gracefully falls back to character-based estimation (typically within 10-15% accuracy). No manual installation is required unless you specifically want accurate counts and the automatic installation failed.
|
|
465
413
|
|
|
466
414
|
### Stats for CI/CD
|
|
467
415
|
|
|
@@ -1064,5 +1012,3 @@ Issues and PRs welcome! This is an open-source project.
|
|
|
1064
1012
|
|
|
1065
1013
|
- [LogicStamp Main Project](https://github.com/LogicStamp/logicstamp)
|
|
1066
1014
|
- [Report Issues](https://github.com/LogicStamp/logicstamp-context/issues)
|
|
1067
|
-
|
|
1068
|
-
#TEST: CI Workflow
|
|
@@ -112,7 +112,7 @@ export async function generateSummary(contracts, manifest, bundles, stats, token
|
|
|
112
112
|
if (!tokenizerStatus.claude) {
|
|
113
113
|
missing.push('@anthropic-ai/tokenizer (Claude)');
|
|
114
114
|
}
|
|
115
|
-
console.log(` 💡 Tip:
|
|
115
|
+
console.log(` 💡 Tip: Tokenizers are included as optional dependencies. If installation failed, manually install ${missing.join(' and/or ')} for accurate token counts`);
|
|
116
116
|
}
|
|
117
117
|
console.log(` ⚠️ Current mode = tokenizer-based.`);
|
|
118
118
|
console.log(` Other modes / raw source = heuristic.`);
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"statsCalculator.js","sourceRoot":"","sources":["../../../../src/cli/commands/context/statsCalculator.ts"],"names":[],"mappings":"AAAA;;GAEG;AAMH,OAAO,EAAE,gBAAgB,EAAE,kBAAkB,EAAE,MAAM,0BAA0B,CAAC;AAQhF;;GAEG;AACH,MAAM,UAAU,cAAc,CAAC,OAA2B;IACxD,MAAM,UAAU,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAC7E,MAAM,UAAU,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAC7E,MAAM,YAAY,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAEhF,OAAO;QACL,UAAU;QACV,UAAU;QACV,YAAY;KACb,CAAC;AACJ,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,mBAAmB,CACjC,SAAwB,EACxB,QAAyB,EACzB,OAA2B,EAC3B,KAAkB,EAClB,cAA8B,EAC9B,OAAe;IAEf,OAAO;QACL,eAAe,EAAE,SAAS,CAAC,MAAM;QACjC,cAAc,EAAE,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM;QAC3C,cAAc,EAAE,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM;QAC5C,gBAAgB,EAAE,OAAO,CAAC,MAAM;QAChC,UAAU,EAAE,KAAK,CAAC,UAAU;QAC5B,UAAU,EAAE,KAAK,CAAC,UAAU;QAC5B,mBAAmB,EAAE,KAAK,CAAC,YAAY;QACvC,UAAU,EAAE,cAAc,CAAC,WAAW;QACtC,YAAY,EAAE,cAAc,CAAC,aAAa;QAC1C,aAAa,EAAE;YACb,IAAI,EAAE,EAAE,IAAI,EAAE,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,EAAE,MAAM,EAAE,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,MAAM,EAAE;YACxG,MAAM,EAAE,EAAE,IAAI,EAAE,cAAc,CAAC,aAAa,CAAC,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,cAAc,CAAC,aAAa,CAAC,MAAM,CAAC,MAAM,EAAE;YAC9G,IAAI,EAAE,EAAE,IAAI,EAAE,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,EAAE,MAAM,EAAE,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,MAAM,EAAE;SACzG;QACD,WAAW,EAAE,cAAc,CAAC,WAAW;QACvC,aAAa,EAAE,cAAc,CAAC,aAAa;QAC3C,SAAS,EAAE,OAAO;KACnB,CAAC;AACJ,CAAC;AAED;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,eAAe,CACnC,SAAwB,EACxB,QAAyB,EACzB,OAA2B,EAC3B,KAAkB,EAClB,cAA8B,EAC9B,OAkBC;IAED,OAAO,CAAC,GAAG,CAAC,eAAe,CAAC,CAAC;IAC7B,OAAO,CAAC,GAAG,CAAC,wBAAwB,SAAS,CAAC,MAAM,EAAE,CAAC,CAAC;IACxD,OAAO,CAAC,GAAG,CAAC,uBAAuB,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,EAAE,CAAC,CAAC;IAClE,OAAO,CAAC,GAAG,CAAC,uBAAuB,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC;IACnE,OAAO,CAAC,GAAG,CAAC,yBAAyB,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;IACvD,OAAO,CAAC,GAAG,CAAC,8BAA8B,KAAK,CAAC,UAAU,EAAE,CAAC,CAAC;IAC9D,OAAO,CAAC,GAAG,CAAC,mBAAmB,KAAK,CAAC,UAAU,EAAE,CAAC,CAAC;IACnD,OAAO,CAAC,GAAG,CAAC,4BAA4B,KAAK,CAAC,YAAY,EAAE,CAAC,CAAC;IAE9D,uBAAuB;IACvB,MAAM,QAAQ,GAAG,OAAO,CAAC,YAAY,KAAK,IAAI,CAAC;IAC/C,MAAM,YAAY,GAAG,OAAO,CAAC,WAAW,KAAK,QAAQ,CAAC;IAEtD,IAAI,SAAiB,CAAC;IACtB,IAAI,YAAY,IAAI,QAAQ,EAAE,CAAC;QAC7B,SAAS,GAAG,cAAc,CAAC;IAC7B,CAAC;SAAM,IAAI,YAAY,IAAI,CAAC,QAAQ,EAAE,CAAC;QACrC,SAAS,GAAG,QAAQ,CAAC;IACvB,CAAC;SAAM,CAAC;QACN,SAAS,GAAG,OAAO,CAAC,WAAW,KAAK,MAAM,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,OAAO,CAAC,WAAW,CAAC;IAClF,CAAC;IAED,kHAAkH;IAClH,IAAI,iBAAyB,CAAC;IAC9B,IAAI,mBAA2B,CAAC;IAChC,IAAI,mBAA2B,CAAC;IAChC,IAAI,qBAA6B,CAAC;IAElC,IAAI,YAAY,IAAI,QAAQ,EAAE,CAAC;QAC7B,gEAAgE;QAChE,iBAAiB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC,CAAC,4BAA4B;QACvF,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;QAC9D,mBAAmB,GAAG,OAAO,CAAC,WAAW,CAAC;QAC1C,qBAAqB,GAAG,OAAO,CAAC,aAAa,CAAC;IAChD,CAAC;SAAM,IAAI,YAAY,IAAI,CAAC,QAAQ,EAAE,CAAC;QACrC,gEAAgE;QAChE,iBAAiB,GAAG,OAAO,CAAC,WAAW,CAAC;QACxC,mBAAmB,GAAG,OAAO,CAAC,aAAa,CAAC;QAC5C,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC,CAAC,kBAAkB;QAC/E,qBAAqB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;IAClE,CAAC;SAAM,CAAC;QACN,sCAAsC;QACtC,iBAAiB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC;QAC1D,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;QAC9D,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC;QAC5D,qBAAqB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;IAClE,CAAC;IAED,8CAA8C;IAC9C,MAAM,iBAAiB,GAAG,cAAc,CAAC,gBAAgB,GAAG,CAAC;QAC3D,CAAC,CAAC,CAAC,CAAC,cAAc,CAAC,gBAAgB,GAAG,iBAAiB,CAAC,GAAG,cAAc,CAAC,gBAAgB,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAC5G,CAAC,CAAC,GAAG,CAAC;IACR,MAAM,sBAAsB,GAAG,cAAc,CAAC,gBAAgB,GAAG,CAAC;QAChE,CAAC,CAAC,CAAC,CAAC,cAAc,CAAC,gBAAgB,GAAG,mBAAmB,CAAC,GAAG,cAAc,CAAC,gBAAgB,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAC9G,CAAC,CAAC,GAAG,CAAC;IAER,wCAAwC;IACxC,MAAM,eAAe,GAAG,MAAM,kBAAkB,EAAE,CAAC;IACnD,MAAM,UAAU,GAAG,eAAe,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,eAAe,CAAC;IACvE,MAAM,YAAY,GAAG,eAAe,CAAC,MAAM,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,eAAe,CAAC;IAE5E,OAAO,CAAC,GAAG,CAAC,yBAAyB,SAAS,SAAS,CAAC,CAAC;IACzD,OAAO,CAAC,GAAG,CAAC,gCAAgC,UAAU,eAAe,YAAY,GAAG,CAAC,CAAC;IACtF,IAAI,CAAC,eAAe,CAAC,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,EAAE,CAAC;QACrD,MAAM,OAAO,GAAa,EAAE,CAAC;QAC7B,IAAI,CAAC,eAAe,CAAC,IAAI,EAAE,CAAC;YAC1B,OAAO,CAAC,IAAI,CAAC,wBAAwB,CAAC,CAAC;QACzC,CAAC;QACD,IAAI,CAAC,eAAe,CAAC,MAAM,EAAE,CAAC;YAC5B,OAAO,CAAC,IAAI,CAAC,kCAAkC,CAAC,CAAC;QACnD,CAAC;QACD,OAAO,CAAC,GAAG,CAAC,
|
|
1
|
+
{"version":3,"file":"statsCalculator.js","sourceRoot":"","sources":["../../../../src/cli/commands/context/statsCalculator.ts"],"names":[],"mappings":"AAAA;;GAEG;AAMH,OAAO,EAAE,gBAAgB,EAAE,kBAAkB,EAAE,MAAM,0BAA0B,CAAC;AAQhF;;GAEG;AACH,MAAM,UAAU,cAAc,CAAC,OAA2B;IACxD,MAAM,UAAU,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAC7E,MAAM,UAAU,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAC7E,MAAM,YAAY,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAEhF,OAAO;QACL,UAAU;QACV,UAAU;QACV,YAAY;KACb,CAAC;AACJ,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,mBAAmB,CACjC,SAAwB,EACxB,QAAyB,EACzB,OAA2B,EAC3B,KAAkB,EAClB,cAA8B,EAC9B,OAAe;IAEf,OAAO;QACL,eAAe,EAAE,SAAS,CAAC,MAAM;QACjC,cAAc,EAAE,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM;QAC3C,cAAc,EAAE,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM;QAC5C,gBAAgB,EAAE,OAAO,CAAC,MAAM;QAChC,UAAU,EAAE,KAAK,CAAC,UAAU;QAC5B,UAAU,EAAE,KAAK,CAAC,UAAU;QAC5B,mBAAmB,EAAE,KAAK,CAAC,YAAY;QACvC,UAAU,EAAE,cAAc,CAAC,WAAW;QACtC,YAAY,EAAE,cAAc,CAAC,aAAa;QAC1C,aAAa,EAAE;YACb,IAAI,EAAE,EAAE,IAAI,EAAE,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,EAAE,MAAM,EAAE,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,MAAM,EAAE;YACxG,MAAM,EAAE,EAAE,IAAI,EAAE,cAAc,CAAC,aAAa,CAAC,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,cAAc,CAAC,aAAa,CAAC,MAAM,CAAC,MAAM,EAAE;YAC9G,IAAI,EAAE,EAAE,IAAI,EAAE,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,EAAE,MAAM,EAAE,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,MAAM,EAAE;SACzG;QACD,WAAW,EAAE,cAAc,CAAC,WAAW;QACvC,aAAa,EAAE,cAAc,CAAC,aAAa;QAC3C,SAAS,EAAE,OAAO;KACnB,CAAC;AACJ,CAAC;AAED;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,eAAe,CACnC,SAAwB,EACxB,QAAyB,EACzB,OAA2B,EAC3B,KAAkB,EAClB,cAA8B,EAC9B,OAkBC;IAED,OAAO,CAAC,GAAG,CAAC,eAAe,CAAC,CAAC;IAC7B,OAAO,CAAC,GAAG,CAAC,wBAAwB,SAAS,CAAC,MAAM,EAAE,CAAC,CAAC;IACxD,OAAO,CAAC,GAAG,CAAC,uBAAuB,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,EAAE,CAAC,CAAC;IAClE,OAAO,CAAC,GAAG,CAAC,uBAAuB,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC;IACnE,OAAO,CAAC,GAAG,CAAC,yBAAyB,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;IACvD,OAAO,CAAC,GAAG,CAAC,8BAA8B,KAAK,CAAC,UAAU,EAAE,CAAC,CAAC;IAC9D,OAAO,CAAC,GAAG,CAAC,mBAAmB,KAAK,CAAC,UAAU,EAAE,CAAC,CAAC;IACnD,OAAO,CAAC,GAAG,CAAC,4BAA4B,KAAK,CAAC,YAAY,EAAE,CAAC,CAAC;IAE9D,uBAAuB;IACvB,MAAM,QAAQ,GAAG,OAAO,CAAC,YAAY,KAAK,IAAI,CAAC;IAC/C,MAAM,YAAY,GAAG,OAAO,CAAC,WAAW,KAAK,QAAQ,CAAC;IAEtD,IAAI,SAAiB,CAAC;IACtB,IAAI,YAAY,IAAI,QAAQ,EAAE,CAAC;QAC7B,SAAS,GAAG,cAAc,CAAC;IAC7B,CAAC;SAAM,IAAI,YAAY,IAAI,CAAC,QAAQ,EAAE,CAAC;QACrC,SAAS,GAAG,QAAQ,CAAC;IACvB,CAAC;SAAM,CAAC;QACN,SAAS,GAAG,OAAO,CAAC,WAAW,KAAK,MAAM,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,OAAO,CAAC,WAAW,CAAC;IAClF,CAAC;IAED,kHAAkH;IAClH,IAAI,iBAAyB,CAAC;IAC9B,IAAI,mBAA2B,CAAC;IAChC,IAAI,mBAA2B,CAAC;IAChC,IAAI,qBAA6B,CAAC;IAElC,IAAI,YAAY,IAAI,QAAQ,EAAE,CAAC;QAC7B,gEAAgE;QAChE,iBAAiB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC,CAAC,4BAA4B;QACvF,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;QAC9D,mBAAmB,GAAG,OAAO,CAAC,WAAW,CAAC;QAC1C,qBAAqB,GAAG,OAAO,CAAC,aAAa,CAAC;IAChD,CAAC;SAAM,IAAI,YAAY,IAAI,CAAC,QAAQ,EAAE,CAAC;QACrC,gEAAgE;QAChE,iBAAiB,GAAG,OAAO,CAAC,WAAW,CAAC;QACxC,mBAAmB,GAAG,OAAO,CAAC,aAAa,CAAC;QAC5C,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC,CAAC,kBAAkB;QAC/E,qBAAqB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;IAClE,CAAC;SAAM,CAAC;QACN,sCAAsC;QACtC,iBAAiB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC;QAC1D,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;QAC9D,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC;QAC5D,qBAAqB,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;IAClE,CAAC;IAED,8CAA8C;IAC9C,MAAM,iBAAiB,GAAG,cAAc,CAAC,gBAAgB,GAAG,CAAC;QAC3D,CAAC,CAAC,CAAC,CAAC,cAAc,CAAC,gBAAgB,GAAG,iBAAiB,CAAC,GAAG,cAAc,CAAC,gBAAgB,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAC5G,CAAC,CAAC,GAAG,CAAC;IACR,MAAM,sBAAsB,GAAG,cAAc,CAAC,gBAAgB,GAAG,CAAC;QAChE,CAAC,CAAC,CAAC,CAAC,cAAc,CAAC,gBAAgB,GAAG,mBAAmB,CAAC,GAAG,cAAc,CAAC,gBAAgB,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAC9G,CAAC,CAAC,GAAG,CAAC;IAER,wCAAwC;IACxC,MAAM,eAAe,GAAG,MAAM,kBAAkB,EAAE,CAAC;IACnD,MAAM,UAAU,GAAG,eAAe,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,eAAe,CAAC;IACvE,MAAM,YAAY,GAAG,eAAe,CAAC,MAAM,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,eAAe,CAAC;IAE5E,OAAO,CAAC,GAAG,CAAC,yBAAyB,SAAS,SAAS,CAAC,CAAC;IACzD,OAAO,CAAC,GAAG,CAAC,gCAAgC,UAAU,eAAe,YAAY,GAAG,CAAC,CAAC;IACtF,IAAI,CAAC,eAAe,CAAC,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,EAAE,CAAC;QACrD,MAAM,OAAO,GAAa,EAAE,CAAC;QAC7B,IAAI,CAAC,eAAe,CAAC,IAAI,EAAE,CAAC;YAC1B,OAAO,CAAC,IAAI,CAAC,wBAAwB,CAAC,CAAC;QACzC,CAAC;QACD,IAAI,CAAC,eAAe,CAAC,MAAM,EAAE,CAAC;YAC5B,OAAO,CAAC,IAAI,CAAC,kCAAkC,CAAC,CAAC;QACnD,CAAC;QACD,OAAO,CAAC,GAAG,CAAC,yGAAyG,OAAO,CAAC,IAAI,CAAC,UAAU,CAAC,4BAA4B,CAAC,CAAC;IAC7K,CAAC;IACD,OAAO,CAAC,GAAG,CAAC,wCAAwC,CAAC,CAAC;IACtD,OAAO,CAAC,GAAG,CAAC,6CAA6C,CAAC,CAAC;IAC3D,OAAO,CAAC,GAAG,CAAC,4EAA4E,CAAC,CAAC;IAC1F,OAAO,CAAC,GAAG,CAAC,mBAAmB,gBAAgB,CAAC,cAAc,CAAC,WAAW,CAAC,SAAS,CAAC,CAAC;IACtF,OAAO,CAAC,GAAG,CAAC,mBAAmB,gBAAgB,CAAC,cAAc,CAAC,aAAa,CAAC,SAAS,CAAC,CAAC;IACxF,OAAO,CAAC,GAAG,CAAC,kBAAkB,CAAC,CAAC;IAChC,OAAO,CAAC,GAAG,CAAC,2EAA2E,CAAC,CAAC;IACzF,OAAO,CAAC,GAAG,CAAC,6EAA6E,CAAC,CAAC;IAC3F,OAAO,CAAC,GAAG,CAAC,uBAAuB,gBAAgB,CAAC,cAAc,CAAC,gBAAgB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,gBAAgB,CAAC,cAAc,CAAC,kBAAkB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,OAAO,CAAC,CAAC;IAChL,OAAO,CAAC,GAAG,CAAC,uBAAuB,gBAAgB,CAAC,iBAAiB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,gBAAgB,CAAC,mBAAmB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,iBAAiB,GAAG,CAAC,CAAC;IACvK,OAAO,CAAC,GAAG,CAAC,uBAAuB,gBAAgB,CAAC,mBAAmB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,gBAAgB,CAAC,qBAAqB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,sBAAsB,GAAG,CAAC,CAAC;IAChL,OAAO,CAAC,GAAG,CAAC,oCAAoC,gBAAgB,CAAC,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,mBAAmB,gBAAgB,CAAC,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,CAAC;IAEhM,OAAO,CAAC,GAAG,CAAC,+BAA+B,CAAC,CAAC;IAC7C,OAAO,CAAC,GAAG,CAAC,wCAAwC,CAAC,CAAC;IACtD,OAAO,CAAC,GAAG,CAAC,6CAA6C,CAAC,CAAC;IAC3D,OAAO,CAAC,GAAG,CAAC,4EAA4E,CAAC,CAAC;IAC1F,OAAO,CAAC,GAAG,CAAC,mBAAmB,gBAAgB,CAAC,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAClG,yFAAyF;IACzF,MAAM,gBAAgB,GAAG,SAAS,CAAC;IACnC,OAAO,CAAC,GAAG,CAAC,MAAM,gBAAgB,CAAC,MAAM,CAAC,EAAE,CAAC,KAAK,gBAAgB,CAAC,cAAc,CAAC,WAAW,CAAC,SAAS,CAAC,CAAC;IACzG,OAAO,CAAC,GAAG,CAAC,mBAAmB,gBAAgB,CAAC,cAAc,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAElG,IAAI,KAAK,CAAC,YAAY,GAAG,CAAC,EAAE,CAAC;QAC3B,OAAO,CAAC,GAAG,CAAC,oDAAoD,CAAC,CAAC;QAClE,MAAM,UAAU,GAAG,IAAI,GAAG,EAAU,CAAC;QACrC,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE;YAClB,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,UAAU,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC;QAC1D,CAAC,CAAC,CAAC;QACH,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,OAAO,CAAC,GAAG,CAAC,QAAQ,IAAI,EAAE,CAAC,CAAC,CAAC;QACjF,IAAI,UAAU,CAAC,IAAI,GAAG,EAAE,EAAE,CAAC;YACzB,OAAO,CAAC,GAAG,CAAC,cAAc,UAAU,CAAC,IAAI,GAAG,EAAE,OAAO,CAAC,CAAC;QACzD,CAAC;IACH,CAAC;AACH,CAAC"}
|
|
@@ -230,7 +230,7 @@ export async function displayModeComparison(comparison, elapsed) {
|
|
|
230
230
|
if (!tokenizerStatus.claude) {
|
|
231
231
|
missing.push('@anthropic-ai/tokenizer (Claude)');
|
|
232
232
|
}
|
|
233
|
-
console.log(` 💡 Tip:
|
|
233
|
+
console.log(` 💡 Tip: Tokenizers are included as optional dependencies. If installation failed, manually install ${missing.join(' and/or ')} for accurate token counts`);
|
|
234
234
|
}
|
|
235
235
|
console.log('\n Comparison:');
|
|
236
236
|
console.log(' Mode | Tokens GPT-4o | Tokens Claude | Savings vs Raw Source');
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"tokenEstimator.js","sourceRoot":"","sources":["../../../../src/cli/commands/context/tokenEstimator.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,OAAO,EAAE,kBAAkB,EAAE,oBAAoB,EAAE,gBAAgB,EAAE,kBAAkB,EAAE,MAAM,0BAA0B,CAAC;AAK1H,OAAO,EAAE,IAAI,EAAE,MAAM,uBAAuB,CAAC;AAC7C,OAAO,EAAE,aAAa,EAAE,MAAM,kCAAkC,CAAC;AACjE,OAAO,EAAE,eAAe,EAAE,MAAM,4BAA4B,CAAC;AAC7D,OAAO,EAAE,oBAAoB,EAAE,MAAM,iCAAiC,CAAC;AACvE,OAAO,EAAE,gBAAgB,EAAE,MAAM,uBAAuB,CAAC;AACzD,OAAO,EAAE,OAAO,EAAE,MAAM,UAAU,CAAC;AACnC,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AA+BrD;;GAEG;AACH,MAAM,CAAC,KAAK,UAAU,uBAAuB,CAC3C,MAAc,EACd,eAAuB,EACvB,WAAmB,EACnB,aAAqB;IAErB,sCAAsC;IACtC,MAAM,gBAAgB,GAAG,IAAI,CAAC,IAAI,CAAC,eAAe,GAAG,CAAC,CAAC,CAAC;IACxD,MAAM,kBAAkB,GAAG,IAAI,CAAC,IAAI,CAAC,eAAe,GAAG,GAAG,CAAC,CAAC;IAE5D,yDAAyD;IACzD,sEAAsE;IACtE,sDAAsD;IACtD,6CAA6C;IAC7C,MAAM,aAAa,GAAG;QACpB,IAAI,EAAE;YACJ,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,WAAW,GAAG,GAAG,CAAC;YAClC,MAAM,EAAE,IAAI,CAAC,IAAI,CAAC,aAAa,GAAG,GAAG,CAAC;SACvC;QACD,MAAM,EAAE;YACN,IAAI,EAAE,WAAW;YACjB,MAAM,EAAE,aAAa;SACtB;QACD,IAAI,EAAE;YACJ,IAAI,EAAE,WAAW,GAAG,gBAAgB;YACpC,MAAM,EAAE,aAAa,GAAG,kBAAkB;SAC3C;KACF,CAAC;IAEF,wDAAwD;IACxD,MAAM,WAAW,GAAG,aAAa,CAAC,IAAI,CAAC,IAAI,GAAG,CAAC;QAC7C,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,GAAG,WAAW,CAAC,GAAG,aAAa,CAAC,IAAI,CAAC,IAAI,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QACtF,CAAC,CAAC,GAAG,CAAC;IACR,MAAM,aAAa,GAAG,aAAa,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC;QACjD,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,MAAM,GAAG,aAAa,CAAC,GAAG,aAAa,CAAC,IAAI,CAAC,MAAM,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAC5F,CAAC,CAAC,GAAG,CAAC;IAER,OAAO;QACL,WAAW;QACX,aAAa;QACb,gBAAgB;QAChB,kBAAkB;QAClB,aAAa;QACb,WAAW;QACX,aAAa;KACd,CAAC;AACJ,CAAC;AAED;;GAEG;AACH,MAAM,CAAC,KAAK,UAAU,sBAAsB,CAC1C,KAAe,EACf,QAAyB,EACzB,WAAmB,EACnB,WAAmB,EACnB,aAAqB,EACrB,eAAuB,EACvB,OAWC;IAED,MAAM,QAAQ,GAAG,OAAO,CAAC,YAAY,KAAK,IAAI,CAAC;IAC/C,MAAM,YAAY,GAAG,OAAO,CAAC,WAAW,KAAK,QAAQ,CAAC;IAEtD,sCAAsC;IACtC,MAAM,sBAAsB,GAAG,IAAI,CAAC,IAAI,CAAC,eAAe,GAAG,CAAC,CAAC,CAAC;IAC9D,MAAM,wBAAwB,GAAG,IAAI,CAAC,IAAI,CAAC,eAAe,GAAG,GAAG,CAAC,CAAC;IAElE,mDAAmD;IACnD,IAAI,iBAAyB,CAAC;IAC9B,IAAI,mBAA2B,CAAC;IAChC,IAAI,mBAA2B,CAAC;IAChC,IAAI,qBAA6B,CAAC;IAElC,IAAI,YAAY,IAAI,QAAQ,EAAE,CAAC;QAC7B,qFAAqF;QACrF,mBAAmB,GAAG,WAAW,CAAC;QAClC,qBAAqB,GAAG,aAAa,CAAC;QAEtC,8EAA8E;QAC9E,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC;YACnB,OAAO,CAAC,GAAG,CAAC,iEAAiE,CAAC,CAAC;QACjF,CAAC;QAED,MAAM,gBAAgB,GAAkB,EAAE,CAAC;QAC3C,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;YACzB,IAAI,CAAC;gBACH,MAAM,GAAG,GAAG,MAAM,eAAe,CAAC,IAAI,CAAC,CAAC;gBACxC,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,gBAAgB,CAAC,IAAI,CAAC,CAAC;gBAE9C,MAAM,MAAM,GAAG,aAAa,CAAC,IAAI,EAAE,GAAG,EAAE;oBACtC,MAAM,EAAE,MAAM;oBACd,UAAU,EAAE,IAAI;oBAChB,iBAAiB,EAAE,OAAO,CAAC,eAAe;oBAC1C,aAAa,EAAE,SAAS,EAAE,sBAAsB;iBACjD,CAAC,CAAC;gBAEH,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;oBACpB,gBAAgB,CAAC,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;gBACzC,CAAC;YACH,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,oCAAoC;YACtC,CAAC;QACH,CAAC;QAED,2CAA2C;QAC3C,MAAM,mBAAmB,GAAG,IAAI,GAAG,CAAC,gBAAgB,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;QAC/E,MAAM,cAAc,GAAG,MAAM,OAAO,CAAC,GAAG,CACtC,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAChC,IAAI,CAAC,MAAM,EAAE,QAAQ,EAAE;YACrB,KAAK,EAAE,OAAO,CAAC,KAAK;YACpB,WAAW,EAAE,OAAO,CAAC,WAAW;YAChC,QAAQ,EAAE,OAAO,CAAC,QAAQ;YAC1B,YAAY,EAAE,mBAAmB;YACjC,MAAM,EAAE,OAAO,CAAC,MAAM;YACtB,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,KAAK;YACnC,MAAM,EAAE,OAAO,CAAC,MAAM,IAAI,KAAK;YAC/B,YAAY,EAAE,OAAO,CAAC,YAAY,KAAK,KAAK;SAC7C,EAAE,WAAW,CAAC,CAChB,CACF,CAAC;QAEF,6CAA6C;QAC7C,MAAM,aAAa,GAAG,aAAa,CAAC,cAAc,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC;QAEpE,iBAAiB,GAAG,MAAM,kBAAkB,CAAC,aAAa,CAAC,CAAC;QAC5D,mBAAmB,GAAG,MAAM,oBAAoB,CAAC,aAAa,CAAC,CAAC;IAClE,CAAC;SAAM,IAAI,YAAY,IAAI,CAAC,QAAQ,EAAE,CAAC;QACrC,0FAA0F;QAC1F,iBAAiB,GAAG,WAAW,CAAC;QAChC,mBAAmB,GAAG,aAAa,CAAC;QAEpC,iFAAiF;QACjF,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC;YACnB,OAAO,CAAC,GAAG,CAAC,8DAA8D,CAAC,CAAC;QAC9E,CAAC;QAED,MAAM,YAAY,GAAG,IAAI,OAAO,CAAC;YAC/B,2BAA2B,EAAE,IAAI;YACjC,eAAe,EAAE;gBACf,GAAG,EAAE,CAAC;gBACN,MAAM,EAAE,EAAE;aACX;SACF,CAAC,CAAC;QAEH,MAAM,cAAc,GAAkB,EAAE,CAAC;QACzC,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;YACzB,IAAI,CAAC;gBACH,MAAM,GAAG,GAAG,MAAM,eAAe,CAAC,IAAI,CAAC,CAAC;gBACxC,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,gBAAgB,CAAC,IAAI,CAAC,CAAC;gBAE9C,IAAI,aAAa,CAAC;gBAClB,IAAI,CAAC;oBACH,MAAM,UAAU,GAAG,YAAY,CAAC,mBAAmB,CAAC,IAAI,CAAC,CAAC;oBAC1D,aAAa,GAAG,MAAM,oBAAoB,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC;gBAC/D,CAAC;gBAAC,OAAO,UAAU,EAAE,CAAC;oBACpB,+BAA+B;gBACjC,CAAC;gBAED,MAAM,MAAM,GAAG,aAAa,CAAC,IAAI,EAAE,GAAG,EAAE;oBACtC,MAAM,EAAE,MAAM;oBACd,UAAU,EAAE,IAAI;oBAChB,iBAAiB,EAAE,OAAO,CAAC,eAAe;oBAC1C,aAAa;iBACd,CAAC,CAAC;gBAEH,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;oBACpB,cAAc,CAAC,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;gBACvC,CAAC;YACH,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,oCAAoC;YACtC,CAAC;QACH,CAAC;QAED,gDAAgD;QAChD,MAAM,iBAAiB,GAAG,IAAI,GAAG,CAAC,cAAc,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;QAC3E,MAAM,YAAY,GAAG,MAAM,OAAO,CAAC,GAAG,CACpC,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAChC,IAAI,CAAC,MAAM,EAAE,QAAQ,EAAE;YACrB,KAAK,EAAE,OAAO,CAAC,KAAK;YACpB,WAAW,EAAE,OAAO,CAAC,WAAW;YAChC,QAAQ,EAAE,OAAO,CAAC,QAAQ;YAC1B,YAAY,EAAE,iBAAiB;YAC/B,MAAM,EAAE,OAAO,CAAC,MAAM;YACtB,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,KAAK;YACnC,MAAM,EAAE,OAAO,CAAC,MAAM,IAAI,KAAK;YAC/B,YAAY,EAAE,OAAO,CAAC,YAAY,KAAK,KAAK;SAC7C,EAAE,WAAW,CAAC,CAChB,CACF,CAAC;QAEF,0CAA0C;QAC1C,MAAM,WAAW,GAAG,aAAa,CAAC,YAAY,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC;QAEhE,mBAAmB,GAAG,MAAM,kBAAkB,CAAC,WAAW,CAAC,CAAC;QAC5D,qBAAqB,GAAG,MAAM,oBAAoB,CAAC,WAAW,CAAC,CAAC;IAClE,CAAC;SAAM,CAAC;QACN,gCAAgC;QAChC,iBAAiB,GAAG,IAAI,CAAC,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC;QAClD,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;QACtD,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC;QACpD,qBAAqB,GAAG,IAAI,CAAC,IAAI,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;IAC1D,CAAC;IAED,2BAA2B;IAC3B,MAAM,aAAa,GAAG;QACpB,IAAI,EAAE;YACJ,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,iBAAiB,GAAG,GAAG,CAAC;YACxC,MAAM,EAAE,IAAI,CAAC,IAAI,CAAC,mBAAmB,GAAG,GAAG,CAAC;SAC7C;QACD,MAAM,EAAE;YACN,IAAI,EAAE,iBAAiB;YACvB,MAAM,EAAE,mBAAmB;SAC5B;QACD,WAAW,EAAE;YACX,IAAI,EAAE,mBAAmB;YACzB,MAAM,EAAE,qBAAqB;SAC9B;QACD,IAAI,EAAE;YACJ,IAAI,EAAE,iBAAiB,GAAG,sBAAsB;YAChD,MAAM,EAAE,mBAAmB,GAAG,wBAAwB;SACvD;KACF,CAAC;IAEF,OAAO;QACL,iBAAiB;QACjB,mBAAmB;QACnB,mBAAmB;QACnB,qBAAqB;QACrB,gBAAgB,EAAE,sBAAsB;QACxC,kBAAkB,EAAE,wBAAwB;QAC5C,aAAa;KACd,CAAC;AACJ,CAAC;AAED;;GAEG;AACH,MAAM,CAAC,KAAK,UAAU,qBAAqB,CACzC,UAAgC,EAChC,OAAe;IAEf,MAAM,EAAE,iBAAiB,EAAE,mBAAmB,EAAE,mBAAmB,EAAE,qBAAqB,EAAE,gBAAgB,EAAE,kBAAkB,EAAE,aAAa,EAAE,GAAG,UAAU,CAAC;IAE/J,8CAA8C;IAC9C,MAAM,iBAAiB,GAAG,gBAAgB,GAAG,CAAC;QAC5C,CAAC,CAAC,CAAC,CAAC,gBAAgB,GAAG,iBAAiB,CAAC,GAAG,gBAAgB,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAC9E,CAAC,CAAC,GAAG,CAAC;IACR,MAAM,sBAAsB,GAAG,gBAAgB,GAAG,CAAC;QACjD,CAAC,CAAC,CAAC,CAAC,gBAAgB,GAAG,mBAAmB,CAAC,GAAG,gBAAgB,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAChF,CAAC,CAAC,GAAG,CAAC;IAER,yBAAyB;IACzB,MAAM,eAAe,GAAG,MAAM,kBAAkB,EAAE,CAAC;IACnD,MAAM,UAAU,GAAG,eAAe,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,eAAe,CAAC;IACvE,MAAM,YAAY,GAAG,eAAe,CAAC,MAAM,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,eAAe,CAAC;IAE5E,OAAO,CAAC,GAAG,CAAC,wBAAwB,CAAC,CAAC;IACtC,OAAO,CAAC,GAAG,CAAC,gCAAgC,UAAU,eAAe,YAAY,GAAG,CAAC,CAAC;IACtF,IAAI,CAAC,eAAe,CAAC,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,EAAE,CAAC;QACrD,MAAM,OAAO,GAAa,EAAE,CAAC;QAC7B,IAAI,CAAC,eAAe,CAAC,IAAI,EAAE,CAAC;YAC1B,OAAO,CAAC,IAAI,CAAC,wBAAwB,CAAC,CAAC;QACzC,CAAC;QACD,IAAI,CAAC,eAAe,CAAC,MAAM,EAAE,CAAC;YAC5B,OAAO,CAAC,IAAI,CAAC,kCAAkC,CAAC,CAAC;QACnD,CAAC;QACD,OAAO,CAAC,GAAG,CAAC,
|
|
1
|
+
{"version":3,"file":"tokenEstimator.js","sourceRoot":"","sources":["../../../../src/cli/commands/context/tokenEstimator.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,OAAO,EAAE,kBAAkB,EAAE,oBAAoB,EAAE,gBAAgB,EAAE,kBAAkB,EAAE,MAAM,0BAA0B,CAAC;AAK1H,OAAO,EAAE,IAAI,EAAE,MAAM,uBAAuB,CAAC;AAC7C,OAAO,EAAE,aAAa,EAAE,MAAM,kCAAkC,CAAC;AACjE,OAAO,EAAE,eAAe,EAAE,MAAM,4BAA4B,CAAC;AAC7D,OAAO,EAAE,oBAAoB,EAAE,MAAM,iCAAiC,CAAC;AACvE,OAAO,EAAE,gBAAgB,EAAE,MAAM,uBAAuB,CAAC;AACzD,OAAO,EAAE,OAAO,EAAE,MAAM,UAAU,CAAC;AACnC,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AA+BrD;;GAEG;AACH,MAAM,CAAC,KAAK,UAAU,uBAAuB,CAC3C,MAAc,EACd,eAAuB,EACvB,WAAmB,EACnB,aAAqB;IAErB,sCAAsC;IACtC,MAAM,gBAAgB,GAAG,IAAI,CAAC,IAAI,CAAC,eAAe,GAAG,CAAC,CAAC,CAAC;IACxD,MAAM,kBAAkB,GAAG,IAAI,CAAC,IAAI,CAAC,eAAe,GAAG,GAAG,CAAC,CAAC;IAE5D,yDAAyD;IACzD,sEAAsE;IACtE,sDAAsD;IACtD,6CAA6C;IAC7C,MAAM,aAAa,GAAG;QACpB,IAAI,EAAE;YACJ,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,WAAW,GAAG,GAAG,CAAC;YAClC,MAAM,EAAE,IAAI,CAAC,IAAI,CAAC,aAAa,GAAG,GAAG,CAAC;SACvC;QACD,MAAM,EAAE;YACN,IAAI,EAAE,WAAW;YACjB,MAAM,EAAE,aAAa;SACtB;QACD,IAAI,EAAE;YACJ,IAAI,EAAE,WAAW,GAAG,gBAAgB;YACpC,MAAM,EAAE,aAAa,GAAG,kBAAkB;SAC3C;KACF,CAAC;IAEF,wDAAwD;IACxD,MAAM,WAAW,GAAG,aAAa,CAAC,IAAI,CAAC,IAAI,GAAG,CAAC;QAC7C,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,GAAG,WAAW,CAAC,GAAG,aAAa,CAAC,IAAI,CAAC,IAAI,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QACtF,CAAC,CAAC,GAAG,CAAC;IACR,MAAM,aAAa,GAAG,aAAa,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC;QACjD,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,MAAM,GAAG,aAAa,CAAC,GAAG,aAAa,CAAC,IAAI,CAAC,MAAM,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAC5F,CAAC,CAAC,GAAG,CAAC;IAER,OAAO;QACL,WAAW;QACX,aAAa;QACb,gBAAgB;QAChB,kBAAkB;QAClB,aAAa;QACb,WAAW;QACX,aAAa;KACd,CAAC;AACJ,CAAC;AAED;;GAEG;AACH,MAAM,CAAC,KAAK,UAAU,sBAAsB,CAC1C,KAAe,EACf,QAAyB,EACzB,WAAmB,EACnB,WAAmB,EACnB,aAAqB,EACrB,eAAuB,EACvB,OAWC;IAED,MAAM,QAAQ,GAAG,OAAO,CAAC,YAAY,KAAK,IAAI,CAAC;IAC/C,MAAM,YAAY,GAAG,OAAO,CAAC,WAAW,KAAK,QAAQ,CAAC;IAEtD,sCAAsC;IACtC,MAAM,sBAAsB,GAAG,IAAI,CAAC,IAAI,CAAC,eAAe,GAAG,CAAC,CAAC,CAAC;IAC9D,MAAM,wBAAwB,GAAG,IAAI,CAAC,IAAI,CAAC,eAAe,GAAG,GAAG,CAAC,CAAC;IAElE,mDAAmD;IACnD,IAAI,iBAAyB,CAAC;IAC9B,IAAI,mBAA2B,CAAC;IAChC,IAAI,mBAA2B,CAAC;IAChC,IAAI,qBAA6B,CAAC;IAElC,IAAI,YAAY,IAAI,QAAQ,EAAE,CAAC;QAC7B,qFAAqF;QACrF,mBAAmB,GAAG,WAAW,CAAC;QAClC,qBAAqB,GAAG,aAAa,CAAC;QAEtC,8EAA8E;QAC9E,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC;YACnB,OAAO,CAAC,GAAG,CAAC,iEAAiE,CAAC,CAAC;QACjF,CAAC;QAED,MAAM,gBAAgB,GAAkB,EAAE,CAAC;QAC3C,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;YACzB,IAAI,CAAC;gBACH,MAAM,GAAG,GAAG,MAAM,eAAe,CAAC,IAAI,CAAC,CAAC;gBACxC,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,gBAAgB,CAAC,IAAI,CAAC,CAAC;gBAE9C,MAAM,MAAM,GAAG,aAAa,CAAC,IAAI,EAAE,GAAG,EAAE;oBACtC,MAAM,EAAE,MAAM;oBACd,UAAU,EAAE,IAAI;oBAChB,iBAAiB,EAAE,OAAO,CAAC,eAAe;oBAC1C,aAAa,EAAE,SAAS,EAAE,sBAAsB;iBACjD,CAAC,CAAC;gBAEH,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;oBACpB,gBAAgB,CAAC,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;gBACzC,CAAC;YACH,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,oCAAoC;YACtC,CAAC;QACH,CAAC;QAED,2CAA2C;QAC3C,MAAM,mBAAmB,GAAG,IAAI,GAAG,CAAC,gBAAgB,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;QAC/E,MAAM,cAAc,GAAG,MAAM,OAAO,CAAC,GAAG,CACtC,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAChC,IAAI,CAAC,MAAM,EAAE,QAAQ,EAAE;YACrB,KAAK,EAAE,OAAO,CAAC,KAAK;YACpB,WAAW,EAAE,OAAO,CAAC,WAAW;YAChC,QAAQ,EAAE,OAAO,CAAC,QAAQ;YAC1B,YAAY,EAAE,mBAAmB;YACjC,MAAM,EAAE,OAAO,CAAC,MAAM;YACtB,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,KAAK;YACnC,MAAM,EAAE,OAAO,CAAC,MAAM,IAAI,KAAK;YAC/B,YAAY,EAAE,OAAO,CAAC,YAAY,KAAK,KAAK;SAC7C,EAAE,WAAW,CAAC,CAChB,CACF,CAAC;QAEF,6CAA6C;QAC7C,MAAM,aAAa,GAAG,aAAa,CAAC,cAAc,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC;QAEpE,iBAAiB,GAAG,MAAM,kBAAkB,CAAC,aAAa,CAAC,CAAC;QAC5D,mBAAmB,GAAG,MAAM,oBAAoB,CAAC,aAAa,CAAC,CAAC;IAClE,CAAC;SAAM,IAAI,YAAY,IAAI,CAAC,QAAQ,EAAE,CAAC;QACrC,0FAA0F;QAC1F,iBAAiB,GAAG,WAAW,CAAC;QAChC,mBAAmB,GAAG,aAAa,CAAC;QAEpC,iFAAiF;QACjF,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC;YACnB,OAAO,CAAC,GAAG,CAAC,8DAA8D,CAAC,CAAC;QAC9E,CAAC;QAED,MAAM,YAAY,GAAG,IAAI,OAAO,CAAC;YAC/B,2BAA2B,EAAE,IAAI;YACjC,eAAe,EAAE;gBACf,GAAG,EAAE,CAAC;gBACN,MAAM,EAAE,EAAE;aACX;SACF,CAAC,CAAC;QAEH,MAAM,cAAc,GAAkB,EAAE,CAAC;QACzC,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;YACzB,IAAI,CAAC;gBACH,MAAM,GAAG,GAAG,MAAM,eAAe,CAAC,IAAI,CAAC,CAAC;gBACxC,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,gBAAgB,CAAC,IAAI,CAAC,CAAC;gBAE9C,IAAI,aAAa,CAAC;gBAClB,IAAI,CAAC;oBACH,MAAM,UAAU,GAAG,YAAY,CAAC,mBAAmB,CAAC,IAAI,CAAC,CAAC;oBAC1D,aAAa,GAAG,MAAM,oBAAoB,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC;gBAC/D,CAAC;gBAAC,OAAO,UAAU,EAAE,CAAC;oBACpB,+BAA+B;gBACjC,CAAC;gBAED,MAAM,MAAM,GAAG,aAAa,CAAC,IAAI,EAAE,GAAG,EAAE;oBACtC,MAAM,EAAE,MAAM;oBACd,UAAU,EAAE,IAAI;oBAChB,iBAAiB,EAAE,OAAO,CAAC,eAAe;oBAC1C,aAAa;iBACd,CAAC,CAAC;gBAEH,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;oBACpB,cAAc,CAAC,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;gBACvC,CAAC;YACH,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,oCAAoC;YACtC,CAAC;QACH,CAAC;QAED,gDAAgD;QAChD,MAAM,iBAAiB,GAAG,IAAI,GAAG,CAAC,cAAc,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;QAC3E,MAAM,YAAY,GAAG,MAAM,OAAO,CAAC,GAAG,CACpC,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAChC,IAAI,CAAC,MAAM,EAAE,QAAQ,EAAE;YACrB,KAAK,EAAE,OAAO,CAAC,KAAK;YACpB,WAAW,EAAE,OAAO,CAAC,WAAW;YAChC,QAAQ,EAAE,OAAO,CAAC,QAAQ;YAC1B,YAAY,EAAE,iBAAiB;YAC/B,MAAM,EAAE,OAAO,CAAC,MAAM;YACtB,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,KAAK;YACnC,MAAM,EAAE,OAAO,CAAC,MAAM,IAAI,KAAK;YAC/B,YAAY,EAAE,OAAO,CAAC,YAAY,KAAK,KAAK;SAC7C,EAAE,WAAW,CAAC,CAChB,CACF,CAAC;QAEF,0CAA0C;QAC1C,MAAM,WAAW,GAAG,aAAa,CAAC,YAAY,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC;QAEhE,mBAAmB,GAAG,MAAM,kBAAkB,CAAC,WAAW,CAAC,CAAC;QAC5D,qBAAqB,GAAG,MAAM,oBAAoB,CAAC,WAAW,CAAC,CAAC;IAClE,CAAC;SAAM,CAAC;QACN,gCAAgC;QAChC,iBAAiB,GAAG,IAAI,CAAC,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC;QAClD,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;QACtD,mBAAmB,GAAG,IAAI,CAAC,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC,CAAC;QACpD,qBAAqB,GAAG,IAAI,CAAC,IAAI,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;IAC1D,CAAC;IAED,2BAA2B;IAC3B,MAAM,aAAa,GAAG;QACpB,IAAI,EAAE;YACJ,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,iBAAiB,GAAG,GAAG,CAAC;YACxC,MAAM,EAAE,IAAI,CAAC,IAAI,CAAC,mBAAmB,GAAG,GAAG,CAAC;SAC7C;QACD,MAAM,EAAE;YACN,IAAI,EAAE,iBAAiB;YACvB,MAAM,EAAE,mBAAmB;SAC5B;QACD,WAAW,EAAE;YACX,IAAI,EAAE,mBAAmB;YACzB,MAAM,EAAE,qBAAqB;SAC9B;QACD,IAAI,EAAE;YACJ,IAAI,EAAE,iBAAiB,GAAG,sBAAsB;YAChD,MAAM,EAAE,mBAAmB,GAAG,wBAAwB;SACvD;KACF,CAAC;IAEF,OAAO;QACL,iBAAiB;QACjB,mBAAmB;QACnB,mBAAmB;QACnB,qBAAqB;QACrB,gBAAgB,EAAE,sBAAsB;QACxC,kBAAkB,EAAE,wBAAwB;QAC5C,aAAa;KACd,CAAC;AACJ,CAAC;AAED;;GAEG;AACH,MAAM,CAAC,KAAK,UAAU,qBAAqB,CACzC,UAAgC,EAChC,OAAe;IAEf,MAAM,EAAE,iBAAiB,EAAE,mBAAmB,EAAE,mBAAmB,EAAE,qBAAqB,EAAE,gBAAgB,EAAE,kBAAkB,EAAE,aAAa,EAAE,GAAG,UAAU,CAAC;IAE/J,8CAA8C;IAC9C,MAAM,iBAAiB,GAAG,gBAAgB,GAAG,CAAC;QAC5C,CAAC,CAAC,CAAC,CAAC,gBAAgB,GAAG,iBAAiB,CAAC,GAAG,gBAAgB,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAC9E,CAAC,CAAC,GAAG,CAAC;IACR,MAAM,sBAAsB,GAAG,gBAAgB,GAAG,CAAC;QACjD,CAAC,CAAC,CAAC,CAAC,gBAAgB,GAAG,mBAAmB,CAAC,GAAG,gBAAgB,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;QAChF,CAAC,CAAC,GAAG,CAAC;IAER,yBAAyB;IACzB,MAAM,eAAe,GAAG,MAAM,kBAAkB,EAAE,CAAC;IACnD,MAAM,UAAU,GAAG,eAAe,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,eAAe,CAAC;IACvE,MAAM,YAAY,GAAG,eAAe,CAAC,MAAM,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,eAAe,CAAC;IAE5E,OAAO,CAAC,GAAG,CAAC,wBAAwB,CAAC,CAAC;IACtC,OAAO,CAAC,GAAG,CAAC,gCAAgC,UAAU,eAAe,YAAY,GAAG,CAAC,CAAC;IACtF,IAAI,CAAC,eAAe,CAAC,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,EAAE,CAAC;QACrD,MAAM,OAAO,GAAa,EAAE,CAAC;QAC7B,IAAI,CAAC,eAAe,CAAC,IAAI,EAAE,CAAC;YAC1B,OAAO,CAAC,IAAI,CAAC,wBAAwB,CAAC,CAAC;QACzC,CAAC;QACD,IAAI,CAAC,eAAe,CAAC,MAAM,EAAE,CAAC;YAC5B,OAAO,CAAC,IAAI,CAAC,kCAAkC,CAAC,CAAC;QACnD,CAAC;QACD,OAAO,CAAC,GAAG,CAAC,yGAAyG,OAAO,CAAC,IAAI,CAAC,UAAU,CAAC,4BAA4B,CAAC,CAAC;IAC7K,CAAC;IACD,OAAO,CAAC,GAAG,CAAC,kBAAkB,CAAC,CAAC;IAChC,OAAO,CAAC,GAAG,CAAC,2EAA2E,CAAC,CAAC;IACzF,OAAO,CAAC,GAAG,CAAC,6EAA6E,CAAC,CAAC;IAC3F,OAAO,CAAC,GAAG,CAAC,uBAAuB,gBAAgB,CAAC,gBAAgB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,gBAAgB,CAAC,kBAAkB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,OAAO,CAAC,CAAC;IAClJ,OAAO,CAAC,GAAG,CAAC,uBAAuB,gBAAgB,CAAC,iBAAiB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,gBAAgB,CAAC,mBAAmB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,iBAAiB,GAAG,CAAC,CAAC;IACvK,OAAO,CAAC,GAAG,CAAC,uBAAuB,gBAAgB,CAAC,mBAAmB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,gBAAgB,CAAC,qBAAqB,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,sBAAsB,GAAG,CAAC,CAAC;IAChL,OAAO,CAAC,GAAG,CAAC,sBAAsB,CAAC,CAAC;IACpC,OAAO,CAAC,GAAG,CAAC,6EAA6E,CAAC,CAAC;IAC3F,OAAO,CAAC,GAAG,CAAC,+EAA+E,CAAC,CAAC;IAE7F,MAAM,KAAK,GAA0D;QACnE,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,aAAa,CAAC,IAAI,CAAC,IAAI,EAAE,MAAM,EAAE,aAAa,CAAC,IAAI,CAAC,MAAM,EAAE;QAClF,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,iBAAiB,EAAE,MAAM,EAAE,mBAAmB,EAAE;QACxE,EAAE,IAAI,EAAE,cAAc,EAAE,IAAI,EAAE,mBAAmB,EAAE,MAAM,EAAE,qBAAqB,EAAE;QAClF,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,aAAa,CAAC,IAAI,CAAC,IAAI,EAAE,MAAM,EAAE,aAAa,CAAC,IAAI,CAAC,MAAM,EAAE;KACnF,CAAC;IAEF,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE;QACnB,MAAM,OAAO,GAAG,aAAa,CAAC,IAAI,CAAC,IAAI,GAAG,CAAC;YACzC,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,GAAG,aAAa,CAAC,IAAI,CAAC,IAAI,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;YACpF,CAAC,CAAC,GAAG,CAAC;QAER,OAAO,CAAC,GAAG,CACT,QAAQ,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,MAAM,gBAAgB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,gBAAgB,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,OAAO,GAAG,CAC3I,CAAC;IACJ,CAAC,CAAC,CAAC;IAEH,OAAO,CAAC,GAAG,CAAC,qBAAqB,OAAO,IAAI,CAAC,CAAC;AAChD,CAAC"}
|
package/dist/utils/tokens.d.ts
CHANGED
|
@@ -2,7 +2,10 @@
|
|
|
2
2
|
* Token estimation utilities for GPT and Claude models
|
|
3
3
|
* Uses character-based approximations by default
|
|
4
4
|
*
|
|
5
|
-
* For accurate token counts
|
|
5
|
+
* For accurate token counts: These are included as optional dependencies in package.json.
|
|
6
|
+
* npm automatically attempts to install them when installing logicstamp-context.
|
|
7
|
+
* If installation succeeds, accurate token counts are used.
|
|
8
|
+
* If installation fails or is skipped, falls back to character-based estimation.
|
|
6
9
|
* - @dqbd/tiktoken for GPT models (gpt-4o encoding)
|
|
7
10
|
* - @anthropic-ai/tokenizer for Claude (claude-3-5-sonnet-20241022 encoding)
|
|
8
11
|
*/
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"tokens.d.ts","sourceRoot":"","sources":["../../src/utils/tokens.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"tokens.d.ts","sourceRoot":"","sources":["../../src/utils/tokens.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AA6DH;;;GAGG;AACH,wBAAsB,kBAAkB,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CActE;AAED;;;GAGG;AACH,wBAAsB,oBAAoB,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAiBxE;AAED;;GAEG;AACH,wBAAgB,gBAAgB,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,CAEtD;AAED;;;GAGG;AACH,wBAAsB,kBAAkB,IAAI,OAAO,CAAC;IAAE,IAAI,EAAE,OAAO,CAAC;IAAC,MAAM,EAAE,OAAO,CAAA;CAAE,CAAC,CAItF"}
|
package/dist/utils/tokens.js
CHANGED
|
@@ -2,7 +2,10 @@
|
|
|
2
2
|
* Token estimation utilities for GPT and Claude models
|
|
3
3
|
* Uses character-based approximations by default
|
|
4
4
|
*
|
|
5
|
-
* For accurate token counts
|
|
5
|
+
* For accurate token counts: These are included as optional dependencies in package.json.
|
|
6
|
+
* npm automatically attempts to install them when installing logicstamp-context.
|
|
7
|
+
* If installation succeeds, accurate token counts are used.
|
|
8
|
+
* If installation fails or is skipped, falls back to character-based estimation.
|
|
6
9
|
* - @dqbd/tiktoken for GPT models (gpt-4o encoding)
|
|
7
10
|
* - @anthropic-ai/tokenizer for Claude (claude-3-5-sonnet-20241022 encoding)
|
|
8
11
|
*/
|
package/dist/utils/tokens.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"tokens.js","sourceRoot":"","sources":["../../src/utils/tokens.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"tokens.js","sourceRoot":"","sources":["../../src/utils/tokens.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AAEH,oDAAoD;AACpD,IAAI,eAAe,GAAQ,IAAI,CAAC;AAChC,IAAI,kBAAkB,GAAQ,IAAI,CAAC;AACnC,IAAI,cAAc,GAAG,KAAK,CAAC;AAC3B,IAAI,eAAe,GAAG,KAAK,CAAC;AAE5B;;GAEG;AACH,KAAK,UAAU,YAAY;IACzB,IAAI,cAAc,EAAE,CAAC;QACnB,OAAO,eAAe,KAAK,IAAI,CAAC;IAClC,CAAC;IACD,cAAc,GAAG,IAAI,CAAC;IAEtB,IAAI,CAAC;QACH,MAAM,QAAQ,GAAG,MAAM,MAAM,CAAC,gBAAgB,CAAC,CAAC;QAChD,+CAA+C;QAC/C,IAAI,OAAO,QAAQ,CAAC,kBAAkB,KAAK,UAAU,EAAE,CAAC;YACtD,eAAe,GAAG,QAAQ,CAAC,kBAAkB,CAAC,QAAQ,CAAC,CAAC;YACxD,OAAO,IAAI,CAAC;QACd,CAAC;QACD,wCAAwC;QACxC,IAAI,OAAO,QAAQ,CAAC,YAAY,KAAK,UAAU,EAAE,CAAC;YAChD,eAAe,GAAG,QAAQ,CAAC,YAAY,CAAC,aAAa,CAAC,CAAC;YACvD,OAAO,IAAI,CAAC;QACd,CAAC;QACD,iDAAiD;QACjD,OAAO,KAAK,CAAC;IACf,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,wCAAwC;QACxC,OAAO,KAAK,CAAC;IACf,CAAC;AACH,CAAC;AAED;;GAEG;AACH,KAAK,UAAU,sBAAsB;IACnC,IAAI,eAAe,EAAE,CAAC;QACpB,OAAO,kBAAkB,KAAK,IAAI,CAAC;IACrC,CAAC;IACD,eAAe,GAAG,IAAI,CAAC;IAEvB,IAAI,CAAC;QACH,MAAM,SAAS,GAAG,MAAM,MAAM,CAAC,yBAAyB,CAAC,CAAC;QAC1D,gEAAgE;QAChE,IAAI,OAAO,SAAS,CAAC,WAAW,KAAK,UAAU,EAAE,CAAC;YAChD,kBAAkB,GAAG,SAAS,CAAC;YAC/B,OAAO,IAAI,CAAC;QACd,CAAC;QACD,qDAAqD;QACrD,OAAO,KAAK,CAAC;IACf,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,yCAAyC;QACzC,OAAO,KAAK,CAAC;IACf,CAAC;AACH,CAAC;AAED;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,kBAAkB,CAAC,IAAY;IACnD,MAAM,WAAW,GAAG,MAAM,YAAY,EAAE,CAAC;IAEzC,IAAI,WAAW,IAAI,eAAe,EAAE,CAAC;QACnC,IAAI,CAAC;YACH,OAAO,eAAe,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC;QAC7C,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,6CAA6C;QAC/C,CAAC;IACH,CAAC;IAED,0CAA0C;IAC1C,6DAA6D;IAC7D,OAAO,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;AACpC,CAAC;AAED;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,oBAAoB,CAAC,IAAY;IACrD,MAAM,YAAY,GAAG,MAAM,sBAAsB,EAAE,CAAC;IAEpD,IAAI,YAAY,IAAI,kBAAkB,EAAE,CAAC;QACvC,IAAI,CAAC;YACH,gEAAgE;YAChE,IAAI,OAAO,kBAAkB,CAAC,WAAW,KAAK,UAAU,EAAE,CAAC;gBACzD,OAAO,kBAAkB,CAAC,WAAW,CAAC,IAAI,CAAC,CAAC;YAC9C,CAAC;QACH,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,6CAA6C;QAC/C,CAAC;IACH,CAAC;IAED,0CAA0C;IAC1C,gEAAgE;IAChE,OAAO,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,GAAG,CAAC,CAAC;AACtC,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,gBAAgB,CAAC,KAAa;IAC5C,OAAO,KAAK,CAAC,cAAc,EAAE,CAAC;AAChC,CAAC;AAED;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,kBAAkB;IACtC,MAAM,IAAI,GAAG,MAAM,YAAY,EAAE,CAAC;IAClC,MAAM,MAAM,GAAG,MAAM,sBAAsB,EAAE,CAAC;IAC9C,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC;AAC1B,CAAC"}
|