@vernikr/size-report 2.5.0 → 2.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/src/tokens.js CHANGED
@@ -4,7 +4,7 @@ import { loadOptional } from './optional.js';
4
4
  /* The tokenizer follows the same discipline as the minifier: an optional dependency, loaded
5
5
  * lazily (how that works: `src/optional.js`). One difference: the tokenizer takes any text and
6
6
  * has nothing to refuse, so a missing dependency is not a refusal but a different count — an
7
- * estimate by length, marked as an approximation in the metric label.
7
+ * estimate by length, named as such in the metric label.
8
8
  *
9
9
  * The family is about models, the encoding about the number: the same file counts differently
10
10
  * under `cl100k_base` and `o200k_base`, which is why the encoding is chosen next to the family
@@ -21,8 +21,7 @@ export const TOKEN_DEFAULTS = { family: 'openai', encoding: 'o200k_base' };
21
21
  /* The estimate without a dictionary. The coefficient was taken from this repository's own
22
22
  * texts (Russian documents and code): `README.md` gave 3.1 characters per token, the archived
23
23
  * journal `worklog/archive/WORKLOG.md` about 3.0. For Latin script the same estimate overstates
24
- * the count (about 4 characters per token there), which is why it is marked as an
25
- * approximation. */
24
+ * the count (about 4 characters per token there), which is why the method says so in words. */
26
25
  export const CHARS_PER_TOKEN = 3;
27
26
 
28
27
  /* Formats for which counting tokens makes no sense: a picture, a font or an archive is bytes,
@@ -53,7 +52,7 @@ function familyOf(settings) {
53
52
  }
54
53
 
55
54
  /* Counting one text: with the dictionary if there is one, by estimate otherwise. Both answers
56
- * are a number of text units, and what tells them apart is the metric label (`accuracy`) rather
55
+ * are a number of text units, and what tells them apart is the metric label (`method`) rather
57
56
  * than the value, which is why one cannot be passed off as the other. */
58
57
  export function tokenCount(text, settings) {
59
58
  const { tool } = tokenizer(settings);
@@ -70,8 +69,8 @@ export function estimate(text) {
70
69
  }
71
70
 
72
71
  /* Whether the file is binary, since counting tokens means nothing for it. The list of formats
73
- * is owned here, so both the metric label and the cell mark ask about a file here instead of
74
- * keeping a list of their own. */
72
+ * is owned here, so the metric label asks about a file here instead of keeping a list of its
73
+ * own. */
75
74
  export function isBinary(file) {
76
75
  return BINARY_EXTS.indexOf(path.extname(file).toLowerCase()) >= 0;
77
76
  }
@@ -37,9 +37,9 @@ What is worth knowing about the template's values:
37
37
  - `metrics: ["raw", "min", "tok"]` — the report's three measurements. `min` is counted by the real minifier
38
38
  (`minify.engine: "esbuild"`) and `tok` by the `o200k_base` dictionary. Both travel as optional
39
39
  dependencies of the package and are installed by an ordinary installation; without them (an installation
40
- without the optional dependencies, a platform that has none) the tool still works, but says honestly that
41
- the numbers were counted another way, and returns **code 4** — that is a named approximation rather than
42
- an error of the settings.
40
+ without the optional dependencies, a platform that has none) the tool still works, but says that the
41
+ numbers were counted another way, and returns **code 4** — that is a named different count rather than an
42
+ error of the settings.
43
43
  - `fixCommand` — the command the report's signature and the refusals quote. In the template it is
44
44
  `node node_modules/@vernikr/size-report/bin/size.js --write` — the path to the installed package inside
45
45
  the project. **The package name as a command cannot stand here:** `npx <name>` in a project without the