@dev-pi2pie/word-counter 0.1.5-canary.3 → 0.1.5-canary.5

package/README.md

@@ -4,7 +4,7 @@ Locale-aware word counting powered by the Web API [`Intl.Segmenter`](https://dev
 
 ## Quick Start (npx)
 
-Runtime requirement: Node.js `>=20`.
+Runtime requirement: Node.js `>=22.18.0`.
 
 Run without installing:
 
@@ -101,18 +101,145 @@ Enable the optional WASM detector for ambiguous Latin and Han routes:
 ```bash
 word-counter --detector wasm "This sentence should clearly be detected as English for the wasm detector path."
 word-counter --detector wasm "漢字測試需要更多內容才能觸發偵測"
+word-counter --detector wasm --content-gate strict "Internationalization documentation remains understandable."
+word-counter --detector wasm --content-gate loose "四字成語"
+word-counter --detector wasm --content-gate off "mode: debug\ntee: true\npath: logs\nUse this for testing."
+```
+
+Inspect detector behavior without count output:
+
+```bash
+word-counter inspect "こんにちは、世界！これはテストです。"
+word-counter inspect --detector wasm --view engine "This sentence should clearly be detected as English for the wasm detector path."
+word-counter inspect --detector regex -f json "こんにちは、世界！これはテストです。"
+word-counter inspect --detector regex -f json --pretty "こんにちは、世界！これはテストです。"
+word-counter inspect --detector wasm --content-gate off "mode: debug\ntee: true\npath: logs\nUse this for testing."
+word-counter inspect -p ./examples/yaml-basic.md
+word-counter inspect -p ./examples/test-case-multi-files-support
+word-counter inspect -p ./examples/test-case-multi-files-support --section content -f json --pretty
 ```
 
 Detector mode notes:
 
 - `--detector regex` is the default behavior.
 - `--detector wasm` only runs for ambiguous `und-Latn` and `und-Hani` chunks.
+- `--content-gate default|strict|loose|off` configures the shared detector policy mode used by the WASM detector path.
+  - `default`: current fixture-backed project policy
+  - `strict`: raises detector eligibility thresholds and makes more borderline windows fall back
+  - `loose`: lowers detector eligibility thresholds and makes more borderline windows eligible or upgradable
+  - `off`: bypasses `contentGate` evaluation only
+- mode behavior differs by route:
+  - `und-Latn`: `default|strict|loose` affect both eligibility and the Latin prose-style `contentGate`
+  - `und-Hani`: `default|strict|loose` affect eligibility only, while `contentGate` still reports `policy=none`
+- current Hani behavior:
+  - `default`: keeps the current Hani diagnostic-sample threshold
+  - `strict`: raises the Hani diagnostic-sample threshold
+  - `loose`: uses a short-window Han-focused threshold so idiom-length samples such as `四字成語` can become eligible
+  - `off`: keeps the same Hani eligibility thresholds as `default`
 - `--detector regex` keeps the original script/regex chunk-first detection path.
 - `--detector wasm` uses a detector-oriented ambiguous-window scoring pass before accepted tags are projected back onto the counting chunks.
 - In `--detector wasm` mode, Latin hint rules and explicit Latin hint flags are deferred until after detector evaluation and only relabel unresolved `und-Latn` output.
 - Very short chunks stay on the original `und-*` fallback.
 - Low-confidence or unsupported detector results fall back to `und-*`.
 - Technical-noise-heavy Latin windows stay conservative and may remain `und-Latn` even when the detector produces a wrong-but-confident language guess.
+- inspect/debug disclosure uses `contentGate` as the canonical gate field.
+- legacy debug/evidence payloads still emit `qualityGate` as a compatibility alias derived from `contentGate.passed`.
+- for practical verification, use `inspect` to compare direct mode outcomes across `default`, `strict`, `loose`, and `off`; use `--debug --detector-evidence` when you specifically need counting-flow event details or legacy `qualityGate` compatibility
+- `word-counter inspect` supports:
+  - positional text input
+  - one direct `-p, --path <file>` input
+  - repeated `-p, --path` inputs for batch inspect
+  - directory inputs in default `--path-mode auto`
+  - literal file-only path handling in `--path-mode manual`
+  - `--section all|frontmatter|content`
+- batch inspect keeps counting-style path acquisition but not counting aggregation:
+  - no inspect `--merged`
+  - no inspect `--per-file`
+  - no inspect `--jobs`
+
+### Config Files
+
+`word-counter` supports config files in these canonical names:
+
+- `wc-intl-seg.config.toml`
+- `wc-intl-seg.config.json`
+- `wc-intl-seg.config.jsonc`
+
+Config precedence is:
+
+```text
+built-in defaults
+< user config dir / wc-intl-seg.config.{toml|jsonc|json}
+< cwd / wc-intl-seg.config.{toml|jsonc|json}
+< environment variables
+< flag options
+```
+
+Same-scope file priority is `toml > jsonc > json`.
+If lower-priority sibling config files are ignored, the CLI emits a warning.
+
+Detector config notes:
+
+- counting defaults to `regex`
+- `inspect` also defaults to `regex`
+- root `detector` controls normal counting
+- optional `inspect.detector` overrides inspect-only behavior
+- root `contentGate.mode` controls detector-policy defaults for counting
+- optional `inspect.contentGate.mode` overrides inspect-only detector-policy behavior
+- `WORD_COUNTER_CONTENT_GATE` overrides config-derived content-gate defaults
+- `--content-gate` stays the highest-precedence detector-policy override
+- `inspect --detector` only affects the current inspect invocation
+
+Examples:
+
+```bash
+word-counter -d wasm "This sentence should clearly be detected as English for the wasm detector path."
+word-counter --content-gate strict "Internationalization documentation remains understandable."
+word-counter inspect -d regex -f json "こんにちは、世界！これはテストです。"
+word-counter inspect --content-gate off "mode: debug\ntee: true\npath: logs\nUse this for testing."
+word-counter --path ./examples/test-case-multi-files-support --format json
+```
+
+Default-reference config examples live under:
+
+- `examples/wc-config/wc-intl-seg.config.toml`
+- `examples/wc-config/wc-intl-seg.config.json`
+- `examples/wc-config/wc-intl-seg.config.jsonc`
+
+For full config behavior, platform-specific user config locations, merge rules, and examples, see [`docs/config-usage-guide.md`](docs/config-usage-guide.md).
+
+### Detector Subpath (`@dev-pi2pie/word-counter/detector`)
+
+Use the detector subpath when you need async detector-aware APIs directly in library code.
+
+```ts
+import {
+  inspectTextWithDetector,
+  segmentTextByLocaleWithDetector,
+  wordCounterWithDetector,
+} from "@dev-pi2pie/word-counter/detector";
+
+const inspectResult = await inspectTextWithDetector("こんにちは、世界！これはテストです。", {
+  detector: "wasm",
+  view: "pipeline",
+});
+const countResult = await wordCounterWithDetector(
+  "Internationalization documentation remains understandable.",
+  {
+    detector: "wasm",
+    contentGate: { mode: "strict" },
+  },
+);
+```
+
+Detector subpath notes:
+
+- detector entrypoints are async
+- use the root package for normal counting when you do not need detector-specific control
+- detector-subpath APIs that execute detector policy also accept:
+  - `contentGate: { mode: "default" | "strict" | "loose" | "off" }`
+- use `detectorDebug` for counting-flow runtime diagnostics
+- use `inspectTextWithDetector()` for direct detector diagnosis as structured data
 
 Collect non-words (emoji/symbols/punctuation):
 
@@ -140,6 +267,7 @@ Directory scans are recursive by default:
 
 ```bash
 word-counter --path ./examples/test-case-multi-files-support
+word-counter --path ./examples/test-case-multi-files-support --recursive
 word-counter --path ./examples/test-case-multi-files-support --no-recursive
 ```
 
@@ -153,6 +281,7 @@ Progress behavior in standard batch mode:
 
 ```bash
 word-counter --path ./examples/test-case-multi-files-support
+word-counter --path ./examples/test-case-multi-files-support --progress
 word-counter --path ./examples/test-case-multi-files-support --no-progress
 word-counter --path ./examples/test-case-multi-files-support --keep-progress
 ```
@@ -174,7 +303,7 @@ Quick policy:
 - `--jobs 1`: async main-thread `load+count` baseline.
 - `--jobs > 1`: worker `load+count` with async fallback when workers are unavailable.
 - if requested `--jobs` exceeds host `suggestedMaxJobs` (from `--print-jobs-limit`), the CLI warns and runs with the suggested limit as a safety cap.
-- use `--quiet-warnings` to suppress non-fatal warning lines (for example jobs-limit advisory and worker-fallback warning).
+- use `--quiet-warnings` to suppress non-fatal warning lines (for example config discovery notes, jobs-limit advisory, and worker-fallback warning).
 
 Inspect host jobs diagnostics:
 
@@ -196,7 +325,7 @@ word-counter doctor --format json --pretty
 
 Doctor scope in v1:
 
-- checks runtime support policy against Node.js `>=20`
+- checks runtime support policy against Node.js `>=22.18.0`
 - verifies `Intl.Segmenter` availability plus word/grapheme constructor health
 - reports batch jobs host limits using the same heuristics as `--print-jobs-limit`
 - reports worker-route preflight signals and the worker-disable env toggle that affects worker availability
@@ -216,7 +345,9 @@ For full policy details, JSON parity expectations (`--misc`, `--total-of whitesp
 ### Stable Path Resolution Contract
 
 - Repeated `--path` values are accepted as mixed inputs (file + directory).
-- In `--path-mode auto` (default), directory inputs are expanded to files (recursive unless `--no-recursive`).
+- In `--path-mode auto` (default), directory inputs are expanded to files.
+  `--recursive` explicitly enables recursive traversal and overrides non-recursive config/env defaults.
+  `--no-recursive` explicitly disables recursive traversal for the current invocation.
 - In `--path-mode manual`, `--path` values are treated as literal file inputs; `--path <dir>` is not supported and is skipped as `not a regular file`.
 - Extension and regex filters apply only to files discovered from directory expansion.
 - Direct file inputs are always considered regardless of `--include-ext` / `--exclude-ext` / `--regex`.
@@ -500,6 +631,7 @@ Import from `@dev-pi2pie/word-counter/detector` for the explicit detector-enable
 | `wordCounterWithDetector`     | function | Async detector-aware counting entrypoint.       |
 | `segmentTextByLocaleWithDetector` | function | Async detector-aware locale segmentation.  |
 | `countSectionsWithDetector`   | function | Async detector-aware section counting.          |
+| `inspectTextWithDetector`     | function | Async detector-aware inspect entrypoint.        |
 | `DEFAULT_DETECTOR_MODE`       | value    | Current default detector mode (`regex`).        |
 | `DETECTOR_MODES`              | value    | Supported detector modes.                       |