kotoshu 1.0.1 → 1.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: fbf9efcde52fd9741e0453ff9c8e58117ec4ed1d2c8bddf6c00f07851c7dfdbe
4
- data.tar.gz: 838132ed1c7b305e1004b29b747024239dcb9c347e7eab0566c6cedbb839752f
3
+ metadata.gz: 15c72e0b9eecd7ad94734eb9f8cab33e693ceff9642898fb7227f68121759425
4
+ data.tar.gz: 7da7d2b1d3b8bc5bdfa5651455c47ef602d0bbac53516795a51888b573adf5e8
5
5
  SHA512:
6
- metadata.gz: caf90e1d64a196f6d866354cc811ee710302e924d9ecae40e0d216f06ae3f1b214e7637dd585a9d44855685109655aca67c6b1910508db76e3d78bdd537adb77
7
- data.tar.gz: 290bbfbeb0d6da02733d617690dcb3cb4a30e850b183287c6b371e160b73b5c1f0af81c83eb6419999382150d2f7ba67d522381429239b55f4a4420d7311d9f4
6
+ metadata.gz: 9c14c361ee5c5cb94e3db292cd801e3af0993797643de17bf5a40b2fe0e08f2d27e8060d92290103b1db038766cddf3ee0ff8a5b9c0afc621fb4f3373f50e81e
7
+ data.tar.gz: 43186beae6c49a91cde7bdd66674906eb2936f75cf25c6df118c2682a26aa578a453031a903b02b08f6c8dc83b295b889da56308ffcf89ce39c5873be4dce541
@@ -1,6 +1,10 @@
1
1
  # Plan 118 — kotoshu-server 1.0 must ride kotoshu 1.0
2
2
 
3
- Status: pending (P0 — broken release constraint)
3
+ Status: executed (kotoshu-server PR #8 + release 1.0.1, 2026-09-11)
4
+
5
+ gemspec + Gemfile + lock floor ~> 1.0; guard spec added; server 1.0.1
6
+ live on RubyGems (verified). The suite ran green on kotoshu 1.0.0
7
+ (25/0) - the 0.6-era engine had been serving this repo's own CI.
4
8
  Priority: P0 (a shipped 1.0.0 release cannot see the 1.0 gem)
5
9
 
6
10
  ## Problem
@@ -1,6 +1,12 @@
1
1
  # Plan 119 — Embed the frozen Kelly tiers in the gem
2
2
 
3
- Status: pending
3
+ Status: executed (gem PR #181, 2026-09-11)
4
+
5
+ Acceptance exceeded: XDG_CACHE_HOME=<empty> compare is 2630/0/0 -
6
+ determinism by construction, no CI seeding needed. Embedded tiers are
7
+ Set-equal to the checksummed cache tiers; data ships as JSON generated
8
+ by scripts/generate_frozen_tiers.rb (provenance + sha inside the
9
+ file); other languages unchanged. Full suite 4010/0, rubocop clean.
4
10
  Depends on: plan 117 (the dataset-divergence forensics)
5
11
 
6
12
  ## Problem
@@ -1,6 +1,12 @@
1
1
  # Plan 121 — Jekyll generator skips the sweep for baseline-covered words
2
2
 
3
- Status: pending
3
+ Status: executed (gem PR #181, 2026-09-11)
4
+
5
+ Third call site wired: the generator builds the per-file budget from
6
+ its own baseline store and threads suggestions_filter through
7
+ Kotoshu.check. Generator spec proves a beyond-budget occurrence
8
+ surfaces with suggestions while covered words never pay for the
9
+ sweep.
4
10
  Depends on: plan 116 (the suggestions_filter mechanism)
5
11
 
6
12
  ## Problem
@@ -1,6 +1,22 @@
1
1
  # Plan 122 — KPK2: compressed pack sections
2
2
 
3
- Status: designed (execution-ready; next session — multi-repo pipeline)
3
+ Status: executed as REJECTED on the pre-declared gate (2026-09-12 measurement)
4
+
5
+ zstd-19 per-section, en-1.5.0.bin (15,173,525 bytes):
6
+
7
+ aff 3,086 -> 908 (3.40x)
8
+ dic 551,762 -> 169,700 (3.25x)
9
+ model 3,040,752 -> 2,769,252 (1.10x)
10
+ vocab 202,597 -> 59,342 (3.41x)
11
+ buckets 11,375,135 -> 9,854,872 (1.15x)
12
+ total -> 12,854,074 (84.7% of KPK1)
13
+
14
+ The acceptance gate was <= 60%. The pack is 75% int8 bucket table and
15
+ 20% int8 ONNX weights — nearly incompressible; the sections that win
16
+ 3.4x are 5% of the bytes. A 15% wire saving does not pay for a format
17
+ generation, a decompressor on the wasm surface, and a registry
18
+ re-cut. Revisit only if buckets shrink or sub-int8 quantization lands
19
+ (the thing that would actually move the 84.7%).
4
20
  Depends on: plan 113 (KPK1), the 2026-09-10 model-efficiency research
5
21
 
6
22
  ## Problem
@@ -1,6 +1,20 @@
1
1
  # Plan 123 — Verifiable typo corpora, synthesized at scale
2
2
 
3
- Status: designed (execution-ready; next session — models-repo arc)
3
+ Status: executed (models PR #32, 2026-09-12)
4
+
5
+ de/es: 5,000 pairs each, dictionary-grounded, deterministic. The
6
+ admission rule was SHAPED BY MEASUREMENT: the first draft sampled raw
7
+ dictionary stems and 77% of corrections landed outside the model
8
+ vocab - the corpus would have measured the vocabulary cut, not
9
+ ranking; the checked-in rule samples the rank-ordered dict-vocab
10
+ intersection (correction-OOV now 0). make_typo_traced added to
11
+ noise.py (op provenance, weighting unchanged). Headline tier finding:
12
+ full-tier typo-OOV on these typos is 98.5% de / 98.6% es - the
13
+ quantified gap the hybrid thread claims to close. Tests in the repo's
14
+ unittest convention (admission invariants, histogram consistency,
15
+ dictionary verification, byte-deterministic rebuild). The hybrid
16
+ pricing rerun (plan 115 ship decision) still needs the plan-114
17
+ training environment reassembled; the data blocker is closed.
4
18
  Depends on: plan 115 (hybrid pricing blocked on real de/es corpora);
5
19
  validated direction: DeepSeek-V4.1-Flash §5.1 — data/environment
6
20
  synthesis exceeds algorithmic novelty
@@ -0,0 +1,43 @@
1
+ # Plan 125 — Dependency-floor guards in every consumer repo
2
+
3
+ Status: executed (docker PR #1; site push 70df393; 2026-09-12)
4
+
5
+ The guard found a REAL break on its first run: the docker CI image
6
+ could not install kotoshu at all since the native extension shipped
7
+ (ruby:slim lacks make; the 'no native ext' comment was stale). Fix:
8
+ build-essential + clang/libclang + rustup in the builder stage.
9
+ Dispositions: kotoshu-go has no artifact pin to guard (pure /v1 HTTP
10
+ client); kotoshu-python's native extra rides the deliberately-0.x
11
+ kotoshu-native line - a floor lands when that line crosses 1.0
12
+ (tracked in plan 124).
13
+ Depends on: plan 118 (the bug class: silent floor drift)
14
+
15
+ ## Problem
16
+
17
+ Plan 118 found kotoshu-server shipping 1.0.0 with a runtime floor
18
+ (`~> 0.6`) that excluded the gem it was releasing alongside — and its
19
+ own CI had been bundling kotoshu 0.6.0 the whole time. The same
20
+ silent-drift class lives wherever a consumer pins a kotoshu artifact:
21
+ docker-kotoshu-ci (base image gem), kotoshu-go (embedded wasm/native
22
+ pins or docs), kotoshu-python (wheel cargo/git pin of kotoshu-rs).
23
+ Nothing reads those pins after they are written.
24
+
25
+ ## Fix
26
+
27
+ Each repo gains the same guard shape the server got: a spec/test that
28
+ loads the pin (gemspec/Dockerfile/dependency manifest) and asserts it
29
+ admits the artifact version the repo actually builds/tests against.
30
+ A constraint nobody reads is guarded by a test that reads it.
31
+
32
+ - docker-kotoshu-ci: a CI step that installs the pinned gem and
33
+ asserts `kotoshu --version` >= the floor the Dockerfile names.
34
+ - kotoshu-python / kotoshu-go: assert the pinned kotoshu-rs rev/tag
35
+ resolves to the crate the wheels/binaries were built from (build
36
+ metadata comparison), or at minimum that the pin admits the
37
+ workspace version.
38
+
39
+ ## Acceptance
40
+
41
+ - Each repo's CI fails if its pin excludes the version it builds.
42
+ - 124's inventory item "floors to check on next cuts" shrinks to
43
+ "guards exist".
@@ -0,0 +1,33 @@
1
+ # Plan 126 — Publish the measured error budget per tier
2
+
3
+ Status: executed (site 70df393, 2026-09-12)
4
+
5
+ /performance carries the table: 55-language worst-case rank_corr /
6
+ top-1 agreement per tier against the gates, the vocabulary-cut
7
+ caveat, and the plan-123 finding that even the full tier embeds only
8
+ ~1.5% of dictionary-grounded typos.
9
+ Depends on: the 2026-09-10 model-efficiency research (DeepSeek-V4.1-Flash:
10
+ every "cheaper" claim carries its measured number)
11
+
12
+ ## Problem
13
+
14
+ The site documents tier sizes and the near-lossless headline
15
+ (fluency rank_corr 0.9999) but not a per-tier, per-language error
16
+ budget a user can reason about: what quality do I trade for mini's
17
+ 3 MB vs fluency's 15 MB vs full? The numbers exist (gates.json,
18
+ corpus benches); they are just not published as a table.
19
+
20
+ ## Fix
21
+
22
+ - /docs/performance gains an "Error budget" section: per tier, the
23
+ frozen gate values (fluency rank_corr, top-1/top-5 agreement deltas
24
+ vs full on the real corpora) with corpus sizes stated, plus the
25
+ dictionary-only baseline for contrast.
26
+ - The table cites its source (eval gates at registry tag) and its
27
+ date; numbers regenerate with each registry release (script optional
28
+ — a hand-truthed table with provenance beats a stale pipeline).
29
+
30
+ ## Acceptance
31
+
32
+ - A user can answer "what do I lose on mini?" with one number pair.
33
+ - The section states corpora sizes and OOV caveats honestly.
@@ -0,0 +1,34 @@
1
+ # Plan 127 — Report-a-wrong-suggestion: backendless feedback loop
2
+
3
+ Status: executed (site 70df393, 2026-09-12)
4
+
5
+ The popover carries a 'wrong suggestions? report' link opening a
6
+ pre-filled issue (word, top-5 rows with source+confidence, language,
7
+ engine version - no page text).
8
+ Depends on: the 2026-09-11 recommendation (real failure reports feed
9
+ plan 123's corpora; DeepSeek's environments-from-real-failures lesson)
10
+
11
+ ## Problem
12
+
13
+ The playground is our only surface where real users meet real
14
+ suggestions, and when a suggestion is wrong the feedback evaporates.
15
+ Plan 123 synthesizes pairs; the higher-value stream is REAL wrong-
16
+ suggestion reports, and capturing them needs no backend: a GitHub
17
+ issue prefill carries the diagnostic.
18
+
19
+ ## Fix
20
+
21
+ - Playground suggestion popover gains a small "report" affordance on
22
+ each row: opens github.com/kotoshu/kotoshu/issues/new?title=... with
23
+ a body template containing the word, the offered suggestions, the
24
+ language, the engine version, and (already public) tier info — no
25
+ page content, no user text beyond what the user chooses to add.
26
+ - The issue template lands in the gem repo (.github/ISSUE_TEMPLATE/
27
+ wrong-suggestion.md) so reports arrive structured; a triage note
28
+ points verified pairs at eval/corpora for the next reprice.
29
+
30
+ ## Acceptance
31
+
32
+ - One click from a wrong suggestion to a structured, pre-filled issue.
33
+ - No PII beyond the single word the user is already looking at; the
34
+ template says so.
@@ -0,0 +1,47 @@
1
+ # Plan 128 — The hybrid ship decision: price C-retrieve + fastText-rescore on the closed data blocker
2
+
3
+ Status: executed — VERDICT SHIP=True (models PR #34, 2026-09-12)
4
+
5
+ The full arc: frozen cbench rebuilt byte-faithfully (every sha256
6
+ reproduced; only the receipt wall-clock differed, reverted), c_typo_v2
7
+ retrained (0.481 MB, probe_hit5 0.410), pricing verdict per the
8
+ pre-declared rule: hybrid beats full on top-1 AND top-5 in all four
9
+ REAL components (en +6.3pp top5 CI [+4.9,+7.7] decisive; de +28.6,
10
+ ru +11.1, es +10.0), degrades nowhere, 25.8 MB int8 at ~3-4 ms.
11
+ Densified: on ~4,700 synth pairs/lang the hybrid evaluates essentially
12
+ all (top5 0.467-0.478) where full embeds only 69-85 pairs. Engine
13
+ productization = the follow-on plan (131).
14
+ Depends on: plan 123 (corpora shipped; the data blocker that parked
15
+ plan 115), plan 114 (the frozen C-benchmark + pre-declared verdict rule)
16
+
17
+ ## Problem
18
+
19
+ Plan 115 priced the hybrid retrieval thread and parked on missing de/es
20
+ typo data. Plan 123 closed that blocker (5,000 dictionary-grounded
21
+ pairs each). The plan-114 artifacts (frozen cbench files, c_typo_v2
22
+ encoder) were gitignored and wiped in the September cleanup — but the
23
+ corpus is vendored, the builder and trainer are committed, and the
24
+ whole chain is deterministic. The verdict arc is executable end to end.
25
+
26
+ ## Execution
27
+
28
+ 1. Rebuild the frozen C-benchmark: `scripts/build_cbench.py` — the
29
+ receipt (eval/reports/cbench.frozen.json) carries per-file sha256s;
30
+ the rebuild must reproduce them byte for byte or the freeze is
31
+ broken and the run stops (integrity before evidence).
32
+ 2. Retrain `c_typo_v2`: `scripts/train_typo_biencoder_v2.py` (char
33
+ bi-GRU, CPU, minutes; artifacts gitignored as before).
34
+ 3. Price: `eval/hybrid_pricing_bench.py` over en/de/ru/es — the frozen
35
+ REAL components are the ship gate (unchanged), the plan-123 corpora
36
+ are additional labeled evidence for the de/es gap.
37
+ 4. Verdict per the plan-114 pre-declared rule (wins on real, degrades
38
+ nowhere with n>=20) — ship, or reject on data like 114 did. Either
39
+ outcome closes the thread honestly.
40
+
41
+ ## Acceptance
42
+
43
+ - The cbench rebuild reproduces every frozen sha256.
44
+ - The pricing report lands (eval/reports/hybrid-pricing.json refresh)
45
+ with real + synth components labeled, and the verdict is recorded in
46
+ the plan status with numbers.
47
+ - No frozen artifact changes; new evidence only.
@@ -0,0 +1,23 @@
1
+ # Plan 129 — pt/fr synthetic corpora (plan 123 stretch targets)
2
+
3
+ Status: executed (models PR #34, 2026-09-12) — 5,000 pairs each,
4
+ same rule, tests at four languages; tier numbers in the refreshed
5
+ synth-corpus summary.
6
+ Depends on: plan 123 (the generator)
7
+
8
+ ## Problem
9
+
10
+ Plan 123 shipped de/es; pt and fr were declared stretch targets. The
11
+ generator, tests, and bench all generalize by construction — extending
12
+ is a parameter, not new code.
13
+
14
+ ## Fix
15
+
16
+ - Generate eval/corpora/synth/{pt,fr}.json at the same 5,000-pair
17
+ target with the same admission rule (pt: ABNT qwerty + accents; fr:
18
+ azerty alternates — both already in noise.py's layout registry).
19
+ - Extend the tier baseline bench + tests to the new languages.
20
+
21
+ ## Acceptance
22
+
23
+ - Same invariants as 123, same test coverage, tier numbers recorded.
@@ -0,0 +1,23 @@
1
+ # Plan 130 — News truth: the 1.0.1 patch and the 09-12 wave
2
+
3
+ Status: executed (site 6daa72d, 2026-09-12) — gem-1-0-1 and
4
+ evidence-2026-09 entries live; NewsKind widened with 'docs'.
5
+ Depends on: plans 119/121 (1.0.1 content), 123/125/126/127
6
+
7
+ ## Problem
8
+
9
+ Two shipped changes have no news entries: gem 1.0.1 (embedded frozen
10
+ tiers, Jekyll sweep skip) and the 09-12 wave (corpora, error budgets,
11
+ the feedback loop, the docker image fix). The site's discipline is one
12
+ entry per user-visible change.
13
+
14
+ ## Fix
15
+
16
+ - One entry for 1.0.1 (determinism by construction: cache-cold == warm
17
+ == frozen vectors).
18
+ - One entry for the wave: corpora + the 1.5% typo-embedding finding +
19
+ error-budget table + report-a-suggestion link.
20
+
21
+ ## Acceptance
22
+
23
+ - Entries live, dates and version references truthful.
@@ -0,0 +1,42 @@
1
+ # Plan 131 — Productize the hybrid: C-retrieve + fastText-rescore in the engines
2
+
3
+ Status: proposed (the decision evidence is complete; plan 128 verdict SHIP)
4
+ Depends on: plan 128 (verdict + artifacts), plan 113 (packs — the
5
+ delivery vehicle), registry v1.6.0 (typo-biencoder assets already opt-in)
6
+
7
+ ## What ships
8
+
9
+ The hybrid becomes the opt-in "typo retrieval" layer, the same shape
10
+ as the semantic tier: per-language int8 C-matrix (25.8 MB) + the char
11
+ bi-encoder (0.48 MB) fetched from the registry, C-retrieve top-20 over
12
+ the vocabulary, fastText-rescore of the candidates, merged into the
13
+ suggestion pipeline ahead of frequency ranking.
14
+
15
+ ## Surfaces
16
+
17
+ - kotoshu-rs: char-bi-encoder ONNX inference (the pure-Rust ONNX
18
+ reader already exists for the tiers), the retrieval+rescore path,
19
+ wasm surface (`loadTypo` + `typoSuggest`), KPK1 pack extension (the
20
+ C-matrix rides the pack for pack languages).
21
+ - gem: native-ext accelerated path where present; the merge into
22
+ Suggestions::Pipeline as a strategy (OCP: a new strategy class, no
23
+ pipeline surgery).
24
+ - Conformance: new vectors for the hybrid path are NOT frozen
25
+ initially (it is opt-in, tier-like, gated by eval not conformance —
26
+ the tier precedent).
27
+ - Registry: promote the typo-biencoder entries from opt-in to exposed;
28
+ server/playground/worker flips follow the semantic-tier pattern.
29
+
30
+ ## Gates (pre-declared)
31
+
32
+ - The frozen real components + synth corpora re-run against the
33
+ SHIPPED engines must reproduce the PR-34 numbers within noise.
34
+ - Latency: per-suggest budget ≤ 10 ms amortized (the plan-112 gates
35
+ tighten for the shipped path).
36
+ - Degrades nowhere vs dictionary-only when disabled; zero behavior
37
+ change at default settings.
38
+
39
+ ## Non-goals
40
+
41
+ - No retraining; the v2 artifacts are frozen (probe_hit5 0.410).
42
+ - No default-on: opt-in per language, like the semantic tier.
@@ -0,0 +1,27 @@
1
+ # Plan 132 — The techniques page; professional prose across the site
2
+
3
+ Status: executed (2026-09-12, site push; this plan)
4
+
5
+ ## What shipped
6
+
7
+ 1. /docs/techniques — a complete catalog of the stack's techniques in
8
+ four groups (engines, models, delivery, reliability), each entry
9
+ naming the mechanism and its measured effect in full sentences.
10
+ Written as the site's first MDX page: @astrojs/mdx installed and
11
+ wired, DocsLayout applied through an imported component wrapper.
12
+ 2. A professional rewrite of the published copy: 34 fragment-style
13
+ news titles became declarative sentences, and 69 fragment-stacked
14
+ summaries and senses across all 36 entries were rewritten as full
15
+ sentences with clear subjects. The worst patterns eliminated:
16
+ em-dash fragment leads ("What 1.0 freezes:"), colon-list openers
17
+ ("Also in the patch:"), trailing fragment appends ("— no download,
18
+ no CI seeding"), and lowercase clause starts.
19
+ 3. The docs pages received the same treatment where fragments
20
+ appeared: the performance error-budget section, the caching
21
+ language-packs section, and three headlines.
22
+
23
+ ## Conventions established
24
+
25
+ - Text-heavy documentation pages are written in MDX, not .astro HTML.
26
+ - News titles are complete clauses; summaries and senses are full
27
+ sentences. Release titles use the "package X.Y.Z: description" form.
@@ -0,0 +1,68 @@
1
+ # Plan 133 — Rust-first distribution: precompiled native gems and complete wheels
2
+
3
+ Status: proposed (execution-ready)
4
+ Depends on: the 1.0 conformance contract (identical outputs make the
5
+ backend default flip correctness-free); plan 93 (the python wheel matrix)
6
+
7
+ ## Problem
8
+
9
+ The Rust engine is the faster implementation (~10-15x on the sweep) and
10
+ the strategic default, but today reaching it requires a build toolchain:
11
+ the gem compiles its native extension at install time, and the python
12
+ native wheel matrix is incomplete on PyPI. Users who cannot run a
13
+ toolchain get pure Ruby whether they wanted it or not — and users who
14
+ wanted Rust pay a compile for it. The architecture makes both engines
15
+ byte-identical, so distribution — not capability — is the only gap.
16
+
17
+ ## Design
18
+
19
+ ### Ruby: precompiled platform gems (the rb_sys flow)
20
+
21
+ 1. New gem workflow `release-native.yml`: build matrix over
22
+ {x86_64-linux, aarch64-linux, x86_64-linux-musl, arm64-darwin,
23
+ x86_64-darwin, x64-mingw-ucrt} x {ruby 3.1..4.0} using
24
+ rb-sys-dock images for the Linux targets and native runners for
25
+ macOS/Windows; each job builds and pushes
26
+ `kotoshu-X.Y.Z-<platform>.gem` through trusted publishing. The
27
+ pure-Ruby platform gem continues to publish exactly as today, so
28
+ every unmatched platform resolves to it.
29
+ 2. Backend default `ruby` -> `auto`: use the native engine when the
30
+ extension loads, fall back silently to pure Ruby when it does not.
31
+ Auto never downloads and never fails a check; specs pin both
32
+ branches; the conformance job already runs both engines.
33
+ 3. The docker CI image drops its build-tool layer (it installs the
34
+ platform gem instead) — smaller image, faster builds, and its
35
+ version guard gains a native-backend assertion.
36
+
37
+ ### Python: complete the kotoshu-native wheel matrix
38
+
39
+ 1. Audit PyPI's published wheels for kotoshu-native against the
40
+ plan-93 matrix: manylinux x86_64/aarch64, macOS x86_64/arm64,
41
+ windows x86_64 (+ arm64 if tractable), sdist.
42
+ 2. Fill the gaps in release-pypi.yml (maturin + the standard target
43
+ matrix), publish a patch release, and verify pip resolves a wheel
44
+ (no source build) on each platform.
45
+ 3. Add the trove classifiers for every published target.
46
+
47
+ ### Site
48
+
49
+ Install page's engines-and-bindings section gains the resolution
50
+ sentence: the native engine installs automatically wherever a prebuilt
51
+ artifact exists for your platform, and pure Ruby is always the
52
+ fallback; python's entry notes the wheel coverage.
53
+
54
+ ## Gates
55
+
56
+ - A clean-machine `gem install kotoshu` on each published platform
57
+ loads the native backend with no toolchain present (CI-verifiable
58
+ on the matrix runners).
59
+ - The 2,630-vector conformance compare passes under `auto` on both a
60
+ native and a pure-Ruby install.
61
+ - pip on each target platform resolves a wheel, not the sdist.
62
+ - The pure-Ruby path is unchanged: same gem, same outputs, same specs.
63
+
64
+ ## Non-goals
65
+
66
+ - No behavior change of any kind — outputs are frozen; only which
67
+ engine computes them changes.
68
+ - No deprecation of anything: pure Ruby is a feature, not a legacy.
@@ -207,7 +207,7 @@ module Kotoshu
207
207
  backend: {
208
208
  env: "KOTOSHU_BACKEND",
209
209
  default: "ruby",
210
- description: "Engine backend for correct?/suggest (ruby, native, or auto)",
210
+ description: "Engine backend for correct?/suggest (auto by default: native when the extension is available, ruby otherwise; or force either)",
211
211
  type: String
212
212
  },
213
213
  personal_dictionary: {
@@ -235,7 +235,12 @@ module Kotoshu
235
235
  default_language: "en",
236
236
  resource_pin: "main",
237
237
  semantic_cascade_threshold: 1.0,
238
- backend: "ruby",
238
+ # "auto" (plan 133): the Rust engine whenever the native
239
+ # extension loaded and the dictionary is a file-backed Hunspell,
240
+ # pure Ruby otherwise — identical outputs either way (the
241
+ # conformance contract), so the default is the faster engine
242
+ # with the pure fallback, never a failure.
243
+ backend: "auto",
239
244
  personal_dictionary: true
240
245
  }.freeze
241
246
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Kotoshu
4
- VERSION = '1.0.1'
4
+ VERSION = '1.0.2'
5
5
  end
@@ -0,0 +1,45 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ # Build a precompiled platform gem (plan 133): assemble kotoshu's
5
+ # gemspec around the ALREADY-COMPILED extension binary, exactly the
6
+ # surgery rb_sys's cross-compiling blocks perform — the rb_sys build
7
+ # dependency goes away (nothing builds at install), the Rust sources,
8
+ # Cargo files, and extconf drop out of the file list, and the platform
9
+ # tag makes RubyGems resolve this gem on matching platforms while the
10
+ # pure-Ruby gem keeps serving everything else.
11
+ #
12
+ # Usage: ruby scripts/build_native_gem.rb [platform]
13
+ # platform defaults to Gem::Platform.local; pass e.g. x86_64-linux
14
+ # when the binary was produced for another host. The compiled
15
+ # extension must already sit at lib/kotoshu/kotoshu_native.<dlext>
16
+ # (rake compile on the matching platform).
17
+
18
+ require "rubygems/package"
19
+ require "rbconfig"
20
+
21
+ platform = ARGV[0]
22
+ dlext = RbConfig::CONFIG["DLEXT"]
23
+ binary = File.join("lib", "kotoshu", "kotoshu_native.#{dlext}")
24
+
25
+ unless File.exist?(binary)
26
+ abort "missing #{binary} — run `rake compile` on the target platform first"
27
+ end
28
+
29
+ spec = Gem::Specification.load("kotoshu.gemspec")
30
+ spec.platform = platform ? Gem::Platform.new(platform) : Gem::Platform.local
31
+ # The darwin deployment-target suffix (-23, -25, ...) differs per build
32
+ # machine; RubyGems matches a versionless darwin tag on any macOS of
33
+ # the same CPU, so normalize it off (the convention shipped binary
34
+ # gems use). Linux and mingw tags carry no version to begin with.
35
+ if spec.platform.os == "darwin" && spec.platform.version
36
+ spec.platform = Gem::Platform.new("#{spec.platform.cpu}-darwin")
37
+ end
38
+ spec.files += [binary]
39
+ spec.extensions = []
40
+ spec.dependencies.reject! { |d| d.name == "rb_sys" }
41
+ spec.files.reject! { |f| f.start_with?("ext/") }
42
+
43
+ path = Gem::Package.build(spec)
44
+ puts "built #{path}"
45
+ path
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: kotoshu
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.1
4
+ version: 1.0.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ribose Inc.
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2026-09-11 00:00:00.000000000 Z
11
+ date: 2026-09-12 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: thor
@@ -130,6 +130,15 @@ files:
130
130
  - TODO.impl/122-pack-compression-kpk2.md
131
131
  - TODO.impl/123-typo-corpus-synthesis.md
132
132
  - TODO.impl/124-owner-residue.md
133
+ - TODO.impl/125-consumer-floor-guards.md
134
+ - TODO.impl/126-tier-error-budgets.md
135
+ - TODO.impl/127-suggestion-feedback-loop.md
136
+ - TODO.impl/128-hybrid-pricing-verdict.md
137
+ - TODO.impl/129-pt-fr-corpora.md
138
+ - TODO.impl/130-news-truth-0912.md
139
+ - TODO.impl/131-hybrid-productization.md
140
+ - TODO.impl/132-techniques-page-and-prose.md
141
+ - TODO.impl/133-rust-first-distribution.md
133
142
  - TODO.impl/20-encapsulation-cleanup.md
134
143
  - TODO.impl/21-autoload-migration.md
135
144
  - TODO.impl/22-url-pin-single-source-of-truth.md
@@ -521,6 +530,7 @@ files:
521
530
  - lib/kotoshu/version.rb
522
531
  - rakelib/compile.rake
523
532
  - script/validate_all_dictionaries.rb
533
+ - scripts/build_native_gem.rb
524
534
  - scripts/extract_eval_grids.rb
525
535
  - scripts/generate_frozen_tiers.rb
526
536
  - sig/kotoshu.rbs