kotoshu-server 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 80541f5f06f9c4f0c41cbdecfbf195fec21d630f9e89711c061f4a3796753cfc
4
- data.tar.gz: dae5536d6035f67546136f12ffe1fc52d602bdffc8dd5eeba120fec2313c4cd7
3
+ metadata.gz: 6fa24971e66e791bd277c8fe878890693c6a624a3de1c488dbb7f5b38f629f0a
4
+ data.tar.gz: eabc7a8dd8124cf20a1c05b32257e9fdfdcdebeb72ac6aceae249c2aba692976
5
5
  SHA512:
6
- metadata.gz: b3bcab28c17474fbc07c721bb7686ae6d7505580a70e03042438aef75d032b846efc9576520351f1967a4e5fc04d8e9b21590a4961b7f9bbc420bb3ff10505b6
7
- data.tar.gz: 8cf8d948c812075b068479e25737a0d43999831f2dea87a018d6d62fe8c1049d96bb27076f88173ed91d4bd4ae1939669d768402ee9e8ce8e5c215c14014d909
6
+ metadata.gz: fac8b861344a723119bf84e7c078c772d491d974cb37a9338b2100acb0e28cf9e17d83e06b4f18aad41ecba9d2ec78bc6249575830b485f03f6f1200ab58daa0
7
+ data.tar.gz: bd7c11018743035167858f90f8a7b70aa29efdfb6313496b3f6d06a1e297931985e6259625d3b43b34a5236d03179f58851a33f388f9f5a6e2cbcb0701ab745b
data/LICENSE ADDED
@@ -0,0 +1,26 @@
1
+ BSD 2-Clause License
2
+
3
+ Copyright (c) 2025-2026, Kotoshu contributors
4
+ All rights reserved.
5
+
6
+ Redistribution and use in source and binary forms, with or without
7
+ modification, are permitted provided that the following conditions are met:
8
+
9
+ 1. Redistributions of source code must retain the above copyright notice, this
10
+ list of conditions and the following disclaimer.
11
+
12
+ 2. Redistributions in binary form must reproduce the above copyright notice,
13
+ this list of conditions and the following disclaimer in the documentation
14
+ and/or other materials provided with the distribution.
15
+
16
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
17
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
19
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
20
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
22
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
23
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
24
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
26
+
data/README.md ADDED
@@ -0,0 +1,141 @@
1
+ # kotoshu-server
2
+
3
+ Self-hostable HTTP API wrapping the [Kotoshu](https://github.com/kotoshu/kotoshu) spell checker.
4
+
5
+ ## Status
6
+
7
+ MVP. Six endpoints over JSON. Rack/Sinatra + Puma. Pre-warms
8
+ languages on boot. Designed as the deployment surface for non-Ruby
9
+ SDKs (`kotoshu-python`, `kotoshu-js`, `kotoshu-go`).
10
+
11
+ See `TODO.impl/64-http-api-and-sdks.md` for the full plan.
12
+
13
+ ## Install
14
+
15
+ **Run from source.** The published `kotoshu-server 0.1.0` gem is
16
+ empty (a gemspec file-list bug, fixed on main). Use source until the
17
+ owner republishes 0.1.1:
18
+
19
+ ```bash
20
+ cd kotoshu-server && bundle install && bundle exec exe/kotoshu-server
21
+ ```
22
+
23
+ ## Run
24
+
25
+ ```bash
26
+ # Default: localhost:9292, English pre-warmed
27
+ kotoshu-server
28
+
29
+ # Custom port + multi-language pre-warm
30
+ KOTOSHU_SERVER_PORT=8080 \
31
+ KOTOSHU_SERVER_LANGUAGES="en de fr" \
32
+ kotoshu-server
33
+ ```
34
+
35
+ ## Semantic models (opt-in)
36
+
37
+ By default the server is dictionary-only — exactly the 0.1.0
38
+ behavior. Semantic reranking is a boot-time opt-in; the server never
39
+ downloads a model implicitly:
40
+
41
+ ```bash
42
+ KOTOSHU_SERVER_LANGUAGES="en de" \
43
+ KOTOSHU_SERVER_MODEL_LANGS="en" \
44
+ kotoshu-server
45
+ ```
46
+
47
+ - `KOTOSHU_SERVER_MODEL_LANGS` — languages the pre-warm thread sets
48
+ up with spelling + semantic model (space separated). Unset means no
49
+ models, no downloads, no behavior change.
50
+ - `KOTOSHU_SERVER_MODEL_TIER` — model tier to set up and resolve:
51
+ `fluency` (default), `full`, or `mini`.
52
+
53
+ **Requires kotoshu >= 0.7.0.** Model tiers, the resource registry,
54
+ and the confidence cascade ship in the 0.7.0 cut. The gemspec keeps
55
+ its `kotoshu ~> 0.6` constraint (dependency floors are the owner's
56
+ decision), so the server checks the *installed* gem instead: setting
57
+ `KOTOSHU_SERVER_MODEL_LANGS` with an older kotoshu fails fast at
58
+ boot with a clear error, and explicit `"model": true` requests
59
+ return 503.
60
+
61
+ Once a language has a model set up, `POST /v1/check` accepts an
62
+ optional `"model": true|false` flag (default: whether the language
63
+ has a model set up server-side). With it on, each error's
64
+ suggestions are reranked by the semantic analyzer; the gem's
65
+ confidence cascade (`KOTOSHU_SEMANTIC_CASCADE_THRESHOLD`) decides per
66
+ word whether the ONNX rerank actually runs. `GET /v1/languages`
67
+ reports `"model": {"en": true}` per cached language.
68
+
69
+ Memory and latency: budget roughly 15 MB resident per language at
70
+ the default fluency tier (the full tier is far larger), plus
71
+ one-time model load on the first model-enabled request. Listing the
72
+ language in `KOTOSHU_SERVER_MODEL_LANGS` warms it at boot instead.
73
+
74
+ ## Endpoints
75
+
76
+ | Method | Path | Body | Returns |
77
+ |---|---|---|---|
78
+ | `GET` | `/` | — | service metadata |
79
+ | `GET` | `/v1/health` | — | `{ status, ready, timestamp }` |
80
+ | `GET` | `/v1/version` | — | `{ server, kotoshu, ruby }` |
81
+ | `GET` | `/v1/languages` | — | `{ cached, supported, model }` |
82
+ | `POST` | `/v1/check` | `{ text, language?, format?, model? }` | `{ file, word_count, errors: [...] }` |
83
+ | `POST` | `/v1/suggest` | `{ word, language?, max? }` | `{ word, suggestions: [...] }` |
84
+ | `POST` | `/v1/detect` | `{ text }` | `{ language, confidence }` |
85
+
86
+ Each `error` and `suggestion` mirrors the lutaml-model serialization
87
+ (`word`, `distance`, `confidence`, `source`).
88
+
89
+ ## curl examples
90
+
91
+ ```bash
92
+ # Check a document
93
+ curl -X POST http://localhost:9292/v1/check \
94
+ -H "Content-Type: application/json" \
95
+ -d '{"text":"helo wrold","language":"en"}'
96
+
97
+ # Suggestions for one word
98
+ curl -X POST http://localhost:9292/v1/suggest \
99
+ -H "Content-Type: application/json" \
100
+ -d '{"word":"helo","language":"en","max":3}'
101
+
102
+ # Detect language
103
+ curl -X POST http://localhost:9292/v1/detect \
104
+ -H "Content-Type: application/json" \
105
+ -d '{"text":"bonjour le monde"}'
106
+ ```
107
+
108
+ ## Docker
109
+
110
+ ```bash
111
+ docker build -t kotoshu-server .
112
+
113
+ docker run --rm -p 9292:9292 \
114
+ -e KOTOSHU_SERVER_LANGUAGES="en de" \
115
+ kotoshu-server
116
+ ```
117
+
118
+ Healthcheck probes `/v1/health` every 30s.
119
+
120
+ ## Configuration
121
+
122
+ | Env var | Default | Purpose |
123
+ |---|---|---|
124
+ | `KOTOSHU_SERVER_PORT` | `9292` | Listen port |
125
+ | `KOTOSHU_SERVER_BIND` | `0.0.0.0` | Bind address |
126
+ | `KOTOSHU_SERVER_LANGUAGES` | `en` | Languages to pre-warm on boot |
127
+ | `KOTOSHU_SERVER_MODEL_LANGS` | unset | Languages to set up with a semantic model (requires kotoshu >= 0.7) |
128
+ | `KOTOSHU_SERVER_MODEL_TIER` | `fluency` | Model tier for `KOTOSHU_SERVER_MODEL_LANGS` |
129
+ | `KOTOSHU_SERVER_LAZY` | `0` | Skip pre-warm; load on first request |
130
+ | `KOTOSHU_SERVER_DEFAULT_LANG` | `en` | When client omits `language` |
131
+ | `KOTOSHU_SERVER_LOG_LEVEL` | `info` | `debug`/`info`/`warn`/`error` |
132
+ | `KOTOSHU_OFFLINE` | `1` (in Docker) | Never trigger downloads |
133
+
134
+ ## OpenAPI
135
+
136
+ The full OpenAPI 3.1 spec is at [`openapi.yaml`](./openapi.yaml). Use
137
+ `openapi-generator` to spin up SDKs in Rust / .NET / Java.
138
+
139
+ ## License
140
+
141
+ BSD-2-Clause, same as Kotoshu.
@@ -0,0 +1,7 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ $LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
5
+ require "kotoshu/server"
6
+
7
+ Kotoshu::Server.run!
@@ -0,0 +1,510 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "json"
4
+ require "logger"
5
+ require "sinatra/base"
6
+ require "kotoshu"
7
+
8
+ module Kotoshu
9
+ module Server
10
+ # Raised at boot when KOTOSHU_SERVER_MODEL_LANGS / _TIER are set
11
+ # but cannot be honored (kotoshu gem too old, unknown tier).
12
+ class ModelConfigError < StandardError; end
13
+
14
+ class App < Sinatra::Base
15
+ VERSION = "0.1.0".freeze
16
+
17
+ # Semantic models (tiers, registry, confidence cascade) ship in
18
+ # kotoshu 0.7.0. The gemspec keeps its `kotoshu ~> 0.6`
19
+ # constraint (dependency floors are the owner's call), so the
20
+ # server validates the *installed* gem at boot instead.
21
+ MODEL_MIN_KOTOSHU = Gem::Version.new("0.7.0")
22
+
23
+ # Languages to set up at boot, from KOTOSHU_SERVER_LANGUAGES
24
+ # (space separated). Single source of truth for the env var: the
25
+ # boot pre-warm and /v1/health both read it here.
26
+ #
27
+ # @return [Array<String>] Language codes to pre-warm
28
+ def self.configured_languages
29
+ ENV.fetch("KOTOSHU_SERVER_LANGUAGES", "en").split
30
+ end
31
+
32
+ # Languages to set up with spelling + semantic model at boot,
33
+ # from KOTOSHU_SERVER_MODEL_LANGS (space separated, e.g. "en de").
34
+ # Empty when unset — boot-time opt-in only, never an implicit
35
+ # download (the gem's two-stage promise).
36
+ #
37
+ # @return [Array<String>] Language codes to set up with a model
38
+ def self.model_languages
39
+ ENV.fetch("KOTOSHU_SERVER_MODEL_LANGS", "").split
40
+ end
41
+
42
+ # Model tier used for setup and resolution, from
43
+ # KOTOSHU_SERVER_MODEL_TIER. Defaults to "fluency", the
44
+ # ecosystem default (owner decision 2026-09-04).
45
+ #
46
+ # @return [String] "full", "fluency", or "mini"
47
+ def self.model_tier
48
+ ENV.fetch("KOTOSHU_SERVER_MODEL_TIER", "fluency")
49
+ end
50
+
51
+ # Whether the installed kotoshu gem carries the semantic-model
52
+ # surface the server wires against (0.7.0+).
53
+ #
54
+ # @return [Boolean]
55
+ def self.semantic_models_supported?
56
+ Gem::Version.new(Kotoshu::VERSION) >= MODEL_MIN_KOTOSHU
57
+ end
58
+
59
+ # Whether a semantic model is actually set up server-side for a
60
+ # language (cache-only lookup; false on gems without model
61
+ # support, so the /v1/check default flag never turns models on
62
+ # underneath an old install).
63
+ #
64
+ # @param language [String] Language code
65
+ # @return [Boolean]
66
+ def self.model_available?(language)
67
+ return false unless semantic_models_supported?
68
+
69
+ Kotoshu.setup?(language.to_sym, :model)
70
+ rescue StandardError
71
+ false
72
+ end
73
+
74
+ # Validate the semantic-model environment before anything is
75
+ # set up. Raises {ModelConfigError} with an actionable message
76
+ # when KOTOSHU_SERVER_MODEL_LANGS is set but the installed
77
+ # kotoshu gem predates model support, or the configured tier is
78
+ # unknown. A no-op when the vars are unset — boot behavior is
79
+ # then exactly today's.
80
+ #
81
+ # @raise [ModelConfigError] on an unsatisfiable configuration
82
+ # @return [void]
83
+ def self.validate_model_config!
84
+ return if model_languages.empty?
85
+
86
+ unless semantic_models_supported?
87
+ raise ModelConfigError,
88
+ "KOTOSHU_SERVER_MODEL_LANGS requires kotoshu >= 0.7.0 " \
89
+ "(installed: #{Kotoshu::VERSION}). Semantic models ship in " \
90
+ "kotoshu 0.7.0; unset KOTOSHU_SERVER_MODEL_LANGS or upgrade " \
91
+ "the kotoshu gem."
92
+ end
93
+
94
+ begin
95
+ Kotoshu::Cache::ModelCache.normalize_tier(model_tier)
96
+ rescue ArgumentError => e
97
+ raise ModelConfigError, "KOTOSHU_SERVER_MODEL_TIER: #{e.message}"
98
+ end
99
+ end
100
+
101
+ # Synchronously set up the given languages (downloads on a cold
102
+ # or expired cache). Runs on the pre-warm thread; also usable
103
+ # directly by embedders that want a blocking warm-up.
104
+ #
105
+ # Languages listed in KOTOSHU_SERVER_MODEL_LANGS are set up with
106
+ # spelling + model (at KOTOSHU_SERVER_MODEL_TIER) in one setup
107
+ # call; the rest get spelling only, as before. Both lists are
108
+ # unioned, and setup is idempotent, so a language in both is
109
+ # set up once with the model.
110
+ #
111
+ # @param languages [Array<String>] Language codes
112
+ # @return [void]
113
+ def self.prewarm!(languages)
114
+ logger = Logger.new($stderr)
115
+ model_langs = model_languages
116
+ tier = model_tier
117
+ (languages | model_langs).each do |lang|
118
+ with_model = model_langs.include?(lang)
119
+ logger.info("pre-warming #{lang}#{with_model ? " (spelling + model, #{tier} tier)" : ""}")
120
+ begin
121
+ if with_model
122
+ Kotoshu.setup(lang.to_sym, want: %i[spelling model], tier: tier)
123
+ else
124
+ Kotoshu.setup(lang.to_sym)
125
+ end
126
+ logger.info("pre-warm #{lang} complete")
127
+ rescue StandardError => e
128
+ logger.warn("pre-warm #{lang} failed: #{e.message}")
129
+ end
130
+ end
131
+ end
132
+
133
+ # Start pre-warming in a detached background thread and return
134
+ # immediately. The server must bind and serve within seconds of
135
+ # boot, while resource setup can block on the network
136
+ # (download retries, and DNS resolution that no Net::HTTP
137
+ # timeout bounds) — so setup never runs on the boot path.
138
+ # Progress and completion are logged; /v1/health reports
139
+ # per-language readiness while it runs.
140
+ #
141
+ # @param languages [Array<String>] Language codes
142
+ # @return [Thread] the detached pre-warm thread
143
+ def self.prewarm_async!(languages)
144
+ validate_model_config! # fail fast, on the caller's thread
145
+ Thread.new do
146
+ Thread.current.name = "kotoshu-server-prewarm"
147
+ prewarm!(languages)
148
+ end
149
+ end
150
+
151
+ # ---- Semantic analyzers (memoized per language + model file) ----
152
+
153
+ @semantic_analyzers = {}
154
+ @semantic_analyzers_mutex = Mutex.new
155
+
156
+ # A memoized {Kotoshu::Analyzers::SemanticAnalyzer} for the model
157
+ # file the ResourceManager resolved for `language`. Loading a
158
+ # model (vocab + ONNX session) is expensive; one analyzer per
159
+ # language is kept for the process lifetime. A changed model
160
+ # file (different path) loads a fresh analyzer.
161
+ #
162
+ # @param language [String] Language code
163
+ # @param model_info [Hash] bundle.model from ResourceManager
164
+ # (needs :model_path)
165
+ # @return [Kotoshu::Analyzers::SemanticAnalyzer]
166
+ def self.semantic_analyzer_for(language, model_info)
167
+ path = model_info[:model_path]
168
+ @semantic_analyzers_mutex.synchronize do
169
+ @semantic_analyzers[[language.to_s, path]] ||= begin
170
+ model = Kotoshu::Models::OnnxModel.from_file(path, language_code: language.to_s)
171
+ Kotoshu::Analyzers::SemanticAnalyzer.new(model)
172
+ end
173
+ end
174
+ end
175
+
176
+ configure do
177
+ set :logging, false
178
+ set :show_exceptions, false
179
+
180
+ app_logger = Logger.new($stderr)
181
+ app_logger.level = ENV.fetch("KOTOSHU_SERVER_LOG_LEVEL", Logger::INFO)
182
+ set :app_logger, app_logger
183
+ end
184
+
185
+ # ---- Endpoints ----
186
+
187
+ get "/" do
188
+ content_type :json
189
+ {
190
+ name: "kotoshu-server",
191
+ version: VERSION,
192
+ kotoshu_version: Kotoshu::VERSION,
193
+ docs: "/v1/health, /v1/version, /v1/languages, /v1/check, /v1/suggest, /v1/detect"
194
+ }.to_json
195
+ end
196
+
197
+ get "/v1/health" do
198
+ content_type :json
199
+ ready = self.class.configured_languages.map { |l| [l, Kotoshu.setup?(l.to_sym, :spelling)] }.to_h
200
+ { status: "ok", ready: ready, timestamp: Time.now.utc.iso8601 }.to_json
201
+ end
202
+
203
+ get "/v1/version" do
204
+ content_type :json
205
+ { server: VERSION, kotoshu: Kotoshu::VERSION, ruby: RUBY_VERSION }.to_json
206
+ end
207
+
208
+ get "/v1/languages" do
209
+ content_type :json
210
+ cached = Kotoshu.languages_setup
211
+ model = cached.to_h { |lang| [lang, self.class.model_available?(lang)] }
212
+ { cached: cached, supported: cached, model: model }.to_json
213
+ end
214
+
215
+ post "/v1/check" do
216
+ body = parse_json_body(request.body.read)
217
+ text = body["text"]
218
+ language = body["language"] || ENV.fetch("KOTOSHU_SERVER_DEFAULT_LANG", "en")
219
+ format_hint = body["format"] || "full"
220
+ want_model = model_request_flag(body["model"], language)
221
+
222
+ halt_with_error(400, "missing 'text'") unless text.is_a?(String)
223
+
224
+ want = want_model ? %i[spelling model] : %i[spelling]
225
+ result = with_resource(language, want: want) do |checker, bundle|
226
+ if want_model && bundle.model.nil?
227
+ # The gem resolves a nil model for languages that cannot
228
+ # have one; an explicit request must not silently degrade.
229
+ halt 422, { "Content-Type" => "application/json" },
230
+ {
231
+ error: "resource_not_setup",
232
+ message: "no semantic model set up for language '#{language}'",
233
+ hint: "list it in KOTOSHU_SERVER_MODEL_LANGS and restart"
234
+ }.to_json
235
+ end
236
+ check_result = checker.check(text)
237
+ want_model ? rerank_semantic(language, check_result, text, bundle.model) : check_result
238
+ end
239
+
240
+ content_type :json
241
+ case format_hint
242
+ when "errors" then serialize_errors(result).to_json
243
+ else serialize_full(result).to_json
244
+ end
245
+ end
246
+
247
+ post "/v1/suggest" do
248
+ body = parse_json_body(request.body.read)
249
+ word = body["word"]
250
+ language = body["language"] || ENV.fetch("KOTOSHU_SERVER_DEFAULT_LANG", "en")
251
+ max = body["max"]&.to_i
252
+
253
+ halt_with_error(400, "missing 'word'") unless word.is_a?(String)
254
+
255
+ result = with_resource(language) do |checker|
256
+ checker.suggest(word, max_suggestions: max)
257
+ end
258
+
259
+ content_type :json
260
+ { word: word, suggestions: serialize_suggestions(result) }.to_json
261
+ end
262
+
263
+ post "/v1/detect" do
264
+ body = parse_json_body(request.body.read)
265
+ text = body["text"]
266
+ halt_with_error(400, "missing 'text'") unless text.is_a?(String)
267
+
268
+ lang, confidence = Kotoshu.detect_language_with_confidence(text)
269
+ content_type :json
270
+ { language: lang, confidence: confidence }.to_json
271
+ end
272
+
273
+ # ---- Error handling ----
274
+
275
+ error Kotoshu::Server::ModelConfigError do
276
+ status 500
277
+ content_type :json
278
+ { error: "model_config", message: env["sinatra.error"].message }.to_json
279
+ end
280
+
281
+ error Kotoshu::Models::OnnxModel::OnnxUnavailable do
282
+ status 503
283
+ content_type :json
284
+ { error: "onnx_unavailable", message: env["sinatra.error"].message,
285
+ hint: "install the onnxruntime gem, or stop setting model langs" }.to_json
286
+ end
287
+
288
+ error Kotoshu::ResourceNotSetupError do
289
+ e = env["sinatra.error"]
290
+ status 422
291
+ content_type :json
292
+ { error: "resource_not_setup", message: e.message,
293
+ hint: "POST /v1/admin/setup with {language: \"...\"} (admin only)" }.to_json
294
+ end
295
+
296
+ error JSON::ParserError do
297
+ status 400
298
+ content_type :json
299
+ { error: "invalid_json", message: env["sinatra.error"].message }.to_json
300
+ end
301
+
302
+ error StandardError do
303
+ e = env["sinatra.error"]
304
+ settings.app_logger&.error("unhandled: #{e.class}: #{e.message}")
305
+ status 500
306
+ content_type :json
307
+ { error: "internal", message: e.message }.to_json
308
+ end
309
+
310
+ # ---- Helpers ----
311
+
312
+ helpers do
313
+ def parse_json_body(body)
314
+ return {} if body.nil? || body.empty?
315
+
316
+ JSON.parse(body)
317
+ end
318
+
319
+ def halt_with_error(code, message)
320
+ halt code, { "Content-Type" => "application/json" },
321
+ { error: "invalid_request", message: message }.to_json
322
+ end
323
+
324
+ def with_resource(language, want: %i[spelling])
325
+ Kotoshu.reset_spellchecker if Kotoshu.instance_variable_get(:@spellcheckers).nil?
326
+ bundle = Kotoshu::ResourceManager.resolve(language: language, want: want)
327
+ checker = Kotoshu::Spellchecker.new(resource_bundle: bundle)
328
+ yield checker, bundle
329
+ end
330
+
331
+ # Resolve the effective "model" flag for /v1/check: an explicit
332
+ # request value wins; otherwise the default is whether the
333
+ # language has a model set up server-side (cache-only). An
334
+ # explicit true on a kotoshu install without model support is
335
+ # a 503 naming the requirement, not a silent dictionary-only
336
+ # answer.
337
+ #
338
+ # @param value [Boolean, nil] the request's "model" field
339
+ # @param language [String] resolved request language
340
+ # @return [Boolean]
341
+ def model_request_flag(value, language)
342
+ case value
343
+ when nil then self.class.model_available?(language)
344
+ when true
345
+ unless self.class.semantic_models_supported?
346
+ halt 503, { "Content-Type" => "application/json" },
347
+ {
348
+ error: "model_unsupported",
349
+ message: "semantic models require kotoshu >= 0.7.0 " \
350
+ "(installed: #{Kotoshu::VERSION})"
351
+ }.to_json
352
+ end
353
+ true
354
+ when false then false
355
+ else halt_with_error(400, "'model' must be true or false")
356
+ end
357
+ end
358
+
359
+ # Rerank a traditional check result with the gem's semantic
360
+ # analyzer (dictionary verdict first, neural rerank for
361
+ # uncertain candidates — the gem's confidence cascade decides
362
+ # per word whether the ONNX rerank runs at all). Semantic
363
+ # candidates lead the merged suggestion list; traditional
364
+ # candidates follow, deduplicated by word. Any per-word
365
+ # analyzer failure keeps the traditional suggestions for that
366
+ # word — one bad word never fails the request.
367
+ #
368
+ # @param language [String] Language code
369
+ # @param result [Kotoshu::Models::Result::DocumentResult]
370
+ # @param text [String] the checked text (context source)
371
+ # @param model_info [Hash, nil] bundle.model from the resolve
372
+ # @return [Kotoshu::Models::Result::DocumentResult]
373
+ def rerank_semantic(language, result, text, model_info)
374
+ return result if model_info.nil? || model_info[:model_path].nil?
375
+ errors = Array(result.errors)
376
+ return result if errors.empty?
377
+
378
+ analyzer = self.class.semantic_analyzer_for(language, model_info)
379
+ cascade = Kotoshu::Suggestions::SemanticCascade.from_configuration(Kotoshu.configuration)
380
+
381
+ reranked = errors.map { |error| rerank_error(analyzer, cascade, text, error) }
382
+
383
+ Kotoshu::Models::Result::DocumentResult.new(
384
+ file: result.file,
385
+ errors: reranked,
386
+ suppressed_errors: result.suppressed_errors,
387
+ word_count: result.word_count
388
+ )
389
+ end
390
+
391
+ # Rerank one error's suggestions, or return the error unchanged
392
+ # when the cascade skips it or the analyzer has nothing better
393
+ # (including an analyzer failure on this word — traditional
394
+ # suggestions survive).
395
+ #
396
+ # @param analyzer [Kotoshu::Analyzers::SemanticAnalyzer]
397
+ # @param cascade [Kotoshu::Suggestions::SemanticCascade]
398
+ # @param text [String] the checked text
399
+ # @param error [Kotoshu::Models::Result::WordResult]
400
+ # @return [Kotoshu::Models::Result::WordResult]
401
+ def rerank_error(analyzer, cascade, text, error)
402
+ traditional = error.suggestions.to_a
403
+ return error if traditional.empty? || cascade.skip?(traditional)
404
+
405
+ semantic = begin
406
+ analyzer.suggest_corrections(error.word, context: semantic_context(text, error))
407
+ rescue StandardError
408
+ []
409
+ end
410
+ return error if semantic.empty?
411
+
412
+ Kotoshu::Models::Result::WordResult.new(
413
+ word: error.word,
414
+ correct: false,
415
+ position: error.position,
416
+ suggestions: merge_suggestions(semantic, traditional),
417
+ suppressed: error.suppressed,
418
+ suppressed_by: error.suppressed_by
419
+ )
420
+ end
421
+
422
+ # Merge analyzer suggestions (semantic, context-ranked) ahead
423
+ # of the traditional ones, deduplicated by word, mapped to the
424
+ # wire Suggestion shape /v1/check already serializes.
425
+ #
426
+ # @param semantic [Array<Kotoshu::Models::Suggestion>]
427
+ # @param traditional [Array<Kotoshu::Suggestions::Suggestion>]
428
+ # @return [Array<Kotoshu::Suggestions::Suggestion>]
429
+ def merge_suggestions(semantic, traditional)
430
+ merged = []
431
+ seen = {}
432
+ semantic.each do |s|
433
+ next if seen[s.word]
434
+
435
+ seen[s.word] = true
436
+ merged << Kotoshu::Suggestions::Suggestion.new(
437
+ word: s.word,
438
+ distance: s.metadata[:distance].to_i,
439
+ confidence: s.confidence,
440
+ source: :semantic
441
+ )
442
+ end
443
+ traditional.each do |s|
444
+ next if seen[s.word]
445
+
446
+ seen[s.word] = true
447
+ merged << s
448
+ end
449
+ merged
450
+ end
451
+
452
+ # The text window around an error, in the shape the analyzer's
453
+ # own context ranking expects ({Models::Context} before /
454
+ # current / after slices; nil when the error has no position).
455
+ #
456
+ # @param text [String] the checked text
457
+ # @param error [Kotoshu::Models::Result::WordResult]
458
+ # @return [Kotoshu::Models::Context, nil]
459
+ def semantic_context(text, error)
460
+ pos = error.position
461
+ return nil unless pos.is_a?(Integer)
462
+
463
+ window = 32
464
+ word_end = pos + error.word.length
465
+ Kotoshu::Models::Context.new(
466
+ before: text[[pos - window, 0].max...pos] || "",
467
+ current: text[pos...word_end] || "",
468
+ after: text[word_end...(word_end + window)] || "",
469
+ location: nil
470
+ )
471
+ end
472
+
473
+ def serialize_full(result)
474
+ {
475
+ file: result.file,
476
+ word_count: result.word_count,
477
+ errors: serialize_errors(result)
478
+ }
479
+ end
480
+
481
+ def serialize_errors(result)
482
+ (result.errors || []).map do |err|
483
+ {
484
+ word: err.word,
485
+ position: err.position,
486
+ suggestions: err.suggestions.map { |s| serialize_suggestion(s) }
487
+ }
488
+ end
489
+ end
490
+
491
+ def serialize_suggestions(set)
492
+ return [] unless set.respond_to?(:suggestions)
493
+
494
+ set.suggestions.map { |s| serialize_suggestion(s) }
495
+ end
496
+
497
+ def serialize_suggestion(s)
498
+ {
499
+ word: s.word,
500
+ distance: s.distance,
501
+ confidence: s.confidence,
502
+ source: s.source
503
+ }
504
+ end
505
+ end
506
+ end
507
+ end
508
+ end
509
+
510
+ require "time"
@@ -0,0 +1,7 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Kotoshu
4
+ module Server
5
+ VERSION = "0.1.1"
6
+ end
7
+ end
@@ -0,0 +1,27 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative "server/version"
4
+ require_relative "server/app"
5
+
6
+ module Kotoshu
7
+ module Server
8
+ # Boot the server.
9
+ #
10
+ # Pre-warm starts in a detached thread and never blocks: Puma binds
11
+ # and serves within seconds regardless of how long resource setup
12
+ # (cold-cache downloads) takes. KOTOSHU_SERVER_LAZY=1 skips the
13
+ # pre-warm entirely; /v1/health reports readiness while it runs.
14
+ #
15
+ # Semantic-model env vars (KOTOSHU_SERVER_MODEL_LANGS / _TIER) are
16
+ # validated before the bind, so an unsatisfiable model config
17
+ # fails fast with a clear error instead of surprising per-request
18
+ # failures. Unset, boot is exactly as before.
19
+ def self.run!(port: ENV.fetch("KOTOSHU_SERVER_PORT", 9292), bind: ENV.fetch("KOTOSHU_SERVER_BIND", "0.0.0.0"))
20
+ App.validate_model_config!
21
+ unless ENV.fetch("KOTOSHU_SERVER_LAZY", "0") == "1"
22
+ App.prewarm_async!(App.configured_languages)
23
+ end
24
+ App.run!(port: port.to_i, bind: bind)
25
+ end
26
+ end
27
+ end
data/openapi.yaml ADDED
@@ -0,0 +1,244 @@
1
+ openapi: 3.1.0
2
+ info:
3
+ title: Kotoshu HTTP API
4
+ version: 0.1.0
5
+ description: |
6
+ Self-hostable HTTP API wrapping the Kotoshu spell checker.
7
+ Used by the Python, JavaScript, and Go SDKs.
8
+ license:
9
+ name: BSD-2-Clause
10
+ url: https://github.com/kotoshu/kotoshu/blob/main/LICENSE
11
+ servers:
12
+ - url: http://localhost:9292
13
+ description: Local server
14
+ paths:
15
+ /:
16
+ get:
17
+ summary: Service metadata
18
+ responses:
19
+ "200":
20
+ description: OK
21
+ content:
22
+ application/json:
23
+ schema:
24
+ $ref: "#/components/schemas/Metadata"
25
+ /v1/health:
26
+ get:
27
+ summary: Liveness probe + cache state
28
+ responses:
29
+ "200":
30
+ description: OK
31
+ content:
32
+ application/json:
33
+ schema:
34
+ $ref: "#/components/schemas/Health"
35
+ /v1/version:
36
+ get:
37
+ summary: Server + kotoshu versions
38
+ responses:
39
+ "200":
40
+ description: OK
41
+ content:
42
+ application/json:
43
+ schema:
44
+ $ref: "#/components/schemas/Version"
45
+ /v1/languages:
46
+ get:
47
+ summary: Languages with cached dictionaries and model availability
48
+ responses:
49
+ "200":
50
+ description: OK
51
+ content:
52
+ application/json:
53
+ schema:
54
+ $ref: "#/components/schemas/Languages"
55
+ /v1/check:
56
+ post:
57
+ summary: Check text for spelling errors
58
+ description: |
59
+ Optional `"model": true` reranks each error's suggestions with
60
+ the semantic model set up server-side for the language
61
+ (requires KOTOSHU_SERVER_MODEL_LANGS at boot, kotoshu >= 0.7).
62
+ The default is whether a model is set up for the language.
63
+ requestBody:
64
+ required: true
65
+ content:
66
+ application/json:
67
+ schema:
68
+ $ref: "#/components/schemas/CheckRequest"
69
+ responses:
70
+ "200":
71
+ description: Result
72
+ content:
73
+ application/json:
74
+ schema:
75
+ $ref: "#/components/schemas/DocumentResult"
76
+ "400":
77
+ $ref: "#/components/responses/BadRequest"
78
+ "422":
79
+ $ref: "#/components/responses/ResourceNotSetup"
80
+ "503":
81
+ $ref: "#/components/responses/ModelUnsupported"
82
+ /v1/suggest:
83
+ post:
84
+ summary: Get suggestions for a single word
85
+ requestBody:
86
+ required: true
87
+ content:
88
+ application/json:
89
+ schema:
90
+ $ref: "#/components/schemas/SuggestRequest"
91
+ responses:
92
+ "200":
93
+ description: Suggestions
94
+ content:
95
+ application/json:
96
+ schema:
97
+ $ref: "#/components/schemas/Suggestions"
98
+ "400":
99
+ $ref: "#/components/responses/BadRequest"
100
+ /v1/detect:
101
+ post:
102
+ summary: Detect the language of text
103
+ requestBody:
104
+ required: true
105
+ content:
106
+ application/json:
107
+ schema:
108
+ $ref: "#/components/schemas/DetectRequest"
109
+ responses:
110
+ "200":
111
+ description: Detected language
112
+ content:
113
+ application/json:
114
+ schema:
115
+ $ref: "#/components/schemas/Detection"
116
+ "400":
117
+ $ref: "#/components/responses/BadRequest"
118
+ components:
119
+ responses:
120
+ BadRequest:
121
+ description: Malformed request
122
+ content:
123
+ application/json:
124
+ schema:
125
+ $ref: "#/components/schemas/Error"
126
+ ResourceNotSetup:
127
+ description: Language not set up on the server
128
+ content:
129
+ application/json:
130
+ schema:
131
+ $ref: "#/components/schemas/Error"
132
+ ModelUnsupported:
133
+ description: Model requested but the installed kotoshu gem is older than 0.7.0
134
+ content:
135
+ application/json:
136
+ schema:
137
+ $ref: "#/components/schemas/Error"
138
+ schemas:
139
+ Metadata:
140
+ type: object
141
+ required: [name, version, kotoshu_version]
142
+ properties:
143
+ name: { type: string, example: kotoshu-server }
144
+ version: { type: string, example: 0.1.0 }
145
+ kotoshu_version: { type: string, example: 0.6.0 }
146
+ docs: { type: string }
147
+ Health:
148
+ type: object
149
+ required: [status, ready]
150
+ properties:
151
+ status: { type: string, example: ok }
152
+ ready:
153
+ type: object
154
+ additionalProperties: { type: boolean }
155
+ example: { en: true }
156
+ timestamp: { type: string, format: date-time }
157
+ Version:
158
+ type: object
159
+ required: [server, kotoshu]
160
+ properties:
161
+ server: { type: string }
162
+ kotoshu: { type: string }
163
+ ruby: { type: string }
164
+ Languages:
165
+ type: object
166
+ required: [cached, model]
167
+ properties:
168
+ cached:
169
+ type: array
170
+ items: { type: string }
171
+ supported:
172
+ type: array
173
+ items: { type: string }
174
+ model:
175
+ type: object
176
+ description: Whether a semantic model is set up per cached language
177
+ additionalProperties: { type: boolean }
178
+ example: { en: true, de: false }
179
+ CheckRequest:
180
+ type: object
181
+ required: [text]
182
+ properties:
183
+ text: { type: string }
184
+ language: { type: string, default: en }
185
+ format: { type: string, enum: [full, errors], default: full }
186
+ model:
187
+ type: boolean
188
+ description: |
189
+ Rerank suggestions with the language's semantic model.
190
+ Default is whether a model is set up server-side. true on a
191
+ language without a cached model returns 422.
192
+ SuggestRequest:
193
+ type: object
194
+ required: [word]
195
+ properties:
196
+ word: { type: string }
197
+ language: { type: string, default: en }
198
+ max: { type: integer, minimum: 1, maximum: 50 }
199
+ DetectRequest:
200
+ type: object
201
+ required: [text]
202
+ properties:
203
+ text: { type: string }
204
+ DocumentResult:
205
+ type: object
206
+ properties:
207
+ file: { type: string, nullable: true }
208
+ word_count: { type: integer }
209
+ errors:
210
+ type: array
211
+ items: { $ref: "#/components/schemas/WordError" }
212
+ WordError:
213
+ type: object
214
+ properties:
215
+ word: { type: string }
216
+ position: { type: integer, nullable: true }
217
+ suggestions:
218
+ type: array
219
+ items: { $ref: "#/components/schemas/Suggestion" }
220
+ Suggestions:
221
+ type: object
222
+ properties:
223
+ word: { type: string }
224
+ suggestions:
225
+ type: array
226
+ items: { $ref: "#/components/schemas/Suggestion" }
227
+ Suggestion:
228
+ type: object
229
+ properties:
230
+ word: { type: string }
231
+ distance: { type: integer }
232
+ confidence: { type: number, format: float }
233
+ source: { type: string }
234
+ Detection:
235
+ type: object
236
+ properties:
237
+ language: { type: string, nullable: true }
238
+ confidence: { type: number, format: float }
239
+ Error:
240
+ type: object
241
+ properties:
242
+ error: { type: string }
243
+ message: { type: string }
244
+ hint: { type: string }
metadata CHANGED
@@ -1,13 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: kotoshu-server
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ribose Inc.
8
+ autorequire:
8
9
  bindir: exe
9
10
  cert_chain: []
10
- date: 1980-01-02 00:00:00.000000000 Z
11
+ date: 2026-09-06 00:00:00.000000000 Z
11
12
  dependencies:
12
13
  - !ruby/object:Gem::Dependency
13
14
  name: kotoshu
@@ -69,10 +70,18 @@ description: Self-hostable HTTP API that exposes Kotoshu's check / suggest / det
69
70
  over JSON. Designed as the deployment surface for non-Ruby SDKs (Python, JS, Go).
70
71
  email:
71
72
  - open.source@ribose.com
72
- executables: []
73
+ executables:
74
+ - kotoshu-server
73
75
  extensions: []
74
76
  extra_rdoc_files: []
75
- files: []
77
+ files:
78
+ - LICENSE
79
+ - README.md
80
+ - exe/kotoshu-server
81
+ - lib/kotoshu/server.rb
82
+ - lib/kotoshu/server/app.rb
83
+ - lib/kotoshu/server/version.rb
84
+ - openapi.yaml
76
85
  homepage: https://github.com/kotoshu/kotoshu-server
77
86
  licenses:
78
87
  - BSD-2-Clause
@@ -80,6 +89,7 @@ metadata:
80
89
  homepage_uri: https://github.com/kotoshu/kotoshu-server
81
90
  source_code_uri: https://github.com/kotoshu/kotoshu-server/tree/main
82
91
  rubygems_mfa_required: 'true'
92
+ post_install_message:
83
93
  rdoc_options: []
84
94
  require_paths:
85
95
  - lib
@@ -94,7 +104,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
94
104
  - !ruby/object:Gem::Version
95
105
  version: '0'
96
106
  requirements: []
97
- rubygems_version: 4.0.16
107
+ rubygems_version: 3.5.22
108
+ signing_key:
98
109
  specification_version: 4
99
110
  summary: HTTP API server wrapping the Kotoshu spell checker
100
111
  test_files: []