kotoshu 0.9.2 → 0.9.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +24 -0
- data/TODO.impl/00-vision.md +22 -0
- data/TODO.impl/100-full-feature-batch3.md +28 -0
- data/TODO.impl/101-int4-tier-experiment.md +32 -0
- data/TODO.impl/102-lid-chain.md +28 -0
- data/TODO.impl/103-oov-bucket-tables.md +27 -0
- data/TODO.impl/104-vscode-marketplace-readiness.md +32 -0
- data/TODO.impl/99-site-truth-performance-docs.md +31 -0
- data/lib/kotoshu/cache/language_cache.rb +4 -1
- data/lib/kotoshu/keyboard/layouts/arabic.rb +59 -0
- data/lib/kotoshu/keyboard/layouts/bulgarian.rb +57 -0
- data/lib/kotoshu/keyboard/layouts/hebrew.rb +57 -0
- data/lib/kotoshu/keyboard/layouts/hrsl.rb +78 -0
- data/lib/kotoshu/keyboard/layouts/latin.rb +47 -1
- data/lib/kotoshu/keyboard/layouts/persian.rb +58 -0
- data/lib/kotoshu/keyboard/layouts/serbian.rb +59 -0
- data/lib/kotoshu/keyboard/registry.rb +20 -4
- data/lib/kotoshu/keyboard.rb +9 -0
- data/lib/kotoshu/languages/ar/language.rb +14 -0
- data/lib/kotoshu/languages/bg/language.rb +37 -0
- data/lib/kotoshu/languages/et/language.rb +24 -0
- data/lib/kotoshu/languages/fa/language.rb +16 -0
- data/lib/kotoshu/languages/he/language.rb +15 -0
- data/lib/kotoshu/languages/hr/language.rb +24 -0
- data/lib/kotoshu/languages/id/language.rb +25 -0
- data/lib/kotoshu/languages/lt/language.rb +24 -0
- data/lib/kotoshu/languages/lv/language.rb +24 -0
- data/lib/kotoshu/languages/sk/language.rb +25 -0
- data/lib/kotoshu/languages/sl/language.rb +25 -0
- data/lib/kotoshu/languages/sr/language.rb +38 -0
- data/lib/kotoshu/languages.rb +12 -0
- data/lib/kotoshu/version.rb +1 -1
- data/scripts/extract_eval_grids.rb +4 -1
- data/sig/kotoshu.rbs +24 -0
- metadata +22 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4891dc67bd85b5c342fdbc7453085990a1e9ff9bea5b394a67f87e11b4bbd6c7
|
|
4
|
+
data.tar.gz: 378d1b77bea8113d155774bc07e481afaa9229d6e1edcd7d5fda2805d28a963c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: bebc7b8be31a4f9f0127bddf2cb4a8de23c9908b05ddf91f51a43af5927af4608a0398090cce27deefce0e5c86f4a7d9edb0ffdf6b122f849bb065cfcee44e2d
|
|
7
|
+
data.tar.gz: 1240bb5e7d513c4d3b3dd936042c3d177cbad6aebc402fc6ef91f86e3163bc0ba2dcf516f93d7658171d2c952c1f2fed6162928c7a50bdef3e5703233acce1ce
|
data/CHANGELOG.md
CHANGED
|
@@ -7,7 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
### Added
|
|
11
|
+
- **Full-feature language batch 3** (plan 100) - twelve languages gain the
|
|
12
|
+
downloadable spelling dictionary, a keyboard layout, and a gem module:
|
|
13
|
+
`ar` `id` `fa` `he` `bg` `sr` `hr` `sk` `sl` `lt` `lv` `et` (the RTL
|
|
14
|
+
ar/fa/he modules are promoted from tokenizer-only wiring; the other nine
|
|
15
|
+
are new thin modules - id hr sl sk et lt lv on `LatinBase`, bg sr with
|
|
16
|
+
the Cyrillic tokenizer). Six national key grids join the layout registry
|
|
17
|
+
(Arabic 101, Persian standard ISIRI 9147, Hebrew SI-1452, Bulgarian BDS,
|
|
18
|
+
Serbian Cyrillic, and the shared Croatian/Slovenian QWERTZ), all mirrored
|
|
19
|
+
from the models repo eval harness and drift-checked against it; five new
|
|
20
|
+
Latin family members (Indonesian Slovak Estonian Lithuanian Latvian)
|
|
21
|
+
type on the US QWERTY grid like the eval model. Bulgarian and Serbian
|
|
22
|
+
register ahead of JCUKEN so their Cyrillic grids win the lookup. The
|
|
23
|
+
eval-grid drift fixture now covers all nine national grids. Specimen
|
|
24
|
+
misspelling -> correction pairs for every language are engine-verified
|
|
25
|
+
against the staged dictionaries in a `:network` spec.
|
|
26
|
+
|
|
10
27
|
### Fixed
|
|
28
|
+
- **RTL word extraction on the check path** - the plan-91 script-aware
|
|
29
|
+
word regex never reached the Arabic, Persian and Hebrew tokenizers, so
|
|
30
|
+
`Kotoshu.check` / `kotoshu check -l ar|fa|he` extracted zero words and
|
|
31
|
+
silently passed every RTL document. The three tokenizers now declare
|
|
32
|
+
their script sets (`\p{Arabic}`, `\p{Arabic}` + ZWNJ for Persian
|
|
33
|
+
compounds like میروم, `\p{Hebrew}` + geresh/gershayim), and RTL
|
|
34
|
+
misspellings surface with ranked suggestions.
|
|
11
35
|
- **Suggestion candidate sweep missed transpositions, substitutions, and
|
|
12
36
|
dictionary forms** - `EditDistanceStrategy` only scored raw dictionary
|
|
13
37
|
stems, so valid dictionary forms could never be suggested no matter how
|
data/TODO.impl/00-vision.md
CHANGED
|
@@ -188,6 +188,28 @@ convertible model pool exhausted at 55 languages; owner-side
|
|
|
188
188
|
remnants (PyPI registrations, marketplace PAT, nds/fi calls) recorded
|
|
189
189
|
in the plans, not pending work.
|
|
190
190
|
|
|
191
|
+
Plans 99-104 are **wave 6, the live-quality follow-through**
|
|
192
|
+
(2026-09-07), written after the sweep-performance arc (gem 0.9.1 ->
|
|
193
|
+
0.9.2, @kotoshu/wasm 0.3.0 -> 0.3.2) and a fresh state audit: 55 model
|
|
194
|
+
languages x 3 tiers confirmed (ja/ko/zh rerank-only by design), no LID
|
|
195
|
+
resource mirrored, lsp/server `~> 0.6` constraints already resolving
|
|
196
|
+
current engines, site docs carrying stale version references.
|
|
197
|
+
|
|
198
|
+
| # | Plan | Gap it closes |
|
|
199
|
+
|---|---|---|
|
|
200
|
+
| 99 | [Site truth + performance docs](99-site-truth-performance-docs.md) | Docs misstate current versions; the measured per-language latency is unpublished |
|
|
201
|
+
| 100 | [Full-feature batch 3](100-full-feature-batch3.md) | ~35 languages have dictionaries + models but no gem module - the big-audience remainder (ar id fa he ...) |
|
|
202
|
+
| 101 | [int4 tier experiment](101-int4-tier-experiment.md) | Halving tier bytes was named (B1) but never run through the gates |
|
|
203
|
+
| 102 | [LID chain](102-lid-chain.md) | No detection in the registry, wasm, or playground - users pick the language blind |
|
|
204
|
+
| 103 | [OOV bucket tables](103-oov-bucket-tables.md) | Model candidates miss n-grams absent from vocab (the Teh gap's model half) |
|
|
205
|
+
| 104 | [VS Code marketplace readiness](104-vscode-marketplace-readiness.md) | The extension is built but not installable; everything short of the owner PAT |
|
|
206
|
+
|
|
207
|
+
Execution notes: plan 103 is sequenced after 101 (same repo, one
|
|
208
|
+
worktree at a time); all site-facing flips (99, 100, 102's playground
|
|
209
|
+
half, 104's link line) land through one coordinated site pass to keep
|
|
210
|
+
the repo conflict-free; lsp/server 0.1.2 bumps are deliberately NOT
|
|
211
|
+
planned - the `~> 0.6` floor already resolves 0.9.2 for fresh installs.
|
|
212
|
+
|
|
191
213
|
Dependency order across the ecosystem track:
|
|
192
214
|
|
|
193
215
|
```
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Plan 100 — Full-feature language batch 3: the highest-audience remainder
|
|
2
|
+
|
|
3
|
+
## Why
|
|
4
|
+
20 languages are full-feature (gem module + keyboard layout + verified
|
|
5
|
+
specimens); the dictionaries manifest and the model registry carry ~35 more
|
|
6
|
+
with both resources. Every remaining language without a module leaves
|
|
7
|
+
speakers on the degraded path. The gaps are the big-audience ones: ar (~400M),
|
|
8
|
+
id (~200M), fa (~80M), he, bg, sr, hr, sk, sl, and more.
|
|
9
|
+
|
|
10
|
+
## Work (gem repo only — the site flip rides a follow-up)
|
|
11
|
+
1. Enumerate: languages present in BOTH the dictionaries manifest and the
|
|
12
|
+
models registry that lack a gem module. Rank by speaker count.
|
|
13
|
+
2. For the top ~12 (expected: ar id fa he bg sr hr sk sl lt lv et — confirm
|
|
14
|
+
against data): create the language module (pattern of the 19 from PR
|
|
15
|
+
#123), keyboard layout (Arabic 101 for ar; Persian for fa; Hebrew for he;
|
|
16
|
+
Cyrillic/YUSI for bg/sr; QWERTZ/Slovak for the Latinate), AVAILABLE_LANGUAGES
|
|
17
|
+
extension, and specimen pairs verified against the STAGED dictionary via
|
|
18
|
+
the real engine (correct()/suggest() round trips) — the PR #123 discipline.
|
|
19
|
+
3. RTL already wired (ar/id from batch 2); no tokenizer work needed.
|
|
20
|
+
4. Suite green; rubocop clean.
|
|
21
|
+
|
|
22
|
+
## Verification
|
|
23
|
+
One spec per module (specimen corrections engine-verified, layout smoke);
|
|
24
|
+
`kotoshu setup <lang>` + `check -l <lang>` exercised live for at least ar,
|
|
25
|
+
id, fa, he. Deliver the verified specimen table + sizes for the site flip.
|
|
26
|
+
|
|
27
|
+
## Status
|
|
28
|
+
Executed 2026-09-07 — gem PR #151 merged. Twelve languages (ar id fa he bg sr hr sk sl lt lv et) full-feature; AVAILABLE_LANGUAGES 20→32; 49 specimens; RTL tokenizer fix for ar/fa/he. Site flip PR #27.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Plan 101 — int4 tier experiment (models repo), eval-gated
|
|
2
|
+
|
|
3
|
+
## Why
|
|
4
|
+
The tier recipe settled at int8 (rank_corr 0.9999, near-lossless) after SVD
|
|
5
|
+
failed its gates. int4 halves the bytes again — mini ~1.5 MB, fluency ~8 MB —
|
|
6
|
+
which changes browser adoption math for the wasm playground. It was named in
|
|
7
|
+
the original kotoshu-rs TODO (B1) but never run through the gates.
|
|
8
|
+
|
|
9
|
+
## Work (models repo)
|
|
10
|
+
1. Quantization pass: per-row int4 (packed nibbles) + fp16 scale, over the
|
|
11
|
+
same vocab cuts as fluency/mini.
|
|
12
|
+
2. The existing eval gates unchanged and unweakened: rank correlation vs
|
|
13
|
+
full float32 on the keyboard-aware eval; accept only if rank_corr within
|
|
14
|
+
0.001 of int8's and top-1 agreement holds on the probe-limited languages.
|
|
15
|
+
3. If gates pass: registry v1.4.0 adding the int4 variants as NEW resources
|
|
16
|
+
(`{lang}/mini4`? or a field) — additive, never replacing; mirror with
|
|
17
|
+
ACAO:* like v1.2.1+. If gates fail: record the numbers as the verdict
|
|
18
|
+
(the SVD precedent — measured, not assumed).
|
|
19
|
+
4. ONNX reader side (kotoshu-rs) only if the gates pass.
|
|
20
|
+
|
|
21
|
+
## Verification
|
|
22
|
+
Gates table per language; byte sizes; the release registry byte-identical to
|
|
23
|
+
the committed one; no existing tier URLs change.
|
|
24
|
+
|
|
25
|
+
## Status
|
|
26
|
+
Executed 2026-09-07 — REJECTED on data (models PR #17, merged). int4-per-row
|
|
27
|
+
(one fp16 scale, packed nibbles) over 8 languages: rank_corr deltas -0.0146
|
|
28
|
+
to -0.0199 vs int8 (15-20x outside the 0.001 window), top-1 fails on 8/8
|
|
29
|
+
fluency4 and 2/8 mini4. Sizes halve (mini4 1.52 MB, fluency4 7.60 MB) but
|
|
30
|
+
accuracy does not hold. With the earlier group-128/64/32 sweep the scale
|
|
31
|
+
ladder is measured end to end — int8-per-row stands final. Registry
|
|
32
|
+
untouched; evidence in models eval/reports/.
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Plan 102 — Language detection end to end: registry mirror, wasm, playground
|
|
2
|
+
|
|
3
|
+
## Why
|
|
4
|
+
The gem detects document language (server /v1/detect) using its own detection
|
|
5
|
+
model — but the registry mirrors none of it, the wasm surface has no detect
|
|
6
|
+
API, and the playground makes a user pick a language dropdown blind. A
|
|
7
|
+
writer pasting text should be told what it is. The FastText LID model is
|
|
8
|
+
tiny (~1 MB) and the mirror pattern is proven (v1.2.1+).
|
|
9
|
+
|
|
10
|
+
## Work
|
|
11
|
+
1. Models repo: add the LID model as a registry resource (converted like the
|
|
12
|
+
tiers, int8-quantized if gates hold vs the float32 labels), mirrored with
|
|
13
|
+
ACAO:*; registry v1.4.x alongside plan 101 (coordinate the version).
|
|
14
|
+
2. kotoshu-rs: `detect_language(bytes, vocab_bytes) -> {code, score}` on the
|
|
15
|
+
model surface beside loadModel/rerank; wasm export `detectLanguage`;
|
|
16
|
+
tests against the gem's detection outputs on a fixed multilingual corpus.
|
|
17
|
+
3. Playground (rides the site follow-up with plans 99/100 flips): on load
|
|
18
|
+
and on demand ("detect" affordance beside the lang select), load the LID
|
|
19
|
+
model (~1 MB, cached like the semantic tier) and set/select the language,
|
|
20
|
+
showing the detected code + confidence. Never auto-switch a user's
|
|
21
|
+
explicit choice silently — propose it.
|
|
22
|
+
|
|
23
|
+
## Verification
|
|
24
|
+
Registry mirror ACAO + size checks; wasm parity vs gem detect on the corpus;
|
|
25
|
+
playground verified with multilingual paste (en/de/ru/ja/ar).
|
|
26
|
+
|
|
27
|
+
## Status
|
|
28
|
+
Executed 2026-09-07 (core) — models PR #18 (lid-176, registry v1.4.0) + kotoshu-rs PR #25 (loadLid/detectLanguage). Playground propose-UI still pending a wasm cut that ships the export.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Plan 103 — OOV bucket-table artifacts: model-generated candidates for the last mile
|
|
2
|
+
|
|
3
|
+
## Why
|
|
4
|
+
semanticSuggest embeds an OOV word via subword n-grams PRESENT IN THE VOCAB;
|
|
5
|
+
a typo whose n-grams never appear (Teh — only n-gram "teh") embeds nothing
|
|
6
|
+
and returns empty. fastText's real OOV path hashes n-grams into buckets the
|
|
7
|
+
training wrote to; exporting the bucket table closes that gap so the model,
|
|
8
|
+
not just the Damerau sweep, can reach the intended word. Recorded as the
|
|
9
|
+
honest gap in kotoshu-rs oov.rs docs and PR #18's report.
|
|
10
|
+
|
|
11
|
+
## Work (models repo, then kotoshu-rs)
|
|
12
|
+
1. Export the bucket-table rows the FastText training produced (hash bucket
|
|
13
|
+
→ vector) for each tier alongside the vocab; size audit first — the
|
|
14
|
+
bucket table may exceed the tier itself; if so, gate to the top-K buckets
|
|
15
|
+
by usage and measure the quality delta honestly.
|
|
16
|
+
2. kotoshu-rs: extend the int8 reader to consult bucket rows for n-grams
|
|
17
|
+
absent from vocab (feature-gated like `model`); semantic_neighbors then
|
|
18
|
+
covers bucket-backed OOV.
|
|
19
|
+
3. Gates: existing rerank conformance must not move (frozen); new specs for
|
|
20
|
+
bucket-backed neighbors (Teh→the expected); eval on the rerank probe set.
|
|
21
|
+
|
|
22
|
+
## Verification
|
|
23
|
+
Bucket-backed neighbor specs; size deltas recorded; rerank conformance 0
|
|
24
|
+
change; a 0.4.x wasm release only after both engines agree.
|
|
25
|
+
|
|
26
|
+
## Status
|
|
27
|
+
Executed 2026-09-07 — models PR #20 (en/de buckets) + kotoshu-rs PR #26 (bucket-backed OOV). Teh→the via buckets. Release assets + wasm 0.4.0 pin still owner/coordinated cut. (sequenced AFTER plan 101 — same repo, one worktree at a time)
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Plan 104 — VS Code marketplace + Open VSX readiness
|
|
2
|
+
|
|
3
|
+
## Why
|
|
4
|
+
The extension is built, CI-green, with a .vsix artifact — but it is not
|
|
5
|
+
installable without the marketplace listing, which is the single largest
|
|
6
|
+
adoption blocker for the editor audience. Publishing needs the owner's
|
|
7
|
+
publisher PAT (a credential we never hold); everything else can be ready
|
|
8
|
+
so publishing is one secret + one click.
|
|
9
|
+
|
|
10
|
+
## Work (vscode repo)
|
|
11
|
+
1. Open VSX publish workflow (release-ovsx.yml, secret-gated) beside the
|
|
12
|
+
existing CI — Open VSX serves VSCodium/Code-OSS users without the
|
|
13
|
+
Microsoft marketplace.
|
|
14
|
+
2. Metadata polish: gallery banner/colors, keywords, categories, repository
|
|
15
|
+
links, the kotoshu-vscode README (quickstart with kotoshu-lsp install,
|
|
16
|
+
settings reference: log level, personal dictionary command).
|
|
17
|
+
3. PREPUBLISH.md: exact owner steps — create publisher, add PAT as secret,
|
|
18
|
+
run the workflow; nothing else.
|
|
19
|
+
4. Site follow-up line (rides plan 99's pass): /docs/clients/lsp gains the
|
|
20
|
+
marketplace link placeholder to fill on publish day.
|
|
21
|
+
|
|
22
|
+
## Verification
|
|
23
|
+
`vsce package` clean with no warnings; workflow lint-valid; PREPUBLISH
|
|
24
|
+
steps verified against the actual GitHub UI labels.
|
|
25
|
+
|
|
26
|
+
## Status
|
|
27
|
+
Executed 2026-09-07 — vscode PR #3, merged. release-marketplace.yml (VSCE_PAT)
|
|
28
|
+
+ release-ovsx.yml (OPEN_VSX_TOKEN), both version-gated; PREPUBLISH.md with
|
|
29
|
+
verified owner steps; package.json publisher/categories/keywords/banner;
|
|
30
|
+
README quickstart against gem 0.1.1; vsce package warning-clean. Publishing
|
|
31
|
+
itself is one PAT secret + one tag — owner action. The /docs/clients/lsp
|
|
32
|
+
marketplace link rides the coordinated site pass.
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Plan 99 — Site truth pass + measured performance documentation
|
|
2
|
+
|
|
3
|
+
## Why
|
|
4
|
+
The docs were written against the versions live at the time; three releases
|
|
5
|
+
later some pages misstate reality. Users deserve the measured numbers the
|
|
6
|
+
perf work produced — no spellchecker site publishes per-language sweep
|
|
7
|
+
latency; we have it, verified, for eight languages.
|
|
8
|
+
|
|
9
|
+
## Work (site repo)
|
|
10
|
+
1. `/docs/performance` (new page): the measured tables — whole-text check
|
|
11
|
+
latency (1-33 ms per 100 words), per-sweep averages post-0.3.2
|
|
12
|
+
(en 45 ms ... pt 423 ms, fr worst-case note), tier sizes (full/fluency/mini),
|
|
13
|
+
and tuning: personal dictionaries, baselines, tier choice guidance
|
|
14
|
+
(`--tier`), backend choice (ruby vs native vs wasm). Link from the
|
|
15
|
+
playground footnote, /languages, and /install.
|
|
16
|
+
2. Truth pass: /docs/clients/action still says the action "installs the gem
|
|
17
|
+
from RubyGems where 0.7.0 gates six languages" — stale; the action now
|
|
18
|
+
rides 0.9.2 (20 full-feature languages, baselines, directory mode).
|
|
19
|
+
Audit every page for version references (0.7.0/0.8.0/0.1.1/wasm 0.2.0)
|
|
20
|
+
and flip to current-with-dates. /playground/server: verify the hosted
|
|
21
|
+
demo state and the run-your-own instructions.
|
|
22
|
+
3. Footer/readout: confirm every registry badge resolves (RubyGems 0.9.2,
|
|
23
|
+
npm 0.3.2, PyPI, crates.io).
|
|
24
|
+
|
|
25
|
+
## Verification
|
|
26
|
+
Rendered text extraction on the built site for stale version strings;
|
|
27
|
+
every measured number in /docs/performance traceable to the 0.3.2 bench
|
|
28
|
+
harness (kotoshu-rs PR #23 table) — no inferred numbers.
|
|
29
|
+
|
|
30
|
+
## Status
|
|
31
|
+
Executed 2026-09-07 — site PR #26: /docs/performance + truth pass. Action gap found and closed via action-kotoshu PR #2 + site PR #27 docs flip.
|
|
@@ -31,8 +31,11 @@ module Kotoshu
|
|
|
31
31
|
# modules (lib/kotoshu/languages/). Staged-only dictionaries in
|
|
32
32
|
# the dictionaries repo stay outside this list until their
|
|
33
33
|
# module lands; local files come via setup --aff/--dic instead.
|
|
34
|
+
# Batch 3 (plan 100) promotes the RTL wiring ar/fa/he and adds
|
|
35
|
+
# bg et hr id lt lv sk sl sr.
|
|
34
36
|
AVAILABLE_LANGUAGES = %w[
|
|
35
|
-
ca cs da de el en es
|
|
37
|
+
ar bg ca cs da de el en es et fa fr he hr hu id it lt lv nb nl
|
|
38
|
+
pl pt ro ru sk sl sr sv tr uk vi
|
|
36
39
|
].freeze
|
|
37
40
|
|
|
38
41
|
# Get or download spelling dictionary for a language.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Keyboard
|
|
5
|
+
module Layouts
|
|
6
|
+
# Arabic 101 keyboard layout (standard IBM PC Arabic layout).
|
|
7
|
+
#
|
|
8
|
+
# The Arabic letters sit on the QWERTY key faces: the number row
|
|
9
|
+
# is shared with ASCII, and ض ص ث ق ف غ ع ه خ ح ج د / ش س ي ب ل ا
|
|
10
|
+
# ت ن م ك ط / ئ ء ؤ ر لا ى ة و ز ظ fill the three letter rows.
|
|
11
|
+
# Diacritics (tashkeel) are dead keys and are not in the grid;
|
|
12
|
+
# the hamza carriers أ إ آ and the ه/ة ي/ى spelling slips are
|
|
13
|
+
# handled at the normalizer level, not as physical keys.
|
|
14
|
+
#
|
|
15
|
+
# The key grid is mirrored from the models repo eval harness
|
|
16
|
+
# (kotoshu/models-fasttext-onnx eval/noise.py _AR_101, plan 83
|
|
17
|
+
# batch 2 of the model coverage expansion, verified there
|
|
18
|
+
# against the kbdlayout.info Arabic 101 driver table) so
|
|
19
|
+
# typo-adjacency used for suggestion ranking matches the
|
|
20
|
+
# adjacency used to evaluate the models.
|
|
21
|
+
# spec/kotoshu/keyboard/layouts/eval_grid_drift_spec.rb enforces
|
|
22
|
+
# the sync.
|
|
23
|
+
#
|
|
24
|
+
# Languages: ar, ar-SA
|
|
25
|
+
class Arabic101 < Layout
|
|
26
|
+
# Key positions for the Arabic 101 layout.
|
|
27
|
+
# Each key maps to [row, column] coordinates.
|
|
28
|
+
KEY_POSITIONS = {
|
|
29
|
+
# Number row
|
|
30
|
+
'ذ' => [0, 0], '1' => [0, 1], '2' => [0, 2], '3' => [0, 3],
|
|
31
|
+
'4' => [0, 4], '5' => [0, 5], '6' => [0, 6], '7' => [0, 7],
|
|
32
|
+
'8' => [0, 8], '9' => [0, 9], '0' => [0, 10], '-' => [0, 11],
|
|
33
|
+
'=' => [0, 12],
|
|
34
|
+
# Top row (ض ص ث ق ف غ ع ه خ ح ج د)
|
|
35
|
+
'ض' => [1, 0], 'ص' => [1, 1], 'ث' => [1, 2], 'ق' => [1, 3],
|
|
36
|
+
'ف' => [1, 4], 'غ' => [1, 5], 'ع' => [1, 6], 'ه' => [1, 7],
|
|
37
|
+
'خ' => [1, 8], 'ح' => [1, 9], 'ج' => [1, 10], 'د' => [1, 11],
|
|
38
|
+
# Home row (ش س ي ب ل ا ت ن م ك ط)
|
|
39
|
+
'ش' => [2, 0], 'س' => [2, 1], 'ي' => [2, 2], 'ب' => [2, 3],
|
|
40
|
+
'ل' => [2, 4], 'ا' => [2, 5], 'ت' => [2, 6], 'ن' => [2, 7],
|
|
41
|
+
'م' => [2, 8], 'ك' => [2, 9], 'ط' => [2, 10],
|
|
42
|
+
# Bottom row (ئ ء ؤ ر لا ى ة و ز ظ)
|
|
43
|
+
'ئ' => [3, 0], 'ء' => [3, 1], 'ؤ' => [3, 2], 'ر' => [3, 3],
|
|
44
|
+
'لا' => [3, 4], 'ى' => [3, 5], 'ة' => [3, 6], 'و' => [3, 7],
|
|
45
|
+
'ز' => [3, 8], 'ظ' => [3, 9]
|
|
46
|
+
}.freeze
|
|
47
|
+
|
|
48
|
+
# Initialize the Arabic 101 layout.
|
|
49
|
+
def initialize
|
|
50
|
+
super(
|
|
51
|
+
name: 'Arabic-101',
|
|
52
|
+
language_codes: %w[ar ar-SA],
|
|
53
|
+
key_positions: KEY_POSITIONS
|
|
54
|
+
)
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Keyboard
|
|
5
|
+
module Layouts
|
|
6
|
+
# Bulgarian BDS keyboard layout (БДС 5237:1978, KBDBUL.DLL).
|
|
7
|
+
#
|
|
8
|
+
# The national Bulgarian phonetic-adjacent ordering: ы у е и ш щ
|
|
9
|
+
# к с д з ц / ь я а о ж г т н в м ч / ю й ъ э ф х п р л б. The
|
|
10
|
+
# Bulgarian-only letters ъ ь are real keys (unlike JCUKEN, where
|
|
11
|
+
# ъ sits top-right and ы is the home-row opener).
|
|
12
|
+
#
|
|
13
|
+
# The key grid is mirrored from the models repo eval harness
|
|
14
|
+
# (kotoshu/models-fasttext-onnx eval/noise.py _BG_BDS, plan 83
|
|
15
|
+
# batch 2 of the model coverage expansion, verified there
|
|
16
|
+
# against the kbdlayout.info KBDBUL driver table) so
|
|
17
|
+
# typo-adjacency used for suggestion ranking matches the
|
|
18
|
+
# adjacency used to evaluate the models.
|
|
19
|
+
# spec/kotoshu/keyboard/layouts/eval_grid_drift_spec.rb enforces
|
|
20
|
+
# the sync.
|
|
21
|
+
#
|
|
22
|
+
# Languages: bg, bg-BG
|
|
23
|
+
class BulgarianBds < Layout
|
|
24
|
+
# Key positions for the Bulgarian BDS layout.
|
|
25
|
+
# Each key maps to [row, column] coordinates.
|
|
26
|
+
KEY_POSITIONS = {
|
|
27
|
+
# Number row
|
|
28
|
+
')' => [0, 0], '1' => [0, 1], '2' => [0, 2], '3' => [0, 3],
|
|
29
|
+
'4' => [0, 4], '5' => [0, 5], '6' => [0, 6], '7' => [0, 7],
|
|
30
|
+
'8' => [0, 8], '9' => [0, 9], '0' => [0, 10], '$' => [0, 11],
|
|
31
|
+
'€' => [0, 12],
|
|
32
|
+
# Top row (ы у е и ш щ к с д з ц)
|
|
33
|
+
'ы' => [1, 0], 'у' => [1, 1], 'е' => [1, 2], 'и' => [1, 3],
|
|
34
|
+
'ш' => [1, 4], 'щ' => [1, 5], 'к' => [1, 6], 'с' => [1, 7],
|
|
35
|
+
'д' => [1, 8], 'з' => [1, 9], 'ц' => [1, 10],
|
|
36
|
+
# Home row (ь я а о ж г т н в м ч)
|
|
37
|
+
'ь' => [2, 0], 'я' => [2, 1], 'а' => [2, 2], 'о' => [2, 3],
|
|
38
|
+
'ж' => [2, 4], 'г' => [2, 5], 'т' => [2, 6], 'н' => [2, 7],
|
|
39
|
+
'в' => [2, 8], 'м' => [2, 9], 'ч' => [2, 10],
|
|
40
|
+
# Bottom row (ю й ъ э ф х п р л б)
|
|
41
|
+
'ю' => [3, 0], 'й' => [3, 1], 'ъ' => [3, 2], 'э' => [3, 3],
|
|
42
|
+
'ф' => [3, 4], 'х' => [3, 5], 'п' => [3, 6], 'р' => [3, 7],
|
|
43
|
+
'л' => [3, 8], 'б' => [3, 9]
|
|
44
|
+
}.freeze
|
|
45
|
+
|
|
46
|
+
# Initialize the Bulgarian BDS layout.
|
|
47
|
+
def initialize
|
|
48
|
+
super(
|
|
49
|
+
name: 'Bulgarian-BDS',
|
|
50
|
+
language_codes: %w[bg bg-BG],
|
|
51
|
+
key_positions: KEY_POSITIONS
|
|
52
|
+
)
|
|
53
|
+
end
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Keyboard
|
|
5
|
+
module Layouts
|
|
6
|
+
# Hebrew standard keyboard layout (SI-1452).
|
|
7
|
+
#
|
|
8
|
+
# The 27 Hebrew letters all sit on key faces in the standard
|
|
9
|
+
# ordering, including the five final-letter forms ם ן ך ף ץ as
|
|
10
|
+
# real keys. Niqqud (vowel points) are dead keys and are not in
|
|
11
|
+
# the grid; geresh/gershayim are shift states, not base keys.
|
|
12
|
+
#
|
|
13
|
+
# The key grid is mirrored from the models repo eval harness
|
|
14
|
+
# (kotoshu/models-fasttext-onnx eval/noise.py _HE_SI1452, plan 83
|
|
15
|
+
# batch 2 of the model coverage expansion, verified there
|
|
16
|
+
# against the Culmus SI-1452 layout table) so typo-adjacency
|
|
17
|
+
# used for suggestion ranking matches the adjacency used to
|
|
18
|
+
# evaluate the models.
|
|
19
|
+
# spec/kotoshu/keyboard/layouts/eval_grid_drift_spec.rb enforces
|
|
20
|
+
# the sync.
|
|
21
|
+
#
|
|
22
|
+
# Languages: he, he-IL
|
|
23
|
+
class HebrewSI1452 < Layout
|
|
24
|
+
# Key positions for the Hebrew SI-1452 layout.
|
|
25
|
+
# Each key maps to [row, column] coordinates.
|
|
26
|
+
KEY_POSITIONS = {
|
|
27
|
+
# Number row
|
|
28
|
+
'`' => [0, 0], '1' => [0, 1], '2' => [0, 2], '3' => [0, 3],
|
|
29
|
+
'4' => [0, 4], '5' => [0, 5], '6' => [0, 6], '7' => [0, 7],
|
|
30
|
+
'8' => [0, 8], '9' => [0, 9], '0' => [0, 10], '-' => [0, 11],
|
|
31
|
+
'=' => [0, 12],
|
|
32
|
+
# Top row (ק ר א ט ו ן ם פ)
|
|
33
|
+
'/' => [1, 0], "'" => [1, 1], 'ק' => [1, 2], 'ר' => [1, 3],
|
|
34
|
+
'א' => [1, 4], 'ט' => [1, 5], 'ו' => [1, 6], 'ן' => [1, 7],
|
|
35
|
+
'ם' => [1, 8], 'פ' => [1, 9], '[' => [1, 10], ']' => [1, 11],
|
|
36
|
+
# Home row (ש ד ג כ ע י ח ל ך ף)
|
|
37
|
+
'ש' => [2, 0], 'ד' => [2, 1], 'ג' => [2, 2], 'כ' => [2, 3],
|
|
38
|
+
'ע' => [2, 4], 'י' => [2, 5], 'ח' => [2, 6], 'ל' => [2, 7],
|
|
39
|
+
'ך' => [2, 8], 'ף' => [2, 9], ',' => [2, 10],
|
|
40
|
+
# Bottom row (ז ס ב ה נ מ צ ת ץ)
|
|
41
|
+
'ז' => [3, 0], 'ס' => [3, 1], 'ב' => [3, 2], 'ה' => [3, 3],
|
|
42
|
+
'נ' => [3, 4], 'מ' => [3, 5], 'צ' => [3, 6], 'ת' => [3, 7],
|
|
43
|
+
'ץ' => [3, 8], '.' => [3, 9]
|
|
44
|
+
}.freeze
|
|
45
|
+
|
|
46
|
+
# Initialize the Hebrew SI-1452 layout.
|
|
47
|
+
def initialize
|
|
48
|
+
super(
|
|
49
|
+
name: 'Hebrew-SI-1452',
|
|
50
|
+
language_codes: %w[he he-IL],
|
|
51
|
+
key_positions: KEY_POSITIONS
|
|
52
|
+
)
|
|
53
|
+
end
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Keyboard
|
|
5
|
+
module Layouts
|
|
6
|
+
# Croatian and Slovenian QWERTZ keyboard layouts (KBDCRO /
|
|
7
|
+
# KBDYCL driver tables — the two national standards share one
|
|
8
|
+
# physical grid).
|
|
9
|
+
#
|
|
10
|
+
# The five South-Slavic Latin letters are real keys: š đ take
|
|
11
|
+
# the top-row bracket faces, č ć ž the home-row umlaut faces
|
|
12
|
+
# and the key right of them. Everything else is the QWERTZ
|
|
13
|
+
# physical arrangement (z/y swapped), so diacritic slips are not
|
|
14
|
+
# possible for these five — they are key slips.
|
|
15
|
+
#
|
|
16
|
+
# The shared key grid is mirrored from the models repo eval
|
|
17
|
+
# harness (kotoshu/models-fasttext-onnx eval/noise.py _HRSL,
|
|
18
|
+
# plan 83 batch 2 of the model coverage expansion, verified
|
|
19
|
+
# there against the kbdlayout.info driver tables) so
|
|
20
|
+
# typo-adjacency used for suggestion ranking matches the
|
|
21
|
+
# adjacency used to evaluate the models.
|
|
22
|
+
# spec/kotoshu/keyboard/layouts/eval_grid_drift_spec.rb enforces
|
|
23
|
+
# the sync.
|
|
24
|
+
module HrSlQwertz
|
|
25
|
+
# Key positions for the Croatian/Slovenian QWERTZ grid.
|
|
26
|
+
# Each key maps to [row, column] coordinates.
|
|
27
|
+
KEY_POSITIONS = {
|
|
28
|
+
# Number row
|
|
29
|
+
'‚' => [0, 0], '1' => [0, 1], '2' => [0, 2], '3' => [0, 3],
|
|
30
|
+
'4' => [0, 4], '5' => [0, 5], '6' => [0, 6], '7' => [0, 7],
|
|
31
|
+
'8' => [0, 8], '9' => [0, 9], '0' => [0, 10],
|
|
32
|
+
"'" => [0, 11], '+' => [0, 12],
|
|
33
|
+
# Top row (QWERTZ + š đ)
|
|
34
|
+
'q' => [1, 0], 'w' => [1, 1], 'e' => [1, 2], 'r' => [1, 3],
|
|
35
|
+
't' => [1, 4], 'z' => [1, 5], 'u' => [1, 6], 'i' => [1, 7],
|
|
36
|
+
'o' => [1, 8], 'p' => [1, 9], 'š' => [1, 10], 'đ' => [1, 11],
|
|
37
|
+
# Home row (ASDFG + č ć ž)
|
|
38
|
+
'a' => [2, 0], 's' => [2, 1], 'd' => [2, 2], 'f' => [2, 3],
|
|
39
|
+
'g' => [2, 4], 'h' => [2, 5], 'j' => [2, 6], 'k' => [2, 7],
|
|
40
|
+
'l' => [2, 8], 'č' => [2, 9], 'ć' => [2, 10], 'ž' => [2, 11],
|
|
41
|
+
# Bottom row (YXCVB)
|
|
42
|
+
'y' => [3, 0], 'x' => [3, 1], 'c' => [3, 2], 'v' => [3, 3],
|
|
43
|
+
'b' => [3, 4], 'n' => [3, 5], 'm' => [3, 6],
|
|
44
|
+
',' => [3, 7], '.' => [3, 8], '-' => [3, 9]
|
|
45
|
+
}.freeze
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
# Croatian QWERTZ (KBDCRO / Croatian standard grid).
|
|
49
|
+
#
|
|
50
|
+
# Languages: hr, hr-HR
|
|
51
|
+
class Croatian < Layout
|
|
52
|
+
# Initialize the Croatian layout.
|
|
53
|
+
def initialize
|
|
54
|
+
super(
|
|
55
|
+
name: 'Croatian-QWERTZ',
|
|
56
|
+
language_codes: %w[hr hr-HR],
|
|
57
|
+
key_positions: HrSlQwertz::KEY_POSITIONS
|
|
58
|
+
)
|
|
59
|
+
end
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
# Slovenian QWERTZ (KBDYCL / Slovenian standard grid — the same
|
|
63
|
+
# physical arrangement as Croatian).
|
|
64
|
+
#
|
|
65
|
+
# Languages: sl, sl-SI
|
|
66
|
+
class Slovenian < Layout
|
|
67
|
+
# Initialize the Slovenian layout.
|
|
68
|
+
def initialize
|
|
69
|
+
super(
|
|
70
|
+
name: 'Slovenian-QWERTZ',
|
|
71
|
+
language_codes: %w[sl sl-SI],
|
|
72
|
+
key_positions: HrSlQwertz::KEY_POSITIONS
|
|
73
|
+
)
|
|
74
|
+
end
|
|
75
|
+
end
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
end
|
|
@@ -15,7 +15,8 @@ module Kotoshu
|
|
|
15
15
|
# (kotoshu/models-fasttext-onnx eval/noise.py: LANG_LAYOUT maps
|
|
16
16
|
# the Latin newcomers onto the qwerty/qwertz grids and handles
|
|
17
17
|
# diacritics as _LANG_ALTERNATES slips), keeping gem suggestion
|
|
18
|
-
# adjacency and eval adjacency in sync.
|
|
18
|
+
# adjacency and eval adjacency in sync. Batch 3 (plan 100) adds
|
|
19
|
+
# id sk et lt lv on the same qwerty base grid.
|
|
19
20
|
#
|
|
20
21
|
# One family, per-language declarations — no near-duplicate
|
|
21
22
|
# layout files. The five pre-existing layouts (qwerty qwertz
|
|
@@ -214,6 +215,51 @@ module Kotoshu
|
|
|
214
215
|
removed_keys %w([ ] ; ')
|
|
215
216
|
extra_positions({ 'å' => [1, 10], 'ä' => [2, 9], 'ö' => [2, 10] })
|
|
216
217
|
end
|
|
218
|
+
|
|
219
|
+
# Indonesian: US QWERTY physical layout — the Indonesian
|
|
220
|
+
# national standard types on the unmodified US grid, and the
|
|
221
|
+
# eval harness maps id the same way (plan 83 batch 2).
|
|
222
|
+
class Indonesian < Latin
|
|
223
|
+
layout_name 'Indonesian-QWERTY'
|
|
224
|
+
languages %w[id id-ID]
|
|
225
|
+
base_grid :qwerty
|
|
226
|
+
end
|
|
227
|
+
|
|
228
|
+
# Slovak: US QWERTY physical layout; the eval harness maps sk
|
|
229
|
+
# to the qwerty grid (plan 83 batch 2) with the háčik
|
|
230
|
+
# diacritics as dead-key slips, exactly the Polish
|
|
231
|
+
# programmers' pattern. The Slovak QWERTZ national variant
|
|
232
|
+
# exists, but Slovak QWERTY is the widely-typed form.
|
|
233
|
+
class Slovak < Latin
|
|
234
|
+
layout_name 'Slovak-QWERTY'
|
|
235
|
+
languages %w[sk sk-SK]
|
|
236
|
+
base_grid :qwerty
|
|
237
|
+
end
|
|
238
|
+
|
|
239
|
+
# Estonian: US QWERTY physical layout; ä ö õ ü š ž are dead-key
|
|
240
|
+
# or AltGr sequences on the eval model grid (plan 83 batch 2).
|
|
241
|
+
class Estonian < Latin
|
|
242
|
+
layout_name 'Estonian-QWERTY'
|
|
243
|
+
languages %w[et et-EE]
|
|
244
|
+
base_grid :qwerty
|
|
245
|
+
end
|
|
246
|
+
|
|
247
|
+
# Lithuanian: US QWERTY physical layout; ą č ę ė į š ų ū ž are
|
|
248
|
+
# dead-key or AltGr sequences (plan 83 batch 2 eval mapping).
|
|
249
|
+
class Lithuanian < Latin
|
|
250
|
+
layout_name 'Lithuanian-QWERTY'
|
|
251
|
+
languages %w[lt lt-LT]
|
|
252
|
+
base_grid :qwerty
|
|
253
|
+
end
|
|
254
|
+
|
|
255
|
+
# Latvian: US QWERTY physical layout; ā č ē ģ ī ķ ļ ņ š ū ž
|
|
256
|
+
# are dead-key or AltGr sequences (plan 83 batch 2 eval
|
|
257
|
+
# mapping).
|
|
258
|
+
class Latvian < Latin
|
|
259
|
+
layout_name 'Latvian-QWERTY'
|
|
260
|
+
languages %w[lv lv-LV]
|
|
261
|
+
base_grid :qwerty
|
|
262
|
+
end
|
|
217
263
|
end
|
|
218
264
|
end
|
|
219
265
|
end
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Keyboard
|
|
5
|
+
module Layouts
|
|
6
|
+
# Persian standard keyboard layout (ISIRI 9147).
|
|
7
|
+
#
|
|
8
|
+
# Shares the Arabic letter skeleton but carries the four Persian
|
|
9
|
+
# letters as real keys: پ چ ژ گ replace their Arabic equivalents
|
|
10
|
+
# at standard positions, and ک ی are the Persian keheh/yeh
|
|
11
|
+
# keycaps. The ZWNJ and diacritics are dead keys and stay out of
|
|
12
|
+
# the grid; the Arabic→Persian letter slips (ك→ک ي→ی) are
|
|
13
|
+
# handled at the normalizer level.
|
|
14
|
+
#
|
|
15
|
+
# The key grid is mirrored from the models repo eval harness
|
|
16
|
+
# (kotoshu/models-fasttext-onnx eval/noise.py _FA_STD, plan 83
|
|
17
|
+
# batch 2 of the model coverage expansion, verified there
|
|
18
|
+
# against ISIRI 9147 renderings) so typo-adjacency used for
|
|
19
|
+
# suggestion ranking matches the adjacency used to evaluate the
|
|
20
|
+
# models. spec/kotoshu/keyboard/layouts/eval_grid_drift_spec.rb
|
|
21
|
+
# enforces the sync.
|
|
22
|
+
#
|
|
23
|
+
# Languages: fa, fa-IR
|
|
24
|
+
class PersianStandard < Layout
|
|
25
|
+
# Key positions for the Persian standard layout.
|
|
26
|
+
# Each key maps to [row, column] coordinates.
|
|
27
|
+
KEY_POSITIONS = {
|
|
28
|
+
# Number row
|
|
29
|
+
'`' => [0, 0], '1' => [0, 1], '2' => [0, 2], '3' => [0, 3],
|
|
30
|
+
'4' => [0, 4], '5' => [0, 5], '6' => [0, 6], '7' => [0, 7],
|
|
31
|
+
'8' => [0, 8], '9' => [0, 9], '0' => [0, 10], '-' => [0, 11],
|
|
32
|
+
'=' => [0, 12],
|
|
33
|
+
# Top row (ض ص ث ق ف غ ع ه خ ح ج چ)
|
|
34
|
+
'ض' => [1, 0], 'ص' => [1, 1], 'ث' => [1, 2], 'ق' => [1, 3],
|
|
35
|
+
'ف' => [1, 4], 'غ' => [1, 5], 'ع' => [1, 6], 'ه' => [1, 7],
|
|
36
|
+
'خ' => [1, 8], 'ح' => [1, 9], 'ج' => [1, 10], 'چ' => [1, 11],
|
|
37
|
+
# Home row (ش س ی ب ل ا ت ن م ک گ)
|
|
38
|
+
'ش' => [2, 0], 'س' => [2, 1], 'ی' => [2, 2], 'ب' => [2, 3],
|
|
39
|
+
'ل' => [2, 4], 'ا' => [2, 5], 'ت' => [2, 6], 'ن' => [2, 7],
|
|
40
|
+
'م' => [2, 8], 'ک' => [2, 9], 'گ' => [2, 10],
|
|
41
|
+
# Bottom row (ظ ط ز ر ذ د پ و ۀ .)
|
|
42
|
+
'ظ' => [3, 0], 'ط' => [3, 1], 'ز' => [3, 2], 'ر' => [3, 3],
|
|
43
|
+
'ذ' => [3, 4], 'د' => [3, 5], 'پ' => [3, 6], 'و' => [3, 7],
|
|
44
|
+
'ۀ' => [3, 8], '.' => [3, 9]
|
|
45
|
+
}.freeze
|
|
46
|
+
|
|
47
|
+
# Initialize the Persian standard layout.
|
|
48
|
+
def initialize
|
|
49
|
+
super(
|
|
50
|
+
name: 'Persian-Standard',
|
|
51
|
+
language_codes: %w[fa fa-IR],
|
|
52
|
+
key_positions: KEY_POSITIONS
|
|
53
|
+
)
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
|
58
|
+
end
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Keyboard
|
|
5
|
+
module Layouts
|
|
6
|
+
# Serbian Cyrillic keyboard layout (ЉЊЕРТЗУИОПШ /
|
|
7
|
+
# АСДФГХЈКЛЧЋ / ЏЦВБНМ, KBDSR driver table).
|
|
8
|
+
#
|
|
9
|
+
# The Serbian-specific letters љ њ ђ ћ џ ј are real keys: љ њ
|
|
10
|
+
# take the q/w faces, ђ sits where QWERTY carries the right
|
|
11
|
+
# bracket, and џ ц в б н м fill the bottom row. Serbian Latin
|
|
12
|
+
# (sr-Latn) types on a QWERTZ physical grid but is not claimed
|
|
13
|
+
# by this layout — the gem wires Cyrillic as the primary Serbian
|
|
14
|
+
# script, matching the dictionaries repo dictionary.
|
|
15
|
+
#
|
|
16
|
+
# The key grid is mirrored from the models repo eval harness
|
|
17
|
+
# (kotoshu/models-fasttext-onnx eval/noise.py _SR_CYR, plan 83
|
|
18
|
+
# batch 2 of the model coverage expansion) so typo-adjacency
|
|
19
|
+
# used for suggestion ranking matches the adjacency used to
|
|
20
|
+
# evaluate the models.
|
|
21
|
+
# spec/kotoshu/keyboard/layouts/eval_grid_drift_spec.rb enforces
|
|
22
|
+
# the sync.
|
|
23
|
+
#
|
|
24
|
+
# Languages: sr, sr-RS
|
|
25
|
+
class SerbianCyrillic < Layout
|
|
26
|
+
# Key positions for the Serbian Cyrillic layout.
|
|
27
|
+
# Each key maps to [row, column] coordinates.
|
|
28
|
+
KEY_POSITIONS = {
|
|
29
|
+
# Number row
|
|
30
|
+
'`' => [0, 0], '1' => [0, 1], '2' => [0, 2], '3' => [0, 3],
|
|
31
|
+
'4' => [0, 4], '5' => [0, 5], '6' => [0, 6], '7' => [0, 7],
|
|
32
|
+
'8' => [0, 8], '9' => [0, 9], '0' => [0, 10],
|
|
33
|
+
"'" => [0, 11], '+' => [0, 12],
|
|
34
|
+
# Top row (љ њ е р т з у и о п ш ђ)
|
|
35
|
+
'љ' => [1, 0], 'њ' => [1, 1], 'е' => [1, 2], 'р' => [1, 3],
|
|
36
|
+
'т' => [1, 4], 'з' => [1, 5], 'у' => [1, 6], 'и' => [1, 7],
|
|
37
|
+
'о' => [1, 8], 'п' => [1, 9], 'ш' => [1, 10], 'ђ' => [1, 11],
|
|
38
|
+
# Home row (а с д ф г х ј к л ч ћ ж)
|
|
39
|
+
'а' => [2, 0], 'с' => [2, 1], 'д' => [2, 2], 'ф' => [2, 3],
|
|
40
|
+
'г' => [2, 4], 'х' => [2, 5], 'ј' => [2, 6], 'к' => [2, 7],
|
|
41
|
+
'л' => [2, 8], 'ч' => [2, 9], 'ћ' => [2, 10], 'ж' => [2, 11],
|
|
42
|
+
# Bottom row (џ ц в б н м , . -)
|
|
43
|
+
'џ' => [3, 0], 'ц' => [3, 1], 'в' => [3, 2], 'б' => [3, 3],
|
|
44
|
+
'н' => [3, 4], 'м' => [3, 5], ',' => [3, 6], '.' => [3, 7],
|
|
45
|
+
'-' => [3, 8]
|
|
46
|
+
}.freeze
|
|
47
|
+
|
|
48
|
+
# Initialize the Serbian Cyrillic layout.
|
|
49
|
+
def initialize
|
|
50
|
+
super(
|
|
51
|
+
name: 'Serbian-Cyrillic',
|
|
52
|
+
language_codes: %w[sr sr-RS],
|
|
53
|
+
key_positions: KEY_POSITIONS
|
|
54
|
+
)
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
@@ -132,11 +132,14 @@ module Kotoshu
|
|
|
132
132
|
register(Layouts::QWERTY)
|
|
133
133
|
register(Layouts::QWERTZ)
|
|
134
134
|
register(Layouts::AZERTY)
|
|
135
|
-
# Ukrainian
|
|
136
|
-
# blanket-claims uk, but
|
|
137
|
-
#
|
|
138
|
-
# first-registered-wins, so order here is
|
|
135
|
+
# Ukrainian, Bulgarian and Serbian register before JCUKEN: the
|
|
136
|
+
# base JCUKEN layout blanket-claims uk bg, but typists of those
|
|
137
|
+
# languages get their dedicated grids (plan 84 uk, plan 100
|
|
138
|
+
# bg/sr). Lookup is first-registered-wins, so order here is
|
|
139
|
+
# load-bearing for uk bg sr.
|
|
139
140
|
register(Layouts::Ukrainian)
|
|
141
|
+
register(Layouts::BulgarianBds)
|
|
142
|
+
register(Layouts::SerbianCyrillic)
|
|
140
143
|
register(Layouts::JCUKEN)
|
|
141
144
|
register(Layouts::Dvorak)
|
|
142
145
|
# Wave-1 national layouts (plan 84)
|
|
@@ -154,6 +157,19 @@ module Kotoshu
|
|
|
154
157
|
register(Layouts::Latin::Danish)
|
|
155
158
|
register(Layouts::Latin::Norwegian)
|
|
156
159
|
register(Layouts::Latin::Swedish)
|
|
160
|
+
# Batch-3 national layouts (plan 100): ar fa he type on their
|
|
161
|
+
# national-script grids; hr and sl share the South-Slavic QWERTZ
|
|
162
|
+
# grid; the id sk et lt lv Latin members follow.
|
|
163
|
+
register(Layouts::Arabic101)
|
|
164
|
+
register(Layouts::PersianStandard)
|
|
165
|
+
register(Layouts::HebrewSI1452)
|
|
166
|
+
register(Layouts::Croatian)
|
|
167
|
+
register(Layouts::Slovenian)
|
|
168
|
+
register(Layouts::Latin::Indonesian)
|
|
169
|
+
register(Layouts::Latin::Slovak)
|
|
170
|
+
register(Layouts::Latin::Estonian)
|
|
171
|
+
register(Layouts::Latin::Lithuanian)
|
|
172
|
+
register(Layouts::Latin::Latvian)
|
|
157
173
|
end
|
|
158
174
|
end
|
|
159
175
|
end
|
data/lib/kotoshu/keyboard.rb
CHANGED
|
@@ -32,6 +32,15 @@ module Kotoshu
|
|
|
32
32
|
# Parameterized Latin family (plan 84): one file, per-language
|
|
33
33
|
# declarations over the qwerty/qwertz base grids.
|
|
34
34
|
autoload :Latin, "kotoshu/keyboard/layouts/latin"
|
|
35
|
+
# Batch-3 national layouts (plan 100): grids mirrored from the
|
|
36
|
+
# models repo eval harness (plan 83 batch-2 grids).
|
|
37
|
+
autoload :Arabic101, "kotoshu/keyboard/layouts/arabic"
|
|
38
|
+
autoload :PersianStandard, "kotoshu/keyboard/layouts/persian"
|
|
39
|
+
autoload :HebrewSI1452, "kotoshu/keyboard/layouts/hebrew"
|
|
40
|
+
autoload :BulgarianBds, "kotoshu/keyboard/layouts/bulgarian"
|
|
41
|
+
autoload :SerbianCyrillic, "kotoshu/keyboard/layouts/serbian"
|
|
42
|
+
autoload :Croatian, "kotoshu/keyboard/layouts/hrsl"
|
|
43
|
+
autoload :Slovenian, "kotoshu/keyboard/layouts/hrsl"
|
|
35
44
|
end
|
|
36
45
|
|
|
37
46
|
class << self
|
|
@@ -14,6 +14,11 @@ module Kotoshu
|
|
|
14
14
|
# lang = Kotoshu::Languages::Arabic.new
|
|
15
15
|
# lang.script_type # => :arabic
|
|
16
16
|
# lang.rtl? # => true
|
|
17
|
+
#
|
|
18
|
+
# Full-feature since plan 100 (batch 3): the spelling dictionary
|
|
19
|
+
# downloads from the dictionaries repo (AVAILABLE_LANGUAGES) and
|
|
20
|
+
# keyboard proximity uses the Arabic 101 grid
|
|
21
|
+
# (Keyboard::Layouts::Arabic101).
|
|
17
22
|
class Arabic < Language::Base
|
|
18
23
|
register "ar"
|
|
19
24
|
register "ar-SA"
|
|
@@ -69,6 +74,15 @@ module Kotoshu
|
|
|
69
74
|
.reject { |token| skip_token?(token) }
|
|
70
75
|
end
|
|
71
76
|
|
|
77
|
+
# Spell-check word characters (plan 91 pattern, added by
|
|
78
|
+
# plan 100): Arabic-script letters. Arabic needs no ZWNJ (the
|
|
79
|
+
# staged dictionary carries none) and no apostrophe.
|
|
80
|
+
#
|
|
81
|
+
# @return [Regexp] Regex matching a single word character
|
|
82
|
+
def spellcheck_word_regex
|
|
83
|
+
/\p{Arabic}/
|
|
84
|
+
end
|
|
85
|
+
|
|
72
86
|
protected
|
|
73
87
|
|
|
74
88
|
def normalize(token)
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Languages
|
|
5
|
+
# Bulgarian language implementation (plan 100, batch 3).
|
|
6
|
+
#
|
|
7
|
+
# Cyrillic composition like Ukrainian: the Cyrillic tokenizer
|
|
8
|
+
# keeps Bulgarian-script words, the base normalizer suffices —
|
|
9
|
+
# Bulgarian case folding has no special pairs. The Bulgarian-only
|
|
10
|
+
# letter ъ (and ь) are real keys on the Bulgarian BDS grid
|
|
11
|
+
# (Keyboard::Layouts::BulgarianBds), not the Russian JCUKEN the
|
|
12
|
+
# pre-existing Cyrillic layout models.
|
|
13
|
+
class Bulgarian < LatinBase
|
|
14
|
+
register "bg"
|
|
15
|
+
register "bg-BG"
|
|
16
|
+
|
|
17
|
+
def initialize(code: "bg", name: "Bulgarian")
|
|
18
|
+
super
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
# Cyrillic-script tokenizer.
|
|
22
|
+
#
|
|
23
|
+
# @return [Language::Tokenizer::CyrillicTokenizer]
|
|
24
|
+
def tokenizer
|
|
25
|
+
@tokenizer ||= Language::Tokenizer::CyrillicTokenizer.new
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def script_type
|
|
29
|
+
:cyrillic
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def default_dictionary_paths
|
|
33
|
+
["/usr/share/dict/bulgarian"]
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
end
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Languages
|
|
5
|
+
# Estonian language implementation (plan 100, batch 3).
|
|
6
|
+
#
|
|
7
|
+
# Thin composition: shared Latin tokenizer, base normalizer,
|
|
8
|
+
# registry entry. The Estonian letters ä ö õ ü š ž are dead-key /
|
|
9
|
+
# AltGr sequences over the US QWERTY grid
|
|
10
|
+
# (Keyboard::Layouts::Latin::Estonian), like the eval model.
|
|
11
|
+
class Estonian < LatinBase
|
|
12
|
+
register "et"
|
|
13
|
+
register "et-EE"
|
|
14
|
+
|
|
15
|
+
def initialize(code: "et", name: "Estonian")
|
|
16
|
+
super
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def default_dictionary_paths
|
|
20
|
+
["/usr/share/dict/estonian"]
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
@@ -13,6 +13,11 @@ module Kotoshu
|
|
|
13
13
|
# lang = Kotoshu::Languages::Persian.new
|
|
14
14
|
# lang.script_type # => :arabic
|
|
15
15
|
# lang.rtl? # => true
|
|
16
|
+
#
|
|
17
|
+
# Full-feature since plan 100 (batch 3): the spelling dictionary
|
|
18
|
+
# downloads from the dictionaries repo (AVAILABLE_LANGUAGES) and
|
|
19
|
+
# keyboard proximity uses the Persian standard grid
|
|
20
|
+
# (Keyboard::Layouts::PersianStandard).
|
|
16
21
|
class Persian < Language::Base
|
|
17
22
|
register "fa"
|
|
18
23
|
register "fa-IR"
|
|
@@ -60,6 +65,17 @@ module Kotoshu
|
|
|
60
65
|
.reject { |token| skip_token?(token) }
|
|
61
66
|
end
|
|
62
67
|
|
|
68
|
+
# Spell-check word characters (plan 91 pattern, added by
|
|
69
|
+
# plan 100): Arabic-script letters plus ZWNJ (U+200C) —
|
|
70
|
+
# Persian compounds like میروم keep their nimb-fāsele inside
|
|
71
|
+
# the word, and the staged dictionary carries tens of
|
|
72
|
+
# thousands of ZWNJ forms.
|
|
73
|
+
#
|
|
74
|
+
# @return [Regexp] Regex matching a single word character
|
|
75
|
+
def spellcheck_word_regex
|
|
76
|
+
/[\p{Arabic}\u200C]/
|
|
77
|
+
end
|
|
78
|
+
|
|
63
79
|
protected
|
|
64
80
|
|
|
65
81
|
def normalize(token)
|
|
@@ -14,6 +14,11 @@ module Kotoshu
|
|
|
14
14
|
# lang = Kotoshu::Languages::Hebrew.new
|
|
15
15
|
# lang.script_type # => :hebrew
|
|
16
16
|
# lang.rtl? # => true
|
|
17
|
+
#
|
|
18
|
+
# Full-feature since plan 100 (batch 3): the spelling dictionary
|
|
19
|
+
# downloads from the dictionaries repo (AVAILABLE_LANGUAGES) and
|
|
20
|
+
# keyboard proximity uses the Hebrew SI-1452 grid
|
|
21
|
+
# (Keyboard::Layouts::HebrewSI1452).
|
|
17
22
|
class Hebrew < Language::Base
|
|
18
23
|
register "he"
|
|
19
24
|
register "he-IL"
|
|
@@ -68,6 +73,16 @@ module Kotoshu
|
|
|
68
73
|
.reject { |token| skip_token?(token) }
|
|
69
74
|
end
|
|
70
75
|
|
|
76
|
+
# Spell-check word characters (plan 91 pattern, added by
|
|
77
|
+
# plan 100): Hebrew letters plus the geresh/gershayim used
|
|
78
|
+
# inside acronyms. Niqqud are combining marks, not letters —
|
|
79
|
+
# the Hebrew normalizer strips them from input before lookup.
|
|
80
|
+
#
|
|
81
|
+
# @return [Regexp] Regex matching a single word character
|
|
82
|
+
def spellcheck_word_regex
|
|
83
|
+
/[\p{Hebrew}\u05F3\u05F4]/
|
|
84
|
+
end
|
|
85
|
+
|
|
71
86
|
protected
|
|
72
87
|
|
|
73
88
|
def normalize(token)
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Languages
|
|
5
|
+
# Croatian language implementation (plan 100, batch 3).
|
|
6
|
+
#
|
|
7
|
+
# Thin composition: shared Latin tokenizer, base normalizer,
|
|
8
|
+
# registry entry. The five Croatian digraph letters š đ č ć ž are
|
|
9
|
+
# real keys on the South-Slavic QWERTZ physical layout
|
|
10
|
+
# (Keyboard::Layouts::Croatian), shared with Slovenian.
|
|
11
|
+
class Croatian < LatinBase
|
|
12
|
+
register "hr"
|
|
13
|
+
register "hr-HR"
|
|
14
|
+
|
|
15
|
+
def initialize(code: "hr", name: "Croatian")
|
|
16
|
+
super
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def default_dictionary_paths
|
|
20
|
+
["/usr/share/dict/croatian"]
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Languages
|
|
5
|
+
# Indonesian language implementation (plan 100, batch 3).
|
|
6
|
+
#
|
|
7
|
+
# Thin composition like the wave-1 members: shared Latin
|
|
8
|
+
# tokenizer, base normalizer, registry entry. Indonesian is typed
|
|
9
|
+
# on the unmodified US QWERTY grid (Keyboard::Layouts::Latin::Indonesian)
|
|
10
|
+
# — the language has no diacritics, so the Latin base covers it
|
|
11
|
+
# without any per-language care.
|
|
12
|
+
class Indonesian < LatinBase
|
|
13
|
+
register "id"
|
|
14
|
+
register "id-ID"
|
|
15
|
+
|
|
16
|
+
def initialize(code: "id", name: "Indonesian")
|
|
17
|
+
super
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def default_dictionary_paths
|
|
21
|
+
["/usr/share/dict/indonesian"]
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Languages
|
|
5
|
+
# Lithuanian language implementation (plan 100, batch 3).
|
|
6
|
+
#
|
|
7
|
+
# Thin composition: shared Latin tokenizer, base normalizer,
|
|
8
|
+
# registry entry. The Lithuanian letters ą č ę ė į š ų ū ž are
|
|
9
|
+
# dead-key / AltGr sequences over the US QWERTY grid
|
|
10
|
+
# (Keyboard::Layouts::Latin::Lithuanian), like the eval model.
|
|
11
|
+
class Lithuanian < LatinBase
|
|
12
|
+
register "lt"
|
|
13
|
+
register "lt-LT"
|
|
14
|
+
|
|
15
|
+
def initialize(code: "lt", name: "Lithuanian")
|
|
16
|
+
super
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def default_dictionary_paths
|
|
20
|
+
["/usr/share/dict/lithuanian"]
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Languages
|
|
5
|
+
# Latvian language implementation (plan 100, batch 3).
|
|
6
|
+
#
|
|
7
|
+
# Thin composition: shared Latin tokenizer, base normalizer,
|
|
8
|
+
# registry entry. The Latvian letters ā č ē ģ ī ķ ļ ņ š ū ž are
|
|
9
|
+
# dead-key / AltGr sequences over the US QWERTY grid
|
|
10
|
+
# (Keyboard::Layouts::Latin::Latvian), like the eval model.
|
|
11
|
+
class Latvian < LatinBase
|
|
12
|
+
register "lv"
|
|
13
|
+
register "lv-LV"
|
|
14
|
+
|
|
15
|
+
def initialize(code: "lv", name: "Latvian")
|
|
16
|
+
super
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def default_dictionary_paths
|
|
20
|
+
["/usr/share/dict/latvian"]
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Languages
|
|
5
|
+
# Slovak language implementation (plan 100, batch 3).
|
|
6
|
+
#
|
|
7
|
+
# Thin composition: shared Latin tokenizer, base normalizer,
|
|
8
|
+
# registry entry. Slovak diacritics (á ä č ď é í ľ ĺ ň ó ô ŕ š ť
|
|
9
|
+
# ú ý ž) are dead keys over the US QWERTY grid
|
|
10
|
+
# (Keyboard::Layouts::Latin::Slovak), the Polish programmers'
|
|
11
|
+
# pattern the eval harness also models.
|
|
12
|
+
class Slovak < LatinBase
|
|
13
|
+
register "sk"
|
|
14
|
+
register "sk-SK"
|
|
15
|
+
|
|
16
|
+
def initialize(code: "sk", name: "Slovak")
|
|
17
|
+
super
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def default_dictionary_paths
|
|
21
|
+
["/usr/share/dict/slovak"]
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Languages
|
|
5
|
+
# Slovenian language implementation (plan 100, batch 3).
|
|
6
|
+
#
|
|
7
|
+
# Thin composition: shared Latin tokenizer, base normalizer,
|
|
8
|
+
# registry entry. The Slovenian letters č š ž (plus the Croatian
|
|
9
|
+
# đ ć the grid also carries) are real keys on the South-Slavic
|
|
10
|
+
# QWERTZ physical layout (Keyboard::Layouts::Slovenian), shared
|
|
11
|
+
# with Croatian.
|
|
12
|
+
class Slovenian < LatinBase
|
|
13
|
+
register "sl"
|
|
14
|
+
register "sl-SI"
|
|
15
|
+
|
|
16
|
+
def initialize(code: "sl", name: "Slovenian")
|
|
17
|
+
super
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def default_dictionary_paths
|
|
21
|
+
["/usr/share/dict/slovenian"]
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kotoshu
|
|
4
|
+
module Languages
|
|
5
|
+
# Serbian language implementation (plan 100, batch 3).
|
|
6
|
+
#
|
|
7
|
+
# Cyrillic composition like Bulgarian/Ukrainian: Serbian is
|
|
8
|
+
# written in both scripts, but Cyrillic is the constitutional
|
|
9
|
+
# primary and the script the staged dictionary carries — the
|
|
10
|
+
# module registers sr for the Cyrillic grid
|
|
11
|
+
# (Keyboard::Layouts::SerbianCyrillic); Serbian Latin (sr-Latn)
|
|
12
|
+
# stays unwired like nn until its own module lands. The Serbian
|
|
13
|
+
# letters љ њ ђ ћ џ ј are real keys on that grid.
|
|
14
|
+
class Serbian < LatinBase
|
|
15
|
+
register "sr"
|
|
16
|
+
register "sr-RS"
|
|
17
|
+
|
|
18
|
+
def initialize(code: "sr", name: "Serbian")
|
|
19
|
+
super
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
# Cyrillic-script tokenizer.
|
|
23
|
+
#
|
|
24
|
+
# @return [Language::Tokenizer::CyrillicTokenizer]
|
|
25
|
+
def tokenizer
|
|
26
|
+
@tokenizer ||= Language::Tokenizer::CyrillicTokenizer.new
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
def script_type
|
|
30
|
+
:cyrillic
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
def default_dictionary_paths
|
|
34
|
+
["/usr/share/dict/serbian"]
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
end
|
data/lib/kotoshu/languages.rb
CHANGED
|
@@ -36,5 +36,17 @@ module Kotoshu
|
|
|
36
36
|
autoload :Turkish, "kotoshu/languages/tr/language"
|
|
37
37
|
autoload :Ukrainian, "kotoshu/languages/uk/language"
|
|
38
38
|
autoload :Vietnamese, "kotoshu/languages/vi/language"
|
|
39
|
+
# Batch-3 modules (plan 100): id hr sl sk et lt lv (Latin) and
|
|
40
|
+
# bg sr (Cyrillic). The pre-existing ar fa he modules become
|
|
41
|
+
# full-feature in this batch (layout + downloadable dictionary).
|
|
42
|
+
autoload :Bulgarian, "kotoshu/languages/bg/language"
|
|
43
|
+
autoload :Croatian, "kotoshu/languages/hr/language"
|
|
44
|
+
autoload :Estonian, "kotoshu/languages/et/language"
|
|
45
|
+
autoload :Indonesian, "kotoshu/languages/id/language"
|
|
46
|
+
autoload :Latvian, "kotoshu/languages/lv/language"
|
|
47
|
+
autoload :Lithuanian, "kotoshu/languages/lt/language"
|
|
48
|
+
autoload :Serbian, "kotoshu/languages/sr/language"
|
|
49
|
+
autoload :Slovak, "kotoshu/languages/sk/language"
|
|
50
|
+
autoload :Slovenian, "kotoshu/languages/sl/language"
|
|
39
51
|
end
|
|
40
52
|
end
|
data/lib/kotoshu/version.rb
CHANGED
|
@@ -16,7 +16,10 @@ abort "noise.py not found: #{source}" unless File.exist?(source)
|
|
|
16
16
|
|
|
17
17
|
src = File.read(source)
|
|
18
18
|
grids = {}
|
|
19
|
-
%w[
|
|
19
|
+
%w[
|
|
20
|
+
_TR_Q _UK_JCUKEN _EL_PHONETIC _AR_101 _FA_STD _HE_SI1452 _BG_BDS
|
|
21
|
+
_SR_CYR _HRSL
|
|
22
|
+
].each do |name|
|
|
20
23
|
body = src[/^#{name} = \{(.*?)^\}/m, 1]
|
|
21
24
|
abort "grid #{name} not found in #{source}" unless body
|
|
22
25
|
grid = {}
|
data/sig/kotoshu.rbs
CHANGED
|
@@ -152,6 +152,30 @@ def spellcheck_word_regex: () -> Regexp
|
|
|
152
152
|
end
|
|
153
153
|
class Vietnamese < LatinBase
|
|
154
154
|
end
|
|
155
|
+
# Batch-3 modules (plan 100): id hr sl sk et lt lv (Latin) and
|
|
156
|
+
# bg sr (Cyrillic).
|
|
157
|
+
class Bulgarian < LatinBase
|
|
158
|
+
def tokenizer: () -> Language::Tokenizer::CyrillicTokenizer
|
|
159
|
+
def script_type: () -> :cyrillic
|
|
160
|
+
end
|
|
161
|
+
class Croatian < LatinBase
|
|
162
|
+
end
|
|
163
|
+
class Estonian < LatinBase
|
|
164
|
+
end
|
|
165
|
+
class Indonesian < LatinBase
|
|
166
|
+
end
|
|
167
|
+
class Latvian < LatinBase
|
|
168
|
+
end
|
|
169
|
+
class Lithuanian < LatinBase
|
|
170
|
+
end
|
|
171
|
+
class Serbian < LatinBase
|
|
172
|
+
def tokenizer: () -> Language::Tokenizer::CyrillicTokenizer
|
|
173
|
+
def script_type: () -> :cyrillic
|
|
174
|
+
end
|
|
175
|
+
class Slovak < LatinBase
|
|
176
|
+
end
|
|
177
|
+
class Slovenian < LatinBase
|
|
178
|
+
end
|
|
155
179
|
end
|
|
156
180
|
|
|
157
181
|
module Cli
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kotoshu
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.9.
|
|
4
|
+
version: 0.9.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Ribose Inc.
|
|
@@ -103,6 +103,11 @@ files:
|
|
|
103
103
|
- TODO.impl/08-grammar-engine.md
|
|
104
104
|
- TODO.impl/09-integrity-security.md
|
|
105
105
|
- TODO.impl/10-testing-ci.md
|
|
106
|
+
- TODO.impl/100-full-feature-batch3.md
|
|
107
|
+
- TODO.impl/101-int4-tier-experiment.md
|
|
108
|
+
- TODO.impl/102-lid-chain.md
|
|
109
|
+
- TODO.impl/103-oov-bucket-tables.md
|
|
110
|
+
- TODO.impl/104-vscode-marketplace-readiness.md
|
|
106
111
|
- TODO.impl/11-release-v1.md
|
|
107
112
|
- TODO.impl/20-encapsulation-cleanup.md
|
|
108
113
|
- TODO.impl/21-autoload-migration.md
|
|
@@ -174,6 +179,7 @@ files:
|
|
|
174
179
|
- TODO.impl/96-server-semantic-models.md
|
|
175
180
|
- TODO.impl/97-yard-execution.md
|
|
176
181
|
- TODO.impl/98-campaign-cleanup.md
|
|
182
|
+
- TODO.impl/99-site-truth-performance-docs.md
|
|
177
183
|
- conformance/vectors.jsonl
|
|
178
184
|
- data/en/grammar/rules.yaml
|
|
179
185
|
- docs/cache-architecture.md
|
|
@@ -330,13 +336,19 @@ files:
|
|
|
330
336
|
- lib/kotoshu/jekyll.rb
|
|
331
337
|
- lib/kotoshu/keyboard.rb
|
|
332
338
|
- lib/kotoshu/keyboard/layout.rb
|
|
339
|
+
- lib/kotoshu/keyboard/layouts/arabic.rb
|
|
333
340
|
- lib/kotoshu/keyboard/layouts/azerty.rb
|
|
341
|
+
- lib/kotoshu/keyboard/layouts/bulgarian.rb
|
|
334
342
|
- lib/kotoshu/keyboard/layouts/dvorak.rb
|
|
335
343
|
- lib/kotoshu/keyboard/layouts/greek.rb
|
|
344
|
+
- lib/kotoshu/keyboard/layouts/hebrew.rb
|
|
345
|
+
- lib/kotoshu/keyboard/layouts/hrsl.rb
|
|
336
346
|
- lib/kotoshu/keyboard/layouts/jcuken.rb
|
|
337
347
|
- lib/kotoshu/keyboard/layouts/latin.rb
|
|
348
|
+
- lib/kotoshu/keyboard/layouts/persian.rb
|
|
338
349
|
- lib/kotoshu/keyboard/layouts/qwerty.rb
|
|
339
350
|
- lib/kotoshu/keyboard/layouts/qwertz.rb
|
|
351
|
+
- lib/kotoshu/keyboard/layouts/serbian.rb
|
|
340
352
|
- lib/kotoshu/keyboard/layouts/turkish_q.rb
|
|
341
353
|
- lib/kotoshu/keyboard/layouts/ukrainian.rb
|
|
342
354
|
- lib/kotoshu/keyboard/registry.rb
|
|
@@ -379,6 +391,7 @@ files:
|
|
|
379
391
|
- lib/kotoshu/language/tokenizer/spanish_tokenizer.rb
|
|
380
392
|
- lib/kotoshu/languages.rb
|
|
381
393
|
- lib/kotoshu/languages/ar/language.rb
|
|
394
|
+
- lib/kotoshu/languages/bg/language.rb
|
|
382
395
|
- lib/kotoshu/languages/ca/language.rb
|
|
383
396
|
- lib/kotoshu/languages/cs/language.rb
|
|
384
397
|
- lib/kotoshu/languages/da/language.rb
|
|
@@ -386,19 +399,27 @@ files:
|
|
|
386
399
|
- lib/kotoshu/languages/el/language.rb
|
|
387
400
|
- lib/kotoshu/languages/en/language.rb
|
|
388
401
|
- lib/kotoshu/languages/es/language.rb
|
|
402
|
+
- lib/kotoshu/languages/et/language.rb
|
|
389
403
|
- lib/kotoshu/languages/fa/language.rb
|
|
390
404
|
- lib/kotoshu/languages/fr/language.rb
|
|
391
405
|
- lib/kotoshu/languages/he/language.rb
|
|
406
|
+
- lib/kotoshu/languages/hr/language.rb
|
|
392
407
|
- lib/kotoshu/languages/hu/language.rb
|
|
408
|
+
- lib/kotoshu/languages/id/language.rb
|
|
393
409
|
- lib/kotoshu/languages/it/language.rb
|
|
394
410
|
- lib/kotoshu/languages/ja/language.rb
|
|
395
411
|
- lib/kotoshu/languages/latin_base.rb
|
|
412
|
+
- lib/kotoshu/languages/lt/language.rb
|
|
413
|
+
- lib/kotoshu/languages/lv/language.rb
|
|
396
414
|
- lib/kotoshu/languages/nb/language.rb
|
|
397
415
|
- lib/kotoshu/languages/nl/language.rb
|
|
398
416
|
- lib/kotoshu/languages/pl/language.rb
|
|
399
417
|
- lib/kotoshu/languages/pt/language.rb
|
|
400
418
|
- lib/kotoshu/languages/ro/language.rb
|
|
401
419
|
- lib/kotoshu/languages/ru/language.rb
|
|
420
|
+
- lib/kotoshu/languages/sk/language.rb
|
|
421
|
+
- lib/kotoshu/languages/sl/language.rb
|
|
422
|
+
- lib/kotoshu/languages/sr/language.rb
|
|
402
423
|
- lib/kotoshu/languages/sv/language.rb
|
|
403
424
|
- lib/kotoshu/languages/tr/language.rb
|
|
404
425
|
- lib/kotoshu/languages/uk/language.rb
|