khmerthings 0.4.3__tar.gz

khmerthings-0.4.3/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,28 @@
+# Summary
+
+<!-- What does this PR change, and why? -->
+
+## Type of change
+
+- [ ] New tool / feature
+- [ ] Bug fix
+- [ ] Lexicon update (`words.txt`)
+- [ ] Tests / CI / tooling
+- [ ] Documentation
+
+## Checklist
+
+- [ ] `uv run pytest` passes; new behavior is covered by tests
+- [ ] `uv run mypy src tests` is clean
+- [ ] `uv run ruff check` and `uv run ruff format --check` are clean
+- [ ] Change is fully deterministic (no randomness, no ML/LLM inference)
+- [ ] No third-party Khmer NLP code or data introduced; zero new runtime deps
+- [ ] Lexicon edits (if any): NFC, Khmer-only, no duplicates, both ្ត/្ដ
+      variants where applicable
+- [ ] Public API changes re-exported in `__init__.py` and documented in
+      `README.md` / `AGENTS.md`
+- [ ] `CHANGELOG.md` updated under `[Unreleased]` (any user-visible change)
+- [ ] Affected docs updated (`README.md`, `AGENTS.md`,
+      `DEVELOPMENT_GUIDE.md`, docstrings)
+- [ ] Tool changes: per-tool doc in `docs/<tool>.md` added/updated, with
+      real (executed) example outputs

khmerthings-0.4.3/.github/workflows/ci.yml

@@ -0,0 +1,69 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  lint:
+    name: Lint & type check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+      - name: Install
+        run: uv sync --locked
+      - name: Ruff lint
+        run: uv run ruff check --output-format=github
+      - name: Ruff format
+        run: uv run ruff format --check
+      - name: Mypy
+        run: uv run mypy src tests
+
+  test:
+    name: Test (Python ${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12", "3.13", "3.14"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+          python-version: ${{ matrix.python-version }}
+      - name: Install
+        run: uv sync --locked
+      - name: Run tests
+        run: uv run pytest --cov=khmerthings --cov-report=term-missing
+
+  build:
+    name: Build package
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+      - name: Build sdist and wheel
+        run: uv build
+      - name: Smoke test the wheel
+        run: |
+          uv venv /tmp/smoke
+          uv pip install --python /tmp/smoke/bin/python dist/*.whl
+          echo "ខ្ញុំស្រឡាញ់ភាសាខ្មែរ" | /tmp/smoke/bin/python -m khmerthings count -
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/

khmerthings-0.4.3/.github/workflows/publish.yml

@@ -0,0 +1,58 @@
+name: Publish
+
+on:
+  push:
+    tags: ["v*"]
+
+permissions:
+  contents: read
+
+jobs:
+  checks:
+    name: Full checks
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+      - name: Install
+        run: uv sync --locked
+      - name: Ruff lint
+        run: uv run ruff check --output-format=github
+      - name: Ruff format
+        run: uv run ruff format --check
+      - name: Mypy
+        run: uv run mypy src tests
+      - name: Run tests
+        run: uv run pytest
+
+  publish:
+    name: Build and publish to PyPI
+    runs-on: ubuntu-latest
+    needs: checks
+    environment: pypi
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+      - name: Check tag matches package version
+        run: |
+          version=$(uv run --no-sync python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])")
+          tag="${GITHUB_REF_NAME#v}"
+          if [ "$version" != "$tag" ]; then
+            echo "Tag v$tag does not match pyproject.toml version $version" >&2
+            exit 1
+          fi
+      - name: Build sdist and wheel
+        run: uv build
+      - name: Smoke test the wheel
+        run: |
+          uv venv /tmp/smoke
+          uv pip install --python /tmp/smoke/bin/python dist/*.whl
+          echo "ខ្ញុំស្រឡាញ់ភាសាខ្មែរ" | /tmp/smoke/bin/python -m khmerthings count -
+      - name: Publish to PyPI
+        run: uv publish --token "$PYPI_API_TOKEN"
+        env:
+          PYPI_API_TOKEN: ${{ secrets.PYPI_API_TOKEN }}

khmerthings-0.4.3/.gitignore

@@ -0,0 +1,8 @@
+.venv/
+__pycache__/
+*.egg-info/
+dist/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage

khmerthings-0.4.3/AGENTS.md

@@ -0,0 +1,130 @@
+# khmerthings — agent guide
+
+Deterministic Khmer language tools in Python. Note: the repo directory is
+`libkhm`, but the package, CLI, and PyPI name is **khmerthings**.
+
+## Hard constraints (non-negotiable)
+
+- **Deterministic only.** Every tool must be rule/algorithm/dictionary-based.
+  Same input → same output, always. No probabilistic models, no LLMs, no ML
+  inference at runtime.
+- **No third-party Khmer NLP code; word data is self-curated.** Do not add
+  dependencies on or copy from other Khmer NLP projects, and never
+  bulk-import someone else's wordlist. Web research to *find and verify*
+  candidate words/names/slang is allowed (user decision, 2026-07-03) — but
+  every entry is curated individually, spellings cross-checked, and sources
+  noted in the data file header. Data files under `src/khmerthings/data/`:
+  `words.txt` (core), `names.txt` (names, surnames, titles), `modern.txt`
+  (slang, informal, loanwords, trending). All three are growable and merged
+  via `load_lexicon(*sources)`.
+- **Zero runtime dependencies.** Stdlib only. Dev tools (pytest, ruff, mypy)
+  are the only allowed dependencies.
+- **Tests are the top priority.** Every module ships with table-driven unit
+  tests and invariant checks. Write/update tests with every change.
+
+## Commands
+
+```sh
+uv sync                      # env + dev deps
+uv run pytest                # tests (must stay green)
+uv run mypy src tests        # strict mode, must be clean
+uv run ruff check --fix && uv run ruff format
+uv build                     # sdist + wheel
+echo "ខ្មែរ" | uv run khmerthings count   # CLI smoke test
+```
+
+Run all four checks (pytest, mypy, ruff check, ruff format --check) before
+considering any change done.
+
+## Architecture
+
+`src/` layout; modules build bottom-up, each layer a primitive for the next:
+
+1. `chars.py` — Khmer Unicode character classification (pure functions,
+   single-character contract: multi-char input raises `ValueError`).
+2. `clusters.py` — Khmer character-cluster (KCC) segmentation. Cluster
+   boundaries are the only legal word boundaries.
+3. `lexicon.py` + `data/*.txt` — wordlists (`words`/`names`/`modern`) +
+   trie keyed by clusters; `longest_match` is the segmentation primitive;
+   `load_lexicon(*sources)` merges sources (cached), `--include` on the
+   CLI exposes the extra ones.
+4. `tokenizer.py` — lossless typed tokenization (Khmer words via greedy
+   longest-match; unknown Khmer spans become `KHMER_UNKNOWN`, never dropped).
+5. `counter.py` — word counter tool (`count_words`, `analyze`).
+6. `segmenter.py` — word breaker tool (`break_words`, `mark_boundaries`),
+   a thin first-class wrapper over `tokenize`; invariant
+   `len(break_words(t)) == count_words(t)`.
+7. `sorting.py` — Khmer dictionary-order line sorting (`sort_lines`,
+   `khmer_sort_key`: per-cluster key `(base, coengs, vowels, signs)` —
+   naive codepoint order is wrong for subscripts).
+8. `cli.py` — argparse subcommands, one per tool (`khmerthings count ...`,
+   `khmerthings segment ...`, `khmerthings sort ...`).
+
+Planned tools (spellchecker/spellfixer — blocked on lexicon size, POS
+tagger, intent detector, paragraph categorizer) follow the same pattern:
+new module in `src/khmerthings/`, re-export in `__init__.py`, new CLI
+subcommand in `cli.py`, new `tests/test_<module>.py`, and a
+**per-tool document `docs/<tool>.md`** (see below).
+
+## Invariants to preserve (enforced by tests)
+
+- `"".join(segment_clusters(t)) == unicodedata.normalize("NFC", t)` — cluster
+  segmentation never drops or reorders characters, even on malformed input.
+- Tokenization is lossless: concatenated token texts equal the NFC input, and
+  token offsets are contiguous.
+- All text is NFC-normalized at entry points; lexicon entries must be NFC,
+  Khmer-letters-only, and unique (loader raises otherwise).
+- A lexicon match can never split a character cluster.
+
+## Documentation upkeep (do this every change)
+
+Docs must always reflect the current state of the code. As part of any
+change — not as an afterthought:
+
+- **Self-update this file (AGENTS.md)** when architecture, constraints,
+  commands, tools, or conventions change (e.g. a new module or subcommand).
+- **Every main (end-user) tool has its own `docs/<tool>.md`** written for
+  community users *and for AI agents driving the tools autonomously*,
+  following the shared template: What it does / Quick start / CLI
+  reference / Python API / How it works / Guarantees & limitations / Task
+  recipes / Related tools. Requirements: **every** CLI flag and API
+  parameter gets a concrete example with its output; exit codes and error
+  output are documented; a "Task recipes" table maps goals → exact
+  commands/calls. Low-level primitives are documented via docstrings only,
+  not `docs/`. All example outputs must be real — run the command and
+  paste the actual output, never invent it (a wrong "expected" output has
+  already been caught this way).
+- **Update the other docs** touched by the change: `README.md` (landing
+  page: tool table, roadmap, examples), the affected `docs/*.md`,
+  `DEVELOPMENT_GUIDE.md` (workflow, recipes), docstrings.
+- **Keep `CHANGELOG.md` current**: add an entry under `[Unreleased]` for
+  every user-visible change (Keep a Changelog format: Added / Changed /
+  Fixed / Removed). On release, rename `[Unreleased]` to the version + date.
+- A PR that changes behavior but touches no docs/changelog is incomplete —
+  the PR template checklist enforces this.
+
+## Releasing
+
+1. Bump `version` in `pyproject.toml` **and** `__version__` in
+   `src/khmerthings/__init__.py` (they must stay in sync).
+2. Turn the `[Unreleased]` section of `CHANGELOG.md` into `[X.Y.Z] - date`.
+3. `uv sync` (refresh lockfile), run all four checks, commit.
+4. `git tag vX.Y.Z && git push origin main --tags`.
+5. The tag push triggers `.github/workflows/publish.yml`, which re-runs the
+   checks, verifies the tag matches `pyproject.toml`, builds, smoke-tests the
+   wheel, and publishes to PyPI (needs the `PYPI_API_TOKEN` repo secret,
+   configured in repo settings).
+
+## Conventions
+
+- Python ≥ 3.11, mypy `strict`, ruff line length 100.
+- Public API re-exported in `__init__.py` with `__all__`; keep `py.typed`.
+- Frozen dataclasses for result types (`Token`, `WordCount`).
+- Wordlist files (`words.txt`, `names.txt`, `modern.txt`): one entry per
+  line, UTF-8, NFC, Khmer letters/marks only, `#` comments, grouped by
+  category, sources noted in the header. High-frequency words with
+  subscript ta/da (្ត/្ដ) spelling variation are listed in both spellings —
+  real-world text mixes them. Within a file duplicates are a load error;
+  the same entry in different files is fine (merged at load).
+- Khmer test strings: verify codepoints carefully (visually identical strings
+  can differ, e.g. ្ត vs ្ដ); assert exact expected values, hand-verified.

khmerthings-0.4.3/CHANGELOG.md

@@ -0,0 +1,131 @@
+# Changelog
+
+All notable changes to khmerthings are documented here.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.4.3] - 2026-07-03
+
+### Added
+
+- GitHub Actions publish workflow: pushing a `vX.Y.Z` tag now runs the full
+  checks, verifies the tag matches the package version, builds, smoke-tests
+  the wheel, and uploads to PyPI (requires the `PYPI_API_TOKEN` secret).
+
+## [0.4.2] - 2026-07-03
+
+### Added
+
+- Lexicon: ខ្សោយ (weak) — reported missing via
+  `khmerthings count` showing it as an unknown span.
+
+## [0.4.1] - 2026-07-03
+
+### Changed
+
+- Per-tool docs rewritten for autonomous (AI-agent and scripted) use:
+  every CLI flag and Python API parameter now has a concrete example with
+  real executed output, exit codes and error output are documented, and
+  each doc ends with a "Task recipes" table mapping goals to exact
+  commands/calls.
+
+### Fixed
+
+- CLI: a missing/unreadable input file now exits with code 1 and a
+  one-line `khmerthings: error: ...` message instead of a Python traceback.
+
+## [0.4.0] - 2026-07-03
+
+### Added
+
+- Two new growable wordlist sources alongside the core vocabulary
+  (candidates researched from public sources — Wikipedia's Cambodian-name
+  article, Behind the Name, Khmer Wiktionary and Khmer media — and curated
+  entry by entry, spellings cross-checked):
+  - `names.txt` (200 entries): Khmer surnames, given names, and honorific
+    titles (ឯកឧត្តម, សម្តេច, …).
+  - `modern.txt` (30 entries): slang (ឡូយ, ស្ទាវ), informal register,
+    tech/media loanwords (ហ្វេសប៊ុក, អនឡាញ), and everyday modern loanwords.
+- `load_lexicon(*sources)` public API: merge any combination of `words`,
+  `names`, `modern` (cached, per-file validation, cross-file duplicates
+  merged); `WORD_SOURCES` lists the available sources.
+- `--include names,modern` flag on `khmerthings count` and
+  `khmerthings segment` to match against the extra wordlists.
+
+### Changed
+
+- Data policy clarified: web research to find/verify candidate entries is
+  allowed; bulk-importing third-party wordlists remains forbidden. Sources
+  are noted in each data file header.
+- Total curated entries: 802 across the three sources.
+
+## [0.3.0] - 2026-07-03
+
+### Added
+
+- Word breaker tool: `khmerthings.segmenter` with `break_words(text)`
+  (words as a list; length always equals `count_words`) and
+  `mark_boundaries(text, separator="​")` (insert separators at Khmer
+  word boundaries, everything else preserved).
+- `khmerthings segment [files|-] [--separator SEP] [--mark]` CLI subcommand.
+- Per-tool community documentation in `docs/`: `word-breaker.md`,
+  `word-counter.md`, `line-sorter.md` — each with quick start, CLI and
+  Python API reference, how-it-works, and guarantees/limitations. All
+  example outputs are real, executed outputs.
+- Lexicon batch 2: +290 hand-curated words (family & occupations, food &
+  drink, verbs, adjectives, connectives, society & learning, time & places);
+  now 582 words.
+
+### Changed
+
+- `README.md` restructured as a community landing page: tool table linking
+  to per-tool docs, roadmap (spellchecker/spellfixer next, pending lexicon
+  coverage), contributing pointers.
+- Doc-upkeep rules in `AGENTS.md`/`DEVELOPMENT_GUIDE.md`/PR template now
+  require a `docs/<tool>.md` for every main tool, with verified outputs.
+
+## [0.2.0] - 2026-07-03
+
+### Added
+
+- Khmer dictionary-order line sorting: `khmerthings.sorting` module with
+  `sort_lines(lines, descending=False)` and `khmer_sort_key(text)` (a
+  collation key usable directly with `sorted()`). Sorting is per character
+  cluster — base consonant, then subscript consonants, then dependent
+  vowels, then signs — approximating Chuon Nath dictionary order, which
+  naive codepoint sorting gets wrong for subscript consonants.
+- `khmerthings sort [files|-] [--desc]` CLI subcommand (stdin by default,
+  multiple files merged).
+- `CHANGELOG.md` (this file) and documentation-upkeep rules in `AGENTS.md`.
+
+## [0.1.0] - 2026-07-03
+
+### Added
+
+- Initial release: deterministic Khmer language toolkit with zero runtime
+  dependencies, built from first principles.
+- `khmerthings.chars`: Khmer Unicode character classification (consonants,
+  vowels, signs, coeng, digits, punctuation, script classes).
+- `khmerthings.clusters`: Khmer character-cluster (KCC) segmentation with a
+  losslessness invariant (never drops or reorders characters).
+- `khmerthings.lexicon`: hand-curated seed lexicon (~290 words) with
+  trie-based longest-match lookup keyed by clusters.
+- `khmerthings.tokenizer`: lossless typed tokenization of mixed
+  Khmer/Latin text; unknown Khmer spans preserved as `KHMER_UNKNOWN`.
+- `khmerthings.counter`: word counter (`count_words`, `analyze`) aware that
+  Khmer writes no spaces between words.
+- `khmerthings count [files|-] [--json]` CLI, installable globally.
+- Full test suite (207 tests), strict mypy, ruff, GitHub Actions CI
+  (lint, Python 3.11–3.14 test matrix, build + wheel smoke test), MIT
+  license, AGENTS.md/CLAUDE.md, DEVELOPMENT_GUIDE.md, PR template.
+
+[Unreleased]: https://github.com/spkskx/khmerthings/compare/v0.4.2...HEAD
+[0.4.2]: https://github.com/spkskx/khmerthings/compare/v0.4.1...v0.4.2
+[0.4.1]: https://github.com/spkskx/khmerthings/compare/v0.4.0...v0.4.1
+[0.4.0]: https://github.com/spkskx/khmerthings/compare/v0.3.0...v0.4.0
+[0.3.0]: https://github.com/spkskx/khmerthings/compare/v0.2.0...v0.3.0
+[0.2.0]: https://github.com/spkskx/khmerthings/compare/v0.1.0...v0.2.0
+[0.1.0]: https://github.com/spkskx/khmerthings/releases/tag/v0.1.0

khmerthings-0.4.3/CLAUDE.md

@@ -0,0 +1 @@
+Refer to @AGENTS.md for project guidance, constraints, commands, and conventions.

khmerthings-0.4.3/DEVELOPMENT_GUIDE.md

@@ -0,0 +1,123 @@
+# Development guide
+
+How to work on khmerthings. Agent-oriented rules live in [AGENTS.md](AGENTS.md);
+this guide is for humans. The short version: deterministic algorithms only,
+self-owned data only, and tests come first.
+
+## Setup
+
+Requires [uv](https://docs.astral.sh/uv/). Python 3.11+ is provisioned
+automatically.
+
+```sh
+git clone git@github.com:spkskx/khmerthings.git
+cd khmerthings
+uv sync          # creates .venv with dev dependencies
+```
+
+## Everyday commands
+
+```sh
+uv run pytest                          # run the test suite
+uv run pytest tests/test_clusters.py  # one file
+uv run mypy src tests                  # type check (strict)
+uv run ruff check --fix                # lint (+autofix)
+uv run ruff format                     # format
+uv run khmerthings count file.txt      # run the CLI from source
+uv build                               # build sdist + wheel
+```
+
+All of pytest, mypy, `ruff check`, and `ruff format --check` must pass before
+a change is done — CI enforces exactly these on every PR.
+
+## Branching & PRs
+
+- `main` is the only long-lived branch and is what CI, tags, and releases
+  point at. Work on short-lived feature branches and open a PR into `main`.
+- CI runs lint/type checks, the test matrix (Python 3.11–3.14), a package
+  build, and a wheel smoke test. All jobs must be green to merge.
+- Keep `uv.lock` committed; CI installs with `uv sync --locked`.
+
+## Project rules
+
+1. **Deterministic only.** Same input → same output, always. Dictionary- and
+   rule-based algorithms; no ML inference, no LLMs, no randomness.
+2. **Self-owned everything.** No third-party Khmer NLP libraries and no
+   bulk-imported wordlists/corpora. Word data is hand-curated entry by
+   entry in `src/khmerthings/data/` — web research to find and verify
+   candidates is fine; copying someone's wordlist wholesale is not. Note
+   candidate sources in the data file header.
+3. **Zero runtime dependencies.** Stdlib only; dev tooling is the only
+   exception.
+4. **Tests first.** Every behavior change comes with tests. Prefer
+   table-driven `pytest.mark.parametrize` cases plus invariant tests (see
+   `tests/test_clusters.py::TestInvariants` for the pattern).
+
+## Adding a new tool
+
+Follow the existing bottom-up architecture (chars → clusters → lexicon →
+tokenizer → tools). To add a tool, e.g. a spellchecker:
+
+1. Create `src/khmerthings/spellcheck.py`, building on the existing
+   primitives (`segment_clusters`, `Lexicon.longest_match`, `tokenize`).
+2. Re-export the public API in `src/khmerthings/__init__.py` (`__all__`).
+3. Add a CLI subcommand in `src/khmerthings/cli.py` (`khmerthings spellcheck`).
+4. Add `tests/test_spellcheck.py` with unit + invariant tests.
+5. Write its per-tool document `docs/spellcheck.md` following the shared
+   template used by the existing docs (What it does / Quick start / CLI
+   reference / Python API / How it works / Guarantees & limitations /
+   Related tools). All example outputs must be real, verified outputs.
+6. Add it to the tool table and roadmap in `README.md`, and to `AGENTS.md`.
+
+## Editing the wordlists (`src/khmerthings/data/`)
+
+Three growable files, merged on demand via `load_lexicon(*sources)`:
+
+| File | Source name | Contents |
+|---|---|---|
+| `words.txt` | `words` | core vocabulary (always loaded) |
+| `names.txt` | `names` | personal names, surnames, honorific titles |
+| `modern.txt` | `modern` | slang, informal register, loanwords, trending terms |
+
+Put new entries in the right file: standard dictionary vocabulary → `words`;
+anything people are called → `names`; register that shifts with time
+(slang, borrowings, internet vocabulary) → `modern`.
+
+- One entry per line, UTF-8, NFC-normalized, Khmer letters/marks only —
+  the loader rejects anything else (including within-file duplicates) at
+  load time. The same entry may appear in different files (merged at load).
+- Keep entries in their category section; add a new `# --- section ---` when
+  needed. Note research sources in the file header comments.
+- Words with subscript ta/da (្ត/្ដ) spelling variation should be added in
+  **both** spellings — the two render identically and real text mixes them.
+- Beware visually identical strings with different codepoints; verify with
+  `python -c "print([hex(ord(c)) for c in 'word'])"` when in doubt.
+- Adding words is the highest-leverage accuracy improvement for every tool.
+
+## Khmer script primer (why the code looks like this)
+
+- Khmer writes **no spaces between words**; spaces separate phrases. Word
+  boundaries must be inferred via the lexicon.
+- A **character cluster** (base consonant + subscript consonants + vowel +
+  signs) is the smallest indivisible unit; word boundaries can only fall on
+  cluster boundaries. `clusters.py` implements this.
+- The zero-width space (U+200B) is used in real Khmer text as an explicit
+  word delimiter; the tokenizer treats it as whitespace.
+
+## Documentation & changelog
+
+Docs are part of every change, not a follow-up: update `README.md`,
+`AGENTS.md`, `DEVELOPMENT_GUIDE.md`, and docstrings whenever behavior,
+architecture, or workflow changes, and add an entry to the `[Unreleased]`
+section of `CHANGELOG.md` (Keep a Changelog format) for every user-visible
+change. A behavior-changing PR with no docs/changelog update is incomplete.
+
+## Releasing
+
+1. Bump `version` in `pyproject.toml` and `__version__` in
+   `src/khmerthings/__init__.py` (keep them in sync).
+2. Rename the `[Unreleased]` section of `CHANGELOG.md` to `[X.Y.Z] - date`
+   and update the compare links at the bottom.
+3. Update the lockfile: `uv sync`.
+4. Commit, then tag: `git tag vX.Y.Z && git push origin main --tags`.
+5. Build artifacts with `uv build` (CI also builds and uploads them).

khmerthings-0.4.3/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 spkskx
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

khmerthings-0.4.3/PKG-INFO

@@ -0,0 +1,101 @@
+Metadata-Version: 2.4
+Name: khmerthings
+Version: 0.4.3
+Summary: Deterministic Khmer language tools: word counter, segmentation primitives, and more.
+Author: spkskx
+License: MIT
+License-File: LICENSE
+Keywords: khmer,nlp,segmentation,unicode,word-count
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+
+# khmerthings
+
+Deterministic Khmer language tools for Python — built as **community
+building blocks**: small, correct, dependency-free primitives you can
+compose into bigger systems.
+
+No machine-learning models, no third-party NLP dependencies, no network
+calls. Every result is reproducible and explainable. Khmer script writes no
+spaces between words, so even "simple" operations like counting or sorting
+need real language handling — khmerthings implements that from first
+principles.
+
+## Tools
+
+Each tool is available both as a Python API and a CLI subcommand, and has
+its own detailed document:
+
+| Tool | CLI | Python | Docs |
+|---|---|---|---|
+| **Word breaker** — split Khmer text into words | `khmerthings segment` | `break_words`, `mark_boundaries` | [docs/word-breaker.md](docs/word-breaker.md) |
+| **Word counter** — count words in Khmer/mixed text | `khmerthings count` | `count_words`, `analyze` | [docs/word-counter.md](docs/word-counter.md) |
+| **Line sorter** — Khmer dictionary-order sorting | `khmerthings sort` | `sort_lines`, `khmer_sort_key` | [docs/line-sorter.md](docs/line-sorter.md) |
+
+## Install
+
+```sh
+pip install khmerthings          # library
+uv tool install khmerthings     # global CLI
+```
+
+## A taste
+
+```sh
+$ echo "ខ្ញុំស្រឡាញ់ភាសាខ្មែរ" | khmerthings segment
+ខ្ញុំ ស្រឡាញ់ ភាសា ខ្មែរ
+```
+
+```python
+from khmerthings import break_words, count_words, sort_lines
+
+break_words("ខ្ញុំស្រឡាញ់ភាសាខ្មែរ")   # ['ខ្ញុំ', 'ស្រឡាញ់', 'ភាសា', 'ខ្មែរ']
+count_words("ខ្ញុំមានឆ្កែ ២ ក្បាល and 3 cats")   # 8
+sort_lines(["ក្រ", "កា", "កក"])                    # ['កក', 'កា', 'ក្រ']
+```
+
+## Design principles
+
+- **Deterministic**: same input, same output, always. Rule- and
+  dictionary-based algorithms only; nothing probabilistic.
+- **Self-contained**: zero runtime dependencies; all word data is our own
+  hand-curated set of growable wordlists — `words` (core vocabulary),
+  `names` (people's names & titles), and `modern` (slang, loanwords,
+  trending terms) — 802 entries and growing. Candidates are researched from
+  public sources and verified entry by entry; no wordlist is imported
+  wholesale.
+- **Lossless**: no character is ever dropped — unknown Khmer spans are
+  reported, not discarded.
+- **Tested first**: every module ships with table-driven unit tests and
+  invariant checks (258 tests as of v0.3.0).
+
+Under the hood, the tools share deterministic primitives (character
+classification, character-cluster segmentation, a cluster-keyed lexicon
+trie, lossless tokenization) in `src/khmerthings/` — see the module
+docstrings if you want to build on them directly.
+
+## Roadmap
+
+- ✅ Word counter, line sorter, word breaker
+- ⏳ Wordlist growth across all three sources (`words`, `names`, `modern`) —
+  hand-curated batches each release; the accuracy lever for every
+  dictionary-based tool
+- 🔜 Spellchecker & spellfixer (engine is feasible today; waiting on lexicon
+  coverage to make its verdicts trustworthy)
+- Later: part-of-speech tagger, intent detection, paragraph categorization
+
+## Contributing
+
+See [DEVELOPMENT_GUIDE.md](DEVELOPMENT_GUIDE.md) for setup, the
+architecture, the rules (determinism, self-owned data, tests first), and
+how to add words to the lexicon — the single most valuable contribution.
+Changes are tracked in [CHANGELOG.md](CHANGELOG.md).
+
+## License
+
+[MIT](LICENSE)