orthomni-codeco 0.1.0__tar.gz

orthomni_codeco-0.1.0/.gitignore

@@ -0,0 +1,11 @@
+__pycache__/
+*.pyc
+*.egg-info/
+.pytest_cache/
+.venv/
+build/
+dist/
+# Model INT8 cục bộ (~1.1 GB) - phân phối qua Hugging Face Hub, không commit/đóng gói.
+model/
+*.staging/
+_legacy/

orthomni_codeco-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Nhat H. Pham
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

orthomni_codeco-0.1.0/PKG-INFO

@@ -0,0 +1,218 @@
+Metadata-Version: 2.5
+Name: orthomni-codeco
+Version: 0.1.0
+Summary: Vietnamese spelling correction (BARTpho seq2seq) on CPU with ONNX Runtime INT8 - no PyTorch needed
+Project-URL: Homepage, https://huggingface.co/nhatho1999/orthomni-codeco
+Author-email: "Nhat H. Pham" <nhatho1999.ai@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: bartpho,nlp,onnx,onnxruntime,spelling-correction,vietnamese
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Natural Language :: Vietnamese
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Typing :: Typed
+Requires-Python: >=3.8
+Requires-Dist: huggingface-hub>=0.20
+Requires-Dist: numpy>=1.21
+Requires-Dist: onnxruntime>=1.16
+Requires-Dist: sentencepiece>=0.1.99
+Provides-Extra: dev
+Requires-Dist: build>=1.0; extra == 'dev'
+Requires-Dist: httpx>=0.27; extra == 'dev'
+Requires-Dist: pytest>=7; extra == 'dev'
+Requires-Dist: twine>=5.0; extra == 'dev'
+Provides-Extra: export
+Requires-Dist: optimum-onnx[onnxruntime]>=0.1.0; extra == 'export'
+Requires-Dist: torch>=2.1; extra == 'export'
+Requires-Dist: transformers<4.58,>=4.45; extra == 'export'
+Provides-Extra: server
+Requires-Dist: fastapi>=0.110; extra == 'server'
+Requires-Dist: pydantic>=2.6; extra == 'server'
+Requires-Dist: uvicorn[standard]>=0.29; extra == 'server'
+Description-Content-Type: text/markdown
+
+# orthomni-codeco
+
+Vietnamese spelling correction on **CPU** with **ONNX Runtime INT8**, **no PyTorch required**. The model is a Seq2Seq BARTpho/MBart (~400M parameters).
+
+- **Lightweight:** the only dependencies are `onnxruntime`, `numpy`, `sentencepiece` and `huggingface-hub` (~120 MB, versus ~1.3 GB with torch + transformers).
+- **Automatic model download:** the wheel contains no weights. On first run, the INT8 model (~1.1 GB) is downloaded from the Hugging Face Hub to `~/.cache/orthomni_codeco/` with a progress bar. Later runs work offline.
+- **Batteries included:** FastAPI server, CLI and Dockerfile.
+
+## Installation
+
+```bash
+pip install orthomni-codeco            # library
+pip install "orthomni-codeco[server]"  # + FastAPI/uvicorn
+```
+
+Requires Python >= 3.8.
+
+## Quickstart
+
+```python
+from orthomni_codeco import OrthoMNIModel
+
+model = OrthoMNIModel()                         # first run: downloads the model to the cache, then loads it (~2 s)
+model.correct("hom nay troi dep qua")           # 'Hôm nay trời đẹp quá.'
+model.correct(["xin chao", "toi di hoc"])       # ['Xin chào.', 'Tôi đi học.']
+```
+
+`predict()` and `generate()` are aliases of `correct()`. `Corrector` is the old name of `OrthoMNIModel`.
+
+## Library usage
+
+```python
+from orthomni_codeco import OrthoMNIModel, CorrectorConfig, InputTooLongError, get_corrector, ensure_model
+
+model = get_corrector()                                   # per-process singleton, config read from ORTHOMNI_* env vars
+model = OrthoMNIModel(CorrectorConfig(intra_op_threads=4))
+model = OrthoMNIModel(CorrectorConfig(model_dir="/path/to/int8"))  # use a local model, no download
+
+await model.acorrect("xin chao")                          # async, does not block the event loop
+
+try:
+    model.correct(very_long_text)
+except InputTooLongError as e:
+    print(e.indices, e.max_tokens)                        # which inputs exceed the limit -> split sentences first
+
+ensure_model()                                            # only download the model to the cache, returns its path
+```
+
+- **Load once:** create `OrthoMNIModel` once and reuse it.
+- **Thread-safe:** calls are serialized with a lock, since ONNX Runtime already uses all the threads it is given.
+- **Lightweight import:** `import orthomni_codeco` does not load onnxruntime or numpy.
+
+### Model source
+
+In order of priority:
+1. `model_dir` / `ORTHOMNI_MODEL_DIR`: a local model directory, nothing is downloaded.
+2. `repo_id` + `revision` (`ORTHOMNI_REPO_ID`, `ORTHOMNI_REVISION`): downloaded from the Hugging Face Hub to `cache_dir` (`ORTHOMNI_CACHE_DIR`, default `~/.cache/orthomni_codeco/<org>--<repo>/<revision>/`).
+
+Private repos use `HF_TOKEN` or `hf auth login`. For offline environments, pre-download with `orthomni download`, then set `HF_HUB_OFFLINE=1`.
+
+## Mounting into an existing FastAPI app
+
+```python
+import asyncio
+from contextlib import asynccontextmanager
+from fastapi import FastAPI
+from orthomni_codeco import get_corrector
+from orthomni_codeco.server import create_router
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    await asyncio.to_thread(get_corrector)   # load the model at startup
+    yield
+
+app = FastAPI(lifespan=lifespan)
+app.include_router(create_router(), prefix="/spell")   # -> POST /spell/predict, GET /spell/health
+```
+
+## CLI and standalone server
+
+```bash
+orthomni download                                   # download the model to the cache, print its path
+orthomni correct "toi di hoc" "xin chao"            # quick test
+orthomni serve --port 8006                          # workers/threads derived from the vCPU count
+orthomni serve --workers 1 --threads 8              # lowest latency
+```
+
+`python -m orthomni_codeco ...` is equivalent to `orthomni ...`.
+
+| Method | Path | Description |
+|---|---|---|
+| GET | `/health` | `200 {"status":"ok"}` when the model is ready, `503` otherwise |
+| POST | `/predict` | Body `{"text": str}` **or** `{"texts": list[str]}` |
+
+```bash
+curl -X POST localhost:8006/predict -H 'content-type: application/json' -d '{"text":"hom nay troi dep qua"}'
+# {"result":"Hôm nay trời đẹp quá.","latency_ms":113.7}
+```
+
+Error codes: `422` invalid input (both `text` and `texts`, batch too large, empty text), `413` sentence longer than `MAX_INPUT_TOKENS` (the server does not truncate), `500` inference error.
+
+## Configuration (environment variables)
+
+| Variable | Default | Meaning |
+|---|---|---|
+| `ORTHOMNI_MODEL_DIR` | (empty) | Local model directory; when set, no download happens |
+| `ORTHOMNI_REPO_ID` / `ORTHOMNI_REVISION` | see `config.py` | Hugging Face Hub repo and revision |
+| `ORTHOMNI_CACHE_DIR` | `~/.cache/orthomni_codeco` | Model cache directory |
+| `ORTHOMNI_INTRA_OP_THREADS` | `0` (= vCPU / workers) | ORT threads within one operator. **Biggest effect on latency** |
+| `ORTHOMNI_INTER_OP_THREADS` | `1` | Parallelism across graph branches |
+| `WEB_CONCURRENCY` | `vCPU / 4` | Number of uvicorn workers; each holds its own model copy (~1.7 GB RAM) |
+| `ORTHOMNI_MAX_INPUT_TOKENS` | `256` | Maximum tokens per sentence |
+| `ORTHOMNI_INFER_BATCH_SIZE` | `8` | Sub-batch size (sorted by length to reduce padding) |
+| `ORTHOMNI_NEW_TOKENS_RATIO` / `_OFFSET` / `_CAP` | `1.3` / `8` / `320` | `max_new_tokens = min(CAP, len_input × RATIO + OFFSET)` |
+| `ORTHOMNI_WARMUP` | `true` | Run one dummy inference on load |
+| `ORTHOMNI_MAX_BATCH_SIZE` | `32` | Maximum number of `texts` per request |
+| `ORTHOMNI_MAX_TEXT_CHARS` | `4000` | Maximum characters per text |
+| `ORTHOMNI_MAX_CONCURRENCY` | `1` | Concurrent generate requests per worker |
+
+**Avoid thread thrashing:** keep `workers × ORTHOMNI_INTRA_OP_THREADS ≤ vCPUs`. The `serve` command reads CPU limits from cgroups (`docker --cpus`, K8s limits). Fewer workers with more threads gives lower latency; more workers with fewer threads gives higher throughput.
+
+## Docker
+
+Requires the INT8 model in `./model` (see below):
+
+```bash
+docker build -t orthomni-codeco:cpu .
+docker run --rm -p 8006:8006 --cpus 8 --memory 4g orthomni-codeco:cpu
+```
+
+## Development
+
+### Environment setup
+
+```bash
+python -m venv .venv && . .venv/bin/activate
+pip install -e ".[server,dev]"
+ORTHOMNI_MODEL_DIR=./model pytest -q        # model tests are skipped if ./model is missing
+./run.sh                                    # local server using ./model, port 7428
+```
+
+### Exporting a new model
+
+The current INT8 model is quantized for **arm64** (see `model_info.json`). It still runs on x86 but not optimally, so re-export on a machine with the same CPU architecture as the serving machine:
+
+```bash
+pip install torch --index-url https://download.pytorch.org/whl/cpu   # CPU-only torch first
+pip install -e ".[export]"
+orthomni export --model_dir /path/to/checkpoint --output_dir ./model [--target avx512_vnni]
+```
+
+Quantization is dynamic INT8 **per-channel**, applied only to `MatMul`; embeddings stay FP32. Per-tensor quantization produced incorrect sentences.
+
+### Publishing the model to the Hugging Face Hub
+
+```bash
+hf auth login
+python scripts/upload_model.py --model_dir ./model --repo_id <org>/orthomni-codeco
+```
+
+The script prints a commit hash. Update `DEFAULT_REPO_ID` and `DEFAULT_REVISION` in `src/orthomni_codeco/config.py` before releasing, so each package version always downloads the same set of weights.
+
+### Build and publish
+
+```bash
+pip install -U build twine
+rm -rf dist/ && python -m build                  # -> dist/*.whl + dist/*.tar.gz
+twine check dist/*                               # check metadata and that the README renders on PyPI
+unzip -l dist/*.whl                              # confirm no .onnx files are inside
+
+# Try the wheel in a clean venv
+python -m venv /tmp/t && /tmp/t/bin/pip install dist/*.whl
+/tmp/t/bin/python -c "from orthomni_codeco import OrthoMNIModel; print(OrthoMNIModel().correct('xin chao'))"
+
+twine upload --repository testpypi dist/*        # try on TestPyPI first
+twine upload dist/*                              # real PyPI
+```
+
+## License
+
+MIT

orthomni_codeco-0.1.0/README.md

@@ -0,0 +1,181 @@
+# orthomni-codeco
+
+Vietnamese spelling correction on **CPU** with **ONNX Runtime INT8**, **no PyTorch required**. The model is a Seq2Seq BARTpho/MBart (~400M parameters).
+
+- **Lightweight:** the only dependencies are `onnxruntime`, `numpy`, `sentencepiece` and `huggingface-hub` (~120 MB, versus ~1.3 GB with torch + transformers).
+- **Automatic model download:** the wheel contains no weights. On first run, the INT8 model (~1.1 GB) is downloaded from the Hugging Face Hub to `~/.cache/orthomni_codeco/` with a progress bar. Later runs work offline.
+- **Batteries included:** FastAPI server, CLI and Dockerfile.
+
+## Installation
+
+```bash
+pip install orthomni-codeco            # library
+pip install "orthomni-codeco[server]"  # + FastAPI/uvicorn
+```
+
+Requires Python >= 3.8.
+
+## Quickstart
+
+```python
+from orthomni_codeco import OrthoMNIModel
+
+model = OrthoMNIModel()                         # first run: downloads the model to the cache, then loads it (~2 s)
+model.correct("hom nay troi dep qua")           # 'Hôm nay trời đẹp quá.'
+model.correct(["xin chao", "toi di hoc"])       # ['Xin chào.', 'Tôi đi học.']
+```
+
+`predict()` and `generate()` are aliases of `correct()`. `Corrector` is the old name of `OrthoMNIModel`.
+
+## Library usage
+
+```python
+from orthomni_codeco import OrthoMNIModel, CorrectorConfig, InputTooLongError, get_corrector, ensure_model
+
+model = get_corrector()                                   # per-process singleton, config read from ORTHOMNI_* env vars
+model = OrthoMNIModel(CorrectorConfig(intra_op_threads=4))
+model = OrthoMNIModel(CorrectorConfig(model_dir="/path/to/int8"))  # use a local model, no download
+
+await model.acorrect("xin chao")                          # async, does not block the event loop
+
+try:
+    model.correct(very_long_text)
+except InputTooLongError as e:
+    print(e.indices, e.max_tokens)                        # which inputs exceed the limit -> split sentences first
+
+ensure_model()                                            # only download the model to the cache, returns its path
+```
+
+- **Load once:** create `OrthoMNIModel` once and reuse it.
+- **Thread-safe:** calls are serialized with a lock, since ONNX Runtime already uses all the threads it is given.
+- **Lightweight import:** `import orthomni_codeco` does not load onnxruntime or numpy.
+
+### Model source
+
+In order of priority:
+1. `model_dir` / `ORTHOMNI_MODEL_DIR`: a local model directory, nothing is downloaded.
+2. `repo_id` + `revision` (`ORTHOMNI_REPO_ID`, `ORTHOMNI_REVISION`): downloaded from the Hugging Face Hub to `cache_dir` (`ORTHOMNI_CACHE_DIR`, default `~/.cache/orthomni_codeco/<org>--<repo>/<revision>/`).
+
+Private repos use `HF_TOKEN` or `hf auth login`. For offline environments, pre-download with `orthomni download`, then set `HF_HUB_OFFLINE=1`.
+
+## Mounting into an existing FastAPI app
+
+```python
+import asyncio
+from contextlib import asynccontextmanager
+from fastapi import FastAPI
+from orthomni_codeco import get_corrector
+from orthomni_codeco.server import create_router
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    await asyncio.to_thread(get_corrector)   # load the model at startup
+    yield
+
+app = FastAPI(lifespan=lifespan)
+app.include_router(create_router(), prefix="/spell")   # -> POST /spell/predict, GET /spell/health
+```
+
+## CLI and standalone server
+
+```bash
+orthomni download                                   # download the model to the cache, print its path
+orthomni correct "toi di hoc" "xin chao"            # quick test
+orthomni serve --port 8006                          # workers/threads derived from the vCPU count
+orthomni serve --workers 1 --threads 8              # lowest latency
+```
+
+`python -m orthomni_codeco ...` is equivalent to `orthomni ...`.
+
+| Method | Path | Description |
+|---|---|---|
+| GET | `/health` | `200 {"status":"ok"}` when the model is ready, `503` otherwise |
+| POST | `/predict` | Body `{"text": str}` **or** `{"texts": list[str]}` |
+
+```bash
+curl -X POST localhost:8006/predict -H 'content-type: application/json' -d '{"text":"hom nay troi dep qua"}'
+# {"result":"Hôm nay trời đẹp quá.","latency_ms":113.7}
+```
+
+Error codes: `422` invalid input (both `text` and `texts`, batch too large, empty text), `413` sentence longer than `MAX_INPUT_TOKENS` (the server does not truncate), `500` inference error.
+
+## Configuration (environment variables)
+
+| Variable | Default | Meaning |
+|---|---|---|
+| `ORTHOMNI_MODEL_DIR` | (empty) | Local model directory; when set, no download happens |
+| `ORTHOMNI_REPO_ID` / `ORTHOMNI_REVISION` | see `config.py` | Hugging Face Hub repo and revision |
+| `ORTHOMNI_CACHE_DIR` | `~/.cache/orthomni_codeco` | Model cache directory |
+| `ORTHOMNI_INTRA_OP_THREADS` | `0` (= vCPU / workers) | ORT threads within one operator. **Biggest effect on latency** |
+| `ORTHOMNI_INTER_OP_THREADS` | `1` | Parallelism across graph branches |
+| `WEB_CONCURRENCY` | `vCPU / 4` | Number of uvicorn workers; each holds its own model copy (~1.7 GB RAM) |
+| `ORTHOMNI_MAX_INPUT_TOKENS` | `256` | Maximum tokens per sentence |
+| `ORTHOMNI_INFER_BATCH_SIZE` | `8` | Sub-batch size (sorted by length to reduce padding) |
+| `ORTHOMNI_NEW_TOKENS_RATIO` / `_OFFSET` / `_CAP` | `1.3` / `8` / `320` | `max_new_tokens = min(CAP, len_input × RATIO + OFFSET)` |
+| `ORTHOMNI_WARMUP` | `true` | Run one dummy inference on load |
+| `ORTHOMNI_MAX_BATCH_SIZE` | `32` | Maximum number of `texts` per request |
+| `ORTHOMNI_MAX_TEXT_CHARS` | `4000` | Maximum characters per text |
+| `ORTHOMNI_MAX_CONCURRENCY` | `1` | Concurrent generate requests per worker |
+
+**Avoid thread thrashing:** keep `workers × ORTHOMNI_INTRA_OP_THREADS ≤ vCPUs`. The `serve` command reads CPU limits from cgroups (`docker --cpus`, K8s limits). Fewer workers with more threads gives lower latency; more workers with fewer threads gives higher throughput.
+
+## Docker
+
+Requires the INT8 model in `./model` (see below):
+
+```bash
+docker build -t orthomni-codeco:cpu .
+docker run --rm -p 8006:8006 --cpus 8 --memory 4g orthomni-codeco:cpu
+```
+
+## Development
+
+### Environment setup
+
+```bash
+python -m venv .venv && . .venv/bin/activate
+pip install -e ".[server,dev]"
+ORTHOMNI_MODEL_DIR=./model pytest -q        # model tests are skipped if ./model is missing
+./run.sh                                    # local server using ./model, port 7428
+```
+
+### Exporting a new model
+
+The current INT8 model is quantized for **arm64** (see `model_info.json`). It still runs on x86 but not optimally, so re-export on a machine with the same CPU architecture as the serving machine:
+
+```bash
+pip install torch --index-url https://download.pytorch.org/whl/cpu   # CPU-only torch first
+pip install -e ".[export]"
+orthomni export --model_dir /path/to/checkpoint --output_dir ./model [--target avx512_vnni]
+```
+
+Quantization is dynamic INT8 **per-channel**, applied only to `MatMul`; embeddings stay FP32. Per-tensor quantization produced incorrect sentences.
+
+### Publishing the model to the Hugging Face Hub
+
+```bash
+hf auth login
+python scripts/upload_model.py --model_dir ./model --repo_id <org>/orthomni-codeco
+```
+
+The script prints a commit hash. Update `DEFAULT_REPO_ID` and `DEFAULT_REVISION` in `src/orthomni_codeco/config.py` before releasing, so each package version always downloads the same set of weights.
+
+### Build and publish
+
+```bash
+pip install -U build twine
+rm -rf dist/ && python -m build                  # -> dist/*.whl + dist/*.tar.gz
+twine check dist/*                               # check metadata and that the README renders on PyPI
+unzip -l dist/*.whl                              # confirm no .onnx files are inside
+
+# Try the wheel in a clean venv
+python -m venv /tmp/t && /tmp/t/bin/pip install dist/*.whl
+/tmp/t/bin/python -c "from orthomni_codeco import OrthoMNIModel; print(OrthoMNIModel().correct('xin chao'))"
+
+twine upload --repository testpypi dist/*        # try on TestPyPI first
+twine upload dist/*                              # real PyPI
+```
+
+## License
+
+MIT

orthomni_codeco-0.1.0/pyproject.toml

@@ -0,0 +1,54 @@
+[build-system]
+requires = ["hatchling>=1.27"]
+build-backend = "hatchling.build"
+
+[project]
+name = "orthomni-codeco"
+version = "0.1.0"
+description = "Vietnamese spelling correction (BARTpho seq2seq) on CPU with ONNX Runtime INT8 - no PyTorch needed"
+readme = "README.md"
+requires-python = ">=3.8"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [
+    { name = "Nhat H. Pham", email = "nhatho1999.ai@gmail.com" },
+]
+keywords = ["vietnamese", "spelling-correction", "onnx", "onnxruntime", "bartpho", "nlp"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Natural Language :: Vietnamese",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Text Processing :: Linguistic",
+    "Typing :: Typed",
+]
+# Inference chỉ cần ONNX Runtime + numpy + sentencepiece. Model (~1.1 GB) tải từ Hugging Face Hub lúc chạy.
+dependencies = [
+    "onnxruntime>=1.16",
+    "numpy>=1.21",
+    "sentencepiece>=0.1.99",
+    "huggingface-hub>=0.20",
+]
+
+[project.optional-dependencies]
+server = ["fastapi>=0.110", "uvicorn[standard]>=0.29", "pydantic>=2.6"]
+# Chỉ cần khi export checkpoint PyTorch -> ONNX INT8. Nên cài torch CPU-only trước (xem README).
+export = ["torch>=2.1", "optimum-onnx[onnxruntime]>=0.1.0", "transformers>=4.45,<4.58"]
+dev = ["pytest>=7", "httpx>=0.27", "build>=1.0", "twine>=5.0"]
+
+[project.urls]
+Homepage = "https://huggingface.co/nhatho1999/orthomni-codeco"
+
+[project.scripts]
+orthomni = "orthomni_codeco.__main__:cli"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/orthomni_codeco"]
+
+[tool.hatch.build.targets.sdist]
+include = ["src/", "tests/", "README.md", "LICENSE", "pyproject.toml"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

orthomni_codeco-0.1.0/src/orthomni_codeco/__init__.py

@@ -0,0 +1,43 @@
+"""orthomni_codeco - Sửa chính tả tiếng Việt (Seq2Seq BARTpho, ONNX Runtime INT8, CPU).
+
+    from orthomni_codeco import OrthoMNIModel
+    model = OrthoMNIModel()             # lần đầu tự tải model về ~/.cache/orthomni_codeco
+    model.correct("toi di hoc")         # -> "Tôi đi học."
+
+Import package rất nhẹ: onnxruntime/numpy chỉ được import khi dùng tới OrthoMNIModel.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+from .config import DEFAULT_REPO_ID, DEFAULT_REVISION, CorrectorConfig, ServerConfig, available_cpus
+from .downloader import ensure_model
+from .exceptions import InputTooLongError, ModelNotFoundError, OrthomniError
+
+if TYPE_CHECKING:
+    from .model import Corrector, OrthoMNIModel, get_corrector
+
+__version__ = "0.1.0"
+__all__ = [
+    "OrthoMNIModel",
+    "Corrector",
+    "CorrectorConfig",
+    "ServerConfig",
+    "get_corrector",
+    "ensure_model",
+    "available_cpus",
+    "DEFAULT_REPO_ID",
+    "DEFAULT_REVISION",
+    "OrthomniError",
+    "ModelNotFoundError",
+    "InputTooLongError",
+]
+
+
+def __getattr__(name: str) -> Any:
+    if name in ("OrthoMNIModel", "Corrector", "get_corrector"):
+        from . import model
+
+        return getattr(model, name)
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

orthomni_codeco-0.1.0/src/orthomni_codeco/__main__.py

@@ -0,0 +1,141 @@
+"""CLI:
+    python -m orthomni_codeco serve  [--port 8006] [--workers N] [--threads T]
+    python -m orthomni_codeco export --model_dir /path/to/checkpoint [--target auto]
+    python -m orthomni_codeco correct "toi di hoc" "xin chao"
+    python -m orthomni_codeco download [--repo_id ...] [--revision ...]
+"""
+
+from __future__ import annotations
+
+import argparse
+import logging
+import os
+import sys
+from typing import List, Optional
+
+
+def _thread_env_defaults() -> None:
+    # Tính toán nặng nằm trong thread pool của ONNX Runtime; OpenMP/BLAS (numpy, torch khi export)
+    # để 1 luồng để tránh thread thrashing. Phải set trước khi import các thư viện này.
+    for var in ("OMP_NUM_THREADS", "MKL_NUM_THREADS", "OPENBLAS_NUM_THREADS"):
+        os.environ.setdefault(var, "1")
+    os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")
+
+
+def cmd_serve(args: argparse.Namespace) -> int:
+    import uvicorn
+
+    from .config import available_cpus
+
+    # Quy tắc: workers x intra_op_threads <= số vCPU (đã tính cgroup quota).
+    cpus = available_cpus()
+    workers = args.workers or int(os.getenv("WEB_CONCURRENCY", "0")) or max(1, cpus // 4)
+    threads = args.threads or int(os.getenv("ORT_INTRA_OP_THREADS", os.getenv("ORTHOMNI_INTRA_OP_THREADS", "0"))) \
+        or max(1, cpus // workers)
+    # Worker là process con -> truyền cấu hình qua env.
+    os.environ["WEB_CONCURRENCY"] = str(workers)
+    os.environ["ORTHOMNI_INTRA_OP_THREADS"] = str(threads)
+    # Tải model 1 lần ở process cha, tránh N worker cùng tải song song.
+    _ensure_model_from_env()
+    print(f"[orthomni] cpus={cpus} workers={workers} ort_intra_threads={threads}", flush=True)
+
+    uvicorn.run(
+        "orthomni_codeco.server:app_factory",
+        factory=True,
+        host=args.host,
+        port=args.port,
+        workers=workers,
+        access_log=False,
+        timeout_keep_alive=30,
+        timeout_graceful_shutdown=30,
+        limit_concurrency=args.limit_concurrency,
+        backlog=2048,
+    )
+    return 0
+
+
+def _ensure_model_from_env() -> None:
+    from .config import CorrectorConfig
+    from .downloader import ensure_model
+
+    cfg = CorrectorConfig.from_env()
+    if not cfg.model_dir:
+        ensure_model(cfg.repo_id, cfg.revision, cfg.cache_dir or None)
+
+
+def cmd_download(args: argparse.Namespace) -> int:
+    from .config import CorrectorConfig
+    from .downloader import ensure_model
+
+    cfg = CorrectorConfig.from_env()
+    path = ensure_model(
+        args.repo_id or cfg.repo_id,
+        args.revision or cfg.revision,
+        args.cache_dir or cfg.cache_dir or None,
+        force=args.force,
+    )
+    print(path)
+    return 0
+
+
+def cmd_correct(args: argparse.Namespace) -> int:
+    from .model import OrthoMNIModel
+
+    model = OrthoMNIModel()
+    for src, out in zip(args.texts, model.correct(args.texts)):
+        print(f"{src}\t->\t{out}")
+    return 0
+
+
+def main(argv: Optional[List[str]] = None) -> int:
+    _thread_env_defaults()
+    logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
+    parser = argparse.ArgumentParser(prog="python -m orthomni_codeco")
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    p_serve = sub.add_parser("serve", help="Chạy FastAPI server (cần extra [server]).")
+    p_serve.add_argument("--host", default=os.getenv("HOST", "0.0.0.0"))
+    p_serve.add_argument("--port", type=int, default=int(os.getenv("PORT", "8006")))
+    p_serve.add_argument("--workers", type=int, default=0, help="0 = 1 worker / 4 vCPU.")
+    p_serve.add_argument("--threads", type=int, default=0, help="Luồng ORT mỗi worker. 0 = vCPU / workers.")
+    p_serve.add_argument("--limit_concurrency", type=int, default=int(os.getenv("LIMIT_CONCURRENCY", "256")))
+
+    p_download = sub.add_parser("download", help="Tải model từ Hugging Face Hub về cache (in ra đường dẫn).")
+    p_download.add_argument("--repo_id", default=None)
+    p_download.add_argument("--revision", default=None)
+    p_download.add_argument("--cache_dir", default=None)
+    p_download.add_argument("--force", action="store_true", help="Tải lại kể cả khi cache đã đủ.")
+
+    p_export = sub.add_parser("export", help="Export checkpoint PyTorch -> ONNX INT8 (cần extra [export]).")
+    from .export import add_arguments
+
+    add_arguments(p_export)
+
+    p_correct = sub.add_parser("correct", help="Sửa nhanh vài câu từ dòng lệnh.")
+    p_correct.add_argument("texts", nargs="+")
+
+    args = parser.parse_args(argv)
+    if args.command == "serve":
+        return cmd_serve(args)
+    if args.command == "download":
+        return cmd_download(args)
+    if args.command == "export":
+        from .export import main as export_main
+
+        return export_main(args)
+    return cmd_correct(args)
+
+
+def cli() -> int:
+    """Entry point: lỗi đã biết (thiếu model, tải thất bại...) in 1 dòng thay vì traceback."""
+    from .exceptions import OrthomniError
+
+    try:
+        return main()
+    except OrthomniError as exc:
+        print(f"[orthomni] Lỗi: {exc}", file=sys.stderr)
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(cli())