modelfile-doctor 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. modelfile_doctor-0.3.0/.gitignore +41 -0
  2. modelfile_doctor-0.3.0/CHANGELOG.md +38 -0
  3. modelfile_doctor-0.3.0/LICENSE +21 -0
  4. modelfile_doctor-0.3.0/PKG-INFO +330 -0
  5. modelfile_doctor-0.3.0/README.ko.md +304 -0
  6. modelfile_doctor-0.3.0/README.md +306 -0
  7. modelfile_doctor-0.3.0/examples/Modelfile.chatml-good +8 -0
  8. modelfile_doctor-0.3.0/examples/Modelfile.mismatch +4 -0
  9. modelfile_doctor-0.3.0/examples/Modelfile.syntax-errors +10 -0
  10. modelfile_doctor-0.3.0/pyproject.toml +52 -0
  11. modelfile_doctor-0.3.0/src/modelfile_doctor/__init__.py +29 -0
  12. modelfile_doctor-0.3.0/src/modelfile_doctor/__main__.py +5 -0
  13. modelfile_doctor-0.3.0/src/modelfile_doctor/cli.py +192 -0
  14. modelfile_doctor-0.3.0/src/modelfile_doctor/core.py +134 -0
  15. modelfile_doctor-0.3.0/src/modelfile_doctor/decode.py +42 -0
  16. modelfile_doctor-0.3.0/src/modelfile_doctor/demo.py +94 -0
  17. modelfile_doctor-0.3.0/src/modelfile_doctor/errors.py +45 -0
  18. modelfile_doctor-0.3.0/src/modelfile_doctor/gguf_header.py +253 -0
  19. modelfile_doctor-0.3.0/src/modelfile_doctor/gotemplate.py +116 -0
  20. modelfile_doctor-0.3.0/src/modelfile_doctor/i18n.py +53 -0
  21. modelfile_doctor-0.3.0/src/modelfile_doctor/locales/en.json +109 -0
  22. modelfile_doctor-0.3.0/src/modelfile_doctor/locales/ko.json +109 -0
  23. modelfile_doctor-0.3.0/src/modelfile_doctor/ollama.py +73 -0
  24. modelfile_doctor-0.3.0/src/modelfile_doctor/output.py +70 -0
  25. modelfile_doctor-0.3.0/src/modelfile_doctor/params.py +76 -0
  26. modelfile_doctor-0.3.0/src/modelfile_doctor/parser.py +100 -0
  27. modelfile_doctor-0.3.0/src/modelfile_doctor/py.typed +0 -0
  28. modelfile_doctor-0.3.0/src/modelfile_doctor/report.py +83 -0
  29. modelfile_doctor-0.3.0/src/modelfile_doctor/rules_ct.py +69 -0
  30. modelfile_doctor-0.3.0/src/modelfile_doctor/rules_mf.py +155 -0
  31. modelfile_doctor-0.3.0/src/modelfile_doctor/rules_tp.py +63 -0
  32. modelfile_doctor-0.3.0/src/modelfile_doctor/tokens.py +53 -0
  33. modelfile_doctor-0.3.0/tests/conftest.py +128 -0
  34. modelfile_doctor-0.3.0/tests/test_cli.py +136 -0
  35. modelfile_doctor-0.3.0/tests/test_decode_parser.py +47 -0
  36. modelfile_doctor-0.3.0/tests/test_demo.py +59 -0
  37. modelfile_doctor-0.3.0/tests/test_gguf_header.py +83 -0
  38. modelfile_doctor-0.3.0/tests/test_gotemplate.py +60 -0
  39. modelfile_doctor-0.3.0/tests/test_i18n.py +265 -0
  40. modelfile_doctor-0.3.0/tests/test_ollama_stub.py +90 -0
  41. modelfile_doctor-0.3.0/tests/test_precommit_hook.py +22 -0
  42. modelfile_doctor-0.3.0/tests/test_rules_ct.py +63 -0
  43. modelfile_doctor-0.3.0/tests/test_rules_mf.py +89 -0
@@ -0,0 +1,41 @@
1
+ # Python
2
+ __pycache__/
3
+ *.py[cod]
4
+ .venv/
5
+ .pytest_cache/
6
+ .ruff_cache/
7
+ .mypy_cache/
8
+ *.egg-info/
9
+
10
+ # Node
11
+ node_modules/
12
+ dist/
13
+ .next/
14
+
15
+ # 실행 산출물·모델·데이터 캐시
16
+ runs/
17
+ outputs/
18
+ .cache/
19
+ *.safetensors
20
+ *.gguf
21
+ *.bin
22
+
23
+ # 환경·비밀
24
+ .env
25
+ .env.*
26
+ !.env.example
27
+
28
+ # OS·에디터
29
+ .DS_Store
30
+ .vscode/
31
+ .idea/
32
+
33
+ # Flutter·Dart
34
+ .dart_tool/
35
+ build/
36
+ .flutter-plugins
37
+ .flutter-plugins-dependencies
38
+ coverage/
39
+
40
+ # Local tool state
41
+ .claude/
@@ -0,0 +1,38 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project are documented here.
4
+ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses [Semantic Versioning](https://semver.org/).
5
+
6
+ ## [0.3.0] - 2026-11-13
7
+
8
+ ### Added
9
+ - `.pre-commit-hooks.yaml`: use modelfile-doctor as a pre-commit hook.
10
+
11
+ ## [0.2.0] - 2026-11-13
12
+
13
+ ### Added
14
+ - `MF021`: `ADAPTER` path that does not exist (relative to the Modelfile, `~` expanded; model-name values are not checked; files and directories both pass).
15
+
16
+ ### Changed
17
+ - Existing Modelfiles may now report a new error and fail CI. To turn the rule off, use `--ignore MF021`.
18
+
19
+ ## [0.1.1] - 2026-11-13
20
+
21
+ ### Fixed
22
+ - `--list-rules`: the header line's Description column was two columns to the right of the rows in both languages.
23
+
24
+ ## [0.1.0] - 2026-11-12
25
+
26
+ ### Added
27
+ - Modelfile checks: encoding (UTF-8/BOM/UTF-16), instructions, PARAMETER names/types/ranges, MESSAGE roles, duplicate TEMPLATE/SYSTEM, stop values, FROM paths (MF rules).
28
+ - Go `text/template` structure checks: delimiters, block balance, Jinja leftovers, unknown variables (TP rules).
29
+ - Dependency-free GGUF key/value reader (v2/v3) that never reads tensor data.
30
+ - Comparison of TEMPLATE/stop/num_ctx with GGUF metadata (CT rules), also for model names through a local Ollama server (`--ollama`).
31
+ - Text and JSON output, exit codes 0/1/2, `--strict`, `--select`, `--ignore`, `--list-rules`.
32
+ - English and Korean output (`--lang`, `MODELFILE_DOCTOR_LANG`, `LC_ALL`/`LC_MESSAGES`/`LANG`).
33
+ - `demo` command (synthetic GGUF without weights) and `inspect` command.
34
+
35
+ [0.3.0]: https://github.com/LeeGyeongPil/modelfile-doctor/compare/v0.2.0...v0.3.0
36
+ [0.2.0]: https://github.com/LeeGyeongPil/modelfile-doctor/compare/v0.1.1...v0.2.0
37
+ [0.1.1]: https://github.com/LeeGyeongPil/modelfile-doctor/compare/v0.1.0...v0.1.1
38
+ [0.1.0]: https://github.com/LeeGyeongPil/modelfile-doctor/releases/tag/v0.1.0
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 LeeGyeongPil
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,330 @@
1
+ Metadata-Version: 2.5
2
+ Name: modelfile-doctor
3
+ Version: 0.3.0
4
+ Summary: Check an Ollama Modelfile against the GGUF metadata it points to, before you run `ollama create`.
5
+ Project-URL: Homepage, https://github.com/LeeGyeongPil/modelfile-doctor
6
+ Project-URL: Issues, https://github.com/LeeGyeongPil/modelfile-doctor/issues
7
+ Author: LeeGyeongPil
8
+ License: MIT
9
+ License-File: LICENSE
10
+ Keywords: chat-template,gguf,lint,llm,modelfile,ollama
11
+ Classifier: Development Status :: 4 - Beta
12
+ Classifier: Environment :: Console
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: License :: OSI Approved :: MIT License
15
+ Classifier: Programming Language :: Python :: 3
16
+ Classifier: Programming Language :: Python :: 3.10
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Topic :: Software Development :: Quality Assurance
21
+ Classifier: Typing :: Typed
22
+ Requires-Python: >=3.10
23
+ Description-Content-Type: text/markdown
24
+
25
+ # modelfile-doctor
26
+
27
+ **English** | [한국어](https://github.com/LeeGyeongPil/modelfile-doctor/blob/v0.3.0/README.ko.md)
28
+
29
+ [![CI](https://github.com/LeeGyeongPil/modelfile-doctor/actions/workflows/ci.yml/badge.svg)](https://github.com/LeeGyeongPil/modelfile-doctor/actions/workflows/ci.yml) ![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue) [![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
30
+
31
+ Check an Ollama `Modelfile` before you run `ollama create`. It reports every syntax error at once, with line numbers, and compares your `TEMPLATE` and `stop` values with the chat format stored inside the GGUF model file, which Ollama itself does not do. English and Korean output, no runtime dependencies.
32
+
33
+ ![modelfile-doctor demo run: a good Modelfile passes, a bad one gets six findings](https://raw.githubusercontent.com/LeeGyeongPil/modelfile-doctor/v0.3.0/docs/images/01-demo.png)
34
+ *The built-in demo: `Modelfile.good` passes, `Modelfile.bad` (a Llama 2 style template on a ChatML model) gets three errors and three warnings.*
35
+
36
+ ## Why
37
+ A Modelfile like this passes `ollama create`, because Ollama only checks that the file parses:
38
+
39
+ ```text
40
+ FROM ./model.gguf
41
+ PARAMETER stop "</s>"
42
+ TEMPLATE """[INST] {{ .Prompt }} [/INST]"""
43
+ ```
44
+ If the model is a ChatML model, `[INST]` is not a special token for it and `</s>` is never produced, so the model answers badly or never stops. `modelfile-doctor` compares these lines with the chat format stored in the GGUF and says so before you create the model.
45
+
46
+ ## Who this is for
47
+ - You download a GGUF file and write your own `Modelfile`, and `ollama create` printed `success` but the model rambles or never stops.
48
+ - You want every typo in a `Modelfile` (parameter names, value types, message roles, template blocks) listed in one run instead of one error per attempt.
49
+ - You want a check you can run in CI or a pre-commit step, with JSON output and exit codes.
50
+
51
+ ## Install
52
+ Needs Python 3.10 or newer. Install [uv](https://docs.astral.sh/uv/) if you do not have it (`curl -LsSf https://astral.sh/uv/install.sh | sh`); on a minimal Linux image also make sure `git` and `curl` are installed.
53
+
54
+ ```bash
55
+ # from a clone (what this README was verified with)
56
+ git clone https://github.com/LeeGyeongPil/modelfile-doctor
57
+ cd modelfile-doctor
58
+ uv sync
59
+ uv run modelfile-doctor --version
60
+ ```
61
+ ```text
62
+ modelfile-doctor 0.3.0
63
+ ```
64
+
65
+ To put the command on your PATH: `uv tool install git+https://github.com/LeeGyeongPil/modelfile-doctor`. There are no runtime dependencies; `gguf` and `numpy` are only used by the tests.
66
+
67
+ ## Quick start
68
+ The demo needs no model and no Ollama server. It writes a tiny practice GGUF (header only, no weights, nothing you could run) plus two Modelfiles, and checks them with the real rule engine.
69
+
70
+ ```bash
71
+ uv run modelfile-doctor demo
72
+ ```
73
+ ```text
74
+ practice fixture: synthetic GGUF without weights (nothing here is a real model)
75
+ Files were written to a temporary directory (/tmp/...) and are removed afterwards. Use --out DIR to keep them.
76
+
77
+ Modelfile.good: no problems
78
+
79
+ Modelfile.bad:2: error MF011 Unknown PARAMETER "TEMPERATURE". Did you mean "temperature"? (names are case-sensitive)
80
+ Modelfile.bad:3: warning CT001 The model ends its turn with <|im_end|>, but PARAMETER stop does not list it.
81
+ Modelfile.bad:3: warning CT004 stop "</s>" is a token this model never produces.
82
+ Modelfile.bad:4: warning CT002 The model's chat template uses <|im_start|> <|im_end|>, but TEMPLATE does not contain them.
83
+ Modelfile.bad:4: error CT003 TEMPLATE uses [INST], which is not in this model's vocabulary (it would be read as plain text).
84
+ Modelfile.bad:4: error CT003 TEMPLATE uses [/INST], which is not in this model's vocabulary (it would be read as plain text).
85
+
86
+ Summary: 3 error(s), 3 warning(s), 0 info in 2 file(s)
87
+ Modelfile.bad has a ChatML model with a Llama 2 template; `ollama create` would still print success. Try it on your own model: modelfile-doctor Modelfile
88
+ ```
89
+ Each line is `file:line: severity RULE message`. `MF` rules are about the Modelfile syntax, `CT` rules compare it with the model. The demo is a practice mode: the model is synthetic, so it shows how the checks behave, not how a real model would answer.
90
+
91
+ ## Usage
92
+ | Command / option | What it does | Example |
93
+ |---|---|---|
94
+ | `modelfile-doctor PATH...` | Check one or more Modelfiles | `modelfile-doctor Modelfile` |
95
+ | `demo [--out DIR]` | Run the practice demo; `--out` keeps the generated files | `modelfile-doctor demo --out ./demo-files` |
96
+ | `inspect FILE.gguf` | Print the chat-related header fields of a GGUF file | `modelfile-doctor inspect model.gguf` |
97
+ | `--ollama [URL]` | Resolve `FROM model:tag` through a local Ollama server (default `$OLLAMA_HOST` or `http://127.0.0.1:11434`) | `modelfile-doctor --ollama Modelfile` |
98
+ | `--format text\|json` | Output format | `modelfile-doctor --format json Modelfile` |
99
+ | `--strict` | Warnings also give exit code 1 | `modelfile-doctor --strict Modelfile` |
100
+ | `--select IDS` / `--ignore IDS` | Only run / skip rules (ids or prefixes, comma separated) | `modelfile-doctor --select CT Modelfile` |
101
+ | `--list-rules` | Show every rule with its severity | `modelfile-doctor --list-rules` |
102
+ | `--lang en\|ko`, `--no-color` | Output language, no colors | `modelfile-doctor --lang ko Modelfile` |
103
+
104
+ Exit codes: `0` no errors, `1` errors found (or warnings with `--strict`), `2` usage or I/O problem (missing file, unreadable GGUF, Ollama not reachable).
105
+
106
+ ### Scenario 1: find every syntax error in one run
107
+ The repository ships `examples/Modelfile.syntax-errors`, with a typo in almost every line:
108
+
109
+ ```bash
110
+ uv run modelfile-doctor --no-color --lang en examples/Modelfile.syntax-errors
111
+ ```
112
+ ```text
113
+ examples/Modelfile.syntax-errors:1: info CT010 FROM "qwen3:4b" is a model name, so TEMPLATE/stop were not compared. Pass --ollama to compare.
114
+ examples/Modelfile.syntax-errors:2: error MF011 Unknown PARAMETER "TEMPERATURE". Did you mean "temperature"? (names are case-sensitive)
115
+ examples/Modelfile.syntax-errors:3: error MF012 PARAMETER num_ctx needs a value of type int, got "4k".
116
+ examples/Modelfile.syntax-errors:4: warning MF010 PARAMETER mirostat is deprecated; Ollama drops it.
117
+ examples/Modelfile.syntax-errors:5: warning MF013 PARAMETER temperature = 5 is outside the usual range (0..2).
118
+ examples/Modelfile.syntax-errors:6: error MF014 MESSAGE role "User" must be system, user or assistant (lower case).
119
+ examples/Modelfile.syntax-errors:7: error MF004 Unknown instruction "SYSTM". Did you mean SYSTEM?
120
+ examples/Modelfile.syntax-errors:8: error TP004 The range block is never closed with an end action.
121
+ examples/Modelfile.syntax-errors:8: warning TP005 Unknown template variable .Contnt. Did you mean .Content?
122
+ examples/Modelfile.syntax-errors:9: error TP004 The if block is never closed with an end action.
123
+ Summary: 6 error(s), 3 warning(s), 1 info in 1 file(s)
124
+ ```
125
+ The exit code is `1` because errors were found. The `CT010` line is information only: `FROM qwen3:4b` is a model name, so there is no file to compare against unless you pass `--ollama`.
126
+
127
+ ![The same checks on a Modelfile full of typos](https://raw.githubusercontent.com/LeeGyeongPil/modelfile-doctor/v0.3.0/docs/images/02-syntax-errors.png)
128
+ *Ten findings, each with its own line number.*
129
+
130
+ ### Scenario 2: compare a Modelfile with the model (the part `ollama create` skips)
131
+ Keep the demo files, then check the bad Modelfile against the synthetic GGUF it points to:
132
+
133
+ ```bash
134
+ uv run modelfile-doctor demo --out ./demo-files # leaves Modelfile.* and a .gguf there; use a /tmp path if you prefer
135
+ uv run modelfile-doctor --no-color --lang en ./demo-files/Modelfile.bad
136
+ ```
137
+ ```text
138
+ ./demo-files/Modelfile.bad:2: error MF011 Unknown PARAMETER "TEMPERATURE". Did you mean "temperature"? (names are case-sensitive)
139
+ ./demo-files/Modelfile.bad:3: warning CT001 The model ends its turn with <|im_end|>, but PARAMETER stop does not list it.
140
+ ...
141
+ Summary: 3 error(s), 3 warning(s), 0 info in 1 file(s)
142
+ ```
143
+ (Run it to see all six lines; the output is the same as in the demo.) Read it like this: the model's own chat template uses `<|im_start|>` and `<|im_end|>` (CT002), it ends turns with `<|im_end|>` that `stop` does not list (CT001), `</s>` is never produced by this model (CT004), and `[INST]` is not even in the vocabulary, so it would be fed to the model as plain letters (CT003).
144
+
145
+ To look at what the tool read from the GGUF header:
146
+
147
+ ```bash
148
+ uv run modelfile-doctor --lang en inspect ./demo-files/synthetic-chatml.gguf
149
+ ```
150
+ ```text
151
+ architecture: llama
152
+ name: practice-fixture
153
+ GGUF version: 3
154
+ context length: 4096
155
+ vocabulary: 6 tokens
156
+ EOS token: <|im_end|>
157
+ BOS token: -
158
+ EOT token: -
159
+ chat template: 201 characters
160
+ role tokens: <|im_start|> <|im_end|>
161
+ ```
162
+
163
+ ### Scenario 3: a model name instead of a file (`--ollama`)
164
+ If `FROM` is a model name such as `qwen3:4b`, the tool asks your local Ollama server (`/api/show`) where the model file is and compares against its header. Without `--ollama` it only prints the `CT010` info line and exits `0`. If the server is not running you get a one-line error and exit code `2`:
165
+
166
+ ```bash
167
+ uv run modelfile-doctor --no-color --lang en --ollama http://127.0.0.1:9 examples/Modelfile.syntax-errors
168
+ ```
169
+ ```text
170
+ error: Cannot reach Ollama at http://127.0.0.1:9 ([Errno 61] Connection refused). Start it with `ollama serve` or pass --ollama URL.
171
+ ```
172
+ The number in `[Errno 61]` is the macOS value; the same failure shows `[Errno 111]` on Linux.
173
+
174
+ This mode was verified here only against a small stub server in the tests, not against a real Ollama install. The tool runs on the machine where the model file is, so it needs that file to be readable.
175
+
176
+ ### The other files in `examples/`
177
+ `examples/Modelfile.mismatch` and `examples/Modelfile.chatml-good` are two deliberate practice files for the same synthetic model: the first has the Llama 2 style template, the second is the matching ChatML one. They point to `./synthetic-chatml.gguf`, which is not in the repository, so run them next to a generated copy:
178
+
179
+ ```bash
180
+ uv run modelfile-doctor demo --out ./demo-files
181
+ cp examples/Modelfile.mismatch examples/Modelfile.chatml-good ./demo-files/
182
+ uv run modelfile-doctor --no-color --lang en ./demo-files/Modelfile.mismatch ./demo-files/Modelfile.chatml-good
183
+ ```
184
+ Expected: `Modelfile.mismatch` gives 2 errors and 3 warnings (the CT001 to CT004 lines from the demo), `Modelfile.chatml-good` gives `no problems`. Run directly from the repository (without the `.gguf` beside them) you only get `MF020 FROM path ... does not exist`, which is expected. `examples/Modelfile.syntax-errors` is the one from Scenario 1; its errors are intentional.
185
+
186
+ ### Scenario 4: CI and scripts
187
+ ```bash
188
+ uv run modelfile-doctor --format json --select CT ./demo-files/Modelfile.bad
189
+ ```
190
+ prints a JSON array with one object per file: `path`, `encoding` and `diagnostics`, each diagnostic having `rule`, `severity`, `line`, `message_key`, `params`, `message` and `fixable` (always `false` for now). `message_key` and `params` do not depend on the language; only `message` is translated.
191
+
192
+ ![All rules the checker knows](https://raw.githubusercontent.com/LeeGyeongPil/modelfile-doctor/v0.3.0/docs/images/03-list-rules.png)
193
+ *`--list-rules` shows every rule id, its severity and a short description.*
194
+
195
+ Use it as a [pre-commit](https://pre-commit.com) hook; in `.pre-commit-config.yaml`:
196
+ ```yaml
197
+ repos:
198
+ - repo: https://github.com/LeeGyeongPil/modelfile-doctor
199
+ rev: v0.3.0
200
+ hooks:
201
+ - id: modelfile-doctor
202
+ ```
203
+ The hook runs on files named `Modelfile` or `Modelfile.*`. Pass options with `args:` (for example `args: [--ignore, MF020]` if the model files are not in the repository), and use `exclude:` to skip files such as `Modelfile.gguf`.
204
+
205
+ ## Configuration
206
+ There is no config file. Only environment variables:
207
+
208
+ | Variable | Meaning |
209
+ |---|---|
210
+ | `MODELFILE_DOCTOR_LANG` | Output language `en` or `ko`. Order: `--lang`, then this variable, then `LC_ALL`, `LC_MESSAGES`, `LANG` (first non-empty wins; anything not starting with `ko` means English). |
211
+ | `OLLAMA_HOST` | Used by `--ollama` when you give no URL. Default `http://127.0.0.1:11434`. |
212
+ | `MODELFILE_DOCTOR_LIVE` | Tests only: `1` runs the test that needs a real local Ollama. |
213
+
214
+ The Korean screenshots in [README.ko.md](https://github.com/LeeGyeongPil/modelfile-doctor/blob/v0.3.0/README.ko.md) were taken with `--lang ko`.
215
+
216
+ ## Python API
217
+ The checks are also a small library (no extra dependencies). Everything below is exported from `modelfile_doctor`.
218
+
219
+ | Name | What it does |
220
+ |---|---|
221
+ | `lint_path(path, *, ollama_url=None, select=None, ignore=None)` | Read and lint a Modelfile; returns a `Report` |
222
+ | `lint_text(text, *, base_dir=".", gguf_meta=None, ...)` | Lint Modelfile text you already have; pass `gguf_meta` to compare against a model without touching disk |
223
+ | `read_gguf_metadata(path)` | Read the GGUF header into a `GGUFMetadata` (architecture, `chat_template`, `tokens`, EOS id, ...) |
224
+ | `Report.diagnostics` | List of `Diagnostic` with `rule`, `severity`, `line`, `message_key`, `params`; also `Report.errors`, `.warnings`, `.infos` |
225
+ | `DoctorError.code`, `.params` | Raised for I/O problems (missing file, GGUF passed as Modelfile, unreachable Ollama); `code` and `params` do not depend on the language |
226
+ | `render_error(exc, lang)` / `render(key, params, lang)` | Turn an error or a diagnostic key into a sentence in `en` or `ko` |
227
+
228
+ ```python
229
+ from modelfile_doctor import lint_path, render_error, DoctorError
230
+
231
+ report = lint_path("examples/Modelfile.syntax-errors")
232
+ print(sorted({d.rule for d in report.diagnostics}))
233
+ for d in report.diagnostics[:2]:
234
+ print(d.line, d.severity.value, d.rule, d.message_key, d.params)
235
+
236
+ try:
237
+ lint_path("no-such-file")
238
+ except DoctorError as exc:
239
+ print(exc.code, exc.params)
240
+ print(render_error(exc, "en"))
241
+ ```
242
+ ```text
243
+ ['CT010', 'MF004', 'MF010', 'MF011', 'MF012', 'MF013', 'MF014', 'TP004', 'TP005']
244
+ 1 info CT010 diag.CT010 {'name': 'qwen3:4b'}
245
+ 2 error MF011 diag.MF011.suggest {'name': 'TEMPERATURE', 'suggestion': 'temperature'}
246
+ io.not_found {'path': 'no-such-file'}
247
+ Cannot find "no-such-file". Check the path.
248
+ ```
249
+ Run it from the repository root (`uv run python script.py`).
250
+
251
+ ## How it works
252
+ ```mermaid
253
+ flowchart LR
254
+ A[Modelfile] --> B[Decode: UTF-8, BOM, UTF-16]
255
+ B --> C[Parse instructions]
256
+ C --> D[MF rules: names, types, ranges]
257
+ C --> E[TP rules: Go template blocks]
258
+ C --> F[FROM path or model name]
259
+ F -->|file| G[Read GGUF header only]
260
+ F -->|name + --ollama| H[Ask local Ollama for the file]
261
+ H --> G
262
+ G --> I[CT rules: TEMPLATE, stop, num_ctx vs model]
263
+ D --> J[Report: text or JSON]
264
+ E --> J
265
+ I --> J
266
+ ```
267
+ - The file is decoded first, so a UTF-16 file without BOM is reported (MF001) instead of failing later.
268
+ - The parser splits the file into instructions with their line numbers; all later checks use those lines.
269
+ - MF rules check parameter names, value types and ranges, message roles and instructions; TP rules check the Go template (balanced `range`/`if`/`end`, leftover Jinja syntax, unknown variables).
270
+ - The GGUF reader reads only the key/value header (chat template, tokens, token types, end-of-sequence token, context length) and never the weights.
271
+ - CT rules compare what `TEMPLATE` and `stop` use with what the model actually knows and produces.
272
+
273
+ ## Syntax rules (MF, TP)
274
+ `--list-rules` prints the full list; these are the ones you will meet most.
275
+
276
+ | Rule | Severity | Example |
277
+ |---|---|---|
278
+ | `MF002` / `MF004` | error | No `FROM` line, or an unknown instruction such as `SYSTM` |
279
+ | `MF011` / `MF012` | error | Unknown `PARAMETER` name (`TEMPERATURE`), or a value of the wrong type (`num_ctx 4k`) |
280
+ | `MF010` / `MF013` | warning | Deprecated parameter (`mirostat`), or a value outside the usual range (`temperature 5`) |
281
+ | `MF014` | error | `MESSAGE` role is not `system`, `user` or `assistant` |
282
+ | `MF020` | error | `FROM` path does not exist or is not a GGUF file |
283
+ | `MF021` | error | `ADAPTER` path does not exist (skip with `--ignore MF021`) |
284
+ | `TP001` to `TP004` | error | Unbalanced `{{ }}`, Jinja syntax in a Go template, stray `end`/`else`, a block that is never closed |
285
+ | `TP005` | warning | Unknown template variable (`.Contnt`) |
286
+
287
+ ## Model comparison rules (CT)
288
+ These are the checks that compare your Modelfile with the model. `--list-rules` shows all other rules too. Rule sources: Ollama's [Modelfile reference](https://docs.ollama.com/modelfile) and its [parser.go](https://github.com/ollama/ollama/blob/main/parser/parser.go).
289
+
290
+ | Rule | Severity | Example |
291
+ |---|---|---|
292
+ | `CT001` | warning | The model ends turns with `<\|im_end\|>` but `PARAMETER stop` does not list it, so generation may not stop |
293
+ | `CT002` | warning | The model's chat template uses `<\|im_start\|>`, your `TEMPLATE` does not |
294
+ | `CT003` | error / warning | `TEMPLATE` uses `[INST]`, which is not in the vocabulary (error, it is read as plain text), or a token the model's own chat template never uses (warning) |
295
+ | `CT004` | warning | `stop "</s>"` is a token this model never produces |
296
+ | `CT005` | warning | `num_ctx` is larger than the model's context length |
297
+ | `CT010` / `CT011` | info | The comparison was skipped (`FROM` is a model name, or the GGUF header could not be read) |
298
+
299
+ ## When to use it
300
+ - Good fit: you write or edit Modelfiles by hand for GGUF files and want a check before `ollama create`, or a CI step that fails on a broken Modelfile.
301
+ - Other tools for other jobs: a plain syntax-only Modelfile linter such as [owist-modelfile-lint](https://pypi.org/project/owist-modelfile-lint/) (check its README for what it covers), watching what Ollama does at request time → [Ollama](https://ollama.com) itself and its [Modelfile reference](https://docs.ollama.com/modelfile), inspecting GGUF files in depth → the [`gguf`](https://pypi.org/project/gguf/) Python package from llama.cpp.
302
+
303
+ ## Limitations
304
+ - Which tokens are role or control tokens is decided by a pattern match plus the GGUF `token_type` values. This is a heuristic, so `CT002`, `CT003` and `CT004` can have false positives or miss something. If a finding is wrong for your model, skip it with `--ignore CT003`.
305
+ - `CT001` assumes that a turn-ending token missing from `stop` can make generation run on. How the Ollama runner treats it can differ by version, so read it as a warning, not as proof.
306
+ - The `PARAMETER` names and instruction tables are a snapshot of Ollama 0.40.15 (checked 2026-11-12). A newer Ollama may add parameters that this tool reports as `MF011` until the table is updated.
307
+ - `--ollama` only works when the model file that Ollama reports is readable on the same machine; remote Ollama servers are not supported. It was verified here only against a stub server, not a real install.
308
+ - `fixable` is always `false` for now: the tool reports problems but does not rewrite your file.
309
+ - Test gaps (known): there is no test that fails if the 4 MiB cap is removed from the Modelfile read, and the test for the "cannot create demo directory" message only checks the exit code and language, not the wording.
310
+ - A Modelfile larger than 4 MiB is refused, and a GGUF file passed as the Modelfile gets a hint to use `inspect`.
311
+
312
+ ## Troubleshooting
313
+ | Symptom | Cause | Fix |
314
+ |---|---|---|
315
+ | `error: Cannot find "…"` and exit 2 | The path is wrong | Check the path; directories are not searched |
316
+ | `MF020 FROM path ... does not exist` | `FROM ./x.gguf` is resolved relative to the Modelfile, not to where you run the command | Fix the path or run on the right Modelfile |
317
+ | `MF021 ADAPTER path ... does not exist` | `ADAPTER` is resolved relative to the Modelfile (`~` is expanded); a directory counts as existing | Fix the path, or turn the rule off with `--ignore MF021` |
318
+ | `CT010` info only | `FROM` is a model name | Pass `--ollama` (Ollama must be running) |
319
+ | `CT011` info | The GGUF header could not be read (truncated or unsupported version) | Run `modelfile-doctor inspect FILE.gguf` to see the exact reason |
320
+ | Output is in the wrong language | Locale variables win only after `--lang` and `MODELFILE_DOCTOR_LANG` | Pass `--lang en` or `--lang ko` |
321
+
322
+ ## Development
323
+ ```bash
324
+ uv sync
325
+ uv run pytest -q
326
+ ```
327
+ Tests build their GGUF fixtures on the fly; no model files and no Ollama are needed (one live test is skipped unless `MODELFILE_DOCTOR_LIVE=1`). Bugs and ideas are welcome as issues. Changes are listed in [CHANGELOG.md](https://github.com/LeeGyeongPil/modelfile-doctor/blob/v0.3.0/CHANGELOG.md).
328
+
329
+ ## License
330
+ MIT. The practice GGUF is generated by the tool itself and contains no model weights.