ai-template-runner 1.5.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 brunodavi
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,245 @@
1
+ Metadata-Version: 2.4
2
+ Name: ai-template-runner
3
+ Version: 1.5.0
4
+ Summary: Simple AI workflow based on Markdown templates
5
+ Keywords: ai,llm,templates,llamacpp,local-models,prompt
6
+ Author: brunodavi
7
+ Author-email: brunodavi <48166556+brunodavi@users.noreply.github.com>
8
+ License-Expression: MIT
9
+ License-File: LICENSE
10
+ Classifier: Development Status :: 4 - Beta
11
+ Classifier: Environment :: Console
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Programming Language :: Python :: 3.12
15
+ Classifier: Programming Language :: Python :: 3.13
16
+ Classifier: Programming Language :: Python :: 3.14
17
+ Classifier: Topic :: Software Development
18
+ Requires-Python: >=3.12
19
+ Project-URL: Repository, https://github.com/brunodavi/ai-template-runner
20
+ Project-URL: Issues, https://github.com/brunodavi/ai-template-runner/issues
21
+ Project-URL: Changelog, https://github.com/brunodavi/ai-template-runner/blob/master/CHANGELOG.md
22
+ Description-Content-Type: text/markdown
23
+
24
+ # AI Template Runner
25
+
26
+ Small local models are cheap and fast — they also lose structure the moment a prompt gets long. `ait` turns that weakness into a workflow: each template is a single Markdown file that gathers context, prompts the model, validates the answer and renders the result. Deterministic scaffolding around a dumb-but-fast model, even one running on your phone.
27
+
28
+ ```
29
+ ┌ template file ─────────────────────────────────┐
30
+ │ ```toml → model, endpoint, sampling │
31
+ │ ``` │
32
+ │ --- → system prompt │
33
+ │ user prompt: {cmd.version} output + question │
34
+ │ [test] / [out] → validate + render output │
35
+ └────────────────────────────────────────────────┘
36
+ ```
37
+
38
+ One file, one command, one predictable answer — no agent framework needed.
39
+
40
+ **Why it matters:** a small model cannot keep a workflow in its head, but it can fill in a well-shaped slot. Templates put the workflow in the file so the model only has to do the part it is good at. Failures become visible and reproducible through `[test]`, instead of vague "the model is dumb" sessions.
41
+
42
+ ## Quick start
43
+
44
+ Point `ait` at any OpenAI-compatible chat endpoint (llama.cpp server, Ollama, LM Studio, ...):
45
+
46
+ 1. Start a model server, e.g. llama.cpp:
47
+
48
+ ```bash
49
+ llama-server -hf unsloth/Qwen3-0.6B-GGUF --port 8080
50
+ ```
51
+
52
+ 2. Run a ready template:
53
+
54
+ ```bash
55
+ uvx ai-template-runner run TEMPLATE "What is Python?"
56
+ ```
57
+
58
+ Or clone the repository and try it against a fixture without a model at
59
+ all — the `examples/simulation/` templates run against fixed
60
+ responses, so you can feel the `[test]`/`[out]` mechanics in seconds:
61
+
62
+ ```bash
63
+ git clone https://github.com/brunodavi/ai-template-runner.git
64
+ cd ai-template-runner
65
+ uvx ai-template-runner run examples/simulation/conventional-commits/feat.md --test
66
+ ```
67
+
68
+ Running on a phone? See [`docs/termux.md`](docs/termux.md).
69
+
70
+ ## Installation
71
+
72
+ | Command | When to use |
73
+ | --- | --- |
74
+ | `uvx ai-template-runner --help` | zero-install trial from [PyPI](https://pypi.org/project/ai-template-runner/) |
75
+ | `pipx install ai-template-runner` | persistent install, any platform (Termux ships pipx) |
76
+ | `pip install ai-template-runner` | any pip environment |
77
+ | `uv tool install ai-template-runner` | [uv](https://docs.astral.sh/uv/) managed installs |
78
+ | `uv tool install git+https://github.com/brunodavi/ai-template-runner.git` | latest `master` |
79
+
80
+ Requires Python 3.12+. The CLI is `ait` (or `ai-template-runner`); the package has **zero runtime dependencies**.
81
+
82
+ ## Usage
83
+
84
+ ```bash
85
+ ait run examples/features/full.md "What is Python?"
86
+ ait run examples/features/full.md "What is Python?" --quiet
87
+ ait run examples/simulation/conventional-commits/feat.md --test
88
+ ait --help
89
+ ait run --help
90
+ ```
91
+
92
+ - `-q` / `--quiet`: display only the AI response (stdout stays clean for pipes).
93
+ - `-t` / `--test`: validate the response against the template's `[test]` table. PASS/FAIL lines go to stderr and the exit code is 1 when any check fails.
94
+
95
+ ## Templates
96
+
97
+ Templates combine **TOML configuration** and Markdown prompts in a single file. The configuration is defined in a ` ```toml ` code block at the beginning of the file, and is parsed and removed from the prompt content. Two ` --- ` separators delimit an optional system prompt; everything after them becomes the user prompt. Templates can also execute terminal commands and inject their output directly into prompts.
98
+
99
+ ````markdown
100
+ ```toml
101
+ [provider]
102
+ base_url = 'http://localhost:8080/v1/chat/completions'
103
+ model = 'lfm230m'
104
+ temperature = 0.2
105
+ top_p = 0.9
106
+ top_k = 40
107
+ min_p = 0.05
108
+ max_tokens = 220
109
+ seed = 42
110
+ stop = []
111
+ frequency_penalty = 0.0
112
+ presence_penalty = 0.0
113
+ repeat_penalty = 1.1
114
+
115
+ [provider.extras]
116
+ typical_p = 1.0
117
+ repeat_last_n = 64
118
+
119
+ [cmd]
120
+ message = 'echo hello'
121
+ ```
122
+
123
+ ---
124
+ You are a concise assistant. Use the command output as context.
125
+ ---
126
+
127
+ Command output:
128
+ {cmd.message}
129
+
130
+ Answer the following question concisely: {question}
131
+ ````
132
+
133
+ The TOML block is parsed and removed from the prompt content. The optional system prompt is delimited by two `---` separators; everything after the second separator is the user prompt. When the separators are omitted, the whole content after the configuration block is the user prompt.
134
+
135
+ The `{question}` placeholder is replaced with the CLI input. The `{cmd.message}` placeholder executes the configured command and inserts its output into the prompt. Placeholders can be used in both the system and user prompts.
136
+
137
+ The provider supports these generation parameters:
138
+
139
+ - `temperature`, `top_p`, `top_k`, `min_p`: control sampling.
140
+ - `max_tokens`: limits the generated response length.
141
+ - `seed`: sets the random seed when supported.
142
+ - `stop`: defines stop sequences.
143
+ - `frequency_penalty`, `presence_penalty`, `repeat_penalty`: control repetition.
144
+ - `extras`: passes additional parameters to the API, such as llama.cpp-specific options.
145
+
146
+ `base_url` and `model` are required. Parameter support depends on the API and model server being used.
147
+
148
+ A template can inherit configuration, commands, and prompts from another template with a top-level `extends` key placed before any TOML table:
149
+
150
+ ```toml
151
+ extends = 'base.md'
152
+
153
+ [provider]
154
+ temperature = 0.2
155
+ ```
156
+
157
+ The `extends` path is resolved relative to the template file itself, so [`examples/features/`](examples/features/) templates use `extends = '../base.md'`. Child values win per key; nested tables such as `[provider.extras]` and `[cmd]` are merged key by key. Empty child prompts fall back to the parent prompts, which lets a template reuse only the parent prompts and configuration.
158
+
159
+ A template can validate its own output with a `[test]` table and the `--test` flag:
160
+
161
+ ```toml
162
+ [test]
163
+ type = 'feat' # JSON output field equality
164
+ message = 'add health check endpoint' # exact value comparison
165
+ "user.name" = 'Alice' # nested JSON path (quoted dotted key)
166
+
167
+ text = 'exact raw output' # exact raw text comparison
168
+ text_contains = ['GET_health'] # every fragment present in the raw text
169
+ text_regex = '^\{' # regex search on the raw text
170
+ ```
171
+
172
+ Run with validation:
173
+
174
+ ```bash
175
+ ait run examples/simulation/conventional-commits/feat.md --test
176
+ ```
177
+
178
+ PASS/FAIL lines go to stderr (stdout stays clean for pipes) and the exit code is 1 when any check fails. Missing JSON fields and invalid JSON output fail instead of raising. The `[test]` table participates in `extends` inheritance like the other config tables.
179
+
180
+ A template can also render its own display output with an `[out]` table. The `text` key is a render template formatted with the parsed JSON fields (`{out.type}`, `{out.message}`, ...); missing fields render as empty:
181
+
182
+ ```toml
183
+ [out]
184
+ text = '{out.type}: {out.message}'
185
+ ```
186
+
187
+ Transforms run in two channels that never share the field namespace (`out.count` is always the JSON field, never `str.count`):
188
+
189
+ ```toml
190
+ [out]
191
+ text = '{out.message:strip} | {out.message:replace:GET:SET} | {out.message[:4]}'
192
+
193
+ [[out.upper]] # one step: str.upper() with no args
194
+
195
+ [[out.replace]] # named args match the real str signature
196
+ old = 'GET_health'
197
+ new = 'GET'
198
+
199
+ [[out.slice]] # helper: builtin slice(start, end)
200
+ start = 1
201
+ end = 10
202
+ ```
203
+
204
+ The `[[out.<method>]]` tables run in real document order (one table = one step, an empty table means no arguments) and apply to the rendered `text` — or to the raw response when there is no `text`. Steps are real `str` methods, with named arguments mapped positionally by the real signature. The inline spec is a single step from `method:arg1:arg2` (argument values may not contain ':'), and `{out.field[1:]}` follows pythonic slice syntax. Without `[out]` (or an empty table) the raw AI response is printed. The `[out]` config participates in `extends` inheritance per key: the child `text` and step lists replace the parent's.
205
+
206
+ The same inline transforms work on every interpolated value in the prompts — `{question:strip}`, `{question[:80]:lower}` and `{cmd.message:upper}`, `{cmd.status[1:20]}` — since the value stays lazy for `{cmd.*}` (the command only runs when a field is actually used).
207
+
208
+ ## Examples and docs
209
+
210
+ See [`examples/base.md`](examples/base.md) for the shared base used by the examples, [`examples/features/`](examples/features/) for one example per feature, [`examples/`](examples/) for ready-to-use task templates such as [`email-subject.md`](examples/email-subject.md) and [`conventional-commits.md`](examples/conventional-commits.md), and [`examples/simulation/`](examples/simulation/) for simulations that run task templates against fixed fixtures.
211
+
212
+ For deeper material, read [`docs/architecture.md`](docs/architecture.md) (why the code is
213
+ shaped the way it is), [`docs/termux.md`](docs/termux.md) (running local models on a phone)
214
+ and [`docs/maintenance.md`](docs/maintenance.md) (release routine for maintainers).
215
+
216
+ ## Development
217
+
218
+ ```bash
219
+ git clone https://github.com/brunodavi/ai-template-runner.git
220
+ cd ai-template-runner
221
+ uv sync
222
+ uv pip install -e .
223
+ ```
224
+
225
+ Editable installation (`-e .`) lets you run `ait` directly while developing, so source changes are available without reinstalling.
226
+
227
+ Run a template to test your changes:
228
+
229
+ ```bash
230
+ ait run examples/features/full.md "What is Python?"
231
+ ```
232
+
233
+ Run the test suite (from the repository root):
234
+
235
+ ```bash
236
+ uv run python -m unittest
237
+ ```
238
+
239
+ Contributions are welcome — CI runs the suite on Python 3.12, 3.13 and 3.14, so that is the floor.
240
+
241
+ ## Status
242
+
243
+ AIT is in active development for local AI experimentation and lightweight, template-driven workflows. Expect the template syntax to be stable and everything around it to keep evolving. See [`CHANGELOG.md`](CHANGELOG.md) for release history.
244
+
245
+ Licensed under the [MIT license](LICENSE).
@@ -0,0 +1,222 @@
1
+ # AI Template Runner
2
+
3
+ Small local models are cheap and fast — they also lose structure the moment a prompt gets long. `ait` turns that weakness into a workflow: each template is a single Markdown file that gathers context, prompts the model, validates the answer and renders the result. Deterministic scaffolding around a dumb-but-fast model, even one running on your phone.
4
+
5
+ ```
6
+ ┌ template file ─────────────────────────────────┐
7
+ │ ```toml → model, endpoint, sampling │
8
+ │ ``` │
9
+ │ --- → system prompt │
10
+ │ user prompt: {cmd.version} output + question │
11
+ │ [test] / [out] → validate + render output │
12
+ └────────────────────────────────────────────────┘
13
+ ```
14
+
15
+ One file, one command, one predictable answer — no agent framework needed.
16
+
17
+ **Why it matters:** a small model cannot keep a workflow in its head, but it can fill in a well-shaped slot. Templates put the workflow in the file so the model only has to do the part it is good at. Failures become visible and reproducible through `[test]`, instead of vague "the model is dumb" sessions.
18
+
19
+ ## Quick start
20
+
21
+ Point `ait` at any OpenAI-compatible chat endpoint (llama.cpp server, Ollama, LM Studio, ...):
22
+
23
+ 1. Start a model server, e.g. llama.cpp:
24
+
25
+ ```bash
26
+ llama-server -hf unsloth/Qwen3-0.6B-GGUF --port 8080
27
+ ```
28
+
29
+ 2. Run a ready template:
30
+
31
+ ```bash
32
+ uvx ai-template-runner run TEMPLATE "What is Python?"
33
+ ```
34
+
35
+ Or clone the repository and try it against a fixture without a model at
36
+ all — the `examples/simulation/` templates run against fixed
37
+ responses, so you can feel the `[test]`/`[out]` mechanics in seconds:
38
+
39
+ ```bash
40
+ git clone https://github.com/brunodavi/ai-template-runner.git
41
+ cd ai-template-runner
42
+ uvx ai-template-runner run examples/simulation/conventional-commits/feat.md --test
43
+ ```
44
+
45
+ Running on a phone? See [`docs/termux.md`](docs/termux.md).
46
+
47
+ ## Installation
48
+
49
+ | Command | When to use |
50
+ | --- | --- |
51
+ | `uvx ai-template-runner --help` | zero-install trial from [PyPI](https://pypi.org/project/ai-template-runner/) |
52
+ | `pipx install ai-template-runner` | persistent install, any platform (Termux ships pipx) |
53
+ | `pip install ai-template-runner` | any pip environment |
54
+ | `uv tool install ai-template-runner` | [uv](https://docs.astral.sh/uv/) managed installs |
55
+ | `uv tool install git+https://github.com/brunodavi/ai-template-runner.git` | latest `master` |
56
+
57
+ Requires Python 3.12+. The CLI is `ait` (or `ai-template-runner`); the package has **zero runtime dependencies**.
58
+
59
+ ## Usage
60
+
61
+ ```bash
62
+ ait run examples/features/full.md "What is Python?"
63
+ ait run examples/features/full.md "What is Python?" --quiet
64
+ ait run examples/simulation/conventional-commits/feat.md --test
65
+ ait --help
66
+ ait run --help
67
+ ```
68
+
69
+ - `-q` / `--quiet`: display only the AI response (stdout stays clean for pipes).
70
+ - `-t` / `--test`: validate the response against the template's `[test]` table. PASS/FAIL lines go to stderr and the exit code is 1 when any check fails.
71
+
72
+ ## Templates
73
+
74
+ Templates combine **TOML configuration** and Markdown prompts in a single file. The configuration is defined in a ` ```toml ` code block at the beginning of the file, and is parsed and removed from the prompt content. Two ` --- ` separators delimit an optional system prompt; everything after them becomes the user prompt. Templates can also execute terminal commands and inject their output directly into prompts.
75
+
76
+ ````markdown
77
+ ```toml
78
+ [provider]
79
+ base_url = 'http://localhost:8080/v1/chat/completions'
80
+ model = 'lfm230m'
81
+ temperature = 0.2
82
+ top_p = 0.9
83
+ top_k = 40
84
+ min_p = 0.05
85
+ max_tokens = 220
86
+ seed = 42
87
+ stop = []
88
+ frequency_penalty = 0.0
89
+ presence_penalty = 0.0
90
+ repeat_penalty = 1.1
91
+
92
+ [provider.extras]
93
+ typical_p = 1.0
94
+ repeat_last_n = 64
95
+
96
+ [cmd]
97
+ message = 'echo hello'
98
+ ```
99
+
100
+ ---
101
+ You are a concise assistant. Use the command output as context.
102
+ ---
103
+
104
+ Command output:
105
+ {cmd.message}
106
+
107
+ Answer the following question concisely: {question}
108
+ ````
109
+
110
+ The TOML block is parsed and removed from the prompt content. The optional system prompt is delimited by two `---` separators; everything after the second separator is the user prompt. When the separators are omitted, the whole content after the configuration block is the user prompt.
111
+
112
+ The `{question}` placeholder is replaced with the CLI input. The `{cmd.message}` placeholder executes the configured command and inserts its output into the prompt. Placeholders can be used in both the system and user prompts.
113
+
114
+ The provider supports these generation parameters:
115
+
116
+ - `temperature`, `top_p`, `top_k`, `min_p`: control sampling.
117
+ - `max_tokens`: limits the generated response length.
118
+ - `seed`: sets the random seed when supported.
119
+ - `stop`: defines stop sequences.
120
+ - `frequency_penalty`, `presence_penalty`, `repeat_penalty`: control repetition.
121
+ - `extras`: passes additional parameters to the API, such as llama.cpp-specific options.
122
+
123
+ `base_url` and `model` are required. Parameter support depends on the API and model server being used.
124
+
125
+ A template can inherit configuration, commands, and prompts from another template with a top-level `extends` key placed before any TOML table:
126
+
127
+ ```toml
128
+ extends = 'base.md'
129
+
130
+ [provider]
131
+ temperature = 0.2
132
+ ```
133
+
134
+ The `extends` path is resolved relative to the template file itself, so [`examples/features/`](examples/features/) templates use `extends = '../base.md'`. Child values win per key; nested tables such as `[provider.extras]` and `[cmd]` are merged key by key. Empty child prompts fall back to the parent prompts, which lets a template reuse only the parent prompts and configuration.
135
+
136
+ A template can validate its own output with a `[test]` table and the `--test` flag:
137
+
138
+ ```toml
139
+ [test]
140
+ type = 'feat' # JSON output field equality
141
+ message = 'add health check endpoint' # exact value comparison
142
+ "user.name" = 'Alice' # nested JSON path (quoted dotted key)
143
+
144
+ text = 'exact raw output' # exact raw text comparison
145
+ text_contains = ['GET_health'] # every fragment present in the raw text
146
+ text_regex = '^\{' # regex search on the raw text
147
+ ```
148
+
149
+ Run with validation:
150
+
151
+ ```bash
152
+ ait run examples/simulation/conventional-commits/feat.md --test
153
+ ```
154
+
155
+ PASS/FAIL lines go to stderr (stdout stays clean for pipes) and the exit code is 1 when any check fails. Missing JSON fields and invalid JSON output fail instead of raising. The `[test]` table participates in `extends` inheritance like the other config tables.
156
+
157
+ A template can also render its own display output with an `[out]` table. The `text` key is a render template formatted with the parsed JSON fields (`{out.type}`, `{out.message}`, ...); missing fields render as empty:
158
+
159
+ ```toml
160
+ [out]
161
+ text = '{out.type}: {out.message}'
162
+ ```
163
+
164
+ Transforms run in two channels that never share the field namespace (`out.count` is always the JSON field, never `str.count`):
165
+
166
+ ```toml
167
+ [out]
168
+ text = '{out.message:strip} | {out.message:replace:GET:SET} | {out.message[:4]}'
169
+
170
+ [[out.upper]] # one step: str.upper() with no args
171
+
172
+ [[out.replace]] # named args match the real str signature
173
+ old = 'GET_health'
174
+ new = 'GET'
175
+
176
+ [[out.slice]] # helper: builtin slice(start, end)
177
+ start = 1
178
+ end = 10
179
+ ```
180
+
181
+ The `[[out.<method>]]` tables run in real document order (one table = one step, an empty table means no arguments) and apply to the rendered `text` — or to the raw response when there is no `text`. Steps are real `str` methods, with named arguments mapped positionally by the real signature. The inline spec is a single step from `method:arg1:arg2` (argument values may not contain ':'), and `{out.field[1:]}` follows pythonic slice syntax. Without `[out]` (or an empty table) the raw AI response is printed. The `[out]` config participates in `extends` inheritance per key: the child `text` and step lists replace the parent's.
182
+
183
+ The same inline transforms work on every interpolated value in the prompts — `{question:strip}`, `{question[:80]:lower}` and `{cmd.message:upper}`, `{cmd.status[1:20]}` — since the value stays lazy for `{cmd.*}` (the command only runs when a field is actually used).
184
+
185
+ ## Examples and docs
186
+
187
+ See [`examples/base.md`](examples/base.md) for the shared base used by the examples, [`examples/features/`](examples/features/) for one example per feature, [`examples/`](examples/) for ready-to-use task templates such as [`email-subject.md`](examples/email-subject.md) and [`conventional-commits.md`](examples/conventional-commits.md), and [`examples/simulation/`](examples/simulation/) for simulations that run task templates against fixed fixtures.
188
+
189
+ For deeper material, read [`docs/architecture.md`](docs/architecture.md) (why the code is
190
+ shaped the way it is), [`docs/termux.md`](docs/termux.md) (running local models on a phone)
191
+ and [`docs/maintenance.md`](docs/maintenance.md) (release routine for maintainers).
192
+
193
+ ## Development
194
+
195
+ ```bash
196
+ git clone https://github.com/brunodavi/ai-template-runner.git
197
+ cd ai-template-runner
198
+ uv sync
199
+ uv pip install -e .
200
+ ```
201
+
202
+ Editable installation (`-e .`) lets you run `ait` directly while developing, so source changes are available without reinstalling.
203
+
204
+ Run a template to test your changes:
205
+
206
+ ```bash
207
+ ait run examples/features/full.md "What is Python?"
208
+ ```
209
+
210
+ Run the test suite (from the repository root):
211
+
212
+ ```bash
213
+ uv run python -m unittest
214
+ ```
215
+
216
+ Contributions are welcome — CI runs the suite on Python 3.12, 3.13 and 3.14, so that is the floor.
217
+
218
+ ## Status
219
+
220
+ AIT is in active development for local AI experimentation and lightweight, template-driven workflows. Expect the template syntax to be stable and everything around it to keep evolving. See [`CHANGELOG.md`](CHANGELOG.md) for release history.
221
+
222
+ Licensed under the [MIT license](LICENSE).
@@ -0,0 +1,44 @@
1
+ [project]
2
+ name = "ai-template-runner"
3
+ version = "1.5.0"
4
+ description = "Simple AI workflow based on Markdown templates"
5
+ readme = "README.md"
6
+ license = "MIT"
7
+ license-files = ["LICENSE"]
8
+ keywords = [
9
+ "ai",
10
+ "llm",
11
+ "templates",
12
+ "llamacpp",
13
+ "local-models",
14
+ "prompt",
15
+ ]
16
+ classifiers = [
17
+ "Development Status :: 4 - Beta",
18
+ "Environment :: Console",
19
+ "Intended Audience :: Developers",
20
+ "Programming Language :: Python :: 3",
21
+ "Programming Language :: Python :: 3.12",
22
+ "Programming Language :: Python :: 3.13",
23
+ "Programming Language :: Python :: 3.14",
24
+ "Topic :: Software Development",
25
+ ]
26
+ requires-python = ">=3.12"
27
+ dependencies = []
28
+
29
+ [[project.authors]]
30
+ name = "brunodavi"
31
+ email = "48166556+brunodavi@users.noreply.github.com"
32
+
33
+ [project.urls]
34
+ Repository = "https://github.com/brunodavi/ai-template-runner"
35
+ Issues = "https://github.com/brunodavi/ai-template-runner/issues"
36
+ Changelog = "https://github.com/brunodavi/ai-template-runner/blob/master/CHANGELOG.md"
37
+
38
+ [project.scripts]
39
+ ai-template-runner = "ai_template_runner:main"
40
+ ait = "ai_template_runner:main"
41
+
42
+ [build-system]
43
+ requires = ["uv_build>=0.12.19,<0.13.0"]
44
+ build-backend = "uv_build"
@@ -0,0 +1,36 @@
1
+ [project]
2
+ name = "ai-template-runner"
3
+ version = "1.5.0"
4
+ description = "Simple AI workflow based on Markdown templates"
5
+ readme = "README.md"
6
+ license = "MIT"
7
+ license-files = ["LICENSE"]
8
+ authors = [
9
+ { name = "brunodavi", email = "48166556+brunodavi@users.noreply.github.com" }
10
+ ]
11
+ keywords = ["ai", "llm", "templates", "llamacpp", "local-models", "prompt"]
12
+ classifiers = [
13
+ "Development Status :: 4 - Beta",
14
+ "Environment :: Console",
15
+ "Intended Audience :: Developers",
16
+ "Programming Language :: Python :: 3",
17
+ "Programming Language :: Python :: 3.12",
18
+ "Programming Language :: Python :: 3.13",
19
+ "Programming Language :: Python :: 3.14",
20
+ "Topic :: Software Development",
21
+ ]
22
+ requires-python = ">=3.12"
23
+ dependencies = []
24
+
25
+ [project.urls]
26
+ Repository = "https://github.com/brunodavi/ai-template-runner"
27
+ Issues = "https://github.com/brunodavi/ai-template-runner/issues"
28
+ Changelog = "https://github.com/brunodavi/ai-template-runner/blob/master/CHANGELOG.md"
29
+
30
+ [project.scripts]
31
+ ai-template-runner = "ai_template_runner:main"
32
+ ait = "ai_template_runner:main"
33
+
34
+ [build-system]
35
+ requires = ["uv_build>=0.12.19,<0.13.0"]
36
+ build-backend = "uv_build"
@@ -0,0 +1,4 @@
1
+ from .cli import cli
2
+
3
+ def main():
4
+ cli()
@@ -0,0 +1,94 @@
1
+ import json
2
+ import re
3
+
4
+
5
+ class CheckResult:
6
+ def __init__(self, name: str, passed: bool, detail: str = ''):
7
+ self.name = name
8
+ self.passed = passed
9
+ self.detail = detail
10
+
11
+
12
+ TEXT_KEY = 'text'
13
+ TEXT_CONTAINS_KEY = 'text_contains'
14
+ TEXT_REGEX_KEY = 'text_regex'
15
+
16
+
17
+ def _quoted(value) -> str:
18
+ return f'{value!r}'
19
+
20
+
21
+ def _json_get(data: dict, dotted_path: str):
22
+ current = data
23
+ for part in dotted_path.split('.'):
24
+ if not isinstance(current, dict) or part not in current:
25
+ return None
26
+ current = current[part]
27
+ return current
28
+
29
+
30
+ def run_checks(test_config: dict, text: str) -> list[CheckResult]:
31
+ """Validate the AI response against the template [test] table.
32
+
33
+ Reserved keys (text, text_contains, text_regex) check the raw
34
+ text; any other key is a dotted path into the parsed JSON output.
35
+ Missing JSON fields and invalid JSON fail without raising.
36
+ """
37
+ results: list[CheckResult] = []
38
+
39
+ expected_json = {
40
+ key: value
41
+ for key, value in test_config.items()
42
+ if key not in (TEXT_KEY, TEXT_CONTAINS_KEY, TEXT_REGEX_KEY)
43
+ }
44
+
45
+ try:
46
+ parsed = json.loads(text)
47
+ parse_error = None
48
+ except (json.JSONDecodeError, TypeError) as err:
49
+ parsed = None
50
+ parse_error = err
51
+
52
+ for key, expected in expected_json.items():
53
+ if parse_error is not None:
54
+ results.append(CheckResult(
55
+ key,
56
+ passed=False,
57
+ detail=f'invalid json output: {text[:100]!r}',
58
+ ))
59
+ continue
60
+
61
+ got = _json_get(parsed, key)
62
+ passed = got == expected
63
+ detail = '' if passed else (
64
+ f'expected {_quoted(expected)}, got {_quoted(got)}'
65
+ )
66
+ results.append(CheckResult(key, passed, detail))
67
+
68
+ if TEXT_KEY in test_config:
69
+ expected = test_config[TEXT_KEY]
70
+ passed = text == expected
71
+ detail = '' if passed else f'expected {_quoted(expected)}, got {_quoted(text)}'
72
+ results.append(CheckResult(TEXT_KEY, passed, detail))
73
+
74
+ if TEXT_CONTAINS_KEY in test_config:
75
+ fragments = test_config[TEXT_CONTAINS_KEY]
76
+ missing = [
77
+ fragment for fragment in fragments
78
+ if fragment not in text
79
+ ]
80
+ passed = not missing
81
+ detail = '' if passed else f'missing fragments: {_quoted(missing)}'
82
+ results.append(CheckResult(TEXT_CONTAINS_KEY, passed, detail))
83
+
84
+ if TEXT_REGEX_KEY in test_config:
85
+ pattern = test_config[TEXT_REGEX_KEY]
86
+ try:
87
+ passed = re.search(pattern, text) is not None
88
+ detail = '' if passed else f'pattern {_quoted(pattern)} not found'
89
+ except re.error as err:
90
+ passed = False
91
+ detail = f'invalid regex {_quoted(pattern)}: {err}'
92
+ results.append(CheckResult(TEXT_REGEX_KEY, passed, detail))
93
+
94
+ return results
@@ -0,0 +1,104 @@
1
+ from argparse import ArgumentParser
2
+ from pathlib import Path
3
+ import sys
4
+
5
+ from .processor import (
6
+ process_template,
7
+ process_prompt,
8
+ process_system_prompt,
9
+ )
10
+ from .providers import llamacpp
11
+ from .checks import run_checks
12
+ from .output import render_out
13
+
14
+
15
+ def cli(argv: list[str] = None):
16
+ parser = ArgumentParser(
17
+ prog="ait",
18
+ description="Simple AI workflow based on Markdown templates",
19
+ )
20
+
21
+ subparsers = parser.add_subparsers(
22
+ dest="command",
23
+ required=True,
24
+ )
25
+
26
+ run = subparsers.add_parser(
27
+ "run",
28
+ help="Run a template",
29
+ description="Run a template",
30
+ )
31
+
32
+ run.add_argument(
33
+ "template_path",
34
+ help="Path template markdown to run",
35
+ type=Path,
36
+ )
37
+
38
+ run.add_argument(
39
+ "question",
40
+ help="Question to answer",
41
+ nargs="?",
42
+ default="",
43
+ )
44
+
45
+ run.add_argument(
46
+ "-q", "--quiet",
47
+ action="store_true",
48
+ help="Show only the AI response",
49
+ )
50
+
51
+ run.add_argument(
52
+ "-t", "--test",
53
+ action="store_true",
54
+ help="Validate the AI response against the [test] table",
55
+ )
56
+
57
+ args = parser.parse_args(argv)
58
+
59
+ if args.command == "run":
60
+ template_path: Path = args.template_path
61
+
62
+ if not args.quiet:
63
+ print(f"Running template: {template_path.name}\n")
64
+
65
+ template = process_template(template_path)
66
+ prompt = process_prompt(template, args.question)
67
+ system_prompt = process_system_prompt(template, args.question)
68
+
69
+ if not args.quiet and system_prompt:
70
+ print("System:", system_prompt)
71
+ print()
72
+
73
+ if not args.quiet:
74
+ print("User:", prompt)
75
+ print()
76
+
77
+ response_json = llamacpp(
78
+ template.provider,
79
+ prompt,
80
+ system_prompt,
81
+ )
82
+
83
+ ai_response = (
84
+ response_json["choices"][0]["message"]["content"]
85
+ )
86
+
87
+ rendered = render_out(template.out, ai_response)
88
+
89
+ if args.quiet:
90
+ print(rendered)
91
+ else:
92
+ print("AI:", rendered)
93
+
94
+ if args.test:
95
+ failed = False
96
+
97
+ for result in run_checks(template.test, ai_response):
98
+ status = 'PASS' if result.passed else 'FAIL'
99
+ line = f'{status} {result.name}' + (f': {result.detail}' if result.detail else '')
100
+ print(line, file=sys.stderr)
101
+ failed = failed or not result.passed
102
+
103
+ if failed:
104
+ raise SystemExit(1)
@@ -0,0 +1,16 @@
1
+ import subprocess
2
+
3
+ from .output import Smart
4
+
5
+ class Command:
6
+ def __init__(self, **kwargs):
7
+ self.commands = kwargs
8
+
9
+ def __getattr__(self, name):
10
+ command = self.commands[name]
11
+ return Smart(subprocess.run(
12
+ command,
13
+ shell=True,
14
+ capture_output=True,
15
+ text=True
16
+ ).stdout.strip())
@@ -0,0 +1,29 @@
1
+ from dataclasses import dataclass, field
2
+ from typing import Any
3
+
4
+
5
+ @dataclass
6
+ class Provider:
7
+ base_url: str
8
+ model: str
9
+
10
+ # Sampling
11
+ temperature: float | None = None
12
+ top_p: float | None = None
13
+ top_k: int | None = None
14
+ min_p: float | None = None
15
+
16
+ # Output
17
+ max_tokens: int | None = None
18
+ stop: list[str] = field(default_factory=list)
19
+
20
+ # Reproducibility
21
+ seed: int | None = None
22
+
23
+ # Repetition control
24
+ frequency_penalty: float | None = None
25
+ presence_penalty: float | None = None
26
+ repeat_penalty: float | None = None
27
+
28
+ # Less common or provider-specific options
29
+ extras: dict[str, Any] = field(default_factory=dict)
@@ -0,0 +1,13 @@
1
+ from dataclasses import dataclass, field
2
+ from ..command import Command
3
+ from .provider import Provider
4
+
5
+ @dataclass
6
+ class Template:
7
+ user_prompt: str
8
+ system_prompt: str = ''
9
+
10
+ provider: Provider = None
11
+ cmd: Command = None
12
+ test: dict = field(default_factory=dict)
13
+ out: dict = field(default_factory=dict)
@@ -0,0 +1,221 @@
1
+ """The [out] table: render of the AI response with transforms.
2
+
3
+ Two mechanisms share one transform executor:
4
+
5
+ - [[out.<method>]] tables build a pipeline applied in real document
6
+ order (array-of-tables group by key in tomllib, so the raw headers
7
+ are numbered with a suffix before parsing and un-numbered after);
8
+ - inline spec inside the render text: `{out.field:method:arg1:arg2}`
9
+ and the pythonic slice `{out.field[1:]}` through a Smart wrapper.
10
+ """
11
+
12
+ import inspect
13
+ import json
14
+ import re
15
+ from tomllib import loads as toml_loads
16
+
17
+
18
+ OUT_HEADER = re.compile(r'\[\[out\.(\w+)\]\]')
19
+
20
+ SLICE_METHOD = 'slice'
21
+ TEXT_KEY = 'text'
22
+ STEPS_KEY = 'steps'
23
+
24
+
25
+ def number_out_headers(toml_str: str) -> str:
26
+ """Number [[out.<name>]] headers so document order survives tomllib.
27
+
28
+ tomllib groups array-of-tables by key, losing the order between
29
+ different method names. Adding a per-name occurrence suffix gives
30
+ every header a unique key; build_out() removes the suffix again.
31
+ The suffix is not matched by the pattern itself (\w stops at '-'),
32
+ so numbering stays idempotent.
33
+ """
34
+ counters: dict[str, int] = {}
35
+
36
+ def rename(match: re.Match) -> str:
37
+ name = match.group(1)
38
+ counters[name] = counters.get(name, 0) + 1
39
+ return f'[[out.{name}-{counters[name]}]]'
40
+
41
+ return OUT_HEADER.sub(rename, toml_str)
42
+
43
+
44
+ def build_out(raw_out: dict | None) -> dict:
45
+ """Canonicalize the parsed [out] table into {text, steps}.
46
+
47
+ Numbered keys (built with number_out_headers) are un-suffixed and
48
+ flattened into an ordered steps list of (method, args) tuples,
49
+ where args is the step table (empty table = no arguments).
50
+ """
51
+ if not raw_out:
52
+ return {}
53
+
54
+ built = {}
55
+
56
+ if raw_out.get(TEXT_KEY):
57
+ built[TEXT_KEY] = raw_out[TEXT_KEY]
58
+
59
+ steps = []
60
+ for key, entries in raw_out.items():
61
+ if key == TEXT_KEY:
62
+ continue
63
+ method = key.rsplit('-', 1)[0]
64
+ for entry in entries:
65
+ steps.append((method, dict(entry)))
66
+
67
+ if steps:
68
+ built[STEPS_KEY] = steps
69
+
70
+ return built
71
+
72
+
73
+ def _positional_by_signature(method, args: dict, name: str) -> list:
74
+ """Map named args to positional order using the method's real signature."""
75
+ try:
76
+ params = [
77
+ param
78
+ for param in inspect.signature(method).parameters
79
+ if param != 'self'
80
+ ]
81
+ except (ValueError, TypeError):
82
+ raise ValueError(
83
+ f'[out] cannot map named arguments for transform {name!r}'
84
+ ) from None
85
+
86
+ unknown = set(args) - set(params)
87
+ if unknown:
88
+ raise ValueError(
89
+ f'[out] unknown arguments for {name}: {sorted(unknown)}'
90
+ )
91
+
92
+ return [args[param] for param in params if param in args]
93
+
94
+
95
+ def _slice_key(args) -> slice:
96
+ if isinstance(args, dict):
97
+ return slice(args.get('start'), args.get('end'))
98
+
99
+ start = args[0] if len(args) > 0 else None
100
+ end = args[1] if len(args) > 1 else None
101
+ return slice(
102
+ int(start) if start not in (None, '') else None,
103
+ int(end) if end not in (None, '') else None,
104
+ )
105
+
106
+
107
+ def apply_op(text: str, name: str, args) -> str:
108
+ """Run one transform: a real str method, or the 'slice' helper.
109
+
110
+ args is a named dict (from [[out.<method>]] tables, mapped to
111
+ positional by the method signature) or a positional list (from an
112
+ inline spec). An empty args table calls the method without args.
113
+ """
114
+ if name == SLICE_METHOD:
115
+ return text[_slice_key(args)]
116
+
117
+ if not hasattr(str, name) or name.startswith('_'):
118
+ raise ValueError(f'[out] unknown transform: {name!r}')
119
+
120
+ method = getattr(str, name)
121
+
122
+ if not args:
123
+ return method(text)
124
+
125
+ if isinstance(args, dict):
126
+ args = _positional_by_signature(method, args, name)
127
+
128
+ return method(text, *args)
129
+
130
+
131
+ def run_spec(text: str, spec: str) -> str:
132
+ """Run the inline spec: 'method' | 'method:arg1:arg2' | 'slice:1:10'.
133
+
134
+ One step only, positional args separated by ':' (values may not
135
+ contain ':'). An empty spec is a pass-through.
136
+ """
137
+ if not spec:
138
+ return text
139
+
140
+ name, *args = spec.split(':')
141
+ return apply_op(text, name, args)
142
+
143
+
144
+ def parse_slice_key(key: str) -> slice:
145
+ """Parse a format subscript key like '1:', ':4' or '1:10' into a slice."""
146
+ start, _, end = key.partition(':')
147
+ return slice(
148
+ int(start) if start else None,
149
+ int(end) if end else None,
150
+ )
151
+
152
+
153
+ class Smart(str):
154
+ """Interpolable value: transforms only via :spec and [slice].
155
+
156
+ The format machinery passes subscript keys as strings to
157
+ __getitem__ ('1:' -> slice) and the format spec to __format__
158
+ ('replace:GET:SET'), where we run them through apply_op. Because
159
+ transforms never use attribute names, JSON fields keep their own
160
+ namespace (a field called 'count' is data, never str.count).
161
+ """
162
+
163
+ def __getitem__(self, key):
164
+ if isinstance(key, str):
165
+ key = parse_slice_key(key)
166
+ return Smart(super().__getitem__(key))
167
+
168
+ def __format__(self, spec: str) -> str:
169
+ return run_spec(str(self), spec)
170
+
171
+
172
+ class Rec:
173
+ """Wraps the parsed JSON response so {out.*} resolves to fields.
174
+
175
+ Missing fields render as ''; nested dicts may be addressed with
176
+ dotted attribute access. Values are Smart, so :spec and [slice]
177
+ work on any field.
178
+ """
179
+
180
+ def __init__(self, fields: dict):
181
+ object.__setattr__(self, '_fields', fields or {})
182
+
183
+ def __getattr__(self, field: str):
184
+ if field.startswith('_'):
185
+ raise AttributeError(field)
186
+
187
+ if field not in self._fields:
188
+ return Smart('')
189
+
190
+ value = self._fields[field]
191
+ return Rec(value) if isinstance(value, dict) else Smart(value)
192
+
193
+
194
+ def _response_fields(ai_response: str) -> dict:
195
+ try:
196
+ parsed = json.loads(ai_response)
197
+ except (json.JSONDecodeError, TypeError):
198
+ return {}
199
+
200
+ return parsed if isinstance(parsed, dict) else {}
201
+
202
+
203
+ def render_out(out: dict | None, ai_response: str) -> str:
204
+ """Render the AI response through the [out] config.
205
+
206
+ Without [out] (or an empty table) the raw response passes through.
207
+ With 'text', the parsed JSON fields are formatted through Rec and
208
+ Smart; without 'text' the raw response is the starting value. The
209
+ steps from [[out.<method>]] tables are applied in document order.
210
+ """
211
+ if not out:
212
+ return ai_response
213
+
214
+ base = out.get(TEXT_KEY) or ai_response
215
+ if base is not ai_response:
216
+ base = base.format(out=Rec(_response_fields(ai_response)))
217
+
218
+ for name, args in out.get(STEPS_KEY, []):
219
+ base = apply_op(base, name, args)
220
+
221
+ return base
@@ -0,0 +1,191 @@
1
+ from pathlib import Path
2
+ from tomllib import loads
3
+
4
+ from .models.template import Template
5
+ from .models.provider import Provider
6
+ from .command import Command
7
+ from .output import build_out, number_out_headers, Smart
8
+
9
+ TOML_FENCE = '```toml'
10
+ FENCE_END = '```'
11
+ SYSTEM_SEPARATOR = '---'
12
+
13
+ EXTENDS_KEY = 'extends'
14
+
15
+
16
+ def extract_toml_block(template_str: str) -> tuple[str, str]:
17
+ """Extract the TOML configuration block from the template.
18
+
19
+ The block must be the first content of the Markdown file and is
20
+ delimited by a ```toml fenced code block. Returns the TOML source
21
+ and the template content without the block.
22
+ """
23
+ lines = template_str.splitlines()
24
+
25
+ start = 0
26
+ while start < len(lines) and not lines[start].strip():
27
+ start += 1
28
+
29
+ if start == len(lines) or lines[start].strip() != TOML_FENCE:
30
+ raise ValueError(
31
+ "Template must begin with a '```toml' configuration block"
32
+ )
33
+
34
+ end = next(
35
+ (
36
+ i
37
+ for i in range(start + 1, len(lines))
38
+ if lines[i].strip() == FENCE_END
39
+ ),
40
+ None,
41
+ )
42
+
43
+ if end is None:
44
+ raise ValueError(
45
+ "The '```toml' configuration block is not closed"
46
+ )
47
+
48
+ toml_str = '\n'.join(lines[start + 1:end])
49
+ content = '\n'.join(lines[end + 1:])
50
+
51
+ return toml_str, content
52
+
53
+
54
+ def split_prompts(content: str) -> tuple[str, str]:
55
+ """Split the template content into a system prompt and a user prompt.
56
+
57
+ The system prompt is optional and delimited by two '---' separators.
58
+ Everything after the second separator is the user prompt.
59
+ """
60
+ lines = content.splitlines()
61
+ separators = [
62
+ i for i, line in enumerate(lines) if line.strip() == SYSTEM_SEPARATOR
63
+ ]
64
+
65
+ if len(separators) < 2:
66
+ return '', content.strip()
67
+
68
+ [first, second] = separators[:2]
69
+ system_prompt = '\n'.join(lines[first + 1:second])
70
+ user_prompt = '\n'.join(lines[second + 1:])
71
+
72
+ return system_prompt.strip(), user_prompt.strip()
73
+
74
+
75
+ def merge_configs(parent: dict, child: dict) -> dict:
76
+ """Merge two TOML config dicts, child values win.
77
+
78
+ Nested tables (e.g. [provider.extras] or [cmd]) are merged
79
+ per additional table level, so child tables inherit parent keys.
80
+ """
81
+ merged = dict(parent)
82
+ for key, value in child.items():
83
+ if isinstance(value, dict) and isinstance(merged.get(key), dict):
84
+ merged[key] = merge_configs(merged[key], value)
85
+ else:
86
+ merged[key] = value
87
+ return merged
88
+
89
+
90
+ def load_template(
91
+ template_path: Path,
92
+ _extends_stack: frozenset[Path] = frozenset(),
93
+ ) -> tuple[dict, dict, dict, dict, str, str]:
94
+ """Load a template file and resolve its `extends` chain.
95
+
96
+ Returns (provider config, cmd config, test config, out config,
97
+ system prompt, user prompt). The `extends` path is resolved relative
98
+ to the template file itself; parent provider/cmd/test/out configs
99
+ are merged per key and parent prompts are used wherever the child
100
+ prompts are empty.
101
+ """
102
+ template_str = template_path.read_text()
103
+
104
+ toml_str, content = extract_toml_block(template_str)
105
+
106
+ config = loads(number_out_headers(toml_str))
107
+ extends = config.pop(EXTENDS_KEY, None)
108
+
109
+ provider = dict(config.get('provider', {}))
110
+ cmd = dict(config.get('cmd', {}))
111
+ test = dict(config.get('test', {}))
112
+ out = build_out(config.get('out'))
113
+
114
+ system_prompt, user_prompt = split_prompts(content)
115
+
116
+ if extends:
117
+ parent_path = template_path.parent / extends
118
+
119
+ if not parent_path.is_file():
120
+ raise ValueError(
121
+ f"Template 'extends' file not found: {extends}"
122
+ )
123
+
124
+ parent_resolved = parent_path.resolve()
125
+
126
+ if parent_resolved in _extends_stack:
127
+ raise ValueError(
128
+ f"circular 'extends' in template chain: {extends}"
129
+ )
130
+
131
+ parent = load_template(
132
+ parent_path,
133
+ _extends_stack | {parent_resolved},
134
+ )
135
+
136
+ parent_provider, parent_cmd, parent_test, parent_out, parent_system, parent_user = (
137
+ parent
138
+ )
139
+
140
+ provider = merge_configs(parent_provider, provider)
141
+ cmd = merge_configs(parent_cmd, cmd)
142
+ test = merge_configs(parent_test, test)
143
+ out = merge_configs(parent_out, out)
144
+
145
+ system_prompt = system_prompt or parent_system
146
+ user_prompt = user_prompt or parent_user
147
+
148
+ return provider, cmd, test, out, system_prompt, user_prompt
149
+
150
+
151
+ def process_template(template_path: Path) -> Template:
152
+ provider_config, cmd_config, test_config, out_config, system_prompt, user_prompt = (
153
+ load_template(template_path)
154
+ )
155
+
156
+ if not provider_config.get('base_url') or not provider_config.get('model'):
157
+ raise ValueError(
158
+ "Provider 'base_url' and 'model' are required"
159
+ )
160
+
161
+ return Template(
162
+ user_prompt,
163
+ system_prompt=system_prompt,
164
+ provider=Provider(**provider_config),
165
+ cmd=Command(**cmd_config),
166
+ test=test_config,
167
+ out=out_config,
168
+ )
169
+
170
+
171
+ def interpolate(text: str, template: Template, question: str | None) -> str:
172
+ return text.format(
173
+ question=Smart(question) if isinstance(question, str) else question,
174
+ cmd=template.cmd,
175
+ ).strip()
176
+
177
+
178
+ def process_prompt(
179
+ template: Template,
180
+ question: str | None = None
181
+ ) -> str:
182
+
183
+ return interpolate(template.user_prompt, template, question)
184
+
185
+
186
+ def process_system_prompt(
187
+ template: Template,
188
+ question: str | None = None
189
+ ) -> str:
190
+
191
+ return interpolate(template.system_prompt, template, question)
@@ -0,0 +1,50 @@
1
+ import json
2
+ import urllib.request
3
+
4
+ from .models.provider import Provider
5
+
6
+
7
+ def post_json(url, payload):
8
+ data = json.dumps(payload).encode()
9
+ req = urllib.request.Request(
10
+ url,
11
+ data=data,
12
+ headers={"Content-Type": "application/json"}
13
+ )
14
+ with urllib.request.urlopen(req, timeout=180) as response:
15
+ return json.loads(response.read())
16
+
17
+
18
+ def llamacpp(
19
+ provider: Provider,
20
+ user_prompt: str,
21
+ system_prompt: str = '',
22
+ ):
23
+
24
+ url = provider.base_url
25
+
26
+ payload = {
27
+ "model": provider.model,
28
+ "messages": [
29
+ {"role": "system", "content": system_prompt},
30
+ {"role": "user", "content": user_prompt},
31
+ ],
32
+ "temperature": provider.temperature,
33
+ "top_p": provider.top_p,
34
+ "top_k": provider.top_k,
35
+ "min_p": provider.min_p,
36
+
37
+ "max_tokens": provider.max_tokens,
38
+ "seed": provider.seed,
39
+ "stop": provider.stop,
40
+
41
+ "frequency_penalty": provider.frequency_penalty,
42
+ "presence_penalty": provider.presence_penalty,
43
+ "repeat_penalty": provider.repeat_penalty,
44
+
45
+ "stream": False,
46
+
47
+ **provider.extras,
48
+ }
49
+
50
+ return post_json(url, payload)