samagotchi 0.94.0 → 0.95.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +38 -1
- data/README.md +7 -7
- data/bin/chi +6 -2
- data/docs/architecture.md +4 -5
- data/docs/cli.md +3 -3
- data/docs/configuration.md +49 -86
- data/docs/hooks.md +1 -1
- data/docs/internals/context-telemetry.md +1 -1
- data/docs/internals/engine.md +7 -4
- data/docs/testing.md +6 -9
- data/lib/samagotchi/bootstrap/config_writer.rb +1 -5
- data/lib/samagotchi/bundle_command/try/sandbox.rb +8 -2
- data/lib/samagotchi/bundles/system/config_modification_protocol.md +6 -7
- data/lib/samagotchi/bundles/system/manifest.yml +3 -3
- data/lib/samagotchi/bundles/system/self_map.md +1 -1
- data/lib/samagotchi/client.rb +59 -196
- data/lib/samagotchi/config.rb +74 -15
- data/lib/samagotchi/engine.rb +7 -7
- data/lib/samagotchi/host_config.rb +20 -27
- data/lib/samagotchi/host_registry.rb +8 -8
- data/lib/samagotchi/kernel_loop.rb +46 -8
- data/lib/samagotchi/model_profile.rb +2 -3
- data/lib/samagotchi/self_report.rb +0 -2
- data/lib/samagotchi/system_prompt.rb +7 -21
- data/lib/samagotchi/terminal_ui/thinking_line.rb +2 -4
- data/lib/samagotchi/thinking.rb +35 -2
- data/lib/samagotchi/thinking_notices.rb +11 -4
- data/lib/samagotchi/version.rb +1 -1
- data/lib/samagotchi/vision_support.rb +0 -3
- metadata +1 -1
|
@@ -18,7 +18,7 @@ Single registry `Samagotchi::Config` (`Config::ENTRIES` in `lib/samagotchi/confi
|
|
|
18
18
|
|
|
19
19
|
Sections forbid `_`/`-` (`SECTION_RE` `/\A[a-z0-9]+\z/`); leaves keep `snake_case` in YAML (`base_url`) and become kebab in CLI (`base-url`) via registry derivation — no generic string split, registry lookup avoids flat vs nested collision.
|
|
20
20
|
|
|
21
|
-
**Universal entries** (`expose: [:env,:config,:cli]`): `default.model`, `server.host/port/
|
|
21
|
+
**Universal entries** (`expose: [:env,:config,:cli]`): `default.model`, `server.host/port/open_timeout/read_timeout`, `server.first_token_timeout` (env and config only), `recap.model/base_url/host_ref/inactivity/timeout/min_user_turns/sentences`, `session.retention_days/max_count/keep_status/sweep_interval_hours/idle_exit_minutes`, `session.shared/keep_empty/max_children` (env and config only), `log.file/disable`, `status.line`, `context.status/window_tokens/chars_per_token/status_thresholds`, `memory.index_warn_tokens`, `default.max_tokens`, `max_tool_output_chars`, `retry.max/base_delay/max_delay`, `read.*`, `execute.*`, `web.port/host`, `no_interrupt`, `no_default_input` etc. (`Config::ENTRIES`; `expose` says which of env/config/cli each takes). Precedence is `CLI > ENV > file > default`.
|
|
22
22
|
|
|
23
23
|
Example `config.yml` (new nested form, preferred):
|
|
24
24
|
|
|
@@ -29,7 +29,6 @@ default:
|
|
|
29
29
|
server:
|
|
30
30
|
host: localhost
|
|
31
31
|
port: 8080
|
|
32
|
-
transport: llama_cpp # llama_cpp|mlx|omlx
|
|
33
32
|
first_token_timeout: 120 # seconds a request may wait for its first token; 0 = off; unset: 120 for remote hosts, none for local
|
|
34
33
|
recap:
|
|
35
34
|
host_ref: small-box # a name under hosts:, or base_url: http://...
|
|
@@ -57,8 +56,8 @@ Only the nested form is read. An env name used as a top-level key (`SAMAGOTCHI_D
|
|
|
57
56
|
**Excluded maps** (YAML-only, not part of the flat registry; skipped by scalar loader):
|
|
58
57
|
|
|
59
58
|
- `model_aliases:` map of alias → model id (`ModelRef.parse`, through `HostRegistry#resolve`). Keys lowercased on write (`ConfigFile.write_model_alias!`). Values may be bare `model` or qualified `host:model` (hybrid; only `:` names a host, `org/model` is an id). Aliases apply once: never point an alias at another alias.
|
|
60
|
-
- `hosts:` map of `name → {host, port | url,
|
|
61
|
-
- `models:` map of model id or alias → `{profile, vision, sampling, thinking, reasoning_history}` (`ConfigFile.model_settings`). `thinking:` (here, on a host, or `thinking.level` for every model) is `off|low|medium|high|default` (`Thinking`; unquoted `off` works, `on` is not a level; `default` = send nothing). Precedence: the session's own level (`/thinking LEVEL`, saved in the session; `chi --thinking` on a session run, `chi send --new --thinking`) > `chi web --thinking`/`SAMAGOTCHI_THINKING_LEVEL` > `models:` > `hosts.<name>.thinking` > `thinking.level` in config.yml > `default`. A `sampling:` key wins over the fields a level sends (`null` drops one); `docs/configuration.md` "Thinking". `reasoning_history:` (here or on a host) is `drop|turn|all`, which of its reasoning a chat host's model gets back. Precedence: the session's own level (`/reasoning-history LEVEL`, saved in the session) > `SAMAGOTCHI_REASONING_HISTORY` > `models:` > `hosts.<name>.reasoning_history` > `turn`; `docs/configuration.md` "Reasoning history". `/thinking` and `/reasoning-history` ask before a change that restarts the prompt cache; `--yes` applies it (`chi send` commands apply directly). `sampling:` (here or on a host) is a map of request fields passed to the provider as written (`temperature`, `top_p`, `presence_penalty`, `repeat_penalty`, …; a model's fields win over its host's per field; `null` = don't send; chi's own fields like `max_tokens`/`stream` are refused with a warning; `docs/configuration.md` "Sampling"). Keys match case-insensitively. `profile` (here or on a host) is `qwen36|gemma4`: the raw prompt format for native hosts. Precedence: `--profile`/`SAMAGOTCHI_MODEL_PROFILE` > `models:` > `hosts.<name>.profile` > the llama.cpp server's chat template > the name (`qwen`/`gemma`) > `qwen36` (`ModelProfile.resolve`). Set one when a model's name hides its family (e.g. a Qwen fine-tune under another name
|
|
59
|
+
- `hosts:` map of `name → {host, port | url, api, api_key_env, auth, dialect, profile, first_token_timeout, vision, sampling, thinking, enabled, models}` (`ConfigFile.hosts_config`, `host_registry.rb` `HostEntry`). Names lowercased; `url:` (http/https, optional path) replaces host/port, never both; `api_key_env:` names the env var holding the API key (never write a key into config.yml); `auth:` (`bearer|x-api-key`) is the header the key goes in (default `x-api-key` on `api: anthropic`, Bearer elsewhere); `dialect:` (`fireworks`, or `standard` for the default; `api: anthropic` only) adjusts the Messages wire for that provider (Bearer by default, no thinking `display`, no `cache_control`, `x-session-affinity`; `LLM::Dialect`, docs/configuration.md "Anthropic Messages hosts"); `first_token_timeout` (seconds, `0` = off; a negative or non-number warns and is ignored) overrides `server.first_token_timeout` for that host; `models:` (a map of model ids, or a plain list) declares ids the host serves whatever its `/v1/models` lists: no "doesn't list" warning, and a bare id routes there as if listed (`HostModel`); an entry may hold `price: {input, output, cache_read?, cache_write?}` in USD per 1M tokens (`ModelPrice`; an unset cache rate is the input rate), which estimates a cost the provider doesn't report or reports as 0 (`cost_estimate`, shown as `~$`), and `served: [ids] | any`, the models a gateway may answer that id with (no served-model warning for them); model settings stay in the root `models:`; a worker reads `hosts:` from the file as it starts (no environment variable sets hosts).
|
|
60
|
+
- `models:` map of model id or alias → `{profile, vision, sampling, thinking, reasoning_history}` (`ConfigFile.model_settings`). `thinking:` (here, on a host, or `thinking.level` for every model) is `off|low|medium|high|default` (`Thinking`; unquoted `off` works, `on` is not a level; `default` = send nothing). Precedence: the session's own level (`/thinking LEVEL`, saved in the session; `chi --thinking` on a session run, `chi send --new --thinking`) > `chi web --thinking`/`SAMAGOTCHI_THINKING_LEVEL` > `models:` > `hosts.<name>.thinking` > `thinking.level` in config.yml > `default`. A `sampling:` key wins over the fields a level sends (`null` drops one); `docs/configuration.md` "Thinking". `reasoning_history:` (here or on a host) is `drop|turn|all`, which of its reasoning a chat host's model gets back. Precedence: the session's own level (`/reasoning-history LEVEL`, saved in the session) > `SAMAGOTCHI_REASONING_HISTORY` > `models:` > `hosts.<name>.reasoning_history` > `turn`; `docs/configuration.md` "Reasoning history". `/thinking` and `/reasoning-history` ask before a change that restarts the prompt cache; `--yes` applies it (`chi send` commands apply directly). `sampling:` (here or on a host) is a map of request fields passed to the provider as written (`temperature`, `top_p`, `presence_penalty`, `repeat_penalty`, …; a model's fields win over its host's per field; `null` = don't send; chi's own fields like `max_tokens`/`stream` are refused with a warning; `docs/configuration.md` "Sampling"). Keys match case-insensitively. `profile` (here or on a host) is `qwen36|gemma4`: the raw prompt format for native hosts. Precedence: `--profile`/`SAMAGOTCHI_MODEL_PROFILE` > `models:` > `hosts.<name>.profile` > the llama.cpp server's chat template > the name (`qwen`/`gemma`) > `qwen36` (`ModelProfile.resolve`). Set one when a model's name hides its family (e.g. a Qwen fine-tune under another name whose server publishes no template).
|
|
62
61
|
- `hooks:` map of `hooks_dir` + per-event lists `{path, on_error}` (`Hooks::Loader.load`). `hooks_dir` may start with `~`.
|
|
63
62
|
- `guardrails:` tool-call rules (`docs/guardrails.md`; read in `Engine#guardrail_rules`, parsed by `Guardrails::Rules.parse`): `enabled` (bool, default true; `false` drops rules and hooks' asks, a deny still applies; config.yml only, like every `guardrails` setting: no env variable), `parent_answers` (`soft`, the default: a parent agent's `chi answer` may deny, answer, Continue and give "Allow once"; `standard`: no allow; `hard`: deny, Stop and plain answers only; never a wider scope; replaces `parent_approvals` and `turn.parent_continue`; a convention for an honest parent, not a security boundary, and the user's choice: set it only when the user asks), `rules:` (list), `disable:` (a rule id or a list of them, `id` or `bundle:id`, switching off a bundle's or config rule without editing it), `small_models` (`auto`, the default: small = 32B or less read from the name; a glob or list of globs; `[]` for none). A rule takes only `id`, `tool`, `command`, `path`, `git`, `models`, `verdict`, `reason`, `scopes`: `id` required; at least one of `tool` (a name, `shell` = execute + task_create, a `File.fnmatch` glob like `"mcp_*"`, or a list), `command` (a Ruby regex on the shell command), `path` (a glob, or `outside_repo`: outside the session's repo, tmp folders and memory files excepted), `git` (only `outside_repo`: a shell call runs commit/add/reset/… in another checkout); `models` (optional: `small`, a model-name glob, or a list; without it the rule is for every model); `verdict` `ask|deny`; `scopes` (for `ask`) a subset of `once, session, repo, rule`. **Any parse error (an unknown key, a bad regex, no verdict) makes chi deny every tool call** until fixed, so validate with `YAML.safe_load` and keep the list shape. Rules are read again on the next tool call after config.yml or a bundle's rules change: no restart needed. Installed bundles' rule files (`chi bundle install guardrails`) add to them; `/guardrails` lists what loaded.
|
|
64
63
|
|
|
@@ -132,8 +131,8 @@ A key never goes into config.yml: `api_key_env: VAR` names it. chi reads `VAR` f
|
|
|
132
131
|
## Validations
|
|
133
132
|
|
|
134
133
|
- **Model name** (`default.model` / `SAMAGOTCHI_DEFAULT_MODEL`): `ModelProfile.required_model_name` — non-empty string, otherwise harness fails fast at startup. Via `Config.get("default.model")`.
|
|
135
|
-
- **Host api** (`hosts.<name>.api`): `llama_cpp
|
|
136
|
-
- **
|
|
134
|
+
- **Host api** (`hosts.<name>.api`): `llama_cpp` (raw-prompt loop over llama.cpp's own API), `openai` (chat loop at `http://HOST:PORT/v1`) or `anthropic` (chat loop over the Messages API at `{url}/v1/messages`; without `url:`/`host:` it is `https://api.anthropic.com`). Absent: raw-prompt loop. It replaces the removed `backend` setting.
|
|
135
|
+
- **No transport key**: `server.transport`, `hosts.<name>.transport` and `SAMAGOTCHI_SERVER_TRANSPORT` were removed (llama.cpp is the only raw-prompt server; never write them). Set to `mlx`/`omlx` (or `api: mlx`/`omlx`), chi stops at start with one line (`ConfigFile::RemovedApi`): mlx_lm.server and oMLX are `url: http://HOST:PORT/v1, api: openai` hosts. Any other value warns at start (`ConfigFile.removed_transport_problem`); remove the key.
|
|
137
136
|
- **Profile** (`models.<id>.profile`, `hosts.<name>.profile`, `SAMAGOTCHI_MODEL_PROFILE`): enum `qwen36|gemma4`; an unknown one warns and is ignored.
|
|
138
137
|
- **Alias name** (`write_model_alias!`):
|
|
139
138
|
- required, non-empty, no whitespace, not starting with `-`, no `/`, must match `/\A[a-z0-9][a-z0-9._-]*\z/i`
|
|
@@ -141,7 +140,7 @@ A key never goes into config.yml: `api_key_env: VAR` names it. chi reads `VAR` f
|
|
|
141
140
|
- must not point to itself (case-insensitive)
|
|
142
141
|
- keys are normalized to downcase on write — `Qwen` and `qwen` collide
|
|
143
142
|
- **Alias target**: non-empty string (model id).
|
|
144
|
-
- **Hosts**: each entry needs `host` (or `url:`, an http(s) URL with an optional path, replacing `host`/`port`; never both), `port` 1-65535 (default 8080), `enabled: false` skips the entry, `
|
|
143
|
+
- **Hosts**: each entry needs `host` (or `url:`, an http(s) URL with an optional path, replacing `host`/`port`; never both), `port` 1-65535 (default 8080), `enabled: false` skips the entry, `api` optional (`llama_cpp`, `openai`, `anthropic`), name must match `/\A[a-z0-9][a-z0-9._-]*\z/i`.
|
|
145
144
|
- **Hooks**: each entry must have `path` (relative to `hooks_dir`), `on_error` is `skip` (default) or `log`; `required: true` on a `before_tool_call` hook denies the call when the hook raises. Class name must match file basename snake→Pascal.
|
|
146
145
|
- **Scalars via registry**: `Config.coerce` validates `String/Numeric/true/false` per `type: :string/:integer/:float/:bool/:enum`; invalid values warn and fall back to entry `default`.
|
|
147
146
|
- **Keys**: `validate_yaml_sections` flags unknown keys (with a suggestion), a registry section that isn't a mapping, and env/CLI-only keys (`model.profile`).
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: samagotchi-system
|
|
3
|
-
version: 0.
|
|
3
|
+
version: 0.95.0
|
|
4
4
|
scope: system
|
|
5
5
|
description: Default system memories — identity, self map, config modification protocol, memory guide and the delegated-session rules
|
|
6
6
|
files:
|
|
7
7
|
identity.md: sha256:29c2f7df9f6c5acbc5ec17000a4490741e55e426af15ca0d81c8706edd9bd027
|
|
8
|
-
self_map.md: sha256:
|
|
9
|
-
config_modification_protocol.md: sha256:
|
|
8
|
+
self_map.md: sha256:df3f8da539a9f5812f79e089c2c8b9894da6124a89654f292160046a6bb060da
|
|
9
|
+
config_modification_protocol.md: sha256:ff4efccf2bdde453fc4e6741df8c18f78da78e353f6b1a5a64d0e7a631aea258
|
|
10
10
|
memory_guide.md: sha256:c48eea744db69e4f3e9bb415e9a03a8c78bbcaa1205ad3b1fb157a079f92d22e
|
|
11
11
|
delegated.md: sha256:a0d6ddcd4a8be06526d964898611775fd251cfd9d145a9d16afaccaa503597bd
|
|
@@ -68,7 +68,7 @@
|
|
|
68
68
|
|
|
69
69
|
## Where things live (relative to the source dir)
|
|
70
70
|
- `docs/`: `cli.md` (flags, subcommands, REPL commands), `configuration.md` (config.yml keys,
|
|
71
|
-
hosts,
|
|
71
|
+
hosts, the server api), `hooks.md` (hook events table), `plugins.md` (plugin API, btw/mcp bundles),
|
|
72
72
|
`guardrails.md`, `memory.md`, `sessions.md`. `README.md` is only the quickstart.
|
|
73
73
|
- `lib/samagotchi/tools/<tool>.rb`: each tool's real limits and defaults (e.g. `execute.rb`,
|
|
74
74
|
`output_guardrails.rb`). Tool descriptions are summaries, not the spec.
|
data/lib/samagotchi/client.rb
CHANGED
|
@@ -12,14 +12,13 @@ require_relative "vision_support"
|
|
|
12
12
|
require_relative "sampling_settings"
|
|
13
13
|
|
|
14
14
|
module Samagotchi
|
|
15
|
-
# Thin HTTP client for llama.cpp's native /completion endpoint
|
|
16
|
-
#
|
|
15
|
+
# Thin HTTP client for llama.cpp's native /completion endpoint (chi's
|
|
16
|
+
# raw-prompt loop; any other server is a chat host, api: openai).
|
|
17
17
|
# Configured by the server.* settings (Samagotchi::Config):
|
|
18
18
|
# server.host (default: localhost)
|
|
19
|
-
# server.port (default: 8080
|
|
19
|
+
# server.port (default: 8080)
|
|
20
20
|
# server.open_timeout (default: 10 seconds)
|
|
21
21
|
# server.read_timeout (default: 600 seconds)
|
|
22
|
-
# server.transport (llama_cpp|mlx|omlx, default: llama_cpp)
|
|
23
22
|
class Client
|
|
24
23
|
# The shared HTTP layer's errors, under their old names.
|
|
25
24
|
RequestCancelled = LLM::RequestCancelled
|
|
@@ -36,8 +35,13 @@ module Samagotchi
|
|
|
36
35
|
# #clear_props_failures!).
|
|
37
36
|
PROPS_FAILURE_TTL = 30
|
|
38
37
|
|
|
39
|
-
|
|
40
|
-
|
|
38
|
+
# llama.cpp's native API: the name in error lines (when the host has
|
|
39
|
+
# none) and its endpoints. llama.cpp's /props carries the per-slot
|
|
40
|
+
# n_ctx (-c split across --parallel slots) and the chat template.
|
|
41
|
+
LABEL = "llama.cpp"
|
|
42
|
+
COMPLETION_PATH = "/completion"
|
|
43
|
+
MODELS_PATH = "/models"
|
|
44
|
+
PROPS_PATH = "/props"
|
|
41
45
|
|
|
42
46
|
# One /props cache per host per process. A host has more than one Client
|
|
43
47
|
# in a run (the registry's, a rebuilt HostRegistry's, the Engines of a
|
|
@@ -64,98 +68,30 @@ module Samagotchi
|
|
|
64
68
|
end
|
|
65
69
|
end
|
|
66
70
|
|
|
67
|
-
#
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
class Transport
|
|
73
|
-
attr_reader :name
|
|
74
|
-
|
|
75
|
-
# @param name [Symbol] one of Client::VALID_TRANSPORTS
|
|
76
|
-
# @param model_resolver [Proc, nil] client-installed resolver for the
|
|
77
|
-
# request's `model` field (oMLX resolves against its /v1/models list;
|
|
78
|
-
# mlx installs one that always returns nil to omit the field); nil
|
|
79
|
-
# means forward the selector verbatim (llama.cpp default)
|
|
80
|
-
def initialize(name, model_resolver: nil)
|
|
81
|
-
@name = name
|
|
82
|
-
@model_resolver = model_resolver
|
|
83
|
-
end
|
|
84
|
-
|
|
85
|
-
def label
|
|
86
|
-
# oMLX gets its own label so its error paths read "omlx ...", not "mlx ...".
|
|
87
|
-
@name == :llama_cpp ? "llama.cpp" : @name.to_s
|
|
88
|
-
end
|
|
89
|
-
|
|
90
|
-
def completion_path
|
|
91
|
-
openai_compatible? ? "/v1/completions" : "/completion"
|
|
92
|
-
end
|
|
93
|
-
|
|
94
|
-
def models_path
|
|
95
|
-
openai_compatible? ? "/v1/models" : "/models"
|
|
96
|
-
end
|
|
97
|
-
|
|
98
|
-
def token_limit_key
|
|
99
|
-
openai_compatible? ? :max_tokens : :n_predict
|
|
100
|
-
end
|
|
101
|
-
|
|
102
|
-
# Where the server describes itself (context window, chat template), or
|
|
103
|
-
# nil when it has no such route. llama.cpp's /props carries the per-slot
|
|
104
|
-
# n_ctx (-c split across --parallel slots) and the chat template.
|
|
105
|
-
# mlx_lm.server and oMLX expose neither.
|
|
106
|
-
def props_path
|
|
107
|
-
openai_compatible? ? nil : "/props"
|
|
108
|
-
end
|
|
109
|
-
|
|
110
|
-
def context_window_from(body)
|
|
111
|
-
n_ctx = body.is_a?(Hash) ? body.dig("default_generation_settings", "n_ctx") : nil
|
|
112
|
-
n_ctx.is_a?(Integer) && n_ctx.positive? ? n_ctx : nil
|
|
113
|
-
end
|
|
114
|
-
|
|
115
|
-
# Text content carried by one streamed `data:` payload.
|
|
116
|
-
def content_from_payload(payload)
|
|
117
|
-
openai_compatible? ? payload.dig("choices", 0, "text").to_s : payload.fetch("content", "")
|
|
118
|
-
end
|
|
71
|
+
# The context window (n_ctx) in a /props body, or nil.
|
|
72
|
+
def self.context_window_from(body)
|
|
73
|
+
n_ctx = body.is_a?(Hash) ? body.dig("default_generation_settings", "n_ctx") : nil
|
|
74
|
+
n_ctx.is_a?(Integer) && n_ctx.positive? ? n_ctx : nil
|
|
75
|
+
end
|
|
119
76
|
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
return nil unless payload.is_a?(Hash)
|
|
127
|
-
return payload.dig("choices", 0, "finish_reason") if openai_compatible?
|
|
128
|
-
|
|
129
|
-
case payload["stop_type"]
|
|
130
|
-
when "limit" then "length"
|
|
131
|
-
when "eos", "word" then "stop"
|
|
132
|
-
end
|
|
133
|
-
end
|
|
77
|
+
# Why the stream stopped, from its last payload, as an OpenAI finish
|
|
78
|
+
# reason ("length" for the token cap or a full context, "stop"
|
|
79
|
+
# otherwise); nil for a payload that doesn't say. llama.cpp's
|
|
80
|
+
# /completion names it stop_type ("limit", "eos", "word").
|
|
81
|
+
def self.finish_reason_from(payload)
|
|
82
|
+
return nil unless payload.is_a?(Hash)
|
|
134
83
|
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
# server's own `--model` CLI flag).
|
|
139
|
-
# - oMLX: MUST send a model id that exists in the server's `/v1/models`
|
|
140
|
-
# list, or oMLX 400s with "model: Field required". The client-installed
|
|
141
|
-
# resolver maps the short selector (e.g. `gemma-4-26b-a4b-it-4bit`) to
|
|
142
|
-
# the exact registered id (which may be prefixed, e.g.
|
|
143
|
-
# `mlx-community--...`) by matching against the loaded /v1/models list.
|
|
144
|
-
def model_for_payload(model)
|
|
145
|
-
return @model_resolver.call(model) if @model_resolver
|
|
146
|
-
|
|
147
|
-
value = model.to_s.strip
|
|
148
|
-
value.empty? ? nil : value
|
|
84
|
+
case payload["stop_type"]
|
|
85
|
+
when "limit" then "length"
|
|
86
|
+
when "eos", "word" then "stop"
|
|
149
87
|
end
|
|
88
|
+
end
|
|
150
89
|
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
def openai_compatible?
|
|
157
|
-
@name == :mlx || @name == :omlx
|
|
158
|
-
end
|
|
90
|
+
# The request's `model` field: the selector verbatim, nil (omit the
|
|
91
|
+
# field) when blank.
|
|
92
|
+
def self.model_for_payload(model)
|
|
93
|
+
value = model.to_s.strip
|
|
94
|
+
value.empty? ? nil : value
|
|
159
95
|
end
|
|
160
96
|
|
|
161
97
|
# One /props probe's outcome. `answered?` is false when the probe failed:
|
|
@@ -191,14 +127,14 @@ module Samagotchi
|
|
|
191
127
|
# @param first_token_timeout [Numeric, nil] seconds a completion may take
|
|
192
128
|
# to stream its first text (LLM::HTTP); nil: no limit
|
|
193
129
|
# @param name [String, nil] the host's config name, for error lines
|
|
194
|
-
# (default:
|
|
130
|
+
# (default: LABEL)
|
|
195
131
|
# @param api_key_env [String, nil] the variable holding the host's API
|
|
196
132
|
# key (llama.cpp's --api-key), sent as a bearer token on every request;
|
|
197
133
|
# nil sends no Authorization header
|
|
198
134
|
# @param auth [Symbol, nil] the header the key goes in (LLM::ApiKey::SCHEMES;
|
|
199
135
|
# HostEntry#auth_scheme), nil: Bearer
|
|
200
136
|
# @param env [Hash] where the key variable is read
|
|
201
|
-
def initialize(host: nil, port: nil, open_timeout: nil, read_timeout: nil,
|
|
137
|
+
def initialize(host: nil, port: nil, open_timeout: nil, read_timeout: nil, sleeper: nil, scheme: nil,
|
|
202
138
|
first_token_timeout: nil, name: nil, api_key_env: nil, auth: nil, env: ENV)
|
|
203
139
|
# Unified config precedence: CLI > ENV > file > default (via Samagotchi::Config)
|
|
204
140
|
@host = host || Samagotchi::Config.get("server.host")
|
|
@@ -206,7 +142,6 @@ module Samagotchi
|
|
|
206
142
|
@scheme = scheme || "http"
|
|
207
143
|
@open_timeout = Samagotchi::Config.positive_seconds("server.open_timeout", open_timeout)
|
|
208
144
|
@read_timeout = Samagotchi::Config.positive_seconds("server.read_timeout", read_timeout)
|
|
209
|
-
@transport = build_transport(resolve_transport(transport))
|
|
210
145
|
# This host's process-wide /props entry (see self.props_entry): its
|
|
211
146
|
# answers and failures are shared with every other Client of the same
|
|
212
147
|
# server.
|
|
@@ -216,7 +151,7 @@ module Samagotchi
|
|
|
216
151
|
@props_mutex = @props_entry[:mutex]
|
|
217
152
|
@first_token_timeout = first_token_timeout
|
|
218
153
|
@host_name = name
|
|
219
|
-
@label = name.to_s.empty? ?
|
|
154
|
+
@label = name.to_s.empty? ? LABEL : name.to_s
|
|
220
155
|
@http = LLM::HTTP.new(label: @label, open_timeout: @open_timeout, read_timeout: @read_timeout,
|
|
221
156
|
sleeper: sleeper, first_token_timeout: first_token_timeout,
|
|
222
157
|
api_key: LLM::ApiKey.for(api_key_env, host: @label, env: env, scheme: auth))
|
|
@@ -228,9 +163,6 @@ module Samagotchi
|
|
|
228
163
|
# The host's config name, or nil when built without one.
|
|
229
164
|
attr_reader :host_name
|
|
230
165
|
|
|
231
|
-
# The wire-format strategy for this client's transport.
|
|
232
|
-
attr_reader :transport
|
|
233
|
-
|
|
234
166
|
# Send a raw prompt and return the model's completion text.
|
|
235
167
|
#
|
|
236
168
|
# llama.cpp can stream completion chunks as newline-delimited `data: {...}`
|
|
@@ -286,23 +218,20 @@ module Samagotchi
|
|
|
286
218
|
|
|
287
219
|
# Prefill +prompt+ and keep it in the server's cache, so the next
|
|
288
220
|
# request that starts with it only prefills what follows (PromptWarmup:
|
|
289
|
-
# the turn-end warm-up).
|
|
290
|
-
# any other transport. One non-streamed request generating one token
|
|
221
|
+
# the turn-end warm-up). One non-streamed request generating one token
|
|
291
222
|
# (n_predict 0 does the same on current builds, 1 on every one): no
|
|
292
223
|
# retries, no first-token limit, no stream events. Errors are raised.
|
|
293
224
|
# @param slot [Integer, nil] the slot to run on (`id_slot`): the one the
|
|
294
225
|
# last request used, which still holds its state
|
|
295
226
|
# @param images [Array<String>] as #complete's
|
|
296
|
-
# @return [Warmup
|
|
227
|
+
# @return [Warmup]
|
|
297
228
|
def warm_up(prompt, model: nil, slot: nil, images: [])
|
|
298
|
-
return nil unless @transport.native?
|
|
299
|
-
|
|
300
229
|
images = Array(images)
|
|
301
230
|
text = LLM::Utf8Scrub.call(images.empty? ? prompt : prompt.gsub(ImagePlan::NATIVE_PLACEHOLDER, media_marker!(model)))
|
|
302
231
|
payload = { prompt: images.empty? ? text : { prompt_string: text, multimodal_data: images },
|
|
303
232
|
n_predict: 1, cache_prompt: true, stream: false }
|
|
304
233
|
payload[:id_slot] = slot if slot.is_a?(Integer)
|
|
305
|
-
model_name =
|
|
234
|
+
model_name = Client.model_for_payload(model)
|
|
306
235
|
payload[:model] = model_name if model_name
|
|
307
236
|
uri = completion_uri
|
|
308
237
|
request = Net::HTTP::Post.new(uri)
|
|
@@ -317,13 +246,13 @@ module Samagotchi
|
|
|
317
246
|
|
|
318
247
|
# Whether llama.cpp's slot +slot+ is processing a request now (GET
|
|
319
248
|
# /slots, its is_processing): :busy or :idle. nil when unknown: no slot,
|
|
320
|
-
#
|
|
249
|
+
# the endpoint off (--no-slots), a slot it doesn't
|
|
321
250
|
# list, a bad body or a network error. One quick attempt (PromptWarmup:
|
|
322
251
|
# a warm-up pinned to a busy slot waits behind that request, and the
|
|
323
252
|
# next turn behind the warm-up).
|
|
324
253
|
# @return [Symbol, nil]
|
|
325
254
|
def slot_status(slot, model: nil)
|
|
326
|
-
return nil unless
|
|
255
|
+
return nil unless slot.is_a?(Integer)
|
|
327
256
|
|
|
328
257
|
model = model.to_s.strip
|
|
329
258
|
query = model.empty? ? "" : "?#{URI.encode_www_form(model: model)}"
|
|
@@ -368,7 +297,7 @@ module Samagotchi
|
|
|
368
297
|
next unless on_chunk
|
|
369
298
|
|
|
370
299
|
chunk = { content: content, payload: payload }
|
|
371
|
-
finish_reason =
|
|
300
|
+
finish_reason = Client.finish_reason_from(payload)
|
|
372
301
|
chunk[:finish_reason] = finish_reason if finish_reason
|
|
373
302
|
on_chunk.call(**chunk)
|
|
374
303
|
end
|
|
@@ -378,26 +307,26 @@ module Samagotchi
|
|
|
378
307
|
clear_props_failures!
|
|
379
308
|
result
|
|
380
309
|
rescue LLM::BadRequest => e
|
|
381
|
-
raise unless e.status == 404
|
|
310
|
+
raise unless e.status == 404
|
|
382
311
|
|
|
383
312
|
# No native /completion here: likely an OpenAI-compatible server.
|
|
384
313
|
raise LLM::BadRequest.new(e.message, host: e.host, status: e.status, attempts: e.attempts, hint: OPENAI_API_HINT)
|
|
385
314
|
rescue RequestCancelled, LLM::ProviderError
|
|
386
315
|
raise
|
|
387
316
|
rescue StandardError => e
|
|
388
|
-
raise "#{
|
|
317
|
+
raise "#{LABEL} request failed (#{@host}:#{@port}): #{e.message}"
|
|
389
318
|
end
|
|
390
319
|
|
|
391
320
|
# The running llama.cpp's media marker (/props), or a VisionUnsupported.
|
|
392
321
|
def media_marker!(model)
|
|
393
|
-
marker =
|
|
322
|
+
marker = VisionSupport.media_marker(server_props(model: model))
|
|
394
323
|
return marker if marker
|
|
395
324
|
|
|
396
325
|
raise LLM::VisionUnsupported.new("#{@label}: can't reach /props for the media marker", host: @label)
|
|
397
326
|
end
|
|
398
327
|
|
|
399
328
|
def list_models
|
|
400
|
-
uri = URI("#{@scheme}://#{@host}:#{@port}#{
|
|
329
|
+
uri = URI("#{@scheme}://#{@host}:#{@port}#{MODELS_PATH}")
|
|
401
330
|
response = @http.fetch(uri, Net::HTTP::Get.new(uri), log_fields: { purpose: "models" })
|
|
402
331
|
clear_props_failures!
|
|
403
332
|
parsed = JSON.parse(response.body.to_s)
|
|
@@ -405,11 +334,11 @@ module Samagotchi
|
|
|
405
334
|
rescue LLM::ProviderError
|
|
406
335
|
raise
|
|
407
336
|
rescue StandardError => e
|
|
408
|
-
raise "#{
|
|
337
|
+
raise "#{LABEL} model listing failed (#{@host}:#{@port}): #{e.message}"
|
|
409
338
|
end
|
|
410
339
|
|
|
411
|
-
# What the running server says about itself (/props), as a ServerProps
|
|
412
|
-
#
|
|
340
|
+
# What the running server says about itself (/props), as a ServerProps.
|
|
341
|
+
# One GET with short
|
|
413
342
|
# timeouts and no retry: it runs before generation and must never hold up
|
|
414
343
|
# a turn. The probe names the model (`?model=`): a llama.cpp router
|
|
415
344
|
# answers a stub without it, and a single-model server ignores it.
|
|
@@ -420,9 +349,6 @@ module Samagotchi
|
|
|
420
349
|
# or model list this client gets through clears the failures: a server
|
|
421
350
|
# that just answered is not the down or hung one the window remembered.
|
|
422
351
|
def server_props(model: nil)
|
|
423
|
-
path = @transport.props_path
|
|
424
|
-
return nil unless path
|
|
425
|
-
|
|
426
352
|
key = model.to_s
|
|
427
353
|
@props_mutex.synchronize do
|
|
428
354
|
return @props_cache[key] if @props_cache.key?(key)
|
|
@@ -431,7 +357,7 @@ module Samagotchi
|
|
|
431
357
|
return failed if failed && monotonic_now - failed_at < PROPS_FAILURE_TTL
|
|
432
358
|
end
|
|
433
359
|
|
|
434
|
-
props = probe_props(
|
|
360
|
+
props = probe_props(PROPS_PATH, key)
|
|
435
361
|
@props_mutex.synchronize do
|
|
436
362
|
case props.status
|
|
437
363
|
when :cancelled then nil
|
|
@@ -451,10 +377,10 @@ module Samagotchi
|
|
|
451
377
|
end
|
|
452
378
|
|
|
453
379
|
# The context window (tokens) the running server was started with, or nil
|
|
454
|
-
# when the
|
|
380
|
+
# when the server reports none or the probe fails (see #server_props).
|
|
455
381
|
def context_window(model: nil)
|
|
456
382
|
props = server_props(model: model)
|
|
457
|
-
props&.answered? ?
|
|
383
|
+
props&.answered? ? Client.context_window_from(props.body) : nil
|
|
458
384
|
rescue StandardError
|
|
459
385
|
nil
|
|
460
386
|
end
|
|
@@ -505,28 +431,8 @@ module Samagotchi
|
|
|
505
431
|
nil
|
|
506
432
|
end
|
|
507
433
|
|
|
508
|
-
def resolve_transport(transport)
|
|
509
|
-
value = (transport || Samagotchi::Config.get("server.transport")).to_s.strip.downcase.to_sym
|
|
510
|
-
VALID_TRANSPORTS.include?(value) ? value : DEFAULT_TRANSPORT
|
|
511
|
-
end
|
|
512
|
-
|
|
513
|
-
# Build the wire-format strategy for a resolved transport name. oMLX gets
|
|
514
|
-
# a resolver that maps the short selector to the exact /v1/models id (see
|
|
515
|
-
# #resolve_omlx_model); mlx gets a resolver that always returns nil so the
|
|
516
|
-
# `model` field is omitted; llama.cpp uses the strategy's default (forward
|
|
517
|
-
# the selector verbatim).
|
|
518
|
-
def build_transport(name)
|
|
519
|
-
resolver = case name
|
|
520
|
-
when :omlx
|
|
521
|
-
->(model) { resolve_omlx_model(model) }
|
|
522
|
-
when :mlx
|
|
523
|
-
->(_model) {}
|
|
524
|
-
end
|
|
525
|
-
Transport.new(name, model_resolver: resolver)
|
|
526
|
-
end
|
|
527
|
-
|
|
528
434
|
def completion_uri
|
|
529
|
-
URI("#{@scheme}://#{@host}:#{@port}#{
|
|
435
|
+
URI("#{@scheme}://#{@host}:#{@port}#{COMPLETION_PATH}")
|
|
530
436
|
end
|
|
531
437
|
|
|
532
438
|
def completion_payload(prompt, stop:, n_predict:, model:, sampling: {}, slot: nil, cache_prompt: true)
|
|
@@ -534,16 +440,14 @@ module Samagotchi
|
|
|
534
440
|
# A non-positive cap is dropped, which leaves the length to the server
|
|
535
441
|
# (unbounded): 0 here never means "only process the prompt". The
|
|
536
442
|
# turn-end warm-up (#warm_up) sends its own payload for that reason.
|
|
537
|
-
payload[
|
|
538
|
-
model_name =
|
|
443
|
+
payload[:n_predict] = n_predict if n_predict && n_predict.to_i.positive?
|
|
444
|
+
model_name = Client.model_for_payload(model)
|
|
539
445
|
payload[:model] = model_name if model_name
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
payload[:id_slot] = slot if slot.is_a?(Integer)
|
|
546
|
-
end
|
|
446
|
+
# llama.cpp's default today, sent anyway: chi's next turn relies on it.
|
|
447
|
+
# false only for the retry of a malformed generation (KernelLoop),
|
|
448
|
+
# which a poisoned cache must not produce again.
|
|
449
|
+
payload[:cache_prompt] = cache_prompt != false
|
|
450
|
+
payload[:id_slot] = slot if slot.is_a?(Integer)
|
|
547
451
|
sendable_sampling(sampling).merge(payload)
|
|
548
452
|
end
|
|
549
453
|
|
|
@@ -553,48 +457,8 @@ module Samagotchi
|
|
|
553
457
|
(sampling || {}).reject { |key, value| value.nil? || ConfigFile::SAMPLING_RESERVED_KEYS.include?(key.to_s) }
|
|
554
458
|
end
|
|
555
459
|
|
|
556
|
-
# Resolve a user-facing SAMAGOTCHI_DEFAULT_MODEL selector to the exact id oMLX
|
|
557
|
-
# expects in the request body (an id from its `/v1/models` list).
|
|
558
|
-
#
|
|
559
|
-
# Resolution order: exact (case-insensitive) match first, then the first
|
|
560
|
-
# substring match, else the selector passes through unchanged so oMLX returns
|
|
561
|
-
# its own 404 listing the available models. An empty selector yields nil (no
|
|
562
|
-
# model field). The id list is loaded once per client and memoized, but
|
|
563
|
-
# resolution runs on every completion so a runtime model switch re-resolves.
|
|
564
|
-
def resolve_omlx_model(raw)
|
|
565
|
-
value = raw.to_s.strip
|
|
566
|
-
return nil if value.empty?
|
|
567
|
-
|
|
568
|
-
ids = fetch_omlx_model_ids
|
|
569
|
-
return value if ids.empty?
|
|
570
|
-
|
|
571
|
-
ids.find { |id| id.casecmp?(value) } ||
|
|
572
|
-
ids.select { |id| id.downcase.include?(value.downcase) }.first ||
|
|
573
|
-
value
|
|
574
|
-
end
|
|
575
|
-
|
|
576
|
-
# Load and memoize the list of model ids from oMLX's `/v1/models`.
|
|
577
|
-
#
|
|
578
|
-
# Only memoize on success: a failed `list_models` must leave the cache unset
|
|
579
|
-
# so the next completion retries (a transient blip shouldn't disable
|
|
580
|
-
# resolution for the whole session). On failure we return [] so an unknown
|
|
581
|
-
# selector still passes through raw, letting oMLX return its own 400/404
|
|
582
|
-
# (server decides), as documented in docs/configuration.md.
|
|
583
|
-
def fetch_omlx_model_ids
|
|
584
|
-
return @omlx_model_ids if defined?(@omlx_model_ids)
|
|
585
|
-
|
|
586
|
-
begin
|
|
587
|
-
ids = list_models.map { |m| m.is_a?(Hash) ? m["id"] : m }
|
|
588
|
-
rescue StandardError
|
|
589
|
-
return []
|
|
590
|
-
end
|
|
591
|
-
|
|
592
|
-
@omlx_model_ids = ids
|
|
593
|
-
ids
|
|
594
|
-
end
|
|
595
|
-
|
|
596
460
|
# Returns [content, payload] for a streamed SSE line, or nil to skip
|
|
597
|
-
# (blank lines, non-data lines, and
|
|
461
|
+
# (blank lines, non-data lines, and a `[DONE]` sentinel).
|
|
598
462
|
# Raises the ProviderError of a server's error event.
|
|
599
463
|
def parse_stream_line(line)
|
|
600
464
|
error = LLM::HTTP.sse_error(line, host: @label)
|
|
@@ -609,8 +473,7 @@ module Samagotchi
|
|
|
609
473
|
rescue JSON::ParserError => e
|
|
610
474
|
raise LLM::ProtocolError.new("#{@label}: malformed stream chunk: #{e.message[0, 200]}", host: @label)
|
|
611
475
|
end
|
|
612
|
-
content
|
|
613
|
-
[content, payload]
|
|
476
|
+
[payload.fetch("content", ""), payload]
|
|
614
477
|
end
|
|
615
478
|
end
|
|
616
479
|
end
|