ruby_llm-providers-lms 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +46 -0
- data/README.md +124 -10
- data/lib/ruby_llm/providers/lms/decisions.rb +35 -0
- data/lib/ruby_llm/providers/lms/models.rb +19 -3
- data/lib/ruby_llm/providers/lms/native_v1.rb +8 -3
- data/lib/ruby_llm/providers/lms/system_one.rb +45 -0
- data/lib/ruby_llm/providers/lms/version.rb +1 -1
- data/lib/ruby_llm/providers/lms.rb +22 -1
- metadata +55 -50
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ddec10e0078375411b8c6edfa0b2a476665402b3ee58bd58cf1038efa8e647bd
|
|
4
|
+
data.tar.gz: 7aacef68d4b0e03034647931f3106d65f4aa611059fb2bce1000d61740280a79
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 6e90e96b77af748c918eaf933311d0990b013bfbeef8aad9d145b4f62f7a374826d8799420051f58a189b10db5ab03105a14ea88d40d2d0d677115b6fea85f5d
|
|
7
|
+
data.tar.gz: 57d08ddeaf9b6b9f23449bd4351648eafaa36c5d57323df138a11a1b6be3d17a5547528d7286252b414e18ea5340dfe76f915a0606b0212e1fb3356f9e27b0a5
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,52 @@ All notable changes to this project are documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
|
|
12
|
+
- Judgments with LM Studio decision models (LM Studio 0.4.26 and later), through
|
|
13
|
+
RubyLLM 2.1's `RubyLLM::Judge` and `RubyLLM.judge`. Two new protocols:
|
|
14
|
+
- `:decisions`: OpenAI's Decisions API at `POST /v1/decisions`, and the default.
|
|
15
|
+
Accepts images through `with:`, sent as inline base64 data URLs. Answers are
|
|
16
|
+
matched to questions by name, since RubyLLM 2.1.1 pairs them by position.
|
|
17
|
+
If the response names no model, the requested model id is used.
|
|
18
|
+
- `:system_one`: the Jev (TypeSafe) System One API at `POST /v1/systemone`.
|
|
19
|
+
Accepts images, sent as inline base64 data URLs in the top-level `images`
|
|
20
|
+
array where LM Studio reads them (the stock protocol refuses images). An
|
|
21
|
+
image judged without text sends an empty `state`, which the endpoint
|
|
22
|
+
requires.
|
|
23
|
+
- Live specs and cassettes for judgments with Kev 0.8B, D1 Omni 600M and Clef
|
|
24
|
+
Flash, over both endpoints, including image input.
|
|
25
|
+
- Examples 16-20, on open-source decision models: the three question types, a
|
|
26
|
+
reusable `RubyLLM::Judge` subclass, a sweep of every decision model over both
|
|
27
|
+
endpoints, image judgments, and a decision model screening an inbox ahead of
|
|
28
|
+
a chat model. `examples/common.rb` gains `DECISION_MODEL`
|
|
29
|
+
(`LMS_DECISION_MODEL`, default `d1-omni-600m`) and `judgment_context`.
|
|
30
|
+
- A documentation site (MkDocs Material, `docs/` and `mkdocs.yml`) published
|
|
31
|
+
to GitHub Pages: getting started, the chat protocols, judgments and decision
|
|
32
|
+
models (with measured results per model and endpoint), models and model
|
|
33
|
+
management, architecture, API reference, examples and development notes.
|
|
34
|
+
- `lms_judgment_protocol` configuration option to choose between the two
|
|
35
|
+
endpoints (`:decisions` by default). Judgments go to a decision endpoint
|
|
36
|
+
whatever `lms_protocol` selects.
|
|
37
|
+
- Decision models (`type: "decision"` in `GET /api/v1/models`) are listed as
|
|
38
|
+
judgment models: output modality and capability `judgment`, image input when
|
|
39
|
+
LM Studio reports a vision adapter, and `decision_type` in the metadata.
|
|
40
|
+
|
|
41
|
+
### Changed
|
|
42
|
+
|
|
43
|
+
- Require `ruby_llm >= 2.1.1`, the first release line with judgments.
|
|
44
|
+
- Specs follow RubyLLM 2.1, which releases a request's body once the response
|
|
45
|
+
arrives (`raw.env.request_body` is now `nil`).
|
|
46
|
+
|
|
47
|
+
## [0.2.1] - 2026-09-18
|
|
48
|
+
|
|
49
|
+
### Changed
|
|
50
|
+
|
|
51
|
+
- Require the released `ruby_llm >= 2.0.0` instead of the `2.0.0.rc1`
|
|
52
|
+
prerelease floor. Tested against RubyLLM 2.0.0.
|
|
53
|
+
|
|
8
54
|
## [0.2.0] - 2026-09-14
|
|
9
55
|
|
|
10
56
|
### Added
|
data/README.md
CHANGED
|
@@ -44,6 +44,10 @@ Messages API, and LM Studio's own **native REST** API — which adds inference
|
|
|
44
44
|
stats, server-stored conversations, and model management. The provider works
|
|
45
45
|
out of the box: no API key, no configuration.
|
|
46
46
|
|
|
47
|
+
**Documentation:** <https://madbomber.github.io/ruby_llm-providers-lms> —
|
|
48
|
+
guides for every protocol, judgments with open-source decision models, model
|
|
49
|
+
management, and the examples.
|
|
50
|
+
|
|
47
51
|
## Installation
|
|
48
52
|
|
|
49
53
|
Add this line to your application's Gemfile:
|
|
@@ -95,6 +99,7 @@ provider.list_models.each { |model| puts model.id }
|
|
|
95
99
|
| [OpenAI](#the-openai-api) | `POST /v1/chat/completions`, `POST /v1/responses`, `GET /v1/models`, `POST /v1/embeddings`, `POST /v1/completions` | `:chat_completions` (default), `:responses` |
|
|
96
100
|
| [Anthropic](#the-anthropic-api) | `POST /v1/messages` | `:anthropic` |
|
|
97
101
|
| [Native REST](#the-native-rest-api) | v0: `POST /api/v0/chat/completions`, `GET /api/v0/models`, `POST /api/v0/embeddings`, … <br> v1: `POST /api/v1/chat`, `GET /api/v1/models`, `POST /api/v1/models/load`, `/unload`, `/download` | `:native_v0`, `:native_chat` |
|
|
102
|
+
| [Decisions](#judgments-with-decision-models) | OpenAI: `POST /v1/decisions` <br> Jev System One: `POST /v1/systemone` | `:decisions` (default for judgments), `:system_one` |
|
|
98
103
|
|
|
99
104
|
Everything but the default is opt-in. Pick a protocol process-wide, for one
|
|
100
105
|
conversation, or for a single chat:
|
|
@@ -121,10 +126,11 @@ Stay on the default unless you need something only another column offers.
|
|
|
121
126
|
| Inference stats | no | no | no | yes, via `raw` | yes, via `tokens` |
|
|
122
127
|
| LM Studio request extras | yes | yes | yes | yes | sampling only — see below |
|
|
123
128
|
| Server-stored conversations | no | no | no | no | yes |
|
|
124
|
-
| Reasoning control | yes | yes |
|
|
125
|
-
| Reasoning text returned | yes | yes |
|
|
129
|
+
| Reasoning control | yes | yes | yes | yes | yes |
|
|
130
|
+
| Reasoning text returned | yes | yes | yes | yes | yes |
|
|
126
131
|
|
|
127
|
-
Checked against LM Studio 0.4.24+1
|
|
132
|
+
Checked against LM Studio 0.4.24+1, the `:anthropic` reasoning rows against
|
|
133
|
+
0.4.26 (see [Tested against](#tested-against)).
|
|
128
134
|
Where a protocol ignores something silently rather than raising, the section
|
|
129
135
|
below says so — those are the cases that cost you an afternoon.
|
|
130
136
|
|
|
@@ -196,10 +202,10 @@ Model 'bible-study-phi3-mini' does not expose reasoning configuration.
|
|
|
196
202
|
```
|
|
197
203
|
|
|
198
204
|
So on `:native_chat`, only ask for reasoning from a model whose listing
|
|
199
|
-
reports it. `with_thinking` and `with_thinking(false)`
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
server and gets the error above.
|
|
205
|
+
reports it. `with_thinking` and `with_thinking(false)` refuse before any
|
|
206
|
+
request is sent (on `ask`) for a model that reports no reasoning options —
|
|
207
|
+
one in the registry without them, or one not in your catalog at all. Only an
|
|
208
|
+
explicit `with_thinking(effort:)` reaches the server and gets the error above.
|
|
203
209
|
|
|
204
210
|
Which efforts you can ask for depends on what you have downloaded:
|
|
205
211
|
|
|
@@ -298,6 +304,10 @@ Plain exchanges, streaming (proper `message_start` / `content_block_delta`
|
|
|
298
304
|
SSE events) and tool calls (`tool_use` blocks with `stop_reason: "tool_use"`,
|
|
299
305
|
executed as ordinary `RubyLLM::Tool` objects in your process) all work; the
|
|
300
306
|
live specs in `spec/ruby_llm/chat_anthropic_spec.rb` cover all three.
|
|
307
|
+
Reasoning works too: `with_thinking(effort: :low)` sends Anthropic's
|
|
308
|
+
`thinking` field, and the reasoning comes back as a `thinking` content block
|
|
309
|
+
in `response.thinking.text` (no thinking token count, since Anthropic's usage
|
|
310
|
+
block has none).
|
|
301
311
|
|
|
302
312
|
**`with_schema` does not work here, and fails quietly.** LM Studio ignores the
|
|
303
313
|
schema and nothing raises. The same model and schema, two protocols:
|
|
@@ -384,7 +394,9 @@ response = chat.ask('Hello')
|
|
|
384
394
|
filtering. Executed tool calls appear in `response.raw.body['output']`.
|
|
385
395
|
- **Per-request context length** — `with_provider_options(context_length: 8192)`.
|
|
386
396
|
- **Opting out of storage** — `with_provider_options(store: false)` keeps the
|
|
387
|
-
conversation off the server
|
|
397
|
+
conversation off the server. The chat is then single-turn: a second `ask`
|
|
398
|
+
raises, because the endpoint cannot replay history and has nothing stored
|
|
399
|
+
to continue from.
|
|
388
400
|
|
|
389
401
|
- **Sampling settings** — `top_p`, `top_k`, `min_p` and `repeat_penalty` are
|
|
390
402
|
part of the native request schema, so `with_provider_options(top_k: 40)` and
|
|
@@ -431,6 +443,100 @@ API — set idle TTL per request through the OpenAI endpoints (see
|
|
|
431
443
|
|
|
432
444
|
---
|
|
433
445
|
|
|
446
|
+
# Judgments with decision models
|
|
447
|
+
|
|
448
|
+
LM Studio 0.4.26 added two endpoints for **decision models**: models that
|
|
449
|
+
answer typed questions with probabilities instead of generating text.
|
|
450
|
+
RubyLLM 2.1 calls these **judgments** (`RubyLLM::Judge`, `RubyLLM.judge`),
|
|
451
|
+
and this gem points them at LM Studio. One call can ask three kinds of
|
|
452
|
+
question:
|
|
453
|
+
|
|
454
|
+
| Question | Answer |
|
|
455
|
+
| --- | --- |
|
|
456
|
+
| `probability` | the chance a condition holds, 0 to 1 |
|
|
457
|
+
| `choice` | one option from a fixed set, a probability for every option, and a confidence |
|
|
458
|
+
| `score` | a position on an ordered scale (it can fall between levels), per-level probabilities, and a confidence |
|
|
459
|
+
|
|
460
|
+
```ruby
|
|
461
|
+
class TicketTriage < RubyLLM::Judge
|
|
462
|
+
model 'clef-flash', provider: :lms
|
|
463
|
+
|
|
464
|
+
probability :urgent, 'Does this need attention today?'
|
|
465
|
+
|
|
466
|
+
choice :department, 'Which team should handle this?' do
|
|
467
|
+
billing 'Payments and refunds'
|
|
468
|
+
technical 'Bugs and integrations'
|
|
469
|
+
other 'Everything else'
|
|
470
|
+
end
|
|
471
|
+
|
|
472
|
+
score :frustration, 'How frustrated is the customer?', %w[Calm Frustrated Angry]
|
|
473
|
+
end
|
|
474
|
+
|
|
475
|
+
judgment = TicketTriage.judge('I was charged twice. Please fix it today.')
|
|
476
|
+
judgment.urgent.probability # => 0.96
|
|
477
|
+
judgment.department.choice # => :billing
|
|
478
|
+
judgment.department.confidence # => 0.87
|
|
479
|
+
judgment.frustration.score # => 1.68
|
|
480
|
+
```
|
|
481
|
+
|
|
482
|
+
Those numbers came from Clef Flash. Smaller models are faster but less sure:
|
|
483
|
+
on the same ticket, D1 Omni 600M also picks `:billing` (confidence 0.78), while
|
|
484
|
+
Kev 0.8B picks `:other`.
|
|
485
|
+
|
|
486
|
+
RubyLLM's [judgments guide](https://rubyllm.com/judgments/) covers the full
|
|
487
|
+
DSL: structured input, dynamic options, and thresholds.
|
|
488
|
+
|
|
489
|
+
**You need a decision model.** A chat model is rejected outright — *"Invalid
|
|
490
|
+
model identifier"*, or *"There are no downloaded decision models"* when you
|
|
491
|
+
have none at all. A decision model has to be one LM Studio
|
|
492
|
+
itself identifies as such. A model sharing a decision model's name is not
|
|
493
|
+
enough: an MLX conversion of Clef Flash loads as an ordinary LLM and is
|
|
494
|
+
rejected by both endpoints. `GET /api/v1/models` reports the ones LM Studio
|
|
495
|
+
recognizes with `type: "decision"`, and `provider.list_models` lists them as
|
|
496
|
+
judgment models (output modality `judgment`, capability `judgment`, plus
|
|
497
|
+
`decision_type` in the metadata).
|
|
498
|
+
|
|
499
|
+
These GGUF decision models are known to work (download with `lms get <URL>`):
|
|
500
|
+
|
|
501
|
+
| Model | Hugging Face | Size | Images |
|
|
502
|
+
| --- | --- | --- | --- |
|
|
503
|
+
| Kev 0.8B | [ggml-org/Kev-0.8B-GGUF](https://huggingface.co/ggml-org/Kev-0.8B-GGUF) | 0.8 GB | no |
|
|
504
|
+
| D1 Omni 600M | [LiquidAI/d1-omni-600M-GGUF](https://huggingface.co/LiquidAI/d1-omni-600M-GGUF) | 0.8 GB | yes |
|
|
505
|
+
| Clef Flash | [ggml-org/Clef-Flash-GGUF](https://huggingface.co/ggml-org/Clef-Flash-GGUF) | 7.4 GB | yes |
|
|
506
|
+
|
|
507
|
+
Other repositories tagged
|
|
508
|
+
[`decision-model`](https://huggingface.co/models?other=decision-model) on
|
|
509
|
+
Hugging Face should work too, as long as LM Studio lists them as decision
|
|
510
|
+
models.
|
|
511
|
+
|
|
512
|
+
## Choosing the endpoint
|
|
513
|
+
|
|
514
|
+
Judgments go to a decision endpoint whatever `lms_protocol` says; a chat
|
|
515
|
+
protocol never handles them. `lms_judgment_protocol` picks which one:
|
|
516
|
+
|
|
517
|
+
| | `:decisions` (default) | `:system_one` |
|
|
518
|
+
| --- | --- | --- |
|
|
519
|
+
| Endpoint | `POST /v1/decisions` (OpenAI's Decisions API) | `POST /v1/systemone` (Jev / TypeSafe System One) |
|
|
520
|
+
| Images (`with:`) | inline base64 data URLs in the input content | inline base64 data URLs in a top-level `images` array |
|
|
521
|
+
|
|
522
|
+
```ruby
|
|
523
|
+
RubyLLM.configure { |config| config.lms_judgment_protocol = :system_one }
|
|
524
|
+
```
|
|
525
|
+
|
|
526
|
+
Both endpoints answer the same questions with the same models, and both take
|
|
527
|
+
images for a model with a vision adapter (`list_models` shows `image` among its
|
|
528
|
+
input modalities):
|
|
529
|
+
|
|
530
|
+
```ruby
|
|
531
|
+
TicketTriage.judge('Customer photo of the delivered box', with: 'box.png')
|
|
532
|
+
```
|
|
533
|
+
|
|
534
|
+
Each answer reports the instance that served it as `judgment.model`. When LM
|
|
535
|
+
Studio loads a model on demand beside a copy you loaded yourself, that is the
|
|
536
|
+
instance's identifier (`clef-flash:2`), not the model id.
|
|
537
|
+
|
|
538
|
+
---
|
|
539
|
+
|
|
434
540
|
# Models on your machine
|
|
435
541
|
|
|
436
542
|
## What listing enrichment adds
|
|
@@ -454,6 +560,7 @@ up whatever the server knows:
|
|
|
454
560
|
| `reasoning_options` | v1 | `[{type: "effort", values: ["none", "low", "medium", "xhigh"], default: "xhigh"}, {type: "toggle"}]` |
|
|
455
561
|
| `loaded_context_length` | v1 | `8192` (the loaded instance's window, which can be smaller than `context_window`) |
|
|
456
562
|
| `remaining_ttl_seconds` | v1 | `1050` — how long before an idle JIT-loaded instance evicts |
|
|
563
|
+
| `decision_type` | v1, decision models only | the kind of decision model, when LM Studio reports one |
|
|
457
564
|
|
|
458
565
|
`Model#capabilities` picks up `function_calling` and `tool_choice` from the
|
|
459
566
|
model's tool-use training, `vision` from its vision flag, and `reasoning` from
|
|
@@ -463,6 +570,10 @@ from your schema, so structured output works even on weights with no tool
|
|
|
463
570
|
training (whether the *values* are any good is a separate question — see the
|
|
464
571
|
gpt-oss caveat in `spec/support/models.rb`).
|
|
465
572
|
|
|
573
|
+
Decision models are the exception: they don't chat, so they get the
|
|
574
|
+
`judgment` capability and output modality and nothing else. They take image
|
|
575
|
+
input only when LM Studio reports a vision adapter for them.
|
|
576
|
+
|
|
466
577
|
## The packaged catalog (models.json)
|
|
467
578
|
|
|
468
579
|
The gem ships a `models.json` at its root. **This file is only a sample**,
|
|
@@ -525,8 +636,9 @@ models are resident right now.
|
|
|
525
636
|
| Component | Version |
|
|
526
637
|
| --- | --- |
|
|
527
638
|
| LM Studio (macOS) | 0.4.24+1 |
|
|
639
|
+
| LM Studio (macOS), judgments and `:anthropic` reasoning | 0.4.26 |
|
|
528
640
|
| `lms` CLI | commit `ff50809` |
|
|
529
|
-
| RubyLLM | 2.0.0
|
|
641
|
+
| RubyLLM | 2.1.1 (cassettes recorded on 2.0.0) |
|
|
530
642
|
|
|
531
643
|
This is the server the committed VCR cassettes were recorded from, and the
|
|
532
644
|
version every endpoint and capability claim above was checked against. LM
|
|
@@ -571,7 +683,9 @@ the ones that need a particular kind of model are commented there:
|
|
|
571
683
|
| `REASONING_OFF_MODELS` | one of those that also reports LM Studio's `off` |
|
|
572
684
|
| `VISION_MODELS` | a model flagged `vision: true` |
|
|
573
685
|
| `EMBEDDING_MODELS` | an embedding model |
|
|
574
|
-
| `NATIVE_CHAT_MODELS`, `
|
|
686
|
+
| `NATIVE_CHAT_MODELS`, `MANAGEMENT_MODEL` | small models, so load/unload cycles stay fast |
|
|
687
|
+
| `NATIVE_REASONING_MODELS` | a model that emits reasoning items on the native endpoint (gpt-oss does; most instruct fine-tunes do not) |
|
|
688
|
+
| `JUDGMENT_MODELS` | decision models LM Studio tags `type: "decision"`; `vision: true` for ones with a vision adapter |
|
|
575
689
|
|
|
576
690
|
The portable contract specs are adapted from
|
|
577
691
|
[RubyLLM's live specs](https://github.com/crmne/ruby_llm/tree/main/spec/ruby_llm).
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module RubyLLM
|
|
4
|
+
module Providers
|
|
5
|
+
class LMS < Provider
|
|
6
|
+
# LM Studio serves OpenAI's Decisions API at POST /v1/decisions for
|
|
7
|
+
# decision models (LM Studio 0.4.26 and later), and RubyLLM's stock
|
|
8
|
+
# Decisions protocol drives it — RubyLLM::Judge and RubyLLM.judge
|
|
9
|
+
# reach it through LMS#judge. Questions go out as an Array and the
|
|
10
|
+
# answers come back as one, each echoing its question's name.
|
|
11
|
+
#
|
|
12
|
+
# RubyLLM 2.1.1 pairs answers with questions by position, which holds
|
|
13
|
+
# only while the server answers in the order it was asked. This reads
|
|
14
|
+
# them by name instead (as RubyLLM's main branch now does), and takes
|
|
15
|
+
# the model id from the request when the response omits it.
|
|
16
|
+
class Decisions < Protocols::OpenAI::Decisions
|
|
17
|
+
def parse_judgment_response(response, questions:)
|
|
18
|
+
body = response.body
|
|
19
|
+
named = body.fetch('answers').to_h { |answer| [answer.fetch('name'), answer] }
|
|
20
|
+
answers = questions.values.to_h do |question|
|
|
21
|
+
[question.name, parse_answer(named.fetch(question.name.to_s), question)]
|
|
22
|
+
end
|
|
23
|
+
RubyLLM::Judgment.new(answers:, model: body['model'] || judged_model_id, raw: response,
|
|
24
|
+
model_info: @model, tokens: parse_tokens(body['usage']))
|
|
25
|
+
rescue KeyError, NoMethodError, TypeError => e
|
|
26
|
+
raise Error.new("LM Studio Decisions returned an invalid judgment: #{e.message}", response:)
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
def judged_model_id
|
|
30
|
+
@model.respond_to?(:id) ? @model.id : @model
|
|
31
|
+
end
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
end
|
|
@@ -57,22 +57,31 @@ module RubyLLM
|
|
|
57
57
|
end
|
|
58
58
|
end
|
|
59
59
|
|
|
60
|
+
# A decision model answers judgments (RubyLLM.judge), not chat, and
|
|
61
|
+
# reads images only when LM Studio reports a vision adapter for it.
|
|
60
62
|
def build_modalities(detail)
|
|
61
63
|
return { input: %w[text], output: %w[embeddings] } if embedding?(detail)
|
|
62
64
|
|
|
63
65
|
input = %w[text]
|
|
64
66
|
input << 'image' if vision?(detail)
|
|
65
|
-
{ input: input, output: %w[text] }
|
|
67
|
+
{ input: input, output: decision?(detail) ? %w[judgment] : %w[text] }
|
|
66
68
|
end
|
|
67
69
|
|
|
68
70
|
def build_capabilities(detail)
|
|
69
71
|
return [] if embedding?(detail)
|
|
72
|
+
return %w[judgment] if decision?(detail)
|
|
70
73
|
|
|
74
|
+
model_capabilities + chat_capabilities(detail)
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
# The capabilities a chat model's own weights bring, beyond what the
|
|
78
|
+
# server provides for every chat model (model_capabilities).
|
|
79
|
+
def chat_capabilities(detail)
|
|
71
80
|
reported = Array(detail['capabilities'])
|
|
72
81
|
derived = CAPABILITY_MAP.filter_map { |native, names| names if reported.include?(native) }.flatten
|
|
73
82
|
derived << 'vision' if vision?(detail) && !derived.include?('vision')
|
|
74
83
|
derived << 'reasoning' if detail['reasoning_options']
|
|
75
|
-
|
|
84
|
+
derived
|
|
76
85
|
end
|
|
77
86
|
|
|
78
87
|
def build_metadata(model, detail)
|
|
@@ -92,7 +101,8 @@ module RubyLLM
|
|
|
92
101
|
selected_variant: detail['selected_variant'],
|
|
93
102
|
reasoning_options: detail['reasoning_options'],
|
|
94
103
|
loaded_context_length: detail['loaded_context_length'],
|
|
95
|
-
remaining_ttl_seconds: detail['remaining_ttl_seconds']
|
|
104
|
+
remaining_ttl_seconds: detail['remaining_ttl_seconds'],
|
|
105
|
+
decision_type: detail['decision_type']
|
|
96
106
|
}.compact
|
|
97
107
|
end
|
|
98
108
|
|
|
@@ -100,7 +110,13 @@ module RubyLLM
|
|
|
100
110
|
detail['type'] == 'embeddings'
|
|
101
111
|
end
|
|
102
112
|
|
|
113
|
+
def decision?(detail)
|
|
114
|
+
detail['type'] == 'decision'
|
|
115
|
+
end
|
|
116
|
+
|
|
103
117
|
def vision?(detail)
|
|
118
|
+
return detail['has_vision_adapter'] == true if decision?(detail)
|
|
119
|
+
|
|
104
120
|
detail['type'] == 'vlm' || Array(detail['capabilities']).include?('vision')
|
|
105
121
|
end
|
|
106
122
|
|
|
@@ -45,14 +45,19 @@ module RubyLLM
|
|
|
45
45
|
'selected_variant' => model['selected_variant'],
|
|
46
46
|
'reasoning_options' => normalize_v1_reasoning(capabilities['reasoning']),
|
|
47
47
|
'loaded_context_length' => instance&.dig('config', 'context_length'),
|
|
48
|
-
'remaining_ttl_seconds' => instance&.dig('remaining_ttl_seconds')
|
|
48
|
+
'remaining_ttl_seconds' => instance&.dig('remaining_ttl_seconds'),
|
|
49
|
+
'decision_type' => model['decision_type'],
|
|
50
|
+
'has_vision_adapter' => model['has_vision_adapter']
|
|
49
51
|
}.compact
|
|
50
52
|
end
|
|
51
53
|
|
|
52
|
-
# v1 reports
|
|
53
|
-
# capability; v0 spells
|
|
54
|
+
# v1 reports 'llm', 'embedding', and 'decision', and states vision as
|
|
55
|
+
# a capability; v0 spells the first two 'embeddings' and 'vlm'. A
|
|
56
|
+
# decision model stays 'decision' even when it reads images — that
|
|
57
|
+
# is what has_vision_adapter says.
|
|
54
58
|
def normalize_v1_type(model, capabilities)
|
|
55
59
|
return 'embeddings' if model['type'] == 'embedding'
|
|
60
|
+
return 'decision' if model['type'] == 'decision'
|
|
56
61
|
return 'vlm' if capabilities['vision']
|
|
57
62
|
|
|
58
63
|
model['type']
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module RubyLLM
|
|
4
|
+
module Providers
|
|
5
|
+
class LMS < Provider
|
|
6
|
+
# LM Studio also serves the Jev (TypeSafe) System One API at
|
|
7
|
+
# POST /v1/systemone for decision models. Questions go out as a Hash
|
|
8
|
+
# keyed by name and the answers come back keyed the same way, so the
|
|
9
|
+
# stock protocol needs only two changes:
|
|
10
|
+
#
|
|
11
|
+
# - The path. It spells it 'v1/systemone' for TypeSafe's bare host,
|
|
12
|
+
# which beside an api_base ending in /v1 would land on
|
|
13
|
+
# /v1/v1/systemone.
|
|
14
|
+
# - Images. The stock protocol refuses them; LM Studio reads them from a
|
|
15
|
+
# top-level `images` array of inline base64 data URLs. (A
|
|
16
|
+
# Responses-style content array in `state` is accepted but its images
|
|
17
|
+
# are ignored -- the answer doesn't change with the picture.) `state`
|
|
18
|
+
# is required even then, so an image judged without text sends an
|
|
19
|
+
# empty one.
|
|
20
|
+
#
|
|
21
|
+
# Select it over Decisions with `config.lms_judgment_protocol =
|
|
22
|
+
# :system_one`.
|
|
23
|
+
class SystemOne < Protocols::SystemOne
|
|
24
|
+
def judgment_url
|
|
25
|
+
'systemone'
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def render_judgment_payload(input, with: [], **)
|
|
29
|
+
payload = super(input, with: [], **)
|
|
30
|
+
return payload if with.empty?
|
|
31
|
+
|
|
32
|
+
payload.merge(state: payload[:state] || '', images: render_images(with))
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def render_images(attachments)
|
|
36
|
+
attachments.map do |attachment|
|
|
37
|
+
raise UnsupportedAttachmentError, attachment.mime_type unless attachment.image?
|
|
38
|
+
|
|
39
|
+
attachment.for_llm
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
end
|
|
@@ -8,6 +8,8 @@ require 'ruby_llm/providers/lms/native_v1'
|
|
|
8
8
|
require 'ruby_llm/providers/lms/models'
|
|
9
9
|
require 'ruby_llm/providers/lms/responses'
|
|
10
10
|
require 'ruby_llm/providers/lms/anthropic_messages'
|
|
11
|
+
require 'ruby_llm/providers/lms/decisions'
|
|
12
|
+
require 'ruby_llm/providers/lms/system_one'
|
|
11
13
|
require 'ruby_llm/providers/lms/model_management'
|
|
12
14
|
require 'ruby_llm/providers/lms/native_chat/conversation'
|
|
13
15
|
require 'ruby_llm/providers/lms/native_chat/streaming'
|
|
@@ -49,6 +51,10 @@ module RubyLLM
|
|
|
49
51
|
protocol :native_chat, NativeChat
|
|
50
52
|
protocol :native_v0, NativeChatCompletions
|
|
51
53
|
protocol :anthropic, AnthropicMessages
|
|
54
|
+
protocol :decisions, Decisions
|
|
55
|
+
protocol :system_one, SystemOne
|
|
56
|
+
|
|
57
|
+
DEFAULT_JUDGMENT_PROTOCOL = :decisions
|
|
52
58
|
|
|
53
59
|
include ModelManagement
|
|
54
60
|
|
|
@@ -80,13 +86,20 @@ module RubyLLM
|
|
|
80
86
|
embedding_protocol(model).render_embedding(text, model: model_id_for(model), dimensions: dimensions)
|
|
81
87
|
end
|
|
82
88
|
|
|
89
|
+
# Which of LM Studio's two decision endpoints RubyLLM.judge talks to:
|
|
90
|
+
# :decisions (POST /v1/decisions) unless lms_judgment_protocol says
|
|
91
|
+
# :system_one (POST /v1/systemone).
|
|
92
|
+
def judgment_protocol
|
|
93
|
+
@config.lms_judgment_protocol || DEFAULT_JUDGMENT_PROTOCOL
|
|
94
|
+
end
|
|
95
|
+
|
|
83
96
|
class << self
|
|
84
97
|
def display_name
|
|
85
98
|
'LM Studio'
|
|
86
99
|
end
|
|
87
100
|
|
|
88
101
|
def configuration_options
|
|
89
|
-
%i[lms_api_base lms_api_key]
|
|
102
|
+
%i[lms_api_base lms_api_key lms_judgment_protocol]
|
|
90
103
|
end
|
|
91
104
|
|
|
92
105
|
def configuration_requirements
|
|
@@ -100,6 +113,14 @@ module RubyLLM
|
|
|
100
113
|
|
|
101
114
|
private
|
|
102
115
|
|
|
116
|
+
# Judgments go to a decision endpoint whichever chat dialect is in
|
|
117
|
+
# effect: lms_protocol would otherwise route RubyLLM.judge to, say,
|
|
118
|
+
# the Anthropic protocol, which has no judgments at all.
|
|
119
|
+
def resolve_protocol(name, model, operation: nil, **)
|
|
120
|
+
name ||= judgment_protocol if operation == :judge
|
|
121
|
+
super
|
|
122
|
+
end
|
|
123
|
+
|
|
103
124
|
def embedding_protocol(model)
|
|
104
125
|
protocols.fetch(:chat_completions).new(self, model)
|
|
105
126
|
end
|
metadata
CHANGED
|
@@ -1,76 +1,81 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ruby_llm-providers-lms
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
|
-
- madbomber
|
|
7
|
+
- madbomber
|
|
8
8
|
bindir: bin
|
|
9
9
|
cert_chain: []
|
|
10
10
|
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
11
|
dependencies:
|
|
12
|
-
- !ruby/object:Gem::Dependency
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: ruby_llm
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
-
|
|
17
|
+
- ">="
|
|
18
|
+
- !ruby/object:Gem::Version
|
|
19
|
+
version: 2.1.1
|
|
20
|
+
type: :runtime
|
|
21
|
+
prerelease: false
|
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
23
|
+
requirements:
|
|
24
|
+
-
|
|
25
|
+
- ">="
|
|
26
|
+
- !ruby/object:Gem::Version
|
|
27
|
+
version: 2.1.1
|
|
28
|
+
description: "Adds LM Studio provider support to RubyLLM via LM Studio's OpenAI-compatible local server, with model listings enriched from LM Studio's native REST API."
|
|
28
29
|
email:
|
|
29
|
-
- dvanhoozer@gmail.com
|
|
30
|
+
- dvanhoozer@gmail.com
|
|
30
31
|
executables: []
|
|
31
32
|
extensions: []
|
|
32
33
|
extra_rdoc_files: []
|
|
33
34
|
files:
|
|
34
|
-
- CHANGELOG.md
|
|
35
|
-
- LICENSE
|
|
36
|
-
- README.md
|
|
37
|
-
- lib/ruby_llm/providers/lms.rb
|
|
38
|
-
- lib/ruby_llm/providers/lms/anthropic_messages.rb
|
|
39
|
-
- lib/ruby_llm/providers/lms/connection_guard.rb
|
|
40
|
-
- lib/ruby_llm/providers/lms/
|
|
41
|
-
- lib/ruby_llm/providers/lms/
|
|
42
|
-
- lib/ruby_llm/providers/lms/
|
|
43
|
-
- lib/ruby_llm/providers/lms/
|
|
44
|
-
- lib/ruby_llm/providers/lms/native_chat
|
|
45
|
-
- lib/ruby_llm/providers/lms/native_chat/
|
|
46
|
-
- lib/ruby_llm/providers/lms/
|
|
47
|
-
- lib/ruby_llm/providers/lms/
|
|
48
|
-
- lib/ruby_llm/providers/lms/
|
|
49
|
-
-
|
|
50
|
-
|
|
35
|
+
- CHANGELOG.md
|
|
36
|
+
- LICENSE
|
|
37
|
+
- README.md
|
|
38
|
+
- lib/ruby_llm/providers/lms.rb
|
|
39
|
+
- lib/ruby_llm/providers/lms/anthropic_messages.rb
|
|
40
|
+
- lib/ruby_llm/providers/lms/connection_guard.rb
|
|
41
|
+
- lib/ruby_llm/providers/lms/decisions.rb
|
|
42
|
+
- lib/ruby_llm/providers/lms/endpoints.rb
|
|
43
|
+
- lib/ruby_llm/providers/lms/model_management.rb
|
|
44
|
+
- lib/ruby_llm/providers/lms/models.rb
|
|
45
|
+
- lib/ruby_llm/providers/lms/native_chat.rb
|
|
46
|
+
- lib/ruby_llm/providers/lms/native_chat/conversation.rb
|
|
47
|
+
- lib/ruby_llm/providers/lms/native_chat/streaming.rb
|
|
48
|
+
- lib/ruby_llm/providers/lms/native_v1.rb
|
|
49
|
+
- lib/ruby_llm/providers/lms/responses.rb
|
|
50
|
+
- lib/ruby_llm/providers/lms/system_one.rb
|
|
51
|
+
- lib/ruby_llm/providers/lms/version.rb
|
|
52
|
+
- models.json
|
|
53
|
+
homepage: "https://github.com/madbomber/ruby_llm-providers-lms"
|
|
51
54
|
licenses:
|
|
52
|
-
- MIT
|
|
55
|
+
- MIT
|
|
53
56
|
metadata:
|
|
54
|
-
homepage_uri: https://github.com/madbomber/ruby_llm-providers-lms
|
|
55
|
-
source_code_uri: https://github.com/madbomber/ruby_llm-providers-lms
|
|
56
|
-
changelog_uri: https://github.com/madbomber/ruby_llm-providers-lms/blob/main/CHANGELOG.md
|
|
57
|
-
bug_tracker_uri: https://github.com/madbomber/ruby_llm-providers-lms/issues
|
|
58
|
-
rubygems_mfa_required:
|
|
57
|
+
homepage_uri: "https://github.com/madbomber/ruby_llm-providers-lms"
|
|
58
|
+
source_code_uri: "https://github.com/madbomber/ruby_llm-providers-lms"
|
|
59
|
+
changelog_uri: "https://github.com/madbomber/ruby_llm-providers-lms/blob/main/CHANGELOG.md"
|
|
60
|
+
bug_tracker_uri: "https://github.com/madbomber/ruby_llm-providers-lms/issues"
|
|
61
|
+
rubygems_mfa_required: "true"
|
|
59
62
|
rdoc_options: []
|
|
60
63
|
require_paths:
|
|
61
|
-
- lib
|
|
64
|
+
- lib
|
|
62
65
|
required_ruby_version: !ruby/object:Gem::Requirement
|
|
63
66
|
requirements:
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
+
-
|
|
68
|
+
- ">="
|
|
69
|
+
- !ruby/object:Gem::Version
|
|
70
|
+
version: "3.3"
|
|
67
71
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
68
72
|
requirements:
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
73
|
+
-
|
|
74
|
+
- ">="
|
|
75
|
+
- !ruby/object:Gem::Version
|
|
76
|
+
version: "0"
|
|
72
77
|
requirements: []
|
|
73
|
-
rubygems_version: 4.0.
|
|
78
|
+
rubygems_version: 4.1.0.beta2
|
|
74
79
|
specification_version: 4
|
|
75
80
|
summary: RubyLLM provider for LM Studio.
|
|
76
81
|
test_files: []
|