ollama-ai 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.gitignore +2 -0
- data/.rubocop.yml +6 -0
- data/.ruby-version +1 -0
- data/Gemfile +10 -0
- data/Gemfile.lock +62 -0
- data/LICENSE +9 -0
- data/README.md +852 -0
- data/components/errors.rb +24 -0
- data/controllers/client.rb +144 -0
- data/ollama-ai.gemspec +35 -0
- data/ports/dsl/ollama-ai/errors.rb +5 -0
- data/ports/dsl/ollama-ai.rb +14 -0
- data/static/gem.rb +15 -0
- data/tasks/generate-readme.clj +39 -0
- data/template.md +817 -0
- metadata +76 -0
data/README.md
ADDED
@@ -0,0 +1,852 @@
|
|
1
|
+
# Ollama AI
|
2
|
+
|
3
|
+
A Ruby gem for interacting with [Ollama](https://github.com/jmorganca/ollama)'s API that allows you to run open source AI LLMs (Large Language Models) locally.
|
4
|
+
|
5
|
+
![The image presents a llama's head merged with a red ruby gemstone against a light beige background. The red facets form both the ruby and the contours of the llama, creating a clever visual fusion.](https://raw.githubusercontent.com/gbaptista/assets/main/ollama-ai/ollama-ai-canvas.png)
|
6
|
+
|
7
|
+
> _This Gem is designed to provide low-level access to Ollama, enabling people to build abstractions on top of it. If you are interested in more high-level abstractions or more user-friendly tools, you may want to consider [Nano Bots](https://github.com/icebaker/ruby-nano-bots) 💎 🤖._
|
8
|
+
|
9
|
+
## TL;DR and Quick Start
|
10
|
+
|
11
|
+
```ruby
|
12
|
+
gem 'ollama-ai', '~> 1.0.0'
|
13
|
+
```
|
14
|
+
|
15
|
+
```ruby
|
16
|
+
require 'ollama-ai'
|
17
|
+
|
18
|
+
client = Ollama.new(
|
19
|
+
credentials: { address: 'http://localhost:11434' },
|
20
|
+
options: { server_sent_events: true }
|
21
|
+
)
|
22
|
+
|
23
|
+
result = client.generate(
|
24
|
+
{ model: 'dolphin-phi',
|
25
|
+
prompt: 'Hi!' }
|
26
|
+
)
|
27
|
+
```
|
28
|
+
|
29
|
+
Result:
|
30
|
+
```ruby
|
31
|
+
[{ 'model' => 'dolphin-phi',
|
32
|
+
'created_at' => '2024-01-06T16:53:21.357816652Z',
|
33
|
+
'response' => 'Hello',
|
34
|
+
'done' => false },
|
35
|
+
{ 'model' => 'dolphin-phi',
|
36
|
+
'created_at' => '2024-01-06T16:53:21.490053654Z',
|
37
|
+
'response' => '!',
|
38
|
+
'done' => false },
|
39
|
+
# ...
|
40
|
+
{ 'model' => 'dolphin-phi',
|
41
|
+
'created_at' => '2024-01-06T16:53:24.82505599Z',
|
42
|
+
'response' => '.',
|
43
|
+
'done' => false },
|
44
|
+
{ 'model' => 'dolphin-phi',
|
45
|
+
'created_at' => '2024-01-06T16:53:24.956774721Z',
|
46
|
+
'response' => '',
|
47
|
+
'done' => true,
|
48
|
+
'context' =>
|
49
|
+
[50_296, 10_057,
|
50
|
+
# ...
|
51
|
+
1037, 13],
|
52
|
+
'total_duration' => 5_702_027_026,
|
53
|
+
'load_duration' => 649_711,
|
54
|
+
'prompt_eval_count' => 25,
|
55
|
+
'prompt_eval_duration' => 2_227_159_000,
|
56
|
+
'eval_count' => 39,
|
57
|
+
'eval_duration' => 3_466_593_000 }]
|
58
|
+
```
|
59
|
+
|
60
|
+
## Index
|
61
|
+
|
62
|
+
- [TL;DR and Quick Start](#tldr-and-quick-start)
|
63
|
+
- [Index](#index)
|
64
|
+
- [Setup](#setup)
|
65
|
+
- [Installing](#installing)
|
66
|
+
- [Usage](#usage)
|
67
|
+
- [Client](#client)
|
68
|
+
- [Methods](#methods)
|
69
|
+
- [generate: Generate a completion](#generate-generate-a-completion)
|
70
|
+
- [Without Streaming Events](#without-streaming-events)
|
71
|
+
- [Receiving Stream Events](#receiving-stream-events)
|
72
|
+
- [chat: Generate a chat completion](#chat-generate-a-chat-completion)
|
73
|
+
- [Back-and-Forth Conversations](#back-and-forth-conversations)
|
74
|
+
- [embeddings: Generate Embeddings](#embeddings-generate-embeddings)
|
75
|
+
- [Models](#models)
|
76
|
+
- [create: Create a Model](#create-create-a-model)
|
77
|
+
- [tags: List Local Models](#tags-list-local-models)
|
78
|
+
- [show: Show Model Information](#show-show-model-information)
|
79
|
+
- [copy: Copy a Model](#copy-copy-a-model)
|
80
|
+
- [delete: Delete a Model](#delete-delete-a-model)
|
81
|
+
- [pull: Pull a Model](#pull-pull-a-model)
|
82
|
+
- [push: Push a Model](#push-push-a-model)
|
83
|
+
- [Streaming and Server-Sent Events (SSE)](#streaming-and-server-sent-events-sse)
|
84
|
+
- [Server-Sent Events (SSE) Hang](#server-sent-events-sse-hang)
|
85
|
+
- [New Functionalities and APIs](#new-functionalities-and-apis)
|
86
|
+
- [Request Options](#request-options)
|
87
|
+
- [Timeout](#timeout)
|
88
|
+
- [Error Handling](#error-handling)
|
89
|
+
- [Rescuing](#rescuing)
|
90
|
+
- [For Short](#for-short)
|
91
|
+
- [Errors](#errors)
|
92
|
+
- [Development](#development)
|
93
|
+
- [Purpose](#purpose)
|
94
|
+
- [Publish to RubyGems](#publish-to-rubygems)
|
95
|
+
- [Updating the README](#updating-the-readme)
|
96
|
+
- [Resources and References](#resources-and-references)
|
97
|
+
- [Disclaimer](#disclaimer)
|
98
|
+
|
99
|
+
## Setup
|
100
|
+
|
101
|
+
### Installing
|
102
|
+
|
103
|
+
```sh
|
104
|
+
gem install ollama-ai -v 1.0.0
|
105
|
+
```
|
106
|
+
|
107
|
+
```sh
|
108
|
+
gem 'ollama-ai', '~> 1.0.0'
|
109
|
+
```
|
110
|
+
|
111
|
+
## Usage
|
112
|
+
|
113
|
+
### Client
|
114
|
+
|
115
|
+
Create a new client:
|
116
|
+
```ruby
|
117
|
+
require 'ollama-ai'
|
118
|
+
|
119
|
+
client = Ollama.new(
|
120
|
+
credentials: { address: 'http://localhost:11434' },
|
121
|
+
options: { server_sent_events: true }
|
122
|
+
)
|
123
|
+
```
|
124
|
+
|
125
|
+
### Methods
|
126
|
+
|
127
|
+
```ruby
|
128
|
+
client.generate
|
129
|
+
client.chat
|
130
|
+
client.embeddings
|
131
|
+
|
132
|
+
client.create
|
133
|
+
client.tags
|
134
|
+
client.show
|
135
|
+
client.copy
|
136
|
+
client.delete
|
137
|
+
client.pull
|
138
|
+
client.push
|
139
|
+
```
|
140
|
+
|
141
|
+
#### generate: Generate a completion
|
142
|
+
|
143
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#generate-a-completion
|
144
|
+
|
145
|
+
##### Without Streaming Events
|
146
|
+
|
147
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#generate-a-completion
|
148
|
+
|
149
|
+
```ruby
|
150
|
+
result = client.generate(
|
151
|
+
{ model: 'dolphin-phi',
|
152
|
+
prompt: 'Hi!',
|
153
|
+
stream: false }
|
154
|
+
)
|
155
|
+
```
|
156
|
+
|
157
|
+
Result:
|
158
|
+
```ruby
|
159
|
+
[{ 'model' => 'dolphin-phi',
|
160
|
+
'created_at' => '2024-01-06T17:47:26.443128626Z',
|
161
|
+
'response' =>
|
162
|
+
"Hello! How can I assist you today? Do you have any questions or problems that you'd like help with?",
|
163
|
+
'done' => true,
|
164
|
+
'context' =>
|
165
|
+
[50_296, 10_057,
|
166
|
+
# ...
|
167
|
+
351, 30],
|
168
|
+
'total_duration' => 6_495_278_960,
|
169
|
+
'load_duration' => 1_434_052_851,
|
170
|
+
'prompt_eval_count' => 25,
|
171
|
+
'prompt_eval_duration' => 1_938_861_000,
|
172
|
+
'eval_count' => 23,
|
173
|
+
'eval_duration' => 3_119_030_000 }]
|
174
|
+
```
|
175
|
+
|
176
|
+
##### Receiving Stream Events
|
177
|
+
|
178
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#generate-a-completion
|
179
|
+
|
180
|
+
Ensure that you have enabled [Server-Sent Events](#streaming-and-server-sent-events-sse) before using blocks for streaming. `stream: true` is not necessary, as `true` is the [default](https://github.com/jmorganca/ollama/blob/main/docs/api.md#generate-a-completion):
|
181
|
+
|
182
|
+
```ruby
|
183
|
+
client.generate(
|
184
|
+
{ model: 'dolphin-phi',
|
185
|
+
prompt: 'Hi!' }
|
186
|
+
) do |event, raw|
|
187
|
+
puts event
|
188
|
+
end
|
189
|
+
```
|
190
|
+
|
191
|
+
Event:
|
192
|
+
```ruby
|
193
|
+
{ 'model' => 'dolphin-phi',
|
194
|
+
'created_at' => '2024-01-06T17:27:29.366879586Z',
|
195
|
+
'response' => 'Hello',
|
196
|
+
'done' => false }
|
197
|
+
```
|
198
|
+
|
199
|
+
You can get all the receive events at once as an array:
|
200
|
+
```ruby
|
201
|
+
result = client.generate(
|
202
|
+
{ model: 'dolphin-phi',
|
203
|
+
prompt: 'Hi!' }
|
204
|
+
)
|
205
|
+
```
|
206
|
+
|
207
|
+
Result:
|
208
|
+
```ruby
|
209
|
+
[{ 'model' => 'dolphin-phi',
|
210
|
+
'created_at' => '2024-01-06T16:53:21.357816652Z',
|
211
|
+
'response' => 'Hello',
|
212
|
+
'done' => false },
|
213
|
+
{ 'model' => 'dolphin-phi',
|
214
|
+
'created_at' => '2024-01-06T16:53:21.490053654Z',
|
215
|
+
'response' => '!',
|
216
|
+
'done' => false },
|
217
|
+
# ...
|
218
|
+
{ 'model' => 'dolphin-phi',
|
219
|
+
'created_at' => '2024-01-06T16:53:24.82505599Z',
|
220
|
+
'response' => '.',
|
221
|
+
'done' => false },
|
222
|
+
{ 'model' => 'dolphin-phi',
|
223
|
+
'created_at' => '2024-01-06T16:53:24.956774721Z',
|
224
|
+
'response' => '',
|
225
|
+
'done' => true,
|
226
|
+
'context' =>
|
227
|
+
[50_296, 10_057,
|
228
|
+
# ...
|
229
|
+
1037, 13],
|
230
|
+
'total_duration' => 5_702_027_026,
|
231
|
+
'load_duration' => 649_711,
|
232
|
+
'prompt_eval_count' => 25,
|
233
|
+
'prompt_eval_duration' => 2_227_159_000,
|
234
|
+
'eval_count' => 39,
|
235
|
+
'eval_duration' => 3_466_593_000 }]
|
236
|
+
```
|
237
|
+
|
238
|
+
You can mix both as well:
|
239
|
+
```ruby
|
240
|
+
result = client.generate(
|
241
|
+
{ model: 'dolphin-phi',
|
242
|
+
prompt: 'Hi!' }
|
243
|
+
) do |event, raw|
|
244
|
+
puts event
|
245
|
+
end
|
246
|
+
```
|
247
|
+
|
248
|
+
#### chat: Generate a chat completion
|
249
|
+
|
250
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#generate-a-chat-completion
|
251
|
+
|
252
|
+
```ruby
|
253
|
+
result = client.chat(
|
254
|
+
{ model: 'dolphin-phi',
|
255
|
+
messages: [
|
256
|
+
{ role: 'user', content: 'Hi! My name is Purple.' }
|
257
|
+
] }
|
258
|
+
) do |event, raw|
|
259
|
+
puts event
|
260
|
+
end
|
261
|
+
```
|
262
|
+
|
263
|
+
Event:
|
264
|
+
```ruby
|
265
|
+
{ 'model' => 'dolphin-phi',
|
266
|
+
'created_at' => '2024-01-06T18:17:22.468231988Z',
|
267
|
+
'message' => { 'role' => 'assistant', 'content' => 'Hello' },
|
268
|
+
'done' => false }
|
269
|
+
```
|
270
|
+
|
271
|
+
Result:
|
272
|
+
```ruby
|
273
|
+
[{ 'model' => 'dolphin-phi',
|
274
|
+
'created_at' => '2024-01-06T18:17:22.468231988Z',
|
275
|
+
'message' => { 'role' => 'assistant', 'content' => 'Hello' },
|
276
|
+
'done' => false },
|
277
|
+
{ 'model' => 'dolphin-phi',
|
278
|
+
'created_at' => '2024-01-06T18:17:22.594414415Z',
|
279
|
+
'message' => { 'role' => 'assistant', 'content' => ' Purple' },
|
280
|
+
'done' => false },
|
281
|
+
# ...
|
282
|
+
{ 'model' => 'dolphin-phi',
|
283
|
+
'created_at' => '2024-01-06T18:17:25.491597233Z',
|
284
|
+
'message' => { 'role' => 'assistant', 'content' => '?' },
|
285
|
+
'done' => false },
|
286
|
+
{ 'model' => 'dolphin-phi',
|
287
|
+
'created_at' => '2024-01-06T18:17:25.578463723Z',
|
288
|
+
'message' => { 'role' => 'assistant', 'content' => '' },
|
289
|
+
'done' => true,
|
290
|
+
'total_duration' => 5_274_177_696,
|
291
|
+
'load_duration' => 1_565_325,
|
292
|
+
'prompt_eval_count' => 30,
|
293
|
+
'prompt_eval_duration' => 2_284_638_000,
|
294
|
+
'eval_count' => 29,
|
295
|
+
'eval_duration' => 2_983_962_000 }]
|
296
|
+
```
|
297
|
+
|
298
|
+
##### Back-and-Forth Conversations
|
299
|
+
|
300
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#generate-a-chat-completion
|
301
|
+
|
302
|
+
To maintain a back-and-forth conversation, you need to append the received responses and build a history for your requests:
|
303
|
+
|
304
|
+
```ruby
|
305
|
+
result = client.chat(
|
306
|
+
{ model: 'dolphin-phi',
|
307
|
+
messages: [
|
308
|
+
{ role: 'user', content: 'Hi! My name is Purple.' },
|
309
|
+
{ role: 'assistant',
|
310
|
+
content: "Hi, Purple! It's nice to meet you. I am Dolphin. How can I help you today?" },
|
311
|
+
{ role: 'user', content: "What's my name?" }
|
312
|
+
] }
|
313
|
+
) do |event, raw|
|
314
|
+
puts event
|
315
|
+
end
|
316
|
+
```
|
317
|
+
|
318
|
+
Event:
|
319
|
+
|
320
|
+
```ruby
|
321
|
+
{ 'model' => 'dolphin-phi',
|
322
|
+
'created_at' => '2024-01-06T19:07:51.05465997Z',
|
323
|
+
'message' => { 'role' => 'assistant', 'content' => 'Your' },
|
324
|
+
'done' => false }
|
325
|
+
```
|
326
|
+
|
327
|
+
Result:
|
328
|
+
```ruby
|
329
|
+
[{ 'model' => 'dolphin-phi',
|
330
|
+
'created_at' => '2024-01-06T19:07:51.05465997Z',
|
331
|
+
'message' => { 'role' => 'assistant', 'content' => 'Your' },
|
332
|
+
'done' => false },
|
333
|
+
{ 'model' => 'dolphin-phi',
|
334
|
+
'created_at' => '2024-01-06T19:07:51.184476541Z',
|
335
|
+
'message' => { 'role' => 'assistant', 'content' => ' name' },
|
336
|
+
'done' => false },
|
337
|
+
# ...
|
338
|
+
{ 'model' => 'dolphin-phi',
|
339
|
+
'created_at' => '2024-01-06T19:07:56.526297223Z',
|
340
|
+
'message' => { 'role' => 'assistant', 'content' => '.' },
|
341
|
+
'done' => false },
|
342
|
+
{ 'model' => 'dolphin-phi',
|
343
|
+
'created_at' => '2024-01-06T19:07:56.667809424Z',
|
344
|
+
'message' => { 'role' => 'assistant', 'content' => '' },
|
345
|
+
'done' => true,
|
346
|
+
'total_duration' => 12_169_557_266,
|
347
|
+
'load_duration' => 4_486_689,
|
348
|
+
'prompt_eval_count' => 95,
|
349
|
+
'prompt_eval_duration' => 6_678_566_000,
|
350
|
+
'eval_count' => 40,
|
351
|
+
'eval_duration' => 5_483_133_000 }]
|
352
|
+
```
|
353
|
+
|
354
|
+
#### embeddings: Generate Embeddings
|
355
|
+
|
356
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#generate-embeddings
|
357
|
+
|
358
|
+
```ruby
|
359
|
+
result = client.embeddings(
|
360
|
+
{ model: 'dolphin-phi',
|
361
|
+
prompt: 'Hi!' }
|
362
|
+
)
|
363
|
+
```
|
364
|
+
|
365
|
+
Result:
|
366
|
+
```ruby
|
367
|
+
[{ 'embedding' =>
|
368
|
+
[1.0372048616409302,
|
369
|
+
1.0635842084884644,
|
370
|
+
# ...
|
371
|
+
-0.5416496396064758,
|
372
|
+
0.051569778472185135] }]
|
373
|
+
```
|
374
|
+
|
375
|
+
#### Models
|
376
|
+
|
377
|
+
##### create: Create a Model
|
378
|
+
|
379
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#create-a-model
|
380
|
+
|
381
|
+
```ruby
|
382
|
+
result = client.create(
|
383
|
+
{ name: 'mario',
|
384
|
+
modelfile: "FROM dolphin-phi\nSYSTEM You are mario from Super Mario Bros." }
|
385
|
+
) do |event, raw|
|
386
|
+
puts event
|
387
|
+
end
|
388
|
+
```
|
389
|
+
|
390
|
+
Event:
|
391
|
+
```ruby
|
392
|
+
{ 'status' => 'reading model metadata' }
|
393
|
+
```
|
394
|
+
|
395
|
+
Result:
|
396
|
+
```ruby
|
397
|
+
[{ 'status' => 'reading model metadata' },
|
398
|
+
{ 'status' => 'creating system layer' },
|
399
|
+
{ 'status' =>
|
400
|
+
'using already created layer sha256:4eca7304a07a42c48887f159ef5ad82ed5a5bd30fe52db4aadae1dd938e26f70' },
|
401
|
+
{ 'status' =>
|
402
|
+
'using already created layer sha256:876a8d805b60882d53fed3ded3123aede6a996bdde4a253de422cacd236e33d3' },
|
403
|
+
{ 'status' =>
|
404
|
+
'using already created layer sha256:a47b02e00552cd7022ea700b1abf8c572bb26c9bc8c1a37e01b566f2344df5dc' },
|
405
|
+
{ 'status' =>
|
406
|
+
'using already created layer sha256:f02dd72bb2423204352eabc5637b44d79d17f109fdb510a7c51455892aa2d216' },
|
407
|
+
{ 'status' =>
|
408
|
+
'writing layer sha256:1741cf59ce26ff01ac614d31efc700e21e44dd96aed60a7c91ab3f47e440ef94' },
|
409
|
+
{ 'status' =>
|
410
|
+
'writing layer sha256:e8bcbb2eebad88c2fa64bc32939162c064be96e70ff36aff566718fc9186b427' },
|
411
|
+
{ 'status' => 'writing manifest' },
|
412
|
+
{ 'status' => 'success' }]
|
413
|
+
```
|
414
|
+
|
415
|
+
After creation, you can use it:
|
416
|
+
```ruby
|
417
|
+
client.generate(
|
418
|
+
{ model: 'mario',
|
419
|
+
prompt: 'Hi! Who are you?' }
|
420
|
+
) do |event, raw|
|
421
|
+
print event['response']
|
422
|
+
end
|
423
|
+
```
|
424
|
+
|
425
|
+
> _Hello! I'm Mario, a character from the popular video game series Super Mario Bros. My goal is to rescue Princess Peach from the evil Bowser and his minions, so we can live happily ever after in the Mushroom Kingdom! 🍄🐥_
|
426
|
+
>
|
427
|
+
> _What brings you here? How can I help you on your journey?_
|
428
|
+
|
429
|
+
##### tags: List Local Models
|
430
|
+
|
431
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#list-local-models
|
432
|
+
|
433
|
+
```ruby
|
434
|
+
result = client.tags
|
435
|
+
```
|
436
|
+
|
437
|
+
Result:
|
438
|
+
```ruby
|
439
|
+
[{ 'models' =>
|
440
|
+
[{ 'name' => 'dolphin-phi:latest',
|
441
|
+
'modified_at' => '2024-01-06T12:20:42.778120982-03:00',
|
442
|
+
'size' => 1_602_473_850,
|
443
|
+
'digest' =>
|
444
|
+
'c5761fc772409945787240af89a5cce01dd39dc52f1b7b80d080a1163e8dbe10',
|
445
|
+
'details' =>
|
446
|
+
{ 'format' => 'gguf',
|
447
|
+
'family' => 'phi2',
|
448
|
+
'families' => ['phi2'],
|
449
|
+
'parameter_size' => '3B',
|
450
|
+
'quantization_level' => 'Q4_0' } },
|
451
|
+
{ 'name' => 'mario:latest',
|
452
|
+
'modified_at' => '2024-01-06T16:19:11.340234644-03:00',
|
453
|
+
'size' => 1_602_473_846,
|
454
|
+
'digest' =>
|
455
|
+
'582e668feaba3fcb6add3cee26046a1d6a0c940b86a692ea30d5100aec90135f',
|
456
|
+
'details' =>
|
457
|
+
{ 'format' => 'gguf',
|
458
|
+
'family' => 'phi2',
|
459
|
+
'families' => ['phi2'],
|
460
|
+
'parameter_size' => '3B',
|
461
|
+
'quantization_level' => 'Q4_0' } }] }]
|
462
|
+
```
|
463
|
+
|
464
|
+
##### show: Show Model Information
|
465
|
+
|
466
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#show-model-information
|
467
|
+
|
468
|
+
```ruby
|
469
|
+
result = client.show(
|
470
|
+
{ name: 'dolphin-phi' }
|
471
|
+
)
|
472
|
+
```
|
473
|
+
|
474
|
+
Result:
|
475
|
+
```ruby
|
476
|
+
[{ 'license' =>
|
477
|
+
"MICROSOFT RESEARCH LICENSE TERMS\n" \
|
478
|
+
# ...
|
479
|
+
'It also applies even if Microsoft knew or should have known about the possibility...',
|
480
|
+
'modelfile' =>
|
481
|
+
"# Modelfile generated by \"ollama show\"\n" \
|
482
|
+
# ...
|
483
|
+
'PARAMETER stop "<|im_end|>"',
|
484
|
+
'parameters' =>
|
485
|
+
"stop <|im_start|>\n" \
|
486
|
+
'stop <|im_end|>',
|
487
|
+
'template' =>
|
488
|
+
"<|im_start|>system\n" \
|
489
|
+
"{{ .System }}<|im_end|>\n" \
|
490
|
+
"<|im_start|>user\n" \
|
491
|
+
"{{ .Prompt }}<|im_end|>\n" \
|
492
|
+
"<|im_start|>assistant\n",
|
493
|
+
'system' => 'You are Dolphin, a helpful AI assistant.',
|
494
|
+
'details' =>
|
495
|
+
{ 'format' => 'gguf',
|
496
|
+
'family' => 'phi2',
|
497
|
+
'families' => ['phi2'],
|
498
|
+
'parameter_size' => '3B',
|
499
|
+
'quantization_level' => 'Q4_0' } }]
|
500
|
+
```
|
501
|
+
|
502
|
+
##### copy: Copy a Model
|
503
|
+
|
504
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#copy-a-model
|
505
|
+
|
506
|
+
```ruby
|
507
|
+
result = client.copy(
|
508
|
+
{ source: 'dolphin-phi',
|
509
|
+
destination: 'dolphin-phi-backup' }
|
510
|
+
)
|
511
|
+
```
|
512
|
+
|
513
|
+
Result:
|
514
|
+
```ruby
|
515
|
+
true
|
516
|
+
```
|
517
|
+
|
518
|
+
If the source model does not exist:
|
519
|
+
```ruby
|
520
|
+
begin
|
521
|
+
result = client.copy(
|
522
|
+
{ source: 'purple',
|
523
|
+
destination: 'purple-backup' }
|
524
|
+
)
|
525
|
+
rescue Ollama::Errors::OllamaError => error
|
526
|
+
puts error.class # Ollama::Errors::RequestError
|
527
|
+
puts error.message # 'the server responded with status 404'
|
528
|
+
|
529
|
+
puts error.payload
|
530
|
+
# { source: 'purple',
|
531
|
+
# destination: 'purple-backup',
|
532
|
+
# ...
|
533
|
+
# }
|
534
|
+
|
535
|
+
puts error.request.inspect
|
536
|
+
# #<Faraday::ResourceNotFound response={:status=>404, :headers...
|
537
|
+
end
|
538
|
+
```
|
539
|
+
|
540
|
+
##### delete: Delete a Model
|
541
|
+
|
542
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#delete-a-model
|
543
|
+
|
544
|
+
```ruby
|
545
|
+
result = client.delete(
|
546
|
+
{ name: 'dolphin-phi' }
|
547
|
+
)
|
548
|
+
```
|
549
|
+
|
550
|
+
Result:
|
551
|
+
```ruby
|
552
|
+
true
|
553
|
+
```
|
554
|
+
|
555
|
+
If the model does not exist:
|
556
|
+
```ruby
|
557
|
+
begin
|
558
|
+
result = client.delete(
|
559
|
+
{ name: 'dolphin-phi' }
|
560
|
+
)
|
561
|
+
rescue Ollama::Errors::OllamaError => error
|
562
|
+
puts error.class # Ollama::Errors::RequestError
|
563
|
+
puts error.message # 'the server responded with status 404'
|
564
|
+
|
565
|
+
puts error.payload
|
566
|
+
# { name: 'dolphin-phi',
|
567
|
+
# ...
|
568
|
+
# }
|
569
|
+
|
570
|
+
puts error.request.inspect
|
571
|
+
# #<Faraday::ResourceNotFound response={:status=>404, :headers...
|
572
|
+
end
|
573
|
+
```
|
574
|
+
|
575
|
+
##### pull: Pull a Model
|
576
|
+
|
577
|
+
API Documentation: https://github.com/jmorganca/ollama/blob/main/docs/api.md#pull-a-model
|
578
|
+
|
579
|
+
```ruby
|
580
|
+
result = client.pull(
|
581
|
+
{ name: 'dolphin-phi' }
|
582
|
+
) do |event, raw|
|
583
|
+
puts event
|
584
|
+
end
|
585
|
+
```
|
586
|
+
|
587
|
+
Event:
|
588
|
+
```ruby
|
589
|
+
{ 'status' => 'pulling manifest' }
|
590
|
+
```
|
591
|
+
|
592
|
+
Result:
|
593
|
+
```ruby
|
594
|
+
[{ 'status' => 'pulling manifest' },
|
595
|
+
{ 'status' => 'pulling 4eca7304a07a',
|
596
|
+
'digest' =>
|
597
|
+
'sha256:4eca7304a07a42c48887f159ef5ad82ed5a5bd30fe52db4aadae1dd938e26f70',
|
598
|
+
'total' => 1_602_463_008,
|
599
|
+
'completed' => 1_602_463_008 },
|
600
|
+
# ...
|
601
|
+
{ 'status' => 'verifying sha256 digest' },
|
602
|
+
{ 'status' => 'writing manifest' },
|
603
|
+
{ 'status' => 'removing any unused layers' },
|
604
|
+
{ 'status' => 'success' }]
|
605
|
+
```
|
606
|
+
|
607
|
+
##### push: Push a Model
|
608
|
+
|
609
|
+
Documentation: [API](https://github.com/jmorganca/ollama/blob/main/docs/api.md#push-a-model) and [_Publishing Your Model_](https://github.com/jmorganca/ollama/blob/main/docs/import.md#publishing-your-model-optional--early-alpha).
|
610
|
+
|
611
|
+
|
612
|
+
You need to create an account at https://ollama.ai and add your Public Key at https://ollama.ai/settings/keys.
|
613
|
+
|
614
|
+
Your keys are located in `/usr/share/ollama/.ollama/`. You may need to copy them to your user directory:
|
615
|
+
|
616
|
+
```sh
|
617
|
+
sudo cp /usr/share/ollama/.ollama/id_ed25519 ~/.ollama/
|
618
|
+
sudo cp /usr/share/ollama/.ollama/id_ed25519.pub ~/.ollama/
|
619
|
+
```
|
620
|
+
|
621
|
+
Copy your model to your user namespace:
|
622
|
+
|
623
|
+
```ruby
|
624
|
+
client.copy(
|
625
|
+
{ source: 'mario',
|
626
|
+
destination: 'your-user/mario' }
|
627
|
+
)
|
628
|
+
```
|
629
|
+
|
630
|
+
And push it:
|
631
|
+
|
632
|
+
```ruby
|
633
|
+
result = client.push(
|
634
|
+
{ name: 'your-user/mario' }
|
635
|
+
) do |event, raw|
|
636
|
+
puts event
|
637
|
+
end
|
638
|
+
```
|
639
|
+
|
640
|
+
Event:
|
641
|
+
```ruby
|
642
|
+
{ 'status' => 'retrieving manifest' }
|
643
|
+
```
|
644
|
+
|
645
|
+
Result:
|
646
|
+
```ruby
|
647
|
+
[{ 'status' => 'retrieving manifest' },
|
648
|
+
{ 'status' => 'pushing 4eca7304a07a',
|
649
|
+
'digest' =>
|
650
|
+
'sha256:4eca7304a07a42c48887f159ef5ad82ed5a5bd30fe52db4aadae1dd938e26f70',
|
651
|
+
'total' => 1_602_463_008,
|
652
|
+
'completed' => 1_602_463_008 },
|
653
|
+
# ...
|
654
|
+
{ 'status' => 'pushing e8bcbb2eebad',
|
655
|
+
'digest' =>
|
656
|
+
'sha256:e8bcbb2eebad88c2fa64bc32939162c064be96e70ff36aff566718fc9186b427',
|
657
|
+
'total' => 555,
|
658
|
+
'completed' => 555 },
|
659
|
+
{ 'status' => 'pushing manifest' },
|
660
|
+
{ 'status' => 'success' }]
|
661
|
+
```
|
662
|
+
|
663
|
+
### Streaming and Server-Sent Events (SSE)
|
664
|
+
|
665
|
+
[Server-Sent Events (SSE)](https://en.wikipedia.org/wiki/Server-sent_events) is a technology that allows certain endpoints to offer streaming capabilities, such as creating the impression that "the model is typing along with you," rather than delivering the entire answer all at once.
|
666
|
+
|
667
|
+
You can set up the client to use Server-Sent Events (SSE) for all supported endpoints:
|
668
|
+
```ruby
|
669
|
+
client = Ollama.new(
|
670
|
+
credentials: { address: 'http://localhost:11434' },
|
671
|
+
options: { server_sent_events: true }
|
672
|
+
)
|
673
|
+
```
|
674
|
+
|
675
|
+
Or, you can decide on a request basis:
|
676
|
+
```ruby
|
677
|
+
result = client.generate(
|
678
|
+
{ model: 'dolphin-phi',
|
679
|
+
prompt: 'Hi!' },
|
680
|
+
server_sent_events: true
|
681
|
+
) do |event, raw|
|
682
|
+
puts event
|
683
|
+
end
|
684
|
+
```
|
685
|
+
|
686
|
+
With Server-Sent Events (SSE) enabled, you can use a block to receive partial results via events. This feature is particularly useful for methods that offer streaming capabilities, such as `generate`: [Receiving Stream Events](#receiving-stream-events)
|
687
|
+
|
688
|
+
#### Server-Sent Events (SSE) Hang
|
689
|
+
|
690
|
+
Method calls will _hang_ until the server-sent events finish, so even without providing a block, you can obtain the final results of the received events: [Receiving Stream Events](#receiving-stream-events)
|
691
|
+
|
692
|
+
### New Functionalities and APIs
|
693
|
+
|
694
|
+
Ollama may launch a new endpoint that we haven't covered in the Gem yet. If that's the case, you may still be able to use it through the `request` method. For example, `generate` is just a wrapper for `api/generate`, which you can call directly like this:
|
695
|
+
|
696
|
+
```ruby
|
697
|
+
result = client.request(
|
698
|
+
'api/generate',
|
699
|
+
{ model: 'dolphin-phi',
|
700
|
+
prompt: 'Hi!' },
|
701
|
+
request_method: 'POST', server_sent_events: true
|
702
|
+
)
|
703
|
+
```
|
704
|
+
|
705
|
+
### Request Options
|
706
|
+
|
707
|
+
#### Timeout
|
708
|
+
|
709
|
+
You can set the maximum number of seconds to wait for the request to complete with the `timeout` option:
|
710
|
+
|
711
|
+
```ruby
|
712
|
+
client = Ollama.new(
|
713
|
+
credentials: { address: 'http://localhost:11434' },
|
714
|
+
options: { connection: { request: { timeout: 5 } } }
|
715
|
+
)
|
716
|
+
```
|
717
|
+
|
718
|
+
You can also have more fine-grained control over [Faraday's Request Options](https://lostisland.github.io/faraday/#/customization/request-options?id=request-options) if you prefer:
|
719
|
+
|
720
|
+
```ruby
|
721
|
+
client = Ollama.new(
|
722
|
+
credentials: { address: 'http://localhost:11434' },
|
723
|
+
options: {
|
724
|
+
connection: {
|
725
|
+
request: {
|
726
|
+
timeout: 5,
|
727
|
+
open_timeout: 5,
|
728
|
+
read_timeout: 5,
|
729
|
+
write_timeout: 5
|
730
|
+
}
|
731
|
+
}
|
732
|
+
}
|
733
|
+
)
|
734
|
+
```
|
735
|
+
|
736
|
+
### Error Handling
|
737
|
+
|
738
|
+
#### Rescuing
|
739
|
+
|
740
|
+
```ruby
|
741
|
+
require 'ollama-ai'
|
742
|
+
|
743
|
+
begin
|
744
|
+
client.chat_completions(
|
745
|
+
{ model: 'dolphin-phi',
|
746
|
+
prompt: 'Hi!' }
|
747
|
+
)
|
748
|
+
rescue Ollama::Errors::OllamaError => error
|
749
|
+
puts error.class # Ollama::Errors::RequestError
|
750
|
+
puts error.message # 'the server responded with status 500'
|
751
|
+
|
752
|
+
puts error.payload
|
753
|
+
# { model: 'dolphin-phi',
|
754
|
+
# prompt: 'Hi!',
|
755
|
+
# ...
|
756
|
+
# }
|
757
|
+
|
758
|
+
puts error.request.inspect
|
759
|
+
# #<Faraday::ServerError response={:status=>500, :headers...
|
760
|
+
end
|
761
|
+
```
|
762
|
+
|
763
|
+
#### For Short
|
764
|
+
|
765
|
+
```ruby
|
766
|
+
require 'ollama-ai/errors'
|
767
|
+
|
768
|
+
begin
|
769
|
+
client.chat_completions(
|
770
|
+
{ model: 'dolphin-phi',
|
771
|
+
prompt: 'Hi!' }
|
772
|
+
)
|
773
|
+
rescue OllamaError => error
|
774
|
+
puts error.class # Ollama::Errors::RequestError
|
775
|
+
end
|
776
|
+
```
|
777
|
+
|
778
|
+
#### Errors
|
779
|
+
|
780
|
+
```ruby
|
781
|
+
OllamaError
|
782
|
+
|
783
|
+
BlockWithoutServerSentEventsError
|
784
|
+
|
785
|
+
RequestError
|
786
|
+
```
|
787
|
+
|
788
|
+
## Development
|
789
|
+
|
790
|
+
```bash
|
791
|
+
bundle
|
792
|
+
rubocop -A
|
793
|
+
```
|
794
|
+
|
795
|
+
### Purpose
|
796
|
+
|
797
|
+
This Gem is designed to provide low-level access to Ollama, enabling people to build abstractions on top of it. If you are interested in more high-level abstractions or more user-friendly tools, you may want to consider [Nano Bots](https://github.com/icebaker/ruby-nano-bots) 💎 🤖.
|
798
|
+
|
799
|
+
### Publish to RubyGems
|
800
|
+
|
801
|
+
```bash
|
802
|
+
gem build ollama-ai.gemspec
|
803
|
+
|
804
|
+
gem signin
|
805
|
+
|
806
|
+
gem push ollama-ai-1.0.0.gem
|
807
|
+
```
|
808
|
+
|
809
|
+
### Updating the README
|
810
|
+
|
811
|
+
Install [Babashka](https://babashka.org):
|
812
|
+
|
813
|
+
```sh
|
814
|
+
curl -s https://raw.githubusercontent.com/babashka/babashka/master/install | sudo bash
|
815
|
+
```
|
816
|
+
|
817
|
+
Update the `template.md` file and then:
|
818
|
+
|
819
|
+
```sh
|
820
|
+
bb tasks/generate-readme.clj
|
821
|
+
```
|
822
|
+
|
823
|
+
Trick for automatically updating the `README.md` when `template.md` changes:
|
824
|
+
|
825
|
+
```sh
|
826
|
+
sudo pacman -S inotify-tools # Arch / Manjaro
|
827
|
+
sudo apt-get install inotify-tools # Debian / Ubuntu / Raspberry Pi OS
|
828
|
+
sudo dnf install inotify-tools # Fedora / CentOS / RHEL
|
829
|
+
|
830
|
+
while inotifywait -e modify template.md; do bb tasks/generate-readme.clj; done
|
831
|
+
```
|
832
|
+
|
833
|
+
Trick for Markdown Live Preview:
|
834
|
+
```sh
|
835
|
+
pip install -U markdown_live_preview
|
836
|
+
|
837
|
+
mlp README.md -p 8076
|
838
|
+
```
|
839
|
+
|
840
|
+
## Resources and References
|
841
|
+
|
842
|
+
These resources and references may be useful throughout your learning process:
|
843
|
+
|
844
|
+
- [Ollama Official Website](https://ollama.ai)
|
845
|
+
- [Ollama GitHub](https://github.com/jmorganca/ollama)
|
846
|
+
- [Ollama API Documentation](https://github.com/jmorganca/ollama/blob/main/docs/api.md)
|
847
|
+
|
848
|
+
## Disclaimer
|
849
|
+
|
850
|
+
This is not an official Ollama project, nor is it affiliated with Ollama in any way.
|
851
|
+
|
852
|
+
This software is distributed under the [MIT License](https://github.com/gbaptista/ollama-ai/blob/main/LICENSE). This license includes a disclaimer of warranty. Moreover, the authors assume no responsibility for any damage or costs that may result from using this project. Use the Ollama AI Ruby Gem at your own risk.
|