ollama-ruby 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGES.md +135 -110
- data/bin/ollama_chat +22 -8
- data/docker-compose.yml +1 -1
- data/lib/ollama/documents.rb +2 -1
- data/lib/ollama/utils/fetcher.rb +2 -0
- data/lib/ollama/version.rb +1 -1
- data/ollama-ruby.gemspec +3 -3
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7a6a94882e3f76e84fbceadd80ec844f9a93febd18ed294b01f7874867c53e68
|
|
4
|
+
data.tar.gz: 47cda3eef10438ff1cd136ffcc4469d2275bc2e26cdab985c7a001e2d0178b7f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5c42f47be513e41465693c47dee68447ac42a967f7ecaaf88ea51df6ea30eb9beb61b25595fc145909a0c9b8838d22756f47e71bc6acdd33bb47e4876545ccdc
|
|
7
|
+
data.tar.gz: b4ac6bf59882a23774fbfb3503607c1a22d200037b31613a8f9d565ebf174b99d66259cc62d99f0635e7ac99fd07cc51aec735dd7dbb31259dc767e403eaa5d7
|
data/CHANGES.md
CHANGED
|
@@ -1,99 +1,124 @@
|
|
|
1
1
|
# Changes
|
|
2
2
|
|
|
3
|
+
## 2024-10-07 v0.8.0
|
|
4
|
+
|
|
5
|
+
* **Refactor source handling in Ollama chat**:
|
|
6
|
+
+ Update regular expression to match dot-dot, dot or tilde prefix for local files
|
|
7
|
+
+ Add error handling for non-existent files in `Utils::Fetcher.read`
|
|
8
|
+
+ Stop matching the "hostname" part in file:// URLs
|
|
9
|
+
* **Update voice list command to suppress errors**:
|
|
10
|
+
+ Changed `say` command in OllamaChatConfig to `-v ? 2>/dev/null`
|
|
11
|
+
* **Add 'List:' and list of collections to the collection stats output**
|
|
12
|
+
* **Update collection stats display**:
|
|
13
|
+
+ Added new stat: `#Tags` to `collection_stats` method in `ollama_chat.rb`
|
|
14
|
+
+ Displaying number of tags for the current document collection
|
|
15
|
+
* **Refactor embed_source function to include document count**:
|
|
16
|
+
+ Added `count` parameter to `embed_source` method
|
|
17
|
+
+ Updated `puts` statements in `embed_source` and main loop to display document counts
|
|
18
|
+
+ Incremented `count` variable in main loop to track total documents embedded
|
|
19
|
+
+ Passed `count:` keyword argument to `embed_source` method
|
|
20
|
+
* **Add config option for batch_size and update document adding logic**:
|
|
21
|
+
+ Added `batch_size` option to `OllamaChatConfig`
|
|
22
|
+
+ Updated `embed_source` method in `bin/ollama_chat` to use `batch_size` from `config`
|
|
23
|
+
+ Updated `add` method in `lib/ollama/documents.rb` to accept `batch_size` and default to 10
|
|
24
|
+
* **Update Redis image to valkey/valkey:7.2.7-alpine**:
|
|
25
|
+
+ Updated `image` in `docker-compose.yml` to `valkey/valkey:7.2.7-alpine`
|
|
26
|
+
* **Reformat CHANGES.md**
|
|
27
|
+
|
|
3
28
|
## 2024-10-03 v0.7.0
|
|
4
29
|
|
|
5
30
|
* **Refactor command line interface**
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
31
|
+
+ Moved case order
|
|
32
|
+
+ Renamed `/collection clear [tag]|change` to `/collection (clear|change)`
|
|
33
|
+
+ Improved help message, added /info
|
|
9
34
|
* **Update README.md**
|
|
10
|
-
|
|
35
|
+
+ Update README.md to reflect changed/added commands
|
|
11
36
|
* **Add support for reading PostScript**
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
37
|
+
+ Extracted `pdf_read` method to read PDF files using `PDF::Reader`
|
|
38
|
+
+ Added `ps_read` method to read PostScript files by converting them to PDF with Ghostscript and using `pdf_read`.
|
|
39
|
+
+ Updated `parse_source` method to handle PostScript files
|
|
15
40
|
* **Update read buffer size for tempfile writes**
|
|
16
|
-
|
|
41
|
+
+ Updated `tmp.write` to use a larger buffer size (**16KB**) in IO.popen block.
|
|
17
42
|
* **Refactor Collection Chooser and usages**
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
43
|
+
+ Added confirmation prompt before clearing collection
|
|
44
|
+
+ Improved collection chooser with `[EXIT]` and `[ALL]` options
|
|
45
|
+
+ Added `ask?` method for user input
|
|
21
46
|
* **Add prompt to choose method**
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
47
|
+
+ Added `prompt` parameter to `choose` method in `Ollama::Utils::Chooser`
|
|
48
|
+
+ Modified output formatting for selected entry in `choose` method
|
|
49
|
+
+ Updated `choose` method to handle cases better where no entry was chosen
|
|
25
50
|
* **Fix Redis cache expiration logic**
|
|
26
|
-
|
|
51
|
+
+ Update `set` method to delete key expiration time is less than 1 second.
|
|
27
52
|
* **Update dependencies and add source tracking**
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
53
|
+
- Remove `sorted_set` dependency from Rakefile
|
|
54
|
+
- Modify `Ollama::Documents` class to track source of tags
|
|
55
|
+
- Update `Ollama::Utils::Tags` class to include source in tag output and add methods for tracking source
|
|
56
|
+
- Update tests for `Ollama::Utils::Tags` class
|
|
32
57
|
* **Refactor width calculation and add tests for wrap and truncate methods.**
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
58
|
+
+ Extend `Term::ANSIColor` in `Ollama::Utils::Width`
|
|
59
|
+
+ Update `width` method to use ellipsis length when truncating text
|
|
60
|
+
+ Add tests for `wrap` and `truncate` methods with percentage and length arguments
|
|
36
61
|
* **Add attr_reader for data and update equality check**
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
62
|
+
+ Added `attr_reader :data` to Ollama::Image class
|
|
63
|
+
+ Updated `==` method in Ollama::Image class to use `other.data`
|
|
64
|
+
+ Added test case in `image_spec.rb` to verify equality of images
|
|
40
65
|
|
|
41
66
|
## 2024-09-30 v0.6.0
|
|
42
67
|
|
|
43
68
|
### Significant Changes
|
|
44
69
|
|
|
45
70
|
* **Added voice toggle and change functionality**:
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
71
|
+
+ Removed `-v` command line switch
|
|
72
|
+
+ Added new Switch class for voice output
|
|
73
|
+
+ Added new method `change_voice` to toggle or change voice output
|
|
74
|
+
+ Updated `info` method to display current voice output if enabled
|
|
75
|
+
+ Updated `display_chat_help` method to include /voice command
|
|
51
76
|
* **Added expiring cache support**:
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
77
|
+
+ Added `Ollama::Utils::CacheFetcher` class for caching HTTP responses
|
|
78
|
+
+ Modified `Ollama::Utils::Fetcher` to use the new cache class
|
|
79
|
+
+ Updated `ollama_chat` script to use the cache when fetching sources
|
|
80
|
+
+ Added specs for the new cache fetcher class
|
|
56
81
|
* **Added change system prompt feature**:
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
82
|
+
+ Added `/system` command to change system prompt
|
|
83
|
+
+ Implemented `set_system_prompt` and `change_system_prompt` methods in `bin/ollama_chat`
|
|
84
|
+
+ Updated help messages in `README.md`
|
|
60
85
|
|
|
61
86
|
### Other Changes
|
|
62
87
|
|
|
63
88
|
* **Updated dependencies**:
|
|
64
|
-
|
|
65
|
-
|
|
89
|
+
+ Updated version of `xdg` gem to **7.0**
|
|
90
|
+
+ Added `xdg` dependency to Rakefile
|
|
66
91
|
* **Refactored error handling**:
|
|
67
|
-
|
|
68
|
-
|
|
92
|
+
+ Warn message updated to include more context about the error
|
|
93
|
+
+ `warn` statement now mentions "while pulling model"
|
|
69
94
|
* **Updated chat commands and added clipboard functionality**:
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
95
|
+
+ Added `/copy` command to copy last response to clipboard
|
|
96
|
+
+ Implemented `copy_to_clipboard` method in `ollama_chat`
|
|
97
|
+
+ Updated chat help display to include new `/copy` command
|
|
73
98
|
* **Refactored Ollama::Utils::Fetcher**:
|
|
74
|
-
|
|
75
|
-
|
|
99
|
+
+ Made instance methods private and only exposed class methods
|
|
100
|
+
+ Added `expose` method to `Ollama::Utils::FetcherSpec` for testing
|
|
76
101
|
* **Added version command to ollama chat binary**:
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
102
|
+
+ Added `version` method to print Ollama version and exit
|
|
103
|
+
+ Updated `$opts` string in `ollama` script to include `-V` option for version command
|
|
104
|
+
+ Added call to `version` method when `-V` option is used
|
|
80
105
|
* **Updated system prompt display**:
|
|
81
|
-
|
|
106
|
+
+ Changed `Ollama::Utils::Width.wrap` to `Ollama::Utils::ANSIMarkdown.parse` in `show_system_prompt` method
|
|
82
107
|
* **Added system prompt configuration via search_ui for ? argument value**:
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
108
|
+
+ Added `show_system_prompt` method to print configured system prompt
|
|
109
|
+
+ Modified `info` method to include system prompt in output
|
|
110
|
+
+ Implemented option `-s ?` to choose or specify system prompt
|
|
86
111
|
|
|
87
112
|
## 2024-09-26 v0.5.0
|
|
88
113
|
|
|
89
114
|
### New Features
|
|
90
115
|
|
|
91
116
|
* Add stdin substitution and variable expansion to `ollama_cli`:
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
117
|
+
+ Added support for `%{stdin}` in prompts, substituting with actual input
|
|
118
|
+
+ Added `-P` option to set prompt variables from command line arguments
|
|
119
|
+
+ Added handling of multiple placeholders in prompts
|
|
95
120
|
* Add proxy support to Ollama chat client:
|
|
96
|
-
|
|
121
|
+
+ Add `tins/xt/hash_union` gem to dependencies
|
|
97
122
|
+ Update `OllamaChatConfig` with new `proxy` option
|
|
98
123
|
+ Modify `http_options` method to include proxy and SSL verify peer options
|
|
99
124
|
based on config settings
|
|
@@ -125,7 +150,7 @@
|
|
|
125
150
|
+ Added support for displaying a specific number of messages with `/list
|
|
126
151
|
[n]`
|
|
127
152
|
* Update chat commands' quit functionality:
|
|
128
|
-
|
|
153
|
+
+ Moved `/quit` command to exit the program
|
|
129
154
|
* Refactor OllamaChatConfig web prompt:
|
|
130
155
|
+ Add `web` prompt to `OllamaChatConfig` class
|
|
131
156
|
+ Replace hardcoded content with variable `content`
|
|
@@ -155,10 +180,10 @@
|
|
|
155
180
|
### Refactoring
|
|
156
181
|
|
|
157
182
|
* Refactor `file_argument.rb` for better readability:
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
183
|
+
+ Update conditionals in Ollama::Utils::FileArgument module
|
|
184
|
+
+ Simplify logic with improved variable usage
|
|
185
|
+
+ Remove unnecessary elsif statement
|
|
186
|
+
+ Use consistent indentation and spacing throughout the code
|
|
162
187
|
* Refactor Redis-backed memory cache:
|
|
163
188
|
+ Removed `pre` and `unpre` methods from `Ollama::Documents` use mixin
|
|
164
189
|
instead.
|
|
@@ -166,12 +191,12 @@
|
|
|
166
191
|
### Documentation
|
|
167
192
|
|
|
168
193
|
* Update README.md to reflect changes in `ollama_chat` functionality.
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
194
|
+
+ Modified commands:
|
|
195
|
+
- `/import source` to import the source's content
|
|
196
|
+
- `/embed source` to embed the source's content
|
|
172
197
|
- `/summarize [n] source` to summarize the source's content in n words
|
|
173
198
|
- `/embedding` to toggle embedding paused or not
|
|
174
|
-
|
|
199
|
+
- `/embed source` to embed the source's content
|
|
175
200
|
|
|
176
201
|
### Dependencies and Date Updates
|
|
177
202
|
|
|
@@ -179,13 +204,13 @@
|
|
|
179
204
|
+ Added `logger` (~> **1.0**) and `json` (~> **2.0**) as runtime
|
|
180
205
|
dependencies to Rakefile and ollama-ruby.gemspec.
|
|
181
206
|
+ Updated date in ollama-ruby.gemspec from "2024-09-21" to "2024-09-22".
|
|
182
|
-
|
|
207
|
+
+ Added `require 'logger'` to lib/ollama.rb.
|
|
183
208
|
|
|
184
209
|
### Other Changes
|
|
185
210
|
|
|
186
211
|
* Add SSL no verify option to OllamaChatConfig and Utils::Fetcher:
|
|
187
|
-
|
|
188
|
-
|
|
212
|
+
+ Added `ssl_no_verify` option to OllamaChatConfig
|
|
213
|
+
+ Updated Utils::Fetcher to take an
|
|
189
214
|
|
|
190
215
|
## 2024-09-21 v0.4.0
|
|
191
216
|
|
|
@@ -218,20 +243,20 @@
|
|
|
218
243
|
## 2024-09-15 v0.3.2
|
|
219
244
|
|
|
220
245
|
* Add color support to chooser module:
|
|
221
|
-
|
|
246
|
+
+ Include `Term::ANSIColor` in `Ollama::Utils::Chooser` module
|
|
222
247
|
+ Use `blue`, `on_blue` ANSI color for selected item in query method
|
|
223
248
|
* Refactor summarize method to also import sources:
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
249
|
+
+ Added `content` variable to store result of `parse_source`
|
|
250
|
+
+ Replaced `or return` with explicit assignment and return
|
|
251
|
+
+ Added calls to `source_io.rewind` and `import_document`
|
|
227
252
|
* Add new test for `file_argument_spec.rb`
|
|
228
253
|
* Refactor tag list initialization and merging:
|
|
229
|
-
|
|
230
|
-
|
|
254
|
+
+ Use array literals for initializing tags lists
|
|
255
|
+
+ Use array literals for passing to merge method
|
|
231
256
|
* Update dependencies and dates in Rakefile and gemspec:
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
257
|
+
+ Removed '.utilsrc' from ignored files in Rakefile
|
|
258
|
+
+ Updated date in `ollama-ruby.gemspec` to "2024-09-13"
|
|
259
|
+
+ Removed 'utils' development dependency from `ollama-ruby.gemspec`
|
|
235
260
|
* Refactor `search_web` method to allow n parameter to be optional and default
|
|
236
261
|
to 1.
|
|
237
262
|
|
|
@@ -306,31 +331,31 @@
|
|
|
306
331
|
### Changes
|
|
307
332
|
|
|
308
333
|
* **Added Web Search Functionality to `ollama_chat`**
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
334
|
+
+ Added `/web` command to fetch search results from DuckDuckGo
|
|
335
|
+
+ Updated `/summarize` command to handle cases where summarization fails
|
|
336
|
+
+ Fix bug in parsing content type of source document
|
|
312
337
|
* **Refactored Options Class and Usage**
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
338
|
+
+ Renamed `options` variable to use `Options[]` method in ollama_chat script
|
|
339
|
+
+ Added `[](value)` method to Ollama::Options class for casting hashes
|
|
340
|
+
+ Updated options_spec.rb with tests for casting hashes and error handling
|
|
316
341
|
* **Refactored Web Search Command**
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
342
|
+
+ Added support for specifying a page number in `/web` command
|
|
343
|
+
+ Updated regular expression to match new format
|
|
344
|
+
+ Passed page number as an argument to `search_web` method
|
|
345
|
+
+ Updated content string to reference the query and sources correctly
|
|
321
346
|
* **DTO Class Changes**
|
|
322
|
-
|
|
323
|
-
|
|
347
|
+
+ Renamed `json_create` method to `from_hash` in Ollama::DTO class
|
|
348
|
+
+ Updated `as_json` method to remove now unnecessary hash creation
|
|
324
349
|
* **Message and Tool Spec Changes**
|
|
325
|
-
|
|
326
|
-
|
|
350
|
+
+ Removed `json_class` from JSON serialization in message_spec
|
|
351
|
+
+ Removed `json_class` from JSON serialization in tool_spec
|
|
327
352
|
* **Command Spec Changes**
|
|
328
|
-
|
|
353
|
+
+ Removed `json_class` from JSON serialization in various command specs (e.g. generate_spec, pull_spec, etc.)
|
|
329
354
|
* **Miscellaneous Changes**
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
355
|
+
+ Improved width calculation for text truncation
|
|
356
|
+
+ Updated FollowChat class to display evaluation statistics
|
|
357
|
+
+ Update OllamaChatConfig to use EOT instead of end for heredoc syntax
|
|
358
|
+
+ Add .keep file to tmp directory
|
|
334
359
|
|
|
335
360
|
## 2024-08-30 v0.1.0
|
|
336
361
|
|
|
@@ -339,18 +364,18 @@
|
|
|
339
364
|
#### Significant Changes
|
|
340
365
|
|
|
341
366
|
* **Document Splitting and Embedding Functionality**: Added `Ollama::Documents` class with methods for adding documents, checking existence, deleting documents, and finding similar documents.
|
|
342
|
-
|
|
343
|
-
|
|
367
|
+
+ Introduced two types of caches: `MemoryCache` and `RedisCache`
|
|
368
|
+
+ Implemented `SemanticSplitter` class to split text into sentences based on semantic similarity
|
|
344
369
|
* **Improved Ollama Chat Client**: Added support for document embeddings and web/file RAG
|
|
345
|
-
|
|
346
|
-
|
|
370
|
+
+ Allowed configuration per yaml file
|
|
371
|
+
+ Parse user input for URLs or files to send images to multimodal models
|
|
347
372
|
* **Redis Docker Service**: Set `REDIS_URL` environment variable to `redis://localhost:9736`
|
|
348
|
-
|
|
373
|
+
+ Added Redis service to `docker-compose.yml`
|
|
349
374
|
* **Status Display and Progress Updates**: Added infobar.label = response.status when available
|
|
350
|
-
|
|
351
|
-
|
|
375
|
+
+ Updated infobar with progress message on each call if total and completed are set
|
|
376
|
+
+ Display error message from response.error if present
|
|
352
377
|
* **Refactored Chat Commands**: Simplified regular expression patterns for `/pop`, `/save`, `/load`, and `/image` commands
|
|
353
|
-
|
|
378
|
+
+ Added whitespace to some command patterns for better readability
|
|
354
379
|
|
|
355
380
|
#### Other Changes
|
|
356
381
|
|
|
@@ -366,13 +391,13 @@
|
|
|
366
391
|
## 2024-08-16 v0.0.1
|
|
367
392
|
|
|
368
393
|
* **New Features**
|
|
369
|
-
|
|
370
|
-
|
|
394
|
+
+ Added missing options parameter to Embed command
|
|
395
|
+
+ Documented new `/api/embed` endpoint
|
|
371
396
|
* **Improvements**
|
|
372
|
-
|
|
397
|
+
+ Improved example in README.md
|
|
373
398
|
* **Code Refactoring**
|
|
374
|
-
|
|
375
|
-
|
|
399
|
+
+ Renamed `client` to `ollama` in client and command specs
|
|
400
|
+
+ Updated expectations to use `ollama` instead of `client`
|
|
376
401
|
|
|
377
402
|
## 2024-08-12 v0.0.0
|
|
378
403
|
|
data/bin/ollama_chat
CHANGED
|
@@ -49,7 +49,7 @@ class OllamaChatConfig
|
|
|
49
49
|
voice:
|
|
50
50
|
enabled: false
|
|
51
51
|
default: Samantha
|
|
52
|
-
list: <%= `say -v
|
|
52
|
+
list: <%= `say -v ? 2>/dev/null`.lines.map { _1[/^(.+?)\s+[a-z]{2}_[a-zA-Z0-9]{2,}/, 1] }.uniq.sort.to_s.force_encoding('ASCII-8BIT') %>
|
|
53
53
|
markdown: true
|
|
54
54
|
stream: true
|
|
55
55
|
embedding:
|
|
@@ -59,6 +59,7 @@ class OllamaChatConfig
|
|
|
59
59
|
options: {}
|
|
60
60
|
# Retrieval prompt template:
|
|
61
61
|
prompt: 'Represent this sentence for searching relevant passages: %s'
|
|
62
|
+
batch_size: 10
|
|
62
63
|
collection: <%= ENV['OLLAMA_CHAT_COLLECTION'] %>
|
|
63
64
|
found_texts_size: 4096
|
|
64
65
|
found_texts_count: null
|
|
@@ -478,9 +479,14 @@ def parse_source(source_io)
|
|
|
478
479
|
end
|
|
479
480
|
end
|
|
480
481
|
|
|
481
|
-
def embed_source(source_io, source)
|
|
482
|
+
def embed_source(source_io, source, count: nil)
|
|
482
483
|
$embedding.on? or return parse_source(source_io)
|
|
483
|
-
|
|
484
|
+
m = "Embedding #{italic { source_io&.content_type }} document #{source.to_s.inspect}."
|
|
485
|
+
if count
|
|
486
|
+
puts '%u. %s' % [ count, m ]
|
|
487
|
+
else
|
|
488
|
+
puts m
|
|
489
|
+
end
|
|
484
490
|
text = parse_source(source_io) or return
|
|
485
491
|
text.downcase!
|
|
486
492
|
splitter_config = $config.embedding.splitter
|
|
@@ -517,7 +523,7 @@ def embed_source(source_io, source)
|
|
|
517
523
|
length: 10
|
|
518
524
|
)
|
|
519
525
|
end
|
|
520
|
-
$documents.add(inputs, source:)
|
|
526
|
+
$documents.add(inputs, source:, batch_size: $config.embedding.batch_size?)
|
|
521
527
|
end
|
|
522
528
|
|
|
523
529
|
def add_image(images, source_io, source)
|
|
@@ -554,7 +560,7 @@ def fetch_source(source, &block)
|
|
|
554
560
|
) do |tmp|
|
|
555
561
|
block.(tmp)
|
|
556
562
|
end
|
|
557
|
-
when %r(\Afile://(
|
|
563
|
+
when %r(\Afile://(/\S*)|\A((?:\.\.|[~.]?)/\S*))
|
|
558
564
|
filename = $~.captures.compact.first
|
|
559
565
|
filename = File.expand_path(filename)
|
|
560
566
|
Utils::Fetcher.read(filename) do |tmp|
|
|
@@ -612,7 +618,7 @@ def parse_content(content, images)
|
|
|
612
618
|
images.clear
|
|
613
619
|
tags = Utils::Tags.new
|
|
614
620
|
|
|
615
|
-
content.scan(%r([.~]?/\S+|https?://\S+|#\S+)).each do |source|
|
|
621
|
+
content.scan(%r((?:\.\.|[.~])?/\S+|https?://\S+|#\S+)).each do |source|
|
|
616
622
|
case source
|
|
617
623
|
when /\A#(\S+)/
|
|
618
624
|
tags.add($1, source:)
|
|
@@ -672,12 +678,18 @@ ensure
|
|
|
672
678
|
end
|
|
673
679
|
|
|
674
680
|
def collection_stats
|
|
681
|
+
list = $documents.collections.sort.map { |c|
|
|
682
|
+
' ' + ($documents.collection == c ? bold { c } : c).to_s
|
|
683
|
+
}.join(?\n)
|
|
675
684
|
puts <<~EOT
|
|
676
|
-
Collection
|
|
685
|
+
Current Collection
|
|
677
686
|
Name: #{bold{$documents.collection}}
|
|
678
687
|
Embedding model: #{bold{$embedding_model}}
|
|
679
688
|
#Embeddings: #{$documents.size}
|
|
689
|
+
#Tags: #{$documents.tags.size}
|
|
680
690
|
Tags: #{$documents.tags}
|
|
691
|
+
List:
|
|
692
|
+
#{list}
|
|
681
693
|
EOT
|
|
682
694
|
end
|
|
683
695
|
|
|
@@ -889,11 +901,13 @@ if $embedding.on?
|
|
|
889
901
|
end
|
|
890
902
|
end
|
|
891
903
|
puts "Collection #{bold{collection}}: Adding #{document_list.size} documents…"
|
|
904
|
+
count = 1
|
|
892
905
|
document_list.each_slice(25) do |docs|
|
|
893
906
|
docs.each do |doc|
|
|
894
907
|
fetch_source(doc) do |doc_io|
|
|
895
|
-
embed_source(doc_io, doc)
|
|
908
|
+
embed_source(doc_io, doc, count:)
|
|
896
909
|
end
|
|
910
|
+
count += 1
|
|
897
911
|
end
|
|
898
912
|
end
|
|
899
913
|
end
|
data/docker-compose.yml
CHANGED
data/lib/ollama/documents.rb
CHANGED
|
@@ -55,8 +55,9 @@ class Ollama::Documents
|
|
|
55
55
|
@cache.prefix = prefix
|
|
56
56
|
end
|
|
57
57
|
|
|
58
|
-
def add(inputs, batch_size:
|
|
58
|
+
def add(inputs, batch_size: nil, source: nil, tags: [])
|
|
59
59
|
inputs = Array(inputs)
|
|
60
|
+
batch_size ||= 10
|
|
60
61
|
tags = Ollama::Utils::Tags.new(tags, source:)
|
|
61
62
|
if source
|
|
62
63
|
tags.add(File.basename(source).gsub(/\?.*/, ''), source:)
|
data/lib/ollama/utils/fetcher.rb
CHANGED
data/lib/ollama/version.rb
CHANGED
data/ollama-ruby.gemspec
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
# -*- encoding: utf-8 -*-
|
|
2
|
-
# stub: ollama-ruby 0.
|
|
2
|
+
# stub: ollama-ruby 0.8.0 ruby lib
|
|
3
3
|
|
|
4
4
|
Gem::Specification.new do |s|
|
|
5
5
|
s.name = "ollama-ruby".freeze
|
|
6
|
-
s.version = "0.
|
|
6
|
+
s.version = "0.8.0".freeze
|
|
7
7
|
|
|
8
8
|
s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version=
|
|
9
9
|
s.require_paths = ["lib".freeze]
|
|
10
10
|
s.authors = ["Florian Frank".freeze]
|
|
11
|
-
s.date = "2024-10-
|
|
11
|
+
s.date = "2024-10-06"
|
|
12
12
|
s.description = "Library that allows interacting with the Ollama API".freeze
|
|
13
13
|
s.email = "flori@ping.de".freeze
|
|
14
14
|
s.executables = ["ollama_console".freeze, "ollama_chat".freeze, "ollama_update".freeze, "ollama_cli".freeze]
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ollama-ruby
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.8.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Florian Frank
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2024-10-
|
|
11
|
+
date: 2024-10-06 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: gem_hadar
|