ollama-ruby 0.7.0 → 0.8.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGES.md +135 -110
- data/bin/ollama_chat +22 -8
- data/docker-compose.yml +1 -1
- data/lib/ollama/documents.rb +2 -1
- data/lib/ollama/utils/fetcher.rb +2 -0
- data/lib/ollama/version.rb +1 -1
- data/ollama-ruby.gemspec +3 -3
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7a6a94882e3f76e84fbceadd80ec844f9a93febd18ed294b01f7874867c53e68
|
|
4
|
+
data.tar.gz: 47cda3eef10438ff1cd136ffcc4469d2275bc2e26cdab985c7a001e2d0178b7f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5c42f47be513e41465693c47dee68447ac42a967f7ecaaf88ea51df6ea30eb9beb61b25595fc145909a0c9b8838d22756f47e71bc6acdd33bb47e4876545ccdc
|
|
7
|
+
data.tar.gz: b4ac6bf59882a23774fbfb3503607c1a22d200037b31613a8f9d565ebf174b99d66259cc62d99f0635e7ac99fd07cc51aec735dd7dbb31259dc767e403eaa5d7
|
data/CHANGES.md
CHANGED
|
@@ -1,99 +1,124 @@
|
|
|
1
1
|
# Changes
|
|
2
2
|
|
|
3
|
+
## 2024-10-07 v0.8.0
|
|
4
|
+
|
|
5
|
+
* **Refactor source handling in Ollama chat**:
|
|
6
|
+
+ Update regular expression to match dot-dot, dot or tilde prefix for local files
|
|
7
|
+
+ Add error handling for non-existent files in `Utils::Fetcher.read`
|
|
8
|
+
+ Stop matching the "hostname" part in file:// URLs
|
|
9
|
+
* **Update voice list command to suppress errors**:
|
|
10
|
+
+ Changed `say` command in OllamaChatConfig to `-v ? 2>/dev/null`
|
|
11
|
+
* **Add 'List:' and list of collections to the collection stats output**
|
|
12
|
+
* **Update collection stats display**:
|
|
13
|
+
+ Added new stat: `#Tags` to `collection_stats` method in `ollama_chat.rb`
|
|
14
|
+
+ Displaying number of tags for the current document collection
|
|
15
|
+
* **Refactor embed_source function to include document count**:
|
|
16
|
+
+ Added `count` parameter to `embed_source` method
|
|
17
|
+
+ Updated `puts` statements in `embed_source` and main loop to display document counts
|
|
18
|
+
+ Incremented `count` variable in main loop to track total documents embedded
|
|
19
|
+
+ Passed `count:` keyword argument to `embed_source` method
|
|
20
|
+
* **Add config option for batch_size and update document adding logic**:
|
|
21
|
+
+ Added `batch_size` option to `OllamaChatConfig`
|
|
22
|
+
+ Updated `embed_source` method in `bin/ollama_chat` to use `batch_size` from `config`
|
|
23
|
+
+ Updated `add` method in `lib/ollama/documents.rb` to accept `batch_size` and default to 10
|
|
24
|
+
* **Update Redis image to valkey/valkey:7.2.7-alpine**:
|
|
25
|
+
+ Updated `image` in `docker-compose.yml` to `valkey/valkey:7.2.7-alpine`
|
|
26
|
+
* **Reformat CHANGES.md**
|
|
27
|
+
|
|
3
28
|
## 2024-10-03 v0.7.0
|
|
4
29
|
|
|
5
30
|
* **Refactor command line interface**
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
31
|
+
+ Moved case order
|
|
32
|
+
+ Renamed `/collection clear [tag]|change` to `/collection (clear|change)`
|
|
33
|
+
+ Improved help message, added /info
|
|
9
34
|
* **Update README.md**
|
|
10
|
-
|
|
35
|
+
+ Update README.md to reflect changed/added commands
|
|
11
36
|
* **Add support for reading PostScript**
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
37
|
+
+ Extracted `pdf_read` method to read PDF files using `PDF::Reader`
|
|
38
|
+
+ Added `ps_read` method to read PostScript files by converting them to PDF with Ghostscript and using `pdf_read`.
|
|
39
|
+
+ Updated `parse_source` method to handle PostScript files
|
|
15
40
|
* **Update read buffer size for tempfile writes**
|
|
16
|
-
|
|
41
|
+
+ Updated `tmp.write` to use a larger buffer size (**16KB**) in IO.popen block.
|
|
17
42
|
* **Refactor Collection Chooser and usages**
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
43
|
+
+ Added confirmation prompt before clearing collection
|
|
44
|
+
+ Improved collection chooser with `[EXIT]` and `[ALL]` options
|
|
45
|
+
+ Added `ask?` method for user input
|
|
21
46
|
* **Add prompt to choose method**
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
47
|
+
+ Added `prompt` parameter to `choose` method in `Ollama::Utils::Chooser`
|
|
48
|
+
+ Modified output formatting for selected entry in `choose` method
|
|
49
|
+
+ Updated `choose` method to handle cases better where no entry was chosen
|
|
25
50
|
* **Fix Redis cache expiration logic**
|
|
26
|
-
|
|
51
|
+
+ Update `set` method to delete key expiration time is less than 1 second.
|
|
27
52
|
* **Update dependencies and add source tracking**
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
53
|
+
- Remove `sorted_set` dependency from Rakefile
|
|
54
|
+
- Modify `Ollama::Documents` class to track source of tags
|
|
55
|
+
- Update `Ollama::Utils::Tags` class to include source in tag output and add methods for tracking source
|
|
56
|
+
- Update tests for `Ollama::Utils::Tags` class
|
|
32
57
|
* **Refactor width calculation and add tests for wrap and truncate methods.**
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
58
|
+
+ Extend `Term::ANSIColor` in `Ollama::Utils::Width`
|
|
59
|
+
+ Update `width` method to use ellipsis length when truncating text
|
|
60
|
+
+ Add tests for `wrap` and `truncate` methods with percentage and length arguments
|
|
36
61
|
* **Add attr_reader for data and update equality check**
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
62
|
+
+ Added `attr_reader :data` to Ollama::Image class
|
|
63
|
+
+ Updated `==` method in Ollama::Image class to use `other.data`
|
|
64
|
+
+ Added test case in `image_spec.rb` to verify equality of images
|
|
40
65
|
|
|
41
66
|
## 2024-09-30 v0.6.0
|
|
42
67
|
|
|
43
68
|
### Significant Changes
|
|
44
69
|
|
|
45
70
|
* **Added voice toggle and change functionality**:
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
71
|
+
+ Removed `-v` command line switch
|
|
72
|
+
+ Added new Switch class for voice output
|
|
73
|
+
+ Added new method `change_voice` to toggle or change voice output
|
|
74
|
+
+ Updated `info` method to display current voice output if enabled
|
|
75
|
+
+ Updated `display_chat_help` method to include /voice command
|
|
51
76
|
* **Added expiring cache support**:
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
77
|
+
+ Added `Ollama::Utils::CacheFetcher` class for caching HTTP responses
|
|
78
|
+
+ Modified `Ollama::Utils::Fetcher` to use the new cache class
|
|
79
|
+
+ Updated `ollama_chat` script to use the cache when fetching sources
|
|
80
|
+
+ Added specs for the new cache fetcher class
|
|
56
81
|
* **Added change system prompt feature**:
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
82
|
+
+ Added `/system` command to change system prompt
|
|
83
|
+
+ Implemented `set_system_prompt` and `change_system_prompt` methods in `bin/ollama_chat`
|
|
84
|
+
+ Updated help messages in `README.md`
|
|
60
85
|
|
|
61
86
|
### Other Changes
|
|
62
87
|
|
|
63
88
|
* **Updated dependencies**:
|
|
64
|
-
|
|
65
|
-
|
|
89
|
+
+ Updated version of `xdg` gem to **7.0**
|
|
90
|
+
+ Added `xdg` dependency to Rakefile
|
|
66
91
|
* **Refactored error handling**:
|
|
67
|
-
|
|
68
|
-
|
|
92
|
+
+ Warn message updated to include more context about the error
|
|
93
|
+
+ `warn` statement now mentions "while pulling model"
|
|
69
94
|
* **Updated chat commands and added clipboard functionality**:
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
95
|
+
+ Added `/copy` command to copy last response to clipboard
|
|
96
|
+
+ Implemented `copy_to_clipboard` method in `ollama_chat`
|
|
97
|
+
+ Updated chat help display to include new `/copy` command
|
|
73
98
|
* **Refactored Ollama::Utils::Fetcher**:
|
|
74
|
-
|
|
75
|
-
|
|
99
|
+
+ Made instance methods private and only exposed class methods
|
|
100
|
+
+ Added `expose` method to `Ollama::Utils::FetcherSpec` for testing
|
|
76
101
|
* **Added version command to ollama chat binary**:
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
102
|
+
+ Added `version` method to print Ollama version and exit
|
|
103
|
+
+ Updated `$opts` string in `ollama` script to include `-V` option for version command
|
|
104
|
+
+ Added call to `version` method when `-V` option is used
|
|
80
105
|
* **Updated system prompt display**:
|
|
81
|
-
|
|
106
|
+
+ Changed `Ollama::Utils::Width.wrap` to `Ollama::Utils::ANSIMarkdown.parse` in `show_system_prompt` method
|
|
82
107
|
* **Added system prompt configuration via search_ui for ? argument value**:
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
108
|
+
+ Added `show_system_prompt` method to print configured system prompt
|
|
109
|
+
+ Modified `info` method to include system prompt in output
|
|
110
|
+
+ Implemented option `-s ?` to choose or specify system prompt
|
|
86
111
|
|
|
87
112
|
## 2024-09-26 v0.5.0
|
|
88
113
|
|
|
89
114
|
### New Features
|
|
90
115
|
|
|
91
116
|
* Add stdin substitution and variable expansion to `ollama_cli`:
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
117
|
+
+ Added support for `%{stdin}` in prompts, substituting with actual input
|
|
118
|
+
+ Added `-P` option to set prompt variables from command line arguments
|
|
119
|
+
+ Added handling of multiple placeholders in prompts
|
|
95
120
|
* Add proxy support to Ollama chat client:
|
|
96
|
-
|
|
121
|
+
+ Add `tins/xt/hash_union` gem to dependencies
|
|
97
122
|
+ Update `OllamaChatConfig` with new `proxy` option
|
|
98
123
|
+ Modify `http_options` method to include proxy and SSL verify peer options
|
|
99
124
|
based on config settings
|
|
@@ -125,7 +150,7 @@
|
|
|
125
150
|
+ Added support for displaying a specific number of messages with `/list
|
|
126
151
|
[n]`
|
|
127
152
|
* Update chat commands' quit functionality:
|
|
128
|
-
|
|
153
|
+
+ Moved `/quit` command to exit the program
|
|
129
154
|
* Refactor OllamaChatConfig web prompt:
|
|
130
155
|
+ Add `web` prompt to `OllamaChatConfig` class
|
|
131
156
|
+ Replace hardcoded content with variable `content`
|
|
@@ -155,10 +180,10 @@
|
|
|
155
180
|
### Refactoring
|
|
156
181
|
|
|
157
182
|
* Refactor `file_argument.rb` for better readability:
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
183
|
+
+ Update conditionals in Ollama::Utils::FileArgument module
|
|
184
|
+
+ Simplify logic with improved variable usage
|
|
185
|
+
+ Remove unnecessary elsif statement
|
|
186
|
+
+ Use consistent indentation and spacing throughout the code
|
|
162
187
|
* Refactor Redis-backed memory cache:
|
|
163
188
|
+ Removed `pre` and `unpre` methods from `Ollama::Documents` use mixin
|
|
164
189
|
instead.
|
|
@@ -166,12 +191,12 @@
|
|
|
166
191
|
### Documentation
|
|
167
192
|
|
|
168
193
|
* Update README.md to reflect changes in `ollama_chat` functionality.
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
194
|
+
+ Modified commands:
|
|
195
|
+
- `/import source` to import the source's content
|
|
196
|
+
- `/embed source` to embed the source's content
|
|
172
197
|
- `/summarize [n] source` to summarize the source's content in n words
|
|
173
198
|
- `/embedding` to toggle embedding paused or not
|
|
174
|
-
|
|
199
|
+
- `/embed source` to embed the source's content
|
|
175
200
|
|
|
176
201
|
### Dependencies and Date Updates
|
|
177
202
|
|
|
@@ -179,13 +204,13 @@
|
|
|
179
204
|
+ Added `logger` (~> **1.0**) and `json` (~> **2.0**) as runtime
|
|
180
205
|
dependencies to Rakefile and ollama-ruby.gemspec.
|
|
181
206
|
+ Updated date in ollama-ruby.gemspec from "2024-09-21" to "2024-09-22".
|
|
182
|
-
|
|
207
|
+
+ Added `require 'logger'` to lib/ollama.rb.
|
|
183
208
|
|
|
184
209
|
### Other Changes
|
|
185
210
|
|
|
186
211
|
* Add SSL no verify option to OllamaChatConfig and Utils::Fetcher:
|
|
187
|
-
|
|
188
|
-
|
|
212
|
+
+ Added `ssl_no_verify` option to OllamaChatConfig
|
|
213
|
+
+ Updated Utils::Fetcher to take an
|
|
189
214
|
|
|
190
215
|
## 2024-09-21 v0.4.0
|
|
191
216
|
|
|
@@ -218,20 +243,20 @@
|
|
|
218
243
|
## 2024-09-15 v0.3.2
|
|
219
244
|
|
|
220
245
|
* Add color support to chooser module:
|
|
221
|
-
|
|
246
|
+
+ Include `Term::ANSIColor` in `Ollama::Utils::Chooser` module
|
|
222
247
|
+ Use `blue`, `on_blue` ANSI color for selected item in query method
|
|
223
248
|
* Refactor summarize method to also import sources:
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
249
|
+
+ Added `content` variable to store result of `parse_source`
|
|
250
|
+
+ Replaced `or return` with explicit assignment and return
|
|
251
|
+
+ Added calls to `source_io.rewind` and `import_document`
|
|
227
252
|
* Add new test for `file_argument_spec.rb`
|
|
228
253
|
* Refactor tag list initialization and merging:
|
|
229
|
-
|
|
230
|
-
|
|
254
|
+
+ Use array literals for initializing tags lists
|
|
255
|
+
+ Use array literals for passing to merge method
|
|
231
256
|
* Update dependencies and dates in Rakefile and gemspec:
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
257
|
+
+ Removed '.utilsrc' from ignored files in Rakefile
|
|
258
|
+
+ Updated date in `ollama-ruby.gemspec` to "2024-09-13"
|
|
259
|
+
+ Removed 'utils' development dependency from `ollama-ruby.gemspec`
|
|
235
260
|
* Refactor `search_web` method to allow n parameter to be optional and default
|
|
236
261
|
to 1.
|
|
237
262
|
|
|
@@ -306,31 +331,31 @@
|
|
|
306
331
|
### Changes
|
|
307
332
|
|
|
308
333
|
* **Added Web Search Functionality to `ollama_chat`**
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
334
|
+
+ Added `/web` command to fetch search results from DuckDuckGo
|
|
335
|
+
+ Updated `/summarize` command to handle cases where summarization fails
|
|
336
|
+
+ Fix bug in parsing content type of source document
|
|
312
337
|
* **Refactored Options Class and Usage**
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
338
|
+
+ Renamed `options` variable to use `Options[]` method in ollama_chat script
|
|
339
|
+
+ Added `[](value)` method to Ollama::Options class for casting hashes
|
|
340
|
+
+ Updated options_spec.rb with tests for casting hashes and error handling
|
|
316
341
|
* **Refactored Web Search Command**
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
342
|
+
+ Added support for specifying a page number in `/web` command
|
|
343
|
+
+ Updated regular expression to match new format
|
|
344
|
+
+ Passed page number as an argument to `search_web` method
|
|
345
|
+
+ Updated content string to reference the query and sources correctly
|
|
321
346
|
* **DTO Class Changes**
|
|
322
|
-
|
|
323
|
-
|
|
347
|
+
+ Renamed `json_create` method to `from_hash` in Ollama::DTO class
|
|
348
|
+
+ Updated `as_json` method to remove now unnecessary hash creation
|
|
324
349
|
* **Message and Tool Spec Changes**
|
|
325
|
-
|
|
326
|
-
|
|
350
|
+
+ Removed `json_class` from JSON serialization in message_spec
|
|
351
|
+
+ Removed `json_class` from JSON serialization in tool_spec
|
|
327
352
|
* **Command Spec Changes**
|
|
328
|
-
|
|
353
|
+
+ Removed `json_class` from JSON serialization in various command specs (e.g. generate_spec, pull_spec, etc.)
|
|
329
354
|
* **Miscellaneous Changes**
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
355
|
+
+ Improved width calculation for text truncation
|
|
356
|
+
+ Updated FollowChat class to display evaluation statistics
|
|
357
|
+
+ Update OllamaChatConfig to use EOT instead of end for heredoc syntax
|
|
358
|
+
+ Add .keep file to tmp directory
|
|
334
359
|
|
|
335
360
|
## 2024-08-30 v0.1.0
|
|
336
361
|
|
|
@@ -339,18 +364,18 @@
|
|
|
339
364
|
#### Significant Changes
|
|
340
365
|
|
|
341
366
|
* **Document Splitting and Embedding Functionality**: Added `Ollama::Documents` class with methods for adding documents, checking existence, deleting documents, and finding similar documents.
|
|
342
|
-
|
|
343
|
-
|
|
367
|
+
+ Introduced two types of caches: `MemoryCache` and `RedisCache`
|
|
368
|
+
+ Implemented `SemanticSplitter` class to split text into sentences based on semantic similarity
|
|
344
369
|
* **Improved Ollama Chat Client**: Added support for document embeddings and web/file RAG
|
|
345
|
-
|
|
346
|
-
|
|
370
|
+
+ Allowed configuration per yaml file
|
|
371
|
+
+ Parse user input for URLs or files to send images to multimodal models
|
|
347
372
|
* **Redis Docker Service**: Set `REDIS_URL` environment variable to `redis://localhost:9736`
|
|
348
|
-
|
|
373
|
+
+ Added Redis service to `docker-compose.yml`
|
|
349
374
|
* **Status Display and Progress Updates**: Added infobar.label = response.status when available
|
|
350
|
-
|
|
351
|
-
|
|
375
|
+
+ Updated infobar with progress message on each call if total and completed are set
|
|
376
|
+
+ Display error message from response.error if present
|
|
352
377
|
* **Refactored Chat Commands**: Simplified regular expression patterns for `/pop`, `/save`, `/load`, and `/image` commands
|
|
353
|
-
|
|
378
|
+
+ Added whitespace to some command patterns for better readability
|
|
354
379
|
|
|
355
380
|
#### Other Changes
|
|
356
381
|
|
|
@@ -366,13 +391,13 @@
|
|
|
366
391
|
## 2024-08-16 v0.0.1
|
|
367
392
|
|
|
368
393
|
* **New Features**
|
|
369
|
-
|
|
370
|
-
|
|
394
|
+
+ Added missing options parameter to Embed command
|
|
395
|
+
+ Documented new `/api/embed` endpoint
|
|
371
396
|
* **Improvements**
|
|
372
|
-
|
|
397
|
+
+ Improved example in README.md
|
|
373
398
|
* **Code Refactoring**
|
|
374
|
-
|
|
375
|
-
|
|
399
|
+
+ Renamed `client` to `ollama` in client and command specs
|
|
400
|
+
+ Updated expectations to use `ollama` instead of `client`
|
|
376
401
|
|
|
377
402
|
## 2024-08-12 v0.0.0
|
|
378
403
|
|
data/bin/ollama_chat
CHANGED
|
@@ -49,7 +49,7 @@ class OllamaChatConfig
|
|
|
49
49
|
voice:
|
|
50
50
|
enabled: false
|
|
51
51
|
default: Samantha
|
|
52
|
-
list: <%= `say -v
|
|
52
|
+
list: <%= `say -v ? 2>/dev/null`.lines.map { _1[/^(.+?)\s+[a-z]{2}_[a-zA-Z0-9]{2,}/, 1] }.uniq.sort.to_s.force_encoding('ASCII-8BIT') %>
|
|
53
53
|
markdown: true
|
|
54
54
|
stream: true
|
|
55
55
|
embedding:
|
|
@@ -59,6 +59,7 @@ class OllamaChatConfig
|
|
|
59
59
|
options: {}
|
|
60
60
|
# Retrieval prompt template:
|
|
61
61
|
prompt: 'Represent this sentence for searching relevant passages: %s'
|
|
62
|
+
batch_size: 10
|
|
62
63
|
collection: <%= ENV['OLLAMA_CHAT_COLLECTION'] %>
|
|
63
64
|
found_texts_size: 4096
|
|
64
65
|
found_texts_count: null
|
|
@@ -478,9 +479,14 @@ def parse_source(source_io)
|
|
|
478
479
|
end
|
|
479
480
|
end
|
|
480
481
|
|
|
481
|
-
def embed_source(source_io, source)
|
|
482
|
+
def embed_source(source_io, source, count: nil)
|
|
482
483
|
$embedding.on? or return parse_source(source_io)
|
|
483
|
-
|
|
484
|
+
m = "Embedding #{italic { source_io&.content_type }} document #{source.to_s.inspect}."
|
|
485
|
+
if count
|
|
486
|
+
puts '%u. %s' % [ count, m ]
|
|
487
|
+
else
|
|
488
|
+
puts m
|
|
489
|
+
end
|
|
484
490
|
text = parse_source(source_io) or return
|
|
485
491
|
text.downcase!
|
|
486
492
|
splitter_config = $config.embedding.splitter
|
|
@@ -517,7 +523,7 @@ def embed_source(source_io, source)
|
|
|
517
523
|
length: 10
|
|
518
524
|
)
|
|
519
525
|
end
|
|
520
|
-
$documents.add(inputs, source:)
|
|
526
|
+
$documents.add(inputs, source:, batch_size: $config.embedding.batch_size?)
|
|
521
527
|
end
|
|
522
528
|
|
|
523
529
|
def add_image(images, source_io, source)
|
|
@@ -554,7 +560,7 @@ def fetch_source(source, &block)
|
|
|
554
560
|
) do |tmp|
|
|
555
561
|
block.(tmp)
|
|
556
562
|
end
|
|
557
|
-
when %r(\Afile://(
|
|
563
|
+
when %r(\Afile://(/\S*)|\A((?:\.\.|[~.]?)/\S*))
|
|
558
564
|
filename = $~.captures.compact.first
|
|
559
565
|
filename = File.expand_path(filename)
|
|
560
566
|
Utils::Fetcher.read(filename) do |tmp|
|
|
@@ -612,7 +618,7 @@ def parse_content(content, images)
|
|
|
612
618
|
images.clear
|
|
613
619
|
tags = Utils::Tags.new
|
|
614
620
|
|
|
615
|
-
content.scan(%r([.~]?/\S+|https?://\S+|#\S+)).each do |source|
|
|
621
|
+
content.scan(%r((?:\.\.|[.~])?/\S+|https?://\S+|#\S+)).each do |source|
|
|
616
622
|
case source
|
|
617
623
|
when /\A#(\S+)/
|
|
618
624
|
tags.add($1, source:)
|
|
@@ -672,12 +678,18 @@ ensure
|
|
|
672
678
|
end
|
|
673
679
|
|
|
674
680
|
def collection_stats
|
|
681
|
+
list = $documents.collections.sort.map { |c|
|
|
682
|
+
' ' + ($documents.collection == c ? bold { c } : c).to_s
|
|
683
|
+
}.join(?\n)
|
|
675
684
|
puts <<~EOT
|
|
676
|
-
Collection
|
|
685
|
+
Current Collection
|
|
677
686
|
Name: #{bold{$documents.collection}}
|
|
678
687
|
Embedding model: #{bold{$embedding_model}}
|
|
679
688
|
#Embeddings: #{$documents.size}
|
|
689
|
+
#Tags: #{$documents.tags.size}
|
|
680
690
|
Tags: #{$documents.tags}
|
|
691
|
+
List:
|
|
692
|
+
#{list}
|
|
681
693
|
EOT
|
|
682
694
|
end
|
|
683
695
|
|
|
@@ -889,11 +901,13 @@ if $embedding.on?
|
|
|
889
901
|
end
|
|
890
902
|
end
|
|
891
903
|
puts "Collection #{bold{collection}}: Adding #{document_list.size} documents…"
|
|
904
|
+
count = 1
|
|
892
905
|
document_list.each_slice(25) do |docs|
|
|
893
906
|
docs.each do |doc|
|
|
894
907
|
fetch_source(doc) do |doc_io|
|
|
895
|
-
embed_source(doc_io, doc)
|
|
908
|
+
embed_source(doc_io, doc, count:)
|
|
896
909
|
end
|
|
910
|
+
count += 1
|
|
897
911
|
end
|
|
898
912
|
end
|
|
899
913
|
end
|
data/docker-compose.yml
CHANGED
data/lib/ollama/documents.rb
CHANGED
|
@@ -55,8 +55,9 @@ class Ollama::Documents
|
|
|
55
55
|
@cache.prefix = prefix
|
|
56
56
|
end
|
|
57
57
|
|
|
58
|
-
def add(inputs, batch_size:
|
|
58
|
+
def add(inputs, batch_size: nil, source: nil, tags: [])
|
|
59
59
|
inputs = Array(inputs)
|
|
60
|
+
batch_size ||= 10
|
|
60
61
|
tags = Ollama::Utils::Tags.new(tags, source:)
|
|
61
62
|
if source
|
|
62
63
|
tags.add(File.basename(source).gsub(/\?.*/, ''), source:)
|
data/lib/ollama/utils/fetcher.rb
CHANGED
data/lib/ollama/version.rb
CHANGED
data/ollama-ruby.gemspec
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
# -*- encoding: utf-8 -*-
|
|
2
|
-
# stub: ollama-ruby 0.
|
|
2
|
+
# stub: ollama-ruby 0.8.0 ruby lib
|
|
3
3
|
|
|
4
4
|
Gem::Specification.new do |s|
|
|
5
5
|
s.name = "ollama-ruby".freeze
|
|
6
|
-
s.version = "0.
|
|
6
|
+
s.version = "0.8.0".freeze
|
|
7
7
|
|
|
8
8
|
s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version=
|
|
9
9
|
s.require_paths = ["lib".freeze]
|
|
10
10
|
s.authors = ["Florian Frank".freeze]
|
|
11
|
-
s.date = "2024-10-
|
|
11
|
+
s.date = "2024-10-06"
|
|
12
12
|
s.description = "Library that allows interacting with the Ollama API".freeze
|
|
13
13
|
s.email = "flori@ping.de".freeze
|
|
14
14
|
s.executables = ["ollama_console".freeze, "ollama_chat".freeze, "ollama_update".freeze, "ollama_cli".freeze]
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ollama-ruby
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.8.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Florian Frank
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2024-10-
|
|
11
|
+
date: 2024-10-06 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: gem_hadar
|