prompt_manager 0.3.1 → 0.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +10 -1
- data/README.md +123 -32
- data/examples/rgfzf +44 -0
- data/examples/simple.rb +20 -3
- data/examples/using_search_proc.rb +71 -0
- data/lib/prompt_manager/prompt.rb +41 -5
- data/lib/prompt_manager/storage/active_record_adapter.rb +101 -116
- data/lib/prompt_manager/storage/file_system_adapter.rb +2 -2
- data/lib/prompt_manager/version.rb +1 -1
- metadata +19 -4
- data/lib/prompt_manager/storage/sqlite_adapter.rb +0 -156
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 60ae8d4c5f669935906248aa0e5ffc628e6ef0fbcc28eb31c889eb85b200d370
|
4
|
+
data.tar.gz: 44e51b38634667930b5524232cefd63fe0dab3ef334500714361680009186bfd
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 634cc2d1cfdedaf1ca5f5a232334b7e8ccbad25dbb48e669daddf831e43eb45f03367f2921561cdb8ddcf1675ea48f1dcf8cc40defb479aa1eced3bb493442f9
|
7
|
+
data.tar.gz: f04cc92a7ac3c286fb3958eaa44358c402899a121e89ca41aa7bddd30c41bd704bed55dd57775340c622671d0fbf36eef48171ffc3d290f220d6306622486ef3
|
data/CHANGELOG.md
CHANGED
@@ -1,4 +1,13 @@
|
|
1
|
-
## [
|
1
|
+
## [0.4.0] = 2023-12-19
|
2
|
+
- Add "//directives param(s)" with keywords just like the prompt text.
|
3
|
+
|
4
|
+
## [0.3.3] = 2023-12-01
|
5
|
+
- Added example of using the `search_proc` config parameter with the FileSystemAdapter.
|
6
|
+
|
7
|
+
## [0.3.2] = 2023-12-01
|
8
|
+
|
9
|
+
- The ActiveRecordAdapter is passing its unit tests
|
10
|
+
- Dropped the concept of an sqlite3 adapter since active record can be used to access sqlite3 databases as well as the big boys.
|
2
11
|
|
3
12
|
## [0.3.0] = 2023-11-28
|
4
13
|
|
data/README.md
CHANGED
@@ -14,18 +14,24 @@ Manage the parameterized prompts (text) used in generative AI (aka chatGPT, Open
|
|
14
14
|
- [Overview](#overview)
|
15
15
|
- [Generative AI (gen-AI)](#generative-ai-gen-ai)
|
16
16
|
- [What does a keyword look like?](#what-does-a-keyword-look-like)
|
17
|
-
|
18
|
-
- [
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
- [
|
24
|
-
- [
|
25
|
-
|
26
|
-
- [
|
27
|
-
- [
|
28
|
-
|
17
|
+
- [All about directives](#all-about-directives)
|
18
|
+
- [Comments Are Ignored](#comments-are-ignored)
|
19
|
+
- [Storage Adapters](#storage-adapters)
|
20
|
+
- [FileSystemAdapter](#filesystemadapter)
|
21
|
+
- [Configuration](#configuration)
|
22
|
+
- [prompts_dir](#prompts_dir)
|
23
|
+
- [search_proc](#search_proc)
|
24
|
+
- [File Extensions](#file-extensions)
|
25
|
+
- [Example Prompt Text File](#example-prompt-text-file)
|
26
|
+
- [Example Prompt Parameters JSON File](#example-prompt-parameters-json-file)
|
27
|
+
- [Extra Functionality](#extra-functionality)
|
28
|
+
- [ActiveRecordAdapter](#activerecordadapter)
|
29
|
+
- [Configuration](#configuration-1)
|
30
|
+
- [model](#model)
|
31
|
+
- [id_column](#id_column)
|
32
|
+
- [text_column](#text_column)
|
33
|
+
- [parameters_column](#parameters_column)
|
34
|
+
- [Other Potential Storage Adapters](#other-potential-storage-adapters)
|
29
35
|
- [Development](#development)
|
30
36
|
- [Contributing](#contributing)
|
31
37
|
- [License](#license)
|
@@ -46,8 +52,12 @@ If bundler is not being used to manage dependencies, install the gem by executin
|
|
46
52
|
|
47
53
|
See [examples/simple.rb](examples/simple.rb)
|
48
54
|
|
55
|
+
See also [examples/using_search_proc.rb](examples/using_search_proc.rb)
|
56
|
+
|
49
57
|
## Overview
|
50
58
|
|
59
|
+
The `prompt_manager` gem provides functionality to manage prompts that have keywords and directives for use with generative AI processes.
|
60
|
+
|
51
61
|
### Generative AI (gen-AI)
|
52
62
|
|
53
63
|
Gen-AI deals with the conversion (some would say execution) of a human natural language text (the "prompt") into somthing else using what are known as large language models (LLM) such as those available from OpenAI. A parameterized prompt is one in which there are embedded keywords (parameters) which are place holders for other text to be inserted into the prompt.
|
@@ -60,13 +70,57 @@ The current hard-coded REGEX for a [KEYWORD] identifies any all [UPPERCASE_TEXT]
|
|
60
70
|
|
61
71
|
This is just the initial convention adopted by prompt_manager. It is intended that this REGEX be configurable so that the prompt_manager can be used with other conventions.
|
62
72
|
|
63
|
-
|
73
|
+
#### All about directives
|
74
|
+
|
75
|
+
A directive is a line in the prompt text that starts with the two characters '//' - slash slash - just like in the old days of IBM JCL - Job Control Language. A prompt can have zero or more directives. Directives can have parameters and can make use of keywords.
|
76
|
+
|
77
|
+
The `prompt_manager` only collections directives. It extracts keywords from directive lines and provides the substitution of those keywords with other text just like it does for the prompt. Remember a directive is part of the prompt.
|
78
|
+
|
79
|
+
Here is an example problem with comments, directives and keywords:
|
80
|
+
|
81
|
+
```
|
82
|
+
# prompts/sing_a_song.txt
|
83
|
+
# Desc: Has the computer sing a song
|
84
|
+
|
85
|
+
//TextToSpeech [LANGUAGE] [VOICE NAME]
|
86
|
+
|
87
|
+
Say the lyrics to the song [SONG NAME]. Please provide only the lyrics without commentary.
|
88
|
+
|
89
|
+
__END__
|
90
|
+
Computers will never replace Frank Sinatra
|
91
|
+
```
|
92
|
+
|
93
|
+
Getting directives from a prompt is as easy as getting the kewyords:
|
94
|
+
|
95
|
+
```ruby
|
96
|
+
prompt = PromptManager::Prompt.new(...)
|
97
|
+
prompt.keywords #=> an Array
|
98
|
+
prompt.directives #=> a Hash
|
99
|
+
|
100
|
+
# to_s builds the prompt by substituting
|
101
|
+
# values for keywords amd removing comments.
|
102
|
+
# The resulting text contains directives and
|
103
|
+
# prompt text ready for the LLM process.
|
104
|
+
puts prompt.to_s
|
105
|
+
```
|
106
|
+
|
107
|
+
The Hash returned by the `prompt.directives` method has as its key the directive name (without the double-slash). Its also case-sensitive. So any typo made in editing the prompt with regard to the case of the directive's name will impact the backend processing of the prompt.
|
108
|
+
|
109
|
+
The value for each key is a String exactly as entered in the prompt file.
|
110
|
+
|
111
|
+
#### Comments Are Ignored
|
112
|
+
|
113
|
+
The `prompt_manager` gem ignores comments. A line that begins with the '#' - pound (aka hash) character - is a line comment. Any lines that follow a line that is '__END__ at the end of a file are considered comments. Basically the '__END__' the end of the file. Nothing is process following that line.
|
114
|
+
|
115
|
+
The gem also ignores blank lines.
|
116
|
+
|
117
|
+
## Storage Adapters
|
64
118
|
|
65
119
|
A storage adapter is a class instance that ties the `PromptManager::Prompt` class to a storage facility that holds the actual prompts. Currently there are 3 storage adapters planned for implementation.
|
66
120
|
|
67
121
|
The `PromptManager::Prompt` to support a small set of methods. A storage adapter can provide "extra" class or instance methods that can be used through the Prompt class. See the `test/prompt_manager/prompt_test.rb` for guidance on creating a new storage adapter.
|
68
122
|
|
69
|
-
|
123
|
+
### FileSystemAdapter
|
70
124
|
|
71
125
|
This is the first storage adapter developed. It saves prompts as text files within the file system inside a designated `prompts_dir` (directory) such as `~/.prompts` or where it makes the most sense to you. Another example would be to have your directory on a shared file system so that others can use the same prompts.
|
72
126
|
|
@@ -74,16 +128,16 @@ The `prompt ID` is the basename of the text file. For example `todo.txt` is the
|
|
74
128
|
|
75
129
|
The parameters for the `todo` prompt ID are saved in the same directory as `todo.txt` in a JSON file named `todo.json` (also in the examples directory.)
|
76
130
|
|
77
|
-
|
131
|
+
#### Configuration
|
78
132
|
|
79
133
|
Use a `config` block to establish the configuration for the class.
|
80
134
|
|
81
135
|
```ruby
|
82
136
|
PromptManager::Storage::FileSystemAdapter.config do |o|
|
83
|
-
o.prompts_dir
|
84
|
-
o.search_proc
|
85
|
-
o.prompt_extension
|
86
|
-
o.params_extension
|
137
|
+
o.prompts_dir = "path/to/prompts_directory"
|
138
|
+
o.search_proc = nil # default
|
139
|
+
o.prompt_extension = '.txt' # default
|
140
|
+
o.params_extension = '.json' # default
|
87
141
|
end
|
88
142
|
```
|
89
143
|
|
@@ -98,19 +152,21 @@ PromptManager::Prompt
|
|
98
152
|
end.new
|
99
153
|
```
|
100
154
|
|
101
|
-
|
155
|
+
##### prompts_dir
|
102
156
|
|
103
157
|
This is either a `String` or a `Pathname` object. All file paths are maintained in the class as `Pathname` objects. If you provide a `String` it will be converted. Relative paths will be converted to absolute paths.
|
104
158
|
|
105
159
|
An `ArgumentError` will be raised when `prompts_dir` does not exist or if it is not a directory.
|
106
160
|
|
107
|
-
|
161
|
+
##### search_proc
|
162
|
+
|
163
|
+
The default for `search_proc` is nil which means that the search will be preformed by a default `search` method which is basically reading all the prompt files to see which ones contain the search term. It will return an Array of prompt IDs for each prompt file found that contains the search term. Its up to the application to select which returned prompt ID to use.
|
108
164
|
|
109
|
-
|
165
|
+
There are faster ways to search and select files. For example there are specialized search and selection utilities that are available for the command line. The `examples` directory contains a `bash` script named `rgfzf` that uses `rg` (aka `ripgrep`) to do the searching and `fzf` to do the selecting.
|
110
166
|
|
111
|
-
|
167
|
+
See [examples/using_search_proc.rb](examples/using_search_proc.rb)
|
112
168
|
|
113
|
-
|
169
|
+
##### File Extensions
|
114
170
|
|
115
171
|
These two configuration options are `String` objects that must start with a period "." utherwise an `ArgumentError` will be raised.
|
116
172
|
|
@@ -121,7 +177,7 @@ Currently the `FileSystemAdapter` only supports a JSON serializer for its parame
|
|
121
177
|
|
122
178
|
They exist so that there is a platform on to which other storage adapters can be built or serializers added. This is not currently on the roadmap.
|
123
179
|
|
124
|
-
|
180
|
+
#### Example Prompt Text File
|
125
181
|
|
126
182
|
```text
|
127
183
|
# ~/.prompts/joke.txt
|
@@ -132,7 +188,7 @@ Tell me a few [KIND] jokes about [SUBJECT]
|
|
132
188
|
|
133
189
|
Note the command lines at the top. This is a convention I use. It is not part of the software. I find it helpful in documenting the prompt.
|
134
190
|
|
135
|
-
|
191
|
+
#### Example Prompt Parameters JSON File
|
136
192
|
|
137
193
|
```json
|
138
194
|
{
|
@@ -151,7 +207,7 @@ Note the command lines at the top. This is a convention I use. It is not part
|
|
151
207
|
|
152
208
|
The last value in the keyword's Array is the most recent value used for that keyword. This is a functionality established since v0.3.0. Its purpose is to provide a history of values from which a user can select to repeat a previous value or to select ta previous value and edit it into something new.
|
153
209
|
|
154
|
-
|
210
|
+
#### Extra Functionality
|
155
211
|
|
156
212
|
The `FileSystemAdapter` adds two new methods for use by the `Prompt` class:
|
157
213
|
* list - returns an Array of prompt IDs
|
@@ -160,15 +216,50 @@ The `FileSystemAdapter` adds two new methods for use by the `Prompt` class:
|
|
160
216
|
Use the `path(prompt_id)` form against the `Prompt` class
|
161
217
|
Use `prompt.path` when you have an instance of a `Prompt`
|
162
218
|
|
163
|
-
|
219
|
+
### ActiveRecordAdapter
|
220
|
+
|
221
|
+
The `ActiveRecordAdapter` assumes that there is a database already configured by the application program that is requiring `prompt_manager` which has a model that contains prompt content. This model must have at least three columns which contain content for:
|
222
|
+
|
223
|
+
- a prompt ID
|
224
|
+
- prompt text
|
225
|
+
- prompt parameters
|
226
|
+
|
227
|
+
The model and the columns for these three elements can have any name. Those names are provided to the `ActiveRecordAdapter` in its config block.
|
228
|
+
|
229
|
+
|
230
|
+
#### Configuration
|
231
|
+
|
232
|
+
Use a `config` block to establish the configuration for the class.
|
233
|
+
|
234
|
+
The `PromptManager::Prompt` class expects an instance of a storage adapter class. By convention storage adapter class config methods will return `self` so that a simple `new` after the config will establish the instance.
|
235
|
+
|
236
|
+
```ruby
|
237
|
+
PromptManager::Prompt
|
238
|
+
.storage_adapter =
|
239
|
+
PromptManager::Storage::ActiveRecordAdapter.config do |config|
|
240
|
+
config.model = DbPromptModel # any ActiveRecord::Base model
|
241
|
+
config.id_column = :prompt_name
|
242
|
+
config.text_column = :prompt_text
|
243
|
+
config.parameters_column = :prompt_params
|
244
|
+
end.new # adapters an instances of the adapter class
|
245
|
+
```
|
246
|
+
|
247
|
+
##### model
|
248
|
+
The `model` configuration parameter is the actual class name of the `ActiveRecord::Base` or `ApplicationRecord` (if you are using a rails application) that contains the content used for prompts.
|
249
|
+
|
250
|
+
##### id_column
|
251
|
+
The `id_column` contains the name of the column that contains the "prompt ID" content. It can be either a `String` or `Symbol` value.
|
252
|
+
|
253
|
+
##### text_column
|
254
|
+
The `text_column` contains name of the column that contains the actual raw text of the prompt. This raw text can include the keywords which will be replaced by values from the parameters Hash. The column name value can be either a `String` or a `Symbol`.
|
164
255
|
|
165
|
-
|
256
|
+
##### parameters_column
|
257
|
+
The `parameters_column` contains the name of the column that contains the parameters used to replace keywords in the prompt text. This column in the database model is expected to be serialized. The `ActiveRecordAdapter` currently has a kludge bit of code that assumes that the serialization is done with JSON. The value of the parameters_column can be either a `String` or a `Symbol`.
|
166
258
|
|
167
|
-
|
259
|
+
TODO: fix the kludge so that any serialization can be used.
|
168
260
|
|
169
|
-
TODO: Still looking for requirements on how to integrate with an existing `rails` app. Looking for ideas.
|
170
261
|
|
171
|
-
|
262
|
+
### Other Potential Storage Adapters
|
172
263
|
|
173
264
|
There are many possibilities to example this plugin concept of the storage adapter. Here are some for consideration:
|
174
265
|
|
data/examples/rgfzf
ADDED
@@ -0,0 +1,44 @@
|
|
1
|
+
#!/usr/bin/env bash
|
2
|
+
# Refactored examples/rgfzf to take search term and directory path as parameters
|
3
|
+
|
4
|
+
search_term="$1"
|
5
|
+
directory_path="${2:-$(pwd)}" # Use given directory path, or current working directory if not provided
|
6
|
+
|
7
|
+
# Verify that a search term is provided
|
8
|
+
if [[ -z "$search_term" ]]; then
|
9
|
+
echo "Usage: $0 search_term [directory_path]"
|
10
|
+
exit 1
|
11
|
+
fi
|
12
|
+
|
13
|
+
# Ensure ripgrep (rg) and fzf are installed
|
14
|
+
if ! command -v rg &> /dev/null || ! command -v fzf &> /dev/null; then
|
15
|
+
echo "Please ensure ripgrep and fzf are installed before running this script."
|
16
|
+
exit 1
|
17
|
+
fi
|
18
|
+
|
19
|
+
# Perform the file search using ripgrep and selection using fzf
|
20
|
+
selected_file=$(
|
21
|
+
rg --files-with-matches \
|
22
|
+
--no-messages \
|
23
|
+
--smart-case "$search_term" "$directory_path" |
|
24
|
+
sed "s#^$directory_path/##" | # Strip out the directory path
|
25
|
+
fzf --ansi \
|
26
|
+
--color "hl:-1:underline,hl+:-1:underline:reverse" \
|
27
|
+
--preview "cat $directory_path/{}" \
|
28
|
+
--preview-window "up,60%,border-bottom" \
|
29
|
+
--no-multi \
|
30
|
+
--exit-0 \
|
31
|
+
--query "$search_term"
|
32
|
+
)
|
33
|
+
|
34
|
+
|
35
|
+
# If no file was selected, exit the script
|
36
|
+
if [[ -z "$selected_file" ]]; then
|
37
|
+
# echo "No file selected."
|
38
|
+
exit 0
|
39
|
+
fi
|
40
|
+
|
41
|
+
# Remove the file extension
|
42
|
+
prompt_id=$(echo "$selected_file" | sed "s/\.[^.]*$//")
|
43
|
+
|
44
|
+
echo "$prompt_id"
|
data/examples/simple.rb
CHANGED
@@ -9,9 +9,6 @@
|
|
9
9
|
## By: Dewayne VanHoozer (dvanhoozer@gmail.com)
|
10
10
|
##
|
11
11
|
#
|
12
|
-
# TODO: Add `list` to get an Array of prompt IDs
|
13
|
-
# TODO: Add `path` to get a path to the prompt file
|
14
|
-
#
|
15
12
|
|
16
13
|
require 'prompt_manager'
|
17
14
|
require 'prompt_manager/storage/file_system_adapter'
|
@@ -140,3 +137,23 @@ puts <<~EOS
|
|
140
137
|
EOS
|
141
138
|
|
142
139
|
puts PromptManager::Prompt.path('toy/8-ball')
|
140
|
+
|
141
|
+
puts
|
142
|
+
|
143
|
+
puts "Default Search for Prompts"
|
144
|
+
puts "=========================="
|
145
|
+
|
146
|
+
print "Search Proc Class: "
|
147
|
+
puts PromptManager::Prompt.storage_adapter.search_proc.class
|
148
|
+
|
149
|
+
search_term = "txt" # some comment lines show the file name example: todo.txt
|
150
|
+
|
151
|
+
puts "Search for '#{search_term}' ..."
|
152
|
+
|
153
|
+
prompt_ids = PromptManager::Prompt.search search_term
|
154
|
+
|
155
|
+
# NOTE: prompt+ids is an Array of prompt IDs even if there is only one entry.
|
156
|
+
# or and empty array if there are no prompts have the search term.
|
157
|
+
|
158
|
+
puts "Found: #{prompt_ids}"
|
159
|
+
|
@@ -0,0 +1,71 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
# encoding: utf-8
|
3
|
+
# frozen_string_literal: true
|
4
|
+
# warn_indent: true
|
5
|
+
##########################################################
|
6
|
+
###
|
7
|
+
## File: using_search_proc.rb
|
8
|
+
## Desc: Simple demo of the PromptManager and FileStorageAdapter
|
9
|
+
## By: Dewayne VanHoozer (dvanhoozer@gmail.com)
|
10
|
+
##
|
11
|
+
#
|
12
|
+
|
13
|
+
require 'debug_me'
|
14
|
+
include DebugMe
|
15
|
+
|
16
|
+
require 'prompt_manager'
|
17
|
+
require 'prompt_manager/storage/file_system_adapter'
|
18
|
+
|
19
|
+
require 'amazing_print'
|
20
|
+
require 'pathname'
|
21
|
+
|
22
|
+
HERE = Pathname.new( __dir__ )
|
23
|
+
PROMPTS_DIR = HERE + "prompts_dir"
|
24
|
+
SEARCH_SCRIPT = HERE + 'rgfzf' # a bash script using rg and fzf
|
25
|
+
|
26
|
+
######################################################
|
27
|
+
# Main
|
28
|
+
|
29
|
+
at_exit do
|
30
|
+
puts
|
31
|
+
puts "Done."
|
32
|
+
puts
|
33
|
+
end
|
34
|
+
|
35
|
+
# Configure the Storage Adapter to use
|
36
|
+
PromptManager::Storage::FileSystemAdapter.config do |config|
|
37
|
+
config.prompts_dir = PROMPTS_DIR
|
38
|
+
config.search_proc = ->(q) {`#{SEARCH_SCRIPT} #{q} #{PROMPTS_DIR}`} # default
|
39
|
+
# config.prompt_extension = '.txt' # default
|
40
|
+
# config.parms+_extension = '.json' # default
|
41
|
+
end
|
42
|
+
|
43
|
+
PromptManager::Prompt.storage_adapter = PromptManager::Storage::FileSystemAdapter.new
|
44
|
+
|
45
|
+
|
46
|
+
|
47
|
+
puts "Using Custom Search Proc"
|
48
|
+
puts "========================"
|
49
|
+
|
50
|
+
print "Search Proc Class: "
|
51
|
+
puts PromptManager::Prompt.storage_adapter.search_proc.class
|
52
|
+
|
53
|
+
search_term = "txt" # some comment lines show the file name example: todo.txt
|
54
|
+
|
55
|
+
puts "Search for '#{search_term}' ..."
|
56
|
+
|
57
|
+
prompt_id = PromptManager::Prompt.search search_term
|
58
|
+
|
59
|
+
# NOTE: the search proc uses fzf as a selection tool. In this
|
60
|
+
# case only one selected prompt ID that matches the search
|
61
|
+
# term will be returned.
|
62
|
+
|
63
|
+
puts "Found: '#{prompt_id}' which is a #{prompt_id.class}. empty? #{prompt_id.empty?}"
|
64
|
+
|
65
|
+
puts <<~EOS
|
66
|
+
|
67
|
+
When the rgfzf bash script does not find a prompt ID or if the
|
68
|
+
ESC key is pressed, the prompt ID that is returned will be an empty String.
|
69
|
+
|
70
|
+
EOS
|
71
|
+
|
@@ -5,9 +5,20 @@
|
|
5
5
|
# as well as building prompts with replacement of parameterized values and
|
6
6
|
# comment removal. It communicates with a storage system through a storage
|
7
7
|
# adapter.
|
8
|
+
#
|
9
|
+
# Directives can be anything required by the backend
|
10
|
+
# prompt processing functionality. Directives are
|
11
|
+
# available along with the prompt text w/o comments.
|
12
|
+
# Keywords can be used in the directives.
|
13
|
+
#
|
14
|
+
# It is expected that the backend prompt processes will
|
15
|
+
# act on and remove the directives before passing the
|
16
|
+
# prompt text on through the LLM.
|
8
17
|
|
9
18
|
class PromptManager::Prompt
|
10
|
-
|
19
|
+
COMMENT_SIGNAL = '#' # lines beginning with this are a comment
|
20
|
+
DIRECTIVE_SIGNAL = '//' # Like the old IBM JCL
|
21
|
+
PARAMETER_REGEX = /(\[[A-Z _|]+\])/
|
11
22
|
@storage_adapter = nil
|
12
23
|
|
13
24
|
class << self
|
@@ -47,8 +58,8 @@ class PromptManager::Prompt
|
|
47
58
|
|
48
59
|
# SMELL: Does the db (aka storage adapter) really need
|
49
60
|
# to be accessible by the main program?
|
50
|
-
attr_accessor :db, :id, :text, :parameters
|
51
|
-
|
61
|
+
attr_accessor :db, :id, :text, :parameters
|
62
|
+
|
52
63
|
|
53
64
|
# Retrieve the specific prompt ID from the Storage system.
|
54
65
|
def initialize(
|
@@ -64,9 +75,12 @@ class PromptManager::Prompt
|
|
64
75
|
@record = db.get(id: id)
|
65
76
|
@text = @record[:text]
|
66
77
|
@parameters = @record[:parameters]
|
67
|
-
@keywords = []
|
78
|
+
@keywords = [] # Array of String
|
79
|
+
@directives = {} # Hash with directive as key, parameters as value
|
68
80
|
|
69
81
|
update_keywords
|
82
|
+
update_directives
|
83
|
+
|
70
84
|
build
|
71
85
|
end
|
72
86
|
|
@@ -120,6 +134,12 @@ class PromptManager::Prompt
|
|
120
134
|
update_keywords
|
121
135
|
end
|
122
136
|
|
137
|
+
|
138
|
+
def directives
|
139
|
+
update_directives
|
140
|
+
end
|
141
|
+
|
142
|
+
|
123
143
|
######################################
|
124
144
|
private
|
125
145
|
|
@@ -133,10 +153,26 @@ class PromptManager::Prompt
|
|
133
153
|
end
|
134
154
|
|
135
155
|
|
156
|
+
def update_directives
|
157
|
+
@text.split("\n").each do |a_line|
|
158
|
+
line = a_line.strip
|
159
|
+
next unless line.start_with?(DIRECTIVE_SIGNAL)
|
160
|
+
|
161
|
+
parts = line.split(' ')
|
162
|
+
directive = parts.shift()[DIRECTIVE_SIGNAL.length..] # drop the directive signal
|
163
|
+
@directives[directive] = parts.join(' ')
|
164
|
+
end
|
165
|
+
|
166
|
+
@directives
|
167
|
+
end
|
168
|
+
|
169
|
+
|
136
170
|
def remove_comments
|
137
171
|
lines = @prompt
|
138
172
|
.split("\n")
|
139
|
-
.reject{|a_line|
|
173
|
+
.reject{|a_line|
|
174
|
+
a_line.strip.start_with?(COMMENT_SIGNAL)
|
175
|
+
}
|
140
176
|
|
141
177
|
# Remove empty lines at the start of the prompt
|
142
178
|
#
|
@@ -1,161 +1,146 @@
|
|
1
1
|
# prompt_manager/lib/prompt_manager/storage/active_record_adapter.rb
|
2
2
|
|
3
|
-
|
4
|
-
|
5
|
-
#
|
6
|
-
#
|
7
|
-
# with the things that the Prompt class wants.
|
3
|
+
# This class acts as an adapter for interacting with an ActiveRecord model
|
4
|
+
# to manage storage operations for PromptManager::Prompt instances. It defines
|
5
|
+
# methods that allow for saving, searching, retrieving by ID, and deleting
|
6
|
+
# prompts.
|
8
7
|
|
9
8
|
class PromptManager::Storage::ActiveRecordAdapter
|
10
|
-
|
9
|
+
|
10
|
+
class << self
|
11
|
+
attr_accessor :model,
|
12
|
+
:id_column,
|
13
|
+
:text_column,
|
14
|
+
:parameters_column
|
11
15
|
|
12
|
-
|
13
|
-
|
14
|
-
|
16
|
+
def config
|
17
|
+
if block_given?
|
18
|
+
yield self
|
19
|
+
validate_configuration
|
20
|
+
else
|
21
|
+
raise ArgumentError, "No block given to config"
|
22
|
+
end
|
23
|
+
|
24
|
+
self
|
25
|
+
end
|
15
26
|
|
16
27
|
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
28
|
+
def validate_configuration
|
29
|
+
validate_model
|
30
|
+
validate_columns
|
31
|
+
end
|
21
32
|
|
22
33
|
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
34
|
+
def validate_model
|
35
|
+
raise ArgumentError, "AR Model not set" unless model
|
36
|
+
raise ArgumentError, "AR Model is not an ActiveRecord model" unless model < ActiveRecord::Base
|
37
|
+
end
|
27
38
|
|
28
39
|
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
40
|
+
def validate_columns
|
41
|
+
columns = model.column_names # Array of Strings
|
42
|
+
[id_column, text_column, parameters_column].each do |column|
|
43
|
+
raise ArgumentError, "#{column} is not a valid column for model #{model}" unless columns.include?(column.to_s)
|
44
|
+
end
|
45
|
+
end
|
35
46
|
|
36
47
|
|
37
|
-
def delete(prompt_id)
|
38
|
-
prompt = find_prompt(prompt_id)
|
39
|
-
prompt.destroy
|
40
|
-
end
|
41
48
|
|
49
|
+
def method_missing(method_name, *args, &block)
|
50
|
+
if model.respond_to?(method_name)
|
51
|
+
model.send(method_name, *args, &block)
|
52
|
+
else
|
53
|
+
super
|
54
|
+
end
|
55
|
+
end
|
42
56
|
|
43
|
-
def search(for_what)
|
44
|
-
# TODO: search through all prompts. Return an Array of
|
45
|
-
# prompt_id where the text of the prompt contains
|
46
|
-
# for_what is being searched.
|
47
57
|
|
48
|
-
|
58
|
+
def respond_to_missing?(method_name, include_private = false)
|
59
|
+
model.respond_to?(method_name, include_private) || super
|
60
|
+
end
|
49
61
|
end
|
62
|
+
|
50
63
|
|
64
|
+
##############################################
|
65
|
+
attr_accessor :record
|
51
66
|
|
52
|
-
class << self
|
53
|
-
def config
|
54
|
-
# TODO: establish a connection to the database
|
55
|
-
# maybe define the prompts table and its
|
56
|
-
# columns of interest.
|
57
|
-
end
|
58
|
-
end
|
59
67
|
|
60
|
-
|
61
|
-
|
68
|
+
# Avoid code littered with self.class prefixes ...
|
69
|
+
def model = self.class.model
|
70
|
+
def id_column = self.class.id_column
|
71
|
+
def text_column = self.class.text_column
|
72
|
+
def parameters_column = self.class.parameters_column
|
73
|
+
|
62
74
|
|
63
|
-
def
|
64
|
-
|
75
|
+
def initialize
|
76
|
+
self.class.send(:validate_configuration) # send gets around private designations of a method
|
77
|
+
@record = model.first
|
65
78
|
end
|
66
|
-
end
|
67
79
|
|
68
80
|
|
69
|
-
|
81
|
+
def get(id:)
|
82
|
+
@record = model.find_by(id_column => id)
|
83
|
+
raise ArgumentError, "Prompt not found with id: #{id}" unless @record
|
70
84
|
|
71
|
-
#
|
85
|
+
# kludge? testing showed that parameters was being
|
86
|
+
# returned as a String. Did serialization fail or is
|
87
|
+
# there something else going on?
|
88
|
+
# FIXME: expected the parameters_column to be a HAsh after de-serialization
|
89
|
+
parameters = @record[parameters_column]
|
72
90
|
|
73
|
-
|
91
|
+
if parameters.is_a? String
|
92
|
+
parameters = JSON.parse parameters
|
93
|
+
end
|
74
94
|
|
75
|
-
|
76
|
-
|
77
|
-
|
95
|
+
{
|
96
|
+
id: id, # same as the id_column
|
97
|
+
text: @record[text_column],
|
98
|
+
parameters: parameters
|
99
|
+
}
|
100
|
+
end
|
78
101
|
|
79
|
-
# Define models for ActiveRecord
|
80
|
-
class Prompt < ActiveRecord::Base
|
81
|
-
validates :unique_id, presence: true
|
82
|
-
validates :text, presence: true
|
83
|
-
end
|
84
102
|
|
85
|
-
|
86
|
-
|
87
|
-
validates :key, presence: true
|
88
|
-
serialize :value
|
89
|
-
end
|
103
|
+
def save(id:, text: "", parameters: {})
|
104
|
+
@record = model.find_or_initialize_by(id_column => id)
|
90
105
|
|
91
|
-
|
92
|
-
|
93
|
-
|
94
|
-
|
95
|
-
end
|
106
|
+
@record[text_column] = text
|
107
|
+
@record[parameters_column] = parameters
|
108
|
+
@record.save!
|
109
|
+
end
|
96
110
|
|
97
|
-
def get(id:)
|
98
|
-
prompt = Prompt.find_by(unique_id: id)
|
99
|
-
return nil unless prompt
|
100
111
|
|
101
|
-
|
112
|
+
def delete(id:)
|
113
|
+
@record = model.find_by(id_column => id)
|
114
|
+
@record&.destroy
|
115
|
+
end
|
102
116
|
|
103
|
-
{
|
104
|
-
id: prompt.unique_id,
|
105
|
-
text: prompt.text,
|
106
|
-
parameters: parameters.transform_values(&:value)
|
107
|
-
}
|
108
|
-
end
|
109
117
|
|
110
|
-
|
111
|
-
|
112
|
-
|
113
|
-
|
118
|
+
|
119
|
+
def list(*)
|
120
|
+
model.all.pluck(id_column)
|
121
|
+
end
|
114
122
|
|
115
|
-
parameters.each do |key, value|
|
116
|
-
parameter = PromptParameter.find_or_initialize_by(prompt: prompt, key: key)
|
117
|
-
parameter.value = value
|
118
|
-
parameter.save!
|
119
|
-
end
|
120
|
-
end
|
121
123
|
|
122
|
-
|
123
|
-
|
124
|
-
|
125
|
-
|
126
|
-
prompt.prompt_parameters.destroy_all
|
127
|
-
prompt.destroy
|
128
|
-
end
|
124
|
+
def search(for_what)
|
125
|
+
model.where("#{text_column} LIKE ?", "%#{for_what}%").pluck(id_column)
|
126
|
+
end
|
129
127
|
|
130
|
-
def search(for_what)
|
131
|
-
query = '%' + for_what.downcase + '%'
|
132
|
-
Prompt.where('LOWER(text) LIKE ?', query).pluck(:unique_id)
|
133
|
-
end
|
134
128
|
|
135
|
-
|
136
|
-
|
137
|
-
end
|
129
|
+
##############################################
|
130
|
+
private
|
138
131
|
|
139
|
-
private
|
140
132
|
|
141
|
-
|
142
|
-
|
143
|
-
|
144
|
-
|
145
|
-
|
146
|
-
database: 'prompts.db'
|
147
|
-
)
|
148
|
-
end
|
133
|
+
def method_missing(method_name, *args, &block)
|
134
|
+
if @record.respond_to?(method_name)
|
135
|
+
model.send(method_name, args.first, &block)
|
136
|
+
else
|
137
|
+
super
|
149
138
|
end
|
150
139
|
end
|
151
|
-
end
|
152
|
-
|
153
|
-
# After this, you would need to create a database migration to generate the required tables.
|
154
|
-
# Additionally, you have to establish an ActiveRecord connection before using this adapter,
|
155
|
-
# typically in the environment setup of your application.
|
156
140
|
|
157
|
-
# Keep in mind you need to create migrations for both the Prompt and PromptParameter models,
|
158
|
-
# and manage the database schema using ActiveRecord migrations. This adapter assumes that the
|
159
|
-
# database structure is already in place and follows the schema inferred by the models in the adapter.
|
160
141
|
|
142
|
+
def respond_to_missing?(method_name, include_private = false)
|
143
|
+
model.respond_to?(method_name, include_private) || super
|
144
|
+
end
|
145
|
+
end
|
161
146
|
|
@@ -211,8 +211,8 @@ class PromptManager::Storage::FileSystemAdapter
|
|
211
211
|
def search(for_what)
|
212
212
|
search_term = for_what.downcase
|
213
213
|
|
214
|
-
if
|
215
|
-
|
214
|
+
if search_proc.is_a? Proc
|
215
|
+
search_proc.call(search_term)
|
216
216
|
else
|
217
217
|
search_prompts(search_term)
|
218
218
|
end
|
metadata
CHANGED
@@ -1,15 +1,29 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: prompt_manager
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Dewayne VanHoozer
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2023-
|
11
|
+
date: 2023-12-19 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: activerecord
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '0'
|
20
|
+
type: :development
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '0'
|
13
27
|
- !ruby/object:Gem::Dependency
|
14
28
|
name: amazing_print
|
15
29
|
requirement: !ruby/object:Gem::Requirement
|
@@ -84,13 +98,14 @@ files:
|
|
84
98
|
- examples/prompts_dir/todo.json
|
85
99
|
- examples/prompts_dir/todo.txt
|
86
100
|
- examples/prompts_dir/toy/8-ball.txt
|
101
|
+
- examples/rgfzf
|
87
102
|
- examples/simple.rb
|
103
|
+
- examples/using_search_proc.rb
|
88
104
|
- lib/prompt_manager.rb
|
89
105
|
- lib/prompt_manager/prompt.rb
|
90
106
|
- lib/prompt_manager/storage.rb
|
91
107
|
- lib/prompt_manager/storage/active_record_adapter.rb
|
92
108
|
- lib/prompt_manager/storage/file_system_adapter.rb
|
93
|
-
- lib/prompt_manager/storage/sqlite_adapter.rb
|
94
109
|
- lib/prompt_manager/version.rb
|
95
110
|
homepage: https://github.com/MadBomber/prompt_manager
|
96
111
|
licenses:
|
@@ -115,7 +130,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
115
130
|
- !ruby/object:Gem::Version
|
116
131
|
version: '0'
|
117
132
|
requirements: []
|
118
|
-
rubygems_version: 3.
|
133
|
+
rubygems_version: 3.5.1
|
119
134
|
signing_key:
|
120
135
|
specification_version: 4
|
121
136
|
summary: Manage prompts for use with gen-AI processes
|
@@ -1,156 +0,0 @@
|
|
1
|
-
# prompt_manager/lib/prompt_manager/storage/sqlite_adapter.rb
|
2
|
-
|
3
|
-
require 'json'
|
4
|
-
require 'sqlite3'
|
5
|
-
|
6
|
-
class PromptManager::Storage::SqliteAdapter
|
7
|
-
def initialize(db_path = 'prompts.db')
|
8
|
-
@db = SQLite3::Database.new(db_path)
|
9
|
-
create_tables
|
10
|
-
end
|
11
|
-
|
12
|
-
def save_prompt(prompt_id, text, params)
|
13
|
-
@db.execute("INSERT INTO prompts (id, text, params) VALUES (?, ?, ?)",
|
14
|
-
[prompt_id, text, params.to_json])
|
15
|
-
end
|
16
|
-
|
17
|
-
def prompt_text(prompt_id)
|
18
|
-
result = @db.get_first_value("SELECT text FROM prompts WHERE id = ?", [prompt_id])
|
19
|
-
raise 'Prompt not found' if result.nil?
|
20
|
-
result
|
21
|
-
end
|
22
|
-
|
23
|
-
def parameter_values(prompt_id)
|
24
|
-
json_content = @db.get_first_value("SELECT params FROM prompts WHERE id = ?", [prompt_id])
|
25
|
-
raise 'Parameters not found' if json_content.nil?
|
26
|
-
JSON.parse(json_content, symbolize_names: true)
|
27
|
-
end
|
28
|
-
|
29
|
-
def save(prompt_id, prompt_text, parameter_values)
|
30
|
-
@db.execute(
|
31
|
-
'REPLACE INTO prompts (id, text, params) VALUES (?, ?, ?)',
|
32
|
-
[prompt_id, prompt_text, parameter_values.to_json]
|
33
|
-
)
|
34
|
-
end
|
35
|
-
|
36
|
-
def delete(prompt_id)
|
37
|
-
@db.execute('DELETE FROM prompts WHERE id = ?', [prompt_id])
|
38
|
-
end
|
39
|
-
|
40
|
-
|
41
|
-
def search(for_what)
|
42
|
-
# TODO: search through all prompts. Return an Array of
|
43
|
-
# prompt_id where the text of the prompt contains
|
44
|
-
# for_what is being searched.
|
45
|
-
|
46
|
-
[]
|
47
|
-
end
|
48
|
-
|
49
|
-
###################################################
|
50
|
-
private
|
51
|
-
|
52
|
-
def create_tables
|
53
|
-
@db.execute <<-SQL
|
54
|
-
CREATE TABLE IF NOT EXISTS prompts (
|
55
|
-
id TEXT PRIMARY KEY,
|
56
|
-
text TEXT NOT NULL,
|
57
|
-
params TEXT NOT NULL
|
58
|
-
);
|
59
|
-
SQL
|
60
|
-
end
|
61
|
-
end
|
62
|
-
|
63
|
-
|
64
|
-
__END__
|
65
|
-
|
66
|
-
|
67
|
-
require 'sequel'
|
68
|
-
require 'pathname'
|
69
|
-
require 'json'
|
70
|
-
|
71
|
-
module PromptManager
|
72
|
-
module Storage
|
73
|
-
class SqliteAdapter
|
74
|
-
DEFAULT_DB_FILE = 'prompts.sqlite3'.freeze
|
75
|
-
|
76
|
-
class << self
|
77
|
-
attr_accessor :db_file, :db
|
78
|
-
|
79
|
-
def config
|
80
|
-
if block_given?
|
81
|
-
yield self
|
82
|
-
else
|
83
|
-
raise ArgumentError, 'No block given to config'
|
84
|
-
end
|
85
|
-
|
86
|
-
self.db = Sequel.sqlite(db_file || DEFAULT_DB_FILE) # Use provided db_file or default
|
87
|
-
create_tables unless db.table_exists?(:prompts)
|
88
|
-
end
|
89
|
-
|
90
|
-
# Define the necessary tables within the SQLite database if they don't exist
|
91
|
-
def create_tables
|
92
|
-
db.create_table :prompts do
|
93
|
-
primary_key :id
|
94
|
-
String :prompt_id, unique: true, null: false
|
95
|
-
Text :text
|
96
|
-
Json :parameters
|
97
|
-
|
98
|
-
index :prompt_id
|
99
|
-
end
|
100
|
-
end
|
101
|
-
end
|
102
|
-
|
103
|
-
def initialize
|
104
|
-
@db = self.class.db
|
105
|
-
end
|
106
|
-
|
107
|
-
def get(id:)
|
108
|
-
validate_id(id)
|
109
|
-
result = @db[:prompts].where(prompt_id: id).first
|
110
|
-
raise ArgumentError, 'Prompt not found' unless result
|
111
|
-
|
112
|
-
{
|
113
|
-
id: result[:prompt_id],
|
114
|
-
text: result[:text],
|
115
|
-
parameters: result[:parameters]
|
116
|
-
}
|
117
|
-
end
|
118
|
-
|
119
|
-
def save(id:, text: '', parameters: {})
|
120
|
-
validate_id(id)
|
121
|
-
rec = @db[:prompts].where(prompt_id: id).first
|
122
|
-
if rec
|
123
|
-
@db[:prompts].where(prompt_id: id).update(text: text, parameters: Sequel.pg_json(parameters))
|
124
|
-
else
|
125
|
-
@db[:prompts].insert(prompt_id: id, text: text, parameters: Sequel.pg_json(parameters))
|
126
|
-
end
|
127
|
-
end
|
128
|
-
|
129
|
-
def delete(id:)
|
130
|
-
validate_id(id)
|
131
|
-
@db[:prompts].where(prompt_id: id).delete
|
132
|
-
end
|
133
|
-
|
134
|
-
# Return an Array of prompt IDs
|
135
|
-
def list()
|
136
|
-
@db[:prompts].select_map(:prompt_id)
|
137
|
-
end
|
138
|
-
|
139
|
-
def search(for_what)
|
140
|
-
search_term = for_what.downcase
|
141
|
-
@db[:prompts].where(Sequel.ilike(:text, "%#{search_term}%")).select_map(:prompt_id)
|
142
|
-
end
|
143
|
-
|
144
|
-
private
|
145
|
-
|
146
|
-
# Validate that the ID contains good characters.
|
147
|
-
def validate_id(id)
|
148
|
-
raise ArgumentError, "Invalid ID format id: #{id}" unless id =~ /^[a-zA-Z0-9\-\/_]+$/
|
149
|
-
end
|
150
|
-
end
|
151
|
-
end
|
152
|
-
end
|
153
|
-
|
154
|
-
|
155
|
-
|
156
|
-
|