prompt_manager 0.3.3 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -1
- data/README.md +99 -19
- data/lib/prompt_manager/prompt.rb +41 -5
- data/lib/prompt_manager/version.rb +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 91badda19121014eee4d6c28b1cb882347d951af971d9afb021b00e558cb3fdc
|
|
4
|
+
data.tar.gz: 500c401e8ced2be25adacaf374a6e18e73c42ec811e7bb632155de9d5626370c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 89c274000ee45ddb047eb33ab232d0e300fab51be512b1181e6d9d9dbf6abe5213013229bfd0450e948b78b09aa00aab882131ed066c7529e27c5865b8251647
|
|
7
|
+
data.tar.gz: 4fc94add6f43ab7b0f289849e68c3c6c9f35ca24a90983a459cd2ea9043de03b8b2f9ed6701a3a614be7efecd1fc5557ab0fb0d252e84dc628d38f9f80d2d9ba
|
data/CHANGELOG.md
CHANGED
|
@@ -1,4 +1,9 @@
|
|
|
1
|
-
## [
|
|
1
|
+
## [0.4.1] = 2023-12-29
|
|
2
|
+
- Changed @directives from Hash to an Array
|
|
3
|
+
- Fixed keywords not being substituted in directives
|
|
4
|
+
|
|
5
|
+
## [0.4.0] = 2023-12-19
|
|
6
|
+
- Add "//directives param(s)" with keywords just like the prompt text.
|
|
2
7
|
|
|
3
8
|
## [0.3.3] = 2023-12-01
|
|
4
9
|
- Added example of using the `search_proc` config parameter with the FileSystemAdapter.
|
data/README.md
CHANGED
|
@@ -9,30 +9,36 @@ Manage the parameterized prompts (text) used in generative AI (aka chatGPT, Open
|
|
|
9
9
|
|
|
10
10
|
## Table of Contents
|
|
11
11
|
|
|
12
|
-
- [Installation](#installation)
|
|
13
|
-
- [Usage](#usage)
|
|
14
|
-
- [Overview](#overview)
|
|
12
|
+
- [Installation](#installation)
|
|
13
|
+
- [Usage](#usage)
|
|
14
|
+
- [Overview](#overview)
|
|
15
15
|
- [Generative AI (gen-AI)](#generative-ai-gen-ai)
|
|
16
16
|
- [What does a keyword look like?](#what-does-a-keyword-look-like)
|
|
17
|
-
- [
|
|
17
|
+
- [All about directives](#all-about-directives)
|
|
18
|
+
- [Example Prompt with Directives](#example-prompt-with-directives)
|
|
19
|
+
- [Accessing Directives](#accessing-directives)
|
|
20
|
+
- [Dynamic Directives](#dynamic-directives)
|
|
21
|
+
- [Executing Directives](#executing-directives)
|
|
22
|
+
- [Comments Are Ignored](#comments-are-ignored)
|
|
23
|
+
- [Storage Adapters](#storage-adapters)
|
|
18
24
|
- [FileSystemAdapter](#filesystemadapter)
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
25
|
+
- [Configuration](#configuration)
|
|
26
|
+
- [prompts_dir](#prompts_dir)
|
|
27
|
+
- [search_proc](#search_proc)
|
|
28
|
+
- [File Extensions](#file-extensions)
|
|
29
|
+
- [Example Prompt Text File](#example-prompt-text-file)
|
|
30
|
+
- [Example Prompt Parameters JSON File](#example-prompt-parameters-json-file)
|
|
31
|
+
- [Extra Functionality](#extra-functionality)
|
|
26
32
|
- [ActiveRecordAdapter](#activerecordadapter)
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
33
|
+
- [Configuration](#configuration-1)
|
|
34
|
+
- [model](#model)
|
|
35
|
+
- [id_column](#id_column)
|
|
36
|
+
- [text_column](#text_column)
|
|
37
|
+
- [parameters_column](#parameters_column)
|
|
32
38
|
- [Other Potential Storage Adapters](#other-potential-storage-adapters)
|
|
33
|
-
- [Development](#development)
|
|
34
|
-
- [Contributing](#contributing)
|
|
35
|
-
- [License](#license)
|
|
39
|
+
- [Development](#development)
|
|
40
|
+
- [Contributing](#contributing)
|
|
41
|
+
- [License](#license)
|
|
36
42
|
|
|
37
43
|
<!-- Tocer[finish]: Auto-generated, don't remove. -->
|
|
38
44
|
|
|
@@ -54,6 +60,8 @@ See also [examples/using_search_proc.rb](examples/using_search_proc.rb)
|
|
|
54
60
|
|
|
55
61
|
## Overview
|
|
56
62
|
|
|
63
|
+
The `prompt_manager` gem provides functionality to manage prompts that have keywords and directives for use with generative AI processes.
|
|
64
|
+
|
|
57
65
|
### Generative AI (gen-AI)
|
|
58
66
|
|
|
59
67
|
Gen-AI deals with the conversion (some would say execution) of a human natural language text (the "prompt") into somthing else using what are known as large language models (LLM) such as those available from OpenAI. A parameterized prompt is one in which there are embedded keywords (parameters) which are place holders for other text to be inserted into the prompt.
|
|
@@ -66,6 +74,78 @@ The current hard-coded REGEX for a [KEYWORD] identifies any all [UPPERCASE_TEXT]
|
|
|
66
74
|
|
|
67
75
|
This is just the initial convention adopted by prompt_manager. It is intended that this REGEX be configurable so that the prompt_manager can be used with other conventions.
|
|
68
76
|
|
|
77
|
+
#### All about directives
|
|
78
|
+
|
|
79
|
+
A directive is a line in the prompt text that starts with the two characters '//' - slash slash - just like in the old days of IBM JCL - Job Control Language. A prompt can have zero or more directives. Directives can have parameters and can make use of keywords.
|
|
80
|
+
|
|
81
|
+
The `prompt_manager` only collects directives. It extracts keywords from directive lines and provides the substitution of those keywords with other text just like it does for the prompt.
|
|
82
|
+
|
|
83
|
+
##### Example Prompt with Directives
|
|
84
|
+
|
|
85
|
+
Here is an example prompt text file with comments, directives and keywords:
|
|
86
|
+
|
|
87
|
+
```
|
|
88
|
+
# prompts/sing_a_song.txt
|
|
89
|
+
# Desc: Has the computer sing a song
|
|
90
|
+
|
|
91
|
+
//TextToSpeech [LANGUAGE] [VOICE NAME]
|
|
92
|
+
|
|
93
|
+
Say the lyrics to the song [SONG NAME]. Please provide only the lyrics without commentary.
|
|
94
|
+
|
|
95
|
+
__END__
|
|
96
|
+
Computers will never replace Frank Sinatra
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
##### Accessing Directives
|
|
100
|
+
|
|
101
|
+
Getting directives from a prompt is as easy as getting the kewyords:
|
|
102
|
+
|
|
103
|
+
```ruby
|
|
104
|
+
prompt = PromptManager::Prompt.new(...)
|
|
105
|
+
prompt.keywords #=> an Array
|
|
106
|
+
prompt.directives #=> an Array of entries like: ['directive', 'parameters']
|
|
107
|
+
|
|
108
|
+
# to_s builds the prompt by substituting
|
|
109
|
+
# values for keywords amd removing comments.
|
|
110
|
+
# The resulting text contains directives and
|
|
111
|
+
# prompt text ready for the LLM process.
|
|
112
|
+
puts prompt.to_s
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
The entries in the Array returned by the `prompt.directives` method is in the order that the directives were defined within the prompt. Each entry has two elements:
|
|
116
|
+
|
|
117
|
+
- directive name (without the // characters)
|
|
118
|
+
- parameter string for the directive
|
|
119
|
+
|
|
120
|
+
##### Dynamic Directives
|
|
121
|
+
|
|
122
|
+
Since directies are collected after the keywords in the prompt have been substituted for their values, it is possible to have dynamically generated directives as part of a prompt. For example:
|
|
123
|
+
|
|
124
|
+
```
|
|
125
|
+
//[COMMAND] [OPTIONS]
|
|
126
|
+
# or
|
|
127
|
+
[SOMETHING]
|
|
128
|
+
```
|
|
129
|
+
... where [COMMAND] gets replaced by some directive name. [SOMETHING] could be replaced by "//directive options"
|
|
130
|
+
|
|
131
|
+
##### Executing Directives
|
|
132
|
+
|
|
133
|
+
The `prompt_manager` gem only collects directives. Executing those directives is left up to some down stream process. Here are some ideas on how directives could be used in prompt downstream process:
|
|
134
|
+
|
|
135
|
+
- "//model gpt-5" could be used to set the LLM model to be used for a specific prompt.
|
|
136
|
+
- "//backend mods" could be used to set the backend prompt processor on the command line to be the `mods` utility.
|
|
137
|
+
- "//include path_to_file" could be used to add the contents of a file to the prompt.
|
|
138
|
+
- "//chat" could be used to send the prompts and then start up a chat session about the prompt and its response.
|
|
139
|
+
|
|
140
|
+
Its all up to how your application wants to support directives or not.
|
|
141
|
+
|
|
142
|
+
|
|
143
|
+
#### Comments Are Ignored
|
|
144
|
+
|
|
145
|
+
The `prompt_manager` gem ignores comments. A line that begins with the '#' - pound (aka hash) character - is a line comment. Any lines that follow a line that is '__END__ at the end of a file are considered comments. Basically the '__END__' the end of the file. Nothing is process following that line.
|
|
146
|
+
|
|
147
|
+
The gem also ignores blank lines.
|
|
148
|
+
|
|
69
149
|
## Storage Adapters
|
|
70
150
|
|
|
71
151
|
A storage adapter is a class instance that ties the `PromptManager::Prompt` class to a storage facility that holds the actual prompts. Currently there are 3 storage adapters planned for implementation.
|
|
@@ -5,9 +5,23 @@
|
|
|
5
5
|
# as well as building prompts with replacement of parameterized values and
|
|
6
6
|
# comment removal. It communicates with a storage system through a storage
|
|
7
7
|
# adapter.
|
|
8
|
+
#
|
|
9
|
+
# Directives are collected into an Array where each entry is an Array
|
|
10
|
+
# of two elements. The first is the directive name as a String. The
|
|
11
|
+
# second is a string of parameters used by the directive.
|
|
12
|
+
#
|
|
13
|
+
# Directives are collected from the prompt after keyword
|
|
14
|
+
# substitution has occured. This means that directives within a
|
|
15
|
+
# prompt can be dynamic.
|
|
16
|
+
#
|
|
17
|
+
# PromptManager does not execute directives. They
|
|
18
|
+
# are made available to be passed on to down stream
|
|
19
|
+
# process.
|
|
8
20
|
|
|
9
21
|
class PromptManager::Prompt
|
|
10
|
-
|
|
22
|
+
COMMENT_SIGNAL = '#' # lines beginning with this are a comment
|
|
23
|
+
DIRECTIVE_SIGNAL = '//' # Like the old IBM JCL
|
|
24
|
+
PARAMETER_REGEX = /(\[[A-Z _|]+\])/
|
|
11
25
|
@storage_adapter = nil
|
|
12
26
|
|
|
13
27
|
class << self
|
|
@@ -47,7 +61,7 @@ class PromptManager::Prompt
|
|
|
47
61
|
|
|
48
62
|
# SMELL: Does the db (aka storage adapter) really need
|
|
49
63
|
# to be accessible by the main program?
|
|
50
|
-
attr_accessor :db, :id, :text, :parameters
|
|
64
|
+
attr_accessor :db, :id, :text, :parameters, :directives
|
|
51
65
|
|
|
52
66
|
|
|
53
67
|
# Retrieve the specific prompt ID from the Storage system.
|
|
@@ -64,9 +78,11 @@ class PromptManager::Prompt
|
|
|
64
78
|
@record = db.get(id: id)
|
|
65
79
|
@text = @record[:text]
|
|
66
80
|
@parameters = @record[:parameters]
|
|
67
|
-
@keywords = []
|
|
81
|
+
@keywords = [] # Array of String
|
|
82
|
+
@directives = [] # Array of arrays. directive is first entry, rest are parameters
|
|
68
83
|
|
|
69
84
|
update_keywords
|
|
85
|
+
|
|
70
86
|
build
|
|
71
87
|
end
|
|
72
88
|
|
|
@@ -111,7 +127,8 @@ class PromptManager::Prompt
|
|
|
111
127
|
param_name = match
|
|
112
128
|
Array(parameters[param_name]).last || match
|
|
113
129
|
end
|
|
114
|
-
|
|
130
|
+
|
|
131
|
+
save_directives(@prompt)
|
|
115
132
|
remove_comments
|
|
116
133
|
end
|
|
117
134
|
|
|
@@ -134,10 +151,29 @@ class PromptManager::Prompt
|
|
|
134
151
|
end
|
|
135
152
|
|
|
136
153
|
|
|
154
|
+
def save_directives(keyword_substituted_string)
|
|
155
|
+
@directives = []
|
|
156
|
+
|
|
157
|
+
keyword_substituted_string.split("\n").each do |a_line|
|
|
158
|
+
line = a_line.strip
|
|
159
|
+
next unless line.start_with?(DIRECTIVE_SIGNAL)
|
|
160
|
+
|
|
161
|
+
parts = line.split(' ')
|
|
162
|
+
directive = parts.shift[DIRECTIVE_SIGNAL.length..] # drop the directive signal
|
|
163
|
+
@directives << [directive, parts.join(' ')]
|
|
164
|
+
end
|
|
165
|
+
|
|
166
|
+
@directives
|
|
167
|
+
end
|
|
168
|
+
|
|
169
|
+
|
|
137
170
|
def remove_comments
|
|
138
171
|
lines = @prompt
|
|
139
172
|
.split("\n")
|
|
140
|
-
.reject{|a_line|
|
|
173
|
+
.reject{|a_line|
|
|
174
|
+
a_line.strip.start_with?(COMMENT_SIGNAL) ||
|
|
175
|
+
a_line.strip.start_with?(DIRECTIVE_SIGNAL)
|
|
176
|
+
}
|
|
141
177
|
|
|
142
178
|
# Remove empty lines at the start of the prompt
|
|
143
179
|
#
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: prompt_manager
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.4.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Dewayne VanHoozer
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2023-12-
|
|
11
|
+
date: 2023-12-29 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: activerecord
|
|
@@ -130,7 +130,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
130
130
|
- !ruby/object:Gem::Version
|
|
131
131
|
version: '0'
|
|
132
132
|
requirements: []
|
|
133
|
-
rubygems_version: 3.
|
|
133
|
+
rubygems_version: 3.5.3
|
|
134
134
|
signing_key:
|
|
135
135
|
specification_version: 4
|
|
136
136
|
summary: Manage prompts for use with gen-AI processes
|