prompt_manager 0.3.3 → 0.4.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -1
- data/README.md +99 -19
- data/lib/prompt_manager/prompt.rb +41 -5
- data/lib/prompt_manager/version.rb +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 91badda19121014eee4d6c28b1cb882347d951af971d9afb021b00e558cb3fdc
|
|
4
|
+
data.tar.gz: 500c401e8ced2be25adacaf374a6e18e73c42ec811e7bb632155de9d5626370c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 89c274000ee45ddb047eb33ab232d0e300fab51be512b1181e6d9d9dbf6abe5213013229bfd0450e948b78b09aa00aab882131ed066c7529e27c5865b8251647
|
|
7
|
+
data.tar.gz: 4fc94add6f43ab7b0f289849e68c3c6c9f35ca24a90983a459cd2ea9043de03b8b2f9ed6701a3a614be7efecd1fc5557ab0fb0d252e84dc628d38f9f80d2d9ba
|
data/CHANGELOG.md
CHANGED
|
@@ -1,4 +1,9 @@
|
|
|
1
|
-
## [
|
|
1
|
+
## [0.4.1] = 2023-12-29
|
|
2
|
+
- Changed @directives from Hash to an Array
|
|
3
|
+
- Fixed keywords not being substituted in directives
|
|
4
|
+
|
|
5
|
+
## [0.4.0] = 2023-12-19
|
|
6
|
+
- Add "//directives param(s)" with keywords just like the prompt text.
|
|
2
7
|
|
|
3
8
|
## [0.3.3] = 2023-12-01
|
|
4
9
|
- Added example of using the `search_proc` config parameter with the FileSystemAdapter.
|
data/README.md
CHANGED
|
@@ -9,30 +9,36 @@ Manage the parameterized prompts (text) used in generative AI (aka chatGPT, Open
|
|
|
9
9
|
|
|
10
10
|
## Table of Contents
|
|
11
11
|
|
|
12
|
-
- [Installation](#installation)
|
|
13
|
-
- [Usage](#usage)
|
|
14
|
-
- [Overview](#overview)
|
|
12
|
+
- [Installation](#installation)
|
|
13
|
+
- [Usage](#usage)
|
|
14
|
+
- [Overview](#overview)
|
|
15
15
|
- [Generative AI (gen-AI)](#generative-ai-gen-ai)
|
|
16
16
|
- [What does a keyword look like?](#what-does-a-keyword-look-like)
|
|
17
|
-
- [
|
|
17
|
+
- [All about directives](#all-about-directives)
|
|
18
|
+
- [Example Prompt with Directives](#example-prompt-with-directives)
|
|
19
|
+
- [Accessing Directives](#accessing-directives)
|
|
20
|
+
- [Dynamic Directives](#dynamic-directives)
|
|
21
|
+
- [Executing Directives](#executing-directives)
|
|
22
|
+
- [Comments Are Ignored](#comments-are-ignored)
|
|
23
|
+
- [Storage Adapters](#storage-adapters)
|
|
18
24
|
- [FileSystemAdapter](#filesystemadapter)
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
25
|
+
- [Configuration](#configuration)
|
|
26
|
+
- [prompts_dir](#prompts_dir)
|
|
27
|
+
- [search_proc](#search_proc)
|
|
28
|
+
- [File Extensions](#file-extensions)
|
|
29
|
+
- [Example Prompt Text File](#example-prompt-text-file)
|
|
30
|
+
- [Example Prompt Parameters JSON File](#example-prompt-parameters-json-file)
|
|
31
|
+
- [Extra Functionality](#extra-functionality)
|
|
26
32
|
- [ActiveRecordAdapter](#activerecordadapter)
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
33
|
+
- [Configuration](#configuration-1)
|
|
34
|
+
- [model](#model)
|
|
35
|
+
- [id_column](#id_column)
|
|
36
|
+
- [text_column](#text_column)
|
|
37
|
+
- [parameters_column](#parameters_column)
|
|
32
38
|
- [Other Potential Storage Adapters](#other-potential-storage-adapters)
|
|
33
|
-
- [Development](#development)
|
|
34
|
-
- [Contributing](#contributing)
|
|
35
|
-
- [License](#license)
|
|
39
|
+
- [Development](#development)
|
|
40
|
+
- [Contributing](#contributing)
|
|
41
|
+
- [License](#license)
|
|
36
42
|
|
|
37
43
|
<!-- Tocer[finish]: Auto-generated, don't remove. -->
|
|
38
44
|
|
|
@@ -54,6 +60,8 @@ See also [examples/using_search_proc.rb](examples/using_search_proc.rb)
|
|
|
54
60
|
|
|
55
61
|
## Overview
|
|
56
62
|
|
|
63
|
+
The `prompt_manager` gem provides functionality to manage prompts that have keywords and directives for use with generative AI processes.
|
|
64
|
+
|
|
57
65
|
### Generative AI (gen-AI)
|
|
58
66
|
|
|
59
67
|
Gen-AI deals with the conversion (some would say execution) of a human natural language text (the "prompt") into somthing else using what are known as large language models (LLM) such as those available from OpenAI. A parameterized prompt is one in which there are embedded keywords (parameters) which are place holders for other text to be inserted into the prompt.
|
|
@@ -66,6 +74,78 @@ The current hard-coded REGEX for a [KEYWORD] identifies any all [UPPERCASE_TEXT]
|
|
|
66
74
|
|
|
67
75
|
This is just the initial convention adopted by prompt_manager. It is intended that this REGEX be configurable so that the prompt_manager can be used with other conventions.
|
|
68
76
|
|
|
77
|
+
#### All about directives
|
|
78
|
+
|
|
79
|
+
A directive is a line in the prompt text that starts with the two characters '//' - slash slash - just like in the old days of IBM JCL - Job Control Language. A prompt can have zero or more directives. Directives can have parameters and can make use of keywords.
|
|
80
|
+
|
|
81
|
+
The `prompt_manager` only collects directives. It extracts keywords from directive lines and provides the substitution of those keywords with other text just like it does for the prompt.
|
|
82
|
+
|
|
83
|
+
##### Example Prompt with Directives
|
|
84
|
+
|
|
85
|
+
Here is an example prompt text file with comments, directives and keywords:
|
|
86
|
+
|
|
87
|
+
```
|
|
88
|
+
# prompts/sing_a_song.txt
|
|
89
|
+
# Desc: Has the computer sing a song
|
|
90
|
+
|
|
91
|
+
//TextToSpeech [LANGUAGE] [VOICE NAME]
|
|
92
|
+
|
|
93
|
+
Say the lyrics to the song [SONG NAME]. Please provide only the lyrics without commentary.
|
|
94
|
+
|
|
95
|
+
__END__
|
|
96
|
+
Computers will never replace Frank Sinatra
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
##### Accessing Directives
|
|
100
|
+
|
|
101
|
+
Getting directives from a prompt is as easy as getting the kewyords:
|
|
102
|
+
|
|
103
|
+
```ruby
|
|
104
|
+
prompt = PromptManager::Prompt.new(...)
|
|
105
|
+
prompt.keywords #=> an Array
|
|
106
|
+
prompt.directives #=> an Array of entries like: ['directive', 'parameters']
|
|
107
|
+
|
|
108
|
+
# to_s builds the prompt by substituting
|
|
109
|
+
# values for keywords amd removing comments.
|
|
110
|
+
# The resulting text contains directives and
|
|
111
|
+
# prompt text ready for the LLM process.
|
|
112
|
+
puts prompt.to_s
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
The entries in the Array returned by the `prompt.directives` method is in the order that the directives were defined within the prompt. Each entry has two elements:
|
|
116
|
+
|
|
117
|
+
- directive name (without the // characters)
|
|
118
|
+
- parameter string for the directive
|
|
119
|
+
|
|
120
|
+
##### Dynamic Directives
|
|
121
|
+
|
|
122
|
+
Since directies are collected after the keywords in the prompt have been substituted for their values, it is possible to have dynamically generated directives as part of a prompt. For example:
|
|
123
|
+
|
|
124
|
+
```
|
|
125
|
+
//[COMMAND] [OPTIONS]
|
|
126
|
+
# or
|
|
127
|
+
[SOMETHING]
|
|
128
|
+
```
|
|
129
|
+
... where [COMMAND] gets replaced by some directive name. [SOMETHING] could be replaced by "//directive options"
|
|
130
|
+
|
|
131
|
+
##### Executing Directives
|
|
132
|
+
|
|
133
|
+
The `prompt_manager` gem only collects directives. Executing those directives is left up to some down stream process. Here are some ideas on how directives could be used in prompt downstream process:
|
|
134
|
+
|
|
135
|
+
- "//model gpt-5" could be used to set the LLM model to be used for a specific prompt.
|
|
136
|
+
- "//backend mods" could be used to set the backend prompt processor on the command line to be the `mods` utility.
|
|
137
|
+
- "//include path_to_file" could be used to add the contents of a file to the prompt.
|
|
138
|
+
- "//chat" could be used to send the prompts and then start up a chat session about the prompt and its response.
|
|
139
|
+
|
|
140
|
+
Its all up to how your application wants to support directives or not.
|
|
141
|
+
|
|
142
|
+
|
|
143
|
+
#### Comments Are Ignored
|
|
144
|
+
|
|
145
|
+
The `prompt_manager` gem ignores comments. A line that begins with the '#' - pound (aka hash) character - is a line comment. Any lines that follow a line that is '__END__ at the end of a file are considered comments. Basically the '__END__' the end of the file. Nothing is process following that line.
|
|
146
|
+
|
|
147
|
+
The gem also ignores blank lines.
|
|
148
|
+
|
|
69
149
|
## Storage Adapters
|
|
70
150
|
|
|
71
151
|
A storage adapter is a class instance that ties the `PromptManager::Prompt` class to a storage facility that holds the actual prompts. Currently there are 3 storage adapters planned for implementation.
|
|
@@ -5,9 +5,23 @@
|
|
|
5
5
|
# as well as building prompts with replacement of parameterized values and
|
|
6
6
|
# comment removal. It communicates with a storage system through a storage
|
|
7
7
|
# adapter.
|
|
8
|
+
#
|
|
9
|
+
# Directives are collected into an Array where each entry is an Array
|
|
10
|
+
# of two elements. The first is the directive name as a String. The
|
|
11
|
+
# second is a string of parameters used by the directive.
|
|
12
|
+
#
|
|
13
|
+
# Directives are collected from the prompt after keyword
|
|
14
|
+
# substitution has occured. This means that directives within a
|
|
15
|
+
# prompt can be dynamic.
|
|
16
|
+
#
|
|
17
|
+
# PromptManager does not execute directives. They
|
|
18
|
+
# are made available to be passed on to down stream
|
|
19
|
+
# process.
|
|
8
20
|
|
|
9
21
|
class PromptManager::Prompt
|
|
10
|
-
|
|
22
|
+
COMMENT_SIGNAL = '#' # lines beginning with this are a comment
|
|
23
|
+
DIRECTIVE_SIGNAL = '//' # Like the old IBM JCL
|
|
24
|
+
PARAMETER_REGEX = /(\[[A-Z _|]+\])/
|
|
11
25
|
@storage_adapter = nil
|
|
12
26
|
|
|
13
27
|
class << self
|
|
@@ -47,7 +61,7 @@ class PromptManager::Prompt
|
|
|
47
61
|
|
|
48
62
|
# SMELL: Does the db (aka storage adapter) really need
|
|
49
63
|
# to be accessible by the main program?
|
|
50
|
-
attr_accessor :db, :id, :text, :parameters
|
|
64
|
+
attr_accessor :db, :id, :text, :parameters, :directives
|
|
51
65
|
|
|
52
66
|
|
|
53
67
|
# Retrieve the specific prompt ID from the Storage system.
|
|
@@ -64,9 +78,11 @@ class PromptManager::Prompt
|
|
|
64
78
|
@record = db.get(id: id)
|
|
65
79
|
@text = @record[:text]
|
|
66
80
|
@parameters = @record[:parameters]
|
|
67
|
-
@keywords = []
|
|
81
|
+
@keywords = [] # Array of String
|
|
82
|
+
@directives = [] # Array of arrays. directive is first entry, rest are parameters
|
|
68
83
|
|
|
69
84
|
update_keywords
|
|
85
|
+
|
|
70
86
|
build
|
|
71
87
|
end
|
|
72
88
|
|
|
@@ -111,7 +127,8 @@ class PromptManager::Prompt
|
|
|
111
127
|
param_name = match
|
|
112
128
|
Array(parameters[param_name]).last || match
|
|
113
129
|
end
|
|
114
|
-
|
|
130
|
+
|
|
131
|
+
save_directives(@prompt)
|
|
115
132
|
remove_comments
|
|
116
133
|
end
|
|
117
134
|
|
|
@@ -134,10 +151,29 @@ class PromptManager::Prompt
|
|
|
134
151
|
end
|
|
135
152
|
|
|
136
153
|
|
|
154
|
+
def save_directives(keyword_substituted_string)
|
|
155
|
+
@directives = []
|
|
156
|
+
|
|
157
|
+
keyword_substituted_string.split("\n").each do |a_line|
|
|
158
|
+
line = a_line.strip
|
|
159
|
+
next unless line.start_with?(DIRECTIVE_SIGNAL)
|
|
160
|
+
|
|
161
|
+
parts = line.split(' ')
|
|
162
|
+
directive = parts.shift[DIRECTIVE_SIGNAL.length..] # drop the directive signal
|
|
163
|
+
@directives << [directive, parts.join(' ')]
|
|
164
|
+
end
|
|
165
|
+
|
|
166
|
+
@directives
|
|
167
|
+
end
|
|
168
|
+
|
|
169
|
+
|
|
137
170
|
def remove_comments
|
|
138
171
|
lines = @prompt
|
|
139
172
|
.split("\n")
|
|
140
|
-
.reject{|a_line|
|
|
173
|
+
.reject{|a_line|
|
|
174
|
+
a_line.strip.start_with?(COMMENT_SIGNAL) ||
|
|
175
|
+
a_line.strip.start_with?(DIRECTIVE_SIGNAL)
|
|
176
|
+
}
|
|
141
177
|
|
|
142
178
|
# Remove empty lines at the start of the prompt
|
|
143
179
|
#
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: prompt_manager
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.4.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Dewayne VanHoozer
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2023-12-
|
|
11
|
+
date: 2023-12-29 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: activerecord
|
|
@@ -130,7 +130,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
130
130
|
- !ruby/object:Gem::Version
|
|
131
131
|
version: '0'
|
|
132
132
|
requirements: []
|
|
133
|
-
rubygems_version: 3.
|
|
133
|
+
rubygems_version: 3.5.3
|
|
134
134
|
signing_key:
|
|
135
135
|
specification_version: 4
|
|
136
136
|
summary: Manage prompts for use with gen-AI processes
|