prompt_manager 0.3.3 → 0.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +2 -1
- data/README.md +67 -19
- data/lib/prompt_manager/prompt.rb +38 -3
- data/lib/prompt_manager/version.rb +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 60ae8d4c5f669935906248aa0e5ffc628e6ef0fbcc28eb31c889eb85b200d370
|
|
4
|
+
data.tar.gz: 44e51b38634667930b5524232cefd63fe0dab3ef334500714361680009186bfd
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 634cc2d1cfdedaf1ca5f5a232334b7e8ccbad25dbb48e669daddf831e43eb45f03367f2921561cdb8ddcf1675ea48f1dcf8cc40defb479aa1eced3bb493442f9
|
|
7
|
+
data.tar.gz: f04cc92a7ac3c286fb3958eaa44358c402899a121e89ca41aa7bddd30c41bd704bed55dd57775340c622671d0fbf36eef48171ffc3d290f220d6306622486ef3
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
|
@@ -9,30 +9,32 @@ Manage the parameterized prompts (text) used in generative AI (aka chatGPT, Open
|
|
|
9
9
|
|
|
10
10
|
## Table of Contents
|
|
11
11
|
|
|
12
|
-
- [Installation](#installation)
|
|
13
|
-
- [Usage](#usage)
|
|
14
|
-
- [Overview](#overview)
|
|
12
|
+
- [Installation](#installation)
|
|
13
|
+
- [Usage](#usage)
|
|
14
|
+
- [Overview](#overview)
|
|
15
15
|
- [Generative AI (gen-AI)](#generative-ai-gen-ai)
|
|
16
16
|
- [What does a keyword look like?](#what-does-a-keyword-look-like)
|
|
17
|
-
- [
|
|
17
|
+
- [All about directives](#all-about-directives)
|
|
18
|
+
- [Comments Are Ignored](#comments-are-ignored)
|
|
19
|
+
- [Storage Adapters](#storage-adapters)
|
|
18
20
|
- [FileSystemAdapter](#filesystemadapter)
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
21
|
+
- [Configuration](#configuration)
|
|
22
|
+
- [prompts_dir](#prompts_dir)
|
|
23
|
+
- [search_proc](#search_proc)
|
|
24
|
+
- [File Extensions](#file-extensions)
|
|
25
|
+
- [Example Prompt Text File](#example-prompt-text-file)
|
|
26
|
+
- [Example Prompt Parameters JSON File](#example-prompt-parameters-json-file)
|
|
27
|
+
- [Extra Functionality](#extra-functionality)
|
|
26
28
|
- [ActiveRecordAdapter](#activerecordadapter)
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
29
|
+
- [Configuration](#configuration-1)
|
|
30
|
+
- [model](#model)
|
|
31
|
+
- [id_column](#id_column)
|
|
32
|
+
- [text_column](#text_column)
|
|
33
|
+
- [parameters_column](#parameters_column)
|
|
32
34
|
- [Other Potential Storage Adapters](#other-potential-storage-adapters)
|
|
33
|
-
- [Development](#development)
|
|
34
|
-
- [Contributing](#contributing)
|
|
35
|
-
- [License](#license)
|
|
35
|
+
- [Development](#development)
|
|
36
|
+
- [Contributing](#contributing)
|
|
37
|
+
- [License](#license)
|
|
36
38
|
|
|
37
39
|
<!-- Tocer[finish]: Auto-generated, don't remove. -->
|
|
38
40
|
|
|
@@ -54,6 +56,8 @@ See also [examples/using_search_proc.rb](examples/using_search_proc.rb)
|
|
|
54
56
|
|
|
55
57
|
## Overview
|
|
56
58
|
|
|
59
|
+
The `prompt_manager` gem provides functionality to manage prompts that have keywords and directives for use with generative AI processes.
|
|
60
|
+
|
|
57
61
|
### Generative AI (gen-AI)
|
|
58
62
|
|
|
59
63
|
Gen-AI deals with the conversion (some would say execution) of a human natural language text (the "prompt") into somthing else using what are known as large language models (LLM) such as those available from OpenAI. A parameterized prompt is one in which there are embedded keywords (parameters) which are place holders for other text to be inserted into the prompt.
|
|
@@ -66,6 +70,50 @@ The current hard-coded REGEX for a [KEYWORD] identifies any all [UPPERCASE_TEXT]
|
|
|
66
70
|
|
|
67
71
|
This is just the initial convention adopted by prompt_manager. It is intended that this REGEX be configurable so that the prompt_manager can be used with other conventions.
|
|
68
72
|
|
|
73
|
+
#### All about directives
|
|
74
|
+
|
|
75
|
+
A directive is a line in the prompt text that starts with the two characters '//' - slash slash - just like in the old days of IBM JCL - Job Control Language. A prompt can have zero or more directives. Directives can have parameters and can make use of keywords.
|
|
76
|
+
|
|
77
|
+
The `prompt_manager` only collections directives. It extracts keywords from directive lines and provides the substitution of those keywords with other text just like it does for the prompt. Remember a directive is part of the prompt.
|
|
78
|
+
|
|
79
|
+
Here is an example problem with comments, directives and keywords:
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
# prompts/sing_a_song.txt
|
|
83
|
+
# Desc: Has the computer sing a song
|
|
84
|
+
|
|
85
|
+
//TextToSpeech [LANGUAGE] [VOICE NAME]
|
|
86
|
+
|
|
87
|
+
Say the lyrics to the song [SONG NAME]. Please provide only the lyrics without commentary.
|
|
88
|
+
|
|
89
|
+
__END__
|
|
90
|
+
Computers will never replace Frank Sinatra
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
Getting directives from a prompt is as easy as getting the kewyords:
|
|
94
|
+
|
|
95
|
+
```ruby
|
|
96
|
+
prompt = PromptManager::Prompt.new(...)
|
|
97
|
+
prompt.keywords #=> an Array
|
|
98
|
+
prompt.directives #=> a Hash
|
|
99
|
+
|
|
100
|
+
# to_s builds the prompt by substituting
|
|
101
|
+
# values for keywords amd removing comments.
|
|
102
|
+
# The resulting text contains directives and
|
|
103
|
+
# prompt text ready for the LLM process.
|
|
104
|
+
puts prompt.to_s
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
The Hash returned by the `prompt.directives` method has as its key the directive name (without the double-slash). Its also case-sensitive. So any typo made in editing the prompt with regard to the case of the directive's name will impact the backend processing of the prompt.
|
|
108
|
+
|
|
109
|
+
The value for each key is a String exactly as entered in the prompt file.
|
|
110
|
+
|
|
111
|
+
#### Comments Are Ignored
|
|
112
|
+
|
|
113
|
+
The `prompt_manager` gem ignores comments. A line that begins with the '#' - pound (aka hash) character - is a line comment. Any lines that follow a line that is '__END__ at the end of a file are considered comments. Basically the '__END__' the end of the file. Nothing is process following that line.
|
|
114
|
+
|
|
115
|
+
The gem also ignores blank lines.
|
|
116
|
+
|
|
69
117
|
## Storage Adapters
|
|
70
118
|
|
|
71
119
|
A storage adapter is a class instance that ties the `PromptManager::Prompt` class to a storage facility that holds the actual prompts. Currently there are 3 storage adapters planned for implementation.
|
|
@@ -5,9 +5,20 @@
|
|
|
5
5
|
# as well as building prompts with replacement of parameterized values and
|
|
6
6
|
# comment removal. It communicates with a storage system through a storage
|
|
7
7
|
# adapter.
|
|
8
|
+
#
|
|
9
|
+
# Directives can be anything required by the backend
|
|
10
|
+
# prompt processing functionality. Directives are
|
|
11
|
+
# available along with the prompt text w/o comments.
|
|
12
|
+
# Keywords can be used in the directives.
|
|
13
|
+
#
|
|
14
|
+
# It is expected that the backend prompt processes will
|
|
15
|
+
# act on and remove the directives before passing the
|
|
16
|
+
# prompt text on through the LLM.
|
|
8
17
|
|
|
9
18
|
class PromptManager::Prompt
|
|
10
|
-
|
|
19
|
+
COMMENT_SIGNAL = '#' # lines beginning with this are a comment
|
|
20
|
+
DIRECTIVE_SIGNAL = '//' # Like the old IBM JCL
|
|
21
|
+
PARAMETER_REGEX = /(\[[A-Z _|]+\])/
|
|
11
22
|
@storage_adapter = nil
|
|
12
23
|
|
|
13
24
|
class << self
|
|
@@ -64,9 +75,12 @@ class PromptManager::Prompt
|
|
|
64
75
|
@record = db.get(id: id)
|
|
65
76
|
@text = @record[:text]
|
|
66
77
|
@parameters = @record[:parameters]
|
|
67
|
-
@keywords = []
|
|
78
|
+
@keywords = [] # Array of String
|
|
79
|
+
@directives = {} # Hash with directive as key, parameters as value
|
|
68
80
|
|
|
69
81
|
update_keywords
|
|
82
|
+
update_directives
|
|
83
|
+
|
|
70
84
|
build
|
|
71
85
|
end
|
|
72
86
|
|
|
@@ -121,6 +135,11 @@ class PromptManager::Prompt
|
|
|
121
135
|
end
|
|
122
136
|
|
|
123
137
|
|
|
138
|
+
def directives
|
|
139
|
+
update_directives
|
|
140
|
+
end
|
|
141
|
+
|
|
142
|
+
|
|
124
143
|
######################################
|
|
125
144
|
private
|
|
126
145
|
|
|
@@ -134,10 +153,26 @@ class PromptManager::Prompt
|
|
|
134
153
|
end
|
|
135
154
|
|
|
136
155
|
|
|
156
|
+
def update_directives
|
|
157
|
+
@text.split("\n").each do |a_line|
|
|
158
|
+
line = a_line.strip
|
|
159
|
+
next unless line.start_with?(DIRECTIVE_SIGNAL)
|
|
160
|
+
|
|
161
|
+
parts = line.split(' ')
|
|
162
|
+
directive = parts.shift()[DIRECTIVE_SIGNAL.length..] # drop the directive signal
|
|
163
|
+
@directives[directive] = parts.join(' ')
|
|
164
|
+
end
|
|
165
|
+
|
|
166
|
+
@directives
|
|
167
|
+
end
|
|
168
|
+
|
|
169
|
+
|
|
137
170
|
def remove_comments
|
|
138
171
|
lines = @prompt
|
|
139
172
|
.split("\n")
|
|
140
|
-
.reject{|a_line|
|
|
173
|
+
.reject{|a_line|
|
|
174
|
+
a_line.strip.start_with?(COMMENT_SIGNAL)
|
|
175
|
+
}
|
|
141
176
|
|
|
142
177
|
# Remove empty lines at the start of the prompt
|
|
143
178
|
#
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: prompt_manager
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.4.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Dewayne VanHoozer
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2023-12-
|
|
11
|
+
date: 2023-12-19 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: activerecord
|
|
@@ -130,7 +130,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
130
130
|
- !ruby/object:Gem::Version
|
|
131
131
|
version: '0'
|
|
132
132
|
requirements: []
|
|
133
|
-
rubygems_version: 3.
|
|
133
|
+
rubygems_version: 3.5.1
|
|
134
134
|
signing_key:
|
|
135
135
|
specification_version: 4
|
|
136
136
|
summary: Manage prompts for use with gen-AI processes
|