prompt_manager 0.4.2 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6c566743ba6741a0787db6ea680e77676ab18282013e3475cad1c30b85b9ce1b
-  data.tar.gz: 1e9e57eecaf97d05799d6b173b1ce90518dc02abeeddbd50eed90c34c376509f
+  metadata.gz: 267e48a65e377f2fc3390d3bbff69a6e1f241c96951fc86437f5ab02d2e93daa
+  data.tar.gz: db8af7d8515e72bf12f847e3544a7413a0acb0707a8321bcae7f13e1ffa1ff86
 SHA512:
-  metadata.gz: e1bcda6797d895b4dd78baf6c6c6cd2b83e2f06bb3446f9112795a6d216eb9450af9f1c2d0b3f762d4f21ee40a1257fec8610eadec33f7f19dabf078e7a859bf
-  data.tar.gz: f69ed112a2b22801dc830d87b11f0a231755141fa4eccded5ee2dc8019343cf6cf61705af560fed211866869940029bbde88cca7c821423da245593839515e03
+  metadata.gz: 259cb1494a9390f94d40c669bd87960d9257a9126c3d71cbc6fdcdbaa82dae3e0209f23acc322d019e781abe48090a070fba20761a2ffe8a1818a303c8ef2741
+  data.tar.gz: 97e6ce3eb0743b02804b44c745a2710f15e4048957c7a039c6313b2d061d65660f9d4435465fd0139ff8e348ba841cb4ae543d06a7709391fab19a6419ea1e55

data/CHANGELOG.md

@@ -1,6 +1,13 @@
 ## Unreleased
 
 ## Released
+### [0.5.0] = 2025-03-29
+- Major refactoring of to improve processing of parameters and directives.
+- Added PromptManager::DirectiveProcessor as an example of how to implement custom directives.
+- Added support for //include directive that protects against loops.
+- Added support for embedding system environment variables.
+- Added support for ERB processing within a prompt.
+- Improved test coverage.
 
 ### [0.4.2] = 2024-10-26
 - Added configurable parameter_regex to customize keyword pattern

data/README.md

@@ -9,6 +9,8 @@ Manage the parameterized prompts (text) used in generative AI (aka chatGPT, Open
 
 ## Table of Contents
 
+- [PromptManager](#promptmanager)
+  - [Table of Contents](#table-of-contents)
   - [Installation](#installation)
   - [Usage](#usage)
   - [Overview](#overview)
@@ -16,15 +18,15 @@ Manage the parameterized prompts (text) used in generative AI (aka chatGPT, Open
       - [What does a keyword look like?](#what-does-a-keyword-look-like)
       - [All about directives](#all-about-directives)
         - [Example Prompt with Directives](#example-prompt-with-directives)
-        - [Accessing Directives](#accessing-directives)
+        - [Accessing Directives and Setting Parameter Values](#accessing-directives-and-setting-parameter-values)
         - [Dynamic Directives](#dynamic-directives)
         - [Executing Directives](#executing-directives)
       - [Comments Are Ignored](#comments-are-ignored)
   - [Storage Adapters](#storage-adapters)
     - [FileSystemAdapter](#filesystemadapter)
       - [Configuration](#configuration)
-        - [prompts_dir](#prompts_dir)
-        - [search_proc](#search_proc)
+        - [prompts\_dir](#prompts_dir)
+        - [search\_proc](#search_proc)
         - [File Extensions](#file-extensions)
       - [Example Prompt Text File](#example-prompt-text-file)
       - [Example Prompt Parameters JSON File](#example-prompt-parameters-json-file)
@@ -32,15 +34,21 @@ Manage the parameterized prompts (text) used in generative AI (aka chatGPT, Open
     - [ActiveRecordAdapter](#activerecordadapter)
       - [Configuration](#configuration-1)
         - [model](#model)
-        - [id_column](#id_column)
-        - [text_column](#text_column)
-        - [parameters_column](#parameters_column)
+        - [id\_column](#id_column)
+        - [text\_column](#text_column)
+        - [parameters\_column](#parameters_column)
     - [Other Potential Storage Adapters](#other-potential-storage-adapters)
   - [Development](#development)
   - [Contributing](#contributing)
   - [License](#license)
 
 <!-- Tocer[finish]: Auto-generated, don't remove. -->
+### Latest Capabilities
+- **Directive Processing:** Processes directives such as `//include` (aliased as `//import`) with loop protection.
+- **ERB Processing:** Supports ERB templating within prompts.
+- **Environment Variable Embedding:** Automatically substitutes system environment variables in prompts.
+- **Improved Parameter Handling:** Refactored to maintain a history of parameter values.
+- **ActiveRecord Adapter:** Facilitates storing and retrieving prompts via an ActiveRecord model.
 
 ## Installation
 
@@ -60,11 +68,19 @@ See also [examples/using_search_proc.rb](examples/using_search_proc.rb)
 
 ## Overview
 
+### Prompt Initialization Options
+- `id`: A String name for the prompt.
+- `context`: An Array for additional context.
+- `directives_processor`: An instance of PromptManager::DirectiveProcessor (default), can be customized.
+- `external_binding`: A Ruby binding to be used for ERB processing.
+- `erb_flag`: Boolean flag to enable ERB processing in the prompt text.
+- `envar_flag`: Boolean flag to enable environment variable substitution in the prompt text.
+
 The `prompt_manager` gem provides functionality to manage prompts that have keywords and directives for use with generative AI processes.
 
 ### Generative AI (gen-AI)
 
-Gen-AI deals with the conversion (some would say execution) of a human natural language text (the "prompt") into somthing else using what are known as large language models (LLM) such as those available from OpenAI.  A parameterized prompt is one in which there are embedded keywords (parameters) which are place holders for other text to be inserted into the prompt.
+Gen-AI deals with the conversion (some would say execution) of a human natural language text (the "prompt") into something else using what are known as large language models (LLM) such as those available from OpenAI.  A parameterized prompt is one in which there are embedded keywords (parameters) which are place holders for other text to be inserted into the prompt.
 
 The prompt_manager uses a regular expression to identify these keywords within the prompt. It uses the keywords as keys in a `parameters` Hash which is stored with the prompt text in a serialized form - for example as JSON.
 
@@ -84,7 +100,7 @@ The regex must include capturing parentheses () to extract the keyword. The defa
 
 A directive is a line in the prompt text that starts with the two characters '//' - slash slash - just like in the old days of IBM JCL - Job Control Language.  A prompt can have zero or more directives.  Directives can have parameters and can make use of keywords.
 
-The `prompt_manager` only collects directives.  It extracts keywords from directive lines and provides the substitution of those keywords with other text just like it does for the prompt.  
+The `prompt_manager` only collects directives.  It extracts keywords from directive lines and provides the substitution of those keywords with other text just like it does for the prompt.
 
 ##### Example Prompt with Directives
 
@@ -102,20 +118,30 @@ __END__
 Computers will never replace Frank Sinatra
 ```
 
-##### Accessing Directives
+##### Accessing Directives and Setting Parameter Values
 
-Getting directives from a prompt is as easy as getting the kewyords:
+Getting directives and keywords from a prompt is straightforward:
 
 ```ruby
-prompt = PromptManager::Prompt.new(...)
-prompt.keywords   #=> an Array
+prompt = PromptManager::Prompt.new(id: 'some_id')
+prompt.keywords   #=> an Array of keywords found in the prompt text
 prompt.directives #=> an Array of entries like: ['directive', 'parameters']
 
+# Or directly update the parameters hash
+prompt.parameters = {
+  "[KEYWORD1]" => "value1",
+  "[KEYWORD2]" => "value2"
+}
+
+# After setting parameter values, call to_s to build the final prompt
 # to_s builds the prompt by substituting
-# values for keywords amd removing comments.
+# values for keywords and removing comments.
 # The resulting text contains directives and
 # prompt text ready for the LLM process.
-puts prompt.to_s  
+final_text = prompt.to_s
+
+# To persist any parameter changes to storage
+prompt.save
 ```
 
 The entries in the Array returned by the `prompt.directives` method is in the order that the directives were defined within the prompt.  Each entry has two elements:
@@ -138,7 +164,7 @@ Since directies are collected after the keywords in the prompt have been substit
 
 The `prompt_manager` gem only collects directives.  Executing those directives is left up to some down stream process.  Here are some ideas on how directives could be used in prompt downstream process:
 
-- "//model gpt-5" could be used to set the LLM model to be used for a specific prompt.  
+- "//model gpt-5" could be used to set the LLM model to be used for a specific prompt.
 - "//backend mods" could be used to set the backend prompt processor on the command line to be the `mods` utility.
 - "//include path_to_file" could be used to add the contents of a file to the prompt.
 - "//chat" could be used to send the prompts and then start up a chat session about the prompt and its response.
@@ -172,8 +198,8 @@ Use a `config` block to establish the configuration for the class.
 
 ```ruby
 PromptManager::Storage::FileSystemAdapter.config do |o|
-  o.prompts_dir       = "path/to/prompts_directory" 
-  o.search_proc       = nil     # default 
+  o.prompts_dir       = "path/to/prompts_directory"
+  o.search_proc       = nil     # default
   o.prompt_extension  = '.txt'  # default
   o.params_extension  = '.json' # default
 end
@@ -183,7 +209,7 @@ The `config` block returns `self` so that means you can do this to setup the sto
 
 ```ruby
 PromptManager::Prompt
-  .storage_adapter = 
+  .storage_adapter =
     PromptManager::Storage::FileSystemAdapter
       .config do |config|
         config.prompts_dir = 'path/to/prompts_dir'
@@ -198,7 +224,7 @@ An `ArgumentError` will be raised when `prompts_dir` does not exist or if it is
 
 ##### search_proc
 
-The default for `search_proc` is nil which means that the search will be preformed by a default `search` method which is basically reading all the prompt files to see which ones contain the search term. It will return an Array of prompt IDs for each prompt file found that contains the search term.  Its up to the application to select which returned prompt ID to use. 
+The default for `search_proc` is nil which means that the search will be preformed by a default `search` method which is basically reading all the prompt files to see which ones contain the search term. It will return an Array of prompt IDs for each prompt file found that contains the search term.  Its up to the application to select which returned prompt ID to use.
 
 There are faster ways to search and select files.  For example there are specialized search and selection utilities that are available for the command line. The `examples` directory contains a `bash` script named `rgfzf` that uses `rg` (aka `ripgrep`) to do the searching and `fzf` to do the selecting.
 
@@ -273,7 +299,7 @@ The `PromptManager::Prompt` class expects an instance of a storage adapter class
 
 ```ruby
 PromptManager::Prompt
-  .storage_adapter = 
+  .storage_adapter =
     PromptManager::Storage::ActiveRecordAdapter.config do |config|
       config.model              = DbPromptModel # any ActiveRecord::Base model
       config.id_column          = :prompt_name

data/lib/prompt_manager/directive_processor.rb

@@ -0,0 +1,47 @@
+# lib/prompt_manager/directive_processor.rb
+
+# This is an example of a directive processor class.
+# It only supports the //include directive which is also
+# aliased as //import.
+
+module PromptManager
+  class DirectiveProcessor
+    EXCLUDED_METHODS = %w[ run initialize ]
+
+    def initialize
+      @prefix_size    = PromptManager::Prompt::DIRECTIVE_SIGNAL.size
+      @included_files = []
+    end
+
+    def run(directives)
+      return {} if directives.nil? || directives.empty?
+      directives.each do |key, _|
+        sans_prefix = key[@prefix_size..]
+        args        = sans_prefix.split(' ')
+        method_name = args.shift
+
+        if EXCLUDED_METHODS.include?(method_name)
+          directives[key] = "Error: #{method_name} is not a valid directive: #{key}"
+        elsif respond_to?(method_name)
+          directives[key] =  send(method_name, *args)
+        else
+          directives[key] = "Error: Unknown directive '#{key}'"
+        end
+      end
+      directives
+    end
+
+    def include(file_path)
+      if  File.exist?(file_path) &&
+          File.readable?(file_path) &&
+          !@included_files.include?(file_path)
+        content = File.read(file_path)
+        @included_files << file_path
+        content
+      else
+        "Error: File '#{file_path}' not accessible"
+      end
+    end
+    alias_method :import, :include
+  end
+end

data/lib/prompt_manager/prompt.rb

@@ -1,34 +1,22 @@
 # prompt_manager/lib/prompt_manager/prompt.rb
 
-# This class is responsible for managing prompts which can be utilized by
-# generative AI processes. This includes creation, retrieval, storage management,
-# as well as building prompts with replacement of parameterized values and 
-# comment removal. It communicates with a storage system through a storage 
-# adapter.
-#
-# Directives are collected into an Array where each entry is an Array
-# of two elements.  The first is the directive name as a String. The 
-# second is a string of parameters used by the directive.
-# 
-# Directives are collected from the prompt after keyword
-# substitution has occured.  This means that directives within a
-# prompt can be dynamic.
-#
-# PromptManager does not execute directives.  They
-# are made available to be passed on to down stream
-# process.
+require_relative "directive_processor"
 
 class PromptManager::Prompt
-  COMMENT_SIGNAL    = '#'   # lines beginning with this are a comment
-  DIRECTIVE_SIGNAL  = '//'  # Like the old IBM JCL
+  COMMENT_SIGNAL          = '#'   # lines beginning with this are a comment
+  DIRECTIVE_SIGNAL        = '//'  # Like the old IBM JCL
   DEFAULT_PARAMETER_REGEX = /(\[[A-Z _|]+\])/
-  @storage_adapter  = nil
-  @parameter_regex  = DEFAULT_PARAMETER_REGEX
+  @parameter_regex        = DEFAULT_PARAMETER_REGEX
+
+  ##############################################
+  ## Public class methods
 
   class << self
     attr_accessor :storage_adapter, :parameter_regex
 
-    alias_method :get, :new
+    def get(id:)
+      storage_adapter.get(id: id)  # Return the hash directly from storage
+    end
 
     def create(id:, text: "", parameters: {})
       storage_adapter.save(
@@ -37,15 +25,22 @@ class PromptManager::Prompt
         parameters: parameters
       )
 
-      new(id: id)
+      ::PromptManager::Prompt.new(id: id, context: [], directives_processor: PromptManager::DirectiveProcessor.new)
     end
 
+    def find(id:)
+      ::PromptManager::Prompt.new(id: id, context: [], directives_processor: PromptManager::DirectiveProcessor.new)
+    end
+
+    def destroy(id:)
+      prompt = find(id: id)
+      prompt.delete
+    end
 
     def search(for_what)
       storage_adapter.search(for_what)
     end
 
-
     def method_missing(method_name, *args, &block)
       if storage_adapter.respond_to?(method_name)
         storage_adapter.send(method_name, *args, &block)
@@ -54,176 +49,108 @@ class PromptManager::Prompt
       end
     end
 
-
     def respond_to_missing?(method_name, include_private = false)
       storage_adapter.respond_to?(method_name, include_private) || super
     end
   end
-  
-  # SMELL:  Does the db (aka storage adapter) really need
-  #         to be accessible by the main program?
-  attr_accessor :db, :id, :text, :parameters, :directives
+
+  ##############################################
+  ## Public Instance Methods
+
+  attr_accessor :id,           # String name for the prompt
+                :text,         # String, full text of the prompt
+                :parameters    # Hash, Key and Value are Strings
 
 
-  # Retrieve the specific prompt ID from the Storage system.
   def initialize(
-      id:       nil,  # A String name for the prompt
-      context:  []    # FIXME: Array of Strings or Pathname?
+      id:       nil,    # A String name for the prompt
+      context:  [],     # TODO: Array of Strings or Pathname?
+      directives_processor: PromptManager::DirectiveProcessor.new,
+      external_binding:     binding,
+      erb_flag:             false,    # replace $ENVAR and ${ENVAR} when true
+      envar_flag:           false     # process ERB against the external_binding when true
     )
 
     @id  = id
-    @db  = self.class.storage_adapter
-    
-    validate_arguments(@id, @db)
+    @directives_processor = directives_processor
 
-    @record     = db.get(id: id)
-    @text       = @record[:text]
-    @parameters = @record[:parameters]
-    @keywords   = []  # Array of String
-    @directives = []  # Array of arrays. directive is first entry, rest are parameters
+    validate_arguments(@id)
 
-    update_keywords
-
-    build
+    @record           = db.get(id: id)
+    @text             = @record[:text]        || ""
+    @parameters       = @record[:parameters]  || {}
+    @directives       = {}
+    @external_binding = external_binding
+    @erb_flag         = erb_flag
+    @envar_flag       = envar_flag
   end
 
-
-  # Make sure the ID and DB are good-to-go
-  def validate_arguments(prompt_id, prompts_db)
-    raise ArgumentError, 'id cannot be blank'           if prompt_id.nil? || id.strip.empty?
+  def validate_arguments(prompt_id, prompts_db=db)
+    raise ArgumentError, 'id cannot be blank'           if prompt_id.nil? || prompt_id.strip.empty?
     raise(ArgumentError, 'storage_adapter is not set')  if prompts_db.nil?
   end
 
-
-  # Return tje prompt text suitable for passing to a
-  # gen-AI process.
   def to_s
-    build
+    processed_text = remove_comments
+    processed_text = substitute_values(processed_text, @parameters)
+    processed_text = substitute_env_vars(processed_text)
+    processed_text = process_directives(processed_text)
+    process_erb(processed_text)
   end
-  alias_method :prompt, :to_s
-
 
-  # Save the prompt to the Storage system
   def save
     db.save(
-      id:         id, 
-      text:       text, 
+      id:         id,
+      text:       text,       # Save the original text
       parameters: parameters
-    )    
-  end
-
-
-  # Delete this prompt from the Storage system
-  def delete
-    db.delete(id: id)  
-  end
-
-
-  # Build the @prompt String by replacing the keywords
-  # with there parameterized values and removing all
-  # the comments.
-  #  
-  def build
-    @prompt = text.gsub(self.class.parameter_regex) do |match|
-                param_name = match
-                Array(parameters[param_name]).last || match
-              end
-              
-    save_directives(@prompt)
-    remove_comments
+    )
   end
 
-
-  def keywords
-    update_keywords
-  end
+  def delete = db.delete(id: id)
 
 
   ######################################
   private
 
-  def update_keywords
-    @keywords = @text.scan(self.class.parameter_regex).flatten.uniq
-    @keywords.each do |kw|
-      @parameters[kw] = [] unless @parameters.has_key?(kw)
-    end
+  def db = self.class.storage_adapter
 
-    @keywords
+  def remove_comments
+    lines = @text.split("\n")
+    end_index = lines.index("__END__") || lines.size
+    lines[0...end_index].reject { |line| line.strip.start_with?(COMMENT_SIGNAL) }.join("\n")
   end
 
-
-  def save_directives(keyword_substituted_string)
-    @directives = []
-
-    keyword_substituted_string.split("\n").each do |a_line|
-      line = a_line.strip
-      next unless line.start_with?(DIRECTIVE_SIGNAL)
-      
-      parts       = line.split(' ')
-      directive   = parts.shift[DIRECTIVE_SIGNAL.length..] # drop the directive signal
-      @directives << [directive, parts.join(' ')]
+  def substitute_values(input_text, values_hash)
+    if values_hash.is_a?(Hash) && !values_hash.empty?
+      values_hash.each do |key, value|
+        input_text = input_text.gsub(key, value)
+      end
     end
-
-    @directives
+    input_text
   end
 
+  def erb?      = @erb_flag
+  def envvar?   = @envvar_flag
 
-  def remove_comments
-    lines           = @prompt
-                        .split("\n")
-                        .reject{|a_line| 
-                          a_line.strip.start_with?(COMMENT_SIGNAL)  ||
-                          a_line.strip.start_with?(DIRECTIVE_SIGNAL)
-                        }
-
-    # Remove empty lines at the start of the prompt
-    #
-    lines = lines.drop_while(&:empty?)
-
-    # Drop all the lines at __END__ and after
-    #
-    logical_end_inx = lines.index("__END__")
-
-    @prompt = if logical_end_inx
-                lines[0...logical_end_inx] # NOTE: ... means to not include last index
-              else
-                lines
-              end.join("\n") 
-  end
-  
-
-  # Handle storage errors
-  # SMELL:  Just raise them or get out of their way and let the
-  #         main program do tje job.
-  def handle_storage_error(error)
-    # Log the error message, notify, or take appropriate action
-    log_error("Storage operation failed: #{error.message}")
-    # Re-raise the error if necessary, or define recovery steps
-    raise error
-  end
-
+  def substitute_env_vars(input_text)
+    return input_text unless envvar?
 
-  # Let the storage adapter instance take a crake at
-  # these unknown methods.  Don't care what the args
-  # are, just pass the prompt's ID.
-  def method_missing(method_name, *args, &block)
-    if db.respond_to?(method_name)
-      db.send(method_name, id, &block)
-    else
-      super
+    input_text.gsub(/\$(\w+)|\$\{(\w+)\}/) do |match|
+      env_var = $1 || $2
+      ENV[env_var] || match
     end
   end
 
-
-  def respond_to_missing?(method_name, include_private = false)
-    db.respond_to?(method_name, include_private) || super
+  def process_directives(input_text)
+    directive_lines = input_text.split("\n").select { |line| line.strip.start_with?(DIRECTIVE_SIGNAL) }
+    @directives = directive_lines.each_with_object({}) { |line, hash| hash[line.strip] = "" }
+    @directives = @directives_processor.run(@directives)
+    substitute_values(input_text, @directives)
   end
 
+  def process_erb(input_text)
+    return input_text unless erb?
 
-  # SMELL:  should this gem log errors or is that a function of
-  #         main program?  I believe its the main program's job.
-  def log_error(message)
-    puts "ERROR: #{message}"
+    ERB.new(input_text).result(@external_binding)
   end
 end
-

data/lib/prompt_manager/storage/active_record_adapter.rb

@@ -1,18 +1,28 @@
 # prompt_manager/lib/prompt_manager/storage/active_record_adapter.rb
 
 # This class acts as an adapter for interacting with an ActiveRecord model
+require 'active_record'
 # to manage storage operations for PromptManager::Prompt instances. It defines
 # methods that allow for saving, searching, retrieving by ID, and deleting
 # prompts.
+#
+# To use this adapter, you must configure it with an ActiveRecord model and
+# the column names for ID, text content, and parameters. The adapter will
+# handle serialization and deserialization of parameters.
+#
+# This adapter is used by PromptManager::Prompt as its storage backend, enabling CRUD operations on persistent prompt data.
 
 class PromptManager::Storage::ActiveRecordAdapter
   
   class << self
-    attr_accessor :model, 
-                  :id_column, 
-                  :text_column, 
+    # Configure the ActiveRecord model and column mappings
+    attr_accessor :model,
+                  :id_column,
+                  :text_column,
                   :parameters_column
 
+    # Configure the adapter with the required settings
+    # Must be called with a block before using the adapter
     def config
       if block_given?
         yield self
@@ -24,19 +34,19 @@ class PromptManager::Storage::ActiveRecordAdapter
       self
     end
 
-
+    # Validate that all required configuration is present and valid
     def validate_configuration
       validate_model
       validate_columns
     end
 
-
+    # Ensure the provided model is a valid ActiveRecord model
     def validate_model
       raise ArgumentError, "AR Model not set" unless model
       raise ArgumentError, "AR Model is not an ActiveRecord model" unless model < ActiveRecord::Base
     end
 
-
+    # Verify that all required columns exist in the model
     def validate_columns
       columns = model.column_names # Array of Strings
       [id_column, text_column, parameters_column].each do |column|
@@ -44,8 +54,7 @@ class PromptManager::Storage::ActiveRecordAdapter
       end
     end
 
-
-
+    # Delegate unknown methods to the ActiveRecord model
     def method_missing(method_name, *args, &block)
       if model.respond_to?(method_name)
         model.send(method_name, *args, &block)
@@ -54,7 +63,7 @@ class PromptManager::Storage::ActiveRecordAdapter
       end
     end
 
-
+    # Support respond_to? for delegated methods
     def respond_to_missing?(method_name, include_private = false)
       model.respond_to?(method_name, include_private) || super
     end
@@ -62,30 +71,29 @@ class PromptManager::Storage::ActiveRecordAdapter
   
 
   ##############################################
+  # The ActiveRecord object representing the current prompt
   attr_accessor :record
 
+  # Accessor methods to avoid repeated self.class prefixes
+  def model             = self.class.model
+  def id_column         = self.class.id_column
+  def text_column       = self.class.text_column
+  def parameters_column = self.class.parameters_column
 
-  # Avoid code littered with self.class prefixes ...
-  def model               = self.class.model
-  def id_column    = self.class.id_column
-  def text_column  = self.class.text_column
-  def parameters_column   = self.class.parameters_column
-
-
+  # Initialize the adapter and validate configuration
   def initialize
     self.class.send(:validate_configuration) # send gets around private designations of a method
     @record = model.first
   end
 
-
+  # Retrieve a prompt by its ID
+  # Returns a hash with id, text, and parameters
   def get(id:)
     @record = model.find_by(id_column => id)
     raise ArgumentError, "Prompt not found with id: #{id}" unless @record
 
-    # kludge? testing showed that parameters was being
-    # returned as a String.  Did serialization fail or is
-    # there something else going on?
-    # FIXME: expected the parameters_column to be a HAsh after de-serialization
+    # Handle case where parameters might be stored as a JSON string
+    # instead of a native Hash
     parameters = @record[parameters_column]
 
     if parameters.is_a? String
@@ -93,54 +101,57 @@ class PromptManager::Storage::ActiveRecordAdapter
     end
 
     {
-      id:         id, # same as the id_column
+      id:         id,
       text:       @record[text_column],
       parameters: parameters
     }
   end
 
-
+  # Save a prompt with the given ID, text, and parameters
+  # Creates a new record if one doesn't exist, otherwise updates existing record
   def save(id:, text: "", parameters: {})
     @record = model.find_or_initialize_by(id_column => id) 
 
     @record[text_column] = text
-    @record[parameters_column]  = parameters
+    @record[parameters_column] = parameters
     @record.save!
   end
 
-
+  # Delete a prompt with the given ID
   def delete(id:)
     @record = model.find_by(id_column => id)
     @record&.destroy
   end
 
-
-  
+  # Return an array of all prompt IDs
   def list(*)
     model.all.pluck(id_column)
   end
 
-
+  # Search for prompts containing the given text
+  # Returns an array of matching prompt IDs
   def search(for_what)
     model.where("#{text_column} LIKE ?", "%#{for_what}%").pluck(id_column)
   end
 
-
   ##############################################
   private
 
-
+  # Delegate unknown methods to the current record
   def method_missing(method_name, *args, &block)
-    if @record.respond_to?(method_name)
-      model.send(method_name, args.first, &block)
+    if @record && @record.respond_to?(method_name)
+      @record.send(method_name, *args, &block)
+    elsif model.respond_to?(method_name)
+      model.send(method_name, *args, &block)
     else
       super
     end
   end
 
-
+  # Support respond_to? for delegated methods
   def respond_to_missing?(method_name, include_private = false)
-    model.respond_to?(method_name, include_private) || super
+    (model.respond_to?(method_name, include_private) ||
+     (@record && @record.respond_to?(method_name, include_private)) ||
+     super)
   end
 end
-

data/lib/prompt_manager/storage/file_system_adapter.rb

@@ -3,36 +3,45 @@
 # Use the local (or remote) file system as a place to
 # store and access prompts.
 #
-# Adds two additional methods to the Promp class:
+# Adds two additional methods to the Prompt class:
 #   list - returns Array of prompt IDs
-#   path = returns a Pathname object to the prompt's text file
+#   path - returns a Pathname object to the prompt's text file
 #   path(prompt_id) - same as path on the prompt instance
 #
 # Allows sub-directories of the prompts_dir to be
-# used like categories.  For example the prompt_id "toy/magic"
+# used like categories. For example the prompt_id "toy/magic"
 # is found in the `magic.txt` file inside the `toy` sub-directory
 # of the prompts_dir.
 #
-# There can man be many layers of categories (sub-directories)
+# There can be many layers of categories (sub-directories)
 #
+# This adapter serves as the file-based storage backend for PromptManager::Prompt,
+# enabling prompt retrieval and persistence using file system operations.
 
 require 'json'      # basic serialization of parameters
 require 'pathname'
+require 'fileutils'
 
 class PromptManager::Storage::FileSystemAdapter
-  SEARCH_PROC       = nil # placeholder
+  # Placeholder for search proc
+  SEARCH_PROC       = nil
+  # File extension for parameters
   PARAMS_EXTENSION  = '.json'.freeze
+  # File extension for prompts
   PROMPT_EXTENSION  = '.txt'.freeze
+  # Regular expression for valid prompt IDs
   PROMPT_ID_FORMAT  = /^[a-zA-Z0-9\-\/_]+$/
 
   class << self
-    attr_accessor :prompts_dir, :search_proc, 
+    # Accessors for configuration options
+    attr_accessor :prompts_dir, :search_proc,
                   :params_extension, :prompt_extension
-  
+
+    # Configure the adapter
     def config
       if block_given?
         yield self
-        validate_configuration     
+        validate_configuration
       else
         raise ArgumentError, "No block given to config"
       end
@@ -48,7 +57,6 @@ class PromptManager::Storage::FileSystemAdapter
       new.list
     end
 
-
     def path(prompt_id)
       new.path(prompt_id)
     end
@@ -56,6 +64,7 @@ class PromptManager::Storage::FileSystemAdapter
     #################################################
     private
 
+    # Validate the configuration
     def validate_configuration
       validate_prompts_dir
       validate_search_proc
@@ -63,28 +72,33 @@ class PromptManager::Storage::FileSystemAdapter
       validate_params_extension
     end
 
-
+    # Validate the prompts directory
     def validate_prompts_dir
-      # This is a work around for a Ruby scope issue where the 
-      # class getter/setter method is becoming confused with a 
-      # local variable when anything other than plain 'ol get and 
-      # set are used.'This error is in both Ruby v3.2.2 and
+      # This is a work around for a Ruby scope issue where the
+      # class getter/setter method is becoming confused with a
+      # local variable when anything other than plain 'ol get and
+      # set are used. This error is in both Ruby v3.2.2 and
       # v3.3.0-preview3.
       #
       prompts_dir_local = self.prompts_dir
 
+      raise ArgumentError, "prompts_dir must be set" if prompts_dir_local.nil? || prompts_dir_local.to_s.strip.empty?
+
       unless prompts_dir_local.is_a?(Pathname)
-        prompts_dir_local = Pathname.new(prompts_dir_local) unless prompts_dir_local.nil?
+        prompts_dir_local = Pathname.new(prompts_dir_local)
       end
 
       prompts_dir_local = prompts_dir_local.expand_path
 
-      raise(ArgumentError, "prompts_dir: #{prompts_dir_local}") unless prompts_dir_local.exist? && prompts_dir_local.directory?
-      
+      unless prompts_dir_local.exist?
+        FileUtils.mkdir_p(prompts_dir_local)
+      end
+      raise(ArgumentError, "prompts_dir: #{prompts_dir_local} is not a directory") unless prompts_dir_local.directory?
+
       self.prompts_dir = prompts_dir_local
     end
 
-
+    # Validate the search proc
     def validate_search_proc
       search_proc_local = self.search_proc
 
@@ -97,7 +111,7 @@ class PromptManager::Storage::FileSystemAdapter
       self.search_proc = search_proc_local
     end
 
-
+    # Validate the prompt extension
     def validate_prompt_extension
       prompt_extension_local = self.prompt_extension
 
@@ -113,7 +127,7 @@ class PromptManager::Storage::FileSystemAdapter
       self.prompt_extension = prompt_extension_local
     end
 
-
+    # Validate the params extension
     def validate_params_extension
       params_extension_local = self.params_extension
 
@@ -130,25 +144,25 @@ class PromptManager::Storage::FileSystemAdapter
     end
   end
 
-
   ##################################################
   ###
   ##  Instance
   #
 
+  # Accessors for instance variables
   def prompts_dir       = self.class.prompts_dir
   def search_proc       = self.class.search_proc
   def prompt_extension  = self.class.prompt_extension
   def params_extension  = self.class.params_extension
 
-
+  # Initialize the adapter
   def initialize
     # NOTE: validate because main program may have made
     #       changes outside of the config block
     self.class.send(:validate_configuration) # send gets around private designations of a method
   end
 
-
+  # Get a prompt by ID
   def get(id:)
     validate_id(id)
     verify_id(id)
@@ -160,17 +174,15 @@ class PromptManager::Storage::FileSystemAdapter
     }
   end
 
-
   # Retrieve prompt text by its id
   def prompt_text(prompt_id)
     read_file(file_path(prompt_id, prompt_extension))
   end
 
-
   # Retrieve parameter values by its id
   def parameter_values(prompt_id)
     params_path = file_path(prompt_id, params_extension)
-    
+
     if params_path.exist?
       parms_content = read_file(params_path)
       deserialize(parms_content)
@@ -179,35 +191,33 @@ class PromptManager::Storage::FileSystemAdapter
     end
   end
 
-
   # Save prompt text and parameter values to corresponding files
   def save(
-      id:, 
-      text: "", 
+      id:,
+      text: "",
       parameters: {}
     )
     validate_id(id)
 
     prompt_filepath = file_path(id, prompt_extension)
     params_filepath = file_path(id, params_extension)
-    
+
     write_with_error_handling(prompt_filepath, text)
     write_with_error_handling(params_filepath, serialize(parameters))
   end
 
-
-  # Delete prompted text and parameter values files
+  # Delete prompt text and parameter values files
   def delete(id:)
     validate_id(id)
 
     prompt_filepath = file_path(id, prompt_extension)
     params_filepath = file_path(id, params_extension)
-    
+
     delete_with_error_handling(prompt_filepath)
     delete_with_error_handling(params_filepath)
   end
 
-
+  # Search for prompts
   def search(for_what)
     search_term = for_what.downcase
 
@@ -218,25 +228,23 @@ class PromptManager::Storage::FileSystemAdapter
     end
   end
 
-
   # Return an Array of prompt IDs
   def list(*)
     prompt_ids = []
-    
+
     Pathname.glob(prompts_dir.join("**/*#{prompt_extension}")).each do |file_path|
       prompt_id = file_path.relative_path_from(prompts_dir).to_s.gsub(prompt_extension, '')
       prompt_ids << prompt_id
     end
 
-    prompt_ids
+    prompt_ids.sort
   end
 
-
   # Returns a Pathname object for a prompt ID text file
   # However, it is possible that the file does not exist.
   def path(id)
     validate_id(id)
-    file_path(id, prompt_extension) 
+    file_path(id, prompt_extension)
   end
 
   ##########################################
@@ -247,14 +255,14 @@ class PromptManager::Storage::FileSystemAdapter
     raise ArgumentError, "Invalid ID format id: #{id}" unless id =~ PROMPT_ID_FORMAT
   end
 
-
+  # Verify that the ID exists
   def verify_id(id)
     unless file_path(id, prompt_extension).exist?
       raise ArgumentError, "Invalid prompt_id: #{id}"
     end
   end
 
-
+  # Write to a file with error handling
   def write_with_error_handling(file_path, content)
     begin
       file_path.write content
@@ -264,8 +272,7 @@ class PromptManager::Storage::FileSystemAdapter
     end
   end
 
-
-  # file_path (Pathname)
+  # Delete a file with error handling
   def delete_with_error_handling(file_path)
     begin
       file_path.delete
@@ -275,39 +282,37 @@ class PromptManager::Storage::FileSystemAdapter
     end
   end
 
-
+  # Get the file path for a prompt ID and extension
   def file_path(id, extension)
     prompts_dir + "#{id}#{extension}"
   end
 
-
+  # Read a file
   def read_file(full_path)
     raise IOError, 'File does not exist' unless File.exist?(full_path)
     File.read(full_path)
   end
 
-
+  # Search for prompts
   def search_prompts(search_term)
     prompt_ids = []
-    
+
     Pathname.glob(prompts_dir.join("**/*#{prompt_extension}")).each do |prompt_path|
       if prompt_path.read.downcase.include?(search_term)
-        prompt_id = prompt_path.relative_path_from(prompts_dir).to_s.gsub(prompt_extension, '')    
+        prompt_id = prompt_path.relative_path_from(prompts_dir).to_s.gsub(prompt_extension, '')
         prompt_ids << prompt_id
       end
     end
 
-    prompt_ids
+    prompt_ids.sort
   end
 
-
-  # TODO: Should the serializer be generic?
-
+  # Serialize data to JSON
   def serialize(data)
     data.to_json
   end
 
-
+  # Deserialize JSON data
   def deserialize(data)
     JSON.parse(data)
   end

data/lib/prompt_manager/storage.rb

@@ -1,7 +1,34 @@
 # prompt_manager/lib/prompt_manager/storage.rb
 
+# The Storage module provides a namespace for different storage adapters
+# that handle persistence of prompts. Each adapter implements a common
+# interface for saving, retrieving, searching, and deleting prompts.
+#
+# Available adapters:
+# - FileSystemAdapter: Stores prompts in text files on the local filesystem
+# - ActiveRecordAdapter: Stores prompts in a database using ActiveRecord
+#
+# To use an adapter, configure it before using PromptManager::
+#
+# Example with FileSystemAdapter:
+#   PromptManager::Storage::FileSystemAdapter.config do |config|
+#     config.prompts_dir = Pathname.new('/path/to/prompts')
+#   end
+#   PromptManager::Prompt.storage_adapter = PromptManager::Storage::FileSystemAdapter.new
+#
+# Example with ActiveRecordAdapter:
+#   PromptManager::Storage::ActiveRecordAdapter.config do |config|
+#     config.model = MyPromptModel
+#     config.id_column = :prompt_id
+#     config.text_column = :content
+#     config.parameters_column = :params
+#   end
+#   PromptManager::Prompt.storage_adapter = PromptManager::Storage::ActiveRecordAdapter.new
 module PromptManager
+  # The Storage module provides adapters for different storage backends.
+  # Each adapter implements a common interface for managing prompts.
+  # Note: PromptManager::Prompt uses one of these adapters as its storage backend to
+  # perform all CRUD operations on prompt data.
   module Storage
   end
 end
-

data/lib/prompt_manager/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module PromptManager
-  VERSION = "0.4.2"
+  VERSION = "0.5.0"
 end

data/lib/prompt_manager.rb

@@ -5,10 +5,20 @@
 require 'ostruct'
 
 require_relative "prompt_manager/version"
-require_relative "prompt_manager/storage"
 require_relative "prompt_manager/prompt"
+require_relative "prompt_manager/storage"
+require_relative "prompt_manager/storage/file_system_adapter"
 
+# The PromptManager module provides functionality for managing, storing,
+# retrieving, and parameterizing text prompts used with generative AI systems.
+# It supports different storage backends through adapters and offers features
+# like parameter substitution, directives processing, and comment handling.
 module PromptManager
+  # Base error class for all PromptManager-specific errors
   class Error < StandardError; end
-  # TODO: Add some more module specific errors here
+
+  # TODO: Add additional module-specific error classes such as:
+  # - StorageError - For issues with storing or retrieving prompts
+  # - ParameterError - For issues with parameter substitution
+  # - ConfigurationError - For setup and configuration issues
 end

metadata

@@ -1,14 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: prompt_manager
 version: !ruby/object:Gem::Version
-  version: 0.4.2
+  version: 0.5.0
 platform: ruby
 authors:
 - Dewayne VanHoozer
-autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-10-26 00:00:00.000000000 Z
+date: 2025-03-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -139,7 +138,6 @@ files:
 - README.md
 - Rakefile
 - examples/directives.rb
-- examples/prompts_dir/directive_example.txt
 - examples/prompts_dir/todo.json
 - examples/prompts_dir/todo.txt
 - examples/prompts_dir/toy/8-ball.txt
@@ -147,6 +145,7 @@ files:
 - examples/simple.rb
 - examples/using_search_proc.rb
 - lib/prompt_manager.rb
+- lib/prompt_manager/directive_processor.rb
 - lib/prompt_manager/prompt.rb
 - lib/prompt_manager/storage.rb
 - lib/prompt_manager/storage/active_record_adapter.rb
@@ -160,7 +159,6 @@ metadata:
   homepage_uri: https://github.com/MadBomber/prompt_manager
   source_code_uri: https://github.com/MadBomber/prompt_manager
   changelog_uri: https://github.com/MadBomber/prompt_manager
-post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -175,8 +173,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.5.22
-signing_key:
+rubygems_version: 3.6.6
 specification_version: 4
 summary: Manage prompts for use with gen-AI processes
 test_files: []

data/examples/prompts_dir/directive_example.txt

@@ -1,14 +0,0 @@
-# directive_example.txt
-# Desc: Shows how directives work
-
-//good_directive  param1 param2
-//bad_directive   param3 param4
-
-say hello to me in {language} as well as English.
-
-# The default parameter is delimited by square brackets
-# and is all uppercase
-
-write a [PROGRAMMING LANGUAGE] program that predicts the lottery.
-
-