prompt_manager 0.1.1 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8b3b9572e08d0f90b7857d9451a13e0ed5d3b089761a1ce820d3b4cf9a47b805
-  data.tar.gz: aa374359e5200177d631ce9607bac8559529bdc968a661820110017021414605
+  metadata.gz: dbcdfe0c23335596eaad240b45fdb1451cab1be2569e511d21ddaf58ea986128
+  data.tar.gz: ce09536239ef6d06c8e42a120d0fd6661f3e42b72a4e9bea6dc998be12024956
 SHA512:
-  metadata.gz: 992bc7f9e337d3fadbc99d0d192a325640bc113f244dd0bf448273398717e9e0d60d0e2a2ee02d18f617e3c9687efaee8fc36c6594f0a3819ca9e877140d6164
-  data.tar.gz: a7d7891c715aa4beea562a82135e2f945d36d4e2132eb39efaf71a8795572dc804ef55bd253a34734a4d84bfb1e516fd6f973dc0c62da0b63fb4ad59cd18f5c2
+  metadata.gz: 1a603d11d09c89ba70e4577db65e0feb31246406dadb0b118717a51b1b4831d54dde782e90511a4d2e44733dd255c501296c97639b446d8f458903d391e3aa0a
+  data.tar.gz: bf48bf56a2e28ccde3ec5673ca9d54e5994ca4b7b71b35714177b2fcb8fbe07bf64521e5f4e9d8d1faf0ca4ec86f8ed98c5f879d532245e8978c200b319fee71

data/CHANGELOG.md

@@ -1,5 +1,10 @@
 ## [Unreleased]
 
+## [0.2.0] - 2023-11-21
+
+- **Breaking change to FileSystemAdapter config process**
+- added list and path as extra methods in FileSystemAdapter
+
 ## [0.1.0] - 2023-11-16
 
 - Initial release using the FileSystemAdapter

data/README.md

@@ -2,6 +2,33 @@
 
 Manage the parameterized prompts (text) used in generative AI (aka chatGPT, OpenAI, _et.al._) using storage adapters such as FileSystemAdapter, SqliteAdapter and ActiveRecordAdapter.
 
+**Breaking Change** in version 0.2.0 for `FileSystemAdapter` configuration. See [Configuration](#configuration) to see how the new `config` block works.
+
+<!-- Tocer[start]: Auto-generated, don't remove. -->
+
+## Table of Contents
+
+  - [Installation](#installation)
+  - [Usage](#usage)
+  - [Overview](#overview)
+    - [Generative AI (gen-AI)](#generative-ai-gen-ai)
+      - [What does a keyword look like?](#what-does-a-keyword-look-like)
+    - [Storage Adapters](#storage-adapters)
+      - [FileSystemAdapter](#filesystemadapter)
+        - [Configuration](#configuration)
+          - [prompts_dir](#prompts_dir)
+          - [search_proc](#search_proc)
+          - [File Extensions](#file-extensions)
+        - [Extra Functionality](#extra-functionality)
+      - [SqliteAdapter](#sqliteadapter)
+      - [ActiveRecordAdapter](#activerecordadapter)
+      - [Other Potential Storage Adapters](#other-potential-storage-adapters)
+  - [Development](#development)
+  - [Contributing](#contributing)
+  - [License](#license)
+
+<!-- Tocer[finish]: Auto-generated, don't remove. -->
+
 ## Installation
 
 Install the gem and add to the application's Gemfile by executing:
@@ -34,6 +61,8 @@ This is just the initial convention adopted by prompt_manager. It is intended th
 
 A storage adapter is a class instance that ties the `PromptManager::Prompt` class to a storage facility that holds the actual prompts. Currently there are 3 storage adapters planned for implementation.
 
+The `PromptManager::Prompt` to support a small set of methods.  A storage adapter can provide "extra" class or instance methods that can be used through the Prompt class.  See the `test/prompt_manager/prompt_test.rb` for guidance on creating a new storage adapter.
+
 #### FileSystemAdapter
 
 This is the first storage adapter developed. It saves prompts as text files within the file system inside a designated `prompts_dir` (directory) such as `~/.prompts` or where it makes the most sense to you.  Another example would be to have your directory on a shared file system so that others can use the same prompts.
@@ -42,6 +71,62 @@ The `prompt ID` is the basename of the text file. For example `todo.txt` is the
 
 The parameters for the `todo` prompt ID are saved in the same directory as `todo.txt` in a JSON file named `todo.json` (also in the examples directory.)
 
+##### Configuration
+
+Use a `config` block toe establish the configuration for the class.
+
+```ruby
+PromptManager::Storage::FileSystemAdapter.config do |o|
+  o.prompts_dir = "path/to/prompts_directory" 
+  o.search_proc = -> (q) { "ag -l #{q} #{prompts_dir} | reformat" } 
+  o.prompt_extension = '.txt' # default
+  o.params_extension = '.json' # the default
+end
+```
+
+The `config` block returns `self` so that means you can do this to setup the storage adapter with the Prompt class:
+
+```ruby
+PromptManager::Prompt
+  .storage_adapter = 
+    PromptManager::Storage::FileSystemAdapter
+      .config do |config|
+        config.prompts_dir = 'path/to/prompts_dir'
+      end.new
+```
+
+###### prompts_dir
+
+This is either a `String` or a `Pathname` object.  All file paths are maintained in the class as `Pathname` objects.  If you provide a `String` it will be converted.  Relative paths will be converted to absolute paths.
+
+An `ArgumentError` will be raised when `prompts_dir` does not exist or if it is not a directory.
+
+###### search_proc
+
+The default for `search_proc` is nil.  In this case the search will be preformed by a default `search` method which is basically reading all the prompt files to see which ones contain the search term.  There are faster ways to do this kind of thing using CLI=based utilities.
+
+TODO: add a example to the examples directory on how to integrate with command line utilities.
+
+###### File Extensions
+
+These two configuration options are `String` objects that must start with a period "." utherwise an `ArgumentError` will be raised.
+
+* prompt_extension - default: '.txt'
+* params_extension - default: '.json'
+
+Currently the `FileSystemAdapter` only supports a JSON serializer for its parameters Hash.  Using any other values for these extensions will cause problems.
+
+They exist so that there is a platform on to which other storage adapters can be built or serializers added.  This is not currently on the roadmap.
+
+##### Extra Functionality
+
+The `FileSystemAdapter` adds two new methods for use by the `Prompt` class:
+* list - returns an Array of prompt IDs
+* path and path(prompt_id) - returns a `Pathname` object to the prompt file
+
+Use the `path(prompt_id)` form against the `Prompt` class
+Use `prompt.path` when you have an instance of a `Prompt`
+
 #### SqliteAdapter
 
 TODO: This may be the next adapter to be implemented.

data/Rakefile

@@ -1,5 +1,14 @@
 # frozen_string_literal: true
 
+begin
+  require "tocer/rake/register"
+rescue LoadError => error
+  puts error.message
+end
+
+Tocer::Rake::Register.call
+
+
 require "bundler/gem_tasks"
 require "rake/testtask"
 

data/examples/prompts_dir/toy/8-ball.txt

@@ -0,0 +1,4 @@
+# prompt_manager/examples/prompts_dir/toy/8-ball.txt
+# Desc: In the late 1940s the toy "Magic 8 Ball" came to market
+
+As a magic 8-ball, provide me with a terse answer to what you suspect that I am thinking about.

data/examples/simple.rb

@@ -9,7 +9,9 @@
 ##  By:   Dewayne VanHoozer (dvanhoozer@gmail.com)
 ##
 #
-
+# TODO: Add `list` to get an Array of prompt IDs
+# TODO: Add `path` to get a path to the prompt file
+#
 
 require 'prompt_manager'
 require 'prompt_manager/storage/file_system_adapter'
@@ -31,11 +33,14 @@ at_exit do
 end
 
 # Configure the Storage Adapter to use
+PromptManager::Storage::FileSystemAdapter.config do |config|
+  config.prompts_dir        = PROMPTS_DIR
+  # config.search_proc      = nil     # default
+  # config.prompt_extension = '.txt'  # default
+  # config.parms+_extension = '.json' # default
+end
 
-PromptManager::Prompt.storage_adapter = 
-  PromptManager::Storage::FileSystemAdapter.new(
-    prompts_dir: PROMPTS_DIR
-  )
+PromptManager::Prompt.storage_adapter = PromptManager::Storage::FileSystemAdapter.new
 
 # Get a prompt
 
@@ -87,3 +92,51 @@ EOS
 
 puts todo.to_s
 
+puts <<~EOS
+
+  When using the FileSystemAdapter for prompt storage you can have within 
+  the prompts_dir you can have many sub-directories. These sub-directories 
+  act like categories.  The prompt ID is composed for the sub-directory name, 
+  a "/" character and then the normal prompt ID.  For example "toy/8-ball"
+
+EOS
+
+magic = PromptManager::Prompt.get( id: 'toy/8-ball' )
+
+puts "The magic PROMPT is:"
+puts magic
+puts
+puts "Remember if you want to see the full text of the prompt file:"
+puts magic.text
+
+puts "="*64
+
+puts <<~EOS
+
+  The FileSystemAdapter also adds two new methods to the Prompt class:
+
+    list - provides an Array of pompt IDs
+    path(prompt_id) - Returns a Pathname object to the prompt file
+
+EOS
+
+puts "List of prompts available"
+puts "========================="
+
+puts PromptManager::Prompt.list
+
+puts <<~EOS
+
+  And the path to the "toy/8-ball" prompt file is:
+
+  #{magic.path}
+
+  Use "your_prompt.path" for when you want to do something with the
+  the prompt file like send it to a text editor.
+
+  Your can also use the class method if you supply a prompt_id
+  like this:
+
+EOS
+
+puts PromptManager::Prompt.path('toy/8-ball')

data/lib/prompt_manager/prompt.rb

@@ -25,9 +25,24 @@ class PromptManager::Prompt
       new(id: id)
     end
 
+
     def search(for_what)
       storage_adapter.search(for_what)
     end
+
+
+    def method_missing(method_name, *args, &block)
+      if storage_adapter.respond_to?(method_name)
+        storage_adapter.send(method_name, *args, &block)
+      else
+        super
+      end
+    end
+
+
+    def respond_to_missing?(method_name, include_private = false)
+      storage_adapter.respond_to?(method_name, include_private) || super
+    end
   end
   
   # SMELL:  Does the db (aka storage adapter) really need
@@ -142,6 +157,23 @@ class PromptManager::Prompt
   end
 
 
+  # Let the storage adapter instance take a crake at
+  # these unknown methods.  Don't care what the args
+  # are, just pass the prompt's ID.
+  def method_missing(method_name, *args, &block)
+    if db.respond_to?(method_name)
+      db.send(method_name, id, &block)
+    else
+      super
+    end
+  end
+
+
+  def respond_to_missing?(method_name, include_private = false)
+    db.respond_to?(method_name, include_private) || super
+  end
+
+
   # SMELL:  should this gem log errors or is that a function of
   #         main program?  I believe its the main program's job.
   def log_error(message)

data/lib/prompt_manager/storage/file_system_adapter.rb

@@ -1,26 +1,151 @@
 # prompt_manager/lib/prompt_manager/storage/file_system_adapter.rb
 
-
-require 'json'
+# Use the local (or remote) file system as a place to
+# store and access prompts.
+#
+# Adds two additional methods to the Promp class:
+#   list - returns Array of prompt IDs
+#   path = returns a Pathname object to the prompt's text file
+#   path(prompt_id) - same as path on the prompt instance
+#
+# Allows sub-directories of the prompts_dir to be
+# used like categories.  For example the prompt_id "toy/magic"
+# is found in the `magic.txt` file inside the `toy` sub-directory
+# of the prompts_dir.
+#
+# There can man be many layers of categories (sub-directories)
+#
+
+require 'json'      # basic serialization of parameters
+require 'pathname'
 
 class PromptManager::Storage::FileSystemAdapter
-  PARAMS_EXTENSION = '.json'.freeze
-  PROMPT_EXTENSION = '.txt'.freeze
+  SEARCH_PROC       = nil # placeholder
+  PARAMS_EXTENSION  = '.json'.freeze
+  PROMPT_EXTENSION  = '.txt'.freeze
+  PROMPT_ID_FORMAT  = /^[a-zA-Z0-9\-\/_]+$/
+
+  class << self
+    attr_accessor :prompts_dir, :search_proc, 
+                  :params_extension, :prompt_extension
+  
+    def config
+      if block_given?
+        yield self
+        validate_configuration     
+      else
+        raise ArgumentError, "No block given to config"
+      end
 
-  attr_reader :prompts_dir
+      self
+    end
 
-  def initialize(
-      prompts_dir:  '.prompts',
-      search_proc:  nil   # Example: ->(q) {`ag -l #{q}`}
-    )
+    # Expansion methods on the Prompt class specific to
+    # this storage adapter.
+
+    # Ignore the incoming prompt_id
+    def list(prompt_id = nil)
+      new.list
+    end
+
+
+    def path(prompt_id)
+      new.path(prompt_id)
+    end
+
+    #################################################
+    private
+
+    def validate_configuration
+      validate_prompts_dir
+      validate_search_proc
+      validate_prompt_extension
+      validate_params_extension
+    end
+
+
+    def validate_prompts_dir
+      # This is a work around for a Ruby scope issue where the 
+      # class getter/setter method is becoming confused with a 
+      # local variable when anything other than plain 'ol get and 
+      # set are used.'This error is in both Ruby v3.2.2 and
+      # v3.3.0-preview3.
+      #
+      prompts_dir_local = self.prompts_dir
+
+      unless prompts_dir_local.is_a?(Pathname)
+        prompts_dir_local = Pathname.new(prompts_dir_local) unless prompts_dir_local.nil?
+      end
 
-    # validate that prompts_dir exist and is in fact a directory.
-    unless Dir.exist?(prompts_dir)
-      raise "Directory #{prompts_dir} does not exist or is not a directory"
+      prompts_dir_local = prompts_dir_local.expand_path
+
+      raise(ArgumentError, "prompts_dir: #{prompts_dir_local}") unless prompts_dir_local.exist? && prompts_dir_local.directory?
+      
+      self.prompts_dir = prompts_dir_local
+    end
+
+
+    def validate_search_proc
+      search_proc_local = self.search_proc
+
+      if search_proc_local.nil?
+        search_proc_local = SEARCH_PROC
+      else
+        raise(ArgumentError, "search_proc invalid; does not respond to call") unless search_proc_local.respond_to?(:call)
+      end
+
+      self.search_proc = search_proc_local
+    end
+
+
+    def validate_prompt_extension
+      prompt_extension_local = self.prompt_extension
+
+      if prompt_extension_local.nil?
+        prompt_extension_local = PROMPT_EXTENSION
+      else
+        unless  prompt_extension_local.is_a?(String)    &&
+                prompt_extension_local.start_with?('.')
+          raise(ArgumentError, "Invalid prompt_extension: #{prompt_extension_local}")
+        end
+      end
+
+      self.prompt_extension = prompt_extension_local
     end
 
-    @prompts_dir  = prompts_dir
-    @search_proc  = search_proc
+
+    def validate_params_extension
+      params_extension_local = self.params_extension
+
+      if params_extension_local.nil?
+        params_extension_local = PARAMS_EXTENSION
+      else
+        unless  params_extension_local.is_a?(String)    &&
+                params_extension_local.start_with?('.')
+          raise(ArgumentError, "Invalid params_extension: #{params_extension_local}")
+        end
+      end
+
+      self.params_extension = params_extension_local
+    end
+  end
+
+
+  ##################################################
+  ###
+  ##  Instance
+  #
+
+  def prompts_dir       = self.class.prompts_dir
+  def search_proc       = self.class.search_proc
+  def prompt_extension  = self.class.prompt_extension
+  def params_extension  = self.class.params_extension
+
+
+  def initialize
+    # NOTE: validate because main program may have made
+    #       changes outside of the config block
+    self.class.send(:validate_configuration) # send gets around private designations of a method
   end
 
 
@@ -37,14 +162,20 @@ class PromptManager::Storage::FileSystemAdapter
 
   # Retrieve prompt text by its id
   def prompt_text(prompt_id)
-    read_file(file_path(prompt_id, PROMPT_EXTENSION))
+    read_file(file_path(prompt_id, prompt_extension))
   end
 
 
   # Retrieve parameter values by its id
   def parameter_values(prompt_id)
-    json_content = read_file(file_path(prompt_id, PARAMS_EXTENSION))
-    deserialize(json_content)
+    params_path = file_path(prompt_id, params_extension)
+    
+    if params_path.exist?
+      parms_content = read_file(params_path)
+      deserialize(parms_content)
+    else
+      {}
+    end
   end
 
 
@@ -56,8 +187,8 @@ class PromptManager::Storage::FileSystemAdapter
     )
     validate_id(id)
 
-    prompt_filepath = file_path(id, PROMPT_EXTENSION)
-    params_filepath = file_path(id, PARAMS_EXTENSION)
+    prompt_filepath = file_path(id, prompt_extension)
+    params_filepath = file_path(id, params_extension)
     
     write_with_error_handling(prompt_filepath, text)
     write_with_error_handling(params_filepath, serialize(parameters))
@@ -68,8 +199,8 @@ class PromptManager::Storage::FileSystemAdapter
   def delete(id:)
     validate_id(id)
 
-    prompt_filepath = file_path(id, PROMPT_EXTENSION)
-    params_filepath = file_path(id, PARAMS_EXTENSION)
+    prompt_filepath = file_path(id, prompt_extension)
+    params_filepath = file_path(id, params_extension)
     
     delete_with_error_handling(prompt_filepath)
     delete_with_error_handling(params_filepath)
@@ -85,26 +216,54 @@ class PromptManager::Storage::FileSystemAdapter
   end
 
 
+  # Return an Array of prompt IDs
+  def list(*)
+    prompt_ids = []
+    
+    Pathname.glob(prompts_dir.join("**/*#{prompt_extension}")).each do |file_path|
+      prompt_id = file_path.relative_path_from(prompts_dir).to_s.gsub(prompt_extension, '')
+      prompt_ids << prompt_id
+    end
+
+    prompt_ids
+  end
+
+
+  # Returns a Pathname object for a prompt ID text file
+  # However, it is possible that the file does not exist.
+  def path(id)
+    validate_id(id)
+    file_path(id, prompt_extension) 
+  end
+
   ##########################################
   private
 
+  # Validate that the ID contains good characters.
   def validate_id(id)
-    raise ArgumentError, 'Invalid ID format' unless id =~ /^[a-zA-Z0-9\-_]+$/
+    raise ArgumentError, "Invalid ID format id: #{id}" unless id =~ PROMPT_ID_FORMAT
+  end
+
+
+  # Verify that the ID actually exists
+  def verify_id(id)
+    file_path(id, prompt_extension).exist?
   end
 
 
   def write_with_error_handling(file_path, content)
     begin
-      File.write(file_path, content)
+      file_path.write content
     rescue IOError => e
       raise "Failed to write to file: #{e.message}"
     end
   end
 
 
+  # file_path (Pathname)
   def delete_with_error_handling(file_path)
     begin
-      FileUtils.rm_f(file_path)
+      file_path.delete
     rescue IOError => e
       raise "Failed to delete file: #{e.message}"
     end
@@ -112,7 +271,7 @@ class PromptManager::Storage::FileSystemAdapter
 
 
   def file_path(id, extension)
-    File.join(@prompts_dir, "#{id}#{extension}")
+    prompts_dir + "#{id}#{extension}"
   end
 
 
@@ -123,21 +282,22 @@ class PromptManager::Storage::FileSystemAdapter
 
 
   def search_prompts(search_term)
-    query_term = search_term.downcase
-
-    Dir.glob(File.join(@prompts_dir, "*#{PROMPT_EXTENSION}")).each_with_object([]) do |file_path, ids|
-      File.open(file_path) do |file|
-        file.each_line do |line|
-          if line.downcase.include?(query_term)
-            ids << File.basename(file_path, PROMPT_EXTENSION)
-            next
-          end
-        end
+    prompt_ids = []
+    
+    Pathname.glob(prompts_dir.join("**/*#{prompt_extension}")).each do |prompt_path|
+      if prompt_path.read.downcase.include?(search_term)
+        prompt_id = prompt_path.relative_path_from(prompts_dir).to_s.gsub(prompt_extension, '')    
+        prompt_ids << prompt_id
       end
-    end.uniq
+    end
+
+    prompt_ids
+
   end
 
 
+  # TODO: Should the serializer be generic?
+
   def serialize(data)
     data.to_json
   end

data/lib/prompt_manager/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module PromptManager
-  VERSION = "0.1.1"
+  VERSION = "0.2.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: prompt_manager
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.2.0
 platform: ruby
 authors:
 - Dewayne VanHoozer
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2023-11-19 00:00:00.000000000 Z
+date: 2023-11-22 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: amazing_print
@@ -25,7 +25,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: fakefs
+  name: debug_me
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -52,6 +52,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: tocer
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: "Manage the parameterized prompts (text) used in generative AI (aka chatGPT,
   \nOpenAI, et.al.) using storage adapters such as FileSystemAdapter, \nSqliteAdapter
   and ActiveRecordAdapter.\n"
@@ -69,6 +83,7 @@ files:
 - Rakefile
 - examples/prompts_dir/todo.json
 - examples/prompts_dir/todo.txt
+- examples/prompts_dir/toy/8-ball.txt
 - examples/simple.rb
 - lib/prompt_manager.rb
 - lib/prompt_manager/prompt.rb