aia 0.4.1 → 0.4.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0542f46fbbf4d3dd960229ae8f707f89b2cbab347d763d2b8db11aa8f4e2e552
-  data.tar.gz: 6dd9b949fbdc26019c64f2ece134114b515782395bce52a71b0f5d281042071b
+  metadata.gz: 3e8cad3b88bdc76e72dce9b7fa202923ab655f2da811039bf18df52c5fe714ae
+  data.tar.gz: 57ae0a1dd04c0fbd237ba8daa3210ae273a8db21e0efe5c154ffbe72c9afd0e5
 SHA512:
-  metadata.gz: 2eefbbd2215bc82361d1335da437714760ca7c9db5237b89280dd5cdbe220da2dd14f48890ca56641758186c0a54d5cc199b22b18535fa62ed147c52bda2a77a
-  data.tar.gz: '03990e11ef3fa9960fa8c3bd8aa34f667cf9fdd10b2e37646d2c54b199ea629fb58c1cafaeeb328bff1cbda2ce219300ab3831ab95894958317bc97c2835f18f'
+  metadata.gz: 0cc90b667bd99aae7688231286c58ee3cc7a3ad4a1b4202c531a71113911d24c084f6ccce5d154d57e0059a2a0b3d6d1706f3ee9a5fae5bf229b2604d1982d4d
+  data.tar.gz: 18dafdbfdeb257a2597f53070feebcb05f409f101b06ff17fdbe81cc813ebbf335833a0eace414638e437425ca47298b26655768890737cb065e3b15cfbf405e

data/.semver

@@ -1,6 +1,6 @@
 ---
 :major: 0
 :minor: 4
-:patch: 1
+:patch: 3
 :special: ''
 :metadata: ''

data/CHANGELOG.md

@@ -1,5 +1,12 @@
 ## [Unreleased]
-## [0.4.1] 2023-31
+## [0.4.3] 2023-12-31
+- added --env to process embedded system environment variables and shell commands within a prompt.
+- added --erb to process Embedded RuBy within a prompt because have embedded shell commands will only get you in a trouble.  Having ERB will really get you into trouble.  Remember the simple prompt is usually the best prompt.
+
+## [0.4.2] 2023-12-31
+- added the --role CLI option to pre-pend a "role" prompt to the front of a primary prompt.
+
+## [0.4.1] 2023-12-31
 - added a chat mode
 - prompt directives now supported
 - version bumped to match the `prompt_manager` gem

data/README.md

@@ -6,7 +6,9 @@ Uses the gem "prompt_manager" to manage the prompts sent to the `mods` command-l
 
 **Most Recent Change**
 
-v0.4.1 - Added a chat mode.  Prompt directives are now supported.
+v0.4.3 - Added the --env and --erb options.  Prompts can now be dynamic.
+v0.4.2 - Added a --role option to be prepended to a primary prompt.
+v0.4.1 - Added --chat and --speak options.  Prompt directives are now supported.
 
 <!-- Tocer[start]: Auto-generated, don't remove. -->
 
@@ -15,9 +17,14 @@ v0.4.1 - Added a chat mode.  Prompt directives are now supported.
   - [Installation](#installation)
   - [Usage](#usage)
   - [System Environment Variables (envar)](#system-environment-variables-envar)
+    - [System Environment Variables inside of a Prompt](#system-environment-variables-inside-of-a-prompt)
+  - [*E*mbedded *R*u*B*y (ERB)](#embedded-ruby-erb)
   - [Prompt Directives](#prompt-directives)
+  - [All About ROLES](#all-about-roles)
+    - [Other Ways to Insert Roles into Prompts](#other-ways-to-insert-roles-into-prompts)
   - [External CLI Tools Used](#external-cli-tools-used)
   - [Shell Completion](#shell-completion)
+  - [Ny Most Powerful Prompt](#ny-most-powerful-prompt)
   - [Development](#development)
   - [Contributing](#contributing)
   - [License](#license)
@@ -38,13 +45,14 @@ Install the command-line utilities by executing:
 
 You will also need to establish a directory in your file system where your prompt text files, last used parameters and usage log files are kept.
 
-Setup a system environment variable named "PROMPTS_DIR" that points to your prompts directory.  The default is in your HOME directory named ".prompts_dir"
+Setup a system environment variable named "AIA_PROMPTS_DIR" that points to your prompts directory.  The default is in your HOME directory named ".prompts_dir"
 
-You will also need to source the completion script.
+You may also want to install the completion script for your shell.  To get a copy of the completion script do:
 
-TODO: May want to add a `setup` function to the command-line options that will create the directory, and do something with the completion function.
+`aia --completion bash`
+
+`fish` and `zsh` are also available.
 
-TODO: don't forget to mention have access token (API keys) setup as envars for the various backend services like OpenAI... if they are still in business.
 
 ## Usage
 
@@ -89,6 +97,24 @@ OPTIONS
 
        --dump FORMAT
 
+       -e, --edit
+              Invokes an editor on the prompt file.  You can make changes to the prompt file,
+              save it and the newly saved prompt will be processed by the backend.
+
+       --env  This option tells aia to replace references to system environment variables in
+              the prompt with the value of the envar.  envars are like $HOME and ${HOME} in
+              this example their occurance will be replaced by the value of ENV[‘HOME’].
+              Also the dynamic shell command in the pattern $(shell command) will be executed
+              and its output replaces its pattern.  It does not matter if your shell uses
+              different patters than BASH since the replacement is being done within a Ruby
+              context.
+
+       --erb  If dynamic prompt content using $(...) wasn’t enough here is ERB.  Embedded
+              RUby.  <%= ruby code %> within a prompt will have its ruby code executed and
+              the results of that execution will be inserted into the prompt.  I’m sure we
+              will find a way to truly misuse this capability.  Remember, some say that the
+              simple prompt is the best prompt.
+
        --model NAME
               Name of the LLM model to use - default is gpt-4-1106-preview
 
@@ -133,6 +159,11 @@ OPTIONS
        -p, --prompts PATH_TO_DIRECTORY
               Directory containing the prompt files - default is ~/.prompts
 
+       -r, --role ROLE_ID
+              A role ID is the same as a prompt ID.  A “role” is a specialized prompt that
+              gets pre-pended to another prompt.  It’s purpose is to configure the LLM into a
+              certain orientation within which to resolve its primary prompt.
+
        -v, --verbose
               Be Verbose - default is false
 
@@ -217,7 +248,6 @@ AUTHOR
        Dewayne VanHoozer <dvanhoozer@gmail.com>
 
 AIA                                       2024-01-01                                   aia(1)
-
 ```
 
 ## System Environment Variables (envar)
@@ -244,6 +274,20 @@ See the `@options` hash in the `cli.rb` file for a complete list.  There are som
 
 In addition to these config items for `aia` the optional command line parameters for the backend prompt processing utilities (mods and sgpt) can also be set using envars with the "AIA_" prefix.  For example "export AIA_TOPP=1.0" will set the "--topp 1.0" command line option for the `mods` utility when its used as the backend processor.
 
+### System Environment Variables inside of a Prompt
+
+The cpmmand line option "--env" instructs `aia` to replace any system environment variable references in the prompt text with the value of the system envornment variable.  So patterns like $HOME and ${HOME} in the prompt will be replaced with the value of ENV['HOME'].
+
+As an added bonus, dynamic content can be inserted into the prompt using the pattern $(shell command) where the output of the shell command will replace the $(...) pattern.
+
+That's enough power to get anyone into deep trouble.  Wait there's more coming.  Do you want a full programming language in your prompt?  Feel like running the prompt through ERB before sending it to the backend?  I've been thinking about it.  Should be a simple thing to do. Yes it was.
+
+## *E*mbedded *R*u*B*y (ERB)
+
+The --erb options now submits the prompt text through the ERB parser for execution before environment variable, keyword and dynamic shell replace take place.  With this much dynamic prompt manipulation we should really be able to get into trouble!
+
+If you are not a Rubyist and do not know about ERB see this [webpage](https://ruby-doc.org/stdlib-3.0.0/libdoc/erb/rdoc/ERB.html) for some information about the power of embedded Ruby.
+
 ## Prompt Directives
 
 Downstream processing directives were added to the `prompt_manager` gem at version 0.4.1.  These directives are lines in the prompt text file that begin with "//"
@@ -256,9 +300,35 @@ That prompt will enter the chat loop regardles of the presents of a "--chat" CLI
 
 BTW did I mention that `aia` supports a chat mode where you can send an initial prompt to the backend and then followup the backend's reponse with additional keyboard entered questions, instructions, prompts etc.
 
-## External CLI Tools Used
+See the [AIA::Directives](lib/aia/directives.rb) class to see what directives are available on the fromend within `aia`.
+
+See the [AIA::Mods](lib/aia/tools/mods.rb) class to for directives that are available to the `mods` backend.
+
+See the [AIA::Sgpt](lib/aia/tools/sgpt.rb) class to for directives that are available to the `sgpt` backend.
+
+## All About ROLES
+
+`aia` provides the "-r --role" CLI option to identify a prompt ID within your prompts directory which defines the context within which the LLM is to provide its response.  The text of the role ID is pre-pended to the text of the primary prompt to form a complete prompt to be processed by the backend.
+
+For example consider:
+
+> aia -r ruby refactor my_class.rb
+
+Within the prompts directory the contents of the text file `ruby.txt` will be pre-pre-pended to the contents of the refactor.txt file to produce a complete prompt.  That complete prompt will have any parameters then directives processed before sending the prompt text to the backend.
+
+Note that "role" is just a way of saying add this prompt to the front of this other prompt.  The contents of the "role" prompt could be anything.  It does not necessarily have be an actual role.
+
+### Other Ways to Insert Roles into Prompts
 
-From the verbose help text ...
+Since `aia` supports parameterized prompts you could make a keyword like "[ROLE]" be part of your prompt.  For example consider this prompt:
+
+```text
+As a [ROLE] tell me what you think about [SUBJECT]
+```
+
+When this prompt is processed, `aia` will ask you for a value for the keyword "ROLE" and the keyword "SUBJECT" to complete the prompt.  Since `aia` maintains a history your previous answers, you could just choose something that you used in the past or answer with a completely new value.
+
+## External CLI Tools Used
 
 ```text
 External Tools Used
@@ -296,6 +366,22 @@ If you're not a fan of "born again" replace `bash` with one of the others.
 
 Copy the function to a place where it can be installed in your shell's instance.  This might be a `.profile` or `.bashrc` file, etc.
 
+## Ny Most Powerful Prompt
+
+This is just between you and me so don't go blabbing this around to everyone.  My most power prompt is in a file named `ad_hoc.txt`. It looks like this:
+
+> [WHAT NOW HUMAN]
+
+Yep.  Just a single parameter for which I can provide a value of anything that is on my mind at the time.  Its advantage is that I do not pollute my shell's command history with lots of text.
+
+Which do you think is better to have in your shell's history file?
+
+> mods "As a certified public accountant specializing in forensic audit and analysis of public company financial statements, what do you think of mine?  What is the best way to hide the millions dracma that I've skimmed?"  < financial_statement.txt
+
+> aia ad_hoc financial_statement.txt
+
+Both do the same thing; however, aia does not put the text of the prompt into the shell's history file.... of course the keyword/parameter value is saved in the prompt's JSON file and the prompt with the response are logged unless --no-log is specified; but, its not messing up the shell history!
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
@@ -304,6 +390,21 @@ After checking out the repo, run `bin/setup` to install dependencies. Then, run
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/MadBomber/aia.
 
+I've designed `aia` so that it should be easy to integrate other backend LLM processors.  If you've found one that you like, send me a pull request or a feature request.
+
+When you find problems with `aia` please note them as an issue.  This thing was written mostly by a human and you know how error prone humans are.  There should be plenty of errors to find.
+
+Also I'm interested in doing more with the prompt directives.  I'm thinking that there is a way to include dynamic content into the prompt computationally.  Maybe something like tjos wpi;d be easu tp dp:
+
+> //insert url https://www.whitehouse.gov/briefing-room/
+> //insert file path_to_file.txt
+
+or maybe incorporating the contents of system environment variables into prompts using $UPPERCASE or $(command) or ${envar_name} patterns.
+
+I've also been thinking that the REGEX used to identify a keyword within a prompt could be a configuration item.  I chose to use square brackets and uppercase in the default regex; maybe, you have a collection of prompt files that use some other regex.  Why should it be one way and not the other.
+
+Also I'm not happy with the way where I hve some command line options for external command hard coded.  I think they should be part of the configuration as well.  For example the way I'm using `rg` and `fzf` may not be the way that you want to use them.
+
 ## License
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/lib/aia/cli.rb

@@ -126,6 +126,8 @@ class AIA::Cli
       completion: [nil,       "--completion"],
       #
       edit?:      [false,     "-e --edit"],
+      env?:       [false,     "--env"],
+      erb?:       [false,     "--erb"],
       debug?:     [false,     "-d --debug"],
       verbose?:   [false,     "-v --verbose"],
       version?:   [false,     "--version"],
@@ -136,11 +138,7 @@ class AIA::Cli
       terse?:     [false,     "--terse"],
       speak?:     [false,     "--speak"],
       #
-      # TODO: May have to process the
-      #       "~" character and replace it with HOME
-      #
-      # TODO: Consider using standard suffix of _dif and _file
-      #       to signal Pathname objects fo validation
+      role:       ['',        "-r --role"],
       #
       config_file:[nil,                       "-c --config"],
       prompts_dir:["~/.prompts",              "-p --prompts"],

data/lib/aia/prompt.rb

@@ -1,6 +1,7 @@
 # lib/aia/prompt.rb
 
 require 'reline'
+require 'erb'
 
 class AIA::Prompt
   #
@@ -22,9 +23,25 @@ class AIA::Prompt
 
   # setting build: false supports unit testing.
   def initialize(build: true)
+    if AIA.config.role.empty?
+      @role = nil
+    else
+      AIA.config.arguments.prepend AIA.config.role
+      get_prompt
+      @role = @prompt.dup
+    end
+
     get_prompt
 
-    process_prompt if build
+    unless @role.nil?
+      @prompt.text.prepend @role.text
+    end
+
+    if build
+      @prompt.text = render_erb(@prompt.text)   if AIA.config.erb?
+      @prompt.text = replace_env(@prompt.text)  if AIA.config.env?
+      process_prompt 
+    end
   end
 
 
@@ -65,6 +82,25 @@ class AIA::Prompt
   end
 
 
+  # inserts environmant variables and dynamic content into a prompt
+  # replaces patterns like $HOME and ${HOME} with the value of ENV['HOME']
+  # replaces patterns like $(shell command) with the output of the shell command
+  #
+  def replace_env(a_string)
+    a_string.gsub(/\$(\w+|\{\w+\})/) do |match|
+      ENV[match.tr('$', '').tr('{}', '')]
+    end.gsub(/\$\((.*?)\)/) do |match|
+      `#{match[2..-2]}`.chomp
+    end
+  end
+
+
+  # You are just asking for trouble!
+  def render_erb(a_string)
+    ERB.new(a_string).result(binding)
+  end
+
+
   def replace_keywords
     puts
     puts "ID: #{@prompt.id}"
@@ -128,7 +164,7 @@ class AIA::Prompt
 
     puts "Parameter #{kw} ..."
 
-    if default.empty?
+    if default&.empty?
       user_prompt = "\n-=> "
     else
       user_prompt = "\n(#{default}) -=>"

data/man/aia.1

@@ -29,6 +29,15 @@ begin a chat session with the backend after the initial prompt response;  will s
 .TP
 \fB\-\-dump\fR \fIFORMAT\fP
 .TP
+\fB\-e\fR, \fB\-\-edit\fR
+Invokes an editor on the prompt file\.  You can make changes to the prompt file, save it and the newly saved prompt will be processed by the backend\.
+.TP
+\fB\-\-env\fR
+This option tells \fBaia\fR to replace references to system environment variables in the prompt with the value of the envar\.  envars are like \[Do]HOME and \[Do]\[lC]HOME\[rC] in this example their occurance will be replaced by the value of ENV\[lB]\[oq]HOME\[cq]\[rB]\.  Also the dynamic shell command in the pattern \[Do](shell command) will be executed and its output replaces its pattern\.  It does not matter if your shell uses different patters than BASH since the replacement is being done within a Ruby context\.
+.TP
+\fB\-\-erb\fR
+If dynamic prompt content using \[Do](\.\.\.) wasn\[cq]t enough here is ERB\.  Embedded RUby\.  <%\[eq] ruby code %> within a prompt will have its ruby code executed and the results of that execution will be inserted into the prompt\.  I\[cq]m sure we will find a way to truly misuse this capability\.  Remember, some say that the simple prompt is the best prompt\.
+.TP
 \fB\-\-model\fR \fINAME\fP
 Name of the LLM model to use \- default is gpt\-4\-1106\-preview
 .TP
@@ -71,6 +80,9 @@ Out FILENAME \- default is \.\[sl]temp\.md
 \fB\-p\fR, \fB\-\-prompts\fR \fIPATH\[ru]TO\[ru]DIRECTORY\fP
 Directory containing the prompt files \- default is \[ti]\[sl]\.prompts
 .TP
+\fB\-r\fR, \fB\-\-role\fR \fIROLE\[ru]ID\fP
+A role ID is the same as a prompt ID\.  A \[lq]role\[rq] is a specialized prompt that gets pre\-pended to another prompt\.  It\[cq]s purpose is to configure the LLM into a certain orientation within which to resolve its primary prompt\.
+.TP
 \fB\-v\fR, \fB\-\-verbose\fR
 Be Verbose \- default is false
 .SH CONFIGURATION HIERARCHY

data/man/aia.1.md

@@ -34,6 +34,15 @@ The aia command-line tool is an interface for interacting with an AI model backe
 `--dump` *FORMAT*
 : Dump a Config File in [yaml | toml] to STDOUT - default is nil
 
+`-e`, `--edit`
+: Invokes an editor on the prompt file.  You can make changes to the prompt file, save it and the newly saved prompt will be processed by the backend.
+
+`--env`
+: This option tells `aia` to replace references to system environment variables in the prompt with the value of the envar.  envars are like $HOME and ${HOME} in this example their occurance will be replaced by the value of ENV['HOME'].  Also the dynamic shell command in the pattern $(shell command) will be executed and its output replaces its pattern.  It does not matter if your shell uses different patters than BASH since the replacement is being done within a Ruby context.
+
+`--erb`
+: If dynamic prompt content using $(...) wasn't enough here is ERB.  Embedded RUby.  <%= ruby code %> within a prompt will have its ruby code executed and the results of that execution will be inserted into the prompt.  I'm sure we will find a way to truly misuse this capability.  Remember, some say that the simple prompt is the best prompt.
+
 `--model` *NAME*
 : Name of the LLM model to use - default is gpt-4-1106-preview
 
@@ -76,6 +85,9 @@ The aia command-line tool is an interface for interacting with an AI model backe
 `-p`, `--prompts` *PATH_TO_DIRECTORY*
 : Directory containing the prompt files - default is ~/.prompts
 
+`-r`, `--role` *ROLE_ID*
+: A role ID is the same as a prompt ID.  A "role" is a specialized prompt that gets pre-pended to another prompt.  It's purpose is to configure the LLM into a certain orientation within which to resolve its primary prompt.
+
 `-v`, `--verbose`
 : Be Verbose - default is false
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: aia
 version: !ruby/object:Gem::Version
-  version: 0.4.1
+  version: 0.4.3
 platform: ruby
 authors:
 - Dewayne VanHoozer
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2023-12-31 00:00:00.000000000 Z
+date: 2024-01-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: hashie
@@ -205,7 +205,6 @@ files:
 - lib/aia/tools/vim.rb
 - lib/aia/version.rb
 - lib/core_ext/string_wrap.rb
-- lib/modularization_plan.md
 - main.just
 - man/aia.1
 - man/aia.1.md

data/lib/modularization_plan.md

@@ -1,126 +0,0 @@
-## Suggested Refactoring into Modules
-
-### ConfigurationModule
-
-This module could encapsulate all the constants and environment-dependent settings.
-
-```ruby
-module Configuration
-  HOME            = Pathname.new(ENV['HOME'])
-  PROMPTS_DIR     = Pathname.new(ENV['PROMPTS_DIR'] || (HOME + ".prompts_dir"))
-  AI_CLI_PROGRAM  = "mods"
-  EDITOR          = ENV['EDITOR'] || 'edit'
-  MY_NAME         = Pathname.new(__FILE__).basename.to_s.split('.')[0]
-  MODS_MODEL      = ENV['MODS_MODEL'] || 'gpt-4-1106-preview'
-  OUTPUT          = Pathname.pwd + "temp.md"
-  PROMPT_LOG      = PROMPTS_DIR  + "_prompts.log"
-  USAGE = <<~EOUSAGE
-    AI Assistant (aia)
-    ==================
-    The AI cli program being used is: #{AI_CLI_PROGRAM}
-    You can pass additional CLI options to #{AI_CLI_PROGRAM} like this:
-    "#{MY_NAME} my options -- options for #{AI_CLI_PROGRAM}"
-  EOUSAGE
-end
-```
-
-### OptionParsingModule
-
-This module could manage the parsing of command-line arguments and configuring the options for the application.
-
-```ruby
-module OptionParsing
-  def build_reader_methods
-    # ... method definition ...
-  end
-
-  def process_arguments
-    # ... method definition ...
-  end
-
-  def check_for(an_option)
-    # ... method definition ...
-  end
-
-  def process_option(option_sym, switches)
-    # ... method definition ...
-  end
-end
-```
-
-### CommandLineInterfaceModule
-
-This module would manage interactions with the command-line interface including editing of prompts and selection processes.
-
-```ruby
-module CommandLineInterface
-  def keyword_value(kw, default)
-    # ... method definition ...
-  end
-
-  def handle_multiple_prompts(found_these, while_looking_for_this)
-    # ... method definition ...
-  end
-end
-```
-
-### LoggingModule
-
-Responsible for logging the results of the command.
-
-```ruby
-module Logging
-  def write_to_log(answer)
-    # ... method definition ...
-  end
-end
-```
-
-### AICommandModule
-
-Manages the building and execution of the AI CLI command.
-
-```ruby
-module AICommand
-  def setup_cli_program
-    # ... method definition ...
-  end
-
-  def build_command
-    # ... method definition ...
-  end
-
-  def execute_and_log_command(command)
-    # ... method definition ...
-  end
-end
-```
-
-### PromptProcessingModule
-
-Handles prompt retrieval, existing check, and keyword processing.
-
-```ruby
-module PromptProcessing
-  def existing_prompt?(prompt_id)
-    # ... method definition ...
-  end
-
-  def process_prompt
-    # ... method definition ...
-  end
-
-  def replace_keywords
-    # ... method definition ...
-  end
-
-  def search_for_a_matching_prompt(prompt_id)
-    # ... method definition ...
-  end
-end
-```
-
-Each module should only contain the methods relevant to that module's purpose. After defining these modules, they can be included in the `AIA::Main` class where appropriate. Note that the method `get_prompt_id` didn't fit neatly into one of the outlined modules; it may remain in the main class or be included in a module if additional context becomes available or if it can be logically grouped with similar methods.
-
-The `__END__` block and the Readline history management could be encapsulated into a separate module for terminal interactions if that block grows in complexity or moves out of the overall class definition.
-