agent_c 2.9

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: d647a877e7e6660d1c9ab7f5e2ce9e00d8f932c78d9ca6a47467265cc2c62e4e
+  data.tar.gz: 83e0796a0545c207c5afdebd457ac3134ea0b8bd5b9cd327a02de5f2049cc02b
+SHA512:
+  metadata.gz: a248627251111087a4814c86a5c980cdafa7936f0ae2b420b397485b367c7e0f693b2d4f6dcc6a46070c4381d2e2eaff775e31615bfdd5c0ea72e984655bec74
+  data.tar.gz: ce5adb048f393142b008a15943e2337597f2969a76cd8446a1890e5440ad9c73d39af74052085d916b58fbbbd551a23385db2b6285f68e359f676b5fb5fa67d4

data/.rubocop.yml

@@ -0,0 +1,10 @@
+AllCops:
+  Exclude: [ "bin/*" ]
+  TargetRubyVersion: 3.2
+  DisabledByDefault: true
+
+Style/FrozenStringLiteralComment:
+  Enabled: true
+
+Layout/EmptyLineAfterMagicComment:
+  Enabled: true

data/.ruby-version

@@ -0,0 +1 @@
+3.2.9

data/CLAUDE.md

@@ -0,0 +1,21 @@
+# Rules
+
+- Leave modules modules if they do not need state. If they apply behaviors to other schema/classes leave them modules.
+- Prefer to not store lambdas as variables unless necessary. If they are just going to be passed to other methods, leave them as blocks
+- If a class is not trivial (eg, more than one method and/or more than like 30 lines) then extract it to its own file.
+- This project is using Zeitwerk. You should not use require_relative, just match the module names to file path and it will load automatically.
+- When you commit, use the --no-gpg-sign flag. Start commit messages with "claude: "
+- DO NOT add example scripts. Either add it to the readme or make a test.
+- DO NOT add documentation outside of the README
+- DO NOT program defensively. If something should respond_to?() a method then just invoke the method. An error is better than a false positive
+- DO NOT write a one-off script for debugging, write a test-case and run it instead.
+- DO NOT edit the singleton class of an object. If you think you need to do this, ideas for avoiding: inject an object, create a module and include it, make a base class.
+
+# TESTING
+
+- We do not use stubbing in our test. If you need to stub something (or monkey-patch it) to test it, that thing should be injectable.
+- Run tests with `bin/rake test` You can pass TESTOPTS to run a specific file.
+
+# Style
+
+- For multiline Strings always use a HEREDOC

data/README.md

@@ -0,0 +1,360 @@
+# AgentC
+
+A small Ruby wrapper around [RubyLLM](https://github.com/alexrudall/ruby_llm) that helps you write a pipeline of AI prompts and run it many times, in bulk. Built for automating repetitive refactors across a large codebase.
+
+<small>Most of what's below is generated by an LLM. I take no responsibility for any of it, unless it's awesome... then it was pure prompting skills which I will take credit for.</small>
+
+## Overview
+
+AgentC provides batch processing and pipeline orchestration for AI-powered tasks:
+
+- **Batch Processing** - Execute pipelines across multiple records with automatic parallelization via worktrees
+- **Pipeline Orchestration** - Define multi-step workflows with AI-powered agent steps and custom logic
+- **Resumable Execution** - Automatically skip completed steps when pipelines are rerun
+- **Automatic query persistence** - All interactions saved to SQLite
+- **Cost tracking** - Detailed reports on token usage and costs
+- **Custom tools** - File operations, grep, Rails tests, and more
+- **Schema validation** - RubyLLM Schema support for structured responses
+
+## Installation
+
+This gem is not pushed to rubygems. Instead, you should add a git reference to your Gemfile (use a revision because I'm going to make changes with complete disregard for backwards compatibility).
+
+## Example template
+
+See an [example template](./template) you can run in the `template/` directory of this repo. Poke around there after perusing this section.
+
+You can copy this template to start building your own.
+
+## Quick Start
+
+A "Pipeline" is a series of prompts for Claude to perform. Data gathered from prior steps are fed into subsequent steps (you'll define an ActiveRecord class to capture the data). If any step fails, the pipeline aborts.
+
+A "Batch" is a collection of pipelines to be run. They can be run against a single directory in series, or concurrently across multiple git worktrees. If a pipeline fails, the failure will be recorded but the batch will continue.
+
+### The necessary structures
+
+In this example, we'll have Claude choose a random file, summarize its contents in a language of our choosing, then write it to disk and commit.
+
+```ruby
+# Define the records your agent will interact with.
+# Normally you'd only have one record.
+#
+# A versioned store saves a full db backup per-transaction
+# so that you can recover from any step of the process.
+# Just trying to save tokens...
+class MyStore < VersionedStore::Base
+  include AgentC::Store
+
+  record(:summary) do
+
+    # the migration schema is defined in line
+    schema do |t|
+      # we'll input this data
+      t.string(:language)
+
+      # claude will generate this data
+      t.string(:input_path)
+      t.text(:summary_text)
+      t.text(:summary_path)
+    end
+
+    # this is the body of your ActiveRecord class
+    # add methods here as needed
+  end
+end
+
+# A "pipeline" processes a single record
+class MyPipeline < AgentC::Pipeline
+  # The prompts for these steps will
+  # live in our prompts.yml file
+  agent_step(:analyze_code)
+  agent_step(:write_summary_to_file)
+
+  step(:finalize) do
+    repo.commit_all("claude: analyzed code")
+  end
+
+  # if this pipeline fails, we want to
+  # leave the repo in a clean state
+  # for the next pipeline.
+  on_failure do
+    repo.reset_hard_all
+  end
+end
+```
+```yaml
+# define your prompts in a prompts.yml file:
+
+en:
+
+  # the key names must match up to the `agent_step` invocation above
+  analyze_code:
+    # prompts here will be cached across pipelines.
+    # These prompts cannot interpolate any attributes.
+    # Suggested use is to put as much in the cached_prompts
+    # as possible and put variable data in the prompt.
+    cached_prompts:
+      - "Choose a random file. Read it and summarize it in the provided language."
+
+    # You can interpolate any attribute from your record class
+    prompt: "lanuage: %{language}"
+
+    # Tools available:
+    # - dir_glob
+    # - read_file
+    # - edit_file
+    # - grep
+    # - run_rails_test
+    # you can add more...
+    tools: [read_file, dir_glob]
+
+    # The response schema defines what Claude will return.
+    # The keys must be attributes from your record. What Claude
+    # returns will automatically be saved to your record.
+    response_schema:
+      summary_text:
+        type: string # this is the default
+        required: true # this is the default
+        description: "The summary text"
+      input_path:
+        type: string # this is the default
+        required: true # this is the default
+        description: "The path of the file you summarized"
+
+  write_summary_to_file:
+    cached_prompts:
+      - |
+      You will be given some text.
+      Choose a well-named file and write the text to it"
+
+    prompt: "Here is the text to write: %{summary_text}"
+    tools: [edit_file]
+    response_schema:
+      summary_path:
+        description: "the path of the file you wrote"
+```
+
+Now, make a Batch and invoke it. A batch requires a lot of configuration, related to data storage, where your repo is, and claude API credentials:
+
+```ruby
+
+batch = Batch.new(
+  record_type: :summary, # the class name you want to work on
+  pipeline: Pipeline, # the Pipeline class you made
+
+  # A batch has a "project" and a "run". These are ways
+  # to track Claude usage. Your Batch will have a
+  # "project". Each time you call batch.new you get
+  # a new "run".
+  project: "TemplateProject",
+
+  # We'll set some spending limits. Once these are
+  # reached, the Batch will abort.
+  max_spend_project: 100.0,
+  max_spend_run: 20.0,
+
+  store: {
+    class: Store, # the Store class you made
+    config: {
+      logger: Logger.new("/dev/null"), # a logger for the store
+      dir: "/where/you/want/your/store/saved"
+    }
+  },
+
+  # Where Claude will work
+  workspace: {
+    dir: "/where/claude/will/be/working",
+    env: {
+      # available to your tools
+      # only used by run_rails_test currently
+      SOME_ENV_VAR: "1"
+    }
+  }
+
+  # If you prefer, you can have the Batch manage
+  # some git worktrees for you. It will parallelize
+  # your tasks across your worktrees for MAXIMUM
+  # TOKEN BURN.
+  #
+  # Worktrees will be created for you if you are
+  # starting a new Batch. If you are continuing an
+  # existing Batch (after an error, for example),
+  # the worktrees will be left in their current
+  # state.
+  #
+  # You must pass *either* a workspace or a repo
+  repo: {
+    dir: "/path/to/your/repo",
+
+    # an existing git revision or branch name
+    initial_revision: "main",
+
+     # optional: limit Claude to a subdir from your repo
+    working_subdir: "./",
+
+    # Where to put your worktrees
+    worktrees_root_dir: "/tmp/example-worktrees",
+
+    # Each worktree gets a branch, they'll be suffixed
+    # with a counter
+    worktree_branch_prefix: "summary-examples",
+
+    # Currently, this defines how many worktrees to
+    # create. It's obnoxious I know, but hey, it works.
+    worktree_envs: [{}, {}],
+  }
+
+  # The claude configuration:
+  session: {
+    # all chats with claude are saved to a sqlite db.
+    # this is separate than your Store's db because
+    # why throw anything away. Can be useful for
+    # debugging why Claude did what it did
+    agent_db_path: "/path/to/your/claude/db.sqlite",
+    logger: Logger.new("/dev/null"), # probably use the same logger for everything...
+    i18n_path: "/path/to/your/prompts.yml",
+
+    # as you debug your pipeline, you'll probably run it
+    # many times. We tag all Claude chat records with a
+    # project so you can track costs.
+    project: "SomeProject",
+
+    # only available for Bedrock...
+    ruby_llm: {
+      bedrock_api_key: ENV.fetch("AWS_ACCESS_KEY_ID"),
+      bedrock_secret_key: ENV.fetch("AWS_SECRET_ACCESS_KEY"),
+      bedrock_session_token: ENV.fetch("AWS_SESSION_TOKEN"),
+      bedrock_region: ENV.fetch("AWS_REGION", "us-west-2"),
+      default_model: ENV.fetch("LLM_MODEL", "us.anthropic.claude-sonnet-4-5-20250929-v1:0")
+    }
+  },
+)
+
+# WHEW that's a lot of config,
+
+# Now we add some records for processing.
+# The batches "store" is just a bunch of
+# ActiveRecord classes, but you reference
+# them by the name you gave them in the
+# store.
+#
+# We'll add some summary records.
+# This seeded data represents the input
+# into your pipelines.
+#
+# Because your batch can be stopped and
+# restarted, we need our data creation
+# to be idemptotent.
+record_1 = (
+  batch
+    .store
+    .summary
+    .find_or_create_by!(language: "english")
+)
+record_2 = (
+  batch
+    .store
+    .summary
+    .find_or_create_by!(language: "spanish")
+)
+
+# Add the records to be processed.
+# add_task is idempotent
+batch.add_task(record_1)
+batch.add_task(record_2)
+
+batch.call
+
+# See the details of what happened
+puts batch.report
+# =>
+# Summary report:
+# Succeeded: 2
+# Pending: 0
+# Failed: 0
+# Run cost: $2.34
+# Project total cost: $10.40
+# ---
+# task: 1 - wrote summary to /tmp/example-worktrees/summary-examples-0/CHAT_TEST_SUMMARY.md
+# task: 2 - wrote summary to /tmp/example-worktrees/summary-examples-1/RESUMEN_BASE.md
+
+# Get a more detailed breakdown
+cost = batch.cost
+
+# Explore the records created
+tasks = batch.store.task.all
+summaries = batch.store.summary.all
+```
+
+You can tail your logs to see what's happening. The full text of you Claude chats are logged to DEBUG.
+
+If you just want to see your pipeline's progression:
+
+```shell
+# Only see INFO
+tail -f /path/to/log.log | grep INFO
+```
+
+#### Batch errors
+
+If your batch is interupted (by an exception or you kill it), you can continue it by simply running your batch again. The progress is persisted in the Batch's store.
+
+If you need to correct any data or go back in time, you can peruse the store's versions by doing:
+
+```ruby
+# see how many versions
+puts batch.store.versions.count
+
+# peruse your store:
+batch.store.versions[12].summary.count
+
+# restore a prior version
+batch.store.version[12].restore
+
+# re-run the batch
+batch.call
+```
+
+### Resetting a Batch
+
+You can delete the sqlite database for your store.
+
+Delete the database you configured at `store: { config: { dir: "/path/to/db" } }`
+
+### Debugging a Batch
+
+If you make multiple worktrees, they will be processed concurrently. This makes things hard to debug using `binding.irb`.
+
+I suggest making one worktree until it's running successfully.
+
+### Structuring your project
+
+I suggest following the structure of the [example template](./template).
+
+# Detailed Documentation
+
+Detailed guides for all features:
+
+- **[Batch](docs/batch.md)** - Batch configuration, methods, and pipeline integration
+- **[Pipeline Tips and Tricks](docs/pipeline-tips-and-tricks.md)** - Useful patterns and techniques for pipelines
+- **[Chat Methods](docs/chat-methods.md)** - Using session.prompt and session.chat for direct interactions
+- **[Tools](docs/tools.md)** - Built-in tools for file operations and code interaction
+- **[Testing](docs/testing.md)** - Using TestHelpers::DummyChat for testing without real LLM calls
+- **[Cost Reporting](docs/cost-reporting.md)** - Track token usage and costs
+- **[Session Configuration](docs/session-configuration.md)** - All configuration options
+- **[Store Versioning](docs/versioned-store.md)** - All configuration options
+
+## Requirements
+
+- Ruby >= 3.0.0
+- AWS credentials configured for Bedrock access
+- SQLite3
+
+## License
+
+WTFPL - Do What The Fuck You Want To Public License
+
+## Author
+
+Pete Kinnecom (git@k7u7.com)

data/Rakefile

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "."
+  t.test_files = FileList["test/**/*_test.rb"]
+  t.verbose = false
+  t.warning = false
+  # Enable parallel test execution based on number of processors
+  ENV["MT_CPU"] ||= Etc.nprocessors.to_s
+end
+
+task default: :test

data/TODO.md

@@ -0,0 +1,105 @@
+# TODOs
+
+Things I'd like to work on:
+
+- Make injecting a Chat record simpler.
+- Make injecting Git simpler (make injecting anything easier)
+- Add a request queue to AgentC::Chat so that we can rate-limit and retry on error
+- Use spring for run_rails_test, but add a timeout condition where it kills the process if no stdout appears for a while and tries again without spring.
+- tool calls should write the full results to file (except for readfile) and pass back a reference for future queries. For example, if RunRailsTest gives way too much output, we have to truncate but how to see the rest?
+
+## Immplement plan, implement, review looping
+
+Some scratch:
+
+```ruby
+agent_iterate_loop(
+  :implement_query_object,
+  max_tries: 17,
+  plan: [
+    :plan_step_1,
+    :plan_step_2,
+  ],
+  implement: [
+    :implement_step_1,
+    :implement_step_2,
+  ],
+  # optional: defaults to implement
+  iterate: [
+    :address_feedback_1
+  ]
+  review: [
+    :review_step_1,
+    :review_step_2
+  ],
+)
+```
+
+Thoughts:
+
+- This makes a demand on the `response_schema` of the prompts.
+- Does the `task` track the looping here or the `record`?
+  - Is the progress tracked in memory? This can be one step...
+- What if you have multiple `agent_iterate_loop` invocations? How do we store diffs/reviews for each of those?
+- The `agent_step` applies the result to the record. If we track it on the task, we could get the attributes to the right place. Can we extend the response schema? The `plan` step definitely needs to update the record, so it needs to specify response_schema. The `implement/iterate` steps *might* need to accept response_schema (eg, path of file created? Or not necessary?, maybe not necessary.)
+- Does a failed review trigger plan -> iterate -> review or just iterate -> review
+  - The implement/iterate arrays could include "plan" steps so that they know if they're starting from scratch or not.
+  - Do we run every review or stop at the first one?
+
+
+```ruby
+def agent_iterate_loop(
+  name,
+  max_tries: 3,
+  implement: [],
+  iterate: implement,
+  review: [],
+)
+  step(name) do
+    tries = 0
+
+    while(tries < max_tries)
+      if tries == 0
+        implement.each do |name|
+          process_prompt(name)
+        end
+      else
+        iterate.each do |name|
+          process_prompt(
+            name,
+            additional_i18n_attrs: {
+              feedback: feedback.join("\n---\n")
+            }
+          )
+        end
+      end
+
+      feedbacks = []
+
+      diff = git.diff
+      review.each do |name|
+        result = process_prompt(
+          name,
+          schema: -> {
+            t.boolean(:pass)
+            t.string(:feedback)
+          },
+          additional_i18n_attrs: {
+            diff:
+          }
+        )
+
+        if !result.fetch("pass")
+          feedbacks << result.fetch("feedback")
+        end
+      end
+
+      if record.respond_to?(:add_review)
+        record.add_review(diff:, feedbacks:)
+      end
+
+      break if feedbacks.empty?
+    end
+  end
+end
+```

data/agent_c.gemspec

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+require_relative "lib/agent_c/version"
+
+Gem::Specification.new do |spec|
+  spec.name = "agent_c"
+  spec.version = AgentC::VERSION
+  spec.authors = ["Pete Kinnecom"]
+  spec.email = ["git@k7u7.com"]
+
+  spec.summary = <<~TEXT.strip
+    Batch processing for pipelines of steps for AI. AgentC, get it?
+  TEXT
+  spec.homepage = "https://github.com/petekinnecom/agent_c"
+  spec.license = "WTFPL"
+  spec.required_ruby_version = ">= 3.0.0"
+  spec.metadata["allowed_push_host"] = "https://rubygems.org"
+  spec.metadata["homepage_uri"] = spec.homepage
+  spec.metadata["source_code_uri"] = spec.homepage
+  spec.metadata["changelog_uri"] = spec.homepage
+
+  spec.files = Dir.chdir(__dir__) do
+    `git ls-files -z`.split("\x0").reject do |f|
+      (File.expand_path(f) == __FILE__) ||
+        f.start_with?(*%w[bin/ test/ spec/ features/ .git .circleci appveyor Gemfile])
+    end
+  end
+  spec.bindir = "exe"
+  spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency("zeitwerk", "~> 2.7")
+  spec.add_dependency("activerecord", "~> 8.1")
+  spec.add_dependency("sqlite3", "~> 2.9")
+  spec.add_dependency("async", "~> 2.35")
+  spec.add_dependency("ruby_llm", "~> 1.9")
+  spec.add_dependency("json-schema", "~> 6.1")
+end