greenlit-ai 0.3.0__tar.gz

greenlit_ai-0.3.0/PKG-INFO

@@ -0,0 +1,95 @@
+Metadata-Version: 2.4
+Name: greenlit-ai
+Version: 0.3.0
+Summary: greenlit: coding agents hear about each other's plans before they edit, and ask a person only when your rules say to
+License: Proprietary
+Project-URL: Homepage, https://neelsck.github.io/greenlit/
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: numpy>=1.24
+Requires-Dist: httpx>=0.24
+Provides-Extra: claude
+Requires-Dist: anthropic; extra == "claude"
+Provides-Extra: openai
+Requires-Dist: openai; extra == "openai"
+Provides-Extra: proxy
+Requires-Dist: httpx; extra == "proxy"
+Provides-Extra: prior
+Requires-Dist: lightgbm>=4; extra == "prior"
+Provides-Extra: all
+Requires-Dist: anthropic; extra == "all"
+Requires-Dist: openai; extra == "all"
+Requires-Dist: httpx; extra == "all"
+Requires-Dist: lightgbm>=4; extra == "all"
+Requires-Dist: duckdb>=1.0; extra == "all"
+Provides-Extra: report
+Requires-Dist: duckdb>=1.0; extra == "report"
+Requires-Dist: anthropic; extra == "report"
+Requires-Dist: httpx; extra == "report"
+Provides-Extra: team
+Requires-Dist: httpx; extra == "team"
+Provides-Extra: bugs
+Requires-Dist: pytest>=7; extra == "bugs"
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: anthropic; extra == "dev"
+Requires-Dist: openai; extra == "dev"
+Requires-Dist: httpx; extra == "dev"
+Requires-Dist: lightgbm>=4; extra == "dev"
+Requires-Dist: pyarrow; extra == "dev"
+Requires-Dist: duckdb>=1.0; extra == "dev"
+
+# greenlit (`pip install greenlit-ai`, command `grlt`)
+
+greenlit for teams running coding agents. When two agents are about to change the same thing in different ways,
+greenlit tells both of them, at plan time, before either edits. It asks a person only when your repo's rules say to,
+or when the agents still collide after being told.
+
+## Quickstart
+
+```bash
+pip install greenlit-ai
+cd your-repo
+grlt up            # starts a team on the hosted hub, wires Claude Code in this checkout, prints an invite link
+claude             # as usual
+```
+
+A teammate runs `grlt join '<invite link>'` in their own checkout. On your own, run a second agent in another worktree:
+
+```bash
+git worktree add ../your-repo-2 -b second && cd ../your-repo-2 && grlt join '<invite link>'
+```
+
+`grlt up` writes `goal.md` if the repo has none: what you're shipping, the decisions agents must hold to, and a
+**Don't change without asking** list. A commit that still changes something on that list after its agent was told is
+held until someone approves, denies, or tells the agents to settle it, in the web app's **Needs you**.
+
+## Inside Claude Code
+
+- Your agent gets greenlit's notes in its context, as facts about teammates' work.
+- You see each note too: `greenlit told your agent: ...` appears in the session.
+- The status line reads `greenlit · 2 agents · 1 note · nothing needs you`.
+
+## The web app
+
+The invite link is also the web app: **Needs you** (held commits, both sides, the rule from goal.md, approve / deny /
+settle), **Map** (only the files in someone's plan, each agent's planned path, forecast crossings before any edit) and
+**Threads** (one per clash: forecast, notes, holds, decision, outcome, a rating on each note).
+
+## What leaves your laptop
+
+The agent's plan text, file paths, changed lines and the functions around them, and the task text go to the hub.
+Never your prompts, your code base or your API keys: model traffic goes from your machine to Anthropic through a local
+proxy, and passes through untouched if the hub is down or slow.
+
+## Commands
+
+| | |
+|---|---|
+| `grlt up` | start a team for this repo, wire Claude Code here |
+| `grlt join <link>` | join a team from your own checkout |
+| `grlt demo` | watch the shop demo's catch in the web app (a replay; no agents, no cost) |
+| `grlt demo --seed DIR` | make the shop repo with two checkouts, to run it with real Claude Code |
+| `grlt goal` | send an edited goal.md to the team |
+| `grlt stats` | what greenlit did for this team |
+| `grlt down` | unwire this checkout: Claude Code's settings go back exactly as they were |

greenlit_ai-0.3.0/README.md

@@ -0,0 +1,141 @@
+# fsm-agent
+
+fsm-agent forecasts what an LLM agent will spend, in tokens and dollars, before each task and before each model call, learning from the agent's own history, and controls that spend: budgets that stop a run before it goes over, phase budgets for an agent you already have, and a phase controller for an agent you build on it. For a task the agent repeats, it also keeps the routes: the tree of tool calls earlier runs made, the cheapest one that succeeded, and a note to the agent at the start, at each stage and when it strays ([docs/paths.md](docs/paths.md)). It works with any agent, in any language, on Anthropic's or OpenAI's API or any OpenAI-compatible server, and it runs on your machine: your keys and your prompts never leave it.
+
+It also contains the paired experiment the controller was developed with: the same tasks with and without it, each arm billed to its own API key and checked against Anthropic's usage records.
+
+## Install
+
+Python 3.9 or later.
+
+```bash
+git clone https://github.com/neelsck/fsm-agent.git && cd fsm-agent
+python3 -m venv .venv && source .venv/bin/activate     # once per clone; activate again in a new terminal
+python3 -m pip install -e ".[dev]"                     # numpy, the anthropic and openai SDKs, httpx, pytest
+python3 -m pytest -q                                   # about a minute and a half; nothing reaches a real API
+```
+
+## Use it with your agent
+
+**New here? [docs/quickstart.md](docs/quickstart.md) gets your agent's runs forecast in about 15 minutes, step by step.**
+
+How the forecasts are made, and how accurate they have been: [docs/forecasting.md](docs/forecasting.md).
+
+```bash
+python3 -m fsm_agent import calls.jsonl --agent support-triage     # your history, from logs it already writes
+python3 -m fsm_agent proxy --agent support-triage                  # from now on: point the agent's base URL here
+python3 -m fsm_agent report fsm_runs/support-triage                # forecast against actual, the step graph, a dashboard
+python3 -m fsm_agent ui fsm_runs/support-triage                    # a local page following its runs as they happen
+python3 -m fsm_agent compare --agent support-triage --task "..." -- ./my-agent "{task}"   # one task, as it is and steered
+python3 -m fsm_agent mode --agent support-triage controller        # steer every run (compare: none)
+python3 -m fsm_agent paths fsm_runs/support-triage                 # what its repeated tasks did, and the cheapest successful route
+```
+
+| Way in | For |
+|---|---|
+| `import` | past runs, from logged request/response bodies (Anthropic, OpenAI), usage rows (CSV or JSON lines) or OpenTelemetry traces |
+| `proxy` | any agent in any language: set `ANTHROPIC_BASE_URL` or `OPENAI_BASE_URL` (or any OpenAI-compatible base URL) to it; runs by header or by conversation; budgets that stop a call before it is sent |
+| `track.wrap(client)` | a Python agent: wrap its `anthropic` or `openai` client |
+| `mcp` | Claude Code, Codex, Cursor or any MCP client: forecast a task, ask for the best known path through it, check a run's spend, report how it went |
+| a `policy.json` | phase budgets for an agent you already have, enforced by the proxy or the wrapper |
+| `fsm_agent.agent` | the phase controller for your own tools: phases, a check after each change, undo and retry, on Claude or any OpenAI-compatible model |
+
+```python
+from fsm_agent import agent
+
+flow = agent.Workflow(name="refunds", system="You handle refund requests.",
+                      phases=[agent.Phase("research", tools=[search_orders], max_calls=6),
+                              agent.Phase("refund", tools=[issue_refund], max_calls=2, commits=True)],
+                      check=refund_is_right)
+result = agent.run(flow, task="Customer 4411 says they paid twice for order 991.", model="claude-sonnet-5-5")
+```
+
+Every forecast is written to disk before the run or call it is about, so the report is an honest test of what you would have known at the time. [docs/onboarding.md](docs/onboarding.md) is the guide for all of this; [docs/traces.md](docs/traces.md) describes the files.
+
+## The experiment
+
+```bash
+python3 -m fsm_agent experiment --dry-run --n 20              # no key, no cost: a scripted stand-in plays the agent
+python3 -m fsm_agent experiment --n 3                         # a trial with a real model: 3 runs in each arm
+python3 -m fsm_agent experiment --n 50                        # the experiment
+python3 -m fsm_agent experiment --tasks bugs --n 12           # the bug-fix task set
+```
+
+**The arms.** Both arms are the same agent with the same tools on the same tasks, and both finish the same number of runs.
+
+- `baseline`: no phases. The harness runs the check after every change the agent makes (the test suite, or the booking check), as the controller does, so the comparison is about the phases and nothing else. `--plain-baseline` instead leaves checking to the agent.
+- `fsm`: the phase controller. The task is split into phases (for a bug: reproduce, locate, read, edit, verify), each with its own tools and a call budget; after an edit the harness runs the check; a failed attempt is cut from the context and the change undone, up to three attempts.
+
+**The task sets** (`--tasks`):
+
+- `bookings` (the default): meeting requests on a sandbox company calendar (14 people in five time zones, rooms, two weeks of meetings), graded against a hidden specification. About a cent a run.
+- `bugs`: 36 bugs seeded into [boltons](https://github.com/mahmoud/boltons) (BSD-licensed, pinned at commit `4e5faa3d`), 12 easy, 13 medium and 11 hard, each with a bug report written the way a user would. A run passes when the library's own tests for that module pass. Tens of cents a run. The first run fetches boltons with git into `~/.cache/fsm-agent/` (or `$FSM_AGENT_CACHE`). `python3 -m fsm_agent tasks --check` proves every bug is real: its tests fail with the bug and pass without it.
+
+**API keys.** A real run asks for two API keys, one per arm, each from its own workspace in the Claude Console, so each arm's spend shows up separately in your organization's usage. It also asks, optionally, for an Admin key, used only for read-only requests to fetch Anthropic's usage report for the audit. Keys are pasted at hidden prompts (or set as `ANTHROPIC_API_KEY_BASELINE`, `ANTHROPIC_API_KEY_FSM` and `ANTHROPIC_ADMIN_KEY`) and kept in memory: files and output identify a key only the way the Console does, by its first and last few characters. The plan and the spend limits are shown before anything starts, and nothing is spent until you answer `y`.
+
+**What it writes**, in a new folder under `experiments/`:
+
+| file | what |
+|---|---|
+| `baseline.jsonl`, `fsm.jsonl` | every step of every run ([docs/traces.md](docs/traces.md)) |
+| `forecasts.jsonl` | each run's cost forecast, written before the run started |
+| `forecast.md`, `dashboard.html`, `forecast.json` | the report: forecast against actual per run, per call and during a run; the step graph; the dashboard |
+| `audit.md`, `usage_ledger.jsonl` | what each key sent, against what Anthropic recorded for it, and where to see the same in the Console |
+| `patches/` | bugs: each run's change as a diff |
+| `tasks.json`, `experiment.json` | the tasks with their hidden answers, and the settings |
+
+**Forecasts** learn from the runs of the same arm that finished earlier, and start from earlier experiments with the same task set and model (`--learn-from FOLDER ...` to choose, `--fresh` for none).
+
+**Publishing the bug fixes** (`--publish OWNER/REPO`, bugs only). Every run is pushed to that repository as a branch, `runs/<experiment>/<arm>/<task>-r<n>`, on top of a branch holding the task's starting point, `tasks/<task>`, with a pull request whose description has the bug report, the agent's steps and the cost against its forecast. An index of every run is committed to `main` under `experiments/`. Use a repository of its own for this, not this one. It needs push access; the pull requests are opened with the GitHub CLI if you are logged in (`gh auth login`), else with `GITHUB_TOKEN`, and otherwise the run's link to open one is printed.
+
+## How the forecast works
+
+- **Before a call.** Most of a request repeats an earlier one, block for block; the model's reply and the tool's result are what was added. So the input is forecast from the earlier request's actual input, plus the reply carried over and what was added, and the cache split follows from the same match. The reply's length is learned per kind of step.
+- **Before a run.** Two methods. The *step graph*: an absorbing Markov chain over the kinds of step the agent takes (the tool each call asked for, or a plain reply), with each step's cost, which gives an expected cost and a range by simulation. And a *regression on the request's features*: whatever is known about a task before it starts (for bookings, the number of attendees and whether a room is needed; for bugs, the size of the module and of the bug report; for your agent, the `features` you name). The regression is used once there are enough runs to fit it, because it is what tells cheap requests from expensive ones in advance.
+- **During a run.** The projected total is what has been spent plus what is still to come, combining the step graph's view from the current step with the run's forecast.
+
+Costs are in input-token equivalents (ITE): one uncached input token is 1, and cache reads, cache writes and output tokens are weighted by the model's prices (on Claude Sonnet 5.5: 0.05, 1.25 and 5). An agent on several models counts every call in units of its first model, converted at list prices. Dollar figures come from the price table in `fsm_agent/prices.json` (Anthropic, OpenAI and Google, with sources), to which you can add your own models. Output tokens cost five times as much as input or more, so most of a call's forecast error is the length of the reply.
+
+## Commands
+
+```
+python3 -m fsm_agent proxy      --agent NAME [--port 8788] [--upstream NAME=URL] [--budget ITE --on-budget stop] [--policy FILE]
+python3 -m fsm_agent import     LOGS --agent NAME [--features a,b]   logs: messages, usage rows (CSV/JSONL) or OpenTelemetry
+python3 -m fsm_agent report     [FOLDER] [--full] [--replay]       an agent's or an experiment's folder; default: the latest
+python3 -m fsm_agent mcp        [--runs DIR] [--proxy URL]         the MCP server (stdio)
+python3 -m fsm_agent skill      [--to DIR]                         the Skill for the MCP tools
+python3 -m fsm_agent experiment [--tasks bookings|bugs] [--n 10] [--dry-run] [--publish OWNER/REPO]
+python3 -m fsm_agent audit      [FOLDER]                            an experiment's bill against Anthropic's usage report
+python3 -m fsm_agent show       TRACE.jsonl [--run ID]              one run, step by step
+python3 -m fsm_agent ui         [FOLDER] [--page FILE]             a local page following an agent's runs live
+python3 -m fsm_agent evaluate   [FOLDER] [--skip 10]                how good its forecasts are, replayed stage by stage
+python3 -m fsm_agent compare    --agent NAME --task T [--isolate] [--check CMD] [--split-keys VAR] -- CMD "{task}"
+python3 -m fsm_agent serve      --agent NAME [--port 8788] [--ui-port 8765]   the proxy, the runner and the console in one process (docs/runner.md)
+python3 -m fsm_agent mode       --agent NAME [compare|controller]   whether fsm-agent steers the agent's runs
+python3 -m fsm_agent paths      [FOLDER] [--family F] [--from a,b]   the tree of tool calls per repeated task, the cheapest successful path, advice from a point
+python3 -m fsm_agent tasks      [--tasks bugs] [--n 10] [--check]   the tasks an experiment would run
+```
+
+`python3 -m fsm_agent <command> --help` lists every option.
+
+## The code
+
+| | |
+|---|---|
+| `fsm_agent/proxy.py` | the local proxy |
+| `fsm_agent/track.py` | the wrapper, and the bookkeeping the proxy and `import` share with it |
+| `fsm_agent/providers.py` | the API shapes: Anthropic's Messages, OpenAI's Chat Completions and Responses |
+| `fsm_agent/cost.py`, `prices.json` | prices for any model, and costs in a common unit |
+| `fsm_agent/importers.py` | history from logs |
+| `fsm_agent/policy.py` | guardrails for an agent you already have |
+| `fsm_agent/paths.py` | per repeated task: the tree of tool calls, the cheapest successful path, the advice |
+| `fsm_agent/agent.py` | the phase controller as a library |
+| `fsm_agent/mcp.py`, `skill/SKILL.md` | the MCP server and its Skill |
+python3 -m fsm_agent ui fsm_runs/support-triage                    # a local page following its runs as they happen
+| `fsm_agent/forecast.py`, `graph.py`, `forecast_report.py` | the forecaster, the step graph, the report |
+| `fsm_agent/controller.py`, `spec.py` | the phase controller and the workflow definitions |
+| `fsm_agent/models/` | the controller's model adapters: Anthropic's Messages, OpenAI's Chat Completions |
+| `fsm_agent/experiment.py`, `billing.py` | the paired experiment and the billing audit |
+| `fsm_agent/bookings/`, `fsm_agent/bugs/` | the two task sets (each: workflow, environment, tasks, a scripted stand-in); `bugs/publish.py` publishes runs to GitHub |
+| `fsm_agent/dashboard.py`, `dashboard_template.html` | the dashboard |
+| `tests/` | `stub_api.py` and `stub_openai.py` are local stand-ins for Anthropic's and OpenAI's APIs that enforce their documented rules, so tests drive the real SDKs over HTTP without a key |

greenlit_ai-0.3.0/docs/grlt.md

@@ -0,0 +1,54 @@
+# greenlit (`pip install greenlit-ai`, command `grlt`)
+
+greenlit for teams running coding agents. When two agents are about to change the same thing in different ways,
+greenlit tells both of them, at plan time, before either edits. It asks a person only when your repo's rules say to,
+or when the agents still collide after being told.
+
+## Quickstart
+
+```bash
+pip install greenlit-ai
+cd your-repo
+grlt up            # starts a team on the hosted hub, wires Claude Code in this checkout, prints an invite link
+claude             # as usual
+```
+
+A teammate runs `grlt join '<invite link>'` in their own checkout. On your own, run a second agent in another worktree:
+
+```bash
+git worktree add ../your-repo-2 -b second && cd ../your-repo-2 && grlt join '<invite link>'
+```
+
+`grlt up` writes `goal.md` if the repo has none: what you're shipping, the decisions agents must hold to, and a
+**Don't change without asking** list. A commit that still changes something on that list after its agent was told is
+held until someone approves, denies, or tells the agents to settle it, in the web app's **Needs you**.
+
+## Inside Claude Code
+
+- Your agent gets greenlit's notes in its context, as facts about teammates' work.
+- You see each note too: `greenlit told your agent: ...` appears in the session.
+- The status line reads `greenlit · 2 agents · 1 note · nothing needs you`.
+
+## The web app
+
+The invite link is also the web app: **Needs you** (held commits, both sides, the rule from goal.md, approve / deny /
+settle), **Map** (only the files in someone's plan, each agent's planned path, forecast crossings before any edit) and
+**Threads** (one per clash: forecast, notes, holds, decision, outcome, a rating on each note).
+
+## What leaves your laptop
+
+The agent's plan text, file paths, changed lines and the functions around them, and the task text go to the hub.
+Never your prompts, your code base or your API keys: model traffic goes from your machine to Anthropic through a local
+proxy, and passes through untouched if the hub is down or slow.
+
+## Commands
+
+| | |
+|---|---|
+| `grlt up` | start a team for this repo, wire Claude Code here |
+| `grlt join <link>` | join a team from your own checkout |
+| `grlt demo` | watch the shop demo's catch in the web app (a replay; no agents, no cost) |
+| `grlt demo --seed DIR` | make the shop repo with two checkouts, to run it with real Claude Code |
+| `grlt goal` | send an edited goal.md to the team |
+| `grlt stats` | what greenlit did for this team |
+| `grlt down` | unwire this checkout: Claude Code's settings go back exactly as they were |

greenlit_ai-0.3.0/fsm_agent/__init__.py

@@ -0,0 +1,7 @@
+"""fsm-agent: forecast the token spend of LLM agents, before every run and every call.
+
+  fsm_agent.track       wrap the Claude client your agent already uses; every run and call is forecast and recorded
+  fsm_agent.importers   bring an agent's history in from its logs (Messages API logs, OpenTelemetry)
+  fsm_agent.experiment  the paired experiment: the same tasks without and with the phase controller
+"""
+__version__ = "0.2.0"

greenlit_ai-0.3.0/fsm_agent/__main__.py

@@ -0,0 +1,3 @@
+from .cli import main
+
+main()