sql-harness 0.1.0__tar.gz

sql_harness-0.1.0/.claude-plugin/marketplace.json

@@ -0,0 +1,26 @@
+{
+  "name": "sql-harness",
+  "description": "Install sql-harness skills directly from this repo.",
+  "owner": {
+    "name": "much-bigpy",
+    "url": "https://github.com/zhaoliuxue"
+  },
+  "plugins": [
+    {
+      "name": "sql-harness",
+      "description": "Direct SQL access for agents: query, execute, schema inspection, transactions across PostgreSQL and MySQL. Plaintext credentials in one TOML file; auto-imported helpers. Ships skills only; the `sql-harness` CLI is a one-time `uv sync` install prerequisite.",
+      "category": "automation",
+      "source": ".",
+      "homepage": "https://github.com/zhaoliuxue/much_bigpy",
+      "keywords": [
+        "sql",
+        "postgres",
+        "mysql",
+        "redis",
+        "agent",
+        "automation",
+        "database"
+      ]
+    }
+  ]
+}

sql_harness-0.1.0/.claude-plugin/plugin.json

@@ -0,0 +1,13 @@
+{
+  "name": "sql-harness",
+  "version": "0.1.0",
+  "description": "Single-process SQL CLI for LLM agents. Drives PostgreSQL, MySQL (and Redis soon) via SQLAlchemy engines, with auto-imported Python helpers and plaintext credentials in one TOML file. Requires the one-time `uv sync` install (see the skill's references/install.md).",
+  "author": {
+    "name": "much-bigpy",
+    "url": "https://github.com/zhaoliuxue"
+  },
+  "homepage": "https://github.com/zhaoliuxue/much_bigpy",
+  "repository": "https://github.com/zhaoliuxue/much_bigpy",
+  "license": "MIT",
+  "keywords": ["sql", "postgres", "mysql", "redis", "agent", "cli", "automation", "database"]
+}

sql_harness-0.1.0/.env.example

@@ -0,0 +1,20 @@
+# Copy to .env — auto-loaded. Only needed for integration tests.
+#
+# sql-harness does NOT use an API key. Instead, prod secrets live in
+# connections.toml via ${env:VAR} indirection, e.g.:
+#
+#   [[connections]]
+#   name = "prod_pg"
+#   driver = "postgres"
+#   url = "${env:PROD_PG_URL}"
+#
+# Set these env vars (here or in your shell) for integration tests.
+
+# PostgreSQL integration test target.
+BH_PG_URL=postgresql://postgres:postgres@localhost:5432/postgres
+
+# MySQL integration test target.
+BH_MYSQL_URL=mysql+pymysql://root:root@localhost:3306/mysql?charset=utf8mb4
+
+# Example prod secret referenced via ${env:PROD_PG_URL} in connections.toml.
+# PROD_PG_URL=postgresql://user:pw@prod-host:5432/db

sql_harness-0.1.0/.github/ISSUE_TEMPLATE/bug-report.yml

@@ -0,0 +1,49 @@
+name: Bug report
+description: Report a reproducible bug in sql-harness.
+labels: [bug]
+body:
+  - type: checkboxes
+    id: preflight
+    attributes:
+      label: Before submitting
+      options:
+        - label: I searched existing issues for duplicates.
+          required: true
+        - label: I ran `sql-harness doctor` and read the output.
+          required: true
+        - label: I read the troubleshooting section of `install.md`.
+          required: true
+        - label: This is a reproducible bug in sql-harness — not a question, feature request, or connection-config problem.
+          required: true
+
+  - type: textarea
+    id: summary
+    attributes:
+      label: Summary
+      description: What's broken, in one or two sentences.
+    validations:
+      required: true
+
+  - type: textarea
+    id: repro
+    attributes:
+      label: Repro
+      description: Numbered steps. Include the exact command, the full output, and the driver being used.
+      placeholder: |
+        1. PostgreSQL 17, localhost:5432
+        2. sql-harness test my_pg
+        3. ConnectionError: password authentication failed
+    validations:
+      required: true
+
+  - type: textarea
+    id: environment
+    attributes:
+      label: Environment
+      placeholder: |
+        OS:
+        Database version (PG/MySQL/Redis):
+        sql-harness --version:
+        sql-harness doctor output:
+    validations:
+      required: true

sql_harness-0.1.0/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,8 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Question or how-to
+    url: https://github.com/zhaoliuxue/much_bigpy/discussions
+    about: Ask in Discussions, not Issues.
+  - name: Install or setup troubleshooting
+    url: https://github.com/zhaoliuxue/much_bigpy/blob/main/lab/sql_harness/install.md
+    about: Most connection and config errors are covered here.

sql_harness-0.1.0/.github/ISSUE_TEMPLATE/feature-request.yml

@@ -0,0 +1,37 @@
+name: Feature request
+description: Propose a new feature or change.
+labels: [feature-request]
+body:
+  - type: checkboxes
+    id: preflight
+    attributes:
+      label: Before submitting
+      options:
+        - label: I searched existing issues and discussions.
+          required: true
+        - label: This is a feature request, not a bug.
+          required: true
+
+  - type: textarea
+    id: problem
+    attributes:
+      label: Problem
+      description: What user pain or limitation motivates this? (Missing driver? Connection pain? Missing skill?)
+    validations:
+      required: true
+
+  - type: textarea
+    id: proposal
+    attributes:
+      label: Proposal
+      description: What you'd like to happen.
+    validations:
+      required: true
+
+  - type: textarea
+    id: alternatives
+    attributes:
+      label: Alternatives considered
+      description: What else you tried, or why other approaches fall short.
+    validations:
+      required: true

sql_harness-0.1.0/.gitignore

@@ -0,0 +1,34 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+
+# Build / dist
+build/
+dist/
+*.egg-info/
+.eggs/
+
+# Virtual envs
+.venv/
+venv/
+.tool-venv/
+
+# Test / coverage
+.pytest_cache/
+.coverage
+htmlcov/
+
+# IDE
+.vscode/
+.idea/
+*.swp
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Secrets (the connections.toml at runtime can have real DSNs; never commit)
+.env
+*.env.local

sql_harness-0.1.0/AGENTS.md

@@ -0,0 +1,37 @@
+browser-harness is a thin layer that connects agents to browsers via a CDP daemon.
+sql-harness is the same idea for SQL — connects agents to relational databases via SQLAlchemy engines. Single-process; no daemon.
+
+# Code priorities
+- Clarity
+- Precision
+- Low verbosity
+- Versatility
+
+# Overview
+Core code lives in `lab/sql_harness/src/sql_harness/` (src-layout, mirrors browser-harness's `src/browser_harness/`):
+- `config.py` — TOML schema + ConnectionConfig / ConnectionsConfig dataclasses
+- `paths.py` — XDG-style state directory resolution (mirrors browser-harness paths.py)
+- `drivers/` — backend abstractions (postgres, mysql, redis stub, sqlite for tests)
+- `manager.py` — SqlHarness class: holds engines, workspaces, skills registry
+- `helpers.py` — heredoc helpers (auto-imported)
+- `agent_loader.py` — loads `$BH_SQL_AGENT_WORKSPACE/agent_helpers.py` (mirrors browser-harness helpers.py:493-508)
+- `run.py` — heredoc entry; delegates to CLI otherwise
+- `cli.py` — argparse subparser CLI (~10 subcommands)
+
+`SKILL.md` tells agents how to use the harness.
+`install.md` tells agents how to install and configure it.
+
+An agent operating the harness only edits inside `agent-workspace/`:
+- `agent_helpers.py` — task-specific query helpers the agent adds
+- `skills/<name>.md` — per-task or per-table knowledge the agent writes/reads
+
+# Contributing
+Consider what is really needed. Prefer the smallest diff that fixes the bug.
+
+# Driver extension
+To add a new backend (Redis is the obvious next target):
+1. Add the package to `pyproject.toml` via `uv add <pkg>`.
+2. Create `drivers/<name>.py` implementing the `Driver` protocol (see `drivers/__init__.py`).
+3. Register it in `drivers/__init__.py:get_driver()`.
+4. Add a unit test in `tests/unit/test_<name>.py`.
+5. Optionally add an integration test in `tests/integration/test_<name>.py` gated on `BH_<NAME>_URL`.

sql_harness-0.1.0/PKG-INFO

@@ -0,0 +1,137 @@
+Metadata-Version: 2.4
+Name: sql-harness
+Version: 0.1.0
+Summary: Single-process SQL + SSH CLI for LLM agents. PostgreSQL/MySQL/SSH via SQLAlchemy + paramiko. Plaintext credentials in one TOML file. Helpers auto-injected into the heredoc namespace.
+Project-URL: Source, https://github.com/zhaoliuxue/much_bigpy/tree/master/lab/sql_harness
+Project-URL: Issues, https://github.com/zhaoliuxue/much_bigpy/issues
+Author: much-bigpy
+License: MIT
+Keywords: agent,cli,database,heredoc,mysql,postgres,sql,ssh
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Database
+Classifier: Topic :: System :: Shells
+Requires-Python: >=3.12
+Requires-Dist: paramiko<4,>=3.4
+Requires-Dist: psycopg[binary]<4,>=3.2
+Requires-Dist: pymysql<2,>=1.1
+Requires-Dist: sqlalchemy<3,>=2.0
+Description-Content-Type: text/markdown
+
+# sql-harness
+
+A thin, single-process SQL CLI for LLM agents. Mirrors [browser-harness](https://github.com/browser-use/browser-harness)'s structure but targets relational databases (Postgres, MySQL, Redis-soon).
+
+## Quickstart
+
+```bash
+# 1. Install deps (one-time)
+uv sync
+
+# 2. Scaffold a starter connections.toml
+uv run sql-harness init
+
+# 3. Add a connection
+uv run sql-harness add local_pg --driver postgres --url 'postgresql://postgres:postgres@localhost:5432/postgres'
+
+# 4. Test it
+uv run sql-harness test local_pg
+
+# 5. Use it
+uv run sql-harness <<'PY'
+use_workspace("local_pg")
+print(query("SELECT version()"))
+print(list_tables())
+print(describe("pg_class"))
+PY
+```
+
+## Architecture (~1k lines across 8 core files)
+
+- `install.md` — first-time install + first connection
+- `SKILL.md` — day-to-day usage
+- `lab/sql_harness/src/sql_harness/` — protected core package (src-layout, mirrors browser-harness)
+- `${XDG_CONFIG_HOME:-~/.config}/sql-harness/connections.toml` — plaintext credentials in ONE place
+- `${XDG_CONFIG_HOME:-~/.config}/sql-harness/agent-workspace/agent_helpers.py` — task-specific helpers
+- `${XDG_CONFIG_HOME:-~/.config}/sql-harness/agent-workspace/skills/` — per-task/per-table skills
+
+## Why mirror browser-harness?
+
+- Same operational ergonomics for agents: heredoc CLI with auto-imported helpers.
+- Same agent-workspace pattern (helpers + skills stay editable per-project).
+- Same path/state layout (XDG-style state directory).
+- Same doc layering: `SKILL.md` body + `interaction-skills/*.md` for stuck points.
+
+## What this is NOT
+
+- Not a database migration framework.
+- Not an ORM.
+- Not a connection-string parser (`pgcli`/`mycli` already exist for interactive REPL use; this is for *agents*).
+- Not a long-running daemon. SQL connections don't need one.
+
+## Connection file format
+
+Default: `${XDG_CONFIG_HOME:-~/.config}/sql-harness/connections.toml`.
+
+```toml
+default_workspace = "local_pg"
+
+[pool_defaults]
+size = 5
+recycle = 3600
+pre_ping = true
+echo = false
+
+[[connections]]
+name = "local_pg"
+driver = "postgres"
+url = "postgresql+psycopg://postgres:postgres@localhost:5432/postgres"
+description = "Local PostgreSQL for dev"
+
+[[connections]]
+name = "prod_pg"
+driver = "postgres"
+url = "${env:PROD_PG_URL}"            # env-var indirection
+pool_size = 10
+```
+
+Use `${env:VAR}` placeholders inside `url` for prod secrets — they're expanded at load time from the process environment.
+
+## Configuration env vars
+
+| Var | Purpose |
+|---|---|
+| `BH_SQL_HOME` | Override state root (`~/.config/sql-harness` by default) |
+| `BH_SQL_CONFIG_FILE` | Override connections.toml path |
+| `BH_SQL_AGENT_WORKSPACE` | Override agent-workspace directory |
+| `BH_SQL_RUNTIME_DIR` | Override runtime state dir |
+| `BH_SQL_TMP_DIR` | Override temp dir |
+| `BH_PG_URL` | Used by integration tests (skipped if unset) |
+| `BH_MYSQL_URL` | Used by integration tests (skipped if unset) |
+
+## Drivers
+
+| Driver | Backend | Package | Status |
+|---|---|---|---|
+| `postgres` | PostgreSQL | `psycopg[binary]>=3.2` | ✅ |
+| `mysql` | MySQL | `pymysql>=1.1` | ✅ |
+| `redis` | Redis | `redis>=5.0` (not yet added) | 🚧 stub |
+| `sqlite` | SQLite | stdlib (`sqlite3`) | ✅ (test-only) |
+| `ssh` / `ssh+password` / `ssh+key` | remote shell + SFTP (paramiko) | `paramiko>=3.4` | ✅ |
+
+To enable Redis: `uv add "redis>=5.0,<6"` then implement `drivers/redis.py`.
+To add another SSH host: append a `[[connections]]` block with `driver = "ssh"` and an `ssh://...` URL.
+
+## Contributing
+
+PRs and improvements welcome. See `AGENTS.md` for code priorities.
+
+- **Skills are written by the harness, not by you.** When you figure out a non-obvious SQL flow (a weird schema, a slow query, a JSON column trick), file a skill in `agent-workspace/skills/<name>.md`. Future sessions will read it before re-discovering it.
+- Bug fixes, new drivers, helper additions all welcome.
+
+## License
+
+MIT.

sql_harness-0.1.0/README.md

@@ -0,0 +1,114 @@
+# sql-harness
+
+A thin, single-process SQL CLI for LLM agents. Mirrors [browser-harness](https://github.com/browser-use/browser-harness)'s structure but targets relational databases (Postgres, MySQL, Redis-soon).
+
+## Quickstart
+
+```bash
+# 1. Install deps (one-time)
+uv sync
+
+# 2. Scaffold a starter connections.toml
+uv run sql-harness init
+
+# 3. Add a connection
+uv run sql-harness add local_pg --driver postgres --url 'postgresql://postgres:postgres@localhost:5432/postgres'
+
+# 4. Test it
+uv run sql-harness test local_pg
+
+# 5. Use it
+uv run sql-harness <<'PY'
+use_workspace("local_pg")
+print(query("SELECT version()"))
+print(list_tables())
+print(describe("pg_class"))
+PY
+```
+
+## Architecture (~1k lines across 8 core files)
+
+- `install.md` — first-time install + first connection
+- `SKILL.md` — day-to-day usage
+- `lab/sql_harness/src/sql_harness/` — protected core package (src-layout, mirrors browser-harness)
+- `${XDG_CONFIG_HOME:-~/.config}/sql-harness/connections.toml` — plaintext credentials in ONE place
+- `${XDG_CONFIG_HOME:-~/.config}/sql-harness/agent-workspace/agent_helpers.py` — task-specific helpers
+- `${XDG_CONFIG_HOME:-~/.config}/sql-harness/agent-workspace/skills/` — per-task/per-table skills
+
+## Why mirror browser-harness?
+
+- Same operational ergonomics for agents: heredoc CLI with auto-imported helpers.
+- Same agent-workspace pattern (helpers + skills stay editable per-project).
+- Same path/state layout (XDG-style state directory).
+- Same doc layering: `SKILL.md` body + `interaction-skills/*.md` for stuck points.
+
+## What this is NOT
+
+- Not a database migration framework.
+- Not an ORM.
+- Not a connection-string parser (`pgcli`/`mycli` already exist for interactive REPL use; this is for *agents*).
+- Not a long-running daemon. SQL connections don't need one.
+
+## Connection file format
+
+Default: `${XDG_CONFIG_HOME:-~/.config}/sql-harness/connections.toml`.
+
+```toml
+default_workspace = "local_pg"
+
+[pool_defaults]
+size = 5
+recycle = 3600
+pre_ping = true
+echo = false
+
+[[connections]]
+name = "local_pg"
+driver = "postgres"
+url = "postgresql+psycopg://postgres:postgres@localhost:5432/postgres"
+description = "Local PostgreSQL for dev"
+
+[[connections]]
+name = "prod_pg"
+driver = "postgres"
+url = "${env:PROD_PG_URL}"            # env-var indirection
+pool_size = 10
+```
+
+Use `${env:VAR}` placeholders inside `url` for prod secrets — they're expanded at load time from the process environment.
+
+## Configuration env vars
+
+| Var | Purpose |
+|---|---|
+| `BH_SQL_HOME` | Override state root (`~/.config/sql-harness` by default) |
+| `BH_SQL_CONFIG_FILE` | Override connections.toml path |
+| `BH_SQL_AGENT_WORKSPACE` | Override agent-workspace directory |
+| `BH_SQL_RUNTIME_DIR` | Override runtime state dir |
+| `BH_SQL_TMP_DIR` | Override temp dir |
+| `BH_PG_URL` | Used by integration tests (skipped if unset) |
+| `BH_MYSQL_URL` | Used by integration tests (skipped if unset) |
+
+## Drivers
+
+| Driver | Backend | Package | Status |
+|---|---|---|---|
+| `postgres` | PostgreSQL | `psycopg[binary]>=3.2` | ✅ |
+| `mysql` | MySQL | `pymysql>=1.1` | ✅ |
+| `redis` | Redis | `redis>=5.0` (not yet added) | 🚧 stub |
+| `sqlite` | SQLite | stdlib (`sqlite3`) | ✅ (test-only) |
+| `ssh` / `ssh+password` / `ssh+key` | remote shell + SFTP (paramiko) | `paramiko>=3.4` | ✅ |
+
+To enable Redis: `uv add "redis>=5.0,<6"` then implement `drivers/redis.py`.
+To add another SSH host: append a `[[connections]]` block with `driver = "ssh"` and an `ssh://...` URL.
+
+## Contributing
+
+PRs and improvements welcome. See `AGENTS.md` for code priorities.
+
+- **Skills are written by the harness, not by you.** When you figure out a non-obvious SQL flow (a weird schema, a slow query, a JSON column trick), file a skill in `agent-workspace/skills/<name>.md`. Future sessions will read it before re-discovering it.
+- Bug fixes, new drivers, helper additions all welcome.
+
+## License
+
+MIT.

sql_harness-0.1.0/SKILL.md

@@ -0,0 +1,61 @@
+---
+name: sql-harness
+description: "Use sql-harness for SQL work — querying, schema inspection, migrations, and cross-database workflows."
+---
+
+# sql-harness
+
+Direct SQL access via a thin Python heredoc CLI. Connections live in plaintext in a single TOML file (default: `~/.config/sql-harness/connections.toml`).
+
+For setup or install problems, read `install.md`. For stuck-point mechanics, see `interaction-skills/` (cross-DB) and `interaction-skills/postgres/` (PG optimization depth: plan reading, B-Tree scans, specialized indexes, table optimization, slow-query detection).
+
+## Usage
+
+```bash
+sql-harness --help
+sql-harness list
+sql-harness add <name> --driver postgres --url 'postgresql://...'
+sql-harness test <name>
+
+sql-harness <<'PY'
+use_workspace("local_pg")
+print(query("SELECT version()"))
+print(list_tables())
+PY
+```
+
+- Helpers are pre-imported; you can call them by name in heredoc mode.
+- **First call requires `use_workspace(name)`** — there is no implicit default.
+- For task-specific helpers, drop them into `$BH_SQL_AGENT_WORKSPACE/agent_helpers.py` and they'll merge into the heredoc namespace.
+
+## Connection-pool skill + workspace skill
+
+Read these when working across multiple connections or tuning pool behavior:
+
+- `agent-workspace/skills/pool.md` — pool sizing, `pre_ping`, idle reuse
+- `agent-workspace/skills/workspace.md` — workspace isolation, multi-DB workflows
+- `interaction-skills/save-run-cycle.md` — the correct workflow: write a block, `save` it, `run` it; repeat. Don't batch-save at the end.
+- `interaction-skills/zone-skill-auto-surface.md` — set `$BH_SQL_ZONE_SKILLS=1` and `use_workspace(name)` returns a dict auto-attaching `zone_skills` + `zone_scripts` (mirrors browser-harness's `BH_DOMAIN_SKILLS=1` + `goto_url` returning domain skills).
+
+## Local Chrome? No, this is SQL.
+
+There is no Chrome and no remote "cloud" backend. SQL is single-process. All engines live in `SqlHarness`'s in-memory registry.
+
+## Design Constraints
+
+- One connection = one workspace; never share engines across workspaces.
+- Connection pool defaults: `size=5, recycle=3600, pre_ping=True` (per-connection overrides win).
+- `with_transaction()` yields a `Connection`; use `conn.execute(text(...))` for raw control.
+- `query()` returns `list[dict]`; use `execute()` for INSERT/UPDATE/DELETE.
+- `list_tables()` and `describe()` are read-only schema introspection helpers.
+
+## Gotchas
+
+- Driver label in TOML must match the URL scheme (`postgres` ⇄ `postgresql://`, `mysql` ⇄ `mysql+pymysql://`).
+- Passwords in TOML are plaintext; use `${env:VAR}` indirection for prod secrets.
+- `query()` is for SELECTs only. For INSERTs, use `execute()`.
+- For tables > 10k rows, use `engine.connect().execution_options(stream_results=True)` and iterate manually — `query()` loads everything into memory.
+
+## Domain / table skills
+
+PG optimization practice: 71 real [pgexercises](https://pgexercises.com/) problems in `practice/pgexercises/` — one `.sql` file per problem (header = question, body = answer). Load schema/seed and replay an answer with `run_sql_file`; tune it with `explain_analyze`. To add a per-table or per-schema skill, drop a markdown file into `agent-workspace/skills/<name>.md`. Read with `apply_skill(name)`.

sql_harness-0.1.0/agent-workspace/README.md

@@ -0,0 +1,42 @@
+# Generated-code examples (mirrors browser-harness/agent-workspace/domain-skills/)
+
+These are **committed examples** of code the harness generated during real
+sessions. The live runtime copies live in `$BH_SQL_AGENT_WORKSPACE` (default
+`~/.config/sql-harness/agent-workspace/`); these repo copies are the
+"historical code" you can read in-tree, the same way browser-harness ships
+`agent-workspace/domain-skills/<site>/` examples.
+
+## Contents
+
+- `scripts/sh_demo_crud.py` — full e-commerce CRUD walkthrough against a live
+  PostgreSQL. Generated via `sql-harness save sh_demo_crud`. Idempotent (drops
+  + recreates the `sh_demo` schema each run). Re-run with
+  `sql-harness run sh_demo_crud`.
+- `agent_helpers.py` — reusable helpers (`seed_users`, `place_order`,
+  `ship_pending_orders`, `list_orders`, `user_by_email`, `product_by_sku`)
+  auto-imported into the heredoc/run namespace.
+- `skills/sh_demo-schema.md` — schema knowledge the agent captured for reuse.
+
+## How this code was produced
+
+```bash
+# 1. Save the DSN once
+sql-harness add --name test_pg --driver postgres --url 'postgres://...'
+
+# 2. Iterate in heredoc mode until the flow works
+sql-harness <<'PY'
+use_workspace("test_pg")
+...
+PY
+
+# 3. Persist the working heredoc as reusable code
+sql-harness save sh_demo_crud <<'PY'
+... the working code ...
+PY
+
+# 4. Re-run anytime (no re-typing)
+sql-harness run sh_demo_crud
+```
+
+That last step — **executed code is saved and re-runnable** — is the essence
+this harness inherits from browser-harness.