sql-harness 0.3.1__tar.gz → 0.3.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (94) hide show
  1. sql_harness-0.3.2/AGENTS.md +19 -0
  2. {sql_harness-0.3.1 → sql_harness-0.3.2}/PKG-INFO +5 -5
  3. sql_harness-0.3.2/SKILL.md +195 -0
  4. {sql_harness-0.3.1 → sql_harness-0.3.2/docs}/AGENTS.md +4 -3
  5. {sql_harness-0.3.1 → sql_harness-0.3.2/docs}/README.md +3 -3
  6. {sql_harness-0.3.1 → sql_harness-0.3.2/docs}/install.md +5 -1
  7. sql_harness-0.3.2/docs/maintainer.md +58 -0
  8. {sql_harness-0.3.1 → sql_harness-0.3.2}/docs/standalone-install.md +3 -3
  9. {sql_harness-0.3.1 → sql_harness-0.3.2}/pyproject.toml +2 -2
  10. sql_harness-0.3.2/skills/sql-harness/README.txt +8 -0
  11. sql_harness-0.3.2/skills/sql-harness/references/install.md +145 -0
  12. {sql_harness-0.3.1 → sql_harness-0.3.2}/sql-harness +1 -1
  13. sql_harness-0.3.2/src/sql_harness/analytics.py +290 -0
  14. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/cli.py +28 -1
  15. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/helpers.py +6 -3
  16. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/manager.py +4 -1
  17. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/run.py +15 -4
  18. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/integration/test_pgexercises_practice.py +8 -4
  19. sql_harness-0.3.2/tests/unit/test_analytics.py +323 -0
  20. {sql_harness-0.3.1 → sql_harness-0.3.2}/uv.lock +1 -1
  21. sql_harness-0.3.1/SKILL.md +0 -112
  22. sql_harness-0.3.1/docs/maintainer.md +0 -60
  23. sql_harness-0.3.1/skills/sql-harness/README.txt +0 -8
  24. sql_harness-0.3.1/skills/sql-harness/references/install.md +0 -93
  25. {sql_harness-0.3.1 → sql_harness-0.3.2}/.claude-plugin/marketplace.json +0 -0
  26. {sql_harness-0.3.1 → sql_harness-0.3.2}/.claude-plugin/plugin.json +0 -0
  27. {sql_harness-0.3.1 → sql_harness-0.3.2}/.env.example +0 -0
  28. {sql_harness-0.3.1 → sql_harness-0.3.2}/.github/ISSUE_TEMPLATE/bug-report.yml +0 -0
  29. {sql_harness-0.3.1 → sql_harness-0.3.2}/.github/ISSUE_TEMPLATE/config.yml +0 -0
  30. {sql_harness-0.3.1 → sql_harness-0.3.2}/.github/ISSUE_TEMPLATE/feature-request.yml +0 -0
  31. {sql_harness-0.3.1 → sql_harness-0.3.2}/.gitignore +0 -0
  32. {sql_harness-0.3.1 → sql_harness-0.3.2}/agent-workspace/README.md +0 -0
  33. {sql_harness-0.3.1 → sql_harness-0.3.2}/agent-workspace/agent_helpers.example.py +0 -0
  34. {sql_harness-0.3.1 → sql_harness-0.3.2}/agent-workspace/agent_helpers.py +0 -0
  35. {sql_harness-0.3.1 → sql_harness-0.3.2}/agent-workspace/zones/example_pg/scripts/sh_demo_crud.py +0 -0
  36. {sql_harness-0.3.1 → sql_harness-0.3.2}/agent-workspace/zones/example_pg/skills/sh_demo-schema.md +0 -0
  37. {sql_harness-0.3.1 → sql_harness-0.3.2}/agent-workspace/zones/meta/skills/pool.md +0 -0
  38. {sql_harness-0.3.1 → sql_harness-0.3.2}/agent-workspace/zones/meta/skills/workspace.md +0 -0
  39. {sql_harness-0.3.1 → sql_harness-0.3.2}/docs/connections-toml-example.toml +0 -0
  40. {sql_harness-0.3.1 → sql_harness-0.3.2}/docs/headless-notes.md +0 -0
  41. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/aggregations-advanced.md +0 -0
  42. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/aggregations.md +0 -0
  43. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/encoding-and-charset.md +0 -0
  44. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/joins-advanced.md +0 -0
  45. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/joins.md +0 -0
  46. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/large-result-sets.md +0 -0
  47. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/migrations.md +0 -0
  48. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/mysql/json-columns.md +0 -0
  49. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/pooling.md +0 -0
  50. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/postgres/btree-indexes.md +0 -0
  51. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/postgres/indexes-and-explain.md +0 -0
  52. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/postgres/json-columns.md +0 -0
  53. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/postgres/migrations.md +0 -0
  54. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/postgres/plan-reading.md +0 -0
  55. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/postgres/slow-queries-joins.md +0 -0
  56. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/postgres/specialized-indexes.md +0 -0
  57. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/postgres/table-optimization.md +0 -0
  58. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/recursive-ctes-advanced.md +0 -0
  59. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/recursive-ctes.md +0 -0
  60. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/save-run-cycle.md +0 -0
  61. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/schema-introspection.md +0 -0
  62. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/ssh/auth-and-tunnels.md +0 -0
  63. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/ssh/docker-via-ssh.md +0 -0
  64. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/ssh/ssh.md +0 -0
  65. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/timeouts-and-cancellation.md +0 -0
  66. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/transactions.md +0 -0
  67. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/window-functions-advanced.md +0 -0
  68. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/window-functions.md +0 -0
  69. {sql_harness-0.3.1 → sql_harness-0.3.2}/interaction-skills/zone-skill-auto-surface.md +0 -0
  70. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/__init__.py +0 -0
  71. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/agent_loader.py +0 -0
  72. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/config.py +0 -0
  73. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/drivers/__init__.py +0 -0
  74. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/drivers/mysql.py +0 -0
  75. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/drivers/postgres.py +0 -0
  76. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/drivers/redis.py +0 -0
  77. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/drivers/sqlite.py +0 -0
  78. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/drivers/ssh.py +0 -0
  79. {sql_harness-0.3.1 → sql_harness-0.3.2}/src/sql_harness/paths.py +0 -0
  80. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/__init__.py +0 -0
  81. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/conftest.py +0 -0
  82. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/integration/__init__.py +0 -0
  83. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/integration/test_mysql.py +0 -0
  84. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/integration/test_pg_optimization.py +0 -0
  85. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/integration/test_postgres.py +0 -0
  86. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/integration/test_ssh.py +0 -0
  87. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/unit/__init__.py +0 -0
  88. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/unit/test_agent_loader.py +0 -0
  89. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/unit/test_cli.py +0 -0
  90. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/unit/test_config.py +0 -0
  91. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/unit/test_helpers.py +0 -0
  92. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/unit/test_manager.py +0 -0
  93. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/unit/test_paths.py +0 -0
  94. {sql_harness-0.3.1 → sql_harness-0.3.2}/tests/unit/test_ssh.py +0 -0
@@ -0,0 +1,19 @@
1
+ # AGENTS.md — pointer
2
+
3
+ > All contributor documentation lives in `docs/AGENTS.md` (single docs directory).
4
+ > This pointer file exists only because agent frameworks auto-load `AGENTS.md`
5
+ > by filename at the repo root — read the real content at `docs/AGENTS.md`.
6
+
7
+ # Code priorities
8
+
9
+ - Clarity
10
+ - Precision
11
+ - Low verbosity
12
+ - Versatility
13
+
14
+ # One-line summary
15
+
16
+ sql-harness is a single-process SQL + SSH CLI for agents: plaintext
17
+ connections in one TOML file, heredoc mode with auto-injected helpers.
18
+ Core code in `src/sql_harness/`, agent-editable workspace under
19
+ `agent-workspace/`. Full module list + contributing rules: `docs/AGENTS.md`.
@@ -1,6 +1,6 @@
1
- Metadata-Version: 2.4
1
+ Metadata-Version: 2.5
2
2
  Name: sql-harness
3
- Version: 0.3.1
3
+ Version: 0.3.2
4
4
  Summary: Single-process SQL + SSH CLI for LLM agents. PostgreSQL/MySQL/SSH via SQLAlchemy + paramiko. Plaintext credentials in one TOML file. Helpers auto-injected into the heredoc namespace.
5
5
  Project-URL: Source, https://github.com/zhaoliuxue/much_bigpy/tree/master/lab/sql_harness
6
6
  Project-URL: Issues, https://github.com/zhaoliuxue/much_bigpy/issues
@@ -25,7 +25,7 @@ Description-Content-Type: text/markdown
25
25
 
26
26
  A thin, single-process SQL CLI for LLM agents. Mirrors [browser-harness](https://github.com/browser-use/browser-harness)'s structure but targets relational databases (Postgres, MySQL, Redis-soon).
27
27
 
28
- Connection file format, env vars, and driver list: see `install.md`.
28
+ Connection file format, env vars, and driver list: see `docs/install.md`.
29
29
 
30
30
  ## Quickstart
31
31
 
@@ -53,7 +53,7 @@ PY
53
53
 
54
54
  ## Architecture (~1k lines across 8 core files)
55
55
 
56
- - `install.md` — first-time install + first connection
56
+ - `docs/install.md` — first-time install + first connection
57
57
  - `SKILL.md` — day-to-day usage
58
58
  - `lab/sql_harness/src/sql_harness/` — protected core package
59
59
  - `${XDG_CONFIG_HOME:-~/.config}/sql-harness/connections.toml` — plaintext credentials in ONE place
@@ -77,7 +77,7 @@ PY
77
77
 
78
78
  ## Contributing
79
79
 
80
- PRs and improvements welcome. See `AGENTS.md` for code priorities.
80
+ PRs and improvements welcome. See `docs/AGENTS.md` for code priorities.
81
81
 
82
82
  - **Skills are written by the harness, not by you.** When you figure out a non-obvious SQL flow (a weird schema, a slow query, a JSON column trick), file a skill in `agent-workspace/zones/<conn>/skills/<name>.md` for that DSN, or `agent-workspace/zones/meta/skills/<name>.md` if it generalizes. Future sessions will read it before re-discovering it.
83
83
  - Bug fixes, new drivers, helper additions all welcome.
@@ -0,0 +1,195 @@
1
+ ---
2
+ name: sql-harness
3
+ description: "Always use sql-harness for SQL, SSH, and database operations — querying any DSN (Postgres/MySQL/SQLite/Redis), inspecting schemas, running EXPLAIN plans, indexing analysis, schema migrations, shell exec over SSH, SFTP upload/download, and any cross-DSN workflow. Reach for it whenever the task touches a database, a remote host, or schema work — not just for `query()` calls."
4
+ ---
5
+
6
+ # sql-harness
7
+
8
+ A single-process SQL + SSH CLI for agents. 5 drivers (`postgres`, `mysql`, `sqlite`, `redis`, `ssh`) share one heredoc namespace. Connections live in plaintext in one TOML file.
9
+
10
+ For setup, read `install.md`. For stuck-point mechanics, see `interaction-skills/` (cross-DB) and `interaction-skills/postgres/` (PG depth).
11
+
12
+ ## Reach for sql-harness when...
13
+
14
+ - Any DB task: query, schema, migration, EXPLAIN, indexing, slow-query, stats
15
+ - "Run this on the prod box": SSH workspace (`ssh_exec` / `ssh_upload` / `ssh_run_script`)
16
+ - "Deploy this compose stack to remote": SSH + `interaction-skills/ssh/docker-via-ssh.md`
17
+ - "The DB is behind a bastion / can't reach directly": SSH port-forward / tunnel
18
+ - "Why is this query slow?": `explain_analyze` + `slow_queries` + `index_usage_stats`
19
+ - "Which indexes are dead?": `unused_indexes` + `seq_scan_heavy`
20
+ - "Compare row counts prod vs staging": `use_workspace("prod")` → `query()` → `use_workspace("staging")` → `query()`
21
+ - "Save this heredoc as a reusable script": `sql-harness save <name> -c <conn>`
22
+ - Anything where you would otherwise reach for `psql`, `mysql`, `pgcli`, or hand-rolled `subprocess` over SSH
23
+
24
+ ## Usage
25
+
26
+ ```bash
27
+ sql-harness --help
28
+ sql-harness list
29
+ sql-harness add <name> --driver postgres --url 'postgresql://user:pw@host/db'
30
+ sql-harness add <name> --driver mysql --url 'mysql+pymysql://user:pw@host/db'
31
+ sql-harness add <name> --driver ssh --url 'ssh://user@host:22?key=/path/id_ed25519'
32
+ sql-harness test <name>
33
+
34
+ sql-harness <<'PY'
35
+ use_workspace("local_pg")
36
+ print(query("SELECT version()"))
37
+ print(list_tables())
38
+ PY
39
+ ```
40
+
41
+ - Helpers are pre-imported; call by name in heredoc mode.
42
+ - **First call requires `use_workspace(name)`** — no implicit default.
43
+ - Set `BH_SQL_ZONE_SKILLS=1` once so `use_workspace()` auto-surfaces that zone's per-DSN skill docs (see `interaction-skills/zone-skill-auto-surface.md`).
44
+ - For task-specific helpers, drop them into `agent-workspace/zones/<conn>/helpers.py` and they merge into the heredoc namespace (per-zone, overriding the shared base).
45
+ - Other subcommands: `init` (scaffold connections.toml + install preset zone skills), `doctor` (SELECT 1 on every connection), `edit`/`remove`/`show`, `workspace {list,use,close,show}`, `save`/`run`/`scripts`, `skill` (bare: emit packaged SKILL.md; `install`/`list`/`show`), `paths`, `open`, `stats`, `version`, `ssh {exec,upload,download,run-script,info}`.
46
+ - Zone-scoped subcommands (`save`, `run`, `scripts`, `skill list/show`, `ssh`) need `-c <conn>` or `$BH_SQL_ACTIVE_CONNECTION`.
47
+ - **Analytics (ON by default)**: every CLI call and every doc read (`apply_skill`, `skill`, `skill show`) is counted into `$BH_SQL_HOME/analytics.json`; view with `sql-harness stats`. Every executed command (CLI argv or heredoc body) plus failures is appended to `$BH_SQL_HOME/sql-harness.log` (NDJSON, one record per line). Disable both with `BH_SQL_ANALYTICS=0|off|false|no`. Extension point: `from sql_harness.analytics import on, CLI_INVOKED, DOC_READ, COMMAND_EXECUTED` to attach custom listeners (observer pattern).
48
+
49
+ ## Drivers & capabilities
50
+
51
+ Five drivers, one heredoc namespace. URL scheme must match the backend: MySQL requires `mysql+pymysql://`; PostgreSQL accepts `postgres://` / `postgresql://` (normalized to `postgresql+psycopg://`); SSH accepts `ssh://`, `ssh+password://`, `ssh+key://`.
52
+
53
+ ### Postgres (`postgres://`)
54
+
55
+ Full SQLAlchemy + psycopg3 + 8 PG-only introspection helpers. Reach for these whenever the task is "PG + performance" or "PG + schema":
56
+
57
+ - `explain(sql)` — `EXPLAIN` plan (estimated)
58
+ - `explain_analyze(sql, buffers=True)` — `EXPLAIN (ANALYZE, BUFFERS)` (real times)
59
+ - `table_stats(name, schema=None)` — `pg_stat_user_tables` (seq vs idx scan counts, dead tuples)
60
+ - `index_usage_stats(table=None, schema=None)` — `pg_stat_user_indexes` (idx scan counts per index)
61
+ - `unused_indexes(...)` — indexes with zero idx scans (candidates for DROP)
62
+ - `seq_scan_heavy(...)` — tables routinely full-scanned (candidates for new indexes)
63
+ - `slow_queries(limit=10)` — top by `pg_stat_statements.mean_exec_time`
64
+ - `missing_indexes_hint(...)` — heuristic: many seq scans + table size > threshold
65
+
66
+ See `interaction-skills/postgres/` for the 8 depth docs (plan-reading, B-Tree, specialized indexes, table optimization, slow-queries-joins, jsonb, migrations, indexes-and-explain).
67
+
68
+ ### MySQL (`mysql+pymysql://`)
69
+
70
+ Standard SQLAlchemy + PyMySQL. JSON syntax differs from PG — see `interaction-skills/mysql/json-columns.md`.
71
+
72
+ ### SQLite (`sqlite:///path.db`)
73
+
74
+ Local dev / tests. All standard helpers work. Use `sql-harness add --driver sqlite --url 'sqlite:///<absolute-path>'` (SQLite needs absolute path).
75
+
76
+ ### Redis (`redis://host:port`)
77
+
78
+ Stub driver — wired up but minimal helpers. Prefer `query()` won't work (Redis is not SQL). Skip if your task needs Redis; flag in your reply.
79
+
80
+ ### SSH (`ssh://user@host:port?key=...`)
81
+
82
+ **Not a database** — a remote shell workspace. Reach for ssh whenever the task says "on the box", "the prod host", "deploy this", "upload that file", "run that command remotely":
83
+
84
+ ```python
85
+ use_workspace("prod-app")
86
+ r = ssh_exec("systemctl status myapp") # exit_code + stdout/stderr
87
+ ssh_upload("./deploy.sh", "/tmp/deploy.sh") # SFTP
88
+ ssh_download("/var/log/app.log", "./app.log") # SFTP
89
+ ssh_run_script("./scripts/deploy.sh") # upload + execute
90
+ print(ssh_info()) # user/host/port/SFTP available
91
+ ```
92
+
93
+ CLI equivalent: `sql-harness ssh -c <conn> {exec|upload|download|run-script|info} <args>`.
94
+
95
+ See `interaction-skills/ssh/` for: `ssh.md` (helpers + CLI + gotchas), `docker-via-ssh.md` (docker-compose deploy loop over SSH), `auth-and-tunnels.md` (auth scheme 选型 + 堡垒机 + 端口转发).
96
+
97
+ ## Generic SQL helpers (all DB drivers)
98
+
99
+ | Helper | Returns | Use |
100
+ |---|---|---|
101
+ | `query(sql, params=None)` | `list[dict]` | SELECTs only |
102
+ | `execute(sql, params=None)` | `dict` (`lastrowid`, `rowcount`) | INSERT/UPDATE/DELETE/DDL |
103
+ | `list_tables(schema=None)` | `list[str]` | Schema recon — always start here |
104
+ | `describe(table, schema=None)` | `list[dict]` | Columns + types — before writing SQL |
105
+ | `table(name, schema=None, limit=None)` | `list[dict]` | Quick peek: `SELECT * FROM x LIMIT n` |
106
+ | `with_transaction()` | context manager yielding `Connection` | ≥ 2 statements / DDL / atomicity |
107
+ | `run_sql_file(path)` | `list[dict]` | Multi-statement `.sql`, one transaction |
108
+ | `server_version()` | `str` | Round-trip liveness check |
109
+ | `connection_info()` | `dict` | Current workspace metadata |
110
+ | `use_workspace_info(name)` | `dict` | Zone info + skills + scripts (always surfaces) |
111
+
112
+ ## Interaction Skills
113
+
114
+ Stuck-point mechanics — read on demand by filename. Top-level (cross-DB):
115
+
116
+ - `interaction-skills/aggregations.md` (+ `aggregations-advanced.md`)
117
+ - `interaction-skills/encoding-and-charset.md`
118
+ - `interaction-skills/joins.md` (+ `joins-advanced.md`)
119
+ - `interaction-skills/large-result-sets.md`
120
+ - `interaction-skills/migrations.md`
121
+ - `interaction-skills/pooling.md`
122
+ - `interaction-skills/recursive-ctes.md` (+ `recursive-ctes-advanced.md`)
123
+ - `interaction-skills/save-run-cycle.md`
124
+ - `interaction-skills/schema-introspection.md`
125
+ - `interaction-skills/timeouts-and-cancellation.md`
126
+ - `interaction-skills/transactions.md`
127
+ - `interaction-skills/window-functions.md` (+ `window-functions-advanced.md`)
128
+ - `interaction-skills/zone-skill-auto-surface.md`
129
+
130
+ PG-specific depth — `interaction-skills/postgres/`:
131
+
132
+ - `interaction-skills/postgres/btree-indexes.md`
133
+ - `interaction-skills/postgres/indexes-and-explain.md`
134
+ - `interaction-skills/postgres/json-columns.md`
135
+ - `interaction-skills/postgres/migrations.md`
136
+ - `interaction-skills/postgres/plan-reading.md`
137
+ - `interaction-skills/postgres/slow-queries-joins.md`
138
+ - `interaction-skills/postgres/specialized-indexes.md`
139
+ - `interaction-skills/postgres/table-optimization.md`
140
+
141
+ MySQL-specific — `interaction-skills/mysql/`:
142
+
143
+ - `interaction-skills/mysql/json-columns.md`
144
+
145
+ SSH-specific — `interaction-skills/ssh/`:
146
+
147
+ - `interaction-skills/ssh/auth-and-tunnels.md`
148
+ - `interaction-skills/ssh/docker-via-ssh.md`
149
+ - `interaction-skills/ssh/ssh.md`
150
+
151
+ ## Cross-DSN strategy skills
152
+
153
+ Reached via `apply_skill("pool")` etc. from any active zone — `zones/meta/skills/` is the passive fallback layer.
154
+
155
+ - `agent-workspace/zones/meta/skills/pool.md` — pool sizing, `pre_ping`, idle reuse.
156
+ - `agent-workspace/zones/meta/skills/workspace.md` — workspace isolation, multi-DSN workflows.
157
+
158
+ ## What actually works (field-tested)
159
+
160
+ - **Start with `list_tables` + `describe`, not `query`.** Before writing SQL on an unknown DB, run `list_tables(schema=None)` then `describe("table", schema=None)` to learn the columns. Skips the trial-and-error SELECT cycle.
161
+ - **Set `BH_SQL_ZONE_SKILLS=1` once** so every `use_workspace()` surfaces that zone's per-DSN skill docs as a hint list (10 filenames max). Then `apply_skill(name)` reads the body when relevant.
162
+ - **Default to `with_transaction()`** for any write with ≥ 2 statements (insert + update, drop + create, …). Atomic + automatic rollback on exception.
163
+ - **For PG performance tasks, reach for the 8 PG-only helpers first** (`explain_analyze`, `slow_queries`, `unused_indexes`, `seq_scan_heavy`, `table_stats`, `index_usage_stats`, `missing_indexes_hint`). Don't hand-parse `pg_stat_*` SQL.
164
+ - **For remote tasks, reach for SSH workspace first.** Don't reach for raw `subprocess` + ssh-key files in agent code — `ssh_exec("systemctl restart myapp")` is one line.
165
+ - **For "insert returning the new id"**: psycopg supports `INSERT … RETURNING id`; MySQL has no `RETURNING` — use `execute(...)` and read `result.lastrowid`.
166
+ - **Streaming large result sets**: `query()` loads every row into memory. For >10k rows, use `ws.engine.connect().execution_options(stream_results=True)` and iterate manually.
167
+ - **Read-only "is this alive?" check**: `print(server_version())` — single round-trip, works on every DSN, no per-driver fudging.
168
+ - **Auth wall**: redirected to login → stop and ask the user. Don't type credentials from `connection_info`/`describe`.
169
+ - **Credentials in TOML**: use `${env:VAR}` indirection for prod secrets. The connections.toml loader expands these at load time.
170
+ - **Cross-DB SQL**: prefer ANSI syntax (`'string'`, `LIMIT n OFFSET m`, `COALESCE`, `CURRENT_TIMESTAMP`). PG-specific: `::TYPE`, `JSONB`, `RETURNING`. MySQL-specific: `AUTO_INCREMENT`, backticks, `?` placeholders (sqlalchemy normalizes to `:name`).
171
+ - **Save the workflow**: every working block → `sql-harness save <name> -c <conn>`. Tomorrow's session re-runs with one command. See `interaction-skills/save-run-cycle.md`.
172
+
173
+ ## Design Constraints
174
+
175
+ - One connection = one workspace; never share engines across workspaces.
176
+ - Connection pool defaults: `size=5, recycle=3600, pre_ping=True` (per-connection overrides win).
177
+ - `with_transaction()` yields a `Connection`; use `conn.execute(text(...))` for raw control.
178
+ - `query()` returns `list[dict]`; use `execute()` for INSERT/UPDATE/DELETE.
179
+ - `list_tables()` and `describe()` are read-only schema introspection helpers.
180
+ - SSH workspaces expose `ssh_exec` / `ssh_upload` / etc. Calling them on a non-SSH workspace raises `RuntimeError`.
181
+ - PG-only helpers (`explain_analyze`, `table_stats`, ...) raise `NotImplementedError` on non-PG workspaces. Detect with `ws.driver.name == "postgres"`.
182
+ - All engines live in `SqlHarness`'s in-memory registry. Single-process, no daemon, no remote backend.
183
+
184
+ ## Gotchas
185
+
186
+ - Driver label in TOML must match the URL scheme (`postgres` ⇄ `postgresql://`, `mysql` ⇄ `mysql+pymysql://`, `ssh` ⇄ `ssh://` / `ssh+password://` / `ssh+key://`).
187
+ - Passwords in TOML are plaintext; use `${env:VAR}` indirection for prod secrets.
188
+ - `query()` is for SELECTs only. For INSERTs, use `execute()`.
189
+ - For tables > 10k rows, use `engine.connect().execution_options(stream_results=True)` and iterate manually — `query()` loads everything into memory.
190
+ - SSH SFTP subsystem may be disabled on the remote host. `ssh_upload/ssh_download` raise `RuntimeError("SFTP subsystem not available")`; fall back to `tar -czf - | ssh host tar -xzf -` via `ssh_exec`.
191
+ - Default `ssh_exec` timeout is 30 s. Long-running commands (e.g. `pg_dump`, `tar czf`) need explicit `timeout=`.
192
+
193
+ ## Domain / table skills
194
+
195
+ Per-table or per-schema skills: drop a markdown file into `agent-workspace/zones/<conn>/skills/<name>.md`, read with `apply_skill(name)`. Cross-DSN strategy goes to `zones/meta/skills/` instead. PG optimization depth (plan reading, indexes, `table_stats`, `slow_queries`, …) lives in `interaction-skills/postgres/`.
@@ -15,11 +15,12 @@ Core code lives in `lab/sql_harness/src/sql_harness/`:
15
15
  - `manager.py` — SqlHarness class: holds engines, workspaces, skills registry
16
16
  - `helpers.py` — heredoc helpers (auto-imported)
17
17
  - `agent_loader.py` — loads `$BH_SQL_AGENT_WORKSPACE/agent_helpers.py`
18
+ - `analytics.py` — event bus (observer) + JSON stats + NDJSON execution log: CLI call counts, doc-read tracking, command/error history; `BH_SQL_ANALYTICS` (default ON), view with `sql-harness stats`
18
19
  - `run.py` — heredoc entry; delegates to CLI otherwise
19
- - `cli.py` — argparse subparser CLI (~10 subcommands)
20
+ - `cli.py` — argparse subparser CLI (18 subcommands)
20
21
 
21
22
  `SKILL.md` tells agents how to use the harness.
22
- `install.md` tells agents how to install and configure it.
23
+ `docs/install.md` tells agents how to install and configure it.
23
24
 
24
25
  An agent operating the harness only edits inside `agent-workspace/`:
25
26
  - `agent_helpers.py` — task-specific query helpers the agent adds
@@ -28,4 +29,4 @@ An agent operating the harness only edits inside `agent-workspace/`:
28
29
  # Contributing
29
30
  Consider what is really needed. Prefer the smallest diff that fixes the bug.
30
31
 
31
- To add a backend: implement `drivers/<name>.py` against the Driver protocol, register in `drivers/__init__.py:get_driver()`, add `tests/unit/test_<name>.py` (see install.md §Drivers).
32
+ To add a backend: implement `drivers/<name>.py` against the Driver protocol, register in `drivers/__init__.py:get_driver()`, add `tests/unit/test_<name>.py` (see `docs/install.md` §Drivers).
@@ -2,7 +2,7 @@
2
2
 
3
3
  A thin, single-process SQL CLI for LLM agents. Mirrors [browser-harness](https://github.com/browser-use/browser-harness)'s structure but targets relational databases (Postgres, MySQL, Redis-soon).
4
4
 
5
- Connection file format, env vars, and driver list: see `install.md`.
5
+ Connection file format, env vars, and driver list: see `docs/install.md`.
6
6
 
7
7
  ## Quickstart
8
8
 
@@ -30,7 +30,7 @@ PY
30
30
 
31
31
  ## Architecture (~1k lines across 8 core files)
32
32
 
33
- - `install.md` — first-time install + first connection
33
+ - `docs/install.md` — first-time install + first connection
34
34
  - `SKILL.md` — day-to-day usage
35
35
  - `lab/sql_harness/src/sql_harness/` — protected core package
36
36
  - `${XDG_CONFIG_HOME:-~/.config}/sql-harness/connections.toml` — plaintext credentials in ONE place
@@ -54,7 +54,7 @@ PY
54
54
 
55
55
  ## Contributing
56
56
 
57
- PRs and improvements welcome. See `AGENTS.md` for code priorities.
57
+ PRs and improvements welcome. See `docs/AGENTS.md` for code priorities.
58
58
 
59
59
  - **Skills are written by the harness, not by you.** When you figure out a non-obvious SQL flow (a weird schema, a slow query, a JSON column trick), file a skill in `agent-workspace/zones/<conn>/skills/<name>.md` for that DSN, or `agent-workspace/zones/meta/skills/<name>.md` if it generalizes. Future sessions will read it before re-discovering it.
60
60
  - Bug fixes, new drivers, helper additions all welcome.
@@ -88,10 +88,14 @@ pool_size = 10
88
88
  | Var | Purpose |
89
89
  |---|---|
90
90
  | `BH_SQL_HOME` | Override state root (`~/.config/sql-harness` by default) |
91
+ | `BH_SQL_CONFIG_DIR` | Override config dir (defaults to `BH_SQL_HOME`) |
91
92
  | `BH_SQL_CONFIG_FILE` | Override connections.toml path |
92
93
  | `BH_SQL_AGENT_WORKSPACE` | Override agent-workspace directory |
93
94
  | `BH_SQL_RUNTIME_DIR` | Override runtime state dir |
94
95
  | `BH_SQL_TMP_DIR` | Override temp dir |
96
+ | `BH_SQL_ACTIVE_CONNECTION` | Active DSN zone for zone-scoped CLI (`save`/`run`/`scripts`/`skill list/show`) when no `-c` flag |
97
+ | `BH_SQL_ZONE_SKILLS=1` | `use_workspace()` auto-surfaces the zone's skills/scripts |
98
+ | `BH_SQL_ANALYTICS` | Analysis mode (CLI call counts + doc-read tracking + execution log). **Default ON**; set `0`/`off`/`false`/`no` to disable. Data: `$BH_SQL_HOME/analytics.json` (view with `sql-harness stats`) + `$BH_SQL_HOME/sql-harness.log` (append-only NDJSON: every executed command and its outcome) |
95
99
  | `BH_PG_URL` | Used by integration tests (skipped if unset) |
96
100
  | `BH_MYSQL_URL` | Used by integration tests (skipped if unset) |
97
101
 
@@ -115,7 +119,7 @@ To add another SSH host: append a `[[connections]]` block with `driver = "ssh"`
115
119
  | `ModuleNotFoundError: No module named 'psycopg'` | Run `uv sync` |
116
120
  | `sql-harness: command not found` | Use `uv run sql-harness ...` or activate `.venv` |
117
121
  | `password authentication failed` | Check URL; use `${env:VAR}` for prod secrets |
118
- | `unknown driver: 'X'` | Only `postgres`, `mysql`, `redis`, `sqlite` are built-in |
122
+ | `unknown driver: 'X'` | Only `postgres`, `mysql`, `redis`, `sqlite`, `ssh`/`ssh+password`/`ssh+key` are built-in |
119
123
  | Heredoc mode complains `no workspace is active` | You forgot `use_workspace(name)` |
120
124
  | Heredoc mode complains `name 'X' is not defined` | You forgot the import — all helpers are pre-imported, just call them |
121
125
 
@@ -0,0 +1,58 @@
1
+ # sql-harness standalone project
2
+
3
+ ## Quick reference for the maintainer
4
+
5
+ | Task | Command |
6
+ |---|---|
7
+ | Install (editable dev mode) | `uv tool install --python 3.12 -e .` |
8
+ | Build wheel + sdist | `uv build` |
9
+ | Publish to TestPyPI | `uv publish --repository testpypi --token $TEST_PYPI_TOKEN` |
10
+ | Publish to PyPI | `uv publish --token $PYPI_TOKEN` |
11
+ | Refresh an agent's registered skill copy | `uv run sql-harness skill install` (or `skill > ~/.claude/skills/sql-harness/SKILL.md`) |
12
+
13
+ ## Why the SKILL.md sync step exists
14
+
15
+ The in-package skill docs are **generated at build time**, not kept as
16
+ hand-synced copies. `pyproject.toml` `[tool.hatch.build.targets.wheel.force-include]`
17
+ maps the repo docs into the wheel:
18
+
19
+ - `SKILL.md` → `sql_harness/_skills/SKILL.md`
20
+ - `interaction-skills` → `sql_harness/_skills/interaction-skills`
21
+ - `agent-workspace/zones/meta/skills` → `sql_harness/_skills/zones-meta-skills`
22
+
23
+ The root `SKILL.md` is the **single source of truth** (v0.3.0 consolidation —
24
+ the old `skills/sql-harness/SKILL.md` and `src/sql_harness/SKILL.md` copies
25
+ were removed). After editing it, just `uv build` again. `sql-harness skill`
26
+ reads `sql_harness/_skills/SKILL.md` from the installed package and falls
27
+ back to the repo-root `SKILL.md` in a source checkout.
28
+
29
+ ## Build outputs
30
+
31
+ ```bash
32
+ $ uv build
33
+ Building source distribution...
34
+ Building wheel from source distribution...
35
+ Successfully built dist\sql_harness-0.3.0.tar.gz
36
+ Successfully built dist\sql_harness-0.3.0-py3-none-any.whl
37
+ ```
38
+
39
+ The wheel contains the package modules + the `_skills/` doc bundle
40
+ (SKILL.md + interaction-skills tree + zones-meta-skills).
41
+
42
+ ## Test before publish
43
+
44
+ ```bash
45
+ # standalone venv
46
+ uv sync
47
+ uv run pytest tests/unit -q # 80 tests must pass
48
+ uv run sql-harness --help # CLI works
49
+ uv run sql-harness skill # SKILL.md emit works
50
+ ```
51
+
52
+ ## Local PyPI dry-run (no upload)
53
+
54
+ ```bash
55
+ uv publish --dry-run
56
+ # → checks both dist files against upload.pypi.org/legacy/
57
+ # → errors with "no OIDC token" (expected; pass --token to actually publish)
58
+ ```
@@ -162,6 +162,6 @@ monorepo access to `from sql_harness import ...` for testing — no breakage.
162
162
 
163
163
  ## See also
164
164
 
165
- - `lab/sql_harness/install.md` — first-time install + troubleshooting
166
- - `lab/sql_harness/AGENTS.md` — code priorities + extension recipe
167
- - `lab/sql_harness/README.md` — user-facing quickstart
165
+ - `lab/sql_harness/docs/install.md` — first-time install + troubleshooting
166
+ - `lab/sql_harness/docs/AGENTS.md` — code priorities + extension recipe
167
+ - `lab/sql_harness/docs/README.md` — user-facing quickstart
@@ -1,8 +1,8 @@
1
1
  [project]
2
2
  name = "sql-harness"
3
- version = "0.3.1"
3
+ version = "0.3.2"
4
4
  description = "Single-process SQL + SSH CLI for LLM agents. PostgreSQL/MySQL/SSH via SQLAlchemy + paramiko. Plaintext credentials in one TOML file. Helpers auto-injected into the heredoc namespace."
5
- readme = "README.md"
5
+ readme = "docs/README.md"
6
6
  license = { text = "MIT" }
7
7
  authors = [
8
8
  { name = "much-bigpy" },
@@ -0,0 +1,8 @@
1
+ # sql-harness — skills bundle (mirrors browser-harness/skills/browser-harness/)
2
+
3
+ v0.3.0 起 SKILL.md 副本已移除:分发版由 pyproject force-include 在 build 时
4
+ 从根 SKILL.md 自动生成到 wheel 的 sql_harness/_skills/(本目录不再放 SKILL.md)。
5
+
6
+ Contents:
7
+ - `references/install.md` — one-time install + first-connection guide
8
+ (synced copy of the repo-root install.md)
@@ -0,0 +1,145 @@
1
+ ---
2
+ name: sql-install
3
+ description: Install sql-harness and configure the first connection.
4
+ ---
5
+
6
+ # sql-harness install
7
+
8
+ Use once. For SQL work, read `SKILL.md`.
9
+
10
+ ## 1. Install deps
11
+
12
+ ```bash
13
+ cd /path/to/this/repo
14
+ uv sync
15
+ ```
16
+
17
+ This pulls in `psycopg[binary]`, `pymysql`, `sqlalchemy`, plus the dev deps.
18
+
19
+ ## 2. Scaffold a starter connections.toml
20
+
21
+ ```bash
22
+ uv run sql-harness init
23
+ ```
24
+
25
+ Writes `~/.config/sql-harness/connections.toml` (or `$BH_SQL_HOME/connections.toml`) with two example connections — `local_pg` and `local_mysql`. Edit the file to add real URLs.
26
+
27
+ ## 3. Add your first connection
28
+
29
+ ```bash
30
+ uv run sql-harness add my_pg \
31
+ --driver postgres \
32
+ --url 'postgresql://user:pw@localhost:5432/db' \
33
+ --description "My dev DB"
34
+ ```
35
+
36
+ Or edit `connections.toml` directly. The full schema for `connections.toml` is documented below.
37
+
38
+ For prod secrets, use `${env:VAR}` inside `url` — they're expanded at load time from the process environment.
39
+
40
+ ## 4. Verify
41
+
42
+ ```bash
43
+ uv run sql-harness test my_pg
44
+ ```
45
+
46
+ Should print `{"name": "my_pg", "ok": true, "select_1": 1, "latency_ms": ...}`.
47
+
48
+ ## 5. Use it
49
+
50
+ ```bash
51
+ uv run sql-harness <<'PY'
52
+ use_workspace("my_pg")
53
+ print(query("SELECT version()"))
54
+ print(list_tables())
55
+ PY
56
+ ```
57
+
58
+ ## Connection file format
59
+
60
+ Default path: `${XDG_CONFIG_HOME:-~/.config}/sql-harness/connections.toml`. Override with `BH_SQL_CONFIG_FILE`.
61
+
62
+ ```toml
63
+ default_workspace = "local_pg"
64
+
65
+ [pool_defaults]
66
+ size = 5
67
+ recycle = 3600
68
+ pre_ping = true
69
+ echo = false
70
+
71
+ [[connections]]
72
+ name = "local_pg"
73
+ driver = "postgres"
74
+ url = "postgresql+psycopg://postgres:postgres@localhost:5432/postgres"
75
+ description = "Local PostgreSQL for dev"
76
+
77
+ [[connections]]
78
+ name = "prod_pg"
79
+ driver = "postgres"
80
+ url = "${env:PROD_PG_URL}" # env-var indirection
81
+ pool_size = 10
82
+ ```
83
+
84
+ `${env:VAR}` placeholders inside `url` are expanded at load time from the process environment — keep prod secrets out of the file.
85
+
86
+ ## Configuration env vars
87
+
88
+ | Var | Purpose |
89
+ |---|---|
90
+ | `BH_SQL_HOME` | Override state root (`~/.config/sql-harness` by default) |
91
+ | `BH_SQL_CONFIG_DIR` | Override config dir (defaults to `BH_SQL_HOME`) |
92
+ | `BH_SQL_CONFIG_FILE` | Override connections.toml path |
93
+ | `BH_SQL_AGENT_WORKSPACE` | Override agent-workspace directory |
94
+ | `BH_SQL_RUNTIME_DIR` | Override runtime state dir |
95
+ | `BH_SQL_TMP_DIR` | Override temp dir |
96
+ | `BH_SQL_ACTIVE_CONNECTION` | Active DSN zone for zone-scoped CLI (`save`/`run`/`scripts`/`skill list/show`) when no `-c` flag |
97
+ | `BH_SQL_ZONE_SKILLS=1` | `use_workspace()` auto-surfaces the zone's skills/scripts |
98
+ | `BH_SQL_ANALYTICS` | Analysis mode (CLI call counts + doc-read tracking + execution log). **Default ON**; set `0`/`off`/`false`/`no` to disable. Data: `$BH_SQL_HOME/analytics.json` (view with `sql-harness stats`) + `$BH_SQL_HOME/sql-harness.log` (append-only NDJSON: every executed command and its outcome) |
99
+ | `BH_PG_URL` | Used by integration tests (skipped if unset) |
100
+ | `BH_MYSQL_URL` | Used by integration tests (skipped if unset) |
101
+
102
+ ## Drivers
103
+
104
+ | Driver | Backend | Package | Status |
105
+ |---|---|---|---|
106
+ | `postgres` | PostgreSQL | `psycopg[binary]>=3.2` | ✅ |
107
+ | `mysql` | MySQL | `pymysql>=1.1` | ✅ |
108
+ | `redis` | Redis | `redis>=5.0` (not yet added) | 🚧 stub |
109
+ | `sqlite` | SQLite | stdlib (`sqlite3`) | ✅ (test-only) |
110
+ | `ssh` / `ssh+password` / `ssh+key` | remote shell + SFTP (paramiko) | `paramiko>=3.4` | ✅ |
111
+
112
+ To enable Redis: `uv add "redis>=5.0,<6"` then implement `drivers/redis.py`.
113
+ To add another SSH host: append a `[[connections]]` block with `driver = "ssh"` and an `ssh://...` URL.
114
+
115
+ ## Troubleshooting
116
+
117
+ | Problem | Fix |
118
+ | -------------------------------------------------- | --------------------------------------------------------------------- |
119
+ | `ModuleNotFoundError: No module named 'psycopg'` | Run `uv sync` |
120
+ | `sql-harness: command not found` | Use `uv run sql-harness ...` or activate `.venv` |
121
+ | `password authentication failed` | Check URL; use `${env:VAR}` for prod secrets |
122
+ | `unknown driver: 'X'` | Only `postgres`, `mysql`, `redis`, `sqlite`, `ssh`/`ssh+password`/`ssh+key` are built-in |
123
+ | Heredoc mode complains `no workspace is active` | You forgot `use_workspace(name)` |
124
+ | Heredoc mode complains `name 'X' is not defined` | You forgot the import — all helpers are pre-imported, just call them |
125
+
126
+ ## Windows notes
127
+
128
+ `$XDG_CONFIG_HOME` doesn't exist by default; sql-harness falls back to `%USERPROFILE%\.config\sql-harness`. To override, set `BH_SQL_HOME`.
129
+
130
+ PyMySQL is pure-Python so it works on Windows without native deps.
131
+
132
+ ## Linux headless
133
+
134
+ If running sql-harness in a headless Linux container, make sure the connections you're testing are reachable from the container's network namespace.
135
+
136
+ ## If Still Broken
137
+
138
+ Inspect the source for ground truth:
139
+
140
+ - `src/sql_harness/config.py` — TOML schema + `ConnectionConfig` / `ConnectionsConfig` dataclasses
141
+ - `src/sql_harness/paths.py` — XDG-style state-dir resolution
142
+ - `src/sql_harness/manager.py` — `SqlHarness` engine registry
143
+ - `src/sql_harness/drivers/__init__.py:get_driver()` — backend dispatch
144
+
145
+ State lives under `${XDG_CONFIG_HOME:-~/.config}/sql-harness` by default: connections.toml, agent workspace, runtime sockets, and temp files. Override with `BH_SQL_HOME`.
@@ -16,7 +16,7 @@ set -euo pipefail
16
16
  # browser-harness keeps the launcher at repo root; sql-harness's "repo root" is lab/sql_harness/).
17
17
  SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
18
18
  ROOT="$SCRIPT_DIR"
19
- SRC="$ROOT/src"
19
+ SRC="$ROOT/src"r
20
20
 
21
21
  # Isolate dev state.
22
22
  export BH_SQL_HOME="$ROOT/.sql-harness-dev"