postgresai 0.18.0-dev.9 → 0.18.0-rc.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -216,21 +216,57 @@ and the result's `provider` remains `self-managed` for compatibility. It works w
216
216
  PostgreSQL endpoint where the supplied user can create the monitoring role and grant privileges.
217
217
  Use a direct endpoint, not a pooler.
218
218
 
219
- The endpoint must be reachable from this machine, which prepares the database, and from
220
- PostgresAI. Before provisioning, PostgresAI tests the connection from its platform, whose
221
- address is not published, so the port must accept connections from the internet at that point.
222
- After provisioning, the monitoring server connects from its own IP address, shown on its page in
223
- the Console (Monitoring, then the instance). A database that must stay behind an IP allowlist
224
- needs AWS PrivateLink (RDS) or self-hosted monitoring (`--self-hosted`).
225
-
226
- Without `--provider`, RDS and Supabase hostnames still go to their console integration.
227
- `generic` does not configure AWS PrivateLink, open a firewall, or remove TLS, permission, or
219
+ The endpoint must be reachable from this machine, which prepares the database. The monitoring
220
+ box connects from its own static IP. When PostgresAI can't reach the database the way an IP
221
+ allowlist refuses (a timeout, a refused connection, no `pg_hba.conf` entry), deploy reserves the
222
+ box's IP before the box exists and stops at it: at a terminal it shows the IP and waits (Enter
223
+ continues, `s` stops); elsewhere it exits 3 with `"step":"allow_ip"` and `ip`, `cidr`, `host`,
224
+ `port`, `reserved_until`. Allow the IP on the database, then run `pgai mon deploy <url> --yes
225
+ --ip-allowed` to create the box with it. `--ip-first` asks for the IP first even when PostgresAI
226
+ can reach the database. A re-run never reserves a second IP; the IP is held 7 days and stays the
227
+ box's for its life. If the box still can't connect, it waits for up to 90 minutes, then is removed
228
+ and its IP kept: allow it, and run with `--ip-allowed` again. Any database can also use self-hosted
229
+ monitoring (`--self-hosted`).
230
+
231
+ An Amazon RDS that is not publicly accessible is monitored over AWS PrivateLink: nothing becomes public and no port is
232
+ opened. Run the deploy anywhere: where the database is reachable (in its VPC) it runs the setup SQL itself and checks it
233
+ before the box exists; where it is not, it prints the SQL to run as the master user on a machine that reaches the
234
+ database, and the box checks it over PrivateLink:
235
+
236
+ ```bash
237
+ pgai mon deploy 'postgresql://postgres:<password>@<instance>.<id>.<region>.rds.amazonaws.com:5432/app'
238
+ ```
239
+
240
+ It reads the instance with the aws CLI when it can (read-only: `sts get-caller-identity`, `rds describe-db-instances`,
241
+ `cloudformation describe-stacks`, `ec2 describe-instance-types`; the profile it uses is printed, `--aws-profile`
242
+ picks one, `--no-aws` skips AWS), else asks public DNS whether the RDS is public (inside the VPC, AWS resolves even a
243
+ public RDS to a private address). Then: the price; a CloudFormation link that creates the stack in your AWS account,
244
+ filled in with what AWS said (`--terraform` writes the same as a Terraform module); the `postgres_ai_mon` role; the box
245
+ on AWS in the RDS's region; and its check of the database over PrivateLink. At a terminal it waits up to 30 minutes for
246
+ the stack, under any name: it is found by its `RdsInstanceIdentifier` (with several for the RDS, you pick one; scripts get
247
+ them listed in `stacks`). `[o]` opens the link where a browser can open. Without AWS, the link leaves the VPC, subnets and
248
+ security group to choose in the form (listed above it), and deploy asks for the stack's `VpceServiceName`. Scripts and agents get
249
+ exit 3 with `"step":"privatelink_stack"` and `cfn_url`: create the stack, then re-run (with `--vpce-service <name>`
250
+ when AWS is not read here). When the URL's user can't create the role, the SQL comes back (`"step":"prepare_db"` with
251
+ `sql`, `nonce`, `monitoring_password`): run it as the master user in that database, then re-run with
252
+ `PGAI_MON_PASSWORD` and `PGAI_MON_NONCE` set to those. `--host-stats` adds host CPU, memory, disk and IOPS through a
253
+ read-only role the stack creates. `--privatelink` / `--no-privatelink` decide public or private instead. Aurora and RDS
254
+ Proxy endpoints are not supported over PrivateLink. A publicly accessible RDS is monitored directly.
255
+
256
+ Supabase hostnames go to the Supabase consent flow (`pgai mon deploy` for Supabase).
257
+
258
+ A Supabase project needs no URL at all: `pgai mon deploy --provider supabase` connects the Supabase organization
259
+ if needed and lists its projects. At a terminal you pick one; agents and scripts pass
260
+ `--supabase-project <name or project ID>`. Without either, with `--yes`, or not at a terminal, it stops (exit 3) with the list
261
+ in the JSON result (`projects`, each with `selectable` and `reason`). The vCPUs come from the project's compute size.
262
+
263
+ `generic` does not configure AWS PrivateLink (`rds` does, for a private RDS), open a firewall, or remove TLS, permission, or
228
264
  payment checks. A private endpoint needs a network path, or monitoring hosted in that network
229
265
  (`--self-hosted`).
230
266
 
231
267
  `pgai mon deploy` and `pgai dblab deploy` share their options and results: the database URL as the
232
- argument (or `--db-url`), `--name` (default: from the database), `--location` (a Hetzner location;
233
- default: any with capacity), `--wait <minutes>` / `--no-wait`, `--yes` (never prompt), `--json`, the
268
+ argument (or `--db-url`), `--name` (default: from the database), `--cloud` and `--location` (where the box
269
+ runs; see the table), `--wait <minutes>` / `--no-wait`, `--yes` (never prompt), `--json`, the
234
270
  `status` words and exit codes below, and `instances list | watch | delete`, which take an id or a name
235
271
  (a name two instances have is refused, with their ids). `watch` is also `status`.
236
272
 
@@ -253,14 +289,20 @@ terminal, or with `--json`, it only points to `pgai mon deploy`.
253
289
  | `--clickhouse-key <key-id>:<key-secret>` | ClickHouse Cloud API key, for CPU, memory and disk |
254
290
  | `--self-hosted` | run the stack on this machine (`mon local-install`) instead of PostgresAI Cloud |
255
291
  | `--reset-password` | the monitoring role exists and its password is lost: set a new one (see below) |
256
- | `--monitoring-user <name>` | the monitoring role to create or use (default `postgres_ai_mon`; at most 63 bytes; MCP: `monitoring_user`). A custom role always carries its own password: PostgresAI keeps only `postgres_ai_mon`'s, so creating or resetting a custom role needs `PGAI_MON_PASSWORD` (an agent passes the role's URL). A name with `'`, `$`, `%`, backslash or a control character is refused, and so is a role that is a superuser or has `CREATEDB`, `CREATEROLE`, `REPLICATION` or `BYPASSRLS`, is a member of a role other than `pg_monitor`, `pg_read_all_stats`, `pg_read_all_settings` and `pg_stat_scan_tables` (directly or through other roles, also through `pg_monitor` itself), owns objects, has grants in another database of the server, or has grants in the database beyond `CONNECT`, `USAGE` on schemas other than `pg_toast`, `SELECT` outside `pg_catalog` and `pg_toast`, and the grants pgai makes, or holds any grant `WITH GRANT OPTION` or a membership `WITH ADMIN OPTION`. So a custom role used for databases on one server is refused for the second one. `SELECT` on application tables and views is allowed, including a view over `pg_authid`: what a role may read through views is the owner's choice. Ignored for a host handed to the Console |
292
+ | `--monitoring-user <name>` | the monitoring role to create or use (default `postgres_ai_mon`; at most 63 bytes; MCP: `monitoring_user`). A custom role always carries its own password: PostgresAI keeps only `postgres_ai_mon`'s, so creating or resetting a custom role needs `PGAI_MON_PASSWORD` (an agent passes the role's URL). A name with `'`, `$`, `%`, backslash or a control character is refused, and so is a role that is a superuser or has `CREATEDB`, `CREATEROLE`, `REPLICATION` or `BYPASSRLS`, is a member of a role other than `pg_monitor`, `pg_read_all_stats`, `pg_read_all_settings` and `pg_stat_scan_tables` (directly or through other roles, also through `pg_monitor` itself), owns objects, has grants in another database of the server, or has grants in the database beyond `CONNECT`, `USAGE` on schemas other than `pg_toast`, `SELECT` outside `pg_catalog` and `pg_toast`, and the grants pgai makes, or holds any grant `WITH GRANT OPTION` or a membership `WITH ADMIN OPTION`. So a custom role used for databases on one server is refused for the second one. `SELECT` on application tables and views is allowed, including a view over `pg_authid`: what a role may read through views is the owner's choice. Refused for a private RDS over AWS PrivateLink (the box uses `postgres_ai_mon`) and for Supabase, where PostgresAI creates `postgres_ai_mon` through the Supabase Management API |
257
293
  | `--db-url <url>` | the database URL, instead of the argument |
258
294
  | `--name <name>` | the instance's name (default: `host[:port]/database`); ignored, with a note, for a database already monitored |
259
- | `--location <location>` | Hetzner location `fsn1`, `nbg1` or `hel1` (default: any with capacity) |
295
+ | `--privatelink` / `--no-privatelink` | RDS: over AWS PrivateLink (not publicly accessible) or directly; default: decided from AWS, else public DNS (see the PrivateLink paragraph above) |
296
+ | `--vpce-service <name>` | RDS over PrivateLink: the stack's `VpceServiceName` output, when AWS is not read here |
297
+ | `--host-stats` / `--host-stats-role <arn>` | RDS over PrivateLink: host CPU, memory, disk and IOPS through the stack's read-only role (its `HostStatsRoleArn` output, when not read from AWS) |
298
+ | `--aws-profile <name>` / `--no-aws` | RDS: the aws CLI profile that reads the instance and its stack (read-only); `--no-aws` skips AWS |
299
+ | `--terraform` | RDS over PrivateLink: write the stack as a Terraform module (`./postgresai-privatelink-<instance>/`) instead of the CloudFormation link |
300
+ | `--cloud <cloud>` | `mon deploy`: `aws` or `hetzner`; default: the database's cloud (its RDS, Aurora, EC2 or Supabase region), else AWS. `dblab deploy`: `hetzner`, `aws` or `gcp` (default `hetzner`) |
301
+ | `--location <location>` | a location of that cloud: an AWS region, or a Hetzner location (`fsn1`, `nbg1`, `hel1`, `ash`, `hil`). `mon deploy` default: the database's region, else us-east-1 on AWS; a location of another cloud is refused before anything is created |
260
302
  | `--vcpus <n>` | `mon deploy` only: the database server's vCPUs. Without it there is no AAS (database load) data; at a terminal you are asked. On a database already monitored, it sets missing vCPUs (a set value is changed with `mon instances set`) |
261
303
  | `--wait <minutes>` | how long to wait for the monitoring box; `0` does not wait (default 20) |
262
304
  | `--no-wait` | the same as `--wait 0` |
263
- | `-y, --yes` | never prompt; accept a billed box's price (`mon deploy` does not start the browser sign-in; `mon instances delete` does not ask) |
305
+ | `-y, --yes` | never prompt: deploy where shown and accept the price (`mon deploy` does not start the browser sign-in; `mon instances delete` does not ask). Without it, `mon deploy` asks before creating any box, free or billed, and at a terminal lets you choose another cloud and location; not at a terminal it stops (exit 3) with the place, the price and `choices` |
264
306
  | `--coupon <code>` | promotion code for the organization's monitoring subscription |
265
307
  | `--json` | JSON output, also on a terminal |
266
308
 
@@ -270,6 +312,7 @@ and asks you to re-run with `--yes` to accept it.
270
312
  | Environment | |
271
313
  |---|---|
272
314
  | `PGAI_API_KEY` | the API key, instead of signing in (agents, CI) |
315
+ | `PGAI_MON_NONCE` | RDS over PrivateLink: with `PGAI_MON_PASSWORD`, the printed SQL (`"step":"prepare_db"`) ran: deploy goes on with its password and nonce, without a new SQL |
273
316
  | `PGAI_MON_PASSWORD` | the password of `postgres_ai_mon`. For a new role it is set; for an existing role it is checked by logging in, and never changed. Without it a new role gets a generated password. Not needed for another database on a server PostgresAI already monitors for the organization, with `sslmode=require` or `verify-*` in the URL (see below) |
274
317
  | `PGPASSWORD` | the password for a URL without one. With a `postgres_ai_mon` URL it is the password the monitoring box gets, once the server has checked it |
275
318
  | `PGSSLMODE` | the `sslmode` for a URL without one (`sslmode` in the URL wins, as in libpq) |
@@ -339,13 +382,18 @@ do `pgai dblab deploy` and `pgai dblab instances`:
339
382
  | `status` | Meaning | Exit code |
340
383
  |---|---|---|
341
384
  | `ready` | monitoring works; `health_url` (when the project is found) and `dashboard_url` are set | 0 |
342
- | `in_progress` | the monitoring box is being set up (also when `--wait` ran out); follow it with `pgai mon instances watch <id>` | 0 |
385
+ | `in_progress` | the monitoring box is being set up (`pgai mon deploy --no-wait`; also when the `--wait` of `pgai mon instances watch` or `pgai dblab deploy` ran out); follow it with the `next` command (`pgai mon instances watch <id>`; for DBLab, `pgai dblab instances watch <id>`) | 0 |
343
386
  | `deleting` | a delete is in progress, also the result of `pgai mon instances delete` | 0 |
344
387
  | `deleted` | it is gone | 0 |
345
388
  | `inactive` | deactivated (billing) | 1 |
346
- | `action_required` | do what `next` says, then re-run | 3 |
389
+ | `action_required` | do what `next` says, then re-run; also when the `--wait` of `pgai mon deploy` ran out with the box still starting (`next`: "Still in progress ... resume with pgai mon instances watch <id>") | 3 |
347
390
  | `failed` | `next` has the reason | 1 |
348
391
 
392
+ When PostgresAI is slow to answer the box request for a database URL, `pgai mon deploy` says it is still
393
+ creating the monitoring box and looks for the box for up to 5 minutes (30 seconds from the MCP tool)
394
+ before `--wait` starts, also with `--no-wait`; then it follows the box as usual. If no box appears, it
395
+ stops with `action_required` and says to check `pgai mon instances list` before running deploy again.
396
+
349
397
  `pgai init` exits with 130 when cancelled at a prompt (Ctrl-C, Ctrl-D).
350
398
 
351
399
  `--self-hosted` starts `mon local-install` with the monitoring URL, the API key and the ClickHouse
@@ -421,16 +469,29 @@ required tool argument instead of falling back to a stored default.
421
469
  ### DBLab in PostgresAI cloud: `pgai dblab deploy`
422
470
 
423
471
  ```bash
424
- PGPASSWORD=<password> pgai dblab deploy 'postgresql://<user>@<host>:5432/<db>' --ssh-key <key name or id>
472
+ PGPASSWORD=<password> pgai dblab deploy 'postgresql://<user>@<host>:5432/<db>'
425
473
  ```
426
474
 
427
475
  The same options as `pgai mon deploy` (`--db-url`, `--name`, `--location`, `--wait`, `--no-wait`, `--yes`, `--json`),
428
- plus `--size S|M|L` (default `S`), `--disk <GiB>` (default 50), `--ssh-key` (required: one of the organization's
429
- SSH keys, for reaching the server and its clones) and `--provider generic|rds|supabase` (default: detected; `self-managed`
430
- remains an alias of `generic`). `--name` defaults to the database's name; `--wait` to 120 minutes.
476
+ plus `--size S|M|L` (default `S`), `--disk <GiB>` (default 50), `--cloud hetzner|aws|gcp` and
477
+ `--provider generic|rds|supabase` (default: detected; `self-managed` remains an alias of `generic`). `--name` defaults to
478
+ the database's name; `--wait` to 120 minutes.
431
479
  It shows the estimated monthly cost before asking; with no payment method it stops with the billing page (exit 3).
432
480
  `pgai dblab instances list | watch (status) | delete` take an id or a name.
433
481
 
482
+ The server gets no SSH keys of yours: clones are reached through an SSH tunnel as `dblab-tunnel`, a user that can only
483
+ forward to the clone ports. Allow your key after the deploy (organization admins; it applies within a minute):
484
+
485
+ ```bash
486
+ pgai dblab access add <id or name> --public-key-file ~/.ssh/id_ed25519.pub # or --ssh-key <org key name or id>
487
+ pgai dblab access list <id or name> # the allowed keys, and the tunnel command
488
+ pgai dblab access remove <id or name> --ssh-key <org key name or id>
489
+ ssh -N -L 6000:127.0.0.1:6000 dblab-tunnel@<server ip> # then: psql "host=127.0.0.1 port=6000 ..."
490
+ ```
491
+
492
+ `--ssh-key` on `pgai dblab deploy` puts organization keys on the server itself, only in an environment that allows
493
+ that.
494
+
434
495
  ### Start monitoring
435
496
 
436
497
  Start monitoring with demo database:
@@ -577,7 +638,7 @@ Tools exposed:
577
638
  - `update_issue_comment`: update an existing comment (args: `{ comment_id, org_id, content?, attachments?, debug? }`).
578
639
  - `upload_file`: upload a local file and return the storage URL plus a ready-to-paste markdown link (args: `{ path, org_id, debug? }`).
579
640
  - `download_file`: download a file from storage (args: `{ url, org_id, output_path?, debug? }`).
580
- - `deploy_monitoring`: `pgai mon deploy` as a tool (args: `{ database_url, org_id, provider?, clickhouse_key?, yes?, coupon?, monitoring_user?, debug? }`). Returns the same JSON (`status`: `ready`, `in_progress`, `deleting`, `deleted`, `inactive`, `action_required` or `failed`; `health_url`; `dashboard_url`; `next`). For a billed box, the tool first returns the price with `action_required`; an explicit call with `yes: true` accepts it. `coupon` supplies a promotion code. It does not wait for the monitoring box: call it again to see the status. `database_url` must carry its password, and only the query parameters `sslmode`, `channel_binding`, `application_name` (and `password`); `PGAI_MON_PASSWORD`, `PGPASSWORD` and `CLICKHOUSE_KEY_ID` + `CLICKHOUSE_KEY_SECRET` of the server process are not used, a TLS failure is not retried in plaintext, and `--self-hosted` is CLI-only. For another database on a server PostgresAI already monitors for the organization, an admin `database_url` with `sslmode=require` (or `verify-*`) is enough: PostgresAI fills in the password of `postgres_ai_mon` it keeps.
641
+ - `deploy_monitoring`: `pgai mon deploy` as a tool (args: `{ database_url, org_id, provider?, clickhouse_key?, yes?, coupon?, monitoring_user?, debug? }`). Returns the same JSON (`status`: `ready`, `in_progress`, `deleting`, `deleted`, `inactive`, `action_required` or `failed`; `health_url`; `dashboard_url`; `next`). For a billed box, the tool first returns the price with `action_required`; an explicit call with `yes: true` accepts it. `coupon` supplies a promotion code. It does not wait for the monitoring box: call it again to see the status. Supabase needs no password (PostgresAI prepares the database through the Supabase Management API): the first time, the result is `action_required` with a consent link in `next`; the user opens and approves it, then the tool is called again. Elsewhere `database_url` must carry its password, and only the query parameters `sslmode`, `channel_binding`, `application_name` (and `password`); `PGAI_MON_PASSWORD`, `PGPASSWORD` and `CLICKHOUSE_KEY_ID` + `CLICKHOUSE_KEY_SECRET` of the server process are not used, a TLS failure is not retried in plaintext, and `--self-hosted` is CLI-only. For another database on a server PostgresAI already monitors for the organization, an admin `database_url` with `sslmode=require` (or `verify-*`) is enough: PostgresAI fills in the password of `postgres_ai_mon` it keeps.
581
642
 
582
643
  #### `attachments` parameter (issue/comment tools)
583
644
 
@@ -726,6 +787,17 @@ pgai orgs create --name "My team" --alias my-team --json
726
787
  - You become the owner and an Admin member. No organization selection is sent, and your saved configuration does not change. Use the returned alias with `--org` afterwards.
727
788
  - Requires the Platform change https://gitlab.com/postgres-ai/platform-all/-/merge_requests/1157 to be deployed.
728
789
 
790
+ ### Check for a payment method on file
791
+
792
+ ```bash
793
+ pgai billing status --org my-team # Payment method on file for my-team.
794
+ pgai billing status --org my-team --json # {"org", "has_payment_method", "billing_url"}
795
+ ```
796
+
797
+ - Exit 0: a payment method is on file. Exit 3: none; add one at `billing_url`. Exit 1: an error, or it could not be checked just now.
798
+ - It is the check `pgai mon deploy` and `pgai dblab deploy` make before they create anything. Any member of the organization can run it.
799
+ - Read-only and safe to repeat, so a script or an agent can wait for the card while you add it. It checks no promo code: a coupon is checked only by the deploy.
800
+
729
801
  ## Configuration
730
802
 
731
803
  The CLI stores configuration in `~/.config/postgresai/config.json` including:
@@ -892,16 +964,16 @@ provider-specific behavior.
892
964
  | Schema-valid JSON reports | Local (CLI) | Yes |
893
965
  | Severity summaries (`summary`: `status` + `message`) | Local (CLI) | Yes |
894
966
  | Local conclusions/recommendations for the checks that implement them (e.g. F003, H001) | Local (CLI) | Yes |
895
- | Rich markdown analysis and prose recommendations | **Server-side** (PostgresAI API, via `--markdown`) | No — requires a network call and, for full detail, an API key |
967
+ | Issues with findings and recommendations | **Server-side** (PostgresAI) | No — requires an API key |
896
968
 
897
969
  In short: the local embed gives you structured JSON, per-check severity, and
898
- the conclusions each check implements today. The full narrative analysis is an
899
- API-side capability. Embedders that only need structured findings and severity
970
+ the conclusions each check implements today. Issues with findings are a
971
+ PostgresAI capability on the server side. Embedders that only need structured findings and severity
900
972
  never have to call the API.
901
973
 
902
974
  For every new or updated check, local JSON must include `conclusions` and
903
- `recommendations`. Server-side markdown analysis can enrich those verdicts, but
904
- must never be their only source. The coverage checklist in
975
+ `recommendations`. Server-side issues can enrich those verdicts, but must never
976
+ be their only source. The coverage checklist in
905
977
  [work item #285](https://gitlab.com/postgres-ai/postgresai/-/work_items/285)
906
978
  tracks which existing checks have completed this migration.
907
979