pgkiln 0.48.0 → 0.49.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/SECURITY.md +2 -2
  2. package/db/migrations/116_dump_jobs.sql +41 -0
  3. package/db/migrations/117_rest_handlers.sql +63 -0
  4. package/db/migrations/118_change_log.sql +54 -0
  5. package/docs/apex-feature-parity.md +7 -7
  6. package/docs/guide/01-installation.md +4 -0
  7. package/docs/guide/03-builder.md +179 -5
  8. package/docs/guide/08-security.md +11 -0
  9. package/docs/guide/12-development.md +44 -6
  10. package/docs/guide/13-rest-api.md +59 -1
  11. package/docs/guide/15-extensions.md +1 -1
  12. package/examples/hr/hr_62_rest_module.sql +39 -0
  13. package/package.json +1 -1
  14. package/public/app.css +1 -1
  15. package/public/builder-icons.svg +2 -0
  16. package/public/builder.css +420 -173
  17. package/public/builder.js +48 -0
  18. package/public/sql-monitor.js +73 -0
  19. package/src/app.ts +6 -0
  20. package/src/automations.ts +6 -0
  21. package/src/binds.ts +2 -1
  22. package/src/builder/advisor.ts +24 -2
  23. package/src/builder/autorest.ts +3 -2
  24. package/src/builder/changelog.ts +240 -0
  25. package/src/builder/components.ts +1 -1
  26. package/src/builder/datapump.ts +364 -0
  27. package/src/builder/ddl-tools.ts +13 -4
  28. package/src/builder/find-workspace.ts +2 -2
  29. package/src/builder/plan-view.ts +25 -0
  30. package/src/builder/rest-modules.ts +629 -0
  31. package/src/builder/rest.ts +1 -0
  32. package/src/builder/routes.ts +4 -2
  33. package/src/builder/scheduling.ts +495 -0
  34. package/src/builder/sql-monitor.ts +397 -0
  35. package/src/builder/sql-tools.ts +6 -2
  36. package/src/builder/sql-utilities.ts +7 -39
  37. package/src/builder/sql.ts +6 -0
  38. package/src/builder/table-editor.ts +4 -1
  39. package/src/builder/top-sql.ts +24 -6
  40. package/src/builder/ui.ts +6 -4
  41. package/src/builder/worksheet.ts +5 -21
  42. package/src/changelog.ts +109 -0
  43. package/src/cron-schedule.ts +282 -0
  44. package/src/datapump.ts +546 -0
  45. package/src/dumpscript.ts +407 -0
  46. package/src/pgcron.ts +225 -0
  47. package/src/runtime/rest-handlers.ts +75 -0
  48. package/src/runtime/rest.ts +116 -22
package/SECURITY.md CHANGED
@@ -29,7 +29,7 @@ first version (MVP). Every finding below has a regression test in
29
29
  | Passwords | Password policy (length, letters and digits, no username), expiry and change on first use enforced before a session exists; changing a password needs the current one and ends other sessions. |
30
30
  | Approvals | Task tables are closed to application roles; `meta.tasks` and the `meta.*_task` functions check, for every call, who may see, claim, decide, delegate or cancel (one rights function for both). Initiators can't decide their own requests unless the definition allows it. The completion SQL runs as the application's role in the same transaction, so RLS applies and an error undoes the decision. |
31
31
  | Workflows | Steps run as the application's role (grants and RLS apply), variables are binds (escaped literals), the instance tables are closed to application roles; only initiators and administrators see a workflow, terminate it, and only administrators retry. |
32
- | REST modules | Bearer tokens only (HS256 with `API_JWT_SECRET`, the app claim must match; no cookies, so no CSRF), the account (active, access) or OAuth client (not revoked) checked on every request, handler roles via `meta.has_role()`, SQL as the application's role (RLS) with every value a bind; `:APP_USER`, `:APP_ID` and the other built-in binds come from the server, never from the request. |
32
+ | REST modules | Bearer tokens only (HS256 with `API_JWT_SECRET`, the app claim must match; no cookies, so no CSRF), the account (active, access) or OAuth client (not revoked) checked on every request, handler roles via `meta.has_role()`, SQL as the application's role (RLS) with every value a bind; `:APP_USER`, `:APP_ID` and the other built-in binds come from the server, never from the request. Handlers (SQL, PL/pgSQL and media) refuse with 403 to run as a superuser, a role with BYPASSRLS or a member of pgkiln's owner, so a missing `db_role` on the owner connection can't turn an endpoint into owner access; the Advisor reports this as an error. PL/pgSQL handlers get request values as transaction-local settings (`meta.rest_param`), never as SQL text, inside a dollar-quote tag the body may not contain. |
33
33
  | Progressive Web Apps | Off by default, per app. Kept pages (opt-in) are removed at every sign-in and sign-out; forms kept offline are sent only for the user who filled them in, with a fresh CSRF token, and processed at most once (submission ids); the record a form was opened for is signed with the form. `Permissions-Policy` allows camera and geolocation for the application itself only (`camera=(self), microphone=(), geolocation=(self)`). |
34
34
  | Outgoing web requests (REST data sources) | Off until `PGKILN_REST_ALLOWED_HOSTS` lists hosts. Only http/https without user:password; the resolved address is checked at connect time and is the one connected to (no DNS rebinding); private, loopback, link-local (cloud metadata), CGNAT, multicast and documentation ranges are refused, also as IPv4-mapped, NAT64 and 6to4 addresses, unless the host is in `PGKILN_REST_PRIVATE_HOSTS`; at most 3 redirects, each re-checked, credential headers dropped when a redirect leaves the origin; time and size limits (also after decompression). Parameter values are URL-encoded after a fixed host (`.`/`..` refused), header values are one line, and `Authorization`, `Cookie`, `Host` and transport headers can't be set on a source. Response values reach SQL as one escaped literal through `jsonb_to_recordset` with checked column names, as the app's role. |
35
35
  | Web credential secrets | Encrypted with AES-256-GCM using `PGKILN_SECRET_KEY`, which lives outside the database (a dump alone reveals nothing); write-only in the builder, never exported, logged or shown in errors; the runtime role's column grant leaves out `secret_enc`. Changing the key means entering the secrets again. |
@@ -212,7 +212,7 @@ A second full review, mainly of what end users and API callers can reach. Regres
212
212
  2. `alter role pgkiln_runtime password '...'` and set `RUNTIME_DATABASE_URL`.
213
213
  3. Run behind HTTPS with `COOKIE_SECURE=true`, and set `TRUST_PROXY=true` behind one reverse proxy (a number for several, or their addresses; client IPs are needed for throttling).
214
214
  4. Keep `debug` off for production apps (see the Settings → Security checklist in the builder).
215
- 5. Give each app its own `db_role` with the minimum grants; add RLS where rows are per user or per team.
215
+ 5. Give each app its own `db_role` with the minimum grants; add RLS where rows are per user or per team. REST modules and AutoREST refuse to run without one on a privileged connection (403); run the Advisor after upgrading.
216
216
  6. Restrict network access to `/builder` (reverse proxy or firewall) if developers are a small group.
217
217
  7. With a REST API: set a long random `API_JWT_SECRET` (the same in PostgREST), change the `pgkiln_authenticator` password, configure `db-pre-request = meta.api_check`, expose only the `api` schema, and run PostgREST behind HTTPS.
218
218
  8. With REST data sources: list only the hosts you need in `PGKILN_REST_ALLOWED_HOSTS` (avoid `*`), set a long random `PGKILN_SECRET_KEY` (32+ characters) kept outside the database and its backups, and use `PGKILN_REST_PRIVATE_HOSTS` only for internal services you mean to reach.
@@ -0,0 +1,41 @@
1
+ -- ---------------------------------------------------------------------
2
+ -- 116: Schema Export / Import jobs (sprint 103, Database Actions DA-16:
3
+ -- Data Pump). Administration › Schema Export / Import runs pg_dump for a
4
+ -- schema and pg_restore into a new or existing schema as background jobs
5
+ -- (src/datapump.ts). A job's file lives in the dump directory
6
+ -- (PGKILN_DUMP_DIR) until it expires (PGKILN_DUMP_KEEP_DAYS); the row
7
+ -- keeps the job's options, size, timing and log.
8
+ --
9
+ -- The builder reads and writes it as the owner, for administrators only;
10
+ -- nobody else has grants. Installation data, not exported.
11
+ -- ---------------------------------------------------------------------
12
+ create table meta.dump_job (
13
+ id bigint generated always as identity primary key,
14
+ kind text not null check (kind in ('export', 'import')),
15
+ -- uploaded: a file waiting for its import options
16
+ status text not null check (status in ('uploaded', 'running', 'succeeded', 'failed', 'cancelled')),
17
+ username text not null,
18
+ -- export: the schema dumped; import: the schema in the dump
19
+ schema_name text check (length(schema_name) between 1 and 63),
20
+ -- import: the schema the objects went to
21
+ target_schema text check (length(target_schema) between 1 and 63),
22
+ options jsonb not null default '{}',
23
+ -- the file in the dump directory (a plain name, no path); null once removed
24
+ file_name text check (file_name ~ '^[A-Za-z0-9_.-]{1,120}$'),
25
+ -- an upload's own file name, as shown
26
+ original_name text check (length(original_name) <= 255),
27
+ -- an import of an export's file
28
+ source_job bigint references meta.dump_job on delete set null,
29
+ bytes bigint,
30
+ log text not null default '',
31
+ error text,
32
+ created_at timestamptz not null default now(),
33
+ started_at timestamptz,
34
+ finished_at timestamptz,
35
+ -- a running job's server writes it every few seconds; a job without it for minutes was interrupted
36
+ heartbeat_at timestamptz,
37
+ expires_at timestamptz
38
+ );
39
+ create index dump_job_created on meta.dump_job (created_at desc);
40
+ revoke all on meta.dump_job from public;
41
+ comment on table meta.dump_job is 'Schema Export / Import jobs (116): pg_dump and pg_restore runs with their options, files, sizes and logs; installation data, not exported';
@@ -0,0 +1,63 @@
1
+ -- ---------------------------------------------------------------------
2
+ -- 117: RESTful Services modules, templates and handlers (sprint 104,
3
+ -- Database Actions DA-11 part 2; ORDS: REST › Modules). SQL Workshop ›
4
+ -- RESTful Services edits an application's REST modules (024) as ORDS does:
5
+ -- a module (its base path), its templates (URIs with :parameters) and their
6
+ -- handlers (method and source type: collection query, single row, SQL,
7
+ -- PL/pgSQL, media).
8
+ --
9
+ -- The handlers stay in meta.rest_module.handlers (jsonb), so modules travel
10
+ -- with the application's export as before (export_app takes the whole row:
11
+ -- the new columns go along without redefining it). New here:
12
+ --
13
+ -- roles the module's privilege: a caller needs one of these
14
+ -- application roles (meta.has_role) for every handler of the
15
+ -- module, on top of a handler's own roles
16
+ -- templates [{"uri": "emp/:id", "description": "…"}]: a template can
17
+ -- exist before its first handler, and its description goes
18
+ -- into the OpenAPI document
19
+ --
20
+ -- PL/pgSQL handlers run their block as an anonymous DO block as the
21
+ -- application's database role. A DO block takes no parameters, so the
22
+ -- request's binds are set as transaction-local settings (query parameters:
23
+ -- never part of the SQL text) and :NAME in the block reads them through
24
+ -- meta.rest_param('NAME'); meta.rest_reply(body, status) sets the response.
25
+ -- ---------------------------------------------------------------------
26
+ alter table meta.rest_module
27
+ add column roles text[] not null default '{}',
28
+ add column templates jsonb not null default '[]' check (jsonb_typeof(templates) = 'array');
29
+
30
+ -- an export from before 117 has no "roles" or "templates": jsonb_populate_record gives NULL, the default applies
31
+ create function meta.rest_module_defaults() returns trigger
32
+ language plpgsql set search_path = pg_catalog as $$
33
+ begin
34
+ new.roles := coalesce(new.roles, '{}');
35
+ new.templates := coalesce(new.templates, '[]');
36
+ return new;
37
+ end $$;
38
+ create trigger rest_module_defaults before insert or update on meta.rest_module
39
+ for each row execute function meta.rest_module_defaults();
40
+
41
+ -- A bind of the REST request a PL/pgSQL handler runs for (NULL when the request has none).
42
+ create function meta.rest_param(p_name text) returns text
43
+ language sql stable set search_path = pg_catalog as $$
44
+ select nullif(current_setting('pgkiln_bind.' || lower(p_name), true), '')
45
+ $$;
46
+ comment on function meta.rest_param(text) is 'REST handlers (117): a bind of the request (path, query or body parameter; BODY is the JSON body); :NAME in a PL/pgSQL handler reads it';
47
+
48
+ -- The response of a PL/pgSQL REST handler: a JSON body and an optional HTTP status (200-599).
49
+ create function meta.rest_reply(p_body jsonb, p_status int default null) returns void
50
+ language plpgsql set search_path = pg_catalog as $$
51
+ begin
52
+ if p_status is not null and (p_status < 200 or p_status > 599) then
53
+ raise exception 'meta.rest_reply: the status is 200 to 599, not %', p_status;
54
+ end if;
55
+ perform set_config('pgkiln_rest.body', coalesce(p_body::text, ''), true);
56
+ perform set_config('pgkiln_rest.status', coalesce(p_status::text, ''), true);
57
+ end $$;
58
+ comment on function meta.rest_reply(jsonb, int) is 'REST handlers (117): the JSON response of a PL/pgSQL handler and its status';
59
+
60
+ grant execute on function meta.rest_param(text), meta.rest_reply(jsonb, int) to public;
61
+
62
+ comment on column meta.rest_module.roles is 'The module''s privilege (117): the caller needs one of these application roles for every handler';
63
+ comment on column meta.rest_module.templates is 'Templates (117): [{"uri": "emp/:id", "description": "…"}]; handlers name their template in "path"';
@@ -0,0 +1,54 @@
1
+ -- ---------------------------------------------------------------------
2
+ -- 118: the change log (sprint 106, Database Actions DA-17; Liquibase /
3
+ -- Flyway). The DDL a developer runs from the SQL Worksheet, Table
4
+ -- Properties and Schema Comparison is recorded as a change set: who, when,
5
+ -- the SQL, the schema it changes, where it came from, and a number per
6
+ -- workspace and schema (seq) that stays when other change sets are deleted,
7
+ -- so exported Flyway versions (V0001__add_bonus.sql) never change
8
+ -- (meta.change_set_counter).
9
+ -- SQL Workshop › Change Log lists them per schema and writes them as
10
+ -- migration files; exported_at says when a change set was last written.
11
+ --
12
+ -- The builder reads and writes it as the owner; nobody else has grants.
13
+ -- Installation data like the worksheet's history: not exported with an
14
+ -- application (the schema's DDL is not part of an application).
15
+ -- ---------------------------------------------------------------------
16
+ create table meta.change_set (
17
+ id bigint generated always as identity primary key,
18
+ workspace_id int not null default 1 references meta.workspace on delete cascade,
19
+ schema_name text not null check (length(schema_name) between 1 and 63),
20
+ seq int not null,
21
+ -- the description in file names: V0001__<name>.sql
22
+ name text not null check (name ~ '^[a-z0-9][a-z0-9_]{0,59}$'),
23
+ author text not null,
24
+ source text not null check (source in ('worksheet', 'table', 'compare')),
25
+ sql text not null check (length(sql) between 1 and 1000000),
26
+ object_name text check (length(object_name) <= 63),
27
+ created_at timestamptz not null default now(),
28
+ exported_at timestamptz,
29
+ unique (workspace_id, schema_name, seq)
30
+ );
31
+ create index change_set_schema_idx on meta.change_set (workspace_id, schema_name, seq);
32
+ revoke all on meta.change_set from public;
33
+ comment on table meta.change_set is 'The change log (118): DDL run from the SQL Workshop as change sets per workspace and schema; installation data, not exported';
34
+
35
+ -- the last number given per workspace and schema: a deleted change set's number is never given again
36
+ create table meta.change_set_counter (
37
+ workspace_id int not null references meta.workspace on delete cascade,
38
+ schema_name text not null,
39
+ last_seq int not null,
40
+ primary key (workspace_id, schema_name)
41
+ );
42
+ revoke all on meta.change_set_counter from public;
43
+ comment on table meta.change_set_counter is 'The change log (118): the last change set number per workspace and schema; installation data';
44
+
45
+ create function meta.change_set_seq() returns trigger
46
+ language plpgsql set search_path = pg_catalog as $$
47
+ begin
48
+ insert into meta.change_set_counter as c (workspace_id, schema_name, last_seq) values (new.workspace_id, new.schema_name, 1)
49
+ on conflict (workspace_id, schema_name) do update set last_seq = c.last_seq + 1
50
+ returning last_seq into new.seq;
51
+ return new;
52
+ end $$;
53
+ create trigger change_set_seq before insert on meta.change_set
54
+ for each row execute function meta.change_set_seq();
@@ -7,7 +7,7 @@ open item and open an issue or pull request; see [CONTRIBUTING.md](../CONTRIBUTI
7
7
  Legend: ✅ available · 🟡 partial (see notes) · ❌ not yet · ➖ not planned (a deliberate choice,
8
8
  or better served by the PostgreSQL ecosystem; see the notes and [extensions](guide/15-extensions.md))
9
9
 
10
- Last reviewed: 2026-10-16 (sprint 100: RESTful Services, tables and views published as REST endpoints with paging, filters, keys and OpenAPI; sprint 99: Administration → Database Users for PostgreSQL roles; sprint 98: the Data Modeler with relational diagrams, DDL and a data dictionary; sprint 97: charts of a query in the worksheet, saved charts and their dashboard; sprint 96: the SQL Workshop's Launchpad with pins and tool groups, the builder's search with Ctrl+K; sprint 95: Table Properties: create, edit or copy a table in a form with a DDL preview and Apply in one transaction; sprint 94: an object's menu with Open in a panel, Quick DDL, Add index / trigger / policy and Drop with dependents; sprint 93: worksheets saved as SQL Scripts with tabs and a Files list, Format, Convert case, find and replace, bind and substitution variables; sprint 92: the worksheet's Query Result with sort, value dialog, copy and downloads, plans as a tree with the slowest step and misestimates, SQL History with status, search and run again; sprint 91: the SQL Worksheet like Database Actions' SQL: navigator, Run Statement at the cursor and Run Script with a log per statement, Explain Plan, Autotrace, notices, history, Run as an application role; sprint 90: Comment Out in the page designer, gallery labels in title case; sprint 89: the Object Browser's schema, one row of tabs, blank NULLs, Modify column; sprint 88: home cards with colour bands and round icons, the hero on the page background, a page footer with the version and Back to top; sprint 87: the Create Application wizard reloads the page list for a schema, RLS badges, the alias from the name; sprint 86: the page designer by page number, HTML error pages in the builder; sprint 85: the Form Designer, beyond APEX; sprint 84: the date picker, facet menus; sprint 83: Component Settings and Component Groups; sprint 82: Report Queries; sprint 81: Application Computations and Shortcuts in Shared Components; sprint 80: the master grid's first row selected when the page opens and after a search; grid cells as text with their format mask until edited, with cell navigation by arrow keys, Enter, F2 and Escape; sprint 79: generated dashboards with sums, donuts and lines, leaving out charts that say nothing; applications open in their own light or dark mode until the user chooses, per application; a user menu with Appearance and Sign in when signed out; the developer toolbar's App and Page texts and debug state; the report search's column scope in the magnifier; sprint 78: generated faceted search reports with one column per foreign key, the faceted search's total row count, Show all and Yes/No values, one search field, friendly headings and number formats in generated grids and master-detail pages, friendly names in the Create Application wizard's page list; sprint 77: the Page Designer's layout and tree by display position, button positions, the tree's context menu with Duplicate and Copy to other page, Show Common / Show All, Go to group and the Template Options dialog; display positions, button positions and side-column page templates at runtime; sprint 76: the interactive report's menus and pagination, date items in the date format; sprint 75: the SQL Workshop's Object Browser and tabs; sprint 74: the builder's information architecture: Shared Components grid, Settings tabs, application home search and tasks; sprint 73: the Create Application wizard on one page with page types and features; sprint 72: the runtime chrome compared with APEX 26.2's: the developer toolbar, the Iris look, the navigation bar's Help menu and Install button; sprint 71: generated applications compared with APEX 26.2's: labels, foreign key names in reports, file columns, format masks, the Home page and the dashboard; sprint 70: treemap, sunburst and timeline charts; sprint 69: dragging tasks on a Gantt chart; sprint 68: the Data Reporter's welcome page, Find my workspace; sprint 67: REST source catalogs from OpenAPI documents, a workspace's export and import (two rows added); sprint 66: a blueprint from an existing application; sprint 65: facet charts; sprint 64: approval vacation rules and expiration policies; sprint 63: range, box plot and stock charts; sprint 62: workspace preferences (SQL Workshop and RESTful services per workspace), extension links, Manage backups across applications, SQL Workshop utilities with object reports, About Database and a database monitor; sprint 61: the builder's screens compared with APEX 26.1: the workspace sign-in page, the Administration menu with workspace message, environment banner and utilization, the SQL Workshop home, the App Gallery, application groups and cross application reports (four rows added); sprint 60: isolated workspaces with their own database logins, row-level security on pgkiln's tables and workspace database roles, workspace administrators; sprint 59: the Page Designer's help pane, Quick edit from the developer toolbar; sprint 58: page groups; sprint 57: region templates hero, title bar, buttons container, inline dialog and drawer, wizard with wizard progress, the carousel; sprint 56: Generate DDL and Schema Comparison in the SQL Workshop, `pgkiln ddl` and `pgkiln compare`; sprint 55: an editable login page with a Login region and the Minimal page template, Forgot password through the application's mail service; sprint 54: interactive grid views, the single row view, group by and chart; sprint 53: application settings, user preferences and items kept per user, dynamic translations, `meta.data_export` and data export downloads, display image and geocoded address items; sprint 52: every dynamic action event and action, Ajax callbacks, an error handling function, the session timeout warning; sprint 51: checked imports, application backups, availability and Run only, feedback, type-specific attribute help; sprint 50: lost update detection, warn on unsaved changes, the URL and search regions with search configurations, the developer toolbar with a session state viewer, and e-mail through SMTP with OAuth2, Microsoft Graph, Amazon SES and HTTP APIs; import checks, application availability, property help and e-mail compared again; 2026-10-06: pgkiln 0.31.0, in progress: static application files and the Execute JavaScript dynamic action; region, item, dynamic action and process plug-ins; conditional and dynamic theme styles; XML and Excel in `meta.parse_data`, zips in SQL, `meta.v_boolean`; file items in S3-compatible object storage; map layers by the visible area and as vector tiles; a graphical query builder; tenants for workflows and tasks; application files as YAML and region static IDs; Lucide icons and icon modifiers; PWA push notifications; 0.30.0: 136 icons and an icon picker; an axe-core accessibility audit; cropping pictures; session sharing between applications; twenty-two built-in languages; 0.29.0: workspaces; the Iris base style; drawers; built-in template components including the metric card; Theme Roller dark-mode colours, live preview and template options on items and report columns; report selection across pages; instance settings; fourteen built-in languages; 0.28.0: AI with Claude and OpenAI; 0.27.0: Data Reporter; sample data generator; create application from several sheets, pasted data or existing tables; 0.26.0: working copies with a three-way merge; theme, library and boilerplate applications with subscriptions; 0.25.0: Gantt, pyramid and polar charts; map layers, clustering and server-side spatial filters; REST data sources that write back from forms and grids and synchronise into tables, OAuth2 password and refresh-token grants; debug messages per request and an install/upgrade log; `meta.web_request()` and `meta.parse_data()`; Theme Roller style variants and template options; 0.24.0: automations with several actions, error handling per row and runs from SQL; invoke-API workflow steps; Unload Data in the Data Workshop; create page wizards for cards, calendar, chart, map, faceted search, form and master-detail; create an application from a file; 0.23.0: interactive grid aggregates, frozen/moved/resized columns, layouts and saved grid reports, master-detail, row actions and copy/paste; download processes, execution chains in the background, workflow processes, branches to a function's URL or another app, the Dialog Closed event; number format masks, automatic time zone, runtime messages in German, French and Spanish; SQL scripts, Quick SQL, a query builder, XML loading and data load definitions; custom authentication, lists, page and application locks with developer comments, supporting objects; 0.22.0: keyset paging, report PDFs and REST collections streamed from a cursor, database-account authentication; 0.21.0: HTTP-header authentication behind a reverse proxy; 0.20.0: a popup list of values with server-side search; 0.19.0: row-range pagination, maximum row counts and row limits, lazy loading and caching of regions, streamed CSV/Excel downloads; 0.18.0: calendar week/day/list views with drag and drop, bubble/gauge/funnel/radar charts and drill-down, smart filters, a region display selector and more facet types, computations, conditional branches, build options, menu buttons, REST data sources and web credentials, rich text/Markdown, rating, combobox, date range and QR code items; 0.17.0: an App Builder home, workspace dashboard and utilities like APEX's; 0.16.0: a Page Designer with drag-and-drop layout, a code editor with autocomplete, template components and plug-ins, parallel branches and versions of workflows, the `pgkiln` command line with one file per component; 0.15.0: drop and paste files, heat maps, filtering a report by the map area; 0.14.0: stacked, combo, scatter and pie charts, several files per upload item; 0.13.0: map and tree regions; 0.12.0: REST modules in the builder; 0.11.0: workflows and Progressive Web Apps; 0.10.0: LDAP, SAML, "Keep me signed in", document templates, JSON loading, approvals and the task list; pgkiln installs no application, HR is an example).
10
+ Last reviewed: 2026-10-16 (sprint 106: the change log of DDL change sets with Flyway, Liquibase and SQL downloads; sprint 104: RESTful Services modules, templates and handlers with PL/pgSQL and media handlers, privileges, OpenAPI and Try it; sprint 103: Schema Export / Import with pg_dump and pg_restore; sprint 102: the SQL Monitor with live sessions, Top SQL and a statement's plan; sprint 101: Scheduling of pg_cron jobs with a schedule builder, run now and run history; sprint 100: RESTful Services, tables and views published as REST endpoints with paging, filters, keys and OpenAPI; sprint 99: Administration → Database Users for PostgreSQL roles; sprint 98: the Data Modeler with relational diagrams, DDL and a data dictionary; sprint 97: charts of a query in the worksheet, saved charts and their dashboard; sprint 96: the SQL Workshop's Launchpad with pins and tool groups, the builder's search with Ctrl+K; sprint 95: Table Properties: create, edit or copy a table in a form with a DDL preview and Apply in one transaction; sprint 94: an object's menu with Open in a panel, Quick DDL, Add index / trigger / policy and Drop with dependents; sprint 93: worksheets saved as SQL Scripts with tabs and a Files list, Format, Convert case, find and replace, bind and substitution variables; sprint 92: the worksheet's Query Result with sort, value dialog, copy and downloads, plans as a tree with the slowest step and misestimates, SQL History with status, search and run again; sprint 91: the SQL Worksheet like Database Actions' SQL: navigator, Run Statement at the cursor and Run Script with a log per statement, Explain Plan, Autotrace, notices, history, Run as an application role; sprint 90: Comment Out in the page designer, gallery labels in title case; sprint 89: the Object Browser's schema, one row of tabs, blank NULLs, Modify column; sprint 88: home cards with colour bands and round icons, the hero on the page background, a page footer with the version and Back to top; sprint 87: the Create Application wizard reloads the page list for a schema, RLS badges, the alias from the name; sprint 86: the page designer by page number, HTML error pages in the builder; sprint 85: the Form Designer, beyond APEX; sprint 84: the date picker, facet menus; sprint 83: Component Settings and Component Groups; sprint 82: Report Queries; sprint 81: Application Computations and Shortcuts in Shared Components; sprint 80: the master grid's first row selected when the page opens and after a search; grid cells as text with their format mask until edited, with cell navigation by arrow keys, Enter, F2 and Escape; sprint 79: generated dashboards with sums, donuts and lines, leaving out charts that say nothing; applications open in their own light or dark mode until the user chooses, per application; a user menu with Appearance and Sign in when signed out; the developer toolbar's App and Page texts and debug state; the report search's column scope in the magnifier; sprint 78: generated faceted search reports with one column per foreign key, the faceted search's total row count, Show all and Yes/No values, one search field, friendly headings and number formats in generated grids and master-detail pages, friendly names in the Create Application wizard's page list; sprint 77: the Page Designer's layout and tree by display position, button positions, the tree's context menu with Duplicate and Copy to other page, Show Common / Show All, Go to group and the Template Options dialog; display positions, button positions and side-column page templates at runtime; sprint 76: the interactive report's menus and pagination, date items in the date format; sprint 75: the SQL Workshop's Object Browser and tabs; sprint 74: the builder's information architecture: Shared Components grid, Settings tabs, application home search and tasks; sprint 73: the Create Application wizard on one page with page types and features; sprint 72: the runtime chrome compared with APEX 26.2's: the developer toolbar, the Iris look, the navigation bar's Help menu and Install button; sprint 71: generated applications compared with APEX 26.2's: labels, foreign key names in reports, file columns, format masks, the Home page and the dashboard; sprint 70: treemap, sunburst and timeline charts; sprint 69: dragging tasks on a Gantt chart; sprint 68: the Data Reporter's welcome page, Find my workspace; sprint 67: REST source catalogs from OpenAPI documents, a workspace's export and import (two rows added); sprint 66: a blueprint from an existing application; sprint 65: facet charts; sprint 64: approval vacation rules and expiration policies; sprint 63: range, box plot and stock charts; sprint 62: workspace preferences (SQL Workshop and RESTful services per workspace), extension links, Manage backups across applications, SQL Workshop utilities with object reports, About Database and a database monitor; sprint 61: the builder's screens compared with APEX 26.1: the workspace sign-in page, the Administration menu with workspace message, environment banner and utilization, the SQL Workshop home, the App Gallery, application groups and cross application reports (four rows added); sprint 60: isolated workspaces with their own database logins, row-level security on pgkiln's tables and workspace database roles, workspace administrators; sprint 59: the Page Designer's help pane, Quick edit from the developer toolbar; sprint 58: page groups; sprint 57: region templates hero, title bar, buttons container, inline dialog and drawer, wizard with wizard progress, the carousel; sprint 56: Generate DDL and Schema Comparison in the SQL Workshop, `pgkiln ddl` and `pgkiln compare`; sprint 55: an editable login page with a Login region and the Minimal page template, Forgot password through the application's mail service; sprint 54: interactive grid views, the single row view, group by and chart; sprint 53: application settings, user preferences and items kept per user, dynamic translations, `meta.data_export` and data export downloads, display image and geocoded address items; sprint 52: every dynamic action event and action, Ajax callbacks, an error handling function, the session timeout warning; sprint 51: checked imports, application backups, availability and Run only, feedback, type-specific attribute help; sprint 50: lost update detection, warn on unsaved changes, the URL and search regions with search configurations, the developer toolbar with a session state viewer, and e-mail through SMTP with OAuth2, Microsoft Graph, Amazon SES and HTTP APIs; import checks, application availability, property help and e-mail compared again; 2026-10-06: pgkiln 0.31.0, in progress: static application files and the Execute JavaScript dynamic action; region, item, dynamic action and process plug-ins; conditional and dynamic theme styles; XML and Excel in `meta.parse_data`, zips in SQL, `meta.v_boolean`; file items in S3-compatible object storage; map layers by the visible area and as vector tiles; a graphical query builder; tenants for workflows and tasks; application files as YAML and region static IDs; Lucide icons and icon modifiers; PWA push notifications; 0.30.0: 136 icons and an icon picker; an axe-core accessibility audit; cropping pictures; session sharing between applications; twenty-two built-in languages; 0.29.0: workspaces; the Iris base style; drawers; built-in template components including the metric card; Theme Roller dark-mode colours, live preview and template options on items and report columns; report selection across pages; instance settings; fourteen built-in languages; 0.28.0: AI with Claude and OpenAI; 0.27.0: Data Reporter; sample data generator; create application from several sheets, pasted data or existing tables; 0.26.0: working copies with a three-way merge; theme, library and boilerplate applications with subscriptions; 0.25.0: Gantt, pyramid and polar charts; map layers, clustering and server-side spatial filters; REST data sources that write back from forms and grids and synchronise into tables, OAuth2 password and refresh-token grants; debug messages per request and an install/upgrade log; `meta.web_request()` and `meta.parse_data()`; Theme Roller style variants and template options; 0.24.0: automations with several actions, error handling per row and runs from SQL; invoke-API workflow steps; Unload Data in the Data Workshop; create page wizards for cards, calendar, chart, map, faceted search, form and master-detail; create an application from a file; 0.23.0: interactive grid aggregates, frozen/moved/resized columns, layouts and saved grid reports, master-detail, row actions and copy/paste; download processes, execution chains in the background, workflow processes, branches to a function's URL or another app, the Dialog Closed event; number format masks, automatic time zone, runtime messages in German, French and Spanish; SQL scripts, Quick SQL, a query builder, XML loading and data load definitions; custom authentication, lists, page and application locks with developer comments, supporting objects; 0.22.0: keyset paging, report PDFs and REST collections streamed from a cursor, database-account authentication; 0.21.0: HTTP-header authentication behind a reverse proxy; 0.20.0: a popup list of values with server-side search; 0.19.0: row-range pagination, maximum row counts and row limits, lazy loading and caching of regions, streamed CSV/Excel downloads; 0.18.0: calendar week/day/list views with drag and drop, bubble/gauge/funnel/radar charts and drill-down, smart filters, a region display selector and more facet types, computations, conditional branches, build options, menu buttons, REST data sources and web credentials, rich text/Markdown, rating, combobox, date range and QR code items; 0.17.0: an App Builder home, workspace dashboard and utilities like APEX's; 0.16.0: a Page Designer with drag-and-drop layout, a code editor with autocomplete, template components and plug-ins, parallel branches and versions of workflows, the `pgkiln` command line with one file per component; 0.15.0: drop and paste files, heat maps, filtering a report by the map area; 0.14.0: stacked, combo, scatter and pie charts, several files per upload item; 0.13.0: map and tree regions; 0.12.0: REST modules in the builder; 0.11.0: workflows and Progressive Web Apps; 0.10.0: LDAP, SAML, "Keep me signed in", document templates, JSON loading, approvals and the task list; pgkiln installs no application, HR is an example).
11
11
 
12
12
  ## At a glance
13
13
 
@@ -184,7 +184,7 @@ two, REST source catalogs and the workspace export, with their features.)
184
184
 
185
185
  | APEX | pgkiln | Notes |
186
186
  |---|---|---|
187
- | SQL Workshop: SQL commands, object browser | ✅ | (sprint 61) A **home page** like APEX's: tiles, recently created tables, your recent SQL commands and the workspace's recent scripts, and the session's **default schema** (first in the search path of the SQL Worksheet and Scripts). (sprint 91) **SQL Commands is the SQL Worksheet** like Database Actions' SQL: a navigator, the statement at the cursor or a script with a log per statement, Explain Plan and Autotrace, notices and history (see the Database Actions section below). (sprint 75) The **Object Browser** like APEX's: a schema in the header, a tree by object type (tables, views, materialized views, indexes, sequences, types, functions, procedures, triggers) with a filter and a Create menu, system and extension objects hidden unless asked, and a table in tabs (columns with Add/Rename/Drop column writing the ALTER TABLE to review, data in pages, indexes, constraints, triggers, grants, row level security, dependencies, statistics, DDL); the tools as main tabs and a More menu. (sprint 62) **Utilities** like APEX's: the tools on one page, **Object Reports** (tables without a primary key, unindexed foreign keys, row-level security without policies or not enabled, PUBLIC grants, unused indexes, largest tables, missing comments), **About Database** and a **Database Monitor** of sessions and blocked queries for administrators. **Missing:** a recycle bin (PostgreSQL has none), user interface defaults (pgkiln's wizards read the catalog and comments) |
187
+ | SQL Workshop: SQL commands, object browser | ✅ | (sprint 61) A **home page** like APEX's: tiles, recently created tables, your recent SQL commands and the workspace's recent scripts, and the session's **default schema** (first in the search path of the SQL Worksheet and Scripts). (sprint 91) **SQL Commands is the SQL Worksheet** like Database Actions' SQL: a navigator, the statement at the cursor or a script with a log per statement, Explain Plan and Autotrace, notices and history (see the Database Actions section below). (sprint 75) The **Object Browser** like APEX's: a schema in the header, a tree by object type (tables, views, materialized views, indexes, sequences, types, functions, procedures, triggers) with a filter and a Create menu, system and extension objects hidden unless asked, and a table in tabs (columns with Add/Rename/Drop column writing the ALTER TABLE to review, data in pages, indexes, constraints, triggers, grants, row level security, dependencies, statistics, DDL); the tools as main tabs and a More menu. (sprint 62) **Utilities** like APEX's: the tools on one page, **Object Reports** (tables without a primary key, unindexed foreign keys, row-level security without policies or not enabled, PUBLIC grants, unused indexes, largest tables, missing comments), **About Database** and (sprint 102) the **SQL Monitor** of sessions, blocked queries and Top SQL for administrators. **Missing:** a recycle bin (PostgreSQL has none), user interface defaults (pgkiln's wizards read the catalog and comments) |
188
188
  | RESTful services (ORDS) | ✅ | [PostgREST](https://postgrest.org) next to pgkiln: `api` schema, the same RLS as the UI, per-app API role, tokens in the builder, a pre-request check, and **OAuth clients** (client credentials, like ORDS `oauth.create_client`) so tokens renew themselves |
189
189
  | REST handler editor, REST-enabled SQL | ✅ | **REST modules** in the builder: handlers (method, path with parameters, SQL as collection, item or statements, roles, public) served by pgkiln with bearer tokens and an OpenAPI description; plus PostgREST for schema-wide APIs. **Missing:** REST-enabled SQL (rarely desirable) |
190
190
  | SQL scripts, query builder, Quick SQL | ✅ | **SQL Scripts**: saved, upload/download, a result per statement, stop or continue on errors, optionally one transaction, run history. **Quick SQL** (a subset: tables, child tables, types, the main column directives, `/auditcols`, settings, views). A graphical **query builder**: tables as boxes on a canvas (dragged, placed where left), joins along foreign keys or drawn from column to column (inner or left), column functions with automatic `group by`, conditions, sort, open in SQL Commands; without JavaScript the same as forms ([chapter 3](guide/03-builder.md#sql-workshop)) |
@@ -352,11 +352,11 @@ AI and row level security badges.
352
352
  | Object context menu, table properties dialog, Quick DDL | An object's menu (sprint 94): Open in a panel, Quick DDL to the worksheet or a file, Load data, Add index / trigger / policy, Drop with its dependents; **Table Properties** (sprint 95): columns, keys, foreign keys, checks, indexes, comment, RLS, partitioning, DDL preview, Apply in one transaction, use as template | ✅ |
353
353
  | Format, Convert Case, Find and replace, bind variables | Format (Ctrl+F7), Convert case, find and replace, Go to line, comment lines, `:NAME` and `&NAME.` asked for before a run (sprint 93); no guided tour | ✅ |
354
354
  | Data Modeler (relational diagrams) | SQL Workshop → Data Modeler (sprint 98): diagrams per schema, boxes with columns and keys, foreign key lines with crow's feet, positions saved, zoom, fit, auto layout, mini-map, SVG / PNG export, DDL of the shown objects, a data dictionary | ✅ |
355
- | REST (modules, AutoREST) | SQL Workshop → RESTful Services (sprint 100): a table or view published through an application with GET (paging, filters, search, order), item GET, POST, PUT, DELETE as the app role with its tokens and RLS, OpenAPI; custom modules, templates and handlers are application REST modules (Shared Components), not yet managed from the SQL Workshop | 🟡 |
356
- | Scheduling | pgkiln's own jobs and automations | ❌ |
355
+ | REST (modules, AutoREST) | SQL Workshop → RESTful Services (sprint 100): a table or view published through an application with GET (paging, filters, search, order), item GET, POST, PUT, DELETE as the app role with its tokens and RLS, OpenAPI. (sprint 104) *Modules*: module (base path) › templates (URIs with `:parameters`) › handlers (GET/POST/PUT/PATCH/DELETE; collection query, single row, SQL, PL/pgSQL with `meta.rest_reply`, media), module privilege and handler roles mapped to application roles, documented parameters, OpenAPI per module, a Try it panel; modules export with the application. **Missing:** ORDS's own OAuth client and role administration per module (pgkiln uses the application's OAuth clients and roles), header and OUT parameters, ETags | ✅ |
356
+ | Scheduling | SQL Workshop → Scheduling (sprint 101): pg_cron's jobs with the schedule in words and next runs, create / edit with a cron builder and the next five runs, SQL reviewed then applied in one transaction, disable, run now (pgkiln runs the command as the job's role; not in pg_cron's history), unschedule, run history with filters and purge; without pg_cron an install note and pgkiln's own automations and process jobs. No chains (pg_cron has none) | ✅ |
357
357
  | Database Users | Administration → Database users (sprint 99): roles with sign-in, attributes, expiry, memberships and owned objects; create and edit with the SQL reviewed (password masked, a SCRAM verifier) and applied in one transaction; drop with reassign or drop owned; pgkiln's roles protected. PostgreSQL has no created date or quota | ✅ |
358
- | Real Time SQL Monitor | Database Monitor (sessions) and Top SQL per application role | 🟡 |
358
+ | Real Time SQL Monitor | SQL Workshop → SQL Monitor (sprint 102): live sessions (state, wait event, blockers, running time) refreshed every 5 / 15 / 60 s with Pause, Cancel / Terminate with confirmation, wait events, progress of VACUUM / ANALYZE / CREATE INDEX / CLUSTER / COPY; Top SQL from `pg_stat_statements` with bars, sort by any column, reset; a statement's totals per role, Explain Plan (EXPLAIN only, generic plan for parameters) and Open in worksheet; administrators only. No historical (past) monitoring beyond the statistics | ✅ |
359
359
  | Launchpad, Search Database (Ctrl+K) | The SQL Workshop home as a Launchpad (sprint 96): pinned and recently visited, Development / Administration / Monitoring tabs; Search (Ctrl+K) for tools, applications, pages and database objects; no guided walk-through | ✅ |
360
- | Data Pump | Application backups and Unload Data | 🟡 |
361
- | Liquibase | Schema Comparison | 🟡 |
360
+ | Data Pump | Schema Export / Import (sprint 103): `pg_dump` of a schema (custom or plain format, definitions and / or data, include / exclude tables) and `pg_restore` of a one-schema archive into a new or existing schema (create schema as: renamed in one transaction), background jobs with status, size, duration and log, files kept N days; plus application backups and Unload Data. Plain dumps can't be imported here; one schema per job | ✅ |
361
+ | Liquibase | Schema Comparison; (sprint 106) SQL Workshop → **Change Log**: DDL run from the SQL Worksheet, Table Properties and Schema Comparison (*Apply and record*) is recorded as change sets per schema (author, time, SQL, source, a stable number), downloaded as Flyway files (`V0001__add_bonus.sql`), a Liquibase formatted SQL changelog or a plain script, and marked against the latest export of an application. **Missing:** rollback blocks, preconditions and contexts; DDL run outside the SQL Workshop (psql, migrations) is not recorded | ✅ |
362
362
  | JavaScript (MLE) | ➖ Not planned: PostgreSQL has plv8 as an extension | ➖ |
@@ -158,6 +158,10 @@ which is read at startup; real environment variables take precedence.
158
158
  | `DB_ACQUIRE_TIMEOUT_MS` | `30000` | How long a request waits for a free connection before it fails with an error |
159
159
  | `DB_IDLE_IN_TRANSACTION_MS` | `300000` | The database ends a transaction left idle this long (milliseconds), so a hung request frees its connection |
160
160
  | `MAX_UPLOAD_MB` | `10` | Largest file a file item accepts (an item's `max_mb` can only lower it) |
161
+ | `PGKILN_PG_BIN` | (searched) | (sprint 103) The directory of pg_dump and pg_restore for Schema Export / Import (the server's major version or newer) |
162
+ | `PGKILN_DUMP_DIR` | `<temp>/pgkiln-dumps` | Where Schema Export / Import keeps its files (the Docker image: `/app/dumps`, a volume) |
163
+ | `PGKILN_DUMP_KEEP_DAYS` | `7` | Days an export or upload is kept |
164
+ | `PGKILN_DUMP_MAX_MB` | `2048` | Largest dump an administrator uploads |
161
165
  | `DATA_LOAD_MAX_MB` | `50` | Largest file for SQL Workshop → Load Data and Create → From a file |
162
166
  | `DATA_LOAD_MAX_ROWS` | `100000` | Most rows loaded from one file |
163
167
  | `PDF_MAX_ROWS` | `5000` | Most rows in a report PDF (1 to 100,000; read with a cursor in batches) |
@@ -102,7 +102,41 @@ Links), added on the workspace's page by administrators and the workspace's admi
102
102
  `https://` URL, an optional description and an order. While the workspace is current an **Extensions** icon in the
103
103
  rail opens them, each in a new tab (`rel="noopener noreferrer"`).
104
104
 
105
- ### Export and import
105
+ #### Scheduling with pg_cron
106
+
107
+ (sprint 101; Database Actions: Scheduling) **SQL Workshop → Scheduling** (administrators only) manages the jobs of
108
+ [pg_cron](https://github.com/citusdata/pg_cron), the PostgreSQL extension that runs SQL on a schedule inside the
109
+ database server. pgkiln needs pg_cron 1.4 or later, created in pgkiln's database; the page says what is missing when
110
+ it isn't (the package, `shared_preload_libraries = 'pg_cron'`, `cron.database_name`, `create extension pg_cron`).
111
+ Without pg_cron it lists **pgkiln's own scheduled work** read-only: every application's automations (Shared Components
112
+ → Automations, [chapter 6](06-processing.md)) with their schedule, next and last run, and the latest background process
113
+ jobs. With pg_cron that list is behind a toggle under the jobs.
114
+
115
+ - **Jobs** (`cron.job`): name, the command, the schedule with its meaning in words (*At 02:00, every day*), the **next
116
+ run**, the last run's status and time (and how many of the last ten failed), database and the role it runs as;
117
+ a search field. Times are in pg_cron's time zone (`cron.timezone`, GMT unless set). pg_cron shows each role only its
118
+ own jobs: when pgkiln's owner connection is neither a superuser nor `BYPASSRLS`, the page says so.
119
+ - **Create job** / **Edit**: name (unique per role), database, *Run as* (a role that can sign in; only a superuser
120
+ schedules for another role), *Active*, the command (SQL, run outside a transaction block as pg_cron runs it, so
121
+ `vacuum` works) and the **schedule**: five cron fields, `n seconds` (1 to 59) or `@hourly` … `@yearly`, `@reboot`.
122
+ A **schedule builder** (one field each for minute, hour, day of month, month and day of week, `$` for the last day
123
+ of the month) and **common schedules** fill it in; the **preview** shows it in words with the **next five runs**,
124
+ as you type (without JavaScript: the *Use* and *Preview* buttons). pgkiln explains schedules as pg_cron reads them:
125
+ lists, ranges, steps, month and day names, 7 for Sunday, and a day matching either the day of month or the day of
126
+ week when both are restricted.
127
+ - **Review SQL** shows the statements (`cron.schedule_in_database`, `cron.alter_job` with only what changed, an
128
+ `update cron.job` for a new name; the command dollar-quoted); **Apply** runs them in one transaction and only if they
129
+ are still what was reviewed. **Disable / Enable** (`cron.alter_job(…, active := …)`) applies at once and shows the
130
+ statement; **Unschedule…** (`cron.unschedule`) after a review. Every change goes to the activity log (event `cron_job`).
131
+ - **Run now**: pg_cron has no run-now, so pgkiln runs the job's stored command once itself: a new connection to the
132
+ job's database with the owner connection's login, `SET ROLE` to the job's role (the owner must be allowed to), no
133
+ transaction block around it, a 10-minute statement timeout. The page shows the result, the duration and notices.
134
+ The run is not in pg_cron's history; the activity log has it (event `cron_run_now`).
135
+ - **Run history** (`cron.job_run_details`, newest first, 200 at most): status, start, duration and message, filtered
136
+ by job, status or a text in the message or command. **Purge…** deletes runs older than 7, 30 or 90 days, or all
137
+ (of every job or one), after showing the `delete` and how many runs it removes: pg_cron keeps every run until then.
138
+
139
+ ## Export and import
106
140
 
107
141
  Workspace utilities → **Export and import** (`/builder/workspace-export`; APEX: Workspace Utilities → Export), for
108
142
  administrators and the workspace's administrators: the current workspace's components as one JSON file (format
@@ -199,6 +233,13 @@ Every builder page has the same frame, modelled on an IDE:
199
233
 
200
234
  On phones the rail gets narrower and the breadcrumb shows only the current page.
201
235
 
236
+ The builder's look comes from design tokens at the top of `public/builder.css` (sprint 105): CSS custom properties
237
+ for colours (a warm neutral palette with the kiln's ember for primary buttons, links, focus rings and selections),
238
+ surfaces and their shadows, corner radii, the type scale and transitions, in a light and a dark set. The rail, the
239
+ toolbar and the status bar stay dark in both themes. Transitions are switched off when the operating system asks
240
+ for reduced motion. The builder uses the system's fonts (Inter when it is installed); it loads nothing from
241
+ other sites.
242
+
202
243
  (sprint 86) An address under `/builder` that leads nowhere (a deleted application or page, a typo, an
203
244
  application of a workspace you are not a member of) shows a **Page not found** page in the builder's
204
245
  theme with a link to the App Builder home; a server error shows **Something went wrong** (the error
@@ -1454,7 +1495,8 @@ the SQL Workshop's connection may use. Without JavaScript the header's field ope
1454
1495
  replaces views and functions, drops and creates constraints, indexes, triggers, policies and
1455
1496
  grants that differ, and lists for review what it doesn't change: drops of what only the target
1456
1497
  has (commented out), columns whose identity or generation differs, and types, domains and
1457
- sequences that differ. The same from the command line (`pgkiln ddl`, `pgkiln compare`,
1498
+ sequences that differ. (sprint 106) *Apply and record* runs the script on the target in one transaction and records
1499
+ it in the Change Log (below). The same from the command line (`pgkiln ddl`, `pgkiln compare`,
1458
1500
  [chapter 18](18-cli.md)) and for AI agents (`generate_ddl`, `compare_schemas`).
1459
1501
  - **RESTful Services** (`/builder/sql/rest`, sprint 100; Database Actions: REST › AutoREST): publish a table or
1460
1502
  view as REST endpoints. Choose the object (of a schema the SQL Workshop may use), the **application** it is
@@ -1465,6 +1507,25 @@ the SQL Workshop's connection may use. Without JavaScript the header's field ope
1465
1507
  accounts' roles. The object's page checks the role's grants per method, row level security and (for a view)
1466
1508
  `security_invoker`, lists the endpoints with their URLs, and shows cURL and `fetch` examples and the OpenAPI link.
1467
1509
  Details in [chapter 13](13-rest-api.md#autorest-tables-and-views-from-the-sql-workshop).
1510
+ (sprint 104; Database Actions: REST › Modules) The **Modules** tab edits the REST modules of the workspace's
1511
+ applications as a tree: a module (its base path and its privilege: roles every handler needs), its templates (URIs
1512
+ with `:parameters`) and their handlers (method, source type: collection query, single row, SQL, PL/pgSQL or media,
1513
+ who may call it, roles, documented parameters). Each handler has a **Try it** panel: the request goes to the real
1514
+ endpoint without a token or as an account of the application, and the response is shown in the page. Details in
1515
+ [chapter 13](13-rest-api.md#modules-templates-and-handlers-in-the-sql-workshop).
1516
+ - **Change Log** (`/builder/sql/changes`, sprint 106; Database Actions: Liquibase): DDL that succeeds in the **SQL
1517
+ Worksheet**, **Table Properties** (*Apply*) and **Schema Comparison** (*Apply and record*: the change script run on
1518
+ the target in one transaction) is recorded as a *change set*: the schema it changes, its SQL, who ran it, when,
1519
+ where from and a number per schema that is never given again (deleting a change set keeps the others' numbers).
1520
+ Statements without a schema-qualified name count for the worksheet's schema. Only schema DDL is kept (CREATE,
1521
+ ALTER, DROP, COMMENT, GRANT and REVOKE on objects): no queries or data changes, and nothing about roles, databases
1522
+ or servers, so passwords never end up in the log. The page lists a schema's change sets with their SQL; rename one
1523
+ (its file name) or delete it from the log (the database is not changed). It marks which change sets were made
1524
+ before the latest **Export** of an application of the workspace (*in it*) and which after (*still to ship*).
1525
+ *Download migration* writes all of them, those not yet written to a file, or those after the latest export, as
1526
+ **Flyway** versioned migrations (`V0001__add_bonus.sql`; several in a zip), a **Liquibase** formatted SQL changelog
1527
+ (`--liquibase formatted sql`, `--changeset author:0001-add_bonus`) or one plain SQL script; each change set also
1528
+ has its own Flyway file. Change sets belong to the workspace and are not part of an application export.
1468
1529
  - **Utilities** (`/builder/sql/utilities`; APEX: SQL Workshop → Utilities): tiles for Load and Unload Data, Sample
1469
1530
  Data, the Query Builder, DDL and Compare, and three pages of their own. Each runs in a read-only transaction with a
1470
1531
  15-second statement timeout, on the SQL Workshop's connection (an isolated workspace's developers see only the
@@ -1475,14 +1536,51 @@ the SQL Workshop's connection may use. Without JavaScript the header's field ope
1475
1536
  to the Object Browser. pgkiln's own `meta` schema and the system's are left out.
1476
1537
  - **About Database**: the PostgreSQL version, database size, uptime, a few settings (encoding, time zone,
1477
1538
  connections, memory, row security), the installed extensions and the schemas with their size.
1478
- - **Database Monitor** (administrators only): the client sessions of this database from `pg_stat_activity`, active
1479
- first, with their state, wait event, running time, the sessions that block them and the query (as far as the
1480
- owner may read other roles' statistics). Read only: it cancels nothing.
1539
+ - **SQL Monitor** (administrators only): see [SQL Monitor](#sql-monitor) below (it replaced the Database Monitor;
1540
+ the old address `/builder/sql/utilities/monitor` redirects).
1481
1541
  - **Database Users** (administrators only; Administration menu → *Database users*, `/builder/db-users`; sprint 99,
1482
1542
  Database Actions: Database Users): see [Database users](#database-users).
1543
+ - **Schema Export / Import** (administrators only; `/builder/sql/data-pump`; sprint 103, Database Actions: Data
1544
+ Pump): see [Schema export and import](#schema-export-and-import).
1545
+ - **Scheduling** (administrators only; `/builder/sql/scheduling`, on the Launchpad under *Administration*, the
1546
+ Utilities page and the search; sprint 101, Database Actions: Scheduling): [pg_cron](#scheduling-with-pg_cron) jobs.
1483
1547
 
1484
1548
  Because the SQL Workshop runs as the owner, restrict who gets a developer account.
1485
1549
 
1550
+ ### SQL Monitor
1551
+
1552
+ (sprint 102; Database Actions: Real Time SQL Monitor and Performance Hub) **SQL Workshop → SQL Monitor**
1553
+ (`/builder/sql/monitor`, administrators only; on the Launchpad under *Monitoring*, in the Utilities and in the search,
1554
+ Ctrl+K) shows what the database is doing, from PostgreSQL's statistics views:
1555
+
1556
+ - **Sessions**: tiles with the sessions (of `max_connections`), the active ones, those idle in a transaction, the
1557
+ blocked ones and the longest running statement; then the client sessions of this database (`pg_stat_activity`;
1558
+ idle sessions on request, *Show idle sessions*) with user, application and client address, state (idle in a
1559
+ transaction with its time), wait event, running time, the sessions that **block** it (the row is highlighted) and
1560
+ the query. A query with a `query_id` links to its statement page. **Cancel** (`pg_cancel_backend`: stop the running
1561
+ statement) and **Terminate** (`pg_terminate_backend`: roll back and disconnect) ask for a confirmation first; only
1562
+ client sessions of this database can be signalled, and the owner role needs the right to (a superuser, the
1563
+ session's own role, or `pg_signal_backend` for non-superusers' sessions).
1564
+ - **Wait events**: what the active sessions of the server wait for now, counted per type and event (*CPU: running*
1565
+ when they don't wait).
1566
+ - **In progress** (only when something runs): VACUUM, ANALYZE, CREATE INDEX, CLUSTER and COPY from the
1567
+ `pg_stat_progress_*` views, with the phase and a progress bar.
1568
+ - The tiles and tables **refresh in place** every 5, 15 or 60 seconds, with a **Pause** button; the choice is
1569
+ remembered in the browser. Without JavaScript, *Refresh* reloads the page.
1570
+ - **Top SQL**: the statements of this database from `pg_stat_statements`, every role: calls, total and mean time
1571
+ (with bars), rows, shared blocks hit and read; sort by any column (click the heading again to reverse), search the
1572
+ text, **Reset statistics** (the whole server, after a confirmation). Literals show as `$1`, `$2`….
1573
+ - **A statement**: its text, its totals (calls, total and mean time, rows, blocks, temporary blocks) and a row per
1574
+ role with standard deviation, minimum and maximum; **Explain** shows its plan as a tree like the worksheet's. It is
1575
+ `EXPLAIN` only, never `EXPLAIN ANALYZE` (the statement is someone else's): a text with parameters gets a generic
1576
+ plan (`EXPLAIN (GENERIC_PLAN)`, PostgreSQL 16 or later), in a read-only transaction that is rolled back, with
1577
+ a 5-second timeout and one statement only. Choose the schema its unqualified names live in. **Open in worksheet**
1578
+ puts the text in the SQL Worksheet.
1579
+
1580
+ Top SQL needs the `pg_stat_statements` extension, which the server must load (`shared_preload_libraries`); pgkiln
1581
+ creates the extension when it can (migration 016). Without it the page says how to turn it on. The per-application
1582
+ **Top SQL** on an application's Activity page shows the same statistics for its database role only.
1583
+
1486
1584
  ### Database users
1487
1585
 
1488
1586
  (sprint 99; Database Actions: Database Users) **Administration → Database users** (administrators only; also on the
@@ -1518,6 +1616,82 @@ schema, REST API role) or an isolated workspace can be changed but not dropped.
1518
1616
  `pgkiln`. The owner connection needs `CREATEROLE` (and, on PostgreSQL 16 and later, ADMIN on roles it didn't create)
1519
1617
  for these changes; PostgreSQL's own error says so otherwise.
1520
1618
 
1619
+ ### Schema export and import
1620
+
1621
+ (sprint 103; Database Actions: Data Pump) **SQL Workshop → Schema Export / Import** (administrators only; on the
1622
+ Launchpad under *Administration* and on the Utilities page) dumps a schema to a file with **pg_dump** and loads a
1623
+ file into a schema with **pg_restore**, as background jobs. The list shows every job: export or import, the schema,
1624
+ its status (*running*, *succeeded*, *failed*, *cancelled*, or *uploaded* for a file waiting for its options), the
1625
+ file's size, when it started, how long it took and who ran it. A job's page has its options, pg_dump's or the
1626
+ import's **log**, and **Download**, **Import…**, **Cancel** (while it runs) and **Delete** (the job and its file).
1627
+
1628
+ **Export a schema**: pick a schema (not pgkiln's own `meta`, PostgreSQL's or an extension's), then
1629
+
1630
+ - the **format**: *custom* (compressed, `pg_dump -Fc`; what Import reads) or *plain SQL* (for `psql`; not imported
1631
+ here; owners are left out);
1632
+ - the **content**: definitions and data, definitions only (`--schema-only`) or data only (`--data-only`), with or
1633
+ without **grants**;
1634
+ - the **tables, views and sequences**: the whole schema, only the checked ones (`--table`: the file then has no
1635
+ CREATE SCHEMA and no functions or types), or all but the checked ones (`--exclude-table`).
1636
+
1637
+ pg_dump runs with an argument list (no shell) on the owner connection (`DATABASE_URL`); the password goes in
1638
+ `PGPASSWORD`, never on the command line. The file is kept in the dump directory for `PGKILN_DUMP_KEEP_DAYS` days
1639
+ (default 7); then the scheduler removes it and the job stays in the list.
1640
+
1641
+ **Import**: upload a custom or tar archive of **one schema** (`pg_dump -Fc -n <schema>`, up to `PGKILN_DUMP_MAX_MB`,
1642
+ default 2048 MB; the file is streamed to the dump directory), or pick a kept export. The next page shows what the
1643
+ archive holds (`pg_restore --list`): the schema, the tables, the object types and how many of each, the server and
1644
+ pg_dump versions. Then choose
1645
+
1646
+ - the **schema** to import into: the dump's own name or another one, **new or existing**. Another name renames the
1647
+ dump's schema in the same transaction (*create schema as*);
1648
+ - the **content** (definitions and data, definitions only, data only: data goes into the tables that are there, in
1649
+ foreign key order, parents first);
1650
+ - **Drop the dump's objects in the schema first** (`pg_restore --clean --if-exists`, to replace what an earlier import
1651
+ made) and **Grants**.
1652
+
1653
+ pg_restore writes the script (`pg_restore -f -`, without a connection of its own) and pgkiln runs it on one
1654
+ connection in **one transaction**, so an error rolls everything back:
1655
+
1656
+ ```sql
1657
+ begin;
1658
+ alter schema hr rename to pgkiln_dump_42; -- importing hr as hr_copy while hr exists: the original steps aside
1659
+ alter schema hr_copy rename to hr; -- only when hr_copy exists: the dump's names now mean it
1660
+ -- the dump's statements and COPY data (CREATE SCHEMA left out when the schema exists)
1661
+ alter schema hr rename to hr_copy;
1662
+ alter schema pgkiln_dump_42 rename to hr;
1663
+ commit;
1664
+ ```
1665
+
1666
+ Other sessions keep seeing the original schema until the commit. Views, column defaults, triggers, policies and
1667
+ SQL-standard function bodies refer to objects, so they follow the rename; **names inside PL/pgSQL bodies** and other
1668
+ text (dynamic SQL, a function's `SET search_path`) still say the dump's schema after a rename. Objects are owned by
1669
+ pgkiln's owner role (`--no-owner`); security labels, tablespaces, publications and subscriptions are left out.
1670
+
1671
+ Every statement is checked before it runs: the settings pg_dump writes, CREATE / ALTER / COMMENT / GRANT / DROP of
1672
+ objects **in the dump's schema**, `COPY … FROM stdin` into its tables and `setval` of its sequences. Anything else is
1673
+ refused and rolls the import back: objects in other schemas (`meta` among them), roles and role memberships, `SET
1674
+ ROLE`, `OWNER TO`, `SET SCHEMA`, `DO`, `COPY` from or to files and programs, extensions, and psql meta-commands (other
1675
+ than pg_dump's `\restrict`). An archive with several schemas, extensions or large objects is refused at the preview.
1676
+ A dump is still code: its functions and triggers run as pgkiln's owner role. **Import only dumps you trust.**
1677
+
1678
+ **The client programs.** pg_dump can't dump a server newer than itself, so pgkiln looks for pg_dump and pg_restore of
1679
+ the server's major version or newer, the newest it finds: `PGKILN_PG_BIN` (a directory), else
1680
+ `/usr/lib/postgresql/<n>/bin` (Debian, Ubuntu), `/usr/pgsql-<n>/bin` (RHEL), Homebrew's `postgresql@<n>`, else `PATH`.
1681
+ Without them the page says what to install (`postgresql-client-17` for a PostgreSQL 17 server). The Docker image
1682
+ includes PostgreSQL 17's client programs and keeps the files in `/app/dumps` (a volume in `deploy/compose.yaml`).
1683
+
1684
+ | Variable | Default | |
1685
+ |---|---|---|
1686
+ | `PGKILN_PG_BIN` | (searched) | the directory of pg_dump and pg_restore |
1687
+ | `PGKILN_DUMP_DIR` | `<temp>/pgkiln-dumps` | where the files are kept; one server's directory: with several pgkiln servers, a job's file is on the server that ran it |
1688
+ | `PGKILN_DUMP_KEEP_DAYS` | 7 | days a file is kept |
1689
+ | `PGKILN_DUMP_MAX_MB` | 2048 | the largest upload |
1690
+
1691
+ A job runs in the server that started it: when that server stops, the scheduler marks its running jobs *failed
1692
+ (interrupted)* within two minutes. Exports, uploads, imports and downloads go to the activity log (`schema_export`,
1693
+ `schema_upload`, `schema_import`, `schema_download`).
1694
+
1521
1695
  ## Export and import
1522
1696
 
1523
1697
  **Export** downloads the application as JSON: pages, all components, shared components and
@@ -516,6 +516,17 @@ Workshop → RESTful Services**, AutoREST) run as the app role too, never as the
516
516
  and the row level security below decide what an API caller may read and change. AutoREST refuses a role that owns
517
517
  the published object or bypasses row level security, and an object answers only to a bearer token of its application
518
518
  unless the developer chose *public* for it ([chapter 13](13-rest-api.md#autorest-tables-and-views-from-the-sql-workshop)).
519
+ (sprint 104) REST module handlers refuse to run as a superuser, a role that bypasses row level security or a member
520
+ of pgkiln's owner (403). Their parameters are bound: SQL handlers get escaped literals, PL/pgSQL handlers read
521
+ transaction-local settings through `meta.rest_param()`; a request can't set the server's binds (`APP_USER`, …) or
522
+ real settings. Media handlers are served with a sandboxing Content-Security-Policy and `nosniff`. *Try it* in the
523
+ SQL Workshop calls an endpoint as an account of the application, as a developer issuing a token could
524
+ ([chapter 13](13-rest-api.md#modules-templates-and-handlers-in-the-sql-workshop)).
525
+
526
+ (sprint 106) The SQL Workshop's **Change Log** keeps the schema DDL developers run from the worksheet, Table
527
+ Properties and Schema Comparison, per workspace (only the workspace's developers see it). Statements about roles,
528
+ databases, servers, user mappings and subscriptions are never recorded, so passwords and connection strings stay
529
+ out of it; it holds what the DDL says otherwise (defaults, comments, function bodies).
519
530
 
520
531
  ### Database users and their passwords
521
532