pgkiln 0.48.0 → 0.50.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/SECURITY.md +3 -2
- package/db/migrations/116_dump_jobs.sql +41 -0
- package/db/migrations/117_rest_handlers.sql +63 -0
- package/db/migrations/118_change_log.sql +54 -0
- package/docs/apex-feature-parity.md +9 -9
- package/docs/guide/01-installation.md +7 -0
- package/docs/guide/03-builder.md +227 -10
- package/docs/guide/08-security.md +11 -0
- package/docs/guide/12-development.md +64 -13
- package/docs/guide/13-rest-api.md +59 -1
- package/docs/guide/15-extensions.md +1 -1
- package/examples/hr/hr_62_rest_module.sql +39 -0
- package/package.json +1 -1
- package/public/app.css +1 -1
- package/public/builder-icons.svg +2 -0
- package/public/builder.css +478 -173
- package/public/builder.js +48 -0
- package/public/code-editor.css +32 -79
- package/public/code-editor.js +58 -3
- package/public/data-modeler.css +17 -14
- package/public/sql-monitor.js +73 -0
- package/public/worksheet.css +14 -7
- package/public/worksheet.js +119 -3
- package/src/app.ts +6 -0
- package/src/automations.ts +6 -0
- package/src/binds.ts +2 -1
- package/src/builder/advisor.ts +24 -2
- package/src/builder/autorest.ts +3 -2
- package/src/builder/changelog.ts +240 -0
- package/src/builder/components.ts +1 -1
- package/src/builder/data-modeler.ts +1 -1
- package/src/builder/datapump.ts +364 -0
- package/src/builder/db-users.ts +2 -2
- package/src/builder/ddl-tools.ts +13 -4
- package/src/builder/find-workspace.ts +2 -2
- package/src/builder/object-browser.ts +4 -4
- package/src/builder/plan-view.ts +25 -0
- package/src/builder/querybuilder.ts +1 -1
- package/src/builder/quicksql.ts +1 -1
- package/src/builder/rest-modules.ts +629 -0
- package/src/builder/rest.ts +1 -0
- package/src/builder/routes.ts +4 -2
- package/src/builder/scheduling.ts +495 -0
- package/src/builder/scripts.ts +1 -1
- package/src/builder/sql-charts.ts +1 -1
- package/src/builder/sql-monitor.ts +397 -0
- package/src/builder/sql-tools.ts +6 -2
- package/src/builder/sql-utilities.ts +7 -39
- package/src/builder/sql.ts +6 -0
- package/src/builder/table-editor.ts +6 -3
- package/src/builder/top-sql.ts +24 -6
- package/src/builder/ui.ts +6 -4
- package/src/builder/worksheet.ts +173 -41
- package/src/changelog.ts +109 -0
- package/src/cron-schedule.ts +282 -0
- package/src/datapump.ts +546 -0
- package/src/dumpscript.ts +407 -0
- package/src/pgcron.ts +225 -0
- package/src/runtime/rest-handlers.ts +75 -0
- package/src/runtime/rest.ts +116 -22
- package/src/session.ts +5 -0
- package/src/sqlworksheet.ts +9 -6
- package/src/worksheet-txn.ts +270 -0
package/docs/guide/03-builder.md
CHANGED
|
@@ -102,7 +102,41 @@ Links), added on the workspace's page by administrators and the workspace's admi
|
|
|
102
102
|
`https://` URL, an optional description and an order. While the workspace is current an **Extensions** icon in the
|
|
103
103
|
rail opens them, each in a new tab (`rel="noopener noreferrer"`).
|
|
104
104
|
|
|
105
|
-
|
|
105
|
+
#### Scheduling with pg_cron
|
|
106
|
+
|
|
107
|
+
(sprint 101; Database Actions: Scheduling) **SQL Workshop → Scheduling** (administrators only) manages the jobs of
|
|
108
|
+
[pg_cron](https://github.com/citusdata/pg_cron), the PostgreSQL extension that runs SQL on a schedule inside the
|
|
109
|
+
database server. pgkiln needs pg_cron 1.4 or later, created in pgkiln's database; the page says what is missing when
|
|
110
|
+
it isn't (the package, `shared_preload_libraries = 'pg_cron'`, `cron.database_name`, `create extension pg_cron`).
|
|
111
|
+
Without pg_cron it lists **pgkiln's own scheduled work** read-only: every application's automations (Shared Components
|
|
112
|
+
→ Automations, [chapter 6](06-processing.md)) with their schedule, next and last run, and the latest background process
|
|
113
|
+
jobs. With pg_cron that list is behind a toggle under the jobs.
|
|
114
|
+
|
|
115
|
+
- **Jobs** (`cron.job`): name, the command, the schedule with its meaning in words (*At 02:00, every day*), the **next
|
|
116
|
+
run**, the last run's status and time (and how many of the last ten failed), database and the role it runs as;
|
|
117
|
+
a search field. Times are in pg_cron's time zone (`cron.timezone`, GMT unless set). pg_cron shows each role only its
|
|
118
|
+
own jobs: when pgkiln's owner connection is neither a superuser nor `BYPASSRLS`, the page says so.
|
|
119
|
+
- **Create job** / **Edit**: name (unique per role), database, *Run as* (a role that can sign in; only a superuser
|
|
120
|
+
schedules for another role), *Active*, the command (SQL, run outside a transaction block as pg_cron runs it, so
|
|
121
|
+
`vacuum` works) and the **schedule**: five cron fields, `n seconds` (1 to 59) or `@hourly` … `@yearly`, `@reboot`.
|
|
122
|
+
A **schedule builder** (one field each for minute, hour, day of month, month and day of week, `$` for the last day
|
|
123
|
+
of the month) and **common schedules** fill it in; the **preview** shows it in words with the **next five runs**,
|
|
124
|
+
as you type (without JavaScript: the *Use* and *Preview* buttons). pgkiln explains schedules as pg_cron reads them:
|
|
125
|
+
lists, ranges, steps, month and day names, 7 for Sunday, and a day matching either the day of month or the day of
|
|
126
|
+
week when both are restricted.
|
|
127
|
+
- **Review SQL** shows the statements (`cron.schedule_in_database`, `cron.alter_job` with only what changed, an
|
|
128
|
+
`update cron.job` for a new name; the command dollar-quoted); **Apply** runs them in one transaction and only if they
|
|
129
|
+
are still what was reviewed. **Disable / Enable** (`cron.alter_job(…, active := …)`) applies at once and shows the
|
|
130
|
+
statement; **Unschedule…** (`cron.unschedule`) after a review. Every change goes to the activity log (event `cron_job`).
|
|
131
|
+
- **Run now**: pg_cron has no run-now, so pgkiln runs the job's stored command once itself: a new connection to the
|
|
132
|
+
job's database with the owner connection's login, `SET ROLE` to the job's role (the owner must be allowed to), no
|
|
133
|
+
transaction block around it, a 10-minute statement timeout. The page shows the result, the duration and notices.
|
|
134
|
+
The run is not in pg_cron's history; the activity log has it (event `cron_run_now`).
|
|
135
|
+
- **Run history** (`cron.job_run_details`, newest first, 200 at most): status, start, duration and message, filtered
|
|
136
|
+
by job, status or a text in the message or command. **Purge…** deletes runs older than 7, 30 or 90 days, or all
|
|
137
|
+
(of every job or one), after showing the `delete` and how many runs it removes: pg_cron keeps every run until then.
|
|
138
|
+
|
|
139
|
+
## Export and import
|
|
106
140
|
|
|
107
141
|
Workspace utilities → **Export and import** (`/builder/workspace-export`; APEX: Workspace Utilities → Export), for
|
|
108
142
|
administrators and the workspace's administrators: the current workspace's components as one JSON file (format
|
|
@@ -199,6 +233,15 @@ Every builder page has the same frame, modelled on an IDE:
|
|
|
199
233
|
|
|
200
234
|
On phones the rail gets narrower and the breadcrumb shows only the current page.
|
|
201
235
|
|
|
236
|
+
The builder's look comes from design tokens at the top of `public/builder.css` (sprint 105): CSS custom properties
|
|
237
|
+
for colours (a warm neutral palette with the kiln's ember for primary buttons, links, focus rings and selections),
|
|
238
|
+
surfaces and their shadows, corner radii, the type scale and transitions, in a light and a dark set; the code editor's syntax colours (`--ce-*`: keywords, strings, numbers, comments, binds,
|
|
239
|
+
quoted names, operators, the matching bracket, the current line, the selection and the line numbers) are tokens there
|
|
240
|
+
too (sprint 108). The rail, the
|
|
241
|
+
toolbar and the status bar stay dark in both themes. Transitions are switched off when the operating system asks
|
|
242
|
+
for reduced motion. The builder uses the system's fonts (Inter when it is installed); it loads nothing from
|
|
243
|
+
other sites.
|
|
244
|
+
|
|
202
245
|
(sprint 86) An address under `/builder` that leads nowhere (a deleted application or page, a typo, an
|
|
203
246
|
application of a workspace you are not a member of) shows a **Page not found** page in the builder's
|
|
204
247
|
theme with a link to the App Builder home; a server error shows **Something went wrong** (the error
|
|
@@ -663,7 +706,13 @@ handlers, the SQL Workshop and the import box) opens in a small code editor:
|
|
|
663
706
|
binds, `&ITEM.` substitutions, `$$` bodies), JSON and HTML (tags, attributes, `&ITEM.` and
|
|
664
707
|
`{{tags}}`). The language follows the component type: a `static` region's source is HTML, a
|
|
665
708
|
`regex` validation is plain text, an `execute_sql` dynamic action is PL/pgSQL.
|
|
666
|
-
- **Line numbers**, the line and column of the cursor,
|
|
709
|
+
- **Line numbers**, the line and column of the cursor, the **current line** (a band and its number in bold
|
|
710
|
+
while the field has the focus) and the **matching bracket** marked.
|
|
711
|
+
- **Colours from the builder's theme** (sprint 108): keywords take the kiln's ember, types, functions, strings,
|
|
712
|
+
numbers, binds and quoted names cool hues, comments a muted grey, each at least 4.5:1 (WCAG AA) on the editor's
|
|
713
|
+
background in the light and the dark theme. Read-only SQL (the Object Browser's DDL and source tabs, the Data
|
|
714
|
+
Modeler's DDL, generated SQL in Quick SQL, Query Builder, Scheduling, Database Users and the table editor, the SQL
|
|
715
|
+
Monitor's text, a saved chart's query, the Change Log, a script's results) is highlighted the same way.
|
|
667
716
|
- **Tab / Shift+Tab** indent and outdent (the selected lines, too); **Enter** keeps the
|
|
668
717
|
indentation and indents after `(`, `begin`, `loop`, `then`; brackets and quotes are closed for
|
|
669
718
|
you and typing the closer steps over it. Press **Escape, then Tab** to move on to the next field.
|
|
@@ -1340,11 +1389,41 @@ the SQL Workshop's connection may use. Without JavaScript the header's field ope
|
|
|
1340
1389
|
(**Ctrl + F**, **Ctrl + H**) with a regular expression or exact case, **Go to line** (**Ctrl + G**), and
|
|
1341
1390
|
**Ctrl + /** comments the selected lines out or in. The **?** menu lists every key; **F1** opens this help. The
|
|
1342
1391
|
upload button opens Load Data for the navigator's schema.
|
|
1343
|
-
- Each statement commits on its own. Statements run as the **owner connection** (an isolated workspace's
|
|
1392
|
+
- Each statement commits on its own (**Autocommit**, like Database Actions). Statements run as the **owner connection** (an isolated workspace's
|
|
1344
1393
|
developer login for its developers), on a connection of their own that is closed afterwards, so `SET ROLE` or
|
|
1345
1394
|
`SET search_path` doesn't leak. **Run as** picks an application of your workspace instead: the statements then
|
|
1346
1395
|
run as its database role, with `pgkiln.app_id` set, so its grants and row level security policies apply. Use
|
|
1347
|
-
it to test policies; it is no security boundary, since the SQL could `RESET ROLE`.
|
|
1396
|
+
it to test policies; it is no security boundary, since the SQL could `RESET ROLE`. A `BEGIN` typed in
|
|
1397
|
+
Autocommit lasts for the run only: `begin; …; commit;` in one Run Script works, but a transaction still open at
|
|
1398
|
+
the end of the run is rolled back, with a warning under *Notices*.
|
|
1399
|
+
- (sprint 107) **Transaction mode** (more than Database Actions, which always autocommits): the toolbar's
|
|
1400
|
+
**Commit** menu switches a worksheet tab between *Autocommit* and *Transaction*. In Transaction mode the tab's
|
|
1401
|
+
statements (Run Statement and Run Script) stay in one transaction on a connection of its own until you press
|
|
1402
|
+
**Commit** or **Rollback**: a SELECT in the same tab sees the changes, other sessions (and your other tabs) don't
|
|
1403
|
+
until you commit. A badge shows *Transaction open*, the number of statements and its age; after a reload it shows
|
|
1404
|
+
the server's real state, and leaving the page with an open transaction asks first.
|
|
1405
|
+
- `COMMIT` or `ROLLBACK` typed in the editor end the transaction too (the next run starts a new one); a `BEGIN`
|
|
1406
|
+
typed inside it only warns. Statements PostgreSQL refuses inside a transaction (`VACUUM`, `CREATE INDEX
|
|
1407
|
+
CONCURRENTLY`, `ALTER SYSTEM`, `CREATE DATABASE`, …) are refused before they run, so they can't abort it.
|
|
1408
|
+
- A statement that fails leaves the transaction **aborted** (*Transaction failed*): every later statement fails
|
|
1409
|
+
until **Rollback**; Commit then rolls back as well, and says so.
|
|
1410
|
+
- **Explain Plan** and **Autotrace** run inside the open transaction behind a savepoint: the plan sees its changes,
|
|
1411
|
+
Autotrace's own changes are undone and the transaction stays as it was.
|
|
1412
|
+
- **Run as**: the transaction keeps the role it started with. Changing *Run as* while it is open is refused:
|
|
1413
|
+
commit or roll back first. `SET ROLE` typed in the editor reaches only the roles of the connection's login, as
|
|
1414
|
+
in Autocommit.
|
|
1415
|
+
- DDL reaches the **Change Log** when the transaction commits, never when it is rolled back. SQL History records
|
|
1416
|
+
each statement as it runs.
|
|
1417
|
+
- The transaction is **rolled back** when it is left without a run for `WORKSHEET_TXN_IDLE_SECONDS` (default
|
|
1418
|
+
300, five minutes; the badge and the next run say so), when you close its worksheet tab or switch it to
|
|
1419
|
+
Autocommit (after asking), sign out, switch workspaces, and when the server stops. The database ends the
|
|
1420
|
+
connection itself a minute later (`idle_in_transaction_session_timeout`) should the server not.
|
|
1421
|
+
- Each open transaction holds a database connection outside the pools: a developer has at most
|
|
1422
|
+
`WORKSHEET_TXN_MAX_PER_DEVELOPER` (default 3) and the server `WORKSHEET_TXN_MAX` (default 20); past that a run in
|
|
1423
|
+
Transaction mode is refused. A transaction belongs to the builder session, workspace and worksheet tab that
|
|
1424
|
+
started it: another developer, or another sign-in of yours, never reaches it. It lives in the server's memory,
|
|
1425
|
+
so with several pgkiln servers behind a load balancer the browser must stay on one (sticky sessions).
|
|
1426
|
+
- Without JavaScript the worksheet runs in Autocommit only.
|
|
1348
1427
|
- With script, the panes fill the window. Drag the splitters (or focus one and use the arrow keys) to resize
|
|
1349
1428
|
them, hide the navigator, or use **Fullscreen** to see only the editor (**Escape** leaves). Sizes are
|
|
1350
1429
|
remembered in the browser. Without script, the buttons post the form and the panes stack; Run Statement then
|
|
@@ -1412,8 +1491,9 @@ the SQL Workshop's connection may use. Without JavaScript the header's field ope
|
|
|
1412
1491
|
when the foreign key may be empty. A box's name opens **Table Properties** (a view: the Object
|
|
1413
1492
|
Browser). The toolbar: **Save** (Ctrl+S), **Undo** / **Redo** (Ctrl+Z, Ctrl+Y), **Zoom out** / **Zoom
|
|
1414
1493
|
in** (or Ctrl+wheel), **Fit**, **Auto layout** (referenced tables to the left of the tables that
|
|
1415
|
-
reference them; undoable), a **Mini-map** (click or drag in it to move around
|
|
1416
|
-
exports of the diagram as shown,
|
|
1494
|
+
reference them; undoable), a **Mini-map** (click or drag in it to move around; the button shows when it is on),
|
|
1495
|
+
**SVG** and **PNG** exports of the diagram as shown (in light colours, whichever theme the builder uses; on screen the
|
|
1496
|
+
boxes, lines and mini-map follow the light or dark theme), **Print**, **Generate DDL** (the CREATE statements of the shown
|
|
1417
1497
|
objects with their keys, indexes, triggers, row level security, policies, comments and grants: to
|
|
1418
1498
|
download or open in the SQL Worksheet) and **Data dictionary** (each object's columns with type,
|
|
1419
1499
|
nullability, default, keys and comments, its constraints and what references it; to print, save as
|
|
@@ -1454,7 +1534,8 @@ the SQL Workshop's connection may use. Without JavaScript the header's field ope
|
|
|
1454
1534
|
replaces views and functions, drops and creates constraints, indexes, triggers, policies and
|
|
1455
1535
|
grants that differ, and lists for review what it doesn't change: drops of what only the target
|
|
1456
1536
|
has (commented out), columns whose identity or generation differs, and types, domains and
|
|
1457
|
-
sequences that differ.
|
|
1537
|
+
sequences that differ. (sprint 106) *Apply and record* runs the script on the target in one transaction and records
|
|
1538
|
+
it in the Change Log (below). The same from the command line (`pgkiln ddl`, `pgkiln compare`,
|
|
1458
1539
|
[chapter 18](18-cli.md)) and for AI agents (`generate_ddl`, `compare_schemas`).
|
|
1459
1540
|
- **RESTful Services** (`/builder/sql/rest`, sprint 100; Database Actions: REST › AutoREST): publish a table or
|
|
1460
1541
|
view as REST endpoints. Choose the object (of a schema the SQL Workshop may use), the **application** it is
|
|
@@ -1465,6 +1546,29 @@ the SQL Workshop's connection may use. Without JavaScript the header's field ope
|
|
|
1465
1546
|
accounts' roles. The object's page checks the role's grants per method, row level security and (for a view)
|
|
1466
1547
|
`security_invoker`, lists the endpoints with their URLs, and shows cURL and `fetch` examples and the OpenAPI link.
|
|
1467
1548
|
Details in [chapter 13](13-rest-api.md#autorest-tables-and-views-from-the-sql-workshop).
|
|
1549
|
+
(sprint 104; Database Actions: REST › Modules) The **Modules** tab edits the REST modules of the workspace's
|
|
1550
|
+
applications as a tree: a module (its base path and its privilege: roles every handler needs), its templates (URIs
|
|
1551
|
+
with `:parameters`) and their handlers (method, source type: collection query, single row, SQL, PL/pgSQL or media,
|
|
1552
|
+
who may call it, roles, documented parameters). Each handler has a **Try it** panel: the request goes to the real
|
|
1553
|
+
endpoint without a token or as an account of the application, and the response is shown in the page. Details in
|
|
1554
|
+
[chapter 13](13-rest-api.md#modules-templates-and-handlers-in-the-sql-workshop).
|
|
1555
|
+
- **Change Log** (`/builder/sql/changes`, sprint 106; Database Actions: Liquibase): DDL that succeeds in the **SQL
|
|
1556
|
+
Worksheet**, **Table Properties** (*Apply*) and **Schema Comparison** (*Apply and record*: the change script run on
|
|
1557
|
+
the target in one transaction) is recorded as a *change set*: the schema it changes, its SQL, who ran it, when,
|
|
1558
|
+
where from and a number per schema that is never given again (deleting a change set keeps the others' numbers).
|
|
1559
|
+
Statements without a schema-qualified name count for the worksheet's schema. Only schema DDL is kept (CREATE,
|
|
1560
|
+
ALTER, DROP, COMMENT, GRANT and REVOKE on objects): no queries or data changes, and nothing about roles, databases
|
|
1561
|
+
or servers, so passwords never end up in the log. The page lists a schema's change sets with their SQL; rename one
|
|
1562
|
+
(its file name) or delete it from the log (the database is not changed). It marks which change sets were made
|
|
1563
|
+
before the latest **Export** of an application of the workspace (*in it*) and which after (*still to ship*).
|
|
1564
|
+
*Download migration* writes all of them, those not yet written to a file, or those after the latest export, as
|
|
1565
|
+
**Flyway** versioned migrations (`V0001__add_bonus.sql`; several in a zip), a **Liquibase** formatted SQL changelog
|
|
1566
|
+
(`--liquibase formatted sql`, `--changeset author:0001-add_bonus`) or one plain SQL script; each change set also
|
|
1567
|
+
has its own Flyway file. Change sets belong to the workspace and are not part of an application export.
|
|
1568
|
+
- **JavaScript (MLE)**: not planned. Database Actions has a JavaScript tool for Oracle's Multilingual Engine;
|
|
1569
|
+
PostgreSQL runs JavaScript in the database only through the [plv8](15-extensions.md) extension. Where plv8 is
|
|
1570
|
+
installed, write `create function … language plv8` in the SQL Worksheet like any other function: the navigator,
|
|
1571
|
+
the Object Browser and Quick DDL show it. pgkiln adds no JavaScript editor of its own.
|
|
1468
1572
|
- **Utilities** (`/builder/sql/utilities`; APEX: SQL Workshop → Utilities): tiles for Load and Unload Data, Sample
|
|
1469
1573
|
Data, the Query Builder, DDL and Compare, and three pages of their own. Each runs in a read-only transaction with a
|
|
1470
1574
|
15-second statement timeout, on the SQL Workshop's connection (an isolated workspace's developers see only the
|
|
@@ -1475,14 +1579,51 @@ the SQL Workshop's connection may use. Without JavaScript the header's field ope
|
|
|
1475
1579
|
to the Object Browser. pgkiln's own `meta` schema and the system's are left out.
|
|
1476
1580
|
- **About Database**: the PostgreSQL version, database size, uptime, a few settings (encoding, time zone,
|
|
1477
1581
|
connections, memory, row security), the installed extensions and the schemas with their size.
|
|
1478
|
-
- **
|
|
1479
|
-
|
|
1480
|
-
owner may read other roles' statistics). Read only: it cancels nothing.
|
|
1582
|
+
- **SQL Monitor** (administrators only): see [SQL Monitor](#sql-monitor) below (it replaced the Database Monitor;
|
|
1583
|
+
the old address `/builder/sql/utilities/monitor` redirects).
|
|
1481
1584
|
- **Database Users** (administrators only; Administration menu → *Database users*, `/builder/db-users`; sprint 99,
|
|
1482
1585
|
Database Actions: Database Users): see [Database users](#database-users).
|
|
1586
|
+
- **Schema Export / Import** (administrators only; `/builder/sql/data-pump`; sprint 103, Database Actions: Data
|
|
1587
|
+
Pump): see [Schema export and import](#schema-export-and-import).
|
|
1588
|
+
- **Scheduling** (administrators only; `/builder/sql/scheduling`, on the Launchpad under *Administration*, the
|
|
1589
|
+
Utilities page and the search; sprint 101, Database Actions: Scheduling): [pg_cron](#scheduling-with-pg_cron) jobs.
|
|
1483
1590
|
|
|
1484
1591
|
Because the SQL Workshop runs as the owner, restrict who gets a developer account.
|
|
1485
1592
|
|
|
1593
|
+
### SQL Monitor
|
|
1594
|
+
|
|
1595
|
+
(sprint 102; Database Actions: Real Time SQL Monitor and Performance Hub) **SQL Workshop → SQL Monitor**
|
|
1596
|
+
(`/builder/sql/monitor`, administrators only; on the Launchpad under *Monitoring*, in the Utilities and in the search,
|
|
1597
|
+
Ctrl+K) shows what the database is doing, from PostgreSQL's statistics views:
|
|
1598
|
+
|
|
1599
|
+
- **Sessions**: tiles with the sessions (of `max_connections`), the active ones, those idle in a transaction, the
|
|
1600
|
+
blocked ones and the longest running statement; then the client sessions of this database (`pg_stat_activity`;
|
|
1601
|
+
idle sessions on request, *Show idle sessions*) with user, application and client address, state (idle in a
|
|
1602
|
+
transaction with its time), wait event, running time, the sessions that **block** it (the row is highlighted) and
|
|
1603
|
+
the query. A query with a `query_id` links to its statement page. **Cancel** (`pg_cancel_backend`: stop the running
|
|
1604
|
+
statement) and **Terminate** (`pg_terminate_backend`: roll back and disconnect) ask for a confirmation first; only
|
|
1605
|
+
client sessions of this database can be signalled, and the owner role needs the right to (a superuser, the
|
|
1606
|
+
session's own role, or `pg_signal_backend` for non-superusers' sessions).
|
|
1607
|
+
- **Wait events**: what the active sessions of the server wait for now, counted per type and event (*CPU: running*
|
|
1608
|
+
when they don't wait).
|
|
1609
|
+
- **In progress** (only when something runs): VACUUM, ANALYZE, CREATE INDEX, CLUSTER and COPY from the
|
|
1610
|
+
`pg_stat_progress_*` views, with the phase and a progress bar.
|
|
1611
|
+
- The tiles and tables **refresh in place** every 5, 15 or 60 seconds, with a **Pause** button; the choice is
|
|
1612
|
+
remembered in the browser. Without JavaScript, *Refresh* reloads the page.
|
|
1613
|
+
- **Top SQL**: the statements of this database from `pg_stat_statements`, every role: calls, total and mean time
|
|
1614
|
+
(with bars), rows, shared blocks hit and read; sort by any column (click the heading again to reverse), search the
|
|
1615
|
+
text, **Reset statistics** (the whole server, after a confirmation). Literals show as `$1`, `$2`….
|
|
1616
|
+
- **A statement**: its text, its totals (calls, total and mean time, rows, blocks, temporary blocks) and a row per
|
|
1617
|
+
role with standard deviation, minimum and maximum; **Explain** shows its plan as a tree like the worksheet's. It is
|
|
1618
|
+
`EXPLAIN` only, never `EXPLAIN ANALYZE` (the statement is someone else's): a text with parameters gets a generic
|
|
1619
|
+
plan (`EXPLAIN (GENERIC_PLAN)`, PostgreSQL 16 or later), in a read-only transaction that is rolled back, with
|
|
1620
|
+
a 5-second timeout and one statement only. Choose the schema its unqualified names live in. **Open in worksheet**
|
|
1621
|
+
puts the text in the SQL Worksheet.
|
|
1622
|
+
|
|
1623
|
+
Top SQL needs the `pg_stat_statements` extension, which the server must load (`shared_preload_libraries`); pgkiln
|
|
1624
|
+
creates the extension when it can (migration 016). Without it the page says how to turn it on. The per-application
|
|
1625
|
+
**Top SQL** on an application's Activity page shows the same statistics for its database role only.
|
|
1626
|
+
|
|
1486
1627
|
### Database users
|
|
1487
1628
|
|
|
1488
1629
|
(sprint 99; Database Actions: Database Users) **Administration → Database users** (administrators only; also on the
|
|
@@ -1518,6 +1659,82 @@ schema, REST API role) or an isolated workspace can be changed but not dropped.
|
|
|
1518
1659
|
`pgkiln`. The owner connection needs `CREATEROLE` (and, on PostgreSQL 16 and later, ADMIN on roles it didn't create)
|
|
1519
1660
|
for these changes; PostgreSQL's own error says so otherwise.
|
|
1520
1661
|
|
|
1662
|
+
### Schema export and import
|
|
1663
|
+
|
|
1664
|
+
(sprint 103; Database Actions: Data Pump) **SQL Workshop → Schema Export / Import** (administrators only; on the
|
|
1665
|
+
Launchpad under *Administration* and on the Utilities page) dumps a schema to a file with **pg_dump** and loads a
|
|
1666
|
+
file into a schema with **pg_restore**, as background jobs. The list shows every job: export or import, the schema,
|
|
1667
|
+
its status (*running*, *succeeded*, *failed*, *cancelled*, or *uploaded* for a file waiting for its options), the
|
|
1668
|
+
file's size, when it started, how long it took and who ran it. A job's page has its options, pg_dump's or the
|
|
1669
|
+
import's **log**, and **Download**, **Import…**, **Cancel** (while it runs) and **Delete** (the job and its file).
|
|
1670
|
+
|
|
1671
|
+
**Export a schema**: pick a schema (not pgkiln's own `meta`, PostgreSQL's or an extension's), then
|
|
1672
|
+
|
|
1673
|
+
- the **format**: *custom* (compressed, `pg_dump -Fc`; what Import reads) or *plain SQL* (for `psql`; not imported
|
|
1674
|
+
here; owners are left out);
|
|
1675
|
+
- the **content**: definitions and data, definitions only (`--schema-only`) or data only (`--data-only`), with or
|
|
1676
|
+
without **grants**;
|
|
1677
|
+
- the **tables, views and sequences**: the whole schema, only the checked ones (`--table`: the file then has no
|
|
1678
|
+
CREATE SCHEMA and no functions or types), or all but the checked ones (`--exclude-table`).
|
|
1679
|
+
|
|
1680
|
+
pg_dump runs with an argument list (no shell) on the owner connection (`DATABASE_URL`); the password goes in
|
|
1681
|
+
`PGPASSWORD`, never on the command line. The file is kept in the dump directory for `PGKILN_DUMP_KEEP_DAYS` days
|
|
1682
|
+
(default 7); then the scheduler removes it and the job stays in the list.
|
|
1683
|
+
|
|
1684
|
+
**Import**: upload a custom or tar archive of **one schema** (`pg_dump -Fc -n <schema>`, up to `PGKILN_DUMP_MAX_MB`,
|
|
1685
|
+
default 2048 MB; the file is streamed to the dump directory), or pick a kept export. The next page shows what the
|
|
1686
|
+
archive holds (`pg_restore --list`): the schema, the tables, the object types and how many of each, the server and
|
|
1687
|
+
pg_dump versions. Then choose
|
|
1688
|
+
|
|
1689
|
+
- the **schema** to import into: the dump's own name or another one, **new or existing**. Another name renames the
|
|
1690
|
+
dump's schema in the same transaction (*create schema as*);
|
|
1691
|
+
- the **content** (definitions and data, definitions only, data only: data goes into the tables that are there, in
|
|
1692
|
+
foreign key order, parents first);
|
|
1693
|
+
- **Drop the dump's objects in the schema first** (`pg_restore --clean --if-exists`, to replace what an earlier import
|
|
1694
|
+
made) and **Grants**.
|
|
1695
|
+
|
|
1696
|
+
pg_restore writes the script (`pg_restore -f -`, without a connection of its own) and pgkiln runs it on one
|
|
1697
|
+
connection in **one transaction**, so an error rolls everything back:
|
|
1698
|
+
|
|
1699
|
+
```sql
|
|
1700
|
+
begin;
|
|
1701
|
+
alter schema hr rename to pgkiln_dump_42; -- importing hr as hr_copy while hr exists: the original steps aside
|
|
1702
|
+
alter schema hr_copy rename to hr; -- only when hr_copy exists: the dump's names now mean it
|
|
1703
|
+
-- the dump's statements and COPY data (CREATE SCHEMA left out when the schema exists)
|
|
1704
|
+
alter schema hr rename to hr_copy;
|
|
1705
|
+
alter schema pgkiln_dump_42 rename to hr;
|
|
1706
|
+
commit;
|
|
1707
|
+
```
|
|
1708
|
+
|
|
1709
|
+
Other sessions keep seeing the original schema until the commit. Views, column defaults, triggers, policies and
|
|
1710
|
+
SQL-standard function bodies refer to objects, so they follow the rename; **names inside PL/pgSQL bodies** and other
|
|
1711
|
+
text (dynamic SQL, a function's `SET search_path`) still say the dump's schema after a rename. Objects are owned by
|
|
1712
|
+
pgkiln's owner role (`--no-owner`); security labels, tablespaces, publications and subscriptions are left out.
|
|
1713
|
+
|
|
1714
|
+
Every statement is checked before it runs: the settings pg_dump writes, CREATE / ALTER / COMMENT / GRANT / DROP of
|
|
1715
|
+
objects **in the dump's schema**, `COPY … FROM stdin` into its tables and `setval` of its sequences. Anything else is
|
|
1716
|
+
refused and rolls the import back: objects in other schemas (`meta` among them), roles and role memberships, `SET
|
|
1717
|
+
ROLE`, `OWNER TO`, `SET SCHEMA`, `DO`, `COPY` from or to files and programs, extensions, and psql meta-commands (other
|
|
1718
|
+
than pg_dump's `\restrict`). An archive with several schemas, extensions or large objects is refused at the preview.
|
|
1719
|
+
A dump is still code: its functions and triggers run as pgkiln's owner role. **Import only dumps you trust.**
|
|
1720
|
+
|
|
1721
|
+
**The client programs.** pg_dump can't dump a server newer than itself, so pgkiln looks for pg_dump and pg_restore of
|
|
1722
|
+
the server's major version or newer, the newest it finds: `PGKILN_PG_BIN` (a directory), else
|
|
1723
|
+
`/usr/lib/postgresql/<n>/bin` (Debian, Ubuntu), `/usr/pgsql-<n>/bin` (RHEL), Homebrew's `postgresql@<n>`, else `PATH`.
|
|
1724
|
+
Without them the page says what to install (`postgresql-client-17` for a PostgreSQL 17 server). The Docker image
|
|
1725
|
+
includes PostgreSQL 17's client programs and keeps the files in `/app/dumps` (a volume in `deploy/compose.yaml`).
|
|
1726
|
+
|
|
1727
|
+
| Variable | Default | |
|
|
1728
|
+
|---|---|---|
|
|
1729
|
+
| `PGKILN_PG_BIN` | (searched) | the directory of pg_dump and pg_restore |
|
|
1730
|
+
| `PGKILN_DUMP_DIR` | `<temp>/pgkiln-dumps` | where the files are kept; one server's directory: with several pgkiln servers, a job's file is on the server that ran it |
|
|
1731
|
+
| `PGKILN_DUMP_KEEP_DAYS` | 7 | days a file is kept |
|
|
1732
|
+
| `PGKILN_DUMP_MAX_MB` | 2048 | the largest upload |
|
|
1733
|
+
|
|
1734
|
+
A job runs in the server that started it: when that server stops, the scheduler marks its running jobs *failed
|
|
1735
|
+
(interrupted)* within two minutes. Exports, uploads, imports and downloads go to the activity log (`schema_export`,
|
|
1736
|
+
`schema_upload`, `schema_import`, `schema_download`).
|
|
1737
|
+
|
|
1521
1738
|
## Export and import
|
|
1522
1739
|
|
|
1523
1740
|
**Export** downloads the application as JSON: pages, all components, shared components and
|
|
@@ -516,6 +516,17 @@ Workshop → RESTful Services**, AutoREST) run as the app role too, never as the
|
|
|
516
516
|
and the row level security below decide what an API caller may read and change. AutoREST refuses a role that owns
|
|
517
517
|
the published object or bypasses row level security, and an object answers only to a bearer token of its application
|
|
518
518
|
unless the developer chose *public* for it ([chapter 13](13-rest-api.md#autorest-tables-and-views-from-the-sql-workshop)).
|
|
519
|
+
(sprint 104) REST module handlers refuse to run as a superuser, a role that bypasses row level security or a member
|
|
520
|
+
of pgkiln's owner (403). Their parameters are bound: SQL handlers get escaped literals, PL/pgSQL handlers read
|
|
521
|
+
transaction-local settings through `meta.rest_param()`; a request can't set the server's binds (`APP_USER`, …) or
|
|
522
|
+
real settings. Media handlers are served with a sandboxing Content-Security-Policy and `nosniff`. *Try it* in the
|
|
523
|
+
SQL Workshop calls an endpoint as an account of the application, as a developer issuing a token could
|
|
524
|
+
([chapter 13](13-rest-api.md#modules-templates-and-handlers-in-the-sql-workshop)).
|
|
525
|
+
|
|
526
|
+
(sprint 106) The SQL Workshop's **Change Log** keeps the schema DDL developers run from the worksheet, Table
|
|
527
|
+
Properties and Schema Comparison, per workspace (only the workspace's developers see it). Statements about roles,
|
|
528
|
+
databases, servers, user mappings and subscriptions are never recorded, so passwords and connection strings stay
|
|
529
|
+
out of it; it holds what the DDL says otherwise (defaults, comments, function bodies).
|
|
519
530
|
|
|
520
531
|
### Database users and their passwords
|
|
521
532
|
|
|
@@ -68,8 +68,15 @@ src/
|
|
|
68
68
|
xml.ts safe XML reader (no DTDs or entities, limits) and xmlTable(): rows from a repeating element (unit tested)
|
|
69
69
|
sqlscript.ts SQL scripts: splitScript() (statements, line numbers and offsets, psql commands), runScript() (stop/continue, transaction, savepoints)
|
|
70
70
|
sqlworksheet.ts (sprint 91) the worksheet's runs: pickStatements() (at the cursor, a selection, all), runStatements() (a log per
|
|
71
|
-
statement, psql command tags, notices, error line and column), explainStatement() (EXPLAIN / rolled-back ANALYZE
|
|
71
|
+
statement, psql command tags, notices, error line and column), explainStatement() (EXPLAIN / rolled-back ANALYZE;
|
|
72
|
+
a savepoint inside a worksheet transaction)
|
|
73
|
+
worksheet-txn.ts (sprint 107) the worksheet's transaction mode: an in-memory registry of pinned pg.Clients per (builder session,
|
|
74
|
+
worksheet tab), beginTxn() / endTxn() (caps per developer and server), accountRun() (statement count, DDL kept for
|
|
75
|
+
the Change Log until COMMIT), outsideTxnOnly(), sweepTxns() (idle timeout, ended sessions); onSessionEnd and
|
|
76
|
+
onClosePools roll back; WORKSHEET_TXN_IDLE_SECONDS / _MAX_PER_DEVELOPER / _MAX
|
|
72
77
|
datamodel.ts (sprint 98) Data Modeler diagrams: readModel() (catalog), boxSize(), fkPath() (crow's feet), autoLayout(), diagramSvg()
|
|
78
|
+
changelog.ts (sprint 106, migration 118) the change log: isSchemaDdl(), ddlTarget() (schema and object), changeName()
|
|
79
|
+
(add_bonus, create_index_x), recordChanges() (grouped per schema), Flyway / Liquibase / plain SQL writers
|
|
73
80
|
tabledef.ts (sprint 95) a table as a definition: readTable(), checkTable(), tableSql() (CREATE, or ALTER from the catalog's version)
|
|
74
81
|
sqlformat.ts (sprint 93) formatSql(): the worksheet's Format (clauses per line, keyword case; strings and comments kept)
|
|
75
82
|
sqlplan.ts (sprint 92) EXPLAIN (FORMAT JSON) as a tree: parsePlan() (self time, slowest step, misestimates), planText()
|
|
@@ -87,9 +94,19 @@ src/
|
|
|
87
94
|
mvt.ts Mapbox Vector Tile encoding for map layers served as tiles
|
|
88
95
|
yamltext.ts the text style of the directory export: a strict YAML subset, written and read
|
|
89
96
|
webclient.ts outgoing HTTP to web services: allow-list, address checks at connect time (SSRF), redirects, limits
|
|
97
|
+
datapump.ts (sprint 103, DA-16) Schema Export / Import jobs (meta.dump_job): findTools (pg_dump/pg_restore ≥ the
|
|
98
|
+
server's major), startExport (pg_dump, argument array, PGPASSWORD), readToc/parseToc (pg_restore --list),
|
|
99
|
+
startImport (pg_restore -f - run in one transaction between schema renames; data-only in FK order),
|
|
100
|
+
cancel, delete, dumpTick (interrupted jobs, expired files; called by the scheduler)
|
|
101
|
+
dumpscript.ts (sprint 103) the import's script runner: ScriptLexer (psql-style split, COPY data, \restrict),
|
|
102
|
+
checkStatement (only the dump's schema; no roles, DO, COPY to/from files…), COPY FROM STDIN as a pg submittable
|
|
90
103
|
dbroles.ts (sprint 99, DA-13) Database Users' SQL: roleSql (CREATE/ALTER ROLE, memberships, search path, schema
|
|
91
104
|
and default privileges: only what differs from the catalog), dropSql, checkRole/checkPassword,
|
|
92
105
|
scramVerifier (SCRAM-SHA-256, SASLprep) and the password slot (maskSql for review and log, withVerifier to run)
|
|
106
|
+
cron-schedule.ts (sprint 101, DA-12) pg_cron schedules without a database: parseSchedule (cron fields, $, n seconds, macros),
|
|
107
|
+
nextRuns in a time zone, describeSchedule in words, the builder's presets and fields
|
|
108
|
+
pgcron.ts (sprint 101, DA-12) pg_cron for SQL Workshop → Scheduling: cronState (installed, loaded, cron.database_name,
|
|
109
|
+
access), jobs and runs, jobSql/activeSql/unscheduleSql/purgeSql (reviewed, then applySql), runNow
|
|
93
110
|
secrets.ts secrets at rest (web credentials): AES-256-GCM with PGKILN_SECRET_KEY
|
|
94
111
|
websources.ts web credentials (OAuth2 client credentials/password/refresh token grants, token cache, stored refresh
|
|
95
112
|
tokens) and REST data sources: requests, JSON paths, typed rows, response cache, write-back operations
|
|
@@ -152,7 +169,10 @@ src/
|
|
|
152
169
|
maps.ts map region (data for Leaflet: layers with a query each, markers, clusters or heat, PostGIS geometry as GeoJSON, report filter by area or distance; list fallback, head assets)
|
|
153
170
|
spatial.ts spatial filtering on the server: map area and distance parsing, PostGIS detection and SQL (ST_Intersects, ST_DWithin), lat/lng fallback (bounding box, haversine)
|
|
154
171
|
pwa.ts Progressive Web App: manifest, service worker route, icons (PNG encoder), offline page
|
|
155
|
-
rest.ts REST modules: handler checks, matching, bearer tokens,
|
|
172
|
+
rest.ts REST modules: handler and template checks, matching, bearer tokens, module privilege, the constrained-role
|
|
173
|
+
check, execution (collections stream from a cursor), OpenAPI (templates, documented parameters, roles)
|
|
174
|
+
rest-handlers.ts (sprint 104, migration 117) PL/pgSQL handlers (binds as settings, DO block, meta.rest_reply) and media
|
|
175
|
+
handlers (content type and file name cleaned, sandboxed response)
|
|
156
176
|
autorest.ts (sprint 100, migration 113) AutoREST: /a/:alias/autorest/<object>/ for the tables and views SQL Workshop →
|
|
157
177
|
RESTful Services published (meta.rest_object): caller() of rest.ts, SET ROLE to the app role, owner/BYPASSRLS
|
|
158
178
|
check, columns from the catalog (column grants), filters/q/order/paging, POST/PUT/DELETE by primary key, OpenAPI
|
|
@@ -234,7 +254,8 @@ src/
|
|
|
234
254
|
worksheet's Chart tab, SQL Workshop → Charts (meta.sql_chart, 110; read-only runs)
|
|
235
255
|
sql-tools.ts (sprint 96) SQL_TOOLS: every SQL Workshop tool with its group, tab and description (home, More menu, search)
|
|
236
256
|
worksheet.ts (sprint 91) the SQL Worksheet (Database Actions' SQL; /sql/commands too): navigator (pickSchema, objects with
|
|
237
|
-
columns, objectStatements), toolbar, output tabs, history (meta.sql_history, 108), plan trees, Run as an app's role, (sprint 93) Save / Rename / Download as SQL Scripts, bind and substitution values, POST /sql/worksheet/format; GET /sql/worksheet/navigator
|
|
257
|
+
columns, objectStatements), toolbar, output tabs, history (meta.sql_history, 108), plan trees, Run as an app's role, (sprint 93) Save / Rename / Download as SQL Scripts, bind and substitution values, POST /sql/worksheet/format; GET /sql/worksheet/navigator;
|
|
258
|
+
(sprint 107) transaction mode: the txn field, Commit / Rollback (mode=commit|rollback), txnBadge(), GET/POST /sql/worksheet/txn (JSON)
|
|
238
259
|
object-browser.ts (sprint 75) the Object Browser: objects by type, system objects hidden, a table in tabs, column actions as SQL to review
|
|
239
260
|
table-editor.ts (sprint 95) Table Properties: GET/POST /sql/objects/table (create, edit ?o=, ?template=), fromForm(), Preview / Apply
|
|
240
261
|
(one transaction) / Open in the worksheet
|
|
@@ -246,9 +267,14 @@ src/
|
|
|
246
267
|
quicksql.ts SQL Workshop → Quick SQL page (preview, save as script, run)
|
|
247
268
|
querybuilder.ts SQL Workshop → Query Builder: catalog, joins by foreign key or drawn, functions/group by, buildQuery() from the URL; the canvas is in builder.js
|
|
248
269
|
users.ts user directory and identity providers
|
|
270
|
+
datapump.ts (sprint 103, DA-16) SQL Workshop → Schema Export / Import (/builder/sql/data-pump, administrators only):
|
|
271
|
+
jobs, export form, upload (streamed to PGKILN_DUMP_DIR), archive preview and import options, job log, download
|
|
249
272
|
db-users.ts (sprint 99, DA-13) Administration → Database Users (/builder/db-users, administrators only): PostgreSQL
|
|
250
273
|
roles with highlights, create/edit (review the masked SQL, then Apply in one transaction), drop with
|
|
251
274
|
reassigning; pgkiln's own roles read-only, roles of applications and workspaces not dropped
|
|
275
|
+
scheduling.ts (sprint 101, DA-12) SQL Workshop → Scheduling (/builder/sql/scheduling, administrators only): pg_cron jobs
|
|
276
|
+
with next runs, create/edit with a schedule builder and preview (review the SQL, then Apply), disable,
|
|
277
|
+
run now, unschedule, run history and purge; without pg_cron the install note and pgkiln's automations
|
|
252
278
|
api.ts per-app REST API page (API role, tokens)
|
|
253
279
|
globalization.ts translations, XLIFF/CSV, text messages
|
|
254
280
|
dataload.ts SQL Workshop → Load Data (with definitions, save a mapping as one); data load definition spec (Shared Components)
|
|
@@ -260,11 +286,18 @@ src/
|
|
|
260
286
|
region-settings.ts page designer: settings forms for grid, chart (gauge, drill-down), cards, calendar (views, create, drag and drop), facets, smart filters, display selector, list
|
|
261
287
|
search.ts app search, "where used" (appEntries, search, whereUsed, usedInPanel)
|
|
262
288
|
advisor.ts Advisor: EXPLAIN every SQL fragment, reference checks, plpgsql_check
|
|
263
|
-
ddl-tools.ts SQL Workshop → DDL and Compare (src/ddl.ts)
|
|
289
|
+
ddl-tools.ts SQL Workshop → DDL and Compare (src/ddl.ts); (sprint 106) the Apply and record form
|
|
290
|
+
changelog.ts (sprint 106) SQL Workshop → Change Log (/builder/sql/changes): change sets per schema, latest app export,
|
|
291
|
+
downloads (POST, mark exported_at), rename, delete; /sql/compare/apply (one transaction, recorded)
|
|
264
292
|
sql-utilities.ts SQL Workshop → Utilities (sprint 62): the tools' tiles, OBJECT_REPORTS (fixed catalogue SQL over the
|
|
265
|
-
connection's usable schemas), About Database
|
|
293
|
+
connection's usable schemas), About Database; read only, with a timeout; /sql/utilities/monitor redirects to the SQL Monitor
|
|
266
294
|
login-page.ts Application settings → Login page and Forgot password; Create a login page (page 9999)
|
|
267
|
-
top-sql.ts Top SQL per app role from pg_stat_statements
|
|
295
|
+
top-sql.ts Top SQL per app role from pg_stat_statements; (sprint 102) statementsState / statementsNote (installed and loaded?)
|
|
296
|
+
sql-monitor.ts (sprint 102, DA-14) SQL Workshop → SQL Monitor (administrators): /sql/monitor (sessions, wait events, progress;
|
|
297
|
+
?fragment=1 the live part), POST /sql/monitor/session (cancel / terminate), /sql/monitor/top (topStatements:
|
|
298
|
+
pg_stat_statements sortable, bars), /reset, /statements/:queryid (totals by role, explainForMonitor: EXPLAIN only,
|
|
299
|
+
one statement by the extended protocol, read only, generic plan for $1), /statements/:queryid/worksheet
|
|
300
|
+
plan-view.ts (sprint 92; shared since 102) planView(): a plan (src/sqlplan.ts) as a tree, for the worksheet and the SQL Monitor
|
|
268
301
|
diagnostics.ts Activity → Debug messages (level, list, one request's entries, purge); Workspace utilities →
|
|
269
302
|
Installation (version, install/upgrade runs, applied and missing migrations; administrators)
|
|
270
303
|
ldap.ts Users → LDAP directories
|
|
@@ -279,6 +312,9 @@ src/
|
|
|
279
312
|
autorest.ts (sprint 100) SQL Workshop → RESTful Services (/builder/sql/rest): publish a table or view through an
|
|
280
313
|
application of the workspace (checks: schema the workshop may use, app role not the owner), its page
|
|
281
314
|
(grants per method, RLS, security_invoker, endpoints, cURL/fetch, OpenAPI link), settings, disable
|
|
315
|
+
rest-modules.ts (sprint 104) SQL Workshop → RESTful Services → Modules (/builder/sql/rest/modules): modules of the
|
|
316
|
+
workspace's applications, the module › template › handler tree, handler forms, Try it (inject with a
|
|
317
|
+
short-lived token), Run only / lock checks
|
|
282
318
|
workflows.ts workflow versions, diagram and instances (Shared Components)
|
|
283
319
|
process-jobs.ts page designer: the Jobs tab of a background chain process
|
|
284
320
|
template-spec.ts template component property form (Shared Components)
|
|
@@ -309,14 +345,20 @@ src/
|
|
|
309
345
|
public/
|
|
310
346
|
app.css theme (light/dark, responsive; --font, --font-size, --radius for style variants; template option classes to-*)
|
|
311
347
|
app.js client runtime: dialogs (dialog_closed actions), popup LOVs, dynamic actions (focus, classes, messages), grids (add/duplicate rows, master-detail refresh, move/resize columns, copy/paste of cell ranges), menus, lazy regions, maps (Leaflet layers, marker clusters, heat layer, layer legend, area/distance filter) (no inline JS)
|
|
312
|
-
code-editor.js, .css builder code editor: enhances <textarea data-code>, highlighting, suggestions (no dependencies)
|
|
348
|
+
code-editor.js, .css builder code editor: enhances <textarea data-code>, highlighting, suggestions (no dependencies);
|
|
349
|
+
(sprint 108) the current line, and read-only <pre data-code-view="sql|plpgsql|json|html"> highlighted;
|
|
350
|
+
its colours are the --ce-* tokens in builder.css
|
|
313
351
|
palette.js (sprint 96) the builder's search dialog (Ctrl+K), recently visited SQL Workshop pages, the Launchpad's tabs
|
|
352
|
+
sql-monitor.js (sprint 102) the SQL Monitor: reloads the sessions' live part every 5 / 15 / 60 s, Pause (localStorage pgkiln.sqlMonitor)
|
|
314
353
|
data-modeler.js, .css (sprint 98) the Data Modeler: drag boxes, undo/redo, save, zoom/fit, auto layout, mini-map, SVG/PNG export, print
|
|
315
354
|
worksheet.js, .css (sprint 91) the SQL Worksheet: run keys with the cursor, output tabs, navigator swap and search, drag and drop, splitters;
|
|
316
|
-
(92) grid sort, value dialog, history filter; (93) worksheet tabs in localStorage, Files, binds dialog, format, find, case
|
|
317
|
-
|
|
355
|
+
(92) grid sort, value dialog, history filter; (93) worksheet tabs in localStorage, Files, binds dialog, format, find, case;
|
|
356
|
+
(107) Autocommit / Transaction per tab, the badge from the server, beforeunload, rollback on closing a tab
|
|
357
|
+
builder.css builder only: IDE look (dark chrome, icon rail, panes); design tokens at the top (colour,
|
|
358
|
+
elevation, radius, type, motion; light and dark sets, sprint 105; code editor --ce-*, sprint 108),
|
|
359
|
+
then the shared controls
|
|
318
360
|
builder.js builder only: tabs, component tree, property filter, drag and drop on the layout
|
|
319
|
-
builder-icons.svg builder only: icons of the rail, toolbar and designer (b-*)
|
|
361
|
+
builder-icons.svg builder only: icons of the rail, toolbar and designer (b-*); b-kiln is the brand mark (brandMark() in ui.ts)
|
|
320
362
|
test/
|
|
321
363
|
binds.test.ts unit tests
|
|
322
364
|
security.test.ts security regression tests (in-process, against the database)
|
|
@@ -333,13 +375,20 @@ test/
|
|
|
333
375
|
unload.test.ts Unload Data: CSV/JSON/XLSX/XML output, read back with Load Data, read-only and one-statement checks, streaming
|
|
334
376
|
app-from-file.test.ts Create → From a file: proposed names and types, app + table + rows + pages, row errors, login, validation
|
|
335
377
|
workshop.test.ts SQL scripts, Quick SQL pages, query builder, data load definitions (Load Data, the process, export)
|
|
378
|
+
cron-schedule.test.ts (sprint 101) pg_cron schedules: parsing and errors, next runs across time zones and daylight saving, words
|
|
379
|
+
scheduling.test.ts (sprint 101) SQL Workshop → Scheduling: the SQL, the no-pg_cron page, create/edit/disable/run now/unschedule,
|
|
380
|
+
history and purge; against pg_cron when the test database has it, else test/pgcron-mock.ts (a stand-in)
|
|
336
381
|
sql-charts.test.ts (sprint 97) the worksheet's Chart tab, saving a chart, the Charts dashboard, a chart's page, read-only runs
|
|
337
382
|
launchpad.test.ts (sprint 96) Launchpad groups, pins (builder pages only, at most 24), the search (JSON and page)
|
|
338
383
|
tabledef.test.ts (sprint 95) table definitions round trip: CREATE, read, ALTER, read again; mistakes
|
|
384
|
+
changelog.test.ts (sprint 106) DDL recognition and names, recording from Table Properties, the worksheet and Compare,
|
|
385
|
+
downloads (Flyway, Liquibase, SQL), stable numbers, the latest application export
|
|
339
386
|
table-editor.test.ts (sprint 95) the Table Properties page: preview, apply, a failing apply rolls back, template, worksheet
|
|
340
387
|
object-actions.test.ts (sprint 94) object menus: Quick DDL, Drop with dependents, Add index / trigger / policy, the panel, not-found
|
|
341
388
|
data-modeler.test.ts (sprint 98) Data Modeler: "hr core" with its FK lines, saved positions, SVG, objects, DDL, dictionary, geometry
|
|
342
389
|
worksheet.test.ts (sprint 91) the SQL Worksheet: statement at the cursor, Run Script log, plans, notices, history, navigator, Run as
|
|
390
|
+
worksheet-txn.test.ts (sprint 107) transaction mode: the tab's transaction, Commit / Rollback, typed COMMIT, aborted, idle timeout, refusals,
|
|
391
|
+
Explain in it, Change Log at COMMIT, the JSON endpoint, sign-out
|
|
343
392
|
quicksql.test.ts Quick SQL parser and DDL generator
|
|
344
393
|
xml.test.ts XML reader: rows, attributes, paths, refused DTDs and entities, limits
|
|
345
394
|
printing.test.ts report PDFs
|
|
@@ -352,6 +401,7 @@ test/
|
|
|
352
401
|
charts.test.ts chart markup per kind (geometry as classes), gauges, Gantt time axis and dependencies, pyramid, polar, range, box plot, stock, drill-down links
|
|
353
402
|
calendar.test.ts calendar views, create links, moving events (pure and over HTTP)
|
|
354
403
|
rest-sources.test.ts REST data sources, web credentials, SSRF checks, invoke_api (mock service + HR page 23)
|
|
404
|
+
rest-modules.test.ts (sprint 104) modules, templates, PL/pgSQL and media handlers, OpenAPI, the builder's tree and Try it
|
|
355
405
|
workflow-invoke.test.ts workflow invoke_api steps: the call between transactions, faults, retry, lease, Advisor, export (mock service)
|
|
356
406
|
large-tables.test.ts row ranges, max_rows, row limits, lazy regions, region caching, streamed downloads (HR page 25)
|
|
357
407
|
grid.test.ts interactive grid: aggregates, layouts per user, saved grid reports, master-detail, row actions (HR page 27)
|
|
@@ -384,9 +434,10 @@ test/
|
|
|
384
434
|
ai-mock.ts a local mock of the Claude Messages API (SSE stream) and OpenAI Chat Completions
|
|
385
435
|
helpers.ts a cookie-keeping test browser
|
|
386
436
|
e2e/responsive.test.ts browser tests at phone/tablet/desktop widths (Playwright)
|
|
387
|
-
e2e/code-editor.test.ts the code editor in a browser: highlighting, keys, suggestions, touch, screen readers
|
|
437
|
+
e2e/code-editor.test.ts the code editor in a browser: highlighting, keys, suggestions, touch, screen readers, syntax colours at AA in both themes (sprint 108)
|
|
388
438
|
e2e/data-modeler.test.ts (sprint 98) the Data Modeler in a browser: drag, save, reload, undo/redo, zoom, mini-map, exports, phone
|
|
389
|
-
e2e/worksheet.test.ts (sprint 91) the SQL Worksheet in a browser: run keys, tabs, navigator, drag and drop, splitters, phone, no JavaScript
|
|
439
|
+
e2e/worksheet.test.ts (sprint 91) the SQL Worksheet in a browser: run keys, tabs, navigator, drag and drop, splitters, phone, no JavaScript;
|
|
440
|
+
(107) transaction mode: badge, reload, Rollback, closing a tab, Autocommit switch, beforeunload
|
|
390
441
|
e2e/items.test.ts sprint 26 item types in a browser: editors, tags, stars, dates, reveal; without JavaScript
|
|
391
442
|
e2e/designer.test.ts page designer: panes per width, drag and drop, keyboard, Arrange buttons, builder theme
|
|
392
443
|
e2e/calendar.test.ts calendar drag and drop and create on click, view switching, chart drill-down
|
|
@@ -437,7 +488,7 @@ CI (`.github/workflows/ci.yml`) runs these jobs against PostgreSQL 17:
|
|
|
437
488
|
- **test**: typecheck and `npm test` on a fresh database;
|
|
438
489
|
- **e2e**: the browser tests, uploading the screenshots as an artifact; `test/e2e/accessibility.test.ts` runs
|
|
439
490
|
axe-core (WCAG 2.1 A and AA rules) on every page of the HR example (light, dark, Iris) and the builder's main pages
|
|
440
|
-
and allows no violation;
|
|
491
|
+
(in both builder themes, with the SQL Workshop and the wizards, since sprint 108) and allows no violation;
|
|
441
492
|
- **docker**: builds the image and starts the compose stack as a user would.
|
|
442
493
|
|
|
443
494
|
- **upgrade**: installs each earlier release (`v0.31.0` …) with its HR example (`git archive <tag> | tar -x`,
|
|
@@ -31,11 +31,12 @@ answer 404.
|
|
|
31
31
|
|---|---|
|
|
32
32
|
| `method` | `GET`, `POST`, `PUT`, `PATCH` or `DELETE` |
|
|
33
33
|
| `path` | Segments with `:parameters`, e.g. `orders/:id/lines` |
|
|
34
|
-
| `type` | `collection`: a SELECT, returned page by page as `{items, offset, limit, has_more}` (`?limit=` up to 500, `?offset=`, `page_size` for the default); the page streams from a cursor as a chunked JSON array (an error before the first rows is still a 4xx/5xx, a failure after them ends the response short) · `item`: one row as an object, 404 when there is none · `sql`: statements; the first row of the last one is the response (201 for POST, 204 without a row; `status` overrides) |
|
|
34
|
+
| `type` | `collection`: a SELECT, returned page by page as `{items, offset, limit, has_more}` (`?limit=` up to 500, `?offset=`, `page_size` for the default); the page streams from a cursor as a chunked JSON array (an error before the first rows is still a 4xx/5xx, a failure after them ends the response short) · `item`: one row as an object, 404 when there is none · `sql`: statements; the first row of the last one is the response (201 for POST, 204 without a row; `status` overrides) · `plpgsql` (sprint 104): a PL/pgSQL block that answers with `meta.rest_reply(body, status)` ([below](#modules-templates-and-handlers-in-the-sql-workshop)) · `media`: a query whose row has `content_type`, `content` and optionally `filename`, sent as that content |
|
|
35
35
|
| `source` | The SQL. Binds: path parameters, query parameters and the fields of a JSON (or form) body, upper case (`:ID`, `:CUSTOMER`); `:BODY` is the whole JSON body. `:APP_USER` (the caller), `:APP_ID` and `:APP_ALIAS` are set by the server: a parameter of that name (or `APP_SESSION`, `APP_PAGE_ID`, `REQUEST`, `APP_LANGUAGE`) is ignored, and can't be a path parameter |
|
|
36
36
|
| `roles` | The caller needs one of these roles (as `meta.has_role()` sees them) |
|
|
37
37
|
| `auth` | `token` (default) or `public` (no token; the SQL runs with `meta.app_user()` = `nobody`) |
|
|
38
38
|
| `description` | For the OpenAPI description |
|
|
39
|
+
| `params` | (sprint 104) Documented parameters: `[{"name": "start_date", "in": "body", "type": "string", "required": true, "description": "…"}]` (`in`: `query` or `body`; types `string`, `integer`, `number`, `boolean`). They go into the OpenAPI document; a required one that is missing or empty answers 400 |
|
|
39
40
|
|
|
40
41
|
**Calling them.** Send `Authorization: Bearer <token>` with a token from **App → REST API**: a token
|
|
41
42
|
issued for an account, or one from `POST /oauth/token` for an OAuth client ([below](#tokens)). An
|
|
@@ -63,6 +64,63 @@ curl -X POST http://127.0.0.1:3100/a/hr/rest/v1/leave -H "Authorization: Bearer
|
|
|
63
64
|
-H "Content-Type: application/json" -d '{"start_date": "2027-05-03", "end_date": "2027-05-04", "reason": "dentist"}'
|
|
64
65
|
```
|
|
65
66
|
|
|
67
|
+
## Modules, templates and handlers in the SQL Workshop
|
|
68
|
+
|
|
69
|
+
**SQL Workshop → RESTful Services → Modules** (sprint 104; Oracle Database Actions / ORDS: REST › Modules) edits
|
|
70
|
+
the same REST modules the ORDS way, for every application of the workspace:
|
|
71
|
+
|
|
72
|
+
- a **module** is a base path, `/a/<alias>/rest/<name>/`, with a title, a description, *Enabled* and its
|
|
73
|
+
**privilege**: application roles (as `meta.has_role()` sees them) of which a caller needs one for *every*
|
|
74
|
+
handler of the module, public ones included;
|
|
75
|
+
- a **template** is a URI below it with `:parameters`, e.g. `emp/:id`, with a description for the OpenAPI document
|
|
76
|
+
(a template can exist before its first handler; renaming it moves its handlers);
|
|
77
|
+
- a **handler** answers one method of a template: its source type (*collection query*, *single row*, *SQL*,
|
|
78
|
+
*PL/pgSQL*, *media*), the source, who may call it (a token, or explicitly *public*), further roles, the page size
|
|
79
|
+
or status, and documented parameters, one per line: `start_date body string required First day`.
|
|
80
|
+
|
|
81
|
+
The tree on the left of each page shows module › templates › handlers. A Run only or locked application refuses
|
|
82
|
+
changes. The module's page checks the role the handlers run as and links the module's **OpenAPI** document.
|
|
83
|
+
|
|
84
|
+
**PL/pgSQL handlers** run their block as a `DO` statement as the application's database role. A `DO` block takes
|
|
85
|
+
no parameters, so pgkiln sets the request's binds as transaction-local settings (with query parameters: values
|
|
86
|
+
never become SQL text) and turns `:NAME` in the block into `meta.rest_param('NAME')` (text, NULL when absent):
|
|
87
|
+
|
|
88
|
+
```sql
|
|
89
|
+
declare
|
|
90
|
+
v_id int;
|
|
91
|
+
begin
|
|
92
|
+
v_id := sales.create_order(:CUSTOMER, :TOTAL::numeric);
|
|
93
|
+
perform meta.rest_reply(jsonb_build_object('id', v_id), 201);
|
|
94
|
+
end
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
`meta.rest_reply(body jsonb, status int)` sets the response (status 200–599, also 4xx for your own errors; the
|
|
98
|
+
transaction is still committed); without it the handler answers 204. A `raise exception` answers 400 with its
|
|
99
|
+
message and rolls back. Statements without `begin … end` are wrapped in one.
|
|
100
|
+
|
|
101
|
+
**Media handlers** run a query whose first row has `content_type`, `content` (bytea or text) and optionally
|
|
102
|
+
`filename`; the response is that content, sent with `X-Content-Type-Options: nosniff` and a sandboxing
|
|
103
|
+
Content-Security-Policy, so HTML or SVG from a table can't run script in the application's origin. No row: 404.
|
|
104
|
+
|
|
105
|
+
**Try it.** A handler's page sends a request to the real endpoint, with path parameters, a query string and a JSON
|
|
106
|
+
body, either without a token (to see the 401) or *as* an account of the application with a short-lived token (the
|
|
107
|
+
same as issuing one on App → REST API). A PUT, POST or DELETE changes data. The response's status and body are
|
|
108
|
+
shown in the page; it works without JavaScript.
|
|
109
|
+
|
|
110
|
+
**Example:** the HR example's module `hr` (`examples/hr/hr_62_rest_module.sql`): `GET emp/:id` (single row),
|
|
111
|
+
`GET emp/:id/photo` (media), `POST leave` (PL/pgSQL with body parameters) and `GET payroll/:deptno` (roles manager or
|
|
112
|
+
admin):
|
|
113
|
+
|
|
114
|
+
```bash
|
|
115
|
+
curl http://127.0.0.1:3100/a/hr/rest/hr/emp/7839 -H "Authorization: Bearer $TOKEN"
|
|
116
|
+
curl -X POST http://127.0.0.1:3100/a/hr/rest/hr/leave -H "Authorization: Bearer $TOKEN" \
|
|
117
|
+
-H "Content-Type: application/json" -d '{"start_date": "2027-05-03", "end_date": "2027-05-04", "reason": "dentist"}'
|
|
118
|
+
# {"id": 12, "status": "requested"}
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Handlers never run as a superuser, a role that bypasses row level security or pgkiln's owner: such a request
|
|
122
|
+
answers 403 (set `RUNTIME_DATABASE_URL` and give the application its own database role).
|
|
123
|
+
|
|
66
124
|
## AutoREST: tables and views from the SQL Workshop
|
|
67
125
|
|
|
68
126
|
**SQL Workshop → RESTful Services** (`/builder/sql/rest`; Database Actions: a table's *REST › Enable*, ORDS
|
|
@@ -19,7 +19,7 @@ setup`) and on practically every managed service. Install with `create extension
|
|
|
19
19
|
| **unaccent** | Accent-insensitive search (`café` finds `cafe`), essential in multilingual apps | Text search options |
|
|
20
20
|
| **citext** | Case-insensitive text columns (e-mail addresses, codes) without `lower()` everywhere | NLS_COMP=LINGUISTIC |
|
|
21
21
|
| **btree_gist** | Exclusion constraints on ranges, e.g. **no overlapping bookings/leave** enforced by the database, race-free | Custom PL/SQL checks |
|
|
22
|
-
| **pg_stat_statements** | Which SQL is slow or called most, per role, so per app (`db_role`) | Monitor Activity → Top SQL |
|
|
22
|
+
| **pg_stat_statements** | Which SQL is slow or called most, per role, so per app (`db_role`) | Monitor Activity → Top SQL; SQL Workshop → SQL Monitor (Top SQL, a statement's totals) |
|
|
23
23
|
| **tablefunc** | `crosstab()` for pivot tables | Interactive report *Pivot* |
|
|
24
24
|
| **ltree** | Hierarchies (org charts, categories, menus) with fast subtree queries | Tree region, `CONNECT BY` |
|
|
25
25
|
| **postgres_fdw**, **file_fdw** | Query another PostgreSQL database, or a CSV file, as if it were a table | Remote servers / database links |
|