pgkiln 0.35.0 → 0.37.0

package/SECURITY.md

@@ -67,7 +67,8 @@ first version (MVP). Every finding below has a regression test in
 | AI services (0.28.0) | Administrators only configure services (anonymous → sign-in, other developers 403, CSRF); API keys are encrypted with `PGKILN_SECRET_KEY`, write-only, never exported, logged or shown; all AI tables are revoked from public and the runtime role. Model, base URL and limits are checked; models are never changed silently. Item values go into prompts as escaped `<data>` text marked as data, never instructions; session ids and password items are never sent. Answers only become item values and are escaped when shown. Usage is logged without prompt or answer text (except at debug level 9). Data in prompts leaves the instance for the chosen provider. AI calls don't use `PGKILN_REST_ALLOWED_HOSTS`: only administrators set the base URL. |
 | AI assistant, NL2IR (0.28.0) | CSRF and region visibility are checked before any AI call; assistants are for signed-in users unless marked public. Conversations belong to one session and region, are deleted with the session and are closed to the runtime and application roles. Context queries and tools run as the app's role (grants, RLS): arguments are bound literals checked against the declared parameters (unknown arguments refused), each tool is one subquery with a row limit and a 15 s timeout and is rolled back unless the developer marks it as writing; tools can be limited by an authorization scheme; REST tool arguments are never `&ITEM.`-substituted. Natural-language report filters only produce filters a user could set by hand, on visible columns. Answers are escaped (paragraphs, lists, bold, code only). A writing tool lets the model change data on the user's behalf within the role's rights: design those with care. |
 | App Builder AI and blueprints (0.28.0) | Developer session and CSRF; only administrators choose the builder's AI service. Generated SQL is shown, never run automatically; page proposals are checked (page types, tables the app role can read, free page numbers) and created only after the developer confirms. The model sees names, types and keys, never rows. A blueprint is validated (identifiers, references, rows) and turned into quoted SQL; Create only accepts the exact text the review showed, signed with the developer's session, in one transaction. `meta.builder_ai`, `meta.ai_table_note` and `meta.blueprint` are closed to the runtime role. |
-| Workspaces (0.29.0) | Every builder request under `/apps/:id` or `/pages/:pid` (GET and POST), a working copy's main application, a boilerplate and the code editor's application are checked against the developer's workspaces in `developer()` and answer 404 outside them; lists are filtered by the current workspace. Switching needs CSRF and membership; managing workspaces needs an administrator and CSRF; names are escaped; subscriptions only reach theme and library applications of the same workspace. Workspace tables are closed to the runtime role. **Not a tenant boundary:** the SQL Workshop runs as the owner and application code on the runtime connection, so developers who write SQL can reach other workspaces; accounts, identity providers, LDAP directories and AI services are installation-wide. |
+| Workspaces (0.29.0) | Every builder request under `/apps/:id` or `/pages/:pid` (GET and POST), a working copy's main application, a boilerplate and the code editor's application are checked against the developer's workspaces in `developer()` and answer 404 outside them; lists are filtered by the current workspace. Switching needs CSRF and membership; managing workspaces needs an administrator and CSRF; names are escaped; subscriptions only reach theme and library applications of the same workspace. Workspace tables are closed to the runtime role. **A shared workspace is not a tenant boundary:** its SQL Workshop runs as the owner and its application code on the shared runtime connection; isolate a workspace for that (below). |
+| Isolated workspaces (sprint 60) | The boundary is the PostgreSQL login (`session_user`, which SQL can't change): an isolated workspace's applications run on its own run login and its developers' SQL on its own developer login (`src/workspace-db.ts`; passwords derived from `PGKILN_SECRET_KEY`, never stored), so `RESET ROLE` or `set_config('role', …)` only reach that login. The run login has `pgkiln_app_code` (the runtime role's rights in `meta`, kept in step after every migration by `meta.sync_app_code_grants()`, checked by a test) and the workspace's roles; the developer login only the workspace's roles. Row-level security on every `meta` table the runtime reads keys on the login (`meta.visible_apps()`), both ways: isolated workspaces don't see others, the shared runtime doesn't see isolated ones; the owner isn't affected. `meta.app_id()` and the eight functions that take an application id (`meta.checked_app`) refuse an application the login may not see. A workspace's database roles are an administrator's list (`meta.workspace_role`): an application there may only run as one of them, a role belongs to one workspace, and superusers, roles with CREATEROLE/BYPASSRLS/REPLICATION and pgkiln's own roles are refused; memberships follow by triggers (`meta.workspace_sync_roles()`). A new application gets a new role and a new or own schema. Every builder tool that runs a developer's SQL uses the workspace's developer login (`workshopDb`, `devDbForApp`); the import check plans the imported SQL on it, outside the import's owner transaction. Saved scripts, runs, blueprints and generators are per workspace. **Not covered:** administrators and developers who also belong to a shared workspace (owner rights there), the `pg_catalog` (definitions are readable by every login), PostgREST (one authenticator), identity providers, LDAP, mail and AI services (installation-wide). |
 | Drawers, built-in template components, Theme Roller additions (0.29.0) | Dialog positions and sizes, base styles, dark-mode colours, item and report-column template options only reach the page from fixed lists (checked when saved and again when rendered: a hand-edited value is ignored); the browser gets dialog shapes as JSON values from those lists. Built-in template components pass the same template checks as plug-ins; *Copy into this application* takes a static id from the fixed list, needs CSRF and the application in the developer's workspace, and never overwrites the application's own component. |
 | Report row selection across pages (0.29.0) | `POST …/report/<id>/select` needs the CSRF token, page access and a visible report region whose `selection` names the item (and the item visible); it only changes that item's session state; values are bounded (5000, 400 characters, no `:`) and treated as user input, like a submit; output is escaped. |
 | Instance settings (0.29.0) | Administrators only, with CSRF; only whole numbers within each setting's range are stored (a bad stored value falls back to the environment variable or default); the configuration overview never shows secrets (connection strings, keys: only whether they are set); changes are logged.
@@ -184,18 +185,19 @@ A second full review, mainly of what end users and API callers can reach. Regres
   can do what the runtime role can: read every application's metadata and sessions, set
   `pgkiln.app_id` / `pgkiln.app_user` and call `meta.page_url()` for any application and user (valid
   checksums), or change sessions. Treat every developer as trusted with all applications of the
-  instance, as the workspace notes say; end users never write SQL. A separate login role per
-  application would close this.
+  instance's shared workspaces, as the workspace notes say; end users never write SQL. An isolated workspace
+  (sprint 60) closes this between workspaces: its SQL resets to a login of its own.
 - **Administrators and developers**: SQL Commands, SQL Scripts and the other SQL Workshop tools run on
-  the owner connection for every developer, so a developer can change anything, including
+  the owner connection for administrators and the developers of shared workspaces, so a developer can change anything, including
   `meta.developer.is_admin`. The administrator-only pages (instance settings, AI services, identity
   providers, LDAP directories, the password policy, developers) prevent mistakes; they are not a
   boundary against a developer who means harm.
-- **App id from SQL (known gap, to fix)**: `meta.app_id()` reads the `pgkiln.app_id` setting, which an
-  application's own SQL can change. Requests queued from SQL (`meta.web_request`, `meta.ai_generate`) could
-  therefore be made under another application's id, using its web credentials or AI services and quota.
-  Page processes are not affected (they use the server's app id). Until this is fixed, treat applications
-  on one instance as mutually trusted for these features.
+- **App id from SQL (shared workspaces)**: `meta.app_id()` reads the `pgkiln.app_id` setting, which an
+  application's own SQL can change. In shared workspaces, requests queued from SQL (`meta.web_request`,
+  `meta.ai_generate`) could therefore be made under another shared application's id, using its web credentials or
+  AI services and quota. Page processes are not affected (they use the server's app id). Since sprint 60,
+  `meta.app_id()` refuses an application outside the login's workspace: isolate a workspace whose applications must
+  not be trusted by the others.
 - **Lockout trade-off**: per-user throttling lets an attacker lock a known
   account for 15 minutes. The per-IP limit and the activity monitor help you
   spot this.

package/db/migrations/087_login_page.sql

@@ -0,0 +1,175 @@
+-- =====================================================================
+-- 087: an editable login page and "Forgot password?" for app users
+--
+-- meta.app.login_page: the page that replaces the built-in sign-in page
+--   (APEX: the login page, usually 9999). /a/<alias>/login sends visitors
+--   there; it must not require authentication. Its "login" region is the
+--   sign-in form, which posts to /a/<alias>/login as before.
+-- meta.page.page_template: 'minimal' shows a page without the header and
+--   navigation (APEX: Minimal (No Navigation)); login pages use it.
+-- meta.app.password_reset: the sign-in page offers "Forgot password?" to
+--   accounts with a password in the user directory, when the application
+--   has an enabled mail service. A reset link is single-use and expires
+--   after 30 minutes; only a SHA-256 hash of its token is stored.
+-- meta.password_reset: the tokens (owner only; the functions below are
+--   the runtime's way in).
+-- =====================================================================
+alter table meta.app add column login_page int check (login_page > 0);
+alter table meta.app add column password_reset boolean not null default false;
+alter table meta.page add column page_template text not null default 'standard' check (page_template in ('standard', 'minimal'));
+
+-- an export made before 087 has neither: NULL becomes the default (like 077)
+create function meta.login_page_defaults() returns trigger
+language plpgsql set search_path = pg_catalog as $$
+begin
+  if tg_table_name = 'app' then
+    new.password_reset := coalesce(new.password_reset, false);
+  else
+    new.page_template := coalesce(new.page_template, 'standard');
+  end if;
+  return new;
+end
+$$;
+create trigger app_login_page_defaults before insert or update on meta.app
+  for each row execute function meta.login_page_defaults();
+create trigger page_login_page_defaults before insert or update on meta.page
+  for each row execute function meta.login_page_defaults();
+
+alter table meta.region drop constraint region_type_check;
+alter table meta.region add constraint region_type_check
+  check (type in ('report', 'form', 'chart', 'cards', 'static', 'grid', 'calendar', 'dynamic', 'facets', 'tasks', 'workflows', 'map', 'tree',
+                  'template_component', 'smart_filters', 'display_selector', 'list', 'data_reporter', 'ai_assistant', 'plugin', 'url', 'search',
+                  'login'));
+
+create table meta.password_reset (
+  id          bigint generated always as identity primary key,
+  app_id      int not null references meta.app on delete cascade,
+  account_id  int not null references meta.account on delete cascade,
+  token_hash  text not null unique check (token_hash ~ '^[0-9a-f]{64}$'),
+  ip          text,
+  created_at  timestamptz not null default now(),
+  expires_at  timestamptz not null default now() + interval '30 minutes',
+  used_at     timestamptz
+);
+create index password_reset_account on meta.password_reset (account_id, created_at);
+revoke all on meta.password_reset from public;
+
+-- Whether the application offers "Forgot password?": turned on, app users with local passwords,
+-- and an enabled mail service with a From address.
+create function meta.password_reset_available(p_app_id int) returns boolean
+language sql stable security definer set search_path = meta, pg_catalog as $$
+  select exists (
+    select 1
+      from meta.app a
+      join meta.app_mail_service ams on ams.app_id = a.id
+      join meta.mail_service s on s.id = ams.service_id
+     where a.id = p_app_id and a.password_reset and a.authentication = 'app_users' and a.local_login
+       and s.enabled and nullif(btrim(ams.from_address), '') is not null)
+$$;
+revoke all on function meta.password_reset_available(int) from public;
+grant execute on function meta.password_reset_available(int) to pgkiln_runtime;
+
+-- A reset request for a user name or e-mail address. When it matches one active account with a
+-- password and an e-mail address that may use the application, keeps the token's hash and queues
+-- the mail (#USER# in the subject and bodies becomes the user name; the caller puts the link in).
+-- Says nothing about whether an account matched: the caller shows the same answer either way.
+-- At most 3 requests per account per hour.
+create function meta.request_password_reset(p_app_id int, p_login text, p_token_hash text, p_ip text,
+                                            p_subject text, p_body text, p_body_html text) returns void
+language plpgsql volatile security definer set search_path = meta, public, pg_catalog as $$
+declare
+  v_acc   meta.account;
+  v_n     int;
+  v_from  text;
+  v_send  text[];
+  v_max   int;
+  v_vals  jsonb;
+  v_id    bigint;
+begin
+  if not meta.password_reset_available(p_app_id) or p_token_hash !~ '^[0-9a-f]{64}$' or btrim(coalesce(p_login, '')) = '' then
+    return;
+  end if;
+  -- a user name, else an e-mail address that only one active account has
+  select * into v_acc from meta.account where lower(username) = lower(btrim(p_login)) and active;
+  if not found then
+    select count(*) into v_n from meta.account where lower(email) = lower(btrim(p_login)) and active;
+    if v_n <> 1 then
+      return;
+    end if;
+    select * into v_acc from meta.account where lower(email) = lower(btrim(p_login)) and active;
+  end if;
+  if v_acc.password_hash is null or nullif(btrim(v_acc.email), '') is null then
+    return;
+  end if;
+  if not exists (select 1 from meta.app a where a.id = p_app_id and a.access_control = 'any_user')
+     and not exists (select 1 from meta.app_access where app_id = p_app_id and account_id = v_acc.id) then
+    return;
+  end if;
+  select count(*) into v_n from meta.password_reset where account_id = v_acc.id and created_at > now() - interval '1 hour';
+  if v_n >= 3 then
+    return;
+  end if;
+  select ams.from_address, s.senders, ams.max_per_day into v_from, v_send, v_max
+    from meta.app_mail_service ams join meta.mail_service s on s.id = ams.service_id
+   where ams.app_id = p_app_id;
+  if not meta.mail_sender_allowed(v_send, v_from) then
+    raise warning 'password reset for app %: the mail service may not send as %', p_app_id, v_from;
+    return;
+  end if;
+  if v_max is not null and (select count(*) from meta.mail_queue
+                             where app_id = p_app_id and created_at >= date_trunc('day', now() at time zone 'UTC') at time zone 'UTC'
+                               and status <> 'cancelled') >= v_max then
+    raise warning 'password reset for app %: the daily mail limit is reached', p_app_id;
+    return;
+  end if;
+  delete from meta.password_reset where created_at < now() - interval '7 days';
+  insert into meta.password_reset (app_id, account_id, token_hash, ip) values (p_app_id, v_acc.id, p_token_hash, left(p_ip, 100));
+  v_vals := jsonb_build_object('USER', v_acc.username);
+  insert into meta.mail_queue (app_id, username, from_address, to_addresses, subject, body_text, body_html, template)
+  values (p_app_id, v_acc.username, v_from, array[v_acc.email], left(replace(replace(meta.mail_fill(p_subject, v_vals, false), E'\r', ''), E'\n', ' '), 998),
+          meta.mail_fill(p_body, v_vals, false), meta.mail_fill(p_body_html, v_vals, true), 'PASSWORD_RESET')
+  returning id into v_id;
+  perform pg_notify('pgkiln_mail', v_id::text);
+end
+$$;
+revoke all on function meta.request_password_reset(int, text, text, text, text, text, text) from public;
+grant execute on function meta.request_password_reset(int, text, text, text, text, text, text) to pgkiln_runtime;
+
+-- The user name a valid (unused, unexpired) token is for, or null.
+create function meta.password_reset_user(p_app_id int, p_token_hash text) returns text
+language sql stable security definer set search_path = meta, pg_catalog as $$
+  select ac.username
+    from meta.password_reset r join meta.account ac on ac.id = r.account_id
+   where r.app_id = p_app_id and r.token_hash = p_token_hash and r.used_at is null and r.expires_at > now() and ac.active
+$$;
+revoke all on function meta.password_reset_user(int, text) from public;
+grant execute on function meta.password_reset_user(int, text) to pgkiln_runtime;
+
+-- Set the new password with a valid token: the token and the account's other open requests are used
+-- up, every session and "keep me signed in" token of the account ends. Returns the user name, or
+-- null when the token is not valid (any more).
+create function meta.reset_password(p_app_id int, p_token_hash text, p_new text) returns text
+language plpgsql volatile security definer set search_path = meta, public, pg_catalog as $$
+declare
+  v_reset meta.password_reset;
+  v_acc   meta.account;
+begin
+  select * into v_reset from meta.password_reset
+   where app_id = p_app_id and token_hash = p_token_hash and used_at is null and expires_at > now()
+   for update;
+  if not found or coalesce(p_new, '') = '' then
+    return null;
+  end if;
+  select * into v_acc from meta.account where id = v_reset.account_id and active;
+  if not found then
+    return null;
+  end if;
+  update meta.account set password_hash = crypt(p_new, gen_salt('bf', 10)), must_change_password = false where id = v_acc.id;
+  update meta.password_reset set used_at = now() where account_id = v_acc.id and used_at is null;
+  delete from meta.session where app_id is not null and lower(username) = lower(v_acc.username);
+  delete from meta.persistent_login where account_id = v_acc.id;
+  return v_acc.username;
+end
+$$;
+revoke all on function meta.reset_password(int, text, text) from public;
+grant execute on function meta.reset_password(int, text, text) to pgkiln_runtime;

package/db/migrations/088_region_templates.sql

@@ -0,0 +1,18 @@
+-- =====================================================================
+-- 088: more region templates (APEX: Hero, Title Bar, Buttons Container,
+-- Inline Dialog, Inline Drawer, Wizard Container)
+--
+-- hero              a large banner: title, text, an icon, buttons
+-- title_bar         the title and buttons on one line, without a card
+-- buttons_container the buttons only (and any text), without a header
+-- inline_dialog     hidden until opened (dynamic action "Open region", or
+--                   a link to #R<id>); a dialog over the page
+-- inline_drawer     the same, sliding in from the side
+-- wizard            a narrow card for one step of a wizard (with the list
+--                   template "wizard_progress" above it)
+-- The carousel is a style of the display selector region ({"style":
+-- "carousel"}); it needs no column.
+-- =====================================================================
+alter table meta.region drop constraint region_template_check;
+alter table meta.region add constraint region_template_check
+  check (template in ('standard', 'plain', 'collapsible', 'hero', 'title_bar', 'buttons_container', 'inline_dialog', 'inline_drawer', 'wizard'));

package/db/migrations/089_page_groups.sql

@@ -0,0 +1,330 @@
+-- =====================================================================
+-- 089: page groups (APEX: Shared Components → Page Groups)
+--
+-- meta.page_group: an application's named groups of pages, to organise
+--   them in the App Builder (the page list by group, a filter). A page's
+--   group is meta.page.page_group (the group's name; renaming a group in
+--   the builder renames it on its pages). They change nothing at run time.
+-- export_app / import_app are redefined from 086 to carry page_groups.
+-- =====================================================================
+create table meta.page_group (
+  id          serial primary key,
+  app_id      int  not null references meta.app on delete cascade,
+  name        text not null check (name ~ '^[A-Za-z0-9][A-Za-z0-9 _.-]{0,59}$'),
+  description text,
+  unique (app_id, name)
+);
+revoke all on meta.page_group from public;
+grant select on meta.page_group to pgkiln_runtime;
+comment on table meta.page_group is 'Page groups (089): named groups of an application''s pages, for the App Builder';
+
+alter table meta.page add column page_group text;
+
+-- renaming a group renames it on its pages; deleting it takes the pages out of it
+create function meta.page_group_follow() returns trigger
+language plpgsql set search_path = meta, pg_catalog as $$
+begin
+  if tg_op = 'DELETE' then
+    update meta.page set page_group = null where app_id = old.app_id and page_group = old.name;
+    return old;
+  end if;
+  if new.name is distinct from old.name then
+    update meta.page set page_group = new.name where app_id = new.app_id and page_group = old.name;
+  end if;
+  return new;
+end
+$$;
+create trigger page_group_follow after update or delete on meta.page_group
+  for each row execute function meta.page_group_follow();
+
+create or replace function meta.export_app(p_alias text) returns jsonb
+language sql stable set search_path = meta, pg_catalog as $$
+  select jsonb_build_object(
+    'format', 'pgkiln/2',
+    -- (083) availability and build status belong to the installation
+    'app', to_jsonb(a) - 'id' - 'created_at' - 'updated_at' - 'availability' - 'availability_message' - 'availability_users' - 'build_status',
+    'authz_schemes', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.authz_scheme x where x.app_id = a.id), '[]'),
+    'app_items', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.app_item x where x.app_id = a.id), '[]'),
+    'app_processes', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.seq, x.id) from meta.app_process x where x.app_id = a.id), '[]'),
+    'lovs', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.lov x where x.app_id = a.id), '[]'),
+    'group_roles', coalesce((select jsonb_agg(to_jsonb(x) - 'app_id' order by x.group_name, x.role) from meta.app_group_role x where x.app_id = a.id), '[]'),
+    'text_messages', coalesce((select jsonb_agg(to_jsonb(x) - 'app_id' order by x.name, x.language) from meta.text_message x where x.app_id = a.id), '[]'),
+    'translations', coalesce((select jsonb_agg(to_jsonb(x) - 'app_id' order by x.language, x.source) from meta.translation x where x.app_id = a.id), '[]'),
+    'report_layouts', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' - 'logo' || jsonb_build_object('logo', encode(x.logo, 'base64')) order by x.name)
+                                  from meta.report_layout x where x.app_id = a.id), '[]'),
+    'automations', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' - 'next_run_at' - 'last_run_at' - 'last_status' - 'code' order by x.name)
+                               from meta.automation x where x.app_id = a.id), '[]'),
+    -- (044) the actions of the automations, by automation name
+    'automation_actions', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.automation_name collate "C", x.seq, x.name collate "C")
+                                      from meta.automation_action x where x.app_id = a.id), '[]'),
+    'document_templates', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.document_template x where x.app_id = a.id), '[]'),
+    'task_definitions', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.task_definition x where x.app_id = a.id), '[]'),
+    'workflow_definitions', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.workflow_definition x where x.app_id = a.id), '[]'),
+    'rest_modules', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.rest_module x where x.app_id = a.id), '[]'),
+    'template_components', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.static_id) from meta.template_component x where x.app_id = a.id), '[]'),
+    'build_options', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.build_option x where x.app_id = a.id), '[]'),
+    -- (030) secrets never leave the installation: they are entered again after an import
+    -- (050) nor do the password and refresh token of the OAuth2 flows, or the token state
+    'web_credentials', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' - 'secret_enc' - 'password_enc' - 'refresh_token_enc' - 'token_refreshed_at' order by x.name)
+                                   from meta.web_credential x where x.app_id = a.id), '[]'),
+    -- (050) the synchronisation's state (next and last run) belongs to the installation
+    'rest_sources', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' - 'sync_next_at' - 'sync_last_at' - 'sync_last_status' order by x.name)
+                                from meta.rest_source x where x.app_id = a.id), '[]'),
+    -- (041) data load definitions
+    'data_load_definitions', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.data_load_def x where x.app_id = a.id), '[]'),
+    -- (042) lists; entries keep their ids so parents can be remapped on import
+    'lists', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.list x where x.app_id = a.id), '[]'),
+    'list_entries', coalesce((select jsonb_agg(to_jsonb(x) - 'app_id' order by x.list_name, x.parent_id nulls first, x.seq, x.id) from meta.list_entry x where x.app_id = a.id), '[]'),
+    -- (042) supporting objects: install, upgrade and deinstall scripts (never run on import)
+    'supporting_scripts', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.supporting_script x where x.app_id = a.id), '[]'),
+    -- (069) plug-ins (their files are static files, their templates template components)
+    'plugins', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.plugin x where x.app_id = a.id), '[]'),
+    -- (080) search configurations
+    'search_configs', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.search_config x where x.app_id = a.id), '[]'),
+    -- (081) e-mail templates
+    'email_templates', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.email_template x where x.app_id = a.id), '[]'),
+    -- (086) application settings (with their values) and dynamic translations
+    'app_settings', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.app_setting x where x.app_id = a.id), '[]'),
+    'page_groups', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.name) from meta.page_group x where x.app_id = a.id), '[]'),
+    'dynamic_translations', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' order by x.language, x.source collate "C") from meta.dynamic_translation x where x.app_id = a.id), '[]'),
+    -- (068) static application files (JavaScript, CSS, images, fonts) as base64
+    'static_files', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'app_id' - 'content' - 'updated_at' || jsonb_build_object('content', translate(encode(x.content, 'base64'), E'\n', '')) order by x.name)
+                                from meta.static_file x where x.app_id = a.id), '[]'),
+    -- nav entries and regions keep their ids, so parents and references can be remapped on import
+    'nav', coalesce((select jsonb_agg(to_jsonb(x) - 'app_id' order by x.parent_id nulls first, x.seq, x.id) from meta.nav_entry x where x.app_id = a.id), '[]'),
+    'pages', coalesce((
+      select jsonb_agg(to_jsonb(p) - 'id' - 'app_id' || jsonb_build_object(
+        'regions', coalesce((select jsonb_agg(to_jsonb(r) - 'page_id' order by r.seq, r.id) from meta.region r where r.page_id = p.id), '[]'),
+        'items', coalesce((select jsonb_agg(to_jsonb(i) - 'id' - 'page_id' order by i.seq, i.id) from meta.item i where i.page_id = p.id), '[]'),
+        'buttons', coalesce((select jsonb_agg(to_jsonb(b) - 'id' - 'page_id' order by b.seq, b.id) from meta.button b where b.page_id = p.id), '[]'),
+        'dynamic_actions', coalesce((select jsonb_agg(to_jsonb(d) - 'id' - 'page_id' order by d.seq, d.id) from meta.dynamic_action d where d.page_id = p.id), '[]'),
+        'validations', coalesce((select jsonb_agg(to_jsonb(v) - 'id' - 'page_id' order by v.seq, v.id) from meta.validation v where v.page_id = p.id), '[]'),
+        'processes', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'page_id' order by x.seq, x.id) from meta.process x where x.page_id = p.id), '[]'),
+        'computations', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'page_id' order by x.seq, x.id) from meta.computation x where x.page_id = p.id), '[]'),
+        'branches', coalesce((select jsonb_agg(to_jsonb(x) - 'id' - 'page_id' order by x.seq, x.id) from meta.branch x where x.page_id = p.id), '[]')
+      ) order by p.page_no)
+      from meta.page p where p.app_id = a.id), '[]'))
+  from meta.app a
+  where a.alias = p_alias
+$$;
+
+-- ---------------------------------------------------------------- import
+-- Same as 081, plus app_settings and dynamic_translations.
+
+create or replace function meta.import_app(p_doc jsonb, p_alias text default null) returns int
+language plpgsql set search_path = meta, pg_catalog as $$
+declare
+  v_app_id  int;
+  v_page_id int;
+  v_page    jsonb;
+  v_e       jsonb;
+  v_rmap    jsonb;
+  v_nmap    jsonb := '{}';
+  v_lmap    jsonb := '{}';
+  v_new_id  int;
+begin
+  if p_doc->>'format' is distinct from 'pgkiln/2' then
+    raise exception 'unsupported export format %', coalesce(p_doc->>'format', '(none)');
+  end if;
+
+  insert into meta.app
+  select (jsonb_populate_record(null::meta.app, p_doc->'app' || jsonb_build_object(
+            'id', nextval('meta.app_id_seq'),
+            'alias', coalesce(p_alias, p_doc->'app'->>'alias'),
+            'created_at', now(), 'updated_at', now()))).*
+  returning id into v_app_id;
+
+  insert into meta.authz_scheme
+  select (jsonb_populate_record(null::meta.authz_scheme, e || jsonb_build_object('id', nextval('meta.authz_scheme_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'authz_schemes', '[]')) e;
+  insert into meta.app_item
+  select (jsonb_populate_record(null::meta.app_item, e || jsonb_build_object('id', nextval('meta.app_item_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'app_items', '[]')) e;
+  insert into meta.app_process
+  select (jsonb_populate_record(null::meta.app_process, e || jsonb_build_object('id', nextval('meta.app_process_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'app_processes', '[]')) e;
+  insert into meta.lov
+  select (jsonb_populate_record(null::meta.lov, e || jsonb_build_object('id', nextval('meta.lov_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'lovs', '[]')) e;
+  insert into meta.app_group_role
+  select (jsonb_populate_record(null::meta.app_group_role, e || jsonb_build_object('app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'group_roles', '[]')) e;
+  insert into meta.text_message
+  select (jsonb_populate_record(null::meta.text_message, e || jsonb_build_object('app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'text_messages', '[]')) e;
+  insert into meta.translation
+  select (jsonb_populate_record(null::meta.translation, e || jsonb_build_object('app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'translations', '[]')) e;
+  insert into meta.report_layout
+  select (jsonb_populate_record(null::meta.report_layout, (e - 'logo') || jsonb_build_object(
+            'id', nextval('meta.report_layout_id_seq'), 'app_id', v_app_id,
+            'logo', null))).*
+    from jsonb_array_elements(coalesce(p_doc->'report_layouts', '[]')) e;
+  update meta.report_layout l
+     set logo = decode(e->>'logo', 'base64')
+    from jsonb_array_elements(coalesce(p_doc->'report_layouts', '[]')) e
+   where l.app_id = v_app_id and l.name = e->>'name' and e->>'logo' is not null;
+  -- imported automations start switched off: the copy must not run the original's jobs unasked.
+  -- (044) a file with actions: the automations' code is ignored; an older file: code becomes the single action
+  insert into meta.automation
+  select (jsonb_populate_record(null::meta.automation, '{"error_handling": "stop"}'::jsonb
+            || jsonb_strip_nulls(case when p_doc ? 'automation_actions' then e - 'code' else e end)
+            || jsonb_build_object('id', nextval('meta.automation_id_seq'), 'app_id', v_app_id, 'enabled', false))).*
+    from jsonb_array_elements(coalesce(p_doc->'automations', '[]')) e;
+  insert into meta.automation_action
+  select (jsonb_populate_record(null::meta.automation_action, '{"seq": 10}'::jsonb || jsonb_strip_nulls(e)
+            || jsonb_build_object('id', nextval('meta.automation_action_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'automation_actions', '[]')) e;
+  insert into meta.document_template
+  select (jsonb_populate_record(null::meta.document_template, e || jsonb_build_object('id', nextval('meta.document_template_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'document_templates', '[]')) e;
+  insert into meta.task_definition
+  select (jsonb_populate_record(null::meta.task_definition, e || jsonb_build_object('id', nextval('meta.task_definition_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'task_definitions', '[]')) e;
+  insert into meta.workflow_definition
+  select (jsonb_populate_record(null::meta.workflow_definition, e || jsonb_build_object('id', nextval('meta.workflow_definition_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'workflow_definitions', '[]')) e;
+  insert into meta.rest_module
+  select (jsonb_populate_record(null::meta.rest_module, e || jsonb_build_object('id', nextval('meta.rest_module_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'rest_modules', '[]')) e;
+  -- (028) template components: regions and report columns refer to them by static id
+  insert into meta.template_component
+  select (jsonb_populate_record(null::meta.template_component, e || jsonb_build_object('id', nextval('meta.template_component_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'template_components', '[]')) e;
+  -- (029) build options (components name them by name)
+  insert into meta.build_option
+  select (jsonb_populate_record(null::meta.build_option, e || jsonb_build_object('id', nextval('meta.build_option_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'build_options', '[]')) e;
+  -- (030) web credentials arrive without a secret, whatever the document holds
+  -- (050) nor a password or refresh token
+  insert into meta.web_credential
+  select (jsonb_populate_record(null::meta.web_credential, '{"type": "basic", "valid_for": [], "grant_type": "client_credentials"}'::jsonb
+            || jsonb_strip_nulls(e - 'secret_enc' - 'password_enc' - 'refresh_token_enc' - 'token_refreshed_at')
+            || jsonb_build_object('id', nextval('meta.web_credential_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'web_credentials', '[]')) e;
+  -- (050) write-back operations and synchronisation; an imported synchronisation starts switched off
+  insert into meta.rest_source
+  select (jsonb_populate_record(null::meta.rest_source, '{"method": "GET", "headers": {}, "params": [], "columns": [], "cache_seconds": 0, "timeout_s": 10, "max_rows": 1000,
+                                                          "operations": {}, "key_columns": [], "sync_mode": "merge", "sync_delete": false, "sync_time_zone": "UTC"}'::jsonb
+            || jsonb_strip_nulls(e - 'sync_next_at' - 'sync_last_at' - 'sync_last_status')
+            || jsonb_build_object('id', nextval('meta.rest_source_id_seq'), 'app_id', v_app_id, 'sync_enabled', false))).*
+    from jsonb_array_elements(coalesce(p_doc->'rest_sources', '[]')) e;
+
+  -- (041) data load definitions (the data_load process names them)
+  insert into meta.data_load_def
+  select (jsonb_populate_record(null::meta.data_load_def, '{"format": "auto", "headers": true, "mode": "append", "skip_errors": false, "columns": []}'::jsonb
+            || jsonb_strip_nulls(e) || jsonb_build_object('id', nextval('meta.data_load_def_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'data_load_definitions', '[]')) e;
+
+  -- (042) lists and their entries, supporting scripts. Entries are inserted
+  -- without a parent first and the parents set afterwards, so the order of
+  -- the entries in the document doesn't matter (a moved entry may have a
+  -- higher id than its children).
+  insert into meta.list
+  select (jsonb_populate_record(null::meta.list, '{"type": "static"}'::jsonb || jsonb_strip_nulls(e) || jsonb_build_object('id', nextval('meta.list_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'lists', '[]')) e;
+  for v_e in select * from jsonb_array_elements(coalesce(p_doc->'list_entries', '[]')) loop
+    insert into meta.list_entry
+    select (jsonb_populate_record(null::meta.list_entry, '{"target_items": {}}'::jsonb || jsonb_strip_nulls(v_e) || jsonb_build_object(
+              'id', nextval('meta.list_entry_id_seq'), 'app_id', v_app_id, 'parent_id', null))).*
+    returning id into v_new_id;
+    if v_e->>'id' is not null then
+      v_lmap := v_lmap || jsonb_build_object(v_e->>'id', v_new_id);
+    end if;
+  end loop;
+  update meta.list_entry x
+     set parent_id = (v_lmap->>(e->>'parent_id'))::int
+    from jsonb_array_elements(coalesce(p_doc->'list_entries', '[]')) e
+   where e->>'parent_id' is not null and v_lmap ? (e->>'parent_id') and v_lmap ? (e->>'id')
+     and x.id = (v_lmap->>(e->>'id'))::int;
+  insert into meta.supporting_script
+  select (jsonb_populate_record(null::meta.supporting_script, '{"kind": "install", "seq": 10, "script": ""}'::jsonb || jsonb_strip_nulls(e)
+            || jsonb_build_object('id', nextval('meta.supporting_script_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'supporting_scripts', '[]')) e;
+
+  -- (069) plug-ins
+  insert into meta.plugin
+  select (jsonb_populate_record(null::meta.plugin, e || jsonb_build_object('id', nextval('meta.plugin_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'plugins', '[]')) e;
+
+  -- (080) search configurations (search regions name them)
+  insert into meta.search_config
+  select (jsonb_populate_record(null::meta.search_config, '{"seq": 10, "source_type": "table", "searchable_columns": [], "max_results": 10}'::jsonb
+            || jsonb_strip_nulls(e) || jsonb_build_object('id', nextval('meta.search_config_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'search_configs', '[]')) e;
+
+  -- (081) e-mail templates (meta.send_mail_template and the send_email process name them)
+  insert into meta.email_template
+  select (jsonb_populate_record(null::meta.email_template, e || jsonb_build_object('id', nextval('meta.email_template_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'email_templates', '[]')) e;
+
+  -- (089) page groups
+  insert into meta.page_group
+  select (jsonb_populate_record(null::meta.page_group, e || jsonb_build_object('id', nextval('meta.page_group_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'page_groups', '[]')) e;
+
+  -- (086) application settings and dynamic translations
+  insert into meta.app_setting
+  select (jsonb_populate_record(null::meta.app_setting, e || jsonb_build_object('id', nextval('meta.app_setting_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'app_settings', '[]')) e;
+  insert into meta.dynamic_translation
+  select (jsonb_populate_record(null::meta.dynamic_translation, e || jsonb_build_object('id', nextval('meta.dynamic_translation_id_seq'), 'app_id', v_app_id))).*
+    from jsonb_array_elements(coalesce(p_doc->'dynamic_translations', '[]')) e;
+
+  -- (068) static application files
+  insert into meta.static_file (app_id, name, mime, content, updated_at)
+  select v_app_id, e->>'name', e->>'mime', decode(coalesce(e->>'content', ''), 'base64'), now()
+    from jsonb_array_elements(coalesce(p_doc->'static_files', '[]')) e;
+
+  for v_e in select * from jsonb_array_elements(coalesce(p_doc->'nav', '[]')) loop
+    insert into meta.nav_entry
+    select (jsonb_populate_record(null::meta.nav_entry, v_e || jsonb_build_object(
+              'id', nextval('meta.nav_entry_id_seq'), 'app_id', v_app_id,
+              'parent_id', v_nmap->>(v_e->>'parent_id')))).*
+    returning id into v_new_id;
+    v_nmap := v_nmap || jsonb_build_object(v_e->>'id', v_new_id);
+  end loop;
+
+  for v_page in select * from jsonb_array_elements(coalesce(p_doc->'pages', '[]')) loop
+    insert into meta.page
+    select (jsonb_populate_record(null::meta.page, v_page || jsonb_build_object('id', nextval('meta.page_id_seq'), 'app_id', v_app_id))).*
+    returning id into v_page_id;
+
+    v_rmap := '{}';
+    for v_e in select * from jsonb_array_elements(coalesce(v_page->'regions', '[]')) loop
+      insert into meta.region
+      select (jsonb_populate_record(null::meta.region, v_e || jsonb_build_object('id', nextval('meta.region_id_seq'), 'page_id', v_page_id))).*
+      returning id into v_new_id;
+      v_rmap := v_rmap || jsonb_build_object(v_e->>'id', v_new_id);
+    end loop;
+    -- facet and map regions point at their report region by id
+    update meta.region r
+       set config = jsonb_set(r.config, '{report}', to_jsonb((v_rmap->>(r.config->>'report'))::int))
+     where r.page_id = v_page_id and r.type in ('facets', 'smart_filters', 'map') and v_rmap ? (r.config->>'report');
+
+    insert into meta.item
+    select (jsonb_populate_record(null::meta.item, e || jsonb_build_object('id', nextval('meta.item_id_seq'), 'page_id', v_page_id, 'region_id', v_rmap->>(e->>'region_id')))).*
+      from jsonb_array_elements(coalesce(v_page->'items', '[]')) e;
+    insert into meta.button
+    select (jsonb_populate_record(null::meta.button, e || jsonb_build_object('id', nextval('meta.button_id_seq'), 'page_id', v_page_id, 'region_id', v_rmap->>(e->>'region_id')))).*
+      from jsonb_array_elements(coalesce(v_page->'buttons', '[]')) e;
+    insert into meta.dynamic_action
+    select (jsonb_populate_record(null::meta.dynamic_action, e || jsonb_build_object('id', nextval('meta.dynamic_action_id_seq'), 'page_id', v_page_id, 'affected_region_id', v_rmap->>(e->>'affected_region_id')))).*
+      from jsonb_array_elements(coalesce(v_page->'dynamic_actions', '[]')) e;
+    insert into meta.validation
+    select (jsonb_populate_record(null::meta.validation, e || jsonb_build_object('id', nextval('meta.validation_id_seq'), 'page_id', v_page_id))).*
+      from jsonb_array_elements(coalesce(v_page->'validations', '[]')) e;
+    insert into meta.process
+    select (jsonb_populate_record(null::meta.process, e || jsonb_build_object('id', nextval('meta.process_id_seq'), 'page_id', v_page_id, 'region_id', v_rmap->>(e->>'region_id')))).*
+      from jsonb_array_elements(coalesce(v_page->'processes', '[]')) e;
+    insert into meta.computation
+    select (jsonb_populate_record(null::meta.computation, e || jsonb_build_object('id', nextval('meta.computation_id_seq'), 'page_id', v_page_id))).*
+      from jsonb_array_elements(coalesce(v_page->'computations', '[]')) e;
+    insert into meta.branch
+    select (jsonb_populate_record(null::meta.branch, e || jsonb_build_object('id', nextval('meta.branch_id_seq'), 'page_id', v_page_id))).*
+      from jsonb_array_elements(coalesce(v_page->'branches', '[]')) e;
+  end loop;
+
+  return v_app_id;
+end
+$$;