limitsdb 0.5.0__tar.gz → 0.6.0__tar.gz

{limitsdb-0.5.0 → limitsdb-0.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: limitsdb
-Version: 0.5.0
+Version: 0.6.0
 Summary: Configuration-driven ILM (Information Lifecycle Management) for production and history databases
 License: Apache License
          Version 2.0, January 2004
@@ -207,6 +207,52 @@ source .venv/bin/activate
 pip install limitsdb
 ```
 
+### Air-gapped or internet-restricted servers
+
+For production database servers without internet access (common in enterprise Oracle
+environments on Linux, AIX, or Solaris), download the packages on a machine that
+does have access and transfer them to the target server.
+
+**Step 1 — Download on a connected machine**
+
+If the target server is Linux x86_64:
+
+```bash
+pip download limitsdb \
+  --dest ./limitsdb-bundle \
+  --platform linux_x86_64 \
+  --python-version 312 \
+  --only-binary=:all:
+```
+
+For the same platform as the current machine (Windows or macOS build host
+copying to an identical Linux target, omit `--platform` and `--python-version`):
+
+```bash
+pip download limitsdb --dest ./limitsdb-bundle
+```
+
+**Step 2 — Transfer to the target server**
+
+```bash
+scp -r ./limitsdb-bundle user@server:/opt/limitsdb-bundle
+```
+
+**Step 3 — Install on the target server (no internet required)**
+
+```bash
+python3 -m venv /opt/limitsdb-venv
+source /opt/limitsdb-venv/bin/activate
+pip install --no-index --find-links /opt/limitsdb-bundle limitsdb
+ldb-run --help
+```
+
+> **Note for AIX and Solaris**: `oracledb` may not have a prebuilt wheel for your
+> platform. If `pip download --only-binary=:all:` finds nothing, the package must
+> be compiled from source on the target, which requires Python development headers
+> and a C compiler (`gcc` or `xlc`). Contact your system administrator before
+> proceeding.
+
 For development setup (cloning the repo, running tests, CI) see [CONTRIBUTING.md](CONTRIBUTING.md).
 
 ## Quickstart
@@ -227,7 +273,7 @@ Spin up a minimal end-to-end run with the built-in scaffolding tools:
    ldb-crypt --schema billing --profile dev
    ```
 
-4. **Dry-run the ILM plan** (no data changes):
+4. **Preview the ILM run** (applies the required DDL and grants, moves no data):
 
    ```bash
    ldb-run --schema billing --profile dev --action SOURCE_ILM --mode PREVIEW
@@ -356,9 +402,32 @@ What it creates in both source and history environments:
 - **Database link** — bidirectional links between source and history so the ILM
   engine can reach both environments from either side.
 
-`ldb-impl` requires all four credential pairs to be present and encrypted in
-`secrets[.<PROFILE>].json`: `source_password`, `history_password`,
-`admin_source_password`, and `admin_history_password`. Run `ldb-crypt` first.
+`ldb-impl` needs the runtime credentials (`source_password`, `history_password`)
+encrypted in `secrets[.<PROFILE>].json`. Run `ldb-crypt` first.
+
+### Without admin credentials
+
+Admin credentials (`admin_source_*`, `admin_history_*`) are optional per
+database. When they are absent for a database, `ldb-impl` creates nothing in it
+and writes a SQL*Plus script for a DBA instead, one file per database:
+`bootstrap_<schema>_<source|history>_<YYYYMMDD_HHMMSS>.sql` under
+`ddl_pending_dir` (the same parameter as `ldb-run`: `--ddl-pending-dir`,
+`LDB_DDL_PENDING_DIR`, default `ldb_ddl_pending`). A database whose admin credentials are present is still
+provisioned directly. Generating scripts is not an error (exit code 0, with a
+warning naming each file); if a script cannot be written, `ldb-impl` exits 1.
+
+The script:
+
+- stops at the first error (`whenever sqlerror exit failure`) and ends with a
+  `LimitsDb: OK` banner on success;
+- contains no passwords: it prompts for them (hidden input);
+- can be rerun: each object is created in a PL/SQL block that ignores only its
+  own "already exists" error, and an existing database link is skipped;
+- creates the database link after reconnecting as the runtime user, who owns it
+  (it asks for that user's password).
+
+Run it in SQL*Plus connected as a user with administrative privileges:
+`sqlplus admin@<dsn> @bootstrap_<schema>_<env>_<timestamp>.sql`.
 
 ## Run ILM
 
@@ -372,9 +441,11 @@ ldb-run --schema <SCHEMA> [--profile <PROFILE>] --action SOURCE_ILM|HISTORY_ILM
 Common overrides:
 
 ```
---mode VALIDATE|PLAN|PREVIEW|SCRIPT|EXECUTE   # VALIDATE setup, PLAN DAG, PREVIEW simulate, SCRIPT emit SQL, EXECUTE run
+--mode VALIDATE|PLAN|PREVIEW|SCRIPT|EXECUTE   # VALIDATE setup, PLAN DAG, PREVIEW run without moving data, SCRIPT emit SQL, EXECUTE run
 --chunk-size <int>                            # Rows per chunk when processing large tables
 --parallel-max <int>                          # Maximum number of tables processed in parallel
+--ddl-pending-dir <path>                      # Where failed history DDL is written for a DBA (default: ldb_ddl_pending)
+--history-baseline EMPTY|SOURCE               # SCRIPT without a readable history: assume no history objects (default) or the same objects as source
 --log-level DEBUG|INFO|WARNING|ERROR|CRITICAL # Adjust logger verbosity
 --ilm-config-file <path>                      # Point to a specific ILM YAML file
 --config-dir <root>                           # Override config root discovery
@@ -396,24 +467,84 @@ ldb-run --schema billing --profile prod --action SOURCE_ILM --mode EXECUTE --set
 
 `ldb-run` uses admin credentials (`admin_source_password`,
 `admin_history_password`) for two operations: granting table privileges to the
-application role, and creating or altering history tables before an `EXECUTE`
-run. Both operations are idempotent.
+application role, and creating or altering history tables before a `PREVIEW` or
+`EXECUTE` run (`PREVIEW` needs the objects in place to validate and time the
+queries; it skips only the data movement). Both operations are idempotent.
 
 If admin credentials are absent from `secrets[.<PROFILE>].json`, the privilege
 grants are skipped silently — the assumption is that grants are already in
-place. History table management in `EXECUTE` mode still requires admin
-credentials in the current version.
+place.
+
+History table management follows the same principle:
+
+- **Admin credentials present, tables already up to date**: the admin connection
+  is not opened. No DDL is executed.
+- **Admin credentials present, DDL changes are required**: the admin connection
+  is opened and the changes are applied as before.
+- **Admin credentials absent, tables already up to date**: `ldb-run` proceeds
+  without an admin connection. This is the normal case for environments where a
+  DBA pre-creates all history objects and admin credentials are not stored.
+- **Admin credentials absent, history schema cannot be inspected** (for example
+  the runtime connection or dictionary query fails): `ldb-run` exits with code 1
+  and logs the cause. It does not run ILM against a history schema whose state is
+  unknown.
+- **Admin credentials absent, DDL changes are required**: `ldb-run` applies the
+  DDL directly over the runtime history connection, with no prior privilege
+  check. It succeeds when the runtime user owns the objects or holds the needed
+  privileges (for example `ALTER`). Any DDL statement that fails, for whatever
+  reason, is recorded and the remaining statements are still attempted. If at least one
+  statement failed, `ldb-run` writes the failed statements, each preceded by its
+  error, to `history_ddl_<schema>_<YYYYMMDD_HHMMSS>.sql` under `ddl_pending_dir`
+  (default `ldb_ddl_pending`), logs an error naming the file, and exits with
+  code 1. The file is plain SQL for SQL*Plus (no PL/SQL): it stops at the first
+  failing statement with a non-zero exit code, and only a run that reaches the
+  final `LimitsDb: OK` banner applied everything. The recorded errors are
+  comments. A DBA reviews and runs the file, then `ldb-run` is rerun. Nothing is
+  written to stdout, and the directory is created only when a file is written.
+  If the file cannot be written (for example the directory is not writable),
+  `ldb-run` logs the error with the failed statements and exits with code 1.
+
+### History DDL in `--mode SCRIPT`
+
+`--mode SCRIPT` applies nothing to the databases. The ILM SQL blocks, which
+run many times, go to stdout. The one-time statements go to files under
+`ddl_pending_dir`, so they stay out of the script that is rerun:
+
+- `history_ddl_<schema>_<YYYYMMDD_HHMMSS>.sql`: the history table DDL followed by
+  the grants on the history tables for the history role. Run it against the
+  history database.
+- `source_grants_<schema>_<YYYYMMDD_HHMMSS>.sql` (`SOURCE_ILM` only): the grants
+  on the source tables for the source role. Run it against the source database.
+
+Both are plain SQL for SQL*Plus that stop at the first failing statement; the
+grants are unconditional and safe to run again. The history DDL is always
+emitted, computed like this:
+
+- **A history connection works**: the admin connection is used when admin
+  credentials exist, the runtime one otherwise. The current history structure is
+  read, and the file holds only the missing changes.
+- **Neither works or there are no credentials**: `history_baseline` states what to
+  assume. `EMPTY` (default) assumes history has no objects, so every table, key
+  and index is created. `SOURCE` assumes history holds the same objects as
+  source, so only the history-specific differences are emitted (for example the
+  `LDB_PROCESS_DATE` column and the matching primary key).
+
+Without admin credentials, `ldb-run` applies the history grants right after the
+DDL over the runtime connection. Any grant the runtime user cannot apply goes to
+the same pending file as the failed DDL, so a DBA can run it.
 
 ### Configuration parameters reference
 
 | Parameter                | Purpose                                                                                                                                                         | Default / values                                                                     |
 | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
 | `action`                 | ILM target (`SOURCE_ILM` moves/purges source, `HISTORY_ILM` cleans downstream history).                                                                         | Default `SOURCE_ILM`; choices `SOURCE_ILM`, `HISTORY_ILM`.                           |
-| `mode`                   | Execution mode: `VALIDATE` config/credentials, `PLAN` dependency order, `PREVIEW` simulate (a.k.a. legacy `DRY_RUN`), `SCRIPT` emit SQL, `EXECUTE` run changes. | Default `PREVIEW`; choices `VALIDATE`, `PLAN`, `PREVIEW`, `SCRIPT`, `EXECUTE`.       |
+| `mode`                   | Execution mode: `VALIDATE` config/credentials, `PLAN` dependency order, `PREVIEW` run the real queries and apply DDL and grants but do not insert or delete data (a.k.a. legacy `DRY_RUN`), `SCRIPT` emit the history DDL and the ILM SQL without applying anything, `EXECUTE` run changes. | Default `PREVIEW`; choices `VALIDATE`, `PLAN`, `PREVIEW`, `SCRIPT`, `EXECUTE`.       |
 | `chunk_size`             | Rows per chunk when processing large tables.                                                                                                                    | Default `100000`.                                                                    |
 | `use_added_columns`      | Populate derived columns in history tables.                                                                                                                     | Default `True` (boolean toggle).                                                     |
 | `add_ldb_columns`        | Add LimitsDb execution-date columns in history tables.                                                                                                        | Default `True` (boolean toggle).                                                     |
 | `parallel_max`           | Maximum number of parallel processes.                                                                                                                           | Default `10`.                                                                        |
+| `ddl_pending_dir`        | Directory where `ldb-run` writes history DDL it could not apply without admin credentials, and the DDL and grants files of `--mode SCRIPT`.                                                                      | Default `ldb_ddl_pending`.                                                           |
+| `history_baseline`       | State assumed for the history tables by `--mode SCRIPT` when it cannot read them over the runtime history connection: `EMPTY` (no objects) or `SOURCE` (same objects as source). | Default `EMPTY`; choices `EMPTY`, `SOURCE`. |
 | `db_engine`              | Database engine.                                                                                                                                                | Default and only current choice: `"oracle"`.                                        |
 | `log_level`              | Logging level.                                                                                                                                                  | Default `"INFO"`; choices `"DEBUG"`, `"INFO"`, `"WARNING"`, `"ERROR"`, `"CRITICAL"`. |
 | `schema`                 | Configuration set name: identifies a system or application (e.g., `billing`). Not a database schema — it is the folder that groups all configuration files for one target. | Required on CLI; no persisted default. |
@@ -440,6 +571,8 @@ credentials in the current version.
 | `use_added_columns`      | —                          | —                            | `config[.<PROFILE>].yml: use_added_columns`        |
 | `add_ldb_columns`        | —                          | —                            | `config[.<PROFILE>].yml: add_ldb_columns`          |
 | `parallel_max`           | `--parallel-max`           | `LDB_PARALLEL_MAX`           | `config[.<PROFILE>].yml: parallel_max`             |
+| `ddl_pending_dir`        | `--ddl-pending-dir`        | `LDB_DDL_PENDING_DIR`        | `config[.<PROFILE>].yml: ddl_pending_dir`          |
+| `history_baseline`       | `--history-baseline`       | `LDB_HISTORY_BASELINE`       | `config[.<PROFILE>].yml: history_baseline`         |
 | `db_engine`              | `--db-engine`              | `LDB_DB_ENGINE`              | `config[.<PROFILE>].yml: db_engine`                |
 | `log_level`              | `--log-level`              | `LDB_LOG_LEVEL`              | `config[.<PROFILE>].yml: log_level`                |
 | `schema`                 | `--schema`                 | `LDB_SCHEMA`                 | CLI only (not stored).                             |
@@ -497,6 +630,8 @@ Every key is commented; the default value and short help appear inline. Example:
 # use_added_columns: True  # Populate derived columns in history tables
 # add_ldb_columns: True  # Add LimitsDb execution-date columns in history tables
 # parallel_max: 10      # Maximum number of parallel processes
+# ddl_pending_dir: "ldb_ddl_pending"  # Where failed history DDL is written for a DBA
+# history_baseline: "EMPTY"  # SCRIPT without readable history: EMPTY | SOURCE
 # db_engine: "oracle"   # Database engine
 # log_level: "INFO"     # Logging level
 ```

{limitsdb-0.5.0 → limitsdb-0.6.0}/README.md

@@ -61,6 +61,52 @@ source .venv/bin/activate
 pip install limitsdb
 ```
 
+### Air-gapped or internet-restricted servers
+
+For production database servers without internet access (common in enterprise Oracle
+environments on Linux, AIX, or Solaris), download the packages on a machine that
+does have access and transfer them to the target server.
+
+**Step 1 — Download on a connected machine**
+
+If the target server is Linux x86_64:
+
+```bash
+pip download limitsdb \
+  --dest ./limitsdb-bundle \
+  --platform linux_x86_64 \
+  --python-version 312 \
+  --only-binary=:all:
+```
+
+For the same platform as the current machine (Windows or macOS build host
+copying to an identical Linux target, omit `--platform` and `--python-version`):
+
+```bash
+pip download limitsdb --dest ./limitsdb-bundle
+```
+
+**Step 2 — Transfer to the target server**
+
+```bash
+scp -r ./limitsdb-bundle user@server:/opt/limitsdb-bundle
+```
+
+**Step 3 — Install on the target server (no internet required)**
+
+```bash
+python3 -m venv /opt/limitsdb-venv
+source /opt/limitsdb-venv/bin/activate
+pip install --no-index --find-links /opt/limitsdb-bundle limitsdb
+ldb-run --help
+```
+
+> **Note for AIX and Solaris**: `oracledb` may not have a prebuilt wheel for your
+> platform. If `pip download --only-binary=:all:` finds nothing, the package must
+> be compiled from source on the target, which requires Python development headers
+> and a C compiler (`gcc` or `xlc`). Contact your system administrator before
+> proceeding.
+
 For development setup (cloning the repo, running tests, CI) see [CONTRIBUTING.md](CONTRIBUTING.md).
 
 ## Quickstart
@@ -81,7 +127,7 @@ Spin up a minimal end-to-end run with the built-in scaffolding tools:
    ldb-crypt --schema billing --profile dev
    ```
 
-4. **Dry-run the ILM plan** (no data changes):
+4. **Preview the ILM run** (applies the required DDL and grants, moves no data):
 
    ```bash
    ldb-run --schema billing --profile dev --action SOURCE_ILM --mode PREVIEW
@@ -210,9 +256,32 @@ What it creates in both source and history environments:
 - **Database link** — bidirectional links between source and history so the ILM
   engine can reach both environments from either side.
 
-`ldb-impl` requires all four credential pairs to be present and encrypted in
-`secrets[.<PROFILE>].json`: `source_password`, `history_password`,
-`admin_source_password`, and `admin_history_password`. Run `ldb-crypt` first.
+`ldb-impl` needs the runtime credentials (`source_password`, `history_password`)
+encrypted in `secrets[.<PROFILE>].json`. Run `ldb-crypt` first.
+
+### Without admin credentials
+
+Admin credentials (`admin_source_*`, `admin_history_*`) are optional per
+database. When they are absent for a database, `ldb-impl` creates nothing in it
+and writes a SQL*Plus script for a DBA instead, one file per database:
+`bootstrap_<schema>_<source|history>_<YYYYMMDD_HHMMSS>.sql` under
+`ddl_pending_dir` (the same parameter as `ldb-run`: `--ddl-pending-dir`,
+`LDB_DDL_PENDING_DIR`, default `ldb_ddl_pending`). A database whose admin credentials are present is still
+provisioned directly. Generating scripts is not an error (exit code 0, with a
+warning naming each file); if a script cannot be written, `ldb-impl` exits 1.
+
+The script:
+
+- stops at the first error (`whenever sqlerror exit failure`) and ends with a
+  `LimitsDb: OK` banner on success;
+- contains no passwords: it prompts for them (hidden input);
+- can be rerun: each object is created in a PL/SQL block that ignores only its
+  own "already exists" error, and an existing database link is skipped;
+- creates the database link after reconnecting as the runtime user, who owns it
+  (it asks for that user's password).
+
+Run it in SQL*Plus connected as a user with administrative privileges:
+`sqlplus admin@<dsn> @bootstrap_<schema>_<env>_<timestamp>.sql`.
 
 ## Run ILM
 
@@ -226,9 +295,11 @@ ldb-run --schema <SCHEMA> [--profile <PROFILE>] --action SOURCE_ILM|HISTORY_ILM
 Common overrides:
 
 ```
---mode VALIDATE|PLAN|PREVIEW|SCRIPT|EXECUTE   # VALIDATE setup, PLAN DAG, PREVIEW simulate, SCRIPT emit SQL, EXECUTE run
+--mode VALIDATE|PLAN|PREVIEW|SCRIPT|EXECUTE   # VALIDATE setup, PLAN DAG, PREVIEW run without moving data, SCRIPT emit SQL, EXECUTE run
 --chunk-size <int>                            # Rows per chunk when processing large tables
 --parallel-max <int>                          # Maximum number of tables processed in parallel
+--ddl-pending-dir <path>                      # Where failed history DDL is written for a DBA (default: ldb_ddl_pending)
+--history-baseline EMPTY|SOURCE               # SCRIPT without a readable history: assume no history objects (default) or the same objects as source
 --log-level DEBUG|INFO|WARNING|ERROR|CRITICAL # Adjust logger verbosity
 --ilm-config-file <path>                      # Point to a specific ILM YAML file
 --config-dir <root>                           # Override config root discovery
@@ -250,24 +321,84 @@ ldb-run --schema billing --profile prod --action SOURCE_ILM --mode EXECUTE --set
 
 `ldb-run` uses admin credentials (`admin_source_password`,
 `admin_history_password`) for two operations: granting table privileges to the
-application role, and creating or altering history tables before an `EXECUTE`
-run. Both operations are idempotent.
+application role, and creating or altering history tables before a `PREVIEW` or
+`EXECUTE` run (`PREVIEW` needs the objects in place to validate and time the
+queries; it skips only the data movement). Both operations are idempotent.
 
 If admin credentials are absent from `secrets[.<PROFILE>].json`, the privilege
 grants are skipped silently — the assumption is that grants are already in
-place. History table management in `EXECUTE` mode still requires admin
-credentials in the current version.
+place.
+
+History table management follows the same principle:
+
+- **Admin credentials present, tables already up to date**: the admin connection
+  is not opened. No DDL is executed.
+- **Admin credentials present, DDL changes are required**: the admin connection
+  is opened and the changes are applied as before.
+- **Admin credentials absent, tables already up to date**: `ldb-run` proceeds
+  without an admin connection. This is the normal case for environments where a
+  DBA pre-creates all history objects and admin credentials are not stored.
+- **Admin credentials absent, history schema cannot be inspected** (for example
+  the runtime connection or dictionary query fails): `ldb-run` exits with code 1
+  and logs the cause. It does not run ILM against a history schema whose state is
+  unknown.
+- **Admin credentials absent, DDL changes are required**: `ldb-run` applies the
+  DDL directly over the runtime history connection, with no prior privilege
+  check. It succeeds when the runtime user owns the objects or holds the needed
+  privileges (for example `ALTER`). Any DDL statement that fails, for whatever
+  reason, is recorded and the remaining statements are still attempted. If at least one
+  statement failed, `ldb-run` writes the failed statements, each preceded by its
+  error, to `history_ddl_<schema>_<YYYYMMDD_HHMMSS>.sql` under `ddl_pending_dir`
+  (default `ldb_ddl_pending`), logs an error naming the file, and exits with
+  code 1. The file is plain SQL for SQL*Plus (no PL/SQL): it stops at the first
+  failing statement with a non-zero exit code, and only a run that reaches the
+  final `LimitsDb: OK` banner applied everything. The recorded errors are
+  comments. A DBA reviews and runs the file, then `ldb-run` is rerun. Nothing is
+  written to stdout, and the directory is created only when a file is written.
+  If the file cannot be written (for example the directory is not writable),
+  `ldb-run` logs the error with the failed statements and exits with code 1.
+
+### History DDL in `--mode SCRIPT`
+
+`--mode SCRIPT` applies nothing to the databases. The ILM SQL blocks, which
+run many times, go to stdout. The one-time statements go to files under
+`ddl_pending_dir`, so they stay out of the script that is rerun:
+
+- `history_ddl_<schema>_<YYYYMMDD_HHMMSS>.sql`: the history table DDL followed by
+  the grants on the history tables for the history role. Run it against the
+  history database.
+- `source_grants_<schema>_<YYYYMMDD_HHMMSS>.sql` (`SOURCE_ILM` only): the grants
+  on the source tables for the source role. Run it against the source database.
+
+Both are plain SQL for SQL*Plus that stop at the first failing statement; the
+grants are unconditional and safe to run again. The history DDL is always
+emitted, computed like this:
+
+- **A history connection works**: the admin connection is used when admin
+  credentials exist, the runtime one otherwise. The current history structure is
+  read, and the file holds only the missing changes.
+- **Neither works or there are no credentials**: `history_baseline` states what to
+  assume. `EMPTY` (default) assumes history has no objects, so every table, key
+  and index is created. `SOURCE` assumes history holds the same objects as
+  source, so only the history-specific differences are emitted (for example the
+  `LDB_PROCESS_DATE` column and the matching primary key).
+
+Without admin credentials, `ldb-run` applies the history grants right after the
+DDL over the runtime connection. Any grant the runtime user cannot apply goes to
+the same pending file as the failed DDL, so a DBA can run it.
 
 ### Configuration parameters reference
 
 | Parameter                | Purpose                                                                                                                                                         | Default / values                                                                     |
 | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
 | `action`                 | ILM target (`SOURCE_ILM` moves/purges source, `HISTORY_ILM` cleans downstream history).                                                                         | Default `SOURCE_ILM`; choices `SOURCE_ILM`, `HISTORY_ILM`.                           |
-| `mode`                   | Execution mode: `VALIDATE` config/credentials, `PLAN` dependency order, `PREVIEW` simulate (a.k.a. legacy `DRY_RUN`), `SCRIPT` emit SQL, `EXECUTE` run changes. | Default `PREVIEW`; choices `VALIDATE`, `PLAN`, `PREVIEW`, `SCRIPT`, `EXECUTE`.       |
+| `mode`                   | Execution mode: `VALIDATE` config/credentials, `PLAN` dependency order, `PREVIEW` run the real queries and apply DDL and grants but do not insert or delete data (a.k.a. legacy `DRY_RUN`), `SCRIPT` emit the history DDL and the ILM SQL without applying anything, `EXECUTE` run changes. | Default `PREVIEW`; choices `VALIDATE`, `PLAN`, `PREVIEW`, `SCRIPT`, `EXECUTE`.       |
 | `chunk_size`             | Rows per chunk when processing large tables.                                                                                                                    | Default `100000`.                                                                    |
 | `use_added_columns`      | Populate derived columns in history tables.                                                                                                                     | Default `True` (boolean toggle).                                                     |
 | `add_ldb_columns`        | Add LimitsDb execution-date columns in history tables.                                                                                                        | Default `True` (boolean toggle).                                                     |
 | `parallel_max`           | Maximum number of parallel processes.                                                                                                                           | Default `10`.                                                                        |
+| `ddl_pending_dir`        | Directory where `ldb-run` writes history DDL it could not apply without admin credentials, and the DDL and grants files of `--mode SCRIPT`.                                                                      | Default `ldb_ddl_pending`.                                                           |
+| `history_baseline`       | State assumed for the history tables by `--mode SCRIPT` when it cannot read them over the runtime history connection: `EMPTY` (no objects) or `SOURCE` (same objects as source). | Default `EMPTY`; choices `EMPTY`, `SOURCE`. |
 | `db_engine`              | Database engine.                                                                                                                                                | Default and only current choice: `"oracle"`.                                        |
 | `log_level`              | Logging level.                                                                                                                                                  | Default `"INFO"`; choices `"DEBUG"`, `"INFO"`, `"WARNING"`, `"ERROR"`, `"CRITICAL"`. |
 | `schema`                 | Configuration set name: identifies a system or application (e.g., `billing`). Not a database schema — it is the folder that groups all configuration files for one target. | Required on CLI; no persisted default. |
@@ -294,6 +425,8 @@ credentials in the current version.
 | `use_added_columns`      | —                          | —                            | `config[.<PROFILE>].yml: use_added_columns`        |
 | `add_ldb_columns`        | —                          | —                            | `config[.<PROFILE>].yml: add_ldb_columns`          |
 | `parallel_max`           | `--parallel-max`           | `LDB_PARALLEL_MAX`           | `config[.<PROFILE>].yml: parallel_max`             |
+| `ddl_pending_dir`        | `--ddl-pending-dir`        | `LDB_DDL_PENDING_DIR`        | `config[.<PROFILE>].yml: ddl_pending_dir`          |
+| `history_baseline`       | `--history-baseline`       | `LDB_HISTORY_BASELINE`       | `config[.<PROFILE>].yml: history_baseline`         |
 | `db_engine`              | `--db-engine`              | `LDB_DB_ENGINE`              | `config[.<PROFILE>].yml: db_engine`                |
 | `log_level`              | `--log-level`              | `LDB_LOG_LEVEL`              | `config[.<PROFILE>].yml: log_level`                |
 | `schema`                 | `--schema`                 | `LDB_SCHEMA`                 | CLI only (not stored).                             |
@@ -351,6 +484,8 @@ Every key is commented; the default value and short help appear inline. Example:
 # use_added_columns: True  # Populate derived columns in history tables
 # add_ldb_columns: True  # Add LimitsDb execution-date columns in history tables
 # parallel_max: 10      # Maximum number of parallel processes
+# ddl_pending_dir: "ldb_ddl_pending"  # Where failed history DDL is written for a DBA
+# history_baseline: "EMPTY"  # SCRIPT without readable history: EMPTY | SOURCE
 # db_engine: "oracle"   # Database engine
 # log_level: "INFO"     # Logging level
 ```

{limitsdb-0.5.0 → limitsdb-0.6.0}/limitsdb/cli/ldb_impl.py

@@ -3,6 +3,9 @@
 from __future__ import annotations
 
 import argparse
+from datetime import datetime
+from pathlib import Path
+from typing import Any
 
 from limitsdb.core.ldb_crypto import load_or_create_key
 from limitsdb.core.ldb_errors import LimitsDbError
@@ -33,6 +36,9 @@ def _parse_args() -> argparse.Namespace:
     parser.add_argument(
         "--set", action="append", default=[], help="Overrides like key=value; supports dotted keys for nesting"
     )
+    parser.add_argument(
+        "--ddl-pending-dir", help="Directory for the bootstrap scripts written when admin credentials are absent"
+    )
     parser.add_argument(
         "--log-level", default="INFO", choices=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"], help="Logging level"
     )
@@ -101,10 +107,6 @@ def run_cli() -> None:
         connections = config.connections
         administration = config.administration
 
-        if not administration.source.username or not administration.source.password:
-            raise ValueError("admin_source_username and admin_source_password are required")
-        if not administration.history.username or not administration.history.password:
-            raise ValueError("admin_history_username and admin_history_password are required")
         if not connections.source.username or not connections.source.password:
             raise ValueError("source_username and source_password are required")
         if not connections.history.username or not connections.history.password:
@@ -158,48 +160,100 @@ def run_cli() -> None:
     )
 
     summary: dict[str, list[str]] = {}
+    script_files: list[Path] = []
+    source_has_admin = bool(administration.source.username and administration.source.password)
+    history_has_admin = bool(administration.history.username and administration.history.password)
 
-    source_admin_conn = engine.get_connection(config, admin=True, env="SOURCE")
-    try:
-        summary["source_roles"] = engine.ensure_roles(source_admin_conn, source_roles)
-        summary["source_users"] = engine.ensure_users(source_admin_conn, [source_user])
-        summary["source_tables"] = engine.ensure_tables(source_admin_conn, source_tables)
-        summary["source_sequences"] = engine.ensure_sequences(source_admin_conn, source_sequences)
-        summary["source_supporting_objects"] = engine.ensure_supporting_objects(
-            source_admin_conn, connections.source.username
-        )
-    finally:
-        engine.close_connection(source_admin_conn)
+    def write_script(env: str, **definitions: Any) -> None:
+        content = engine.generate_bootstrap_script(title=f"{env} database of schema {config.schema}", **definitions)
+        stamp = datetime.now().strftime("%Y%m%d_%H%M%S")
+        path = Path(config.execution.ddl_pending_dir) / f"bootstrap_{config.schema}_{env.lower()}_{stamp}.sql"
+        try:
+            path.parent.mkdir(parents=True, exist_ok=True)
+            path.write_text(content, encoding="utf-8")
+        except OSError as exc:
+            logger.error(
+                "Admin credentials for %s are absent and the script could not be written to %s: %s", env, path, exc
+            )
+            raise SystemExit(1) from exc
+        script_files.append(path)
 
-    history_admin_conn = engine.get_connection(config, admin=True, env="HISTORY")
-    try:
-        summary["history_roles"] = engine.ensure_roles(history_admin_conn, history_roles)
-        summary["history_users"] = engine.ensure_users(history_admin_conn, [history_user])
-        summary["history_tables"] = engine.ensure_tables(history_admin_conn, history_tables)
-        summary["history_sequences"] = engine.ensure_sequences(history_admin_conn, history_sequences)
-        summary["history_supporting_objects"] = engine.ensure_supporting_objects(
-            history_admin_conn, connections.history.username
+    if not source_has_admin:
+        write_script(
+            "SOURCE",
+            roles=source_roles,
+            users=[source_user],
+            tables=source_tables,
+            sequences=source_sequences,
+            supporting_owner=connections.source.username,
+            links=[source_db_link],
+            runtime_username=connections.source.username,
+            runtime_dsn=connections.source.dsn,
+        )
+    if not history_has_admin:
+        write_script(
+            "HISTORY",
+            roles=history_roles,
+            users=[history_user],
+            tables=history_tables,
+            sequences=history_sequences,
+            supporting_owner=connections.history.username,
+            links=[history_db_link],
+            runtime_username=connections.history.username,
+            runtime_dsn=connections.history.dsn,
         )
-    finally:
-        engine.close_connection(history_admin_conn)
 
-    source_user_conn = engine.get_connection(config, admin=False, env="SOURCE")
-    try:
-        summary["source_database_links"] = engine.ensure_database_links(source_user_conn, [source_db_link])
-    finally:
-        engine.close_connection(source_user_conn)
+    if source_has_admin:
+        source_admin_conn = engine.get_connection(config, admin=True, env="SOURCE")
+        try:
+            summary["source_roles"] = engine.ensure_roles(source_admin_conn, source_roles)
+            summary["source_users"] = engine.ensure_users(source_admin_conn, [source_user])
+            summary["source_tables"] = engine.ensure_tables(source_admin_conn, source_tables)
+            summary["source_sequences"] = engine.ensure_sequences(source_admin_conn, source_sequences)
+            summary["source_supporting_objects"] = engine.ensure_supporting_objects(
+                source_admin_conn, connections.source.username
+            )
+        finally:
+            engine.close_connection(source_admin_conn)
 
-    history_user_conn = engine.get_connection(config, admin=False, env="HISTORY")
-    try:
-        summary["history_database_links"] = engine.ensure_database_links(history_user_conn, [history_db_link])
-    finally:
-        engine.close_connection(history_user_conn)
+    if history_has_admin:
+        history_admin_conn = engine.get_connection(config, admin=True, env="HISTORY")
+        try:
+            summary["history_roles"] = engine.ensure_roles(history_admin_conn, history_roles)
+            summary["history_users"] = engine.ensure_users(history_admin_conn, [history_user])
+            summary["history_tables"] = engine.ensure_tables(history_admin_conn, history_tables)
+            summary["history_sequences"] = engine.ensure_sequences(history_admin_conn, history_sequences)
+            summary["history_supporting_objects"] = engine.ensure_supporting_objects(
+                history_admin_conn, connections.history.username
+            )
+        finally:
+            engine.close_connection(history_admin_conn)
+
+    if source_has_admin:
+        source_user_conn = engine.get_connection(config, admin=False, env="SOURCE")
+        try:
+            summary["source_database_links"] = engine.ensure_database_links(source_user_conn, [source_db_link])
+        finally:
+            engine.close_connection(source_user_conn)
+
+    if history_has_admin:
+        history_user_conn = engine.get_connection(config, admin=False, env="HISTORY")
+        try:
+            summary["history_database_links"] = engine.ensure_database_links(history_user_conn, [history_db_link])
+        finally:
+            engine.close_connection(history_user_conn)
 
     anything_created = any(summary.values())
     for key, values in summary.items():
         if values:
             logger.info("Created %s: %s", key.replace("_", " "), ", ".join(values))
-    if not anything_created:
+    for script_file in script_files:
+        logger.warning(
+            "Admin credentials are absent for this database, so nothing was created there. "
+            "Ask a DBA to run %s in SQL*Plus.",
+            script_file,
+        )
+    if not anything_created and not script_files:
         logger.info("All LimitsDb database objects are already present.")