ligoj-cli 1.0.2__tar.gz → 1.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/PKG-INFO +985 -29
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/README.md +984 -28
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligoj_cli.egg-info/PKG-INFO +985 -29
- ligoj_cli-1.2.0/ligoj_cli.egg-info/SOURCES.txt +47 -0
- ligoj_cli-1.2.0/ligojcli/assets/logo.png +0 -0
- ligoj_cli-1.2.0/ligojcli/dev_backup.py +1015 -0
- ligoj_cli-1.2.0/ligojcli/dev_debug.py +692 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/__init__.py +152 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/_common.py +49 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/_seed.py +285 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/_subscribe.py +291 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/plugin_build_jenkins.py +41 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/plugin_id_ldap.py +147 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/plugin_prov_aws.py +38 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/plugin_prov_azure.py +48 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/plugin_qa_sonarqube.py +106 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/plugin_registry_artifactory.py +56 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/plugin_registry_harbor.py +42 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/plugin_registry_nexus.py +46 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/plugin_scm_github.py +65 -0
- ligoj_cli-1.2.0/ligojcli/dev_demo/plugin_scm_gitlab.py +46 -0
- ligoj_cli-1.2.0/ligojcli/dev_package.py +266 -0
- ligoj_cli-1.2.0/ligojcli/dev_plugin.py +1301 -0
- ligoj_cli-1.2.0/ligojcli/dev_test.py +544 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/ligoj.py +20 -4
- ligoj_cli-1.2.0/ligojcli/plugins/build.py +103 -0
- ligoj_cli-1.2.0/ligojcli/plugins/dev.py +3341 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/ligoj.py +0 -186
- ligoj_cli-1.2.0/ligojcli/plugins/prov.py +688 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/utils.py +11 -2
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/pyproject.toml +5 -2
- ligoj_cli-1.0.2/ligoj_cli.egg-info/SOURCES.txt +0 -24
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/LICENSE +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligoj_cli.egg-info/dependency_links.txt +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligoj_cli.egg-info/entry_points.txt +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligoj_cli.egg-info/requires.txt +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligoj_cli.egg-info/top_level.txt +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/__init__.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/__init__.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/alfresco.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/argocd.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/bootstrap.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/gitlab.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/harbor.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/id.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/jenkins.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/nexus.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/sonarqube.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/ligojcli/plugins/ssl.py +0 -0
- {ligoj_cli-1.0.2 → ligoj_cli-1.2.0}/setup.cfg +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: ligoj-cli
|
|
3
|
-
Version: 1.0
|
|
3
|
+
Version: 1.2.0
|
|
4
4
|
Summary: Ligoj CLI
|
|
5
5
|
Author-email: Fabrice Daugan <fdaugan@kloudy.io>
|
|
6
6
|
License-Expression: MIT
|
|
@@ -38,10 +38,36 @@ Ligoj CLI makes REST calls to a remote Ligoj instance, with parameters and error
|
|
|
38
38
|
# Requirements
|
|
39
39
|
|
|
40
40
|
- Python 3.11+
|
|
41
|
-
- Connectivity and API keys to target endpoints `Ligoj
|
|
42
|
-
- `pip`
|
|
41
|
+
- Connectivity and API keys to the target endpoints: `Ligoj` (required) and, for `bootstrap` actions, `Nexus`, `Jenkins`, `SonarQube`
|
|
43
42
|
- Valid [credentials](#credentials)
|
|
44
43
|
|
|
44
|
+
# Installation
|
|
45
|
+
|
|
46
|
+
Ligoj CLI is published on [PyPI](https://pypi.org/project/ligoj-cli/) as `ligoj-cli` and provides the `ligoj` command.
|
|
47
|
+
|
|
48
|
+
Install it as an isolated tool with [`uv`](https://docs.astral.sh/uv/) (recommended):
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
uv tool install ligoj-cli
|
|
52
|
+
ligoj --version
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Alternatively, use `pipx` or `pip`:
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
pipx install ligoj-cli
|
|
59
|
+
# or
|
|
60
|
+
pip install ligoj-cli
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
To try the latest pre-release published on [TestPyPI](https://test.pypi.org/project/ligoj-cli/):
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ ligoj-cli
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
For local development from a checkout, see [Development](#development).
|
|
70
|
+
|
|
45
71
|
|
|
46
72
|
# Configuration
|
|
47
73
|
|
|
@@ -86,7 +112,7 @@ Options are sourced in the following order of priority, from highest to lowest:
|
|
|
86
112
|
|
|
87
113
|
### Configuration files
|
|
88
114
|
|
|
89
|
-
Sections in these `.ini` files correspond to profile names.
|
|
115
|
+
Sections in these `.ini` files correspond to profile names. When neither the `--profile` option nor the `LIGOJ_PROFILE` environment variable is provided, the default profile is `default` — except for the [`dev`](#dev-environment) command, which defaults to the `dev` profile. See [Profile](#profile) for the full resolution order.
|
|
90
116
|
|
|
91
117
|
In the file `~/.ligoj/config`, default configurations can be specified. No secrets are sourced from this file.
|
|
92
118
|
|
|
@@ -230,12 +256,26 @@ ligoj --api-local-roles session get
|
|
|
230
256
|
|
|
231
257
|
### Profile
|
|
232
258
|
|
|
233
|
-
|
|
259
|
+
A **profile** is a named section shared across the three `.ini` files under `~/.ligoj/` ([`config`, `credentials`, `sessions`](#configuration-files)): the settings, secrets, and session for a given profile all live under the same `[<profile>]` section. Profiles let you keep several isolated sets of endpoints and credentials — for example one per Ligoj instance, or one for the local [dev environment](#dev-environment) — and switch between them without editing files.
|
|
260
|
+
|
|
261
|
+
Select the profile with the `--profile` option or the `LIGOJ_PROFILE` environment variable:
|
|
234
262
|
|
|
235
263
|
```bash
|
|
236
|
-
ligoj --profile some
|
|
264
|
+
ligoj --profile some node list
|
|
265
|
+
# or
|
|
266
|
+
LIGOJ_PROFILE=some ligoj node list
|
|
237
267
|
```
|
|
238
268
|
|
|
269
|
+
The profile name is resolved in the following order (first match wins):
|
|
270
|
+
|
|
271
|
+
1. The `--profile` command-line option.
|
|
272
|
+
2. The `LIGOJ_PROFILE` environment variable.
|
|
273
|
+
3. The default profile, which depends on the command:
|
|
274
|
+
- **`default`** for every command,
|
|
275
|
+
- **`dev`** for the [`dev`](#dev-environment) command — its local development stack reads and writes the `[dev]` section populated by `dev init`.
|
|
276
|
+
|
|
277
|
+
For example, `ligoj dev demo` uses the `[dev]` profile automatically, while `ligoj node list` uses `[default]`; either can still be overridden with `--profile` or `LIGOJ_PROFILE`.
|
|
278
|
+
|
|
239
279
|
### From
|
|
240
280
|
|
|
241
281
|
JSON content to load. Use the `--from` option. The following forms are available:
|
|
@@ -1651,6 +1691,893 @@ For current user:
|
|
|
1651
1691
|
ligoj id:user reset-password
|
|
1652
1692
|
```
|
|
1653
1693
|
|
|
1694
|
+
# Plugin prov
|
|
1695
|
+
|
|
1696
|
+
Operations related to [plugin-prov](https://github.com/ligoj/plugin-prov) and its provider
|
|
1697
|
+
sub-plugins (AWS, Azure, GCP, …), exposed under `prov:<resource>` services. They drive a
|
|
1698
|
+
*provisioning quote* identified by a **subscription** id (`--subscription`/`-s`), the project's
|
|
1699
|
+
subscription to a `service:prov:*` node.
|
|
1700
|
+
|
|
1701
|
+
| Service | Actions | REST base |
|
|
1702
|
+
| ---------------- | ------------------------------------------------ | ---------------------------------- |
|
|
1703
|
+
| `prov:quote` | `get`, `update`, `refresh`, `refresh-cost`, `locations` | `service/prov/{subscription}` |
|
|
1704
|
+
| `prov:instance` | `lookup`, `create`, `update`, `delete`, `delete-all` | `service/prov/instance` |
|
|
1705
|
+
| `prov:container` | `lookup`, `create`, `update`, `delete`, `delete-all` | `service/prov/container` |
|
|
1706
|
+
| `prov:database` | `lookup`, `create`, `update`, `delete`, `delete-all` | `service/prov/database` |
|
|
1707
|
+
| `prov:function` | `lookup`, `create`, `update`, `delete`, `delete-all` | `service/prov/function` |
|
|
1708
|
+
| `prov:storage` | `lookup`, `create`, `update`, `delete`, `delete-all` | `service/prov/storage` |
|
|
1709
|
+
| `prov:support` | `lookup`, `create`, `update`, `delete`, `delete-all` | `service/prov/support` |
|
|
1710
|
+
| `prov:usage` | `list`, `create`, `update`, `delete` | `service/prov/{subscription}/usage` |
|
|
1711
|
+
| `prov:budget` | `list`, `create`, `update`, `delete` | `service/prov/{subscription}/budget` |
|
|
1712
|
+
| `prov:optimizer` | `list`, `create`, `update`, `delete` | `service/prov/{subscription}/optimizer` |
|
|
1713
|
+
| `prov:tag` | `create`, `update`, `delete` | `service/prov/{subscription}/tag` |
|
|
1714
|
+
| `prov:catalog` | `list`, `status`, `update`, `cancel` | `service/prov/catalog` |
|
|
1715
|
+
| `prov:upload` | `resources` | `service/prov/{subscription}/upload` |
|
|
1716
|
+
|
|
1717
|
+
Run `ligoj prov:<service> --help` (and `... <action> --help`) for the full argument list. Every
|
|
1718
|
+
`create`/`update` also accepts `--data '<json>'` to set or override any backend field not exposed
|
|
1719
|
+
as a flag.
|
|
1720
|
+
|
|
1721
|
+
|
|
1722
|
+
## Quote configuration (`prov:quote`)
|
|
1723
|
+
|
|
1724
|
+
```bash
|
|
1725
|
+
# Inspect the full quote (resources + total cost)
|
|
1726
|
+
ligoj prov:quote get --subscription 12
|
|
1727
|
+
|
|
1728
|
+
# Set defaults and recompute
|
|
1729
|
+
ligoj prov:quote update -s 12 --location "eu-west-1" --usage "dev" --license "BYOL"
|
|
1730
|
+
ligoj prov:quote refresh -s 12
|
|
1731
|
+
```
|
|
1732
|
+
|
|
1733
|
+
|
|
1734
|
+
## Compute, database, container, function
|
|
1735
|
+
|
|
1736
|
+
The typical flow is *lookup a price → create the resource with that price id*:
|
|
1737
|
+
|
|
1738
|
+
```bash
|
|
1739
|
+
# 1. find the cheapest instance matching the requirement
|
|
1740
|
+
ligoj prov:instance lookup -s 12 --cpu 2 --ram 4096 --os LINUX
|
|
1741
|
+
|
|
1742
|
+
# 2. create it from the returned price id
|
|
1743
|
+
ligoj prov:instance create -s 12 --name "web" --price 4211 --cpu 2 --ram 4096 \
|
|
1744
|
+
--os LINUX --internet PUBLIC --min-quantity 1 --max-quantity 3
|
|
1745
|
+
|
|
1746
|
+
# database / container / function follow the same pattern
|
|
1747
|
+
ligoj prov:database create -s 12 --name "db" --price 5120 --cpu 1 --ram 2048 --engine MYSQL
|
|
1748
|
+
ligoj prov:function create -s 12 --name "fn" --price 77 --runtime Python --nb-requests 5
|
|
1749
|
+
|
|
1750
|
+
# update / delete
|
|
1751
|
+
ligoj prov:instance update -s 12 --id 99 --max-quantity 5
|
|
1752
|
+
ligoj prov:instance delete --id 99
|
|
1753
|
+
ligoj prov:instance delete-all -s 12
|
|
1754
|
+
```
|
|
1755
|
+
|
|
1756
|
+
|
|
1757
|
+
## Storage, usage, budget, optimizer, tag
|
|
1758
|
+
|
|
1759
|
+
```bash
|
|
1760
|
+
ligoj prov:storage create -s 12 --name "data" --type "gp2" --size 100 --instance 99
|
|
1761
|
+
ligoj prov:usage create -s 12 --name "dev" --rate 50 --duration 12
|
|
1762
|
+
ligoj prov:budget create -s 12 --name "2026" --initial-cost 10000
|
|
1763
|
+
ligoj prov:tag create -s 12 --name "env" --value "prod" --type INSTANCE --resource 99
|
|
1764
|
+
```
|
|
1765
|
+
|
|
1766
|
+
|
|
1767
|
+
## Catalog and bulk upload
|
|
1768
|
+
|
|
1769
|
+
```bash
|
|
1770
|
+
# trigger and follow a provider catalog import
|
|
1771
|
+
ligoj prov:catalog update --node "service:prov:aws:test" --force
|
|
1772
|
+
ligoj prov:catalog status --node "service:prov:aws:test"
|
|
1773
|
+
|
|
1774
|
+
# bulk-create resources from a CSV
|
|
1775
|
+
ligoj prov:upload resources -s 12 --from ./resources.csv --merge update
|
|
1776
|
+
```
|
|
1777
|
+
|
|
1778
|
+
# Plugin build
|
|
1779
|
+
|
|
1780
|
+
Operations related to [plugin-build](https://github.com/ligoj/plugin-build) and its CI provider
|
|
1781
|
+
sub-plugins (Jenkins, Travis), exposed under the `build:job` service. The build provider is taken
|
|
1782
|
+
from `--provider`, inferred from the `--node` identifier (`service:build:<provider>:…`), or resolved
|
|
1783
|
+
from the subscription's node.
|
|
1784
|
+
|
|
1785
|
+
> Note: this drives the **Ligoj** build service (`service/build/<provider>`). The separate `jenkins`
|
|
1786
|
+
> service talks **directly** to a Jenkins server instead.
|
|
1787
|
+
|
|
1788
|
+
| Action | Arguments | REST |
|
|
1789
|
+
| ----------- | ---------------------------------- | -------------------------------------------------- |
|
|
1790
|
+
| `trigger` | `--subscription` [`--provider`] | `POST service/build/<provider>/build/{subscription}` |
|
|
1791
|
+
| `find` | `--node --criteria` [`--provider`] | `GET service/build/<provider>/{node}/{criteria}` |
|
|
1792
|
+
| `templates` | `--node --criteria` [`--provider`] | `GET service/build/<provider>/template/{node}/{criteria}` (Jenkins) |
|
|
1793
|
+
| `get` | `--node --id` [`--provider`] | `GET service/build/<provider>/{node}/job/{id}` |
|
|
1794
|
+
|
|
1795
|
+
```bash
|
|
1796
|
+
# Trigger the build configured for a subscription (provider inferred from the subscription)
|
|
1797
|
+
ligoj build:job trigger --subscription 42
|
|
1798
|
+
|
|
1799
|
+
# Search jobs / templates on a node (provider inferred from the node)
|
|
1800
|
+
ligoj build:job find --node "service:build:jenkins:dev" --criteria "my-app"
|
|
1801
|
+
ligoj build:job templates --node "service:build:jenkins:dev" --criteria "template"
|
|
1802
|
+
|
|
1803
|
+
# Return a single job by id
|
|
1804
|
+
ligoj build:job get --node "service:build:jenkins:dev" --id "my-app"
|
|
1805
|
+
```
|
|
1806
|
+
|
|
1807
|
+
# Dev environment
|
|
1808
|
+
|
|
1809
|
+
`dev init` brings up, on **Kubernetes**, the backing services a Ligoj developer needs and wires the
|
|
1810
|
+
resulting endpoints and credentials into the **`[dev]` section** of `~/.ligoj/credentials`. Every
|
|
1811
|
+
`dev` command uses the `dev` [profile](#profile) by default (no `--profile` needed), and the
|
|
1812
|
+
generated `[dev]` profile can be reused by any other command with `--profile dev`.
|
|
1813
|
+
|
|
1814
|
+
> Unlike every other command, `dev` is purely local and does **not** require `LIGOJ_ENDPOINT`.
|
|
1815
|
+
|
|
1816
|
+
Two runtimes, both driven by **podman** — no heavyweight cluster unless you ask for Harbor:
|
|
1817
|
+
|
|
1818
|
+
- **`podman kube play`** for every self-contained service. Each becomes a `Pod` (manifest written to
|
|
1819
|
+
`~/.ligoj/dev/k8s/<service>.yaml`); `PersistentVolumeClaim`s map to podman named volumes so data
|
|
1820
|
+
survives restarts. A pre-existing raw container of the same name is migrated to a pod automatically.
|
|
1821
|
+
- **a `kind` cluster** (podman provider, `ligoj-dev`) created on demand **only for Harbor**, which
|
|
1822
|
+
needs a real cluster; Harbor is installed with its Helm chart and exposed on `localhost` via a
|
|
1823
|
+
NodePort + kind `extraPortMappings`.
|
|
1824
|
+
|
|
1825
|
+
The init phase **bootstraps its own tooling**: a missing CLI (`podman`, and `kind`/`helm`/`kubectl`
|
|
1826
|
+
for Harbor) is installed with Homebrew, and the podman machine is initialized/started as needed — so
|
|
1827
|
+
a fresh machine can go from nothing to a running stack with one command. (If Homebrew is absent, you
|
|
1828
|
+
get a clear message to install the tool yourself.) The whole stack (GitLab + SonarQube + Nexus +
|
|
1829
|
+
Artifactory + kind at once) is heavy, so `dev init` also **enforces the podman machine's resources**:
|
|
1830
|
+
if it has fewer than **6 vCPU** or **23 GB RAM** it is **stopped, resized up to that minimum, and
|
|
1831
|
+
restarted** (a fresh machine is created already sized; a machine that already exceeds the minimum is
|
|
1832
|
+
left untouched). On macOS it also checks the tools `dev demo` needs
|
|
1833
|
+
later — **Java 21** (installed as the Temurin JDK cask via Homebrew) and **Maven 3.9.6** (installed
|
|
1834
|
+
via [SDKMAN](https://sdkman.io), bootstrapped if missing); these two are best-effort and never abort
|
|
1835
|
+
`init`. Pass `--skip-prereqs` to skip all these checks. Each service streams live progress and is
|
|
1836
|
+
**idempotent** — a running pod is reused; use `--recreate` to replace it.
|
|
1837
|
+
|
|
1838
|
+
| Service | Pod / release | Default port | Image | What `dev init` configures |
|
|
1839
|
+
| ------------ | ------------- | ------------ | ------------------------------------- | -------------------------- |
|
|
1840
|
+
| `postgresql` | `ligoj-db` | `5432` | `postgres:17` | `ligoj/ligoj` user/db (what `ligoj-api` expects), persistent volume `ligoj_db_data` |
|
|
1841
|
+
| `openldap` | `openldap` | `1389` | `bitnamilegacy/openldap:latest` | `Manager` / `dc=sample,dc=com`; generates the admin password if missing; volume `openldap_data` |
|
|
1842
|
+
| `keycloak` | `keycloak` | `9083` | `quay.io/keycloak/keycloak:26.6.1` | **backed by the shared `postgresql`** (dedicated `keycloak` database); realm `ligoj`, LDAP user federation, confidential `ligoj` client; prints Spring Boot properties |
|
|
1843
|
+
| `jenkins` | `jenkins` | `8085` | `jenkins/jenkins:2.570-slim-jdk25` | volume `jenkins_home`; provisions the admin user and generates an API token |
|
|
1844
|
+
| `sonarqube` | `sonarqube` | `9000` | `sonarqube:26.6.0.123539-community` | **backed by the shared `postgresql`** (dedicated `sonarqube` database); changes the default admin password and creates an API token |
|
|
1845
|
+
| `gitlab` | `gitlab` | `8929` (+ssh `2289`) | `gitlab/gitlab-ce:latest` | omnibus CE (single container), trimmed footprint; root password in `[dev]` |
|
|
1846
|
+
| `harbor` | `harbor` (Helm, on `kind`) | `8088` | `goharbor/harbor` chart | minimal Harbor (no trivy/metrics); admin password in `[dev]` |
|
|
1847
|
+
| `nexus` | `nexus` | `8181` | `sonatype/nexus3:latest` | volume `nexus_data`; resets the generated initial admin password and stores it in `[dev]` (host `8181` leaves `8081` free for the Ligoj API) |
|
|
1848
|
+
| `artifactory`| `artifactory` | `8082` | `jfrog/artifactory-oss:7.111.9` (pinned) | **backed by the shared `postgresql`** (dedicated `artifactory` database, Derby is refused), volume `artifactory_data`; admin `password` in `[dev]`. Forces IPv4, **self-heals a hung boot** (see below), and is **pinned below 7.125** to keep the web UI usable (see below) |
|
|
1849
|
+
| `argocd` | `argocd` (Helm, on `kind`) | `8083` | `argo/argo-cd` chart | `role:ligoj` RBAC + `ligoj` account & API token; Dex **LDAP federation** to OpenLDAP |
|
|
1850
|
+
|
|
1851
|
+
## Options
|
|
1852
|
+
|
|
1853
|
+
| Option | Description |
|
|
1854
|
+
| ------------- | --------------------------------------------------------------------------- |
|
|
1855
|
+
| `--only`, `-O` | Limit to a subset, e.g. `--only postgresql keycloak` (default: all) |
|
|
1856
|
+
| `--recreate`, `-R` | Delete and recreate the pods / kind cluster (named volumes are kept) |
|
|
1857
|
+
| `--wait`, `-w` | How long to wait for readiness, with **live progress**: omit = wait until done or Ctrl+C; `0` = no wait (skip readiness/token steps); `N` = up to N seconds. Also accepted by `start` / `stop` / `restart`. |
|
|
1858
|
+
| `--ldap-port` / `--jenkins-port` / `--sonar-port` / `--db-port` / `--keycloak-port` / `--gitlab-port` / `--harbor-port` / `--nexus-port` / `--artifactory-port` / `--argocd-port` | Override a host port |
|
|
1859
|
+
|
|
1860
|
+
Every value can also be set in the `[dev]` section or as an environment variable. The credentials
|
|
1861
|
+
are **read** before a secret is generated, so you stay in control:
|
|
1862
|
+
|
|
1863
|
+
| Service | Inputs (option · env · `[dev]` key) |
|
|
1864
|
+
| ------------ | -------------------------------------------------------------------------------------------------- |
|
|
1865
|
+
| `postgresql` | `DB_IMAGE`·`db_image`, `DB_PORT`·`db_port`, `POSTGRES_USER`·`db_user`, `POSTGRES_PASSWORD`·`db_password`, `POSTGRES_DB`·`db_name` |
|
|
1866
|
+
| `openldap` | `LDAP_IMAGE`·`ldap_image`, `LDAP_PORT`·`ldap_port`, `LDAP_ADMIN_USERNAME`·`ldap_admin_user`, `LDAP_ROOT`·`ldap_root`, `LDAP_ADMIN_PASSWORD`·`ldap_admin_password`, `LDAP_SCHEMA_DIR`·`ldap_schema_dir` |
|
|
1867
|
+
| `keycloak` | `KEYCLOAK_IMAGE`·`keycloak_image`, `KEYCLOAK_PORT`·`keycloak_port`, `KC_BOOTSTRAP_ADMIN_USERNAME`·`keycloak_admin_user`, `KC_BOOTSTRAP_ADMIN_PASSWORD`·`keycloak_admin_password`, `KEYCLOAK_LDAP_URL`·`keycloak_ldap_url`, `KEYCLOAK_DB_PASSWORD`·`keycloak_db_password` (shared-DB role) |
|
|
1868
|
+
| `jenkins` | `JENKINS_IMAGE`·`jenkins_image`, `JENKINS_PORT`·`jenkins_port`, `JENKINS_API_USER`·`jenkins_api_user`, `JENKINS_ADMIN_PASSWORD`·`jenkins_admin_password`, `JENKINS_API_TOKEN`·`jenkins_api_token` |
|
|
1869
|
+
| `sonarqube` | `SONAR_IMAGE`·`sonar_image`, `SONAR_PORT`·`sonar_port`, `SONAR_ADMIN_PASSWORD`·`sonar_admin_password`, `SONAR_DB_PASSWORD`·`sonar_db_password` (shared-DB role) |
|
|
1870
|
+
| `gitlab` | `GITLAB_IMAGE`·`gitlab_image`, `GITLAB_PORT`·`gitlab_port`, `GITLAB_SSH_PORT`·`gitlab_ssh_port`, `GITLAB_ROOT_PASSWORD`·`gitlab_root_password` |
|
|
1871
|
+
| `harbor` | `HARBOR_PORT`·`harbor_port`, `HARBOR_NODE_PORT`·`harbor_node_port`, `HARBOR_ADMIN_PASSWORD`·`harbor_admin_password`, `HARBOR_REDIS_IMAGE`·`harbor_redis_image` |
|
|
1872
|
+
| `nexus` | `NEXUS_IMAGE`·`nexus_image`, `NEXUS_PORT`·`nexus_port`, `NEXUS_ADMIN_PASSWORD`·`nexus_admin_password` |
|
|
1873
|
+
| `artifactory`| `ARTIFACTORY_IMAGE`·`artifactory_image`, `ARTIFACTORY_PORT`·`artifactory_port`, `ARTIFACTORY_USER`·`artifactory_user`, `ARTIFACTORY_PASSWORD`·`artifactory_password`, `ARTIFACTORY_DB_PASSWORD`·`artifactory_db_password` (shared-DB role), `ARTIFACTORY_HEAL_AFTER`·`artifactory_heal_after` (boot self-heal grace, default `90`, `0` disables) |
|
|
1874
|
+
| `argocd` | `ARGOCD_PORT`·`argocd_port`, `ARGOCD_NODE_PORT`·`argocd_node_port` (LDAP fields reuse the `openldap` keys) |
|
|
1875
|
+
|
|
1876
|
+
In return, `dev init` **writes** to `[dev]`: `db_*` (host/port/name/user/password/url), `ldap_url` and
|
|
1877
|
+
`ldap_admin_password`, `keycloak_endpoint` / `keycloak_admin_password` / `keycloak_client_secret` /
|
|
1878
|
+
`keycloak_issuer_uri` / `keycloak_db_password`, `jenkins_endpoint` / `jenkins_admin_password` / `jenkins_api_token`,
|
|
1879
|
+
`sonar_endpoint` / `sonar_admin_password` / `sonar_api_token` / `sonar_db_password`, `gitlab_endpoint` /
|
|
1880
|
+
`gitlab_root_password` / `gitlab_token`, `harbor_endpoint` / `harbor_admin_password`,
|
|
1881
|
+
`nexus_endpoint` / `nexus_admin_password`, `artifactory_endpoint` / `artifactory_password` /
|
|
1882
|
+
`artifactory_db_password`, and
|
|
1883
|
+
`argocd_endpoint` / `argocd_admin_password` / `argocd_account` / `argocd_api_token`. **Keycloak**,
|
|
1884
|
+
**SonarQube** and **Artifactory** each get a dedicated role + database in the shared `postgresql`
|
|
1885
|
+
(reached from their pods via `host.containers.internal`). For **Jenkins**,
|
|
1886
|
+
**SonarQube**, **GitLab** and **ArgoCD** an API token is generated (and reused on later runs), so
|
|
1887
|
+
`--profile dev` can drive their APIs straight away. The **GitLab** token is a `root` personal access
|
|
1888
|
+
token minted via the Rails console (`api`, `read_api`, `read_repository`, `write_repository` scopes).
|
|
1889
|
+
|
|
1890
|
+
```bash
|
|
1891
|
+
# Bring up the whole local stack (Harbor pulls in a kind cluster)
|
|
1892
|
+
ligoj dev init
|
|
1893
|
+
|
|
1894
|
+
# Only the database and Keycloak, on custom ports
|
|
1895
|
+
ligoj dev init --only postgresql keycloak --db-port 5432 --keycloak-port 9083
|
|
1896
|
+
|
|
1897
|
+
# Recreate Jenkins to apply a freshly configured admin token
|
|
1898
|
+
JENKINS_API_USER=admin JENKINS_API_TOKEN="$(cat token.txt)" ligoj dev init --only jenkins --recreate
|
|
1899
|
+
|
|
1900
|
+
# Reuse the generated credentials in subsequent commands
|
|
1901
|
+
ligoj --profile dev sonar project list
|
|
1902
|
+
```
|
|
1903
|
+
|
|
1904
|
+
> **Footprint**: GitLab (~4 GB) and Harbor (a kind node + ~7 pods) are heavy; running all services at
|
|
1905
|
+
> once needs a roomy podman machine. Use `--only` to bring up just what you need.
|
|
1906
|
+
|
|
1907
|
+
> **Data safety**: for `postgresql`, an existing `ligoj-db` keeps its current data source (named
|
|
1908
|
+
> volume *or* host bind mount) and image **even on `--recreate`**, so a `--recreate` never orphans
|
|
1909
|
+
> the database nor swaps the PostgreSQL major version under an existing data directory.
|
|
1910
|
+
|
|
1911
|
+
## Status
|
|
1912
|
+
|
|
1913
|
+
`dev status` reports, for every service, its runtime state, a health probe **from the host** and the
|
|
1914
|
+
URL to reach it — handy after an `init` or to see what is still up:
|
|
1915
|
+
|
|
1916
|
+
```text
|
|
1917
|
+
SERVICE STATUS HEALTH URL
|
|
1918
|
+
---------- -------------- ------ ---------------------------------------
|
|
1919
|
+
postgresql running OK postgresql://ligoj@localhost:5432/ligoj
|
|
1920
|
+
openldap running OK ldap://localhost:1389
|
|
1921
|
+
keycloak running OK http://localhost:9083
|
|
1922
|
+
jenkins running OK http://localhost:8085
|
|
1923
|
+
sonarqube running OK http://localhost:9000
|
|
1924
|
+
gitlab running OK http://localhost:8929
|
|
1925
|
+
harbor running (kind) OK http://localhost:8088
|
|
1926
|
+
nexus running OK http://localhost:8181
|
|
1927
|
+
artifactory running OK http://localhost:8082/artifactory
|
|
1928
|
+
```
|
|
1929
|
+
|
|
1930
|
+
`STATUS` is the pod (or, for Harbor, the kind cluster) state — `running` / `stopped` / `absent`;
|
|
1931
|
+
`HEALTH` is probed from your machine (an HTTP check for web services, a TCP connect for
|
|
1932
|
+
PostgreSQL/LDAP). It reads the ports/URLs recorded in `[dev]`, so it works without contacting podman.
|
|
1933
|
+
|
|
1934
|
+
### Artifactory boot self-heal
|
|
1935
|
+
|
|
1936
|
+
Artifactory's JFrog microservice mesh occasionally deadlocks on boot: the internal services fail to
|
|
1937
|
+
join over `localhost` (they try the IPv6 `::1` loopback and get *connection refused*), so the
|
|
1938
|
+
container stays up but its router never binds `:8082` — the port `dev` and the Ligoj node probe — and
|
|
1939
|
+
the readiness wait would otherwise hang forever. Two mitigations are built in: the pod is started
|
|
1940
|
+
with `-Djava.net.preferIPv4Stack=true`, and if the router is still not listening after a grace period
|
|
1941
|
+
(default **90 s**, set `ARTIFACTORY_HEAL_AFTER` seconds, `0` disables) the wait **restarts the pod
|
|
1942
|
+
once** — a fresh boot usually wins the race (~40 s). A healthy-but-slow boot (which answers `503`
|
|
1943
|
+
while starting) is never restarted.
|
|
1944
|
+
|
|
1945
|
+
### Artifactory version pin (usable UI)
|
|
1946
|
+
|
|
1947
|
+
The image is pinned to `artifactory-oss:7.111.9`, **not** `:latest`, on purpose. From ~7.125 the OSS
|
|
1948
|
+
web UI's frontend service (`jffe`) is stuck in an unbounded retry loop on a *"first-time entitlement
|
|
1949
|
+
fetch"* — a licensing/entitlements gRPC call that returns **404 UNIMPLEMENTED** because that service
|
|
1950
|
+
ships only with the commercial JFrog Platform, not OSS. Until it "succeeds" (it never does) `jffe`
|
|
1951
|
+
won't serve UI data, so **every** `/ui/api/v1/ui/*` call hangs ~12 s and the UI is effectively
|
|
1952
|
+
unusable (long JFrog splash, every screen crawls). The REST API and all `dev demo` Maven operations
|
|
1953
|
+
are unaffected — only the browser UI. Tested across versions: 7.146/7.133/7.125 all hang; **7.111.9**
|
|
1954
|
+
is the newest OSS tag whose UI answers in milliseconds with no entitlement loop. (The many `404`s the
|
|
1955
|
+
browser logs for `xray`, `mc`, `distribution`, `apptrust`, … microfrontends are unrelated and
|
|
1956
|
+
harmless — those are commercial modules absent from OSS.) Override the pin with `ARTIFACTORY_IMAGE` /
|
|
1957
|
+
`[dev] artifactory_image` if you need a specific version; a **downgrade needs a fresh volume and
|
|
1958
|
+
database** (`podman pod rm -f artifactory`, `podman volume rm artifactory_data`, and drop the
|
|
1959
|
+
`artifactory` DB), because a newer schema/master-key won't start on an older binary.
|
|
1960
|
+
|
|
1961
|
+
## Per-service config
|
|
1962
|
+
|
|
1963
|
+
`dev config <service>` prints the key properties of a single service — URL, admin user and password,
|
|
1964
|
+
and the other connection details — read straight from `[dev]`:
|
|
1965
|
+
|
|
1966
|
+
```text
|
|
1967
|
+
# sonarqube
|
|
1968
|
+
url http://localhost:9000
|
|
1969
|
+
admin user admin
|
|
1970
|
+
admin password v1JE…
|
|
1971
|
+
api token squ_…
|
|
1972
|
+
```
|
|
1973
|
+
|
|
1974
|
+
Works for `postgresql`, `openldap`, `keycloak`, `jenkins`, `sonarqube`, `gitlab`, `harbor`, `nexus`,
|
|
1975
|
+
`artifactory` and `argocd`. Each
|
|
1976
|
+
service adds its own relevant fields — e.g. PostgreSQL the host/port/database, OpenLDAP the bind/base
|
|
1977
|
+
DN, Keycloak the realm + issuer URI + client id/secret, GitLab the SSH URL, Harbor the registry host.
|
|
1978
|
+
|
|
1979
|
+
Omit the service to get a summary **table** of every service at once:
|
|
1980
|
+
|
|
1981
|
+
```text
|
|
1982
|
+
SERVICE URL USER PASSWORD TOKEN/SECRET
|
|
1983
|
+
---------- --------------------------------------- ------- -------- ------------
|
|
1984
|
+
postgresql postgresql://ligoj@localhost:5432/ligoj ligoj ligoj -
|
|
1985
|
+
keycloak http://localhost:9083 admin admin z1n7…
|
|
1986
|
+
jenkins http://localhost:8085 admin cfu_… 11651…
|
|
1987
|
+
sonarqube http://localhost:9000 admin v1JE… squ_…
|
|
1988
|
+
…
|
|
1989
|
+
```
|
|
1990
|
+
|
|
1991
|
+
## Restart, stop and start
|
|
1992
|
+
|
|
1993
|
+
`dev restart` restarts every service; pass a service to restart just one:
|
|
1994
|
+
|
|
1995
|
+
```bash
|
|
1996
|
+
ligoj dev restart # all services
|
|
1997
|
+
ligoj dev restart jenkins # one kube-play service -> podman pod restart
|
|
1998
|
+
ligoj dev restart argocd # one kind service -> kubectl rollout restart
|
|
1999
|
+
```
|
|
2000
|
+
|
|
2001
|
+
kube-play services are restarted with `podman pod restart`; `harbor`/`argocd` with a
|
|
2002
|
+
`kubectl rollout restart` of their deployments/statefulsets (the kind node is started first if it was
|
|
2003
|
+
stopped). A service that hasn't been created yet is reported and skipped.
|
|
2004
|
+
|
|
2005
|
+
`dev stop` stops everything to free resources; pass a service to stop just one:
|
|
2006
|
+
|
|
2007
|
+
```bash
|
|
2008
|
+
ligoj dev stop # all services
|
|
2009
|
+
ligoj dev stop gitlab # one kube-play service -> podman pod stop
|
|
2010
|
+
ligoj dev stop harbor # one kind service -> scale workloads to 0
|
|
2011
|
+
```
|
|
2012
|
+
|
|
2013
|
+
A kube-play service is stopped with `podman pod stop`; a single kind service has its workloads
|
|
2014
|
+
scaled to 0. Stopping **all** also stops the kind node (pausing Harbor + ArgoCD together while keeping
|
|
2015
|
+
their replica counts).
|
|
2016
|
+
|
|
2017
|
+
`dev start` is the inverse — it starts stopped services without the full `init` reconcile:
|
|
2018
|
+
|
|
2019
|
+
```bash
|
|
2020
|
+
ligoj dev start # all services
|
|
2021
|
+
ligoj dev start sonarqube # one kube-play service -> podman pod start
|
|
2022
|
+
ligoj dev start harbor # one kind service -> start node + scale workloads to 1
|
|
2023
|
+
```
|
|
2024
|
+
|
|
2025
|
+
A kube-play service is started with `podman pod start`; a kind service starts the node (if the
|
|
2026
|
+
whole-cluster stop stopped it) and scales its workloads back to 1. (`dev init` also brings everything
|
|
2027
|
+
up, additionally re-running the chart upgrades and token/realm steps.)
|
|
2028
|
+
|
|
2029
|
+
All of `init`, `start`, `stop` and `restart` take `--wait` and stream **live progress** while waiting
|
|
2030
|
+
for the target state (services up, or down for `stop`): omit it to wait until done (or Ctrl+C), `0`
|
|
2031
|
+
to return immediately, or `N` to cap the wait at N seconds — e.g. `dev restart --wait 120`.
|
|
2032
|
+
|
|
2033
|
+
### Whole-environment `up` / `down`
|
|
2034
|
+
|
|
2035
|
+
`dev down` and `dev up` power the environment off and on at the **podman-machine** level rather than
|
|
2036
|
+
service by service:
|
|
2037
|
+
|
|
2038
|
+
```bash
|
|
2039
|
+
ligoj dev down # hard stop: stop the podman machine, then quit Podman Desktop
|
|
2040
|
+
ligoj dev up # start podman + its machine, launch Podman Desktop, then start every service
|
|
2041
|
+
```
|
|
2042
|
+
|
|
2043
|
+
`dev down` is a **hard stop** of the whole environment: it stops the podman machine (a single VM
|
|
2044
|
+
stop takes every service and the kind node down at once) and then **quits the Podman Desktop app** —
|
|
2045
|
+
which otherwise keeps the machine managed/alive — force-terminating it if it does not quit cleanly.
|
|
2046
|
+
|
|
2047
|
+
`dev up` is the inverse: it starts podman and the machine (installing/creating them if missing, same
|
|
2048
|
+
as `init`, but **without** the resource resize), **launches Podman Desktop**, waits for the machine to
|
|
2049
|
+
be ready, then runs the `dev start` actions to bring the pods and kind workloads back. `dev up` takes
|
|
2050
|
+
`--wait` like `start`. (Podman Desktop is only touched on macOS, and only when it is installed.)
|
|
2051
|
+
|
|
2052
|
+
## Configure Ligoj with `dev demo`
|
|
2053
|
+
|
|
2054
|
+
While `dev init` brings up the backing **services**, `dev demo` configures a **running Ligoj
|
|
2055
|
+
instance** to use them. The Ligoj API can run either as a container or from IntelliJ — the command
|
|
2056
|
+
only needs it reachable at the configured `--endpoint` (default `http://localhost:8080/ligoj`).
|
|
2057
|
+
|
|
2058
|
+
```bash
|
|
2059
|
+
ligoj dev demo # configure every installed plugin that has a demo
|
|
2060
|
+
ligoj dev demo --list # just list installed plugins (id, name, version) and exit
|
|
2061
|
+
ligoj dev demo --only plugin-id-ldap plugin-build-jenkins
|
|
2062
|
+
```
|
|
2063
|
+
|
|
2064
|
+
It (1) checks Ligoj is up via `/manage/health`, (2) lists the installed plugins with
|
|
2065
|
+
[`plugin list`](#plugin), (3) runs the demo registered for each one, then (4) creates the demo
|
|
2066
|
+
projects and their [link subscriptions](#demo-projects-and-link-subscriptions). Connection values
|
|
2067
|
+
(URLs, users, passwords/tokens) are read back from the `[dev]` credentials section that `dev init`
|
|
2068
|
+
wrote, so the created nodes point at the local services.
|
|
2069
|
+
|
|
2070
|
+
Each plugin's demo lives in its own module under `ligojcli/dev_demo/`:
|
|
2071
|
+
|
|
2072
|
+
| Plugin artifact | What the demo does |
|
|
2073
|
+
| ----------------------------- | --------------------------------------------------------------------------------------- |
|
|
2074
|
+
| `plugin-id-ldap` | Upserts the `service:id:ldap:local` node (from [docs/nodes/ldap.local.json](docs/nodes/ldap.local.json), with the live URL / bind DN / password), makes it the primary IAM, restarts the context, then creates the reference OUs, company/group container scopes and technical groups |
|
|
2075
|
+
| `plugin-build-jenkins` | Upserts the `service:build:jenkins:local` node (url / user / api-token) |
|
|
2076
|
+
| `plugin-scm-gitlab` | Upserts the `service:scm:gitlab:local` node (url / user / auth-key) |
|
|
2077
|
+
| `plugin-registry-harbor` | Upserts the `service:registry:harbor:local` node (url / user / password) |
|
|
2078
|
+
| `plugin-registry-nexus` | Upserts the `service:registry:nexus:local` node (from [docs/nodes/nexus.local.json](docs/nodes/nexus.local.json), with the live url / user / password) |
|
|
2079
|
+
| `plugin-registry-artifactory` | Upserts the `service:registry:artifactory:local` node (from [docs/nodes/artifactory.local.json](docs/nodes/artifactory.local.json), with the live url / user / password) |
|
|
2080
|
+
| `plugin-prov-aws` | Upserts the `service:prov:aws:local` node from the `[dev]` AWS credentials — `aws_access_key_id`, `aws_secret_access_key`, `aws_account_id` (all required) |
|
|
2081
|
+
| `plugin-prov-azure` | Upserts the `service:prov:azure:local` node from the `[dev]` Azure service principal — `azure_tenant_id`, `azure_subscription_id`, `azure_application_id`, `azure_client_secret` (required) and `azure_resource_group` (optional) |
|
|
2082
|
+
|
|
2083
|
+
Each demo is idempotent (nodes are upserted, OUs/scopes/groups skip when they already exist), so
|
|
2084
|
+
re-running is safe. A plugin whose required values are missing (e.g. no Jenkins API token in `[dev]`)
|
|
2085
|
+
is skipped with a warning instead of aborting the run. Use `--wait` to bound the LDAP context restart
|
|
2086
|
+
(defaults to 60s).
|
|
2087
|
+
|
|
2088
|
+
### Demo projects and link subscriptions
|
|
2089
|
+
|
|
2090
|
+
After the nodes are configured, the demo also creates three projects — **Démo #1** (`demo-1`),
|
|
2091
|
+
**Démo #2** (`demo-2`) and **Démo #3** (`demo-3`) — owned by the current API user. (Ligoj project
|
|
2092
|
+
keys match `^([a-z]|\d+-?[a-z])[a-z\d\-]*$`, so `demo:1` is written `demo-1`.)
|
|
2093
|
+
|
|
2094
|
+
Only **`demo-1`** receives subscriptions; `demo-2` / `demo-3` stay empty. For each active tool node a
|
|
2095
|
+
subscription is created in **`link` mode**, which requires the referenced resource to already exist
|
|
2096
|
+
on the remote tool — so the demo first provisions that resource via the tool's own REST API (using
|
|
2097
|
+
the `[dev]` credentials), then links it. The two **provisioning** plugins are the exception: they
|
|
2098
|
+
subscribe in **`create` mode** (a new, empty quote) — which Ligoj rejects until a price **catalog**
|
|
2099
|
+
has been imported for that provider, so that step is best-effort and reported when the catalog is
|
|
2100
|
+
missing. The plugins are processed **in parallel, one worker per plugin**:
|
|
2101
|
+
|
|
2102
|
+
| Plugin | Resource created on the tool | Link parameter(s) |
|
|
2103
|
+
| ----------------------------- | ----------------------------------------------------- | ---------------------------------------- |
|
|
2104
|
+
| `plugin-id-ldap` | a group `demo-1` (Project scope) | `service:id:group` |
|
|
2105
|
+
| `plugin-build-jenkins` | a free-style job `demo-1` | `service:build:jenkins:job` |
|
|
2106
|
+
| `plugin-scm-gitlab` | a project `demo-1` | `service:scm:gitlab:repository` |
|
|
2107
|
+
| `plugin-scm-github` | — (links the public `ligoj/plugin-ui` repo) | `service:scm:github:repository` |
|
|
2108
|
+
| `plugin-qa-sonarqube` | a dedicated `ligoj` admin user + the empty project `org.ligoj.plugin:plugin-ui` | `service:qa:sonarqube:project` |
|
|
2109
|
+
| `plugin-registry-harbor` | a project `demo-1` (docker/OCI only) | `type` = `docker`, `registry` |
|
|
2110
|
+
| `plugin-registry-nexus` | hosted repositories `demo-1-docker`, `demo-1-maven` | one subscription per type (`type`, `registry`) |
|
|
2111
|
+
| `plugin-registry-artifactory` | — (**not usable on OSS**; subscription skipped) | `type`, `registry` (Pro only) |
|
|
2112
|
+
| `plugin-prov-aws` | an empty provisioning **quote** on `demo-1` (`create` mode) | *(none — needs a catalog first)* |
|
|
2113
|
+
| `plugin-prov-azure` | an empty provisioning **quote** on `demo-1` (`create` mode) | *(none — needs a catalog first)* |
|
|
2114
|
+
|
|
2115
|
+
For registry plugins, the demo provisions and subscribes one repository **per supported type**
|
|
2116
|
+
(currently `docker` and `maven`, intersected with what the tool advertises — Harbor is docker-only).
|
|
2117
|
+
The supported types and the parameter definitions are discovered from the node itself
|
|
2118
|
+
(`node/<id>/parameter/link`), so the demo adapts to each plugin. Anything that cannot be created is
|
|
2119
|
+
skipped with a warning and never aborts the run.
|
|
2120
|
+
|
|
2121
|
+
**Artifactory OSS cannot be used by the Ligoj registry plugin at all** — it needs Pro-only REST APIs.
|
|
2122
|
+
OSS blocks both repository *creation* (`PUT /api/repositories/<key>`) **and** per-repository *reads*
|
|
2123
|
+
(`GET /api/repositories/<key>`), and the plugin's `artifactory-registry` validator relies on the
|
|
2124
|
+
latter — so even a Maven repo you create by hand in the UI cannot be linked, and OSS has no Docker
|
|
2125
|
+
package type at all (Docker is Pro-only, hence greyed out in the UI). Artifactory OSS is therefore
|
|
2126
|
+
only useful here as a plain Maven **deploy target** (the seed pushes to its default `example-repo-local`
|
|
2127
|
+
over the non-Pro deploy API). For demo **registry subscriptions**, use **Nexus** (docker + maven) and
|
|
2128
|
+
**Harbor** (docker), which work fully.
|
|
2129
|
+
|
|
2130
|
+
The **GitHub** link points at the real public `ligoj/plugin-ui` repository, which GitHub validates
|
|
2131
|
+
against its API — so a token is required (`[dev]` `github_token` / `GITHUB_TOKEN`, or the locally
|
|
2132
|
+
authenticated `gh` CLI). Without one, the GitHub demo is skipped.
|
|
2133
|
+
|
|
2134
|
+
The **SonarQube** plugin authenticates with a login + password that must have admin rights, so the
|
|
2135
|
+
demo — using the `sonar_api_token` from `dev init` — provisions a dedicated `ligoj` admin user
|
|
2136
|
+
(password `sonar_demo_password`, default `Ligoj-Demo-Pass1!`; SonarQube requires ≥ 12 characters with
|
|
2137
|
+
upper/lower/digit/special) rather than reusing the admin account, then links the
|
|
2138
|
+
`org.ligoj.plugin:plugin-ui` project. That Sonar project is created empty up front (link
|
|
2139
|
+
mode needs it to exist); the seed phase's `sonar:sonar` analysis fills it in afterwards. Without a
|
|
2140
|
+
token, the SonarQube demo is skipped.
|
|
2141
|
+
|
|
2142
|
+
Every subscription is created idempotently and its status is refreshed (validated) right after
|
|
2143
|
+
linking.
|
|
2144
|
+
|
|
2145
|
+
### Tool data seeding
|
|
2146
|
+
|
|
2147
|
+
Finally, `dev demo` fills the tools with real data so the demo project has something to show. This
|
|
2148
|
+
step is **heavy** (image pulls, two Maven builds, a Sonar analysis and two git mirrors — several
|
|
2149
|
+
minutes) and runs on a full `dev demo` (it is skipped when you pass `--only`). The tools are seeded
|
|
2150
|
+
in parallel, best-effort — a failing tool logs a warning and never aborts the rest:
|
|
2151
|
+
|
|
2152
|
+
| Tool | Data seeded |
|
|
2153
|
+
| ---- | ----------- |
|
|
2154
|
+
| **Harbor** | pulls 2 small public images (`busybox`, `alpine`) and pushes them into the `demo-1` project |
|
|
2155
|
+
| **Nexus (docker)** | pushes the same 2 images to the Nexus docker registry connector (port `8182`, see `--nexus-docker-port`) |
|
|
2156
|
+
| **Nexus / Artifactory (maven)** | builds `plugin-ui` and `plugin-id` and deploys the artifacts to `demo-1-maven` (Nexus) and to `example-repo-local` (Artifactory OSS's default repo) |
|
|
2157
|
+
| **SonarQube** | runs `mvn … sonar:sonar` on both plugins (project keys `org.ligoj.plugin:plugin-ui` / `plugin-id`) |
|
|
2158
|
+
| **GitLab** | mirrors the `github.com/ligoj/plugin-ui` and `plugin-id` repositories |
|
|
2159
|
+
|
|
2160
|
+
Prerequisites: **podman**, **mvn** (JDK 21), **git**, network access to Docker Hub / GitHub, and the
|
|
2161
|
+
plugin sources under `LIGOJ_PLUGINS_DIR` (default `~/git/ligoj-plugins`). Nexus Community Edition
|
|
2162
|
+
requires its EULA (accepted automatically by `dev init`) and a docker connector port; Artifactory OSS
|
|
2163
|
+
has no docker registry, so it only receives the Maven artifacts.
|
|
2164
|
+
|
|
2165
|
+
## Keycloak realm, federation and client
|
|
2166
|
+
|
|
2167
|
+
For the `keycloak` service, `dev init` drives the Keycloak Admin REST API to create (idempotently):
|
|
2168
|
+
|
|
2169
|
+
- the **`ligoj` realm**;
|
|
2170
|
+
- an **LDAP user federation** bound to the OpenLDAP pod — reachable from inside the Keycloak
|
|
2171
|
+
pod through `ldap://host.containers.internal:<ldap-port>` — using
|
|
2172
|
+
`cn=<ldap_admin_user>,<ldap_root>` (`READ_ONLY`, users DN `<ldap_root>`, `inetOrgPerson`, subtree);
|
|
2173
|
+
- a confidential **`ligoj` OIDC client** (standard flow, redirect URIs for the UI on `:5173` and the
|
|
2174
|
+
API on `:8080`) whose generated secret is stored in `[dev] keycloak_client_secret`.
|
|
2175
|
+
|
|
2176
|
+
It then prints ready-to-paste Spring Boot properties:
|
|
2177
|
+
|
|
2178
|
+
```properties
|
|
2179
|
+
security=OAuth2Bff
|
|
2180
|
+
ligoj.security.oauth2.username-attribute = email
|
|
2181
|
+
ligoj.security.login.url = /oauth2/authorization/keycloak
|
|
2182
|
+
spring.security.oauth2.client.provider.keycloak.issuer-uri=http://localhost:9083/realms/ligoj
|
|
2183
|
+
spring.security.oauth2.client.registration.keycloak.provider=keycloak
|
|
2184
|
+
spring.security.oauth2.client.registration.keycloak.authorization-grant-type=authorization_code
|
|
2185
|
+
spring.security.oauth2.client.registration.keycloak.client-id=ligoj
|
|
2186
|
+
spring.security.oauth2.client.registration.keycloak.client-secret=<generated>
|
|
2187
|
+
spring.security.oauth2.client.registration.keycloak.scope=openid
|
|
2188
|
+
```
|
|
2189
|
+
|
|
2190
|
+
## Debug the Ligoj apps from the IDE (`dev debug`) — macOS
|
|
2191
|
+
|
|
2192
|
+
While `dev init` brings up the backing **services**, `dev debug` drives the local **application**
|
|
2193
|
+
stack you actually debug: IntelliJ IDEA plus the two Ligoj Spring Boot apps and the Vite dev server.
|
|
2194
|
+
|
|
2195
|
+
| Component | Started by `dev debug` | Endpoint / path |
|
|
2196
|
+
| --------------- | ------------------------------------------------------- | --------------- |
|
|
2197
|
+
| IntelliJ IDEA | `open -a "IntelliJ IDEA" <project>` (if stopped) | `~/git/ligoj` |
|
|
2198
|
+
| `ligoj-api` | the **dedicated launcher app**, in Debug mode | `http://localhost:8081/ligoj-api` |
|
|
2199
|
+
| `ligoj-ui` | the **dedicated launcher app**, in Debug mode | `http://localhost:8080/ligoj` |
|
|
2200
|
+
| Vite (app-ui) | `npm run dev` in `app-ui/src/main/webapp` | `http://localhost:5173/ligoj/` |
|
|
2201
|
+
|
|
2202
|
+
```bash
|
|
2203
|
+
ligoj dev debug init # compile the dedicated launcher app (one-time; re-run after renaming a config)
|
|
2204
|
+
ligoj dev debug start # open IntelliJ + Debug-launch the API/UI + start Vite (only those stopped)
|
|
2205
|
+
ligoj dev debug status # show what is running (process) and reachable (port), no changes
|
|
2206
|
+
ligoj dev debug stop # stop the API/UI/Vite apps (IntelliJ stays open to protect unsaved work)
|
|
2207
|
+
ligoj dev debug restart # stop then start the apps
|
|
2208
|
+
ligoj dev debug start -w 60 # same live '--wait' as the other dev commands (0 = no wait)
|
|
2209
|
+
```
|
|
2210
|
+
|
|
2211
|
+
**Why `init` / the launcher app.** IntelliJ has no headless "run this configuration" command, and
|
|
2212
|
+
scripting its UI needs the broad macOS **Accessibility** permission (control any app + read the
|
|
2213
|
+
screen). Instead of granting that to your whole terminal, `dev debug init` compiles a tiny dedicated
|
|
2214
|
+
app (default `~/Applications/Ligoj Debug.app`) that drives IntelliJ's *Run ▶ Debug…* chooser for
|
|
2215
|
+
`ligoj-api` / `ligoj-ui` (skipping any already running). You grant Accessibility to **that app only**
|
|
2216
|
+
— the first `dev debug start` triggers the macOS prompt — and can then revoke your terminal's grant.
|
|
2217
|
+
When `dev debug start` launches a **cold** IntelliJ, the launcher first **waits (up to 180 s) for the
|
|
2218
|
+
IDE to become UI-ready** — its *Run* menu populated, i.e. the project frame is up — before sending any
|
|
2219
|
+
keystroke, so a not-yet-started IDE no longer drops the Debug commands. Because that logic is baked
|
|
2220
|
+
into the compiled app, `dev debug start` warns and asks you to **re-run `dev debug init`** whenever the
|
|
2221
|
+
installed launcher predates this behavior (also re-run it after renaming a run config).
|
|
2222
|
+
|
|
2223
|
+
Everything else needs no permission: all four components are detected by process
|
|
2224
|
+
(`org.ligoj.boot.api.Application` / `…web.Application`, the project's vite process) and by TCP port,
|
|
2225
|
+
so `status`/`stop`/`restart` cover them however they were started. `stop` (and the stop half of
|
|
2226
|
+
`restart`) terminates the API/UI processes even though the IDE launched them. Overridable via
|
|
2227
|
+
`[dev]`/env: `LIGOJ_PROJECT_DIR`·`ligoj_project_dir`, `IDEA_APP`·`idea_app`,
|
|
2228
|
+
`LIGOJ_DEBUG_APP`·`ligoj_debug_app` (launcher app location).
|
|
2229
|
+
|
|
2230
|
+
> The Ligoj API dev app binds `8081`, so the `nexus` service publishes `8181` (not its own `8081`) to
|
|
2231
|
+
> avoid the clash — run both at once without conflict.
|
|
2232
|
+
|
|
2233
|
+
## Scaffold a new plugin (`dev plugin create`)
|
|
2234
|
+
|
|
2235
|
+
`dev plugin create <plugin>` generates a brand-new Ligoj plugin project in the current directory. The
|
|
2236
|
+
`<plugin>` is the full Maven artifact and **must start with `plugin-`**; its fragments decide the type,
|
|
2237
|
+
exactly like the real plugins:
|
|
2238
|
+
|
|
2239
|
+
- **one fragment** → a **service** plugin, e.g. `plugin-km` (like `plugin-id`)
|
|
2240
|
+
- **two+ fragments** → a **tool** plugin, e.g. `plugin-km-confluence` (like `plugin-id-ldap`), which
|
|
2241
|
+
extends the service `plugin-<first-fragment>` via a `provided` Maven dependency.
|
|
2242
|
+
|
|
2243
|
+
You're prompted for a display name and description (or pass `--name` / `--description`):
|
|
2244
|
+
|
|
2245
|
+
```bash
|
|
2246
|
+
ligoj dev plugin create plugin-km # a service plugin
|
|
2247
|
+
ligoj dev plugin create plugin-km-confluence # a tool plugin of plugin-km
|
|
2248
|
+
ligoj dev plugin create plugin-foo --name "Ligoj - Plugin Foo" --description "Foo service."
|
|
2249
|
+
```
|
|
2250
|
+
|
|
2251
|
+
It generates a complete, buildable project (`mvn verify` compiles, tests, and builds the Vue bundle),
|
|
2252
|
+
with **100% coverage of the generated code**:
|
|
2253
|
+
|
|
2254
|
+
| Area | Files |
|
|
2255
|
+
| ---- | ----- |
|
|
2256
|
+
| Maven | `pom.xml` (correct parent + coordinates; a tool adds the `provided` parent-service dep) |
|
|
2257
|
+
| Java | the `AbstractServicePlugin` / `AbstractToolPluginResource` skeleton (+ the `<Service>ServicePlugin` interface for a service), package `org.ligoj.app.plugin.<fragment>` |
|
|
2258
|
+
| Java test | plain JUnit 5 asserting `getKey()` |
|
|
2259
|
+
| Vue UI (`ui/`) | `index.js` + `service.js` (`@ligoj/host` integration), i18n `en`/`fr`, `package.json` (+ generated `package-lock.json`), `vite.config.js`, `eslint.config.js`, and a Vitest test mocking `@ligoj/host` |
|
|
2260
|
+
| Resources | `csv/node.csv` (+ `csv/parameter.csv` for a tool) |
|
|
2261
|
+
| Project | `README.md`, `LICENSE` (MIT), `.gitignore`, `.codeclimate.yml`, GitHub SonarCloud workflow |
|
|
2262
|
+
|
|
2263
|
+
Options: `--name`, `--description`, `--dir` (parent directory, default the current one). The generated
|
|
2264
|
+
`vite.config.js` / Vitest expect the Ligoj UI host as a sibling checkout at `../../../ligoj` (same
|
|
2265
|
+
convention as the existing plugins), and `npm` is used once at creation to produce the lockfile the
|
|
2266
|
+
Maven frontend build (`npm ci`) requires. After creation: `cd <plugin> && mvn verify`.
|
|
2267
|
+
|
|
2268
|
+
## Build plugin frontends (`dev plugin build`)
|
|
2269
|
+
|
|
2270
|
+
Each Ligoj plugin ships a frontend under `<plugin>/ui/` built with `npm run build` (Vite). `dev plugin
|
|
2271
|
+
build` runs that build for every **live** plugin — the ones installed in the running Ligoj instance
|
|
2272
|
+
(`system/plugin`) that also have a local `<plugin>/ui/` under the plugins directory:
|
|
2273
|
+
|
|
2274
|
+
```bash
|
|
2275
|
+
# Rebuild the frontend of every live plugin (in parallel)
|
|
2276
|
+
ligoj dev plugin build
|
|
2277
|
+
|
|
2278
|
+
# Build only specific plugins (skips the live lookup, so Ligoj need not be running)
|
|
2279
|
+
ligoj dev plugin build --only plugin-ui plugin-id
|
|
2280
|
+
|
|
2281
|
+
# Limit parallelism
|
|
2282
|
+
ligoj dev plugin build --jobs 2
|
|
2283
|
+
```
|
|
2284
|
+
|
|
2285
|
+
Dependencies are installed automatically on first build (`npm ci` when a `package-lock.json` is
|
|
2286
|
+
present, otherwise `npm install`) before `npm run build`. Builds run in parallel (default
|
|
2287
|
+
`min(4, CPUs)`, `--jobs` to change) and each plugin is reported `OK` / `FAILED` independently — one
|
|
2288
|
+
failing frontend never aborts the others. The plugins directory is `LIGOJ_PLUGINS_DIR` /
|
|
2289
|
+
`[dev] ligoj_plugins_dir` (default `~/git/ligoj-plugins`), and `npm` must be on the `PATH`. Without
|
|
2290
|
+
`--only`, Ligoj must be reachable so the live plugin set can be listed.
|
|
2291
|
+
|
|
2292
|
+
## Renovate a plugin's dependencies (`dev plugin renovate`)
|
|
2293
|
+
|
|
2294
|
+
`dev plugin renovate` updates a plugin's dependency descriptors, ported from the `renovate` mode of
|
|
2295
|
+
`commands/release.sh` but scoped to **editing the files** — it does **not** commit, leaving the edits
|
|
2296
|
+
in the working tree for you to review:
|
|
2297
|
+
|
|
2298
|
+
- **`pom.xml`** — bumps the `org.ligoj.api:plugin-parent` `<version>` to the target (default: the
|
|
2299
|
+
latest local `org.ligoj.api:parent` release; override with `--parent-version`). A current version
|
|
2300
|
+
newer than the target is left alone; the project's own `<version>` is never touched.
|
|
2301
|
+
- **`package.json`** — re-pins only the npm dependency constraints that are **also** declared by the
|
|
2302
|
+
host UI to the host's versions (none added/removed; `scripts` and non-shared deps untouched).
|
|
2303
|
+
- **`package-lock.json`** — regenerated from `package.json` (`npm install --package-lock-only`) so the
|
|
2304
|
+
plugin's `npm ci` stays in sync.
|
|
2305
|
+
|
|
2306
|
+
```bash
|
|
2307
|
+
# Renovate the plugin in the current directory
|
|
2308
|
+
ligoj dev plugin renovate
|
|
2309
|
+
|
|
2310
|
+
# A specific plugin (artifact under LIGOJ_PLUGINS_DIR, or a path)
|
|
2311
|
+
ligoj dev plugin renovate plugin-km
|
|
2312
|
+
|
|
2313
|
+
# Every plugin under LIGOJ_PLUGINS_DIR
|
|
2314
|
+
ligoj dev plugin renovate --all
|
|
2315
|
+
```
|
|
2316
|
+
|
|
2317
|
+
The host UI referential is `LIGOJ_HOST_PACKAGE_JSON` (default
|
|
2318
|
+
`~/git/ligoj/app-ui/src/main/webapp/package.json`, override with `--host-package-json`), the plugins
|
|
2319
|
+
root is `LIGOJ_PLUGINS_DIR` / `--plugins-dir` (default `~/git/ligoj-plugins`), and `npm` must be on the
|
|
2320
|
+
`PATH`. A plugin must be an `org.ligoj.api:plugin-parent` project — anything else is skipped (with
|
|
2321
|
+
`--all`) or reported as an error (when named).
|
|
2322
|
+
|
|
2323
|
+
## Build the app container images (`dev package`)
|
|
2324
|
+
|
|
2325
|
+
`dev package` builds the two Ligoj application container images **locally**, straight from the
|
|
2326
|
+
`app-api/` and `app-ui/` Dockerfiles with podman (or docker) — no external script. It produces exactly
|
|
2327
|
+
the images `dev test start` runs, and pushes nothing:
|
|
2328
|
+
|
|
2329
|
+
```bash
|
|
2330
|
+
ligoj dev package # build ligoj-api + ligoj-ui for the native arch (fast)
|
|
2331
|
+
ligoj dev package --only api # build just one image
|
|
2332
|
+
ligoj dev package --tag 4.0.2-rc1 # override the image tag
|
|
2333
|
+
ligoj dev package --platform all # multi-arch manifest (linux/amd64 + linux/arm64, podman)
|
|
2334
|
+
```
|
|
2335
|
+
|
|
2336
|
+
Every option resolves from the flag, then the environment, then `[dev]`:
|
|
2337
|
+
|
|
2338
|
+
| Option | Default | Env / `[dev]` key |
|
|
2339
|
+
| ------ | ------- | ----------------- |
|
|
2340
|
+
| `--project DIR` (Ligoj checkout) | `~/git/ligoj` | `LIGOJ_DIR` / `ligoj_dir` |
|
|
2341
|
+
| `--tag TAG` | project `<version>` without `-SNAPSHOT` | `LIGOJ_PACKAGE_TAG` / `ligoj_package_tag` |
|
|
2342
|
+
| `--only api\|ui` | both | — |
|
|
2343
|
+
| `--platform` (single arch, comma list, or `all`) | host native arch | — |
|
|
2344
|
+
| `--runtime docker\|podman` | `docker` if present, else `podman` | `LIGOJ_TEST_RUNTIME` / `ligoj_test_runtime` |
|
|
2345
|
+
|
|
2346
|
+
The **API image bundles all three JDBC drivers** (`db-postgresql`, `db-mysql`, `db-mariadb`): the build
|
|
2347
|
+
passes `--build-arg MAVEN_PROFILES=…` to force those Maven profiles. They are `activeByDefault` in
|
|
2348
|
+
`app-api/pom.xml`, but Maven silently disables *every* default profile as soon as a `settings.xml`
|
|
2349
|
+
`<activeProfiles>` (or any `-P`) is in play — which had dropped all drivers from the war and made the API
|
|
2350
|
+
fail at boot with `ClassNotFoundException: org.postgresql.Driver`. Forcing them makes the image
|
|
2351
|
+
DB-capable regardless of the build's `settings.xml`. `~/.ligoj/plugin-vendors.p12` is bundled into the
|
|
2352
|
+
API image when present (password from `PLUGIN_VENDORS_STOREPASS`, the `ligoj.release.vendors-storepass`
|
|
2353
|
+
keychain entry, or `changeit`).
|
|
2354
|
+
|
|
2355
|
+
Only the host arch is built by default — much faster than a QEMU-emulated multi-arch build, and all
|
|
2356
|
+
`dev test` needs. This builds **local images only**; for a full release (deploy, tags, Docker Hub) use
|
|
2357
|
+
the release helper (`commands/release.sh`).
|
|
2358
|
+
|
|
2359
|
+
## Test the released app containers (`dev test`)
|
|
2360
|
+
|
|
2361
|
+
While `dev debug` runs the apps from your IDE, `dev test` runs the **released Docker images**
|
|
2362
|
+
(`ligoj/ligoj-api` + `ligoj/ligoj-ui`) against the local dev stack — the quickest way to smoke-test a
|
|
2363
|
+
published build. It starts both containers in the background, waits until **both are healthy**, then
|
|
2364
|
+
opens the UI in your browser at `http://localhost:<ui-port>/ligoj/`:
|
|
2365
|
+
|
|
2366
|
+
```bash
|
|
2367
|
+
ligoj dev test start # run both, wait for health, open the browser
|
|
2368
|
+
ligoj dev test stop # stop and remove both containers
|
|
2369
|
+
ligoj dev test start --tag 4.0.2-SNAPSHOT-101 --port 8089 --api-port 8088
|
|
2370
|
+
ligoj dev test start --no-browser --no-wait # start detached, don't wait or open the browser
|
|
2371
|
+
ligoj dev test -h # full option + '-D' reference (mirrors ligoj/DOC.md)
|
|
2372
|
+
|
|
2373
|
+
# Free-form JVM options, grouped per container with --api / --ui:
|
|
2374
|
+
ligoj dev test start \
|
|
2375
|
+
--api -Dlog.level=INFO -Dligoj.sslVerify=false \
|
|
2376
|
+
--ui -Dsecurity=Trusted -Dlog.level=info
|
|
2377
|
+
```
|
|
2378
|
+
|
|
2379
|
+
The containers mount **`LIGOJ_HOME`** (default `~/.ligoj`, override with `--home` / `LIGOJ_HOME` /
|
|
2380
|
+
`[dev] ligoj_home`) at `/home/ligoj`, with `hooks/` and `files/` subdirectories — this replaces the
|
|
2381
|
+
`/var/lib/ligoj` of the DOC.md sample commands with your user home. Under **podman** they run as
|
|
2382
|
+
`--user 0` so the app can write that host-owned mount. The UI's `ENDPOINT` points at the API on `--api-port`.
|
|
2383
|
+
|
|
2384
|
+
**Database:** by default the API is wired to the **dev-stack PostgreSQL** from `[dev]`
|
|
2385
|
+
(`-Djdbc.vendor=postgresql` + host/port/db/user/password + the PG dialect), so it targets the dev DB out
|
|
2386
|
+
of the box. The API image must **bundle the PostgreSQL JDBC driver** (`org.postgresql.Driver`) or it
|
|
2387
|
+
fails at boot with `ClassNotFoundException` — the released `ligoj-api` image defaults to MySQL only, so
|
|
2388
|
+
add the driver to your build. Pass an explicit `--api …` group to take full control of the DB options.
|
|
2389
|
+
|
|
2390
|
+
**Networking** adapts to the runtime: **docker/Linux** uses `--network=host` (the API reaches the dev DB
|
|
2391
|
+
on `localhost`); **podman-machine** publishes ports (`-p <port>:<port>`) so the mac can reach
|
|
2392
|
+
`localhost:<port>`, and the containers reach the machine host — the other container and the dev DB — via
|
|
2393
|
+
`host.containers.internal`. Override with `--net host|publish`.
|
|
2394
|
+
|
|
2395
|
+
Every option resolves from the CLI flag, then the environment, then `~/.ligoj/config` / `~/.ligoj/credentials`:
|
|
2396
|
+
|
|
2397
|
+
| Option | Default | Env / `[dev]` key |
|
|
2398
|
+
| ------ | ------- | ----------------- |
|
|
2399
|
+
| `--port` (UI port, also the browser port) | `8089` | `LIGOJ_UI_PORT` / `ligoj_ui_port` |
|
|
2400
|
+
| `--api-port` (UI `ENDPOINT` + API exposed port) | `8088` | `LIGOJ_API_PORT` / `ligoj_api_port` |
|
|
2401
|
+
| `--home` (`LIGOJ_HOME`) | `~/.ligoj` | `LIGOJ_HOME` / `ligoj_home` |
|
|
2402
|
+
| `--tag` / `--api-tag` / `--ui-tag` | newest **local** build, else latest published | `LIGOJ_TEST_TAG` / `ligoj_test_tag` |
|
|
2403
|
+
| `--runtime` | `docker` if present, else `podman` | `LIGOJ_TEST_RUNTIME` / `ligoj_test_runtime` |
|
|
2404
|
+
| `--net` (`host` \| `publish`) | `publish` for podman, else `host` | `LIGOJ_TEST_NETWORK` / `ligoj_test_network` |
|
|
2405
|
+
| `--api` / `--ui` `-D…` (replace the defaults) | see below | `LIGOJ_TEST_API_OPTS` / `ligoj_test_api_opts` |
|
|
2406
|
+
| `--wait N` / `--no-wait` / `--no-browser` / `--pull` | wait 300s, open browser | — |
|
|
2407
|
+
|
|
2408
|
+
The `--api` / `--ui` groups take **any number of `-D…` options** (each replaces that container's
|
|
2409
|
+
defaults). Defaults: API `--enable-preview -Dlog.level=INFO <single-connection LDAP pool>
|
|
2410
|
+
-Dligoj.sslVerify=false`; UI `-Dsecurity=Trusted -Dlog.level=info`. Some properties have well-known
|
|
2411
|
+
values (from DOC.md's *Application level properties*): UI `-Dsecurity=Trusted|Rest|OAuth2Bff`, both
|
|
2412
|
+
`-Dlog.level=trace|debug|info|warn|error` and `-Dlogging.level.<category>=<level>`, API
|
|
2413
|
+
`-Djdbc.vendor=mysql|postgresql|mariadb`, `-Djpa.hbm2ddl=update|none|validate`,
|
|
2414
|
+
`-Dligoj.plugin.repository=central|nexus`. Run `ligoj dev test -h` for the full annotated list.
|
|
2415
|
+
|
|
2416
|
+
> `-Dsecurity=Trusted` (the UI default here) runs Ligoj **without password verification** (RBAC still
|
|
2417
|
+
> enforced) — convenient for local testing, never for a publicly reachable instance.
|
|
2418
|
+
|
|
2419
|
+
## Back up & restore a service's data (`dev backup`)
|
|
2420
|
+
|
|
2421
|
+
`dev backup` snapshots, lists and reloads the database rows owned by a Ligoj **service**, using the
|
|
2422
|
+
PostgreSQL client tools (`psql` / `pg_dump`, from `brew install libpq`). Only **`service:prov`** is
|
|
2423
|
+
supported today (its catalog is expensive to re-import). Everything lives under one command:
|
|
2424
|
+
|
|
2425
|
+
| Command | What it does |
|
|
2426
|
+
| ------- | ------------ |
|
|
2427
|
+
| `dev backup` | take the snapshot (no sub-command) |
|
|
2428
|
+
| `dev backup list` | list the snapshots stored locally |
|
|
2429
|
+
| `dev backup restore` | reload a snapshot into the target DB |
|
|
2430
|
+
|
|
2431
|
+
The services to snapshot are selected with **`--services` / `-S`**, which takes a list — space- or
|
|
2432
|
+
comma-separated, with the `service:` prefix optional. Omit it to act on every supported service.
|
|
2433
|
+
|
|
2434
|
+
```bash
|
|
2435
|
+
ligoj dev backup # every supported service
|
|
2436
|
+
ligoj dev backup --services service:prov # -> ~/.ligoj/backup/prov-<timestamp>/
|
|
2437
|
+
ligoj dev backup -S prov # same, short form
|
|
2438
|
+
ligoj dev backup list # list what is stored locally
|
|
2439
|
+
ligoj dev backup restore # list backups, pick one interactively
|
|
2440
|
+
ligoj dev backup restore --service service:prov # same, explicit service
|
|
2441
|
+
ligoj dev backup restore prov-20260706-232820 # restore a specific backup
|
|
2442
|
+
```
|
|
2443
|
+
|
|
2444
|
+
> Because `--services` takes a *list*, write the sub-command **first** — `dev backup list --services
|
|
2445
|
+
> prov`, not `dev backup --services prov list` (the latter would read `list` as a service name; the
|
|
2446
|
+
> CLI says so and names the fix).
|
|
2447
|
+
|
|
2448
|
+
> `dev backup restore` uses the singular **`--service`** / `-S` (same value forms — `prov` or
|
|
2449
|
+
> `service:prov`), because a restore targets exactly one backup id; its only positional is the
|
|
2450
|
+
> optional `backup_id`. Omit it to pick one from an interactive list.
|
|
2451
|
+
|
|
2452
|
+
### Listing backups (`dev backup list`)
|
|
2453
|
+
|
|
2454
|
+
`dev backup list` lists the snapshots under `~/.ligoj/backup`, newest first, in three renderings selected
|
|
2455
|
+
with `--format` / `-F`. It takes the same `--services` / `-S` filter as `dev backup`:
|
|
2456
|
+
|
|
2457
|
+
```bash
|
|
2458
|
+
ligoj dev backup list # table (default), all services
|
|
2459
|
+
ligoj dev backup list -F text # tab-separated rows, for cut/awk/grep
|
|
2460
|
+
ligoj dev backup list -F json # full records, for jq
|
|
2461
|
+
ligoj dev backup list --services service:prov # restrict to one (or more) services
|
|
2462
|
+
```
|
|
2463
|
+
|
|
2464
|
+
```
|
|
2465
|
+
ID CREATED SERVICE SUBS TABLES ROWS SIZE
|
|
2466
|
+
-------------------- ---------------- ------------ ---- ------ -------- -------
|
|
2467
|
+
prov-20260727-105521 2026-07-27 10:57 service:prov 13 34 24360846 580.0MB
|
|
2468
|
+
prov-20260706-232820 2026-07-06 23:28 service:prov 12 29 2126023 116.8MB
|
|
2469
|
+
|
|
2470
|
+
[INFO ] [backups] 2 backup(s), 813.6MB on disk in /Users/me/.ligoj/backup
|
|
2471
|
+
```
|
|
2472
|
+
|
|
2473
|
+
`SUBS` is the number of backed-up subscriptions, `TABLES` the number of `ligoj_prov_*` tables the
|
|
2474
|
+
snapshot holds (a useful tell that a backup predates a plugin-prov schema change), `ROWS` their total
|
|
2475
|
+
row count and `SIZE` the compressed dump. A snapshot whose `prov-data.sql.gz` is missing is flagged
|
|
2476
|
+
**INCOMPLETE** and called out as not restorable, rather than failing later during a restore.
|
|
2477
|
+
|
|
2478
|
+
The `text` mode emits the same columns tab-separated (`ligoj dev backup list -F text | cut -f1` gives just
|
|
2479
|
+
the ids); `json` adds the raw `dump_bytes` / `disk_bytes`, `duration_seconds`, `source_db`, `path` and
|
|
2480
|
+
`complete` fields, and prints `[]` rather than an error when there is nothing stored. When `--format`
|
|
2481
|
+
is omitted, an explicit global `-o/--output` is honoured — so `ligoj -o json dev backup list` works like
|
|
2482
|
+
the rest of the CLI — and `table` is now accepted by `-o` too.
|
|
2483
|
+
|
|
2484
|
+
**What a `service:prov` backup captures.** Every `ligoj_prov_*` table (the whole catalog + quotes),
|
|
2485
|
+
dumped in bulk with `pg_dump` — the price tables reach tens of millions of rows, so their content is
|
|
2486
|
+
stored compressed and restored **verbatim**. The table set is **globbed from the catalog**, never
|
|
2487
|
+
hard-coded, so tables a newer `plugin-prov` adds are captured automatically. Alongside, the
|
|
2488
|
+
cross-referenced core rows that keep the quotes valid are exported as CSV: the `ligoj_subscription`
|
|
2489
|
+
rows on prov nodes (referenced by `ligoj_prov_quote` and the other subscription-scoped prov tables),
|
|
2490
|
+
their `ligoj_node` rows, the `ligoj_parameter_value` rows of those nodes/subscriptions, and the
|
|
2491
|
+
`ligoj_project` rows behind them. A `metadata.json` records the id, timestamp, and per-table row
|
|
2492
|
+
counts; the whole thing lands under **`~/.ligoj/backup/<id>/`**. Progress, per-table stats, and a
|
|
2493
|
+
duration summary are printed throughout.
|
|
2494
|
+
|
|
2495
|
+
**Restore is an id-aware reload** (the target is a live Ligoj DB with its own ids), run as a single
|
|
2496
|
+
transaction (all-or-nothing):
|
|
2497
|
+
|
|
2498
|
+
1. drop the prov foreign keys, empty the `ligoj_prov_*` tables, and delete the current prov
|
|
2499
|
+
subscriptions / instance nodes / parameter values (+ their transient status events);
|
|
2500
|
+
2. **projects** are matched by `pkey` or name and **reused** (their id is remapped into the restored
|
|
2501
|
+
rows), else inserted; **nodes** are inserted only when missing (string ids);
|
|
2502
|
+
3. **subscriptions** are inserted with **fresh ids** (the backup's ids may already be taken by
|
|
2503
|
+
unrelated subscriptions in the target) and their project remapped — the subscription-scoped
|
|
2504
|
+
**parameter values** that reference them are repointed to the new ids; a node's parameter values
|
|
2505
|
+
are dropped then re-inserted;
|
|
2506
|
+
4. the `ligoj_prov_*` tables are **bulk-reloaded verbatim**, then **every prov column with a foreign
|
|
2507
|
+
key to `ligoj_subscription` is repointed** at the new ids. That list is **discovered from the
|
|
2508
|
+
catalog**, not hard-coded — today it covers `quote`, `quote_view`, `quote_snapshot`, `comparison`
|
|
2509
|
+
and `lookup_error` (the last two through *both* a `subscription` and a `main_subscription` column),
|
|
2510
|
+
and it picks up whatever `plugin-prov` adds next. This matters because the foreign keys are re-added
|
|
2511
|
+
`NOT VALID` (instant — no rescan of the huge tables), so a missed column would leave rows silently
|
|
2512
|
+
pointing at unrelated subscriptions instead of raising an error. Finally every touched sequence is
|
|
2513
|
+
bumped to `max(id) + allocation-size` so Hibernate's next id block can't collide.
|
|
2514
|
+
|
|
2515
|
+
The restore log shows only the **five phases**, each with its own duration (the noisy per-statement
|
|
2516
|
+
`ALTER` / `DELETE` / `COPY` output is suppressed); a `>> [n/5] … done (Xs)` line marks each phase.
|
|
2517
|
+
|
|
2518
|
+
Restore is **resilient to a Postgres / schema mismatch** between the backup and the target — e.g. a
|
|
2519
|
+
backup taken against a newer server restored into an older one:
|
|
2520
|
+
|
|
2521
|
+
- the version-specific session settings pg_dump writes in its header (such as PG 17's
|
|
2522
|
+
`SET transaction_timeout`) are stripped from the bulk dump before load, so an older server doesn't
|
|
2523
|
+
abort on an unknown parameter;
|
|
2524
|
+
- the core rows are loaded **by column name** (from each CSV header, intersected with the target
|
|
2525
|
+
table), not by position — so a differing column order or an extra/missing column can't shift a value
|
|
2526
|
+
into the wrong column;
|
|
2527
|
+
- the bulk prov `COPY` blocks are **aligned to the target's columns** the same way: a column the
|
|
2528
|
+
target lacks (e.g. a field a newer prov plugin added) is dropped from the block's header **and**
|
|
2529
|
+
every data row (a `WARN` reports it — that column's data is lost), a table the target doesn't have
|
|
2530
|
+
is skipped, and any target column the backup lacks keeps its default;
|
|
2531
|
+
- the prov **`UNIQUE` constraints** are dropped for the load (like the FKs) and each is re-added only
|
|
2532
|
+
if the reloaded rows satisfy it; one the backup's older data violates (e.g. two same-named storages
|
|
2533
|
+
in one quote, which a newer model now forbids) is **left dropped** with a `WARNING` naming it, so
|
|
2534
|
+
the restore still completes — dedup and re-add it by hand if you need it.
|
|
2535
|
+
|
|
2536
|
+
Without an id, `restore` lists the available backups (id, creation time, subscription count, prov row
|
|
2537
|
+
count, size) for keyboard selection. The **target database** comes from the active `--profile`'s
|
|
2538
|
+
section when it carries `db_*` keys, else `[dev]` for backup and `[restore]` for restore — so a typical
|
|
2539
|
+
restore-into-another-DB is `ligoj --profile restore dev backup restore --service service:prov`. Each section needs
|
|
2540
|
+
`db_host` / `db_port` / `db_name` / `db_user` / `db_password`, and the target must be a real Ligoj DB
|
|
2541
|
+
with the provisioning plugin installed (its parameter definitions and provider nodes must already
|
|
2542
|
+
exist).
|
|
2543
|
+
|
|
2544
|
+
## Harbor and ArgoCD on kind
|
|
2545
|
+
|
|
2546
|
+
Harbor and ArgoCD are the services that need a real cluster. They **share** a single-node **`kind`
|
|
2547
|
+
cluster** `ligoj-dev` (podman provider), created on first use with the host→NodePort mappings for
|
|
2548
|
+
**both** baked in (Harbor `8088→30088`, ArgoCD `8083→30083`) — mappings are immutable after creation,
|
|
2549
|
+
so the cluster carries all of them up front. Each service is then `helm upgrade --install`ed.
|
|
2550
|
+
|
|
2551
|
+
`dev init --only harbor` installs the Harbor chart in namespace `harbor` — a **minimal** install
|
|
2552
|
+
(`trivy` and `metrics` disabled), TLS off, `externalURL=http://localhost:8088`; stores
|
|
2553
|
+
`harbor_endpoint` and `harbor_admin_password`.
|
|
2554
|
+
|
|
2555
|
+
> **Apple Silicon (arm64)**: Harbor's official images are amd64-only. Most run under emulation, but
|
|
2556
|
+
> `goharbor/redis-photon` segfaults under QEMU, so `dev init` swaps the internal cache for a
|
|
2557
|
+
> multi-arch `redis` image (`HARBOR_REDIS_IMAGE`·`harbor_redis_image`, default `docker.io/redis:7.4.1`).
|
|
2558
|
+
> The remaining components run amd64-under-QEMU (functional, slower).
|
|
2559
|
+
|
|
2560
|
+
`dev init --only argocd` installs the ArgoCD chart in namespace `argocd` (insecure/HTTP, NodePort),
|
|
2561
|
+
then:
|
|
2562
|
+
|
|
2563
|
+
1. configures a **Dex LDAP connector** to the OpenLDAP pod — reachable from inside kind via the
|
|
2564
|
+
podman host IP (resolved from `host.containers.internal` on the node), bind
|
|
2565
|
+
`cn=<ldap_admin_user>,<ldap_root>`;
|
|
2566
|
+
2. creates a **`role:ligoj`** RBAC role (apps/projects/clusters/repos) bound to a `ligoj` account
|
|
2567
|
+
(`apiKey, login`) and the `ligoj` LDAP group;
|
|
2568
|
+
3. reads the admin password from `argocd-initial-admin-secret` and **generates an API token** for the
|
|
2569
|
+
`ligoj` account — both stored in `[dev]`.
|
|
2570
|
+
|
|
2571
|
+
> Adding ArgoCD to a cluster that only had Harbor (or vice-versa) requires `--recreate` once, since
|
|
2572
|
+
> the new service's port mapping must be baked into the cluster. After that both coexist.
|
|
2573
|
+
|
|
2574
|
+
> After a **podman machine restart** the kube-play pods and the kind node are left stopped; just
|
|
2575
|
+
> re-run `dev init` (or `dev init --only <kind-service>`) — it restarts the pods, starts the kind node
|
|
2576
|
+
> and refreshes its kubeconfig automatically.
|
|
2577
|
+
|
|
2578
|
+
Tear the cluster down with `kind delete cluster --name ligoj-dev` (or `dev init --only harbor argocd
|
|
2579
|
+
--recreate` to rebuild it).
|
|
2580
|
+
|
|
1654
2581
|
# Bootstrap
|
|
1655
2582
|
|
|
1656
2583
|
The following commands can be executed to perform several API commands following a complex workflow.
|
|
@@ -2318,33 +3245,62 @@ docker run -e CUSTOM_OPTS='-Djavax.net.ssl.trustStore=/home/ligoj/ligoj.jks' \
|
|
|
2318
3245
|
```
|
|
2319
3246
|
|
|
2320
3247
|
|
|
2321
|
-
# Development
|
|
3248
|
+
# Development
|
|
3249
|
+
|
|
3250
|
+
Everything goes through the [`Makefile`](Makefile), powered by
|
|
3251
|
+
[`uv`](https://docs.astral.sh/uv/). `make init` installs `uv` if it is missing, then creates the
|
|
3252
|
+
virtual environment and installs the exact runtime and dev dependencies from
|
|
3253
|
+
[`pyproject.toml`](pyproject.toml) — no manual `venv` / `pip` / `pyenv` steps.
|
|
3254
|
+
|
|
3255
|
+
```bash
|
|
3256
|
+
make init # install uv if needed + create the venv + install deps
|
|
3257
|
+
make run ARGS="--version" # run the CLI from the source tree
|
|
3258
|
+
```
|
|
3259
|
+
|
|
3260
|
+
Run `make help` to list every target:
|
|
3261
|
+
|
|
3262
|
+
| Command | Description |
|
|
3263
|
+
| ------------------- | ------------------------------------------------------- |
|
|
3264
|
+
| `make init` | Install uv if missing, create the venv, install deps |
|
|
3265
|
+
| `make run` | Run the CLI, e.g. `make run ARGS="info status"` |
|
|
3266
|
+
| `make format` | Auto-format and apply safe fixes with `ruff` |
|
|
3267
|
+
| `make lint` | Static analysis with `ruff` and `flake8` |
|
|
3268
|
+
| `make test` | Full local gate: lint, format check, build, twine check |
|
|
3269
|
+
| `make build` | Build the sdist and wheel into `dist/` |
|
|
3270
|
+
| `make release-test` | Publish a dev build to TestPyPI and wait until live |
|
|
3271
|
+
| `make release` | Cut a PyPI release (bump, tag, publish, wait) |
|
|
3272
|
+
| `make clean` | Remove build artifacts and caches |
|
|
3273
|
+
|
|
3274
|
+
# Releasing
|
|
2322
3275
|
|
|
2323
|
-
|
|
3276
|
+
Releases are driven entirely from `make`; the GitHub Actions workflows only build and publish via
|
|
3277
|
+
[PyPI Trusted Publishing](https://docs.pypi.org/trusted-publishers/) (OIDC, no API token). Both
|
|
3278
|
+
commands print step-by-step progress and block until the package is actually live on the index. See
|
|
3279
|
+
[RELEASE.md](RELEASE.md) for one-time setup and troubleshooting.
|
|
3280
|
+
|
|
3281
|
+
| Stage | Command | Index | Workflow |
|
|
3282
|
+
| ---------------- | ------------------- | ---------------------------------------------------- | ---------------------------------------------------- |
|
|
3283
|
+
| **Test publish** | `make release-test` | [TestPyPI](https://test.pypi.org/project/ligoj-cli/) | [deploy-test.yml](.github/workflows/deploy-test.yml) |
|
|
3284
|
+
| **Release** | `make release` | [PyPI](https://pypi.org/project/ligoj-cli/) | [deploy.yml](.github/workflows/deploy.yml) |
|
|
3285
|
+
|
|
3286
|
+
## Test publish (TestPyPI)
|
|
2324
3287
|
|
|
2325
3288
|
```bash
|
|
2326
|
-
|
|
2327
|
-
|
|
2328
|
-
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
|
|
2329
|
-
eval "$(pyenv init -)"' >> ~/.bashrc
|
|
2330
|
-
source ~/.bashrc
|
|
2331
|
-
brew install pyenv-virtualenv 3.11 ligoj # See https://github.com/pyenv/pyenv-virtualenv
|
|
2332
|
-
pyenv install 3.11
|
|
2333
|
-
pyenv virtualenv 3.11 ligoj
|
|
2334
|
-
pyenv activate ligoj
|
|
3289
|
+
make release-test
|
|
3290
|
+
```
|
|
2335
3291
|
|
|
2336
|
-
|
|
2337
|
-
|
|
2338
|
-
export http_proxy="http://10.154.154.154:3128"
|
|
2339
|
-
export https_proxy="http://10.154.154.154:3128"
|
|
2340
|
-
export NO_PROXY="localhost,*.rie.gouv.fr,127.0.0.1,0.0.0.0,ligoj.$TENANT"
|
|
3292
|
+
Pushes the current `HEAD` to `develop`, which builds a unique `.dev<run-number>` version and uploads
|
|
3293
|
+
it to TestPyPI. The command waits until that build is live and prints the install line.
|
|
2341
3294
|
|
|
2342
|
-
|
|
2343
|
-
pip install -U --root-user-action=ignore pip -e .
|
|
3295
|
+
## Release (PyPI)
|
|
2344
3296
|
|
|
2345
|
-
|
|
2346
|
-
|
|
2347
|
-
|
|
2348
|
-
ruff check . --fix
|
|
2349
|
-
flake8 .
|
|
3297
|
+
```bash
|
|
3298
|
+
make release # bump the minor version (e.g. 1.0.2 -> 1.1.0)
|
|
3299
|
+
make release PART=patch # or bump patch / major instead
|
|
2350
3300
|
```
|
|
3301
|
+
|
|
3302
|
+
This runs the full quality gate, bumps `version` in [pyproject.toml](pyproject.toml), commits, tags
|
|
3303
|
+
`vX.Y.Z`, pushes, creates the GitHub Release (which triggers
|
|
3304
|
+
[deploy.yml](.github/workflows/deploy.yml)), then waits until the version is live on
|
|
3305
|
+
[PyPI](https://pypi.org/project/ligoj-cli/). You are asked to confirm before anything is pushed —
|
|
3306
|
+
pass `YES=1` to skip the prompt.
|