bisos.airflow 0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,878 @@
1
+ Metadata-Version: 2.2
2
+ Name: bisos.airflow
3
+ Version: 0.1
4
+ Summary: bisos.airflow: Apache Airflow platform management via BISOS Capability Bundles
5
+ Author: Mohsen Banan
6
+ Author-email: libre@mohsen.1.banan.byname.net
7
+ Maintainer: Mohsen Banan
8
+ Maintainer-email: libre@mohsen.1.banan.byname.net
9
+ License: AGPL
10
+ Classifier: Development Status :: 5 - Production/Stable
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: License :: OSI Approved :: GNU Affero General Public License v3
13
+ Classifier: Operating System :: POSIX
14
+ Classifier: Programming Language :: Python
15
+ Classifier: Topic :: Software Development :: Libraries
16
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
17
+ Requires-Dist: bisos
18
+ Requires-Dist: bisos.b
19
+ Requires-Dist: bisos.banna
20
+ Requires-Dist: bisos.basics
21
+ Requires-Dist: bisos.capability
22
+ Requires-Dist: bisos.common
23
+ Requires-Dist: bisos.csPlayer
24
+ Requires-Dist: bisos.csSeed
25
+ Requires-Dist: bisos.debian
26
+ Requires-Dist: bisos.dnsCap
27
+ Requires-Dist: bisos.sbom
28
+ Requires-Dist: bisos.webCap
29
+ Requires-Dist: setuptools==75.8.0
30
+ Requires-Dist: wheel==0.38.4
31
+ Dynamic: author
32
+ Dynamic: author-email
33
+ Dynamic: classifier
34
+ Dynamic: description
35
+ Dynamic: license
36
+ Dynamic: maintainer
37
+ Dynamic: maintainer-email
38
+ Dynamic: requires-dist
39
+ Dynamic: summary
40
+
41
+ ==============================================================================
42
+ bisos.airflow: Apache Airflow platform management via BISOS Capability Bundles
43
+ ==============================================================================
44
+
45
+ .. contents::
46
+ :depth: 3
47
+ ..
48
+
49
+ Overview
50
+ ========
51
+
52
+ *bisos.airflow* is a BISOS package for managing an Apache Airflow
53
+ platform (webserver, scheduler, triggerer, database, and DAG
54
+ development/testing) as a set of BISOS Capability Bundles (CBS/CBM) and
55
+ ``systemd``-planted Command Services. It is under active initial
56
+ development; this README is a starting point and will be expanded as the
57
+ package matures.
58
+
59
+ **The remainder of this document is carried over from**
60
+ ``bisos.dockerProc`` \*(the starting-point template this package was
61
+ copied from) and still describes docker/podman container lifecycle
62
+ management. It will be rewritten to describe bisos.airflow as the
63
+ package develops.\*
64
+
65
+ Capability Materialization
66
+ ==========================
67
+
68
+ The figure below shows the top-down layering of ``airflow.here``'s
69
+ Capability Materialization: the planted CBM pointer leaf
70
+ (``/bisos/platform/sys/cbm/collective/hereWeb/airflow/cbmProc.spcs``)
71
+ points to (and does **not** duplicate) the CBS
72
+ (``py3/bin/airflow-cbs.pcs``), which declares the bundle of
73
+ ``airflow-sbom.pcs``, the 4 ``airflow-*-sysd.pcs`` units, and
74
+ ``airflow-assemble.cs``. ``py3/bin/airflowAdmin.cs`` sits below all of
75
+ that as the day-to-day admin-facing CS an operator runs against the
76
+ materialized installation.
77
+
78
+ **Regenerate this figure** (after editing
79
+ ``py3/images/airflow-graphviz.pcs``):
80
+
81
+ ::
82
+
83
+ cd py3/images
84
+ ./airflow-graphviz.pcs --format=all -i ngProcess all
85
+
86
+ | The companion image-specifications repo lives at
87
+ | https://github.com/bxObjects/bro_dockerfiles — a set of Dockerfiles
88
+ for XFCE4 desktops over VNC / noVNC / SSH on Debian 12 and 13,
89
+ organised by init/isolation profile.
90
+
91
+ *bisos.dockerProc* is a python package that uses the
92
+ `PyCS-Framework <https://github.com/bisos-pip/pycs>`__. It is a
93
+ BISOS-Capability and a Standalone-BISOS-Package.
94
+
95
+ **Architectural context.** ``bisos.dockerProc`` is **one realization**
96
+ of a general BISOS facility for building walkable trees on top of
97
+ ``.spcs`` (Spread Planted Command Services) — currently the first
98
+ realization, with ``bisos.lcnt`` (LaTeX content), static-web
99
+ publication, and others planned. The ``.spcs`` mechanism itself lives in
100
+ ``bisos.csSeed``; the tree walker and ``WalkExampleSpec`` dataclass live
101
+ in ``bisos.fileObj``; the architecture that composes them (three-box
102
+ layered figure, authoring recipe, canonical references) is documented at
103
+ ```bisos.pycs/spcs/README.org`` </bisos/git/auth/bxRepos/bisos-pip/pycs/spcs/README.org>`__.
104
+ This README describes what makes *this* realization specific —
105
+ path-anchoring on ``debian/``, per-profile ports, cgroup variants,
106
+ container-in-container safety, backup workflow, the six-image matrix of
107
+ the companion ``bro_dockerfiles`` tree.
108
+
109
+ Package Documentation At Github
110
+ ===============================
111
+
112
+ The information below is a subset of the full of documentation for this
113
+ bisos-pip package. More complete documentation is available at:
114
+ https://github.com/bisos-pip/capability-cs
115
+
116
+ Use Cases
117
+ =========
118
+
119
+ *bisos.dockerProc* addresses several deployment scenarios in which the
120
+ usual "one Dockerfile, one ``docker run`` line" pattern falls short. If
121
+ any of the following describes your situation, this package is built for
122
+ you.
123
+
124
+ RawBisos-Reconstruction — BISOS's Equivalent of the Common CI/CD Pipeline
125
+ -------------------------------------------------------------------------
126
+
127
+ **BISOS's equivalent of the common CI/CD pipeline is called
128
+ RawBisos-Reconstruction.** Two execution substrates exist:
129
+
130
+ - **Vagrant-based variant** (canonical, in ``bxGenesis/start``) —
131
+ provisions a full VM from scratch, runs
132
+ ``raw-bisos.sh -i installUnsitedBisos`` against it, validates the
133
+ resulting environment. Heavyweight; suitable for full-release
134
+ validation.
135
+ - **Container-based variant** (this package + ``bro_dockerfiles``) —
136
+ provisions a fresh Debian container, runs ``installRawBisos.sh``
137
+ inside it, validates the resulting environment. Lightweight; suitable
138
+ for on-demand validation during development.
139
+
140
+ The container-based variant runs entirely from planted ``.spcs`` files.
141
+ No CI service, no VM lifecycle. The six ``bro_dockerfiles`` leaves
142
+ (deb12/deb13 × confined/privileged/rootless-sysd) are the test matrix.
143
+
144
+ **Typical one-shot per variant** (from any leaf):
145
+
146
+ ::
147
+
148
+ cd bro_dockerfiles/debian/12/confined/vnc/xfce/bisos_deb12-fresh
149
+ ./dockerProc.spcs -i containerProc_imageBuild
150
+ ./dockerProc.spcs -i containerProc_instanceUp
151
+ ./dockerProc.spcs -i containerProc_exec_installRawBisos # bxGenesis bootstrap inside container
152
+ ./dockerProc.spcs -i containerProc_instanceVerify
153
+ ./dockerProc.spcs -i containerProc_fullClean
154
+
155
+ Or **across the whole matrix** from ``bro_dockerfiles/debian/``:
156
+
157
+ ::
158
+
159
+ ./ftoBranchProc.spcs -i fto_forwardToLeaves --cmndName=containerProc_exec_installRawBisos
160
+
161
+ Each of the six leaves subprocess-runs the bootstrap independently (via
162
+ ``bisos.fileObj`` Stage 2 Deliverable 6 subprocess-per-branch). Results
163
+ stream to stdout; a failing bootstrap on one variant does not mask
164
+ successful bootstraps on the others.
165
+
166
+ The ``installRawBisos.sh`` script (baked into each image at
167
+ ``~/raw-bisos/``) refreshes ``raw-bisos.sh`` from ``bxGenesis/start``
168
+ via wget and invokes it non-interactively with
169
+ ``-v -n showRun -i installUnsitedBisos``. This means every
170
+ reconstruction is against the **current** head of ``bxGenesis/start``,
171
+ not a pinned snapshot — catches regressions as they land.
172
+
173
+ Reproducible XFCE Desktops on Debian 12 and 13
174
+ ----------------------------------------------
175
+
176
+ You want an XFCE4 desktop, accessible via VNC / noVNC / SSH, built the
177
+ same way on Debian 12 and Debian 13 so team members on either release
178
+ get a consistent environment. Instead of maintaining two divergent
179
+ Dockerfiles, ``bro_dockerfiles`` has one leaf per (release × profile ×
180
+ image), and ``dockerProc.spcs`` builds and runs any of them from its own
181
+ directory. Same commands, different leaves.
182
+
183
+ Choosing Among Three Init/Isolation Profiles for the Same Image
184
+ ---------------------------------------------------------------
185
+
186
+ The same desktop content can be shipped under three fundamentally
187
+ different container postures — and which one to use depends on the
188
+ host's security policy, not the image content:
189
+
190
+ - ``confined`` — unprivileged docker; no systemd; entrypoint script
191
+ launches the services. Runs anywhere docker runs.
192
+ - ``privileged`` — docker with ``--privileged``; systemd as PID 1;
193
+ services managed as systemd units. Full systemd fidelity, at the cost
194
+ of a privileged container.
195
+ - ``rootless-sysd`` — rootless podman; systemd as PID 1; unprivileged
196
+ posture. Requires cgroup v2 with controller delegation on the host.
197
+ Delivers systemd fidelity **and** an unprivileged security posture.
198
+
199
+ The three profiles sit side-by-side in the ``bro_dockerfiles`` tree — a
200
+ single leaf-path change switches profile. This is unusual: most projects
201
+ pick one and hide the others.
202
+
203
+ Multi-Tenant: N Engineers on One Powerful VM
204
+ --------------------------------------------
205
+
206
+ One VM, many users, each user running one or more container instances of
207
+ their own. This is the scenario that motivates the Platform Registrar
208
+ (design-only in this phase; see `Full Design at a
209
+ Glance <#full-design-at-a-glance>`__): per-user, per-image, per-instance
210
+ port assignments so users don't collide on the fixed 2222/5901/6901
211
+ host-side port table.
212
+
213
+ The rootless-sysd profile is what makes this scenario safe — each user's
214
+ containers run in their own user namespace with no ``--privileged``
215
+ flag, so isolation between users is real and not a policy promise.
216
+
217
+ Corporate Firewall: Only Port 22 Outbound
218
+ -----------------------------------------
219
+
220
+ A common corporate deployment target: engineers' workstations can reach
221
+ the internet, but only outbound port 22 is permitted. VNC, noVNC, and
222
+ SSH-alt ports to the container platform are blocked. The Remote User's
223
+ ``.ssh/config`` is populated (from the registrar) with ``LocalForward``
224
+ entries, so all container-side ports are tunnelled through the single
225
+ port-22 ssh connection. The user then points local ssh / vnc / browser
226
+ clients at ``localhost:<forwarded-port>``.
227
+
228
+ Path-Derived Configuration Instead of Per-Leaf Config Files
229
+ -----------------------------------------------------------
230
+
231
+ You want to add a new image without writing a new configuration file. In
232
+ ``bro_dockerfiles`` the leaf's **path**
233
+ (``debian/<release>/<profile>/vnc/xfce/<imageName>/``) supplies engine,
234
+ profile, ports, base image, and cgroup support — no ``config.yaml``, no
235
+ environment file, no shell variables to set. Copy an existing leaf to a
236
+ new path, drop in a Dockerfile, plant the appropriate ``.spcs``, and the
237
+ seed does the rest. Configuration lives in filesystem structure, not in
238
+ files.
239
+
240
+ Full Design at a Glance
241
+ =======================
242
+
243
+ The figure below captures the **full design** of ``bisos.dockerProc`` —
244
+ the three execution platforms it spans, the pieces on each, and the
245
+ flows between them. Part of the design (the **Platform Registrar**) is
246
+ not yet implemented and is shown with a dashed border. Scan the legend
247
+ first, then read the figure.
248
+
249
+ **Legend:**
250
+
251
+ +---------------------------+-----------------------------------------+
252
+ | Element | Meaning |
253
+ +===========================+=========================================+
254
+ | Light-blue cluster (left) | **Remote User** — corporate host, only |
255
+ | | port 22 outbound. |
256
+ +---------------------------+-----------------------------------------+
257
+ | Pink cluster (middle) | **Container Platform** — the powerful |
258
+ | | VM; hosts engines, ``.spcs``, |
259
+ | | registrar. |
260
+ +---------------------------+-----------------------------------------+
261
+ | Green cluster (right) | **Container Instances** — one image, |
262
+ | | many running instances per user. |
263
+ +---------------------------+-----------------------------------------+
264
+ | Dashed border (registrar) | **Design-only in this phase** — shape |
265
+ | | reserved, not yet implemented. |
266
+ +---------------------------+-----------------------------------------+
267
+ | Yellow cylinder | Persistent store / path-derived |
268
+ | | identity source. |
269
+ +---------------------------+-----------------------------------------+
270
+ | Salmon ``rarrow`` shape | CS / ``.spcs`` dispatcher. |
271
+ +---------------------------+-----------------------------------------+
272
+ | Blue rectangle | Engine / CLI parameter / port. |
273
+ +---------------------------+-----------------------------------------+
274
+ | Purple edge | ssh tunnel path (Remote User → Platform |
275
+ | | via port 22 → Instances). |
276
+ +---------------------------+-----------------------------------------+
277
+ | Blue dashed edge | Registrar read (design-only; two |
278
+ | | consumers: ``.spcs`` and Remote User). |
279
+ +---------------------------+-----------------------------------------+
280
+ | Dotted "planted in" edge | Which leaves in ``bro_dockerfiles`` |
281
+ | | plant which ``.spcs``. |
282
+ +---------------------------+-----------------------------------------+
283
+
284
+ **Reading the figure, left to right:**
285
+
286
+ - **Container Platform (middle) — the anchor.** This is the powerful VM
287
+ where everything runs. Image identity comes from the leaf directory
288
+ path in ``bro_dockerfiles``
289
+ (``debian/<release>/<profile>/vnc/xfce/<imageName>/``):
290
+ ``dockerProc.spcs`` is planted in ``confined`` and ``privileged``
291
+ leaves; ``podmanProc.spcs`` is planted in ``rootless-sysd`` leaves.
292
+ Both delegate to the engine-agnostic seed ``containerProc-seed.cs``,
293
+ which drives docker or podman according to the leaf's profile.
294
+
295
+ - **Platform Registrar (middle, dashed) — design-only in this phase.**
296
+ The current implementation derives host-side ports from the leaf path
297
+ (fixed 2222/2223/… table). This works for one user with one instance
298
+ per profile. The moment you have multiple users on one VM — or
299
+ multiple instances of the same image per user — fixed ports collide.
300
+ The Registrar is the reserved slot for that: a per-user, per-image,
301
+ per-instance store at
302
+ ``/bisos/platform/registrar/dockerProc/<user>/<imageName>/instance0..N/``
303
+ that assigns and tracks host-side ports. Path gives the **image**
304
+ identity; registrar gives the **instance** identity.
305
+
306
+ - **Two consumers of the registrar** (both drawn as blue dashed edges).
307
+ ``.spcs`` reads assigned ports on the platform side to do ``run`` /
308
+ ``composeUp`` / ``verify`` for the right host-side ports. Remote User
309
+ reads the same assignments (over ssh) to generate its local
310
+ ``.ssh/config`` ``LocalForward`` entries so its ssh client tunnels
311
+ the right host ports to the right instances.
312
+
313
+ - **Remote User (left) — corporate host, port-22-only.** Typical
314
+ deployment target: a workstation behind corporate networking where
315
+ everything but outbound port 22 is blocked. The user cannot open
316
+ VNC/noVNC/SSH-alt ports directly to the platform. Instead, the Remote
317
+ User's ``.ssh/config`` is populated with ``LocalForward`` entries
318
+ that tunnel the assigned host-side ports through the single port-22
319
+ ssh connection. The user then points local ssh/vnc/browser clients at
320
+ ``localhost:<forwarded-port>``. The ``.ssh/config`` entries are a
321
+ generated artefact: their content comes from the registrar.
322
+
323
+ - **Container Instances (right) — many per user.** Each running
324
+ container exposes the same fixed **container-side** ports (22 for
325
+ ssh, 5901 for VNC, 6901 for noVNC). What varies per instance is the
326
+ **host-side** mapping. The figure shows ``image-A / inst0``,
327
+ ``inst1``, ``image-B / inst0``, ``... instN`` as representative — one
328
+ image can have many instances; a user can have many images.
329
+
330
+ **What is implemented today vs what the figure describes:**
331
+
332
+ - Implemented: image path → ``.spcs`` → seed → engine → running
333
+ container. The ``dockerProc.spcs`` and ``podmanProc.spcs`` planting,
334
+ the seed's ``build`` / ``composeUp`` / ``run`` / ``verify`` commands,
335
+ and ``podmanHostVerify.cs`` for host readiness are all in place.
336
+ Verified on both docker (deb13 privileged sysd, cgroup v1 + v2) and
337
+ podman (deb13 rootless-sysd).
338
+ - Not yet implemented: everything downstream of **Platform Registrar**.
339
+ Its shape is fixed in the figure so subsequent multi-tenant work can
340
+ add it without redesign.
341
+
342
+ .. _table-of-contents:
343
+
344
+ Table of Contents TOC
345
+ =====================
346
+
347
+ - `Overview <#overview>`__
348
+ - `Capability Materialization <#capability-materialization>`__
349
+ - `Package Documentation At
350
+ Github <#package-documentation-at-github>`__
351
+ - `Use Cases <#use-cases>`__
352
+
353
+ - `RawBisos-Reconstruction — BISOS's Equivalent of the Common CI/CD
354
+ Pipeline <#rawbisos-reconstruction-----bisoss-equivalent-of-the-common-cicd-pipeline>`__
355
+ - `Reproducible XFCE Desktops on Debian 12 and
356
+ 13 <#reproducible-xfce-desktops-on-debian-12-and-13>`__
357
+ - `Choosing Among Three Init/Isolation Profiles for the Same
358
+ Image <#choosing-among-three-initisolation-profiles-for-the-same-image>`__
359
+ - `Multi-Tenant: N Engineers on One Powerful
360
+ VM <#multi-tenant-n-engineers-on-one-powerful-vm>`__
361
+ - `Corporate Firewall: Only Port 22
362
+ Outbound <#corporate-firewall-only-port-22-outbound>`__
363
+ - `Path-Derived Configuration Instead of Per-Leaf Config
364
+ Files <#path-derived-configuration-instead-of-per-leaf-config-files>`__
365
+
366
+ - `Full Design at a Glance <#full-design-at-a-glance>`__
367
+ - `Part of BISOS — ByStar Internet Services Operating
368
+ System <#part-of-bisos--bystar-internet-services-operating-system>`__
369
+ - `bisos.dockerProc is a Command-Only PyCS
370
+ Facility <#bisosdockerproc-is-a-command-only-pycs-facility>`__
371
+ - `The spcs Pattern: Directory Path as
372
+ Configuration <#the-spcs-pattern-directory-path-as-configuration>`__
373
+
374
+ - `Companion Repo:
375
+ bro\ dockerfiles <#companion-repo-bro_dockerfiles>`__
376
+ - `Path-to-parameter Mapping <#path-to-parameter-mapping>`__
377
+
378
+ - `Three Init/Isolation Profiles <#three-initisolation-profiles>`__
379
+ - `Installation <#installation>`__
380
+
381
+ - `Installation With pip <#installation-with-pip>`__
382
+ - `Installation With pipx <#installation-with-pipx>`__
383
+
384
+ - `Usage <#usage>`__
385
+
386
+ - `Planting a ``.spcs`` File in a Leaf
387
+ Directory <#planting-a-spcs-file-in-a-leaf-directory>`__
388
+ - `The Seed Commands: image, instance, verify+status,
389
+ combined <#the-seed-commands-image-instance-verifystatus-combined>`__
390
+ - `Host Readiness Check:
391
+ ``podmanHostVerify.cs`` <#host-readiness-check-podmanhostverifycs>`__
392
+ - `Docker / Podman Installation via ``-sbom.pcs``
393
+ Files <#docker--podman-installation-via--sbompcs-files>`__
394
+ - `Cheat Sheet: ``dockerCmnds.cs`` /
395
+ ``podmanCmnds.cs`` <#cheat-sheet-dockercmndscs--podmancmndscs>`__
396
+
397
+ - `Key Files <#key-files>`__
398
+ - `Documentation and Blee-Panels <#documentation-and-blee-panels>`__
399
+ - `Support <#support>`__
400
+
401
+ Part of BISOS — ByStar Internet Services Operating System
402
+ =========================================================
403
+
404
+ Layered on top of Debian, **BISOS** (By\* Internet Services Operating
405
+ System) is a unified and universal framework for developing both
406
+ internet services and software-service continuums that use internet
407
+ services. See `Bootstrapping ByStar, BISOS and
408
+ Blee <https://github.com/bxGenesis/start>`__ for information about
409
+ getting started with BISOS.
410
+
411
+ **BISOS** is a foundation for **The Libre-Halaal ByStar Digital
412
+ Ecosystem** which is described as a cure for losses of autonomy and
413
+ privacy in a book titled: `Nature of
414
+ Polyexistentials <https://github.com/bxplpc/120033>`__
415
+
416
+ *bisos.dockerProc* is part of BISOS. It is a standalone package that can
417
+ be used independently of the full BISOS environment.
418
+
419
+ bisos.dockerProc is a Command-Only PyCS Facility
420
+ ================================================
421
+
422
+ bisos.dockerProc is a command-line tool. It is a PyCS multi-unit command
423
+ service. PyCS is a framework that converges development of CLI tools and
424
+ services. PyCS is an alternative to FastAPI, Typer and Click.
425
+
426
+ bisos.dockerProc uses the PyCS-Framework to:
427
+
428
+ #. Provide a **seed** (``containerProc-seed.cs``) and two **planted**
429
+ command services (``dockerProc.spcs`` for docker leaves,
430
+ ``podmanProc.spcs`` for rootless-sysd leaves) that manage container
431
+ image build, compose, run, verify, status, and clean.
432
+ #. Derive all operating parameters from the leaf directory path in which
433
+ each ``.spcs`` is planted — there are no per-leaf parameter files.
434
+ #. Provide standalone helpers: ``podmanHostVerify.cs`` (host readiness
435
+ check for rootless-sysd), ``dockerProc-sbom.pcs`` /
436
+ ``podman-sbom.pcs`` (install docker / podman via bisos.sbom), and
437
+ ``dockerCmnds.cs`` / ``podmanCmnds.cs`` (cheat sheet of direct-engine
438
+ invocations).
439
+
440
+ The core of PyCS-Framework is the
441
+ `bisos.b <https://github.com/bisos-pip/b>`__ package (the
442
+ PyCS-Foundation).
443
+
444
+ The spcs Pattern: Directory Path as Configuration
445
+ =================================================
446
+
447
+ /=.spcs= is a general BISOS facility, not a ``bisos.dockerProc``
448
+ invention. For the architectural picture — three-box layered figure, the
449
+ ``WalkExampleSpec`` typed-data mechanism, the branch-side walker Cmnds
450
+ (``fto_forwardToLeaves``, ``fto_walkRunExternal``), and how to build a
451
+ new consumer — see
452
+ ```bisos.pycs/spcs/README.org`` </bisos/git/auth/bxRepos/bisos-pip/pycs/spcs/README.org>`__.
453
+ This section summarises how ``bisos.dockerProc`` uses ``.spcs``./
454
+
455
+ A ``.spcs`` file (**Spread Planted Command Service**) is a thin Python
456
+ file whose behaviour is entirely context-dependent on the directory in
457
+ which it is planted. The **same** file content is spread across many
458
+ directories; each instance is contextualised by its location. The leaf
459
+ directory path encodes all parameters — there are no per-leaf
460
+ configuration files.
461
+
462
+ Two ``.spcs`` variants exist here:
463
+
464
+ - ``dockerProc.spcs`` — planted in leaves that use docker.
465
+ - ``podmanProc.spcs`` — planted in leaves that use rootless podman.
466
+
467
+ Both delegate to the same engine-agnostic seed,
468
+ ``containerProc-seed.cs``, which parses the leaf path (via
469
+ ``bisos.dockerProc.containerProc_seedInfo.paramsFromPlantPath()``) to
470
+ derive engine, profile, ports, base image, and cgroup support.
471
+
472
+ Companion Repo: bro\ :sub:`dockerfiles`
473
+ ---------------------------------------
474
+
475
+ | The image specifications live in a separate repo,
476
+ | https://github.com/bxObjects/bro_dockerfiles. Its directory hierarchy
477
+ encodes the configuration:
478
+
479
+ ::
480
+
481
+ debian/
482
+ <majorRelease>/ 12 or 13
483
+ confined/vnc/xfce/ unprivileged, entrypoint.sh init
484
+ <imageName>/
485
+ privileged/vnc/xfce/ --privileged (docker), systemd PID 1
486
+ <imageName>/
487
+ rootless-sysd/vnc/xfce/ rootless podman, systemd PID 1
488
+ <imageName>/
489
+
490
+ Clone it alongside your other repos — in BISOS the canonical location is
491
+ ``/bisos/git/bxRepos/bxObjects/``:
492
+
493
+ .. code:: bash
494
+
495
+ cd /bisos/git/bxRepos/bxObjects
496
+ git clone https://github.com/bxObjects/bro_dockerfiles.git
497
+
498
+ Path-to-parameter Mapping
499
+ -------------------------
500
+
501
+ Each segment of the leaf path
502
+ ``debian/<release>/<profile>/vnc/xfce/<imageName>/`` encodes one
503
+ dimension of the container's configuration:
504
+
505
+ +-----------------+---------------------------------------------------+
506
+ | Path segment | Parameter derived |
507
+ +=================+===================================================+
508
+ | ``debian`` | distro (anchor segment: ``paramsFromPlantPath()`` |
509
+ | | anchors on this) |
510
+ +-----------------+---------------------------------------------------+
511
+ | ``<release>`` | 12 or 13 → base OS, base image tag |
512
+ +-----------------+---------------------------------------------------+
513
+ | ``<profile>`` | ``confined`` / ``privileged`` / ``rootless-sysd`` |
514
+ | | → engine, init, privilege |
515
+ +-----------------+---------------------------------------------------+
516
+ | ``vnc/xfce`` | desktop type → VNC/noVNC ports, xstartup variant |
517
+ +-----------------+---------------------------------------------------+
518
+ | ``<imageName>`` | image name → DockerHub name, container name |
519
+ +-----------------+---------------------------------------------------+
520
+
521
+ From these, ``paramsFromPlantPath()`` derives: engine (docker vs
522
+ podman), base image name, host port assignments (ssh / vnc / novnc),
523
+ ``--privileged`` flag, ``--isolation=chroot`` for rootless podman
524
+ builds, ``--systemd=always`` for podman run, and cgroup-variant support.
525
+
526
+ Three Init/Isolation Profiles
527
+ =============================
528
+
529
+ The ``<profile>`` path segment selects one of three init/isolation
530
+ profiles:
531
+
532
+ +--------------+--------+--------------+--------------+--------------+
533
+ | Profile | Engine | Init | Privilege | Host cgroup |
534
+ +==============+========+==============+==============+==============+
535
+ | ``confined`` | docker | ``ent | unprivileged | v1 or v2 |
536
+ | | | rypoint.sh`` | | |
537
+ | | | (no systemd) | | |
538
+ +--------------+--------+--------------+--------------+--------------+
539
+ | `` | docker | `` | ``-- | v1 or v2 (v1 |
540
+ | privileged`` | | /sbin/init`` | privileged`` | needs cgv1 |
541
+ | | | (systemd PID | | overlay) |
542
+ | | | 1) | | |
543
+ +--------------+--------+--------------+--------------+--------------+
544
+ | ``roo | podman | `` | rootless | v2 only |
545
+ | tless-sysd`` | | /sbin/init`` | | |
546
+ | | | (systemd PID | | |
547
+ | | | 1) | | |
548
+ +--------------+--------+--------------+--------------+--------------+
549
+
550
+ - **confined**: unprivileged, no systemd; services (VNC / noVNC / sshd)
551
+ launched manually by ``entrypoint.sh``.
552
+ - **privileged**: full systemd as PID 1; runs ``--privileged`` on
553
+ docker. Requires the ``docker-compose.cgv1.yml`` overlay on cgroup-v1
554
+ hosts.
555
+ - **rootless-sysd**: full systemd as PID 1, but **unprivileged** via
556
+ rootless podman. Requires cgroup v2 with controller delegation.
557
+ Motivated by the multi-tenant "N engineers on one shared VM" use case
558
+ where ``--privileged`` is a security dealbreaker.
559
+
560
+ Rootless-sysd delivers the **systemd fidelity** of the privileged
561
+ profile with the **security posture** of the confined profile. It is the
562
+ target model for multi-tenant use once the Platform Registrar (see the
563
+ `Full Design at a Glance <#full-design-at-a-glance>`__ figure) is
564
+ implemented.
565
+
566
+ Installation
567
+ ============
568
+
569
+ | The sources for the bisos.dockerProc pip package are maintained at:
570
+ | https://github.com/bisos-pip/dockerProc
571
+
572
+ | The bisos.dockerProc pip package is available at PYPI as
573
+ | https://pypi.org/project/bisos.dockerProc
574
+
575
+ You can install bisos.dockerProc with pip or pipx.
576
+
577
+ Installation With pip
578
+ ---------------------
579
+
580
+ If you need access to bisos.dockerProc as a python module, install it
581
+ with pip:
582
+
583
+ .. code:: bash
584
+
585
+ pip install bisos.dockerProc
586
+
587
+ Installation With pipx
588
+ ----------------------
589
+
590
+ If you only need access to bisos.dockerProc on the command line, install
591
+ it with pipx:
592
+
593
+ .. code:: bash
594
+
595
+ pipx install bisos.dockerProc
596
+
597
+ The following commands are made available:
598
+
599
+ - ``containerProc-seed.cs`` — the engine-agnostic seed both ``.spcs``
600
+ files delegate to.
601
+ - ``dockerProc.spcs`` — Spread Planted CS for docker leaves (confined +
602
+ privileged).
603
+ - ``podmanProc.spcs`` — Spread Planted CS for rootless-sysd leaves.
604
+ - ``dockerCmnds.cs`` / ``podmanCmnds.cs`` — cheat-sheet of
605
+ direct-engine invocations (podmanCmnds.cs is a symlink to
606
+ dockerCmnds.cs; the CS dispatches on ``argv[0]``).
607
+ - ``podmanHostVerify.cs`` — host readiness check for rootless-sysd
608
+ containers.
609
+ - ``dockerProc-sbom.pcs`` / ``podman-sbom.pcs`` — install docker /
610
+ podman via `bisos.sbom <https://github.com/bisos-pip/sbom>`__.
611
+
612
+ Usage
613
+ =====
614
+
615
+ The primary workflow is: clone ``bro_dockerfiles``, ``cd`` into a leaf
616
+ directory, and run ``dockerProc.spcs`` or ``podmanProc.spcs`` there. The
617
+ ``.spcs`` file is already planted in each leaf as part of the companion
618
+ repo.
619
+
620
+ Planting a ``.spcs`` File in a Leaf Directory
621
+ ---------------------------------------------
622
+
623
+ For most users this is already done — the ``bro_dockerfiles`` repo ships
624
+ with ``dockerProc.spcs`` planted in each of the four docker leaves and
625
+ ``podmanProc.spcs`` planted in each of the two rootless-sysd leaves. If
626
+ you create a new leaf, plant the appropriate ``.spcs``:
627
+
628
+ .. code:: bash
629
+
630
+ cp $(which dockerProc.spcs) .
631
+
632
+ cp $(which podmanProc.spcs) .
633
+
634
+ chmod +x dockerProc.spcs # or podmanProc.spcs
635
+
636
+ The leaf's path (``debian/<release>/<profile>/vnc/xfce/<imageName>/``)
637
+ supplies all parameters.
638
+
639
+ The Seed Commands: image, instance, verify+status, combined
640
+ -----------------------------------------------------------
641
+
642
+ From any planted leaf, run the ``.spcs`` file with no arguments to see
643
+ the menu of common invocations — filtered to just the commands relevant
644
+ to that leaf (docker leaves don't show podman-only options, and vice
645
+ versa).
646
+
647
+ .. code:: bash
648
+
649
+ cd /bisos/git/bxRepos/bxObjects/bro_dockerfiles/debian/13/privileged/vnc/xfce/bisos_deb13-sysd
650
+ ./dockerProc.spcs
651
+
652
+ The Cmnd surface uses two noun prefixes — ``image*`` for commands that
653
+ operate on the container image, and ``instance*`` for commands that
654
+ operate on the running (or stopped) container instance. Plus a ``full*``
655
+ combined command.
656
+
657
+ **Image commands (image = the built artefact):**
658
+
659
+ +-------------------------------+-------------------------------------+
660
+ | Command | Purpose |
661
+ +===============================+=====================================+
662
+ | ``containerProc_imageBuild`` | ``docker build`` or |
663
+ | | ``podman build``. |
664
+ | | ``--noCache``"true"= for clean |
665
+ | | build. |
666
+ +-------------------------------+-------------------------------------+
667
+ | ``containerProc_imageDelete`` | ``rmi`` (image only; does NOT touch |
668
+ | | instances). |
669
+ +-------------------------------+-------------------------------------+
670
+
671
+ **Instance commands (instance = a container built from the image):**
672
+
673
+ +----------------------------------+----------------------------------+
674
+ | Command | Purpose |
675
+ +==================================+==================================+
676
+ | ``containerProc_instanceUp`` | Docker: |
677
+ | | ``docker compose up -d``. |
678
+ | | Podman: |
679
+ | | ``podman run --systemd=always``. |
680
+ +----------------------------------+----------------------------------+
681
+ | ``containerProc_instanceDown`` | Docker: ``docker compose down``. |
682
+ | | Podman: ``podman stop`` (does |
683
+ | | NOT rm). |
684
+ +----------------------------------+----------------------------------+
685
+ | ``containerProc_instanceDelete`` | Stop + remove instance (image |
686
+ | | preserved). |
687
+ +----------------------------------+----------------------------------+
688
+ | ` | Stop + start in place (state |
689
+ | `containerProc_instanceRestart`` | preserved). |
690
+ +----------------------------------+----------------------------------+
691
+ | ``containerProc_instancePs`` | ``ps -a`` filtered to this |
692
+ | | leaf's container name. |
693
+ +----------------------------------+----------------------------------+
694
+ | ``containerProc_instanceLogs`` | ``logs`` (``--follow``"true"= to |
695
+ | | stream). |
696
+ +----------------------------------+----------------------------------+
697
+ | ``containerProc_instanceExec`` | ``exec -it <container> bash`` |
698
+ | | (or ``--execCmd``\ …= for other |
699
+ | | command). |
700
+ +----------------------------------+----------------------------------+
701
+
702
+ **Verify + status:**
703
+
704
+ +----------------------------------+----------------------------------+
705
+ | Command | Purpose |
706
+ +==================================+==================================+
707
+ | ``containerProc_instanceVerify`` | Host-side smoke test: port + |
708
+ | | noVNC HTTP + SSH-based systemd |
709
+ | | checks. |
710
+ +----------------------------------+----------------------------------+
711
+ | ``containerProc_instanceStatus`` | Engine inspect + SSH systemd |
712
+ | | status. |
713
+ +----------------------------------+----------------------------------+
714
+
715
+ **Combined:**
716
+
717
+ +-----------------------------+---------------------------------------+
718
+ | Command | Purpose |
719
+ +=============================+=======================================+
720
+ | ``containerProc_fullClean`` | ``instanceDelete`` + ``imageDelete`` |
721
+ | | (from-scratch rebuild). |
722
+ +-----------------------------+---------------------------------------+
723
+
724
+ Additional notes:
725
+
726
+ - ``imageBuild`` on a rootless-sysd leaf auto-builds the confined base
727
+ image if it is missing from podman's store.
728
+ - ``instanceVerify`` is **exec-free** for rootless (uses SSH instead of
729
+ ``podman exec``, which is unreliable on old Podman for systemd
730
+ containers).
731
+ - For docker leaves, ``--cgroupVer``"v1"= selects
732
+ ``docker-compose.cgv1.yml`` when the host is on cgroup v1.
733
+ - The old flat names (``build``, ``composeUp``, ``composeDown``,
734
+ ``run``, ``verify``, ``status``, ``clean``) are kept as deprecated
735
+ aliases for one release — they still work but emit a
736
+ ``DeprecationWarning``.
737
+
738
+ Example — build and start the deb13 privileged image on a cgroup-v2
739
+ host:
740
+
741
+ .. code:: bash
742
+
743
+ cd /bisos/git/bxRepos/bxObjects/bro_dockerfiles/debian/13/privileged/vnc/xfce/bisos_deb13-sysd
744
+ ./dockerProc.spcs -i containerProc_imageBuild
745
+ ./dockerProc.spcs -i containerProc_instanceUp
746
+ ./dockerProc.spcs -i containerProc_instanceVerify
747
+
748
+ On a cgroup-v1 host (e.g. RHEL 8), pass ``cgroupVer=v1`` to instance
749
+ up/down:
750
+
751
+ .. code:: bash
752
+
753
+ ./dockerProc.spcs -i containerProc_instanceUp --cgroupVer="v1"
754
+ ./dockerProc.spcs -i containerProc_instanceDown --cgroupVer="v1"
755
+
756
+ For rootless-sysd (podman) leaves:
757
+
758
+ .. code:: bash
759
+
760
+ cd /bisos/git/bxRepos/bxObjects/bro_dockerfiles/debian/13/rootless-sysd/vnc/xfce/bisos_deb13-rootless-sysd
761
+ ./podmanProc.spcs -i containerProc_imageBuild
762
+ ./podmanProc.spcs -i containerProc_instanceUp --detach="true"
763
+ ./podmanProc.spcs -i containerProc_instanceVerify
764
+
765
+ Host Readiness Check: ``podmanHostVerify.cs``
766
+ ---------------------------------------------
767
+
768
+ Before running rootless-sysd containers on a host for the first time,
769
+ run ``podmanHostVerify.cs -i verify`` to check the host meets the
770
+ requirements: non-root user, podman installed, cgroup v2, crun OCI
771
+ runtime, subuid/subgid ranges, XDG\ :sub:`RUNTIMEDIR` + user
772
+ systemd/D-Bus session, linger enabled, cgroup-v2 controller delegation,
773
+ graphroot on local disk with free space.
774
+
775
+ .. code:: bash
776
+
777
+ podmanHostVerify.cs -i verify
778
+
779
+ Reports PASS / WARN / FAIL for each check and an overall GO / NO-GO exit
780
+ status.
781
+
782
+ Docker / Podman Installation via ``-sbom.pcs`` Files
783
+ ----------------------------------------------------
784
+
785
+ Two ``.pcs`` files use
786
+ `bisos.sbom <https://github.com/bisos-pip/sbom>`__ to install docker or
787
+ podman on a Debian host with all their supporting packages:
788
+
789
+ .. code:: bash
790
+
791
+ dockerProc-sbom.pcs -i sbom_apt_install # installs docker-ce + friends
792
+ podman-sbom.pcs -i sbom_apt_install # installs podman + rootless deps
793
+
794
+ The docker sbom adds Docker's official apt repository first (mirrors
795
+ what ``dockerInstall.sh`` used to do); the podman sbom uses Debian's own
796
+ packages.
797
+
798
+ Cheat Sheet: ``dockerCmnds.cs`` / ``podmanCmnds.cs``
799
+ ----------------------------------------------------
800
+
801
+ For direct docker / podman command references (not routed through the
802
+ seed — just a documented list of the underlying engine commands), run:
803
+
804
+ .. code:: bash
805
+
806
+ dockerCmnds.cs
807
+ podmanCmnds.cs
808
+
809
+ Both print an examples menu covering inspect / images / run / compose /
810
+ exec / cleanup. ``podmanCmnds.cs`` is a symlink to ``dockerCmnds.cs``;
811
+ the CS dispatches on ``argv[0]`` to select the docker or podman variant.
812
+ The podman menu also includes rootless-sysd-specific commands
813
+ (``--systemd=always``, ``--isolation=chroot``, cgroup check, linger,
814
+ host verify).
815
+
816
+ Key Files
817
+ =========
818
+
819
+ An overview of the relevant files of the bisos.dockerProc package
820
+ (starting-point template; to be updated to reflect bisos.airflow's
821
+ actual files):
822
+
823
+ - ``py3/bin/containerProc-seed.cs`` — the engine-agnostic seed both
824
+ ``.spcs`` delegate to.
825
+ - ``py3/bin/dockerProc.spcs`` — Spread Planted CS for docker leaves.
826
+ - ``py3/bin/podmanProc.spcs`` — Spread Planted CS for rootless-sysd
827
+ leaves.
828
+ - ``py3/bin/podmanHostVerify.cs`` — host readiness check for
829
+ rootless-sysd.
830
+ - ``py3/bin/dockerProc-sbom.pcs`` / ``py3/bin/podman-sbom.pcs`` —
831
+ engine install via
832
+ `bisos.sbom <https://github.com/bisos-pip/sbom>`__.
833
+ - ``py3/bin/dockerCmnds.cs`` / ``py3/bin/podmanCmnds.cs`` — cheat-sheet
834
+ CS (symlinked).
835
+ - ``py3/bisos/dockerProc/containerProc_seedInfo.py`` —
836
+ ``Engine=/=Profile=/=CgroupVer`` enums, ``ContainerParams``
837
+ dataclass, ``paramsFromPlantPath()`` pure function.
838
+ - ``py3/bisos/dockerProc/containerProc_seed.py`` — atexit registration.
839
+ - ``py3/bisos/dockerProc/containerProc_csu.py`` — CS command
840
+ implementations. Cmnds are organized by noun:
841
+ ``containerProc_image*`` (build/delete), ``containerProc_instance*``
842
+ (up/down/delete/restart/ps/logs/exec/verify/status), and
843
+ ``containerProc_fullClean`` (combined).
844
+ - ``py3/images/containerProc-graphviz.pcs`` — source for the "Full
845
+ Design at a Glance" figure.
846
+ - ``py3/setup.py``, ``py3/pypiProc.sh`` — PyPI packaging (setup.py is
847
+ dblock-driven; do not hand-edit).
848
+
849
+ Documentation and Blee-Panels
850
+ =============================
851
+
852
+ bisos.airflow is part of the ByStar Digital Ecosystem
853
+ http://www.by-star.net.
854
+
855
+ This module's primary documentation is in the form of Blee-Panels.
856
+ Blee-Panels are in the ``./panels`` directory. From within Blee and
857
+ BISOS these panels are accessible under the Blee "Panels" menu.
858
+
859
+ See
860
+ `file:./panels/bisos.airflow/_nodeBase_/fullUsagePanel-en.org <./panels/bisos.airflow/_nodeBase_/fullUsagePanel-en.org>`__
861
+ for a starting point.
862
+
863
+ The companion repo ``bro_dockerfiles`` has its own README with details
864
+ on each image variant, host cgroup v1/v2 compatibility, and the per-leaf
865
+ build / verify recipes.
866
+
867
+ *bisos.dockerProc* is best developed with
868
+ `Blee <https://github.com/bx-blee>`__, the *By\* BISOS Libre-Halaal
869
+ Emacs Environment* — a layer on top of Emacs and BISOS which creates a
870
+ comprehensive integrated usage and development environment.
871
+
872
+ Support
873
+ =======
874
+
875
+ | For support, criticism, comments and questions; please contact the
876
+ author/maintainer
877
+ | `Mohsen Banan <http://mohsen.1.banan.byname.net>`__ at:
878
+ http://mohsen.1.banan.byname.net/contact