@git.zone/tsdocker 1.15.1 → 1.17.0

package/readme.md

@@ -1,6 +1,6 @@
 # @git.zone/tsdocker
 
-> 🐳 The ultimate Docker development toolkit for TypeScript projects — build, test, and ship containerized applications with ease.
+> 🐳 The ultimate Docker development toolkit for TypeScript projects — build, test, and ship multi-arch containerized applications with zero friction.
 
 ## Issue Reporting and Security
 
@@ -8,15 +8,18 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
 
 ## What is tsdocker?
 
-**tsdocker** is a comprehensive Docker development and building tool that handles everything from testing npm packages in clean environments to building and pushing multi-architecture Docker images across multiple registries.
+**tsdocker** is a comprehensive Docker development and build tool that handles everything from testing npm packages in clean environments to building and pushing multi-architecture Docker images across multiple registries — all from a single CLI.
 
 ### 🎯 Key Capabilities
 
 - 🧪 **Containerized Testing** — Run your tests in pristine Docker environments
 - 🏗️ **Smart Docker Builds** — Automatically discover, sort, and build Dockerfiles by dependency
-- 🚀 **Multi-Registry Push** — Ship to Docker Hub, GitLab, GitHub Container Registry, and more
-- 🔧 **Multi-Architecture** — Build for `amd64` and `arm64` with Docker Buildx
-- ⚡ **Zero Config Start** — Works out of the box, scales with your needs
+- 🌍 **True Multi-Architecture** — Build for `amd64` and `arm64` simultaneously with Docker Buildx
+- 🚀 **Multi-Registry Push** — Ship to Docker Hub, GitLab, GitHub Container Registry, and more via OCI Distribution API
+- ⚡ **Parallel Builds** — Level-based parallel builds with configurable concurrency
+- 🗄️ **Persistent Local Registry** — All images flow through a local OCI registry with persistent storage
+- 📦 **Build Caching** — Skip unchanged Dockerfiles with content-hash caching
+- 🔧 **Zero Config Start** — Works out of the box, scales with your needs
 
 ## Installation
 
@@ -53,6 +56,7 @@ tsdocker will:
 2. 📊 Analyze `FROM` dependencies between them
 3. 🔄 Sort them topologically
 4. 🏗️ Build each image in the correct order
+5. 📦 Push every image to a persistent local registry (`.nogit/docker-registry/`)
 
 ### 📤 Push to Registries
 
@@ -63,33 +67,52 @@ Ship your images to one or all configured registries:
 tsdocker push
 
 # Push to a specific registry
-tsdocker push registry.gitlab.com
+tsdocker push --registry=registry.gitlab.com
 ```
 
+Under the hood, `tsdocker push` uses the **OCI Distribution API** to copy images directly from the local registry to remote registries. This means multi-arch manifest lists are preserved end-to-end — no more single-platform-only pushes.
+
 ## CLI Commands
 
 | Command | Description |
 |---------|-------------|
-| `tsdocker` | Run tests in a fresh Docker container |
+| `tsdocker` | Run tests in a fresh Docker container (legacy mode) |
 | `tsdocker build` | Build all Dockerfiles with dependency ordering |
-| `tsdocker push [registry]` | Push images to configured registries |
+| `tsdocker push` | Build + push images to configured registries |
 | `tsdocker pull <registry>` | Pull images from a specific registry |
-| `tsdocker test` | Run container test scripts (test_*.sh) |
+| `tsdocker test` | Build + run container test scripts (`test_*.sh`) |
 | `tsdocker login` | Authenticate with configured registries |
 | `tsdocker list` | Display discovered Dockerfiles and their dependencies |
-| `tsdocker clean --all` | ⚠️ Aggressively clean Docker environment |
+| `tsdocker clean` | Interactively clean Docker environment |
 | `tsdocker vscode` | Launch containerized VS Code in browser |
 
+### Build Flags
+
+| Flag | Description |
+|------|-------------|
+| `--platform=linux/arm64` | Override build platform for a single architecture |
+| `--timeout=600` | Build timeout in seconds |
+| `--no-cache` | Force rebuild without Docker layer cache |
+| `--cached` | Skip unchanged Dockerfiles (content-hash based) |
+| `--verbose` | Stream raw `docker build` output |
+| `--parallel` | Enable level-based parallel builds (default concurrency: 4) |
+| `--parallel=8` | Parallel builds with custom concurrency |
+| `--context=mycontext` | Use a specific Docker context |
+
+### Clean Flags
+
+| Flag | Description |
+|------|-------------|
+| `--all` | Include all images and volumes (not just dangling) |
+| `-y` | Auto-confirm all prompts |
+
 ## Configuration
 
-Configure tsdocker in your `package.json` or `npmextra.json`:
+Configure tsdocker in your `package.json` or `npmextra.json` under the `@git.zone/tsdocker` key:
 
 ```json
 {
   "@git.zone/tsdocker": {
-    "baseImage": "node:20",
-    "command": "npm test",
-    "dockerSock": false,
     "registries": ["registry.gitlab.com", "docker.io"],
     "registryRepoMap": {
       "registry.gitlab.com": "myorg/myproject"
@@ -98,7 +121,6 @@ Configure tsdocker in your `package.json` or `npmextra.json`:
       "NODE_VERSION": "NODE_VERSION"
     },
     "platforms": ["linux/amd64", "linux/arm64"],
-    "push": false,
     "testDir": "./test"
   }
 }
@@ -106,24 +128,73 @@ Configure tsdocker in your `package.json` or `npmextra.json`:
 
 ### Configuration Options
 
-#### Testing Options (Legacy)
+#### Build & Push Options
 
-| Option | Type | Description |
-|--------|------|-------------|
-| `baseImage` | `string` | Docker image for test environment (default: `hosttoday/ht-docker-node:npmdocker`) |
-| `command` | `string` | Command to run inside container (default: `npmci npm test`) |
-| `dockerSock` | `boolean` | Mount Docker socket for DinD scenarios (default: `false`) |
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `registries` | `string[]` | `[]` | Registry URLs to push to |
+| `registryRepoMap` | `object` | `{}` | Map registries to different repository paths |
+| `buildArgEnvMap` | `object` | `{}` | Map Docker build ARGs to environment variables |
+| `platforms` | `string[]` | `["linux/amd64"]` | Target architectures for multi-arch builds |
+| `testDir` | `string` | `./test` | Directory containing test scripts |
 
-#### Build & Push Options
+#### Legacy Testing Options
 
-| Option | Type | Description |
-|--------|------|-------------|
-| `registries` | `string[]` | Registry URLs to push to |
-| `registryRepoMap` | `object` | Map registries to different repository paths |
-| `buildArgEnvMap` | `object` | Map Docker build ARGs to environment variables |
-| `platforms` | `string[]` | Target architectures (default: `["linux/amd64"]`) |
-| `push` | `boolean` | Auto-push after build (default: `false`) |
-| `testDir` | `string` | Directory containing test scripts |
+These options configure the `tsdocker` default command (containerized test runner):
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `baseImage` | `string` | `hosttoday/ht-docker-node:npmdocker` | Docker image for test environment |
+| `command` | `string` | `npmci npm test` | Command to run inside the container |
+| `dockerSock` | `boolean` | `false` | Mount Docker socket for DinD scenarios |
+
+## Architecture: How tsdocker Works
+
+tsdocker uses a **local OCI registry** as the canonical store for all built images. This design solves fundamental problems with Docker's local daemon, which cannot hold multi-architecture manifest lists.
+
+### 📐 Build Flow
+
+```
+┌─────────────────────────────────────────────────┐
+│  tsdocker build                                 │
+│                                                 │
+│  1. Start local registry (localhost:5234)        │
+│     └── Persistent volume: .nogit/docker-registry/
+│                                                 │
+│  2. For each Dockerfile (topological order):     │
+│     ├── Multi-platform: buildx --push → registry │
+│     └── Single-platform: docker build → registry │
+│                                                 │
+│  3. Stop local registry (data persists on disk)  │
+└─────────────────────────────────────────────────┘
+```
+
+### 📤 Push Flow
+
+```
+┌──────────────────────────────────────────────────┐
+│  tsdocker push                                   │
+│                                                  │
+│  1. Start local registry (loads persisted data)   │
+│                                                  │
+│  2. For each image × each remote registry:        │
+│     └── OCI Distribution API copy:               │
+│         ├── Fetch manifest (single or multi-arch) │
+│         ├── Copy blobs (skip if already exist)    │
+│         └── Push manifest with destination tag    │
+│                                                  │
+│  3. Stop local registry                          │
+└──────────────────────────────────────────────────┘
+```
+
+### 🔑 Why a Local Registry?
+
+| Problem | Solution |
+|---------|----------|
+| `docker buildx --load` fails for multi-arch images | `buildx --push` to local registry works for any number of platforms |
+| `docker push` only pushes single-platform manifests | OCI API copy preserves full manifest lists (multi-arch) |
+| Images lost between build and push phases | Persistent storage at `.nogit/docker-registry/` survives restarts |
+| Redundant blob uploads on incremental pushes | HEAD checks skip blobs that already exist on the remote |
 
 ## Registry Authentication
 
@@ -140,13 +211,17 @@ export DOCKER_REGISTRY_USER="username"
 export DOCKER_REGISTRY_PASSWORD="password"
 ```
 
+### Docker Config Fallback
+
+When pushing, tsdocker will also read credentials from `~/.docker/config.json` if no explicit credentials are provided via environment variables. This means `docker login` credentials work automatically.
+
 ### Login Command
 
 ```bash
 tsdocker login
 ```
 
-Authenticates with all configured registries.
+Authenticates with all configured registries using the provided environment variables.
 
 ## Advanced Usage
 
@@ -162,7 +237,27 @@ Build for multiple platforms using Docker Buildx:
 }
 ```
 
-tsdocker automatically sets up a Buildx builder when multiple platforms are specified.
+tsdocker automatically:
+- Sets up a Buildx builder with `--driver-opt network=host` (so buildx can reach the local registry)
+- Pushes multi-platform images to the local registry via `buildx --push`
+- Copies the full manifest list (including all platform variants) to remote registries on `tsdocker push`
+
+### ⚡ Parallel Builds
+
+Speed up builds by building independent images concurrently:
+
+```bash
+# Default concurrency (4 workers)
+tsdocker build --parallel
+
+# Custom concurrency
+tsdocker build --parallel=8
+
+# Works with caching too
+tsdocker build --parallel --cached
+```
+
+tsdocker groups Dockerfiles into **dependency levels** using topological analysis. Images within the same level have no dependencies on each other and build in parallel. Each level completes before the next begins.
 
 ### 📦 Dockerfile Naming Conventions
 
@@ -190,7 +285,7 @@ COPY . .
 RUN npm run build
 ```
 
-tsdocker automatically detects that `Dockerfile_app` depends on `Dockerfile_base` and builds them in the correct order.
+tsdocker automatically detects that `Dockerfile_app` depends on `Dockerfile_base`, builds them in the correct order, and makes the base image available to dependent builds via the local registry (using `--build-context` for buildx).
 
 ### 🧪 Container Test Scripts
 
@@ -210,6 +305,8 @@ Run with:
 tsdocker test
 ```
 
+This builds all images, starts the local registry (so multi-arch images can be pulled), and runs each matching test script inside a container.
+
 ### 🔧 Build Args from Environment
 
 Pass environment variables as Docker build arguments:
@@ -232,6 +329,24 @@ FROM node:${NODE_VERSION}
 RUN echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > ~/.npmrc
 ```
 
+### 🗺️ Registry Repo Mapping
+
+Use different repository names for different registries:
+
+```json
+{
+  "@git.zone/tsdocker": {
+    "registries": ["registry.gitlab.com", "docker.io"],
+    "registryRepoMap": {
+      "registry.gitlab.com": "mygroup/myproject",
+      "docker.io": "myuser/myproject"
+    }
+  }
+}
+```
+
+When pushing, tsdocker maps the local repo name to the registry-specific path. For example, a locally built `myproject:latest` becomes `registry.gitlab.com/mygroup/myproject:latest` and `docker.io/myuser/myproject:latest`.
+
 ### 🐳 Docker-in-Docker Testing
 
 Test Docker-related tools by mounting the Docker socket:
@@ -259,59 +374,33 @@ Output:
 Discovered Dockerfiles:
 ========================
 
-1. Dockerfile_base
+1. /path/to/Dockerfile_base
    Tag: myproject:base
    Base Image: node:20-alpine
    Version: base
 
-2. Dockerfile_app
+2. /path/to/Dockerfile_app
    Tag: myproject:app
    Base Image: myproject:base
    Version: app
    Depends on: myproject:base
 ```
 
-### 🗺️ Registry Repo Mapping
+## Examples
 
-Use different repository names for different registries:
+### Minimal Build & Push
 
 ```json
 {
   "@git.zone/tsdocker": {
-    "registries": ["registry.gitlab.com", "docker.io"],
-    "registryRepoMap": {
-      "registry.gitlab.com": "mygroup/myproject",
-      "docker.io": "myuser/myproject"
-    }
+    "registries": ["docker.io"],
+    "platforms": ["linux/amd64"]
   }
 }
 ```
 
-## Environment Variables
-
-### qenv Integration
-
-tsdocker automatically loads environment variables from `qenv.yml`:
-
-```yaml
-# qenv.yml
-API_KEY: your-api-key
-DATABASE_URL: postgres://localhost/test
-```
-
-These are injected into your test container automatically.
-
-## Examples
-
-### Basic Test Configuration
-
-```json
-{
-  "@git.zone/tsdocker": {
-    "baseImage": "node:20",
-    "command": "npm test"
-  }
-}
+```bash
+tsdocker push
 ```
 
 ### Full Production Setup
@@ -319,8 +408,6 @@ These are injected into your test container automatically.
 ```json
 {
   "@git.zone/tsdocker": {
-    "baseImage": "node:20-alpine",
-    "command": "pnpm test",
     "registries": ["registry.gitlab.com", "ghcr.io", "docker.io"],
     "registryRepoMap": {
       "registry.gitlab.com": "myorg/myapp",
@@ -338,57 +425,37 @@ These are injected into your test container automatically.
 
 ### CI/CD Integration
 
+**GitLab CI:**
+
 ```yaml
-# .gitlab-ci.yml
-build:
+build-and-push:
   stage: build
   script:
     - npm install -g @git.zone/tsdocker
-    - tsdocker build
     - tsdocker push
+  variables:
+    DOCKER_REGISTRY_1: "registry.gitlab.com|$CI_REGISTRY_USER|$CI_REGISTRY_PASSWORD"
+```
+
+**GitHub Actions:**
 
-# GitHub Actions
+```yaml
 - name: Build and Push
   run: |
     npm install -g @git.zone/tsdocker
     tsdocker login
-    tsdocker build
     tsdocker push
   env:
     DOCKER_REGISTRY_1: "ghcr.io|${{ github.actor }}|${{ secrets.GITHUB_TOKEN }}"
 ```
 
-## Requirements
-
-- **Docker** — Docker Engine or Docker Desktop must be installed
-- **Node.js** — Version 18 or higher (ESM support required)
-- **Docker Buildx** — Required for multi-architecture builds (included in Docker Desktop)
-
-## Why tsdocker?
-
-### 🎯 The Problem
-
-Managing Docker workflows manually is tedious:
-- Remembering build order for dependent images
-- Pushing to multiple registries with different credentials
-- Setting up Buildx for multi-arch builds
-- Ensuring consistent test environments
-
-### ✨ The Solution
-
-tsdocker automates the entire workflow:
-- **One command** to build all images in dependency order
-- **One command** to push to all registries
-- **Automatic** Buildx setup for multi-platform builds
-- **Consistent** containerized test environments
-
 ## TypeScript API
 
-tsdocker exposes its types for programmatic use:
+tsdocker can also be used programmatically:
 
 ```typescript
-import type { ITsDockerConfig } from '@git.zone/tsdocker/dist_ts/interfaces/index.js';
 import { TsDockerManager } from '@git.zone/tsdocker/dist_ts/classes.tsdockermanager.js';
+import type { ITsDockerConfig } from '@git.zone/tsdocker/dist_ts/interfaces/index.js';
 
 const config: ITsDockerConfig = {
   baseImage: 'node:20',
@@ -396,15 +463,21 @@ const config: ITsDockerConfig = {
   dockerSock: false,
   keyValueObject: {},
   registries: ['docker.io'],
-  platforms: ['linux/amd64'],
+  platforms: ['linux/amd64', 'linux/arm64'],
 };
 
 const manager = new TsDockerManager(config);
 await manager.prepare();
-await manager.build();
+await manager.build({ parallel: true });
 await manager.push();
 ```
 
+## Requirements
+
+- **Docker** — Docker Engine 20+ or Docker Desktop
+- **Node.js** — Version 18 or higher (for native `fetch` and ESM support)
+- **Docker Buildx** — Required for multi-architecture builds (included in Docker Desktop)
+
 ## Troubleshooting
 
 ### "docker not found"
@@ -417,11 +490,10 @@ docker --version
 
 ### Multi-arch build fails
 
-Make sure Docker Buildx is available:
+Make sure Docker Buildx is available. tsdocker will set up the builder automatically, but you can verify:
 
 ```bash
 docker buildx version
-docker buildx create --use
 ```
 
 ### Registry authentication fails
@@ -433,19 +505,22 @@ echo $DOCKER_REGISTRY_1
 tsdocker login
 ```
 
+tsdocker also falls back to `~/.docker/config.json` — ensure you've run `docker login` for your target registries.
+
 ### Circular dependency detected
 
 Review your Dockerfiles' `FROM` statements — you have images depending on each other in a loop.
 
-## Performance Tips
-
-🚀 **Use specific tags**: `node:20-alpine` is smaller and faster than `node:latest`
+### Build context too large
 
-🚀 **Leverage caching**: Docker layers are cached — your builds get faster over time
+Use a `.dockerignore` file to exclude `node_modules`, `.git`, `.nogit`, and other large directories:
 
-🚀 **Prune regularly**: `docker system prune` reclaims disk space
-
-🚀 **Use .dockerignore**: Exclude `node_modules`, `.git`, etc. from build context
+```
+node_modules
+.git
+.nogit
+dist_ts
+```
 
 ## Migration from Legacy
 

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@git.zone/tsdocker',
-  version: '1.15.1',
+  version: '1.17.0',
   description: 'develop npm modules cross platform with docker'
 }

package/ts/classes.dockercontext.ts

@@ -1,4 +1,5 @@
 import * as plugins from './tsdocker.plugins.js';
+import * as fs from 'fs';
 import { logger } from './tsdocker.logging.js';
 import type { IDockerContextInfo } from './interfaces/index.js';
 
@@ -38,19 +39,28 @@ export class DockerContext {
       isRootless = infoResult.stdout.includes('name=rootless');
     }
 
-    this.contextInfo = { name, endpoint, isRootless, dockerHost: process.env.DOCKER_HOST };
+    // Detect topology
+    let topology: 'socket-mount' | 'dind' | 'local' = 'local';
+    if (process.env.DOCKER_HOST && process.env.DOCKER_HOST.startsWith('tcp://')) {
+      topology = 'dind';
+    } else if (fs.existsSync('/.dockerenv')) {
+      topology = 'socket-mount';
+    }
+
+    this.contextInfo = { name, endpoint, isRootless, dockerHost: process.env.DOCKER_HOST, topology };
     return this.contextInfo;
   }
 
   /** Logs context info prominently. */
   public logContextInfo(): void {
     if (!this.contextInfo) return;
-    const { name, endpoint, isRootless, dockerHost } = this.contextInfo;
+    const { name, endpoint, isRootless, dockerHost, topology } = this.contextInfo;
     logger.log('info', '=== DOCKER CONTEXT ===');
     logger.log('info', `Context:  ${name}`);
     logger.log('info', `Endpoint: ${endpoint}`);
     if (dockerHost) logger.log('info', `DOCKER_HOST: ${dockerHost}`);
     logger.log('info', `Rootless: ${isRootless ? 'yes' : 'no'}`);
+    logger.log('info', `Topology: ${topology || 'local'}`);
   }
 
   /** Emits rootless-specific warnings. */