@mnci/cli 4.14.0 → 4.16.0

package/README.md

@@ -75,7 +75,7 @@ helpers — they have no interest in adding a project. The concept now sits in i
 own slice that depends on nothing above it (`file-system` alone), and all three
 consumers point at it.
 
-## Commands (deliberately just six)
+## Commands (deliberately just seven)
 
 ```sh
 mnci new my-repo            # create a monorepo (prompts scope + registry)
@@ -103,6 +103,7 @@ mnci add python-vendor shared --lib core  # wire core's module into shared's bui
 # Go (@nx-go/nx-go — one root go.mod, golangci-lint + go test)
 mnci add go-app api            # executable -> apps/ (binary, zipped into the drop)
 mnci add go-app cli --release  # ...released: tag + per-platform zips on the GitHub Release
+mnci add go-app tray --cgo     # needs a C toolchain: built on a runner of each OS
 mnci add go-function-app fn    # serverless handler -> apps/
 mnci add go-lib core           # publishable (by git tag) -> packages/
 mnci add go-internal-lib util  # private shared package -> libs/
@@ -122,6 +123,8 @@ mnci upgrade --agent windows-latest   # ...with an explicit override
 
 mnci doctor                   # check this workspace's invariants (read-only)
 
+mnci ci verify                # the pipeline's verify phase, run here: see `mnci ci` below
+
 mnci sync                     # converge dependency ranges + nx sync (TS project refs)
 mnci sync --check             # ...report and exit non-zero, writing nothing
 
@@ -319,6 +322,35 @@ TTY, so a piped or CI run reports instead of hanging on a prompt), `-y/--yes`
 (take everything), `--ecosystem`, and `--no-install` (edit the manifests but skip
 the reinstall).
 
+## `mnci ci`: the pipeline's phases, as a command
+
+`mnci ci verify` runs the pipeline's verify phase on your machine: `nx sync:check`, then
+every project (or, when a pull request's target branch is set, only the projects affected
+since the merge-base with it) through `lint`, `typecheck`, `test` and `build`. It exits with
+the failing command's own status, so it can stand in for the pipeline's step.
+
+It is not a shortcut for those Nx targets, and the CLI still has no wrapper for `nx test`.
+It is the pipeline's own logic, which until now lived as a `node -e` one-liner inside the
+generated YAML, ported to tested code, so a laptop and a pipeline run the same thing. The
+design (one `npx mnci ci` call in the pipeline, with room for your own steps around it) is
+tracked in #269, and this is its first phase: **the generated pipelines do not call it yet**
+and are unchanged, so nothing about an existing workspace changes. Phases follow as their
+guards are ported.
+
+- **Same scoping as the guard.** No pull-request target means every project, so a push to
+  main verifies in full. A pull request verifies what is affected since the merge-base with
+  `origin/<target>`, fetching the target once if that ref is missing, and falls back to every
+  project when no merge-base can be found, since a run that verifies too little still reports
+  green. The target comes from `GITHUB_BASE_REF` or `SYSTEM_PULLREQUEST_TARGETBRANCH`, so to
+  reproduce a pull request run locally, set one: `GITHUB_BASE_REF=main mnci ci verify`.
+- **Native (cgo) apps are left out**, as in the pipeline, since one agent cannot build them.
+- **Log groups.** Under GitHub Actions and Azure Pipelines each part is a collapsible group
+  (`::group::`, `##[group]`); locally it is a plain heading.
+- **Checked against the guard it replaces.** An integration spec runs the inline guard and
+  this command against the same real git repository, with a recording stand-in for `npx`,
+  across a push, a pull request, Azure's `refs/heads/` form, a missing `origin/<target>` and
+  an unresolvable one, and requires the same Nx commands in each.
+
 ## `mnci doctor`: checking the invariants actually hold
 
 Read-only — it never edits the workspace. Every failing finding names the command
@@ -688,6 +720,64 @@ the generated tree, so it runs afterwards, but still before the first write:
 a refusal leaves the target byte-identical to how it was found, and the staging
 copy is discarded.
 
+### Adopting a flat Go module
+
+A repository with its own `go.mod`, a root `main.go` and `internal/` packages is the
+other common starting point, and `--into` takes it as it is. Measured on a clone of a
+real one (twelve `internal/` packages, its own `release.yml`):
+
+- **`go.mod` and `go.sum` are never touched.** `add go-*` skips the module bootstrap
+  when a `go.mod` exists, so the module path, the `require` lines and the Go version
+  survive byte for byte, and no `go.work` is created.
+- **The Go plugin is registered anyway.** The bootstrap that registers it in `nx.json`
+  is the one that is skipped, which used to leave the plugin installed and unlisted:
+  every target worked, and Nx had **no Go project graph**, so `nx affected` skipped an
+  app that imports a changed library, silently. `add go-*` now registers it, `mnci
+  upgrade` repairs a workspace that adopted before, and `mnci doctor` fails while Go
+  projects exist and it is missing.
+- **Your own workflow coexists.** A release workflow that fires on `v*` tags and the
+  generated `ci.yml`, which fires on pushes and pull requests to `main`, do not overlap.
+  mnci tags a released app `<name>@<version>`, so once `--release` and its GitHub
+  Release assets do what yours did, delete the old workflow.
+- **The format step can fail on a file you already had**, and says so without failing
+  the run: a UTF-16 `.json` at the root made `eslint .` report a parse error. The root
+  lint covers root-level files, so a repository's existing JSON, Markdown and YAML are
+  now inside it; fix or delete what it names.
+
+Landing the code is mechanical, so it is a recipe, not a command (`git mv` and one
+import rewrite are the whole job, and a command would have to guess your layout). With
+`youtube-downloader` as the module path, `mvd-cli` as the app and `mvd-core` as the
+library:
+
+```sh
+mnci add go-app mvd-cli
+mnci add go-internal-lib mvd-core
+
+# The generated starters are placeholders: drop them.
+rm apps/mvd-cli/main.go apps/mvd-cli/main_test.go
+rm -r libs/mvd-core/mvdcore
+
+# git mv, not mv, so `git log --follow` reaches the history before the move.
+git mv main.go apps/mvd-cli/main.go
+for slice in internal/*/; do git mv "$slice" "libs/mvd-core/$(basename "$slice")"; done
+
+# Rewrite the import paths (macOS: sed -i '').
+grep -rl 'youtube-downloader/internal/' --include=*.go apps libs \
+  | xargs sed -i 's#youtube-downloader/internal/#youtube-downloader/libs/mvd-core/#g'
+
+npx nx run-many -t test,build --projects=mvd-cli,mvd-core
+```
+
+Use `nx` and not a bare `go test ./...` from the root: `./...` also walks
+`node_modules`, where npm packages that contain Go code sit (one did here). On the
+repository measured, this left `go build` clean, every test passing, 62 renames
+detected by git, and the app depending on the library in the project graph. The e2e
+(`go adoption`) runs this recipe on a fixture and asserts each of those.
+
+Not measured here: whether `golangci-lint`, which is what each Go project's `lint`
+runs, passes on a codebase that has never been linted. Run `nx run <lib>:lint` before
+relying on it, and expect to fix or silence what it reports.
+
 ## Layout convention = release scoping
 
 | Directory          | Contents                                                       | Released?                              |
@@ -1439,6 +1529,51 @@ pipeline installs `golangci-lint` itself (see below).
   `nx-release-publish` target is written: there is nothing to push. The only
   real difference between `go-lib` and `go-internal-lib` is intent, recorded
   in the `type:go-lib` tag and the `packages/` location.
+- **Pure-Go apps are cross-compiled; an app that needs a C toolchain is not
+  (`mnci add go-app <name> --cgo`).** `build-all` builds six static binaries from
+  one machine with `CGO_ENABLED=0`, which is right until the app needs cgo: a
+  system-tray icon (Cocoa on macOS, GTK on Linux), a native GUI toolkit, a cgo
+  database driver. Those can only be built where the C toolchain and the target
+  OS's libraries are, so a `--cgo` app is tagged `build:cgo` and gets
+  `build-native` and `package-native` (this machine only, `CGO_ENABLED=1`, the
+  same `VERSION` stamp and `-trimpath`, the zip named as `package-all` names a
+  platform's) **instead of** `package`, `build-all` and `package-all`. It keeps
+  `build`, `test`, `lint` and `start` for local work.
+  - **A `native` job, only when such an app exists.** The generated pipeline
+    gains a job with one leg each on `windows-latest`, `macos-latest` and
+    `ubuntu-latest` (GitHub Actions: a matrix; Azure Pipelines: a matrix of
+    `vmImage`s, with the original job moved under `jobs:`). Each leg lints,
+    tests, builds and packages the native apps and publishes its zips as an
+    artifact. The single-agent verify excludes them (`--exclude=tag:build:cgo`),
+    because it cannot build them. A workspace with no native app keeps a
+    pipeline byte-identical to the one it had: the job is decided when the file is
+    written, since neither provider can skip a whole job on a file's existence.
+  - **Run `mnci upgrade` after adding one.** `add` does not rewrite the
+    pipeline files, so until then CI would verify the app on the one agent that
+    cannot build it. `mnci add` says so, and `mnci doctor` fails while a
+    pipeline lacks the job, and while no C compiler is installed on the machine
+    (it reads `CC`, then `gcc`, `clang`, `cc`).
+  - **Linux prerequisites are yours to finish.** The leg installs a C compiler
+    and `pkg-config`, which is all mnci can know. The `-dev` packages your app
+    links (`libgtk-3-dev` and `libayatana-appindicator3-dev` for a tray icon) go
+    on that one marked line in the pipeline. The macOS runner ships Xcode's tools,
+    and the Windows leg relies on the hosted image's MinGW-w64 `gcc`: the nightly
+    e2e builds a cgo app on `windows-latest`, and says so loudly if it finds no
+    compiler there rather than passing.
+  - **Architectures: one per OS, the runner's own.** `windows-latest` and
+    `ubuntu-latest` are amd64 and `macos-latest` is arm64, so a native app ships
+    `windows-amd64`, `linux-amd64` and `darwin-arm64`. `darwin-amd64`,
+    `linux-arm64` and `windows-arm64` are not built: they need an Intel macOS
+    runner, an arm Linux runner or a cross-compiler. The legs are fixed in the
+    generated pipeline today, so adding one means editing a file `mnci upgrade`
+    rewrites (russoedu/MoNecromanCi#269 is about giving that a safe place).
+  - **Releasing one.** With `--release` as well, each leg, on a push to main and
+    after the `ci` job has tagged, runs
+    `node tools/go-app-release.cjs assets --native`, which builds
+    `package-native` with `VERSION` set to the tag's version and uploads that
+    OS's zip to the same GitHub Release, so one release collects a zip from every
+    runner. Azure Pipelines has no GitHub Release to attach to and stops at the
+    artifact.
 - **Releasing a Go app is an opt-in: `mnci add go-app <name> --release`.** The
   app is tagged `release:go`, which `release.projects` selects by tag (as it
   does for a VS Code extension, so `mnci upgrade` keeps it). Without the flag
@@ -1472,8 +1607,8 @@ pipeline installs `golangci-lint` itself (see below).
     commits that touch an app's dependencies, not only its own folder (measured:
     a `fix:` touching only an imported `go-internal-lib` produced a patch
     release of the app), which is right for a binary that links the library in.
-  - Not covered yet: apps that need a native toolchain (cgo) and so one runner
-    per OS (russoedu/MoNecromanCi#263). The asset step builds all six
+  - An app that needs a C toolchain is released from several runners instead:
+    see _Pure-Go apps are cross-compiled_ above. This step builds all six
     platforms on one runner, which is what `CGO_ENABLED=0` allows.
 - **No publish-time dependency injection**, unlike Python's vendoring: `go
 build` links statically, so the binary in the drop already contains