matsuri 0.1.0 → 0.3.0

data/docs/local-dev.md

@@ -0,0 +1,173 @@
+# Local Development
+
+Matsuri grew up as a local development harness: build an image, point
+the pod at your working tree, converge, and iterate. These workflows are
+still in the gem. This page documents them as they exist, along with
+where the ecosystem has since produced better tools.
+
+## Docker Build and Push
+
+A stack can own its image lifecycle. The `build` and `push` verbs drive
+hooks that the stack definition supplies — the gem provides the wiring,
+the platform provides the meaning:
+
+```bash
+matsuri dev build myapp --version v42 --branch main
+matsuri dev push  myapp --version v42
+```
+
+```ruby
+Matsuri.define :stack, 'myapp' do
+  def build!(opt = {})
+    docker! "build -t myapp:#{opt[:version] || 'latest'} #{config.docker_path}"
+  end
+
+  def push!(opt = {})
+    docker! "push myapp:#{opt[:version] || 'latest'}"
+  end
+end
+```
+
+Without a definition, both hooks fail with a message saying so; there is
+no default build behavior. `build` accepts `--dev`, `--version`,
+`--branch`, `--github-user`, and `--repo`; `push` accepts `--dev` and
+`--version`. What the options mean is up to the hook.
+
+Conventional paths for build inputs live in `Matsuri::Config`:
+
+- `docker_path` (`docker/`) — Dockerfiles and build contexts
+- `build_path` (`build/`) — build output
+- `images_path` (`platform/images`) — image definitions
+
+## Live Mounts
+
+The iterate-without-rebuilding loop: mount the working tree into the pod
+with a hostPath volume, so edits on the host appear in the running
+container. Pod definitions get path helpers that resolve against the
+platform repo:
+
+- `src_path('myapp')` — `src/myapp` under the repo; the working tree
+- `cache_path('bundle')` — `.cache/bundle`; persistent caches for gems,
+  node_modules, and other artifacts that should outlive the pod
+- `base_path('etc/myapp.conf')` — any path under the repo
+- `host_path_volume(name, path)` — the hostPath volume entry
+
+```ruby
+Matsuri.define :pod, 'myapp-dev' do
+  let(:volumes) do
+    [
+      host_path_volume('src',   src_path('myapp')),
+      host_path_volume('gems',  cache_path('bundle'))
+    ]
+  end
+
+  let(:container) do
+    {
+      name:         'myapp',
+      image:        'myapp:latest',
+      volumeMounts: [
+        { name: 'src',  mountPath: '/app' },
+        { name: 'gems', mountPath: '/bundle' }
+      ]
+    }
+  end
+end
+```
+
+This pattern assumes the kubelet can reach the host filesystem — true
+for single-node local clusters where the node is your machine (or a VM
+with the repo mounted into it). On Docker Desktop for Mac, hostPath file
+I/O crosses the VM boundary and has historically been slow enough to
+defeat the purpose; Linux with a local cluster is where this loop
+shines.
+
+## Interactive Verbs: sh, console, status
+
+> **Deprecation notice**: [k9s](https://k9scli.io/) is a better tool for
+> all of this — live resource views, shell into any container, logs,
+> port-forwards, and resource metrics in one TUI, without any stack
+> definition support. These verbs may be deprecated in a future release.
+
+`sh` and `console` shell into a stack's primary workload. Like `build!`,
+they drive hooks the stack definition supplies — `sh!(opt, args)` and
+`console!(opt, args)` — and fail with a message when undefined:
+
+```bash
+matsuri dev sh myapp              # shell, as the container user
+matsuri dev sh myapp -p web-0     # pick a pod
+matsuri dev sh! myapp             # shell, as root
+matsuri dev console myapp         # app console (rails c, iex, etc.)
+```
+
+`sh` accepts `--root`/`-r`, `--user`/`-u`, and `--pod`/`-p`; `sh!` is
+`sh` with root; `console` accepts `--root`, `--user`, and `--pod`. The
+distinction is convention: `sh` promises a shell, `console` promises the
+application's REPL — what command actually runs is the hook's call.
+
+`status` reports on resources, with per-resource subcommands and two
+kubectl-top wrappers:
+
+```bash
+matsuri dev status stack myapp
+matsuri dev status pod myapp-web
+matsuri dev status pods           # kubectl top pods
+matsuri dev status nodes          # kubectl top nodes
+matsuri dev top pods              # top is an alias for status
+```
+
+Resource-level `status` calls the resource's `status!`, which a
+definition may implement; the Stack base class requires the definition
+to supply it.
+
+## Deploy Verbs: reload, update, migrate
+
+These drive stack-definition hooks the same way `build` and `push` do,
+but they encode an application deploy lifecycle rather than an image
+one:
+
+```bash
+matsuri dev reload  myapp           # push config without recreating pods
+matsuri dev update  myapp v42       # migrate, then roll to VERSION
+matsuri dev migrate myapp v42       # run migrations only
+```
+
+`update` accepts `--skip-migrations`. The intended shape was for the
+stack definition to supply `update!(version, opt)` as its own deploy
+choreography — run database migrations, then roll pods to the new
+version — with `migrate` and `reload` as the narrower pieces.
+
+**These verbs have not been modernized to current Kubernetes flows.**
+They predate Deployments: rolling to a version is now the manifest
+path's job (`converge <stack> <image_tag>`), and migrations are better
+expressed as invoking a Job (or a `:task` dependency in the stack's
+build order) than as imperative hook choreography. The wiring also shows
+its age: `migrate` routes to a `migrate!` hook that has no stub on the
+Stack base class, so without a definition it fails with a raw
+NoMethodError rather than a message; and the CLI invokes `reload!` with
+only the options hash, so the hook's `version` parameter never receives
+a version. Prefer `converge` and Job resources; these verbs remain for
+stacks that already define the hooks.
+
+## Incomplete Features
+
+Pieces of the local-dev story that were started and never finished, or
+whose finishing era passed. They are not wired to any CLI verb; they are
+kept here so their intent is on the record.
+
+- `matsuri <env> rebuild` — prints "Rebuild not implemented yet" and
+  exits 1. The intended loop was `build!` then `recreate!`, which
+  `Stack#rebuild!` still implements for anyone who wires it up.
+- `Matsuri::Tasks::Docker#kill_all!` — the cluster reset button from
+  when the cluster was a pile of containers on your own machine: stop
+  and force-remove **every container on the Docker daemon**, then
+  re-converge from definitions. Indiscriminate against a modern shared
+  daemon; do not wire this up casually.
+- `Matsuri::Tasks::Docker#fix_pts!` — prints (does not run)
+  `sudo mount -o remount,ptmxmode=666 /dev/pts`, the boot-ritual fix for
+  an early-Docker bug where restrictive devpts permissions broke PTY
+  allocation, so `docker exec -it` — and therefore `sh` and `console` —
+  could not open a terminal.
+- `Matsuri::Tasks::Pod#build!` and `#list!` — empty method bodies; a
+  per-pod image build task that never grew content.
+- `Matsuri::Config.host_osx?` — platform detection left over from the
+  boot2docker era; currently has no callers.

data/docs/presenters.md

@@ -0,0 +1,317 @@
+# Presenters
+
+A presenter is one manifest-shaped step in a delivery pipeline:
+`Manifest -> Manifest`. Resources stay pure generators; a presenter
+transforms the rendered manifest after generation, at the delivery
+boundary, before the payload is serialized for kubectl.
+
+The promises:
+
+- Zero-config and context-free. Construction takes nothing; every
+  parameter a step needs is bound before `#call` ever runs.
+- The input manifest is never mutated. `#call` is framework-owned: it
+  makes one deep copy and hands it to `#transform`, so the copy is a
+  constraint, not a convention an author can forget.
+- The empty pipeline is the identity. A stack with no declarations sends
+  payloads byte-identical to its rendered manifests.
+- The diff previews the delivery. One site builds the presented payload
+  for both `diff` and `apply`, so what diff shows is what apply sends —
+  through presenters, with no extra work. The client-side `--hash-diff`
+  reads drift against the presented declaration the same way.
+
+## Defining Presenters
+
+Presenters are defined in `platform/presenters`. Authors implement
+`#transform` against a manifest that is already theirs to mutate, and
+return it. The Kubernetes dialect (`set_env`, `set_label`,
+`append_volume`, `update_in`, and friends from
+`Matsuri::Transform::Kubernetes`) is bare vocabulary inside
+`#transform`.
+
+```ruby
+Matsuri.define :presenter, 'fluentd-sidecar' do
+  def transform(manifest)
+    update_in(manifest, [:spec, :template, :spec]) do |pod_spec|
+      append_volume(pod_spec, name: 'varlog', emptyDir: {})
+      append_to(pod_spec, [:containers],
+                name:         'fluentd',
+                image:        'fluentd:v1.16',
+                volumeMounts: [{ name: 'varlog', mountPath: '/var/log' }])
+    end
+  end
+end
+```
+
+As with acceptors, the registry definition is the golden path, not a
+requirement: any subclass of `Matsuri::Presenter` works, and anything
+`#call`-able can join a pipeline.
+
+## Declaring Pipelines on Stacks
+
+A stack declares its pipelines with `present`. Unlike `accept`, where a
+collection names a slot and the last declaration wins, a scenario
+accumulates: each `present` appends, and declaration order is member
+order.
+
+```ruby
+Matsuri.define :stack, 'app' do
+  needs :config_map, 'app-settings'
+  needs :deployment, 'app-web'
+  needs :service,    'app-web'
+
+  present :on_apply,  match_deployments('fluentd-sidecar')
+  present :on_create, 'team-label'
+end
+```
+
+Members may be presenter registry names, presenter classes, `Callable`
+instances, or anything `#call`-able.
+
+### Scenarios Are Intent Regimes
+
+Scenario names do not name kubectl verbs. There are two regimes:
+
+- `:on_create` — bringing a resource into existence.
+- `:on_apply` — delivering declared state to something that exists.
+
+Each delivery verb assembles the pipeline for the regime it operates in:
+`create` and the create leg of `recreate` assemble `:on_create`;
+`apply`, `replace`, and `diff` assemble `:on_apply`. Neither replace nor
+server-side apply is a third regime — `apply_via` (`:apply`, `:replace`,
+`:server_side_apply`) picks how the payload travels, and the scenario
+picks what the payload says; the two axes stay orthogonal, so flipping a
+resource's `apply_via` never changes which presenters run. Diff is not a
+scenario either: its meaning is "what converge will send", so it
+assembles `:on_apply` and the preview extends through presenters for
+free.
+
+Declarations never cross a stack boundary: a nested stack assembles from
+its own declarations, the same rule stack-config follows.
+
+## Declaring on Resources
+
+A resource declares its own pipelines with the same `present` grammar;
+the default is none. When a resource converges through a stack, the two
+declaration sites merge into one pipeline: **resource members first,
+stack members last**. These are not slot semantics — both run. Position
+is about who sees the completed manifest: the resource's pipeline
+completes its self-knowledge (closer to generation than to delivery),
+and the stack's steps sit where the server's own mutating admission sits
+relative to the client's manifest — they admit the finished product,
+including whatever the resource's steps produced. Delivery-context
+additions mostly commute and lose nothing by running late; a step that
+enforces a boundary is only enforcement if it runs last.
+
+Direct resource verbs (no stack context) assemble the resource's own
+declarations alone, so a resource behaves identically converged directly
+or through an undeclared stack.
+
+## Selective Steps: Combinators
+
+A stack pipeline runs across every member manifest. Combinators make a
+step selective without the step knowing: `match` wraps an element and
+applies it only to manifests meeting the declared
+`{kind:, name:, namespace:}` criteria — the conjunction of what is
+declared, the same addressing grammar imported-manifest lookup uses —
+and is the identity otherwise. An unknown criteria key is an error, so a
+typo cannot become match-all.
+
+```ruby
+present :on_apply, match({ kind: 'Deployment', name: 'app-web' }, 'fluentd-sidecar')
+present :on_apply, match_kind('CronJob', 'team-label')
+present :on_apply, match_deployments('fluentd-sidecar')
+```
+
+Combinators are machinery, not presenters: they are never registered or
+named in the registry. Names cross the declaration boundary; machinery
+does not.
+
+## Parameterized Steps
+
+Presenters take no configuration, and that is a feature: a step that
+needs a value binds it before the pipeline runs, in one of three ways.
+
+A definition fills a concern's declared hole with a `let`:
+
+```ruby
+Matsuri.define :presenter, 'app-image' do
+  include MyPlatform::SeedImage          # declares let(:image) { fail ... }
+  let(:image) { "registry.example.com/app:#{Matsuri::Config.image_tag}" }
+end
+```
+
+A declaration closes over a one-off value in a labeled leaf:
+
+```ruby
+present :on_apply, Matsuri::Transform::Callable.new(label: 'pin-replicas') { |m|
+  m.set([:spec, :replicas], 3)
+}
+```
+
+Or a helper manufactures an anonymous definition — the named form minus
+the registry entry, with the boundary copy riding along:
+
+```ruby
+def seed_image_presenter(image_value)
+  Class.new(Matsuri::Presenter) do
+    include MyPlatform::SeedImage
+    let(:image) { image_value }
+  end
+end
+
+present :on_create, match_deployments(seed_image_presenter('app:v12'))
+```
+
+The leaf form receives the pipeline carrier directly — a `Map` — and no
+boundary copy of its own, so prefer the presenter forms for anything
+reused.
+
+## Seeds
+
+A seed completes a create payload with a field's boot value — the
+replica count, the boot image, initial resource settings — without ever
+contesting a declaration. The model:
+
+- **Additive on create.** Seeds ride `:on_create` declarations on stacks
+  and resources; the ongoing `:on_apply` payload never carries them.
+- **Don't-clobber is structural.** A field only seeds when the manifest
+  omits it. A workload that declares `spec.replicas` has opted out — the
+  seed yields, nothing overwrites. (Deployment's spec compacts, so
+  `let(:replicas) { nil }` is clean omission.)
+- **Controllers take ownership uncontested.** Because the create runs
+  without writing the seeded fields into last-applied (`save_config`
+  governs this), ongoing applies of a manifest that omits them have
+  nothing to say about them — an HPA that scales is never fought. The
+  cluster side of this is pinned by `tests/bin/probe-save-config.sh`.
+
+This is matsuri's whole answer to externally-owned fields — there is no
+read-back. Matsuri never reads live cluster state to make a declaration
+decision: a field someone else rolls (an image tag updated by a deploy
+operator, replicas moved by an HPA) is omitted from the ongoing payload,
+not read back and re-sent. Under client-side apply the omission
+guarantees non-interference (the probe harness pins it); under an
+ongoing deployment operator the field belongs to that operator's field
+manager via server-side apply. If you deliver tags manually, name the
+tag — the invocation positional or the stack's `:image_tag` acceptor is
+the source, never the cluster.
+
+Under server-side delivery (`apply_via :server_side_apply`), ongoing
+applies merge by field ownership under the repo's field manager
+(`Matsuri::Platform.field_manager`, default `matsuri`), and creation is
+a protocol rather than a single verb (`Tasks::Stacks::Seed`): create the
+seeded payload as `<field_manager>-seed` (the seed owner), deliver the
+declared payload with an ordinary server-side apply, then atomically
+remove the seed owner's managedFields entry. The field manager ends sole
+owner of the declared fields and the seeds end unowned — a deliverer
+rolling the image, or a controller scaling, takes them uncontested. The
+order is not stylistic: a server-side apply against an object with empty
+managedFields attributes every pre-existing field to a ghost manager
+that never relinquishes. `tests/bin/probe-ssa.sh` pins the protocol and
+the refuted alternatives. The operator-facing walkthrough — declaring
+only what you manage, conflicts, interrupted converges — is [Server-Side
+Apply](ssa.md).
+
+The delivery verbs honor the protocol themselves: `apply` (and so
+converge) routes a server-side resource that is not yet live through
+`Seed`, `create` brings one into existence through it, and `recreate`
+tears down and reseeds. Every later apply is an ordinary server-side
+apply. Converge is sequential, so a stack member later in `build_order`
+converges only after an earlier member is fully seeded and ceded — place
+whatever hands a field to another manager after the workload it manages.
+
+Three concerns ship, each a machinery + declared-hole pair:
+`Matsuri::Presenters::SeedReplicas` (`let(:replicas)`), `SeedImage`
+(`let(:image)`, seeding only containers without one), and
+`SeedResources` (`let(:resources)`, likewise). One seed concern per
+presenter — compose seeds in the pipeline, not by stacking includes.
+Each has a declaration-surface constructor closing the hole over its
+argument:
+
+```ruby
+Matsuri.define :stack, 'app' do
+  needs :deployment, 'app-web'
+  needs :deployment, 'app-worker'
+
+  present :on_create, match_deployments(seed_replicas(2))
+  present :on_create, match_deployments(seed_resources(
+    requests: { cpu: '500m', memory: '512Mi' }
+  ))
+  present :on_create, match({ kind: 'Deployment', name: 'app-worker' },
+                            seed_replicas(1))
+end
+```
+
+Seed values close at declaration time, so they must be literals or reads
+of global state (`Matsuri::Platform`, environment config) — instance
+values like the converge positional do not reach class-level
+declarations.
+
+### save_config
+
+`save_config` feeds create/recreate's `--save-config` flag, consulted
+only when ongoing delivery consults last-applied (`apply_via :apply`):
+under `:replace` or `:server_side_apply` it is pinned false, because a
+seeded last-applied is a prune trap for any later client-side apply. It
+resolves most-specific-first: a definition's own `let(:save_config)`,
+then the owning stack's `save_config false` declaration, then the
+repo-wide `Matsuri::Platform.save_config` (default `true`). Like
+stack-config and `present` declarations, a stack's policy never crosses
+to nested stacks.
+
+## Prune Empty
+
+`Matsuri::Presenters::PruneEmpty` prunes empty values — `nil`, `{}`,
+`[]` — from the payload, bottom-up (a parent emptied by pruning prunes
+too). Empty strings survive: `""` is a value Kubernetes assigns meaning
+to. Array elements are never dropped — removing elements changes list
+semantics — only their interiors prune.
+
+Under server-side apply this is more than hygiene. An empty collection
+in a payload is an ownership claim over a field the definition does not
+manage, and inside an atomic struct (`.spec.selector`) it diverges from
+the server's persisted normal form — the server drops empty members on
+storage — so the seeding protocol's equal-values co-ownership turns into
+a loud conflict. A pruned payload declares in persistence normal form:
+ceding a field and saying nothing are the same act.
+
+Declare it on both intent regimes, after any seeds, so the create's echo
+equals the declaration:
+
+```ruby
+present :on_create, prune_empty
+present :on_apply,  prune_empty
+```
+
+The merge is not uniformly monoidal — for some fields an empty value is
+a genuine assertion (an atomic field held empty against other writers).
+`let(:exceptions)` carries those paths, each preserved even when empty;
+it ships with none. Fill it through the helper or a named definition:
+
+```ruby
+present :on_apply, prune_empty(except: [%i[spec selector matchExpressions]])
+
+Matsuri.define :presenter, 'prune-empty' do
+  include Matsuri::Presenters::PruneEmpty
+  let(:exceptions) { [%i[spec selector matchExpressions]] }
+end
+```
+
+Paths address hash keys only; they do not descend into arrays.
+
+## Composition
+
+Pipelines are `Matsuri::Transform::Callable` values: `new` is the
+identity, `new(label:) { }` a leaf, `new([a, b])` a composite, and `+`
+concatenates. Composition obeys the monoid laws, so bundles nest without
+surprises: a composite passed as a member stays one member, and its
+label rides along as trace metadata.
+
+## What Presenters Are Not
+
+Presenters are not the place for a resource's own vocabulary — a
+deployment that always wants a label declares it in its manifest.
+Presenters carry what the *delivery context* adds: the stack-wide
+sidecar, the environment stamp, the value seeded only at creation.
+Generation stays in definitions, delivery policy in lets
+(`converge_via`, `apply_via`, `save_config`), and presentation in the
+pipeline between them.

data/docs/resources.md

@@ -0,0 +1,121 @@
+# Resources
+
+Resources are defined using `let()` extracted from `rspec` (shipped as
+the `rlet` gem). Each resource has a `manifest` let that assembles the
+complete resource manifest. Every value in it is constructed by a
+predefined `let()` that you override with your own `let()`: the manifest
+is a composition of small named values, and redefining one name changes
+exactly that leaf. Values that have no sensible default — a deployment's
+`replicas`, its `match_labels` — are declared holes that fail fast until
+a definition fills them.
+
+Resource definitions are Ruby classes, so inheritance can be used: a
+definition can inherit from another definition and override only the
+lets that differ.
+
+You can create mixins with `Concern` that carry their own `let()`. This
+allows you to include them across definitions. Matsuri comes with a few
+builtin mixins:
+
+- [`Concerns::Pods::SpreadTopology`](../lib/matsuri/concerns/pods/spread_topology.rb)
+  — topology spread constraint vocabulary for balancing same-labeled
+  pods across zones or nodes.
+- [`Concerns::Pods::SpreadPods`](../lib/matsuri/concerns/pods/spread_pods.rb)
+  — pod anti-affinity vocabulary for excluding co-location within a node
+  or zone. Complements SpreadTopology: spread expresses balance,
+  anti-affinity expresses exclusion.
+- [`Concerns::PodTemplate`](../lib/matsuri/concerns/pod_template.rb) —
+  embeds a pod definition as a workload's pod template; this is how
+  Deployment, StatefulSet, and the other workloads reference pods.
+- [`Concerns::ImportManifest`](../lib/matsuri/concerns/import_manifest.rb)
+  — imports vendor YAML manifests as source material and transforms them
+  with the same algebra.
+
+## Defining a Resource in the Registry
+
+Definitions live under `platform/`, one directory per resource type, one
+file per definition. The registry resolves `(type, name)` to the file by
+convention — `Matsuri.define :pod, 'web'` is found in
+`platform/pods/web.rb`:
+
+    platform/
+      acceptors/app.rb
+      config_maps/app-settings.rb
+      deployments/web.rb
+      pods/web.rb
+      presenters/team-label.rb
+      services/web.rb
+      stacks/app.rb
+
+`Matsuri.define` registers the definition and returns the class, so a
+file can also refine what it defines (or define bare classes and
+register them directly — the registry is the golden path, not a
+requirement).
+
+### Environments
+
+A definition can be specialized per environment by path: the registry
+consults `platform/pods/<environment>/web.rb` before
+`platform/pods/web.rb`, so an environment-specific file shadows the
+shared one. The environment is the CLI's first argument
+(`matsuri staging ...`); see [Setup](setup.md) for how environments bind
+to kubectl contexts.
+
+See also the rspec fixture platform, which is a small, complete example
+repo: [spec/fixture/platform](../spec/fixture/platform).
+
+## Examples
+
+### Pods
+
+The smallest definition overrides `spec` outright:
+
+```ruby
+# platform/pods/hello.rb
+Matsuri.define :pod, 'hello' do
+  let(:spec) do
+    { containers: [{ name: 'hello', image: 'nginx:1.25' }] }
+  end
+end
+```
+
+Most definitions override narrower lets and let the predefined `spec`
+assemble around them:
+
+```ruby
+# platform/pods/web.rb
+Matsuri.define :pod, 'web' do
+  let(:container) { { name: 'web', image: "app:#{image_tag}" } }
+  let(:volumes)   { [] }
+end
+```
+
+Placement vocabulary (`node_selector`, `tolerations`, `affinity`,
+`topology_spread_constraints`) is predefined on Pod; the spread concerns
+above supply strategies for the affinity and spread lets.
+
+See also the pod fixtures:
+[spec/fixture/platform/pods](../spec/fixture/platform/pods).
+
+### Deployments
+
+A deployment embeds a pod definition as its template by naming it —
+`let(:pod_name)` is the reference, and the pod's labels, annotations,
+and spec ride in through `Concerns::PodTemplate`:
+
+```ruby
+# platform/deployments/web.rb
+Matsuri.define :deployment, 'web' do
+  let(:replicas)     { 2 }
+  let(:match_labels) { { 'app' => 'web' } }
+  let(:pod_name)     { 'web' } # embeds pod 'web' as the pod template
+end
+```
+
+`replicas` and `match_labels` are declared holes — a deployment that
+does not define them fails at first render, not at apply time. Rollout
+knobs (`strategy`, `max_unavailable`, `max_surge`, `min_ready_seconds`)
+are predefined lets with conventional defaults.
+
+See also the deployment fixtures:
+[spec/fixture/platform/deployments](../spec/fixture/platform/deployments).