matsuri 0.1.0 → 0.3.0

data/README.md

@@ -1,8 +1,56 @@
 # Matsuri
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/matsuri`. To experiment with that code, run `bin/console` for an interactive prompt.
-
-TODO: Delete this and the text above, and describe your gem
+Matsuri manages Kubernetes manifests. Matsuri uses the full Ruby
+language for generating manifests. Unlike Helm, it constructs these with
+Ruby objects instead of using string-based templating.
+
+Resources are defined by `let()` extracted from `rspec`. These values
+can be applied monoidally in a flat keyspace, through mixins
+(`Concerns`) or Ruby inheritance. It ultimately constructs a hash data
+structure which the harness feeds into `kubectl` to apply to a target
+cluster.
+
+Since Matsuri *generates* manifests from hash objects, values can be
+safely inserted. Fragments are just more hash objects. Helper methods
+can be written. [Acceptors](docs/acceptors.md) can be defined to import
+configuration values and resource definitions can inject them in.
+
+Similar to Kustomize's design, Matsuri also includes a full transform
+algebra, both for importing upstream manifests, as well as applying
+transforms after generation. [Presenters](docs/presenters.md) can be
+used to create seed values on creation. Subsequent applications through
+Server Side Apply can omit seed presenters, allowing Matsuri to cede
+those values for other operators to manage.
+
+The final hashes are encoded as JSON or YAML, and not before. This
+allows Matsuri to inject, manipulate, and transform manifests as
+structured data, instead of as strings.
+
+Finally, Matsuri provides a [Stack](docs/stacks.md) abstraction. These
+bundle together a collection of resources that can be applied together.
+When combined with acceptors, this allows for automatically standing up
+preview environments.
+
+## History
+
+Matsuri was originally created to help stand up a local K8S cluster for
+a dev environment, while also managing deployments to a staging and
+production environment. Better tools, such as `k9s` and `minikube`, have
+since been developed by the Kubernetes community.
+
+While Helm has emerged as the de facto industry standard, and kustomize
+closes many of Helm's design flaws, Matsuri can do something neither do
+well: it can dynamically generate whole stacks.
+
+Unlike cdk8s, Matsuri's use of monoidal merges allows for a far greater
+degree of composability and extensibility. Its tradeoff is the use of
+Ruby, which not all platform engineering teams may have expertise in.
+
+In an era of AI coding assistance, platform teams can leverage coding
+agents to assist with using Matsuri. The current release includes the
+design grammar, rspec demonstrating golden path patterns, and usage
+documentation to help get things started. Consider using Matsuri if you
+need to dynamically set up Kubernetes resources.
 
 ## Installation
 
@@ -22,13 +70,43 @@ Or install it yourself as:
 
 ## Usage
 
-TODO: Write usage instructions here
+The documentation covers the golden paths:
+
+- [Setup](docs/setup.md) — repo layout, environments, kubectl contexts,
+  and namespaces
+- [Boot](docs/boot.md) — invocation and the boot sequence
+- [Resources](docs/resources.md) — defining resources with `let()`
+- [Stacks](docs/stacks.md) — bundling resources and converging them
+  together
+- [Acceptors](docs/acceptors.md) — importing configuration values
+- [Presenters](docs/presenters.md) — transforms, seeds, and server-side
+  apply
+- [Server-Side Apply](docs/ssa.md) — field ownership, normal form, and
+  the creation protocol
+- [Diff](docs/diff.md) — previewing what converge will send
+- [Local Development](docs/local-dev.md) — the local-dev workflows
+- [Known Issues](docs/known-issues.md)
+
+Design notes — the constraints and the properties they induce — live in
+[docs/design](docs/design).
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+There is a Nix flake containing the dependencies for developing Matsuri.
+This includes Ruby and minikube. A `.envrc` is set up for `direnv`
+users. The gems are not a part of the flake so you will still have to
+run
+
+    bundle install
+
+Coding agents can invoke commands with
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+    direnv exec . bundle exec rake
+
+To release a new version, update the version number in `version.rb`, and
+then run `bundle exec rake release`, which will create a git tag for the
+version, push git commits and tags, and push the `.gem` file to
+[rubygems.org](https://rubygems.org).
 
 ## License
 
@@ -36,5 +114,5 @@ Matsuri is distributed under the MIT License
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/matsuri.
-
+Bug reports and pull requests are welcome on GitHub at
+<https://github.com/matsuri-rb/matsuri>.

data/Rakefile

@@ -1,6 +1,26 @@
+# frozen_string_literal: true
+
 require "bundler/gem_tasks"
 require "rspec/core/rake_task"
 
 RSpec::Core::RakeTask.new(:spec)
 
+namespace :docs do
+  desc "Reformat docs/**/*.md in place with pandoc (gfm, 72 columns)"
+  task :fmt do
+    FileList["docs/**/*.md"].each do |path|
+      formatted = IO.popen(%w[pandoc -f gfm -t gfm-smart --columns=72], "r+") do |io|
+        io.write File.read(path)
+        io.close_write
+        io.read
+      end
+      fail "pandoc failed on #{path}" unless $?.success?
+      # pandoc writes "``` ruby"; conventional GFM is "```ruby"
+      formatted = formatted.gsub(/^(\s*)``` (\S)/, '\1```\2')
+      File.write(path, formatted)
+      puts path
+    end
+  end
+end
+
 task :default => :spec

data/bin/console

@@ -1,4 +1,5 @@
 #!/usr/bin/env ruby
+# frozen_string_literal: true
 
 require "bundler/setup"
 require "matsuri"

data/bin/matsuri

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'matsuri'
+
+Matsuri::Initializer.start(ARGV)

data/bin/setup

@@ -1,4 +1,6 @@
 #!/bin/bash
+# frozen_string_literal: true
+
 set -euo pipefail
 IFS=$'\n\t'
 

data/docs/acceptors.md

@@ -0,0 +1,181 @@
+# Acceptors
+
+An acceptor maps the information from a different bounded context into
+the bounded context of Matsuri -- K8S resources and Matsuri abstractions
+such as `stacks`.
+
+The promises:
+
+- Fields the acceptor does not declare are ignored. Upstream can add
+  vocabulary freely without breaking anything downstream.
+- Reads the acceptor does not declare raise `NoMethodError`. Nothing
+  reaches past the declared surface.
+- There is no fallback to the underlying value. When upstream renames a
+  field, the remap happens in the acceptor, in one place, and consumers
+  do not move.
+
+## Defining Acceptors
+
+Acceptors are defined in `platform/acceptors`. The raw value arrives
+whole as `options`; each `let` declares one read on the accepted
+surface.
+
+```ruby
+Matsuri.define :acceptor, 'app' do
+  # the artifact anchor owns the empty value
+  let(:artifact) { options || {} }
+
+  # identity
+  let(:instance_name) { artifact[:name] }
+  let(:namespace)     { artifact[:namespace] }
+
+  # declared settings, with the defaults this surface promises
+  let(:log_level)    { artifact[:log_level] || 'info' }
+  let(:worker_queue) { artifact[:worker_queue] || 'default' }
+
+  # a reference: the default points at the shared tier, an entry in the
+  # artifact overrides it
+  let(:search_api_url) do
+    (artifact[:pins] || {})['search-api'] || 'https://search.staging.svc.cluster.local'
+  end
+end
+```
+
+Keep the declared surface to flat scalar values. Lets that return nested
+structures will work, but you give up treating an instance config as a
+sparse overlay of the defaults — and deep-merging nested returns is
+where that goes wrong.
+
+Because every declared read carries its default, an empty value is a
+valid one: every read lands on its default. A sparse artifact overrides
+only what it names.
+
+The empty value belongs to the acceptor, because what empty means is
+specific to the vocabulary it admits: every declared read is total over
+nil. A hash-shaped surface anchors once —
+`let(:artifact) { options || {} }` — and reads fields off the anchor; a
+scalar surface reads `options` whole, as the built-in image tag acceptor
+does.
+
+## Reading
+
+An acceptor is instantiated with the raw value and read through its
+declared surface:
+
+```ruby
+acceptor = Matsuri::Registry.acceptor('app').new(config)
+
+acceptor.namespace       # declared       => the artifact's value
+acceptor.log_level       # not in config  => 'info'
+acceptor.owner           # not declared   => NoMethodError
+```
+
+Note on defaults: the artifact usually arrives as a `Map`, and
+`Map#fetch` does not honor Hash's default-value argument (it requires a
+block). Use `options[:key] || default`.
+
+## Bare Classes
+
+The registry definition is the golden path, not a requirement. Any class
+that is instantiated with the raw value and answers the declared reads
+is an acceptor:
+
+```ruby
+class AppConfig
+  include Let
+  include RLet::LazyOptions
+
+  let(:namespace) { options[:namespace] }
+end
+```
+
+## Built-in Acceptors
+
+A stack registers acceptors under named collections with `accept`; last
+declaration per collection wins. The collection name is the options key:
+each acceptor admits its own component of the invocation options, whole,
+and cannot see another collection's component. Whoever constructs the
+stack supplies data under those keys.
+
+`Matsuri::Stack` ships two built-ins:
+
+```ruby
+accept :image_tag, Matsuri::Acceptors::ImageTag
+accept :config,    Matsuri::Acceptor
+```
+
+The `:image_tag` collection carries a scalar. The converge positional
+(`matsuri converge app v42`) arrives whole, and the acceptor's single
+read, `image_tag`, owns the `latest` default — the only place that
+default lives. With no positional, the acceptor admits the empty value
+and the default answers.
+
+The `:config` collection carries the stack-config artifact. Its built-in
+is the empty acceptor: the collection is total — `stack_config` is the
+same kind of value on every stack — and its surface is empty until a
+platform re-declares the collection with its own acceptor.
+
+A stack overrides a built-in by re-declaring its collection with its own
+acceptor (registry name or class):
+
+```ruby
+Matsuri.define :stack, 'app' do
+  accept :image_tag, 'pinned-image-tag'
+  accept :config,    'app'
+
+  needs :deployment, 'app-web'
+end
+```
+
+Downstream resources do not have to listen. Members that never read
+`image_tag` are unaffected by what the acceptor admits, and the workload
+classes themselves carry no default — a definition instantiated outside
+a stack that wants one declares its own.
+
+## Acceptors and Stacks
+
+Stacks are the first consumer. The stack never loads anything: the CLI
+owns the boundary mapping between the user invocation and the stack's
+promise, loading the file named by `--stack-config=` and supplying the
+data under the `:config` key. Anything else that constructs a stack — a
+spec, a controller, other tooling — supplies data the same way, with no
+file involved. The stack binds its acceptor once per instance, and
+resource members read the acceptor instance as `stack_config` — one
+binding per converge, so members cannot disagree about what was
+accepted. The pod template forwards the instance, so a pod definition
+reads the same binding the owning resource holds. A missing flag is the
+empty value: every declared read lands on its default. A flag with a bad
+path fails loudly; there is no fallback. The stack wiring is described
+in [stacks](stacks.md).
+
+## One Artifact, Many Consumers
+
+An artifact usually carries vocabulary for more than one consumer, and
+an acceptor claims only its own reads. Consider a feature-environment
+artifact — an engineer names the environment, and the provisioning side
+derives the rest:
+
+```yaml
+name: feat-checkout
+namespace: feat-checkout
+dns_name: feat-checkout.example.dev
+expires_at: "2026-11-15"
+contact: team-checkout
+```
+
+The stack's acceptor declares the fields the stack coordinates on — the
+namespace, and the dns name it needs to configure CORS:
+
+```ruby
+Matsuri.define :acceptor, 'app' do
+  let(:namespace) { options[:namespace] }
+  let(:dns_name)  { options[:dns_name] }
+end
+```
+
+`expires_at` and `contact` pass through this acceptor in silence. They
+are not dead vocabulary — they are the coordination path between an
+environment reaper and whoever provisions environments. The reaper does
+not have to be a matsuri consumer at all; it reads the same artifact on
+its own terms. Adding those fields required no change to this acceptor,
+and the stack never sees them.

data/docs/boot.md

@@ -0,0 +1,101 @@
+# Boot
+
+This covers what happens between invoking `matsuri` and a command
+running: how invocation works and where your own code enters. The repo
+layout itself is in [Setup](setup.md); the constraints behind this
+design — the promise layers and the properties they induce — are in
+[docs/design/boot.md](design/boot.md).
+
+## Invocation
+
+    matsuri <environment> <verb> ...
+
+Invoke from the repo base — where the Gemfile is, in the manner of
+terraform or chef. The environment is the first positional argument and
+has no other source: there is no environment variable and no
+significance to the binary's name. What you typed is what runs.
+[Setup](setup.md) covers how an environment name binds to a kubectl
+context and namespace.
+
+## The Wrapper
+
+The repo's `bin/matsuri` makes the Gemfile's version binding unavoidable
+from any shell. Pin and exec is its entire job:
+
+```bash
+#!/bin/bash -e
+SCRIPT_DIR="$(dirname "$(realpath "${BASH_SOURCE[0]}")")"
+export BUNDLE_GEMFILE="${SCRIPT_DIR}/../Gemfile"
+exec bundle exec matsuri "$@"
+```
+
+A per-environment entry point is argument injection in a sibling script,
+not wrapper logic:
+
+```bash
+#!/bin/bash -e
+# bin/staging
+exec "$(dirname "$0")/matsuri" staging "$@"
+```
+
+## The Boot Sequence
+
+1.  **Discovery.** Walk up from the working directory to the nearest
+    `config/matsuri.rb`; that directory is the repo base. No repo found
+    is a fatal error before any command logic runs.
+2.  **Load path.** The repo's `lib/` is prepended to the load path.
+    Everything after this point may require the repo's own code.
+3.  **Environment.** The first positional argument, verbatim.
+4.  **Configuration.** `config/matsuri.rb` is loaded for `Config`
+    overrides, then the platform config (`config/platform.rb`, plus a
+    repo-local `.platform.rb` if present) for environment definitions.
+    Configuration is declarative — defaults and definitions, no
+    requires.
+5.  **Activation.** `config/initializer.rb` is required, if present.
+    This is the one site that loads code: plugins and your extension
+    library. It runs after configuration, so it may branch on the
+    environment or any resolved setting.
+6.  **Dispatch.** The remaining arguments go to the CLI.
+
+## Your Extension Library
+
+Name a module after your organization, give it a file in `lib/`, and
+autoload your mixins from it:
+
+```ruby
+# lib/yourco.rb
+module Yourco
+  module Pods
+    autoload :Healthcheck, 'yourco/pods/healthcheck'
+  end
+end
+```
+
+```ruby
+# config/initializer.rb
+require 'yourco'
+```
+
+Because boot put `lib/` on the load path before the initializer runs,
+this works the same under `bundle exec`, a bare gem install, or a
+checkout. The repo needs no gemspec. The mixins themselves are ordinary
+concerns carrying `let()` definitions — see [Resources](resources.md)
+for how they compose into definitions.
+
+## Plugins
+
+The Gemfile declares what exists; the initializer activates it:
+
+```ruby
+# Gemfile
+gem 'matsuri', '~> 0.3'
+gem 'matsuri-traefik'
+```
+
+```ruby
+# config/initializer.rb
+require 'matsuri/traefik'
+```
+
+A gem declared but never required costs nothing and breaks nothing —
+adding it to the Gemfile is not yet a decision about boot.

data/docs/design/acceptors.md

@@ -0,0 +1,157 @@
+# Acceptors Design
+
+This describes the constraints and properties of the acceptor design.
+This is intended to provide both a grammar on how acceptors can be used,
+but also extended.
+
+## Framing
+
+An acceptor maps the information from a different bounded context into
+the bounded context of Matsuri -- K8S resources and Matsuri abstractions
+such as `stacks`.
+
+This unidirectional mapping deconstructs potentially complex, nested
+data into a flat let() space. The acceptor provides a stable keyspace
+which can be used within Matsuri resources. Acceptors are not intended
+to keep the domain closed, and are typically used to coordinate across
+promise layers. Coordination here are promises (intentions) crossing
+between contexts: a declared read is one such crossing, demanded by the
+consumer that declares it, arriving in the receiver's own terms.
+
+## Grammar
+
+The whole grammar is two promises. An acceptor is instantiated with the
+admitted value, whole. It answers exactly the reads it declares, each
+one a named definition on a flat surface. Everything else in this
+document is a constraint on those two promises and the properties the
+constraint induces.
+
+## Constraint: the accepted surface is declared, never inferred
+
+Every read on the surface is an explicit declaration. There is no
+delegation, no method_missing, no fallback to the underlying value.
+
+Properties:
+
+- Failure at the boundary is loud and immediate. A read the surface does
+  not answer raises at the point of use, instead of a nil surfacing
+  later inside a rendered manifest.
+- Consumers couple to an owned interface, never to the artifact's
+  schema. When upstream renames a field, the remap happens in the
+  acceptor, in one place, and no consumer moves.
+- Consumption is auditable. The full set of reads a platform permits
+  against an artifact is the acceptor's declarations — one file, not a
+  search across every consumer.
+- Moving a consumer is legible. Whether a different surface can host it
+  reduces to one question — does that surface answer these reads — and a
+  wrong answer fails loudly, at the cut.
+
+## Constraint: the value is taken whole and deconstructed once
+
+An acceptor receives the entire value at instantiation. There is one
+parse site; consumers receive the acceptor instance, never fragments of
+the raw value.
+
+Properties:
+
+- Consumers cannot disagree. Every read in an instance derives from the
+  same binding, so consistency is structural rather than disciplined.
+- The boundary has an address. Questions about what the artifact means
+  to this platform have exactly one place to be answered, and changes to
+  that meaning have exactly one place to happen.
+
+## Constraint: unconsumed vocabulary passes in silence
+
+An acceptor reads the fields it declares and is silent about the rest.
+The silence is directional and deliberate: silent toward the side that
+owns the vocabulary, loud toward the side that owns the surface.
+
+Properties:
+
+- Upstream can grow without coordination. New fields in the artifact are
+  a non-event for every acceptor that does not read them; no release of
+  the platform is needed for the artifact to say more.
+- One artifact serves many consumers. A field this acceptor passes over
+  may be another consumer's declared read — the artifact carries many
+  parties' coordination at once, each acceptor selecting its own. A
+  partial consumer that rejected unknown fields would claim the whole
+  vocabulary and break the sharing; enforcement of the full vocabulary
+  belongs to whatever system owns it, at its own boundary.
+
+## Constraint: every declared read carries its default
+
+Defaults live on the surface, in the declaration of each read — never
+scattered through consumers.
+
+Properties:
+
+- The empty value is valid. An acceptor over an empty artifact answers
+  every read with its default; emptiness is the identity, not an error
+  case.
+- Sparse values are the normal case. An artifact overrides only what it
+  names and inherits everything else, so the cost of an instance config
+  scales with its difference from the defaults, not with the size of the
+  surface.
+- Shared-tier referents are expressions, not machinery. "This reference
+  defaults to the standing instance" is written as the default of one
+  read, visible at the declaration, overridable by the artifact.
+
+## Constraint: flat scalars are the golden path, documented rather than enforced
+
+The surface is intended to be granular scalar reads. Declarations that
+return nested structures are not prevented.
+
+Properties:
+
+- On the flat surface, override composition stays monoidically lawful:
+  defaults are the identity, a sparse artifact is an overlay, and no
+  merge semantics are ever needed.
+- What a nested return forfeits is named rather than policed: the
+  sparse-overlay property stops at that read, and deep-merging the
+  structure is where the loss compounds. The cost falls on the
+  definition that chose the structure, not on the rest of the surface.
+
+## Constraint: an acceptor does not know its boundary; the accepting site binds it
+
+An acceptor does not know which boundary it stands at. The declaration
+that accepts — a stack naming its config acceptor, or any future
+accepting site — binds the acceptor at its boundary, under a named
+collection; the class itself promises only the grammar above.
+
+Properties:
+
+- One noun serves every foreign admission. New accepting sites bind
+  existing machinery; they do not mint new types.
+- Acceptors are testable in isolation: hand one a value, read the
+  surface. No stack, environment, or cluster is implied.
+- The registry definition and a bare class are interchangeable, because
+  the contract is only the two promises of the grammar. The registry is
+  a convenience of definition, not part of the contract.
+
+## Constraint: acceptors stand only at foreign boundaries
+
+An acceptor is warranted where the admitted value's vocabulary has an
+independent owner and lifecycle. Values the platform passes to itself
+are already deconstructed by their consumers' own definitions.
+
+Properties:
+
+- The pattern cannot sprawl into a general indirection layer. Where no
+  independent owner exists there is no churn to absorb and no remap to
+  localize — an acceptor there adds a layer and no property, and the
+  definition excludes it rather than a style rule.
+- The discriminator is decidable. Whether a boundary deserves an
+  acceptor reduces to one question: who owns this vocabulary, and does
+  it change on a lifecycle that is not ours?
+
+## Vocabulary note
+
+The noun names the receiving side's act, and the declaration site is its
+only habitat: consumers read the accepted value under the collection's
+own name, so "acceptor" appears exactly where the policy is declared and
+nowhere downstream. The inbound posture — read only what you declare —
+has an outbound mirror in manifest transforms, which write only what
+they own; the two share a discipline of least contact at boundaries,
+never machinery. Resemblance to presenters, adapters, or anti-corruption
+layers in other ecosystems may be adjacent and carries none of their
+promises.