@appthrust/kest 0.4.0 → 0.4.1

package/README.md

@@ -462,6 +462,280 @@ Duration strings support units like `"200ms"`, `"5s"`, `"1m"`.
 
 ## Best Practices
 
+### Test API contracts, not controllers
+
+E2E tests should describe **how an API behaves from the user's perspective**, not how a specific controller implements that behavior internally. The subject of every test should be the API resource, not the controller behind it.
+
+Why? Controllers are an implementation detail. They get renamed, split, merged, or rewritten -- but as long as the API contract is unchanged, users are unaffected. If your tests are written in terms of controllers, a harmless refactor can break your entire test suite.
+
+```ts
+// ✅ Good — the subject is the API resource
+test("Tenant API creates namespaces for each tenant", async (s) => {
+  s.given("a Tenant resource is applied");
+  const ns = await s.newNamespace();
+  await ns.apply({
+    apiVersion: "example.com/v1",
+    kind: "Tenant",
+    metadata: { name: "acme" },
+    spec: { namespaces: ["dev", "staging"] },
+  });
+
+  s.then("the Tenant reports Ready=True");
+  await ns.assert({
+    apiVersion: "example.com/v1",
+    kind: "Tenant",
+    name: "acme",
+    test() {
+      expect(this.status?.conditions).toContainEqual(
+        expect.objectContaining({ type: "Ready", status: "True" }),
+      );
+    },
+  });
+});
+```
+
+```ts
+// ❌ Bad — the subject is the controller (implementation detail)
+test("tenant-controller creates namespaces", async (s) => {
+  s.given("tenant-controller is running");
+  // ...
+  s.then("tenant-controller creates child namespaces");
+  // ...
+});
+```
+
+The same principle applies to BDD annotations -- keep `s.given()`, `s.when()`, and `s.then()` free of controller names:
+
+| ❌ Controller-centric                             | ✅ API-centric                            |
+| ------------------------------------------------- | ----------------------------------------- |
+| `s.given("tenant-controller is running")`         | `s.given("a Tenant resource exists")`     |
+| `s.when("tenant-controller reconciles")`          | `s.when("the Tenant spec is updated")`    |
+| `s.then("tenant-controller creates a Namespace")` | `s.then("the expected Namespace exists")` |
+
+### Choosing what to test in E2E
+
+E2E tests are powerful for validating **user-observable behavior** but expensive for verifying internal details. Placing implementation details in E2E tests makes refactoring harder without giving users any extra confidence.
+
+**Good candidates for E2E (API contract):**
+
+- Status transitions -- e.g. a resource reaches `Ready=True` after creation
+- Error feedback -- e.g. invalid input produces an explanatory condition like `Ready=False, reason=InvalidSpec`
+- User-facing side effects -- e.g. resources that users are expected to observe or interact with
+
+```ts
+// ✅ Assert a user-observable status condition
+await ns.assert({
+  apiVersion: "example.com/v1",
+  kind: "Database",
+  name: "my-db",
+  test() {
+    expect(this.status?.conditions).toContainEqual(
+      expect.objectContaining({ type: "Ready", status: "True" }),
+    );
+  },
+});
+```
+
+**Better left to unit / integration tests (implementation details):**
+
+- Internal label keys, annotation formats, or hash values
+- Intermediate resources that users don't directly interact with
+- Controller-internal reconciliation logic and branching
+
+```ts
+// ❌ Avoid — internal label format is an implementation detail
+await ns.assert({
+  apiVersion: "example.com/v1",
+  kind: "Database",
+  name: "my-db",
+  test() {
+    // This label may change without affecting users
+    expect(this.metadata?.labels?.["internal.example.com/config-hash"]).toBe("a1b2c3");
+  },
+});
+```
+
+When you find yourself wanting to E2E-test an intermediate resource, ask: _"Is this part of the public API contract?"_ If yes, document it as such and test it. If no, push the assertion down to a cheaper test layer and keep E2E focused on what users actually see.
+
+### Organizing test files
+
+Structure test directories around **API resources**, not controllers. This makes the test suite resilient to internal refactoring and immediately tells readers _which API behavior_ is being verified.
+
+```
+# ✅ Good — organized by API resource
+tests/e2e/
+├── tenant-api/
+│   ├── creation.test.ts
+│   └── deletion.test.ts
+├── database-api/
+│   └── provisioning.test.ts
+
+# ❌ Bad — organized by controller (implementation detail)
+tests/e2e/
+├── tenant-controller/
+│   ├── creation.test.ts
+│   └── deletion.test.ts
+├── database-controller/
+│   └── provisioning.test.ts
+```
+
+**Refactoring-friendliness checklist** -- the more "yes" answers, the better your E2E tests:
+
+- [ ] Is the subject of every test an API resource (not a controller)?
+- [ ] Can a reader understand the test from the manifest and assertions alone?
+- [ ] Do `then` assertions only check user-observable state (`status`, contracted outputs)?
+- [ ] Would splitting, merging, or renaming controllers leave all tests passing?
+
+### One scenario per file
+
+Put exactly one test scenario in one file -- from the start, not "when it gets big enough."
+
+It may be tempting to group several short scenarios into one file while they are small. But E2E tests grow: assertions get richer, edge-case inputs appear, debug aids are added. Splitting later is far harder than starting separate, and deciding _when_ to split is an unnecessary judgment call. The simplest policy is the best one: one scenario, one file, always.
+
+```
+tests/e2e/tenant-api/
+├── creates-namespaces-for-each-tenant.test.ts
+├── rejects-duplicate-tenant-names.test.ts
+├── updates-status-on-namespace-failure.test.ts
+└── deletes-child-namespaces-on-removal.test.ts
+```
+
+**Why not "split when it gets big"?**
+
+- Tests grow incrementally -- no single commit feels like "the moment to split," so it never happens.
+- Splitting a file retroactively means rewriting imports, moving fixtures, and touching unrelated tests in the same PR.
+- The threshold itself becomes a debate ("Is 250 lines too many? 400?"). A universal rule eliminates the discussion.
+
+Starting with one file per scenario reserves room to grow. Each file has built-in headroom for richer assertions, additional setup steps, and debug annotations -- without ever crowding a neighbor.
+
+**What you get:**
+
+- **Self-contained reading** -- open one file, see the full Given/When/Then without scrolling past unrelated scenarios.
+- **Surgical diffs** -- a change touches exactly one scenario, keeping PRs small and reviews focused.
+- **Failure as an address** -- a failing file name tells you which API contract broke, before you read a single line of output.
+- **Conflict-free collaboration** -- teammates edit different files, not different sections of the same file.
+
+**Name files after the behavior they verify.** A reader should know what a test checks without opening the file:
+
+| ✅ Good | ❌ Bad |
+| --- | --- |
+| `creates-namespaces-with-labels.test.ts` | `tenant-test-1.test.ts` |
+| `rejects-reserved-selector-labels.test.ts` | `validation.test.ts` |
+| `rolls-out-when-image-changes.test.ts` | `deployment-tests.test.ts` |
+
+**Exceptions.** Tiny negative-case variations of the same API -- where each case is only a few lines and they are always read together (e.g. boundary-condition lists) -- may share a file. When you do this, leave a comment explaining why, because the exception easily becomes the norm.
+
+**One-scenario-per-file checklist:**
+
+- [ ] Does each test file contain exactly one scenario?
+- [ ] Does the file name describe the behavior under test (not the controller)?
+- [ ] If a file has multiple scenarios, is there a comment justifying the exception?
+
+### Keep manifests visible in your tests
+
+E2E tests double as living documentation. Every test should read as a self-contained specification: what was applied (Given), what changed (When), and what should be true (Then). When a helper function assembles the entire manifest behind the scenes, the test body may _look_ cleaner -- but it stops serving as documentation because the reader can no longer see the actual input.
+
+```ts
+// ❌ Bad — looks tidy, but the actual manifest is hidden inside makeDeployment().
+//    A reader cannot tell what is being applied without jumping to the helper.
+//    The diff between v1 and v2 is buried in a parameter change.
+function makeDeployment(name: string, image: string) {
+  return {
+    apiVersion: "apps/v1",
+    kind: "Deployment",
+    metadata: { name, labels: { app: name } },
+    spec: {
+      replicas: 2,
+      selector: { matchLabels: { app: name } },
+      template: {
+        metadata: { labels: { app: name } },
+        spec: { containers: [{ name: "app", image }] },
+      },
+    },
+  };
+}
+
+test("rollout updates the image", async (s) => {
+  const ns = await s.newNamespace();
+  await ns.apply(makeDeployment("my-app", "my-app:v1"));
+  await ns.apply(makeDeployment("my-app", "my-app:v2"));
+  // What exactly changed? You have to read makeDeployment to find out.
+});
+```
+
+```ts
+// ✅ Good — the full manifest is right here; the diff between steps is obvious.
+test("rollout updates the image", async (s) => {
+  const ns = await s.newNamespace();
+
+  s.when("I apply a Deployment at v1");
+  await ns.apply({
+    apiVersion: "apps/v1",
+    kind: "Deployment",
+    metadata: { name: "my-app" },
+    spec: {
+      replicas: 2,
+      selector: { matchLabels: { app: "my-app" } },
+      template: {
+        metadata: { labels: { app: "my-app" } },
+        spec: { containers: [{ name: "app", image: "my-app:v1" }] },
+      },
+    },
+  });
+
+  s.when("I update to v2");
+  await ns.apply({
+    apiVersion: "apps/v1",
+    kind: "Deployment",
+    metadata: { name: "my-app" },
+    spec: {
+      replicas: 2,
+      selector: { matchLabels: { app: "my-app" } },
+      template: {
+        metadata: { labels: { app: "my-app" } },
+        spec: { containers: [{ name: "app", image: "my-app:v2" }] },
+        //                                          ^^^^^^^^^ the diff
+      },
+    },
+  });
+});
+```
+
+The duplication is intentional -- in E2E tests, **readability beats DRY**. Duplicated manifests make each test self-explanatory and keep failure reports easy to match against the test source. Notice the `^^^ the diff` comment in the example above -- adding a short inline comment to highlight the key change is a simple technique that makes the intent even clearer.
+
+When manifests become too large to inline comfortably, extract them into **static fixture files** instead of helper functions. Both YAML and TypeScript files work:
+
+```ts
+// ✅ Good — static fixture files keep the input visible at a glance
+await ns.apply(import("./fixtures/deployment-v1.yaml"));
+await ns.apply(import("./fixtures/deployment-v2.yaml"));
+
+// TypeScript files work too — useful when converting from inline objects
+await ns.apply(import("./fixtures/deployment-v1.ts"));
+```
+
+If you do use helpers, limit them to **name generation** -- never to assembling `spec`:
+
+```ts
+// ✅ Good — helper generates a name; the manifest stays in the test
+const name = s.generateName("deploy-");
+await ns.apply({
+  apiVersion: "apps/v1",
+  kind: "Deployment",
+  metadata: { name },
+  spec: {
+    /* full spec here, not hidden in a function */
+  },
+});
+```
+
+**Manifest-visibility checklist:**
+
+- [ ] Can you understand the full input by reading just the test file?
+- [ ] Is the diff between test steps visible as a spec-level change?
+- [ ] Can you reconstruct the applied manifest from the failure report alone?
+- [ ] Are helper functions limited to name generation (no `spec` assembly)?
+
 ### Avoiding naming collisions between tests
 
 When tests run in parallel, hard-coded resource names can collide (especially when you create cluster-scoped resources).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@appthrust/kest",
-  "version": "0.4.0",
+  "version": "0.4.1",
   "description": "Kubernetes E2E testing framework designed for humans and AI alike",
   "type": "module",
   "module": "ts/index.ts",