agentme 0.25.2 → 0.26.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.filedist-package.yml +1 -1
- package/.xdrs/agentme/edrs/application/003-javascript-project-tooling.md +4 -4
- package/.xdrs/agentme/edrs/application/010-golang-project-tooling.md +4 -4
- package/.xdrs/agentme/edrs/application/014-python-project-tooling.md +10 -10
- package/.xdrs/agentme/edrs/application/018-ai-llm-development-standards.md +1 -1
- package/.xdrs/agentme/edrs/application/019-ai-agents-development-standards.md +8 -8
- package/.xdrs/agentme/edrs/application/021-ai-workflow-development-standards.md +3 -6
- package/.xdrs/agentme/edrs/application/025-ai-agent-xdrs-knowledge-layer.md +1 -1
- package/.xdrs/agentme/edrs/application/026-pragmatic-hexagonal-architecture.md +2 -2
- package/.xdrs/agentme/edrs/application/028-ai-eval-standards.md +3 -3
- package/.xdrs/agentme/edrs/application/029-ai-workflow-naming-conventions.md +1 -1
- package/.xdrs/agentme/edrs/application/030-ai-test-types-taxonomy.md +2 -2
- package/.xdrs/agentme/edrs/devops/005-monorepo-structure.md +25 -25
- package/.xdrs/agentme/edrs/devops/006-github-pipelines.md +2 -2
- package/.xdrs/agentme/edrs/devops/008-common-targets.md +33 -33
- package/.xdrs/agentme/edrs/devops/017-tool-execution-and-scripting.md +1 -1
- package/.xdrs/agentme/edrs/devops/027-environment-variable-configuration.md +3 -3
- package/.xdrs/agentme/edrs/governance/013-contributing-guide-requirements.md +35 -9
- package/.xdrs/agentme/edrs/observability/011-service-health-check-endpoint.md +1 -1
- package/.xdrs/agentme/edrs/principles/004-unit-test-requirements.md +6 -4
- package/.xdrs/agentme/edrs/principles/007-project-quality-standards.md +21 -21
- package/.xdrs/agentme/edrs/principles/009-error-handling.md +1 -1
- package/.xdrs/agentme/edrs/principles/012-continuous-xdr-enrichment.md +27 -9
- package/.xdrs/agentme/edrs/principles/016-cross-language-module-structure.md +1 -1
- package/.xdrs/agentme/edrs/principles/022-secrets-management.md +26 -24
- package/.xdrs/agentme/edrs/principles/023-coding-abstraction-practices.md +1 -1
- package/package.json +1 -1
package/.filedist-package.yml
CHANGED
|
@@ -61,7 +61,7 @@ coverageProvider: 'v8',
|
|
|
61
61
|
coverageDirectory: '.cache/coverage',
|
|
62
62
|
```
|
|
63
63
|
|
|
64
|
-
Builds that miss the threshold
|
|
64
|
+
Builds that miss the threshold MUST NOT be merged.
|
|
65
65
|
|
|
66
66
|
#### Project structure
|
|
67
67
|
|
|
@@ -105,7 +105,7 @@ Internal source code MUST be organized following [agentme-edr-026](026-pragmatic
|
|
|
105
105
|
|
|
106
106
|
When a repository contains multiple JavaScript/TypeScript packages, each package MUST live in its own module folder such as `lib/my-package/` or `services/my-service/`, each with its own `Makefile`, `README.md`, `dist/`, and `.cache/`.
|
|
107
107
|
|
|
108
|
-
All tool caches, incremental state files, and workspace-local config outputs MUST be written under `.cache/`. This applies to every tool without exception. Cache and state paths MUST be declared in the tool's own configuration file —
|
|
108
|
+
All tool caches, incremental state files, and workspace-local config outputs MUST be written under `.cache/`. This applies to every tool without exception. Cache and state paths MUST be declared in the tool's own configuration file — MUST NOT be on the command line — so that the location is enforced regardless of how the tool is invoked:
|
|
109
109
|
|
|
110
110
|
| Tool | Config file | Setting | Value |
|
|
111
111
|
|------|------------|---------|-------|
|
|
@@ -114,7 +114,7 @@ All tool caches, incremental state files, and workspace-local config outputs MUS
|
|
|
114
114
|
| **TypeScript** | `tsconfig.json` | `tsBuildInfoFile` | `.cache/tsbuildinfo` |
|
|
115
115
|
| **Jest coverage** | `jest.config.js` | `coverageDirectory` | `.cache/coverage` |
|
|
116
116
|
|
|
117
|
-
|
|
117
|
+
Tools MUST NOT write cache or state files to the project root, `src/`, or any other directory outside `.cache/`. Passing cache paths as Makefile or CLI flags instead of config-file settings is not allowed.
|
|
118
118
|
|
|
119
119
|
Contributors and CI MUST invoke the commands below as `make <target>`. The Makefile recipes themselves MUST call the underlying tools through `mise exec -- <tool> ...`.
|
|
120
120
|
|
|
@@ -144,7 +144,7 @@ Contributors and CI MUST invoke the commands below as `make <target>`. The Makef
|
|
|
144
144
|
|
|
145
145
|
Each sub-folder under `examples/` is an independent package. The Makefile installs the locally built `.tgz` pack from `lib/dist/` so examples simulate real external usage.
|
|
146
146
|
|
|
147
|
-
Examples MUST remain outside the module root and MUST consume the package through the packed artifact in `dist
|
|
147
|
+
Examples MUST remain outside the module root and MUST consume the package through the packed artifact in `dist/`. MUST NOT use `../src` imports or other direct source links.
|
|
148
148
|
|
|
149
149
|
Module-specific integration tests that are not just runnable examples belong in `lib/tests_integration/` or a sibling `tests_integration/` when they cover multiple modules.
|
|
150
150
|
|
|
@@ -30,7 +30,7 @@ A predictable layout and minimal external tooling keep Go projects approachable,
|
|
|
30
30
|
| **golangci-lint** | Linting — aggregates many linters in one fast run; configured via `.golangci.yml` |
|
|
31
31
|
| **monotag** | Version tagging from git history for the `publish` target |
|
|
32
32
|
|
|
33
|
-
All commands
|
|
33
|
+
All commands MUST be run exclusively through the Makefile and MUST NOT be called ad-hoc. The project root **MUST** define a `.mise.toml` that pins `go`, `golangci-lint`, and any other Go-related CLIs used by the project. Contributors and CI **MUST** bootstrap with `make setup` or `mise install`, then invoke routine work with `make <target>`. Each Makefile recipe **MUST** execute the underlying tool through `mise exec -- <tool> ...`, following [agentme-edr-017](../devops/017-tool-execution-and-scripting.md).
|
|
34
34
|
Direct installation of project-required Go CLIs with `go install ...@latest` as a repair step is **NOT** allowed unless an XDR for that repository explicitly permits it.
|
|
35
35
|
|
|
36
36
|
#### Project structure
|
|
@@ -80,7 +80,7 @@ Direct installation of project-required Go CLIs with `go install ...@latest` as
|
|
|
80
80
|
- Business logic lives in named feature packages under `app/` (e.g., `app/ownership/`, `app/changes/`). These packages are importable and testable without any CLI or adapter concerns.
|
|
81
81
|
- `adapters/cli/` packages own flag parsing, output formatting, and the wiring between flags and `app/` functions. No business logic lives in adapter packages.
|
|
82
82
|
- Outbound adapters live under `adapters/connectors/` with one subfolder per external resource, named descriptively (e.g., `postgres/`, `stripe-api/`, `redis-cache/`).
|
|
83
|
-
- `shared/`
|
|
83
|
+
- `shared/` MUST contain only infrastructure-agnostic utilities — not business rules or domain logic.
|
|
84
84
|
- Packages are flat by default; sub-packages are only introduced when a feature package itself exceeds ~400 lines or has clearly separable sub-concerns.
|
|
85
85
|
- Application MAY import from Adapters when it simplifies the design (pragmatic coupling per edr-022 rule 05).
|
|
86
86
|
- Consumer examples for reusable libraries belong in a sibling `examples/` folder and MUST import the public module path rather than reaching into internal source paths. Because Go libraries are not typically consumed from a local packaged artifact, local example validation may use a temporary module replacement for resolution, but the import path MUST remain the public module path.
|
|
@@ -120,7 +120,7 @@ make test
|
|
|
120
120
|
make lint
|
|
121
121
|
```
|
|
122
122
|
|
|
123
|
-
The Makefile recipes themselves
|
|
123
|
+
The Makefile recipes themselves MUST use `mise exec --` for the underlying tool commands.
|
|
124
124
|
|
|
125
125
|
#### Cross-platform binary distribution
|
|
126
126
|
|
|
@@ -148,7 +148,7 @@ All tool caches, incremental state files, and build outputs MUST be written unde
|
|
|
148
148
|
| **golangci-lint cache** | `GOLANGCI_LINT_CACHE` env var | `export GOLANGCI_LINT_CACHE := $(CURDIR)/.cache/golangci-lint` |
|
|
149
149
|
| **Test coverage output** | `-coverprofile` flag in `test` target | `.cache/coverage.out` |
|
|
150
150
|
|
|
151
|
-
|
|
151
|
+
Tools MUST NOT write cache or state files to the project root or any directory outside `.cache/`. Passing cache paths as per-recipe environment overrides instead of top-level Makefile exports is not allowed.
|
|
152
152
|
|
|
153
153
|
#### Linting
|
|
154
154
|
|
|
@@ -34,13 +34,13 @@ A single dependency manager, isolated package internals under `lib/`, and a stan
|
|
|
34
34
|
| **pytest-cov** | Coverage reporting and threshold enforcement |
|
|
35
35
|
| **pip-audit** | Dependency CVE audit |
|
|
36
36
|
|
|
37
|
-
All routine commands
|
|
37
|
+
All routine commands MUST run through the project `Makefile`. MUST NOT call `uv`, `ruff`, `pytest`, or `ty` directly in docs, CI, or daily development workflows.
|
|
38
38
|
|
|
39
39
|
The repository root MUST define a `.mise.toml` that pins Python and uv. Contributors and CI MUST bootstrap with `make setup` or `mise install`, then invoke routine work with `make <target>`. Each Makefile recipe MUST execute the underlying tool through `mise exec -- <tool> ...`, following [agentme-edr-017](../devops/017-tool-execution-and-scripting.md). Using routine project CLI commands directly outside the Makefile contract is not allowed.
|
|
40
40
|
|
|
41
|
-
The root `.venv/` is the canonical environment location for both the library and all examples. Subdirectory commands
|
|
41
|
+
The root `.venv/` is the canonical environment location for both the library and all examples. Subdirectory commands MUST set `UV_PROJECT_ENVIRONMENT` to the workspace root `.venv/` instead of creating nested virtual environments.
|
|
42
42
|
|
|
43
|
-
All tool caches, incremental state files, and workspace-local outputs MUST be written under `.cache/`. Cache paths MUST be declared in the tool's own configuration file —
|
|
43
|
+
All tool caches, incremental state files, and workspace-local outputs MUST be written under `.cache/`. Cache paths MUST be declared in the tool's own configuration file — MUST NOT be on the command line or as Makefile CLI flags — so the location is enforced regardless of how the tool is invoked. Configure the following in `lib/pyproject.toml`:
|
|
44
44
|
|
|
45
45
|
| Tool | Config section | Setting | Value |
|
|
46
46
|
|------|---------------|---------|-------|
|
|
@@ -50,7 +50,7 @@ All tool caches, incremental state files, and workspace-local outputs MUST be wr
|
|
|
50
50
|
| **coverage HTML** | `[tool.coverage.html]` | `directory` | `".cache/coverage-html"` |
|
|
51
51
|
| **uv** | `[tool.uv]` in `lib/pyproject.toml` | `cache-dir` | `".cache/uv"` |
|
|
52
52
|
|
|
53
|
-
|
|
53
|
+
Tools MUST NOT write cache or state files to the project root, `src/`, `tests/`, or any directory outside `.cache/`. Passing cache paths as CLI flags or Makefile recipe-level env overrides instead of `pyproject.toml` settings is not allowed.
|
|
54
54
|
|
|
55
55
|
#### Project structure
|
|
56
56
|
|
|
@@ -98,7 +98,7 @@ Use the `lib/src/` layout for import safety and packaging clarity. Keep tests un
|
|
|
98
98
|
|
|
99
99
|
Internal source code MUST be organized following [agentme-edr-026](026-pragmatic-hexagonal-architecture.md): `adapters/` (inbound and outbound I/O boundaries), `app/` (business logic), and `shared/` (infrastructure-agnostic utilities).
|
|
100
100
|
|
|
101
|
-
Libraries and shared utilities
|
|
101
|
+
Libraries and shared utilities MUST include an `examples/` folder and wire example execution into the root `test` flow, following [agentme-edr-007](../principles/007-project-quality-standards.md). Each example directory is its own Python project with its own `pyproject.toml`, and examples MUST import the library as a consumer would rather than reaching back into `lib/src/` with relative imports. Local example verification MUST install the wheel built into `lib/dist/`; do not use editable or path-based dependencies back to `lib/`.
|
|
102
102
|
|
|
103
103
|
Python keeps unit tests under `lib/tests/` by default because that remains the more common and maintainable convention for typed/package-based projects than co-locating tests beside every source file. Integration tests belong in `lib/tests_integration/`, and benchmark harnesses belong in `lib/tests_benchmark/` when they are more than a single micro-benchmark helper.
|
|
104
104
|
|
|
@@ -114,7 +114,7 @@ When ty runs from `lib/`, it auto-discovers the virtual environment via the `VIR
|
|
|
114
114
|
|
|
115
115
|
Ruff is the default formatter and linter. Do not add Black, isort, or Flake8 unless another XDR for that repository explicitly requires them.
|
|
116
116
|
|
|
117
|
-
All Python projects
|
|
117
|
+
All Python projects MUST configure the following sections in `lib/pyproject.toml`. The cache-related settings are mandatory per the `.cache/` policy above:
|
|
118
118
|
|
|
119
119
|
```toml
|
|
120
120
|
[tool.pytest.ini_options]
|
|
@@ -155,11 +155,11 @@ ignore = ["ANN002", "ANN003", "ANN401", "D100", "D101", "D102", "D103", "D104",
|
|
|
155
155
|
ignore-overlong-task-comments = true
|
|
156
156
|
```
|
|
157
157
|
|
|
158
|
-
Adjust `target-version` to match the project's minimum supported Python version. The `cache-dir` keeps Ruff's cache under `.cache/ruff` alongside other tool caches. The `src` list
|
|
158
|
+
Adjust `target-version` to match the project's minimum supported Python version. The `cache-dir` keeps Ruff's cache under `.cache/ruff` alongside other tool caches. The `src` list MUST include every directory that contains importable Python code. The `select` list enables a broad set of rules covering style, correctness, performance, security, and documentation. The `ignore` list suppresses rules that are either too noisy or conflict with the chosen docstring style.
|
|
159
159
|
|
|
160
|
-
ty
|
|
160
|
+
ty MUST run on every lint pass. The default rule set is the minimum baseline; projects may enable stricter rules as the codebase matures.
|
|
161
161
|
|
|
162
|
-
Pytest coverage
|
|
162
|
+
Pytest coverage MUST fail below 80% line and branch coverage, following [agentme-edr-004](../principles/004-unit-test-requirements.md).
|
|
163
163
|
|
|
164
164
|
#### Makefile targets
|
|
165
165
|
|
|
@@ -198,7 +198,7 @@ The root `Makefile` is the only contract for CI and contributors. It delegates l
|
|
|
198
198
|
| `dev` | Same as `run`, optionally with repository-specific dev defaults |
|
|
199
199
|
| `publish` | `mise exec -- uv publish --project .` after versioning and packaging are complete |
|
|
200
200
|
|
|
201
|
-
The root `Makefile`
|
|
201
|
+
The root `Makefile` MUST remain the only contract for CI and contributors, in line with [agentme-edr-008](../devops/008-common-targets.md).
|
|
202
202
|
|
|
203
203
|
## Considered Options
|
|
204
204
|
|
|
@@ -51,7 +51,7 @@ Every component that interacts with an LLM MUST be classified as exactly one of
|
|
|
51
51
|
All direct LLM calls MUST use **LangChain** via the `langchain` packages.
|
|
52
52
|
|
|
53
53
|
- Use `langchain-openai` as the provider integration layer. It supports both OpenAI and Azure OpenAI.
|
|
54
|
-
-
|
|
54
|
+
- LLM providers MUST be configured using explicit library attributes such as `api_key`, `base_url`, `model`, `api_version`, etc. MUST NOT rely on environment variables for LLM configuration.
|
|
55
55
|
- Configuration MUST be passed via constructor parameters or configuration objects, making dependencies explicit and testable.
|
|
56
56
|
|
|
57
57
|
**Example of explicit configuration:**
|
|
@@ -46,11 +46,11 @@ When an agent requires a **local sandbox** — an isolated environment where the
|
|
|
46
46
|
Use deepagents sandbox whenever ANY of the following is true:
|
|
47
47
|
- The agent needs to execute shell commands or scripts in a controlled environment.
|
|
48
48
|
- The agent needs to list, read, or search files across multiple directories at runtime.
|
|
49
|
-
- The agent operates on user-supplied or generated file trees that
|
|
49
|
+
- The agent operates on user-supplied or generated file trees that MUST NOT escape a sandboxed boundary.
|
|
50
50
|
|
|
51
51
|
**Integration requirements:**
|
|
52
52
|
|
|
53
|
-
- The sandbox MUST
|
|
53
|
+
- The sandbox MUST be initialized with `virtual_mode=True` to prevent the agent from reading or writing files outside the mounted workspace. Omitting this flag allows the agent unrestricted host filesystem access, which is a security violation.
|
|
54
54
|
- Initialize the sandbox at the start of the agent run and shut it down in the same `try/finally` block.
|
|
55
55
|
- Pass the sandbox handle into the agent's state so all tool calls share the same sandbox instance.
|
|
56
56
|
- If the host-side code needs to pass files into the sandbox (e.g. generated config or input data), create a temporary directory with `tempfile.mkdtemp()`, write the files there, and mount it into the sandbox. Clean it up in the `finally` block.
|
|
@@ -120,11 +120,11 @@ When multiple agents are needed, one of these composition patterns MUST be chose
|
|
|
120
120
|
|---|---|
|
|
121
121
|
| Single agent + tools | All tools serve the same goal; agent completes in one session |
|
|
122
122
|
| Multiple workflow-orchestrated agents | Each agent has a distinct goal; outputs flow between agents; deterministic sequencing needed |
|
|
123
|
-
| Nested agents (FORBIDDEN) |
|
|
123
|
+
| Nested agents (FORBIDDEN) | MUST NOT use nested agents; MUST use workflow orchestration instead |
|
|
124
124
|
|
|
125
125
|
#### 06-agent-system-prompt-structure
|
|
126
126
|
|
|
127
|
-
Every agent system prompt MUST follow this XML-section template. Sections
|
|
127
|
+
Every agent system prompt MUST follow this XML-section template. Sections MUST appear in this order. Required sections MUST be present; optional sections may be omitted when they genuinely do not apply; MUST NOT be reordered.
|
|
128
128
|
|
|
129
129
|
```xml
|
|
130
130
|
[specific task description to the agent. if not defined use the default prompt "Execute your objective taking into consideration the inputs provided and all the sections described below"]
|
|
@@ -189,7 +189,7 @@ The current OS is: [operating system name].
|
|
|
189
189
|
|---|---|---|
|
|
190
190
|
| `<SYSTEM_CONTEXT>` | Optional | Runtime environment context injected at invocation time (e.g., current date in YYYY-MM-DD, OS). Include whenever the agent may need temporal or environment awareness. Time MUST NOT be included — it changes every second and breaks prompt caching. |
|
|
191
191
|
| `<OBJECTIVE>` | Required | One or two sentences summarising the agent's main deliverable. |
|
|
192
|
-
| `<
|
|
192
|
+
| `<AGENT_ROLE>` | Required | Agent persona and expertise. When inside a workflow, MUST reference its node name from `<WORKFLOW_CONTEXT>`. |
|
|
193
193
|
| `<INPUT>` | Required | List ALL inputs. For workflow agents: workflow-level inputs first, then agent-specific inputs. |
|
|
194
194
|
| `<STEPS>` | Optional | Include when the agent follows a non-trivial numbered sequence of steps. |
|
|
195
195
|
| `<TOOL_GUIDANCE>` | Optional | Include when tool use order or conditions need explicit direction. |
|
|
@@ -200,7 +200,7 @@ The current OS is: [operating system name].
|
|
|
200
200
|
**Formatting rules:**
|
|
201
201
|
|
|
202
202
|
- MUST use XML tags to delimit every section.
|
|
203
|
-
- The content of each section MUST start on the line immediately after the opening tag —
|
|
203
|
+
- The content of each section MUST start on the line immediately after the opening tag — MUST NOT be inline with it.
|
|
204
204
|
- Each closing tag MUST be followed by a blank line before the next opening tag, so sections are visually separated.
|
|
205
205
|
|
|
206
206
|
```xml
|
|
@@ -208,9 +208,9 @@ The current OS is: [operating system name].
|
|
|
208
208
|
Produce a plan for the current batch of files.
|
|
209
209
|
</OBJECTIVE>
|
|
210
210
|
|
|
211
|
-
<
|
|
211
|
+
<AGENT_ROLE>
|
|
212
212
|
You are the batch_plan_agent.
|
|
213
|
-
</
|
|
213
|
+
</AGENT_ROLE>
|
|
214
214
|
```
|
|
215
215
|
|
|
216
216
|
#### 07-agent-output-format
|
|
@@ -41,7 +41,7 @@ Projects MUST follow the eval dataset and implementation requirements defined in
|
|
|
41
41
|
|
|
42
42
|
#### 05-flow-documentation
|
|
43
43
|
|
|
44
|
-
Each workflow MUST be documented as a **Mermaid graph** in a `README.md`. The diagram
|
|
44
|
+
Each workflow MUST be documented as a **Mermaid graph** in a `README.md`. The diagram MUST match the LangGraph `StateGraph` definition:
|
|
45
45
|
|
|
46
46
|
- Use `graph TD` or `graph LR` direction.
|
|
47
47
|
- Label each node with its Python function name.
|
|
@@ -109,12 +109,9 @@ Nodes MUST follow the naming conventions defined in [agentme-edr-029](029-ai-wor
|
|
|
109
109
|
|
|
110
110
|
#### 10-workflow-unit-testing
|
|
111
111
|
|
|
112
|
-
All LLM calls within workflow nodes are external API calls and MUST be mocked in unit tests per [agentme-edr-018](018-ai-llm-development-standards.md) rule `04-unit-test-mocking`. Workflow unit tests
|
|
112
|
+
All LLM calls within workflow nodes are external API calls and MUST be mocked in unit tests per [agentme-edr-018](018-ai-llm-development-standards.md) rule `04-unit-test-mocking`. Workflow unit tests MUST run fully offline with no real LLM provider calls.
|
|
113
113
|
|
|
114
|
-
Choose the mock utility
|
|
115
|
-
|
|
116
|
-
- Use **`FakeListChatModel`** when nodes only read `AIMessage.content` (e.g. a routing node that checks a text label).
|
|
117
|
-
- Use **`GenericFakeChatModel`** when any node in the workflow expects tool calls, structured outputs, or when the workflow contains `_agent` nodes that drive a tool-invocation loop.
|
|
114
|
+
Choose the mock utility per [agentme-edr-018](018-ai-llm-development-standards.md) rule `04-unit-test-mocking`. For workflows containing `_agent` nodes that drive a tool-invocation loop, MUST use `GenericFakeChatModel`.
|
|
118
115
|
|
|
119
116
|
**Example — workflow with plain-text LLM nodes:**
|
|
120
117
|
|
|
@@ -79,7 +79,7 @@ Follow [agentme-edr-019 rule 02-local-sandbox](019-ai-agents-development-standar
|
|
|
79
79
|
| `lib/data/.xdrs/` | XDRS Policy and Skill documents | `/.xdrs/` (read-only) |
|
|
80
80
|
| Generated at startup | `AGENTS.md` instructing the agent to consult XDRS | `/AGENTS.md` (read-only) |
|
|
81
81
|
|
|
82
|
-
XDRS documents MUST
|
|
82
|
+
XDRS documents MUST be mounted at `/.xdrs/`. `AGENTS.md` MUST be placed at the sandbox root (`/AGENTS.md`).
|
|
83
83
|
|
|
84
84
|
Example XDRS mount additions:
|
|
85
85
|
|
|
@@ -139,7 +139,7 @@ When a mock implementation needs to be **reused across multiple tests or importe
|
|
|
139
139
|
- Single-test use → define the mock inline inside the test file (per rule `09` example; no file needed)
|
|
140
140
|
- Reusable across multiple tests OR used from `eval.py` → define in a separate `_mock` file
|
|
141
141
|
|
|
142
|
-
**Scope:** applies to any source file in `adapters/connectors/`, `app/`, or `shared/`. MUST NOT be used for inbound adapters (`cli/`, `http/`, `grpc/`) — those are entry points and
|
|
142
|
+
**Scope:** applies to any source file in `adapters/connectors/`, `app/`, or `shared/`. MUST NOT be used for inbound adapters (`cli/`, `http/`, `grpc/`) — those are entry points and MUST NOT be mocked (rule `09`).
|
|
143
143
|
|
|
144
144
|
**Naming:** insert `_mock` immediately before the file extension:
|
|
145
145
|
|
|
@@ -155,7 +155,7 @@ When a mock implementation needs to be **reused across multiple tests or importe
|
|
|
155
155
|
|
|
156
156
|
**Mock contract:**
|
|
157
157
|
- MUST accept a `fixtures` parameter (constructor argument or factory function argument); the value is whatever `mock_fixtures[key]` contains from the dataset entry — its internal structure is opaque and interpreted by the mock implementation
|
|
158
|
-
- MUST NOT fall back to real external calls under any circumstance — if a call cannot be satisfied from the provided fixtures, MUST raise an explicit error (
|
|
158
|
+
- MUST NOT fall back to real external calls under any circumstance — if a call cannot be satisfied from the provided fixtures, MUST raise an explicit error (MUST NOT silently return `null`, `undefined`, or an empty value)
|
|
159
159
|
|
|
160
160
|
## References
|
|
161
161
|
|
|
@@ -88,8 +88,8 @@ Each `eval.py` script MUST:
|
|
|
88
88
|
|
|
89
89
|
- Load the golden dataset from `golden_dataset/` in the same eval folder, following [agentme-edr-024](024-ml-dataset-structure.md) and the entry envelope in [agentme-edr-030](030-ai-test-types-taxonomy.md) rule `02` (one JSON file per entry, `test_types` array, `input`, `expected_output`, optional `mock_fixtures`).
|
|
90
90
|
- Accept a required `--type=<test_type>|all` CLI argument and filter entries whose `test_types` array contains the requested value; `--type=all` includes every entry.
|
|
91
|
-
- Iterate **entry-first**: for each entry in the filtered set, invoke the real component exactly once; then score that single `actual_output` for every `test_types` value the entry carries that falls within the current `--type` scope —
|
|
92
|
-
- When an entry contains `mock_fixtures` ([agentme-edr-030](030-ai-test-types-taxonomy.md) rule `02`), configure each named mock adapter with its fixture data BEFORE invoking the component for that entry. Each entry MUST use fresh mock instances so fixture state does not bleed across entries. `mock_fixtures` applies to all test types including `human`. `mock_fixtures` MUST NOT configure LLM adapters — the LLM call MUST
|
|
91
|
+
- Iterate **entry-first**: for each entry in the filtered set, invoke the real component exactly once; then score that single `actual_output` for every `test_types` value the entry carries that falls within the current `--type` scope — MUST NOT invoke the component more than once per entry per run.
|
|
92
|
+
- When an entry contains `mock_fixtures` ([agentme-edr-030](030-ai-test-types-taxonomy.md) rule `02`), configure each named mock adapter with its fixture data BEFORE invoking the component for that entry. Each entry MUST use fresh mock instances so fixture state does not bleed across entries. `mock_fixtures` applies to all test types including `human`. `mock_fixtures` MUST NOT configure LLM adapters — the LLM call MUST be real (see [agentme-edr-030](030-ai-test-types-taxonomy.md) rule `03`). How mock adapters are discovered and instantiated is left to the project; see [agentme-edr-026](026-pragmatic-hexagonal-architecture.md) rule `10` for the `_mock` file naming and placement convention.
|
|
93
93
|
- Run every component invocation against **real LLM providers** (not mocked responses), to capture model drift.
|
|
94
94
|
- For `human` entries: invoke the component to capture `actual_output`, export each entry's `input`, `expected_output.human_test` instructions, and `actual_output` into a manual-review checklist (`report-human.md`). MUST NOT invoke an automated scorer and MUST NOT enforce a pass/fail threshold for it. Other `test_types` on the same entry (e.g. `functional`) are still scored automatically.
|
|
95
95
|
- After all entries are processed, compute aggregate metrics per test type, log them to a local MLflow experiment (see rule `04`), write one `report-<type>.md` per evaluated test type (rule `03`), and exit with a non-zero status when any metric falls below its defined threshold per [agentme-edr-007](../principles/007-project-quality-standards.md) rule `07-statistical-models-must-have-eval-targets`. The `human` type has no threshold and does not trigger a non-zero exit.
|
|
@@ -235,7 +235,7 @@ Where $\hat{p}$ is observed accuracy and $n$ is sample count. Accuracy and F1 ar
|
|
|
235
235
|
- MLflow run: experiment `workflow-document-review/eval-basic`, tag `test_types=functional` — view with `mlflow ui`
|
|
236
236
|
```
|
|
237
237
|
|
|
238
|
-
**`human` type artifact:** instead of `report-human.md` with metrics, `--type=human` produces a checklist artifact (still named `report-human.md`) listing, per entry, its `input`, `expected_output.human_test` instructions, and the captured `actual_output` — with no Overall Results table, threshold, or PASS/FAIL section, since this type
|
|
238
|
+
**`human` type artifact:** instead of `report-human.md` with metrics, `--type=human` produces a checklist artifact (still named `report-human.md`) listing, per entry, its `input`, `expected_output.human_test` instructions, and the captured `actual_output` — with no Overall Results table, threshold, or PASS/FAIL section, since this type MUST NOT be auto-scored.
|
|
239
239
|
|
|
240
240
|
#### 04-eval-mlflow-unique-port
|
|
241
241
|
|
|
@@ -46,7 +46,7 @@ def code_reviewer_agent(state): ...
|
|
|
46
46
|
graph.add_node("code_reviewer_agent", code_reviewer_agent)
|
|
47
47
|
```
|
|
48
48
|
|
|
49
|
-
Names MUST NOT use generic labels such as `node1`, `process`, or `run`. Each name
|
|
49
|
+
Names MUST NOT use generic labels such as `node1`, `process`, or `run`. Each name MUST clearly express what action the node performs.
|
|
50
50
|
|
|
51
51
|
Judge nodes use a **prefix** convention instead of a suffix: the name MUST start with `evaluate_` followed by the subject being judged (e.g. `evaluate_progress`, `evaluate_quality`, `evaluate_completeness`, `evaluate_relevance`). This makes judge nodes immediately distinguishable from all other node types at a glance.
|
|
52
52
|
|
|
@@ -41,8 +41,8 @@ Every golden dataset entry (a JSON file in `golden_dataset/data/`) MUST have thi
|
|
|
41
41
|
|
|
42
42
|
- `test_types` — array, values MUST come from rule `04`'s enum, MUST contain at least one value. An entry MAY carry more than one value additively (e.g. `["functional", "smoke", "human"]`) — no test type excludes another.
|
|
43
43
|
- `input` — for Prompt-tier components, a raw prompt string or the prompt template's input parameters object; for Agent/Workflow-tier components, the input attributes object passed to the component.
|
|
44
|
-
- `expected_output` — the fields used to score the entry under each of its automated `test_types`: output attributes for an LLM-as-judge rubric, a target for vector-similarity scoring, or exact attribute values for strict comparison. When `human` is one of the entry's `test_types`, `expected_output` MUST additionally include a `human_test` string field with manual-verification instructions (e.g. `"check for ethical issues, verify record change in system X"`) — this supplements, and
|
|
45
|
-
- `mock_fixtures` — optional object; keys identify the adapter or external system to mock (SHOULD match the connector folder name under `adapters/connectors/<name>` for readability, though not enforced), values are any valid JSON interpreted by the mock implementation. When present, eval.py MUST configure each named mock adapter with its fixture data BEFORE invoking the component for that entry; each entry MUST use fresh mock instances to prevent state from bleeding across entries. `mock_fixtures` applies to all `test_types` including `human` — the component is still invoked for human entries to capture `actual_output`. `mock_fixtures` MUST NOT include keys for LLM adapters: all golden dataset test types are rated `mocks disallowed for LLM calls` (rule `03`), so the LLM call MUST
|
|
44
|
+
- `expected_output` — the fields used to score the entry under each of its automated `test_types`: output attributes for an LLM-as-judge rubric, a target for vector-similarity scoring, or exact attribute values for strict comparison. When `human` is one of the entry's `test_types`, `expected_output` MUST additionally include a `human_test` string field with manual-verification instructions (e.g. `"check for ethical issues, verify record change in system X"`) — this supplements, and MUST NOT replace, the entry's automated scoring fields.
|
|
45
|
+
- `mock_fixtures` — optional object; keys identify the adapter or external system to mock (SHOULD match the connector folder name under `adapters/connectors/<name>` for readability, though not enforced), values are any valid JSON interpreted by the mock implementation. When present, eval.py MUST configure each named mock adapter with its fixture data BEFORE invoking the component for that entry; each entry MUST use fresh mock instances to prevent state from bleeding across entries. `mock_fixtures` applies to all `test_types` including `human` — the component is still invoked for human entries to capture `actual_output`. `mock_fixtures` MUST NOT include keys for LLM adapters: all golden dataset test types are rated `mocks disallowed for LLM calls` (rule `03`), so the LLM call MUST be real; LLM provider mocking belongs exclusively to unit tests via [agentme-edr-018](018-ai-llm-development-standards.md) rule `04`. See [agentme-edr-026](026-pragmatic-hexagonal-architecture.md) rule `10` for the `_mock` file naming and placement convention.
|
|
46
46
|
- The dataset's `dataset.schema.json` MUST require `test_types`, `input`, and `expected_output`, and SHOULD declare `mock_fixtures` as optional (`"type": "object", "additionalProperties": {}`), per [agentme-edr-024](024-ml-dataset-structure.md) rule `04`.
|
|
47
47
|
|
|
48
48
|
#### 03-mocks-allowed-values
|
|
@@ -53,60 +53,60 @@ Module folder responsibilities, artifact locations, and test-folder conventions
|
|
|
53
53
|
#### 02-application-folders
|
|
54
54
|
|
|
55
55
|
- Represent a cohesive unit with its own lifecycle (e.g., `mymobileapp`, `graph-visualizer`).
|
|
56
|
-
-
|
|
57
|
-
-
|
|
58
|
-
-
|
|
56
|
+
- MUST depend only on resources in `/shared/`. Direct cross-application dependencies are forbidden; use published artifacts (container images, published libraries) instead.
|
|
57
|
+
- MUST contain a `README.md` with: purpose, architecture overview, how to build, and how to run.
|
|
58
|
+
- MAY contain `examples/`, `tests_integration/`, and `tests_benchmark/` when those artifacts apply to multiple modules inside the application.
|
|
59
59
|
|
|
60
60
|
#### 03-module-folders
|
|
61
61
|
|
|
62
62
|
- A module is a subfolder inside an application that is independently compilable and produces a build artifact.
|
|
63
63
|
- May depend on sibling modules within the same application or on `/shared/` resources.
|
|
64
|
-
-
|
|
65
|
-
-
|
|
66
|
-
-
|
|
67
|
-
-
|
|
64
|
+
- MUST NOT depend on modules from other applications.
|
|
65
|
+
- MUST contain its own `Makefile`, `README.md`, and language/tooling configuration.
|
|
66
|
+
- MUST keep build outputs under `dist/` and persistent caches under `.cache/`, following [agentme-edr-016](../principles/016-cross-language-module-structure.md).
|
|
67
|
+
- MUST NOT keep consumer examples inside the module folder; those belong in a sibling `examples/` folder at the nearest parent aggregation root.
|
|
68
68
|
|
|
69
69
|
#### 04-naming-conventions
|
|
70
70
|
|
|
71
|
-
- All folder and file names
|
|
71
|
+
- All folder and file names MUST be lowercase.
|
|
72
72
|
- Use hyphens (`-`) to separate words (e.g., `data-loader`, `graph-visualizer`).
|
|
73
73
|
- Avoid abbreviations unless universally understood in the domain (e.g., `cli`, `api`).
|
|
74
74
|
|
|
75
75
|
#### 05-makefiles-at-every-level
|
|
76
76
|
|
|
77
|
-
A `Makefile`
|
|
77
|
+
A `Makefile` MUST be present at the repository root, in every application folder, and in every module folder.
|
|
78
78
|
|
|
79
|
-
All Makefiles
|
|
79
|
+
All Makefiles MUST use the shared target vocabulary from [agentme-edr-008](008-common-targets.md).
|
|
80
80
|
|
|
81
|
-
Repository, application, and module Makefiles
|
|
81
|
+
Repository, application, and module Makefiles MUST define at minimum: `all`, `build`, `lint`, `test`, and `clean`.
|
|
82
82
|
|
|
83
|
-
Module Makefiles
|
|
83
|
+
Module Makefiles SHOULD also provide `lint-fix` and `install` when the underlying tooling supports them.
|
|
84
84
|
|
|
85
|
-
The root `Makefile`
|
|
86
|
-
The root `setup` target
|
|
85
|
+
The root `Makefile` MUST also define a `setup` target that guides a new contributor to prepare their machine.
|
|
86
|
+
The root `setup` target MUST run `mise install` and any small repository bootstrap required before routine targets work.
|
|
87
87
|
|
|
88
88
|
#### 06-mise-for-tooling-management
|
|
89
89
|
|
|
90
|
-
- [Mise](https://mise.jdx.dev/)
|
|
91
|
-
- A `.mise.toml`
|
|
92
|
-
- Every language runtime or CLI referenced by any module `Makefile`, CI workflow, or README command
|
|
93
|
-
- Contributors and CI run `make setup` after cloning or checkout; this target
|
|
94
|
-
- Agents and contributors
|
|
95
|
-
- When `.mise.toml` exists, all build, test, lint, and code-generation commands
|
|
96
|
-
- If a required tool is missing, the first remediation step
|
|
97
|
-
- Root and module `Makefile` targets
|
|
90
|
+
- [Mise](https://mise.jdx.dev/) MUST be used to pin all tool versions (compilers, runtimes, CLI tools).
|
|
91
|
+
- A `.mise.toml` MUST exist at the repository root.
|
|
92
|
+
- Every language runtime or CLI referenced by any module `Makefile`, CI workflow, or README command MUST be pinned in `.mise.toml`.
|
|
93
|
+
- Contributors and CI run `make setup` after cloning or checkout; this target MUST call `mise install`.
|
|
94
|
+
- Agents and contributors MUST check `.mise.toml` before using a system-installed compiler, runtime, or CLI.
|
|
95
|
+
- When `.mise.toml` exists, all build, test, lint, and code-generation commands MUST run through `make <target>`, and the Makefile recipes MUST execute the underlying tools via `mise exec -- <command>`, following [agentme-edr-017](017-tool-execution-and-scripting.md).
|
|
96
|
+
- If a required tool is missing, the first remediation step MUST be to update `.mise.toml` or run `mise install`, not to install ad-hoc global tools with language-specific installers such as `go install`, `npm install -g`, `pip install --user`, or `cargo install`.
|
|
97
|
+
- Root and module `Makefile` targets MUST work when invoked as plain `make <target>` after `make setup`.
|
|
98
98
|
|
|
99
99
|
#### 07-root-readme
|
|
100
100
|
|
|
101
|
-
The root `README.md`
|
|
101
|
+
The root `README.md` MUST include: overview, machine setup, quickstart, and a repository map.
|
|
102
102
|
|
|
103
103
|
#### 08-root-gitignore
|
|
104
104
|
|
|
105
|
-
The repository root
|
|
105
|
+
The repository root MUST ignore `dist/` and `.cache/` so module artifacts and tool caches MUST NOT be committed accidentally.
|
|
106
106
|
|
|
107
107
|
#### 09-git-tagging-and-artifact-versioning
|
|
108
108
|
|
|
109
|
-
All releases
|
|
109
|
+
All releases MUST be tagged using the format `<module-name>/<semver>` (e.g., `graphvisualizer/renderer/1.0.0`, `shared/libs/mylib/2.1.0`).
|
|
110
110
|
|
|
111
111
|
`<module-name>` is preferably the path-like identifier of the module being released. A custom name is allowed but the folder name is strongly preferred.
|
|
112
112
|
|
|
@@ -29,7 +29,7 @@ Separating these concerns eliminates accidental publishes from CI runs, ensures
|
|
|
29
29
|
| `release.yml` | `workflow_dispatch` | Tag the next version using monotag |
|
|
30
30
|
| `publish.yml` | `push` of tags matching `*` | Publish artifacts for the tagged version |
|
|
31
31
|
|
|
32
|
-
All workflows run on `ubuntu-latest`. Tool versions MUST be managed by Mise via `jdx/mise-action`. Projects
|
|
32
|
+
All workflows run on `ubuntu-latest`. Tool versions MUST be managed by Mise via `jdx/mise-action`. Projects SHOULD have a `.mise.toml` file to configure it
|
|
33
33
|
|
|
34
34
|
---
|
|
35
35
|
|
|
@@ -69,7 +69,7 @@ File: `.github/workflows/release.yml`
|
|
|
69
69
|
|
|
70
70
|
Projects MUST use this manually dispatched (`workflow_dispatch`) workflow. It calculates the next semantic version tag using **monotag** and pushes that tag to the repository. Pushing the tag then automatically triggers the publish workflow.
|
|
71
71
|
|
|
72
|
-
The checkout step
|
|
72
|
+
The checkout step MUST use `fetch-depth: 0` so monotag can traverse the full commit history to determine the correct next version.
|
|
73
73
|
|
|
74
74
|
```yaml
|
|
75
75
|
name: release
|
|
@@ -23,18 +23,18 @@ Standardizing both the target names and the execution chain removes per-project
|
|
|
23
23
|
|
|
24
24
|
#### 01-every-project-must-have-root-makefile
|
|
25
25
|
|
|
26
|
-
The project root
|
|
26
|
+
The project root MUST contain a single authoritative `Makefile` that exposes the standard target names defined in rule 3. Developers and CI pipelines MUST invoke routine actions through this `Makefile`, MUST NOT call underlying tools directly in documentation, CI, or daily workflow commands.
|
|
27
27
|
|
|
28
28
|
`make <target>` is the shared contract across projects and languages.
|
|
29
29
|
|
|
30
|
-
- The root `Makefile`
|
|
31
|
-
- The root `Makefile`
|
|
32
|
-
- Reverse-compatibility wrappers are allowed when an ecosystem expects them, but they
|
|
30
|
+
- The root `Makefile` MUST be the entry point for both developers and pipelines.
|
|
31
|
+
- The root `Makefile` MUST expose at minimum the common targets defined in this XDR.
|
|
32
|
+
- Reverse-compatibility wrappers are allowed when an ecosystem expects them, but they MUST stay trivial.
|
|
33
33
|
- Allowed: `package.json` script `"test": "make test"`
|
|
34
34
|
- Not allowed: `make test` -> `npm run test` -> tool command
|
|
35
|
-
- Project logic
|
|
35
|
+
- Project logic MUST NOT live in npm scripts, Mise tasks, shell wrappers, or other secondary runners when the same logic belongs in the `Makefile`.
|
|
36
36
|
|
|
37
|
-
*Why:* The project entry point
|
|
37
|
+
*Why:* The project entry point MUST stay language-agnostic and obvious. A developer SHOULD be able to inspect the `Makefile` and immediately see which real tool commands will run.
|
|
38
38
|
|
|
39
39
|
#### 02-makefile-recipes-must-use-mise
|
|
40
40
|
|
|
@@ -47,10 +47,10 @@ make <target>
|
|
|
47
47
|
-> explicit tool command
|
|
48
48
|
```
|
|
49
49
|
|
|
50
|
-
- The `setup` target
|
|
51
|
-
- Routine targets such as `build`, `lint`, `test`, `run`, and `publish`
|
|
52
|
-
- Each Makefile recipe
|
|
53
|
-
- Makefile recipes
|
|
50
|
+
- The `setup` target MUST run `mise install` and any small project-specific bootstrap needed before normal targets work.
|
|
51
|
+
- Routine targets such as `build`, `lint`, `test`, `run`, and `publish` MUST be invoked as `make <target>` by both contributors and CI.
|
|
52
|
+
- Each Makefile recipe MUST call the real underlying command through `mise exec --`, following [agentme-edr-017](017-tool-execution-and-scripting.md).
|
|
53
|
+
- Makefile recipes MUST NOT add extra script layers such as `npm run`, `pnpm run`, `yarn run`, `mise run`, `mise tasks`, or shell aliases when those layers only forward to another command.
|
|
54
54
|
- Calling the actual tool is allowed even when that tool itself launches another program as part of its normal interface.
|
|
55
55
|
- Allowed: `mise exec -- pnpm exec eslint ./src`
|
|
56
56
|
- Allowed: `mise exec -- go test -cover ./...`
|
|
@@ -66,7 +66,7 @@ make <target>
|
|
|
66
66
|
|
|
67
67
|
#### 03-standard-target-groups-and-names
|
|
68
68
|
|
|
69
|
-
Targets are organized into five lifecycle groups. Projects
|
|
69
|
+
Targets are organized into five lifecycle groups. Projects MUST use these names unchanged. Extensions are allowed (see rule 5) but the core names MUST NOT be repurposed.
|
|
70
70
|
|
|
71
71
|
##### Developer group
|
|
72
72
|
|
|
@@ -150,28 +150,6 @@ The prefix convention ensures developers can infer the purpose of any target wit
|
|
|
150
150
|
|
|
151
151
|
---
|
|
152
152
|
|
|
153
|
-
#### 09-ai-project-dev-targets
|
|
154
|
-
|
|
155
|
-
AI-based projects (LLM, Agent, and Workflow tiers as defined in [agentme-edr-018](../application/018-ai-llm-development-standards.md)) MUST expose a `dev-mlflow` target that starts a local MLflow tracking server for development inspection.
|
|
156
|
-
|
|
157
|
-
**Example implementation:**
|
|
158
|
-
|
|
159
|
-
```makefile
|
|
160
|
-
dev-mlflow:
|
|
161
|
-
mise exec -- mlflow ui --host 0.0.0.0 --port 5000
|
|
162
|
-
open http://localhost:5000/
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
---
|
|
166
|
-
|
|
167
|
-
#### 08-default-targets-must-only-include-offline-subtargets
|
|
168
|
-
|
|
169
|
-
`make all`, `make test`, and `make lint` **MUST** include every subtarget that runs **offline** — meaning it requires no external credentials, no running servers, no paid APIs, and no environment-specific configuration outside the repository.
|
|
170
|
-
|
|
171
|
-
Subtargets that require external dependencies (e.g., `test-integration` against a live database, `test-e2e` against a staging environment, `lint-api` against a remote schema registry) **MUST** exist as named targets so developers can invoke them explicitly, but **MUST NOT** be invoked from `all`, `test`, or `lint`.
|
|
172
|
-
|
|
173
|
-
---
|
|
174
|
-
|
|
175
153
|
#### 06-monorepo-usage
|
|
176
154
|
|
|
177
155
|
In a monorepo, each module has its own `Makefile` with its own `build`, `lint`, `test`, and `deploy` targets scoped to that module. Parent-level Makefiles (at the application or repo root) delegate to child Makefiles in sequence. The parent Makefile **SHOULD** call `$(MAKE) -C <child> <target>` directly, while each child `Makefile` runs its actual tool commands through `mise exec --`.
|
|
@@ -238,6 +216,28 @@ make clean
|
|
|
238
216
|
make all
|
|
239
217
|
```
|
|
240
218
|
|
|
219
|
+
---
|
|
220
|
+
|
|
221
|
+
#### 08-default-targets-must-only-include-offline-subtargets
|
|
222
|
+
|
|
223
|
+
`make all`, `make test`, and `make lint` **MUST** include every subtarget that runs **offline** — meaning it requires no external credentials, no running servers, no paid APIs, and no environment-specific configuration outside the repository.
|
|
224
|
+
|
|
225
|
+
Subtargets that require external dependencies (e.g., `test-integration` against a live database, `test-e2e` against a staging environment, `lint-api` against a remote schema registry) **MUST** exist as named targets so developers can invoke them explicitly, but **MUST NOT** be invoked from `all`, `test`, or `lint`.
|
|
226
|
+
|
|
227
|
+
---
|
|
228
|
+
|
|
229
|
+
#### 09-ai-project-dev-targets
|
|
230
|
+
|
|
231
|
+
AI-based projects (LLM, Agent, and Workflow tiers as defined in [agentme-edr-018](../application/018-ai-llm-development-standards.md)) MUST expose a `dev-mlflow` target that starts a local MLflow tracking server for development inspection.
|
|
232
|
+
|
|
233
|
+
**Example implementation:**
|
|
234
|
+
|
|
235
|
+
```makefile
|
|
236
|
+
dev-mlflow:
|
|
237
|
+
mise exec -- mlflow ui --host 0.0.0.0 --port 5000
|
|
238
|
+
open http://localhost:5000/
|
|
239
|
+
```
|
|
240
|
+
|
|
241
241
|
## Considered Options
|
|
242
242
|
|
|
243
243
|
* (REJECTED) **Language-native entry points only** - Use `npm run`, `python -m`, `go run`, and similar tool-specific commands directly as the standard surface
|
|
@@ -24,7 +24,7 @@ This keeps local development and CI aligned, reduces indirection, and lets contr
|
|
|
24
24
|
- Every project MUST use a root `Makefile` as the authoritative entry point for developer and pipeline commands.
|
|
25
25
|
- The target names in that `Makefile` MUST follow [agentme-edr-008](008-common-targets.md).
|
|
26
26
|
- CI pipelines MUST run `make <target>` from the relevant root instead of calling language-specific scripts such as `npm run`, `pnpm run`, shell wrappers, or secondary task runners.
|
|
27
|
-
- A Makefile target MUST execute the real operation through `mise exec --` before invoking the tool itself, so it
|
|
27
|
+
- A Makefile target MUST execute the real operation through `mise exec --` before invoking the tool itself, so it MUST use the version pinned in `.mise.toml`. Avoid intermediate script layers that hide the actual command.
|
|
28
28
|
- Every Makefile target MUST start by echoing a concise summary of the target and folder or context, using fewer than 10 words. When delegating to another Makefile, echo the child path and delegated target before invoking it.
|
|
29
29
|
- Direct delegation to another Makefile is allowed when traversing repo, app, or module boundaries, for example `$(MAKE) -C lib build`.
|
|
30
30
|
- Calling the actual tool binary through its native executable launcher is allowed when that is the direct command under `mise exec --`, for example `mise exec -- pnpm exec eslint ./src`, `mise exec -- uv run ty check`, `mise exec -- go test`, or `mise exec -- npx -y monotag`.
|
|
@@ -17,9 +17,9 @@ How should projects manage environment variable configuration and CLI invocation
|
|
|
17
17
|
|
|
18
18
|
## Decision Outcome
|
|
19
19
|
|
|
20
|
-
**Use YAML config files for CLI invocation configuration with multiple attributes; use `.env` files to supply environment variables to spawned processes and to hold uncommitted values referenced by config files. Load `.env` exclusively at process launch time —
|
|
20
|
+
**Use YAML config files for CLI invocation configuration with multiple attributes; use `.env` files to supply environment variables to spawned processes and to hold uncommitted values referenced by config files. Load `.env` exclusively at process launch time — MUST NOT be loaded inside application code.**
|
|
21
21
|
|
|
22
|
-
Secrets (API keys, passwords, tokens)
|
|
22
|
+
Secrets (API keys, passwords, tokens) MUST NOT be placed in `.env` files. Those are handled by [agentme-edr-022](../principles/022-secrets-management.md).
|
|
23
23
|
|
|
24
24
|
### Details
|
|
25
25
|
|
|
@@ -43,7 +43,7 @@ FEATURE_FLAG_NEW_UI=false
|
|
|
43
43
|
|
|
44
44
|
#### 02-dotenv-not-committed
|
|
45
45
|
|
|
46
|
-
`.env` MUST be listed in `.gitignore` and
|
|
46
|
+
`.env` MUST be listed in `.gitignore` and MUST NOT be committed to the repository. It is intended for local use in standalone projects and libraries that do not have a formal deployment pipeline.
|
|
47
47
|
|
|
48
48
|
---
|
|
49
49
|
|
|
@@ -21,15 +21,41 @@ Projects MUST keep a `CONTRIBUTING.md` file at the repository root. The file MUS
|
|
|
21
21
|
|
|
22
22
|
### Details
|
|
23
23
|
|
|
24
|
-
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
-
|
|
24
|
+
#### 01-contributing-md-is-required
|
|
25
|
+
|
|
26
|
+
Every project MUST have a root `CONTRIBUTING.md` file at the repository root. The file MUST explain where bugs, feature discussions, and code changes belong so contributors follow a predictable workflow before opening pull requests.
|
|
27
|
+
|
|
28
|
+
#### 02-guide-must-direct-bug-reports-to-issues
|
|
29
|
+
|
|
30
|
+
The guide MUST direct bug reports to issues.
|
|
31
|
+
|
|
32
|
+
#### 03-guide-must-route-feature-discussions-to-issues
|
|
33
|
+
|
|
34
|
+
The guide MUST direct feature ideas and feature discussions to issues before implementation starts.
|
|
35
|
+
|
|
36
|
+
#### 04-guide-must-require-pull-requests
|
|
37
|
+
|
|
38
|
+
The guide MUST state that fixes and features are contributed through pull requests.
|
|
39
|
+
|
|
40
|
+
#### 05-guide-must-require-feature-branches
|
|
41
|
+
|
|
42
|
+
The guide MUST state that pull requests come from feature branches targeting `main`.
|
|
43
|
+
|
|
44
|
+
#### 06-guide-must-require-conventional-comments
|
|
45
|
+
|
|
46
|
+
The guide MUST ask reviewers and contributors to use [Conventional Comments](https://conventionalcomments.org/) for review feedback.
|
|
47
|
+
|
|
48
|
+
#### 07-guide-must-ask-for-small-pull-requests
|
|
49
|
+
|
|
50
|
+
The guide MUST ask contributors to keep pull requests small enough to keep review and discussion focused.
|
|
51
|
+
|
|
52
|
+
#### 08-scaffolding-should-create-contributing-md
|
|
53
|
+
|
|
54
|
+
Project scaffolding skills SHOULD create the file by default when they initialize a repository.
|
|
55
|
+
|
|
56
|
+
#### 09-content-should-be-concise
|
|
57
|
+
|
|
58
|
+
The content SHOULD stay concise and practical; do not turn `CONTRIBUTING.md` into a duplicate of `README.md`.
|
|
33
59
|
|
|
34
60
|
## Considered Options
|
|
35
61
|
|
|
@@ -45,7 +45,7 @@ All services **MUST** expose a `GET /health` endpoint that validates external de
|
|
|
45
45
|
|
|
46
46
|
- `health` (required): overall state — `OK`, `WARNING`, or `ERROR`
|
|
47
47
|
- `latencyMs` (required): total milliseconds to run all checks
|
|
48
|
-
- `message` (required): human-readable summary;
|
|
48
|
+
- `message` (required): human-readable summary; MUST NOT expose credentials, internal IPs, or stack traces
|
|
49
49
|
|
|
50
50
|
**Dependency validation rules:**
|
|
51
51
|
|
|
@@ -15,13 +15,15 @@ What unit testing practices should be followed to ensure tests are meaningful, r
|
|
|
15
15
|
|
|
16
16
|
## Decision Outcome
|
|
17
17
|
|
|
18
|
-
**Every test
|
|
18
|
+
**Every test MUST assert behavior, run offline without external dependencies, enforce 80% coverage, centralize shared setup, and prefer real code over mocks.**
|
|
19
19
|
|
|
20
20
|
### Details
|
|
21
21
|
|
|
22
22
|
#### 01-must-have-at-least-one-assertion-per-test
|
|
23
23
|
|
|
24
24
|
Every test MUST have at least one assertion that validates the expected behavior.
|
|
25
|
+
|
|
26
|
+
```typescript
|
|
25
27
|
// bad — no assertion; passes even when code is broken
|
|
26
28
|
it("processes the order", () => { processOrder(mockOrder); });
|
|
27
29
|
|
|
@@ -36,7 +38,7 @@ it("processes the order and returns a confirmation id", () => {
|
|
|
36
38
|
|
|
37
39
|
#### 02-must-run-offline
|
|
38
40
|
|
|
39
|
-
Unit tests MUST NOT depend on any external resources: no network calls, no running databases, no external APIs, no file system paths outside the repo. Tests
|
|
41
|
+
Unit tests MUST NOT depend on any external resources: no network calls, no running databases, no external APIs, no file system paths outside the repo. Tests MUST pass with only static code available.
|
|
40
42
|
|
|
41
43
|
```typescript
|
|
42
44
|
// bad — hits a real HTTP endpoint
|
|
@@ -83,7 +85,7 @@ src/mymodule/group1/file1.ts ← source
|
|
|
83
85
|
src/mymodule/group1/file1.test.ts ← test (same directory)
|
|
84
86
|
```
|
|
85
87
|
|
|
86
|
-
**Exception — separate test folder:** When the framework makes co-location impractical (e.g. Python's common `tests/` convention), or when the community strongly favors a separate folder, a dedicated test root (e.g. `tests/`) is allowed. In that case the test folder
|
|
88
|
+
**Exception — separate test folder:** When the framework makes co-location impractical (e.g. Python's common `tests/` convention), or when the community strongly favors a separate folder, a dedicated test root (e.g. `tests/`) is allowed. In that case the test folder MUST mirror the source folder structure exactly:
|
|
87
89
|
|
|
88
90
|
```
|
|
89
91
|
src/mymodule/group1/file1.py ← source
|
|
@@ -111,7 +113,7 @@ export function makeOrder(overrides: Partial<Order> = {}): Order {
|
|
|
111
113
|
|
|
112
114
|
Tests SHOULD use the lowest-cost alternative that exercises real behavior:
|
|
113
115
|
|
|
114
|
-
1. **Real implementation** —
|
|
116
|
+
1. **Real implementation** — MUST be preferred
|
|
115
117
|
2. **In-memory / lightweight fake** — e.g. in-memory DB, stub HTTP server
|
|
116
118
|
3. **Recorded fixture** — replay captured real responses
|
|
117
119
|
4. **Mock / stub** — only for external APIs, irreversible operations, or hardware I/O
|
|
@@ -15,9 +15,9 @@ What minimum quality standards must every project in the organization meet to en
|
|
|
15
15
|
|
|
16
16
|
## Decision Outcome
|
|
17
17
|
|
|
18
|
-
Every project
|
|
18
|
+
Every project MUST meet the minimum quality standards: a Getting Started section in its README, unit tests that run on every release, compliance with workspace XDRs, active linting enforcement, a structure that is clear to new developers, and — for libraries and utilities — a runnable examples folder verified on every test run. Integration tests are advised but not required. Projects with statistical models MUST have evaluation targets with performance thresholds.
|
|
19
19
|
|
|
20
|
-
These standards form a non-negotiable baseline. Individual projects may raise the bar but
|
|
20
|
+
These standards form a non-negotiable baseline. Individual projects may raise the bar but MUST NOT fall below it.
|
|
21
21
|
|
|
22
22
|
### Details
|
|
23
23
|
|
|
@@ -52,14 +52,14 @@ myFunction({ input: "value" });
|
|
|
52
52
|
|
|
53
53
|
#### 02-unit-tests-must-run-on-every-release
|
|
54
54
|
|
|
55
|
-
A unit test suite MUST run automatically before every release. Failing tests
|
|
55
|
+
A unit test suite MUST run automatically before every release. Failing tests MUST block the release — no silent skips or overrides.
|
|
56
56
|
|
|
57
57
|
**Requirements:**
|
|
58
|
-
- A `make test` target
|
|
59
|
-
- CI/CD
|
|
58
|
+
- A `make test` target MUST exist and run the full suite
|
|
59
|
+
- CI/CD MUST invoke it before publish/deploy
|
|
60
60
|
- Test failures block the release
|
|
61
61
|
|
|
62
|
-
**Exception:** Projects with fewer than 100 lines of code, or whose `README.md` prominently marks them as a **Spike** or **Experiment**, are exempt from this requirement. Such projects
|
|
62
|
+
**Exception:** Projects with fewer than 100 lines of code, or whose `README.md` prominently marks them as a **Spike** or **Experiment**, are exempt from this requirement. Such projects MUST NOT be deployed to production.
|
|
63
63
|
|
|
64
64
|
**Reference:** [agentme-edr-004](004-unit-test-requirements.md) for detailed unit test requirements.
|
|
65
65
|
|
|
@@ -85,7 +85,7 @@ Projects larger than 10 files or 200 lines of code MUST have a linter configured
|
|
|
85
85
|
- Linter config is checked in (e.g., `.eslintrc.js`, `pyproject.toml`, `.golangci.yml`)
|
|
86
86
|
- CI runs `make lint` before merging or releasing
|
|
87
87
|
|
|
88
|
-
**Exception:** Projects with fewer than 100 lines of code, or whose `README.md` prominently marks them as a **Spike** or **Experiment**, are exempt from this requirement. Such projects
|
|
88
|
+
**Exception:** Projects with fewer than 100 lines of code, or whose `README.md` prominently marks them as a **Spike** or **Experiment**, are exempt from this requirement. Such projects MUST NOT be deployed to production.
|
|
89
89
|
|
|
90
90
|
**Reference:** [agentme-edr-003](../application/003-javascript-project-tooling.md) for JavaScript-specific tooling.
|
|
91
91
|
|
|
@@ -93,11 +93,11 @@ Projects larger than 10 files or 200 lines of code MUST have a linter configured
|
|
|
93
93
|
|
|
94
94
|
#### 05-project-structure-must-be-clear
|
|
95
95
|
|
|
96
|
-
Directory and file layout MUST be self-explanatory: source code, tests, configuration, and examples
|
|
96
|
+
Directory and file layout MUST be self-explanatory: source code, tests, configuration, and examples MUST be clearly separated and named.
|
|
97
97
|
|
|
98
98
|
**Requirements:**
|
|
99
|
-
- Directory names
|
|
100
|
-
- README
|
|
99
|
+
- Directory names MUST reflect their purpose (`src/`, `lib/`, `tests/`, `examples/`, `docs/`)
|
|
100
|
+
- README MUST describe the top-level layout if non-obvious
|
|
101
101
|
- No orphaned or unexplained directories or files at the project root
|
|
102
102
|
|
|
103
103
|
**Example layout (TypeScript project):**
|
|
@@ -118,14 +118,14 @@ Directory and file layout MUST be self-explanatory: source code, tests, configur
|
|
|
118
118
|
|
|
119
119
|
#### 06-libraries-must-have-runnable-examples
|
|
120
120
|
|
|
121
|
-
Projects that are libraries or shared utilities MUST include an `examples/` directory. Each subdirectory represents a usage scenario and
|
|
121
|
+
Projects that are libraries or shared utilities MUST include an `examples/` directory. Each subdirectory represents a usage scenario and MUST be independently runnable. Examples that are "offline" (require no external credentials, no running servers, no paid APIs, and no environment-specific configuration outside the repository) MUST be executed as part of `make test`. Examples that depend on external entities may be left out of `make test`.
|
|
122
122
|
|
|
123
123
|
**Requirements:**
|
|
124
|
-
- `examples/`
|
|
125
|
-
- Each scenario subdirectory
|
|
126
|
-
- Examples
|
|
127
|
-
- `make test` in the root
|
|
128
|
-
- Examples that depend on external entities
|
|
124
|
+
- `examples/` MUST contain at least one subdirectory per major usage scenario
|
|
125
|
+
- Each scenario subdirectory MUST have a `Makefile` with a `run` target
|
|
126
|
+
- Examples MUST import the library as an external consumer (not via relative `../src` imports)
|
|
127
|
+
- `make test` in the root MUST run all offline examples; failures block CI and releases
|
|
128
|
+
- Examples that depend on external entities MUST NOT be included in `make test`
|
|
129
129
|
|
|
130
130
|
**Directory layout:**
|
|
131
131
|
|
|
@@ -173,13 +173,13 @@ all:
|
|
|
173
173
|
Projects that contain statistical models (e.g., ML models, LLM-based evaluators, classifiers, ranking systems, or any component whose output quality is measured probabilistically) MUST define measurable performance thresholds and verify them automatically.
|
|
174
174
|
|
|
175
175
|
**Requirements:**
|
|
176
|
-
- A `make eval` target
|
|
177
|
-
- Each evaluation
|
|
178
|
-
- Thresholds
|
|
179
|
-
- `make eval`
|
|
176
|
+
- A `make eval` target MUST exist and execute all performance evaluations
|
|
177
|
+
- Each evaluation MUST have a **documented minimum performance threshold** (e.g., accuracy ≥ 0.85, F1 ≥ 0.80, BLEU ≥ 0.70)
|
|
178
|
+
- Thresholds MUST be declared explicitly in the project (e.g., in a config file, `Makefile` variable, or documented in `README.md`)
|
|
179
|
+
- `make eval` MUST **exit with a non-zero status** (fail) if:
|
|
180
180
|
- The evaluation cannot be executed (missing data, environment errors, model load failures)
|
|
181
181
|
- Any metric falls below its defined minimum threshold
|
|
182
|
-
- CI/CD
|
|
182
|
+
- CI/CD MUST invoke `make eval` before releasing any version that changes model weights, prompts, or evaluation logic
|
|
183
183
|
|
|
184
184
|
**Threshold declaration example (Makefile):**
|
|
185
185
|
|
|
@@ -211,7 +211,7 @@ Every system boundary MUST signal failure explicitly:
|
|
|
211
211
|
|
|
212
212
|
- **OS processes** MUST exit with a **non-zero exit code** when something went wrong. Exit code `0` means success.
|
|
213
213
|
- **HTTP services** MUST return a **non-2xx/3xx status code** on error, accompanied by a response body that describes the problem without exposing internal system details (stack traces, SQL queries, internal paths, etc.).
|
|
214
|
-
- **All error responses** SHOULD be logged to the console/structured logger, especially system-level or unexpected errors. Operational teams
|
|
214
|
+
- **All error responses** SHOULD be logged to the console/structured logger, especially system-level or unexpected errors. Operational teams MUST be able to find the cause from logs alone.
|
|
215
215
|
|
|
216
216
|
**Examples:**
|
|
217
217
|
|
|
@@ -21,15 +21,33 @@ Developers MUST treat reusable missing guidance discovered during implementation
|
|
|
21
21
|
|
|
22
22
|
### Details
|
|
23
23
|
|
|
24
|
-
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
-
|
|
24
|
+
#### 01-reusable-guidance-must-become-shared-xdr
|
|
25
|
+
|
|
26
|
+
Developers MUST treat reusable missing guidance discovered during implementation as an XDR gap to be proposed and reviewed, not as permanent prompt-only context or repeated vibe coding. The main objective is sharing, discussing, and converging practices across teams. Controlled divergence during exploration is acceptable, but recurring successful decisions MUST be converged into shared XDRs.
|
|
27
|
+
|
|
28
|
+
#### 02-local-scope-must-be-for-truly-specific-decisions
|
|
29
|
+
|
|
30
|
+
Decisions placed in `_local` SHOULD be truly specific to the needs of a single application or repository. The non-`_local` scope exists to share practices across projects, company areas, and functionally organized teams.
|
|
31
|
+
|
|
32
|
+
#### 03-steering-needs-must-trigger-xdr-reflection
|
|
33
|
+
|
|
34
|
+
When developers or coding agents need too much detailed steering to complete a task, they MUST reflect on whether those details would help other teams or future implementations. If yes, create or update an XDR proposal in the broadest appropriate shared scope. This includes cases where an agent implemented a feature without a framework, pattern, coding standard, or other practice that should likely be standardized. Missing reusable guardrails SHOULD trigger an XDR proposal.
|
|
35
|
+
|
|
36
|
+
#### 04-coverage-target-should-be-80-percent
|
|
37
|
+
|
|
38
|
+
Teams SHOULD aim to keep at least 80% of big coding decisions covered by accepted XDRs. Big decisions include framework or tool selection, overall code organization, monorepo structure, complex business flows, and coding standards. If a big decision is not yet covered, developers SHOULD either propose a new XDR or document why the decision is intentionally local and should not be shared.
|
|
39
|
+
|
|
40
|
+
#### 05-leaders-must-review-xdr-proposals
|
|
41
|
+
|
|
42
|
+
Leaders responsible for the affected scope MUST review XDR proposals, adjust them as needed, and publish the accepted decision.
|
|
43
|
+
|
|
44
|
+
#### 06-query-agents-for-missing-xdrs
|
|
45
|
+
|
|
46
|
+
It is good practice to ask the coding agent which missing XDRs made the task harder, increased adjustment rounds, or forced more vibe coding. Those gaps SHOULD feed the XDR backlog.
|
|
47
|
+
|
|
48
|
+
#### 07-xdrs-are-not-feature-specs
|
|
49
|
+
|
|
50
|
+
In SDD, specifications describe the feature being built; XDRs describe reusable decisions and guardrails that MUST survive beyond one feature. Do not keep durable engineering policy only inside feature specs.
|
|
33
51
|
|
|
34
52
|
## Considered Options
|
|
35
53
|
|
|
@@ -88,7 +88,7 @@ Examples that demonstrate how to consume a library or reusable module MUST live
|
|
|
88
88
|
Examples MUST exercise the module through its public distribution surface:
|
|
89
89
|
|
|
90
90
|
- use the package built into `dist/` when the ecosystem supports local packaged artifacts
|
|
91
|
-
- otherwise use the public module path or equivalent consumer-facing import surface;
|
|
91
|
+
- otherwise use the public module path or equivalent consumer-facing import surface; MUST NOT use relative source-file imports or direct references to internal implementation paths
|
|
92
92
|
|
|
93
93
|
Example:
|
|
94
94
|
|
|
@@ -23,13 +23,13 @@ All implementation practices derive from three guiding principles:
|
|
|
23
23
|
|
|
24
24
|
1. **Least exposure** — minimize the means, timespan, and surface of contact with the secret.
|
|
25
25
|
2. **Easiness in secret rotation** — design so rotating a secret requires no code change or redeployment.
|
|
26
|
-
3. **Support for local and cloud deployment runs** — the same application code
|
|
26
|
+
3. **Support for local and cloud deployment runs** — the same application code MUST work transparently in both environments.
|
|
27
27
|
|
|
28
28
|
### Details
|
|
29
29
|
|
|
30
30
|
#### 01-no-secrets-on-disk
|
|
31
31
|
|
|
32
|
-
Secrets MUST
|
|
32
|
+
Secrets MUST NOT be stored on the disk of a developer machine or server. This includes `.env` files (even when gitignored), plaintext config files, embedded in source code, or any other file-based storage.
|
|
33
33
|
|
|
34
34
|
The only acceptable local persistence is through the operating system's native secret manager (e.g., macOS Keychain, Windows Credential Manager, Linux Secret Service).
|
|
35
35
|
|
|
@@ -96,31 +96,11 @@ $ make run
|
|
|
96
96
|
# Application starts successfully
|
|
97
97
|
```
|
|
98
98
|
|
|
99
|
-
#### 05a-makefile-uses-security-utility
|
|
100
|
-
|
|
101
|
-
Makefile targets (e.g., `setup-secrets`) must use the macOS native `security` CLI to store and retrieve secrets from the keychain. This restricts Makefile-based secret management to macOS developer machines, which is acceptable since all contributors are expected to use macOS.
|
|
102
|
-
|
|
103
|
-
Do **not** use `keyring` or other cross-platform libraries in Makefiles — `security` is simpler to invoke from shell and requires no additional dependencies.
|
|
104
|
-
|
|
105
|
-
Storing a secret:
|
|
106
|
-
```makefile
|
|
107
|
-
security add-generic-password -a "$(USER)" -s "mymodule/api-key" -w "$(SECRET_VALUE)" -U
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
Retrieving a secret (e.g., to pass to a command):
|
|
111
|
-
```makefile
|
|
112
|
-
SECRET_VALUE := $(shell security find-generic-password -a "$(USER)" -s "mymodule/api-key" -w 2>/dev/null)
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
The `-U` flag updates the entry if it already exists. Use the format `<group>/<secret-id>` as the service name (`-s`) to mirror the module name and cloud secret manager ID convention defined in rule 02 and 05.
|
|
116
|
-
|
|
117
|
-
In library code (Python, JS/TS, Go), continue using the cross-platform libraries defined in rule 02 (`keyring`, `cross-keychain`, `go-keyring`). The `security` utility is only for Makefile scripts.
|
|
118
|
-
|
|
119
99
|
---
|
|
120
100
|
|
|
121
101
|
#### 06-never-log-or-leak-secrets
|
|
122
102
|
|
|
123
|
-
Secrets MUST
|
|
103
|
+
Secrets MUST NOT be logged under any circumstance or sent to any service that is not clearly the intended consumer of that secret (authentication, encryption, etc.). This applies to all log levels including debug and trace. Error messages MUST reference the secret name or identifier, MUST NOT include its value.
|
|
124
104
|
|
|
125
105
|
---
|
|
126
106
|
|
|
@@ -132,7 +112,7 @@ Wherever possible, secrets SHOULD be fetched dynamically from the secret manager
|
|
|
132
112
|
- Immediate propagation of rotated secrets.
|
|
133
113
|
- Reduced window of exposure if memory is compromised.
|
|
134
114
|
|
|
135
|
-
Short-lived caching (e.g., a few minutes) is acceptable when performance requires it, but
|
|
115
|
+
Short-lived caching (e.g., a few minutes) is acceptable when performance requires it, but MUST have an explicit TTL.
|
|
136
116
|
|
|
137
117
|
---
|
|
138
118
|
|
|
@@ -160,6 +140,28 @@ def test_service_uses_api_key():
|
|
|
160
140
|
|
|
161
141
|
Integration tests MAY use the real keychain on developer machines or CI after `make setup-secrets` has been run.
|
|
162
142
|
|
|
143
|
+
---
|
|
144
|
+
|
|
145
|
+
#### 10-makefile-uses-security-utility
|
|
146
|
+
|
|
147
|
+
Makefile targets (e.g., `setup-secrets`) MUST use the macOS native `security` CLI to store and retrieve secrets from the keychain. This restricts Makefile-based secret management to macOS developer machines, which is acceptable since all contributors are expected to use macOS.
|
|
148
|
+
|
|
149
|
+
Do not use `keyring` or other cross-platform libraries in Makefiles — `security` is simpler to invoke from shell and requires no additional dependencies.
|
|
150
|
+
|
|
151
|
+
Storing a secret:
|
|
152
|
+
```makefile
|
|
153
|
+
security add-generic-password -a "$(USER)" -s "mymodule/api-key" -w "$(SECRET_VALUE)" -U
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Retrieving a secret (e.g., to pass to a command):
|
|
157
|
+
```makefile
|
|
158
|
+
SECRET_VALUE := $(shell security find-generic-password -a "$(USER)" -s "mymodule/api-key" -w 2>/dev/null)
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
The `-U` flag updates the entry if it already exists. Use the format `<group>/<secret-id>` as the service name (`-s`) to mirror the module name and cloud secret manager ID convention defined in rule 02 and 05.
|
|
162
|
+
|
|
163
|
+
In library code (Python, JS/TS, Go), continue using the cross-platform libraries defined in rule 02 (`keyring`, `cross-keychain`, `go-keyring`). The `security` utility is only for Makefile scripts.
|
|
164
|
+
|
|
163
165
|
## References
|
|
164
166
|
|
|
165
167
|
- [agentme-edr-008](../devops/008-common-targets.md) - Common development script names (defines Makefile target conventions)
|
|
@@ -41,7 +41,7 @@ These patterns obfuscate the main program flow and create behavioral indirection
|
|
|
41
41
|
|
|
42
42
|
#### 03-trivial-wrappers-are-prohibited
|
|
43
43
|
|
|
44
|
-
A function that merely delegates to another function or API call without adding meaningful logic, domain intent, or readability
|
|
44
|
+
A function that merely delegates to another function or API call without adding meaningful logic, domain intent, or readability MUST be inlined. A wrapper is justified only when it:
|
|
45
45
|
|
|
46
46
|
- Encapsulates non-trivial logic (validation, retry, transformation).
|
|
47
47
|
- Communicates a domain concept the underlying expression does not convey.
|