agentme 0.25.2 → 0.27.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.filedist-package.yml +1 -1
- package/.xdrs/agentme/edrs/application/003-javascript-project-tooling.md +4 -4
- package/.xdrs/agentme/edrs/application/010-golang-project-tooling.md +4 -4
- package/.xdrs/agentme/edrs/application/014-python-project-tooling.md +10 -10
- package/.xdrs/agentme/edrs/application/018-ai-llm-development-standards.md +4 -2
- package/.xdrs/agentme/edrs/application/019-ai-agents-development-standards.md +8 -8
- package/.xdrs/agentme/edrs/application/020-ai-agents-quality-standards.md +3 -1
- package/.xdrs/agentme/edrs/application/021-ai-workflow-development-standards.md +9 -9
- package/.xdrs/agentme/edrs/application/025-ai-agent-xdrs-knowledge-layer.md +1 -1
- package/.xdrs/agentme/edrs/application/026-pragmatic-hexagonal-architecture.md +2 -2
- package/.xdrs/agentme/edrs/application/028-ai-eval-core-standards.md +120 -0
- package/.xdrs/agentme/edrs/application/029-ai-workflow-naming-conventions.md +1 -1
- package/.xdrs/agentme/edrs/application/030-ai-test-types-taxonomy.md +43 -25
- package/.xdrs/agentme/edrs/application/031-ai-eval-script.md +136 -0
- package/.xdrs/agentme/edrs/application/032-ai-eval-report-format.md +171 -0
- package/.xdrs/agentme/edrs/application/033-ai-eval-repeatability.md +75 -0
- package/.xdrs/agentme/edrs/devops/005-monorepo-structure.md +25 -25
- package/.xdrs/agentme/edrs/devops/006-github-pipelines.md +2 -2
- package/.xdrs/agentme/edrs/devops/008-common-targets.md +33 -33
- package/.xdrs/agentme/edrs/devops/017-tool-execution-and-scripting.md +1 -1
- package/.xdrs/agentme/edrs/devops/027-environment-variable-configuration.md +3 -3
- package/.xdrs/agentme/edrs/governance/013-contributing-guide-requirements.md +35 -9
- package/.xdrs/agentme/edrs/index.md +20 -5
- package/.xdrs/agentme/edrs/observability/011-service-health-check-endpoint.md +1 -1
- package/.xdrs/agentme/edrs/principles/004-unit-test-requirements.md +6 -4
- package/.xdrs/agentme/edrs/principles/007-project-quality-standards.md +35 -30
- package/.xdrs/agentme/edrs/principles/009-error-handling.md +1 -1
- package/.xdrs/agentme/edrs/principles/012-continuous-xdr-enrichment.md +27 -9
- package/.xdrs/agentme/edrs/principles/016-cross-language-module-structure.md +1 -1
- package/.xdrs/agentme/edrs/principles/022-secrets-management.md +26 -24
- package/.xdrs/agentme/edrs/principles/023-coding-abstraction-practices.md +1 -1
- package/package.json +1 -1
- package/.xdrs/agentme/edrs/application/028-ai-eval-standards.md +0 -257
|
@@ -21,15 +21,41 @@ Projects MUST keep a `CONTRIBUTING.md` file at the repository root. The file MUS
|
|
|
21
21
|
|
|
22
22
|
### Details
|
|
23
23
|
|
|
24
|
-
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
-
|
|
24
|
+
#### 01-contributing-md-is-required
|
|
25
|
+
|
|
26
|
+
Every project MUST have a root `CONTRIBUTING.md` file at the repository root. The file MUST explain where bugs, feature discussions, and code changes belong so contributors follow a predictable workflow before opening pull requests.
|
|
27
|
+
|
|
28
|
+
#### 02-guide-must-direct-bug-reports-to-issues
|
|
29
|
+
|
|
30
|
+
The guide MUST direct bug reports to issues.
|
|
31
|
+
|
|
32
|
+
#### 03-guide-must-route-feature-discussions-to-issues
|
|
33
|
+
|
|
34
|
+
The guide MUST direct feature ideas and feature discussions to issues before implementation starts.
|
|
35
|
+
|
|
36
|
+
#### 04-guide-must-require-pull-requests
|
|
37
|
+
|
|
38
|
+
The guide MUST state that fixes and features are contributed through pull requests.
|
|
39
|
+
|
|
40
|
+
#### 05-guide-must-require-feature-branches
|
|
41
|
+
|
|
42
|
+
The guide MUST state that pull requests come from feature branches targeting `main`.
|
|
43
|
+
|
|
44
|
+
#### 06-guide-must-require-conventional-comments
|
|
45
|
+
|
|
46
|
+
The guide MUST ask reviewers and contributors to use [Conventional Comments](https://conventionalcomments.org/) for review feedback.
|
|
47
|
+
|
|
48
|
+
#### 07-guide-must-ask-for-small-pull-requests
|
|
49
|
+
|
|
50
|
+
The guide MUST ask contributors to keep pull requests small enough to keep review and discussion focused.
|
|
51
|
+
|
|
52
|
+
#### 08-scaffolding-should-create-contributing-md
|
|
53
|
+
|
|
54
|
+
Project scaffolding skills SHOULD create the file by default when they initialize a repository.
|
|
55
|
+
|
|
56
|
+
#### 09-content-should-be-concise
|
|
57
|
+
|
|
58
|
+
The content SHOULD stay concise and practical; do not turn `CONTRIBUTING.md` into a duplicate of `README.md`.
|
|
33
59
|
|
|
34
60
|
## Considered Options
|
|
35
61
|
|
|
@@ -27,21 +27,36 @@ Synthetic views combining agentme XDRs and skills around a specific topic.
|
|
|
27
27
|
|
|
28
28
|
Language and framework-specific tooling and project structure.
|
|
29
29
|
|
|
30
|
+
### Language and framework tooling
|
|
31
|
+
|
|
30
32
|
- [agentme-edr-003](application/003-javascript-project-tooling.md) - **JavaScript project tooling and structure** - Scaffold JavaScript libraries with the standard toolchain *(includes skill: [001-create-javascript-project](application/skills/001-create-javascript-project/SKILL.md))*
|
|
31
33
|
- [agentme-edr-010](application/010-golang-project-tooling.md) - **Go project tooling and structure** - Scaffold Go CLIs and libraries with the standard layout *(includes skill: [003-create-golang-project](application/skills/003-create-golang-project/SKILL.md))*
|
|
32
34
|
- [agentme-edr-014](application/014-python-project-tooling.md) - **Python project tooling and structure** - Scaffold Python packages and CLIs with the standard layout *(includes skill: [005-create-python-project](application/skills/005-create-python-project/SKILL.md))*
|
|
33
35
|
- [agentme-edr-015](application/015-cli-tool-standards.md) - **CLI tool standards** - Define command UX and behavior for CLI tools
|
|
36
|
+
- [agentme-edr-026](application/026-pragmatic-hexagonal-architecture.md) - **Pragmatic hexagonal architecture** - Organize application layers as External/Adapters/Application with practical coupling rules
|
|
37
|
+
- [004-select-relevant-xdrs](application/skills/004-select-relevant-xdrs/SKILL.md) - **Select relevant XDRs**
|
|
38
|
+
|
|
39
|
+
### AI development
|
|
40
|
+
|
|
41
|
+
Standards for building LLM, Agent, and Workflow components.
|
|
42
|
+
|
|
34
43
|
- [agentme-edr-018](application/018-ai-llm-development-standards.md) - **AI LLM development standards** - Standard framework (LangChain) and patterns for simple LLM calls with explicit configuration (no environment variables)
|
|
35
44
|
- [agentme-edr-019](application/019-ai-agents-development-standards.md) - **AI agents development standards** - Structural patterns for agents: framework selection, sandbox setup, naming conventions, composition, and system prompt structure
|
|
36
45
|
- [agentme-edr-020](application/020-ai-agents-quality-standards.md) - **AI agents implementation quality standards** - Tool definition patterns, error handling, observability, and unit testing for agents
|
|
37
46
|
- [agentme-edr-021](application/021-ai-workflow-development-standards.md) - **AI workflow development standards** - Standard toolchain (LangGraph), evaluation, and testing patterns for workflow projects
|
|
38
47
|
- [agentme-edr-029](application/029-ai-workflow-naming-conventions.md) - **AI workflow naming conventions** - Node suffix/prefix roles, state type and attribute naming, judge output schema, workflow class names, and cross-element coherence rules
|
|
39
|
-
- [agentme-edr-028](application/028-ai-eval-standards.md) - **AI eval standards** - Folder structure, script requirements, and MLflow tracking for eval tests across LLM, Agent, and Workflow tiers
|
|
40
|
-
- [agentme-edr-030](application/030-ai-test-types-taxonomy.md) - **AI test types taxonomy** - Names AI test types (safety, responsible-AI, quality-eval, prompt, code-level) with group, objective, mocking constraint, and relevance, and defines the shared golden dataset entry envelope
|
|
41
|
-
- [agentme-edr-024](application/024-ml-dataset-structure.md) - **ML dataset structure** - Standard folder layout and file conventions for ML datasets
|
|
42
48
|
- [agentme-edr-025](application/025-ai-agent-xdrs-knowledge-layer.md) - **AI agent XDRS knowledge layer** - How to integrate XDRS as the runtime source of truth for policies and skills in AI agents (apply only when the project explicitly uses XDRS)
|
|
43
|
-
|
|
44
|
-
|
|
49
|
+
|
|
50
|
+
### AI evaluation and testing
|
|
51
|
+
|
|
52
|
+
Standards for eval datasets, scripts, reports, and test type taxonomy.
|
|
53
|
+
|
|
54
|
+
- [agentme-edr-030](application/030-ai-test-types-taxonomy.md) - **AI test types taxonomy** - Names AI test types (`functional`, `safety`, `smoke`, `repeatability`, `adversarial`, `fairness`, and 6 others) with group, objective, mocking constraint, and relevance, and defines the shared golden dataset entry envelope
|
|
55
|
+
- [agentme-edr-028](application/028-ai-eval-core-standards.md) - **AI eval core standards** - Eval folder structure and Makefile interface; LLM-as-judge binary scoring contract applicable to all AI tiers and test types
|
|
56
|
+
- [agentme-edr-031](application/031-ai-eval-script.md) - **AI eval script** - eval.py requirements: entry-first loop, --type filtering, mock_fixtures wiring, human entries, threshold enforcement, and MLflow experiment conventions
|
|
57
|
+
- [agentme-edr-032](application/032-ai-eval-report-format.md) - **AI eval report format** - report-<type>.md template, Wilson score confidence interval, convergence analysis, and human-type checklist artifact
|
|
58
|
+
- [agentme-edr-033](application/033-ai-eval-repeatability.md) - **AI eval repeatability** - Repeatability test type: REPEAT_COUNT loop exception, semantic-similarity and LLM-as-judge scoring, repeatability_accuracy metric, report shape, and run cadence
|
|
59
|
+
- [agentme-edr-024](application/024-ml-dataset-structure.md) - **ML dataset structure** - Standard folder layout and file conventions for ML datasets
|
|
45
60
|
|
|
46
61
|
## Devops
|
|
47
62
|
|
|
@@ -45,7 +45,7 @@ All services **MUST** expose a `GET /health` endpoint that validates external de
|
|
|
45
45
|
|
|
46
46
|
- `health` (required): overall state — `OK`, `WARNING`, or `ERROR`
|
|
47
47
|
- `latencyMs` (required): total milliseconds to run all checks
|
|
48
|
-
- `message` (required): human-readable summary;
|
|
48
|
+
- `message` (required): human-readable summary; MUST NOT expose credentials, internal IPs, or stack traces
|
|
49
49
|
|
|
50
50
|
**Dependency validation rules:**
|
|
51
51
|
|
|
@@ -15,13 +15,15 @@ What unit testing practices should be followed to ensure tests are meaningful, r
|
|
|
15
15
|
|
|
16
16
|
## Decision Outcome
|
|
17
17
|
|
|
18
|
-
**Every test
|
|
18
|
+
**Every test MUST assert behavior, run offline without external dependencies, enforce 80% coverage, centralize shared setup, and prefer real code over mocks.**
|
|
19
19
|
|
|
20
20
|
### Details
|
|
21
21
|
|
|
22
22
|
#### 01-must-have-at-least-one-assertion-per-test
|
|
23
23
|
|
|
24
24
|
Every test MUST have at least one assertion that validates the expected behavior.
|
|
25
|
+
|
|
26
|
+
```typescript
|
|
25
27
|
// bad — no assertion; passes even when code is broken
|
|
26
28
|
it("processes the order", () => { processOrder(mockOrder); });
|
|
27
29
|
|
|
@@ -36,7 +38,7 @@ it("processes the order and returns a confirmation id", () => {
|
|
|
36
38
|
|
|
37
39
|
#### 02-must-run-offline
|
|
38
40
|
|
|
39
|
-
Unit tests MUST NOT depend on any external resources: no network calls, no running databases, no external APIs, no file system paths outside the repo. Tests
|
|
41
|
+
Unit tests MUST NOT depend on any external resources: no network calls, no running databases, no external APIs, no file system paths outside the repo. Tests MUST pass with only static code available.
|
|
40
42
|
|
|
41
43
|
```typescript
|
|
42
44
|
// bad — hits a real HTTP endpoint
|
|
@@ -83,7 +85,7 @@ src/mymodule/group1/file1.ts ← source
|
|
|
83
85
|
src/mymodule/group1/file1.test.ts ← test (same directory)
|
|
84
86
|
```
|
|
85
87
|
|
|
86
|
-
**Exception — separate test folder:** When the framework makes co-location impractical (e.g. Python's common `tests/` convention), or when the community strongly favors a separate folder, a dedicated test root (e.g. `tests/`) is allowed. In that case the test folder
|
|
88
|
+
**Exception — separate test folder:** When the framework makes co-location impractical (e.g. Python's common `tests/` convention), or when the community strongly favors a separate folder, a dedicated test root (e.g. `tests/`) is allowed. In that case the test folder MUST mirror the source folder structure exactly:
|
|
87
89
|
|
|
88
90
|
```
|
|
89
91
|
src/mymodule/group1/file1.py ← source
|
|
@@ -111,7 +113,7 @@ export function makeOrder(overrides: Partial<Order> = {}): Order {
|
|
|
111
113
|
|
|
112
114
|
Tests SHOULD use the lowest-cost alternative that exercises real behavior:
|
|
113
115
|
|
|
114
|
-
1. **Real implementation** —
|
|
116
|
+
1. **Real implementation** — MUST be preferred
|
|
115
117
|
2. **In-memory / lightweight fake** — e.g. in-memory DB, stub HTTP server
|
|
116
118
|
3. **Recorded fixture** — replay captured real responses
|
|
117
119
|
4. **Mock / stub** — only for external APIs, irreversible operations, or hardware I/O
|
|
@@ -15,9 +15,9 @@ What minimum quality standards must every project in the organization meet to en
|
|
|
15
15
|
|
|
16
16
|
## Decision Outcome
|
|
17
17
|
|
|
18
|
-
Every project
|
|
18
|
+
Every project MUST meet the minimum quality standards: a Getting Started section in its README, unit tests that run on every release, compliance with workspace XDRs, active linting enforcement, a structure that is clear to new developers, and — for libraries and utilities — a runnable examples folder verified on every test run. Integration tests are advised but not required. Projects with statistical models MUST have evaluation targets with performance thresholds.
|
|
19
19
|
|
|
20
|
-
These standards form a non-negotiable baseline. Individual projects may raise the bar but
|
|
20
|
+
These standards form a non-negotiable baseline. Individual projects may raise the bar but MUST NOT fall below it.
|
|
21
21
|
|
|
22
22
|
### Details
|
|
23
23
|
|
|
@@ -52,14 +52,14 @@ myFunction({ input: "value" });
|
|
|
52
52
|
|
|
53
53
|
#### 02-unit-tests-must-run-on-every-release
|
|
54
54
|
|
|
55
|
-
A unit test suite MUST run automatically before every release. Failing tests
|
|
55
|
+
A unit test suite MUST run automatically before every release. Failing tests MUST block the release — no silent skips or overrides.
|
|
56
56
|
|
|
57
57
|
**Requirements:**
|
|
58
|
-
- A `make test` target
|
|
59
|
-
- CI/CD
|
|
58
|
+
- A `make test` target MUST exist and run the full suite
|
|
59
|
+
- CI/CD MUST invoke it before publish/deploy
|
|
60
60
|
- Test failures block the release
|
|
61
61
|
|
|
62
|
-
**Exception:** Projects with fewer than 100 lines of code, or whose `README.md` prominently marks them as a **Spike** or **Experiment**, are exempt from this requirement. Such projects
|
|
62
|
+
**Exception:** Projects with fewer than 100 lines of code, or whose `README.md` prominently marks them as a **Spike** or **Experiment**, are exempt from this requirement. Such projects MUST NOT be deployed to production.
|
|
63
63
|
|
|
64
64
|
**Reference:** [agentme-edr-004](004-unit-test-requirements.md) for detailed unit test requirements.
|
|
65
65
|
|
|
@@ -85,7 +85,7 @@ Projects larger than 10 files or 200 lines of code MUST have a linter configured
|
|
|
85
85
|
- Linter config is checked in (e.g., `.eslintrc.js`, `pyproject.toml`, `.golangci.yml`)
|
|
86
86
|
- CI runs `make lint` before merging or releasing
|
|
87
87
|
|
|
88
|
-
**Exception:** Projects with fewer than 100 lines of code, or whose `README.md` prominently marks them as a **Spike** or **Experiment**, are exempt from this requirement. Such projects
|
|
88
|
+
**Exception:** Projects with fewer than 100 lines of code, or whose `README.md` prominently marks them as a **Spike** or **Experiment**, are exempt from this requirement. Such projects MUST NOT be deployed to production.
|
|
89
89
|
|
|
90
90
|
**Reference:** [agentme-edr-003](../application/003-javascript-project-tooling.md) for JavaScript-specific tooling.
|
|
91
91
|
|
|
@@ -93,11 +93,11 @@ Projects larger than 10 files or 200 lines of code MUST have a linter configured
|
|
|
93
93
|
|
|
94
94
|
#### 05-project-structure-must-be-clear
|
|
95
95
|
|
|
96
|
-
Directory and file layout MUST be self-explanatory: source code, tests, configuration, and examples
|
|
96
|
+
Directory and file layout MUST be self-explanatory: source code, tests, configuration, and examples MUST be clearly separated and named.
|
|
97
97
|
|
|
98
98
|
**Requirements:**
|
|
99
|
-
- Directory names
|
|
100
|
-
- README
|
|
99
|
+
- Directory names MUST reflect their purpose (`src/`, `lib/`, `tests/`, `examples/`, `docs/`)
|
|
100
|
+
- README MUST describe the top-level layout if non-obvious
|
|
101
101
|
- No orphaned or unexplained directories or files at the project root
|
|
102
102
|
|
|
103
103
|
**Example layout (TypeScript project):**
|
|
@@ -118,14 +118,14 @@ Directory and file layout MUST be self-explanatory: source code, tests, configur
|
|
|
118
118
|
|
|
119
119
|
#### 06-libraries-must-have-runnable-examples
|
|
120
120
|
|
|
121
|
-
Projects that are libraries or shared utilities MUST include an `examples/` directory. Each subdirectory represents a usage scenario and
|
|
121
|
+
Projects that are libraries or shared utilities MUST include an `examples/` directory. Each subdirectory represents a usage scenario and MUST be independently runnable. Examples that are "offline" (require no external credentials, no running servers, no paid APIs, and no environment-specific configuration outside the repository) MUST be executed as part of `make test`. Examples that depend on external entities may be left out of `make test`.
|
|
122
122
|
|
|
123
123
|
**Requirements:**
|
|
124
|
-
- `examples/`
|
|
125
|
-
- Each scenario subdirectory
|
|
126
|
-
- Examples
|
|
127
|
-
- `make test` in the root
|
|
128
|
-
- Examples that depend on external entities
|
|
124
|
+
- `examples/` MUST contain at least one subdirectory per major usage scenario
|
|
125
|
+
- Each scenario subdirectory MUST have a `Makefile` with a `run` target
|
|
126
|
+
- Examples MUST import the library as an external consumer (not via relative `../src` imports)
|
|
127
|
+
- `make test` in the root MUST run all offline examples; failures block CI and releases
|
|
128
|
+
- Examples that depend on external entities MUST NOT be included in `make test`
|
|
129
129
|
|
|
130
130
|
**Directory layout:**
|
|
131
131
|
|
|
@@ -173,25 +173,30 @@ all:
|
|
|
173
173
|
Projects that contain statistical models (e.g., ML models, LLM-based evaluators, classifiers, ranking systems, or any component whose output quality is measured probabilistically) MUST define measurable performance thresholds and verify them automatically.
|
|
174
174
|
|
|
175
175
|
**Requirements:**
|
|
176
|
-
- A `make eval` target
|
|
177
|
-
- Each evaluation
|
|
178
|
-
- Thresholds
|
|
179
|
-
- `make eval`
|
|
176
|
+
- A `make eval` target MUST exist and execute all performance evaluations
|
|
177
|
+
- Each evaluation MUST have a **documented minimum performance threshold** (e.g., accuracy ≥ 0.85, F1 ≥ 0.80, BLEU ≥ 0.70)
|
|
178
|
+
- Thresholds and all scoring parameters MUST be declared as constants in `eval.py` — they are design decisions about acceptable quality for the component under test, not runtime configuration, and MUST NOT be passed as Makefile variables or CLI flags. See [agentme-edr-031](../application/031-ai-eval-script.md) rule `01`.
|
|
179
|
+
- `make eval` MUST **exit with a non-zero status** (fail) if:
|
|
180
180
|
- The evaluation cannot be executed (missing data, environment errors, model load failures)
|
|
181
181
|
- Any metric falls below its defined minimum threshold
|
|
182
|
-
- CI/CD
|
|
182
|
+
- CI/CD MUST invoke `make eval` before releasing any version that changes model weights, prompts, or evaluation logic
|
|
183
183
|
|
|
184
|
-
**Threshold declaration example (
|
|
184
|
+
**Threshold declaration example (eval.py):**
|
|
185
185
|
|
|
186
|
-
```
|
|
187
|
-
EVAL_MIN_ACCURACY
|
|
188
|
-
EVAL_MIN_F1
|
|
186
|
+
```python
|
|
187
|
+
EVAL_MIN_ACCURACY = {"functional": 0.85, "smoke": 0.90}
|
|
188
|
+
EVAL_MIN_F1 = {"functional": 0.80}
|
|
189
|
+
|
|
190
|
+
# Thresholds are declared here and enforced by the script:
|
|
191
|
+
if accuracy < EVAL_MIN_ACCURACY.get(test_type, 0):
|
|
192
|
+
raise SystemExit(f"Eval failed: {test_type} accuracy {accuracy:.2f} < {EVAL_MIN_ACCURACY[test_type]}")
|
|
193
|
+
```
|
|
189
194
|
|
|
195
|
+
**Makefile target (delegates to eval.py — no threshold variables here):**
|
|
196
|
+
|
|
197
|
+
```makefile
|
|
190
198
|
eval:
|
|
191
|
-
python eval.py
|
|
192
|
-
--min-accuracy $(EVAL_MIN_ACCURACY) \
|
|
193
|
-
--min-f1 $(EVAL_MIN_F1) \
|
|
194
|
-
|| (echo "Evaluation failed: metrics below threshold"; exit 1)
|
|
199
|
+
mise exec -- uv run --project . python eval.py --type=all
|
|
195
200
|
```
|
|
196
201
|
|
|
197
202
|
---
|
|
@@ -261,5 +266,5 @@ AI projects are classified into three tiers — LLM, Agent, and Workflow — def
|
|
|
261
266
|
- Evals MUST be executed before every release.
|
|
262
267
|
- Accuracy below project-defined thresholds MUST block the release. Thresholds MUST be documented in the eval Makefile or README.
|
|
263
268
|
- Evals MUST run against real LLM providers (not mocks) to capture model drift.
|
|
264
|
-
- For eval folder structure and
|
|
269
|
+
- For eval folder structure and LLM-as-judge scoring, see [agentme-edr-028](../application/028-ai-eval-core-standards.md). For eval script requirements, see [agentme-edr-031](../application/031-ai-eval-script.md).
|
|
265
270
|
- For the taxonomy of AI test types (safety, responsible-AI, quality-eval, prompt, code-level) and the golden dataset entry format, see [agentme-edr-030](../application/030-ai-test-types-taxonomy.md).
|
|
@@ -211,7 +211,7 @@ Every system boundary MUST signal failure explicitly:
|
|
|
211
211
|
|
|
212
212
|
- **OS processes** MUST exit with a **non-zero exit code** when something went wrong. Exit code `0` means success.
|
|
213
213
|
- **HTTP services** MUST return a **non-2xx/3xx status code** on error, accompanied by a response body that describes the problem without exposing internal system details (stack traces, SQL queries, internal paths, etc.).
|
|
214
|
-
- **All error responses** SHOULD be logged to the console/structured logger, especially system-level or unexpected errors. Operational teams
|
|
214
|
+
- **All error responses** SHOULD be logged to the console/structured logger, especially system-level or unexpected errors. Operational teams MUST be able to find the cause from logs alone.
|
|
215
215
|
|
|
216
216
|
**Examples:**
|
|
217
217
|
|
|
@@ -21,15 +21,33 @@ Developers MUST treat reusable missing guidance discovered during implementation
|
|
|
21
21
|
|
|
22
22
|
### Details
|
|
23
23
|
|
|
24
|
-
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
-
|
|
24
|
+
#### 01-reusable-guidance-must-become-shared-xdr
|
|
25
|
+
|
|
26
|
+
Developers MUST treat reusable missing guidance discovered during implementation as an XDR gap to be proposed and reviewed, not as permanent prompt-only context or repeated vibe coding. The main objective is sharing, discussing, and converging practices across teams. Controlled divergence during exploration is acceptable, but recurring successful decisions MUST be converged into shared XDRs.
|
|
27
|
+
|
|
28
|
+
#### 02-local-scope-must-be-for-truly-specific-decisions
|
|
29
|
+
|
|
30
|
+
Decisions placed in `_local` SHOULD be truly specific to the needs of a single application or repository. The non-`_local` scope exists to share practices across projects, company areas, and functionally organized teams.
|
|
31
|
+
|
|
32
|
+
#### 03-steering-needs-must-trigger-xdr-reflection
|
|
33
|
+
|
|
34
|
+
When developers or coding agents need too much detailed steering to complete a task, they MUST reflect on whether those details would help other teams or future implementations. If yes, create or update an XDR proposal in the broadest appropriate shared scope. This includes cases where an agent implemented a feature without a framework, pattern, coding standard, or other practice that should likely be standardized. Missing reusable guardrails SHOULD trigger an XDR proposal.
|
|
35
|
+
|
|
36
|
+
#### 04-coverage-target-should-be-80-percent
|
|
37
|
+
|
|
38
|
+
Teams SHOULD aim to keep at least 80% of big coding decisions covered by accepted XDRs. Big decisions include framework or tool selection, overall code organization, monorepo structure, complex business flows, and coding standards. If a big decision is not yet covered, developers SHOULD either propose a new XDR or document why the decision is intentionally local and should not be shared.
|
|
39
|
+
|
|
40
|
+
#### 05-leaders-must-review-xdr-proposals
|
|
41
|
+
|
|
42
|
+
Leaders responsible for the affected scope MUST review XDR proposals, adjust them as needed, and publish the accepted decision.
|
|
43
|
+
|
|
44
|
+
#### 06-query-agents-for-missing-xdrs
|
|
45
|
+
|
|
46
|
+
It is good practice to ask the coding agent which missing XDRs made the task harder, increased adjustment rounds, or forced more vibe coding. Those gaps SHOULD feed the XDR backlog.
|
|
47
|
+
|
|
48
|
+
#### 07-xdrs-are-not-feature-specs
|
|
49
|
+
|
|
50
|
+
In SDD, specifications describe the feature being built; XDRs describe reusable decisions and guardrails that MUST survive beyond one feature. Do not keep durable engineering policy only inside feature specs.
|
|
33
51
|
|
|
34
52
|
## Considered Options
|
|
35
53
|
|
|
@@ -88,7 +88,7 @@ Examples that demonstrate how to consume a library or reusable module MUST live
|
|
|
88
88
|
Examples MUST exercise the module through its public distribution surface:
|
|
89
89
|
|
|
90
90
|
- use the package built into `dist/` when the ecosystem supports local packaged artifacts
|
|
91
|
-
- otherwise use the public module path or equivalent consumer-facing import surface;
|
|
91
|
+
- otherwise use the public module path or equivalent consumer-facing import surface; MUST NOT use relative source-file imports or direct references to internal implementation paths
|
|
92
92
|
|
|
93
93
|
Example:
|
|
94
94
|
|
|
@@ -23,13 +23,13 @@ All implementation practices derive from three guiding principles:
|
|
|
23
23
|
|
|
24
24
|
1. **Least exposure** — minimize the means, timespan, and surface of contact with the secret.
|
|
25
25
|
2. **Easiness in secret rotation** — design so rotating a secret requires no code change or redeployment.
|
|
26
|
-
3. **Support for local and cloud deployment runs** — the same application code
|
|
26
|
+
3. **Support for local and cloud deployment runs** — the same application code MUST work transparently in both environments.
|
|
27
27
|
|
|
28
28
|
### Details
|
|
29
29
|
|
|
30
30
|
#### 01-no-secrets-on-disk
|
|
31
31
|
|
|
32
|
-
Secrets MUST
|
|
32
|
+
Secrets MUST NOT be stored on the disk of a developer machine or server. This includes `.env` files (even when gitignored), plaintext config files, embedded in source code, or any other file-based storage.
|
|
33
33
|
|
|
34
34
|
The only acceptable local persistence is through the operating system's native secret manager (e.g., macOS Keychain, Windows Credential Manager, Linux Secret Service).
|
|
35
35
|
|
|
@@ -96,31 +96,11 @@ $ make run
|
|
|
96
96
|
# Application starts successfully
|
|
97
97
|
```
|
|
98
98
|
|
|
99
|
-
#### 05a-makefile-uses-security-utility
|
|
100
|
-
|
|
101
|
-
Makefile targets (e.g., `setup-secrets`) must use the macOS native `security` CLI to store and retrieve secrets from the keychain. This restricts Makefile-based secret management to macOS developer machines, which is acceptable since all contributors are expected to use macOS.
|
|
102
|
-
|
|
103
|
-
Do **not** use `keyring` or other cross-platform libraries in Makefiles — `security` is simpler to invoke from shell and requires no additional dependencies.
|
|
104
|
-
|
|
105
|
-
Storing a secret:
|
|
106
|
-
```makefile
|
|
107
|
-
security add-generic-password -a "$(USER)" -s "mymodule/api-key" -w "$(SECRET_VALUE)" -U
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
Retrieving a secret (e.g., to pass to a command):
|
|
111
|
-
```makefile
|
|
112
|
-
SECRET_VALUE := $(shell security find-generic-password -a "$(USER)" -s "mymodule/api-key" -w 2>/dev/null)
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
The `-U` flag updates the entry if it already exists. Use the format `<group>/<secret-id>` as the service name (`-s`) to mirror the module name and cloud secret manager ID convention defined in rule 02 and 05.
|
|
116
|
-
|
|
117
|
-
In library code (Python, JS/TS, Go), continue using the cross-platform libraries defined in rule 02 (`keyring`, `cross-keychain`, `go-keyring`). The `security` utility is only for Makefile scripts.
|
|
118
|
-
|
|
119
99
|
---
|
|
120
100
|
|
|
121
101
|
#### 06-never-log-or-leak-secrets
|
|
122
102
|
|
|
123
|
-
Secrets MUST
|
|
103
|
+
Secrets MUST NOT be logged under any circumstance or sent to any service that is not clearly the intended consumer of that secret (authentication, encryption, etc.). This applies to all log levels including debug and trace. Error messages MUST reference the secret name or identifier, MUST NOT include its value.
|
|
124
104
|
|
|
125
105
|
---
|
|
126
106
|
|
|
@@ -132,7 +112,7 @@ Wherever possible, secrets SHOULD be fetched dynamically from the secret manager
|
|
|
132
112
|
- Immediate propagation of rotated secrets.
|
|
133
113
|
- Reduced window of exposure if memory is compromised.
|
|
134
114
|
|
|
135
|
-
Short-lived caching (e.g., a few minutes) is acceptable when performance requires it, but
|
|
115
|
+
Short-lived caching (e.g., a few minutes) is acceptable when performance requires it, but MUST have an explicit TTL.
|
|
136
116
|
|
|
137
117
|
---
|
|
138
118
|
|
|
@@ -160,6 +140,28 @@ def test_service_uses_api_key():
|
|
|
160
140
|
|
|
161
141
|
Integration tests MAY use the real keychain on developer machines or CI after `make setup-secrets` has been run.
|
|
162
142
|
|
|
143
|
+
---
|
|
144
|
+
|
|
145
|
+
#### 10-makefile-uses-security-utility
|
|
146
|
+
|
|
147
|
+
Makefile targets (e.g., `setup-secrets`) MUST use the macOS native `security` CLI to store and retrieve secrets from the keychain. This restricts Makefile-based secret management to macOS developer machines, which is acceptable since all contributors are expected to use macOS.
|
|
148
|
+
|
|
149
|
+
Do not use `keyring` or other cross-platform libraries in Makefiles — `security` is simpler to invoke from shell and requires no additional dependencies.
|
|
150
|
+
|
|
151
|
+
Storing a secret:
|
|
152
|
+
```makefile
|
|
153
|
+
security add-generic-password -a "$(USER)" -s "mymodule/api-key" -w "$(SECRET_VALUE)" -U
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Retrieving a secret (e.g., to pass to a command):
|
|
157
|
+
```makefile
|
|
158
|
+
SECRET_VALUE := $(shell security find-generic-password -a "$(USER)" -s "mymodule/api-key" -w 2>/dev/null)
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
The `-U` flag updates the entry if it already exists. Use the format `<group>/<secret-id>` as the service name (`-s`) to mirror the module name and cloud secret manager ID convention defined in rule 02 and 05.
|
|
162
|
+
|
|
163
|
+
In library code (Python, JS/TS, Go), continue using the cross-platform libraries defined in rule 02 (`keyring`, `cross-keychain`, `go-keyring`). The `security` utility is only for Makefile scripts.
|
|
164
|
+
|
|
163
165
|
## References
|
|
164
166
|
|
|
165
167
|
- [agentme-edr-008](../devops/008-common-targets.md) - Common development script names (defines Makefile target conventions)
|
|
@@ -41,7 +41,7 @@ These patterns obfuscate the main program flow and create behavioral indirection
|
|
|
41
41
|
|
|
42
42
|
#### 03-trivial-wrappers-are-prohibited
|
|
43
43
|
|
|
44
|
-
A function that merely delegates to another function or API call without adding meaningful logic, domain intent, or readability
|
|
44
|
+
A function that merely delegates to another function or API call without adding meaningful logic, domain intent, or readability MUST be inlined. A wrapper is justified only when it:
|
|
45
45
|
|
|
46
46
|
- Encapsulates non-trivial logic (validation, retry, transformation).
|
|
47
47
|
- Communicates a domain concept the underlying expression does not convey.
|