agentme 0.20.0 → 0.21.0

package/.xdrs/agentme/edrs/application/019-ai-agents-development-standards.md

@@ -178,7 +178,7 @@ e.g.: Respond with a JSON object matching this schema: ... ALWAYS return valid J
 </WORKFLOW_CONTEXT>
 
 <SYSTEM_CONTEXT>
-The current date/time is [now in ISO 8601 format].
+The current date is [today in YYYY-MM-DD format].
 The current OS is: [operating system name].
 </SYSTEM_CONTEXT>
 ```
@@ -187,7 +187,7 @@ The current OS is: [operating system name].
 
 | Section | Required? | Notes |
 |---|---|---|
-| `<SYSTEM_CONTEXT>` | Optional | Runtime environment context injected at invocation time (e.g., current date/time in ISO 8601, OS). Include whenever the agent may need temporal or environment awareness. |
+| `<SYSTEM_CONTEXT>` | Optional | Runtime environment context injected at invocation time (e.g., current date in YYYY-MM-DD, OS). Include whenever the agent may need temporal or environment awareness. Time MUST NOT be included — it changes every second and breaks prompt caching. |
 | `<OBJECTIVE>` | Required | One or two sentences summarising the agent's main deliverable. |
 | `<ROLE>` | Required | Agent persona and expertise. When inside a workflow, MUST reference its node name from `<WORKFLOW_CONTEXT>`. |
 | `<INPUT>` | Required | List ALL inputs. For workflow agents: workflow-level inputs first, then agent-specific inputs. |

package/.xdrs/agentme/edrs/application/028-ai-eval-standards.md

@@ -23,36 +23,41 @@ For when evals are required per AI tier, see [agentme-edr-007](../principles/007
 
 #### 01-eval-folder-structure
 
-Each named eval is a self-contained unit. Create one directory per eval under `evals/` at the same level as `lib/` and `examples/`:
+Evals are grouped first by the component being evaluated, then by the specific evaluation scenario. Create one directory per component under `evals/`, and one directory per eval scenario inside it. Place `evals/` at the same level as `lib/` and `examples/`:
 
 ```text
 evals/
-  eval-<name>/
-    dataset/              # EDR-024 compliant dataset (README.md, dataset.schema.json, data/)
-    eval-<name>.py        # evaluation script
-    eval-report.md        # generated report (overwritten on each run — see rule 03)
-    Makefile              # eval and run targets
-  eval-<name2>/
+  <component>/           # the component being evaluated (e.g., workflow-x, agent-y, model-z)
+    eval-<name>/
+      dataset/           # EDR-024 compliant dataset (README.md, dataset.schema.json, data/)
+      eval-<name>.py     # evaluation script
+      eval-<name>-report.md     # generated report (overwritten on each run — see rule 03)
+      Makefile           # eval and run targets
+    eval-<name2>/
+      ...
+  <component2>/
     ...
 ```
 
-Where `<name>` identifies the specific evaluation scenario (e.g., `eval-basic`, `eval-complex`, `eval-edge-cases`).
+`<component>` MUST match the name of the component under evaluation and use lowercase hyphen-separated words (e.g., `workflow-document-review`, `agent-support`, `model-classifier`).
+
+`<name>` identifies the specific evaluation scenario using lowercase hyphen-separated words (e.g., `eval-basic`, `eval-complex`, `eval-edge-cases`, `eval-bias-test`).
 
 The `dataset/` subfolder MUST be a valid [agentme-edr-024](024-ml-dataset-structure.md) dataset — it MUST include `README.md` and `dataset.schema.json` at its root. For input/output pairs, use JSONL files per `agentme-edr-024.04-complex-structured-datasets-must-use-jsonl`.
 
-Each `evals/eval-<name>/Makefile` MUST define:
+Each `evals/<component>/eval-<name>/Makefile` MUST define:
 
 | Target | Behaviour |
 |---|---|
 | `eval` | Runs the eval with threshold enforcement; exits non-zero on failure (CI-safe) |
 | `run` | Runs the eval without threshold enforcement (exploration / debugging) |
 
-The module root Makefile MUST expose a `make eval` target that delegates to `eval` in every `evals/eval-<name>/Makefile`:
+The module root Makefile MUST expose a `make eval` target that delegates to `eval` in every `evals/<component>/eval-<name>/Makefile`:
 
 ```makefile
 eval:
-	$(MAKE) -C evals/eval-basic eval
-	$(MAKE) -C evals/eval-complex eval
+	$(MAKE) -C evals/workflow-document-review/eval-basic eval
+	$(MAKE) -C evals/workflow-document-review/eval-complex eval
 ```
 
 #### 02-eval-script-requirements
@@ -63,7 +68,7 @@ Each `eval-<name>.py` script MUST:
 - Run every input through the live component against **real LLM providers** (not mocked responses), to capture model drift.
 - Log per-sample and aggregate metrics to an MLflow experiment that runs **locally** — a remote MLflow server MUST NOT be required.
 - Compare outputs to expected values using project-defined quality thresholds. Thresholds MUST be declared explicitly (e.g., in a Makefile variable or README).
-- Write `eval-report.md` in the same folder per rule `03-eval-report-file`.
+- Write `eval-<name>-report.md` in the same folder per rule `03-eval-report-file`.
 - Exit with a non-zero status when any metric falls below its defined threshold, consistent with [agentme-edr-007](../principles/007-project-quality-standards.md) rule `07-statistical-models-must-have-eval-targets`.
 
 **Example:**
@@ -91,7 +96,7 @@ with mlflow.start_run() as run:
 
 #### 03-eval-report-file
 
-Each eval script MUST produce `eval-report.md` in the same `evals/eval-<name>/` folder and overwrite it on every run.
+Each eval script MUST produce `eval-<name>-report.md` in the same `evals/<component>/eval-<name>/` folder and overwrite it on every run.
 
 **Generation constraint:** The report MUST be produced programmatically, reading raw metric values directly from MLflow. No LLM or generative model may write, summarize, or paraphrase any section of the report, to prevent hallucinated metric values.
 
@@ -137,7 +142,7 @@ $$\frac{\hat{p} + \frac{z^2}{2n} \pm z\sqrt{\frac{\hat{p}(1-\hat{p})}{n} + \frac
 
 Where $\hat{p}$ is observed accuracy and $n$ is sample count. Accuracy and F1 are required; precision and recall are recommended.
 
-**Filled-in example** (`evals/eval-basic/eval-report.md` for a document review workflow):
+**Filled-in example** (`evals/workflow-document-review/eval-basic/eval-basic-report.md` for a document review workflow):
 
 ```markdown
 # Eval Report: eval-basic
@@ -174,7 +179,7 @@ Where $\hat{p}$ is observed accuracy and $n$ is sample count. Accuracy and F1 ar
 ## Notes
 
 - Sample 005 misclassified: redlined IP clause not flagged as escalation trigger. Possible model drift.
-- MLflow run: experiment `eval_basic` — view with `mlflow ui`
+- MLflow run: experiment `workflow-document-review/eval-basic` — view with `mlflow ui`
 ```
 
 ## References

package/.xdrs/agentme/edrs/principles/007-project-quality-standards.md

@@ -118,13 +118,14 @@ Directory and file layout must be self-explanatory: source code, tests, configur
 
 #### 06-libraries-must-have-runnable-examples
 
-Projects that are libraries or shared utilities must include an `examples/` directory. Each subdirectory represents a usage scenario and must be independently runnable. Examples are executed as part of `make test`.
+Projects that are libraries or shared utilities must include an `examples/` directory. Each subdirectory represents a usage scenario and must be independently runnable. Examples that are "offline" (require no external credentials, no running servers, no paid APIs, and no environment-specific configuration outside the repository) must be executed as part of `make test`. Examples that depend on external entities may be left out of `make test`.
 
 **Requirements:**
 - `examples/` must contain at least one subdirectory per major usage scenario
 - Each scenario subdirectory must have a `Makefile` with a `run` target
 - Examples must import the library as an external consumer (not via relative `../src` imports)
-- `make test` in the root must run all examples; failures block CI and releases
+- `make test` in the root must run all offline examples; failures block CI and releases
+- Examples that depend on external entities must not be included in `make test`
 
 **Directory layout:**
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agentme",
-  "version": "0.20.0",
+  "version": "0.21.0",
   "description": "",
   "dependencies": {
     "filedist": "^0.36.0"