open-agents-ai 0.97.0 → 0.100.0

package/README.md

@@ -1459,7 +1459,7 @@ The agent auto-detects the provider, normalizes the URL (strips `/v1/chat/comple
 
 ## Evaluation Suite
 
-40 evaluation tasks test the agent's autonomous capabilities across coding, web research, SDLC analysis, tool creation, multi-file reasoning, and memory systems:
+46 evaluation tasks test the agent's autonomous capabilities across coding, web research, SDLC analysis, tool creation, multi-file reasoning, memory systems, and context engineering:
 
 ```bash
 node eval/run-agentic.mjs                          # Run all tasks
@@ -1503,19 +1503,26 @@ node eval/run-agentic.mjs --model qwen2.5-coder:32b  # Different model
 | 38 | Read configs, write to multiple memory topics | Memory Multi-Topic |
 | 39 | Search pre-loaded memories across 3 topics | Memory Pre-Loaded Search |
 | 40 | Combined explore_tools + memory analysis pipeline | Explore + Memory |
+| ce-01 | Instruction hierarchy (Priority 0 vs injected Priority 30) | Context Engineering |
+| ce-02 | Memory-backed context assembly | Context Engineering |
+| ce-03 | Progressive skill loading from SKILL.md | Context Engineering |
+| ce-04 | Multi-step error recovery chain (3 sequential bugs) | Context Engineering |
+| ce-05 | 8-file pipeline trace with context compression | Context Engineering |
+| ce-06 | Meta-analysis: write tests, find bugs, fix, document | Context Engineering |
 
-Tasks 31-33 are designed for small model (≤9B) evaluation using `file_edit` patterns. Tasks 34-40 test the memory system (read/write/search) and tool discovery.
+Tasks 31-33 are designed for small model (≤9B) evaluation using `file_edit` patterns. Tasks 34-40 test the memory system (read/write/search) and tool discovery. Tasks ce-01 through ce-06 validate context engineering capabilities grounded in current research (see Context Engineering section below).
 
 ### Benchmark Results
 
 ```
-Qwen3.5-122B: 100% pass rate (37/37 tasks, including memory tasks 34-40)
-Qwen3.5-27B:  100% pass rate (30/30 tasks)
+Qwen3.5-122B: 100% pass rate (37/37 core + 6/6 CE tasks)
+Qwen3.5-27B:  100% pass rate (30/30 core + 5/6 CE tasks)
 Qwen3.5-9B:   100% pass rate (tasks 31-33, file_edit-optimized)
               71% pass rate (5/7 memory tasks 34-40)
+              83% pass rate (5/6 CE tasks)
 ```
 
-The eval runner includes model-tier-aware features: automatic tool set filtering, HTTP 500 recovery with file_edit hints, loop detection with tool banning, and tier-based output truncation.
+The eval runner supports `--runs N` for pass^k reliability measurement (consistency across N independent runs, not just single-pass accuracy). Includes model-tier-aware features: automatic tool set filtering, HTTP 500 recovery with file_edit hints, proactive quality guidance (contextual next-step suggestions instead of tool banning), and tier-based output truncation.
 
 ## AIWG Integration
 
@@ -1534,21 +1541,46 @@ oa "analyze this project's SDLC health and set up documentation"
 | **85+ Agents** | Specialized AI personas (Test Engineer, Security Auditor, API Designer) |
 | **Traceability** | @-mention system links requirements to code to tests |
 
+## Context Engineering
+
+The agent implements structured context assembly based on current research in context engineering, modular prompt optimization, and instruction hierarchy:
+
+```
+C = A(c_instr, c_know, c_tools, c_mem, c_state, c_query)
+```
+
+| Component | Priority | Description |
+|-----------|----------|-------------|
+| `c_instr` | P0 (highest) | Core system instructions — immutable, cannot be overridden |
+| `c_state` | P10 | Personality profile, session state |
+| `c_know` | P20 | Dynamic project context, retrieved knowledge |
+| `c_tools` | P30 (lowest) | Tool outputs — may contain untrusted content |
+
+Key design decisions grounded in research:
+
+- **Instruction hierarchy** — 4-tier priority system (P0/P10/P20/P30) prevents prompt injection from tool outputs overriding system rules. Implemented across all 3 prompt tiers (large/medium/small) with model-appropriate verbosity
+- **Proactive quality guidance** — instead of banning tools after repeated use, the agent receives contextual next-step suggestions appended to tool output, preserving tool availability while steering toward productive actions
+- **Tiered system prompts** — large (≥30B), medium (8-29B), and small (≤7B) models get appropriately sized instruction sets, balancing capability with context budget
+- **Context composition tracing** — every context assembly emits a structured event showing section labels and token estimates for eval observability
+
+Research provenance: grounded in "A Survey of Context Engineering for LLMs" (context assembly equation), "Modular Prompt Optimization" (section-local textual gradients), "Reasoning Up the Instruction Ladder" (priority hierarchy), "GEPA" (reflective prompt evolution), and "Prompt Flow Integrity" (least-privilege context passing).
+
 ## Architecture
 
-The core is `AgenticRunner` — a multi-turn tool-calling loop with context management:
+The core is `AgenticRunner` — a multi-turn tool-calling loop with structured context assembly:
 
 ```
-User task → System prompt + tools → LLM → tool_calls → Execute → Feed results → LLM
-                                          ↓                                      ↑
-                                    Compaction check ─── Memex archive ─── Context restore
-                                          (repeat until task_complete or max turns)
+User task → assembleContext(c_instr, c_state, c_know) → LLM → tool_calls → Execute → Feed results → LLM
+                                                                ↓                                      ↑
+                                                          Compaction check ─── Memex archive ─── Context restore
+                                                                (repeat until task_complete or max turns)
 ```
 
+- **Context-first** — structured context assembly (C = A equation) replaces ad-hoc prompt construction
 - **Tool-first** — the model explores via tools, not pre-stuffed context
 - **Iterative** — tests, sees failures, fixes them
 - **Parallel-safe** — read-only tools concurrent, mutating tools sequential
-- **Observable** — every tool call and result emitted as a real-time event
+- **Observable** — every tool call, context composition, and result emitted as a real-time event
 - **Bounded** — max turns, timeout, output limits prevent runaway loops
 - **Context-aware** — dynamic compaction, Memex archiving, session persistence, model-tier scaling
 - **Brute-force** — optional auto re-engagement when turn limit is hit (keeps going until task_complete or user abort)