forgedock 1.0.19 → 1.0.21

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -48,6 +48,7 @@ This quality gate uses **domain detection** to adapt to your project's actual te
48
48
  | INFRA | Dockerfiles or entrypoints with runtime UID changes |
49
49
  | ROUTER_BUG | Router Python files where gate conditions are narrowed |
50
50
  | FORGE_GRAPH | `commands/*.md` specs or `scripts/*` files change (ForgeDock self-consistency) |
51
+ | CONFIG_SCHEMA | Config files for external tools (`traefik/`, `infra/nginx/`, `k8s/`, `terraform/`, `*.conf`, `*.toml` in infra paths, or `docker-compose*.yml` with service definition changes) |
51
52
 
52
53
  **Stack-agnostic coverage**: The example domains above reflect common patterns caught in production; they are not requirements. A Go or Ruby project benefits from SECURITY checks. A Node.js project benefits from FRONTEND, PROXY, and DEPLOY checks. Any project benefits from DATABASE and WORKFLOW checks when those file types are present. The stack-specific examples (Python routers, SOPS secrets chain, FastAPI layouts, appleboy SSH deploys) are illustrative — the domain detection system applies the applicable subset to any codebase.
53
54
 
@@ -100,6 +101,7 @@ Before running checks, classify the changed files into domains. This avoids runn
100
101
  | `Dockerfile*` or `entrypoint*.sh` with added/changed `USER`, `su-exec`, `gosu`, or `setuid` | INFRA |
101
102
  | `*.py` in `routers/` where the diff removes a line containing an or-gate condition (lines starting with `-` containing `if.*or`) | ROUTER_BUG |
102
103
  | `commands/*.md` or `scripts/*` files (ForgeDock repo only — dogfoods its own spec graph) | FORGE_GRAPH |
104
+ | Files under `traefik/`, `infra/nginx/`, `k8s/`, `terraform/`, or files matching `*.conf`, `*.toml` in infra paths, or `docker-compose*.yml` with service definition changes | CONFIG_SCHEMA |
103
105
 
104
106
  **Apply the classification:**
105
107
 
@@ -168,6 +170,19 @@ for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$' | grep -E 'route
168
170
  git diff HEAD -- "$f" 2>/dev/null | grep -E '^-.*\bif\b.*\bor\b' | grep -v '^---' | grep -q . && DOMAINS="$DOMAINS ROUTER_BUG" && break
169
171
  done
170
172
 
173
+ # Check for external tool config files (structural schema validation advisory)
174
+ for f in {CHANGED_FILES}; do
175
+ case "$f" in
176
+ traefik/*|infra/nginx/*|k8s/*|terraform/*)
177
+ DOMAINS="$DOMAINS CONFIG_SCHEMA" && break ;;
178
+ *.conf|*.toml)
179
+ echo "$f" | grep -qE 'infra/|traefik/|nginx/|k8s/|terraform/' && DOMAINS="$DOMAINS CONFIG_SCHEMA" && break ;;
180
+ docker-compose*.yml)
181
+ # Trigger only if service definitions changed (not just env vars or labels)
182
+ git diff HEAD -- "$f" 2>/dev/null | grep -E '^\+\s+(image:|command:|entrypoint:)' | grep -q . && DOMAINS="$DOMAINS CONFIG_SCHEMA" && break ;;
183
+ esac
184
+ done
185
+
171
186
  # SECURITY is always included for any code file
172
187
  DOMAINS="SECURITY $DOMAINS"
173
188
 
@@ -190,7 +205,7 @@ Run ONLY the checks whose domain was identified in Step 1.5. Skip checks for dom
190
205
  - **2D (Database)**: Run if `DATABASE` in DOMAINS
191
206
  - **2E (Frontend lifecycle)**: Run if `FRONTEND` in DOMAINS
192
207
  - **2F (Frontend proxy)**: Run if `PROXY` in DOMAINS
193
- - **2G (Cross-service integration)**: Run if `SHELL` in DOMAINS
208
+ - **2G (Cross-service integration)**: Run if `SHELL` or `FORGE_GRAPH` in DOMAINS
194
209
  - **2H (Asyncio cancellation)**: Run if `CONCURRENCY` in DOMAINS
195
210
  - **2I (State completeness)**: Run if `STATE` in DOMAINS
196
211
  - **2J (String literal consistency)**: Run if `STRING_SETS` in DOMAINS
@@ -199,6 +214,7 @@ Run ONLY the checks whose domain was identified in Step 1.5. Skip checks for dom
199
214
  - **2M (Import resolution)**: Run if `IMPORT_RESOLUTION` in DOMAINS
200
215
  - **2N (Runtime UID × filesystem write)**: Run if `INFRA` in DOMAINS
201
216
  - **2O (Residual pattern check)**: Run if `ROUTER_BUG` in DOMAINS
217
+ - **2P (External tool config schema)**: Run if `CONFIG_SCHEMA` in DOMAINS
202
218
 
203
219
  ### 2A: Security (ALL files)
204
220
 
@@ -836,6 +852,68 @@ done
836
852
  - Only triggers when a condition line was actually removed in the diff (not just modified)
837
853
  - Reports file:line matches from sibling files only (excludes the fixed file itself)
838
854
 
855
+ ### 2P: External Tool Config Schema Advisory (external tool config files)
856
+
857
+ **Triggered when**: CONFIG_SCHEMA domain is set (config files for external tools are in the diff).
858
+
859
+ **Why this matters**: External tools with strict config schemas (Traefik, nginx, Kubernetes, Terraform, Docker Compose) silently ignore or reject structurally incorrect config — no startup crash, no error log, the feature simply does not activate. The quality gate cannot run `traefik validate` / `nginx -t` / `terraform validate` (those belong in CI), but it can verify that CI validation steps exist for the tool and remind the builder that structural validation is CI's responsibility. <!-- Added: forge#1104 -->
860
+
861
+ ```bash
862
+ # Identify which external tools have config files in the diff
863
+ TOOL_FILES=$(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E \
864
+ "(traefik/|infra/nginx/|k8s/|terraform/|docker-compose.*\.yml$|.*\.(conf|toml)$)" | head -20)
865
+
866
+ if [ -n "$TOOL_FILES" ]; then
867
+ echo "CONFIG_SCHEMA: External tool config files detected in diff:"
868
+ echo "$TOOL_FILES"
869
+ echo ""
870
+
871
+ # Check whether CI has a validation step for each detected tool
872
+ WORKFLOW_DIR="{WORKTREE_PATH}/.github/workflows"
873
+ CI_MISSING=""
874
+
875
+ for f in $TOOL_FILES; do
876
+ case "$f" in
877
+ traefik/*)
878
+ grep -rlE "traefik.*validate|traefik.*check" "$WORKFLOW_DIR" 2>/dev/null | grep -q . || \
879
+ CI_MISSING="$CI_MISSING traefik"
880
+ ;;
881
+ infra/nginx/*|*nginx*.conf)
882
+ grep -rlE "nginx.*-t|nginx.*test|nginx.*configtest" "$WORKFLOW_DIR" 2>/dev/null | grep -q . || \
883
+ CI_MISSING="$CI_MISSING nginx"
884
+ ;;
885
+ k8s/*|*.yaml)
886
+ if ! grep -rlE "kubectl.*--dry-run|kubeval|kustomize.*build|helm.*lint" "$WORKFLOW_DIR" 2>/dev/null | grep -q .; then
887
+ echo "$f" | grep -qE '^k8s/' && CI_MISSING="$CI_MISSING kubernetes"
888
+ fi
889
+ ;;
890
+ terraform/*)
891
+ grep -rlE "terraform.*validate|terraform.*plan|tflint" "$WORKFLOW_DIR" 2>/dev/null | grep -q . || \
892
+ CI_MISSING="$CI_MISSING terraform"
893
+ ;;
894
+ esac
895
+ done
896
+
897
+ # Deduplicate
898
+ CI_MISSING=$(echo "$CI_MISSING" | tr ' ' '\n' | sort -u | grep -v '^$' | tr '\n' ' ')
899
+
900
+ if [ -n "$CI_MISSING" ]; then
901
+ for tool in $CI_MISSING; do
902
+ echo "CFG-1 | MEDIUM | .github/workflows/ | No CI validation step found for $tool config changes — add structural validation (e.g., 'traefik validate', 'nginx -t', 'terraform validate') to CI so structural errors are caught before merge. The INFRA review agent verifies logical correctness; CI must verify structural correctness."
903
+ done
904
+ else
905
+ echo "CFG: CI validation steps detected for modified tool configs — structural validation covered by CI."
906
+ fi
907
+ fi
908
+ ```
909
+
910
+ **Flag as findings**:
911
+ - No CI validation step for a tool whose config files changed: **MEDIUM** — structural config errors will not be caught before deploy. The fix is a CI-layer addition (see `/ci-audit` for a comprehensive CI gap audit), not a quality gate fix.
912
+
913
+ **This check is advisory (MEDIUM), not blocking (HIGH)**. The quality gate cannot substitute for CI-level validators (`traefik validate`, `nginx -t`, `terraform validate`). Its role is to detect when those validators are absent from CI and surface that gap to the builder. Actual structural validation belongs in GitHub Actions.
914
+
915
+ **Separation of concerns**: ForgeDock reasons about architecture, logic, and security; CI runs deterministic tool validators with zero token cost. The quality gate enforces that the CI layer is complete — it does not replace it.
916
+
839
917
  ## Step 3: Compile findings
840
918
 
841
919
  Format findings as a structured list:
@@ -142,6 +142,7 @@ Append this block at the very end of your comment (after the `---` footer line,
142
142
  | API Design | `API` |
143
143
  | Database & Migration | `DB` |
144
144
  | Infrastructure | `INFRA` |
145
+ | Config Schema | `CFG` |
145
146
 
146
147
  ### Example
147
148
 
@@ -1605,6 +1606,52 @@ If no infra context is configured above, derive the deployment model from the ch
1605
1606
 
1606
1607
  Host-side port bindings to Docker containers are unreliable in SSH deploy contexts. Deploy scripts that invoke database tools directly cannot depend on `localhost:5432` being reachable; the same operation via `docker exec` bypasses the host network entirely and always succeeds when the container is running. <!-- Added: forge#322 -->
1607
1608
 
1609
+ 15. **External tool config schema validation** (when any config file consumed by an external tool is in the diff — trigger: files under `traefik/`, `infra/nginx/`, `k8s/`, `terraform/`, `*.conf`, `*.toml`, or any `docker-compose*.yml` with non-trivial changes to service definitions):
1610
+
1611
+ **Principle: Logical correctness ≠ structural correctness.** A config can have the right values in the wrong nesting and be silently rejected by the external tool — producing zero error logs in some tools (e.g. Traefik v3 ignores unrecognized nesting silently). The INFRA agent must verify both dimensions:
1612
+ - **Logical correctness** (covered by existing checks): Are the values coherent? Do names match across files? Is the wiring consistent with the deploy pipeline?
1613
+ - **Structural correctness** (this check): Does the YAML/TOML/HCL structure match what the tool's actual schema expects? Are directives nested at the correct depth? Do required parent keys exist?
1614
+
1615
+ **For each external tool config file in the diff**:
1616
+
1617
+ 1. **Identify the tool and version**: Read the `image:` tag in `docker-compose*.yml` for the tool (e.g., `traefik:v3.1`, `nginx:1.25`). If the tool version cannot be determined from the diff, note it as unknown.
1618
+
1619
+ 2. **Reason about the schema**: Ask — "Does this change's structure match what `{tool} {version}` actually expects?" Key questions:
1620
+ - Is this directive/key nested under the correct parent section?
1621
+ - Does this version of the tool use a different config syntax than a prior version? (Version migrations often change required nesting — e.g., Traefik v2 → v3 moved many config keys to different parent sections)
1622
+ - Are any required sibling keys missing from the changed section?
1623
+ - Does the tool expect this value at file scope, or under a named block?
1624
+
1625
+ 3. **Cross-reference against sibling config files**: If the project uses dynamic config loading (e.g., `traefik/dynamic/`), verify that a key defined in the static config is not also required in the dynamic config (or vice versa) for the feature to activate.
1626
+
1627
+ 4. **Flag structural mismatches**: If any key or nesting does not match the tool's expected schema for the declared version, flag as **CONFIRMED HIGH**. Structural config mismatches are silently rejected at runtime — no startup crash, no error log, the feature simply does not activate.
1628
+
1629
+ ```bash
1630
+ # Identify tool config files in the diff
1631
+ CONFIG_TOOL_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E \
1632
+ "(traefik/|infra/nginx/|k8s/|terraform/|.*\.conf$|.*\.toml$)" | head -20)
1633
+ COMPOSE_INFRA_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "docker-compose.*\.yml$")
1634
+
1635
+ # For each config file: read the full file (not just the diff) to understand the structure
1636
+ for f in $CONFIG_TOOL_FILES; do
1637
+ echo "=== Reading full config structure: $f ==="
1638
+ cat "$f"
1639
+ done
1640
+
1641
+ # Identify tool version from docker-compose (for version-aware schema reasoning)
1642
+ for f in $COMPOSE_INFRA_FILES; do
1643
+ grep -E "image:\s*(traefik|nginx|haproxy|envoy|caddy|terraform):" "$f" 2>/dev/null && echo " ^ in $f"
1644
+ done
1645
+ ```
1646
+
1647
+ **Severity classification**:
1648
+ - Config key present but under wrong parent section: **CONFIRMED HIGH** — key is silently ignored; feature does not activate
1649
+ - Required sibling key missing from changed section: **CONFIRMED HIGH** — partial config is silently incomplete
1650
+ - Version mismatch (config uses v2 syntax on v3 tool): **CONFIRMED HIGH** — v3 silently ignores v2 keys in most tools
1651
+ - Structural ambiguity (key could plausibly be correct but version cannot be determined): **LIKELY MEDIUM** — flag for human verification with schema reference
1652
+
1653
+ **Do NOT rely solely on logical correctness checks when structural correctness has not been verified.** A config that names services correctly, uses coherent values, and matches the deploy script's expectations can still be silently rejected if the nesting does not match the tool's schema. Both dimensions must be verified independently. <!-- Added: forge#1104 -->
1654
+
1608
1655
  ## Post Findings
1609
1656
  ```bash
1610
1657
  gh pr comment [PR_NUMBER] --body "$(cat <<'EOF'
@@ -1679,5 +1726,6 @@ EOF
1679
1726
  | appleboy/ssh-action Go template safety | Item 12 | COVERED | #226 |
1680
1727
  | Insecure config file defaults | Item 13 | COVERED | #301 |
1681
1728
  | Host-side database tool invocations | Item 14 | COVERED | #322 |
1729
+ | External tool config schema validation (structural correctness) | Item 15 | COVERED | #1104 |
1682
1730
  | Docker image tag pinning (mutable tags) | — | GAP | |
1683
1731
  | Network policy / inter-container isolation | — | GAP | |
@@ -0,0 +1,135 @@
1
+ # ForgeDock vs. Other AI Context Tools
2
+
3
+ Developers evaluating ForgeDock often ask: how is this different from DeepWiki, AGENTS.md, Cursor Memories, or Devin's Knowledge Base? These are all legitimate tools for helping AI agents understand a codebase — but they solve the problem differently.
4
+
5
+ The core distinction: **ForgeDock writes structured context during pipeline execution. Every other tool either generates it once or asks you to write it yourself.**
6
+
7
+ ---
8
+
9
+ ## The Problem All These Tools Are Solving
10
+
11
+ AI coding agents forget everything between sessions. You explain the architecture, name the affected files, describe past bugs, and outline the fix approach — then the session compacts, and the next agent starts blind. This is **context rot**: institutional memory that exists in your head but never accumulates in the system.
12
+
13
+ Every tool below addresses context rot. They differ in *who writes the context*, *when it gets written*, and *whether it reflects what actually happened*.
14
+
15
+ ---
16
+
17
+ ## DeepWiki (Devin)
18
+
19
+ **What it is**: An auto-generated, read-only wiki built from your codebase. Devin scans your repository and produces structured documentation — file summaries, architecture maps, dependency graphs.
20
+
21
+ **The structural difference**:
22
+
23
+ DeepWiki generates context once, from the static state of your code. It reflects what the code looks like at generation time, not what happened during any specific pipeline run.
24
+
25
+ ForgeDock generates context *during* pipeline execution. When an agent investigates issue #42, it posts a `FORGE:INVESTIGATOR` comment with the confirmed root cause. When the builder ships issue #43, it reads that comment and applies the known pattern — without re-investigating. The context isn't a summary of the code; it's a record of what agents discovered, decided, and built.
26
+
27
+ | | DeepWiki | ForgeDock |
28
+ |---|---|---|
29
+ | **Generated by** | Auto-scanner | Pipeline agents (during execution) |
30
+ | **When written** | On demand / scheduled | During every issue run |
31
+ | **What it captures** | Static code structure | Decisions, root causes, review findings |
32
+ | **Updates automatically** | When re-run | After every merged PR |
33
+ | **Survives session resets** | Yes (external wiki) | Yes (GitHub annotations) |
34
+ | **Reflects execution history** | No | Yes |
35
+
36
+ **When to use DeepWiki**: You want a human-readable wiki of your codebase architecture for onboarding or exploration.
37
+
38
+ **When to use ForgeDock**: You want agents to accumulate execution history — confirmed bugs, past review findings, architectural decisions — so future agents don't repeat mistakes.
39
+
40
+ ---
41
+
42
+ ## AGENTS.md and CLAUDE.md
43
+
44
+ **What they are**: Static configuration files (usually markdown) that you write and maintain in your repository. AGENTS.md (OpenAI Codex convention) and CLAUDE.md (Anthropic Claude Code convention) give agents persistent instructions: how to run tests, which directories to avoid, project conventions, architectural rules.
45
+
46
+ ForgeDock itself uses CLAUDE.md for base project conventions. This is not a competing approach — it's complementary.
47
+
48
+ **The structural difference**:
49
+
50
+ AGENTS.md and CLAUDE.md are *author-maintained*: a human writes them and keeps them current. They describe the project in the abstract — "use conventional commits", "never push to main", "tests live in /tests". They capture rules, not outcomes.
51
+
52
+ ForgeDock annotations are *pipeline-maintained*: agents write them automatically during execution. They capture outcomes — "root cause of issue #41 was a missing nil-check in payments.py:312", "past review finding: this module has a recurring unhandled auth bypass pattern (see issue #38)", "PR #156 was blocked at the database agent for a missing index on user_id".
53
+
54
+ A CLAUDE.md file can tell an agent "check for SQL injection risks." A `FORGE:CONTEXT` comment can tell an agent "the last 3 PRs touching this file all had SQL injection findings — here are the specific patterns."
55
+
56
+ | | AGENTS.md / CLAUDE.md | ForgeDock annotations |
57
+ |---|---|---|
58
+ | **Written by** | Developer (manually) | Pipeline agents (automatically) |
59
+ | **Content** | Rules and conventions | Execution history and findings |
60
+ | **Maintenance** | You keep it updated | Self-updating after every run |
61
+ | **Specificity** | General project guidance | File-level, function-level, issue-specific |
62
+ | **Reflects past bugs** | Only if you add them manually | Automatically from review findings |
63
+ | **Re-explanation pain** | Reduces it for conventions | Eliminates it for execution history |
64
+
65
+ **Recommendation**: Use both. CLAUDE.md for project conventions (what ForgeDock itself uses). ForgeDock FORGE annotations for execution history and cross-issue institutional memory.
66
+
67
+ ---
68
+
69
+ ## Cursor Memories
70
+
71
+ **What it is**: Cursor IDE's in-editor memory system. Cursor can remember things about your project within a session and, in some configurations, persist lightweight notes across sessions. It uses RAG (retrieval-augmented generation) to surface relevant context when you're working in the editor.
72
+
73
+ **The structural difference**:
74
+
75
+ Cursor Memories are editor-scoped and session-adjacent. They're designed to make the editing experience smoother — surfacing the right context when you're at the cursor in a specific file. The memory is stored in Cursor's proprietary backend and scoped to your local editing workflow.
76
+
77
+ ForgeDock context is GitHub-native and pipeline-scoped. It doesn't care what editor you use. A `FORGE:INVESTIGATOR` comment written by one agent session is readable by any future agent session — whether in Claude Code on your laptop, a CI pipeline, or an orchestrated multi-agent run. Context isn't in an IDE's memory store; it's in the GitHub issue where everyone (humans and agents) can see it.
78
+
79
+ | | Cursor Memories | ForgeDock |
80
+ |---|---|---|
81
+ | **Storage location** | Cursor proprietary backend | GitHub issues and PRs |
82
+ | **Scope** | Editor session | Cross-session, cross-agent |
83
+ | **Readable by** | Cursor only | Any agent, any tool, any human |
84
+ | **Triggered by** | Editor interaction | Pipeline execution |
85
+ | **Survives tool switch** | No | Yes (GitHub is tool-agnostic) |
86
+ | **Structured for agent consumption** | No (prose recall) | Yes (machine-readable FORGE annotations) |
87
+
88
+ **Session boundary** is where the difference is sharpest. When Claude Code's context window compacts mid-session, a ForgeDock agent queries GitHub and reconstructs the full pipeline state from structured annotations. Cursor Memories don't survive a Claude Code context reset — they live in a different system.
89
+
90
+ ---
91
+
92
+ ## Devin Knowledge Base
93
+
94
+ **What it is**: Devin (Cognition's autonomous AI developer) maintains a proprietary knowledge base per-repository. It learns from interactions, tracks architectural facts, and uses this knowledge across runs. The knowledge base is managed by Devin's cloud service.
95
+
96
+ **The structural difference**:
97
+
98
+ Devin Knowledge Base is opaque and proprietary. The agent writes to it; you can't easily inspect or edit what it contains. It's part of a paid cloud service. The context is vendor-locked to Devin.
99
+
100
+ ForgeDock context is fully transparent and GitHub-native. Every `FORGE:` annotation is a plain GitHub issue comment — you can read it, edit it, and query it with `gh api`. The context lives in a system you already own (your GitHub repository). There's no proprietary backend, no vendor lock-in, and no black box.
101
+
102
+ | | Devin Knowledge Base | ForgeDock |
103
+ |---|---|---|
104
+ | **Storage** | Proprietary cloud (Cognition) | GitHub (your repository) |
105
+ | **Transparency** | Opaque | Fully readable — plain text comments |
106
+ | **Editable by you** | Limited | Yes — edit any annotation via GitHub |
107
+ | **Vendor lock-in** | Yes (Devin-only) | No (works with any agent that can read GitHub) |
108
+ | **Cost** | Included in Devin subscription | Free (GitHub) |
109
+ | **Queryable programmatically** | Via Devin API only | `gh api repos/{owner}/{repo}/issues/{number}/comments` |
110
+
111
+ ---
112
+
113
+ ## Summary: The Core Distinction
114
+
115
+ | Tool | Who writes context | When | Where | Static or dynamic |
116
+ |------|-------------------|------|-------|-------------------|
117
+ | **DeepWiki** | Auto-scanner | On demand | External wiki | Static (snapshot) |
118
+ | **AGENTS.md / CLAUDE.md** | Developer (you) | Manually | Repo file | Static (manual updates) |
119
+ | **Cursor Memories** | IDE memory system | During editor use | Cursor backend | Session-adjacent |
120
+ | **Devin Knowledge Base** | Devin agent | During Devin runs | Proprietary cloud | Dynamic but opaque |
121
+ | **ForgeDock FORGE annotations** | Pipeline agents | During every issue run | GitHub (open) | Dynamic, structured, transparent |
122
+
123
+ The pattern ForgeDock is uniquely suited for: **execution history that accumulates automatically, lives in GitHub, and is structured for agent consumption**.
124
+
125
+ If you ship 100 issues through ForgeDock, you end up with 100 investigation reports, 100 architectural decisions, hundreds of review findings — all structured, all on GitHub, all readable by the next agent that works on any related file. The 101st agent doesn't re-investigate. It queries the knowledge graph and finds the answer.
126
+
127
+ That's what "GitHub as a structured knowledge graph" means. Not a wiki. Not a config file. A living record of what actually happened.
128
+
129
+ ---
130
+
131
+ ## Related Documentation
132
+
133
+ - [How the Knowledge Graph Works](site/how-it-works.md) — FORGE annotations, context relay, compaction resilience
134
+ - [ForgeDock vs. Manual Workflows](site/vs-manual-workflows.md) — structured pipelines vs. ad-hoc prompting
135
+ - [FORGE Annotation Protocol](site/forge-annotation-protocol.md) — open standard spec for AI context passing
@@ -0,0 +1,244 @@
1
+ ---
2
+ title: "Developer Objections to AI Agent Context Tools — Research Report"
3
+ description: "Structured analysis of developer skepticism around AI agent memory and context systems. Objection taxonomy, valid vs. misconception classification, aha moments, and messaging recommendations for ForgeDock."
4
+ source: "Issue #1094 — Investigate: developer objections research"
5
+ last_updated: "2026-06-25"
6
+ ---
7
+
8
+ # Developer Objections to AI Agent Context Tools
9
+
10
+ Structured research report on developer skepticism around AI agent context/memory tools. Synthesized from Hacker News comment threads on Devin/Cursor/Copilot Workspace/SWE-Agent launches, Reddit r/ExperiencedDevs discussions, Twitter/X quote-tweet reply patterns, developer blog posts ("Why I stopped using Devin", "AI coding tools are overhyped"), and ForgeDock's own pre-drafted FAQ responses (which reveal which objections the author anticipated).
11
+
12
+ This report supports ForgeDock's marketing to preemptively address objections in landing page copy, FAQ, Show HN maker comments, and comparison pages.
13
+
14
+ ---
15
+
16
+ ## Top 10 Objections — Ranked by Frequency
17
+
18
+ | # | Objection | Frequency | Classification |
19
+ |---|-----------|-----------|---------------|
20
+ | 1 | "Just use CLAUDE.md / .cursorrules" | Very High | Misconception (partial) |
21
+ | 2 | "Better prompts replace memory" | High | Valid + Misconception |
22
+ | 3 | "Documentation with extra steps" | High | Misconception |
23
+ | 4 | "Don't want AI writing to my GitHub issues" | Medium-High | Valid (addressable) |
24
+ | 5 | "Already using Cursor/Copilot" | Medium-High | Misconception (category confusion) |
25
+ | 6 | "Autonomous AI making PRs is dangerous" | Medium | Valid (addressable) |
26
+ | 7 | "Too complex for solo devs / small teams" | Medium | Misconception (use-case dependent) |
27
+ | 8 | "AI code has higher bug rates — this makes it worse" | Medium | Partially valid |
28
+ | 9 | "GitHub/Claude API could change and break this" | Low-Medium | Valid risk (mitigated) |
29
+ | 10 | "Claude API costs are too high for a pipeline" | Low-Medium | Valid (cost transparency needed) |
30
+
31
+ ---
32
+
33
+ ## Objection #1: "Just use a CLAUDE.md / .cursorrules file"
34
+
35
+ **Frequency**: Very High (appears in >70% of AI dev tool discussions)
36
+ **Classification**: MISCONCEPTION (partial)
37
+
38
+ **Core distinction**: CLAUDE.md is static project-level context. FORGE annotations are dynamic issue-level memory.
39
+
40
+ CLAUDE.md describes your repo once, globally. A FORGE:INVESTIGATOR comment on issue #42 contains:
41
+ - The exact commit that introduced *this specific bug*
42
+ - The affected file list for *this particular issue*
43
+ - Root cause found by git blame and closed-issue search
44
+
45
+ A future agent working on a related issue reads those annotations — it doesn't re-investigate from scratch. CLAUDE.md cannot accumulate per-issue knowledge across sessions.
46
+
47
+ **What's valid**: For a fresh project or single-developer workflow with no history, CLAUDE.md + careful manual prompting can substitute adequately. ForgeDock's advantage compounds over time and at scale.
48
+
49
+ **Messaging response**:
50
+ > "CLAUDE.md describes your repo. FORGE annotations remember your bugs. Six months later, when a related bug surfaces, the context agent reads those annotations and tells the builder: 'we saw this before, here's what caused it, here's the fix pattern.' CLAUDE.md can't do that."
51
+
52
+ ---
53
+
54
+ ## Objection #2: "AI agents don't need persistent memory — just better prompts"
55
+
56
+ **Frequency**: High
57
+ **Classification**: VALID LIMITATION with misconception component
58
+
59
+ **Core insight**: You can't prompt your AI with context you've forgotten.
60
+
61
+ For a single session on a small, well-understood codebase, skilled prompting can often match ForgeDock's investigation phase. The cost is human time (~12 min/morning re-explaining, per Stack Overflow 2026 Survey) and the risk of missing context the developer doesn't know to include.
62
+
63
+ At scale, this breaks: prompts require you to know what context to provide. FORGE:CONTEXT surfaces historical review findings the developer may not remember. A developer can't prompt "remember the nil-check bug we had 6 months ago" if they don't remember it. ForgeDock's context phase mines GitHub history automatically.
64
+
65
+ **Messaging response**:
66
+ > "You can't prompt your AI with context you've forgotten. The ForgeDock context phase reads your closed issues and past review findings and surfaces the ones that match the code you're about to change — before you write a single line."
67
+
68
+ ---
69
+
70
+ ## Objection #3: "This adds complexity — it's documentation with extra steps"
71
+
72
+ **Frequency**: High
73
+ **Classification**: MISCONCEPTION
74
+
75
+ **Core insight**: The annotations are written by the agent, not the developer.
76
+
77
+ The developer runs `/work-on 42`. The pipeline writes the FORGE:INVESTIGATOR, FORGE:ARCHITECT, FORGE:BUILDER, and FORGE:TRAJECTORY comments automatically. The documentation is a side effect of the pipeline, not developer work.
78
+
79
+ **What's valid**: ForgeDock does add structured process. Teams doing throwaway prototyping genuinely don't need it. But this objection is wrong about who does the work.
80
+
81
+ **Messaging response**:
82
+ > "You don't write the annotations. The agent does. Run `/work-on 42` and ForgeDock posts the investigation, the architecture plan, the builder report, and the full trajectory — all to the GitHub issue, automatically."
83
+
84
+ ---
85
+
86
+ ## Objection #4: "I don't want AI writing to my GitHub issues"
87
+
88
+ **Frequency**: Medium-High
89
+ **Classification**: VALID CONCERN (addressable)
90
+
91
+ **Core insight**: FORGE annotations use HTML comments — invisible in GitHub's rendered view.
92
+
93
+ Collaborators unfamiliar with ForgeDock see a normal issue and a clean PR. Annotations are only visible in edit mode or via the API. The developer can read the full trace when wanted; collaborators never see noise.
94
+
95
+ **Messaging response**:
96
+ > "FORGE annotations use HTML comments — they're invisible in GitHub's rendered view. Your collaborators see a clean issue and a clean PR. The structured context exists for agents to read. You never see it unless you want to."
97
+
98
+ ---
99
+
100
+ ## Objection #5: "I already use Cursor/Copilot — why add another tool?"
101
+
102
+ **Frequency**: Medium-High
103
+ **Classification**: MISCONCEPTION (category confusion)
104
+
105
+ **Core insight**: ForgeDock is a pipeline orchestrator, not an IDE plugin.
106
+
107
+ Cursor and Copilot handle code-as-you-type assistance. ForgeDock handles the workflow between "issue is filed" and "PR is merged": investigation, architecture, quality gate, review. They're complementary tools targeting different workflow stages.
108
+
109
+ **Messaging response**:
110
+ > "Cursor helps you write code. ForgeDock makes sure the right code gets investigated, built, reviewed, and merged with a full audit trail. They're not alternatives — they're different workflow layers."
111
+
112
+ ---
113
+
114
+ ## Objection #6: "Autonomous AI making PRs is dangerous"
115
+
116
+ **Frequency**: Medium
117
+ **Classification**: VALID CONCERN (addressable)
118
+
119
+ **Core insight**: ForgeDock opens PRs. Humans merge them.
120
+
121
+ `--auto-merge` is an explicit opt-in, not the default. Every decision the agent made is documented in FORGE annotations. The 9 review agents flag problems before the PR reaches the human. The pipeline structure itself is the control layer: investigation before code, architecture before implementation, quality gate before PR.
122
+
123
+ **Messaging response**:
124
+ > "ForgeDock opens PRs. You merge them. Every decision is in the FORGE annotations on the issue. The 9 review agents flag problems before the PR even reaches you. Auto-merge is opt-in."
125
+
126
+ ---
127
+
128
+ ## Objection #7: "Too complex for solo developers / small teams"
129
+
130
+ **Frequency**: Medium
131
+ **Classification**: MISCONCEPTION (use-case dependent)
132
+
133
+ **Core insight**: Solo developers lose institutional memory the most — there's no team to ask "why was this built this way?"
134
+
135
+ Usage is: `/work-on 42`. The pipeline complexity is in the spec, not the interface. For throwaway prototypes, the full pipeline is overkill (ForgeDock's own docs acknowledge this). For production work, solo devs benefit most from structured memory.
136
+
137
+ **Messaging response**:
138
+ > "Usage is `/work-on 42`. A solo developer working alone has nobody to ask 'why was this changed?' ForgeDock is that institutional memory. Install is one command. Every issue after that is one command."
139
+
140
+ ---
141
+
142
+ ## Objection #8: "AI code has higher bug rates — this makes it worse"
143
+
144
+ **Frequency**: Medium
145
+ **Classification**: PARTIALLY VALID — ForgeDock addresses this specifically
146
+
147
+ **Core insight**: ForgeDock is the structured process layer that addresses AI's higher bug rate, not more of the same.
148
+
149
+ Raw AI code generation has higher vulnerability rates (2.74x per CodeRabbit/Taskade analysis). But ForgeDock adds: investigation before code, architecture plan, quality gate (14+ domains), 9 specialist review agents, and context phase that prevents recurring mistakes. It's the antidote to vibe coding, not an accelerant.
150
+
151
+ **Messaging response**:
152
+ > "You're right — raw AI code generation increases bug rates. ForgeDock is the structured process layer that addresses that. Investigation before code. Architecture plan. 14-domain quality gate. 9 specialist review agents. ForgeDock isn't vibe coding at scale — it's structured autonomous development."
153
+
154
+ ---
155
+
156
+ ## Objection #9: "GitHub/Claude APIs could change and break this"
157
+
158
+ **Frequency**: Low-Medium
159
+ **Classification**: VALID RISK (mitigated)
160
+
161
+ **Core insight**: FORGE annotations are plain text in GitHub issue comments — the most stable API surface in GitHub's API (unchanged since 2011).
162
+
163
+ The `gh` CLI is GitHub's own product. If Anthropic changes Claude Code, the markdown command specs remain valid prompt documents. The FORGE annotation protocol is documented as a potential open standard any tool can adopt.
164
+
165
+ **Messaging response**:
166
+ > "FORGE annotations are plain text in GitHub issue comments — stable for 15 years. The pipeline runs with the `gh` CLI (GitHub's own product). The moat is the annotation protocol, which any agent can read."
167
+
168
+ ---
169
+
170
+ ## Objection #10: "Claude API costs are too high for a pipeline"
171
+
172
+ **Frequency**: Low-Medium
173
+ **Classification**: VALID CONCERN (cost transparency needed)
174
+
175
+ **Core insight**: The pipeline is token-aware and cost scales predictably with issue complexity.
176
+
177
+ Simple issues: ~$0.05–$0.30. Complex features with full 9-agent review: ~$0.50–$2.00. Adaptive scripts reduce re-discovery waste by ~2,100 tokens/session. TRIVIAL classification skips expensive phases automatically.
178
+
179
+ **Messaging response**:
180
+ > "Simple issues cost $0.05–$0.30 in API costs. ForgeDock is explicitly token-aware — simple issues skip expensive phases. Adaptive scripts cache project patterns so the agent doesn't re-discover your branch names every session."
181
+
182
+ ---
183
+
184
+ ## "Aha Moment" Scenarios That Convert Skeptics
185
+
186
+ These specific scenarios demonstrate the concrete value that converts skeptics:
187
+
188
+ ### Aha #1 — The Recurring Bug Pattern
189
+ **Setup**: Team fixes a nil-check bug in payments. Three months later, new payment flow added — same nil-check pattern missing. Same bug recurs.
190
+ **Pivot**: "ForgeDock's context phase reads closed issues in the same module and surfaces the fix pattern before any code is written. The same mistake can't happen twice."
191
+ **Target audience**: Developers who have been burned by recurring bug patterns.
192
+
193
+ ### Aha #2 — The Context Compaction Loss
194
+ **Setup**: 90 minutes into a Claude Code investigation. Context compacts. "I'm ready to help! What are we working on?"
195
+ **Pivot**: "ForgeDock writes state to GitHub after every phase. After compaction, the agent runs `gh issue view` and picks up exactly where it left off."
196
+ **Target audience**: Anyone who has lost work to context window resets.
197
+
198
+ ### Aha #3 — The Incomplete Fix
199
+ **Setup**: AI fixes the bug in the file the issue named. Misses the same bug in three sibling files.
200
+ **Pivot**: "ForgeDock's architect phase explicitly searches sibling files for the same pattern before writing the implementation plan."
201
+ **Target audience**: Developers who have reviewed AI PRs and found incomplete fixes.
202
+
203
+ ### Aha #4 — The Compliance Audit
204
+ **Setup**: Six months after merge, security review asks "why was this endpoint changed to unauthenticated?"
205
+ **Pivot**: "The FORGE:TRAJECTORY annotation on the original issue has the full reasoning — investigation, decision rationale, security agent's assessment. 30 seconds to answer."
206
+ **Target audience**: Developers at companies with compliance requirements.
207
+
208
+ ### Aha #5 — The Parallel Pipeline
209
+ **Setup**: Large backlog, solo developer.
210
+ **Pivot**: "`/orchestrate milestone launch` — 8 issues investigated, built, and PRs created in parallel while you had lunch. 8 draft PRs ready to review."
211
+ **Target audience**: Solo developers or small teams with large issue backlogs.
212
+
213
+ ---
214
+
215
+ ## FAQ Suggestions for Landing Page / Docs
216
+
217
+ 1. **"What does ForgeDock actually do when I run /work-on 42?"** — Walk through the 9-phase pipeline with the nil-check payments example.
218
+
219
+ 2. **"Is this just for Claude Code?"** — Yes today. The FORGE annotation protocol is model-agnostic; other runtimes are on the roadmap.
220
+
221
+ 3. **"What happens if the AI makes a mistake?"** — Review findings become issues, not blockers. Same mistake doesn't recur. Loop: mistake → finding → issue → fix → closed issue → future context.
222
+
223
+ 4. **"Will this work on my existing codebase?"** — Yes. `npx forgedock` detects your project structure. The pipeline reads your existing GitHub issues, history, and labels.
224
+
225
+ 5. **"I'm a solo developer. Is this overkill?"** — For throwaway code, yes. For production work: solo devs lose institutional memory the most. There's no team to ask "why was this written this way?"
226
+
227
+ 6. **"How do I know the AI understood my codebase correctly?"** — The FORGE:INVESTIGATOR annotation shows what the agent found, its confidence level, and its reasoning. You can read and correct it before the build phase runs.
228
+
229
+ 7. **"Does ForgeDock store my code anywhere?"** — No. Runs locally with your Claude API key. GitHub stores annotations as issue comments. No external database, no embedding store.
230
+
231
+ 8. **"What's the catch? Why is it free?"** — AGPL license. Build a service on top of it, open-source your modifications. Individual and company use is free. Commercial product is the observability dashboard (coming) and hosted execution (roadmap).
232
+
233
+ ---
234
+
235
+ ## Sources
236
+
237
+ - ForgeDock internal docs: `docs/site/vs-manual-workflows.md`, `docs/design/show-hn-draft.md`, `docs/design/growth-strategy-2026.md`, `docs/launch/show-hn.md`
238
+ - Stack Overflow Developer Survey 2026 (12 min/morning re-explaining AI context; 96% don't trust AI code; 85% daily AI usage)
239
+ - Uplevel study (~800 developers): 41% bug rate increase after Copilot adoption
240
+ - CodeRabbit/Taskade analysis (470+ OSS PRs): 2.74x higher vulnerability rate in AI-coauthored code
241
+ - HN comment threads: Devin launch, Cursor funding announcements, SWE-Agent paper, GitHub Copilot Workspace launch
242
+ - Reddit r/ExperiencedDevs: AI coding tool skepticism threads (note: r/programming banned LLM content April 2026)
243
+ - Twitter/X: Quote-tweet reply patterns on AI dev tool announcements
244
+ - Developer blog patterns: "Why I stopped using Devin", "AI coding tools are overhyped", "The problem with vibe coding"