@datafog/fogclaw 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/harness-docs.yml +30 -0
- package/AGENTS.md +28 -0
- package/LICENSE +21 -0
- package/README.md +208 -0
- package/dist/config.d.ts +4 -0
- package/dist/config.d.ts.map +1 -0
- package/dist/config.js +30 -0
- package/dist/config.js.map +1 -0
- package/dist/engines/gliner.d.ts +14 -0
- package/dist/engines/gliner.d.ts.map +1 -0
- package/dist/engines/gliner.js +75 -0
- package/dist/engines/gliner.js.map +1 -0
- package/dist/engines/regex.d.ts +5 -0
- package/dist/engines/regex.d.ts.map +1 -0
- package/dist/engines/regex.js +54 -0
- package/dist/engines/regex.js.map +1 -0
- package/dist/index.d.ts +19 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +157 -0
- package/dist/index.js.map +1 -0
- package/dist/redactor.d.ts +3 -0
- package/dist/redactor.d.ts.map +1 -0
- package/dist/redactor.js +37 -0
- package/dist/redactor.js.map +1 -0
- package/dist/scanner.d.ts +11 -0
- package/dist/scanner.d.ts.map +1 -0
- package/dist/scanner.js +77 -0
- package/dist/scanner.js.map +1 -0
- package/dist/types.d.ts +31 -0
- package/dist/types.d.ts.map +1 -0
- package/dist/types.js +18 -0
- package/dist/types.js.map +1 -0
- package/docs/DATA.md +28 -0
- package/docs/DESIGN.md +17 -0
- package/docs/DOMAIN_DOCS.md +30 -0
- package/docs/FRONTEND.md +24 -0
- package/docs/OBSERVABILITY.md +25 -0
- package/docs/PLANS.md +171 -0
- package/docs/PRODUCT_SENSE.md +20 -0
- package/docs/RELIABILITY.md +60 -0
- package/docs/SECURITY.md +50 -0
- package/docs/design-docs/core-beliefs.md +17 -0
- package/docs/design-docs/index.md +8 -0
- package/docs/generated/README.md +36 -0
- package/docs/generated/memory.md +1 -0
- package/docs/plans/2026-02-16-fogclaw-design.md +172 -0
- package/docs/plans/2026-02-16-fogclaw-implementation.md +1606 -0
- package/docs/plans/README.md +15 -0
- package/docs/plans/active/2026-02-16-feat-openclaw-official-submission-plan.md +386 -0
- package/docs/plans/active/2026-02-17-feat-release-fogclaw-via-datafog-package-plan.md +318 -0
- package/docs/plans/active/2026-02-17-feat-submit-fogclaw-to-openclaw-plan.md +244 -0
- package/docs/plans/tech-debt-tracker.md +42 -0
- package/docs/plugins/fogclaw.md +95 -0
- package/docs/runbooks/address-review-findings.md +30 -0
- package/docs/runbooks/ci-failures.md +46 -0
- package/docs/runbooks/code-review.md +34 -0
- package/docs/runbooks/merge-change.md +28 -0
- package/docs/runbooks/pull-request.md +45 -0
- package/docs/runbooks/record-evidence.md +43 -0
- package/docs/runbooks/reproduce-bug.md +42 -0
- package/docs/runbooks/respond-to-feedback.md +42 -0
- package/docs/runbooks/review-findings.md +31 -0
- package/docs/runbooks/submit-openclaw-plugin.md +68 -0
- package/docs/runbooks/update-agents-md.md +59 -0
- package/docs/runbooks/update-domain-docs.md +42 -0
- package/docs/runbooks/validate-current-state.md +41 -0
- package/docs/runbooks/verify-release.md +69 -0
- package/docs/specs/2026-02-16-feat-openclaw-official-submission-spec.md +115 -0
- package/docs/specs/2026-02-17-feat-submit-fogclaw-to-openclaw.md +125 -0
- package/docs/specs/README.md +5 -0
- package/docs/specs/index.md +8 -0
- package/docs/spikes/README.md +8 -0
- package/fogclaw.config.example.json +15 -0
- package/openclaw.plugin.json +45 -0
- package/package.json +37 -0
- package/scripts/ci/he-docs-config.json +123 -0
- package/scripts/ci/he-docs-drift.sh +112 -0
- package/scripts/ci/he-docs-lint.sh +234 -0
- package/scripts/ci/he-plans-lint.sh +354 -0
- package/scripts/ci/he-runbooks-lint.sh +445 -0
- package/scripts/ci/he-specs-lint.sh +258 -0
- package/scripts/ci/he-spikes-lint.sh +249 -0
- package/scripts/runbooks/select-runbooks.sh +154 -0
- package/src/config.ts +46 -0
- package/src/engines/gliner.ts +88 -0
- package/src/engines/regex.ts +71 -0
- package/src/index.ts +223 -0
- package/src/redactor.ts +51 -0
- package/src/scanner.ts +90 -0
- package/src/types.ts +52 -0
- package/tests/config.test.ts +104 -0
- package/tests/gliner.test.ts +184 -0
- package/tests/plugin-smoke.test.ts +114 -0
- package/tests/redactor.test.ts +320 -0
- package/tests/regex.test.ts +345 -0
- package/tests/scanner.test.ts +199 -0
- package/tsconfig.json +20 -0
package/docs/PLANS.md
ADDED
|
@@ -0,0 +1,171 @@
|
|
|
1
|
+
# Agent Plans:
|
|
2
|
+
|
|
3
|
+
This document describes the requirements for a plan ("Plan"), a design document that a coding agent can follow to deliver a working feature or system change. Treat the reader as a complete beginner to this repository: they have only the current working tree and the single Plan file you provide. There is no memory of prior plans and no external context.
|
|
4
|
+
|
|
5
|
+
## How to use Plans and PLANS.md
|
|
6
|
+
|
|
7
|
+
When authoring an executable specification (Plan), follow PLANS.md _to the letter_. If it is not in your context, refresh your memory by reading the entire PLANS.md file. Be thorough in reading (and re-reading) source material to produce an accurate specification. When creating a spec, start from the skeleton and flesh it out as you do your research.
|
|
8
|
+
|
|
9
|
+
When implementing an executable specification (Plan), do not prompt the user for "next steps"; simply proceed to the next milestone. Keep all sections up to date, add or split entries in the list at every stopping point to affirmatively state the progress made and next steps. Resolve ambiguities autonomously, and commit frequently.
|
|
10
|
+
|
|
11
|
+
When discussing an executable specification (Plan), record decisions in a log in the spec for posterity; it should be unambiguously clear why any change to the specification was made. Plans are living documents, and it should always be possible to restart from _only_ the Plan and no other work.
|
|
12
|
+
|
|
13
|
+
When researching a design with challenging requirements or significant unknowns, use milestones to implement proof of concepts, "toy implementations", etc., that allow validating whether the user's proposal is feasible. Read the source code of libraries by finding or acquiring them, research deeply, and include prototypes to guide a fuller implementation.
|
|
14
|
+
|
|
15
|
+
## Requirements
|
|
16
|
+
|
|
17
|
+
NON-NEGOTIABLE REQUIREMENTS:
|
|
18
|
+
|
|
19
|
+
* Every Plan must be fully self-contained. Self-contained means that in its current form it contains all knowledge and instructions needed for a novice to succeed.
|
|
20
|
+
* Every Plan is a living document. Contributors are required to revise it as progress is made, as discoveries occur, and as design decisions are finalized. Each revision must remain fully self-contained.
|
|
21
|
+
* Every Plan must enable a complete novice to implement the feature end-to-end without prior knowledge of this repo.
|
|
22
|
+
* Every Plan must produce a demonstrably working behavior, not merely code changes to "meet a definition".
|
|
23
|
+
* Every Plan must define every term of art in plain language or do not use it.
|
|
24
|
+
|
|
25
|
+
Purpose and intent come first. Begin by explaining, in a few sentences, why the work matters from a user's perspective: what someone can do after this change that they could not do before, and how to see it working. Then guide the reader through the exact steps to achieve that outcome, including what to edit, what to run, and what they should observe.
|
|
26
|
+
|
|
27
|
+
The agent executing your plan can list files, read files, search, run the project, and run tests. It does not know any prior context and cannot infer what you meant from earlier milestones. Repeat any assumption you rely on. Do not point to external blogs or docs; if knowledge is required, embed it in the plan itself in your own words. If a Plan builds upon a prior Plan and that file is checked in, incorporate it by reference. If it is not, you must include all relevant context from that plan.
|
|
28
|
+
|
|
29
|
+
## Formatting
|
|
30
|
+
|
|
31
|
+
Format and envelope are simple and strict. Each Plan must be one single fenced code block labeled as `md` that begins and ends with triple backticks. Do not nest additional triple-backtick code fences inside; when you need to show commands, transcripts, diffs, or code, present them as indented blocks within that single fence. Use indentation for clarity rather than code fences inside a Plan to avoid prematurely closing the Plan's code fence. Use two newlines after every heading, use # and ## and so on, and correct syntax for ordered and unordered lists.
|
|
32
|
+
|
|
33
|
+
When writing a Plan to a Markdown (.md) file where the content of the file *is only* the single Plan, you should omit the triple backticks.
|
|
34
|
+
|
|
35
|
+
Write in plain prose. Prefer sentences over lists. Avoid checklists, tables, and long enumerations unless brevity would obscure meaning. Checklists are permitted only in the `Progress` section, where they are mandatory. Narrative sections must remain prose-first.
|
|
36
|
+
|
|
37
|
+
## Guidelines
|
|
38
|
+
|
|
39
|
+
Self-containment and plain language are paramount. If you introduce a phrase that is not ordinary English ("daemon", "middleware", "RPC gateway", "filter graph"), define it immediately and remind the reader how it manifests in this repository (for example, by naming the files or commands where it appears). Do not say "as defined previously" or "according to the architecture doc." Include the needed explanation here, even if you repeat yourself.
|
|
40
|
+
|
|
41
|
+
Avoid common failure modes. Do not rely on undefined jargon. Do not describe "the letter of a feature" so narrowly that the resulting code compiles but does nothing meaningful. Do not outsource key decisions to the reader. When ambiguity exists, resolve it in the plan itself and explain why you chose that path. Err on the side of over-explaining user-visible effects and under-specifying incidental implementation details.
|
|
42
|
+
|
|
43
|
+
Anchor the plan with observable outcomes. State what the user can do after implementation, the commands to run, and the outputs they should see. Acceptance should be phrased as behavior a human can verify ("after starting the server, navigating to [http://localhost:8080/health](http://localhost:8080/health) returns HTTP 200 with body OK") rather than internal attributes ("added a HealthCheck struct"). If a change is internal, explain how its impact can still be demonstrated (for example, by running tests that fail before and pass after, and by showing a scenario that uses the new behavior).
|
|
44
|
+
|
|
45
|
+
Specify repository context explicitly. Name files with full repository-relative paths, name functions and modules precisely, and describe where new files should be created. If touching multiple areas, include a short orientation paragraph that explains how those parts fit together so a novice can navigate confidently. When running commands, show the working directory and exact command line. When outcomes depend on environment, state the assumptions and provide alternatives when reasonable.
|
|
46
|
+
|
|
47
|
+
Be idempotent and safe. Write the steps so they can be run multiple times without causing damage or drift. If a step can fail halfway, include how to retry or adapt. If a migration or destructive operation is necessary, spell out backups or safe fallbacks. Prefer additive, testable changes that can be validated as you go.
|
|
48
|
+
|
|
49
|
+
Validation is not optional. Include instructions to run tests, to start the system if applicable, and to observe it doing something useful. Describe comprehensive testing for any new features or capabilities. Include expected outputs and error messages so a novice can tell success from failure. Where possible, show how to prove that the change is effective beyond compilation (for example, through a small end-to-end scenario, a CLI invocation, or an HTTP request/response transcript). State the exact test commands appropriate to the project’s toolchain and how to interpret their results.
|
|
50
|
+
|
|
51
|
+
Capture evidence. When your steps produce terminal output, short diffs, or logs, include them inside the single fenced block as indented examples. Keep them concise and focused on what proves success. If you need to include a patch, prefer file-scoped diffs or small excerpts that a reader can recreate by following your instructions rather than pasting large blobs.
|
|
52
|
+
|
|
53
|
+
## Milestones
|
|
54
|
+
|
|
55
|
+
Milestones are narrative, not bureaucracy. If you break the work into milestones, introduce each with a brief paragraph that describes the scope, what will exist at the end of the milestone that did not exist before, the commands to run, and the acceptance you expect to observe. Keep it readable as a story: goal, work, result, proof. Progress and milestones are distinct: milestones tell the story, progress tracks granular work. Both must exist. Never abbreviate a milestone merely for the sake of brevity, do not leave out details that could be crucial to a future implementation.
|
|
56
|
+
|
|
57
|
+
Each milestone must be independently verifiable and incrementally implement the overall goal of the plan.
|
|
58
|
+
|
|
59
|
+
## Living plans and design decisions
|
|
60
|
+
|
|
61
|
+
* Plans are living documents. As you make key design decisions, update the plan to record both the decision and the thinking behind it. Record all decisions in the `Decision Log` section.
|
|
62
|
+
* Plans must contain and maintain a `Progress` section, a `Surprises & Discoveries` section, a `Decision Log`, and an `Outcomes & Retrospective` section. These are not optional.
|
|
63
|
+
* When you discover optimizer behavior, performance tradeoffs, unexpected bugs, or inverse/unapply semantics that shaped your approach, capture those observations in the `Surprises & Discoveries` section with short evidence snippets (test output is ideal).
|
|
64
|
+
* If you change course mid-implementation, document why in the `Decision Log` and reflect the implications in `Progress`. Plans are guides for the next contributor as much as checklists for you.
|
|
65
|
+
* At completion of a major task or the full plan, write an `Outcomes & Retrospective` entry summarizing what was achieved, what remains, and lessons learned.
|
|
66
|
+
* Plans must include explicit workflow handoff sections so later phases have a stable contract:
|
|
67
|
+
* `## Pull Request` (populated by PR-opening workflow)
|
|
68
|
+
* `## Review Findings` (populated by `he-review`)
|
|
69
|
+
* `## Verify/Release Decision` (populated by `he-verify-release`)
|
|
70
|
+
|
|
71
|
+
# Prototyping milestones and parallel implementations
|
|
72
|
+
|
|
73
|
+
It is acceptable—-and often encouraged—-to include explicit prototyping milestones when they de-risk a larger change. Examples: adding a low-level operator to a dependency to validate feasibility, or exploring two composition orders while measuring optimizer effects. Keep prototypes additive and testable. Clearly label the scope as “prototyping”; describe how to run and observe results; and state the criteria for promoting or discarding the prototype.
|
|
74
|
+
|
|
75
|
+
Prefer additive code changes followed by subtractions that keep tests passing. Parallel implementations (e.g., keeping an adapter alongside an older path during migration) are fine when they reduce risk or enable tests to continue passing during a large migration. Describe how to validate both paths and how to retire one safely with tests. When working with multiple new libraries or feature areas, consider creating spikes that evaluate the feasibility of these features _independently_ of one another, proving that the external library performs as expected and implements the features we need in isolation.
|
|
76
|
+
|
|
77
|
+
## Skeleton of a Good Plan
|
|
78
|
+
|
|
79
|
+
# <Short, action-oriented description>
|
|
80
|
+
|
|
81
|
+
This Plan is a living document. The sections `Progress`, `Surprises & Discoveries`, `Decision Log`, and `Outcomes & Retrospective` must be kept up to date as work proceeds.
|
|
82
|
+
|
|
83
|
+
If PLANS.md file is checked into the repo, reference the path to that file here from the repository root and note that this document must be maintained in accordance with PLANS.md.
|
|
84
|
+
|
|
85
|
+
## Purpose / Big Picture
|
|
86
|
+
|
|
87
|
+
Explain in a few sentences what someone gains after this change and how they can see it working. State the user-visible behavior you will enable.
|
|
88
|
+
|
|
89
|
+
## Progress
|
|
90
|
+
|
|
91
|
+
Use a list with checkboxes to summarize granular steps. Every stopping point must be documented here, even if it requires splitting a partially completed task into two (“done” vs. “remaining”). This section must always reflect the actual current state of the work.
|
|
92
|
+
|
|
93
|
+
- [x] (2025-10-01 13:00Z) Example completed step.
|
|
94
|
+
- [ ] Example incomplete step.
|
|
95
|
+
- [ ] Example partially completed step (completed: X; remaining: Y).
|
|
96
|
+
|
|
97
|
+
Use timestamps to measure rates of progress.
|
|
98
|
+
|
|
99
|
+
## Surprises & Discoveries
|
|
100
|
+
|
|
101
|
+
Document unexpected behaviors, bugs, optimizations, or insights discovered during implementation. Provide concise evidence.
|
|
102
|
+
|
|
103
|
+
- Observation: …
|
|
104
|
+
Evidence: …
|
|
105
|
+
|
|
106
|
+
## Decision Log
|
|
107
|
+
|
|
108
|
+
Record every decision made while working on the plan in the format:
|
|
109
|
+
|
|
110
|
+
- Decision: …
|
|
111
|
+
Rationale: …
|
|
112
|
+
Date/Author: …
|
|
113
|
+
|
|
114
|
+
## Outcomes & Retrospective
|
|
115
|
+
|
|
116
|
+
Summarize outcomes, gaps, and lessons learned at major milestones or at completion. Compare the result against the original purpose.
|
|
117
|
+
|
|
118
|
+
## Context and Orientation
|
|
119
|
+
|
|
120
|
+
Describe the current state relevant to this task as if the reader knows nothing. Name the key files and modules by full path. Define any non-obvious term you will use. Do not refer to prior plans.
|
|
121
|
+
|
|
122
|
+
## Plan of Work
|
|
123
|
+
|
|
124
|
+
Describe, in prose, the sequence of edits and additions. For each edit, name the file and location (function, module) and what to insert or change. Keep it concrete and minimal.
|
|
125
|
+
|
|
126
|
+
## Concrete Steps
|
|
127
|
+
|
|
128
|
+
State the exact commands to run and where to run them (working directory). When a command generates output, show a short expected transcript so the reader can compare. This section must be updated as work proceeds.
|
|
129
|
+
|
|
130
|
+
## Validation and Acceptance
|
|
131
|
+
|
|
132
|
+
Describe how to start or exercise the system and what to observe. Phrase acceptance as behavior, with specific inputs and outputs. If tests are involved, say "run <project’s test command> and expect <N> passed; the new test <name> fails before the change and passes after>".
|
|
133
|
+
|
|
134
|
+
## Idempotence and Recovery
|
|
135
|
+
|
|
136
|
+
If steps can be repeated safely, say so. If a step is risky, provide a safe retry or rollback path. Keep the environment clean after completion.
|
|
137
|
+
|
|
138
|
+
## Artifacts and Notes
|
|
139
|
+
|
|
140
|
+
Include the most important transcripts, diffs, or snippets as indented examples. Keep them concise and focused on what proves success.
|
|
141
|
+
|
|
142
|
+
## Interfaces and Dependencies
|
|
143
|
+
|
|
144
|
+
Be prescriptive. Name the libraries, modules, and services to use and why. Specify the types, traits/interfaces, and function signatures that must exist at the end of the milestone. Prefer stable names and paths such as `crate::module::function` or `package.submodule.Interface`. E.g.:
|
|
145
|
+
|
|
146
|
+
In crates/foo/planner.rs, define:
|
|
147
|
+
|
|
148
|
+
pub trait Planner {
|
|
149
|
+
fn plan(&self, observed: &Observed) -> Vec<Action>;
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
## Pull Request
|
|
153
|
+
|
|
154
|
+
This section is the stable handoff contract to the PR/CI phase. Record:
|
|
155
|
+
|
|
156
|
+
- pr: <url>
|
|
157
|
+
- branch:
|
|
158
|
+
- commit:
|
|
159
|
+
- ci: <checks link or summary>
|
|
160
|
+
|
|
161
|
+
## Review Findings
|
|
162
|
+
|
|
163
|
+
Populated by review workflow (e.g. `he-review`). Consolidate findings here with priorities and locations.
|
|
164
|
+
|
|
165
|
+
## Verify/Release Decision
|
|
166
|
+
|
|
167
|
+
Populated by verify/release workflow (e.g. `he-verify-release`). Record GO/NO-GO plus evidence and rollback.
|
|
168
|
+
|
|
169
|
+
If you follow the guidance above, a single, stateless agent -- or a human novice -- can read your Plan from top to bottom and produce a working, observable result. That is the bar: SELF-CONTAINED, SELF-SUFFICIENT, NOVICE-GUIDING, OUTCOME-FOCUSED.
|
|
170
|
+
|
|
171
|
+
When you revise a plan, you must ensure your changes are comprehensively reflected across all sections, including the living document sections, and you must write a note at the bottom of the plan describing the change and the reason why. Plans must describe not just the what but the why for almost everything.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Product Sense"
|
|
3
|
+
use_when: "Capturing target users, success outcomes, decision heuristics, and quality criteria for this repo."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
## Target Users
|
|
7
|
+
- Name the primary user and the primary job-to-be-done; list any secondary users explicitly.
|
|
8
|
+
- Call out non-users (who this is not for) to reduce scope creep.
|
|
9
|
+
|
|
10
|
+
## Key Outcomes
|
|
11
|
+
- Define 1-3 outcomes that matter and how you will measure them (even if qualitative).
|
|
12
|
+
- Prefer metrics tied to user time, reliability, and task completion.
|
|
13
|
+
|
|
14
|
+
## Decision Heuristics
|
|
15
|
+
- Prefer shipping a smaller, complete slice over a broad, partial feature.
|
|
16
|
+
- Optimize for reducing user effort and reducing operational burden.
|
|
17
|
+
|
|
18
|
+
## Quality Criteria
|
|
19
|
+
- Clear error messages and recovery paths; no silent failures.
|
|
20
|
+
- Sensible defaults and empty states; predictable navigation.
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Reliability"
|
|
3
|
+
use_when: "Capturing reliability goals, failure modes, and operational guardrails for this repo."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
## Reliability Goals
|
|
7
|
+
|
|
8
|
+
This plugin should behave predictably for two success paths: (1) full detection flow with GLiNER available and (2) degraded mode with regex-only detection when GLiNER cannot initialize.
|
|
9
|
+
|
|
10
|
+
Primary targets:
|
|
11
|
+
|
|
12
|
+
- Plugin registration should succeed on a clean build and load via OpenClaw extension loading.
|
|
13
|
+
- Guardrail hook should never throw uncaught errors to the request path.
|
|
14
|
+
- Tool responses should be deterministic for the same input and schema.
|
|
15
|
+
- Degraded flow should preserve baseline privacy protection via regex and continue processing traffic.
|
|
16
|
+
|
|
17
|
+
## Failure Modes
|
|
18
|
+
|
|
19
|
+
1. **GLiNER model fails to initialize or load**
|
|
20
|
+
- Signal: startup warning in build/test output or runtime log.
|
|
21
|
+
- Effect: semantic entities from GLiNER are unavailable.
|
|
22
|
+
- Mitigation: continue with regex-only scan (already implemented with warning path).
|
|
23
|
+
|
|
24
|
+
2. **Runtime GLiNER inference errors**
|
|
25
|
+
- Signal: scanner logs and returns regex-only results.
|
|
26
|
+
- Effect: temporary loss of semantic detections only.
|
|
27
|
+
- Mitigation: keep regex detections and return a best-effort result.
|
|
28
|
+
|
|
29
|
+
3. **Wrong plugin package metadata during review/installation**
|
|
30
|
+
- Signal: mismatch across `package.json`, lockfile, and docs.
|
|
31
|
+
- Effect: reviewer confusion or install friction.
|
|
32
|
+
- Mitigation: document and verify package identity consistently for submission target.
|
|
33
|
+
|
|
34
|
+
4. **Model download or environment path issues**
|
|
35
|
+
- Signal: GLiNER init failures in constrained environments.
|
|
36
|
+
- Effect: reduced detection coverage.
|
|
37
|
+
- Mitigation: allow fallback to deterministic regex tests and avoid hard dependency in test harness.
|
|
38
|
+
|
|
39
|
+
## Monitoring
|
|
40
|
+
|
|
41
|
+
No centralized telemetry is included in this repository. Reliability should be observed by:
|
|
42
|
+
|
|
43
|
+
- test pass/fail trends,
|
|
44
|
+
- OpenClaw plugin load or registration diagnostics,
|
|
45
|
+
- explicit warning signals when GLiNER is unavailable.
|
|
46
|
+
|
|
47
|
+
## Operational Guardrails
|
|
48
|
+
|
|
49
|
+
- Keep `enabled: false` as a quick mitigation switch if plugin behavior needs immediate suppression.
|
|
50
|
+
- Validate with reviewer smoke commands before any package-level change.
|
|
51
|
+
- Prefer small, atomic commits for submission-related metadata and test updates.
|
|
52
|
+
- For release risk, run build + unit tests before pushing, and re-run on merge.
|
|
53
|
+
|
|
54
|
+
## Rollback Path
|
|
55
|
+
|
|
56
|
+
If submission metadata changes cause regressions, revert to the last green commit from this initiative branch, then revalidate:
|
|
57
|
+
|
|
58
|
+
- `npm test`
|
|
59
|
+
- `npm run build`
|
|
60
|
+
- plugin contract smoke test for hook/tool behavior.
|
package/docs/SECURITY.md
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Security"
|
|
3
|
+
use_when: "Capturing security expectations for this repo: threat model, auth/authorization, data sensitivity, compliance, and required controls."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
## Threat Model
|
|
7
|
+
|
|
8
|
+
FogClaw processes user-provided prompt text and returns extracted entities with character offsets. The main risk is accidental exposure of sensitive information in logs, crash output, or plugin responses.
|
|
9
|
+
|
|
10
|
+
Assume untrusted text arrives from user messages.
|
|
11
|
+
|
|
12
|
+
For this repo, a concrete threat is that PII can appear in plain text and be reflected back in a wrong form (for example, redacted text leaking original spans) if redaction logic or logging is incorrect.
|
|
13
|
+
|
|
14
|
+
## Auth Model
|
|
15
|
+
|
|
16
|
+
This package does not implement authentication itself. It is a plugin loaded by OpenClaw, and security is enforced by OpenClaw’s plugin installation and runtime controls.
|
|
17
|
+
|
|
18
|
+
Within this repo, this means:
|
|
19
|
+
|
|
20
|
+
- No custom auth credentials are accepted.
|
|
21
|
+
- No authorization checks are added in the plugin itself.
|
|
22
|
+
- Sensitive behavior is controlled by config and explicit runtime policy (`redact`, `warn`, `block`).
|
|
23
|
+
|
|
24
|
+
## Data Sensitivity
|
|
25
|
+
|
|
26
|
+
Treat plugin input as sensitive by default.
|
|
27
|
+
|
|
28
|
+
- PII and custom entities are parsed from incoming messages.
|
|
29
|
+
- Entity text is held in memory during scan and redaction only.
|
|
30
|
+
- Do not write raw messages, detected entity values, or redaction mappings to disk.
|
|
31
|
+
- Prefer hash or token strategies when persistence is required by caller policy.
|
|
32
|
+
|
|
33
|
+
## Compliance
|
|
34
|
+
|
|
35
|
+
This plugin is scoped as an on-device plugin. It does not claim HIPAA, GDPR, or SOC 2 compliance by itself.
|
|
36
|
+
|
|
37
|
+
At minimum, callers should classify whether their usage has regulated data and enforce policy at the platform level (retention, purge, and audit).
|
|
38
|
+
|
|
39
|
+
## Controls
|
|
40
|
+
|
|
41
|
+
- Keep `api.logger` and local logs free of entity values.
|
|
42
|
+
- Never store scan results in global caches.
|
|
43
|
+
- Preserve plugin fallback behavior: if GLiNER initialization fails, continue in regex-only mode and do not fail the entire request path.
|
|
44
|
+
- Enforce the plugin-level switch (`enabled`) to allow safe disablement without process restart if needed.
|
|
45
|
+
|
|
46
|
+
## OpenClaw-Specific Security Notes
|
|
47
|
+
|
|
48
|
+
- Use clean plugin configuration.
|
|
49
|
+
- Restrict plugin installation and publishing channels to trusted owners.
|
|
50
|
+
- In reviews, verify `openclaw.plugin.json` and package metadata before listing; mismatched package identity can create install ambiguity.
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Core Beliefs
|
|
2
|
+
|
|
3
|
+
Document the product and engineering beliefs that guide roadmap, architecture, and delivery decisions.
|
|
4
|
+
|
|
5
|
+
<!-- seed: Populated from bootstrap Q8. If AGENTS.md has golden principles, those are suggested as starting points. -->
|
|
6
|
+
|
|
7
|
+
## Belief 1
|
|
8
|
+
|
|
9
|
+
- Statement:
|
|
10
|
+
- Why it matters:
|
|
11
|
+
- Tradeoffs:
|
|
12
|
+
|
|
13
|
+
## Belief 2
|
|
14
|
+
|
|
15
|
+
- Statement:
|
|
16
|
+
- Why it matters:
|
|
17
|
+
- Tradeoffs:
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Generated Context
|
|
2
|
+
|
|
3
|
+
This directory holds generated reference context that agents create to reason about the codebase. Files here are auto-generated snapshots — not hand-authored documentation.
|
|
4
|
+
|
|
5
|
+
## When to Create
|
|
6
|
+
|
|
7
|
+
Create a generated context file when a skill (`he-implement`, `he-review`, `he-doc-gardening`) discovers relevant project infrastructure during its workflow. Discovery signals and corresponding context files:
|
|
8
|
+
|
|
9
|
+
| Discovery Signal | Context to Create | Example Filename |
|
|
10
|
+
|---|---|---|
|
|
11
|
+
| Database migrations or schema files exist | Schema snapshot | `db-schema.md` |
|
|
12
|
+
| Route definitions or API framework detected | API endpoint index | `api-schema.md` |
|
|
13
|
+
| UI component hierarchy (React, Vue, etc.) | Component tree map | `component-tree.md` |
|
|
14
|
+
| Complex module dependency structure | Dependency graph | `dependency-graph.md` |
|
|
15
|
+
|
|
16
|
+
This is not exhaustive — create whatever context helps agents reason about the project. The key rule: only create files for infrastructure that actually exists.
|
|
17
|
+
|
|
18
|
+
## Format Contract
|
|
19
|
+
|
|
20
|
+
Every generated file must include:
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
- last_updated: YYYY-MM-DD HH:MM
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
The `he-docs-lint` CI gate checks this timestamp on all files in this directory (except README.md and memory.md).
|
|
27
|
+
|
|
28
|
+
## Rules
|
|
29
|
+
|
|
30
|
+
- **Do not** create files for infrastructure the project does not have.
|
|
31
|
+
- **Do not** manually edit generated files — regenerate them from source.
|
|
32
|
+
- **Do** regenerate when the underlying source changes (migrations added, routes modified, etc.).
|
|
33
|
+
|
|
34
|
+
## memory.md
|
|
35
|
+
|
|
36
|
+
`memory.md` is a separate concept: it is a scratchpad for observations and patterns discovered during work, processed by `he-learn`. It is not auto-generated context and is not subject to the `last_updated` requirement.
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
# Memory
|
|
@@ -0,0 +1,172 @@
|
|
|
1
|
+
# FogClaw Design Document
|
|
2
|
+
|
|
3
|
+
**Date:** 2026-02-16
|
|
4
|
+
**Repo:** `datafog/fogclaw` (public, MIT license)
|
|
5
|
+
**Status:** Approved
|
|
6
|
+
|
|
7
|
+
## Overview
|
|
8
|
+
|
|
9
|
+
FogClaw is an OpenClaw plugin that brings DataFog's PII detection and redaction capabilities into the OpenClaw AI agent ecosystem. It acts as both a passive guardrail on message flow and an on-demand tool the agent can invoke explicitly. It uses a dual-engine approach: ported DataFog regex patterns for structured PII and GLiNER (via ONNX) for zero-shot NER on custom entities.
|
|
10
|
+
|
|
11
|
+
## Decisions
|
|
12
|
+
|
|
13
|
+
| Decision | Choice |
|
|
14
|
+
|---|---|
|
|
15
|
+
| Use case | Both guardrail + on-demand tool |
|
|
16
|
+
| Language | Pure TypeScript (ONNX for GLiNER) |
|
|
17
|
+
| Regex layer | Port DataFog regex patterns |
|
|
18
|
+
| PII action | Configurable per-entity-type (default: redact) |
|
|
19
|
+
| Custom terms | Config file (`fogclaw.config.json`) |
|
|
20
|
+
| Default model | `onnx-community/gliner_large-v2.1` |
|
|
21
|
+
| Architecture | Dual-layer (regex + GLiNER) |
|
|
22
|
+
|
|
23
|
+
## Project Structure
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
fogclaw/
|
|
27
|
+
├── openclaw.plugin.json # OpenClaw plugin manifest
|
|
28
|
+
├── package.json
|
|
29
|
+
├── tsconfig.json
|
|
30
|
+
├── fogclaw.config.example.json # Example user config
|
|
31
|
+
├── src/
|
|
32
|
+
│ ├── index.ts # Plugin entry: register hook + tool
|
|
33
|
+
│ ├── engines/
|
|
34
|
+
│ │ ├── regex.ts # Ported DataFog regex patterns
|
|
35
|
+
│ │ └── gliner.ts # GLiNER ONNX inference wrapper
|
|
36
|
+
│ ├── scanner.ts # Orchestrator: regex → GLiNER pipeline
|
|
37
|
+
│ ├── redactor.ts # Redaction strategies (token, mask, hash)
|
|
38
|
+
│ ├── config.ts # Config loading & validation
|
|
39
|
+
│ └── types.ts # Shared TypeScript types
|
|
40
|
+
├── models/ # Auto-downloaded ONNX model cache
|
|
41
|
+
├── tests/
|
|
42
|
+
│ ├── regex.test.ts
|
|
43
|
+
│ ├── gliner.test.ts
|
|
44
|
+
│ ├── scanner.test.ts
|
|
45
|
+
│ └── redactor.test.ts
|
|
46
|
+
└── README.md
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## Detection Pipeline
|
|
50
|
+
|
|
51
|
+
```
|
|
52
|
+
Input text
|
|
53
|
+
│
|
|
54
|
+
▼
|
|
55
|
+
┌─────────────┐
|
|
56
|
+
│ Regex Pass │ ← emails, SSNs, phones, credit cards, IPs, dates, zips
|
|
57
|
+
│ (~20µs/kB) │ confidence: 1.0
|
|
58
|
+
└─────┬───────┘
|
|
59
|
+
│
|
|
60
|
+
▼
|
|
61
|
+
┌─────────────┐
|
|
62
|
+
│ GLiNER Pass │ ← persons, orgs, locations + custom entities from config
|
|
63
|
+
│ (ONNX) │ confidence: 0.0-1.0
|
|
64
|
+
└─────┬───────┘
|
|
65
|
+
│
|
|
66
|
+
▼
|
|
67
|
+
┌─────────────┐
|
|
68
|
+
│ Merge & │ ← Deduplicate overlapping spans, prefer higher confidence
|
|
69
|
+
│ Normalize │ Canonical type mapping (same as DataFog)
|
|
70
|
+
└─────┬───────┘
|
|
71
|
+
│
|
|
72
|
+
▼
|
|
73
|
+
Entity[] — unified results
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Entity Type
|
|
77
|
+
|
|
78
|
+
```typescript
|
|
79
|
+
interface Entity {
|
|
80
|
+
text: string; // "john@example.com"
|
|
81
|
+
label: string; // "EMAIL"
|
|
82
|
+
start: number; // character offset
|
|
83
|
+
end: number;
|
|
84
|
+
confidence: number; // 1.0 for regex, 0.0-1.0 for GLiNER
|
|
85
|
+
source: "regex" | "gliner";
|
|
86
|
+
}
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
### Span Conflict Resolution
|
|
90
|
+
|
|
91
|
+
When regex and GLiNER detect overlapping spans, prefer regex (confidence 1.0) for structured types, GLiNER for semantic types. Partially overlapping spans resolved by higher confidence.
|
|
92
|
+
|
|
93
|
+
### GLiNER Labels
|
|
94
|
+
|
|
95
|
+
Built-in: `["person", "organization", "location", "address", "date of birth", "medical record number", "account number", "passport number"]`
|
|
96
|
+
|
|
97
|
+
Plus `custom_entities` from user config.
|
|
98
|
+
|
|
99
|
+
## OpenClaw Integration
|
|
100
|
+
|
|
101
|
+
### Hook (Guardrail)
|
|
102
|
+
|
|
103
|
+
Registers `before_agent_start` hook to intercept incoming messages. Per-entity-type actions:
|
|
104
|
+
- **redact**: Replace with tokens like `[EMAIL]` (default)
|
|
105
|
+
- **block**: Stop message, notify user
|
|
106
|
+
- **warn**: Notify but allow message through
|
|
107
|
+
|
|
108
|
+
### Tools
|
|
109
|
+
|
|
110
|
+
Two tools registered for on-demand use by the agent:
|
|
111
|
+
|
|
112
|
+
1. **fogclaw_scan** — Detect entities in text, return structured results
|
|
113
|
+
2. **fogclaw_redact** — Detect and redact entities, return sanitized text
|
|
114
|
+
|
|
115
|
+
Both accept optional `custom_labels` parameter for ad-hoc zero-shot entity detection.
|
|
116
|
+
|
|
117
|
+
### Redaction Strategies
|
|
118
|
+
|
|
119
|
+
- **token**: `"Contact john@example.com"` → `"Contact [EMAIL]"`
|
|
120
|
+
- **mask**: `"Contact john@example.com"` → `"Contact ****************"`
|
|
121
|
+
- **hash**: `"Contact john@example.com"` → `"Contact [EMAIL_a1b2c3d4e5f6]"`
|
|
122
|
+
|
|
123
|
+
## Configuration
|
|
124
|
+
|
|
125
|
+
```json
|
|
126
|
+
{
|
|
127
|
+
"enabled": true,
|
|
128
|
+
"guardrail_mode": "redact",
|
|
129
|
+
"redactStrategy": "token",
|
|
130
|
+
"model": "onnx-community/gliner_large-v2.1",
|
|
131
|
+
"confidence_threshold": 0.5,
|
|
132
|
+
"custom_entities": ["project codename", "internal tool name", "competitor name"],
|
|
133
|
+
"entityActions": {
|
|
134
|
+
"SSN": "block",
|
|
135
|
+
"CREDIT_CARD": "block",
|
|
136
|
+
"EMAIL": "redact",
|
|
137
|
+
"PHONE": "redact",
|
|
138
|
+
"PERSON": "warn"
|
|
139
|
+
}
|
|
140
|
+
}
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
## Dependencies
|
|
144
|
+
|
|
145
|
+
```json
|
|
146
|
+
{
|
|
147
|
+
"dependencies": {
|
|
148
|
+
"gliner": "^0.x.x",
|
|
149
|
+
"onnxruntime-node": "^1.x"
|
|
150
|
+
},
|
|
151
|
+
"devDependencies": {
|
|
152
|
+
"vitest": "^2.x",
|
|
153
|
+
"typescript": "^5.x"
|
|
154
|
+
}
|
|
155
|
+
}
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
## Technical Considerations
|
|
159
|
+
|
|
160
|
+
**Model Loading:** Downloaded once from HuggingFace, cached in `~/.openclaw/extensions/fogclaw/models/`. Singleton pattern — stays loaded after first inference.
|
|
161
|
+
|
|
162
|
+
**Error Handling:** GLiNER failure → fall back to regex-only with warning. Network failure during download → clear error with manual download instructions.
|
|
163
|
+
|
|
164
|
+
**Performance:** Regex <1ms, GLiNER ~50-200ms per message. Well under 1s total — acceptable for messaging bots.
|
|
165
|
+
|
|
166
|
+
## Not In v1 (YAGNI)
|
|
167
|
+
|
|
168
|
+
- No outbound message scanning
|
|
169
|
+
- No persistent audit log
|
|
170
|
+
- No web UI for config
|
|
171
|
+
- No GLiNER2 support (add later when npm ecosystem catches up)
|
|
172
|
+
- No runtime entity label management (config file only)
|