@llnvd/openclaw-url-guard 0.0.1

package/openspec/changes/e2e-real-openclaw-tests/specs/e2e-real-instance/spec.md

@@ -0,0 +1,197 @@
+# E2E Real Instance Integration Tests
+
+## ADDED Requirements
+
+### Requirement: Gateway lifecycle management
+
+The system SHALL provide helper functions to start, health-check, and stop a real OpenClaw gateway instance for testing purposes.
+
+#### Scenario: Start gateway with custom config
+
+- **WHEN** a test suite starts with a given `UrlGuardConfig`
+- **THEN** the helper SHALL generate a temp config file with the plugin configured
+- **AND** spawn `openclaw gateway` as a subprocess on a dynamically assigned port
+- **AND** wait for the health endpoint to respond (HTTP 200) with exponential backoff
+- **AND** fail the test suite if the gateway does not become healthy within 30 seconds
+
+#### Scenario: Stop gateway on suite teardown
+
+- **WHEN** a test suite completes (pass or fail)
+- **THEN** the helper SHALL kill the gateway subprocess and delete temp config files
+- **AND** if the test runner crashes, a `process.on('exit')` handler SHALL kill any tracked gateway PIDs
+
+#### Scenario: Port isolation
+
+- **WHEN** multiple test suites run (sequentially or in parallel)
+- **THEN** each suite SHALL use a unique port (via port 0 binding or find-free-port)
+- **AND** no port collisions SHALL occur
+
+### Requirement: Allowlist enforcement through real gateway
+
+The system SHALL verify that allowlist mode works correctly through the real OpenClaw gateway.
+
+#### Scenario: Allowed domain passes through
+
+- **GIVEN** a gateway configured with mode `allowlist` and allowlist `["example.com"]`
+- **WHEN** a tool call for `web_fetch` with URL `https://example.com/page` is sent to the gateway
+- **THEN** the response SHALL indicate the request was allowed
+
+#### Scenario: Unlisted domain is blocked
+
+- **GIVEN** a gateway configured with mode `allowlist` and allowlist `["example.com"]`
+- **WHEN** a tool call for `web_fetch` with URL `https://evil.com/payload` is sent
+- **THEN** the response SHALL be a structured block response
+- **AND** the `reason` field SHALL be `"not in allowlist"`
+- **AND** the `policy` field SHALL be `"allowlist"`
+
+#### Scenario: Wildcard allowlist matching
+
+- **GIVEN** a gateway configured with allowlist `["*.example.com"]`
+- **WHEN** a tool call for `https://api.example.com/data` is sent
+- **THEN** the request SHALL be allowed
+- **AND** a request for `https://example.com/data` (bare domain) SHALL be blocked
+
+### Requirement: Blocklist enforcement through real gateway
+
+The system SHALL verify blocklist mode through the real gateway.
+
+#### Scenario: Blocked domain is rejected
+
+- **GIVEN** a gateway configured with mode `blocklist` and blocklist `["evil.com", "*.malware.org"]`
+- **WHEN** a tool call for `https://evil.com/exploit` is sent
+- **THEN** the response SHALL be a structured block response with reason `"in blocklist"`
+
+#### Scenario: Unlisted domain passes in blocklist mode
+
+- **GIVEN** a gateway configured with mode `blocklist` and blocklist `["evil.com"]`
+- **WHEN** a tool call for `https://safe.com/page` is sent
+- **THEN** the request SHALL be allowed
+
+### Requirement: Hybrid mode enforcement through real gateway
+
+The system SHALL verify hybrid mode (allowlist + blocklist) through the real gateway.
+
+#### Scenario: Allowed and not blocked passes
+
+- **GIVEN** a gateway with mode `hybrid`, allowlist `["*.example.com"]`, blocklist `["blocked.example.com"]`
+- **WHEN** a tool call for `https://api.example.com/data` is sent
+- **THEN** the request SHALL be allowed
+
+#### Scenario: Allowed but also blocked is rejected
+
+- **GIVEN** the same hybrid config
+- **WHEN** a tool call for `https://blocked.example.com/data` is sent
+- **THEN** the response SHALL be a structured block response
+
+#### Scenario: Not in allowlist is rejected in hybrid mode
+
+- **GIVEN** the same hybrid config
+- **WHEN** a tool call for `https://outside.com/page` is sent
+- **THEN** the response SHALL be blocked with reason `"not in allowlist"`
+
+### Requirement: SSRF protection through real gateway
+
+The system SHALL verify that private/internal IP addresses are blocked at the integration level.
+
+#### Scenario: Localhost is blocked
+
+- **GIVEN** a gateway configured with `blockPrivateIps: true`
+- **WHEN** a tool call for `http://127.0.0.1/admin` is sent
+- **THEN** the response SHALL be a structured block response with reason `"private or internal IP blocked"`
+
+#### Scenario: Link-local address is blocked
+
+- **GIVEN** the same config
+- **WHEN** a tool call for `http://169.254.169.254/metadata` is sent (cloud metadata endpoint)
+- **THEN** the response SHALL be blocked
+
+#### Scenario: IPv6 loopback is blocked
+
+- **GIVEN** the same config
+- **WHEN** a tool call for `http://[::1]/admin` is sent
+- **THEN** the response SHALL be blocked
+
+### Requirement: URLhaus blocking with fixture feed
+
+The system SHALL verify URLhaus-based blocking using a deterministic fixture feed.
+
+#### Scenario: Known malicious URL is blocked
+
+- **GIVEN** a gateway configured with URLhaus enabled, pointing to `tests/integration/fixtures/urlhaus-sample.csv`
+- **WHEN** a tool call for a URL present in the fixture feed is sent
+- **THEN** the response SHALL be blocked with a reason indicating URLhaus match
+
+#### Scenario: Clean URL passes URLhaus check
+
+- **GIVEN** the same config
+- **WHEN** a tool call for `https://clean-example.com/safe` (not in fixture) is sent
+- **THEN** the request SHALL be allowed (assuming it passes other policy checks)
+
+### Requirement: Config validation error handling
+
+The system SHALL produce clear errors when given malformed configuration.
+
+#### Scenario: Missing required mode field
+
+- **GIVEN** a config file with the plugin configured but `mode` field omitted
+- **WHEN** the gateway starts
+- **THEN** the gateway SHALL fail to start or the plugin SHALL report a clear validation error
+- **AND** the error message SHALL mention the missing `mode` field
+
+#### Scenario: Invalid mode value
+
+- **GIVEN** a config with `mode: "invalid_mode"`
+- **WHEN** the gateway starts
+- **THEN** a clear validation error SHALL be reported
+
+#### Scenario: Allowlist mode without allowlist array
+
+- **GIVEN** a config with `mode: "allowlist"` but no `allowlist` field
+- **WHEN** a tool call is made
+- **THEN** the behavior SHALL be well-defined (either block all, or error clearly)
+
+### Requirement: Structured block response format
+
+All blocked requests through the real gateway SHALL return a structured response.
+
+#### Scenario: Block response contains required fields
+
+- **WHEN** any request is blocked by the plugin
+- **THEN** the response SHALL contain at minimum:
+  - `blocked`: `true`
+  - `reason`: a human-readable explanation (matching `getBlockReason()` output)
+  - `url`: the original requested URL
+
+### Requirement: Test gating and CI integration
+
+The integration tests SHALL be gated and integrated into CI.
+
+#### Scenario: Tests skip when not opted in
+
+- **WHEN** `INTEGRATION_TESTS` env var is not set or not `"true"`
+- **THEN** all integration tests SHALL be skipped gracefully (not fail)
+
+#### Scenario: npm script available
+
+- **WHEN** a developer runs `npm run test:integration`
+- **THEN** integration tests SHALL execute with `INTEGRATION_TESTS=true` set automatically
+
+#### Scenario: CI runs on main only
+
+- **GIVEN** the CI workflow configuration
+- **THEN** the integration test job SHALL only run on pushes to `main` (not on PRs)
+- **AND** the job SHALL have a timeout of 5 minutes
+- **AND** the job SHALL depend on unit and E2E test jobs passing first
+
+### Requirement: Documentation
+
+#### Scenario: Testing docs exist
+
+- **THEN** `docs/testing.md` SHALL document all test tiers (unit, E2E mock, E2E integration)
+- **AND** explain how to run integration tests locally
+- **AND** explain the env var gating mechanism
+
+#### Scenario: README updated
+
+- **THEN** the README testing section SHALL reference the integration test tier
+- **AND** include the `npm run test:integration` command

package/openspec/changes/e2e-real-openclaw-tests/tasks.md

@@ -0,0 +1,34 @@
+# Tasks
+
+## Phase 1: Infrastructure
+
+- [ ] Create `tests/integration/helpers/gateway.ts` — gateway lifecycle (start, stop, health check, PID tracking, cleanup)
+- [ ] Create `tests/integration/helpers/config.ts` — temp config file generation from `UrlGuardConfig`
+- [ ] Create `tests/integration/helpers/client.ts` — HTTP client for sending tool calls to gateway
+- [ ] Create `tests/integration/fixtures/urlhaus-sample.csv` — static URLhaus feed snapshot (5-10 entries)
+- [ ] Add `test:integration` script to `package.json` (sets `INTEGRATION_TESTS=true`, runs vitest on `tests/integration/`)
+
+## Phase 2: Core Test Suites
+
+- [ ] Create `tests/integration/allowlist.test.ts` — allowlist enforcement (allowed, blocked, wildcard, bare domain)
+- [ ] Create `tests/integration/blocklist.test.ts` — blocklist enforcement (blocked, unlisted passes)
+- [ ] Create `tests/integration/hybrid.test.ts` — hybrid mode (allowed+not-blocked passes, allowed+blocked rejected, not-in-allowlist rejected)
+- [ ] Create `tests/integration/ssrf.test.ts` — SSRF protection (localhost, link-local, IPv6 loopback, cloud metadata)
+- [ ] Create `tests/integration/urlhaus.test.ts` — URLhaus fixture-based blocking
+
+## Phase 3: Error Handling & Validation
+
+- [ ] Create `tests/integration/config-validation.test.ts` — malformed config errors (missing mode, invalid mode, missing allowlist)
+- [ ] Create `tests/integration/error-handling.test.ts` — gateway error behavior, structured block response format validation
+
+## Phase 4: CI & Documentation
+
+- [ ] Add `test-integration-real` job to `.forgejo/workflows/ci.yaml` (main-only, 5min timeout, depends on unit+e2e)
+- [ ] Create `docs/testing.md` — document all test tiers, local setup, env var gating
+- [ ] Update `README.md` testing section — reference integration tier, `npm run test:integration`
+
+## Dependencies
+
+- Phase 2 depends on Phase 1 (helpers must exist)
+- Phase 3 depends on Phase 1
+- Phase 4 can start after Phase 1 (CI job) or after Phase 3 (docs)

package/openspec/config.yaml

@@ -0,0 +1,8 @@
+schema: spec-driven
+
+context: |
+  Tech stack: TypeScript, Vitest, Node.js 20
+  OpenClaw plugin for URL security (allowlist/blocklist/hybrid modes)
+  Uses URLhaus threat feed, trust scoring, SSRF protection
+  CI: Forgejo Actions on Codeberg (codeberg-tiny/codeberg-small runners)
+  Conventions: conventional commits, ESLint + Prettier, vitest for testing

package/openspec/specs/e2e-real-instance/spec.md

@@ -0,0 +1,197 @@
+# E2E Real Instance Integration Tests
+
+## ADDED Requirements
+
+### Requirement: Gateway lifecycle management
+
+The system SHALL provide helper functions to start, health-check, and stop a real OpenClaw gateway instance for testing purposes.
+
+#### Scenario: Start gateway with custom config
+
+- **WHEN** a test suite starts with a given `UrlGuardConfig`
+- **THEN** the helper SHALL generate a temp config file with the plugin configured
+- **AND** spawn `openclaw gateway` as a subprocess on a dynamically assigned port
+- **AND** wait for the health endpoint to respond (HTTP 200) with exponential backoff
+- **AND** fail the test suite if the gateway does not become healthy within 30 seconds
+
+#### Scenario: Stop gateway on suite teardown
+
+- **WHEN** a test suite completes (pass or fail)
+- **THEN** the helper SHALL kill the gateway subprocess and delete temp config files
+- **AND** if the test runner crashes, a `process.on('exit')` handler SHALL kill any tracked gateway PIDs
+
+#### Scenario: Port isolation
+
+- **WHEN** multiple test suites run (sequentially or in parallel)
+- **THEN** each suite SHALL use a unique port (via port 0 binding or find-free-port)
+- **AND** no port collisions SHALL occur
+
+### Requirement: Allowlist enforcement through real gateway
+
+The system SHALL verify that allowlist mode works correctly through the real OpenClaw gateway.
+
+#### Scenario: Allowed domain passes through
+
+- **GIVEN** a gateway configured with mode `allowlist` and allowlist `["example.com"]`
+- **WHEN** a tool call for `web_fetch` with URL `https://example.com/page` is sent to the gateway
+- **THEN** the response SHALL indicate the request was allowed
+
+#### Scenario: Unlisted domain is blocked
+
+- **GIVEN** a gateway configured with mode `allowlist` and allowlist `["example.com"]`
+- **WHEN** a tool call for `web_fetch` with URL `https://evil.com/payload` is sent
+- **THEN** the response SHALL be a structured block response
+- **AND** the `reason` field SHALL be `"not in allowlist"`
+- **AND** the `policy` field SHALL be `"allowlist"`
+
+#### Scenario: Wildcard allowlist matching
+
+- **GIVEN** a gateway configured with allowlist `["*.example.com"]`
+- **WHEN** a tool call for `https://api.example.com/data` is sent
+- **THEN** the request SHALL be allowed
+- **AND** a request for `https://example.com/data` (bare domain) SHALL be blocked
+
+### Requirement: Blocklist enforcement through real gateway
+
+The system SHALL verify blocklist mode through the real gateway.
+
+#### Scenario: Blocked domain is rejected
+
+- **GIVEN** a gateway configured with mode `blocklist` and blocklist `["evil.com", "*.malware.org"]`
+- **WHEN** a tool call for `https://evil.com/exploit` is sent
+- **THEN** the response SHALL be a structured block response with reason `"in blocklist"`
+
+#### Scenario: Unlisted domain passes in blocklist mode
+
+- **GIVEN** a gateway configured with mode `blocklist` and blocklist `["evil.com"]`
+- **WHEN** a tool call for `https://safe.com/page` is sent
+- **THEN** the request SHALL be allowed
+
+### Requirement: Hybrid mode enforcement through real gateway
+
+The system SHALL verify hybrid mode (allowlist + blocklist) through the real gateway.
+
+#### Scenario: Allowed and not blocked passes
+
+- **GIVEN** a gateway with mode `hybrid`, allowlist `["*.example.com"]`, blocklist `["blocked.example.com"]`
+- **WHEN** a tool call for `https://api.example.com/data` is sent
+- **THEN** the request SHALL be allowed
+
+#### Scenario: Allowed but also blocked is rejected
+
+- **GIVEN** the same hybrid config
+- **WHEN** a tool call for `https://blocked.example.com/data` is sent
+- **THEN** the response SHALL be a structured block response
+
+#### Scenario: Not in allowlist is rejected in hybrid mode
+
+- **GIVEN** the same hybrid config
+- **WHEN** a tool call for `https://outside.com/page` is sent
+- **THEN** the response SHALL be blocked with reason `"not in allowlist"`
+
+### Requirement: SSRF protection through real gateway
+
+The system SHALL verify that private/internal IP addresses are blocked at the integration level.
+
+#### Scenario: Localhost is blocked
+
+- **GIVEN** a gateway configured with `blockPrivateIps: true`
+- **WHEN** a tool call for `http://127.0.0.1/admin` is sent
+- **THEN** the response SHALL be a structured block response with reason `"private or internal IP blocked"`
+
+#### Scenario: Link-local address is blocked
+
+- **GIVEN** the same config
+- **WHEN** a tool call for `http://169.254.169.254/metadata` is sent (cloud metadata endpoint)
+- **THEN** the response SHALL be blocked
+
+#### Scenario: IPv6 loopback is blocked
+
+- **GIVEN** the same config
+- **WHEN** a tool call for `http://[::1]/admin` is sent
+- **THEN** the response SHALL be blocked
+
+### Requirement: URLhaus blocking with fixture feed
+
+The system SHALL verify URLhaus-based blocking using a deterministic fixture feed.
+
+#### Scenario: Known malicious URL is blocked
+
+- **GIVEN** a gateway configured with URLhaus enabled, pointing to `tests/integration/fixtures/urlhaus-sample.csv`
+- **WHEN** a tool call for a URL present in the fixture feed is sent
+- **THEN** the response SHALL be blocked with a reason indicating URLhaus match
+
+#### Scenario: Clean URL passes URLhaus check
+
+- **GIVEN** the same config
+- **WHEN** a tool call for `https://clean-example.com/safe` (not in fixture) is sent
+- **THEN** the request SHALL be allowed (assuming it passes other policy checks)
+
+### Requirement: Config validation error handling
+
+The system SHALL produce clear errors when given malformed configuration.
+
+#### Scenario: Missing required mode field
+
+- **GIVEN** a config file with the plugin configured but `mode` field omitted
+- **WHEN** the gateway starts
+- **THEN** the gateway SHALL fail to start or the plugin SHALL report a clear validation error
+- **AND** the error message SHALL mention the missing `mode` field
+
+#### Scenario: Invalid mode value
+
+- **GIVEN** a config with `mode: "invalid_mode"`
+- **WHEN** the gateway starts
+- **THEN** a clear validation error SHALL be reported
+
+#### Scenario: Allowlist mode without allowlist array
+
+- **GIVEN** a config with `mode: "allowlist"` but no `allowlist` field
+- **WHEN** a tool call is made
+- **THEN** the behavior SHALL be well-defined (either block all, or error clearly)
+
+### Requirement: Structured block response format
+
+All blocked requests through the real gateway SHALL return a structured response.
+
+#### Scenario: Block response contains required fields
+
+- **WHEN** any request is blocked by the plugin
+- **THEN** the response SHALL contain at minimum:
+  - `blocked`: `true`
+  - `reason`: a human-readable explanation (matching `getBlockReason()` output)
+  - `url`: the original requested URL
+
+### Requirement: Test gating and CI integration
+
+The integration tests SHALL be gated and integrated into CI.
+
+#### Scenario: Tests skip when not opted in
+
+- **WHEN** `INTEGRATION_TESTS` env var is not set or not `"true"`
+- **THEN** all integration tests SHALL be skipped gracefully (not fail)
+
+#### Scenario: npm script available
+
+- **WHEN** a developer runs `npm run test:integration`
+- **THEN** integration tests SHALL execute with `INTEGRATION_TESTS=true` set automatically
+
+#### Scenario: CI runs on main only
+
+- **GIVEN** the CI workflow configuration
+- **THEN** the integration test job SHALL only run on pushes to `main` (not on PRs)
+- **AND** the job SHALL have a timeout of 5 minutes
+- **AND** the job SHALL depend on unit and E2E test jobs passing first
+
+### Requirement: Documentation
+
+#### Scenario: Testing docs exist
+
+- **THEN** `docs/testing.md` SHALL document all test tiers (unit, E2E mock, E2E integration)
+- **AND** explain how to run integration tests locally
+- **AND** explain the env var gating mechanism
+
+#### Scenario: README updated
+
+- **THEN** the README testing section SHALL reference the integration test tier
+- **AND** include the `npm run test:integration` command

package/package.json

@@ -0,0 +1,39 @@
+{
+  "name": "@llnvd/openclaw-url-guard",
+  "version": "0.0.1",
+  "description": "OpenClaw plugin for URL allowlisting/blocklisting in web_fetch and web_search tools",
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "scripts": {
+    "build": "tsc -p tsconfig.json",
+    "test": "vitest run",
+    "test:e2e": "vitest run tests/e2e/",
+    "test:integration": "INTEGRATION_TESTS=true vitest run tests/integration/",
+    "lint": "oxlint src/ tests/",
+    "lint:fix": "oxlint --fix src/ tests/",
+    "format": "prettier --write 'src/**/*.ts' 'tests/**/*.ts'",
+    "format:check": "prettier --check 'src/**/*.ts' 'tests/**/*.ts'",
+    "check": "npm run lint && npm run format:check && npm run test",
+    "prepare": "husky"
+  },
+  "lint-staged": {
+    "*.ts": [
+      "oxlint --fix",
+      "prettier --write"
+    ]
+  },
+  "devDependencies": {
+    "@types/node": "^25.3.5",
+    "husky": "^9.1.7",
+    "lint-staged": "^15.4.3",
+    "oxlint": "^1.51.0",
+    "prettier": "^3.5.3",
+    "typescript": "^5.6.3",
+    "vitest": "^2.1.8"
+  },
+  "openclaw": {
+    "extensions": [
+      "./dist/src/index.js"
+    ]
+  }
+}