@wenathlan/saddle 1.8.6 → 1.8.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +114 -103
- package/docs/actionsincident.md +4 -0
- package/docs/release.md +5 -5
- package/docs/releaseassets.md +1 -1
- package/docs/securityaudit-1.8.7.md +21 -0
- package/extension/manifest.json +1 -1
- package/package.json +2 -3
- package/readme.txt +0 -163
- package/scrape/package-lock.json +0 -9397
- package/scrape/package.json +0 -1420
package/README.md
CHANGED
|
@@ -1,22 +1,20 @@
|
|
|
1
|
-
# Saddle
|
|
2
|
-
|
|
3
1
|
<p align="center">
|
|
4
2
|
<img src="docs/assets/saddlemark.svg" alt="Saddle" width="720" />
|
|
5
3
|
</p>
|
|
6
4
|
|
|
7
5
|
<p align="center">
|
|
8
6
|
<strong>Storage-backed jobs, scraping contracts and portable runners for Node.js.</strong><br/>
|
|
9
|
-
<strong>Binary computing
|
|
7
|
+
<strong>Binary computing engine, agent browser, scraper and packager.</strong><br/>
|
|
10
8
|
<a href="https://github.com/wenathlan/saddle/actions/workflows/ci.yml"><img src="https://github.com/wenathlan/saddle/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
|
|
11
|
-
<a href="https://github.com/wenathlan/saddle/releases/tag/v1.8.
|
|
9
|
+
<a href="https://github.com/wenathlan/saddle/releases/tag/v1.8.7"><img src="https://img.shields.io/badge/release-v1.8.7-d35d3d" alt="Release 1.8.7" /></a>
|
|
12
10
|
<a href="https://github.com/wenathlan/saddle/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-GPL--3.0-202a2f" alt="GPL 3.0 license" /></a>
|
|
13
11
|
</p>
|
|
14
12
|
|
|
15
|
-
> **Core idea:** storage is the durable side of the working set; the runner is replaceable; the artifact is the boundary. **Storage == Compute**
|
|
13
|
+
> **Core idea:** storage is the durable side of the working set; the runner is replaceable; the artifact is the boundary. **Storage == Compute** means that the same bytes can be retained or processed according to an explicit usage flag.
|
|
16
14
|
|
|
17
|
-
Saddle is a **JavaScript ESM engine** for jobs that move data between storage, a working set,
|
|
15
|
+
Saddle is a **JavaScript ESM engine** for jobs that move data between storage, a bounded working set, a caller-injected runner and durable artifacts. It is also a virtual machine published as a package: the caller can run it on GitHub Actions, Forgejo, Gitea, GitLab, Codeberg, Docker or another third-party compute surface. The engine does not require the operator's local machine, does not embed credentials and does not choose a mandatory cloud provider.
|
|
18
16
|
|
|
19
|
-
|
|
17
|
+
The canonical JavaScript package is `@wenathlan/saddle`. GitHub Packages npm, Maven and GHCR use the `wenathlan` owner namespace; NuGet and RubyGems retain their ecosystem package names. Older `@devthink`, `@iakadion` and `io.devthink` references in archived documents are historical records, not current package identities.
|
|
20
18
|
|
|
21
19
|
## Start here
|
|
22
20
|
|
|
@@ -35,160 +33,173 @@ const context = formatforagent(result, { maxchunksize: 2000, keypoints: 4 });
|
|
|
35
33
|
console.log(context.summary);
|
|
36
34
|
```
|
|
37
35
|
|
|
38
|
-
|
|
36
|
+
The deterministic examples and tests do not require network access or real credentials:
|
|
39
37
|
|
|
40
38
|
```bash
|
|
41
39
|
node examples/publicapi.js
|
|
40
|
+
npm test
|
|
42
41
|
```
|
|
43
42
|
|
|
44
|
-
##
|
|
43
|
+
## Progressive architecture
|
|
44
|
+
|
|
45
|
+
The project documentation follows a progressive arc. The foundation describes the storage and runner model; the engine describes the contracts that make the model executable; productization describes the package, extension, workflow and web surfaces.
|
|
46
|
+
|
|
47
|
+
### Foundation: storage, runners and working sets
|
|
48
|
+
|
|
49
|
+
Saddle treats a repository, bucket or object store as durable state and a third-party runner as a replaceable processor. GitHub Actions is one adapter, not the core. Forgejo, Gitea, GitLab, Codeberg, Docker and caller-owned runners can implement the same runner contracts.
|
|
50
|
+
|
|
51
|
+
The physical limit remains explicit: remote storage is not VRAM. A storage-to-RAM bridge can stage a bounded working set through a local filesystem, tmpfs, mmap, cache or caller-owned storage adapter, but it cannot remove network latency or create the bandwidth of a GPU bus. The engine exposes that distinction instead of hiding it behind marketing language.
|
|
52
|
+
|
|
53
|
+
The execution model is:
|
|
54
|
+
|
|
55
|
+
```text
|
|
56
|
+
repository or bucket -> runner working set -> process -> durable artifact
|
|
57
|
+
persistent state virtual processor published boundary
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
The repository may act as a disk, a CI workflow may act as a function call, Pages may act as a static bus and a release artifact may act as the durable boundary. `workflow_dispatch`, `repository_dispatch` and HTTP adapters remain caller-configured interfaces.
|
|
61
|
+
|
|
62
|
+
### Engine: contracts instead of vendor lock-in
|
|
45
63
|
|
|
46
|
-
| Area |
|
|
64
|
+
| Area | Contracts shipped | Result |
|
|
47
65
|
| --- | --- | --- |
|
|
48
|
-
| Jobs | `engine`, `scheduler`, `inprocess` | `prepare
|
|
49
|
-
| Storage | local, chunked, content-addressed, S3-compatible, GitHub Contents
|
|
50
|
-
| Working set | memory bridge, modes, objects
|
|
51
|
-
| Scraping | robots, cache, extraction, semantic facts, schema
|
|
52
|
-
| Crawl | normalization, priority frontier, BFS crawler
|
|
53
|
-
| Browser | snapshots, tabs, frames, actions, fingerprint, session
|
|
66
|
+
| Jobs | `engine`, `scheduler`, `inprocess` | `prepare -> process -> sync -> cleanup` |
|
|
67
|
+
| Storage | local, chunked, content-addressed, S3-compatible, GitHub Contents and file-hosting adapters | durable objects, ranges, dedupe and sync |
|
|
68
|
+
| Working set | memory bridge, modes, objects and transforms | storage-to-compute and compute-to-storage flows |
|
|
69
|
+
| Scraping | robots, cache, extraction, semantic facts, schema and normalization | bounded text, metadata, links, controls and structured output |
|
|
70
|
+
| Crawl | normalization, priority frontier, BFS crawler and persistent frontier contracts | domain-aware bounded crawling |
|
|
71
|
+
| Browser | snapshots, tabs, frames, actions, fingerprint, session and replay contracts | caller-owned browser automation without a mandatory provider |
|
|
54
72
|
| Operations | queues, idempotency, saga, retry, circuit breaker, health and heartbeat | controlled execution and recovery |
|
|
55
73
|
| Protocols | JSON, NDJSON, SSE, blocks, API envelopes and MCP | transport-neutral messages |
|
|
56
|
-
| Delivery | manifests, workflow registry,
|
|
74
|
+
| Delivery | manifests, workflow registry, extension packaging and release assets | repeatable package and runner surfaces |
|
|
57
75
|
| Integrations | GitHub, GitLab, Forgejo, app lifecycle, command scopes and delivery adapters | caller-owned provider connectivity |
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
76
|
+
|
|
77
|
+
The root entry point is transport-neutral. Node filesystem, HTTP server, persistent sessions and Playwright are explicit subpaths or optional adapters. The library accepts caller-provided fetchers, browser transports, storage adapters, persistence, proxy pools, captcha evidence handlers, webhook secrets and remote credentials.
|
|
78
|
+
|
|
79
|
+
### Productization: one engine, many shells
|
|
80
|
+
|
|
81
|
+
The same contracts can be surfaced as an npm library, CLI, binary, Manifest V3 browser extension, webhook server, MCP transport, workflow action, container image, Maven package, NuGet package or RubyGem. These surfaces are adapters around the engine; they are not separate sources of truth.
|
|
62
82
|
|
|
63
83
|
## Public API
|
|
64
84
|
|
|
65
85
|
| Export | Purpose |
|
|
66
86
|
| --- | --- |
|
|
67
|
-
| `saddleurl` | choose fetch or injected browser path |
|
|
68
|
-
| `scrapeurl` | fetch one URL and extract |
|
|
69
|
-
| `scrapehtml` | extract from HTML without network |
|
|
87
|
+
| `saddleurl` | choose a fetch or caller-injected browser path |
|
|
88
|
+
| `scrapeurl` | fetch one URL and extract bounded content |
|
|
89
|
+
| `scrapehtml` | extract from HTML without network access |
|
|
70
90
|
| `extractcontent` | structured extraction |
|
|
71
|
-
| `serializeresult` | serialize
|
|
72
|
-
| `formatforagent` | summary, chunks
|
|
91
|
+
| `serializeresult` | serialize JSON, Markdown or XML results |
|
|
92
|
+
| `formatforagent` | summary, chunks and token count |
|
|
73
93
|
| `batchscrape` | bounded URL groups |
|
|
74
|
-
| `crawlurl` | crawl contract |
|
|
75
|
-
| `browseragent` | navigation, click, type
|
|
76
|
-
| `mcpserver` / `mcptransport` | MCP tools over JSONL
|
|
94
|
+
| `crawlurl` | crawl contract with domain and budget controls |
|
|
95
|
+
| `browseragent` | caller-owned navigation, click, type and screenshot actions |
|
|
96
|
+
| `mcpserver` / `mcptransport` | MCP tools over JSONL or HTTP |
|
|
77
97
|
| `nodeserver` | Web Request/Response handler |
|
|
98
|
+
| `engine` / `scheduler` | job lifecycle and runner dispatch |
|
|
99
|
+
| `release-assets` | SHA256SUMS, SBOM and provenance metadata for caller-selected artifacts |
|
|
78
100
|
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
## The execution model
|
|
82
|
-
|
|
83
|
-
Saddle coordinates contracts instead of hiding providers. A repo + CI runner is a virtual processor:
|
|
84
|
-
|
|
85
|
-
- Repo = Disk (persistent state)
|
|
86
|
-
- CI = CPU (workflow_dispatch = function call)
|
|
87
|
-
- Pages = Bus + CDN
|
|
88
|
-
- Static site = BIOS
|
|
89
|
-
- repository_dispatch = IPC
|
|
90
|
-
|
|
91
|
-
```js
|
|
92
|
-
import { engine, eventbus, inprocess, scheduler } from "@wenathlan/saddle";
|
|
93
|
-
import { localmemory } from "@wenathlan/saddle/memory-node";
|
|
94
|
-
import { localstorage } from "@wenathlan/saddle/storage-node";
|
|
95
|
-
const events = eventbus();
|
|
96
|
-
const run = engine({
|
|
97
|
-
storage: localstorage("./.saddle-data"),
|
|
98
|
-
memory: localmemory(),
|
|
99
|
-
scheduler: scheduler([inprocess()]),
|
|
100
|
-
events
|
|
101
|
-
});
|
|
102
|
-
const result = await run.run(
|
|
103
|
-
{ name: "example", input: { value: 42 } },
|
|
104
|
-
({ job }) => ({ jobid: job.id, ok: true })
|
|
105
|
-
);
|
|
106
|
-
```
|
|
107
|
-
|
|
108
|
-
The caller still chooses how to provide `fetcher`, browser transport, persistence, proxy pool, captcha solver, webhook secret and remote credentials. The root entry is transport-neutral; Node filesystem and HTTP adapters are explicit subpaths such as `@wenathlan/saddle/storage-node`, `@wenathlan/saddle/memory-node`, `@wenathlan/saddle/server-node`, `@wenathlan/saddle/sessions-file` and `@wenathlan/saddle/queue-persistent`. Saddle does not embed secrets, fixed hosts or a mandatory cloud vendor.
|
|
101
|
+
The complete export map is documented in [`docs/libraryapi.md`](docs/libraryapi.md). The product index is in [`docs/productindex.md`](docs/productindex.md), and runnable examples are in [`docs/usage.md`](docs/usage.md).
|
|
109
102
|
|
|
110
103
|
## Browser extension
|
|
111
104
|
|
|
112
|
-
|
|
105
|
+
The extension is a pure JavaScript Manifest V3 reference surface in [`extension/`](extension/). It contains a popup, service worker, isolated content bridge, read-only page-world `pagefacts` boundary, snapshot diffs and persisted window/tab/frame context for explicit resume.
|
|
113
106
|
|
|
114
107
|
```bash
|
|
115
108
|
# load the unpacked extension from chrome://extensions
|
|
116
109
|
ls extension/manifest.json extension/worker.js extension/content.js extension/popup.html
|
|
117
110
|
|
|
118
|
-
# build an isolated
|
|
111
|
+
# build an isolated artifact using the version supplied by the caller or release tag
|
|
119
112
|
npm run extension:build -- --output build/extension
|
|
120
113
|
```
|
|
121
114
|
|
|
122
|
-
The
|
|
115
|
+
The base permission set is `activeTab`, `scripting` and `storage`. It does not request broad host permissions, cookies, `webRequest`, debugger access or arbitrary page code execution. Optional host escalation remains caller-owned. Releases attach `saddle-extension-<version>.zip`; cross-browser profiles remain adapter work.
|
|
123
116
|
|
|
124
|
-
##
|
|
117
|
+
## Security boundaries
|
|
118
|
+
|
|
119
|
+
| Boundary | Policy |
|
|
120
|
+
| --- | --- |
|
|
121
|
+
| Credentials | injected by the caller or repository secret; never committed or printed |
|
|
122
|
+
| Network | HTTP/HTTPS targets are validated; private-target access remains caller policy |
|
|
123
|
+
| Crawling | robots rules, crawl delay, limits and budgets are explicit |
|
|
124
|
+
| Storage | adapters are replaceable; the core does not own a provider account |
|
|
125
|
+
| Runtime | Node-only filesystem, HTTP, Playwright and release metadata stay outside the transport-neutral root |
|
|
126
|
+
| Extension | page-world reads are bounded, token-correlated and read-only |
|
|
127
|
+
| Failure | retry, circuit breaker, idempotency and resume are configurable |
|
|
128
|
+
| Releases | version comes from the `vX.Y.Z` tag and must match `package.json` |
|
|
129
|
+
|
|
130
|
+
Version 1.8.7 also removes the obsolete nested `scrape` package manifests and lockfile that generated a separate stale dependency graph. The dependency-free JavaScript scrape contracts remain in `scrape/`. The root lockfile is regenerated and CI runs `npm audit --audit-level=high` plus dependency review for pull requests. See [`docs/securityaudit-1.8.7.md`](docs/securityaudit-1.8.7.md) for the baseline and remediation record.
|
|
131
|
+
|
|
132
|
+
## Package surfaces and release automation
|
|
133
|
+
|
|
134
|
+
Workflows use the release tag and the local `releaseversion` action. They do not contain a manually edited version number. The action fetches the tag, checks out its commit and rejects a release when the tag version does not match the root `package.json`.
|
|
135
|
+
|
|
136
|
+
| Registry | Artifact | Workflow |
|
|
137
|
+
| --- | --- | --- |
|
|
138
|
+
| GitHub Packages npm | `@wenathlan/saddle@<version>` | `publishgithubnpm.yml` |
|
|
139
|
+
| Public npmjs | `@wenathlan/saddle@<version>` | `publishnpmjs.yml` |
|
|
140
|
+
| GHCR | `ghcr.io/wenathlan/saddle:<version>` | `publishghcr.yml` |
|
|
141
|
+
| Maven | `io.wenathlan:saddle:<version>` | `publishmaven.yml` |
|
|
142
|
+
| NuGet | `Saddle.<version>.nupkg` | `publishnuget.yml` |
|
|
143
|
+
| RubyGems | `saddle <version>` | `publishrubygems.yml` |
|
|
144
|
+
|
|
145
|
+
Release assets are caller-selected and deterministic: `SHA256SUMS`, `sbom.cdx.json` in CycloneDX 1.5 shape and `provenance.intoto.jsonl` in an in-toto statement shape. The adapter does not publish, authenticate or choose a registry. The npm token previously sent in chat is compromised and must never be used; public npmjs publication uses only the owner-managed `NPM_TOKEN` repository secret.
|
|
146
|
+
|
|
147
|
+
## GitHub Pages web surface
|
|
148
|
+
|
|
149
|
+
The marketing site lives under [`web/`](web/) with a root-based TypeScript/React layout. It has no `client/` or `src/` subdirectory. Vite normalizes the base path and all visual assets resolve through a shared helper, so the same build works at `/` and `/saddle/`.
|
|
125
150
|
|
|
126
151
|
```bash
|
|
127
|
-
|
|
128
|
-
saddle
|
|
129
|
-
saddle runexample
|
|
130
|
-
saddle mcp
|
|
152
|
+
npm run web:check
|
|
153
|
+
VITE_BASE_PATH=/saddle npm run web:build:pages
|
|
131
154
|
```
|
|
132
155
|
|
|
133
|
-
|
|
156
|
+
Small public configuration and visual assets live under `web/public/`. The development collector is `web/public/debugcollector.js` and uses `/debuglogs`; it is not part of the production build. The obsolete `web/public/__manus__` directory is intentionally absent.
|
|
134
157
|
|
|
135
|
-
|
|
136
|
-
| --- | --- |
|
|
137
|
-
| Credentials | injected at runtime; never committed |
|
|
138
|
-
| Network | http/https validated; private targets blocked |
|
|
139
|
-
| Crawling | robots rules and crawl delay explicit |
|
|
140
|
-
| Storage | adapters replaceable |
|
|
141
|
-
| Runtime | Node HTTP isolated |
|
|
142
|
-
| Failure | retry, circuit breaker, idempotency configurable |
|
|
143
|
-
|
|
144
|
-
## Package surfaces
|
|
145
|
-
|
|
146
|
-
| Registry | Artifact | Workflow | Status |
|
|
147
|
-
| --- | --- | --- | --- |
|
|
148
|
-
| GitHub npm | `@wenathlan/saddle@1.8.6` | publishgithubnpm.yml | pending release |
|
|
149
|
-
| GHCR | `ghcr.io/wenathlan/saddle:1.8.6` and `latest` | publishghcr.yml | pending release |
|
|
150
|
-
| Maven | `io.wenathlan:saddle:1.8.6` | publishmaven.yml | pending release |
|
|
151
|
-
| NuGet | `Saddle.1.8.6.nupkg` | publishnuget.yml | pending release |
|
|
152
|
-
| RubyGems | `saddle 1.8.6` | publishrubygems.yml | pending release |
|
|
153
|
-
| npmjs | `@wenathlan/saddle@1.8.6` | publishnpmjs.yml | pending release |
|
|
154
|
-
|
|
155
|
-
## Development
|
|
158
|
+
## Development and release gates
|
|
156
159
|
|
|
157
160
|
```bash
|
|
158
161
|
npm ci
|
|
159
|
-
npm test
|
|
160
162
|
npm run check
|
|
161
163
|
npm run formatcheck
|
|
164
|
+
npm test
|
|
162
165
|
npm run pack:check
|
|
166
|
+
npm audit --audit-level=high
|
|
167
|
+
npm run web:check
|
|
168
|
+
VITE_BASE_PATH=/saddle npm run web:build:pages
|
|
163
169
|
```
|
|
164
170
|
|
|
165
|
-
|
|
171
|
+
The engine test suite is deterministic and does not require real credentials or network access. The release path is: update `package.json` and the manifest files, update `changelog.md`, run all gates, create `v<package-version>`, push the tag and create the GitHub release. Registry workflows then derive the same version from that release tag.
|
|
166
172
|
|
|
167
173
|
## Repository map
|
|
168
174
|
|
|
169
|
-
```
|
|
170
|
-
core/ errors, events and
|
|
175
|
+
```text
|
|
176
|
+
core/ errors, events, identifiers and hashing
|
|
171
177
|
domain/ jobs, artifacts, sessions and providers
|
|
172
178
|
memory/ working-set bridge, modes, objects and transforms
|
|
173
179
|
storage/ local, chunked, remote and file-hosting adapters
|
|
174
|
-
scrape/ robots, cache, extraction, schema and
|
|
180
|
+
scrape/ dependency-free robots, cache, extraction, schema and normalization contracts
|
|
175
181
|
crawl/ URL normalization, crawler and persistent frontier
|
|
176
182
|
queue/ queue, idempotency, saga and recovery
|
|
177
|
-
browser/ fingerprint, session and
|
|
178
|
-
|
|
179
|
-
mcp/ optional server and JSONL/HTTP transport
|
|
183
|
+
browser/ fingerprint, session, agent and Playwright adapter contracts
|
|
184
|
+
extension/ Manifest V3 reference surface and packager
|
|
180
185
|
protocol/ JSON, NDJSON, SSE and block serializers
|
|
181
|
-
workflow/ manifests, templates and registry
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
186
|
+
workflow/ manifests, templates and registry contracts
|
|
187
|
+
packager/ package and publication plans
|
|
188
|
+
release/ checksums, SBOM and provenance metadata
|
|
189
|
+
web/ root-based static marketing site
|
|
190
|
+
tests/ deterministic engine and extension coverage
|
|
191
|
+
docs/ architecture, API, security, release and registry notes
|
|
185
192
|
```
|
|
186
193
|
|
|
187
|
-
|
|
194
|
+
The engine remains pure JavaScript ESM with JSDoc comments in English. The web surface is TypeScript/React, while the published library has no TypeScript build requirement and no hardcoded host, port or credential.
|
|
195
|
+
|
|
196
|
+
## Historical documentation
|
|
197
|
+
|
|
198
|
+
Earlier README snapshots remain in `docs/plans/README.md`, `docs/talks9/README.md` and `docs/talks9/README (2).md` as archival evidence. Their useful architecture ideas were consolidated here, while stale `@devthink`, `@iakadion`, `io.devthink`, Node 20/22, `client/src` and speculative provider quotas were not copied into the canonical contract.
|
|
188
199
|
|
|
189
200
|
## Current scope
|
|
190
201
|
|
|
191
|
-
Version 1.8.
|
|
202
|
+
Version 1.8.7 extends the 1.8.6 engine with dependency remediation, explicit security gates, base-aware Pages assets, removal of the obsolete public debug directory and consolidated documentation. Browser binaries, provider credentials, n8n host registration, persistent databases, captcha solvers and production deployment remain caller-selected adapters. Future work should extend contracts without coupling the core to one forge, registry, browser or storage vendor.
|
|
192
203
|
|
|
193
204
|
## License
|
|
194
205
|
|
package/docs/actionsincident.md
CHANGED
|
@@ -27,3 +27,7 @@ The repository metadata audit found `main` as the default branch and one open De
|
|
|
27
27
|
The repository homepage About editor was opened in the authenticated browser and saved with the canonical Saddle description. The repository API should now report that description instead of the placeholder `saddle`.
|
|
28
28
|
|
|
29
29
|
The six non-main Dependabot tips were preserved as `archive-dependabot-*` tags, their PRs were closed, and their branch refs were removed. The cleanup leaves `main` as the only active branch without deleting the archived commit objects.
|
|
30
|
+
|
|
31
|
+
The first `v1.8.6` release fan-out passed release validation, GitHub Packages npm, public npmjs, Maven, NuGet, RubyGems and extension packaging. GHCR alone failed because the Docker image ran `npm ci --omit=dev` against the root manifest, whose dev-only Vite peer graph is rejected by npm's strict peer resolver. The corrective Dockerfile now sets `NPM_CONFIG_LEGACY_PEER_DEPS=true` and passes `--legacy-peer-deps`; the GHCR workflow also checks out the release tag for both release and manual dispatch paths.
|
|
32
|
+
|
|
33
|
+
The corrected manual GHCR run [31706464064](https://github.com/wenathlan/saddle/actions/runs/31706464064) completed successfully, including the Docker build and push. The six registry outcomes for `v1.8.6` are therefore green: GitHub Packages npm, public npmjs, Maven, NuGet, RubyGems and GHCR. The Pages deployment remains green in [run 31705301175](https://github.com/wenathlan/saddle/actions/runs/31705301175).
|
package/docs/release.md
CHANGED
|
@@ -6,10 +6,10 @@ The release path is intentionally split into source validation, package validati
|
|
|
6
6
|
|
|
7
7
|
| step | owner | condition |
|
|
8
8
|
|---|---|---|
|
|
9
|
-
| package version | repository | `package.json`
|
|
9
|
+
| package version | repository | `package.json` matches the release tag, for example `1.8.7` |
|
|
10
10
|
| quality gate | GitHub Actions | `npm run pack:check` passes |
|
|
11
|
-
| tag | repository owner | tag `v1.8.
|
|
12
|
-
| GitHub release | repository owner | release `v1.8.
|
|
11
|
+
| tag | repository owner | tag `v1.8.7` points to the validated release commit |
|
|
12
|
+
| GitHub release | repository owner | release `v1.8.7` is created from the validated tag |
|
|
13
13
|
| GitHub Packages | GitHub Actions | `publishgithubnpm.yml`, `publishghcr.yml`, `publishmaven.yml`, `publishnuget.yml`, and `publishrubygems.yml` use `GITHUB_TOKEN` |
|
|
14
14
|
| public npmjs | owner-managed GitHub Actions secret | `publishnpmjs.yml` uses `NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}` and derives the version from the release tag or latest release in manual maintenance runs |
|
|
15
15
|
| extension zip | GitHub Actions | `buildextension.yml` derives the version from the release tag, validates the unpacked artifact and attaches `saddle-extension-<version>.zip` |
|
|
@@ -22,8 +22,8 @@ The npm token previously sent in chat is compromised and must not be used. GitHu
|
|
|
22
22
|
|
|
23
23
|
```text
|
|
24
24
|
npm run pack:check
|
|
25
|
-
git tag v1.8.
|
|
26
|
-
git push origin v1.8.
|
|
25
|
+
git tag v1.8.7
|
|
26
|
+
git push origin v1.8.7
|
|
27
27
|
```
|
|
28
28
|
|
|
29
29
|
The release-created event is the publication and extension-asset trigger for the release workflows. A dry-run verifies package shape and local tests, but cannot verify registry ownership, Trusted Publisher configuration, package scope authorization, package visibility or browser-store submission; those remain settings controlled by the owner.
|
package/docs/releaseassets.md
CHANGED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# Security and web audit for 1.8.7
|
|
2
|
+
|
|
3
|
+
This audit records the baseline collected before remediation. The repository owner can cross-check the live GitHub view at [Saddle Security](https://github.com/wenathlan/saddle/security) and the [Dependabot alerts API](https://docs.github.com/en/rest/dependabot/alerts).
|
|
4
|
+
|
|
5
|
+
## Baseline
|
|
6
|
+
|
|
7
|
+
GitHub reported **42 open Dependabot alerts**: 2 critical, 14 high, 23 medium and 3 low. Five alerts were associated with the root `package-lock.json`; the remaining alerts were associated with the tracked `scrape/package-lock.json`. The largest groups were `undici`, `hono`, `vite`, `shell-quote`, `brace-expansion`, `postcss`, `ip-address` and `sharp` in the nested scrape manifest.
|
|
8
|
+
|
|
9
|
+
The local root `npm audit` reported 4 vulnerabilities in the current root installation: 2 moderate, 1 high and 1 critical. The root audit is a separate view from GitHub's repository-wide alert count and does not include the stale nested scrape dependency graph unless that directory is audited independently.
|
|
10
|
+
|
|
11
|
+
Code scanning returned no analysis found, and secret scanning returned that the feature is disabled. These are coverage gaps, not evidence that the repository has no code or secret findings. The remediation therefore includes enabling or documenting the appropriate GitHub security controls without fabricating a clean result.
|
|
12
|
+
|
|
13
|
+
## Web inventory
|
|
14
|
+
|
|
15
|
+
The requested directory `web/public/manos` does not exist. The actual platform directory is `web/public/__manus__`, containing `debug-collector.js`; it is a small runtime support directory rather than a product asset directory. The public visual assets are tracked under `web/public/assets/` as four WebP files.
|
|
16
|
+
|
|
17
|
+
The first path audit found root-absolute application entry and route paths, while the asset files themselves are in the correct public directory. The Pages site is served below `/saddle/`, so every asset and internal route must be resolved through the Vite base path rather than assuming `/`.
|
|
18
|
+
|
|
19
|
+
## Remediation policy
|
|
20
|
+
|
|
21
|
+
The 1.8.7 work will prioritize Node.js built-ins for new logic, update direct and transitive dependencies through the root lockfile, isolate or remove the obsolete nested scrape dependency graph when it is not part of the published engine, preserve the small `__manus__` support directory unless its script is proven unnecessary, and make web assets base-aware. A final audit will distinguish resolved advisories from external or unfixable advisories rather than hiding them.
|
package/extension/manifest.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"manifest_version": 3,
|
|
3
3
|
"name": "Saddle browser bridge",
|
|
4
|
-
"version": "1.8.
|
|
4
|
+
"version": "1.8.7",
|
|
5
5
|
"description": "User initiated page snapshots through the Saddle browser contract.",
|
|
6
6
|
"minimum_chrome_version": "110",
|
|
7
7
|
"permissions": ["activeTab", "scripting", "storage"],
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@wenathlan/saddle",
|
|
3
|
-
"version": "1.8.
|
|
3
|
+
"version": "1.8.7",
|
|
4
4
|
"description": "binary computing engine that turns distributed storage into a publishable working set",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"private": false,
|
|
@@ -178,7 +178,6 @@
|
|
|
178
178
|
"vaul": "^1.1.2",
|
|
179
179
|
"wouter": "^3.3.5",
|
|
180
180
|
"zod": "^4.1.12",
|
|
181
|
-
"@builder.io/vite-plugin-jsx-loc": "^0.1.1",
|
|
182
181
|
"@tailwindcss/typography": "^0.5.15",
|
|
183
182
|
"@tailwindcss/vite": "^4.1.3",
|
|
184
183
|
"@types/express": "4.17.21",
|
|
@@ -198,6 +197,6 @@
|
|
|
198
197
|
"typescript": "5.6.3",
|
|
199
198
|
"vite": "^7.1.7",
|
|
200
199
|
"vite-plugin-manus-runtime": "^0.0.58",
|
|
201
|
-
"vitest": "^
|
|
200
|
+
"vitest": "^4.1.10"
|
|
202
201
|
}
|
|
203
202
|
}
|
package/readme.txt
DELETED
|
@@ -1,163 +0,0 @@
|
|
|
1
|
-
SADDLE - README
|
|
2
|
-
Version 1.0, August 2026
|
|
3
|
-
|
|
4
|
-
Copyright (C) August 2026 devthink, nathlan, iakadion, nathu filho, allan neris, andraneris
|
|
5
|
-
Everyone is permitted to view this document, but changing it
|
|
6
|
-
is not allowed. This document is part of Project saddle.
|
|
7
|
-
|
|
8
|
-
Preamble
|
|
9
|
-
|
|
10
|
-
Project saddle unifies both README sources.
|
|
11
|
-
|
|
12
|
-
Saddle is a JavaScript ESM engine for jobs that move data between storage,
|
|
13
|
-
a working set, an injected runner and durable artifacts. It includes
|
|
14
|
-
contracts for scraping, crawling, browser agents, queues, persistence,
|
|
15
|
-
MCP transport, webhooks and package delivery.
|
|
16
|
-
|
|
17
|
-
Core thesis: storage bytes and compute-memory bytes are the same bytes.
|
|
18
|
-
A Node.js framework runs on other people's runners, loading storage
|
|
19
|
-
buckets as virtual RAM/GPU via storage->RAM bridge.
|
|
20
|
-
|
|
21
|
-
This README is the single source of truth combining Foundation, Engine,
|
|
22
|
-
and Productization sections from both original READMEs.
|
|
23
|
-
|
|
24
|
-
TERMS AND CONDITIONS
|
|
25
|
-
|
|
26
|
-
0. Overview.
|
|
27
|
-
|
|
28
|
-
See full readme.md for complete documentation, API, execution model,
|
|
29
|
-
CLI, security boundaries, package surfaces, development, and repository
|
|
30
|
-
map.
|
|
31
|
-
|
|
32
|
-
1. What is Included.
|
|
33
|
-
|
|
34
|
-
Jobs, Storage, Working set, Scraping, Crawl, Browser, Operations,
|
|
35
|
-
Protocols, Delivery, Agent Browser, Compute Backends, Storage Backends.
|
|
36
|
-
|
|
37
|
-
2. License.
|
|
38
|
-
|
|
39
|
-
Proprietary - View Only. See license.txt.
|
|
40
|
-
|
|
41
|
-
END OF TERMS AND CONDITIONS
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
# Saddle
|
|
45
|
-
|
|
46
|
-
<p align="center">
|
|
47
|
-
<img src="docs/assets/saddlemark.svg" alt="Saddle" width="720" />
|
|
48
|
-
</p>
|
|
49
|
-
|
|
50
|
-
<p align="center">
|
|
51
|
-
<strong>Storage-backed jobs, scraping contracts and portable runners for Node.js.</strong><br/>
|
|
52
|
-
<strong>Binary computing agent, agent browser, computer-use, scraper and packager.</strong><br/>
|
|
53
|
-
<a href="https://github.com/wenathlan/saddle/actions/workflows/ci.yml"><img src="https://github.com/wenathlan/saddle/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
|
|
54
|
-
<a href="https://github.com/wenathlan/saddle/releases/tag/v1.8.2"><img src="https://img.shields.io/badge/release-v1.8.2-d35d3d" alt="Release 1.8.2" /></a>
|
|
55
|
-
<a href="https://github.com/wenathlan/saddle/blob/main/license.md"><img src="https://img.shields.io/badge/license-Proprietary--View--Only-202a2f" alt="Proprietary View Only" /></a>
|
|
56
|
-
</p>
|
|
57
|
-
|
|
58
|
-
> **Core idea:** storage is the durable side of the working set; the runner is replaceable; the artifact is the boundary. **Storage == Compute** — RAM and disk are the same construct, differing only by usage flag.
|
|
59
|
-
|
|
60
|
-
Saddle is a **JavaScript ESM engine** for jobs that move data between storage, a working set, an injected runner and durable artifacts. It is also a **virtual machine you publish as a package** that runs on other people's computers (GitHub Actions, Forgejo, Gitea, GitLab, Codeberg, free Docker containers) and turns unlimited third-party storage buckets into virtual RAM/GPU/CPU. Nothing runs on the operator's local machine.
|
|
61
|
-
|
|
62
|
-
Ships as a library, CLI, binary, n8n node, CRX extension, Android/iOS and Tauri desktop app. The canonical JavaScript package is `@wenathlan/saddle`; GitHub Packages npm, Maven and GHCR use the transferred `wenathlan` owner namespace, while NuGet and RubyGems retain their unscoped ecosystem package names.
|
|
63
|
-
|
|
64
|
-
## Start here
|
|
65
|
-
|
|
66
|
-
Saddle requires **Node.js 22 or newer**.
|
|
67
|
-
|
|
68
|
-
```bash
|
|
69
|
-
npm install @wenathlan/saddle
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
```js
|
|
73
|
-
import { scrapeurl, formatforagent } from "@wenathlan/saddle";
|
|
74
|
-
|
|
75
|
-
const result = await scrapeurl("https://example.com", { format: "markdown" });
|
|
76
|
-
const context = formatforagent(result, { maxchunksize: 2000, keypoints: 4 });
|
|
77
|
-
|
|
78
|
-
console.log(context.summary);
|
|
79
|
-
```
|
|
80
|
-
|
|
81
|
-
Deterministic example with no network:
|
|
82
|
-
|
|
83
|
-
```bash
|
|
84
|
-
node examples/publicapi.js
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
## What is included
|
|
88
|
-
|
|
89
|
-
| Area | Contract | Result |
|
|
90
|
-
| --- | --- | --- |
|
|
91
|
-
| Jobs | `engine`, `scheduler`, `inprocess` | `prepare → process → sync → cleanup` |
|
|
92
|
-
| Storage | local, chunked, S3-compatible, GitHub Contents, file hosting | durable objects and chunks |
|
|
93
|
-
| Working set | memory bridge, modes, objects, transforms | storage-to-compute and compute-to-storage |
|
|
94
|
-
| Scraping | robots, cache, extraction, schema, scraper | text, metadata, links and structured output |
|
|
95
|
-
| Crawl | normalization, BFS crawler, persistent frontier | bounded domain-aware crawling |
|
|
96
|
-
| Browser | fingerprint, session, replay and injected agent | browser actions without vendor lock-in |
|
|
97
|
-
| Operations | queues, idempotency, saga, retry, circuit breaker | controlled execution and recovery |
|
|
98
|
-
| Protocols | JSON, NDJSON, SSE, blocks and MCP | transport-neutral messages |
|
|
99
|
-
| Delivery | manifests, workflow registry, binary/container plans | package and runner surfaces |
|
|
100
|
-
| Agent Browser | capture & replay, stealth, fingerprint | Brave capture, movement replay, session recording |
|
|
101
|
-
| Compute Backends | github-actions, huggingface, gitlab-ci, kaggle, oracle-cloud | free runners chain |
|
|
102
|
-
| Storage Backends | HF, Kaggle, Terabox, R2, Telegram, Discord via rclone | unlimited disk as RAM |
|
|
103
|
-
|
|
104
|
-
## Public API
|
|
105
|
-
|
|
106
|
-
| Export | Purpose |
|
|
107
|
-
| --- | --- |
|
|
108
|
-
| `saddleurl` | choose fetch or injected browser path |
|
|
109
|
-
| `scrapeurl` | fetch one URL and extract |
|
|
110
|
-
| `scrapehtml` | extract from HTML without network |
|
|
111
|
-
| `extractcontent` | structured extraction |
|
|
112
|
-
| `serializeresult` | serialize as JSON, Markdown, XML |
|
|
113
|
-
| `formatforagent` | summary, chunks, token count |
|
|
114
|
-
| `batchscrape` | bounded URL groups |
|
|
115
|
-
| `crawlurl` | crawl contract |
|
|
116
|
-
| `browseragent` | navigation, click, type, screenshot |
|
|
117
|
-
| `mcpserver` / `mcptransport` | MCP tools over JSONL/HTTP |
|
|
118
|
-
| `nodeserver` | Web Request/Response handler |
|
|
119
|
-
|
|
120
|
-
Complete API: `docs/libraryapi.md`
|
|
121
|
-
|
|
122
|
-
## The execution model
|
|
123
|
-
|
|
124
|
-
Saddle coordinates contracts instead of hiding providers. A repo + CI runner is a virtual processor:
|
|
125
|
-
|
|
126
|
-
- Repo = Disk (persistent state)
|
|
127
|
-
- CI = CPU (workflow_dispatch = function call)
|
|
128
|
-
- Pages = Bus + CDN
|
|
129
|
-
- Static site = BIOS
|
|
130
|
-
- repository_dispatch = IPC
|
|
131
|
-
|
|
132
|
-
```js
|
|
133
|
-
import { engine, eventbus, inprocess, localmemory, localstorage, scheduler } from "@wenathlan/saddle";
|
|
134
|
-
const events = eventbus();
|
|
135
|
-
const run = engine({
|
|
136
|
-
storage: localstorage("./.saddle-data"),
|
|
137
|
-
memory: localmemory(),
|
|
138
|
-
scheduler: scheduler([inprocess()]),
|
|
139
|
-
events
|
|
140
|
-
});
|
|
141
|
-
const result = await run.run(
|
|
142
|
-
{ name: "example", input: { value: 42 } },
|
|
143
|
-
({ job }) => ({ jobid: job.id, ok: true })
|
|
144
|
-
);
|
|
145
|
-
```
|
|
146
|
-
|
|
147
|
-
## CLI
|
|
148
|
-
|
|
149
|
-
```bash
|
|
150
|
-
saddle help
|
|
151
|
-
saddle modes
|
|
152
|
-
saddle runexample
|
|
153
|
-
saddle mcp
|
|
154
|
-
saddle capture --url <url>
|
|
155
|
-
saddle bot --platform github --token $SBOT_TOKEN
|
|
156
|
-
saddle memory --load repo://owner/repo/path/file.json
|
|
157
|
-
saddle deploy --target netlify
|
|
158
|
-
```
|
|
159
|
-
|
|
160
|
-
## Security boundaries
|
|
161
|
-
|
|
162
|
-
| Boundary | Policy |
|
|
163
|
-
| --- | --- |
|