openclaw-workspace-sync 2.2.0 → 2.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +395 -262
- package/README.src.md +412 -262
- package/dist/backup-manager.d.ts +67 -0
- package/dist/backup-manager.d.ts.map +1 -0
- package/dist/backup-manager.js +520 -0
- package/dist/backup-manager.js.map +1 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +247 -27
- package/dist/index.js.map +1 -1
- package/dist/types.d.ts +82 -0
- package/dist/types.d.ts.map +1 -1
- package/docs/diagrams/mode-3.svg +42 -0
- package/openclaw.plugin.json +122 -1
- package/package.json +2 -2
- package/skills/workspace-sync/SKILL.md +57 -10
package/README.md
CHANGED
|
@@ -1,31 +1,108 @@
|
|
|
1
|
-
# OpenClaw Workspace
|
|
1
|
+
# OpenClaw Workspace Sync & Backup Plugin
|
|
2
2
|
|
|
3
|
-
Sync your OpenClaw agent workspace
|
|
3
|
+
Sync and back up your OpenClaw agent workspace to cloud storage via [rclone](https://rclone.org/).
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
**Sync** your workspace to Dropbox, Google Drive, OneDrive, S3, or [70+ providers](https://rclone.org/overview/) with mailbox, mirror, or bisync modes. **Back up** your entire agent system — workspace, config, sessions, memory — as encrypted snapshots to S3, R2, B2, or any rclone backend.
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## What's included
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
9
|
+
| Feature | What it does | Cost |
|
|
10
|
+
|---------|-------------|------|
|
|
11
|
+
| [**Sync**](#sync) | Live mirror of your workspace to/from cloud storage. Three modes: mailbox (safest), mirror, bisync. | Zero LLM cost — pure file ops |
|
|
12
|
+
| [**Encrypted Backup**](#encrypted-backups) | Streaming encrypted snapshots of your entire agent system to your own bucket. Automatic retention. | Zero LLM cost, zero extra disk |
|
|
13
|
+
|
|
14
|
+
Both features use rclone under the hood and share provider credentials. You can use sync alone, backup alone, or both together with different providers — e.g. sync to Dropbox for daily access, backup to R2 for disaster recovery.
|
|
15
|
+
|
|
16
|
+
## Install
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
openclaw plugins install openclaw-workspace-sync
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Or clone into your extensions directory:
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
cd ~/.openclaw/extensions
|
|
26
|
+
git clone https://github.com/ashbrener/openclaw-workspace-sync workspace-sync
|
|
27
|
+
cd workspace-sync && npm install --omit=dev
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Quick start
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
# Interactive setup wizard (recommended)
|
|
34
|
+
openclaw workspace-sync setup
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
The setup wizard guides you through:
|
|
38
|
+
1. Checking/installing rclone
|
|
39
|
+
2. Selecting cloud provider
|
|
40
|
+
3. Choosing sync mode
|
|
41
|
+
4. Dropbox app folder option (for scoped access)
|
|
42
|
+
5. Background sync interval
|
|
43
|
+
6. OAuth authorization
|
|
44
|
+
7. First sync
|
|
45
|
+
|
|
46
|
+
Or configure manually — see [Configuration](#configuration) below.
|
|
47
|
+
|
|
48
|
+
## Configuration
|
|
49
|
+
|
|
50
|
+
Add to your `openclaw.json`. The `sync` and `backup` blocks are independent — use one or both:
|
|
51
|
+
|
|
52
|
+
```json
|
|
53
|
+
{
|
|
54
|
+
"plugins": {
|
|
55
|
+
"entries": {
|
|
56
|
+
"openclaw-workspace-sync": {
|
|
57
|
+
"enabled": true,
|
|
58
|
+
"config": {
|
|
59
|
+
"sync": {
|
|
60
|
+
"provider": "dropbox",
|
|
61
|
+
"mode": "mailbox",
|
|
62
|
+
"remotePath": "",
|
|
63
|
+
"localPath": "/",
|
|
64
|
+
"interval": 180,
|
|
65
|
+
"onSessionStart": true,
|
|
66
|
+
"exclude": [".git/**", "node_modules/**", "*.log"]
|
|
67
|
+
},
|
|
68
|
+
"backup": {
|
|
69
|
+
"enabled": true,
|
|
70
|
+
"provider": "s3",
|
|
71
|
+
"bucket": "my-backups",
|
|
72
|
+
"prefix": "habibi/",
|
|
73
|
+
"interval": 86400,
|
|
74
|
+
"encrypt": true,
|
|
75
|
+
"passphrase": "${BACKUP_PASSPHRASE}",
|
|
76
|
+
"include": ["workspace", "config", "cron", "memory"],
|
|
77
|
+
"retain": 7
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
}
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
> **Flat format still works.** Putting `provider`, `mode`, etc. at the config root (without `sync`) is supported for backwards compatibility. The nested `{ sync, backup }` format is recommended for clarity.
|
|
12
87
|
|
|
13
|
-
|
|
88
|
+
---
|
|
14
89
|
|
|
15
|
-
|
|
90
|
+
## Sync
|
|
16
91
|
|
|
17
|
-
|
|
92
|
+
Live workspace mirroring via rclone. The remote gateway workspace is the **source of truth**. Changes made by the agent flow down to your local machine through cloud storage. You can send files to the agent through an optional inbox.
|
|
93
|
+
|
|
94
|
+
<p align="center">
|
|
95
|
+
<img src="https://raw.githubusercontent.com/ashbrener/openclaw-workspace-sync/main/docs/how-it-works.png" alt="How it works — Local Machine syncs to Cloud Provider syncs to Remote Gateway" width="600" />
|
|
96
|
+
</p>
|
|
18
97
|
|
|
19
98
|
<p align="center">
|
|
20
99
|
<img src="https://raw.githubusercontent.com/ashbrener/openclaw-workspace-sync/main/docs/architecture.png" alt="Plugin architecture — CLI, Hooks, and Sync Manager feed into rclone wrapper" width="600" />
|
|
21
100
|
</p>
|
|
22
101
|
|
|
23
|
-
|
|
102
|
+
### Sync modes (breaking change in v2.0)
|
|
24
103
|
|
|
25
104
|
**`mode` is now required.** Previous versions used bidirectional bisync implicitly. Starting with v2.0, you must explicitly set `"mode"` in your config. The plugin will refuse to start and log an error until `mode` is set. This prevents accidental data loss from an unexpected sync direction.
|
|
26
105
|
|
|
27
|
-
The plugin supports three sync modes. Choose the one that fits your workflow:
|
|
28
|
-
|
|
29
106
|
| Mode | Direction | Description |
|
|
30
107
|
|------|-----------|-------------|
|
|
31
108
|
| `mailbox` | Push + inbox/outbox | Workspace pushes to cloud; users drop files in `_outbox` to send them to the agent. **Safest.** |
|
|
@@ -34,7 +111,7 @@ The plugin supports three sync modes. Choose the one that fits your workflow:
|
|
|
34
111
|
|
|
35
112
|
**Upgrading from a previous version?** If you were using bisync before, add `"mode": "bisync"` to your config to preserve the existing behavior. For the safest option, use `"mode": "mailbox"`.
|
|
36
113
|
|
|
37
|
-
|
|
114
|
+
#### `mailbox` mode (recommended)
|
|
38
115
|
|
|
39
116
|
The agent workspace is the source of truth. Each sync cycle:
|
|
40
117
|
|
|
@@ -56,7 +133,7 @@ On startup, the plugin bootstraps both directories:
|
|
|
56
133
|
|
|
57
134
|
Because the push explicitly excludes `_inbox/**` and `_outbox/**`, there is no risk of sync loops or accidental overwrites. Files only flow in one direction through each channel.
|
|
58
135
|
|
|
59
|
-
|
|
136
|
+
##### Inbox notifications (optional)
|
|
60
137
|
|
|
61
138
|
By default, mailbox mode is silent — files land in `_inbox` without waking the agent. This keeps costs at zero.
|
|
62
139
|
|
|
@@ -79,7 +156,7 @@ This wakes the agent on its next heartbeat. The agent sees the message and can p
|
|
|
79
156
|
}
|
|
80
157
|
```
|
|
81
158
|
|
|
82
|
-
|
|
159
|
+
#### `mirror` mode
|
|
83
160
|
|
|
84
161
|
The agent workspace is the source of truth. Every sync cycle copies the latest workspace state down to your local folder. Local files outside the workspace are never sent up.
|
|
85
162
|
|
|
@@ -89,7 +166,7 @@ The agent workspace is the source of truth. Every sync cycle copies the latest w
|
|
|
89
166
|
|
|
90
167
|
This is safe: even if something goes wrong, only your local copy is affected — the workspace stays untouched.
|
|
91
168
|
|
|
92
|
-
|
|
169
|
+
##### `ingest` option (mirror mode only)
|
|
93
170
|
|
|
94
171
|
Want to send files to the agent while using mirror mode? Enable the `ingest` option. This creates a local `inbox/` folder (sibling to the sync folder) that syncs one-way **up** to the workspace. Drop a file in the inbox — it appears on the remote workspace. The inbox is separate from the mirror, so there is no risk of overwriting workspace files.
|
|
95
172
|
|
|
@@ -105,7 +182,7 @@ When enabled, a local `inbox/` folder syncs its contents to `<remotePath>/inbox/
|
|
|
105
182
|
|
|
106
183
|
> For a more robust file-exchange pattern, consider `mailbox` mode instead. Mailbox uses `rclone move` to drain files (deleting from the source after transfer), which prevents duplicates and is easier to reason about.
|
|
107
184
|
|
|
108
|
-
|
|
185
|
+
#### `bisync` mode (advanced)
|
|
109
186
|
|
|
110
187
|
Full bidirectional sync using rclone bisync. Changes on either side propagate to the other.
|
|
111
188
|
|
|
@@ -123,11 +200,99 @@ Use this only if you understand the trade-offs:
|
|
|
123
200
|
|
|
124
201
|
If you are running on a container platform, `mailbox` mode is strongly recommended.
|
|
125
202
|
|
|
126
|
-
|
|
203
|
+
### Sync config reference
|
|
204
|
+
|
|
205
|
+
| Key | Type | Default | Description |
|
|
206
|
+
|-----|------|---------|-------------|
|
|
207
|
+
| `provider` | string | `"off"` | `dropbox` \| `gdrive` \| `onedrive` \| `s3` \| `custom` \| `off` |
|
|
208
|
+
| `mode` | string | **required** | `mailbox` \| `mirror` \| `bisync` — see [Sync modes](#sync-modes-breaking-change-in-v20) |
|
|
209
|
+
| `ingest` | boolean | `false` | Enable local inbox for sending files to the agent (mirror mode only) |
|
|
210
|
+
| `ingestPath` | string | `"inbox"` | Local subfolder name for ingestion (relative to `localPath`) |
|
|
211
|
+
| `notifyOnInbox` | boolean | `false` | Wake the agent when files arrive in `_inbox` (mailbox mode). Off by default — enabling this costs LLM credits per notification. |
|
|
212
|
+
| `remotePath` | string | `"openclaw-share"` | Folder name in cloud storage |
|
|
213
|
+
| `localPath` | string | `"shared"` | Subfolder within workspace to sync |
|
|
214
|
+
| `interval` | number | `0` | Background sync interval in seconds (0 = manual only, min 60) |
|
|
215
|
+
| `timeout` | number | `1800` | Max seconds for a single rclone sync operation (min 60) |
|
|
216
|
+
| `onSessionStart` | boolean | `false` | Sync when an agent session begins |
|
|
217
|
+
| `onSessionEnd` | boolean | `false` | Sync when an agent session ends |
|
|
218
|
+
| `remoteName` | string | `"cloud"` | rclone remote name |
|
|
219
|
+
| `configPath` | string | auto | Path to rclone.conf |
|
|
220
|
+
| `conflictResolve` | string | `"newer"` | `newer` \| `local` \| `remote` (bisync only) |
|
|
221
|
+
| `exclude` | string[] | see below | Glob patterns to exclude |
|
|
222
|
+
| `copySymlinks` | boolean | `false` | Follow symlinks during sync |
|
|
223
|
+
|
|
224
|
+
Default excludes: `**/.DS_Store`
|
|
225
|
+
|
|
226
|
+
### Sync CLI commands
|
|
227
|
+
|
|
228
|
+
```bash
|
|
229
|
+
# Interactive setup wizard
|
|
230
|
+
openclaw workspace-sync setup
|
|
231
|
+
|
|
232
|
+
# Check sync status
|
|
233
|
+
openclaw workspace-sync status
|
|
234
|
+
|
|
235
|
+
# Sync (behavior depends on mode)
|
|
236
|
+
openclaw workspace-sync sync
|
|
237
|
+
|
|
238
|
+
# Preview changes without syncing
|
|
239
|
+
openclaw workspace-sync sync --dry-run
|
|
240
|
+
|
|
241
|
+
# One-way sync (explicit, overrides mode for this run)
|
|
242
|
+
openclaw workspace-sync sync --direction pull # remote -> local
|
|
243
|
+
openclaw workspace-sync sync --direction push # local -> remote
|
|
244
|
+
|
|
245
|
+
# Force re-establish bisync baseline (bisync mode only)
|
|
246
|
+
openclaw workspace-sync sync --resync
|
|
247
|
+
|
|
248
|
+
# Authorize with cloud provider
|
|
249
|
+
openclaw workspace-sync authorize
|
|
250
|
+
openclaw workspace-sync authorize --provider gdrive
|
|
251
|
+
|
|
252
|
+
# List remote files
|
|
253
|
+
openclaw workspace-sync list
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
### Auto-sync
|
|
257
|
+
|
|
258
|
+
#### Session hooks
|
|
259
|
+
|
|
260
|
+
Sync automatically when sessions start or end. These run during existing agent activity and incur zero LLM cost:
|
|
261
|
+
|
|
262
|
+
```json
|
|
263
|
+
{
|
|
264
|
+
"onSessionStart": true,
|
|
265
|
+
"onSessionEnd": false
|
|
266
|
+
}
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
#### Periodic background sync
|
|
270
|
+
|
|
271
|
+
Set `interval` to enable automatic background sync (in seconds):
|
|
272
|
+
|
|
273
|
+
```json
|
|
274
|
+
{
|
|
275
|
+
"interval": 300,
|
|
276
|
+
"timeout": 3600
|
|
277
|
+
}
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
The gateway runs sync in the background at this interval. Minimum interval is 60 seconds. The `timeout` controls how long each sync operation is allowed to run (default: 1800s / 30 min). Increase this for large workspaces or slow connections.
|
|
281
|
+
|
|
282
|
+
In `mailbox` mode, periodic sync pushes the workspace to the cloud and drains the `_outbox`. In `mirror` mode, periodic sync pulls the latest workspace state down to local. In `bisync` mode, it runs a full bidirectional sync.
|
|
283
|
+
|
|
284
|
+
#### External cron (alternative)
|
|
285
|
+
|
|
286
|
+
```bash
|
|
287
|
+
# Add to crontab (crontab -e)
|
|
288
|
+
*/5 * * * * openclaw workspace-sync sync >> /var/log/openclaw-sync.log 2>&1
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
### Before your first sync
|
|
127
292
|
|
|
128
293
|
Getting the initial state right prevents data loss. Each mode has different requirements:
|
|
129
294
|
|
|
130
|
-
|
|
295
|
+
#### `mailbox` mode — starting state
|
|
131
296
|
|
|
132
297
|
The first sync **pushes** your local workspace to the cloud. This means rclone makes cloud match local exactly — any files on cloud that don't exist locally will be **deleted**.
|
|
133
298
|
|
|
@@ -144,17 +309,17 @@ rclone sync <remote>:<path> /data/workspace/ --config <config-path> \
|
|
|
144
309
|
|
|
145
310
|
Then verify local matches remote before enabling the plugin.
|
|
146
311
|
|
|
147
|
-
|
|
312
|
+
#### `mirror` mode — starting state
|
|
148
313
|
|
|
149
314
|
The first sync **pulls** from cloud to local. Local can be empty, stale, or corrupted — the pull simply overwrites it. **No preparation needed.**
|
|
150
315
|
|
|
151
|
-
|
|
316
|
+
#### `bisync` mode — starting state
|
|
152
317
|
|
|
153
318
|
The first sync requires `--resync`, which copies everything from both sides to the other. Any stale or unwanted files on either side will propagate.
|
|
154
319
|
|
|
155
320
|
**Recommended starting state:** Both sides are identical, or one side is empty and the other has the data you want. Verify both before running `--resync`.
|
|
156
321
|
|
|
157
|
-
|
|
322
|
+
#### General first-sync checklist
|
|
158
323
|
|
|
159
324
|
1. Run a `--dry-run` first to see what would happen: `openclaw workspace-sync sync --dry-run`
|
|
160
325
|
2. Check the output for unexpected deletions
|
|
@@ -163,104 +328,237 @@ The first sync requires `--resync`, which copies everything from both sides to t
|
|
|
163
328
|
|
|
164
329
|
> For maintenance, recovery, and common problems, see [TROUBLESHOOTING.md](./TROUBLESHOOTING.md).
|
|
165
330
|
|
|
166
|
-
|
|
331
|
+
### Sync safety
|
|
167
332
|
|
|
168
|
-
|
|
333
|
+
Cloud sync involves two copies of your data. When things go wrong, one side can overwrite the other. Here is what to keep in mind:
|
|
169
334
|
|
|
170
|
-
|
|
171
|
-
2. **Verify the remote** is accessible: `openclaw workspace-sync status`
|
|
172
|
-
3. **Run a dry-run first** to see what would happen: `openclaw workspace-sync sync --dry-run`
|
|
173
|
-
4. **Run the first sync**: `openclaw workspace-sync sync`
|
|
174
|
-
- In `mailbox` mode, this pushes the workspace and drains the `_outbox`
|
|
175
|
-
- In `mirror` mode, this pulls the current workspace down
|
|
176
|
-
- In `bisync` mode, this requires `--resync` to establish the baseline
|
|
177
|
-
5. **Enable periodic sync** by setting `interval` in your config
|
|
335
|
+
**Mailbox mode is the safest.** The workspace pushes to the cloud; users send files via `_outbox`. The two streams never overlap. Even if your local folder is wiped, the next push re-creates everything. Even if the `_outbox` has stale files, they just land in `_inbox` for the agent to handle.
|
|
178
336
|
|
|
179
|
-
|
|
337
|
+
**Mirror mode is safe by design.** The remote workspace is the authority. Local is a read-only copy. Even if your local folder is empty, stale, or corrupted, the next sync just re-downloads the workspace. The agent's work is never affected by local state.
|
|
180
338
|
|
|
181
|
-
|
|
339
|
+
**Bisync requires both sides to agree.** Bisync tracks what changed since the last sync. If that tracking state is lost (deploy, disk wipe, moving to a new machine), rclone does not know what changed and requires a `--resync`. A resync copies everything from both sides — if one side has stale or unwanted files, they propagate to the other.
|
|
340
|
+
|
|
341
|
+
**Common pitfalls to avoid:**
|
|
342
|
+
- Changing `remotePath` or `localPath` while periodic sync is enabled
|
|
343
|
+
- Running `--resync` without checking both sides first
|
|
344
|
+
- Using `bisync` on container platforms where state is ephemeral
|
|
345
|
+
- Syncing very large directories (use `exclude` patterns liberally)
|
|
346
|
+
|
|
347
|
+
**If in doubt, use `mailbox` mode.** It gives you a live local mirror of the workspace and a clean way to send files to the agent, with no risk of data loss.
|
|
348
|
+
|
|
349
|
+
> For recovery procedures, mode switching, and maintenance tips, see [TROUBLESHOOTING.md](./TROUBLESHOOTING.md).
|
|
350
|
+
|
|
351
|
+
#### Important: `--resync` is destructive (bisync only)
|
|
352
|
+
|
|
353
|
+
**Never use `--resync` unless you know exactly what it does.** The `--resync` flag tells rclone to throw away its knowledge of what has changed and do a full reconciliation — it copies every file that exists on either side to the other side. This means:
|
|
354
|
+
|
|
355
|
+
- Files you deleted remotely will come back from local (and vice versa)
|
|
356
|
+
- It transfers your **entire** sync scope, not just recent changes
|
|
357
|
+
- On a large Dropbox, this can take 30+ minutes and fill your disk
|
|
358
|
+
|
|
359
|
+
Normal bisync (without `--resync`) only transfers files that changed since the last sync. The plugin **never** auto-resyncs. If bisync's internal state gets corrupted, it will log a message telling you to run `--resync` manually — but only do this after confirming both sides are in the state you want.
|
|
182
360
|
|
|
183
361
|
```bash
|
|
184
|
-
|
|
362
|
+
# Only when you explicitly need to re-establish the baseline:
|
|
363
|
+
openclaw workspace-sync sync --resync
|
|
185
364
|
```
|
|
186
365
|
|
|
187
|
-
|
|
366
|
+
### Sync troubleshooting
|
|
367
|
+
|
|
368
|
+
#### Token expired
|
|
188
369
|
|
|
189
370
|
```bash
|
|
190
|
-
|
|
191
|
-
git clone https://github.com/ashbrener/openclaw-workspace-sync workspace-sync
|
|
192
|
-
cd workspace-sync && npm install --omit=dev
|
|
371
|
+
openclaw workspace-sync authorize
|
|
193
372
|
```
|
|
194
373
|
|
|
195
|
-
|
|
374
|
+
#### Conflicts (bisync only)
|
|
375
|
+
|
|
376
|
+
Files modified on both sides get a `.conflict` suffix. The winner is determined by `conflictResolve` (default: `newer`). To find conflict files:
|
|
196
377
|
|
|
197
378
|
```bash
|
|
198
|
-
|
|
199
|
-
openclaw workspace-sync setup
|
|
379
|
+
find <workspace>/shared -name "*.conflict"
|
|
200
380
|
```
|
|
201
381
|
|
|
202
|
-
|
|
203
|
-
1. Checking/installing rclone
|
|
204
|
-
2. Selecting cloud provider
|
|
205
|
-
3. Choosing sync mode
|
|
206
|
-
4. Dropbox app folder option (for scoped access)
|
|
207
|
-
5. Background sync interval
|
|
208
|
-
6. OAuth authorization
|
|
209
|
-
7. First sync
|
|
382
|
+
#### Stale lock files
|
|
210
383
|
|
|
211
|
-
|
|
384
|
+
The plugin automatically handles stale rclone lock files. If a sync is interrupted (timeout, crash, kill), the next run detects the stale lock, clears it, and retries. Lock files older than 15 minutes are treated as expired by rclone's `--max-lock` flag.
|
|
212
385
|
|
|
213
|
-
|
|
386
|
+
If you still see lock errors, you can manually clear them:
|
|
214
387
|
|
|
215
|
-
|
|
388
|
+
```bash
|
|
389
|
+
rclone deletefile ~/.cache/rclone/bisync/<lockfile>.lck
|
|
390
|
+
```
|
|
391
|
+
|
|
392
|
+
#### Sync times out
|
|
393
|
+
|
|
394
|
+
Increase the `timeout` config (in seconds). The default is 1800 (30 min). For large workspaces:
|
|
216
395
|
|
|
217
396
|
```json
|
|
218
397
|
{
|
|
219
|
-
"
|
|
220
|
-
"entries": {
|
|
221
|
-
"openclaw-workspace-sync": {
|
|
222
|
-
"enabled": true,
|
|
223
|
-
"config": {
|
|
224
|
-
"provider": "dropbox",
|
|
225
|
-
"mode": "mailbox",
|
|
226
|
-
"remotePath": "",
|
|
227
|
-
"localPath": "/",
|
|
228
|
-
"interval": 60,
|
|
229
|
-
"timeout": 1800,
|
|
230
|
-
"onSessionStart": true,
|
|
231
|
-
"onSessionEnd": false,
|
|
232
|
-
"exclude": [".git/**", "node_modules/**", "*.log"]
|
|
233
|
-
}
|
|
234
|
-
}
|
|
235
|
-
}
|
|
236
|
-
}
|
|
398
|
+
"timeout": 3600
|
|
237
399
|
}
|
|
238
400
|
```
|
|
239
401
|
|
|
240
|
-
|
|
402
|
+
#### Permission errors
|
|
403
|
+
|
|
404
|
+
```bash
|
|
405
|
+
chmod -R 755 <workspace>/shared
|
|
406
|
+
```
|
|
407
|
+
|
|
408
|
+
#### Dropbox rate limiting
|
|
409
|
+
|
|
410
|
+
Dropbox enforces API rate limits (`too_many_requests`). If your workspace has many files (10k+), each sync cycle can consume a large number of API calls just for checking. To avoid hitting limits:
|
|
411
|
+
|
|
412
|
+
- **Set `interval` high enough** for the sync to complete between cycles. A workspace with ~40k files takes ~2 minutes to scan. An `interval` of 180 (3 min) is the minimum; 300 (5 min) is safer.
|
|
413
|
+
- **Use `exclude` patterns liberally** — skip `node_modules`, `.git`, `__pycache__`, build output, and anything you don't need synced. Fewer files = fewer API calls.
|
|
414
|
+
- **If you see `too_many_requests` errors** in the logs, increase the interval and add more excludes.
|
|
415
|
+
|
|
416
|
+
---
|
|
417
|
+
|
|
418
|
+
## Encrypted backups
|
|
419
|
+
|
|
420
|
+
Back up your entire agent system — workspace, config, cron jobs, memory, sessions — as encrypted snapshots to your own cloud storage. Your bucket, your encryption key, zero monthly fees.
|
|
421
|
+
|
|
422
|
+
### How it works
|
|
423
|
+
|
|
424
|
+
<p align="center">
|
|
425
|
+
<img src="https://raw.githubusercontent.com/ashbrener/openclaw-workspace-sync/main/docs/diagrams/mode-3.svg" alt="backup pipeline diagram" width="700" />
|
|
426
|
+
</p>
|
|
427
|
+
|
|
428
|
+
1. **Streams directly** — `tar | [openssl enc] | rclone rcat` piped straight to the remote. Zero local temp files, zero extra disk needed. A 10 GB workspace on a 1 GB free volume? No problem.
|
|
429
|
+
2. Optionally encrypts with AES-256 (client-side, before upload) via `openssl`
|
|
430
|
+
3. Uploads via rclone to any supported provider (S3, R2, Backblaze B2, Dropbox, etc.)
|
|
431
|
+
4. Prunes old snapshots based on your retention policy
|
|
432
|
+
|
|
433
|
+
> **Disk-constrained?** Because backups stream directly, you don't need any free disk space for the backup itself. Only the restore downloads to a staging directory.
|
|
434
|
+
|
|
435
|
+
### Backup config reference
|
|
241
436
|
|
|
242
437
|
| Key | Type | Default | Description |
|
|
243
438
|
|-----|------|---------|-------------|
|
|
244
|
-
| `
|
|
245
|
-
| `
|
|
246
|
-
| `
|
|
247
|
-
| `
|
|
248
|
-
| `
|
|
249
|
-
| `
|
|
250
|
-
| `
|
|
251
|
-
| `
|
|
252
|
-
| `
|
|
253
|
-
| `
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
|
258
|
-
|
|
259
|
-
| `
|
|
439
|
+
| `enabled` | boolean | `false` | Enable scheduled backups |
|
|
440
|
+
| `provider` | string | parent provider | Cloud provider for backup storage |
|
|
441
|
+
| `bucket` | string | — | S3/R2 bucket name |
|
|
442
|
+
| `prefix` | string | `""` | Path prefix within the bucket (e.g., `habibi/`) |
|
|
443
|
+
| `interval` | number | `86400` | Backup interval in seconds (86400 = daily; clamped to min 300) |
|
|
444
|
+
| `encrypt` | boolean | `false` | Encrypt snapshots with AES-256 before upload |
|
|
445
|
+
| `passphrase` | string | — | Encryption passphrase (use `${BACKUP_PASSPHRASE}` env var) |
|
|
446
|
+
| `include` | string[] | see below | What to back up |
|
|
447
|
+
| `retain` | number or object | `7` | Retention: `7` = keep 7 latest, or `{ daily: 7, weekly: 4 }` |
|
|
448
|
+
| `exclude` | string[] | parent excludes | Glob patterns to exclude from workspace backup |
|
|
449
|
+
|
|
450
|
+
### Include options
|
|
451
|
+
|
|
452
|
+
| Item | What gets backed up |
|
|
453
|
+
|------|---------------------|
|
|
454
|
+
| `workspace` | The workspace directory (agent files, projects, etc.) |
|
|
455
|
+
| `config` | `openclaw.json` |
|
|
456
|
+
| `cron` | Cron job schedules and state |
|
|
457
|
+
| `memory` | Memory files (MEMORY.md, etc.) |
|
|
458
|
+
| `sessions` | Session metadata and store |
|
|
459
|
+
| `credentials` | Auth profile credentials |
|
|
460
|
+
| `skills` | Skill files |
|
|
461
|
+
| `hooks` | Webhook configurations and state (Gmail watch, custom hooks) |
|
|
462
|
+
| `extensions` | Installed plugins/extensions (for reproducible restores) |
|
|
463
|
+
| `env` | Environment variables file (`.env`) |
|
|
464
|
+
| `agents` | Multi-agent state (per-agent sessions, subagent registry) |
|
|
465
|
+
| `pages` | Custom pages served by the gateway |
|
|
466
|
+
| `transcripts` | Full conversation logs (JSONL session transcripts) |
|
|
467
|
+
|
|
468
|
+
Default: `["workspace", "config", "cron", "memory"]`
|
|
469
|
+
|
|
470
|
+
### Backup CLI commands
|
|
260
471
|
|
|
261
|
-
|
|
472
|
+
```bash
|
|
473
|
+
# Create a backup now
|
|
474
|
+
openclaw workspace-sync backup now
|
|
475
|
+
|
|
476
|
+
# List available snapshots
|
|
477
|
+
openclaw workspace-sync backup list
|
|
478
|
+
|
|
479
|
+
# Restore the latest snapshot
|
|
480
|
+
openclaw workspace-sync backup restore
|
|
481
|
+
|
|
482
|
+
# Restore a specific snapshot
|
|
483
|
+
openclaw workspace-sync backup restore --snapshot backup-20260310T020000Z.tar.gz.enc
|
|
484
|
+
|
|
485
|
+
# Restore to a specific directory (safe — doesn't overwrite live data)
|
|
486
|
+
openclaw workspace-sync backup restore --to /tmp/restore-test
|
|
487
|
+
|
|
488
|
+
# Check backup service status
|
|
489
|
+
openclaw workspace-sync backup status
|
|
490
|
+
```
|
|
491
|
+
|
|
492
|
+
### Restore safety
|
|
493
|
+
|
|
494
|
+
By default, `restore` extracts to a staging directory (`~/.openclaw/.backup-restore/`), not directly over your live workspace. This lets you inspect the contents before copying them into place. Use `--to` to control where files land.
|
|
262
495
|
|
|
263
|
-
###
|
|
496
|
+
### Backup provider examples
|
|
497
|
+
|
|
498
|
+
**Cloudflare R2 (free tier: 10GB):**
|
|
499
|
+
|
|
500
|
+
```json
|
|
501
|
+
{
|
|
502
|
+
"backup": {
|
|
503
|
+
"enabled": true,
|
|
504
|
+
"provider": "s3",
|
|
505
|
+
"encrypt": true,
|
|
506
|
+
"passphrase": "${BACKUP_PASSPHRASE}",
|
|
507
|
+
"s3": {
|
|
508
|
+
"endpoint": "https://<account-id>.r2.cloudflarestorage.com",
|
|
509
|
+
"bucket": "openclaw-backups",
|
|
510
|
+
"region": "auto",
|
|
511
|
+
"accessKeyId": "${R2_ACCESS_KEY}",
|
|
512
|
+
"secretAccessKey": "${R2_SECRET_KEY}"
|
|
513
|
+
}
|
|
514
|
+
}
|
|
515
|
+
}
|
|
516
|
+
```
|
|
517
|
+
|
|
518
|
+
**AWS S3:**
|
|
519
|
+
|
|
520
|
+
```json
|
|
521
|
+
{
|
|
522
|
+
"backup": {
|
|
523
|
+
"enabled": true,
|
|
524
|
+
"provider": "s3",
|
|
525
|
+
"encrypt": true,
|
|
526
|
+
"passphrase": "${BACKUP_PASSPHRASE}",
|
|
527
|
+
"s3": {
|
|
528
|
+
"bucket": "my-openclaw-backups",
|
|
529
|
+
"region": "us-east-1",
|
|
530
|
+
"accessKeyId": "${AWS_ACCESS_KEY_ID}",
|
|
531
|
+
"secretAccessKey": "${AWS_SECRET_ACCESS_KEY}"
|
|
532
|
+
}
|
|
533
|
+
}
|
|
534
|
+
}
|
|
535
|
+
```
|
|
536
|
+
|
|
537
|
+
**Backblaze B2:**
|
|
538
|
+
|
|
539
|
+
```json
|
|
540
|
+
{
|
|
541
|
+
"backup": {
|
|
542
|
+
"enabled": true,
|
|
543
|
+
"provider": "s3",
|
|
544
|
+
"encrypt": true,
|
|
545
|
+
"passphrase": "${BACKUP_PASSPHRASE}",
|
|
546
|
+
"s3": {
|
|
547
|
+
"endpoint": "https://s3.us-west-002.backblazeb2.com",
|
|
548
|
+
"bucket": "openclaw-backups",
|
|
549
|
+
"region": "us-west-002",
|
|
550
|
+
"accessKeyId": "${B2_KEY_ID}",
|
|
551
|
+
"secretAccessKey": "${B2_APP_KEY}"
|
|
552
|
+
}
|
|
553
|
+
}
|
|
554
|
+
}
|
|
555
|
+
```
|
|
556
|
+
|
|
557
|
+
---
|
|
558
|
+
|
|
559
|
+
## Provider-specific options
|
|
560
|
+
|
|
561
|
+
These provider blocks work for both sync and backup. Place them inside the `sync` or `backup` config block as needed.
|
|
264
562
|
|
|
265
563
|
**Dropbox with app folder (recommended for security):**
|
|
266
564
|
|
|
@@ -346,72 +644,7 @@ With `appFolder: true`, set `remotePath` to `""`. The app folder root is your sy
|
|
|
346
644
|
|
|
347
645
|
The `custom` provider accepts any [rclone backend type](https://rclone.org/overview/) and passes `rcloneOptions` directly to the rclone config. This gives you config-driven access to all 70+ providers without manually editing `rclone.conf`.
|
|
348
646
|
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
```bash
|
|
352
|
-
# Interactive setup wizard
|
|
353
|
-
openclaw workspace-sync setup
|
|
354
|
-
|
|
355
|
-
# Check sync status
|
|
356
|
-
openclaw workspace-sync status
|
|
357
|
-
|
|
358
|
-
# Sync (behavior depends on mode)
|
|
359
|
-
openclaw workspace-sync sync
|
|
360
|
-
|
|
361
|
-
# Preview changes without syncing
|
|
362
|
-
openclaw workspace-sync sync --dry-run
|
|
363
|
-
|
|
364
|
-
# One-way sync (explicit, overrides mode for this run)
|
|
365
|
-
openclaw workspace-sync sync --direction pull # remote -> local
|
|
366
|
-
openclaw workspace-sync sync --direction push # local -> remote
|
|
367
|
-
|
|
368
|
-
# Force re-establish bisync baseline (bisync mode only)
|
|
369
|
-
openclaw workspace-sync sync --resync
|
|
370
|
-
|
|
371
|
-
# Authorize with cloud provider
|
|
372
|
-
openclaw workspace-sync authorize
|
|
373
|
-
openclaw workspace-sync authorize --provider gdrive
|
|
374
|
-
|
|
375
|
-
# List remote files
|
|
376
|
-
openclaw workspace-sync list
|
|
377
|
-
```
|
|
378
|
-
|
|
379
|
-
## Auto-sync
|
|
380
|
-
|
|
381
|
-
### Session hooks
|
|
382
|
-
|
|
383
|
-
Sync automatically when sessions start or end. These run during existing agent activity and incur zero LLM cost:
|
|
384
|
-
|
|
385
|
-
```json
|
|
386
|
-
{
|
|
387
|
-
"onSessionStart": true,
|
|
388
|
-
"onSessionEnd": false
|
|
389
|
-
}
|
|
390
|
-
```
|
|
391
|
-
|
|
392
|
-
### Periodic background sync
|
|
393
|
-
|
|
394
|
-
Set `interval` to enable automatic background sync (in seconds):
|
|
395
|
-
|
|
396
|
-
```json
|
|
397
|
-
{
|
|
398
|
-
"interval": 300,
|
|
399
|
-
"timeout": 3600
|
|
400
|
-
}
|
|
401
|
-
```
|
|
402
|
-
|
|
403
|
-
The gateway runs sync in the background at this interval. Minimum interval is 60 seconds. The `timeout` controls how long each sync operation is allowed to run (default: 1800s / 30 min). Increase this for large workspaces or slow connections.
|
|
404
|
-
|
|
405
|
-
In `mailbox` mode, periodic sync pushes the workspace to the cloud and drains the `_outbox`. In `mirror` mode, periodic sync pulls the latest workspace state down to local. In `bisync` mode, it runs a full bidirectional sync.
|
|
406
|
-
|
|
407
|
-
### External cron (alternative)
|
|
408
|
-
|
|
409
|
-
```bash
|
|
410
|
-
# Add to crontab (crontab -e)
|
|
411
|
-
*/5 * * * * openclaw workspace-sync sync >> /var/log/openclaw-sync.log 2>&1
|
|
412
|
-
```
|
|
413
|
-
|
|
414
|
-
## Supported providers
|
|
647
|
+
### Supported providers
|
|
415
648
|
|
|
416
649
|
| Provider | Config value | Auth method | Config-driven |
|
|
417
650
|
|----------|-------------|-------------|---------------|
|
|
@@ -423,24 +656,7 @@ In `mailbox` mode, periodic sync pushes the workspace to the cloud and drains th
|
|
|
423
656
|
|
|
424
657
|
All providers are fully config-driven — no manual `rclone.conf` editing needed. The `custom` provider gives access to all [70+ rclone backends](https://rclone.org/overview/) (SFTP, B2, Mega, pCloud, Azure Blob, etc.).
|
|
425
658
|
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
If you prefer to skip the interactive wizard, configure the plugin in `openclaw.json` and use the CLI:
|
|
429
|
-
|
|
430
|
-
```bash
|
|
431
|
-
# 1. Authorize with your cloud provider
|
|
432
|
-
openclaw workspace-sync authorize --provider dropbox
|
|
433
|
-
|
|
434
|
-
# 2. Run a dry-run to preview what will sync
|
|
435
|
-
openclaw workspace-sync sync --dry-run
|
|
436
|
-
|
|
437
|
-
# 3. Run the first sync
|
|
438
|
-
openclaw workspace-sync sync
|
|
439
|
-
```
|
|
440
|
-
|
|
441
|
-
The plugin handles rclone installation, config generation, and token storage automatically based on your `openclaw.json` settings.
|
|
442
|
-
|
|
443
|
-
## Dropbox app folder access (recommended)
|
|
659
|
+
### Dropbox app folder access (recommended)
|
|
444
660
|
|
|
445
661
|
For better security, create a scoped Dropbox app that only accesses a single folder:
|
|
446
662
|
|
|
@@ -461,90 +677,7 @@ Benefits:
|
|
|
461
677
|
- If token is compromised, blast radius is limited
|
|
462
678
|
- Clean separation — sync folder lives under `Apps/<your-app-name>/`
|
|
463
679
|
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
Dropbox enforces API rate limits (`too_many_requests`). If your workspace has many files (10k+), each sync cycle can consume a large number of API calls just for checking. To avoid hitting limits:
|
|
467
|
-
|
|
468
|
-
- **Set `interval` high enough** for the sync to complete between cycles. A workspace with ~40k files takes ~2 minutes to scan. An `interval` of 180 (3 min) is the minimum; 300 (5 min) is safer.
|
|
469
|
-
- **Use `exclude` patterns liberally** — skip `node_modules`, `.git`, `__pycache__`, build output, and anything you don't need synced. Fewer files = fewer API calls.
|
|
470
|
-
- **If you see `too_many_requests` errors** in the logs, increase the interval and add more excludes.
|
|
471
|
-
|
|
472
|
-
## Understanding sync safety
|
|
473
|
-
|
|
474
|
-
Cloud sync involves two copies of your data. When things go wrong, one side can overwrite the other. Here is what to keep in mind:
|
|
475
|
-
|
|
476
|
-
**Mailbox mode is the safest.** The workspace pushes to the cloud; users send files via `_outbox`. The two streams never overlap. Even if your local folder is wiped, the next push re-creates everything. Even if the `_outbox` has stale files, they just land in `_inbox` for the agent to handle.
|
|
477
|
-
|
|
478
|
-
**Mirror mode is safe by design.** The remote workspace is the authority. Local is a read-only copy. Even if your local folder is empty, stale, or corrupted, the next sync just re-downloads the workspace. The agent's work is never affected by local state.
|
|
479
|
-
|
|
480
|
-
**Bisync requires both sides to agree.** Bisync tracks what changed since the last sync. If that tracking state is lost (deploy, disk wipe, moving to a new machine), rclone does not know what changed and requires a `--resync`. A resync copies everything from both sides — if one side has stale or unwanted files, they propagate to the other.
|
|
481
|
-
|
|
482
|
-
**Common pitfalls to avoid:**
|
|
483
|
-
- Changing `remotePath` or `localPath` while periodic sync is enabled
|
|
484
|
-
- Running `--resync` without checking both sides first
|
|
485
|
-
- Using `bisync` on container platforms where state is ephemeral
|
|
486
|
-
- Syncing very large directories (use `exclude` patterns liberally)
|
|
487
|
-
|
|
488
|
-
**If in doubt, use `mailbox` mode.** It gives you a live local mirror of the workspace and a clean way to send files to the agent, with no risk of data loss.
|
|
489
|
-
|
|
490
|
-
> For recovery procedures, mode switching, and maintenance tips, see [TROUBLESHOOTING.md](./TROUBLESHOOTING.md).
|
|
491
|
-
|
|
492
|
-
## Important: `--resync` is destructive (bisync only)
|
|
493
|
-
|
|
494
|
-
**Never use `--resync` unless you know exactly what it does.** The `--resync` flag tells rclone to throw away its knowledge of what has changed and do a full reconciliation — it copies every file that exists on either side to the other side. This means:
|
|
495
|
-
|
|
496
|
-
- Files you deleted remotely will come back from local (and vice versa)
|
|
497
|
-
- It transfers your **entire** sync scope, not just recent changes
|
|
498
|
-
- On a large Dropbox, this can take 30+ minutes and fill your disk
|
|
499
|
-
|
|
500
|
-
Normal bisync (without `--resync`) only transfers files that changed since the last sync. The plugin **never** auto-resyncs. If bisync's internal state gets corrupted, it will log a message telling you to run `--resync` manually — but only do this after confirming both sides are in the state you want.
|
|
501
|
-
|
|
502
|
-
```bash
|
|
503
|
-
# Only when you explicitly need to re-establish the baseline:
|
|
504
|
-
openclaw workspace-sync sync --resync
|
|
505
|
-
```
|
|
506
|
-
|
|
507
|
-
## Troubleshooting
|
|
508
|
-
|
|
509
|
-
### Token expired
|
|
510
|
-
|
|
511
|
-
```bash
|
|
512
|
-
openclaw workspace-sync authorize
|
|
513
|
-
```
|
|
514
|
-
|
|
515
|
-
### Conflicts (bisync only)
|
|
516
|
-
|
|
517
|
-
Files modified on both sides get a `.conflict` suffix. The winner is determined by `conflictResolve` (default: `newer`). To find conflict files:
|
|
518
|
-
|
|
519
|
-
```bash
|
|
520
|
-
find <workspace>/shared -name "*.conflict"
|
|
521
|
-
```
|
|
522
|
-
|
|
523
|
-
### Stale lock files
|
|
524
|
-
|
|
525
|
-
The plugin automatically handles stale rclone lock files. If a sync is interrupted (timeout, crash, kill), the next run detects the stale lock, clears it, and retries. Lock files older than 15 minutes are treated as expired by rclone's `--max-lock` flag.
|
|
526
|
-
|
|
527
|
-
If you still see lock errors, you can manually clear them:
|
|
528
|
-
|
|
529
|
-
```bash
|
|
530
|
-
rclone deletefile ~/.cache/rclone/bisync/<lockfile>.lck
|
|
531
|
-
```
|
|
532
|
-
|
|
533
|
-
### Sync times out
|
|
534
|
-
|
|
535
|
-
Increase the `timeout` config (in seconds). The default is 1800 (30 min). For large workspaces:
|
|
536
|
-
|
|
537
|
-
```json
|
|
538
|
-
{
|
|
539
|
-
"timeout": 3600
|
|
540
|
-
}
|
|
541
|
-
```
|
|
542
|
-
|
|
543
|
-
### Permission errors
|
|
544
|
-
|
|
545
|
-
```bash
|
|
546
|
-
chmod -R 755 <workspace>/shared
|
|
547
|
-
```
|
|
680
|
+
---
|
|
548
681
|
|
|
549
682
|
## Deployment recommendations
|
|
550
683
|
|
|
@@ -553,15 +686,15 @@ If you are running OpenClaw on a cloud container (Fly.io, Railway, Render) or a
|
|
|
553
686
|
- **Use a separate persistent volume for the workspace.** Container root filesystems are ephemeral — a redeploy wipes everything. Mount a dedicated volume (e.g., Fly.io volumes, EBS, DigitalOcean block storage) at your workspace path so data survives deploys and restarts.
|
|
554
687
|
- **Enable daily volume snapshots.** Most cloud providers offer automated snapshots (Fly.io does this by default with 5-day retention). If something goes wrong — a bad sync, accidental deletion, or a failed reorganization — a recent snapshot lets you restore in minutes instead of rebuilding from scratch.
|
|
555
688
|
- **Test your restore process.** A backup you have never restored is a backup you do not have. Create a volume from a snapshot at least once to confirm the process works and you know the steps.
|
|
556
|
-
|
|
557
|
-
These recommendations apply regardless of whether you use this plugin. Cloud sync adds convenience but is not a substitute for proper backups.
|
|
689
|
+
- **Use encrypted backups for off-site disaster recovery.** Volume snapshots protect against accidental deletes, but not against provider outages or account issues. Streaming encrypted backups to a separate provider (e.g., R2, B2) gives you a fully independent recovery path.
|
|
558
690
|
|
|
559
691
|
## Security notes
|
|
560
692
|
|
|
561
693
|
- **Token storage**: rclone tokens are stored in `rclone.conf` with `0600` permissions
|
|
562
694
|
- **Sensitive files**: Don't sync secrets, API keys, or credentials
|
|
563
|
-
- **Encryption**:
|
|
695
|
+
- **Encryption**: Backups support AES-256 client-side encryption. For sync, consider [rclone crypt](https://rclone.org/crypt/) for sensitive data.
|
|
564
696
|
- **App folder**: Use Dropbox app folder access for minimal permissions
|
|
697
|
+
- **Passphrase safety**: Use environment variables (`${BACKUP_PASSPHRASE}`) — never hardcode passphrases in config files
|
|
565
698
|
|
|
566
699
|
## Development
|
|
567
700
|
|