@pixlcore/xycli 1.0.8 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -3
- package/docs/help.md +21 -7
- package/docs/sync.md +85 -21
- package/docs/two-way-delete-tutorial.md +259 -0
- package/docs/two-way-git-tutorial.md +13 -7
- package/index.js +22 -1
- package/lib/apikey.js +1 -1
- package/lib/marketplace.js +1 -1
- package/lib/sync-buckets.js +35 -1
- package/lib/sync-syshook.js +207 -0
- package/lib/sync.js +161 -26
- package/lib/transfer.js +3 -3
- package/lib/utils.js +6 -2
- package/package.json +2 -2
- package/test/README.md +3 -3
- package/test/help.test.js +56 -0
- package/test/sync-buckets-unit.test.js +31 -0
- package/test/sync-buckets.test.js +36 -0
- package/test/sync-syshook-unit.test.js +231 -0
- package/test/sync-unit.test.js +215 -7
- package/test/sync.test.js +42 -0
- package/test/tickets.test.js +4 -1
- package/test/toast.test.js +3 -3
package/README.md
CHANGED
|
@@ -27,6 +27,7 @@ Browse the [screenshot gallery](docs/screenshots.md) for more views, or jump to
|
|
|
27
27
|
- [Global CLI switches](#global-cli-switches)
|
|
28
28
|
- [Tutorial: your first Event, job, and Web Hook](docs/tutorial.md)
|
|
29
29
|
- [Tutorial: two-way sync with Git](docs/two-way-git-tutorial.md)
|
|
30
|
+
- [Tutorial: two-way sync with event-driven deletes](docs/two-way-delete-tutorial.md)
|
|
30
31
|
- [Explore and manage xyOps](#explore-and-manage-xyops)
|
|
31
32
|
- [Transfer data between systems](#transfer-data-between-systems)
|
|
32
33
|
- [Sync your automation with local files](#sync-your-automation-with-local-files)
|
|
@@ -134,7 +135,7 @@ These switches control output or execution across many commands. Add them to the
|
|
|
134
135
|
| Switch | Short form | What it does |
|
|
135
136
|
| --- | --- | --- |
|
|
136
137
|
| `--verbose` | `-v` | Show extra detail. Resource views may expand scripts, parameters, or monitoring sections; many mutation commands also display API requests and responses. |
|
|
137
|
-
| `--quiet` | `-q` | Suppress ordinary terminal output and progress displays. JSON data still prints, and
|
|
138
|
+
| `--quiet` | `-q` | Suppress ordinary terminal output and progress displays. JSON data still prints, and human-readable warnings and errors remain visible on stderr. |
|
|
138
139
|
| `--dry` | | Preview a supported operation without applying it. Resource mutations, imports, exports, sync, and API calls use this to show planned changes or requests. It does not prevent connection checks or reads, and does not apply to local `xy config` updates. |
|
|
139
140
|
| `--format json` | `-f json` | Display supported results as indented JSON instead of the usual report or table. |
|
|
140
141
|
| `--format jsonc` | `-f jsonc` | Display supported results as compact JSON on one line. Here, `jsonc` means compact JSON. |
|
|
@@ -157,7 +158,7 @@ XYOPS_COLOR=false xy events --format json --quiet > ./events.json
|
|
|
157
158
|
xy categories -f jsonc -q > ./categories.json
|
|
158
159
|
```
|
|
159
160
|
|
|
160
|
-
|
|
161
|
+
Human-readable warnings and errors print to stderr even in quiet mode, so redirecting JSON output to a file leaves those diagnostics visible in your terminal. Structured JSON results may also include warnings and errors. Sync warnings and errors still produce a nonzero exit status. Check exit status and any structured results when scripting.
|
|
161
162
|
|
|
162
163
|
JSON output varies by command: some local collection views return all matches even when their table view is paginated, while database searches may return only the selected page. Some commands print multiple JSON values, such as the request and response from `xy api`. Check the command reference when using the output in a script.
|
|
163
164
|
|
|
@@ -513,10 +514,12 @@ The push above updates xyOps immediately. Remove `--dry` from the pull command t
|
|
|
513
514
|
|
|
514
515
|
Sync supports Alerts, API Keys, Categories, Channels, Events and workflows, Groups, Monitors, Plugins, Tags, and Web Hooks. It updates existing objects and files; use creation commands, imports, setup, or exports to introduce new ones. Two-way sync is experimental and uses modification times, so one-way sync is a better fit for a Git-based source of truth.
|
|
515
516
|
|
|
516
|
-
Delete mode
|
|
517
|
+
Delete mode works in one direction per run. Up-only deletion removes eligible xyOps objects missing from your local inventory; down-only deletion removes local XYPDF definitions and Bucket files missing from xyOps. Objects with a `stock` or `marketplace` property are protected from deletion. Keep a complete inventory on the authoritative side for each selected type, and review a dry run before applying it. Sync applies changes without a confirmation step unless you use `--dry`.
|
|
517
518
|
|
|
518
519
|
Follow the [Two-Way Git Sync Tutorial](docs/two-way-git-tutorial.md) to connect one persistent checkout to a shared repository, commit downloaded changes, discover new Events, and schedule the complete workflow with cron or xySat.
|
|
519
520
|
|
|
521
|
+
For an event-driven workflow that deletes in both directions, creates new definitions only in xyOps, and uses no recurring sync schedule, follow the [Two-Way Git Sync with Event-Driven Deletes Tutorial](docs/two-way-delete-tutorial.md).
|
|
522
|
+
|
|
520
523
|
Read the dedicated [Sync Guide](docs/sync.md) for setup, file layouts, every option, saved defaults, two-way sync, deletion, notifications, and automation with cron, Git hooks, and GitHub Actions.
|
|
521
524
|
|
|
522
525
|
- **Command reference:** [Sync commands](docs/help.md#sync) and [Sync setup](docs/help.md#sync-setup).
|
package/docs/help.md
CHANGED
|
@@ -188,7 +188,7 @@ Your configured permissions still apply during import. If an item fails, import
|
|
|
188
188
|
|
|
189
189
|
## sync
|
|
190
190
|
|
|
191
|
-
For setup, configuration, file layouts, and automation examples, see the dedicated [Filesystem Sync Guide](sync.md).
|
|
191
|
+
For setup, configuration, file layouts, and automation examples, see the dedicated [Filesystem Sync Guide](sync.md). For separate Git- and xyOps-triggered delete passes, see the [event-driven two-way delete tutorial](two-way-delete-tutorial.md).
|
|
192
192
|
|
|
193
193
|
Sync existing xyOps definitions with a directory of XYPDF files. Each run scans the directory and its subdirectories, shows any differences, and applies changes in the directions you select. **Changes are applied immediately unless you add `--dry`; there is no `--confirm` step.**
|
|
194
194
|
|
|
@@ -213,6 +213,7 @@ Pass one or more base directories, or omit them to scan the current directory. F
|
|
|
213
213
|
- `--down TYPES` updates local files from xyOps.
|
|
214
214
|
- `--dry` previews changes without applying them.
|
|
215
215
|
- `--verbose` shows full diffs and API requests and responses.
|
|
216
|
+
- `--ignore_missing` skips local sources whose type and ID are absent from xyOps, without a warning or failure. It leaves their files untouched.
|
|
216
217
|
|
|
217
218
|
You must enable at least one direction. `TYPES` is a comma-separated list of `alerts`, `api_keys`, `buckets`, `categories`, `channels`, `events`, `groups`, `monitors`, `plugins`, `tags`, or `web_hooks`. Use `all` to select every supported type. Workflows are included under `events`. Bucket sync requires xyOps v1.1.2 or later. Secrets, Users, and Roles are not supported.
|
|
218
219
|
|
|
@@ -224,20 +225,32 @@ Repeated sources with the same type and ID generate warnings, even if their cont
|
|
|
224
225
|
|
|
225
226
|
External string properties are loaded automatically from adjacent files named `BASENAME-PROPERTY.EXT`, such as `My-Plugin-script.js` beside `My-Plugin.json`, or `My-Event-params.script.sh` beside `My-Event.json`. The extension is your choice. Keep only one external file per property, and rename its basename along with the JSON file.
|
|
226
227
|
|
|
227
|
-
Bucket metadata uses a normal XYPDF source. For a Bucket titled `My Bucket Name`, its separate JSON data and uploaded files live beside `buckets/My-Bucket-Name.json` in `buckets/My-Bucket-Name/data.json` and `buckets/My-Bucket-Name/files/`. Both `data.json` and `files/` are required for a selected local bucket. Bucket data upsync replaces the entire remote data object. Bucket files are matched by xyOps filename normalization and transferred according to size and modification time. Two-way bucket sync uses the data record and file timestamps from xyOps; a transfer updates local file times to match.
|
|
228
|
+
Bucket metadata uses a normal XYPDF source. For a Bucket titled `My Bucket Name`, its separate JSON data and uploaded files live beside `buckets/My-Bucket-Name.json` in `buckets/My-Bucket-Name/data.json` and `buckets/My-Bucket-Name/files/`. Both `data.json` and `files/` are required for a selected local bucket. Bucket data upsync replaces the entire remote data object. Bucket files are matched by xyOps filename normalization and transferred according to size and modification time. Two-way bucket sync uses the data record and file timestamps from xyOps; a transfer updates local file times to match. Up-only `--delete buckets` removes remote files missing locally; down-only `--delete buckets` removes local files missing remotely.
|
|
228
229
|
|
|
229
230
|
**Delete mode**
|
|
230
231
|
|
|
231
|
-
Add `--delete TYPES` to
|
|
232
|
+
Add `--delete TYPES` to remove definitions missing from the authoritative side of a one-way sync. With `--up` only, it deletes xyOps objects missing from the local tree. With `--down` only, it deletes local XYPDF sources missing from xyOps, plus their external property files. The CLI rejects deletion when both directions or neither direction are enabled, including saved defaults and dry runs. Use `--up false` or `--down false` to disable an inherited direction. See [Delete mode](sync.md#delete-mode) before using it.
|
|
232
233
|
|
|
233
234
|
```sh
|
|
234
235
|
xy sync ./ --up events,plugins --delete events --dry
|
|
235
236
|
xy sync ./ --up events,plugins --delete events
|
|
237
|
+
xy sync ./ --up false --down events,plugins --delete events,plugins --dry
|
|
238
|
+
xy sync ./ --up false --down events,plugins --delete events,plugins
|
|
236
239
|
```
|
|
237
240
|
|
|
238
|
-
**Objects with a `stock` or `marketplace` property are
|
|
241
|
+
**Objects with a `stock` or `marketplace` property are ignored by deletion**, including during dry runs. Upward deletion checks the xyOps definition; downward deletion checks the local XYPDF. Setup inclusion flags cannot override this protection.
|
|
239
242
|
|
|
240
|
-
**The
|
|
243
|
+
**The authoritative side must have a complete inventory of the objects you intend to keep for every type selected for deletion.** Delete mode is not limited to previously synced objects. A missing local file can delete a remote object during upsync; a new local file not yet created in xyOps can be deleted during downsync. Ensure the API Key can see the complete remote inventory, review a dry run, and coordinate separate up and down jobs before enabling automatic deletion. Warnings or errors before the delete phase skip that phase; earlier successful updates remain in place.
|
|
244
|
+
|
|
245
|
+
**System Hook deletions**
|
|
246
|
+
|
|
247
|
+
Use `--syshook ACTION` when a matching xyOps System Hook launches the CLI. This supports deletion hooks for every standard sync definition type except Buckets. It reads the activity JSON from STDIN and removes only the XYPDF with the matching type and ID, plus its neighboring property files. It does not run ordinary sync, compare definitions, or call the xyOps API. Pass exactly one sync directory, and use an absolute path because System Hooks start in a temporary directory. See the [supported actions](sync.md#system-hook-deletion) for the exact hook names, including `apikey_delete` for API Keys.
|
|
248
|
+
|
|
249
|
+
```sh
|
|
250
|
+
xy sync /srv/xyops-automation --syshook event_delete --down_cmd /usr/local/sbin/commit-xyops-deletion
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
`--down_cmd` runs from the sync directory only after files are actually removed. A missing local source is a successful no-op; duplicate matching sources stop the command without deleting either one. `--dry` previews the files without deleting them or running `down_cmd`. No xyOps URL or API Key is needed. See [System Hook deletion](sync.md#system-hook-deletion) for configuration and operational details.
|
|
241
254
|
|
|
242
255
|
**Completion commands and notifications**
|
|
243
256
|
|
|
@@ -254,7 +267,7 @@ xy sync ./ --up events,plugins \
|
|
|
254
267
|
- `--error_email ADDRESS` sends warnings and errors by email through xyOps.
|
|
255
268
|
- `--error_event EVENT_ID` runs an Event on warnings or errors.
|
|
256
269
|
|
|
257
|
-
Completion commands run in a local shell in the first base directory.
|
|
270
|
+
Completion commands run in a local shell in the first base directory. Successful up-only deletion triggers `up_cmd`; down-only and System Hook deletions trigger `down_cmd` when it is configured. Event notifications receive `input.data.errors` and `input.data.warnings`.
|
|
258
271
|
|
|
259
272
|
The configured API Key needs the appropriate resource permissions and `update_state` for sync tracking updates. Notification actions also need their usual permissions. Earlier successful changes are not rolled back if a later operation fails. Retry the sync after correcting an error; previously applied updates compare equal, and deletion remains disabled until the pre-delete phase completes without warnings or errors.
|
|
260
273
|
|
|
@@ -306,6 +319,7 @@ For each Bucket, setup also writes `BUCKET-SLUG/data.json` and downloads its fil
|
|
|
306
319
|
- `--file_props PATHS` extracts string properties into adjacent files.
|
|
307
320
|
- `--default_ext EXT` sets the fallback neighbor extension instead of `txt`.
|
|
308
321
|
- `--new` writes only remote definitions not already represented locally.
|
|
322
|
+
- `--ignore_missing` lets `--new` skip local sources whose type and ID are absent from xyOps.
|
|
309
323
|
- `--down_cmd COMMAND` runs after setup writes one or more definitions.
|
|
310
324
|
- `--cmd_timeout SECONDS` changes the command timeout from 30 seconds.
|
|
311
325
|
- `--stock` includes stock objects, which are omitted by default.
|
|
@@ -317,7 +331,7 @@ For `--file_props`, use comma-separated property names or dot paths, such as `sc
|
|
|
317
331
|
|
|
318
332
|
The same property selection also checks every workflow node automatically. For example, `params.script` in a node's `data` becomes a neighbor such as `My-Workflow-workflow.nt3sr3y4.params.script.sh`. The node ID in the filename identifies which node owns the property. No additional setup option is needed.
|
|
319
333
|
|
|
320
|
-
Use `--new` from the root of an existing sync tree to add definitions that exist in xyOps but have no local source. Setup scans recursively and matches existing sources by item type and exact ID, so renamed files and custom folder layouts are preserved. New definitions use the normal suggested type and Category folders. Malformed, duplicate, unsupported, or orphaned local sources stop the operation before any new files are written. `--new` cannot be combined with `--force`, and it does not create new definitions in xyOps.
|
|
334
|
+
Use `--new` from the root of an existing sync tree to add definitions that exist in xyOps but have no local source. Setup scans recursively and matches existing sources by item type and exact ID, so renamed files and custom folder layouts are preserved. New definitions use the normal suggested type and Category folders. Malformed, duplicate, unsupported, or orphaned local sources stop the operation before any new files are written. With `--ignore_missing`, only sources whose type and ID are absent from xyOps are skipped. `--new` cannot be combined with `--force`, and it does not create new definitions in xyOps.
|
|
321
335
|
|
|
322
336
|
An explicitly supplied `--down_cmd` runs from the current setup directory after at least one definition is written. It does not run when `--new` finds nothing or during a dry run. Saved sync completion commands are not inherited by setup.
|
|
323
337
|
|
package/docs/sync.md
CHANGED
|
@@ -40,10 +40,12 @@ Sync has two directions. **Up** sends local definitions to xyOps. **Down** write
|
|
|
40
40
|
|
|
41
41
|
**Sync applies changes immediately. There is no confirmation step.** Add `--dry` to preview a run before allowing it to change xyOps or your files.
|
|
42
42
|
|
|
43
|
-
Sync updates *changes* to objects that already exist on both sides. It does not create new xyOps objects
|
|
43
|
+
Sync updates *changes* to objects that already exist on both sides. It does not create new xyOps objects or generate local files for newly created objects (except during [setup](#set-up-your-first-sync-tree)). Use resource creation commands or imports to create objects, and setup or exports to introduce their local files. One-way [delete mode](#delete-mode) can remove definitions missing from the authoritative side.
|
|
44
44
|
|
|
45
45
|
For a Git-based deployment workflow, start with up-only sync. A fresh checkout is a good source for an explicit push, but its filesystem timestamps are not reliable evidence for two-way conflict decisions.
|
|
46
46
|
|
|
47
|
+
For two-way deletion driven by accepted Git pushes and xyOps System Activity triggers, see the [event-driven two-way delete tutorial](two-way-delete-tutorial.md).
|
|
48
|
+
|
|
47
49
|
## Supported resources and requirements
|
|
48
50
|
|
|
49
51
|
Install and connect xyCLI as described in [Getting started](../README.md#getting-started). The CLI requires Node.js 20.19.0 or later and xyOps 1.0.96 or later. The machine running sync needs network access to the xyOps API and access to the local directories you select.
|
|
@@ -70,7 +72,7 @@ Secret Vaults, Users, and Roles are not supported by sync. Individual workers, j
|
|
|
70
72
|
|
|
71
73
|
### Permissions
|
|
72
74
|
|
|
73
|
-
Your API Key needs permission to read the definitions you scan and to edit each type you push.
|
|
75
|
+
Your API Key needs permission to read the definitions you scan and to edit each type you push. Up-only delete mode additionally requires the corresponding delete privileges. Down-only deletion removes local files and does not call the remote delete APIs. Resource restrictions, such as Category and Group access, still apply.
|
|
74
76
|
|
|
75
77
|
Sync also maintains remote-management tracking which requires the [update_state](https://docs.xyops.io/privileges/update_state) privilege. A pull or two-way run can update this tracking too. Notification email requires the [send_emails](https://docs.xyops.io/privileges/send_emails) privilege; an error Event requires the [run_jobs](https://docs.xyops.io/privileges/run_jobs) privilege, permission to use that Event's resources, and an enabled manual trigger.
|
|
76
78
|
|
|
@@ -153,7 +155,7 @@ Setup normally omits objects marked as stock or installed from the Marketplace.
|
|
|
153
155
|
xy sync setup all --stock --marketplace --file_props script,params.script
|
|
154
156
|
```
|
|
155
157
|
|
|
156
|
-
These switches are optional export choices, not requirements for delete mode.
|
|
158
|
+
These switches are optional export choices, not requirements for delete mode. Up-only deletion excludes remote objects marked `stock` or `marketplace`. Down-only deletion excludes local sources carrying either property. Leave Marketplace Plugins managed by the Marketplace and its upgrade system; a standard sync tree can focus on definitions you maintain yourself.
|
|
157
159
|
|
|
158
160
|
### Rerun setup carefully
|
|
159
161
|
|
|
@@ -172,7 +174,7 @@ xy sync setup events plugins categories --new --file_props script,params.script
|
|
|
172
174
|
xy sync setup events plugins categories --new --file_props script,params.script
|
|
173
175
|
```
|
|
174
176
|
|
|
175
|
-
New-only setup scans the current directory recursively and indexes existing XYPDF sources by item type and exact ID. Renamed files and custom folders are preserved. Remote definitions not found in that index are written to setup's normal type and Category folders; existing definitions are skipped without being rewritten. Malformed, duplicate, unsupported, or locally orphaned sources stop the operation before new files are written. `--new` cannot be combined with `--force`.
|
|
177
|
+
New-only setup scans the current directory recursively and indexes existing XYPDF sources by item type and exact ID. Renamed files and custom folders are preserved. Remote definitions not found in that index are written to setup's normal type and Category folders; existing definitions are skipped without being rewritten. Malformed, duplicate, unsupported, or locally orphaned sources stop the operation before new files are written. Add `--ignore_missing` to skip only local sources whose type and ID are absent from xyOps. `--new` cannot be combined with `--force`.
|
|
176
178
|
|
|
177
179
|
This is an additive local export. It does not create definitions in xyOps, and ordinary upsync or downsync still updates only definitions that already exist on both sides.
|
|
178
180
|
|
|
@@ -393,16 +395,18 @@ Relative paths resolve against the CLI's current working directory. Completion c
|
|
|
393
395
|
| --- | --- | --- | --- |
|
|
394
396
|
| `--up TYPES` | `sync.up` | Disabled | Sync upwards to xyOps from local sources for these types. |
|
|
395
397
|
| `--down TYPES` | `sync.down` | Disabled | Down downwards to existing local sources from xyOps for these types. |
|
|
396
|
-
| `--delete TYPES` | `sync.delete` | Disabled | Delete
|
|
398
|
+
| `--delete TYPES` | `sync.delete` | Disabled | Delete definitions missing from the sole enabled direction's authoritative side. Requires up-only or down-only sync; see [Delete mode](#delete-mode). |
|
|
399
|
+
| `--syshook ACTION` | n/a | Disabled | Handle one supported definition deletion activity from STDIN as a local-only deletion; see [System Hook deletion](#system-hook-deletion). |
|
|
400
|
+
| `--ignore_missing` | `sync.ignore_missing` | Off | Skip local sources whose type and ID are absent from xyOps, without a warning or failure. Down-only deletion takes precedence for types selected by `--delete`. |
|
|
397
401
|
| `--up_cmd COMMAND` | `sync.up_cmd` | None | Run a local shell command after at least one successful upsync. |
|
|
398
|
-
| `--down_cmd COMMAND` | `sync.down_cmd` | None | Run a local shell command after at least one successful downsync. |
|
|
402
|
+
| `--down_cmd COMMAND` | `sync.down_cmd` | None | Run a local shell command after at least one successful downsync, or an explicit command after a System Hook deletion. |
|
|
399
403
|
| `--cmd_timeout SECONDS` | `sync.cmd_timeout` | `30` | Timeout for each completion command. |
|
|
400
404
|
| `--error_email ADDRESS` | `sync.error_email` | None | Email collected warnings and errors through xyOps. |
|
|
401
405
|
| `--error_event EVENT_ID` | `sync.error_event` | None | Launch an Event with collected warnings and errors in its input data. |
|
|
402
406
|
| `--lock_file PATH` | `sync.lock_file` | Temporary file keyed by `base_url` | Use an explicit path for the built-in PID lock. |
|
|
403
407
|
| `--dry` | n/a | Off | Dry-Run Mode: Preview without writes, completion commands, or notifications. |
|
|
404
408
|
| `--verbose`, `-v` | n/a | Off | Show complete diffs and additional request, response, and write details. |
|
|
405
|
-
| `--quiet`, `-q` | n/a | Off | Suppress routine output and progress.
|
|
409
|
+
| `--quiet`, `-q` | n/a | Off | Suppress routine output and progress. Warnings and errors remain visible on standard error. |
|
|
406
410
|
|
|
407
411
|
At least one direction must be enabled. The command is a single run, not a background watcher; repeat it through a scheduler or hook for automatic operation.
|
|
408
412
|
|
|
@@ -417,6 +421,7 @@ At least one direction must be enabled. The command is a single run, not a backg
|
|
|
417
421
|
| `--file_props PATHS` | Extract string properties into adjacent neighbor files. Accepts comma-separated dot paths or a JSON array. |
|
|
418
422
|
| `--default_ext EXT` | Use this extension instead of `txt` when a neighbor's type cannot be detected. A leading dot is optional. |
|
|
419
423
|
| `--new` | Export only remote definitions whose type and ID are not already present in the current local tree. |
|
|
424
|
+
| `--ignore_missing` | With `--new`, skip local sources whose type and ID are absent from xyOps. Leave their files untouched. |
|
|
420
425
|
| `--down_cmd COMMAND` | Run a local shell command after setup writes one or more definitions. |
|
|
421
426
|
| `--cmd_timeout SECONDS` | Set the setup completion-command timeout in seconds. The default is `30`. |
|
|
422
427
|
| `--stock` | Include stock definitions that ships wit xyOps (e.g. "Shell Plugin"), which setup normally omits. |
|
|
@@ -450,16 +455,54 @@ For automatic two-way operation, use a persistent tree and run every minute. Bot
|
|
|
450
455
|
|
|
451
456
|
### Remove a definition during two-way sync
|
|
452
457
|
|
|
453
|
-
|
|
458
|
+
One command with both sync directions does not propagate deletions. Keep `--delete false` in a scheduled two-way command, and remove a definition from both sides deliberately:
|
|
454
459
|
|
|
455
460
|
1. Delete the definition in the xyOps web interface first.
|
|
456
461
|
2. Remove its local XYPDF JSON file and any neighboring external property files. If the tree is stored in Git, commit and push their removal so the dedicated sync checkout receives the deletion.
|
|
457
|
-
3. Run sync again, or resume the schedule, after the local files are gone.
|
|
462
|
+
3. Run sync again, or resume the schedule, after the local files are gone. Without `--ignore_missing`, pause the schedule during these steps if you want to avoid a failed pass between the two removals.
|
|
458
463
|
|
|
459
|
-
If a sync runs after the xyOps deletion but before local cleanup, the leftover source produces a missing-object warning
|
|
464
|
+
If a sync runs after the xyOps deletion but before local cleanup, the leftover source normally produces a missing-object warning and stops the run before any definition changes. Add `--ignore_missing` to ordinary sync and any following `sync setup --new` command to skip that source and keep processing other definitions. This does not remove the local files or propagate the deletion to Git. Remove those files yourself to complete the deletion. Other scan warnings and errors still stop the run. For an event-driven workflow with one authoritative direction per run, see [Delete mode](#delete-mode).
|
|
460
465
|
|
|
461
466
|
Do not remove the Git files first while the definition still exists in xyOps. A scheduled `sync setup --new` pass can export the still-existing definition back into the repository. See the [Two-Way Git Sync Tutorial](two-way-git-tutorial.md#delete-a-definition-from-both-sides) for the Git workflow.
|
|
462
467
|
|
|
468
|
+
### System Hook deletion
|
|
469
|
+
|
|
470
|
+
xyOps [System Hooks](https://docs.xyops.io/#Docs/syshooks) can launch xyCLI when a standard synced definition is deleted. Configure each deletion action you need separately. The `--syshook` command reads the activity JSON from STDIN, checks that its action matches the command, and uses the deleted object's type and exact ID to find one local XYPDF source. It removes that source and its neighboring external property files. Other local definitions, including new ones not yet in xyOps, are left alone.
|
|
471
|
+
|
|
472
|
+
| Sync selection | System Hook action |
|
|
473
|
+
| --- | --- |
|
|
474
|
+
| `alerts` | `alert_delete` |
|
|
475
|
+
| `api_keys` | `apikey_delete` |
|
|
476
|
+
| `categories` | `category_delete` |
|
|
477
|
+
| `channels` | `channel_delete` |
|
|
478
|
+
| `events` (including workflows) | `event_delete` |
|
|
479
|
+
| `groups` | `group_delete` |
|
|
480
|
+
| `monitors` | `monitor_delete` |
|
|
481
|
+
| `plugins` | `plugin_delete` |
|
|
482
|
+
| `tags` | `tag_delete` |
|
|
483
|
+
| `web_hooks` | `web_hook_delete` |
|
|
484
|
+
|
|
485
|
+
Buckets are excluded because a Bucket may have separate JSON data and uploaded files alongside its XYPDF source. The `bucket_delete` activity is not supported by `--syshook`.
|
|
486
|
+
|
|
487
|
+
For example, configure the following hooks on the primary conductor, adjusting executable and checkout paths:
|
|
488
|
+
|
|
489
|
+
```json
|
|
490
|
+
{
|
|
491
|
+
"hooks": {
|
|
492
|
+
"event_delete": {
|
|
493
|
+
"shell_exec": "/usr/local/bin/xy sync /srv/xyops-automation --syshook event_delete --down_cmd /usr/local/sbin/commit-xyops-deletion"
|
|
494
|
+
},
|
|
495
|
+
"plugin_delete": {
|
|
496
|
+
"shell_exec": "/usr/local/bin/xy sync /srv/xyops-automation --syshook plugin_delete --down_cmd /usr/local/sbin/commit-xyops-deletion"
|
|
497
|
+
}
|
|
498
|
+
}
|
|
499
|
+
}
|
|
500
|
+
```
|
|
501
|
+
|
|
502
|
+
The `--down_cmd` script can stage, commit, and push the deletions in a dedicated Git checkout. It runs only after at least one file was removed. A missing source is a successful no-op, and duplicate matching sources stop the command before any files are removed. Use `--dry` to preview the selected files without changing them or running the completion command.
|
|
503
|
+
|
|
504
|
+
This mode does not run ordinary sync, fetch remote definitions, or change xyOps. It ignores saved sync direction settings and requires exactly one directory. No xyOps URL or API Key is needed. The CLI still uses a host-local lock, with a fixed default lock key when no `base_url` is configured. System shell hooks run on the primary conductor from a temporary directory, and xyOps limits them to 60 seconds. The conductor must be able to access the checkout and Git credentials. If regular sync runs on another host, its local lock cannot protect this checkout; arrange for both operations to use the same host and lock if they share a checkout. Watch hook failures and Git push failures, because a failed hook is not automatically retried.
|
|
505
|
+
|
|
463
506
|
## Remote-management warnings
|
|
464
507
|
|
|
465
508
|
Up-only sync records the managed definitions in xyOps's global sync state. The web interface and xyCLI update commands use this information to warn that a definition is under remote management and should not be edited directly.
|
|
@@ -482,9 +525,15 @@ Make sure only one sync instance talks to one xyOps installation.
|
|
|
482
525
|
|
|
483
526
|
## Delete mode
|
|
484
527
|
|
|
485
|
-
Delete mode
|
|
528
|
+
Delete mode removes definitions missing from the authoritative side of a **one-way** sync. With `--up` only, it deletes xyOps definitions that have no local XYPDF source. With `--down` only, it deletes local XYPDF sources that have no matching visible xyOps definition, along with their external property files. Select the affected resource types with `--delete TYPES`. The CLI rejects deletion if both directions, or neither direction, are enabled, including directions inherited from saved configuration. A single two-way sync command cannot use delete mode.
|
|
529
|
+
|
|
530
|
+
The chosen direction applies to every missing definition of the selected types. A newly added Git file that has not yet been imported into xyOps looks the same as a definition deleted in xyOps, so a down-only delete run removes that file. Likewise, an xyOps definition that has not yet been exported looks the same as one deleted from Git to an up-only delete run. Coordinate Git-triggered and xyOps-triggered jobs so they do not overlap or process pending changes out of order. A host-local PID lock cannot serialize jobs on different machines.
|
|
531
|
+
|
|
532
|
+
The [event-driven two-way delete tutorial](two-way-delete-tutorial.md) shows separate authoritative triggers for Git pushes, xyOps updates and deletions, and xyOps creations.
|
|
486
533
|
|
|
487
|
-
###
|
|
534
|
+
### Up-only deletion
|
|
535
|
+
|
|
536
|
+
#### Prepare a complete inventory
|
|
488
537
|
|
|
489
538
|
The scanned directories must contain a source for **every other remote object of each type selected for deletion** that you intend to keep. Delete mode considers all eligible objects in the visible remote collection, not just objects previously synced or carrying a remote-management warning.
|
|
490
539
|
|
|
@@ -500,7 +549,7 @@ xy sync setup events plugins categories
|
|
|
500
549
|
|
|
501
550
|
Verify that every object eligible for deletion that you intend to keep has a corresponding source, including workflows under the `events` selection. Stock and Marketplace objects may be omitted. Avoid filename collisions, omitted resources, missing mounts, and hidden source directories.
|
|
502
551
|
|
|
503
|
-
|
|
552
|
+
#### Preview a deliberate deletion
|
|
504
553
|
|
|
505
554
|
Remove the source file(s) for the Event you intend to delete, then preview from the complete inventory:
|
|
506
555
|
|
|
@@ -518,10 +567,25 @@ Deleting an Event does not automatically remove its dependencies. Remote API rul
|
|
|
518
567
|
|
|
519
568
|
Deletion only begins after every update has completed and the sync-tracking state has been written without warnings or errors. If either phase fails, the entire delete pass is skipped for that run. Earlier successful updates remain in place, so correct the error and retry; definitions already updated will compare equal. Once deletion begins, successful earlier deletions are not rolled back if a later deletion fails.
|
|
520
569
|
|
|
521
|
-
With `--delete buckets`, a remote Bucket file missing from a selected local Bucket's `files/` directory is deleted individually. Removing the Bucket XYPDF source deletes the whole Bucket, including all its data and files. A missing or invalid `data.json`, incomplete file inventory, or failed transfer prevents Bucket file deletion and the whole-object delete pass.
|
|
570
|
+
With up-only `--delete buckets`, a remote Bucket file missing from a selected local Bucket's `files/` directory is deleted individually. Removing the Bucket XYPDF source deletes the whole Bucket, including all its data and files. A missing or invalid `data.json`, incomplete file inventory, or failed transfer prevents Bucket file deletion and the whole-object delete pass.
|
|
522
571
|
|
|
523
572
|
An unavailable base directory stops the scan, but an existing empty directory can look like an empty inventory. Do not treat that guard as protection against an empty mount, accidental file removal, or missing exports of your own definitions.
|
|
524
573
|
|
|
574
|
+
### Down-only deletion
|
|
575
|
+
|
|
576
|
+
Run a down-only pass when xyOps should determine which local definitions remain. Specify the full local sync tree, and use `--up false` if saved configuration would otherwise enable upsync:
|
|
577
|
+
|
|
578
|
+
```sh
|
|
579
|
+
xy sync /srv/xyops-sync --up false --down events,plugins --delete events,plugins --dry
|
|
580
|
+
xy sync /srv/xyops-sync --up false --down events,plugins --delete events,plugins --down_cmd ./commit-downloads.sh
|
|
581
|
+
```
|
|
582
|
+
|
|
583
|
+
Down-only deletion removes matching local XYPDF sources and their validated external property files. It matches by item type and exact ID, not filename. A missing local source is not created by ordinary downsync; run `xy sync setup TYPES --new` separately to export newly created xyOps definitions. Setup runs from the current directory, so run it from the sync tree's root.
|
|
584
|
+
|
|
585
|
+
For Buckets, down-only `--delete buckets` also removes local uploaded files absent from an existing remote Bucket. If the entire Bucket is absent from xyOps, it removes the local Bucket XYPDF, `data.json`, and uploaded files, then removes the empty content directories. Unexpected content in a Bucket directory stops the whole-object local delete pass. Stock and Marketplace markers in a local XYPDF exclude that definition from deletion.
|
|
586
|
+
|
|
587
|
+
The local delete pass starts only after the source scan, definition downloads, sync-state update, and Bucket content pass finish without warnings or errors. It checks every candidate before removing any whole-object files. A successful local deletion triggers `down_cmd`, including when there was no ordinary download. If a later file removal fails, earlier removals remain and the run exits nonzero; inspect the tree and retry.
|
|
588
|
+
|
|
525
589
|
Keep deletion disabled in general-purpose cron, Git hooks, and CI until you have a deliberate policy for maintaining and checking the complete inventory. Disable saved deletion for other runs with `--delete false`.
|
|
526
590
|
|
|
527
591
|
It is also highly recommended that you [keep daily backups of critical data](https://docs.xyops.io/hosting/daily-backups) before enabling sync, especially sync with delete.
|
|
@@ -544,7 +608,7 @@ xy sync ./ --down events,plugins --down_cmd "./commit-downloads.sh" --cmd_timeou
|
|
|
544
608
|
### When commands run
|
|
545
609
|
|
|
546
610
|
- `up_cmd` runs if at least one object was successfully upsynced (this includes deletes).
|
|
547
|
-
- `down_cmd` runs if at least one object was successfully
|
|
611
|
+
- `down_cmd` runs if at least one object was successfully downloaded or locally deleted by down-only sync. In `--syshook` mode, an explicitly supplied `down_cmd` runs after the matching local files are deleted.
|
|
548
612
|
- During setup, an explicitly supplied `down_cmd` runs if at least one definition was written.
|
|
549
613
|
- If both apply, `up_cmd` runs first, then `down_cmd`.
|
|
550
614
|
- Neither runs for an identical/no-op run, a scan that stopped before changes, or a dry run.
|
|
@@ -635,7 +699,7 @@ Runs with no warnings or errors exit with status `0`. Scan warnings prevent chan
|
|
|
635
699
|
|
|
636
700
|
Every `xy sync` command acquires a host-local PID lock before it loads remote definitions or touches local files. This includes ordinary up, down, two-way, delete, dry-run, and setup operations. The lock remains held through completion commands and error notifications, so every CLI entry point uses the same overlap protection automatically.
|
|
637
701
|
|
|
638
|
-
By default, xyCLI creates a lock file in the operating system's temporary directory. Its filename contains a hash of the normalized `base_url`, so syncs targeting the same xyOps instance on one host serialize even when they use different directories or API Keys. Syncs for different instances use different default locks.
|
|
702
|
+
By default, xyCLI creates a lock file in the operating system's temporary directory. Its filename contains a hash of the normalized `base_url`, so syncs targeting the same xyOps instance on one host serialize even when they use different directories or API Keys. Syncs for different instances use different default locks. System Hook deletions without a configured `base_url` use a fixed local key instead.
|
|
639
703
|
|
|
640
704
|
When another live sync process owns the lock, the new command stops before doing any work, prints the owning PID and lock path, and exits nonzero. It does not wait or queue. Schedulers can try again on their next run. If a process was forcibly terminated and left its PID file behind, the next invocation detects that the PID is dead, removes the stale file, atomically acquires a replacement, and continues.
|
|
641
705
|
|
|
@@ -691,7 +755,7 @@ PATH=/usr/local/bin:/usr/bin:/bin
|
|
|
691
755
|
|
|
692
756
|
Choose only the item types your tree manages. This example explicitly disables downsync and deletion. Add `--error_event EVENT_ID` if you prefer an error-handling Event, or configure notifications in the account's CLI config file.
|
|
693
757
|
|
|
694
|
-
`--quiet` keeps routine progress and diffs out of cron's output, avoiding an email for every ordinary run.
|
|
758
|
+
`--quiet` keeps routine progress and diffs out of cron's output, avoiding an email for every ordinary run. Warnings and errors still print to standard error. Configure notifications if you also want xyOps to send a message or launch an Event when sync encounters a problem.
|
|
695
759
|
|
|
696
760
|
If the previous run is still active, the built-in PID lock rejects the overlapping invocation with a nonzero exit status. The next minute tries again. A single sync can therefore take longer than a minute without starting another copy.
|
|
697
761
|
|
|
@@ -937,16 +1001,16 @@ Repository concurrency controls these workflow runs. Coordinate all writers to t
|
|
|
937
1001
|
| The wrong directory is scanned | Positional paths take precedence, followed by `base_dirs`, then the current directory. Check your command and config; use absolute paths in automation. |
|
|
938
1002
|
| A config override seems to vanish | The user file replaces a system `sync` object as a whole. Environment settings take precedence over files. See [configuration](#configuration-and-option-reference). |
|
|
939
1003
|
| An object is missing from the local tree | Ordinary downsync does not discover it. Export that object or run setup in a separate staging directory. |
|
|
940
|
-
| A source cannot find its remote object | Match its exact `type` and `data.id` to an existing visible object in this xyOps installation. Create/import missing objects separately. |
|
|
1004
|
+
| A source cannot find its remote object | Match its exact `type` and `data.id` to an existing visible object in this xyOps installation. Create/import missing objects separately, or use down-only delete mode if xyOps is authoritative and the local source should be removed. |
|
|
941
1005
|
| A duplicate-source warning stops sync | Keep one file per resource type and ID, even when the copies are identical. Remove duplicates and avoid overlapping base directories, then retry. |
|
|
942
1006
|
| One bad file stops unrelated updates | The scan validates all JSON sources before applying changes. Fix malformed JSON, multi-item XYPDF, unsupported types, missing objects, or unreadable property files. |
|
|
943
1007
|
| A script edit has no effect | Check the matching JSON stem, property suffix, filename extension, and property path. Hidden paths are skipped. |
|
|
944
1008
|
| Two-way sync overwrites an unexpected side | Compare the remote modification time with the newest JSON/property-file mtime. Fresh checkouts, clock skew, or touched files can make local sources appear newer. |
|
|
945
1009
|
| An Event or workflow does not execute | Sync saves definitions; it does not run them. Use `xy run EVENT_ID` separately. |
|
|
946
|
-
| `down_cmd` does not run | Normal sync requires a successful download. Setup requires at least one written definition and an explicit `--down_cmd`. No-op and dry runs do not trigger it. |
|
|
1010
|
+
| `down_cmd` does not run | Normal sync requires a successful download or down-only deletion. Setup requires at least one written definition and an explicit `--down_cmd`. No-op and dry runs do not trigger it. |
|
|
947
1011
|
| A download commit launches another sync | Disable deployment Git hooks for commits made by `down_cmd`. The nested sync will otherwise be rejected by the built-in PID lock. |
|
|
948
1012
|
| Sync reports another PID is running | Another local sync owns the lock. Let it finish. If the PID is no longer alive, the next run recovers the stale file automatically. If the PID belongs to an unrelated recycled process, verify that no sync is active before removing the reported lock file. |
|
|
949
|
-
| A scheduled run prints nothing | `--quiet` suppresses routine output and
|
|
1013
|
+
| A scheduled run prints nothing | `--quiet` suppresses routine output. Warnings and errors still print to standard error. Check that the scheduler captures it. |
|
|
950
1014
|
| Sync exits nonzero after a warning report | Scan warnings prevent changes from being applied, so warnings and errors both exit with status `1`. Resolve the reported problem before retrying. |
|
|
951
1015
|
| Notifications never arrive | Check API access, email setup and privileges, the error Event's manual trigger, and failures occurring before reporting. |
|
|
952
1016
|
| API Key updates behave differently after import | Portable definitions do not transfer usable secret key material. Treat key credentials separately from their definitions. |
|
|
@@ -961,7 +1025,7 @@ Before important deployments, review a dry run, retain recoverable versions of b
|
|
|
961
1025
|
|
|
962
1026
|
Sync uses IDs rather than titles or filenames, and does not remap IDs for a different xyOps installation. Duplicate sources for the same type and ID generate warnings and stop the run before changes; keep one source per object and avoid overlapping base directories. Split inventories by ownership when useful, but remember that each run replaces the shared up-only remote-management tracking map rather than combining independent jobs' maps.
|
|
963
1027
|
|
|
964
|
-
New
|
|
1028
|
+
New definitions still require explicit creation or setup export. Deletion requires separate one-way runs with a complete inventory on the authoritative side. Two-way sync remains experimental and resolves existing whole objects by modification time, without a merge or conflict prompt.
|
|
965
1029
|
|
|
966
1030
|
- **Command reference:** [Sync](help.md#sync), [Sync setup](help.md#sync-setup), [Exports](help.md#export), [Imports](help.md#import), and [Run](help.md#run).
|
|
967
1031
|
- **Learn more about xyOps:** [xyOps Portable Data Format](https://docs.xyops.io/xypdf) and [xyOps API Keys](https://docs.xyops.io/api#api-keys).
|