@pixlcore/xycli 1.0.1 → 1.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,39 @@
1
+ name: Offline Tests
2
+
3
+ on:
4
+ push:
5
+ pull_request:
6
+
7
+ permissions:
8
+ contents: read
9
+
10
+ jobs:
11
+ test:
12
+ name: ${{ matrix.os }} / Node.js ${{ matrix.node }}
13
+ runs-on: ${{ matrix.os }}
14
+
15
+ strategy:
16
+ fail-fast: false
17
+ matrix:
18
+ os:
19
+ - ubuntu-latest
20
+ - windows-latest
21
+ node:
22
+ - 20.19.0
23
+ - 24
24
+
25
+ steps:
26
+ - name: Check out repository
27
+ uses: actions/checkout@v7
28
+
29
+ - name: Use Node.js ${{ matrix.node }}
30
+ uses: actions/setup-node@v7
31
+ with:
32
+ node-version: ${{ matrix.node }}
33
+ cache: npm
34
+
35
+ - name: Install dependencies
36
+ run: npm ci
37
+
38
+ - name: Run offline tests
39
+ run: npm run test:offline
package/CHANGELOG.md ADDED
@@ -0,0 +1,26 @@
1
+ # xyCLI Changelog
2
+
3
+ ## Version v1.0.2
4
+
5
+ > September 20, 2026
6
+
7
+ - [`7b9023b`](https://github.com/pixlcore/xycli/commit/7b9023b390ea45b0615ff43b4d5a8d3daa3a68f3): Version 1.0.2
8
+ - [`2114287`](https://github.com/pixlcore/xycli/commit/2114287dc98ca2cedcc6cf1b15eb7e164f697851): Feature: Sync Setup: Write out events and workflows into category sub-folders. Fixes #2.
9
+ - [`3f3e8a5`](https://github.com/pixlcore/xycli/commit/3f3e8a515be030e551c33a701c43e2bb3ff72569): Doc: Add two-way git tutorial
10
+ - [`4336dc2`](https://github.com/pixlcore/xycli/commit/4336dc25924616038213b77c5892990c09fa22f5): Feature: Sync setup now supports its own "--down_cmd" for running a custom command if and only if files were written out.
11
+ - [`7b94e63`](https://github.com/pixlcore/xycli/commit/7b94e632e2c0d86eaef5211325b259195cb30f3c): Feature: Add optional "--new" flag to sync setup, to add newly added items not already in sync set
12
+ - [`90d065f`](https://github.com/pixlcore/xycli/commit/90d065f941578ed83783cc79bebbbdc7018a6664): Allow "--default_ext" switch to override default ".txt" file extension for neighbor property files when language cannot be detected. Ref #1.
13
+
14
+ ## Version v1.0.1
15
+
16
+ > September 20, 2026
17
+
18
+ - [`ee12585`](https://github.com/pixlcore/xycli/commit/ee12585b36e7237d8514b9c34ce0411debf16494): Version 1.0.1
19
+ - [`c18090a`](https://github.com/pixlcore/xycli/commit/c18090a03ebdb91772fc565b59eec22e4378df3f): Fix: Do not start delete pass if any warnings or errors occurred during the sync.
20
+ - [`ef4e68b`](https://github.com/pixlcore/xycli/commit/ef4e68b229b8e0114c0af4a44dc1b61ba81e9e39): Test: Add more event unit tests
21
+
22
+ ## Version v1.0.0
23
+
24
+ > September 20, 2026
25
+
26
+ - Initial release!
package/README.md CHANGED
@@ -26,6 +26,7 @@ Browse the [screenshot gallery](docs/screenshots.md) for more views, or jump to
26
26
  - [Configuration](#configuration)
27
27
  - [Global CLI switches](#global-cli-switches)
28
28
  - [Tutorial: your first Event, job, and Web Hook](docs/tutorial.md)
29
+ - [Tutorial: two-way sync with Git](docs/two-way-git-tutorial.md)
29
30
  - [Explore and manage xyOps](#explore-and-manage-xyops)
30
31
  - [Transfer data between systems](#transfer-data-between-systems)
31
32
  - [Sync your automation with local files](#sync-your-automation-with-local-files)
@@ -48,7 +49,7 @@ Save your xyOps URL and API Key:
48
49
  xy config --base_url https://xyops.example.com --api_key YOUR_API_KEY
49
50
  ```
50
51
 
51
- Use the base URL of your xyOps instance, without the `/api` endpoint suffix. The command creates `~/.config/xyops/cli.json` and saves the settings there.
52
+ Use the base URL of your xyOps instance, without the `/api` endpoint suffix. On Unix and macOS, the command creates `~/.config/xyops/cli.json`. On Windows, it creates `%USERPROFILE%\.config\xyops\cli.json`.
52
53
 
53
54
  Now check your connection:
54
55
 
@@ -62,11 +63,11 @@ You should see the dashboard. If you need to set up xyOps itself, start with its
62
63
 
63
64
  xyCLI loads optional JSON configuration files on every invocation, then applies environment variables. Settings are merged in this order:
64
65
 
65
- | Order | Source | Purpose |
66
- | --- | --- | --- |
67
- | 1 | `/etc/xyops/cli.json` | Shared settings for the machine. |
68
- | 2 | `~/.config/xyops/cli.json` | Settings for your user account, overriding matching system settings. |
69
- | 3 | `XYOPS_` environment variables | Overrides for the current shell or command. |
66
+ | Order | Unix and macOS | Windows | Purpose |
67
+ | --- | --- | --- | --- |
68
+ | 1 | `/etc/xyops/cli.json` | `%PROGRAMDATA%\xyops\cli.json` | Shared settings for the machine. |
69
+ | 2 | `~/.config/xyops/cli.json` | `%USERPROFILE%\.config\xyops\cli.json` | Settings for your user account, overriding matching system settings. |
70
+ | 3 | `XYOPS_` environment variables | `XYOPS_` environment variables | Overrides for the current shell or command. |
70
71
 
71
72
  Neither file is required if you supply your connection settings through the environment. Missing files are skipped; files you create must contain valid JSON. Matching top-level properties are replaced by later sources, so a user-level object such as `sync` replaces the system-level object of the same name.
72
73
 
@@ -91,7 +92,7 @@ xy config --suggest false
91
92
  xy config --color false
92
93
  ```
93
94
 
94
- `xy config` masks your API Key when displaying it. Updates load the **user configuration file** separately, apply only the settings specified on the command line, and save it with owner-only read/write permissions. Other user settings are preserved; settings inherited from the system file or environment are not copied into the user file. If the user file does not exist, it is created with only the settings you supply. Environment overrides still take precedence on the next invocation.
95
+ `xy config` masks your API Key when displaying it. Updates load the **user configuration file** separately, apply only the settings specified on the command line, and save it with owner-only read/write permissions on Unix and macOS. On Windows, the file inherits the access controls of your user profile because Windows does not implement Unix permission modes. Other user settings are preserved; settings inherited from the system file or environment are not copied into the user file. If the user file does not exist, it is created with only the settings you supply. Environment overrides still take precedence on the next invocation.
95
96
 
96
97
  Every `XYOPS_` variable maps to a lowercase configuration key. For example, `XYOPS_BASE_URL` sets `base_url` and `XYOPS_ITEMS_PER_PAGE` sets `items_per_page`. Literal `true` and `false` become booleans, and numeric strings become numbers.
97
98
 
@@ -103,6 +104,14 @@ export XYOPS_API_KEY="YOUR_API_KEY"
103
104
  xy
104
105
  ```
105
106
 
107
+ In Windows PowerShell, use the equivalent environment syntax:
108
+
109
+ ```powershell
110
+ $env:XYOPS_BASE_URL = "https://xyops.example.com"
111
+ $env:XYOPS_API_KEY = "YOUR_API_KEY"
112
+ xy
113
+ ```
114
+
106
115
  Or override a display setting for one command:
107
116
 
108
117
  ```sh
@@ -483,7 +492,7 @@ cd xyops-automation
483
492
  xy sync setup events plugins categories --file_props script,params.script
484
493
  ```
485
494
 
486
- Setup creates one file per definition and can extract scripts into adjacent source files. You choose how to organize the tree afterward; sync scans subdirectories and matches definitions by resource type and exact ID.
495
+ Setup creates one file per definition and can extract scripts into adjacent source files. Events and workflows are automatically grouped into subfolders named after their Categories. You can reorganize the tree afterward; sync scans subdirectories and matches definitions by resource type and exact ID.
487
496
 
488
497
  Preview a push or pull, then apply the direction you want:
489
498
 
@@ -499,6 +508,8 @@ Sync supports Alerts, API Keys, Categories, Channels, Events and workflows, Grou
499
508
 
500
509
  Delete mode is for up-sync only: it removes eligible xyOps objects missing from your local inventory. Objects with a `stock` or `marketplace` property are always protected from deletion and need no local files. Keep a complete inventory of the other objects you intend to retain for each selected type, and review a dry run before applying it. Sync applies changes without a confirmation step unless you use `--dry`.
501
510
 
511
+ Follow the [Two-Way Git Sync Tutorial](docs/two-way-git-tutorial.md) to connect one persistent checkout to a shared repository, commit downloaded changes, discover new Events, and schedule the complete workflow with cron or xySat.
512
+
502
513
  Read the dedicated [Sync Guide](docs/sync.md) for setup, file layouts, every option, saved defaults, two-way sync, deletion, notifications, and automation with cron, Git hooks, and GitHub Actions.
503
514
 
504
515
  - **Command reference:** [Sync commands](docs/help.md#sync) and [Sync setup](docs/help.md#sync-setup).
@@ -540,4 +551,4 @@ xy api runEvent --id EVENT_ID --params.example VALUE --dry
540
551
  xy api updateEvent --json @./request.json --dry
541
552
  ```
542
553
 
543
- The examples with `--dry` display the request without calling the API. Many resource views also accept `--format json` to display their data as JSON. See the [xyOps API Reference](https://docs.xyops.io/api), [API command](docs/help.md#api), and [documentation command](docs/help.md#doc) for more.
554
+ The examples with `--dry` display the request without calling the API. Many resource views also accept `--format json` to display their data as JSON. See the [xyOps API Reference](https://docs.xyops.io/#Docs/api), [API command](docs/help.md#api), and [documentation command](docs/help.md#doc) for more.
package/docs/help.md CHANGED
@@ -258,7 +258,7 @@ Every sync and setup command uses a built-in host-local PID lock keyed by `base_
258
258
 
259
259
  **Saved defaults**
260
260
 
261
- Put sync defaults under a `sync` object in `/etc/xyops/cli.json` or `~/.config/xyops/cli.json`:
261
+ Put sync defaults under a `sync` object in the machine or user configuration file. These are `/etc/xyops/cli.json` and `~/.config/xyops/cli.json` on Unix and macOS, or `%PROGRAMDATA%\xyops\cli.json` and `%USERPROFILE%\.config\xyops\cli.json` on Windows:
262
262
 
263
263
  ```json
264
264
  {
@@ -281,6 +281,9 @@ Export existing xyOps definitions into a starter sync tree in the current direct
281
281
  ```sh
282
282
  xy sync setup events plugins categories --dry
283
283
  xy sync setup events plugins categories --file_props script,params.script
284
+ xy sync setup events --file_props params.script --default_ext ps1
285
+ xy sync setup events plugins --new
286
+ xy sync setup events --new --down_cmd "./commit-downloads.sh"
284
287
  xy sync setup all
285
288
  xy sync setup all --stock --marketplace
286
289
  xy sync setup events plugins --force
@@ -289,17 +292,25 @@ xy sync --setup events,plugins --file_props script,params.script
289
292
 
290
293
  Choose one or more resource types separated by spaces, or use `all`. Supported types are the same as for `xy sync`. The `--setup TYPES` form also accepts a comma-separated list.
291
294
 
292
- Setup creates one folder per resource type and one XYPDF JSON file per object, using its title as a filename slug. Selecting `events` exports both Events and workflows, with workflows placed in a separate `workflows` folder. Both are synced with `--up events` or `--down events`.
295
+ Setup creates one folder per resource type and one XYPDF JSON file per object, using its title as a filename slug. Selecting `events` exports both Events and workflows, with workflows placed in a separate `workflows` folder. Events and workflows are grouped into subfolders using their Category title as a slug. Both are synced with `--up events` or `--down events`. Setup stops before writing if an Event or workflow selected for export refers to a missing Category.
293
296
 
294
297
  - `--file_props PATHS` extracts string properties into adjacent files.
298
+ - `--default_ext EXT` sets the fallback neighbor extension instead of `txt`.
299
+ - `--new` writes only remote definitions not already represented locally.
300
+ - `--down_cmd COMMAND` runs after setup writes one or more definitions.
301
+ - `--cmd_timeout SECONDS` changes the command timeout from 30 seconds.
295
302
  - `--stock` includes stock objects, which are omitted by default.
296
303
  - `--marketplace` includes Marketplace objects, which are omitted by default.
297
304
  - `--force` allows generated files to overwrite existing destinations.
298
305
  - `--dry` previews the layout without creating directories or writing files.
299
306
 
300
- For `--file_props`, use comma-separated property names or dot paths, such as `script,params.script`. Only nonempty string values are extracted. The JSON value becomes `(External)`, and sync loads the external file automatically.
307
+ For `--file_props`, use comma-separated property names or dot paths, such as `script,params.script`. Only nonempty string values are extracted. The JSON value becomes `(External)`, and sync loads the external file automatically. Setup chooses extensions from executable commands, shebangs, and JSON content when possible. Use `--default_ext ps1` (or `--default_ext .ps1`) to replace the `.txt` fallback for properties whose type cannot be detected. The override does not replace an extension that setup successfully detects.
301
308
 
302
- Without `--force`, setup stops when an output file already exists. Files written before an error are not rolled back. Ensure object titles produce distinct filename slugs, especially when using `--force`, and back up any local edits before rerunning setup. For a delete-mode inventory, verify that every object eligible for deletion that you intend to keep has a source file. Stock and Marketplace objects are protected from deletion and may be omitted.
309
+ Use `--new` from the root of an existing sync tree to add definitions that exist in xyOps but have no local source. Setup scans recursively and matches existing sources by item type and exact ID, so renamed files and custom folder layouts are preserved. New definitions use the normal suggested type and Category folders. Malformed, duplicate, unsupported, or orphaned local sources stop the operation before any new files are written. `--new` cannot be combined with `--force`, and it does not create new definitions in xyOps.
310
+
311
+ An explicitly supplied `--down_cmd` runs from the current setup directory after at least one definition is written. It does not run when `--new` finds nothing or during a dry run. Saved sync completion commands are not inherited by setup.
312
+
313
+ Without `--force`, ordinary setup stops when an output file already exists. Files written before an error are not rolled back. Ensure object titles produce distinct filename slugs, especially when using `--force`, and back up any local edits before rerunning setup. For a delete-mode inventory, verify that every object eligible for deletion that you intend to keep has a source file. Stock and Marketplace objects are protected from deletion and may be omitted.
303
314
 
304
315
  ## api
305
316
 
@@ -330,7 +341,7 @@ Type `.exit` or hit `Ctrl-C` to exit.
330
341
 
331
342
  ## config
332
343
 
333
- View the effective CLI configuration or save one or more user settings. API keys are masked when displayed, and updated values are written to `~/.config/xyops/cli.json`.
344
+ View the effective CLI configuration or save one or more user settings. API keys are masked when displayed. Updated values are written to `~/.config/xyops/cli.json` on Unix and macOS, or `%USERPROFILE%\.config\xyops\cli.json` on Windows.
334
345
 
335
346
  ```sh
336
347
  xy config
@@ -342,7 +353,7 @@ xy config --base_url http://localhost:5522 --api_key YOUR_API_KEY
342
353
 
343
354
  Updates load the user file separately and apply only the settings specified on the command line, preserving other user settings. System settings and environment overrides are not copied into the file. If the user file does not exist, it is created with only the supplied settings.
344
355
 
345
- Configuration loads `/etc/xyops/cli.json` first, then `~/.config/xyops/cli.json`, then `XYOPS_`-prefixed environment variables. Later sources override matching top-level settings. Environment overrides still take precedence over saved user settings. Common keys include `api_key`, `base_url`, `temp_dir`, `color`, `suggest`, `items_per_page`, `cache_ttl`, `raw`, and `invisible`.
356
+ Configuration loads the machine file first, then the user file, then `XYOPS_`-prefixed environment variables. The machine and user files are `/etc/xyops/cli.json` and `~/.config/xyops/cli.json` on Unix and macOS, or `%PROGRAMDATA%\xyops\cli.json` and `%USERPROFILE%\.config\xyops\cli.json` on Windows. Later sources override matching top-level settings. Environment overrides still take precedence over saved user settings. Common keys include `api_key`, `base_url`, `temp_dir`, `color`, `suggest`, `items_per_page`, `cache_ttl`, `raw`, and `invisible`.
346
357
 
347
358
  ## doc
348
359
 
package/docs/sync.md CHANGED
@@ -113,18 +113,20 @@ xyops-automation/
113
113
  categories/
114
114
  Backups.json
115
115
  events/
116
- Nightly-Backup.json
117
- Nightly-Backup-params.script.sh
116
+ Operations/
117
+ Nightly-Backup.json
118
+ Nightly-Backup-params.script.sh
118
119
  plugins/
119
120
  Custom-Runner.json
120
121
  Custom-Runner-script.js
121
122
  workflows/
122
- Release-Pipeline.json
123
+ Deployments/
124
+ Release-Pipeline.json
123
125
  ```
124
126
 
125
- Setup creates one folder per selected resource type and one JSON file per definition. Event and workflow exports are separated into `events/` and `workflows/`. Both participate in normal sync through the `events` selection.
127
+ Setup creates one folder per selected resource type and one JSON file per definition. Event and workflow exports are separated into `events/` and `workflows/`, then grouped into subfolders using their Category titles. Both participate in normal sync through the `events` selection.
126
128
 
127
- Filenames come from each definition's title: sequences of non-word characters become hyphens, and leading or trailing hyphens are removed. For example, `Nightly Backup` becomes `Nightly-Backup.json`. Different titles can produce the same filename, so review the preview for collisions.
129
+ Folder and filenames come from Category and definition titles: sequences of non-word characters become hyphens, and leading or trailing hyphens are removed. For example, the Event `Nightly Backup` in the `Operations` Category becomes `events/Operations/Nightly-Backup.json`. Different titles can produce the same path, so review the preview for collisions. Setup stops before writing anything if an Event or workflow selected for export refers to a Category that is not present in xyOps.
128
130
 
129
131
  ### 4. Review and version the tree
130
132
 
@@ -159,6 +161,17 @@ xy sync setup events plugins --force
159
161
 
160
162
  `--force` allows generated JSON and external files to be overwritten. It does not clean up old filenames, solve filename collisions, or preserve your local edits. Export into a fresh directory when adding newly created objects, then move their reviewed files into the existing tree.
161
163
 
164
+ To add only definitions that are new to an existing sync tree, run setup from the tree's root with `--new`:
165
+
166
+ ```sh
167
+ xy sync setup events plugins categories --new --file_props script,params.script --dry
168
+ xy sync setup events plugins categories --new --file_props script,params.script
169
+ ```
170
+
171
+ New-only setup scans the current directory recursively and indexes existing XYPDF sources by item type and exact ID. Renamed files and custom folders are preserved. Remote definitions not found in that index are written to setup's normal type and Category folders; existing definitions are skipped without being rewritten. Malformed, duplicate, unsupported, or locally orphaned sources stop the operation before new files are written. `--new` cannot be combined with `--force`.
172
+
173
+ This is an additive local export. It does not create definitions in xyOps, and ordinary upsync or downsync still updates only definitions that already exist on both sides.
174
+
162
175
  Alternatively, you can export single objects at a time:
163
176
 
164
177
  ```sh
@@ -232,6 +245,8 @@ This extracts the selected content, if found, and moves it to a neighboring file
232
245
 
233
246
  The general naming pattern is **`BASENAME-PROPERTY.EXT`**, in the same directory as `BASENAME.json`. Dot paths such as `params.script` identify nested properties. Setup chooses an extension from the executable command or a script's shebang when possible, falling back to JSON-looking content or `.txt`. Normal sync accepts your chosen extension.
234
247
 
248
+ Use `--default_ext EXT` to replace the `.txt` fallback when setup cannot detect a type. For example, `--default_ext ps1` and `--default_ext .ps1` both produce `.ps1` neighbors. Detected interpreter and JSON extensions still take precedence. The fallback applies to every undetected property selected by that setup command, so group properties with the same expected file type or run setup separately when needed.
249
+
235
250
  External properties are detected automatically when sync runs; you do not need to repeat `--file_props`. The neighbor's text replaces that property in the local definition before comparison and before an upload. A pull writes the latest value into the existing neighbor file.
236
251
 
237
252
  You can use this for other string properties too, such as `body` or `notes`. Keep only one neighbor file per unique property path. A neighbor must resolve to a valid property path, and multiple matching neighbors are not treated as alternative versions.
@@ -293,7 +308,7 @@ After changes begin, individual API or write failures are logged and the engine
293
308
 
294
309
  ### Save defaults in a CLI config file
295
310
 
296
- Put a `sync` object in `/etc/xyops/cli.json` for machine or service defaults, or `~/.config/xyops/cli.json` for your own user account:
311
+ Put a `sync` object in the machine configuration file for machine or service defaults, or the user configuration file for your own account. These are `/etc/xyops/cli.json` and `~/.config/xyops/cli.json` on Unix and macOS, or `%PROGRAMDATA%\xyops\cli.json` and `%USERPROFILE%\.config\xyops\cli.json` on Windows:
297
312
 
298
313
  ```json
299
314
  {
@@ -393,12 +408,16 @@ At least one direction must be enabled. The command is a single run, not a backg
393
408
  | `xy sync setup TYPES...` | Export the selected types to the current directory. Separate type names with spaces, or use `all`. |
394
409
  | `--setup TYPES` | Alternative setup syntax, accepting comma-separated type names. For example, `xy sync --setup events,plugins`. |
395
410
  | `--file_props PATHS` | Extract string properties into adjacent neighbor files. Accepts comma-separated dot paths or a JSON array. |
411
+ | `--default_ext EXT` | Use this extension instead of `txt` when a neighbor's type cannot be detected. A leading dot is optional. |
412
+ | `--new` | Export only remote definitions whose type and ID are not already present in the current local tree. |
413
+ | `--down_cmd COMMAND` | Run a local shell command after setup writes one or more definitions. |
414
+ | `--cmd_timeout SECONDS` | Set the setup completion-command timeout in seconds. The default is `30`. |
396
415
  | `--stock` | Include stock definitions that ships wit xyOps (e.g. "Shell Plugin"), which setup normally omits. |
397
416
  | `--marketplace` | Include Marketplace Plugins, which setup normally omits. |
398
417
  | `--force` | Allow generated files to overwrite existing destinations. |
399
418
  | `--dry` | Preview setup's output without creating folders or files. |
400
419
 
401
- Workflow defaults in the `sync` configuration object are not used by setup. Pass setup choices explicitly on the command line. The common `lock_file` setting still applies so setup cannot overlap another sync operation for the same instance.
420
+ Workflow defaults in the `sync` configuration object are not used by setup. Pass setup choices explicitly on the command line. An explicitly supplied `down_cmd` runs from the current setup directory only after real files are written, so it does not run during a dry run or when `--new` finds nothing. The common `lock_file` setting still applies so setup cannot overlap another sync operation for the same instance.
402
421
 
403
422
  ## Two-way sync and modification times
404
423
 
@@ -501,6 +520,7 @@ xy sync ./ --down events,plugins --down_cmd "./commit-downloads.sh" --cmd_timeou
501
520
 
502
521
  - `up_cmd` runs if at least one object was successfully upsynced (this includes deletes).
503
522
  - `down_cmd` runs if at least one object was successfully downsynced.
523
+ - During setup, an explicitly supplied `down_cmd` runs if at least one definition was written.
504
524
  - If both apply, `up_cmd` runs first, then `down_cmd`.
505
525
  - Neither runs for an identical/no-op run, a scan that stopped before changes, or a dry run.
506
526
  - A completion command can run after partial success even if another item failed. It is not proof that the whole run succeeded.
@@ -898,7 +918,7 @@ Repository concurrency controls these workflow runs. Coordinate all writers to t
898
918
  | A script edit has no effect | Check the matching JSON stem, property suffix, filename extension, and property path. Hidden paths are skipped. |
899
919
  | Two-way sync overwrites an unexpected side | Compare the remote modification time with the newest JSON/property-file mtime. Fresh checkouts, clock skew, or touched files can make local sources appear newer. |
900
920
  | An Event or workflow does not execute | Sync saves definitions; it does not run them. Use `xy run EVENT_ID` separately. |
901
- | `down_cmd` does not run | It requires at least one successful download. A no-op, dry run, scan failure, or deletion alone does not trigger it. |
921
+ | `down_cmd` does not run | Normal sync requires a successful download. Setup requires at least one written definition and an explicit `--down_cmd`. No-op and dry runs do not trigger it. |
902
922
  | A download commit launches another sync | Disable deployment Git hooks for commits made by `down_cmd`. The nested sync will otherwise be rejected by the built-in PID lock. |
903
923
  | Sync reports another PID is running | Another local sync owns the lock. Let it finish. If the PID is no longer alive, the next run recovers the stale file automatically. If the PID belongs to an unrelated recycled process, verify that no sync is active before removing the reported lock file. |
904
924
  | A scheduled run prints nothing | `--quiet` suppresses routine output and collected reports. Remove it to investigate and configure notifications. |
@@ -0,0 +1,262 @@
1
+ # Two-Way Git Sync Tutorial
2
+
3
+ > [!IMPORTANT]
4
+ > Some of the features used in this tutorial require xyCLI v1.0.2 or newer.
5
+
6
+ This tutorial sets up one persistent Git checkout as the bridge between xyOps and a shared repository. Team members can edit and push Event files through Git, while changes made in the xyOps web interface are downloaded, committed, and pushed back to the repository.
7
+
8
+ The finished workflow looks like this:
9
+
10
+ ```text
11
+ Developers <-> Git repository <-> Dedicated sync checkout <-> xyOps
12
+ ```
13
+
14
+ One machine and operating-system account should own the dedicated sync checkout. That machine can be an xyOps worker running xySat, but xySat is not required. Cron, another scheduler, or a manually launched script works too.
15
+
16
+ This tutorial manages Events and workflows. You can expand the commands to other [supported resource types](sync.md#supported-resources-and-requirements) after the Event flow is working.
17
+
18
+ ## Before you begin
19
+
20
+ You will need:
21
+
22
+ - xyCLI installed and connected to xyOps.
23
+ - A dedicated Git repository and a persistent clone on the sync machine.
24
+ - A xyOps API Key that can read and edit Events and update the global sync state.
25
+ - Git credentials that allow the sync account to pull and push without an interactive prompt.
26
+ - A configured Git author name and email for automated commits.
27
+ - Accurate clocks on the sync machine and the xyOps conductor, preferably synchronized through NTP.
28
+ - The `flock` command on the scheduler host if you plan to use cron.
29
+
30
+ Two-way sync is experimental. When the local and remote definitions differ, xyCLI compares the xyOps modification timestamp with the newest modification time among the local JSON file and its external property files. The newer side wins, and equal timestamps favor the local copy.
31
+
32
+ Do not periodically replace the dedicated checkout with a fresh clone. Git checkout operations can change filesystem timestamps, which are part of two-way conflict resolution.
33
+
34
+ ## 1. Prepare the dedicated checkout
35
+
36
+ Clone the repository into a permanent location. Replace the example URL and path with your own:
37
+
38
+ ```sh
39
+ git clone git@github.com:YOUR_ORG/xyops-automation.git /srv/xyops-automation
40
+ cd /srv/xyops-automation
41
+ ```
42
+
43
+ Configure the identity used for commits created by sync:
44
+
45
+ ```sh
46
+ git config user.name "xyOps Sync"
47
+ git config user.email "xyops-sync@example.com"
48
+ ```
49
+
50
+ Run all remaining setup commands as the same operating-system account that will run the scheduled job. Configure xyCLI for that account if you have not already done so:
51
+
52
+ ```sh
53
+ xy config --base_url https://xyops.example.com --api_key YOUR_API_KEY
54
+ xy dashboard
55
+ ```
56
+
57
+ The dashboard command verifies the connection. Keep the API Key and Git credentials out of the repository.
58
+
59
+ ## 2. Export the initial Event tree
60
+
61
+ Preview the initial setup from the repository root:
62
+
63
+ ```sh
64
+ cd /srv/xyops-automation
65
+ xy sync setup events --file_props script,params.script --default_ext ps1 --dry
66
+ ```
67
+
68
+ The `events` selection includes both Events and workflows. Setup places workflows in their own `workflows/` directory, then groups Events and workflows into subfolders named after their Categories.
69
+
70
+ `--file_props script,params.script` extracts common Plugin and Event script properties into neighboring files. If your Plugin uses another property, add its dot path to the list. For example, use `script,params.script,params.command` when an Event stores its PowerShell source in `params.command`.
71
+
72
+ The optional `--default_ext ps1` uses `.ps1` when setup cannot detect a neighbor file's language from an executable command, shebang, or JSON content. Remove it or choose another extension if PowerShell is not the appropriate fallback for your repository.
73
+
74
+ When the preview looks correct, write the files:
75
+
76
+ ```sh
77
+ xy sync setup events --file_props script,params.script --default_ext ps1
78
+ ```
79
+
80
+ Review the generated files before publishing them. Definitions can contain scripts, request headers, and other sensitive configuration.
81
+
82
+ ```sh
83
+ git status --short
84
+ find events workflows -type f -print 2>/dev/null
85
+ ```
86
+
87
+ Commit and push the initial tree:
88
+
89
+ ```sh
90
+ git add --all
91
+ git commit -m "Add xyOps Event definitions"
92
+ git push -u origin HEAD
93
+ ```
94
+
95
+ Setup is only responsible for creating the local source files. Normal sync identifies each definition by its resource type and exact ID inside the JSON, so you may reorganize or rename the generated files afterward.
96
+
97
+ ## 3. Preview two-way sync
98
+
99
+ Run a dry pass before enabling automation:
100
+
101
+ ```sh
102
+ xy sync . --up events --down events --delete false --dry
103
+ ```
104
+
105
+ This considers both directions for every local Event and workflow source:
106
+
107
+ - A newer local file is uploaded to its existing xyOps definition.
108
+ - A newer xyOps definition is downloaded to its existing local file.
109
+ - Identical definitions are left alone.
110
+
111
+ The `--delete false` switch is intentional. Two-way sync does not propagate deletions safely. Delete an item deliberately on both sides, and keep deletion disabled in the scheduled command.
112
+
113
+ Review the displayed diffs carefully. If the wrong side would win, check the clocks and file modification times before continuing.
114
+
115
+ ## 4. Create the scheduled sync script
116
+
117
+ Save the following script as `/usr/local/sbin/xyops-git-sync`. Adjust `PATH`, `REPO_DIR`, `XY_BIN`, the extracted property list, and the fallback extension for your environment.
118
+
119
+ ```sh
120
+ #!/bin/sh
121
+ set -eu
122
+
123
+ # Cron and xySat jobs often receive a minimal environment. Include the
124
+ # directories containing Node.js, xyCLI, Git, and standard system tools.
125
+ PATH="/usr/local/bin:/usr/bin:/bin"
126
+ export PATH
127
+
128
+ REPO_DIR="/srv/xyops-automation"
129
+ XY_BIN="/usr/local/bin/xy"
130
+
131
+ cd "$REPO_DIR"
132
+
133
+ # This should be a dedicated automated checkout. Stop instead of accidentally
134
+ # committing an operator's unfinished work or leftovers from a failed run.
135
+ if [ -n "$(git status --porcelain)" ]; then
136
+ echo "The sync checkout has uncommitted changes. Resolve them before retrying."
137
+ exit 1
138
+ fi
139
+
140
+ # Bring in changes pushed by contributors. The extra push retries a commit
141
+ # that may have succeeded during an earlier run whose network push then failed.
142
+ git pull --quiet --ff-only
143
+ git push --quiet
144
+
145
+ # xyCLI runs this only after it writes local definitions.
146
+ GIT_DOWN_CMD="git add --all && git commit -m 'Sync from xyOps' && git push"
147
+
148
+ # Sync changes to existing local definitions in both directions. If this
149
+ # command fails, set -e stops the script before the setup pass.
150
+ "$XY_BIN" sync . \
151
+ --up events \
152
+ --down events \
153
+ --delete false \
154
+ --down_cmd "$GIT_DOWN_CMD" \
155
+ --cmd_timeout 120
156
+
157
+ # Discover definitions created in xyOps since the initial setup. Existing
158
+ # sources are identified by type and ID, regardless of their paths or names.
159
+ # The setup completion command runs only when new files were actually written.
160
+ "$XY_BIN" sync setup events \
161
+ --new \
162
+ --file_props script,params.script \
163
+ --default_ext ps1 \
164
+ --down_cmd "$GIT_DOWN_CMD" \
165
+ --cmd_timeout 120
166
+ ```
167
+
168
+ Make the script executable:
169
+
170
+ ```sh
171
+ chmod 755 /usr/local/sbin/xyops-git-sync
172
+ ```
173
+
174
+ Run it manually as the eventual scheduler account before installing a schedule:
175
+
176
+ ```sh
177
+ /usr/local/sbin/xyops-git-sync
178
+ ```
179
+
180
+ The script intentionally does not implement its own outer lock. A cron caller will use `flock` to protect the complete pull, normal sync, and new-definition setup sequence. When xyOps schedules the script through xySat, the Event's job concurrency setting provides this protection instead. xyCLI's built-in lock still protects each individual sync command.
181
+
182
+ The two uses of `--down_cmd` are deliberately identical:
183
+
184
+ - Ordinary sync runs it only after at least one existing definition is successfully downloaded.
185
+ - `sync setup --new` runs it only after at least one new local definition is written.
186
+ - A no-op or dry run does not create a commit or push.
187
+
188
+ The command stages the entire dedicated repository with `git add --all`. Do not use this script in a checkout containing unrelated work. If your repository contains other material, narrow the Git paths in `GIT_DOWN_CMD`.
189
+
190
+ ## 5. Schedule the script with cron
191
+
192
+ Open the crontab for the sync account:
193
+
194
+ ```sh
195
+ crontab -e
196
+ ```
197
+
198
+ Locate `flock` and adjust the path in the cron entry if necessary:
199
+
200
+ ```sh
201
+ command -v flock
202
+ ```
203
+
204
+ Run the script every minute, with `flock` outside the script:
205
+
206
+ ```cron
207
+ * * * * * /usr/bin/flock --nonblock --conflict-exit-code 0 /tmp/xyops-git-sync.lock /usr/local/sbin/xyops-git-sync >/dev/null
208
+ ```
209
+
210
+ Cron normally emails job output when local mail delivery is configured. This entry discards routine standard output so successful runs remain silent, but leaves standard error alone so failures can still trigger mail. The xyCLI commands deliberately omit `--quiet`, because that option can suppress collected sync warnings and error reports along with routine output.
211
+
212
+ To keep successful output in a log instead, replace `>/dev/null` with an append redirect to a file that the sync account can write. Do not place that log inside the Git repository because the completion command stages the whole tree.
213
+
214
+ `--nonblock` makes an overlapping invocation exit immediately instead of waiting. `--conflict-exit-code 0` treats that expected skip as a successful scheduler run, while an executed sync script still returns its own exit status. The operating system releases the advisory lock automatically when `flock` exits, including when the sync script fails. The `/tmp/xyops-git-sync.lock` file may remain afterward, which is normal and does not mean the lock is still held. See the [`flock` manual](https://man7.org/linux/man-pages/man1/flock.1.html) for details.
215
+
216
+ ## 6. Or schedule it through xySat
217
+
218
+ On a machine running xySat, create an Event using the Shell Plugin and target only the dedicated sync worker. Give it a recurring one-minute schedule and use this script body:
219
+
220
+ ```sh
221
+ /usr/local/sbin/xyops-git-sync
222
+ ```
223
+
224
+ Keep the Event's [job concurrency setting](https://docs.xyops.io/#Docs/limits/max-jobs-limit) at `1`, which is the default. xyOps will then prevent a second copy of the Event from running while the first job is still active, so this invocation does not need `flock`.
225
+
226
+ The xySat service account must be able to execute the script, read and write the repository, run Node.js and xyCLI, access that account's xyCLI configuration, and use the Git credentials. Test the Event manually before enabling its schedule.
227
+
228
+ Schedule only one copy of this workflow, and do not run both cron and xySat for the same checkout.
229
+
230
+ ## 7. Understand the daily workflow
231
+
232
+ After setup, contributors work through Git as usual:
233
+
234
+ ```sh
235
+ git pull
236
+ # Edit Event JSON or neighboring script files.
237
+ git add --all
238
+ git commit -m "Update nightly report"
239
+ git push
240
+ ```
241
+
242
+ The next scheduled pass pulls the commit and uploads newer local definitions to xyOps. If someone edits an existing Event in the xyOps web interface, the next pass downloads the newer definition, and `--down_cmd` commits and pushes the changed files.
243
+
244
+ When someone creates a new Event in xyOps, the ordinary sync pass does not know where to create its local source. The following `sync setup events --new` pass discovers it, creates the suggested files in its Category folder, then commits and pushes them through its own `--down_cmd`.
245
+
246
+ `--new` is additive. It does not overwrite existing sources, and it matches existing definitions by type and exact ID rather than filename. This preserves renamed files and custom folder layouts.
247
+
248
+ ## Troubleshooting
249
+
250
+ | Symptom | What to check |
251
+ | --- | --- |
252
+ | `git pull --ff-only` fails | The sync clone and remote branch have diverged, or the working tree changed. Resolve the Git state manually. Do not force-reset the automated checkout without reviewing its commits. |
253
+ | `git push` prompts or fails under cron or xySat | Configure noninteractive SSH or HTTPS credentials for the scheduler account and confirm the branch allows that account to push. |
254
+ | Git reports an unknown author | Set `user.name` and `user.email` in the dedicated clone for the scheduler account. |
255
+ | The wrong side wins a two-way conflict | Check NTP, the xyOps conductor clock, and the modification times of the JSON and neighboring files. A checkout, restore, or file copy can refresh local timestamps. |
256
+ | A new Event is not exported | Confirm `sync setup events --new` runs from the repository root and that the API Key can see the Event. Add the correct property path if its script is stored somewhere other than `params.script`. |
257
+ | A new Git file is not created in xyOps | Ordinary sync updates definitions that already exist on both sides. Create or import the definition into xyOps first, then let sync manage subsequent changes. |
258
+ | A deleted item reappears or causes an error | Two-way deletion is not supported. Remove the local source and the xyOps definition deliberately as one coordinated operation. |
259
+ | A cron run is skipped | Another cron invocation may hold the `flock` lock. The lock file itself may remain when no lock is held, so check for a running sync process rather than treating the file as stale. |
260
+ | A completion command times out | Increase `--cmd_timeout`, investigate Git network latency, and confirm that Git cannot wait for interactive input. |
261
+
262
+ For the complete behavior and option reference, see the [Filesystem Sync Guide](sync.md) and the [`sync` command reference](help.md#sync).
package/lib/config.js CHANGED
@@ -1,21 +1,37 @@
1
1
  // Config Layer
2
2
 
3
3
  const fs = require('fs');
4
+ const os = require('os');
4
5
  const Path = require('path');
5
6
  const cli = require('pixl-cli');
6
7
  const Tools = cli.Tools;
7
8
  const chalk = cli.chalk;
8
9
 
9
- module.exports = {
10
+ function getConfigFiles(platform, env, home_dir) {
11
+ // Keep the existing Unix paths, but use Windows's shared application-data
12
+ // directory instead of resolving /etc against whichever drive is current.
13
+ // ProgramData should always exist in an ordinary Windows login. If a custom
14
+ // environment omits it, skip the optional machine file rather than guessing.
15
+ var path_api = (platform == 'win32') ? Path.win32 : Path.posix;
16
+ var config_files = [];
17
+ var system_dir = (platform == 'win32') ?
18
+ (env.ProgramData || env.PROGRAMDATA || env.ALLUSERSPROFILE) : '/etc';
19
+
20
+ if (system_dir) config_files.push( path_api.join(system_dir, 'xyops', 'cli.json') );
21
+
22
+ // os.homedir() uses USERPROFILE on Windows and has an operating-system
23
+ // fallback, unlike reading HOME or HOMEPATH directly.
24
+ config_files.push( path_api.join(home_dir, '.config', 'xyops', 'cli.json') );
25
+ return config_files;
26
+ }
27
+
28
+ var config_layer = module.exports = {
10
29
 
11
30
  loadConfig() {
12
31
  // handle loading config at startup
13
32
  // optional config files, sync with env, two-way
14
33
  var config = {};
15
- var config_files = this.configFiles = [
16
- Path.join( '/etc', 'xyops', 'cli.json' ),
17
- Path.join( process.env.HOME, '.config', 'xyops', 'cli.json' )
18
- ];
34
+ var config_files = this.configFiles = getConfigFiles(process.platform, process.env, os.homedir());
19
35
 
20
36
  config_files.forEach( function(file) {
21
37
  if (!fs.existsSync(file)) return;
@@ -80,7 +96,13 @@ module.exports = {
80
96
  Tools.mkdirpSync( Path.dirname(dest_file) );
81
97
  println( "\n " + bold.cyan("Writing to: ") + dest_file );
82
98
  fs.writeFileSync( dest_file, JSON.stringify(user_config, null, "\t") + "\n", { mode: 0o600 } );
83
- fs.chmodSync(dest_file, 0o600);
99
+
100
+ // Windows inherits an ACL from the user's profile directory and does not
101
+ // implement Unix owner/group/other mode distinctions.
102
+ if (process.platform != 'win32') fs.chmodSync(dest_file, 0o600);
84
103
  }
85
-
86
104
  };
105
+
106
+ // Expose the pure path selector to unit tests without merging another command
107
+ // method into the runtime app object.
108
+ Object.defineProperty(config_layer, 'getConfigFiles', { value: getConfigFiles });