accrete-kit 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (134) hide show
  1. package/CHANGELOG.md +43 -0
  2. package/LICENSE +21 -0
  3. package/README.md +148 -2
  4. package/bin/accrete-kit.mjs +4 -0
  5. package/dist/kernel/darwin-arm64/LICENSE +21 -0
  6. package/dist/kernel/darwin-arm64/THIRD-PARTY-NOTICES +5780 -0
  7. package/dist/kernel/darwin-arm64/accrete +0 -0
  8. package/dist/kernel/darwin-arm64/licenses/RUST-COPYRIGHT-library.html +7038 -0
  9. package/dist/kernel/darwin-arm64/licenses/musl-COPYRIGHT +193 -0
  10. package/dist/kernel/darwin-arm64/licenses/rust/Apache-2.0.txt +73 -0
  11. package/dist/kernel/darwin-arm64/licenses/rust/BSD-2-Clause.txt +9 -0
  12. package/dist/kernel/darwin-arm64/licenses/rust/CC-BY-SA-4.0.txt +427 -0
  13. package/dist/kernel/darwin-arm64/licenses/rust/GCC-exception-3.1.txt +30 -0
  14. package/dist/kernel/darwin-arm64/licenses/rust/GPL-2.0-only.txt +133 -0
  15. package/dist/kernel/darwin-arm64/licenses/rust/GPL-3.0-or-later.txt +202 -0
  16. package/dist/kernel/darwin-arm64/licenses/rust/ISC.txt +7 -0
  17. package/dist/kernel/darwin-arm64/licenses/rust/LLVM-exception.txt +15 -0
  18. package/dist/kernel/darwin-arm64/licenses/rust/MIT.txt +9 -0
  19. package/dist/kernel/darwin-arm64/licenses/rust/NCSA.txt +29 -0
  20. package/dist/kernel/darwin-arm64/licenses/rust/OFL-1.1.txt +43 -0
  21. package/dist/kernel/darwin-arm64/licenses/rust/Unicode-3.0.txt +39 -0
  22. package/dist/kernel/darwin-x64/LICENSE +21 -0
  23. package/dist/kernel/darwin-x64/THIRD-PARTY-NOTICES +5780 -0
  24. package/dist/kernel/darwin-x64/accrete +0 -0
  25. package/dist/kernel/darwin-x64/licenses/RUST-COPYRIGHT-library.html +7038 -0
  26. package/dist/kernel/darwin-x64/licenses/musl-COPYRIGHT +193 -0
  27. package/dist/kernel/darwin-x64/licenses/rust/Apache-2.0.txt +73 -0
  28. package/dist/kernel/darwin-x64/licenses/rust/BSD-2-Clause.txt +9 -0
  29. package/dist/kernel/darwin-x64/licenses/rust/CC-BY-SA-4.0.txt +427 -0
  30. package/dist/kernel/darwin-x64/licenses/rust/GCC-exception-3.1.txt +30 -0
  31. package/dist/kernel/darwin-x64/licenses/rust/GPL-2.0-only.txt +133 -0
  32. package/dist/kernel/darwin-x64/licenses/rust/GPL-3.0-or-later.txt +202 -0
  33. package/dist/kernel/darwin-x64/licenses/rust/ISC.txt +7 -0
  34. package/dist/kernel/darwin-x64/licenses/rust/LLVM-exception.txt +15 -0
  35. package/dist/kernel/darwin-x64/licenses/rust/MIT.txt +9 -0
  36. package/dist/kernel/darwin-x64/licenses/rust/NCSA.txt +29 -0
  37. package/dist/kernel/darwin-x64/licenses/rust/OFL-1.1.txt +43 -0
  38. package/dist/kernel/darwin-x64/licenses/rust/Unicode-3.0.txt +39 -0
  39. package/dist/kernel/kernel.json +17 -0
  40. package/dist/kernel/linux-arm64/LICENSE +21 -0
  41. package/dist/kernel/linux-arm64/THIRD-PARTY-NOTICES +5780 -0
  42. package/dist/kernel/linux-arm64/accrete +0 -0
  43. package/dist/kernel/linux-arm64/licenses/RUST-COPYRIGHT-library.html +7038 -0
  44. package/dist/kernel/linux-arm64/licenses/musl-COPYRIGHT +193 -0
  45. package/dist/kernel/linux-arm64/licenses/rust/Apache-2.0.txt +73 -0
  46. package/dist/kernel/linux-arm64/licenses/rust/BSD-2-Clause.txt +9 -0
  47. package/dist/kernel/linux-arm64/licenses/rust/CC-BY-SA-4.0.txt +427 -0
  48. package/dist/kernel/linux-arm64/licenses/rust/GCC-exception-3.1.txt +30 -0
  49. package/dist/kernel/linux-arm64/licenses/rust/GPL-2.0-only.txt +133 -0
  50. package/dist/kernel/linux-arm64/licenses/rust/GPL-3.0-or-later.txt +202 -0
  51. package/dist/kernel/linux-arm64/licenses/rust/ISC.txt +7 -0
  52. package/dist/kernel/linux-arm64/licenses/rust/LLVM-exception.txt +15 -0
  53. package/dist/kernel/linux-arm64/licenses/rust/MIT.txt +9 -0
  54. package/dist/kernel/linux-arm64/licenses/rust/NCSA.txt +29 -0
  55. package/dist/kernel/linux-arm64/licenses/rust/OFL-1.1.txt +43 -0
  56. package/dist/kernel/linux-arm64/licenses/rust/Unicode-3.0.txt +39 -0
  57. package/dist/kernel/linux-x64/LICENSE +21 -0
  58. package/dist/kernel/linux-x64/THIRD-PARTY-NOTICES +5780 -0
  59. package/dist/kernel/linux-x64/accrete +0 -0
  60. package/dist/kernel/linux-x64/licenses/RUST-COPYRIGHT-library.html +7038 -0
  61. package/dist/kernel/linux-x64/licenses/musl-COPYRIGHT +193 -0
  62. package/dist/kernel/linux-x64/licenses/rust/Apache-2.0.txt +73 -0
  63. package/dist/kernel/linux-x64/licenses/rust/BSD-2-Clause.txt +9 -0
  64. package/dist/kernel/linux-x64/licenses/rust/CC-BY-SA-4.0.txt +427 -0
  65. package/dist/kernel/linux-x64/licenses/rust/GCC-exception-3.1.txt +30 -0
  66. package/dist/kernel/linux-x64/licenses/rust/GPL-2.0-only.txt +133 -0
  67. package/dist/kernel/linux-x64/licenses/rust/GPL-3.0-or-later.txt +202 -0
  68. package/dist/kernel/linux-x64/licenses/rust/ISC.txt +7 -0
  69. package/dist/kernel/linux-x64/licenses/rust/LLVM-exception.txt +15 -0
  70. package/dist/kernel/linux-x64/licenses/rust/MIT.txt +9 -0
  71. package/dist/kernel/linux-x64/licenses/rust/NCSA.txt +29 -0
  72. package/dist/kernel/linux-x64/licenses/rust/OFL-1.1.txt +43 -0
  73. package/dist/kernel/linux-x64/licenses/rust/Unicode-3.0.txt +39 -0
  74. package/dist/scaffold/assets/contract/cli-v1.json +296 -0
  75. package/dist/scaffold/assets/contract/example-inputs.json +38 -0
  76. package/dist/scaffold/assets/contract/input-drafts.schema.json +654 -0
  77. package/dist/scaffold/assets/generated/cli-commands.md +288 -0
  78. package/dist/scaffold/assets/generated/release.json +11 -0
  79. package/dist/scaffold/assets/skills/accrete-data/SKILL.md +73 -0
  80. package/dist/scaffold/assets/skills/accrete-data/references/commands.md +193 -0
  81. package/dist/scaffold/assets/skills/accrete-data/references/objects-and-native-work.md +123 -0
  82. package/dist/scaffold/assets/skills/accrete-data/references/remote-and-retention.md +98 -0
  83. package/dist/scaffold/assets/skills/accrete-memory/SKILL.md +61 -0
  84. package/dist/scaffold/assets/skills/accrete-memory/references/commands.md +124 -0
  85. package/dist/scaffold/assets/skills/accrete-memory/references/handovers-and-remote-work.md +78 -0
  86. package/dist/scaffold/assets/skills/accrete-memory/references/writing-and-continuity.md +101 -0
  87. package/dist/scaffold/assets/skills/accrete-models/SKILL.md +46 -0
  88. package/dist/scaffold/assets/skills/accrete-models/references/cli.md +117 -0
  89. package/dist/scaffold/assets/skills/accrete-models/references/objects.md +67 -0
  90. package/dist/scaffold/assets/skills/accrete-models/references/working-practice.md +48 -0
  91. package/dist/scaffold/assets/skills/accrete-results/SKILL.md +69 -0
  92. package/dist/scaffold/assets/skills/accrete-results/references/activities-and-relations.md +116 -0
  93. package/dist/scaffold/assets/skills/accrete-results/references/commands.md +191 -0
  94. package/dist/scaffold/assets/skills/accrete-results/references/scientific-practice.md +89 -0
  95. package/dist/scaffold/assets/skills/accrete-tools/SKILL.md +51 -0
  96. package/dist/scaffold/assets/skills/accrete-tools/references/cli.md +87 -0
  97. package/dist/scaffold/assets/skills/accrete-tools/references/objects.md +66 -0
  98. package/dist/scaffold/assets/skills/accrete-tools/references/working-practice.md +51 -0
  99. package/dist/scaffold/assets/template/.accrete/PROJECT.md +31 -0
  100. package/dist/scaffold/assets/template/.accrete/config-examples/local-compute.toml.example +27 -0
  101. package/dist/scaffold/assets/template/.accrete/config-examples/local-workstation.toml.example +21 -0
  102. package/dist/scaffold/assets/template/.accrete/config-examples/project.toml.example +27 -0
  103. package/dist/scaffold/assets/template/.accrete/docs/cli-contract.md +393 -0
  104. package/dist/scaffold/assets/template/.accrete/docs/configuration.md +117 -0
  105. package/dist/scaffold/assets/template/.accrete/docs/coordinator.md +102 -0
  106. package/dist/scaffold/assets/template/.accrete/docs/long-term-work.md +108 -0
  107. package/dist/scaffold/assets/template/.accrete/docs/overview.md +118 -0
  108. package/dist/scaffold/assets/template/.accrete/docs/remote-work.md +125 -0
  109. package/dist/scaffold/assets/template/.accrete/docs/subagents.md +78 -0
  110. package/dist/scaffold/assets/template/.accrete/examples/README.md +78 -0
  111. package/dist/scaffold/assets/template/.accrete/examples/data-register.json +19 -0
  112. package/dist/scaffold/assets/template/.accrete/examples/iteration-close.json +5 -0
  113. package/dist/scaffold/assets/template/.accrete/examples/iteration-open.json +8 -0
  114. package/dist/scaffold/assets/template/.accrete/examples/memory-entry.md +10 -0
  115. package/dist/scaffold/assets/template/.accrete/examples/model-register.json +7 -0
  116. package/dist/scaffold/assets/template/.accrete/examples/observation.md +9 -0
  117. package/dist/scaffold/assets/template/.accrete/examples/result-amend.json +5 -0
  118. package/dist/scaffold/assets/template/.accrete/examples/result-record.json +11 -0
  119. package/dist/scaffold/assets/template/.accrete/examples/tool-admit.json +8 -0
  120. package/dist/scaffold/assets/template/.accrete/examples/tool-retire.json +3 -0
  121. package/dist/scaffold/assets/template/.accrete/examples/tool-update.json +3 -0
  122. package/dist/scaffold/assets/template/.gitattributes +17 -0
  123. package/dist/scaffold/assets/template/AGENTS.md +23 -0
  124. package/dist/scaffold/assets/template/CLAUDE.md +5 -0
  125. package/dist/scaffold/assets/template/_gitignore +12 -0
  126. package/dist/scaffold/config/defaults.toml +20 -0
  127. package/dist/scaffold/manifest.json +622 -0
  128. package/lib/cli.mjs +188 -0
  129. package/lib/fsutil.mjs +135 -0
  130. package/lib/install.mjs +439 -0
  131. package/lib/kernel.mjs +76 -0
  132. package/lib/scaffold.mjs +129 -0
  133. package/lib/workspace.mjs +278 -0
  134. package/package.json +18 -4
@@ -0,0 +1,288 @@
1
+ # CLI command map
2
+
3
+ Generated from the command catalog of this ACCRETE release. Read
4
+ [CLI semantics](cli-contract.md) for input fields, publication outcomes and
5
+ recovery, and `accrete COMMAND --help` for the same flags at the terminal.
6
+
7
+ Creating a Workspace and upgrading its template files are not `accrete` commands:
8
+ see [the generator](cli-contract.md#workspace-creation-and-template-upgrades).
9
+
10
+ Use `--project DIRECTORY` to select the Workspace explicitly and `--json` for receipts.
11
+ Uppercase metavariables must be replaced with actual values. Object arguments
12
+ accept a stable ID, a name or an alias. These flags repeat: `--result`, `--model`,
13
+ `--tool`, `--data` and `--supersedes` on `memory append`, `--commit` on
14
+ `model iteration close`, `--source-scope` on `model register`, `--path` on
15
+ `result capture`, and `--id` on `export`.
16
+
17
+ ## project.migrate.legacy
18
+
19
+ ```sh
20
+ accrete migrate --from LEGACY [--dry-run]
21
+ ```
22
+
23
+ Convert the old scaffold's metadata, retaining original backups and an identity map.
24
+
25
+ ## project.migrate.format
26
+
27
+ ```sh
28
+ accrete migrate [--dry-run]
29
+ ```
30
+
31
+ Handle persisted format changes, keeping identities and relationships. With only format 1 in existence it reports the project is current and changes nothing.
32
+
33
+ ## config.resolve
34
+
35
+ ```sh
36
+ accrete config resolve --json
37
+ ```
38
+
39
+ Report resolved project roots, host, metadata view, state directory and index path without disclosing credentials or writing objects.
40
+
41
+ ## interface
42
+
43
+ ```sh
44
+ accrete interface --json
45
+ ```
46
+
47
+ Report CLI, interface and format versions and exit codes; needs no Workspace.
48
+
49
+ ## model.register
50
+
51
+ ```sh
52
+ accrete model register [--from FILE] [--checkout PATH] [--id ID] [--name NAME] [--description TEXT] [--source-scope PATH]
53
+ ```
54
+
55
+ Allocate a Model identity for an existing Git checkout or a repository locator supplied in JSON. Does not clone or fetch. A checkout becomes a host-local binding.
56
+
57
+ ## model.bind
58
+
59
+ ```sh
60
+ accrete model bind MODEL_ID --checkout PATH
61
+ ```
62
+
63
+ Set this host's checkout binding for a known Model.
64
+
65
+ ## model.update
66
+
67
+ ```sh
68
+ accrete model update MODEL_ID --from FILE --expect REVISION
69
+ ```
70
+
71
+ Amend the Model descriptor; source history stays in Git.
72
+
73
+ ## model.retire
74
+
75
+ ```sh
76
+ accrete model retire MODEL_ID --expect REVISION [--from FILE]
77
+ ```
78
+
79
+ End current use while preserving identity, repository and earlier associations. The optional JSON may carry reason.
80
+
81
+ ## model.iteration.open
82
+
83
+ ```sh
84
+ accrete model iteration open MODEL_ID [--from FILE] [--id ID] [--name NAME] [--purpose TEXT] [--branch BRANCH] [--worktree PATH] [--start REVISION] [--existing]
85
+ ```
86
+
87
+ Describe an iteration. --worktree creates a worktree of the Model's repository unless --existing attaches one; without --worktree no storage is allocated. Flags override --from fields of the same name.
88
+
89
+ ## model.iteration.close
90
+
91
+ ```sh
92
+ accrete model iteration close ITERATION_ID --expect REVISION [--from FILE] [--outcome TEXT] [--commit COMMIT]
93
+ ```
94
+
95
+ Retain the outcome, summary and every selected commit. Outcome is required from the file or --outcome; --commit repeats. Merging and worktree removal stay separate.
96
+
97
+ ## model.iteration.remove-worktree
98
+
99
+ ```sh
100
+ accrete model iteration remove-worktree ITERATION_ID [--force]
101
+ ```
102
+
103
+ Remove the iteration's bound worktree through Git, keeping Git's refusal for changed work unless --force is given.
104
+
105
+ ## tool.admit
106
+
107
+ ```sh
108
+ accrete tool admit [--from FILE] [--id ID] [--name NAME] [--description TEXT] [--source PATH] [--source-mode live|retained] [--revision COMMIT]
109
+ ```
110
+
111
+ Give an existing capability a Tool identity and explicit live or retained source selection.
112
+
113
+ ## tool.bind
114
+
115
+ ```sh
116
+ accrete tool bind TOOL_ID --source PATH
117
+ ```
118
+
119
+ Set this host's source binding for a known Tool.
120
+
121
+ ## tool.update
122
+
123
+ ```sh
124
+ accrete tool update TOOL_ID --from FILE --expect REVISION
125
+ ```
126
+
127
+ Amend metadata or source selection without changing the user's active implementation checkout.
128
+
129
+ ## tool.expose
130
+
131
+ ```sh
132
+ accrete tool expose TOOL_ID --agent codex|claude [--mode wrapper|symlink] [--name NAME] [--destination DIRECTORY]
133
+ ```
134
+
135
+ Create a project-local discovery binding to actual source material; default to a portable source-reading wrapper.
136
+
137
+ ## tool.refresh
138
+
139
+ ```sh
140
+ accrete tool refresh TOOL_ID [--agent codex|claude]
141
+ ```
142
+
143
+ Refresh existing managed exposures, reporting unavailable or pending source material; a retired Tool's local exposures are removed where possible.
144
+
145
+ ## tool.retire
146
+
147
+ ```sh
148
+ accrete tool retire TOOL_ID --expect REVISION [--from FILE]
149
+ ```
150
+
151
+ Retain identity and associations while ending normal discovery; preserve source material and running work. The optional JSON may carry reason.
152
+
153
+ ## tool.export
154
+
155
+ ```sh
156
+ accrete tool export TOOL_ID --to PATH
157
+ ```
158
+
159
+ Create a new local directory with tool.json and source/; external publication remains a separate action.
160
+
161
+ ## data.register
162
+
163
+ ```sh
164
+ accrete data register --from FILE [--id ID] [--name NAME] [--description TEXT] [--result RESULT_ID] [--live]
165
+ ```
166
+
167
+ Describe Data and usable locations without mandatory payload copying, packing or hashing.
168
+
169
+ ## data.register.path
170
+
171
+ ```sh
172
+ accrete data register --path PATH [--id ID] [--name NAME] [--description TEXT] [--kind file|directory|collection|external] [--hash] [--result RESULT_ID] [--live]
173
+ ```
174
+
175
+ Register a local borrowed path in place. --hash computes SHA-256 for one regular file.
176
+
177
+ ## data.amend
178
+
179
+ ```sh
180
+ accrete data amend DATA_ID --from FILE --expect REVISION
181
+ ```
182
+
183
+ Change descriptive metadata. Closed content and the locations array cannot change this way.
184
+
185
+ ## data.location.add
186
+
187
+ ```sh
188
+ accrete data location add DATA_ID [--from FILE] [--location NAME] [--path PATH] [--kind filesystem|rclone|external] [--expect REVISION]
189
+ ```
190
+
191
+ Add a borrowed locator for the same described content; do not redefine earlier Result inputs. A repeated matching location is a no-op.
192
+
193
+ ## data.materialize
194
+
195
+ ```sh
196
+ accrete data materialize DATA_ID [--to LOCATION] [--source SOURCE_LOCATION] [--expect REVISION]
197
+ ```
198
+
199
+ Produce and publish a managed replica at a configured filesystem location (default working) using local transfer or the configured rclone adapter.
200
+
201
+ ## data.evict
202
+
203
+ ```sh
204
+ accrete data evict DATA_ID --location LOCATION [--expect REVISION]
205
+ ```
206
+
207
+ Remove precisely one Kernel-managed replica named by location ID or name, preserving the Data identity and other locations. An ambiguous name is a conflict.
208
+
209
+ ## data.retire
210
+
211
+ ```sh
212
+ accrete data retire DATA_ID --expect REVISION [--from FILE]
213
+ ```
214
+
215
+ End current use while preserving identity, locations and earlier associations; no bytes are removed. The optional JSON may carry reason.
216
+
217
+ ## result.record
218
+
219
+ ```sh
220
+ accrete result record [--from FILE] [--id ID] [--name NAME] [--description TEXT] [--observe-checkout PATH]
221
+ ```
222
+
223
+ Persist a useful activity description and its supplied Model, Tool and Data associations. --observe-checkout adds a separately labelled recording-time observation.
224
+
225
+ ## result.amend
226
+
227
+ ```sh
228
+ accrete result amend RESULT_ID --from FILE --expect REVISION
229
+ ```
230
+
231
+ Apply an explicit correction or enrichment against the actual current document bytes.
232
+
233
+ ## result.observe
234
+
235
+ ```sh
236
+ accrete result observe RESULT_ID --text-file FILE [--operation-id ID]
237
+ ```
238
+
239
+ Append an independent observation to the observations array; an operation ID makes retries unambiguous.
240
+
241
+ ## result.capture
242
+
243
+ ```sh
244
+ accrete result capture --checkout PATH --path SOURCE_PATH --to DIRECTORY
245
+ ```
246
+
247
+ Write a selected binary Git diff and selected nonignored untracked files to a new directory with a manifest of the captured scope. --path repeats. It is not a complete executable environment.
248
+
249
+ ## memory.append
250
+
251
+ ```sh
252
+ accrete memory append --from FILE [--id ID] [--title TITLE] [--authored-at RFC3339] [--result ID] [--model ID] [--tool ID] [--data ID] [--supersedes ID] [--links-from FILE]
253
+ ```
254
+
255
+ Publish complete authored Markdown with a stable ID, timezone-based path and repeatable object links. Retry a lost acknowledgement with the same --id.
256
+
257
+ ## index.refresh
258
+
259
+ ```sh
260
+ accrete index refresh
261
+ ```
262
+
263
+ Refresh the host-local disposable SQLite projection from the selected metadata view.
264
+
265
+ ## export
266
+
267
+ ```sh
268
+ accrete export --to PATH [--id ID]
269
+ ```
270
+
271
+ Write a metadata bundle of selected or all managed documents. Payloads, Git objects, Tool source and machine bindings stay out. --id repeats.
272
+
273
+ ## import
274
+
275
+ ```sh
276
+ accrete import --from PATH [--allow-foreign]
277
+ ```
278
+
279
+ Import a metadata bundle object by object, reporting each outcome. A batch is not atomic; repeating it completes missing entries.
280
+
281
+ ## recover
282
+
283
+ ```sh
284
+ accrete recover [--operation OPERATION_ID]
285
+ ```
286
+
287
+ Reconcile interrupted transfers, evictions, worktree creation and uncertain publications from their receipts. Never relaunches scientific computation. --operation limits Data reconciliation to one operation.
288
+
@@ -0,0 +1,11 @@
1
+ {
2
+ "template_id": "accrete-foundation",
3
+ "template_version": "0.1.0",
4
+ "kernel_version": "0.1.0",
5
+ "cli_contract": "accrete-template-cli/v1",
6
+ "project_formats": {
7
+ "min": 1,
8
+ "max": 1
9
+ },
10
+ "architecture_edition": "2026-11-03"
11
+ }
@@ -0,0 +1,73 @@
1
+ ---
2
+ name: accrete-data
3
+ description: Retain, locate, materialize, and manage scientific inputs and outputs as ACCRETE Data. Use when a dataset, restart, model output, analysis product, external dataset version, or collection needs a durable identity, a Result association, another replica, or explicit replica eviction. Read available data with native scientific tools.
4
+ ---
5
+
6
+ # ACCRETE Data
7
+
8
+ Give useful material a continuing project identity when later work needs to
9
+ find it or understand its use. A Data object can describe a file, a directory,
10
+ a collection, or an external dataset version. Its ID denotes the intended
11
+ material; locations say where copies can be accessed. A checksum describes
12
+ bytes within its stated scope. Keep those three meanings separate.
13
+
14
+ ## Choose the appropriate action
15
+
16
+ | Need | Action |
17
+ | --- | --- |
18
+ | Read an available array, table, or document | Use its normal reader and concrete path. |
19
+ | Produce intermediate outputs or explore a transformation | Work in Scratch or the producer's own directory. |
20
+ | Retain a useful product in place | Call `accrete data register`. |
21
+ | Describe an external dataset before downloading it | Register its locator and known content/version information. |
22
+ | Correct a description | Call `accrete data amend` using the current revision. |
23
+ | Associate an existing additional copy | Call `accrete data location add`. |
24
+ | Obtain a managed usable copy | Call `accrete data materialize`. |
25
+ | Remove one managed copy while preserving its meaning | Call `accrete data evict` for that exact replica. |
26
+ | End current use without removing bytes | Call `accrete data retire`. |
27
+
28
+ Registration need not copy, archive, upload, traverse, or fully hash a dataset.
29
+ Supply the information already available, and request additional byte work
30
+ only when it serves the task. A Data object may be valid while no copy is
31
+ accessible on the current machine.
32
+
33
+ ## Preserve content meaning
34
+
35
+ Register a completed product when its relevant writes have finished. When
36
+ working with a live source or open collection, describe it that way. A stable
37
+ path to changing bytes does not identify a historical snapshot. Adding a
38
+ replica or correcting a description changes an existing descriptor; corrected
39
+ scientific content normally receives a new Data ID naming the previous one in
40
+ `replaces`. Keep earlier Results associated with the material they
41
+ actually used.
42
+
43
+ Use the Result Skill to describe the producing or consuming activity. Link
44
+ Data as an input or output when known; ordinary input/output participation
45
+ does not assert that every output derives from every input. Use Memory for
46
+ interpretation, retention decisions, and lessons that later work should use.
47
+ Keep NetCDF, HDF5, Zarr, Parquet, archives, and domain collections in their
48
+ normal representations. Domain Tools choose transformations and packaging.
49
+
50
+ ## Work through the native boundary
51
+
52
+ Read shared `.accrete/project.toml` and host-local `.accrete/local.toml` to
53
+ resolve store and location bindings. Paths in examples are illustrative;
54
+ configured paths and mutation receipts identify the actual locations. When
55
+ working outside the ACCRETE root, select it explicitly with `--project`.
56
+
57
+ Draft registration inputs in ordinary files; publish managed descriptors
58
+ through `accrete`. Use its returned ID and concrete path. If metadata
59
+ was committed but indexing is pending, refresh the index without registering
60
+ another object. If a transfer or eviction stopped partway, run `accrete recover`;
61
+ it finishes from the receipt and reports an incomplete transfer with its paths.
62
+ Metadata repair never requires rerunning the producer.
63
+
64
+ ## Read when needed
65
+
66
+ - [Objects and native work](references/objects-and-native-work.md): identity,
67
+ content editions, collections, relationships, and reading existing material.
68
+ - [Command manual](references/commands.md): registration, amendment, retirement,
69
+ location changes, materialization, eviction, input drafts, and operation outcomes.
70
+ - [Remote work and retention](references/remote-and-retention.md): host
71
+ placement, scientific outputs, large transfers, and borrowed locations.
72
+
73
+ `accrete data COMMAND --help` prints the flags.
@@ -0,0 +1,193 @@
1
+ # Data command manual
2
+
3
+ ## Contents
4
+
5
+ - Interface and project selection
6
+ - Register a product in place
7
+ - Draft structured Data input
8
+ - Amend or retire a description
9
+ - Associate another location
10
+ - Materialize a managed replica
11
+ - Evict an exact managed replica
12
+ - Understand operation outcomes
13
+
14
+ ## Interface and project selection
15
+
16
+ This manual covers the Data commands. `accrete data COMMAND --help` prints the flags.
17
+
18
+ Global options are `--project DIRECTORY` and `--json`. `ACCRETE_PROJECT` can select
19
+ the project when no explicit `--project` is present. Otherwise the Kernel finds
20
+ the nearest project marker from the current directory. Invocation selection
21
+ takes precedence over the environment. Relative shared and local configured
22
+ paths resolve against the selected project root.
23
+
24
+ The full contract is installed at project-root path
25
+ `.accrete/docs/cli-contract.md`; the generated command map is at
26
+ `.accrete/docs/cli-commands.md`. JSON output goes to stdout and diagnostics to
27
+ stderr. Use returned paths and IDs instead of deriving filenames yourself.
28
+
29
+ ## Register a product in place
30
+
31
+ After a producer has closed a directory selected for retention:
32
+
33
+ ```sh
34
+ accrete --json data register --name year-1-output \
35
+ --path /scratch/project-a/run/year-1/output \
36
+ --kind directory --result "$RESULT_ID"
37
+ ```
38
+
39
+ `RESULT_ID` denotes the actual producing Result when known. Omit `--result`
40
+ when no Result exists or the relationship is unknown. The command establishes
41
+ a Data identity and a borrowed location. Registration does not copy the
42
+ directory or imply permission to delete it. A local path may be outside the
43
+ project root. A relative `--path` resolves from the project root, and a symlink
44
+ is refused.
45
+
46
+ Registration is metadata-proportional by default. `--hash` computes SHA-256 for
47
+ one regular file; directories can refer to a producer-owned manifest. Recursive
48
+ inventory, archiving, and upload are separate explicit choices. Do not request
49
+ them merely to make a descriptor look complete. `--live` describes mutable
50
+ material, and `--id` gives a stable ID for retries.
51
+
52
+ ## Draft structured Data input
53
+
54
+ Use a structured draft for richer content and locations:
55
+
56
+ ```sh
57
+ accrete --json data register --from scratch/data.json
58
+ ```
59
+
60
+ The following is a complete example of the input fields. Replace its example
61
+ path and host with actual configured context before use. The Kernel assigns the
62
+ object envelope, and a UUID unless you supply `id`.
63
+
64
+ ```json
65
+ {
66
+ "name": "year-1-diagnostics",
67
+ "description": "Selected diagnostics from the completed first simulation year.",
68
+ "content": {
69
+ "kind": "directory",
70
+ "mode": "closed",
71
+ "format": "NetCDF collection"
72
+ },
73
+ "origin": {
74
+ "description": "Produced by the first-year simulation; selection preserved by the domain launcher."
75
+ },
76
+ "locations": [
77
+ {
78
+ "name": "compute-output",
79
+ "kind": "filesystem",
80
+ "host": "gpu-a",
81
+ "path": "/scratch/project-a/year-1/diagnostics",
82
+ "managed": false
83
+ }
84
+ ]
85
+ }
86
+ ```
87
+
88
+ Draft fields:
89
+
90
+ | Field | Meaning |
91
+ | --- | --- |
92
+ | `name`, `description` | Human organization and intended material. |
93
+ | `content.kind` | `file`, `directory`, `collection`, or `external`. Without `--path` it defaults to `external`. |
94
+ | `content.mode` | `closed` (the default) for an intended fixed edition; `live` for changing content. |
95
+ | `content.format` | Optional native format description. |
96
+ | `content.edition` | Optional words describing the intended edition, such as a provider's release name. |
97
+ | `content.size_bytes` | Optional known size. Materialization checks it for a file. |
98
+ | `content.sha256` | Optional SHA-256 of a regular file's bytes. Materialization checks it. |
99
+ | `origin.description` | How the material arose. |
100
+ | `origin.result_id`, `origin.uri` | Optional producing activity or provider location. |
101
+ | `replaces` | Optional earlier Data ID that this corrected edition replaces. |
102
+ | `locations` | Access locations, each with its own generated ID. |
103
+ | `locations[].name` | Optional. A configured location name supplies its kind and root or remote. |
104
+ | `locations[].kind` | `filesystem`, configured `rclone`, or `external` for a locator the Kernel records and never transfers. |
105
+ | `locations[].host` | Host context for a physical path; filesystem locations default to the current host. |
106
+ | `locations[].path` | Filesystem path, backend-relative path, or external locator. |
107
+ | `locations[].managed` | `false` or omitted; `true` is refused. The Kernel sets managed ownership for replicas it creates. |
108
+
109
+ A location is a usable locator; an origin explanation alone is insufficient for
110
+ materialization. An external version can be registered without downloading it.
111
+ Do not add a `sha256` whose bytes have not been hashed. A backend `version`,
112
+ `version_id` or `s3_version_at` on a location is kept as an opaque token, and
113
+ `data materialize` refuses a source that carries one rather than silently
114
+ fetching current bytes. Use a version-specific exported locator, or retrieve
115
+ that version with the provider's tool and register the local copy.
116
+
117
+ ## Amend or retire a description
118
+
119
+ ```sh
120
+ accrete --json data amend "$DATA_ID" --from scratch/change.json --expect "$REVISION"
121
+ accrete --json data retire "$DATA_ID" --expect "$REVISION"
122
+ ```
123
+
124
+ `data amend` takes a JSON Merge Patch of descriptive fields. It refuses to change
125
+ closed `content` or replace `locations`; those need a new Data ID or the location
126
+ commands below. `data retire` changes status and removes no bytes. Its optional
127
+ `--from FILE` document can carry a `reason`, stored on the descriptor.
128
+
129
+ ## Associate another location
130
+
131
+ When a known copy already exists at a configured location:
132
+
133
+ ```sh
134
+ accrete --json data location add "$DATA_ID" \
135
+ --location archive --path retained/year-1
136
+ ```
137
+
138
+ `--location` names the location, `--kind filesystem|rclone|external` overrides its
139
+ kind, and `--from FILE` accepts the same fields as JSON. `--expect REVISION` is
140
+ optional. A path relative to a configured filesystem root resolves there. A
141
+ repeated matching location is a no-op. The command records an association. It does
142
+ not perform the transfer, rewrite content identity, or turn a borrowed path into a
143
+ Kernel-owned replica. Keep backend credentials in native configuration.
144
+
145
+ ## Materialize a managed replica
146
+
147
+ ```sh
148
+ accrete --json data materialize "$DATA_ID" --to working
149
+ ```
150
+
151
+ `--to` is a configured filesystem location name, by default `working`. The command
152
+ selects a usable source, stages the transfer, publishes the completed replica at
153
+ `ROOT/DATA_ID/REPLICA_UUID/payload`, then records its location. The receipt supplies
154
+ the path to read with ordinary scientific tools and names the content checks actually
155
+ performed. If the source is ambiguous, select it with `--source LOCATION_ID` or an
156
+ unambiguous name. `--expect REVISION` ensures the descriptor has not changed first.
157
+
158
+ A repeat can return an existing present replica at the same root. Partial transfer
159
+ bytes remain separate from published usable material. Symlinks and special files are
160
+ refused; package them first. For a project already using DVC or DataLad,
161
+ materialization through those native tools can be followed by `data location add`
162
+ for the obtained copy.
163
+
164
+ ## Evict an exact managed replica
165
+
166
+ ```sh
167
+ accrete --json data evict "$DATA_ID" --location "$LOCATION_ID"
168
+ ```
169
+
170
+ `--location` takes the location's ID or name; an ambiguous name is a conflict. Choose
171
+ the exact recorded managed location after deciding that removing this copy serves the
172
+ task. The Kernel removes that managed replica and updates its availability while
173
+ preserving the Data identity and Result associations. Borrowed paths are not eligible
174
+ merely because they were registered. The other recorded locations can inform retention
175
+ decisions; their availability may be dated, and local metadata does not describe all
176
+ external uses.
177
+
178
+ ## Understand operation outcomes
179
+
180
+ | Outcome | Meaning and next action |
181
+ | --- | --- |
182
+ | `committed: true`, `index_pending: false` | The document was published and the local projection updated. Use its ID/path. |
183
+ | `committed: true`, `index_pending: true` | The document exists. Run `accrete index refresh` to repair the projection. |
184
+ | `published: true`, uncertain durability (exit 8) | The destination is visible; run `accrete recover`. Do not allocate a replacement identity. |
185
+ | Conflict before publication (exit 4) | Retain the current descriptor and input draft; resolve the specific differing content or binding. |
186
+ | Transfer side effect with descriptor pending (exit 7) | Run `accrete recover`, or `accrete recover --operation OPERATION_ID` for one operation. |
187
+
188
+ `accrete recover` finishes publishing a completed transfer or eviction from its
189
+ receipt. A receipt for an incomplete transfer is reported with its paths and left for
190
+ explicit retry or cleanup; recovery does not start the transfer again. A retry of the
191
+ same persistent operation must not change into a new Data object or a second scientific
192
+ execution. Data registration and a subsequent Result update can complete separately;
193
+ finish the missing association without undoing the already-retained product.
@@ -0,0 +1,123 @@
1
+ # Data objects and native work
2
+
3
+ ## Contents
4
+
5
+ - Identity and content
6
+ - Choosing what to retain
7
+ - Collections and scientific representations
8
+ - Relations to other objects
9
+ - Reading and changing material
10
+
11
+ ## Identity and content
12
+
13
+ Keep these descriptions distinct when interpreting a Data descriptor:
14
+
15
+ | Description | Meaning |
16
+ | --- | --- |
17
+ | Stable ID | The project object meant by later links. |
18
+ | Name or alias | A readable label that may change. |
19
+ | Content description | The intended file, collection, provider edition, or live source. |
20
+ | Origin | How the material arose: provider, extraction, generating Result, or explanation. |
21
+ | Location | A concrete filesystem or backend locator through which bytes may be accessed. |
22
+ | Availability observation | What was known about a location at a particular time. |
23
+ | `content.sha256` or manifest | Optional identification of a file's bytes or of explicitly scoped content. |
24
+ | Descriptor revision | The current metadata bytes used to detect conflicting edits. |
25
+
26
+ Two Data IDs may intentionally denote different products with identical bytes.
27
+ Storage deduplication does not merge their identities. Conversely, a path may
28
+ keep its spelling while its contents change. Naming a live path alone does
29
+ not identify what a historical activity consumed.
30
+
31
+ A closed object describes one intended content edition. Add locations and
32
+ correct metadata without changing that edition. If a numerical fix produces
33
+ different output, register a new object and relate it to the previous one.
34
+ A provider-native immutable version can identify an edition, described in
35
+ `content.edition`. Keep opaque backend tokens as supplied; an mtime or ETag is not a
36
+ SHA-256 content digest, and a backend version selector is not something generic
37
+ materialization will fetch.
38
+
39
+ The revision returned after a descriptor amendment serves optimistic editing.
40
+ It is separate from the content identity: adding an archive copy changes the
41
+ descriptor while leaving the scientific product unchanged.
42
+
43
+ ## Choosing what to retain
44
+
45
+ Retain material that gives subsequent work something useful to reuse, inspect,
46
+ compare, or cite. A small diagnostic table may deserve a stable identity while
47
+ large regenerable intermediates remain Scratch. Existing provider data may
48
+ need an identity before any download. Compiler caches can remain ordinary
49
+ performance aids without becoming scientific Data.
50
+
51
+ No fixed size threshold, retention duration, or sequence of registration steps
52
+ applies to every project. Choose the smallest useful description. Examples of
53
+ reasonable choices include:
54
+
55
+ - Register an existing restart directory in place after the producer closes it.
56
+ - Register a provider's immutable dataset version and its remote locator.
57
+ - Preserve selected variables from a large simulation through a domain Tool,
58
+ then register the selected product as new Data linked to its generating Result.
59
+ - Leave an exploratory plot in Scratch until it is useful to retain.
60
+ - Describe a mutable observational feed explicitly as live, with the version
61
+ or extraction used by a particular Result when available.
62
+
63
+ Neither registration nor an archive locator proves that restoration will be
64
+ possible indefinitely. Availability descriptions are dated observations.
65
+
66
+ ## Collections and scientific representations
67
+
68
+ A directory, a manifest-defined set, and a provider dataset version are useful
69
+ forms of Data. Preserve their native reading behavior. A Zarr collection need
70
+ not be converted into a tarball, and its chunks need not each become ACCRETE
71
+ objects. A domain reader understands array metadata and calendars; the Kernel
72
+ understands identity and location.
73
+
74
+ For output produced in stages, retain completed products independently when
75
+ useful. A multiyear simulation might produce one retained restart and one
76
+ diagnostic collection per completed year. A parent collection may link them;
77
+ the project need not adopt a universal Year/Run/Experiment hierarchy.
78
+
79
+ An open collection can acquire membership observations. A Result requiring
80
+ a fixed membership must identify the edition or manifest it used. Merely
81
+ adding a later partition to a changing directory does not retrospectively
82
+ change the set consumed by an earlier analysis.
83
+
84
+ The producer remains responsible for safe concurrent reading and completion
85
+ of its format. Registering a partly written HDF5 file cannot make it a usable
86
+ snapshot. Follow the scientific program's closure, flush, or snapshot behavior.
87
+
88
+ ## Relations to other objects
89
+
90
+ Use Result input and output links to preserve how Data participated in an
91
+ activity. Data origin can name a generating Result where useful. A Model
92
+ describes the program; a Result describes a particular use of that program;
93
+ Data describes retained material. A Tool may perform extraction, comparison,
94
+ transfer preparation, or format conversion. Memory explains what the material
95
+ means for later decisions.
96
+
97
+ Keep supplied roles such as `initial_state`, `forcing`, `restart`, or
98
+ `diagnostics` as domain descriptions. They are not globally fixed Data classes.
99
+ Cross-host relations may be unresolved locally until their documents arrive.
100
+ Do not download a dataset merely to make its ID locally resolvable.
101
+
102
+ ## Reading and changing material
103
+
104
+ Read small descriptors with ordinary file tools, and read content with the
105
+ appropriate scientific software. For default roots, useful starting points are:
106
+
107
+ ```sh
108
+ rg 'restart|forcing|diagnostic' .accrete/objects
109
+ jq '.' /path/from/a/data-receipt.json
110
+ ```
111
+
112
+ Replace the example path with the actual descriptor path. Use `sqlite3
113
+ -readonly` on the host-local index, at the path in `.local_state.index` from `accrete
114
+ config resolve --json`, for relational questions, after checking that its projection
115
+ covers the documents needed. Use native SQL schema discovery (`.schema`) instead of
116
+ assuming a column. A brief "database is locked" during a concurrent refresh means retry. `accrete index refresh` rebuilds a stale projection from small
117
+ documents; it does not reprocess scientific payloads.
118
+
119
+ Edit scientific source or working content with ordinary tools. Publish Data
120
+ registration and location changes with the Kernel. If directly editing managed
121
+ metadata is explicitly required, coordinate with other writers and run `accrete index
122
+ refresh` afterward. A direct file edit does not participate in the Kernel's local
123
+ concurrency protection.