alfrd-gsheet 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. alfrd_gsheet-0.2.0/PKG-INFO +530 -0
  2. alfrd_gsheet-0.2.0/README.md +522 -0
  3. alfrd_gsheet-0.2.0/alfrd_gsheet/__init__.py +33 -0
  4. alfrd_gsheet-0.2.0/alfrd_gsheet/a1.py +63 -0
  5. alfrd_gsheet-0.2.0/alfrd_gsheet/actions.py +454 -0
  6. alfrd_gsheet-0.2.0/alfrd_gsheet/cli.py +274 -0
  7. alfrd_gsheet-0.2.0/alfrd_gsheet/client.py +348 -0
  8. alfrd_gsheet-0.2.0/alfrd_gsheet/columns.py +94 -0
  9. alfrd_gsheet-0.2.0/alfrd_gsheet/loopback.py +214 -0
  10. alfrd_gsheet-0.2.0/alfrd_gsheet/mapping.py +380 -0
  11. alfrd_gsheet-0.2.0/alfrd_gsheet/oauth.py +362 -0
  12. alfrd_gsheet-0.2.0/alfrd_gsheet/oauth_client.json +1 -0
  13. alfrd_gsheet-0.2.0/alfrd_gsheet/pages/picker.html +35 -0
  14. alfrd_gsheet-0.2.0/alfrd_gsheet/pages/signed_in.html +2 -0
  15. alfrd_gsheet-0.2.0/alfrd_gsheet/project.py +260 -0
  16. alfrd_gsheet-0.2.0/alfrd_gsheet/row_match.py +43 -0
  17. alfrd_gsheet-0.2.0/alfrd_gsheet/settings.py +144 -0
  18. alfrd_gsheet-0.2.0/alfrd_gsheet/studio.py +119 -0
  19. alfrd_gsheet-0.2.0/alfrd_gsheet/sync.py +505 -0
  20. alfrd_gsheet-0.2.0/alfrd_gsheet/web/index.css +106 -0
  21. alfrd_gsheet-0.2.0/alfrd_gsheet/web/index.js +88 -0
  22. alfrd_gsheet-0.2.0/alfrd_gsheet/web/project_ui.js +959 -0
  23. alfrd_gsheet-0.2.0/alfrd_gsheet.egg-info/PKG-INFO +530 -0
  24. alfrd_gsheet-0.2.0/alfrd_gsheet.egg-info/SOURCES.txt +38 -0
  25. alfrd_gsheet-0.2.0/alfrd_gsheet.egg-info/dependency_links.txt +1 -0
  26. alfrd_gsheet-0.2.0/alfrd_gsheet.egg-info/entry_points.txt +2 -0
  27. alfrd_gsheet-0.2.0/alfrd_gsheet.egg-info/requires.txt +2 -0
  28. alfrd_gsheet-0.2.0/alfrd_gsheet.egg-info/top_level.txt +1 -0
  29. alfrd_gsheet-0.2.0/pyproject.toml +20 -0
  30. alfrd_gsheet-0.2.0/setup.cfg +4 -0
  31. alfrd_gsheet-0.2.0/tests/test_client.py +380 -0
  32. alfrd_gsheet-0.2.0/tests/test_core.py +537 -0
  33. alfrd_gsheet-0.2.0/tests/test_gsheet_actions.py +536 -0
  34. alfrd_gsheet-0.2.0/tests/test_gsheet_cli.py +209 -0
  35. alfrd_gsheet-0.2.0/tests/test_gsheet_studio.py +188 -0
  36. alfrd_gsheet-0.2.0/tests/test_loopback.py +119 -0
  37. alfrd_gsheet-0.2.0/tests/test_oauth.py +278 -0
  38. alfrd_gsheet-0.2.0/tests/test_oauth_config.py +248 -0
  39. alfrd_gsheet-0.2.0/tests/test_real_auth.py +51 -0
  40. alfrd_gsheet-0.2.0/tests/test_runner.py +108 -0
@@ -0,0 +1,530 @@
1
+ Metadata-Version: 2.4
2
+ Name: alfrd-gsheet
3
+ Version: 0.2.0
4
+ Requires-Python: >=3.10
5
+ Description-Content-Type: text/markdown
6
+ Requires-Dist: google-auth[requests]==2.61.0
7
+ Requires-Dist: requests==2.32.5
8
+
9
+ # Google Sheet step sync
10
+
11
+ Writes each finished plan step's status, timing, usage and other fields into
12
+ a Google Sheet row for that target. It can also read notes or todo/skip
13
+ requests from the sheet into the plan. Mirroring runs in the plan runner's step
14
+ hooks. The plugin also provides the `alfrd gsheet` CLI, a Studio panel and a
15
+ palette check.
16
+
17
+ Status: offline transport tests, the scheduler integration, an isolated
18
+ install and the Studio browser tests pass. Live verification against a real
19
+ Google Sheet is still pending.
20
+ Installing the plugin registers hooks; projects without a mapping make no API
21
+ calls and do not need credentials.
22
+
23
+ ## Setup walkthrough: Google sign-in (desktop)
24
+
25
+ 1. Install the plugin and restart `alfrd serve`:
26
+ `alfrd plugin install ./examples/plugins/alfrd-gsheet`.
27
+ 2. Open a project's **Google Sheet → Connect & validate…** dialog and choose
28
+ **Sign in with Google…**. Complete Google's consent screen in the browser on
29
+ the same computer as ALFRD. Select `oauth` in **Settings → Plugins → Google
30
+ Sheet → Sign-in method** if you previously used a service-account key.
31
+ 3. **Connect sheet… → Choose from Google Drive…**, or enter a title and
32
+ **Create new spreadsheet**. Picker grants ALFRD access to the chosen file.
33
+ Pasting an arbitrary spreadsheet URL alone cannot grant access. Creating a
34
+ sheet starts it empty: open its link, add headers and target rows, then Load.
35
+ 4. Select the tab, header row and row-match column, then **Attach**. Validate
36
+ the mapping and preview an export before writing run results.
37
+
38
+ Google sign-in requests only `drive.file`, `openid` and email. It can read and
39
+ edit files chosen with Picker or created by ALFRD, rather than all Drive files.
40
+ **Choose sheet again…** restores a file grant without changing the mapping.
41
+ A read-only sheet needs Editor permission from its owner. Expired or revoked
42
+ sign-in stops subsequent sync requests until you sign in again. The local panel
43
+ shows a sign-in or file-access recovery message after a failed sync.
44
+
45
+ **Sign out…** attempts to revoke the grant, then deletes the local refresh token
46
+ whether or not Google is reachable. Google's revocation affects **all clients
47
+ of the same Google project**, including ALFRD on other computers. Previously
48
+ chosen files may need choosing again. If revoke fails, remove ALFRD manually at
49
+ <https://myaccount.google.com/connections>.
50
+
51
+ CLI equivalents:
52
+
53
+ ```sh
54
+ alfrd gsheet auth login
55
+ alfrd gsheet auth status
56
+ alfrd gsheet auth logout --yes
57
+ alfrd gsheet create --title "Lab runs"
58
+ alfrd gsheet init <project> --spreadsheet <returned-id>
59
+ ```
60
+
61
+ `auth login --no-browser --timeout 300` prints a URL and waits; advanced remote
62
+ users must forward the printed loopback port themselves. Normal login refuses
63
+ headless/SSH sessions and starts no listener there. Servers and CI should use
64
+ **service_account**, below. Hooks never initiate login.
65
+
66
+ ### OAuth client configuration
67
+
68
+ **End users of the official ALFRD release need no OAuth client setup or credentials.**
69
+ The release CI supplies the bundled client; users just choose **Sign in with Google**.
70
+
71
+ The repository ships **empty placeholders** in `alfrd_gsheet/oauth_client.json`.
72
+ For source checkouts, maintainers can save their client outside the repository at
73
+ `~/.config/alfrd/gsheet_oauth_client.json` (or
74
+ `$XDG_CONFIG_HOME/alfrd/gsheet_oauth_client.json` when set). Discovery is automatic.
75
+ Use Google's downloaded Desktop `{"installed": {...}}` JSON or a flat object
76
+ with the keys below. Optional `picker_api_key` and `project_number` keys can be
77
+ included in the same object as the client keys; `project_id` is not the project number.
78
+ Web clients are rejected. Use `chmod 600` on the file; group- or world-readable
79
+ client files trigger a warning without preventing sign-in.
80
+
81
+ Precedence is **per key**, highest first: non-empty individual environment
82
+ variables → `ALFRD_GSHEET_OAUTH_CLIENT_FILE` → per-user file → bundled defaults.
83
+ The explicit file variable accepts the same JSON formats. Missing per-user files
84
+ are ignored; malformed or unreadable files produce a redacted configuration error.
85
+ Set overrides in both the Studio server and runner/CLI environments.
86
+ Maintainer setup hints are logged by the server when the client is unavailable;
87
+ Studio directs users to the official release or **Advanced → service account**.
88
+ See the [maintainer setup guide](../../../docs/plugins/gsheet-oauth-setup.md).
89
+
90
+ | File key | Environment override | Format |
91
+ |---|---|---|
92
+ | `client_id` | `ALFRD_GSHEET_OAUTH_CLIENT_ID` | Desktop client ending in `.apps.googleusercontent.com` |
93
+ | `client_secret` | `ALFRD_GSHEET_OAUTH_CLIENT_SECRET` | Non-empty desktop client secret, at most 256 characters |
94
+ | `picker_api_key` | `ALFRD_GSHEET_PICKER_API_KEY` | API key, 30–64 letters/digits/underscore/hyphen |
95
+ | `project_number` | `ALFRD_GSHEET_GCP_PROJECT_NUMBER` | Numeric Cloud project number, 6–20 digits |
96
+
97
+ Use one Google Cloud project for the Desktop OAuth client and Picker project
98
+ number. Enable Sheets, Drive and Google Picker APIs. Restrict the API key to
99
+ the Google Picker API. Loopback origins use random ports; validate any website
100
+ restrictions before distributing your build. Without Picker configuration,
101
+ sign-in and Create work; choosing existing files is disabled.
102
+
103
+ Configure the external consent screen for the three scopes above and publish
104
+ it **In production** before release. In **Testing**, grants containing
105
+ `drive.file` normally expire after **7 days**. Complete applicable Google brand
106
+ verification. No real client credentials, API keys or user tokens belong in
107
+ source control.
108
+
109
+ Refresh tokens live separately from plugin settings in
110
+ `<user_config_dir("alfrd")>/gsheet/oauth-token.json` (directory 0700, file 0600,
111
+ atomic replacement, honours `XDG_CONFIG_HOME`). Access tokens remain in process
112
+ memory and the isolated Picker page; they are never returned in Studio actions.
113
+ This is local file protection, not an OS keyring. Only short-lived nonces and
114
+ URLs pass between Studio and the `127.0.0.1:<random-port>` helper. The helper
115
+ checks Host, state/PKCE and single-use callbacks, and expires automatically.
116
+
117
+ **Release validation still needed:** the brief's S0 checks A1–A4 require a
118
+ maintainer Google project/account: Desktop-token Picker on the random loopback
119
+ origin, API-key restrictions/CSP, inaccessible-file 403/404, and re-picking after
120
+ revocation. These assumptions have not been live-verified by the offline tests.
121
+ If Desktop-token web Picker is rejected, the brief describes a separate
122
+ `trigger_onepick` desktop fallback; this build implements the web Picker path.
123
+ See [Google installed-app OAuth](https://developers.google.com/identity/protocols/oauth2/native-app)
124
+ and [Google Picker integration](https://developers.google.com/workspace/drive/picker/guides/web-picker).
125
+
126
+ ## Advanced: service account (headless)
127
+
128
+ 1. **Install** the plugin and restart `alfrd serve`:
129
+ `alfrd plugin install ./examples/plugins/alfrd-gsheet`.
130
+ 2. **Create a service account** in a Google Cloud project (IAM & Admin →
131
+ Service accounts), add a JSON key, and download it. Enable the
132
+ **Google Sheets API** in the same project. The **Google Drive API** is only
133
+ needed if you tick **Check edit permission with the Drive API**.
134
+ 3. **Share the spreadsheet** with the key's `client_email`
135
+ (`…@….iam.gserviceaccount.com`) as an **Editor**. Use a dedicated sheet or
136
+ a copy until you trust the mapping.
137
+ 4. **Configure**: in Settings → Plugins → Google Sheet → Configure, select
138
+ `service_account` as **Sign-in method**, paste the
139
+ whole JSON key and set the default spreadsheet id or URL. Press **Test**. It
140
+ writes nothing; it reports Editor access and the number of worksheets.
141
+ 5. **Generate a mapping**:
142
+ `alfrd gsheet init <project> [--worksheet Targets]`. The sheet needs a header
143
+ row and a column whose values match the plan's target names.
144
+ Edit the file to add fields, formats or inbound rules (see below).
145
+ 6. **Validate**: `alfrd gsheet validate <project>`, or **Google Sheet: validate
146
+ mapping** in the Studio palette. Fix each reported column or step.
147
+ 7. **Preview, then backfill**: `alfrd gsheet push <project> --dry-run` shows the
148
+ cells it would write. Run without `--dry-run` to fill in steps that finished
149
+ before the mapping existed. From then on, every launched step syncs on its
150
+ own.
151
+ 8. Optionally add the [Studio panel](#studio-panel-and-validation).
152
+
153
+ To stop syncing a project, set `enabled: false` or delete `alfrd.gsheet.yaml`.
154
+ The setting applies from the next launched unit.
155
+
156
+ ## Development
157
+
158
+ From the alfrd repository, run the offline tests with the host environment:
159
+
160
+ ```sh
161
+ uv run python -m pytest -q examples/plugins/alfrd-gsheet/tests
162
+ ```
163
+
164
+ No service account, Google libraries or network are needed to run these tests.
165
+ The optional real-library signing and OAuth refresh tests skip when Google's libraries are absent.
166
+ To include them without accessing a real sheet:
167
+
168
+ ```sh
169
+ uv run --with 'google-auth[requests]==2.61.0' --with 'requests==2.32.5' \
170
+ python -m pytest -q examples/plugins/alfrd-gsheet/tests
171
+ ```
172
+
173
+ The suite blocks non-loopback socket connections. Loopback HTTP handler tests
174
+ use in-memory connections and fake Google responses, so even restricted CI
175
+ needs no listening sockets. Browser-origin/Google interoperability remains a
176
+ manual release check; JS syntax is checked with `node --check`.
177
+
178
+ The engine accepts a `SheetsClient`; its protocol is in `alfrd_gsheet/client.py`.
179
+ The transport dependencies are pinned in `pyproject.toml`; alfrd and its
180
+ PyYAML/typer dependencies come from the host, not a second alfrd install.
181
+
182
+ Install from the repository:
183
+
184
+ ```sh
185
+ alfrd plugin install ./examples/plugins/alfrd-gsheet
186
+ ```
187
+
188
+ ## Settings and Test
189
+
190
+ **Sign-in method** accepts `oauth` or `service_account`. Empty/missing uses
191
+ `service_account` when `credentials_json` is present, otherwise `oauth`. Existing
192
+ key-only configurations need no migration or file rewrite. Explicit `oauth`
193
+ ignores a leftover key without deleting it; switching back restores that key.
194
+ OAuth can be saved without any service-account key. Without a default sheet,
195
+ OAuth Test refreshes sign-in and reports the account; set a default to also test
196
+ sheet access.
197
+
198
+ For Advanced mode, select `service_account`, paste a service-account JSON
199
+ key into **Service-account key JSON (Advanced)** and set **Default spreadsheet id or URL**.
200
+ Enable the Sheets API in the key's Google Cloud project and share the
201
+ spreadsheet with the key's `client_email` as an Editor. Credentials stay
202
+ in the host's private plugin settings file; public settings show only whether
203
+ the key is set. Do not put the key in the project mapping.
204
+
205
+ **Test** reads spreadsheet metadata and lists worksheets without writing; a
206
+ successful result reports the worksheet count. By default it checks read access
207
+ only, and edit access is verified on the first write. Tick **Check edit
208
+ permission with the Drive API** to have Test also ask Drive whether the account
209
+ can edit; that needs the Drive API enabled in the same project. Syncing itself
210
+ uses only the Sheets API.
211
+ Read-only access asks you to grant Editor permission; 403/404 errors ask you to
212
+ check sharing, the spreadsheet id and enabled APIs. In service-account mode, Test requires a default
213
+ spreadsheet even when individual mappings specify their own ids.
214
+
215
+ Defaults: `value_input_option: RAW`, `conflict_policy: skip`,
216
+ `request_timeout: 15`, `dry_run: false`, `check_drive: false`. `USER_ENTERED` lets Sheets interpret
217
+ values as formulas/numbers/dates. Request timeout must be positive and finite.
218
+ Credentials are cached per process, while each API call uses a fresh session.
219
+ Authentication, lock waits, up to three attempts for 429/5xx, and exponential
220
+ backoff with jitter share the caller's absolute deadline. Socket timeouts are
221
+ capped by the request timeout and remaining budget. A wall-time guard also
222
+ bounds authentication/library waits; an already-sent HTTP write may finish
223
+ after timeout, so a timeout does not prove the write was rejected. No subsequent
224
+ retry or API request starts after the deadline.
225
+
226
+ ## Mapping file
227
+
228
+ Save `<project>/alfrd.gsheet.yaml`. This file is separate from `alfrd.yaml` and
229
+ is re-read before each launched unit. A unit keeps the mapping that it started
230
+ with. Missing files, `enabled: false`, and units with no matching rules do
231
+ nothing. Skipped, blocked and never-launched units get no lifecycle calls.
232
+
233
+ ```yaml
234
+ version: 1
235
+ enabled: true
236
+ spreadsheet_id: 1AbCdEfGhIjKlMnOpQrStUvWxYz
237
+ worksheet: Targets # or gid: 0 instead of worksheet
238
+ header_row: 1
239
+ rows:
240
+ key_column: TARGET_NAME
241
+ code_column: PROJECT_CODE # optional; distinguishes repeated targets
242
+ missing_row: skip # skip | append
243
+ read_range: A1:ZZ # optional; must include header_row
244
+ verify_before_write: true
245
+ outbound:
246
+ - step: calibrate
247
+ column: calibrate
248
+ field: status
249
+ - step: calibrate
250
+ column: calibrate RAM
251
+ field: usage.peak_mem
252
+ format: bytes
253
+ - step: '*'
254
+ column: '{step} finished'
255
+ field: finished
256
+ format: datetime
257
+ when: [done, failed]
258
+ - step: calibrate
259
+ column: calibrate summary
260
+ field: template
261
+ template: '{status} ({duration_s:.0f}s)'
262
+ inbound:
263
+ - column: notes
264
+ to: plan_column
265
+ plan_column: notes
266
+ - column: image requested
267
+ to: plan_cell
268
+ step: image
269
+ ```
270
+
271
+ Headers match after trimming whitespace and ignoring case. A matching header
272
+ takes precedence over a literal column letter, including letters beyond Z.
273
+ Ambiguous headers and repeated row identities fail validation. Outbound rules
274
+ cannot change the target/code columns. Wildcards expand over project steps;
275
+ every substituted header must exist. Duplicate `(step, column)` targets are
276
+ rejected, including duplicates expressed once as a header and once as letters.
277
+
278
+ Inbound `plan_column` must name an existing extra column. It cannot change a
279
+ workflow step or the target, code, workdir or files columns. Inbound `plan_cell`
280
+ accepts only `todo` and `skip`. It calls the host's CSV updater under the plan
281
+ lock with an `only_if` guard excluding `running`. No step in the about-to-run
282
+ unit is touched; inbound requests affect later steps. Dry run changes neither
283
+ the sheet nor the plan CSV.
284
+
285
+ ### Matching rows by filenames
286
+
287
+ Existing mappings default to target-name matching. To match an AVICA sheet's
288
+ `FILENAMES` column, choose **Row-match column → FILENAMES** and **Match against →
289
+ Plan filenames** in Connect sheet, or edit Sync options. The persisted setting is:
290
+
291
+ ```yaml
292
+ rows:
293
+ key_column: FILENAMES
294
+ match_against: files
295
+ # code_column: PROJECT_CODE # optional disambiguation
296
+ ```
297
+
298
+ The backend resolves sheet values to the plan's targets; target names in the plan
299
+ are preserved. Commas, whitespace and newlines separate filenames, as in ALFRD's
300
+ plan CSV. Duplicate filenames and list order do not matter. Relative path segments
301
+ are normalized (`./a.fits` matches `a.fits`); matching is case-sensitive and does
302
+ not guess basenames or depend on files being present on disk. Every filename in a
303
+ sheet cell must belong to the same plan target. A cell can contain one file from
304
+ a target that has several files.
305
+
306
+ Setup shows backend-resolved samples (`sheet value → target`). Unknown file lists
307
+ are left unmatched; ambiguous lists prevent attachment/validation/sync. Two sheet
308
+ rows resolving to the same target also prevent sync. A configured project-code
309
+ column disambiguates shared filenames. If missing-row append is enabled, the new
310
+ row uses the plan's complete file list, never its target name; empty or ambiguous
311
+ file lists cannot be appended. Hooks read the running plan CSV; Studio validation
312
+ and connection samples read the configured project plan CSV.
313
+
314
+ ### Creating missing destination columns
315
+
316
+ When a named outbound destination is missing, the editor lists its concrete names
317
+ (including each `{step}` expansion). **Review column creation…** shows the worksheet,
318
+ header row and destination letters. Confirm **Create N columns** to insert new
319
+ columns and write their headers. Existing headers/data are kept; your draft stays
320
+ open, and **Save** remains a separate action. “Add status columns for all steps”
321
+ continues to add mapping rules for existing headers only.
322
+
323
+ New columns are inserted at the sheet's actual grid end, after its existing empty
324
+ columns too. Widen or clear a bounded read range if the reviewed letters are outside
325
+ it. Missing row keys, inbound sources, duplicate headers and literal column letters
326
+ need manual correction. Review rechecks current headers, and stale reviews cannot
327
+ write. After a failure, review again to reconcile any completed write before retrying;
328
+ insertion requests are never automatically replayed. Settings dry-run offers
329
+ **Preview column creation** and writes nothing.
330
+
331
+ ## Fields and formats
332
+
333
+ Fields: `status`, `exit_code`, `error`, `started`, `finished`, `duration_s`,
334
+ `log_path`, `log_path_rel`, `total_cost_usd`, `model`, `outcome`, `unit_id`,
335
+ `plan_id`, `target`, `code`, `usage.<key>`, `agent_usage.<key>`, `results.<key>`,
336
+ and `template`. Status comes from the individual row/step cell, falling back to
337
+ the unit status when unavailable. Agent/results dotted paths may be nested.
338
+
339
+ Usage keys: `wall_s`, `cpu_s`, `avg_cores`, `peak_mem`, `peak_mem_kind`,
340
+ `peak_rss`, `max_rss`, `read_bytes`, `write_bytes`, `samples`, `interval_s`,
341
+ `limited`, `counted`, `cgroup_memory_peak`, `workdir`, `workdir_start`,
342
+ `workdir_end`. Compatibility aliases: `wall` → `wall_s`, `cpu` → `cpu_s`,
343
+ `cores` → `avg_cores`, `io_read` → `read_bytes`, `io_write` → `write_bytes`.
344
+
345
+ | Format | Example |
346
+ | --- | --- |
347
+ | `raw` (default) | `done`, `0`; None/NaN becomes empty |
348
+ | `bytes` | `1048576` → `1 MiB` |
349
+ | `seconds` | `12.5` → `12.5s` |
350
+ | `duration` | `3661` → `01:01:01` |
351
+ | `datetime` | ISO timestamp (retains timezone) |
352
+ | `percent` | `0.125` → `12.5%` |
353
+ | `json` | Compact JSON with sorted object keys |
354
+
355
+ Templates use the same fields and Python format specifications. For example,
356
+ `{status} ({duration_s:.0f}s)`. Missing values are empty; a numeric specification
357
+ on a missing value is a validation/formatting error and suppresses that sync.
358
+
359
+ ## Writes and conflicts
360
+
361
+ Ordinary units read once before launch. After completion, an empty diff makes
362
+ no further API calls. Changed cells are checked in one `batchGet` (unless
363
+ `verify_before_write: false`) and sent in one `batchUpdate`, each range naming
364
+ one cell. Appending a missing target takes an additional full used-range read
365
+ under the sheet lock to choose vacant rows safely. Existing rows retain their
366
+ original snapshot for conflict checks.
367
+
368
+ Policies: `skip` omits cells edited since the snapshot; `overwrite` writes
369
+ anyway; `fail-soft` writes nothing for that unit and reports a hook warning.
370
+ Append refuses a row whose identity or header changed before writing, even
371
+ under `overwrite`. Re-adopted units take a fresh snapshot and do not replay
372
+ inbound changes.
373
+
374
+ The engine has a 25-second total budget per call, leaving margin for the
375
+ 30-second public hook timeout. All transport methods receive the same absolute
376
+ deadline, and both sheet and plan lock acquisition are bounded. POSIX flock
377
+ serializes processes under `<project>/.alfrd/locks/`; on Windows the sheet lock
378
+ serializes threads in this process only, matching the host's flock limitation.
379
+
380
+ History is written to `.alfrd/gsheet/sync.jsonl`, one entry per outcome or failed
381
+ before call. It includes changed cell addresses, counts, conflicts and safe
382
+ error text. Values, settings and context payloads are omitted to keep secrets
383
+ out of history. The core reports failures in `runner.log` and
384
+ `plugin.hook_failed` without changing the unit's status.
385
+
386
+ ## Command line
387
+
388
+ All commands take the project folder. Errors print one `gsheet: …` line on
389
+ stderr and exit 1. Credentials and Google response bodies are never printed.
390
+
391
+ | Command | What it does |
392
+ |---|---|
393
+ | `alfrd gsheet init <project> [--spreadsheet ID\|URL] [--worksheet NAME] [--header-row N] [--force]` | Reads the header row (default: first tab, row 1) and writes a starter `alfrd.gsheet.yaml`. Each step id that matches a column name gets a `status` rule. The file also includes commented usage/time/log examples. Without `--spreadsheet`, the default spreadsheet from Settings is used and not written to the file. The command refuses to overwrite the file without `--force`. |
394
+ | `alfrd gsheet validate <project> [--offline]` | Checks the mapping against the project's steps, then against the live header (one read). `--offline` skips the sheet. A missing or disabled mapping reports "Sync is off" and exits 0. |
395
+ | `alfrd gsheet diff <project> --unit ID [--plan ID]` | Replays one finished unit's after-sync against the current sheet. Prints `address: 'old' → 'new'` for each cell that would change. Writes nothing, including history. |
396
+ | `alfrd gsheet push <project> [--targets T1,T2] [--steps s1,s2] [--plan ID] [--dry-run]` | Backfills from finished units in all plans (or `--plan`). It takes the newest unit per target and step and uses the same diff and conflict policy as the hooks: one read, one conflict read, one write. The outcome is recorded in history as unit `push`. The `dry_run` setting also turns a push into a preview. |
397
+ | `alfrd gsheet status <project> [-n 20] [--json]` | The most recent `sync.jsonl` entries (UTC time, unit, steps, result, cells, conflicts, error). |
398
+
399
+ ## Google quotas
400
+
401
+ Requests per launched unit:
402
+
403
+ | Case | Reads | Writes |
404
+ |---|---|---|
405
+ | Unit whose mapped cells don't change | 1 | 0 |
406
+ | Ordinary unit | 2 (snapshot + conflict check) | 1 `batchUpdate` |
407
+ | `verify_before_write: false` | 1 | 1 |
408
+ | Appending a missing target row | +1 full used-range read | (same write) |
409
+ | Mapping with `gid` instead of `worksheet` | +1 metadata read | — |
410
+ | `dry_run` | 1 | 0 |
411
+
412
+ `push` makes one read, one conflict read and one write for any number of
413
+ units. `validate` makes one read; **Test** makes one metadata read and one
414
+ Drive permission read. A batch write counts as one request, however many
415
+ cells it changes.
416
+
417
+ The Sheets API limits requests per minute both per Cloud project and per
418
+ user (here, the service account). The published defaults are 300 per
419
+ project and 60 per user, counted separately for reads and writes. Check
420
+ current values under APIs & Services → Google Sheets API → Quotas. About 30
421
+ units finishing in one minute under one service account can therefore reach
422
+ the per-user read limit. 429 responses are retried up to three times within
423
+ the 25-second budget, and then the sync for that unit fails and is recorded.
424
+ Use `alfrd gsheet push` later to fill in what was missed, or give busy
425
+ projects separate service accounts.
426
+
427
+ ## Troubleshooting
428
+
429
+ Failures never change a step's result. They appear in `runner.log`, as a
430
+ `plugin.hook_failed` notification, in `alfrd gsheet status` and in the Studio panel.
431
+
432
+ | Message | Cause and fix |
433
+ |---|---|
434
+ | `Google denied access (403, Sheets API): share the sheet …` | Share the spreadsheet with the key's `client_email` as Editor. |
435
+ | `Google denied access (403): the Drive API is disabled in the key's Google Cloud project …` | Only Test's edit check uses Drive. Open the printed link and enable it (allow a few minutes), or untick **Check edit permission with the Drive API**. |
436
+ | `Google denied access (403): the Sheets API is disabled …` | Enable the Sheets API with the printed link. |
437
+ | `Google denied access (403): … quota exceeded` | Too many requests per minute; it clears by itself. |
438
+ | `Google Sheet not found (404) …` | Wrong spreadsheet id or URL, or the sheet isn't shared with the service account. |
439
+ | `The service account has read-only access …` | It is shared as Viewer or Commenter; change it to Editor. |
440
+ | `Google rejected the service-account credentials (401).` / `Service-account authentication failed …` | The key was deleted or disabled, or the account was removed. Create a new key and paste it again. |
441
+ | `Service-account credentials must be valid JSON.` / `… must contain a service-account client_email and private_key.` | Paste the whole downloaded JSON file, not an OAuth client or an API key. |
442
+ | `Set a default spreadsheet id or URL before running Test.` | Test needs the default spreadsheet, even when mappings name their own. |
443
+ | `Google Sheet unavailable after 3 attempts (HTTP 429)` | Quota exceeded; see [Google quotas](#google-quotas). Backfill later with `push`. |
444
+ | `Google Sheet request deadline exceeded.` / `… sync time budget exhausted` | The network is slow or Google responded slowly. Lower `request_timeout` so retries fit in the budget. A timed-out write may still have landed; check with `diff`. |
445
+ | `Google Sheet sync lock timed out` | Another unit or `push` held the sheet lock for the whole budget. It usually clears; otherwise look for a stuck `alfrd gsheet` process. |
446
+ | `unknown sheet column 'X'` / `N problems: unknown column …` | A header was renamed or deleted, or a `{step}` wildcard names a column that doesn't exist. Rename the header or fix the rule; then run `validate`. |
447
+ | `ambiguous sheet column …` | Two headers have the same name after trimming and ignoring case. Rename one, or use the column letter. |
448
+ | `duplicate sheet row identity …` | The key column repeats a target. Add `rows.code_column` or remove the duplicate row. |
449
+ | `unknown step 'x'` | The rule names a step that isn't in the project's workflow. |
450
+ | Cells listed under *conflicts* and not written | Someone edited the cell while the step ran (`conflict_policy: skip`). Rerun `push` once the edit is resolved, or choose `overwrite`. |
451
+ | Nothing is written | Check that `enabled: true` is set and the target is in the key column. With `missing_row: skip`, absent targets are ignored, and the `when` filters must match. A `Dry run` chip or setting writes nothing. |
452
+ | Panel says "No step has finished since sync was set up." | No hooked unit has finished yet. The legacy `alfrd runtime` worker path has no hooks; use `push`. |
453
+
454
+ ## Studio panel and validation
455
+
456
+ After installing and restarting Studio, the panel appears automatically for any
457
+ project with an `alfrd.gsheet.yaml`, including disabled or invalid mappings. An
458
+ explicit `views.metadata` entry is optional; existing entries keep working and
459
+ are not duplicated:
460
+
461
+ ```yaml
462
+ views:
463
+ metadata:
464
+ - panel: gsheet_sync
465
+ title: Google Sheet
466
+ scope: project
467
+ ```
468
+
469
+ The panel reads only `alfrd.gsheet.yaml` and `.alfrd/gsheet/sync.jsonl`. Opening it
470
+ makes no Google requests. It shows the newest sync per step, ten initially, with
471
+ **Show all** to expand the table. Counts are unit totals when a unit covers several
472
+ steps. History scanning is limited to the latest 1 MiB; `alfrd gsheet status` can
473
+ inspect older records. Interrupted or malformed history lines are ignored.
474
+
475
+ Use **Google Sheet: validate mapping** in the command palette for an explicit live,
476
+ read-only check of the selected project. It shares validation with the CLI and
477
+ reports mapping problems or connection errors in a toast. The request uses the
478
+ host's authenticated, loopback and CSRF-protected project-check endpoint.
479
+
480
+ Because the panel reads no global settings, its dry-run chip reflects the most
481
+ recent recorded sync. A sheet id inherited from Settings becomes a link after the
482
+ first recorded sync. A mapping with a numeric `gid` links directly to that tab;
483
+ worksheet titles link to the spreadsheet because resolving a title to a gid would
484
+ require a Google metadata request. Validate after changing global settings. The
485
+ panel checks intrinsic mapping syntax; workflow step and live-column checks belong
486
+ to the explicit validation command.
487
+
488
+ ## Studio
489
+
490
+ Overview shows a **Google Sheet ▾** menu next to **Export** (it appears after restarting
491
+ `alfrd serve` with the plugin enabled). Everything below works without a terminal
492
+ and never edits `alfrd.yaml`.
493
+
494
+ - **Connect & validate…** opens the connection dialog, with sheet name, worksheet, mapping facts and last sync. A sheet without a mapping shows **Not connected** and **Connect sheet…**.
495
+ A saved mapping shows **Connected**; this indicates attachment, not a live
496
+ connectivity check. Use **Validate** to check sheet access. An unreadable mapping
497
+ shows **Needs attention**, with its error and recovery actions.
498
+ - **Connect sheet** (or the palette command **Google Sheet: attach…**) opens a
499
+ three-step dialog: paste a sheet URL or ID (or use the Settings default), **Load**
500
+ it, pick the tab and header row, then the target-name column (guessed from
501
+ `TARGET_NAME`, `target`, `source`, `name`) and an optional project-code column.
502
+ The first key values and how many match the project's plan targets are shown before
503
+ you attach. A URL with `#gid=` preselects that tab. Without a service-account key
504
+ the connection dialog links to Settings → Plugins → Google Sheet instead.
505
+ - **Edit mapping** (or **Google Sheet: open mapping editor**) edits outbound rules
506
+ (step, column, field with usage/results key, format, when, template), inbound rules
507
+ and options. Rows can be added, duplicated, moved and deleted; **Add status columns
508
+ for all steps** adds the `status` rules `init` would add. Problems from live
509
+ validation (600 ms after a change) appear next to the control and in a summary;
510
+ **Save** stays disabled until they are fixed. Saving rewrites the file in a stable
511
+ order and removes comments, so the first save asks for confirmation. If the file
512
+ changed on disk since it was opened, Save offers **Reload** instead of overwriting.
513
+ - **Validate connection**, **Preview changes…** (cell, target, step, column, old → new; the first 500 are
514
+ listed) and the menu’s **Export to sheet…** (preview first, then "Write N cells to <tab>?"). With
515
+ Settings dry run on, backfill writes nothing and says so.
516
+ - The menu’s **Sync** focuses the **Update this sheet during runs** switch and **Detach…**: disable sync and keep the file, or delete
517
+ `alfrd.gsheet.yaml` (sync history stays).
518
+
519
+ The menu, connection dialog, attach and editor call protected project actions. `plugin_actions: false` in
520
+ `plugins.json` (or `alfrd serve --no-plugin-actions`) disables Attach, Save, the switch,
521
+ Backfill and Detach; Validate and Preview stay available. Mapping edits check a SHA-256
522
+ revision under `.alfrd/locks/alfrd.gsheet.yaml.lock` and replace the file atomically;
523
+ CLI init uses the same lock, so hand edits and the Studio never silently overwrite
524
+ each other.
525
+
526
+ The form uses labelled sections, associated helper text, visible keyboard focus
527
+ and a single-column layout on narrow screens. The mapping tables scroll inside
528
+ their own labelled regions. See [OVERVIEW-UX.md](OVERVIEW-UX.md) and [STUDIO-UX.md](STUDIO-UX.md) for the presentation
529
+ specification and the proposed file-matching and column-creation flows (those two
530
+ flows are not implemented yet).