@datarails-ai/dr-cli 1.6.94 → 1.6.96
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli.js +7001 -6316
- package/dist/scopes-map.json +21 -1
- package/dist/skills/create-connector-lib/SKILL.md +6 -2
- package/dist/skills/datarails-connectors-onboard-connector/SKILL.md +20 -9
- package/dist/skills/dr-cli/SKILL.md +5 -1
- package/dist/skills/dr-cli/reference/filebox-documents.md +32 -2
- package/dist/skills/dr-cli/reference/integrations.md +34 -0
- package/dist/skills/dr-cli/reference/reconciliation-mec.md +14 -2
- package/package.json +2 -2
package/dist/scopes-map.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"version": 1,
|
|
3
|
-
"cliVersion": "1.6.
|
|
3
|
+
"cliVersion": "1.6.96",
|
|
4
4
|
"_contract": "Resolve a command path through `aliases` (canonical prefix + alias segment -> canonical path) before any lookup. A path listed in `pendingCommands`, or whose first segment is a key of `pending`, is UNDECIDED, not unrestricted: fail closed or skip it, and never read a path's absence from `commands` as 'no requirement'. An entry carrying `denied` is UNAVAILABLE to any AGENT-SCOPED credential — one whose `scopes` claim carries at least one `resource:action` entry; a claim that is absent, empty, capability-only or wholly unparseable is UNRESTRICTED, and this mark does not apply to it — whatever else that entry says: it is not a ceiling and it is not always-available, its `denied.reason` says why and `denied.until` names the ticket that lifts it, and the absence of `requires` on it must NEVER be read as 'no requirement'. Only an entry in `commands` with an empty `requires` means always-available.",
|
|
5
5
|
"commands": {
|
|
6
6
|
"login": {
|
|
@@ -1795,6 +1795,26 @@
|
|
|
1795
1795
|
}
|
|
1796
1796
|
]
|
|
1797
1797
|
},
|
|
1798
|
+
"integrations sftp create": {
|
|
1799
|
+
"requires": [
|
|
1800
|
+
{
|
|
1801
|
+
"resource": "integrations",
|
|
1802
|
+
"action": "write"
|
|
1803
|
+
},
|
|
1804
|
+
{
|
|
1805
|
+
"resource": "fileboxes",
|
|
1806
|
+
"action": "write"
|
|
1807
|
+
}
|
|
1808
|
+
]
|
|
1809
|
+
},
|
|
1810
|
+
"integrations sftp provision": {
|
|
1811
|
+
"requires": [
|
|
1812
|
+
{
|
|
1813
|
+
"resource": "integrations",
|
|
1814
|
+
"action": "write"
|
|
1815
|
+
}
|
|
1816
|
+
]
|
|
1817
|
+
},
|
|
1798
1818
|
"connectors publish": {
|
|
1799
1819
|
"denied": {
|
|
1800
1820
|
"reason": "The connectors control plane runs no dr-auth-sdk guard and checks no scope, so the `integrations` ceiling this command declares would be enforced by nobody. Closed to scoped credentials until that control plane adopts the SDK.",
|
|
@@ -60,7 +60,9 @@ parameter fails in `connect()` before the vendor is called.
|
|
|
60
60
|
synthetic. Per-customer values (columns, category or custom-field names) live only in thin files.
|
|
61
61
|
Before committing, grep the lib diff for the customer's name, domain and org id.
|
|
62
62
|
- **Reuse before adding.** Use the SDK kit first (`from dr_integrations_sdk import kit`: logger, query
|
|
63
|
-
vars, HTTP client, CSV writer, paging, auth, dates
|
|
63
|
+
vars, HTTP client, CSV writer, paging, auth, dates; on kit 1.1 also `config.is_unlimited`,
|
|
64
|
+
`http.json`, `http.same_origin`, `paging.gather_or_cancel`, `runtime.Deadline` and
|
|
65
|
+
`CsvWriter(block_bytes=)`), then `_core` helpers and an existing connection
|
|
64
66
|
that fits; never a second copy. Name modules after what they do, never after who uses them. Hand the
|
|
65
67
|
user a list of lib improvements you noticed (duplicated helpers, code that belongs in `_core`, a
|
|
66
68
|
hand-written copy of a kit helper), done in their own PR.
|
|
@@ -81,7 +83,9 @@ parameter fails in `connect()` before the vendor is called.
|
|
|
81
83
|
the header-once, one-block-per-page writer and takes `csv.writer` dialect kwargs
|
|
82
84
|
(`lineterminator="\n"`, `quoting=`), but writes each value as `csv.writer` does (`str()`, `None` as an
|
|
83
85
|
empty cell), so the engine still pre-formats every cell the way the landed file shows (`true`, not
|
|
84
|
-
`True`).
|
|
86
|
+
`True`). Kit 1.1's `CsvWriter(cell=kit.csv.cell)` is a generic formatter, not a parity one: use it
|
|
87
|
+
only when its output matches the landed file byte for byte, else pass the engine's own function as
|
|
88
|
+
`cell=`.
|
|
85
89
|
- **Credentials resolve at runtime from the datasource** (`get_credentials`), never from code, tests
|
|
86
90
|
or logs (`tech/security.md` §4). Check a migrated credential's shape without printing it; a stored
|
|
87
91
|
header value may carry its scheme (`Basic <base64>`).
|
|
@@ -194,22 +194,32 @@ dataquery that defines its own `_get_logger`, `_query_vars`, `_safe_snippet`, `_
|
|
|
194
194
|
call below before it is reviewed. A template gives you the structure; the logger, config, retry, CSV
|
|
195
195
|
and paging code comes from
|
|
196
196
|
`from dr_integrations_sdk import kit` (import the package, not `kit.csv`/`kit.logging` by name — they
|
|
197
|
-
would shadow the stdlib).
|
|
198
|
-
|
|
199
|
-
|
|
197
|
+
would shadow the stdlib). Every template is built on it (`sample_http_vendor`, `sample_streaming_fanout`,
|
|
198
|
+
`lib/sample_vendor`), so a copy starts with no hand-written helpers. Reference: dr-connectors-runner
|
|
199
|
+
`docs/connector-kit.md`.
|
|
200
200
|
|
|
201
201
|
| Need | Kit |
|
|
202
202
|
|---|---|
|
|
203
203
|
| a logger whose lines survive the sandbox | `kit.logging.get_logger("<vendor>__<dq>")` |
|
|
204
204
|
| the merged config channels, required variables | `kit.config.query_vars(req)`, `kit.config.require(vars, *names, context=)` (raises `PermanentError` naming each missing one) |
|
|
205
|
+
| *(1.1)* a cap variable that may mean "no cap" | `kit.config.is_unlimited(value)`: `None`, `""`, `0` and the string `"0"` all mean unlimited |
|
|
205
206
|
| vendor text in a log line or an error | `kit.redact.safe_snippet` / `log_safe` (redacts credentials); `one_line` for ids and cursors |
|
|
206
207
|
| HTTP with retry and 4xx classification | `kit.http.client(...)`, an `httpx.AsyncClient` (every httpx kwarg passes through): retries 429 (honouring `Retry-After`), 500/502/503/504 and transport errors; 401/403 raise `AuthError`, any other 4xx or an unfollowed redirect `PermanentError`; a 429/5xx still failing after the last attempt raises `httpx.HTTPStatusError` (runtime retry). `raise_for_status=False` hands 4xx back to you |
|
|
208
|
+
| *(1.1)* a JSON body you must not misread as an empty page | `kit.http.json(response, expect=list)`: `PermanentError` with a redacted snippet when the body is not JSON or not that type |
|
|
209
|
+
| *(1.1)* a vendor next link fetched with the credential | `kit.paging.cursor_pages(page, same_origin=base_url)` refuses a link to another scheme, host or port before fetching it; `kit.http.same_origin(url, base)` / `require_same_origin` for links you follow yourself; *(1.2)* `kit.http.client(base_url=..., same_origin_only=True)` refuses every request off that origin, redirect hops included |
|
|
207
210
|
| the CSV header once, one block per page | `kit.csv.CsvWriter(fields).blocks(req, pages)`; `.rows` counts rows written |
|
|
211
|
+
| *(1.1)* many small pages | `CsvWriter(fields, block_bytes=kit.csv.BLOCK_BYTES)` coalesces them into ~4 MB blocks, far under Azure's 50,000-block limit |
|
|
212
|
+
| *(1.1)* typed values as CSV cells | `CsvWriter(fields, cell=kit.csv.cell)`: bool `true`/`false`, `Decimal` without exponent, tz-aware datetime as naive UTC, bytes hex, dict/list compact JSON. Not on a Workato migration: there the cell shape comes from the landed file (see *create-connector-lib*) |
|
|
208
213
|
| paging that fails loudly | `kit.paging.cursor_pages` / `offset_pages`; `kit.paging.waves(items, worker, concurrency=)` |
|
|
214
|
+
| *(1.1)* a handful of concurrent vendor calls | `kit.paging.gather_or_cancel(awaitables)`: cancels the siblings on the first failure and re-raises it unwrapped. Never `asyncio.TaskGroup`: its `ExceptionGroup` turns a `PermanentError` into 5 retries |
|
|
215
|
+
| *(1.1)* the hook's time budget | `deadline = kit.runtime.Deadline("<vendor> <dq>")` in `fetch`: *(1.2)* it counts from the runner's own start of the hook and stops 60 s short of the timeout the runner kills at, watching CPU time too. `deadline.check(progress)` per page, `await deadline.sleep(s, progress)` for poll waits, and `kit.http.client(deadline=deadline)` for retry waits. Each raises `PermanentError` naming the progress |
|
|
209
216
|
| credentials and tokens | `kit.auth.credential(creds, *keys, vendor=)`, `client_credentials` / `password_grant` → `Token.header()`, `basic_header`, `bearer_header` |
|
|
210
217
|
| a date window | `kit.dates.date_window(vars, start_keys=, end_keys=, required=)`, `DateWindow.chunks(days)`, `parse_date` |
|
|
211
218
|
|
|
212
|
-
**
|
|
219
|
+
**Kit 1.2 is live in every production ring** (runner image `connectors-runner-base` v1.35.0, since
|
|
220
|
+
2026-10-02), so every row above is usable there and the kit is never a reason to fall back to copied
|
|
221
|
+
helpers. The *(1.1)* / *(1.2)* marks say which kit version a row needs, for an environment whose runner
|
|
222
|
+
image is older.
|
|
213
223
|
At runtime the kit comes from the **runner image**, not from `.sdk-ref` (that pins only what CI tests
|
|
214
224
|
against). An image with the kit prints
|
|
215
225
|
`INFO:dr.sdk: hook=… correlation_id=… kit=<version>` as each hook's first log line. On a dev or test
|
|
@@ -309,9 +319,10 @@ Two more sandbox ceilings worth knowing, both undocumented elsewhere:
|
|
|
309
319
|
ThreadPoolExecutor **never reaps its threads** — so an unbounded pool can claim ~256 MB of the
|
|
310
320
|
512 MB and never give it back. That is why a brief concurrency spike used to poison every *later*
|
|
311
321
|
phase of a run with `RuntimeError: can't start new thread` and SSL allocation failures long after
|
|
312
|
-
the spike finished.
|
|
313
|
-
`
|
|
314
|
-
|
|
322
|
+
the spike finished. **The runner now caps it at 8 threads** before every hook
|
|
323
|
+
(`sandbox_entry.py` `SANDBOX_EXECUTOR_THREADS`, DR-53651), so a connector no longer installs its own
|
|
324
|
+
`set_default_executor(...)`; remove one when you touch the file.
|
|
325
|
+
- **`RLIMIT_CPU = 2700s`** sits right next to the memory limit in `sandbox_entry.py`. Exceeding it
|
|
315
326
|
SIGKILLs the hook (runner reports `sandbox exited -24`) with no `MemoryError` to explain it. Async
|
|
316
327
|
I/O waiting is free, but TLS handshakes, gzip and JSON parsing are not — log
|
|
317
328
|
`time.process_time()` alongside wall clock so you can see how close you are.
|
|
@@ -871,8 +882,8 @@ unmerged connector has no version to ingest, bind, or run.
|
|
|
871
882
|
| Confused about publish vs ingest | CI publishes to JFrog on merge (cuts the git tag `vN`); the CLI ingests a published version into a DC | `dr connectors publish <path/to/dataqueries/<dq>.py> --server DEV --account <customer-user-email> --json` reads the newest git tag (`git fetch --tags` + max `vN`, or `--tag vN`) — **no local compose** — then the control-plane **pulls that tag from JFrog** (no upload) and registers it (needs `NODE_EXTRA_CA_CERTS`); returns `{name, version: "vN", digest}`. |
|
|
872
883
|
| Run `Failed`, `connect` completion `verdict: ok` but `fetch` completion `verdict: retry`, error `'async for' requires an object with __aiter__ method, got function` | Passed the async-generator **function** to `write_output` — `write_output(req, gen)` instead of **calling** it `write_output(req, gen())`. `write_output` does `async for chunk in <arg>`, which needs an async **iterator**, not a function. (The connect gate passing but fetch dying instantly is the tell.) | Call the generator: `write_output(req, gen())`. The golden template already shows `write_output(req, rows())` / `write_output(req, blocks())` — match it exactly; the missing `()` is easy to drop. |
|
|
873
884
|
| Vendor **4xx** (e.g. `402 Payment Required`, `401`, `403`, bad-symbol `404`) makes the `fetch` completion `verdict: retry` and the run **retries forever**; the HTTP status shows ONLY in the completion store, never in the Inngest run `output` (which is `null`). | A bare `r.raise_for_status()` raises `httpx.HTTPStatusError` for **every** status ≥ 400, and the runtime classifies that as **transient → retry**. But a plan/auth/bad-request 4xx will never succeed on retry. | Use `kit.http.client` (skill §1): 401/403 raise `AuthError` and any other 4xx `PermanentError`, each naming the status and a redacted body; 429 and 5xx are retried, then left to the runtime's retry (genuinely transient). Concrete case: FMP's free tier answers **402** when `limit` exceeds its historical allowance (dev key caps statements at the last **5** annual records) — a small `limit` or a paid key fixes that one; the PermanentError stops the infinite loop for the general case. The golden `sample_http_vendor` template is built on this client. |
|
|
874
|
-
| Connector works perfectly in local/dev testing, then in production `fetch` dies with `MemoryError`, `MemoryError: Unable to allocate output buffer`, an SSL/crypto allocation failure, or `RuntimeError: can't start new thread` — often within a second of "fetch start", and deterministically on every one of the 5 attempts | The sandbox caps **virtual address space at `RLIMIT_AS` = 512 MB** (`dr-connectors-runner/src/runner/sandbox_entry.py`), and the runner pod's limit is also 512Mi. The guard is `if sys.platform != "win32"`, so **the ceiling does not exist on a Windows dev box** — a buffering connector cannot fail locally. Two causes, usually together: (a) the fetch accumulates the result set (a `rows` list, `asyncio.gather` over every parent, a full CSV string + its `.encode()` copy) — journal_lines measured **378 MB live to emit 67 MB**, 243 MB of it row dicts at 846 B/row; (b) asyncio's **default** DNS ThreadPoolExecutor grew to `min(32, node_cores + 4)` threads × 8 MB stack ≈ **256 MB of address space that is never reclaimed**, which is why a brief concurrency spike poisons every later phase of the run. | Stream: do the fetching **inside** the generator passed to `write_output`, replace gather-over-everything with a bounded worker pool feeding a bounded `asyncio.Queue`, format rows to CSV bytes immediately (never hold dicts), and re-batch to ~4 MB blocks.
|
|
875
|
-
| Hook dies with no traceback and no `MemoryError`; runner reports `sandbox exited -24` | **`RLIMIT_CPU =
|
|
885
|
+
| Connector works perfectly in local/dev testing, then in production `fetch` dies with `MemoryError`, `MemoryError: Unable to allocate output buffer`, an SSL/crypto allocation failure, or `RuntimeError: can't start new thread` — often within a second of "fetch start", and deterministically on every one of the 5 attempts | The sandbox caps **virtual address space at `RLIMIT_AS` = 512 MB** (`dr-connectors-runner/src/runner/sandbox_entry.py`), and the runner pod's limit is also 512Mi. The guard is `if sys.platform != "win32"`, so **the ceiling does not exist on a Windows dev box** — a buffering connector cannot fail locally. Two causes, usually together: (a) the fetch accumulates the result set (a `rows` list, `asyncio.gather` over every parent, a full CSV string + its `.encode()` copy) — journal_lines measured **378 MB live to emit 67 MB**, 243 MB of it row dicts at 846 B/row; (b) asyncio's **default** DNS ThreadPoolExecutor grew to `min(32, node_cores + 4)` threads × 8 MB stack ≈ **256 MB of address space that is never reclaimed**, which is why a brief concurrency spike poisons every later phase of the run. | Stream: do the fetching **inside** the generator passed to `write_output`, replace gather-over-everything with a bounded worker pool feeding a bounded `asyncio.Queue`, format rows to CSV bytes immediately (never hold dicts), and re-batch to ~4 MB blocks (`kit.csv.CsvWriter(..., block_bytes=kit.csv.BLOCK_BYTES)` on kit 1.1). The executor cap (b) is now set by the runner (8 threads); a connector installs none. Full rules in §1 *Memory*; working template `samples/sample_streaming_fanout/`. Reproduce locally in `python:3.12-slim` under the same rlimits — never trust a Windows run for this. |
|
|
886
|
+
| Hook dies with no traceback and no `MemoryError`; runner reports `sandbox exited -24` | **`RLIMIT_CPU = 2700s`** (`sandbox_entry.py`, right beside the memory limit) — 2700 seconds of **CPU** time, summed over all threads, then SIGXCPU/SIGKILL. Async I/O waiting is free, but TLS handshakes, gzip and JSON parsing across thousands of requests are not. | Cut CPU per row (fewer/larger vendor pages, `onlyData` where the vendor supports it, no per-row dict churn) and log `time.process_time()` next to wall clock so proximity to the cap is visible before it kills a run. |
|
|
876
887
|
| A sync `Completed` and the file **looks** complete, but the customer's totals are short | A page-limit "safety cap" (`_LINE_PAGE_LIMIT`, `_HEADER_PAGE_LIMIT`, …) tripped and the code **logged a warning and returned what it had**, so a partial result landed as if whole. Same failure class as a missing offset follow-up: journal_lines silently dropped **113,171 rows** of one customer's month this way. | Caps must `raise PermanentError`, never warn-and-return — a failed sync is recoverable, a silently partial landing gets reconciled against and believed. Audit every `log.warning` that is followed by `return`. |
|
|
877
888
|
|
|
878
889
|
## Env quick reference (dev germany)
|
|
@@ -299,6 +299,7 @@ area's gotchas.
|
|
|
299
299
|
reconnect before the next sync._
|
|
300
300
|
- `integrations share users|groups`
|
|
301
301
|
- `integrations it list|get|add|update|delete|resync` (IT-admin file-sync; redacts secrets by default)
|
|
302
|
+
- `integrations sftp create|provision` — self-serve SFTP fileboxes: `create --name <n>` makes a filebox and its SFTP folder `/<name>_<id>` in one step; `provision <filebox_id>` does it for an existing filebox
|
|
302
303
|
- `entities list|qbd-availability|netsuite-connections|rename|rename-group|delete|mark-completed` (alias `companies`)
|
|
303
304
|
- `entities create qbo|xero|qbd|netsuite` (connected accounting systems)
|
|
304
305
|
- `error-enrichments list|stats|get|create|update|delete|test-regex|export|import` — the
|
|
@@ -611,7 +612,10 @@ network traffic or hand-edits JSON. Full recipes in the relevant reference file.
|
|
|
611
612
|
date to whatever granularity the filebox's own DATE tag actually uses —
|
|
612
613
|
no need to know if it's yearly/quarterly/monthly/daily/weekly. `--tag` is
|
|
613
614
|
an unvalidated escape hatch that forwards straight to the server; a
|
|
614
|
-
malformed value 500s with no error body.
|
|
615
|
+
malformed value 500s with no error body. To know when it finished, add
|
|
616
|
+
`--wait` — never poll `status` or the version count: an unchanged result
|
|
617
|
+
writes no new version and still reads `SYNCED`. See
|
|
618
|
+
reference/filebox-documents.md.
|
|
615
619
|
- **"User has no table permissions" is almost always a misread.** To see which
|
|
616
620
|
tables a user can access, use `dr users permissions get <id>` — never the raw
|
|
617
621
|
`GET /api/it/users/<id>` object's `permissions` field. That field is Django
|
|
@@ -169,8 +169,12 @@ set-version-include` (below).
|
|
|
169
169
|
only allow-listed keys are accepted (currently `is_bank_integration`); unknown
|
|
170
170
|
keys are rejected client-side (the server silently accepts dead keys).
|
|
171
171
|
- `filebox data-query get|set|clear|sync <fb_id>` — read/write/delete/poke the
|
|
172
|
-
Workato data-query. `get` shows `sync_interval` (cron)
|
|
173
|
-
|
|
172
|
+
Workato data-query. `get` shows `sync_interval` (cron), `last_sync` and
|
|
173
|
+
`Last start` (when the latest sync started); `set --payload-file` PUTs a
|
|
174
|
+
create/update body; `clear` is two-step. `last_sync` doubles as the
|
|
175
|
+
auto-sync debounce, so it can read minutes in the future. Older servers do
|
|
176
|
+
not return either timestamp: `last_sync` then reads `never` and `Last start`
|
|
177
|
+
is omitted.
|
|
174
178
|
- `filebox data-query sync <fb_id> [--date YYYY-MM-DD]` — trigger an async
|
|
175
179
|
sync, optionally scoped to specific period(s). `--date` is repeatable and
|
|
176
180
|
takes one plain calendar date — the CLI reads the filebox's own DATE
|
|
@@ -195,6 +199,32 @@ set-version-include` (below).
|
|
|
195
199
|
constant — varies per tenant) and the epoch just needs to fall within the
|
|
196
200
|
target period (the server normalizes it). Prefer `--date`; reach for raw
|
|
197
201
|
`--tag` only when it doesn't cover your case.
|
|
202
|
+
- **Waiting for a sync: `filebox data-query sync <fb_id> --date … --wait`.**
|
|
203
|
+
Never poll `status` or the version count yourself — both lie:
|
|
204
|
+
- With `sync_only_changes` (the default), a sync whose output is identical
|
|
205
|
+
to the stored latest version for that period writes **no new version**.
|
|
206
|
+
`status` still goes to `SYNCED`, and the version count never moves. This
|
|
207
|
+
is not an error.
|
|
208
|
+
- Right after a sync is queued, `status` can still show the previous run's
|
|
209
|
+
value (or `-`) until the sync starts. A loop that waits for "not
|
|
210
|
+
SYNCING" exits at once.
|
|
211
|
+
- A multi-period sync flips `status` to `SYNCED` after each period, so
|
|
212
|
+
"status is SYNCED" can mean one period of several.
|
|
213
|
+
- A premature exit followed by `filebox download` silently returns the
|
|
214
|
+
**previous** version.
|
|
215
|
+
|
|
216
|
+
`--wait` snapshots each `--date` period's versions before triggering, then
|
|
217
|
+
polls until every period either gained a new version (**new version**) or
|
|
218
|
+
had its latest version re-confirmed — the server bumps that version's
|
|
219
|
+
`last_sync` instead of writing one (**unchanged**). It exits non-zero when
|
|
220
|
+
the data-query goes to `ERROR` (with the server's `last_sync_error`) or on
|
|
221
|
+
`--timeout <seconds>` (default 1800). `--json` prints one object:
|
|
222
|
+
`{outcome, periods: [{period, result: new_version|no_change|pending,
|
|
223
|
+
version_id, version}], unmatched_new_versions, error, waited_seconds}`.
|
|
224
|
+
Without `--date`, it waits for the first period to finish. It cannot be
|
|
225
|
+
combined with `--tag`. One blind spot: a sync that returns **no rows**
|
|
226
|
+
writes no version and re-confirms none, so it ends in a timeout whose
|
|
227
|
+
message says so — check `dr filebox data-query get <fb_id>`.
|
|
198
228
|
- `filebox data-query set-sync-interval <fb_id> --cron "<expr>"` — change how
|
|
199
229
|
often the data-query polls (Workato-backed **and** connector-platform /
|
|
200
230
|
`DatarailsConnector` fileboxes both use this cron). **Use this for sync
|
|
@@ -34,6 +34,40 @@ secrets by default.
|
|
|
34
34
|
`--set-config KEY=VAL`), `integrations it delete` (two-step confirm),
|
|
35
35
|
`integrations it resync`.
|
|
36
36
|
|
|
37
|
+
## Self-serve SFTP (`integrations sftp`)
|
|
38
|
+
|
|
39
|
+
The embedded SFTP integration: each filebox gets its own folder in the org's SFTP
|
|
40
|
+
container, and files dropped there land in the filebox.
|
|
41
|
+
|
|
42
|
+
- `integrations sftp create --name <n> [flags]` creates an SFTP filebox in one step
|
|
43
|
+
(`--scenario`, `--date-sub-type`, `--parent`, `--no-date` as on `filebox create`).
|
|
44
|
+
`integrations sftp provision <filebox_id> [flags]` sets up SFTP on a filebox that
|
|
45
|
+
already exists. Both make the UI wizard's call,
|
|
46
|
+
`POST /api/v3/data_integrations/fileboxes/<id>/sftp/provision`, which creates the
|
|
47
|
+
folder `/<filebox name>_<filebox id>` (with a `.keep` file) and, for the org's
|
|
48
|
+
first SFTP filebox, the org's SFTP user and password.
|
|
49
|
+
- The org is the logged-in account's org (`--server`, `--account`; `dr login-as` to
|
|
50
|
+
act as a customer). There is no org flag.
|
|
51
|
+
- Give the client the printed **Folder** unchanged. An SFTP login opens inside the
|
|
52
|
+
org's container, so the container name is never part of the path. **Host** is the
|
|
53
|
+
bare hostname.
|
|
54
|
+
- `--date-method filename|upload_date`. A date-tagged filebox defaults to
|
|
55
|
+
`upload_date`. `filename` also needs `--date-format` (one of the 10 formats the UI
|
|
56
|
+
offers) and `--date-position beginning|end`. `--backfill <n>` adds subfolders
|
|
57
|
+
`<folder>/-1` … `<folder>/-N` (`upload_date` only). A `--no-date` filebox takes none
|
|
58
|
+
of these flags.
|
|
59
|
+
- Safe to re-run: no folder is duplicated, and flags left out keep their stored values.
|
|
60
|
+
- The password is redacted unless `--show-secrets`, in `--json` output too. It is the
|
|
61
|
+
org's password: one SFTP user serves every SFTP filebox in the org, so revealing it
|
|
62
|
+
gives access to all of their folders.
|
|
63
|
+
- Errors: 404 "not enabled" means the org's `integrations__sftp` flag is off. 403
|
|
64
|
+
means the account does not own the filebox, or is a viewer. 400 "already connected" means the
|
|
65
|
+
filebox is linked to another data source. 502 means the gateway is unavailable;
|
|
66
|
+
re-run. When `integrations sftp create` fails at the SFTP step, the filebox
|
|
67
|
+
already exists, so finish with `integrations sftp provision <id>`.
|
|
68
|
+
- Do not copy SFTP settings from another filebox with `filebox data-query set`. That
|
|
69
|
+
links the filebox but creates no folder, so the client has nowhere to drop files.
|
|
70
|
+
|
|
37
71
|
## Entities — company data sources (`entities`, alias `companies`)
|
|
38
72
|
|
|
39
73
|
The connected accounting systems (QuickBooks Online/Desktop, Xero, NetSuite) —
|
|
@@ -22,7 +22,14 @@ Account assignment / sign-off (all take the bank-account UUID via
|
|
|
22
22
|
|
|
23
23
|
- `reconciliation accounts update-settings --file <p>` — admin-only PATCH.
|
|
24
24
|
Two-step confirm; the token binds to a hash of the file's rows, so editing
|
|
25
|
-
the file between plan and confirm invalidates it.
|
|
25
|
+
the file between plan and confirm invalidates it. Rows are matched to the
|
|
26
|
+
tenant by `(entity, account_group, name)`; only rows that differ are sent,
|
|
27
|
+
in PATCHes of at most 256 KB (one whole-tenant PATCH near 1 MB returns
|
|
28
|
+
HTTP 500). The plan reports rows to change, rows skipped as already
|
|
29
|
+
matching, and the changes by target frequency (`Exclude` is how an account
|
|
30
|
+
is excluded from reconciliation). If a PATCH fails, the error says how many
|
|
31
|
+
rows landed; re-run the same file without a token and the new plan covers
|
|
32
|
+
only the rest.
|
|
26
33
|
- `reconciliation accounts set-assignee <account_uuid> --date <YYYY-MM-DD>
|
|
27
34
|
--type Preparer|Reviewer --user-id <n> [--user-name <s> --user-email <s>]` —
|
|
28
35
|
resets any prior sign-off; name/email are auto-fetched if not supplied.
|
|
@@ -38,7 +45,12 @@ Account assignment / sign-off (all take the bank-account UUID via
|
|
|
38
45
|
resets. Every account is checked before anything is written, and the
|
|
39
46
|
write goes out in PATCHes of 100 accounts. If one fails, the error says
|
|
40
47
|
how many accounts landed; re-run the same file without a token and the
|
|
41
|
-
new plan covers only the rest.
|
|
48
|
+
new plan covers only the rest. Before each PATCH after the first, the
|
|
49
|
+
accounts are re-read; if any account in the next chunk is gone (a
|
|
50
|
+
trial-balance rebuild re-mints account ids) or its member slots changed
|
|
51
|
+
since the plan, the run halts, says how many accounts landed, and names
|
|
52
|
+
the drifted accounts. Gone accounts mean the file must be rebuilt from
|
|
53
|
+
`accounts list` once the rebuild settles. Do not script a loop of single
|
|
42
54
|
`set-assignee` calls for this.
|
|
43
55
|
- `reconciliation accounts clear-assignee <account_uuid> --date <YYYY-MM-DD>
|
|
44
56
|
--type Preparer|Reviewer` — unassigns (preserves due_date, resets sign-off).
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@datarails-ai/dr-cli",
|
|
3
|
-
"version": "1.6.
|
|
3
|
+
"version": "1.6.96",
|
|
4
4
|
"description": "Datarails CLI",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"engines": {
|
|
@@ -17,7 +17,7 @@
|
|
|
17
17
|
"access": "public"
|
|
18
18
|
},
|
|
19
19
|
"dependencies": {
|
|
20
|
-
"@datarails-ai/dr-appspace-cli": "0.10.
|
|
20
|
+
"@datarails-ai/dr-appspace-cli": "0.10.20",
|
|
21
21
|
"@modelcontextprotocol/sdk": "^1.29.0",
|
|
22
22
|
"@napi-rs/keyring": "^1.3.0",
|
|
23
23
|
"commander": "^12.1.0",
|