@foggy-projects/deepseek-harness-plugin 0.4.0-beta.14 → 0.4.0-beta.15
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +18 -14
- package/docs/PUBLIC-BETA-READINESS.md +25 -5
- package/docs/WINDOWS-BETA-ACCEPTANCE.md +17 -14
- package/experience/linux/prepare.sh +1 -1
- package/package.json +1 -1
- package/skills/foggy-deepseek-onboarding/SKILL.md +106 -127
- package/skills/foggy-deepseek-onboarding/assets/connection.schema.json +6 -8
- package/skills/foggy-deepseek-onboarding/assets/datasource.example.json +3 -2
- package/skills/foggy-deepseek-onboarding/assets/env.example +3 -3
- package/skills/foggy-deepseek-onboarding/assets/onboarding-state.schema.json +23 -1
- package/skills/foggy-deepseek-onboarding/assets/versions.json +1 -1
- package/skills/foggy-deepseek-onboarding/references/onboarding-workflow.md +113 -123
- package/skills/foggy-deepseek-onboarding/scripts/onboarding.py +131 -20
package/README.md
CHANGED
|
@@ -14,7 +14,7 @@ modify `PATH`, or register Python globally. Advanced users may explicitly set
|
|
|
14
14
|
## Local beta installation
|
|
15
15
|
|
|
16
16
|
```powershell
|
|
17
|
-
dsh plugin --profile web add --workspace-root ./foggy-projects-deepseek-harness-plugin-0.4.0-beta.
|
|
17
|
+
dsh plugin --profile web add --workspace-root ./foggy-projects-deepseek-harness-plugin-0.4.0-beta.15.tgz
|
|
18
18
|
```
|
|
19
19
|
|
|
20
20
|
Restart `dsh web`, use the browser it opens (or the complete printed URL,
|
|
@@ -30,9 +30,13 @@ dsh plugin --profile web add --workspace-root @foggy-projects/deepseek-harness-p
|
|
|
30
30
|
For development, `FOGGY_ASSET_CACHE_DIRS` can contain platform-delimited verified
|
|
31
31
|
asset-cache directories. It is not required for a normal install.
|
|
32
32
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
33
|
+
The bundled onboarding Skill is intentionally development-first. Users may
|
|
34
|
+
provide a local datasource password in the conversation, a user-managed
|
|
35
|
+
connection file, an Agent environment variable, or a compatible Runtime
|
|
36
|
+
Console. Direct passwords are submitted to the already-running Runtime without
|
|
37
|
+
being copied into onboarding state or evidence. The Skill then drives schema
|
|
38
|
+
discovery, semantic drafting, local Bundle registration, and bounded query
|
|
39
|
+
verification without restarting Runtime.
|
|
36
40
|
|
|
37
41
|
## Native Skill and workspace contract
|
|
38
42
|
|
|
@@ -42,14 +46,12 @@ are available in every DSH workspace without copying or symlinking `.agents`.
|
|
|
42
46
|
The current session `cwd` remains the workspace boundary for semantic drafts and
|
|
43
47
|
evidence.
|
|
44
48
|
|
|
45
|
-
Opaque CLI profiles
|
|
46
|
-
`<dataRoot>/cli-profiles` directory
|
|
49
|
+
Opaque CLI profiles remain available for users who prefer them and default to
|
|
50
|
+
the private persistent `<dataRoot>/cli-profiles` directory, but they are not a
|
|
51
|
+
prerequisite for ordinary development. Composite onboarding commands are
|
|
47
52
|
idempotent: unchanged completed phases are resumed rather than re-adding a
|
|
48
|
-
datasource or re-registering a
|
|
49
|
-
|
|
50
|
-
persistent store. A completed profile can also be bound non-destructively to an
|
|
51
|
-
additional DSH workspace when its reviewed connection contract and published
|
|
52
|
-
semantic digest are unchanged.
|
|
53
|
+
datasource or re-registering a local Bundle. Settings detects legacy temporary
|
|
54
|
+
profiles and offers an explicit, validated migration into the persistent store.
|
|
53
55
|
|
|
54
56
|
Private Python, CLI, Launcher, the analysis Skill, install state, and Runtime
|
|
55
57
|
state live in the user-level Foggy component directories. The managed CLI is intentionally isolated
|
|
@@ -73,9 +75,11 @@ same wildcard binding used by the Java server, so a conflicting application or W
|
|
|
73
75
|
port proxy produces an immediate, high-visibility error instead of a readiness timeout.
|
|
74
76
|
The CLI and Skills resolve the resulting stable Runtime URL from managed state.
|
|
75
77
|
|
|
76
|
-
This beta remains a local dev/test integration.
|
|
77
|
-
|
|
78
|
-
a
|
|
78
|
+
This beta remains a local dev/test integration. It does not automatically extend
|
|
79
|
+
local credentials or approvals into a formal environment. Production model
|
|
80
|
+
publication should use a separate manual or dedicated deployment workflow with
|
|
81
|
+
an explicit target, model Git commit/tag, credentials, verification, and
|
|
82
|
+
rollback plan.
|
|
79
83
|
|
|
80
84
|
See [`docs/PUBLIC-BETA-READINESS.md`](./docs/PUBLIC-BETA-READINESS.md) for the
|
|
81
85
|
tested public Beta scope, release gates, and stable-release blockers.
|
|
@@ -1,10 +1,10 @@
|
|
|
1
|
-
# Public beta readiness: 0.4.0-beta.
|
|
1
|
+
# Public beta readiness: 0.4.0-beta.15
|
|
2
2
|
|
|
3
3
|
Assessment date: 2026-09-04
|
|
4
4
|
|
|
5
5
|
## Verdict
|
|
6
6
|
|
|
7
|
-
`0.4.0-beta.
|
|
7
|
+
`0.4.0-beta.15` is suitable for a scoped public beta on Windows 10/11 x64 and
|
|
8
8
|
Linux/WSL2 x64. It is not a general-availability release.
|
|
9
9
|
|
|
10
10
|
The plugin package stays small and downloads large components only during
|
|
@@ -26,8 +26,22 @@ system PATH, registry, or a user's existing Python installation.
|
|
|
26
26
|
- WSL2 reported the existing managed Python, CLI, Launcher, and analysis Skill
|
|
27
27
|
as ready; an isolated Windows profile correctly reported its fresh component
|
|
28
28
|
state and rejected the host's Java 12 as below the Java 17 prerequisite.
|
|
29
|
+
- Onboarding is development-first: a direct password or Agent environment
|
|
30
|
+
variable is submitted to the public datasource API of an already-running
|
|
31
|
+
Runtime. The password is excluded from onboarding state and evidence, and
|
|
32
|
+
datasource setup does not trigger Runtime restart. Opaque profiles remain an
|
|
33
|
+
optional advanced path.
|
|
34
|
+
- A real Launcher 0.1.18 smoke on Java 17 accepted a synthetic inline-password
|
|
35
|
+
datasource through the public Runtime API while retaining the same Java PID.
|
|
36
|
+
The isolated registry confirmed the documented local-development plaintext
|
|
37
|
+
persistence boundary; the test datasource was then removed and the temporary
|
|
38
|
+
Runtime stopped.
|
|
39
|
+
- A full isolated `onboard-datasource-run` then created, tested, bound, and
|
|
40
|
+
inspected a SQLite datasource containing one table. Runtime retained the same
|
|
41
|
+
PID, while the persisted onboarding state and command evidence contained no
|
|
42
|
+
copy of the synthetic password.
|
|
29
43
|
- The UI reported Python 3.12.13 as `Foggy private`, and CLI 0.1.23, Launcher
|
|
30
|
-
0.1.18, analysis Skill 0.1.17, and onboarding Skill 0.4.0-beta.
|
|
44
|
+
0.1.18, analysis Skill 0.1.17, and onboarding Skill 0.4.0-beta.15 as installed.
|
|
31
45
|
- Runtime startup now reports the current phase, elapsed/timeout seconds, and a
|
|
32
46
|
bounded progress value. A real Windows launch with Temurin 17 reached readiness
|
|
33
47
|
and passed capabilities in 18.9 seconds. A controlled post-launch Java exit was
|
|
@@ -47,8 +61,10 @@ system PATH, registry, or a user's existing Python installation.
|
|
|
47
61
|
downloaded artifacts are not trusted until the complete SHA256 matches.
|
|
48
62
|
- Windows progress writes tolerate transient sharing violations without losing
|
|
49
63
|
the previous valid progress document or leaving temporary files behind.
|
|
50
|
-
-
|
|
51
|
-
|
|
64
|
+
- Current automated regression passes 20 Node tests and 20 Python tests. The
|
|
65
|
+
new datasource tests cover secret-free persisted state, direct Runtime API
|
|
66
|
+
submission, CLI bypass for inline development credentials, and output
|
|
67
|
+
redaction.
|
|
52
68
|
|
|
53
69
|
## Beta boundaries
|
|
54
70
|
|
|
@@ -63,6 +79,10 @@ system PATH, registry, or a user's existing Python installation.
|
|
|
63
79
|
- Runtime production authentication is outside this beta's default local
|
|
64
80
|
onboarding mode. The UI and doctor continue to report `productionReady=false`
|
|
65
81
|
for the development-only no-auth mode.
|
|
82
|
+
- Direct development passwords are stored by the current Runtime in its private
|
|
83
|
+
local datasource registry. Production publication is a separate manual or
|
|
84
|
+
future dedicated deployment workflow and should use the target environment's
|
|
85
|
+
credential controls.
|
|
66
86
|
|
|
67
87
|
## Release gate after publication
|
|
68
88
|
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Native Windows Beta acceptance
|
|
2
2
|
|
|
3
3
|
This runbook validates DeepSeek Harness `0.1.2-rc.1` with Foggy plugin
|
|
4
|
-
`0.4.0-beta.
|
|
4
|
+
`0.4.0-beta.15` on a clean 64-bit Windows 10 or Windows 11 machine. Use a local
|
|
5
5
|
directory that is not synchronized by OneDrive.
|
|
6
6
|
|
|
7
7
|
## Prerequisites
|
|
@@ -54,7 +54,7 @@ Expected component state:
|
|
|
54
54
|
- CLI 0.1.23;
|
|
55
55
|
- Launcher 0.1.18;
|
|
56
56
|
- analysis Skill 0.1.17;
|
|
57
|
-
- onboarding Skill 0.4.0-beta.
|
|
57
|
+
- onboarding Skill 0.4.0-beta.15;
|
|
58
58
|
- Java 17+ available;
|
|
59
59
|
- native DSH Skill registration available.
|
|
60
60
|
|
|
@@ -74,21 +74,24 @@ starting again must clear the applicable conflict state.
|
|
|
74
74
|
|
|
75
75
|
## Database and semantic-layer acceptance
|
|
76
76
|
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
77
|
+
Use a development or test database. For this local acceptance run, connection
|
|
78
|
+
details and a password may be supplied directly in the prompt or in a local
|
|
79
|
+
connection JSON file. The onboarding wrapper must configure the datasource
|
|
80
|
+
online without stopping or restarting an already-running Runtime. It must not
|
|
81
|
+
copy the password into onboarding state, evidence, TM/QM files, or Git.
|
|
81
82
|
|
|
82
83
|
Use a natural prompt such as:
|
|
83
84
|
|
|
84
85
|
```text
|
|
85
|
-
请帮我用 Foggy
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
86
|
+
请帮我用 Foggy 分析这个测试数据库:地址、数据库名、账号和密码分别是
|
|
87
|
+
<连接信息>。直接创建开发数据源并绑定一个合适的 namespace,不要重启
|
|
88
|
+
Runtime。读取表结构后,在当前工作目录创建订单语义模型,包含订单数量、
|
|
89
|
+
状态和开单时间,调试到模型通过,并执行一个不超过 20 行的只读查询。
|
|
89
90
|
```
|
|
90
91
|
|
|
91
|
-
Pass only when Runtime readiness and capabilities succeed, datasource
|
|
92
|
-
can be discovered, TM/QM models validate and
|
|
93
|
-
|
|
94
|
-
|
|
92
|
+
Pass only when Runtime readiness and capabilities succeed, datasource setup
|
|
93
|
+
does not restart Runtime, metadata can be discovered, TM/QM models validate and
|
|
94
|
+
become effective in the local Runtime, the bounded read-only query succeeds,
|
|
95
|
+
diagnostics contain no credentials, and stopping Runtime frees the configured
|
|
96
|
+
port (default `18166`). Recommend Git management for the completed model files;
|
|
97
|
+
do not treat this acceptance run as production publication.
|
package/package.json
CHANGED
|
@@ -1,137 +1,116 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: foggy-deepseek-onboarding
|
|
3
|
-
description:
|
|
3
|
+
description: Set up and use Foggy Runtime for data-source exploration and semantic-model development inside DeepSeek Harness. Use for local experience, datasource onboarding, TM/QM authoring, validation, and development publication; route production deployment or updates to a separate release workflow.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Foggy DeepSeek onboarding
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
5. For a new business database, read [references/onboarding-workflow.md](references/onboarding-workflow.md)
|
|
89
|
-
and prefer its two composite `onboard-datasource-run` / `onboard-semantic-run` commands. Require the
|
|
90
|
-
trusted operator to create the private CLI profile outside Harness. Unless the operator explicitly
|
|
91
|
-
overrides it, use the persistent profile store reported by the wrapper under the Foggy data root
|
|
92
|
-
(`<dataRoot>/cli-profiles`), never `/tmp`. Accept only the opaque profile
|
|
93
|
-
ID, exact revision, datasource name/type, and namespace; never request JDBC URL, username,
|
|
94
|
-
password, or password environment-variable name in Harness.
|
|
95
|
-
If plugin settings report a legacy temporary profile, use the explicit migration action before
|
|
96
|
-
onboarding. It moves only validated connection metadata and environment-variable references; it
|
|
97
|
-
never copies a password value and leaves a recoverable private backup below the Foggy data root.
|
|
98
|
-
6. After schema discovery, use `foggy-ai-analysis` only to author TM/QM files in the standard project-local
|
|
99
|
-
draft directory. Register, validate, publish, and verify them through this Skill's wrapper using the deterministic commands in
|
|
100
|
-
[references/onboarding-workflow.md](references/onboarding-workflow.md). Do not publish, prune, replace
|
|
101
|
-
a bundle, or execute a business-data query without the matching explicit flag and user approval.
|
|
102
|
-
7. Stop only the Runtime PID recorded by this package. Preserve Runtime data unless the user explicitly
|
|
103
|
-
requests purge.
|
|
104
|
-
|
|
105
|
-
## Command contract
|
|
106
|
-
|
|
107
|
-
All package scripts return one JSON object on stdout. Treat `success=false` or a nonzero exit code as a
|
|
108
|
-
failure. Do not infer readiness from a fixed sleep; require CLI `wait-ready` and `capabilities`.
|
|
109
|
-
|
|
110
|
-
Use this analysis order after setup:
|
|
8
|
+
Use this Skill for the normal DeepSeek Harness experience: connect a development datasource, inspect
|
|
9
|
+
it, create TM/QM files in the current workspace, and iterate until queries pass. Use the managed
|
|
10
|
+
`foggy-runtime` CLI and this Skill's wrappers; do not configure Foggy MCP for this local workflow.
|
|
11
|
+
|
|
12
|
+
## Product boundary
|
|
13
|
+
|
|
14
|
+
- Treat the plugin and bundled Launcher as local experience and development tooling.
|
|
15
|
+
- Datasources, namespace bindings, model Bundles, refreshes, and query checks are online Runtime
|
|
16
|
+
operations. They do not require a Runtime restart.
|
|
17
|
+
- Start Runtime only when it is absent. Reuse a healthy running Runtime. Restart only when the user
|
|
18
|
+
asks, a Launcher/JAR upgrade requires it, Runtime is unhealthy, or an actual startup setting such as
|
|
19
|
+
port, JVM options, Runtime authentication, or an opt-in module changes.
|
|
20
|
+
- Do not turn production publication into a continuation of local onboarding. When the user wants to
|
|
21
|
+
deploy or update a formal environment, recommend a manual release or a dedicated production
|
|
22
|
+
deployment workflow with separately supplied target access, credentials, version, verification, and
|
|
23
|
+
rollback information.
|
|
24
|
+
- Do not modify Foggy engine or CLI source. If progress genuinely requires either change, stop and ask
|
|
25
|
+
for explicit authorization.
|
|
26
|
+
|
|
27
|
+
## Managed installation
|
|
28
|
+
|
|
29
|
+
This Skill is registered through DeepSeek Harness's native Skill registry. Its scripts and references
|
|
30
|
+
are authoritative; do not copy the Skill into the current workspace.
|
|
31
|
+
|
|
32
|
+
- Linux install state: `${XDG_DATA_HOME:-$HOME/.local/share}/foggy/deepseek-harness/install-state.json`.
|
|
33
|
+
- Windows install state: `%LOCALAPPDATA%\Foggy\DeepSeekHarness\install-state.json`.
|
|
34
|
+
- Run this Skill's `doctor` wrapper with the current DSH workspace as `--project-root`. The managed CLI
|
|
35
|
+
is intentionally isolated, so absence from `PATH` does not mean it is missing.
|
|
36
|
+
- Use plugin settings to initialize or repair private Python, CLI, Launcher, or the managed analysis
|
|
37
|
+
Skill. Do not independently reinstall them during a normal analysis session.
|
|
38
|
+
- Treat the current DSH workspace as `projectRoot`. Keep model drafts and final model files there rather
|
|
39
|
+
than redirecting them to an example repository.
|
|
40
|
+
|
|
41
|
+
## Development workflow
|
|
42
|
+
|
|
43
|
+
1. Run `doctor`. Start Runtime only if it is not already healthy; require `wait-ready` and
|
|
44
|
+
`capabilities` after a new start.
|
|
45
|
+
2. Accept datasource connection details from the user's message, a user-supplied local JSON file, an
|
|
46
|
+
environment variable, or Runtime Console. Direct `password` is supported for local development.
|
|
47
|
+
The wrapper submits it to the public Runtime API without copying it into onboarding state or
|
|
48
|
+
evidence. Do not echo it in the response.
|
|
49
|
+
3. Create or select the namespace, add/test the datasource online, and bind it to the namespace. Do
|
|
50
|
+
not stop or restart Runtime to make a password available. A running Runtime can accept a direct
|
|
51
|
+
development password through its datasource API.
|
|
52
|
+
4. Inspect tables, columns, keys, and relationships. Small, bounded, read-only SQL samples are allowed
|
|
53
|
+
when they help infer captions, enums, units, or date semantics. Do not run mutations unless the user
|
|
54
|
+
explicitly requests them.
|
|
55
|
+
5. Load `foggy-ai-analysis` from the native registry only for TM/QM authoring and query tuning; its
|
|
56
|
+
generic installation, datasource-secret, and production-deployment guidance does not override this
|
|
57
|
+
development workflow. Default to a project-local `models/` directory unless the user specifies
|
|
58
|
+
another directory.
|
|
59
|
+
6. Iterate through model validation, Bundle registration/update, model refresh/describe, query
|
|
60
|
+
validation, and bounded query execution. Development publication means making the local model
|
|
61
|
+
directory effective in this local Runtime; it is not a production release.
|
|
62
|
+
7. When the model works, recommend committing the model directory to the user's own Git repository.
|
|
63
|
+
Do not initialize a repository, commit, push, or create a remote unless the user requests it.
|
|
64
|
+
|
|
65
|
+
Read [references/onboarding-workflow.md](references/onboarding-workflow.md) for the wrapper commands and
|
|
66
|
+
credential shapes. Use `foggy-ai-analysis` references for detailed TM/QM modeling and query tuning.
|
|
67
|
+
|
|
68
|
+
## Credential choices for local development
|
|
69
|
+
|
|
70
|
+
Choose the simplest source the user provides:
|
|
71
|
+
|
|
72
|
+
- `password` in a connection file: simplest for an experience session; the source file remains under
|
|
73
|
+
the user's control and should normally stay outside Git.
|
|
74
|
+
- `passwordEnv`: the wrapper reads the variable from the Agent process and submits the value online;
|
|
75
|
+
Runtime does not need to inherit it at startup.
|
|
76
|
+
- Opaque profile: optional for users who already have one; never require it for ordinary onboarding.
|
|
77
|
+
- Runtime Console: when a Launcher exposing Runtime Console is installed, the user may enter the
|
|
78
|
+
connection there using the management token shown by the host/plugin.
|
|
79
|
+
|
|
80
|
+
Keep only a minimal development safety baseline: do not echo passwords, put them in TM/QM files,
|
|
81
|
+
include them in evidence/diagnostics, or commit them to Git. Do not impose production IAM, audit,
|
|
82
|
+
approval, secret-store, or network-governance requirements on this local flow.
|
|
83
|
+
|
|
84
|
+
## Command behavior
|
|
85
|
+
|
|
86
|
+
All scripts emit one JSON object. Treat `success=false` or a nonzero exit code as failure. Prefer the
|
|
87
|
+
two resumable composite commands for a complete requested experience:
|
|
111
88
|
|
|
112
89
|
```text
|
|
113
|
-
datasource
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
90
|
+
onboard-datasource-run --project-root <workspace> --connection-file <json> \
|
|
91
|
+
--approve-configure --approve-bind --include-indexes
|
|
92
|
+
|
|
93
|
+
onboard-semantic-run --project-root <workspace> --semantic-plan <json> \
|
|
94
|
+
--query-payload <json> --approve-validate --approve-publish --approve-execute
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
When the user has already asked to connect, build, and test a new local model, those flags implement
|
|
98
|
+
that request and do not require separate question-by-question confirmation. Replacement, pruning, broad
|
|
99
|
+
queries, destructive SQL, Git push, and production deployment still require their own clear scope.
|
|
100
|
+
|
|
101
|
+
The normal analysis order is:
|
|
102
|
+
|
|
103
|
+
```text
|
|
104
|
+
datasource add/test -> namespace bind -> table/schema inspection
|
|
105
|
+
TM/QM authoring -> models validate -> bundle add/update -> refresh/describe
|
|
106
|
+
query validate -> bounded query execute -> tune -> optional Git handoff
|
|
117
107
|
```
|
|
118
108
|
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
query
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
An already-completed profile may be reused from another DSH workspace when the approved connection
|
|
129
|
-
contract is byte-for-byte equivalent. Run the datasource composite in the new workspace first; it adds
|
|
130
|
-
that workspace as a non-destructive binding and reuses datasource/schema checkpoints. Then pass the
|
|
131
|
-
current workspace explicitly as `--project-root` to the semantic composite. A secondary workspace may
|
|
132
|
-
reuse an identical published semantic digest and run its own bounded verification query, but it cannot
|
|
133
|
-
replace the published semantic layer; changes must be published from the original workspace or a new
|
|
134
|
-
profile.
|
|
135
|
-
|
|
136
|
-
Keep user business data separate from the sales-drop SQLite demo. Prefer a read-only database account,
|
|
137
|
-
opaque CLI profile references, and bounded query limits.
|
|
109
|
+
Stop only the PID recorded by this package and preserve the Runtime work directory unless the user
|
|
110
|
+
explicitly requests removal.
|
|
111
|
+
|
|
112
|
+
## Result
|
|
113
|
+
|
|
114
|
+
Report the Runtime URL, namespace, datasource name, model directory, Bundle and QueryModel names,
|
|
115
|
+
validation/query status, and useful evidence paths. Never include the password or business row values
|
|
116
|
+
unless the user specifically asks for those values.
|
|
@@ -12,6 +12,7 @@
|
|
|
12
12
|
"type": {"enum": ["sqlite", "mysql", "postgres", "postgresql"]},
|
|
13
13
|
"jdbcUrl": {"type": "string", "minLength": 1},
|
|
14
14
|
"username": {"type": "string"},
|
|
15
|
+
"password": {"type": "string", "description": "Optional local development password. It is submitted once and is not copied into onboarding state or evidence."},
|
|
15
16
|
"passwordEnv": {"type": "string", "pattern": "^[A-Za-z_][A-Za-z0-9_]*$"},
|
|
16
17
|
"opaqueProfileId": {"type": "string", "pattern": "^fop_[a-f0-9]{32}$"},
|
|
17
18
|
"opaqueRevision": {"type": "string", "pattern": "^sha256:[a-f0-9]{64}$"},
|
|
@@ -27,6 +28,7 @@
|
|
|
27
28
|
"not": {"anyOf": [
|
|
28
29
|
{"required": ["jdbcUrl"]},
|
|
29
30
|
{"required": ["username"]},
|
|
31
|
+
{"required": ["password"]},
|
|
30
32
|
{"required": ["passwordEnv"]}
|
|
31
33
|
]}
|
|
32
34
|
},
|
|
@@ -38,13 +40,9 @@
|
|
|
38
40
|
]}
|
|
39
41
|
}
|
|
40
42
|
],
|
|
41
|
-
"allOf": [
|
|
42
|
-
{
|
|
43
|
-
"
|
|
44
|
-
"required": ["jdbcUrl"],
|
|
45
|
-
"properties": {"type": {"not": {"const": "sqlite"}}}
|
|
46
|
-
},
|
|
47
|
-
"then": {"required": ["passwordEnv"]}
|
|
43
|
+
"allOf": [{
|
|
44
|
+
"not": {
|
|
45
|
+
"required": ["password", "passwordEnv"]
|
|
48
46
|
}
|
|
49
|
-
]
|
|
47
|
+
}]
|
|
50
48
|
}
|
|
@@ -3,8 +3,9 @@
|
|
|
3
3
|
"profile": "business",
|
|
4
4
|
"name": "business-db",
|
|
5
5
|
"type": "mysql",
|
|
6
|
-
"
|
|
7
|
-
"
|
|
6
|
+
"jdbcUrl": "jdbc:mysql://127.0.0.1:3306/business",
|
|
7
|
+
"username": "business_dev",
|
|
8
|
+
"password": "replace-for-local-development",
|
|
8
9
|
"namespace": "business",
|
|
9
10
|
"schemas": ["public"],
|
|
10
11
|
"modelsDir": "models",
|
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
#
|
|
2
|
-
#
|
|
1
|
+
# Optional development configuration. Direct passwords in a user-supplied connection file are also
|
|
2
|
+
# supported; the onboarding wrapper does not copy them into state or evidence.
|
|
3
3
|
|
|
4
4
|
FOGGY_RUNTIME_API_URL=http://127.0.0.1:18166
|
|
5
5
|
FOGGY_NAMESPACE=default
|
|
@@ -10,5 +10,5 @@ FOGGY_NAMESPACE=default
|
|
|
10
10
|
# Optional data-plane authorization value.
|
|
11
11
|
# FOGGY_RUNTIME_AUTHORIZATION=
|
|
12
12
|
|
|
13
|
-
#
|
|
13
|
+
# Optional alternative to a direct development password.
|
|
14
14
|
# FOGGY_DATASOURCE_PASSWORD=
|
|
@@ -34,7 +34,29 @@
|
|
|
34
34
|
}
|
|
35
35
|
},
|
|
36
36
|
"runtime": {"type": "object"},
|
|
37
|
-
"connection": {
|
|
37
|
+
"connection": {
|
|
38
|
+
"type": "object",
|
|
39
|
+
"additionalProperties": false,
|
|
40
|
+
"required": ["schemaVersion", "connectionMode", "name", "type", "namespace"],
|
|
41
|
+
"properties": {
|
|
42
|
+
"schemaVersion": {"const": "foggy-deepseek-connection/v1"},
|
|
43
|
+
"connectionMode": {"enum": ["legacy-inline", "opaque-profile"]},
|
|
44
|
+
"credentialMode": {"enum": ["inline-development", "agent-environment", "opaque-profile", "none"]},
|
|
45
|
+
"name": {"type": "string"},
|
|
46
|
+
"type": {"type": "string"},
|
|
47
|
+
"jdbcUrl": {"type": ["string", "null"]},
|
|
48
|
+
"username": {"type": ["string", "null"]},
|
|
49
|
+
"passwordEnv": {"type": ["string", "null"]},
|
|
50
|
+
"opaqueProfileId": {"type": "string"},
|
|
51
|
+
"opaqueRevision": {"type": "string"},
|
|
52
|
+
"namespace": {"type": "string"},
|
|
53
|
+
"schemas": {"type": "array", "items": {"type": "string"}},
|
|
54
|
+
"modelsDir": {"type": "string"},
|
|
55
|
+
"readOnlyRecommended": {"type": "boolean"},
|
|
56
|
+
"profile": {"type": "string"},
|
|
57
|
+
"evidenceDir": {"type": "string"}
|
|
58
|
+
}
|
|
59
|
+
},
|
|
38
60
|
"semantic": {"type": "object"},
|
|
39
61
|
"steps": {"type": "object"},
|
|
40
62
|
"artifacts": {"type": "object"}
|
|
@@ -1,156 +1,146 @@
|
|
|
1
|
-
#
|
|
2
|
-
|
|
3
|
-
Use the
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
1
|
+
# Development datasource and semantic-model workflow
|
|
2
|
+
|
|
3
|
+
Use this workflow after the Foggy plugin is initialized. It is designed for local experience and
|
|
4
|
+
model development in DeepSeek Harness, not for production deployment.
|
|
5
|
+
|
|
6
|
+
## Connection input
|
|
7
|
+
|
|
8
|
+
The simplest local connection contract may contain a direct password:
|
|
9
|
+
|
|
10
|
+
```json
|
|
11
|
+
{
|
|
12
|
+
"schemaVersion": "foggy-deepseek-connection/v1",
|
|
13
|
+
"profile": "business",
|
|
14
|
+
"name": "business-db",
|
|
15
|
+
"type": "mysql",
|
|
16
|
+
"jdbcUrl": "jdbc:mysql://127.0.0.1:3306/business",
|
|
17
|
+
"username": "business_dev",
|
|
18
|
+
"password": "local-development-password",
|
|
19
|
+
"namespace": "business",
|
|
20
|
+
"modelsDir": "models",
|
|
21
|
+
"evidenceDir": ".foggy/onboarding-command-evidence/business"
|
|
22
|
+
}
|
|
23
|
+
```
|
|
12
24
|
|
|
13
|
-
|
|
25
|
+
The wrapper uses the direct password only while submitting the datasource to the public Runtime API.
|
|
26
|
+
It does not copy it into onboarding state, command evidence, TM/QM files, or conversational output.
|
|
27
|
+
The source connection file is user-managed; do not commit it. The local Runtime currently stores a
|
|
28
|
+
direct password in its private datasource registry, so use `passwordRef`-style production credential
|
|
29
|
+
management when moving beyond local development.
|
|
30
|
+
|
|
31
|
+
An environment variable is an optional alternative:
|
|
32
|
+
|
|
33
|
+
```json
|
|
34
|
+
{
|
|
35
|
+
"schemaVersion": "foggy-deepseek-connection/v1",
|
|
36
|
+
"profile": "business",
|
|
37
|
+
"name": "business-db",
|
|
38
|
+
"type": "mysql",
|
|
39
|
+
"jdbcUrl": "jdbc:mysql://127.0.0.1:3306/business",
|
|
40
|
+
"username": "business_dev",
|
|
41
|
+
"passwordEnv": "FOGGY_BUSINESS_DB_PASSWORD",
|
|
42
|
+
"namespace": "business",
|
|
43
|
+
"modelsDir": "models"
|
|
44
|
+
}
|
|
45
|
+
```
|
|
14
46
|
|
|
15
|
-
|
|
47
|
+
Here `passwordEnv` is read by the onboarding wrapper and submitted online to the already-running
|
|
48
|
+
Runtime. Do not restart Runtime merely to make the variable part of the Java process environment.
|
|
49
|
+
Opaque CLI profiles remain supported for users who already prefer them, but are not required for a
|
|
50
|
+
normal DSH experience.
|
|
16
51
|
|
|
17
|
-
|
|
18
|
-
2. datasource name, database type, and namespace from the CLI's public profile summary;
|
|
19
|
-
3. optional schemas, the project-relative semantic model directory, and a project-relative evidence directory.
|
|
52
|
+
## Normal sequence
|
|
20
53
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
line, logs, or evidence. For non-SQLite databases, ask the user to set the named environment variable
|
|
24
|
-
before Runtime starts so the Java process inherits it. Recommend a read-only database account.
|
|
54
|
+
Run the platform-specific wrapper from this Skill directory. `onboard-datasource-run` performs the
|
|
55
|
+
online datasource setup, namespace binding, and schema discovery against the healthy Runtime:
|
|
25
56
|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
57
|
+
```text
|
|
58
|
+
onboard-datasource-run --project-root <current-workspace> \
|
|
59
|
+
--connection-file <connection-json> \
|
|
60
|
+
--approve-configure --approve-bind --include-indexes
|
|
61
|
+
```
|
|
31
62
|
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
63
|
+
If the user already requested the complete connection experience, include both approval flags in the
|
|
64
|
+
first call. They refer only to adding the named development datasource and binding the requested
|
|
65
|
+
namespace. Do not add `--replace` unless replacement was explicitly requested.
|
|
35
66
|
|
|
36
|
-
|
|
37
|
-
Migration validates the opaque profile schema, rejects embedded passwords and conflicts, writes the
|
|
38
|
-
destination with private permissions, verifies it, and moves the legacy JSON to a recoverable private
|
|
39
|
-
backup below the Foggy data root.
|
|
40
|
-
Do not manually copy or edit opaque profile JSON.
|
|
67
|
+
The command performs:
|
|
41
68
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
69
|
+
```text
|
|
70
|
+
datasource add -> datasource test -> namespace bind -> diagnostics
|
|
71
|
+
tables list -> bounded schema inspection
|
|
72
|
+
```
|
|
46
73
|
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
normalized contracts to the user before accepting mutation approvals; do not combine contract drafting
|
|
50
|
-
and approval into an unseen one-turn mutation.
|
|
74
|
+
It does not stop or restart Runtime. Repeated calls reuse completed checkpoints when the public
|
|
75
|
+
connection contract is unchanged. Inline password values are excluded from the comparison.
|
|
51
76
|
|
|
52
|
-
|
|
53
|
-
|
|
77
|
+
After discovery, use `foggy-ai-analysis` to create TM/QM drafts from actual tables, columns, keys,
|
|
78
|
+
small read-only samples when useful, and the user's business questions. Keep the files in the current
|
|
79
|
+
workspace, normally:
|
|
54
80
|
|
|
55
|
-
|
|
81
|
+
```text
|
|
82
|
+
models/
|
|
83
|
+
model/
|
|
84
|
+
<Name>.tm
|
|
85
|
+
query/
|
|
86
|
+
<Name>QueryModel.qm
|
|
87
|
+
```
|
|
56
88
|
|
|
57
|
-
|
|
58
|
-
evidence internally and refuses to cross an unapproved mutation gate:
|
|
89
|
+
Then run the semantic composite:
|
|
59
90
|
|
|
60
91
|
```text
|
|
61
|
-
onboard-
|
|
62
|
-
--
|
|
63
|
-
--
|
|
64
|
-
|
|
65
|
-
# After authoring the registered TM/QM draft from schema metadata:
|
|
66
|
-
onboard-semantic-run --project-root <current-session-workspace> \
|
|
67
|
-
--semantic-plan <approved-json> --query-payload <approved-json> \
|
|
92
|
+
onboard-semantic-run --project-root <current-workspace> \
|
|
93
|
+
--semantic-plan <semantic-json> \
|
|
94
|
+
--query-payload <bounded-query-json> \
|
|
68
95
|
--approve-validate --approve-publish --approve-execute
|
|
69
96
|
```
|
|
70
97
|
|
|
71
|
-
|
|
72
|
-
|
|
98
|
+
For an explicitly requested end-to-end local experience, these flags may be used together. The command
|
|
99
|
+
validates the model files, copies them to the approved model directory when needed, registers or
|
|
100
|
+
updates the local Runtime Bundle, refreshes/describes the declared QueryModels, validates the bounded
|
|
101
|
+
query, and executes it. It never means production publication.
|
|
73
102
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
registering the same bundle twice. If query validation fails after publication, correct the project-local
|
|
77
|
-
payload and rerun `onboard-semantic-run`; it resumes at query verification and leaves the active bundle in
|
|
78
|
-
place. A same-name datasource is accepted idempotently only when its public name and database type match
|
|
79
|
-
the approved plan; otherwise replacement still requires explicit approval.
|
|
103
|
+
Use `--watch` only when the user wants the local Runtime to follow model-file edits. Use `--prune` or
|
|
104
|
+
bundle replacement only when the user clearly asks for those changes.
|
|
80
105
|
|
|
81
|
-
|
|
82
|
-
current workspace as a binding only if the approved connection contract is identical and the datasource,
|
|
83
|
-
schema, and semantic publication checkpoints are complete. The semantic composite then accepts the same
|
|
84
|
-
published draft digest from the bound workspace and performs a workspace-specific bounded verification
|
|
85
|
-
query. It refuses semantic replacement from the secondary workspace.
|
|
106
|
+
## Iteration and troubleshooting
|
|
86
107
|
|
|
87
|
-
The granular commands
|
|
88
|
-
the composite commands into this list during a normal Harness turn.
|
|
108
|
+
The granular commands remain available when a specific stage needs repair:
|
|
89
109
|
|
|
90
110
|
```text
|
|
91
111
|
onboard-plan --connection-file <json> --profile <profile>
|
|
92
|
-
datasource-configure --profile <profile>
|
|
93
|
-
datasource-
|
|
94
|
-
|
|
95
|
-
datasource-verify --profile <profile> --bind
|
|
96
|
-
schema-discover --profile <profile> --include-indexes
|
|
112
|
+
datasource-configure --profile <profile> [--apply]
|
|
113
|
+
datasource-verify --profile <profile> [--bind]
|
|
114
|
+
schema-discover --profile <profile> [--include-indexes]
|
|
97
115
|
semantic-draft --profile <profile> --semantic-plan <json>
|
|
98
|
-
semantic-validate --profile <profile>
|
|
99
|
-
semantic-
|
|
100
|
-
semantic-
|
|
101
|
-
semantic-publish --profile <profile> --apply
|
|
102
|
-
semantic-verify --profile <profile> --project-root <current-session-workspace> --query-payload <json>
|
|
103
|
-
semantic-verify --profile <profile> --project-root <current-session-workspace> --query-payload <json> --execute
|
|
116
|
+
semantic-validate --profile <profile> [--apply]
|
|
117
|
+
semantic-publish --profile <profile> [--apply]
|
|
118
|
+
semantic-verify --profile <profile> --query-payload <json> [--execute]
|
|
104
119
|
onboard-status --profile <profile>
|
|
120
|
+
onboard-resume --profile <profile>
|
|
105
121
|
```
|
|
106
122
|
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
explicitly approves replacing an existing datasource definition.
|
|
111
|
-
|
|
112
|
-
Schema discovery is metadata-only and inspects at most 25 tables by default. Use repeated `--schema` or
|
|
113
|
-
`--table` arguments to narrow scope, `--list-only` for inventory only, and raise `--max-tables` only when
|
|
114
|
-
the user needs broader inspection. The resulting evidence remains under the private Runtime data root.
|
|
115
|
-
|
|
116
|
-
## Semantic authoring and publishing
|
|
117
|
-
|
|
118
|
-
After `schemaDiscovered=completed`, review table names, columns, keys, and relationships with the user.
|
|
119
|
-
Use `foggy-ai-analysis` for TM/QM authoring in a separate project-local draft directory. Do not invent
|
|
120
|
-
business definitions, joins, units, enum meanings, or date semantics. Create a plan using
|
|
121
|
-
`assets/semantic-plan.schema.json`; its declared query-model names must exist in the draft `.qm` files.
|
|
122
|
-
|
|
123
|
-
`semantic-draft` records TM/QM hashes. Any later file change invalidates the recorded validation and must
|
|
124
|
-
be registered and validated again. `semantic-validate` is a dry run until `--apply`; applying it may
|
|
125
|
-
replace the Runtime validation catalog, so explain that mutation before approval.
|
|
126
|
-
|
|
127
|
-
`semantic-publish` shows added, updated, unchanged, preserved, and optionally removed files. Applying it:
|
|
123
|
+
A standalone `datasource-configure --apply` can resolve `passwordEnv` from its current Agent process.
|
|
124
|
+
For a direct `password`, use the composite datasource command with the original connection file so the
|
|
125
|
+
secret stays ephemeral to that invocation.
|
|
128
126
|
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
4. registers the bundle;
|
|
133
|
-
5. refreshes only the declared query models.
|
|
127
|
+
When a failure occurs, repair only the failed input and rerun the same composite command. Do not remove
|
|
128
|
+
a successfully registered Bundle merely because later query validation failed. Keep SQL samples small,
|
|
129
|
+
read-only, and relevant to semantic authoring.
|
|
134
130
|
|
|
135
|
-
|
|
136
|
-
when the user approves replacing an existing Runtime bundle, and `--watch` only when file watching is
|
|
137
|
-
desired. A failure before refresh restores project files; a refresh failure is reported as a partial
|
|
138
|
-
Runtime publication and must be diagnosed before retrying.
|
|
131
|
+
## Git handoff
|
|
139
132
|
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
133
|
+
When validation and representative questions pass, recommend placing the model directory under the
|
|
134
|
+
user's existing Git repository. Git is the source of truth for model history; Runtime Bundle
|
|
135
|
+
registration only makes a selected local directory active. Do not create commits, remotes, tags, or
|
|
136
|
+
pushes without the user's request.
|
|
144
137
|
|
|
145
|
-
|
|
146
|
-
`projectRoot`, has a bounded limit, and targets a query model declared in the approved semantic plan.
|
|
147
|
-
After publication, `semantic-verify` always describes the live model before validation. If a field is
|
|
148
|
-
rejected, update the payload from those described names and rerun the same composite command.
|
|
138
|
+
## Production handoff
|
|
149
139
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
140
|
+
Stop the local onboarding flow when the user asks to publish or update a formal environment. A separate
|
|
141
|
+
manual or dedicated deployment workflow should collect the exact target, Launcher version, model Git
|
|
142
|
+
commit/tag, namespace, production datasource procedure, access credentials, verification plan, and
|
|
143
|
+
rollback point. It should then install/start Runtime if needed, prepare the target datasource, register
|
|
144
|
+
the versioned model directory, refresh, and run a narrow smoke check.
|
|
154
145
|
|
|
155
|
-
|
|
156
|
-
or environment, and run `onboard-resume` to continue from the first incomplete step.
|
|
146
|
+
Local onboarding credentials and approvals do not authorize production access or publication.
|
|
@@ -18,6 +18,7 @@ import subprocess
|
|
|
18
18
|
import sys
|
|
19
19
|
import tempfile
|
|
20
20
|
import time
|
|
21
|
+
import urllib.error
|
|
21
22
|
import urllib.request
|
|
22
23
|
import venv
|
|
23
24
|
import zipfile
|
|
@@ -918,7 +919,7 @@ def validate_connection(payload: dict) -> dict:
|
|
|
918
919
|
if payload.get("schemaVersion") != CONNECTION_SCHEMA:
|
|
919
920
|
raise OnboardingError(f"connection schemaVersion must be {CONNECTION_SCHEMA}")
|
|
920
921
|
allowed = {
|
|
921
|
-
"schemaVersion", "name", "type", "jdbcUrl", "username", "passwordEnv",
|
|
922
|
+
"schemaVersion", "name", "type", "jdbcUrl", "username", "password", "passwordEnv",
|
|
922
923
|
"opaqueProfileId", "opaqueRevision",
|
|
923
924
|
"profile", "namespace", "schemas", "modelsDir", "evidenceDir", "readOnlyRecommended",
|
|
924
925
|
}
|
|
@@ -930,6 +931,7 @@ def validate_connection(payload: dict) -> dict:
|
|
|
930
931
|
for name in required:
|
|
931
932
|
if not isinstance(payload.get(name), str) or not payload[name].strip():
|
|
932
933
|
raise OnboardingError(f"connection.{name} must be a non-empty string")
|
|
934
|
+
password = payload.get("password")
|
|
933
935
|
password_env = payload.get("passwordEnv")
|
|
934
936
|
jdbc_url = None
|
|
935
937
|
if opaque:
|
|
@@ -937,15 +939,19 @@ def validate_connection(payload: dict) -> dict:
|
|
|
937
939
|
raise OnboardingError("connection.opaqueProfileId must be an opaque Foggy profile ID")
|
|
938
940
|
if not isinstance(payload.get("opaqueRevision"), str) or not OPAQUE_REVISION_PATTERN.fullmatch(payload["opaqueRevision"]):
|
|
939
941
|
raise OnboardingError("connection.opaqueRevision must be a sha256 revision")
|
|
940
|
-
exposed = sorted(name for name in ("jdbcUrl", "username", "passwordEnv") if name in payload)
|
|
942
|
+
exposed = sorted(name for name in ("jdbcUrl", "username", "password", "passwordEnv") if name in payload)
|
|
941
943
|
if exposed:
|
|
942
944
|
raise OnboardingError(f"Opaque connection plans must not contain: {', '.join(exposed)}")
|
|
943
945
|
else:
|
|
946
|
+
if password is not None and not isinstance(password, str):
|
|
947
|
+
raise OnboardingError("connection.password must be a string")
|
|
944
948
|
if password_env is not None and (not isinstance(password_env, str) or not ENV_NAME_PATTERN.fullmatch(password_env)):
|
|
945
949
|
raise OnboardingError("connection.passwordEnv must be an environment variable name")
|
|
950
|
+
if password is not None and password_env is not None:
|
|
951
|
+
raise OnboardingError("Provide either connection.password or connection.passwordEnv, not both")
|
|
946
952
|
jdbc_url = payload["jdbcUrl"].strip()
|
|
947
953
|
if re.search(r"(?i)(?:password|passwd|pwd)\s*=", jdbc_url) or re.search(r"//[^/@:]+:[^/@]+@", jdbc_url):
|
|
948
|
-
raise OnboardingError("Do not embed passwords in jdbcUrl; use passwordEnv")
|
|
954
|
+
raise OnboardingError("Do not embed passwords in jdbcUrl; use password or passwordEnv")
|
|
949
955
|
schemas = payload.get("schemas", [])
|
|
950
956
|
if not isinstance(schemas, list) or any(not isinstance(item, str) or not item.strip() for item in schemas):
|
|
951
957
|
raise OnboardingError("connection.schemas must be an array of non-empty strings")
|
|
@@ -973,6 +979,7 @@ def validate_connection(payload: dict) -> dict:
|
|
|
973
979
|
else:
|
|
974
980
|
result["jdbcUrl"] = jdbc_url
|
|
975
981
|
result["username"] = payload.get("username")
|
|
982
|
+
result["password"] = password
|
|
976
983
|
result["passwordEnv"] = password_env
|
|
977
984
|
if payload.get("profile") is not None:
|
|
978
985
|
result["profile"] = safe_profile(payload["profile"])
|
|
@@ -980,11 +987,21 @@ def validate_connection(payload: dict) -> dict:
|
|
|
980
987
|
result["evidenceDir"] = payload["evidenceDir"].strip()
|
|
981
988
|
if result["type"] not in {"sqlite", "mysql", "postgres", "postgresql"}:
|
|
982
989
|
raise OnboardingError("Initial onboarding supports sqlite, mysql, postgres, and postgresql")
|
|
983
|
-
if not opaque and result["type"] != "sqlite" and not result["passwordEnv"]:
|
|
984
|
-
raise OnboardingError("Non-SQLite connections require passwordEnv")
|
|
985
990
|
return result
|
|
986
991
|
|
|
987
992
|
|
|
993
|
+
def persisted_connection(connection: dict) -> dict:
|
|
994
|
+
"""Return the resumable connection contract without an inline development password."""
|
|
995
|
+
persisted = {key: value for key, value in connection.items() if key != "password"}
|
|
996
|
+
persisted["credentialMode"] = (
|
|
997
|
+
"inline-development" if connection.get("password") is not None
|
|
998
|
+
else "agent-environment" if connection.get("passwordEnv")
|
|
999
|
+
else "opaque-profile" if connection.get("connectionMode") == "opaque-profile"
|
|
1000
|
+
else "none"
|
|
1001
|
+
)
|
|
1002
|
+
return persisted
|
|
1003
|
+
|
|
1004
|
+
|
|
988
1005
|
def validate_semantic_plan(payload: dict) -> dict:
|
|
989
1006
|
if payload.get("schemaVersion") != SEMANTIC_PLAN_SCHEMA:
|
|
990
1007
|
raise OnboardingError(f"semantic plan schemaVersion must be {SEMANTIC_PLAN_SCHEMA}")
|
|
@@ -1780,6 +1797,64 @@ def cli_json(install_state: dict, runtime_state: dict, namespace: str, command:
|
|
|
1780
1797
|
return payload
|
|
1781
1798
|
|
|
1782
1799
|
|
|
1800
|
+
def runtime_api_json(
|
|
1801
|
+
runtime_state: dict,
|
|
1802
|
+
namespace: str,
|
|
1803
|
+
method: str,
|
|
1804
|
+
path: str,
|
|
1805
|
+
body: dict | None,
|
|
1806
|
+
label: str,
|
|
1807
|
+
timeout: int = 60,
|
|
1808
|
+
) -> dict:
|
|
1809
|
+
"""Call the public Runtime API for development inputs not yet exposed by the pinned CLI."""
|
|
1810
|
+
base_url = str(runtime_state["runtimeUrl"]).rstrip("/")
|
|
1811
|
+
request_body = json.dumps(body).encode("utf-8") if body is not None else None
|
|
1812
|
+
headers = {
|
|
1813
|
+
"Accept": "application/json",
|
|
1814
|
+
"Content-Type": "application/json",
|
|
1815
|
+
"X-NS": namespace,
|
|
1816
|
+
}
|
|
1817
|
+
auth_code = os.environ.get("FOGGY_RUNTIME_API_AUTH_CODE")
|
|
1818
|
+
if auth_code:
|
|
1819
|
+
headers["X-Foggy-Runtime-Code"] = auth_code
|
|
1820
|
+
request = urllib.request.Request(
|
|
1821
|
+
f"{base_url}{path}",
|
|
1822
|
+
data=request_body,
|
|
1823
|
+
headers=headers,
|
|
1824
|
+
method=method,
|
|
1825
|
+
)
|
|
1826
|
+
try:
|
|
1827
|
+
with urllib.request.urlopen(request, timeout=timeout) as response:
|
|
1828
|
+
raw = response.read().decode("utf-8", errors="replace")
|
|
1829
|
+
except urllib.error.HTTPError as exc:
|
|
1830
|
+
raw = exc.read().decode("utf-8", errors="replace")
|
|
1831
|
+
try:
|
|
1832
|
+
failed = json.loads(raw)
|
|
1833
|
+
except json.JSONDecodeError:
|
|
1834
|
+
failed = None
|
|
1835
|
+
error = failed.get("error") if isinstance(failed, dict) else None
|
|
1836
|
+
detail = (
|
|
1837
|
+
" | ".join(str(error.get(name)) for name in ("code", "phase", "message") if error.get(name))
|
|
1838
|
+
if isinstance(error, dict)
|
|
1839
|
+
else str(exc.reason)
|
|
1840
|
+
)
|
|
1841
|
+
raise OnboardingError(f"{label} failed with HTTP {exc.code}: {detail or exc.reason}") from exc
|
|
1842
|
+
except urllib.error.URLError as exc:
|
|
1843
|
+
raise OnboardingError(f"{label} could not reach Runtime: {exc.reason}") from exc
|
|
1844
|
+
try:
|
|
1845
|
+
payload = json.loads(raw)
|
|
1846
|
+
except json.JSONDecodeError as exc:
|
|
1847
|
+
raise OnboardingError(f"{label} did not return JSON") from exc
|
|
1848
|
+
if payload.get("success") is not True:
|
|
1849
|
+
error = payload.get("error")
|
|
1850
|
+
if isinstance(error, dict):
|
|
1851
|
+
detail = " | ".join(str(error.get(name)) for name in ("code", "phase", "message") if error.get(name))
|
|
1852
|
+
else:
|
|
1853
|
+
detail = str(error or "unknown Runtime error")
|
|
1854
|
+
raise OnboardingError(f"{label} returned success=false: {detail}")
|
|
1855
|
+
return payload
|
|
1856
|
+
|
|
1857
|
+
|
|
1783
1858
|
CONNECTION_SECRET_KEYS = {
|
|
1784
1859
|
"jdbcurl", "url", "username", "password", "passwordenv", "passwordref",
|
|
1785
1860
|
}
|
|
@@ -1837,7 +1912,8 @@ def onboarding_plan_command(args: argparse.Namespace) -> dict:
|
|
|
1837
1912
|
if not project_root.is_dir():
|
|
1838
1913
|
raise OnboardingError(f"Project root not found: {project_root}")
|
|
1839
1914
|
connection_file = normalized(args.connection_file)
|
|
1840
|
-
|
|
1915
|
+
requested_connection = validate_connection(read_json_object(connection_file, "Connection plan"))
|
|
1916
|
+
connection = persisted_connection(requested_connection)
|
|
1841
1917
|
if connection.get("connectionMode") == "opaque-profile":
|
|
1842
1918
|
require_opaque_profile_cli(install_state)
|
|
1843
1919
|
existing = read_onboarding_state(data_root, profile, required=False)
|
|
@@ -1882,7 +1958,10 @@ def onboarding_plan_command(args: argparse.Namespace) -> dict:
|
|
|
1882
1958
|
"schemaVersion": "foggy-deepseek-onboarding-plan-result/v1",
|
|
1883
1959
|
"profile": profile,
|
|
1884
1960
|
"statePath": str(path),
|
|
1885
|
-
"connection": {
|
|
1961
|
+
"connection": {
|
|
1962
|
+
**connection,
|
|
1963
|
+
"passwordEnvPresent": bool(password_env and os.environ.get(password_env)),
|
|
1964
|
+
},
|
|
1886
1965
|
"runtimeAvailable": runtime_state is not None,
|
|
1887
1966
|
"next": "run datasource-configure --apply after reviewing the plan",
|
|
1888
1967
|
"productionReady": False,
|
|
@@ -1913,10 +1992,11 @@ def datasource_configure_command(args: argparse.Namespace) -> dict:
|
|
|
1913
1992
|
if opaque:
|
|
1914
1993
|
plan.update({"profileId": connection["opaqueProfileId"], "revision": connection["opaqueRevision"]})
|
|
1915
1994
|
else:
|
|
1995
|
+
inline_password = getattr(args, "runtime_password", None)
|
|
1916
1996
|
plan.update({
|
|
1917
1997
|
"jdbcUrl": connection["jdbcUrl"],
|
|
1918
1998
|
"username": connection.get("username"),
|
|
1919
|
-
"
|
|
1999
|
+
"credentialMode": connection.get("credentialMode", "none"),
|
|
1920
2000
|
})
|
|
1921
2001
|
if not args.apply:
|
|
1922
2002
|
return {"success": True, "dryRun": True, "profile": state["profile"], "plan": plan, "next": "rerun with --apply after approval"}
|
|
@@ -1928,22 +2008,50 @@ def datasource_configure_command(args: argparse.Namespace) -> dict:
|
|
|
1928
2008
|
]
|
|
1929
2009
|
label = "opaque profile configure"
|
|
1930
2010
|
else:
|
|
2011
|
+
inline_password = getattr(args, "runtime_password", None)
|
|
1931
2012
|
password_env = connection.get("passwordEnv")
|
|
1932
|
-
if
|
|
2013
|
+
if connection.get("credentialMode") == "inline-development" and inline_password is None:
|
|
2014
|
+
raise OnboardingError(
|
|
2015
|
+
"Direct development password is not persisted; rerun onboard-datasource-run with the original connection file"
|
|
2016
|
+
)
|
|
2017
|
+
if inline_password is None and password_env and os.environ.get(password_env) is None:
|
|
1933
2018
|
raise OnboardingError(f"Required password environment variable is not present: {password_env}")
|
|
1934
|
-
command = ["datasources", "add", "--name", connection["name"], "--type", connection["type"], "--jdbc-url", connection["jdbcUrl"]]
|
|
1935
|
-
if connection.get("username"):
|
|
1936
|
-
command.extend(["--username", connection["username"]])
|
|
1937
|
-
if password_env:
|
|
1938
|
-
command.extend(["--password-env", password_env])
|
|
1939
2019
|
label = "datasources add"
|
|
1940
|
-
|
|
1941
|
-
|
|
2020
|
+
resolved_password = inline_password if inline_password is not None else (
|
|
2021
|
+
os.environ.get(password_env) if password_env else None
|
|
2022
|
+
)
|
|
2023
|
+
command = None
|
|
1942
2024
|
already_present = False
|
|
1943
2025
|
try:
|
|
1944
|
-
|
|
1945
|
-
|
|
1946
|
-
|
|
2026
|
+
if not opaque and resolved_password is not None:
|
|
2027
|
+
body = {
|
|
2028
|
+
"name": connection["name"],
|
|
2029
|
+
"type": connection["type"],
|
|
2030
|
+
"jdbcUrl": connection["jdbcUrl"],
|
|
2031
|
+
"replace": bool(args.replace),
|
|
2032
|
+
"enabled": True,
|
|
2033
|
+
"password": resolved_password,
|
|
2034
|
+
}
|
|
2035
|
+
if connection.get("username"):
|
|
2036
|
+
body["username"] = connection["username"]
|
|
2037
|
+
result = redact_connection_material(runtime_api_json(
|
|
2038
|
+
runtime_state,
|
|
2039
|
+
connection["namespace"],
|
|
2040
|
+
"POST",
|
|
2041
|
+
"/api/v1/datasources",
|
|
2042
|
+
body,
|
|
2043
|
+
label,
|
|
2044
|
+
))
|
|
2045
|
+
else:
|
|
2046
|
+
if not opaque:
|
|
2047
|
+
command = ["datasources", "add", "--name", connection["name"], "--type", connection["type"], "--jdbc-url", connection["jdbcUrl"]]
|
|
2048
|
+
if connection.get("username"):
|
|
2049
|
+
command.extend(["--username", connection["username"]])
|
|
2050
|
+
if args.replace:
|
|
2051
|
+
command.append("--replace")
|
|
2052
|
+
result = redact_connection_material(
|
|
2053
|
+
cli_json(install_state, runtime_state, connection["namespace"], command, label)
|
|
2054
|
+
)
|
|
1947
2055
|
except OnboardingError as exc:
|
|
1948
2056
|
if "DATASOURCE_ALREADY_EXISTS" not in str(exc):
|
|
1949
2057
|
raise
|
|
@@ -2641,6 +2749,7 @@ def datasource_run_command(args: argparse.Namespace) -> dict:
|
|
|
2641
2749
|
_install_root, install_state, data_root, _runtime_state = onboarding_context(args, require_runtime=True)
|
|
2642
2750
|
project_root = normalized(args.project_root or Path.cwd())
|
|
2643
2751
|
requested_connection = validate_connection(read_json_object(normalized(args.connection_file), "Connection plan"))
|
|
2752
|
+
resumable_connection = persisted_connection(requested_connection)
|
|
2644
2753
|
if not requested_connection.get("profile"):
|
|
2645
2754
|
raise OnboardingError("Composite datasource onboarding requires connection.profile in the approved contract")
|
|
2646
2755
|
profile = requested_connection["profile"]
|
|
@@ -2654,7 +2763,7 @@ def datasource_run_command(args: argparse.Namespace) -> dict:
|
|
|
2654
2763
|
files: list[str] = []
|
|
2655
2764
|
existing = read_onboarding_state(data_root, profile, required=False)
|
|
2656
2765
|
if existing:
|
|
2657
|
-
if existing.get("connection") !=
|
|
2766
|
+
if persisted_connection(existing.get("connection", {})) != resumable_connection:
|
|
2658
2767
|
raise OnboardingError("Existing onboarding profile does not match the requested connection plan")
|
|
2659
2768
|
adopted = bind_completed_workspace(existing, data_root, project_root)
|
|
2660
2769
|
plan_result = {
|
|
@@ -2687,6 +2796,7 @@ def datasource_run_command(args: argparse.Namespace) -> dict:
|
|
|
2687
2796
|
else:
|
|
2688
2797
|
configure_dry = datasource_configure_command(argparse.Namespace(
|
|
2689
2798
|
install_root=args.install_root, data_root=args.data_root, profile=profile, apply=False, replace=False,
|
|
2799
|
+
runtime_password=requested_connection.get("password"),
|
|
2690
2800
|
))
|
|
2691
2801
|
save_composite_result(evidence_dir, "02-datasource-dry.json", configure_dry, files)
|
|
2692
2802
|
if not args.approve_configure:
|
|
@@ -2702,6 +2812,7 @@ def datasource_run_command(args: argparse.Namespace) -> dict:
|
|
|
2702
2812
|
}
|
|
2703
2813
|
configured = datasource_configure_command(argparse.Namespace(
|
|
2704
2814
|
install_root=args.install_root, data_root=args.data_root, profile=profile, apply=True, replace=False,
|
|
2815
|
+
runtime_password=requested_connection.get("password"),
|
|
2705
2816
|
))
|
|
2706
2817
|
save_composite_result(evidence_dir, "03-datasource-apply.json", configured, files)
|
|
2707
2818
|
state = read_onboarding_state(data_root, profile)
|