@descix/cli 1.0.17 → 1.0.19

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/agent-assets/guides/README.md +2 -1
  2. package/agent-assets/guides/dynamic-site-pattern.md +6 -10
  3. package/agent-assets/guides/knowledge-only-pattern.md +12 -11
  4. package/agent-assets/guides/microservice-pattern.md +13 -16
  5. package/agent-assets/guides/static-site-pattern.md +6 -7
  6. package/agent-assets/instructions/app-asset-validation.md +61 -59
  7. package/agent-assets/instructions/onboarding-flow.md +57 -74
  8. package/agent-assets/instructions/standard-folder-structure.md +39 -36
  9. package/agent-assets/reference/app-creation.md +27 -33
  10. package/agent-assets/reference/architecture.md +85 -148
  11. package/agent-assets/reference/local-dev.md +5 -83
  12. package/agent-assets/reference/mcp-integration.md +59 -49
  13. package/agent-assets/reference/workspace-config.md +9 -10
  14. package/bin/descix.js +20 -151
  15. package/lib/command-visibility.js +1 -1
  16. package/lib/commands/airdrop.js +6 -1
  17. package/lib/commands/init.js +2 -2
  18. package/lib/commands/kb-sync-surface.js +10 -0
  19. package/lib/commands/kb.js +1 -1
  20. package/lib/commands/microservice-auth-model.js +19 -0
  21. package/lib/commands/retired-verbs.js +106 -0
  22. package/lib/help-guard.js +85 -0
  23. package/lib/service-api-client.js +7 -6
  24. package/lib/verb-invokes.generated.json +0 -5
  25. package/lib/workspace-config.js +1 -1
  26. package/package.json +3 -3
  27. package/templates/agent-claude.md +6 -4
  28. package/templates/agent-cline.md +2 -2
  29. package/templates/agent-copilot.md +2 -2
  30. package/templates/agent-cursor.md +2 -2
  31. package/templates/scaffolds/microservice/SERVICE_README_sdk.md +6 -5
  32. package/templates/scaffolds/microservice/app.js +7 -3
  33. package/templates/scaffolds/microservice/defaults-config.json +2 -1
  34. package/templates/scaffolds/microservice/dev-overrides.example.json +2 -1
  35. package/templates/scaffolds/microservice/manifest.json +2 -1
  36. package/templates/scaffolds/microservice/package.json +2 -2
  37. package/templates/scaffolds/microservice/templates/SERVICE_README_TEMPLATE.md +3 -2
  38. package/lib/commands/retired-kb-sync.js +0 -112
@@ -23,13 +23,15 @@ DeSciX uses a "Mono-Repo" style structure where multiple communities and apps ca
23
23
  │ │ ├── icon.png
24
24
  │ │ └── system_instructions.md (AI Persona)
25
25
  │ │
26
- │ ├── kb/ # Knowledge Base (RAG)
27
- │ │ ├── staging/ # Local files to push to Drive
26
+ │ ├── .descix/
27
+ │ │ └── manifests/
28
+ │ │ └── General.json # Corpus manifest — names the KB's git-tracked sources
29
+ │ │
30
+ │ ├── kb/ # Knowledge Base content — OPTIONAL location, any path works
31
+ │ │ ├── staging/ # (if using Drive) Local files to push to Drive
28
32
  │ │ │ └── new-doc.pdf
29
- │ │ ├── General/ # Converted markdown (from Drive pull)
30
- │ │ │ └── *.md
31
- │ │ └── chunks/ # JSON chunk files (for vectorization)
32
- │ │ └── *.chunks.json
33
+ │ │ └── General/ # (if using Drive) Converted markdown from `descix drive pull`
34
+ │ │ └── *.md
33
35
  │ │
34
36
  │ ├── site/ # Frontend UI (Optional)
35
37
  │ │ ├── index.html
@@ -66,19 +68,24 @@ Self-contained folder for a single application.
66
68
  - **`system_instructions.md`**: Instructions for the AI Agent that interacts with users of this app.
67
69
  - **`icon.png`**: App Icon.
68
70
 
69
- #### `kb/` - Knowledge Base
70
- The KB folder supports both local-first (Git mode) and Drive-based workflows:
71
+ #### `.descix/manifests/<KB>.json` - Corpus Manifest
72
+ **This is what makes a KB syncable — not any folder's existence.** It names the git-tracked
73
+ path(s) `descix kb corpus sync` walks, chunks and upserts to Pinecone. No manifest means nothing
74
+ to sync, regardless of what is under `kb/`.
75
+
76
+ #### `kb/` - Knowledge Base content (optional convention)
77
+ There is no required KB folder. `kb/` is one common place to keep content, especially if you also
78
+ use Drive-based authoring:
71
79
  - **`staging/`**: Place raw files here (PDFs, images) to push them to Drive via `descix drive push`.
72
- - **`General/`**: The canonical source of text content. Files here are pulled from Drive and converted to Markdown via `descix drive pull`.
73
- - **`chunks/`**: Processed JSON chunk files ready for vectorization via `descix kb corpus sync`.
80
+ - **`General/`**: Files pulled from Drive and converted to Markdown via `descix drive pull`. Still
81
+ needs to be named as a source in the corpus manifest to be indexed — pulling alone does not sync.
74
82
 
75
83
  **CLI-Centric Processing (Git Mode):**
76
84
  ```bash
77
- descix drive push # Push staging/ -> Drive
78
- descix drive pull # Pull Drive -> General/ (converted to markdown)
79
- descix kb corpus sync # Generate chunks from General/ -> chunks/
80
- descix kb corpus sync # Push chunks to Pinecone via service layer
81
- descix kb corpus sync # Convenience: pull → chunk → sync
85
+ descix drive push # (optional) Push staging/ -> Drive
86
+ descix drive pull # (optional) Pull Drive -> local markdown, converted
87
+ # Commit the markdown, name its folder in .descix/manifests/<KB>.json, then:
88
+ descix kb corpus sync # Walk the manifest's sources, chunk what changed, upsert + purge in Pinecone
82
89
  ```
83
90
 
84
91
  #### `site/` - Frontend
@@ -94,34 +101,30 @@ descix kb corpus sync # Convenience: pull → chunk → sync
94
101
 
95
102
  ## Configuration Files
96
103
 
97
- ### `.descix/workspace.json` (Root)
104
+ ### `.descix/workspace.json` (Root) — v2.1, the only supported format
105
+
98
106
  ```json
99
107
  {
100
- "version": "2.0",
101
- "communities": {
102
- "my-dao": {
103
- "apps": {
104
- "governance": {
105
- "localPath": "my-dao/governance",
106
- "sync_mode": "git"
107
- },
108
- "treasury": {
109
- "localPath": "my-dao/treasury",
110
- "sync_mode": "drive"
111
- }
112
- }
113
- }
108
+ "version": "2.1",
109
+ "env": {
110
+ "environment": "DEV",
111
+ "apiUrl": "https://dev.descix.net",
112
+ "products": [
113
+ { "appId": "my-dao-governance", "communityId": "my-dao", "localPath": "my-dao/governance", "kbId": "General" },
114
+ { "appId": "my-dao-treasury", "communityId": "my-dao", "localPath": "my-dao/treasury", "kbId": "General" }
115
+ ]
114
116
  }
115
117
  }
116
118
  ```
117
119
 
118
- **`sync_mode` Options:**
119
- | Mode | KB Source | Use Case |
120
- |------|-----------|----------|
121
- | `git` | Local files | CLI-managed apps, developers (default for CLI) |
122
- | `drive` | Pull from Drive | PWA-created apps, non-technical users |
120
+ Written by CLI verbs only — `descix config init`, `descix app init`, `descix app set-site`,
121
+ `descix app set-port`. A `communities`-keyed block with no `env` block is the removed v1 shape and
122
+ is refused on load: `v1 workspace format is not supported. Migrate to v2.1.`
123
123
 
124
- **Note:** The CLI only supports `git` mode operations. If you are using the CLI, you are effectively in `git` mode.
124
+ **Sync mode:** The CLI only performs **git-mode** operations (local files → corpus manifest →
125
+ Pinecone, via `descix kb corpus sync`). Drive-mode (Drive → GCS → Pinecone, fully server-side) is
126
+ PWA-only and is never triggered by the CLI. There is no per-app `sync_mode` switch in workspace.json
127
+ — using the CLI at all means git mode.
125
128
 
126
129
  ## Agent Workflow for Migration
127
130
 
@@ -44,16 +44,9 @@ This separation ensures:
44
44
 
45
45
  ### Drive Templates
46
46
 
47
- **Location:** Google Drive, owned by `dip@descix.net`
48
- **SDK Reference:** `DeSciX_Core/descix-cli/templates/drive/`
49
- **Template IDs:** Configured in `DeSciX_Cloud/microservice/defaults-config.json`
50
-
51
- ```json
52
- {
53
- "DRIVE_COMMUNITY_TEMPLATE_FOLDER_ID": "1ABC...",
54
- "DRIVE_AGENT_APP_TEMPLATE_FOLDER_ID": "1XYZ..."
55
- }
56
- ```
47
+ **Location:** Google Drive, owned by the platform. The CLI ships its own copy of the template
48
+ layout under `templates/drive/`; the platform's Drive folder IDs are server-side config, not
49
+ something the CLI or a developer needs to touch.
57
50
 
58
51
  **Community Template:**
59
52
  ```
@@ -80,9 +73,10 @@ templates/drive/app/
80
73
 
81
74
  ### Git Scaffolds
82
75
 
83
- **Location:** microservice — `DeSciX_Core/descix-cli/templates/scaffolds/`;
84
- site — `DeSciX_Core/descix-app-sdk/scaffold/` (owned there, and exported as `SITE_SCAFFOLD_DIR`
85
- from `@descix/app-sdk/scaffold`, so no caller re-derives the path).
76
+ **Location:** the microservice scaffold ships inside the CLI package (`templates/scaffolds/`); the
77
+ site scaffold ships inside `@descix/app-sdk` and is exported as `SITE_SCAFFOLD_DIR` from
78
+ `@descix/app-sdk/scaffold`, so no caller re-derives the path. `descix microservice init` /
79
+ `descix site init` copy these for you — you do not need to locate them yourself.
86
80
 
87
81
  **Site Scaffold:**
88
82
  ```
@@ -243,12 +237,16 @@ detection.
243
237
  | Folder | Purpose | Sync Direction |
244
238
  |--------|---------|----------------|
245
239
  | `assets/` | App metadata (icon, description) | Bidirectional |
246
- | `kb/staging/` | Local files to push to Drive | Local → Drive |
247
- | `kb/General/` | Text-converted files from Drive | Drive → Local |
248
- | `kb/chunks/` | JSON chunks for Pinecone | Local only |
240
+ | `.descix/manifests/<KB>.json` | Names the KB's git-tracked sources — REQUIRED for a KB to sync | Local, git-tracked |
241
+ | `kb/staging/` (optional) | Local files to push to Drive | Local → Drive |
242
+ | `kb/General/` (optional) | Text-converted files from Drive | Drive → Local |
249
243
  | `site/` | Static site files | Local → GCS |
250
244
  | `microservice/` | Service code | Local → GCS |
251
245
 
246
+ There is no required KB folder and no persisted chunk directory — chunking happens in memory
247
+ during `descix kb corpus sync`. A knowledge base is whatever git-tracked path(s) the corpus
248
+ manifest names; `kb/General/` is a common convention, not a requirement.
249
+
252
250
  ### Required Files
253
251
 
254
252
  **`assets/` folder:**
@@ -256,9 +254,10 @@ detection.
256
254
  - `app_description.md` - Markdown description for App Store
257
255
  - `system_instructions.md` - AI agent persona and instructions
258
256
 
259
- **`kb/General/` folder:**
260
- - Contains reference documents for RAG
261
- - All formats converted to Markdown/text
257
+ **`.descix/manifests/<KB>.json`:**
258
+ - Names the source path(s) that make up the KB
259
+ - `descix kb corpus sync` walks them at a git ref and publishes to Pinecone — no manifest means
260
+ nothing to sync
262
261
 
263
262
  ---
264
263
 
@@ -322,17 +321,12 @@ descix site upload -a <app_id> # deploy the static site
322
321
 
323
322
  ---
324
323
 
325
- ## 9. File References
326
-
327
- | Component | Path | Description |
328
- |-----------|------|-------------|
329
- | Community Management | `DeSciX_Cloud/microservice/services/communityManagement.js` | Community/App creation |
330
- | App Commands | `DeSciX_Cloud/microservice/services/commandHandlers/appCommands.js` | Server-side app ops |
331
- | Google Storage Service | `DeSciX_Cloud/microservice/services/googleStorageService.js` | Drive/GCS operations |
332
- | Template Config | `DeSciX_Cloud/microservice/defaults-config.json` | Template folder IDs |
333
- | Drive Templates | `DeSciX_Core/descix-cli/templates/drive/` | Content templates |
334
- | Git Scaffolds | `DeSciX_Core/descix-cli/templates/scaffolds/` | Code scaffolds |
335
- | Hydrator | `DeSciX_Core/descix-cli/lib/core/Hydrator.js` | `copyScaffold` — copies the site / microservice scaffold into an app |
336
- | WorkspaceConfig | `DeSciX_Core/descix-cli/lib/workspace-config.js` | CLI configuration |
337
- | Setup Command | `DeSciX_Core/descix-cli/lib/wizard/setup.js` | Initial setup |
338
- | Scaffold Command | `DeSciX_Core/descix-cli/bin/descix.js` | CLI scaffolds |
324
+ ## 9. Summary of Verbs
325
+
326
+ | Concept | Owned by |
327
+ |---------|----------|
328
+ | Community/App creation | Platform backend (server-side; not part of the SDK you have locally) |
329
+ | Drive/GCS content operations | Platform backend, reached via `descix app init`, `descix drive pull/push` |
330
+ | Content templates (Drive) | Shipped inside the CLI package; copied by `descix app init` |
331
+ | Code scaffolds (site/microservice) | Shipped inside the CLI/app-sdk packages; copied by `descix site init` / `descix microservice init` |
332
+ | Local workspace configuration | `.descix/workspace.json`, written only by CLI verbs — see `workspace-config.md` |
@@ -12,93 +12,55 @@ This document is the canonical reference for the DeSciX SDK architecture, coveri
12
12
 
13
13
  ### Core Philosophy
14
14
 
15
- 1. **Drive as Backend:** Google Drive is the canonical source for raw assets (PDFs, documents, images).
16
- 2. **Git as Truth:** The local Git repository is the canonical source for *processed* text and code.
17
- 3. **Client-Side Mediation:** The CLI communicates directly with Drive via ADC (Application Default Credentials), removing server-side proxy overhead for file transfers.
18
- 4. **No Local Tools:** The CLI relies on Drive's internal conversion capabilities (OCR, Doc-to-Markdown) instead of bundling heavy local dependencies.
15
+ 1. **Git as Truth:** The git repository — whatever files a corpus manifest names — is the
16
+ canonical source for what gets published to a knowledge base. There is no required KB folder.
17
+ 2. **Manifest-Driven Publish:** `descix kb corpus sync` walks the sources a manifest names at a
18
+ git ref, chunks what changed, and upserts/purges against Pinecone in one pass. This is the
19
+ ONE sync surface; there is no separate chunk step or push step for the developer to run.
20
+ 3. **Drive as an optional authoring front-end:** Google Drive is one way to *author* raw content
21
+ (PDFs, documents, images) before it lands in git — via `descix drive pull`/`descix drive push`
22
+ (ADC-based, client-side, no server-side proxy for the file transfer). It is not required, and
23
+ it is not itself a sync step — content still has to be committed and named in a manifest.
24
+ 4. **No Local Tools:** Drive-based conversion (OCR, Doc-to-Markdown) uses Drive's own capabilities
25
+ instead of bundling heavy local dependencies.
19
26
 
20
27
  ### Data Flow Summary
21
28
 
22
29
  ```
23
- Drive (raw assets) → CLI (pull + convert) → Local Git (text + chunks) → Backend API → Pinecone
30
+ Any git-tracked source, optionally authored via Drive:
31
+ Drive (raw assets, optional) → CLI (pull + convert) → committed to git
32
+ │
33
+ named in .descix/manifests/<KB>.json
34
+ │
35
+ ▼
36
+ descix kb corpus sync → Backend API → Pinecone
24
37
  ```
25
38
 
26
39
  ---
27
40
 
28
- ## 2. Core Components
41
+ ## 2. Core Components (SDK internals — not a developer-facing API)
29
42
 
30
- All core SDK logic resides in `DeSciX_Core/descix-cli/lib/core/`.
43
+ These modules are internal to the CLI. A developer never calls them directly; they run underneath
44
+ the CLI verbs below.
31
45
 
32
- ### 2.1 Hydrator (`Hydrator.js`)
46
+ ### 2.1 Hydrator
33
47
 
34
- **Responsibilities:**
35
- - Pull content from Drive to local filesystem
36
- - Push staging files to Drive
37
- - Convert files using Drive's native capabilities (PDF/Image → Google Doc → Markdown)
38
- - Conflict detection and resolution during hydration
48
+ Drive-side operations, invoked by `descix drive pull` / `descix drive push`:
49
+ - Pulls content from Drive to the local filesystem, converting Drive's native formats to Markdown
50
+ (PDF/DOCX/Image → temp Google Doc → Markdown export; Google Docs → direct Markdown export;
51
+ Sheets → CSV; plain text → downloaded as-is)
52
+ - Pushes staging files to Drive
53
+ - Also copies the `site`/`microservice` scaffold into an app (`descix site init` /
54
+ `descix microservice init`)
39
55
 
40
- **Key Functions:**
56
+ ### 2.2 Chunker and Syncer
41
57
 
42
- | Function | Description |
43
- |----------|-------------|
44
- | `copyScaffold(type, appPath, options)` | Copy the `site` or `microservice` scaffold into an app (`descix site init` / `descix microservice init`) |
45
- | `hydrateApp(config, options)` | Pull entire app from Drive |
46
- | `hydrateKb(config, options)` | Pull and convert KB files from Drive |
47
- | `pushStaging(config, options)` | Upload files from `kb/staging/` to Drive |
48
- | `convertAndSave(driveFile, localDir)` | Convert and save a single file |
49
-
50
- **Drive-Native Conversion Logic:**
51
- 1. PDF/DOCX/Images: Copy with `convert=true` → Creates temp Google Doc → Export as Markdown → Delete temp doc
52
- 2. Google Docs: Export directly as `text/markdown`
53
- 3. Google Sheets: Export as `text/csv`
54
- 4. Plain text files: Download as-is
55
-
56
- ### 2.2 Chunker (`Chunker.js`)
57
-
58
- **Responsibilities:**
59
- - Chunk documents for RAG embedding
60
- - Support multiple chunking strategies (semantic, sliding window)
61
- - Generate chunk records with metadata for Pinecone
62
-
63
- **Key Functions:**
64
-
65
- | Function | Description |
66
- |----------|-------------|
67
- | `categorizeFile(fileName, mimeType)` | Detect file type (code/docs/papers/generic) |
68
- | `chunkFile(fileInfo, options)` | Chunk a file based on its type |
69
- | `chunkDocument(content, metadata, rules)` | Markdown-aware section-based chunking |
70
- | `processKb(config, options)` | Process entire KB directory into chunks |
71
- | `loadChunks(config)` | Load all chunks from JSON files |
72
-
73
- **Chunking Parameters (defaults):**
74
- - `maxChunkSize`: 2000 characters
75
- - `maxCodeFileSize`: 8000 characters
76
- - `overlapSize`: 500 characters
77
-
78
- **Chunking Strategies:**
79
- - **Documents:** Split by Markdown headers (`#`, `##`, `###`) with overlap
80
- - **Code:** Split by logical boundaries (functions, classes, contracts)
81
- - **Generic:** Sliding window with overlap
82
-
83
- ### 2.3 Syncer (`Syncer.js`)
84
-
85
- **Responsibilities:**
86
- - Sync local chunks to Pinecone via backend API
87
- - Compute delta between local and remote chunks
88
- - Manage chunk lifecycle (upsert/delete)
89
-
90
- **Key Functions:**
91
-
92
- | Function | Description |
93
- |----------|-------------|
94
- | `syncKb(apiClient, config, options)` | Full sync workflow (delta + upsert + delete) |
95
- | `computeChunkDelta(localChunks, remoteIds)` | Compare local vs remote chunks |
96
- | `getRemoteChunkIds(apiClient, communityId, appId, kbId)` | Get existing chunk IDs from Pinecone |
97
- | `upsertChunks(apiClient, ...)` | Push chunks to backend |
98
- | `deleteStaleChunks(apiClient, ...)` | Remove deleted chunks |
99
- | `getSyncStatus(apiClient, config)` | Check sync state |
100
-
101
- **Security Note:** The CLI never communicates with Pinecone directly. All chunk operations go through the backend API, which validates metadata and manages Pinecone credentials.
58
+ Invoked by `descix kb corpus sync`, never called separately: chunk documents (Markdown-aware
59
+ section splitting, with a code-aware and a generic sliding-window strategy for non-Markdown files),
60
+ then upsert/delete against Pinecone via the backend API — computing the delta between what the
61
+ manifest's git walk found and what is already indexed. **The CLI never communicates with Pinecone
62
+ directly.** All chunk operations go through the backend API, which validates metadata and manages
63
+ Pinecone credentials.
102
64
 
103
65
  ---
104
66
 
@@ -112,15 +74,15 @@ All core SDK logic resides in `DeSciX_Core/descix-cli/lib/core/`.
112
74
  │ ├── icon.png
113
75
  │ ├── app_description.md
114
76
  │ └── system_instructions.md
115
- ├── kb/
116
- │ ├── staging/ # Local files to push to Drive
77
+ ├── .descix/
78
+ │ └── manifests/
79
+ │ └── General.json # Corpus manifest — names the KB's git-tracked sources
80
+ ├── kb/ # OPTIONAL — content can live anywhere; kb/ is a common convention
81
+ │ ├── staging/ # (if using Drive) Local files to push to Drive
117
82
  │ │ └── research.pdf
118
- │ ├── General/ # Text-converted mirror of Drive
119
- │ │ ├── research.md # Converted from PDF
120
- │ │ └── notes.md # Converted from Google Doc
121
- │ └── chunks/ # Processed JSON chunks (Git-tracked)
122
- │ ├── research.chunks.json
123
- │ └── notes.chunks.json
83
+ │ └── General/ # (if using Drive) Text-converted mirror of Drive
84
+ │ ├── research.md # Converted from PDF
85
+ │ └── notes.md # Converted from Google Doc
124
86
  ├── site/ # Static site files (push-only)
125
87
  │ ├── index.html
126
88
  │ └── DeSciXAppSDK.js # The DeSciX bridge (generated)
@@ -129,14 +91,17 @@ All core SDK logic resides in `DeSciX_Core/descix-cli/lib/core/`.
129
91
  └── services/ # Service code — there is no src/
130
92
  ```
131
93
 
94
+ Chunking happens in memory during `descix kb corpus sync` — there is no persisted `kb/chunks/`
95
+ folder of JSON chunk files to manage.
96
+
132
97
  ### Folder Purposes
133
98
 
134
99
  | Folder | Sync Direction | Purpose |
135
100
  |--------|----------------|---------|
136
101
  | `assets/` | Bidirectional | App metadata (icon, description, instructions) |
137
- | `kb/staging/` | Local → Drive | Raw files waiting to be pushed |
138
- | `kb/General/` | Drive → Local | Text-converted files from Drive |
139
- | `kb/chunks/` | Local only | JSON chunk files for Pinecone sync |
102
+ | `.descix/manifests/<KB>.json` | Local, git-tracked | Names the sources `descix kb corpus sync` walks — REQUIRED for a KB to sync |
103
+ | `kb/staging/` | Local → Drive (optional) | Raw files waiting to be pushed via `descix drive push` |
104
+ | `kb/General/` | Drive → Local (optional) | Text-converted files from `descix drive pull` |
140
105
  | `site/` | Local → GCS | Static site deployment |
141
106
  | `microservice/` | Local → GCS | Backend service deployment |
142
107
 
@@ -206,8 +171,7 @@ The DeSciX CLI publishes content with one verb per plane. Each command resolves
206
171
  | `descix site upload` | Deploy site to GCS |
207
172
  | `descix app set-site -a <app_id> --port <port>` | Register the local dev-server port for the app's site (`env.products[].site.port`) |
208
173
  | `descix microservice init` | Copy microservice template to current app's `microservice/` folder |
209
- | `descix microservice register` | Register microservice with gateway |
210
- | `descix microservice vectorize` | Vectorize README for discovery |
174
+ | `descix microservice register -r SERVICE_README.md` | Register microservice AND vectorize the README for `tell_me_how` discovery, in one step |
211
175
 
212
176
  **Note:** Site and microservice commands are Git-based code operations. Drive templates contain content (assets, KB), while site/microservice commands handle code (HTML, JS, Dockerfile, etc.).
213
177
 
@@ -284,47 +248,49 @@ CLI → kb_sync_chunks API → pineconeService.upsertChunkRecords() → Pinecone
284
248
 
285
249
  ## 6. Data Flows
286
250
 
287
- ### Push-Pull-Convert Cycle
251
+ ### Publish Cycle
288
252
 
289
253
  ```mermaid
290
254
  flowchart LR
291
- subgraph local [Local Filesystem]
292
- staging[kb/staging/]
293
- general[kb/General/]
294
- chunks[kb/chunks/]
255
+ subgraph drive [Google Drive — optional authoring]
256
+ driveKb[Drive folder]
295
257
  end
296
-
297
- subgraph drive [Google Drive]
298
- driveKb[kb/General/]
258
+
259
+ subgraph local [Local Git repo]
260
+ staging[kb/staging/ — optional]
261
+ docs[docs (or any git-tracked path)]
262
+ manifest[".descix/manifests/&lt;KB&gt;.json"]
299
263
  end
300
-
264
+
301
265
  subgraph backend [Backend]
302
266
  api[kb_sync_chunks]
303
267
  pinecone[Pinecone]
304
268
  end
305
-
306
- staging -->|"kb push"| driveKb
307
- driveKb -->|"kb pull (convert)"| general
308
- general -->|"kb chunk"| chunks
309
- chunks -->|"kb sync"| api
269
+
270
+ staging -->|"descix drive push"| driveKb
271
+ driveKb -->|"descix drive pull (convert)"| docs
272
+ docs -->|"named as a source in"| manifest
273
+ manifest -->|"descix kb corpus sync (walk + chunk in memory + upsert)"| api
310
274
  api --> pinecone
311
275
  ```
312
276
 
313
277
  ### Detailed Flow
314
278
 
315
- 1. **Stage:** User adds `paper.pdf` to `kb/staging/`
316
- 2. **Push:** `descix drive push` uploads to Drive `kb/General/`
317
- 3. **Pull:** `descix drive pull` downloads and converts:
279
+ 1. **(Optional) Stage:** User adds `paper.pdf` to `kb/staging/`
280
+ 2. **(Optional) Push:** `descix drive push` uploads to a Drive folder
281
+ 3. **(Optional) Pull:** `descix drive pull` downloads and converts:
318
282
  - Sees `paper.pdf` in Drive
319
283
  - Copies with `convert=true` → temp Google Doc
320
- - Exports as Markdown → `kb/General/paper.md`
284
+ - Exports as Markdown → local `paper.md`
321
285
  - Deletes temp doc
322
- 4. **Chunk:** `descix kb corpus sync` processes `paper.md`:
323
- - Generates `kb/chunks/paper.chunks.json`
324
- 5. **Sync:** `descix kb corpus sync` pushes to backend:
325
- - Backend validates metadata
326
- - Forwards to Pinecone
327
- - Pinecone embeds and stores
286
+ 4. **Commit + name:** commit `paper.md` and name its folder as a source in
287
+ `.descix/manifests/<KB>.json` (skip steps 1-3 entirely if the content already lives in git —
288
+ Drive is optional)
289
+ 5. **Sync:** `descix kb corpus sync` does the rest in one pass:
290
+ - Walks the manifest's sources at a git ref, chunking each file in memory (nothing is written
291
+ to disk)
292
+ - Pushes changed chunks to the backend, which validates metadata and forwards to Pinecone
293
+ - Purges any chunk that is live in Pinecone but no longer in the manifest's walk
328
294
 
329
295
  ---
330
296
 
@@ -352,49 +318,20 @@ CLI authenticates to backend via device login:
352
318
 
353
319
  ---
354
320
 
355
- ## 8. Migration Notes
356
-
357
- ### From `kb/src/` to `kb/staging/`
358
-
359
- The v2 architecture renamed `kb/src/` to `kb/staging/` for clarity. The Hydrator includes automatic migration:
360
-
361
- ```javascript
362
- // If old kb/src/ exists, rename to kb/staging/
363
- const oldSrcDir = path.join(kbBaseDir, 'src');
364
- if (await exists(oldSrcDir)) {
365
- await rename(oldSrcDir, stagingDir);
366
- }
367
- ```
368
-
369
- ### Git Mode vs Drive Mode
321
+ ## 8. Git Mode vs Drive Mode
370
322
 
371
323
  Apps operate in one of two modes, determined by how they are managed:
372
324
 
373
325
  | Mode | Source of Truth | Versioning | User Type | Tool |
374
326
  |------|-----------------|------------|-----------|------|
375
- | **git** | Local Git repo | Git | Developers | CLI (`descix kb *`) |
327
+ | **git** | Local Git repo | Git | Developers | CLI (`descix kb corpus sync`, `descix drive pull/push`) |
376
328
  | **drive** | Google Drive | GCS/Firestore | Non-developers | PWA only |
377
329
 
378
330
  **Important:** The CLI only supports **git-mode** operations. Drive-mode apps are managed entirely through the PWA with server-side processing. There is no CLI flag to switch modes - if you're using the CLI, you're in git-mode.
379
331
 
380
- Both modes start with content on Google Drive (users create documents there). The difference is:
381
- - **Git mode**: Developer pulls content from Drive, then manages text/chunks locally with Git
382
- - **Drive mode**: PWA triggers server-side pipeline (Drive → GCS → Pinecone) automatically
383
-
384
- ---
385
-
386
- ## 9. File References
387
-
388
- | Component | Path | Description |
389
- |-----------|------|-------------|
390
- | WorkspaceConfig | `DeSciX_Core/descix-cli/lib/workspace-config.js` | Sole configuration class |
391
- | GlobalConfig | `DeSciX_Core/descix-cli/lib/global-config.js` | User-level settings (~/.descix) |
392
- | Hydrator | `DeSciX_Core/descix-cli/lib/core/Hydrator.js` | Drive sync operations |
393
- | Chunker | `DeSciX_Core/descix-cli/lib/core/Chunker.js` | KB chunking |
394
- | Syncer | `DeSciX_Core/descix-cli/lib/core/Syncer.js` | Pinecone sync |
395
- | KB Commands | `DeSciX_Core/descix-cli/lib/commands/kb.js` | Git-mode KB processing |
396
- | Setup Command | `DeSciX_Core/descix-cli/lib/wizard/setup.js` | Initial workspace setup |
397
- | Drive ADC | `DeSciX_Core/descix-cli/lib/google-storage-adc.js` | Google Drive API wrapper |
398
- | Backend KB API | `DeSciX_Cloud/microservice/services/commandHandlers/appCommands.js` | Server-side processing |
399
- | Pinecone Service | `DeSciX_Cloud/microservice/services/pineconeService.js` | Vector database |
400
- | Server Chunking | `DeSciX_Cloud/microservice/services/chunkingUtils.js` | Drive-mode chunking |
332
+ Git mode does not require Drive at all: content can be authored directly in the repo. Drive is one
333
+ optional way to bring raw content (PDFs, images) in as converted Markdown, via `descix drive pull`.
334
+ - **Git mode**: developer commits content (optionally pulled from Drive first), names it in a
335
+ corpus manifest, and `descix kb corpus sync` publishes it
336
+ - **Drive mode**: PWA triggers a server-side pipeline (Drive → GCS → Pinecone) automatically; the
337
+ CLI never triggers it
@@ -169,9 +169,11 @@ the survey outranks the paper for exactly the reader who does not yet know the p
169
169
 
170
170
  **There is no port-derived step for the API.** An environment NAME never becomes an origin, and
171
171
  `env.platform.microservice.port` is not consulted: a local backend is a URL you name
172
- (`descix config set-env dev --url https://localhost:4000`, or `env.apiUrl` direct). The single
173
- owner of this order is `lib/origin.js::PRECEDENCE`; this table is a description of it, not a
174
- second copy of the rule.
172
+ (`descix config set-env dev --url https://localhost:4000`, or `env.apiUrl` direct). **This is
173
+ platform-dev mode** — running your own local backend (Scenario 2 below), not part of standard app
174
+ development, which targets a hosted environment (`descix config set-env dev` with no `--url`, or
175
+ `demo`/`prod`). The single owner of this order is `lib/origin.js::PRECEDENCE`; this table is a
176
+ description of it, not a second copy of the rule.
175
177
 
176
178
  **Every resolution carries its SOURCE, and every network-bound command prints it on stderr** —
177
179
  always, not only when it lands on the default:
@@ -250,83 +252,3 @@ human it is unverified.
250
252
  * **The credential is a body field**, not a cookie and not a header: every call is a relative `POST /apifront/` with the token in the JSON body. There is no `SameSite` problem, no credentialed CORS preflight, and the browser origin is irrelevant to the mesh — same-origin is required for *frame scripting*, not for auth.
251
253
  * **Set your framework's base path to `/p/{appId}`.** The gateway does not rewrite paths, so assets requested at `/` will 404.
252
254
 
253
- ## 6. Running from a git worktree
254
-
255
- `.descix/workspace.json` is gitignored, so no worktree has one. The gateway walks **up** from cwd to find it, so an in-repo worktree silently inherits the **main checkout's** workspace — its port, its products, its `localPath`s. Copying the file into a worktree does not help: a saved workspace bakes an absolute `workspaceRoot` and prefers it over the discovered root, pointing back at the original checkout. Out-of-repo worktrees fail earlier and more confusingly: a credential-loading command reports **"Authentication required… Run: descix login"** when the real cause is a missing workspace file.
256
-
257
- Rules of thumb: run the CLI from the canonical checkout, and treat an auth error raised from a worktree as a workspace-resolution suspect *before* re-authenticating.
258
-
259
- ### Deploying: probe the credential the action actually uses
260
-
261
- `deploy-env.sh` will refuse to start rather than fail halfway:
262
-
263
- ```
264
- gcloud credentials are NOT usable (expired user session, or no active account)...
265
- Refusing to start a real deploy that would fail partway through.
266
- ```
267
-
268
- That refusal is the script working correctly, and **an expired gcloud user session alongside live
269
- ADC is a normal steady state**, not a broken machine. The two are different credentials behind one
270
- word: the deploy path needs the **user session**, while most quick probes reach for **ADC**.
271
-
272
- This is the trap: `gcloud auth application-default print-access-token` returns a token, you record
273
- "gcloud is fine", and that observation is *true and irrelevant*. **Probe the credential the action
274
- actually uses, not an adjacent one that happens to answer.**
275
-
276
- The remedy is one line, needs no re-auth prompt, and the script prints it itself:
277
-
278
- ```bash
279
- export CLOUDSDK_AUTH_ACCESS_TOKEN="$(gcloud auth application-default print-access-token)"
280
- [ -n "$CLOUDSDK_AUTH_ACCESS_TOKEN" ] || { echo "empty token"; exit 1; }
281
- ```
282
-
283
- Assert it non-empty before launching — an empty token reproduces the same refusal one layer later,
284
- where it is harder to recognise.
285
-
286
- **Why this is written down rather than left to be rediscovered:** without it, the next agent reads
287
- a correct fail-loud refusal as *platform unreachable* and stops under the platform-reachability
288
- rule — halting a lane and escalating, over a one-line export the script had already told it. A
289
- fail-loud message nobody knows how to act on still costs a lane.
290
-
291
- ### Exit codes from wrappers are not evidence
292
-
293
- A deploy wrapper ending in `| tail` returns **tail's** status, not the deploy's. A run can report
294
- exit 0 while the deploy correctly returned 1. Capture the real status *before* anything pipes it:
295
-
296
- ```bash
297
- deploy-env.sh dev; rc=$? # capture FIRST
298
- deploy-env.sh dev 2>&1 | tail -40 # then look at output separately
299
- ```
300
-
301
- To confirm a deploy succeeded, **read the log, not the exit code** — and where the artifact is
302
- observable (a served object's timestamp, a string in the shipped bundle), check that instead. A
303
- pipe silently drops information in both directions: `| tail` hides a failure, and `| head` hides
304
- later matches from a search that looks complete.
305
-
306
- ### The SDK you edit is not always the SDK you serve
307
-
308
- A second, sharper worktree trap, and it is silent. `DeSciX_Cloud/site/node_modules/@descix/app-sdk`
309
- is a **symlink into the canonical Core checkout**:
310
-
311
- ```
312
- $ readlink DeSciX_Cloud/site/node_modules/@descix/app-sdk
313
- ../../../../DeSciX_Core/descix-app-sdk
314
- ```
315
-
316
- and `@descix/app-sdk` has **no build step** — its `main` and its `.` export both point straight at
317
- `./src/index.js`, so Vite serves the package source as-is. Put those two facts together: **`descix
318
- serve` serves whatever the CANONICAL Core working tree contains**, no matter which worktree you are
319
- editing in. Change `app-sdk` on a branch, reload, and you are still running canonical `main` — your
320
- edit is not in the page, nothing errors, and the obvious conclusion ("my change had no effect") is
321
- wrong.
322
-
323
- Verify what is actually being served before you conclude anything about a change:
324
-
325
- ```bash
326
- readlink DeSciX_Cloud/site/node_modules/@descix/app-sdk # where the symlink points
327
- node -p "require.resolve('@descix/app-sdk')" # from the site directory
328
- ```
329
-
330
- If it resolves to the canonical checkout and you are editing elsewhere, you are measuring the wrong
331
- tree. Re-point the symlink at your worktree for the measurement **and restore it afterwards** — it
332
- is untracked, so nothing will remind you.