@descix/cli 1.0.16 → 1.0.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agent-assets/guides/README.md +2 -1
- package/agent-assets/guides/dynamic-site-pattern.md +6 -10
- package/agent-assets/guides/knowledge-only-pattern.md +12 -11
- package/agent-assets/guides/microservice-pattern.md +13 -16
- package/agent-assets/guides/static-site-pattern.md +6 -7
- package/agent-assets/instructions/app-asset-validation.md +61 -59
- package/agent-assets/instructions/onboarding-flow.md +57 -73
- package/agent-assets/instructions/standard-folder-structure.md +39 -36
- package/agent-assets/reference/app-creation.md +27 -33
- package/agent-assets/reference/architecture.md +85 -148
- package/agent-assets/reference/local-dev.md +5 -83
- package/agent-assets/reference/mcp-integration.md +32 -39
- package/agent-assets/reference/workspace-config.md +9 -10
- package/bin/descix.js +30 -133
- package/lib/commands/appInit.js +154 -0
- package/lib/commands/init.js +14 -54
- package/lib/help-guard.js +85 -0
- package/lib/verb-invokes.generated.json +5 -1
- package/package.json +1 -1
- package/templates/agent-claude.md +14 -9
- package/templates/agent-cline.md +7 -7
- package/templates/agent-copilot.md +7 -7
- package/templates/agent-cursor.md +7 -7
|
@@ -23,13 +23,15 @@ DeSciX uses a "Mono-Repo" style structure where multiple communities and apps ca
|
|
|
23
23
|
│ │ ├── icon.png
|
|
24
24
|
│ │ └── system_instructions.md (AI Persona)
|
|
25
25
|
│ │
|
|
26
|
-
│ ├──
|
|
27
|
-
│ │
|
|
26
|
+
│ ├── .descix/
|
|
27
|
+
│ │ └── manifests/
|
|
28
|
+
│ │ └── General.json # Corpus manifest — names the KB's git-tracked sources
|
|
29
|
+
│ │
|
|
30
|
+
│ ├── kb/ # Knowledge Base content — OPTIONAL location, any path works
|
|
31
|
+
│ │ ├── staging/ # (if using Drive) Local files to push to Drive
|
|
28
32
|
│ │ │ └── new-doc.pdf
|
|
29
|
-
│ │
|
|
30
|
-
│ │
|
|
31
|
-
│ │ └── chunks/ # JSON chunk files (for vectorization)
|
|
32
|
-
│ │ └── *.chunks.json
|
|
33
|
+
│ │ └── General/ # (if using Drive) Converted markdown from `descix drive pull`
|
|
34
|
+
│ │ └── *.md
|
|
33
35
|
│ │
|
|
34
36
|
│ ├── site/ # Frontend UI (Optional)
|
|
35
37
|
│ │ ├── index.html
|
|
@@ -66,19 +68,24 @@ Self-contained folder for a single application.
|
|
|
66
68
|
- **`system_instructions.md`**: Instructions for the AI Agent that interacts with users of this app.
|
|
67
69
|
- **`icon.png`**: App Icon.
|
|
68
70
|
|
|
69
|
-
#### `
|
|
70
|
-
|
|
71
|
+
#### `.descix/manifests/<KB>.json` - Corpus Manifest
|
|
72
|
+
**This is what makes a KB syncable — not any folder's existence.** It names the git-tracked
|
|
73
|
+
path(s) `descix kb corpus sync` walks, chunks and upserts to Pinecone. No manifest means nothing
|
|
74
|
+
to sync, regardless of what is under `kb/`.
|
|
75
|
+
|
|
76
|
+
#### `kb/` - Knowledge Base content (optional convention)
|
|
77
|
+
There is no required KB folder. `kb/` is one common place to keep content, especially if you also
|
|
78
|
+
use Drive-based authoring:
|
|
71
79
|
- **`staging/`**: Place raw files here (PDFs, images) to push them to Drive via `descix drive push`.
|
|
72
|
-
- **`General/`**:
|
|
73
|
-
|
|
80
|
+
- **`General/`**: Files pulled from Drive and converted to Markdown via `descix drive pull`. Still
|
|
81
|
+
needs to be named as a source in the corpus manifest to be indexed — pulling alone does not sync.
|
|
74
82
|
|
|
75
83
|
**CLI-Centric Processing (Git Mode):**
|
|
76
84
|
```bash
|
|
77
|
-
descix drive push
|
|
78
|
-
descix drive pull
|
|
79
|
-
|
|
80
|
-
descix kb corpus sync
|
|
81
|
-
descix kb corpus sync # Convenience: pull → chunk → sync
|
|
85
|
+
descix drive push # (optional) Push staging/ -> Drive
|
|
86
|
+
descix drive pull # (optional) Pull Drive -> local markdown, converted
|
|
87
|
+
# Commit the markdown, name its folder in .descix/manifests/<KB>.json, then:
|
|
88
|
+
descix kb corpus sync # Walk the manifest's sources, chunk what changed, upsert + purge in Pinecone
|
|
82
89
|
```
|
|
83
90
|
|
|
84
91
|
#### `site/` - Frontend
|
|
@@ -94,34 +101,30 @@ descix kb corpus sync # Convenience: pull → chunk → sync
|
|
|
94
101
|
|
|
95
102
|
## Configuration Files
|
|
96
103
|
|
|
97
|
-
### `.descix/workspace.json` (Root)
|
|
104
|
+
### `.descix/workspace.json` (Root) — v2.1, the only supported format
|
|
105
|
+
|
|
98
106
|
```json
|
|
99
107
|
{
|
|
100
|
-
"version": "2.
|
|
101
|
-
"
|
|
102
|
-
"
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
"treasury": {
|
|
109
|
-
"localPath": "my-dao/treasury",
|
|
110
|
-
"sync_mode": "drive"
|
|
111
|
-
}
|
|
112
|
-
}
|
|
113
|
-
}
|
|
108
|
+
"version": "2.1",
|
|
109
|
+
"env": {
|
|
110
|
+
"environment": "DEV",
|
|
111
|
+
"apiUrl": "https://dev.descix.net",
|
|
112
|
+
"products": [
|
|
113
|
+
{ "appId": "my-dao-governance", "communityId": "my-dao", "localPath": "my-dao/governance", "kbId": "General" },
|
|
114
|
+
{ "appId": "my-dao-treasury", "communityId": "my-dao", "localPath": "my-dao/treasury", "kbId": "General" }
|
|
115
|
+
]
|
|
114
116
|
}
|
|
115
117
|
}
|
|
116
118
|
```
|
|
117
119
|
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
| `git` | Local files | CLI-managed apps, developers (default for CLI) |
|
|
122
|
-
| `drive` | Pull from Drive | PWA-created apps, non-technical users |
|
|
120
|
+
Written by CLI verbs only — `descix config init`, `descix app init`, `descix app set-site`,
|
|
121
|
+
`descix app set-port`. A `communities`-keyed block with no `env` block is the removed v1 shape and
|
|
122
|
+
is refused on load: `v1 workspace format is not supported. Migrate to v2.1.`
|
|
123
123
|
|
|
124
|
-
**
|
|
124
|
+
**Sync mode:** The CLI only performs **git-mode** operations (local files → corpus manifest →
|
|
125
|
+
Pinecone, via `descix kb corpus sync`). Drive-mode (Drive → GCS → Pinecone, fully server-side) is
|
|
126
|
+
PWA-only and is never triggered by the CLI. There is no per-app `sync_mode` switch in workspace.json
|
|
127
|
+
— using the CLI at all means git mode.
|
|
125
128
|
|
|
126
129
|
## Agent Workflow for Migration
|
|
127
130
|
|
|
@@ -44,16 +44,9 @@ This separation ensures:
|
|
|
44
44
|
|
|
45
45
|
### Drive Templates
|
|
46
46
|
|
|
47
|
-
**Location:** Google Drive, owned by
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
```json
|
|
52
|
-
{
|
|
53
|
-
"DRIVE_COMMUNITY_TEMPLATE_FOLDER_ID": "1ABC...",
|
|
54
|
-
"DRIVE_AGENT_APP_TEMPLATE_FOLDER_ID": "1XYZ..."
|
|
55
|
-
}
|
|
56
|
-
```
|
|
47
|
+
**Location:** Google Drive, owned by the platform. The CLI ships its own copy of the template
|
|
48
|
+
layout under `templates/drive/`; the platform's Drive folder IDs are server-side config, not
|
|
49
|
+
something the CLI or a developer needs to touch.
|
|
57
50
|
|
|
58
51
|
**Community Template:**
|
|
59
52
|
```
|
|
@@ -80,9 +73,10 @@ templates/drive/app/
|
|
|
80
73
|
|
|
81
74
|
### Git Scaffolds
|
|
82
75
|
|
|
83
|
-
**Location:** microservice
|
|
84
|
-
site
|
|
85
|
-
|
|
76
|
+
**Location:** the microservice scaffold ships inside the CLI package (`templates/scaffolds/`); the
|
|
77
|
+
site scaffold ships inside `@descix/app-sdk` and is exported as `SITE_SCAFFOLD_DIR` from
|
|
78
|
+
`@descix/app-sdk/scaffold`, so no caller re-derives the path. `descix microservice init` /
|
|
79
|
+
`descix site init` copy these for you — you do not need to locate them yourself.
|
|
86
80
|
|
|
87
81
|
**Site Scaffold:**
|
|
88
82
|
```
|
|
@@ -243,12 +237,16 @@ detection.
|
|
|
243
237
|
| Folder | Purpose | Sync Direction |
|
|
244
238
|
|--------|---------|----------------|
|
|
245
239
|
| `assets/` | App metadata (icon, description) | Bidirectional |
|
|
246
|
-
| `
|
|
247
|
-
| `kb/
|
|
248
|
-
| `kb/
|
|
240
|
+
| `.descix/manifests/<KB>.json` | Names the KB's git-tracked sources — REQUIRED for a KB to sync | Local, git-tracked |
|
|
241
|
+
| `kb/staging/` (optional) | Local files to push to Drive | Local → Drive |
|
|
242
|
+
| `kb/General/` (optional) | Text-converted files from Drive | Drive → Local |
|
|
249
243
|
| `site/` | Static site files | Local → GCS |
|
|
250
244
|
| `microservice/` | Service code | Local → GCS |
|
|
251
245
|
|
|
246
|
+
There is no required KB folder and no persisted chunk directory — chunking happens in memory
|
|
247
|
+
during `descix kb corpus sync`. A knowledge base is whatever git-tracked path(s) the corpus
|
|
248
|
+
manifest names; `kb/General/` is a common convention, not a requirement.
|
|
249
|
+
|
|
252
250
|
### Required Files
|
|
253
251
|
|
|
254
252
|
**`assets/` folder:**
|
|
@@ -256,9 +254,10 @@ detection.
|
|
|
256
254
|
- `app_description.md` - Markdown description for App Store
|
|
257
255
|
- `system_instructions.md` - AI agent persona and instructions
|
|
258
256
|
|
|
259
|
-
|
|
260
|
-
-
|
|
261
|
-
-
|
|
257
|
+
**`.descix/manifests/<KB>.json`:**
|
|
258
|
+
- Names the source path(s) that make up the KB
|
|
259
|
+
- `descix kb corpus sync` walks them at a git ref and publishes to Pinecone — no manifest means
|
|
260
|
+
nothing to sync
|
|
262
261
|
|
|
263
262
|
---
|
|
264
263
|
|
|
@@ -322,17 +321,12 @@ descix site upload -a <app_id> # deploy the static site
|
|
|
322
321
|
|
|
323
322
|
---
|
|
324
323
|
|
|
325
|
-
## 9.
|
|
326
|
-
|
|
327
|
-
|
|
|
328
|
-
|
|
329
|
-
| Community
|
|
330
|
-
|
|
|
331
|
-
|
|
|
332
|
-
|
|
|
333
|
-
|
|
|
334
|
-
| Git Scaffolds | `DeSciX_Core/descix-cli/templates/scaffolds/` | Code scaffolds |
|
|
335
|
-
| Hydrator | `DeSciX_Core/descix-cli/lib/core/Hydrator.js` | `copyScaffold` — copies the site / microservice scaffold into an app |
|
|
336
|
-
| WorkspaceConfig | `DeSciX_Core/descix-cli/lib/workspace-config.js` | CLI configuration |
|
|
337
|
-
| Setup Command | `DeSciX_Core/descix-cli/lib/wizard/setup.js` | Initial setup |
|
|
338
|
-
| Scaffold Command | `DeSciX_Core/descix-cli/bin/descix.js` | CLI scaffolds |
|
|
324
|
+
## 9. Summary of Verbs
|
|
325
|
+
|
|
326
|
+
| Concept | Owned by |
|
|
327
|
+
|---------|----------|
|
|
328
|
+
| Community/App creation | Platform backend (server-side; not part of the SDK you have locally) |
|
|
329
|
+
| Drive/GCS content operations | Platform backend, reached via `descix app init`, `descix drive pull/push` |
|
|
330
|
+
| Content templates (Drive) | Shipped inside the CLI package; copied by `descix app init` |
|
|
331
|
+
| Code scaffolds (site/microservice) | Shipped inside the CLI/app-sdk packages; copied by `descix site init` / `descix microservice init` |
|
|
332
|
+
| Local workspace configuration | `.descix/workspace.json`, written only by CLI verbs — see `workspace-config.md` |
|
|
@@ -12,93 +12,55 @@ This document is the canonical reference for the DeSciX SDK architecture, coveri
|
|
|
12
12
|
|
|
13
13
|
### Core Philosophy
|
|
14
14
|
|
|
15
|
-
1. **
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
15
|
+
1. **Git as Truth:** The git repository — whatever files a corpus manifest names — is the
|
|
16
|
+
canonical source for what gets published to a knowledge base. There is no required KB folder.
|
|
17
|
+
2. **Manifest-Driven Publish:** `descix kb corpus sync` walks the sources a manifest names at a
|
|
18
|
+
git ref, chunks what changed, and upserts/purges against Pinecone in one pass. This is the
|
|
19
|
+
ONE sync surface; there is no separate chunk step or push step for the developer to run.
|
|
20
|
+
3. **Drive as an optional authoring front-end:** Google Drive is one way to *author* raw content
|
|
21
|
+
(PDFs, documents, images) before it lands in git — via `descix drive pull`/`descix drive push`
|
|
22
|
+
(ADC-based, client-side, no server-side proxy for the file transfer). It is not required, and
|
|
23
|
+
it is not itself a sync step — content still has to be committed and named in a manifest.
|
|
24
|
+
4. **No Local Tools:** Drive-based conversion (OCR, Doc-to-Markdown) uses Drive's own capabilities
|
|
25
|
+
instead of bundling heavy local dependencies.
|
|
19
26
|
|
|
20
27
|
### Data Flow Summary
|
|
21
28
|
|
|
22
29
|
```
|
|
23
|
-
|
|
30
|
+
Any git-tracked source, optionally authored via Drive:
|
|
31
|
+
Drive (raw assets, optional) → CLI (pull + convert) → committed to git
|
|
32
|
+
│
|
|
33
|
+
named in .descix/manifests/<KB>.json
|
|
34
|
+
│
|
|
35
|
+
▼
|
|
36
|
+
descix kb corpus sync → Backend API → Pinecone
|
|
24
37
|
```
|
|
25
38
|
|
|
26
39
|
---
|
|
27
40
|
|
|
28
|
-
## 2. Core Components
|
|
41
|
+
## 2. Core Components (SDK internals — not a developer-facing API)
|
|
29
42
|
|
|
30
|
-
|
|
43
|
+
These modules are internal to the CLI. A developer never calls them directly; they run underneath
|
|
44
|
+
the CLI verbs below.
|
|
31
45
|
|
|
32
|
-
### 2.1 Hydrator
|
|
46
|
+
### 2.1 Hydrator
|
|
33
47
|
|
|
34
|
-
|
|
35
|
-
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
-
|
|
48
|
+
Drive-side operations, invoked by `descix drive pull` / `descix drive push`:
|
|
49
|
+
- Pulls content from Drive to the local filesystem, converting Drive's native formats to Markdown
|
|
50
|
+
(PDF/DOCX/Image → temp Google Doc → Markdown export; Google Docs → direct Markdown export;
|
|
51
|
+
Sheets → CSV; plain text → downloaded as-is)
|
|
52
|
+
- Pushes staging files to Drive
|
|
53
|
+
- Also copies the `site`/`microservice` scaffold into an app (`descix site init` /
|
|
54
|
+
`descix microservice init`)
|
|
39
55
|
|
|
40
|
-
|
|
56
|
+
### 2.2 Chunker and Syncer
|
|
41
57
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
| `convertAndSave(driveFile, localDir)` | Convert and save a single file |
|
|
49
|
-
|
|
50
|
-
**Drive-Native Conversion Logic:**
|
|
51
|
-
1. PDF/DOCX/Images: Copy with `convert=true` → Creates temp Google Doc → Export as Markdown → Delete temp doc
|
|
52
|
-
2. Google Docs: Export directly as `text/markdown`
|
|
53
|
-
3. Google Sheets: Export as `text/csv`
|
|
54
|
-
4. Plain text files: Download as-is
|
|
55
|
-
|
|
56
|
-
### 2.2 Chunker (`Chunker.js`)
|
|
57
|
-
|
|
58
|
-
**Responsibilities:**
|
|
59
|
-
- Chunk documents for RAG embedding
|
|
60
|
-
- Support multiple chunking strategies (semantic, sliding window)
|
|
61
|
-
- Generate chunk records with metadata for Pinecone
|
|
62
|
-
|
|
63
|
-
**Key Functions:**
|
|
64
|
-
|
|
65
|
-
| Function | Description |
|
|
66
|
-
|----------|-------------|
|
|
67
|
-
| `categorizeFile(fileName, mimeType)` | Detect file type (code/docs/papers/generic) |
|
|
68
|
-
| `chunkFile(fileInfo, options)` | Chunk a file based on its type |
|
|
69
|
-
| `chunkDocument(content, metadata, rules)` | Markdown-aware section-based chunking |
|
|
70
|
-
| `processKb(config, options)` | Process entire KB directory into chunks |
|
|
71
|
-
| `loadChunks(config)` | Load all chunks from JSON files |
|
|
72
|
-
|
|
73
|
-
**Chunking Parameters (defaults):**
|
|
74
|
-
- `maxChunkSize`: 2000 characters
|
|
75
|
-
- `maxCodeFileSize`: 8000 characters
|
|
76
|
-
- `overlapSize`: 500 characters
|
|
77
|
-
|
|
78
|
-
**Chunking Strategies:**
|
|
79
|
-
- **Documents:** Split by Markdown headers (`#`, `##`, `###`) with overlap
|
|
80
|
-
- **Code:** Split by logical boundaries (functions, classes, contracts)
|
|
81
|
-
- **Generic:** Sliding window with overlap
|
|
82
|
-
|
|
83
|
-
### 2.3 Syncer (`Syncer.js`)
|
|
84
|
-
|
|
85
|
-
**Responsibilities:**
|
|
86
|
-
- Sync local chunks to Pinecone via backend API
|
|
87
|
-
- Compute delta between local and remote chunks
|
|
88
|
-
- Manage chunk lifecycle (upsert/delete)
|
|
89
|
-
|
|
90
|
-
**Key Functions:**
|
|
91
|
-
|
|
92
|
-
| Function | Description |
|
|
93
|
-
|----------|-------------|
|
|
94
|
-
| `syncKb(apiClient, config, options)` | Full sync workflow (delta + upsert + delete) |
|
|
95
|
-
| `computeChunkDelta(localChunks, remoteIds)` | Compare local vs remote chunks |
|
|
96
|
-
| `getRemoteChunkIds(apiClient, communityId, appId, kbId)` | Get existing chunk IDs from Pinecone |
|
|
97
|
-
| `upsertChunks(apiClient, ...)` | Push chunks to backend |
|
|
98
|
-
| `deleteStaleChunks(apiClient, ...)` | Remove deleted chunks |
|
|
99
|
-
| `getSyncStatus(apiClient, config)` | Check sync state |
|
|
100
|
-
|
|
101
|
-
**Security Note:** The CLI never communicates with Pinecone directly. All chunk operations go through the backend API, which validates metadata and manages Pinecone credentials.
|
|
58
|
+
Invoked by `descix kb corpus sync`, never called separately: chunk documents (Markdown-aware
|
|
59
|
+
section splitting, with a code-aware and a generic sliding-window strategy for non-Markdown files),
|
|
60
|
+
then upsert/delete against Pinecone via the backend API — computing the delta between what the
|
|
61
|
+
manifest's git walk found and what is already indexed. **The CLI never communicates with Pinecone
|
|
62
|
+
directly.** All chunk operations go through the backend API, which validates metadata and manages
|
|
63
|
+
Pinecone credentials.
|
|
102
64
|
|
|
103
65
|
---
|
|
104
66
|
|
|
@@ -112,15 +74,15 @@ All core SDK logic resides in `DeSciX_Core/descix-cli/lib/core/`.
|
|
|
112
74
|
│ ├── icon.png
|
|
113
75
|
│ ├── app_description.md
|
|
114
76
|
│ └── system_instructions.md
|
|
115
|
-
├──
|
|
116
|
-
│
|
|
77
|
+
├── .descix/
|
|
78
|
+
│ └── manifests/
|
|
79
|
+
│ └── General.json # Corpus manifest — names the KB's git-tracked sources
|
|
80
|
+
├── kb/ # OPTIONAL — content can live anywhere; kb/ is a common convention
|
|
81
|
+
│ ├── staging/ # (if using Drive) Local files to push to Drive
|
|
117
82
|
│ │ └── research.pdf
|
|
118
|
-
│
|
|
119
|
-
│
|
|
120
|
-
│
|
|
121
|
-
│ └── chunks/ # Processed JSON chunks (Git-tracked)
|
|
122
|
-
│ ├── research.chunks.json
|
|
123
|
-
│ └── notes.chunks.json
|
|
83
|
+
│ └── General/ # (if using Drive) Text-converted mirror of Drive
|
|
84
|
+
│ ├── research.md # Converted from PDF
|
|
85
|
+
│ └── notes.md # Converted from Google Doc
|
|
124
86
|
├── site/ # Static site files (push-only)
|
|
125
87
|
│ ├── index.html
|
|
126
88
|
│ └── DeSciXAppSDK.js # The DeSciX bridge (generated)
|
|
@@ -129,14 +91,17 @@ All core SDK logic resides in `DeSciX_Core/descix-cli/lib/core/`.
|
|
|
129
91
|
└── services/ # Service code — there is no src/
|
|
130
92
|
```
|
|
131
93
|
|
|
94
|
+
Chunking happens in memory during `descix kb corpus sync` — there is no persisted `kb/chunks/`
|
|
95
|
+
folder of JSON chunk files to manage.
|
|
96
|
+
|
|
132
97
|
### Folder Purposes
|
|
133
98
|
|
|
134
99
|
| Folder | Sync Direction | Purpose |
|
|
135
100
|
|--------|----------------|---------|
|
|
136
101
|
| `assets/` | Bidirectional | App metadata (icon, description, instructions) |
|
|
137
|
-
| `
|
|
138
|
-
| `kb/
|
|
139
|
-
| `kb/
|
|
102
|
+
| `.descix/manifests/<KB>.json` | Local, git-tracked | Names the sources `descix kb corpus sync` walks — REQUIRED for a KB to sync |
|
|
103
|
+
| `kb/staging/` | Local → Drive (optional) | Raw files waiting to be pushed via `descix drive push` |
|
|
104
|
+
| `kb/General/` | Drive → Local (optional) | Text-converted files from `descix drive pull` |
|
|
140
105
|
| `site/` | Local → GCS | Static site deployment |
|
|
141
106
|
| `microservice/` | Local → GCS | Backend service deployment |
|
|
142
107
|
|
|
@@ -206,8 +171,7 @@ The DeSciX CLI publishes content with one verb per plane. Each command resolves
|
|
|
206
171
|
| `descix site upload` | Deploy site to GCS |
|
|
207
172
|
| `descix app set-site -a <app_id> --port <port>` | Register the local dev-server port for the app's site (`env.products[].site.port`) |
|
|
208
173
|
| `descix microservice init` | Copy microservice template to current app's `microservice/` folder |
|
|
209
|
-
| `descix microservice register` | Register microservice
|
|
210
|
-
| `descix microservice vectorize` | Vectorize README for discovery |
|
|
174
|
+
| `descix microservice register -r SERVICE_README.md` | Register microservice AND vectorize the README for `tell_me_how` discovery, in one step |
|
|
211
175
|
|
|
212
176
|
**Note:** Site and microservice commands are Git-based code operations. Drive templates contain content (assets, KB), while site/microservice commands handle code (HTML, JS, Dockerfile, etc.).
|
|
213
177
|
|
|
@@ -284,47 +248,49 @@ CLI → kb_sync_chunks API → pineconeService.upsertChunkRecords() → Pinecone
|
|
|
284
248
|
|
|
285
249
|
## 6. Data Flows
|
|
286
250
|
|
|
287
|
-
###
|
|
251
|
+
### Publish Cycle
|
|
288
252
|
|
|
289
253
|
```mermaid
|
|
290
254
|
flowchart LR
|
|
291
|
-
subgraph
|
|
292
|
-
|
|
293
|
-
general[kb/General/]
|
|
294
|
-
chunks[kb/chunks/]
|
|
255
|
+
subgraph drive [Google Drive — optional authoring]
|
|
256
|
+
driveKb[Drive folder]
|
|
295
257
|
end
|
|
296
|
-
|
|
297
|
-
subgraph
|
|
298
|
-
|
|
258
|
+
|
|
259
|
+
subgraph local [Local Git repo]
|
|
260
|
+
staging[kb/staging/ — optional]
|
|
261
|
+
docs[docs (or any git-tracked path)]
|
|
262
|
+
manifest[".descix/manifests/<KB>.json"]
|
|
299
263
|
end
|
|
300
|
-
|
|
264
|
+
|
|
301
265
|
subgraph backend [Backend]
|
|
302
266
|
api[kb_sync_chunks]
|
|
303
267
|
pinecone[Pinecone]
|
|
304
268
|
end
|
|
305
|
-
|
|
306
|
-
staging -->|"
|
|
307
|
-
driveKb -->|"
|
|
308
|
-
|
|
309
|
-
|
|
269
|
+
|
|
270
|
+
staging -->|"descix drive push"| driveKb
|
|
271
|
+
driveKb -->|"descix drive pull (convert)"| docs
|
|
272
|
+
docs -->|"named as a source in"| manifest
|
|
273
|
+
manifest -->|"descix kb corpus sync (walk + chunk in memory + upsert)"| api
|
|
310
274
|
api --> pinecone
|
|
311
275
|
```
|
|
312
276
|
|
|
313
277
|
### Detailed Flow
|
|
314
278
|
|
|
315
|
-
1. **Stage:** User adds `paper.pdf` to `kb/staging/`
|
|
316
|
-
2. **Push:** `descix drive push` uploads to Drive
|
|
317
|
-
3. **Pull:** `descix drive pull` downloads and converts:
|
|
279
|
+
1. **(Optional) Stage:** User adds `paper.pdf` to `kb/staging/`
|
|
280
|
+
2. **(Optional) Push:** `descix drive push` uploads to a Drive folder
|
|
281
|
+
3. **(Optional) Pull:** `descix drive pull` downloads and converts:
|
|
318
282
|
- Sees `paper.pdf` in Drive
|
|
319
283
|
- Copies with `convert=true` → temp Google Doc
|
|
320
|
-
- Exports as Markdown → `
|
|
284
|
+
- Exports as Markdown → local `paper.md`
|
|
321
285
|
- Deletes temp doc
|
|
322
|
-
4. **
|
|
323
|
-
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
-
|
|
327
|
-
|
|
286
|
+
4. **Commit + name:** commit `paper.md` and name its folder as a source in
|
|
287
|
+
`.descix/manifests/<KB>.json` (skip steps 1-3 entirely if the content already lives in git —
|
|
288
|
+
Drive is optional)
|
|
289
|
+
5. **Sync:** `descix kb corpus sync` does the rest in one pass:
|
|
290
|
+
- Walks the manifest's sources at a git ref, chunking each file in memory (nothing is written
|
|
291
|
+
to disk)
|
|
292
|
+
- Pushes changed chunks to the backend, which validates metadata and forwards to Pinecone
|
|
293
|
+
- Purges any chunk that is live in Pinecone but no longer in the manifest's walk
|
|
328
294
|
|
|
329
295
|
---
|
|
330
296
|
|
|
@@ -352,49 +318,20 @@ CLI authenticates to backend via device login:
|
|
|
352
318
|
|
|
353
319
|
---
|
|
354
320
|
|
|
355
|
-
## 8.
|
|
356
|
-
|
|
357
|
-
### From `kb/src/` to `kb/staging/`
|
|
358
|
-
|
|
359
|
-
The v2 architecture renamed `kb/src/` to `kb/staging/` for clarity. The Hydrator includes automatic migration:
|
|
360
|
-
|
|
361
|
-
```javascript
|
|
362
|
-
// If old kb/src/ exists, rename to kb/staging/
|
|
363
|
-
const oldSrcDir = path.join(kbBaseDir, 'src');
|
|
364
|
-
if (await exists(oldSrcDir)) {
|
|
365
|
-
await rename(oldSrcDir, stagingDir);
|
|
366
|
-
}
|
|
367
|
-
```
|
|
368
|
-
|
|
369
|
-
### Git Mode vs Drive Mode
|
|
321
|
+
## 8. Git Mode vs Drive Mode
|
|
370
322
|
|
|
371
323
|
Apps operate in one of two modes, determined by how they are managed:
|
|
372
324
|
|
|
373
325
|
| Mode | Source of Truth | Versioning | User Type | Tool |
|
|
374
326
|
|------|-----------------|------------|-----------|------|
|
|
375
|
-
| **git** | Local Git repo | Git | Developers | CLI (`descix kb
|
|
327
|
+
| **git** | Local Git repo | Git | Developers | CLI (`descix kb corpus sync`, `descix drive pull/push`) |
|
|
376
328
|
| **drive** | Google Drive | GCS/Firestore | Non-developers | PWA only |
|
|
377
329
|
|
|
378
330
|
**Important:** The CLI only supports **git-mode** operations. Drive-mode apps are managed entirely through the PWA with server-side processing. There is no CLI flag to switch modes - if you're using the CLI, you're in git-mode.
|
|
379
331
|
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
- **
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
## 9. File References
|
|
387
|
-
|
|
388
|
-
| Component | Path | Description |
|
|
389
|
-
|-----------|------|-------------|
|
|
390
|
-
| WorkspaceConfig | `DeSciX_Core/descix-cli/lib/workspace-config.js` | Sole configuration class |
|
|
391
|
-
| GlobalConfig | `DeSciX_Core/descix-cli/lib/global-config.js` | User-level settings (~/.descix) |
|
|
392
|
-
| Hydrator | `DeSciX_Core/descix-cli/lib/core/Hydrator.js` | Drive sync operations |
|
|
393
|
-
| Chunker | `DeSciX_Core/descix-cli/lib/core/Chunker.js` | KB chunking |
|
|
394
|
-
| Syncer | `DeSciX_Core/descix-cli/lib/core/Syncer.js` | Pinecone sync |
|
|
395
|
-
| KB Commands | `DeSciX_Core/descix-cli/lib/commands/kb.js` | Git-mode KB processing |
|
|
396
|
-
| Setup Command | `DeSciX_Core/descix-cli/lib/wizard/setup.js` | Initial workspace setup |
|
|
397
|
-
| Drive ADC | `DeSciX_Core/descix-cli/lib/google-storage-adc.js` | Google Drive API wrapper |
|
|
398
|
-
| Backend KB API | `DeSciX_Cloud/microservice/services/commandHandlers/appCommands.js` | Server-side processing |
|
|
399
|
-
| Pinecone Service | `DeSciX_Cloud/microservice/services/pineconeService.js` | Vector database |
|
|
400
|
-
| Server Chunking | `DeSciX_Cloud/microservice/services/chunkingUtils.js` | Drive-mode chunking |
|
|
332
|
+
Git mode does not require Drive at all: content can be authored directly in the repo. Drive is one
|
|
333
|
+
optional way to bring raw content (PDFs, images) in as converted Markdown, via `descix drive pull`.
|
|
334
|
+
- **Git mode**: developer commits content (optionally pulled from Drive first), names it in a
|
|
335
|
+
corpus manifest, and `descix kb corpus sync` publishes it
|
|
336
|
+
- **Drive mode**: PWA triggers a server-side pipeline (Drive → GCS → Pinecone) automatically; the
|
|
337
|
+
CLI never triggers it
|
|
@@ -169,9 +169,11 @@ the survey outranks the paper for exactly the reader who does not yet know the p
|
|
|
169
169
|
|
|
170
170
|
**There is no port-derived step for the API.** An environment NAME never becomes an origin, and
|
|
171
171
|
`env.platform.microservice.port` is not consulted: a local backend is a URL you name
|
|
172
|
-
(`descix config set-env dev --url https://localhost:4000`, or `env.apiUrl` direct).
|
|
173
|
-
|
|
174
|
-
|
|
172
|
+
(`descix config set-env dev --url https://localhost:4000`, or `env.apiUrl` direct). **This is
|
|
173
|
+
platform-dev mode** — running your own local backend (Scenario 2 below), not part of standard app
|
|
174
|
+
development, which targets a hosted environment (`descix config set-env dev` with no `--url`, or
|
|
175
|
+
`demo`/`prod`). The single owner of this order is `lib/origin.js::PRECEDENCE`; this table is a
|
|
176
|
+
description of it, not a second copy of the rule.
|
|
175
177
|
|
|
176
178
|
**Every resolution carries its SOURCE, and every network-bound command prints it on stderr** —
|
|
177
179
|
always, not only when it lands on the default:
|
|
@@ -250,83 +252,3 @@ human it is unverified.
|
|
|
250
252
|
* **The credential is a body field**, not a cookie and not a header: every call is a relative `POST /apifront/` with the token in the JSON body. There is no `SameSite` problem, no credentialed CORS preflight, and the browser origin is irrelevant to the mesh — same-origin is required for *frame scripting*, not for auth.
|
|
251
253
|
* **Set your framework's base path to `/p/{appId}`.** The gateway does not rewrite paths, so assets requested at `/` will 404.
|
|
252
254
|
|
|
253
|
-
## 6. Running from a git worktree
|
|
254
|
-
|
|
255
|
-
`.descix/workspace.json` is gitignored, so no worktree has one. The gateway walks **up** from cwd to find it, so an in-repo worktree silently inherits the **main checkout's** workspace — its port, its products, its `localPath`s. Copying the file into a worktree does not help: a saved workspace bakes an absolute `workspaceRoot` and prefers it over the discovered root, pointing back at the original checkout. Out-of-repo worktrees fail earlier and more confusingly: a credential-loading command reports **"Authentication required… Run: descix login"** when the real cause is a missing workspace file.
|
|
256
|
-
|
|
257
|
-
Rules of thumb: run the CLI from the canonical checkout, and treat an auth error raised from a worktree as a workspace-resolution suspect *before* re-authenticating.
|
|
258
|
-
|
|
259
|
-
### Deploying: probe the credential the action actually uses
|
|
260
|
-
|
|
261
|
-
`deploy-env.sh` will refuse to start rather than fail halfway:
|
|
262
|
-
|
|
263
|
-
```
|
|
264
|
-
gcloud credentials are NOT usable (expired user session, or no active account)...
|
|
265
|
-
Refusing to start a real deploy that would fail partway through.
|
|
266
|
-
```
|
|
267
|
-
|
|
268
|
-
That refusal is the script working correctly, and **an expired gcloud user session alongside live
|
|
269
|
-
ADC is a normal steady state**, not a broken machine. The two are different credentials behind one
|
|
270
|
-
word: the deploy path needs the **user session**, while most quick probes reach for **ADC**.
|
|
271
|
-
|
|
272
|
-
This is the trap: `gcloud auth application-default print-access-token` returns a token, you record
|
|
273
|
-
"gcloud is fine", and that observation is *true and irrelevant*. **Probe the credential the action
|
|
274
|
-
actually uses, not an adjacent one that happens to answer.**
|
|
275
|
-
|
|
276
|
-
The remedy is one line, needs no re-auth prompt, and the script prints it itself:
|
|
277
|
-
|
|
278
|
-
```bash
|
|
279
|
-
export CLOUDSDK_AUTH_ACCESS_TOKEN="$(gcloud auth application-default print-access-token)"
|
|
280
|
-
[ -n "$CLOUDSDK_AUTH_ACCESS_TOKEN" ] || { echo "empty token"; exit 1; }
|
|
281
|
-
```
|
|
282
|
-
|
|
283
|
-
Assert it non-empty before launching — an empty token reproduces the same refusal one layer later,
|
|
284
|
-
where it is harder to recognise.
|
|
285
|
-
|
|
286
|
-
**Why this is written down rather than left to be rediscovered:** without it, the next agent reads
|
|
287
|
-
a correct fail-loud refusal as *platform unreachable* and stops under the platform-reachability
|
|
288
|
-
rule — halting a lane and escalating, over a one-line export the script had already told it. A
|
|
289
|
-
fail-loud message nobody knows how to act on still costs a lane.
|
|
290
|
-
|
|
291
|
-
### Exit codes from wrappers are not evidence
|
|
292
|
-
|
|
293
|
-
A deploy wrapper ending in `| tail` returns **tail's** status, not the deploy's. A run can report
|
|
294
|
-
exit 0 while the deploy correctly returned 1. Capture the real status *before* anything pipes it:
|
|
295
|
-
|
|
296
|
-
```bash
|
|
297
|
-
deploy-env.sh dev; rc=$? # capture FIRST
|
|
298
|
-
deploy-env.sh dev 2>&1 | tail -40 # then look at output separately
|
|
299
|
-
```
|
|
300
|
-
|
|
301
|
-
To confirm a deploy succeeded, **read the log, not the exit code** — and where the artifact is
|
|
302
|
-
observable (a served object's timestamp, a string in the shipped bundle), check that instead. A
|
|
303
|
-
pipe silently drops information in both directions: `| tail` hides a failure, and `| head` hides
|
|
304
|
-
later matches from a search that looks complete.
|
|
305
|
-
|
|
306
|
-
### The SDK you edit is not always the SDK you serve
|
|
307
|
-
|
|
308
|
-
A second, sharper worktree trap, and it is silent. `DeSciX_Cloud/site/node_modules/@descix/app-sdk`
|
|
309
|
-
is a **symlink into the canonical Core checkout**:
|
|
310
|
-
|
|
311
|
-
```
|
|
312
|
-
$ readlink DeSciX_Cloud/site/node_modules/@descix/app-sdk
|
|
313
|
-
../../../../DeSciX_Core/descix-app-sdk
|
|
314
|
-
```
|
|
315
|
-
|
|
316
|
-
and `@descix/app-sdk` has **no build step** — its `main` and its `.` export both point straight at
|
|
317
|
-
`./src/index.js`, so Vite serves the package source as-is. Put those two facts together: **`descix
|
|
318
|
-
serve` serves whatever the CANONICAL Core working tree contains**, no matter which worktree you are
|
|
319
|
-
editing in. Change `app-sdk` on a branch, reload, and you are still running canonical `main` — your
|
|
320
|
-
edit is not in the page, nothing errors, and the obvious conclusion ("my change had no effect") is
|
|
321
|
-
wrong.
|
|
322
|
-
|
|
323
|
-
Verify what is actually being served before you conclude anything about a change:
|
|
324
|
-
|
|
325
|
-
```bash
|
|
326
|
-
readlink DeSciX_Cloud/site/node_modules/@descix/app-sdk # where the symlink points
|
|
327
|
-
node -p "require.resolve('@descix/app-sdk')" # from the site directory
|
|
328
|
-
```
|
|
329
|
-
|
|
330
|
-
If it resolves to the canonical checkout and you are editing elsewhere, you are measuring the wrong
|
|
331
|
-
tree. Re-point the symlink at your worktree for the measurement **and restore it afterwards** — it
|
|
332
|
-
is untracked, so nothing will remind you.
|