newo 3.7.1 → 3.7.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +25 -1
- package/README.md +24 -0
- package/dist/api.d.ts +33 -1
- package/dist/api.js +22 -0
- package/dist/domain/strategies/sync/ProjectSyncStrategy.d.ts +7 -0
- package/dist/domain/strategies/sync/ProjectSyncStrategy.js +81 -1
- package/dist/domain/strategies/sync/V2ProjectSyncStrategy.d.ts +11 -0
- package/dist/domain/strategies/sync/V2ProjectSyncStrategy.js +125 -2
- package/dist/sync/flow-metadata.d.ts +67 -0
- package/dist/sync/flow-metadata.js +283 -0
- package/dist/sync/json-attr-utils.d.ts +40 -11
- package/dist/sync/json-attr-utils.js +89 -22
- package/dist/sync/push.js +65 -2
- package/dist/types.d.ts +37 -0
- package/package.json +1 -1
- package/src/api.ts +61 -0
- package/src/domain/strategies/sync/ProjectSyncStrategy.ts +100 -0
- package/src/domain/strategies/sync/V2ProjectSyncStrategy.ts +147 -0
- package/src/sync/flow-metadata.ts +345 -0
- package/src/sync/json-attr-utils.ts +84 -19
- package/src/sync/push.ts +75 -2
- package/src/types.ts +40 -0
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [3.7.3] - 2026-05-25
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- **Workflow Builder canvas blank-screen, take two** - JSON-typed attribute values that contain Markdown body text with `\_` escapes (Builder uses backslash-underscore to escape underscores) and/or structural newlines from pretty-printed JSON no longer corrupt the canvas on push. Two bugs were leaking through v3.7.1's STRING-coercion fix: (a) `yaml.dump` of a pretty-printed JSON string emitted a double-quoted scalar with `\n` escape sequences, which `patchYamlToPyyaml` then rewrote as a single-quoted scalar where `\n` became two literal chars - Builder's `JSON.parse` then choked on backslash-n as structural whitespace; (b) `\_` is not a valid JSON escape per RFC 8259 and Chrome V8's `JSON.parse` throws on it, silently blanking the Builder. Fix: `normalizeJsonValueForStorage` now strips invalid escape sequences via a quote-aware walker (`fixInvalidJsonEscapes`), then compacts via `JSON.parse` + `JSON.stringify` so the value is a single-line string that survives `yaml.dump` -> `patchYamlToPyyaml` without escape-sequence corruption. Existing pretty-printed canvases in `attributes.yaml` will be reformatted to compact form on next pull (one-time stylistic diff, no semantic loss). Change-detection (`jsonValuesEqual`) continues to canonicalize both sides so the new compact form does not trigger spurious pushes against remotes that may still be pretty. Reported by Bob in [#7](https://github.com/sabbah13/newo-cli/pull/7); 25 regression tests in `test/json-attribute-roundtrip.test.js` covering both bug families and the full pull -> dump -> patch -> reload -> `JSON.parse` pipeline.
|
|
15
|
+
|
|
16
|
+
### Added
|
|
17
|
+
|
|
18
|
+
- **`fixInvalidJsonEscapes(s)`** in `src/sync/json-attr-utils.ts` - quote-aware walker that strips invalid JSON escape sequences (`\_`, `\.`, etc.) inside JSON string values per RFC 8259. Preserves all valid escapes (`\" \\ \/ \b \f \n \r \t \uXXXX`) and structural characters outside strings.
|
|
19
|
+
|
|
20
|
+
## [3.7.2] - 2026-05-17
|
|
21
|
+
|
|
22
|
+
### Fixed
|
|
23
|
+
|
|
24
|
+
- **`newo push` now syncs flow metadata (title, events, state_fields) to the platform** — closes [#3](https://github.com/sabbah13/newo-cli/issues/3). Previously `newo push` only uploaded skill scripts; edits to a flow's `metadata.yaml` (V1) or `{FlowIdn}.yaml` (V2) silently never reached the platform, and events added via `newo create-event` could appear to disappear after a subsequent push because the local-side definition lagged the remote one. Push now performs hash-gated reconciliation per flow: when `metadata.yaml` changes locally it calls `PATCH /api/v1/designer/flows/{id}` for title/description/runner, then create/update/delete child events and state_fields against `/events` and `/states` so the platform mirrors local. Hash-gating is critical — flows whose `metadata.yaml` is untouched are never even compared, so stale local trees cannot accidentally wipe events created out-of-band via the Builder UI. Wired into the legacy `pushChanged` path, `ProjectSyncStrategy`, and `V2ProjectSyncStrategy`. Reported by Bob (issue #3).
|
|
25
|
+
|
|
26
|
+
### Added
|
|
27
|
+
|
|
28
|
+
- **`updateFlow`, `updateFlowEvent`, `updateFlowState`, `deleteFlowState`, `getFlow` API wrappers** in `src/api.ts` for the previously-undocumented `PATCH /api/v1/designer/flows/{id}`, `PATCH /api/v1/designer/flows/events/{id}`, `PUT /api/v1/designer/flows/states/{id}`, `DELETE /api/v1/designer/flows/states/{id}`, and `GET /api/v1/designer/flows/{id}` endpoints. Empty payload returns 500; the platform requires the full descriptor.
|
|
29
|
+
- **`src/sync/flow-metadata.ts`** module — pure-logic reconciler shared by both V1 and V2 push paths. Comparator helpers `flowEventDiffers`/`flowStateDiffers` normalize null/undefined/empty so YAML round-trips don't trigger spurious updates.
|
|
30
|
+
- **18 unit tests** in `test/flow-metadata.test.js` covering create/update/delete event flows, state CRUD in a single pass, no-op when local matches remote, and graceful error handling that records failures without aborting the loop.
|
|
31
|
+
|
|
10
32
|
## [3.7.1] - 2026-04-29
|
|
11
33
|
|
|
12
34
|
### Fixed
|
|
@@ -1039,7 +1061,9 @@ Another Item: $Price [Modifiers: modifier3]
|
|
|
1039
1061
|
- GitHub Actions CI/CD integration
|
|
1040
1062
|
- Robust authentication with token refresh
|
|
1041
1063
|
|
|
1042
|
-
[Unreleased]: https://github.com/sabbah13/newo-cli/compare/v3.7.
|
|
1064
|
+
[Unreleased]: https://github.com/sabbah13/newo-cli/compare/v3.7.3...HEAD
|
|
1065
|
+
[3.7.3]: https://github.com/sabbah13/newo-cli/compare/v3.7.2...v3.7.3
|
|
1066
|
+
[3.7.2]: https://github.com/sabbah13/newo-cli/compare/v3.7.1...v3.7.2
|
|
1043
1067
|
[3.7.1]: https://github.com/sabbah13/newo-cli/compare/v3.7.0...v3.7.1
|
|
1044
1068
|
[3.3.0]: https://github.com/sabbah13/newo-cli/compare/v3.2.0...v3.3.0
|
|
1045
1069
|
[3.2.0]: https://github.com/sabbah13/newo-cli/compare/v3.1.0...v3.2.0
|
package/README.md
CHANGED
|
@@ -8,6 +8,8 @@
|
|
|
8
8
|
**NEWO CLI** - Professional command-line tool for NEWO AI Agent development. Features **modular architecture**, **IDN-based file management**, and **comprehensive multi-customer support**.
|
|
9
9
|
|
|
10
10
|
Sync NEWO "Project → Agent → Flow → Skills" structure to local files with:
|
|
11
|
+
- 🆕 **Canvas blank-screen hardening** (v3.7.3) - JSON-typed attributes (e.g. Workflow Builder canvas) with Markdown `\_` escapes or structural newlines no longer corrupt the canvas on push ([#7](https://github.com/sabbah13/newo-cli/pull/7))
|
|
12
|
+
- 🆕 **Flow metadata sync** (v3.7.2) - `newo push` now reconciles flow title, events, and state_fields from local `metadata.yaml` to the platform (closes [#3](https://github.com/sabbah13/newo-cli/issues/3))
|
|
11
13
|
- 🆕 **Dual format support** (v3.6.0) - `cli_v1` (native) and `newo_v2` (platform compatible), auto-detected per customer
|
|
12
14
|
- 🆕 **Libraries** (v3.6.0) - Pull/push shared reusable skills across agents within a project
|
|
13
15
|
- 🆕 **Bulk export** (v3.6.0) - `newo export` downloads complete V2 ZIP from platform
|
|
@@ -153,6 +155,28 @@ NEWO_REFRESH_URL=custom_refresh_endpoint # Custom refresh endpoint
|
|
|
153
155
|
| `newo import-akb` | Import knowledge base articles | • Structured text parsing<br>• Bulk article import<br>• Validation and error reporting |
|
|
154
156
|
| `newo meta` | Get project metadata (debug) | • Project structure analysis<br>• Metadata validation |
|
|
155
157
|
|
|
158
|
+
### Flow Metadata Sync (NEW v3.7.2)
|
|
159
|
+
|
|
160
|
+
`newo push` now reconciles **flow-level metadata** — title, `events:`, and `state_fields:` — from local YAML to the platform. Before v3.7.2 push only uploaded skill scripts, so edits to a flow's `metadata.yaml` (V1) or `{FlowIdn}.yaml` (V2) silently never reached the platform; events added via `newo create-event` could appear to disappear after a pull → push cycle. Closes [#3](https://github.com/sabbah13/newo-cli/issues/3).
|
|
161
|
+
|
|
162
|
+
**How it works:**
|
|
163
|
+
|
|
164
|
+
| Local change in `metadata.yaml` | What push does |
|
|
165
|
+
|---|---|
|
|
166
|
+
| `title:` changed | `PATCH /api/v1/designer/flows/{id}` with full descriptor |
|
|
167
|
+
| New event in `events:` | `POST /api/v1/designer/flows/{id}/events` |
|
|
168
|
+
| Event field edited | `PATCH /api/v1/designer/flows/events/{eventId}` |
|
|
169
|
+
| Event removed from list | `DELETE /api/v1/designer/flows/events/{eventId}` |
|
|
170
|
+
| Same for `state_fields:` | Mirrored CRUD against `/states` |
|
|
171
|
+
|
|
172
|
+
**Safety:**
|
|
173
|
+
|
|
174
|
+
- **Hash-gated**: flows whose `metadata.yaml` SHA256 is unchanged are *not* compared against the platform. Stale local trees cannot wipe events you created via the Builder UI.
|
|
175
|
+
- **Full sync on changed flows**: when you *do* edit `metadata.yaml`, local becomes the source of truth for that flow. Events present on the platform but missing locally are deleted. To keep events created out-of-band, run `newo pull` before editing.
|
|
176
|
+
- **Push output**: changed flows print `↑ Flow <flow>: events +N/~N/-N, states +N/~N/-N` so you see exactly what synced.
|
|
177
|
+
|
|
178
|
+
**Available in:** legacy V1 push path (`pushChanged`), `ProjectSyncStrategy`, and `V2ProjectSyncStrategy`. Works the same with `newo push --format cli_v1` and `--format newo_v2`.
|
|
179
|
+
|
|
156
180
|
### Lint, Format, Check (NEW v3.7.0)
|
|
157
181
|
|
|
158
182
|
Static-analysis over DSL files, powered by [`newo-dsl-analyzer`](https://www.npmjs.com/package/newo-dsl-analyzer). Same engine that runs in the VS Code extension - no drift between editor and CI.
|
package/dist/api.d.ts
CHANGED
|
@@ -1,10 +1,32 @@
|
|
|
1
1
|
import { type AxiosInstance } from 'axios';
|
|
2
|
-
import type { ProjectMeta, Agent, Skill, FlowEvent, FlowState, AkbImportArticle, CustomerProfile, CustomerAttribute, CustomerAttributesResponse, UserPersonaResponse, UserPersona, ChatHistoryParams, ChatHistoryResponse, CreateAgentRequest, CreateAgentResponse, CreateFlowRequest, CreateFlowResponse, CreateSkillRequest, CreateSkillResponse, CreateFlowEventRequest, CreateFlowEventResponse, CreateFlowStateRequest, CreateFlowStateResponse, CreateSkillParameterRequest, CreateSkillParameterResponse, CreateCustomerAttributeRequest, CreateCustomerAttributeResponse, CreatePersonaRequest, CreatePersonaResponse, CreateProjectRequest, CreateProjectResponse, PublishFlowRequest, PublishFlowResponse, Integration, Connector, CreateSandboxPersonaRequest, CreateSandboxPersonaResponse, CreateActorRequest, CreateActorResponse, SendChatMessageRequest, ConversationActsParams, ConversationActsResponse, ScriptAction, IntegrationSetting, CreateConnectorRequest, CreateConnectorResponse, UpdateConnectorRequest, OutgoingWebhook, IncomingWebhook, PersonaSearchResponse, AkbTopicsResponse, Registry, RegistryItem, AddProjectFromRegistryRequest, CreateNewoCustomerRequest, CreateNewoCustomerResponse, LogsQueryParams, LogsResponse } from './types.js';
|
|
2
|
+
import type { ProjectMeta, Agent, Skill, FlowEvent, FlowState, AkbImportArticle, CustomerProfile, CustomerAttribute, CustomerAttributesResponse, UserPersonaResponse, UserPersona, ChatHistoryParams, ChatHistoryResponse, CreateAgentRequest, CreateAgentResponse, CreateFlowRequest, CreateFlowResponse, CreateSkillRequest, CreateSkillResponse, CreateFlowEventRequest, CreateFlowEventResponse, UpdateFlowEventRequest, CreateFlowStateRequest, CreateFlowStateResponse, UpdateFlowStateRequest, UpdateFlowRequest, CreateSkillParameterRequest, CreateSkillParameterResponse, CreateCustomerAttributeRequest, CreateCustomerAttributeResponse, CreatePersonaRequest, CreatePersonaResponse, CreateProjectRequest, CreateProjectResponse, PublishFlowRequest, PublishFlowResponse, Integration, Connector, CreateSandboxPersonaRequest, CreateSandboxPersonaResponse, CreateActorRequest, CreateActorResponse, SendChatMessageRequest, ConversationActsParams, ConversationActsResponse, ScriptAction, IntegrationSetting, CreateConnectorRequest, CreateConnectorResponse, UpdateConnectorRequest, OutgoingWebhook, IncomingWebhook, PersonaSearchResponse, AkbTopicsResponse, Registry, RegistryItem, AddProjectFromRegistryRequest, CreateNewoCustomerRequest, CreateNewoCustomerResponse, LogsQueryParams, LogsResponse } from './types.js';
|
|
3
3
|
export declare function makeClient(verbose?: boolean, token?: string): Promise<AxiosInstance>;
|
|
4
4
|
export declare function listProjects(client: AxiosInstance): Promise<ProjectMeta[]>;
|
|
5
5
|
export declare function listAgents(client: AxiosInstance, projectId: string): Promise<Agent[]>;
|
|
6
6
|
export declare function getProjectMeta(client: AxiosInstance, projectId: string): Promise<ProjectMeta>;
|
|
7
7
|
export declare function listFlowSkills(client: AxiosInstance, flowId: string): Promise<Skill[]>;
|
|
8
|
+
/**
|
|
9
|
+
* Fetch a single flow's top-level descriptor.
|
|
10
|
+
*
|
|
11
|
+
* Used by push to detect whether local metadata.yaml title/description
|
|
12
|
+
* differ from the platform before patching.
|
|
13
|
+
*/
|
|
14
|
+
export interface FlowDescriptor {
|
|
15
|
+
id: string;
|
|
16
|
+
idn: string;
|
|
17
|
+
title: string;
|
|
18
|
+
description: string | null;
|
|
19
|
+
agent_id: string;
|
|
20
|
+
default_runner_type: string;
|
|
21
|
+
default_model: {
|
|
22
|
+
provider_idn: string;
|
|
23
|
+
model_idn: string;
|
|
24
|
+
};
|
|
25
|
+
publication_datetime?: string;
|
|
26
|
+
last_change_datetime?: string;
|
|
27
|
+
creation_datetime?: string;
|
|
28
|
+
}
|
|
29
|
+
export declare function getFlow(client: AxiosInstance, flowId: string): Promise<FlowDescriptor>;
|
|
8
30
|
export declare function getSkill(client: AxiosInstance, skillId: string): Promise<Skill>;
|
|
9
31
|
export declare function updateSkill(client: AxiosInstance, skillObject: Skill): Promise<void>;
|
|
10
32
|
export declare function listFlowEvents(client: AxiosInstance, flowId: string): Promise<FlowEvent[]>;
|
|
@@ -32,8 +54,18 @@ export declare function createSkill(client: AxiosInstance, flowId: string, skill
|
|
|
32
54
|
export declare function deleteSkill(client: AxiosInstance, skillId: string): Promise<void>;
|
|
33
55
|
export declare function deleteSkillById(client: AxiosInstance, skillId: string): Promise<void>;
|
|
34
56
|
export declare function createFlowEvent(client: AxiosInstance, flowId: string, eventData: CreateFlowEventRequest): Promise<CreateFlowEventResponse>;
|
|
57
|
+
export declare function updateFlowEvent(client: AxiosInstance, eventId: string, eventData: UpdateFlowEventRequest): Promise<void>;
|
|
35
58
|
export declare function deleteFlowEvent(client: AxiosInstance, eventId: string): Promise<void>;
|
|
36
59
|
export declare function createFlowState(client: AxiosInstance, flowId: string, stateData: CreateFlowStateRequest): Promise<CreateFlowStateResponse>;
|
|
60
|
+
export declare function updateFlowState(client: AxiosInstance, stateId: string, stateData: UpdateFlowStateRequest): Promise<void>;
|
|
61
|
+
export declare function deleteFlowState(client: AxiosInstance, stateId: string): Promise<void>;
|
|
62
|
+
/**
|
|
63
|
+
* Update flow metadata (title, description, runner type, default model).
|
|
64
|
+
*
|
|
65
|
+
* Uses PATCH /api/v1/designer/flows/{flowId}. The platform requires the
|
|
66
|
+
* full flow descriptor; sending a partial body returns 500.
|
|
67
|
+
*/
|
|
68
|
+
export declare function updateFlow(client: AxiosInstance, flowId: string, flowData: UpdateFlowRequest): Promise<void>;
|
|
37
69
|
export declare function createSkillParameter(client: AxiosInstance, skillId: string, paramData: CreateSkillParameterRequest): Promise<CreateSkillParameterResponse>;
|
|
38
70
|
export declare function createCustomerAttribute(client: AxiosInstance, attributeData: CreateCustomerAttributeRequest): Promise<CreateCustomerAttributeResponse>;
|
|
39
71
|
export declare function createProject(client: AxiosInstance, projectData: CreateProjectRequest): Promise<CreateProjectResponse>;
|
package/dist/api.js
CHANGED
|
@@ -77,6 +77,10 @@ export async function listFlowSkills(client, flowId) {
|
|
|
77
77
|
const response = await client.get(`/api/v1/designer/flows/${flowId}/skills`);
|
|
78
78
|
return response.data;
|
|
79
79
|
}
|
|
80
|
+
export async function getFlow(client, flowId) {
|
|
81
|
+
const response = await client.get(`/api/v1/designer/flows/${flowId}`);
|
|
82
|
+
return response.data;
|
|
83
|
+
}
|
|
80
84
|
export async function getSkill(client, skillId) {
|
|
81
85
|
const response = await client.get(`/api/v1/designer/skills/${skillId}`);
|
|
82
86
|
return response.data;
|
|
@@ -221,6 +225,9 @@ export async function createFlowEvent(client, flowId, eventData) {
|
|
|
221
225
|
const response = await client.post(`/api/v1/designer/flows/${flowId}/events`, eventData);
|
|
222
226
|
return response.data;
|
|
223
227
|
}
|
|
228
|
+
export async function updateFlowEvent(client, eventId, eventData) {
|
|
229
|
+
await client.patch(`/api/v1/designer/flows/events/${eventId}`, eventData);
|
|
230
|
+
}
|
|
224
231
|
export async function deleteFlowEvent(client, eventId) {
|
|
225
232
|
await client.delete(`/api/v1/designer/flows/events/${eventId}`);
|
|
226
233
|
}
|
|
@@ -228,6 +235,21 @@ export async function createFlowState(client, flowId, stateData) {
|
|
|
228
235
|
const response = await client.post(`/api/v1/designer/flows/${flowId}/states`, stateData);
|
|
229
236
|
return response.data;
|
|
230
237
|
}
|
|
238
|
+
export async function updateFlowState(client, stateId, stateData) {
|
|
239
|
+
await client.put(`/api/v1/designer/flows/states/${stateId}`, stateData);
|
|
240
|
+
}
|
|
241
|
+
export async function deleteFlowState(client, stateId) {
|
|
242
|
+
await client.delete(`/api/v1/designer/flows/states/${stateId}`);
|
|
243
|
+
}
|
|
244
|
+
/**
|
|
245
|
+
* Update flow metadata (title, description, runner type, default model).
|
|
246
|
+
*
|
|
247
|
+
* Uses PATCH /api/v1/designer/flows/{flowId}. The platform requires the
|
|
248
|
+
* full flow descriptor; sending a partial body returns 500.
|
|
249
|
+
*/
|
|
250
|
+
export async function updateFlow(client, flowId, flowData) {
|
|
251
|
+
await client.patch(`/api/v1/designer/flows/${flowId}`, flowData);
|
|
252
|
+
}
|
|
231
253
|
export async function createSkillParameter(client, skillId, paramData) {
|
|
232
254
|
// Debug the parameter creation request
|
|
233
255
|
console.log('Creating parameter for skill:', skillId);
|
|
@@ -75,6 +75,13 @@ export declare class ProjectSyncStrategy implements ISyncStrategy<ProjectMeta, L
|
|
|
75
75
|
* Push changed projects to NEWO platform
|
|
76
76
|
*/
|
|
77
77
|
push(customer: CustomerConfig, changes?: ChangeItem<LocalProjectData>[]): Promise<PushResult>;
|
|
78
|
+
/**
|
|
79
|
+
* Push a flow metadata.yaml change — syncs title, events, and state_fields
|
|
80
|
+
* to the platform. Closes GH issue #3 (events/title silently un-synced).
|
|
81
|
+
*
|
|
82
|
+
* Path shape: newo_customers/{customer}/projects/{project}/{agent}/{flow}/metadata.yaml
|
|
83
|
+
*/
|
|
84
|
+
private pushFlowMetadataUpdate;
|
|
78
85
|
/**
|
|
79
86
|
* Push a skill update
|
|
80
87
|
*/
|
|
@@ -12,7 +12,8 @@
|
|
|
12
12
|
*/
|
|
13
13
|
import fs from 'fs-extra';
|
|
14
14
|
import yaml from 'js-yaml';
|
|
15
|
-
import { listProjects, listAgents, listFlowSkills, listFlowEvents, listFlowStates, updateSkill, publishFlow, listLibraries, updateLibrarySkill, } from '../../../api.js';
|
|
15
|
+
import { listProjects, listAgents, listFlowSkills, listFlowEvents, listFlowStates, updateSkill, publishFlow, listLibraries, updateLibrarySkill, getFlow, } from '../../../api.js';
|
|
16
|
+
import { syncFlowMetadata, emptyFlowSyncCounts, totalFlowSyncOps, describeFlowSyncCounts } from '../../../sync/flow-metadata.js';
|
|
16
17
|
import { ensureState, writeFileSafe, mapPath, projectMetadataPath, agentMetadataPath, flowMetadataPath, skillMetadataPath, skillScriptPath, skillFolderPath, flowsYamlPath, customerProjectsDir, projectDir, libraryMetadataPath, librarySkillMetadataPath, librarySkillScriptPath, } from '../../../fsutil.js';
|
|
17
18
|
import { sha256, saveHashes, loadHashes } from '../../../hash.js';
|
|
18
19
|
import { generateFlowsYaml } from '../../../sync/metadata.js';
|
|
@@ -354,6 +355,23 @@ export class ProjectSyncStrategy {
|
|
|
354
355
|
for (const change of changes) {
|
|
355
356
|
try {
|
|
356
357
|
if (change.operation === 'modified') {
|
|
358
|
+
// Flow-level metadata.yaml needs different handling than a skill
|
|
359
|
+
// script: we sync title/events/state_fields rather than uploading a
|
|
360
|
+
// file. Detected by filename. (GH issue #3)
|
|
361
|
+
if (change.path.endsWith('/metadata.yaml') && !change.path.includes('/libraries/')) {
|
|
362
|
+
const pathParts = change.path.split('/');
|
|
363
|
+
// {customer}/projects/{project}/{agent}/{flow}/metadata.yaml
|
|
364
|
+
// Last 5 segments end with metadata.yaml; skip if it's a skill
|
|
365
|
+
// metadata file (one extra segment) - skill metadata is handled
|
|
366
|
+
// by V1 legacy push, not by this strategy yet.
|
|
367
|
+
const tail = pathParts.slice(-5);
|
|
368
|
+
const isFlowMeta = tail[0] === 'projects' || tail[2] && tail[4] === 'metadata.yaml';
|
|
369
|
+
if (isFlowMeta && tail.length === 5) {
|
|
370
|
+
const updateResult = await this.pushFlowMetadataUpdate(client, change, mapData, newHashes);
|
|
371
|
+
result.updated += updateResult;
|
|
372
|
+
continue;
|
|
373
|
+
}
|
|
374
|
+
}
|
|
357
375
|
const isLibrary = change.path.includes('/libraries/');
|
|
358
376
|
if (isLibrary) {
|
|
359
377
|
const updateResult = await this.pushLibrarySkillUpdate(client, change, mapData, newHashes);
|
|
@@ -381,6 +399,53 @@ export class ProjectSyncStrategy {
|
|
|
381
399
|
}
|
|
382
400
|
return result;
|
|
383
401
|
}
|
|
402
|
+
/**
|
|
403
|
+
* Push a flow metadata.yaml change — syncs title, events, and state_fields
|
|
404
|
+
* to the platform. Closes GH issue #3 (events/title silently un-synced).
|
|
405
|
+
*
|
|
406
|
+
* Path shape: newo_customers/{customer}/projects/{project}/{agent}/{flow}/metadata.yaml
|
|
407
|
+
*/
|
|
408
|
+
async pushFlowMetadataUpdate(client, change, mapData, newHashes) {
|
|
409
|
+
const pathParts = change.path.split('/');
|
|
410
|
+
// Tail: projects/{project}/{agent}/{flow}/metadata.yaml
|
|
411
|
+
const flowIdn = pathParts[pathParts.length - 2] || '';
|
|
412
|
+
const agentIdn = pathParts[pathParts.length - 3] || '';
|
|
413
|
+
const projectIdn = pathParts[pathParts.length - 4] || '';
|
|
414
|
+
const projectData = mapData.projects[projectIdn];
|
|
415
|
+
const agentData = projectData?.agents[agentIdn];
|
|
416
|
+
const flowData = agentData?.flows[flowIdn];
|
|
417
|
+
if (!flowData?.id) {
|
|
418
|
+
this.logger.warn(`Flow metadata change but flow not in project map: ${projectIdn}/${agentIdn}/${flowIdn}`);
|
|
419
|
+
return 0;
|
|
420
|
+
}
|
|
421
|
+
const content = await fs.readFile(change.path, 'utf8');
|
|
422
|
+
let localFlow;
|
|
423
|
+
try {
|
|
424
|
+
localFlow = yaml.load(content);
|
|
425
|
+
}
|
|
426
|
+
catch (error) {
|
|
427
|
+
throw new Error(`Failed to parse ${change.path}: ${error instanceof Error ? error.message : String(error)}`);
|
|
428
|
+
}
|
|
429
|
+
let remoteFlow = null;
|
|
430
|
+
try {
|
|
431
|
+
remoteFlow = await getFlow(client, flowData.id);
|
|
432
|
+
}
|
|
433
|
+
catch (error) {
|
|
434
|
+
this.logger.verbose(`Could not GET flow ${flowIdn}: ${error.response?.status ?? error.message}`);
|
|
435
|
+
}
|
|
436
|
+
const counts = emptyFlowSyncCounts();
|
|
437
|
+
await syncFlowMetadata(client, flowData.id, localFlow, remoteFlow, false, counts);
|
|
438
|
+
const total = totalFlowSyncOps(counts);
|
|
439
|
+
if (total > 0) {
|
|
440
|
+
this.logger.info(`↑ Flow ${flowIdn}: ${describeFlowSyncCounts(counts)}`);
|
|
441
|
+
}
|
|
442
|
+
for (const err of counts.errors) {
|
|
443
|
+
this.logger.warn(err);
|
|
444
|
+
}
|
|
445
|
+
// Stamp the hash so the next push skips this file unless it changes again.
|
|
446
|
+
newHashes[change.path] = sha256(content);
|
|
447
|
+
return total;
|
|
448
|
+
}
|
|
384
449
|
/**
|
|
385
450
|
* Push a skill update
|
|
386
451
|
*/
|
|
@@ -488,6 +553,21 @@ export class ProjectSyncStrategy {
|
|
|
488
553
|
for (const [projectIdn, projectData] of Object.entries(mapData.projects)) {
|
|
489
554
|
for (const [agentIdn, agentData] of Object.entries(projectData.agents)) {
|
|
490
555
|
for (const [flowIdn, flowData] of Object.entries(agentData.flows)) {
|
|
556
|
+
// Flow metadata change detection (title/events/state_fields).
|
|
557
|
+
// Surfaced as a change so push() picks it up alongside skill edits.
|
|
558
|
+
const flowMetaPath = flowMetadataPath(customer.idn, projectIdn, agentIdn, flowIdn);
|
|
559
|
+
if (await fs.pathExists(flowMetaPath)) {
|
|
560
|
+
const content = await fs.readFile(flowMetaPath, 'utf8');
|
|
561
|
+
const currentHash = sha256(content);
|
|
562
|
+
const storedHash = hashes[flowMetaPath];
|
|
563
|
+
if (storedHash !== currentHash) {
|
|
564
|
+
changes.push({
|
|
565
|
+
item: {},
|
|
566
|
+
operation: 'modified',
|
|
567
|
+
path: flowMetaPath
|
|
568
|
+
});
|
|
569
|
+
}
|
|
570
|
+
}
|
|
491
571
|
for (const [skillIdn, _skillData] of Object.entries(flowData.skills)) {
|
|
492
572
|
const skillFile = await getSingleSkillFile(customer.idn, projectIdn, agentIdn, flowIdn, skillIdn);
|
|
493
573
|
if (skillFile) {
|
|
@@ -54,6 +54,17 @@ export declare class V2ProjectSyncStrategy implements ISyncStrategy<ProjectMeta,
|
|
|
54
54
|
*/
|
|
55
55
|
private pullLibrary;
|
|
56
56
|
push(customer: CustomerConfig, changes?: ChangeItem<LocalProjectData>[]): Promise<PushResult>;
|
|
57
|
+
/**
|
|
58
|
+
* Recognize the V2 flow YAML location: .../agents/{agent}/flows/{flow}/{flow}.yaml
|
|
59
|
+
* Distinguishes it from skill scripts, library YAMLs, and attribute files.
|
|
60
|
+
*/
|
|
61
|
+
private isV2FlowYamlPath;
|
|
62
|
+
/**
|
|
63
|
+
* Push V2 flow YAML changes. Closes GH issue #3 for newo_v2 layout.
|
|
64
|
+
* Parses the V2 YAML, converts to V1-shaped FlowMetadata, and reuses the
|
|
65
|
+
* shared syncFlowMetadata routine that calls PATCH/POST/DELETE per child.
|
|
66
|
+
*/
|
|
67
|
+
private pushV2FlowYamlUpdate;
|
|
57
68
|
/**
|
|
58
69
|
* Push a V2 skill update
|
|
59
70
|
*
|
|
@@ -14,12 +14,13 @@
|
|
|
14
14
|
* skills/{SkillIdn}.nsl|.nslg
|
|
15
15
|
*/
|
|
16
16
|
import fs from 'fs-extra';
|
|
17
|
-
import { listProjects, listAgents, listFlowSkills, listFlowEvents, listFlowStates, updateSkill, publishFlow, getProjectAttributes, getCustomerAttributes, listLibraries, updateLibrarySkill, } from '../../../api.js';
|
|
17
|
+
import { listProjects, listAgents, listFlowSkills, listFlowEvents, listFlowStates, updateSkill, publishFlow, getProjectAttributes, getCustomerAttributes, listLibraries, updateLibrarySkill, getFlow, } from '../../../api.js';
|
|
18
|
+
import { syncFlowMetadata, emptyFlowSyncCounts, totalFlowSyncOps, describeFlowSyncCounts } from '../../../sync/flow-metadata.js';
|
|
18
19
|
import { ensureStateOnly, writeFileSafe, mapPath, } from '../../../fsutil.js';
|
|
19
20
|
import { sha256, saveHashes, loadHashes } from '../../../hash.js';
|
|
20
21
|
import { v2ImportVersionPath, v2ProjectYamlPath, v2AgentYamlPath, v2FlowYamlPath, v2SkillScriptPath, v2SkillRelativePath, v2ProjectAttributesPath, v2CustomerAttributesPath, v2AkbDir, v2AkbPath, v2LibraryYamlPath, v2LibrarySkillScriptPath, v2LibrarySkillRelativePath, } from '../../../format/paths-v2.js';
|
|
21
22
|
import { V2_IMPORT_VERSION, } from '../../../format/types.js';
|
|
22
|
-
import { generateV2FlowYaml, generateV2ProjectYaml, generateV2AgentYaml, buildV2InlineSkill, buildV2FlowEvent, buildV2StateField, } from '../../../format/v2-yaml.js';
|
|
23
|
+
import { generateV2FlowYaml, generateV2ProjectYaml, generateV2AgentYaml, buildV2InlineSkill, buildV2FlowEvent, buildV2StateField, parseV2FlowYaml, } from '../../../format/v2-yaml.js';
|
|
23
24
|
import { isContentDifferent } from '../../../sync/skill-files.js';
|
|
24
25
|
import yaml from 'js-yaml';
|
|
25
26
|
import { patchYamlToPyyaml } from '../../../format/yaml-patch.js';
|
|
@@ -440,6 +441,14 @@ export class V2ProjectSyncStrategy {
|
|
|
440
441
|
for (const change of changes) {
|
|
441
442
|
try {
|
|
442
443
|
if (change.operation === 'modified') {
|
|
444
|
+
// V2 flow YAML: newo_customers/{cust}/{proj}/agents/{agent}/flows/{flow}/{flow}.yaml
|
|
445
|
+
// The flow YAML carries title, events, and state_fields inline, so
|
|
446
|
+
// changes there must sync to the platform like V1 metadata.yaml.
|
|
447
|
+
if (this.isV2FlowYamlPath(change.path)) {
|
|
448
|
+
const count = await this.pushV2FlowYamlUpdate(client, change, mapData, newHashes);
|
|
449
|
+
result.updated += count;
|
|
450
|
+
continue;
|
|
451
|
+
}
|
|
443
452
|
// Detect if this is a library skill or flow skill by path
|
|
444
453
|
const isLibrary = change.path.includes('/libraries/');
|
|
445
454
|
const count = isLibrary
|
|
@@ -458,6 +467,103 @@ export class V2ProjectSyncStrategy {
|
|
|
458
467
|
}
|
|
459
468
|
return result;
|
|
460
469
|
}
|
|
470
|
+
/**
|
|
471
|
+
* Recognize the V2 flow YAML location: .../agents/{agent}/flows/{flow}/{flow}.yaml
|
|
472
|
+
* Distinguishes it from skill scripts, library YAMLs, and attribute files.
|
|
473
|
+
*/
|
|
474
|
+
isV2FlowYamlPath(p) {
|
|
475
|
+
const parts = p.split('/');
|
|
476
|
+
const file = parts[parts.length - 1];
|
|
477
|
+
if (!file || !file.endsWith('.yaml'))
|
|
478
|
+
return false;
|
|
479
|
+
// .../agents/{agent}/flows/{flow}/{flow}.yaml → last 5 parts:
|
|
480
|
+
// agents, {agent}, flows, {flow}, {flow}.yaml
|
|
481
|
+
if (parts.length < 5)
|
|
482
|
+
return false;
|
|
483
|
+
const flowsKeyword = parts[parts.length - 3];
|
|
484
|
+
const agentsKeyword = parts[parts.length - 5];
|
|
485
|
+
const flowFolder = parts[parts.length - 2] || '';
|
|
486
|
+
const stem = file.slice(0, -'.yaml'.length);
|
|
487
|
+
return flowsKeyword === 'flows' && agentsKeyword === 'agents' && stem === flowFolder;
|
|
488
|
+
}
|
|
489
|
+
/**
|
|
490
|
+
* Push V2 flow YAML changes. Closes GH issue #3 for newo_v2 layout.
|
|
491
|
+
* Parses the V2 YAML, converts to V1-shaped FlowMetadata, and reuses the
|
|
492
|
+
* shared syncFlowMetadata routine that calls PATCH/POST/DELETE per child.
|
|
493
|
+
*/
|
|
494
|
+
async pushV2FlowYamlUpdate(client, change, mapData, newHashes) {
|
|
495
|
+
const parts = change.path.split('/');
|
|
496
|
+
const flowIdn = parts[parts.length - 2] || '';
|
|
497
|
+
const agentIdn = parts[parts.length - 4] || '';
|
|
498
|
+
const projectIdn = parts[parts.length - 6] || '';
|
|
499
|
+
const projectData = mapData.projects[projectIdn];
|
|
500
|
+
const agentData = projectData?.agents[agentIdn];
|
|
501
|
+
const flowData = agentData?.flows[flowIdn];
|
|
502
|
+
if (!flowData?.id) {
|
|
503
|
+
this.logger.warn(`[newo_v2] Flow YAML change but flow not in project map: ${projectIdn}/${agentIdn}/${flowIdn}`);
|
|
504
|
+
return 0;
|
|
505
|
+
}
|
|
506
|
+
const v2Flow = await parseV2FlowYaml(change.path);
|
|
507
|
+
// Convert V2 → V1-shaped FlowMetadata for the shared sync routine.
|
|
508
|
+
// V2 events lack `id` and `description`; we fill defaults so the shape
|
|
509
|
+
// matches FlowEvent[] / FlowState[] expected by syncFlowMetadata.
|
|
510
|
+
// Optional fields are omitted (not set to undefined) to satisfy
|
|
511
|
+
// exactOptionalPropertyTypes.
|
|
512
|
+
const localMeta = {
|
|
513
|
+
id: flowData.id,
|
|
514
|
+
idn: v2Flow.idn,
|
|
515
|
+
title: v2Flow.title,
|
|
516
|
+
description: v2Flow.description ?? '',
|
|
517
|
+
default_runner_type: v2Flow.default_runner_type || 'guidance',
|
|
518
|
+
default_model: {
|
|
519
|
+
provider_idn: v2Flow.default_provider_idn,
|
|
520
|
+
model_idn: v2Flow.default_model_idn,
|
|
521
|
+
},
|
|
522
|
+
events: (v2Flow.events || []).map((e) => {
|
|
523
|
+
const out = {
|
|
524
|
+
id: '',
|
|
525
|
+
idn: e.idn,
|
|
526
|
+
description: '',
|
|
527
|
+
skill_selector: e.skill_selector,
|
|
528
|
+
interrupt_mode: (e.interrupt_mode || 'queue'),
|
|
529
|
+
...(e.skill_idn != null ? { skill_idn: e.skill_idn } : {}),
|
|
530
|
+
...(e.state_idn != null ? { state_idn: e.state_idn } : {}),
|
|
531
|
+
...(e.integration_idn != null ? { integration_idn: e.integration_idn } : {}),
|
|
532
|
+
...(e.connector_idn != null ? { connector_idn: e.connector_idn } : {}),
|
|
533
|
+
};
|
|
534
|
+
return out;
|
|
535
|
+
}),
|
|
536
|
+
state_fields: (v2Flow.state_fields || []).map((s) => {
|
|
537
|
+
const out = {
|
|
538
|
+
id: '',
|
|
539
|
+
idn: s.idn,
|
|
540
|
+
title: s.title || s.idn,
|
|
541
|
+
scope: (s.scope || 'flow'),
|
|
542
|
+
...(s.default_value != null ? { default_value: s.default_value } : {}),
|
|
543
|
+
};
|
|
544
|
+
return out;
|
|
545
|
+
}),
|
|
546
|
+
};
|
|
547
|
+
let remoteFlow = null;
|
|
548
|
+
try {
|
|
549
|
+
remoteFlow = await getFlow(client, flowData.id);
|
|
550
|
+
}
|
|
551
|
+
catch (error) {
|
|
552
|
+
this.logger.verbose(`[newo_v2] Could not GET flow ${flowIdn}: ${error.response?.status ?? error.message}`);
|
|
553
|
+
}
|
|
554
|
+
const counts = emptyFlowSyncCounts();
|
|
555
|
+
await syncFlowMetadata(client, flowData.id, localMeta, remoteFlow, false, counts);
|
|
556
|
+
const total = totalFlowSyncOps(counts);
|
|
557
|
+
if (total > 0) {
|
|
558
|
+
this.logger.info(`[newo_v2] ↑ Flow ${flowIdn}: ${describeFlowSyncCounts(counts)}`);
|
|
559
|
+
}
|
|
560
|
+
for (const err of counts.errors) {
|
|
561
|
+
this.logger.warn(err);
|
|
562
|
+
}
|
|
563
|
+
const content = await fs.readFile(change.path, 'utf8');
|
|
564
|
+
newHashes[change.path] = sha256(content);
|
|
565
|
+
return total;
|
|
566
|
+
}
|
|
461
567
|
/**
|
|
462
568
|
* Push a V2 skill update
|
|
463
569
|
*
|
|
@@ -562,6 +668,23 @@ export class V2ProjectSyncStrategy {
|
|
|
562
668
|
for (const [projectIdn, projectData] of Object.entries(mapData.projects)) {
|
|
563
669
|
// Flow skills
|
|
564
670
|
for (const [agentIdn, agentData] of Object.entries(projectData.agents)) {
|
|
671
|
+
for (const [flowIdn, _flowData] of Object.entries(agentData.flows)) {
|
|
672
|
+
// V2 stores flow events / state_fields / title inline in the flow
|
|
673
|
+
// YAML. Detect changes here so push() can sync them (GH issue #3).
|
|
674
|
+
const flowYamlPath = v2FlowYamlPath(customer.idn, projectIdn, agentIdn, flowIdn);
|
|
675
|
+
if (await fs.pathExists(flowYamlPath)) {
|
|
676
|
+
const content = await fs.readFile(flowYamlPath, 'utf8');
|
|
677
|
+
const currentHash = sha256(content);
|
|
678
|
+
const storedHash = hashes[flowYamlPath];
|
|
679
|
+
if (storedHash !== currentHash) {
|
|
680
|
+
changes.push({
|
|
681
|
+
item: {},
|
|
682
|
+
operation: 'modified',
|
|
683
|
+
path: flowYamlPath
|
|
684
|
+
});
|
|
685
|
+
}
|
|
686
|
+
}
|
|
687
|
+
}
|
|
565
688
|
for (const [flowIdn, flowData] of Object.entries(agentData.flows)) {
|
|
566
689
|
for (const [skillIdn, skillMeta] of Object.entries(flowData.skills)) {
|
|
567
690
|
const scriptPath = v2SkillScriptPath(customer.idn, projectIdn, agentIdn, flowIdn, skillIdn, skillMeta.runner_type);
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Flow metadata sync — reconciles local flow metadata.yaml with the platform.
|
|
3
|
+
*
|
|
4
|
+
* Closes the gap behind GH issue #3 (push wiping flow event subscriptions and
|
|
5
|
+
* title): before this module existed, push only updated skill scripts. Local
|
|
6
|
+
* edits to flow title, events, or state_fields silently never reached the
|
|
7
|
+
* platform; new events created via `newo create-event` had no path to flow
|
|
8
|
+
* therefore disappeared from local metadata.yaml after a subsequent pull.
|
|
9
|
+
*
|
|
10
|
+
* Reconciliation strategy per flow (only runs when metadata.yaml hash changed):
|
|
11
|
+
* - Compare local FlowMetadata against fresh GET responses from the platform
|
|
12
|
+
* - Update flow title/description/runner via PATCH /api/v1/designer/flows/{id}
|
|
13
|
+
* - For each child collection (events, state_fields):
|
|
14
|
+
* • idn present locally + missing remotely → create
|
|
15
|
+
* • idn present in both, contents differ → update
|
|
16
|
+
* • idn missing locally + present remotely → delete
|
|
17
|
+
*
|
|
18
|
+
* Hash-gating is critical: if the user never touched metadata.yaml, we never
|
|
19
|
+
* compute a remote diff, which means a stale or partially-pulled tree cannot
|
|
20
|
+
* accidentally wipe events that were created out-of-band via the Builder UI.
|
|
21
|
+
*/
|
|
22
|
+
import type { AxiosInstance } from 'axios';
|
|
23
|
+
import type { FlowMetadata, FlowEvent, FlowState } from '../types.js';
|
|
24
|
+
export interface FlowMetadataSyncCounts {
|
|
25
|
+
flowsUpdated: number;
|
|
26
|
+
eventsCreated: number;
|
|
27
|
+
eventsUpdated: number;
|
|
28
|
+
eventsDeleted: number;
|
|
29
|
+
statesCreated: number;
|
|
30
|
+
statesUpdated: number;
|
|
31
|
+
statesDeleted: number;
|
|
32
|
+
errors: string[];
|
|
33
|
+
}
|
|
34
|
+
export declare function emptyFlowSyncCounts(): FlowMetadataSyncCounts;
|
|
35
|
+
/**
|
|
36
|
+
* True when remote FlowEvent fields differ from what the local metadata says.
|
|
37
|
+
* We only compare semantic fields the platform stores - `id` is platform-owned.
|
|
38
|
+
*/
|
|
39
|
+
export declare function flowEventDiffers(local: FlowEvent, remote: FlowEvent): boolean;
|
|
40
|
+
export declare function flowStateDiffers(local: FlowState, remote: FlowState): boolean;
|
|
41
|
+
/**
|
|
42
|
+
* Reconcile one flow's metadata with the platform.
|
|
43
|
+
*
|
|
44
|
+
* @param client authenticated Axios client
|
|
45
|
+
* @param flowId platform flow ID (UUID)
|
|
46
|
+
* @param local parsed FlowMetadata from the customer's local YAML
|
|
47
|
+
* @param remoteFlow flow data fetched from GET /flows/{id} - if null,
|
|
48
|
+
* flow-level updates are skipped (still syncs children).
|
|
49
|
+
* Pass null when caller already knows the GET endpoint
|
|
50
|
+
* will 404 (e.g. legacy data) or wants children-only.
|
|
51
|
+
* @param verbose when true, emits per-operation log lines
|
|
52
|
+
* @param counts shared counter mutated in place across multiple flows
|
|
53
|
+
*/
|
|
54
|
+
export declare function syncFlowMetadata(client: AxiosInstance, flowId: string, local: FlowMetadata, remoteFlow: {
|
|
55
|
+
title: string;
|
|
56
|
+
description: string | null;
|
|
57
|
+
default_runner_type?: string;
|
|
58
|
+
} | null, verbose: boolean, counts: FlowMetadataSyncCounts): Promise<void>;
|
|
59
|
+
/**
|
|
60
|
+
* Combined count of operations across all categories.
|
|
61
|
+
*/
|
|
62
|
+
export declare function totalFlowSyncOps(counts: FlowMetadataSyncCounts): number;
|
|
63
|
+
/**
|
|
64
|
+
* Human-readable summary line for the push report.
|
|
65
|
+
*/
|
|
66
|
+
export declare function describeFlowSyncCounts(counts: FlowMetadataSyncCounts): string;
|
|
67
|
+
//# sourceMappingURL=flow-metadata.d.ts.map
|