newo 3.7.2 → 3.7.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [3.7.4] - 2026-06-10
11
+
12
+ ### Added
13
+
14
+ - **V2 push now creates new skills from inline flow YAML definitions.** Previously the `newo_v2` push path could only update content of skills already known to the local project map; a skill added locally to `{FlowIdn}.yaml` after the last pull was silently ignored, leaving its `.nsl`/`.nslg` script with no way to reach the platform. `V2ProjectSyncStrategy.push()` now reconciles inline skill metadata before iterating script changes: missing remote skills are created via `POST /api/v1/designer/flows/{flowId}/skills`, race-condition duplicates (already-exists on create) fall back to fetching the remote skill via `listFlowSkills` and updating it, and missing skill parameters are filled in via `POST /api/v1/designer/flows/skills/{skillId}/parameters`. The local project map and SHA256 hash store are updated atomically after the reconciliation pass.
15
+ - **Strict model validation before V2 skill writes.** New `assertSkillModelResolved` helper throws a descriptive error ("Set either skill.model.* or flow default_model_idn/default_provider_idn") when neither the inline skill nor the flow declares a model/provider, instead of letting the platform return a generic 4xx with no hint about which YAML field is missing.
16
+ - **33 unit tests** in `test/v2-push-helpers.test.js` covering `isAlreadyExistsApiError`, `assertSkillModelResolved`, `normalizeRunnerType`, `normalizeParameters`, `skillMetadataDiffers`, `buildV2SkillMetadataFromYaml`, and `createMissingSkillParameters` (stubbed API client). Regression-locks the false-positive matcher fix, the model-validation contract, the parameter-creation counting, and the model-key-order false positive.
17
+
18
+ ### Fixed
19
+
20
+ - **`isAlreadyExistsApiError` no longer matches "does not exist".** The previous detector accepted any 400/409/422 response whose body contained the substring `"exist"`, which incorrectly classified "Skill does not exist" / "Flow doesn't exist" errors as duplicates and triggered a spurious reuse fallback. The matcher is now tightened to only `"already exists"` and `"duplicate key"`.
21
+ - **`validate()` reports missing scripts for YAML-declared skills.** Previously `V2ProjectSyncStrategy.validate()` only checked scripts listed in the local map; skills added directly to `{FlowIdn}.yaml` since the last pull were never validated. Validation now walks the YAML when present and surfaces a `Script file not found:` error per skill, matching the new push contract.
22
+ - **Missing-script skills are reported, not silently skipped.** Instead of a `verbose`-only "Skipping skill without local script" log, the push now records `[newo_v2] Missing script for skill {project}/{agent}/{flow}/{skill}: {path}` in the push errors. Failures are isolated per skill: one broken skill no longer aborts the push of every other flow/project in the workspace.
23
+ - **Model validation only runs before actual platform writes.** `assertSkillModelResolved` fires only when a skill is about to be created or its metadata updated — a pre-existing flow YAML without `skill.model.*` / flow `default_model_idn` no longer blocks pushes of unrelated, untouched skills.
24
+ - **Parameter-creation count is no longer inflated by already-existing parameters.** `createMissingSkillParameters` incremented its counter (and logged "Created skill parameter") even when the platform answered "already exists", which spuriously triggered a follow-up `updateSkill` call. Count and log now happen only on successful creation.
25
+ - **`isAlreadyExistsApiError` no longer throws on `null`/`undefined` errors** (optional chaining on `.response`), which previously masked the original failure inside catch handlers.
26
+ - **`skillMetadataDiffers` no longer flags every skill as changed.** The model comparison used `JSON.stringify`, which is key-order-sensitive: the project map stores `{provider_idn, model_idn}` (platform API order) while YAML-built metadata uses `{model_idn, provider_idn}`. Verified against a live account: every push rewrote all 1342 skills. Model is now compared field-by-field and parameters are compared order-insensitively (sorted by name).
27
+ - **Skill parameters are now created explicitly after `createSkill`.** The platform's create endpoint ignores inline `parameters` in the request body (verified against the live platform) — newly created skills silently lost their YAML-declared parameters. The create path now calls `POST /flows/skills/{skillId}/parameters` per parameter, same as the update path.
28
+
29
+ ## [3.7.3] - 2026-05-25
30
+
31
+ ### Fixed
32
+
33
+ - **Workflow Builder canvas blank-screen, take two** - JSON-typed attribute values that contain Markdown body text with `\_` escapes (Builder uses backslash-underscore to escape underscores) and/or structural newlines from pretty-printed JSON no longer corrupt the canvas on push. Two bugs were leaking through v3.7.1's STRING-coercion fix: (a) `yaml.dump` of a pretty-printed JSON string emitted a double-quoted scalar with `\n` escape sequences, which `patchYamlToPyyaml` then rewrote as a single-quoted scalar where `\n` became two literal chars - Builder's `JSON.parse` then choked on backslash-n as structural whitespace; (b) `\_` is not a valid JSON escape per RFC 8259 and Chrome V8's `JSON.parse` throws on it, silently blanking the Builder. Fix: `normalizeJsonValueForStorage` now strips invalid escape sequences via a quote-aware walker (`fixInvalidJsonEscapes`), then compacts via `JSON.parse` + `JSON.stringify` so the value is a single-line string that survives `yaml.dump` -> `patchYamlToPyyaml` without escape-sequence corruption. Existing pretty-printed canvases in `attributes.yaml` will be reformatted to compact form on next pull (one-time stylistic diff, no semantic loss). Change-detection (`jsonValuesEqual`) continues to canonicalize both sides so the new compact form does not trigger spurious pushes against remotes that may still be pretty. Reported by Bob in [#7](https://github.com/sabbah13/newo-cli/pull/7); 25 regression tests in `test/json-attribute-roundtrip.test.js` covering both bug families and the full pull -> dump -> patch -> reload -> `JSON.parse` pipeline.
34
+
35
+ ### Added
36
+
37
+ - **`fixInvalidJsonEscapes(s)`** in `src/sync/json-attr-utils.ts` - quote-aware walker that strips invalid JSON escape sequences (`\_`, `\.`, etc.) inside JSON string values per RFC 8259. Preserves all valid escapes (`\" \\ \/ \b \f \n \r \t \uXXXX`) and structural characters outside strings.
38
+
10
39
  ## [3.7.2] - 2026-05-17
11
40
 
12
41
  ### Fixed
@@ -1051,7 +1080,10 @@ Another Item: $Price [Modifiers: modifier3]
1051
1080
  - GitHub Actions CI/CD integration
1052
1081
  - Robust authentication with token refresh
1053
1082
 
1054
- [Unreleased]: https://github.com/sabbah13/newo-cli/compare/v3.7.1...HEAD
1083
+ [Unreleased]: https://github.com/sabbah13/newo-cli/compare/v3.7.4...HEAD
1084
+ [3.7.4]: https://github.com/sabbah13/newo-cli/compare/v3.7.3...v3.7.4
1085
+ [3.7.3]: https://github.com/sabbah13/newo-cli/compare/v3.7.2...v3.7.3
1086
+ [3.7.2]: https://github.com/sabbah13/newo-cli/compare/v3.7.1...v3.7.2
1055
1087
  [3.7.1]: https://github.com/sabbah13/newo-cli/compare/v3.7.0...v3.7.1
1056
1088
  [3.3.0]: https://github.com/sabbah13/newo-cli/compare/v3.2.0...v3.3.0
1057
1089
  [3.2.0]: https://github.com/sabbah13/newo-cli/compare/v3.1.0...v3.2.0
package/README.md CHANGED
@@ -8,6 +8,9 @@
8
8
  **NEWO CLI** - Professional command-line tool for NEWO AI Agent development. Features **modular architecture**, **IDN-based file management**, and **comprehensive multi-customer support**.
9
9
 
10
10
  Sync NEWO "Project → Agent → Flow → Skills" structure to local files with:
11
+ - 🆕 **V2 skill creation on push** (v3.7.4) - adding a skill inline to a `newo_v2` `{FlowIdn}.yaml` and pushing now creates it on the platform (previously only updates of existing skills worked)
12
+ - 🆕 **Canvas blank-screen hardening** (v3.7.3) - JSON-typed attributes (e.g. Workflow Builder canvas) with Markdown `\_` escapes or structural newlines no longer corrupt the canvas on push ([#7](https://github.com/sabbah13/newo-cli/pull/7))
13
+ - 🆕 **Flow metadata sync** (v3.7.2) - `newo push` now reconciles flow title, events, and state_fields from local `metadata.yaml` to the platform (closes [#3](https://github.com/sabbah13/newo-cli/issues/3))
11
14
  - 🆕 **Dual format support** (v3.6.0) - `cli_v1` (native) and `newo_v2` (platform compatible), auto-detected per customer
12
15
  - 🆕 **Libraries** (v3.6.0) - Pull/push shared reusable skills across agents within a project
13
16
  - 🆕 **Bulk export** (v3.6.0) - `newo export` downloads complete V2 ZIP from platform
@@ -153,6 +156,28 @@ NEWO_REFRESH_URL=custom_refresh_endpoint # Custom refresh endpoint
153
156
  | `newo import-akb` | Import knowledge base articles | • Structured text parsing<br>• Bulk article import<br>• Validation and error reporting |
154
157
  | `newo meta` | Get project metadata (debug) | • Project structure analysis<br>• Metadata validation |
155
158
 
159
+ ### Flow Metadata Sync (NEW v3.7.2)
160
+
161
+ `newo push` now reconciles **flow-level metadata** — title, `events:`, and `state_fields:` — from local YAML to the platform. Before v3.7.2 push only uploaded skill scripts, so edits to a flow's `metadata.yaml` (V1) or `{FlowIdn}.yaml` (V2) silently never reached the platform; events added via `newo create-event` could appear to disappear after a pull → push cycle. Closes [#3](https://github.com/sabbah13/newo-cli/issues/3).
162
+
163
+ **How it works:**
164
+
165
+ | Local change in `metadata.yaml` | What push does |
166
+ |---|---|
167
+ | `title:` changed | `PATCH /api/v1/designer/flows/{id}` with full descriptor |
168
+ | New event in `events:` | `POST /api/v1/designer/flows/{id}/events` |
169
+ | Event field edited | `PATCH /api/v1/designer/flows/events/{eventId}` |
170
+ | Event removed from list | `DELETE /api/v1/designer/flows/events/{eventId}` |
171
+ | Same for `state_fields:` | Mirrored CRUD against `/states` |
172
+
173
+ **Safety:**
174
+
175
+ - **Hash-gated**: flows whose `metadata.yaml` SHA256 is unchanged are *not* compared against the platform. Stale local trees cannot wipe events you created via the Builder UI.
176
+ - **Full sync on changed flows**: when you *do* edit `metadata.yaml`, local becomes the source of truth for that flow. Events present on the platform but missing locally are deleted. To keep events created out-of-band, run `newo pull` before editing.
177
+ - **Push output**: changed flows print `↑ Flow <flow>: events +N/~N/-N, states +N/~N/-N` so you see exactly what synced.
178
+
179
+ **Available in:** legacy V1 push path (`pushChanged`), `ProjectSyncStrategy`, and `V2ProjectSyncStrategy`. Works the same with `newo push --format cli_v1` and `--format newo_v2`.
180
+
156
181
  ### Lint, Format, Check (NEW v3.7.0)
157
182
 
158
183
  Static-analysis over DSL files, powered by [`newo-dsl-analyzer`](https://www.npmjs.com/package/newo-dsl-analyzer). Same engine that runs in the VS Code extension - no drift between editor and CI.
@@ -65,6 +65,39 @@ export declare class V2ProjectSyncStrategy implements ISyncStrategy<ProjectMeta,
65
65
  * shared syncFlowMetadata routine that calls PATCH/POST/DELETE per child.
66
66
  */
67
67
  private pushV2FlowYamlUpdate;
68
+ /**
69
+ * Reconcile inline skill definitions from V2 flow YAML before pushing scripts.
70
+ *
71
+ * V2 keeps skill metadata (model, runner_type, parameters) in the flow YAML,
72
+ * not in a separate skill metadata file. The map only contains the remote IDs
73
+ * from a previous pull, so new local skills must be created before their
74
+ * callers can be published.
75
+ */
76
+ private syncV2FlowYamlDefinitions;
77
+ private createMissingSkillParameters;
78
+ /**
79
+ * Detect "resource already exists" API errors.
80
+ *
81
+ * Matches only on the precise phrases the platform actually returns
82
+ * ("already exists", "duplicate key"). Loose substrings like "exist"
83
+ * would otherwise sweep up unrelated "does not exist" / "doesn't exist"
84
+ * errors and trigger an incorrect reuse fallback.
85
+ */
86
+ private isAlreadyExistsApiError;
87
+ private normalizeRunnerType;
88
+ private normalizeParameters;
89
+ /**
90
+ * Fail fast if no model could be resolved for a V2 skill.
91
+ *
92
+ * `buildV2SkillMetadataFromYaml` falls back to empty strings when neither
93
+ * the skill nor the flow declare a model. The platform rejects empty
94
+ * model_idn/provider_idn at creation/update time, but the error it returns
95
+ * is generic — we surface a clearer message before issuing the request.
96
+ */
97
+ private assertSkillModelResolved;
98
+ private buildV2SkillMetadataFromYaml;
99
+ private skillMetadataDiffers;
100
+ private resolveV2FlowSkillScriptPath;
68
101
  /**
69
102
  * Push a V2 skill update
70
103
  *
@@ -81,6 +114,7 @@ export declare class V2ProjectSyncStrategy implements ISyncStrategy<ProjectMeta,
81
114
  */
82
115
  private publishAllFlows;
83
116
  getChanges(customer: CustomerConfig): Promise<ChangeItem<LocalProjectData>[]>;
117
+ private loadLocalV2FlowSkills;
84
118
  validate(customer: CustomerConfig, _items: LocalProjectData[]): Promise<ValidationResult>;
85
119
  getStatus(customer: CustomerConfig): Promise<StatusSummary>;
86
120
  }
@@ -14,13 +14,13 @@
14
14
  * skills/{SkillIdn}.nsl|.nslg
15
15
  */
16
16
  import fs from 'fs-extra';
17
- import { listProjects, listAgents, listFlowSkills, listFlowEvents, listFlowStates, updateSkill, publishFlow, getProjectAttributes, getCustomerAttributes, listLibraries, updateLibrarySkill, getFlow, } from '../../../api.js';
17
+ import { listProjects, listAgents, listFlowSkills, listFlowEvents, listFlowStates, createSkill, createSkillParameter, updateSkill, publishFlow, getProjectAttributes, getCustomerAttributes, listLibraries, updateLibrarySkill, getFlow, } from '../../../api.js';
18
18
  import { syncFlowMetadata, emptyFlowSyncCounts, totalFlowSyncOps, describeFlowSyncCounts } from '../../../sync/flow-metadata.js';
19
19
  import { ensureStateOnly, writeFileSafe, mapPath, } from '../../../fsutil.js';
20
20
  import { sha256, saveHashes, loadHashes } from '../../../hash.js';
21
- import { v2ImportVersionPath, v2ProjectYamlPath, v2AgentYamlPath, v2FlowYamlPath, v2SkillScriptPath, v2SkillRelativePath, v2ProjectAttributesPath, v2CustomerAttributesPath, v2AkbDir, v2AkbPath, v2LibraryYamlPath, v2LibrarySkillScriptPath, v2LibrarySkillRelativePath, } from '../../../format/paths-v2.js';
21
+ import { v2ImportVersionPath, v2ProjectYamlPath, v2AgentDir, v2AgentYamlPath, v2FlowYamlPath, v2SkillScriptPath, v2SkillRelativePath, v2ProjectAttributesPath, v2CustomerAttributesPath, v2AkbDir, v2AkbPath, v2LibraryYamlPath, v2LibrarySkillScriptPath, v2LibrarySkillRelativePath, } from '../../../format/paths-v2.js';
22
22
  import { V2_IMPORT_VERSION, } from '../../../format/types.js';
23
- import { generateV2FlowYaml, generateV2ProjectYaml, generateV2AgentYaml, buildV2InlineSkill, buildV2FlowEvent, buildV2StateField, parseV2FlowYaml, } from '../../../format/v2-yaml.js';
23
+ import { generateV2FlowYaml, generateV2ProjectYaml, generateV2AgentYaml, parseV2FlowYaml, buildV2InlineSkill, buildV2FlowEvent, buildV2StateField, } from '../../../format/v2-yaml.js';
24
24
  import { isContentDifferent } from '../../../sync/skill-files.js';
25
25
  import yaml from 'js-yaml';
26
26
  import { patchYamlToPyyaml } from '../../../format/yaml-patch.js';
@@ -438,8 +438,15 @@ export class V2ProjectSyncStrategy {
438
438
  return result;
439
439
  }
440
440
  const mapData = await fs.readJson(mapFile);
441
+ const metadataSync = await this.syncV2FlowYamlDefinitions(client, customer, mapData, newHashes);
442
+ result.created += metadataSync.created;
443
+ result.updated += metadataSync.updated;
444
+ result.errors.push(...metadataSync.errors);
441
445
  for (const change of changes) {
442
446
  try {
447
+ if (metadataSync.syncedPaths.has(change.path)) {
448
+ continue;
449
+ }
443
450
  if (change.operation === 'modified') {
444
451
  // V2 flow YAML: newo_customers/{cust}/{proj}/agents/{agent}/flows/{flow}/{flow}.yaml
445
452
  // The flow YAML carries title, events, and state_fields inline, so
@@ -461,6 +468,9 @@ export class V2ProjectSyncStrategy {
461
468
  result.errors.push(`Failed to push ${change.path}: ${error instanceof Error ? error.message : String(error)}`);
462
469
  }
463
470
  }
471
+ if (metadataSync.created > 0 || metadataSync.updated > 0) {
472
+ await writeFileSafe(mapFile, JSON.stringify(mapData, null, 2));
473
+ }
464
474
  await saveHashes(newHashes, customer.idn);
465
475
  if (result.created > 0 || result.updated > 0) {
466
476
  await this.publishAllFlows(client, mapData);
@@ -564,6 +574,245 @@ export class V2ProjectSyncStrategy {
564
574
  newHashes[change.path] = sha256(content);
565
575
  return total;
566
576
  }
577
+ /**
578
+ * Reconcile inline skill definitions from V2 flow YAML before pushing scripts.
579
+ *
580
+ * V2 keeps skill metadata (model, runner_type, parameters) in the flow YAML,
581
+ * not in a separate skill metadata file. The map only contains the remote IDs
582
+ * from a previous pull, so new local skills must be created before their
583
+ * callers can be published.
584
+ */
585
+ async syncV2FlowYamlDefinitions(client, customer, mapData, newHashes) {
586
+ let created = 0;
587
+ let updated = 0;
588
+ const syncedPaths = new Set();
589
+ const errors = [];
590
+ for (const [projectIdn, projectData] of Object.entries(mapData.projects)) {
591
+ for (const [agentIdn, agentData] of Object.entries(projectData.agents)) {
592
+ for (const [flowIdn, flowData] of Object.entries(agentData.flows)) {
593
+ const flowYamlPath = v2FlowYamlPath(customer.idn, projectIdn, agentIdn, flowIdn);
594
+ if (!(await fs.pathExists(flowYamlPath))) {
595
+ continue;
596
+ }
597
+ let flowDef;
598
+ try {
599
+ flowDef = await parseV2FlowYaml(flowYamlPath);
600
+ }
601
+ catch (error) {
602
+ this.logger.warn(`[newo_v2] Failed to parse flow YAML ${flowYamlPath}: ${error instanceof Error ? error.message : String(error)}`);
603
+ continue;
604
+ }
605
+ for (const skill of flowDef.skills || []) {
606
+ const skillLocator = `${projectIdn}/${agentIdn}/${flowIdn}/${skill.idn}`;
607
+ // Per-skill failure isolation: one broken skill must not abort the
608
+ // push of every other project/flow in the workspace.
609
+ try {
610
+ const runnerType = this.normalizeRunnerType(skill.runner_type);
611
+ const scriptPath = await this.resolveV2FlowSkillScriptPath(customer.idn, projectIdn, agentIdn, flowIdn, skill.idn, runnerType, skill.prompt_script);
612
+ if (!(await fs.pathExists(scriptPath))) {
613
+ errors.push(`[newo_v2] Missing script for skill ${skillLocator}: ${scriptPath}`);
614
+ continue;
615
+ }
616
+ const content = await fs.readFile(scriptPath, 'utf8');
617
+ const localMetadata = this.buildV2SkillMetadataFromYaml(skill, flowDef, runnerType, flowData.skills[skill.idn]);
618
+ const existingSkill = flowData.skills[skill.idn];
619
+ if (!existingSkill) {
620
+ this.assertSkillModelResolved(localMetadata, skillLocator);
621
+ try {
622
+ const createdSkill = await createSkill(client, flowData.id, {
623
+ idn: localMetadata.idn,
624
+ title: localMetadata.title,
625
+ prompt_script: content,
626
+ runner_type: localMetadata.runner_type,
627
+ model: localMetadata.model,
628
+ parameters: localMetadata.parameters,
629
+ path: localMetadata.path || ''
630
+ });
631
+ // The create endpoint ignores inline `parameters` (verified
632
+ // against the live platform) — create them explicitly.
633
+ await this.createMissingSkillParameters(client, { ...localMetadata, id: createdSkill.id, parameters: [] }, localMetadata);
634
+ flowData.skills[skill.idn] = {
635
+ ...localMetadata,
636
+ id: createdSkill.id
637
+ };
638
+ newHashes[scriptPath] = sha256(content);
639
+ syncedPaths.add(scriptPath);
640
+ created++;
641
+ this.logger.info(`[newo_v2] Created skill: ${flowIdn}/${skill.idn}`);
642
+ }
643
+ catch (error) {
644
+ if (!this.isAlreadyExistsApiError(error)) {
645
+ throw error;
646
+ }
647
+ const remoteSkills = await listFlowSkills(client, flowData.id);
648
+ const remoteSkill = remoteSkills.find(s => s.idn === skill.idn);
649
+ if (!remoteSkill) {
650
+ throw error;
651
+ }
652
+ const remoteMetadata = {
653
+ id: remoteSkill.id,
654
+ idn: remoteSkill.idn,
655
+ title: remoteSkill.title,
656
+ runner_type: remoteSkill.runner_type,
657
+ model: remoteSkill.model,
658
+ parameters: this.normalizeParameters(remoteSkill.parameters),
659
+ path: remoteSkill.path
660
+ };
661
+ await this.createMissingSkillParameters(client, remoteMetadata, localMetadata);
662
+ await updateSkill(client, {
663
+ id: remoteSkill.id,
664
+ title: localMetadata.title,
665
+ idn: localMetadata.idn,
666
+ prompt_script: content,
667
+ runner_type: localMetadata.runner_type,
668
+ model: localMetadata.model,
669
+ parameters: localMetadata.parameters,
670
+ path: remoteSkill.path || localMetadata.path
671
+ });
672
+ flowData.skills[skill.idn] = {
673
+ ...localMetadata,
674
+ id: remoteSkill.id,
675
+ path: remoteSkill.path || localMetadata.path
676
+ };
677
+ newHashes[scriptPath] = sha256(content);
678
+ syncedPaths.add(scriptPath);
679
+ updated++;
680
+ this.logger.info(`[newo_v2] Reused existing skill: ${flowIdn}/${skill.idn}`);
681
+ }
682
+ continue;
683
+ }
684
+ const createdParameters = await this.createMissingSkillParameters(client, existingSkill, localMetadata);
685
+ if (createdParameters > 0 || this.skillMetadataDiffers(existingSkill, localMetadata)) {
686
+ this.assertSkillModelResolved(localMetadata, skillLocator);
687
+ await updateSkill(client, {
688
+ id: existingSkill.id,
689
+ title: localMetadata.title,
690
+ idn: localMetadata.idn,
691
+ prompt_script: content,
692
+ runner_type: localMetadata.runner_type,
693
+ model: localMetadata.model,
694
+ parameters: localMetadata.parameters,
695
+ path: localMetadata.path
696
+ });
697
+ flowData.skills[skill.idn] = {
698
+ ...localMetadata,
699
+ id: existingSkill.id
700
+ };
701
+ newHashes[scriptPath] = sha256(content);
702
+ syncedPaths.add(scriptPath);
703
+ updated++;
704
+ this.logger.info(`[newo_v2] Updated skill metadata: ${flowIdn}/${skill.idn}`);
705
+ }
706
+ }
707
+ catch (error) {
708
+ errors.push(`Failed to sync skill ${skillLocator}: ${error instanceof Error ? error.message : String(error)}`);
709
+ }
710
+ }
711
+ }
712
+ }
713
+ }
714
+ return { created, updated, syncedPaths, errors };
715
+ }
716
+ async createMissingSkillParameters(client, existing, local) {
717
+ const existingNames = new Set(this.normalizeParameters(existing.parameters).map(p => p.name));
718
+ let created = 0;
719
+ for (const parameter of local.parameters) {
720
+ if (existingNames.has(parameter.name)) {
721
+ continue;
722
+ }
723
+ try {
724
+ await createSkillParameter(client, existing.id, {
725
+ name: parameter.name,
726
+ default_value: parameter.default_value ?? ''
727
+ });
728
+ created++;
729
+ this.logger.info(`[newo_v2] Created skill parameter: ${local.idn}/${parameter.name}`);
730
+ }
731
+ catch (error) {
732
+ if (!this.isAlreadyExistsApiError(error)) {
733
+ throw error;
734
+ }
735
+ }
736
+ existingNames.add(parameter.name);
737
+ }
738
+ return created;
739
+ }
740
+ /**
741
+ * Detect "resource already exists" API errors.
742
+ *
743
+ * Matches only on the precise phrases the platform actually returns
744
+ * ("already exists", "duplicate key"). Loose substrings like "exist"
745
+ * would otherwise sweep up unrelated "does not exist" / "doesn't exist"
746
+ * errors and trigger an incorrect reuse fallback.
747
+ */
748
+ isAlreadyExistsApiError(error) {
749
+ const response = error?.response;
750
+ const status = response?.status;
751
+ if (status !== 400 && status !== 409 && status !== 422) {
752
+ return false;
753
+ }
754
+ const haystack = JSON.stringify(response?.data ?? (error instanceof Error ? error.message : String(error))).toLowerCase();
755
+ return haystack.includes('already exists') || haystack.includes('duplicate key');
756
+ }
757
+ normalizeRunnerType(runnerType) {
758
+ return runnerType === 'nsl' ? 'nsl' : 'guidance';
759
+ }
760
+ normalizeParameters(parameters) {
761
+ return (parameters || []).map(p => ({
762
+ name: p.name,
763
+ default_value: p.default_value ?? ''
764
+ }));
765
+ }
766
+ /**
767
+ * Fail fast if no model could be resolved for a V2 skill.
768
+ *
769
+ * `buildV2SkillMetadataFromYaml` falls back to empty strings when neither
770
+ * the skill nor the flow declare a model. The platform rejects empty
771
+ * model_idn/provider_idn at creation/update time, but the error it returns
772
+ * is generic — we surface a clearer message before issuing the request.
773
+ */
774
+ assertSkillModelResolved(metadata, locator) {
775
+ if (!metadata.model.model_idn || !metadata.model.provider_idn) {
776
+ throw new Error(`[newo_v2] Cannot resolve model for skill ${locator}: ` +
777
+ `model_idn="${metadata.model.model_idn}", provider_idn="${metadata.model.provider_idn}". ` +
778
+ `Set either skill.model.* or flow default_model_idn/default_provider_idn in the flow YAML.`);
779
+ }
780
+ }
781
+ buildV2SkillMetadataFromYaml(skill, flowDef, runnerType, existing) {
782
+ return {
783
+ id: existing?.id || '',
784
+ idn: skill.idn,
785
+ title: skill.title || '',
786
+ runner_type: runnerType,
787
+ model: {
788
+ model_idn: skill.model?.model_idn || flowDef.default_model_idn || '',
789
+ provider_idn: skill.model?.provider_idn || flowDef.default_provider_idn || ''
790
+ },
791
+ parameters: this.normalizeParameters(skill.parameters),
792
+ path: existing?.path || ''
793
+ };
794
+ }
795
+ skillMetadataDiffers(existing, local) {
796
+ // Compare model/parameters field-by-field, never via JSON.stringify of the
797
+ // raw objects: the map stores model keys in platform API order
798
+ // (provider_idn first) while YAML-built metadata uses model_idn first, and
799
+ // a key-order-sensitive comparison flags every skill as changed.
800
+ const paramsKey = (params) => JSON.stringify(this.normalizeParameters(params).sort((a, b) => a.name.localeCompare(b.name)));
801
+ return (existing.title !== local.title ||
802
+ existing.runner_type !== local.runner_type ||
803
+ existing.model.model_idn !== local.model.model_idn ||
804
+ existing.model.provider_idn !== local.model.provider_idn ||
805
+ paramsKey(existing.parameters) !== paramsKey(local.parameters));
806
+ }
807
+ async resolveV2FlowSkillScriptPath(customerIdn, projectIdn, agentIdn, flowIdn, skillIdn, runnerType, promptScript) {
808
+ if (promptScript) {
809
+ const fromPromptScript = `${v2AgentDir(customerIdn, projectIdn, agentIdn)}/${promptScript}`;
810
+ if (await fs.pathExists(fromPromptScript)) {
811
+ return fromPromptScript;
812
+ }
813
+ }
814
+ return v2SkillScriptPath(customerIdn, projectIdn, agentIdn, flowIdn, skillIdn, runnerType);
815
+ }
567
816
  /**
568
817
  * Push a V2 skill update
569
818
  *
@@ -686,8 +935,18 @@ export class V2ProjectSyncStrategy {
686
935
  }
687
936
  }
688
937
  for (const [flowIdn, flowData] of Object.entries(agentData.flows)) {
689
- for (const [skillIdn, skillMeta] of Object.entries(flowData.skills)) {
690
- const scriptPath = v2SkillScriptPath(customer.idn, projectIdn, agentIdn, flowIdn, skillIdn, skillMeta.runner_type);
938
+ const flowYamlSkills = await this.loadLocalV2FlowSkills(customer.idn, projectIdn, agentIdn, flowIdn);
939
+ const skillIdns = new Set([
940
+ ...Object.keys(flowData.skills),
941
+ ...flowYamlSkills.keys()
942
+ ]);
943
+ for (const skillIdn of skillIdns) {
944
+ const yamlSkill = flowYamlSkills.get(skillIdn);
945
+ const skillMeta = flowData.skills[skillIdn];
946
+ const runnerType = this.normalizeRunnerType(yamlSkill?.runner_type || skillMeta?.runner_type);
947
+ const scriptPath = yamlSkill
948
+ ? await this.resolveV2FlowSkillScriptPath(customer.idn, projectIdn, agentIdn, flowIdn, skillIdn, runnerType, yamlSkill.prompt_script)
949
+ : v2SkillScriptPath(customer.idn, projectIdn, agentIdn, flowIdn, skillIdn, runnerType);
691
950
  if (await fs.pathExists(scriptPath)) {
692
951
  const content = await fs.readFile(scriptPath, 'utf8');
693
952
  const currentHash = sha256(content);
@@ -726,6 +985,19 @@ export class V2ProjectSyncStrategy {
726
985
  }
727
986
  return changes;
728
987
  }
988
+ async loadLocalV2FlowSkills(customerIdn, projectIdn, agentIdn, flowIdn) {
989
+ const flowYamlPath = v2FlowYamlPath(customerIdn, projectIdn, agentIdn, flowIdn);
990
+ if (!(await fs.pathExists(flowYamlPath))) {
991
+ return new Map();
992
+ }
993
+ try {
994
+ const flowDef = await parseV2FlowYaml(flowYamlPath);
995
+ return new Map((flowDef.skills || []).map(skill => [skill.idn, skill]));
996
+ }
997
+ catch {
998
+ return new Map();
999
+ }
1000
+ }
729
1001
  async validate(customer, _items) {
730
1002
  const errors = [];
731
1003
  const mapFile = mapPath(customer.idn);
@@ -741,7 +1013,46 @@ export class V2ProjectSyncStrategy {
741
1013
  for (const [projectIdn, projectData] of Object.entries(mapData.projects)) {
742
1014
  for (const [agentIdn, agentData] of Object.entries(projectData.agents)) {
743
1015
  for (const [flowIdn, flowData] of Object.entries(agentData.flows)) {
1016
+ const flowYamlPath = v2FlowYamlPath(customer.idn, projectIdn, agentIdn, flowIdn);
1017
+ let localYamlSkills;
1018
+ if (await fs.pathExists(flowYamlPath)) {
1019
+ try {
1020
+ const flowDef = await parseV2FlowYaml(flowYamlPath);
1021
+ localYamlSkills = new Map((flowDef.skills || []).map(s => [s.idn, s]));
1022
+ const skillIdns = new Set([
1023
+ ...Object.keys(flowData.skills),
1024
+ ...localYamlSkills.keys()
1025
+ ]);
1026
+ for (const skillIdn of skillIdns) {
1027
+ const localYamlSkill = localYamlSkills.get(skillIdn);
1028
+ const skillMeta = flowData.skills[skillIdn];
1029
+ if (!localYamlSkill) {
1030
+ errors.push({
1031
+ field: `skill.${skillIdn}`,
1032
+ message: `Skill exists in project map but is missing from flow YAML: ${flowYamlPath}`,
1033
+ path: flowYamlPath
1034
+ });
1035
+ continue;
1036
+ }
1037
+ const runnerType = this.normalizeRunnerType(localYamlSkill.runner_type || skillMeta?.runner_type);
1038
+ const scriptPath = await this.resolveV2FlowSkillScriptPath(customer.idn, projectIdn, agentIdn, flowIdn, skillIdn, runnerType, localYamlSkill.prompt_script);
1039
+ if (!(await fs.pathExists(scriptPath))) {
1040
+ errors.push({
1041
+ field: `skill.${localYamlSkill.idn}`,
1042
+ message: `Script file not found: ${scriptPath}`,
1043
+ path: scriptPath
1044
+ });
1045
+ }
1046
+ }
1047
+ }
1048
+ catch {
1049
+ localYamlSkills = undefined;
1050
+ }
1051
+ }
744
1052
  for (const [skillIdn, skillMeta] of Object.entries(flowData.skills)) {
1053
+ if (localYamlSkills) {
1054
+ continue;
1055
+ }
745
1056
  const scriptPath = v2SkillScriptPath(customer.idn, projectIdn, agentIdn, flowIdn, skillIdn, skillMeta.runner_type);
746
1057
  if (!(await fs.pathExists(scriptPath))) {
747
1058
  errors.push({
@@ -8,7 +8,7 @@
8
8
  * `value_type: json`. The API may return the `value` field as either a
9
9
  * STRING containing JSON or as an already-parsed OBJECT.
10
10
  *
11
- * Without normalization, two bugs leak through:
11
+ * Without normalization, several bugs leak through:
12
12
  *
13
13
  * 1. When the API returns the value as an OBJECT, `yaml.dump` serializes
14
14
  * it as a YAML structure (mappings/sequences). Pushing back then sends
@@ -21,10 +21,25 @@
21
21
  * string vs object representations it triggers spurious pushes that
22
22
  * overwrite the canvas with the wrong shape (Builder shows blank).
23
23
  *
24
- * The fix is conservative: for `value_type: json` only, always coerce the
25
- * value to a STRING when persisting and when pushing, and use canonical
26
- * JSON for comparisons. String-typed values in the wild are left
27
- * untouched, so no churn for the majority of attributes.
24
+ * 3. (Bug 3.7.2-a) Canvas JSON strings with structural newlines (real
25
+ * U+000A between tokens) can be emitted by yaml.dump as double-quoted
26
+ * scalars with `\n` escape sequences. patchYamlToPyyaml then converts
27
+ * those to single-quoted YAML scalars, where `\n` is treated as two
28
+ * literal chars (backslash + n). On push the platform stores those
29
+ * literal chars and the Builder calls JSON.parse, which fails on
30
+ * backslash-n as structural whitespace.
31
+ *
32
+ * 4. (Bug 3.7.2-b) Canvas body text contains Markdown with `\_`
33
+ * (backslash + underscore). `\_` is not a valid JSON escape sequence
34
+ * per RFC 8259 (valid ones: " \ / b f n r t uXXXX). Chrome V8's
35
+ * JSON.parse is strict: it throws SyntaxError on `\_`, silently
36
+ * blanking the Builder.
37
+ *
38
+ * The fix for (3) and (4): for `value_type: json` string values, strip
39
+ * invalid escape sequences then compact via JSON.parse + JSON.stringify.
40
+ * Compaction removes structural newlines and re-serializes all string
41
+ * values with only valid JSON escapes, producing a single-line string
42
+ * that round-trips through YAML without corruption.
28
43
  */
29
44
  /**
30
45
  * True if the attribute is a JSON-typed attribute (case- and
@@ -32,26 +47,40 @@
32
47
  * `ValueType.JSON`, etc.).
33
48
  */
34
49
  export declare function isJsonValueType(valueType: unknown): boolean;
50
+ /**
51
+ * Fix invalid JSON escape sequences inside JSON string values.
52
+ *
53
+ * Per RFC 8259, valid escape sequences inside a JSON string are:
54
+ * \" \\ \/ \b \f \n \r \t \uXXXX
55
+ * Anything else (e.g. `\_` `\.` from Markdown) is invalid and causes
56
+ * JSON.parse to throw. Fix: drop the backslash (e.g. `\_` → `_`).
57
+ *
58
+ * Only modifies characters inside JSON string values (tracks quote
59
+ * context). Structural characters outside strings are untouched.
60
+ */
61
+ export declare function fixInvalidJsonEscapes(s: string): string;
35
62
  /**
36
63
  * Coerce a JSON-typed attribute's value to a STRING suitable for storage
37
64
  * in attributes.yaml and for sending to the platform.
38
65
  *
39
66
  * - `null` / `undefined` → `''`
40
67
  * - object → compact JSON string (`JSON.stringify(value)`)
41
- * - string → returned as-is (we trust the platform's existing format)
68
+ * - string → fix invalid escapes (e.g. `\_` `_`), then compact via
69
+ * JSON.parse + JSON.stringify. If parsing still fails after
70
+ * fixing escapes, return the fixed string as-is.
42
71
  * - other → `String(value)`
43
72
  *
44
- * We deliberately do NOT re-format string values, even when they look
45
- * like JSON. Many existing canvases are stored pretty-printed and
46
- * reformatting would create huge spurious diffs in users' repos.
73
+ * Compacting removes structural newlines and guarantees a single-line
74
+ * string that yaml.dump serializes without escape-sequence corruption in
75
+ * the patchYamlToPyyaml pass. See module-level comment for full context.
47
76
  */
48
77
  export declare function normalizeJsonValueForStorage(value: unknown): string;
49
78
  /**
50
79
  * Canonical comparison for JSON-typed attribute values.
51
80
  *
52
81
  * Returns the canonical form (compact JSON if parseable, otherwise the
53
- * raw string). Use this on both sides of a comparison so that pretty- vs
54
- * compact-printed JSON does not register as a change, and so that an
82
+ * fixed string). Use this on both sides of a comparison so that pretty-
83
+ * vs compact-printed JSON does not register as a change, and so that an
55
84
  * object on one side equals its stringified form on the other side.
56
85
  */
57
86
  export declare function canonicalJsonValue(value: unknown): string;