newo 3.7.2 → 3.7.3

package/CHANGELOG.md

@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [3.7.3] - 2026-05-25
+
+### Fixed
+
+- **Workflow Builder canvas blank-screen, take two** - JSON-typed attribute values that contain Markdown body text with `\_` escapes (Builder uses backslash-underscore to escape underscores) and/or structural newlines from pretty-printed JSON no longer corrupt the canvas on push. Two bugs were leaking through v3.7.1's STRING-coercion fix: (a) `yaml.dump` of a pretty-printed JSON string emitted a double-quoted scalar with `\n` escape sequences, which `patchYamlToPyyaml` then rewrote as a single-quoted scalar where `\n` became two literal chars - Builder's `JSON.parse` then choked on backslash-n as structural whitespace; (b) `\_` is not a valid JSON escape per RFC 8259 and Chrome V8's `JSON.parse` throws on it, silently blanking the Builder. Fix: `normalizeJsonValueForStorage` now strips invalid escape sequences via a quote-aware walker (`fixInvalidJsonEscapes`), then compacts via `JSON.parse` + `JSON.stringify` so the value is a single-line string that survives `yaml.dump` -> `patchYamlToPyyaml` without escape-sequence corruption. Existing pretty-printed canvases in `attributes.yaml` will be reformatted to compact form on next pull (one-time stylistic diff, no semantic loss). Change-detection (`jsonValuesEqual`) continues to canonicalize both sides so the new compact form does not trigger spurious pushes against remotes that may still be pretty. Reported by Bob in [#7](https://github.com/sabbah13/newo-cli/pull/7); 25 regression tests in `test/json-attribute-roundtrip.test.js` covering both bug families and the full pull -> dump -> patch -> reload -> `JSON.parse` pipeline.
+
+### Added
+
+- **`fixInvalidJsonEscapes(s)`** in `src/sync/json-attr-utils.ts` - quote-aware walker that strips invalid JSON escape sequences (`\_`, `\.`, etc.) inside JSON string values per RFC 8259. Preserves all valid escapes (`\" \\ \/ \b \f \n \r \t \uXXXX`) and structural characters outside strings.
+
 ## [3.7.2] - 2026-05-17
 
 ### Fixed
@@ -1051,7 +1061,9 @@ Another Item: $Price [Modifiers: modifier3]
 - GitHub Actions CI/CD integration
 - Robust authentication with token refresh
 
-[Unreleased]: https://github.com/sabbah13/newo-cli/compare/v3.7.1...HEAD
+[Unreleased]: https://github.com/sabbah13/newo-cli/compare/v3.7.3...HEAD
+[3.7.3]: https://github.com/sabbah13/newo-cli/compare/v3.7.2...v3.7.3
+[3.7.2]: https://github.com/sabbah13/newo-cli/compare/v3.7.1...v3.7.2
 [3.7.1]: https://github.com/sabbah13/newo-cli/compare/v3.7.0...v3.7.1
 [3.3.0]: https://github.com/sabbah13/newo-cli/compare/v3.2.0...v3.3.0
 [3.2.0]: https://github.com/sabbah13/newo-cli/compare/v3.1.0...v3.2.0

package/README.md

@@ -8,6 +8,8 @@
 **NEWO CLI** - Professional command-line tool for NEWO AI Agent development. Features **modular architecture**, **IDN-based file management**, and **comprehensive multi-customer support**.
 
 Sync NEWO "Project → Agent → Flow → Skills" structure to local files with:
+- 🆕 **Canvas blank-screen hardening** (v3.7.3) - JSON-typed attributes (e.g. Workflow Builder canvas) with Markdown `\_` escapes or structural newlines no longer corrupt the canvas on push ([#7](https://github.com/sabbah13/newo-cli/pull/7))
+- 🆕 **Flow metadata sync** (v3.7.2) - `newo push` now reconciles flow title, events, and state_fields from local `metadata.yaml` to the platform (closes [#3](https://github.com/sabbah13/newo-cli/issues/3))
 - 🆕 **Dual format support** (v3.6.0) - `cli_v1` (native) and `newo_v2` (platform compatible), auto-detected per customer
 - 🆕 **Libraries** (v3.6.0) - Pull/push shared reusable skills across agents within a project
 - 🆕 **Bulk export** (v3.6.0) - `newo export` downloads complete V2 ZIP from platform
@@ -153,6 +155,28 @@ NEWO_REFRESH_URL=custom_refresh_endpoint   # Custom refresh endpoint
 | `newo import-akb` | Import knowledge base articles | • Structured text parsing<br>• Bulk article import<br>• Validation and error reporting |
 | `newo meta` | Get project metadata (debug) | • Project structure analysis<br>• Metadata validation |
 
+### Flow Metadata Sync (NEW v3.7.2)
+
+`newo push` now reconciles **flow-level metadata** — title, `events:`, and `state_fields:` — from local YAML to the platform. Before v3.7.2 push only uploaded skill scripts, so edits to a flow's `metadata.yaml` (V1) or `{FlowIdn}.yaml` (V2) silently never reached the platform; events added via `newo create-event` could appear to disappear after a pull → push cycle. Closes [#3](https://github.com/sabbah13/newo-cli/issues/3).
+
+**How it works:**
+
+| Local change in `metadata.yaml` | What push does |
+|---|---|
+| `title:` changed | `PATCH /api/v1/designer/flows/{id}` with full descriptor |
+| New event in `events:` | `POST /api/v1/designer/flows/{id}/events` |
+| Event field edited | `PATCH /api/v1/designer/flows/events/{eventId}` |
+| Event removed from list | `DELETE /api/v1/designer/flows/events/{eventId}` |
+| Same for `state_fields:` | Mirrored CRUD against `/states` |
+
+**Safety:**
+
+- **Hash-gated**: flows whose `metadata.yaml` SHA256 is unchanged are *not* compared against the platform. Stale local trees cannot wipe events you created via the Builder UI.
+- **Full sync on changed flows**: when you *do* edit `metadata.yaml`, local becomes the source of truth for that flow. Events present on the platform but missing locally are deleted. To keep events created out-of-band, run `newo pull` before editing.
+- **Push output**: changed flows print `↑ Flow <flow>: events +N/~N/-N, states +N/~N/-N` so you see exactly what synced.
+
+**Available in:** legacy V1 push path (`pushChanged`), `ProjectSyncStrategy`, and `V2ProjectSyncStrategy`. Works the same with `newo push --format cli_v1` and `--format newo_v2`.
+
 ### Lint, Format, Check (NEW v3.7.0)
 
 Static-analysis over DSL files, powered by [`newo-dsl-analyzer`](https://www.npmjs.com/package/newo-dsl-analyzer). Same engine that runs in the VS Code extension - no drift between editor and CI.

package/dist/sync/json-attr-utils.d.ts

@@ -8,7 +8,7 @@
  * `value_type: json`. The API may return the `value` field as either a
  * STRING containing JSON or as an already-parsed OBJECT.
  *
- * Without normalization, two bugs leak through:
+ * Without normalization, several bugs leak through:
  *
  * 1. When the API returns the value as an OBJECT, `yaml.dump` serializes
  *    it as a YAML structure (mappings/sequences). Pushing back then sends
@@ -21,10 +21,25 @@
  *    string vs object representations it triggers spurious pushes that
  *    overwrite the canvas with the wrong shape (Builder shows blank).
  *
- * The fix is conservative: for `value_type: json` only, always coerce the
- * value to a STRING when persisting and when pushing, and use canonical
- * JSON for comparisons. String-typed values in the wild are left
- * untouched, so no churn for the majority of attributes.
+ * 3. (Bug 3.7.2-a) Canvas JSON strings with structural newlines (real
+ *    U+000A between tokens) can be emitted by yaml.dump as double-quoted
+ *    scalars with `\n` escape sequences. patchYamlToPyyaml then converts
+ *    those to single-quoted YAML scalars, where `\n` is treated as two
+ *    literal chars (backslash + n). On push the platform stores those
+ *    literal chars and the Builder calls JSON.parse, which fails on
+ *    backslash-n as structural whitespace.
+ *
+ * 4. (Bug 3.7.2-b) Canvas body text contains Markdown with `\_`
+ *    (backslash + underscore). `\_` is not a valid JSON escape sequence
+ *    per RFC 8259 (valid ones: " \ / b f n r t uXXXX). Chrome V8's
+ *    JSON.parse is strict: it throws SyntaxError on `\_`, silently
+ *    blanking the Builder.
+ *
+ * The fix for (3) and (4): for `value_type: json` string values, strip
+ * invalid escape sequences then compact via JSON.parse + JSON.stringify.
+ * Compaction removes structural newlines and re-serializes all string
+ * values with only valid JSON escapes, producing a single-line string
+ * that round-trips through YAML without corruption.
  */
 /**
  * True if the attribute is a JSON-typed attribute (case- and
@@ -32,26 +47,40 @@
  * `ValueType.JSON`, etc.).
  */
 export declare function isJsonValueType(valueType: unknown): boolean;
+/**
+ * Fix invalid JSON escape sequences inside JSON string values.
+ *
+ * Per RFC 8259, valid escape sequences inside a JSON string are:
+ *   \" \\ \/ \b \f \n \r \t \uXXXX
+ * Anything else (e.g. `\_` `\.` from Markdown) is invalid and causes
+ * JSON.parse to throw. Fix: drop the backslash (e.g. `\_` → `_`).
+ *
+ * Only modifies characters inside JSON string values (tracks quote
+ * context). Structural characters outside strings are untouched.
+ */
+export declare function fixInvalidJsonEscapes(s: string): string;
 /**
  * Coerce a JSON-typed attribute's value to a STRING suitable for storage
  * in attributes.yaml and for sending to the platform.
  *
  * - `null` / `undefined` → `''`
  * - object → compact JSON string (`JSON.stringify(value)`)
- * - string → returned as-is (we trust the platform's existing format)
+ * - string → fix invalid escapes (e.g. `\_` → `_`), then compact via
+ *            JSON.parse + JSON.stringify. If parsing still fails after
+ *            fixing escapes, return the fixed string as-is.
  * - other → `String(value)`
  *
- * We deliberately do NOT re-format string values, even when they look
- * like JSON. Many existing canvases are stored pretty-printed and
- * reformatting would create huge spurious diffs in users' repos.
+ * Compacting removes structural newlines and guarantees a single-line
+ * string that yaml.dump serializes without escape-sequence corruption in
+ * the patchYamlToPyyaml pass. See module-level comment for full context.
  */
 export declare function normalizeJsonValueForStorage(value: unknown): string;
 /**
  * Canonical comparison for JSON-typed attribute values.
  *
  * Returns the canonical form (compact JSON if parseable, otherwise the
- * raw string). Use this on both sides of a comparison so that pretty- vs
- * compact-printed JSON does not register as a change, and so that an
+ * fixed string). Use this on both sides of a comparison so that pretty-
+ * vs compact-printed JSON does not register as a change, and so that an
  * object on one side equals its stringified form on the other side.
  */
 export declare function canonicalJsonValue(value: unknown): string;

package/dist/sync/json-attr-utils.js

@@ -8,7 +8,7 @@
  * `value_type: json`. The API may return the `value` field as either a
  * STRING containing JSON or as an already-parsed OBJECT.
  *
- * Without normalization, two bugs leak through:
+ * Without normalization, several bugs leak through:
  *
  * 1. When the API returns the value as an OBJECT, `yaml.dump` serializes
  *    it as a YAML structure (mappings/sequences). Pushing back then sends
@@ -21,10 +21,25 @@
  *    string vs object representations it triggers spurious pushes that
  *    overwrite the canvas with the wrong shape (Builder shows blank).
  *
- * The fix is conservative: for `value_type: json` only, always coerce the
- * value to a STRING when persisting and when pushing, and use canonical
- * JSON for comparisons. String-typed values in the wild are left
- * untouched, so no churn for the majority of attributes.
+ * 3. (Bug 3.7.2-a) Canvas JSON strings with structural newlines (real
+ *    U+000A between tokens) can be emitted by yaml.dump as double-quoted
+ *    scalars with `\n` escape sequences. patchYamlToPyyaml then converts
+ *    those to single-quoted YAML scalars, where `\n` is treated as two
+ *    literal chars (backslash + n). On push the platform stores those
+ *    literal chars and the Builder calls JSON.parse, which fails on
+ *    backslash-n as structural whitespace.
+ *
+ * 4. (Bug 3.7.2-b) Canvas body text contains Markdown with `\_`
+ *    (backslash + underscore). `\_` is not a valid JSON escape sequence
+ *    per RFC 8259 (valid ones: " \ / b f n r t uXXXX). Chrome V8's
+ *    JSON.parse is strict: it throws SyntaxError on `\_`, silently
+ *    blanking the Builder.
+ *
+ * The fix for (3) and (4): for `value_type: json` string values, strip
+ * invalid escape sequences then compact via JSON.parse + JSON.stringify.
+ * Compaction removes structural newlines and re-serializes all string
+ * values with only valid JSON escapes, producing a single-line string
+ * that round-trips through YAML without corruption.
  */
 /**
  * True if the attribute is a JSON-typed attribute (case- and
@@ -37,24 +52,84 @@ export function isJsonValueType(valueType) {
     const lower = valueType.toLowerCase();
     return lower === 'json' || lower.endsWith('.json');
 }
+/**
+ * Fix invalid JSON escape sequences inside JSON string values.
+ *
+ * Per RFC 8259, valid escape sequences inside a JSON string are:
+ *   \" \\ \/ \b \f \n \r \t \uXXXX
+ * Anything else (e.g. `\_` `\.` from Markdown) is invalid and causes
+ * JSON.parse to throw. Fix: drop the backslash (e.g. `\_` → `_`).
+ *
+ * Only modifies characters inside JSON string values (tracks quote
+ * context). Structural characters outside strings are untouched.
+ */
+export function fixInvalidJsonEscapes(s) {
+    const VALID_ESCAPES = new Set(['"', '\\', '/', 'b', 'f', 'n', 'r', 't', 'u']);
+    const result = [];
+    let inString = false;
+    let i = 0;
+    while (i < s.length) {
+        const c = s[i];
+        if (inString) {
+            if (c === '\\' && i + 1 < s.length) {
+                const next = s[i + 1];
+                if (VALID_ESCAPES.has(next)) {
+                    result.push(c, next);
+                }
+                else {
+                    result.push(next); // drop the backslash — \_ → _, etc.
+                }
+                i += 2;
+                continue;
+            }
+            else if (c === '"') {
+                inString = false;
+                result.push(c);
+            }
+            else {
+                result.push(c);
+            }
+        }
+        else {
+            if (c === '"') {
+                inString = true;
+                result.push(c);
+            }
+            else {
+                result.push(c);
+            }
+        }
+        i++;
+    }
+    return result.join('');
+}
 /**
  * Coerce a JSON-typed attribute's value to a STRING suitable for storage
  * in attributes.yaml and for sending to the platform.
  *
  * - `null` / `undefined` → `''`
  * - object → compact JSON string (`JSON.stringify(value)`)
- * - string → returned as-is (we trust the platform's existing format)
+ * - string → fix invalid escapes (e.g. `\_` → `_`), then compact via
+ *            JSON.parse + JSON.stringify. If parsing still fails after
+ *            fixing escapes, return the fixed string as-is.
  * - other → `String(value)`
  *
- * We deliberately do NOT re-format string values, even when they look
- * like JSON. Many existing canvases are stored pretty-printed and
- * reformatting would create huge spurious diffs in users' repos.
+ * Compacting removes structural newlines and guarantees a single-line
+ * string that yaml.dump serializes without escape-sequence corruption in
+ * the patchYamlToPyyaml pass. See module-level comment for full context.
  */
 export function normalizeJsonValueForStorage(value) {
     if (value == null)
         return '';
-    if (typeof value === 'string')
-        return value;
+    if (typeof value === 'string') {
+        const fixed = fixInvalidJsonEscapes(value);
+        try {
+            return JSON.stringify(JSON.parse(fixed));
+        }
+        catch {
+            return fixed;
+        }
+    }
     if (typeof value === 'object') {
         try {
             return JSON.stringify(value);
@@ -69,20 +144,12 @@ export function normalizeJsonValueForStorage(value) {
  * Canonical comparison for JSON-typed attribute values.
  *
  * Returns the canonical form (compact JSON if parseable, otherwise the
- * raw string). Use this on both sides of a comparison so that pretty- vs
- * compact-printed JSON does not register as a change, and so that an
+ * fixed string). Use this on both sides of a comparison so that pretty-
+ * vs compact-printed JSON does not register as a change, and so that an
  * object on one side equals its stringified form on the other side.
  */
 export function canonicalJsonValue(value) {
-    const stringified = normalizeJsonValueForStorage(value);
-    if (stringified === '')
-        return '';
-    try {
-        return JSON.stringify(JSON.parse(stringified));
-    }
-    catch {
-        return stringified;
-    }
+    return normalizeJsonValueForStorage(value);
 }
 /**
  * True if two JSON-typed attribute values are semantically equal.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "newo",
-  "version": "3.7.2",
+  "version": "3.7.3",
   "description": "NEWO CLI: Professional command-line tool with modular architecture for NEWO AI Agent development. Features account migration, integration management, webhook automation, AKB knowledge base, project attributes, sandbox testing, IDN-based file management, real-time progress tracking, intelligent sync operations, and comprehensive multi-customer support.",
   "type": "module",
   "bin": {

package/src/sync/json-attr-utils.ts

@@ -8,7 +8,7 @@
  * `value_type: json`. The API may return the `value` field as either a
  * STRING containing JSON or as an already-parsed OBJECT.
  *
- * Without normalization, two bugs leak through:
+ * Without normalization, several bugs leak through:
  *
  * 1. When the API returns the value as an OBJECT, `yaml.dump` serializes
  *    it as a YAML structure (mappings/sequences). Pushing back then sends
@@ -21,10 +21,25 @@
  *    string vs object representations it triggers spurious pushes that
  *    overwrite the canvas with the wrong shape (Builder shows blank).
  *
- * The fix is conservative: for `value_type: json` only, always coerce the
- * value to a STRING when persisting and when pushing, and use canonical
- * JSON for comparisons. String-typed values in the wild are left
- * untouched, so no churn for the majority of attributes.
+ * 3. (Bug 3.7.2-a) Canvas JSON strings with structural newlines (real
+ *    U+000A between tokens) can be emitted by yaml.dump as double-quoted
+ *    scalars with `\n` escape sequences. patchYamlToPyyaml then converts
+ *    those to single-quoted YAML scalars, where `\n` is treated as two
+ *    literal chars (backslash + n). On push the platform stores those
+ *    literal chars and the Builder calls JSON.parse, which fails on
+ *    backslash-n as structural whitespace.
+ *
+ * 4. (Bug 3.7.2-b) Canvas body text contains Markdown with `\_`
+ *    (backslash + underscore). `\_` is not a valid JSON escape sequence
+ *    per RFC 8259 (valid ones: " \ / b f n r t uXXXX). Chrome V8's
+ *    JSON.parse is strict: it throws SyntaxError on `\_`, silently
+ *    blanking the Builder.
+ *
+ * The fix for (3) and (4): for `value_type: json` string values, strip
+ * invalid escape sequences then compact via JSON.parse + JSON.stringify.
+ * Compaction removes structural newlines and re-serializes all string
+ * values with only valid JSON escapes, producing a single-line string
+ * that round-trips through YAML without corruption.
  */
 
 /**
@@ -38,22 +53,78 @@ export function isJsonValueType(valueType: unknown): boolean {
   return lower === 'json' || lower.endsWith('.json');
 }
 
+/**
+ * Fix invalid JSON escape sequences inside JSON string values.
+ *
+ * Per RFC 8259, valid escape sequences inside a JSON string are:
+ *   \" \\ \/ \b \f \n \r \t \uXXXX
+ * Anything else (e.g. `\_` `\.` from Markdown) is invalid and causes
+ * JSON.parse to throw. Fix: drop the backslash (e.g. `\_` → `_`).
+ *
+ * Only modifies characters inside JSON string values (tracks quote
+ * context). Structural characters outside strings are untouched.
+ */
+export function fixInvalidJsonEscapes(s: string): string {
+  const VALID_ESCAPES = new Set(['"', '\\', '/', 'b', 'f', 'n', 'r', 't', 'u']);
+  const result: string[] = [];
+  let inString = false;
+  let i = 0;
+  while (i < s.length) {
+    const c = s[i]!;
+    if (inString) {
+      if (c === '\\' && i + 1 < s.length) {
+        const next = s[i + 1]!;
+        if (VALID_ESCAPES.has(next)) {
+          result.push(c, next);
+        } else {
+          result.push(next); // drop the backslash — \_ → _, etc.
+        }
+        i += 2;
+        continue;
+      } else if (c === '"') {
+        inString = false;
+        result.push(c);
+      } else {
+        result.push(c);
+      }
+    } else {
+      if (c === '"') {
+        inString = true;
+        result.push(c);
+      } else {
+        result.push(c);
+      }
+    }
+    i++;
+  }
+  return result.join('');
+}
+
 /**
  * Coerce a JSON-typed attribute's value to a STRING suitable for storage
  * in attributes.yaml and for sending to the platform.
  *
  * - `null` / `undefined` → `''`
  * - object → compact JSON string (`JSON.stringify(value)`)
- * - string → returned as-is (we trust the platform's existing format)
+ * - string → fix invalid escapes (e.g. `\_` → `_`), then compact via
+ *            JSON.parse + JSON.stringify. If parsing still fails after
+ *            fixing escapes, return the fixed string as-is.
  * - other → `String(value)`
  *
- * We deliberately do NOT re-format string values, even when they look
- * like JSON. Many existing canvases are stored pretty-printed and
- * reformatting would create huge spurious diffs in users' repos.
+ * Compacting removes structural newlines and guarantees a single-line
+ * string that yaml.dump serializes without escape-sequence corruption in
+ * the patchYamlToPyyaml pass. See module-level comment for full context.
  */
 export function normalizeJsonValueForStorage(value: unknown): string {
   if (value == null) return '';
-  if (typeof value === 'string') return value;
+  if (typeof value === 'string') {
+    const fixed = fixInvalidJsonEscapes(value);
+    try {
+      return JSON.stringify(JSON.parse(fixed));
+    } catch {
+      return fixed;
+    }
+  }
   if (typeof value === 'object') {
     try {
       return JSON.stringify(value);
@@ -68,18 +139,12 @@ export function normalizeJsonValueForStorage(value: unknown): string {
  * Canonical comparison for JSON-typed attribute values.
  *
  * Returns the canonical form (compact JSON if parseable, otherwise the
- * raw string). Use this on both sides of a comparison so that pretty- vs
- * compact-printed JSON does not register as a change, and so that an
+ * fixed string). Use this on both sides of a comparison so that pretty-
+ * vs compact-printed JSON does not register as a change, and so that an
  * object on one side equals its stringified form on the other side.
  */
 export function canonicalJsonValue(value: unknown): string {
-  const stringified = normalizeJsonValueForStorage(value);
-  if (stringified === '') return '';
-  try {
-    return JSON.stringify(JSON.parse(stringified));
-  } catch {
-    return stringified;
-  }
+  return normalizeJsonValueForStorage(value);
 }
 
 /**