@solidxai/core 0.1.12-beta.2 → 0.1.12-beta.4

package/docs/workflow/data-engineering-pipeline-example-notes.md

@@ -1,330 +0,0 @@
-# Data Engineering Pipeline Example Notes
-
-## Purpose
-
-This document captures a concrete workflow example and the accompanying UI notes derived from the Kestra reference screens. It is intended to serve as a working design reference for the SolidX workflow-definition list view, detail view, and editor experience.
-
----
-
-## Example Workflow YAML
-
-```yaml
-id: data-engineering-pipeline
-namespace: tutorial
-labels:
-  name: Data Engineering Pipeline
-
-inputs:
-  - id: columns_to_keep
-    type: ARRAY
-    itemType: STRING
-    defaults:
-      - brand
-      - price
-
-tasks:
-  - id: extract
-    type: io.kestra.plugin.core.http.Download
-    description: Download the raw products JSON data.
-    uri: https://dummyjson.com/products
-
-  - id: transform
-    type: io.kestra.plugin.scripts.python.Script
-    description: Filter the product fields and write a cleaned JSON file.
-    containerImage: python:3.11-alpine
-    inputFiles:
-      data.json: "{{ outputs.extract.uri }}"
-    outputFiles:
-      - "*.json"
-    script: |
-      import json
-      import os
-
-      with open("data.json", "r") as file:
-          data = json.load(file)
-
-      filtered_data = [
-          {column: product.get(column, "N/A") for column in {{ inputs.columns_to_keep }}}
-          for product in data["products"]
-      ]
-
-      with open("products.json", "w") as file:
-          json.dump(filtered_data, file, indent=4)
-
-  - id: query
-    type: io.kestra.plugin.jdbc.duckdb.Queries
-    description: Aggregate the cleaned data with DuckDB and store the result.
-    inputFiles:
-      products.json: "{{ outputs.transform.outputFiles['products.json'] }}"
-    sql: |
-      INSTALL json;
-      LOAD json;
-      SELECT brand, round(avg(price), 2) as avg_price
-      FROM read_json_auto('{{ workingDir }}/products.json')
-      GROUP BY brand
-      ORDER BY avg_price DESC;
-    fetchType: STORE
-
-description: |
-  **Use case:** Compact data engineering pipeline that filters product data and aggregates it by brand.
-  **Highlights:**
-  - Download product JSON from a public API.
-  - Run a Python transform that keeps only the requested columns and emits a cleaned file.
-  - Query the cleaned data with DuckDB to compute average prices per brand and store the result for download in the UI.
-```
-
----
-
-## What This Example Represents
-
-This workflow is a good V1 reference because it is:
-
-- Simple enough to understand quickly.
-- Sequential, so it maps well to the current implicit-edge model.
-- Realistic, because it uses HTTP download, transformation, and data querying.
-- Rich enough to exercise inputs, outputs, descriptions, code editing, and topology rendering.
-
-The visual topology implied by this workflow is:
-
-```text
-Start
-  |
-  v
-extract
-  |
-  v
-transform
-  |
-  v
-query
-  |
-  v
-End
-```
-
----
-
-## Detail Page Notes
-
-The reference detail experience suggests a workflow page shaped around tabs, not a single long form.
-
-### Tabs Observed
-
-- `Overview`
-- `Topology`
-- `Executions`
-- `Edit`
-- `Revisions`
-- `Triggers`
-- `Logs`
-- `Metrics`
-- `Dependencies`
-- `Concurrency`
-- `Audit Logs`
-
-### SolidX Interpretation
-
-For SolidX workflow-definition UX, the current recommended prioritization is:
-
-- Keep: `Overview`
-- Keep: `Topology`
-- Keep: `Executions`
-- Keep: `Edit`
-- Keep: `Triggers`
-- Keep: `Logs`
-- Keep: `Metrics`
-- Ignore for now: `Revisions`
-- Ignore for now: `Concurrency`
-- Ignore for now: `Audit Logs`
-
----
-
-## Overview Notes
-
-The overview page should change meaningfully once executions exist.
-
-### Before Executions
-
-The page can show an empty-state hero with a primary call to action:
-
-- execute workflow
-- guide the user toward the first run
-
-### After Executions
-
-The page can evolve into a summary dashboard showing:
-
-- success ratio
-- failed ratio
-- in-progress count
-- pending count
-- total executions
-- execution trend / chart
-- description panel
-
-This means the overview page should not be treated as static documentation. It should become an operational summary once runtime data exists.
-
----
-
-## Topology Notes
-
-Topology is read-only in the reference detail page.
-
-Recommended SolidX interpretation:
-
-- `Topology` tab should be a clean read-only visualization.
-- It should render the workflow structure without edit affordances.
-- It can reuse the same layout logic as the editor flow view, but without mutation controls.
-
----
-
-## Executions Notes
-
-The `Executions` tab should present a list of all executions for the workflow.
-
-Important characteristics from the reference:
-
-- list-based presentation
-- filtering/search support
-- empty state when no executions exist
-
-For SolidX:
-
-- use existing retrieve APIs
-- support server-side pagination, sorting, and search where possible
-- keep the visuals close to SolidX list/table patterns
-
----
-
-## Edit Tab Notes
-
-This is the most important area for the current UI work.
-
-### Desired Structure
-
-- `Edit` is a top-level detail tab.
-- Inside `Edit`, only `Code` and `Flow` should be primary editing surfaces.
-- `Docs` should open in a side panel, not as a main third panel in the base layout.
-
-### Editor Layout
-
-- left: code editor
-- right: flow editor
-- docs: contextual side panel that opens on demand
-
-### Editing Mode
-
-- flow uses React Flow
-- modal editing for nodes
-- docs are contextual to the selected node type
-- save remains manual
-
----
-
-## Flow Canvas Notes
-
-The reference behavior to preserve:
-
-- `+` insertion controls between nodes
-- per-node actions:
-  - edit
-  - delete
-  - view docs
-- directional flow should be visible
-- zoom in / zoom out controls should exist
-- users may drag nodes to rearrange the graph visually
-- dragging is for visual arrangement only, not for defining semantic connections via drag-and-drop wiring
-
-### Important Modeling Note
-
-Workflow structure is still governed by the workflow DSL contract.
-
-That means:
-
-- node order remains driven by workflow structure
-- edges are implicit from the DSL
-- the canvas is a visual/editor representation of the workflow, not the source of truth for execution semantics
-
----
-
-## Triggers Notes
-
-Triggers are wanted, but not the first editing priority.
-
-Current note:
-
-- keep a `Triggers` tab in the long-term detail experience
-- rendering and editing of triggers can come after the core editor is stabilized
-
----
-
-## Logs Notes
-
-Logs are wanted.
-
-Recommended direction:
-
-- separate `Logs` tab
-- searchable / filterable runtime log list
-- use workflow or execution context to scope logs
-
----
-
-## Metrics Notes
-
-Metrics are wanted, but depend on runtime persistence.
-
-Important note:
-
-- metrics are only meaningful once we persist them into backing tables or another queryable runtime store
-
-So for now:
-
-- keep `Metrics` in the design vocabulary
-- treat it as a pending feature unless the persistence model is implemented
-
----
-
-## Pending Feature Notes
-
-The following are intentionally deferred or partially deferred:
-
-- metrics persistence and reporting
-- trigger authoring experience
-- richer overview analytics
-- revisions
-- concurrency controls
-- audit-log specific workflow views
-- blueprints for this area as part of a later phase
-
----
-
-## Screenshot References
-
-These screenshots informed the notes above:
-
-- `/Users/harishpatel/Desktop/Screenshot 2026-07-10 at 16.58.43.png`
-- `/Users/harishpatel/Desktop/Screenshot 2026-07-10 at 16.58.48.png`
-- `/Users/harishpatel/Desktop/Screenshot 2026-07-10 at 16.58.53.png`
-- `/Users/harishpatel/Desktop/Screenshot 2026-07-10 at 16.58.59.png`
-- `/Users/harishpatel/Desktop/Screenshot 2026-07-10 at 16.59.07.png`
-- `/Users/harishpatel/Desktop/Screenshot 2026-07-10 at 16.59.16.png`
-- `/Users/harishpatel/Desktop/Screenshot 2026-07-10 at 16.59.22.png`
-- `/Users/harishpatel/Desktop/Screenshot 2026-07-10 at 16.59.28.png`
-- `/Users/harishpatel/Desktop/Screenshot 2026-07-10 at 17.01.42.png`
-- `/Users/harishpatel/Desktop/Screenshot 2026-07-10 at 17.01.47.png`
-
----
-
-## Recommended Next UI Build Order
-
-Based on this example, the clean next implementation sequence is:
-
-1. Finish the bespoke workflow-definition list view.
-2. Tighten the workflow-definition detail shell with tabs.
-3. Refine the `Edit` tab into:
-   - left code editor
-   - right React Flow editor
-   - contextual docs side panel
-4. Add modal-based node configuration using schema-driven forms.
-5. Add runtime tabs for executions, logs, and later metrics.

package/docs/workflow/foreach-example-notes.md

@@ -1,187 +0,0 @@
-# ForEach Example Notes
-
-## Purpose
-
-This document captures a reference workflow for loop-based control flow using `ForEach`. It fills an important gap in the example set by showing how repeated execution over a list of values should be represented in topology and editor views.
-
----
-
-## Example Workflow YAML
-
-```yaml
-id: foreach_example
-namespace: my.namespace
-
-tasks:
-  - id: for_each_loop
-    type: io.kestra.core.tasks.flows.ForEach
-    values: ["item_1", "item_2", "item_3"]
-    tasks:
-      - id: print_value
-        type: io.kestra.core.tasks.scripts.Bash
-        commands:
-          - echo "Current loop value is {{ taskrun.value }}"
-```
-
----
-
-## What This Example Adds
-
-This example is useful because it shows:
-
-- iterative control flow
-- a loop container node
-- child tasks executed once per loop item
-- loop-scoped runtime values such as `taskrun.value`
-
-It is the cleanest reference for repeated execution semantics in the workflow DSL.
-
----
-
-## High-Level Structure
-
-```text
-Start
-  |
-  v
-for_each_loop
-  |
-  v
-print_value
-  |
-  v
-End
-```
-
-Conceptually, the structure is:
-
-```text
-Workflow
-└── for_each_loop
-    └── print_value
-```
-
-Runtime interpretation:
-
-```text
-for_each_loop over ["item_1", "item_2", "item_3"]
-  - run print_value with taskrun.value = "item_1"
-  - run print_value with taskrun.value = "item_2"
-  - run print_value with taskrun.value = "item_3"
-```
-
----
-
-## Main Note
-
-### Example With `ForEach`
-
-This is the key purpose of the example.
-
-It shows that a loop is not just a visual group. It represents repeated execution of its child tasks across a collection of values.
-
-For SolidX, the topology should communicate:
-
-- the loop container
-- the loop body
-- that child tasks belong to the repeated block
-
-The canvas does not need to simulate every repeated instance visually. It should represent the structural loop definition cleanly.
-
----
-
-## Topology Notes
-
-The screenshot suggests:
-
-- `ForEach` appears as a grouped/container region
-- the loop node is rendered inside that region
-- the child task appears within the same grouped flow area
-- the overall sequence still remains vertical and directional
-
-This means the renderer should support a loop-specific grouped layout similar to other control-flow containers, but with loop semantics instead of branch semantics.
-
----
-
-## Loop Semantics Notes
-
-This example is important because it introduces loop-local context:
-
-- `taskrun.value`
-
-That means the editor and docs system will eventually need to help users understand:
-
-- which context values are available inside loops
-- that those values only exist within the loop body
-- how expressions differ between workflow scope and loop scope
-
-This is a documentation and autocomplete concern as much as a topology concern.
-
----
-
-## Implications For SolidX Flow Modeling
-
-This example implies the flow model should support a distinct category for loop containers.
-
-Useful structural categories now include:
-
-- simple task node
-- trigger group
-- branch/control-flow container
-- nested/group container
-- loop container
-
-For `ForEach` specifically, the model should preserve:
-
-- loop source values
-- child task list
-- loop-local expression scope
-
----
-
-## Editor Notes
-
-This example is a good reminder that the editor should remain DSL-driven.
-
-For a loop:
-
-- the source of truth is the `ForEach` node definition
-- the canvas should reflect loop structure
-- the user should edit loop configuration from metadata/schema, not by drawing repeated paths manually
-
-This fits well with the current direction:
-
-- code + flow are the primary surfaces
-- node editing is modal
-- docs are contextual
-
----
-
-## Future UX Notes
-
-This example points to a few future needs:
-
-- loop-aware docs for `taskrun.value`
-- loop-container editing modal
-- schema-driven configuration for `values`
-- clear distinction between loop definition and loop runtime iterations
-
----
-
-## Screenshot References
-
-This screenshot informed the notes above:
-
-- `/var/folders/z_/h0rqj1693gs4cxm3k_vks86c0000gn/T/TemporaryItems/NSIRD_screencaptureui_ptomjl/Screenshot 2026-07-10 at 17.17.48.png`
-
----
-
-## Recommended Follow-On Use
-
-This example should be used as a reference for:
-
-1. `ForEach` node rendering
-2. loop-container layout rules
-3. loop-local expression awareness
-4. schema-driven loop configuration UX
-5. future autocomplete help for loop scope values