sparkforensics-cli 0.1.0 → 0.2.2

package/export-template/docs/assets/contributor-guide_architecture_worker-protocol.md.IbnfNrV3.js

@@ -0,0 +1,6 @@
+import{_ as o,o as t,c as d,a5 as a}from"./chunks/framework.DSg0KOwT.js";const u=JSON.parse('{"title":"Worker protocol","description":"","frontmatter":{},"headers":[],"relativePath":"contributor-guide/architecture/worker-protocol.md","filePath":"contributor-guide/architecture/worker-protocol.md"}'),c={name:"contributor-guide/architecture/worker-protocol.md"};function s(r,e,i,n,l,p){return t(),d("div",null,[...e[0]||(e[0]=[a(`<h1 id="worker-protocol" tabindex="-1">Worker protocol <a class="header-anchor" href="#worker-protocol" aria-label="Permalink to &quot;Worker protocol&quot;">​</a></h1><h2 id="message-protocol" tabindex="-1">Message protocol <a class="header-anchor" href="#message-protocol" aria-label="Permalink to &quot;Message protocol&quot;">​</a></h2><p>Worker to main:</p><ul><li><code>app</code>, <code>stage</code>, <code>sql</code>, <code>executor</code>, <code>job</code>, <code>progress</code>: progressive, posted as the parser reads.</li><li><code>sqlPlan</code> (<code>{ type: &#39;sqlPlan&#39;, data: { executionId, planTree } }</code>): posted by <code>endSqlExecution</code> once a SQL execution&#39;s plan tree resolves.</li><li><code>runAggregates</code>: one whole-run core-time-series summary (busy-core-ms, peak concurrency, per-stage task-duration sums), emitted just before <code>done</code>.</li><li><code>stageExecutorMetrics</code>: the post-completion re-post described in <a href="./overview.html#streaming">Streaming</a>, also emitted just before <code>done</code>.</li><li><code>done</code>, <code>taskData</code>, <code>error</code>.</li></ul><p>A History Server failure is always the typed, display-safe payload <code>{ type: &#39;error&#39;, source: &#39;shs&#39;, code }</code>, where <code>code</code> is one of <code>local-server-unavailable</code>, <code>upstream-unreachable</code>, <code>application-not-found</code>, <code>access-or-upstream-failure</code>, or <code>invalid-event-log</code>. It never carries an upstream message, URL, status, or response body.</p><p>Evidence availability has no dedicated worker message: the final <code>app</code> message carries a compact <code>evidenceInputs</code> counter summary, and <code>done.skippedLines</code> supplies its parse-integrity input.</p><p>Main to worker: <code>parse</code>, <code>parseFiles(files)</code> (rolling <code>eventlog_v2_*</code> directories, one continuous stream across files), <code>parseFromUrl({ baseUrl, appId, attemptId })</code>, and <code>getTaskData(stageId)</code>. The SHS request object is normalized before it reaches the worker: <code>{ baseUrl: string, appId: string, attemptId: string | null }</code>.</p><p><code>taskData</code> uses structured-clone (<code>.slice()</code>) so the worker retains its own <code>Float64Array</code> for re-renders.</p><p>Post-parse prefetch: after <code>done</code>, main runs <code>analyzer.ts</code> to build the bottleneck catalog, then fires parallel <code>getTaskData</code> for every flagged stage so their widgets render immediately. Unflagged stages are on-demand.</p><h3 id="evidence-availability-worker-input" tabindex="-1">Evidence-availability worker input <a class="header-anchor" href="#evidence-availability-worker-input" aria-label="Permalink to &quot;Evidence-availability worker input&quot;">​</a></h3><p><code>packages/core/src/event-handlers.ts</code>&#39;s <code>createState()</code>/per-event handlers start and increment this fixed <code>evidenceInputs</code> shape on the normalized app model:</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>environmentUpdates, applicationEnds, stageSubmissions, rddStorageSnapshots,</span></span>
+<span class="line"><span>sqlExecutions, resolvedSqlPlans, executorMetricRows, taskRecords</span></span></code></pre></div><p>The counters are a structured-clone-safe summary: they contain counts, not raw events, task records, host names, SQL text, paths, or Spark-property values. The final <code>app</code> message snapshots the counters after all input is processed, immediately before <code>done</code>; a consumer must not add a second worker message just for the ledger. <code>app</code> messages are also emitted mid-parse from <code>SparkListenerApplicationStart</code> and <code>SparkListenerEnvironmentUpdate</code>; those snapshots are partial and non-authoritative. Only the terminal <code>app</code> message emitted by <code>emitParseCompletion</code> (the one immediately before <code>done</code>) should be used for evidence-availability conclusions. On the main thread, <code>useIngest</code> combines the normalized <code>AppModel</code> with <code>done.skippedLines</code>, derives the ledger before calling <code>analyze()</code>, and stores it as <code>appModel.evidenceAvailability</code>.</p><h3 id="evidence-availability-contract-v1" tabindex="-1">Evidence-availability contract (V1) <a class="header-anchor" href="#evidence-availability-contract-v1" aria-label="Permalink to &quot;Evidence-availability contract (V1)&quot;">​</a></h3><p><code>packages/core/src/evidence-availability.ts</code> is the single reusable taxonomy for the browser, future report output, and future headless consumers. Its serialized ledger shape is <code>{ schemaVersion: 1, entries: EvidenceAvailabilityEntry[] }</code>. Entries are ordered by the following fixed eight-key enum:</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>executorMetrics, rddStorageSnapshots, sqlPlan, sparkConfiguration,</span></span>
+<span class="line"><span>taskCoreTime, infrastructureContext, sourceContext, costContext</span></span></code></pre></div><p>Each entry has the stable fields <code>key</code>, <code>state</code>, <code>reasonCode</code>, <code>summary</code>, and optional <code>evidence</code>. The closed V1 state enum is:</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>present, disabled, notEmitted, notApplicable, outsideEventLog, unknown</span></span></code></pre></div><p>The closed V1 reason-code enum is:</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>observed, explicitlyDisabled, noObservedExecutorMetrics,</span></span>
+<span class="line"><span>noObservedStageSubmission, noRddStorageSnapshot, noResolvedSqlPlan,</span></span>
+<span class="line"><span>noSqlExecution, noEnvironmentUpdate, noTaskRecords, noUsableCoreTimeAggregate,</span></span>
+<span class="line"><span>outsideEventLogScope, parseIncomplete</span></span></code></pre></div><p><code>summary</code> is domain-agnostic presentation copy derived from the reason code; the enum values, not labels, are the machine contract. Optional <code>evidence</code> is limited to safe compact provenance of the exact <code>{ eventType, count }</code> shape. <code>eventType</code> values are pinned to the exact <code>evidenceInputs</code> counter keys; they are not shortened or aliased. It must never include raw Spark-property values, host names, paths, SQL text, task records, raw-event identifiers, or other event payloads.</p><table tabindex="0"><thead><tr><th>Key</th><th>Proven <code>present</code> evidence</th><th>Trustworthy absence / boundary result</th></tr></thead><tbody><tr><td><code>executorMetrics</code></td><td><code>executorMetricRows &gt; 0</code></td><td>Explicit observed <code>spark.eventLog.logStageExecutorMetrics=false</code> is <code>disabled</code> / <code>explicitlyDisabled</code>; otherwise <code>notEmitted</code> / <code>noObservedExecutorMetrics</code>. Observed metrics win over the explicit setting.</td></tr><tr><td><code>rddStorageSnapshots</code></td><td><code>rddStorageSnapshots &gt; 0</code> from <code>SparkListenerStageSubmitted</code></td><td>No stage submissions is <code>notEmitted</code> / <code>noObservedStageSubmission</code>; stages submitted without any <code>RDD Info</code> rows is <code>notEmitted</code> / <code>noRddStorageSnapshot</code>. This is RDD snapshot evidence, not block-update telemetry.</td></tr><tr><td><code>sqlPlan</code></td><td><code>resolvedSqlPlans &gt; 0</code></td><td>A SQL execution without a resolved plan is <code>notEmitted</code> / <code>noResolvedSqlPlan</code>; no SQL execution is <code>notApplicable</code> / <code>noSqlExecution</code>.</td></tr><tr><td><code>sparkConfiguration</code></td><td><code>environmentUpdates &gt; 0</code></td><td><code>notEmitted</code> / <code>noEnvironmentUpdate</code>; a normalized empty config object alone is not evidence.</td></tr><tr><td><code>taskCoreTime</code></td><td><code>taskRecords &gt; 0</code> and a usable <code>runAggregates.perStage</code> task count</td><td>No task records is <code>notEmitted</code> / <code>noTaskRecords</code>; task records with no usable aggregate are <code>notEmitted</code> / <code>noUsableCoreTimeAggregate</code>. An empty aggregate object is not usable evidence.</td></tr><tr><td><code>infrastructureContext</code></td><td>none</td><td>Always <code>outsideEventLog</code> / <code>outsideEventLogScope</code>.</td></tr><tr><td><code>sourceContext</code></td><td>none</td><td>Always <code>outsideEventLog</code> / <code>outsideEventLogScope</code>.</td></tr><tr><td><code>costContext</code></td><td>none</td><td>Always <code>outsideEventLog</code> / <code>outsideEventLogScope</code>.</td></tr></tbody></table><p>Parse integrity is fail-closed. A parse is trustworthy only when it skipped no malformed lines and observed an application-end event. For an untrustworthy parse, proof already observed remains <code>present</code>, and an explicitly observed executor-metrics disablement remains <code>disabled</code>; every conclusion based on absence or irrelevance becomes <code>unknown</code> / <code>parseIncomplete</code>. The three outside-event-log product boundaries remain <code>outsideEventLog</code>. In particular, <code>notEmitted</code> and <code>notApplicable</code> are never favorable measurements inferred from an incomplete parse.</p><p>Availability is model metadata, not a detector or finding: it never changes detector thresholds or ordering, impact band, catalog/scorecard counts, no-bottleneck behavior, or detector suppression. <code>session-snapshot.ts</code> captures and restores <code>evidenceAvailability</code> with the normalized model, so a recent-file switch retains the same ledger without reparsing.</p><h3 id="portable-evidence-report-v1" tabindex="-1">Portable evidence report (V1) <a class="header-anchor" href="#portable-evidence-report-v1" aria-label="Permalink to &quot;Portable evidence report (V1)&quot;">​</a></h3><p><code>packages/core/src/evidence-report.ts</code>&#39;s <code>buildEvidenceReport(appModel, { redact })</code> returns <code>{ markdown, json }</code>: a self-contained, byte-stable document that runs the detectors over an <code>appModel</code> and serializes the result for sharing outside the tool. Raw task records are never included; identifier redaction (app id + host names → <code>app-1</code>/<code>host-1</code> pseudonyms via <code>packages/core/src/redact.ts</code>) is opt-in with <code>{ redact: true }</code>. The JSON is pinned by <code>EVIDENCE_SCHEMA_VERSION</code> (currently <code>3</code>, surfaced as <code>json.schemaVersion</code>) and has this fixed top-level key order:</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>schemaVersion, summary, evidenceAvailability, detectors, findings, recommendations, cleanChecks</span></span></code></pre></div><ul><li><code>summary</code> is the run header: <code>{ app: { id, name, sparkVersion }, stageCount, jobCount, sqlExecutionCount, findingCount, impactBandCounts }</code>.</li><li><code>evidenceAvailability</code> is the ledger above (or <code>null</code> when absent).</li><li><code>detectors</code> is <code>detectorCatalog()</code> output: one <code>{ type, version, scope, thresholds, docAnchor }</code> per detector, in <code>DETECTORS</code> order, so the exact threshold set that produced each finding travels with the evidence.</li><li><code>findings</code> are deterministically sorted rows (impact band → type → stage → id), each with a stable <code>id</code>, <code>tag</code>, core columns, an always-present <code>actionLabel</code>, and an <code>evidence</code> sub-object for non-core fields; <code>confidence</code>/ <code>validationRequired</code>/<code>docAnchor</code> appear only when the detector emitted them.</li><li><code>recommendations</code> is the impact-ranked <code>buildRecommendationRollup</code> output (<code>packages/core/src/recommendation-rollup.ts</code>), and <code>cleanChecks</code> lists every detector type that fired zero findings this run: see the 2026-09-03 update below.</li></ul><p>Determinism holds because detector order, finding sort, and object key order are all fixed, so a given <code>appModel</code> serializes identically across calls.</p><p>Redaction enumerates hosts two ways: by walking the findings tree for every string value under a key literally named <code>host</code> (<code>evidence.host</code>, <code>evidence.failedTaskDetails[].host</code>, <code>evidence.retriedTaskDetails[].host</code>, and any future nested <code>host</code> field, all covered without enumerating paths), and by scanning every string value for EC2-style hostnames / bare IPv4 tokens. So identifiers that surface only in free text (recommendation copy, a <code>stageFailed</code> failure-reason value) are pseudonymized too. Pseudonym numbering uses a numeric-aware sort, so re-redacting an already-redacted report is a no-op even past <code>host-10</code>.</p><p>The Markdown rendering mirrors the JSON&#39;s AC3 field set: each finding block prints its <code>detector version</code>, its sorted <code>evidence</code> entries (byte-magnitude keys humanized), and the report ends with a <code>## Detectors</code> catalog carrying the version + threshold set. A finding&#39;s <code>impactEstimate</code> (when its <code>basis</code> isn&#39;t <code>&#39;informational&#39;</code>) prints as its own <code>- impact: </code> line (<code>Estimated &lt;low&gt;-&lt;high&gt;</code> and/or the raw-waste figure, plus <code>estimateMethod</code>), via <code>renderImpactEstimate</code>/<code>formatWallClockRange</code>/<code>formatRawWaste</code> in <code>packages/core/src/evidence-report.ts</code>. <code>EvidenceExport</code> names downloads <code>evidence-&lt;appId&gt;[-redacted].&lt;md|json&gt;</code>, taking the app id from the (already pseudonymized when redacting) report so a redacted file never leaks the real id and is never name-identical to a raw export.</p><p>Decision 9 (design spec): <code>Finding</code>&#39;s <code>impactEstimate</code> field, plus the <code>utilizationFraction</code>/<code>memorySize</code>/<code>diskSize</code>/<code>numCachedPartitions</code>/<code>numPartitions</code> instrumentation fields the impact estimator reads, were added without bumping <code>EVIDENCE_SCHEMA_VERSION</code> past <code>1</code>. Both additions are purely optional and ride <code>Finding</code>&#39;s existing optional-field-plus-catch-all convention, so an evidence report built before these fields existed still deserializes and compares byte-for-byte against one built after: nothing about the schema&#39;s stability guarantee changed, only its surface grew. A future reader who notices <code>impactEstimate</code> in the JSON without a schema bump is looking at this deliberate call, not an oversight.</p><p>2026-08-30 update: <code>EVIDENCE_SCHEMA_VERSION</code> was bumped to <code>2</code> for the occupancy-weighted attribution redesign (see <a href="./impact-estimation.html#occupancy-weighted-attribution">Occupancy-weighted attribution</a>): <code>ImpactEstimate</code>&#39;s shape changed from <code>{low, high}</code> to <code>{basis, wallClock, estimateMethod, rawWaste?}</code>, a real, non-additive breaking change to a field this same Decision 9 previously shipped without a bump. <code>impactEstimate</code> had zero consumers outside <code>packages/core/src/impact-estimator.ts</code> and its own tests at the time of this bump (confirmed by grep across <code>src/view/*</code> and <code>packages/core/src/evidence-report.ts</code>), so no other code needed migrating alongside it.</p><p><code>FindingRow</code>&#39;s schema was extended to surface <code>impactEstimate</code> as a first-class column without further bumping <code>EVIDENCE_SCHEMA_VERSION</code> past <code>2</code>, consistent with <code>Finding</code>&#39;s existing optional-field-plus-catch-all convention. It appears after the pinned core columns (<code>id</code>, <code>type</code>, <code>impactBand</code>, <code>stageId</code>, <code>metric</code>, <code>value</code>, <code>recommendation</code>, <code>detectorVersion</code>, plus optional but pinned <code>confidence</code>, <code>validationRequired</code>, <code>docAnchor</code>) without displacing any of them; byte-for-byte deserializability of existing reports is preserved. <code>FindingRow.impactEstimate</code> carries the full contract documented in <a href="./impact-estimation.html#occupancy-weighted-attribution">Impact estimation</a> (basis, wallClock, estimateMethod, rawWaste).</p><p>2026-09-03 update: the &quot;Fix These First&quot; dashboard redesign (impact-ranked recommendation rollup, short per-finding action labels, a clean-checks table) was UI-only when it shipped; this update ports the underlying data into <code>buildEvidenceReport()</code> so the CLI (<code>packages/cli/bin/sparkforensics-analyze.mjs</code>) and the MCP tool (<code>diagnoseRun</code> in <code>packages/core/src/mcp-tools.ts</code>) get it too, not just the web markdown/JSON download. Three additions, all purely additive, so this does not bump <code>EVIDENCE_SCHEMA_VERSION</code> past <code>2</code>, the same rationale as the <code>impactEstimate</code> addition immediately above: <code>FindingRow.actionLabel</code> is now always present (a short imperative label like &quot;Reduce shuffle size&quot;), sourced from a new core module, <code>packages/core/src/finding-action-label.ts</code>&#39;s <code>coreFindingActionLabel</code>, extracted from the (type, discriminant) switch statement that used to live only in the view layer (<code>src/view/finding-action-label.ts</code>, which now wraps the core function and layers its own <code>REGISTRY</code> fallback on top). <code>EvidenceReportJson.recommendations</code> is the same impact-ranked <code>buildRecommendationRollup</code> grouping (<code>packages/core/src/recommendation-rollup.ts</code>) that <code>FixTheseFirst.tsx</code> renders, so CLI/MCP/download consumers get the same &quot;what&#39;s the highest-impact fix&quot; ranking the dashboard shows, without changing the existing <code>findings</code> array&#39;s own impact-band-sorted order at all. <code>EvidenceReportJson.cleanChecks</code> lists every detector type that fired zero findings this run, each with <code>getThresholdSummary</code>&#39;s one-line &quot;what would have tripped it&quot; sentence; unlike the dashboard&#39;s <code>Alerts.tsx</code> clean-checks table, it deliberately includes the one &quot;always-mounted&quot; reference type (<code>coreLocality</code>) even when it has no findings, since a flat evidence report has no separate always-visible surface for it to already appear on the way that widget does on the board.</p><h3 id="finding-identity" tabindex="-1">Finding identity <a class="header-anchor" href="#finding-identity" aria-label="Permalink to &quot;Finding identity&quot;">​</a></h3><p>Every finding <code>analyzer.ts</code> emits carries a stable <code>id</code> (<code>push()</code>&#39;s <code>findingId()</code> choke point, FNV-1a hash over <code>type | location | metric | value | discriminators</code>) and a <code>detectorVersion</code> (from the emitting <code>DETECTORS</code> entry). The two are deliberately decoupled: <code>id</code> is derived only from a finding&#39;s evidence tuple, never from <code>detectorVersion</code>, so a threshold or logic tweak that bumps a detector&#39;s <code>version</code> does not change the <code>id</code> of findings it still emits at the same location/metric. A saved reference (dashboard bookmark, exported report row, future URL-restored filter) keeps pointing at the same logical finding across detector revisions; <code>detectorVersion</code> is separate provenance metadata for &quot;which ruleset produced this,&quot; not part of identity. See <code>packages/core/test/analyzer-finding-identity.test.js</code> for the decoupling proof.</p><p><code>id</code> stability holds for equivalent reruns of the same schema/analyzer version on the same input. Changing the id-derivation rule itself (the hash algorithm, or which fields feed the location key/discriminators) is an intentional breaking change and must bump <code>EVIDENCE_SCHEMA_VERSION</code> (<code>packages/core/src/evidence-report.ts</code>); there is no separate id-scheme version.</p><h3 id="headless-analysis-cli-v1" tabindex="-1">Headless analysis CLI (V1) <a class="header-anchor" href="#headless-analysis-cli-v1" aria-label="Permalink to &quot;Headless analysis CLI (V1)&quot;">​</a></h3><p><code>packages/cli/bin/sparkforensics-analyze.mjs</code> runs the same parser + detector contracts outside the browser, for CI. It accepts a single event-log file or a rolling-log directory, drives <code>runParse</code>/<code>runParseFiles</code> (<code>packages/core/src/parser-worker.ts</code>) with a Node-only File-like shim (<code>packages/core/src/cli/collect-run.ts</code>), and writes the exact <code>buildEvidenceReport</code> JSON schema described above: one format, not a second. Alternatively, <code>--shs-base-url &lt;url&gt; --app-id &lt;id&gt; [--attempt-id &lt;id&gt;]</code> fetches the run from a Spark History Server instead (mutually exclusive with the positional file/directory argument), calling <code>resolveFromShs</code> (<code>packages/core/src/shs-load.ts</code>, shared with the MCP server&#39;s SHS source path below) directly; no dependency on the <code>packages/server</code> package. A failed SHS fetch reports its message to <code>stderr</code> and exits <code>2</code>, same as a local file that can&#39;t be parsed.</p><p>Optional CLI-flag budgets (<code>--max-runtime &lt;ms&gt;</code>, <code>--max-spill &lt;gb&gt;</code>, <code>--max-skew &lt;ratio&gt;</code>, <code>--max-failed-task-rate &lt;pct&gt;</code>, <code>--min-efficiency &lt;pct&gt;</code>) are evaluated in <code>packages/core/src/cli/budgets.ts</code> against the existing finding catalog (<code>analyze()</code>) and <code>computeEfficiencyModel</code>; there is no second rule engine. A budget whose required evidence is missing (e.g. the run never emitted <code>ApplicationEnd</code>, or has no usable per-task <code>runAggregates</code>) is reported as inconclusive (<code>stderr</code> warning) rather than silently passing, and gets its own exit code distinct from both pass and violation. Exit codes: <code>0</code> pass, <code>1</code> a configured budget was violated, <code>2</code> the input could not be parsed at all, <code>3</code> no violations but at least one budget was inconclusive. A violation always wins over an inconclusive result in the same run (exit <code>1</code>, not <code>3</code>).</p><h3 id="mcp-server-v1" tabindex="-1">MCP server (V1) <a class="header-anchor" href="#mcp-server-v1" aria-label="Permalink to &quot;MCP server (V1)&quot;">​</a></h3><p><code>packages/core/src/mcp-server-factory.ts</code>&#39;s <code>createMcpServer()</code> registers 6 tools: <code>diagnose_run</code> (thresholded findings + remediation text), <code>get_run_summary</code> (app/stage/job/sql counts and duration, no findings), <code>compare_runs</code> (categorized findings delta + metric deltas between two runs), <code>evaluate_budgets</code> (pass/fail budget thresholds against one run, optionally with a second run for regression/fail-on-introduced budgets: the MCP side of the CLI&#39;s <code>evaluateBudgets()</code> gating), <code>get_finding_evidence</code> (raw evidence bundle for one finding, for drill-down after <code>diagnose_run</code>), and <code>get_finding_documentation</code> (detection/tuning reference docs for one finding type, independent of any run). None re-implement detector logic; all repackage <code>analyze</code>/<code>buildEvidenceReport</code>/<code>compareRuns</code>/<code>captureSnapshot</code>/<code>evaluateBudgets</code> from <code>packages/core/src/mcp-tools.ts</code>, which resolves a <code>source</code> (event-log <code>path</code>, or an SHS <code>shsBaseUrl</code>/<code>appId</code>/<code>attemptId</code> triple) into a cached <code>AppModel</code>. The SHS source path calls <code>resolveFromShs</code> (<code>packages/core/src/shs-load.ts</code>, also used directly by the CLI&#39;s <code>--shs-base-url</code> mode above), which reuses two extractions shared with the browser ingestion flow: <code>fetchShsEventLog</code> (<code>packages/core/src/proxy.js</code>) fetches the zip, and <code>decodeShsArchive</code> (<code>packages/core/src/shs-fetch.ts</code>, re-exported from <code>parser-worker.ts</code>&#39;s barrel) decodes it into a parsed <code>AppModel</code>, the same split <code>runParseFromUrl</code> itself calls into.</p><p>Two transports connect to that one factory: <code>packages/mcp/bin/sparkforensics-mcp.mjs</code> (stdio, for local MCP clients) and <code>packages/server/index.js</code>&#39;s <code>/mcp</code> route (streamable HTTP, for the local server). The run cache (<code>packages/core/src/mcp-tools.ts</code>) is a module-level LRU (cap 8 and 15-minute idle TTL by default, overridable via <code>SPARKFORENSICS_MCP_CACHE_CAP</code> and <code>SPARKFORENSICS_MCP_CACHE_TTL_MS</code>, lazily swept on access) keyed by resolved source (path mtime, or SHS baseUrl+appId+attemptId), so a client mints a <code>runId</code> once via <code>resolveOrCreateRun</code> and reuses it across subsequent tool calls instead of re-parsing.</p><p>Every tool failure comes back as <code>{isError: true, content: [...], structuredContent: {code}}</code>, never an HTTP-status-shaped error; <code>code</code> is one of the 5 existing SHS codes (<code>SHS_ERROR_CODES</code>, <code>packages/core/src/shs-request.js</code>) plus <code>run-not-found</code>, <code>finding-not-found</code>, <code>invalid-event-log</code> (also covers a nonexistent <code>path</code> source), and <code>archive-too-large</code> (SHS archive over the <code>SPARKFORENSICS_MAX_ARCHIVE_BYTES</code> byte cap, default 1 GiB, because the MCP path buffers the whole archive in memory, unlike the streaming <code>/shs-proxy</code> route).</p><p><code>scripts/vendor-core.mjs</code> (shared by <code>packages/cli</code>, <code>packages/mcp</code>, and <code>packages/server</code>&#39;s <code>prepack</code> scripts) vendors <code>packages/core/src/</code> wholesale into each package&#39;s own <code>vendor-core/</code> at pack time, pre-stripping TypeScript to plain <code>.js</code> (Node&#39;s native TS stripping refuses to run on <code>.ts</code> files under <code>node_modules</code>, which is exactly where a published <code>vendor-core/</code> lands). Each package&#39;s bin/entry point resolves its needed module from <code>vendor-core/</code> if present, else falls back to the real <code>packages/core/src/</code> sibling loaded as <code>.ts</code> directly (<code>packages/server/index.js</code> mirrors <code>resolveStaticRoot</code>&#39;s <code>public/</code>-vs-<code>../dist</code> pattern for this same fallback).</p><h2 id="typescript-core-and-runtime-event-validation" tabindex="-1">TypeScript core and runtime event validation <a class="header-anchor" href="#typescript-core-and-runtime-event-validation" aria-label="Permalink to &quot;TypeScript core and runtime event validation&quot;">​</a></h2><p>All of <code>src/</code> (browser SPA) and <code>packages/core/src/</code> (shared analysis logic) is strict TypeScript; the only plain-<code>.js</code> holdouts are the two vendored third-party decompressors, <code>packages/core/src/vendor/fflate.js</code> and <code>packages/core/src/vendor/fzstd.js</code> (left untouched deliberately: vendored code, not project code). Every remaining import of a same-repo module uses a <code>.ts</code> specifier (e.g. <code>import { dispatchLine } from &#39;./event-handlers.ts&#39;</code>), not <code>.js</code>: the CLI and MCP entrypoints (<code>packages/cli/bin/sparkforensics-analyze.mjs</code>, <code>packages/mcp/bin/sparkforensics-mcp.mjs</code>) run under plain Node&#39;s ESM resolver, which cannot remap a <code>.js</code> specifier to a same-named <code>.ts</code> file the way Vite/Vitest&#39;s bundler-style resolver can. <code>tsconfig.json</code> sets <code>&quot;allowImportingTsExtensions&quot;: true</code> to make this legal.</p><p>Event schema validation lives in <code>packages/core/src/event-schemas.ts</code>: one zod schema per <code>SparkListener*</code> event variant the parser understands (15 total: <code>LogStart</code>, <code>ApplicationStart</code>, <code>EnvironmentUpdate</code>, <code>ApplicationEnd</code>, <code>JobStart</code>, <code>JobEnd</code>, <code>StageSubmitted</code>, <code>StageCompleted</code>, <code>StageExecutorMetrics</code>, <code>TaskEnd</code>, the three SQL-execution-UI listener events, <code>ExecutorAdded</code>, <code>ExecutorRemoved</code>), combined into <code>SparkEventSchema = z.discriminatedUnion(&#39;Event&#39;, [...])</code>. <code>processEvent</code>&#39;s switch (<code>event-handlers.ts</code>) consumes the resulting <code>SparkEvent</code> union type directly, so a schema change and a handler&#39;s expectations can&#39;t silently drift apart.</p><p>The recursive <code>sparkPlanInfo.children</code> tree carried on SQL-execution-start events is validated by <code>parseSparkPlanInfoTree</code>, an iterative, explicit heap-allocated-stack parser, deliberately not <code>z.lazy()</code>. A real plan tree&#39;s depth is unbounded, and attacker-uncontrolled input should never hand zod&#39;s own recursive schema resolution an arbitrarily deep structure to walk. The iterative parser shallow-validates one node at a time via <code>z.object(...)</code> and builds the tree itself, capping at <code>MAX_PLAN_DEPTH = 500</code> (throws past that). This mirrors <code>packages/core/src/plan-tree-walk.ts</code>&#39;s <code>walkPlanTree</code>, which uses the same iterative-over-recursive approach for the <em>resolved</em> tree; <code>event-schemas.ts</code> applies it one layer earlier, to the raw JSON before it becomes a tree at all.</p><p>There are two external-data boundaries, the two places this codebase parses data it does not control. Both now run that data through a schema, and both treat a validation failure the same way: a silent skip, not a distinct error.</p><ul><li><code>dispatchLine</code> (<code>event-handlers.ts</code>): after <code>JSON.parse</code> succeeds, a line whose <code>Event</code> value is one of the 15 modeled types but fails that type&#39;s own schema now increments <code>skippedLines</code> (previously: no shape validation existed at all, and a malformed event silently corrupted downstream state with no signal anywhere). An <code>Event</code> value outside the 15 modeled types is still silently ignored without incrementing <code>skippedLines</code>, unchanged from before migration (see the note below on why the broader design was rejected).</li><li><code>shs-fetch.ts</code>: on a non-OK response from the local SHS proxy, the JSON error envelope is validated against <code>ShsProxyErrorBodySchema</code> (<code>packages/core/src/shs-schemas.ts</code>, <code>{ code: string }</code>, <code>.passthrough()</code>). A body that isn&#39;t valid JSON, or is JSON but fails that schema, falls back to the generic <code>access-or-upstream-failure</code> code: the same silent-skip treatment as a malformed log line, collapsing what could have been a separate &quot;malformed SHS response&quot; error code into the existing generic one.</li></ul><p>Neither boundary distinguishes &quot;malformed JSON&quot; from &quot;wrong shape&quot; from &quot;unrecognized variant&quot; in what it reports outward: all three collapse into the same skip/fallback path. That was a deliberate scope decision. The correction below says why the <em>line</em> boundary stops there rather than flagging every unrecognized event type.</p><blockquote><p>Correction made during migration, not part of the original design: an earlier draft counted <em>any</em> <code>Event</code> value outside the 15 modeled types toward <code>skippedLines</code>, not just ones that fail their own schema. Running that design against real Spark event logs (which always contain plenty of ordinary event types this tool has never modeled: <code>TaskStart</code>, <code>BlockManagerAdded</code>, <code>ExecutorMetricsUpdate</code>, and others, always silently ignored pre-migration) pushed <code>skippedLines</code> from 0 to the tens of thousands on completely healthy logs. That would have tripped <code>evidence-availability.ts</code>&#39;s fail-closed <code>trustworthy</code> gate and shown a false &quot;malformed JSON&quot; warning on every real file. Caught in review against real fixtures, not synthetic ones; fixed to the narrower rule described above before merging. <code>evidence-availability.ts</code>&#39;s gate itself (<code>trustworthy = skippedLines === 0 &amp;&amp; applicationEnds &gt; 0</code>) needed no code change once this was fixed: it was already correct, the input feeding it was not.</p></blockquote><p>The exhaustiveness convention is <code>packages/core/src/assert-never.ts</code>. <code>assertNever(x: never): never</code> throws at runtime and, more importantly, fails <code>tsc</code> at compile time if <code>x</code> is not actually <code>never</code>, i.e. if some case of a union type isn&#39;t handled. Used today at the <code>default</code> arm of <code>event-handlers.ts</code>&#39;s <code>processEvent</code> switch (over <code>SparkEvent</code>) and <code>analyzer.ts</code>&#39;s scope-dispatch switch (over a detector&#39;s <code>scope: &#39;stage&#39; | &#39;sql&#39; | &#39;app&#39; | &#39;config&#39;</code>). It is the project&#39;s standard pattern for any future exhaustive switch or dispatch over a closed union: add <code>default: return assertNever(x);</code> (or the closest non-returning equivalent) so that adding a new union member without updating every consumer becomes a compile error instead of a silent runtime gap.</p>`,55)])])}const m=o(c,[["render",s]]);export{u as __pageData,m as default};

package/export-template/docs/assets/contributor-guide_architecture_worker-protocol.md.IbnfNrV3.lean.js

@@ -0,0 +1 @@
+import{_ as o,o as t,c as d,a5 as a}from"./chunks/framework.DSg0KOwT.js";const u=JSON.parse('{"title":"Worker protocol","description":"","frontmatter":{},"headers":[],"relativePath":"contributor-guide/architecture/worker-protocol.md","filePath":"contributor-guide/architecture/worker-protocol.md"}'),c={name:"contributor-guide/architecture/worker-protocol.md"};function s(r,e,i,n,l,p){return t(),d("div",null,[...e[0]||(e[0]=[a("",55)])])}const m=o(c,[["render",s]]);export{u as __pageData,m as default};

package/export-template/docs/assets/contributor-guide_contributing.md.CvRsdr6J.js

@@ -0,0 +1 @@
+import{_ as o,o as t,c as i,a5 as n}from"./chunks/framework.DSg0KOwT.js";const u=JSON.parse('{"title":"Contributing","description":"","frontmatter":{},"headers":[],"relativePath":"contributor-guide/contributing.md","filePath":"contributor-guide/contributing.md"}'),s={name:"contributor-guide/contributing.md"};function a(c,e,r,d,h,l){return t(),i("div",null,[...e[0]||(e[0]=[n('<h1 id="contributing" tabindex="-1">Contributing <a class="header-anchor" href="#contributing" aria-label="Permalink to &quot;Contributing&quot;">​</a></h1><h2 id="where-does-new-content-go" tabindex="-1">Where does new content go? <a class="header-anchor" href="#where-does-new-content-go" aria-label="Permalink to &quot;Where does new content go?&quot;">​</a></h2><ul><li><code>docs/</code> is for internal engineering records: ADRs, audits, competitive research. It stays flat, unstructured markdown, read by contributors through GitHub&#39;s own file viewer.</li><li><code>docs-site/</code> (this site) is the published documentation for users and contributors. If you&#39;re writing something a user or a first-time contributor should read, it belongs here, not in <code>docs/</code>.</li></ul><h2 id="linting-before-you-commit" tabindex="-1">Linting before you commit <a class="header-anchor" href="#linting-before-you-commit" aria-label="Permalink to &quot;Linting before you commit&quot;">​</a></h2><p><code>npm install</code> runs the <code>prepare</code> script, which points git at <code>.githooks/</code> (<code>git config core.hooksPath .githooks</code>). After that, every commit runs <code>.githooks/pre-commit</code>: it lints any staged <code>.js</code>/<code>.mjs</code> files with eslint and blocks the commit if linting fails. <code>git commit --no-verify</code> skips it. To run the same linter by hand, use <code>npm run lint</code>.</p><h2 id="changesets-and-releases" tabindex="-1">Changesets and releases <a class="header-anchor" href="#changesets-and-releases" aria-label="Permalink to &quot;Changesets and releases&quot;">​</a></h2><p>A changeset is required for a PR that touches <code>packages/cli</code>, <code>packages/mcp</code>, <code>packages/server</code>, or the root site app (<code>src/</code>, <code>index.html</code>, <code>vite.config.ts</code>, or the root <code>package.json</code>). Run <code>npx changeset add</code>, answer its prompts, and commit the generated <code>.changeset/*.md</code> file. CI&#39;s <code>changeset</code> job (<code>scripts/check-changeset.sh</code>) fails the PR otherwise, with instructions to fix it. <code>docs-site/</code> and <code>docs/</code> are exempt: doc-only changes don&#39;t need one.</p><p>Needing a changeset doesn&#39;t mean getting published, though. Only <code>packages/cli</code>, <code>packages/mcp</code>, and <code>packages/server</code> publish to npm. The root site (<code>sparkforensics</code> itself) is <code>&quot;private&quot;: true</code> and never published; its changeset only bumps its <code>package.json</code> version and writes a <code>CHANGELOG.md</code> entry, via <code>.changeset/config.json</code>&#39;s <code>privatePackages.version</code> setting. <code>@sparkforensics/core</code> is the other private package but stays out of this entirely (<code>.changeset/config.json</code>&#39;s <code>ignore</code> list): it&#39;s vendored into cli/mcp by filesystem copy at pack time, not read as a real dependency, so versioning it would have no consumer.</p><p>A release itself is two merges, not one: merging your feature PR into <code>main</code> runs <code>.github/workflows/release.yml</code>, which opens (or updates) a &quot;Version Packages&quot; PR collecting all pending changesets. Nothing publishes yet. Merging <em>that</em> PR is what actually bumps versions, writes changelogs, and runs <code>npm publish</code> for whichever of the three publishable packages had pending changesets.</p><p>Maintainers: this pipeline depends on the three package names being claimed on npmjs.com and Trusted Publishing being registered against this repo (a one-time manual bootstrap step, tracked separately). Until that&#39;s done, every push to <code>main</code> will fail at the publish step. Check whether it&#39;s been done before merging a &quot;Version Packages&quot; PR.</p><p>For the full setup and test workflow, see <a href="./development-setup.html">Development setup</a> and <a href="./testing.html">Testing &amp; verification</a>.</p>',11)])])}const p=o(s,[["render",a]]);export{u as __pageData,p as default};

package/export-template/docs/assets/contributor-guide_contributing.md.CvRsdr6J.lean.js

@@ -0,0 +1 @@
+import{_ as o,o as t,c as i,a5 as n}from"./chunks/framework.DSg0KOwT.js";const u=JSON.parse('{"title":"Contributing","description":"","frontmatter":{},"headers":[],"relativePath":"contributor-guide/contributing.md","filePath":"contributor-guide/contributing.md"}'),s={name:"contributor-guide/contributing.md"};function a(c,e,r,d,h,l){return t(),i("div",null,[...e[0]||(e[0]=[n("",11)])])}const p=o(s,[["render",a]]);export{u as __pageData,p as default};

package/export-template/docs/assets/contributor-guide_development-setup.md.DvAN_9mK.js

@@ -0,0 +1,12 @@
+import{_ as i,o as a,c as e,a5 as t}from"./chunks/framework.DSg0KOwT.js";const c=JSON.parse('{"title":"Development setup","description":"","frontmatter":{},"headers":[],"relativePath":"contributor-guide/development-setup.md","filePath":"contributor-guide/development-setup.md"}'),n={name:"contributor-guide/development-setup.md"};function p(h,s,l,k,r,d){return a(),e("div",null,[...s[0]||(s[0]=[t(`<h1 id="development-setup" tabindex="-1">Development setup <a class="header-anchor" href="#development-setup" aria-label="Permalink to &quot;Development setup&quot;">​</a></h1><p>Clone the repo, then:</p><div class="language-bash vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">bash</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">git</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> submodule</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> update</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> --init</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> dev/log-corpus</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">   # or: git clone --recurse-submodules</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> install</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> test</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">              # vitest run: the full unit/component suite</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> run</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> test:watch</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">    # vitest, watch mode</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> run</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> dev</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">           # Vite dev server for the app itself</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> run</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> build</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">         # production build, outputs dist/</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> run</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> preview</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">       # serves dist/ locally</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npx</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> tsc</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> --noEmit</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">      # typecheck (strict TypeScript across src/)</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> run</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> lint</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">          # eslint over the repo</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">node</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> packages/cli/bin/sparkforensics-analyze.mjs</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> &lt;</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">file</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">|</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">dir</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">&gt;   </span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;"># run the CLI analyzer against a log, outside the browser</span></span></code></pre></div><p><code>npm install</code> also runs the <code>prepare</code> script, which points git at <code>.githooks/</code>. After that, a pre-commit hook lints staged JS files on every commit. Fix what it reports, or skip it with <code>git commit --no-verify</code>.</p><p><code>dev/log-corpus</code> is a git submodule pointing at the private <code>spark-event-corpus-data</code> repo. If you don&#39;t have access to it, skip the <code>git submodule update</code> step above: <code>packages/server/test/shs-proxy-fixture.test.js</code> will report as skipped rather than failed, which is expected, not a bug.</p><p>The <code>server/</code> package (the optional local-server deploy mode) keeps its own dependencies and test suite. Run them separately:</p><div class="language-bash vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">bash</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> run</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> test:server</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">   # runs server/&#39;s own suite, not part of \`npm test\`</span></span></code></pre></div><p>Build the docs site before packing or releasing <code>server/</code>: run <code>npm run docs:build</code> from the repo root, or the packed artifact ships without <code>/docs/</code>.</p><p>This docs site has its own dev loop:</p><div class="language-bash vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">bash</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> run</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> docs:dev</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">       # VitePress dev server for docs-site/</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> run</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> docs:build</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">     # production build; also checks internal links/anchors</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">npm</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> run</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> docs:preview</span><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">   # serves the built docs site locally</span></span></code></pre></div>`,10)])])}const F=i(n,[["render",p]]);export{c as __pageData,F as default};

package/export-template/docs/assets/contributor-guide_development-setup.md.DvAN_9mK.lean.js

@@ -0,0 +1 @@
+import{_ as i,o as a,c as e,a5 as t}from"./chunks/framework.DSg0KOwT.js";const c=JSON.parse('{"title":"Development setup","description":"","frontmatter":{},"headers":[],"relativePath":"contributor-guide/development-setup.md","filePath":"contributor-guide/development-setup.md"}'),n={name:"contributor-guide/development-setup.md"};function p(h,s,l,k,r,d){return a(),e("div",null,[...s[0]||(s[0]=[t("",10)])])}const F=i(n,[["render",p]]);export{c as __pageData,F as default};

package/export-template/docs/assets/contributor-guide_testing.md.6rIKqSyY.js

@@ -0,0 +1 @@
+import{_ as t,o,c as s,a5 as a}from"./chunks/framework.DSg0KOwT.js";const h=JSON.parse('{"title":"Testing & verification","description":"","frontmatter":{},"headers":[],"relativePath":"contributor-guide/testing.md","filePath":"contributor-guide/testing.md"}'),d={name:"contributor-guide/testing.md"};function r(i,e,c,n,l,u){return o(),s("div",null,[...e[0]||(e[0]=[a('<h1 id="testing-verification" tabindex="-1">Testing &amp; verification <a class="header-anchor" href="#testing-verification" aria-label="Permalink to &quot;Testing &amp; verification&quot;">​</a></h1><p>Before opening a PR:</p><ol><li>Run the full suite: <code>npm test</code> (root) and, if you touched <code>server/</code>, <code>npm run test:server</code> too: it&#39;s a separate package with its own dependencies, not covered by the root suite.</li><li>Typecheck: <code>npx tsc --noEmit</code>.</li><li>Lint: <code>npm run lint</code>. This also runs automatically on every commit via <code>.githooks/pre-commit</code>, but run it yourself so you catch failures before committing.</li><li>For any UI change, verify it in a real browser (Playwright or headless Chrome). Unit and jsdom tests check code correctness; they don&#39;t prove a feature works end-to-end. Drive the app, exercise the golden path and the edge cases, and look for regressions elsewhere on the dashboard.</li><li>Report the passing test count when you&#39;re done, not &quot;tests pass.&quot;</li></ol><p>Sample Spark event logs for manual testing live outside this repo. Keep them in a sibling <code>../spark-log-examples/</code> directory: real <code>.zstd</code> event logs plus baseline/candidate pairs for run comparison.</p><h2 id="testing-layout" tabindex="-1">Testing layout <a class="header-anchor" href="#testing-layout" aria-label="Permalink to &quot;Testing layout&quot;">​</a></h2><p>Vitest throughout, two environments in one run. Core logic tests (<code>tests/*.test.js</code>, covering parser, analyzer, detectors, format-utils, etc.) run under Node. View tests (<code>tests/view/*.test.tsx</code>) declare <code>// @vitest-environment jsdom</code> and use React Testing Library (<code>render</code>/<code>screen</code>/<code>userEvent</code>). There are no hand-rolled DOM-mounting jsdom tests left: the old <code>tests/widgets/*.test.js</code> suite called <code>src/widgets/*.js</code>&#39;s <code>renderX(mount, ...)</code> functions directly, and every behavior it covered now has an RTL equivalent.</p><p><code>server/</code> is a separate npm package with its own Vitest config and <code>npm test</code>, not run by the root suite. One test there, <code>server/test/bin.test.js</code>, packs and installs the real published tarball and spawns the real <code>.bin</code> entry point: the only test that exercises the production build end-to-end.</p><p>Two contract tests guard invariants that a future edit could silently break. The &quot;detector contract&quot; suite in <code>tests/analyzer.test.js</code> asserts <code>stageSlowness</code> stays array-index-after <code>slowHost</code> in <code>DETECTORS</code> (see <a href="./architecture/detector-contract.html#detector-contract">Detector contract</a>). <code>tests/view/detector-registry.test.tsx</code> asserts <code>REGISTRY</code> completeness against every <code>DETECTORS</code> type, including the <code>underBroadcast</code>/<code>overBroadcast</code> correction: <code>broadcastSizing</code> itself is never an emitted <code>finding.type</code>, as the doc comment above <code>REGISTRY</code> in <code>src/view/detector-registry.tsx</code> explains.</p><h2 id="test-fixtures" tabindex="-1">Test fixtures <a class="header-anchor" href="#test-fixtures" aria-label="Permalink to &quot;Test fixtures&quot;">​</a></h2><p>Example event logs live under <code>examples/</code>. They are real Spark History NDJSON logs, not text, and gitignored for size.</p><ul><li><code>small-application_1777489669889_56601</code> (~76 MB): fast end-to-end smoke test.</li><li><code>big-application_1777489669889_51251_1</code> (~1.3 GB): stress-test the Web Worker streaming path.</li><li><code>lz4-application_1777393442674_464993</code> (~120 MB): decompressed from a real Spark History Server download. Use it to check the <code>LZ4Block</code> codec decoder by hand. The automated suite never touches it; <code>tests/lz4-block.test.js</code> uses a small hardcoded byte array instead.</li></ul><p>Of the finding types moved onto the Gold Standard row/expand contract in the widget-gold-standard plan, only <code>retryWaste</code> and <code>autoscalingChurn</code> fire on any of the 15 real logs in <code>../spark-log-examples/</code>. <code>failures</code>, <code>stageFailed</code>, and <code>jobFailureRate</code> fire on none of them; they were verified with unit tests only (<code>tests/view/failures.test.tsx</code>, <code>tests/view/job-failures.test.tsx</code>), not against a real log.</p><p><code>dev/log-corpus/</code> is a git submodule pointing at the private <code>spark-event-corpus-data</code> repo, pinned to a tag. It backs <code>packages/server/test/shs-proxy-fixture.test.js</code>, which loads a real <code>*-parquet-baseline.ndjson</code> fixture and asserts the <code>/shs-proxy</code> route streams it back unmodified. Populate it locally with <code>git submodule update --init dev/log-corpus</code>; without it, the test suite skips (see <a href="./development-setup.html">Development setup</a>).</p>',13)])])}const g=t(d,[["render",r]]);export{h as __pageData,g as default};

package/export-template/docs/assets/contributor-guide_testing.md.6rIKqSyY.lean.js

@@ -0,0 +1 @@
+import{_ as t,o,c as s,a5 as a}from"./chunks/framework.DSg0KOwT.js";const h=JSON.parse('{"title":"Testing & verification","description":"","frontmatter":{},"headers":[],"relativePath":"contributor-guide/testing.md","filePath":"contributor-guide/testing.md"}'),d={name:"contributor-guide/testing.md"};function r(i,e,c,n,l,u){return o(),s("div",null,[...e[0]||(e[0]=[a("",13)])])}const g=t(d,[["render",r]]);export{h as __pageData,g as default};

package/export-template/docs/assets/dag-stages.DSz_S937.svg

@@ -0,0 +1 @@
+<svg id="gd13" width="100%" xmlns="http://www.w3.org/2000/svg" class="flowchart" viewBox="-8 -17.3125 597.09375 608.2625" role="graphics-document document" aria-roledescription="flowchart-v2" style="max-width: 597.09375px;"><style>#gd13{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:16px;fill:#151b1e;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#gd13 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#gd13 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#gd13 .error-icon{fill:#eef1f6;}#gd13 .error-text{fill:#110e09;stroke:#110e09;}#gd13 .edge-thickness-normal{stroke-width:1px;}#gd13 .edge-thickness-thick{stroke-width:3.5px;}#gd13 .edge-pattern-solid{stroke-dasharray:0;}#gd13 .edge-thickness-invisible{stroke-width:0;fill:none;}#gd13 .edge-pattern-dashed{stroke-dasharray:3;}#gd13 .edge-pattern-dotted{stroke-dasharray:2;}#gd13 .marker{fill:#4f5f60;stroke:#4f5f60;}#gd13 .marker.cross{stroke:#4f5f60;}#gd13 svg{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:16px;}#gd13 p{margin:0;}#gd13 .label{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;color:#151b1e;}#gd13 .cluster-label text{fill:#151b1e;}#gd13 .cluster-label span{color:#151b1e;}#gd13 .cluster-label span p{background-color:transparent;}#gd13 .label text,#gd13 span{fill:#151b1e;color:#151b1e;}#gd13 .node rect,#gd13 .node circle,#gd13 .node ellipse,#gd13 .node polygon,#gd13 .node path{fill:#ffffff;stroke:#9aa8a3;stroke-width:1px;}#gd13 .rough-node .label text,#gd13 .node .label text,#gd13 .image-shape .label,#gd13 .icon-shape .label{text-anchor:middle;}#gd13 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#gd13 .rough-node .label,#gd13 .node .label,#gd13 .image-shape .label,#gd13 .icon-shape .label{text-align:center;}#gd13 .node.clickable{cursor:pointer;}#gd13 .root .anchor path{fill:#4f5f60!important;stroke-width:0;stroke:#4f5f60;}#gd13 .arrowheadPath{fill:rgba(255, 255, 255, 0);}#gd13 .edgePath .path{stroke:#4f5f60;stroke-width:1px;}#gd13 .flowchart-link{stroke:#4f5f60;fill:none;}#gd13 .edgeLabel{background-color:#eef1f6;text-align:center;}#gd13 .edgeLabel p{background-color:#eef1f6;}#gd13 .edgeLabel rect{opacity:0.5;background-color:#eef1f6;fill:#eef1f6;}#gd13 .labelBkg{background-color:rgba(238, 241, 246, 0.5);}#gd13 .cluster rect{fill:#f2f5f9;stroke:#dbe2df;stroke-width:1px;}#gd13 .cluster text{fill:#151b1e;}#gd13 .cluster span{color:#151b1e;}#gd13 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:12px;background:#eef1f6;border:1px solid hsl(217.5, 0%, 84.9019607843%);border-radius:2px;pointer-events:none;z-index:100;}#gd13 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#151b1e;}#gd13 rect.text{fill:none;stroke-width:0;}#gd13 .icon-shape,#gd13 .image-shape{background-color:#eef1f6;text-align:center;}#gd13 .icon-shape p,#gd13 .image-shape p{background-color:#eef1f6;padding:2px;}#gd13 .icon-shape .label rect,#gd13 .image-shape .label rect{opacity:0.5;background-color:#eef1f6;fill:#eef1f6;}#gd13 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#gd13 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#gd13 .node .neo-node{stroke:#9aa8a3;}#gd13 [data-look=&quot;neo&quot;].node rect,#gd13 [data-look=&quot;neo&quot;].cluster rect,#gd13 [data-look=&quot;neo&quot;].node polygon{stroke:url(#gd13-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd13 [data-look=&quot;neo&quot;].swimlane.cluster rect{filter:none;}#gd13 [data-look=&quot;neo&quot;].node path{stroke:url(#gd13-gradient);stroke-width:1px;}#gd13 [data-look=&quot;neo&quot;].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd13 [data-look=&quot;neo&quot;].node .neo-line path{stroke:#9aa8a3;filter:none;}#gd13 [data-look=&quot;neo&quot;].node circle{stroke:url(#gd13-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd13 [data-look=&quot;neo&quot;].node circle .state-start{fill:#000000;}#gd13 [data-look=&quot;neo&quot;].icon-shape .icon{fill:url(#gd13-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd13 [data-look=&quot;neo&quot;].icon-shape .icon-neo path{stroke:url(#gd13-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd13 :root{--mermaid-font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;}</style><g><marker id="gd13_flowchart-v2-pointEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 0 L 10 5 L 0 10 z" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></path></marker><marker id="gd13_flowchart-v2-pointStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="4.5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 5 L 10 10 L 10 0 z" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></path></marker><marker id="gd13_flowchart-v2-pointEnd-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="11.5" refY="7" markerUnits="userSpaceOnUse" markerWidth="10.5" markerHeight="14" orient="auto"><path d="M 0 0 L 11.5 7 L 0 14 z" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></path></marker><marker id="gd13_flowchart-v2-pointStart-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="1" refY="7" markerUnits="userSpaceOnUse" markerWidth="11.5" markerHeight="14" orient="auto"><polygon points="0,7 11.5,14 11.5,0" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></polygon></marker><marker id="gd13_flowchart-v2-circleEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="11" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></circle></marker><marker id="gd13_flowchart-v2-circleStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-1" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></circle></marker><marker id="gd13_flowchart-v2-circleEnd-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refY="5" refX="12.25" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></circle></marker><marker id="gd13_flowchart-v2-circleStart-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-2" refY="5" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></circle></marker><marker id="gd13_flowchart-v2-crossEnd" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="12" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width:2;stroke-dasharray:1,0"></path></marker><marker id="gd13_flowchart-v2-crossStart" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="-1" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width:2;stroke-dasharray:1,0"></path></marker><marker id="gd13_flowchart-v2-crossEnd-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="17.7" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width:2.5"></path></marker><marker id="gd13_flowchart-v2-crossStart-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="-3.5" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width:2.5;stroke-dasharray:1,0"></path></marker><g class="root"><g class="clusters"><g class="cluster " id="gd13-S3" data-look="classic"><rect x="8" y="484.32500000000005" width="175.140625" height="98.625"></rect><g class="cluster-label " transform="translate(64.8203125, 484.32500000000005)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Stage</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> 3</tspan></tspan></text></g></g></g><g class="cluster " id="gd13-S2" data-look="classic"><rect x="12.796875" y="295.475" width="165.546875" height="98.625"></rect><g class="cluster-label " transform="translate(64.8203125, 295.475)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Stage</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> 2</tspan></tspan></text></g></g></g><g class="cluster " id="gd13-S1" data-look="classic"><rect x="11.9609375" y="8" width="167.21875" height="197.25"></rect><g class="cluster-label " transform="translate(64.8203125, 8)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Stage</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> 1</tspan></tspan></text></g></g></g></g><g class="edgePaths"><path d="M95.57,81.625L95.57,85.792C95.57,89.958,95.57,98.292,95.57,105.958C95.57,113.625,95.57,120.625,95.57,124.125L95.57,127.625" id="gd13-L_F_M1_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_F_M1_0" data-points="W3sieCI6OTUuNTcwMzEyNSwieSI6ODEuNjI1fSx7IngiOjk1LjU3MDMxMjUsInkiOjEwNi42MjV9LHsieCI6OTUuNTcwMzEyNSwieSI6MTMxLjYyNX1d" data-look="classic" marker-end="url(#gd13_flowchart-v2-pointEnd)" style=";"></path><path d="M95.57,180.25L95.57,184.417C95.57,188.583,95.57,196.917,95.57,208.602C95.57,220.288,95.57,235.325,95.57,250.363C95.57,265.4,95.57,280.438,95.57,291.456C95.57,302.475,95.57,309.475,95.57,312.975L95.57,316.475" id="gd13-L_M1_M2_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_M1_M2_0" data-points="W3sieCI6OTUuNTcwMzEyNSwieSI6MTgwLjI1fSx7IngiOjk1LjU3MDMxMjUsInkiOjIwNS4yNX0seyJ4Ijo5NS41NzAzMTI1LCJ5IjoyNTAuMzYyNX0seyJ4Ijo5NS41NzAzMTI1LCJ5IjoyOTUuNDc1fSx7IngiOjk1LjU3MDMxMjUsInkiOjMyMC40NzV9XQ==" data-look="classic" marker-end="url(#gd13_flowchart-v2-pointEnd)" style=";"></path><path d="M95.57,369.1L95.57,373.267C95.57,377.433,95.57,385.767,95.57,397.452C95.57,409.138,95.57,424.175,95.57,439.213C95.57,454.25,95.57,469.288,95.57,480.306C95.57,491.325,95.57,498.325,95.57,501.825L95.57,505.325" id="gd13-L_M2_C_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_M2_C_0" data-points="W3sieCI6OTUuNTcwMzEyNSwieSI6MzY5LjF9LHsieCI6OTUuNTcwMzEyNSwieSI6Mzk0LjF9LHsieCI6OTUuNTcwMzEyNSwieSI6NDM5LjIxMjUwMDAwMDAwMDAzfSx7IngiOjk1LjU3MDMxMjUsInkiOjQ4NC4zMjUwMDAwMDAwMDAwNX0seyJ4Ijo5NS41NzAzMTI1LCJ5Ijo1MDkuMzI1MDAwMDAwMDAwMDV9XQ==" data-look="classic" marker-end="url(#gd13_flowchart-v2-pointEnd)" style=";"></path></g><g class="edgeLabels"><g class="edgeLabel"><g class="label" data-id="L_F_M1_0" transform="translate(0, -10.4609375)"><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"></tspan></text></g></g><g><rect class="background" style="stroke: none"></rect></g><g class="edgeLabel" transform="translate(95.5703125, 250.3625)"><g class="label" data-id="L_M1_M2_0" transform="translate(0, -19.2609375)"><g><rect class="background" x="-56.63671875" y="-0.8515625" width="113.2734375" height="40.225"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">groupByKey</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">(wide</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> shuffle)</tspan></tspan></text></g></g></g><g class="edgeLabel" transform="translate(95.5703125, 439.21250000000003)"><g class="label" data-id="L_M2_C_0" transform="translate(0, -19.2609375)"><g><rect class="background" x="-56.63671875" y="-0.8515625" width="113.2734375" height="40.225"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">sortByKey</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">(wide</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> shuffle)</tspan></tspan></text></g></g></g></g><g class="nodes"><g class="node default  " id="gd13-flowchart-F-0" data-look="classic" transform="translate(95.5703125, 57.3125)"><rect class="basic label-container" x="-48.609375" y="-24.3125" width="97.21875" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">filter</tspan></tspan></text></g></g></g><g class="node default  " id="gd13-flowchart-M1-1" data-look="classic" transform="translate(95.5703125, 155.9375)"><rect class="basic label-container" x="-47.7734375" y="-24.3125" width="95.546875" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">map</tspan></tspan></text></g></g></g><g class="node default  " id="gd13-flowchart-M2-2" data-look="classic" transform="translate(95.5703125, 344.7875)"><rect class="basic label-container" x="-47.7734375" y="-24.3125" width="95.546875" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">map</tspan></tspan></text></g></g></g><g class="node default  " id="gd13-flowchart-C-3" data-look="classic" transform="translate(95.5703125, 533.6375)"><rect class="basic label-container" x="-52.5703125" y="-24.3125" width="105.140625" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">count</tspan></tspan></text></g></g></g><g class="node default  " id="gd13-flowchart-N-8" data-look="classic" transform="translate(397.63671875, 57.3125)"><rect class="basic label-container" x="-183.45703125" y="-24.3125" width="366.9140625" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Each</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> stage</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> runs</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> one</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> task</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> per</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> partition</tspan></tspan></text></g></g></g></g></g></g><defs><filter id="gd13-drop-shadow" height="130%" width="130%"><feDropShadow dx="4" dy="4" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><defs><filter id="gd13-drop-shadow-small" height="150%" width="150%"><feDropShadow dx="2" dy="2" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><linearGradient id="gd13-gradient" gradientUnits="objectBoundingBox" x1="0%" y1="0%" x2="100%" y2="0%"><stop offset="0%" stop-color="#9aa8a3" stop-opacity="1"></stop><stop offset="100%" stop-color="hsl(214.2857142857, 0%, 86.2745098039%)" stop-opacity="1"></stop></linearGradient></svg>

package/export-template/docs/assets/dag-stages.dark.F72UzxH4.svg

@@ -0,0 +1 @@
+<svg id="gd14" width="100%" xmlns="http://www.w3.org/2000/svg" class="flowchart" viewBox="-8 -17.3125 597.09375 608.2625" role="graphics-document document" aria-roledescription="flowchart-v2" style="max-width: 597.09375px;"><style>#gd14{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:16px;fill:#f4f4f5;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#gd14 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#gd14 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#gd14 .error-icon{fill:#1a232f;}#gd14 .error-text{fill:#e5dcd0;stroke:#e5dcd0;}#gd14 .edge-thickness-normal{stroke-width:1px;}#gd14 .edge-thickness-thick{stroke-width:3.5px;}#gd14 .edge-pattern-solid{stroke-dasharray:0;}#gd14 .edge-thickness-invisible{stroke-width:0;fill:none;}#gd14 .edge-pattern-dashed{stroke-dasharray:3;}#gd14 .edge-pattern-dotted{stroke-dasharray:2;}#gd14 .marker{fill:#a9b5b4;stroke:#a9b5b4;}#gd14 .marker.cross{stroke:#a9b5b4;}#gd14 svg{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:16px;}#gd14 p{margin:0;}#gd14 .label{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;color:#f4f4f5;}#gd14 .cluster-label text{fill:#f4f4f5;}#gd14 .cluster-label span{color:#f4f4f5;}#gd14 .cluster-label span p{background-color:transparent;}#gd14 .label text,#gd14 span{fill:#f4f4f5;color:#f4f4f5;}#gd14 .node rect,#gd14 .node circle,#gd14 .node ellipse,#gd14 .node polygon,#gd14 .node path{fill:#17202c;stroke:#3a4a52;stroke-width:1px;}#gd14 .rough-node .label text,#gd14 .node .label text,#gd14 .image-shape .label,#gd14 .icon-shape .label{text-anchor:middle;}#gd14 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#gd14 .rough-node .label,#gd14 .node .label,#gd14 .image-shape .label,#gd14 .icon-shape .label{text-align:center;}#gd14 .node.clickable{cursor:pointer;}#gd14 .root .anchor path{fill:#a9b5b4!important;stroke-width:0;stroke:#a9b5b4;}#gd14 .arrowheadPath{fill:rgba(255, 255, 255, 0);}#gd14 .edgePath .path{stroke:#a9b5b4;stroke-width:1px;}#gd14 .flowchart-link{stroke:#a9b5b4;fill:none;}#gd14 .edgeLabel{background-color:#11171a;text-align:center;}#gd14 .edgeLabel p{background-color:#11171a;}#gd14 .edgeLabel rect{opacity:0.5;background-color:#11171a;fill:#11171a;}#gd14 .labelBkg{background-color:rgba(17, 23, 26, 0.5);}#gd14 .cluster rect{fill:#11171a;stroke:#233035;stroke-width:1px;}#gd14 .cluster text{fill:#f4f4f5;}#gd14 .cluster span{color:#f4f4f5;}#gd14 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:12px;background:#1a232f;border:1px solid hsl(214.2857142857, 0%, 4.3137254902%);border-radius:2px;pointer-events:none;z-index:100;}#gd14 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#f4f4f5;}#gd14 rect.text{fill:none;stroke-width:0;}#gd14 .icon-shape,#gd14 .image-shape{background-color:#11171a;text-align:center;}#gd14 .icon-shape p,#gd14 .image-shape p{background-color:#11171a;padding:2px;}#gd14 .icon-shape .label rect,#gd14 .image-shape .label rect{opacity:0.5;background-color:#11171a;fill:#11171a;}#gd14 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#gd14 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#gd14 .node .neo-node{stroke:#3a4a52;}#gd14 [data-look=&quot;neo&quot;].node rect,#gd14 [data-look=&quot;neo&quot;].cluster rect,#gd14 [data-look=&quot;neo&quot;].node polygon{stroke:url(#gd14-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd14 [data-look=&quot;neo&quot;].swimlane.cluster rect{filter:none;}#gd14 [data-look=&quot;neo&quot;].node path{stroke:url(#gd14-gradient);stroke-width:1px;}#gd14 [data-look=&quot;neo&quot;].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd14 [data-look=&quot;neo&quot;].node .neo-line path{stroke:#3a4a52;filter:none;}#gd14 [data-look=&quot;neo&quot;].node circle{stroke:url(#gd14-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd14 [data-look=&quot;neo&quot;].node circle .state-start{fill:#000000;}#gd14 [data-look=&quot;neo&quot;].icon-shape .icon{fill:url(#gd14-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd14 [data-look=&quot;neo&quot;].icon-shape .icon-neo path{stroke:url(#gd14-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd14 :root{--mermaid-font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;}</style><g><marker id="gd14_flowchart-v2-pointEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 0 L 10 5 L 0 10 z" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></path></marker><marker id="gd14_flowchart-v2-pointStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="4.5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 5 L 10 10 L 10 0 z" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></path></marker><marker id="gd14_flowchart-v2-pointEnd-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="11.5" refY="7" markerUnits="userSpaceOnUse" markerWidth="10.5" markerHeight="14" orient="auto"><path d="M 0 0 L 11.5 7 L 0 14 z" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></path></marker><marker id="gd14_flowchart-v2-pointStart-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="1" refY="7" markerUnits="userSpaceOnUse" markerWidth="11.5" markerHeight="14" orient="auto"><polygon points="0,7 11.5,14 11.5,0" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></polygon></marker><marker id="gd14_flowchart-v2-circleEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="11" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></circle></marker><marker id="gd14_flowchart-v2-circleStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-1" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></circle></marker><marker id="gd14_flowchart-v2-circleEnd-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refY="5" refX="12.25" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></circle></marker><marker id="gd14_flowchart-v2-circleStart-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-2" refY="5" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></circle></marker><marker id="gd14_flowchart-v2-crossEnd" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="12" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width:2;stroke-dasharray:1,0"></path></marker><marker id="gd14_flowchart-v2-crossStart" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="-1" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width:2;stroke-dasharray:1,0"></path></marker><marker id="gd14_flowchart-v2-crossEnd-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="17.7" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width:2.5"></path></marker><marker id="gd14_flowchart-v2-crossStart-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="-3.5" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width:2.5;stroke-dasharray:1,0"></path></marker><g class="root"><g class="clusters"><g class="cluster " id="gd14-S3" data-look="classic"><rect x="8" y="484.32500000000005" width="175.140625" height="98.625"></rect><g class="cluster-label " transform="translate(64.8203125, 484.32500000000005)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Stage</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> 3</tspan></tspan></text></g></g></g><g class="cluster " id="gd14-S2" data-look="classic"><rect x="12.796875" y="295.475" width="165.546875" height="98.625"></rect><g class="cluster-label " transform="translate(64.8203125, 295.475)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Stage</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> 2</tspan></tspan></text></g></g></g><g class="cluster " id="gd14-S1" data-look="classic"><rect x="11.9609375" y="8" width="167.21875" height="197.25"></rect><g class="cluster-label " transform="translate(64.8203125, 8)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Stage</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> 1</tspan></tspan></text></g></g></g></g><g class="edgePaths"><path d="M95.57,81.625L95.57,85.792C95.57,89.958,95.57,98.292,95.57,105.958C95.57,113.625,95.57,120.625,95.57,124.125L95.57,127.625" id="gd14-L_F_M1_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_F_M1_0" data-points="W3sieCI6OTUuNTcwMzEyNSwieSI6ODEuNjI1fSx7IngiOjk1LjU3MDMxMjUsInkiOjEwNi42MjV9LHsieCI6OTUuNTcwMzEyNSwieSI6MTMxLjYyNX1d" data-look="classic" marker-end="url(#gd14_flowchart-v2-pointEnd)" style=";"></path><path d="M95.57,180.25L95.57,184.417C95.57,188.583,95.57,196.917,95.57,208.602C95.57,220.288,95.57,235.325,95.57,250.363C95.57,265.4,95.57,280.438,95.57,291.456C95.57,302.475,95.57,309.475,95.57,312.975L95.57,316.475" id="gd14-L_M1_M2_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_M1_M2_0" data-points="W3sieCI6OTUuNTcwMzEyNSwieSI6MTgwLjI1fSx7IngiOjk1LjU3MDMxMjUsInkiOjIwNS4yNX0seyJ4Ijo5NS41NzAzMTI1LCJ5IjoyNTAuMzYyNX0seyJ4Ijo5NS41NzAzMTI1LCJ5IjoyOTUuNDc1fSx7IngiOjk1LjU3MDMxMjUsInkiOjMyMC40NzV9XQ==" data-look="classic" marker-end="url(#gd14_flowchart-v2-pointEnd)" style=";"></path><path d="M95.57,369.1L95.57,373.267C95.57,377.433,95.57,385.767,95.57,397.452C95.57,409.138,95.57,424.175,95.57,439.213C95.57,454.25,95.57,469.288,95.57,480.306C95.57,491.325,95.57,498.325,95.57,501.825L95.57,505.325" id="gd14-L_M2_C_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_M2_C_0" data-points="W3sieCI6OTUuNTcwMzEyNSwieSI6MzY5LjF9LHsieCI6OTUuNTcwMzEyNSwieSI6Mzk0LjF9LHsieCI6OTUuNTcwMzEyNSwieSI6NDM5LjIxMjUwMDAwMDAwMDAzfSx7IngiOjk1LjU3MDMxMjUsInkiOjQ4NC4zMjUwMDAwMDAwMDAwNX0seyJ4Ijo5NS41NzAzMTI1LCJ5Ijo1MDkuMzI1MDAwMDAwMDAwMDV9XQ==" data-look="classic" marker-end="url(#gd14_flowchart-v2-pointEnd)" style=";"></path></g><g class="edgeLabels"><g class="edgeLabel"><g class="label" data-id="L_F_M1_0" transform="translate(0, -10.4609375)"><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"></tspan></text></g></g><g><rect class="background" style="stroke: none"></rect></g><g class="edgeLabel" transform="translate(95.5703125, 250.3625)"><g class="label" data-id="L_M1_M2_0" transform="translate(0, -19.2609375)"><g><rect class="background" x="-56.63671875" y="-0.8515625" width="113.2734375" height="40.225"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">groupByKey</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">(wide</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> shuffle)</tspan></tspan></text></g></g></g><g class="edgeLabel" transform="translate(95.5703125, 439.21250000000003)"><g class="label" data-id="L_M2_C_0" transform="translate(0, -19.2609375)"><g><rect class="background" x="-56.63671875" y="-0.8515625" width="113.2734375" height="40.225"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">sortByKey</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">(wide</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> shuffle)</tspan></tspan></text></g></g></g></g><g class="nodes"><g class="node default  " id="gd14-flowchart-F-0" data-look="classic" transform="translate(95.5703125, 57.3125)"><rect class="basic label-container" x="-48.609375" y="-24.3125" width="97.21875" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">filter</tspan></tspan></text></g></g></g><g class="node default  " id="gd14-flowchart-M1-1" data-look="classic" transform="translate(95.5703125, 155.9375)"><rect class="basic label-container" x="-47.7734375" y="-24.3125" width="95.546875" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">map</tspan></tspan></text></g></g></g><g class="node default  " id="gd14-flowchart-M2-2" data-look="classic" transform="translate(95.5703125, 344.7875)"><rect class="basic label-container" x="-47.7734375" y="-24.3125" width="95.546875" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">map</tspan></tspan></text></g></g></g><g class="node default  " id="gd14-flowchart-C-3" data-look="classic" transform="translate(95.5703125, 533.6375)"><rect class="basic label-container" x="-52.5703125" y="-24.3125" width="105.140625" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">count</tspan></tspan></text></g></g></g><g class="node default  " id="gd14-flowchart-N-8" data-look="classic" transform="translate(397.63671875, 57.3125)"><rect class="basic label-container" x="-183.45703125" y="-24.3125" width="366.9140625" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Each</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> stage</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> runs</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> one</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> task</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> per</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> partition</tspan></tspan></text></g></g></g></g></g></g><defs><filter id="gd14-drop-shadow" height="130%" width="130%"><feDropShadow dx="4" dy="4" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><defs><filter id="gd14-drop-shadow-small" height="150%" width="150%"><feDropShadow dx="2" dy="2" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><linearGradient id="gd14-gradient" gradientUnits="objectBoundingBox" x1="0%" y1="0%" x2="100%" y2="0%"><stop offset="0%" stop-color="#3a4a52" stop-opacity="1"></stop><stop offset="100%" stop-color="hsl(200, 0%, 0%)" stop-opacity="1"></stop></linearGradient></svg>

package/export-template/docs/assets/driver-executor.D5pQ7YN1.svg

@@ -0,0 +1 @@
+<svg id="gd15" width="100%" xmlns="http://www.w3.org/2000/svg" class="flowchart" viewBox="-8 -17.3125 945.12109375 616.0875" role="graphics-document document" aria-roledescription="flowchart-v2" style="max-width: 945.12109375px;"><style>#gd15{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:16px;fill:#151b1e;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#gd15 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#gd15 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#gd15 .error-icon{fill:#eef1f6;}#gd15 .error-text{fill:#110e09;stroke:#110e09;}#gd15 .edge-thickness-normal{stroke-width:1px;}#gd15 .edge-thickness-thick{stroke-width:3.5px;}#gd15 .edge-pattern-solid{stroke-dasharray:0;}#gd15 .edge-thickness-invisible{stroke-width:0;fill:none;}#gd15 .edge-pattern-dashed{stroke-dasharray:3;}#gd15 .edge-pattern-dotted{stroke-dasharray:2;}#gd15 .marker{fill:#4f5f60;stroke:#4f5f60;}#gd15 .marker.cross{stroke:#4f5f60;}#gd15 svg{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:16px;}#gd15 p{margin:0;}#gd15 .label{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;color:#151b1e;}#gd15 .cluster-label text{fill:#151b1e;}#gd15 .cluster-label span{color:#151b1e;}#gd15 .cluster-label span p{background-color:transparent;}#gd15 .label text,#gd15 span{fill:#151b1e;color:#151b1e;}#gd15 .node rect,#gd15 .node circle,#gd15 .node ellipse,#gd15 .node polygon,#gd15 .node path{fill:#ffffff;stroke:#9aa8a3;stroke-width:1px;}#gd15 .rough-node .label text,#gd15 .node .label text,#gd15 .image-shape .label,#gd15 .icon-shape .label{text-anchor:middle;}#gd15 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#gd15 .rough-node .label,#gd15 .node .label,#gd15 .image-shape .label,#gd15 .icon-shape .label{text-align:center;}#gd15 .node.clickable{cursor:pointer;}#gd15 .root .anchor path{fill:#4f5f60!important;stroke-width:0;stroke:#4f5f60;}#gd15 .arrowheadPath{fill:rgba(255, 255, 255, 0);}#gd15 .edgePath .path{stroke:#4f5f60;stroke-width:1px;}#gd15 .flowchart-link{stroke:#4f5f60;fill:none;}#gd15 .edgeLabel{background-color:#eef1f6;text-align:center;}#gd15 .edgeLabel p{background-color:#eef1f6;}#gd15 .edgeLabel rect{opacity:0.5;background-color:#eef1f6;fill:#eef1f6;}#gd15 .labelBkg{background-color:rgba(238, 241, 246, 0.5);}#gd15 .cluster rect{fill:#f2f5f9;stroke:#dbe2df;stroke-width:1px;}#gd15 .cluster text{fill:#151b1e;}#gd15 .cluster span{color:#151b1e;}#gd15 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:12px;background:#eef1f6;border:1px solid hsl(217.5, 0%, 84.9019607843%);border-radius:2px;pointer-events:none;z-index:100;}#gd15 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#151b1e;}#gd15 rect.text{fill:none;stroke-width:0;}#gd15 .icon-shape,#gd15 .image-shape{background-color:#eef1f6;text-align:center;}#gd15 .icon-shape p,#gd15 .image-shape p{background-color:#eef1f6;padding:2px;}#gd15 .icon-shape .label rect,#gd15 .image-shape .label rect{opacity:0.5;background-color:#eef1f6;fill:#eef1f6;}#gd15 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#gd15 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#gd15 .node .neo-node{stroke:#9aa8a3;}#gd15 [data-look=&quot;neo&quot;].node rect,#gd15 [data-look=&quot;neo&quot;].cluster rect,#gd15 [data-look=&quot;neo&quot;].node polygon{stroke:url(#gd15-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd15 [data-look=&quot;neo&quot;].swimlane.cluster rect{filter:none;}#gd15 [data-look=&quot;neo&quot;].node path{stroke:url(#gd15-gradient);stroke-width:1px;}#gd15 [data-look=&quot;neo&quot;].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd15 [data-look=&quot;neo&quot;].node .neo-line path{stroke:#9aa8a3;filter:none;}#gd15 [data-look=&quot;neo&quot;].node circle{stroke:url(#gd15-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd15 [data-look=&quot;neo&quot;].node circle .state-start{fill:#000000;}#gd15 [data-look=&quot;neo&quot;].icon-shape .icon{fill:url(#gd15-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd15 [data-look=&quot;neo&quot;].icon-shape .icon-neo path{stroke:url(#gd15-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd15 :root{--mermaid-font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;}</style><g><marker id="gd15_flowchart-v2-pointEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 0 L 10 5 L 0 10 z" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></path></marker><marker id="gd15_flowchart-v2-pointStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="4.5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 5 L 10 10 L 10 0 z" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></path></marker><marker id="gd15_flowchart-v2-pointEnd-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="11.5" refY="7" markerUnits="userSpaceOnUse" markerWidth="10.5" markerHeight="14" orient="auto"><path d="M 0 0 L 11.5 7 L 0 14 z" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></path></marker><marker id="gd15_flowchart-v2-pointStart-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="1" refY="7" markerUnits="userSpaceOnUse" markerWidth="11.5" markerHeight="14" orient="auto"><polygon points="0,7 11.5,14 11.5,0" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></polygon></marker><marker id="gd15_flowchart-v2-circleEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="11" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></circle></marker><marker id="gd15_flowchart-v2-circleStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-1" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></circle></marker><marker id="gd15_flowchart-v2-circleEnd-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refY="5" refX="12.25" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></circle></marker><marker id="gd15_flowchart-v2-circleStart-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-2" refY="5" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></circle></marker><marker id="gd15_flowchart-v2-crossEnd" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="12" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width:2;stroke-dasharray:1,0"></path></marker><marker id="gd15_flowchart-v2-crossStart" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="-1" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width:2;stroke-dasharray:1,0"></path></marker><marker id="gd15_flowchart-v2-crossEnd-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="17.7" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width:2.5"></path></marker><marker id="gd15_flowchart-v2-crossStart-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="-3.5" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width:2.5;stroke-dasharray:1,0"></path></marker><g class="root"><g class="clusters"><g class="cluster " id="gd15-CM" data-look="classic"><rect x="8" y="171.85000000000002" width="343.98828125" height="418.92499999999995"></rect><g class="cluster-label " transform="translate(113.099609375, 171.85000000000002)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Cluster</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> manager</tspan></tspan></text></g></g></g><g class="cluster " id="gd15-Driver" data-look="classic"><rect x="371.98828125" y="8" width="557.1328125" height="418.92499999999995"></rect><g class="cluster-label " transform="translate(625.9375, 8)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Driver</tspan></tspan></text></g></g></g></g><g class="edgePaths"><path d="M511.531,254.275L511.531,261.794C511.531,269.313,511.531,284.35,511.531,297.254C511.531,310.158,511.531,320.929,511.531,326.315L511.531,331.7" id="gd15-L_DAG_TS_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_DAG_TS_0" data-points="W3sieCI6NTExLjUzMTI1LCJ5IjoyNTQuMjc0OTk5OTk5OTk5OTh9LHsieCI6NTExLjUzMTI1LCJ5IjoyOTkuMzg3NX0seyJ4Ijo1MTEuNTMxMjUsInkiOjMzNS43fV0=" data-look="classic" marker-end="url(#gd15_flowchart-v2-pointEnd)" style=";"></path><path d="M406.988,340.919L381.046,333.997C355.104,327.075,303.219,313.231,271.608,300.725C239.997,288.219,228.66,277.051,222.992,271.466L217.323,265.882" id="gd15-L_TS_E1_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_TS_E1_0" data-points="W3sieCI6NDA2Ljk4ODI4MTI1LCJ5IjozNDAuOTE4NjgwMzMxOTI5NjR9LHsieCI6MjUxLjMzMzk4NDM3NSwieSI6Mjk5LjM4NzV9LHsieCI6MjE0LjQ3MzUxMTQ4OTQ2NzA2LCJ5IjoyNjMuMDc1fV0=" data-look="classic" marker-end="url(#gd15_flowchart-v2-pointEnd)" style=";"></path><path d="M483.171,401.925L479.603,406.092C476.034,410.258,468.897,418.592,406.477,428.81C344.058,439.029,226.356,451.133,173.174,462.77C119.991,474.406,131.328,485.574,136.997,491.159L142.665,496.743" id="gd15-L_TS_E2_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_TS_E2_0" data-points="W3sieCI6NDgzLjE3MTQ2MDM0NDI5NDUsInkiOjQwMS45MjV9LHsieCI6NDYxLjc1OTc2NTYyNSwieSI6NDI2LjkyNDk5OTk5OTk5OTk1fSx7IngiOjEwOC42NTQyOTY4NzUsInkiOjQ2My4yMzc0OTk5OTk5OTk5NX0seyJ4IjoxNDUuNTE0NzY5NzYwNTMyOTQsInkiOjQ5OS41NDk5OTk5OTk5OTk5fV0=" data-look="classic" marker-end="url(#gd15_flowchart-v2-pointEnd)" style=";"></path><path d="M147.249,263.075L141.106,269.127C134.962,275.179,122.675,287.283,165.308,301.777C207.941,316.271,305.494,333.154,354.271,341.596L403.047,350.037" id="gd15-L_E1_TS_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_E1_TS_0" data-points="W3sieCI6MTQ3LjI0OTE0NDc2MDUzMjk0LCJ5IjoyNjMuMDc1fSx7IngiOjExMC4zODg2NzE4NzUsInkiOjI5OS4zODc1fSx7IngiOjQwNi45ODgyODEyNSwieSI6MzUwLjcxOTQ0MjgxNDcxMzgzfV0=" data-look="classic" marker-end="url(#gd15_flowchart-v2-pointEnd)" style=";"></path><path d="M212.739,499.55L218.883,493.498C225.026,487.446,237.313,475.342,320.244,463.237C403.175,451.133,556.751,439.029,618.47,428.572C680.189,418.115,650.051,409.305,634.982,404.9L619.914,400.495" id="gd15-L_E2_TS_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_E2_TS_0" data-points="W3sieCI6MjEyLjczOTEzNjQ4OTQ2NzA2LCJ5Ijo0OTkuNTQ5OTk5OTk5OTk5OX0seyJ4IjoyNDkuNTk5NjA5Mzc1LCJ5Ijo0NjMuMjM3NDk5OTk5OTk5OTV9LHsieCI6NzEwLjMyNjE3MTg3NSwieSI6NDI2LjkyNDk5OTk5OTk5OTk1fSx7IngiOjYxNi4wNzQyMTg3NSwieSI6Mzk5LjM3MjkwNDczMzU5OTl9XQ==" data-look="classic" marker-end="url(#gd15_flowchart-v2-pointEnd)" style=";"></path><path d="M650.555,99.225L650.555,105.277C650.555,111.329,650.555,123.433,650.555,135.538C650.555,147.642,650.555,159.746,601.46,167.515C552.365,175.285,454.175,178.72,405.081,180.438L355.986,182.155" id="gd15-L_SC_CM_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_SC_CM_0" data-points="W3sieCI6NjUwLjU1NDY4NzUsInkiOjk5LjIyNX0seyJ4Ijo2NTAuNTU0Njg3NSwieSI6MTM1LjUzNzV9LHsieCI6NjUwLjU1NDY4NzUsInkiOjE3MS44NX0seyJ4IjoyNjcuNTY0NDUzMTI1LCJ5IjoyMTkuMjM1MjEzMjQ3OTIxODh9XQ==" data-look="classic" marker-end="url(#gd15_flowchart-v2-pointEnd)" style=";"></path></g><g class="edgeLabels"><g class="edgeLabel"><g class="label" data-id="L_DAG_TS_0" transform="translate(0, -10.4609375)"><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"></tspan></text></g></g><g><rect class="background" style="stroke: none"></rect></g><g class="edgeLabel" transform="translate(304.16433, 313.48352)"><g class="label" data-id="L_TS_E1_0" transform="translate(0, -10.4609375)"><g><rect class="background" x="-59.60546875" y="-0.8515625" width="119.2109375" height="22.625"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">dispatch</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> tasks</tspan></tspan></text></g></g></g><g class="edgeLabel" transform="translate(108.654296875, 463.23749999999995)"><g class="label" data-id="L_TS_E2_0" transform="translate(0, -10.4609375)"><g><rect class="background" x="-59.60546875" y="-0.8515625" width="119.2109375" height="22.625"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">dispatch</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> tasks</tspan></tspan></text></g></g></g><g class="edgeLabel" transform="translate(233.19616, 320.64156)"><g class="label" data-id="L_E1_TS_0" transform="translate(0, -10.4609375)"><g><rect class="background" x="-61.33984375" y="-0.8515625" width="122.6796875" height="22.625"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">status</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> /</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> results</tspan></tspan></text></g></g></g><g class="edgeLabel" transform="translate(503.11804, 443.25626)"><g class="label" data-id="L_E2_TS_0" transform="translate(0, -10.4609375)"><g><rect class="background" x="-61.33984375" y="-0.8515625" width="122.6796875" height="22.625"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">status</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> /</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> results</tspan></tspan></text></g></g></g><g class="edgeLabel" transform="translate(537.56178, 175.80303)"><g class="label" data-id="L_SC_CM_0" transform="translate(0, -10.4609375)"><g><rect class="background" x="-74.6640625" y="-0.8515625" width="149.328125" height="22.625"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">request</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> executors</tspan></tspan></text></g></g></g></g><g class="nodes"><g class="node default  " id="gd15-flowchart-SC-0" data-look="classic" transform="translate(650.5546875, 66.1125)"><rect class="basic label-container" x="-89.69921875" y="-33.1125" width="179.3984375" height="66.225"></rect><g class="label" transform="translate(0, -18.1125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">SparkContext</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> /</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">SparkSession</tspan></tspan></text></g></g></g><g class="node default  " id="gd15-flowchart-DAG-1" data-look="classic" transform="translate(511.53125, 229.96249999999998)"><rect class="basic label-container" x="-87.8828125" y="-24.3125" width="175.765625" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">DAGScheduler</tspan></tspan></text></g></g></g><g class="node default  " id="gd15-flowchart-TS-2" data-look="classic" transform="translate(511.53125, 368.8125)"><rect class="basic label-container" x="-104.54296875" y="-33.1125" width="209.0859375" height="66.225"></rect><g class="label" transform="translate(0, -18.1125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">TaskScheduler</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> +</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">SchedulerBackend</tspan></tspan></text></g></g></g><g class="node default  " id="gd15-flowchart-BMM-3" data-look="classic" transform="translate(780.09765625, 368.8125)"><rect class="basic label-container" x="-114.0234375" y="-24.3125" width="228.046875" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">BlockManagerMaster</tspan></tspan></text></g></g></g><g class="node default  " id="gd15-flowchart-E1-6" data-look="classic" transform="translate(180.861328125, 229.96249999999998)"><rect class="basic label-container" x="-86.703125" y="-33.1125" width="173.40625" height="66.225"></rect><g class="label" transform="translate(0, -18.1125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Executor</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">BlockManager</tspan></tspan></text></g></g></g><g class="node default  " id="gd15-flowchart-E2-7" data-look="classic" transform="translate(179.126953125, 532.6624999999999)"><rect class="basic label-container" x="-86.703125" y="-33.1125" width="173.40625" height="66.225"></rect><g class="label" transform="translate(0, -18.1125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Executor</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">BlockManager</tspan></tspan></text></g></g></g></g></g></g><defs><filter id="gd15-drop-shadow" height="130%" width="130%"><feDropShadow dx="4" dy="4" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><defs><filter id="gd15-drop-shadow-small" height="150%" width="150%"><feDropShadow dx="2" dy="2" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><linearGradient id="gd15-gradient" gradientUnits="objectBoundingBox" x1="0%" y1="0%" x2="100%" y2="0%"><stop offset="0%" stop-color="#9aa8a3" stop-opacity="1"></stop><stop offset="100%" stop-color="hsl(214.2857142857, 0%, 86.2745098039%)" stop-opacity="1"></stop></linearGradient></svg>

package/export-template/docs/assets/driver-executor.dark.BmX9cPvh.svg

@@ -0,0 +1 @@
+<svg id="gd16" width="100%" xmlns="http://www.w3.org/2000/svg" class="flowchart" viewBox="-8 -17.3125 945.12109375 616.0875" role="graphics-document document" aria-roledescription="flowchart-v2" style="max-width: 945.12109375px;"><style>#gd16{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:16px;fill:#f4f4f5;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#gd16 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#gd16 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#gd16 .error-icon{fill:#1a232f;}#gd16 .error-text{fill:#e5dcd0;stroke:#e5dcd0;}#gd16 .edge-thickness-normal{stroke-width:1px;}#gd16 .edge-thickness-thick{stroke-width:3.5px;}#gd16 .edge-pattern-solid{stroke-dasharray:0;}#gd16 .edge-thickness-invisible{stroke-width:0;fill:none;}#gd16 .edge-pattern-dashed{stroke-dasharray:3;}#gd16 .edge-pattern-dotted{stroke-dasharray:2;}#gd16 .marker{fill:#a9b5b4;stroke:#a9b5b4;}#gd16 .marker.cross{stroke:#a9b5b4;}#gd16 svg{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:16px;}#gd16 p{margin:0;}#gd16 .label{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;color:#f4f4f5;}#gd16 .cluster-label text{fill:#f4f4f5;}#gd16 .cluster-label span{color:#f4f4f5;}#gd16 .cluster-label span p{background-color:transparent;}#gd16 .label text,#gd16 span{fill:#f4f4f5;color:#f4f4f5;}#gd16 .node rect,#gd16 .node circle,#gd16 .node ellipse,#gd16 .node polygon,#gd16 .node path{fill:#17202c;stroke:#3a4a52;stroke-width:1px;}#gd16 .rough-node .label text,#gd16 .node .label text,#gd16 .image-shape .label,#gd16 .icon-shape .label{text-anchor:middle;}#gd16 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#gd16 .rough-node .label,#gd16 .node .label,#gd16 .image-shape .label,#gd16 .icon-shape .label{text-align:center;}#gd16 .node.clickable{cursor:pointer;}#gd16 .root .anchor path{fill:#a9b5b4!important;stroke-width:0;stroke:#a9b5b4;}#gd16 .arrowheadPath{fill:rgba(255, 255, 255, 0);}#gd16 .edgePath .path{stroke:#a9b5b4;stroke-width:1px;}#gd16 .flowchart-link{stroke:#a9b5b4;fill:none;}#gd16 .edgeLabel{background-color:#11171a;text-align:center;}#gd16 .edgeLabel p{background-color:#11171a;}#gd16 .edgeLabel rect{opacity:0.5;background-color:#11171a;fill:#11171a;}#gd16 .labelBkg{background-color:rgba(17, 23, 26, 0.5);}#gd16 .cluster rect{fill:#11171a;stroke:#233035;stroke-width:1px;}#gd16 .cluster text{fill:#f4f4f5;}#gd16 .cluster span{color:#f4f4f5;}#gd16 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:12px;background:#1a232f;border:1px solid hsl(214.2857142857, 0%, 4.3137254902%);border-radius:2px;pointer-events:none;z-index:100;}#gd16 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#f4f4f5;}#gd16 rect.text{fill:none;stroke-width:0;}#gd16 .icon-shape,#gd16 .image-shape{background-color:#11171a;text-align:center;}#gd16 .icon-shape p,#gd16 .image-shape p{background-color:#11171a;padding:2px;}#gd16 .icon-shape .label rect,#gd16 .image-shape .label rect{opacity:0.5;background-color:#11171a;fill:#11171a;}#gd16 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#gd16 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#gd16 .node .neo-node{stroke:#3a4a52;}#gd16 [data-look=&quot;neo&quot;].node rect,#gd16 [data-look=&quot;neo&quot;].cluster rect,#gd16 [data-look=&quot;neo&quot;].node polygon{stroke:url(#gd16-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd16 [data-look=&quot;neo&quot;].swimlane.cluster rect{filter:none;}#gd16 [data-look=&quot;neo&quot;].node path{stroke:url(#gd16-gradient);stroke-width:1px;}#gd16 [data-look=&quot;neo&quot;].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd16 [data-look=&quot;neo&quot;].node .neo-line path{stroke:#3a4a52;filter:none;}#gd16 [data-look=&quot;neo&quot;].node circle{stroke:url(#gd16-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd16 [data-look=&quot;neo&quot;].node circle .state-start{fill:#000000;}#gd16 [data-look=&quot;neo&quot;].icon-shape .icon{fill:url(#gd16-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd16 [data-look=&quot;neo&quot;].icon-shape .icon-neo path{stroke:url(#gd16-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd16 :root{--mermaid-font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;}</style><g><marker id="gd16_flowchart-v2-pointEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 0 L 10 5 L 0 10 z" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></path></marker><marker id="gd16_flowchart-v2-pointStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="4.5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 5 L 10 10 L 10 0 z" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></path></marker><marker id="gd16_flowchart-v2-pointEnd-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="11.5" refY="7" markerUnits="userSpaceOnUse" markerWidth="10.5" markerHeight="14" orient="auto"><path d="M 0 0 L 11.5 7 L 0 14 z" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></path></marker><marker id="gd16_flowchart-v2-pointStart-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="1" refY="7" markerUnits="userSpaceOnUse" markerWidth="11.5" markerHeight="14" orient="auto"><polygon points="0,7 11.5,14 11.5,0" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></polygon></marker><marker id="gd16_flowchart-v2-circleEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="11" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></circle></marker><marker id="gd16_flowchart-v2-circleStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-1" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></circle></marker><marker id="gd16_flowchart-v2-circleEnd-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refY="5" refX="12.25" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></circle></marker><marker id="gd16_flowchart-v2-circleStart-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-2" refY="5" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></circle></marker><marker id="gd16_flowchart-v2-crossEnd" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="12" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width:2;stroke-dasharray:1,0"></path></marker><marker id="gd16_flowchart-v2-crossStart" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="-1" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width:2;stroke-dasharray:1,0"></path></marker><marker id="gd16_flowchart-v2-crossEnd-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="17.7" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width:2.5"></path></marker><marker id="gd16_flowchart-v2-crossStart-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="-3.5" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width:2.5;stroke-dasharray:1,0"></path></marker><g class="root"><g class="clusters"><g class="cluster " id="gd16-CM" data-look="classic"><rect x="8" y="171.85000000000002" width="343.98828125" height="418.92499999999995"></rect><g class="cluster-label " transform="translate(113.099609375, 171.85000000000002)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Cluster</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> manager</tspan></tspan></text></g></g></g><g class="cluster " id="gd16-Driver" data-look="classic"><rect x="371.98828125" y="8" width="557.1328125" height="418.92499999999995"></rect><g class="cluster-label " transform="translate(625.9375, 8)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Driver</tspan></tspan></text></g></g></g></g><g class="edgePaths"><path d="M511.531,254.275L511.531,261.794C511.531,269.313,511.531,284.35,511.531,297.254C511.531,310.158,511.531,320.929,511.531,326.315L511.531,331.7" id="gd16-L_DAG_TS_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_DAG_TS_0" data-points="W3sieCI6NTExLjUzMTI1LCJ5IjoyNTQuMjc0OTk5OTk5OTk5OTh9LHsieCI6NTExLjUzMTI1LCJ5IjoyOTkuMzg3NX0seyJ4Ijo1MTEuNTMxMjUsInkiOjMzNS43fV0=" data-look="classic" marker-end="url(#gd16_flowchart-v2-pointEnd)" style=";"></path><path d="M406.988,340.919L381.046,333.997C355.104,327.075,303.219,313.231,271.608,300.725C239.997,288.219,228.66,277.051,222.992,271.466L217.323,265.882" id="gd16-L_TS_E1_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_TS_E1_0" data-points="W3sieCI6NDA2Ljk4ODI4MTI1LCJ5IjozNDAuOTE4NjgwMzMxOTI5NjR9LHsieCI6MjUxLjMzMzk4NDM3NSwieSI6Mjk5LjM4NzV9LHsieCI6MjE0LjQ3MzUxMTQ4OTQ2NzA2LCJ5IjoyNjMuMDc1fV0=" data-look="classic" marker-end="url(#gd16_flowchart-v2-pointEnd)" style=";"></path><path d="M483.171,401.925L479.603,406.092C476.034,410.258,468.897,418.592,406.477,428.81C344.058,439.029,226.356,451.133,173.174,462.77C119.991,474.406,131.328,485.574,136.997,491.159L142.665,496.743" id="gd16-L_TS_E2_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_TS_E2_0" data-points="W3sieCI6NDgzLjE3MTQ2MDM0NDI5NDUsInkiOjQwMS45MjV9LHsieCI6NDYxLjc1OTc2NTYyNSwieSI6NDI2LjkyNDk5OTk5OTk5OTk1fSx7IngiOjEwOC42NTQyOTY4NzUsInkiOjQ2My4yMzc0OTk5OTk5OTk5NX0seyJ4IjoxNDUuNTE0NzY5NzYwNTMyOTQsInkiOjQ5OS41NDk5OTk5OTk5OTk5fV0=" data-look="classic" marker-end="url(#gd16_flowchart-v2-pointEnd)" style=";"></path><path d="M147.249,263.075L141.106,269.127C134.962,275.179,122.675,287.283,165.308,301.777C207.941,316.271,305.494,333.154,354.271,341.596L403.047,350.037" id="gd16-L_E1_TS_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_E1_TS_0" data-points="W3sieCI6MTQ3LjI0OTE0NDc2MDUzMjk0LCJ5IjoyNjMuMDc1fSx7IngiOjExMC4zODg2NzE4NzUsInkiOjI5OS4zODc1fSx7IngiOjQwNi45ODgyODEyNSwieSI6MzUwLjcxOTQ0MjgxNDcxMzgzfV0=" data-look="classic" marker-end="url(#gd16_flowchart-v2-pointEnd)" style=";"></path><path d="M212.739,499.55L218.883,493.498C225.026,487.446,237.313,475.342,320.244,463.237C403.175,451.133,556.751,439.029,618.47,428.572C680.189,418.115,650.051,409.305,634.982,404.9L619.914,400.495" id="gd16-L_E2_TS_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_E2_TS_0" data-points="W3sieCI6MjEyLjczOTEzNjQ4OTQ2NzA2LCJ5Ijo0OTkuNTQ5OTk5OTk5OTk5OX0seyJ4IjoyNDkuNTk5NjA5Mzc1LCJ5Ijo0NjMuMjM3NDk5OTk5OTk5OTV9LHsieCI6NzEwLjMyNjE3MTg3NSwieSI6NDI2LjkyNDk5OTk5OTk5OTk1fSx7IngiOjYxNi4wNzQyMTg3NSwieSI6Mzk5LjM3MjkwNDczMzU5OTl9XQ==" data-look="classic" marker-end="url(#gd16_flowchart-v2-pointEnd)" style=";"></path><path d="M650.555,99.225L650.555,105.277C650.555,111.329,650.555,123.433,650.555,135.538C650.555,147.642,650.555,159.746,601.46,167.515C552.365,175.285,454.175,178.72,405.081,180.438L355.986,182.155" id="gd16-L_SC_CM_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_SC_CM_0" data-points="W3sieCI6NjUwLjU1NDY4NzUsInkiOjk5LjIyNX0seyJ4Ijo2NTAuNTU0Njg3NSwieSI6MTM1LjUzNzV9LHsieCI6NjUwLjU1NDY4NzUsInkiOjE3MS44NX0seyJ4IjoyNjcuNTY0NDUzMTI1LCJ5IjoyMTkuMjM1MjEzMjQ3OTIxODh9XQ==" data-look="classic" marker-end="url(#gd16_flowchart-v2-pointEnd)" style=";"></path></g><g class="edgeLabels"><g class="edgeLabel"><g class="label" data-id="L_DAG_TS_0" transform="translate(0, -10.4609375)"><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"></tspan></text></g></g><g><rect class="background" style="stroke: none"></rect></g><g class="edgeLabel" transform="translate(304.16433, 313.48352)"><g class="label" data-id="L_TS_E1_0" transform="translate(0, -10.4609375)"><g><rect class="background" x="-59.60546875" y="-0.8515625" width="119.2109375" height="22.625"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">dispatch</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> tasks</tspan></tspan></text></g></g></g><g class="edgeLabel" transform="translate(108.654296875, 463.23749999999995)"><g class="label" data-id="L_TS_E2_0" transform="translate(0, -10.4609375)"><g><rect class="background" x="-59.60546875" y="-0.8515625" width="119.2109375" height="22.625"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">dispatch</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> tasks</tspan></tspan></text></g></g></g><g class="edgeLabel" transform="translate(233.19616, 320.64156)"><g class="label" data-id="L_E1_TS_0" transform="translate(0, -10.4609375)"><g><rect class="background" x="-61.33984375" y="-0.8515625" width="122.6796875" height="22.625"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">status</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> /</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> results</tspan></tspan></text></g></g></g><g class="edgeLabel" transform="translate(503.11804, 443.25626)"><g class="label" data-id="L_E2_TS_0" transform="translate(0, -10.4609375)"><g><rect class="background" x="-61.33984375" y="-0.8515625" width="122.6796875" height="22.625"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">status</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> /</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> results</tspan></tspan></text></g></g></g><g class="edgeLabel" transform="translate(537.56178, 175.80303)"><g class="label" data-id="L_SC_CM_0" transform="translate(0, -10.4609375)"><g><rect class="background" x="-74.6640625" y="-0.8515625" width="149.328125" height="22.625"></rect><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">request</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> executors</tspan></tspan></text></g></g></g></g><g class="nodes"><g class="node default  " id="gd16-flowchart-SC-0" data-look="classic" transform="translate(650.5546875, 66.1125)"><rect class="basic label-container" x="-89.69921875" y="-33.1125" width="179.3984375" height="66.225"></rect><g class="label" transform="translate(0, -18.1125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">SparkContext</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> /</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">SparkSession</tspan></tspan></text></g></g></g><g class="node default  " id="gd16-flowchart-DAG-1" data-look="classic" transform="translate(511.53125, 229.96249999999998)"><rect class="basic label-container" x="-87.8828125" y="-24.3125" width="175.765625" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">DAGScheduler</tspan></tspan></text></g></g></g><g class="node default  " id="gd16-flowchart-TS-2" data-look="classic" transform="translate(511.53125, 368.8125)"><rect class="basic label-container" x="-104.54296875" y="-33.1125" width="209.0859375" height="66.225"></rect><g class="label" transform="translate(0, -18.1125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">TaskScheduler</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> +</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">SchedulerBackend</tspan></tspan></text></g></g></g><g class="node default  " id="gd16-flowchart-BMM-3" data-look="classic" transform="translate(780.09765625, 368.8125)"><rect class="basic label-container" x="-114.0234375" y="-24.3125" width="228.046875" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">BlockManagerMaster</tspan></tspan></text></g></g></g><g class="node default  " id="gd16-flowchart-E1-6" data-look="classic" transform="translate(180.861328125, 229.96249999999998)"><rect class="basic label-container" x="-86.703125" y="-33.1125" width="173.40625" height="66.225"></rect><g class="label" transform="translate(0, -18.1125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Executor</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">BlockManager</tspan></tspan></text></g></g></g><g class="node default  " id="gd16-flowchart-E2-7" data-look="classic" transform="translate(179.126953125, 532.6624999999999)"><rect class="basic label-container" x="-86.703125" y="-33.1125" width="173.40625" height="66.225"></rect><g class="label" transform="translate(0, -18.1125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Executor</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">BlockManager</tspan></tspan></text></g></g></g></g></g></g><defs><filter id="gd16-drop-shadow" height="130%" width="130%"><feDropShadow dx="4" dy="4" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><defs><filter id="gd16-drop-shadow-small" height="150%" width="150%"><feDropShadow dx="2" dy="2" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><linearGradient id="gd16-gradient" gradientUnits="objectBoundingBox" x1="0%" y1="0%" x2="100%" y2="0%"><stop offset="0%" stop-color="#3a4a52" stop-opacity="1"></stop><stop offset="100%" stop-color="hsl(200, 0%, 0%)" stop-opacity="1"></stop></linearGradient></svg>

package/export-template/docs/assets/duplicate-plan-subtree.B4cvN6fj.svg

@@ -0,0 +1 @@
+<svg id="gd17" width="100%" xmlns="http://www.w3.org/2000/svg" class="flowchart" viewBox="-8 -17.3125 1268.84375 268.1625" role="graphics-document document" aria-roledescription="flowchart-v2" style="max-width: 1268.84375px;"><style>#gd17{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:16px;fill:#151b1e;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#gd17 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#gd17 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#gd17 .error-icon{fill:#eef1f6;}#gd17 .error-text{fill:#110e09;stroke:#110e09;}#gd17 .edge-thickness-normal{stroke-width:1px;}#gd17 .edge-thickness-thick{stroke-width:3.5px;}#gd17 .edge-pattern-solid{stroke-dasharray:0;}#gd17 .edge-thickness-invisible{stroke-width:0;fill:none;}#gd17 .edge-pattern-dashed{stroke-dasharray:3;}#gd17 .edge-pattern-dotted{stroke-dasharray:2;}#gd17 .marker{fill:#4f5f60;stroke:#4f5f60;}#gd17 .marker.cross{stroke:#4f5f60;}#gd17 svg{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:16px;}#gd17 p{margin:0;}#gd17 .label{font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;color:#151b1e;}#gd17 .cluster-label text{fill:#151b1e;}#gd17 .cluster-label span{color:#151b1e;}#gd17 .cluster-label span p{background-color:transparent;}#gd17 .label text,#gd17 span{fill:#151b1e;color:#151b1e;}#gd17 .node rect,#gd17 .node circle,#gd17 .node ellipse,#gd17 .node polygon,#gd17 .node path{fill:#ffffff;stroke:#9aa8a3;stroke-width:1px;}#gd17 .rough-node .label text,#gd17 .node .label text,#gd17 .image-shape .label,#gd17 .icon-shape .label{text-anchor:middle;}#gd17 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#gd17 .rough-node .label,#gd17 .node .label,#gd17 .image-shape .label,#gd17 .icon-shape .label{text-align:center;}#gd17 .node.clickable{cursor:pointer;}#gd17 .root .anchor path{fill:#4f5f60!important;stroke-width:0;stroke:#4f5f60;}#gd17 .arrowheadPath{fill:rgba(255, 255, 255, 0);}#gd17 .edgePath .path{stroke:#4f5f60;stroke-width:1px;}#gd17 .flowchart-link{stroke:#4f5f60;fill:none;}#gd17 .edgeLabel{background-color:#eef1f6;text-align:center;}#gd17 .edgeLabel p{background-color:#eef1f6;}#gd17 .edgeLabel rect{opacity:0.5;background-color:#eef1f6;fill:#eef1f6;}#gd17 .labelBkg{background-color:rgba(238, 241, 246, 0.5);}#gd17 .cluster rect{fill:#f2f5f9;stroke:#dbe2df;stroke-width:1px;}#gd17 .cluster text{fill:#151b1e;}#gd17 .cluster span{color:#151b1e;}#gd17 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:12px;background:#eef1f6;border:1px solid hsl(217.5, 0%, 84.9019607843%);border-radius:2px;pointer-events:none;z-index:100;}#gd17 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#151b1e;}#gd17 rect.text{fill:none;stroke-width:0;}#gd17 .icon-shape,#gd17 .image-shape{background-color:#eef1f6;text-align:center;}#gd17 .icon-shape p,#gd17 .image-shape p{background-color:#eef1f6;padding:2px;}#gd17 .icon-shape .label rect,#gd17 .image-shape .label rect{opacity:0.5;background-color:#eef1f6;fill:#eef1f6;}#gd17 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#gd17 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#gd17 .node .neo-node{stroke:#9aa8a3;}#gd17 [data-look=&quot;neo&quot;].node rect,#gd17 [data-look=&quot;neo&quot;].cluster rect,#gd17 [data-look=&quot;neo&quot;].node polygon{stroke:url(#gd17-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd17 [data-look=&quot;neo&quot;].swimlane.cluster rect{filter:none;}#gd17 [data-look=&quot;neo&quot;].node path{stroke:url(#gd17-gradient);stroke-width:1px;}#gd17 [data-look=&quot;neo&quot;].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd17 [data-look=&quot;neo&quot;].node .neo-line path{stroke:#9aa8a3;filter:none;}#gd17 [data-look=&quot;neo&quot;].node circle{stroke:url(#gd17-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd17 [data-look=&quot;neo&quot;].node circle .state-start{fill:#000000;}#gd17 [data-look=&quot;neo&quot;].icon-shape .icon{fill:url(#gd17-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd17 [data-look=&quot;neo&quot;].icon-shape .icon-neo path{stroke:url(#gd17-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#gd17 :root{--mermaid-font-family:Recursive,ui-sans-serif,system-ui,-apple-system,sans-serif;}</style><g><marker id="gd17_flowchart-v2-pointEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 0 L 10 5 L 0 10 z" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></path></marker><marker id="gd17_flowchart-v2-pointStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="4.5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 5 L 10 10 L 10 0 z" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></path></marker><marker id="gd17_flowchart-v2-pointEnd-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="11.5" refY="7" markerUnits="userSpaceOnUse" markerWidth="10.5" markerHeight="14" orient="auto"><path d="M 0 0 L 11.5 7 L 0 14 z" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></path></marker><marker id="gd17_flowchart-v2-pointStart-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="1" refY="7" markerUnits="userSpaceOnUse" markerWidth="11.5" markerHeight="14" orient="auto"><polygon points="0,7 11.5,14 11.5,0" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></polygon></marker><marker id="gd17_flowchart-v2-circleEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="11" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></circle></marker><marker id="gd17_flowchart-v2-circleStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-1" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:1;stroke-dasharray:1,0"></circle></marker><marker id="gd17_flowchart-v2-circleEnd-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refY="5" refX="12.25" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></circle></marker><marker id="gd17_flowchart-v2-circleStart-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-2" refY="5" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width:0;stroke-dasharray:1,0"></circle></marker><marker id="gd17_flowchart-v2-crossEnd" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="12" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width:2;stroke-dasharray:1,0"></path></marker><marker id="gd17_flowchart-v2-crossStart" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="-1" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width:2;stroke-dasharray:1,0"></path></marker><marker id="gd17_flowchart-v2-crossEnd-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="17.7" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width:2.5"></path></marker><marker id="gd17_flowchart-v2-crossStart-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="-3.5" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width:2.5;stroke-dasharray:1,0"></path></marker><g class="root"><g class="clusters"></g><g class="edgePaths"></g><g class="edgeLabels"></g><g class="nodes"><g class="root" transform="translate(0, 4.400000000000006)"><g class="clusters"><g class="cluster " id="gd17-AFTER" data-look="classic"><rect x="8" y="8" width="615.4921875" height="226.05"></rect><g class="cluster-label " transform="translate(208.484375, 8)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">After:</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> subtree</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> shared</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> once</tspan></tspan></text></g></g></g></g><g class="edgePaths"><path d="M349.642,87.913L360.833,85.213C372.024,82.513,394.407,77.113,411.181,74.413C427.956,71.713,439.122,71.713,444.706,71.712L450.289,71.713" id="gd17-L_SH_C1_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_SH_C1_0" data-points="W3sieCI6MzQ5LjY0MTk2MTczOTU0MzcsInkiOjg3LjkxMjUwMDAwMDAwMDAxfSx7IngiOjQxNi43ODkwNjI1LCJ5Ijo3MS43MTI1fSx7IngiOjQ1NC4yODkwNjI1LCJ5Ijo3MS43MTI1fV0=" data-look="classic" marker-end="url(#gd17_flowchart-v2-pointEnd)" style=";"></path><path d="M349.642,154.138L360.833,156.838C372.024,159.538,394.407,164.938,411.181,167.638C427.956,170.338,439.122,170.338,444.706,170.338L450.289,170.338" id="gd17-L_SH_C2_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_SH_C2_0" data-points="W3sieCI6MzQ5LjY0MTk2MTczOTU0MzcsInkiOjE1NC4xMzc1fSx7IngiOjQxNi43ODkwNjI1LCJ5IjoxNzAuMzM3NX0seyJ4Ijo0NTQuMjg5MDYyNSwieSI6MTcwLjMzNzV9XQ==" data-look="classic" marker-end="url(#gd17_flowchart-v2-pointEnd)" style=";"></path></g><g class="edgeLabels"><g class="edgeLabel"><g class="label" data-id="L_SH_C1_0" transform="translate(0, -10.4609375)"><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"></tspan></text></g></g><g><rect class="background" style="stroke: none"></rect></g><g class="edgeLabel"><g class="label" data-id="L_SH_C2_0" transform="translate(0, -10.4609375)"><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"></tspan></text></g></g><g><rect class="background" style="stroke: none"></rect></g></g><g class="nodes"><g class="node default  " id="gd17-flowchart-SH-4" data-look="classic" transform="translate(212.39453125, 121.025)"><rect class="basic label-container" x="-166.89453125" y="-33.1125" width="333.7890625" height="66.225"></rect><g class="label" transform="translate(0, -18.1125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Scan</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> +</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> operators</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">(exchange</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> reuse</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> or</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> cache/persist)</tspan></tspan></text></g></g></g><g class="node default  " id="gd17-flowchart-C1-5" data-look="classic" transform="translate(520.140625, 71.7125)"><rect class="basic label-container" x="-65.8515625" y="-24.3125" width="131.703125" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Branch</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> 1</tspan></tspan></text></g></g></g><g class="node default  " id="gd17-flowchart-C2-7" data-look="classic" transform="translate(520.140625, 170.3375)"><rect class="basic label-container" x="-65.8515625" y="-24.3125" width="131.703125" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Branch</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> 2</tspan></tspan></text></g></g></g></g></g><g class="root" transform="translate(673.4921875, 0)"><g class="clusters"><g class="cluster " id="gd17-BEFORE" data-look="classic"><rect x="8" y="8" width="571.3515625" height="234.85000000000002"></rect><g class="cluster-label " transform="translate(164.83984375, 8)"><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Before:</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> subtree</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> computed</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> twice</tspan></tspan></text></g></g></g></g><g class="edgePaths"><path d="M290.137,67.313L303.889,67.313C317.641,67.313,345.145,67.313,364.48,67.313C383.815,67.313,394.982,67.313,400.565,67.313L406.148,67.313" id="gd17-L_SA_B1_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_SA_B1_0" data-points="W3sieCI6MjkwLjEzNjcxODc1LCJ5Ijo2Ny4zMTI1fSx7IngiOjM3Mi42NDg0Mzc1LCJ5Ijo2Ny4zMTI1fSx7IngiOjQxMC4xNDg0Mzc1LCJ5Ijo2Ny4zMTI1fV0=" data-look="classic" marker-end="url(#gd17_flowchart-v2-pointEnd)" style=";"></path><path d="M335.148,174.738L341.398,174.737C347.648,174.738,360.148,174.738,371.982,174.737C383.815,174.738,394.982,174.738,400.565,174.737L406.148,174.738" id="gd17-L_SB_B2_0" class=" edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" data-edge="true" data-et="edge" data-id="L_SB_B2_0" data-points="W3sieCI6MzM1LjE0ODQzNzUsInkiOjE3NC43Mzc1fSx7IngiOjM3Mi42NDg0Mzc1LCJ5IjoxNzQuNzM3NX0seyJ4Ijo0MTAuMTQ4NDM3NSwieSI6MTc0LjczNzV9XQ==" data-look="classic" marker-end="url(#gd17_flowchart-v2-pointEnd)" style=";"></path></g><g class="edgeLabels"><g class="edgeLabel"><g class="label" data-id="L_SA_B1_0" transform="translate(0, -10.4609375)"><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"></tspan></text></g></g><g><rect class="background" style="stroke: none"></rect></g><g class="edgeLabel"><g class="label" data-id="L_SB_B2_0" transform="translate(0, -10.4609375)"><text y="-10.1" text-anchor="middle"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em" text-anchor="middle"></tspan></text></g></g><g><rect class="background" style="stroke: none"></rect></g></g><g class="nodes"><g class="node default  " id="gd17-flowchart-SA-0" data-look="classic" transform="translate(190.32421875, 67.3125)"><rect class="basic label-container" x="-99.8125" y="-24.3125" width="199.625" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Scan</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> +</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> operators</tspan></tspan></text></g></g></g><g class="node default  " id="gd17-flowchart-B1-1" data-look="classic" transform="translate(476, 67.3125)"><rect class="basic label-container" x="-65.8515625" y="-24.3125" width="131.703125" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Branch</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> 1</tspan></tspan></text></g></g></g><g class="node default  " id="gd17-flowchart-SB-2" data-look="classic" transform="translate(190.32421875, 174.7375)"><rect class="basic label-container" x="-144.82421875" y="-33.1125" width="289.6484375" height="66.225"></rect><g class="label" transform="translate(0, -18.1125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Scan</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> +</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> operators</tspan></tspan><tspan class="text-outer-tspan row" x="0" y="1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">(same</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> subtree,</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> recomputed)</tspan></tspan></text></g></g></g><g class="node default  " id="gd17-flowchart-B2-3" data-look="classic" transform="translate(476, 174.7375)"><rect class="basic label-container" x="-65.8515625" y="-24.3125" width="131.703125" height="48.625"></rect><g class="label" transform="translate(0, -9.3125)"><rect></rect><g><rect class="background" style="stroke: none"></rect><text y="-10.1"><tspan class="text-outer-tspan row" x="0" y="-0.1em" dy="1.1em"><tspan font-style="normal" class="text-inner-tspan" font-weight="normal">Branch</tspan><tspan font-style="normal" class="text-inner-tspan" font-weight="normal"> 2</tspan></tspan></text></g></g></g></g></g></g></g></g><defs><filter id="gd17-drop-shadow" height="130%" width="130%"><feDropShadow dx="4" dy="4" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><defs><filter id="gd17-drop-shadow-small" height="150%" width="150%"><feDropShadow dx="2" dy="2" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><linearGradient id="gd17-gradient" gradientUnits="objectBoundingBox" x1="0%" y1="0%" x2="100%" y2="0%"><stop offset="0%" stop-color="#9aa8a3" stop-opacity="1"></stop><stop offset="100%" stop-color="hsl(214.2857142857, 0%, 86.2745098039%)" stop-opacity="1"></stop></linearGradient></svg>