create-yss-spec 3.5.2 → 3.5.4

package/template/.codex/skills/data-analytics/skills/user-context/references/semantic-layer/setup.md

@@ -1,65 +0,0 @@
-# Semantic Layer Setup
-
-Use this reference when onboarding, direct setup, or maintenance work needs to create, refresh, inspect, or repair a Data Analytics semantic layer for a product or business area.
-
-A semantic layer is a source-backed local skill that helps future Data Analytics runs answer data questions with the right metrics, tables, joins, grains, dimensions, caveats, and query patterns. The output is an agent-facing Codex skill, not a warehouse migration, semantic modeling project, or BI rebuild.
-
-## Required References
-
-- Read `source-intake.md` before asking the user for inputs or deciding which sources are enough.
-- Read `connector-playbook.md` before crawling links, dashboards, team communication channels, data warehouse sources, code repositories, docs, or local skills.
-- Read `skill-template.md` before drafting or creating the target semantic-layer skill.
-- Read `weekly-polling-automation.md` before offering, creating, or updating recurring source polling.
-
-## Setup Contract
-
-- Identify or infer one coherent product, business, metric, source, or reporting area from the user's provided context or sources. Also identify intended users, destination, and whether the user wants intake-only planning, evidence crawling, a draft skill, direct local skill creation, refresh, inspection, or repair.
-- Default direct local creations to `$CODEX_HOME/skills/<area>-semantic-layer` unless the user chose another destination. Ask before placing a generated semantic-layer skill inside a plugin.
-- Create one target semantic-layer skill per coherent area by default. Keep unrelated areas separate unless the user explicitly wants one cross-product layer.
-- Do not require the user to name the area up front. Require user-provided or user-approved useful context or at least one starting source before a source-backed crawl. If the user has only broad intent and no source or context, ask for one item from `source-intake.md`; if they want a skeleton only, label it as ungrounded.
-- Build a source inventory before crawling. For direct creation or draft-file work, write it to `references/source-inventory.md`; for intake-only work, return it in chat. Before saving, summarize sources checked, missing high-value source lanes, and rejected or lower-confidence candidates.
-- Resolve connectors lazily when the active source lane needs them. Prefer the most specific installed first-party connector or source tool; when unavailable, ask for the smallest useful export, pasted excerpt, local checkout, SQL text, schema description, screenshot, or alternate source. If the user selected a source during onboarding but the current semantic-layer lane has not read it yet, use the configured route from `../source-category-runtime.md` and attempt the actual read only when that lane needs the source.
-- Do not create or refresh a semantic-layer skill from ambient connector availability, saved user context, or guessed app results alone. The user must provide or approve the inputs, or explicitly choose to skip or defer setup.
-- Keep raw sensitive data, credentials, secrets, row-level customer examples, and long private messages out of generated files.
-
-## Evidence And Synthesis
-
-Cover the available evidence lanes: tables and namespaces, verified dashboards, raw SQL, team communication, data documentation, code repositories, and existing local skills.
-
-Infer expected source lanes from the user's target metric, grain, domain, and supplied starting points. For example, a daily revenue semantic layer should look for daily-grain warehouse evidence, revenue metric docs or dashboards, and revenue-specific team communication before accepting weekly subscription or generic analytics sources as sufficient. If the best reachable sources are nearby but not exact, label the layer `Limited` or `Directional`, record the mismatch, and ask for the smallest source that would close the gap when the mismatch changes future answers.
-
-Use this source precedence unless the user provides a stronger local rule:
-
-1. Transformation code, tests, and authoritative data documentation, including maintained docs, metric dictionaries, source-backed semantic-layer skills, and user-trusted canonical data skills.
-2. Verified dashboards and reviewed SQL.
-3. Table metadata, lineage, schema comments, and owners.
-4. Query history as observed usage evidence.
-5. Team communication announcements or discussion, weighted by recency, authoritativeness, and durable links.
-6. Other existing local skills, treated as helpful context until corroborated.
-
-A user can promote a local data skill by naming it as canonical or trusted for the current domain; treat that as authoritative data documentation when it has clear scope and explicit metric, table, grain, or exclusion rules. Still verify high-stakes, stale, time-sensitive, or conflicting claims against the underlying cited source or connected app when possible.
-
-Flag unresolved conflicts instead of smoothing them over. The generated skill should say which metric or table definition is canonical, which is deprecated or dashboard-specific, and what future agents should verify before high-stakes answers.
-
-## Create Or Refresh The Target Skill
-
-- Name the target skill after the area and job, usually `<area>-semantic-layer` or `<area>-data-semantics`.
-- Keep the generated `SKILL.md` focused on how future agents should answer data questions, where to find canonical context, and when to verify. Put detailed metrics, tables, query patterns, gotchas, and evidence into directly linked references.
-- Make the layer discoverable by recording the durable pointer in `$CODEX_HOME/state/plugins/data-analytics/user-context.md` under `# Semantic Layers`: area, skill name, absolute path, source inventory path, last updated date, and future-use guidance.
-- Record operational setup or refresh status in `$CODEX_HOME/state/plugins/data-analytics/onboarding-state.json`.
-- Validate generated standalone skills with the relevant authoring validator when files are created locally. Do not claim the skill is invokable until the runtime can see the top-level skill; Codex reload may be required.
-
-## Output Contract
-
-- Intake-only: return source inventory, missing high-value inputs, connector readiness, and crawl plan.
-- Crawl-and-synthesis: return semantic-layer summary, source coverage, unresolved conflicts, confidence notes, and target skill structure.
-- Direct create or refresh: return created or updated file paths, durable pointer written, validation results, source-use checkpoint, connector gaps, activation note when relevant, and the weekly polling resolution.
-- Blocked: state the exact missing connector, permission, source, or destination decision and provide the best manual fallback.
-
-## Quality Bar
-
-- Preserve provenance for every important metric, table, query pattern, and caveat.
-- Keep generated instructions operational: future agents should know which source to check first and what to do when sources disagree.
-- Prefer compact semantic-layer references over copied artifacts.
-- Continue with a partial skill when enough source-backed context exists, but label coverage honestly.
-- Ask before external writes, dashboard changes, team communication posts, repo changes, connector installs, or recurring automation creation.

package/template/.codex/skills/data-analytics/skills/user-context/references/semantic-layer/skill-template.md

@@ -1,160 +0,0 @@
-# Semantic Layer Skill Template
-
-Use this reference when drafting the generated semantic-layer skill. The target skill should be small enough to load quickly and should route detailed data knowledge into references.
-
-## Recommended File Shape
-
-```text
-<skills-root>/
-  <area>-semantic-layer/
-    SKILL.md
-    references/
-      source-inventory.md
-      semantic-layer.md
-      evidence.md
-```
-
-Use more reference files only when the area is large enough to justify them, such as `metrics.md`, `tables.md`, `query-patterns.md`, or `gotchas.md`.
-
-For multiple product or business areas, create one folder per area. Keep shared source links or cross-skill references explicit rather than merging unrelated semantics into one large skill.
-
-## Target SKILL.md Shape
-
-```markdown
----
-name: <area>-semantic-layer
-description: Use when answering data questions for <area>, including metric definitions, table choice, joins, dashboard reconciliation, freshness, and known caveats.
----
-
-# <Area> Semantic Layer
-
-Use this skill to answer <area> data questions with the source-backed context in `references/semantic-layer.md`.
-
-## Start Here
-
-1. Read `references/semantic-layer.md`.
-2. Use the listed canonical metrics, tables, grains, joins, filters, and caveats.
-3. Check freshness before answering time-sensitive questions.
-4. When sources disagree or coverage is weak, say so and verify against the cited source.
-
-## References
-
-- `references/semantic-layer.md`: metrics, tables, filters, query patterns, gotchas, freshness, and open questions.
-- `references/source-inventory.md`: sources checked, coverage level, permissions, rejected candidates, and update boundaries.
-- `references/evidence.md`: detailed provenance, only when the layer needs separate evidence tracking.
-
-## Answering Rules
-
-- Treat this skill as source-selection guidance, not as a substitute for live reads.
-- Preserve metric grain, time zone, date columns, filters, and join keys.
-- Label stale, inferred, partial, or conflicted evidence.
-```
-
-## semantic-layer.md Shape
-
-```markdown
-# <Area> Semantic Layer
-
-## Quick Reference
-
-- Area:
-- Intended users:
-- Coverage level:
-- Source inventory: `references/source-inventory.md`
-- Last synthesized:
-- Freshness expectations:
-- Default date and time zone rules:
-
-## Entity Clarification
-
-Use this section to disambiguate names that future agents may confuse.
-
-| Entity | Means | Does Not Mean | Primary IDs | Grain Notes | Sources |
-| --- | --- | --- | --- | --- | --- |
-
-## Key Metrics
-
-| Metric | Definition | Numerator | Denominator | Time Grain | Canonical Source | Caveats |
-| --- | --- | --- | --- | --- | --- | --- |
-
-## Standard Filters And Dimensions
-
-| Filter Or Dimension | Default Logic | Override When | Applies To | Sources |
-| --- | --- | --- | --- | --- |
-
-## Key Tables
-
-| Table | When To Use | Grain | Join Keys | Freshness | Caveats | Sources |
-| --- | --- | --- | --- | --- | --- | --- |
-
-## Query Patterns
-
-- Pattern:
-  - Use when:
-  - Key tables:
-  - Required filters:
-  - Common joins:
-  - Example skeleton:
-
-## Gotchas
-
-- Gotcha:
-  - Impact:
-  - How to avoid:
-  - Source:
-
-## Related Dashboards And Docs
-
-| Source | Use It For | Caveats |
-| --- | --- | --- |
-
-## Open Questions
-
-- Question:
-  - Why it matters:
-  - Best owner or source to check next:
-```
-
-## evidence.md Shape
-
-Create `references/evidence.md` only when separate evidence tracking is useful, such as conflicting sources, multiple high-stakes metrics, or a large crawl. Small layers may keep source pointers directly in `references/semantic-layer.md`.
-
-```markdown
-# Evidence Register
-
-| Fact Or Claim | Source Type | Source Link Or Path | Retrieved Or Observed | Confidence | Notes |
-| --- | --- | --- | --- | --- | --- |
-```
-
-## source-inventory.md Shape
-
-Create `references/source-inventory.md` whenever the workflow creates or drafts a target semantic-layer skill package. Intake-only runs may return the same inventory in chat without creating the file.
-
-```markdown
-# Source Inventory
-
-## Coverage
-
-- Coverage level:
-- Sources checked:
-- Missing high-value lanes:
-- Rejected or lower-confidence candidates:
-
-## Sources
-
-| Source | Type | Locator | Connector Or Tool | Permission Status | Last Checked | Supports | Gaps Or Caveats | Automation Eligible | Update Boundary |
-| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
-```
-
-Use `Sources checked`, `Missing high-value lanes`, and `Rejected or lower-confidence candidates` as the pre-save source-use checkpoint. Use `Automation Eligible` to distinguish sources the weekly automation can poll directly from sources that require manual export, one-off permission, or user review. Use `Update Boundary` to say whether the automation may update semantic-layer references automatically, must draft a proposed update, or must only report changes.
-
-## Drafting Rules
-
-- Put durable semantic facts in the generated skill's references, not in chat-only prose.
-- Keep `SKILL.md` operational and short; move table catalogs, metric dictionaries, query examples, and evidence into references.
-- Do not put setup-time policy or full evidence procedures into generated `SKILL.md` files; keep those guardrails in the setup flow and validators.
-- Keep the source inventory current enough that a weekly polling automation can determine what to check and what update boundary applies.
-- Use source links, file paths, dashboard IDs, table names, and repo paths as provenance.
-- Label stale, inferred, query-history-only, or team-communication-only facts.
-- Preserve unresolved conflicts as explicit open questions.
-- Avoid raw sensitive data, credentials, long message quotes, and copied dashboard exports.

package/template/.codex/skills/data-analytics/skills/user-context/references/semantic-layer/source-intake.md

@@ -1,75 +0,0 @@
-# Source Intake
-
-Use this reference to gather enough information to build a source-backed semantic-layer skill without turning intake into a long questionnaire.
-
-## Minimum Viable Intake
-
-Proceed when the user gives enough useful context or sources to organize a layer, such as:
-
-- a target product, feature, business area, metric, dashboard, report, table, question, or team;
-- at least one starting source, such as a table namespace, verified dashboard, raw SQL query, data doc, code repository, notebook, team communication channel, or existing local skill;
-- permission to read the supplied sources, when permission is not already implied by the user's request and available connectors.
-
-Do not require the user to name the product or business area up front. Infer the likely organizing area from the supplied context, split unrelated areas when needed, and ask a concise follow-up only when the missing answer changes the crawl or destination materially. Good first follow-ups are: "Should I split these into separate semantic layers?", "Which source should I treat as canonical if these disagree?", or "Where should the generated skill live?"
-
-## Multiple Areas
-
-Create separate semantic-layer skills for separate product or business areas by default. If the user asks for multiple areas, collect a target area and source inventory for each one, then run the crawl and synthesis separately so each output stays compact and triggerable. Use a shared/common semantic layer only when multiple areas depend on the same substantial set of canonical metrics, tables, joins, or caveats.
-
-## Starting Point Menu
-
-Ask for any starting points the user has and explain why each kind can help. Tailor the menu to source lanes they have not already supplied instead of repeating known inputs. Do not imply that all inputs are required.
-
-Infer useful source families from the user's question even when they do not name connectors. For metric-heavy requests, expect structured data plus metric docs, dashboards, or reviewed SQL. For definition or business-context requests, expect docs and team communication. For freshness, lineage, or grain questions, expect table metadata, transformation code, or owner notes. Use connected apps that can satisfy those source families, but do not treat ambient app availability as user approval to create a layer.
-
-Use or adapt this intake prompt:
-
-```text
-Set Up Data Context
-
-Codex gets better at data work when it understands the metric definitions, tables, filters, and source-of-truth context your team already uses.
-
-Send anything you would point a new analyst to. I'll organize what you send into reusable guidance for future analysis.
-
-Good starting points:
-1. What this should help with, like a product or business area.
-2. The source of truth for definitions or logic, like transformation code or repos, metric docs, a trusted data skill, reviewed SQL, dashboards, or recurring reports.
-3. Data tables or catalogs you frequently use.
-4. Places where data definitions, caveats, or changes are discussed, like team channels or threads.
-
-Reply with any starting points, or say `skip for now`.
-```
-
-If the user supplies only a broad area name and no starting points or useful context, ask for one or more starting points from this menu before crawling. If they already supplied starting points or useful context, organize those inputs, ask once for any missing high-value lane only when it would materially improve coverage, then proceed from the available inventory. If they cannot provide any starting points, offer a draft-only semantic-layer skill skeleton and label it as ungrounded.
-
-## High-Value Inputs
-
-Request these inputs when available, and record unknowns explicitly when they are not available:
-
-| Input | What To Ask For | Why It Matters |
-| --- | --- | --- |
-| Scope | Product, metric, dashboard, report, audience, or business question | Sets skill scope and trigger language |
-| Canonical guidance | Transformation code, data docs, metric dictionaries, trusted data skills, or source-backed semantic layers | Supplies authoritative metric, table, grain, and exclusion rules |
-| Existing analysis | Verified dashboards, reviewed SQL, notebooks, reports, or recurring analyses | Shows how definitions are used in practice |
-| Data entry points | Tables, schemas, catalogs, namespaces, warehouses, or workspaces | Provides table metadata, lineage, grain, joins, freshness, and query history |
-| Team context | Channels, threads, owner notes, or announcement windows | Captures corrections, deprecations, and known gotchas |
-| Destination and boundaries | Target skill location, source permissions, sensitivity limits, freshness needs, and automation preference | Keeps creation, crawling, and future updates within the user's intent |
-
-## Source Coverage Levels
-
-Use these labels in final output:
-
-- `Limited`: one useful source was crawled, but important lanes are missing.
-- `Directional`: two or more lanes agree on core tables or metrics, but gaps remain.
-- `Strong`: authoritative docs or transformation code plus dashboard or table evidence support the key facts.
-- `Conflicted`: important definitions disagree and need user or owner resolution.
-- `Blocked`: required connector, permission, source, or destination is unavailable.
-
-## Intake Defaults
-
-- Prefer direct links, table names, repo paths, and channel names over broad descriptions.
-- Treat verified dashboards and transformation code as higher-value starting points than team communication discussion.
-- Use recent query history and team communication discussion as observed behavior, not as canonical definitions by default.
-- If the user says "build what you can", produce a partial skill and name the missing source lanes.
-- After a direct creation or refresh with a usable source inventory, offer weekly polling as an optional follow-up instead of assuming the user wants a recurring automation.
-- If the user asks only for a source audit, stop before creating files and return a crawl plan plus coverage assessment.

package/template/.codex/skills/data-analytics/skills/user-context/references/semantic-layer/weekly-polling-automation.md

@@ -1,96 +0,0 @@
-# Weekly Polling Automation
-
-Use this reference when the user wants the generated semantic-layer skill to stay current by polling the given sources every week.
-
-## When To Offer
-
-Offer weekly polling after semantic-layer creation or refresh when the generated skill has a stable target path and a usable `references/source-inventory.md`. Also offer it when the user asks to keep the semantic layer fresh, monitor sources, watch dashboards, or update the skill if definitions change.
-
-Do not create an automation silently. Ask for approval unless the user already gave a direct instruction such as "set up weekly polling" or "keep this updated every week".
-
-Present the offer as its own user-facing follow-up section or final question, not as a caveat, validation note, or buried automation-status sentence. Use a direct question, for example:
-
-```text
-Do you want me to set up weekly source polling for this semantic layer so changes in dashboards, docs, SQL, repos, or team communication are summarized?
-```
-
-If weekly polling cannot be offered yet, name the missing prerequisite and ask for the specific input that would unblock it. If polling was already created or updated, state that clearly with the automation name and cadence instead of asking again.
-
-After creating or repairing the automation, read it back through the available automation inspection path before claiming readiness. If readback fails, report the concrete blocker and do not say weekly polling is active.
-
-## Automation Tool
-
-Use the Codex app automation tool, not raw automation directives. If the automation tool is not already available, use `tool_search` to find `automation_update`.
-
-Choose the automation kind by the update model:
-
-- Use a `cron` automation when the job should run as a detached weekly workspace job and can safely read the source inventory and update local skill files.
-- Use a `heartbeat` automation attached to the current thread when the user wants weekly review in this conversation, wants to approve changes before writes, or the target files are not safely writable from a detached workspace job.
-
-Prefer updating an existing matching automation over creating a duplicate. Name the automation after the generated semantic-layer skill, such as `<area> semantic layer refresh`.
-
-## User-Facing Updates
-
-Every weekly run must end with a concise user-facing run summary. For a `heartbeat`, post the summary in the attached thread. For a detached `cron`, make the summary the automation run output so the user can review what happened from the automation run history.
-
-Use this summary shape:
-
-- Status: `no change`, `updated`, `blocked`, or `conflicted`.
-- Sources checked: list source names and any sources skipped.
-- Changes found: summarize source-backed metric, table, query-pattern, caveat, or source-inventory changes.
-- Files changed: list local file paths, or say `none`.
-- Validation: report pass, fail, or not run.
-- Needs review: list conflicts, permission gaps, connector gaps, or proposed changes that were not written.
-
-For no-change runs, explicitly say that no source-backed changes were found and list the sources checked. Do not send a silent or empty update.
-
-## Schedule Defaults
-
-If the user asks for weekly polling but does not specify timing, propose a practical weekly cadence in the user's locale, such as Monday morning. Do not show raw RRULE strings to the user. When creating the automation, pass the schedule through the automation tool fields rather than embedding the schedule in the prompt.
-
-## Prompt Requirements
-
-The automation prompt must be self-contained. Include:
-
-- target semantic-layer skill path;
-- source inventory path;
-- source precedence and evidence standard;
-- update boundary for each source, or an instruction to read it from `source-inventory.md`;
-- validation command or validator path when known;
-- output expectations for no-change, changed, blocked, and conflicted runs.
-
-Do not include schedule details in the prompt. The schedule belongs in the automation fields.
-
-Use this prompt skeleton and fill in concrete paths and source names:
-
-```text
-Poll the source inventory for the <area> semantic-layer skill and update the skill only if source-backed changes are needed.
-
-Target skill: <absolute-or-user-provided-skill-path>
-Source inventory: <target-skill>/references/source-inventory.md
-
-Read the source inventory first. For each automation-eligible source, use the listed connector or tool to check for relevant changes since the last checked date. Focus on metric definitions, canonical dashboards, table grain, join keys, freshness, owner notes, deprecations, query patterns, and caveats. Respect the update boundary listed for each source.
-
-Apply this source precedence: transformation code, tests, and authoritative data documentation, including maintained docs, metric dictionaries, source-backed semantic-layer skills, and user-trusted canonical data skills; then verified dashboards; then table metadata and lineage; then query history; then team communication context; then other local skills as supporting context. Do not update from team-communication-only, query-history-only, or generic-local-skill-only evidence unless the source inventory says that is acceptable.
-
-If changes are clear, source-backed, and within the allowed update boundary, update the semantic-layer references and evidence register. Keep raw sensitive data, credentials, long private messages, and row-level examples out of the files. Preserve unresolved conflicts as open questions instead of choosing silently.
-
-After edits, validate the target skill if a validator is available. End every run with a user-facing run summary using this shape:
-
-- Status: no change, updated, blocked, or conflicted.
-- Sources checked: source names plus any skipped sources.
-- Changes found: source-backed metric, table, query-pattern, caveat, or source-inventory changes.
-- Files changed: local file paths, or none.
-- Validation: pass, fail, or not run.
-- Needs review: conflicts, permission gaps, connector gaps, or proposed changes that were not written.
-
-If nothing material changed, say that no source-backed changes were found and list the sources checked. Do not produce a silent or empty update.
-```
-
-## Update Rules
-
-- Update generated semantic-layer references only when a change is corroborated by an authoritative source or allowed by the source inventory's update boundary.
-- Preserve user edits and manual notes. If a local manual note conflicts with a source update, report the conflict instead of overwriting it.
-- Never change external dashboards, team communication channels, docs, repos, or source systems as part of the weekly polling automation.
-- If a connector is missing, permission has expired, or a source cannot be read, record the gap in the run summary and leave the skill unchanged for that source.
-- If the automation updates files, validate the standalone skill before reporting success.