htmlineage 0.0.0 → 0.1.0-beta.0

package/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or Derivative
+          Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Varun Jain
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

package/README.md

@@ -1,5 +1,111 @@
-# htmlineage
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset=".github/assets/logo-dark.svg">
+  <img src=".github/assets/logo-light.svg" alt="htmlineage" width="420">
+</picture>
 
-Lineage for AI-generated HTML reports. Every number, chart, and claim traces back to the query that produced it.
+**A provenance layer for AI-generated reports.** Every number, chart, and claim traces back to the query that produced it.
 
-This is a placeholder reserving the name. The first release is coming soon.
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset=".github/assets/viewer-dark.png">
+  <img src=".github/assets/viewer-light.png" alt="The htmlineage viewer: a quarterly business review on the left, and on the right how its &quot;Gap to year-end target&quot; figure was computed">
+</picture>
+
+<sub>The viewer on a sample report. Halden is a fictional company and its data is made up.</sub>
+
+AI writes analytics reports faster than data teams can vet them. htmlineage records where each figure came from — `source → view → figure → insight` — so a reviewer can see the lineage behind any claim before it reaches a CXO deck. A skill teaches your AI assistant to record that lineage while it builds a report, and a small CLI checks it, composes it into one SQL script, and shows it beside the report.
+
+- **Visibility, not verification.** htmlineage shows the lineage; it doesn't assert the numbers are right.
+- **Zero builder friction.** Reports stay plain `.html`. Lineage rides along in an inert `<script type="application/lineage+json">` block and `data-lineage` attributes — invisible to the reader.
+- **SQL-first evidence.** Compose an entire report back into one runnable SQL script for review.
+- **Pinned time.** Queries use absolute date ranges, never `now()`, so a report reproduces exactly.
+
+## Status
+
+**Beta.** The lineage format may still change before `0.1.0`.
+
+## Install
+
+**The skill** follows the open [Agent Skills](https://agentskills.io) standard, so it works with Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot and other tools that read skills. Install it with [skills.sh](https://skills.sh):
+
+```bash
+npx skills add htmlineage/htmlineage -s htmlineage
+```
+
+Or copy [`skills/htmlineage/`](skills/htmlineage/) into your tool's skills folder (for example `.claude/skills/` for Claude Code, `.agents/skills/` for most others).
+
+The skill is opt-in: your assistant uses it when you ask for a report that will be reviewed, traced or shared with leadership, and not for quick questions. To use it for every report in a repository, add a line such as "Every report built here uses htmlineage" to the instructions file your tool reads (`AGENTS.md`, `CLAUDE.md` or similar).
+
+**The CLI** needs Node 18 or later and nothing else. There is nothing to install: run it with `npx`.
+
+```bash
+npx htmlineage --help
+```
+
+## Build and check a report
+
+### 1. Ask for a report
+
+Tell your assistant you want a report that will be reviewed, traced or shared with leadership. The assistant writes the report as plain HTML and, beside it, the lineage graph in `.lineage/<report>.json`.
+
+### 2. Check it
+
+This validates the graph against the report, embeds it into the HTML, and prints a summary. It exits `0` when the report passes (warnings still pass) and `1` when it does not.
+
+```bash
+npx htmlineage check report.html
+```
+
+It reads the graph from `.lineage/<report>.json` beside the report, or else the block already embedded in the report; `--graph <path>` reads that file instead. Use `--no-embed` to leave the file untouched and `--json` for machine-readable output. The rules and what each message means are in [CHECKS.md](https://github.com/htmlineage/htmlineage/blob/main/CHECKS.md).
+
+### 3. Open it
+
+This serves a two-pane page on 127.0.0.1: the report on the left, the lineage of whatever you click on the right.
+
+```bash
+npx htmlineage view report.html
+```
+
+### 4. Compose the SQL
+
+This joins every source, view, and figure into a single script a reviewer can paste into the warehouse. A report that fails `check` is refused.
+
+```bash
+npx htmlineage sql report.html > report.sql
+npx htmlineage sql report.html --id f3 > f3.sql
+```
+
+`--id <node>` composes only that node and everything it rests on.
+
+Each command has its own options: run `npx htmlineage <command> --help`.
+
+## Review a report
+
+To review a report someone else built, ask your assistant to review it. The skill then follows [the review guide](skills/htmlineage/references/review.md): read the check summary, trace the numbers that carry the report, and say where the evidence is weak.
+
+To do it yourself, run `check`, click through the report in `view`, and run `sql --id` on the numbers that matter.
+
+`view` serves the report, its graph and files beside the report such as images, only on 127.0.0.1. It never serves any other file whose name starts with a dot or that lies outside the report's folder, and it needs no database or credentials. More in [SECURITY.md](SECURITY.md).
+
+## How lineage is recorded
+
+A report's graph is a JSON object with a `nodes` map. Each node is one of four steps:
+
+| Step        | Id  | What it is                                                              |
+| ----------- | --- | ----------------------------------------------------------------------- |
+| **source**  | `q` | One wide query per dataset, with a pinned date range                    |
+|             | `x` | An external source htmlineage cannot see inside (a CSV, a document)     |
+| **view**    | `v` | A `select` over sources or views, shown as a table or chart             |
+| **figure**  | `f` | A single value picked from a view, or a SQL expression over figures     |
+| **insight** | `i` | A claim resting on views, figures or insights, or flagged as ungrounded |
+
+Elements on the page point at nodes:
+
+```html
+<div class="kpi-value" data-lineage="f1">46.6%</div>
+```
+
+Every field of every step is in the [field reference](skills/htmlineage/references/format.md), and a complete short report with its graph is in the [worked example](skills/htmlineage/references/example/).
+
+## License
+
+Apache-2.0 © Varun Jain

package/index.js

@@ -0,0 +1,4 @@
+#!/usr/bin/env node
+import { main } from "./src/cli.js";
+
+process.exitCode = await main();

package/package.json

@@ -1,7 +1,57 @@
 {
   "name": "htmlineage",
-  "version": "0.0.0",
-  "description": "Lineage for AI-generated HTML reports — every number, chart, and claim traces back to the query that produced it. (Name reserved; first release coming soon.)",
+  "version": "0.1.0-beta.0",
+  "description": "A provenance layer for AI-generated reports — every number, chart, and claim traces back to the query that produced it.",
+  "type": "module",
+  "main": "index.js",
+  "bin": {
+    "htmlineage": "index.js"
+  },
+  "files": [
+    "index.js",
+    "src/",
+    "skills/"
+  ],
+  "engines": {
+    "node": ">=18"
+  },
+  "scripts": {
+    "test": "node --test test/*.test.js",
+    "lint": "eslint .",
+    "lint:fix": "eslint . --fix",
+    "format": "prettier --write .",
+    "format:check": "prettier --check .",
+    "check:package": "node scripts/check-package.mjs",
+    "start": "node index.js"
+  },
+  "keywords": [
+    "htmlineage",
+    "provenance",
+    "lineage",
+    "data-lineage",
+    "sql",
+    "reporting",
+    "analytics",
+    "ai"
+  ],
   "author": "Varun Jain <varunjainvj1@gmail.com>",
-  "license": "Apache-2.0"
+  "license": "Apache-2.0",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/htmlineage/htmlineage.git"
+  },
+  "bugs": {
+    "url": "https://github.com/htmlineage/htmlineage/issues"
+  },
+  "homepage": "https://github.com/htmlineage/htmlineage#readme",
+  "devDependencies": {
+    "@commitlint/cli": "^19.6.1",
+    "@commitlint/config-conventional": "^19.6.0",
+    "@eslint/js": "^9.17.0",
+    "eslint": "^9.17.0",
+    "prettier": "^3.4.2"
+  },
+  "dependencies": {
+    "node-html-parser": "^7.1.0"
+  }
 }

package/skills/htmlineage/SKILL.md

@@ -0,0 +1,173 @@
+---
+name: htmlineage
+description: Records where every number in an HTML data report came from, so a reviewer can trace each figure, chart and claim back to the query behind it. Use when the user asks for a report that will be reviewed, traced or shared with leadership, asks for lineage or htmlineage by name, or when project instructions say reports use htmlineage. Also use to review such a report or explain how one of its numbers was computed. Skip it for quick, throwaway analysis unless asked.
+---
+
+# htmlineage
+
+While you build a report, you also write down how each thing on the page was
+made: which query, which step, which value. That record is the lineage graph.
+`htmlineage check` validates it and embeds it in the report. A reviewer can
+then open any number and see the SQL behind it.
+
+This changes nothing about how you query, chart or write. The report looks the
+same with or without it. You add the graph and you run `check`.
+
+## When to use it
+
+Use it when the report will be reviewed before it is shared: the user says so,
+asks for lineage or htmlineage, or the project's instructions say every report
+there uses it. Do not use it for a quick look at some data, or when the user
+says they do not need it. When you are unsure, ask once.
+
+## The shape
+
+Every value reaches the page by one path:
+
+1. **Source.** Pull once, wide, per mart: one query per dataset, with the
+   reporting period as literal dates.
+2. **View.** Make a view for everything you show: each chart and each table
+   is a SELECT over sources or other views.
+3. **Figure.** Pick each number from a view: one cell. Or derive it from other
+   figures with an expression.
+4. **Insight.** Write each claim over the figures, views and insights it
+   rests on.
+
+If a claim rests on nothing you queried, keep it and mark it ungrounded. Never
+leave it out to make the report look cleaner.
+
+## Workflow
+
+1. Query the warehouse the way you normally would. Write each source into the
+   graph as you run it.
+2. Write the graph to `.lineage/<report-stem>.json`, beside the report. For
+   `growth-review.html` that is `.lineage/growth-review.json`. Add views,
+   figures and insights to it as you build the page, not at the end.
+3. Put `data-lineage="<id>"` on every element that shows a view, figure or
+   insight. A number inside a sentence gets its own span.
+4. Run `npx htmlineage check <report>.html`.
+5. Fix every error. Most messages end with what to change. Run `check` again.
+6. Fix the warnings you can. A warning does not fail the report, but each one
+   is something a reviewer will see.
+7. When you edit the page after this, update the graph to match and run
+   `check` again. It is the last thing you do before you hand the report over.
+
+`check` exits 0 when the report passes, with or without warnings, and 1 when
+it does not. Exit 2 means the command was wrong (a bad flag, or no report
+named): fix the command, not the graph. Never write the embedded
+`<script>` block yourself; `check` does.
+
+## Pin time
+
+Every source names its period with literal dates, both in `period` and in its
+SQL:
+
+```sql
+where order_date between '2026-08-01' and '2026-08-31'
+```
+
+Never use `current_date`, `now()`, `dateadd(…, current_date)` or "last 30
+days". A report must give the same numbers when it is run again next month.
+
+## Patterns
+
+- **Ranks.** Add a `rank() over (…)` column to the view already on the page,
+  then pick the rank as a figure. Do not make a separate unbound view for it.
+- **Shares.** Pick from the view on the page with a subquery in the figure's
+  SQL, for example a channel's count over the total, when no shown view has
+  the share as a column.
+- **A change or a ratio.** A figure with `expr` over other figures, such as
+  `(f1 - f2) / f2`. It has no `sql`.
+- **A change in a rate.** When a percentage moves, say whether the change is
+  in points or relative: "up 4.9 points" or "a relative gain of 11.8%", never
+  a bare "up 11.8%".
+- **A name.** A figure can hold text, such as the top channel's name. Its
+  `value` and `display` are the name.
+- **A number you were told.** When someone hands you a number or a file (a
+  target, a forecast, a CSV), make an external source with `given` saying what
+  you were handed and by whom. Put a figure over it, and bind the figure. Never
+  bind the source.
+- **Copy `display` exactly.** The span's text is the figure's `display`,
+  character for character: `46.6%`, not `46.60%` or `47%`.
+- **Metric names.** Write `30-day retention`, not "retention over 30 days".
+  `check` reads a number joined to a time unit by a hyphen as part of a
+  definition, not a claim to trace.
+- **Opinion and forecast.** A claim from judgement or outside knowledge is an
+  insight with `"ungrounded": true` and `"from": []`. Ungrounded means no data
+  backs it. A projection you can compute from figures, such as "at this pace
+  we end the year at $1.47M", is an `expr` figure and the claim rests on it.
+- **Every number a claim leans on.** If a claim depends on a quantity, that
+  quantity is a figure shown on the page and the claim rests on it. "All of
+  the growth came from new customers" needs new MRR on the page, not only the
+  totals beside it.
+
+## Describe each step
+
+Sql sources, views and figures carry a `desc`: one line, 120 characters at
+most, saying how the step was computed. A reviewer reads it before the SQL.
+
+- Use business words. Name the population, the filter and the time window.
+- No result values: the desc says how, not what came out.
+- No SQL words: no `select`, `join`, column names or table names.
+
+| Step               | Good                                                                          | Bad                                                           |
+| ------------------ | ----------------------------------------------------------------------------- | ------------------------------------------------------------- |
+| sql source         | Retention by channel and cohort month, January to August 2026                 | select channel, retention_30d from marts.retention_by_channel |
+| view               | Average 30-day retention for each channel and month, ranked within each month | Retention by channel, paid social leads at 46.6%              |
+| figure with `sql`  | Paid social's 30-day retention for the August 2026 cohort                     | Pick from v1 where channel = paid_social                      |
+| figure with `expr` | Change in paid social retention from July to August, relative to July         | (f1 - f2) / f2                                                |
+
+A figure's `label` is what the number _is_ ("Paid social retention,
+August"). Its `desc` is how it was _computed_.
+
+## SQL style
+
+The composed SQL is what a reviewer reads, so write it to be read:
+
+- Lowercase keywords, one clause per line, one column per line.
+- Name every computed column: `avg(retention_30d) as retention`.
+- Aliases of three letters or more.
+- A view or figure names its parents by id, as table names: `from q1`,
+  `from v2`.
+- List columns. No `select *`.
+
+## Don't
+
+- Give a figure its own query against the warehouse. It picks from a view or
+  source that is already there.
+- Leave a number in prose outside a `data-lineage` span.
+- Use `current_date` or any relative date, anywhere.
+- Chain views deeply. Each link is one more thing a reviewer has to read;
+  two deep is usually enough.
+- Drop a claim you cannot source. Mark it ungrounded instead.
+
+## When asked how a number was computed
+
+For a number, chart or claim in a report, your own or one you were handed,
+take the id from the element's `data-lineage` attribute and run:
+
+```
+npx htmlineage sql <report>.html --id <id>
+```
+
+It prints one runnable script: the source, every view and the figure, in
+order. Answer from that script rather than by reading the graph by hand.
+`npx htmlineage view <report>.html` opens the report beside its lineage for a
+person to click through.
+
+## When asked to review a report
+
+Follow [references/review.md](references/review.md). In short: read the check
+summary with `--no-embed`, trace the numbers the report rests on with
+`sql --id`, and report what is weak. Do not edit the report.
+
+## References
+
+- [references/format.md](references/format.md): every field of every kind of
+  node, what may rest on what, ids, and binding. Open it when you write your
+  first graph, or when `check` names a field you do not recognise.
+- [references/example/](references/example/): a complete short report with
+  its graph, the order it was built in, and the `check` output it gives. Read
+  it before your first report; copy its shape.
+- [references/review.md](references/review.md): how to review a report
+  someone else built. Open it only when asked to review.