tackbox 0.1.69 → 0.1.71

package/README.md

@@ -323,29 +323,40 @@ every lint and writes no clean-cache markers. A `java`-format clone that
 lies entirely within both files' headers (package, imports, leading
 comments) has no extractable code and is dropped before it is reported.
 
-### Markdown: ASCII and the language marker
+### Markdown: declared charset
 
-The Markdown engine runs the standard markdownlint rules plus `MD-ASCII`,
-which flags any non-ASCII character (any codepoint above U+007F) - em
-dashes, curly quotes, other scripts, emoji - keeping docs portable and
-grep-friendly.
+The Markdown engine does not enforce Markdown or prose style. It runs the
+four link-reference built-ins - `MD011` (reversed links), `MD042` (empty
+links), `MD051` (in-file link fragments), `MD052` (reference links
+defined) - plus `MD-CHARS`, which checks a file's character repertoire
+against a declaration the file makes about itself.
 
-One HTML comment on one of the first five lines widens the alphabet for
-a single file:
+The check is opt-in and declaration-driven. With no marker, the charset
+is not checked. One HTML comment on one of the first five lines turns it
+on and names the allowed sets:
 
 ```text
-<!-- tackbox: lang=ru personal experimental repo -->
+<!-- tackbox: chars=cyrillic,punct -->
 ```
 
-It widens the allowed set to that language's script plus its typographic
-punctuation - `ru` today: Cyrillic, guillemets, em/en dash, ellipsis,
-curly quotes, NBSP - and nothing else: every other non-ASCII character,
-emoji and other scripts included, is still flagged.
+With a marker present, every codepoint must be in the always-allowed
+ASCII base (U+0000-U+007F) or in one of the declared sets; anything else
+is a finding. The sets are named by character repertoire, not language:
 
-The marker is single-use and never disables the rule. A second marker, a
-marker past the fifth line, a missing code, or an unknown language code
-is itself a finding and leaves the whole file strict ASCII. Any text
-after the code (as above) is a free-form note.
+- `ascii` adds nothing (it declares the check with no extension);
+- `cyrillic` adds the Cyrillic block U+0400-U+04FF;
+- `punct` adds em/en dash, guillemets, ellipsis, curly and low quotes,
+  and NBSP.
+
+Sets are comma-joined and unioned; a space after a comma is fine
+(`chars=ascii, cyrillic`). Russian prose declares `chars=cyrillic,punct`;
+a grep-friendly zone declares `chars=cyrillic`.
+
+The marker strengthens the check, so it is not a suppression: it draws no
+approval and does not appear in the escapes inventory. An invalid marker
+(an unknown set, an empty list, a duplicate set, a duplicate marker, or a
+marker past the fifth line) is itself a finding, and the content charset
+is then not checked - a broken declaration does not pass silently.
 
 ## No configuration
 
@@ -396,8 +407,6 @@ ordinary comment token:
   engine also accepts a standalone single-line `/* ... */` (see its
   section).
 - Python: a `#` comment.
-- Markdown: the `tackbox: lang=` HTML comment (see the Markdown
-  engine section) is the only Markdown marker.
 - Svelte: inside `<script>` blocks the `//` form works as in JS/TS;
   the template adds two forms - a `//` comment inside a `{...}`
   expression (line-adjacent, as ever) and an HTML comment
@@ -457,6 +466,74 @@ event, `dev.py check`, and CI reports until the entry lands or the
 marker is reverted. Removing a manifest line is free; a marker whose
 text, scope, or count changes needs its entry updated the same way.
 
+## Generated and vendored code
+
+Committed code that carries a generated or vendored git attribute is
+excluded from the whole lint - findings there are not fixable in the
+file (they belong in the generator), and a suppression marker cannot
+survive regeneration. tackbox honors exactly three attributes,
+`linguist-generated`, `gitlab-generated`, and `linguist-vendored`,
+read from `.gitattributes` the same way the host (GitHub, GitLab)
+reads them - no exclude surface of tackbox's own. The best fix stays
+organizational: generated code should normally not be committed at
+all; this serves the forced residue.
+
+A file is excluded when `git check-attr` reports one of the three as
+set. Semantics by example:
+
+```text
+gen/**       linguist-generated
+vendor/**    linguist-vendored
+gen/keep.go  linguist-generated=false
+```
+
+`gen/**` excludes everything under `gen/`; `vendor/**` the same for
+vendored. Note `dir/**`, not `dir/` - gitattributes patterns, unlike
+gitignore, do not match a trailing-slash directory form. `=false`
+re-includes a single file inside an excluded tree (`gen/keep.go` above
+is linted normally); `-attr` and `!attr` also leave a file in. Only
+`set` / `=true` excludes.
+
+The exclusion covers everything: per-file engines, the erclint Go
+package run (a mixed package's excluded file is compiled but its
+findings drop; a compile break still fails loudly), duplication, the
+CodeClimate report, and the marker inventory - an excluded file's
+markers are dead, so a manifest entry addressing one orphans. A lint
+run whose scope touches excluded files prints one summary line:
+
+```text
+excluded by attributes: 12 files in scope (tackbox escapes lists all)
+```
+
+It counts unique excluded files in the current scope (absent at zero),
+so scoped runs are not wallpapered with a global constant;
+`tackbox escapes` lists the full population as `attribute-excluded`
+entries.
+
+Because the excluded population is where the lint, the marker
+inventory, and host diff review are all blind, the agent hook makes
+the two ways into it loud:
+
+- adding a positive exclusion line (a bare `<attr>` or `<attr>=true`)
+  to any `.gitattributes` draws a PreToolUse ask, one joint ask per
+  edit listing every added line; removals, `=false`, `-attr`, and
+  non-exclusion lines are free;
+- editing (or creating) a file that is effective-excluded draws an
+  ask naming the attributes.
+
+Generators run through Bash and are unaffected - the boundary is that
+the change stays in the commit/PR diff, and hosts collapse
+excluded-file diffs, so reviewers must expand them.
+
+`tackbox doctor` adds an informational `attributes` section (never a
+check, no exit-code effect) naming local conditions that can make a
+run diverge from a clean CI clone - an `info/attributes` or an
+untracked/index-hidden `.gitattributes` carrier mentioning the
+attributes, or a neutralized attribute source override. `tackbox
+escapes --since <rev>` resolves the baseline's attributes as of the
+rev and so needs git >= 2.40 (older git is a named infra error on the
+`--since` path only; the plain listing needs no version bump).
+
 ## Runtime reporting helpers
 
 Direct reporting helpers ship per language; their shared runtime behavior -
@@ -483,9 +560,11 @@ Claude Code hook event on stdin and dispatches by `hook_event_name`:
   nothing, and the block repeats on every event until the tree is
   consistent. The authoritative gate stays pre-commit / CI.
 - **PreToolUse** asks for approval before a new `.tackbox/approvals`
-  line or a new `.tackbox/reporters` line lands; removing one is
-  free. Editing markers in code draws no Pre ask - the consistency
-  check owns them.
+  line or a new `.tackbox/reporters` line lands, before a positive
+  exclusion line is added to any `.gitattributes`, and before an edit
+  to an attribute-excluded file (see "Generated and vendored code");
+  removing a line is free. Editing markers in code draws no Pre ask -
+  the consistency check owns them.
 
 Only markers in files an engine would lint participate in the check
 (D012): a marker in a Go `testdata/` path or a non-lintable fixture
@@ -522,8 +601,8 @@ cheap command that review tooling of any harness can consume (D013). It
 enumerates:
 
 - **suppression markers** (`// no-report`, `// parse-skip`,
-  `// nil-return`, `// long-comment`, `// test-skip`, `// dup-ok`, plus
-  the markdown `tackbox: lang=` marker), each with its reason;
+  `// nil-return`, `// long-comment`, `// test-skip`, `// dup-ok`), each
+  with its reason;
 - **`.tackbox/reporters` declarations** - the tier-2 sinks;
 - **notify / quiet lane choices** - the call sites of the user-lane-only
   `notify` and the telemetry-only `quiet` verbs.
@@ -543,9 +622,11 @@ uvx tackbox@latest escapes --since origin/main --context 5
 
 ```json
 {
-  "version": 1,
+  "version": 2,
   "since": null,
   "entries": [
+    {"kind": "attribute-excluded", "file": "gen/api.pb.go",
+     "attribute": "linguist-generated"},
     {"kind": "marker", "file": "a/b.py", "line": 12,
      "text": "no-report: central boundary already captures it",
      "reason": "central boundary already captures it",
@@ -559,29 +640,39 @@ uvx tackbox@latest escapes --since origin/main --context 5
     {"kind": "quiet-site", "file": "go/x.go", "line": 9,
      "text": "report.Quiet(ctx, ...)", "context": ["..."]}
   ],
-  "counts": {"marker": 1, "reporter-decl": 1, "notify-site": 1, "quiet-site": 1}
+  "counts": {"marker": 1, "reporter-decl": 1, "notify-site": 1,
+             "quiet-site": 1, "attribute-excluded": 1}
 }
 ```
 
-- `version` is the schema version (`1`); `counts` always carries all four
-  kinds, even at zero, so consumers see a stable shape.
+- `version` is the schema version (`2`); `counts` always carries all five
+  kinds, even at zero, so consumers see a stable shape. Every count is an
+  entry count except `attribute-excluded`, which counts unique files.
 - `since` echoes the `--since` rev, or `null`.
+- `attribute-excluded` entries carry only `kind` / `file` / `attribute` (no
+  line or text): the whole file is the bypass, one entry per set attribute
+  of the three (`linguist-generated`, `gitlab-generated`,
+  `linguist-vendored`). See "Generated and vendored code".
 - `text` is the trimmed source line; for a marker it runs from the marker
   keyword to end of line.
 - `reason` (markers only) is what follows the keyword's colon, trimmed -
-  possibly empty (the `tackbox: lang=` marker carries none).
+  possibly empty.
 - `context` is the surrounding source, `--context N` lines each side
   (default 3), inclusive of the entry line itself - the window
   `[line-N, line+N]`, clipped at file edges, each line trimmed of trailing
   whitespace. It is plain source; the entry line is not marked.
-- `entries` are sorted by `(file, line)` for stable output.
+- `entries` are sorted by `(file, kind, kind-subkey)` - the subkey is
+  `(line, text)` for the line-bearing kinds and `(attribute,)` for
+  `attribute-excluded`.
 
 ### Scope and detection
 
 The scan covers the same lintable source set the linter would scan (the
 D012 predicate: extension match plus each engine's path filter, so a Go
-`testdata/` file is out), plus the root `.tackbox/reporters` (every
-non-empty line is one declaration - the file has no comment syntax).
+`testdata/` file is out) minus the attribute-excluded files, plus the root
+`.tackbox/reporters` (every non-empty line is one declaration - the file has
+no comment syntax). An attribute-excluded file's own markers are dead, so it
+surfaces only as its `attribute-excluded` entries.
 notify / quiet call sites are detected **textually per language**
 (`report_quiet` / `notify` in Python, `reportQuiet` / `notify` in the JS
 family, `.Quiet(` / `.Notify(` in Go, `.quiet(` / `.notify(` in Java),
@@ -592,12 +683,19 @@ this is observability, not a lint.
 ### `--since <rev>`
 
 `--since <rev>` prints only entries **new against `<rev>`**, compared by
-content identity `(kind, file, text)` - the same extraction run against
-the tree at `<rev>` (via `git ls-tree` + `git show`) subtracted, count
-aware, from the current tree's entries. It over-reports on moved code (a
-new file path is a new identity) but never silently drops an entry - the
-conservative direction for a review aid. A bad rev is the one infra error:
-one stderr line, exit 1.
+content identity (`(kind, file, text)`, or `(kind, file, attribute)` for
+attribute-excluded) - the same extraction run against the tree at `<rev>`
+(via `git ls-tree` + `git show`) subtracted, count aware, from the current
+tree's entries. The baseline is attribute-aware: it resolves the attributes
+as of `<rev>` (via the seam's `git check-attr --source`), so an attribute
+added since the rev reports its newly-excluded files, a removed one
+re-activates its markers as new (never a silent subtraction), and an
+unchanged one adds no noise. Because `--source` needs git >= 2.40, an older
+git is a named infra error on the `--since` path only (the plain listing
+needs no version bump). It over-reports on moved code (a new file path is a
+new identity) but never silently drops an entry - the conservative direction
+for a review aid. A bad rev is the other infra error: one stderr line,
+exit 1.
 
 ## Layout
 

package/bin/tackbox-mdlint.js

@@ -1,7 +1,8 @@
 #!/usr/bin/env node
 const fs = require('fs')
 const { lint } = require('markdownlint/promise')
-const noNonAscii = require('../js/markdownlint-rules/no-non-ascii')
+const declaredChars = require('../js/markdownlint-rules/declared-chars')
+const linkIntegrity = require('../js/markdownlint-rules/link-integrity')
 
 // readFilesFrom reads a newline-separated UTF-8 list-file into its non-empty
 // paths. Additive to positional paths - the bin is public on npm.
@@ -9,25 +10,67 @@ function readFilesFrom(listPath) {
   return fs.readFileSync(listPath, 'utf8').split(/\r?\n/).filter(Boolean)
 }
 
+// Parse the link-target inventory list-file (LF, `<kind>\t<path>` per line) the
+// tackbox CLI writes: F = linkable files, L = tracked symlinks, G = gitlink
+// roots. Repo-relative paths, kept verbatim.
+function readLinkTargets(listPath) {
+  const F = new Set()
+  const L = new Set()
+  const G = []
+  for (const line of readFilesFrom(listPath)) {
+    const tab = line.indexOf('\t')
+    const kind = line.slice(0, tab)
+    const p = line.slice(tab + 1)
+    if (kind === 'F') F.add(p)
+    else if (kind === 'L') L.add(p)
+    else if (kind === 'G') G.push(p)
+  }
+  return { F, L, G }
+}
+
+const USAGE =
+  'tackbox-mdlint: --repo-root <dir> and --link-targets-from <list-file> are ' +
+  'required (the cross-file link rule needs the whole-tree target inventory)\n'
+
 async function run() {
   const argv = process.argv.slice(2)
   const machine = argv.includes('--machine')
   const files = []
+  let repoRoot = null
+  let linkTargetsFrom = null
   for (let i = 0; i < argv.length; i++) {
     const a = argv[i]
     if (a === '--machine') continue
     // The file set rides a list-file, not positional argv (ARG_MAX safety).
     if (a === '--files-from') { files.push(...readFilesFrom(argv[++i])); continue }
+    if (a === '--repo-root') { repoRoot = argv[++i]; continue }
+    if (a === '--link-targets-from') { linkTargetsFrom = argv[++i]; continue }
     files.push(a)
   }
+  if (!repoRoot || !linkTargetsFrom) {
+    process.stderr.write(USAGE)
+    process.exit(2)
+  }
   if (files.length === 0) {
     process.stderr.write('tackbox-mdlint: no files supplied\n')
     process.exit(2)
   }
+  const { F, L, G } = readLinkTargets(linkTargetsFrom)
   const result = await lint({
     files,
-    config: { default: true, 'no-non-ascii': true },
-    customRules: [noNonAscii],
+    // Style preset off; only the link-reference built-ins plus the declared-
+    // charset and cross-file link rules run (D017/D018). noInlineConfig blocks
+    // in-file rule toggles.
+    config: {
+      default: false,
+      MD011: true,
+      MD042: true,
+      MD051: true,
+      MD052: true,
+      'declared-chars': true,
+      'link-integrity': true,
+    },
+    customRules: [declaredChars, linkIntegrity.makeRule({ repoRoot, F, L, G })],
     noInlineConfig: true,
   })
   let count = 0

package/js/README.md

@@ -138,6 +138,47 @@ siblings still report. Recognition reads the enclosing element's
 preceding `SvelteHTMLComment` sibling; `/* ... */` block comments
 are never markers.
 
+## Markdown
+
+The CLI also lints `.md` files through `tackbox-mdlint`, a thin markdownlint
+wrapper. The style preset is off; only the link-reference built-ins (MD011,
+MD042, MD051, MD052) plus two tackbox rules run:
+
+- `declared-chars` (MD-CHARS) - a file's character repertoire is checked only
+  when it declares one with `<!-- tackbox: chars=... -->` (D017).
+- `link-integrity` (MD-LINK) - a relative link or image must resolve to an
+  existing in-repo target, and a `#fragment` into a target `.md` must name a
+  real heading slug or HTML anchor (D018). Cross-file only; MD051 holds a
+  file's own fragments. Fully offline: any URI scheme or absolute path is out
+  of scope.
+
+The link rule needs the whole-tree set of valid targets, so the bin takes two
+mandatory flags - a call missing either is a usage error (exit 2):
+
+```bash
+tackbox-mdlint --repo-root <dir> --link-targets-from <list-file> \
+  [--files-from <list-file>] [files...]
+```
+
+- `--repo-root <dir>` - the repository root; linted files and targets are
+  normalized to it, so a call from a subdirectory resolves correctly.
+- `--link-targets-from <list-file>` - the link-target inventory (below).
+- `--files-from <list-file>` - the files to lint, one per line (additive to
+  positional paths; keeps the spawn under ARG_MAX).
+
+The inventory list-file is LF-terminated, one `<kind>\t<path>` per line, paths
+repo-relative:
+
+- `F` - a linkable file (a source-set file, before generated/vendored
+  exclusion: existence is not linting).
+- `L` - a tracked symlink; its target exists but is not dereferenced and its
+  fragment is not checked.
+- `G` - a gitlink (submodule) root; a target under it is skipped.
+
+The tackbox CLI builds this inventory from the git listing and passes it in;
+the flags exist for that one caller and are a deliberate breaking change to the
+public bin.
+
 ## Reporter recognition
 
 A call counts as a reporter only when its callee resolves to one of the

package/js/markdownlint-rules/declared-chars.js

@@ -0,0 +1,155 @@
+// Custom markdownlint rule: a Markdown file's character repertoire is checked
+// only when the file declares one. Semantics are declaration-driven, not a
+// strict-ASCII default (D017):
+//
+//   * No marker -> the charset is not checked at all.
+//   * A marker -> every codepoint must be in the always-allowed ASCII base
+//     (U+0000-U+007F, the Markdown syntax alphabet) or in one of the declared
+//     named sets. Everything else is a finding.
+//
+//   <!-- tackbox: chars=cyrillic,punct -->
+//
+// Sets are named by character repertoire, not by language (a set proves nothing
+// about the prose's language). The marker lists them comma-joined (union);
+// tokens are trimmed, so a space after a comma is allowed. An invalid marker -
+// an unknown set, an empty token, a duplicate set, a duplicate marker, or a
+// marker below the fifth line - is a finding on the marker itself, and the
+// content charset is then not checked (a broken declaration does not pass
+// silently; there is no default to fall back to).
+//
+// The marker is read from micromark HTML-comment tokens, not params.lines:
+// markdownlint masks HTML-comment interiors in `lines`, so the raw code is
+// only visible in the parse tree.
+
+// Named character sets: extra codepoints a set adds beyond the ASCII base. Add
+// a set by adding one entry (its script range(s) and/or individual points).
+const CHAR_SETS = {
+  // Declares the check with no extension; the ASCII base is always allowed.
+  ascii: { ranges: [], points: [] },
+  // Cyrillic block U+0400-U+04FF in full.
+  cyrillic: { ranges: [[0x0400, 0x04ff]], points: [] },
+  // Typographic punctuation: em/en dash, guillemets, ellipsis, curly
+  // single/double quotes (incl. low opening quotes), NBSP.
+  punct: {
+    ranges: [],
+    points: [
+      0x2014, 0x2013, 0x00ab, 0x00bb, 0x2026,
+      0x2018, 0x2019, 0x201c, 0x201d, 0x201e, 0x201a, 0x00a0,
+    ],
+  },
+}
+
+const MARKER_MAX_LINE = 5
+const MARKER_RE = /<!--\s*tackbox:\s*chars=([^>]*?)\s*-->/g
+
+function collectMarkers(token, found) {
+  for (const m of token.text.matchAll(MARKER_RE)) {
+    const before = token.text.slice(0, m.index)
+    const lineOffset = (before.match(/\n/g) || []).length
+    const lastNl = before.lastIndexOf('\n')
+    found.push({
+      lineNumber: token.startLine + lineOffset,
+      list: m[1],
+      col: lineOffset === 0 ? token.startColumn + m.index : m.index - lastNl,
+      len: m[0].length,
+    })
+  }
+}
+
+// Every marker occurrence in the file's HTML comments: {lineNumber, list, col,
+// len}. Walks the micromark tree; htmlFlow / htmlText carry the raw comment
+// text (their children just re-slice it, so we do not descend).
+function findMarkers(tokens) {
+  const found = []
+  const walk = (toks) => {
+    for (const t of toks) {
+      if (t.type === 'htmlFlow' || t.type === 'htmlText') {
+        collectMarkers(t, found)
+        continue
+      }
+      if (t.children && t.children.length) walk(t.children)
+    }
+  }
+  walk(tokens)
+  return found
+}
+
+// Validate the marker set; emit findings for a misplaced / duplicate / invalid
+// marker. Return the allowed set list for a single valid marker (to check
+// content against), or null (do not check content) for no marker or any invalid
+// one.
+function resolveMarkers(markers, onError) {
+  if (markers.length === 0) return null
+  const markerErr = (m, detail) =>
+    onError({ lineNumber: m.lineNumber, detail, range: [m.col, m.len] })
+
+  if (markers.length > 1) {
+    for (const dup of markers.slice(1)) {
+      markerErr(dup, 'duplicate tackbox chars marker (one marker per file)')
+    }
+    return null
+  }
+  const m = markers[0]
+  if (m.lineNumber > MARKER_MAX_LINE) {
+    markerErr(m, `tackbox chars marker must be within the first ${MARKER_MAX_LINE} lines`)
+    return null
+  }
+  const tokens = m.list.split(',').map((t) => t.trim())
+  if (tokens.some((t) => t === '')) {
+    markerErr(m, 'tackbox chars marker: empty character-set list')
+    return null
+  }
+  const seen = new Set()
+  const allowed = []
+  for (const t of tokens) {
+    if (seen.has(t)) {
+      markerErr(m, `tackbox chars marker: duplicate set '${t}'`)
+      return null
+    }
+    seen.add(t)
+    const set = CHAR_SETS[t]
+    if (!set) {
+      markerErr(m, `tackbox chars marker: unknown character set '${t}'`)
+      return null
+    }
+    allowed.push(set)
+  }
+  return allowed
+}
+
+function isAllowed(code, allowed) {
+  if (code <= 0x7f) return true
+  for (const set of allowed) {
+    for (const [lo, hi] of set.ranges) {
+      if (code >= lo && code <= hi) return true
+    }
+    if (set.points.includes(code)) return true
+  }
+  return false
+}
+
+module.exports = {
+  names: ['MD-CHARS', 'declared-chars'],
+  description: 'Character outside declared repertoire',
+  tags: ['charset'],
+  parser: 'micromark',
+  function: function rule(params, onError) {
+    const allowed = resolveMarkers(findMarkers(params.parsers.micromark.tokens), onError)
+    if (!allowed) return
+    params.lines.forEach((line, idx) => {
+      let col = 0
+      for (const ch of line) {
+        const code = ch.codePointAt(0)
+        if (!isAllowed(code, allowed)) {
+          onError({
+            lineNumber: idx + 1,
+            detail:
+              'U+' + code.toString(16).toUpperCase() + ' (' + ch + ') is not in the declared character set',
+            range: [col + 1, ch.length],
+          })
+        }
+        col += ch.length
+      }
+    })
+  },
+}

package/js/markdownlint-rules/link-integrity.js

@@ -0,0 +1,275 @@
+// Custom markdownlint rule: every relative Markdown link / image target must
+// exist and stay inside the repo, and a #fragment into a target .md must name a
+// real anchor (D018). Cross-file only - MD051 (enabled alongside) already holds
+// same-file fragments.
+//
+// Target existence is decided against a whole-tree link-target inventory the
+// tackbox CLI builds from the raw git listing, NOT fs.exists: a link to a
+// gitignored file is broken in a clean clone, so it is a finding. The inventory
+// is a factory input, so the rule is constructed per run with the repo root and
+// the parsed inventory baked in.
+//
+// Anchor semantics follow the MD051 / GitHub contract. markdownlint does not
+// export its heading-fragment helper, so the GitHub slugger is ported here and
+// pinned by fixtures. A nested markdownlint lint is not an option: its per-file
+// token cache is module-level and reset per file, so parsing a target inside a
+// rule would corrupt the linted file's own results.
+
+const fs = require('fs')
+const path = require('path')
+
+// RFC3986 scheme: an external URL (http, mailto, tel, data, ftp, ...) is out of
+// scope - the rule is fully offline, no network anywhere.
+const SCHEME_RE = /^[a-zA-Z][a-zA-Z0-9+.-]*:/
+
+// GitHub heading slugger, ported verbatim from markdownlint's md051
+// convertHeadingToHTMLFragment minus the encodeURIComponent wrapper: anchors are
+// compared decoded, so unicode headings (legal since the charset flip) match a
+// literal or a percent-encoded link fragment alike.
+function slug(text) {
+  return text
+    .toLowerCase()
+    .replace(/[^\p{Letter}\p{Mark}\p{Number}\p{Connector_Punctuation}\- ]/gu, '')
+    .replace(/ /gu, '-')
+}
+
+// Reduce a raw heading line to its plain inline text before slugging: images
+// drop entirely (GitHub excludes their alt), links keep their visible text, raw
+// HTML / autolinks drop, word-boundary underscore emphasis is removed (intraword
+// `_` stays literal per CommonMark), and backslash escapes become the char. The
+// remaining markup punctuation (`*`, backticks, brackets) is stripped by slug's
+// own punctuation filter, so code spans and asterisk emphasis need no pass here.
+function headingInlineText(raw) {
+  return raw
+    .replace(/!\[[^\]]*\]\([^)]*\)/g, '')
+    .replace(/!\[[^\]]*\](\[[^\]]*\])?/g, '')
+    .replace(/\[([^\]]*)\]\([^)]*\)/g, '$1')
+    .replace(/\[([^\]]*)\]\[[^\]]*\]/g, '$1')
+    .replace(/\[([^\]]*)\]/g, '$1')
+    .replace(/<[^>]*>/g, '')
+    .replace(/(^|[^\p{L}\p{N}])_{1,3}(?=\S)/gu, '$1')
+    .replace(/(?<=\S)_{1,3}($|[^\p{L}\p{N}])/gu, '$1')
+    .replace(/\\(.)/g, '$1')
+}
+
+const ID_RE = /<[^>]*?\bid\s*=\s*["']([^"']+)["']/gi
+const NAME_RE = /<a\b[^>]*?\bname\s*=\s*["']([^"']+)["']/gi
+
+function collectHtmlAnchors(line, anchors) {
+  let m
+  while ((m = ID_RE.exec(line)) !== null) anchors.add(m[1])
+  while ((m = NAME_RE.exec(line)) !== null) anchors.add(m[1])
+}
+
+function addHeadingAnchor(rawText, anchors, counts) {
+  const s = slug(headingInlineText(rawText))
+  if (s === '') return
+  const c = counts.get(s) || 0
+  // A duplicate heading slug gets the -1 / -2 ... suffix GitHub assigns.
+  if (c > 0) anchors.add(s + '-' + c)
+  anchors.add(s)
+  counts.set(s, c + 1)
+}
+
+// Anchor set of a target .md's text: heading slugs (with GitHub duplicate
+// suffixes), HTML id= / <a name=> anchors, and #top (always valid). Text-based
+// so no markdownlint reentrancy; generous extraction (extra anchors) only ever
+// misses a broken link, never invents one.
+function computeAnchors(text) {
+  const anchors = new Set(['top'])
+  const counts = new Map()
+  const lines = text.split(/\r?\n/)
+  let fence = null
+  for (let i = 0; i < lines.length; i++) {
+    const line = lines[i]
+    const open = line.match(/^ {0,3}(`{3,}|~{3,})/)
+    if (fence === null && open) {
+      fence = open[1][0]
+      continue
+    }
+    if (fence !== null) {
+      if (new RegExp('^ {0,3}' + fence + '{3,}[ \\t]*$').test(line)) fence = null
+      continue
+    }
+    const atx = line.match(/^ {0,3}(#{1,6})(?:[ \t]+(.*?))?[ \t]*$/)
+    if (atx) {
+      addHeadingAnchor((atx[2] || '').replace(/[ \t]+#+[ \t]*$/, ''), anchors, counts)
+      collectHtmlAnchors(line, anchors)
+      continue
+    }
+    const next = i + 1 < lines.length ? lines[i + 1] : ''
+    if (
+      line.trim() !== '' &&
+      !/^ {0,3}#/.test(line) &&
+      /^ {0,3}(=+|-+)[ \t]*$/.test(next)
+    ) {
+      addHeadingAnchor(line.trim(), anchors, counts)
+    }
+    collectHtmlAnchors(line, anchors)
+  }
+  return anchors
+}
+
+// Percent-decode without throwing (unlike decodeURIComponent): each maximal run
+// of %XX escapes is decoded as UTF-8 bytes, invalid bytes become U+FFFD, and a
+// stray % stays literal. A malformed link must never crash the lint.
+function percentDecode(s) {
+  return s.replace(/(?:%[0-9A-Fa-f]{2})+/g, (seq) => {
+    const bytes = new Uint8Array(seq.length / 3)
+    for (let i = 0; i < bytes.length; i++) {
+      bytes[i] = parseInt(seq.slice(i * 3 + 1, i * 3 + 3), 16)
+    }
+    return new TextDecoder('utf-8').decode(bytes)
+  })
+}
+
+// Split a raw destination into { pathPart, fragment }, or null to skip it. The
+// fragment is everything after the first '#'; the query before it is dropped.
+// External schemes, absolute paths, empty and same-file (#...) destinations are
+// skipped - the latter is MD051's job.
+function splitDest(dest) {
+  if (dest === '' || SCHEME_RE.test(dest) || dest.startsWith('/') || dest.startsWith('#')) {
+    return null
+  }
+  const hash = dest.indexOf('#')
+  let before = hash >= 0 ? dest.slice(0, hash) : dest
+  const fragment = hash >= 0 ? dest.slice(hash + 1) : null
+  const q = before.indexOf('?')
+  if (q >= 0) before = before.slice(0, q)
+  if (before === '') return null
+  return { pathPart: percentDecode(before), fragment: fragment === null ? null : percentDecode(fragment) }
+}
+
+// The rule's own destination string(s), never a nested link/image's: prune the
+// walk at nested link/image boundaries so `[![alt](img)](target)` reports both
+// img and target, each against its own token.
+function ownLinkData(linkToken) {
+  const data = { dest: null, ref: null, label: null }
+  const rec = (toks) => {
+    for (const t of toks) {
+      if (t !== linkToken && (t.type === 'link' || t.type === 'image')) continue
+      if (t.type === 'resourceDestinationString' && data.dest === null) data.dest = t.text
+      if (t.type === 'referenceString' && data.ref === null) data.ref = t.text
+      if (t.type === 'labelText' && data.label === null) data.label = t.text
+      if (t.children && t.children.length) rec(t.children)
+    }
+  }
+  rec([linkToken])
+  return data
+}
+
+function normLabel(s) {
+  return s.trim().replace(/\s+/g, ' ').toLowerCase()
+}
+
+// Every link / image in the file as { dest, line, endLine, startColumn,
+// endColumn }: inline destinations plus reference / collapsed / shortcut ones
+// resolved through the file's link definitions.
+function collectLinks(tokens) {
+  const definitions = new Map()
+  const links = []
+  const walk = (toks) => {
+    for (const t of toks) {
+      if (t.type === 'definition') {
+        let label = null
+        let dest = null
+        const rec = (xs) => {
+          for (const x of xs) {
+            if (x.type === 'definitionLabelString' && label === null) label = x.text
+            if (x.type === 'definitionDestinationString' && dest === null) dest = x.text
+            if (x.children && x.children.length) rec(x.children)
+          }
+        }
+        rec(t.children || [])
+        if (label !== null && dest !== null) definitions.set(normLabel(label), dest)
+      }
+      if (t.type === 'link' || t.type === 'image') {
+        links.push({ token: t, data: ownLinkData(t) })
+      }
+      if (t.children && t.children.length) walk(t.children)
+    }
+  }
+  walk(tokens)
+  const out = []
+  for (const { token, data } of links) {
+    let dest = data.dest
+    if (dest === null) {
+      const key = data.ref && data.ref !== '' ? data.ref : data.label
+      if (key === null) continue
+      dest = definitions.get(normLabel(key))
+      if (dest === undefined || dest === null) continue
+    }
+    out.push({
+      dest,
+      line: token.startLine,
+      endLine: token.endLine,
+      startColumn: token.startColumn,
+      endColumn: token.endColumn,
+    })
+  }
+  return out
+}
+
+// Build the rule bound to `repoRoot` (absolute) and the parsed inventory. F =
+// linkable files, L = tracked symlinks (exist, not dereferenced), G = gitlink
+// roots (targets under them are skipped). dirs holds every ancestor prefix of an
+// F/L entry, so a directory link resolves in O(1).
+function makeRule({ repoRoot, F, L, G }) {
+  // realpath both the root and each linted file: a symlinked temp root (macOS
+  // /tmp -> /private/tmp) would otherwise make path.relative diverge, since a
+  // spawned node resolves process.cwd() through the symlink.
+  const absRoot = fs.realpathSync(path.resolve(repoRoot))
+  const dirs = new Set()
+  for (const p of [...F, ...L]) {
+    const parts = p.split('/')
+    for (let i = 1; i < parts.length; i++) dirs.add(parts.slice(0, i).join('/'))
+  }
+  const anchorCache = new Map()
+  const anchorsOf = (rel) => {
+    if (!anchorCache.has(rel)) {
+      anchorCache.set(rel, computeAnchors(fs.readFileSync(path.resolve(absRoot, rel), 'utf8')))
+    }
+    return anchorCache.get(rel)
+  }
+  const underGitlink = (rel) => G.some((g) => rel === g || rel.startsWith(g + '/'))
+
+  return {
+    names: ['MD-LINK', 'link-integrity'],
+    description: 'Broken relative link target',
+    tags: ['links'],
+    parser: 'micromark',
+    function: function rule(params, onError) {
+      const abs = fs.realpathSync(path.resolve(process.cwd(), params.name))
+      const relDir = path.relative(absRoot, abs).split(path.sep).join('/').replace(/[^/]*$/, '')
+      for (const link of collectLinks(params.parsers.micromark.tokens)) {
+        const split = splitDest(link.dest)
+        if (split === null) continue
+        const rel = path.posix.normalize(path.posix.join(relDir, split.pathPart)).replace(/\/$/, '')
+        const report = (detail) => {
+          const single = link.line === link.endLine
+          onError({
+            lineNumber: link.line,
+            detail,
+            range: single ? [link.startColumn, link.endColumn - link.startColumn] : undefined,
+          })
+        }
+        if (rel === '..' || rel.startsWith('../')) {
+          report('link target escapes the repository root: ' + split.pathPart)
+          continue
+        }
+        if (underGitlink(rel)) continue
+        if (F.has(rel)) {
+          if (split.fragment && rel.endsWith('.md') && !anchorsOf(rel).has(split.fragment)) {
+            report('link fragment not found in target: ' + rel + '#' + split.fragment)
+          }
+          continue
+        }
+        if (L.has(rel)) continue
+        if (dirs.has(rel)) continue
+        report('link target does not exist: ' + rel)
+      }
+    },
+  }
+}
+
+module.exports = { makeRule, computeAnchors, slug, headingInlineText, percentDecode, splitDest }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tackbox",
-  "version": "0.1.69",
+  "version": "0.1.71",
   "description": "ESLint and Markdown lint plugins plus direct error-reporting helpers for JavaScript and TypeScript.",
   "license": "MIT",
   "main": "./js/eslint-plugin.js",

package/js/markdownlint-rules/no-non-ascii.js

@@ -1,132 +0,0 @@
-// Custom markdownlint rule: flag any character outside the printable
-// ASCII range (codepoints > 0x7F). Keeps docs strictly ASCII so
-// em-dashes, curly quotes, Cyrillic, box-drawing chars, emoji, etc.
-// cannot leak into prose, tables, or code fences.
-//
-// One escape hatch: a language marker in an HTML comment within the first
-// 5 lines widens the alphabet for that one file to a declared language's
-// script (plus a little typographic punctuation). It never disables the
-// rule - every other non-ASCII character (emoji, zero-width, other
-// scripts) is still flagged, and a misplaced / duplicate / malformed
-// marker is a finding that leaves the file strict-ASCII.
-//
-//   <!-- tackbox lang ru personal experimental repo -->
-//
-// The marker is read from micromark HTML-comment tokens, not params.lines:
-// markdownlint masks HTML-comment interiors in `lines`, so the raw code is
-// only visible in the parse tree.
-
-// code -> extra codepoints allowed when a valid marker declares it. Add a
-// language by adding one entry: its script range(s) plus the typographic
-// punctuation its prose uses.
-const LANG_SCRIPTS = {
-  ru: {
-    // Cyrillic (U+0400-U+04FF).
-    ranges: [[0x0400, 0x04ff]],
-    // Typographic punctuation common in Russian prose: em/en dash,
-    // guillemets, ellipsis, curly single/double quotes (incl. low
-    // opening quotes), NBSP.
-    punct: [
-      0x2014, 0x2013, 0x00ab, 0x00bb, 0x2026,
-      0x2018, 0x2019, 0x201c, 0x201d, 0x201e, 0x201a, 0x00a0,
-    ],
-  },
-}
-
-const MARKER_MAX_LINE = 5
-const MARKER_RE = /<!--\s*tackbox:\s*lang=([^\s>]*)[^>]*-->/g
-
-function collectMarkers(token, found) {
-  for (const m of token.text.matchAll(MARKER_RE)) {
-    const before = token.text.slice(0, m.index)
-    const lineOffset = (before.match(/\n/g) || []).length
-    const lastNl = before.lastIndexOf('\n')
-    found.push({
-      lineNumber: token.startLine + lineOffset,
-      code: m[1],
-      col: lineOffset === 0 ? token.startColumn + m.index : m.index - lastNl,
-      len: m[0].length,
-    })
-  }
-}
-
-// Every marker occurrence in the file's HTML comments: {lineNumber, code,
-// col, len}. Walks the micromark tree; htmlFlow / htmlText carry the raw
-// comment text (their children just re-slice it, so we do not descend).
-function findMarkers(tokens) {
-  const found = []
-  const walk = (toks) => {
-    for (const t of toks) {
-      if (t.type === 'htmlFlow' || t.type === 'htmlText') {
-        collectMarkers(t, found)
-        continue
-      }
-      if (t.children && t.children.length) walk(t.children)
-    }
-  }
-  walk(tokens)
-  return found
-}
-
-// Validate the marker set; emit findings for a misplaced / duplicate /
-// malformed / unknown marker. Return the allow-config for a single valid
-// marker, or null (strict ASCII-only) for no marker or any invalid one.
-function resolveMarkers(markers, onError) {
-  if (markers.length === 0) return null
-  const markerErr = (m, detail) =>
-    onError({ lineNumber: m.lineNumber, detail, range: [m.col, m.len] })
-
-  if (markers.length > 1) {
-    for (const dup of markers.slice(1)) {
-      markerErr(dup, 'duplicate tackbox lang marker (one marker per file)')
-    }
-    return null
-  }
-  const m = markers[0]
-  if (m.lineNumber > MARKER_MAX_LINE) {
-    markerErr(m, `tackbox lang marker must be within the first ${MARKER_MAX_LINE} lines`)
-    return null
-  }
-  if (m.code === '') {
-    markerErr(m, 'tackbox lang marker is missing a language code')
-    return null
-  }
-  const cfg = LANG_SCRIPTS[m.code]
-  if (!cfg) {
-    markerErr(m, `tackbox lang marker: unsupported language code '${m.code}'`)
-    return null
-  }
-  return cfg
-}
-
-function isWidened(code, allow) {
-  if (!allow) return false
-  for (const [lo, hi] of allow.ranges) {
-    if (code >= lo && code <= hi) return true
-  }
-  return allow.punct.includes(code)
-}
-
-module.exports = {
-  names: ['MD-ASCII', 'no-non-ascii'],
-  description: 'Non-ASCII character',
-  tags: ['ascii'],
-  parser: 'micromark',
-  function: function rule(params, onError) {
-    const allow = resolveMarkers(findMarkers(params.parsers.micromark.tokens), onError)
-    params.lines.forEach((line, idx) => {
-      let col = 0
-      for (const ch of line) {
-        const code = ch.codePointAt(0)
-        if (code > 0x7f && !isWidened(code, allow)) {
-          onError({
-            lineNumber: idx + 1,
-            detail: 'Non-ASCII character U+' + code.toString(16).toUpperCase() + ' (' + ch + ')',
-            range: [col + 1, ch.length],
-          })
-        }
-        col += ch.length
-      }
-    })
-  },
-}