@catheadowl/md-tools 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # @catheadowl/md-tools
2
2
 
3
- **Keep Markdown references true while a repository moves.** `doc-rename` moves files and directories and rebases every in-repo reference as one atomic plan; `doc-link` checks that every internal link and heading anchor resolves; `doc-normalize` migrates `/`-prefixed internal links to document-relative form. The same engine ships in this package as an importable library.
3
+ **Keep Markdown references true while a repository moves.** `doc-rename` moves files and directories and rebases every in-repo reference as one atomic plan; `doc-link` checks that every internal link and heading anchor resolves; `doc-normalize` migrates `/`-prefixed internal links to document-relative form. The same engine ships in this package as an importable library, together with a document-structure measurement entry.
4
4
 
5
5
  ## Install
6
6
 
@@ -25,7 +25,7 @@ doc-rename --sink <dir> <sub>
25
25
  # Check every internal link and anchor resolves (read-only audit)
26
26
  doc-link
27
27
 
28
- # Machine-readable violations (file:line, url, reason)
28
+ # Machine-readable results (file, line, class, URL, reason, remedy); classes are target-missing, anchor-missing, or outside-repository
29
29
  doc-link --json
30
30
 
31
31
  # One-time migration of /-prefixed internal links to document-relative form
@@ -44,7 +44,7 @@ doc-normalize [--fix] [--json]
44
44
 
45
45
  ## Declared freeze policy
46
46
 
47
- A directory whose `manifest.yml` declares `archived: true` is frozen history, picked up automatically: `doc-link` exempts frozen files as *sources* (their out-links are still checked) and `doc-rename` treats them as read-only. `--frozen <dirName>` replaces the declared policy for one run; `--no-manifest-freeze` skips it. Flag semantics: `doc-link --help`; the pinned read and its fail-loud rules: `skills/md-tools/references/freeze.md`.
47
+ A directory whose `manifest.yml` declares `archived: true` is frozen history, picked up automatically: `doc-link` does not report out-links from frozen files, while still checking active files that link to frozen targets; `doc-rename` treats frozen files as read-only. `--frozen <dirName>` replaces the declared policy for one run; `--no-manifest-freeze` skips it. Flag semantics: `doc-link --help`; the pinned read and its fail-loud rules: `skills/md-tools/references/freeze.md`.
48
48
 
49
49
  ## Agent skill
50
50
 
@@ -64,7 +64,25 @@ for (const v of checkRepository(root)) {
64
64
  }
65
65
  ```
66
66
 
67
- The engine classifies, extracts, resolves, anchor-checks and plans byte-preserving rewrites of links, images, definitions, `[[wikilinks]]` and registered scheme addresses over a git tree — the file universe is `git ls-files`; extraction and rewrite planning are pure string functions. It never decides when to check or what to exempt: freeze policy, deny lists and wiki disambiguation flow in as parameters. The typed surface is the API reference — `lib/index.d.ts`, with the data plane at `@catheadowl/md-tools/links/index.js` and the report face at `@catheadowl/md-tools/gate-check.js`.
67
+ The engine classifies, extracts, resolves, anchor-checks and plans byte-preserving rewrites of links, images, definitions, `[[wikilinks]]` and registered scheme addresses over a git tree — the file universe is `git ls-files`; extraction and rewrite planning are pure string functions. It never decides when to check or what to exempt: freeze policy, deny lists and wiki disambiguation flow in as parameters. The typed surface is the API reference — `lib/index.d.ts`, with the data plane at `@catheadowl/md-tools/links/index.js`, the report face at `@catheadowl/md-tools/gate-check.js`, and the measurement entry at `@catheadowl/md-tools/measure/index.js`.
68
+
69
+ ### Measuring document structure
70
+
71
+ A separate narrow entry measures Markdown text — physical lines, sections, structure facts — without any triage policy. Budgets, thresholds and routing stay with the caller.
72
+
73
+ ```js
74
+ import { createMarkdownDocument, measureMarkdownLines, partitionMarkdownSections } from '@catheadowl/md-tools/measure/index.js'
75
+
76
+ const document = createMarkdownDocument(source) // one parse, shared by every consumer
77
+ const { sections } = partitionMarkdownSections(document, { levels: [1, 2], scope: 'all' })
78
+ const nonEmptyBodyLines = sections.map(section => measureMarkdownLines(document, {
79
+ range: section.body,
80
+ blankness: 'js-trim',
81
+ exclude: ['tables', 'fencedCode', 'indentedCode', 'definitions'],
82
+ }).count)
83
+ ```
84
+
85
+ One document is one parse: headings, code fences (including unclosed and container-nested ones), tables and definitions are facts over the same tree, positions are original UTF-16 offsets, and line ranges are document-owned. Blankness (`none` / `space-tab` / `js-trim`) and exclusions are always explicit — each count names its own caliber instead of one ambiguous `sectionLines`. `visitMarkdownDocument` walks the shared tree for anything the facts do not carry.
68
86
 
69
87
  Ports that do not import the library can prove agreement against the shipped conformance corpus at `@catheadowl/md-tools/conformance/cases.json` — pure data, one case per entry, tagged by capability.
70
88
 
package/cli/doc-link.mjs CHANGED
@@ -80,7 +80,8 @@ if (isMain) {
80
80
  console.log(
81
81
  'usage: doc-link [--frozen <dirName>]... [--no-manifest-freeze] [--json]\n' +
82
82
  ' freeze policy: folder-manifest archived: true declarations, discovered automatically;\n' +
83
- ' --frozen replaces the whole policy for this run, --no-manifest-freeze skips manifest reading')
83
+ ' --frozen replaces the whole policy for this run; a source matches when an ancestor directory name matches\n' +
84
+ ' (the source file name does not count); --no-manifest-freeze skips manifest reading')
84
85
  process.exit(0)
85
86
  } else {
86
87
  console.error(`unexpected argument: ${a}`)
@@ -121,6 +122,8 @@ if (isMain) {
121
122
  process.exit(2)
122
123
  }
123
124
  const violations = outcome
125
+ const outsideRepositoryCount = violations.filter(v => v.class === 'outside-repository').length
126
+ const violationCount = violations.length - outsideRepositoryCount
124
127
 
125
128
  if (json) {
126
129
  console.log(JSON.stringify({ checked: 'repository', violations }, null, 2))
@@ -128,9 +131,12 @@ if (isMain) {
128
131
  if (violations.length === 0) {
129
132
  console.log('clean — every internal Markdown reference resolves')
130
133
  } else {
131
- console.log(`${violations.length} violation(s):`)
134
+ const summary = []
135
+ if (violationCount > 0) summary.push(`${violationCount} violation(s)`)
136
+ if (outsideRepositoryCount > 0) summary.push(`${outsideRepositoryCount} outside-repository target(s)`)
137
+ console.log(`${summary.join(', ')}:`)
132
138
  for (const v of violations) {
133
- console.log(` ${v.file}:${v.line} ${v.reason}`)
139
+ console.log(` [${v.class}] ${v.file}:${v.line} ${v.reason}`)
134
140
  console.log(` remedy: ${v.remedy.guidance}`)
135
141
  }
136
142
  }
@@ -12,6 +12,8 @@ export interface ChangeSet {
12
12
  opaque?: boolean;
13
13
  }
14
14
  export interface ReportViolation {
15
+ /** Stable report category; additive to the existing report shape. */
16
+ class: ReportViolationClass;
15
17
  file: string;
16
18
  line: number;
17
19
  reason: string;
@@ -20,6 +22,8 @@ export interface ReportViolation {
20
22
  guidance: string;
21
23
  };
22
24
  }
25
+ /** Stable categories exposed by the doc-link report face. */
26
+ export type ReportViolationClass = 'target-missing' | 'anchor-missing' | 'outside-repository';
23
27
  /**
24
28
  * Structured "this check does not apply here" outcome: the workspace sits
25
29
  * outside any git repository (or git itself is unavailable), so there is no
package/lib/gate-check.js CHANGED
@@ -27,12 +27,18 @@
27
27
  */
28
28
  import { readFileSync } from 'node:fs';
29
29
  import { join } from 'node:path';
30
- import { REASON_ANCHOR_MISSING, canonicalPath, checkRepository, discoverRepositoryRoot, documentAnchorPairs, pathInside, repositoryRoot } from './links/index.js';
31
- const MANUAL_REMEDY = {
30
+ import { REASON_ANCHOR_MISSING, REASON_OUTSIDE_REPOSITORY, REASON_TARGET_MISSING, canonicalPath, checkRepository, discoverRepositoryRoot, documentAnchorPairs, pathInside, repositoryRoot } from './links/index.js';
31
+ const TARGET_MISSING_REMEDY = {
32
32
  kind: 'manual',
33
- guidance: 'Repair the reference so its target file and #fragment resolve (document-relative path; '
34
- + '//, /, and scheme targets are skipped; absolute local paths such as D:/… are reported '
35
- + 'as outside the repository), or create/move the target it points to.',
33
+ guidance: 'Update the reference to an existing target, or add/move the intended target into the repository.',
34
+ };
35
+ const ANCHOR_MISSING_REMEDY = {
36
+ kind: 'manual',
37
+ guidance: 'Update the #fragment to match a heading or explicit anchor in the target document.',
38
+ };
39
+ const OUTSIDE_REPOSITORY_REMEDY = {
40
+ kind: 'manual',
41
+ guidance: 'Confirm this outside-repository reference is intentional; doc-link cannot verify its target as part of this repository.',
36
42
  };
37
43
  /** Maximum hint lines appended to an anchor-missing remedy guidance. */
38
44
  const MAX_ANCHOR_HINTS = 3;
@@ -105,10 +111,25 @@ function buildAttributionPredicate(root, changes) {
105
111
  }
106
112
  function toReportViolation(violation) {
107
113
  const reason = `unresolved Markdown reference ${JSON.stringify(violation.url)} (${violation.reason})`;
108
- const guidance = violation.reason === REASON_ANCHOR_MISSING
109
- ? MANUAL_REMEDY.guidance + anchorHints(fragmentOf(violation.url), violation.targetAbs)
110
- : MANUAL_REMEDY.guidance;
114
+ let violationClass;
115
+ let guidance;
116
+ if (violation.reason === REASON_ANCHOR_MISSING) {
117
+ violationClass = 'anchor-missing';
118
+ guidance = ANCHOR_MISSING_REMEDY.guidance + anchorHints(fragmentOf(violation.url), violation.targetAbs);
119
+ }
120
+ else if (violation.reason === REASON_TARGET_MISSING) {
121
+ violationClass = 'target-missing';
122
+ guidance = TARGET_MISSING_REMEDY.guidance;
123
+ }
124
+ else if (violation.reason === REASON_OUTSIDE_REPOSITORY) {
125
+ violationClass = 'outside-repository';
126
+ guidance = OUTSIDE_REPOSITORY_REMEDY.guidance;
127
+ }
128
+ else {
129
+ throw new Error(`doc-link report: unclassified violation reason ${JSON.stringify(violation.reason)}`);
130
+ }
111
131
  return {
132
+ class: violationClass,
112
133
  file: violation.file,
113
134
  line: violation.line,
114
135
  reason,
@@ -12,8 +12,21 @@ export declare function captureCliStdout(command: string, args: readonly string[
12
12
  * errors that never produced a subprocess (binary missing, sandbox refusal)
13
13
  * still throw — they carry no exit status to return — and callers that treat
14
14
  * an absent binary as a degrade shape catch that throw at their boundary.
15
+ *
16
+ * `options.win32Shell` launches through the platform shell on win32. CLIs
17
+ * that arrive as script shims — npm bin entries are `name`/`name.cmd`/
18
+ * `name.ps1` wrappers, never .exe — cannot be launched by a direct spawn:
19
+ * CreateProcess has no executable to run and spawn ENOENTs even though the
20
+ * CLI is installed and healthy. Opt in ONLY for fixed literal args: under
21
+ * the shell the args array is joined into one command line that cmd
22
+ * re-parses, so variable args (paths, revs — the git seam's shape) must stay
23
+ * on the default direct spawn. Inert off win32. Side effect of the opt-in:
24
+ * a missing CLI answers as a non-zero exit (cmd's not-found code) instead of
25
+ * a spawn throw — for probes both land in the same degrade classification.
15
26
  */
16
- export declare function captureCliExit(command: string, args: readonly string[], tempName: string): {
27
+ export declare function captureCliExit(command: string, args: readonly string[], tempName: string, options?: {
28
+ win32Shell?: boolean;
29
+ }): {
17
30
  status: number | null;
18
31
  stdout: string;
19
32
  stderr: string;
package/lib/links/cli.js CHANGED
@@ -41,8 +41,19 @@ export function captureCliStdout(command, args, tempName) {
41
41
  * errors that never produced a subprocess (binary missing, sandbox refusal)
42
42
  * still throw — they carry no exit status to return — and callers that treat
43
43
  * an absent binary as a degrade shape catch that throw at their boundary.
44
+ *
45
+ * `options.win32Shell` launches through the platform shell on win32. CLIs
46
+ * that arrive as script shims — npm bin entries are `name`/`name.cmd`/
47
+ * `name.ps1` wrappers, never .exe — cannot be launched by a direct spawn:
48
+ * CreateProcess has no executable to run and spawn ENOENTs even though the
49
+ * CLI is installed and healthy. Opt in ONLY for fixed literal args: under
50
+ * the shell the args array is joined into one command line that cmd
51
+ * re-parses, so variable args (paths, revs — the git seam's shape) must stay
52
+ * on the default direct spawn. Inert off win32. Side effect of the opt-in:
53
+ * a missing CLI answers as a non-zero exit (cmd's not-found code) instead of
54
+ * a spawn throw — for probes both land in the same degrade classification.
44
55
  */
45
- export function captureCliExit(command, args, tempName) {
56
+ export function captureCliExit(command, args, tempName, options = {}) {
46
57
  const dir = mkdtempSync(join(tmpdir(), 'doc-core-cli-'));
47
58
  const outFile = join(dir, tempName);
48
59
  const errFile = join(dir, `${tempName}-err`);
@@ -51,7 +62,10 @@ export function captureCliExit(command, args, tempName) {
51
62
  const errFd = openSync(errFile, 'w');
52
63
  try {
53
64
  try {
54
- execFileSync(command, args, { stdio: ['ignore', outFd, errFd] });
65
+ execFileSync(command, args, {
66
+ stdio: ['ignore', outFd, errFd],
67
+ ...(options.win32Shell && process.platform === 'win32' ? { shell: true } : {}),
68
+ });
55
69
  }
56
70
  catch (error) {
57
71
  const status = error.status;
@@ -20,7 +20,7 @@ export { githubSlug, documentAnchors, documentAnchorPairs, anchorCache } from '.
20
20
  export type { DocumentAnchorPair } from './anchors.js';
21
21
  export { gitTopLevel, discoverRepositoryRoot, gitLinkPaths, gitLsFiles, gitTrackedUnder, gitGrep, gitMove, GitMoveError, RepositoryRootError, captureGitStdout, gitStatusPorcelain, gitTreeHas, gitLogRenameRecords, gitLogLastTouched, } from './git.js';
22
22
  export type { GitLsFiles, GitStatusRecord, GitRenameRecord, RepositoryRootDiscovery, RepositoryUnavailableCode } from './git.js';
23
- export { REASON_ANCHOR_MISSING, pathInside, posixRelative, canonicalPath, collectAllFiles, collectMarkdownSources, repositoryRoot, extractReferences, resolveReference, resolveReferenceLexically, enumerateWikiCandidates, targetProbeCache, checkRepository, } from './resolve.js';
23
+ export { REASON_ANCHOR_MISSING, REASON_TARGET_MISSING, REASON_OUTSIDE_REPOSITORY, pathInside, posixRelative, canonicalPath, collectAllFiles, collectMarkdownSources, repositoryRoot, extractReferences, resolveReference, resolveReferenceLexically, enumerateWikiCandidates, targetProbeCache, checkRepository, } from './resolve.js';
24
24
  export type { CheckRepositoryOptions, LinkReference, Resolution, LinkViolation, ReferenceKind, TargetProbe, WikiCandidateOptions } from './resolve.js';
25
25
  export { recognizeSchemeReference, registeredSchemes, resolveSchemeReference, parseUkpListJson, fetchUkpRegistry, resolveUkpWith, } from './scheme.js';
26
26
  export type { SchemeReference, SchemeResolution, SchemeResolver, SchemeOptions, UkpEndpoint, UkpRegistry } from './scheme.js';
@@ -17,7 +17,7 @@ export { parseMarkdown, visitMarkdown, markdownDestination, markdownLabel, scanW
17
17
  export { captureCliExit } from './cli.js';
18
18
  export { githubSlug, documentAnchors, documentAnchorPairs, anchorCache } from './anchors.js';
19
19
  export { gitTopLevel, discoverRepositoryRoot, gitLinkPaths, gitLsFiles, gitTrackedUnder, gitGrep, gitMove, GitMoveError, RepositoryRootError, captureGitStdout, gitStatusPorcelain, gitTreeHas, gitLogRenameRecords, gitLogLastTouched, } from './git.js';
20
- export { REASON_ANCHOR_MISSING, pathInside, posixRelative, canonicalPath, collectAllFiles, collectMarkdownSources, repositoryRoot, extractReferences, resolveReference, resolveReferenceLexically, enumerateWikiCandidates, targetProbeCache, checkRepository, } from './resolve.js';
20
+ export { REASON_ANCHOR_MISSING, REASON_TARGET_MISSING, REASON_OUTSIDE_REPOSITORY, pathInside, posixRelative, canonicalPath, collectAllFiles, collectMarkdownSources, repositoryRoot, extractReferences, resolveReference, resolveReferenceLexically, enumerateWikiCandidates, targetProbeCache, checkRepository, } from './resolve.js';
21
21
  export { recognizeSchemeReference, registeredSchemes, resolveSchemeReference, parseUkpListJson, fetchUkpRegistry, resolveUkpWith, } from './scheme.js';
22
22
  export { rebaseDestination, rebaseLabel } from './rebase.js';
23
23
  export { rebaseHref, planRename, planSink, applyRenamePlan, RenameApplyError, relabelFor, REASON_NO_RENAME_EVIDENCE, RENAME_CONFLICT_CODES, RENAME_CONFLICT_MESSAGES, POSTHOC_WITNESS_SHAPES, RENAME_HISTORY_EVIDENCE_SHAPES, } from './rename.js';
@@ -77,6 +77,8 @@ export interface MarkdownLabel {
77
77
  * the node's children and the reference kind).
78
78
  */
79
79
  export declare function markdownLabel(source: string, node: MarkdownDestinationNode): MarkdownLabel;
80
+ /** Text a reader sees from one Markdown node; raw HTML itself contributes none. */
81
+ export declare function renderedText(node: Nodes): string;
80
82
  /** Return every parsed Markdown heading with its rendered text and source line. */
81
83
  export declare function markdownHeadingLines(source: string): MarkdownHeadingLine[];
82
84
  /** One authored `[[wiki-link]]`, located byte-exactly in the source. */
@@ -160,7 +160,7 @@ export function markdownLabel(source, node) {
160
160
  return { ...absolute, text: source.slice(absolute.start, absolute.end) };
161
161
  }
162
162
  /** Text a reader sees from one Markdown node; raw HTML itself contributes none. */
163
- function renderedText(node) {
163
+ export function renderedText(node) {
164
164
  if (node.type === 'text' || node.type === 'inlineCode')
165
165
  return node.value;
166
166
  if (node.type === 'image' || node.type === 'imageReference')
@@ -15,6 +15,8 @@ import { type GitLsFiles } from './git.js';
15
15
  import { type MarkdownLabel } from './markdown.js';
16
16
  /** Stable reason strings the doc-link gate branches its remedy on — reword here, never in the gate. */
17
17
  export declare const REASON_ANCHOR_MISSING = "anchor does not exist";
18
+ export declare const REASON_TARGET_MISSING = "target does not exist";
19
+ export declare const REASON_OUTSIDE_REPOSITORY = "outside repository";
18
20
  export type ReferenceKind = 'link' | 'image' | 'definition';
19
21
  /** One authored link/image/definition, located byte-exactly in the source. */
20
22
  export interface LinkReference {
@@ -18,6 +18,8 @@ import { gitLinkPaths, gitLsFiles, gitTopLevel } from './git.js';
18
18
  import { markdownDestination, markdownLabel, parseMarkdown, visitMarkdown, isWindowsDriveUrl } from './markdown.js';
19
19
  /** Stable reason strings the doc-link gate branches its remedy on — reword here, never in the gate. */
20
20
  export const REASON_ANCHOR_MISSING = 'anchor does not exist';
21
+ export const REASON_TARGET_MISSING = 'target does not exist';
22
+ export const REASON_OUTSIDE_REPOSITORY = 'outside repository';
21
23
  /** Whether `target` is at or below `root` (lexical containment). */
22
24
  export function pathInside(root, target) {
23
25
  const rel = relative(root, target);
@@ -204,10 +206,10 @@ export function resolveReference(reference, sourceFile, root, probe) {
204
206
  if (!exists) {
205
207
  if (temporarilyUnverifiable(root, resolved, probe))
206
208
  return { ignored: true };
207
- return { reason: 'target does not exist', fragment };
209
+ return { reason: REASON_TARGET_MISSING, fragment };
208
210
  }
209
211
  if (!existingPathInside(root, resolved, probe))
210
- return { reason: 'outside repository', fragment };
212
+ return { reason: REASON_OUTSIDE_REPOSITORY, fragment };
211
213
  return { abs: resolved, fragment };
212
214
  }
213
215
  /**
@@ -95,10 +95,13 @@ export declare function parseUkpListJson(stdout: string): UkpRegistry;
95
95
  * question, since a document may legitimately point at an endpoint outside
96
96
  * the current workspace scope, and resolution must not inherit that scope.
97
97
  * Builds predating the flag answer non-zero and land in the degrade path,
98
- * same as no `--json`. Every failure mode — no ukp on PATH, a failed spawn,
99
- * non-zero exit, unparseable output — is registry-unavailable, which
100
- * downgrades resolution to a warning: a missing machine contract must not
101
- * fail a repo.
98
+ * same as no `--json`. win32 launches through the shell (`win32Shell` — npm
99
+ * bin shims are script wrappers, not executables; args here are fixed
100
+ * literals, the opt-in's contract), so a missing ukp there answers as a
101
+ * non-zero exit rather than a failed spawn — same classification. Every
102
+ * failure mode — no ukp on PATH, a failed spawn, non-zero exit, unparseable
103
+ * output — is registry-unavailable, which downgrades resolution to a
104
+ * warning: a missing machine contract must not fail a repo.
102
105
  */
103
106
  export declare function fetchUkpRegistry(): UkpRegistry;
104
107
  /**
@@ -146,10 +146,13 @@ let ukpRegistrySnapshot;
146
146
  * question, since a document may legitimately point at an endpoint outside
147
147
  * the current workspace scope, and resolution must not inherit that scope.
148
148
  * Builds predating the flag answer non-zero and land in the degrade path,
149
- * same as no `--json`. Every failure mode — no ukp on PATH, a failed spawn,
150
- * non-zero exit, unparseable output — is registry-unavailable, which
151
- * downgrades resolution to a warning: a missing machine contract must not
152
- * fail a repo.
149
+ * same as no `--json`. win32 launches through the shell (`win32Shell` — npm
150
+ * bin shims are script wrappers, not executables; args here are fixed
151
+ * literals, the opt-in's contract), so a missing ukp there answers as a
152
+ * non-zero exit rather than a failed spawn — same classification. Every
153
+ * failure mode — no ukp on PATH, a failed spawn, non-zero exit, unparseable
154
+ * output — is registry-unavailable, which downgrades resolution to a
155
+ * warning: a missing machine contract must not fail a repo.
153
156
  */
154
157
  export function fetchUkpRegistry() {
155
158
  if (ukpRegistrySnapshot === undefined) {
@@ -160,7 +163,7 @@ export function fetchUkpRegistry() {
160
163
  function probeUkpRegistry() {
161
164
  let captured;
162
165
  try {
163
- captured = captureCliExit('ukp', ['list', '--json', '-g'], 'ukp-list');
166
+ captured = captureCliExit('ukp', ['list', '--json', '-g'], 'ukp-list', { win32Shell: true });
164
167
  }
165
168
  catch (error) {
166
169
  return { status: 'unavailable', reason: `ukp CLI could not be launched (${error instanceof Error ? error.message : String(error)})` };
@@ -0,0 +1,47 @@
1
+ /**
2
+ * Physical-line counting over one measurement document. Range, blankness
3
+ * and exclusions are all explicit — there is no budget and no file-type
4
+ * default. Blankness is judged on the original physical rows with their
5
+ * container prefixes still in place, so a row holding only `>` counts as
6
+ * non-empty here even when its decoded content is not; consumers choose
7
+ * which axis they are counting. Exclusions remove the whole physical rows
8
+ * a structure occupies — a row shared by a list marker and a fence
9
+ * opener loses both. The result returns the count together with the
10
+ * ordered maximal included runs that sum to it, so every number can be
11
+ * explained row by row.
12
+ */
13
+ import type { MarkdownDocument, MarkdownLineRange } from './document.js';
14
+ /**
15
+ * Whole-row exclusion families; overlapping families union, and a row
16
+ * shared by a list marker and a fence opener is removed once, for both.
17
+ * `fencedCode` removes delimiter rows together with payload rows;
18
+ * `fenceDelimiters` removes only the actual opening and closing rows of
19
+ * fenced code, leaving payload rows counted; `prefix` removes the rows
20
+ * the caller-claimed prefix covers. All families apply at every AST
21
+ * depth, inside containers included.
22
+ */
23
+ export type MarkdownLineExclusion = 'prefix' | 'headings' | 'tables' | 'fencedCode' | 'fenceDelimiters' | 'indentedCode' | 'definitions';
24
+ export interface MarkdownLineSelection {
25
+ readonly range: MarkdownLineRange;
26
+ /**
27
+ * Judged on the original physical rows, container prefixes still in
28
+ * place. `none` counts every physical row. `space-tab` treats a row as
29
+ * blank when its content is empty or only U+0020/U+0009. `js-trim`
30
+ * treats a row as blank when ECMAScript's trim would empty it — the
31
+ * wider whitespace set, NBSP included.
32
+ */
33
+ readonly blankness: 'none' | 'space-tab' | 'js-trim';
34
+ readonly exclude: readonly MarkdownLineExclusion[];
35
+ }
36
+ export interface MarkdownLineMeasurement {
37
+ readonly count: number;
38
+ /** Ordered maximal runs of included physical rows; explain count without duplicate rows. */
39
+ readonly included: readonly MarkdownLineRange[];
40
+ }
41
+ /**
42
+ * Count the physical rows of one document-owned range under explicit
43
+ * blankness and exclusion rules. An unknown blankness or exclusion value
44
+ * throws TypeError; a foreign or malformed range throws TypeError via
45
+ * the document's ownership check.
46
+ */
47
+ export declare function measureMarkdownLines(document: MarkdownDocument, selection: MarkdownLineSelection): MarkdownLineMeasurement;
@@ -0,0 +1,91 @@
1
+ /**
2
+ * Physical-line counting over one measurement document. Range, blankness
3
+ * and exclusions are all explicit — there is no budget and no file-type
4
+ * default. Blankness is judged on the original physical rows with their
5
+ * container prefixes still in place, so a row holding only `>` counts as
6
+ * non-empty here even when its decoded content is not; consumers choose
7
+ * which axis they are counting. Exclusions remove the whole physical rows
8
+ * a structure occupies — a row shared by a list marker and a fence
9
+ * opener loses both. The result returns the count together with the
10
+ * ordered maximal included runs that sum to it, so every number can be
11
+ * explained row by row.
12
+ */
13
+ import { requireOwnedRange } from './document.js';
14
+ const EXCLUSIONS = [
15
+ 'prefix', 'headings', 'tables', 'fencedCode', 'fenceDelimiters', 'indentedCode', 'definitions',
16
+ ];
17
+ const BLANKNESS = ['none', 'space-tab', 'js-trim'];
18
+ /** Every row range one exclusion family touches, at any AST depth. */
19
+ function excludedRows(document, exclusion) {
20
+ const { structure } = document;
21
+ switch (exclusion) {
22
+ case 'prefix': return [document.prefix];
23
+ case 'headings': return structure.headings.map(fact => fact.rows);
24
+ case 'tables': return structure.tables.map(fact => fact.rows);
25
+ case 'fencedCode': return structure.code.filter(fact => fact.kind === 'fenced').map(fact => fact.rows);
26
+ case 'indentedCode': return structure.code.filter(fact => fact.kind === 'indented').map(fact => fact.rows);
27
+ case 'definitions': return structure.definitions.map(fact => fact.rows);
28
+ case 'fenceDelimiters': {
29
+ const delimiters = [];
30
+ for (const fact of structure.code) {
31
+ if (fact.kind !== 'fenced')
32
+ continue;
33
+ if (fact.opening !== null)
34
+ delimiters.push(fact.opening);
35
+ if (fact.closing !== null)
36
+ delimiters.push(fact.closing);
37
+ }
38
+ return delimiters;
39
+ }
40
+ }
41
+ }
42
+ /**
43
+ * Count the physical rows of one document-owned range under explicit
44
+ * blankness and exclusion rules. An unknown blankness or exclusion value
45
+ * throws TypeError; a foreign or malformed range throws TypeError via
46
+ * the document's ownership check.
47
+ */
48
+ export function measureMarkdownLines(document, selection) {
49
+ const span = requireOwnedRange(document, selection.range);
50
+ if (!BLANKNESS.includes(selection.blankness)) {
51
+ throw new TypeError(`measure: blankness must be one of ${JSON.stringify(BLANKNESS)}, got ${JSON.stringify(selection.blankness)}`);
52
+ }
53
+ for (const exclusion of selection.exclude) {
54
+ if (!EXCLUSIONS.includes(exclusion)) {
55
+ throw new TypeError(`measure: unknown line exclusion ${JSON.stringify(exclusion)}`);
56
+ }
57
+ }
58
+ const excluded = new Set();
59
+ for (const exclusion of selection.exclude) {
60
+ for (const rows of excludedRows(document, exclusion)) {
61
+ for (let line = Math.max(rows.startLine, span.startLine); line < Math.min(rows.endLineExclusive, span.endLineExclusive); line++) {
62
+ excluded.add(line);
63
+ }
64
+ }
65
+ }
66
+ const included = [];
67
+ let count = 0;
68
+ let runStart = null;
69
+ const closeRun = (endLineExclusive) => {
70
+ if (runStart !== null) {
71
+ included.push(document.range({ startLine: runStart, endLineExclusive }));
72
+ runStart = null;
73
+ }
74
+ };
75
+ for (let line = span.startLine; line < span.endLineExclusive; line++) {
76
+ const physical = document.lines[line - 1];
77
+ const content = physical === undefined ? '' : document.source.slice(physical.startOffset, physical.contentEndOffset);
78
+ const blank = selection.blankness === 'space-tab' ? /^[ \t]*$/.test(content)
79
+ : selection.blankness === 'js-trim' ? content.trim() === ''
80
+ : false;
81
+ if (excluded.has(line) || blank) {
82
+ closeRun(line);
83
+ continue;
84
+ }
85
+ count += 1;
86
+ if (runStart === null)
87
+ runStart = line;
88
+ }
89
+ closeRun(span.endLineExclusive);
90
+ return Object.freeze({ count, included: Object.freeze(included) });
91
+ }
@@ -0,0 +1,105 @@
1
+ /**
2
+ * Measurement document — the single parse every measurement consumer
3
+ * shares. `createMarkdownDocument` keeps the original source string,
4
+ * scans its physical lines (LF, CRLF and CR each end one line; CRLF is a
5
+ * single terminator; a trailing terminator never creates an extra line),
6
+ * claims a caller-recognized leading prefix, parses the remaining body
7
+ * once and restores every tree position to original-source coordinates:
8
+ * UTF-16 offsets from 0, 1-based line/column, exclusive ends.
9
+ *
10
+ * Row ranges are document-owned: `range` validates the boundaries and
11
+ * brands the result, and a range handed to another document is a
12
+ * TypeError, never a silently accepted look-alike span.
13
+ */
14
+ import type { Nodes, Root } from 'mdast';
15
+ import type { MarkdownStructure } from './structure.js';
16
+ /** Recursively readonly view of a shared, frozen tree. */
17
+ export type DeepReadonly<T> = T extends (...args: never[]) => unknown ? T : T extends readonly unknown[] ? {
18
+ readonly [K in keyof T]: DeepReadonly<T[K]>;
19
+ } : T extends object ? {
20
+ readonly [K in keyof T]: DeepReadonly<T[K]>;
21
+ } : T;
22
+ export interface MarkdownPoint {
23
+ readonly line: number;
24
+ readonly column: number;
25
+ /** Original JavaScript string index in UTF-16 code units. */
26
+ readonly offset: number;
27
+ }
28
+ export interface MarkdownPosition {
29
+ readonly start: MarkdownPoint;
30
+ /** Exclusive endpoint; may be the parser's point after the last physical line. */
31
+ readonly end: MarkdownPoint;
32
+ }
33
+ export interface MarkdownLineSpan {
34
+ /** One-based inclusive physical line boundary. */
35
+ readonly startLine: number;
36
+ /** One-based exclusive boundary, at most physicalLineCount + 1. */
37
+ readonly endLineExclusive: number;
38
+ }
39
+ declare const rangeOwner: unique symbol;
40
+ /** Created only through the owning document; foreign-document use is an error. */
41
+ export interface MarkdownLineRange extends MarkdownLineSpan {
42
+ readonly [rangeOwner]: unknown;
43
+ }
44
+ export interface MarkdownPhysicalLine {
45
+ /** One-based line number, matching the tree coordinate system. */
46
+ readonly line: number;
47
+ readonly startOffset: number;
48
+ /** Excludes the line terminator. */
49
+ readonly contentEndOffset: number;
50
+ /** Includes the line terminator, when present. */
51
+ readonly endOffset: number;
52
+ readonly ending: '' | '\n' | '\r\n' | '\r';
53
+ }
54
+ export interface MarkdownDocumentOptions {
55
+ /**
56
+ * Caller-recognized leading prefix [0, offset) in UTF-16 code units:
57
+ * frontmatter the caller recognized, never parsed here. Must end after
58
+ * a complete line terminator or at actual end of input — a mid-line or
59
+ * CRLF-splitting offset is a RangeError, as is a non-integer, negative
60
+ * or out-of-source offset. `source` itself stays the full original
61
+ * string, BOM included; a leading U+FEFF rides on the source axis and
62
+ * shifts offsets and first-line columns of the restored tree, while the
63
+ * parser consumes it as a file marker with no syntax effect.
64
+ */
65
+ readonly prefixEndOffset?: number;
66
+ }
67
+ export interface MarkdownDocument {
68
+ readonly source: string;
69
+ /** Every physical line, 1-based in the `line` field, in source order. */
70
+ readonly lines: readonly MarkdownPhysicalLine[];
71
+ /** All physical rows, prefix rows included. */
72
+ readonly whole: MarkdownLineRange;
73
+ /** The rows after the prefix; an empty prefix leaves it equal to `whole`. */
74
+ readonly body: MarkdownLineRange;
75
+ /** The rows the claimed prefix covers; empty when no prefix is claimed. */
76
+ readonly prefix: MarkdownLineRange;
77
+ readonly structure: MarkdownStructure;
78
+ /** Same tree on every access, with original-source coordinates. */
79
+ readonly tree: DeepReadonly<Root>;
80
+ /**
81
+ * Validates integral boundaries and creates a document-owned row range.
82
+ * A non-integer, reversed or out-of-bounds span throws RangeError; a
83
+ * range created by another document throws TypeError.
84
+ */
85
+ range(span: MarkdownLineSpan): MarkdownLineRange;
86
+ }
87
+ /**
88
+ * Resolve one document-owned row range to its span. A range from another
89
+ * document — or a plain object shaped like one — is a TypeError: row math
90
+ * silently applied to the wrong document is how same-named metrics drift.
91
+ */
92
+ export declare function requireOwnedRange(document: MarkdownDocument, range: MarkdownLineRange): MarkdownLineSpan;
93
+ /**
94
+ * Create the shared measurement document for one Markdown source. The
95
+ * body is parsed exactly once; structure facts, section partition and
96
+ * line counting all read that same tree. A non-string source throws
97
+ * TypeError; the empty string is a valid zero-line document. The empty
98
+ * body's tree root sits at the real end-of-input point, and line ranges
99
+ * of an empty body are [L+1, L+1).
100
+ */
101
+ export declare function createMarkdownDocument(source: string, options?: MarkdownDocumentOptions): MarkdownDocument;
102
+ export type MarkdownVisitor = (node: DeepReadonly<Nodes>, ancestors: readonly DeepReadonly<Nodes>[]) => false | void;
103
+ /** Pre-order visit over the document's shared tree; returning false prunes a node's children. Never parses again. */
104
+ export declare function visitMarkdownDocument(document: MarkdownDocument, visitor: MarkdownVisitor): void;
105
+ export {};
@@ -0,0 +1,173 @@
1
+ /**
2
+ * Measurement document — the single parse every measurement consumer
3
+ * shares. `createMarkdownDocument` keeps the original source string,
4
+ * scans its physical lines (LF, CRLF and CR each end one line; CRLF is a
5
+ * single terminator; a trailing terminator never creates an extra line),
6
+ * claims a caller-recognized leading prefix, parses the remaining body
7
+ * once and restores every tree position to original-source coordinates:
8
+ * UTF-16 offsets from 0, 1-based line/column, exclusive ends.
9
+ *
10
+ * Row ranges are document-owned: `range` validates the boundaries and
11
+ * brands the result, and a range handed to another document is a
12
+ * TypeError, never a silently accepted look-alike span.
13
+ */
14
+ import { parseMarkdown } from '../links/markdown.js';
15
+ import { buildStructure } from './structure.js';
16
+ const rangeOwner = Symbol('markdownLineRangeOwner');
17
+ /** Owner identity for the ranges each document hands out. */
18
+ const documentOwners = new WeakMap();
19
+ /**
20
+ * Resolve one document-owned row range to its span. A range from another
21
+ * document — or a plain object shaped like one — is a TypeError: row math
22
+ * silently applied to the wrong document is how same-named metrics drift.
23
+ */
24
+ export function requireOwnedRange(document, range) {
25
+ const owner = documentOwners.get(document);
26
+ if (owner === undefined || typeof range !== 'object' || range === null
27
+ || range[rangeOwner] !== owner) {
28
+ throw new TypeError('measure: line range was not created by this document');
29
+ }
30
+ return { startLine: range.startLine, endLineExclusive: range.endLineExclusive };
31
+ }
32
+ /** Split the source into physical lines without normalizing any terminator. */
33
+ function scanLines(source) {
34
+ const lines = [];
35
+ let start = 0;
36
+ while (start < source.length) {
37
+ let contentEnd = start;
38
+ while (contentEnd < source.length && source[contentEnd] !== '\n' && source[contentEnd] !== '\r')
39
+ contentEnd += 1;
40
+ if (contentEnd === source.length) {
41
+ lines.push({ line: lines.length + 1, startOffset: start, contentEndOffset: contentEnd, endOffset: contentEnd, ending: '' });
42
+ break;
43
+ }
44
+ const ending = source[contentEnd] === '\r'
45
+ ? (source[contentEnd + 1] === '\n' ? '\r\n' : '\r')
46
+ : '\n';
47
+ const endOffset = contentEnd + ending.length;
48
+ lines.push({ line: lines.length + 1, startOffset: start, contentEndOffset: contentEnd, endOffset, ending });
49
+ start = endOffset;
50
+ }
51
+ return lines;
52
+ }
53
+ /**
54
+ * The zero-width point at actual end of input: still on the last physical
55
+ * line when it has no terminator, on the next line's column 1 when it does.
56
+ */
57
+ function eofPoint(source, lines) {
58
+ const last = lines[lines.length - 1];
59
+ if (last === undefined)
60
+ return { line: 1, column: 1, offset: 0 };
61
+ if (last.ending === '') {
62
+ return { line: last.line, column: last.contentEndOffset - last.startOffset + 1, offset: last.contentEndOffset };
63
+ }
64
+ return { line: last.line + 1, column: 1, offset: source.length };
65
+ }
66
+ /**
67
+ * Shift every position in the body-local tree back onto the original
68
+ * source. Offsets and lines always shift; a leading U+FEFF BOM shifts
69
+ * only the first line's columns, because the parser consumes it as a
70
+ * file marker while it still occupies one column of the original line.
71
+ */
72
+ function restorePositions(root, offsetShift, lineShift, firstColumnShift) {
73
+ const restorePoint = (point) => {
74
+ if (point.line === 1)
75
+ point.column += firstColumnShift;
76
+ point.line += lineShift;
77
+ point.offset = (point.offset ?? 0) + offsetShift;
78
+ };
79
+ const restore = (node) => {
80
+ if (node.position !== undefined) {
81
+ restorePoint(node.position.start);
82
+ restorePoint(node.position.end);
83
+ }
84
+ if ('children' in node)
85
+ for (const child of node.children)
86
+ restore(child);
87
+ };
88
+ restore(root);
89
+ }
90
+ function deepFreeze(value) {
91
+ if (value !== null && typeof value === 'object' && !Object.isFrozen(value)) {
92
+ Object.freeze(value);
93
+ for (const key of Object.getOwnPropertyNames(value)) {
94
+ deepFreeze(value[key]);
95
+ }
96
+ }
97
+ return value;
98
+ }
99
+ /**
100
+ * Create the shared measurement document for one Markdown source. The
101
+ * body is parsed exactly once; structure facts, section partition and
102
+ * line counting all read that same tree. A non-string source throws
103
+ * TypeError; the empty string is a valid zero-line document. The empty
104
+ * body's tree root sits at the real end-of-input point, and line ranges
105
+ * of an empty body are [L+1, L+1).
106
+ */
107
+ export function createMarkdownDocument(source, options = {}) {
108
+ if (typeof source !== 'string')
109
+ throw new TypeError('measure: source must be a string');
110
+ const prefixEndOffset = options.prefixEndOffset ?? 0;
111
+ if (!Number.isInteger(prefixEndOffset) || prefixEndOffset < 0 || prefixEndOffset > source.length) {
112
+ throw new RangeError(`measure: prefixEndOffset ${prefixEndOffset} is outside the source`);
113
+ }
114
+ if (prefixEndOffset !== 0 && prefixEndOffset !== source.length) {
115
+ const previous = source[prefixEndOffset - 1];
116
+ const afterTerminator = previous === '\n' || (previous === '\r' && source[prefixEndOffset] !== '\n');
117
+ if (!afterTerminator) {
118
+ throw new RangeError('measure: prefixEndOffset must end after a complete line terminator or at actual end of input');
119
+ }
120
+ }
121
+ const lines = scanLines(source);
122
+ let prefixLines = 0;
123
+ while (prefixLines < lines.length && lines[prefixLines].endOffset <= prefixEndOffset)
124
+ prefixLines += 1;
125
+ const bodyText = source.slice(prefixEndOffset);
126
+ const bomShift = bodyText.charCodeAt(0) === 0xfeff ? 1 : 0;
127
+ const tree = parseMarkdown(bodyText);
128
+ if (bodyText === '') {
129
+ const point = eofPoint(source, lines);
130
+ tree.position = { start: point, end: point };
131
+ }
132
+ else if (prefixEndOffset + bomShift > 0) {
133
+ restorePositions(tree, prefixEndOffset + bomShift, prefixLines, bomShift);
134
+ }
135
+ const owner = Symbol('owner');
136
+ const range = (span) => {
137
+ const { startLine, endLineExclusive } = span ?? {};
138
+ if (!Number.isInteger(startLine) || !Number.isInteger(endLineExclusive)
139
+ || startLine < 1 || startLine > endLineExclusive || endLineExclusive > lines.length + 1) {
140
+ throw new RangeError(`measure: invalid line span [${startLine}, ${endLineExclusive}) for a document with ${lines.length} physical lines`);
141
+ }
142
+ return Object.freeze({ startLine, endLineExclusive, [rangeOwner]: owner });
143
+ };
144
+ const whole = range({ startLine: 1, endLineExclusive: lines.length + 1 });
145
+ const prefix = range({ startLine: 1, endLineExclusive: prefixLines + 1 });
146
+ const body = range({ startLine: prefixLines + 1, endLineExclusive: lines.length + 1 });
147
+ const structure = buildStructure(tree, source, lines, range);
148
+ const document = {
149
+ source,
150
+ lines: deepFreeze(lines),
151
+ whole,
152
+ prefix,
153
+ body,
154
+ structure,
155
+ tree: deepFreeze(tree),
156
+ range,
157
+ };
158
+ Object.freeze(document);
159
+ documentOwners.set(document, owner);
160
+ return document;
161
+ }
162
+ /** Pre-order visit over the document's shared tree; returning false prunes a node's children. Never parses again. */
163
+ export function visitMarkdownDocument(document, visitor) {
164
+ const walk = (node, ancestors) => {
165
+ if (visitor(node, ancestors) === false)
166
+ return;
167
+ if ('children' in node) {
168
+ for (const child of node.children)
169
+ walk(child, [...ancestors, node]);
170
+ }
171
+ };
172
+ walk(document.tree, []);
173
+ }
@@ -0,0 +1,22 @@
1
+ /**
2
+ * @catheadowl/md-tools measurement face — one shared parse per document,
3
+ * structure facts, section partition and explicit physical-line counting
4
+ * over Markdown text. Deliberately free of triage policy: budgets,
5
+ * genres, thresholds and routing stay with consumers. Import as
6
+ * `@catheadowl/md-tools/measure/index.js`.
7
+ *
8
+ * Sibling modules are this entry's implementation carriers; their shipped
9
+ * `.d.ts` files exist for type resolution, and only what is re-exported
10
+ * here is the supported surface: `document.js` owns the one-parse
11
+ * document and its row ranges, `structure.js` projects headings, code,
12
+ * tables and definitions off that tree, `sections.js` partitions it, and
13
+ * `count.js` counts rows.
14
+ * @module @catheadowl/md-tools/measure
15
+ */
16
+ export { createMarkdownDocument, visitMarkdownDocument } from './document.js';
17
+ export type { DeepReadonly, MarkdownDocument, MarkdownDocumentOptions, MarkdownLineRange, MarkdownLineSpan, MarkdownPhysicalLine, MarkdownPoint, MarkdownPosition, MarkdownVisitor, } from './document.js';
18
+ export type { MarkdownBlock, MarkdownCode, MarkdownContainer, MarkdownHeading, MarkdownStructure } from './structure.js';
19
+ export { partitionMarkdownSections } from './sections.js';
20
+ export type { MarkdownPartition, MarkdownSection, MarkdownSectionSelection } from './sections.js';
21
+ export { measureMarkdownLines } from './count.js';
22
+ export type { MarkdownLineExclusion, MarkdownLineMeasurement, MarkdownLineSelection } from './count.js';
@@ -0,0 +1,18 @@
1
+ /**
2
+ * @catheadowl/md-tools measurement face — one shared parse per document,
3
+ * structure facts, section partition and explicit physical-line counting
4
+ * over Markdown text. Deliberately free of triage policy: budgets,
5
+ * genres, thresholds and routing stay with consumers. Import as
6
+ * `@catheadowl/md-tools/measure/index.js`.
7
+ *
8
+ * Sibling modules are this entry's implementation carriers; their shipped
9
+ * `.d.ts` files exist for type resolution, and only what is re-exported
10
+ * here is the supported surface: `document.js` owns the one-parse
11
+ * document and its row ranges, `structure.js` projects headings, code,
12
+ * tables and definitions off that tree, `sections.js` partitions it, and
13
+ * `count.js` counts rows.
14
+ * @module @catheadowl/md-tools/measure
15
+ */
16
+ export { createMarkdownDocument, visitMarkdownDocument } from './document.js';
17
+ export { partitionMarkdownSections } from './sections.js';
18
+ export { measureMarkdownLines } from './count.js';
@@ -0,0 +1,38 @@
1
+ /**
2
+ * Section partition over one measurement document's structure. Heading
3
+ * levels and scope are always explicit: every selected heading opens a
4
+ * section — even with an empty body — unselected deeper headings stay
5
+ * body content, no section accumulates another, and the preamble spans
6
+ * the parsed body up to the first selected heading. Nothing about file
7
+ * names or genres decides the selection.
8
+ */
9
+ import type { MarkdownDocument, MarkdownLineRange } from './document.js';
10
+ import type { MarkdownHeading } from './structure.js';
11
+ export interface MarkdownSectionSelection {
12
+ readonly levels: readonly (1 | 2 | 3 | 4 | 5 | 6)[];
13
+ /**
14
+ * `all` selects every matching heading at any container depth, in
15
+ * global source order; `root` selects only headings directly under the
16
+ * root. Under `all`, a container heading's section body still runs to
17
+ * the next selected heading's first row — container end does not end
18
+ * the section, so a root heading after a quote closes the quoted
19
+ * section at its own row.
20
+ */
21
+ readonly scope: 'all' | 'root';
22
+ }
23
+ export interface MarkdownSection {
24
+ readonly heading: MarkdownHeading;
25
+ /** Excludes every source row occupied by the opening heading; empty body stays a section. */
26
+ readonly body: MarkdownLineRange;
27
+ }
28
+ export interface MarkdownPartition {
29
+ readonly preamble: MarkdownLineRange;
30
+ readonly sections: readonly MarkdownSection[];
31
+ }
32
+ /**
33
+ * Partition the document's body into explicit-level sections, in source
34
+ * order. With no selected heading the sections array is empty and the
35
+ * preamble covers the whole body. Duplicate levels collapse; a level
36
+ * outside 1..6 throws RangeError, a non-empty levels array is required.
37
+ */
38
+ export declare function partitionMarkdownSections(document: MarkdownDocument, selection: MarkdownSectionSelection): MarkdownPartition;
@@ -0,0 +1,37 @@
1
+ /**
2
+ * Section partition over one measurement document's structure. Heading
3
+ * levels and scope are always explicit: every selected heading opens a
4
+ * section — even with an empty body — unselected deeper headings stay
5
+ * body content, no section accumulates another, and the preamble spans
6
+ * the parsed body up to the first selected heading. Nothing about file
7
+ * names or genres decides the selection.
8
+ */
9
+ /**
10
+ * Partition the document's body into explicit-level sections, in source
11
+ * order. With no selected heading the sections array is empty and the
12
+ * preamble covers the whole body. Duplicate levels collapse; a level
13
+ * outside 1..6 throws RangeError, a non-empty levels array is required.
14
+ */
15
+ export function partitionMarkdownSections(document, selection) {
16
+ const levels = [...new Set(selection.levels)];
17
+ if (levels.length === 0) {
18
+ throw new RangeError('measure: partition levels must be a non-empty subset of 1..6');
19
+ }
20
+ if (levels.some(depth => !Number.isInteger(depth) || depth < 1 || depth > 6)) {
21
+ throw new RangeError(`measure: partition levels must stay within 1..6, got ${JSON.stringify(selection.levels)}`);
22
+ }
23
+ if (selection.scope !== 'all' && selection.scope !== 'root') {
24
+ throw new TypeError(`measure: partition scope must be 'all' or 'root', got ${JSON.stringify(selection.scope)}`);
25
+ }
26
+ const wanted = new Set(levels);
27
+ const selected = document.structure.headings.filter(heading => wanted.has(heading.depth) && (selection.scope === 'all' || heading.containers.length === 0));
28
+ const sections = selected.map((heading, index) => {
29
+ const endLineExclusive = index + 1 < selected.length ? selected[index + 1].rows.startLine : document.lines.length + 1;
30
+ return Object.freeze({ heading, body: document.range({ startLine: heading.rows.endLineExclusive, endLineExclusive }) });
31
+ });
32
+ const preamble = document.range({
33
+ startLine: document.body.startLine,
34
+ endLineExclusive: selected.length === 0 ? document.body.endLineExclusive : selected[0].rows.startLine,
35
+ });
36
+ return Object.freeze({ preamble, sections: Object.freeze(sections) });
37
+ }
@@ -0,0 +1,53 @@
1
+ /**
2
+ * Structure facts over one measurement document's shared tree: every real
3
+ * heading, code block, GFM table and link definition, each with its
4
+ * original position, its occupied physical rows and its container
5
+ * ancestors. Ancestors never mask descendants and never replace the AST
6
+ * classification — a table inside a blockquote is a table fact whose
7
+ * containers name the blockquote, not a blockquote-only row.
8
+ *
9
+ * Fenced code additionally gets a source projection the parser does not
10
+ * hand out directly: opening/closing delimiter rows and raw payload rows.
11
+ * A closing row exists only where a legal closer terminates the fence —
12
+ * shorter, wrong-character or junk-carrying rows stay payload, and a
13
+ * fence left open by its container ends with the container instead of
14
+ * swallowing the following root-level content.
15
+ */
16
+ import type { Nodes } from 'mdast';
17
+ import type { MarkdownLineRange, MarkdownLineSpan, MarkdownPhysicalLine, MarkdownPosition } from './document.js';
18
+ export interface MarkdownContainer {
19
+ readonly type: 'blockquote' | 'list' | 'listItem' | 'footnoteDefinition';
20
+ readonly position: MarkdownPosition;
21
+ }
22
+ export interface MarkdownBlock {
23
+ readonly position: MarkdownPosition;
24
+ readonly rows: MarkdownLineRange;
25
+ readonly containers: readonly MarkdownContainer[];
26
+ }
27
+ export interface MarkdownHeading extends MarkdownBlock {
28
+ readonly depth: 1 | 2 | 3 | 4 | 5 | 6;
29
+ readonly style: 'atx' | 'setext';
30
+ readonly text: string;
31
+ }
32
+ export interface MarkdownCode extends MarkdownBlock {
33
+ readonly kind: 'fenced' | 'indented';
34
+ /** Null for indented code. Closing delimiter is null for an unclosed fence. */
35
+ readonly opening: MarkdownLineRange | null;
36
+ readonly closing: MarkdownLineRange | null;
37
+ /** Raw rows within this AST code node, excluding delimiters; includes container prefixes. An unclosed container fence may end before document EOF. */
38
+ readonly payloadRows: MarkdownLineRange;
39
+ /** Parser-decoded code value, without container prefixes; not a source slice. */
40
+ readonly value: string;
41
+ readonly lang: string | null;
42
+ readonly meta: string | null;
43
+ }
44
+ export interface MarkdownStructure {
45
+ readonly headings: readonly MarkdownHeading[];
46
+ readonly code: readonly MarkdownCode[];
47
+ readonly tables: readonly MarkdownBlock[];
48
+ readonly definitions: readonly MarkdownBlock[];
49
+ }
50
+ type RangeFactory = (span: MarkdownLineSpan) => MarkdownLineRange;
51
+ /** Collect the structure facts of one shared tree, in source order, without masking ancestors. */
52
+ export declare function buildStructure(tree: Nodes, source: string, lines: readonly MarkdownPhysicalLine[], rangeFor: RangeFactory): MarkdownStructure;
53
+ export {};
@@ -0,0 +1,186 @@
1
+ /**
2
+ * Structure facts over one measurement document's shared tree: every real
3
+ * heading, code block, GFM table and link definition, each with its
4
+ * original position, its occupied physical rows and its container
5
+ * ancestors. Ancestors never mask descendants and never replace the AST
6
+ * classification — a table inside a blockquote is a table fact whose
7
+ * containers name the blockquote, not a blockquote-only row.
8
+ *
9
+ * Fenced code additionally gets a source projection the parser does not
10
+ * hand out directly: opening/closing delimiter rows and raw payload rows.
11
+ * A closing row exists only where a legal closer terminates the fence —
12
+ * shorter, wrong-character or junk-carrying rows stay payload, and a
13
+ * fence left open by its container ends with the container instead of
14
+ * swallowing the following root-level content.
15
+ */
16
+ import { renderedText } from '../links/markdown.js';
17
+ function isContainer(node) {
18
+ return node.type === 'blockquote' || node.type === 'list' || node.type === 'listItem' || node.type === 'footnoteDefinition';
19
+ }
20
+ function pointFact(point) {
21
+ return { line: point.line, column: point.column, offset: point.offset ?? 0 };
22
+ }
23
+ function positionFact(position) {
24
+ return { start: pointFact(position.start), end: pointFact(position.end) };
25
+ }
26
+ function containerFacts(ancestors) {
27
+ return Object.freeze(ancestors
28
+ .filter(node => isContainer(node) && node.position !== undefined)
29
+ .map(node => Object.freeze({ type: node.type, position: positionFact(node.position) })));
30
+ }
31
+ /**
32
+ * The physical rows a node actually occupies. An exclusive end sitting at
33
+ * column 1 of the next line does not claim that line, and no node claims
34
+ * rows past the document's real physical end.
35
+ */
36
+ function rowsFor(position, lines, rangeFor) {
37
+ const endLineExclusive = position.end.column === 1 ? position.end.line : position.end.line + 1;
38
+ return rangeFor({ startLine: position.start.line, endLineExclusive: Math.min(endLineExclusive, lines.length + 1) });
39
+ }
40
+ function lineText(source, lines, line) {
41
+ const physical = lines[line - 1];
42
+ return physical === undefined ? '' : source.slice(physical.startOffset, physical.contentEndOffset);
43
+ }
44
+ /** Expand tabs to four-column tab stops, the way block structure reads indentation. */
45
+ function expandTabs(text) {
46
+ if (!text.includes('\t'))
47
+ return text;
48
+ let expanded = '';
49
+ for (const char of text) {
50
+ if (char === '\t')
51
+ expanded += ' '.repeat(4 - (expanded.length % 4));
52
+ else
53
+ expanded += char;
54
+ }
55
+ return expanded;
56
+ }
57
+ /**
58
+ * Content column of a list item's first row: marker plus following spaces
59
+ * when content shares the row, one past the bare marker when it does not.
60
+ * Columns follow tab stops, so the marker is measured on the expanded row.
61
+ */
62
+ function listItemContentIndent(item, source, lines) {
63
+ if (item.position === undefined)
64
+ return 1;
65
+ const physical = lines[item.position.start.line - 1];
66
+ if (physical === undefined)
67
+ return 1;
68
+ const rowText = source.slice(physical.startOffset, physical.contentEndOffset);
69
+ const lead = expandTabs(rowText.slice(0, (item.position.start.offset ?? 0) - physical.startOffset));
70
+ const fromMarker = expandTabs(rowText).slice(lead.length);
71
+ const marker = /^[ ]{0,3}(?:[-+*]|[0-9]{1,9}[.)])([ ]+|$)/.exec(fromMarker);
72
+ if (marker === null)
73
+ return 1;
74
+ if (marker[1] === undefined)
75
+ return marker[0].length + 1;
76
+ if (marker[0].length === fromMarker.length)
77
+ return marker[0].replace(/[ ]+$/, '').length + 1;
78
+ return marker[0].length;
79
+ }
80
+ /**
81
+ * Whether one row is a legal closing fence for an opening run of
82
+ * `fenceLength` `fenceChar` characters: the same container continuation
83
+ * prefixes (blockquote markers, list-item content indent), then up to
84
+ * three spaces, the run, and nothing but spaces or tabs. Indentation is
85
+ * judged with tabs expanded to tab stops — a tab-indented continuation
86
+ * row is as legal as a space-indented one.
87
+ */
88
+ function isClosingFenceRow(rowText, ancestors, source, lines, fenceChar, fenceLength) {
89
+ let rest = expandTabs(rowText);
90
+ for (const ancestor of ancestors) {
91
+ if (ancestor.type === 'blockquote') {
92
+ const prefix = /^[ ]{0,3}>[ ]?/.exec(rest);
93
+ if (prefix === null)
94
+ return false;
95
+ rest = rest.slice(prefix[0].length);
96
+ }
97
+ else if (ancestor.type === 'listItem') {
98
+ const indent = ' '.repeat(listItemContentIndent(ancestor, source, lines));
99
+ if (!rest.startsWith(indent))
100
+ return false;
101
+ rest = rest.slice(indent.length);
102
+ }
103
+ }
104
+ return new RegExp(`^[ ]{0,3}${fenceChar === '`' ? '`' : '~'}{${fenceLength},}[ \t]*$`).test(rest);
105
+ }
106
+ function codeFact(node, position, source, lines, rangeFor, ancestors, rows, containers) {
107
+ const startOffset = position.start.offset ?? 0;
108
+ const fenceChar = source[startOffset];
109
+ const fenced = fenceChar === '`' || fenceChar === '~';
110
+ let opening = null;
111
+ let closing = null;
112
+ if (fenced) {
113
+ // The node starts at the fence character itself; an indented code node
114
+ // starts at its first content column instead.
115
+ opening = rangeFor({ startLine: rows.startLine, endLineExclusive: rows.startLine + 1 });
116
+ let fenceLength = 0;
117
+ while (source[startOffset + fenceLength] === fenceChar)
118
+ fenceLength += 1;
119
+ const lastRow = rows.endLineExclusive - 1;
120
+ // A closed fence always ends on its closing row, so only that row can
121
+ // carry the closer; an opener-only node never closes onto itself.
122
+ if (lastRow > rows.startLine
123
+ && isClosingFenceRow(lineText(source, lines, lastRow), ancestors, source, lines, fenceChar, fenceLength)) {
124
+ closing = rangeFor({ startLine: lastRow, endLineExclusive: lastRow + 1 });
125
+ }
126
+ }
127
+ const payloadRows = fenced
128
+ ? rangeFor({ startLine: rows.startLine + 1, endLineExclusive: closing === null ? rows.endLineExclusive : closing.startLine })
129
+ : rows;
130
+ return Object.freeze({
131
+ position: positionFact(position),
132
+ rows,
133
+ containers,
134
+ kind: fenced ? 'fenced' : 'indented',
135
+ opening,
136
+ closing,
137
+ payloadRows,
138
+ value: node.value,
139
+ lang: node.lang ?? null,
140
+ meta: node.meta ?? null,
141
+ });
142
+ }
143
+ /** Collect the structure facts of one shared tree, in source order, without masking ancestors. */
144
+ export function buildStructure(tree, source, lines, rangeFor) {
145
+ const headings = [];
146
+ const code = [];
147
+ const tables = [];
148
+ const definitions = [];
149
+ const walk = (node, ancestors) => {
150
+ if (node.position !== undefined) {
151
+ const containers = containerFacts(ancestors);
152
+ const rows = rowsFor(node.position, lines, rangeFor);
153
+ if (node.type === 'heading') {
154
+ headings.push(Object.freeze({
155
+ position: positionFact(node.position),
156
+ rows,
157
+ containers,
158
+ depth: node.depth,
159
+ style: node.position.end.line > node.position.start.line ? 'setext' : 'atx',
160
+ text: renderedText(node),
161
+ }));
162
+ }
163
+ else if (node.type === 'code') {
164
+ code.push(codeFact(node, node.position, source, lines, rangeFor, ancestors, rows, containers));
165
+ }
166
+ else if (node.type === 'table' || node.type === 'definition') {
167
+ const fact = Object.freeze({ position: positionFact(node.position), rows, containers });
168
+ if (node.type === 'table')
169
+ tables.push(fact);
170
+ else
171
+ definitions.push(fact);
172
+ }
173
+ }
174
+ if ('children' in node) {
175
+ for (const child of node.children)
176
+ walk(child, [...ancestors, node]);
177
+ }
178
+ };
179
+ walk(tree, []);
180
+ return Object.freeze({
181
+ headings: Object.freeze(headings),
182
+ code: Object.freeze(code),
183
+ tables: Object.freeze(tables),
184
+ definitions: Object.freeze(definitions),
185
+ });
186
+ }
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@catheadowl/md-tools",
3
- "version": "0.1.0",
4
- "description": "Markdown reference toolchain for git repositories — doc-rename moves files with every reference rebased, doc-link checks links and anchors, doc-normalize migrates link style; the engine ships as an importable library.",
3
+ "version": "0.2.0",
4
+ "description": "Markdown reference toolchain for git repositories — doc-rename moves files with every reference rebased, doc-link checks links and anchors, doc-normalize migrates link style; the engine ships as an importable library for reference rebase/check and document structure measurement.",
5
5
  "license": "MIT",
6
6
  "type": "module",
7
7
  "packageManager": "pnpm@12.8.1",
@@ -20,6 +20,10 @@
20
20
  "types": "./lib/links/index.d.ts",
21
21
  "default": "./lib/links/index.js"
22
22
  },
23
+ "./measure/index.js": {
24
+ "types": "./lib/measure/index.d.ts",
25
+ "default": "./lib/measure/index.js"
26
+ },
23
27
  "./conformance/cases.json": "./conformance/cases.json"
24
28
  },
25
29
  "bin": {
@@ -49,7 +53,10 @@
49
53
  "anchors",
50
54
  "git",
51
55
  "documentation",
52
- "cli"
56
+ "cli",
57
+ "measure",
58
+ "structure",
59
+ "sections"
53
60
  ],
54
61
  "publishConfig": {
55
62
  "access": "public"
@@ -57,7 +64,7 @@
57
64
  "scripts": {
58
65
  "build": "tsc -p tsconfig.json",
59
66
  "typecheck": "tsc --noEmit -p tsconfig.json",
60
- "test": "npm run build && node --test --test-isolation=none test/anchors.test.mjs test/markdown.test.mjs test/resolve.test.mjs test/rebase.test.mjs test/rename.test.mjs test/rename-posthoc.test.mjs test/rename-manifest.test.mjs test/normalize.test.mjs test/git-cache.test.mjs test/spawn-count.test.mjs test/gate-check.test.mjs test/conformance.test.mjs test/wiki.test.mjs test/scheme.test.mjs test/source-hygiene.test.mjs && node --test test/cli.test.mjs test/skills.test.mjs test/publish-hygiene.test.mjs",
67
+ "test": "npm run build && node --test --test-isolation=none test/anchors.test.mjs test/markdown.test.mjs test/measure.test.mjs test/resolve.test.mjs test/rebase.test.mjs test/rename.test.mjs test/rename-posthoc.test.mjs test/rename-manifest.test.mjs test/normalize.test.mjs test/git-cache.test.mjs test/spawn-count.test.mjs test/gate-check.test.mjs test/conformance.test.mjs test/wiki.test.mjs test/scheme.test.mjs test/source-hygiene.test.mjs && node --test test/cli.test.mjs test/skills.test.mjs test/publish-hygiene.test.mjs",
61
68
  "prepublishOnly": "npm test",
62
69
  "check": "node cli/doc-link.mjs",
63
70
  "setup": "node -e \"if(parseInt(process.versions.node)<18){console.error('md-tools needs node >= 18');process.exit(1)}\" && npm run build && npm link",
@@ -3,7 +3,7 @@ name: md-tools
3
3
  description: Markdown link toolchain for git repos — move files with reference rebasing (doc-rename), link/anchor integrity check (doc-link), /-prefixed link migration (doc-normalize).
4
4
  metadata:
5
5
  author: catheadowl
6
- version: "0.1.0"
6
+ version: "0.2.0"
7
7
  ---
8
8
 
9
9
  # md-tools
@@ -6,7 +6,8 @@ description: md-tools declared freeze policy — manifest.yml archived declarati
6
6
 
7
7
  Any directory whose `manifest.yml` declares `archived: true` is frozen history. Picked up automatically — no flags, no extra file.
8
8
 
9
- - `doc-link` exempts frozen files as *sources*: their out-links are still checked, because frozen content is read-only by policy, so its rotting out-links are noise, not findings.
9
+ - `doc-link` exempts frozen files as sources: it does not report their outbound links, because frozen content is read-only and its rotting links are not actionable findings.
10
+ - Active files remain checked when they link to targets inside a frozen tree; freezing a target does not exempt incoming references.
10
11
  - `doc-rename` treats frozen files as read-only.
11
12
 
12
13
  The read is narrowly pinned: only `manifest.yml`, a YAML mapping, `schema_version: 1`, and a boolean `archived` are recognized — anything else is ignored.