throughline-transform 0.2.0__tar.gz → 0.4.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {throughline_transform-0.2.0/src/throughline_transform.egg-info → throughline_transform-0.4.0}/PKG-INFO +21 -17
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/README.md +19 -14
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/pyproject.toml +2 -2
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/cli.py +4 -5
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/notes.py +54 -3
- throughline_transform-0.4.0/src/throughline_transform/progress.py +47 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/project.py +11 -3
- {throughline_transform-0.2.0 → throughline_transform-0.4.0/src/throughline_transform.egg-info}/PKG-INFO +21 -17
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform.egg-info/SOURCES.txt +1 -0
- throughline_transform-0.4.0/src/throughline_transform.egg-info/requires.txt +4 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/tests/conftest.py +1 -1
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/tests/test_outputs.py +14 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/tests/test_surface.py +48 -0
- throughline_transform-0.2.0/src/throughline_transform.egg-info/requires.txt +0 -5
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/LICENSE +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/MANIFEST.in +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/NOTICE +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/setup.cfg +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/__init__.py +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/blocks.py +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/html.py +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/markdown.py +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/model.py +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/office.py +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/package.py +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/provenance.py +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/shape.py +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/tables.py +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform.egg-info/dependency_links.txt +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform.egg-info/entry_points.txt +0 -0
- {throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform.egg-info/top_level.txt +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: throughline-transform
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.4.0
|
|
4
4
|
Summary: Turn a throughline requirements graph on disk, committed or not, into the forms its readers open: Markdown, a folder of Obsidian notes, CSV, an Excel workbook, a Word document, or a print-ready HTML document. Ships the tl-transform CLI.
|
|
5
5
|
Author-email: Henry J Grech-Cini <henry.grechcini@gmail.com>
|
|
6
6
|
License-Expression: Apache-2.0
|
|
@@ -21,8 +21,7 @@ Requires-Python: >=3.11
|
|
|
21
21
|
Description-Content-Type: text/markdown
|
|
22
22
|
License-File: LICENSE
|
|
23
23
|
License-File: NOTICE
|
|
24
|
-
Requires-Dist: throughline
|
|
25
|
-
Requires-Dist: throughline>=2.2.0
|
|
24
|
+
Requires-Dist: throughline>=3.11.1
|
|
26
25
|
Provides-Extra: dev
|
|
27
26
|
Requires-Dist: pytest>=8.0; extra == "dev"
|
|
28
27
|
Dynamic: license-file
|
|
@@ -33,7 +32,7 @@ A [throughline](https://github.com/rhodium-org/throughline) requirements graph o
|
|
|
33
32
|
disk — committed or not — in the forms its readers open. One command, one file:
|
|
34
33
|
|
|
35
34
|
```bash
|
|
36
|
-
pipx install throughline-transform # pulls tl
|
|
35
|
+
pipx install throughline-transform # pulls tl along too
|
|
37
36
|
|
|
38
37
|
tl-transform md -C idd # Markdown, written by the tool itself
|
|
39
38
|
tl-transform notes -C idd # a folder of notes for Obsidian (zip)
|
|
@@ -64,16 +63,16 @@ worse than none. That is [`NG-0001`](idd/non-goals/NG-0001.yml).
|
|
|
64
63
|
**It ships no PDF writer** ([`NG-0002`](idd/non-goals/NG-0002.yml)). Every
|
|
65
64
|
browser prints to PDF with a renderer that is already on the machine.
|
|
66
65
|
|
|
67
|
-
**It depends on nothing but the family.** `throughline`
|
|
68
|
-
|
|
69
|
-
formats, which are written from their own specifications.
|
|
66
|
+
**It depends on nothing but the family.** `throughline` alone (composition is
|
|
67
|
+
the Tool's own since 3.11.1), and the standard library's `zipfile` for the two
|
|
68
|
+
Office formats, which are written from their own specifications.
|
|
70
69
|
|
|
71
70
|
## Which tool answers
|
|
72
71
|
|
|
73
|
-
|
|
74
|
-
`
|
|
75
|
-
|
|
76
|
-
|
|
72
|
+
Every graph is read through `tl`, which composes a graph that declares
|
|
73
|
+
`[[sources]]` itself, so a borrowed clause is written with the reference number
|
|
74
|
+
a conformance document exists to carry; `tl-compose` is a second name for the
|
|
75
|
+
same program.
|
|
77
76
|
The tool is found beside this package's interpreter first and on the path
|
|
78
77
|
second, and is run as a command — its command line is the surface it publishes.
|
|
79
78
|
|
|
@@ -82,7 +81,7 @@ second, and is run as a command — its command line is the surface it publishes
|
|
|
82
81
|
| Format | What you get |
|
|
83
82
|
|---|---|
|
|
84
83
|
| `md` | The tool's own document: a summary, a catalogue of every item in full, the mirrored clauses of adopted sources. `--body table` or `both` for a one-line-per-item table, `--matrix` for one traceability matrix per grounding link type, `--markers` to keep the region markers so `tl docs` can refill the file later. |
|
|
85
|
-
| `notes` | One Markdown note per item in a folder per register
|
|
84
|
+
| `notes` | One Markdown note per item in a folder per register: YAML front matter as properties (type, status, attributes, the title as an alias, a tag each for type and status), a heading of the identifier and title, the item's words, rationale and links, then what links to it — `Implements this`, `Derives from this` — every identifier a `[[wikilink]]`. Cited clauses under `sources/<namespace>/`, the colon in their name replaced by a space. Open the folder as an Obsidian vault: the graph view is the intent graph, and a note's backlinks are what rests on it. |
|
|
86
85
|
| `csv` | A zip of one file of items (or one per register with `--split per-register`), `links.csv` with one row per link naming the sheet at each end, one file per cited source, and the provenance. |
|
|
87
86
|
| `xlsx` | The same as sheets, read-only by recommendation with the structure locked and no password. Sorting and filtering stay allowed; a spreadsheet nobody can sort is one nobody can review. |
|
|
88
87
|
| `docx` | A title page with the provenance, a contents field Word fills in, a section per register, a heading per item with its type and status beneath, its words as a quotation, its rationale, its attributes in small print, and one appendix per cited source. |
|
|
@@ -96,14 +95,19 @@ knows them without git — the editor holds them from its clone — states them
|
|
|
96
95
|
`--repository`, `--ref`, `--commit` and `--tree clean|dirty|untracked`, and git
|
|
97
96
|
is then not consulted.
|
|
98
97
|
|
|
98
|
+
A quick run is silent. When a step — reading the graph, asking the tool to
|
|
99
|
+
write the document, writing the file — has run for a second, one line on
|
|
100
|
+
stderr says which, because a composed graph fetching its sources for the first
|
|
101
|
+
time otherwise reads as a hang.
|
|
102
|
+
|
|
99
103
|
## One implementation, two surfaces
|
|
100
104
|
|
|
101
105
|
The [throughline editor](https://github.com/rhodium-org/throughline-editor)
|
|
102
|
-
runs this package as a wheel under Pyodide beside `tl
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
106
|
+
runs this package as a wheel under Pyodide beside `tl`, so an export from a
|
|
107
|
+
browser tab and an export from a terminal are the same bytes for the same graph
|
|
108
|
+
at the same commit. Pyodide has no processes, so there the tool calls the
|
|
109
|
+
console entry point `tl` declares in its package metadata — the same function
|
|
110
|
+
the command runs — instead of spawning it.
|
|
107
111
|
|
|
108
112
|
## Self-hosting
|
|
109
113
|
|
|
@@ -4,7 +4,7 @@ A [throughline](https://github.com/rhodium-org/throughline) requirements graph o
|
|
|
4
4
|
disk — committed or not — in the forms its readers open. One command, one file:
|
|
5
5
|
|
|
6
6
|
```bash
|
|
7
|
-
pipx install throughline-transform # pulls tl
|
|
7
|
+
pipx install throughline-transform # pulls tl along too
|
|
8
8
|
|
|
9
9
|
tl-transform md -C idd # Markdown, written by the tool itself
|
|
10
10
|
tl-transform notes -C idd # a folder of notes for Obsidian (zip)
|
|
@@ -35,16 +35,16 @@ worse than none. That is [`NG-0001`](idd/non-goals/NG-0001.yml).
|
|
|
35
35
|
**It ships no PDF writer** ([`NG-0002`](idd/non-goals/NG-0002.yml)). Every
|
|
36
36
|
browser prints to PDF with a renderer that is already on the machine.
|
|
37
37
|
|
|
38
|
-
**It depends on nothing but the family.** `throughline`
|
|
39
|
-
|
|
40
|
-
formats, which are written from their own specifications.
|
|
38
|
+
**It depends on nothing but the family.** `throughline` alone (composition is
|
|
39
|
+
the Tool's own since 3.11.1), and the standard library's `zipfile` for the two
|
|
40
|
+
Office formats, which are written from their own specifications.
|
|
41
41
|
|
|
42
42
|
## Which tool answers
|
|
43
43
|
|
|
44
|
-
|
|
45
|
-
`
|
|
46
|
-
|
|
47
|
-
|
|
44
|
+
Every graph is read through `tl`, which composes a graph that declares
|
|
45
|
+
`[[sources]]` itself, so a borrowed clause is written with the reference number
|
|
46
|
+
a conformance document exists to carry; `tl-compose` is a second name for the
|
|
47
|
+
same program.
|
|
48
48
|
The tool is found beside this package's interpreter first and on the path
|
|
49
49
|
second, and is run as a command — its command line is the surface it publishes.
|
|
50
50
|
|
|
@@ -53,7 +53,7 @@ second, and is run as a command — its command line is the surface it publishes
|
|
|
53
53
|
| Format | What you get |
|
|
54
54
|
|---|---|
|
|
55
55
|
| `md` | The tool's own document: a summary, a catalogue of every item in full, the mirrored clauses of adopted sources. `--body table` or `both` for a one-line-per-item table, `--matrix` for one traceability matrix per grounding link type, `--markers` to keep the region markers so `tl docs` can refill the file later. |
|
|
56
|
-
| `notes` | One Markdown note per item in a folder per register
|
|
56
|
+
| `notes` | One Markdown note per item in a folder per register: YAML front matter as properties (type, status, attributes, the title as an alias, a tag each for type and status), a heading of the identifier and title, the item's words, rationale and links, then what links to it — `Implements this`, `Derives from this` — every identifier a `[[wikilink]]`. Cited clauses under `sources/<namespace>/`, the colon in their name replaced by a space. Open the folder as an Obsidian vault: the graph view is the intent graph, and a note's backlinks are what rests on it. |
|
|
57
57
|
| `csv` | A zip of one file of items (or one per register with `--split per-register`), `links.csv` with one row per link naming the sheet at each end, one file per cited source, and the provenance. |
|
|
58
58
|
| `xlsx` | The same as sheets, read-only by recommendation with the structure locked and no password. Sorting and filtering stay allowed; a spreadsheet nobody can sort is one nobody can review. |
|
|
59
59
|
| `docx` | A title page with the provenance, a contents field Word fills in, a section per register, a heading per item with its type and status beneath, its words as a quotation, its rationale, its attributes in small print, and one appendix per cited source. |
|
|
@@ -67,14 +67,19 @@ knows them without git — the editor holds them from its clone — states them
|
|
|
67
67
|
`--repository`, `--ref`, `--commit` and `--tree clean|dirty|untracked`, and git
|
|
68
68
|
is then not consulted.
|
|
69
69
|
|
|
70
|
+
A quick run is silent. When a step — reading the graph, asking the tool to
|
|
71
|
+
write the document, writing the file — has run for a second, one line on
|
|
72
|
+
stderr says which, because a composed graph fetching its sources for the first
|
|
73
|
+
time otherwise reads as a hang.
|
|
74
|
+
|
|
70
75
|
## One implementation, two surfaces
|
|
71
76
|
|
|
72
77
|
The [throughline editor](https://github.com/rhodium-org/throughline-editor)
|
|
73
|
-
runs this package as a wheel under Pyodide beside `tl
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
+
runs this package as a wheel under Pyodide beside `tl`, so an export from a
|
|
79
|
+
browser tab and an export from a terminal are the same bytes for the same graph
|
|
80
|
+
at the same commit. Pyodide has no processes, so there the tool calls the
|
|
81
|
+
console entry point `tl` declares in its package metadata — the same function
|
|
82
|
+
the command runs — instead of spawning it.
|
|
78
83
|
|
|
79
84
|
## Self-hosting
|
|
80
85
|
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "throughline-transform"
|
|
7
|
-
version = "0.
|
|
7
|
+
version = "0.4.0"
|
|
8
8
|
description = "Turn a throughline requirements graph on disk, committed or not, into the forms its readers open: Markdown, a folder of Obsidian notes, CSV, an Excel workbook, a Word document, or a print-ready HTML document. Ships the tl-transform CLI."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.11"
|
|
@@ -41,7 +41,7 @@ classifiers = [
|
|
|
41
41
|
# 0.16.1 are the shapes this package reads (SR-0003); it was written against those
|
|
42
42
|
# releases and the floors say so. compose 0.16.1 requires throughline 2.2.0, so the
|
|
43
43
|
# two floors are one fact stated at both ends.
|
|
44
|
-
dependencies = ["throughline
|
|
44
|
+
dependencies = ["throughline>=3.11.1"] # composition is the Tool's own since 3.11.1 (throughline UR-0037); tl-compose is a second name for tl
|
|
45
45
|
|
|
46
46
|
[project.optional-dependencies]
|
|
47
47
|
dev = ["pytest>=8.0"]
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/cli.py
RENAMED
|
@@ -32,6 +32,7 @@ from .model import Graph, register_titles, source_titles
|
|
|
32
32
|
from .notes import to_notes
|
|
33
33
|
from .office import to_docx, to_xlsx
|
|
34
34
|
from .package import Entry, archive, unpack, utf8
|
|
35
|
+
from .progress import slow
|
|
35
36
|
from .project import TransformError, find_graph, load
|
|
36
37
|
from .provenance import TREE_WORDS, Provenance, provenance_of, stated
|
|
37
38
|
from .tables import Table, cited_sources, file_name, links_of, per_register, to_csv, whole_graph
|
|
@@ -42,10 +43,7 @@ FOLDERS = ("notes", "csv")
|
|
|
42
43
|
|
|
43
44
|
|
|
44
45
|
def version_string() -> str:
|
|
45
|
-
return (
|
|
46
|
-
f"tl-transform {_v('throughline-transform')} "
|
|
47
|
-
f"(throughline-compose {_v('throughline-compose')}, throughline {_v('throughline')})"
|
|
48
|
-
)
|
|
46
|
+
return f"tl-transform {_v('throughline-transform')} (throughline {_v('throughline')})"
|
|
49
47
|
|
|
50
48
|
|
|
51
49
|
def build_parser() -> argparse.ArgumentParser:
|
|
@@ -160,7 +158,8 @@ def main(argv: list[str] | None = None) -> int:
|
|
|
160
158
|
else provenance_of(root, graph.tool_version)
|
|
161
159
|
)
|
|
162
160
|
result, default = produce(graph, root, args.format, opts, provenance)
|
|
163
|
-
|
|
161
|
+
with slow(f"still writing {args.output or default}…"):
|
|
162
|
+
target = write(result, default, args.output, args.folder)
|
|
164
163
|
except (TransformError, OSError, ValueError) as exc:
|
|
165
164
|
print(f"tl-transform: {exc}", file=sys.stderr)
|
|
166
165
|
return 1
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/notes.py
RENAMED
|
@@ -10,7 +10,10 @@ clause of an adopted source, and an About note carrying the provenance.
|
|
|
10
10
|
The prose is the tool's. The catalogue is written once and cut here at the
|
|
11
11
|
item line, so the words in a note are the words ``tl docs`` publishes. What is
|
|
12
12
|
added is what the tool has no view on: file boundaries, YAML front matter the
|
|
13
|
-
note tool reads as properties, and
|
|
13
|
+
note tool reads as properties, link syntax, and the body reshaped for a note —
|
|
14
|
+
the item line as the note's heading, the attribute line left to the front
|
|
15
|
+
matter, and the incoming links written out so a reader sees what rests on the
|
|
16
|
+
item without opening a pane (SR-0009, reversed on first use in Obsidian).
|
|
14
17
|
"""
|
|
15
18
|
|
|
16
19
|
from __future__ import annotations
|
|
@@ -22,7 +25,7 @@ from dataclasses import dataclass, field
|
|
|
22
25
|
from .model import Graph, Item, prefix_of, split_borrowed
|
|
23
26
|
from .package import Entry, utf8
|
|
24
27
|
from .provenance import Provenance
|
|
25
|
-
from .shape import IDENTIFIER, ITEM_LINE
|
|
28
|
+
from .shape import ATTRS_LINE, IDENTIFIER, ITEM_LINE
|
|
26
29
|
|
|
27
30
|
_HEADING = re.compile(r"^#{1,3}\s+(.*)$")
|
|
28
31
|
|
|
@@ -150,6 +153,54 @@ def wikilink(line: str, own: str, names: dict[str, str]) -> str:
|
|
|
150
153
|
return IDENTIFIER.sub(link, line)
|
|
151
154
|
|
|
152
155
|
|
|
156
|
+
def _link(identifier: str, names: dict[str, str]) -> str:
|
|
157
|
+
name = names[identifier]
|
|
158
|
+
return f"[[{identifier}]]" if name == identifier else f"[[{name}|{identifier}]]"
|
|
159
|
+
|
|
160
|
+
|
|
161
|
+
def _label(link_type: str) -> str:
|
|
162
|
+
"""The tool's own label for a link type — ``derives_from`` -> ``Derives from``."""
|
|
163
|
+
return link_type.replace("_", " ").capitalize()
|
|
164
|
+
|
|
165
|
+
|
|
166
|
+
def note_body(block: NoteBlock, graph: Graph, names: dict[str, str]) -> list[str]:
|
|
167
|
+
"""The tool's block reshaped for a note (SR-0009).
|
|
168
|
+
|
|
169
|
+
The item line becomes a level-one heading with the type and status on the
|
|
170
|
+
line beneath; the words, rationale and outgoing link lines follow as the
|
|
171
|
+
tool wrote them, wikilinked; the attribute line goes, because the front
|
|
172
|
+
matter already carries every attribute; then one line per incoming link
|
|
173
|
+
type names what links to this item, worded as the link type reads with
|
|
174
|
+
the word "this", so a reader of a requirement sees what rests on it.
|
|
175
|
+
"""
|
|
176
|
+
head = ITEM_LINE.match(block.lines[0])
|
|
177
|
+
assert head is not None
|
|
178
|
+
identifier, reference, title, type_, status = head.groups()
|
|
179
|
+
# The heading goes through the same link pass as any line: a title that
|
|
180
|
+
# names another item links to it, and a bracket pair in a title is prose.
|
|
181
|
+
lines = [
|
|
182
|
+
wikilink(f"# {identifier}{f' ({reference})' if reference else ''} — {title}", block.id, names),
|
|
183
|
+
f"{type_.replace('_', ' ')} · {status}",
|
|
184
|
+
]
|
|
185
|
+
for line in block.lines[1:]:
|
|
186
|
+
if ATTRS_LINE.match(line):
|
|
187
|
+
continue
|
|
188
|
+
lines.append(wikilink(line, block.id, names))
|
|
189
|
+
while lines and not lines[-1]:
|
|
190
|
+
lines.pop()
|
|
191
|
+
|
|
192
|
+
by_type: dict[str, list[str]] = {}
|
|
193
|
+
for source, link_type in graph.incoming.get(block.id, []):
|
|
194
|
+
if source in names:
|
|
195
|
+
by_type.setdefault(link_type, []).append(source)
|
|
196
|
+
if by_type:
|
|
197
|
+
lines.append("")
|
|
198
|
+
for link_type in sorted(by_type):
|
|
199
|
+
sources = ", ".join(_link(uid, names) for uid in sorted(by_type[link_type]))
|
|
200
|
+
lines.append(f"*{_label(link_type)} this:* {sources}")
|
|
201
|
+
return lines
|
|
202
|
+
|
|
203
|
+
|
|
153
204
|
def to_notes(markdown: str, graph: Graph, provenance: Provenance, folder: str = "") -> list[Entry]:
|
|
154
205
|
"""The notes, as the files of the folder. ``folder`` prefixes every path when set."""
|
|
155
206
|
blocks = blocks_of(markdown)
|
|
@@ -166,7 +217,7 @@ def to_notes(markdown: str, graph: Graph, provenance: Provenance, folder: str =
|
|
|
166
217
|
raise ValueError(f"the tool wrote {block.id}, which is not in the graph's export")
|
|
167
218
|
head = ITEM_LINE.match(block.lines[0])
|
|
168
219
|
reference = head.group(2) if head else None
|
|
169
|
-
body = "\n".join(
|
|
220
|
+
body = "\n".join(note_body(block, graph, names))
|
|
170
221
|
entries.append(
|
|
171
222
|
(
|
|
172
223
|
f"{prefix}{_folder_of(block, registers)}/{note_name(block.id)}.md",
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# Copyright (c) 2026 Henry J Grech-Cini
|
|
2
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
3
|
+
"""A word when a step is slow (SR-0015).
|
|
4
|
+
|
|
5
|
+
The command says nothing while a step is quick. When a step has run for a
|
|
6
|
+
second and not finished, one line on stderr names it, so a composed graph
|
|
7
|
+
fetching its sources for the first time does not read as a hang. Never on
|
|
8
|
+
stdout, so a script capturing the output is unaffected; never where there are
|
|
9
|
+
no threads, which is Pyodide, where the caller shows its own progress.
|
|
10
|
+
"""
|
|
11
|
+
|
|
12
|
+
from __future__ import annotations
|
|
13
|
+
|
|
14
|
+
import sys
|
|
15
|
+
import threading
|
|
16
|
+
from contextlib import contextmanager
|
|
17
|
+
from typing import Iterator
|
|
18
|
+
|
|
19
|
+
#: How long a step may run in silence.
|
|
20
|
+
SLOW_AFTER = 1.0
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def has_threads() -> bool:
|
|
24
|
+
"""Whether a timer can run beside the step at all."""
|
|
25
|
+
return sys.platform != "emscripten"
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
@contextmanager
|
|
29
|
+
def slow(message: str, after: float | None = None) -> Iterator[None]:
|
|
30
|
+
"""Say ``message`` on stderr if the enclosed step is still running after ``after`` seconds."""
|
|
31
|
+
timer: threading.Timer | None = None
|
|
32
|
+
if has_threads():
|
|
33
|
+
timer = threading.Timer(SLOW_AFTER if after is None else after, _say, args=(message,))
|
|
34
|
+
timer.daemon = True
|
|
35
|
+
try:
|
|
36
|
+
timer.start()
|
|
37
|
+
except RuntimeError: # a platform that has the module but cannot start a thread
|
|
38
|
+
timer = None
|
|
39
|
+
try:
|
|
40
|
+
yield
|
|
41
|
+
finally:
|
|
42
|
+
if timer is not None:
|
|
43
|
+
timer.cancel()
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
def _say(message: str) -> None:
|
|
47
|
+
print(f"tl-transform: {message}", file=sys.stderr, flush=True)
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/project.py
RENAMED
|
@@ -32,6 +32,7 @@ from pathlib import Path
|
|
|
32
32
|
from typing import Any
|
|
33
33
|
|
|
34
34
|
from .model import Graph, index
|
|
35
|
+
from .progress import slow
|
|
35
36
|
|
|
36
37
|
CONFIG = "throughline.toml"
|
|
37
38
|
|
|
@@ -108,7 +109,7 @@ def _in_process(name: str, argv: list[str]) -> tuple[int, str, str]:
|
|
|
108
109
|
|
|
109
110
|
def run(root: Path, args: list[str], composed: bool) -> str:
|
|
110
111
|
"""Run the tool over the graph and hand back what it wrote."""
|
|
111
|
-
name = "tl
|
|
112
|
+
name = "tl" # tl composes a graph that declares sources itself (throughline 3.11.1)
|
|
112
113
|
argv = ["-C", str(root), *args]
|
|
113
114
|
if has_processes():
|
|
114
115
|
command = [executable(name), *argv]
|
|
@@ -127,10 +128,16 @@ def run(root: Path, args: list[str], composed: bool) -> str:
|
|
|
127
128
|
return stdout
|
|
128
129
|
|
|
129
130
|
|
|
131
|
+
def _tool(composed: bool) -> str:
|
|
132
|
+
return "tl" # tl composes a graph that declares sources itself (throughline 3.11.1)
|
|
133
|
+
|
|
134
|
+
|
|
130
135
|
def load(root: Path) -> Graph:
|
|
131
136
|
"""The graph, as the tool exports it."""
|
|
132
137
|
composed = is_composed(root)
|
|
133
|
-
|
|
138
|
+
fetching = " — its sources may be being fetched" if composed else ""
|
|
139
|
+
with slow(f"still reading the graph with {_tool(composed)}{fetching}…"):
|
|
140
|
+
raw: dict[str, Any] = json.loads(run(root, ["dump"], composed))
|
|
134
141
|
return index(raw, composed)
|
|
135
142
|
|
|
136
143
|
|
|
@@ -143,5 +150,6 @@ def docs(root: Path, document: str, composed: bool) -> str:
|
|
|
143
150
|
with tempfile.TemporaryDirectory(prefix="tl-transform-") as scratch:
|
|
144
151
|
path = Path(scratch) / "document.md"
|
|
145
152
|
path.write_text(document, encoding="utf-8")
|
|
146
|
-
|
|
153
|
+
with slow(f"still asking {_tool(composed)} to write the document…"):
|
|
154
|
+
run(root, ["docs", str(path)], composed)
|
|
147
155
|
return path.read_text(encoding="utf-8")
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: throughline-transform
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.4.0
|
|
4
4
|
Summary: Turn a throughline requirements graph on disk, committed or not, into the forms its readers open: Markdown, a folder of Obsidian notes, CSV, an Excel workbook, a Word document, or a print-ready HTML document. Ships the tl-transform CLI.
|
|
5
5
|
Author-email: Henry J Grech-Cini <henry.grechcini@gmail.com>
|
|
6
6
|
License-Expression: Apache-2.0
|
|
@@ -21,8 +21,7 @@ Requires-Python: >=3.11
|
|
|
21
21
|
Description-Content-Type: text/markdown
|
|
22
22
|
License-File: LICENSE
|
|
23
23
|
License-File: NOTICE
|
|
24
|
-
Requires-Dist: throughline
|
|
25
|
-
Requires-Dist: throughline>=2.2.0
|
|
24
|
+
Requires-Dist: throughline>=3.11.1
|
|
26
25
|
Provides-Extra: dev
|
|
27
26
|
Requires-Dist: pytest>=8.0; extra == "dev"
|
|
28
27
|
Dynamic: license-file
|
|
@@ -33,7 +32,7 @@ A [throughline](https://github.com/rhodium-org/throughline) requirements graph o
|
|
|
33
32
|
disk — committed or not — in the forms its readers open. One command, one file:
|
|
34
33
|
|
|
35
34
|
```bash
|
|
36
|
-
pipx install throughline-transform # pulls tl
|
|
35
|
+
pipx install throughline-transform # pulls tl along too
|
|
37
36
|
|
|
38
37
|
tl-transform md -C idd # Markdown, written by the tool itself
|
|
39
38
|
tl-transform notes -C idd # a folder of notes for Obsidian (zip)
|
|
@@ -64,16 +63,16 @@ worse than none. That is [`NG-0001`](idd/non-goals/NG-0001.yml).
|
|
|
64
63
|
**It ships no PDF writer** ([`NG-0002`](idd/non-goals/NG-0002.yml)). Every
|
|
65
64
|
browser prints to PDF with a renderer that is already on the machine.
|
|
66
65
|
|
|
67
|
-
**It depends on nothing but the family.** `throughline`
|
|
68
|
-
|
|
69
|
-
formats, which are written from their own specifications.
|
|
66
|
+
**It depends on nothing but the family.** `throughline` alone (composition is
|
|
67
|
+
the Tool's own since 3.11.1), and the standard library's `zipfile` for the two
|
|
68
|
+
Office formats, which are written from their own specifications.
|
|
70
69
|
|
|
71
70
|
## Which tool answers
|
|
72
71
|
|
|
73
|
-
|
|
74
|
-
`
|
|
75
|
-
|
|
76
|
-
|
|
72
|
+
Every graph is read through `tl`, which composes a graph that declares
|
|
73
|
+
`[[sources]]` itself, so a borrowed clause is written with the reference number
|
|
74
|
+
a conformance document exists to carry; `tl-compose` is a second name for the
|
|
75
|
+
same program.
|
|
77
76
|
The tool is found beside this package's interpreter first and on the path
|
|
78
77
|
second, and is run as a command — its command line is the surface it publishes.
|
|
79
78
|
|
|
@@ -82,7 +81,7 @@ second, and is run as a command — its command line is the surface it publishes
|
|
|
82
81
|
| Format | What you get |
|
|
83
82
|
|---|---|
|
|
84
83
|
| `md` | The tool's own document: a summary, a catalogue of every item in full, the mirrored clauses of adopted sources. `--body table` or `both` for a one-line-per-item table, `--matrix` for one traceability matrix per grounding link type, `--markers` to keep the region markers so `tl docs` can refill the file later. |
|
|
85
|
-
| `notes` | One Markdown note per item in a folder per register
|
|
84
|
+
| `notes` | One Markdown note per item in a folder per register: YAML front matter as properties (type, status, attributes, the title as an alias, a tag each for type and status), a heading of the identifier and title, the item's words, rationale and links, then what links to it — `Implements this`, `Derives from this` — every identifier a `[[wikilink]]`. Cited clauses under `sources/<namespace>/`, the colon in their name replaced by a space. Open the folder as an Obsidian vault: the graph view is the intent graph, and a note's backlinks are what rests on it. |
|
|
86
85
|
| `csv` | A zip of one file of items (or one per register with `--split per-register`), `links.csv` with one row per link naming the sheet at each end, one file per cited source, and the provenance. |
|
|
87
86
|
| `xlsx` | The same as sheets, read-only by recommendation with the structure locked and no password. Sorting and filtering stay allowed; a spreadsheet nobody can sort is one nobody can review. |
|
|
88
87
|
| `docx` | A title page with the provenance, a contents field Word fills in, a section per register, a heading per item with its type and status beneath, its words as a quotation, its rationale, its attributes in small print, and one appendix per cited source. |
|
|
@@ -96,14 +95,19 @@ knows them without git — the editor holds them from its clone — states them
|
|
|
96
95
|
`--repository`, `--ref`, `--commit` and `--tree clean|dirty|untracked`, and git
|
|
97
96
|
is then not consulted.
|
|
98
97
|
|
|
98
|
+
A quick run is silent. When a step — reading the graph, asking the tool to
|
|
99
|
+
write the document, writing the file — has run for a second, one line on
|
|
100
|
+
stderr says which, because a composed graph fetching its sources for the first
|
|
101
|
+
time otherwise reads as a hang.
|
|
102
|
+
|
|
99
103
|
## One implementation, two surfaces
|
|
100
104
|
|
|
101
105
|
The [throughline editor](https://github.com/rhodium-org/throughline-editor)
|
|
102
|
-
runs this package as a wheel under Pyodide beside `tl
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
106
|
+
runs this package as a wheel under Pyodide beside `tl`, so an export from a
|
|
107
|
+
browser tab and an export from a terminal are the same bytes for the same graph
|
|
108
|
+
at the same commit. Pyodide has no processes, so there the tool calls the
|
|
109
|
+
console entry point `tl` declares in its package metadata — the same function
|
|
110
|
+
the command runs — instead of spawning it.
|
|
107
111
|
|
|
108
112
|
## Self-hosting
|
|
109
113
|
|
|
@@ -12,6 +12,7 @@ src/throughline_transform/model.py
|
|
|
12
12
|
src/throughline_transform/notes.py
|
|
13
13
|
src/throughline_transform/office.py
|
|
14
14
|
src/throughline_transform/package.py
|
|
15
|
+
src/throughline_transform/progress.py
|
|
15
16
|
src/throughline_transform/project.py
|
|
16
17
|
src/throughline_transform/provenance.py
|
|
17
18
|
src/throughline_transform/shape.py
|
|
@@ -23,7 +23,7 @@ BIN = Path(sys.executable).parent
|
|
|
23
23
|
|
|
24
24
|
|
|
25
25
|
def _tl(root: Path, *args: str, compose: bool = False) -> str:
|
|
26
|
-
exe = BIN /
|
|
26
|
+
exe = BIN / "tl" # tl composes a graph that declares sources itself (throughline 3.11.1)
|
|
27
27
|
if not exe.exists(): # pragma: no cover - only when the suite runs outside its venv
|
|
28
28
|
exe = Path(shutil.which(exe.name) or exe.name)
|
|
29
29
|
done = subprocess.run([str(exe), "-C", str(root), *args], capture_output=True, text=True, encoding="utf-8")
|
|
@@ -235,8 +235,19 @@ def test_notes_are_one_per_item_with_front_matter_and_links(plain_graph: Path):
|
|
|
235
235
|
sr2 = entries["System requirements/SR-0002.md"].decode()
|
|
236
236
|
assert sr2.startswith('---\nuid: "SR-0002"\n')
|
|
237
237
|
assert 'priority: "should"' in sr2 and ' - "type/system_requirement"' in sr2 and ' - "status/draft"' in sr2
|
|
238
|
+
# The body is the tool's block reshaped for a note (SR-0009): a heading,
|
|
239
|
+
# the type and status beneath, no attribute line, the incoming links.
|
|
240
|
+
body = sr2.split("\n---\n", 1)[1]
|
|
241
|
+
assert body.startswith("# SR-0002 — Declare it in \\[\\[sources\\]\\]\nsystem requirement · draft\n")
|
|
242
|
+
assert "**priority**" not in body and "**origin**" not in body
|
|
238
243
|
assert "*Relates:* [[SR-0001]]" in sr2
|
|
239
244
|
assert "*Implements:* [[UR-0002]]" in sr2
|
|
245
|
+
sr1 = entries["System requirements/SR-0001.md"].decode()
|
|
246
|
+
assert "*Refines this:* [[SR-0003]]" in sr1 and "*Relates this:* [[SR-0002]]" in sr1
|
|
247
|
+
ur1 = entries["User requirements/UR-0001.md"].decode()
|
|
248
|
+
assert "*Implements this:* [[SR-0001]]" in ur1
|
|
249
|
+
int1 = entries["Intents/INT-0001.md"].decode()
|
|
250
|
+
assert "*Derives from this:* [[UR-0001]], [[UR-0002]]" in int1
|
|
240
251
|
# The TOML table in prose is escaped rather than becoming a ghost note. The
|
|
241
252
|
# title in the front matter is a quoted YAML string, which the note tool
|
|
242
253
|
# reads as a property, not as a link.
|
|
@@ -266,6 +277,9 @@ def test_borrowed_clause_notes_replace_the_colon(composed_graph: Path):
|
|
|
266
277
|
assert 'uid: "src:SR-0001"' in clause and 'source: "src"' in clause
|
|
267
278
|
citing = entries["vault/System requirements/SR-0001.md"].decode()
|
|
268
279
|
assert "[[src SR-0001|src:SR-0001]]" in citing
|
|
280
|
+
# The clause shows which local items claim it.
|
|
281
|
+
assert "*Satisfies this:* [[SR-0001]]" in clause
|
|
282
|
+
assert clause.split("\n---\n", 1)[1].startswith("# src:SR-0001 — ")
|
|
269
283
|
assert note_name("asvs:SR-0255") == "asvs SR-0255"
|
|
270
284
|
|
|
271
285
|
|
|
@@ -133,3 +133,51 @@ def test_a_failing_tool_is_reported_in_process(tmp_path: Path, monkeypatch: pyte
|
|
|
133
133
|
with pytest.raises(project.TransformError) as exc:
|
|
134
134
|
project.run(tmp_path, ["dump"], composed=False)
|
|
135
135
|
assert "tl dump failed" in str(exc.value)
|
|
136
|
+
|
|
137
|
+
|
|
138
|
+
# --- a word when a step is slow (SR-0015) -----------------------------------
|
|
139
|
+
|
|
140
|
+
|
|
141
|
+
def _capture(capsys, argv):
|
|
142
|
+
code = main(argv)
|
|
143
|
+
out = capsys.readouterr()
|
|
144
|
+
return code, out.out, out.err
|
|
145
|
+
|
|
146
|
+
|
|
147
|
+
def test_a_quick_run_says_nothing_on_stderr(plain_graph: Path, tmp_path: Path, capsys: pytest.CaptureFixture):
|
|
148
|
+
code, out, err = _capture(capsys, ["md", "-C", str(plain_graph), "-o", str(tmp_path / "q.md")])
|
|
149
|
+
assert code == 0 and err == "" and str(tmp_path / "q.md") in out
|
|
150
|
+
|
|
151
|
+
|
|
152
|
+
def test_a_slow_step_is_named_once_on_stderr(composed_graph: Path, tmp_path: Path, capsys: pytest.CaptureFixture, monkeypatch: pytest.MonkeyPatch):
|
|
153
|
+
from throughline_transform import progress
|
|
154
|
+
|
|
155
|
+
monkeypatch.setattr(progress, "SLOW_AFTER", 0.2)
|
|
156
|
+
real = subprocess.run
|
|
157
|
+
|
|
158
|
+
def dawdle(cmd, *a, **k):
|
|
159
|
+
if "dump" in cmd:
|
|
160
|
+
import time
|
|
161
|
+
|
|
162
|
+
time.sleep(0.5)
|
|
163
|
+
return real(cmd, *a, **k)
|
|
164
|
+
|
|
165
|
+
monkeypatch.setattr(subprocess, "run", dawdle)
|
|
166
|
+
code, out, err = _capture(capsys, ["md", "-C", str(composed_graph), "-o", str(tmp_path / "s.md")])
|
|
167
|
+
assert code == 0
|
|
168
|
+
# Once per slow step: the reading step was made slow, and with the
|
|
169
|
+
# threshold this low the docs step over a composed graph may be too.
|
|
170
|
+
assert err.startswith("tl-transform: still reading the graph with tl — its sources may be being fetched")
|
|
171
|
+
assert err.count("still reading") == 1
|
|
172
|
+
assert all(line.startswith("tl-transform: still ") for line in err.splitlines())
|
|
173
|
+
assert str(tmp_path / "s.md") in out
|
|
174
|
+
|
|
175
|
+
|
|
176
|
+
def test_no_threads_means_no_feedback(composed_graph: Path, tmp_path: Path, capsys: pytest.CaptureFixture, monkeypatch: pytest.MonkeyPatch):
|
|
177
|
+
from throughline_transform import progress
|
|
178
|
+
|
|
179
|
+
monkeypatch.setattr(progress, "SLOW_AFTER", 0.05)
|
|
180
|
+
monkeypatch.setattr(sys, "platform", "emscripten")
|
|
181
|
+
assert not progress.has_threads()
|
|
182
|
+
code, _, err = _capture(capsys, ["notes", "-C", str(composed_graph), "-o", str(tmp_path / "n.zip"), "--repository", "r"])
|
|
183
|
+
assert code == 0 and err == ""
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/__init__.py
RENAMED
|
File without changes
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/blocks.py
RENAMED
|
File without changes
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/html.py
RENAMED
|
File without changes
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/markdown.py
RENAMED
|
File without changes
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/model.py
RENAMED
|
File without changes
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/office.py
RENAMED
|
File without changes
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/package.py
RENAMED
|
File without changes
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/provenance.py
RENAMED
|
File without changes
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/shape.py
RENAMED
|
File without changes
|
{throughline_transform-0.2.0 → throughline_transform-0.4.0}/src/throughline_transform/tables.py
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|