rowcanary 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- rowcanary-1.0.0/.editorconfig +12 -0
- rowcanary-1.0.0/.gitattributes +6 -0
- rowcanary-1.0.0/.github/workflows/publish.yml +57 -0
- rowcanary-1.0.0/.github/workflows/tests.yml +72 -0
- rowcanary-1.0.0/.gitignore +21 -0
- rowcanary-1.0.0/CHANGELOG.md +127 -0
- rowcanary-1.0.0/CONTRIBUTING.md +28 -0
- rowcanary-1.0.0/LICENSE +21 -0
- rowcanary-1.0.0/MANIFEST.in +13 -0
- rowcanary-1.0.0/PKG-INFO +283 -0
- rowcanary-1.0.0/README.md +249 -0
- rowcanary-1.0.0/SECURITY.md +54 -0
- rowcanary-1.0.0/assets/FONT-LICENSE.txt +93 -0
- rowcanary-1.0.0/assets/README.md +43 -0
- rowcanary-1.0.0/assets/icon-128.png +0 -0
- rowcanary-1.0.0/assets/icon-256.png +0 -0
- rowcanary-1.0.0/assets/icon-32.png +0 -0
- rowcanary-1.0.0/assets/icon-64.png +0 -0
- rowcanary-1.0.0/assets/icon.svg +1 -0
- rowcanary-1.0.0/assets/logo-512.png +0 -0
- rowcanary-1.0.0/assets/logo-dark.svg +1 -0
- rowcanary-1.0.0/assets/logo-light.svg +1 -0
- rowcanary-1.0.0/assets/logo.svg +1 -0
- rowcanary-1.0.0/assets/rowcanary-logo.png +0 -0
- rowcanary-1.0.0/assets/social-preview.png +0 -0
- rowcanary-1.0.0/assets/social-preview.svg +1 -0
- rowcanary-1.0.0/docs/compatibility.md +71 -0
- rowcanary-1.0.0/docs/development-plan.md +36 -0
- rowcanary-1.0.0/docs/formats.md +46 -0
- rowcanary-1.0.0/docs/integrations.md +62 -0
- rowcanary-1.0.0/docs/limitations.md +72 -0
- rowcanary-1.0.0/docs/performance.md +46 -0
- rowcanary-1.0.0/docs/publication.md +39 -0
- rowcanary-1.0.0/docs/quickstart.md +56 -0
- rowcanary-1.0.0/docs/release-notes.md +18 -0
- rowcanary-1.0.0/docs/reporting.md +49 -0
- rowcanary-1.0.0/docs/screenshots/desktop.png +0 -0
- rowcanary-1.0.0/docs/screenshots/mobile-dark.png +0 -0
- rowcanary-1.0.0/docs/superpowers/plans/2026-11-03-compatibility.md +27 -0
- rowcanary-1.0.0/docs/superpowers/plans/2026-11-03-local-ui.md +38 -0
- rowcanary-1.0.0/docs/superpowers/plans/2026-11-03-streaming-resources.md +32 -0
- rowcanary-1.0.0/docs/verification.md +191 -0
- rowcanary-1.0.0/docs/web-ui.md +64 -0
- rowcanary-1.0.0/examples/altered.csv +26 -0
- rowcanary-1.0.0/examples/clean.csv +26 -0
- rowcanary-1.0.0/examples/clean.json +98 -0
- rowcanary-1.0.0/examples/compatibility/clean.csv +38 -0
- rowcanary-1.0.0/examples/compatibility/clean.json +142 -0
- rowcanary-1.0.0/examples/compatibility/clean.jsonl +35 -0
- rowcanary-1.0.0/examples/compatibility/clean.tsv +38 -0
- rowcanary-1.0.0/examples/compatibility/clean.xlsx +0 -0
- rowcanary-1.0.0/examples/compatibility/excel/automatic.csv +38 -0
- rowcanary-1.0.0/examples/compatibility/excel/manifest.json +122 -0
- rowcanary-1.0.0/examples/compatibility/excel/text.csv +38 -0
- rowcanary-1.0.0/examples/compatibility/manifest.json +182 -0
- rowcanary-1.0.0/examples/compatibility/semicolon.csv +38 -0
- rowcanary-1.0.0/examples/compatibility/synthetic-changes.csv +39 -0
- rowcanary-1.0.0/examples/compatibility/synthetic-types.json +142 -0
- rowcanary-1.0.0/examples/compatibility/utf16.csv +0 -0
- rowcanary-1.0.0/examples/compatibility/utf16.jsonl +0 -0
- rowcanary-1.0.0/examples/compatibility/utf16.tsv +0 -0
- rowcanary-1.0.0/examples/compatibility/utf32.json +0 -0
- rowcanary-1.0.0/examples/compatibility/utf8-bom.csv +38 -0
- rowcanary-1.0.0/examples/config.json +7 -0
- rowcanary-1.0.0/examples/github-actions.yml +16 -0
- rowcanary-1.0.0/examples/integration.py +20 -0
- rowcanary-1.0.0/pyproject.toml +52 -0
- rowcanary-1.0.0/rowcanary/__init__.py +25 -0
- rowcanary-1.0.0/rowcanary/__main__.py +4 -0
- rowcanary-1.0.0/rowcanary/cli.py +201 -0
- rowcanary-1.0.0/rowcanary/config.py +110 -0
- rowcanary-1.0.0/rowcanary/config.schema.json +17 -0
- rowcanary-1.0.0/rowcanary/core.py +428 -0
- rowcanary-1.0.0/rowcanary/diagnostics.py +119 -0
- rowcanary-1.0.0/rowcanary/formats.py +200 -0
- rowcanary-1.0.0/rowcanary/probes.py +98 -0
- rowcanary-1.0.0/rowcanary/report.schema.json +54 -0
- rowcanary-1.0.0/rowcanary/reporting.py +72 -0
- rowcanary-1.0.0/rowcanary/static/app.css +6 -0
- rowcanary-1.0.0/rowcanary/static/app.js +150 -0
- rowcanary-1.0.0/rowcanary/static/icon.svg +1 -0
- rowcanary-1.0.0/rowcanary/static/index.html +42 -0
- rowcanary-1.0.0/rowcanary/streaming.py +108 -0
- rowcanary-1.0.0/rowcanary/web.py +236 -0
- rowcanary-1.0.0/rowcanary.egg-info/PKG-INFO +283 -0
- rowcanary-1.0.0/rowcanary.egg-info/SOURCES.txt +108 -0
- rowcanary-1.0.0/rowcanary.egg-info/dependency_links.txt +1 -0
- rowcanary-1.0.0/rowcanary.egg-info/entry_points.txt +2 -0
- rowcanary-1.0.0/rowcanary.egg-info/requires.txt +11 -0
- rowcanary-1.0.0/rowcanary.egg-info/top_level.txt +1 -0
- rowcanary-1.0.0/setup.cfg +4 -0
- rowcanary-1.0.0/tests/test_cli.py +34 -0
- rowcanary-1.0.0/tests/test_cli_process.py +239 -0
- rowcanary-1.0.0/tests/test_compatibility.py +77 -0
- rowcanary-1.0.0/tests/test_config.py +145 -0
- rowcanary-1.0.0/tests/test_core.py +142 -0
- rowcanary-1.0.0/tests/test_documentation.py +24 -0
- rowcanary-1.0.0/tests/test_formats.py +175 -0
- rowcanary-1.0.0/tests/test_parsing.py +314 -0
- rowcanary-1.0.0/tests/test_reporting.py +193 -0
- rowcanary-1.0.0/tests/test_resources.py +144 -0
- rowcanary-1.0.0/tests/test_stabilization.py +30 -0
- rowcanary-1.0.0/tests/test_web.py +179 -0
- rowcanary-1.0.0/tools/benchmark.py +90 -0
- rowcanary-1.0.0/tools/build_assets.py +156 -0
- rowcanary-1.0.0/tools/check_assets.py +63 -0
- rowcanary-1.0.0/tools/check_package.py +93 -0
- rowcanary-1.0.0/tools/compatibility.py +114 -0
- rowcanary-1.0.0/tools/excel-roundtrip.ps1 +63 -0
- rowcanary-1.0.0/tools/requirements-assets.txt +4 -0
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
name: Publish to PyPI
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
workflow_dispatch:
|
|
5
|
+
|
|
6
|
+
permissions:
|
|
7
|
+
contents: read
|
|
8
|
+
|
|
9
|
+
concurrency:
|
|
10
|
+
group: pypi-publish
|
|
11
|
+
cancel-in-progress: false
|
|
12
|
+
|
|
13
|
+
jobs:
|
|
14
|
+
build:
|
|
15
|
+
if: github.ref == 'refs/heads/main'
|
|
16
|
+
runs-on: ubuntu-latest
|
|
17
|
+
steps:
|
|
18
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
19
|
+
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
|
|
20
|
+
with:
|
|
21
|
+
python-version: '3.10'
|
|
22
|
+
- name: Build and validate distributions
|
|
23
|
+
run: |
|
|
24
|
+
python -m pip install build twine
|
|
25
|
+
python -m build
|
|
26
|
+
python -m twine check dist/*
|
|
27
|
+
python tools/check_package.py dist
|
|
28
|
+
- name: Smoke test installed wheel outside checkout
|
|
29
|
+
run: |
|
|
30
|
+
python -m venv /tmp/rowcanary-publish
|
|
31
|
+
/tmp/rowcanary-publish/bin/python -m pip install dist/*.whl
|
|
32
|
+
cd /tmp
|
|
33
|
+
/tmp/rowcanary-publish/bin/rowcanary --version
|
|
34
|
+
/tmp/rowcanary-publish/bin/rowcanary demo
|
|
35
|
+
/tmp/rowcanary-publish/bin/rowcanary make publish-probe.csv
|
|
36
|
+
/tmp/rowcanary-publish/bin/rowcanary check publish-probe.csv
|
|
37
|
+
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
|
|
38
|
+
with:
|
|
39
|
+
name: distributions
|
|
40
|
+
path: dist/*
|
|
41
|
+
if-no-files-found: error
|
|
42
|
+
|
|
43
|
+
publish:
|
|
44
|
+
needs: build
|
|
45
|
+
runs-on: ubuntu-latest
|
|
46
|
+
environment:
|
|
47
|
+
name: pypi
|
|
48
|
+
url: https://pypi.org/project/rowcanary/
|
|
49
|
+
permissions:
|
|
50
|
+
id-token: write
|
|
51
|
+
steps:
|
|
52
|
+
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
|
53
|
+
with:
|
|
54
|
+
name: distributions
|
|
55
|
+
path: dist
|
|
56
|
+
- name: Publish with Trusted Publishing
|
|
57
|
+
uses: pypa/gh-action-pypi-publish@7245906fb61fa33a13554f9d5fb856fe61758086 # release/v1
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
name: Checks
|
|
2
|
+
on:
|
|
3
|
+
push:
|
|
4
|
+
pull_request:
|
|
5
|
+
workflow_dispatch:
|
|
6
|
+
|
|
7
|
+
permissions:
|
|
8
|
+
contents: read
|
|
9
|
+
|
|
10
|
+
jobs:
|
|
11
|
+
test:
|
|
12
|
+
runs-on: ${{ matrix.os }}
|
|
13
|
+
strategy:
|
|
14
|
+
fail-fast: false
|
|
15
|
+
matrix:
|
|
16
|
+
os: [ubuntu-latest, macos-15-intel, windows-latest]
|
|
17
|
+
python-version: ['3.10', '3.11', '3.12', '3.13', '3.14']
|
|
18
|
+
steps:
|
|
19
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
20
|
+
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
|
|
21
|
+
with:
|
|
22
|
+
python-version: ${{ matrix.python-version }}
|
|
23
|
+
- name: Install
|
|
24
|
+
run: python -m pip install '.[test]'
|
|
25
|
+
- name: Run tests
|
|
26
|
+
run: python -m unittest discover -s tests -v
|
|
27
|
+
- name: Smoke test
|
|
28
|
+
run: |
|
|
29
|
+
python -m rowcanary demo
|
|
30
|
+
python -m rowcanary make probe.csv
|
|
31
|
+
python -m rowcanary check probe.csv
|
|
32
|
+
rowcanary --version
|
|
33
|
+
|
|
34
|
+
package:
|
|
35
|
+
runs-on: ubuntu-latest
|
|
36
|
+
steps:
|
|
37
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
38
|
+
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
|
|
39
|
+
with:
|
|
40
|
+
python-version: '3.10'
|
|
41
|
+
- name: Install development tools
|
|
42
|
+
run: python -m pip install '.[dev]'
|
|
43
|
+
- name: Static analysis
|
|
44
|
+
run: |
|
|
45
|
+
python -m ruff check .
|
|
46
|
+
python -m ruff format --check rowcanary tests tools
|
|
47
|
+
- name: Build and validate distributions
|
|
48
|
+
run: |
|
|
49
|
+
python -m build
|
|
50
|
+
python -m twine check dist/*
|
|
51
|
+
python tools/check_package.py dist
|
|
52
|
+
- name: Test installed wheel outside the checkout
|
|
53
|
+
run: |
|
|
54
|
+
python -m venv /tmp/rowcanary-wheel
|
|
55
|
+
/tmp/rowcanary-wheel/bin/python -m pip install dist/*.whl
|
|
56
|
+
cd /tmp
|
|
57
|
+
/tmp/rowcanary-wheel/bin/python -m rowcanary demo
|
|
58
|
+
/tmp/rowcanary-wheel/bin/rowcanary make probe.csv
|
|
59
|
+
/tmp/rowcanary-wheel/bin/rowcanary check probe.csv
|
|
60
|
+
/tmp/rowcanary-wheel/bin/python -c "import json; from pathlib import Path; from importlib.resources import files; from rowcanary import Config, check_file, write_probes; c=Config(probe_set='extended'); write_probes(Path('api.json'), **c.make_options()); assert check_file(Path('api.json'), **c.check_options()).ok; json.loads(files('rowcanary').joinpath('config.schema.json').read_bytes())"
|
|
61
|
+
|
|
62
|
+
assets:
|
|
63
|
+
runs-on: ubuntu-latest
|
|
64
|
+
steps:
|
|
65
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
66
|
+
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
|
|
67
|
+
with:
|
|
68
|
+
python-version: '3.14'
|
|
69
|
+
- name: Install asset verification tools
|
|
70
|
+
run: python -m pip install -r tools/requirements-assets.txt
|
|
71
|
+
- name: Validate vectors and PNG exports
|
|
72
|
+
run: python tools/check_assets.py
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
__pycache__/
|
|
2
|
+
*.py[cod]
|
|
3
|
+
.pytest_cache/
|
|
4
|
+
.venv/
|
|
5
|
+
.venv-*/
|
|
6
|
+
.local/
|
|
7
|
+
.ruff_cache/
|
|
8
|
+
venv/
|
|
9
|
+
*.egg-info/
|
|
10
|
+
build/
|
|
11
|
+
dist/
|
|
12
|
+
.coverage
|
|
13
|
+
.serena/
|
|
14
|
+
.DS_Store
|
|
15
|
+
rowcanary-probe.csv
|
|
16
|
+
.trail/cache/
|
|
17
|
+
.trail/graph.db
|
|
18
|
+
.trail/state.json
|
|
19
|
+
Thumbs.db
|
|
20
|
+
.playwright-cli/
|
|
21
|
+
output/playwright/
|
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## [1.0.0] - 2026-11-03
|
|
4
|
+
|
|
5
|
+
- Publish the frozen, backward-compatible CLI, five-format comparison engine,
|
|
6
|
+
diagnostics/reports, local browser interface and Python/configuration API.
|
|
7
|
+
- Complete the sequential roadmap, final regression and packaging review;
|
|
8
|
+
retain bounded inputs, exact values/types and documented application limits.
|
|
9
|
+
- Provide verified wheel, sdist, source ZIP and SHA-256 checksums on GitHub.
|
|
10
|
+
No PyPI publication is included.
|
|
11
|
+
|
|
12
|
+
## [0.9.0] - 2026-11-03
|
|
13
|
+
|
|
14
|
+
- Freeze the 1.0 feature/API surface after focused independent security and
|
|
15
|
+
compatibility review of the new configuration, CLI and browser changes.
|
|
16
|
+
- Update security boundaries for all formats, local HTTP and configuration.
|
|
17
|
+
- Verify the installed public API and configuration schema outside the checkout
|
|
18
|
+
in hosted packaging CI, in addition to existing local wheel/sdist gates.
|
|
19
|
+
- Retain known application limitations and the unchanged runtime dependencies.
|
|
20
|
+
|
|
21
|
+
## [0.8.0] - 2026-11-03
|
|
22
|
+
|
|
23
|
+
- Add a complete first-run guide, spreadsheet guidance and troubleshooting.
|
|
24
|
+
- Clear stale browser reports after input/mode changes; lock fields while busy
|
|
25
|
+
and announce errors assertively. Retain keyboard focus and both themes.
|
|
26
|
+
- Include actual desktop/mobile screenshots and verify browser generation,
|
|
27
|
+
PASS/FAIL, report downloads, labels, skip-link focus and reduced motion.
|
|
28
|
+
|
|
29
|
+
## [0.7.0] - 2026-11-03
|
|
30
|
+
|
|
31
|
+
- Stabilize public Python imports and add immutable, validated Config/load_config.
|
|
32
|
+
- Support bounded UTF-8 JSON configuration for make/check/batch, relative custom
|
|
33
|
+
paths, explicit CLI precedence and --no-strict-order. Preserve prior defaults.
|
|
34
|
+
- Package configuration schema 1.0 and Python, batch and pinned Actions examples.
|
|
35
|
+
- Reject unknown/duplicate keys, invalid option types, encodings and oversized JSON.
|
|
36
|
+
|
|
37
|
+
## [0.6.0] - 2026-11-03
|
|
38
|
+
|
|
39
|
+
- Add 13 deterministic synthetic compatibility fixtures with independent
|
|
40
|
+
expected issue IDs/codes, explicit provenance and SHA-256 manifests.
|
|
41
|
+
- Cover five formats, UTF BOM profiles, semicolon CSV, changed/reordered rows
|
|
42
|
+
and native JSON scalar coercions; verify byte-identical corpus rebuilds.
|
|
43
|
+
- Record native Microsoft Excel 16.0 build 20527 (32-bit) OpenText/CSVUTF8
|
|
44
|
+
observations: text-column import preserved 33/35 controls, automatic inference
|
|
45
|
+
20/35. Retain raw CSV outputs and exact settings; neither profile fully passed.
|
|
46
|
+
- Add an isolated, macro-disabled Excel automation script and an evidence matrix
|
|
47
|
+
that separates parser fixtures, native observations and untested applications.
|
|
48
|
+
- Use fixed ZIP platform metadata and stored XLSX members so generated bytes do
|
|
49
|
+
not vary with host zlib versions. Files are larger; existing output limits apply.
|
|
50
|
+
|
|
51
|
+
## [0.5.0] - 2026-11-03
|
|
52
|
+
|
|
53
|
+
- Stream CSV, TSV and JSONL through bounded, context-managed readers, preserving
|
|
54
|
+
exact CR/LF/CRLF and enforcing byte limits even after stale size checks.
|
|
55
|
+
- Compare streamed records once without consuming JSONL rows during column
|
|
56
|
+
inference; close input handles after success and failure.
|
|
57
|
+
- Bound diagnostic counts/text and JSON numeric fields; reject duplicate XLSX
|
|
58
|
+
archive members and multiple objects per JSONL line.
|
|
59
|
+
- Avoid reserving the entire byte limit for small buffered reads.
|
|
60
|
+
- Bound the local server backlog, test concurrent isolation and hostile framing,
|
|
61
|
+
and render only the first 200 UI items while retaining complete downloads.
|
|
62
|
+
- Add reproducible synthetic time/Python-allocation benchmarks; document that
|
|
63
|
+
JSON arrays/XLSX remain buffered and measurements do not represent process RSS.
|
|
64
|
+
|
|
65
|
+
## [0.4.0] - 2026-11-03
|
|
66
|
+
|
|
67
|
+
- Add `rowcanary serve` with a browser interface bound only to `127.0.0.1`.
|
|
68
|
+
- Generate/download controls and upload/check exports in all five formats,
|
|
69
|
+
including custom controls, exact types and strict-order comparison.
|
|
70
|
+
- Download versioned JSON and escaped standalone HTML reports.
|
|
71
|
+
- Include labeled keyboard-accessible controls, responsive light/dark themes,
|
|
72
|
+
the existing canary identity and no external resources or runtime dependencies.
|
|
73
|
+
- Enforce exact Host/Origin and per-session token checks, fixed static routes,
|
|
74
|
+
request/upload limits, temporary files and no-store/security headers.
|
|
75
|
+
|
|
76
|
+
## [0.3.0] - 2026-11-03
|
|
77
|
+
|
|
78
|
+
- Add evidence-based classifications, severity, scalar types, observations,
|
|
79
|
+
tentative possible causes and practical guidance without changing issue codes.
|
|
80
|
+
- Observe reordered rows without failing by default; add opt-in `--strict-order`.
|
|
81
|
+
- Add deterministic, escaped standalone HTML and JSON exports with exclusive
|
|
82
|
+
creation through `check --report` and `batch --report`.
|
|
83
|
+
- Check multiple files with `batch`, retaining every result; operational errors
|
|
84
|
+
take exit-code precedence over integrity differences.
|
|
85
|
+
- Package report JSON Schema 1.0 and validate real single and batch reports.
|
|
86
|
+
- Keep schema validation a test dependency; runtime dependencies are unchanged.
|
|
87
|
+
|
|
88
|
+
## [0.2.0] - 2026-11-02
|
|
89
|
+
|
|
90
|
+
- Generate and audit CSV, TSV, JSON, JSONL and single-sheet XLSX files.
|
|
91
|
+
- Add explicit format/encoding options, conservative content detection and
|
|
92
|
+
UTF-16/32 BOM decoding without normalizing in-cell line breaks.
|
|
93
|
+
- Add deterministic extended/category probe sets and validated custom scalar
|
|
94
|
+
controls, preserving the original default probe IDs and CLI commands.
|
|
95
|
+
- Preserve JSON/XLSX scalar types; refuse lossy XLSX integer/text generation.
|
|
96
|
+
- Store XLSX strings as literal text and reject returned formulas without
|
|
97
|
+
executing them. Bound ZIP expansion and validate malformed workbooks.
|
|
98
|
+
- Write deterministic OpenXML/ZIP metadata and preserve embedded CR/LF/CRLF
|
|
99
|
+
across platforms without passing generated data through text-mode writers.
|
|
100
|
+
|
|
101
|
+
## [0.1.1] - 2026-11-02
|
|
102
|
+
|
|
103
|
+
- Enforce the documented string-field safety bound in JSON, including nested
|
|
104
|
+
extra data, so JSON cannot bypass the bound applied to CSV fields.
|
|
105
|
+
- Reject invalid API column names before reading a file, with an actionable error.
|
|
106
|
+
- Add regression coverage for both validation failures. Preserve all existing
|
|
107
|
+
CLI commands, default probes and exact CR/LF comparisons.
|
|
108
|
+
|
|
109
|
+
## [0.1.0] - 2026-11-02
|
|
110
|
+
|
|
111
|
+
- Generate 24 deterministic, realistic edge-case CSV controls.
|
|
112
|
+
- Compare round-tripped CSV or JSON by stable probe IDs, regardless of row order.
|
|
113
|
+
- Detect dropped, duplicate, added, and changed control records.
|
|
114
|
+
- Print human-readable or JSON reports with appropriate exit codes.
|
|
115
|
+
- Include an isolated demonstration, automated tests and CI configuration.
|
|
116
|
+
- Preserve in-cell CR/LF/CRLF exactly and report blank CSV records.
|
|
117
|
+
- Support explicit CSV delimiters without automatic data normalization.
|
|
118
|
+
- Reject duplicate JSON keys and nonstandard constants; retain numeric token
|
|
119
|
+
precision for diagnostics without accepting numbers as strings.
|
|
120
|
+
- Bound reads to 32 MiB, exported records to 100,000 and CSV fields to 128 Ki
|
|
121
|
+
characters; refuse existing output symlinks, including dangling links.
|
|
122
|
+
- Escape untrusted terminal text and support restricted output encodings.
|
|
123
|
+
- Return structured JSON errors for `check --json` input and I/O failures.
|
|
124
|
+
- Clarify spreadsheet formula risks and the limits of a passing result.
|
|
125
|
+
- Reconstruct the original bird/table logo as self-contained SVG paths, with
|
|
126
|
+
PNG exports and a 1280 × 640 social preview.
|
|
127
|
+
- Include package inspection, asset verification and regression tests.
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
Use an issue to describe a reproducible data-change case, including Python/OS
|
|
4
|
+
versions and the exact commands used. Include synthetic input and observed
|
|
5
|
+
output. Do not share customer records, credentials or unredacted private paths.
|
|
6
|
+
Read [SECURITY.md](SECURITY.md) before reporting a vulnerability.
|
|
7
|
+
|
|
8
|
+
For code changes:
|
|
9
|
+
|
|
10
|
+
1. Install development tools: `python -m pip install -e ".[dev]"`.
|
|
11
|
+
2. Add a focused regression test that fails before the fix.
|
|
12
|
+
3. Preserve exact strings and JSON types. Do not hide data changes through
|
|
13
|
+
trimming, newline replacement, Unicode normalization or numeric coercion.
|
|
14
|
+
4. Prefer the Python standard library and small changes at the responsible
|
|
15
|
+
layer. Keep Python 3.10 compatibility.
|
|
16
|
+
5. Run `python -m unittest discover -s tests -v` and `python -m ruff check .`.
|
|
17
|
+
6. For a release, also run `python -m build`, `python -m twine check dist/*` and
|
|
18
|
+
`python tools/check_package.py dist` in a directory containing one current
|
|
19
|
+
wheel and one current source distribution.
|
|
20
|
+
|
|
21
|
+
CLI exit statuses are part of the interface: 0 for matching controls or
|
|
22
|
+
successful generation/demo, 1 for integrity differences, 2 for invalid input or
|
|
23
|
+
I/O/usage errors. `check --json` operational errors produce an `error` object;
|
|
24
|
+
argument parsing errors remain stderr usage diagnostics.
|
|
25
|
+
|
|
26
|
+
Tests must never evaluate spreadsheet formulas. Use only synthetic fixtures
|
|
27
|
+
and temporary directories; verify that generation cannot overwrite an existing
|
|
28
|
+
path. See [asset instructions](assets/README.md) before changing artwork.
|
rowcanary-1.0.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 RowCanary contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
include README.md LICENSE CHANGELOG.md CONTRIBUTING.md SECURITY.md
|
|
2
|
+
include .editorconfig .gitignore .gitattributes
|
|
3
|
+
recursive-include .github/workflows *.yml
|
|
4
|
+
recursive-include assets *.svg *.png *.md *.txt
|
|
5
|
+
recursive-include docs *.md *.png
|
|
6
|
+
recursive-include examples *.csv *.tsv *.json *.jsonl *.xlsx *.md *.py *.yml
|
|
7
|
+
recursive-include rowcanary/static *.html *.css *.js *.svg
|
|
8
|
+
recursive-include tests *.py
|
|
9
|
+
recursive-include tools *.py *.ps1 *.txt
|
|
10
|
+
global-exclude *.py[cod] .DS_Store
|
|
11
|
+
prune .trail
|
|
12
|
+
prune .local
|
|
13
|
+
prune .venv
|
rowcanary-1.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,283 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: rowcanary
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Detect silent data corruption in CSV import/export workflows with control samples
|
|
5
|
+
Author: RowCanary contributors
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Source, https://github.com/spockplayer2-alt/rowcanary
|
|
8
|
+
Project-URL: Issues, https://github.com/spockplayer2-alt/rowcanary/issues
|
|
9
|
+
Project-URL: Changelog, https://github.com/spockplayer2-alt/rowcanary/blob/main/CHANGELOG.md
|
|
10
|
+
Keywords: csv,import,export,data-integrity,testing,cli
|
|
11
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
18
|
+
Classifier: Operating System :: OS Independent
|
|
19
|
+
Classifier: Topic :: Software Development :: Testing
|
|
20
|
+
Classifier: Topic :: Utilities
|
|
21
|
+
Requires-Python: >=3.10
|
|
22
|
+
Description-Content-Type: text/markdown
|
|
23
|
+
License-File: LICENSE
|
|
24
|
+
Requires-Dist: openpyxl<4,>=3.1.5
|
|
25
|
+
Requires-Dist: defusedxml<1,>=0.7.1
|
|
26
|
+
Provides-Extra: dev
|
|
27
|
+
Requires-Dist: build>=1.2; extra == "dev"
|
|
28
|
+
Requires-Dist: ruff>=0.11; extra == "dev"
|
|
29
|
+
Requires-Dist: twine>=6; extra == "dev"
|
|
30
|
+
Requires-Dist: jsonschema<5,>=4.23; extra == "dev"
|
|
31
|
+
Provides-Extra: test
|
|
32
|
+
Requires-Dist: jsonschema<5,>=4.23; extra == "test"
|
|
33
|
+
Dynamic: license-file
|
|
34
|
+
|
|
35
|
+
<div align="center">
|
|
36
|
+
<picture>
|
|
37
|
+
<source media="(prefers-color-scheme: dark)" srcset="assets/logo-dark.svg">
|
|
38
|
+
<img src="assets/logo.svg" width="156" height="132" alt="RowCanary: a canary shaped like a data table">
|
|
39
|
+
</picture>
|
|
40
|
+
<h1>RowCanary</h1>
|
|
41
|
+
<p><strong>Detect silent data changes during import/export workflows.</strong></p>
|
|
42
|
+
<p>Local Python CLI and browser UI · CSV, TSV, JSON, JSONL and XLSX · MIT</p>
|
|
43
|
+
</div>
|
|
44
|
+
|
|
45
|
+
An import can succeed while changing the data. A spreadsheet might turn
|
|
46
|
+
`00012345` into `12345`, an importer might interpret `5E12` as a number, or an
|
|
47
|
+
export might trim spaces that matter.
|
|
48
|
+
|
|
49
|
+
RowCanary creates 24 known control values. Pass them through the same workflow
|
|
50
|
+
as your data, export the result, and check which values survived. It reports
|
|
51
|
+
exact differences without trimming, numeric conversion or Unicode normalization.
|
|
52
|
+
|
|
53
|
+
## Demonstration
|
|
54
|
+
|
|
55
|
+
Run this from the source directory or after installation:
|
|
56
|
+
|
|
57
|
+
```console
|
|
58
|
+
$ python -m rowcanary demo
|
|
59
|
+
DEMO: three sample values were intentionally modified.
|
|
60
|
+
FAIL 21/24 test values preserved
|
|
61
|
+
[changed] zeroes: Leading zeroes may have been removed
|
|
62
|
+
expected: '00012345'
|
|
63
|
+
actual: '12345'
|
|
64
|
+
[changed] scientific: Identifier may have been read as scientific notation
|
|
65
|
+
expected: '5E12'
|
|
66
|
+
actual: '5000000000000'
|
|
67
|
+
[changed] formula: Formula-like text was evaluated or rewritten
|
|
68
|
+
expected: '=1+1'
|
|
69
|
+
actual: '2'
|
|
70
|
+
Scope: only the provided control samples, not an entire real dataset.
|
|
71
|
+
Use 'rowcanary make probe.csv' to test a real import/export process.
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
The demo deliberately alters three values in a temporary directory. It exits
|
|
75
|
+
successfully when the demonstration runs; this is not an observed result from
|
|
76
|
+
Excel, Google Sheets, a CRM or another third-party application.
|
|
77
|
+
|
|
78
|
+
## Installation
|
|
79
|
+
|
|
80
|
+
Requires **Python 3.10 or newer**. Install from [PyPI](https://pypi.org/project/rowcanary/):
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
pip install rowcanary
|
|
84
|
+
rowcanary --version
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
The version output is `rowcanary 1.0.0`. Use a virtual environment if your Python
|
|
88
|
+
installation requires one. If the console script is not on your `PATH`, use
|
|
89
|
+
`python -m rowcanary` with the Python interpreter that installed the package.
|
|
90
|
+
You can also run the module directly from the extracted source directory.
|
|
91
|
+
|
|
92
|
+
## Quick start
|
|
93
|
+
|
|
94
|
+
For the local browser interface, run `rowcanary serve --open`. It binds only to
|
|
95
|
+
`127.0.0.1`, uses no external resources and provides generation, upload/check,
|
|
96
|
+
light/dark themes and JSON/HTML downloads. Stop it with Ctrl+C. See the
|
|
97
|
+
[local UI guide](docs/web-ui.md) for privacy, limits and port options.
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
python -m rowcanary make probe.csv
|
|
101
|
+
python -m rowcanary check probe.csv
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
The second command checks the unmodified control file:
|
|
105
|
+
|
|
106
|
+
```text
|
|
107
|
+
PASS 24/24 test values preserved
|
|
108
|
+
No changes detected in the test values.
|
|
109
|
+
Scope: only the provided control samples, not an entire real dataset.
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
`make` refuses to overwrite an existing path. Choose a new filename for each
|
|
113
|
+
probe. The default filename, when omitted, is `rowcanary-probe.csv`.
|
|
114
|
+
|
|
115
|
+
**Spreadsheet safety:** the probes deliberately include strings beginning with
|
|
116
|
+
`=`, `+` and `-`, including `=1+1`. Spreadsheet software may evaluate them.
|
|
117
|
+
CSV quoting does not prevent formula evaluation. Use isolated test data and
|
|
118
|
+
inspect import settings before opening a probe. RowCanary itself never evaluates
|
|
119
|
+
formulas or executes input values. It does not sanitize arbitrary CSV files.
|
|
120
|
+
|
|
121
|
+
## Real-world usage
|
|
122
|
+
|
|
123
|
+
1. Generate a probe with `rowcanary make probe.csv`.
|
|
124
|
+
2. Import it into an isolated test area of the application being assessed.
|
|
125
|
+
3. Export those same records through the workflow you want to test.
|
|
126
|
+
4. Run `rowcanary check exported.csv`.
|
|
127
|
+
|
|
128
|
+
Keep `_rowcanary_id` and `_rowcanary_value` in the export. Records can be
|
|
129
|
+
reordered and additional named columns are allowed. The file should contain
|
|
130
|
+
only the probe records: other records are reported as unexpected.
|
|
131
|
+
|
|
132
|
+
If the application renames the columns:
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
rowcanary check exported.csv --id-column probe_id --value-column probe_value
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
For a semicolon-delimited workflow, explicitly choose the same delimiter for
|
|
139
|
+
generation and checking:
|
|
140
|
+
|
|
141
|
+
```bash
|
|
142
|
+
rowcanary make probe-semicolon.csv --delimiter ";"
|
|
143
|
+
rowcanary check exported.csv --delimiter ";"
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
There is no delimiter detection. A delimiter must be one UTF-8 character; tab
|
|
147
|
+
and pipe are also supported. Newline, carriage return, NUL and double quote
|
|
148
|
+
cannot be delimiters.
|
|
149
|
+
|
|
150
|
+
For JSON exports, use an array of objects with the same column names. Probe
|
|
151
|
+
values and IDs must remain **strings**; a JSON number, boolean or null is a
|
|
152
|
+
change, even if it looks similar when printed.
|
|
153
|
+
See [the complete JSON example](examples/clean.json).
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
rowcanary check exported.json --json
|
|
157
|
+
rowcanary check export.txt --format json --json
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
`--format` chooses the input parser; `--json` chooses the output report.
|
|
161
|
+
Without `--format`, a supported extension selects the parser; recognizable
|
|
162
|
+
content is used for files with unknown extensions.
|
|
163
|
+
`--delimiter` applies only to CSV.
|
|
164
|
+
|
|
165
|
+
| Exit code | Meaning |
|
|
166
|
+
| --- | --- |
|
|
167
|
+
| `0` | All controls match, or `make` / `demo` completed |
|
|
168
|
+
| `1` | The export was read, but integrity differences were found |
|
|
169
|
+
| `2` | Invalid input, command usage or an I/O error |
|
|
170
|
+
|
|
171
|
+
Normal JSON reports contain `schema_version`, `ok`, `passed`, `total`, `input`,
|
|
172
|
+
`issues` and non-failing `observations`.
|
|
173
|
+
For input or I/O errors, `check --json` writes `{"error": "..."}` to stdout and
|
|
174
|
+
exits 2. Argument-parsing errors use the usual stderr usage message. Human
|
|
175
|
+
reports escape terminal control characters and characters the output encoding
|
|
176
|
+
cannot represent. JSON escapes preserve the original Unicode values.
|
|
177
|
+
|
|
178
|
+
## Supported formats
|
|
179
|
+
|
|
180
|
+
| Operation | Format |
|
|
181
|
+
| --- | --- |
|
|
182
|
+
| Generate / check | CSV, TSV, JSON array of objects, JSONL, XLSX |
|
|
183
|
+
| Input encoding | UTF-8, UTF-16/32 with BOM; explicit `--encoding` for legacy text |
|
|
184
|
+
| Report | Human-readable text, versioned JSON or standalone HTML |
|
|
185
|
+
|
|
186
|
+
See [reports and batch checks](docs/reporting.md) for diagnostic fields, the
|
|
187
|
+
packaged JSON Schema, `--strict-order`, exclusive `--report` exports and batch
|
|
188
|
+
exit-code precedence.
|
|
189
|
+
|
|
190
|
+
The [compatibility matrix](docs/compatibility.md) separates deterministic parser
|
|
191
|
+
fixtures from actual Excel observations and unverified application workflows.
|
|
192
|
+
|
|
193
|
+
CSV quoting, embedded delimiters, escaped double quotes and embedded line
|
|
194
|
+
breaks are parsed. Line breaks **inside values** are compared exactly, including
|
|
195
|
+
the difference between LF, CR and CRLF. Record terminators may vary.
|
|
196
|
+
|
|
197
|
+
Use `--format` when the extension/content is ambiguous. XLSX uses one worksheet
|
|
198
|
+
and stores generated strings as literal text; returned formulas are refused
|
|
199
|
+
without evaluation. CSV/TSV can still be interpreted as formulas by external
|
|
200
|
+
spreadsheet applications. See [formats and custom probes](docs/formats.md) and
|
|
201
|
+
[the format and resource limits](docs/limitations.md).
|
|
202
|
+
|
|
203
|
+
## What it detects
|
|
204
|
+
|
|
205
|
+
The controls cover leading zeros, large integers, scientific notation, decimal
|
|
206
|
+
formatting, signed values, an international phone number, date/time-like text,
|
|
207
|
+
boolean/null-like strings, empty strings, spaces, tabs, newlines, punctuation,
|
|
208
|
+
Unicode, emoji and formula-like text.
|
|
209
|
+
|
|
210
|
+
It also reports missing, duplicated and unexpected records, non-text IDs and
|
|
211
|
+
missing required fields. A valid export with no records reports 24 missing
|
|
212
|
+
controls. Blank CSV records are reported rather than discarded. Duplicate CSV
|
|
213
|
+
headers, extra CSV fields, duplicate JSON object keys and nonstandard JSON
|
|
214
|
+
constants such as `NaN` are rejected as invalid input.
|
|
215
|
+
|
|
216
|
+
The preserved-value count measures matching known controls; extra or duplicate
|
|
217
|
+
rows can therefore produce a failing report showing `24/24`. Always use the
|
|
218
|
+
exit code or JSON `ok` field to decide whether the check passed.
|
|
219
|
+
|
|
220
|
+
## Limitations
|
|
221
|
+
|
|
222
|
+
A pass applies to **these controls in this specific workflow**. It does not
|
|
223
|
+
prove that an entire real dataset is intact, that formulas never ran, or that
|
|
224
|
+
an application is secure. A formula-like string being changed does not establish
|
|
225
|
+
why it changed. Intentional normalization is reported as a difference.
|
|
226
|
+
|
|
227
|
+
The application must support export or read-back and preserve the two probe
|
|
228
|
+
columns. A rejected import is inconclusive; its schema may legitimately reject
|
|
229
|
+
some controls. RowCanary has no application integrations and does not inspect
|
|
230
|
+
an importer's internal storage. See [limitations](docs/limitations.md) and
|
|
231
|
+
[security guidance](SECURITY.md).
|
|
232
|
+
|
|
233
|
+
## How it works
|
|
234
|
+
|
|
235
|
+
`make` writes the deterministic controls defined in `rowcanary/core.py`.
|
|
236
|
+
`check` reads the bounded UTF-8 export, matches records by their stable IDs and
|
|
237
|
+
compares each value as an exact string. Input order is irrelevant. JSON numeric
|
|
238
|
+
tokens retain their original spelling for diagnostics and remain numbers for
|
|
239
|
+
comparison; they are never silently converted into strings. The CLI wraps
|
|
240
|
+
these checks with text/JSON reports and exit codes.
|
|
241
|
+
|
|
242
|
+
All processing stays local. There are no network calls, accounts or telemetry.
|
|
243
|
+
XLSX support uses openpyxl and defusedxml.
|
|
244
|
+
|
|
245
|
+
## Development
|
|
246
|
+
|
|
247
|
+
From the source directory:
|
|
248
|
+
|
|
249
|
+
```bash
|
|
250
|
+
python -m pip install -e ".[dev]"
|
|
251
|
+
python -m unittest discover -s tests -v
|
|
252
|
+
python -m ruff check .
|
|
253
|
+
python -m build
|
|
254
|
+
python -m twine check dist/*
|
|
255
|
+
python tools/check_package.py dist
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
The tests use the standard library. GitHub Actions is configured for Ubuntu,
|
|
259
|
+
Windows and macOS on Python 3.10–3.14, plus package and static checks. The
|
|
260
|
+
[verification report](docs/verification.md) distinguishes executed checks from
|
|
261
|
+
pending hosted CI and third-party application tests.
|
|
262
|
+
|
|
263
|
+
The existing flat `rowcanary/` package is retained. `tests/` contains regression
|
|
264
|
+
tests, `examples/` contains clean and deliberately altered fixtures, and
|
|
265
|
+
`assets/` contains genuine vector masters and their PNG exports. See
|
|
266
|
+
[asset provenance and regeneration](assets/README.md).
|
|
267
|
+
|
|
268
|
+
## Contributing
|
|
269
|
+
|
|
270
|
+
Reproducible data-change cases and focused improvements are welcome. Read
|
|
271
|
+
[CONTRIBUTING.md](CONTRIBUTING.md). Use synthetic data in reports and never
|
|
272
|
+
include customer records or credentials. Report security issues through the
|
|
273
|
+
process in [SECURITY.md](SECURITY.md).
|
|
274
|
+
|
|
275
|
+
## License
|
|
276
|
+
|
|
277
|
+
RowCanary is distributed under the [MIT license](LICENSE). The social preview
|
|
278
|
+
uses outlined Noto Sans glyphs; its [font license](assets/FONT-LICENSE.txt) is
|
|
279
|
+
included.
|
|
280
|
+
|
|
281
|
+
Reusable JSON options, Python API and batch/Actions examples: [integration guide](docs/integrations.md).
|
|
282
|
+
|
|
283
|
+
Start here: [first control check and troubleshooting](docs/quickstart.md).
|