markback 0.1.5__tar.gz → 0.2.1__tar.gz

{markback-0.1.5 → markback-0.2.1}/.claude/settings.local.json

@@ -17,7 +17,18 @@
       "Bash(ls:*)",
       "Bash(pip install:*)",
       "Bash(PYTHONPATH=/src/markback EDITOR=cat python3:*)",
-      "Bash(PYTHONPATH=/src/markback python3:*)"
+      "Bash(PYTHONPATH=/src/markback python3:*)",
+      "Bash(python3 -m markback.cli --help)",
+      "Read(//tmp/**)",
+      "Bash(python3 -c:*)",
+      "Bash(node -e:*)",
+      "Bash(git add:*)",
+      "Bash(git commit -m ':*)",
+      "Bash(git push:*)",
+      "Bash(git commit -m ' *)",
+      "Bash(tar tzf *)",
+      "Bash(.venv/bin/python *)",
+      "Bash(bash -n /src/markback/scripts/bump-version.sh)"
     ]
   }
 }

{markback-0.1.5 → markback-0.2.1}/.gitignore

@@ -194,6 +194,9 @@ cython_debug/
 # PyPI configuration file
 .pypirc
 
+# Devbox
+.devbox/
+
 # Cursor
 #  Cursor is an AI-powered code editor. `.cursorignore` specifies files/directories to
 #  exclude from AI features like autocomplete and code analysis. Recommended for sensitive data

markback-0.2.1/AGENTS.md

@@ -0,0 +1,135 @@
+# Writing MarkBack V2 (.mb) Files
+
+MarkBack pairs content with single-line feedback using `<<<` as the delimiter.
+
+## Minimal record
+
+```
+Some content here.
+<<< positive
+```
+
+## Record with headers
+
+```
+@id item-001
+@by reviewer@example.com
+@file ./file.txt
+@input ./prompt.txt
+@tag review p1
+
+Inline content goes here.
+<<< good; quality=high
+```
+
+Headers: `@id`, `@by`, `@tag`, `@input`, `@file`. All optional. Order: id, by, tag, input, file.
+
+## Rules
+
+- `<<<` must be followed by one space then feedback text — all on one line
+- A blank line is **required** between headers and inline content
+- `@file` + inline content can coexist (file is provenance, content is snapshot)
+- Records in multi-record files are separated by `---`
+- Files must be UTF-8 with LF line endings
+- `@id` values are plain strings (no URI validation)
+
+## Compact format (one record per line)
+
+```
+@file ./images/001.jpg <<< approved; scene=beach
+@file ./images/002.jpg <<< rejected; too dark
+```
+
+No `---` separator needed between compact records. `@id` can go on the line above:
+
+```
+@id item-001
+@file ./file.txt <<< good
+```
+
+## Multi-record file
+
+```
+@id first
+
+First content.
+<<< positive
+
+---
+@id second
+
+Second content.
+<<< negative; needs work
+```
+
+## File-level headers (% prefix)
+
+```
+%markback 2
+%scope issue-A issue-B
+%covers ./gen/batch3/*.txt
+
+@file ./gen/batch3/file2.txt <<< issue-B; tone is off
+```
+
+- `%markback 2` — version declaration
+- `%scope` — what issues are being checked (sweep pattern)
+- `%covers` — glob of all files reviewed (absence = clean for scope)
+
+## Tags
+
+```
+@id item-001
+@tag training positive-examples batch-2024-03
+@file ./data/example.txt
+<<< approved
+```
+
+Space-separated. Multiple `@tag` lines merge.
+
+## Feedback format
+
+Feedback is freeform text. Optional structured convention:
+
+| Pattern | Meaning |
+|---------|---------|
+| `<<< positive` | label |
+| `<<< negative; too vague` | label + comment |
+| `<<< good; quality=high` | label + attribute |
+| `<<< quality=high; confidence=0.9` | attributes only |
+| `<<< json:{"key":"value"}` | JSON mode |
+
+Segments are separated by `; ` (semicolon + space). Segments with `=` are key-value attributes; without are labels or comments.
+
+## Sidecar files
+
+Content in `name.ext`, annotation in `name.ext.mb`:
+
+**report.pdf** — the content
+**report.pdf.mb:**
+```
+@id report-001
+<<< good; grade=B+
+```
+
+## Line/character ranges
+
+`@file` and `@input` support position references:
+
+```
+@file ./code.py:42          ← line 42
+@file ./code.py:42-50       ← lines 42–50
+@file ./code.py:10:5-15:20  ← line 10 col 5 to line 15 col 20
+```
+
+## V1 backward compatibility
+
+V1 headers (`@uri`, `@source`, `@prior`) are automatically mapped to V2 (`@id`, `@file`, `@input`) with a W010 warning.
+
+## Quick checklist
+
+- [ ] Every record has exactly one `<<<` line
+- [ ] Feedback is a single line (no newlines)
+- [ ] Blank line before inline content
+- [ ] `---` between full records; not needed between compact records
+- [ ] File ends with a newline

markback-0.2.1/IMPLEMENTATION_NOTES.md

@@ -0,0 +1,79 @@
+# Implementation Notes — MarkBack V2
+
+## V1 → V2 Changes
+
+### Header Renames
+- `@uri` → `@id` (plain string, no URI validation)
+- `@source` → `@file` (the content being annotated)
+- `@prior` → `@input` (what produced the content)
+
+### New Features
+- `@tag` header — space-separated tags for categorization
+- `%markback 2` — optional version declaration
+- `%scope` / `%covers` — sweep pattern (meaningful absence)
+- `@file` + inline content can coexist (file is provenance, content is snapshot)
+- Sidecar convention simplified to `name.ext.mb` only
+
+### Removed
+- LLM workflow layer (`llm.py`, `workflow.py`) — not core to MarkBack
+- `httpx` dependency
+- `config.py` / `python-dotenv` dependency / `--init` command — not core to MarkBack
+- RFC 3986 URI validation on `@id`
+- E003 (malformed URI) no longer emitted
+- E005 (content with @source) no longer emitted — coexistence is valid
+- `.label.txt` / `.feedback.txt` as primary sidecar convention (kept as V1 legacy in discovery)
+
+### API Changes
+- `SourceRef` → `FileRef` (alias preserved)
+- `Record` fields: `uri→id`, `source→file`, `prior→input`, added `tags`
+- New `write()` / `append()` / `write_string()` functions
+- New `normalize()` function
+- New `discover_sidecars()` function
+- `ParseResult` gains `scope`, `covers`, `version`, `covered_files()`
+- V1 compat aliases preserved for all renamed functions
+
+## Design Decisions
+
+### Parser Architecture
+Same state-machine approach as V1. Now also handles:
+- File-level `%` headers (parsed before records, must be at top of file)
+- V1 header mapping (detected by keyword, mapped with W010 warning)
+- `@tag` with whitespace splitting and merge across multiple lines
+
+### V1 Backward Compatibility
+The parser transparently reads V1 files:
+1. `@uri` → mapped to `record.id`
+2. `@source` → mapped to `record.file`
+3. `@prior` → mapped to `record.input`
+4. `@source ... <<<` compact → handled alongside `@file ... <<<`
+5. Each V1 header emits W010 warning
+
+### Sweep Pattern
+`%scope` + `%covers` enable "meaningful absence":
+- `%scope` declares what issues are being checked
+- `%covers` declares the complete file set under review
+- Files matching `%covers` with no record are implicitly clean for all scope items
+- `ParseResult.covered_files()` resolves the glob for programmatic access
+
+### Writer Simplification
+V1 had 5+ writer functions. V2 has:
+- `write()` — write records to a file (auto-format)
+- `append()` — add a record to existing file
+- `write_string()` — write records to string
+- `normalize()` — canonical rewrite
+All V1 functions preserved as aliases.
+
+### Sidecar Convention
+V2: `name.ext.mb` (append `.mb` to the full filename)
+V1 legacy: `.label.txt`, `.feedback.txt` still discovered for backward compat
+
+## Testing Strategy
+
+### Unit Tests
+- Parser tests cover V2 format, V1 backward compat, file-level headers, tags, sweep pattern
+- Writer tests cover canonical output, version headers, scope/covers, round-trip
+- Linter tests verify all error/warning codes with V2 semantics
+- Type tests verify FileRef, Record with new fields, V1 compat aliases
+
+### Fixtures
+All fixtures updated to V2 format. Error fixtures updated to reflect V2 semantics (e.g., content_with_source is now valid).

markback-0.2.1/PKG-INFO

@@ -0,0 +1,240 @@
+Metadata-Version: 2.4
+Name: markback
+Version: 0.2.1
+Summary: A compact, human-writable format for storing content paired with feedback/labels
+Project-URL: Homepage, https://github.com/dandriscoll/markback
+Project-URL: Repository, https://github.com/dandriscoll/markback
+Project-URL: Documentation, https://github.com/dandriscoll/markback#readme
+Project-URL: Issues, https://github.com/dandriscoll/markback/issues
+Author: Dan Driscoll
+License-Expression: MIT
+License-File: LICENSE
+Keywords: annotation,data-labeling,feedback,labeling,llm,markdown
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Text Processing :: Markup
+Requires-Python: >=3.10
+Requires-Dist: rich>=13.0.0
+Requires-Dist: typer>=0.9.0
+Provides-Extra: dev
+Requires-Dist: build>=1.0.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
+Requires-Dist: pytest>=7.0.0; extra == 'dev'
+Requires-Dist: twine>=5.0.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# MarkBack V2
+
+A compact, human-writable format for storing content paired with feedback/labels.
+
+## Installation
+
+```bash
+pip install -e .
+```
+
+## Quick Start
+
+### Parse a MarkBack file
+
+```python
+from markback import parse_file, parse_string
+
+# Parse a file
+result = parse_file("labels.mb")
+for record in result.records:
+    print(f"{record.id}: {record.feedback}")
+
+# Parse a string
+text = """
+@id example
+
+Some content here.
+<<< positive; good quality
+"""
+result = parse_string(text)
+```
+
+### Write MarkBack files
+
+```python
+from markback import Record, FileRef, write, append
+
+# Write records to a file
+records = [
+    Record(feedback="good", id="item-1", content="First item"),
+    Record(feedback="bad", id="item-2", content="Second item"),
+]
+write("output.mb", records)
+
+# Append a single record
+append("output.mb", Record(feedback="great", id="item-3", content="Third"))
+```
+
+### Lint files
+
+```python
+from markback import lint_file
+
+result = lint_file("myfile.mb")
+if result.has_errors:
+    for d in result.diagnostics:
+        print(d)
+```
+
+## CLI Usage
+
+The CLI is available via `markback` or `mb` (shorthand).
+
+### Annotate files
+
+```bash
+# Single file — inline feedback, appends to myfile.txt.mb
+mb myfile.txt "good; clear writing"
+
+# URL target — derives sidecar from last path segment (or hostname)
+mb https://example.com/blog/post.html "great explanation"
+# → writes post.html.mb with @file https://example.com/blog/post.html
+
+# Quote a passage by editing the .mb file directly: inline content
+# under an @file header can be a full snapshot OR an excerpt.
+#   @file https://example.com/post.html
+#
+#   the quick brown fox jumps over the lazy dog
+#   <<< awkward phrasing
+
+# Multi-segment section: several comments on one source, no repeated headers.
+#   @file ./essay.txt
+#
+#   the lazy fox
+#   <<< awkward
+#
+#   weak ending
+#   <<< needs punch
+
+# With input reference (what produced the file)
+mb output.txt "accurate" --input prompt.txt
+
+# With tags and attribution
+mb file.txt "good" --tag "review p1" --by alice@example.com
+
+# Multiple files — same feedback for all
+mb *.jpg -f "approved"
+
+# Interactive mode — steps through each file
+mb *.jpg --print
+
+# Sweep pattern — track issues across batches
+mb *.txt -f "issue-A" --scope "issue-A issue-B" --covers "./*.txt"
+```
+
+### Utility commands
+
+```bash
+# Lint
+mb --lint myfile.mb
+mb --lint --json ./data/
+
+# List records
+mb --list myfile.mb
+
+# Statistics
+mb --stats myfile.mb
+
+# Normalize to canonical format
+mb --normalize input.mb
+mb --normalize --in-place input.mb
+
+# Convert between formats
+mb --convert --to multi -o output.mb input.mb
+mb --convert --to compact -o output.mb input.mb
+
+# Upgrade V1 files to V2
+mb --upgrade *.mb              # preview
+mb --upgrade --apply --in-place *.mb  # apply
+```
+
+## File Format
+
+### V2 Headers
+
+| Header | Purpose |
+|--------|---------|
+| `@id` | Record identifier (plain string) |
+| `@by` | Who provided feedback |
+| `@tag` | Space-separated tags |
+| `@input` | What produced the content (e.g., a prompt) |
+| `@file` | Path to the content being annotated |
+
+### File-level headers (% prefix)
+
+```
+%markback 2
+%scope issue-A issue-B
+%covers ./gen/batch3/*.txt
+```
+
+### Record examples
+
+```
+@id review-001
+@by alice@company.com
+@file ./src/auth.py:45-67
+@tag security p0
+
+<<< vulnerable; sql-injection in query builder
+```
+
+### Compact label list
+
+```
+@file ./images/001.jpg <<< approved; scene=beach
+@file ./images/002.jpg <<< rejected; too dark
+```
+
+### Sidecar files
+
+Content in `report.pdf`, annotation in `report.pdf.mb`:
+
+```
+@id report-001
+<<< good; grade=B+
+```
+
+### Sweep pattern
+
+Track issues across batches with meaningful absence:
+
+```
+%markback 2
+%scope issue-A issue-B
+%covers ./gen/batch3/*.txt
+
+@file ./gen/batch3/file2.txt <<< issue-B; tone is off
+@file ./gen/batch3/file5.txt <<< issue-A; issue-B; both problems
+```
+
+Files matching `%covers` without annotations are implicitly clean for all `%scope` items.
+
+## V1 Backward Compatibility
+
+V1 headers (`@uri`, `@source`, `@prior`) are automatically mapped to V2 equivalents with a W010 warning. The V2 parser reads V1 files transparently.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+## License
+
+MIT

markback-0.2.1/README.md

@@ -0,0 +1,207 @@
+# MarkBack V2
+
+A compact, human-writable format for storing content paired with feedback/labels.
+
+## Installation
+
+```bash
+pip install -e .
+```
+
+## Quick Start
+
+### Parse a MarkBack file
+
+```python
+from markback import parse_file, parse_string
+
+# Parse a file
+result = parse_file("labels.mb")
+for record in result.records:
+    print(f"{record.id}: {record.feedback}")
+
+# Parse a string
+text = """
+@id example
+
+Some content here.
+<<< positive; good quality
+"""
+result = parse_string(text)
+```
+
+### Write MarkBack files
+
+```python
+from markback import Record, FileRef, write, append
+
+# Write records to a file
+records = [
+    Record(feedback="good", id="item-1", content="First item"),
+    Record(feedback="bad", id="item-2", content="Second item"),
+]
+write("output.mb", records)
+
+# Append a single record
+append("output.mb", Record(feedback="great", id="item-3", content="Third"))
+```
+
+### Lint files
+
+```python
+from markback import lint_file
+
+result = lint_file("myfile.mb")
+if result.has_errors:
+    for d in result.diagnostics:
+        print(d)
+```
+
+## CLI Usage
+
+The CLI is available via `markback` or `mb` (shorthand).
+
+### Annotate files
+
+```bash
+# Single file — inline feedback, appends to myfile.txt.mb
+mb myfile.txt "good; clear writing"
+
+# URL target — derives sidecar from last path segment (or hostname)
+mb https://example.com/blog/post.html "great explanation"
+# → writes post.html.mb with @file https://example.com/blog/post.html
+
+# Quote a passage by editing the .mb file directly: inline content
+# under an @file header can be a full snapshot OR an excerpt.
+#   @file https://example.com/post.html
+#
+#   the quick brown fox jumps over the lazy dog
+#   <<< awkward phrasing
+
+# Multi-segment section: several comments on one source, no repeated headers.
+#   @file ./essay.txt
+#
+#   the lazy fox
+#   <<< awkward
+#
+#   weak ending
+#   <<< needs punch
+
+# With input reference (what produced the file)
+mb output.txt "accurate" --input prompt.txt
+
+# With tags and attribution
+mb file.txt "good" --tag "review p1" --by alice@example.com
+
+# Multiple files — same feedback for all
+mb *.jpg -f "approved"
+
+# Interactive mode — steps through each file
+mb *.jpg --print
+
+# Sweep pattern — track issues across batches
+mb *.txt -f "issue-A" --scope "issue-A issue-B" --covers "./*.txt"
+```
+
+### Utility commands
+
+```bash
+# Lint
+mb --lint myfile.mb
+mb --lint --json ./data/
+
+# List records
+mb --list myfile.mb
+
+# Statistics
+mb --stats myfile.mb
+
+# Normalize to canonical format
+mb --normalize input.mb
+mb --normalize --in-place input.mb
+
+# Convert between formats
+mb --convert --to multi -o output.mb input.mb
+mb --convert --to compact -o output.mb input.mb
+
+# Upgrade V1 files to V2
+mb --upgrade *.mb              # preview
+mb --upgrade --apply --in-place *.mb  # apply
+```
+
+## File Format
+
+### V2 Headers
+
+| Header | Purpose |
+|--------|---------|
+| `@id` | Record identifier (plain string) |
+| `@by` | Who provided feedback |
+| `@tag` | Space-separated tags |
+| `@input` | What produced the content (e.g., a prompt) |
+| `@file` | Path to the content being annotated |
+
+### File-level headers (% prefix)
+
+```
+%markback 2
+%scope issue-A issue-B
+%covers ./gen/batch3/*.txt
+```
+
+### Record examples
+
+```
+@id review-001
+@by alice@company.com
+@file ./src/auth.py:45-67
+@tag security p0
+
+<<< vulnerable; sql-injection in query builder
+```
+
+### Compact label list
+
+```
+@file ./images/001.jpg <<< approved; scene=beach
+@file ./images/002.jpg <<< rejected; too dark
+```
+
+### Sidecar files
+
+Content in `report.pdf`, annotation in `report.pdf.mb`:
+
+```
+@id report-001
+<<< good; grade=B+
+```
+
+### Sweep pattern
+
+Track issues across batches with meaningful absence:
+
+```
+%markback 2
+%scope issue-A issue-B
+%covers ./gen/batch3/*.txt
+
+@file ./gen/batch3/file2.txt <<< issue-B; tone is off
+@file ./gen/batch3/file5.txt <<< issue-A; issue-B; both problems
+```
+
+Files matching `%covers` without annotations are implicitly clean for all `%scope` items.
+
+## V1 Backward Compatibility
+
+V1 headers (`@uri`, `@source`, `@prior`) are automatically mapped to V2 equivalents with a W010 warning. The V2 parser reads V1 files transparently.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+## License
+
+MIT