snf-peirce 0.1.0__tar.gz

snf_peirce-0.1.0/LICENSE

@@ -0,0 +1,28 @@
+MIT License
+
+Copyright (c) 2026 peirce-lang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---
+
+SNF (Semantic Normalized Form) specification and Peirce Query Language
+specification are original works. Attribution in source code and documentation
+is required. See project licensing documentation for details on Portolan and
+Reckoner licensing.

snf_peirce-0.1.0/PKG-INFO

@@ -0,0 +1,374 @@
+Metadata-Version: 2.4
+Name: snf-peirce
+Version: 0.1.0
+Summary: Python implementation of the SNF stack — Peirce parser, lens authoring, substrate compilation, and query execution
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/peirce-lang/snf-peirce
+Project-URL: Repository, https://github.com/peirce-lang/snf-peirce
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pandas>=1.5
+Requires-Dist: duckdb>=0.9
+Provides-Extra: roaring
+Requires-Dist: pyroaring>=0.4; extra == "roaring"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: jupyter; extra == "dev"
+Dynamic: license-file
+
+# snf-peirce
+
+Python implementation of the SNF (Semantic Normalized Form) stack.
+
+Query any dataset by meaning, not by schema. Author a lens once, query semantically forever.
+
+```python
+from snf_peirce import suggest, compile_data, query
+import pandas as pd
+
+df       = pd.read_csv("my_collection.csv")
+draft    = suggest(df)
+draft.map("Artist", "who", "artist").nucleus("release_id", prefix="discogs:release")
+lens     = draft.to_lens(lens_id="discogs_v1", authority="me")
+compiled = compile_data(df, lens)
+
+query(compiled, 'WHO.artist = "Miles Davis" AND WHEN.released = "1959"')
+```
+
+---
+
+## What is SNF?
+
+SNF (Semantic Normalized Form) is a data model and query protocol built around six universal dimensions:
+
+| Dimension | Meaning | Examples |
+|---|---|---|
+| **WHO** | People, organisations, roles | author, publisher, attorney, artist |
+| **WHAT** | Things, topics, identifiers | title, subject, ISBN, genre |
+| **WHEN** | Dates, years, time periods | publication_date, year, date_added |
+| **WHERE** | Places, locations, regions | publication_place, office, territory |
+| **WHY** | Reasons, types, purposes | matter_type, audience, format_legal |
+| **HOW** | Methods, formats, measurements | carrier_type, cmc, media_type |
+
+Every fact in your data maps to one of these dimensions. Once mapped, you query by meaning — not by column name, table structure, or JOIN logic.
+
+**Peirce** is the query language for SNF. Named after Charles Sanders Peirce, whose triadic sign relation maps directly onto the SNF record structure: Dimension → SemanticKey → Value.
+
+---
+
+## Install
+
+```bash
+pip install snf-peirce
+```
+
+Or clone and use directly:
+
+```bash
+git clone https://github.com/peirce-lang/snf-peirce
+cd snf-peirce
+pip install pandas duckdb requests
+```
+
+---
+
+## Quick start
+
+### From a CSV
+
+```python
+import pandas as pd
+from snf_peirce import suggest, compile_data, query
+
+df    = pd.read_csv("matters.csv")
+draft = suggest(df)
+print(draft)   # renders as a table in Jupyter
+
+draft.map("attorney_name", "who",   "attorney")
+draft.map("matter_type",   "why",   "matter_type")
+draft.map("fiscal_year",   "when",  "year")
+draft.map("office",        "where", "office")
+draft.nucleus_composite(["client_id", "matter_id"],
+                         separator="-", prefix="legal:matter")
+
+lens     = draft.to_lens(lens_id="legal_v1", authority="firm")
+compiled = compile_data(df, lens)
+
+query(compiled, 'WHO.attorney = "Smith" AND WHERE.office = "Seattle"')
+query(compiled, 'WHEN.year BETWEEN "2022" AND "2024"')
+query(compiled, 'WHO.attorney = "Smith" OR WHO.attorney = "Jones"')
+```
+
+### From an existing lens-tool lens
+
+Lenses created by the JavaScript lens-tool are directly compatible:
+
+```python
+from snf_peirce import load, compile_data, query
+import pandas as pd
+
+lens     = load("discogs_community_v1.json")
+df       = pd.read_csv("discogs_sample.csv")
+compiled = compile_data(df, lens)
+
+query(compiled, 'WHO.author = "Miles Davis"')
+query(compiled, 'WHEN.publication_date BETWEEN "1955" AND "1965"')
+```
+
+---
+
+## Interactive shell
+
+```bash
+python shell.py csv://my_spoke_dir
+```
+
+```
+peirce> WHO.author = "Miles Davis"
+peirce> WHEN.publication_date BETWEEN "1955" AND "1965"
+peirce> WHO.author = "Miles Davis" AND WHEN.publication_date = "1959"
+peirce> \schema             — show all dimensions and fields
+peirce> \schema WHO         — show fields in WHO with counts
+peirce> \explain            — show execution plan for last query
+peirce> WHO.<TAB>           — TAB-completes field names from substrate
+peirce> \pivot              — toggle wide table view
+peirce> exit
+```
+
+Shell features beyond the JS version:
+- **TAB completion** — field names from the actual substrate
+- **`\explain`** — execution plan with cardinality bars
+- **`\schema`** — dimensions, fields, entity and value counts
+- **Discovery expressions** — `WHO|*`, `WHAT|genre|*` work inline
+
+---
+
+## Guided setup (no coding required)
+
+```bash
+python guided_ingest.py
+python guided_ingest.py mydata.csv
+```
+
+Walks through CSV → lens authoring → compilation → shell with prompts. No code required.
+
+---
+
+## Fetch from public APIs
+
+### Scryfall (Magic: The Gathering)
+
+```bash
+pip install requests
+python fetch_scryfall.py              # Guilds of Ravnica (default)
+python fetch_scryfall.py war          # War of the Spark
+python fetch_scryfall.py --list       # see available sets
+```
+
+```
+peirce> WHAT.guild = "Dimir"
+peirce> WHAT.color = "Blue" AND WHAT.color = "Black"
+peirce> WHAT.card_type = "Creature" AND HOW.cmc BETWEEN "1" AND "3"
+peirce> WHAT.keyword = "Surveil"
+peirce> WHO.artist = "Seb McKinnon"
+peirce> WHAT|guild|*
+```
+
+### Library of Congress catalog
+
+```bash
+python fetch_loc.py                          # default: jazz music
+python fetch_loc.py "toni morrison"          # keyword search
+python fetch_loc.py --subject "cooking"      # subject search
+python fetch_loc.py --author "hemingway"     # author search
+python fetch_loc.py --marc-file catalog.mrc  # from a .mrc file
+```
+
+```
+peirce> WHO.author CONTAINS "Morrison"
+peirce> WHAT.subject_topic CONTAINS "Jazz"
+peirce> WHEN.publication_date BETWEEN "1950" AND "1970"
+peirce> WHERE.publication_place = "New York"
+peirce> WHAT|subject_topic|*
+```
+
+### Build your own fetcher
+
+```python
+from base_fetcher import SNFFetcher, fact, facts, facts_from_list
+
+class MyAPIFetcher(SNFFetcher):
+    lens_id   = "myapi_v1"
+    set_name  = "My Dataset"
+    spoke_dir = "myapi_spoke"
+
+    def fetch(self):
+        import requests
+        return requests.get("https://api.example.com/data").json()["items"]
+
+    def entity_id(self, item):
+        return f"myapi:{item['id']}"
+
+    def translate(self, item):
+        eid = self.entity_id(item)
+        return [
+            *facts(
+                (eid, "what", "title",  item.get("title")),
+                (eid, "who",  "author", item.get("author")),
+                (eid, "when", "year",   item.get("year")),
+            ),
+            *facts_from_list(eid, "what", "genre", item.get("genres", [])),
+        ]
+
+if __name__ == "__main__":
+    MyAPIFetcher().run()
+```
+
+---
+
+## Peirce query syntax
+
+```
+WHO.artist = "Miles Davis"                         equality
+WHO.artist != "Miles Davis"                        not equal
+WHEN.released > "1960"                             comparison
+WHEN.released BETWEEN "1955" AND "1965"            range (inclusive)
+WHAT.title CONTAINS "Blue"                         substring
+WHO.artist PREFIX "Miles"                          starts with
+NOT WHERE.office = "Seattle"                       negation
+WHO.artist = "Miles Davis" AND WHEN.released = "1959"    AND (intersection)
+WHO.artist = "Miles Davis" OR WHO.artist = "Coltrane"    OR (union)
+(WHO.artist = "Miles Davis" AND WHEN.released = "1959")
+OR
+(WHO.artist = "John Coltrane" AND WHEN.released = "1964")  DNF
+
+# Discovery (shell only — shows schema)
+*                 all dimensions
+WHO|*             all fields in WHO
+WHO|artist|*      all values for WHO.artist
+```
+
+---
+
+## MARC support
+
+snf-peirce ships with the MARC Bibliographic Lens v1.0 — a complete
+field mapping from MARC21 tags to SNF dimensions. Python port of
+`MARCTranslator_v3.js`. No extra dependencies required.
+
+```python
+from parse_marc import parse_mrc
+from marc_translator import MARCTranslator
+
+records    = parse_mrc("catalog.mrc")
+translator = MARCTranslator(source_id="loc")
+
+for record in records:
+    facts = translator.translate_record(record)
+```
+
+Key field mappings:
+
+| MARC tag | → | Dimension | Semantic key |
+|---|---|---|---|
+| 100$a | → | WHO | author |
+| 245$a+$b | → | WHAT | title |
+| 260$b / 264$b | → | WHO | publisher |
+| 260$a / 264$a | → | WHERE | publication_place |
+| 260$c / 264$c | → | WHEN | publication_date |
+| 650$a | → | WHAT | subject_topic |
+| 651$a | → | WHERE | subject_place |
+| 600$a | → | WHO | subject_person |
+| 655$a | → | WHAT | genre |
+| 020$a | → | WHAT | isbn (nucleus) |
+
+---
+
+## Jupyter workflow
+
+Results render as tables inline. Output is pandas.
+
+```python
+import pandas as pd
+from snf_peirce import suggest, compile_data, query
+
+df       = pd.read_csv("my_collection.csv")
+draft    = suggest(df)             # renders as mapping table
+compiled = compile_data(df, lens)  # renders substrate summary
+
+query(compiled, 'WHO.artist = "Miles Davis"')  # renders result table inline
+
+result   = query(compiled, 'WHEN.released BETWEEN "1955" AND "1965"', limit=None)
+df_result = result.to_dataframe()   # pandas DataFrame — use anything
+df_result.groupby("semantic_key")["value"].value_counts()
+```
+
+---
+
+## File inventory
+
+| File | Purpose |
+|---|---|
+| `parser.py` | Peirce query parser — conformant with JS reference |
+| `lens.py` | Lens authoring: `suggest()`, `LensDraft`, `load()`, `save()` |
+| `compile.py` | Data compilation: `compile_data()`, `Substrate` |
+| `peirce.py` | Query execution: `query()`, `execute()`, `ResultSet` |
+| `shell.py` | Interactive query shell |
+| `guided_ingest.py` | Guided setup script — no coding required |
+| `base_fetcher.py` | Base class for API fetchers |
+| `fetch_scryfall.py` | Scryfall / Magic: The Gathering fetcher |
+| `fetch_loc.py` | Library of Congress catalog fetcher |
+| `marc_translator.py` | MARC Bibliographic Lens v1.0 |
+| `parse_marc.py` | Pure Python binary MARC / MARCXML parser |
+
+---
+
+## Running the tests
+
+```bash
+python -m pytest test_parser.py -v    # 59 tests — parser conformance
+python test_lens.py                    # 69 tests — lens authoring
+python test_compile.py                 # 57 tests — compilation and queries
+python test_peirce.py                  # 57 tests — end-to-end query
+python test_conformance.py             # 37 tests — cross-language proof
+```
+
+---
+
+## Architecture
+
+```
+SNF / Peirce specification   open protocol (MIT)
+           ↓
+     snf-peirce              Python runtime / engine  ← this package
+           ↓
+       Reckoner               visual application for non-technical users
+```
+
+snf-peirce is Reckoner's data engine. It is also usable standalone
+for data practitioners who want SNF in Python and Jupyter workflows.
+
+---
+
+## Note on Portolan
+
+The `\explain` shell command displays a simplified execution plan
+showing constraints ordered by estimated cardinality. This implements
+the same ordering heuristic as Portolan's I1 algorithm for display
+purposes.
+
+Full Portolan — schema validation, type checking, query rejection,
+composite constraint reasoning — is a separate licensed component
+not included in this package.
+
+---
+
+## License
+
+MIT. See LICENSE file.
+
+SNF specification, Peirce query language, and MARC Bibliographic
+Lens v1.0 are original works. Attribution required in source code
+and documentation. See project licensing documentation for details
+on Portolan and Reckoner licensing.