opentaxonomy 0.1.0__tar.gz

opentaxonomy-0.1.0/.gitignore

@@ -0,0 +1,34 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+*.egg
+dist/
+build/
+.eggs/
+
+# Virtual environments
+.venv/
+venv/
+env/
+
+# Distribution
+dist/
+
+# Environment
+.env
+.env.*
+
+# IDE
+.vscode/
+.idea/
+
+# Taxonomy output (generated data — not source)
+taxonomy/
+
+# Claude Code
+.claude/
+
+# OS
+.DS_Store
+Thumbs.db

opentaxonomy-0.1.0/PKG-INFO

@@ -0,0 +1,130 @@
+Metadata-Version: 2.4
+Name: opentaxonomy
+Version: 0.1.0
+Summary: LLM-powered semantic taxonomy generator for raw categorical data
+License: MIT
+Requires-Python: >=3.11
+Requires-Dist: anthropic>=0.40.0
+Requires-Dist: click>=8.1.0
+Requires-Dist: openpyxl>=3.1.0
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: pyarrow>=14.0.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: sqlalchemy>=2.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# OpenTaxonomy
+
+LLM-powered semantic taxonomy generator for raw categorical data.
+
+## What it does
+
+OpenTaxonomy takes a column of messy raw values (bank transactions, product names, survey responses — anything) and generates a structured semantic taxonomy from it using Claude as the reasoning engine.
+
+**Output for each value:**
+- `{column}_normalized` — cleaned entity name (e.g. `"REWE SAGT DANKE 46654184/..."` → `"REWE"`)
+- `canonical_id` — taxonomy path (e.g. `ft.expenditure.variable.groceries`)
+
+The taxonomy itself is a set of YAML files — a `seed.yaml` capturing the domain structure, one `node.yaml` per tree node (each an ontological contract with inclusion/exclusion criteria and a decision record), and a `placement_map.yaml` that is the source of truth for all mappings.
+
+## Architecture
+
+The core is the **Prima Seed** — a universal questioning protocol that generates domain-specific taxonomic trees from any categorical data:
+
+- **Q0** Identify the Form: what unifies all values?
+- **Q0b** Establish context: what operational realm governs classification?
+- **Q1** Primary differentiation: the most essential splitting criterion
+- **Q2** Recursive differentiation: applied per branch at each level
+- **Q3** Dialectical check: values that resist placement expose flawed criteria
+
+## Installation
+
+```bash
+pip install opentaxonomy
+```
+
+Requires Python 3.11+ and an [Anthropic API key](https://console.anthropic.com/).
+
+## Usage
+
+```bash
+export ANTHROPIC_API_KEY=sk-ant-...
+
+# Generate a new taxonomy from raw data
+opentaxonomy create -i transactions.csv -c description -o ./taxonomy
+
+# Place new/unseen values into an existing taxonomy
+opentaxonomy run -i new_data.csv -c description -s ./taxonomy
+```
+
+### Supported input formats
+
+| Format | Example |
+|--------|---------|
+| CSV / TSV | `data.csv`, `data.tsv` |
+| JSON | `data.json` |
+| Excel | `data.xlsx` |
+| Parquet | `data.parquet` |
+| Database | `postgresql://user:pass@host/db` + `--db-table` |
+
+### Options
+
+```
+opentaxonomy create
+  -i, --input        Input file or database connection string  [required]
+  -c, --column       Column containing raw values to classify  [required]
+  -o, --output-dir   Where to write taxonomy files  [default: ./taxonomy]
+  --domain-hint      Optional hint to guide the LLM (e.g. "German grocery products")
+  --model            Claude model  [default: claude-sonnet-4-6]
+  --api-key          Anthropic API key (or set ANTHROPIC_API_KEY)
+
+opentaxonomy run
+  -i, --input        Input file or database connection string  [required]
+  -c, --column       Column containing raw values to classify  [required]
+  -s, --seed-dir     Directory with seed.yaml and placement_map.yaml  [default: ./taxonomy]
+  --model            Claude model  [default: claude-sonnet-4-6]
+  --api-key          Anthropic API key (or set ANTHROPIC_API_KEY)
+```
+
+## Output structure
+
+```
+taxonomy/
+├── seed.yaml                  # Domain seed: context, levels, edge cases
+├── placement_map.yaml         # Raw values → canonical IDs (source of truth)
+└── nodes/
+    ├── root.yaml              # Root node
+    ├── expenditure.yaml       # Internal node with decision record
+    ├── groceries.yaml         # Leaf node with inclusion/exclusion criteria
+    └── ...
+```
+
+Each node file is an ontological contract:
+
+```yaml
+node: Groceries
+canonical_id: ft.expenditure.variable.groceries
+question: Is this a purchase of food or household staples from a supermarket or grocery store?
+criteria:
+  includes:
+    - Supermarket purchases (REWE, LIDL, EDEKA, etc.)
+    - Organic/bio market purchases
+  excludes:
+    - Restaurant meals
+    - Drugstore purchases unless food items
+edge_cases:
+  - term: REWE TO GO
+    resolution: Included — still a grocery/convenience purchase
+    decided: true
+parent: Variable Necessities
+children: []
+version: 1.0.0
+```
+
+## License
+
+MIT

opentaxonomy-0.1.0/README.md

@@ -0,0 +1,111 @@
+# OpenTaxonomy
+
+LLM-powered semantic taxonomy generator for raw categorical data.
+
+## What it does
+
+OpenTaxonomy takes a column of messy raw values (bank transactions, product names, survey responses — anything) and generates a structured semantic taxonomy from it using Claude as the reasoning engine.
+
+**Output for each value:**
+- `{column}_normalized` — cleaned entity name (e.g. `"REWE SAGT DANKE 46654184/..."` → `"REWE"`)
+- `canonical_id` — taxonomy path (e.g. `ft.expenditure.variable.groceries`)
+
+The taxonomy itself is a set of YAML files — a `seed.yaml` capturing the domain structure, one `node.yaml` per tree node (each an ontological contract with inclusion/exclusion criteria and a decision record), and a `placement_map.yaml` that is the source of truth for all mappings.
+
+## Architecture
+
+The core is the **Prima Seed** — a universal questioning protocol that generates domain-specific taxonomic trees from any categorical data:
+
+- **Q0** Identify the Form: what unifies all values?
+- **Q0b** Establish context: what operational realm governs classification?
+- **Q1** Primary differentiation: the most essential splitting criterion
+- **Q2** Recursive differentiation: applied per branch at each level
+- **Q3** Dialectical check: values that resist placement expose flawed criteria
+
+## Installation
+
+```bash
+pip install opentaxonomy
+```
+
+Requires Python 3.11+ and an [Anthropic API key](https://console.anthropic.com/).
+
+## Usage
+
+```bash
+export ANTHROPIC_API_KEY=sk-ant-...
+
+# Generate a new taxonomy from raw data
+opentaxonomy create -i transactions.csv -c description -o ./taxonomy
+
+# Place new/unseen values into an existing taxonomy
+opentaxonomy run -i new_data.csv -c description -s ./taxonomy
+```
+
+### Supported input formats
+
+| Format | Example |
+|--------|---------|
+| CSV / TSV | `data.csv`, `data.tsv` |
+| JSON | `data.json` |
+| Excel | `data.xlsx` |
+| Parquet | `data.parquet` |
+| Database | `postgresql://user:pass@host/db` + `--db-table` |
+
+### Options
+
+```
+opentaxonomy create
+  -i, --input        Input file or database connection string  [required]
+  -c, --column       Column containing raw values to classify  [required]
+  -o, --output-dir   Where to write taxonomy files  [default: ./taxonomy]
+  --domain-hint      Optional hint to guide the LLM (e.g. "German grocery products")
+  --model            Claude model  [default: claude-sonnet-4-6]
+  --api-key          Anthropic API key (or set ANTHROPIC_API_KEY)
+
+opentaxonomy run
+  -i, --input        Input file or database connection string  [required]
+  -c, --column       Column containing raw values to classify  [required]
+  -s, --seed-dir     Directory with seed.yaml and placement_map.yaml  [default: ./taxonomy]
+  --model            Claude model  [default: claude-sonnet-4-6]
+  --api-key          Anthropic API key (or set ANTHROPIC_API_KEY)
+```
+
+## Output structure
+
+```
+taxonomy/
+├── seed.yaml                  # Domain seed: context, levels, edge cases
+├── placement_map.yaml         # Raw values → canonical IDs (source of truth)
+└── nodes/
+    ├── root.yaml              # Root node
+    ├── expenditure.yaml       # Internal node with decision record
+    ├── groceries.yaml         # Leaf node with inclusion/exclusion criteria
+    └── ...
+```
+
+Each node file is an ontological contract:
+
+```yaml
+node: Groceries
+canonical_id: ft.expenditure.variable.groceries
+question: Is this a purchase of food or household staples from a supermarket or grocery store?
+criteria:
+  includes:
+    - Supermarket purchases (REWE, LIDL, EDEKA, etc.)
+    - Organic/bio market purchases
+  excludes:
+    - Restaurant meals
+    - Drugstore purchases unless food items
+edge_cases:
+  - term: REWE TO GO
+    resolution: Included — still a grocery/convenience purchase
+    decided: true
+parent: Variable Necessities
+children: []
+version: 1.0.0
+```
+
+## License
+
+MIT

opentaxonomy-0.1.0/files/README.md

@@ -0,0 +1,66 @@
+# OpenTaxonomy v0.1.0 — First Output
+
+## What is this?
+
+This is the first concrete output of the OpenTaxonomy project — a proof-of-concept
+demonstrating the Prima Seed protocol applied to real-world data.
+
+## Structure
+
+```
+opentaxonomy/
+├── README.md                              ← You are here
+└── seeds/
+    ├── prima/
+    │   └── prima-seed.yaml                ← The universal questioning protocol
+    └── personal-finance-transactions/
+        ├── seed.yaml                      ← Domain seed (generated by Prima Seed)
+        ├── placement_map.yaml             ← Raw values → canonical IDs
+        └── nodes/
+            ├── root.yaml                  ← Financial Transaction (root)
+            ├── income.yaml                ← Income
+            ├── employment.yaml            ← Employment Income (leaf)
+            ├── government_benefits.yaml   ← Government Benefits (leaf)
+            ├── expenditure.yaml           ← Expenditure
+            ├── fixed_obligations.yaml     ← Fixed Obligations
+            ├── variable_necessities.yaml  ← Variable Necessities
+            ├── discretionary.yaml         ← Discretionary
+            ├── groceries.yaml             ← Groceries (leaf)
+            ├── dining_out.yaml            ← Dining Out (leaf)
+            └── digital_subscriptions.yaml ← Digital Subscriptions (leaf)
+```
+
+## Key Concepts
+
+- **Prima Seed**: The universal meta-protocol. Domain-agnostic questions that
+  generate domain-specific trees from any categorical data.
+
+- **Domain Seed**: A specific questioning protocol for a domain (e.g., personal
+  finance transactions). Generated by running the Prima Seed against real data.
+  Reusable by anyone with similar data.
+
+- **Node**: An ontological contract — a YAML file carrying inclusion criteria,
+  exclusion criteria, edge cases, and decision records. Each node is one file.
+
+- **Canonical ID**: A deterministic identifier derived from the criteria path
+  (e.g., `ft.expenditure.variable.groceries`). Enables semantic joins.
+
+- **Placement Map**: The mapping of raw data values to canonical IDs. This is
+  where meaning is assigned to data.
+
+## How to Read This
+
+1. Start with `prima/prima-seed.yaml` — understand the questioning protocol
+2. Read `personal-finance-transactions/seed.yaml` — see how a domain seed
+   captures the context and level structure
+3. Browse `nodes/` — each file is a self-contained ontological contract
+4. Check `placement_map.yaml` — see how messy real-world bank transactions
+   get assigned to canonical IDs with meaning
+
+## What's Next
+
+- [ ] Build a CLI runner that executes the Prima Seed against any data column
+- [ ] Define canonical ID hashing algorithm
+- [ ] Build the linking ID mechanism (cross-tree semantic comparison)
+- [ ] Create second domain seed (German Grocery Products) for cross-domain testing
+- [ ] Define the seed registry format for OpenTaxonomy platform

opentaxonomy-0.1.0/files/digital_subscriptions.yaml

@@ -0,0 +1,31 @@
+node: Digital Subscriptions
+canonical_id: "ft.expenditure.fixed.subscriptions"
+question: "Is this a recurring payment for a digital service or platform?"
+criteria:
+  includes:
+    - "Streaming services (Netflix, Disney+, Spotify, MUBI, Audible)"
+    - "Software subscriptions (ChatGPT, Wix, Apple services)"
+    - "Gaming services (Nintendo, Google Play)"
+    - "Internet service (HerzoMedia/HERZOvision)"
+  excludes:
+    - "One-time digital purchases"
+    - "Physical subscription boxes"
+    - "Mobile phone contract (classified under telecommunications)"
+  edge_cases:
+    - term: "Amazon Prime"
+      resolution: "Included — recurring digital service even though it enables physical delivery"
+      decided: true
+    - term: "HerzoMedia/HERZOvision"
+      resolution: "Included — internet is a digital subscription; could also be telecommunications"
+      decided: false
+parent: Fixed Obligations
+children: []
+version: 1.0.0
+decision_record:
+  criterion_chosen: "Recurring billing for digital access"
+  alternatives_considered:
+    - "Split by media type (video, audio, software)"
+  reason: >
+    In budgeting, what matters is that these are fixed monthly
+    costs that can be individually cancelled. Media type is
+    secondary to the financial commitment structure.

opentaxonomy-0.1.0/files/dining_out.yaml

@@ -0,0 +1,28 @@
+node: Dining Out
+canonical_id: "ft.expenditure.discretionary.dining"
+question: "Is this a payment at a restaurant, cafe, bar, or food service establishment?"
+criteria:
+  includes:
+    - "Full-service restaurants"
+    - "Fast food chains (McDonalds, Five Guys, etc.)"
+    - "Cafes and coffee shops (Starbucks, etc.)"
+    - "Bars and beer gardens"
+    - "Takeaway/delivery services (Lieferando, Takeaway.com)"
+    - "Corporate canteens (Eurest)"
+    - "Bakery cafes when dining in"
+  excludes:
+    - "Supermarket food purchases"
+    - "Grocery delivery services"
+  edge_cases:
+    - term: "Bakery (Baeckerei und Konditorei)"
+      resolution: "Included — the purchase is prepared food for immediate consumption"
+      decided: true
+    - term: "Food Affairs GmbH"
+      resolution: "Included — appears to be a food service/catering entity"
+      decided: true
+    - term: "Airport food (Relay, Exki, Gru Fridays)"
+      resolution: "Included — dining out regardless of location"
+      decided: true
+parent: Discretionary
+children: []
+version: 1.0.0

opentaxonomy-0.1.0/files/discretionary.yaml

@@ -0,0 +1,30 @@
+node: Discretionary
+canonical_id: "ft.expenditure.discretionary"
+question: "Is this a lifestyle choice — spending that could be reduced or eliminated without impacting basic needs?"
+criteria:
+  includes:
+    - "Restaurant meals and dining out"
+    - "Fashion and retail shopping"
+    - "Online shopping (non-essential)"
+    - "Entertainment, leisure, cultural activities"
+    - "Travel and accommodation"
+  excludes:
+    - "Contractual obligations"
+    - "Essential groceries and household supplies"
+    - "Health-related spending"
+    - "Commute-related transport"
+  edge_cases:
+    - term: "Fast food (McDonalds)"
+      resolution: "Included — discretionary even if routine; not a nutritional necessity"
+      decided: true
+    - term: "Eurest corporate canteen"
+      resolution: "Included — could bring lunch from home; dining choice"
+      decided: true
+parent: Expenditure
+children:
+  - dining_out
+  - shopping_fashion
+  - shopping_online_general
+  - entertainment_leisure
+  - travel_accommodation
+version: 1.0.0

opentaxonomy-0.1.0/files/expenditure.yaml

@@ -0,0 +1,27 @@
+node: Expenditure
+canonical_id: "ft.expenditure"
+question: "Is money flowing out of the account?"
+criteria:
+  includes:
+    - "Any debit, card payment, direct debit, standing order, or cash withdrawal"
+  excludes:
+    - "Incoming payments"
+parent: Financial Transaction
+children:
+  - fixed_obligations
+  - variable_necessities
+  - discretionary
+  - personal_transfers
+  - cash_withdrawals
+version: 1.0.0
+decision_record:
+  criterion_chosen: "Degree of obligation — how controllable is this spending?"
+  alternatives_considered:
+    - "Merchant type (where did I spend?)"
+    - "Payment method (card vs. transfer vs. cash)"
+    - "Amount range"
+  reason: >
+    In a personal budgeting context, the most actionable split
+    is controllability. Fixed obligations are locked in; variable
+    necessities can be optimized; discretionary can be cut. This
+    maps directly to how a person can act on their budget.

opentaxonomy-0.1.0/files/groceries.yaml

@@ -0,0 +1,22 @@
+node: Groceries
+canonical_id: "ft.expenditure.variable.groceries"
+question: "Is this a purchase of food or household staples from a supermarket or grocery store?"
+criteria:
+  includes:
+    - "Supermarket purchases (REWE, LIDL, EDEKA, etc.)"
+    - "Organic/bio market purchases"
+    - "International grocery equivalents (Carrefour, Pao de Acucar, CONAD, etc.)"
+  excludes:
+    - "Restaurant meals (even takeaway from non-grocery)"
+    - "Specialty food shops that are primarily dining (bakery cafe)"
+    - "Drugstore purchases (Rossmann, DM) unless food items"
+  edge_cases:
+    - term: "REWE TO GO"
+      resolution: "Included — still a grocery/convenience purchase, not a restaurant"
+      decided: true
+    - term: "Biomarkt"
+      resolution: "Included — organic grocery store"
+      decided: true
+parent: Variable Necessities
+children: []
+version: 1.0.0