loom-data 0.1.0__tar.gz

loom_data-0.1.0/PKG-INFO

@@ -0,0 +1,330 @@
+Metadata-Version: 2.4
+Name: loom-data
+Version: 0.1.0
+Summary: Python client and CLI for Loom workspace data and sync workflows.
+Project-URL: Homepage, https://github.com/yechenyan/loom
+Project-URL: Repository, https://github.com/yechenyan/loom
+Project-URL: Issues, https://github.com/yechenyan/loom/issues
+Keywords: cli,data,loom,sync,workspace
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Utilities
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+
+# loom-data
+
+`loom-data` publishes the `loom` Python package and the `loom` CLI.
+
+It is used to:
+
+- read raw Loom data with a local cache
+- scan `loom_raw` datasets into summarized `loom_explore` outputs
+- review and confirm local workspace changes
+- push and pull workspaces against a Loom sync server
+
+The install name is `loom-data`, but the Python import stays `import loom` and the command stays `loom`.
+
+## What Loom does
+
+Loom turns raw datasets under `test-project/loom/loom_raw` into structured summaries under `test-project/loom/loom_explore`.
+
+This lets agents and people read generated overviews, profiles, and data cards first, instead of opening every raw CSV directly.
+
+## Directory layout
+
+Loom expects this workspace layout:
+
+- `test-project/loom/loom_raw/<topic>`: raw source datasets
+- `test-project/loom/loom_explore/<topic>`: generated scan output
+- `test-project/loom/.loom/raw/<workspace>`: local raw cache
+
+In this repository:
+
+- client package source: `packages/loom/src/loom`
+- server package source: `packages/loom-server/src/loom_server`
+
+## Dataset rule
+
+If a directory contains `loom.md`, Loom treats that directory as a dataset root.
+
+That dataset includes CSV files under the directory and its children. If a nested child directory also has its own `loom.md`, it becomes a separate dataset and is not folded into the parent dataset scan.
+
+## Install
+
+Install from PyPI:
+
+```bash
+pip install loom-data
+```
+
+For local development in this repository:
+
+```bash
+uv sync
+uv pip install -e .
+```
+
+If you also want the FastAPI sync server and `loom-server` CLI:
+
+```bash
+uv pip install -e packages/loom-server
+```
+
+## Initial setup
+
+Run this once in a workspace to install the local Codex skill and initialize `loom_explore` as a separate git repository:
+
+```bash
+loom install
+```
+
+Or, if you want to be explicit:
+
+```bash
+loom install --workspace-root /path/to/workspace
+```
+
+This setup is what enables the scan workflow to show pending changes and later confirm them into the `loom_explore` git history.
+
+## Python API
+
+Read one raw file and make sure it exists in the local raw cache:
+
+```python
+import loom
+
+local_path = loom.get("energy/technology-data/costs.csv")
+print(local_path)
+```
+
+Refresh a workspace raw cache:
+
+```python
+import loom
+
+loom.pull("energy")
+```
+
+Parse chat text into a Loom request:
+
+```python
+import loom
+
+request = loom.parse_chat_request("loom scan energy")
+print(request)
+```
+
+## CLI overview
+
+Main commands:
+
+```bash
+loom install
+loom scan <topic>
+loom route "<message>"
+loom get <workspace/path/to/file>
+loom pull-raw [workspace]
+loom status [workspace]
+loom confirm [workspace]
+loom push [workspace]
+loom pull [workspace]
+```
+
+If `workspace` is omitted for `push`, `pull`, or `pull-raw`, Loom uses all available top-level workspaces where that behavior is supported.
+
+## Scan a topic
+
+Run a local scan:
+
+```bash
+loom scan energy
+```
+
+This scan is local Python logic only. It will:
+
+- inspect `test-project/loom/loom_raw/energy`
+- find dataset roots via `loom.md`
+- read CSV files
+- generate dataset summaries and cards
+- write outputs to `test-project/loom/loom_explore/energy`
+- print the resulting pending changes
+
+You can also test whether a chat message would route into a Loom action:
+
+```bash
+loom route "loom scan energy"
+```
+
+## Review and confirm scan output
+
+After a scan:
+
+```bash
+loom status energy
+loom confirm energy
+```
+
+`loom status` shows pending files plus sync metadata.
+
+`loom confirm` creates a commit inside the separate `loom_explore` git repository so the workspace has a local confirmed state before syncing.
+
+## Read raw data
+
+Use the CLI:
+
+```bash
+loom get energy/technology-data/costs.csv
+```
+
+Use the Python API:
+
+```python
+import loom
+
+path = loom.get("energy/technology-data/costs.csv")
+```
+
+Lookup behavior:
+
+- Loom first checks `test-project/loom/.loom/raw/...`
+- if the file is already cached locally, that path is reused
+- if a matching file already exists under `test-project/loom/loom_raw/...` or `.raw_data/...`, Loom links it into `.loom/raw`
+- otherwise Loom fetches the latest version from the Loom sync server
+
+Repeated reads reuse the local cache.
+
+## Refresh raw cache
+
+Refresh one workspace:
+
+```bash
+loom pull-raw energy --server-url http://127.0.0.1:8765
+```
+
+Or in Python:
+
+```python
+import loom
+
+loom.pull("energy")
+```
+
+`loom pull-raw` and `loom.pull(...)` compare the remote raw manifest by `path + sha256` and only refresh changed or deleted files.
+
+## Push and pull workspaces
+
+Push one workspace:
+
+```bash
+loom push energy --server-url http://127.0.0.1:8765
+```
+
+Pull one workspace:
+
+```bash
+loom pull energy --server-url http://127.0.0.1:8765
+```
+
+Pull all workspaces:
+
+```bash
+loom pull --server-url http://127.0.0.1:8765
+```
+
+Current sync behavior is rebase-oriented:
+
+- `push` auto-confirms pending local changes before sending
+- if the remote has advanced, `push` rebases local confirmed work first, then continues
+- `pull` fast-forwards when possible
+- if local confirmed commits exist, `pull` rebases them on top of the latest remote revision
+- rebase conflicts return a non-zero exit code and keep conflict markers in `loom_explore`
+
+After resolving a conflict, run:
+
+```bash
+loom confirm <workspace>
+loom push <workspace>
+```
+
+## Optional server commands
+
+The published `loom-data` package focuses on client workflows. Server commands require the separate `loom-server` package.
+
+Install it first:
+
+```bash
+pip install loom-server
+```
+
+Or in this repository:
+
+```bash
+uv pip install -e packages/loom-server
+```
+
+Then you can use:
+
+```bash
+loom server-init-db
+loom server-run
+```
+
+Default server URL:
+
+```text
+http://127.0.0.1:8765
+```
+
+Default database URL:
+
+```text
+postgresql+psycopg2://loom@127.0.0.1:5432/loom
+```
+
+Initialize the server database:
+
+```bash
+loom server-init-db
+```
+
+Run the server:
+
+```bash
+loom server-run --storage-root /tmp/loom-server-storage
+```
+
+## Scan outputs
+
+After scanning, Loom writes files such as:
+
+- `README.md`: topic-level entry file
+- `overview.md`: dataset overview
+- `profile.json`: dataset-level machine-readable summary
+- `*.card.md`: per-CSV data card
+- `*.profile.json`: per-CSV machine-readable profile
+
+## Release
+
+Build distributions:
+
+```bash
+uv build
+```
+
+Upload to TestPyPI first:
+
+```bash
+uv publish --publish-url https://test.pypi.org/legacy/
+```
+
+Upload to PyPI:
+
+```bash
+uv publish
+```

loom_data-0.1.0/README.md

@@ -0,0 +1,166 @@
+# loom workspace
+
+This repository is configured as a `uv` workspace.
+
+## Quick start
+
+```bash
+uv sync
+```
+
+Install the client package in editable mode if you want both the Python API and the `loom` CLI on your shell path:
+
+```bash
+uv pip install -e .
+```
+
+For PyPI installs, use the published project name `loom-data`:
+
+```bash
+pip install loom-data
+```
+
+Install the service package separately when you want the FastAPI sync service and the `loom-server` CLI:
+
+```bash
+uv pip install -e packages/loom-server
+```
+
+Then you can use either form:
+
+```python
+import loom
+
+path = loom.get("energy/technology-data/costs.csv")
+```
+
+```bash
+loom get energy/technology-data/costs.csv
+loom pull-raw energy
+```
+
+Raw cache and local Loom state now live under:
+
+```text
+test-project/loom/.loom/
+```
+
+Raw access behavior:
+
+- `loom.get("workspace/path/to/file")` first checks `test-project/loom/.loom/raw/...`
+- if the file is already available locally, Loom reuses it
+- if a matching local raw source exists under `test-project/loom/loom_raw` or `.raw_data`, Loom links that file into `.loom/raw`
+- otherwise Loom fetches only the missing latest file from the sync server
+- `loom.pull(...)` and `loom pull-raw ...` compare the remote raw manifest by path and hash, then refresh only changed or deleted cache entries
+
+## Install Loom Scan Fast Path
+
+Run this once to install the local Codex skill for `loom scan <topic>` and initialize the `loom_explore` git repo:
+
+```bash
+uv run python scripts/loom.py install
+```
+
+After that, Codex can recognize chat inputs like `loom scan energy` faster and route them to the scan flow directly.
+
+You can also run the scanner manually:
+
+```bash
+uv run python scripts/loom.py scan energy
+```
+
+After scanning, review changes and confirm them into the local `loom_explore` history:
+
+```bash
+uv run python scripts/loom.py status energy
+uv run python scripts/loom.py confirm energy
+```
+
+## Sync Server
+
+The sync server now lives in the separate `packages/loom-server` package. It uses FastAPI for the API layer and PostgreSQL for revision metadata.
+
+By default, Loom expects a local PostgreSQL instance at:
+
+```text
+postgresql+psycopg2://loom@127.0.0.1:5432/loom
+```
+
+`loom-server init-db` and `loom-server run` will automatically create the `loom` database if the local PostgreSQL server is reachable and the database does not exist yet.
+
+Initialize the server schema:
+
+```bash
+uv run python scripts/loom-server.py init-db
+```
+
+Run the server:
+
+```bash
+uv run python scripts/loom-server.py run --storage-root .loom-server-storage
+```
+
+Push and pull workspaces:
+
+```bash
+uv run python scripts/loom.py push energy --server-url http://127.0.0.1:8765
+uv run python scripts/loom.py pull energy --server-url http://127.0.0.1:8765
+uv run python scripts/loom.py pull-raw energy --server-url http://127.0.0.1:8765
+```
+
+Push and pull now use rebase-style workspace sync:
+
+- `push` will auto-confirm local pending changes, pull the latest remote revision when needed, rebase local confirmed work onto it, then continue the incremental push
+- `pull` will fast-forward when possible, or rebase local confirmed work onto the latest remote revision instead of blindly overwriting it
+- if a rebase conflict happens, Loom keeps conflict markers in `loom_explore`, returns a non-zero exit code, and asks you to resolve the files, run `loom confirm <workspace>`, then `loom push <workspace>`
+- normal `pull` / `push` raw refresh only updates files already present under `test-project/loom/.loom/raw/<workspace>/...`
+- `pull-raw` and `loom.pull(...)` still support intentional full raw cache refresh for a workspace
+- if local `test-project/loom/loom_raw/...` files disagree with the remote raw manifest, Loom will not overwrite them and will write `loom.raw-conflict.md` next to the nearest `loom.md`
+
+## Web App
+
+A React web app lives in [web/package.json](/Users/maxiao/Documents/code2/loom/web/package.json) and reads dataset summaries from the FastAPI server.
+
+Install and run it with `pnpm`:
+
+```bash
+cd /Users/maxiao/Documents/code2/loom/web
+pnpm install
+pnpm dev --host 127.0.0.1
+```
+
+Then open:
+
+```text
+http://127.0.0.1:4173
+```
+
+The Vite dev server proxies `/api` requests to the Loom FastAPI server on `http://127.0.0.1:8765`.
+
+## Workspace layout
+
+- Root workspace config: `pyproject.toml`
+- Client package source: `packages/loom/src/loom`
+- Service package source: `packages/loom-server/src/loom_server`
+
+## Publishing `loom-data`
+
+The import package remains `loom`, but the PyPI project name is `loom-data`.
+
+Build distributions:
+
+```bash
+uv build
+```
+
+Upload to TestPyPI first:
+
+```bash
+uv publish --publish-url https://test.pypi.org/legacy/
+```
+
+Upload to PyPI:
+
+```bash
+uv publish
+```