prisma-api 0.4.5__tar.gz → 0.4.7__tar.gz

{prisma_api-0.4.5 → prisma_api-0.4.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: prisma_api
-Version: 0.4.5
+Version: 0.4.7
 Summary: A package for interacting with PrISMa Platform APIs. PrISMa is a holistic system for numerical synthesis and assessment of Metal-Organic Frameworks and their applications, for Direct Air Capture of CO2. For more information, see: https://prisma.hw.ac.uk.
 Author-email: RCCS-CaptureTeam <team@prisma-platform.org>
 License: GNUv3
@@ -58,6 +58,12 @@ api = prisma_api.init()
 ```
 
 On first use, the package creates a config file and prompts for an API key.
+Without a terminal (e.g. CI) it raises `RuntimeError` naming the environment
+variables to set instead (see below).
+
+`init()` prints whether it connected to production or a local dev server.
+Production is the default; use `prisma_api.init(local_dev=True)` for a local
+dev server on `localhost:<dev_host_port>`.
 
 ### Upgrades
 
@@ -88,19 +94,26 @@ create_config_file(api_key="YOUR_API_KEY_HERE")
 
 ### Environment-variable mode
 
-If you do not want to use the config file:
+When no config file exists, `init()` reads these environment variables
+instead (nothing is written to disk):
 
 ```bash
 export PRISMA_API_KEY="YOUR_API_KEY_HERE"
-export PRISMA_API_DEV="False"
+export PRISMA_API_DEV_API_KEY="YOUR_DEV_KEY"     # optional; defaults to PRISMA_API_KEY
+export PRISMA_API_DEV_HOST_PORT="8000"           # optional; used with local_dev=True
 ```
 
+To ignore any config file and use only the environment variables:
+
 ```python
-from prisma_api.prisma_api import prisma_api
+import prisma_api
 
-api = prisma_api(use_config_file=False)
+api = prisma_api.init(use_config_file=False)
 ```
 
+The target server is chosen only by `init(local_dev=...)`; there is no
+environment variable for it.
+
 ## Quick start
 
 ```python
@@ -129,6 +142,55 @@ flowsheet = api.v2.get_flowsheet(name="dac_min")
 ```
 
 
+## Bring your own dataset
+
+Send a folder of your own results (CSV, JSON or YAML, organised any way) to the platform as one zip. Your files stay as they are; you add one file, `prisma_manifest.yaml`, that says what each file is and where its data goes.
+
+1. **Install and connect.** You need your API key from your platform profile page.
+   ```bash
+   pip install prisma_api
+   python -c "import prisma_api; prisma_api.init()"
+   ```
+2. **Find the dataset on the platform.** Data always goes to a dataset that already exists there. If yours doesn't, create it first on the web portal (from its project's page). On the dataset's page, **Add un-managed dataset** gives you the command for step 4, with the dataset's slug and a tag for its records.
+3. **Leave files out** (optional). The listing skips whatever your `.gitignore` files match, and always leaves out `.git` and cache folders and anything named `*.claude`, `*.venv` or `*.DS_Store` (such as the `.claude` folder the skill is installed in), even if `.gitignore` doesn't list them. No other ignore file counts.
+4. **Send the folder tree.** This sends only the folder structure: each file's path and size. No file contents, zip or manifest are sent. In your folder, paste the command from step 2:
+   ```bash
+   cd my_dataset_folder
+   prisma-dataset tree . --prod --dataset my-dataset --tag "My dataset"
+   ```
+5. **Tick the files.** On the dataset's page, open **View folder tree** and tick the files to send. Every file starts unticked; ticks save automatically.
+6. **Build the manifest and send, with Claude Code.** Copy the command shown above the folder tree, which has the dataset's slug and tag. In your folder:
+   ```bash
+   prisma-dataset install-skill .      # once
+   claude                              # then paste the command, e.g.:
+   /prisma-dataset my-dataset --tag "My dataset" --prod
+   ```
+   Claude writes (or updates) `prisma_manifest.yaml` for the ticked files: what each file holds and where its data goes. It checks the manifest with the platform's dry run (nothing is written) until it is clean, summarises what would be written, and sends the data once you say yes. Agree your first write with the platform team.
+
+   To write the manifest yourself, read `prisma-dataset guide` and `prisma-dataset targets --prod`; `prisma-dataset ticked . --prod --dataset my-dataset` lists the ticked files it must cover. Then `prisma-dataset send . --prod` checks and packs the ticked files, shows the dry run and asks before writing.
+
+When your results change, resend the tree if files were added or removed (`prisma-dataset tree . --prod`), tick any new files, and run the Claude Code command (or `send`) again. The platform updates the dataset to match and keeps a version history:
+
+- **Same files, new data:** the tree is refreshed with your ticks kept. If the upload changes any records, the dataset goes up a minor version (1.0 → 1.1). If it changes nothing, the version stays as it is.
+- **Files added or removed:** if the dataset already has data, the platform makes a new major version: a copy of the dataset called `<slug>-v2`, at version 2.0 and tagged `"<tag> v2"`. Its authors, project links and Zenodo record are copied, and your ticks are carried over. The command prints the new tree's id; `send` uploads to it. Your manifest can keep the original slug.
+
+Each version records what its upload changed against the one before it: files added, removed or modified, and records created, updated or deleted per table and per run. A dry run reports the version it would make. To list a dataset's versions:
+```bash
+prisma-dataset versions my-dataset --prod
+```
+Older versions' values are not kept, only this record of the changes.
+
+`send` does these steps in one go; you can also run them one at a time:
+```bash
+prisma-dataset validate . --prod                         # local check of the ticked files
+prisma-dataset pack . --prod --out ../my_dataset.zip     # write the zip next to the folder, not inside it
+prisma-dataset upload ../my_dataset.zip --prod           # dry run
+prisma-dataset upload ../my_dataset.zip --prod --apply   # write
+```
+`pack` and `upload` take `--tag "Name"` (repeatable) to tag the records with more names than `dataset.tag`.
+
+The same steps are available in Python: `prisma_api.dataset_package.send_tree`, `ticked_files`, `send`, `fetch_selection`, `validate`, `pack`, `upload` and `load_targets`.
+
 ## Return formats
 
 All v2 list endpoints support two formats:
@@ -172,13 +234,14 @@ The main client exposes these v1 methods:
 Configuration helpers on the main client:
 
 - `set_return_format(fmt)`
-- `update_dev_mode(dev)`
+- `update_dev_mode(dev)` — deprecated and no longer changes the target; use `prisma_api.init(local_dev=True|False)`
 
 Module-level helpers exported from the package:
 
 - `prisma_api.init`
+- `prisma_api.__version__`
 - `prisma_api.PrismaAPIv2`
-- `prisma_api.update_dev_mode`
+- `prisma_api.update_dev_mode` — deprecated, as above
 - `prisma_api.update_dev_host_port`
 - `prisma_api.locate_config`
 - `prisma_api.PrismaUpsertError`, `prisma_api.PrismaRowErrorWarning`, `prisma_api.PrismaUnknownFieldsWarning`, `prisma_api.PrismaNewStructureWarning`
@@ -191,11 +254,15 @@ All methods below are available on `api.v2`.
 
 - `health()`
 - `get_flowsheet(name="dac_min")`
+- `get_flowsheet_bundle(name="dac_min")`
+- `upsert_flowsheets(flowsheets, screening_analysis_name=None, on_exists="append", appendix="_v4")`
 
 ### Materials and bundles
 
 - `list_materials(name=None, limit=10000)`
-- `get_material(material_id=None, name=None, bundle=['isotherms','zeopp','water_kpis','cif'])` (`bundle=None` returns root-only material detail)
+- `get_material(material_id=None, *, name=None, bundle=['isotherms','zeopp','water_kpis','cif'], sim_or_exp=None, good_structure=None, limit=500, offset=0, include_cif_text=False, cif_timeout=60)` (`bundle=None` returns root-only material detail; `name` may be a list)
+- `list_cifs(tag=None)`
+- `get_cifs(mof, structured=True, pandas=True, save_dir=None)` — one name or a list; `save_dir` writes the `.cif` files
 - `get_materials_psdi(name=None, limit=500, offset=0)`
 - `get_material_psdi(material_id)`
 - `get_material_property_bundle(mof, sim_or_exp=None, good_structure=None, limit=500, offset=0, query=None)`
@@ -213,6 +280,29 @@ All methods below are available on `api.v2`.
 - `build_case_pack(case_id, scenario_id=None)`
 - `list_case_packs(source=None, sink=None, region=None, study=None, include_scenarios=False, limit=100, offset=0)`
 
+### Case-study library (PrISMa_Platform sync)
+
+- `upsert_case_study_library(document, dry_run=False, max_request_bytes=2_400_000, raise_on_error=False, timeout=600)` — sends a `prisma_v2.case_study_library.bundle.v1` document, split into requests if needed; replaces the pipeline 11 ZIP upload
+- `get_case_study_library_bundle(sections=None, timeout=300)`
+- `get_case_study_bundle(case_id, timeout=120)`
+- `latest_case_study_library_sync()` — `None` before the first sync
+- `list_case_study_library_sync_runs(status=None, limit=50, git_commit=None, offset=0)`
+- `list_stale_case_study_library_records()`
+
+### Datasets and screening outputs
+
+- `get_dataset_targets()` — what a dataset manifest may write on the server: targets, kinds, allowed keys and fields
+- `register_dataset_tree(tree, dataset, manifest=None, tags=None, package=None, timeout=600)` — sends a folder's structure (paths and sizes only) so files can be ticked on the dataset's page; `package` refreshes an existing tree, keeping its ticks. Used by `prisma-dataset tree`
+- `list_dataset_trees(dataset=None, limit=50)` / `get_dataset_tree(package)` — registered trees, and one tree's files with their ticks
+- `ingest_dataset(zip_path, dry_run=True, timeout=1800, tags=None, package=None)` — sends a dataset package (files + `prisma_manifest.yaml`, built with `prisma-dataset pack`) to `POST /api/v2/datasets/ingest/`; returns the server report, including on rejection. Dry run unless `dry_run=False`. The format: `prisma-dataset guide`
+- `list_dataset_versions(dataset, limit=50)` — a dataset's versions, newest first, each with what its upload changed
+- `list_analysis_runs(dataset=None, limit=500, offset=0)` — runs loaded from dataset packages, with conditions, parameter set and row counts
+- `get_analysis_run_rows(run_key, table='results', limit=500, offset=0)` — a run's free-form rows (`results`: metrics with no typed table; `inputs`: per-run inputs), each with its material or reference-system subject
+- `ingest_screening_outputs(zip_path, timeout=7200, background=True, poll_every=15, progress=None)` — sends one batch of v1 case-study screening results (built with `prisma_api.screening_outputs.pack`) to `POST /api/v2/screening-outputs/ingest/` (pipeline 13); each case study / conditions / type becomes a scenario of the batch and its OutputKpi rows are replaced. Writes; no dry run. The server ingests in the background and the client polls until it is done, printing progress
+- `screening_outputs_ingest_status(zip_upload)` — status of a screening-output batch sent earlier
+
+From the command line: `prisma-outputs list <screening_results>` shows the batches, `prisma-outputs send <screening_results> --dev --batch NAME` (or `--all`) sends them after asking (`--yes` skips the question; `--prod` targets production; a failed batch is reported with the server's error and the next is still sent), and `prisma-outputs status <upload> --prod` checks a batch sent earlier.
+
 ### Catalog and reference data
 
 - `get_molecules(name=None, limit=500, offset=0)`
@@ -223,6 +313,8 @@ All methods below are available on `api.v2`.
 - `get_region(region_id)`
 - `get_sources(name=None, limit=500, offset=0)`
 - `get_source(source_id)`
+- `get_scopes(name=None, limit=500, offset=0)`
+- `get_scope(scope_id)`
 - `get_sinks(name=None, limit=500, offset=0)`
 - `get_sink(sink_id)`
 - `get_transport_scenarios(name=None, limit=500, offset=0)`
@@ -399,12 +491,22 @@ pytest tests/ -v --cov=prisma_api --cov-report=term-missing
 - [API_REFERENCE_V1.md](API_REFERENCE_V1.md)
 - [API_REFERENCE_V2.md](API_REFERENCE_V2.md)
 - [DEVELOPMENT_SUMMARY.md](DEVELOPMENT_SUMMARY.md)
-- [User Guide.md](User%20Guide.md)
+- [user_guide.md](user_guide.md)
+- Release notes, one file per minor series: [v0.4.x](https://github.com/RCCS-CaptureTeam/prisma_api/blob/main/docs/release_notes/v0.4.x.md), [v0.3.x](https://github.com/RCCS-CaptureTeam/prisma_api/blob/main/docs/release_notes/v0.3.x.md), [v0.2.x](https://github.com/RCCS-CaptureTeam/prisma_api/blob/main/docs/release_notes/v0.2.x.md)
+
+## Release notes
+
+Plain-language notes for every release, newest first, are in
+[docs/release_notes/](https://github.com/RCCS-CaptureTeam/prisma_api/tree/main/docs/release_notes). Each release section links to its
+GitHub release and to the full diff from the previous release. Check them
+before upgrading for breaking changes and deprecations.
 
 ## Notes
 
 - Production v2 base URL: `https://prisma-platform.org/api/v2`
 - Authentication header: `X-API-Key`
 - Local development routing is selected with `prisma_api.init(local_dev=True)`; the default targets production
+- Server compatibility notes for recent releases are below; see [docs/release_notes/](https://github.com/RCCS-CaptureTeam/prisma_api/tree/main/docs/release_notes) for the full history
 - 0.4.4: the `adsorption-isotherm` endpoints and the 400/notice handling need prisma_cloud >= 0.6.16
 - 0.4.5: H2 isotherms go through `upsert_adsorption_isotherm` (prisma_cloud >= 0.6.16); the H2-only methods and the `isotherm_H2s` collection key are deprecated. New `upsert_autoprism_cifs` uses the existing bundle upsert endpoint. AutoPrism getters take `match="exact"`.
+- 0.4.6: case-study library sync methods (`upsert_case_study_library` and the export, sync-run and stale-record readers) for the prisma_cloud `case-study-library` endpoints, which replace ingest pipeline 11.

{prisma_api-0.4.5 → prisma_api-0.4.7}/README.md

@@ -38,6 +38,12 @@ api = prisma_api.init()
 ```
 
 On first use, the package creates a config file and prompts for an API key.
+Without a terminal (e.g. CI) it raises `RuntimeError` naming the environment
+variables to set instead (see below).
+
+`init()` prints whether it connected to production or a local dev server.
+Production is the default; use `prisma_api.init(local_dev=True)` for a local
+dev server on `localhost:<dev_host_port>`.
 
 ### Upgrades
 
@@ -68,19 +74,26 @@ create_config_file(api_key="YOUR_API_KEY_HERE")
 
 ### Environment-variable mode
 
-If you do not want to use the config file:
+When no config file exists, `init()` reads these environment variables
+instead (nothing is written to disk):
 
 ```bash
 export PRISMA_API_KEY="YOUR_API_KEY_HERE"
-export PRISMA_API_DEV="False"
+export PRISMA_API_DEV_API_KEY="YOUR_DEV_KEY"     # optional; defaults to PRISMA_API_KEY
+export PRISMA_API_DEV_HOST_PORT="8000"           # optional; used with local_dev=True
 ```
 
+To ignore any config file and use only the environment variables:
+
 ```python
-from prisma_api.prisma_api import prisma_api
+import prisma_api
 
-api = prisma_api(use_config_file=False)
+api = prisma_api.init(use_config_file=False)
 ```
 
+The target server is chosen only by `init(local_dev=...)`; there is no
+environment variable for it.
+
 ## Quick start
 
 ```python
@@ -109,6 +122,55 @@ flowsheet = api.v2.get_flowsheet(name="dac_min")
 ```
 
 
+## Bring your own dataset
+
+Send a folder of your own results (CSV, JSON or YAML, organised any way) to the platform as one zip. Your files stay as they are; you add one file, `prisma_manifest.yaml`, that says what each file is and where its data goes.
+
+1. **Install and connect.** You need your API key from your platform profile page.
+   ```bash
+   pip install prisma_api
+   python -c "import prisma_api; prisma_api.init()"
+   ```
+2. **Find the dataset on the platform.** Data always goes to a dataset that already exists there. If yours doesn't, create it first on the web portal (from its project's page). On the dataset's page, **Add un-managed dataset** gives you the command for step 4, with the dataset's slug and a tag for its records.
+3. **Leave files out** (optional). The listing skips whatever your `.gitignore` files match, and always leaves out `.git` and cache folders and anything named `*.claude`, `*.venv` or `*.DS_Store` (such as the `.claude` folder the skill is installed in), even if `.gitignore` doesn't list them. No other ignore file counts.
+4. **Send the folder tree.** This sends only the folder structure: each file's path and size. No file contents, zip or manifest are sent. In your folder, paste the command from step 2:
+   ```bash
+   cd my_dataset_folder
+   prisma-dataset tree . --prod --dataset my-dataset --tag "My dataset"
+   ```
+5. **Tick the files.** On the dataset's page, open **View folder tree** and tick the files to send. Every file starts unticked; ticks save automatically.
+6. **Build the manifest and send, with Claude Code.** Copy the command shown above the folder tree, which has the dataset's slug and tag. In your folder:
+   ```bash
+   prisma-dataset install-skill .      # once
+   claude                              # then paste the command, e.g.:
+   /prisma-dataset my-dataset --tag "My dataset" --prod
+   ```
+   Claude writes (or updates) `prisma_manifest.yaml` for the ticked files: what each file holds and where its data goes. It checks the manifest with the platform's dry run (nothing is written) until it is clean, summarises what would be written, and sends the data once you say yes. Agree your first write with the platform team.
+
+   To write the manifest yourself, read `prisma-dataset guide` and `prisma-dataset targets --prod`; `prisma-dataset ticked . --prod --dataset my-dataset` lists the ticked files it must cover. Then `prisma-dataset send . --prod` checks and packs the ticked files, shows the dry run and asks before writing.
+
+When your results change, resend the tree if files were added or removed (`prisma-dataset tree . --prod`), tick any new files, and run the Claude Code command (or `send`) again. The platform updates the dataset to match and keeps a version history:
+
+- **Same files, new data:** the tree is refreshed with your ticks kept. If the upload changes any records, the dataset goes up a minor version (1.0 → 1.1). If it changes nothing, the version stays as it is.
+- **Files added or removed:** if the dataset already has data, the platform makes a new major version: a copy of the dataset called `<slug>-v2`, at version 2.0 and tagged `"<tag> v2"`. Its authors, project links and Zenodo record are copied, and your ticks are carried over. The command prints the new tree's id; `send` uploads to it. Your manifest can keep the original slug.
+
+Each version records what its upload changed against the one before it: files added, removed or modified, and records created, updated or deleted per table and per run. A dry run reports the version it would make. To list a dataset's versions:
+```bash
+prisma-dataset versions my-dataset --prod
+```
+Older versions' values are not kept, only this record of the changes.
+
+`send` does these steps in one go; you can also run them one at a time:
+```bash
+prisma-dataset validate . --prod                         # local check of the ticked files
+prisma-dataset pack . --prod --out ../my_dataset.zip     # write the zip next to the folder, not inside it
+prisma-dataset upload ../my_dataset.zip --prod           # dry run
+prisma-dataset upload ../my_dataset.zip --prod --apply   # write
+```
+`pack` and `upload` take `--tag "Name"` (repeatable) to tag the records with more names than `dataset.tag`.
+
+The same steps are available in Python: `prisma_api.dataset_package.send_tree`, `ticked_files`, `send`, `fetch_selection`, `validate`, `pack`, `upload` and `load_targets`.
+
 ## Return formats
 
 All v2 list endpoints support two formats:
@@ -152,13 +214,14 @@ The main client exposes these v1 methods:
 Configuration helpers on the main client:
 
 - `set_return_format(fmt)`
-- `update_dev_mode(dev)`
+- `update_dev_mode(dev)` — deprecated and no longer changes the target; use `prisma_api.init(local_dev=True|False)`
 
 Module-level helpers exported from the package:
 
 - `prisma_api.init`
+- `prisma_api.__version__`
 - `prisma_api.PrismaAPIv2`
-- `prisma_api.update_dev_mode`
+- `prisma_api.update_dev_mode` — deprecated, as above
 - `prisma_api.update_dev_host_port`
 - `prisma_api.locate_config`
 - `prisma_api.PrismaUpsertError`, `prisma_api.PrismaRowErrorWarning`, `prisma_api.PrismaUnknownFieldsWarning`, `prisma_api.PrismaNewStructureWarning`
@@ -171,11 +234,15 @@ All methods below are available on `api.v2`.
 
 - `health()`
 - `get_flowsheet(name="dac_min")`
+- `get_flowsheet_bundle(name="dac_min")`
+- `upsert_flowsheets(flowsheets, screening_analysis_name=None, on_exists="append", appendix="_v4")`
 
 ### Materials and bundles
 
 - `list_materials(name=None, limit=10000)`
-- `get_material(material_id=None, name=None, bundle=['isotherms','zeopp','water_kpis','cif'])` (`bundle=None` returns root-only material detail)
+- `get_material(material_id=None, *, name=None, bundle=['isotherms','zeopp','water_kpis','cif'], sim_or_exp=None, good_structure=None, limit=500, offset=0, include_cif_text=False, cif_timeout=60)` (`bundle=None` returns root-only material detail; `name` may be a list)
+- `list_cifs(tag=None)`
+- `get_cifs(mof, structured=True, pandas=True, save_dir=None)` — one name or a list; `save_dir` writes the `.cif` files
 - `get_materials_psdi(name=None, limit=500, offset=0)`
 - `get_material_psdi(material_id)`
 - `get_material_property_bundle(mof, sim_or_exp=None, good_structure=None, limit=500, offset=0, query=None)`
@@ -193,6 +260,29 @@ All methods below are available on `api.v2`.
 - `build_case_pack(case_id, scenario_id=None)`
 - `list_case_packs(source=None, sink=None, region=None, study=None, include_scenarios=False, limit=100, offset=0)`
 
+### Case-study library (PrISMa_Platform sync)
+
+- `upsert_case_study_library(document, dry_run=False, max_request_bytes=2_400_000, raise_on_error=False, timeout=600)` — sends a `prisma_v2.case_study_library.bundle.v1` document, split into requests if needed; replaces the pipeline 11 ZIP upload
+- `get_case_study_library_bundle(sections=None, timeout=300)`
+- `get_case_study_bundle(case_id, timeout=120)`
+- `latest_case_study_library_sync()` — `None` before the first sync
+- `list_case_study_library_sync_runs(status=None, limit=50, git_commit=None, offset=0)`
+- `list_stale_case_study_library_records()`
+
+### Datasets and screening outputs
+
+- `get_dataset_targets()` — what a dataset manifest may write on the server: targets, kinds, allowed keys and fields
+- `register_dataset_tree(tree, dataset, manifest=None, tags=None, package=None, timeout=600)` — sends a folder's structure (paths and sizes only) so files can be ticked on the dataset's page; `package` refreshes an existing tree, keeping its ticks. Used by `prisma-dataset tree`
+- `list_dataset_trees(dataset=None, limit=50)` / `get_dataset_tree(package)` — registered trees, and one tree's files with their ticks
+- `ingest_dataset(zip_path, dry_run=True, timeout=1800, tags=None, package=None)` — sends a dataset package (files + `prisma_manifest.yaml`, built with `prisma-dataset pack`) to `POST /api/v2/datasets/ingest/`; returns the server report, including on rejection. Dry run unless `dry_run=False`. The format: `prisma-dataset guide`
+- `list_dataset_versions(dataset, limit=50)` — a dataset's versions, newest first, each with what its upload changed
+- `list_analysis_runs(dataset=None, limit=500, offset=0)` — runs loaded from dataset packages, with conditions, parameter set and row counts
+- `get_analysis_run_rows(run_key, table='results', limit=500, offset=0)` — a run's free-form rows (`results`: metrics with no typed table; `inputs`: per-run inputs), each with its material or reference-system subject
+- `ingest_screening_outputs(zip_path, timeout=7200, background=True, poll_every=15, progress=None)` — sends one batch of v1 case-study screening results (built with `prisma_api.screening_outputs.pack`) to `POST /api/v2/screening-outputs/ingest/` (pipeline 13); each case study / conditions / type becomes a scenario of the batch and its OutputKpi rows are replaced. Writes; no dry run. The server ingests in the background and the client polls until it is done, printing progress
+- `screening_outputs_ingest_status(zip_upload)` — status of a screening-output batch sent earlier
+
+From the command line: `prisma-outputs list <screening_results>` shows the batches, `prisma-outputs send <screening_results> --dev --batch NAME` (or `--all`) sends them after asking (`--yes` skips the question; `--prod` targets production; a failed batch is reported with the server's error and the next is still sent), and `prisma-outputs status <upload> --prod` checks a batch sent earlier.
+
 ### Catalog and reference data
 
 - `get_molecules(name=None, limit=500, offset=0)`
@@ -203,6 +293,8 @@ All methods below are available on `api.v2`.
 - `get_region(region_id)`
 - `get_sources(name=None, limit=500, offset=0)`
 - `get_source(source_id)`
+- `get_scopes(name=None, limit=500, offset=0)`
+- `get_scope(scope_id)`
 - `get_sinks(name=None, limit=500, offset=0)`
 - `get_sink(sink_id)`
 - `get_transport_scenarios(name=None, limit=500, offset=0)`
@@ -379,12 +471,22 @@ pytest tests/ -v --cov=prisma_api --cov-report=term-missing
 - [API_REFERENCE_V1.md](API_REFERENCE_V1.md)
 - [API_REFERENCE_V2.md](API_REFERENCE_V2.md)
 - [DEVELOPMENT_SUMMARY.md](DEVELOPMENT_SUMMARY.md)
-- [User Guide.md](User%20Guide.md)
+- [user_guide.md](user_guide.md)
+- Release notes, one file per minor series: [v0.4.x](https://github.com/RCCS-CaptureTeam/prisma_api/blob/main/docs/release_notes/v0.4.x.md), [v0.3.x](https://github.com/RCCS-CaptureTeam/prisma_api/blob/main/docs/release_notes/v0.3.x.md), [v0.2.x](https://github.com/RCCS-CaptureTeam/prisma_api/blob/main/docs/release_notes/v0.2.x.md)
+
+## Release notes
+
+Plain-language notes for every release, newest first, are in
+[docs/release_notes/](https://github.com/RCCS-CaptureTeam/prisma_api/tree/main/docs/release_notes). Each release section links to its
+GitHub release and to the full diff from the previous release. Check them
+before upgrading for breaking changes and deprecations.
 
 ## Notes
 
 - Production v2 base URL: `https://prisma-platform.org/api/v2`
 - Authentication header: `X-API-Key`
 - Local development routing is selected with `prisma_api.init(local_dev=True)`; the default targets production
+- Server compatibility notes for recent releases are below; see [docs/release_notes/](https://github.com/RCCS-CaptureTeam/prisma_api/tree/main/docs/release_notes) for the full history
 - 0.4.4: the `adsorption-isotherm` endpoints and the 400/notice handling need prisma_cloud >= 0.6.16
 - 0.4.5: H2 isotherms go through `upsert_adsorption_isotherm` (prisma_cloud >= 0.6.16); the H2-only methods and the `isotherm_H2s` collection key are deprecated. New `upsert_autoprism_cifs` uses the existing bundle upsert endpoint. AutoPrism getters take `match="exact"`.
+- 0.4.6: case-study library sync methods (`upsert_case_study_library` and the export, sync-run and stale-record readers) for the prisma_cloud `case-study-library` endpoints, which replace ingest pipeline 11.

prisma_api-0.4.7/prisma_api/dataset_docs/SKILL.md

@@ -0,0 +1,166 @@
+---
+name: prisma-dataset
+description: Build or update prisma_manifest.yaml for the files ticked in this folder's PrISMa folder tree, check it with the platform's dry run, and send the data, in one command such as `/prisma-dataset my-dataset --tag "My tag" --prod` (the dataset page shows it above the folder tree). Use when asked to prepare, package, send or update this dataset for PrISMa, to write or fix the manifest, or after new results were added to the folder.
+argument-hint: '<dataset-slug> --tag "<tag>" [--dev | --prod]'
+---
+
+# Build the manifest and send this folder to PrISMa
+
+Arguments: `$ARGUMENTS`
+
+The dataset page on the platform gives these arguments, in the command above its
+**View folder tree**:
+
+- the dataset slug (first);
+- the tag the records go under (`--tag "..."`; there may be more than one);
+- the server: `--prod` for the platform, `--dev` for a local development server.
+
+If the slug or the tag is missing, ask the user to copy the command from the
+dataset page; don't invent either. With no server flag, use `--prod`. Below,
+`<server>`, `<slug>` and `<tag>` stand for these arguments; use them in every
+command.
+
+The folder belongs to the researchers: never move, rename or edit their files.
+The only file you write is `prisma_manifest.yaml` at the folder root. The
+format is in [manifest_guide.md](manifest_guide.md); read it first.
+
+`prisma-dataset` comes with the `prisma_api` package. If it is missing, say so
+and suggest `pip install prisma_api`. If commands report a missing API key,
+suggest `python -c "import prisma_api; prisma_api.init()"` (with
+`local_dev=True` inside `init(...)` for `--dev`).
+
+## 1. Find the files to send
+
+```bash
+prisma-dataset ticked . <server> --dataset <slug>
+prisma-dataset ticked . <server> --dataset <slug> --json    # the same, for reading
+```
+
+This lists the files the user ticked "in dataset" in the platform's folder tree,
+with the dataset's title and tags. Only these files are sent, so the manifest
+must cover every one of them; files that are not ticked need no source.
+
+- **No tree yet:** the user first sends the folder structure with the dataset page's **Add un-managed dataset** command (`prisma-dataset tree . <server> --dataset <slug> --tag "<tag>"`), then ticks files in **View folder tree**. Give them that, and stop.
+- **Nothing ticked:** ask them to tick the files to send on the dataset page, and stop.
+- **"Newer than the tree":** these files are not sent. If the user wants them, they resend the tree (`prisma-dataset tree . <server>`) and tick them. Mention them in the hand-over.
+- **"[by others]":** the user marked these files as data by others. Give their sources `by_others: true`.
+
+## 2. Learn what the platform can store
+
+```bash
+prisma-dataset targets <server>
+prisma-dataset targets <server> --json     # the full catalogue, including defaults and allowed keys
+```
+
+Base every target name, field name and key on this output, never on memory.
+Targets whose fields are "any names you choose" take any columns: use them for
+data with no fixed table instead of skipping it.
+
+**A typed target beats a generic one.** Read each target's description and
+decide what the data *means* before choosing where it goes. Generic targets
+(`run_results`, `run_inputs`) store a JSON blob per material: nothing in them is
+a queryable record, and they do not appear as the data's own table on the
+dataset page. Use `run_inputs` only for values a model run *consumed*, and
+`run_results` only when no typed target fits. Measurements and simulation
+outputs that the catalogue has a table for (for example isotherm points)
+belong in that table.
+
+## 3. Profile the ticked files (read-only)
+
+Work out the following. For many files, sample rather than reading them all.
+
+- **File families:** files grouped by naming pattern, with their counts, and the columns or keys of each family. For a git repository, also note the recent history: `git log --stat`, and any change notes the authors wrote.
+- **The id column** of each family, and how ids are spelled across families: suffixes such as `.cif`, case differences, and names of benchmarks that are not materials.
+- **Variants encoded in paths:** cases, conditions, model versions. Each distinct variant becomes a run.
+- **PrISMa v2 flowsheet outputs:** results keyed by flowsheet node (a run store, `run_flowsheet` output, columns or JSON paths such as `tea/ncac_...` or `lca.climate_change`) are the results of a v2 screening analysis. Note the case (e.g. `cases/dac_min/dac_min.json`) and the run id; step 4 links the run to the analysis.
+- **Rollups and copies:** which files summarise others (prefer the rollup where it is complete), and which are identical copies (compare hashes).
+- **Units:** take them from column headers, READMEs and notebooks. Record only conditions you can source.
+- **Wide files:** a file with one column per condition holding the same quantity (`223_K_5_bar`, `223_K_10_bar`, `uptake_30bar`, ...) is a *set of points*, not a set of unrelated columns. Work out the quantity (what is measured, its units), the condition each column stands for (pressure, temperature, time, ...) and the fixed conditions (gas, temperature). Several columns of one quantity at different conditions is the shape of an isotherm or any other curve.
+- **How each value was obtained:** look for a column of notes or a source (`source`, `method`, `origin`) and for flag columns. Values in one file often come from different calculations: simulated or measured, rescaled from another temperature (Clausius-Clapeyron), fitted (Langmuir), interpolated, or not computed at all. Read the distinct texts (`sort | uniq -c`) and note which values each applies to, since a text may cover only some columns. This is data, not decoration: keep it with the values (step 4).
+
+## 4. Build or update the manifest
+
+If `prisma_manifest.yaml` exists, update it. Keep the authors' sources, runs,
+comments and decisions. Add sources for ticked files that no source reads, and
+fix what is out of date. Leave sources whose files are all unticked in place;
+they are simply not sent. Otherwise write a new one with these sections, in
+this order:
+
+1. `dataset`
+2. `material_id`
+3. `reference_systems`
+4. `parameter_sets`
+5. `runs`
+6. `sources`
+
+The `dataset` section:
+
+- `slug`: the `<slug>` argument.
+- `tag`: the `<tag>` argument (a list if there are several).
+- `name`: the dataset's title from step 1.
+- `by_others`: keep the manifest's value. In a new manifest write `by_others: false  # our own data` and point it out in the hand-over. It decides whether the data may appear in the platform's Zenodo publications.
+
+If an existing manifest names another slug or tag, replace it with the
+argument and tell the user. The exception is a slug the step 1 output lists
+under `lineage` (an older version of the same dataset); that one may stay.
+
+For the other sections:
+
+- Map each family to the best target in the catalogue. Map columns to the target's fields; for "any names you choose" targets, pick short snake_case names and give their `units`.
+- A wide file of one quantity at several conditions goes to the catalogue's curve or point target (today `isotherms`: one stored row per material and pressure) with the condition of each column declared in the manifest, never to `run_inputs` or `run_results` as one column per condition. If the catalogue has no such target, say so in the hand-over instead of quietly storing the columns under chosen names.
+- Do not drop how a value was obtained. Where the target can record a basis for each value (`isotherms` has `basis_column` and `basis_rules`), read the distinct notes from step 3 and write one rule per calculation, listing the conditions it applies to; the default basis is what the rest were. Keep the authors' own text (it is stored beside the value) and map flag columns to the target's quality column.
+- For PrISMa v2 flowsheet outputs, give the run `screening_analysis: <analysis name>` (ask the user for the name if the files do not make it certain), send the per-material outputs to `run_results`, and name each column `<node_id>.<output>` after the node in the case's flowsheet, with `units` for every column. Without the node prefix an output only reaches the study page's Visualiser, not its TEA, LCA or Process tab.
+- Use `rows: supplement` for per-material data that should join onto each run's records. Keep `rows: create` on the files that define which materials a run covered.
+- Give every ticked file a source. Put ticked files that hold nothing to ingest (plots, notebooks, copies, partial or temporary files) under `target: skip` with a specific reason.
+- Comment judgement calls in the YAML, e.g. `# 2,525 of 8,236 rows; the full file is X`.
+
+## 5. Check until clean
+
+```bash
+prisma-dataset validate . <server>
+prisma-dataset send . <server> --json        # pack the ticked files and dry run: writes nothing
+```
+
+Fix every error and repeat. Then read the dry-run counts critically:
+
+- Rows per run should match the records in that run's files. For a typed target, find its own table in the dry-run counts (e.g. `adsorption_isotherm: created N`). If the data you meant to store as records only shows as `AnalysisRunInput` / `AnalysisResult`, or the table is absent, the source is on the wrong target: go back to step 4.
+- Blank values make no record. The run's report gives how many were skipped (`isotherm_blank_points`); that should match the empty cells you saw when profiling.
+- Unexpected "materials created" usually mean id spellings need `strip_suffixes` or reference-system `aliases`.
+- Any deletions should be explained.
+
+A refusal about the dataset or the tag (no such dataset, deprecated, tag in use
+elsewhere) is not a manifest problem: tell the user what the platform said and
+stop.
+
+## 6. Confirm, then send
+
+Summarise for the user:
+
+- the runs, with their row counts
+- the benchmarks
+- what was skipped and why
+- the dataset version the upload will make (`version` in the dry run)
+- `by_others`, if you set it
+- files that are newer than the tree, or ticked but missing
+- data problems the authors should check: outliers, files that shrank, partial runs
+
+Then ask once: "Send this to the platform (`<server>`)?" Only on a clear yes, run:
+
+```bash
+prisma-dataset send . <server> --apply
+```
+
+Report the version it made and the dataset page link.
+
+Never run `send --apply` or `upload --apply` without that yes. It writes to the
+platform and removes records that are no longer in the package.
+
+## Later updates
+
+Run the same command again after new results arrive. The manifest is updated
+the same way, and a send that changes records makes a minor version.
+
+If files were added or removed, the user resends the tree first
+(`prisma-dataset tree . <server>`) and ticks the new files. Once data has been
+sent, a changed file list makes a new major version, `<slug>-v2` tagged
+`"<tag> v2"`. The tree command prints the new command to use.