metadata-crawler 2509.0.0__tar.gz → 2509.0.1__tar.gz

{metadata_crawler-2509.0.0 → metadata_crawler-2509.0.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: metadata-crawler
-Version: 2509.0.0
+Version: 2509.0.1
 Summary: Crawl, extract and push climate metadata for indexing.
 Author-email: "DKRZ, Clint" <freva@dkrz.de>
 Requires-Python: >=3.11
@@ -83,10 +83,10 @@ Requires-Dist: pytest-env ; extra == "tests"
 Requires-Dist: requests ; extra == "tests"
 Requires-Dist: pre-commit ; extra == "tests"
 Requires-Dist: toml ; extra == "tests"
-Project-URL: Documentation, https://github.com/freva-org/freva-admin
-Project-URL: Home, https://github.com/freva-org/freva-admin
-Project-URL: Issues, https://github.com/freva-org/freva-admin/issues
-Project-URL: Source, https://github.com/freva-org/freva-admin
+Project-URL: Documentation, https://metadata-crawler.readthedocs.io
+Project-URL: Home, https://github.com/freva-org/metadata-crawler
+Project-URL: Issues, https://github.com/freva-org/metadata-crawler/issues
+Project-URL: Source, https://github.com/freva-org/metadata-crawler
 Provides-Extra: dev
 Provides-Extra: doc
 Provides-Extra: mkdoc
@@ -95,25 +95,25 @@ Provides-Extra: tests
 # metadata-crawler
 
 [![License](https://img.shields.io/badge/License-BSD-purple.svg)](LICENSE)
-[![PyPI](https://img.shields.io/pypi/pyversions/freva-client.svg)](https://pypi.org/project/metadata-crawler/)
+[![PyPI](https://img.shields.io/pypi/pyversions/metadata-crawler.svg)](https://pypi.org/project/metadata-crawler/)
 [![Docs](https://readthedocs.org/projects/metadata-crawler/badge/?version=latest)](https://metadata-crawler.readthedocs.io/en/latest/?badge=latest)
 [![Tests](https://github.com/freva-org/metadata-crawler/actions/workflows/ci_job.yml/badge.svg)](https://github.com/freva-org/metadata-crawler/actions)
 [![Test-Coverage](https://codecov.io/gh/freva-org/metadata-crawler/graph/badge.svg?token=W2YziDnh2N)](https://codecov.io/gh/freva-org/metadata-crawler)
 
 Harvest, normalise, and index climate / earth-system metadata from **POSIX**,
 **S3/MinIO**, and **OpenStack Swift** using configurable **DRS dialects**
-(CMIP6, CMIP5, CORDEX, …). Output to a temporary **catalogue** (DuckDB or
-JSONLines) and then **index** into systems such as **Solr** or **MongoDB**.
+(CMIP6, CMIP5, CORDEX, …). Output to a temporary **catalogue** (JSONLines)
+and then **index** into systems such as **Solr** or **MongoDB**.
 Configuration is **TOML** with inheritance, templating, and computed rules.
 
-> [!TIP]
->   **TL;DR**
->   - Define datasets + dialects in ``drs_config.toml``
->   - ``mdc crawl`` → write a temporary catalogue (``jsonl.gz`` or **DuckDB**)
->   - ``mdc config`` → inspect a the (merged) crawler config.
->   - ``mdc walk-intake`` → inspect the content of an intake catalogue.
->   - ``mdc <backend> index`` → push records from catalogue into your index backend
->   - ``mdc <backend> delete`` → remove records by facet match
+## TL;DR
+
+- Define datasets + dialects in ``drs_config.toml``
+- ``mdc add`` → write a temporary catalogue (``jsonl.gz``)
+- ``mdc config`` → inspect a the (merged) crawler config.
+- ``mdc walk-intake`` → inspect the content of an intake catalogue.
+- ``mdc <backend> index`` → push records from catalogue into your index backend
+- ``mdc <backend> delete`` → remove records by facet match
 
 ## Features
 
@@ -126,7 +126,7 @@ Configuration is **TOML** with inheritance, templating, and computed rules.
   dataset attributes/vars
 - **Special rules**: conditionals, cache lookups and function calls (e.g. CMIP6 realm,
   time aggregation)
-- **Index backends**: DuckDB (local/S3), MongoDB (Motor), Solr
+- **Index backends**: MongoDB (Motor), Solr
 - **Sync + Async APIs** and a clean CLI
 - **Docs**: Sphinx with ``pydata_sphinx_theme``
 
@@ -143,14 +143,14 @@ Configuration is **TOML** with inheritance, templating, and computed rules.
 ```console
 
    # 1) Crawl → write catalogue
-   mdc crawl \
+   mdc add \
      cat.yaml \
      --config-file drs_config.toml \
      --dataset cmip6-fs,obs-fs \
      --threads 4 --batch-size 100
 
-   # 2) Index from catalogue → Solr (or Mongo/DuckDB)
-   mdc soler index \
+   # 2) Index from catalogue → Solr (or Mongo)
+   mdc solr index \
      cat.yaml \
      --server localhot:8983
 

{metadata_crawler-2509.0.0 → metadata_crawler-2509.0.1}/README.md

@@ -1,25 +1,25 @@
 # metadata-crawler
 
 [![License](https://img.shields.io/badge/License-BSD-purple.svg)](LICENSE)
-[![PyPI](https://img.shields.io/pypi/pyversions/freva-client.svg)](https://pypi.org/project/metadata-crawler/)
+[![PyPI](https://img.shields.io/pypi/pyversions/metadata-crawler.svg)](https://pypi.org/project/metadata-crawler/)
 [![Docs](https://readthedocs.org/projects/metadata-crawler/badge/?version=latest)](https://metadata-crawler.readthedocs.io/en/latest/?badge=latest)
 [![Tests](https://github.com/freva-org/metadata-crawler/actions/workflows/ci_job.yml/badge.svg)](https://github.com/freva-org/metadata-crawler/actions)
 [![Test-Coverage](https://codecov.io/gh/freva-org/metadata-crawler/graph/badge.svg?token=W2YziDnh2N)](https://codecov.io/gh/freva-org/metadata-crawler)
 
 Harvest, normalise, and index climate / earth-system metadata from **POSIX**,
 **S3/MinIO**, and **OpenStack Swift** using configurable **DRS dialects**
-(CMIP6, CMIP5, CORDEX, …). Output to a temporary **catalogue** (DuckDB or
-JSONLines) and then **index** into systems such as **Solr** or **MongoDB**.
+(CMIP6, CMIP5, CORDEX, …). Output to a temporary **catalogue** (JSONLines)
+and then **index** into systems such as **Solr** or **MongoDB**.
 Configuration is **TOML** with inheritance, templating, and computed rules.
 
-> [!TIP]
->   **TL;DR**
->   - Define datasets + dialects in ``drs_config.toml``
->   - ``mdc crawl`` → write a temporary catalogue (``jsonl.gz`` or **DuckDB**)
->   - ``mdc config`` → inspect a the (merged) crawler config.
->   - ``mdc walk-intake`` → inspect the content of an intake catalogue.
->   - ``mdc <backend> index`` → push records from catalogue into your index backend
->   - ``mdc <backend> delete`` → remove records by facet match
+## TL;DR
+
+- Define datasets + dialects in ``drs_config.toml``
+- ``mdc add`` → write a temporary catalogue (``jsonl.gz``)
+- ``mdc config`` → inspect a the (merged) crawler config.
+- ``mdc walk-intake`` → inspect the content of an intake catalogue.
+- ``mdc <backend> index`` → push records from catalogue into your index backend
+- ``mdc <backend> delete`` → remove records by facet match
 
 ## Features
 
@@ -32,7 +32,7 @@ Configuration is **TOML** with inheritance, templating, and computed rules.
   dataset attributes/vars
 - **Special rules**: conditionals, cache lookups and function calls (e.g. CMIP6 realm,
   time aggregation)
-- **Index backends**: DuckDB (local/S3), MongoDB (Motor), Solr
+- **Index backends**: MongoDB (Motor), Solr
 - **Sync + Async APIs** and a clean CLI
 - **Docs**: Sphinx with ``pydata_sphinx_theme``
 
@@ -49,14 +49,14 @@ Configuration is **TOML** with inheritance, templating, and computed rules.
 ```console
 
    # 1) Crawl → write catalogue
-   mdc crawl \
+   mdc add \
      cat.yaml \
      --config-file drs_config.toml \
      --dataset cmip6-fs,obs-fs \
      --threads 4 --batch-size 100
 
-   # 2) Index from catalogue → Solr (or Mongo/DuckDB)
-   mdc soler index \
+   # 2) Index from catalogue → Solr (or Mongo)
+   mdc solr index \
      cat.yaml \
      --server localhot:8983
 

{metadata_crawler-2509.0.0 → metadata_crawler-2509.0.1}/docs/source/whatsnew.rst

@@ -3,7 +3,7 @@ What's new
 
 This document highlights major changes and additions across releases.
 
-v2509.0.0
+v2509.0.1
 ----------
 
 * Initial release of the documentation.

{metadata_crawler-2509.0.0 → metadata_crawler-2509.0.1}/pyproject.toml

@@ -100,10 +100,10 @@ tests = [
 ]
 
 [project.urls]
-Documentation = "https://github.com/freva-org/freva-admin"
-Issues = "https://github.com/freva-org/freva-admin/issues"
-Source = "https://github.com/freva-org/freva-admin"
-Home = "https://github.com/freva-org/freva-admin"
+Documentation = "https://metadata-crawler.readthedocs.io"
+Issues = "https://github.com/freva-org/metadata-crawler/issues"
+Source = "https://github.com/freva-org/metadata-crawler"
+Home = "https://github.com/freva-org/metadata-crawler"
 
 [project.scripts]
 metadata-crawler = "metadata_crawler.cli:cli"

metadata_crawler-2509.0.1/src/metadata_crawler/_version.py

@@ -0,0 +1 @@
+__version__ = "2509.0.1"

metadata_crawler-2509.0.0/src/metadata_crawler/_version.py

@@ -1 +0,0 @@
-__version__ = "2509.0.0"