orcli 0.1.0__tar.gz

orcli-0.1.0/LICENSE.md

@@ -0,0 +1,121 @@
+Creative Commons Legal Code
+
+CC0 1.0 Universal
+
+    CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
+    LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
+    ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
+    INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
+    REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
+    PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
+    THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
+    HEREUNDER.
+
+Statement of Purpose
+
+The laws of most jurisdictions throughout the world automatically confer
+exclusive Copyright and Related Rights (defined below) upon the creator
+and subsequent owner(s) (each and all, an "owner") of an original work of
+authorship and/or a database (each, a "Work").
+
+Certain owners wish to permanently relinquish those rights to a Work for
+the purpose of contributing to a commons of creative, cultural and
+scientific works ("Commons") that the public can reliably and without fear
+of later claims of infringement build upon, modify, incorporate in other
+works, reuse and redistribute as freely as possible in any form whatsoever
+and for any purposes, including without limitation commercial purposes.
+These owners may contribute to the Commons to promote the ideal of a free
+culture and the further production of creative, cultural and scientific
+works, or to gain reputation or greater distribution for their Work in
+part through the use and efforts of others.
+
+For these and/or other purposes and motivations, and without any
+expectation of additional consideration or compensation, the person
+associating CC0 with a Work (the "Affirmer"), to the extent that he or she
+is an owner of Copyright and Related Rights in the Work, voluntarily
+elects to apply CC0 to the Work and publicly distribute the Work under its
+terms, with knowledge of his or her Copyright and Related Rights in the
+Work and the meaning and intended legal effect of CC0 on those rights.
+
+1. Copyright and Related Rights. A Work made available under CC0 may be
+protected by copyright and related or neighboring rights ("Copyright and
+Related Rights"). Copyright and Related Rights include, but are not
+limited to, the following:
+
+  i. the right to reproduce, adapt, distribute, perform, display,
+     communicate, and translate a Work;
+ ii. moral rights retained by the original author(s) and/or performer(s);
+iii. publicity and privacy rights pertaining to a person's image or
+     likeness depicted in a Work;
+ iv. rights protecting against unfair competition in regards to a Work,
+     subject to the limitations in paragraph 4(a), below;
+  v. rights protecting the extraction, dissemination, use and reuse of data
+     in a Work;
+ vi. database rights (such as those arising under Directive 96/9/EC of the
+     European Parliament and of the Council of 11 March 1996 on the legal
+     protection of databases, and under any national implementation
+     thereof, including any amended or successor version of such
+     directive); and
+vii. other similar, equivalent or corresponding rights throughout the
+     world based on applicable law or treaty, and any national
+     implementations thereof.
+
+2. Waiver. To the greatest extent permitted by, but not in contravention
+of, applicable law, Affirmer hereby overtly, fully, permanently,
+irrevocably and unconditionally waives, abandons, and surrenders all of
+Affirmer's Copyright and Related Rights and associated claims and causes
+of action, whether now known or unknown (including existing as well as
+future claims and causes of action), in the Work (i) in all territories
+worldwide, (ii) for the maximum duration provided by applicable law or
+treaty (including future time extensions), (iii) in any current or future
+medium and for any number of copies, and (iv) for any purpose whatsoever,
+including without limitation commercial, advertising or promotional
+purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
+member of the public at large and to the detriment of Affirmer's heirs and
+successors, fully intending that such Waiver shall not be subject to
+revocation, rescission, cancellation, termination, or any other legal or
+equitable action to disrupt the quiet enjoyment of the Work by the public
+as contemplated by Affirmer's express Statement of Purpose.
+
+3. Public License Fallback. Should any part of the Waiver for any reason
+be judged legally invalid or ineffective under applicable law, then the
+Waiver shall be preserved to the maximum extent permitted taking into
+account Affirmer's express Statement of Purpose. In addition, to the
+extent the Waiver is so judged Affirmer hereby grants to each affected
+person a royalty-free, non transferable, non sublicensable, non exclusive,
+irrevocable and unconditional license to exercise Affirmer's Copyright and
+Related Rights in the Work (i) in all territories worldwide, (ii) for the
+maximum duration provided by applicable law or treaty (including future
+time extensions), (iii) in any current or future medium and for any number
+of copies, and (iv) for any purpose whatsoever, including without
+limitation commercial, advertising or promotional purposes (the
+"License"). The License shall be deemed effective as of the date CC0 was
+applied by Affirmer to the Work. Should any part of the License for any
+reason be judged legally invalid or ineffective under applicable law, such
+partial invalidity or ineffectiveness shall not invalidate the remainder
+of the License, and in such case Affirmer hereby affirms that he or she
+will not (i) exercise any of his or her remaining Copyright and Related
+Rights in the Work or (ii) assert any associated claims and causes of
+action with respect to the Work, in either case contrary to Affirmer's
+express Statement of Purpose.
+
+4. Limitations and Disclaimers.
+
+ a. No trademark or patent rights held by Affirmer are waived, abandoned,
+    surrendered, licensed or otherwise affected by this document.
+ b. Affirmer offers the Work as-is and makes no representations or
+    warranties of any kind concerning the Work, express, implied,
+    statutory or otherwise, including without limitation warranties of
+    title, merchantability, fitness for a particular purpose, non
+    infringement, or the absence of latent or other defects, accuracy, or
+    the present or absence of errors, whether or not discoverable, all to
+    the greatest extent permissible under applicable law.
+ c. Affirmer disclaims responsibility for clearing rights of other persons
+    that may apply to the Work or any use thereof, including without
+    limitation any person's Copyright and Related Rights in the Work.
+    Further, Affirmer disclaims responsibility for obtaining any necessary
+    consents, permissions or other rights required for any use of the
+    Work.
+ d. Affirmer understands and acknowledges that Creative Commons is not a
+    party to this document and has no duty or obligation with respect to
+    this CC0 or use of the Work.

orcli-0.1.0/PKG-INFO

@@ -0,0 +1,252 @@
+Metadata-Version: 2.4
+Name: orcli
+Version: 0.1.0
+Summary: A Python client for interacting with OpenRefine
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: requests>=2.28.0
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Dynamic: license-file
+
+
+# refine-client
+
+A Python client library for interacting with [OpenRefine](https://openrefine.org/) via its REST API.
+
+For simple project creation, data transformation, metadata management and export operations.
+
+## Features
+
+  - Create and delete projects from local files.
+  - Retrieve and manage project metadata.
+  - Apply OpenRefine operations individually or in batches.
+  - Load operations from JSON files.
+  - Export data in multiple formats (TSV, CSV, JSON).
+  - Retrieve column information and project models, convert and manipulate row data.
+  - Error handling with detailed response logging.
+
+## Installation
+
+### Requirements
+
+Requires Python 3.10+ and a running OpenRefine server instance.
+
+### Quick Start
+
+1. Clone the repository:
+
+```bash
+git clone https://github.com/rkraasch/refine-client.git
+cd refine-client
+```
+
+2. Install dependencies:
+
+```bash
+python -m venv .venv
+source [.venv/bin/activate](.venv/bin/activate)  # Linux and Mac.
+# .venv\Scripts\activate   # Windows.
+pip install -r requirements.txt
+python -m pip list
+deactivate
+```
+
+3. Optionally run tests:
+
+[Download](https://openrefine.org/download) and run OpenRefine, then run the tests as shown below.
+
+```bash
+python -m pytest tests/ -v
+```
+This creates temporary test projects in OpenRefine which should be cleaned up automatically.
+If a project prefixed `pytest_refine-client-integration_` remains visible in the Open project tab (under [#open-project](http://127.0.0.1:3333/#open-project)), something went wrong.
+
+4. Basic Usage:
+
+```python
+from refine_client import Refine
+
+# Initialize the client
+refine = Refine(base_url="http://127.0.0.1:3333")
+
+# Create a project
+project_id = refine.create_project("input_file.csv", "My Project")
+
+# Get column names
+columns = refine.get_column_names(project_id)
+print(f"Columns: {columns}")
+
+# Apply an operation
+operation = {
+    "op": "core/column-removal",
+    "columnName": "unwanted_column",
+    "description": "Remove column"
+}
+refine.apply_operation(operation, project_id)
+
+# Export data
+refine.export_data("output_file.tsv", fmt="tsv", project_id=project_id)
+
+# Clean up
+refine.delete_project(project_id)
+```
+
+5. Access Project Metadata:
+
+```python
+# Set metadata
+refine.set_project_metadata("name", "Project Name", project_id)
+
+# Get all projects
+projects = refine.get_all_projects_metadata()
+for pid, metadata in projects.items():
+    print(f"{metadata['name']} (ID: {pid})")
+
+# Find by name
+project_id = refine.get_project_id_by_name("Project Name")
+```
+
+6. Do Batch Operations:
+
+```python
+# Apply multiple operations
+operations = [
+    {"op": "core/column-removal", "columnName": "col1"},
+    {"op": "core/column-removal", "columnName": "col2"}
+]
+refine.apply_operations(operations, project_id)
+
+# Load from file
+refine.apply_operations_from_file("operations.json", project_id, wait=True)
+```
+
+## Examples
+
+### Example 1: Data Pipeline
+
+```python
+from refine_client import Refine
+
+refine = Refine(verbose=True)
+
+# Create project
+project_id = refine.create_project("raw_data.csv", "Processing")
+
+# Transform data
+operations = [
+    {"op": "core/column-removal", "columnName": "temp_field"},
+    {"op": "core/text-transform", "columnName": "email",
+     "expression": "value.toLowerCase()"}
+]
+refine.apply_operations(operations, project_id, wait=True)
+
+# Export
+refine.export_data("processed_data.csv", fmt="csv", project_id=project_id)
+
+# Cleanup
+refine.delete_project(project_id)
+```
+
+### Example 2: Batch Processing
+
+```python
+import os
+from refine_client import Refine
+
+refine = Refine()
+
+for filename in os.listdir("input_dir/"):
+    if filename.endswith(".csv"):
+        project_id = refine.create_project(f"input_dir/{filename}", filename)
+        refine.apply_operations_from_file("operations.json", project_id, wait=True)
+        refine.export_data(f"output_dir/{filename}", fmt="csv", project_id=project_id)
+        refine.delete_project(project_id)
+```
+
+## API Reference
+
+### Initialization
+
+```python
+Refine(base_url=None, verbose=False, silent=False)
+```
+
+Parameters:
+- base_url: OpenRefine server URL (default: http://127.0.0.1:3333)
+- verbose: Enable verbose logging
+- silent: Suppress logging
+
+### Methods
+
+| Method | Description |
+|--------|-------------|
+| create_project(file, name) | Create project |
+| delete_project(project_id) | Delete project |
+| get_all_projects_metadata() | Get all projects |
+| get_project_id_by_name(name) | Find project by name |
+| set_project_metadata(field, value, id) | Update metadata |
+| apply_operation(op, id, wait) | Apply operation |
+| apply_operations(ops, id, wait) | Apply multiple |
+| apply_operations_from_file(file, id, wait) | Load from file |
+| get_models(id) | Get models |
+| get_column_names(id) | Get columns |
+| export_data(file, fmt, id) | Export data |
+| rows_as_list(data) | Convert rows |
+| wait_until_idle(id, delay) | Wait for completion |
+
+## Configuration
+
+### Logging
+
+```python
+# Verbose mode
+refine = Refine(verbose=True)
+
+# Silent mode
+refine = Refine(silent=True)
+```
+
+### Custom Server
+
+```python
+refine = Refine(base_url="http://example.com:3333")
+```
+
+## Troubleshooting
+
+  - **ConnectionError:** Ensure OpenRefine is running, verify the server URL and check network connectivity.
+  - **CSRF token errors:** Check server status and logs.
+  - **FileNotFoundError:** Use absolute paths, verify the file exists and check permissions.
+
+## Support and Contribute
+
+Open an issue on the project's [issues tab](https://github.com/rkraasch/refine-client/issues) on Github.
+
+Or contribute via Github:
+
+  - fork the repository,
+  - create a feature branch,
+  - commit changes,
+  - push to branch,
+  - open Pull Request.
+
+## References
+
+References:
+
+  - [OpenRefine Documentation](https://docs.openrefine.org/)
+  - [OpenRefine REST API](https://docs.openrefine.org/manual/running)
+
+Similar Projects:
+
+  - [paulmakepeace/refine-client-py](https://github.com/paulmakepeace/refine-client-py): OpenRefine Python 2 Client (last update 11 years ago).
+  - [opencultureconsulting/openrefine-client](https://github.com/opencultureconsulting/openrefine-client): OpenRefine Python Client (archived 2024).
+
+## License
+
+This project is released under [CC0 1.0 Universal License](https://creativecommons.org/publicdomain/zero/1.0/).
+
+For a plain text version see this project's [LICENSE file](./LICENSE.md) or visit [creativecommons.org](https://creativecommons.org/2011/04/15/plaintext-versions-of-creative-commons-licenses-and-cc0/).
+

orcli-0.1.0/README.md

@@ -0,0 +1,240 @@
+
+# refine-client
+
+A Python client library for interacting with [OpenRefine](https://openrefine.org/) via its REST API.
+
+For simple project creation, data transformation, metadata management and export operations.
+
+## Features
+
+  - Create and delete projects from local files.
+  - Retrieve and manage project metadata.
+  - Apply OpenRefine operations individually or in batches.
+  - Load operations from JSON files.
+  - Export data in multiple formats (TSV, CSV, JSON).
+  - Retrieve column information and project models, convert and manipulate row data.
+  - Error handling with detailed response logging.
+
+## Installation
+
+### Requirements
+
+Requires Python 3.10+ and a running OpenRefine server instance.
+
+### Quick Start
+
+1. Clone the repository:
+
+```bash
+git clone https://github.com/rkraasch/refine-client.git
+cd refine-client
+```
+
+2. Install dependencies:
+
+```bash
+python -m venv .venv
+source [.venv/bin/activate](.venv/bin/activate)  # Linux and Mac.
+# .venv\Scripts\activate   # Windows.
+pip install -r requirements.txt
+python -m pip list
+deactivate
+```
+
+3. Optionally run tests:
+
+[Download](https://openrefine.org/download) and run OpenRefine, then run the tests as shown below.
+
+```bash
+python -m pytest tests/ -v
+```
+This creates temporary test projects in OpenRefine which should be cleaned up automatically.
+If a project prefixed `pytest_refine-client-integration_` remains visible in the Open project tab (under [#open-project](http://127.0.0.1:3333/#open-project)), something went wrong.
+
+4. Basic Usage:
+
+```python
+from refine_client import Refine
+
+# Initialize the client
+refine = Refine(base_url="http://127.0.0.1:3333")
+
+# Create a project
+project_id = refine.create_project("input_file.csv", "My Project")
+
+# Get column names
+columns = refine.get_column_names(project_id)
+print(f"Columns: {columns}")
+
+# Apply an operation
+operation = {
+    "op": "core/column-removal",
+    "columnName": "unwanted_column",
+    "description": "Remove column"
+}
+refine.apply_operation(operation, project_id)
+
+# Export data
+refine.export_data("output_file.tsv", fmt="tsv", project_id=project_id)
+
+# Clean up
+refine.delete_project(project_id)
+```
+
+5. Access Project Metadata:
+
+```python
+# Set metadata
+refine.set_project_metadata("name", "Project Name", project_id)
+
+# Get all projects
+projects = refine.get_all_projects_metadata()
+for pid, metadata in projects.items():
+    print(f"{metadata['name']} (ID: {pid})")
+
+# Find by name
+project_id = refine.get_project_id_by_name("Project Name")
+```
+
+6. Do Batch Operations:
+
+```python
+# Apply multiple operations
+operations = [
+    {"op": "core/column-removal", "columnName": "col1"},
+    {"op": "core/column-removal", "columnName": "col2"}
+]
+refine.apply_operations(operations, project_id)
+
+# Load from file
+refine.apply_operations_from_file("operations.json", project_id, wait=True)
+```
+
+## Examples
+
+### Example 1: Data Pipeline
+
+```python
+from refine_client import Refine
+
+refine = Refine(verbose=True)
+
+# Create project
+project_id = refine.create_project("raw_data.csv", "Processing")
+
+# Transform data
+operations = [
+    {"op": "core/column-removal", "columnName": "temp_field"},
+    {"op": "core/text-transform", "columnName": "email",
+     "expression": "value.toLowerCase()"}
+]
+refine.apply_operations(operations, project_id, wait=True)
+
+# Export
+refine.export_data("processed_data.csv", fmt="csv", project_id=project_id)
+
+# Cleanup
+refine.delete_project(project_id)
+```
+
+### Example 2: Batch Processing
+
+```python
+import os
+from refine_client import Refine
+
+refine = Refine()
+
+for filename in os.listdir("input_dir/"):
+    if filename.endswith(".csv"):
+        project_id = refine.create_project(f"input_dir/{filename}", filename)
+        refine.apply_operations_from_file("operations.json", project_id, wait=True)
+        refine.export_data(f"output_dir/{filename}", fmt="csv", project_id=project_id)
+        refine.delete_project(project_id)
+```
+
+## API Reference
+
+### Initialization
+
+```python
+Refine(base_url=None, verbose=False, silent=False)
+```
+
+Parameters:
+- base_url: OpenRefine server URL (default: http://127.0.0.1:3333)
+- verbose: Enable verbose logging
+- silent: Suppress logging
+
+### Methods
+
+| Method | Description |
+|--------|-------------|
+| create_project(file, name) | Create project |
+| delete_project(project_id) | Delete project |
+| get_all_projects_metadata() | Get all projects |
+| get_project_id_by_name(name) | Find project by name |
+| set_project_metadata(field, value, id) | Update metadata |
+| apply_operation(op, id, wait) | Apply operation |
+| apply_operations(ops, id, wait) | Apply multiple |
+| apply_operations_from_file(file, id, wait) | Load from file |
+| get_models(id) | Get models |
+| get_column_names(id) | Get columns |
+| export_data(file, fmt, id) | Export data |
+| rows_as_list(data) | Convert rows |
+| wait_until_idle(id, delay) | Wait for completion |
+
+## Configuration
+
+### Logging
+
+```python
+# Verbose mode
+refine = Refine(verbose=True)
+
+# Silent mode
+refine = Refine(silent=True)
+```
+
+### Custom Server
+
+```python
+refine = Refine(base_url="http://example.com:3333")
+```
+
+## Troubleshooting
+
+  - **ConnectionError:** Ensure OpenRefine is running, verify the server URL and check network connectivity.
+  - **CSRF token errors:** Check server status and logs.
+  - **FileNotFoundError:** Use absolute paths, verify the file exists and check permissions.
+
+## Support and Contribute
+
+Open an issue on the project's [issues tab](https://github.com/rkraasch/refine-client/issues) on Github.
+
+Or contribute via Github:
+
+  - fork the repository,
+  - create a feature branch,
+  - commit changes,
+  - push to branch,
+  - open Pull Request.
+
+## References
+
+References:
+
+  - [OpenRefine Documentation](https://docs.openrefine.org/)
+  - [OpenRefine REST API](https://docs.openrefine.org/manual/running)
+
+Similar Projects:
+
+  - [paulmakepeace/refine-client-py](https://github.com/paulmakepeace/refine-client-py): OpenRefine Python 2 Client (last update 11 years ago).
+  - [opencultureconsulting/openrefine-client](https://github.com/opencultureconsulting/openrefine-client): OpenRefine Python Client (archived 2024).
+
+## License
+
+This project is released under [CC0 1.0 Universal License](https://creativecommons.org/publicdomain/zero/1.0/).
+
+For a plain text version see this project's [LICENSE file](./LICENSE.md) or visit [creativecommons.org](https://creativecommons.org/2011/04/15/plaintext-versions-of-creative-commons-licenses-and-cc0/).
+

orcli-0.1.0/pyproject.toml

@@ -0,0 +1,23 @@
+
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "orcli"
+version = "0.1.0"
+description = "A Python client for interacting with OpenRefine"
+readme = "README.md"
+requires-python = ">=3.9"
+dependencies = [
+    "requests>=2.28.0",
+]
+
+[project.optional-dependencies]
+test = [
+    "pytest",
+]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+

orcli-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

orcli-0.1.0/src/orcli/__init__.py

@@ -0,0 +1,4 @@
+from .client import Refine
+
+__all__ = ["Refine"]
+

orcli-0.1.0/src/orcli/client.py

@@ -0,0 +1,267 @@
+# -*- coding: utf-8 -*-
+# client.py
+# (c) 2025 RK, Lic. CC-0
+
+import json, time, logging, requests
+from pathlib import Path
+from urllib.parse import urlparse, parse_qs
+
+logger = logging.getLogger(__name__)
+# Library logger setup – no default handlers, safe for reuse
+if not logger.hasHandlers():
+    null_handler = logging.NullHandler()
+    logger.addHandler(null_handler)
+
+class Refine:
+    """Client interface for interacting with an OpenRefine server through its REST API."""
+
+    DEFAULT_BASE_URL = "http://127.0.0.1:3333"
+
+    def __init__(self, base_url: str | None = None, verbose: bool = False, silent: bool = False) -> None:
+        self.base_url: str = (base_url or self.DEFAULT_BASE_URL).rstrip("/")
+        self.session = requests.Session()
+        self.csrf_token: str = ""
+        self.project_id: str | None = None
+        self.silent: bool = silent
+        self.verbose: bool = verbose
+        logger.setLevel(logging.INFO)
+        if silent:
+            logger.setLevel(logging.ERROR)
+        elif verbose:
+            logger.setLevel(logging.DEBUG)
+            logger.debug("Verbose logging enabled.")
+        stream_handler = logging.StreamHandler()
+        stream_handler.setLevel(logging.DEBUG)
+        formatter = logging.Formatter(
+            "%(asctime)s - %(levelname)s - %(name)s - %(message)s"
+        )
+        stream_handler.setFormatter(formatter)
+        logger.addHandler(stream_handler)
+        try:
+            response = self.session.get(f"{self.base_url}")
+            response.raise_for_status()
+            logger.info(f"Connected to OpenRefine server at {self.base_url}")
+        except Exception as e:
+            logger.error(f"Failed to connect to OpenRefine server at {self.base_url}: {e}")
+            raise ConnectionError(f"Could not connect to OpenRefine server at {self.base_url}") from e
+
+
+    def _get_csrf_token(self) -> str:
+        """Fetch CSRF token from server and cache it."""
+        if not self.csrf_token:
+            response = self.session.get(f"{self.base_url}/command/core/get-csrf-token")
+            response.raise_for_status()
+            token = response.json().get("token")
+            if not token:
+                logger.error("Failed to acquire CSRF token.")
+                raise ValueError("CSRF token retrieval failed.")
+            self.csrf_token = token
+            logger.debug("CSRF token retrieved successfully.")
+        return self.csrf_token
+
+    def _get_id(self, project_id: str | None) -> str:
+        """Resolve a project ID or use the currently loaded one."""
+        if project_id:
+            return project_id
+        if not self.project_id:
+            raise ValueError("No project ID specified or loaded.")
+        return self.project_id
+
+    def create_project(self, project_file: str, project_name: str) -> str:
+        """Create a new project from a local file."""
+        self._get_csrf_token()
+        file_path = Path(project_file)
+        if not file_path.is_file():
+            logger.error(f"File not found: {project_file}")
+            raise FileNotFoundError(project_file)
+
+        logger.info(f"Creating OpenRefine project: {project_name}")
+        with open(file_path, "rb") as f:
+            files = {"project-file": (file_path.name, f)}
+            params = {"project-name": project_name, "csrf_token": self.csrf_token}
+            response = self.session.post(
+                f"{self.base_url}/command/core/create-project-from-upload",
+                params=params,
+                files=files,
+            )
+
+        if response.status_code != 200:
+            self._save_error_response(response, "Project creation failed.")
+            raise RuntimeError(f"Failed to create project: {response.status_code}")
+
+        parsed_url = urlparse(response.url)
+        self.project_id = parse_qs(parsed_url.query).get("project", [None])[0]
+        if not self.project_id:
+            logger.error("Failed to parse project ID from server response.")
+            raise ValueError("Project ID not found in response.")
+        logger.info(f"Project created successfully: {self.project_id}")
+        return self.project_id
+
+    def apply_operations_from_file(self, operations_file: str, project_id: str | None = None, wait: bool = False):
+        """Apply JSON operations from a given file."""
+        file_path = Path(operations_file)
+        if not file_path.is_file():
+            raise FileNotFoundError(operations_file)
+        with open(file_path, "r", encoding="utf-8") as f:
+            operations = json.load(f)
+        return self.apply_operations(operations, project_id, wait)
+
+    def apply_operations(self, operations: list, project_id: str | None = None, wait: bool = False):
+        """Apply a list of OpenRefine operations."""
+        results = []
+        for op in operations:
+            results.append(self.apply_operation(op, project_id, wait))
+        return results
+
+    def apply_operation(self, operation: dict, project_id: str | None = None, wait: bool = False):
+        """Apply a single operation to a project."""
+        project = self._get_id(project_id)
+        self._get_csrf_token()
+        payload = {"operations": json.dumps([operation]), "csrf_token": self.csrf_token}
+        response = self.session.post(
+            f"{self.base_url}/command/core/apply-operations",
+            params={"project": project},
+            data=payload,
+        )
+        if response.status_code == 200:
+            if self.verbose:
+                desc = operation.get("description", "Unnamed operation")
+                logger.info(f"Applied operation: {desc}")
+        else:
+            self._save_error_response(response, "Operation failed.")
+            raise RuntimeError(f"Operation failed: {response.status_code}")
+        if wait:
+            self.wait_until_idle(project)
+        return response.json()
+
+    def export_data(self, output_file: str = "exported_data.tsv", fmt: str = "tsv", project_id: str | None = None):
+        """Export project data to file in specified format."""
+        project = self._get_id(project_id)
+        self._get_csrf_token()
+        params = {"project": project, "format": fmt, "csrf_token": self.csrf_token}
+        response = self.session.post(f"{self.base_url}/command/core/export-rows", params=params)
+        if response.status_code != 200:
+            self._save_error_response(response, "Export failed.")
+            raise RuntimeError(f"Export failed: {response.status_code}")
+
+        with open(output_file, "wb") as f:
+            f.write(response.content)
+        logger.info(f"Exported data to {output_file}")
+
+    def get_models(self,project_id: str|None = None):
+        """Get project models."""
+        project = self._get_id(project_id)
+        self._get_csrf_token()
+
+        response = self.session.get(f'{self.base_url}/command/core/get-models',
+                                     params={'project': project, 'csrf_token': self.csrf_token})
+        if response.status_code == 200:
+            logger.info(f"Get models {project}")
+            data = response.json()
+            return data
+        else:
+            self._save_error_response(response, 'Get models failed.')
+            raise Exception(f'Get models failed: {response.status_code}')
+
+    def get_column_names(self,project_id: str|None = None):
+        """Get column names from project models."""
+        project = self._get_id(project_id)
+        self._get_csrf_token()
+
+        response = self.session.get(f'{self.base_url}/command/core/get-models',
+                                     params={'project': project, 'csrf_token': self.csrf_token})
+        if response.status_code == 200:
+            logger.info(f"Get column names {project}")
+            data = response.json()["columnModel"]
+            data = [col["name"] for col in data["columns"]]
+            return data
+        else:
+            self._save_error_response(response, 'Get column names failed.')
+            raise Exception(f'Get column names failed: {response.status_code}')
+
+    def delete_project(self, project_id: str | None = None):
+        """Delete a specific project."""
+        project = self._get_id(project_id)
+        self._get_csrf_token()
+        response = self.session.post(
+            f"{self.base_url}/command/core/delete-project",
+            data={"project": project, "csrf_token": self.csrf_token},
+        )
+        if response.status_code == 200:
+            logger.info(f"Deleted project {project}")
+        else:
+            self._save_error_response(response, "Project deletion failed.")
+            raise RuntimeError(f"Deletion failed: {response.status_code}")
+
+    def _save_error_response(self, response, message: str):
+        """Save HTML error responses for debugging."""
+        filename = "response.html"
+        logger.error(f"{message}. Status: {response.status_code}")
+        with open(filename, "w", encoding="utf-8") as f:
+            f.write(response.text)
+        logger.debug(f"Saved error details to {filename}")
+
+    def get_all_projects_metadata(self) -> dict:
+        """Retrieve metadata for all OpenRefine projects."""
+        response = self.session.get(f"{self.base_url}/command/core/get-all-project-metadata")
+        if response.status_code != 200:
+            self._save_error_response(response, "Metadata retrieval failed.")
+            raise RuntimeError("Failed to retrieve metadata.")
+        logger.info("Retrieved all project metadata.")
+        return response.json().get("projects", {})
+
+    def get_project_id_by_name(self, name: str = "") -> str | None :
+        """Get project id by project name"""
+        projects = self.get_all_projects_metadata()
+        for pid, metadata in projects.items():
+            if metadata["name"] == name:
+                logger.info("Retrieved project id.")
+                return pid
+        return None
+
+    def set_project_metadata(self, field_name: str, value: str, id: str | None = None) -> None:
+        project = self._get_id(id)
+        self._get_csrf_token()
+        data = {
+            'project': project,
+            'name': field_name,
+            'value': value,
+            'csrf_token': self.csrf_token
+        }
+        response = self.session.post(f'{self.base_url}/command/core/set-project-metadata', data=data)
+        if response.status_code == 200:
+            if self.verbose:
+                logger.info(f"Project metadata '{field_name}' updated to '{value}'.")
+        else:
+            self._save_error_response(response, "Failed to update project metadata")
+            raise Exception(f"Failed to update project metadata: {response.status_code}")
+
+    def wait_until_idle(self, project_id: str | None = None, delay: float = 0.5):
+        """Wait until OpenRefine finishes processing all jobs."""
+        project = self._get_id(project_id)
+        while True:
+            response = self.session.get(f"{self.base_url}/command/core/get-processes", params={"project": project})
+            processes = response.json().get("processes", [])
+            if processes:
+                time.sleep(delay)
+            else:
+                logger.debug("OpenRefine is idle.")
+                break
+
+    def rows_as_list(self, data:dict)->list:
+        """Return rows as list"""
+        if 'rows' in data:
+            rows_list = [
+                [cell.get("v", None) for cell in row["cells"]]
+                for row in data["rows"]
+            ]
+            return rows_list
+        else:
+            return []
+
+if __name__ == "__main__":
+    refine = Refine(verbose=True)
+    # Example usage:
+    # project_id = refine.create_project("data.csv", "My Project")
+    # refine.apply_operations_from_file("operations.json", project_id, wait=True)
+    # refine.export_data("output.tsv", "tsv", project_id)

orcli-0.1.0/src/orcli.egg-info/PKG-INFO

@@ -0,0 +1,252 @@
+Metadata-Version: 2.4
+Name: orcli
+Version: 0.1.0
+Summary: A Python client for interacting with OpenRefine
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: requests>=2.28.0
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Dynamic: license-file
+
+
+# refine-client
+
+A Python client library for interacting with [OpenRefine](https://openrefine.org/) via its REST API.
+
+For simple project creation, data transformation, metadata management and export operations.
+
+## Features
+
+  - Create and delete projects from local files.
+  - Retrieve and manage project metadata.
+  - Apply OpenRefine operations individually or in batches.
+  - Load operations from JSON files.
+  - Export data in multiple formats (TSV, CSV, JSON).
+  - Retrieve column information and project models, convert and manipulate row data.
+  - Error handling with detailed response logging.
+
+## Installation
+
+### Requirements
+
+Requires Python 3.10+ and a running OpenRefine server instance.
+
+### Quick Start
+
+1. Clone the repository:
+
+```bash
+git clone https://github.com/rkraasch/refine-client.git
+cd refine-client
+```
+
+2. Install dependencies:
+
+```bash
+python -m venv .venv
+source [.venv/bin/activate](.venv/bin/activate)  # Linux and Mac.
+# .venv\Scripts\activate   # Windows.
+pip install -r requirements.txt
+python -m pip list
+deactivate
+```
+
+3. Optionally run tests:
+
+[Download](https://openrefine.org/download) and run OpenRefine, then run the tests as shown below.
+
+```bash
+python -m pytest tests/ -v
+```
+This creates temporary test projects in OpenRefine which should be cleaned up automatically.
+If a project prefixed `pytest_refine-client-integration_` remains visible in the Open project tab (under [#open-project](http://127.0.0.1:3333/#open-project)), something went wrong.
+
+4. Basic Usage:
+
+```python
+from refine_client import Refine
+
+# Initialize the client
+refine = Refine(base_url="http://127.0.0.1:3333")
+
+# Create a project
+project_id = refine.create_project("input_file.csv", "My Project")
+
+# Get column names
+columns = refine.get_column_names(project_id)
+print(f"Columns: {columns}")
+
+# Apply an operation
+operation = {
+    "op": "core/column-removal",
+    "columnName": "unwanted_column",
+    "description": "Remove column"
+}
+refine.apply_operation(operation, project_id)
+
+# Export data
+refine.export_data("output_file.tsv", fmt="tsv", project_id=project_id)
+
+# Clean up
+refine.delete_project(project_id)
+```
+
+5. Access Project Metadata:
+
+```python
+# Set metadata
+refine.set_project_metadata("name", "Project Name", project_id)
+
+# Get all projects
+projects = refine.get_all_projects_metadata()
+for pid, metadata in projects.items():
+    print(f"{metadata['name']} (ID: {pid})")
+
+# Find by name
+project_id = refine.get_project_id_by_name("Project Name")
+```
+
+6. Do Batch Operations:
+
+```python
+# Apply multiple operations
+operations = [
+    {"op": "core/column-removal", "columnName": "col1"},
+    {"op": "core/column-removal", "columnName": "col2"}
+]
+refine.apply_operations(operations, project_id)
+
+# Load from file
+refine.apply_operations_from_file("operations.json", project_id, wait=True)
+```
+
+## Examples
+
+### Example 1: Data Pipeline
+
+```python
+from refine_client import Refine
+
+refine = Refine(verbose=True)
+
+# Create project
+project_id = refine.create_project("raw_data.csv", "Processing")
+
+# Transform data
+operations = [
+    {"op": "core/column-removal", "columnName": "temp_field"},
+    {"op": "core/text-transform", "columnName": "email",
+     "expression": "value.toLowerCase()"}
+]
+refine.apply_operations(operations, project_id, wait=True)
+
+# Export
+refine.export_data("processed_data.csv", fmt="csv", project_id=project_id)
+
+# Cleanup
+refine.delete_project(project_id)
+```
+
+### Example 2: Batch Processing
+
+```python
+import os
+from refine_client import Refine
+
+refine = Refine()
+
+for filename in os.listdir("input_dir/"):
+    if filename.endswith(".csv"):
+        project_id = refine.create_project(f"input_dir/{filename}", filename)
+        refine.apply_operations_from_file("operations.json", project_id, wait=True)
+        refine.export_data(f"output_dir/{filename}", fmt="csv", project_id=project_id)
+        refine.delete_project(project_id)
+```
+
+## API Reference
+
+### Initialization
+
+```python
+Refine(base_url=None, verbose=False, silent=False)
+```
+
+Parameters:
+- base_url: OpenRefine server URL (default: http://127.0.0.1:3333)
+- verbose: Enable verbose logging
+- silent: Suppress logging
+
+### Methods
+
+| Method | Description |
+|--------|-------------|
+| create_project(file, name) | Create project |
+| delete_project(project_id) | Delete project |
+| get_all_projects_metadata() | Get all projects |
+| get_project_id_by_name(name) | Find project by name |
+| set_project_metadata(field, value, id) | Update metadata |
+| apply_operation(op, id, wait) | Apply operation |
+| apply_operations(ops, id, wait) | Apply multiple |
+| apply_operations_from_file(file, id, wait) | Load from file |
+| get_models(id) | Get models |
+| get_column_names(id) | Get columns |
+| export_data(file, fmt, id) | Export data |
+| rows_as_list(data) | Convert rows |
+| wait_until_idle(id, delay) | Wait for completion |
+
+## Configuration
+
+### Logging
+
+```python
+# Verbose mode
+refine = Refine(verbose=True)
+
+# Silent mode
+refine = Refine(silent=True)
+```
+
+### Custom Server
+
+```python
+refine = Refine(base_url="http://example.com:3333")
+```
+
+## Troubleshooting
+
+  - **ConnectionError:** Ensure OpenRefine is running, verify the server URL and check network connectivity.
+  - **CSRF token errors:** Check server status and logs.
+  - **FileNotFoundError:** Use absolute paths, verify the file exists and check permissions.
+
+## Support and Contribute
+
+Open an issue on the project's [issues tab](https://github.com/rkraasch/refine-client/issues) on Github.
+
+Or contribute via Github:
+
+  - fork the repository,
+  - create a feature branch,
+  - commit changes,
+  - push to branch,
+  - open Pull Request.
+
+## References
+
+References:
+
+  - [OpenRefine Documentation](https://docs.openrefine.org/)
+  - [OpenRefine REST API](https://docs.openrefine.org/manual/running)
+
+Similar Projects:
+
+  - [paulmakepeace/refine-client-py](https://github.com/paulmakepeace/refine-client-py): OpenRefine Python 2 Client (last update 11 years ago).
+  - [opencultureconsulting/openrefine-client](https://github.com/opencultureconsulting/openrefine-client): OpenRefine Python Client (archived 2024).
+
+## License
+
+This project is released under [CC0 1.0 Universal License](https://creativecommons.org/publicdomain/zero/1.0/).
+
+For a plain text version see this project's [LICENSE file](./LICENSE.md) or visit [creativecommons.org](https://creativecommons.org/2011/04/15/plaintext-versions-of-creative-commons-licenses-and-cc0/).
+

orcli-0.1.0/src/orcli.egg-info/SOURCES.txt

@@ -0,0 +1,10 @@
+LICENSE.md
+README.md
+pyproject.toml
+src/orcli/__init__.py
+src/orcli/client.py
+src/orcli.egg-info/PKG-INFO
+src/orcli.egg-info/SOURCES.txt
+src/orcli.egg-info/dependency_links.txt
+src/orcli.egg-info/requires.txt
+src/orcli.egg-info/top_level.txt

orcli-0.1.0/src/orcli.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

orcli-0.1.0/src/orcli.egg-info/requires.txt

@@ -0,0 +1,4 @@
+requests>=2.28.0
+
+[test]
+pytest

orcli-0.1.0/src/orcli.egg-info/top_level.txt

@@ -0,0 +1 @@
+orcli