aboa 0.1.0__tar.gz

aboa-0.1.0/AUTHORS

@@ -0,0 +1,5 @@
+ABOA was created by Daniel Brosnan Blázquez.
+
+Major contributing authors include:
+
+- Daniel Brosnan Blázquez

aboa-0.1.0/LICENSE

@@ -0,0 +1,19 @@
+Copyright 2026-2026 BOA authors and contributors <see AUTHORS file>.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
+of the Software, and to permit persons to whom the Software is furnished to do
+so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

aboa-0.1.0/MANIFEST.in

@@ -0,0 +1,3 @@
+include README.md LICENSE AUTHORS
+prune src/tests
+global-exclude __pycache__ *.py[cod] .coverage

aboa-0.1.0/PKG-INFO

@@ -0,0 +1,374 @@
+Metadata-Version: 2.4
+Name: aboa
+Version: 0.1.0
+Summary: Archive for Business Operations Analysis
+Author: Daniel Brosnan Blázquez
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/Daniel-Brosnan-Blazquez/aboa
+Project-URL: Repository, https://github.com/Daniel-Brosnan-Blazquez/aboa
+Project-URL: Issues, https://github.com/Daniel-Brosnan-Blazquez/aboa/issues
+Keywords: archive,inventory,postgresql,retention
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: POSIX
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: System :: Archiving
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: AUTHORS
+Requires-Dist: sqlalchemy==1.3.22
+Requires-Dist: psycopg2-binary==2.9.11
+Requires-Dist: python-dateutil==2.9.0.post0
+Requires-Dist: lxml==6.0.2
+Provides-Extra: tests
+Requires-Dist: pytest==8.4.2; extra == "tests"
+Requires-Dist: pytest-cov==7.0.0; extra == "tests"
+Dynamic: license-file
+
+# ABOA
+
+Archive for Business Operations Analysis.
+
+ABOA stores files on a POSIX filesystem and keeps a PostgreSQL inventory of the
+archived payloads, metadata, configuration history, root-directory history,
+operation failures, trash queues, retention cleanup, and recovery state. The
+current implementation exposes both command line tools and Python interfaces
+through `aboa.engine.engine.Engine` and `aboa.engine.query.Query`.
+
+## Contents
+
+- [What ABOA Provides](#what-aboa-provides)
+- [Quick Start With Docker](#quick-start-with-docker)
+- [Local Python Setup](#local-python-setup)
+- [Install From PyPI](#install-from-pypi)
+- [Runtime Configuration](#runtime-configuration)
+- [Archive Configuration XML](#archive-configuration-xml)
+- [Command Line Tools](#command-line-tools)
+- [Python API](#python-api)
+- [Operation Flow Schematics](#operation-flow-schematics)
+- [Data Model](#data-model)
+- [Development Layout](#development-layout)
+- [Tests](#tests)
+- [Publishing](#publishing)
+
+## What ABOA Provides
+
+- Archive files into a managed root directory using configurable routing rules.
+- Keep inventory metadata in SQLAlchemy/PostgreSQL tables.
+- Track active archive roots and active XML archive-configuration history.
+- Detect duplicate archived file names and route archive failures into the
+  internal `error` area when the input can still be brought under ABOA control.
+- Retrieve inventory rows and optionally copy payloads to a destination folder.
+- Logically delete files, move payloads to trash, permanently delete payloads, or
+  purge inventory entries when no managed payload remains.
+- Recover files from trash or from logical-only deletion.
+- Apply retention cleanup and final trash removal cleanup.
+- Record failed operations in `archive_operations` while successful operations
+  are written only to the log.
+
+## Quick Start With Docker
+
+The development compose file starts an ABOA container and a PostgreSQL/PostGIS
+database container. The source checkout is mounted into the ABOA container at
+`/aboa`.
+
+```bash
+docker compose -f compose_dev.yml up -d --build
+docker compose -f compose_dev.yml exec aboa initialize_aboa_ddbb.sh
+docker compose -f compose_dev.yml exec aboa aboa_archive.py --file /aboa/src/tests/inputs/sample.txt
+docker compose -f compose_dev.yml exec aboa aboa_retrieve.py --name sample.txt --list
+```
+
+`initialize_aboa_ddbb.sh` calls `aboa_init.py -y` and recreates the configured
+database from `src/aboa/datamodel/aboa_data_model.sql`, so it deletes existing ABOA
+inventory data for that database.
+
+## Local Python Setup
+
+For local development outside Docker, use Python 3.9 or newer and install from
+the repository root:
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install -e ".[tests]"
+```
+
+Set the runtime paths before importing or running ABOA:
+
+```bash
+export ABOA_RESOURCES_PATH="$PWD/src/aboa/config"
+export ABOA_LOG_PATH="$PWD/log"
+export ABOA_DEFAULT_ARCHIVE_PATH="/tmp/aboa_archive"
+mkdir -p "$ABOA_LOG_PATH" "$ABOA_DEFAULT_ARCHIVE_PATH"
+```
+
+## Install From PyPI
+
+Once the first release is published, install ABOA in a virtual environment on a
+POSIX system with Python 3.9 or newer:
+
+```bash
+python -m pip install aboa
+```
+
+ABOA requires a PostgreSQL server and the PostgreSQL `psql` command to initialize
+the database. Copy the bundled defaults to a writable configuration directory:
+
+```bash
+export ABOA_RESOURCES_PATH="$PWD/aboa-config"
+export ABOA_LOG_PATH="$PWD/log"
+export ABOA_DEFAULT_ARCHIVE_PATH="$PWD/archive"
+mkdir -p "$ABOA_RESOURCES_PATH" "$ABOA_LOG_PATH" "$ABOA_DEFAULT_ARCHIVE_PATH"
+python -c 'import os, shutil, aboa; from pathlib import Path; shutil.copytree(Path(aboa.__file__).parent / "config", os.environ["ABOA_RESOURCES_PATH"], dirs_exist_ok=True)'
+```
+
+Edit `datamodel.json` for the database connection and set `root_directory` in
+`archive_configurations.xml` to the archive directory. Then run `aboa_init.py`
+and follow its confirmation prompt. Initialization requires PostgreSQL
+administrator access and recreates the configured database.
+
+## Runtime Configuration
+
+ABOA reads JSON and XML runtime resources from `ABOA_RESOURCES_PATH`, falling
+back to bundled defaults when it is unset. The XML schema is bundled with the
+`aboa` package.
+
+Required environment variables:
+
+- `ABOA_LOG_PATH`: directory where rotating log files are written.
+
+Optional environment variables:
+
+- `ABOA_RESOURCES_PATH`: directory containing `datamodel.json`,
+  `engine.json`, and `archive_configurations.xml`; defaults to the packaged
+  configuration directory.
+- `ABOA_DEFAULT_ARCHIVE_PATH`: fallback archive root used when the XML archive
+  configuration cannot be activated.
+- `ABOA_DDBB_HOST`: overrides the host from `datamodel.json`.
+- `ABOA_LOG_LEVEL`: overrides the log level from `engine.json`.
+- `ABOA_STREAM_LOG`: enables stream logging in addition to the rotating file log.
+- `ABOA_LOG_MAX_BYTES` and `ABOA_LOG_MAX_BACKUP`: override log rotation limits.
+
+`src/aboa/config/engine.json` configures logging and archive cleanup behavior:
+
+```json
+{
+  "LOG": {
+    "LEVEL": "INFO",
+    "MAX_BYTES": 50000000,
+    "MAX_BACKUP": 30
+  },
+  "ARCHIVE": {
+    "FINAL_REMOVAL_DELAY_DAYS": 30
+  }
+}
+```
+
+Archived payloads are stored below the active root using:
+
+```text
+<root_directory>/<target_directory>/<YEAR>/<MONTH>/<DAY>/<file_name>
+```
+
+The reserved target directories are `unknown`, `error`, and `trash`.
+
+## Archive Configuration XML
+
+The default configuration lives at `src/aboa/config/archive_configurations.xml`.
+ABOA validates this file with `src/aboa/schemas/aboa_archive_configurations.xsd`.
+
+```xml
+<archive_configurations root_directory="/tmp/aboa_archive">
+  <archive_configuration file_group="invoices">
+    <file_mask>*.pdf</file_mask>
+    <file_directory>invoices</file_directory>
+    <retention_policy active="true" key="archive_date">P180D</retention_policy>
+  </archive_configuration>
+  <retention_policies>
+    <retention_policy active="true" key="validity_stop_date">P365D</retention_policy>
+  </retention_policies>
+</archive_configurations>
+```
+
+Archive rules are evaluated by file mask. A matching rule sets the target
+directory, file group, optional processor module, and optional retention policy.
+If no rule matches, ABOA stores the file in the internal `unknown` area with file
+group `unknown`.
+
+Processors are importable Python modules that expose `process(file_path)` and
+return a metadata dictionary. Supported metadata includes file type, class,
+version, validity dates, generation date, and expiration date.
+
+Retention policies use XML duration values such as `P30D` and can use
+`generation_date`, `reception_date`, `archive_date`, or `validity_stop_date` as
+the base key.
+
+## Command Line Tools
+
+The package installs these console commands:
+
+```bash
+aboa_init.py [-f /path/to/aboa_data_model.sql] [-y]
+aboa_archive.py --file /path/to/file [--delete] [--expiration-date DATETIME]
+aboa_retrieve.py [filters] [--list] [--destination-path /path/to/output]
+aboa_delete.py [filters] [--physical | --permanent] [--purge-entry] [--reason TEXT]
+aboa_recover.py [archived-file filters | trash filters] [--list]
+aboa_clean_up.py [--dry-run]
+aboa_clean_up.py --final-removal [--dry-run]
+aboa_clean_up.py --empty-trash [--dry-run]
+```
+
+Common examples:
+
+```bash
+aboa_archive.py --file /data/incoming/report.txt --delete
+aboa_archive.py --file /data/incoming/invoice.pdf --expiration-date 2026-12-31T00:00:00
+aboa_retrieve.py --name "%.txt" --order-by archive_date --descending --list
+aboa_retrieve.py --uuid <file_uuid> --destination-path /tmp/retrieved
+aboa_delete.py --uuid <file_uuid> --reason manual_delete
+aboa_delete.py --uuid <file_uuid> --physical
+aboa_delete.py --uuid <file_uuid> --permanent --purge-entry
+aboa_recover.py --uuid <file_uuid>
+aboa_recover.py --trash-uuid <file_to_remove_uuid>
+aboa_clean_up.py --dry-run
+aboa_clean_up.py --final-removal
+```
+
+Retrieve, delete, and recover commands share many inventory filters, including:
+
+- Text filters: `--uuid`, `--name`, `--path`, `--root-directory-uuid`,
+  `--file-group`, `--file-type`, `--file-class`, `--file-version`,
+  `--archive-configuration-uuid`, `--delete-archive-configuration-uuid`,
+  `--removal-justification`, and `--checksum`.
+- Date filters: `--reception-date`, `--archive-date`, `--last-access-date`,
+  `--validity-start-date`, `--validity-stop-date`, `--generation-date`,
+  `--expiration-date`, and `--removal-date`.
+- Numeric and boolean filters: `--file-size`, `--available`, and
+  `--physically-available`.
+- Trash recovery filters: `--trash-uuid`, `--trash-path`,
+  `--trash-root-directory-uuid`, and `--trash-removal-date`.
+
+Filter values may use operators such as `==`, `!=`, `<`, `<=`, `>`, and `>=`
+for arithmetic/date filters. Text filters also expose `--<filter>-op` options
+for text operators. Run any command with `--help` for the exact supported options.
+
+## Python API
+
+Use `Engine` for mutating operations and `Query` for read-only inventory access.
+
+```python
+from aboa.engine.engine import Engine
+from aboa.engine.query import Query
+
+engine = Engine()
+try:
+    engine.archive_file("/data/incoming/report.txt", delete=True)
+    rows = engine.retrieve_files(
+        filters={"names": {"filter": "%.txt", "op": "like"}},
+        order_by={"field": "archive_date", "descending": True},
+    )
+finally:
+    engine.close_session()
+
+query = Query()
+try:
+    active_root = query.get_active_root_directory()
+    failures = query.get_archive_operations(
+        operations={"filter": "archive", "op": "like"}
+    )
+finally:
+    query.close_session()
+```
+
+Important write-side methods include `archive_file`, `retrieve_files`,
+`delete_files`, `delete_archived_file_entries`, `recover_files_from_trash`,
+`move_archived_file_to_trash`, and `delete_archived_file_permanently`.
+
+## Operation Flow Schematics
+
+### Shared Operation Rules
+
+![ABOA shared operation rules](https://raw.githubusercontent.com/Daniel-Brosnan-Blazquez/aboa/main/doc/fig/aboa_shared_operation_rules.svg)
+
+### Load Archive Configuration
+
+![ABOA load archive configuration flow](https://raw.githubusercontent.com/Daniel-Brosnan-Blazquez/aboa/main/doc/fig/aboa_load_archive_configuration_flow.svg)
+
+### Archive Operation
+
+![ABOA archive operation flow](https://raw.githubusercontent.com/Daniel-Brosnan-Blazquez/aboa/main/doc/fig/aboa_archive_operation_flow.svg)
+
+### Retrieve Operation
+
+![ABOA retrieve operation flow](https://raw.githubusercontent.com/Daniel-Brosnan-Blazquez/aboa/main/doc/fig/aboa_retrieve_operation_flow.svg)
+
+### Delete Operation
+
+![ABOA delete operation flow](https://raw.githubusercontent.com/Daniel-Brosnan-Blazquez/aboa/main/doc/fig/aboa_delete_operation_flow.svg)
+
+### Physical Deletion And Trash
+
+![ABOA physical deletion and trash flow](https://raw.githubusercontent.com/Daniel-Brosnan-Blazquez/aboa/main/doc/fig/aboa_physical_deletion_trash_flow.svg)
+
+### Recovery Operation
+
+![ABOA recovery operation flow](https://raw.githubusercontent.com/Daniel-Brosnan-Blazquez/aboa/main/doc/fig/aboa_recovery_operation_flow.svg)
+
+### Retention Cleanup
+
+![ABOA retention cleanup flow](https://raw.githubusercontent.com/Daniel-Brosnan-Blazquez/aboa/main/doc/fig/aboa_retention_cleanup_flow.svg)
+
+### Final Removal Cleanup
+
+![ABOA final removal cleanup flow](https://raw.githubusercontent.com/Daniel-Brosnan-Blazquez/aboa/main/doc/fig/aboa_final_removal_cleanup_flow.svg)
+
+## Data Model
+
+The DDBB model, stored in `src/aboa/datamodel/aboa_data_model.dbm`, is built using pgModeler. The tool is then used to generate the SQL instructions, stored in `src/aboa/datamodel/aboa_data_model.sql`, to initialize the DDBB.
+
+![ABOA data model](https://raw.githubusercontent.com/Daniel-Brosnan-Blazquez/aboa/main/doc/fig/aboa_data_model.png)
+
+Main inventory tables:
+
+- `archive_root_directories`: active and historical archive roots.
+- `archive_configurations`: active and historical XML configuration snapshots.
+- `archived_files`: metadata for archived payloads and logical/physical
+  availability.
+- `archive_operations`: failed operation records.
+- `files_to_be_removed`: trash payloads queued for final removal.
+
+## Development Layout
+
+- `src/aboa/datamodel`: SQLAlchemy model, database configuration, exported SQL,
+  and pgModeler model.
+- `src/aboa/engine`: archive, query, configuration, retention, and CLI logic.
+- `src/aboa/processors`: metadata processor contract.
+- `src/aboa/config`: default runtime configuration.
+- `src/aboa/schemas`: bundled XML schemas.
+- `src/aboa/scripts`: console script entry points and database initialization helpers.
+- `src/tests`: pytest-based test suite.
+- `development_plans`: design notes, requirements, and operation flow details.
+- `doc/fig`: data model and operation-flow diagrams.
+
+## Tests
+
+Install the test extra and run the suite from the repository root:
+
+```bash
+python -m pip install -e ".[tests]"
+export ABOA_RESOURCES_PATH="$PWD/src/aboa/config"
+export ABOA_LOG_PATH="$PWD/log"
+export ABOA_DEFAULT_ARCHIVE_PATH="/tmp/aboa_archive"
+mkdir -p "$ABOA_LOG_PATH" "$ABOA_DEFAULT_ARCHIVE_PATH"
+python -m pytest src/tests
+```
+
+## Publishing
+
+See [the publishing guide](https://github.com/Daniel-Brosnan-Blazquez/aboa/blob/main/doc/publishing.md)
+for local build checks, PyPI Trusted Publisher setup, and the GitHub release
+workflow. Package metadata and the release version live in `pyproject.toml`.