enterprise-crawler-framework 1.0.0__tar.gz

enterprise_crawler_framework-1.0.0/CHANGELOG.md

@@ -0,0 +1,88 @@
+# Changelog
+
+All notable changes to Enterprise Crawler Framework are documented in this
+file.
+
+The project follows Semantic Versioning.
+
+## [1.0.0] - 2026-08-20
+
+### Added
+
+- Stable `BaseBot` lifecycle contract.
+- Top-level `Crawler` runtime facade.
+- Public `ExecutionResult` and `ExecutionStatus` contracts.
+- Cooperative shutdown support.
+- Runtime state and execution snapshots.
+- HTTP client with retry and circuit-breaker behavior.
+- Session management.
+- Streaming downloader with size and SHA-256 validation.
+- Opt-in local storage.
+- Atomic file writing.
+- SQLite-backed local state storage.
+- Configuration models and loading helpers.
+- JSON processing.
+- XML processing.
+- CSV processing.
+- HTML processing.
+- Feed processing.
+- PDF processing primitives.
+- Composable processing pipeline.
+- Plugin registry.
+- Plugin manager.
+- Plugin loader.
+- Metadata-only plugin discovery.
+- Python entry-point plugin discovery.
+- Plugin autoload composition.
+- In-memory event queue.
+- Durable SQLite event queue.
+- Claim-token ownership.
+- Queue leases and expired-lease recovery.
+- Event dispatcher.
+- Event worker.
+- Retry classification policy.
+- Exponential retry backoff.
+- Durable scheduled retries.
+- Configurable retry delay caps.
+- Injectable retry jitter.
+- In-memory dead-letter queue.
+- Durable SQLite dead-letter queue.
+- CLI `version` command.
+- CLI `doctor` command.
+- CLI plugin discovery commands.
+- Top-level public API for the primary bot workflow.
+- Runnable `examples/basic_bot/hello_bot.py` quickstart.
+- Wheel and source-distribution build support.
+- Dynamic package versioning from `enterprise_crawler.version.__version__`.
+- Release artifact clean-install smoke testing.
+- GitHub Actions test, lint, and release gates.
+
+### Guarantees
+
+- `BaseBot.run()` is framework-owned and cannot be overridden by subclasses.
+- Injected runtime dependencies are not automatically owned by consumers.
+- Storage and plugins are opt-in.
+- Plugin discovery does not import third-party plugin code.
+- Event acknowledgement and negative acknowledgement require valid ownership.
+- Stale queue claim tokens cannot finalize recovered messages.
+- Lease expiration and retry scheduling remain separate concepts.
+- Scheduled retries survive SQLite reopen and process restart.
+- Retry waiting does not increment delivery count.
+- Dead-letter transfer is fail-closed.
+- Retry exhaustion cannot create an infinite immediate requeue loop.
+- Default retry jitter is disabled and preserves deterministic behavior.
+- Wheel and source-distribution metadata use the same runtime version source.
+
+### Verification
+
+The final pre-release local regression baseline for 1.0.0 is:
+
+- 1629 unit tests passed.
+- 103 integration tests passed.
+- 2 integration tests skipped.
+- 1732 total tests passed.
+- 0 failures.
+- 0 errors.
+
+Release artifacts were additionally verified through clean wheel and source
+distribution installations outside the repository.

enterprise_crawler_framework-1.0.0/CONTRIBUTING.md

@@ -0,0 +1,24 @@
+# Contributing to Enterprise Crawler Framework
+
+Thank you for contributing to Enterprise Crawler Framework.
+
+The project aims to remain a small, reliable, domain-independent foundation for
+building production data-collection systems.
+
+Changes should preserve clear subsystem boundaries and fail-closed behavior.
+
+## Development Requirements
+
+- Python 3.11 or newer
+- pip
+- Git
+
+The framework runtime dependency set is intentionally small.
+
+## Local Setup
+
+Clone the repository and install it in editable mode:
+
+```bash
+python -m pip install --upgrade pip
+python -m pip install -e .

enterprise_crawler_framework-1.0.0/MANIFEST.in

@@ -0,0 +1,10 @@
+include README.md
+include LICENSE
+include CHANGELOG.md
+include SECURITY.md
+include CONTRIBUTING.md
+include CODE_OF_CONDUCT.md
+include pyproject.toml
+include MANIFEST.in
+
+recursive-include examples *.py

enterprise_crawler_framework-1.0.0/PKG-INFO

@@ -0,0 +1,37 @@
+Metadata-Version: 2.4
+Name: enterprise-crawler-framework
+Version: 1.0.0
+Summary: Enterprise-grade crawler framework.
+Author: Lord Jester
+License-Expression: MIT
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.31
+Dynamic: license-file
+
+# Enterprise Crawler Framework
+
+Enterprise Crawler Framework is a Python framework for building reliable,
+reusable data-collection bots and crawler runtimes.
+
+It provides a structured foundation for HTTP collection, lifecycle management,
+storage, processing, plugins, event-driven workloads, retries, durable queues,
+and failure handling without forcing application-specific business rules into
+the framework core.
+
+Current version: **1.0.0**
+
+## Requirements
+
+- Python 3.11 or newer
+- `requests >= 2.31`
+
+## Installation
+
+### From the repository
+
+Clone the repository and install the package:
+
+```bash
+python -m pip install .

enterprise_crawler_framework-1.0.0/README.md

@@ -0,0 +1,25 @@
+# Enterprise Crawler Framework
+
+Enterprise Crawler Framework is a Python framework for building reliable,
+reusable data-collection bots and crawler runtimes.
+
+It provides a structured foundation for HTTP collection, lifecycle management,
+storage, processing, plugins, event-driven workloads, retries, durable queues,
+and failure handling without forcing application-specific business rules into
+the framework core.
+
+Current version: **1.0.0**
+
+## Requirements
+
+- Python 3.11 or newer
+- `requests >= 2.31`
+
+## Installation
+
+### From the repository
+
+Clone the repository and install the package:
+
+```bash
+python -m pip install .

enterprise_crawler_framework-1.0.0/SECURITY.md

@@ -0,0 +1,83 @@
+# Security Policy
+
+Enterprise Crawler Framework is infrastructure software that may interact with
+remote services, local files, persistent state, plugins, and event queues.
+
+Security reports are treated separately from ordinary bug reports.
+
+## Supported Versions
+
+| Version | Security Support |
+| --- | --- |
+| 1.0.x | Supported |
+| < 1.0 | Not supported |
+
+Only the latest patch release of a supported minor version is guaranteed to
+receive security fixes.
+
+## Reporting a Vulnerability
+
+Do not publish exploit details, credentials, tokens, private data, or
+proof-of-concept attacks in a public issue.
+
+Preferred reporting path:
+
+1. Use the repository's private vulnerability reporting feature from the
+   GitHub **Security** tab when it is available.
+2. Include the affected framework version.
+3. Describe the vulnerable component and expected security boundary.
+4. Include minimal reproduction steps.
+5. Explain the potential impact.
+6. Include a proposed mitigation if one is known.
+
+If private vulnerability reporting is not available, open a public issue that
+contains no vulnerability details and request a private contact channel from
+the maintainer.
+
+## Sensitive Information
+
+Never include the following in a report:
+
+- production passwords
+- API keys
+- session cookies
+- private tokens
+- customer data
+- confidential documents
+- private infrastructure addresses
+- exploitable credentials
+
+Use sanitized fixtures whenever possible.
+
+## Security Boundaries
+
+The framework intentionally follows several fail-closed principles.
+
+Injected dependencies are not automatically owned or closed by consumers.
+
+Plugin discovery and plugin loading are separate operations. Discovery of
+entry-point metadata must not require importing third-party plugin code.
+
+Event queue ownership is represented through claim tokens. A stale claim token
+must not acknowledge or negatively acknowledge a recovered message.
+
+Dead-letter transfer must preserve the source event when destination storage
+fails.
+
+Storage and plugin systems are opt-in.
+
+TLS verification is enabled by default. Disabling TLS verification should be
+treated as an explicit operator decision.
+
+## Coordinated Disclosure
+
+Please allow reasonable time for investigation, patch preparation, regression
+testing, and release before publicly disclosing a confirmed vulnerability.
+
+The maintainer may request additional reproduction information when necessary.
+
+## Non-Security Bugs
+
+Crashes, validation errors, incorrect documentation, performance problems, and
+ordinary functional defects that do not create a security impact should be
+reported through the normal issue tracker instead of the security channel.

enterprise_crawler_framework-1.0.0/enterprise_crawler/__init__.py

@@ -0,0 +1,52 @@
+"""
+Enterprise Crawler Framework
+
+Top-level public API.
+
+The root package intentionally exposes only the small set of types required
+for the primary framework workflow:
+
+    BaseBot
+        ↓
+    Crawler
+        ↓
+    ExecutionResult
+
+Subsystem-specific APIs remain available from their dedicated namespaces,
+for example:
+
+    enterprise_crawler.config
+    enterprise_crawler.events
+    enterprise_crawler.plugins
+    enterprise_crawler.processing
+    enterprise_crawler.storage
+"""
+
+from .version import (
+    __version__,
+    __title__,
+    FRAMEWORK_NAME,
+)
+from .contracts import (
+    ExecutionResult,
+)
+from .contracts.enums import (
+    ExecutionStatus,
+)
+from .core.base_bot import (
+    BaseBot,
+)
+from .core.crawler import (
+    Crawler,
+)
+
+
+__all__ = [
+    "__version__",
+    "__title__",
+    "FRAMEWORK_NAME",
+    "BaseBot",
+    "Crawler",
+    "ExecutionResult",
+    "ExecutionStatus",
+]

enterprise_crawler_framework-1.0.0/enterprise_crawler/cli/__init__.py

@@ -0,0 +1,19 @@
+from enterprise_crawler.cli.main import (
+    DoctorCheck,
+    build_parser,
+    main,
+    run_doctor,
+    run_plugins_inspect,
+    run_plugins_list,
+    run_version,
+)
+
+__all__ = [
+    "DoctorCheck",
+    "build_parser",
+    "main",
+    "run_doctor",
+    "run_plugins_inspect",
+    "run_plugins_list",
+    "run_version",
+]

enterprise_crawler_framework-1.0.0/enterprise_crawler/cli/__main__.py

@@ -0,0 +1,7 @@
+from enterprise_crawler.cli.main import main
+
+
+if __name__ == "__main__":
+    raise SystemExit(
+        main()
+    )