docpull 6.1.0__tar.gz → 6.3.0__tar.gz

{docpull-6.1.0/src/docpull.egg-info → docpull-6.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: docpull
-Version: 6.1.0
+Version: 6.3.0
 Summary: Declare, sync, diff, and lock context dependencies for AI agents
 Author-email: Zachary Roth <support@raintree.technology>
 Maintainer-email: Raintree Technology <support@raintree.technology>
@@ -122,6 +122,14 @@ Requires-Dist: types-defusedxml>=0.7.0; extra == "dev"
 Requires-Dist: types-pyyaml>=6.0.0; extra == "dev"
 Dynamic: license-file
 
+<p align="center">
+  <img
+    src="https://raw.githubusercontent.com/raintree-technology/docpull/main/docs/launch-assets/logo-square-light-400.png"
+    alt="DocPull"
+    width="128"
+  />
+</p>
+
 # docpull
 
 **Context dependencies for AI agents. Browser-free by default.**
@@ -139,6 +147,11 @@ web sources an agent depends on, sync them into cited context packs, diff what
 changed, and export reproducible context for Cursor, Claude, OpenAI,
 LlamaIndex, LangChain, MCP clients, and RAG pipelines.
 
+Architecturally, DocPull is the evidence and acquisition engine: it acquires,
+versions, cites, hashes, and replays evidence. Downstream products own
+scheduling, review, approved claims, legal conclusions, and notifications. See
+the [architecture decision](docs/adr/0001-evidence-acquisition-engine.md).
+
 The core workflow is a `docpull.yaml` plus a `.docpull/context.lock.json`,
 similar in spirit to code dependency manifests and lockfiles:
 
@@ -292,8 +305,15 @@ docpull repo-pack psf/requests -o packs/repo --cache
 docpull package-pack pypi:requests -o packs/package
 docpull standards-pack rfc:9110 -o packs/standard
 docpull dataset-pack ./metrics.csv -o packs/dataset
+docpull dataset-pack 'https://data.example.org/rows.json?$limit=100' -o packs/dataset
+docpull relationship-pack example.com --subject "Example Brand" -o packs/relationships
 docpull transcript-pack ./meeting.vtt -o packs/transcript
 docpull wiki-pack wiki:Web_scraping -o packs/wiki
+docpull brand-pack example.com -o packs/brand
+docpull product-pack https://example.com/pricing -o packs/product
+docpull styleguide-pack example.com -o packs/styleguide
+docpull image-pack example.com -o packs/visuals
+docpull policy-pack example.com -o packs/policies
 docpull pack prepare packs/docs --eval-grade
 docpull pack validate packs/docs --level eval
 docpull export packs/docs --format openai-vector-jsonl -o exports/openai.jsonl
@@ -565,9 +585,15 @@ special handling for common web, documentation, and API surfaces.
 | Public GitHub repos | `docpull repo-pack` emits repo metadata, README/docs/examples/changelog files, manifests, and releases |
 | npm / PyPI packages | `docpull package-pack` emits registry metadata, README/description, versions, license, dependencies, and install commands |
 | Standards | `docpull standards-pack` emits RFC, IETF, W3C, and WHATWG metadata plus section-level records |
-| Local datasets | `docpull dataset-pack` emits bounded schema, exact row counts where streamable, column, null-count, and sample summaries |
+| Local/remote datasets | `docpull dataset-pack` emits bounded schema, exact row counts where streamable, column/null/sample summaries, and HTTPS JSON/CSV snapshot provenance |
+| Relationship review | `docpull relationship-pack` emits cited observations and explicit coverage gaps without approving claims or inferring independence |
 | Transcripts | `docpull transcript-pack` emits timestamped segment records from VTT, SRT, text, JSON, or direct transcript URLs |
 | Wikimedia / Wikipedia | `docpull wiki-pack` emits MediaWiki REST page metadata, license/revision metadata, and section-level records |
+| Brand evidence | `docpull brand-pack` emits cited identity, firmographic, social, logo, and color observations |
+| Product and pricing | `docpull product-pack` emits plans, currencies, intervals, trials, feature gates, and page-text provenance |
+| Styleguide | `docpull styleguide-pack` emits cited CSS variables, colors, fonts, spacing, and component samples |
+| Visual assets | `docpull image-pack` emits bounded image candidates and optional validated local assets |
+| Policy documents | `docpull policy-pack` discovers policy/security pages and emits effective dates, stable clauses, and textual change candidates without legal conclusions |
 | Docusaurus / Sphinx / MkDocs | Extracts static article or document regions |
 | VitePress / VuePress / Astro Starlight | Extracts static content regions |
 | GitBook / ReadMe.io | Extracts available article or content regions |
@@ -620,6 +646,16 @@ Every file-backed run writes `corpus.manifest.json` with stable document IDs,
 chunk IDs, hashes, output paths, and chunk counts. See
 [Corpus Manifest](docs/corpus-manifest.md).
 
+Evidence-pack workflows also write `workflow.request.json`,
+`workflow.result.json`, and `artifact.manifest.json`. Export their Draft 2020-12
+schemas with `docpull contracts export -o schemas/`. The complete inventory and
+compatibility rules are in [Public Contracts](docs/contracts.md).
+
+For tracker imports, run `docpull pack intelligence-bundle PACK`. The canonical
+output is `intelligence.bundle.v1.json`; `company_brain.bundle.json` remains a
+compatibility alias. See the
+[competitor-tracker integration contract](docs/competitor-tracker-integration.md).
+
 ## Profiles
 
 ```bash
@@ -736,7 +772,8 @@ part of the package release contract.
   `docpull parse`, `docpull openapi-pack`, `docpull feed-pack`,
   `docpull paper-pack`, `docpull repo-pack`, `docpull package-pack`,
   `docpull standards-pack`, `docpull dataset-pack`, `docpull transcript-pack`,
-  `docpull wiki-pack`,
+  `docpull wiki-pack`, `docpull brand-pack`, `docpull product-pack`,
+  `docpull styleguide-pack`, `docpull image-pack`, `docpull policy-pack`,
   `docpull pack validate`,
   `docpull pack audit`, `docpull export`,
   `docpull serve`, `docpull share`, `docpull render`, `docpull auth check`,

{docpull-6.1.0 → docpull-6.3.0}/README.md

@@ -1,3 +1,11 @@
+<p align="center">
+  <img
+    src="https://raw.githubusercontent.com/raintree-technology/docpull/main/docs/launch-assets/logo-square-light-400.png"
+    alt="DocPull"
+    width="128"
+  />
+</p>
+
 # docpull
 
 **Context dependencies for AI agents. Browser-free by default.**
@@ -15,6 +23,11 @@ web sources an agent depends on, sync them into cited context packs, diff what
 changed, and export reproducible context for Cursor, Claude, OpenAI,
 LlamaIndex, LangChain, MCP clients, and RAG pipelines.
 
+Architecturally, DocPull is the evidence and acquisition engine: it acquires,
+versions, cites, hashes, and replays evidence. Downstream products own
+scheduling, review, approved claims, legal conclusions, and notifications. See
+the [architecture decision](docs/adr/0001-evidence-acquisition-engine.md).
+
 The core workflow is a `docpull.yaml` plus a `.docpull/context.lock.json`,
 similar in spirit to code dependency manifests and lockfiles:
 
@@ -168,8 +181,15 @@ docpull repo-pack psf/requests -o packs/repo --cache
 docpull package-pack pypi:requests -o packs/package
 docpull standards-pack rfc:9110 -o packs/standard
 docpull dataset-pack ./metrics.csv -o packs/dataset
+docpull dataset-pack 'https://data.example.org/rows.json?$limit=100' -o packs/dataset
+docpull relationship-pack example.com --subject "Example Brand" -o packs/relationships
 docpull transcript-pack ./meeting.vtt -o packs/transcript
 docpull wiki-pack wiki:Web_scraping -o packs/wiki
+docpull brand-pack example.com -o packs/brand
+docpull product-pack https://example.com/pricing -o packs/product
+docpull styleguide-pack example.com -o packs/styleguide
+docpull image-pack example.com -o packs/visuals
+docpull policy-pack example.com -o packs/policies
 docpull pack prepare packs/docs --eval-grade
 docpull pack validate packs/docs --level eval
 docpull export packs/docs --format openai-vector-jsonl -o exports/openai.jsonl
@@ -441,9 +461,15 @@ special handling for common web, documentation, and API surfaces.
 | Public GitHub repos | `docpull repo-pack` emits repo metadata, README/docs/examples/changelog files, manifests, and releases |
 | npm / PyPI packages | `docpull package-pack` emits registry metadata, README/description, versions, license, dependencies, and install commands |
 | Standards | `docpull standards-pack` emits RFC, IETF, W3C, and WHATWG metadata plus section-level records |
-| Local datasets | `docpull dataset-pack` emits bounded schema, exact row counts where streamable, column, null-count, and sample summaries |
+| Local/remote datasets | `docpull dataset-pack` emits bounded schema, exact row counts where streamable, column/null/sample summaries, and HTTPS JSON/CSV snapshot provenance |
+| Relationship review | `docpull relationship-pack` emits cited observations and explicit coverage gaps without approving claims or inferring independence |
 | Transcripts | `docpull transcript-pack` emits timestamped segment records from VTT, SRT, text, JSON, or direct transcript URLs |
 | Wikimedia / Wikipedia | `docpull wiki-pack` emits MediaWiki REST page metadata, license/revision metadata, and section-level records |
+| Brand evidence | `docpull brand-pack` emits cited identity, firmographic, social, logo, and color observations |
+| Product and pricing | `docpull product-pack` emits plans, currencies, intervals, trials, feature gates, and page-text provenance |
+| Styleguide | `docpull styleguide-pack` emits cited CSS variables, colors, fonts, spacing, and component samples |
+| Visual assets | `docpull image-pack` emits bounded image candidates and optional validated local assets |
+| Policy documents | `docpull policy-pack` discovers policy/security pages and emits effective dates, stable clauses, and textual change candidates without legal conclusions |
 | Docusaurus / Sphinx / MkDocs | Extracts static article or document regions |
 | VitePress / VuePress / Astro Starlight | Extracts static content regions |
 | GitBook / ReadMe.io | Extracts available article or content regions |
@@ -496,6 +522,16 @@ Every file-backed run writes `corpus.manifest.json` with stable document IDs,
 chunk IDs, hashes, output paths, and chunk counts. See
 [Corpus Manifest](docs/corpus-manifest.md).
 
+Evidence-pack workflows also write `workflow.request.json`,
+`workflow.result.json`, and `artifact.manifest.json`. Export their Draft 2020-12
+schemas with `docpull contracts export -o schemas/`. The complete inventory and
+compatibility rules are in [Public Contracts](docs/contracts.md).
+
+For tracker imports, run `docpull pack intelligence-bundle PACK`. The canonical
+output is `intelligence.bundle.v1.json`; `company_brain.bundle.json` remains a
+compatibility alias. See the
+[competitor-tracker integration contract](docs/competitor-tracker-integration.md).
+
 ## Profiles
 
 ```bash
@@ -612,7 +648,8 @@ part of the package release contract.
   `docpull parse`, `docpull openapi-pack`, `docpull feed-pack`,
   `docpull paper-pack`, `docpull repo-pack`, `docpull package-pack`,
   `docpull standards-pack`, `docpull dataset-pack`, `docpull transcript-pack`,
-  `docpull wiki-pack`,
+  `docpull wiki-pack`, `docpull brand-pack`, `docpull product-pack`,
+  `docpull styleguide-pack`, `docpull image-pack`, `docpull policy-pack`,
   `docpull pack validate`,
   `docpull pack audit`, `docpull export`,
   `docpull serve`, `docpull share`, `docpull render`, `docpull auth check`,

{docpull-6.1.0 → docpull-6.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "docpull"
-version = "6.1.0"
+version = "6.3.0"
 dynamic = []
 description = "Declare, sync, diff, and lock context dependencies for AI agents"
 readme = {file = "README.md", content-type = "text/markdown"}
@@ -202,7 +202,7 @@ where = ["src"]
 include = ["docpull*"]
 
 [tool.setuptools.package-data]
-docpull = ["py.typed", "fixtures/*.json"]
+docpull = ["py.typed", "fixtures/*.json", "schemas/*.schema.json"]
 
 [tool.ruff]
 line-length = 110

docpull-6.3.0/src/docpull/__init__.py

@@ -0,0 +1,287 @@
+"""DocPull — reproducible context dependencies for AI agents.
+
+The public SDK is loaded lazily. Importing :mod:`docpull` is common to every
+CLI command and Python submodule import, so the package root must not eagerly
+load optional workflows or rendering backends. Attribute access and
+``from docpull import ...`` retain the public contract while loading only the
+module that owns the requested symbol.
+"""
+# ruff: noqa: F401 - TYPE_CHECKING imports document the lazy public re-exports.
+
+from __future__ import annotations
+
+from importlib import import_module
+from typing import TYPE_CHECKING, Any
+
+from .surface import PUBLIC_SDK_EXPORTS
+
+__version__ = "6.3.0"
+
+_LAZY_EXPORTS: dict[str, tuple[str, str]] = {
+    **{
+        name: (".context_packs", name)
+        for name in (
+            "async_build_dataset_pack",
+            "async_build_package_pack",
+            "async_build_paper_pack",
+            "async_build_repo_pack",
+            "async_build_standards_pack",
+            "async_build_transcript_pack",
+            "async_build_wiki_pack",
+            "build_brand_pack",
+            "build_dataset_pack",
+            "build_feed_pack",
+            "build_image_pack",
+            "build_openapi_pack",
+            "build_package_pack",
+            "build_paper_pack",
+            "build_policy_pack",
+            "build_relationship_pack",
+            "build_product_pack",
+            "build_repo_pack",
+            "build_standards_pack",
+            "build_styleguide_pack",
+            "build_transcript_pack",
+            "build_wiki_pack",
+            "capture_screenshot_pack",
+        )
+    },
+    **{
+        name: (".contracts", name)
+        for name in (
+            "ArtifactManifest",
+            "ChangeEvent",
+            "EvidenceSpan",
+            "IntelligenceBundle",
+            "CoverageResult",
+            "RelationshipCandidate",
+            "RelationshipPack",
+            "SourceAuthority",
+            "WorkflowRequest",
+            "WorkflowResult",
+            "bundled_schema_path",
+            "write_contract_schemas",
+        )
+    },
+    **{
+        name: (".workflows", name)
+        for name in ("async_run_workflow", "create_workflow_request", "run_workflow")
+    },
+    **{name: (".core.fetcher", name) for name in ("Fetcher", "fetch_blocking", "fetch_one")},
+    **{name: (".conversion.chunking", name) for name in ("Chunk", "TokenCounter", "chunk_markdown")},
+    "PageContext": (".pipeline.base", "PageContext"),
+    **{name: (".pipeline.steps", name) for name in ("SqliteSearchResult", "search_sqlite_documents")},
+    **{
+        name: (".models.config", name)
+        for name in (
+            "BudgetConfig",
+            "CacheConfig",
+            "ContentFilterConfig",
+            "CrawlConfig",
+            "DocpullConfig",
+            "NetworkConfig",
+            "OutputConfig",
+            "PerformanceConfig",
+            "ProfileName",
+            "RenderActionPolicy",
+            "RenderConfig",
+            "RenderViewport",
+        )
+    },
+    **{name: (".models.events", name) for name in ("EventType", "FetchEvent", "FetchStats")},
+    **{name: (".cache", name) for name in ("CacheManager", "StreamingDeduplicator")},
+    "PolicyConfig": (".policy", "PolicyConfig"),
+    **{name: (".local_workflows", name) for name in ("audit_pack", "refresh_pack")},
+    **{
+        name: (".document_parse", name)
+        for name in ("DocumentParseError", "ParsedDocument", "parse_documents", "parse_one_document")
+    },
+    **{
+        name: (".pack_tools", name)
+        for name in (
+            "build_citation_map",
+            "build_company_brain_bundle",
+            "build_intelligence_bundle",
+            "build_research_brief",
+            "diff_packs",
+            "extract_pack_entities",
+            "prepare_pack",
+            "score_pack",
+            "score_pack_sources",
+            "search_pack",
+        )
+    },
+    "validate_pack_contract": (".output_contract", "validate_pack_contract"),
+    **{name: (".context_ci", name) for name in ("CIThresholds", "ContextCIError", "run_context_ci")},
+    **{name: (".exports", name) for name in ("ExportResult", "export_pack")},
+    **{name: (".pack_reader", name) for name in ("LocalPack", "PackReadError", "PackSource", "load_pack")},
+    **{
+        name: (".graph", name)
+        for name in (
+            "GraphError",
+            "build_graph",
+            "graph_neighbors",
+            "graph_status",
+            "load_graph",
+            "query_graph",
+            "refresh_graph",
+        )
+    },
+    **{name: (".server", name) for name in ("PackASGIApp", "PackServerError", "create_pack_app")},
+    **{
+        name: (".share", name)
+        for name in ("ReportHTTPServer", "ShareError", "create_report_server", "render_report_document")
+    },
+    **{
+        name: (".rendering", name)
+        for name in (
+            "AgentBrowserRenderer",
+            "E2BSandboxRenderer",
+            "RenderedPage",
+            "Renderer",
+            "RenderError",
+            "RendererUnavailableError",
+            "VercelSandboxRenderer",
+            "agent_browser_binary",
+            "check_agent_browser_availability",
+            "check_e2b_sandbox_availability",
+            "check_render_backend_availability",
+            "check_vercel_sandbox_availability",
+            "estimate_cloud_render_cost_usd",
+            "render_url",
+            "render_url_to_directory",
+        )
+    },
+}
+
+__all__ = list(PUBLIC_SDK_EXPORTS)
+
+
+def __getattr__(name: str) -> Any:
+    """Load one public SDK symbol on first access and cache it locally."""
+    target = _LAZY_EXPORTS.get(name)
+    if target is None:
+        raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+    module_name, attribute_name = target
+    value = getattr(import_module(module_name, __name__), attribute_name)
+    globals()[name] = value
+    return value
+
+
+def __dir__() -> list[str]:
+    """Include lazy public attributes in interactive discovery."""
+    return sorted(set(globals()) | set(__all__))
+
+
+if TYPE_CHECKING:
+    from .cache import CacheManager, StreamingDeduplicator
+    from .context_ci import CIThresholds, ContextCIError, run_context_ci
+    from .context_packs import (
+        async_build_dataset_pack,
+        async_build_package_pack,
+        async_build_paper_pack,
+        async_build_repo_pack,
+        async_build_standards_pack,
+        async_build_transcript_pack,
+        async_build_wiki_pack,
+        build_brand_pack,
+        build_dataset_pack,
+        build_feed_pack,
+        build_image_pack,
+        build_openapi_pack,
+        build_package_pack,
+        build_paper_pack,
+        build_policy_pack,
+        build_product_pack,
+        build_relationship_pack,
+        build_repo_pack,
+        build_standards_pack,
+        build_styleguide_pack,
+        build_transcript_pack,
+        build_wiki_pack,
+        capture_screenshot_pack,
+    )
+    from .contracts import (
+        ArtifactManifest,
+        ChangeEvent,
+        CoverageResult,
+        EvidenceSpan,
+        IntelligenceBundle,
+        RelationshipCandidate,
+        RelationshipPack,
+        SourceAuthority,
+        WorkflowRequest,
+        WorkflowResult,
+        bundled_schema_path,
+        write_contract_schemas,
+    )
+    from .conversion.chunking import Chunk, TokenCounter, chunk_markdown
+    from .core.fetcher import Fetcher, fetch_blocking, fetch_one
+    from .document_parse import DocumentParseError, ParsedDocument, parse_documents, parse_one_document
+    from .exports import ExportResult, export_pack
+    from .graph import (
+        GraphError,
+        build_graph,
+        graph_neighbors,
+        graph_status,
+        load_graph,
+        query_graph,
+        refresh_graph,
+    )
+    from .local_workflows import audit_pack, refresh_pack
+    from .models.config import (
+        BudgetConfig,
+        CacheConfig,
+        ContentFilterConfig,
+        CrawlConfig,
+        DocpullConfig,
+        NetworkConfig,
+        OutputConfig,
+        PerformanceConfig,
+        ProfileName,
+        RenderActionPolicy,
+        RenderConfig,
+        RenderViewport,
+    )
+    from .models.events import EventType, FetchEvent, FetchStats
+    from .output_contract import validate_pack_contract
+    from .pack_reader import LocalPack, PackReadError, PackSource, load_pack
+    from .pack_tools import (
+        build_citation_map,
+        build_company_brain_bundle,
+        build_intelligence_bundle,
+        build_research_brief,
+        diff_packs,
+        extract_pack_entities,
+        prepare_pack,
+        score_pack,
+        score_pack_sources,
+        search_pack,
+    )
+    from .pipeline.base import PageContext
+    from .pipeline.steps import SqliteSearchResult, search_sqlite_documents
+    from .policy import PolicyConfig
+    from .rendering import (
+        AgentBrowserRenderer,
+        E2BSandboxRenderer,
+        RenderedPage,
+        Renderer,
+        RenderError,
+        RendererUnavailableError,
+        VercelSandboxRenderer,
+        agent_browser_binary,
+        check_agent_browser_availability,
+        check_e2b_sandbox_availability,
+        check_render_backend_availability,
+        check_vercel_sandbox_availability,
+        estimate_cloud_render_cost_usd,
+        render_url,
+        render_url_to_directory,
+    )
+    from .server import PackASGIApp, PackServerError, create_pack_app
+    from .share import ReportHTTPServer, ShareError, create_report_server, render_report_document
+    from .workflows import async_run_workflow, create_workflow_request, run_workflow
+
+
+assert tuple(__all__) == PUBLIC_SDK_EXPORTS
+assert set(_LAZY_EXPORTS) == set(__all__) - {"__version__"}