hugpy-discovery 0.1.1__tar.gz

hugpy_discovery-0.1.1/PKG-INFO

@@ -0,0 +1,16 @@
+Metadata-Version: 2.4
+Name: hugpy-discovery
+Version: 0.1.1
+Summary: Independent model inventory and host-scoped engine route evidence
+Requires-Python: >=3.11
+Requires-Dist: hugpy-resources<0.2,>=0.1.2.8
+Requires-Dist: sentencepiece>=0.2
+Requires-Dist: protobuf>=4
+Requires-Dist: hugpy-settings>=0.1.7
+Provides-Extra: test
+Requires-Dist: pytest<10,>=8; extra == "test"
+Provides-Extra: identity
+Requires-Dist: huggingface_hub>=0.20; extra == "identity"
+Provides-Extra: huggingface
+Requires-Dist: huggingface_hub>=0.20; extra == "huggingface"
+Requires-Dist: requests>=2; extra == "huggingface"

hugpy_discovery-0.1.1/README.md

@@ -0,0 +1,275 @@
+# Hugpy Discovery
+
+Worker engine wiring and the schema-10 history import are documented in
+[Wrapper engine observations](docs/worker-engine-observations.md). The populated
+older snapshots remain historical; imports never promote their readiness gates.
+
+Standalone inventory, verification and pre-engine evidence for a local model tree. Source model files are read without modification; observations and proofs are written to a separate SQLite database. This package has its own scanner, format readers, engine survey, and browser inspector. It does not import the wrapper application or use its model-record writer.
+
+## Source layout
+
+`hugpy-resources` is the canonical reference for generic functions, format
+constants and MIME data. Discovery imports those implementations through its
+`imports/init_imports.py` boundary. Its `imports/__init__.py` and
+`imports/functions/__init__.py` retain the existing shared symbol exports.
+
+| Owner | Responsibility |
+| --- | --- |
+| Resources `imports/functions/json_utils.py`, `formatting.py`, `ast_utils.py` | Serialization, digests, byte formatting and AST helpers |
+| Resources `imports/functions/paths.py`, `locking.py`, `sqlite.py`, `sqlite_connections.py` | Generic filesystem, locking and SQLite utilities |
+| Resources `imports/functions/file_types.py`, `tensor_files.py`, `gguf_*.py` | MIME hints, bounded file identification, tensor headers and GGUF readers |
+| Resources `imports/functions/content_proofs.py` | Stable whole-file SHA256 and Git-blob SHA1 proofs |
+| Resources `imports/constants.py`, `imports/mime_types.json`, `imports/classes.py` | Shared format metadata, MIME definitions, sentinels and parsing errors |
+| Discovery `imports/functions/environment.py`, `repositories.py`, `huggingface.py` | Hugpy environment attribution and provider-specific validation/serialization |
+| Discovery `imports/functions/discovery_paths.py` | Apply Discovery's directory exclusion policy to the generic snapshot helper |
+| Discovery `huggingface/` | HF identity, provider metadata cache, search and explicit snapshot tools |
+
+Moved generic helper scripts are removed from Discovery; Resources owns their
+implementations and data. Source fingerprints include the shared package's
+Python and data files. Optional tokenizer and provider SDKs resolve only when
+the corresponding feature requests them. Resources' base install remains
+standard-library-only; `hugpy-resources[inspection]` supplies optional tokenizer
+inspection dependencies.
+
+Discovery ledger writes, preparation/readiness decisions, engine surveys and
+Hugpy-specific source pointers remain in their domain modules. The standalone
+engine probe lives in `imports.constants.PROBE` and runs in the surveyed
+environment.
+
+Each SQL schema lives with its companion Python code in a package beneath
+`src/hugpy_discovery/`. Each SQL file has a matching Python implementation beside
+it: for example, `preparation/preparation.sql` and `preparation/preparation.py`,
+or `store/schema.sql` and `store/schema.py`. Package `__init__.py` files only
+export the public functions and constants, preserving imports such as
+`hugpy_discovery.preparation`.
+
+| Directory | SQL file | Companion Python responsibility |
+| --- | --- | --- |
+| `store/` | `schema.sql` | SQLite store and migration ordering |
+| `preparation/` | `preparation.sql` | Intake and preparation ledger |
+| `environment/` | `environment.sql` | Environment source registry |
+| `operator_environment/` | `operator_environment.sql` | Shared operator contract projection |
+| `documentation/` | `documentation.sql` | Document imports and revisions |
+| `workflow/` | `workflow.sql` | Connected evidence overview |
+| `formula_catalog/` | `formula_catalog.sql` | Formula contracts, sources and validation |
+| `worker_engines/` | `worker_engines.sql` | Historical engine observation imports |
+| `memory_ticks/` | `memory_ticks.sql` | Capacity, memory scenarios and tick ladders |
+| `wrapper_loads/` | `wrapper_loads.sql` | Historical load measurement imports |
+| `runtime_evidence/` | `runtime_evidence.sql` | Runtime provenance schema migration |
+| `weight_components/` | `weight_components.sql` | Disjoint weight accounting |
+
+`preparation` continues to export its memory scenario helpers, `environment`
+exports `persist_contract`, and `worker_engines` exports `import_loads` for
+existing callers. SQL files are included as subpackage resources in the wheel.
+
+## Preparation and evidence
+
+The [formula and validation reference](docs/formulas-and-validation.md) records
+the offline test suite, ten formula contracts, pinned upstream sources and
+existing geometry pointers. Schema v9 stores validation reports and a separate
+static geometry review; incomplete memory totals remain NULL.
+
+The [initial operator environment contract](docs/operator-environment-contract.md)
+defines the eleven retained keys, their default pointers and migration aliases.
+Schema v6 exposes this contract alongside the broader source audit; third-party
+additions are deferred. Live service configuration has not yet been migrated.
+
+## Assembly before derivation
+
+The current review flow is:
+
+```text
+Saved documentation + eleven-key contract + environment source audit
+                              ↓
+Literal intake → declared stores/owners → filesystem and file locations
+                              ↓
+Provider identity → selected units and dependencies → current file proofs
+                              ↓
+pre_derivation_review: requirements passed or explicit blockers
+                              ↓
+Static geometry / memory evidence review → controlled engine checks (not attempted)
+```
+
+Use `prepare --stop-before-derivation` to refresh and connect the existing
+evidence without computing geometry or memory scenarios. Repeat `--document`
+to capture the associated Markdown in the same workflow:
+
+```sh
+.venv/bin/python -m hugpy_discovery prepare \
+  --database artifacts/pre-engine-v9.sqlite3 \
+  --env-file /etc/hugpy/wrapper.env --stop-before-derivation \
+  --document README.md --document docs/operator-environment-contract.md
+```
+
+For a full documentation snapshot, pass all session documents, as in `save-docs`
+below. With no `--document`, preparation binds the most recent saved import.
+`preparation_context` pins the documentation import, source-registry run and
+exact shared contract JSON/hash. `pre_derivation_review` joins current engine
+candidates to units, ownership, filesystem identity, allowance, selected-file
+verification and eight prerequisite gates. Context blockers remain explicit.
+Preparation rechecks file stat identities and reuses completed proofs; it does
+not read full weight payloads, fetch provider metadata or run benchmarks.
+
+`current_execution_readiness` selects the latest completed preparation run;
+historical candidates remain in `execution_plans`. Earlier geometry and memory
+rows remain available, but deferred stages do not recompute or validate them.
+The separate `formula-catalog --derive-static` command refreshes static evidence
+after a passing offline suite, preserving that preparation history.
+This joins the staging evidence flow; live Wrapper/Core configuration migration
+and undeclared ownership remain separate outstanding work.
+
+## Documentation snapshots
+
+Schema v7 stores full Markdown content with source paths, SHA256 digests,
+capture times and source stat identities. `documentation_current` exposes the
+latest saved text for each document; `documentation_revisions` preserves older
+content. Unchanged imports reuse a revision and record a new observation.
+Source files remain in place. Capturing a document does not turn its historical
+claims into current runtime evidence. Imports are explicit snapshots, not an
+automatic filesystem watcher.
+
+```sh
+.venv/bin/python -m hugpy_discovery save-docs \
+  --database artifacts/pre-engine-v9.sqlite3 \
+  README.md docs/*.md \
+  ../hugpy-settings/package/README.md
+```
+
+The inspector searches document text using its normal table search. Select a
+row in `documentation_current` to read `content_markdown`. Markdown is displayed
+as text, so embedded HTML and scripts are never executed.
+
+## Build and inspect
+
+```sh
+.venv/bin/python -m hugpy_discovery build /path/to/models --database artifacts/discovery.sqlite3
+.venv/bin/python -m hugpy_discovery serve /path/to/models --output artifacts --database artifacts/discovery.sqlite3
+```
+
+`build` creates a **new** database outside the model tree. An existing database is never overwritten. `--engine-python /path/to/python` may be repeated to include specific engine environments. The built-in survey checks a bounded set of local interpreters and native binaries before scanning models. Its results are scoped to this host and the surveyed executables; they do not describe remote fleet capacity.
+
+The static `build` evidence path is:
+
+1. Record engine package presence, Transformers loader registration, and host hardware.
+2. Inventory the model tree, attribute directories/files to shared filesystem records, and inspect bounded file headers, manifests, and config documents.
+3. Form weight sets, shard groups, and component dependencies. Keep alternative weight variants separate.
+4. Derive per-bundle engine routes with explicit blockers or a `static-candidate` state.
+5. Validate database references, estimate coverage, and source metadata stability.
+
+`static-candidate` requires complete dependency evidence and a known loader mapping. Other routes retain `dependency-unresolved` or `loader-unresolved`. Even a `static-candidate` does **not** mean a model will load, fit memory, or run correctly. Static source stability is based on directory metadata plus hashes of inspected evidence bytes; `build` does not content-hash tensor payloads. The separate `prepare` and `verify` stages record readiness requirements and full-file proofs before a guarded engine check. Loader-specific files, custom code, version support, memory allocation, and runtime behavior remain to be checked by engine-specific execution in a later stage.
+
+The original discovery prototype and its SQLite database remain in the wrapper staging tree as historical evidence. New discovery passes and the inspector should use this package and its own database.
+
+## Filesystem attribution
+
+Schema v2 adds `filesystem_info`; `model_records.filesystem_id` and
+`model_record_files.filesystem_id` reference it. Many models share one filesystem
+row. File references describe the resolved file location, including a file on a
+different filesystem from its model directory. The filesystem row records UUID
+and PARTUUID when visible through `/dev/disk`, observed device major/minor and
+path, filesystem type, mount entries/options, capacity and observation time.
+Mount entries are plural because bind mounts can expose the same filesystem at
+several paths with different options.
+
+The primary key uses the host and observed filesystem UUID when available;
+otherwise it uses the host, boot and observed filesystem device. UUID is not
+assumed globally unique across hosts. Unknown identifiers remain NULL.
+This is local observation, not fleet store identity or
+a physical-disk topology inventory. Ownership, model allowance and write/delete
+authority belong to declared stores/access policy; mount `rw` grants none of
+those permissions. No write probe or throughput benchmark runs here.
+
+Opening a v1 database through the writable Store adds the references, leaving
+historical rows unattributed. The read-only inspector does not migrate a database.
+A fresh build populates current filesystem observations. Legacy `drive_*` columns
+remain for compatibility; new consumers should join `filesystem_info`.
+
+## Intake and ownership target
+
+The next rung follows the [D1–D6 storage ownership contract](../hugpy-resources/docs/nomenclature.md): local declarations establish authority; Wrappers host and serve worker and central stores with separate budgets; Core consolidates their evidence and coordinates sharing. Undeclared drives are immutable. Configuration cleanup and host/drive intake must precede engine probes in that target flow.
+
+## Model identity resolver
+
+The HF resolver uses Discovery's own `huggingface.identity` adapter,
+`ModelMetadataStore` and shared `serialize_model_info` helper, extracted from
+Storage. Install the `identity` extra; no Storage package is needed for these
+lookups. For this staging checkout:
+
+```sh
+.venv/bin/python -m pip install -e '.[identity]'
+.venv/bin/python -m hugpy_discovery resolve --database artifacts/discovery.sqlite3 --all
+.venv/bin/python -m hugpy_discovery resolve --database artifacts/discovery.sqlite3 --model-dir /absolute/model/directory --offline
+```
+
+`--all` processes model directories with artifacts or configuration, skipping
+empty inventory-only directories. `--worker` scopes a worker and `--limit` bounds
+a batch. `build --resolve-identities` enriches a fresh inventory after its static
+pass. Ordinary builds remain offline with respect to model providers.
+
+The default cache is `provider-metadata.sqlite3` beside the discovery database
+(or in the inspector's output directory). `--provider-cache` selects another
+cache outside model trees. Successful responses persist without expiry;
+`--refresh` explicitly replaces them, and `--offline` uses only cached results.
+Errors remain distinct from empty search results and receive a ten-minute
+retry delay. Credentials use Hugging Face's environment/saved token resolution;
+requests are restricted to `https://huggingface.co`. Cache keys separate credential
+scopes without storing tokens. No root provisioning helpers are imported.
+
+The inspector serves these operations:
+
+| Method and route | JSON body / query |
+| --- | --- |
+| `POST /api/identity/search` | `{"query":"model name","limit":5}` |
+| `POST /api/identity/describe` | `{"model_id":"owner/repository","revision":"optional revision"}` |
+| `POST /api/identity/resolve` | `{"model_dir":"/absolute/model/directory"}`; optional `worker` |
+| `GET /api/identity/resolution` | `model_dir` and optional `worker`; reads the latest persisted result |
+
+POST bodies must use `Content-Type: application/json`. Optional `offline` and
+`refresh` booleans apply to the three POST operations. The initial provider is
+`hf`; other provider names are rejected explicitly. The inspector's model-record
+details also offer **Resolve identity**.
+
+Schema v3 retains resolution history and candidate/provider evidence. Exact
+repository hints plus matching local-inventory filenames and sizes can produce
+`metadata-supported`; that status populates `hub_id` with an explicit
+`hub_id_source`. Name-search matches stay candidates. Disagreement, ambiguity,
+provider failures and offline cache misses remain visible. Comparisons use the
+recorded scan and retain its input signature; this does not revalidate disk
+contents. A provider's current commit is stored as `provider_revision`, never
+silently assigned to the local `revision`. No weight hashing, downloading,
+loading or geometry derivation is performed by identity resolution.
+
+
+## Hugging Face tools
+
+Discovery owns Hugging Face identity, metadata caching, search, repository size
+lookups, snapshot audit and explicit repair/download helpers under
+`src/hugpy_discovery/huggingface/`. Install `.[huggingface]` for the full tool
+set, including token validation. See [Hugging Face extraction](docs/huggingface-tools.md)
+for source ownership, cache paths and examples. Storage's implementation remains
+intact. File verification now uses the identical content-hashing implementation
+from Resources, so Discovery no longer imports Storage.
+
+
+Internal imports use relative paths. Shared schemas, catalog metadata, status
+sets, and option lists live in `imports/constants.py`; workflow modules import
+them at module scope. Optional provider SDKs, parser backends, and hardware
+libraries remain deferred, as do documented imports that break workflow cycles.
+Resource helpers resolve directly to their definitions in
+`hugpy_resources.imports.functions` and `hugpy_resources.imports.constants`.
+
+Shared metadata and exact NVML observations resolve through the Resources import
+boundary. Discovery no longer imports Wrapper's NVML module. Discovery retains
+provider/repository workflows and persistence; generic parsing stays in Resources
+so lower-level consumers do not acquire a dependency on this application.
+
+## Embedding in Wrapper
+
+`inventory.build_database(..., store=host_store)` and
+`inventory.create_database(..., store=host_store)` accept a host-owned Store.
+Wrapper supplies its own Store and applies its own migrations before Discovery
+writes inventory, filesystem, tensor and route evidence. Discovery neither
+opens its standalone Store nor closes the supplied Store. The existing
+standalone commands retain their own separate schema for standalone use.
+`catalog.discover` supplies bounded traversal with host callbacks for configured
+roots/exclusions, quant election and runtime eligibility.

hugpy_discovery-0.1.1/pyproject.toml

@@ -0,0 +1,32 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "hugpy-discovery"
+version = "0.1.1"
+description = "Independent model inventory and host-scoped engine route evidence"
+requires-python = ">=3.11"
+dependencies = ["hugpy-resources>=0.1.2.8,<0.2", "sentencepiece>=0.2", "protobuf>=4", "hugpy-settings>=0.1.7"]
+
+[project.optional-dependencies]
+test = ["pytest>=8,<10"]
+identity = ["huggingface_hub>=0.20"]
+huggingface = ["huggingface_hub>=0.20", "requests>=2"]
+
+[project.scripts]
+hugpy-discovery = "hugpy_discovery.__main__:main"
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+hugpy_discovery = ["viewer.html"]
+"*" = ["*.sql"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-ra"

hugpy_discovery-0.1.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

hugpy_discovery-0.1.1/src/hugpy_discovery/__init__.py

@@ -0,0 +1,2 @@
+"""Independent, read-only model discovery and engine route assessment."""
+

hugpy_discovery-0.1.1/src/hugpy_discovery/__main__.py

@@ -0,0 +1,187 @@
+"""CLI for the independent discovery package."""
+
+from .inventory import build_database
+from .worker_engines import (
+    import_loads,
+    import_observations,
+)
+from .store import Store
+from .identity import (
+    resolve_database,
+    validate_output,
+)
+from .weight_components import derive
+from .timing_scales import derive as derive_timing
+from .formula_catalog import establish
+from .documentation import import_documents
+from .environment import build_registry
+from .preparation import prepare
+from .verification import verify
+from .viewer import main as viewer_main
+from .imports.init_imports import (
+    Path,
+    argparse,
+    json,
+    sys,
+)
+
+
+def main():
+    parser = argparse.ArgumentParser(description='Local model and engine evidence survey')
+    sub = parser.add_subparsers(dest='command',required=True)
+    build = sub.add_parser('build',help='create a fresh SQLite evidence database')
+    build.add_argument('root',type=Path)
+    build.add_argument('--database',required=True,type=Path)
+    build.add_argument('--engine-python',action='append',default=[])
+    build.add_argument('--resolve-identities', action='store_true', help='resolve observed models through the provider cache/API after inventory')
+    build.add_argument('--provider-cache', type=Path)
+    resolve = sub.add_parser('resolve', help='resolve observed model identities using existing provider metadata')
+    resolve.add_argument('--database', required=True, type=Path)
+    target = resolve.add_mutually_exclusive_group(required=True)
+    target.add_argument('--model-dir', type=Path)
+    target.add_argument('--all', action='store_true', help='resolve all nonempty model directories')
+    resolve.add_argument('--worker')
+    resolve.add_argument('--limit', type=int)
+    resolve.add_argument('--provider-cache', type=Path)
+    resolve.add_argument('--offline', action='store_true')
+    resolve.add_argument('--refresh', action='store_true')
+    prepare = sub.add_parser('prepare', help='build the complete pre-engine evidence/readiness database')
+    prepare.add_argument('--database', required=True, type=Path)
+    prepare.add_argument('--source-database', type=Path, help='copy an existing inventory to a new database first')
+    prepare.add_argument('--env-file', action='append', type=Path, default=[])
+    prepare.add_argument('--declarations', type=Path, help='explicit store ownership, execution paths, allowances and selections')
+    prepare.add_argument('--stop-before-derivation', action='store_true', help='connect existing evidence and record blockers without computing geometry or memory scenarios')
+    prepare.add_argument('--document', type=Path, action='append', default=[], help='Markdown to snapshot and bind to this run; repeatable')
+    verify = sub.add_parser('verify', help='process resumable full-file hash jobs; never loads engines')
+    verify.add_argument('--database', required=True, type=Path)
+    verify.add_argument('--byte-budget', type=int, default=64 * 1024**2)
+    verify.add_argument('--all-bytes', action='store_true', help='remove the per-invocation I/O byte budget')
+    verify.add_argument('--max-files', type=int)
+    verify.add_argument('--model-dir', type=Path)
+    verify.add_argument('--proofs-database', type=Path, help='reuse current Wrapper storage_content_hashes proofs read-only')
+    engines = sub.add_parser('import-wrapper-engines', help='import recorded worker engine readiness; source is read-only')
+    engines.add_argument('--database', required=True, type=Path)
+    engines.add_argument('--wrapper-database', required=True, type=Path)
+    engines.add_argument('--request-id')
+    engines.add_argument('--limit', type=int, default=1000, help='maximum recent calls to import')
+    components = sub.add_parser('derive-weight-components', help='derive expert/non-expert weight breakdown from recorded tensor facts')
+    components.add_argument('--database', required=True, type=Path)
+    timing = sub.add_parser('timing-scales', help='price every bundle, representation and placement per token from the box read rates')
+    timing.add_argument('--database', required=True, type=Path)
+    loads = sub.add_parser('import-wrapper-loads', help='import recorded transformers loads, measured decode and drive reads; source is read-only')
+    loads.add_argument('--database', required=True, type=Path)
+    loads.add_argument('--wrapper-database', required=True, type=Path, action='append')
+    registry = sub.add_parser('env-registry', help='record environment default/source pointers without applying configuration')
+    registry.add_argument('--database', required=True, type=Path)
+    registry.add_argument('--staging', required=True, type=Path)
+    registry.add_argument('--seed-report', required=True, type=Path)
+    registry.add_argument('--report', type=Path)
+    formulas = sub.add_parser('formula-catalog', help='record pinned formulas, implementation pointers and offline suite results')
+    formulas.add_argument('--database', required=True, type=Path)
+    formulas.add_argument('--manifest', required=True, type=Path)
+    formulas.add_argument('--staging', required=True, type=Path)
+    formulas.add_argument('--junit', type=Path)
+    formulas.add_argument('--derive-static', action='store_true')
+    documents = sub.add_parser('save-docs', help='save full Markdown documents and retain content revisions')
+    documents.add_argument('--database', required=True, type=Path)
+    documents.add_argument('paths', nargs='+', type=Path)
+    serve = sub.add_parser('serve',help='open the read-only inspector; build on demand')
+    serve.add_argument('root',type=Path)
+    serve.add_argument('--output',required=True,type=Path)
+    serve.add_argument('--database',type=Path)
+    serve.add_argument('--host',default='127.0.0.1')
+    serve.add_argument('--port',type=int,default=8766)
+    engine_probe = sub.add_parser('engine-discovery', help='collect runtime surveys and correlate probe evidence; never loads models')
+    engine_probe.add_argument('--database', required=True, type=Path)
+    engine_probe.add_argument('--wrapper-database', type=Path)
+    engine_probe.add_argument('--probe-results', action='append', type=Path, default=[])
+    engine_probe.add_argument('--survey', action='store_true')
+    engine_probe.add_argument('--engine-python', action='append', default=[])
+    engine_probe.add_argument('--limit', type=int, default=1000)
+    args = parser.parse_args()
+    if args.command == 'engine-discovery':
+        from .engine_probes import collect
+        print(json.dumps(collect(args.database, wrapper_database=args.wrapper_database,
+              result_paths=args.probe_results, survey=args.survey,
+              engine_paths=args.engine_python, limit=args.limit), indent=2))
+        return
+
+    if args.command == 'import-wrapper-engines':
+        print(json.dumps(import_observations(args.database, args.wrapper_database,
+              request_id=args.request_id, limit=args.limit), indent=2))
+        return
+    if args.command == 'derive-weight-components':
+        database = args.database.resolve(strict=True)
+        validate_output(database, database.parent / 'provider-metadata.sqlite3')
+        db = Store(database)
+        try:
+            with db.transaction() as conn:
+                report = derive(conn)
+            print(json.dumps(report, indent=2))
+        finally:
+            db.close()
+        return
+    if args.command == 'timing-scales':
+        database = args.database.resolve(strict=True)
+        validate_output(database, database.parent / 'provider-metadata.sqlite3')
+        db = Store(database)
+        try:
+            with db.transaction() as conn:
+                report = derive_timing(conn)
+            print(json.dumps(report, indent=2))
+        finally:
+            db.close()
+        return
+    if args.command == 'import-wrapper-loads':
+        print(json.dumps([import_loads(args.database, source) for source in args.wrapper_database], indent=2))
+        return
+    if args.command == 'formula-catalog':
+        print(json.dumps(establish(args.database,manifest=args.manifest,staging=args.staging,
+              junit=args.junit,derive_static=args.derive_static),indent=2))
+        return
+    if args.command == 'save-docs':
+        print(json.dumps(import_documents(args.database, args.paths), indent=2))
+        return
+    if args.command == 'env-registry':
+        print(json.dumps(build_registry(args.database, args.staging, args.seed_report, args.report), indent=2))
+        return
+    if args.command == 'prepare':
+        result = prepare(args.database, source_database=args.source_database, env_files=args.env_file,
+                         declarations_path=args.declarations, stop_before_derivation=args.stop_before_derivation,
+                         document_paths=args.document)
+        print(json.dumps(result, indent=2))
+        return
+    if args.command == 'verify':
+        result = verify(args.database, byte_budget=None if args.all_bytes else args.byte_budget,
+                        max_files=args.max_files, model_dir=args.model_dir, proofs_database=args.proofs_database)
+        print(json.dumps(result, indent=2))
+        return
+    if args.command == 'resolve':
+        results = resolve_database(args.database, cache_path=args.provider_cache, model_dir=args.model_dir,
+                                   worker=args.worker, offline=args.offline, refresh=args.refresh, limit=args.limit)
+        print(json.dumps([dict(model_dir=r['model_dir'], status=r['status'], model_id=r['model_id'],
+                               candidates=len(r['candidates'])) for r in results], indent=2))
+        return
+    if args.command=='serve':
+        sys.argv = [sys.argv[0],str(args.root),'--output',str(args.output),'--host',args.host,'--port',str(args.port)]
+        if args.database:
+            sys.argv.extend(['--database',str(args.database)])
+        viewer_main()
+        return
+    last_phase = None
+    def progress(**event):
+        nonlocal last_phase
+        phase = event.get('phase')
+        if phase != last_phase:
+            print(json.dumps({'phase': phase}), flush=True)
+            last_phase = phase
+    report=build_database(args.root,args.database,engine_paths=args.engine_python,progress=progress)
+    print(json.dumps(report['database'],indent=2,sort_keys=True))
+    if args.resolve_identities:
+        results = resolve_database(args.database, cache_path=args.provider_cache)
+        print(json.dumps({'identities': [dict(model_dir=r['model_dir'], status=r['status'],
+                                              model_id=r['model_id']) for r in results]}, indent=2))
+
+
+if __name__=='__main__':
+    main()