aimarket-histor 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- aimarket_histor-0.2.0/.dockerignore +18 -0
- aimarket_histor-0.2.0/.env.example +44 -0
- aimarket_histor-0.2.0/.gitignore +14 -0
- aimarket_histor-0.2.0/Dockerfile +37 -0
- aimarket_histor-0.2.0/LICENSE +22 -0
- aimarket_histor-0.2.0/Makefile +21 -0
- aimarket_histor-0.2.0/PKG-INFO +195 -0
- aimarket_histor-0.2.0/README.md +167 -0
- aimarket_histor-0.2.0/docs/GLAMA.md +35 -0
- aimarket_histor-0.2.0/docs/api.es.md +192 -0
- aimarket_histor-0.2.0/docs/api.fr.md +195 -0
- aimarket_histor-0.2.0/docs/api.md +190 -0
- aimarket_histor-0.2.0/docs/api.ru.md +192 -0
- aimarket_histor-0.2.0/docs/api.zh.md +183 -0
- aimarket_histor-0.2.0/docs/architecture.es.md +230 -0
- aimarket_histor-0.2.0/docs/architecture.fr.md +230 -0
- aimarket_histor-0.2.0/docs/architecture.md +221 -0
- aimarket_histor-0.2.0/docs/architecture.ru.md +225 -0
- aimarket_histor-0.2.0/docs/architecture.zh.md +192 -0
- aimarket_histor-0.2.0/docs/badges/license.svg +19 -0
- aimarket_histor-0.2.0/docs/behaviour-v3.md +48 -0
- aimarket_histor-0.2.0/docs/labels.es.md +103 -0
- aimarket_histor-0.2.0/docs/labels.fr.md +106 -0
- aimarket_histor-0.2.0/docs/labels.md +97 -0
- aimarket_histor-0.2.0/docs/labels.ru.md +99 -0
- aimarket_histor-0.2.0/docs/labels.zh.md +91 -0
- aimarket_histor-0.2.0/docs/landing/assets/desk.js +914 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/fonts.css +328 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/fraunces-6NU78FyLNQOQZAnv9bYEvDiIdE9Ea92uemAk_WBq8U_9v0c2Wa0KxC9TeP2Xz5c.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/fraunces-6NU78FyLNQOQZAnv9bYEvDiIdE9Ea92uemAk_WBq8U_9v0c2Wa0KxCBTeP2Xz5fU8w.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/fraunces-6NU78FyLNQOQZAnv9bYEvDiIdE9Ea92uemAk_WBq8U_9v0c2Wa0KxCFTeP2Xz5fU8w.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexmono--F63fjptAgt5VM-kVkqdyU8n1i8q131nj-o.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexmono--F63fjptAgt5VM-kVkqdyU8n1iAq131nj-otFQ.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexmono--F63fjptAgt5VM-kVkqdyU8n1iEq131nj-otFQ.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexmono--F63fjptAgt5VM-kVkqdyU8n1iIq131nj-otFQ.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexmono--F63fjptAgt5VM-kVkqdyU8n1isq131nj-otFQ.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexmono--F6qfjptAgt5VM-kVkqdyU8n3twJwl1FgsAXHNlYzg.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexmono--F6qfjptAgt5VM-kVkqdyU8n3twJwl5FgsAXHNlYzg.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexmono--F6qfjptAgt5VM-kVkqdyU8n3twJwl9FgsAXHNlYzg.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexmono--F6qfjptAgt5VM-kVkqdyU8n3twJwlBFgsAXHNk.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexmono--F6qfjptAgt5VM-kVkqdyU8n3twJwlRFgsAXHNlYzg.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexsans-zYXzKVElMYYaJe8bpLHnCwDKr932-G7dytD-Dmu1syxQKYbSB4ZhRNU.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexsans-zYXzKVElMYYaJe8bpLHnCwDKr932-G7dytD-Dmu1syxRKYbSB4ZhRNU.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexsans-zYXzKVElMYYaJe8bpLHnCwDKr932-G7dytD-Dmu1syxTKYbSB4ZhRNU.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexsans-zYXzKVElMYYaJe8bpLHnCwDKr932-G7dytD-Dmu1syxaKYbSB4ZhRNU.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexsans-zYXzKVElMYYaJe8bpLHnCwDKr932-G7dytD-Dmu1syxdKYbSB4ZhRNU.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/fonts/ibmplexsans-zYXzKVElMYYaJe8bpLHnCwDKr932-G7dytD-Dmu1syxeKYbSB4Zh.woff2 +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/histor.css +242 -0
- aimarket_histor-0.2.0/docs/landing/assets/i18n/es.js +354 -0
- aimarket_histor-0.2.0/docs/landing/assets/i18n/fr.js +354 -0
- aimarket_histor-0.2.0/docs/landing/assets/i18n/ru.js +354 -0
- aimarket_histor-0.2.0/docs/landing/assets/i18n/zh.js +354 -0
- aimarket_histor-0.2.0/docs/landing/assets/i18n.js +228 -0
- aimarket_histor-0.2.0/docs/landing/assets/loom.js +862 -0
- aimarket_histor-0.2.0/docs/landing/assets/og.png +0 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/LICENSE +21 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/README.txt +6 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/controls/OrbitControls.js +1963 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/postprocessing/EffectComposer.js +365 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/postprocessing/MaskPass.js +195 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/postprocessing/OutputPass.js +148 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/postprocessing/Pass.js +191 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/postprocessing/RenderPass.js +193 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/postprocessing/ShaderPass.js +135 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/postprocessing/UnrealBloomPass.js +524 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/shaders/CopyShader.js +52 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/shaders/LuminosityHighPassShader.js +68 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/addons/shaders/OutputShader.js +103 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/three.core.min.js +6 -0
- aimarket_histor-0.2.0/docs/landing/assets/vendor/three/three.module.min.js +6 -0
- aimarket_histor-0.2.0/docs/landing/index.html +263 -0
- aimarket_histor-0.2.0/docs/log.es.md +117 -0
- aimarket_histor-0.2.0/docs/log.fr.md +119 -0
- aimarket_histor-0.2.0/docs/log.md +111 -0
- aimarket_histor-0.2.0/docs/log.ru.md +112 -0
- aimarket_histor-0.2.0/docs/log.zh.md +108 -0
- aimarket_histor-0.2.0/docs/operations.es.md +255 -0
- aimarket_histor-0.2.0/docs/operations.fr.md +258 -0
- aimarket_histor-0.2.0/docs/operations.md +258 -0
- aimarket_histor-0.2.0/docs/operations.ru.md +253 -0
- aimarket_histor-0.2.0/docs/operations.zh.md +235 -0
- aimarket_histor-0.2.0/docs/tool-list-cost.es.md +71 -0
- aimarket_histor-0.2.0/docs/tool-list-cost.fr.md +72 -0
- aimarket_histor-0.2.0/docs/tool-list-cost.md +67 -0
- aimarket_histor-0.2.0/docs/tool-list-cost.ru.md +70 -0
- aimarket_histor-0.2.0/docs/tool-list-cost.zh.md +51 -0
- aimarket_histor-0.2.0/docs/tool-list-tokens-2026-10-19.json +160 -0
- aimarket_histor-0.2.0/histor/__init__.py +12 -0
- aimarket_histor-0.2.0/histor/__main__.py +109 -0
- aimarket_histor-0.2.0/histor/app.py +1035 -0
- aimarket_histor-0.2.0/histor/audit.py +141 -0
- aimarket_histor-0.2.0/histor/badge.py +68 -0
- aimarket_histor-0.2.0/histor/check.py +373 -0
- aimarket_histor-0.2.0/histor/classifier.py +262 -0
- aimarket_histor-0.2.0/histor/config.py +242 -0
- aimarket_histor-0.2.0/histor/crawler.py +1181 -0
- aimarket_histor-0.2.0/histor/curated.json +40087 -0
- aimarket_histor-0.2.0/histor/dataset.py +417 -0
- aimarket_histor-0.2.0/histor/dataset_index.py +197 -0
- aimarket_histor-0.2.0/histor/db.py +196 -0
- aimarket_histor-0.2.0/histor/declared.py +253 -0
- aimarket_histor-0.2.0/histor/diffing.py +192 -0
- aimarket_histor-0.2.0/histor/exit_reasons.py +280 -0
- aimarket_histor-0.2.0/histor/keys.py +100 -0
- aimarket_histor-0.2.0/histor/labels.py +252 -0
- aimarket_histor-0.2.0/histor/logbook.py +146 -0
- aimarket_histor-0.2.0/histor/lookalike.py +127 -0
- aimarket_histor-0.2.0/histor/mcp_stdio.py +111 -0
- aimarket_histor-0.2.0/histor/mcpclient.py +400 -0
- aimarket_histor-0.2.0/histor/merkle.py +156 -0
- aimarket_histor-0.2.0/histor/migrations.py +475 -0
- aimarket_histor-0.2.0/histor/netguard.py +106 -0
- aimarket_histor-0.2.0/histor/packages.py +468 -0
- aimarket_histor-0.2.0/histor/receipts.py +203 -0
- aimarket_histor-0.2.0/histor/registry.py +314 -0
- aimarket_histor-0.2.0/histor/scanner.py +95 -0
- aimarket_histor-0.2.0/histor/security_events.py +211 -0
- aimarket_histor-0.2.0/histor/service.py +163 -0
- aimarket_histor-0.2.0/histor/signing.py +308 -0
- aimarket_histor-0.2.0/histor/stdio.py +63 -0
- aimarket_histor-0.2.0/histor/store.py +621 -0
- aimarket_histor-0.2.0/histor/subject.py +231 -0
- aimarket_histor-0.2.0/histor/untrusted.py +64 -0
- aimarket_histor-0.2.0/pyproject.toml +85 -0
- aimarket_histor-0.2.0/sandbox/histor-firewall.sh +52 -0
- aimarket_histor-0.2.0/sandbox/histor-sandbox.service +29 -0
- aimarket_histor-0.2.0/sandbox/histor_observe.py +1401 -0
- aimarket_histor-0.2.0/sandbox/histor_sink.py +369 -0
- aimarket_histor-0.2.0/sandbox/images/node.Dockerfile +16 -0
- aimarket_histor-0.2.0/sandbox/images/python.Dockerfile +20 -0
- aimarket_histor-0.2.0/sandbox/install.sh +152 -0
- aimarket_histor-0.2.0/scanner/package-lock.json +31 -0
- aimarket_histor-0.2.0/scanner/package.json +14 -0
- aimarket_histor-0.2.0/scanner/quality/check-quality-gate.mjs +64 -0
- aimarket_histor-0.2.0/scanner/quality/quality-public-key.pem +3 -0
- aimarket_histor-0.2.0/scanner/scan.mjs +187 -0
- aimarket_histor-0.2.0/scanner/vendor/aimarket-warden-0.18.0.tgz +0 -0
- aimarket_histor-0.2.0/tests/__init__.py +0 -0
- aimarket_histor-0.2.0/tests/conftest.py +211 -0
- aimarket_histor-0.2.0/tests/integration/__init__.py +0 -0
- aimarket_histor-0.2.0/tests/integration/test_live_stack.py +194 -0
- aimarket_histor-0.2.0/tests/integration/world.py +107 -0
- aimarket_histor-0.2.0/tests/test_app.py +668 -0
- aimarket_histor-0.2.0/tests/test_audit.py +103 -0
- aimarket_histor-0.2.0/tests/test_classifier.py +275 -0
- aimarket_histor-0.2.0/tests/test_config.py +53 -0
- aimarket_histor-0.2.0/tests/test_crawler.py +389 -0
- aimarket_histor-0.2.0/tests/test_dataset.py +162 -0
- aimarket_histor-0.2.0/tests/test_dataset_index.py +51 -0
- aimarket_histor-0.2.0/tests/test_declared.py +343 -0
- aimarket_histor-0.2.0/tests/test_diffing.py +100 -0
- aimarket_histor-0.2.0/tests/test_edges.py +241 -0
- aimarket_histor-0.2.0/tests/test_exit_reasons.py +565 -0
- aimarket_histor-0.2.0/tests/test_federation.py +54 -0
- aimarket_histor-0.2.0/tests/test_hostile.py +353 -0
- aimarket_histor-0.2.0/tests/test_keys_and_signing.py +138 -0
- aimarket_histor-0.2.0/tests/test_labels.py +74 -0
- aimarket_histor-0.2.0/tests/test_landing_csp.py +59 -0
- aimarket_histor-0.2.0/tests/test_landing_i18n.py +73 -0
- aimarket_histor-0.2.0/tests/test_lookalike.py +63 -0
- aimarket_histor-0.2.0/tests/test_mcp_stdio.py +113 -0
- aimarket_histor-0.2.0/tests/test_mcpclient.py +84 -0
- aimarket_histor-0.2.0/tests/test_merkle.py +66 -0
- aimarket_histor-0.2.0/tests/test_migrations.py +76 -0
- aimarket_histor-0.2.0/tests/test_netguard.py +50 -0
- aimarket_histor-0.2.0/tests/test_observer.py +525 -0
- aimarket_histor-0.2.0/tests/test_package_lane_memory.py +45 -0
- aimarket_histor-0.2.0/tests/test_packages.py +589 -0
- aimarket_histor-0.2.0/tests/test_receipts.py +177 -0
- aimarket_histor-0.2.0/tests/test_registry.py +117 -0
- aimarket_histor-0.2.0/tests/test_resume.py +163 -0
- aimarket_histor-0.2.0/tests/test_sandbox_pool.py +172 -0
- aimarket_histor-0.2.0/tests/test_scanner.py +65 -0
- aimarket_histor-0.2.0/tests/test_security_events.py +92 -0
- aimarket_histor-0.2.0/tests/test_service_and_cli.py +100 -0
- aimarket_histor-0.2.0/tests/test_sink.py +157 -0
- aimarket_histor-0.2.0/tests/test_strict_signatures.py +57 -0
- aimarket_histor-0.2.0/tests/test_subject.py +101 -0
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
.venv
|
|
2
|
+
.git
|
|
3
|
+
.github
|
|
4
|
+
.pytest_cache
|
|
5
|
+
.ruff_cache
|
|
6
|
+
__pycache__
|
|
7
|
+
*.pyc
|
|
8
|
+
coverage.xml
|
|
9
|
+
.coverage
|
|
10
|
+
dist
|
|
11
|
+
.env
|
|
12
|
+
data
|
|
13
|
+
tests
|
|
14
|
+
scanner/node_modules
|
|
15
|
+
# Docs stay out of the image except the landing, which the service serves at /.
|
|
16
|
+
docs/**
|
|
17
|
+
!docs/landing/
|
|
18
|
+
!docs/landing/**
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# Copy to .env (chmod 600). Nothing here is baked into the image.
|
|
2
|
+
HISTOR_PROFILE=dev # prod requires the three lines below
|
|
3
|
+
HISTOR_PUBLIC_BASE=http://127.0.0.1:9490 # prod: https://histor.modelmarket.dev
|
|
4
|
+
HISTOR_OPERATOR_TOKEN= # prod: >= 24 chars; guards /api/v1/admin/*
|
|
5
|
+
HISTOR_DATABASE_URL= # empty = SQLite (dev); prod: postgresql://…
|
|
6
|
+
HISTOR_POSTGRES_PASSWORD= # used by docker-compose.postgres.yml
|
|
7
|
+
HISTOR_CRAWL_INTERVAL_S=86400 # 0 = crawl only when an operator asks
|
|
8
|
+
HISTOR_CRAWL_ON_START=1
|
|
9
|
+
HISTOR_CRAWL_WORKERS=12
|
|
10
|
+
HISTOR_CRAWL_PER_HOST=2 # politeness: concurrent connections per host
|
|
11
|
+
HISTOR_CRAWL_TIMEOUT_S=20
|
|
12
|
+
HISTOR_CRAWL_LIMIT=0 # dev: observe only the first N endpoints
|
|
13
|
+
HISTOR_CRAWL_BATCH=400 # endpoints saved together; a restart loses at most one batch
|
|
14
|
+
HISTOR_CHECK_RATE_PER_MIN=30
|
|
15
|
+
HISTOR_CHECK_SCAN_RATE_PER_HOUR=20
|
|
16
|
+
HISTOR_PQC=0 # 1 = hybrid Ed25519 + ML-DSA-65 federation signatures
|
|
17
|
+
# Optional meaning-based classifier (OpenAI-compat). Off unless model + key + max-per-crawl are set.
|
|
18
|
+
# Fleet profile switch (scripts/switch_llm_profile.sh) rewrites MODEL / BASE_URL / CLASSIFIER_API_KEY.
|
|
19
|
+
# HISTOR_CLASSIFIER_MODEL=deepseek-v4-pro
|
|
20
|
+
# HISTOR_CLASSIFIER_BASE_URL=https://api.deepseek.com
|
|
21
|
+
# HISTOR_CLASSIFIER_MAX_PER_CRAWL=50
|
|
22
|
+
# DEEPSEEK_API_KEY=
|
|
23
|
+
# HISTOR_OPENROUTER_API_KEY= # used when BASE_URL is OpenRouter; or set HISTOR_CLASSIFIER_API_KEY
|
|
24
|
+
|
|
25
|
+
# Receipts log (histor/receipts.py): comma-separated did:key issuers whose receipt anchors are
|
|
26
|
+
# accepted. Empty keeps the receipts log closed. For modelmarket.dev, the did:key its
|
|
27
|
+
# provenance plugin issues receipts under.
|
|
28
|
+
HISTOR_RECEIPT_ISSUERS=
|
|
29
|
+
# Package sandbox: npm/PyPI MCP servers observed under gVisor on a separate host (histor/sandbox/,
|
|
30
|
+
# docs/operations.md "Package sandbox"). Off unless all three are set.
|
|
31
|
+
HISTOR_SANDBOX_URL=
|
|
32
|
+
HISTOR_SANDBOX_TOKEN=
|
|
33
|
+
HISTOR_SANDBOX_CERT_SHA256=
|
|
34
|
+
HISTOR_SANDBOX_MAX_PER_CRAWL=300
|
|
35
|
+
HISTOR_SANDBOX_CONCURRENCY=2 # = HISTOR_SANDBOX_SLOTS on the sandbox host
|
|
36
|
+
# More sandbox hosts, up to 8 in all (_2_ … _8_). A host is used only when all three of URL,
|
|
37
|
+
# TOKEN and CERT_SHA256 are set; CONCURRENCY = that host's HISTOR_SANDBOX_SLOTS (1-8).
|
|
38
|
+
#HISTOR_SANDBOX_2_URL=https://<second sandbox host>:9443
|
|
39
|
+
#HISTOR_SANDBOX_2_TOKEN=
|
|
40
|
+
#HISTOR_SANDBOX_2_CERT_SHA256=
|
|
41
|
+
#HISTOR_SANDBOX_2_CONCURRENCY=2
|
|
42
|
+
# Observations in flight across all hosts (1-32): each holds a crawl worker and may buffer 32 MiB.
|
|
43
|
+
# Above HISTOR_CRAWL_WORKERS (12) it has no effect unless the workers are raised too.
|
|
44
|
+
HISTOR_SANDBOX_MAX_SLOTS=12
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# HISTOR: the Python service and the WARDEN scanner sidecar in one image. The sidecar is a
|
|
2
|
+
# Node subprocess per batch, so the base is Node and Python comes from Debian (3.11 on bookworm,
|
|
3
|
+
# which pyproject allows): one libc, one libstdc++, no copied binaries.
|
|
4
|
+
FROM node:22-bookworm-slim
|
|
5
|
+
RUN apt-get update -qq \
|
|
6
|
+
&& apt-get install -y -qq --no-install-recommends python3 python3-venv ca-certificates \
|
|
7
|
+
&& rm -rf /var/lib/apt/lists/*
|
|
8
|
+
WORKDIR /app
|
|
9
|
+
COPY scanner/package.json scanner/package-lock.json ./scanner/
|
|
10
|
+
COPY scanner/vendor ./scanner/vendor
|
|
11
|
+
RUN npm ci --prefix scanner --omit=dev --no-audit --no-fund && npm cache clean --force
|
|
12
|
+
COPY scanner/scan.mjs ./scanner/
|
|
13
|
+
COPY scanner/quality ./scanner/quality
|
|
14
|
+
COPY pyproject.toml uv.lock README.md LICENSE ./
|
|
15
|
+
COPY histor ./histor
|
|
16
|
+
COPY docs/landing ./docs/landing
|
|
17
|
+
# Installed from uv.lock, the set CI tests — not whatever pip resolves on build day. CI runs the
|
|
18
|
+
# suite on 3.11 as well, the Python this Debian base ships.
|
|
19
|
+
RUN python3 -m venv /opt/venv && /opt/venv/bin/pip install --no-cache-dir uv==0.11.17 \
|
|
20
|
+
&& UV_PROJECT_ENVIRONMENT=/opt/venv /opt/venv/bin/uv sync --frozen --no-dev --no-editable \
|
|
21
|
+
--extra postgres --extra pqc --python /opt/venv/bin/python \
|
|
22
|
+
&& /opt/venv/bin/pip uninstall -y -q uv && rm -rf /root/.cache \
|
|
23
|
+
&& mkdir -p /data && chown 65532:65532 /data
|
|
24
|
+
ENV PATH="/opt/venv/bin:${PATH}" \
|
|
25
|
+
HISTOR_HOST=0.0.0.0 \
|
|
26
|
+
HISTOR_PORT=9490 \
|
|
27
|
+
HISTOR_DATA_DIR=/data \
|
|
28
|
+
HISTOR_SCANNER_DIR=/app/scanner \
|
|
29
|
+
HISTOR_LANDING_DIR=/app/docs/landing \
|
|
30
|
+
PYTHONDONTWRITEBYTECODE=1 \
|
|
31
|
+
PYTHONUNBUFFERED=1
|
|
32
|
+
USER 65532:65532
|
|
33
|
+
VOLUME ["/data"]
|
|
34
|
+
EXPOSE 9490
|
|
35
|
+
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
|
|
36
|
+
CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:9490/health', timeout=4).read()"
|
|
37
|
+
CMD ["python", "-m", "histor", "serve"]
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 alexar76
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
22
|
+
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
.PHONY: test integration lint run crawl scanner
|
|
2
|
+
|
|
3
|
+
scanner:
|
|
4
|
+
npm ci --prefix scanner --no-audit --no-fund
|
|
5
|
+
|
|
6
|
+
test: scanner
|
|
7
|
+
uv sync --extra dev --project .
|
|
8
|
+
uv run --project . ruff check .
|
|
9
|
+
uv run --project . pytest --cov=histor --cov-branch -m "not integration"
|
|
10
|
+
|
|
11
|
+
integration: scanner
|
|
12
|
+
uv run --project . pytest -m integration
|
|
13
|
+
|
|
14
|
+
lint:
|
|
15
|
+
uv run --project . ruff check .
|
|
16
|
+
|
|
17
|
+
run: scanner
|
|
18
|
+
uv run --project . python -m histor serve
|
|
19
|
+
|
|
20
|
+
crawl: scanner
|
|
21
|
+
uv run --project . python -m histor crawl
|
|
@@ -0,0 +1,195 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: aimarket-histor
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: HISTOR — public transparency log of MCP tool definitions: signed MTL/1 labels over what each server advertised, when it changed, and a Merkle log anyone can audit
|
|
5
|
+
Project-URL: Documentation, https://github.com/alexar76/histor#readme
|
|
6
|
+
Project-URL: Issues, https://github.com/alexar76/histor/issues
|
|
7
|
+
Project-URL: Repository, https://github.com/alexar76/histor
|
|
8
|
+
Project-URL: Homepage, https://histor.modelmarket.dev/
|
|
9
|
+
License: MIT
|
|
10
|
+
License-File: LICENSE
|
|
11
|
+
Requires-Python: >=3.11
|
|
12
|
+
Requires-Dist: awr<3,>=2.0
|
|
13
|
+
Requires-Dist: cryptography>=44
|
|
14
|
+
Requires-Dist: fastapi>=0.115
|
|
15
|
+
Requires-Dist: httpx>=0.27
|
|
16
|
+
Requires-Dist: uvicorn>=0.34
|
|
17
|
+
Provides-Extra: dev
|
|
18
|
+
Requires-Dist: dilithium-py>=1.4.0; extra == 'dev'
|
|
19
|
+
Requires-Dist: psycopg[binary,pool]<4,>=3.2; extra == 'dev'
|
|
20
|
+
Requires-Dist: pytest-cov>=5; extra == 'dev'
|
|
21
|
+
Requires-Dist: pytest>=8; extra == 'dev'
|
|
22
|
+
Requires-Dist: ruff<1,>=0.12; extra == 'dev'
|
|
23
|
+
Provides-Extra: postgres
|
|
24
|
+
Requires-Dist: psycopg[binary,pool]<4,>=3.2; extra == 'postgres'
|
|
25
|
+
Provides-Extra: pqc
|
|
26
|
+
Requires-Dist: dilithium-py>=1.4.0; extra == 'pqc'
|
|
27
|
+
Description-Content-Type: text/markdown
|
|
28
|
+
|
|
29
|
+
# HISTOR
|
|
30
|
+
|
|
31
|
+
<!-- aicom-readme-badges -->
|
|
32
|
+
<p align="center">
|
|
33
|
+
<a href="https://github.com/alexar76/histor/actions/workflows/ci.yml"><img src="https://github.com/alexar76/histor/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
|
|
34
|
+
<a href="https://github.com/alexar76/histor/actions/workflows/pages.yml"><img src="https://github.com/alexar76/histor/actions/workflows/pages.yml/badge.svg" alt="Pages" /></a>
|
|
35
|
+
<a href="https://histor.modelmarket.dev/"><img src="https://img.shields.io/website?url=https%3A%2F%2Fhistor.modelmarket.dev%2Fhealth&label=live&up_message=up&down_message=down" alt="Live service" /></a>
|
|
36
|
+
<a href="https://histor.modelmarket.dev/log"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fhistor.modelmarket.dev%2Fapi%2Fv1%2Fbadges%2Flog" alt="Labels in the log (live)" /></a>
|
|
37
|
+
<a href="https://histor.modelmarket.dev/servers"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fhistor.modelmarket.dev%2Fapi%2Fv1%2Fbadges%2Fpinned" alt="Tool sets pinned (live)" /></a>
|
|
38
|
+
<a href="https://histor.modelmarket.dev/changes"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fhistor.modelmarket.dev%2Fapi%2Fv1%2Fbadges%2Fchanges" alt="Changes in the last 7 days (live)" /></a>
|
|
39
|
+
<a href="https://histor.modelmarket.dev/log"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fhistor.modelmarket.dev%2Fapi%2Fv1%2Fbadges%2Fsth" alt="Age of the last signed tree head (live)" /></a>
|
|
40
|
+
<a href="https://github.com/alexar76/histor/actions/workflows/pages.yml"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Falexar76.github.io%2Fhistor%2Fbadges%2Ftests.json" alt="Tests (measured by CI)" /></a>
|
|
41
|
+
<a href="https://github.com/alexar76/histor/actions/workflows/pages.yml"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Falexar76.github.io%2Fhistor%2Fbadges%2Fcoverage.json" alt="Branch coverage (measured by CI)" /></a>
|
|
42
|
+
<img src="https://img.shields.io/badge/python-%3E%3D3.11-3776AB" alt="Python >=3.11" />
|
|
43
|
+
<img src="https://img.shields.io/badge/store-Postgres%20%C2%B7%20SQLite%20dev-336791" alt="Postgres in production, SQLite for development" />
|
|
44
|
+
<img src="https://img.shields.io/badge/labels-MTL%2F1%20on%20AWR%2F2-8d83ff" alt="MTL/1 labels on AWR/2" />
|
|
45
|
+
<img src="https://img.shields.io/badge/log-RFC%209162-5fe3ff" alt="RFC 9162 Merkle log" />
|
|
46
|
+
<img src="https://img.shields.io/badge/docs-EN%20RU%20ES%20FR%20ZH-9c70ff" alt="Documentation in 5 languages" />
|
|
47
|
+
<a href="https://github.com/alexar76/histor/blob/main/LICENSE"><img src="https://raw.githubusercontent.com/alexar76/histor/refs/heads/main/docs/badges/license.svg" alt="License: MIT" /></a>
|
|
48
|
+
</p>
|
|
49
|
+
<!-- /aicom-readme-badges -->
|
|
50
|
+
|
|
51
|
+
<p align="center">
|
|
52
|
+
<strong>HISTOR</strong> (ἵστωρ, “the one who knows because they saw”) — a public transparency log of MCP tool definitions<br>
|
|
53
|
+
What every remote MCP endpoint in the official registry advertised · when it changed · signed, appended, auditable
|
|
54
|
+
</p>
|
|
55
|
+
|
|
56
|
+
<p align="center">
|
|
57
|
+
<a href="README.md"><b>English</b></a> ·
|
|
58
|
+
<a href="README.ru.md">Русский</a> ·
|
|
59
|
+
<a href="README.es.md">Español</a> ·
|
|
60
|
+
<a href="README.fr.md">Français</a> ·
|
|
61
|
+
<a href="README.zh.md">中文</a>
|
|
62
|
+
</p>
|
|
63
|
+
|
|
64
|
+
> [WARDEN quality cycle: MOMUS → AI-Factory → SKOPOS → shared node agent → deployment](https://github.com/alexar76/momus/blob/main/docs/quality-cycle.md).
|
|
65
|
+
|
|
66
|
+
**Live:** [histor.modelmarket.dev](https://histor.modelmarket.dev) ·
|
|
67
|
+
**Landing:** [alexar76.github.io/histor](https://alexar76.github.io/histor/) ·
|
|
68
|
+
**Capabilities:** `histor.check@v1` · `histor.server@v1` · `histor.changes@v1` ·
|
|
69
|
+
**Port:** `9490`
|
|
70
|
+
|
|
71
|
+
## The problem, in one paragraph
|
|
72
|
+
|
|
73
|
+
An MCP server can pass your review with harmless tool descriptions and change them later —
|
|
74
|
+
a new sentence in a description is all a prompt injection needs, and your client will pass it
|
|
75
|
+
to the model without asking you again. MCP has no content addressing for tool definitions, the
|
|
76
|
+
official registry verifies namespaces rather than contents, and a scanner on your laptop sees
|
|
77
|
+
only what the server shows your laptop. HISTOR is the missing public memory: it records what
|
|
78
|
+
each server advertised, signs it, and tells anyone who asks whether what they received is what
|
|
79
|
+
everyone else is seeing.
|
|
80
|
+
|
|
81
|
+
## What it does
|
|
82
|
+
|
|
83
|
+
```mermaid
|
|
84
|
+
flowchart LR
|
|
85
|
+
R["Official MCP registry<br/>~21k servers"] --> C["Crawler<br/>initialize + tools/list<br/>never a tool call"]
|
|
86
|
+
C --> D["MTL/1 digest<br/>RFC 8785 + SHA-256"]
|
|
87
|
+
D --> L["Four signed labels<br/>AWR/2 · did:key"]
|
|
88
|
+
W["WARDEN sidecar<br/>published pattern set"] --> L
|
|
89
|
+
L --> M["Merkle log<br/>RFC 9162"]
|
|
90
|
+
M --> S["Signed tree head<br/>after every crawl"]
|
|
91
|
+
M --> API["Desk · API · badge · Atom feed"]
|
|
92
|
+
CL["Your client"] -- "tools it received" --> K["/api/v1/check"]
|
|
93
|
+
K -- "signed: same · different · seen before" --> CL
|
|
94
|
+
K -. "opt-in digest count" .-> M
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
- **Reads** every remote endpoint in the official registry: `initialize`, then `tools/list`
|
|
98
|
+
drained across pages. No tool is called, nothing is installed or executed, private addresses
|
|
99
|
+
are refused before a connection opens.
|
|
100
|
+
- **Also reads** a short list of popular remote servers the registry does not carry ([`histor/curated.json`](histor/curated.json), DeepWiki among them), under HISTOR's own namespace `urn:awr:mtl:1:registry:histor.modelmarket.dev/curated`, so no label claims a registry listing. When such a server publishes to the registry, the registry's listing replaces ours.
|
|
101
|
+
- **Runs npm and PyPI packages** — the servers people start locally with `npx` or `uvx` — in a [sandbox](docs/operations.md#package-sandbox): installed without running their code, started under gVisor with decoy credentials and no route out, read with `initialize` + `tools/list`, then up to 15 tools called once with canary arguments, inside the sandbox. gVisor's own trace says what the package did — programs started, names looked up, decoys opened — at install, at startup and when its tools were called. Completed observations are cached per published version; incomplete observations are retried; a new version that changes what its tools tell the model is a change in the log, like any endpoint's.
|
|
102
|
+
- **Digests** the tool set exactly as [MTL/1](https://github.com/alexar76/aicom/blob/main/awr/adoption/mcp-trust-label/PROFILE.md)
|
|
103
|
+
defines it — name, description, input and output schema, UTF-16 code-unit order, RFC 8785.
|
|
104
|
+
- **Signs** four kinds of label, each a standalone AWR/2 `VerificationVerdict`: what was observed,
|
|
105
|
+
what the WARDEN pattern set matched, whether the definitions changed since the previous label,
|
|
106
|
+
and whether the name is on a threat list.
|
|
107
|
+
- **Appends** every label to an RFC 9162 Merkle log and signs a tree head after each crawl, so a
|
|
108
|
+
consistency proof shows the history was only ever appended to.
|
|
109
|
+
- **Answers** `/check`: send the tools your client received; get back, signed, whether HISTOR
|
|
110
|
+
observed the same set at that endpoint, an earlier one, or none.
|
|
111
|
+
|
|
112
|
+
## What a label does not say
|
|
113
|
+
|
|
114
|
+
The profile forbids the words *safe, secure, audited, certified, approved* and *trusted*, and so
|
|
115
|
+
does this README. A pattern match is a reason to read a definition, not a finding. Three of the
|
|
116
|
+
four methods can never return `fail`; the fourth (continuity) fails only in the mechanical sense
|
|
117
|
+
that two digests differ. No source is read, no package resolved, no behaviour observed. A change
|
|
118
|
+
is shown as a date and a diff, never as an accusation. Details: [docs/labels.md](docs/labels.md).
|
|
119
|
+
|
|
120
|
+
**What a tool list costs.** A client resends every connected server's tool list with each model request. Over the 18,204 packages and 16,170 remote servers in the log, a list's median is about 1,300 tokens, the 90th percentile about 8,000, and 31 packages list over 100,000: [docs/tool-list-cost.md](docs/tool-list-cost.md).
|
|
121
|
+
|
|
122
|
+
**Download the data.** Once a month the log's whole view is published as files at [histor.modelmarket.dev/datasets/](https://histor.modelmarket.dev/datasets/): every listed server, every tool set it showed, every recorded change and the WARDEN scans. The files are JSON Lines (gzip) with a manifest signed by the log's key; the compilation is CC BY 4.0. Each snapshot's README lists the fields and loads in pandas or DuckDB ([histor/dataset.py](histor/dataset.py)). Also on [Hugging Face](https://huggingface.co/datasets/alexar76/histor-mcp-tool-definitions); to cite: [doi:10.5281/zenodo.23512759](https://doi.org/10.5281/zenodo.23512759).
|
|
123
|
+
|
|
124
|
+
## Quick start
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
cd histor
|
|
128
|
+
npm ci --prefix scanner # the WARDEN sidecar (node >= 20)
|
|
129
|
+
uv sync --extra dev --project .
|
|
130
|
+
HISTOR_CRAWL_LIMIT=40 uv run --project . python -m histor crawl # observe 40 endpoints
|
|
131
|
+
uv run --project . python -m histor serve # desk + API on :9490 (SQLite under ./data)
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
For a local MCP client or Glama, run `python -m histor mcp`. It exposes
|
|
135
|
+
`histor_changes`, `histor_check` and `histor_server` over stdio using the local log.
|
|
136
|
+
It opens no HTTP port and starts no crawl; use the separate `crawl` command above to
|
|
137
|
+
populate a fresh log. See [Glama build settings](docs/GLAMA.md).
|
|
138
|
+
|
|
139
|
+
Production runs on Postgres, and `HISTOR_PROFILE=prod` refuses to start without it:
|
|
140
|
+
|
|
141
|
+
```bash
|
|
142
|
+
HISTOR_POSTGRES_PASSWORD=… HISTOR_OPERATOR_TOKEN=… HISTOR_PUBLIC_BASE=https://histor.example \
|
|
143
|
+
docker compose -f docker-compose.yml -f docker-compose.postgres.yml up -d
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
Schema changes are numbered migrations (`python -m histor migrate up|status`), applied before the
|
|
147
|
+
service listens; see [docs/operations.md](docs/operations.md).
|
|
148
|
+
|
|
149
|
+
## Ask the log
|
|
150
|
+
|
|
151
|
+
```bash
|
|
152
|
+
# Is what my client received what HISTOR observed at that endpoint?
|
|
153
|
+
curl -sS https://histor.modelmarket.dev/api/v1/check -H 'Content-Type: application/json' \
|
|
154
|
+
-d '{"endpoint":"https://example.com/mcp","tools":[ …the tools/list result… ]}'
|
|
155
|
+
|
|
156
|
+
# The signed tree head, and a proof that today's log extends yesterday's
|
|
157
|
+
curl -sS https://histor.modelmarket.dev/api/v1/log/sth
|
|
158
|
+
curl -sS "https://histor.modelmarket.dev/api/v1/log/proof/consistency?first=1000&second=1200"
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
Every endpoint is in [docs/api.md](docs/api.md). How to audit the log without trusting HISTOR is
|
|
162
|
+
in [docs/log.md](docs/log.md). The internals are in [docs/architecture.md](docs/architecture.md).
|
|
163
|
+
|
|
164
|
+
## Tests
|
|
165
|
+
|
|
166
|
+
```bash
|
|
167
|
+
make test # unit tests; set HISTOR_TEST_DATABASE_URL to run each storage test on Postgres too
|
|
168
|
+
make integration # real sockets: a loopback registry and MCP servers, uvicorn, the real WARDEN sidecar
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
The tests and coverage badges above are measured by CI on every Pages deploy and read through
|
|
172
|
+
shields.io, and the log badges read the live service, so every number there is current.
|
|
173
|
+
|
|
174
|
+
## ERC-8004 identity
|
|
175
|
+
|
|
176
|
+
HISTOR is ERC-8004 agent [`96683` on Base](https://8004scan.io/agents/base/96683), registered on
|
|
177
|
+
2026-10-01 in the canonical IdentityRegistry `0x8004A169…a432` and owned by the AIMarket operator
|
|
178
|
+
wallet `0x1218ff36…Ad0a`. Its [registration file](https://modelmarket.dev/.well-known/erc-8004/histor.json)
|
|
179
|
+
lists the web page, the hosted `histor-check` endpoint and the `did:key` that signs every label, so a
|
|
180
|
+
label's signature can be tied back to the registered agent. Transactions and how to check them:
|
|
181
|
+
[ERC-8004 identities](https://github.com/alexar76/aicom/blob/main/docs/erc-8004-identities.md).
|
|
182
|
+
|
|
183
|
+
## Where it sits
|
|
184
|
+
|
|
185
|
+
| Component | Role |
|
|
186
|
+
|---|---|
|
|
187
|
+
| [WARDEN](https://github.com/alexar76/warden) | checks tool definitions at connect time, on the client |
|
|
188
|
+
| **HISTOR** | remembers what servers advertised, publicly |
|
|
189
|
+
| [THEMIS](https://github.com/alexar76/themis) | admits a capability at publish time on a Hub |
|
|
190
|
+
| [AIMarket Hub](https://modelmarket.dev) | lists `histor.check@v1` in the catalogue |
|
|
191
|
+
| [AWR](https://github.com/alexar76/aicom/tree/main/awr) | the signed document format every label uses |
|
|
192
|
+
|
|
193
|
+
## License
|
|
194
|
+
|
|
195
|
+
MIT — see [LICENSE](LICENSE). Part of the [AIMarket ecosystem](https://ecosystem.modelmarket.dev/).
|
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
# HISTOR
|
|
2
|
+
|
|
3
|
+
<!-- aicom-readme-badges -->
|
|
4
|
+
<p align="center">
|
|
5
|
+
<a href="https://github.com/alexar76/histor/actions/workflows/ci.yml"><img src="https://github.com/alexar76/histor/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
|
|
6
|
+
<a href="https://github.com/alexar76/histor/actions/workflows/pages.yml"><img src="https://github.com/alexar76/histor/actions/workflows/pages.yml/badge.svg" alt="Pages" /></a>
|
|
7
|
+
<a href="https://histor.modelmarket.dev/"><img src="https://img.shields.io/website?url=https%3A%2F%2Fhistor.modelmarket.dev%2Fhealth&label=live&up_message=up&down_message=down" alt="Live service" /></a>
|
|
8
|
+
<a href="https://histor.modelmarket.dev/log"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fhistor.modelmarket.dev%2Fapi%2Fv1%2Fbadges%2Flog" alt="Labels in the log (live)" /></a>
|
|
9
|
+
<a href="https://histor.modelmarket.dev/servers"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fhistor.modelmarket.dev%2Fapi%2Fv1%2Fbadges%2Fpinned" alt="Tool sets pinned (live)" /></a>
|
|
10
|
+
<a href="https://histor.modelmarket.dev/changes"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fhistor.modelmarket.dev%2Fapi%2Fv1%2Fbadges%2Fchanges" alt="Changes in the last 7 days (live)" /></a>
|
|
11
|
+
<a href="https://histor.modelmarket.dev/log"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fhistor.modelmarket.dev%2Fapi%2Fv1%2Fbadges%2Fsth" alt="Age of the last signed tree head (live)" /></a>
|
|
12
|
+
<a href="https://github.com/alexar76/histor/actions/workflows/pages.yml"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Falexar76.github.io%2Fhistor%2Fbadges%2Ftests.json" alt="Tests (measured by CI)" /></a>
|
|
13
|
+
<a href="https://github.com/alexar76/histor/actions/workflows/pages.yml"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Falexar76.github.io%2Fhistor%2Fbadges%2Fcoverage.json" alt="Branch coverage (measured by CI)" /></a>
|
|
14
|
+
<img src="https://img.shields.io/badge/python-%3E%3D3.11-3776AB" alt="Python >=3.11" />
|
|
15
|
+
<img src="https://img.shields.io/badge/store-Postgres%20%C2%B7%20SQLite%20dev-336791" alt="Postgres in production, SQLite for development" />
|
|
16
|
+
<img src="https://img.shields.io/badge/labels-MTL%2F1%20on%20AWR%2F2-8d83ff" alt="MTL/1 labels on AWR/2" />
|
|
17
|
+
<img src="https://img.shields.io/badge/log-RFC%209162-5fe3ff" alt="RFC 9162 Merkle log" />
|
|
18
|
+
<img src="https://img.shields.io/badge/docs-EN%20RU%20ES%20FR%20ZH-9c70ff" alt="Documentation in 5 languages" />
|
|
19
|
+
<a href="https://github.com/alexar76/histor/blob/main/LICENSE"><img src="https://raw.githubusercontent.com/alexar76/histor/refs/heads/main/docs/badges/license.svg" alt="License: MIT" /></a>
|
|
20
|
+
</p>
|
|
21
|
+
<!-- /aicom-readme-badges -->
|
|
22
|
+
|
|
23
|
+
<p align="center">
|
|
24
|
+
<strong>HISTOR</strong> (ἵστωρ, “the one who knows because they saw”) — a public transparency log of MCP tool definitions<br>
|
|
25
|
+
What every remote MCP endpoint in the official registry advertised · when it changed · signed, appended, auditable
|
|
26
|
+
</p>
|
|
27
|
+
|
|
28
|
+
<p align="center">
|
|
29
|
+
<a href="README.md"><b>English</b></a> ·
|
|
30
|
+
<a href="README.ru.md">Русский</a> ·
|
|
31
|
+
<a href="README.es.md">Español</a> ·
|
|
32
|
+
<a href="README.fr.md">Français</a> ·
|
|
33
|
+
<a href="README.zh.md">中文</a>
|
|
34
|
+
</p>
|
|
35
|
+
|
|
36
|
+
> [WARDEN quality cycle: MOMUS → AI-Factory → SKOPOS → shared node agent → deployment](https://github.com/alexar76/momus/blob/main/docs/quality-cycle.md).
|
|
37
|
+
|
|
38
|
+
**Live:** [histor.modelmarket.dev](https://histor.modelmarket.dev) ·
|
|
39
|
+
**Landing:** [alexar76.github.io/histor](https://alexar76.github.io/histor/) ·
|
|
40
|
+
**Capabilities:** `histor.check@v1` · `histor.server@v1` · `histor.changes@v1` ·
|
|
41
|
+
**Port:** `9490`
|
|
42
|
+
|
|
43
|
+
## The problem, in one paragraph
|
|
44
|
+
|
|
45
|
+
An MCP server can pass your review with harmless tool descriptions and change them later —
|
|
46
|
+
a new sentence in a description is all a prompt injection needs, and your client will pass it
|
|
47
|
+
to the model without asking you again. MCP has no content addressing for tool definitions, the
|
|
48
|
+
official registry verifies namespaces rather than contents, and a scanner on your laptop sees
|
|
49
|
+
only what the server shows your laptop. HISTOR is the missing public memory: it records what
|
|
50
|
+
each server advertised, signs it, and tells anyone who asks whether what they received is what
|
|
51
|
+
everyone else is seeing.
|
|
52
|
+
|
|
53
|
+
## What it does
|
|
54
|
+
|
|
55
|
+
```mermaid
|
|
56
|
+
flowchart LR
|
|
57
|
+
R["Official MCP registry<br/>~21k servers"] --> C["Crawler<br/>initialize + tools/list<br/>never a tool call"]
|
|
58
|
+
C --> D["MTL/1 digest<br/>RFC 8785 + SHA-256"]
|
|
59
|
+
D --> L["Four signed labels<br/>AWR/2 · did:key"]
|
|
60
|
+
W["WARDEN sidecar<br/>published pattern set"] --> L
|
|
61
|
+
L --> M["Merkle log<br/>RFC 9162"]
|
|
62
|
+
M --> S["Signed tree head<br/>after every crawl"]
|
|
63
|
+
M --> API["Desk · API · badge · Atom feed"]
|
|
64
|
+
CL["Your client"] -- "tools it received" --> K["/api/v1/check"]
|
|
65
|
+
K -- "signed: same · different · seen before" --> CL
|
|
66
|
+
K -. "opt-in digest count" .-> M
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
- **Reads** every remote endpoint in the official registry: `initialize`, then `tools/list`
|
|
70
|
+
drained across pages. No tool is called, nothing is installed or executed, private addresses
|
|
71
|
+
are refused before a connection opens.
|
|
72
|
+
- **Also reads** a short list of popular remote servers the registry does not carry ([`histor/curated.json`](histor/curated.json), DeepWiki among them), under HISTOR's own namespace `urn:awr:mtl:1:registry:histor.modelmarket.dev/curated`, so no label claims a registry listing. When such a server publishes to the registry, the registry's listing replaces ours.
|
|
73
|
+
- **Runs npm and PyPI packages** — the servers people start locally with `npx` or `uvx` — in a [sandbox](docs/operations.md#package-sandbox): installed without running their code, started under gVisor with decoy credentials and no route out, read with `initialize` + `tools/list`, then up to 15 tools called once with canary arguments, inside the sandbox. gVisor's own trace says what the package did — programs started, names looked up, decoys opened — at install, at startup and when its tools were called. Completed observations are cached per published version; incomplete observations are retried; a new version that changes what its tools tell the model is a change in the log, like any endpoint's.
|
|
74
|
+
- **Digests** the tool set exactly as [MTL/1](https://github.com/alexar76/aicom/blob/main/awr/adoption/mcp-trust-label/PROFILE.md)
|
|
75
|
+
defines it — name, description, input and output schema, UTF-16 code-unit order, RFC 8785.
|
|
76
|
+
- **Signs** four kinds of label, each a standalone AWR/2 `VerificationVerdict`: what was observed,
|
|
77
|
+
what the WARDEN pattern set matched, whether the definitions changed since the previous label,
|
|
78
|
+
and whether the name is on a threat list.
|
|
79
|
+
- **Appends** every label to an RFC 9162 Merkle log and signs a tree head after each crawl, so a
|
|
80
|
+
consistency proof shows the history was only ever appended to.
|
|
81
|
+
- **Answers** `/check`: send the tools your client received; get back, signed, whether HISTOR
|
|
82
|
+
observed the same set at that endpoint, an earlier one, or none.
|
|
83
|
+
|
|
84
|
+
## What a label does not say
|
|
85
|
+
|
|
86
|
+
The profile forbids the words *safe, secure, audited, certified, approved* and *trusted*, and so
|
|
87
|
+
does this README. A pattern match is a reason to read a definition, not a finding. Three of the
|
|
88
|
+
four methods can never return `fail`; the fourth (continuity) fails only in the mechanical sense
|
|
89
|
+
that two digests differ. No source is read, no package resolved, no behaviour observed. A change
|
|
90
|
+
is shown as a date and a diff, never as an accusation. Details: [docs/labels.md](docs/labels.md).
|
|
91
|
+
|
|
92
|
+
**What a tool list costs.** A client resends every connected server's tool list with each model request. Over the 18,204 packages and 16,170 remote servers in the log, a list's median is about 1,300 tokens, the 90th percentile about 8,000, and 31 packages list over 100,000: [docs/tool-list-cost.md](docs/tool-list-cost.md).
|
|
93
|
+
|
|
94
|
+
**Download the data.** Once a month the log's whole view is published as files at [histor.modelmarket.dev/datasets/](https://histor.modelmarket.dev/datasets/): every listed server, every tool set it showed, every recorded change and the WARDEN scans. The files are JSON Lines (gzip) with a manifest signed by the log's key; the compilation is CC BY 4.0. Each snapshot's README lists the fields and loads in pandas or DuckDB ([histor/dataset.py](histor/dataset.py)). Also on [Hugging Face](https://huggingface.co/datasets/alexar76/histor-mcp-tool-definitions); to cite: [doi:10.5281/zenodo.23512759](https://doi.org/10.5281/zenodo.23512759).
|
|
95
|
+
|
|
96
|
+
## Quick start
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
cd histor
|
|
100
|
+
npm ci --prefix scanner # the WARDEN sidecar (node >= 20)
|
|
101
|
+
uv sync --extra dev --project .
|
|
102
|
+
HISTOR_CRAWL_LIMIT=40 uv run --project . python -m histor crawl # observe 40 endpoints
|
|
103
|
+
uv run --project . python -m histor serve # desk + API on :9490 (SQLite under ./data)
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
For a local MCP client or Glama, run `python -m histor mcp`. It exposes
|
|
107
|
+
`histor_changes`, `histor_check` and `histor_server` over stdio using the local log.
|
|
108
|
+
It opens no HTTP port and starts no crawl; use the separate `crawl` command above to
|
|
109
|
+
populate a fresh log. See [Glama build settings](docs/GLAMA.md).
|
|
110
|
+
|
|
111
|
+
Production runs on Postgres, and `HISTOR_PROFILE=prod` refuses to start without it:
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
HISTOR_POSTGRES_PASSWORD=… HISTOR_OPERATOR_TOKEN=… HISTOR_PUBLIC_BASE=https://histor.example \
|
|
115
|
+
docker compose -f docker-compose.yml -f docker-compose.postgres.yml up -d
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
Schema changes are numbered migrations (`python -m histor migrate up|status`), applied before the
|
|
119
|
+
service listens; see [docs/operations.md](docs/operations.md).
|
|
120
|
+
|
|
121
|
+
## Ask the log
|
|
122
|
+
|
|
123
|
+
```bash
|
|
124
|
+
# Is what my client received what HISTOR observed at that endpoint?
|
|
125
|
+
curl -sS https://histor.modelmarket.dev/api/v1/check -H 'Content-Type: application/json' \
|
|
126
|
+
-d '{"endpoint":"https://example.com/mcp","tools":[ …the tools/list result… ]}'
|
|
127
|
+
|
|
128
|
+
# The signed tree head, and a proof that today's log extends yesterday's
|
|
129
|
+
curl -sS https://histor.modelmarket.dev/api/v1/log/sth
|
|
130
|
+
curl -sS "https://histor.modelmarket.dev/api/v1/log/proof/consistency?first=1000&second=1200"
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Every endpoint is in [docs/api.md](docs/api.md). How to audit the log without trusting HISTOR is
|
|
134
|
+
in [docs/log.md](docs/log.md). The internals are in [docs/architecture.md](docs/architecture.md).
|
|
135
|
+
|
|
136
|
+
## Tests
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
make test # unit tests; set HISTOR_TEST_DATABASE_URL to run each storage test on Postgres too
|
|
140
|
+
make integration # real sockets: a loopback registry and MCP servers, uvicorn, the real WARDEN sidecar
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
The tests and coverage badges above are measured by CI on every Pages deploy and read through
|
|
144
|
+
shields.io, and the log badges read the live service, so every number there is current.
|
|
145
|
+
|
|
146
|
+
## ERC-8004 identity
|
|
147
|
+
|
|
148
|
+
HISTOR is ERC-8004 agent [`96683` on Base](https://8004scan.io/agents/base/96683), registered on
|
|
149
|
+
2026-10-01 in the canonical IdentityRegistry `0x8004A169…a432` and owned by the AIMarket operator
|
|
150
|
+
wallet `0x1218ff36…Ad0a`. Its [registration file](https://modelmarket.dev/.well-known/erc-8004/histor.json)
|
|
151
|
+
lists the web page, the hosted `histor-check` endpoint and the `did:key` that signs every label, so a
|
|
152
|
+
label's signature can be tied back to the registered agent. Transactions and how to check them:
|
|
153
|
+
[ERC-8004 identities](https://github.com/alexar76/aicom/blob/main/docs/erc-8004-identities.md).
|
|
154
|
+
|
|
155
|
+
## Where it sits
|
|
156
|
+
|
|
157
|
+
| Component | Role |
|
|
158
|
+
|---|---|
|
|
159
|
+
| [WARDEN](https://github.com/alexar76/warden) | checks tool definitions at connect time, on the client |
|
|
160
|
+
| **HISTOR** | remembers what servers advertised, publicly |
|
|
161
|
+
| [THEMIS](https://github.com/alexar76/themis) | admits a capability at publish time on a Hub |
|
|
162
|
+
| [AIMarket Hub](https://modelmarket.dev) | lists `histor.check@v1` in the catalogue |
|
|
163
|
+
| [AWR](https://github.com/alexar76/aicom/tree/main/awr) | the signed document format every label uses |
|
|
164
|
+
|
|
165
|
+
## License
|
|
166
|
+
|
|
167
|
+
MIT — see [LICENSE](LICENSE). Part of the [AIMarket ecosystem](https://ecosystem.modelmarket.dev/).
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# Glama build — admin form
|
|
2
|
+
|
|
3
|
+
Listing: [glama.ai/mcp/servers/alexar76/histor](https://glama.ai/mcp/servers/alexar76/histor)
|
|
4
|
+
|
|
5
|
+
Admin: […/admin/dockerfile](https://glama.ai/mcp/servers/alexar76/histor/admin/dockerfile)
|
|
6
|
+
|
|
7
|
+
Glama generates the image from this form and does not build the repo [`Dockerfile`](../Dockerfile). That file is the production image: `node:22-bookworm-slim`, Debian Bookworm's Python 3.11, and `python -m histor serve`.
|
|
8
|
+
|
|
9
|
+
The form only offers `debian:bookworm-slim` and `debian:trixie-slim`. Use bookworm, Node 22 (the scanner sidecar) and Python 3.11 (the interpreter that image actually runs).
|
|
10
|
+
|
|
11
|
+
| Field | Value |
|
|
12
|
+
| --- | --- |
|
|
13
|
+
| Base image | `debian:bookworm-slim` |
|
|
14
|
+
| Node.js version | `22` |
|
|
15
|
+
| Python version | `3.11` |
|
|
16
|
+
| Build steps | `["npm ci --prefix scanner --omit=dev --no-audit --no-fund", "uv sync --frozen --no-dev --extra postgres --extra pqc"]` |
|
|
17
|
+
| CMD arguments | `[".venv/bin/python", "-m", "histor", "mcp"]` |
|
|
18
|
+
| Environment variables | `{"type":"object","properties":{},"required":[]}` |
|
|
19
|
+
| Pinned commit SHA | empty |
|
|
20
|
+
|
|
21
|
+
No secret is required for a dev start. `HISTOR_PROFILE=prod` refuses to start until Postgres and the operator settings exist; leave those unset on this form.
|
|
22
|
+
|
|
23
|
+
The `mcp` command serves `histor_changes`, `histor_check` and `histor_server` over
|
|
24
|
+
stdin/stdout using the local log and the existing signed federation handlers. It opens
|
|
25
|
+
no HTTP listener and starts no crawler. Diagnostics go to stderr; stdout contains only
|
|
26
|
+
MCP JSON-RPC. Ordinary `serve` remains the HTTP service on port 9490.
|
|
27
|
+
|
|
28
|
+
A fresh local log has no observations. Populate it separately with `histor crawl`, or
|
|
29
|
+
point `HISTOR_DATA_DIR` at the intended existing log (and preserve its issuer keys).
|
|
30
|
+
The `histor_check` tool may update digest contribution counts when explicitly requested
|
|
31
|
+
with `contribute: true`, just like the HTTP API. No remote service is substituted for
|
|
32
|
+
this local server. EOF closes the MCP session and the database.
|
|
33
|
+
|
|
34
|
+
After uploading the source, sync the Glama repository and replace the saved CMD with
|
|
35
|
+
the value above before clicking Build: the previous `serve` command is still HTTP.
|