log10x-mcp 1.30.37 → 1.30.39
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/build/index.js +1 -1
- package/build/lib/compile-runner.d.ts +1 -1
- package/build/lib/compile-runner.js +1 -1
- package/build/lib/output-types.d.ts +1 -1
- package/build/lib/output-types.js +1 -1
- package/build/lib/runtime-image.d.ts +1 -1
- package/build/lib/runtime-image.js +1 -1
- package/build/lib/runtime-image.js.map +1 -1
- package/build/lib/server-instructions.js +13 -1
- package/build/lib/server-instructions.js.map +1 -1
- package/build/product-kb/docs/api/js.md +1 -1
- package/build/product-kb/docs/apps/mcp/tools/compile/compile-link.md +23 -8
- package/build/product-kb/docs/apps/mcp/tools/compile/compile-status.md +59 -11
- package/build/product-kb/docs/apps/mcp/tools/compile/compile.md +71 -24
- package/build/product-kb/docs/apps/mcp/tools/compile/index.md +8 -6
- package/build/product-kb/docs/apps/mcp/tools/costs/pattern-diff.md +1 -1
- package/build/product-kb/docs/apps/mcp/tools/costs/whats-new.md +1 -1
- package/build/product-kb/docs/apps/receiver/compact/index.md +1 -1
- package/build/product-kb/docs/apps/receiver/run.md +73 -33
- package/build/product-kb/docs/apps/reporter/run.md +72 -32
- package/build/product-kb/docs/apps/retriever/run.md +72 -32
- package/build/product-kb/docs/engine/launcher/extension.md +30 -21
- package/build/product-kb/docs/install/docker.md +8 -7
- package/build/product-kb/docs/run/initialize/message/index.md +25 -8
- package/build/product-kb/docs/run/transform/index.md +2 -0
- package/build/product-kb/docs/run/transform/symbol/index.md +171 -32
- package/build/tools/compile.js +2 -2
- package/default-manifest.json +2 -2
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -91,7 +91,7 @@ All optional. The common path is just `LOG10X_API_KEY` (or no key at all).
|
|
|
91
91
|
| `LOG10X_TENX_PATH` | Path to the local `tenx` binary (used when the resolved mode is `local`). Defaults to `tenx` on PATH. |
|
|
92
92
|
| `LOG10X_RUNTIME_IMAGE` | Engine image for the run path in docker mode (default `log10x/pipeline-10x:latest`). Accepts the alias `native`, which selects the GraalVM-native `log10x/edge-10x`. |
|
|
93
93
|
| `LOG10X_TENX_IMAGE` | Engine image shared by the run path and, as a fallback, `log10x_compile`. A runtime-flavor image here is refused on the compile path. |
|
|
94
|
-
| `LOG10X_COMPILER_IMAGE` | Compiler image for `log10x_compile` / `log10x_compile_link` (default `log10x/compiler-10x:1.1.
|
|
94
|
+
| `LOG10X_COMPILER_IMAGE` | Compiler image for `log10x_compile` / `log10x_compile_link` (default `log10x/compiler-10x:1.1.136` pinned by digest, so every run goes through the same compiler; set it to `:latest` to track releases; falls back to `LOG10X_TENX_IMAGE` when unset). Only the compiler flavor can build a symbol library. |
|
|
95
95
|
| `LOG10X_GH_REPO` / `LOG10X_GITOPS_REPO_PATH` | GitHub repo and local path for GitOps-aware config tools. |
|
|
96
96
|
|
|
97
97
|
## Connect your own data (optional)
|
package/build/index.js
CHANGED
|
@@ -1318,7 +1318,7 @@ async function handleCliFlags() {
|
|
|
1318
1318
|
' LOG10X_TENX_PATH Path to local tenx CLI (used when LOG10X_TENX_MODE=local). Compile and link need the `compiler` flavor; the `mode` arg of log10x_compile states the three flavors and which are refused',
|
|
1319
1319
|
' LOG10X_TENX_IMAGE Docker image when LOG10X_TENX_MODE=docker, SHARED by the run path and (as a fallback) log10x_compile (default: log10x/pipeline-10x:latest). A runtime-flavor image here is refused on the compile path — use LOG10X_RUNTIME_IMAGE instead',
|
|
1320
1320
|
' LOG10X_RUNTIME_IMAGE Docker image for the run-path engine tools only, leaving log10x_compile alone. Accepts a full ref or the alias `native`, which selects the GraalVM-native log10x/edge-10x:latest (391MB vs 926MB, same @apps/mcp output). Wins over LOG10X_TENX_IMAGE',
|
|
1321
|
-
' LOG10X_COMPILER_IMAGE Docker image for log10x_compile. Default log10x/compiler-10x:1.1.
|
|
1321
|
+
' LOG10X_COMPILER_IMAGE Docker image for log10x_compile. Default log10x/compiler-10x:1.1.136 pinned by digest, so every run goes through the same compiler (the image ref is part of the output-folder key); set it to :latest to track releases. Falls back to LOG10X_TENX_IMAGE',
|
|
1322
1322
|
' TENX_LICENSE_KEY License key passed through to the compiler app (log10x_compile); omit to use the image built-in limited license',
|
|
1323
1323
|
' GH_TOKEN GitHub token for log10x_compile github_repos pull (or pass the github_token arg; required even for public repos)',
|
|
1324
1324
|
' DOCKER_USERNAME Registry username for log10x_compile docker_images pull (or pass the docker_username arg; omit for public images)',
|
|
@@ -303,7 +303,7 @@ export declare class HelmRepoAddError extends Error {
|
|
|
303
303
|
*
|
|
304
304
|
* `LOG10X_COMPILER_IMAGE=log10x/compiler-10x:latest` restores the moving tag.
|
|
305
305
|
*/
|
|
306
|
-
export declare const DEFAULT_IMAGE = "log10x/compiler-10x:1.1.
|
|
306
|
+
export declare const DEFAULT_IMAGE = "log10x/compiler-10x:1.1.136@sha256:ffcd9890935804b06fcb1791994dfbb32cdb270f3e35fc423ede78ced2097b1f";
|
|
307
307
|
/** First engine release where a CLI option given after `@apps/compiler` overrides the app's value. */
|
|
308
308
|
export declare const CLI_OVERRIDE_MIN_ENGINE = "1.1.89";
|
|
309
309
|
/**
|
|
@@ -205,7 +205,7 @@ export class HelmRepoAddError extends Error {
|
|
|
205
205
|
*
|
|
206
206
|
* `LOG10X_COMPILER_IMAGE=log10x/compiler-10x:latest` restores the moving tag.
|
|
207
207
|
*/
|
|
208
|
-
export const DEFAULT_IMAGE = 'log10x/compiler-10x:1.1.
|
|
208
|
+
export const DEFAULT_IMAGE = 'log10x/compiler-10x:1.1.136@sha256:ffcd9890935804b06fcb1791994dfbb32cdb270f3e35fc423ede78ced2097b1f';
|
|
209
209
|
/** First engine release where a CLI option given after `@apps/compiler` overrides the app's value. */
|
|
210
210
|
export const CLI_OVERRIDE_MIN_ENGINE = '1.1.89';
|
|
211
211
|
/**
|
|
@@ -32,7 +32,7 @@ export declare const SCHEMA_VERSION: "1.0";
|
|
|
32
32
|
* Kept simple: an ISO date string. Agents do not parse it; they just
|
|
33
33
|
* compare for equality across calls.
|
|
34
34
|
*/
|
|
35
|
-
export declare const SCHEMA_EPOCH: "2026-
|
|
35
|
+
export declare const SCHEMA_EPOCH: "2026-10-06";
|
|
36
36
|
/**
|
|
37
37
|
* Uniform view enum across all default-loaded tools. Per-tool
|
|
38
38
|
* extensions (e.g. POC's "yaml" / "configs") stay tool-local and do
|
|
@@ -32,7 +32,7 @@ export const SCHEMA_VERSION = '1.0';
|
|
|
32
32
|
* Kept simple: an ISO date string. Agents do not parse it; they just
|
|
33
33
|
* compare for equality across calls.
|
|
34
34
|
*/
|
|
35
|
-
export const SCHEMA_EPOCH = '2026-
|
|
35
|
+
export const SCHEMA_EPOCH = '2026-10-06';
|
|
36
36
|
/**
|
|
37
37
|
* Uniform view enum across all default-loaded tools. Per-tool
|
|
38
38
|
* extensions (e.g. POC's "yaml" / "configs") stay tool-local and do
|
|
@@ -74,7 +74,7 @@ export declare const NATIVE_RUNTIME_IMAGE = "log10x/edge-10x:latest";
|
|
|
74
74
|
* pass-through fix (config #74) ships in this image, checked with a Fluent
|
|
75
75
|
* Bit sidecar: 0 fullText fallbacks, 2,000 of 2,000 lines.
|
|
76
76
|
*/
|
|
77
|
-
export declare const EDGE_MANIFEST_IMAGE = "log10x/edge-10x:1.1.
|
|
77
|
+
export declare const EDGE_MANIFEST_IMAGE = "log10x/edge-10x:1.1.136";
|
|
78
78
|
/**
|
|
79
79
|
* Resolve the image the run-path (`@apps/mcp`) docker backend should use.
|
|
80
80
|
*
|
|
@@ -74,7 +74,7 @@ export const NATIVE_RUNTIME_IMAGE = 'log10x/edge-10x:latest';
|
|
|
74
74
|
* pass-through fix (config #74) ships in this image, checked with a Fluent
|
|
75
75
|
* Bit sidecar: 0 fullText fallbacks, 2,000 of 2,000 lines.
|
|
76
76
|
*/
|
|
77
|
-
export const EDGE_MANIFEST_IMAGE = 'log10x/edge-10x:1.1.
|
|
77
|
+
export const EDGE_MANIFEST_IMAGE = 'log10x/edge-10x:1.1.136';
|
|
78
78
|
/** Aliases accepted by LOG10X_RUNTIME_IMAGE in place of a full image ref. */
|
|
79
79
|
const NATIVE_ALIASES = new Set(['native', 'runtime', 'edge']);
|
|
80
80
|
/**
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"runtime-image.js","sourceRoot":"","sources":["../../src/lib/runtime-image.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAoCG;AAEH,0EAA0E;AAC1E,MAAM,CAAC,MAAM,qBAAqB,GAAG,4BAA4B,CAAC;AAElE,uDAAuD;AACvD,MAAM,CAAC,MAAM,oBAAoB,GAAG,wBAAwB,CAAC;AAE7D;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAkCG;AACH,MAAM,CAAC,MAAM,mBAAmB,GAAG,
|
|
1
|
+
{"version":3,"file":"runtime-image.js","sourceRoot":"","sources":["../../src/lib/runtime-image.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAoCG;AAEH,0EAA0E;AAC1E,MAAM,CAAC,MAAM,qBAAqB,GAAG,4BAA4B,CAAC;AAElE,uDAAuD;AACvD,MAAM,CAAC,MAAM,oBAAoB,GAAG,wBAAwB,CAAC;AAE7D;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAkCG;AACH,MAAM,CAAC,MAAM,mBAAmB,GAAG,yBAAyB,CAAC;AAE7D,6EAA6E;AAC7E,MAAM,cAAc,GAAwB,IAAI,GAAG,CAAC,CAAC,QAAQ,EAAE,SAAS,EAAE,MAAM,CAAC,CAAC,CAAC;AAEnF;;;;;GAKG;AACH,MAAM,mBAAmB,GAAsB,CAAC,UAAU,EAAE,YAAY,CAAC,CAAC;AAE1E;;;;;GAKG;AACH,MAAM,UAAU,mBAAmB,CAAC,MAAyB,OAAO,CAAC,GAAG;IACtE,MAAM,QAAQ,GAAG,CAAC,GAAG,CAAC,oBAAoB,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;IACzD,IAAI,QAAQ,EAAE,CAAC;QACb,OAAO,cAAc,CAAC,GAAG,CAAC,QAAQ,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,CAAC,oBAAoB,CAAC,CAAC,CAAC,QAAQ,CAAC;IACtF,CAAC;IACD,MAAM,MAAM,GAAG,CAAC,GAAG,CAAC,iBAAiB,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;IACpD,OAAO,MAAM,IAAI,qBAAqB,CAAC;AACzC,CAAC;AAED,kFAAkF;AAClF,MAAM,UAAU,mBAAmB,CAAC,KAAa;IAC/C,MAAM,IAAI,GAAG,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,WAAW,EAAE,CAAC;IAC7D,OAAO,mBAAmB,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,KAAK,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;AAC/E,CAAC;AAED;;;;;;GAMG;AACH,MAAM,OAAO,+BAAgC,SAAQ,KAAK;IAC/C,KAAK,CAAS;IACvB,YAAY,KAAa,EAAE,YAAqB;QAC9C,MAAM,MAAM,GAAG,YAAY;YACzB,CAAC,CAAC,4FAA4F;YAC9F,CAAC,CAAC,uBAAuB,CAAC;QAC5B,KAAK,CACH;YACE,kCAAkC,KAAK,qKAAqK;YAC5M,EAAE;YACF,gBAAgB,MAAM,GAAG;YACzB,EAAE;YACF,mBAAmB;YACnB,2JAA2J;YAC3J,8GAA8G;SAC/G,CAAC,IAAI,CAAC,IAAI,CAAC,CACb,CAAC;QACF,IAAI,CAAC,IAAI,GAAG,iCAAiC,CAAC;QAC9C,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;IACrB,CAAC;CACF;AAED;;;;GAIG;AACH,MAAM,UAAU,qBAAqB,CAAC,KAAa,EAAE,YAAqB;IACxE,IAAI,mBAAmB,CAAC,KAAK,CAAC;QAAE,MAAM,IAAI,+BAA+B,CAAC,KAAK,EAAE,YAAY,CAAC,CAAC;AACjG,CAAC"}
|
|
@@ -44,7 +44,19 @@ timestamps, and request IDs. That identity is the key to a Prometheus time serie
|
|
|
44
44
|
so any pattern the user has ever emitted is instantly queryable by name, by history, or by sample
|
|
45
45
|
line with zero prior query setup, the observability memory for their logs.
|
|
46
46
|
|
|
47
|
-
|
|
47
|
+
The identity is a function of the line, the symbol library, and the engine version and its naming
|
|
48
|
+
config, so it changes only when one of those does. An engine upgrade that changes the naming rules
|
|
49
|
+
renames the patterns it affects, once: engine 1.1.133 names a multi-line event from its first record
|
|
50
|
+
that holds a message (the .NET console logger's header record no longer names it) and starts the
|
|
51
|
+
message inside the value of a msg, message or body key, quoted or not; in a record that opens with {
|
|
52
|
+
the keys before that key leave the name. Logger and class names stay in names. After such an upgrade
|
|
53
|
+
the renamed patterns appear in log10x_whats_new and log10x_pattern_diff as new, first seen from the
|
|
54
|
+
upgrade on (pattern_diff lists their old names as retired); their history stays under the old name,
|
|
55
|
+
and mutes or per-pattern sample and compact rows keyed on an old name or hash no longer match them.
|
|
56
|
+
When new patterns cluster at an engine upgrade, tell the user they are the same statements under new
|
|
57
|
+
names and offer to re-derive the per-pattern rules from the new names.
|
|
58
|
+
|
|
59
|
+
VOCABULARY: the compact action minifies events: it replaces each event's repeated structure with a template-plus-values encoding so it lands smaller with every field in the encoding, and the destination's expander rebuilds it at read time. Never call it lossless or fully searchable without the destination's terms: on Splunk each event expands exactly up to 256 KB with the current 10x app (a longer one comes back cut) and a search-bar query needs the app's tenxsearch command to see the full text; on Elasticsearch match, match_phrase and multi_match queries (so Kibana and KQL) see the full text and other query types see the encoded form. Timestamps keep their instant: one written in UTC (Z, +0000) or with no zone comes back exactly, and one with any other offset or zone name (-0700, PDT, GMT) can come back written in another zone. When describing it, say "compact" or "minify", never "compress" or "compression": 10x does not do binary or gzip compression, and that word misleads. (Vendor billing terms are different and fine to use as-is, e.g. a destination that bills on "compressed ingest", or Datadog's "compressed GB" rehydration price, refer to the vendor's own compression, not ours.)
|
|
48
60
|
|
|
49
61
|
NON-LOSSY FIRST: the value proposition is cutting cost WITHOUT losing data. Whenever you list the Receiver actions, LEAD with the keep-everything levers and present the lossy ones as opt-ins the user explicitly chooses. Order: (1) compact/minify, keeps everything, where the destination supports it (Splunk, self-hosted Elasticsearch/OpenSearch; a no-op on managed backends like Datadog, CloudWatch, Coralogix, and on ClickHouse, where measurement put it at about 7% of table bytes. ClickHouse: offload is the lever, the bill is compute, and every ClickHouse dollar this server prints is modeled); (2) tier_down, keeps everything, a cheaper tier in the same destination, still queryable, where it applies (Datadog Flex, CloudWatch IA, Azure Monitor Basic/Auxiliary Logs, Coralogix Monitoring); (3) offload, keeps everything, routes to the customer's own S3, recoverable on demand, the one lever that applies on every destination; then (4) sample, lossy, keep 1 in N; (5) drop, lossy, discard. NEVER lead with drop or sample. When you enumerate the actions to a prospect, say which keep everything and which are lossy opt-ins. Do not offer compact as a saving on a destination where it is a no-op; lead with that destination's cheaper tier where it has one (on Datadog that is Flex, because the moved events stay searchable in Datadog), and with offload where it has none.
|
|
50
62
|
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"server-instructions.js","sourceRoot":"","sources":["../../src/lib/server-instructions.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAEH,OAAO,EAAE,4BAA4B,EAAe,MAAM,mBAAmB,CAAC;AAE9E,0EAA0E;AAC1E,+EAA+E;AAC/E,4EAA4E;AAC5E,6EAA6E;AAC7E,MAAM,cAAc,GAClB;IACE,CAAC,QAAQ,EAAE,QAAQ,CAAC;IACpB,CAAC,SAAS,EAAE,SAAS,CAAC;IACtB,CAAC,eAAe,EAAE,eAAe,CAAC;IAClC,CAAC,YAAY,EAAE,YAAY,CAAC;CAE/B;KACE,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,KAAK,CAAC,EAAE,EAAE;IACnB,MAAM,CAAC,GAAG,4BAA4B,CAAC,EAAE,CAAC,CAAC;IAC3C,2EAA2E;IAC3E,4EAA4E;IAC5E,6DAA6D;IAC7D,MAAM,KAAK,GAAG,MAAM,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;IACjE,OAAO,GAAG,KAAK,IAAI,KAAK,KAAK,CAAC;AAChC,CAAC,CAAC;KACD,IAAI,CAAC,IAAI,CAAC,CAAC;AAEd,MAAM,CAAC,MAAM,mBAAmB,GAAG
|
|
1
|
+
{"version":3,"file":"server-instructions.js","sourceRoot":"","sources":["../../src/lib/server-instructions.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAEH,OAAO,EAAE,4BAA4B,EAAe,MAAM,mBAAmB,CAAC;AAE9E,0EAA0E;AAC1E,+EAA+E;AAC/E,4EAA4E;AAC5E,6EAA6E;AAC7E,MAAM,cAAc,GAClB;IACE,CAAC,QAAQ,EAAE,QAAQ,CAAC;IACpB,CAAC,SAAS,EAAE,SAAS,CAAC;IACtB,CAAC,eAAe,EAAE,eAAe,CAAC;IAClC,CAAC,YAAY,EAAE,YAAY,CAAC;CAE/B;KACE,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,KAAK,CAAC,EAAE,EAAE;IACnB,MAAM,CAAC,GAAG,4BAA4B,CAAC,EAAE,CAAC,CAAC;IAC3C,2EAA2E;IAC3E,4EAA4E;IAC5E,6DAA6D;IAC7D,MAAM,KAAK,GAAG,MAAM,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;IACjE,OAAO,GAAG,KAAK,IAAI,KAAK,KAAK,CAAC;AAChC,CAAC,CAAC;KACD,IAAI,CAAC,IAAI,CAAC,CAAC;AAEd,MAAM,CAAC,MAAM,mBAAmB,GAAG;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;EA0LjC,cAAc;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;m2BA0Mm1B,CAAC"}
|
|
@@ -1290,7 +1290,7 @@ if the first type does not yield a result, the next one is tried etc.
|
|
|
1290
1290
|
|
|
1291
1291
|
Supported values: package, class, method, log, enum, const, text, exec, any (case insensitive).
|
|
1292
1292
|
|
|
1293
|
-
Specifying `any` produces Prometheus-compliant metric name comprised of symbol tokens from the current instance's [template](#TenXBaseObject+template) field.
|
|
1293
|
+
Specifying `any` produces Prometheus-compliant metric name comprised of symbol tokens from the current instance's [template](#TenXBaseObject+template) field, starting at the first word past the line's [preamble](https://doc.log10x.com/run/transform/symbol/#symbolpreambleenclosures).
|
|
1294
1294
|
|
|
1295
1295
|
The `field` argument can limit the sequence search to a target extracted field.
|
|
1296
1296
|
For example, specifying an argument value of: "message" will search for matching symbols only in the object's `message` JSON/KV field (if exists).
|
|
@@ -36,7 +36,9 @@ A very large units tree returns a running `job_id` to poll with [`log10x_compile
|
|
|
36
36
|
|
|
37
37
|
## :material-check-decagram-outline: Prerequisites
|
|
38
38
|
|
|
39
|
-
The Compiler flavor, docker-first: `auto` and `docker` pull `log10x/compiler-10x
|
|
39
|
+
The Compiler flavor, docker-first: `auto` and `docker` pull `log10x/compiler-10x:1.1.125` (pinned by digest), and `local` runs a Compiler-flavor `tenx`. The Runtime flavor is refused; the `mode` argument of [`log10x_compile`](compile.md) lists the three flavors.
|
|
40
|
+
|
|
41
|
+
A link into a folder another compile or link job is still writing joins that job instead of starting a second one.
|
|
40
42
|
|
|
41
43
|
This is the CLI tier, so it needs no Kubernetes, deployed app, or Log10x account.
|
|
42
44
|
|
|
@@ -74,9 +76,13 @@ This is the CLI tier, so it needs no Kubernetes, deployed app, or Log10x account
|
|
|
74
76
|
},
|
|
75
77
|
"mode": {
|
|
76
78
|
"type": "string",
|
|
77
|
-
"enum": [
|
|
79
|
+
"enum": [
|
|
80
|
+
"auto",
|
|
81
|
+
"docker",
|
|
82
|
+
"local"
|
|
83
|
+
],
|
|
78
84
|
"default": "auto",
|
|
79
|
-
"description": "Execution backend, same contract as log10x_compile
|
|
85
|
+
"description": "Execution backend, same contract as log10x_compile's `mode`: `auto` prefers the compiler image, `docker` forces it, `local` uses a compiler-flavor tenx. Only the compiler flavor can link; see log10x_compile `mode` for the three flavors and which are refused."
|
|
80
86
|
},
|
|
81
87
|
"timeout_ms": {
|
|
82
88
|
"type": "integer",
|
|
@@ -93,8 +99,11 @@ This is the CLI tier, so it needs no Kubernetes, deployed app, or Log10x account
|
|
|
93
99
|
"description": "How long to wait inline (ms) for the link to finish before handing back a job_id to poll. Default 45,000 (45s). Linking usually finishes inside this and returns the library in ONE call. A very large units tree returns a running job_id you poll with log10x_compile_status. 0 = fire-and-forget."
|
|
94
100
|
}
|
|
95
101
|
},
|
|
96
|
-
"required": [
|
|
97
|
-
|
|
102
|
+
"required": [
|
|
103
|
+
"units_path"
|
|
104
|
+
],
|
|
105
|
+
"additionalProperties": false,
|
|
106
|
+
"$schema": "http://json-schema.org/draft-07/schema#"
|
|
98
107
|
}
|
|
99
108
|
```
|
|
100
109
|
|
|
@@ -129,10 +138,12 @@ This is the CLI tier, so it needs no Kubernetes, deployed app, or Log10x account
|
|
|
129
138
|
"job_id": "7b2e9c41-3f5a-4d8e-9b21-6c0f1a2d7e44",
|
|
130
139
|
"job_status": "completed",
|
|
131
140
|
"mode": "docker",
|
|
132
|
-
"image": "log10x/compiler-10x",
|
|
141
|
+
"image": "log10x/compiler-10x:1.1.125@sha256:ee875d49b9a25ef98b6b9edfe69a4b60888fd68fd370e8a0020f1b4777ecf12b",
|
|
142
|
+
"compiler": null,
|
|
133
143
|
"exit_code": 0,
|
|
134
144
|
"elapsed_ms": 6240,
|
|
135
145
|
"timed_out": false,
|
|
146
|
+
"loss_markers": [],
|
|
136
147
|
"sources": "link /work/symbols-out",
|
|
137
148
|
"output": {
|
|
138
149
|
"folder": "/work/symbols-out",
|
|
@@ -192,12 +203,16 @@ This is the CLI tier, so it needs no Kubernetes, deployed app, or Log10x account
|
|
|
192
203
|
```typescript
|
|
193
204
|
interface CompileLinkPayload {
|
|
194
205
|
job_id: string;
|
|
195
|
-
job_status: 'running' | 'completed' | 'failed' | 'timed_out';
|
|
206
|
+
job_status: 'running' | 'completed' | 'incomplete' | 'failed' | 'timed_out';
|
|
196
207
|
mode: 'docker' | 'local';
|
|
197
208
|
image: string | null;
|
|
209
|
+
// null on a link: the units folder is the caller's, not keyed on a compiler.
|
|
210
|
+
compiler: string | null;
|
|
198
211
|
exit_code: number | null;
|
|
199
212
|
elapsed_ms: number;
|
|
200
213
|
timed_out: boolean;
|
|
214
|
+
// Engine loss markers found in the log; see Status.
|
|
215
|
+
loss_markers: Array<{ kind: string; count: number; sample: string }>;
|
|
201
216
|
sources: string;
|
|
202
217
|
output: {
|
|
203
218
|
folder: string;
|
|
@@ -239,6 +254,6 @@ This is the CLI tier, so it needs no Kubernetes, deployed app, or Log10x account
|
|
|
239
254
|
}
|
|
240
255
|
```
|
|
241
256
|
|
|
242
|
-
When the link overruns `max_wait_ms`, the handoff payload is smaller: `{ job_id, job_status: 'running', mode, image, library_file, runtime_name, sources, started_at, timeout_ms, log_file, units_path, unit_count }`. Poll [`log10x_compile_status`](compile-status.md) with `job_id` for the completed shape above.
|
|
257
|
+
When the link overruns `max_wait_ms`, the handoff payload is smaller: `{ job_id, job_status: 'running', joined_running_job, mode, image, compiler, library_file, runtime_name, sources, started_at, timeout_ms, log_file, units_path, unit_count }`. Poll [`log10x_compile_status`](compile-status.md) with `job_id` for the completed shape above.
|
|
243
258
|
|
|
244
259
|
Envelope-level fields the agent should also read: `status` (top-level mirror of `data.status`), `summary.headline` (1-line answer), `data.human_summary` (the quotable plain-English summary), `actions[]` (next-call chain hints as `{tool, args, reason}`), `truncated`, and `schema_epoch`.
|
|
@@ -6,7 +6,7 @@ icon: material/clipboard-text-clock
|
|
|
6
6
|
|
|
7
7
|
Check on a compile or link job by its `job_id`, the id returned by [`log10x_compile`](compile.md) or [`log10x_compile_link`](compile-link.md).
|
|
8
8
|
|
|
9
|
-
The response reports where the job stands (running, completed, failed, or timed out) and what it has produced:
|
|
9
|
+
The response reports where the job stands (running, completed, incomplete, failed, or timed out) and what it has produced:
|
|
10
10
|
|
|
11
11
|
- the symbol units so far
|
|
12
12
|
- the linked `.10x.tar` file and its size
|
|
@@ -21,6 +21,20 @@ The point of the tool is to keep the [Compiler app](../../../compiler/) legible
|
|
|
21
21
|
|
|
22
22
|
Both fields appear only when the `compiler-10x` image includes the engine diagnostics change. On an older image the response falls back to unit counts and the log tail, and still succeeds.
|
|
23
23
|
|
|
24
|
+
The engine stops a scan on its own caps and still exits 0, so the captured log is checked for its loss markers and a run that carries one is reported as `job_status: "incomplete"` with `data.status: "partial"`. The markers, each with a count and the first matching line, are in `loss_markers`:
|
|
25
|
+
|
|
26
|
+
| `kind` | Engine log line | Meaning |
|
|
27
|
+
|---|---|---|
|
|
28
|
+
| `scan_operation_timeout` | `scan operation timeout: <ms>` | The whole-scan cap (`scanOperationTimeout`) was hit; the scan stopped with files unscanned. |
|
|
29
|
+
| `unit_timeout` | `symbol scan timed out, file dropped: <file>` | The per-file cap (`scanUnitTimeout`, default 30s) dropped a file; it has no unit. |
|
|
30
|
+
| `traverse_aborted` | `traverse aborted for <operation>` / `traverser aborted: <file>` | The tree walk stopped before visiting every file. |
|
|
31
|
+
| `process_output_not_drained` | `process output not fully read: <command>` | A scan subprocess ended with output unread; its units may be missing. |
|
|
32
|
+
| `antlr_parse_timeout` | `timeout exceeded: <ms>ms` | An ANTLR parse gave up on a file; it failed to scan. |
|
|
33
|
+
|
|
34
|
+
An `incomplete` run keeps its units and library, so a re-run with a larger `timeout_ms` (which raises the whole-scan cap) or narrower sources reuses every file already scanned.
|
|
35
|
+
|
|
36
|
+
A zero-byte unit (`empty_unit_count`) is a unit file the engine created and then wrote nothing into: every symbol in the file was filtered out, the file was skipped as a duplicate of one already scanned, or its scan was dropped.
|
|
37
|
+
|
|
24
38
|
Polling is safe to repeat. The first poll after the job finishes records the exit code and releases the container. Later polls keep working, and the compiled library is never removed.
|
|
25
39
|
|
|
26
40
|
## :material-code-braces: Example
|
|
@@ -41,6 +55,7 @@ Polling is safe to repeat. The first poll after the job finishes records the exi
|
|
|
41
55
|
|
|
42
56
|
- *"is compile-7c2f done yet?"*
|
|
43
57
|
- *"which files failed to scan on that job?"*
|
|
58
|
+
- *"did the engine stop early on that job?"*
|
|
44
59
|
- *"show me the last 200 log lines for the failed link"*
|
|
45
60
|
|
|
46
61
|
## :material-check-decagram-outline: Prerequisites
|
|
@@ -79,21 +94,20 @@ The per-language scan failures and the symbol-type counts need a `compiler-10x`
|
|
|
79
94
|
"minimum": 0,
|
|
80
95
|
"maximum": 400,
|
|
81
96
|
"default": 40,
|
|
82
|
-
"description": "How many trailing engine-log lines to include in data.payload.log_tail (credential-redacted). 0 to omit. Raise it when diagnosing a failed run."
|
|
97
|
+
"description": "How many trailing engine-log lines to include in data.payload.log_tail (credential-redacted). 0 to omit. Raise it when diagnosing a failed or incomplete run."
|
|
83
98
|
},
|
|
84
99
|
"view": {
|
|
85
100
|
"type": "string",
|
|
86
|
-
"
|
|
87
|
-
"summary"
|
|
88
|
-
],
|
|
101
|
+
"const": "summary",
|
|
89
102
|
"default": "summary",
|
|
90
|
-
"description": "summary returns the typed envelope (data.payload.job_status, .diagnostics, .output, .log_tail)."
|
|
103
|
+
"description": "summary returns the typed envelope (data.payload.job_status, .loss_markers, .diagnostics, .output, .log_tail)."
|
|
91
104
|
}
|
|
92
105
|
},
|
|
93
106
|
"required": [
|
|
94
107
|
"job_id"
|
|
95
108
|
],
|
|
96
|
-
"additionalProperties": false
|
|
109
|
+
"additionalProperties": false,
|
|
110
|
+
"$schema": "http://json-schema.org/draft-07/schema#"
|
|
97
111
|
}
|
|
98
112
|
```
|
|
99
113
|
|
|
@@ -135,10 +149,12 @@ The per-language scan failures and the symbol-type counts need a `compiler-10x`
|
|
|
135
149
|
"job_id": "compile-7c2f",
|
|
136
150
|
"job_status": "completed",
|
|
137
151
|
"mode": "docker",
|
|
138
|
-
"image": "log10x/compiler-10x",
|
|
152
|
+
"image": "log10x/compiler-10x:1.1.125@sha256:ee875d49b9a25ef98b6b9edfe69a4b60888fd68fd370e8a0020f1b4777ecf12b",
|
|
153
|
+
"compiler": "docker:log10x/compiler-10x:1.1.125@sha256:ee875d49b9a25ef98b6b9edfe69a4b60888fd68fd370e8a0020f1b4777ecf12b",
|
|
139
154
|
"exit_code": 0,
|
|
140
155
|
"elapsed_ms": 184220,
|
|
141
156
|
"timed_out": false,
|
|
157
|
+
"loss_markers": [],
|
|
142
158
|
"sources": "local /src/payments + github acme/payments-api",
|
|
143
159
|
"output": {
|
|
144
160
|
"folder": "/tmp/log10x-mcp-compile/out/payments",
|
|
@@ -202,7 +218,7 @@ The per-language scan failures and the symbol-type counts need a `compiler-10x`
|
|
|
202
218
|
"INFO Linker - merged 4812 units into payments.10x.tar (38.6 MB)"
|
|
203
219
|
]
|
|
204
220
|
},
|
|
205
|
-
"human_summary": "Compile job compile-7c2f completed via docker in 3m04s: 4812 symbol units, linked to /tmp/log10x-mcp-compile/out/payments.10x.tar (38.6 MB). 31 units
|
|
221
|
+
"human_summary": "Compile job compile-7c2f completed via docker in 3m04s: 4812 symbol units, linked to /tmp/log10x-mcp-compile/out/payments.10x.tar (38.6 MB). 31 units are zero bytes: the engine created the unit file and wrote nothing into it, because every symbol in the file was filtered out (the default symbolTypes keep package/class/enum/log/text/exec), the file was skipped as a duplicate of one already scanned, or its scan was dropped. 19 files failed to scan (top: java 11, go 6, python 2). Example: vendor/legacy/Parser.java, unsupported syntax near token 'sealed'. Linked 4812 unit files (37 excluded by folder/name filters); symbols by type: class 18204, log 9661, exec 4113, enum 1290."
|
|
206
222
|
},
|
|
207
223
|
"actions": [
|
|
208
224
|
{
|
|
@@ -218,6 +234,29 @@ The per-language scan failures and the symbol-type counts need a `compiler-10x`
|
|
|
218
234
|
|
|
219
235
|
On an older `compiler-10x` image, `diagnostics.results_available` is `false`, `scan_health` and `link_report` are `null`, and the response carries the unit counts plus `log_tail` alone.
|
|
220
236
|
|
|
237
|
+
A run the engine stopped early comes back as `incomplete`:
|
|
238
|
+
|
|
239
|
+
> _Compile job `compile-7c2f` exited cleanly but stopped early (scan operation timeout x1, unit timeout x2): 3104 units written. The library is missing symbols; see data.payload.loss_markers._
|
|
240
|
+
|
|
241
|
+
with `data.status: "partial"`, `job_status: "incomplete"`, and
|
|
242
|
+
|
|
243
|
+
```json
|
|
244
|
+
"loss_markers": [
|
|
245
|
+
{
|
|
246
|
+
"kind": "scan_operation_timeout",
|
|
247
|
+
"count": 1,
|
|
248
|
+
"sample": "ERROR FileSymbolScanOperation scan operation timeout: 600000 for com.log10x.eng.scanner.operations.FileSymbolScanOperation: {...} after: 10m"
|
|
249
|
+
},
|
|
250
|
+
{
|
|
251
|
+
"kind": "unit_timeout",
|
|
252
|
+
"count": 2,
|
|
253
|
+
"sample": "WARN symbol scan timed out, file dropped: vendor/legacy/Generated.java"
|
|
254
|
+
}
|
|
255
|
+
]
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
The `actions[]` entry then points at a re-run with `timeout_ms` doubled.
|
|
259
|
+
|
|
221
260
|
??? tenx-output-schema "Output schema"
|
|
222
261
|
|
|
223
262
|
The `payload` block inside the [chassis envelope](../index.md):
|
|
@@ -225,12 +264,21 @@ The per-language scan failures and the symbol-type counts need a `compiler-10x`
|
|
|
225
264
|
```typescript
|
|
226
265
|
interface CompileStatusPayload {
|
|
227
266
|
job_id: string;
|
|
228
|
-
|
|
267
|
+
// incomplete: a clean exit whose log carries an engine loss marker.
|
|
268
|
+
job_status: 'running' | 'completed' | 'incomplete' | 'failed' | 'timed_out';
|
|
229
269
|
mode: 'docker' | 'local';
|
|
230
270
|
image: string | null;
|
|
271
|
+
// The identity the output folder is keyed on: `docker:<image ref>` or `local:<engine version>`.
|
|
272
|
+
compiler: string | null;
|
|
231
273
|
exit_code: number | null;
|
|
232
274
|
elapsed_ms: number;
|
|
233
275
|
timed_out: boolean;
|
|
276
|
+
// Engine loss markers found in the log; non-empty on an `incomplete` run.
|
|
277
|
+
loss_markers: Array<{
|
|
278
|
+
kind: 'scan_operation_timeout' | 'unit_timeout' | 'traverse_aborted' | 'process_output_not_drained' | 'antlr_parse_timeout';
|
|
279
|
+
count: number;
|
|
280
|
+
sample: string;
|
|
281
|
+
}>;
|
|
234
282
|
sources: string;
|
|
235
283
|
output: {
|
|
236
284
|
folder: string;
|
|
@@ -271,4 +319,4 @@ The per-language scan failures and the symbol-type counts need a `compiler-10x`
|
|
|
271
319
|
}
|
|
272
320
|
```
|
|
273
321
|
|
|
274
|
-
Envelope-level fields the agent should also read: `summary.headline` (1-line answer), `status` (`success` / `partial` / `no_signal` / `error`), `data.human_summary` (the quotable plain-English summary), and `actions[]` (next-call chain hints as `{tool, args, reason}`; a completed compile chains to `log10x_validate` to smoke-test the library).
|
|
322
|
+
Envelope-level fields the agent should also read: `summary.headline` (1-line answer), `status` (`success` / `partial` / `no_signal` / `error`; `partial` on an `incomplete` run), `data.human_summary` (the quotable plain-English summary), and `actions[]` (next-call chain hints as `{tool, args, reason}`; a completed compile chains to `log10x_validate` to smoke-test the library, an incomplete one to a re-run with a larger `timeout_ms`).
|
|
@@ -6,9 +6,9 @@ icon: material/package-variant-closed
|
|
|
6
6
|
|
|
7
7
|
Compile source code and binaries into a [symbol library](../../../../compile/link/#symbol-library): a per-file `.10x.json` unit for each source file, plus a single linked `.10x.tar` that the 10x runtime later uses to assign hidden classes (TenXTemplates) to log events.
|
|
8
8
|
|
|
9
|
-
The call is bounded-synchronous. It waits inline up to `max_wait_ms` (default 45s), so a small compile, and
|
|
9
|
+
The call is bounded-synchronous. It waits inline up to `max_wait_ms` (default 45s), so a small compile, and a re-run over unchanged local sources, finishes inside the wait and returns the finished library plus the full scan and link diagnostics in one call.
|
|
10
10
|
|
|
11
|
-
A long first compile of a large tree overruns the wait and returns a running `job_id` to poll with [Status](compile-status.md). The run is detached and writes to a pinned output folder
|
|
11
|
+
A long first compile of a large tree overruns the wait and returns a running `job_id` to poll with [Status](compile-status.md). The run is detached and writes to a pinned output folder. Calling Compile again with the same arguments while it runs joins that job (same `job_id`) rather than starting a second engine on the same folder.
|
|
12
12
|
|
|
13
13
|
Set `max_wait_ms: 0` to return the `job_id` immediately for fire-and-forget.
|
|
14
14
|
|
|
@@ -22,7 +22,23 @@ Sources combine freely in one call:
|
|
|
22
22
|
|
|
23
23
|
Docker pulls are daemonless. GitHub and Artifactory pulls each need a token, even for public GitHub repos.
|
|
24
24
|
|
|
25
|
-
The [Compiler app](../../../compiler/) runs the Compiler flavor, docker-first via the `log10x/compiler-10x` image or a local Compiler-flavor `tenx`. It runs without Kubernetes, a deployed app, or a Log10x account.
|
|
25
|
+
The [Compiler app](../../../compiler/) runs the Compiler flavor, docker-first via the `log10x/compiler-10x:1.1.125` image (pinned by digest) or a local Compiler-flavor `tenx`. It runs without Kubernetes, a deployed app, or a Log10x account.
|
|
26
|
+
|
|
27
|
+
What gets scanned, per the shipped scanner config:
|
|
28
|
+
|
|
29
|
+
- parsed with a grammar or AST: Java, Scala, Python, Go, C/C++, C#, JavaScript, TypeScript (`.ts`; `.tsx` goes through the text scanner), Rust, and JVM `.class` bytecode
|
|
30
|
+
- quoted-string extraction only, through the pattern scanner: Kotlin, Ruby, PHP, Lua, Groovy, Swift, bash
|
|
31
|
+
- archives expanded and their entries scanned: `.jar`, `.war`, `.ear`, `.zip`, `.gz`, `.tar` and compressed tarballs
|
|
32
|
+
- config and text through the text scanner: `.json`, `.yaml`, `.xml`, `.properties`, `.csv`, `.txt` and similar
|
|
33
|
+
- native binaries through `strings`: `.so`, `.dylib`, and `.dll` on Windows hosts
|
|
34
|
+
|
|
35
|
+
## :material-refresh: Re-runs
|
|
36
|
+
|
|
37
|
+
The output folder is keyed on the sources, the library name, and the compiler (the image ref, or the local engine version), so units are reused only by the compiler that wrote them.
|
|
38
|
+
|
|
39
|
+
What a re-run skips: the engine skips every local file whose checksum already has a unit in the folder. Pulled sources cost a remote round-trip each run (GitHub branch head, image manifest digest, Artifactory listing) and are downloaded again only when the remote changed; Helm charts are rendered again every run. Only the scan of unchanged files is skipped.
|
|
40
|
+
|
|
41
|
+
The engine's own whole-scan cap (`scanOperationTimeout`, bundled default 10 minutes) is raised to 90% of `timeout_ms` on the command line, so the engine cap follows `timeout_ms`. That needs compiler 1.1.89 or newer; the pinned default is 1.1.125, and an older local engine keeps its 10 minute cap. A run the engine stopped early is reported as `incomplete` by [Status](compile-status.md).
|
|
26
42
|
|
|
27
43
|
## :material-code-braces: Example
|
|
28
44
|
|
|
@@ -36,11 +52,11 @@ The [Compiler app](../../../compiler/) runs the Compiler flavor, docker-first vi
|
|
|
36
52
|
|
|
37
53
|
!!! tenx-ask "You"
|
|
38
54
|
|
|
39
|
-
|
|
55
|
+
run it again, nothing changed
|
|
40
56
|
|
|
41
57
|
!!! tenx-answer "Log10x"
|
|
42
58
|
|
|
43
|
-
Same
|
|
59
|
+
Same sources and compiler, so the pinned output is reused: every local file skipped by checksum, the grafana image digest unchanged. Done in 4s.
|
|
44
60
|
|
|
45
61
|
## :material-chat-question-outline: More to ask
|
|
46
62
|
|
|
@@ -54,7 +70,7 @@ The Compiler flavor needs one of two things: Docker running so the `log10x/compi
|
|
|
54
70
|
|
|
55
71
|
GitHub pull (`github_repos`) needs a token even for public repos, passed as `github_token` or set as `GH_TOKEN` in the MCP server environment. Artifactory pull needs `artifactory_token` or `ARTIFACTORY_TOKEN`.
|
|
56
72
|
|
|
57
|
-
`.jar`
|
|
73
|
+
Archives are scanned as-is: `.jar`, `.war`, `.ear`, `.zip` and tarballs are expanded by the archive scanner and their entries scanned.
|
|
58
74
|
|
|
59
75
|
## :material-code-json: Schema and samples
|
|
60
76
|
|
|
@@ -81,11 +97,13 @@ GitHub pull (`github_repos`) needs a token even for public repos, passed as `git
|
|
|
81
97
|
"properties": {
|
|
82
98
|
"source_path": {
|
|
83
99
|
"type": "string",
|
|
84
|
-
"description": "Absolute path to a local folder of source code / binaries to scan
|
|
100
|
+
"description": "Absolute path to a local folder of source code / binaries to scan, traversed recursively. Parsed with a grammar or AST: Java, Scala, Python, Go, C/C++, C#, JavaScript, TypeScript (.ts; .tsx goes through the text scanner), Rust, and JVM .class bytecode. Other languages in the pattern scanner's file list (Kotlin, Ruby, PHP, Lua, Groovy, Swift, bash) get quoted-string extraction only. Archives (.jar, .war, .ear, .zip, .gz, .tar and compressed tarballs) are expanded and their entries scanned; .json/.yaml/.xml/.properties/.csv/.txt and similar config/text files go through the text scanner; .so/.dylib (and .dll on Windows hosts) through `strings`. Optional when any other source is given; at least one source (source_path, github_repos, docker_images, helm_charts, or artifactory_instance + artifactory_repo) is required."
|
|
85
101
|
},
|
|
86
102
|
"github_repos": {
|
|
87
103
|
"type": "array",
|
|
88
|
-
"items": {
|
|
104
|
+
"items": {
|
|
105
|
+
"type": "string"
|
|
106
|
+
},
|
|
89
107
|
"description": "GitHub repositories to pull (via the GitHub REST API) and scan, each as owner/repo (e.g. [\"apache/commons-cli\"]). REQUIRES a GitHub token, even for public repos: pass github_token, or have GH_TOKEN / GITHUB_TOKEN set in the MCP server environment. Combines freely with source_path."
|
|
90
108
|
},
|
|
91
109
|
"github_branch": {
|
|
@@ -94,7 +112,9 @@ GitHub pull (`github_repos`) needs a token even for public repos, passed as `git
|
|
|
94
112
|
},
|
|
95
113
|
"github_folders": {
|
|
96
114
|
"type": "array",
|
|
97
|
-
"items": {
|
|
115
|
+
"items": {
|
|
116
|
+
"type": "string"
|
|
117
|
+
},
|
|
98
118
|
"description": "Folders within each GitHub repo to pull (e.g. [\"src/main/java\"]). Omit to pull entire repos. Narrowing this speeds up both the pull and the scan."
|
|
99
119
|
},
|
|
100
120
|
"github_token": {
|
|
@@ -103,7 +123,9 @@ GitHub pull (`github_repos`) needs a token even for public repos, passed as `git
|
|
|
103
123
|
},
|
|
104
124
|
"docker_images": {
|
|
105
125
|
"type": "array",
|
|
106
|
-
"items": {
|
|
126
|
+
"items": {
|
|
127
|
+
"type": "string"
|
|
128
|
+
},
|
|
107
129
|
"description": "Docker/OCI images to pull and scan for symbols, as image refs, fully-qualified recommended (e.g. [\"docker.io/grafana/grafana:11.1.0\"]; a port-bearing host like \"harbor.corp:8443/team/app:2.1\" is fine, and a bare \"alpine\" resolves against the engine default registry). The pull is daemonless (podman inside the compiler-10x image, no host docker socket), and the tool automatically grants the compile container `--cap-add SYS_ADMIN` (needed by podman; only when this arg is used). Public images need no credentials. docker_username + docker_token `docker login` to the DEFAULT registry (Docker Hub), so they cover private Docker Hub repos; images on a different private registry must be pre-authenticated on the host/engine. In mode=local the host needs a docker/podman CLI with a working engine, and pulled images are left in its store (remove:false). Combines freely with source_path and github_repos."
|
|
108
130
|
},
|
|
109
131
|
"docker_username": {
|
|
@@ -116,12 +138,16 @@ GitHub pull (`github_repos`) needs a token even for public repos, passed as `git
|
|
|
116
138
|
},
|
|
117
139
|
"helm_charts": {
|
|
118
140
|
"type": "array",
|
|
119
|
-
"items": {
|
|
141
|
+
"items": {
|
|
142
|
+
"type": "string"
|
|
143
|
+
},
|
|
120
144
|
"description": "Helm charts to render and scan, as chart refs. A meta-source: the compiler runs `helm template` / `helm show chart` to extract the docker images and GitHub source repos the chart references, then (by default) pulls those too. OCI refs (e.g. \"oci://ghcr.io/nginxinc/charts/nginx-ingress\") and full URLs resolve standalone; a bare \"repo/chart\" (e.g. \"ingress-nginx/ingress-nginx\") needs a matching helm_repos entry. Combines freely with the other sources. In mode=local the host needs the helm CLI and must have the repos already `helm repo add`-ed."
|
|
121
145
|
},
|
|
122
146
|
"helm_repos": {
|
|
123
147
|
"type": "array",
|
|
124
|
-
"items": {
|
|
148
|
+
"items": {
|
|
149
|
+
"type": "string"
|
|
150
|
+
},
|
|
125
151
|
"description": "Helm chart repositories to register before resolving helm_charts, each as \"name=url\" (e.g. [\"ingress-nginx=https://kubernetes.github.io/ingress-nginx\"]). Required for bare \"repo/chart\" names; unnecessary for OCI/URL refs. Added via `helm repo add` in pre-step containers (docker mode only; in mode=local the host helm config is used as-is). url must be an http(s):// chart-repo index URL; for an OCI registry, put the oci://… ref directly in helm_charts (`helm repo add` does not support oci://)."
|
|
126
152
|
},
|
|
127
153
|
"helm_pull_images": {
|
|
@@ -144,12 +170,16 @@ GitHub pull (`github_repos`) needs a token even for public repos, passed as `git
|
|
|
144
170
|
},
|
|
145
171
|
"artifactory_files": {
|
|
146
172
|
"type": "array",
|
|
147
|
-
"items": {
|
|
173
|
+
"items": {
|
|
174
|
+
"type": "string"
|
|
175
|
+
},
|
|
148
176
|
"description": "Specific files within artifactory_repo to pull, each a repo-relative path (e.g. [\"dist/app-1.0.0.tar.gz\"]). Combine with artifactory_folders; at least one of the two is required when pulling from Artifactory."
|
|
149
177
|
},
|
|
150
178
|
"artifactory_folders": {
|
|
151
179
|
"type": "array",
|
|
152
|
-
"items": {
|
|
180
|
+
"items": {
|
|
181
|
+
"type": "string"
|
|
182
|
+
},
|
|
153
183
|
"description": "Folder paths within artifactory_repo to pull (e.g. [\"com/acme/app\"]). Traversed recursively unless artifactory_recursive is false. At least one of artifactory_files / artifactory_folders is required when pulling from Artifactory."
|
|
154
184
|
},
|
|
155
185
|
"artifactory_recursive": {
|
|
@@ -172,26 +202,31 @@ GitHub pull (`github_repos`) needs a token even for public repos, passed as `git
|
|
|
172
202
|
},
|
|
173
203
|
"mode": {
|
|
174
204
|
"type": "string",
|
|
175
|
-
"enum": [
|
|
205
|
+
"enum": [
|
|
206
|
+
"auto",
|
|
207
|
+
"docker",
|
|
208
|
+
"local"
|
|
209
|
+
],
|
|
176
210
|
"default": "auto",
|
|
177
|
-
"description": "Execution backend. `auto` (default) prefers Docker (
|
|
211
|
+
"description": "Execution backend. `auto` (default) prefers Docker (the compiler image, guaranteed compiler flavor) and falls back to a local compiler-flavor tenx; the backend is chosen once, before the output folder is keyed. `docker` forces the image (LOG10X_COMPILER_IMAGE or LOG10X_TENX_IMAGE, default log10x/compiler-10x:1.1.125 pinned by digest, so every run goes through the same compiler). `local` forces the binary (LOG10X_TENX_PATH or `tenx` on PATH) and refuses it unless its version banner reports the compiler flavor (`compiler`, or `cloud` on an engine built before the flavor rename). Of the three shipped flavors only `compiler` can compile: `runtime` (native binary) and `runtime-jvm` (JVM-packaged runtime, the only runtime available on Windows) are both refused, since neither carries the `generate` pipeline unit. With a local install, local-folder compilation and GitHub pull (REST API + token) work out of the box; docker_images pull additionally needs a container engine (podman or docker) on the host. The docker `compiler-10x` image bundles all of those (podman included, daemonless), which is why Docker is the default."
|
|
178
212
|
},
|
|
179
213
|
"timeout_ms": {
|
|
180
214
|
"type": "integer",
|
|
181
215
|
"minimum": 10000,
|
|
182
216
|
"maximum": 3600000,
|
|
183
217
|
"default": 1800000,
|
|
184
|
-
"description": "Hard cap on compile wall time in milliseconds. Default 1,800,000 (30 min). The
|
|
218
|
+
"description": "Hard cap on compile wall time in milliseconds. Default 1,800,000 (30 min). The engine's own whole-scan cap (scanOperationTimeout, bundled default 10 min) is raised to 90% of this value on the command line, so the engine cap follows this value (needs compiler 1.1.89+; the pinned default is 1.1.125; an older local engine keeps its 10 min cap). A re-run over unchanged local sources skips every file whose checksum already has a unit and finishes in seconds. Pulled sources (GitHub, images, Artifactory) are re-checked remotely each run and downloaded again only when the remote changed; Helm charts are rendered again every run."
|
|
185
219
|
},
|
|
186
220
|
"max_wait_ms": {
|
|
187
221
|
"type": "integer",
|
|
188
222
|
"minimum": 0,
|
|
189
223
|
"maximum": 300000,
|
|
190
224
|
"default": 45000,
|
|
191
|
-
"description": "How long to wait inline (ms) for the compile to finish before handing back a job_id to poll. Default 45,000 (45s): small compiles and re-runs (
|
|
225
|
+
"description": "How long to wait inline (ms) for the compile to finish before handing back a job_id to poll. Default 45,000 (45s): small compiles, and re-runs over unchanged local sources (the engine skips files whose checksum already has a unit), finish inside this and return the library + diagnostics in ONE call. A long first compile returns a running job_id to poll with log10x_compile_status; calling this tool again with the same arguments joins the running job instead of starting a second one. Re-runs that pull remote sources are not instant: GitHub, images and Artifactory are re-checked remotely each run and Helm charts re-rendered. 0 = fire-and-forget (return the job_id immediately)."
|
|
192
226
|
}
|
|
193
227
|
},
|
|
194
|
-
"additionalProperties": false
|
|
228
|
+
"additionalProperties": false,
|
|
229
|
+
"$schema": "http://json-schema.org/draft-07/schema#"
|
|
195
230
|
}
|
|
196
231
|
```
|
|
197
232
|
|
|
@@ -229,10 +264,12 @@ GitHub pull (`github_repos`) needs a token even for public repos, passed as `git
|
|
|
229
264
|
"job_id": "9f2c1a3e-7b4d-4e1a-9f0c-2a1b3c4d5e6f",
|
|
230
265
|
"job_status": "completed",
|
|
231
266
|
"mode": "docker",
|
|
232
|
-
"image": "log10x/compiler-10x:
|
|
267
|
+
"image": "log10x/compiler-10x:1.1.125@sha256:ee875d49b9a25ef98b6b9edfe69a4b60888fd68fd370e8a0020f1b4777ecf12b",
|
|
268
|
+
"compiler": "docker:log10x/compiler-10x:1.1.125@sha256:ee875d49b9a25ef98b6b9edfe69a4b60888fd68fd370e8a0020f1b4777ecf12b",
|
|
233
269
|
"exit_code": 0,
|
|
234
270
|
"elapsed_ms": 38120,
|
|
235
271
|
"timed_out": false,
|
|
272
|
+
"loss_markers": [],
|
|
236
273
|
"sources": "/home/dev/payments-svc + images docker.io/grafana/grafana:11.1.0",
|
|
237
274
|
"output": {
|
|
238
275
|
"folder": "/tmp/log10x-mcp-compile/payments-9f2c1a/symbols",
|
|
@@ -294,7 +331,7 @@ GitHub pull (`github_repos`) needs a token even for public repos, passed as `git
|
|
|
294
331
|
"[link] symbols by type: class 1840, log 612, exec 410, enum 96"
|
|
295
332
|
]
|
|
296
333
|
},
|
|
297
|
-
"human_summary": "Compile job 9f2c1a3e-... completed via docker in 38s: 214 symbol units, linked to /tmp/log10x-mcp-compile/payments-9f2c1a/symbols/payments.10x.tar (4.2 MB). 6 units
|
|
334
|
+
"human_summary": "Compile job 9f2c1a3e-... completed via docker in 38s: 214 symbol units, linked to /tmp/log10x-mcp-compile/payments-9f2c1a/symbols/payments.10x.tar (4.2 MB). 6 units are zero bytes: the engine created the unit file and wrote nothing into it, because every symbol in the file was filtered out (the default symbolTypes keep package/class/enum/log/text/exec), the file was skipped as a duplicate of one already scanned, or its scan was dropped. 3 files failed to scan (top: go 2, python 1). Example: internal/ledger/gen.go, parse error at line 84: unexpected token. Linked 214 unit files (13 excluded by folder/name filters); symbols by type: class 1840, log 612, exec 410, enum 96."
|
|
298
335
|
},
|
|
299
336
|
"actions": [
|
|
300
337
|
{
|
|
@@ -308,21 +345,31 @@ GitHub pull (`github_repos`) needs a token even for public repos, passed as `git
|
|
|
308
345
|
}
|
|
309
346
|
```
|
|
310
347
|
|
|
311
|
-
A long first compile of a large tree overruns `max_wait_ms` and returns `job_status: "running"` with the `job_id` instead. Poll it with [Status](compile-status.md)
|
|
348
|
+
A long first compile of a large tree overruns `max_wait_ms` and returns `job_status: "running"` with the `job_id` instead. Poll it with [Status](compile-status.md). A second call with the same arguments while it runs returns the same handle with `joined_running_job: true`.
|
|
349
|
+
|
|
350
|
+
A run the engine stopped early (whole-scan cap, a dropped file, an aborted tree walk, an abandoned scan subprocess) exits 0 and comes back as `job_status: "incomplete"` with `data.status: "partial"` and the markers in `loss_markers`; see [Status](compile-status.md).
|
|
312
351
|
|
|
313
352
|
??? tenx-output-schema "Output schema"
|
|
314
353
|
|
|
315
|
-
The `data.payload` block inside the [StructuredOutput envelope](../index.md#json-by-default-output). On inline completion the shape matches [Status](compile-status.md); on overrun (or `max_wait_ms: 0`) the running handle carries `job_status: "running"` plus `job_id`, `mode`, `image`, `library_file`, `runtime_name`, `sources`, `started_at`, `timeout_ms`, `log_file`, and `output_folder`.
|
|
354
|
+
The `data.payload` block inside the [StructuredOutput envelope](../index.md#json-by-default-output). On inline completion the shape matches [Status](compile-status.md); on overrun (or `max_wait_ms: 0`) the running handle carries `job_status: "running"` plus `job_id`, `joined_running_job`, `mode`, `image`, `compiler`, `library_file`, `runtime_name`, `sources`, `started_at`, `timeout_ms`, `log_file`, and `output_folder`.
|
|
316
355
|
|
|
317
356
|
```typescript
|
|
318
357
|
interface CompilePayload {
|
|
319
358
|
job_id: string;
|
|
320
|
-
job_status: 'running' | 'completed' | 'failed' | 'timed_out';
|
|
359
|
+
job_status: 'running' | 'completed' | 'incomplete' | 'failed' | 'timed_out';
|
|
321
360
|
mode: 'docker' | 'local';
|
|
322
361
|
image: string | null;
|
|
362
|
+
// The identity the output folder is keyed on: `docker:<image ref>` or `local:<engine version>`.
|
|
363
|
+
compiler: string | null;
|
|
323
364
|
exit_code: number | null;
|
|
324
365
|
elapsed_ms: number;
|
|
325
366
|
timed_out: boolean;
|
|
367
|
+
// Engine loss markers found in the log; non-empty on an `incomplete` run.
|
|
368
|
+
loss_markers: Array<{
|
|
369
|
+
kind: 'scan_operation_timeout' | 'unit_timeout' | 'traverse_aborted' | 'process_output_not_drained' | 'antlr_parse_timeout';
|
|
370
|
+
count: number;
|
|
371
|
+
sample: string;
|
|
372
|
+
}>;
|
|
326
373
|
sources: string;
|
|
327
374
|
output: {
|
|
328
375
|
folder: string;
|