log10x-mcp 1.30.21 → 1.30.22
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -1
- package/build/index.js +26 -4
- package/build/index.js.map +1 -1
- package/build/lib/advisor/reporter-forwarders.js +34 -3
- package/build/lib/advisor/reporter-forwarders.js.map +1 -1
- package/build/lib/advisor/reporter.js +7 -4
- package/build/lib/advisor/reporter.js.map +1 -1
- package/build/lib/cdk-recipes.d.ts +1 -1
- package/build/lib/cdk-recipes.js +4 -5
- package/build/lib/cdk-recipes.js.map +1 -1
- package/build/lib/fenced.d.ts +2 -0
- package/build/lib/fenced.js +2 -2
- package/build/lib/fenced.js.map +1 -1
- package/build/lib/offload-recipes.d.ts +2 -1
- package/build/lib/offload-recipes.js +7 -6
- package/build/lib/offload-recipes.js.map +1 -1
- package/build/lib/poc-report-renderer.js +17 -11
- package/build/lib/poc-report-renderer.js.map +1 -1
- package/build/lib/retriever-api.d.ts +14 -0
- package/build/lib/retriever-api.js +40 -10
- package/build/lib/retriever-api.js.map +1 -1
- package/build/lib/runtime-image.d.ts +4 -2
- package/build/lib/runtime-image.js +4 -2
- package/build/lib/runtime-image.js.map +1 -1
- package/build/lib/siem/_sampling.js +4 -0
- package/build/lib/siem/_sampling.js.map +1 -1
- package/build/lib/siem/cloudwatch.d.ts +18 -0
- package/build/lib/siem/cloudwatch.js +106 -2
- package/build/lib/siem/cloudwatch.js.map +1 -1
- package/build/lib/siem/datadog.js +1 -1
- package/build/lib/siem/datadog.js.map +1 -1
- package/build/lib/siem/elasticsearch.js +13 -1
- package/build/lib/siem/elasticsearch.js.map +1 -1
- package/build/lib/siem/splunk.d.ts +18 -0
- package/build/lib/siem/splunk.js +71 -7
- package/build/lib/siem/splunk.js.map +1 -1
- package/build/product-kb/docs/apps/receiver/compact/index.md +3 -3
- package/build/product-kb/docs/apps/receiver/compact/splunk.md +81 -67
- package/build/product-kb/docs/apps/retriever/faq.md +2 -2
- package/build/product-kb/docs/faq/apps/retriever.md +2 -2
- package/build/product-kb/docs/faq/general.md +1 -1
- package/build/product-kb/docs/faq/stacks/splunk/comparisons.md +3 -3
- package/build/product-kb/docs/faq/stacks/splunk/compatibility.md +19 -242
- package/build/product-kb/docs/faq/stacks/splunk/index.md +1 -1
- package/build/product-kb/docs/faq/stacks/splunk/kv-store.md +41 -159
- package/build/product-kb/docs/faq/stacks/splunk/optimization.md +4 -4
- package/build/product-kb/docs/run/input/forwarder/fluentbit/index.md +1 -1
- package/build/product-kb/docs/run/input/forwarder/fluentd/index.md +1 -1
- package/build/product-kb/docs/run/input/forwarder/otel-collector/index.md +1 -1
- package/build/product-kb/docs/run/input/forwarder/vector/index.md +1 -1
- package/build/tools/advise-install.js +8 -3
- package/build/tools/advise-install.js.map +1 -1
- package/build/tools/doctor.js +3 -2
- package/build/tools/doctor.js.map +1 -1
- package/build/tools/emit-sample-plan.js +7 -1
- package/build/tools/emit-sample-plan.js.map +1 -1
- package/build/tools/poc-from-local.js +7 -6
- package/build/tools/poc-from-local.js.map +1 -1
- package/build/tools/poc-from-siem.js +71 -4
- package/build/tools/poc-from-siem.js.map +1 -1
- package/build/tools/retriever-query.js +1 -1
- package/build/tools/retriever-query.js.map +1 -1
- package/package.json +1 -1
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "
|
|
2
|
+
title: "Log10x App for Splunk"
|
|
3
3
|
description: "Splunk app for transparent expansion of 10x-optimized events"
|
|
4
4
|
source: "https://github.com/log-10x/splunk-app"
|
|
5
5
|
icon: "material/greater-than"
|
|
6
6
|
|
|
7
7
|
---
|
|
8
8
|
|
|
9
|
-
Search and visualize [compact](https://doc.log10x.com/run/transform/#compact) events in Splunk; the original lines expand back exactly at search time. This [open-source app](https://github.com/log-10x/splunk-app){target="_blank"} expands compact events at search time: classic dashboards keep their panels unchanged, the search bar wraps a query in one command, and
|
|
9
|
+
Search and visualize [compact](https://doc.log10x.com/run/transform/#compact) events in Splunk; the original lines expand back exactly at search time. This [open-source app](https://github.com/log-10x/splunk-app){target="_blank"} expands compact events at search time: classic dashboards keep their panels unchanged, the search bar wraps a query in one command, and an alert created in the app's Compile Alert view compiles once so the scheduler runs it natively.
|
|
10
10
|
|
|
11
11
|
Use the [Reporter](https://doc.log10x.com/apps/reporter/) (DaemonSet, pre-ingestion) or the [MCP server](https://github.com/log-10x/log10x-mcp){target="_blank"}'s SIEM-sample tool (agentless, via Splunk REST API) to identify optimization opportunities in your existing Splunk data.
|
|
12
12
|
|
|
@@ -18,22 +18,22 @@ Use the [Reporter](https://doc.log10x.com/apps/reporter/) (DaemonSet, pre-ingest
|
|
|
18
18
|
|
|
19
19
|
## :material-chart-line: Analytics Dashboard
|
|
20
20
|
|
|
21
|
-
The
|
|
21
|
+
The Analytics dashboard shows compact event counts, active templates, compression ratio and storage saved. It finds compact events through the `tenx-events` macro.
|
|
22
22
|
|
|
23
23
|
<figure markdown="span">
|
|
24
|
-
{ loading=lazy }
|
|
25
|
+
<figcaption>The Analytics dashboard</figcaption>
|
|
26
26
|
</figure>
|
|
27
27
|
|
|
28
|
-
|
|
|
28
|
+
| Panel | Description |
|
|
29
29
|
|--------|-------------|
|
|
30
|
-
| **Total Encoded Events** |
|
|
31
|
-
| **Active Templates** |
|
|
32
|
-
| **
|
|
33
|
-
| **Storage Savings** |
|
|
34
|
-
| **Event Volume
|
|
35
|
-
| **Top Templates by Usage** |
|
|
36
|
-
| **
|
|
30
|
+
| **Total Encoded Events** | Compact events indexed |
|
|
31
|
+
| **Active Templates** | Templates in the KV Store |
|
|
32
|
+
| **Compression Ratio** | Original size over compact size, on the latest 5,000 events |
|
|
33
|
+
| **Estimated Storage Savings** | Bytes saved and percentage reduction, on the latest 10,000 events |
|
|
34
|
+
| **Event Volume, Last 7 Days** | Compact events per hour |
|
|
35
|
+
| **Top 10 Templates by Usage** | Template hashes with the most events |
|
|
36
|
+
| **Inflation Success Rate** | Share of the latest 1,000 events that expand |
|
|
37
37
|
|
|
38
38
|
---
|
|
39
39
|
|
|
@@ -41,13 +41,13 @@ The app includes a built-in analytics dashboard providing real-time visibility i
|
|
|
41
41
|
|
|
42
42
|
A [compact event](https://doc.log10x.com/run/transform/#compact) carries a template hash and the event's variable values. The constant words live in the KV Store, so the app puts them back at search time and re-applies the search to the expanded lines.
|
|
43
43
|
|
|
44
|
-
Classic dashboards
|
|
44
|
+
Classic dashboards keep their panels unchanged: the browser sends each panel's search to the app, which rewrites it on the server. Splunk's search page loads no app JavaScript, so a query typed there is wrapped in the `tenxsearch` command, which runs the same rewrite. An alert created in the **Compile Alert** view compiles once and then runs as an ordinary saved search.
|
|
45
45
|
|
|
46
46
|
Compact mode encodes the entire event for Splunk, including the Kubernetes metadata, into a single compact line. Splunk bills against uncompressed ingest volume, so compacting everything maximizes license-cost reduction. The Receiver uses a different mode for Elasticsearch from the same forwarder pipeline; there, only the log message is encoded so plugin queries on metadata still work. The [splunk-app README](https://github.com/log-10x/splunk-app#receiver-side-configuration){target="_blank"} covers the Receiver setting.
|
|
47
47
|
|
|
48
48
|
### :material-arrow-right-bold-outline: Ingestion Flow
|
|
49
49
|
|
|
50
|
-
|
|
50
|
+
The Receiver [compacts](https://doc.log10x.com/run/transform/#compact) events at the edge, and Splunk ingests them into Splunk with reduced payload size:
|
|
51
51
|
|
|
52
52
|
<div style="text-align: center;">
|
|
53
53
|
|
|
@@ -111,7 +111,7 @@ graph LR
|
|
|
111
111
|
|
|
112
112
|
📊 **Results**: fully expanded events, with the original search re-applied so only true matches remain
|
|
113
113
|
|
|
114
|
-
**Scheduled alerts** run server-side, where the browser hook never fires.
|
|
114
|
+
**Scheduled alerts** run server-side, where the browser hook never fires. An alert created in the **Compile Alert** view, or through the `/tenx-alert` endpoint, is compiled once at save time into a native saved search (a template-hash prefilter plus the [`tenx-inflate`](https://github.com/log-10x/splunk-app) macro), so the scheduler runs an ordinary search with no per-run proxy. The **Compile Alert** view and the `/tenx-alert` endpoint handle this, and a recompile pass converts `| tenxsearch` alerts to native form and refreshes their prefilters as new templates appear. See [SAVE_TIME_ALERTS.md](https://github.com/log-10x/splunk-app/blob/main/SAVE_TIME_ALERTS.md){target="_blank"}.
|
|
115
115
|
|
|
116
116
|
---
|
|
117
117
|
|
|
@@ -133,7 +133,7 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
133
133
|
|
|
134
134
|
| Requirement | Description |
|
|
135
135
|
|-------------|-------------|
|
|
136
|
-
| Splunk Enterprise |
|
|
136
|
+
| Splunk Enterprise | 9.4 through 10.4 (bundled Python 3.9 through 3.13) |
|
|
137
137
|
| Admin Access | Required for app installation and KV Store setup |
|
|
138
138
|
|
|
139
139
|
??? tenx-checklist "Verify Installation"
|
|
@@ -141,12 +141,12 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
141
141
|
After restart, confirm the app appears in Splunk:
|
|
142
142
|
|
|
143
143
|
1. Navigate to **Apps** → **Manage Apps**
|
|
144
|
-
2. Search for "10x"
|
|
144
|
+
2. Search for "10x"; the list shows **Log10x App**
|
|
145
145
|
3. Click the app name to open the Analytics dashboard
|
|
146
146
|
|
|
147
147
|
??? tenx-step-config "Step 2: Create HEC Tokens"
|
|
148
148
|
|
|
149
|
-
Create two HTTP Event Collector tokens in Splunk
|
|
149
|
+
Create two HTTP Event Collector tokens in Splunk: one for templates, one for encoded events.
|
|
150
150
|
|
|
151
151
|
**Navigate to HEC Settings:**
|
|
152
152
|
|
|
@@ -168,15 +168,35 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
168
168
|
| Setting | Value |
|
|
169
169
|
|---------|-------|
|
|
170
170
|
| Name | `tenx-encoded` |
|
|
171
|
-
| Source type |
|
|
171
|
+
| Source type | `tenx_encoded` |
|
|
172
172
|
| Index | Your target index |
|
|
173
173
|
| Enable indexer acknowledgement | Off |
|
|
174
174
|
|
|
175
|
+
The app recognizes compact events by their sourcetype. To use a sourcetype other than `tenx_encoded`, add the app's extraction to that sourcetype's `props.conf` stanza:
|
|
176
|
+
|
|
177
|
+
```ini title="props.conf"
|
|
178
|
+
[my_sourcetype]
|
|
179
|
+
REPORT-tenx = tenx-hash-vars-extraction
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
!!! tip "Point the dashboards at these events"
|
|
183
|
+
The app's Analytics and Diagnostics dashboards find compact events through the `tenx-events` search macro, which defaults to `index=* sourcetype=tenx_encoded`. Set it to your index and sourcetype under **Settings** → **Advanced search** → **Search macros**, for example `index=my_compact_index sourcetype=tenx_encoded`.
|
|
184
|
+
|
|
175
185
|
!!! warning "Create Index First"
|
|
176
186
|
If `tenx_dml` index doesn't exist, create it via **Settings** → **Indexes** → **New Index** before creating the token.
|
|
177
187
|
|
|
178
188
|
??? tenx-step-link "Step 3: Configure Forwarder"
|
|
179
189
|
|
|
190
|
+
Set three options in the Receiver's configuration. Each keeps expansion exact:
|
|
191
|
+
|
|
192
|
+
| Option | Value | Why |
|
|
193
|
+
|--------|-------|-----|
|
|
194
|
+
| `varMaxRecurIndexes` | `0` | Every variable value travels in the event |
|
|
195
|
+
| `timestampZone` | `UTC` | The app renders timestamps in UTC |
|
|
196
|
+
| `maxPerObject` | `1` | The app stores one timestamp format per template |
|
|
197
|
+
|
|
198
|
+
A template that breaks the first or third rule stays compact and carries `tenx_expand_refused`, naming the reason.
|
|
199
|
+
|
|
180
200
|
Configure your log forwarder to send encoded events and templates to Splunk.
|
|
181
201
|
|
|
182
202
|
=== ":simple-fluentbit: Fluent-bit"
|
|
@@ -203,7 +223,7 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
203
223
|
event_source fluent-bit-tenx
|
|
204
224
|
event_sourcetype tenx_dml_raw_json
|
|
205
225
|
TLS On
|
|
206
|
-
TLS.Verify
|
|
226
|
+
TLS.Verify On
|
|
207
227
|
|
|
208
228
|
# ========================= ENCODED EVENTS OUTPUT ====================
|
|
209
229
|
# Routes encoded log events to your target index
|
|
@@ -215,14 +235,14 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
215
235
|
Port 8088
|
|
216
236
|
Splunk_Token YOUR_ENCODED_EVENTS_HEC_TOKEN
|
|
217
237
|
event_index your_logs_index
|
|
218
|
-
event_sourcetype
|
|
238
|
+
event_sourcetype tenx_encoded
|
|
219
239
|
event_key $log
|
|
220
240
|
TLS On
|
|
221
|
-
TLS.Verify
|
|
241
|
+
TLS.Verify On
|
|
222
242
|
```
|
|
223
243
|
|
|
224
244
|
!!! danger "Critical: event_key $log"
|
|
225
|
-
Without `event_key $log`, Fluent Bit wraps events in JSON: `{"log":"~hash,var1,var2..."}`. The
|
|
245
|
+
Without `event_key $log`, Fluent Bit wraps events in JSON: `{"log":"~hash,var1,var2..."}`. The app expands the bare `~hash,...` line, which `event_key $log` sends.
|
|
226
246
|
|
|
227
247
|
=== ":simple-fluentd: Fluentd"
|
|
228
248
|
|
|
@@ -254,7 +274,7 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
254
274
|
index tenx_dml
|
|
255
275
|
sourcetype tenx_dml_raw_json
|
|
256
276
|
use_ssl true
|
|
257
|
-
ssl_verify
|
|
277
|
+
ssl_verify true
|
|
258
278
|
</match>
|
|
259
279
|
|
|
260
280
|
# ========================= ENCODED EVENTS OUTPUT ====================
|
|
@@ -267,9 +287,9 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
267
287
|
port 8088
|
|
268
288
|
token YOUR_ENCODED_EVENTS_HEC_TOKEN
|
|
269
289
|
index your_logs_index
|
|
270
|
-
sourcetype
|
|
290
|
+
sourcetype tenx_encoded
|
|
271
291
|
use_ssl true
|
|
272
|
-
ssl_verify
|
|
292
|
+
ssl_verify true
|
|
273
293
|
<format>
|
|
274
294
|
@type single_value
|
|
275
295
|
message_key log
|
|
@@ -278,7 +298,7 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
278
298
|
```
|
|
279
299
|
|
|
280
300
|
!!! danger "Critical: format single_value"
|
|
281
|
-
Without the `<format>` block, Fluentd wraps events in JSON: `{"log":"~hash,var1,var2...","tag":"..."}`. The
|
|
301
|
+
Without the `<format>` block, Fluentd wraps events in JSON: `{"log":"~hash,var1,var2...","tag":"..."}`. The app expands the bare `~hash,...` line, which `format single_value` with `message_key log` sends.
|
|
282
302
|
|
|
283
303
|
=== ":simple-opentelemetry: OTel Collector"
|
|
284
304
|
|
|
@@ -293,16 +313,16 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
293
313
|
index: "tenx_dml"
|
|
294
314
|
sourcetype: "tenx_dml_raw_json"
|
|
295
315
|
tls:
|
|
296
|
-
insecure_skip_verify:
|
|
316
|
+
insecure_skip_verify: false
|
|
297
317
|
|
|
298
318
|
# Encoded events exporter
|
|
299
319
|
splunk_hec/encoded:
|
|
300
320
|
endpoint: "https://your-splunk-host.com:8088/services/collector"
|
|
301
321
|
token: "YOUR_ENCODED_EVENTS_HEC_TOKEN"
|
|
302
322
|
index: "your_logs_index"
|
|
303
|
-
sourcetype: "
|
|
323
|
+
sourcetype: "tenx_encoded"
|
|
304
324
|
tls:
|
|
305
|
-
insecure_skip_verify:
|
|
325
|
+
insecure_skip_verify: false
|
|
306
326
|
|
|
307
327
|
service:
|
|
308
328
|
pipelines:
|
|
@@ -328,16 +348,22 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
328
348
|
```spl
|
|
329
349
|
| inputlookup tenx-dml-lookup | stats count
|
|
330
350
|
```
|
|
331
|
-
:material-check-circle-outline:{ .success } Expected: Count > 0 (
|
|
351
|
+
:material-check-circle-outline:{ .success } Expected: Count > 0 (up to 5 minutes for the first templates)
|
|
332
352
|
|
|
333
353
|
**3. Check compact events expand:**
|
|
334
354
|
```spl
|
|
335
|
-
| tenxsearch searchstring="index=your_logs_index" | head 10
|
|
355
|
+
| tenxsearch searchstring="index=your_logs_index sourcetype=tenx_encoded" | head 10
|
|
336
356
|
```
|
|
337
357
|
:material-check-circle-outline:{ .success } Expected: Full expanded events, not `~hash,var1,var2...` format
|
|
338
358
|
|
|
339
|
-
!!! tip "
|
|
340
|
-
The "Consume KV" saved search
|
|
359
|
+
!!! tip "Templates indexed before the app"
|
|
360
|
+
The "Consume KV" saved search stores new templates every 5 minutes. To load templates indexed before the app was installed, run this search once as an admin or power user:
|
|
361
|
+
|
|
362
|
+
```spl
|
|
363
|
+
index=tenx_dml sourcetype=tenx_dml_raw_json earliest=-30d | sendalert tenx_dml_to_kv
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
Running it again is safe: templates already stored are skipped.
|
|
341
367
|
|
|
342
368
|
??? tenx-step-search "Step 5: Search from the Search Bar"
|
|
343
369
|
|
|
@@ -356,8 +382,8 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
356
382
|
|
|
357
383
|
`NOT`, `OR`, `AND`, parenthesised groups, wildcards, field conditions, inline `earliest=`/`latest=` and a trailing pipeline all behave as they do on the original data. The command also runs in saved searches, alerts and the REST API.
|
|
358
384
|
|
|
359
|
-
!!!
|
|
360
|
-
A
|
|
385
|
+
!!! info "Name compact indexes"
|
|
386
|
+
A search over a compact sourcetype reaches the original words through `tenxsearch`; without it, a keyword search returns no events. Keep compact indexes out of users' default index sets and name them in searches.
|
|
361
387
|
|
|
362
388
|
!!! tip "Dashboards need no command"
|
|
363
389
|
Classic dashboards in an app carrying `dashboard.js` keep their panel SPL unchanged. See [Expansion in Other Splunk Apps](#expansion-in-other-splunk-apps) below.
|
|
@@ -368,7 +394,7 @@ Get encoded events flowing to Splunk in under 15 minutes.
|
|
|
368
394
|
|
|
369
395
|
Use this checklist to diagnose issues at each stage of the pipeline.
|
|
370
396
|
|
|
371
|
-
??? tenx-checklist "Templates Arriving in Splunk
|
|
397
|
+
??? tenx-checklist "Templates Arriving in Splunk"
|
|
372
398
|
|
|
373
399
|
**Test Query:**
|
|
374
400
|
```spl
|
|
@@ -390,7 +416,7 @@ Use this checklist to diagnose issues at each stage of the pipeline.
|
|
|
390
416
|
{"templateHash":"#ABC123xy","template":"INFO $0 - User $1 logged in from $2"}
|
|
391
417
|
```
|
|
392
418
|
|
|
393
|
-
??? tenx-checklist "KV Store Populated
|
|
419
|
+
??? tenx-checklist "KV Store Populated"
|
|
394
420
|
|
|
395
421
|
**Test Query:**
|
|
396
422
|
```spl
|
|
@@ -405,20 +431,20 @@ Use this checklist to diagnose issues at each stage of the pipeline.
|
|
|
405
431
|
**Verify saved search configuration:**
|
|
406
432
|
|
|
407
433
|
1. Go to **Settings** → **Searches, reports, and alerts**
|
|
408
|
-
2. Find "Consume KV" in the
|
|
434
|
+
2. Find "Consume KV" in the Log10x App
|
|
409
435
|
3. Verify search string includes: `index=tenx_dml sourcetype=tenx_dml_raw_json`
|
|
410
436
|
4. Click **Run** to manually trigger
|
|
411
437
|
|
|
412
|
-
??? tenx-checklist "Events Expanding Correctly
|
|
438
|
+
??? tenx-checklist "Events Expanding Correctly"
|
|
413
439
|
|
|
414
440
|
**Test Query:**
|
|
415
441
|
```spl
|
|
416
|
-
index=your_logs_index | head 5
|
|
442
|
+
| tenxsearch searchstring="index=your_logs_index sourcetype=tenx_encoded" | head 5
|
|
417
443
|
```
|
|
418
444
|
|
|
419
445
|
| Result | Meaning | Action |
|
|
420
446
|
|--------|---------|--------|
|
|
421
|
-
| Full expanded events |
|
|
447
|
+
| Full expanded events | Expansion working | None |
|
|
422
448
|
| `~hash,var1,var2...` format | Events not expanding | Check hash exists in KV store |
|
|
423
449
|
| `{"log":"~hash,..."}` | JSON wrapping | Add `event_key $log` to forwarder |
|
|
424
450
|
|
|
@@ -439,21 +465,9 @@ Use this checklist to diagnose issues at each stage of the pipeline.
|
|
|
439
465
|
|
|
440
466
|
| Cause | Solution |
|
|
441
467
|
|-------|----------|
|
|
442
|
-
|
|
|
443
|
-
|
|
|
444
|
-
|
|
|
445
|
-
| Saved search disabled | Enable in **Settings** → **Searches** → "Consume KV" |
|
|
446
|
-
|
|
447
|
-
**Fix saved search manually:**
|
|
448
|
-
|
|
449
|
-
Edit `$SPLUNK_HOME/etc/apps/tenx-for-splunk/local/savedsearches.conf`:
|
|
450
|
-
```ini
|
|
451
|
-
[Consume KV]
|
|
452
|
-
search = index=tenx_dml sourcetype=tenx_dml_raw_json
|
|
453
|
-
dispatch.earliest_time = -1d
|
|
454
|
-
```
|
|
455
|
-
|
|
456
|
-
Restart Splunk or run the search manually to apply.
|
|
468
|
+
| Templates indexed before the app was installed, or while "Consume KV" was down | Run `index=tenx_dml sourcetype=tenx_dml_raw_json earliest=-30d | sendalert tenx_dml_to_kv` once |
|
|
469
|
+
| Saved search disabled | Enable in **Settings** → **Searches, reports, and alerts** → "Consume KV" |
|
|
470
|
+
| Templates in another index | Point the templates HEC token at `tenx_dml` |
|
|
457
471
|
|
|
458
472
|
??? tenx-troubleshoot "Events Show Raw Encoded Format"
|
|
459
473
|
|
|
@@ -463,7 +477,7 @@ Use this checklist to diagnose issues at each stage of the pipeline.
|
|
|
463
477
|
|
|
464
478
|
| Cause | Solution |
|
|
465
479
|
|-------|----------|
|
|
466
|
-
| Template hash not in KV store | Wait
|
|
480
|
+
| Template hash not in KV store | Wait 5 minutes for "Consume KV", or run the backfill search above |
|
|
467
481
|
| Hash mismatch (different data) | Verify encoded events match templates in `tenx_dml` |
|
|
468
482
|
| Search ran without expansion | On a dashboard, ensure `dashboard.js` is in that app; in the search bar, wrap the search in `tenxsearch` |
|
|
469
483
|
|
|
@@ -512,7 +526,7 @@ Use this checklist to diagnose issues at each stage of the pipeline.
|
|
|
512
526
|
port 8088
|
|
513
527
|
token YOUR_TOKEN
|
|
514
528
|
index your_index
|
|
515
|
-
sourcetype
|
|
529
|
+
sourcetype tenx_encoded
|
|
516
530
|
<format>
|
|
517
531
|
@type single_value
|
|
518
532
|
message_key log
|
|
@@ -547,13 +561,13 @@ Use this checklist to diagnose issues at each stage of the pipeline.
|
|
|
547
561
|
| Connection refused | HEC disabled | Enable in **Settings** → **Data inputs** → **HTTP Event Collector** → **Global Settings** |
|
|
548
562
|
| 401 Unauthorized | Invalid token | Verify token value, check token is enabled |
|
|
549
563
|
| 403 Forbidden | Token restrictions | Check token's allowed indexes and sourcetypes |
|
|
550
|
-
| SSL error |
|
|
564
|
+
| SSL error | Forwarder does not trust the HEC certificate | Install Splunk's CA certificate on the forwarder host; turn verification off only on a test instance |
|
|
551
565
|
|
|
552
566
|
---
|
|
553
567
|
|
|
554
568
|
## :material-code-tags: Expansion in Other Splunk Apps
|
|
555
569
|
|
|
556
|
-
`dashboard.js` loads on every classic dashboard in the app that carries it. The
|
|
570
|
+
`dashboard.js` loads on every classic dashboard in the app that carries it. The Log10x App ships it, so its own dashboards expand. To expand classic dashboards in another app, copy the file there:
|
|
557
571
|
|
|
558
572
|
``` { .console .copy }
|
|
559
573
|
cp $SPLUNK_HOME/etc/apps/tenx-for-splunk/appserver/static/dashboard.js \
|
|
@@ -562,8 +576,8 @@ cp $SPLUNK_HOME/etc/apps/tenx-for-splunk/appserver/static/dashboard.js \
|
|
|
562
576
|
|
|
563
577
|
Restart Splunk to load it.
|
|
564
578
|
|
|
565
|
-
!!! info "
|
|
566
|
-
The search bar and Dashboard Studio
|
|
579
|
+
!!! info "Search bar and Dashboard Studio"
|
|
580
|
+
The search bar and Dashboard Studio use the `tenxsearch` command, as in the Quickstart.
|
|
567
581
|
|
|
568
582
|
??? info "Programmatic Hook for Custom Views"
|
|
569
583
|
|
|
@@ -588,10 +602,10 @@ Restart Splunk to load it.
|
|
|
588
602
|
| [**Search Handler**](https://github.com/log-10x/splunk-app/blob/main/tenx-for-splunk/bin/tenx_search_handler.py){target="\_blank"} | REST endpoint transforming SPL queries |
|
|
589
603
|
| [**KV Store**](https://github.com/log-10x/splunk-app/blob/main/tenx-for-splunk/default/collections.conf){target="\_blank"} | Template patterns for event expansion |
|
|
590
604
|
| [**Inflate Macro**](https://github.com/log-10x/splunk-app/blob/main/tenx-for-splunk/default/macros.conf){target="\_blank"} | SPL macro joining events with templates |
|
|
591
|
-
| [**Consume KV Search**](https://github.com/log-10x/splunk-app/blob/main/tenx-for-splunk/default/savedsearches.conf){target="\_blank"} | Scheduled search
|
|
592
|
-
| [**Analytics Dashboard**](https://github.com/log-10x/splunk-app/blob/main/tenx-for-splunk/default/data/ui/views/tenx_dashboard.xml){target="\_blank"} |
|
|
605
|
+
| [**Consume KV Search**](https://github.com/log-10x/splunk-app/blob/main/tenx-for-splunk/default/savedsearches.conf){target="\_blank"} | Scheduled search storing new templates in the KV Store |
|
|
606
|
+
| [**Analytics Dashboard**](https://github.com/log-10x/splunk-app/blob/main/tenx-for-splunk/default/data/ui/views/tenx_dashboard.xml){target="\_blank"} | Compression and template metrics |
|
|
593
607
|
| [**Diagnostics Dashboard**](https://github.com/log-10x/splunk-app/blob/main/tenx-for-splunk/default/data/ui/views/tenx_diagnostics.xml){target="\_blank"} | Troubleshooting and verification tools |
|
|
594
608
|
|
|
595
609
|
---
|
|
596
610
|
|
|
597
|
-
<br/>:material-github: This app is open source. View on [GitHub](https://github.com/log-10x/splunk-app "
|
|
611
|
+
<br/>:material-github: This app is open source. View on [GitHub](https://github.com/log-10x/splunk-app "Log10x App"){target="\_blank"}.
|
|
@@ -124,7 +124,7 @@ The Retriever ships selected events from low-cost storage (S3, Azure Blobs) to l
|
|
|
124
124
|
|
|
125
125
|
**Generic:** Any HTTP endpoint, syslog, or TCP destination. Logs stream with original timestamps preserved.
|
|
126
126
|
|
|
127
|
-
For Splunk users, the optional [Receiver in Compact mode](https://doc.log10x.com/apps/receiver/compact/){target="_blank"} compacts events at the source for an additional reduction in ingestion volume (modeled, varies by data). Events are expanded transparently via the [
|
|
127
|
+
For Splunk users, the optional [Receiver in Compact mode](https://doc.log10x.com/apps/receiver/compact/){target="_blank"} compacts events at the source for an additional reduction in ingestion volume (modeled, varies by data). Events are expanded transparently via the [Log10x App](https://doc.log10x.com/apps/receiver/compact/splunk/){target="_blank"} at search time.
|
|
128
128
|
|
|
129
129
|
For Elasticsearch and OpenSearch users, the [L1ES plugin](https://doc.log10x.com/apps/receiver/compact/elasticsearch/){target="_blank"} expands compact events at query time. Kibana searches, dashboards, and alerts work unchanged on the original full data.
|
|
130
130
|
|
|
@@ -347,7 +347,7 @@ The Retriever ships selected events from low-cost storage (S3, Azure Blobs) to l
|
|
|
347
347
|
The 10x compact format is fully expandable without relying on log10x infrastructure. Multiple open-source expansion tools are available:
|
|
348
348
|
|
|
349
349
|
**1. Splunk App (Apache 2.0)**
|
|
350
|
-
- [
|
|
350
|
+
- [Log10x App](https://github.com/log-10x/splunk-app){target="_blank"}, open-source app that automatically expands compact events
|
|
351
351
|
- Works as search-time macro: all your existing KQL queries work unchanged
|
|
352
352
|
|
|
353
353
|
**2. Java Library and CLI (Apache 2.0)**
|
|
@@ -124,7 +124,7 @@ The Retriever ships selected events from low-cost storage (S3, Azure Blobs) to l
|
|
|
124
124
|
|
|
125
125
|
**Generic:** Any HTTP endpoint, syslog, or TCP destination. Logs stream with original timestamps preserved.
|
|
126
126
|
|
|
127
|
-
For Splunk users, the optional [Receiver in Compact mode](https://doc.log10x.com/apps/receiver/compact/){target="_blank"} compacts events at the source for an additional reduction in ingestion volume (modeled, varies by data). Events are expanded transparently via the [
|
|
127
|
+
For Splunk users, the optional [Receiver in Compact mode](https://doc.log10x.com/apps/receiver/compact/){target="_blank"} compacts events at the source for an additional reduction in ingestion volume (modeled, varies by data). Events are expanded transparently via the [Log10x App](https://doc.log10x.com/apps/receiver/compact/splunk/){target="_blank"} at search time.
|
|
128
128
|
|
|
129
129
|
For Elasticsearch and OpenSearch users, the [L1ES plugin](https://doc.log10x.com/apps/receiver/compact/elasticsearch/){target="_blank"} expands compact events at query time. Kibana searches, dashboards, and alerts work unchanged on the original full data.
|
|
130
130
|
|
|
@@ -313,7 +313,7 @@ The Retriever ships selected events from low-cost storage (S3, Azure Blobs) to l
|
|
|
313
313
|
The 10x compact format is fully expandable without relying on log10x infrastructure. Multiple open-source expansion tools are available:
|
|
314
314
|
|
|
315
315
|
**1. Splunk App (Apache 2.0)**
|
|
316
|
-
- [
|
|
316
|
+
- [Log10x App](https://github.com/log-10x/splunk-app){target="_blank"}, open-source app that automatically expands compact events
|
|
317
317
|
- Works as search-time macro: all your existing KQL queries work unchanged
|
|
318
318
|
|
|
319
319
|
**2. Java Library and CLI (Apache 2.0)**
|
|
@@ -170,7 +170,7 @@ description: "What the 10x Engine does, the measured 63.7% lossless reduction on
|
|
|
170
170
|
- **[Receiver, Compact mode](https://doc.log10x.com/apps/receiver/compact/)**: losslessly compacts events before ingestion where the destination supports it (Splunk, self-hosted Elasticsearch/OpenSearch)
|
|
171
171
|
- **[Retriever](https://doc.log10x.com/apps/retriever/)**: offloads selected events to your own S3 bucket and fetches the exact events back to your analyzer on demand
|
|
172
172
|
|
|
173
|
-
The open-source [
|
|
173
|
+
The open-source [Log10x App](https://doc.log10x.com/apps/receiver/compact/splunk/) auto-expands compact logs at search time. For other analyzers, [Retriever](https://doc.log10x.com/apps/retriever/) expands and streams events on-demand.
|
|
174
174
|
|
|
175
175
|
??? tenx-compare "How does 10x differ from APMs and OpenTelemetry"
|
|
176
176
|
|
|
@@ -73,9 +73,9 @@ title: "Comparisons"
|
|
|
73
73
|
- [ ] Check logs for startup errors: `kubectl logs -l app=log10x-receiver`
|
|
74
74
|
|
|
75
75
|
3. **Install Splunk App**
|
|
76
|
-
- [ ] Download [
|
|
76
|
+
- [ ] Download [Log10x App](https://github.com/log-10x/splunk-app/releases){target="_blank"} from GitHub
|
|
77
77
|
- [ ] Install via Splunk UI: Apps → Install app from file
|
|
78
|
-
- [ ] Verify installation: Settings → Installed apps (should show "
|
|
78
|
+
- [ ] Verify installation: Settings → Installed apps (should show "Log10x App")
|
|
79
79
|
|
|
80
80
|
4. **Configure KV Store**
|
|
81
81
|
- [ ] Create KV Store collection `tenx_dml` with required schema
|
|
@@ -90,7 +90,7 @@ title: "Comparisons"
|
|
|
90
90
|
|
|
91
91
|
**Rollback procedure (if needed):**
|
|
92
92
|
- **Remove Receiver:** `helm uninstall log10x-receiver` (logs resume at full volume)
|
|
93
|
-
- **Remove app:** Splunk UI → Apps → Uninstall
|
|
93
|
+
- **Remove app:** Splunk UI → Apps → Uninstall the Log10x App
|
|
94
94
|
- **Both actions are safe:** No configuration changes needed, no data loss
|
|
95
95
|
|
|
96
96
|
**Compatibility checklist before full deployment:**
|