logstash-codec-cef 6.1.2-java → 6.2.3-java
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +15 -1
- data/docs/index.asciidoc +452 -9
- data/lib/logstash/codecs/cef.rb +272 -111
- data/lib/logstash/codecs/cef/timestamp_normalizer.rb +115 -0
- data/logstash-codec-cef.gemspec +3 -1
- data/spec/codecs/cef/timestamp_normalizer_spec.rb +274 -0
- data/spec/codecs/cef_spec.rb +500 -306
- metadata +37 -7
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d45c024f0bdb71f6056b553e138fff8d57479d021ab3ba8b98ba69384bf9898f
|
|
4
|
+
data.tar.gz: 7ddb2bd1427fcf5c2ca91e762a326d0a9e73e17d911dbadaf4dffc4fdcfc50e8
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 4e1afac1d4c0c05fa8bc4db6f61063d591e47ba85623b8e2530cddbf599e4e2cc6e3b968f1effbe23bb196a1864111d70eee6b354d7404cabb6f27b1fa431be2
|
|
7
|
+
data.tar.gz: 872d25b0b8f8b2aa3f2e884794df1afd614cb26408ce7bdcdc4fc9e7a90cd9b6750659b2da37b4072229cec8f83ebbbd5eca691af32926404e967e1a00c4c628
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
|
+
## 6.2.3
|
|
2
|
+
- Feat: event_factory support [#94](https://github.com/logstash-plugins/logstash-codec-cef/pull/94)
|
|
3
|
+
|
|
4
|
+
## 6.2.2
|
|
5
|
+
- Fixed invalid Field Reference that could occur when ECS mode was enabled and the CEF field `fileHash` was parsed.
|
|
6
|
+
- Added expanded mapping for numbered `deviceCustom*` and `deviceCustom*Label` fields so that all now include numbers 1 through 15. [#89](https://github.com/logstash-plugins/logstash-codec-cef/pull/89).
|
|
7
|
+
|
|
8
|
+
## 6.2.1
|
|
9
|
+
- Added field mapping to docs.
|
|
10
|
+
- Fixed ECS mapping of `deviceMacAddress` field. [#88](https://github.com/logstash-plugins/logstash-codec-cef/pull/88).
|
|
11
|
+
|
|
12
|
+
## 6.2.0
|
|
13
|
+
- Introduce ECS Compatibility mode [#83](https://github.com/logstash-plugins/logstash-codec-cef/pull/83).
|
|
14
|
+
|
|
1
15
|
## 6.1.2
|
|
2
|
-
- Added error log with full payload when something bad happens in decoding a message[#84](https://github.com/logstash-plugins/logstash-codec-cef/pull/84)
|
|
16
|
+
- Added error log with full payload when something bad happens in decoding a message [#84](https://github.com/logstash-plugins/logstash-codec-cef/pull/84)
|
|
3
17
|
|
|
4
18
|
## 6.1.1
|
|
5
19
|
- Improved encoding performance, especially when encoding many extension fields [#81](https://github.com/logstash-plugins/logstash-codec-cef/pull/81)
|
data/docs/index.asciidoc
CHANGED
|
@@ -27,14 +27,385 @@ https://community.saas.hpe.com/dcvta86296/attachments/dcvta86296/connector-docum
|
|
|
27
27
|
If this codec receives a payload from an input that is not a valid CEF message, then it will
|
|
28
28
|
produce an event with the payload as the 'message' field and a '_cefparsefailure' tag.
|
|
29
29
|
|
|
30
|
+
==== Compatibility with the Elastic Common Schema (ECS)
|
|
31
|
+
|
|
32
|
+
This plugin can be used to decode CEF events _into_ the Elastic Common Schema, or to encode ECS-compatible events into CEF.
|
|
33
|
+
It can also be used _without_ ECS, encoding and decoding events using only CEF-defined field names and keys.
|
|
34
|
+
|
|
35
|
+
The ECS Compatibility mode for a specific plugin instance can be controlled by setting <<plugins-{type}s-{plugin}-ecs_compatibility>> when defining the codec:
|
|
36
|
+
|
|
37
|
+
[source,sh]
|
|
38
|
+
-----
|
|
39
|
+
input {
|
|
40
|
+
tcp {
|
|
41
|
+
# ...
|
|
42
|
+
codec => cef {
|
|
43
|
+
ecs_compatibility => v1
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
}
|
|
47
|
+
-----
|
|
48
|
+
|
|
49
|
+
If left unspecified, the value of the `pipeline.ecs_compatibility` setting is used.
|
|
50
|
+
|
|
51
|
+
===== Timestamps and ECS compatiblity
|
|
52
|
+
|
|
53
|
+
When decoding in ECS Compatibility Mode, timestamp-type fields are parsed and normalized
|
|
54
|
+
to specific points on the timeline.
|
|
55
|
+
|
|
56
|
+
Because the CEF format allows ambiguous timestamp formats, some reasonable assumptions are made:
|
|
57
|
+
|
|
58
|
+
- When the timestamp does not include a year, we assume it happened in the recent past
|
|
59
|
+
(or _very_ near future to accommodate out-of-sync clocks and timezone offsets).
|
|
60
|
+
- When the timestamp does not include UTC-offset information, we use the event's
|
|
61
|
+
timezone (`dtz` or `deviceTimeZone` field), or fall through to this plugin's
|
|
62
|
+
<<plugins-{type}s-{plugin}-default_timezone>>.
|
|
63
|
+
- Localized timestamps are parsed using the provided <<plugins-{type}s-{plugin}-locale>>.
|
|
64
|
+
|
|
65
|
+
[id="plugins-{type}s-{plugin}-field-mapping"]
|
|
66
|
+
===== Field mapping
|
|
67
|
+
|
|
68
|
+
The header fields from each CEF payload is expanded to the following fields, depending on whether ECS is enabled.
|
|
69
|
+
|
|
70
|
+
[id="plugins-{type}s-{plugin}-header-field"]
|
|
71
|
+
====== Header field mapping
|
|
72
|
+
|=====
|
|
73
|
+
|ECS Disabled | ECS Field
|
|
74
|
+
|
|
75
|
+
|`cefVersion` |`[cef][version]`
|
|
76
|
+
|`deviceVendor` |`[observer][vendor]`
|
|
77
|
+
|`deviceProduct` |`[observer][product]`
|
|
78
|
+
|`deviceVersion` |`[observer][version]`
|
|
79
|
+
|`deviceEventClassId`|`[event][code]`
|
|
80
|
+
|`name` |`[cef][name]`
|
|
81
|
+
|`severity` |`[event][severity]`
|
|
82
|
+
|=====
|
|
83
|
+
|
|
84
|
+
When decoding CEF payloads with `ecs_compatibility => disabled`, the abbreviated CEF Keys found in extensions are expanded, and CEF Field Names are inserted at the root level of the event.
|
|
85
|
+
|
|
86
|
+
When decoding in an ECS Compatibility mode, the ECS Fields are populated from the corresponding CEF Field Names _or_ CEF Keys found in the payload's extensions.
|
|
87
|
+
|
|
88
|
+
The following is a mapping between these fields.
|
|
89
|
+
|
|
90
|
+
// Templates for short-hand notes in the table below
|
|
91
|
+
:cef-ambiguous-higher: pass:quotes[Multiple possible CEF fields map to this ECS Field. When decoding, the last entry encountered wins. When encoding, this field has _higher_ priority.]
|
|
92
|
+
:cef-ambiguous-lower: pass:quotes[Multiple possible CEF fields map to this ECS Field. When decoding, the last entry encountered wins. When encoding, this field has _lower_ priority.]
|
|
93
|
+
:cef-normalize-timestamp: pass:quotes[This field contains a timestamp. In ECS Compatibility Mode, it is parsed to a specific point in time.]
|
|
94
|
+
:cef-plugin-config-condition: pass:quotes[When plugin configured with]
|
|
95
|
+
|
|
96
|
+
|
|
97
|
+
[id="plugins-{type}s-{plugin}-ext-field"]
|
|
98
|
+
====== Extension field mapping
|
|
99
|
+
|=======================================================================================================================
|
|
100
|
+
|CEF Field Name (optional CEF Key) |ECS Field
|
|
101
|
+
|
|
102
|
+
|`agentAddress` (`agt`) |`[agent][ip]`
|
|
103
|
+
|`agentDnsDomain` |`[cef][agent][registered_domain]`
|
|
104
|
+
|
|
105
|
+
{cef-ambiguous-higher}
|
|
106
|
+
|`agentHostName` (`ahost`) |`[agent][name]`
|
|
107
|
+
|`agentId` (`aid`) |`[agent][id]`
|
|
108
|
+
|`agentMacAddress` (`amac`) |`[agent][mac]`
|
|
109
|
+
|`agentNtDomain` |`[cef][agent][registered_domain]`
|
|
110
|
+
|
|
111
|
+
{cef-ambiguous-lower}
|
|
112
|
+
|`agentReceiptTime` (`art`) |`[event][created]`
|
|
113
|
+
|
|
114
|
+
{cef-normalize-timestamp}
|
|
115
|
+
|`agentTimeZone` (`atz`) |`[cef][agent][timezone]`
|
|
116
|
+
|`agentTranslatedAddress` |`[cef][agent][nat][ip]`
|
|
117
|
+
|`agentTranslatedZoneExternalID` |`[cef][agent][translated_zone][external_id]`
|
|
118
|
+
|`agentTranslatedZoneURI` |`[cef][agent][translated_zone][uri]`
|
|
119
|
+
|`agentType` (`at`) |`[agent][type]`
|
|
120
|
+
|`agentVersion` (`av`) |`[agent][version]`
|
|
121
|
+
|`agentZoneExternalID` |`[cef][agent][zone][external_id]`
|
|
122
|
+
|`agentZoneURI` |`[cef][agent][zone][uri]`
|
|
123
|
+
|`applicationProtocol` (`app`) |`[network][protocol]`
|
|
124
|
+
|`baseEventCount` (`cnt`) |`[cef][base_event_count]`
|
|
125
|
+
|`bytesIn` (`in`) |`[source][bytes]`
|
|
126
|
+
|`bytesOut` (`out`) |`[destination][bytes]`
|
|
127
|
+
|`categoryDeviceType` (`catdt`) |`[cef][device_type]`
|
|
128
|
+
|`customerExternalID` |`[organization][id]`
|
|
129
|
+
|`customerURI` |`[organization][name]`
|
|
130
|
+
|`destinationAddress` (`dst`) |`[destination][ip]`
|
|
131
|
+
|`destinationDnsDomain` |`[destination][registered_domain]`
|
|
132
|
+
|
|
133
|
+
{cef-ambiguous-higher}
|
|
134
|
+
|`destinationGeoLatitude` (`dlat`) |`[destination][geo][location][lat]`
|
|
135
|
+
|`destinationGeoLongitude` (`dlong`) |`[destination][geo][location][lon]`
|
|
136
|
+
|`destinationHostName` (`dhost`) |`[destination][domain]`
|
|
137
|
+
|`destinationMacAddress` (`dmac`) |`[destination][mac]`
|
|
138
|
+
|`destinationNtDomain` (`dntdom`) |`[destination][registered_domain]`
|
|
139
|
+
|
|
140
|
+
{cef-ambiguous-lower}
|
|
141
|
+
|`destinationPort` (`dpt`) |`[destination][port]`
|
|
142
|
+
|`destinationProcessId` (`dpid`) |`[destination][process][pid]`
|
|
143
|
+
|`destinationProcessName` (`dproc`) |`[destination][process][name]`
|
|
144
|
+
|`destinationServiceName` |`[destination][service][name]`
|
|
145
|
+
|`destinationTranslatedAddress` |`[destination][nat][ip]`
|
|
146
|
+
|`destinationTranslatedPort` |`[destination][nat][port]`
|
|
147
|
+
|`destinationTranslatedZoneExternalID` |`[cef][destination][translated_zone][external_id]`
|
|
148
|
+
|`destinationTranslatedZoneURI` |`[cef][destination][translated_zone][uri]`
|
|
149
|
+
|`destinationUserId` (`duid`) |`[destination][user][id]`
|
|
150
|
+
|`destinationUserName` (`duser`) |`[destination][user][name]`
|
|
151
|
+
|`destinationUserPrivileges` (`dpriv`) |`[destination][user][group][name]`
|
|
152
|
+
|`destinationZoneExternalID` |`[cef][destination][zone][external_id]`
|
|
153
|
+
|`destinationZoneURI` |`[cef][destination][zone][uri]`
|
|
154
|
+
|`deviceAction` (`act`) |`[event][action]`
|
|
155
|
+
.2+|`deviceAddress` (`dvc`) |`[observer][ip]`
|
|
156
|
+
|
|
157
|
+
{cef-plugin-config-condition} `device => observer`
|
|
158
|
+
|`[host][ip]`
|
|
159
|
+
|
|
160
|
+
{cef-plugin-config-condition} `device => host`
|
|
161
|
+
|`deviceCustomFloatingPoint1` (`cfp1`) |`[cef][device_custom_floating_point_1][value]`
|
|
162
|
+
|`deviceCustomFloatingPoint1Label` (`cfp1Label`)|`[cef][device_custom_floating_point_1][label]`
|
|
163
|
+
|`deviceCustomFloatingPoint2` (`cfp2`) |`[cef][device_custom_floating_point_2][value]`
|
|
164
|
+
|`deviceCustomFloatingPoint2Label` (`cfp2Label`)|`[cef][device_custom_floating_point_2][label]`
|
|
165
|
+
|`deviceCustomFloatingPoint3` (`cfp3`) |`[cef][device_custom_floating_point_3][value]`
|
|
166
|
+
|`deviceCustomFloatingPoint3Label` (`cfp3Label`)|`[cef][device_custom_floating_point_3][label]`
|
|
167
|
+
|`deviceCustomFloatingPoint4` (`cfp4`) |`[cef][device_custom_floating_point_4][value]`
|
|
168
|
+
|`deviceCustomFloatingPoint4Label` (`cfp4Label`)|`[cef][device_custom_floating_point_4][label]`
|
|
169
|
+
|`deviceCustomFloatingPoint5` (`cfp5`) |`[cef][device_custom_floating_point_5][value]`
|
|
170
|
+
|`deviceCustomFloatingPoint5Label` (`cfp5Label`)|`[cef][device_custom_floating_point_5][label]`
|
|
171
|
+
|`deviceCustomFloatingPoint6` (`cfp6`) |`[cef][device_custom_floating_point_6][value]`
|
|
172
|
+
|`deviceCustomFloatingPoint6Label` (`cfp6Label`)|`[cef][device_custom_floating_point_6][label]`
|
|
173
|
+
|`deviceCustomFloatingPoint7` (`cfp7`) |`[cef][device_custom_floating_point_7][value]`
|
|
174
|
+
|`deviceCustomFloatingPoint7Label` (`cfp7Label`)|`[cef][device_custom_floating_point_7][label]`
|
|
175
|
+
|`deviceCustomFloatingPoint8` (`cfp8`) |`[cef][device_custom_floating_point_8][value]`
|
|
176
|
+
|`deviceCustomFloatingPoint8Label` (`cfp8Label`)|`[cef][device_custom_floating_point_8][label]`
|
|
177
|
+
|`deviceCustomFloatingPoint9` (`cfp9`) |`[cef][device_custom_floating_point_9][value]`
|
|
178
|
+
|`deviceCustomFloatingPoint9Label` (`cfp9Label`)|`[cef][device_custom_floating_point_9][label]`
|
|
179
|
+
|`deviceCustomFloatingPoint10` (`cfp10`) |`[cef][device_custom_floating_point_10][value]`
|
|
180
|
+
|`deviceCustomFloatingPoint10Label` (`cfp10Label`)|`[cef][device_custom_floating_point_10][label]`
|
|
181
|
+
|`deviceCustomFloatingPoint11` (`cfp11`) |`[cef][device_custom_floating_point_11][value]`
|
|
182
|
+
|`deviceCustomFloatingPoint11Label` (`cfp11Label`)|`[cef][device_custom_floating_point_11][label]`
|
|
183
|
+
|`deviceCustomFloatingPoint12` (`cfp12`) |`[cef][device_custom_floating_point_12][value]`
|
|
184
|
+
|`deviceCustomFloatingPoint12Label` (`cfp12Label`)|`[cef][device_custom_floating_point_12][label]`
|
|
185
|
+
|`deviceCustomFloatingPoint13` (`cfp13`) |`[cef][device_custom_floating_point_13][value]`
|
|
186
|
+
|`deviceCustomFloatingPoint13Label` (`cfp13Label`)|`[cef][device_custom_floating_point_13][label]`
|
|
187
|
+
|`deviceCustomFloatingPoint14` (`cfp14`) |`[cef][device_custom_floating_point_14][value]`
|
|
188
|
+
|`deviceCustomFloatingPoint14Label` (`cfp14Label`)|`[cef][device_custom_floating_point_14][label]`
|
|
189
|
+
|`deviceCustomFloatingPoint15` (`cfp15`) |`[cef][device_custom_floating_point_15][value]`
|
|
190
|
+
|`deviceCustomFloatingPoint15Label` (`cfp15Label`)|`[cef][device_custom_floating_point_15][label]`
|
|
191
|
+
|`deviceCustomIPv6Address1` (`c6a1`) |`[cef][device_custom_ipv6_address_1][value]`
|
|
192
|
+
|`deviceCustomIPv6Address1Label` (`c6a1Label`) |`[cef][device_custom_ipv6_address_1][label]`
|
|
193
|
+
|`deviceCustomIPv6Address2` (`c6a2`) |`[cef][device_custom_ipv6_address_2][value]`
|
|
194
|
+
|`deviceCustomIPv6Address2Label` (`c6a2Label`) |`[cef][device_custom_ipv6_address_2][label]`
|
|
195
|
+
|`deviceCustomIPv6Address3` (`c6a3`) |`[cef][device_custom_ipv6_address_3][value]`
|
|
196
|
+
|`deviceCustomIPv6Address3Label` (`c6a3Label`) |`[cef][device_custom_ipv6_address_3][label]`
|
|
197
|
+
|`deviceCustomIPv6Address4` (`c6a4`) |`[cef][device_custom_ipv6_address_4][value]`
|
|
198
|
+
|`deviceCustomIPv6Address4Label` (`c6a4Label`) |`[cef][device_custom_ipv6_address_4][label]`
|
|
199
|
+
|`deviceCustomIPv6Address5` (`c6a5`) |`[cef][device_custom_ipv6_address_5][value]`
|
|
200
|
+
|`deviceCustomIPv6Address5Label` (`c6a5Label`) |`[cef][device_custom_ipv6_address_5][label]`
|
|
201
|
+
|`deviceCustomIPv6Address6` (`c6a6`) |`[cef][device_custom_ipv6_address_6][value]`
|
|
202
|
+
|`deviceCustomIPv6Address6Label` (`c6a6Label`) |`[cef][device_custom_ipv6_address_6][label]`
|
|
203
|
+
|`deviceCustomIPv6Address7` (`c6a7`) |`[cef][device_custom_ipv6_address_7][value]`
|
|
204
|
+
|`deviceCustomIPv6Address7Label` (`c6a7Label`) |`[cef][device_custom_ipv6_address_7][label]`
|
|
205
|
+
|`deviceCustomIPv6Address8` (`c6a8`) |`[cef][device_custom_ipv6_address_8][value]`
|
|
206
|
+
|`deviceCustomIPv6Address8Label` (`c6a8Label`) |`[cef][device_custom_ipv6_address_8][label]`
|
|
207
|
+
|`deviceCustomIPv6Address9` (`c6a9`) |`[cef][device_custom_ipv6_address_9][value]`
|
|
208
|
+
|`deviceCustomIPv6Address9Label` (`c6a9Label`) |`[cef][device_custom_ipv6_address_9][label]`
|
|
209
|
+
|`deviceCustomIPv6Address10` (`c6a10`) |`[cef][device_custom_ipv6_address_10][value]`
|
|
210
|
+
|`deviceCustomIPv6Address10Label` (`c6a10Label`)|`[cef][device_custom_ipv6_address_10][label]`
|
|
211
|
+
|`deviceCustomIPv6Address11` (`c6a11`) |`[cef][device_custom_ipv6_address_11][value]`
|
|
212
|
+
|`deviceCustomIPv6Address11Label` (`c6a11Label`)|`[cef][device_custom_ipv6_address_11][label]`
|
|
213
|
+
|`deviceCustomIPv6Address12` (`c6a12`) |`[cef][device_custom_ipv6_address_12][value]`
|
|
214
|
+
|`deviceCustomIPv6Address12Label` (`c6a12Label`)|`[cef][device_custom_ipv6_address_12][label]`
|
|
215
|
+
|`deviceCustomIPv6Address13` (`c6a13`) |`[cef][device_custom_ipv6_address_13][value]`
|
|
216
|
+
|`deviceCustomIPv6Address13Label` (`c6a13Label`)|`[cef][device_custom_ipv6_address_13][label]`
|
|
217
|
+
|`deviceCustomIPv6Address14` (`c6a14`) |`[cef][device_custom_ipv6_address_14][value]`
|
|
218
|
+
|`deviceCustomIPv6Address14Label` (`c6a14Label`)|`[cef][device_custom_ipv6_address_14][label]`
|
|
219
|
+
|`deviceCustomIPv6Address15` (`c6a15`) |`[cef][device_custom_ipv6_address_15][value]`
|
|
220
|
+
|`deviceCustomIPv6Address15Label` (`c6a15Label`)|`[cef][device_custom_ipv6_address_15][label]`
|
|
221
|
+
|`deviceCustomNumber1` (`cn1`) |`[cef][device_custom_number_1][value]`
|
|
222
|
+
|`deviceCustomNumber1Label` (`cn1Label`) |`[cef][device_custom_number_1][label]`
|
|
223
|
+
|`deviceCustomNumber2` (`cn2`) |`[cef][device_custom_number_2][value]`
|
|
224
|
+
|`deviceCustomNumber2Label` (`cn2Label`) |`[cef][device_custom_number_2][label]`
|
|
225
|
+
|`deviceCustomNumber3` (`cn3`) |`[cef][device_custom_number_3][value]`
|
|
226
|
+
|`deviceCustomNumber3Label` (`cn3Label`) |`[cef][device_custom_number_3][label]`
|
|
227
|
+
|`deviceCustomNumber4` (`cn4`) |`[cef][device_custom_number_4][value]`
|
|
228
|
+
|`deviceCustomNumber4Label` (`cn4Label`) |`[cef][device_custom_number_4][label]`
|
|
229
|
+
|`deviceCustomNumber5` (`cn5`) |`[cef][device_custom_number_5][value]`
|
|
230
|
+
|`deviceCustomNumber5Label` (`cn5Label`) |`[cef][device_custom_number_5][label]`
|
|
231
|
+
|`deviceCustomNumber6` (`cn6`) |`[cef][device_custom_number_6][value]`
|
|
232
|
+
|`deviceCustomNumber6Label` (`cn6Label`) |`[cef][device_custom_number_6][label]`
|
|
233
|
+
|`deviceCustomNumber7` (`cn7`) |`[cef][device_custom_number_7][value]`
|
|
234
|
+
|`deviceCustomNumber7Label` (`cn7Label`) |`[cef][device_custom_number_7][label]`
|
|
235
|
+
|`deviceCustomNumber8` (`cn8`) |`[cef][device_custom_number_8][value]`
|
|
236
|
+
|`deviceCustomNumber8Label` (`cn8Label`) |`[cef][device_custom_number_8][label]`
|
|
237
|
+
|`deviceCustomNumber9` (`cn9`) |`[cef][device_custom_number_9][value]`
|
|
238
|
+
|`deviceCustomNumber9Label` (`cn9Label`) |`[cef][device_custom_number_9][label]`
|
|
239
|
+
|`deviceCustomNumber10` (`cn10`) |`[cef][device_custom_number_10][value]`
|
|
240
|
+
|`deviceCustomNumber10Label` (`cn10Label`) |`[cef][device_custom_number_10][label]`
|
|
241
|
+
|`deviceCustomNumber11` (`cn11`) |`[cef][device_custom_number_11][value]`
|
|
242
|
+
|`deviceCustomNumber11Label` (`cn11Label`) |`[cef][device_custom_number_11][label]`
|
|
243
|
+
|`deviceCustomNumber12` (`cn12`) |`[cef][device_custom_number_12][value]`
|
|
244
|
+
|`deviceCustomNumber12Label` (`cn12Label`) |`[cef][device_custom_number_12][label]`
|
|
245
|
+
|`deviceCustomNumber13` (`cn13`) |`[cef][device_custom_number_13][value]`
|
|
246
|
+
|`deviceCustomNumber13Label` (`cn13Label`) |`[cef][device_custom_number_13][label]`
|
|
247
|
+
|`deviceCustomNumber14` (`cn14`) |`[cef][device_custom_number_14][value]`
|
|
248
|
+
|`deviceCustomNumber14Label` (`cn14Label`) |`[cef][device_custom_number_14][label]`
|
|
249
|
+
|`deviceCustomNumber15` (`cn15`) |`[cef][device_custom_number_15][value]`
|
|
250
|
+
|`deviceCustomNumber15Label` (`cn15Label`) |`[cef][device_custom_number_15][label]`
|
|
251
|
+
|`deviceCustomString1` (`cs1`) |`[cef][device_custom_string_1][value]`
|
|
252
|
+
|`deviceCustomString1Label` (`cs1Label`) |`[cef][device_custom_string_1][label]`
|
|
253
|
+
|`deviceCustomString2` (`cs2`) |`[cef][device_custom_string_2][value]`
|
|
254
|
+
|`deviceCustomString2Label` (`cs2Label`) |`[cef][device_custom_string_2][label]`
|
|
255
|
+
|`deviceCustomString3` (`cs3`) |`[cef][device_custom_string_3][value]`
|
|
256
|
+
|`deviceCustomString3Label` (`cs3Label`) |`[cef][device_custom_string_3][label]`
|
|
257
|
+
|`deviceCustomString4` (`cs4`) |`[cef][device_custom_string_4][value]`
|
|
258
|
+
|`deviceCustomString4Label` (`cs4Label`) |`[cef][device_custom_string_4][label]`
|
|
259
|
+
|`deviceCustomString5` (`cs5`) |`[cef][device_custom_string_5][value]`
|
|
260
|
+
|`deviceCustomString5Label` (`cs5Label`) |`[cef][device_custom_string_5][label]`
|
|
261
|
+
|`deviceCustomString6` (`cs6`) |`[cef][device_custom_string_6][value]`
|
|
262
|
+
|`deviceCustomString6Label` (`cs6Label`) |`[cef][device_custom_string_6][label]`
|
|
263
|
+
|`deviceCustomString7` (`cs7`) |`[cef][device_custom_string_7][value]`
|
|
264
|
+
|`deviceCustomString7Label` (`cs7Label`) |`[cef][device_custom_string_7][label]`
|
|
265
|
+
|`deviceCustomString8` (`cs8`) |`[cef][device_custom_string_8][value]`
|
|
266
|
+
|`deviceCustomString8Label` (`cs8Label`) |`[cef][device_custom_string_8][label]`
|
|
267
|
+
|`deviceCustomString9` (`cs9`) |`[cef][device_custom_string_9][value]`
|
|
268
|
+
|`deviceCustomString9Label` (`cs9Label`) |`[cef][device_custom_string_9][label]`
|
|
269
|
+
|`deviceCustomString10` (`cs10`) |`[cef][device_custom_string_10][value]`
|
|
270
|
+
|`deviceCustomString10Label` (`cs10Label`) |`[cef][device_custom_string_10][label]`
|
|
271
|
+
|`deviceCustomString11` (`cs11`) |`[cef][device_custom_string_11][value]`
|
|
272
|
+
|`deviceCustomString11Label` (`cs11Label`) |`[cef][device_custom_string_11][label]`
|
|
273
|
+
|`deviceCustomString12` (`cs12`) |`[cef][device_custom_string_12][value]`
|
|
274
|
+
|`deviceCustomString12Label` (`cs12Label`) |`[cef][device_custom_string_12][label]`
|
|
275
|
+
|`deviceCustomString13` (`cs13`) |`[cef][device_custom_string_13][value]`
|
|
276
|
+
|`deviceCustomString13Label` (`cs13Label`) |`[cef][device_custom_string_13][label]`
|
|
277
|
+
|`deviceCustomString14` (`cs14`) |`[cef][device_custom_string_14][value]`
|
|
278
|
+
|`deviceCustomString14Label` (`cs14Label`) |`[cef][device_custom_string_14][label]`
|
|
279
|
+
|`deviceCustomString15` (`cs15`) |`[cef][device_custom_string_15][value]`
|
|
280
|
+
|`deviceCustomString15Label` (`cs15Label`) |`[cef][device_custom_string_15][label]`
|
|
281
|
+
|`deviceDirection` |`[network][direction]`
|
|
282
|
+
.2+|`deviceDnsDomain` |`[observer][registered_domain]`
|
|
283
|
+
|
|
284
|
+
{cef-plugin-config-condition} `device => observer`.
|
|
285
|
+
|`[host][registered_domain]`
|
|
286
|
+
|
|
287
|
+
{cef-plugin-config-condition} `device => host`.
|
|
288
|
+
|`deviceEventCategory` (`cat`) |`[cef][category]`
|
|
289
|
+
.2+|`deviceExternalId` |`[observer][name]`
|
|
290
|
+
|
|
291
|
+
{cef-plugin-config-condition} `device => observer`.
|
|
292
|
+
|`[host][id]`
|
|
293
|
+
|
|
294
|
+
{cef-plugin-config-condition} `device => host`.
|
|
295
|
+
|`deviceFacility` |`[log][syslog][facility][code]`
|
|
296
|
+
.2+|`deviceHostName` (`dvchost`) |`[observer][hostname]`
|
|
297
|
+
|
|
298
|
+
{cef-plugin-config-condition} `device => observer`.
|
|
299
|
+
|`[host][name]`
|
|
300
|
+
|
|
301
|
+
{cef-plugin-config-condition} `device => host`.
|
|
302
|
+
|`deviceInboundInterface` |`[observer][ingress][interface][name]`
|
|
303
|
+
.2+|`deviceMacAddress` (`dvcmac`) |`[observer][mac]`
|
|
304
|
+
|
|
305
|
+
{cef-plugin-config-condition} `device => observer`.
|
|
306
|
+
|`[host][mac]`
|
|
307
|
+
|
|
308
|
+
{cef-plugin-config-condition} `device => host`.
|
|
309
|
+
|`deviceNtDomain` |`[cef][nt_domain]`
|
|
310
|
+
|`deviceOutboundInterface` |`[observer][egress][interface][name]`
|
|
311
|
+
|`devicePayloadId` |`[cef][payload_id]`
|
|
312
|
+
|`deviceProcessId` (`dvcpid`) |`[process][pid]`
|
|
313
|
+
|`deviceProcessName` |`[process][name]`
|
|
314
|
+
|`deviceReceiptTime` (`rt`) |`@timestamp`
|
|
315
|
+
|
|
316
|
+
{cef-normalize-timestamp}
|
|
317
|
+
|`deviceTimeZone` (`dtz`) |`[event][timezone]`
|
|
318
|
+
|`deviceTranslatedAddress` |`[host][nat][ip]`
|
|
319
|
+
|`deviceTranslatedZoneExternalID` |`[cef][translated_zone][external_id]`
|
|
320
|
+
|`deviceTranslatedZoneURI` |`[cef][translated_zone][uri]`
|
|
321
|
+
|`deviceVersion` |`[observer][version]`
|
|
322
|
+
|`deviceZoneExternalID` |`[cef][zone][external_id]`
|
|
323
|
+
|`deviceZoneURI` |`[cef][zone][uri]`
|
|
324
|
+
|`endTime` (`end`) |`[event][end]`
|
|
325
|
+
|
|
326
|
+
{cef-normalize-timestamp}
|
|
327
|
+
|`eventId` |`[event][id]`
|
|
328
|
+
|`eventOutcome` (`outcome`) |`[event][outcome]`
|
|
329
|
+
|`externalId` |`[cef][external_id]`
|
|
330
|
+
|`fileCreateTime` |`[file][created]`
|
|
331
|
+
|`fileHash` |`[file][hash]`
|
|
332
|
+
|`fileId` |`[file][inode]`
|
|
333
|
+
|`fileModificationTime` |`[file][mtime]`
|
|
334
|
+
|
|
335
|
+
{cef-normalize-timestamp}
|
|
336
|
+
|`fileName` (`fname`) |`[file][name]`
|
|
337
|
+
|`filePath` |`[file][path]`
|
|
338
|
+
|`filePermission` |`[file][group]`
|
|
339
|
+
|`fileSize` (`fsize`) |`[file][size]`
|
|
340
|
+
|`fileType` |`[file][extension]`
|
|
341
|
+
|`managerReceiptTime` (`mrt`) |`[event][ingested]`
|
|
342
|
+
|
|
343
|
+
{cef-normalize-timestamp}
|
|
344
|
+
|`message` (`msg`) |`[message]`
|
|
345
|
+
|`oldFileCreateTime` |`[cef][old_file][created]`
|
|
346
|
+
|
|
347
|
+
{cef-normalize-timestamp}
|
|
348
|
+
|`oldFileHash` |`[cef][old_file][hash]`
|
|
349
|
+
|`oldFileId` |`[cef][old_file][inode]`
|
|
350
|
+
|`oldFileModificationTime` |`[cef][old_file][mtime]`
|
|
351
|
+
|
|
352
|
+
{cef-normalize-timestamp}
|
|
353
|
+
|`oldFileName` |`[cef][old_file][name]`
|
|
354
|
+
|`oldFilePath` |`[cef][old_file][path]`
|
|
355
|
+
|`oldFilePermission` |`[cef][old_file][group]`
|
|
356
|
+
|`oldFileSize` |`[cef][old_file][size]`
|
|
357
|
+
|`oldFileType` |`[cef][old_file][extension]`
|
|
358
|
+
|`rawEvent` |`[event][original]`
|
|
359
|
+
|`Reason` (`reason`) |`[event][reason]`
|
|
360
|
+
|`requestClientApplication` |`[user_agent][original]`
|
|
361
|
+
|`requestContext` |`[http][request][referrer]`
|
|
362
|
+
|`requestCookies` |`[cef][request][cookies]`
|
|
363
|
+
|`requestMethod` |`[http][request][method]`
|
|
364
|
+
|`requestUrl` (`request`) |`[url][original]`
|
|
365
|
+
|`sourceAddress` (`src`) |`[source][ip]`
|
|
366
|
+
|`sourceDnsDomain` |`[source][registered_domain]`
|
|
367
|
+
|
|
368
|
+
{cef-ambiguous-higher}
|
|
369
|
+
|`sourceGeoLatitude` (`slat`) |`[source][geo][location][lat]`
|
|
370
|
+
|`sourceGeoLongitude` (`slong`) |`[source][geo][location][lon]`
|
|
371
|
+
|`sourceHostName` (`shost`) |`[source][domain]`
|
|
372
|
+
|`sourceMacAddress` (`smac`) |`[source][mac]`
|
|
373
|
+
|`sourceNtDomain` (`sntdom`) |`[source][registered_domain]`
|
|
374
|
+
|
|
375
|
+
{cef-ambiguous-lower}
|
|
376
|
+
|`sourcePort` (`spt`) |`[source][port]`
|
|
377
|
+
|`sourceProcessId` (`spid`) |`[source][process][pid]`
|
|
378
|
+
|`sourceProcessName` (`sproc`) |`[source][process][name]`
|
|
379
|
+
|`sourceServiceName` |`[source][service][name]`
|
|
380
|
+
|`sourceTranslatedAddress` |`[source][nat][ip]`
|
|
381
|
+
|`sourceTranslatedPort` |`[source][nat][port]`
|
|
382
|
+
|`sourceTranslatedZoneExternalID` |`[cef][source][translated_zone][external_id]`
|
|
383
|
+
|`sourceTranslatedZoneURI` |`[cef][source][translated_zone][uri]`
|
|
384
|
+
|`sourceUserId` (`suid`) |`[source][user][id]`
|
|
385
|
+
|`sourceUserName` (`suser`) |`[source][user][name]`
|
|
386
|
+
|`sourceUserPrivileges` (`spriv`) |`[source][user][group][name]`
|
|
387
|
+
|`sourceZoneExternalID` |`[cef][source][zone][external_id]`
|
|
388
|
+
|`sourceZoneURI` |`[cef][source][zone][uri]`
|
|
389
|
+
|`startTime` (`start`) |`[event][start]`
|
|
390
|
+
|
|
391
|
+
{cef-normalize-timestamp}
|
|
392
|
+
|`transportProtocol` (`proto`) |`[network][transport]`
|
|
393
|
+
|`type` |`[cef][type]`
|
|
394
|
+
|=======================================================================================================================
|
|
395
|
+
|
|
396
|
+
|
|
30
397
|
[id="plugins-{type}s-{plugin}-options"]
|
|
31
398
|
==== Cef Codec Configuration Options
|
|
32
399
|
|
|
33
400
|
[cols="<,<,<",options="header",]
|
|
34
401
|
|=======================================================================
|
|
35
402
|
|Setting |Input type|Required
|
|
403
|
+
| <<plugins-{type}s-{plugin}-default_timezone>> |<<string,string>>|No
|
|
36
404
|
| <<plugins-{type}s-{plugin}-delimiter>> |<<string,string>>|No
|
|
405
|
+
| <<plugins-{type}s-{plugin}-device>> |<<string,string>>|No
|
|
406
|
+
| <<plugins-{type}s-{plugin}-ecs_compatibility>> |<<string,string>>|No
|
|
37
407
|
| <<plugins-{type}s-{plugin}-fields>> |<<array,array>>|No
|
|
408
|
+
| <<plugins-{type}s-{plugin}-locale>> |<<string,string>>|No
|
|
38
409
|
| <<plugins-{type}s-{plugin}-name>> |<<string,string>>|No
|
|
39
410
|
| <<plugins-{type}s-{plugin}-product>> |<<string,string>>|No
|
|
40
411
|
| <<plugins-{type}s-{plugin}-reverse_mapping>> |<<boolean,boolean>>|No
|
|
@@ -46,6 +417,21 @@ produce an event with the payload as the 'message' field and a '_cefparsefailure
|
|
|
46
417
|
|
|
47
418
|
|
|
48
419
|
|
|
420
|
+
[id="plugins-{type}s-{plugin}-default_timezone"]
|
|
421
|
+
===== `default_timezone`
|
|
422
|
+
|
|
423
|
+
* Value type is <<string,string>>
|
|
424
|
+
* Supported values are:
|
|
425
|
+
** https://en.wikipedia.org/wiki/List_of_tz_database_time_zones[Timezone names] (such as `Europe/Moscow`, `America/Argentina/Buenos_Aires`)
|
|
426
|
+
** UTC Offsets (such as `-08:00`, `+03:00`)
|
|
427
|
+
* The default value is your system time zone
|
|
428
|
+
* This option has no effect when _encoding_.
|
|
429
|
+
|
|
430
|
+
When parsing timestamp fields in ECS mode and encountering timestamps that
|
|
431
|
+
do not contain UTC-offset information, the `deviceTimeZone` (`dtz`) field
|
|
432
|
+
from the CEF payload is used to interpret the given time. If the event does
|
|
433
|
+
not include timezone information, this `default_timezone` is used instead.
|
|
434
|
+
|
|
49
435
|
[id="plugins-{type}s-{plugin}-delimiter"]
|
|
50
436
|
===== `delimiter`
|
|
51
437
|
|
|
@@ -69,21 +455,69 @@ This setting allows the following character sequences to have special meaning:
|
|
|
69
455
|
* `\\r` (backslash "r") - means carriage return (ASCII 0x0D)
|
|
70
456
|
* `\\n` (backslash "n") - means newline (ASCII 0x0A)
|
|
71
457
|
|
|
458
|
+
[id="plugins-{type}s-{plugin}-device"]
|
|
459
|
+
===== `device`
|
|
460
|
+
|
|
461
|
+
* Value type is <<string,string>>
|
|
462
|
+
* Supported values are:
|
|
463
|
+
** `observer`: indicates that device-specific fields represent the device used to _observe_ the event.
|
|
464
|
+
** `host`: indicates that device-specific fields represent the device on which the event _occurred_.
|
|
465
|
+
* The default value for this setting is `observer`.
|
|
466
|
+
* Option has no effect when <<plugins-{type}s-{plugin}-ecs_compatibility,`ecs_compatibility => disabled`>>.
|
|
467
|
+
* Option has no effect when _encoding_
|
|
468
|
+
|
|
469
|
+
Defines a set of device-specific CEF fields as either representing the device on which an
|
|
470
|
+
event _occurred_, or merely the device from which the event was _observed_.
|
|
471
|
+
This causes the relevant fields to be routed to either the `host` or the `observer`
|
|
472
|
+
top-level groupings.
|
|
473
|
+
|
|
474
|
+
If the codec handles data from a variety of sources, the ECS recommendation is to use `observer`.
|
|
475
|
+
|
|
476
|
+
[id="plugins-{type}s-{plugin}-ecs_compatibility"]
|
|
477
|
+
===== `ecs_compatibility`
|
|
478
|
+
|
|
479
|
+
* Value type is <<string,string>>
|
|
480
|
+
* Supported values are:
|
|
481
|
+
** `disabled`: uses CEF-defined field names in the event (e.g., `bytesIn`, `sourceAddress`)
|
|
482
|
+
** `v1`: supports ECS-compatible event fields (e.g., `[source][bytes]`, `[source][ip]`)
|
|
483
|
+
* Default value depends on which version of Logstash is running:
|
|
484
|
+
** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
|
|
485
|
+
** Otherwise, the default value is `disabled`.
|
|
486
|
+
|
|
487
|
+
Controls this plugin's compatibility with the {ecs-ref}[Elastic Common Schema (ECS)].
|
|
488
|
+
|
|
72
489
|
[id="plugins-{type}s-{plugin}-fields"]
|
|
73
490
|
===== `fields`
|
|
74
491
|
|
|
75
492
|
* Value type is <<array,array>>
|
|
76
493
|
* Default value is `[]`
|
|
494
|
+
* Option has no effect when _decoding_
|
|
495
|
+
|
|
496
|
+
When this codec is used in an Output Plugin, a list of fields can be provided to be included in CEF extensions part as key/value pairs.
|
|
497
|
+
|
|
498
|
+
[id="plugins-{type}s-{plugin}-locale"]
|
|
499
|
+
===== `locale`
|
|
77
500
|
|
|
78
|
-
|
|
501
|
+
* Value type is <<string,string>>
|
|
502
|
+
* Supported values are:
|
|
503
|
+
** Abbreviated language_COUNTRY format (e.g., `en_GB`, `pt_BR`)
|
|
504
|
+
** Valid https://tools.ietf.org/html/bcp47[IETF BCP 47] language tag (e.g., `zh-cmn-Hans-CN`)
|
|
505
|
+
* The default value is your system locale
|
|
506
|
+
* Option has no effect when _encoding_
|
|
507
|
+
|
|
508
|
+
When parsing timestamp fields in ECS mode and encountering timestamps in
|
|
509
|
+
a localized format, this `locale` is used to interpret locale-specific strings
|
|
510
|
+
such as month abbreviations.
|
|
79
511
|
|
|
80
512
|
[id="plugins-{type}s-{plugin}-name"]
|
|
81
513
|
===== `name`
|
|
82
514
|
|
|
83
515
|
* Value type is <<string,string>>
|
|
84
516
|
* Default value is `"Logstash"`
|
|
517
|
+
* Option has no effect when _decoding_
|
|
85
518
|
|
|
86
|
-
|
|
519
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
520
|
+
value of the name field in the CEF header. The new value can include `%{foo}` strings
|
|
87
521
|
to help you build a new value from other parts of the event.
|
|
88
522
|
|
|
89
523
|
[id="plugins-{type}s-{plugin}-product"]
|
|
@@ -91,8 +525,10 @@ to help you build a new value from other parts of the event.
|
|
|
91
525
|
|
|
92
526
|
* Value type is <<string,string>>
|
|
93
527
|
* Default value is `"Logstash"`
|
|
528
|
+
* Option has no effect when _decoding_
|
|
94
529
|
|
|
95
|
-
|
|
530
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
531
|
+
value of the device product field in CEF header. The new value can include `%{foo}` strings
|
|
96
532
|
to help you build a new value from other parts of the event.
|
|
97
533
|
|
|
98
534
|
|
|
@@ -101,6 +537,7 @@ to help you build a new value from other parts of the event.
|
|
|
101
537
|
|
|
102
538
|
* Value type is <<boolean,boolean>>
|
|
103
539
|
* Default value is `false`
|
|
540
|
+
* Option has no effect when _decoding_
|
|
104
541
|
|
|
105
542
|
Set to true to adhere to the specifications and encode using the CEF key name (short name) for the CEF field names.
|
|
106
543
|
|
|
@@ -109,8 +546,10 @@ Set to true to adhere to the specifications and encode using the CEF key name (s
|
|
|
109
546
|
|
|
110
547
|
* Value type is <<string,string>>
|
|
111
548
|
* Default value is `"6"`
|
|
549
|
+
* Option has no effect when _decoding_
|
|
112
550
|
|
|
113
|
-
|
|
551
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
552
|
+
value of the severity field in CEF header. The new value can include `%{foo}` strings
|
|
114
553
|
to help you build a new value from other parts of the event.
|
|
115
554
|
|
|
116
555
|
Defined as field of type string to allow sprintf. The value will be validated
|
|
@@ -122,8 +561,10 @@ All invalid values will be mapped to the default of 6.
|
|
|
122
561
|
|
|
123
562
|
* Value type is <<string,string>>
|
|
124
563
|
* Default value is `"Logstash"`
|
|
564
|
+
* Option has no effect when _decoding_
|
|
125
565
|
|
|
126
|
-
|
|
566
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
567
|
+
value of the signature ID field in CEF header. The new value can include `%{foo}` strings
|
|
127
568
|
to help you build a new value from other parts of the event.
|
|
128
569
|
|
|
129
570
|
[id="plugins-{type}s-{plugin}-vendor"]
|
|
@@ -131,8 +572,10 @@ to help you build a new value from other parts of the event.
|
|
|
131
572
|
|
|
132
573
|
* Value type is <<string,string>>
|
|
133
574
|
* Default value is `"Elasticsearch"`
|
|
575
|
+
* Option has no effect when _decoding_
|
|
134
576
|
|
|
135
|
-
|
|
577
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
578
|
+
value of the device vendor field in CEF header. The new value can include `%{foo}` strings
|
|
136
579
|
to help you build a new value from other parts of the event.
|
|
137
580
|
|
|
138
581
|
[id="plugins-{type}s-{plugin}-version"]
|
|
@@ -140,8 +583,8 @@ to help you build a new value from other parts of the event.
|
|
|
140
583
|
|
|
141
584
|
* Value type is <<string,string>>
|
|
142
585
|
* Default value is `"1.0"`
|
|
586
|
+
* Option has no effect when _decoding_
|
|
143
587
|
|
|
144
|
-
|
|
588
|
+
When this codec is used in an Output Plugin, this option can be used to specify the
|
|
589
|
+
value of the device version field in CEF header. The new value can include `%{foo}` strings
|
|
145
590
|
to help you build a new value from other parts of the event.
|
|
146
|
-
|
|
147
|
-
|