logstash-input-sdee 0.6.9

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 2e5c47c9f0d817a1e1d1fbb697e896ffc7fe0ef4
+  data.tar.gz: 6df5bb5d6942b4554216a032379a4ecf00989573
+SHA512:
+  metadata.gz: 5ef8b0e84db55e36509d00af50cd6134b642ee4ee051c3a07f102cdf86af3dd74fcc217d5877e5451e6251b76a8e1ccb41d14135860955bd7c7fab174540f082
+  data.tar.gz: de12fddd14935ebd9b8e377cd64b605f544ec5b9f8dc354999d7414b1496e8c4a9f76c8116db1a00691723a6866629146d3b3b31c43a04c39e5b37665794488d

data/CHANGELOG.md

@@ -0,0 +1,12 @@
+* 0.6.9
+  - making it ready for RubyGems
+* 0.4.0
+  - msec timestamp precision
+  - SSL notes
+  - config example
+  - multiple hosts support
+  - tempfiles generated per host
+* 0.3.0
+  - pre Alpha release
+* 0.1.0
+  - Initial release

data/CONTRIBUTORS

@@ -0,0 +1,10 @@
+The following is a list of people who have contributed ideas, code, bug
+reports, or in general have helped logstash-input-sdee along its way.
+
+Contributors:
+* Me (roootik@gmail.com)
+
+Note: If you've sent me patches, bug reports, or otherwise contributed to
+logstash, and you aren't on the list above and want to be, please let me know
+and I'll make sure you're here. Contributions from folks like you are what make
+open source awesome.

data/Gemfile

@@ -0,0 +1,2 @@
+source 'https://rubygems.org'
+gemspec

data/LICENSE

@@ -0,0 +1,13 @@
+Copyright (c) 2014–2016 rootik <roootik@gmail.com>
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/README.md

@@ -0,0 +1,190 @@
+# Logstash Cisco SDEE/CIDEE input plugin
+
+This plugin is based off [logstash-input-http_poller](https://github.com/logstash-plugins/logstash-input-http_poller) by @maximede.
+
+This [Logstash](https://github.com/elasticsearch/logstash) input plugin allows you to call a Cisco SDEE/CIDEE HTTP API, decode the output of it into event(s), and send them on their merry way.
+
+The idea behind this plugins came from a need to gather events from Cisco security devices and feed them to ELK stack.
+
+This plugin is tested on:
+* Hardware: Cisco ASA 5585-X IPS SSP-10
+* IPS Version: 7.3(2)E4
+* logstash 2.0.0-beta1
+* Java JRE 1.8.0-60
+
+It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
+
+## TODO
+- Add request options like minThreatRating and maxThreatRating, virtualSensors, errorSeverities, includeStatusCategories, excludeStatusCategories and so on.
+- Gather sensor status and health events
+
+## Config Example
+
+For config examples see `sdee.conf` in `examples` in this repo.
+
+## SSL notes
+
+You need to import host SSL certificate in Java trust store to be able to connect to Cisco IPS device.
+I have had no luck with `truststore` nor `ssl_certificate_validation` configuration options with Java 1.8.0-60 like mentioned in [logstash-mixin-http_client](https://github.com/logstash-plugins/logstash-mixin-http_client) documentation,
+so I'm using system trustStore - `$JAVA_HOME/lib/security/cacerts`. Note, default Java trustStore password is `changeit`.
+Take the following steps:
+
+* Get server certificate from IPS device:
+```sh
+echo | openssl s_client -connect ciscoips:443 2>&1 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > cert.pem
+```
+
+* Import it into Java cacerts:
+```sh
+$JAVA_HOME/bin/keytool -keystore $JAVA_HOME/lib/security/cacerts -importcert -alias ciscoips -file cert.pem
+```
+
+* Verify if import was successful:
+```sh
+$JAVA_HOME/bin/keytool -keystore $JAVA_HOME/lib/security/cacerts -list
+```
+
+* Modify your logstash input config for SSL connection:
+```ruby
+input {
+  sdee { 
+    interval => 60  
+    http => { 
+      truststore_password => "changeit" 
+      url => "https://10.0.2.1"  
+      auth => {
+        user => "cisco"
+        password => "p@ssw0rd"
+      }
+    }
+  }
+}
+```
+
+* Test logstash
+
+## Documentation
+
+Logstash provides infrastructure to automatically generate documentation for this plugin. We use the asciidoc format to write documentation so any comments in the source code will be first converted into asciidoc and then into html. All plugin documentation are placed under one [central location](http://www.elasticsearch.org/guide/en/logstash/current/).
+
+- For formatting code or config example, you can use the asciidoc `[source,ruby]` directive
+- For more asciidoc formatting tips, see the excellent reference here https://github.com/elasticsearch/docs#asciidoc-guide
+
+## Need Help?
+
+Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
+
+## Developing
+
+### 1. Plugin Developement and Testing
+
+#### Code
+- To get started, you'll need JRuby with the Bundler gem installed.
+
+- Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
+
+- Install dependencies
+```sh
+bundle install
+```
+
+#### Test
+
+- Update your dependencies
+
+```sh
+bundle install
+```
+
+- Run tests
+
+```sh
+bundle exec rspec
+```
+
+### 2. Running your unpublished Plugin in Logstash
+
+#### 2.1 Run in a local Logstash clone
+
+- Edit Logstash `Gemfile` and add the local plugin path, for example:
+```ruby
+gem "logstash-input-sdee", :path => "/your/local/logstash-input-sdee"
+```
+- Install plugin
+```sh
+bin/plugin install --no-verify
+```
+- Run Logstash with your plugin
+```sh
+bin/logstash -e 'input {sdee { interval => 60 http => { url => "http://ciscoips" auth => {user => "cisco" password => "p@ssw0rd"}} session_file => "/tmp/session.db" }}'
+```
+At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
+
+Enable signature 2000 (ICMP Echo Reply) on your Cisco IPS device and ping some host on a network, monitored by IPS.
+You should see output like this:
+
+```ruby
+{
+             "@timestamp" => "2015-09-21T12:21:26.000Z",
+               "timezone" => "EEST",
+              "tz_offset" => "180",
+               "event_id" => "6824288790867",
+               "severity" => "informational",
+                 "vendor" => "Cisco",
+                "host_id" => "sensor1",
+               "app_name" => "sensorApp",
+        "app_instance_id" => "26957",
+            "description" => "ICMP Echo Reply",
+                 "sig_id" => "2000",
+            "sig_version" => "S666",
+               "sig_type" => "other",
+            "sig_created" => "20001127",
+              "subsig_id" => "0",
+            "sig_details" => "ICMP Echo Reply",
+        "interface_group" => "vs0",
+                   "vlan" => "0",
+          "attacker_addr" => "10.0.0.1",
+      "attacker_locality" => "OUT",
+            "target_addr" => "10.0.1.1",
+       "target_os_source" => "learned",
+         "target_os_type" => "windows-nt-2k-xp",
+    "target_os_relevance" => "relevant",
+          "alert_details" => "InterfaceAttributes:  context='single_vf' physical='Unknown' backplane='PortChannel0/0' ; ",
+            "risk_rating" => "35",
+            "risk_target" => "medium",
+          "risk_attacker" => "relevant",
+          "threat_rating" => "35",
+              "interface" => "PortChannel0/0",
+                   "host" => "10.0.2.1",
+                   "tags" => "SDEE",
+                "message" => "IdsAlert: 'ICMP Echo Reply' Attacker: '10.0.0.1' Target: '10.0.1.1' SigId: '2000'",
+               "@version" => "1"
+}
+```
+
+#### 2.2 Run in an installed Logstash
+
+You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
+
+- Build your plugin gem
+```sh
+gem build logstash-input-sdee.gemspec
+```
+- Install the plugin from the Logstash home
+```sh
+bin/plugin install /your/local/plugin/logstash-input-sdee.gem
+```
+- Start Logstash and proceed to test the plugin
+
+## Contributing
+
+All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
+
+Programming is not a required skill. Whatever you've seen about open source and maintainers or community members  saying "send patches or die" - you will not see that here.
+
+It is more important to the community that you are able to contribute.
+
+For more information about contributing, see the [CONTRIBUTING](https://github.com/elasticsearch/logstash/blob/master/CONTRIBUTING.md) file.
+
+## Reference
+[Cisco Intrusion Detection Event Exchange (CIDEE) Specification](http://www.cisco.com/c/en/us/td/docs/security/ips/specs/CIDEE_Specification.html)

data/examples/patterns/cisco

@@ -0,0 +1,79 @@
+#== Cisco ASA ==
+HOSTNAME \b(?:[_0-9A-Za-z][_0-9A-Za-z-]{0,62})(?:\.(?:[_0-9A-Za-z][_0-9A-Za-z-]{0,62}))*(\.?|\b)
+CTIMESTAMP %{YEAR}-%{MONTHNUM2}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?
+CISCO_TAGGED %{CTIMESTAMP:ctimestamp}( %{SYSLOGHOST:sysloghost})? %{CISCO_TAG:ciscotag}:
+CISCO_TAG %[A-Z0-9]+-%{INT:cisco_severity}-(?:[A-Z0-9_]+)|WLC[0-9]+
+# Common Particles
+CISCO_ASA_ACTION Built|Teardown|Deny|Denied|denied|requested|permitted|received|denied by ACL|discarded|est-allowed|Dropping|created|deleted|SENDING|RECEIVED|monitored|dropped
+CISCO_ASA_REASON Duplicate TCP SYN|TCP Reset\-O|Failed to locate egress interface|Invalid transport field|No matching connection|DNS Response|DNS Query|(?:%{WORD}\s*)*
+CISCO_ASA_DIRECTION Inbound|inbound|Outbound|outbound
+CISCO_ASA_INTERVAL first hit|%{INT}-second interval
+CISCO_ASA_XLATE_TYPE static|dynamic
+# ASA-2-106001
+CISCOASA106001 %{CISCO_ASA_DIRECTION:direction} %{WORD:protocol} connection %{CISCO_ASA_ACTION:action} from %{IP:src_ip}/%{INT:src_port} to %{IP:dst_ip}/%{INT:dst_port} flags %{GREEDYDATA:tcp_flags} on interface %{GREEDYDATA:interface}
+# ASA-2-106006, ASA-2-106007, ASA-2-106010
+CISCOASA106006_106007_106010 %{CISCO_ASA_ACTION:action} %{CISCO_ASA_DIRECTION:direction} %{WORD:protocol} (?:from|src) %{IP:src_ip}/%{INT:src_port}(\(%{DATA:src_fwuser}\))? (?:to|dst) %{IP:dst_ip}/%{INT:dst_port}(\(%{DATA:dst_fwuser}\))? (?:on interface %{DATA:interface}|due to %{CISCO_ASA_REASON:reason})
+# ASA-3-106014
+CISCOASA106014 %{CISCO_ASA_ACTION:action} %{CISCO_ASA_DIRECTION:direction} %{WORD:protocol} src %{DATA:src_interface}:%{IP:src_ip}(\(%{DATA:src_fwuser}\))? dst %{DATA:dst_interface}:%{IP:dst_ip}(\(%{DATA:dst_fwuser}\))? \(type %{INT:icmp_type}, code %{INT:icmp_code}\)
+# ASA-6-106015
+CISCOASA106015 %{CISCO_ASA_ACTION:action} %{WORD:protocol} \(%{DATA:policy_id}\) from %{IPORHOST:src_ip}/%{INT:src_port} to %{IPORHOST:dst_ip}/%{INT:dst_port} flags %{DATA:tcp_flags}  on interface %{GREEDYDATA:interface}
+# ASA-1-106021
+CISCOASA106021 %{CISCO_ASA_ACTION:action} %{WORD:protocol} reverse path check from %{IP:src_ip} to %{IP:dst_ip} on interface %{GREEDYDATA:interface}
+# ASA-4-106023
+CISCOASA106023 %{CISCO_ASA_ACTION:action} %{WORD:protocol} src %{DATA:src_interface}:%{IPORHOST:src_ip}(/%{INT:src_port})?(\(%{DATA:src_fwuser}\))? dst %{DATA:dst_interface}:%{IPORHOST:dst_ip}(/%{INT:dst_port})?(\(%{DATA:dst_fwuser}\))?( \(type %{INT:icmp_type}, code %{INT:icmp_code}\))? by access-group %{DATA:policy_id} \[%{DATA:hashcode1}, %{DATA:hashcode2}\]
+# ASA-5-106100
+CISCOASA106100 access-list %{WORD:policy_id} %{CISCO_ASA_ACTION:action} %{WORD:protocol} %{DATA:src_interface}/%{IP:src_ip}\(%{INT:src_port}\)(\(%{DATA:src_fwuser}\))? -> %{DATA:dst_interface}/%{IP:dst_ip}\(%{INT:dst_port}\)(\(%{DATA:src_fwuser}\))? hit-cnt %{INT:hit_count} %{CISCO_ASA_INTERVAL:interval} \[%{DATA:hashcode1}, %{DATA:hashcode2}\]
+# ASA-6-110002
+CISCOASA110002 %{CISCO_ASA_REASON:reason} for %{WORD:protocol} from %{DATA:src_interface}:%{IPORHOST:src_ip}/%{INT:src_port} to %{IP:dst_ip}/%{INT:dst_port}
+# ASA-5-111008
+CISCOASA111008 User \'%{DATA:src_fwuser}\' executed the \'%{GREEDYDATA:cmd}\' command\.
+# ASA-7-111009
+CISCOASA111009 User \'%{DATA:src_fwuser}\' executed cmd: %{GREEDYDATA:cmd}
+# ASA-5-111010
+CISCOASA111010 User \'%{DATA:src_fwuser}\', running \'CLI\' from IP %{IPORHOST:src_ip}, executed \'%{GREEDYDATA:cmd}\'
+# ASA-6-302010
+CISCOASA302010 %{INT:connection_count} in use, %{INT:connection_count_max} most used
+# ASA-6-302013, ASA-6-302014, ASA-6-302015, ASA-6-302016
+CISCOASA302013_302014_302015_302016 %{CISCO_ASA_ACTION:action}(?: %{CISCO_ASA_DIRECTION:direction})? %{WORD:protocol} connection %{INT:connection_id} for %{DATA:src_interface}:%{IPORHOST:src_ip}/%{INT:src_port}( \(%{IPORHOST:src_mapped_ip}/%{INT:src_mapped_port}\))?(\(%{DATA:src_fwuser}\))? to %{DATA:dst_interface}:%{IPORHOST:dst_ip}/%{INT:dst_port}( \(%{IPORHOST:dst_mapped_ip}/%{INT:dst_mapped_port}\))?(\(%{DATA:dst_fwuser}\))?( duration %{TIME:duration} bytes %{INT:bytes})?(?: %{CISCO_ASA_REASON:reason})?( \(%{DATA:user}\))?
+# ASA-6-302020, ASA-6-302021
+CISCOASA302020_302021 %{CISCO_ASA_ACTION:action}(?: %{CISCO_ASA_DIRECTION:direction})? %{WORD:protocol} connection for faddr %{IPORHOST:dst_ip}/%{INT:icmp_seq_num}(?:\(%{DATA:fwuser}\))? gaddr %{IPORHOST:src_xlated_ip}/%{INT:icmp_code_xlated} laddr %{IPORHOST:src_ip}/%{INT:icmp_code}( \(%{DATA:user}\))?
+# ASA-3-305006
+CISCOASA305006 regular translation creation failed for %{WORD:protocol} src %{DATA:src_interface}:%{IPORHOST:src_ip}(/%{INT:src_port})? dst %{DATA:dst_interface}:%{IPORHOST:dst_ip}(/%{INT:dst_port})?(?: \(type %{INT:icmp_type}, code %{INT:icmp_code}\))?
+# ASA-6-305011
+CISCOASA305011 %{CISCO_ASA_ACTION:action} %{CISCO_ASA_XLATE_TYPE:xlate_type} %{WORD:protocol} translation from %{DATA:src_interface}:%{IPORHOST:src_ip}(/%{INT:src_port})?(\(%{DATA:src_fwuser}\))? to %{DATA:src_xlated_interface}:%{IPORHOST:src_xlated_ip}/%{DATA:src_xlated_port}
+# ASA-5-305013
+CISCOASA305013 Asymmetric NAT rules matched for forward and reverse flows; Connection for %{WORD:protocol} src %{DATA:src_interface}:%{IPORHOST:src_ip}(/%{INT:src_port})? dst %{DATA:dst_interface}:%{IPORHOST:dst_ip}(/%{INT:dst_port})? %{CISCO_ASA_ACTION:action} due to NAT reverse path failure
+# ASA-3-313001, ASA-3-313004, ASA-3-313008
+CISCOASA313001_313004_313008 %{CISCO_ASA_ACTION:action} %{WORD:protocol} type=%{INT:icmp_type}, code=%{INT:icmp_code} from %{IP:src_ip} on interface %{DATA:interface}( to %{IP:dst_ip})?
+# ASA-4-313005
+CISCOASA313005 %{CISCO_ASA_REASON:reason} for %{WORD:protocol} error message: %{WORD:err_protocol} src %{DATA:err_src_interface}:%{IPORHOST:err_src_ip}(\(%{DATA:err_src_fwuser}\))? dst %{DATA:err_dst_interface}:%{IPORHOST:err_dst_ip}(\(%{DATA:err_dst_fwuser}\))? \(type %{INT:err_icmp_type}, code %{INT:err_icmp_code}\) on %{DATA:interface} interface\.  Original IP payload: %{WORD:protocol} src %{IPORHOST:orig_src_ip}/%{INT:orig_src_port}(\(%{DATA:orig_src_fwuser}\))? dst %{IPORHOST:orig_dst_ip}/%{INT:orig_dst_port}(\(%{DATA:orig_dst_fwuser}\))?
+# ASA-4-338004, ASA-4-338008 
+CISCOASA338004_338008 Dynamic Filter %{CISCO_ASA_ACTION:action} blacklisted %{WORD:protocol} traffic from %{DATA:src_interface}:%{IPORHOST:src_ip}(/%{INT:src_port})?( \(%{IPORHOST:src_mapped_ip}/%{INT:src_mapped_port}\))? to %{DATA:dst_interface}:%{IPORHOST:dst_ip}(/%{INT:dst_port})?( \(%{IPORHOST:dst_mapped_ip}/%{INT:dst_mapped_port}\))?, destination %{IPORHOST:blacklisted_ip} resolved from local list: %{IPORHOST:blacklisted_ip}/%{IPORHOST:blacklisted_netmask}, threat-level: %{DATA:threat_level}, category: %{DATA:category}
+# ASA-4-338008 Dynamic Filter %{CISCO_ASA_ACTION:action} blacklisted %{WORD:protocol} traffic from %{DATA:src_interface}:%{IPORHOST:src_ip}(/%{INT:src_port})?( \(%{IPORHOST:src_mapped_ip}/%{INT:src_mapped_port}\))? to %{DATA:dst_interface}:%{IPORHOST:dst_ip}(/%{INT:dst_port})?( \(%{IPORHOST:dst_mapped_ip}/%{INT:dst_mapped_port}\))?, destination %{IPORHOST:blacklisted_ip} resolved from local list: 221.204.186.0/255.255.255.0, threat-level: very-high, category: admin-added
+# ASA-4-402117
+CISCOASA402117 %{WORD:protocol}: Received a non-IPSec packet \(protocol= %{WORD:orig_protocol}\) from %{IP:src_ip} to %{IP:dst_ip}
+# ASA-4-402119
+CISCOASA402119 %{WORD:protocol}: Received an %{WORD:orig_protocol} packet \(SPI= %{DATA:spi}, sequence number= %{DATA:seq_num}\) from %{IP:src_ip} \(user= %{DATA:user}\) to %{IP:dst_ip} that failed anti-replay checking
+# ASA-4-419001
+CISCOASA419001 %{CISCO_ASA_ACTION:action} %{WORD:protocol} packet from %{DATA:src_interface}:%{IP:src_ip}/%{INT:src_port} to %{DATA:dst_interface}:%{IP:dst_ip}/%{INT:dst_port}, reason: %{GREEDYDATA:reason}
+# ASA-4-419002
+CISCOASA419002 %{CISCO_ASA_REASON:reason} from %{DATA:src_interface}:%{IPORHOST:src_ip}/%{INT:src_port} to %{DATA:dst_interface}:%{IPORHOST:dst_ip}/%{INT:dst_port} with different initial sequence number
+# ASA-4-500004
+CISCOASA500004 %{CISCO_ASA_REASON:reason} for protocol=%{WORD:protocol}, from %{IP:src_ip}/%{INT:src_port} to %{IP:dst_ip}/%{INT:dst_port}
+# ASA-6-602303, ASA-6-602304
+CISCOASA602303_602304 %{WORD:protocol}: An %{CISCO_ASA_DIRECTION:direction} %{GREEDYDATA:tunnel_type} SA \(SPI= %{DATA:spi}\) between %{IP:src_ip} and %{IP:dst_ip} \(user= %{DATA:user}\) has been %{CISCO_ASA_ACTION:action}
+# ASA-7-609001, ASA-7-609002
+CISCOASA609001_609002 %{CISCO_ASA_ACTION:action} local-host %{DATA:src_interface}:%{IPORHOST:src_ip}(?: duration %{TIME:duration})?
+# ASA-7-710001, ASA-7-710002, ASA-7-710003, ASA-7-710005, ASA-7-710006
+CISCOASA710001_710002_710003_710005_710006_710007 %{WORD:protocol} (?:request|access|keepalive) %{CISCO_ASA_ACTION:action} from %{IPORHOST:src_ip}/%{INT:src_port} to %{DATA:dst_interface}:%{IPORHOST:dst_ip}/%{DATA:dst_port}
+# ASA-6-713172
+CISCOASA713172 Group = %{GREEDYDATA:group}, IP = %{IP:src_ip}, Automatic NAT Detection Status:\s+Remote end\s*%{DATA:is_remote_natted}\s*behind a NAT device\s+This\s+end\s*%{DATA:is_local_natted}\s*behind a NAT device
+# ASA-7-713236
+CISCOASA713236 IP = %{IPORHOST:src_ip}, IKE_DECODE %{CISCO_ASA_ACTION} Message \(msgid=%{DATA:msgid}\) with payloads : %{GREEDYDATA:payload} total length : %{INT:length}
+# ASA-7-713906
+CISCOASA713906 IKE Receiver: Packet received on %{IPORHOST:dst_ip}:%{INT:dst_port} from %{IPORHOST:src_ip}:%{INT:src_port}
+# ASA-7-715046
+CISCOASA715036_715046_715047_715075 Group = %{GREEDYDATA:group},(?: Username = %{DATA:src_fwuser},)? IP = %{IP:src_ip},%{GREEDYDATA:vpn_action}
+# ASA-4-733100
+CISCOASA733100 \[\s*%{DATA:drop_type}\s*\] drop %{DATA:drop_rate_id} exceeded. Current burst rate is %{INT:drop_rate_current_burst} per second, max configured rate is %{INT:drop_rate_max_burst}; Current average rate is %{INT:drop_rate_current_avg} per second, max configured rate is %{INT:drop_rate_max_avg}; Cumulative total count is %{INT:drop_total_count}
+#== End Cisco ASA ==

data/examples/sdee.conf

@@ -0,0 +1,35 @@
+input {
+
+  sdee {
+    type => "sdee"
+    interval => 60
+    http => {
+      url => "http://ciscoips1"
+      auth => {
+        user => "cisco"
+        password => "p@ssw0rd"
+      }
+    }
+  }
+
+  sdee {
+    type => "sdee"
+    interval => 60
+    http => {
+      url => "https://ciscoips2"
+      # do not forget, you must add your device or CA certificate to Java trustStore. See README.md
+      truststore_password => "changeit"
+      auth => {
+        user => "cisco"
+        password => "p@ssw0rd"
+      }
+    }
+  }
+
+}
+
+output {
+  stdout {
+    codec => rubydebug
+  }
+}

data/lib/logstash/inputs/sdee.rb

@@ -0,0 +1,354 @@
+# encoding: utf-8
+require "logstash/inputs/base"
+require "logstash/namespace"
+require "logstash/plugin_mixins/http_client"
+
+require "yaml"
+require "uri"
+require "pathname"
+require "time"
+require "rexml/document"
+require "socket" # for Socket.gethostname
+require "manticore"
+include REXML
+
+class LogStash::Inputs::SDEE < LogStash::Inputs::Base
+  include LogStash::PluginMixins::HttpClient
+
+  config_name "sdee"
+  # Do we really need a codec?
+  # default :codec, "plain"
+
+  # A Hash of urls in this format : "name" => "url"
+  # The name and the url will be passed in the outputed event
+  #
+  config :http, :validate => :hash, :required => true
+
+  # How often  (in seconds) the urls will be called
+  config :interval, :validate => :number, :required => true
+
+  # If you'd like to work with the request/response metadata
+  # Set this value to the name of the field you'd like to store a nested
+  # hash of metadata.
+  config :metadata_target, :validate => :string, :default => '@metadata'
+  #, :default => '@metadata'
+
+  # A path to store tempfile with SDEE SubscriptionID and SessionID
+  config :session_path, :validate => :string, :default => '/tmp'
+
+  public
+  def register
+    @host = Socket.gethostname.force_encoding(Encoding::UTF_8)
+    @session_file = Pathname.new(@session_path) + "temp#{URI.parse(@http["url"]).host}.db"
+    @remaining = 0
+    setup_request!(@http)
+
+    @logger.info("Registering SDEE Input", :type => @type,
+                 :http => @http, :interval => @interval, :timeout => @timeout)
+
+    #subscribe to SDEE and store session data
+    @session = subscribe(@request)
+
+    newurl = URI.join(@http["url"], "cgi-bin/sdee-server?subscriptionId=#{@session[:subscriptionid]}&confirm=yes&maxNbrOfEvents=150&timeout=5&sessionId=#{@session[:sessionid]}")
+
+    @request[1] = newurl.to_s
+    rescue StandardError, java.lang.Exception => e
+     @logger.error? && @logger.error("SDEE subscription error!",
+                                    :exception => e,
+                                    :exception_message => e.message,
+                                    :exeption_backtrace => e.backtrace
+     )
+
+  end
+
+  private
+  def subscribe(request)
+    # recover from ungraceful shutdown first
+    if @session_file.exist?
+      yaml = YAML.load_file(@session_file)
+      if (defined?(yaml) && yaml != nil)
+        unsubscribe(request,yaml)
+      end
+    end
+
+    url = URI.join(@http["url"], "cgi-bin/sdee-server?action=open&evIdsAlert&force=yes")
+    request[1] = url.to_s
+    session = Hash.new
+    method, *request_opts = request
+    client.async.send(method, *request_opts).
+      on_success {|response| session = handle_subscription(request, response)}.
+      on_failure {|exception| http_error(request, exception)}
+    client.execute!
+    raise LogStash::ConfigurationError, "SDEE subscription Error! Check your configuration for host url,user,password." unless session
+    session
+  end
+
+  private
+  def handle_subscription(request, response)
+    body = response.body
+    xml = REXML::Document.new body.to_s
+    session = Hash.new
+    sessionid = REXML::XPath.first(xml, "//sd:sessionId")
+    subscriptionid = REXML::XPath.first(xml, "//sd:subscriptionId")
+    session[:sessionid] = sessionid.text if sessionid
+    session[:subscriptionid] = subscriptionid.text if subscriptionid
+    File.open(@session_file, 'w') {|f| f.write session.to_yaml}
+    session
+  end
+
+  private
+  def http_error(request, exeption)
+      @logger.error? && @logger.error("Cannot read URL or send the error as an event! Check your configuration for host url,user,password.",
+                                        :request => structure_request(request),
+                                        :exception => exeption.to_s,
+                                        :exception_backtrace => exeption.backtrace
+      )
+  end
+
+  private
+  def unsubscribe(request,session)
+    #unsubscribe and remove session data file
+    url = URI.join(@http["url"], "cgi-bin/sdee-server?action=close&subscriptionId=#{session[:subscriptionid]}&sessionId=#{session[:sessionid]}")
+    request[1] = url.to_s
+    method, *request_opts = request
+    client.async.send(method, *request_opts).
+      on_success {|response| remove_session}.
+      on_failure {|exception| http_error(request, exception)}
+    client.execute!
+  end
+
+  private
+  def remove_session
+    if @session_file.exist?
+      @session_file.unlink
+    end
+  end
+
+  private
+  def setup_request!(http)
+    @request = normalize_request(http)
+  end
+
+  private
+  def normalize_request(http)
+    if http.is_a?(Hash)
+      # The client will expect keys / values
+      spec = Hash[http.clone.map {|k,v| [k.to_sym, v] }] # symbolize keys
+
+      # method and url aren't really part of the options, so we pull them out
+      method = (spec.delete(:method) || :get).to_sym.downcase
+      url = spec.delete(:url)
+
+      # We need these strings to be keywords!
+      spec[:auth] = {user: spec[:auth]["user"], pass: spec[:auth]["password"]} if spec[:auth]
+      res = [method, url, spec]
+    else
+      raise LogStash::ConfigurationError, "Invalid request spec: '#{http}', expected a Hash!"
+    end
+
+    validate_request!(http, res)
+    res
+  end
+
+  private
+  def validate_request!(http, request)
+    method, url, spec = request
+
+    raise LogStash::ConfigurationError, "No URL provided for request! #{http}" unless url
+    if spec && spec[:auth]
+      if !spec[:auth][:user]
+        raise LogStash::ConfigurationError, "Auth was specified, but 'user' was not!"
+      end
+      if !spec[:auth][:pass]
+        raise LogStash::ConfigurationError, "Auth was specified, but 'password' was not!"
+      end
+    end
+
+    request
+  end
+
+  public
+  def run(queue)
+    while true
+      begin
+        Stud.interval(@interval) do
+          begin
+            run_once(queue)
+          end while (@remaining > 0)
+        end
+      rescue EOFError, LogStash::ShutdownSignal
+        break
+      end
+    end
+  end
+
+  public
+  def teardown
+    @logger.debug? && @logger.debug("SDEE shutting down")
+    unsubscribe(@request,@session) rescue nil
+    finished
+  end # def teardown
+
+  private
+  def run_once(queue)
+    request_async(queue, @request)
+    client.execute!
+  end
+
+  private
+  def request_async(queue, request)
+    @logger.debug? && @logger.debug("Fetching URL", :url => request)
+    started = Time.now
+
+    method, *request_opts = request
+    client.async.send(method, *request_opts).
+      on_success {|response| handle_success(queue, request, response, Time.now - started)}.
+      on_failure {|exception| handle_failure(queue, request, exception, Time.now - started)
+    }
+  end
+
+  private
+  def handle_success(queue, request, response, execution_time)
+    decode(response.body).each_pair do |id,decoded|
+      event = LogStash::Event.new(decoded)
+      handle_decoded_event(queue, request, response, event, execution_time)
+    end
+  end
+
+  private
+  def handle_decoded_event(queue, request, response, event, execution_time)
+    apply_metadata(event, request, response, execution_time) if @metadata_target
+    decorate(event)
+    queue << event
+  rescue StandardError, java.lang.Exception => e
+    @logger.error? && @logger.error("Error eventifying response!",
+                                    :exception => e,
+                                    :exception_message => e.message,
+                                    :exeption_backtrace => e.backtrace,
+                                    :url => request,
+                                    :response => response
+    )
+  end
+
+  private
+  # Beware, on old versions of manticore some uncommon failures are not handled
+  def handle_failure(queue, request, exception, execution_time)
+    event = LogStash::Event.new
+    apply_metadata(event, request)
+
+    event.tag("_sdee_failure")
+
+    # This is also in the metadata, but we send it anyone because we want this
+    # persisted by default, whereas metadata isn't. People don't like mysterious errors
+    event["sdee_failure"] = {
+      "request" => structure_request(request),
+      "error" => exception.to_s,
+      "backtrace" => exception.backtrace,
+      "runtime_seconds" => execution_time
+   }
+
+    queue << event
+  rescue StandardError, java.lang.Exception => e
+      @logger.error? && @logger.error("Cannot read URL or send the error as an event!",
+                                      :exception => e,
+                                      :exception_message => e.message,
+                                      :exception_backtrace => e.backtrace,
+                                      :url => request
+      )
+  end
+
+  private
+  def apply_metadata(event, request, response=nil, execution_time=nil)
+    #return unless @metadata_target
+    event[@metadata_target] = event_metadata(request, response, execution_time)
+  end
+
+  private
+  def event_metadata(request, response=nil, execution_time=nil)
+    m = {
+        "host" => @host,
+        "request" => structure_request(request),
+      }
+
+    m["runtime_seconds"] = execution_time
+
+    if response
+      m["code"] = response.code
+      m["response_headers"] = response.headers
+      m["response_message"] = response.message
+      m["times_retried"] = response.times_retried
+    end
+    m
+  end
+
+  private
+  # Turn [method, url, spec] requests into a hash for friendlier logging / ES indexing
+  def structure_request(request)
+    method, url, spec = request
+    # Flatten everything into the 'spec' hash, also stringify any keys to normalize
+    Hash[(spec||{}).merge({
+      "method" => method.to_s,
+      "url" => url,
+    }).map {|k,v| [k.to_s,v] }]
+  end
+
+  private
+  def decode(body)
+    events = Hash.new
+    xml = REXML::Document.new body.to_s
+    err = REXML::XPath.first(xml, "//env:Reason") if REXML::XPath.first(xml, "//env:Fault")
+    err = err.text.to_s if err
+    if err && err == "Subscription does not exist"
+        subscribe(@request)
+    end
+    rem = REXML::XPath.first(xml, "//sd:remaining-events")
+    @remaining = rem.text.to_i if rem
+    # We use own XML parsing to keep things simple to the user
+    xml.elements.each("*/env:Body/sd:events/sd:evIdsAlert") do |element|
+      eid =  element.attributes["eventId"]
+      # Get timestamp in nsec from sensor
+      timestamp=REXML::XPath.first(element,"./sd:time").text.to_i(10)
+      events[eid] = {
+        "@timestamp" => Time.at(timestamp/10**9,((timestamp%10**9)/1000).to_f).iso8601(3),
+        "timezone" => REXML::XPath.first(element,"./sd:time").attributes["timeZone"],
+        "tz_offset" => REXML::XPath.first(element,"./sd:time").attributes["offset"],
+        "event_id" => eid.to_s,
+        "severity" => element.attributes["severity"].to_s.capitalize,
+        "vendor" => element.attributes["vendor"].to_s,
+        "host_id" => REXML::XPath.first(element,"./sd:originator/sd:hostId").text,
+        "app_name" => REXML::XPath.first(element,"./sd:originator/cid:appName").text,
+        "app_instance_id" => REXML::XPath.first(element,"./sd:originator/cid:appInstanceId").text,
+        "description" => REXML::XPath.first(element,"./sd:signature").attributes["description"],
+        "sig_id" => REXML::XPath.first(element,"./sd:signature").attributes["id"],
+        "sig_version" => REXML::XPath.first(element,"./sd:signature").attributes["cid:version"],
+        "sig_type" => REXML::XPath.first(element,"./sd:signature").attributes["cid:type"],
+        "sig_created" => REXML::XPath.first(element,"./sd:signature").attributes["created"],
+        "subsig_id" => REXML::XPath.first(element,"./sd:signature/cid:subsigId").text,
+        "sig_details" => REXML::XPath.first(element,"./sd:signature/cid:sigDetails").text,
+        "interface_group" => REXML::XPath.first(element,"./sd:interfaceGroup").text,
+        "vlan" => REXML::XPath.first(element,"./sd:vlan").text,
+        "attacker_addr" => REXML::XPath.first(element,"./sd:participants/sd:attacker/sd:addr").text,
+        "attacker_locality" => REXML::XPath.first(element,"./sd:participants/sd:attacker/sd:addr").attributes["locality"],
+        "target_addr" => REXML::XPath.first(element,"./sd:participants/sd:target/sd:addr").text,
+        "targed_locality" => REXML::XPath.first(element,"./sd:participants/sd:target/sd:addr").attributes["locality"],
+        "target_os_source" => REXML::XPath.first(element,"./sd:participants/sd:target/cid:os").attributes["idSource"],
+        "target_os_type" => REXML::XPath.first(element,"./sd:participants/sd:target/cid:os").attributes["type"],
+        "target_os_relevance" => REXML::XPath.first(element,"./sd:participants/sd:target/cid:os").attributes["relevance"],
+        # Need? to parse <cid:summary cid:final='true' cid:initialAlert='6824288769384' cid:summaryType='Regular'>2</cid:summary>
+        "alert_details" => REXML::XPath.first(element,"./cid:alertDetails").text.tr('\\"', '\''),
+        "risk_rating" => REXML::XPath.first(element,"./cid:riskRatingValue").text,
+        "risk_target" => REXML::XPath.first(element,"./cid:riskRatingValue").attributes["targetValueRating"],
+        "risk_attacker" => REXML::XPath.first(element,"./cid:riskRatingValue").attributes["attackRelevanceRating"],
+        "threat_rating" => REXML::XPath.first(element,"./cid:threatRatingValue").text,
+        "interface" => REXML::XPath.first(element,"./cid:interface").text,
+        "host" => URI.parse(@http["url"]).host,
+        "device" => "IPS",
+        "tags" => "SDEE"
+        }
+        events[eid].merge!({"attacker_port" => REXML::XPath.first(element,"./sd:participants/sd:attacker/sd:port").text}) if REXML::XPath.first(element,"./sd:participants/sd:attacker/sd:port")
+        events[eid].merge!({"target_port" => REXML::XPath.first(element,"./sd:participants/sd:target/sd:port").text}) if REXML::XPath.first(element,"./sd:participants/sd:target/sd:port")
+        events[eid].merge!({"message" => "IdsAlert: '#{events[eid]["description"]}' Attacker: '#{events[eid]["attacker_addr"]}' Target: '#{events[eid]["target_addr"]}' SigId: '#{events[eid]["sig_id"]}'"})
+      end
+      events
+  end
+end

data/logstash-input-sdee.gemspec

@@ -0,0 +1,21 @@
+Gem::Specification.new do |s|
+  s.name        = 'logstash-input-sdee'
+  s.version     = '0.6.9'
+  s.date        = '2016-08-14'
+  s.summary     = "Logstah SDEE input from Cisco ASA"
+  s.description = "This Logstash input plugin allows you to call a Cisco SDEE/CIDEE HTTP API, decode the output of it into event(s), and send them on their merry way."
+  s.authors     = ["rootik"]
+  s.email       = 'roootik@gmail.com'
+  s.require_paths = ['lib']
+
+  s.files       = Dir['lib/**/*', 'examples/**/*', '*.gemspec', 'LICENSE', 'Gemfile', 'README.md', 'CHANGELOG.md', 'CONTRIBUTORS']
+  s.homepage    =
+    'http://rubygems.org/gems/logstash-input-sdee'
+  s.metadata = { "logstash_plugin" => "true", "logstash_group" => "input" }
+  s.license       = 'Apache-2.0'
+  s.metadata = { "logstash_plugin" => "true", "logstash_group" => "input" }
+  s.add_runtime_dependency 'logstash-core', '>= 1.4.0', '<= 2.99'
+  s.add_runtime_dependency 'logstash-core-plugin-api', '>= 0.60', '<= 2.99'
+  s.add_runtime_dependency 'logstash-mixin-http_client', '>= 1.0.0', '<= 6.0.0'
+  s.add_runtime_dependency 'rubysl-rexml', '>= 2.0.0', '<= 3.0.0'
+end

metadata

@@ -0,0 +1,135 @@
+--- !ruby/object:Gem::Specification
+name: logstash-input-sdee
+version: !ruby/object:Gem::Version
+  version: 0.6.9
+platform: ruby
+authors:
+- rootik
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2016-08-14 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: logstash-core
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: 1.4.0
+    - - <=
+      - !ruby/object:Gem::Version
+        version: '2.99'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: 1.4.0
+    - - <=
+      - !ruby/object:Gem::Version
+        version: '2.99'
+- !ruby/object:Gem::Dependency
+  name: logstash-core-plugin-api
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0.60'
+    - - <=
+      - !ruby/object:Gem::Version
+        version: '2.99'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0.60'
+    - - <=
+      - !ruby/object:Gem::Version
+        version: '2.99'
+- !ruby/object:Gem::Dependency
+  name: logstash-mixin-http_client
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: 1.0.0
+    - - <=
+      - !ruby/object:Gem::Version
+        version: 6.0.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: 1.0.0
+    - - <=
+      - !ruby/object:Gem::Version
+        version: 6.0.0
+- !ruby/object:Gem::Dependency
+  name: rubysl-rexml
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: 2.0.0
+    - - <=
+      - !ruby/object:Gem::Version
+        version: 3.0.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: 2.0.0
+    - - <=
+      - !ruby/object:Gem::Version
+        version: 3.0.0
+description: This Logstash input plugin allows you to call a Cisco SDEE/CIDEE HTTP
+  API, decode the output of it into event(s), and send them on their merry way.
+email: roootik@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- CONTRIBUTORS
+- Gemfile
+- LICENSE
+- README.md
+- examples/patterns/cisco
+- examples/sdee.conf
+- lib/logstash/inputs/sdee.rb
+- logstash-input-sdee.gemspec
+homepage: http://rubygems.org/gems/logstash-input-sdee
+licenses:
+- Apache-2.0
+metadata:
+  logstash_plugin: 'true'
+  logstash_group: input
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.6
+signing_key: 
+specification_version: 4
+summary: Logstah SDEE input from Cisco ASA
+test_files: []