logstash-lib 1.3.2

data/docs/plugin-synopsis.html.erb

@@ -0,0 +1,24 @@
+<%= name %> {
+<% sorted_attributes.each do |name, config|
+   next if config[:deprecated]
+   if config[:validate].is_a?(Array) 
+     annotation = "string, one of #{config[:validate].inspect}"
+   elsif config[:validate] == :path
+     annotation = "a valid filesystem path"
+   else 
+     annotation = "#{config[:validate]}"
+   end
+
+   if name.is_a?(Regexp)
+     name = "/" + name.to_s.gsub(/^\(\?-mix:/, "").gsub(/\)$/, "") + "/"
+   end
+   if config[:required]
+     annotation += " (required)"
+   else
+     annotation += " (optional)"
+   end
+   annotation += ", default: #{config[:default].inspect}" if config.include?(:default)
+-%>
+    <a href="#<%= name %>"><%= name %></a> => ... # <%= annotation %>
+<% end -%>
+}

data/docs/release-engineering.md

@@ -0,0 +1,46 @@
+---
+title: Release Engineering - logstash
+layout: content_right
+---
+
+# logstash rel-eng.
+
+The version patterns for logstash are x.y.z
+
+* In the same x.y release, no backwards-incompatible changes will be made.
+* Between x.y.z and x.y.(z+1), deprecations are allowed but should be
+  functional through the next release.
+* Any backwards-incompatible changes should be well-documented and, if
+  possible, should include tools to help in migrating.
+* It is OK to add features, plugins, etc, in minor releases as long as they do
+  not break existing functionality.
+
+I do not suspect the 'x' (currently 1) will change frequently. It should only change
+if there are major, backwards-incompatible changes made to logstash, and I'm
+trying to not make those changes, so logstash should forever be at 1.y,z,
+right? ;)
+
+# building a release.
+
+* Make sure all tests pass (make test)
+  * `ruby bin/logstash test`
+  * `java -jar logstash-x.y.z-flatjar.jar test`
+* Update VERSION.rb
+  * VERSION=$(ruby -r./VERSION -e 'puts LOGSTASH_VERSION')
+* Ensure CHANGELOG is up-to-date
+* `git tag v$VERSION; git push origin master; git push --tags`
+* Build binaries
+  * `make jar`
+* make docs
+  * copy build/docs to ../logstash.github.com/docs/$VERSION
+  * Note: you will need to use C-ruby 1.9.2 for this.
+  * You'll need 'bluecloth' and 'cabin' rubygems installed.
+* cd ../logstash.github.com
+  * `make clean update VERSION=$VERSION`
+  * `git add docs/$VERSION docs/latest.html index.html _layouts/*`
+  * `git commit -m "version $VERSION docs" && git push origin master`
+* Publish binaries
+  * Stage binaries at `carrera.databits.net:/home/jls/s/files/logstash/`
+* Update #logstash IRC /topic
+* Send announcement email to logstash-users@, include relevant download URLs &
+  changelog (see past emails for a template)

data/docs/release-test-results.md

@@ -0,0 +1,14 @@
+# Testing for a release
+
+* exec + split + stdout
+* tcp input (server and client modes)
+* tcp output (server and client modes)
+* graphite output (tested server failure conditions, netcat receiver)
+* statsd output (increment, netcat receiver)
+
+## Test Suite
+
+    Finished in 16.826 seconds.
+
+    29 tests, 119 assertions, 0 failures, 0 errors
+

data/docs/repositories.md

@@ -0,0 +1,35 @@
+---
+title: repositories - logstash
+layout: content_right
+---
+# LogStash repositories
+
+We also have Logstash available als APT and YUM repositories.
+
+Our public signing key can be found [here](http://packages.elasticsearch.org/GPG-KEY-elasticsearch)
+
+## Apt based distributions
+
+Add the key:
+
+     wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | apt-key add -
+
+Add the repo to /etc/apt/sources.list
+
+     deb http://packages.elasticsearch.org/logstash/1.3/debian stable main
+
+
+## YUM based distributions
+
+Add the key:
+
+     rpm --import http://packages.elasticsearch.org/GPG-KEY-elasticsearch
+
+Add the repo to /etc/yum.repos.d/ directory
+
+     [logstash-1.3]
+     name=logstash repository for 1.3.x packages
+     baseurl=http://packages.elasticsearch.org/logstash/1.3/centos
+     gpgcheck=1
+     gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
+     enabled=1

data/docs/tutorials/10-minute-walkthrough/apache-elasticsearch.conf

@@ -0,0 +1,35 @@
+input {
+  tcp { 
+    type => "apache"
+    port => 3333
+  } 
+}
+
+filter {
+  if [type] == "apache" {
+    grok {
+      # See the following URL for a complete list of named patterns
+      # logstash/grok ships with by default:
+      # https://github.com/logstash/logstash/tree/master/patterns
+      #
+      # The grok filter will use the below pattern and on successful match use
+      # any captured values as new fields in the event.
+      match => { "message" => "%{COMBINEDAPACHELOG}" }
+    }
+
+    date {
+      # Try to pull the timestamp from the 'timestamp' field (parsed above with
+      # grok). The apache time format looks like: "18/Aug/2011:05:44:34 -0700"
+      match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
+    }
+  }
+}
+
+output {
+  elasticsearch {
+    # Setting 'embedded' will run  a real elasticsearch server inside logstash.
+    # This option below saves you from having to run a separate process just
+    # for ElasticSearch, so you can get started quicker!
+    embedded => true
+  }
+}

data/docs/tutorials/10-minute-walkthrough/apache-parse.conf

@@ -0,0 +1,33 @@
+input {
+  tcp { 
+    type => "apache"
+    port => 3333
+  } 
+}
+
+filter {
+  if [type] == "apache" {
+    grok {
+      # See the following URL for a complete list of named patterns
+      # logstash/grok ships with by default:
+      # https://github.com/logstash/logstash/tree/master/patterns
+      #
+      # The grok filter will use the below pattern and on successful match use
+      # any captured values as new fields in the event.
+      match => { "message" => "%{COMBINEDAPACHELOG}" }
+    }
+
+    date {
+      # Try to pull the timestamp from the 'timestamp' field (parsed above with
+      # grok). The apache time format looks like: "18/Aug/2011:05:44:34 -0700"
+      match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
+    }
+  }
+}
+
+output {
+  # Use stdout in debug mode again to see what logstash makes of the event.
+  stdout {
+    debug => true
+  }
+}

data/docs/tutorials/10-minute-walkthrough/apache_log.1

@@ -0,0 +1 @@
+129.92.249.70 - - [18/Aug/2011:06:00:14 -0700] "GET /style2.css HTTP/1.1" 200 1820 "http://www.semicomplete.com/blog/geekery/bypassing-captive-portals.html" "Mozilla/5.0 (iPad; U; CPU OS 4_3_5 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5"

data/docs/tutorials/10-minute-walkthrough/hello-search.conf

@@ -0,0 +1,25 @@
+input {
+  stdin { 
+    # A type is a label applied to an event. It is used later with filters
+    # to restrict what filters are run against each event.
+    type => "human"
+  } 
+}
+
+output {
+  # Print each event to stdout.
+  stdout {
+    # Enabling 'rubydebug' codec on the stdout output will make logstash
+    # pretty-print the entire event as something similar to a JSON representation.
+    codec => rubydebug
+  }
+  
+  # You can have multiple outputs. All events generally to all outputs.
+  # Output events to elasticsearch
+  elasticsearch {
+    # Setting 'embedded' will run  a real elasticsearch server inside logstash.
+    # This option below saves you from having to run a separate process just
+    # for ElasticSearch, so you can get started quicker!
+    embedded => true
+  }
+}

data/docs/tutorials/10-minute-walkthrough/hello.conf

@@ -0,0 +1,16 @@
+input {
+  stdin { 
+    # A type is a label applied to an event. It is used later with filters
+    # to restrict what filters are run against each event.
+    type => "human"
+  } 
+}
+
+output {
+  # Print each event to stdout.
+  stdout {
+    # Enabling 'rubydebug' codec on the stdout output will make logstash
+    # pretty-print the entire event as something similar to a JSON representation.
+    codec => rubydebug
+  }
+}

data/docs/tutorials/10-minute-walkthrough/index.md

@@ -0,0 +1,124 @@
+---
+title: Logstash 10-Minute Tutorial
+layout: content_right
+---
+# Logstash 10-minute Tutorial
+
+## Step 1 - Download
+
+### Download logstash:
+
+* [logstash-%VERSION%-flatjar.jar](https://download.elasticsearch.org/logstash/logstash/logstash-%VERSION%-flatjar.jar)
+
+### Requirements:
+
+* java
+
+### The Secret:
+
+logstash is written in JRuby, but I release standalone jar files for easy
+deployment, so you don't need to download JRuby or most any other dependencies.
+
+I bake as much as possible into the single release file.
+
+## Step 2 - A hello world.
+
+### Download this config file:
+
+* [hello.conf](hello.conf)
+
+### Run it:
+
+    java -jar logstash-%VERSION%-flatjar.jar agent -f hello.conf
+
+Type stuff on standard input. Press enter. Watch what event logstash sees.
+Press ^C to kill it.
+
+## Step 3 - Add ElasticSearch
+
+### Download this config file:
+
+* [hello-search.conf](hello-search.conf)
+
+### Run it:
+
+    java -jar logstash-%VERSION%-flatjar.jar agent -f hello-search.conf
+
+Same config as step 2, but now we are also writing events to ElasticSearch. Do
+a search for `*` (all):
+
+    curl 'http://localhost:9200/_search?pretty=1&q=*'
+
+## Step 4 - logstash web
+
+The previous step is good, but a better frontend on elasticsearch would help!
+
+The same config as step 3 is used.
+
+### Run it:
+
+    java -jar logstash-%VERSION%-flatjar.jar agent -f hello-search.conf -- web
+
+The above runs both the agent and the logstash web interface in the same
+process. Useful for simple deploys.
+
+### Use it:
+
+Go to the logstash web interface in browser: <http://localhost:9292/>
+
+Type stuff on stdin on the agent, then search for it in the web interface.
+
+## Step 5 - real world example
+
+Let's backfill some old apache logs.  First, let's use grok.
+
+Use the ['grok'](../../filters/grok) logstash filter to parse logs. 
+
+### Download
+
+* [apache-parse.conf](apache-parse.conf)
+* [apache_log.1](apache_log.1) (a single apache log line)
+
+### Run it
+
+    java -jar logstash-%VERSION%-flatjar.jar agent -f apache-parse.conf
+
+Logstash will now be listening on TCP port 3333. Send an apache log message at it:
+
+    nc localhost 3333 < apache_log.1
+
+The expected output can be viewed here: [step-5-output.txt](step-5-output.txt)
+
+## Step 6 - real world example + search
+
+Same as the previous step, but we'll output to ElasticSearch now.
+
+### Download
+
+* [apache-elasticsearch.conf](apache-elasticsearch.conf)
+* [apache_log.2.bz2](apache_log.2.bz2) (2 days of apache logs)
+
+### Run it
+
+    java -jar logstash-%VERSION%-flatjar.jar agent -f apache-elasticsearch.conf -- web
+
+Logstash should be all set for you now. Start feeding it logs:
+
+    bzip2 -d apache_log.2.bz2
+
+    nc localhost 3333 < apache_log.2 
+
+Go to the logstash web interface in browser: <http://localhost:9292/>
+
+Try some search queries. To see all the data, search for `*` (no quotes). Click
+on some results, drill around in some logs.
+
+## Want more?
+
+For further learning, try these:
+
+* [Watch a presentation on logstash](http://www.youtube.com/embed/RuUFnog29M4)
+* [Getting started 'standalone' guide](http://logstash.net/docs/%VERSION%/tutorials/getting-started-simple)
+* [Getting started 'centralized' guide](http://logstash.net/docs/%VERSION%/tutorials/getting-started-centralized) - 
+  learn how to build out your logstash infrastructure and centralize your logs.
+* [Dive into the docs](http://logstash.net/docs/%VERSION%/)

data/docs/tutorials/10-minute-walkthrough/step-5-output.txt

@@ -0,0 +1,17 @@
+{
+  "type"        => "apache",
+  "clientip"          => "129.92.249.70",
+  "ident"             => "-",
+  "auth"              => "-",
+  "timestamp"         => "18/Aug/2011:06:00:14 -0700",
+  "verb"              => "GET",
+  "request"           => "/style2.css",
+  "httpversion"       => "1.1",
+  "response"          => "200",
+  "bytes"             => "1820",
+  "referrer"          => "http://www.semicomplete.com/blog/geekery/bypassing-captive-portals.html",
+  "agent"             => "\"Mozilla/5.0 (iPad; U; CPU OS 4_3_5 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5\"",
+  "@timestamp"   => "2011-08-18T13:00:14.000Z",
+  "host" => "127.0.0.1",
+  "message"     => "129.92.249.70 - - [18/Aug/2011:06:00:14 -0700] \"GET /style2.css HTTP/1.1\" 200 1820 \"http://www.semicomplete.com/blog/geekery/bypassing-captive-portals.html\" \"Mozilla/5.0 (iPad; U; CPU OS 4_3_5 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5\"\n"
+}

data/docs/tutorials/getting-started-centralized-overview-diagram.xml

@@ -0,0 +1 @@
+<mxGraphModel dx="800" dy="800" grid="1" guides="1" tooltips="1" connect="1" fold="1" page="1" pageScale="1" pageWidth="826" pageHeight="1169" style="default-style2"><root><mxCell id="0"/><mxCell id="1" parent="0"/><mxCell id="2" value="Shipper" style="icon;image=http://logstash.net/images/logstash.png;fontSize=18" parent="1" vertex="1"><mxGeometry x="50" y="110" width="60" height="60" as="geometry"/></mxCell><mxCell id="5" value="Broker" style="icon;image=http://www.java-freelance.fr/wp-content/uploads/2012/06/banner_redis-300dpi-0315a8013afee137cce47b474541d7f1.png;fontSize=18" parent="1" vertex="1"><mxGeometry x="180" y="161" width="60" height="60" as="geometry"/></mxCell><mxCell id="7" value="Storage &amp; Search" style="icon;image=http://www.elasticsearch.org/images/set3/bonsai2.png;fontSize=18" parent="1" vertex="1"><mxGeometry x="470" y="161" width="120" height="60" as="geometry"/></mxCell><mxCell id="8" value="" style="endArrow=open;entryX=0;entryY=0.5;fontSize=18" parent="1" source="5" target="11" edge="1"><mxGeometry x="-40" as="geometry"><mxPoint x="520" y="160" as="targetPoint"/></mxGeometry></mxCell><mxCell id="11" value="Indexer" style="icon;image=http://logstash.net/images/logstash.png;fontSize=18" parent="1" vertex="1"><mxGeometry x="280" y="161" width="140" height="60" as="geometry"/></mxCell><mxCell id="13" value="Shipper" style="icon;image=http://logstash.net/images/logstash.png;fontSize=18" parent="1" vertex="1"><mxGeometry x="50" y="240" width="60" height="60" as="geometry"/></mxCell><mxCell id="16" value="...&#xa;" style="text;fontSize=36" parent="1" vertex="1"><mxGeometry x="60" y="178" width="40" height="52" as="geometry"/></mxCell><mxCell id="17" value="" style="endArrow=none;exitX=1;exitY=0.5;entryX=1;entryY=0.5;fontSize=18" parent="1" edge="1"><mxGeometry x="50" y="-7" as="geometry"><mxPoint x="260" y="250" as="sourcePoint"/><mxPoint x="260" y="250" as="targetPoint"/></mxGeometry></mxCell><mxCell id="19" value="" style="endArrow=open;entryX=0;entryY=0.5;exitX=1;exitY=0.5;fontSize=18" parent="1" source="2" target="5" edge="1"><mxGeometry x="-40" as="geometry"><mxPoint x="520" y="160" as="targetPoint"/></mxGeometry></mxCell><mxCell id="20" value="" style="endArrow=open;entryX=0;entryY=0.5;exitX=1;exitY=0.5;fontSize=18" parent="1" source="13" target="5" edge="1"><mxGeometry x="-40" as="geometry"><mxPoint x="520" y="160" as="targetPoint"/></mxGeometry></mxCell><mxCell id="21" value="" style="endArrow=open;exitX=1;exitY=0.5;fontSize=18" parent="1" source="11" target="7" edge="1"><mxGeometry x="-40" as="geometry"><mxPoint x="490" y="200" as="sourcePoint"/><mxPoint x="520" y="160" as="targetPoint"/></mxGeometry></mxCell><mxCell id="26" value="Logstash" style="label;image=http://logstash.net/images/logstash.png;fontSize=12" parent="1" vertex="1"><mxGeometry x="210" y="10" width="140" height="40" as="geometry"/></mxCell><mxCell id="28" value="Redis" style="label;image=http://www.java-freelance.fr/wp-content/uploads/2012/06/banner_redis-300dpi-0315a8013afee137cce47b474541d7f1.png" parent="1" vertex="1"><mxGeometry x="350" y="10" width="110" height="40" as="geometry"/></mxCell><mxCell id="32" value="ElasticSearch" style="label;image=http://www.elasticsearch.org/images/set3/bonsai2.png;fontSize=12" parent="1" vertex="1"><mxGeometry x="460" y="10" width="210" height="40" as="geometry"/></mxCell><mxCell id="34" value="Web Interface" style="icon;image=http://logstash.net/images/logstash.png;fontSize=18" parent="1" vertex="1"><mxGeometry x="630" y="161" width="140" height="60" as="geometry"/></mxCell><mxCell id="35" value="" style="endArrow=open;entryX=0;entryY=0.5;exitX=1;exitY=0.5;fontSize=18" parent="1" source="7" target="34" edge="1"><mxGeometry x="-40" as="geometry"><mxPoint x="520" y="160" as="targetPoint"/></mxGeometry></mxCell><mxCell id="36" value="Legend:" style="text;align=center;verticalAlign=middle;fontSize=18" parent="1" vertex="1"><mxGeometry x="130" y="10" width="70" height="40" as="geometry"/></mxCell><mxCell id="37" value="" style="endArrow=open;entryX=0;entryY=0.5;exitX=1;exitY=0.75" parent="1" source="16" target="5" edge="1"><mxGeometry as="geometry"><mxPoint x="560" y="160" as="targetPoint"/></mxGeometry></mxCell></root></mxGraphModel>

data/docs/tutorials/getting-started-centralized.md

@@ -0,0 +1,217 @@
+---
+title: Getting Started (Centralized Server) - logstash
+layout: content_right
+---
+
+# Getting Started
+
+## Centralized Setup with Event Parsing
+
+This guide shows how to get you going quickly with logstash with multiple
+servers. This guide is for folks who want to ship all their logstash logs to a
+central location for indexing and search.
+
+We'll have two classes of server. First, one that ships logs. Second, one that
+collects and indexes logs.
+
+It's important to note that logstash itself has no concept of "shipper" and
+"collector" - the behavior of an agent depends entirely on how you configure
+it.
+
+This diagram gives you an overview of the architecture:
+
+![Centralized setup overview](getting-started-centralized-overview-diagram.png)
+
+On servers shipping logs:
+
+* Download and run logstash (See section 'logstash log shipper' below)
+
+On the server collecting and indexing your logs:
+
+* Download and run Elasticsearch
+* Download and run Redis
+* Download and run Logstash
+
+## ElasticSearch
+
+Requirements: java
+
+You'll most likely want the version of ElasticSearch specified by the
+[elasticsearch output](../outputs/elasticsearch) docs. Modify this in your shell
+for easy downloading of ElasticSearch:
+
+    ES_PACKAGE=elasticsearch-%ELASTICSEARCH_VERSION%.zip
+    ES_DIR=${ES_PACKAGE%%.zip}
+    SITE=https://download.elasticsearch.org/elasticsearch/elasticsearch
+    if [ ! -d "$ES_DIR" ] ; then
+      wget --no-check-certificate $SITE/$ES_PACKAGE
+      unzip $ES_PACKAGE
+    fi
+
+ElasticSearch requires Java (uses Lucene on the backend; if you want to know
+more read the elasticsearch docs).
+
+To start the service, run `bin/elasticsearch -f`. This will run it in the foreground. We want to keep it this way for debugging for now.
+
+## Redis
+
+Previous versions of this guide used AMQP via RabbitMQ. Due to the complexity of AMQP as well as performance issues related to the Bunny driver we use, we're now recommending Redis instead.
+
+Redis has no external dependencies and has a much simpler configuration in Logstash.
+
+Building and installing Redis is fairly straightforward. While normally this would be out of the scope of this document, as the instructions are so simple we'll include them here:
+
+- Download Redis from http://redis.io/download (The latest stable release is likely what you want)
+- Extract the source, change to the directory and run `make`
+- Run Redis with `src/redis-server --loglevel verbose`
+
+That's it.
+
+## logstash
+
+Once you have elasticsearch and redis running, you're
+ready to configure logstash.
+
+Download the logstash release jar file. The package contains all
+required dependencies to save you time chasing down requirements.
+
+Follow [this link to download logstash-%VERSION%](https://download.elasticsearch.org/logstash/logstash/logstash-%VERSION%-flatjar.jar).
+
+Since we're doing a centralized configuration, you'll have two main
+logstash agent roles: a shipper and an indexer. You will ship logs from
+all servers via Redis and have another agent receive those messages,
+parse them, and index them in elasticsearch.
+
+### logstash log shipper
+
+As with the simple example, we're going to start simple to ensure that events are flowing
+
+    input {
+      stdin {
+        type => "example"
+      }
+    }
+
+    output {
+      stdout { codec => rubydebug }
+      redis { host => "127.0.0.1" data_type => "list" key => "logstash" }
+    }
+
+Put this in a file and call it 'shipper.conf' (or anything, really), and run: 
+
+    java -jar logstash-%VERSION%-flatjar.jar agent -f shipper.conf
+
+This will take anything you type into this console and display it on the console. Additionally it will save events to Redis in a `list` named after the `key` value you provided.
+
+### Testing the Redis output
+
+To verify that the message made it into Redis, check your Redis window. You should see something like the following:
+
+    [83019] 02 Jul 12:51:02 - Accepted 127.0.0.1:58312
+    [83019] 02 Jul 12:51:06 - Client closed connection
+    [83019] 02 Jul 12:51:06 - DB 0: 1 keys (0 volatile) in 4 slots HT.
+
+The redis application ships with a CLI application that you can use to query the data. From your Redis source directory, run the following:
+
+`src/redis-cli`
+
+Once connected, run the following commands:
+
+    redis 127.0.0.1:6379> llen logstash
+    (integer) 1
+    redis 127.0.0.1:6379> lpop logstash
+    "{\"message\":\"hello\",\"@timestamp\":\"2013-09-07T00:59:28.383Z\",\"@version\":\"1\",\"type\":\"stdin\",\"host\":\"pork\"}"
+    redis 127.0.0.1:6379> llen logstash
+    (integer) 0
+
+What we've just done is check the length of the list, read and removed the oldest item in the list, and checked the length again.
+
+This behavior is what Logstash does when it reads from a Redis input (technically logstash performs a blocking lpop). We're essentially using Redis to simulate a queue via the `list` data type.
+
+Go ahead and type a few more entries in the agent window:
+
+- test 1
+- test 2
+- test 3
+
+As you `lpop` you should get them in the correct order of insertion.
+
+### logstash indexer
+
+This agent will parse and index your logs as they come in over Redis. Here's a
+sample config based on the previous section. Save this as `indexer.conf`
+
+    input {
+      redis {
+        host => "127.0.0.1"
+        # these settings should match the output of the agent
+        data_type => "list"
+        key => "logstash"
+
+        # We use the 'json' codec here because we expect to read
+        # json events from redis.
+        codec => json
+      }
+    }
+    
+    output {
+      stdout { debug => true debug_format => "json"}
+
+      elasticsearch {
+        host => "127.0.0.1"
+      }
+    }
+
+The above configuration will attach to Redis and issue a `BLPOP` against the `logstash` list. When an event is recieved, it will be pulled off and sent to Elasticsearch for indexing.
+
+Start the indexer the same way as the agent but specifying the `indexer.conf` file:
+
+`java -jar logstash-%VERSION%-flatjar.jar agent -f indexer.conf`
+
+To verify that your Logstash indexer is connecting to Elasticsearch properly, you should see a message in your Elasticsearch window similar to the following:
+
+`[2012-07-02 13:14:27,008][INFO ][cluster.service          ] [Baron Samedi] added {[Bes][JZQBMR21SUWRNtTMsDV3_g][inet[/192.168.1.194:9301]]{client=true, data=false},}`
+
+The names `Bes` and `Baron Samedi` may differ as ES uses random names for nodes.
+
+### Testing the flow
+Now we want to test the flow. In your agent window, type something to generate an event.
+The indexer should read this and persist it to Elasticsearch. It will also display the event to stdout.
+
+In your Elasticsearch window, you should see something like the following:
+
+    [2012-07-02 13:21:58,982][INFO ][cluster.metadata         ] [Baron Samedi] [logstash-2012.07.02] creating index, cause [auto(index api)], shards [5]/[1], mappings []
+    [2012-07-02 13:21:59,495][INFO ][cluster.metadata         ] [Baron Samedi] [logstash-2012.07.02] update_mapping [stdin-type] (dynamic)
+
+Since indexes are created dynamically, this is the first sign that Logstash was able to write to ES. Let's use curl to verify our data is there:
+Using our curl command from the simple tutorial should let us see the data:
+
+`curl -gs -XGET http://localhost:9200/logstash-*/_search?q=type:example`
+
+You may need to modify the date as this is based on the date this guide was written.
+
+Now we can move on to the final step...
+## logstash web interface
+
+Run this on the same server as your elasticsearch server.
+
+To run the logstash web server, just run the jar with 'web' as the first
+argument. 
+
+    java -jar logstash-%VERSION%-flatjar.jar web
+
+Just point your browser at the http://127.0.0.1:9292/ and start searching
+logs!
+
+The web interface is called 'kibana' - you can learn more about kibana at <http://kibana.org>
+
+# Distributing the load
+At this point we've been simulating a distributed environment on a single machine. If only the world were so easy.
+In all of the example configurations, we've been explicitly setting the connection to connect to `127.0.0.1` despite the fact in most network-related plugins, that's the default host.
+
+Since Logstash is so modular, you can install the various components on different systems.
+
+- If you want to give Redis a dedicated host, simply ensure that the `host` attribute in configurations points to that host.
+- If you want to give Elasticsearch a dedicated host, simple ensure that the `host` attribute is correct as well (in both web and indexer).
+
+As with the simple input example, reading from stdin is fairly useless. Check the Logstash documentation for the various inputs offered and mix and match to taste!