logstash-codec-avro 0.1.0 → 0.1.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (7) hide show
  1. checksums.yaml +4 -4
  2. data/DEVELOPER.md +66 -0
  3. data/LICENSE +1 -1
  4. data/README.md +79 -50
  5. data/lib/logstash/codecs/avro.rb +0 -1
  6. data/logstash-codec-avro.gemspec +1 -1
  7. metadata +17 -16
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: bb0bcc66f0b7e43cae0ac6936ddda5f7cf65dc44
4
- data.tar.gz: 095ac47a666d8b0db08757e07132021db22f83e4
3
+ metadata.gz: 8c15d8d784a754562e9f9bf12de4bcfa337b24dd
4
+ data.tar.gz: e914acd0fcc68cdc20aded6509ac858152a9708c
5
5
  SHA512:
6
- metadata.gz: 69e7e6ca8fcbe43db61ff6ed8246507e0df04e253cb1aaebb0549256bc017c634a12032337a6b9cf1dc539b353b05c6afd8e1dc3f71ad17c001888d5aa8f466b
7
- data.tar.gz: 6f738e2a34e9de2acdf2a67f9d7b982bd8859c2bf5bf1f77d431d227d7901c4fff0960b5b1d1e7a13d8300a4d006c49c3ea70f44cd962192c4702329db21a096
6
+ metadata.gz: 629036bf86db50e57866adf251a373590fafa581dd7cac67b93d8d0c88f5bb3c9803f7e26de0d4899eb728733f07d1521d9c2ac678348bdf982da72f8507ae33
7
+ data.tar.gz: 65839e8c200ca2405258cac49086f4dd6a8d0c07c1f6ccbfeafe577101cdf801563a250dffcecc173255ab722a7fee6e19aa59dec36b55e140192348a938ffdf
data/DEVELOPER.md ADDED
@@ -0,0 +1,66 @@
1
+ Logstash Avro Codec
2
+ ===================
3
+
4
+ How to Install
5
+ --------------
6
+
7
+ ```
8
+ bin/plugin install logstash-avro-codec
9
+ ```
10
+
11
+ How to Use
12
+ ----------
13
+ You can use this codec to decode avro messages
14
+ in a Kafka topic input.
15
+
16
+ Here is an example schema for tweets.
17
+
18
+ ### tweet.avsc
19
+ ```
20
+ {
21
+ "type" : "record",
22
+ "name" : "twitter_schema",
23
+ "namespace" : "com.miguno.avro",
24
+ "fields" : [ {
25
+ "name" : "username",
26
+ "type" : "string",
27
+ "doc" : "Name of the user account on Twitter.com"
28
+ }, {
29
+ "name" : "tweet",
30
+ "type" : "string",
31
+ "doc" : "The content of the user's Twitter message"
32
+ }, {
33
+ "name" : "timestamp",
34
+ "type" : "long",
35
+ "doc" : "Unix epoch time in seconds"
36
+ } ],
37
+ "doc:" : "A basic schema for storing Twitter messages"
38
+ }
39
+ ```
40
+
41
+ Along with the logstash config for reading in messages of this
42
+ type using the avro codec with the logstash-input-kafka plugin.
43
+
44
+ ### logstash.conf
45
+
46
+ ```
47
+ input {
48
+ kafka {
49
+ topic_id => 'test_topic'
50
+ codec => avro {
51
+ schema_file => 'tweet.avsc'
52
+ }
53
+ }
54
+ }
55
+
56
+ output {
57
+ stdout {
58
+ codec => rubydebug
59
+ }
60
+ }
61
+ ```
62
+
63
+ ### Running the setup
64
+ ```
65
+ bin/logstash -f logstash.conf
66
+ ```
data/LICENSE CHANGED
@@ -1,4 +1,4 @@
1
- Copyright (c) 2012-2014 Elasticsearch <http://www.elasticsearch.org>
1
+ Copyright (c) 2012-2015 Elasticsearch <http://www.elasticsearch.org>
2
2
 
3
3
  Licensed under the Apache License, Version 2.0 (the "License");
4
4
  you may not use this file except in compliance with the License.
data/README.md CHANGED
@@ -1,66 +1,95 @@
1
- Logstash Avro Codec
2
- ===================
1
+ # Logstash Plugin
3
2
 
4
- How to Install
5
- --------------
3
+ This is a plugin for [Logstash](https://github.com/elasticsearch/logstash).
6
4
 
7
- ```
8
- bin/plugin install logstash-avro-codec
5
+ It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
6
+
7
+ ## Documentation
8
+
9
+ Logstash provides infrastructure to automatically generate documentation for this plugin. We use the asciidoc format to write documentation so any comments in the source code will be first converted into asciidoc and then into html. All plugin documentation are placed under one [central location](http://www.elasticsearch.org/guide/en/logstash/current/).
10
+
11
+ - For formatting code or config example, you can use the asciidoc `[source,ruby]` directive
12
+ - For more asciidoc formatting tips, see the excellent reference here https://github.com/elasticsearch/docs#asciidoc-guide
13
+
14
+ ## Need Help?
15
+
16
+ Need help? Try #logstash on freenode IRC or the logstash-users@googlegroups.com mailing list.
17
+
18
+ ## Developing
19
+
20
+ ### 1. Plugin Developement and Testing
21
+
22
+ #### Code
23
+ - To get started, you'll need JRuby with the Bundler gem installed.
24
+
25
+ - Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization.
26
+
27
+ - Install dependencies
28
+ ```sh
29
+ bundle install
9
30
  ```
10
31
 
11
- How to Use
12
- ----------
13
- You can use this codec to decode avro messages
14
- in a Kafka topic input.
32
+ #### Test
15
33
 
16
- Here is an example schema for tweets.
34
+ ```sh
35
+ bundle exec rspec
36
+ ```
17
37
 
18
- ### tweet.avsc
38
+ The Logstash code required to run the tests/specs is specified in the `Gemfile` by the line similar to:
39
+ ```ruby
40
+ gem "logstash", :github => "elasticsearch/logstash", :branch => "1.5"
19
41
  ```
20
- {
21
- "type" : "record",
22
- "name" : "twitter_schema",
23
- "namespace" : "com.miguno.avro",
24
- "fields" : [ {
25
- "name" : "username",
26
- "type" : "string",
27
- "doc" : "Name of the user account on Twitter.com"
28
- }, {
29
- "name" : "tweet",
30
- "type" : "string",
31
- "doc" : "The content of the user's Twitter message"
32
- }, {
33
- "name" : "timestamp",
34
- "type" : "long",
35
- "doc" : "Unix epoch time in seconds"
36
- } ],
37
- "doc:" : "A basic schema for storing Twitter messages"
38
- }
42
+ To test against another version or a local Logstash, edit the `Gemfile` to specify an alternative location, for example:
43
+ ```ruby
44
+ gem "logstash", :github => "elasticsearch/logstash", :ref => "master"
45
+ ```
46
+ ```ruby
47
+ gem "logstash", :path => "/your/local/logstash"
48
+ ```
49
+
50
+ Then update your dependencies and run your tests:
51
+
52
+ ```sh
53
+ bundle install
54
+ bundle exec rspec
39
55
  ```
40
56
 
41
- Along with the logstash config for reading in messages of this
42
- type using the avro codec with the logstash-input-kafka plugin.
57
+ ### 2. Running your unpublished Plugin in Logstash
43
58
 
44
- ### logstash.conf
59
+ #### 2.1 Run in a local Logstash clone
45
60
 
61
+ - Edit Logstash `tools/Gemfile` and add the local plugin path, for example:
62
+ ```ruby
63
+ gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
64
+ ```
65
+ - Update Logstash dependencies
66
+ ```sh
67
+ rake vendor:gems
46
68
  ```
47
- input {
48
- kafka {
49
- topic_id => 'test_topic'
50
- codec => avro {
51
- schema_file => 'tweet.avsc'
52
- }
53
- }
54
- }
55
-
56
- output {
57
- stdout {
58
- codec => rubydebug
59
- }
60
- }
69
+ - Run Logstash with your plugin
70
+ ```sh
71
+ bin/logstash -e 'filter {awesome {}}'
61
72
  ```
73
+ At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
74
+
75
+ #### 2.2 Run in an installed Logstash
62
76
 
63
- ### Running the setup
77
+ - Build your plugin gem
78
+ ```sh
79
+ gem build logstash-filter-awesome.gemspec
64
80
  ```
65
- bin/logstash -f logstash.conf
81
+ - Install the plugin from the Logstash home
82
+ ```sh
83
+ bin/plugin install /your/local/plugin/logstash-filter-awesome.gem
66
84
  ```
85
+ - Start Logstash and proceed to test the plugin
86
+
87
+ ## Contributing
88
+
89
+ All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
90
+
91
+ Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
92
+
93
+ It is more important to me that you are able to contribute.
94
+
95
+ For more information about contributing, see the [CONTRIBUTING](https://github.com/elasticsearch/logstash/blob/master/CONTRIBUTING.md) file.
data/lib/logstash/codecs/avro.rb CHANGED
@@ -9,7 +9,6 @@ require "logstash/util"
9
9
  class LogStash::Codecs::Avro < LogStash::Codecs::Base
10
10
  config_name "avro"
11
11
 
12
- milestone 1
13
12
 
14
13
  # schema path to fetch the schema from
15
14
  # This can be a 'http' or 'file' scheme URI
data/logstash-codec-avro.gemspec CHANGED
@@ -1,7 +1,7 @@
1
1
  Gem::Specification.new do |s|
2
2
 
3
3
  s.name = 'logstash-codec-avro'
4
- s.version = '0.1.0'
4
+ s.version = '0.1.1'
5
5
  s.licenses = ['Apache License (2.0)']
6
6
  s.summary = "Encode and decode avro formatted data"
7
7
  s.description = "Encode and decode avro formatted data"
metadata CHANGED
@@ -1,18 +1,17 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-codec-avro
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Elasticsearch
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2014-12-11 00:00:00.000000000 Z
11
+ date: 2015-01-22 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
- name: logstash
15
- version_requirements: !ruby/object:Gem::Requirement
14
+ requirement: !ruby/object:Gem::Requirement
16
15
  requirements:
17
16
  - - '>='
18
17
  - !ruby/object:Gem::Version
@@ -20,7 +19,10 @@ dependencies:
20
19
  - - <
21
20
  - !ruby/object:Gem::Version
22
21
  version: 2.0.0
23
- requirement: !ruby/object:Gem::Requirement
22
+ name: logstash
23
+ prerelease: false
24
+ type: :runtime
25
+ version_requirements: !ruby/object:Gem::Requirement
24
26
  requirements:
25
27
  - - '>='
26
28
  - !ruby/object:Gem::Version
@@ -28,36 +30,34 @@ dependencies:
28
30
  - - <
29
31
  - !ruby/object:Gem::Version
30
32
  version: 2.0.0
31
- prerelease: false
32
- type: :runtime
33
33
  - !ruby/object:Gem::Dependency
34
- name: avro
35
- version_requirements: !ruby/object:Gem::Requirement
36
- requirements:
37
- - - '>='
38
- - !ruby/object:Gem::Version
39
- version: '0'
40
34
  requirement: !ruby/object:Gem::Requirement
41
35
  requirements:
42
36
  - - '>='
43
37
  - !ruby/object:Gem::Version
44
38
  version: '0'
39
+ name: avro
45
40
  prerelease: false
46
41
  type: :runtime
47
- - !ruby/object:Gem::Dependency
48
- name: logstash-devutils
49
42
  version_requirements: !ruby/object:Gem::Requirement
50
43
  requirements:
51
44
  - - '>='
52
45
  - !ruby/object:Gem::Version
53
46
  version: '0'
47
+ - !ruby/object:Gem::Dependency
54
48
  requirement: !ruby/object:Gem::Requirement
55
49
  requirements:
56
50
  - - '>='
57
51
  - !ruby/object:Gem::Version
58
52
  version: '0'
53
+ name: logstash-devutils
59
54
  prerelease: false
60
55
  type: :development
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ requirements:
58
+ - - '>='
59
+ - !ruby/object:Gem::Version
60
+ version: '0'
61
61
  description: Encode and decode avro formatted data
62
62
  email: info@elasticsearch.com
63
63
  executables: []
@@ -65,6 +65,7 @@ extensions: []
65
65
  extra_rdoc_files: []
66
66
  files:
67
67
  - .gitignore
68
+ - DEVELOPER.md
68
69
  - Gemfile
69
70
  - LICENSE
70
71
  - README.md
@@ -94,7 +95,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
94
95
  version: '0'
95
96
  requirements: []
96
97
  rubyforge_project:
97
- rubygems_version: 2.1.9
98
+ rubygems_version: 2.4.5
98
99
  signing_key:
99
100
  specification_version: 4
100
101
  summary: Encode and decode avro formatted data