rabbit-slide-cosmo0920-plazma-slide-20180215 1.0.0 → 1.0.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rabbit +1 -1
- data/config.yaml +9 -9
- data/pdf/plazma-slide-20180215-plazma-slide.pdf +0 -0
- metadata +1 -2
- data/plazma-slide.md +0 -194
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1ecb029369cd650d71a80a5d7099be316b2654ea042c1186b3f1883968d2dee4
|
|
4
|
+
data.tar.gz: 7b6768e478a6962e94cb43e983c4396d2771afec7b16d736f7c4ca192b7861e1
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5194a3a892b84f14c35d99226c8b1b454f38fe3bc564a7696300f257999acc932bc0d3dea4c83d8e3e9137ebdfd11d2a31cd6ab05aeca0cbc25b7f1de4a39533
|
|
7
|
+
data.tar.gz: 9ff7071f3c8087aff0ea1993373d5676f1258f2bdc345c6fe139afdb4f75ebb723c519e85c4d7c85609ba8442b6c752c0779f82243e70a25e35c6d298245a806
|
data/.rabbit
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
plazma-slide.
|
|
1
|
+
plazma-slide.rd
|
data/config.yaml
CHANGED
|
@@ -3,18 +3,18 @@ id: plazma-slide-20180215
|
|
|
3
3
|
base_name: plazma-slide
|
|
4
4
|
tags:
|
|
5
5
|
- plazma-oss-slide
|
|
6
|
-
presentation_date:
|
|
7
|
-
version: 1.0.
|
|
6
|
+
presentation_date:
|
|
7
|
+
version: 1.0.1
|
|
8
8
|
licenses: []
|
|
9
|
-
slideshare_id:
|
|
10
|
-
speaker_deck_id:
|
|
11
|
-
ustream_id:
|
|
12
|
-
vimeo_id:
|
|
13
|
-
youtube_id:
|
|
9
|
+
slideshare_id:
|
|
10
|
+
speaker_deck_id:
|
|
11
|
+
ustream_id:
|
|
12
|
+
vimeo_id:
|
|
13
|
+
youtube_id:
|
|
14
14
|
author:
|
|
15
15
|
markup_language: :rd
|
|
16
16
|
name: Hiroshi Hatake
|
|
17
17
|
email: cosmo0920.wp@gmail.com
|
|
18
18
|
rubygems_user: cosmo0920
|
|
19
|
-
slideshare_user:
|
|
20
|
-
speaker_deck_user:
|
|
19
|
+
slideshare_user:
|
|
20
|
+
speaker_deck_user:
|
|
Binary file
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: rabbit-slide-cosmo0920-plazma-slide-20180215
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.0.
|
|
4
|
+
version: 1.0.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Hiroshi Hatake
|
|
@@ -66,7 +66,6 @@ files:
|
|
|
66
66
|
- images/plugin-list.png
|
|
67
67
|
- images/structure.svg
|
|
68
68
|
- pdf/plazma-slide-20180215-plazma-slide.pdf
|
|
69
|
-
- plazma-slide.md
|
|
70
69
|
- plazma-slide.rd
|
|
71
70
|
- theme.rb
|
|
72
71
|
- vendor/bundle/ruby/2.5.0/gems/gdk_pixbuf2-3.2.1/sample/floppybuddy.gif
|
data/plazma-slide.md
DELETED
|
@@ -1,194 +0,0 @@
|
|
|
1
|
-
# Cooperative works for Fluentd Community
|
|
2
|
-
|
|
3
|
-
author
|
|
4
|
-
: Kenji Okimoto, Hiroshi Hatake
|
|
5
|
-
|
|
6
|
-
institution
|
|
7
|
-
: ClearCode Inc.
|
|
8
|
-
|
|
9
|
-
theme
|
|
10
|
-
: clear-code
|
|
11
|
-
|
|
12
|
-
# Summary
|
|
13
|
-
|
|
14
|
-
* kafka-connect-fluentd and its benckmarking result
|
|
15
|
-
* maintaining fluentd plugins
|
|
16
|
-
|
|
17
|
-
# kafka-connect-fluentd?
|
|
18
|
-
|
|
19
|
-
* Fluentd Forward Protocol Server implemented with Kafka Connect API
|
|
20
|
-
* [fluent/kakfa-connect-fluentd](https://github.com/fluent/kafka-connect-fluentd)
|
|
21
|
-
|
|
22
|
-
# kafka-connect-fluentd?
|
|
23
|
-
|
|
24
|
-
* Use [influent](https://github.com/okumin/influent) for server implementation
|
|
25
|
-
* Partial SSL/TLS support
|
|
26
|
-
* Aim to replace output plugins in fluent-plugin-kafka
|
|
27
|
-
|
|
28
|
-
# Kafka Connect?
|
|
29
|
-
|
|
30
|
-
> Kafka Connect is a framework included in Apache Kafka that integrates Kafka with other systems.
|
|
31
|
-
> Its purpose is to make it easy to add new systems to your scalable and secure stream data pipelines.
|
|
32
|
-
|
|
33
|
-
# Kafka Connect?
|
|
34
|
-
|
|
35
|
-
* Kafka Producer <-> Kafka Connector Source
|
|
36
|
-
* I'll talk about this implementation.
|
|
37
|
-
* Kafka Consumer <-> Kafka Connector Sink
|
|
38
|
-
* This implementation also exists.
|
|
39
|
-
|
|
40
|
-
# fluent-plugin-kafka との違い
|
|
41
|
-
|
|
42
|
-
(ここに図を入れたい fluent-plugin-kafka + kafka)
|
|
43
|
-
|
|
44
|
-
# fluent-plugin-kafka との違い
|
|
45
|
-
|
|
46
|
-
(ここに図を入れたい fluentd + kafka-connect + kafka)
|
|
47
|
-
|
|
48
|
-
# fluent-plugin-kafka との違い
|
|
49
|
-
|
|
50
|
-
* ... TODO
|
|
51
|
-
|
|
52
|
-
# ベンチマークに必要なもの
|
|
53
|
-
|
|
54
|
-
* 使いやすいベンチマークツール
|
|
55
|
-
* Make benchmark tests to be reproducible
|
|
56
|
-
* Compare performance with same basis
|
|
57
|
-
* 結果を確認しやすいこと
|
|
58
|
-
|
|
59
|
-
# 使いやすいベンチマークツール
|
|
60
|
-
|
|
61
|
-
* There is no tool which is easy to use from command line.
|
|
62
|
-
* Dummer + in_tail exist, but it is hard to apply high load.
|
|
63
|
-
|
|
64
|
-
# Crated benchmark test tool
|
|
65
|
-
|
|
66
|
-
* [okkez/fluent-benchmark-client](https://github.com/okkez/fluent-benchmark-client)
|
|
67
|
-
* Implemented by Kotlin language
|
|
68
|
-
* Sending log relies on [komamitsu/fluency](https://github.com/komamitsu/fluency)
|
|
69
|
-
* SSL/TLS is not supported for now.
|
|
70
|
-
|
|
71
|
-
# Make benchmark tests to be reproducible
|
|
72
|
-
|
|
73
|
-
* Built with Terraform + Ansible
|
|
74
|
-
* Developing specific branch on [fluentd-benchmark](https://github.com/okkez/fluentd-benchmark/tree/add-benchmark-using-gcp)
|
|
75
|
-
* Need to tidy up implementation
|
|
76
|
-
|
|
77
|
-
# Compare performance with same basis
|
|
78
|
-
|
|
79
|
-
* [okkez/kafka-fluent-metrics-reporter](https://github.com/okkez/kafka-fluent-metrics-reporter)
|
|
80
|
-
* Kafka plugin which sends Kafka metrics into Fluentd
|
|
81
|
-
* Process with Fluentd, send to influxdb, and visualize with grafana
|
|
82
|
-
* Write a script which parses result of pidstat and sends into Fluentd
|
|
83
|
-
* To measure CPU and memory usage
|
|
84
|
-
|
|
85
|
-
# VS. fluent-plugin-kafka
|
|
86
|
-
|
|
87
|
-
* (ベンチマーク結果の説明)
|
|
88
|
-
* (全部は入れなくていい)
|
|
89
|
-
|
|
90
|
-
# Fluentd Community
|
|
91
|
-
|
|
92
|
-
* Fluentd community has...
|
|
93
|
-
* lots of plugins
|
|
94
|
-
* Over 700+ plugins
|
|
95
|
-
|
|
96
|
-
# Fluentd Community
|
|
97
|
-
|
|
98
|
-
* Fluentd community has...
|
|
99
|
-
* lots of up-to-date plugins
|
|
100
|
-
* lots of outdated plugins
|
|
101
|
-
|
|
102
|
-
# Fluentd Community
|
|
103
|
-
|
|
104
|
-
* Fluentd community has...
|
|
105
|
-
* lots of up-to-date plugins
|
|
106
|
-
* lots of _outdated_ plugins
|
|
107
|
-
|
|
108
|
-
# How does handle outdated plugins?
|
|
109
|
-
|
|
110
|
-
* Send PRs
|
|
111
|
-
* Report issues
|
|
112
|
-
* Become a maintainer
|
|
113
|
-
|
|
114
|
-
# What means _outdated_?
|
|
115
|
-
|
|
116
|
-
* out-of-date dependencies
|
|
117
|
-
* c.f. client libraries
|
|
118
|
-
* out-of-date class inheritance
|
|
119
|
-
* `Fluent::Input`
|
|
120
|
-
* `Fluent::HttpInput` etc.
|
|
121
|
-
* Using old API
|
|
122
|
-
* Gap between v0.12 and v1.0 API
|
|
123
|
-
|
|
124
|
-
# Maintaining Plugins
|
|
125
|
-
|
|
126
|
-
* Normally, same as other rubygems
|
|
127
|
-
|
|
128
|
-
# Maintaining Plugins
|
|
129
|
-
|
|
130
|
-
* Report issues
|
|
131
|
-
* Send PRs
|
|
132
|
-
|
|
133
|
-
# Maintaining Plugins
|
|
134
|
-
|
|
135
|
-
* 130+ PRs still opened...
|
|
136
|
-
* 1400+ PRs merged
|
|
137
|
-
|
|
138
|
-
# Maintaining Plugins
|
|
139
|
-
|
|
140
|
-
* But, sometimes plugin authors are busy....
|
|
141
|
-
|
|
142
|
-
# Let's take over Fluentd plugins project
|
|
143
|
-
|
|
144
|
-
* Ask plugin authors
|
|
145
|
-
* to add project collaborator
|
|
146
|
-
* to add gem owner
|
|
147
|
-
* Create a cooperative working organization
|
|
148
|
-
|
|
149
|
-
# Become project collaborator
|
|
150
|
-
|
|
151
|
-
* Maintaining only
|
|
152
|
-
* takus/fluent-plugin-ec2-metadata
|
|
153
|
-
* y-ken/fluent-plugin-anonymizer
|
|
154
|
-
|
|
155
|
-
# Become project collaborator and gem owner
|
|
156
|
-
|
|
157
|
-
* Still exists in original place, but maintaining ourselves
|
|
158
|
-
* uken/fluent-plugin-elasticsearch
|
|
159
|
-
* y-ken/fluent-plugin-geoip
|
|
160
|
-
* fluent/fluent-plugin-rewrite-tag-filter
|
|
161
|
-
|
|
162
|
-
# Create cooperative working organization
|
|
163
|
-
|
|
164
|
-
* Cooperative working organization
|
|
165
|
-
* Take over outdated fluentd plugin project
|
|
166
|
-
* New plugins which will be widely used
|
|
167
|
-
* c.f. fluent-plugin-concat
|
|
168
|
-
* And some collaborations
|
|
169
|
-
|
|
170
|
-
# Create cooperative working organization
|
|
171
|
-
|
|
172
|
-
* fluent-plugins-nursery
|
|
173
|
-
|
|
174
|
-
# fluent-plugins-nursery
|
|
175
|
-
|
|
176
|
-
* For Fluentd plugins that are not maintained by original authors
|
|
177
|
-
* Aim to provide a sustainable maintenance system for Fluentd community
|
|
178
|
-
|
|
179
|
-
# fluent-plugins-nursery
|
|
180
|
-
|
|
181
|
-
* Almost plugins are taken over maintaining from original author.
|
|
182
|
-
* fluent-plugin-map
|
|
183
|
-
* fluent-plugin-redis
|
|
184
|
-
* fluent-plugin-irc etc.
|
|
185
|
-
|
|
186
|
-
# fluent-plugins-nursery
|
|
187
|
-
|
|
188
|
-
* Let's maintain fluentd plugins by community!
|
|
189
|
-
|
|
190
|
-
# Powered By
|
|
191
|
-
|
|
192
|
-
* Rabbit shocker
|
|
193
|
-
|
|
194
|
-
{:relative_height='80'}
|