fluent-plugin-mixpanel-enchanced 0.0.10
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.gitignore +18 -0
- data/.travis.yml +5 -0
- data/Gemfile +4 -0
- data/LICENSE.txt +13 -0
- data/README.md +172 -0
- data/Rakefile +11 -0
- data/example/http_server.sh +2 -0
- data/example/in_http_mixpanel.conf +12 -0
- data/example/index.html +32 -0
- data/fluent-plugin-mixpanel-enchanced.gemspec +27 -0
- data/lib/fluent/plugin/in_http_mixpanel.rb +33 -0
- data/lib/fluent/plugin/mixpanel_ruby_error_handler.rb +15 -0
- data/lib/fluent/plugin/out_mixpanel.rb +127 -0
- data/test/helper.rb +39 -0
- data/test/plugin/test_in_http_mixpanel.rb +129 -0
- data/test/plugin/test_mixpanel_ruby_error_handler.rb +16 -0
- data/test/plugin/test_out_mixpanel.rb +371 -0
- metadata +149 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 27467730f8f9dcfa247c004aa9ce3e0ca374f993
|
4
|
+
data.tar.gz: 71c01ebc5f14ee468f6685c420db82bbd2cb3d64
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 12c4377d98a3a4b57d75583fa8689fd7c0a79706a2d6c035bf01f887b9b392d655f4da36fcc445eec2aa1c66eeea86eed52011d26d928cdec55e091399935e9f
|
7
|
+
data.tar.gz: 9be23b64240e8e057885519a656a4cb2bd09f095f09ce8210705282702b23cbcbfa8dc355ebc2b2097d061c0ccb00864bd4516c4de91467c40c1d3fa218dc492
|
data/.gitignore
ADDED
data/.travis.yml
ADDED
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,13 @@
|
|
1
|
+
Copyright (c) 2014- Kazuyuki Honda
|
2
|
+
|
3
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
4
|
+
you may not use this file except in compliance with the License.
|
5
|
+
You may obtain a copy of the License at
|
6
|
+
|
7
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
8
|
+
|
9
|
+
Unless required by applicable law or agreed to in writing, software
|
10
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
11
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
|
+
See the License for the specific language governing permissions and
|
13
|
+
limitations under the License.
|
data/README.md
ADDED
@@ -0,0 +1,172 @@
|
|
1
|
+
# fluent-plugin-mixpanel
|
2
|
+
|
3
|
+
[![Build Status](https://travis-ci.org/hakobera/fluent-plugin-mixpanel.png?branch=master)](https://travis-ci.org/hakobera/fluent-plugin-mixpanel)
|
4
|
+
|
5
|
+
**CAUTION** This plugin does not support Ruby < 2.0.
|
6
|
+
|
7
|
+
## Component
|
8
|
+
|
9
|
+
### MixpanelOutput
|
10
|
+
|
11
|
+
[Fluentd](http://fluentd.org) plugin to send event track data to [mixpanel](https://mixpanel.com).
|
12
|
+
|
13
|
+
### HttpMixpanelInput
|
14
|
+
|
15
|
+
[Fluentd](http://fluentd.org) plugin to integrate [mixpanel javascript libraries](https://mixpanel.com/docs/integration-libraries/javascript).
|
16
|
+
|
17
|
+
## Installation
|
18
|
+
|
19
|
+
Install with gem or fluent-gem command as:
|
20
|
+
|
21
|
+
```
|
22
|
+
# for fluentd
|
23
|
+
$ gem install fluent-plugin-mixpanel
|
24
|
+
|
25
|
+
# for td-agent
|
26
|
+
$ sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-mixpanel
|
27
|
+
```
|
28
|
+
|
29
|
+
## Configuration
|
30
|
+
|
31
|
+
### MixpanelOutput
|
32
|
+
|
33
|
+
MixpanelOutput needs mixpanel's `project_token`, that can get from your mixpanel project settings.
|
34
|
+
|
35
|
+
#### Use distinct_id_key and event_key
|
36
|
+
|
37
|
+
You should also specify property key name by `distinct_id_key` and `event_key`.
|
38
|
+
|
39
|
+
```
|
40
|
+
<match output.mixpanel.*>
|
41
|
+
type mixpanel
|
42
|
+
project_token YOUR_PROJECT_TOKEN
|
43
|
+
distinct_id_key user_id
|
44
|
+
event_key event_name
|
45
|
+
</match>
|
46
|
+
```
|
47
|
+
|
48
|
+
If record like this:
|
49
|
+
|
50
|
+
```rb
|
51
|
+
{ user_id: "123", event_name: "event1", key1: "value1", key2: "value2" }
|
52
|
+
```
|
53
|
+
|
54
|
+
above settings send to the following data to mixpanel, using [mixpanel-ruby](https://github.com/mixpanel/mixpanel-ruby) gem.
|
55
|
+
|
56
|
+
```rb
|
57
|
+
tracker = Mixpanel::Tracker.new(YOUR_PROJECT_TOKEN)
|
58
|
+
tracker.track("123", "event1", { key1: "value1", key2: "value2" })
|
59
|
+
```
|
60
|
+
|
61
|
+
#### Use distinct_id_key and event_map_tag
|
62
|
+
|
63
|
+
You can use tag name as event name like this. (see additional tag manipulations options below)
|
64
|
+
|
65
|
+
#### Discarding mixpanel errors
|
66
|
+
|
67
|
+
When delivering events to Mixpanel, Fluent creates a chunk of messages to send. By default, if one event fails to send to Mixpanel, all messages in that chunk are requeued for delivery. Enabling `discard_event_on_send_error` allows you to ignore single delivery failures. The event is logged via `info`, including the record being dropped.
|
68
|
+
|
69
|
+
```
|
70
|
+
<match output.mixpanel.*>
|
71
|
+
...
|
72
|
+
discard_event_on_send_error true
|
73
|
+
...
|
74
|
+
</match>
|
75
|
+
```
|
76
|
+
|
77
|
+
##PLEASE NOTE (breaking api change in a future release)
|
78
|
+
|
79
|
+
The api for remove_tag_prefix will be changing in a future release. There is currently a boolean option,
|
80
|
+
use_legacy_prefix_behavior, which will ensure legacy behavior is maintained until that time. Eventually this option will go away
|
81
|
+
as well and the new behavior will be the only way. The difference is pretty simple, the '.' in the prefix needs to be specified.
|
82
|
+
This change allows this plugin to use Fluet's mixin and unifies syntax across plugins. Currently, use_legacy_prefix_behavior
|
83
|
+
defaults to true, which will work either way, but eventually you will need to specify the '.' in your prefix. Again, use_legacy_prefix_behavior simply removes any '.' along with the specified prefix and will behave properly even after you change your configs
|
84
|
+
to be current as seen below. You do not need to set this option.
|
85
|
+
|
86
|
+
```
|
87
|
+
<match output.mixpanel.*>
|
88
|
+
type mixpanel
|
89
|
+
project_token YOUR_PROJECT_TOKEN
|
90
|
+
distinct_id_key user_id
|
91
|
+
remove_tag_prefix output.mixpanel.
|
92
|
+
event_map_tag true
|
93
|
+
</match>
|
94
|
+
```
|
95
|
+
|
96
|
+
If tag name is `output.mixpanel.event1` and record like this:
|
97
|
+
|
98
|
+
```rb
|
99
|
+
{ user_id: "123", key1: "value1", key2: "value2" }
|
100
|
+
```
|
101
|
+
|
102
|
+
above settings send to the following data to mixpanel, using [mixpanel-ruby](https://github.com/mixpanel/mixpanel-ruby) gem.
|
103
|
+
|
104
|
+
```rb
|
105
|
+
tracker = Mixpanel::Tracker.new(YOUR_PROJECT_TOKEN)
|
106
|
+
tracker.track("123", "event1", { key1: "value1", key2: "value2" })
|
107
|
+
```
|
108
|
+
|
109
|
+
#### Use the import method to post instead of track
|
110
|
+
|
111
|
+
You can use tag name as event name like this.
|
112
|
+
|
113
|
+
```
|
114
|
+
<match output.mixpanel.*>
|
115
|
+
type mixpanel
|
116
|
+
project_token YOUR_PROJECT_TOKEN
|
117
|
+
distinct_id_key user_id
|
118
|
+
remove_tag_prefix output.mixpanel.
|
119
|
+
event_map_tag true
|
120
|
+
use_import true
|
121
|
+
api_key YOUR_API_KEY
|
122
|
+
</match>
|
123
|
+
```
|
124
|
+
|
125
|
+
If tag name is `output.mixpanel.event1` and record like this:
|
126
|
+
|
127
|
+
```rb
|
128
|
+
{ user_id: "123", key1: "value1", key2: "value2" }
|
129
|
+
```
|
130
|
+
|
131
|
+
above settings send to the following data to mixpanel, using [mixpanel-ruby](https://github.com/mixpanel/mixpanel-ruby) gem.
|
132
|
+
|
133
|
+
```rb
|
134
|
+
tracker = Mixpanel::Tracker.new(YOUR_PROJECT_TOKEN)
|
135
|
+
tracker.import(api_key, "123", "event1", { key1: "value1", key2: "value2" })
|
136
|
+
```
|
137
|
+
|
138
|
+
---
|
139
|
+
|
140
|
+
fluentd-plugin-mixpanel also includes the HandleTagNameMixin mixin which allows the following additional options:
|
141
|
+
|
142
|
+
```
|
143
|
+
remove_tag_prefix <tag_prefix_to_remove_including_the_dot>
|
144
|
+
remove_tag_suffix <tag_suffix_to_remove_including_the_dot>
|
145
|
+
add_tag_prefix <tag_prefix_to_add_including_the_dot>
|
146
|
+
add_tag_suffix <tag_suffix_to_add_including_the_dot>
|
147
|
+
```
|
148
|
+
|
149
|
+
### HttpMixpanelInput
|
150
|
+
|
151
|
+
HttpMixpanelInput has same configuration as [http Input Plugin](http://docs.fluentd.org/en/articles/in_http).
|
152
|
+
|
153
|
+
```
|
154
|
+
<source>
|
155
|
+
type http_mixpanel
|
156
|
+
bind 127.0.0.1
|
157
|
+
port 8888
|
158
|
+
body_size_limit 10m
|
159
|
+
keepalive_timeout 5
|
160
|
+
add_http_headers true
|
161
|
+
</source>
|
162
|
+
```
|
163
|
+
|
164
|
+
In example folder, you can see example configuration and HTML.
|
165
|
+
|
166
|
+
## Contributing
|
167
|
+
|
168
|
+
1. Fork it ( http://github.com/hakobera/fluent-plugin-mixpanel/fork )
|
169
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
170
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
171
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
172
|
+
5. Create new Pull Request
|
data/Rakefile
ADDED
data/example/index.html
ADDED
@@ -0,0 +1,32 @@
|
|
1
|
+
<!doctype html>
|
2
|
+
<html>
|
3
|
+
<head>
|
4
|
+
<meta charset="utf-8">
|
5
|
+
<title>in_http_mixpanel example</title>
|
6
|
+
</head>
|
7
|
+
<body>
|
8
|
+
<ul>
|
9
|
+
<li><button data-event='event1' data-value="1">Event1</button></li>
|
10
|
+
<li><button data-event='event2' data-value="2">Event2</button></li>
|
11
|
+
<li><button data-event='event3' data-value="3">Event3</button></li>
|
12
|
+
</ul>
|
13
|
+
<script type="text/javascript">
|
14
|
+
(function(c,a){window.mixpanel=a;var b,d,h,e;b=c.createElement("script");
|
15
|
+
b.type="text/javascript";b.async=!0;b.src=("https:"===c.location.protocol?"https:":"http:")+'//cdn.mxpnl.com/libs/mixpanel-2.2.min.js';d=c.getElementsByTagName("script")[0];d.parentNode.insertBefore(b,d);a._i=[];a.init=function(b,c,f){function d(a,b){var c=b.split(".");2==c.length&&(a=a[c[0]],b=c[1]);a[b]=function(){a.push([b].concat(Array.prototype.slice.call(arguments,0)))}}var g=a;"undefined"!==typeof f?g=a[f]=[]:f="mixpanel";g.people=g.people||[];h=['disable','track','track_pageview','track_links','track_forms','register','register_once','unregister','identify','alias','name_tag','set_config','people.set','people.set_once','people.increment','people.track_charge','people.append'];for(e=0;e<h.length;e++)d(g,h[e]);a._i.push([b,c,f])};a.__SV=1.2;})(document,window.mixpanel||[]);
|
16
|
+
mixpanel.init("dummy", {
|
17
|
+
api_host: '//0.0.0.0:5000',
|
18
|
+
debug: true
|
19
|
+
});
|
20
|
+
</script>
|
21
|
+
<script src="http://code.jquery.com/jquery-1.11.0.min.js"></script>
|
22
|
+
<script>
|
23
|
+
$(function () {
|
24
|
+
$('button').on('click', function () {
|
25
|
+
var evt = $(this).data('event');
|
26
|
+
var value = $(this).data('value');
|
27
|
+
mixpanel.track(evt, { value: value });
|
28
|
+
});
|
29
|
+
});
|
30
|
+
</script>
|
31
|
+
</body>
|
32
|
+
</html>
|
@@ -0,0 +1,27 @@
|
|
1
|
+
# -*- encoding: utf-8 -*-
|
2
|
+
lib = File.expand_path('../lib', __FILE__)
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
+
|
5
|
+
Gem::Specification.new do |spec|
|
6
|
+
spec.name = "fluent-plugin-mixpanel-enchanced"
|
7
|
+
spec.version = "0.0.10"
|
8
|
+
spec.authors = ["zhron4x"]
|
9
|
+
spec.email = ["zhr0n4x@gmail.com"]
|
10
|
+
spec.summary = %q{Fluentd plugin to input/output event track data to mixpanel}
|
11
|
+
spec.description = %q{Fluentd plugin to input/output event track data to mixpanel}
|
12
|
+
spec.homepage = "https://github.com/zhron4x/fluent-plugin-mixpanel-enchanced"
|
13
|
+
spec.license = "Apache License, Version 2.0"
|
14
|
+
|
15
|
+
spec.files = `git ls-files -z`.split("\x0")
|
16
|
+
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
17
|
+
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
|
18
|
+
spec.require_paths = ["lib"]
|
19
|
+
|
20
|
+
spec.add_runtime_dependency "fluentd", ">= 0.10.55"
|
21
|
+
spec.add_runtime_dependency "mixpanel-ruby", "~> 2.2.0"
|
22
|
+
|
23
|
+
spec.add_development_dependency "rake"
|
24
|
+
spec.add_development_dependency "webmock"
|
25
|
+
spec.add_development_dependency "test-unit", "~> 3.0.2"
|
26
|
+
spec.add_development_dependency "pry"
|
27
|
+
end
|
@@ -0,0 +1,33 @@
|
|
1
|
+
require 'fluent/plugin/in_http'
|
2
|
+
require 'base64'
|
3
|
+
require 'pry'
|
4
|
+
|
5
|
+
class Fluent::HttpMixpanelInput < Fluent::HttpInput
|
6
|
+
Fluent::Plugin.register_input('http_mixpanel', self)
|
7
|
+
|
8
|
+
config_param :tag_prefix, :default => 'mixpanel'
|
9
|
+
|
10
|
+
def on_request(path_info, params)
|
11
|
+
data = Base64.decode64(params['data']).force_encoding('utf-8')
|
12
|
+
json = JSON.parse(data)
|
13
|
+
binding.pry
|
14
|
+
props = json['properties']
|
15
|
+
path = "/#{tag_prefix}.#{json['event']}"
|
16
|
+
params['json'] = props.to_json
|
17
|
+
params['time'] = props['time'].to_s if props['time']
|
18
|
+
|
19
|
+
ret = super(path, params)
|
20
|
+
|
21
|
+
headers = {
|
22
|
+
'Access-Control-Allow-Credentials' => true,
|
23
|
+
'Access-Control-Allow-Headers' => 'X-Requested-With',
|
24
|
+
'Access-Control-Allow-Methods' => 'GET, POST, OPTIONS',
|
25
|
+
'Access-Control-Allow-Origin' => params['HTTP_ORIGIN'],
|
26
|
+
'Access-Control-Max-Age' => 1728000,
|
27
|
+
'Cache-Control' => 'no-cache, no-store',
|
28
|
+
'Content-type' => 'text/plain'
|
29
|
+
}
|
30
|
+
|
31
|
+
[ret[0], headers, (ret[0] == '200 OK' ? '1' : '0')]
|
32
|
+
end
|
33
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
require 'mixpanel-ruby'
|
2
|
+
|
3
|
+
class Fluent::MixpanelOutputErrorHandler < Mixpanel::ErrorHandler
|
4
|
+
def initialize(logger)
|
5
|
+
@logger = logger
|
6
|
+
end
|
7
|
+
|
8
|
+
def handle(error)
|
9
|
+
# Default behavior is to not return an error. Mixpanel-ruby gem returns
|
10
|
+
# true/false. If there is an error, an optional error handler is called.
|
11
|
+
# In this case, here, we only want to log the error for future development
|
12
|
+
# of error handling.
|
13
|
+
@logger.error "MixpanelOutputErrorHandler:\n\tClass: #{error.class.to_s}\n\tMessage: #{error.message}\n\tBacktrace: #{error.backtrace}"
|
14
|
+
end
|
15
|
+
end
|
@@ -0,0 +1,127 @@
|
|
1
|
+
require_relative "mixpanel_ruby_error_handler.rb"
|
2
|
+
|
3
|
+
class Fluent::MixpanelOutput < Fluent::BufferedOutput
|
4
|
+
Fluent::Plugin.register_output('mixpanel', self)
|
5
|
+
|
6
|
+
include Fluent::HandleTagNameMixin
|
7
|
+
|
8
|
+
config_param :project_token, :string, :secret => true
|
9
|
+
config_param :api_key, :string, :default => '', :secret => true
|
10
|
+
config_param :use_import, :bool, :default => nil
|
11
|
+
config_param :distinct_id_key, :string
|
12
|
+
config_param :event_key, :string, :default => nil
|
13
|
+
config_param :ip_key, :string, :default => nil
|
14
|
+
config_param :event_map_tag, :bool, :default => false
|
15
|
+
#NOTE: This will be removed in a future release. Please specify the '.' on any prefix
|
16
|
+
config_param :use_legacy_prefix_behavior, :default => true
|
17
|
+
config_param :discard_event_on_send_error, :default => false
|
18
|
+
|
19
|
+
class MixpanelError < StandardError
|
20
|
+
end
|
21
|
+
|
22
|
+
def initialize
|
23
|
+
super
|
24
|
+
require 'mixpanel-ruby'
|
25
|
+
end
|
26
|
+
|
27
|
+
def configure(conf)
|
28
|
+
super
|
29
|
+
@project_tokey = conf['project_token']
|
30
|
+
@distinct_id_key = conf['distinct_id_key']
|
31
|
+
@event_key = conf['event_key']
|
32
|
+
@ip_key = conf['ip_key']
|
33
|
+
@event_map_tag = conf['event_map_tag']
|
34
|
+
@api_key = conf['api_key']
|
35
|
+
@use_import = conf['use_import']
|
36
|
+
@use_legacy_prefix_behavior = conf['use_legacy_prefix_behavior']
|
37
|
+
@discard_event_on_send_error = conf['discard_event_on_send_error']
|
38
|
+
|
39
|
+
if @event_key.nil? and !@event_map_tag
|
40
|
+
raise Fluent::ConfigError, "'event_key' must be specifed when event_map_tag == false."
|
41
|
+
end
|
42
|
+
end
|
43
|
+
|
44
|
+
def start
|
45
|
+
super
|
46
|
+
error_handler = Fluent::MixpanelOutputErrorHandler.new(log)
|
47
|
+
@tracker = Mixpanel::Tracker.new(@project_token, error_handler)
|
48
|
+
end
|
49
|
+
|
50
|
+
def shutdown
|
51
|
+
super
|
52
|
+
end
|
53
|
+
|
54
|
+
def format(tag, time, record)
|
55
|
+
time = record['time'] if record['time'] && @use_import
|
56
|
+
[tag, time, record].to_msgpack
|
57
|
+
end
|
58
|
+
|
59
|
+
def write(chunk)
|
60
|
+
records = []
|
61
|
+
chunk.msgpack_each do |tag, time, record|
|
62
|
+
data = {}
|
63
|
+
prop = data['properties'] = record.dup
|
64
|
+
|
65
|
+
# Ignore token in record
|
66
|
+
prop.delete('token')
|
67
|
+
|
68
|
+
if @event_map_tag
|
69
|
+
tag.gsub!(/^\./, '') if @use_legacy_prefix_behavior
|
70
|
+
data['event'] = tag
|
71
|
+
elsif record[@event_key]
|
72
|
+
data['event'] = record[@event_key]
|
73
|
+
prop.delete(@event_key)
|
74
|
+
else
|
75
|
+
log.warn("no event, tag: #{tag}, time: #{time.to_s}, record: #{record.to_json}")
|
76
|
+
next
|
77
|
+
end
|
78
|
+
|
79
|
+
# Ignore browswer only special event
|
80
|
+
next if data['event'].start_with?('mp_')
|
81
|
+
|
82
|
+
if record[@distinct_id_key]
|
83
|
+
data['distinct_id'] = record[@distinct_id_key]
|
84
|
+
prop.delete(@distinct_id_key)
|
85
|
+
else
|
86
|
+
log.warn("no distinct_id, tag: #{tag}, time: #{time.to_s}, record: #{record.to_json}")
|
87
|
+
next
|
88
|
+
end
|
89
|
+
|
90
|
+
if !@ip_key.nil? and record[@ip_key]
|
91
|
+
prop['ip'] = record[@ip_key]
|
92
|
+
prop.delete(@ip_key)
|
93
|
+
end
|
94
|
+
|
95
|
+
prop.select! {|key, _| !key.start_with?('mp_') }
|
96
|
+
prop.merge!('time' => time.to_i)
|
97
|
+
|
98
|
+
records << data
|
99
|
+
end
|
100
|
+
|
101
|
+
send_to_mixpanel(records)
|
102
|
+
end
|
103
|
+
|
104
|
+
def send_to_mixpanel(records)
|
105
|
+
log.debug("sending #{records.length} to mixpanel")
|
106
|
+
|
107
|
+
records.each do |record|
|
108
|
+
success = true
|
109
|
+
|
110
|
+
if @use_import
|
111
|
+
success = @tracker.import(@api_key, record['distinct_id'], record['event'], record['properties'])
|
112
|
+
else
|
113
|
+
success = @tracker.track(record['distinct_id'], record['event'], record['properties'])
|
114
|
+
end
|
115
|
+
|
116
|
+
unless success
|
117
|
+
if @discard_event_on_send_error
|
118
|
+
msg = "Failed to track event to mixpanel:\n"
|
119
|
+
msg += "\tRecord: #{record.to_json}"
|
120
|
+
log.info(msg)
|
121
|
+
else
|
122
|
+
raise MixpanelError.new("Failed to track event to mixpanel")
|
123
|
+
end
|
124
|
+
end
|
125
|
+
end
|
126
|
+
end
|
127
|
+
end
|
data/test/helper.rb
ADDED
@@ -0,0 +1,39 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
require 'bundler'
|
3
|
+
begin
|
4
|
+
Bundler.setup(:default, :development)
|
5
|
+
rescue Bundler::BundlerError => e
|
6
|
+
$stderr.puts e.message
|
7
|
+
$stderr.puts "Run `bundle install` to install missing gems"
|
8
|
+
exit e.status_code
|
9
|
+
end
|
10
|
+
require 'test/unit'
|
11
|
+
require 'webmock/test_unit'
|
12
|
+
WebMock.disable_net_connect!(:allow_localhost => true)
|
13
|
+
|
14
|
+
$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
|
15
|
+
$LOAD_PATH.unshift(File.dirname(__FILE__))
|
16
|
+
require 'fluent/test'
|
17
|
+
unless ENV.has_key?('VERBOSE')
|
18
|
+
nulllogger = Object.new
|
19
|
+
nulllogger.instance_eval {|obj|
|
20
|
+
def method_missing(method, *args)
|
21
|
+
# pass
|
22
|
+
end
|
23
|
+
}
|
24
|
+
$log = nulllogger
|
25
|
+
end
|
26
|
+
|
27
|
+
def unused_port
|
28
|
+
s = TCPServer.open(0)
|
29
|
+
port = s.addr[1]
|
30
|
+
s.close
|
31
|
+
port
|
32
|
+
end
|
33
|
+
|
34
|
+
require 'fluent/plugin/mixpanel_ruby_error_handler'
|
35
|
+
require 'fluent/plugin/out_mixpanel'
|
36
|
+
require 'fluent/plugin/in_http_mixpanel'
|
37
|
+
|
38
|
+
class Test::Unit::TestCase
|
39
|
+
end
|
@@ -0,0 +1,129 @@
|
|
1
|
+
require 'helper'
|
2
|
+
require 'net/http'
|
3
|
+
require 'base64'
|
4
|
+
require 'fluent/test'
|
5
|
+
require 'net/http'
|
6
|
+
|
7
|
+
class HttpMixpanelInputTest < Test::Unit::TestCase
|
8
|
+
|
9
|
+
class << self
|
10
|
+
def startup
|
11
|
+
socket_manager_path = ServerEngine::SocketManager::Server.generate_path
|
12
|
+
@server = ServerEngine::SocketManager::Server.open(socket_manager_path)
|
13
|
+
ENV['SERVERENGINE_SOCKETMANAGER_PATH'] = socket_manager_path.to_s
|
14
|
+
end
|
15
|
+
|
16
|
+
def shutdown
|
17
|
+
@server.close
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
def setup
|
22
|
+
Fluent::Test.setup
|
23
|
+
end
|
24
|
+
|
25
|
+
PORT = unused_port
|
26
|
+
CONFIG = %[
|
27
|
+
port #{PORT}
|
28
|
+
bind "127.0.0.1"
|
29
|
+
body_size_limit 10m
|
30
|
+
keepalive_timeout 5
|
31
|
+
respond_with_empty_img true
|
32
|
+
]
|
33
|
+
|
34
|
+
def create_driver(conf=CONFIG)
|
35
|
+
Fluent::Test::InputTestDriver.new(Fluent::HttpMixpanelInput).configure(conf, true)
|
36
|
+
end
|
37
|
+
|
38
|
+
def test_configure
|
39
|
+
d = create_driver
|
40
|
+
assert_equal PORT, d.instance.port
|
41
|
+
assert_equal '127.0.0.1', d.instance.bind
|
42
|
+
assert_equal 10*1024*1024, d.instance.body_size_limit
|
43
|
+
assert_equal 5, d.instance.keepalive_timeout
|
44
|
+
assert_equal false, d.instance.add_http_headers
|
45
|
+
end
|
46
|
+
|
47
|
+
def test_time
|
48
|
+
d = create_driver
|
49
|
+
|
50
|
+
time = Time.parse("2011-01-02 13:14:15 UTC").to_i
|
51
|
+
Fluent::Engine.now = time
|
52
|
+
|
53
|
+
d.expect_emit "mixpanel.tag1", time, {"a"=>1}
|
54
|
+
d.expect_emit "mixpanel.tag2", time, {"a"=>2}
|
55
|
+
|
56
|
+
d.run do
|
57
|
+
d.expected_emits.each {|tag, record_time, record|
|
58
|
+
res = track("#{tag}", {"json"=>record})
|
59
|
+
assert_equal "200", res.code
|
60
|
+
assert_equal '1', res.body
|
61
|
+
assert_equal 'true', res['access-control-allow-credentials']
|
62
|
+
assert_equal 'X-Requested-With', res['access-control-allow-headers']
|
63
|
+
assert_equal 'GET, POST, OPTIONS', res['access-control-allow-methods']
|
64
|
+
assert_equal 'http://foo.example', res['access-control-allow-origin']
|
65
|
+
assert_equal '1728000', res['access-control-max-age']
|
66
|
+
assert_equal 'no-cache, no-store', res['cache-control']
|
67
|
+
}
|
68
|
+
end
|
69
|
+
end
|
70
|
+
|
71
|
+
def test_json
|
72
|
+
d = create_driver
|
73
|
+
|
74
|
+
time = Time.parse("2011-01-02 13:14:15 UTC").to_i
|
75
|
+
|
76
|
+
d.expect_emit "mixpanel.tag1", time, {"a"=>1}
|
77
|
+
d.expect_emit "mixpanel.tag2", time, {"a"=>2}
|
78
|
+
|
79
|
+
d.run do
|
80
|
+
d.expected_emits.each {|tag, record_time, record|
|
81
|
+
res = track("#{tag}", {"json"=>record, "time"=>record_time.to_s})
|
82
|
+
assert_equal "200", res.code
|
83
|
+
}
|
84
|
+
end
|
85
|
+
|
86
|
+
d.emit_streams.each { |tag, es|
|
87
|
+
assert !include_http_header?(es.first[1])
|
88
|
+
}
|
89
|
+
end
|
90
|
+
|
91
|
+
def test_json_with_add_http_headers
|
92
|
+
d = create_driver(CONFIG + "add_http_headers true")
|
93
|
+
|
94
|
+
time = Time.parse("2011-01-02 13:14:15 UTC").to_i
|
95
|
+
|
96
|
+
records = [["mixpanel.tag1", time, {"a"=>1}], ["mixpanel.tag2", time, {"a"=>2}]]
|
97
|
+
|
98
|
+
d.run do
|
99
|
+
records.each {|tag, record_time, record|
|
100
|
+
res = track("#{tag}", {"json"=>record, "time"=>record_time.to_s})
|
101
|
+
assert_equal "200", res.code
|
102
|
+
}
|
103
|
+
end
|
104
|
+
|
105
|
+
d.emit_streams.each { |tag, es|
|
106
|
+
assert include_http_header?(es.first[1])
|
107
|
+
}
|
108
|
+
end
|
109
|
+
|
110
|
+
def track(tag, params)
|
111
|
+
event = tag.sub(/^mixpanel\.(.+)$/, '\1')
|
112
|
+
params['json']['time'] = params['time'] if params['time']
|
113
|
+
data = {
|
114
|
+
event: event,
|
115
|
+
properties: params['json']
|
116
|
+
}
|
117
|
+
data = CGI.escape(Base64.encode64(data.to_json))
|
118
|
+
query = "data=#{data}"
|
119
|
+
path = "/track/?#{query}"
|
120
|
+
|
121
|
+
http = Net::HTTP.new("127.0.0.1", PORT)
|
122
|
+
req = Net::HTTP::Get.new(path, { 'origin' => 'http://foo.example' })
|
123
|
+
http.request(req)
|
124
|
+
end
|
125
|
+
|
126
|
+
def include_http_header?(record)
|
127
|
+
record.keys.find { |header| header.start_with?('HTTP_') }
|
128
|
+
end
|
129
|
+
end
|
@@ -0,0 +1,16 @@
|
|
1
|
+
require 'helper'
|
2
|
+
|
3
|
+
class MixpanelRubyErrorHandlerTest < Test::Unit::TestCase
|
4
|
+
|
5
|
+
def test_handle
|
6
|
+
mixpanel_error = Fluent::MixpanelOutput::MixpanelError.new("Foobar failed")
|
7
|
+
@io = StringIO.new
|
8
|
+
logger = Logger.new(@io)
|
9
|
+
error = Fluent::MixpanelOutputErrorHandler.new(logger)
|
10
|
+
error.handle(mixpanel_error)
|
11
|
+
|
12
|
+
output = @io.string
|
13
|
+
expected_output = "MixpanelOutputErrorHandler:\n\tClass: Fluent::MixpanelOutput::MixpanelError\n\tMessage: Foobar failed\n\tBacktrace: \n"
|
14
|
+
assert_match expected_output, output
|
15
|
+
end
|
16
|
+
end
|
@@ -0,0 +1,371 @@
|
|
1
|
+
require 'helper'
|
2
|
+
require 'uri'
|
3
|
+
require 'msgpack'
|
4
|
+
|
5
|
+
class MixpanelOutputTest < Test::Unit::TestCase
|
6
|
+
|
7
|
+
def setup
|
8
|
+
Fluent::Test.setup
|
9
|
+
@out = []
|
10
|
+
end
|
11
|
+
|
12
|
+
CONFIG = %[
|
13
|
+
project_token test_token
|
14
|
+
distinct_id_key user_id
|
15
|
+
]
|
16
|
+
|
17
|
+
IMPORT_CONFIG = CONFIG + %[ api_key test_api_key
|
18
|
+
use_import true
|
19
|
+
use_legacy_prefix_behavior false
|
20
|
+
]
|
21
|
+
|
22
|
+
def create_driver(conf = CONFIG)
|
23
|
+
Fluent::Test::BufferedOutputTestDriver.new(Fluent::MixpanelOutput, 'mixpanel.test').configure(conf)
|
24
|
+
end
|
25
|
+
|
26
|
+
def stub_mixpanel(url="https://api.mixpanel.com/track")
|
27
|
+
stub_request(:post, url).with do |req|
|
28
|
+
body = URI.decode_www_form(req.body)
|
29
|
+
@out << JSON.load(Base64.decode64(body.assoc('data').last))
|
30
|
+
end.to_return(status: 200, body: JSON.generate({ status: 1 }))
|
31
|
+
end
|
32
|
+
|
33
|
+
def stub_mixpanel_import
|
34
|
+
stub_mixpanel("https://api.mixpanel.com/import")
|
35
|
+
end
|
36
|
+
|
37
|
+
def stub_mixpanel_unavailable(url="https://api.mixpanel.com/track")
|
38
|
+
stub_request(:post, url).to_return(status: 503, body: "Service Unavailable")
|
39
|
+
end
|
40
|
+
|
41
|
+
def sample_record
|
42
|
+
{ user_id: "123", event: "event1", key1: "value1", key2: "value2" }
|
43
|
+
end
|
44
|
+
|
45
|
+
def test_configure
|
46
|
+
d = create_driver(CONFIG + "event_key event")
|
47
|
+
|
48
|
+
assert_equal 'test_token', d.instance.project_token
|
49
|
+
assert_equal 'user_id', d.instance.distinct_id_key
|
50
|
+
assert_equal 'event', d.instance.event_key
|
51
|
+
end
|
52
|
+
|
53
|
+
def test_configure_with_ip_key
|
54
|
+
d = create_driver(CONFIG + "event_key event\n ip_key ip")
|
55
|
+
|
56
|
+
assert_equal 'test_token', d.instance.project_token
|
57
|
+
assert_equal 'user_id', d.instance.distinct_id_key
|
58
|
+
assert_equal 'event', d.instance.event_key
|
59
|
+
assert_equal 'ip', d.instance.ip_key
|
60
|
+
end
|
61
|
+
|
62
|
+
def test_configure_with_event_map_tag
|
63
|
+
d = create_driver(CONFIG + "event_map_tag true")
|
64
|
+
|
65
|
+
assert_equal 'test_token', d.instance.project_token
|
66
|
+
assert_equal 'user_id', d.instance.distinct_id_key
|
67
|
+
assert_equal nil, d.instance.event_key
|
68
|
+
assert_equal true.to_s, d.instance.event_map_tag
|
69
|
+
end
|
70
|
+
|
71
|
+
def test_write
|
72
|
+
stub_mixpanel
|
73
|
+
d = create_driver(CONFIG + "event_key event")
|
74
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
75
|
+
d.emit(sample_record, time)
|
76
|
+
d.run
|
77
|
+
|
78
|
+
assert_equal "test_token", @out[0]['properties']['token']
|
79
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
80
|
+
assert_equal "event1", @out[0]['event']
|
81
|
+
assert_equal time, @out[0]['properties']['time']
|
82
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
83
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
84
|
+
end
|
85
|
+
|
86
|
+
def test_write_setting_time_via_export
|
87
|
+
stub_mixpanel_import
|
88
|
+
d = create_driver(CONFIG + "use_import true\nevent_key event")
|
89
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
90
|
+
d.emit(sample_record.merge!('time' => 1435707767), time)
|
91
|
+
d.run
|
92
|
+
|
93
|
+
assert_equal "test_token", @out[0]['properties']['token']
|
94
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
95
|
+
assert_equal "event1", @out[0]['event']
|
96
|
+
assert_equal 1435707767, @out[0]['properties']['time']
|
97
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
98
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
99
|
+
end
|
100
|
+
|
101
|
+
def test_write_multi_request
|
102
|
+
stub_mixpanel_import
|
103
|
+
d = create_driver(IMPORT_CONFIG + "event_key event")
|
104
|
+
time1 = Time.new('2014-01-01T01:23:45+00:00').to_i
|
105
|
+
time2 = Time.new('2014-01-02T01:23:45+00:00').to_i
|
106
|
+
|
107
|
+
d.emit(sample_record, time1)
|
108
|
+
d.emit(sample_record.merge(key3: "value3"), time2)
|
109
|
+
d.run
|
110
|
+
|
111
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
112
|
+
assert_equal "event1", @out[0]['event']
|
113
|
+
assert_equal time1, @out[0]['properties']['time']
|
114
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
115
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
116
|
+
|
117
|
+
assert_equal "123", @out[1]['properties']['distinct_id']
|
118
|
+
assert_equal "event1", @out[1]['event']
|
119
|
+
assert_equal time2, @out[1]['properties']['time']
|
120
|
+
assert_equal "value1", @out[1]['properties']['key1']
|
121
|
+
assert_equal "value2", @out[1]['properties']['key2']
|
122
|
+
assert_equal "value2", @out[1]['properties']['key2']
|
123
|
+
end
|
124
|
+
|
125
|
+
def test_write_with_ip_key
|
126
|
+
stub_mixpanel
|
127
|
+
d = create_driver(CONFIG + "event_key event\n ip_key ip_address")
|
128
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
129
|
+
d.emit(sample_record.merge('ip_address' => '192.168.0.2'), time)
|
130
|
+
d.run
|
131
|
+
|
132
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
133
|
+
assert_equal "event1", @out[0]['event']
|
134
|
+
assert_equal time, @out[0]['properties']['time']
|
135
|
+
assert_equal "192.168.0.2", @out[0]['properties']['ip']
|
136
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
137
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
138
|
+
end
|
139
|
+
|
140
|
+
def test_write_with_no_tag_manipulation
|
141
|
+
stub_mixpanel
|
142
|
+
d = create_driver(CONFIG + "event_map_tag true")
|
143
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
144
|
+
d.emit(sample_record, time)
|
145
|
+
d.run
|
146
|
+
|
147
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
148
|
+
assert_equal "mixpanel.test", @out[0]['event']
|
149
|
+
assert_equal time, @out[0]['properties']['time']
|
150
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
151
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
152
|
+
end
|
153
|
+
|
154
|
+
def test_write_with_event_map_tag_removing_prefix
|
155
|
+
stub_mixpanel
|
156
|
+
d = create_driver(CONFIG + "remove_tag_prefix mixpanel.\n event_map_tag true")
|
157
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
158
|
+
d.emit(sample_record, time)
|
159
|
+
d.run
|
160
|
+
|
161
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
162
|
+
assert_equal "test", @out[0]['event']
|
163
|
+
assert_equal time, @out[0]['properties']['time']
|
164
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
165
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
166
|
+
end
|
167
|
+
|
168
|
+
def test_write_with_event_map_tag_removing_prefix_LEGACY
|
169
|
+
stub_mixpanel
|
170
|
+
d = create_driver(CONFIG + "remove_tag_prefix mixpanel\n event_map_tag true\n use_legacy_prefix_behavior true")
|
171
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
172
|
+
d.emit(sample_record, time)
|
173
|
+
d.run
|
174
|
+
|
175
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
176
|
+
assert_equal "test", @out[0]['event']
|
177
|
+
assert_equal time, @out[0]['properties']['time']
|
178
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
179
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
180
|
+
end
|
181
|
+
|
182
|
+
def test_write_with_event_map_tag_removing_prefix_LEGACY_with_dot
|
183
|
+
stub_mixpanel
|
184
|
+
d = create_driver(CONFIG + "remove_tag_prefix mixpanel.\n event_map_tag true\n use_legacy_prefix_behavior true")
|
185
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
186
|
+
d.emit(sample_record, time)
|
187
|
+
d.run
|
188
|
+
|
189
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
190
|
+
assert_equal "test", @out[0]['event']
|
191
|
+
assert_equal time, @out[0]['properties']['time']
|
192
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
193
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
194
|
+
end
|
195
|
+
|
196
|
+
def test_write_with_event_map_tag_removing_suffix
|
197
|
+
stub_mixpanel
|
198
|
+
d = create_driver(CONFIG + "remove_tag_suffix .test\n event_map_tag true")
|
199
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
200
|
+
d.emit(sample_record, time)
|
201
|
+
d.run
|
202
|
+
|
203
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
204
|
+
assert_equal "mixpanel", @out[0]['event']
|
205
|
+
assert_equal time, @out[0]['properties']['time']
|
206
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
207
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
208
|
+
end
|
209
|
+
|
210
|
+
def test_write_with_event_map_tag_adding_prefix
|
211
|
+
stub_mixpanel
|
212
|
+
d = create_driver(CONFIG + "add_tag_prefix foo.\n event_map_tag true")
|
213
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
214
|
+
d.emit(sample_record, time)
|
215
|
+
d.run
|
216
|
+
|
217
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
218
|
+
assert_equal "foo.mixpanel.test", @out[0]['event']
|
219
|
+
assert_equal time, @out[0]['properties']['time']
|
220
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
221
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
222
|
+
end
|
223
|
+
|
224
|
+
def test_write_with_event_map_tag_adding_suffix
|
225
|
+
stub_mixpanel
|
226
|
+
d = create_driver(CONFIG + "add_tag_suffix .foo\n event_map_tag true")
|
227
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
228
|
+
d.emit(sample_record, time)
|
229
|
+
d.run
|
230
|
+
|
231
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
232
|
+
assert_equal "mixpanel.test.foo", @out[0]['event']
|
233
|
+
assert_equal time, @out[0]['properties']['time']
|
234
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
235
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
236
|
+
end
|
237
|
+
|
238
|
+
def test_write_ignore_special_event
|
239
|
+
stub_mixpanel
|
240
|
+
d = create_driver(CONFIG + "event_key event")
|
241
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
242
|
+
d.emit({ user_id: '123', event: 'mp_page_view' }, time)
|
243
|
+
d.run
|
244
|
+
|
245
|
+
assert_equal 0, @out.length
|
246
|
+
end
|
247
|
+
|
248
|
+
def test_write_ignore_special_property
|
249
|
+
stub_mixpanel
|
250
|
+
d = create_driver(CONFIG + "event_key event")
|
251
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
252
|
+
d.emit(sample_record.merge('mp_event' => '3'), time)
|
253
|
+
d.run
|
254
|
+
|
255
|
+
assert_equal "test_token", @out[0]['properties']['token']
|
256
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
257
|
+
assert_equal "event1", @out[0]['event']
|
258
|
+
assert_equal time, @out[0]['properties']['time']
|
259
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
260
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
261
|
+
assert_equal false, @out[0]['properties'].key?('mp_event')
|
262
|
+
end
|
263
|
+
|
264
|
+
def test_write_delete_supried_token
|
265
|
+
stub_mixpanel
|
266
|
+
d = create_driver(CONFIG + "event_key event")
|
267
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
268
|
+
d.emit(sample_record.merge('token' => '123'), time)
|
269
|
+
d.run
|
270
|
+
|
271
|
+
assert_equal "test_token", @out[0]['properties']['token']
|
272
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
273
|
+
assert_equal "event1", @out[0]['event']
|
274
|
+
assert_equal time, @out[0]['properties']['time']
|
275
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
276
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
277
|
+
assert_equal false, @out[0]['properties'].key?('mp_event')
|
278
|
+
end
|
279
|
+
|
280
|
+
def test_request_error
|
281
|
+
stub_mixpanel_unavailable
|
282
|
+
d = create_driver(CONFIG + "event_key event")
|
283
|
+
d.emit(sample_record)
|
284
|
+
assert_raise(Fluent::MixpanelOutput::MixpanelError) {
|
285
|
+
d.run
|
286
|
+
}
|
287
|
+
end
|
288
|
+
|
289
|
+
def test_multiple_records_1_missing_event
|
290
|
+
stub_mixpanel
|
291
|
+
d = create_driver(CONFIG + "event_key event")
|
292
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
293
|
+
d.emit(sample_record, time)
|
294
|
+
|
295
|
+
broken_record = sample_record.dup.delete(:event)
|
296
|
+
d.emit(broken_record, time)
|
297
|
+
|
298
|
+
d.run
|
299
|
+
|
300
|
+
assert_equal 1, @out.length
|
301
|
+
|
302
|
+
assert_equal "test_token", @out[0]['properties']['token']
|
303
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
304
|
+
assert_equal "event1", @out[0]['event']
|
305
|
+
assert_equal time, @out[0]['properties']['time']
|
306
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
307
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
308
|
+
|
309
|
+
end
|
310
|
+
|
311
|
+
def test_multiple_records_1_missing_distinct_id
|
312
|
+
stub_mixpanel
|
313
|
+
d = create_driver(CONFIG + "event_key event")
|
314
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
315
|
+
d.emit(sample_record, time)
|
316
|
+
|
317
|
+
broken_record = sample_record.dup.delete(:user_id)
|
318
|
+
d.emit(broken_record, time)
|
319
|
+
|
320
|
+
d.run
|
321
|
+
|
322
|
+
assert_equal 1, @out.length
|
323
|
+
|
324
|
+
assert_equal "test_token", @out[0]['properties']['token']
|
325
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
326
|
+
assert_equal "event1", @out[0]['event']
|
327
|
+
assert_equal time, @out[0]['properties']['time']
|
328
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
329
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
330
|
+
|
331
|
+
end
|
332
|
+
|
333
|
+
def test_multiple_records_1_having_mp
|
334
|
+
stub_mixpanel
|
335
|
+
d = create_driver(CONFIG + "event_key event")
|
336
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
337
|
+
d.emit(sample_record, time)
|
338
|
+
|
339
|
+
broken_record = sample_record.dup.merge({ event: 'mp_foo'})
|
340
|
+
d.emit(broken_record, time)
|
341
|
+
|
342
|
+
d.run
|
343
|
+
|
344
|
+
assert_equal 1, @out.length
|
345
|
+
|
346
|
+
assert_equal "test_token", @out[0]['properties']['token']
|
347
|
+
assert_equal "123", @out[0]['properties']['distinct_id']
|
348
|
+
assert_equal "event1", @out[0]['event']
|
349
|
+
assert_equal time, @out[0]['properties']['time']
|
350
|
+
assert_equal "value1", @out[0]['properties']['key1']
|
351
|
+
assert_equal "value2", @out[0]['properties']['key2']
|
352
|
+
|
353
|
+
end
|
354
|
+
|
355
|
+
def test_request_error_discard
|
356
|
+
stub_mixpanel_unavailable
|
357
|
+
d = create_driver(CONFIG + "event_key event\ndiscard_event_on_send_error true")
|
358
|
+
time = Time.new('2014-01-01T01:23:45+00:00').to_i
|
359
|
+
d.emit(sample_record, time)
|
360
|
+
d.run
|
361
|
+
|
362
|
+
logs = d.instance.log.logs
|
363
|
+
|
364
|
+
assert_match "MixpanelOutputErrorHandler:", logs[0]
|
365
|
+
assert_match "Class: Mixpanel::ServerError", logs[0]
|
366
|
+
assert_match "Message: Could not write to Mixpanel, server responded with 503 returning: 'Service Unavailable", logs[0]
|
367
|
+
assert_match "Backtrace", logs[0]
|
368
|
+
assert_match "Failed to track event to mixpanel", logs[1]
|
369
|
+
assert_match 'Record: {"properties":{"key1":"value1","key2":"value2","time":' + time.to_s + '},"event":"event1","distinct_id":"123"}', logs[1]
|
370
|
+
end
|
371
|
+
end
|
metadata
ADDED
@@ -0,0 +1,149 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: fluent-plugin-mixpanel-enchanced
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.0.10
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- zhron4x
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2017-02-08 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: fluentd
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: 0.10.55
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: 0.10.55
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: mixpanel-ruby
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: 2.2.0
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: 2.2.0
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: rake
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - ">="
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: webmock
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - ">="
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '0'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - ">="
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '0'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: test-unit
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - "~>"
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: 3.0.2
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - "~>"
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: 3.0.2
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
name: pry
|
85
|
+
requirement: !ruby/object:Gem::Requirement
|
86
|
+
requirements:
|
87
|
+
- - ">="
|
88
|
+
- !ruby/object:Gem::Version
|
89
|
+
version: '0'
|
90
|
+
type: :development
|
91
|
+
prerelease: false
|
92
|
+
version_requirements: !ruby/object:Gem::Requirement
|
93
|
+
requirements:
|
94
|
+
- - ">="
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
version: '0'
|
97
|
+
description: Fluentd plugin to input/output event track data to mixpanel
|
98
|
+
email:
|
99
|
+
- zhr0n4x@gmail.com
|
100
|
+
executables: []
|
101
|
+
extensions: []
|
102
|
+
extra_rdoc_files: []
|
103
|
+
files:
|
104
|
+
- ".gitignore"
|
105
|
+
- ".travis.yml"
|
106
|
+
- Gemfile
|
107
|
+
- LICENSE.txt
|
108
|
+
- README.md
|
109
|
+
- Rakefile
|
110
|
+
- example/http_server.sh
|
111
|
+
- example/in_http_mixpanel.conf
|
112
|
+
- example/index.html
|
113
|
+
- fluent-plugin-mixpanel-enchanced.gemspec
|
114
|
+
- lib/fluent/plugin/in_http_mixpanel.rb
|
115
|
+
- lib/fluent/plugin/mixpanel_ruby_error_handler.rb
|
116
|
+
- lib/fluent/plugin/out_mixpanel.rb
|
117
|
+
- test/helper.rb
|
118
|
+
- test/plugin/test_in_http_mixpanel.rb
|
119
|
+
- test/plugin/test_mixpanel_ruby_error_handler.rb
|
120
|
+
- test/plugin/test_out_mixpanel.rb
|
121
|
+
homepage: https://github.com/zhron4x/fluent-plugin-mixpanel-enchanced
|
122
|
+
licenses:
|
123
|
+
- Apache License, Version 2.0
|
124
|
+
metadata: {}
|
125
|
+
post_install_message:
|
126
|
+
rdoc_options: []
|
127
|
+
require_paths:
|
128
|
+
- lib
|
129
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
130
|
+
requirements:
|
131
|
+
- - ">="
|
132
|
+
- !ruby/object:Gem::Version
|
133
|
+
version: '0'
|
134
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
135
|
+
requirements:
|
136
|
+
- - ">="
|
137
|
+
- !ruby/object:Gem::Version
|
138
|
+
version: '0'
|
139
|
+
requirements: []
|
140
|
+
rubyforge_project:
|
141
|
+
rubygems_version: 2.6.8
|
142
|
+
signing_key:
|
143
|
+
specification_version: 4
|
144
|
+
summary: Fluentd plugin to input/output event track data to mixpanel
|
145
|
+
test_files:
|
146
|
+
- test/helper.rb
|
147
|
+
- test/plugin/test_in_http_mixpanel.rb
|
148
|
+
- test/plugin/test_mixpanel_ruby_error_handler.rb
|
149
|
+
- test/plugin/test_out_mixpanel.rb
|