aliyun_sls_sdk 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.gitignore +17 -0
- data/Gemfile +4 -0
- data/LICENSE.txt +22 -0
- data/README.md +118 -0
- data/aliyun_sls_sdk.gemspec +28 -0
- data/bin/sls +85 -0
- data/lib/aliyun_sls.rb +13 -0
- data/lib/aliyun_sls_sdk/connection.rb +227 -0
- data/lib/aliyun_sls_sdk/protobuf.rb +50 -0
- data/lib/aliyun_sls_sdk/version.rb +3 -0
- metadata +153 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 38a1c048c663e2c0065649236f08b4fa19366031
|
4
|
+
data.tar.gz: aa466d6252684f31d1b1d7e6e52f24954bcdc0b4
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 0ea508b203c05795d139cdb16f1684f51cfd8d3c37c3369e03c064d4e73beed474504c903cf15c07f25c098cfa57f0d9db80d5dd1113e53ec1ccd1cd8c68526f
|
7
|
+
data.tar.gz: 92cbdfc5e14e81c0422ce9c5be0e1ff197a381624a2bfa9dc4546bf14b67853dc79cd693116fba58f2e446a7db7682b2e686cdc9a0242a772b90ae84ef7ededf
|
data/.gitignore
ADDED
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
Copyright (c) 2015 cuizheng
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
6
|
+
a copy of this software and associated documentation files (the
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
11
|
+
the following conditions:
|
12
|
+
|
13
|
+
The above copyright notice and this permission notice shall be
|
14
|
+
included in all copies or substantial portions of the Software.
|
15
|
+
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,118 @@
|
|
1
|
+
# AliyunSls
|
2
|
+
|
3
|
+
## 阿里云SLS服务Ruby SDK
|
4
|
+
|
5
|
+
|
6
|
+
#### 简单日志服务(`Simple Log Service,简称SLS`)
|
7
|
+
是针对日志收集、存储和查询的平台化服务。服务提供各种类型日志的实时收集,平台化存储及实时查询海量的日志。并可以将日志归档至ODPS,以利用ODPS做大数据分析。除了通过管理控制台操作,SLS还提供了API(Application Programming Interface)方式写入、查询日志数据,管理自己的项目及日志库等。
|
8
|
+
|
9
|
+
#### SLS(简单日志服务)[介绍](http://docs.aliyun.com/?spm=5176.730001.3.10.5GpxDL#/sls)
|
10
|
+
|
11
|
+
------------
|
12
|
+
## 库用法
|
13
|
+
|
14
|
+
### [查询Store清单](http://docs.aliyun.com/#/pub/sls/api/apilist&ListLogstores)
|
15
|
+
|
16
|
+
con = AliyunSls::Connection.new("project", "region", "access_key_secret", "aliyun_access_key")
|
17
|
+
con.list_logstores
|
18
|
+
|
19
|
+
### [上传日志](http://docs.aliyun.com/#/pub/sls/api/apilist&PutLogs)
|
20
|
+
|
21
|
+
log = AliyunSls::Protobuf::Log.new(:time => Time.now.to_i, :contents => [])
|
22
|
+
|
23
|
+
[
|
24
|
+
['value1', '12'],
|
25
|
+
['value2', '24'],
|
26
|
+
['value3', '36'],
|
27
|
+
['value4', '48']
|
28
|
+
].each { |e|
|
29
|
+
k = e[0]
|
30
|
+
v = e[1]
|
31
|
+
log_item = AliyunSls::Protobuf::Log::Content.new(:key => k, :value => v)
|
32
|
+
log.contents << log_item
|
33
|
+
}
|
34
|
+
log_list = AliyunSls::Protobuf::LogGroup.new(:logs => [])
|
35
|
+
log_list.logs << log
|
36
|
+
|
37
|
+
con = AliyunSls::Connection.new("project", "region", "access_key_secret", "aliyun_access_key")
|
38
|
+
con.puts_logs("store", log_list)
|
39
|
+
|
40
|
+
### [列出日志主题](http://docs.aliyun.com/#/pub/sls/api/apilist&ListTopics)
|
41
|
+
|
42
|
+
con.list_topics("store")
|
43
|
+
|
44
|
+
|名称| 类型| 必选| 描述|
|
45
|
+
|-----|-----|----|-----|
|
46
|
+
|logstorename| 字符串| 是| 需要查询的Logstore名称。|
|
47
|
+
|type| 字符串| 是| 查询Logstore数据的类型,在ListTopics接口中该参数必须为"topic"。|
|
48
|
+
|line| 整型| 否| 请求一次返回的Topic最大数目。取值范围为0~100,默认值为100。|
|
49
|
+
|token| 字符串| 否| 请求返回Topic的起始点(按字典序)。默认值为空字符串,表示从头开始查询 |Logstore中的日志Topic。
|
50
|
+
|
51
|
+
### [查询Logstore中的日志在时间轴上的分布](http://docs.aliyun.com/#/pub/sls/api/apilist&GetHistograms)
|
52
|
+
|
53
|
+
con.get_histograms("store")
|
54
|
+
|
55
|
+
|名称| 类型| 必选| 描述|
|
56
|
+
|----|----|------|------|
|
57
|
+
|logstorename| 字符串| 是| 需要查询日志的Logstore名称。|
|
58
|
+
|type| 字符串| 是| 查询Logstore数据的类型,在GetHistograms接口中该参数必须为"histogram"。|
|
59
|
+
|from| 整型| 是| 查询开始时间点(精度为秒,从1970-1-1 00:00:00 UTC计算起的秒数)。|
|
60
|
+
|to| 整型| 是| 查询结束时间点(精度为秒,从1970-1-1 00:00:00 UTC计算起的秒数)。|
|
61
|
+
|topic| 字符串| 否| 查询日志主题。|
|
62
|
+
|query| 字符串| 否| 查询表达式。关于查询表达式的详细语法请参考查询语法。|
|
63
|
+
|
64
|
+
### [查询Logstore中的日志数据](http://docs.aliyun.com/#/pub/sls/api/apilist&GetLogs)
|
65
|
+
|
66
|
+
con.get_logs("store", :topic => "xxx")
|
67
|
+
|
68
|
+
|名称 | 类型| 必选| 描述|
|
69
|
+
|---|---|---|------------|
|
70
|
+
|logstorename | 字符串| 是| 需要查询日志的Logstore名称。|
|
71
|
+
|type | 字符串| 是| 查询Logstore数据的类型,在GetLogs接口中该参数必须为"log"。|
|
72
|
+
|from | 整型| 是| 查询开始时间点(精度为秒,从1970-1-1 00:00:00 UTC计算起的秒数)。|
|
73
|
+
|to | 整型| 是| 查询结束时间点(精度为秒,从1970-1-1 00:00:00 UTC计算起的秒数)。|
|
74
|
+
|topic | 字符串| 否| 查询日志主题。|
|
75
|
+
|query | 字符串| 否| 查询表达式。关于查询表达式的详细语法请参考查询语法。|
|
76
|
+
|line | 整型| 否| 请求返回的最大日志条数。取值范围0~100,默认值为100。|
|
77
|
+
|offset | 整型| 否| 请求返回日志的起始点。取值范围0或正整数,默认值0。|
|
78
|
+
|reverse | 布尔型| 否| 是否按日志时间戳逆序返回日志。true表示逆序,false表示顺序,默认值为false。|
|
79
|
+
|
80
|
+
## 命令行用法
|
81
|
+
|
82
|
+
1. 生成配置文件
|
83
|
+
2. `PROJECT=[PROJECT] REGION=[REGION] SECRET=[SECRET] KEY=[KEY] sls setup`
|
84
|
+
1. 执行上传日志操作
|
85
|
+
2. `[PROJECT=[PROJECT]] sls put log_path store [topic]`
|
86
|
+
2. 上传日志操作是使用Tail方式执行,有增量日志产生的时候,会自动收集增量部分上传到阿里云
|
87
|
+
|
88
|
+
## fluentd-plugin-sls
|
89
|
+
|
90
|
+
可以结合fluentd,将日志解析好之后上传到阿里云上,实现日志的统一存储。
|
91
|
+
|
92
|
+
`gist`地址:`https://gist.github.com/charlescui/d2a231dbc85b11586fa0`
|
93
|
+
|
94
|
+
## Installation
|
95
|
+
|
96
|
+
Add this line to your application's Gemfile:
|
97
|
+
|
98
|
+
gem 'aliyun_sls'
|
99
|
+
|
100
|
+
And then execute:
|
101
|
+
|
102
|
+
$ bundle
|
103
|
+
|
104
|
+
Or install it yourself as:
|
105
|
+
|
106
|
+
$ gem install aliyun_sls
|
107
|
+
|
108
|
+
## Usage
|
109
|
+
|
110
|
+
TODO: Write usage instructions here
|
111
|
+
|
112
|
+
## Contributing
|
113
|
+
|
114
|
+
1. Fork it
|
115
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
116
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
117
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
118
|
+
5. Create new Pull Request
|
@@ -0,0 +1,28 @@
|
|
1
|
+
# coding: utf-8
|
2
|
+
lib = File.expand_path('../lib', __FILE__)
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
+
require 'aliyun_sls_sdk/version'
|
5
|
+
|
6
|
+
Gem::Specification.new do |spec|
|
7
|
+
spec.name = "aliyun_sls_sdk"
|
8
|
+
spec.version = AliyunSlsSdk::VERSION
|
9
|
+
spec.authors = ["linhua.tlh"]
|
10
|
+
spec.email = ["tlh1987@gmail.com"]
|
11
|
+
spec.description = %q{Gem for SDK of SLS of Aliyun}
|
12
|
+
spec.summary = %q{Gem for SDK of SLS of Aliyun, use it at your own risk}
|
13
|
+
spec.homepage = "https://help.aliyun.com/product/28958.html"
|
14
|
+
spec.license = "MIT"
|
15
|
+
|
16
|
+
spec.files = `git ls-files`.split($/)
|
17
|
+
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
18
|
+
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
|
19
|
+
spec.require_paths = ["lib"]
|
20
|
+
|
21
|
+
spec.add_development_dependency "bundler", "~> 1.3"
|
22
|
+
spec.add_development_dependency "rake", "~> 0"
|
23
|
+
spec.add_dependency "beefcake", "~> 0"
|
24
|
+
spec.add_dependency "file-tail", "~> 0"
|
25
|
+
spec.add_dependency "ruby-hmac", "~> 0"
|
26
|
+
spec.add_dependency "addressable", "~> 0"
|
27
|
+
spec.add_dependency "net-http-persistent", "~> 3.0"
|
28
|
+
end
|
data/bin/sls
ADDED
@@ -0,0 +1,85 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
require File.join(File.dirname(__FILE__), '..', 'lib', 'aliyun_sls.rb')
|
3
|
+
require "file-tail"
|
4
|
+
require "byebug"
|
5
|
+
puts "sls [opt] ..." if (ARGV.size < 1)
|
6
|
+
|
7
|
+
def setup
|
8
|
+
h = {}
|
9
|
+
h['project'] = ENV["PROJECT"]
|
10
|
+
h['region'] = ENV["REGION"] || (puts "PROJECT=[PROJECT] REGION=[REGION] SECRET=[SECRET] KEY=[KEY] sls setup";exit)
|
11
|
+
h['access_key_secret'] = ENV["SECRET"] || (puts "PROJECT=[PROJECT] REGION=[REGION] SECRET=[SECRET] KEY=[KEY] sls setup";exit)
|
12
|
+
h['aliyun_access_key'] = ENV["KEY"] || (puts "PROJECT=[PROJECT] REGION=[REGION] SECRET=[SECRET] KEY=[KEY] sls setup";exit)
|
13
|
+
File.open(File.join(Dir.home, '.slsrc'), 'w+'){|f|
|
14
|
+
f << YAML.dump(h)
|
15
|
+
}
|
16
|
+
puts "setup config at ~/.slsrc"
|
17
|
+
end
|
18
|
+
|
19
|
+
def load
|
20
|
+
File.open(File.join(Dir.home, '.slsrc')){|f|
|
21
|
+
$config = YAML.load(f)
|
22
|
+
}
|
23
|
+
end
|
24
|
+
|
25
|
+
def regist_exit
|
26
|
+
puts "Press CTRL-C to quit"
|
27
|
+
Signal.trap(:INT){
|
28
|
+
exit(-1)
|
29
|
+
}
|
30
|
+
end
|
31
|
+
|
32
|
+
def put_logs(path, store, topic)
|
33
|
+
regist_exit
|
34
|
+
(puts "[PROJECT=[PROJECT]] sls put log_path store [topic]";exit(-1)) if !path || !store
|
35
|
+
load
|
36
|
+
(puts "load config failed! run: sls setup first.";exit(-1)) if !$config
|
37
|
+
|
38
|
+
# 如果传递了项目名称
|
39
|
+
# 则使用本次调用传递过来的值
|
40
|
+
# 否则使用配置文件中的值
|
41
|
+
con = AliyunSls::Connection.new(ENV['PROJECT'] || $config['project'], $config['region'], $config['access_key_secret'], $config['aliyun_access_key'])
|
42
|
+
|
43
|
+
File.open(path) do |log|
|
44
|
+
log.extend(File::Tail)
|
45
|
+
log.interval = 3
|
46
|
+
log.backward(0)
|
47
|
+
log.tail { |line|
|
48
|
+
puts line
|
49
|
+
put_line(line, con, store, topic)
|
50
|
+
}
|
51
|
+
end
|
52
|
+
end
|
53
|
+
|
54
|
+
def put_line(line, con, store, topic)
|
55
|
+
log = AliyunSls::Protobuf::Log.new(:time => Time.now.to_i, :contents => [])
|
56
|
+
begin
|
57
|
+
uri = URI.parse(line)
|
58
|
+
uri.query and uri.query.split('&').each{|paire|
|
59
|
+
k, v = paire.split('=')
|
60
|
+
log_item = AliyunSls::Protobuf::Log::Content.new(:key => k, :value => v)
|
61
|
+
log.contents << log_item
|
62
|
+
}
|
63
|
+
log_list = AliyunSls::Protobuf::LogGroup.new(:logs => [])
|
64
|
+
log_list.logs << log
|
65
|
+
log_list.topic = topic
|
66
|
+
log_list.source = uri.host
|
67
|
+
rescue URI::InvalidURIError => e
|
68
|
+
puts e
|
69
|
+
return
|
70
|
+
end
|
71
|
+
|
72
|
+
begin
|
73
|
+
rsp = con.puts_logs(store, log_list)
|
74
|
+
rescue Exception => e
|
75
|
+
puts "Exception with rsp #{e}"
|
76
|
+
end
|
77
|
+
end
|
78
|
+
|
79
|
+
case opt = ARGV[0]
|
80
|
+
when /setup/i
|
81
|
+
setup
|
82
|
+
exit(0)
|
83
|
+
when /put/i
|
84
|
+
put_logs(ARGV[1], ARGV[2], ARGV[3] || 'aliyun_sls_topic')
|
85
|
+
end
|
data/lib/aliyun_sls.rb
ADDED
@@ -0,0 +1,13 @@
|
|
1
|
+
require File.join(File.dirname(__FILE__), 'aliyun_sls_sdk', 'version.rb')
|
2
|
+
require File.join(File.dirname(__FILE__), 'aliyun_sls_sdk', 'protobuf.rb')
|
3
|
+
require File.join(File.dirname(__FILE__), 'aliyun_sls_sdk', 'connection.rb')
|
4
|
+
|
5
|
+
module AliyunSlsSdk
|
6
|
+
class PostBodyInvalid < RuntimeError; end
|
7
|
+
class SLSInvalidTimestamp < RuntimeError; end
|
8
|
+
class SLSInvalidEncoding < RuntimeError; end
|
9
|
+
class SLSInvalidKey < RuntimeError; end
|
10
|
+
class PostBodyTooLarge < RuntimeError; end
|
11
|
+
class PostBodyUncompressError < RuntimeError; end
|
12
|
+
class SLSLogStoreNotExist < RuntimeError; end
|
13
|
+
end
|
@@ -0,0 +1,227 @@
|
|
1
|
+
require "addressable/uri"
|
2
|
+
require 'rest_client'
|
3
|
+
require 'hmac-sha1'
|
4
|
+
require "base64"
|
5
|
+
require "zlib"
|
6
|
+
require "time"
|
7
|
+
require "net/http/persistent"
|
8
|
+
|
9
|
+
module AliyunSlsSdk
|
10
|
+
class Connection
|
11
|
+
|
12
|
+
def initialize(project, region_endpoint, access_key_secret, aliyun_access_key, opts={})
|
13
|
+
default_headers = {
|
14
|
+
"x-sls-apiversion" => "0.4.0",
|
15
|
+
"x-sls-signaturemethod" => "hmac-sha1"
|
16
|
+
}
|
17
|
+
@headers = default_headers.update opts
|
18
|
+
@aliyun_access_key = aliyun_access_key
|
19
|
+
@access_key_secret = access_key_secret
|
20
|
+
@host = "#{project}.#{region_endpoint}"
|
21
|
+
@http = Net::HTTP::Persistent.new
|
22
|
+
# @http.debug_output = $stdout
|
23
|
+
end
|
24
|
+
|
25
|
+
# http://docs.aliyun.com/#/pub/sls/api/apilist&PutLogs
|
26
|
+
def puts_logs(logstorename, content)
|
27
|
+
# 压缩content数据
|
28
|
+
compressed = Zlib::Deflate.deflate(content.encode.to_s)
|
29
|
+
headers = compact_headers(content, compressed)
|
30
|
+
headers["Authorization"] = signature("POST", logstorename, headers, content, {})
|
31
|
+
|
32
|
+
post_uri = URI "http://#{@host}/logstores/#{logstorename}"
|
33
|
+
post = Net::HTTP::Post.new(post_uri.request_uri, headers)
|
34
|
+
post.body = compressed
|
35
|
+
response = @http.request post_uri, post
|
36
|
+
parse_response(response)
|
37
|
+
end
|
38
|
+
|
39
|
+
# http://docs.aliyun.com/#/pub/sls/api/apilist&ListLogstores
|
40
|
+
def list_logstores
|
41
|
+
headers = compact_headers(nil, nil)
|
42
|
+
headers["Authorization"] = signature("GET", nil, headers, nil, {})
|
43
|
+
|
44
|
+
get_uri = URI "http://#{@host}/logstores"
|
45
|
+
headers["Referer"] = get_uri.to_s
|
46
|
+
get = Net::HTTP::Get.new(get_uri.request_uri, headers)
|
47
|
+
response = @http.request get_uri, get
|
48
|
+
parse_response(response)
|
49
|
+
end
|
50
|
+
|
51
|
+
# http://docs.aliyun.com/#/pub/sls/api/apilist&GetLogs
|
52
|
+
def get_logs(logstorename, opts={})
|
53
|
+
default_opts = {
|
54
|
+
:type => "log",
|
55
|
+
:from => Time.now.to_i - 60*5,#默认是五分钟前
|
56
|
+
:to => Time.now.to_i,
|
57
|
+
# :line => 100,
|
58
|
+
# :offset => 0,
|
59
|
+
:reverse => false
|
60
|
+
}
|
61
|
+
opts = default_opts.update opts
|
62
|
+
headers = compact_headers(nil, nil)
|
63
|
+
headers["Authorization"] = signature("GET", logstorename, headers, nil, opts)
|
64
|
+
get_uri = Addressable::URI.parse("http://#{@host}/logstores/#{logstorename}")
|
65
|
+
headers["Referer"] = get_uri.to_s
|
66
|
+
get_uri.query_values = opts
|
67
|
+
puts get_uri.to_s
|
68
|
+
get = Net::HTTP::Get.new(get_uri.request_uri, headers)
|
69
|
+
response = @http.request get_uri, get
|
70
|
+
parse_response(response)
|
71
|
+
end
|
72
|
+
|
73
|
+
# http://docs.aliyun.com/#/pub/sls/api/apilist&ListTopics
|
74
|
+
def list_topics(logstorename, opts={})
|
75
|
+
default_opts = {
|
76
|
+
:type => "topic",
|
77
|
+
:line => 100,
|
78
|
+
:toke => ""
|
79
|
+
}
|
80
|
+
opts = default_opts.update opts
|
81
|
+
headers = compact_headers(nil, nil)
|
82
|
+
headers["Authorization"] = signature("GET", logstorename, headers, nil, opts)
|
83
|
+
|
84
|
+
get_uri = Addressable::URI.parse("http://#{@host}/logstores/#{logstorename}")
|
85
|
+
headers["Referer"] = get_uri.to_s
|
86
|
+
get_uri.query_values = opts
|
87
|
+
puts headers
|
88
|
+
puts get_uri.to_s
|
89
|
+
get = Net::HTTP::Get.new(get_uri.request_uri, headers)
|
90
|
+
response = @http.request get_uri, get
|
91
|
+
parse_response(response)
|
92
|
+
end
|
93
|
+
|
94
|
+
# http://docs.aliyun.com/#/pub/sls/api/apilist&GetHistograms
|
95
|
+
def get_histograms(logstorename, opts={})
|
96
|
+
default_opts = {
|
97
|
+
:type => "histogram",
|
98
|
+
:from => Time.now.to_i - 60*5,#默认是五分钟前
|
99
|
+
:to => Time.now.to_i,
|
100
|
+
:topic => "",
|
101
|
+
:query => "",
|
102
|
+
}
|
103
|
+
opts = default_opts.update opts
|
104
|
+
headers = compact_headers(nil, nil)
|
105
|
+
headers["Authorization"] = signature("GET", logstorename, headers, nil, opts)
|
106
|
+
|
107
|
+
get_uri = Addressable::URI.parse("http://#{@host}/logstores/#{logstorename}")
|
108
|
+
headers["Referer"] = get_uri.to_s
|
109
|
+
get_uri.query_values = opts
|
110
|
+
get = Net::HTTP::Get.new(get_uri.request_uri, headers)
|
111
|
+
response = @http.request get_uri, get
|
112
|
+
parse_response(response)
|
113
|
+
end
|
114
|
+
|
115
|
+
private
|
116
|
+
|
117
|
+
def string_to_sign(verb, logstorename, headers, content, query={})
|
118
|
+
if content
|
119
|
+
string_to_sign_with_content(verb, logstorename, headers, query)
|
120
|
+
else
|
121
|
+
string_to_sign_without_content(verb, logstorename, headers, query)
|
122
|
+
end
|
123
|
+
end
|
124
|
+
|
125
|
+
def string_to_sign_with_content(verb, logstorename, headers, query={})
|
126
|
+
<<-DOC
|
127
|
+
#{verb}
|
128
|
+
#{headers['Content-MD5']}
|
129
|
+
#{headers['Content-Type']}
|
130
|
+
#{headers['Date']}
|
131
|
+
#{canonicalized_sls_headers(headers)}
|
132
|
+
#{canonicalized_resource(logstorename, query)}
|
133
|
+
DOC
|
134
|
+
end
|
135
|
+
|
136
|
+
def string_to_sign_without_content(verb, logstorename, headers, query={})
|
137
|
+
<<-DOC
|
138
|
+
#{verb}
|
139
|
+
|
140
|
+
|
141
|
+
#{headers['Date']}
|
142
|
+
#{canonicalized_sls_headers(headers)}
|
143
|
+
#{canonicalized_resource(logstorename, query)}
|
144
|
+
DOC
|
145
|
+
end
|
146
|
+
|
147
|
+
# “CanonicalizedSLSHeaders”的构造方式如下:
|
148
|
+
# 1. 将所有以“x-sls-”为前缀的HTTP请求头的名字转换成小写字母;
|
149
|
+
# 2. 将上一步得到的所有SLS自定义请求头按照字典序进行升序排序;
|
150
|
+
# 3. 删除请求头和内容之间分隔符两端出现的任何空格;
|
151
|
+
# 4. 将所有的头和内容用\n分隔符组合成最后的CanonicalizedSLSHeader;
|
152
|
+
def canonicalized_sls_headers(headers)
|
153
|
+
h = {}
|
154
|
+
headers.each { |k, v|
|
155
|
+
if k =~ /x-sls-.*/
|
156
|
+
h[k.downcase] = v
|
157
|
+
end
|
158
|
+
}
|
159
|
+
h.keys.sort.map { |e|
|
160
|
+
h[e]
|
161
|
+
"#{e}:#{h[e].gsub(/^\s+/,'')}"
|
162
|
+
}.join($/)
|
163
|
+
end
|
164
|
+
|
165
|
+
# “CanonicalizedResource”的构造方式如下:
|
166
|
+
# 1. 将CanonicalizedResource设置为空字符串("");
|
167
|
+
# 2. 放入要访问的SLS资源:"/logstores/logstorename"(无logstorename则不填);
|
168
|
+
# 3. 如请求包含查询字符串(QUERY_STRING),则在CanonicalizedResource字符串尾部添加“?”和查询字符串。
|
169
|
+
def canonicalized_resource(logstorename, query={})
|
170
|
+
u = logstorename ? Addressable::URI.parse("/logstores/#{logstorename}") : Addressable::URI.parse("/logstores")
|
171
|
+
if query.size != 0
|
172
|
+
# 不能对请求的URL参数做URLEncode编码
|
173
|
+
q_str = query.keys.sort.map { |e|
|
174
|
+
"#{e}=#{query[e]}"
|
175
|
+
}.join('&')
|
176
|
+
"#{u}?#{q_str}"
|
177
|
+
else
|
178
|
+
u.to_s
|
179
|
+
end
|
180
|
+
end
|
181
|
+
|
182
|
+
# 目前,SLS API只支持一种数字签名算法,即默认签名算法"hmac-sha1"。其整个签名公式如下:
|
183
|
+
# Signature = base64(hmac-sha1(UTF8-Encoding-Of(SignString),AccessKeySecret))
|
184
|
+
def sign(method, logstorename, headers, content, query)
|
185
|
+
Base64.encode64((HMAC::SHA1.new(@access_key_secret) << string_to_sign(method, logstorename, headers, content, query).chomp).digest).strip
|
186
|
+
end
|
187
|
+
|
188
|
+
def signature(method, logstorename, headers, content, query)
|
189
|
+
"SLS #{@aliyun_access_key}:#{sign(method, logstorename, headers, content, query)}"
|
190
|
+
end
|
191
|
+
|
192
|
+
# content是LogGroup
|
193
|
+
def compact_headers(content, compressed)
|
194
|
+
headers = @headers.dup
|
195
|
+
# headers["x-sls-date"] =
|
196
|
+
headers["Date"] = DateTime.now.httpdate
|
197
|
+
headers["x-sls-bodyrawsize"] = "0"
|
198
|
+
|
199
|
+
if content and compressed
|
200
|
+
body = content.encode.to_s
|
201
|
+
headers["Content-Length"] = compressed.bytesize.to_s
|
202
|
+
# 日志内容包含的日志必须小于3MB和4096条。
|
203
|
+
raise AliyunSls::PostBodyTooLarge, "content length is larger than 3MB" if headers["Content-Length"].to_i > 3*1024**2*8
|
204
|
+
raise AliyunSls::PostBodyTooLarge, "content size is more than 4096" if content.logs.size > 4096
|
205
|
+
# MD5必须为大写字符串
|
206
|
+
headers["Content-MD5"] = Digest::MD5.hexdigest(compressed).upcase
|
207
|
+
headers["Content-Type"] = "application/x-protobuf"
|
208
|
+
headers["x-sls-bodyrawsize"] = body.bytesize.to_s
|
209
|
+
headers["x-sls-compresstype"] = "deflate"
|
210
|
+
end
|
211
|
+
headers
|
212
|
+
end
|
213
|
+
|
214
|
+
def parse_response(response)
|
215
|
+
# 如果返回结果报错,则解析报错内容打印到日志中
|
216
|
+
if response.code.to_s =~ /[4|5]\d\d/
|
217
|
+
msg = "status #{response.code} body #{response.body}"
|
218
|
+
if $logger and $logger.respond_to?(:error)
|
219
|
+
$logger.error msg
|
220
|
+
else
|
221
|
+
puts msg
|
222
|
+
end
|
223
|
+
end
|
224
|
+
response
|
225
|
+
end
|
226
|
+
end
|
227
|
+
end
|
@@ -0,0 +1,50 @@
|
|
1
|
+
require 'beefcake'
|
2
|
+
|
3
|
+
module AliyunSlsSdk
|
4
|
+
module Protobuf
|
5
|
+
|
6
|
+
# message Log
|
7
|
+
# {
|
8
|
+
# required uint32 time = 1; // UNIX Time Format
|
9
|
+
# message Content
|
10
|
+
# {
|
11
|
+
# required string key = 1;
|
12
|
+
# required string value = 2;
|
13
|
+
# }
|
14
|
+
# repeated Content contents= 2;
|
15
|
+
# }
|
16
|
+
|
17
|
+
# message LogGroup
|
18
|
+
# {
|
19
|
+
# repeated Log logs= 1;
|
20
|
+
# optional string reserved =2; // 内部字段,不需要填写
|
21
|
+
# optional string topic = 3;
|
22
|
+
# optional string source = 4;
|
23
|
+
# }
|
24
|
+
|
25
|
+
class Log
|
26
|
+
include Beefcake::Message
|
27
|
+
|
28
|
+
required :time, :uint32, 1
|
29
|
+
|
30
|
+
class Content
|
31
|
+
include Beefcake::Message
|
32
|
+
|
33
|
+
required :key, :string, 1
|
34
|
+
required :value, :string, 2
|
35
|
+
end
|
36
|
+
|
37
|
+
repeated :contents, Content, 2
|
38
|
+
end
|
39
|
+
|
40
|
+
class LogGroup
|
41
|
+
include Beefcake::Message
|
42
|
+
|
43
|
+
repeated :logs, Log, 1
|
44
|
+
optional :reserved, :string, 2
|
45
|
+
optional :topic, :string, 3
|
46
|
+
optional :source, :string, 4
|
47
|
+
end
|
48
|
+
|
49
|
+
end
|
50
|
+
end
|
metadata
ADDED
@@ -0,0 +1,153 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: aliyun_sls_sdk
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.0.1
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- linhua.tlh
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2017-03-27 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: bundler
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '1.3'
|
20
|
+
type: :development
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '1.3'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: rake
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '0'
|
34
|
+
type: :development
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: beefcake
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - "~>"
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :runtime
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - "~>"
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: file-tail
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - "~>"
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '0'
|
62
|
+
type: :runtime
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '0'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: ruby-hmac
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - "~>"
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '0'
|
76
|
+
type: :runtime
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - "~>"
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '0'
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
name: addressable
|
85
|
+
requirement: !ruby/object:Gem::Requirement
|
86
|
+
requirements:
|
87
|
+
- - "~>"
|
88
|
+
- !ruby/object:Gem::Version
|
89
|
+
version: '0'
|
90
|
+
type: :runtime
|
91
|
+
prerelease: false
|
92
|
+
version_requirements: !ruby/object:Gem::Requirement
|
93
|
+
requirements:
|
94
|
+
- - "~>"
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
version: '0'
|
97
|
+
- !ruby/object:Gem::Dependency
|
98
|
+
name: net-http-persistent
|
99
|
+
requirement: !ruby/object:Gem::Requirement
|
100
|
+
requirements:
|
101
|
+
- - "~>"
|
102
|
+
- !ruby/object:Gem::Version
|
103
|
+
version: '3.0'
|
104
|
+
type: :runtime
|
105
|
+
prerelease: false
|
106
|
+
version_requirements: !ruby/object:Gem::Requirement
|
107
|
+
requirements:
|
108
|
+
- - "~>"
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: '3.0'
|
111
|
+
description: Gem for SDK of SLS of Aliyun
|
112
|
+
email:
|
113
|
+
- tlh1987@gmail.com
|
114
|
+
executables:
|
115
|
+
- sls
|
116
|
+
extensions: []
|
117
|
+
extra_rdoc_files: []
|
118
|
+
files:
|
119
|
+
- ".gitignore"
|
120
|
+
- Gemfile
|
121
|
+
- LICENSE.txt
|
122
|
+
- README.md
|
123
|
+
- aliyun_sls_sdk.gemspec
|
124
|
+
- bin/sls
|
125
|
+
- lib/aliyun_sls.rb
|
126
|
+
- lib/aliyun_sls_sdk/connection.rb
|
127
|
+
- lib/aliyun_sls_sdk/protobuf.rb
|
128
|
+
- lib/aliyun_sls_sdk/version.rb
|
129
|
+
homepage: https://help.aliyun.com/product/28958.html
|
130
|
+
licenses:
|
131
|
+
- MIT
|
132
|
+
metadata: {}
|
133
|
+
post_install_message:
|
134
|
+
rdoc_options: []
|
135
|
+
require_paths:
|
136
|
+
- lib
|
137
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
138
|
+
requirements:
|
139
|
+
- - ">="
|
140
|
+
- !ruby/object:Gem::Version
|
141
|
+
version: '0'
|
142
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
143
|
+
requirements:
|
144
|
+
- - ">="
|
145
|
+
- !ruby/object:Gem::Version
|
146
|
+
version: '0'
|
147
|
+
requirements: []
|
148
|
+
rubyforge_project:
|
149
|
+
rubygems_version: 2.6.11
|
150
|
+
signing_key:
|
151
|
+
specification_version: 4
|
152
|
+
summary: Gem for SDK of SLS of Aliyun, use it at your own risk
|
153
|
+
test_files: []
|