instrumental_tools 1.0.0.rc2 → 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (44) hide show
  1. checksums.yaml +4 -4
  2. data/BUILD.md +50 -0
  3. data/CHANGELOG.md +5 -1
  4. data/CUSTOM_METRICS.md +9 -1
  5. data/INSTALL.md +89 -0
  6. data/LICENSE +1 -1
  7. data/README.md +23 -3
  8. data/Rakefile +257 -0
  9. data/TEST.md +9 -0
  10. data/bin/instrument_server +48 -7
  11. data/chef/.kitchen.yml +16 -0
  12. data/chef/Berksfile +4 -0
  13. data/chef/Berksfile.lock +9 -0
  14. data/chef/instrumental_tools/attributes/default.rb +2 -0
  15. data/chef/instrumental_tools/metadata.rb +3 -0
  16. data/chef/instrumental_tools/recipes/default.rb +21 -0
  17. data/chef/instrumental_tools/templates/instrumental.yml.erb +6 -0
  18. data/conf/instrumental.yml +6 -0
  19. data/debian/after-install.sh +6 -0
  20. data/debian/after-remove.sh +4 -0
  21. data/debian/before-remove.sh +4 -0
  22. data/debian/instrument_server +46 -0
  23. data/examples/mysql/mysql_status.rb +3 -3
  24. data/instrumental_tools.gemspec +32 -4
  25. data/lib/instrumental_tools/metric_script_executor.rb +38 -34
  26. data/lib/instrumental_tools/server_controller.rb +122 -35
  27. data/lib/instrumental_tools/system_inspector/linux.rb +80 -42
  28. data/lib/instrumental_tools/version.rb +1 -1
  29. data/puppet/.kitchen.yml +26 -0
  30. data/puppet/.librarian/puppet/config +2 -0
  31. data/puppet/Puppetfile +4 -0
  32. data/puppet/Puppetfile.lock +17 -0
  33. data/puppet/instrumental_tools/manifests/init.pp +29 -0
  34. data/puppet/instrumental_tools/metadata.json +11 -0
  35. data/puppet/instrumental_tools/templates/instrumental.yml.erb +6 -0
  36. data/puppet/manifests/site.pp +3 -0
  37. data/rpm/after-install.sh +7 -0
  38. data/rpm/after-remove.sh +4 -0
  39. data/rpm/before-remove.sh +5 -0
  40. data/rpm/instrument_server +46 -0
  41. data/systemd/instrument_server.service +13 -0
  42. data/test/integration/default/serverspec/instrumental_tools_spec.rb +29 -0
  43. metadata +164 -11
  44. data/.gitignore +0 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 6737135b417f062d69eaa14a98b0b1746564a7cd
4
- data.tar.gz: 7055fa1984c0b741d95e741f19cd4c602272bdb4
3
+ metadata.gz: 8871b4d7d488bafeb4a58f5d82caad26ba139c00
4
+ data.tar.gz: 5901bebe60f6303dbb00cacba6f376ffa1b61c88
5
5
  SHA512:
6
- metadata.gz: 837a097722be8ec8716c065f0846285346c0bf5e4d3c5d0ca8980ec58b822a2d00a013fe0c96e5ae9078dd390feb7f13452a9abd14938a1cbbee3933784f8511
7
- data.tar.gz: b756454e6b3d7305e354121646a0202045c4f7223155fe1db475e640e401fb0b74885778a39ce82e1601915057c18e849fac3d10ae9f5b67b66fba1800e3ff1c
6
+ metadata.gz: 71a0be3021518f008517cfbf8d0802ca80de6c3362df63f9d70aabaf0a86eac6341fec2018f7c4de1047b0e4e30639f155085182d35c3e59b7b0f23f8a55081c
7
+ data.tar.gz: 5c55425bdb4b2cd64adee1c7aa363d918bbb18175a106796cd29145dc3b17d69be1adba0e5f94af61ee864d98af8665a3b73143420169b2b804e1960207f82b8
data/BUILD.md ADDED
@@ -0,0 +1,50 @@
1
+ # Building instrumental-tools
2
+
3
+ ## The Gem
4
+
5
+ Building the `instrumental_tools` gem can be done via:
6
+
7
+ ```
8
+ rake gem
9
+ ```
10
+
11
+ This will produce a .gem file suitable for release. As a shortcut for the RubyGems release process, you can issue the following command:
12
+
13
+ ```
14
+ rake release
15
+ ```
16
+
17
+ to push a new copy of the gem directly to RubyGems. This presumes you have the correct `rubygems_api_key` available in your system Gem config.
18
+
19
+ ## `deb`, `rpm` and `tgz` packages
20
+
21
+ Building new `deb`, `rpm` and `tgz` packages can be done via the following rake commands:
22
+
23
+ For 32 bit Linux:
24
+
25
+ ```
26
+ rake package:linux-x86:package # builds `rpm` and `deb`
27
+ rake package:linux-x86:tarball # buidls tgz
28
+ ```
29
+
30
+ For 64 bit Linux:
31
+
32
+ ```
33
+ rake package:linux-x86_64:package
34
+ rake package:linux-x86_64:tarball
35
+ ```
36
+
37
+
38
+ For Mac OS X:
39
+
40
+ ```
41
+ rake package:osx:tarball
42
+ ```
43
+
44
+ `deb` and `rpm` packages should be pushed to PackageCloud. You will need to ensure you have the `package_cloud` gem installed (`bundle install` should install it for you - see the [PackageCloud instructions](https://packagecloud.io/docs#cli_install) otherwise). You will also need write credentials to PackageCloud available in `~/.packagecloud`; they will follow the format:
45
+
46
+ ```
47
+ {"url":"https://packagecloud.io","token":"YOUR PACKAGECLOUD API TOKEN"}
48
+ ```
49
+
50
+ On release, the tarball should be uploaded to the Github releases page and linked to from the main README.md.
data/CHANGELOG.md CHANGED
@@ -1,10 +1,14 @@
1
- ### Unreleased
1
+ ### 1.0.0 [?]
2
2
  * Configurable pid and log file locations
3
3
  * Pid and log file default to $HOME
4
4
  * Process control commands do not require API key
5
5
  * Omit "-d" in favor of "start" and "stop", "foreground" runs process in foreground
6
6
  * Configurable reporting interval
7
7
  * Custom scripts may be executed and have their output sent to Instrumental (See [CUSTOM_METRICS.md](CUSTOM_METRICS.md))
8
+ * Support dropping privileges at startup
9
+ * Example scripts for monitoring Docker and MySQL
10
+ * Example startup scripts for starting process at boot time
11
+ * Example Chef and Puppet recipes
8
12
 
9
13
  ### 0.6.0 [August 11th, 2014]
10
14
  * Don't report swap usage if it's zero (Patrick Wyatt)
data/CUSTOM_METRICS.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # Custom Metric Scripts
2
2
 
3
- You may have specific pieces of architecture that you would like `instrument_server` to monitor. As of version 0.7.0 of the `instrument_server` gem, you may pass the `-e` flag to `instrument_server` on startup to enable this functionality. There are several [examples](examples/) of scripts that you may use for your infrastructure, or you can [write your own](#writing_custom_scripts).
3
+ You may have specific pieces of architecture that you would like `instrument_server` to monitor. As of version 1.0.0 of the `instrument_server` gem, you may pass the `-e` flag to `instrument_server` on startup to enable this functionality. There are several [examples](examples/) of scripts that you may use for your infrastructure, or you can [write your own](#writing_custom_scripts).
4
4
 
5
5
  ## Installing Custom Scripts
6
6
 
@@ -28,6 +28,12 @@ Your script is expected to output data in the following format on `STDOUT` in or
28
28
  METRIC_NAME METRIC_VALUE
29
29
  ```
30
30
 
31
+ or
32
+
33
+ ```
34
+ METRIC_NAME METRIC_VALUE UNIX_TIME_IN_SECONDS
35
+ ```
36
+
31
37
  For example, if a script named `application_load` were to report two metrics, `memory` and `load`, to the `instrument_server` process, its output should be:
32
38
 
33
39
  ```
@@ -43,6 +49,8 @@ HOST_NAME.SCRIPT_NAME.METRIC_NAME
43
49
 
44
50
  Using the previous example, if the `application_load` script ran on a host named `app-0001`, its `memory` and `load` metrics would be submitted to Instrumental as `app-0001.application_load.memory` and `app-0001.application_load.load`.
45
51
 
52
+ The optional third parameter of the above formats, `UNIX_TIME_IN_SECONDS`, represents the time under which the submitted metric should be measured. Generally you do not need to provide this value, as `instrument_server` will default to recording the time when it receives the metric from your script as the time under which the measurement should be recorded.
53
+
46
54
  ### Exit Codes
47
55
 
48
56
  If you do not want the output of your script submitted to Instrumental, your process should exit with a non-zero exit code. Its `STDOUT` output will still be provided to your script on the next iteration.
data/INSTALL.md ADDED
@@ -0,0 +1,89 @@
1
+ # Package Locations
2
+
3
+ ## PackageCloud
4
+
5
+ Prebuilt `deb` and `rpm` packages are available via the [packagecloud.io](https://packagecloud.io/) service. These files are also available to download directly:
6
+
7
+ * 64-bit Debian package (Ubuntu, Debian) [https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_amd64.deb](https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_amd64.deb)
8
+ * 32-bit Debian package (Ubuntu, Debian) [https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_i386.deb](https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_i386.deb)
9
+ * 64-bit RPM package (RHEL, Amazon AMI) [https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_amd64.rpm](https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_amd64.rpm)
10
+ * 32-bit RPM package (RHEL, Amazon AMI) [https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_i386.rpm](https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_i386.rpm)
11
+ * 64-bit Linux tarball (CoreOS, etc.) [https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_linux-x86_64.tar.gz](https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_linux-x86_64.tar.gz)
12
+ * 32-bit Linux tarball (CoreOS, etc.) [https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_linux-x86.tar.gz](https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_linux-x86.tar.gz)
13
+ * 64-bit Mac OS X tarball [https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_osx.tar.gz](https://s3.amazonaws.com/instrumental-tools/1.0.0/instrumental-tools_1.0.0_osx.tar.gz)
14
+
15
+ # Ubuntu
16
+
17
+ ```sh
18
+ sudo apt-get install curl
19
+ curl https://packagecloud.io/install/repositories/expectedbehavior/instrumental/script.deb | sudo bash
20
+ sudo apt-get install instrumental-tools
21
+ ```
22
+
23
+ # Debian
24
+
25
+ ```sh
26
+ su -c "apt-get install curl"
27
+ su -c "curl https://packagecloud.io/install/repositories/expectedbehavior/instrumental/script.deb | bash"
28
+ su -c "apt-get install instrumental-tools"
29
+ ```
30
+
31
+ # Enterprise Linux (CentOS, AWS Linux, RedHat)
32
+
33
+ ```sh
34
+ curl https://packagecloud.io/install/repositories/expectedbehavior/instrumental/script.rpm | sudo bash
35
+ sudo yum install instrumental-tools
36
+ ```
37
+
38
+ # Other ( CoreOS, et al )
39
+
40
+ ## Installing the software
41
+
42
+ ```sh
43
+ sudo mkdir -p /opt/instrumental-tools/
44
+ sudo tar -zxvf ./instrumental-tools_1.0.0_linux-x86_64.tar.gz -C /opt/instrumental-tools/ --strip 1
45
+ sudo cp /opt/instrumental-tools/etc/instrumental.yml /etc/
46
+ ```
47
+
48
+ # RubyGems
49
+
50
+ ```sh
51
+ sudo gem install instrumental_tools
52
+ ```
53
+
54
+ Installing `instrumental_tools` via Rubygems will not create the `/opt/instrumental-tools/` directory on your server or setup the process to run on startup. It is advisable that you install the software via the `deb` or `rpm` packages if your system supports its.
55
+
56
+ ## Adding to system startup
57
+
58
+ ### systemd
59
+
60
+ ```sh
61
+ sudo cp /opt/instrumental-tools/lib/app/systemd/instrument_server.service /etc/systemd/system/
62
+ sudo systemctl enable instrument_server.service
63
+ sudo systemctl start instrument_server
64
+ ```
65
+
66
+ ### sysvinit (update-rc.d)
67
+
68
+ ```sh
69
+ sudo cp /opt/instrumental-tools/lib/app/debian/instrument_server /etc/init.d/
70
+ sudo update-rc.d instrument_server defaults
71
+ sudo /etc/init.d/instrument_server start
72
+ ```
73
+
74
+ ### sysvinit (chkconfig)
75
+
76
+ ```sh
77
+ sudo cp /opt/instrumental-tools/lib/app/rpm/instrument_server /etc/init.d/
78
+ sudo chkconfig --add instrument_server
79
+ sudo chkconfig instrument_server on
80
+ sudo service instrument_server start
81
+ ```
82
+
83
+ # Chef
84
+
85
+ An example Chef cookbook for installing `instrumental-tools` is available in [`chef/instrumental_tools`](chef/instrumental_tools).
86
+
87
+ # Puppet
88
+
89
+ An example Puppet module for installing `instrumental-tools` is available in [`puppet/instrumental_tools`](puppet/instrumental_tools).
data/LICENSE CHANGED
@@ -1,4 +1,4 @@
1
- Copyright (c) 2011 Fastest Forward
1
+ Copyright (c) 2015 Expected Behavior
2
2
 
3
3
  Permission is hereby granted, free of charge, to any person obtaining
4
4
  a copy of this software and associated documentation files (the
data/README.md CHANGED
@@ -1,10 +1,30 @@
1
1
  # Instrumental Tools
2
2
 
3
- A collection of scripts useful for monitoring servers and services with Instrumental ([www.instrumentalapp.com](http://www.instrumentalapp.com/)).
3
+ A collection of tools for monitoring servers with Instrumental ([www.instrumentalapp.com](http://www.instrumentalapp.com/)).
4
4
 
5
- ## instrument_server
5
+ ## Operating System Support
6
6
 
7
- Monitor server activity by collecting information on CPU and memory usage, disk IO, filesystem usage, etc.
7
+ `instrumental_tools` is currently officially supported on 32-bit and 64-bit Linux, as well as Mac OS X. There are prebuilt packages available for Debian and RHEL-based systems.
8
+
9
+ ## Installation
10
+
11
+ Installation instructions for supported platforms is available in [INSTALL.md](INSTALL.md). The recommended installation method is to use a prebuilt package, which will automatically install the application as a service in your operating system's startup list.
12
+
13
+ Once you've installed the package, you will want to edit the `/etc/instrumental.yml` file with your Instrumental API key. Example `/etc/instrumental.yml`:
14
+
15
+ ## Sent Metrics
16
+
17
+ The default `instrument_server` behavior will collect metrics on the following data:
18
+
19
+ * CPU (`user`, `nice`, `system`, `idle`, `iowait` and `total in use`)
20
+ * Load (at 1 minute, 5 minute and 15 minute intervals)
21
+ * Memory (`used`, `free`, `buffers`, `cached`, `free_percent` )
22
+ * Swap (`used`, `free`, `free_percent`)
23
+ * Disk Capacity (`total`, `used`, `available`, `available percent` for all mounted disks)
24
+ * Disk Usage (`percent_utilization` for all mounted disks)
25
+ * Filesystem stats (`open_files`, `max_open_files`)
26
+
27
+ ## Command Line Usage
8
28
 
9
29
  Basic usage:
10
30
 
data/Rakefile CHANGED
@@ -1 +1,258 @@
1
+
1
2
  require 'bundler/gem_tasks'
3
+ require 'etc'
4
+ require 'fileutils'
5
+ require 'socket'
6
+ require 'yaml'
7
+
8
+ PACKAGE_CATEGORY = "Utilities"
9
+ PACKAGECLOUD_REPO = "expectedbehavior/instrumental"
10
+ CONFIG_DIR = "conf"
11
+ CONFIG_DEST = "/etc/"
12
+
13
+ GEMSPEC = Bundler::GemHelper.instance.gemspec
14
+ SPEC_PATH = Bundler::GemHelper.instance.spec_path
15
+ PACKAGE_NAME = GEMSPEC.name.gsub("_", "-") # Debian packages cannot include _ in name
16
+ VERSION = GEMSPEC.version
17
+ TRAVELING_RUBY_VERSION = "20150210-2.1.5"
18
+ TRAVELING_RUBY_FILE = "packaging/traveling-ruby-#{TRAVELING_RUBY_VERSION}-%s.tar.gz"
19
+ DEST_DIR = File.join("/opt/", PACKAGE_NAME)
20
+ PACKAGE_OUTPUT_NAME = [PACKAGE_NAME, VERSION].join("_")
21
+ LICENSE = Array(GEMSPEC.licenses).first || "None"
22
+ VENDOR = Array(GEMSPEC.authors).first || Etc.getlogin
23
+ MAINTAINER = Array(GEMSPEC.email).first || [Etc.getlogin, Socket.gethostname].join("@")
24
+ HOMEPAGE = GEMSPEC.homepage || ""
25
+ DESCRIPTION = GEMSPEC.description || ""
26
+ SUPPORTED_DISTROS = {
27
+ 'deb' => ['ubuntu/precise', 'ubuntu/lucid', 'ubuntu/trusty', 'ubuntu/utopic', 'debian/lenny', 'debian/squeeze', 'debian/wheezy'],
28
+ 'rpm' => ['el/5', 'el/6', 'el/7']
29
+ }
30
+ EXTRA_ARGS = {
31
+ 'deb' => '--deb-init debian/instrument_server --after-install debian/after-install.sh --before-remove debian/before-remove.sh --after-remove debian/after-remove.sh --deb-user nobody --deb-group nogroup',
32
+ 'rpm' => '--rpm-init rpm/instrument_server --after-install rpm/after-install.sh --before-remove rpm/before-remove.sh --after-remove rpm/after-remove.sh --rpm-user nobody --rpm-group nobody --rpm-os linux --rpm-attr "-,nobody,nobody:/opt/instrumental-tools/" --directories /opt/instrumental-tools/'
33
+ }
34
+
35
+
36
+ ARCHITECTURES = {
37
+ 'linux-x86' => {
38
+ runtime: TRAVELING_RUBY_FILE % "linux-x86",
39
+ arch: "i386",
40
+ packages: %w{deb rpm},
41
+ platform: "linux",
42
+ packagecloud: true
43
+ },
44
+ 'linux-x86_64' => {
45
+ runtime: TRAVELING_RUBY_FILE % "linux-x86_64",
46
+ arch: "amd64",
47
+ packages: %w{deb rpm},
48
+ platform: "linux",
49
+ packagecloud: true
50
+ },
51
+ 'osx' => {
52
+ runtime: TRAVELING_RUBY_FILE % "osx",
53
+ arch: "x86_64",
54
+ packages: [],
55
+ platform: "darwin",
56
+ packagecloud: false
57
+ }
58
+ }
59
+
60
+
61
+ WRAPPER_SCRIPT = <<-EOSCRIPT
62
+ #!/bin/bash
63
+ set -e
64
+
65
+ # Figure out where this script is located.
66
+ SELFDIR="`dirname \"$0\"`"
67
+ SELFDIR="`cd \"$SELFDIR\" && pwd`"
68
+
69
+ # Tell Bundler where the Gemfile and gems are.
70
+ export BUNDLE_GEMFILE="$SELFDIR/lib/vendor/Gemfile"
71
+ unset BUNDLE_IGNORE_CONFIG
72
+
73
+ # Run the actual app using the bundled Ruby interpreter.
74
+ exec "$SELFDIR/lib/ruby/bin/ruby" -rbundler/setup "$SELFDIR/lib/app/%s" "$@"
75
+ EOSCRIPT
76
+
77
+ BUNDLE_CONFIG = <<-EOBUNDLECONFIG
78
+ BUNDLE_PATH: .
79
+ BUNDLE_WITHOUT: development
80
+ BUNDLE_DISABLE_SHARED_GEMS: '1'
81
+ EOBUNDLECONFIG
82
+
83
+
84
+ desc "Package your app"
85
+ task :package => ARCHITECTURES.map { |name, _| "package:%s" % name }
86
+
87
+ ARCHITECTURES.each do |name, config|
88
+ namespace "package" do
89
+
90
+ has_packaging = Array(config[:packages]).size > 0
91
+
92
+ if has_packaging
93
+ desc "Package your app for %s" % name
94
+ task name => ["%s:package" % name]
95
+ else
96
+ desc "Package your app for %s" % name
97
+ task name => ["%s:tarball" % name]
98
+ end
99
+
100
+ namespace name do
101
+ desc "Create a tarball for %s" % name
102
+ task "tarball" => [:bundle_install, config[:runtime]] do
103
+ create_tarball(create_directory_bundle(name))
104
+ end
105
+
106
+ if has_packaging
107
+ desc "Create packages (%s) for %s" % [config[:packages].join(","), name]
108
+ task "package" => [:bundle_install, config[:runtime]] do
109
+ create_packages(create_tarball(create_directory_bundle(name, DEST_DIR)), config[:platform], config[:arch], config[:packages])
110
+ end
111
+ end
112
+
113
+ if config[:packagecloud]
114
+ namespace "packagecloud" do
115
+ desc "Push packages (%s) to package_cloud" % config[:packages].join(",")
116
+ task "push" do
117
+ packages = create_packages(create_tarball(create_directory_bundle(name, DEST_DIR)), config[:platform], config[:arch], config[:packages])
118
+ by_extension = packages.group_by { |path| File.extname(path)[1..-1] }
119
+ by_extension.each do |extension, files|
120
+ distros = SUPPORTED_DISTROS[extension]
121
+ distros.each do |distro|
122
+ repo = File.join(PACKAGECLOUD_REPO, distro)
123
+ files.each do |file|
124
+ yank_cmd = "package_cloud yank %s %s" % [repo, file]
125
+ puts yank_cmd
126
+ system(yank_cmd)
127
+ sh "package_cloud push %s %s" % [repo, file]
128
+ end
129
+ end
130
+ end
131
+ end
132
+ end
133
+ end
134
+
135
+ end
136
+ end
137
+
138
+
139
+ file config[:runtime] do
140
+ download_runtime(name)
141
+ end
142
+ end
143
+
144
+ namespace "package" do
145
+
146
+ desc "Install gems to local directory"
147
+ task :bundle_install do
148
+ if RUBY_VERSION !~ /^2\.1\./
149
+ abort "You can only 'bundle install' using Ruby 2.1, because that's what Traveling Ruby uses."
150
+ end
151
+
152
+ tmp_package_dir = File.join("packaging", "tmp")
153
+ spec_path = SPEC_PATH
154
+ cache_dir = File.join("packaging", "vendor", "*", "*", "cache", "*")
155
+
156
+ sh "rm -rf %s" % tmp_package_dir
157
+ sh "mkdir -p %s" % tmp_package_dir
158
+ sh "cp %s Gemfile Gemfile.lock %s" % [spec_path, tmp_package_dir]
159
+
160
+ GEMSPEC.require_paths.each do |path|
161
+ sh "ln -sf %s %s" % [File.expand_path(path), tmp_package_dir]
162
+ end
163
+
164
+ Bundler.with_clean_env do
165
+ sh "cd %s && env BUNDLE_IGNORE_CONFIG=1 bundle install --path ../vendor --without development" % tmp_package_dir
166
+ end
167
+
168
+ sh "rm -rf %s" % tmp_package_dir
169
+ sh "rm -f %s" % cache_dir
170
+ end
171
+
172
+ end
173
+
174
+ def create_packages(directory, platform, architecture, package_formats)
175
+ Array(package_formats).map { |pkg| create_package(directory, pkg, platform, architecture) }
176
+ end
177
+
178
+ def create_package(tarball, pkg, platform, architecture)
179
+ output_name = [[PACKAGE_OUTPUT_NAME, architecture].join("_"), pkg].join(".")
180
+ extra_args = EXTRA_ARGS[pkg] || ""
181
+ sh "fpm -s tar -t %s -f -n %s -v %s -a %s --license \"%s\" --vendor \"%s\" --maintainer \"%s\" --url \"%s\" --description \"%s\" --category \"%s\" --config-files %s -C %s -p %s %s %s" % [pkg, PACKAGE_NAME, VERSION, architecture, LICENSE, VENDOR, MAINTAINER, HOMEPAGE, DESCRIPTION, PACKAGE_CATEGORY, CONFIG_DEST, File.basename(tarball, ".tar.gz"), output_name, extra_args, tarball]
182
+ output_name
183
+ end
184
+
185
+ def create_directory_bundle(target, prefix = nil)
186
+ package_dir = [PACKAGE_NAME, VERSION, target].join("_")
187
+ prefixed_dir = if prefix
188
+ File.join(package_dir, prefix)
189
+ else
190
+ package_dir
191
+ end
192
+ lib_dir = File.join(prefixed_dir, "lib")
193
+ config_dest_dir = File.join(package_dir, CONFIG_DEST)
194
+ app_dir = File.join(lib_dir, "app")
195
+ ruby_dir = File.join(lib_dir, "ruby")
196
+ dest_vendor_dir = File.join(lib_dir, "vendor")
197
+ vendor_dir = File.join("packaging", "vendor")
198
+ traveling_ruby_file = "packaging/traveling-ruby-%s-%s.tar.gz" % [TRAVELING_RUBY_VERSION, target]
199
+ spec_path = SPEC_PATH
200
+ bundle_dir = File.join(dest_vendor_dir, ".bundle")
201
+
202
+
203
+ sh "rm -rf %s" % package_dir
204
+ sh "mkdir %s" % package_dir
205
+ sh "mkdir -p %s" % prefixed_dir
206
+ sh "mkdir -p %s" % config_dest_dir
207
+ sh "mkdir -p %s" % app_dir
208
+
209
+ GEMSPEC.files.each do |file|
210
+ destination_dir = File.join(app_dir, File.dirname(file))
211
+ FileUtils.mkdir_p(destination_dir)
212
+
213
+ sh "cp %s %s" % [file, destination_dir]
214
+ end
215
+
216
+ Dir[File.join(CONFIG_DIR, "*")].each do |file|
217
+ sh "cp %s %s" % [file, config_dest_dir]
218
+ end
219
+
220
+ sh "mkdir %s" % ruby_dir
221
+ sh "tar -xzf %s -C %s" % [traveling_ruby_file, ruby_dir]
222
+
223
+ GEMSPEC.executables.each do |file|
224
+ destination = File.join(prefixed_dir, file)
225
+
226
+ File.open(destination, "w") { |f| f.write(WRAPPER_SCRIPT % File.join("bin", file)) }
227
+
228
+ sh "chmod +x %s" % destination
229
+ end
230
+
231
+ sh "cp -pR %s %s" % [vendor_dir, lib_dir]
232
+ sh "cp %s Gemfile Gemfile.lock %s" % [spec_path, dest_vendor_dir]
233
+
234
+ GEMSPEC.require_paths.each do |path|
235
+ sh "ln -sf ../app/%s %s" % [path, File.join(dest_vendor_dir, path)]
236
+ end
237
+
238
+ FileUtils.mkdir_p(bundle_dir)
239
+ File.open(File.join(bundle_dir, "config"), "w") { |f| f.write(BUNDLE_CONFIG) }
240
+ package_dir
241
+ end
242
+
243
+ def create_tarball(package_dir)
244
+ gzip_file = "%s.tar.gz" % package_dir
245
+
246
+ sh "tar -czf %s %s" % [gzip_file, package_dir]
247
+
248
+ gzip_file
249
+ end
250
+
251
+ def download_runtime(target)
252
+ traveling_ruby_name = ["traveling-ruby", TRAVELING_RUBY_VERSION, target].join("-")
253
+ traveling_ruby_file = "%s.tar.gz" % traveling_ruby_name
254
+ traveling_ruby_releases = "http://d6r77u77i8pq3.cloudfront.net/releases"
255
+ traveling_ruby_url = File.join(traveling_ruby_releases, traveling_ruby_file)
256
+
257
+ sh "cd packaging && curl -L -O --fail %s" % traveling_ruby_url
258
+ end
data/TEST.md ADDED
@@ -0,0 +1,9 @@
1
+ # Testing Installation
2
+
3
+ You can test installation of `instrumental_tools` by running the [ServerSpec tests](test/integration/default/serverspec/). From the `chef` or `puppet` directories, run the following command:
4
+
5
+ ```
6
+ bundle exec kitchen verify
7
+ ```
8
+
9
+ to test installation and setup procedures for the `instrumental_tools` command. You must have [Vagrant](https://www.vagrantup.com/) installed; currently the KitchenCI integration is setup to use [VMWare Fusion](http://www.vmware.com/products/fusion) and the [VMWare Fusion Vagrant provider](https://www.vagrantup.com/vmware); you can configure a separate provider for your specific setup by change the `provider` flag in the `.kitchen.yml` file for your particular setup.
@@ -8,35 +8,48 @@ rescue Gem::LoadError
8
8
  puts ' gem install instrumental_agent'
9
9
  exit 1
10
10
  end
11
+
11
12
  require 'etc'
12
13
  require 'instrumental_agent'
13
14
  require 'fileutils'
14
15
  require 'optparse'
15
16
  require 'socket'
17
+ require 'tmpdir'
18
+
16
19
  $: << File.join(File.dirname(__FILE__), "..", "lib")
17
20
  require 'instrumental_tools/version'
18
21
  require 'instrumental_tools/server_controller'
19
22
 
20
23
  def require_api_key(options, parser)
21
- if options[:api_key].to_s.strip.empty?
24
+ if options[:api_key].to_s.strip.empty? && !File.exists?(options[:config_file])
22
25
  print parser.help
23
26
  exit 1
24
27
  end
25
28
  end
26
29
 
27
- default_script_directory = File.join(Dir.home, '.instrumental_scripts')
30
+
31
+ cur_directory = Dir.pwd
32
+ home_directory = Dir.home rescue nil
33
+ script_location = File.expand_path(File.dirname(__FILE__))
34
+ tmp_dir = Dir.tmpdir
35
+ script_data_directory = [cur_directory, home_directory, script_location, tmp_dir].compact.detect { |dir| File.writable?(dir) }
36
+
37
+ default_script_directory = File.join(script_data_directory, '.instrumental_scripts')
28
38
  default_command = :foreground
29
39
 
30
40
  options = {
31
41
  :collector => 'collector.instrumentalapp.com',
32
42
  :port => '8000',
33
43
  :hostname => Socket.gethostname,
34
- :pid_location => File.join(Dir.home, 'instrument_server.pid'),
35
- :log_location => File.join(Dir.home, 'instrument_server.log'),
44
+ :pid_location => File.join(script_data_directory, 'instrument_server.pid'),
45
+ :log_location => File.join(script_data_directory, 'instrument_server.log'),
46
+ :tmp_location => Dir.tmpdir,
36
47
  :enable_scripts => false,
37
48
  :script_location => default_script_directory,
38
49
  :report_interval => 30,
39
- :debug => false
50
+ :debug => false,
51
+ :config_file => '/etc/instrumental.yml',
52
+ :user => nil
40
53
  }
41
54
 
42
55
  option_parser = OptionParser.new do |opts|
@@ -49,6 +62,10 @@ Default command: #{default_command.to_s}
49
62
  options[:api_key] = api_key
50
63
  end
51
64
 
65
+ opts.on('-f', '--config-file PATH', "Config file with location of your API key (default #{options[:config_file]})") do |path|
66
+ options[:config_file] = path
67
+ end
68
+
52
69
  opts.on('-c', '--collector COLLECTOR[:PORT]', "Collector (default #{options[:collector]}:#{options[:port]})") do |collector|
53
70
  address, port = collector.split(':')
54
71
  options[:collector] = address
@@ -79,6 +96,14 @@ Default command: #{default_command.to_s}
79
96
  options[:script_location] = path
80
97
  end
81
98
 
99
+ opts.on('-u', '--user USER_TO_RUN_AS', "User to run instrument_server as. You must have permissions to drop privileges to this user.") do |u|
100
+ options[:user] = u
101
+ end
102
+
103
+ opts.on('-t', '--temp-dir TEMP_DIRECTORY', "Where to store temporary files (default #{options[:tmp_location]})") do |t|
104
+ options[:tmp_location] = t
105
+ end
106
+
82
107
  opts.on('--debug', "Print all sent metrics to the log") do
83
108
  options[:debug] = true
84
109
  end
@@ -91,9 +116,25 @@ end
91
116
 
92
117
  option_parser.parse!
93
118
 
119
+ if options[:user]
120
+ desired_uid = Etc.getpwnam(options[:user]).uid
121
+ Process::Sys.setuid(desired_uid)
122
+ if desired_uid && desired_uid != 0
123
+ begin
124
+ Process::Sys.setuid(0)
125
+ rescue Errno::EPERM
126
+ nil
127
+ else
128
+ puts "Cannot drop privileges to #{options[:user]}"
129
+ exit 1
130
+ end
131
+ end
132
+ end
133
+
94
134
  command = ARGV.first && ARGV.first.to_sym
95
135
  command ||= default_command
96
136
 
137
+
97
138
  options[:api_key] ||= ENV["INSTRUMENTAL_TOKEN"]
98
139
 
99
140
  if options[:pid_location].to_s.strip.empty?
@@ -148,7 +189,7 @@ running_as_daemon = [:start, :restart].include?(command)
148
189
 
149
190
  controller = ServerController.spawn(
150
191
  :name => File.basename(__FILE__),
151
- :path => Dir.pwd,
192
+ :path => options[:tmp_location],
152
193
  :pid_file => options[:pid_location],
153
194
  :verbose => true,
154
195
  :log_file => options[:log_location],
@@ -165,7 +206,7 @@ end
165
206
 
166
207
  if running_as_daemon
167
208
  begin
168
- Timeout.timeout(5) do
209
+ Timeout.timeout(1) do
169
210
  Process.waitpid(controller.pid)
170
211
  end
171
212
  rescue Timeout::Error
data/chef/.kitchen.yml ADDED
@@ -0,0 +1,16 @@
1
+ ---
2
+ driver:
3
+ name: vagrant
4
+ provider: vmware_fusion
5
+
6
+ provisioner:
7
+ name: chef_solo
8
+
9
+ platforms:
10
+ - name: ubuntu-12.04
11
+ - name: centos-6.6
12
+
13
+ suites:
14
+ - name: default
15
+ run_list: ["instrumental_tools"]
16
+ attributes:
data/chef/Berksfile ADDED
@@ -0,0 +1,4 @@
1
+ source "https://supermarket.chef.io"
2
+
3
+ cookbook "packagecloud"
4
+ cookbook "instrumental_tools", path: "instrumental_tools"
@@ -0,0 +1,9 @@
1
+ DEPENDENCIES
2
+ instrumental_tools
3
+ path: instrumental_tools
4
+ packagecloud
5
+
6
+ GRAPH
7
+ instrumental_tools (0.0.0)
8
+ packagecloud (>= 0.0.0)
9
+ packagecloud (0.0.17)
@@ -0,0 +1,2 @@
1
+ default[:instrumental] = {}
2
+ default[:instrumental][:api_key] = nil
@@ -0,0 +1,3 @@
1
+ name "instrumental_tools"
2
+
3
+ depends "packagecloud"