wilbur 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.gitignore +8 -0
- data/.ruby-version +1 -0
- data/Gemfile +5 -0
- data/README.md +93 -0
- data/Rakefile +16 -0
- data/Thorfile +145 -0
- data/Vagrantfile +46 -0
- data/flavors/README.md +33 -0
- data/flavors/ar71xx_nand_3g_usb/dot_config +3932 -0
- data/flavors/flavors.yml +4 -0
- data/lib/wilbur/version.rb +3 -0
- data/openwrt_release +1 -0
- data/provisioning/manifests/builder.pp +35 -0
- data/provisioning/manifests/provisioner.pp +0 -0
- data/provisioning/scripts/openwrt_release.sh +6 -0
- data/wilbur.gemspec +40 -0
- metadata +138 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 417d87a72995963bd253992473a7187df70a3a9d
|
4
|
+
data.tar.gz: ea7e93c56c213dac1aa3516ab3945e6717c05345
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 74b54c9428e99513ef221cf1a59b2895a681c9fb18011163cc5547da8c65f4bd2db8b44c9ab1ae9ef61766585fda837d01c592a5a0c56f677b58fd66defe6eac
|
7
|
+
data.tar.gz: 2505db8a85a9dbb82820fb9326a2570a28f3ee074702f0a3f836e080709967c6269ee0f9ba8b353a5b78406a21665ea9d66065d648ef1bd6521edff2354f2147
|
data/.gitignore
ADDED
data/.ruby-version
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
2.0.0-p195
|
data/Gemfile
ADDED
data/README.md
ADDED
@@ -0,0 +1,93 @@
|
|
1
|
+
# Wilbur: Your OpenWRT Build Assistant
|
2
|
+
|
3
|
+
This project provides a comprehensive infrastructure for building and
|
4
|
+
provisioning OpenWRT images easily and in an automated fashion.
|
5
|
+
|
6
|
+
For more information about OpenWRT, please check http://openwrt.org
|
7
|
+
|
8
|
+
## Project structure
|
9
|
+
* `openwrt_release`: here you specify the name of the release targeted for
|
10
|
+
build and deployment
|
11
|
+
* `flavors/`: contains the different kernel and rootfs configurations used
|
12
|
+
to build working OpenWRT images
|
13
|
+
* `flavors.yml`: serves as a db of flavor configurations. Used to provide
|
14
|
+
more understanding over each configuration and for referencing them more
|
15
|
+
easily by the user.
|
16
|
+
* `<flavor_name>/`: contains all files related to the specific flavor
|
17
|
+
* `dot_config`: Buildroot configuration file used to specify kernel
|
18
|
+
configuration and packages to compile/install in the final image
|
19
|
+
* `patches/`: contains all diff patches to be applied to the OpenWRT's
|
20
|
+
source code prior to building it
|
21
|
+
* `provisioning/`: contains the resources used to provision project's virtual
|
22
|
+
machines
|
23
|
+
* `manifests/`: contains the Puppet manifests used to bring up the building and
|
24
|
+
provisioning VMs
|
25
|
+
* `scripts/`: contains the scripts used with Vagrant to aid Puppet
|
26
|
+
provisioning
|
27
|
+
|
28
|
+
## Usage
|
29
|
+
|
30
|
+
At the moment, all facilites are provided as **Thor**
|
31
|
+
commands. Also, **Bundler** is used to contain the project in its own folder.
|
32
|
+
|
33
|
+
To start using the builder, just let Bundler do its work:
|
34
|
+
|
35
|
+
`bundle install --path vendor/bundle`
|
36
|
+
|
37
|
+
Then, to see a list of the available commands, run:
|
38
|
+
|
39
|
+
`bundle exec thor -T`
|
40
|
+
|
41
|
+
To see a description of the use cases supported by the project, read on this
|
42
|
+
doc file.
|
43
|
+
|
44
|
+
### Declaring OpenWRT release to build
|
45
|
+
In the project's root folder there's a file named `openwrt_release`. This file
|
46
|
+
must contain a single line with the name of the specific OpenWRT release
|
47
|
+
you want to build (e.g. `attitude_adjustment`). If you want to build bleeding
|
48
|
+
edge, just set it to `openwrt`.
|
49
|
+
|
50
|
+
#### Internals
|
51
|
+
The name specified in `openwrt_release` is used to build the URL of the source
|
52
|
+
repository to clone locally (so format matters). Basically, source will be fetched
|
53
|
+
from
|
54
|
+
|
55
|
+
`git://nbd.name/<openwrt_release>.git`
|
56
|
+
|
57
|
+
To provide this same name to Puppet during provisioning, some
|
58
|
+
"pre-provisioning" is done prior to running Puppet; during this step, a custom
|
59
|
+
executable fact will be added to the **building machine**, named
|
60
|
+
`openwrt_release`, that will read the content of the file in the shared project
|
61
|
+
folder. This way, you can specify the release you want to use once and not
|
62
|
+
worry about synchronizing names between building app and provisioning manifest.
|
63
|
+
|
64
|
+
### Building a specific flavor
|
65
|
+
You can easily start a build by invoking the `build` task:
|
66
|
+
|
67
|
+
`bundle exec thor build <flavor>`
|
68
|
+
|
69
|
+
To obtain a list of available flavors, you can ask thor:
|
70
|
+
|
71
|
+
`bundle exec thor flavor_list`
|
72
|
+
|
73
|
+
This will return the list of available flavors. Each row will be in the form:
|
74
|
+
|
75
|
+
`<flavor name> <flavor description>`
|
76
|
+
|
77
|
+
The description is there to clarify what the flavor's purpose is; you can just
|
78
|
+
pick the name and pass it over to the `build` task.
|
79
|
+
|
80
|
+
Also, additional options are available for the `build` command:
|
81
|
+
|
82
|
+
* `--verbose`: let the compilation process be verbose about what's doing
|
83
|
+
(useful for debugging)
|
84
|
+
* `--nuke`: prior to start compilation, delete **everything** that is not bare
|
85
|
+
source code. _Note that in standard operation mode, build command already
|
86
|
+
wipes the output of previous run(s), so in normal circumstances this flag is
|
87
|
+
not needed. You may need it if you mess with the configuration and want to
|
88
|
+
restart with a clean environment. For more information, have a look at
|
89
|
+
[`make distclean` Buildroot command](http://wiki.openwrt.org/doc/howto/build)._
|
90
|
+
|
91
|
+
**NOTE**: At the moment, when the build process ends, nothing is done to gather
|
92
|
+
the produced files. For now, you have to ssh into the **builder** box and find
|
93
|
+
the kernel and rootfs under `attitude_adjustment/bin/<arch>/`.
|
data/Rakefile
ADDED
@@ -0,0 +1,16 @@
|
|
1
|
+
require 'rake/testtask'
|
2
|
+
require 'wilbur/version'
|
3
|
+
|
4
|
+
namespace :gem do
|
5
|
+
require 'bundler/gem_tasks'
|
6
|
+
end
|
7
|
+
|
8
|
+
namespace :test do
|
9
|
+
Rake::TestTask.new(:end_to_end) do |t|
|
10
|
+
t.verbose = true
|
11
|
+
t.libs << "lib"
|
12
|
+
t.libs << "test"
|
13
|
+
t.test_files = FileList["test/end_to_end/**/*_test.rb"]
|
14
|
+
#t.ruby_opts = ["-w"]
|
15
|
+
end
|
16
|
+
end
|
data/Thorfile
ADDED
@@ -0,0 +1,145 @@
|
|
1
|
+
# -*- mode: ruby -*-
|
2
|
+
# vim: ft=ruby
|
3
|
+
|
4
|
+
require 'thor'
|
5
|
+
require 'net/ssh'
|
6
|
+
|
7
|
+
class Default < Thor
|
8
|
+
desc "build FLAVOR", "builds the specified OpenWRT flavor on the building machine"
|
9
|
+
long_desc <<-EOS
|
10
|
+
Builds the specified OpenWRT flavor on the building node.
|
11
|
+
|
12
|
+
This means compiling the kernel and generating one or more OpenWRT image
|
13
|
+
files.
|
14
|
+
\x5The kernel will be configured with the options specified in the flavor
|
15
|
+
configuration file; in the same file (which is a standard kernel dot file
|
16
|
+
suitable for OpenWRT Buildroot) will be described which packages are to be compiled
|
17
|
+
and which ones are to be installed in the preconfigured image.
|
18
|
+
|
19
|
+
You can get a list of available flavors by running:
|
20
|
+
\x5
|
21
|
+
\x5\t thor flavor_list
|
22
|
+
|
23
|
+
The name specified in the first column can be used as an argument for the
|
24
|
+
build command.
|
25
|
+
EOS
|
26
|
+
option :verbose, :type => :boolean, :aliases => :v
|
27
|
+
option :nuke, :type => :boolean, :aliases => :n
|
28
|
+
def build(flavor)
|
29
|
+
if flavors.select { |f| f["name"] == flavor}.empty?
|
30
|
+
puts "No flavor found matching `#{flavor}`"
|
31
|
+
exit 1
|
32
|
+
end
|
33
|
+
|
34
|
+
if not Dir.exists?(File.join('flavors', flavor))
|
35
|
+
puts <<-EOS
|
36
|
+
Sorry, no directory flavors/#{flavor} exist. Please read how flavors
|
37
|
+
management work and fix your configuration before proceeding.
|
38
|
+
EOS
|
39
|
+
exit 1
|
40
|
+
end
|
41
|
+
|
42
|
+
dot_config = File.join(flavor, 'dot_config')
|
43
|
+
if not File.exists?(File.join('flavors', dot_config))
|
44
|
+
puts <<-EOS
|
45
|
+
Sorry, flavors/#{flavor}/dot_config file doesn't exist. Please read how flavors
|
46
|
+
management work and fix your configuration before proceeding.
|
47
|
+
EOS
|
48
|
+
exit 1
|
49
|
+
end
|
50
|
+
|
51
|
+
verbose = options[:verbose] ? "V=s" : ""
|
52
|
+
|
53
|
+
invoke :prepare_builder, [], []
|
54
|
+
|
55
|
+
Net::SSH.start('', nil, ssh_options_for(BUILDER_VM)) do |builder|
|
56
|
+
builder.exec! "cd #{openwrt_release}; make distclean" if options[:nuke]
|
57
|
+
|
58
|
+
puts "Preparing building environment"
|
59
|
+
builder.exec! "
|
60
|
+
cd #{openwrt_release}
|
61
|
+
git pull
|
62
|
+
./scripts/feeds update -a
|
63
|
+
./scripts/feeds install -a" do |channel, stream, data|
|
64
|
+
$stdout << data
|
65
|
+
end
|
66
|
+
|
67
|
+
puts "Syncing OpenWRT dot config file"
|
68
|
+
builder.exec! "cp #{File.join(VAGRANT_SHARE_ROOT, 'flavors', dot_config)} #{File.join(openwrt_release, '.config')}"
|
69
|
+
|
70
|
+
puts "Starting image build process"
|
71
|
+
builder.exec! "
|
72
|
+
cd #{openwrt_release}
|
73
|
+
make clean
|
74
|
+
make #{verbose}" do |channel, stream, data|
|
75
|
+
$stdout << data
|
76
|
+
end
|
77
|
+
|
78
|
+
puts "Build finished."
|
79
|
+
end
|
80
|
+
end
|
81
|
+
|
82
|
+
desc "prepare_builder", "prepares builder VM for subsequent use"
|
83
|
+
def prepare_builder
|
84
|
+
case `vagrant status #{BUILDER_VM}`
|
85
|
+
when /vm is running/i then ['provision']
|
86
|
+
when /to resume this vm/i then ['up', 'provision']
|
87
|
+
else ['up']
|
88
|
+
end.each do |cmd| system 'vagrant', cmd, BUILDER_VM end
|
89
|
+
end
|
90
|
+
|
91
|
+
desc "flavor_list", "list available configurations for kernel compilation and rootfs configuration"
|
92
|
+
def flavor_list
|
93
|
+
flavors.each do |config|
|
94
|
+
puts "#{config["name"]} : #{config["desc"]}"
|
95
|
+
end
|
96
|
+
rescue Exception => e
|
97
|
+
puts e.message
|
98
|
+
exit 1
|
99
|
+
end
|
100
|
+
|
101
|
+
no_commands do
|
102
|
+
BUILDER_VM = 'builder'.freeze
|
103
|
+
VAGRANT_SHARE_ROOT = '/vagrant'.freeze
|
104
|
+
|
105
|
+
def flavors
|
106
|
+
YAML::load_file("flavors/flavors.yml")
|
107
|
+
end
|
108
|
+
|
109
|
+
def ssh_options_for(host)
|
110
|
+
ssh_config = `vagrant ssh-config #{host}`
|
111
|
+
|
112
|
+
return if ssh_config.empty?
|
113
|
+
|
114
|
+
net_ssh_options = {}
|
115
|
+
hashed_ssh_config(ssh_config).each do |option|
|
116
|
+
param = options_map[option[:key]]
|
117
|
+
net_ssh_options[param] = option[:value] if param
|
118
|
+
end
|
119
|
+
|
120
|
+
net_ssh_options
|
121
|
+
end
|
122
|
+
|
123
|
+
def hashed_ssh_config(config_string)
|
124
|
+
config_string.split("\n").map do |option|
|
125
|
+
fields = option.strip.split(' ', 2)
|
126
|
+
{ :key => fields.first, :value => fields.last.gsub('"','') }
|
127
|
+
end
|
128
|
+
end
|
129
|
+
|
130
|
+
def options_map
|
131
|
+
{ "HostName" => :host_name,
|
132
|
+
"User" => :user,
|
133
|
+
"Port" => :port,
|
134
|
+
"IdentityFile" => :keys,
|
135
|
+
"UserKnownHostsFile" => :user_known_hosts_file,
|
136
|
+
"IdentitiesOnly" => :keys_only,
|
137
|
+
}
|
138
|
+
end
|
139
|
+
|
140
|
+
def openwrt_release
|
141
|
+
@openwrt_release = File.read('openwrt_release') unless @openwrt_release
|
142
|
+
@openwrt_release
|
143
|
+
end
|
144
|
+
end
|
145
|
+
end
|
data/Vagrantfile
ADDED
@@ -0,0 +1,46 @@
|
|
1
|
+
# -*- mode: ruby -*-
|
2
|
+
# vi: ft=ruby
|
3
|
+
|
4
|
+
# Adds a `openwrt_relase` custom fact, fetched from projects root
|
5
|
+
$add_facter = <<SCRIPT
|
6
|
+
sudo mkdir -p /etc/facter/facts.d
|
7
|
+
sudo cp /vagrant/provisioning/scripts/openwrt_release.sh /etc/facter/facts.d
|
8
|
+
SCRIPT
|
9
|
+
|
10
|
+
Vagrant.configure("2") do |config|
|
11
|
+
config.vm.define :builder do |builder|
|
12
|
+
builder.vm.box = "ubuntu-base"
|
13
|
+
|
14
|
+
builder.vm.hostname = "openwrt.builders.derecom.it"
|
15
|
+
|
16
|
+
builder.vm.provider :virtualbox do |vbox|
|
17
|
+
vbox.customize ["modifyvm", :id, "--memory", "1024"]
|
18
|
+
end
|
19
|
+
|
20
|
+
builder.ssh.username = "ops"
|
21
|
+
|
22
|
+
builder.vm.provision :shell, :inline => $add_facter
|
23
|
+
|
24
|
+
builder.vm.provision :puppet do |puppet|
|
25
|
+
puppet.manifests_path = "provisioning/manifests"
|
26
|
+
puppet.manifest_file = "builder.pp"
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
config.vm.define :provisioner do |provisioner|
|
31
|
+
provisioner.vm.box = "centos-base"
|
32
|
+
|
33
|
+
provisioner.vm.hostname = "openwrt.provisioning.derecom.it"
|
34
|
+
|
35
|
+
provisioner.vm.provider :virtualbox do |vbox|
|
36
|
+
vbox.customize ["modifyvm", :id, "--memory", "1024"]
|
37
|
+
end
|
38
|
+
|
39
|
+
provisioner.ssh.username = "ops"
|
40
|
+
|
41
|
+
provisioner.vm.provision :puppet do |puppet|
|
42
|
+
puppet.manifests_path = "provisioning/manifests"
|
43
|
+
puppet.manifest_file = "provisioner.pp"
|
44
|
+
end
|
45
|
+
end
|
46
|
+
end
|
data/flavors/README.md
ADDED
@@ -0,0 +1,33 @@
|
|
1
|
+
## Flavors Database
|
2
|
+
This folder contains the flavors' database, which is a list of specific
|
3
|
+
OpenWRT configurations that along with OpenWRT source code suffice to build
|
4
|
+
complete working images. Each flavor might be used as a template for a specific
|
5
|
+
model, as a template for a specific model **and** a specific feature (like 3G
|
6
|
+
over USB support), or even as a complete deployment specification (including
|
7
|
+
particular network configuration, keys, and so on).
|
8
|
+
|
9
|
+
The database is defined in `flavors.yml`, which is basically a YAML file
|
10
|
+
containing an array on its root.
|
11
|
+
|
12
|
+
Each array entry contains the following fields:
|
13
|
+
|
14
|
+
* `name`: the (short) name of the flavor.
|
15
|
+
* `desc`: a description of the flavor, useful to clarify what it contains and
|
16
|
+
when to use it
|
17
|
+
* `type`: can be `rootfs` or `ramdisk`, depending on if the final image is
|
18
|
+
meant to be installed on router's flash or to be loaded entirely in
|
19
|
+
ramdisk (as in the case of a firmware loaded from PXE - useful for
|
20
|
+
provisioning of OpenWRT itself)
|
21
|
+
|
22
|
+
Then, there must exist a folder named after the name of the flavor,
|
23
|
+
containing at least the following file:
|
24
|
+
|
25
|
+
* `dot_config`: typically this will be the `.config` file created in the
|
26
|
+
source root folder after running `make menuconfig` (or similar).
|
27
|
+
Defines the kernel configuration and which packages are to be
|
28
|
+
installed in the final image.
|
29
|
+
|
30
|
+
Also, the following optional sub-entries might be present:
|
31
|
+
|
32
|
+
* `patches/`: contains the diff files to be applied as patches to OpenWRT's
|
33
|
+
source code prior to start the build process.
|