cult 0.1.1.pre

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: cea2fc50c8b92985e08159d77c53f16cb71569ce
+  data.tar.gz: cba93e78130a741980b6f3be07038edb2a9035fc
+SHA512:
+  metadata.gz: 70ccf020691c38b75dce07f9e408cce0f0849a1d54b06a96038de5b0f4c459be024822f6931b433c115e04fa318b461e05f937faad60f7b85ce492474efa72bc
+  data.tar.gz: 98370627f82c3c4aa5c240e66aa2dfce29c17f4d8368e509c205651dfd0fdcd97d4a0aef2f9711fe03f28c5d9e158206ee10771108bc04c5a6b01b8965c02525

data/.gitignore

@@ -0,0 +1,9 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/pkg/
+/spec/reports/
+/tmp/
+.tags

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in cult.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 Mike Owens, meter.md
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,240 @@
+# Cult
+
+## Introduction
+
+Cult is a tool to manage fleets of servers. It tries to work in an obvious
+way, and doesn't intend on you learning a bunch of new metaphors, languages,
+and terminology.
+
+Cult may be what you're looking for if:
+
+  * You like the transparency of shell-script setup but have outgrown it, and
+    it's turning into a tangled mess
+  * "Configuration Management Systems" and "Known Configuration State" stuff
+    rubs you the wrong way
+  * You're not worried about abstracting away Unix, as if you'll be deploying
+    on a herd of Amigas next year
+  * You're not looking to find a cloud-hosted meta-script to effectively
+    `apt-get -y install nginx`
+  * You have no ceremony around spinning up and killing servers. Cult can
+    manage real metal, but its default mindset is: if you fuck up too bad, you
+    can spin up a fresh instance
+  * When you think of a forward migration, the first thing that comes to mind
+    is `#!...`
+  * You don't get why you need more than a working SSH to configure a server,
+    and why you'd want an agent running thereafter.
+  * It doesn't creep you out that Cult can look at your git branch name to
+    decide if you're in production or development mode. There are some
+    conventions going on.
+
+
+Cult's probably not your bourbon and ginger if:
+
+  * You see value in "converging toward a configuration", as if you're guiding
+    your precious children through the path of life, and helping them evolve
+    into better people.
+  * You have one big-ass old server that's gets conservatively upgraded via
+    efforts big enough to have a project name. Cult can help you *out* of
+    that, though.
+  * You're sold on image-in-a-container-in-a-KVM deployment. That's totally
+    reasonable, but Cult doesn't really do anything particularly special
+    to help you if you're happy with the process you have building containers.
+  * You expect to have the same configuration abstractly work on a totally
+    diverse set of platforms (e.g., one "formula" that works on Ubuntu,
+    FreeBSD, and SCO OpenServer 5). Cult can manage these absolutely fine, but
+    you lose a lot of the benefit of its role-based sharing, and will find
+    yourself in the script-equivalent of `#ifdef`-hell.
+  * You've already got a working and complicated network, managed by years
+    worth of highly-tuned inputs into a Configuration Management System. Cult
+    itself does less for you, but requires a lot less of you.
+
+But, what you gain via Cult is transparency, repeatability, and obviousness.
+
+
+## Installation
+
+Cult is written in Ruby and available as a gem, but it is an application, not a
+library. It's not written in, and has no relation to Rails. It depends on
+Ruby 2.3 or greater. If you've got Ruby installed, via any means, the following
+command should handle everything:
+
+    $ gem install cult
+
+Most provider drivers require outside gems, for example, the Linode driver
+requires the Linode API gem, the DigitalOcean driver requires DropletKit,
+etc. Cult will ask, and then install these required gems only when you go to
+use the driver.
+
+Cult requires nothing to be installed on each node, other than an operating SSH
+server and Bourne Shell. If you know you've got Bash on the other end, feel
+free to write your tasks in Bash. If you want to write tasks in Ruby, Python,
+Node or Perl, etc, one of your firsts Tasks in the `all` role should be to
+`apt-get -y install {ruby,python,node}`. All subsequent tasks will have that
+interpreter available.
+
+
+## General Theory
+
+I think a reasonable level of abstraction for cloud deployments is such:
+
+  1. *Nodes*: Actual machines, virtual or otherwise, running somewhere.
+  2. *Roles*: The purpose of a node. e.g., `web-frontend`, `db-master`, or
+     `redis-server`.
+  3. *Tasks*: Roles are made of tasks, which are basically scripts, called
+     things like `install-postgres` or `configure-nginx`. They're written in
+     your language of choice. When you being up a Node with a Role, all of the
+     Role's tasks are executed to get it up and running.
+  4. *Providers*: Your VPS provider, e.g., Linode, DigitalOcean, etc. These
+     allow you to spawn and destroy nodes, and typically charge you a few cents
+     per node per hour. I guess there's also *Drivers*, but a Provider is
+     really just an instance of a *Driver* with an API key configured.
+
+We hate adding anything on top of this because it increases complexity.
+
+Sometimes, what you need is not baked into Cult. So Cult is full of escape
+hatches like ERB templating on shell scripts and JSON files to do weird stuff.
+
+
+### Drivers
+
+Cult provides a handful of Drivers which can talk to common VPS providers,
+initially DigitalOcean, Linode, and Vultr. A driver is typically 200-300 lines
+of Ruby code, and is pretty well isolated from the rest of Cult, so feel free
+to open a PR to add your provider of choice. You can get a current list with:
+
+    $ cult provider drivers
+
+The goal of a driver is to talk to your VPS provider, start a server with a
+size/instance type, zone/region, distribution, and ssh key you provide. It
+then makes sure it's configured to allow a root login with that SSH key.
+
+The Driver gets you to a "pre-bootstrap" stage where your server exists and
+you can connect to it. The `bootstrap` role can then kick in. Your `boostrap`
+role will typically create the `cult` user, disable the root account, etc (the
+default bootstrap role indeed does exactly this.)
+
+### Nodes
+
+A node is a physical instance running somewhere. A node has a name, like
+'web1', but also has a full description, that looks something like:
+
+  web1@ubuntu-16-01.2gb:digitalocean.nyc1
+
+A node named `web1` is an idea, but hasn't spawned. A node named with its full
+description should represent a real server somewhere, that is (hopefully)
+running.
+
+### Roles
+A role is a collection of files (usually configuration files), Tasks (usually
+shell scripts), and a configuration (`role.json`). A role can include other
+roles via `includes:`. A role can state it conflicts with another role via
+`conflicts:`.
+
+A Role's tasks are named like `00000-a-descriptive-name`, because the only
+ordering Cult does is asciibetical.
+
+Every task, file, and even `role.json` is pre-processed with ERB before it gets
+processed by Cult or shipped to a node. This lets you customize behavior based
+on the node, the role, the provider, the project, or anything else you'd like.
+
+#### Special Roles
+There are two Roles generated by default that you should think of as
+special-ish:
+
+  1. `all`: If a Role does not explicitly list an `includes` value, Cult will
+     act as if it specified `includes: ['all']`. In practice, this means tasks
+     in the `all` role are common to all nodes that haven't explicitly opted
+     out of it. A task can opt-out of including `all` with an explicit
+     `includes: []`.
+  2. `bootstrap`: The generator creates this role to be the first one ran on a
+     new node. Before it starts, we have a root account with an SSH key, when
+     `bootstrap` finishes, we have a `cult` user on the node with `sudo` access
+     who uses the same SSH key, and the root account is disabled. `bootstrap`
+     opts-out of `all`. The generator also installs a MOTD banner so you'll
+     know Cult was enabled on the server, has a demo script that sets the
+     hostname.
+
+## UI
+
+Even though you aren't required to use it, Cult includes the terminal-mode GUI
+packages (`$ cult ui`). Because it's 2016, and we're not worried about a few
+MB of extra packages hanging around anymore. We're steadily trying to improve
+this cool-ass tmux-based UI, but it's still new right now.
+
+## Usage
+
+We're going to put together a complete usage guide, tutorial, and example repo
+once Cult has settled down a bit. It's still pre-1.0 software, and we still
+like breaking things to make it work better for us.
+
+### 👻 Spooky Secrets
+
+  * `cult console` is built to be really nice to use. If you're not afraid of
+    Ruby, the method names are chosen to read almost like pseudo-code. It
+    supports IRB, Pry, and Ripl with command-line flags.
+  * Anything that Cult keeps "an Array of", that you'd maybe want to reference
+    by name is stored in a NamedArray, which shares some features with a Hash.
+    This is for convenience. For example, in `cult console`, you can find the
+    first driver with `drivers[0]`, find it by name with `drivers['linode']`,
+    or (*get ready for fancy stuff:*) look it up by a Regexp with
+    `drivers[/ocean/i]`.
+  * The NamedArray stuff even works on the command-line with String arguments,
+    and will convert strings that start with '/' to Regexps to search by name.
+  * Although Cult will only *generate* JSON, not having comments and other
+    stuff is a pain. If you don't care about JSON-readability of your
+    `node.json`s or `role.json`s, you can just rename it to `node.yaml` or
+    `node.yml` and it'll get parsed with a YAML parser. Cult keeps transient
+    state in separate files for this reason: so it doesn't overwrite your
+    long-lived YAML replacements with JSON equivalents.
+
+
+## Development
+
+### History
+
+Cult was developed to basically avoid Puppet, Chef, and Ansible. I know there
+are some really large, successful deployments of all of these. I just think
+they just do too much for me to feel safe with when shit goes crazy.
+
+Some of the things Cult doesn't do are things we haven't had time to implement,
+like fully building out 'cult ui'. A lot of the things Cult doesn't do are
+the reason Cult exists. Some of the limitations are designed to force a
+certain mindset I think is healthy for building resilient systems. In
+particular:
+
+  1. Exercising the server bring-up process from bottom-up as the normal mode
+     of operation. This is why Cult makes you bring up a node from provision/
+     bootstrap each time, instead of using snapshots, a feature virtually every
+     VPS provider supports.
+  2. Making it less reasonable to have nodes hanging around that have ended up
+     in their current state via baby-step migrations for too long. Cult does
+     this fine, but makes it easier to just test a new node with a clean build.
+     This way you're not worried about idempotent transforms, or a lot of
+     conditional logic. If a node is not where you want, bring up another that
+     is, and kill the old one.
+
+We can be convinced otherwise, but these are sort of core tenets of what Cult
+is about.
+
+### Contributing
+
+We greatly appreciate bug reports, pull-requests, questions, and general
+commentary in the GitHub Issues. These are all *contributions*. However,
+before opening an issue demanding us to work on your feature that Cult *has to
+have to be taken seriously*, note:
+
+  *  Cult was built by meter.md nerds to make managing our infrastructure
+     better. Its utility is measured by that metric. We've released the
+     project because we believe in the value of open *collaboration*, not so we
+     can be unpaid software contractors working on the demands of strangers on
+     the internet.
+  *  If your contribution consists of instructing the team how to run the
+     project: Thanks, but we've got that handled.
+
+If you're contributing code, asking a question, or reporting a bug, don't let
+the above items you away.
+
+## License
+
+Cult is available as open source software under the terms of the
+[MIT License](http://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+task :default => :spec
+
+task :tags do
+  sh "ctags -R ."
+end

data/cult

@@ -0,0 +1 @@
+./exe/cult

data/cult.gemspec

@@ -0,0 +1,38 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'cult/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "cult"
+  spec.version       = Cult::VERSION
+  spec.authors       = ["Mike Owens"]
+  spec.email         = ["mike@meter.md"]
+
+  spec.summary       = "Fleet Management like its 1990"
+  spec.homepage      = "https://github.com/metermd/cult"
+  spec.license       = "MIT"
+
+  # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host'
+  # to allow pushing to a single host or delete this section to allow pushing to any host.
+  if spec.respond_to?(:metadata)
+    spec.metadata['allowed_push_host'] = "https://rubygems.org"
+  else
+    raise "RubyGems 2.0 or newer is required to protect against public gem pushes."
+  end
+
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.required_ruby_version = '~> 2.3'
+
+  spec.add_dependency "cri",       "~> 2.7.0"
+  spec.add_dependency "net-ssh",   "~> 3.2.0"
+  spec.add_dependency "net-scp",   "~> 1.2.1"
+  spec.add_dependency "rainbow",   "~> 2.1.0"
+
+  spec.add_development_dependency "bundler", "~> 1.12"
+  spec.add_development_dependency "rake", "~> 10.0"
+end

data/doc/welcome.txt

@@ -0,0 +1 @@
+Welcome to cult.

data/exe/cult

@@ -0,0 +1,86 @@
+#!/usr/bin/env ruby
+$LOAD_PATH << File.expand_path(File.join(__dir__, '../lib'))
+
+require 'cri'
+
+require 'cult/version'
+require 'cult/config'
+require 'cult/project'
+require 'cult/node'
+require 'cult/cli/load'
+require 'cult/drivers/load'
+
+cult = Cri::Command.define do
+  optional_project
+  name        'cult'
+  usage       'cult [options] [command [options...]]'
+  summary     'Control a Fleet of Obedient Zomboid Machines'
+  description <<~EOD.format_description
+    Cult is a tool for creating and then managing a fleet of servers you
+    control.  It operates on a few simple concepts:
+
+      * Nodes: actual servers out there somewhere.  The purpose of using Cult
+        is to end up with nodes doing useful work for you.
+
+      * Roles: Every node has one or more roles, things it plans on being.
+        Roles are composed of...
+
+      * Tasks: Basically shell scripts that run in a specific order.
+
+   Cult has a few more convenience concepts, like Keys and Providers, but you
+   don't end up thinking about them too often.
+
+   To create a new Cult project, use 'cult init DIRECTORY', but see the 'init'
+   help first, with 'cult init --help'
+  EOD
+
+  required :C, :directory, 'Specify a project path' do |value|
+    Cult::CLI.set_project(value)
+  end
+
+  flag :h, :help, 'Show this help' do |value, cmd|
+    puts cmd.help
+  end
+
+  flag :y, :yes, 'Answer "yes" to any questions' do
+    Cult::CLI.yes = true
+  end
+
+  flag :q, :quiet, "Don't show any unnecessary information" do
+    Cult::CLI.quiet = true
+  end
+
+  flag :v, :version, 'Show version information' do
+    puts "Cult #{Cult::VERSION}"
+    puts 'Copyright (C) 2016 Mike A. Owens, meter.md, and Contributors'
+  end
+
+  run(arguments: 0) do |opts, args, cmd|
+    puts cmd.help
+  end
+end
+
+Cult::CLI.commands.each do |m|
+  cult.add_command(m)
+end
+
+if (env = ENV['CULT_PROJECT'])
+  Cult::CLI.set_project(env)
+else
+  Cult.project ||= Cult::Project.from_cwd
+end
+
+if Cult.project
+  Cult.project.execute_cultrc
+end
+
+ERROR_CULT = Rainbow("#{File.basename($0)}:").red
+
+begin
+  cult.run(ARGV)
+rescue Cult::CLI::CLIError, RegexpError => e
+  $stderr.puts "#{ERROR_CULT} #{e.message}"
+  exit 1
+rescue Interrupt
+  exit 1
+end

data/lib/cult/artifact.rb

@@ -0,0 +1,45 @@
+require 'cult/transferable'
+require 'cult/named_array'
+
+module Cult
+  # I'd love to just call this "File", but the ambiguity with ::File would
+  # make it a pain.
+  class Artifact
+    include Transferable
+
+    def self.collection_name
+      "files"
+    end
+
+
+    def relative_path
+      name
+    end
+
+
+    attr_reader :path
+    attr_reader :role
+
+    def initialize(role, path)
+      @role = role
+      @path = path
+    end
+
+
+    def inspect
+      "\#<#{self.class.name} role:#{role&.name.inspect} name:#{name.inspect}>"
+    end
+    alias_method :to_s, :inspect
+
+
+    def self.all_for_role(project, role)
+      Dir.glob(File.join(role.path, "files", "**/*")).map do |filename|
+        next if File.directory?(filename)
+        new(role, filename).tap do |new_file|
+          yield new_file if block_given?
+        end
+      end.compact.to_named_array
+    end
+
+  end
+end