kameleon-builder 2.1.3 → 2.2.0

data/docs/source/index.rst

@@ -1,53 +0,0 @@
-.. Kameleon documentation master file, created by
-   sphinx-quickstart on Thu Feb 13 19:01:14 2014.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
-Welcome to Kameleon's documentation!
-====================================
-
-
-.. note::
-    This documentation is currently being written...
-
-.. image:: _static/kameleon-long.png
-     :align: right
-
------
-About
------
-
-Kameleon should be seen as a simple but powerful tool to generate customized
-appliances. With Kameleon, you make your recipe that describes how to create
-step by step your own distribution. At start Kameleon is used to create custom
-kvm, docker, VirtualBox, ..., but as it is designed to be very generic
-you can probably do a lot more than that.
-
-
-------------------
-User Documentation
-------------------
-
-
-.. toctree::
-    :maxdepth: 2
-
-    installation.rst
-    getting_started.rst
-    use_cases.rst
-    recipe.rst
-    context.rst
-    commands.rst
-    workspace.rst
-    checkpoint.rst
-    persistent_cache.rst
-    aliases.rst
-    faq.rst
-    grid5000_tutorial.rst
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`

data/docs/source/install_atlas.yaml

@@ -1,25 +0,0 @@
-
-- download_tarball:
-    - exec_local: |
-               cd /tmp/
-               wget -q $$repository/$$version/atlas$$version.tar.bz2/download -O atlas$$version.tar.bz2
-    - local2in:
-        - /tmp/atlas$$version.tar.bz2
-        - /root/atlas$$version.tar.bz2
-
-- deactivating_cpu_throttling:
-    - exec_in: cpufreq-selector -g performance
-
-- install:
-    - exec_in: cd /root/
-    - exec_in: bash -c "bunzip2 -c atlas$$version.tar.bz2 | tar xfm - "
-    - exec_in: |
-               mv ATLAS ATLAS$$version                          # get unique dir name
-               mkdir ATLAS$$version/Linux_test                         # create BLDdir
-               cd ATLAS$$version/Linux_test/
-               ../configure -b 64 -D c -DPentiumCPS=2400
-               make build
-               make check
-               make ptcheck ## this is for parallel tests
-               make time ## this is interactive
-               make install

data/docs/source/install_hpl.yaml

@@ -1,24 +0,0 @@
-
-- download_tarball:
-    - exec_local: |
-               cd /tmp/
-               wget -q $$repository/hpl-$$version.tar.gz
-    - local2in:
-        - /tmp/hpl-$$version.tar.gz
-        - /root/hpl-$$version.tar.gz
-
-
-- install:
-    - exec_in: cd /root/
-    - exec_in: |
-               tar -xvf hpl-$$version.tar.gz
-               mv hpl-$$version hpl
-
-    - local2in:
-        - $$hpl_makefile
-        - /root/hpl/Make.Linux
-
-    - exec_in: |
-               rm /usr/lib/libf2c.so
-               ln -s /usr/lib/libf2c.a /usr/lib/libf2c.so
-               make -C /root/hpl/ arch=Linux

data/docs/source/installation.rst

@@ -1,51 +0,0 @@
-.. _`installation`:
-
-------------
-Installation
-------------
-
-Gem package
------------
-
-To install the latest release from `RubyGems`_:
-
-.. _RubyGems: https://rubygems.org/gems/kameleon-builder
-
-::
-
-    gem install kameleon-builder
-
-Or from source::
-
-    git clone https://github.com/oar-team/kameleon.git && cd kameleon
-    gem build kameleon-builder.gemspec
-    gem install kameleon-builder-*.gem
-
-On debian based distribution be sure to install the ``ruby-dev`` package first
-
-
-Dist packages
--------------
-
-These packages contain Kameleon and all its dependencies (Ruby, polipo and all
-gems with their native extensions already compiled). These packages are made
-with `omnibus project`_.
-
-.. _`omnibus project`: https://github.com/opscode/omnibus-ruby
-
-+-------------------------------+------------------------+--------------------------------------------+----------------------------------+
-|                               |        Platform        |                  Download                  |               MD5                |
-+-------------------------------+------------------------+--------------------------------------------+----------------------------------+
-| .. image:: _static/debian.png | **Debian 7 64bit**     | `kameleon_2.1.3-omnibus-1_amd64.deb`_      | dc69d6386b1acb5b9434e8c186ad962c |
-|   :align: center              |                        |                                            |                                  |
-+-------------------------------+------------------------+--------------------------------------------+----------------------------------+
-| .. image:: _static/ubuntu.png | **Ubuntu 12.04 64bit** | `kameleon_2.1.3-omnibus-1_amd64.deb`_      | dc69d6386b1acb5b9434e8c186ad962c |
-|   :align: center              |                        |                                            |                                  |
-+-------------------------------+------------------------+--------------------------------------------+----------------------------------+
-| .. image:: _static/centos.png | **CentOS 6.5 64bit**   | `kameleon-2.1.3_omnibus-1.el6.x86_64.rpm`_ | 7bc5cee07249f5d4c316e5ea885a2949 |
-|   :align: center              |                        |                                            |                                  |
-+-------------------------------+------------------------+--------------------------------------------+----------------------------------+
-
-
-.. _`kameleon_2.1.3-omnibus-1_amd64.deb`: http://kameleon.imag.fr/pkg/kameleon_2.1.3-omnibus-1_amd64.deb
-.. _`kameleon-2.1.3_omnibus-1.el6.x86_64.rpm`: http://kameleon.imag.fr/pkg/kameleon-2.1.3_omnibus-1.el6.x86_64.rpm

data/docs/source/persistent_cache.rst

@@ -1,34 +0,0 @@
-.. _`persistent_cache`:
-
-----------------
-Persistent Cache
-----------------
-
-.. versionadded:: 2.0.1
-
-In order to exactly reconstruct a software appliance with the same exact
-version of packages. Kameleon offer the option of creating a persistent cache
-that will catch all the software packages during the building of the software
-appliance.  Enabling other to reconstruct the same exact software appliance
-with the right package versions. Kameleon uses Polipo [1]_ which is a tiny and
-lightweight web proxy to cache most of the packages that comes form the
-network.  First of all, you have to install Polipo on your system.  If you are
-under a debian distribution you can install it using the package manager::
-
-   sudo apt-get install polipo
-
-You can as well build it from sources and then specify the path of the
-generated binary using the option ``--proxy_path``. To use, you just have to
-add the option ``--cache`` as an argument of the build command.
-For example::
-
-  kameleon build my_recipe.yaml -b /tmp/kameleon/ --cache
-
-This will create a tar file in the build directory ``/tmp/kameleon`` called
-``my_recipe-cache.tar``.  In order to use this generated cache file in
-another build, we have just to use the options ``--from_cache`` as follows::
-
-   kameleon build my_recipe.yaml -b /tmp/kameleon/ --from_cache /tmp/my_recipe-cache.tar
-
-
-.. [1] http://www.pps.univ-paris-diderot.fr/~jch/software/polipo/

data/docs/source/recipe.rst

@@ -1,177 +0,0 @@
-------
-Recipe
-------
-
-Kameleon reads YAML files, named  *recipes*, that describes how you will
-build your appliance. These files are stored in the root of your :ref:`workspace`.
-A recipe is a hierarchical structure of `Section`_, `Step`_, `Microstep`_ and
-:ref:`commands`. Here is an overview of this structure:
-
-.. code-block:: yaml
-
-    recipe
-    |
-    `-- section
-        |
-        `-- step
-            |
-            `-- microstep
-                |
-                `-- command
-
-The recipe also contains set of `Global variables`_ declaration and some
-imports like :ref:`aliases` and :ref:`checkpoint`.
-
-Here is an example of a recipe:
-
-.. literalinclude:: debian7.yaml
-   :language: yaml
-
-Section
--------
-
-Each section is a group of steps. Currently, there are 3 sections:
-
-bootstrap
-    This section contains the bootstrap of the new system and create the *in*
-    context (see :ref:`context`).
-
-setup
-    It is dedicated to install and configuration steps.
-
-export
-    Export the generated appliance in the format of your choice.
-
-
-.. _`step`:
-.. _`microstep`:
-
-Step and microstep
--------------------
-
-Each *step* contains a list of *microsteps* that contains a list of :ref:`commands`
-written in one YAML file.  To be found by Kameleon this file must be named by
-with the step name plus the YAML extension ``.yaml``. For example the
-``software_install.yaml`` step file looks like this:
-
-.. code-block:: yaml
-
-    # Software Install
-    - add_contribs_source:
-      - exec_in: perl -pi -e "s/main$/main contrib non-free/" /etc/apt/sources.list
-    - update_repositories:
-      - exec_in: apt-get -y --force-yes update
-    - upgrade_system:
-      - exec_in: apt-get -y --force-yes dist-upgrade
-    - clean:
-      - on_export_init:
-        - exec_in: apt-get -y --force-yes autoclean
-        - exec_in: apt-get -y --force-yes clean
-        - exec_in: apt-get -y --force-yes autoremove
-    # default packages
-    - packages: "ntp sudo"
-    - extra_packages:
-      - exec_in: apt-get -y --force-yes install $$packages
-
-
-A step will be called like a function in the recipe. You should provide a set
-of local variables if needed by the step or to override default variables (see
-Variables_). Optionally, you can select only some microsteps to execute. Here
-is an example of step call:
-
-.. code-block:: yaml
-
-    - software_install:
-        - update_repositories
-        - add_contribs_source
-        - clean
-        - extra_packages
-        - packages: "debian-keyring ntp zip unzip rsync sudo"
-
-Steps path
-~~~~~~~~~~
-
-The steps are YAML formated files stored in the ``recipe/steps`` directory of
-the :ref:`workspace`. To enable a better recipe reuse and ease of write the steps
-are stored by default in specific folders depending on the sections.
-
-Kameleon is looking for the steps files using the ``include_steps`` list value,
-if it is set in the recipe (NOT mandatory). These includes are often the
-distribution steps. For example if you are building an ubuntu based
-distribution you can use:
-
-.. code-block:: yaml
-
-    include_steps:
-        - ubuntu
-        - debian/wheezy
-        - debian
-
-It also search uppermost within the current section folder. For the previous
-example, in the bootstrap section, the search paths are scanned in this
-order:
-
-.. code-block:: yaml
-
-    steps/bootstrap/ubuntu
-    steps/ubuntu
-    steps/bootstrap/debian/wheezy
-    steps/debian/wheezy
-    steps/bootstrap/debian
-    steps/debian
-    steps/bootstrap/
-    steps/
-
-
-Variables
----------
-
-Kameleon is using preprocessed variables. You can define it with the YAML
-key/value syntax ``my_var: my_value``.To access these variables you have to use
-the two dollars (``$$``) prefix.  Like in a Shell you can also use
-``$${var_name}`` to include your variables in string like this
-``my-$${variable_name}-templated``. It's also possible to use nested variables
-like:
-
-.. code-block:: yaml
-
-    my_var: foo
-    my_nested_var: $${my_var}-bar
-
-Be careful, in YAML you cannot mix dictionary and list on the same level.
-That's why, in the global dictionary, you can define your variables like in the
-example above but, in the recipe or the steps, you must prefix your variable
-with a ``-`` like this ``- my_var: foo``.
-
-
-Global variables
-~~~~~~~~~~~~~~~~~
-
-Global variables are defined in the ``global`` dictionary of the recipe.
-Kameleon use some global variable to enable the appliance build. See :ref:`context`
-and `Steps path`_ for more details
-
-
-Step local variables
-~~~~~~~~~~~~~~~~~~~~
-
-In the recipe, you can provide some variables when you call a step. This
-variable override the global and the default variables.
-
-
-Step default variables
-~~~~~~~~~~~~~~~~~~~~~~
-
-In the step file, you can define some default variables for your microsteps. Be
-careful, to avoid some mistakes, these variables can be override by the step
-local variables but not by the global ones. If this is the behavior you
-expected just add a step local variable that take the global variable value
-like this:
-
-.. code-block:: yaml
-
-    global:
-        foo: bar
-    setup:
-        - my_step:
-            - foo: $$foo

data/docs/source/tau_install.yaml

@@ -1,19 +0,0 @@
-- tau_version: "2.22.2"
-- pdt_version: "3.19"
-- get_tau:
-  - exec_in: cd /tmp/
-  - exec_in: wget  -q http://www.cs.uoregon.edu/research/tau/tau_releases/tau-$$tau_version.tar.gz
-  - exec_in: wget -q http://www.cs.uoregon.edu/research/tau/pdt_releases/pdt-$$pdt_version.tar.gz
-- pdt_install:
-  - exec_in: cd /tmp/
-  - exec_in: tar -xzf pdt-$$pdt_version.tar.gz
-  - exec_in: cd /tmp/pdtoolkit-$$pdt_version
-  - exec_in: ./configure -prefix=/usr/local/pdt-install
-  - exec_in: make clean install
-
-- tau_install:
-  - exec_in: cd /tmp/
-  - exec_in: tar -xzf tau-$$tau_version.tar.gz
-  - exec_in: cd /tmp/tau-$$tau_version
-  - exec_in: ./configure -prefix=/usr/local/tau-install -pdt=/usr/local/pdt-install/ -mpiinc=/usr/local/openmpi-install/include -mpilib=/usr/local/openmpi-install/lib
-  - exec_in: make install

data/docs/source/tau_install_g5k.yaml

@@ -1,25 +0,0 @@
-- tau_version: "2.22.2"
-- pdt_version: "3.19"
-- get_tau:
-  - exec_local: cd /tmp/
-  - exec_local: wget  -q http://www.cs.uoregon.edu/research/tau/tau_releases/tau-$$tau_version.tar.gz
-  - exec_local: wget -q http://www.cs.uoregon.edu/research/tau/pdt_releases/pdt-$$pdt_version.tar.gz
-  - local2in:
-      - /tmp/tau-$$tau_version.tar.gz
-      - /tmp/tau-$$tau_version.tar.gz
-  - local2in:
-      - /tmp/pdt-$$pdt_version.tar.gz
-      - /tmp/pdt-$$pdt_version.tar.gz
-- pdt_install:
-  - exec_in: cd /tmp/
-  - exec_in: tar -xzf pdt-$$pdt_version.tar.gz
-  - exec_in: cd /tmp/pdtoolkit-$$pdt_version
-  - exec_in: ./configure -prefix=/usr/local/pdt-install
-  - exec_in: make clean install
-
-- tau_install:
-  - exec_in: cd /tmp/
-  - exec_in: tar -xzf tau-$$tau_version.tar.gz
-  - exec_in: cd /tmp/tau-$$tau_version
-  - exec_in: ./configure -prefix=/usr/local/tau-install -pdt=/usr/local/pdt-install/ -mpiinc=/usr/local/openmpi-install/include -mpilib=/usr/local/openmpi-install/lib
-  - exec_in: make install

data/docs/source/use_cases.rst

@@ -1,93 +0,0 @@
-.. _`use cases`:
-
----------
-Use Cases
----------
-
-Here it is described different use cases for Kameleon. It should give you a better
-idea, through examples, of how Kameleon is useful for.
-
-Distribute an environnement to your co-workers/students/friends/...
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Whit Kameleon you can easily export your environnement in any format. The
-export section of the recipe is made for this. For example, if you would like
-to export your image in vdi format to use VirtualBox you just have to uncomment
-the right line. Like in the debian7 template::
-
-    #== Export the generated appliance in the format of your choice
-    export:
-      - save_appliance_from_nbd:
-        - mountdir: $$rootfs
-        - filename: "$${kameleon_recipe_name}"
-        - save_as_qcow2
-        # - save_as_qed
-        # - save_as_tgz
-        # - save_as_raw
-        # - save_as_vmdk
-        - save_as_vdi
-
-Make a Linux virtual machine with graphical support
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-You can directly use the templates that provide a desktop. However, here is an
-example of adding desktop capability to the ``debian8`` template. First create
-a new recipe from this template::
-
-     kameleon new debian8-dektop -t debian8
-
-Then edit the recipe file ``debian8-dektop.yaml`` and add ``gnome-core`` and ``xorg``
-packages to the install list::
-
-    setup:
-        # Install
-        - software_install:
-          - packages: >
-              debian-keyring ntp zip unzip rsync sudo less vim bash-completion
-              gnome-core xorg
-
-These packages take some extra space, so add some space on the disk. 4G should
-be enough::
-
-    bootstrap:
-        ...
-        - create_disk_nbd:
-            - image_size: 4G
-
-Build your recipe::
-
-    sudo kameleon build debian8-desktop
-
-When the build has finished, you can try you image with Qemu::
-
-    sudo qemu-system-x86_64 -m 1024 --enable-kvm -vga std \
-        builds/debian8-desktop/debian8-desktop.qcow2
-
-Alternatively, you could use ``virt-manager`` that provide a good GUI to manage
-your virtual machines.
-
-.. note::
-    If you want a better integration between the host and the guest like
-    copy/paste use ``spice`` (http://www.linux-kvm.org/page/SPICE)
-
-
-Create a fully reproducible experimental environement
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-You should use the :ref:`persistent_cache` feature.
-
-.. todo::
-    Give a complete use case.
-
-Create a persistent live USB key
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-A dirty but reliable method to do this is to cat the entire raw disk on the USB
-key. First be sure that the disk size is equal or smaller then your
-Then, export you image in raw format (this is the disk content bit by bit) and
-dump it to your USB key. Once your image is built, if your USB key is the
-``/dev/sdb`` device, be sure that it is not mounted and just do this::
-
-    cat my_image.raw > /dev/my_key
-
-.. warning::
-    This is a dangerous operation, you usb key will erase without
-    warning! Be sure that you pick the right device (use lsblk): it should be
-    ``/dev/sdX`` where X is a letter. Do NOT use the ``dev/sdXY``. unmount it
-    and use the root device instead.