power_stencil 0.3.1

data/doc/plugins.md

@@ -0,0 +1,169 @@
+Plugins
+=======
+
+<!-- TOC -->
+
+- [Why plugins ?](#why-plugins-)
+    - [Capitalize on your custom entity types](#capitalize-on-your-custom-entity-types)
+    - [Functional code growing](#functional-code-growing)
+    - [Need of very specific post actions developed in Ruby](#need-of-very-specific-post-actions-developed-in-ruby)
+    - [Enhance `power_stencil` command line](#enhance-power_stencil-command-line)
+- [What are plugins ?](#what-are-plugins-)
+- [Creating plugin local to the project](#creating-plugin-local-to-the-project)
+
+<!-- /TOC -->
+[:back:][Documentation root]
+
+# Why plugins ?
+
+If you read everything about [entities], [templates] and [builds], you may wonder why you may ever need to create plugins. There are nevertheless good reasons to build real plugins.
+
+## Capitalize on your custom entity types
+
+You have designed super nicely crafted entity types that you may want to **re-use** in other projects.
+
+## Functional code growing
+
+If you start developing a lot of code around your entity types, it may become a bad idea to keep everything within your entity types code and you may want to **structure** your code, add tests to it etc...
+
+## Need of very specific post actions developed in Ruby
+
+If calling an executable after the [build][builds] process completed is not enough and you want some custom Ruby to be run right after the [build][builds] completion.
+
+## Enhance `power_stencil` command line
+
+If you want to have new custom sub-commands or options available.
+
+
+# What are plugins ?
+
+Plugins are actually Ruby Gems with a specific structure. Plugins can be part of the project or provided as a separated stand-alone Gem.
+
+The normal process would be to begin with a plugin within the project and once you're ok with its features you may release it as a standalone Ruby Gem.
+
+- Plugins local to the project are automatically taken in account.
+- To use plugins provided as Gems you have to set the `:plugins:` array property in the `.ps_project/versioned-config.yaml`
+
+:warning: Plugins provided as standalone Gems are not thoroughly tested, for the time being you may use plugins local to the project.
+
+
+# Creating plugin local to the project
+
+The is a command provided for that, that will create a whole plugin skeleton.
+
+```shell
+$ power_stencil new-plugin myplugin
+Generated new plugin 'myplugin'
+
+$ ll .ps_project/plugins/myplugin
+total 60
+drwxrwxr-x 6 laurent laurent 4096 août  28 14:38 ./
+drwxrwxr-x 3 laurent laurent 4096 août  28 14:38 ../
+drwxrwxr-x 2 laurent laurent 4096 août  28 14:38 bin/
+-rw-rw-r-- 1 laurent laurent 3228 août  28 14:38 CODE_OF_CONDUCT.md
+drwxrwxr-x 3 laurent laurent 4096 août  28 14:38 etc/
+-rw-rw-r-- 1 laurent laurent  163 août  28 14:38 Gemfile
+-rw-rw-r-- 1 laurent laurent  113 août  28 14:38 .gitignore
+drwxrwxr-x 3 laurent laurent 4096 août  28 14:38 lib/
+-rw-rw-r-- 1 laurent laurent 1077 août  28 14:38 LICENSE.txt
+-rw-rw-r-- 1 laurent laurent 1491 août  28 14:38 psplugin_myplugin.gemspec
+-rw-rw-r-- 1 laurent laurent  117 août  28 14:38 Rakefile
+-rw-rw-r-- 1 laurent laurent 1770 août  28 14:38 README.md
+-rw-rw-r-- 1 laurent laurent   53 août  28 14:38 .rspec
+drwxrwxr-x 2 laurent laurent 4096 août  28 14:38 spec/
+-rw-rw-r-- 1 laurent laurent   88 août  28 14:38 .travis.yml
+```
+
+You can see that it looks a lot like a Ruby Gem, and it's normal because it is ;-)...
+
+So what was the impact on the project.
+
+If you do a `power_stencil info` you see a new part appeared in the report
+
+```
+--------------------------------------------------------------------------------
+ Plugins:
+ --> Plugin 'myplugin' has following capabilities:
+   - command_line: true
+   - local_config: true
+   - processors: true
+   - build: false
+   - dsl: false
+   - entity_definitions: true
+   - code: true
+   - version: true
+   - templates: true
+```
+Each of the lines correspond to what is called a _plugin capability_.
+
+Obviously by default the plugin does nothing, but it defined some placeholders. For example in the ouput here above it says `command_line: true`. What could it mean ?
+
+Let's try to see to help:
+
+```shell
+$ power_stencil --help
+This is power_stencil. A powerful templating engine.
+-- Options ---------------------------------------------------------------------
+    -v, --verbose  Displays extra runtime information.
+    -h, --help  Displays this help.
+    --program-version, -V, --version  Displays program version.
+    --simulate  Will not perform actual actions
+    --debug  Debug mode
+    --debug-on-err, --debug-on-stderr  Sends debugging to SDTERR
+    --log-level  Defines the level of logging (0 to 5)
+    --log-file  Specifies a file to log into
+    --truncate-log-file  Truncates the log file (appends by default)
+    --project-path  Specifies a startup path to use instead of '.'
+    --auto  Bypasses command-line confirmations to the user
+
+--------------------------------------------------------------------------------
+ Following subcommands exist too:
+ For more information you can always issue sub_command_name --help...
+--------------------------------------------------------------------------------
+ * init: Initializes a PowerStencil repository ...
+ * info: Generic information about the repository ...
+ * new-plugin: Generates the skeleton for a plugin ...
+ * get: Query entities from repository ...
+ * shell: Opens a shell to interact with entities ...
+ * check: Check repository entities consistency ...
+ * create: Creates entities in the repository ...
+ * edit: Edit entities from repository ...
+ * delete: Delete entities from repository ...
+ * build: Builds entities ...
+ * myplugin: Does nothing useful Added by plugin myplugin
+```
+
+You can see that the plugin brought a new sub-command `myplugin`.
+
+Let's try it:
+
+```shell
+$ power_stencil myplugin
+MYPLUGIN PLUGIN WAZ HERE
+```
+
+Wow it worked ! Useless, but worked !
+
+
+The documentation for plugins is not yet done, so you are encouraged to read the code, the capabilities (in the output of `power_stencil info`) give you an idea of what plugins can do.
+
+:warning: As opposed to the rest of `PowerStencil`, the functionnality is nevertheless not completely frozen.
+
+
+[:back:][Documentation root]
+<!-- End of Document -->
+
+<!-- Pages -->
+[Documentation root]: ../README.md "Back to documentation root"
+[entities]: entities.md "Entities in PowerStencil"
+[builds]: builds.md "Builds in PowerStencil"
+[templates]: templates.md "Templates in PowerStencil"
+
+
+<!-- Code links -->
+
+
+<!-- Illustrations -->
+
+
+<!-- External links -->

data/doc/templates.md

@@ -0,0 +1,333 @@
+Templates
+=========
+
+<!-- TOC -->
+
+- [Templates overview](#templates-overview)
+- [Templates discovery with `simple_exec` entity type](#templates-discovery-with-simple_exec-entity-type)
+- [What can I do within templates ?](#what-can-i-do-within-templates-)
+    - [ERB for newbies](#erb-for-newbies)
+    - [The `PowerStencil` DSL](#the-powerstencil-dsl)
+- [Where do I create templates ?](#where-do-i-create-templates-)
+- [Creating templates-templates of your own](#creating-templates-templates-of-your-own)
+    - [basics](#basics)
+    - [`.xxx_ignore` files](#xxx_ignore-files)
+    - [Application to our example](#application-to-our-example)
+
+<!-- /TOC -->
+[:back:][Documentation root]
+
+# Templates overview
+
+Templates are the key feature of `PowerStencil` (hence the name). Templates are what will use the [entities] you carefully crafted in order to produce something during the [build][builds] process.
+
+Currently the underlying engine is [ERB].
+
+When an entity type has some templates associated and you create an entity, it will create a directory `<entity_type>/<entity_name>` where you will fond those templates.
+
+# Templates discovery with `simple_exec` entity type
+
+In the basic entity types, the only one having templates is the `simple_exec` entity type. So let's try with it:
+
+```shell
+$ power_stencil create simple_exec/demo_templates 
+Created 'simple_exec/demo_templates'
+
+$ power_stencil check simple_exec/demo_templates 
+RAW ENTITIES
+ 'simple_exec/demo_templates':
+  - Storage path   : '/tmp/tst3/.ps_project/entities/simple_exec/demo_templates.yaml'
+  - Templates path : '/tmp/tst3/simple_exec/demo_templates'
+  - Status         : Valid 
+  - Buildable      : true
+```
+As you can see the output of `power_stencil check` shows more information than entities we already created. First we see that this entity is `buildable`, and we'll detail that in the [builds] document, but for the moment what we will focus on the `Templates path` information.
+
+And it's true if we have a look in the `simple_exec/demo_templates` from the root of the project, a file appeared there:
+
+```shell
+$ ls -l simple_exec/demo_templates
+total 4
+-rwxrwxr-x 1 laurent laurent 57 août  23 16:44 main.sh
+
+$ cat simple_exec/demo_templates/main.sh 
+#!/usr/bin/env bash
+
+echo "Nothing to do... Easy task..."
+```
+
+Humm... This stuff doesn't look like a template...looks like a regular bash script. It's true and we'll come back to the internals of `simple_exec` in the [builds], where does this script come from etc... So don't focus too much on this file for the moment.
+
+We will just first experiment our first build !
+
+```shell
+$ power_stencil build simple_exec/demo_templates -v
+De-templating files for 'simple_exec/demo_templates' into '/tmp/tst3/build/20190823-1709-54271-simple_exec_demo_templates/simple_exec_demo_templates'
+Entities analysis completed.
+Running '/tmp/tst3/build/20190823-1709-54271-simple_exec_demo_templates/simple_exec_demo_templates/main.sh'
+Nothing to do... Easy task...
+End of process 15572
+ - 'simple_exec/demo_templates' has been correctly built
+```
+
+What can we see there ? 
+
+The first line is the most interesting one for us. It says "De-templating ... into ...", and it's true that a brand new directory named `build` appeared at the root of the project. Again we won't focus too much on the name of the sub-directory as we will see that in the [builds], but if we have a look in the generated directory we see the same content as what we had in `simple_exec/demo_templates`... 
+And by the way the generated script seems to have been run, as we can see in the output.
+
+:warning: If bash is not installed on your system, the output will obviously be different as `PowerStencil` won't be able to run the script, yet all the files should have been generated correctly.
+
+Ok is it possible, that everything inside `simple_exec/demo_templates` is actually considered as templates ? **The answer is yes !!**
+
+Let prove it and replace the content of `simple_exec/demo_templates/main.sh` by the following:
+
+```shell
+#!/usr/bin/env bash
+
+echo "This script has been generated on <%= Time.now %>"
+
+echo "It was done in the scope of '<%= build_target.name %>' (which is of the type '<%= build_target.type %>')..."
+```
+Ah ok... now it looks like an ERB template !
+
+Let's rebuild...
+
+```shell
+$ power_stencil build simple_exec/demo_templates -v
+De-templating files for 'simple_exec/demo_templates' into '/tmp/tst3/build/20190823-1733-08665-simple_exec_demo_templates/simple_exec_demo_templates'
+Entities analysis completed.
+Running '/tmp/tst3/build/20190823-1733-08665-simple_exec_demo_templates/simple_exec_demo_templates/main.sh'
+This script has been generated on 2019-08-23 17:33:08 +0200
+It was done in the scope of 'demo_templates' (which is of the type 'simple_exec')...
+End of process 415
+ - 'simple_exec/demo_templates' has been correctly built
+```
+
+And it's true, if I have a look at the generated script (replace with your own path):
+
+```shell
+$ cat /tmp/tst3/build/20190823-1733-08665-simple_exec_demo_templates/simple_exec_demo_templates/main.sh
+#!/usr/bin/env bash
+
+echo "This script has been generated on 2019-08-23 17:33:08 +0200"
+
+echo "It was done in the scope of 'demo_templates' (which is of the type 'simple_exec')..."
+```
+
+The content has been "de-templated"... :+1: sooo cool !
+
+:information_source: Here the template directory of the `simple_exec/demo_templates` was containing only on file (the bash script), but **any file in any sub-directory of that directory would have been considered as a template** (yet there are some mechanism to control and limit which files have to be processed, like for example imagine there would be images, or zip files, probably you don't want them to be processed. All of this will be covered in the [builds] document).
+
+# What can I do within templates ?
+
+## ERB for newbies
+
+So you noticed the [ERB] syntax. If you already developed within a [Ruby on Rails] project you are probably used to it. Basically [ERB] introduces two main types of tags:
+
+- `<% %>` between those tags you can put **any valid [Ruby] code**.
+- `<%= %>` what is returned by the enclosed [Ruby] code will be inserted _here_ in the resulting document.
+
+There are some other minor tags like comments, or insertion without carriage return... You are encouraged to have a look at [ERB] if you don't know it, but it's fairly simple like most templating engine are.
+
+## The `PowerStencil` DSL
+
+On top of any Ruby legit code you can use special methods provided by `PowerStencil` to ease the access to entities, configuration etc...
+
+Here are the main methods:
+
+- `entity(type, name)` will return any entity from the repository. The method is self-explanatory, this is the most generic way to access any entity of the repository.
+- `entities(criterion: nil, value: nil, &filter_block)` which is a more generic entities query method where `criterion` can be `:by_name` or `:by_type` and that will return an array (that you can potentially filter using the `filter_block`).
+- `build_target` will return the entity that you are currently building.
+- `project_config` is a shortcut to the project entity. You could actually retrieve it using the `entity(type, name)` method. 
+
+And of course you have access like in `power_stencil shell` to any class brought by `PowerStencil`, you could create entities there, destroy some, **but you should not do it**, as it would be very weird to do that during the process of detemplating !
+
+# Where do I create templates ?
+
+**When an entity is [buildable] you can find its templates in the `<entity_type>/<entity_name>` from the root of the project.** 
+
+But by default, that directory is not created, and if no templates is created at this place for a [buildable] entity, the build will fail saying it didn't find any template...
+
+Yet, you probably noticed that when we created an entity of `simple_exec` type, a file (the bash script `main.sh`) _automagically_ appeared in the right place so that we could immediatelly build it. How can you achieve that with entity types of your own ? **This is what is called _templates-templates_**...
+
+First let's have a look at the output of `power_stencil info`, focusing on `simple_exec`: 
+
+```shell
+$ power_stencil info
+    .
+    .
+    .
+ - Type 'simple_exec' --> PowerStencil::SystemEntityDefinitions::SimpleExec (template-template path: '/opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/power_stencil-0.2.18/etc/templates/simple_exec')
+```
+
+`simple_exec` is an entity type coming with `PowerStencil`, and you can see that it takes its _template-template_ from within the `PowerStencil` gem itself !
+
+But within the `PowerStencil` project, there is a place to create _templates-templates_ of your own: `.ps_projet/templates-templates/<entity_type>`.
+
+To summarize about templates and _templates-templates_:
+
+![entity-creation-flow]
+
+:information_source: There is actually a third place where you could find entity types and _templates-templates_, this is within [plugins], but we will cover that in the [plugins] part...
+
+# Creating templates-templates of your own
+
+## basics
+
+Let's imagine we have a custom buildable entity type defined in `.ps_project/entity_definitions/custom_buildable_entity.rb` and which content is:
+
+```ruby
+class CustomBuildableEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_buildable_entity
+  buildable
+end
+```
+We could have (useless) _templates-templates_ defined in `.ps_project/templates-templates/custom_buildable_entity`. 
+
+:information_source: A whole tree structure can be defined there, not only one file.
+
+Let's create there a text file named `a_useless_text_file.txt` with the following content:
+
+```
+This text has been generated on the <%= Time.now %>
+
+Lorem ipsum dolor sit amet, consectetur adipiscing elit,
+sed do eiusmod tempor incididunt ....
+```
+
+Let's now create an entity of this type:
+
+```shell
+$ power_stencil create custom_buildable_entity/test1
+Created 'custom_buildable_entity/test1'
+
+$ ll custom_buildable_entity/test1 
+total 12
+drwxrwxr-x 2 laurent laurent 4096 août  26 13:40 ./
+drwxrwxr-x 3 laurent laurent 4096 août  26 13:40 ../
+-rw-rw-r-- 1 laurent laurent   26 août  26 13:40 a_useless_text_file.txt
+```
+Everything, seems ok. The entity `custom_buildable_entity/test1` has been created and there is now a new folder at the root of the project named `custom_buildable_entity/test1` and which content is a text file named `a_useless_text_file.txt`... So far so good.
+
+And what is the content of this file ?
+
+```shell
+$ cat custom_buildable_entity/test1/a_useless_text_file.txt
+This text has been generated on the 2019-08-26 13:50:35 +0200
+
+Lorem ipsum dolor sit amet, consectetur adipiscing elit,
+sed do eiusmod tempor incididunt ....
+```
+
+**Oops !!** It looks like the _template-template_ content has been processed as templates and the `<%= Time.now %>` code has been replaced by its result. This is not **at all** what we could have expected !! The idea was that the entity type template (the _template-template_) would become the entity template, ready for the build...
+
+**:warning: In `PowerStencil`, the concept is that everywhere there is a template defined (and _templates-templates_ are), they are really treated as templates unless you explicitely say the opposite !**
+
+So, how can we avoid that, so that the _templates-templates_ are not processed as templates ?
+
+A bad answer would be to have [ERB] code to generate [ERB] tags and code... :scream: (although sometimes in very specific cases you may really want to do that).
+
+No in this case we would like the _template-template_ to be simply not processed as a template. There  is a solution for that _.xxx_ignore files_...
+
+## `.xxx_ignore` files
+
+When any file is processed using the `PowerStencil` framework to generate another file, there are 3 operations/checks that are always applied:
+
+1. Is the file eligible to the process (ie should it be completely ignored by the process) ?
+2. Is the file eligible to the templating process ?
+3. Should the file be renamed ?
+
+This is managed through a set of files that work the same way `.git_ignore` files work for Git. You can have those files anywhere within the tree of files that are processed.
+
+Their syntax is very simple. Each line can either be:
+
+- an empty line
+- a comment line starting with `#`
+- a line specifying a [Dir::glob pattern]
+
+As you probably now guessed, there are three different `.xxx_ignore` types of files available:
+
+- the `.copy_ignore` file will define which files should be completely ignored by the process. A good example would be for example a README file describing what is in the directory, but that you don't want to copy. Those files are simply ignored.
+- the `.subst_ignore` file will define which files should be copied as-is and not processed as templates.
+- later in the process files could be renamed (for example, by default `.erb` files see their `.erb` extension removed, but sometimes you may not want this to happen). The `.rename_ignore` file will define which files should not be renamed.
+
+This feature is actually coming from the [dir_glob_ignore] Gem.
+
+## Application to our example
+
+Let's come back to [our example](#basics), and create the relevant `.xxx_ignore` files.
+
+First we don't want the `.ps_project/templates-templates/custom_buildable_entity/a_useless_text_file.txt` to be considered as a template. So let's create the `.ps_project/templates-templates/custom_buildable_entity/.subst_ignore` file with the following content:
+
+```ignore
+# Ignore everything in this directory and sub folders
+**
+```
+
+But if we let things like that, the `.subst_ignore` would not be processed by the templating engine (because it matches `**` from a `Dir::glob` point of view), but would be copied. And we don't want that !
+So let's create a `.ps_project/templates-templates/custom_buildable_entity/.copy_ignore` with the following content:
+
+```ignore
+# This files should not be processed at all
+.subst_ignore
+.copy_ignore
+```
+
+Et voilà, let's verify:
+
+```shell
+$ power_stencil create custom_buildable_entity/test2
+Created 'custom_buildable_entity/test2'
+
+$ ls -la custom_buildable_entity/test2 
+total 12
+drwxrwxr-x 2 laurent laurent 4096 août  26 18:37 .
+drwxrwxr-x 4 laurent laurent 4096 août  26 18:37 ..
+-rw-rw-r-- 1 laurent laurent  148 août  26 18:37 a_useless_text_file.txt
+
+$ cat custom_buildable_entity/test2/a_useless_text_file.txt 
+This text has been generated on the <%= Time.now %>
+
+Lorem ipsum dolor sit amet, consectetur adipiscing elit,
+sed do eiusmod tempor incididunt ....
+```
+:+1: Success ! None of the `.xxx_ignore` files are in the resulting folder, and the `custom_buildable_entity/test2/a_useless_text_file.txt` has not been processed, ready for the [build process][builds]. We can verify this too:
+
+```shell
+$ power_stencil build custom_buildable_entity/test2 -v
+De-templating files for 'custom_buildable_entity/test2' into '/tmp/tst3/build/20190826-1842-27501-custom_buildable_entity_test2/custom_buildable_entity_test2'
+Entities analysis completed.
+ - 'custom_buildable_entity/test2' has been correctly built
+
+$ cat /tmp/tst3/build/20190826-1842-27501-custom_buildable_entity_test2/custom_buildable_entity_test2/a_useless_text_file.txt 
+This text has been generated on the 2019-08-26 18:42:27 +0200
+
+Lorem ipsum dolor sit amet, consectetur adipiscing elit,
+sed do eiusmod tempor incididunt ....
+
+```
+Flawless victory !!
+
+[:back:][Documentation root]
+<!-- End of Document -->
+
+<!-- Pages -->
+[Documentation root]: ../README.md "Back to documentation root"
+[entities]: entities.md "Entities in PowerStencil"
+[builds]: builds.md "Builds in PowerStencil"
+[plugins]: plugins.md "Plugins in PowerStencil"
+[example use cases]: example_use_cases.md "Example uses cases using PowerStencil"
+[buildable]: entities.md#buildable--buildable_by "How to make an entity buildable ?"
+
+<!-- Code links -->
+
+
+<!-- Illustrations -->
+[entity-creation-flow]: images/power-stencil-entity-creation.svg
+
+<!-- External links -->
+[ERB]: https://ruby-doc.org/stdlib-2.6.3/libdoc/erb/rdoc/ERB.html "Quick ERB description"
+[Ruby On Rails]: https://rubyonrails.org/ "One of the best Web framework"
+[Ruby]: https://www.ruby-lang.org "The powerful Ruby language"
+[Dir::glob pattern]: https://ruby-doc.org/core-2.6.3/Dir.html#method-c-glob "Ruby documentation"
+[dir_glob_ignore]: https://gitlab.com/lbriais/dir_glob_ignore "An extension to the Ruby `Dir::glob` feature"