power_stencil 0.3.0

data/doc/entities.md

@@ -0,0 +1,964 @@
+Entities
+========
+
+<!-- TOC -->
+
+- [What are entities ?](#what-are-entities-)
+    - [Default entity types](#default-entity-types)
+    - [Default entities](#default-entities)
+    - [Structure overview](#structure-overview)
+    - [Loose schema entities](#loose-schema-entities)
+- [Manipulating entities](#manipulating-entities)
+    - [Checking entities](#checking-entities)
+    - [Querying entities](#querying-entities)
+    - [Creating entities](#creating-entities)
+        - [Versioned entities](#versioned-entities)
+        - [Local (unversioned) entities](#local-unversioned-entities)
+    - [Updating entities](#updating-entities)
+    - [Deleting entities](#deleting-entities)
+    - [The `PowerStencil` shell](#the-powerstencil-shell)
+- [Creating custom entity types](#creating-custom-entity-types)
+    - [Anatomy of an entity type](#anatomy-of-an-entity-type)
+    - [Where do I create new entity types ?](#where-do-i-create-new-entity-types-)
+    - [My first custom entity](#my-first-custom-entity)
+    - [Directives](#directives)
+        - [entity_type](#entity_type)
+        - [auto_named_entity_type](#auto_named_entity_type)
+        - [field](#field)
+        - [has_one](#has_one)
+        - [has_many](#has_many)
+        - [is_array](#is_array)
+        - [is_hash](#is_hash)
+        - [not_null](#not_null)
+        - [not_empty](#not_empty)
+        - [should_match](#should_match)
+        - [class_name](#class_name)
+        - [buildable / buildable_by](#buildable--buildable_by)
+    - [Module you could include in your entity types](#module-you-could-include-in-your-entity-types)
+    - [Adding functional code](#adding-functional-code)
+
+<!-- /TOC -->
+[:back:][Documentation root]
+
+# What are entities ?
+
+In a nutshell, `entities` are [Ruby] objects persisted into [YAML] files, but unless you have the need the create your own entity definitions (entity types), you don't really need to know the internals as you can fully manage entities without even knowing anything about code.
+
+There is a lot to read there but you may find surprisingly easy how it is to master entities...
+
+## Default entity types
+
+The `power_stencil` CLI provides a number of options to interact with entities. But first let's have a look at default entities and entities definitions. The `power_stencil info` is the perfect command for that:
+
+```shell
+$ power_stencil info
+--------------------------------------------------------------------------------
+PROJECT REPORT
+--------------------------------------------------------------------------------
+ General information:
+ - Project required version: 0.2.18
+ - PowerStencil version: 0.2.18
+--------------------------------------------------------------------------------
+ Paths:
+ - Project root path: '/tmp/tst2'
+ - Project configuration path: '/tmp/tst2/.ps_project'
+ - Local user configuration file: '/tmp/tst2/.ps_project/personal-config.yaml'
+ - Project (versioned) configuration file: '/tmp/tst2/.ps_project/versioned-config.yaml'
+ - Project entities: '/tmp/tst2/.ps_project/entities'
+ - User (unversioned) entities: '/tmp/tst2/.ps_project/user_entities'
+ - Project specific entity definitions: '/tmp/tst2/.ps_project/entity_definitions'
+--------------------------------------------------------------------------------
+ Entities:
+ - Contains 1 entities.
+ -  project_config: 1
+--------------------------------------------------------------------------------
+ Available entity types:
+ - Type 'base_entity' --> PowerStencil::SystemEntityDefinitions::ProjectEntity
+ - Type 'entity_override' --> UniverseCompiler::Entity::Override
+ - Type 'plugin_definition' --> PowerStencil::SystemEntityDefinitions::Plugin (template-template path: '/opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/power_stencil-0.2.18/etc/templates/plugin_definition')
+ - Type 'process_descriptor' --> PowerStencil::SystemEntityDefinitions::ProcessDescriptor
+ - Type 'project_config' --> PowerStencil::SystemEntityDefinitions::ProjectConfig
+ - Type 'simple_exec' --> PowerStencil::SystemEntityDefinitions::SimpleExec (template-template path: '/opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/power_stencil-0.2.18/etc/templates/simple_exec')
+```
+
+It provides a lot of information about the project, but let's focus on the `Entities` and `Available entity types` parts of the report.
+
+You can see here above that only 6 types of entity are defined by default in a brand new created `PowerStencil` project. You can find them below with some of their core features:
+
+| entity type | system | persisted | buildable | has associated files |
+|-------------|:------:|:---------:|:---------:|:--------------------:|
+|base_entity||X
+|entity_override||X
+|plugin_definition|X
+|process_descriptor||X
+|project_config|X
+|simple_exec||X|X|X
+
+- You should not care about _system_ entities unless you are developing features in `PowerStencil`.
+- By default any entity type, including those you may create, is _persistent_.
+- A _buildable_ entity is an entity you can run `power_stencil build` against. More about this in the [builds] part.
+- When an entity _has associated files_, it means its [templates] are in `<project_root>/<entity_type>/<entity_name>` folder. We'll talk about this more in depth in the [templates] and [builds] part.
+
+On top of this `power_stencil info` brings you information about the actual Ruby classes implementing those entity types.
+
+For a typical project which goal is only to generate some files from templates, the only entity types you may need are:
+
+- base_entity
+- entity_override
+- simple_exec
+
+More information about this in [builds].
+
+
+## Default entities
+
+From the `power_stencil info` command output another important information, is that it says that the project contains only one entity, and that this entity is of the type `project_config`.
+
+`project_config` is a special entity type containing the full config of the project as seen by `PowerStencil`, we will here after how to query it (like any other entity). But there are 2 very important things to note about this special entity:
+
+- First, this singleton entity will be available at [build time][builds] from any [template][templates].
+- You can add your own properties in this entity by adding properties to either the project (versioned) configuration file, or to the local user configuration file (see in the `Paths` part of the `power_stencil info` output).
+
+
+## Structure overview
+
+As said in introduction, `entities` are persisted as [YAML] files, and therefore only a light knowledge of the [YAML] format is required.
+
+The type of an entity is actually not part of the entity YAML properties, but it is determined when you [create a new entity](#creating-entities), and it appears at the top of the generated YAML file as a reference to the Ruby class representing the entity type (the ones you can see in the output of `power_stencil info`).
+Then the YAML file should at least contain the `:name` property.
+So for a `basic_entity` which is the most simple entity type, the minimalistic generated YAML file may look like:
+
+```yaml
+--- !ruby/object:PowerStencil::SystemEntityDefinitions::ProjectEntity
+:name: my_1st_entity
+```
+
+The `:name` property may contain any string including space or weird characters. It just has to be a valid YAML string...
+
+:warning: You may have noticed the leading `:` in the YAML file for the name property. We will come back to this in [Creating custom entity types](#creating-custom-entity-types), but a good pratice would be to prefix all your properties with this colon `:` character. Although it is not always mandatory it will ease the readability of your [templates]. Yet it is mandatory for the name property.
+
+:information_source: Each entity has an optional `:description` property.
+
+**:information_source: Each entity has a unique ID which is of the form: `<entity_type>/<entity_name>`**
+
+:warning: Meaning that **for a particular type of entity, its name has to be unique** !
+
+Any valid YAML is actually a valid entity !
+
+## Loose schema entities
+
+An interesting feature of `PowerStencil`, is that we could describe it as _loose-schema enforced_. What does it mean ? Actually entity types can define some of their properties as mandatory, and the rest is completely free !
+For example the `basic_entity` has only `:name` declared mandatory property, and for the rest you do whatever you want.
+
+So as a result the following YAML is a perfectly valid `basic_entity`:
+
+```yaml
+--- !ruby/object:PowerStencil::SystemEntityDefinitions::ProjectEntity
+:name: my_2nd_entity
+:an_array_property:
+    - item 1
+    - item 2
+:a_hash_property:
+    item1: value1
+    item2: value2
+a_non_colon_prefixed_property: blah blah blah
+```
+You will be able to access all those properties in your [templates]. Even those not known by the entity type. Only the way you will access them may differ a bit. We will cover that more in detail when we will create [custom entity types](#creating-custom-entity-types).
+
+
+# Manipulating entities
+
+## Checking entities
+
+We have seen the `power_stencil info` returns (among other) the number of entities present in the repository (grouped by entity type). But we can have a bit more information about each entity using the `power_stencil check` command:
+
+```shell
+$ power_stencil check                                  
+RAW ENTITIES
+ 'base_entity/my_1st_entity':
+  - Storage path: '/tmp/tst2/.ps_project/entities/base_entity/my_1st_entity.yaml'
+  - Status      : Valid 
+  - Buildable   : false
+ 'base_entity/my_2nd_entity':
+  - Storage path: '/tmp/tst2/.ps_project/entities/base_entity/my_2nd_entity.yaml'
+  - Status      : Valid 
+  - Buildable   : false
+ 'project_config/Project Config':
+  - Storage path: ''
+  - Status      : Valid 
+  - Buildable   : false
+```
+
+The goal of this command is normally to check the validity of entities, but it brings as well extra information like if the entity is _buildable_ or even where it is stored in the repository.
+
+:information_source: You can notice that the `project_config` entity has no storage path, the reason being [it is not persisted](#default-entity-types).
+
+This command may show some extra information, for more complex entities. We will see that later on.
+
+By default `power_stencil check` will check all entities in the repository but you can specify on the command line an arbitrary list of entities:
+
+```shell
+$ power_stencil check base_entity/my_1st_entity 'project_config/Project Config'
+RAW ENTITIES
+ 'base_entity/my_1st_entity':
+  - Storage path: '/tmp/tst2/.ps_project/entities/base_entity/my_1st_entity.yaml'
+  - Status      : Valid 
+  - Buildable   : false
+ 'project_config/Project Config':
+  - Storage path: ''
+  - Status      : Valid 
+  - Buildable   : false
+```
+Another possibility is to check a list of objects using a regular expression by using the `--regexp` option:
+
+```shell
+$ power_stencil check base_ --regexp
+RAW ENTITIES
+ 'base_entity/my_1st_entity':
+  - Storage path: '/tmp/tst2/.ps_project/entities/base_entity/my_1st_entity.yaml'
+  - Status      : Valid 
+  - Buildable   : false
+ 'base_entity/my_2nd_entity':
+  - Storage path: '/tmp/tst2/.ps_project/entities/base_entity/my_2nd_entity.yaml'
+  - Status      : Valid 
+  - Buildable   : false
+```
+
+## Querying entities
+
+The command to query entities from the repository is `power_stencil get`, and it works basically like the `power_stencil check` command.
+
+No parameter will return all entities, you can pass multiple ids or regexp on the command line.
+
+By default it will return entities in a format closer to their Ruby counterpart:
+
+```shell
+$ power_stencil get base_entity/my_2nd_entity      
+---
+PowerStencil::SystemEntityDefinitions::ProjectEntity:
+  :type: :base_entity
+  :fields:
+    :name: my_2nd_entity
+    :an_array_property:
+    - item 1
+    - item 2
+    :a_hash_property:
+      item1: value1
+      item2: value2
+    a_non_colon_prefixed_property: blah blah blah
+```
+We will see later why this format.
+
+But you can get the exact YAML version as well using the `--raw` option:
+
+```shell
+$ power_stencil get base_entity/my_2nd_entity --raw
+--- !ruby/object:PowerStencil::SystemEntityDefinitions::ProjectEntity
+:name: my_2nd_entity
+:an_array_property:
+- item 1
+- item 2
+:a_hash_property:
+  item1: value1
+  item2: value2
+a_non_colon_prefixed_property: blah blah blah
+```
+
+You can also get the names only, which is basically only interesting if you query using a regexp:
+
+```shell
+$ power_stencil get base_entity --names-only --regexp
+- my_2nd_entity
+
+- my_1st_entity
+```
+There are 2 other options (`--compiled` and `--scenario`) but they will be covered in the scope of [builds].
+
+## Creating entities
+
+### Versioned entities
+
+You can create new entities in the repository using the `power_stencil create` command.
+
+```
+$ power_stencil create base_entity/my_3rd_entity
+Created 'base_entity/my_3rd_entity'
+```
+
+You can also immediately edit the generated entity using the `--edit` option, which is particularly useful if the entity to created requires some mandatory properties. `PowerStencil` will not let you create invalid entities, so you may have to fill in all mandatory properties before saving.
+
+The editor used can be specified by multiple ways:
+
+- In the `EDITOR` environment variable
+- In one of the two project config files using the `:editor` property
+- Directly on the command line using the `--editor` option
+
+After you saved the entity and exited the editor, if the entity you saved is invalid, `PowerStencil` will reopen the file to give you a chance to fix the issue.
+
+By default the number of retries is specified in the config by the `:max_file_edit_retry_times` property. You can check that in the 
+`project_config/Project Config` entity by issuing:
+
+    $ power_stencil get 'project_config/Project Config'
+
+And check the `:max_file_edit_retry_times` property in the output. The value of this property can be overridden in one of the two project config files (in this case typically the one that is not versioned).
+
+Alternatively, instead of using an editor, you can directly specify a property value directly on the command line using the `--property` option.
+
+```shell
+$ power_stencil create base_entity/my_4th_entity --property extra_prop:foo
+Created 'base_entity/my_4th_entity'
+
+$ power_stencil get base_entity/my_4th_entity --raw
+--- !ruby/object:PowerStencil::SystemEntityDefinitions::ProjectEntity
+:name: my_4th_entity
+:extra_prop: foo
+```
+:warning: Be careful, that any property created this way will be prefixed using `:`
+
+### Local (unversioned) entities
+
+Entities created there are stored in `.ps_project/entities` which is a directory where everything is versioned by default. **But you have the possibility to create entities that will not be versioned**. 
+This is very **useful for developers** for a specific test for example, without risking to commit something they don't want.
+
+For that you just need to use the `--user-storage` option. Then the entity will be persisted in the `.ps_project/user_entities` which is not versioned (see the root `.gitignore` file).
+
+```shell
+$ power_stencil create base_entity/dev_entity --user-storage
+Created 'base_entity/dev_entity'
+
+$ power_stencil check base_entity/dev_entity
+RAW ENTITIES
+ 'base_entity/dev_entity':
+  - Storage path: '/tmp/tst2/.ps_project/user_entities/base_entity/dev_entity.yaml'
+  - Status      : Valid 
+  - Buildable   : false
+```
+You can notice where the YAML file has been created.
+
+Of course you can then delete the entity like any other...
+```shell
+$ power_stencil delete base_entity/dev_entity --auto
+Deleted 'base_entity/dev_entity'
+
+```
+
+
+## Updating entities
+
+Not so much to say about updating entities. It works exactly the same way as the `power_stencil create`.
+
+The command for that is `power_stencil edit`.
+
+## Deleting entities
+
+To delete an entity I guess you get it you have to use the `power_stencil delete` command.
+
+You can delete multiple object in one command using `--regexp`, but `PowerStencil` will ask confirmation for any of the entity requested:
+
+```shell
+$ power_stencil delete base_entity/my_4th_entity                    
+Are you sure you want delete 'base_entity/my_4th_entity' ? (Yes/y/[No]/n):  
+Cancelled by user input.
+$ power_stencil delete base_entity/my_4th_entity 
+Are you sure you want delete 'base_entity/my_4th_entity' ? (Yes/y/[No]/n): y
+Deleted 'base_entity/my_4th_entity'
+```
+
+:warning: You can also pass the `--auto` option that will bypass the user confirmation. **Use with caution!**
+
+## The `PowerStencil` shell
+
+`PowerStencil` provides a full Ruby shell (REPL) based on [Pry].
+It could deserve a whole documentation in its own, because you could do a lot of things there. 
+Basically it is interesting, because you can automate easily the creation of a bunch of entities... But you can actually do anything as it is a fully fledged Ruby console.
+
+Here under just an example of a short session:
+
+```ruby
+$ power_stencil shell
+-------------------------------------------------------------------------------
+# Welcome to the PowerStencil shell session
+# In this shell you have access to anything the templating engine has access to.
+# You can view, edit, save entities.
+#
+# - Retrieve and manipulate entities using the `entities` hash.
+# - Persist your changes using the `save` method on each entity.
+# - Create new project or user entities using `new_<type>` and `user_new_<type>`
+#   methods (see `available_entity_types` for a list of possible types).
+# - And of course, it is a fully fledged Ruby Pry REPL, so you can do anything
+#   you want...
+#
+# Type `exit` to end your session.
+-------------------------------------------------------------------------------
+PowerStencil DSL> entities
+=> [#<PowerStencil::SystemEntityDefinitions::ProjectConfig:47381736532780 composite_key=[:project_config, "Project Config"], @universe='Project entities (1566398755.703200)'>,
+  #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47381735892000 composite_key=[:base_entity, "my_2nd_entity"], @universe='Project entities (1566398755.703200)'>,
+ #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47381735871600 composite_key=[:base_entity, "my_3rd_entity"], @universe='Project entities (1566398755.703200)'>,
+ #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47381735869240 composite_key=[:base_entity, "my_1st_entity"], @universe='Project entities (1566398755.703200)'>]
+PowerStencil DSL> %w(entity1 entity2 entity3).each do |entity_name|
+PowerStencil DSL -->   e = new_base_entity name: entity_name  
+PowerStencil DSL -->   e.save  
+PowerStencil DSL --> end  
+=> ["entity1", "entity2", "entity3"]
+PowerStencil DSL> entities
+=> [#<PowerStencil::SystemEntityDefinitions::ProjectConfig:47381736532780 composite_key=[:project_config, "Project Config"], @universe='Project entities (1566398755.703200)'>
+ #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47381735892000 composite_key=[:base_entity, "my_2nd_entity"], @universe='Project entities (1566398755.703200)'>,
+ #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47381735871600 composite_key=[:base_entity, "my_3rd_entity"], @universe='Project entities (1566398755.703200)'>,
+ #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47381735869240 composite_key=[:base_entity, "my_1st_entity"], @universe='Project entities (1566398755.703200)'>,
+ #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47381741017600 composite_key=[:base_entity, "entity1"], @universe='Project entities (1566398755.703200)'>,
+ #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47381740982060 composite_key=[:base_entity, "entity2"], @universe='Project entities (1566398755.703200)'>,
+ #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47381740930300 composite_key=[:base_entity, "entity3"], @universe='Project entities (1566398755.703200)'>]
+PowerStencil DSL> exit
+```
+And of course after this session the entities are really saved:
+
+```shell
+$ power_stencil get base_entity/ent  --regexp --raw
+--- !ruby/object:PowerStencil::SystemEntityDefinitions::ProjectEntity
+:name: entity3
+
+--- !ruby/object:PowerStencil::SystemEntityDefinitions::ProjectEntity
+:name: entity1
+
+--- !ruby/object:PowerStencil::SystemEntityDefinitions::ProjectEntity
+:name: entity2
+```
+
+Basically, in this shell you can do anything you could do within [templates]. Use it to experiment with `PowerStencil`. It is important to understand that **you can't mess up anything within the shell** (Ok not completely true, as it is a [Pry] console under the hood, you could do everything you could do using the [Ruby] language, including wiping your hard drive, creating botnets or skynet terminators... but let's remain serious, I can't imagine you would do such bad things without knowing what you do ;)), if you don't save your entities, nothing is actually persisted. And if you do, who cares ? Git, your wingman, is there to back you up...
+
+:sparkles: Use the shell Luke !! :sparkles:
+
+
+# Creating custom entity types
+
+
+:warning: As opposed to everything we've done until now, creating custom entity types requires a bit of knowledge of the [Ruby] programming language.
+
+## Anatomy of an entity type
+
+An entity type is actually a Ruby class inheriting from [PowerStencil::SystemEntityDefinitions::ProjectEntity]. Simply inherit from this class and you're good to go with a brand new valid entity type.
+
+Internally an entity is a proxy to a Hash `#fields` which content is exactly the content of the YAML file where the entity is persisted. For example in the `PowerStencil` shell:
+
+```ruby
+PowerStencil DSL> e = entity :base_entity, "my_2nd_entity"
+=> #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47113681164160 composite_key=[:base_entity, "my_2nd_entity"], @universe='Project entities (1566475157.369747)'>
+PowerStencil DSL> e.fields
+=> {:name=>"my_2nd_entity", :an_array_property=>["item 1", "item 2"], :a_hash_property=>{"item1"=>"value1", "item2"=>"value2"}, "a_non_colon_prefixed_property"=>"blah blah blah"}
+PowerStencil DSL> e.fields.to_yaml
+```
+We'll come back later on the DSL included but `entity(type, name)` is a method to retrieve an entity from the repository. So here we retrieve `my_2nd_entity` and we see that the content of the `#fields` is exactly what we defined.
+This is as well a way to see that properties defined with a `:` prefix end up being `Symbols` whereas non-prefixed properties become `Strings`.
+
+The [PowerStencil::SystemEntityDefinitions::ProjectEntity] class extends itself a class from the [universe_compiler] Gem and most of the complexity is hidden there in terms of fields, persistence, relationship between entities, builds, _inheritance_ (we are not talking about class inheritance there), overrides...
+We will clarify some of those concepts here and some other in the [templates] and [builds] part, but you have to know that you don't really need to dig inside the [universe_compiler] unless you are really interested in the deepest internals...
+
+:+1: Basically the main thing you need to understand is that the custom entity types you will create will inherit from that class.
+
+## Where do I create new entity types ?
+
+There mainly two places where you can create custom entity types.
+
+1. Within a plugin. But plugins are the highest level of customization, and are [covered in a separated document][plugins]. But regarding what we will say here the definition of the classes will follow the same rules.
+2. In the `.ps_project/entity_definitions/` project folder. Ruby files that will be created there will be required in their alphabetical order. You can name them how you want (provided the `.rb` extension), as their name don't need to match the name of the classes you will create.
+
+## My first custom entity
+
+Let's create a new file in `.ps_project/entity_definitions/` and let's name it `my_first_custom_entity_type.rb`, then fill it with the following content:
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+    entity_type :custom_entity
+end
+```
+That's all folks ! You have created your first custom entity type. A proof ?
+
+```shell
+$ power_stencil info                                     
+    .
+    .
+    .
+--------------------------------------------------------------------------------
+ Available entity types:
+ - Type 'base_entity' --> PowerStencil::SystemEntityDefinitions::ProjectEntity
+ - Type 'custom_entity' --> MyCustomEntity
+ - Type 'entity_override' --> UniverseCompiler::Entity::Override
+ - Type 'plugin_definition' --> PowerStencil::SystemEntityDefinitions::Plugin (template-template path: '/opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/power_stencil-0.2.18/etc/templates/plugin_definition')
+ - Type 'process_descriptor' --> PowerStencil::SystemEntityDefinitions::ProcessDescriptor
+ - Type 'project_config' --> PowerStencil::SystemEntityDefinitions::ProjectConfig
+ - Type 'simple_exec' --> PowerStencil::SystemEntityDefinitions::SimpleExec (template-template path: '/opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/power_stencil-0.2.18/etc/templates/simple_exec')
+```
+You see that the `custom_entity` type appeared in the list. Need another proof ?
+```shell
+$ power_stencil create custom_entity/my_1st_custom_entity
+Created 'custom_entity/my_1st_custom_entity'
+
+$ power_stencil check cust -r
+RAW ENTITIES
+ 'custom_entity/my_1st_custom_entity':
+  - Storage path: '/tmp/tst2/.ps_project/entities/custom_entity/my_1st_custom_entity.yaml'
+  - Status      : Valid 
+  - Buildable   : false
+
+$ power_stencil get custom_entity/my_1st_custom_entity --raw
+--- !ruby/object:MyCustomEntity
+:name: my_1st_custom_entity
+```
+Ok, I must admit it's a pretty useless entity type, as we didn't add anything compared to the `base_entity` we are inheriting from, but it works, and notice how easy it was ! No complex configuration, no complex code... a breeze.
+
+## Directives
+
+You may have noticed in the previous example the directive `entity_type :custom_entity`... This looks a lot like directives some of you may be used to use in frameworks like [Ruby On Rails], and more specifically in [ActiveRecord]. If it is the case, then you won't be lost in how entity types work in `PowerStencil`. For the others you will discover how nice the Ruby syntax can be.
+
+As said before, an entity is basically a proxy to an internal Hash accessible through the `#fields` accessor. That said, most directives are actually a simple way to define constraints and control over the content of that Hash.
+
+Here are the available directives:
+
+### entity_type
+
+Obviously defines the type of entity. See previous example for how it is used. Although this is not mandatory, if you don't define an `entity_type`, the entity type in not listed in the output of `power_stencil info`. It doesn't mean the entity type is invalid. It just becomes an internal `entity_type`.
+
+### auto_named_entity_type
+
+Sometimes you don't want to give real names to some entity types. Without any parameter this directive will use the `entity_type` as default seed else of course the seed passed to this directive. The name of entities will be of the form `<seed>_ <an incremental counter>`.
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_entity
+
+  auto_named_entity_type
+end
+```
+
+```ruby
+PowerStencil DSL> new_custom_entity 
+=> #<MyCustomEntity:47058432471380 composite_key=[:custom_entity, "custom_entity_1"], @universe='Project entities (1566480766.837926)'>
+PowerStencil DSL> new_custom_entity 
+=> #<MyCustomEntity:47058432258540 composite_key=[:custom_entity, "custom_entity_2"], @universe='Project entities (1566480766.837926)'>
+PowerStencil DSL> new_custom_entity 
+=> #<MyCustomEntity:47058432038400 composite_key=[:custom_entity, "custom_entity_3"], @universe='Project entities (1566480766.837926)'>
+PowerStencil DSL> MyCustomEntity.auto_named_entity_type?
+=> true
+PowerStencil DSL> MyCustomEntity.auto_named_entity_type_seed
+=> "custom_entity"
+```
+
+
+### field
+
+As any persisted data is actually in the `#fields` Hash, you can declare shortcuts using the `field` directive. Let's say we have the following entity type:
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+    entity_type :custom_entity
+
+    field :my_custom_field
+end
+```
+Then it means I can access the `my_custom_field` property in two different ways:
+
+```ruby
+PowerStencil DSL> e = new_custom_entity name: :test_entity
+=> #<MyCustomEntity:47209361877520 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566476673.765155)'>
+PowerStencil DSL> e.my_custom_field
+=> nil
+PowerStencil DSL> e.my_custom_field = 'foo'
+=> "foo"
+PowerStencil DSL> e.my_custom_field
+=> "foo"
+PowerStencil DSL> e.fields[:my_custom_field]
+=> "foo"
+```
+:information_source: You can notice the `new_<entity_type>` DSL method which is a shortcut to create entities.
+
+So you can see the field directive will actually create accessor methods for first level properties in the `#fields` Hash.
+
+:warning: You can do pretty tricky things like:
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+    entity_type :custom_entity
+
+    field :my_custom_field
+    field 'another_field'
+end
+```
+
+```ruby
+PowerStencil DSL> e = new_custom_entity name: :test_entity
+=> #<MyCustomEntity:47332930509380 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566477499.215253)'>
+PowerStencil DSL> e.my_custom_field = :foo
+=> :foo
+PowerStencil DSL> e.another_field = :bar
+=> :bar
+PowerStencil DSL> e.fields
+=> {:name=>"test_entity", :my_custom_field=>:foo, "another_field"=>:bar}
+PowerStencil DSL> e.fields[:my_custom_field]
+=> :foo
+PowerStencil DSL> e.fields['another_field']
+=> :bar
+```
+
+:warning: Do not create two fields with the "same" name, once as a `Symbol` and another as a `String` ! Although it is perfectly legit at `#fields` level or in the YAML persisted file, the accessor methods generated would have the same name and you could expect some strange behaviour...
+
+:information_source: By default the [base_entity][PowerStencil::SystemEntityDefinitions::ProjectEntity] has only two fields declared:
+- The `name` field that we already saw and which is mandatory.
+- The `description` field which is optional, and which is pretty self-explanatory.
+
+### has_one
+
+This property may look familiar to users of [ActiveRecord], it introduces the concept of relationship between entities.
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_entity
+
+  has_one :custom_entity, name: :parent
+end
+```
+
+```ruby
+PowerStencil DSL> e = new_custom_entity name: :test_entity
+=> #<MyCustomEntity:47215552142260 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566478203.584915)'>
+PowerStencil DSL> p = new_custom_entity name: :foo
+=> #<MyCustomEntity:47215551934000 composite_key=[:custom_entity, "foo"], @universe='Project entities (1566478203.584915)'>
+PowerStencil DSL> e.parent = p 
+=> #<MyCustomEntity:47215551934000 composite_key=[:custom_entity, "foo"], @universe='Project entities (1566478203.584915)'>
+PowerStencil DSL> e.fields
+=> {:name=>"test_entity", :parent=>#<MyCustomEntity:47215551934000 composite_key=[:custom_entity, "foo"], @universe='Project entities (1566478203.584915)'>}
+PowerStencil DSL> p.save=> #<MyCustomEntity:47215551934000 composite_key=[:custom_entity, "foo"], @universe='Project entities (1566478203.584915)'>
+PowerStencil DSL> e.save
+=> #<MyCustomEntity:47215552142260 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566478203.584915)'>
+PowerStencil DSL> exit
+```
+
+Which translates at persistence level as:
+
+```shell
+$ power_stencil get custom_entity/test_entity --raw
+--- !ruby/object:MyCustomEntity
+:name: test_entity
+:parent: !psref
+  type: :custom_entity
+  name: foo
+```
+Here we are referencing an entity of the same type, but of course you can reference any type of entity. Using `has_one` will enforce the type of data you reference. Let's try to mess-up:
+
+```ruby
+PowerStencil DSL> e.parent = :bar
+=> :bar
+PowerStencil DSL> e.fields
+=> {:name=>"test_entity", :parent=>:bar}
+PowerStencil DSL> e.valid?
+=> false
+PowerStencil DSL> e.valid? raise_error: true
+UniverseCompiler::Error: Invalid entity '[:custom_entity, "test_entity"]' for fields parent !
+from /opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/universe_compiler-0.2.7/lib/universe_compiler/utils/error_propagation.rb:10:in `false_or_raise'
+PowerStencil DSL> e.save
+UniverseCompiler::Error: Invalid entity '[:custom_entity, "test_entity"]' for fields parent !
+from /opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/universe_compiler-0.2.7/lib/universe_compiler/utils/error_propagation.rb:10:in `false_or_raise'
+```
+So you see that if we try to set parent with something wrong, the accessor seems to accept, you can even see the `#fields` Hash updated. But as soon as you try to save to entity, or if you use the `#valid?` method, it complains about the type... Cool.
+
+:information_source: The `name` property of the `has_one` directive is optional. If not present the field will be named from the entity_type referenced instead...
+
+### has_many
+
+Once you know the `has_one` directive, you should not be surprised by the `has_many` directive...
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_entity
+  
+  has_many :base_entity, name: :sub_properties
+end
+```
+
+```ruby
+PowerStencil DSL> e = new_custom_entity name: :test_entity
+=> #<MyCustomEntity:47312117001560 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566482056.789871)'>
+PowerStencil DSL> sub_prop1 = new_base_entity name: :prop1
+=> #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47312116793880 composite_key=[:base_entity, "prop1"], @universe='Project entities (1566482056.789871)'>
+PowerStencil DSL> e.sub_properties << sub_prop1
+=> [#<PowerStencil::SystemEntityDefinitions::ProjectEntity:47312116793880 composite_key=[:base_entity, "prop1"], @universe='Project entities (1566482056.789871)'>]
+PowerStencil DSL> sub_prop2 = new_base_entity name: :prop2
+=> #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47312116175620 composite_key=[:base_entity, "prop2"], @universe='Project entities (1566482056.789871)'>
+PowerStencil DSL> e.sub_properties << sub_prop2
+=> [#<PowerStencil::SystemEntityDefinitions::ProjectEntity:47312116793880 composite_key=[:base_entity, "prop1"], @universe='Project entities (1566482056.789871)'>,
+ #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47312116175620 composite_key=[:base_entity, "prop2"], @universe='Project entities (1566482056.789871)'>]
+PowerStencil DSL> sub_prop1.save
+=> #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47312116793880 composite_key=[:base_entity, "prop1"], @universe='Project entities (1566482056.789871)'>
+PowerStencil DSL> sub_prop2.save
+=> #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47312116175620 composite_key=[:base_entity, "prop2"], @universe='Project entities (1566482056.789871)'>
+PowerStencil DSL> e.save
+=> #<MyCustomEntity:47312117001560 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566482056.789871)'>
+PowerStencil DSL> exit
+```
+
+Which translates at persistence level as:
+
+```shell
+--- !ruby/object:MyCustomEntity
+:sub_properties:
+- !psref
+  type: :base_entity
+  name: prop1
+- !psref
+  type: :base_entity
+  name: prop2
+:name: test_entity
+```
+Nice !
+
+### is_array
+
+You can define a field as being an array:
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_entity
+
+  field :an_array
+  is_array :an_array
+
+  field :another_array, :is_array
+end
+```
+
+```ruby
+PowerStencil DSL> e = new_custom_entity name: :test_entity
+=> #<MyCustomEntity:47285777457100 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566482755.453123)'>
+PowerStencil DSL> e.fields
+=> {:name=>"test_entity", :an_array=>[], :another_array=>[]}
+```
+
+:information_source: Please note the two syntaxes are equivalent.
+
+### is_hash
+
+You can define a field as being an array:
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_entity
+
+  field :a_hash
+  is_hash :a_hash
+
+  field :another_hash, :is_hash
+end
+```
+
+```ruby
+PowerStencil DSL> e = new_custom_entity name: :test_entity
+=> #<MyCustomEntity:46940048986180 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566483001.470293)'>
+PowerStencil DSL> e.fields
+=> {:name=>"test_entity", :a_hash=>{}, :another_hash=>{}}
+```
+
+:information_source: Please note the two syntaxes are equivalent.
+
+### not_null
+
+I guess you start getting used to what happens:
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_entity
+
+  field :a_field_not_null
+  not_null :a_field_not_null
+
+  field :another_field_not_null, :not_null
+end
+```
+
+```ruby
+PowerStencil DSL> e = new_custom_entity name: :test_entity
+=> #<MyCustomEntity:47124565766660 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566483255.255375)'>
+PowerStencil DSL> e.valid? raise_error: true
+UniverseCompiler::Error: Invalid entity '[:custom_entity, "test_entity"]' for fields a_field_not_null, another_field_not_null !
+from /opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/universe_compiler-0.2.7/lib/universe_compiler/utils/error_propagation.rb:10:in `false_or_raise'
+PowerStencil DSL> e.a_field_not_null = ''
+=> ""
+PowerStencil DSL> e.valid? raise_error: true
+UniverseCompiler::Error: Invalid entity '[:custom_entity, "test_entity"]' for fields another_field_not_null !
+from /opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/universe_compiler-0.2.7/lib/universe_compiler/utils/error_propagation.rb:10:in `false_or_raise'
+PowerStencil DSL> e.another_field_not_null = :yeah
+=> :yeah
+PowerStencil DSL> e.fields
+=> {:name=>"test_entity", :a_field_not_null=>"", :another_field_not_null=>:yeah}
+PowerStencil DSL> e.valid?
+=> true
+```
+Nothing very surprising there. Please note an empty string is not a null value...
+
+### not_empty
+
+This directive will adapt depending on the field type
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_entity
+
+  field :a_non_empty_field, :not_empty
+  field :a_non_empty_array, :is_array, :not_empty
+  field :a_non_empty_hash, :is_hash, :not_empty
+
+  has_many :base_entity, name: :friends
+  not_empty :friends
+end
+```
+
+```ruby
+PowerStencil DSL> e = new_custom_entity name: :test_entity, a_non_empty_field: :foo, a_non_empty_array: [:bar], a_non_empty_hash: {foo: :bar}
+=> #<MyCustomEntity:47261985525080 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566484354.658921)'>
+PowerStencil DSL> e.valid? raise_error: true
+UniverseCompiler::Error: Invalid entity '[:custom_entity, "test_entity"]' for fields friends !
+from /opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/universe_compiler-0.2.7/lib/universe_compiler/utils/error_propagation.rb:10:in `false_or_raise'
+PowerStencil DSL> sub_prop1 = new_base_entity name: :prop1
+=> #<PowerStencil::SystemEntityDefinitions::ProjectEntity:47261985113260 composite_key=[:base_entity, "prop1"], @universe='Project entities (1566484354.658921)'>
+PowerStencil DSL> e.friends << sub_prop1
+=> [#<PowerStencil::SystemEntityDefinitions::ProjectEntity:47261985113260 composite_key=[:base_entity, "prop1"], @universe='Project entities (1566484354.658921)'>]
+PowerStencil DSL> e.valid?
+=> true
+PowerStencil DSL> e.fields
+=> {:name=>"test_entity",
+ :a_non_empty_field=>:foo,
+ :a_non_empty_array=>[:bar],
+ :a_non_empty_hash=>{:foo=>:bar},
+ :friends=>[#<PowerStencil::SystemEntityDefinitions::ProjectEntity:47261985113260 composite_key=[:base_entity, "prop1"], @universe='Project entities (1566484354.658921)'>]}
+```
+
+Pretty simple nope ?
+
+:warning: Please note that neither `has_one` nor `has_many` support the "_condensed_"  version of the syntax (like `field` can). You have to specify the directive on a separated line. 
+
+:information_source: `has_one` does not support `not_empty` but support `not_null`, and the opposite for `has_many`...
+
+### should_match
+
+`should_match` will perform a validation of the content regarding a regular expression or a string. It should work on strings or symbols:
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_entity
+
+  field :ipv4, should_match: /^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$/
+  field :title, :not_null, should_match: /(Mr)|(Ms)/
+end
+```
+
+```ruby
+PowerStencil DSL> e = new_custom_entity name: :test_entity
+=> #<MyCustomEntity:47277853516360 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566485485.196701)'>
+PowerStencil DSL> e.valid? raise_error: true
+UniverseCompiler::Error: Invalid entity '[:custom_entity, "test_entity"]' for fields title !
+from /opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/universe_compiler-0.2.7/lib/universe_compiler/utils/error_propagation.rb:10:in `false_or_raise'
+PowerStencil DSL> e.title = 'PhD'
+=> "PhD"
+PowerStencil DSL> e.valid? raise_error: true
+UniverseCompiler::Error: Invalid entity '[:custom_entity, "test_entity"]' for fields title !
+from /opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/universe_compiler-0.2.7/lib/universe_compiler/utils/error_propagation.rb:10:in `false_or_raise'
+PowerStencil DSL> e.title = 'Mr'
+=> "Mr"
+PowerStencil DSL> e.valid? raise_error: true
+=> true
+PowerStencil DSL> e.ipv4 = 'a.b.c.d'
+=> "a.b.c.d"
+PowerStencil DSL> e.valid? raise_error: true
+UniverseCompiler::Error: Invalid entity '[:custom_entity, "test_entity"]' for fields ipv4 !
+from /opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/universe_compiler-0.2.7/lib/universe_compiler/utils/error_propagation.rb:10:in `false_or_raise'
+PowerStencil DSL> e.ipv4 = '192.168.0.1'
+=> "192.168.0.1"
+PowerStencil DSL> e.valid? raise_error: true
+=> true
+```
+
+:warning: As you can see there, unless specified as `not_null` or `not_empty`, the `should_match` validation will not be performed ! Pretty normal if you think about it...
+
+### class_name
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_entity
+
+  field :should_be_a_string, class_name: String
+end
+```
+
+```ruby
+PowerStencil DSL> e = new_custom_entity name: :test_entity
+=> #<MyCustomEntity:47164532040760 composite_key=[:custom_entity, "test_entity"], @universe='Project entities (1566486839.687064)'>
+PowerStencil DSL> e.valid? raise_error: true
+=> true
+PowerStencil DSL> e.should_be_a_string = 42
+=> 42
+PowerStencil DSL> e.valid? raise_error: true
+UniverseCompiler::Error: Invalid entity '[:custom_entity, "test_entity"]' for fields should_be_a_string !
+from /opt/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/universe_compiler-0.2.8/lib/universe_compiler/utils/error_propagation.rb:10:in `false_or_raise'
+PowerStencil DSL> e.should_be_a_string = 'okay... then'
+=> "okay... then"
+PowerStencil DSL> e.valid? raise_error: true
+=> true
+```
+`class_name` supports actually both a class (like here) or a class name.
+
+:warning: Use this one with caution, as it does not test anything regarding inheritance...
+
+### buildable / buildable_by
+
+One of the most important directives !
+More information is provided in the [builds] documentation, but basically this defines an `entity_type` as a potential target for the build process.
+
+Most probably you will use `buildable`,  `buildable_by` being used in the context of [plugins].
+
+```ruby
+class MyCustomEntity < PowerStencil::SystemEntityDefinitions::ProjectEntity
+  entity_type :custom_entity
+
+  buildable
+end
+```
+
+## Module you could include in your entity types
+
+There is one special module you could include in your entity types.
+
+If you want to define an entity which is not supposed to be persisted you could just include the [PowerStencil::SystemEntityDefinitions::NonPersistent] module et voilà... It could be a directive, but this is not something your are supposed to do a lot. It could become a directive in the future if the need appears.
+
+
+## Adding functional code
+
+Now you know an entity type is just a regular Ruby class. As such you could add any code you want to, for example, get a token from a server, perform a task on a db...whatever...
+
+**:warning: Although you can do anything there, if you see yourself starting writing a lot of code in your custom entities, you should probably consider creating a real [plugin][plugins] ! It will help you maintain and structure your code, on top of providing you a ton of extra cool stuff like extending command line by adding new sub-commands and/or options, having plugin-specific config, having templates-templates (:rofl:), plus the possibility to share/re-use your nice work across multiple projects !!**
+
+
+
+[:back:][Documentation root]
+<!-- End of Document -->
+
+<!-- Pages -->
+[Documentation root]: ../README.md "Back to documentation root"
+[templates]: templates.md "Templates in PowerStencil"
+[builds]: builds.md "Builds in PowerStencil"
+[plugins]: plugins.md "Plugins in PowerStencil"
+[example use cases]: example_use_cases.md "Example uses cases using PowerStencil"
+
+
+<!-- Code links -->
+[PowerStencil::SystemEntityDefinitions::ProjectEntity]: ../lib/power_stencil/system_entity_definitions/project_entity.rb "The base_entity entity type"
+[PowerStencil::SystemEntityDefinitions::NonPersistent]: ../lib/power_stencil/system_entity_definitions/non_persistent.rb "To make your entity type not persistent"
+[PowerStencil::SystemEntityDefinitions::HasAssociatedFiles]: ../lib/power_stencil/system_entity_definitions/has_associated_files.rb "Brings the templating power to your entity types"
+
+<!-- Illustrations -->
+
+
+<!-- External links -->
+[YAML]: https://yaml.org/ "The YAML official site"
+[Ruby]: https://www.ruby-lang.org "The powerful Ruby language"
+[Pry]: https://github.com/pry/pry "The awesome Pry console"
+[ActiveRecord]: https://guides.rubyonrails.org/active_record_basics.html "The ultimate ORM"
+[Ruby On Rails]: https://rubyonrails.org/ "One of the best Web framework"
+[universe_compiler]: https://gitlab.com/tools4devops/universe_compiler "The underlying engine to manage entities and compilation !"