teuton 2.4.1 → 2.4.3

data/docs/ideas/todo.md

@@ -1,124 +1,44 @@
 
-# Ideas
+# TODO list
 
-Esta es una lista de ideas para que no se me olvide... lo podemos ir ajustando 😊
+## Add
 
-1. TEST-EN-CADENA (smart-shell): es un proyecto en curso que trabajará en colaboración ceon teuton. Detecta las acciones del usuario dentro de la shell y automáticamente lanza el test para validar el cumplimiento de los objetivos. Si se supera un % se pasa al siguiente test.
+* Option roadmap_ok_only. When expect returns false... test stop. The rest of targets are forced to fails.
+* `expect_equal "camaleon"`, same as `expect result.equal "camaleon"`.
+* **FEATURE** Define a loop of executions of our activity. For example:
+  every 5 minutes, run the activity, and repeat this 10 times.
+  `start :times=>10, :every=>5 do`
+* Script to set **static IP**
+* Change export to show **command output** into reports.   
+* **TEST**: actualizar las pruebas con vagrant para la versión actual. Mejora de la calidad del código.
+* **DOCS** PROMOCIONAR.  Documentar y hacer videos. Tener foros de la comunidad o presencia en reddit o stackoverflow.... o algo parecido???
+* Create 2 evaluation modes:
+    1. Evaluate targets (current mode): `start eval: :targets`
+    1. Evaluate task: `start eval: :groups` and perhaps add group weight?
+    1. config file with `tt_eval: groups`
+
+Improve RESUME report or create stat-report, with stat information:
+* the worst target/task/case,
+* the best target/task/case,
+* the slowest target/task/case,
+* the fastest target/task/case, etc.
+* Related targets: group of targets that always have the same state in every case.
+
+## Ideas
 
-2. SERVIDOR TEUTON: Teuton web para recibir peticiones de los clientes... ?! Enganchar con teuton-server y teuton-client
-NO. quizás mejor enfoque. Tener un servidor dedicado con una página Web para lanzar test desde el propio servidor.
+Esta es una lista de ideas para que no se me olvide... lo podemos ir ajustando 😊
 
-3. Editor de ficheros de configuración.... una especie de asistente de ayuda..
-no le veo mucho sentido tener un editor específico pero si al ejecutar un reto
-si hay valores con NODATA... entonces que pregunte al usuario y complete la
+1. TEST-EN-CADENA (smart-shell): es un proyecto en curso que trabajará en colaboración ceon teuton. Detecta las acciones del usuario dentro de la shell y automáticamente lanza el test para validar el cumplimiento de los objetivos. Si se supera un % se pasa al siguiente test.
+2. SERVIDOR TEUTON: Teuton web para recibir peticiones de los clientes... ?! Enganchar con teuton-server y teuton-client. NO. quizás mejor enfoque. Tener un servidor dedicado con una página Web para lanzar test desde el propio servidor.
+3. Editor de ficheros de configuración.... una especie de asistente de ayuda.. no le veo mucho sentido tener un editor específico pero si al ejecutar un reto si hay valores con NODATA... entonces que pregunte al usuario y complete la
 configuración sobre la marcha???
-
 4. Editor de tests ?! .... lo veo complejo... a menos que se empiece por una versión simplificada.. El DSL es tan sencillo que no le veo sentido a un editor de retos. Pero si podemos aprovechar y crear una" librería " de los módulos más frecuentes?!... una ayuda para facilitar la creación de test... esto se puede emparejar con el editor de test...
+5. `send :email_to => :members_emails`... esto requiere que cada caso tenga un email y que además el profesor tenga configurado el acceso a su cuenta de correo... para poder enviar copias de los informes....
+6. When students demand help they could recibe some advises. More ideas: bonus, lives, etc.
+7. save partial grades. Before play challenge read previous moodle.cvs. Then play challenge and when creates new moodle.csv... we could preseve old cases(grades) 100% for example.
 
-5. `result.test("...")`
-6. `send :email_to => :members_emails`
-
-7. Empaquetar tests para facilitar su movilidad . Esto se combina muy bien con la opción de búsqueda de retos.... pensar en firma gpg o md5 de comprobación etc..  
-Al ejecutar un test poder hacerlo "teuton run PATH/TO/FILE.zip"
-
-
-# Gamification ideas
-
-* When students demand help they could recibe some advises.
-* Define a loop of executions of our activity. For example:
-  every 5 minutes, run the activity, and repeat this 10 times.
-  `start :times=>10, :duration=>5 do`      
-* More ideas: bonus, lives, etc.
-
-# Ideas para QA
-
-* actualizar las pruebas con vagrant para la versión actual. Mejora de la calidad del código.
-
-# Ideas para PROMOCIONAR
-
-* Documentar y hacer videos.
-* Tener foros de la comunidad o presencia en reddit o stackoverflow.... o algo parecido???
-
-Create 2 evaluation modes:
-1. Evaluate targets (current mode): `start :score=>:targets do ...`
-1. Evaluate task: `start :score=>:groups do ...`
-
-
-## Pending
-
-### moodle.csv and save partial grades.
-
-* Before play challenge read previous moodle.cvs. Then play challenge and when creates new moodle.csv... we could preseve old cases(grades) 100% for example.
-
-### expect keyword
-
-* `expect_equal "camaleon"`, same as `expect result.equal "camaleon"`.
-* `expect_not_equal "lagarto"`, same as not expect equal.
-* `expect regexp("val1|var2")`, regexp function creates regular expresion from String.
-* `expect result.ok?`, result from previous goto exec.
-* `result.exit_code`, exit code from previous goto exec.
-
-### Reports and export keyword
-
-* Work on more output formats: CSV, HTML, XML.
-    * `export :format=>:html, :prefix => IAN`
-    * `export :format=>:xml, :prefix => IAN`
-    * `export :format=>:csv, :prefix => IAN`
-* Improve RESUME report or create stat-report, with stat information:
-    * the worst target/task/case,
-    * the best target/task/case,
-    * the slowest target/task/case,
-    * the fastest target/task/case, etc.
-    * Related targets: group of targets that always have the same state in every case.
-
-### File extension
-
-* Change ".rb" file extension by ".tt" for example. To avoid users think on ruby when using teuton files.
-* So move "ruby language" documentation to other section like "expert mode"...
-* Change "start.rb" or "play.rb" name to "run.tt".
-
-### loop keyword
-
-* Create keyword to do loops without using ruby style:
-```
-loop i: 1..5 do
- puts i
-end
-
-loop :a => [3,5,7] do
-  puts a
-end
-```
-
-### Feedback
-
-Add new features to DSL
-* `feedback or advise "Some usefull information"` provide this information when studends demand help.
-## Working
-
-### Remote temp folder
-
-Modify send keyword so it could works with all OS. Now only work with GNU/Linux. So we need to detect remote OS, then choose remote teuton folder where uploading files.
-
-* First goto open new session to remote host. And then:
-    1. Detect remote OSTYPE: (a) GNU/Linux and MAC using "uname -a", (b) Windows using "ver" command.
-    2. Create teuton remote uploading folder: (a) /tmp/teuton for GNU/Linux and MAC, (b) %windir%/temp/teuton for Windows.
-* Every time we send files to remote host we will use this remote folder.
-
-### Readme
-
-* Pass usefull information into README.md:
-```
-readme do
-  title ...
-  intro ...
-  author ...
-  requirements ...
-  tags ...
-end
-```
-
----
+# Propuestas de sdelquin
 
-* teuton run into
-* txt format -> remove conn_errors when errors = 0
+- Docker en vez de vagrant?? → https://www.cloudbees.com/blog/ssh-into-a-docker-container-how-to-execute-your-commands
+- Tablas "bonitas" en terminal → https://github.com/piotrmurach/tty-markdown#17-table
+- Documentación en https://readthedocs.org/ ??

data/docs/learn/01-cmd_new.md

@@ -0,0 +1,28 @@
+
+[<< back](README.md)
+
+# new test
+
+Create skeleton for a new project: `teuton create foo`
+
+```
+> teuton new foo
+
+[INFO] Creating foo project skeleton
+* Create dir        => foo
+* Create file       => foo/config.yaml
+* Create file       => foo/start.rb
+```
+
+> NOTA: It is posible to create these files by hand.
+
+This command will create:
+
+| File/Directory  | Description    |
+| --------------- | -------------- |
+| foo             | Base directory |
+| foo/start.rb    | Main Script    |
+| foo/config.yaml | YAML configuration file |
+
+
+Now it is time to customize your targets.

data/docs/learn/{01-target.md → 02-target.md}

@@ -1,16 +1,14 @@
 [<< back](README.md)
 
-# Example: target
+# target
 
-A target is a feature you want to check.
+A [target](../dsl/target.md) is a feature you want to check. Targets are defined into `group` sections and every evaluation has 3 parts:
 
-## Definition section
+## Target definition
 
-Test definition section (`group`):
-
-* **target**: Description of the element to be tested.
-* **run**: Execute a command `id david` on localhost.
-* **expect**: Ensure the result contains expected value.
+* [target](dsl/target.md): Description of the element to be tested.
+* [run](../dsl/run.md): Execute a command `id david` on localhost.
+* [expect](../&dsl/expect.md): Verify that the result contains expected value.
 
 ```ruby
 group "Learn about targets" do
@@ -22,14 +20,14 @@ group "Learn about targets" do
 end
 ```
 
-> In this example, localhost's OS must be GNU/Linux (any other compatible OS) because the command used is `id david`.
+> In this example, our localhost's OS is GNU/Linux (or any other compatible OS) because the command executed is `id david`.
 
 ## Execution section
 
-Test execution section (`play`):
+When we run this teuton test, the execution section (`play`) is processed. This seccion now contains this:
 
-* **show**: display process information on screen.
-* **export**: build output reports.
+* [show](../dsl/show.md): display process information on screen.
+* [export](../dsl/export.md): build output reports.
 
 ```ruby
 play do
@@ -38,7 +36,7 @@ play do
 end
 ```
 
-## Running the test
+## Example
 
 Execute this command to run the test:
 
@@ -52,9 +50,7 @@ CASE RESULTS
 +------+-----------+-------+-------+
 ```
 
-## Output reports
-
-Report files created into `var/01-target/`:
+Report files are created into `var/02-target/` folder:
 
 ```console
 var
@@ -67,7 +63,7 @@ var
 Let's see one report:
 
 ```
-> more var/01-target/case-01.txt
+> more var/02-target/case-01.txt
 
 CONFIGURATION
 +-------------+-----------+

data/docs/learn/03-remote_hosts.md

@@ -1,121 +1,93 @@
 [<< back](README.md)
 
-# Example: 03-remote-hosts
+# Remote hosts
 
-Summary:
-* Check a group of remote hosts.
-* Export reports using other output formats.
-* Checking remote machine (host1) with Windows OS.
+To run test on remote host we have to define them into config file.
 
-1. [Config file](#config-file)
-2. [Definition section](#definition-section).
-3. [Execution section](#execution-section).
-4. [Results](#results).
-
-## Config file
-
-This configuration file contains:
-* 2 global params denitions.
-* 3 cases.
-* 4 specific case params definitions.
+Let's see an example with:
+* 1 global param.
+* 3 cases with 3 params.
 
 ```yaml
 ---
 global:
   host1_username: root
-  host1_password: profesor
 cases:
-- tt_members: Darth Maul
+- tt_members: student_1
   host1_ip: 192.168.1.201
-  host1_hostname: siths
-  username: maul
-- tt_members: R2D2
+  host1_password: secret_1
+- tt_members: student_2
   host1_ip: 192.168.1.202
-  host1_hostname: robots
-  username: r2d2
-- tt_members: Obiwan Kenobi
-  host1_ip: 192.168.1.203
-  host1_hostname: jedis
-  username: obiwan
+  host1_password: secret_2
+- tt_members: student_3
+  host1_ip: 127.0.0.1
+  host1_password: secret_3
 ```
 
-## Definition section
-
-Define 3 targets (items to be checked):
+Every remote host definition require some params:
 
-```ruby
-group "How to test remote Windows hosts" do
-  target "Update hostname with #{gett(:host1_hostname)}"
-  run "hostname", on: :host1
-  expect_one get(:host1_hostname)
-
-  target "Ensure network DNS configuration is working"
-  run "nslookup www.google.es", on: :host1
-  expect "Nombre:"
-
-  target "Create user #{gett(:username)}"
-  run "net user", on: :host1
-  expect get(:username)
-end
-```
+| Param | Description    | Default value |
+| ----- | -------------- | ------------- |
+| ip    | Remote host IP | |
+| port  | Remote host port | 22 |
+| username | Remote user account | Not required with public SSH id |
+| password | Remote user pasword | Not required with public SSH id |
+| protocol | SSH or Telner | SSH |
+| route | Defines host2 used as gateway to reach host | |
 
-> NOTE: This example requires Windows OS on remote machine (host1).
+## Definition section
 
-## Execution section
+Define 1 target (item to be checked):
 
 ```ruby
-play do
-  show
-  # export using other output formats
-  export format: :txt
-  export format: :html
-  send copy_to: :host1
+group "Remote hosts" do
+  target "Create user david"
+  run "id david", on: :host1
+  expect "david"
 end
 ```
 
-* `show`, show process log on screen.
-* `export format: :txt`, create output reports files into `var/learn-03-remote-host/` directory using `txt` format.
-* `export format: :html`, create output reports into `var/learn-03-remote-host/` directory using `html` format.
-
-> Several output formats available: txt, colored_text, html, json and yaml.
-
-* `send copy_to: :host1` keyword copy output report into remote machine (host1).
+> NOTE: This example requires GNU/Linux OS on remote machine (host1).
 
-## Screen output
+Execution:
 
 ```
+> teuton run examples/03-remote_hosts
+
 CASE RESULTS
-+------+---------------+-------+-------+
-| CASE | MEMBERS       | GRADE | STATE |
-| 01   | Darth Maul    | 0.0   | ?     |
-| -    | -             | 0.0   |       |
-| 03   | Obiwan Kenobi | 0.0   | ?     |
-+------+---------------+-------+-------+
++------+-----------+-------+-------+
+| CASE | MEMBERS   | GRADE | STATE |
+| 01   | student_1 | 0.0   | ?     |
+| 02   | student_2 | 0.0   | ?     |
+| 03   | student_3 | 100.0 | ✔     |
++------+-----------+-------+-------+
 
 CONN ERRORS
-+------+---------------+-------+------------------+
-| CASE | MEMBERS       | HOST  | ERROR            |
-| 01   | Darth Maul    | host1 | host_unreachable |
-| 03   | Obiwan Kenobi | host1 | host_unreachable |
-+------+---------------+-------+------------------+
++------+-----------+-------+------------------+
+| CASE | MEMBERS   | HOST  | ERROR            |
+| 01   | student_1 | host1 | host_unreachable |
+| 02   | student_2 | host1 | host_unreachable |
++------+-----------+-------+------------------+
 ```
 
-## Results
+Notice that case-03 is 100% and conection works. It is running on localhost because has localhost IP (127.0.0.1)
+
+Results:
 
 ```
-var
-└── 03-remote-hosts
-    ├── case-01.json
-    ├── case-01.txt
-    ├── case-03.json
-    ├── case-03.txt
-    ├── moodle.csv
-    ├── resume.json
-    └── resume.txt
+❯ tree var/03-remote_hosts
+
+var/03-remote_hosts
+├── case-01.txt
+├── case-02.txt
+├── case-03.txt
+├── moodle.csv
+└── resume.txt
 ```
 
-* `case-01`, report with details about case 01 (maul)
-* Case 02 (r2ds) is skipped. So there are no report `case-02`.
-* `case-03`, report with details about case 03 (obiwan)
-* `resume`, report with global resumed information about all cases.
-* `moodle.csv`, CVS file with required fields to upload grades into Moodle eLearning platform.
+## tt_skip param
+
+To disable a case, add skip param `tt_skip: true`.
+* `tt_skip` it is true by default.
+* `tt_skip: false` ignore this case.
+* `tt_skip: true`, evaluate this case.

data/docs/learn/04-config.md

@@ -0,0 +1,147 @@
+[<< back](README.md)
+
+# config
+
+_It's good idea save dinamic data into config file._
+
+By default, `config.yaml` is our config file. Example:
+
+```yaml
+global:
+cases:
+- tt_members: name_student_1
+  username: david
+- tt_members: name_student_2
+  username: fran
+```
+
+> **How to choose another config file?** Read this [document](../commands/example_run.md#3-choosing-config-file).
+
+By default, `start.rb` it's our main execution file. Example:
+
+```ruby
+group "Reading params from config file" do
+
+  target "Create user #{get(:username)}"
+  run "id #{get(:username)}"
+  expect get(:username)
+
+end
+```
+
+* [get](../dsl/get.md) keyword read params from configuration file. It's posible personalize tests with diferent values for every case.
+
+## Example
+
+```console
+> teuton run examples/04-config
+
+CASE RESULTS
++------+----------------+-------+-------+
+| CASE | MEMBERS        | GRADE | STATE |
+| 01   | name_student_1 | 100.0 | ✔     |
+| 02   | name_student_2 | 0.0   | ?     |
++------+----------------+-------+-------+
+```
+
+Reports:
+
+```
+var/04-config
+├── case-01.txt
+├── case-02.txt
+├── moodle.csv
+└── resume.txt
+```
+
+Let's see case 01 report.
+
+```
+> more var/04-config/case-01.txt
+
+CONFIGURATION
++-------------+----------------+
+| tt_members  | name_student_1 |
+| tt_sequence | false          |
+| tt_skip     | false          |
+| tt_testname | 04-config      |
+| username    | david          |
++-------------+----------------+
+
+GROUPS
+- Reading params from config file
+    01 (1.0/1.0)
+        Description : Create user david
+        Command     : id david
+        Duration    : 0.002 (local)
+        Alterations : find(david) & count
+        Expected    : Greater than 0 (String)
+        Result      : 1 (Integer)
+
+RESULTS
++--------------+---------------------------+
+| case_id      | 01                        |
+| start_time   | 2023-01-25 20:46:13 +0000 |
+| finish_time  | 2023-01-25 20:46:13 +0000 |
+| duration     | 0.001778546               |
+| unique_fault | 0                         |
+| max_weight   | 1.0                       |
+| good_weight  | 1.0                       |
+| fail_weight  | 0.0                       |
+| fail_counter | 0                         |
+| grade        | 100                       |
++--------------+---------------------------+```
+```
+
+## Using differents configuration files
+
+Example with 3 config files (yaml files):
+
+```
+❯ tree examples/04-config
+
+examples/04-config
+├── config.yaml
+├── rock.yaml
+├── start.rb
+└── starwars.yaml
+```
+
+Usign default config file (`config.yaml`):
+
+```
+❯ teuton run examples/04-config
+
+CASE RESULTS
++------+----------------+-------+-------+
+| CASE | MEMBERS        | GRADE | STATE |
+| 01   | name_student_1 | 100.0 | ✔     |
+| 02   | name_student_2 | 0.0   | ?     |
++------+----------------+-------+-------+
+```
+
+Using `example/04-config/starwars.yaml`:
+
+```
+❯ teuton run --cname=starwars examples/04-config
+
+CASE RESULTS
++------+------------+-------+-------+
+| CASE | MEMBERS    | GRADE | STATE |
+| 01   | Yoda       | 0.0   | ?     |
+| 02   | Darth Maul | 0.0   | ?     |
++------+------------+-------+-------+
+```
+
+Using `example/04-config/rock.yaml`:
+
+```
+❯ teuton run --cpath=examples/04-config/rock.yaml examples/04-config
+
+CASE RESULTS
++------+------------+-------+-------+
+| CASE | MEMBERS    | GRADE | STATE |
+| 01   | AC/DC band | 0.0   | ?     |
+| 02   | Muse band  | 0.0   | ?     |
++------+------------+-------+-------+
+```