clerq 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d95f522537af88379a6881b154fc50453973102601db1cb4be573f655a33a78d
-  data.tar.gz: cb5e26ff19ff66d0ecc6364decb2caaee046c763956fdd2c39b9e71e932c6abb
+  metadata.gz: c4d9114f078750db97781623afefdbd33f39809dc02c6f68dfe8303d0764b751
+  data.tar.gz: d9cb6491239baa42fdc945ad0282aa3683aa850079de52c26bd5f37a370e01fa
 SHA512:
-  metadata.gz: 2ce20279f674973d75ab59d1cdaf474248459a442dd58eeebb9980c1d59fe4c687c7956e72bade7be78b9b5134b595e35e3859d5a983cb009afcb17481a47ff5
-  data.tar.gz: 55b67ea4ef4b43285c6c5eb51b49d2c3fb2cdef7f323dfe897a4e8a9ecd5644d3b6b3ee563ab78bc06291a2cdc9a3b6ed9e2bcc784669bc842438ac0f96a42b1
+  metadata.gz: cf035ee76e81ff8f565e5d5448773185075839589ebea7773ed8ace9e7477bb8700c805419e17dbc358608b57db01008fc056f7fee203b37bf1966d6ed4c9ff7
+  data.tar.gz: 1758cc13f790117e93944ccc17f97bb738a26cede5c3e70d462dedae9c8deb4aedeaffe868f3056d527ab207864cef083ffdb48b6eabf63c34be3f623639bbb8

data/.gitignore

@@ -1,5 +1,4 @@
 /lib/sandbox/
-/test/.sandbox/
 /.imdone/
 /.bundle/
 /.yardoc
@@ -8,5 +7,4 @@
 /doc/
 /pkg/
 /spec/reports/
-/test/tmp/
-/tmp/
+Gemfile.lock

data/CHANGELOG.md

@@ -0,0 +1,22 @@
+# Change log
+
+## 0.2.0 (2019-11-23)
+
+Changes:
+
+* Started new project [Clerq Video Guide](https://github.com/nvoynov/clerq-video-guide) that provides example of using Clerq.
+* Done massive refactoring of source code; no more gateways.
+* Totally redesigned NodeRepository and TextRepository.
+* Improved interactors caused by gateways throwing out.
+* Improved tests suite; no more DEPRECATED Minitest; Dir.mktmpdir is used for sandbox.
+* Improved README to include all the changes mentioned before.
+* Improved `clerq new PROJECT` command; when the `PROJECT` parameter consists of more than one word, it will create `<project>.thor` file that follows to usual ruby file and class naming conventions; e.g. for `user guide` it will create `user_guide.thor` and `class UserGuide < Thor` inside.
+* Shortened `content.md.tt` content
+* Only two templates in the box are left - `default.md.erb` and `pandoc.md.erb`.
+* Thor `error(msg)` in `cli.rb` changed to `stop!(mgs); raise Thor::Error`.
+* Improved CLI for `build`, `check`, `toc`, `node`; now each one checks if the command is running in clerq project, checked if `clerq.yml` or `src` folder exist.
+* Other small improvements.
+
+## 0.1.0 (2019-11-08)
+
+First release

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    clerq (0.1.0)
+    clerq (0.2.0)
       thor (~> 0.20.3)
 
 GEM

data/README.md

@@ -1,42 +1,139 @@
 # Clerq
 
-Gem `clerq` represents an unusual but effective way of writing and managing requirements. It resembles some static site builders and was inspired by those. Actually this gem provides the ability to write requirements in separate files and combine those to a unified consistent requirements source for any future purposes.
+__What is Clerq?__
 
-Sounds too simple? Combine it with a modern text editor that supports markdown, place repository under `Git`, install `Pandoc` to convert it to any supported format. Create your own commands through `Thor` to automate every piece of work that can be automated. Give it a try!
+The Clerq is a toolbox for manipulating the hierarchy of text data placed in separate markdown files. It implements three basic ideas:
+
+1. Text data repository in file system, based on markdown files with few extra conventions.
+2. Ruby gem that provides access to the text hierarchy from the repository.
+3. Basic CLI to manage the repository and compile the text data to documents based on erb-templates.
+
+__What for?__
+
+The Clerq is suitable for writing thick structured texts. The initial purpose for the system was the "requirements management in file system" and it supposed to help in writing stuff like Vision, RFP, URD, SRS, SAD, and deriving various requirements based artifacts. But now it seems much wider.
 
 ## Installation
 
-Add this line to your application's Gemfile:
+Install it yourself as:
+
+    $ gem install clerq
+
+## Usage
+
+The Clerq is entirely based on one single domain entity `Node` that represents a node of tree hierarchy and provides `id`, `title`, `body`, and `metadata` attributes. It supposes the following simple workflow:
+
+* you create files with text content,
+* and manipulate the data by scripts.
+
+### Project
+
+The Clerq project lives in the following folders structure that will be created by `clerq new <project>`:
+
+* `bin/` - for output documents;
+* `bin/assets` - for assets;
+* `knb/` - knowledge base;
+* `lib/` - place for extra Ruby code;
+* `src/` - source data repository;
+* `tt/` - templates;
+* `<project>.thor` - see [Scripting](#scripting);
+* `clerq.yml` - project settings;
+* `README.md`.
+
+### Writing
+
+The Clerq reads nodes from a set of separate files and assembles it to a single hierarchy. There are a few conventions for a separate file that will become a part of hierarchy.
+
+#### Files
+
+The first convention is the scheme how a markdown content becomes the `Node` entity.
 
-```ruby
-gem 'clerq'
 ```
+# [p2] Part two
+{{parent: top}}
 
-And then execute:
+Body
+```
 
-    $ bundle
+Where
 
-Or install it yourself as:
+* `#` familiar markdown header that indicates a new `node`;
+* `[p1]` is an optional identifier that becomes `node.id`;
+* `Part two` is an optional `node.title`;
+* `{{parent: top}}` in an optional metadata section that becomes `node.meta`;
+* and finally `Body` is an optional `node.body`.
 
-    $ gem install clerq
+Every new header (`#`) at any level indicates a new node. When a file contains headers of different levels, the nodes will be created in a natural hierarchy based on header levels. So as the result of reading the content below, the Clerq will create the natural hierarchy with root node `Top` that holds two child nodes `First` and `Second`.
 
-## Usage
+```markdown
+# Top
+## First
+## Second
+```
+
+One more extra thing is links. You can place links to other nodes in the body section of the file content by using `[[<id>]]` macro. It can be handled in templates.
+
+#### IDs
+
+Each node must have its own unique id so that you can refer to it in other parts of the project. That's why the recommended practice is to put the id straight into the header `# [node id] node title`.
+
+ID can start with one dot, like `[.suffix]`, and clerq will add id of parent node. For the followed example, `[.fm]` will be translated to `[cm.fm]`.
+
+```
+# 3 Function requirements
+## [cm] Components
+### [.fm] File manager
+### Logger
+```
+
+When an id is not provided, Clerq will generate it automatically, and you can freely combine nodes that have id and that has not. For the example above, the `Logger`  will be identified as `[cm.01] Logger`.
+
+#### Meta
+
+The excerpt, the text in brackets `{{ }}` that follows by the header, contains node attributes. And the second convention mentioned in [Writing](#writing) section is two magic metadata attributes that specify parameters of a hierarchy:
+
+1. `parent: <id>` indicates that the node belongs to a node with specified `id`;
+2. `order_index: <id1> <id2>` indicates that child nodes must be lined up in specified order.
+
+You can place in metadata any simple string that suitable for providing additional information like status, originator, author, priority, etc. E.g.
+
+```
+# [r.1]
+{{parent: r, status: draft}}
+
+# [r.2]
+{{parent: r
+}}
+
+# [r.3]
+{{
+parent: r}}
+```
+
+#### Assets
+
+When you want to provide some assets or links to something outside the repository you can provide the lint to the assets. Put the asset in the `bin/assets` folder and specify the link.
+
+```markdown
+# [ent] Entities
+
+The following picture shows something
 
-To see the list of all standard clerq commands type in console `clerq help` and follow the printed instructions.
+![Image](assets/er.png)
+```
 
-To see the list of all specific for the project  commands type `thor help <project>`
+### CLI
 
-Actually the best way to start with Clerq is to see it in action through [Promo](#promo). Just start from copying promo content and running some commands.
+Clerq provides CLI that is based on Thor, so all standard thor features are supported. To print all Clerq commands type `$ clerq help` in your console. To see the list of all the project-specific commands type `thor help <project>`.
 
-### Creating new project
+#### Create new project
 
 To create a new project run `new` command:
 
     $ clerq new <project_name>
 
-### Create new item
+#### Create new file
 
-The simplest way of adding new items to the project is to add a new file to the `src` directory. Of course, Clerq also provides the command `node`
+The simplest way of adding new items to the project is to add a new file to the `src` directory. Of course, Clerq also provides the command `node` that can create template-based files:
 
     $ clerq node ID [TITLE] [-t TEMPLATE]
 
@@ -46,13 +143,13 @@ If you are using images or other assets, you should place it to `bin/assets` dir
 
 __Templates__
 
-You also can prepare your own templates it `tt` folder and provide template through `-t/--template` option. The content of the template file will be placed on the body of the requirement.
+You also can prepare your own templates it `tt` folder and provide template through `-t/--template` option. The content of the template will be placed on the created file.
 
-### Check repository
+#### Check repository
 
 Because of lots of handwriting there can be some specific errors in repository. The most obvious are:
 
-* non-unique requirements identifiers;
+* non-unique identifiers;
 * links to and id that does not exist:
    * for `parent` attribute;
    * in `order_index`;
@@ -62,9 +159,9 @@ The system provides command `clerq check` that will check the repository for the
 
     $ clerq check
 
-### Build project
+#### Build project
 
-Clerq provides the ability to combine all requirements from the project repository and create final document. To create such document you can use `clerq build` command:
+Clerq provides the ability to combine all the text data from the project repository and create the final document. To create such document you can use `clerq build` command:
 
     $ clerq build
 
@@ -81,182 +178,129 @@ You also can specify these settings through `clerq build` options:
 * `-t/--template TEMPLATE` provides the ability to specify template;
 * `-o/--output FILE_NAME` provides the ability to specify output file name.
 
-__Query requirements__
+__Queries__
 
-Clerq also provides ability to query requirements that meet query criteria. To query requirements you should use `-q/--query QUERY_STRING` where `QUERY_STRING` is ruby code that will test if each node meets the  `QUERY_STRING`. For example, `node.tile == 'Functional requirements'` or `node.id == 'us'`.
+Clerq provides the ability to query data that match query criteria. To query data you should use `-q/--query QUERY_STRING` option where `QUERY_STRING` is ruby code that will test if each node matches the  `QUERY_STRING`. For example, `node.tile == 'Functional requirements'` or `node.id == 'us'`.
 
-### Print TOC
+#### Print TOC
 
 Sometimes it helpful to check repository structure by `clerq toc` command. The command also supports `-q/--query QUERY_STRING` option.
 
-## Known issues
+### Scripting
 
-### Failed test
+The section assumes that you are familiar with Ruby or some other programming language.
 
-Some tests of CLI fail by `$ bundle exec rake test` but pass individually one by one through `$ bundle exec rake test TEST=test/cli/cli_build_spec.rb` and I haven't caught the reason.
+Using the basic commands described in [CLI](#cli) section gives you just the ability to create final documents or other output. But this is just the tip of the iceberg, just the beginning, and you can do much more than that with Clerq.
 
-### Thor version
+A usual scenario will consist of two simple steps:
 
-The one issue I certain in is when you are using different version of thor, your custom scripts won't work.
+1. Get data hierarchy from the repository.
+2. Do some processing of the hierarchy.
 
-## Structure
+Instead of adding extra scripts files somewhere in the project, you can write tasks to `<project>.thor` file and access to them through `thor <project>:<your-task> [<params>]`.
 
-### Project
+#### Node class
 
-The Clerq project has the following folders structure by default (that will be created by `clerq new <project>`):
+The [Writing](#writing) section provides the basic knowledge to understand Clerq, and now it is the right time to see the [Node class](https://github.com/nvoynov/clerq/blob/master/lib/clerq/entities/node.rb). It implements the Composite pattern.
 
-* `bin/` - for output documents;
-* `bin/assets` - for assets;
-* `knb/` - knowledge base;
-* `lib/` - place for Ruby code;
-* `src/` - place for requirements;
-* `tt/` - templates;
-* `<project>.thor` - file with automated tasks (see more in [Automating](#automating));
-* `clerq.yml` - project settings;
-* `README.md`.
-
-### Repository
-
-Place requirements to the `src` folder. You can group your requirements by different folders and subfolders - Clerq load all the files of `src` including all subfolders at any level of nesting.
-
-### Node
-
-Each requirement is a markdown file with a few additional compliances (will be explained below) where every file can contain any number of requirements. Let's meet some files ...
+#### Interactors
 
-**content.md**
+Clerq provides five followed interactors:
 
-```markdown
-# 1 Introduction
-{skip_meta: true}
-## 1.1 Purpose
-## 1.2 Scope
-## 1.3 References
-## 1.4 Definitions
-## 1.5 Overview
-# [f] Requirements
-# [i] Interfaces
-# [n] Non-functional requirements
-# [c] Design constraints
-```
+* `QueryAssembly` provides assembly of repository as root Node;
+* `CheckAssembly` checks the assembly for errors (ids and links);
+* `RenderAssembly` render assembly by provided erb-template;
+* `CreateNode` crates new node in the repository;
+* `QueryTemplate` provides text of the template provided as parameter.
 
-**fm.md**
-
-```markdown
-# [.fm] File manager
-{{parent: f}}
+The first part of each repository related task is to get repository assembly. It can be performed through  `NodeRepository#assemble` or `QueryAssembly.call`. Each of these methods returns Node that provides [Enumerable](https://ruby-doc.org/core-2.6.5/Enumerable.html) interface.
 
-The system shall provide the `File Manager` component. The component shall provide the following features:
+Let's invent some advanced scenario. Assume that you develop a "User requirements document" and the project policy requires that each user requirement must have the parameter called `originator`. You can write the policy as followed:
 
-{{@@list}}
-
-## Read folders structure
-## Read file content
-## Load ".md"
-## Load ".tt"
+```ruby
+require 'clerq'
+include Clerk::Interactors
+
+# supposed you have something like user requirements document
+node = QueryAssembly.("node.title == 'User requirements'")
+miss = node.drop(1).select{|n| n[:originator].empty? }
+unless miss.empty?
+  errmsg = "`Originator` is missed for the next nodes:\n"
+  errmsg << miss.map(&:id).join(', ')
+  raise Error, errmsg
+end
 ```
 
-#### Headers
+#### Root Node
 
-Every requirement starts with markdown header. All the text between headers belongs to the requirement.
+A hierarchy starts form root node and Clerq provides the root node with parameter `title` specified in `clerq.yml` file. The subject is a bit tricky actually and there are few extra considerations I try to explain below (and you can always see tests)
 
-#### Identifiers
+When your repository stills empty, the Clerq will provide you with the root node. From one point it resembles the NullObject.
 
-Each requirement must have its own unique identifier so that you can refer to it in other parts of the project. That's why the recommended practice is to put id straight into the header `# [requirement_id] requirement title`.
+When you have a few root nodes in your repository, those become  direct childs of the root node. But when your repository contains single root node, the Clerq will return the single node as root node.
 
-ID can start with one dot, like `[.suffix]`, and clerq will add parent requirement id before. For the followed example, `[.fm]` will be translated to `[cm.fm]`.
+The following example does not provide root node and it causes adding root node from `clerq.yml`.
 
 ```
-# 3 Function requirements
-## [cm] Components
-### [.fm] File manager
-### Logger
+# User requirements
+# Functional requirements
 ```
 
-When an identifier is not provided, Clerq will generate it automatically, and you can combine requirements that have id and requirements that does not. For the example above, the `Logger`  will be identified as `[cm.01] Logger`.
-
-#### Attributes
-
-The excerpt, the text in brackets `{{ }}` that follows by the header, contains requirement attributes. You can place here anything you need to provide additional information, like status, source, author, priority, etc. All the following examples are correct.
+But this one provides, and root node will be `Product SRS` according to rule 1.
 
 ```
-# [r.1]
-{{parent: r; skip_meta: true}}
-
-# [r.2]
-{{parent: r
-skip_meta: true}}
-
-# [r.3]
-{{
-parent: r
-skip_meta: true
-}}
+# Product SRS
+## User requirements
+## Functional requirements
 ```
 
-The next attributes are **system attributes** and these influence to Clerq behavior:
-
-* `order_index: feature1 feature2` will sort child requirements in provided order;
-* `parent: f` will place the requirement as a child of parent requirement `[f]`.
+The QueryAssembly.call(query) follow similar logic
 
-All other attributes (`status`, `source`, etc.) are **user attributes** and do not influence Clerq behavior. These attributes are holding in requirement's attributes and can be used for publishing or automation purpose. One of such `skip_meta: true` used producing output document.
-
-#### Assets
-
-If you need to add an image or some other material into a requirements body, put it in the `bin/assets` folder and specify the link
-
-```markdown
-# [ent] Entities
-
-... the following picture shows conceptual entity relations diagram
-
-![Image](assets/er.png)
-```
+* When query result is empty, the Clerq will provide result with QueryNullNode (node.title == `Query`, node[:query] == QUERY_STRING)
+* When query result contains single node, it becomes a root query node.
+* When query result contains more than one, those becomes a child of root query node.
 
 ### Templating
 
-To customize Clerq output the system provides `erb` templates. You have a lot of possibilities here if you are familiar with Ruby. Clerq distributions provides two basic templates - `default.md.erb` and `pandoc.md.erb`.
-
-The first one combines all requirements to one document and there no requirements transformation actually.
+The Clerq provides the ability to precise adjusting the output for `clerq build` command by erb-templates and gives you two basic templates from the box.
 
-The second one has a lot of features and provides a good example of how to add new features in the requirements body. It handles three macros:
+* [default.md.erb](https://github.com/nvoynov/clerq/blob/master/lib/assets/tt/default.md.erb) that just combines all nodes to one markdown document;
+* [pandoc.md.erb](https://github.com/nvoynov/clerq/blob/master/lib/assets/tt/pandoc.md.erb) is more advanced, it produces [Pandoc's Markdown](https://pandoc.org/MANUAL.html#pandocs-markdown) and provides three followed macros for node body:
+   * `{{@@list}}` - replaces the macro with the list of child requirements;
+   * `{{@@tree}}` - replaces the macro with the tree of child requirements;
+   * `{{@@skip}}` - skip all content inside the brackets.
 
-* `{{@@list}}` - replaces the macro with the list of child requirements;
-* `{{@@tree}}` - replaces the macro with the tree of child requirements;
-* `{{@@skip}}` - skip all content inside the brackets.
+## Known issues
 
-`pandoc.md.erb` generates Pandoc markdown and can be used to convert the output of the template in any supported format. You also can use relative requirements links here. An example of using Pandoc to generate `docx` can be found in [Promo](#promo).
+### Thor version
 
-### Automating
+The one issue I certain in is when you are using different version of thor, your custom scripts won't work.
 
-You can and should extend the standard Clerq CLI by your own commands. See `<project>.thor` file as an example and call for action. It is all the Ruby code and the main point is to get requirements collection and then transform it to anything you want. It is the `Thor` gem that does all work related to CLI.
+## Some considerations
 
-You can find some examples of custom automating in [Promo](#promo), and I hope to provide some more examples in the future.
+### Some obvious things
 
-## Promo    
+Use modern text editor that provides projects tree. like Atom, Sublime, etc.
 
-The clerq provides the promotion project `promo` than contains requirements to the clerq. You can copy the `promo` content to the current clerq repository by `clerq promo` command.
+Hold your projects in Git
 
-The promo provides few specific commands that created to show you the way how to extend clerq. See `promo.thor` for details.
+Use pandoc for generating output in different formats
 
-## History
+### MarkupNode
 
-During 4 years work for my previous employer I participated in dozen software development and software reengineering projects and developed dozen bulk SRS and SAD documents. All those documents were developed in MS Word as a corporate standard format for all project documentation. And every of those often cause some headache usually with casual losses of style formatting or dead halt of Microsoft Word.
+Don't like the current dirty solution with templates and incorporated MarkupNode that does all that stuff with macro. It is the first attempt to provide template that can skipp comments
 
-When I left the employer, I decided to create a small toolkit for writing software documentation. The idea was to hold requirements in Yaml, each item in a separate file, and to provide reach features for effort estimation and prioritizing based on requirements. The first attempt was failed but brought some output. Throw out extra features! One item per file is a hell to author (but for a developer it was rather practical).
+### Several artifacts
 
-As the result of the first fail, the gem `Creq` was born. Exra features were thrown out. An author not limited to Yaml and uses Markdown now; he can write as many topics as he wants in one file. And for my new employer all requirements were developed in Creq. Some lessons were learn, some peculiarities were met, and then I decided to do some reengineering and developed Clerq. So here we are and now I have no claims to the subject of requirements writing and management.
+Because Clerq has `-q/--query QUERY_STRING` option you can be interested in developing several different artifacts in one project.
 
-* One Clerq project per one document!
-* Document versioning and authors collaboration are under Git!
-* Any text editor is suitable for requirements writing! Of course, markdown syntax highlighting feature is desired.
-* Single consistent requirements source and many templates for final documents (I like to left "TBD", "TODO" and other comment for myself, reviewer, and developers but it cannot be left in documents releases.)
-* All extra repeated job for certain project can be automated through Thor.
+I was considering such an example to develop all software project documents in one clerq project but decided that it is more properly to develop one single artifact per project because usually, each artifact has its own develop-review-release cycle.
 
-The last thing should be mentioned is drifting the original purpose of requirements management to compilation bulk documents from small pieces, and a requirement here is just a piece of text that can express just anything.
+Also, I was considering to add some kind of a "top" project that is just a wrapper for individual projects inside (each of them is the clerq project, and the top project just provides a specific set of commands.) I was speculating about some kind of shared content and tracing nodes between different artifacts. But for the moment I have no full-fledged vision.
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+The project is bundled, so after checking out the repo, run `bundle` to install dependencies. Then, run `bundle exec rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).