marko 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: f907c502f514d2cd27122f2c2d6acf4e96c066515075c7b8df69b39c3022ee85
+  data.tar.gz: 075f639071daab8f53d56fec821196514f8c614c58b43b1cbc2306fc616f9f24
+SHA512:
+  metadata.gz: fcdf1096aec13270c9a87ed229feda55c81e95de4073b7de264cb8b42dcd66ca2031f643bba63b40c9db2da2e83e945b817866589e4ecb3e2abebb04e2592ad3
+  data.tar.gz: df5ea3a5f9cdc378aee2bbe8292b3dc3d0e033da11a676a8fcaa5b5c294f614b0e23609ac6a633c79e1725ee9af462be1849a780ff4d1b45f1158ed417601f4e

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+## [Unreleased]
+
+## [0.1.0] - 2023-01-02
+
+- Initial release

data/Gemfile

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in marko.gemspec
+gemspec
+
+gem "rake", "~> 13.0"
+
+gem "minitest", "~> 5.0"

data/Gemfile.lock

@@ -0,0 +1,22 @@
+PATH
+  remote: .
+  specs:
+    marko (0.1.0)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    minitest (5.16.3)
+    rake (13.0.6)
+
+PLATFORMS
+  x64-mingw32
+  x86_64-linux
+
+DEPENDENCIES
+  marko!
+  minitest (~> 5.0)
+  rake (~> 13.0)
+
+BUNDLED WITH
+   2.4.1

data/README.md

@@ -0,0 +1,159 @@
+# Marko
+
+Marko is a markup compiler that builds a tree from separated sources and compiles it into a single deliverable artifact.
+
+Marko supplies a "docs-as-code" approach for compiling bulky software artifacts by providing source storage, original plain text markup, compiler templates, Ruby- and a command-line interface for assembling and compiling.
+
+Having assembled the artifact, it can be analyzed, enriched by extra data, etc.; it can serve as a source for deriving subdued artifacts.  
+
+I've applied the approach for dozens of artifacts for the last six years, mainly writing requirements in Markdown, analyzing quality, deriving estimation sheets, exporting deliverables with Pandoc, and automating some parts of my everyday work.
+
+## Installation
+
+Install the gem by executing:
+
+    $ gem install marko
+
+## Usage
+
+### Interface
+
+Marko provides just basic command-line interface for creating new projects and assembling artifacts - run `$ marko` to see the details.
+
+In addition to the standard CLI, Marko supplies you with Rakefile, that also serves as custom automation example. You can run `rake -T` to see available commands.
+
+To help you with task automation, Marko provides `Marko.assemble` for assembling and `Marko.compile` for compiling artifacts (you could already spot it inside Rakefile.) See [Automation](#automation) section for examples.
+
+### Structure
+
+`marko new PROJECT` command will create a new marko project inside the `PROJECT` directory with following structure:
+
+- [bin/](bin/) - output folder for `build`
+- [bin/assets/](bin/assets/) - assets folder
+- [src/](src/) - markup sources
+- [tt/](tt/) - templates for `build`
+- [marko.yml](marko.yml) - project configuration
+- [Rakefile](Rakefile) - Rake automation file
+- [README.md](README.md) - this file
+
+### Markup
+
+The basic and the only Marko entity is [TreeNode](#github-link) with `id`, `meta`, `body`, and `items` properties.
+
+And the primary activity is just writing source files consisting of the TreeNode, where the source actually just a regular Markdown with an optional metadata excerpt. All lines from `#` until the next `#` are considered TreeNode.
+
+Let's see it by example and assume one has a few separate sources `content.md`, `uc.signup.md`, and `uc.signin.md`.
+`content.md`
+
+```markdown
+# Overview
+# User Requirements
+## Use Cases
+{{id: uc, order_index: .signup .signin}}
+# Functional requirements
+```
+
+`uc.signup.md`
+
+```markdown
+# Sign-Up
+{{id: .signup, parent: uc}}
+
+body markup
+```
+
+`uc.signin.md`
+
+```markdown
+# Sign-In
+{{id: .signin, parent: uc}}
+
+body markup
+```
+
+These sources will be assembled in a single hierarchy as follows
+
+```markdown
+# Overview
+# User Requirements
+## Use Cases
+{{id: uc, order_index: .signup .signin}}
+### Sign-Up
+{{id: .signup, parent: uc}}
+
+body markup
+### Sign-In
+{{id: .signin, parent: uc}}
+
+body markup
+
+# Functional requirements
+```
+
+So all the assemblage magic is just linking TreeNode by using `id`, `parent`, and `order_index` attributes; where `id` and `parent` are just nodes identifiers, and `order_index` is just an array of identifiers that point out the order of getting `items`.
+
+### Metadata
+
+It was shown above how to provide hierarchy attributes by metadata excerpt `{{}}`. But you can also use the excerpt to provide your own attributes, like `source: John Doe`, `affects: some.other.thing`, etc.
+
+### Tree, IDs
+
+When you deal with trees in separated sources, to reference nodes you need identifiers. So when you write `id`, `parent` and `order_index` metadata - you actually deal with TreeNode Id, and it must be unique.  
+
+When one works on a simple parent -> child relationship, identifiers can be shortened by starting from `.`. In the example above `{{order_index: .signup .signin}}`, the parent will find its children by `/.signup$/` and `/.signin$/`; and besides, during the assembling phase those relative identifiers will be turned to full - `uc.signup` and `uc.signin`.
+
+Marko will generate a unique Id for each TreeNode when Id was not provided by the author.
+
+### Macros
+
+The TreeNode.body can include macros. The most helpful one is `[[reference.id]]` that will be substituted by well-formed markdown link `[<node.title>](#<node.url>)`. There are also `@@tree`, `@@list`, `@@todo`, and `@@skip` standard macros; and this list could be extended or shortened through building templates.  
+
+- `@@tree` substituted by the tree of references to all descendants of the current node, might be used for the table of contents;
+- `@@list` substituted by the list of references to node items;
+- `@@todo` will will skip text with the macro till the end of the line
+- `@@skip` will skip the text after the macro
+
+### Templates
+
+Marko uses templates placed under the `tt` folder to compile sources into artifacts. You can use and customize the default one or design your own for particular purposes. It's just pure ERB, where Marko enumerates TreeNodes and renders the node output.
+
+```
+<%= @node.header %>
+<%= @node.meta   %>
+<%= @node.body   %>
+```
+
+The `marko.yml` configuration file sets the building process's default template and other default values.
+
+```yml
+--- !ruby/struct:Marko::Artifact
+id: ed863484-243f-4d46-8012-4b148f8c2910
+title: Marko Artifact
+template: tt/artifact.md.tt
+filename: tt/marko-artifact.md
+```
+
+### Automation
+
+Following quick example will assemble tree, remove TreeNode with id == 'hint', and compile the tree. You can also see Rakefile for other examples.
+
+```ruby
+require 'marko'
+
+def do_remove_hint
+  tree = Marko.assemble
+  hint = tree.find_node('hint')
+  hint.orphan!
+  Marko.compile(tree: tree)  
+end
+```
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/marko.

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/test_*.rb"]
+end
+
+task default: :test

data/exe/marko

@@ -0,0 +1,20 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "marko"
+include Marko
+
+command = ARGV.shift
+case command
+when nil
+  CLI.banner
+when /(n|new)/
+  dir = ARGV.shift
+  CLI.punch(dir)
+when /(d|demo)/
+  CLI.punch_demo
+when /(c|compile)/
+  CLI.compile
+else
+  CLI.banner
+end

data/lib/assets/demo/README.md

@@ -0,0 +1,13 @@
+% Marko Demo Project
+
+# Overview
+
+This project is an example of developing bulky technical artifacts with Marko Markup Compiler. It also serves as a sandbox for testing and features development.
+
+# Usage
+
+Run `$ marko` to see command-line interface
+
+Run `$ rake marko:publish` to publish the aritfact
+
+Run `$ rake -T` to see other custom interface

data/lib/assets/demo/src/fr/assemble.md

@@ -0,0 +1,27 @@
+# Assemble artifact
+{{id: .assemble, parent: fr}}
+
+The system shall provide the function to assemble artifact.
+
+__Input__
+
+* NO
+
+__Output__
+
+Parameter Type     0..* Description
+--------- -------- ---- --------------
+tree      TreeNode 1    assembled tree
+
+__Flow__
+
+@@todo provide id for required functions and change steps with appropriate links
+
+1. get list of project sources
+2. parse sources buffering nodes and errors
+3. fail "parsing errors" if errors.any?
+4. assemble artifact from buffer of nodes
+5. generate and inject auto ids
+6. validate artifact buffering errors
+7. fail "tree errors" if errors.any?
+8. return artifact

data/lib/assets/demo/src/fr/compile.md

@@ -0,0 +1,25 @@
+# Compile artifact
+{{id: .compile, parent: fr}}
+
+The system shall provide the function to create deliverable artifact.
+
+__Input__
+
+Parameter Type     0..* Description
+--------- -------- ---- --------------
+tree      TreeNode 1    assembled tree
+template  String   1    ERB template
+filename  String   1    filename
+
+__Output__
+
+The output of the function must be well-formed deliverable with `filename` built on `template` parameter.
+
+__Flow__
+
+@@todo provide id for required functions and change steps with appropriate links
+
+1. `tree` = [[fr.assemble]] unless `tree`
+2. load template body from `template`
+3. backup `filename` to `<filename>~`
+4. render `template` for each node of `tree` into `filename`

data/lib/assets/demo/src/fr/markup.md

@@ -0,0 +1,111 @@
+# Source Markup
+{{id: .markup, parent: fr}}
+
+The main and only entity in the system is [[fr.treenode]]. The system shall provide the following abilities for the entity:
+
+@@list
+
+## Node Markup
+{{id: .node}}
+
+The system shall support the following node markup.
+
+```markdown
+# <title>
+{{\<meta\>}}
+\<body\>
+```
+
+Where:  
+
+- Each node starts from Markdown header mark `#`.
+- The header mark can be followed by node title.
+- The next line _might_ contain metadata block:
+   - that starts from `{{` and finishes with `}}`;
+   - that can be one or multiline string.
+- The next lines tile the end of file or next header mark `#` are considered as node body.
+
+## Get Sources
+
+The system shall provide function to getting all sources files from project repository.
+
+@@todo define project repository
+
+## Parse Source
+
+The system shall provide the function to parse source file.
+
+During the parsing process the system must record source information within the node parsed, such as origin file name and the number of line inside the origin where the node begins. This information must be stored inside metadata as [[fr.treenode.orig]].
+
+__Input__
+
+Parameter Type   0..* Description
+--------- ------ ---- -----------
+filename  String 1    filename
+
+__Output__
+
+Parameter Type   0..* Description
+--------- ------ ---- -----------
+node      Node   0..n node parsed
+
+## Assemble Tree
+
+The system shall provide the function to assemble the artifact tree. The artifact tree is assembled based on [[fr.treenode.tree]].
+
+@@todo The assemblage algorithm, errors handler
+
+__Input__
+
+Parameter Type   0..* Description
+--------- ------ ---- -----------
+node      Node   0..n node array
+
+__Output__
+
+Parameter Type   0..* Description
+--------- ------ ---- -----------
+node      Node   1    root tree
+
+## Inject Id
+
+The system shall provide each node with unique node Id.
+
+Some nodes can already have ids from source file, especially those that referenced as parent or child and placed in separate files. For those nodes that still have empty id, the system must generate auto id 0..99.
+
+For example, when the system has assembled the tree
+
+```markdown
+# Artifact
+## Introduction
+### Purpose
+### Scope
+# User Requirements #ur
+# Functional Requirements #fr
+```
+
+and then generated id, the generated ids should be as follows:
+
+```markdown
+# Artifact Title #00
+## Introduction #01.01
+### Purpose #01.01.01
+### Scope #01.01.02
+# User Requirements #ur
+# Functional Requirements #fr
+```
+
+__Input__
+
+Parameter Type   0..* Description
+--------- ------ ---- -----------
+root      Node   1    root node
+
+## Checking Tree
+
+The system shall provide the function to check assembled tree for errors related to assembling tree based on [[fr.treenode.tree]]. The system must check the following errors:
+
+- `duplicate id`, finds two or more nodes that share the same id;
+- `unknown parent`, finds nodes that have `parent` metadata, but parent not found in the tree;
+- `unknown index`, finds nodes with `order_index` metadata where one or more children in the index not found;
+- `unknown links`, finds nodes containing links but the links are not found.

data/lib/assets/demo/src/fr/storage.md

@@ -0,0 +1,16 @@
+# Sources Storage
+{{id: .storage, parent: fr}}
+
+The system shall provide a storage for artifact sources. The storage must provide the following features:
+
+- Create a new project
+- Remove project
+- Get the list of project sources
+- Get project source
+
+@@skip
+
+At this stage only option will be developed - the filesystem storage. Other options that should be considered for future stages are
+
+- Database storage (SQL and NoSQL)
+- Confluence Wiki

data/lib/assets/demo/src/fr/treenode.md

@@ -0,0 +1,34 @@
+# TreeNode
+{{id: .treenode, parent: fr}}
+
+The system shall provide `Node` entity with the following properties that provide the following attributes:
+
+Property  Type   *..n Default Description
+--------- ------ ---- ------- -----------
+id        String 1    ""      Node identifier
+parent    Node   1    null    Parent node reference
+title     String 1    ""      Node title
+meta      Hash   1    {}      Node metadata
+body      String 1    ""      Node body
+items     Node   0..n []      Child node reference
+
+## Tree Metadata
+{{id: .tree}}
+
+To assemble the project artifact from nodes placed among several source files, the system shall provide the following optional tree metadata attributes:
+
+Attribute   Type   0..n Description
+----------- ------ ---- --------------
+id          String 0..1 Unique node id
+parent      String 0..1 Parent id
+order_index String 0..1 Children ids delimited by space
+
+## Source Metadata
+{{id: .orig}}
+
+During source file parsing, the system must store the following node origin information:
+
+Attribute   Type   0..n Description
+----------- ------ ---- --------------
+origin      String 1    Source filename
+lineno      String 1    Number of line

data/lib/assets/demo/src/index.md

@@ -0,0 +1,34 @@
+# Intro
+{{id: intro}}
+
+# Users
+{{id: usr}}
+
+The users of the system are different people who play for authoring various sorts of technical documentation. It might be a technical writer, business/systems analyst, developer, etc.
+
+# User Requirements
+{{id: ur, order_index: uc}}
+
+## Use Cases
+{{id: uc, order_index: .create.project .general.flow}}
+
+# Functional Requirements
+{{id: fr, order_index: .treenode .markup .storage .assemble .compile}}
+
+# Interface Requirements
+{{id: ui, order_index: .cli .gem}}
+
+# Assumptions and Dependencies
+{{id: as}}
+
+##
+
+The system requires a user interface for managing markup sources. It is assumed that users utilize any text editor of their choice.
+
+##
+
+The system requires tools for collaboration on artifacts by several authors simultaneously. Because of the plain text nature of the markup sources, It is assumed using Git for managing the artifact sources repository.
+
+##
+
+The system requires the compilation of artifacts into deliverables in form of well-supported formats like pdf, doc, etc. It is assumed using Pandoc for creating deliverables.

data/lib/assets/demo/src/intro.md

@@ -0,0 +1,98 @@
+# Purpose
+{{parent: intro}}
+
+The main purpose of this document is to provide a comprehensive demo project for Marko gem. The other technical purpose is to have Marko Sandbox for testing and development.
+
+# Problem
+{{parent: intro}}
+
+There are a few alternatives for authors who work on bulky software artifacts like requirements specification. They can apply one of the following tools
+
+@@todo find a few popular requirements management tools
+@@todo find a few popular Tex alternatives
+
+- a particular requirements management tool, like Doors
+- a Word Processor, like Microsoft Word, Libra Office, or Google Docs;
+- an elaborate publishing system, like Tex;
+- or just using a simple Wiki system, like Confluence or Redmine.
+
+A requirements management tool might seem to be the best choice there because it is tailored exactly for the required process. But it certainly will be a "hard way" from a cost and time perspective, an elaborate and expensive solution requiring personnel to be trained and vendor support.
+
+Although deliverable artifacts could be seen as a usual document structured and expressed in headers and paragraphs, the meaning of each of those can be different. So word processors and wiki systems might be the perfect choice for presenting artifact deliverables, but they lack of semantic fails for content development and management. The best of them provide a scripting language for document processing, but it is usually too complicated and again lacks particular entities' semantics (stakeholders, requirements, constraints, etc.)
+
+Designing documents of hundreds of pages in word processors brings some peculiar drawbacks. The software tends to become "bulky" in operating, consuming too many system resources and responding with delays; tend to be fragile with styles.
+
+__The problem of__ the lack of simple tools and approaches for writing software artifacts __affects__ technical writers who develop and manage the artifacts __the impact of which is__ they tend to choose a publishing tool or a word processor that does not fit well to work on software artifacts causing lots of manual work and maybe other confusing things like "bulky" documents __a successful solution would be__ the tool that will provide
+
+- a plain text markup for writing artifact items, that will be easy to read for humans, and processed by machines;  
+- a "semantics" layer to distinguish artifact items (actors, requirements, etc.) from supported text
+- the ability to automate tasks based on processing content according to its semantic value;
+- the ability to build a convenient presentation of the artifact (pdf, doc, odt);
+- will not depend on OS platform and any proprietary format or software.
+
+@@skip
+
+It provides a statement summarizing the problem being solved by the project.
+
+__The problem of__ [describe the problem]
+__affects__ [the stakeholders affected by the problem]
+__the impact of which is__ [what is the impact of the problem?]
+__a successful solution would be__ [list some key benefits of a successful solution]
+
+# Product
+{{parent: intro}}
+
+@@todo redesign the product statement for "who" section
+
+__For__ authors of software artifacts __who__ need a reliable and repeatable process of managing big software artifacts __the__ Marko Markup Compiler is a free open source software __that__ brings a "docs-as-code" approach with efficient plain markup for artifact sources, @@todo templates, CLI, Rake ..
+
+@@skip
+
+__For__ [target customer]
+__Who__ [statement of the need or opportunity]
+__The__ [product name] is a [product category]
+__That__ [key benefit, compelling reason to buy]
+__Unlike__ [primary competitive alternative]
+__Our product__ [statement of primary differentiation]
+
+# Scope
+{{parent: intro}}
+
+The developing system will consist of
+
+- the simple plain markup for writing the artifact sources;
+- the repository of artifact sources;
+- the algorithm for assembling the artifact from sources;
+- the markup for templates for publishing the artifact;
+- the command line interface for compiling the artifact into deliverables based on the templates  
+- the Ruby Gem that presents the artifact as the collection o items that can be easily visited for tasks automation;
+- the demo project that will help the customers to adopt the approach and design their own solutions based on the approach.
+
+# Definitions
+{{parent: intro}}
+
+CLI
+
+:   Command-line interface
+
+ERB
+
+:   Ruby Templating system
+
+# References
+{{parent: intro}}
+
+1. [Markdown Guide](https://www.markdownguide.org/)
+2. [Pandoc User's Guide](https://pandoc.org/MANUAL.html)
+3. [Git User's Manual](https://git-scm.com/docs/user-manual.html)
+
+# Overview
+{{parent: intro}}
+
+The remaining sections of this document provide user- and functional requirements of the system.
+
+The [[usr]] chapter introduces users of the system.
+
+The next chapter [[ur]] introduces users requirements that establish the context for the functional requirements.
+
+The following chapter [[fr]] describes detailed requirements for functions and user interfaces.

data/lib/assets/demo/src/ui/cli.md

@@ -0,0 +1,26 @@
+# Command-line interface
+{{id: .cli, parent: ui}}
+
+The system shall provide a command-line interface. The interface must provide the following commands:
+
+@@list
+
+## Create a new project
+
+The system shall provide the `new PROJECT` command. When the user requests the `new PROJECT` command, the system
+
+- ensures that `PROJECT` folder does not exist or fails
+- creates `PROJECT` folder and copies required files for new project
+- reports the user about success  
+
+## Compile artifact
+
+The system shall provide the `compile [-t TEMPALTE] [-o FILENAME]` command. When the user requests the `compile` command, the system
+
+- ensures that current directory is `Marko Project` directory, or fails
+- load `template`,
+  - load from the `TEMPLATE` parameter when provided
+  - or load default template when not
+- creates `tree` by using [[fr.assemble]]
+- compiles the tree by [[fr.compile]].(`tree`, `template`, `filename`)
+- reports the user about success

data/lib/assets/demo/src/ui/gem.md

@@ -0,0 +1,14 @@
+# Gem interface
+{{id: .gem, parent: ui}}
+
+The system shall provide Ruby Gem with the following functions
+
+@@list
+
+## Marko.assemble
+
+@@todo define Marko.assemble
+
+## Marko.compile
+
+@@todo define Marko.compile

data/lib/assets/demo/src/ur/uc.create.project.md

@@ -0,0 +1,8 @@
+# Create a new project
+{{id: uc.create.project, parent: uc}}
+
+__Main Flow__
+
+1. The user requests the system to create a new project passing a project folder for new project.
+2. The system checks for project folder to be unique. When  folder with the same name exists already, the system reports it to the user and the scenario is finished.
+3. The system creates the project folder and furnish it with project structure.

data/lib/assets/demo/src/ur/uc.general.flow.md

@@ -0,0 +1,14 @@
+# General Clerq flow
+{{id: uc.general.flow, parent: uc}}
+
+__Prerequisites__
+
+1. The user inside Marko project (see [[uc.create.project]])
+
+__Main Flow__
+
+1. The user creates and makes changes to the sources of the project. [outside the system]
+2. The user requests the system to `compile` the artifact.
+3. The system assembles the artifact from source files.
+4. The system checks the artifact and reports the errors to the user if any.
+5. The system returns the assembled artifact.