giblish 0.6.1 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8a6dbe01692a92bd2c55c61f756eb8971a256250c21f42e124b072c03af5ec03
-  data.tar.gz: 137d1b2688db5927bdb2841eb74d563adcfb503d4e7565dd71fa65a060baeb46
+  metadata.gz: c845d2e0c00221f3474469a435157b178894d2af53f8e316f4a555d9480643c6
+  data.tar.gz: a1e5204459599f4d270342d0e8bcf72ee4d36b8440b2ff8499b2428fc543972d
 SHA512:
-  metadata.gz: 34450a242e1ff5a5cd80793622a7c1c52b2e98f558b03ea3b83ba551af997d522d3c29bbfbdffc43089effd7b983f32f73fe7fc0ea1c2df070b7858f3a53ada8
-  data.tar.gz: 7ab48efc13679cb564d35b14a5f4843c7288e8e42db058aa460dd423af9ed164a3dd9f15d93995e667fabc7cadd8ce149320ac6927e94b8a30da9b6fde30b4eb
+  metadata.gz: 1c498054328dca32d192d16e7bbfea061417d7835e3ab80e86c998c1e705701cfef9d0ebdfd39cbfef23b954d56fb0d5e111b110def298fb96eddd7c6cecdd41
+  data.tar.gz: f8c52b147ff9a8134fb0829b69eb48cdade6518840253cf5e9b5ef66ea439a2b26d0d14a20b20a3b84b76c63408c59a52dab0363495f951da51d784d5b5b2335

data/Changelog

@@ -0,0 +1,16 @@
+= giblish changelog
+
+== v0.7.0
+
+ * *Breaking change* include the entire path in the -i and -j options
+ * *Breaking change* change the meaning of the -w flag (and rename the long form to --web-path)
+   and add the -mp flag.
+   This was done as a means to remove the hard-coded arguments to the search box html so that one
+   can generate html docs to a destination that is not the same as the one where they are deployed
+   on a web server.
+ * take the idprefix and idseparator into account when indexing sections for text search
+ * Add the font awesome css as link to search box and use the magnifying glass icon
+ * update README file
+ * handle the case where a 'index.adoc' file exists by making the basename configurable using the
+   '--index-basename' flag.
+ * update the dependencies to the latest asciidoctor-pdf release

data/README.adoc

@@ -1,5 +1,7 @@
 = giblish
-Generate docs from asciidoc files in a git repo
+:idseparator:-
+:idprefix:
+:numbered:
 
 image::https://travis-ci.org/rillbert/giblish.svg?branch=master["Build Status", link="https://travis-ci.org/rillbert/giblish"]
 
@@ -7,26 +9,31 @@ image::https://travis-ci.org/rillbert/giblish.svg?branch=master["Build Status",
 
 giblish is used to convert a source directory tree containing AsciiDoc files to
 a destination directory tree containing the corresponding html or pdf files
-and add some handy tools for easier navigation of the resulting files.
+and adds some handy tools for easier navigation of the resulting files.
 
-The tools include:
+An example of how giblish can generate the master branch of the official asciidoc.org
+documentation git repo can be found at http://www.rillbert.se/adoc/examples/adocorg/master/myindex.html
 
- * An index page listing all rendered documents with clickable links
+The added tools include:
+
+ * An index page listing all rendered documents with clickable links.
+ * A (stripped-down but nonetheless useful) text-search of your (html) documents (requires
+   that you view your docs via a web-server.
+ * If the source directory tree is part of a git repository, giblish can generate
+   separate html/pdf trees for branches and/or tags that match a user specified
+   regexp (see examples below).
  * Document ids - Note: the implementation of this is giblish-specific and thus
    you need to render your adoc files using giblish to make this work as intended.
    You can use document ids to:
  ** Reference one doc in the source tree from another doc without depending on file
-    names or relative paths. The referenced doc can thus be moved within the sourc
+    names or relative paths. The referenced doc can thus be moved within the source
     tree or change its file name and the reference will still be valid.
  ** Validate doc id references during document rendering and thus be alerted to
     any invalid doc id references.
  ** Let giblish generate a clickable graph of all document references (requires
     graphviz and the 'dot' tool).
- * A (stripped-down but nonetheless useful) text-search of your documents (requires
-   that you view your docs via a web-server.
- * If the source directory tree is part of a git repository, giblish can generate
-   separate html/pdf trees for branches and/or tags that match a user specified
-   regexp (see examples below).
+
+NOTE: giblish shall be considered stable from a usage perspective, please report bugs to the issue tracker. It's API however, is to be considered alpha, meaning that future versions may break the current usage of cmd line flags, generated artifacts, styling or other functionality.
 
 == Dependencies and credits
 
@@ -37,168 +44,40 @@ Thank you @mojavelinux and others for making these brilliant tools available!!
 
  gem install giblish
 
-== Some random notes
+Want to get started straight away? Go directly to the <<usage_examples>>.
+
+=== Some caveats
 
 When using giblish for generating docs the following applies:
 
  * giblish *will overwrite* files with the same name in the destination directory.
- * make sure that the git working tree and index of the source git repo are clean
-   when generating docs from a git repo.
+ * giblish requires that the git working tree and index of the repo containing source
+   documents are clean when generating documentation.
  * giblish will make explicit check-outs of all the branches or tags that matches
    the selection criteria. The working dir of the source git repo will thus have
    the last branch that giblish checked-out as the current branch after doc
    generation.
 
-== Usage Examples
-
-.Get available options
-====
- giblish -h
-====
-
-.Giblish 'hello world'
-====
- giblish my_src_root my_dst_root
-
-The above will convert all .adoc or .ADOC files under the dir `my_src_root` to
-html and place the resulting files under the `my_dst_root` dir. An index page
-named `index.html` is generated in the `my_dst_root` dir containing links and
-some info about the converted files.
-
-The default asciidoctor css will be used in the html conversion.
-====
-
-.Using a different css
-====
- giblish -r ./path/to/my/resources -s mylayout my_src_root my_dst_root
-
-The above will convert all .adoc or .ADOC files under the dir `my_src_root` to
-html and place the resulting files under the `my_dst_root` dir. An index page
-named `index.html` is generated in the `my_dst_root` dir containing links and
-some info about the converted files.
-
-A css named mylayout.css must be found in the dir
-`<working_dir/path/to/my/resources/css`. The resulting html files will link
-to this css. Fonts and images used from the css must be found under
-`<working_dir/path/to/my/resources/fonts` and
-`<working_dir/path/to/my/resources/images` respectively.
-====
-
-.Generate html from multiple git branches
-====
- giblish -g "feature" my_src_root my_dst_root
-
-The above will check-out all branches matching the regexp "feature" and convert
-the .adoc or .ADOC files under the dir `my_src_root` to html and place the
-resulting files under the `my_dst_root/<branch_name>` dir.
-
-An index page named `index.html` is generated in each `my_dst_root/<branch_name`
-dir containing links and some info about the converted files.
-
-A summary page containing links to all branches will be generated directly in
-the `my_dst_root` dir.
-====
-
-.Generate html from giblish git repo using giblish css
-====
-Assuming you have cloned this git repo to `~/github/giblish` you can do:
-
- giblish -g "master" -r ~/github/giblish/resources ~/github/giblish my_dst_root
-
-The above will check-out all branches matching the regexp "master" and convert
-the .adoc or .ADOC files under the dir `my_src_root` to html and place the
-resulting files under the `my_dst_root/<branch_name>` dir.
-
-An index page named `index.html` is generated in each `my_dst_root/<branch_name`
-dir containing links and some info about the converted files.
-
-A summary page containing links to all branches will be generated directly in
-the `my_dst_root` dir.
-====
-
-.Generate pdf from giblish git repo using the giblish pdf theme
-====
-Assuming you have cloned this git repo to `~/github/giblish` you can do:
-
- giblish -f pdf -g "master" -r ~/github/giblish/resources ~/github/giblish my_dst_root
-
-The above will check-out all branches matching the regexp "master" and convert
-the .adoc or .ADOC files under the dir `my_src_root` to pdf and place the
-resulting files under the `my_dst_root/<branch_name>` dir.
-
-An index page named `index.pdf` is generated in each `my_dst_root/<branch_name`
-dir containing links and some info about the converted files.
-
-A summary page containing links to all branches will be generated directly in
-the `my_dst_root` dir.
-====
-
-.Advanced usage; Publish a static html site from a git repo with search capabilities
-====
-giblish can be used to inject a tree of html docs suitable for serving via a web
-server (e.g. Apache). Below is an example how to create such a tree. If you
-combine this with a server side git hook that invokes this script after push,
-you will have a way of auto publish your latest documents and/or documents at
-specific git tags. A document management system including nice index pages and
-text search capabilities
-
-Assumptions:
-
- * You have a running web server that serves pages from directory root
-   `/var/www/html`
- * You want to access the generated docs from http://your_web_site.com/proddocs
- * The git repo containing the source docs has its working dir at `~/gh/myrepo`
- * You only want to publish the documents in the subfolder `common/Documents` in
-   your git repo.
- * You want to use your own css named `mylayout.css` that internally references
-   fonts and images using relative paths.
- * You have the css and its referenced fonts and images in subfolders
-   of the git repo at `common/resources/css common/resources/fonts common/resources/images`
- * You want to publish the documentation as it looked for your release tags
-   myprod-v1.0-final, myprod-v2.0-final, ...
-
- giblish -m -t "-final$" -r ~/gh/myrepo/common/resources -s mylayout -w /var/www/html ~/gh/myrepo/common/Documents /var/www/html/proddocs
-
-The above will create a tree of html docs under `/var/www/html/proddocs`. Each
-tag will get its own subdir (e.g. `/var/www/html/proddocs/myprod_v1.0_final`).
-The css and referenced assets will be copied to a 'web_assets' dir for each
-subdir and also to the .../proddocs dir.
-
-The `-w` switch above will strip the `/var/www/html` from the css link so that
-the paths to the css will be correct in the context of the serving of the
-pages via the web server.
-
-The `-m` switch above will build a database (JSON file) with enough information
-to enable a cgi-script to provide a text-search capability for your users. The
-cgi-script must be located at http://your_web_site.com/cgi-bin/giblish-search.cgi
-and this gem provides a default implementation that you can copy from the .../lib
-folder to the correct destination.
-====
-
 == Text search implementation
 
 The text search enables a user to search for a text string and receive matching
 sections in the documentation tree.
 
-To make this work, three things are needed
-
- . the source text of all adoc files together with a JSON file that maps sections to
-   their line numbers. This 'search data' is collected by giblish when it generates the
-   html files to the destination directory. The JSON file and all adoc source files
-   are copied to a well-known place in the destination tree (see below).
- . an html form somewhere in the rendered pages that receives the user input (text string
-   and other parameters) to start a search.
- ** giblish injects such an html form in the generated index page when the user
-    specifies the '-m' switch.
- . a server side script that
- ** receive a text string to search for
- ** parses the search data for matches to the text string
- ** presents the result to the user
-
-This gem contains an example implementation of a server side script. It is implemented
-in ruby and uses 'grep' to parse the search data. It then generates a result page where
-each matching section is shown and when clicked, will navigate the user to that section
-in the corresponding document.
+giblish ties together the following three pieces to enable the text search:
+
+. the source text of all adoc files together with a JSON file that maps sections to
+their line numbers.
+** giblish collects this 'search data' when it generates the
+html files to the destination directory. The JSON file and all adoc source files
+are copied to a well-known place in the destination tree (see below).
+. an html form somewhere on the rendered pages where the user can input search queries and
+  initiate a search.
+** giblish injects such an html form in the generated index page when the user
+specifies the '-m' switch.
+. a server side script that handles a user request to search the documents for a specific
+  text string and presents the result to the user.
+** this gem contains an implementation of such a server side script. It is intended to be
+   run as a cgi script and requires ruby and grep to be installed on the server where it runs.
 
 === Search data and html form parameters
 
@@ -225,9 +104,6 @@ root. This is illustrated below.
  |     |- branch_2_top_dir
  |           | ...
 
-assume that the file tree looks like this when not
-rendering a git branch:
-
 .When rendering documents not in a git branch
  dst_root_dir
  |- index.html
@@ -243,13 +119,270 @@ rendering a git branch:
  |     |   |- file2.adoc
  |     |- ...
 
-The parameters that is sent to the cgi script via the html form generated
-by giblish are:
+== Document ids and the reference graph
+
+NOTE: This is a non-standard extension of asciidoc. If you use this feature, you will
+need to generate your documents using giblish to make this work as intended.
+
+giblish extends the cross reference concept in asciidoc with a _document id_ mechanism.
+To use this, you need to:
+
+ . Add a `:docid:` entry in your document's header section. The doc id can consist of
+   up to 10 characters and must be unique within the set of documents generated by
+   giblish.
+ . Refer to a document using the syntax pass:[<<:docid:#DOC_ID#>>].
+ . Run giblish with the -d switch when generating documents.
+
+Using doc ids makes it possible for giblish to do two things:
+
+ . Make the reference from one document to another work even if one of the documents
+   have been moved within the source tree.
+ . Produce a clickable 'map' of the generated documents where the different references
+   are clearly seen (this feature require that the 'dot' tool, part of the graphwiz package
+   is installed on the machine where giblish is run).
+
+The use of the -d switch makes giblish parse the document twice, once to map up the doc ids and
+all references to them, once to actually generate the output documentation. Thus, you pay a
+performance penalty but this should not be a big inconvenience since the generation is quite
+fast in itself.
+
+=== Example of using the docid feature
+
+Consider that you have two documents located somewhere in the same folder tree, document one and
+document two. You could then use the docid feature of giblish to refer to one document from the
+other as in the example below.
+
+Example document one::
+
+[source,asciidoc]
+----
+= Document one
+:toc:
+:numbered:
+:docid: D-001
+
+== Purpose
+
+To illustrate the use of doc id.
+----
+
+Example document two::
+[source,asciidoc]
+----
+= Document two
+:toc:
+:numbered:
+:docid: D-002
+
+== Purpose
+
+To illustrate the use of doc id. You can refer to document one as <<:docid:D-001>>.
+This will display a clickable link with the doc id (D-001 in this case).
+
+You can basically follow the same syntax as the normal asciidoc cross-ref, such as:
+
+ * <<:docid:D-002#purpose>> to refer to a specific section or anchor.
+ * <<:docid:D-002#purpose,The purpose section>> to refer to a specific section and
+   display a specific text for the link.
+----
+
+The above reference will work even if either document changes location or file name as long
+as both documents are parsed by giblish in the same run.
+
+[[usage_examples]]
+== Usage Examples
+
+Here follows a number of usages for giblish in increasing order of complexity.
+
+=== Get available options
+
+ giblish -h
+
+=== Giblish html 'hello world'
+
+ giblish my_src_root my_dst_root
+
+ * convert all .adoc or .ADOC files under the dir `my_src_root` to
+ html and place the resulting files under the `my_dst_root` dir.
+ * generate an index page named `index.html` that contains links and
+ some info about the converted files. The file is placed in the `my_dst_root` dir.
+
+The default asciidoctor css will be used in the html conversion.
+
+=== Giblish pdf 'hello world'
+
+ giblish -f pdf my_src_root my_dst_root
+
+ * convert all .adoc or .ADOC files under the dir `my_src_root` to
+pdf and place the resulting files under the `my_dst_root` dir.
+ * generate an index page named `index.pdf` that contains links and
+some info about the converted files. The file is placed in the `my_dst_root` dir.
+
+The default asciidoctor pdf theme will be used in the pdf conversion.
+
+=== Using a custom css for the generated html
+
+Generate html that can be browsed locally from file:://<my_dst_root>.
+
+ giblish -r path/to/my/resources -s mylayout my_src_root my_dst_root
+
+ * convert all .adoc or .ADOC files under the dir `my_src_root` to
+html and place the resulting files under the `my_dst_root` dir.
+ * generate an index page named `index.html` that contains links and
+some info about the converted files. The file is placed in the `my_dst_root` dir.
+ * copy the `css`, `fonts` and `images` directories found under
+   `<working_dir>/path/to/my/resources` to `my_dst_root/web_assets`
+ * link all generated html files to the css found at
+   `/web_assets/css/mylayout.css`
+
+
+=== Using a custom pdf theme for the generated pdfs
+
+ giblish -f pdf -r path/to/my/resources -s mylayout my_src_root my_dst_root
+
+ * convert all .adoc or .ADOC files under the dir `my_src_root` to
+ pdf and place the resulting files under the `my_dst_root` dir.
+ * generate an index page named `index.pdf` that contains links and
+ some info about the converted files. The file is placed in the `my_dst_root` dir.
+ * the generated pdf will use the theme found at
+   `<working_dir>/path/to/my/resources/themes/mylayout.yml`
+
+=== Generate html from multiple git branches
+
+ giblish -g "feature" my_src_root my_dst_root
+
+ * check-out each branch matching the regexp "feature" in turn
+ * for each checked-out branch,
+ ** convert the .adoc or .ADOC files under the dir `my_src_root` to html.
+ ** place the resulting files under the `my_dst_root/<branch_name>` dir.
+ ** generate an index page named `index.html` that contains links and
+    some info about the converted files. The file is placed in the
+    `my_dst_root/<branch_name` dir.
+ * generate a summary page containing links to a all branches and place it in
+   the `my_dst_root` dir.
+
+=== Publish the asciidoctor.org documents with text search
+
+giblish can be used to generate html docs suitable for serving via a web
+server (e.g. Apache). You can use the cgi script included in the giblish
+gem to provide text search capabilities.
+
+Here is an example of how to publish the docs in the official asciidoctor.org git
+repo to a web server, including index pages and text search.
+
+NOTE: No consideration has been taken to how permissions are set up on the web server.
+Just running the below commands as-is on e.g.a standard apache set-up will bail out
+with 'permission denied' errors.
+
+==== Generating the html documents
+
+The example assumes that you have one machine where you generate the documents and another
+machine that runs a web server to which you have ssh access.
+
+The generated docs shall be accessible via _www.example.com/adocorg/with_search_
+
+You need to provide two pieces of deployment info to giblish when generating the documents:
+
+ * the uri path where to access the deployed docs (/adocorg/with_search in this example)
+ * the path in the local file system on the web server where the search data can be accessed
+   (/var/www/html/site_1/adocorg/with_search in this example)
+
+The following commands will generate the asciidoctor.org documentation and deploy the result
+to the web server.
+
+ . Clone the asciidoctor doc repo to your development machine
++
+ git clone https://github.com/asciidoctor/asciidoctor.org.git
+
+ . Generate the html documentation with the correct deployment info
++
+ giblish -j '^.*_include.*' -m -mp /var/www/html/site_1/adocorg/with_search -w /adocorg/with_search -g master --index-basename "myindex" asciidoctor.org/docs ./generated_docs
++
+Explanation of the parameters and arguments::
+ * *pass:[-j '^.*_include.*']* - exclude everything in the __include_ directory. (the
+                                 asciidoctor.org repo stores partial docs here).
+ * *-m* - assemble the necessary search data to support text search and include this data as part of
+          the generated documents.
+ * *-mp /var/www/html/site_1/adocorg/with_search* - the file system path on the deployment machine where
+                                                    the search data is located
+ * *-w /adocorg/with_search* - the uri path to the deployed docs
+ * *-g master* - publish all git branches that matches the regexp 'master' (i.e. only the 'master'
+                 branch).
+ * *--index-basename "myindex"* - change the default name (index) that giblish uses for the generated
+                                  index page. This is needed since asciidoctor.org contains an
+                                  "index.adoc" file that would otherwise be overwritten by giblish.
+ * *asciidoctor.org/docs* - the root of the source document tree.
+ * *./generated_docs* - a temporary storage for the generated html docs on the local system.
++
+
+ . Copy the generated files to the web server
+
+ scp -r ./generated_docs rillbert@my.web.server.org:/var/www/adocorg/with_search/.
++
+
+
+==== Copy the text search script to the web server
+
+This only needs to be done once (or if a new version of giblish breaks the currently used API).
+
+IMPORTANT: The current version of giblish expects the script to be found at the URI path
+`/cgi-bin/giblish-search.cgi`. This might be customizable in future versions but is currently
+hard-coded. Thus, if your web-server serves pages at www.mywebsite.com, the search script must be
+accessible at www.mywebsite.com/cgi-bin/giblish-search.cgi
+
+ . Find the server side script that implements text search that is included with
+   giblish
+
+ gem which giblish
++
+
+In my case this returns `/var/lib/ruby/gems/2.4.0/gems/giblish-0.5.2/lib/giblish.rb`. This means that I will find the script in the same directory, i.e. `/var/lib/ruby/gems/2.4.0/gems/giblish-0.5.2/lib`.
+
+ . Copy the server side script to the /cgi-bin dir on the web server. In this example the cgi-bin dir
+   is configured to be /var/www/cgi-bin
++
+ scp /var/lib/ruby/gems/2.4.0/gems/giblish-0.5.2/lib/giblish-search.rb rillbert@my.web.server.org:/var/www/cgi-bin/giblish-search.cgi
+
+////
+
+If you
+combine this with a server side git hook that invokes this script after push,
+you will have a way of auto publish your latest documents and/or documents at
+specific git tags. A document management system including nice index pages and
+text search capabilities
+
+=== Generate html from giblish git repo using giblish css
+
+Assuming you have cloned this git repo to `~/github/giblish` you can do:
+
+ giblish -g "master" -r ~/github/giblish/resources ~/github/giblish my_dst_root
+
+The above will check-out all branches matching the regexp "master" and convert
+the .adoc or .ADOC files under the dir `my_src_root` to html and place the
+resulting files under the `my_dst_root/<branch_name>` dir.
+
+An index page named `index.html` is generated in each `my_dst_root/<branch_name`
+dir containing links and some info about the converted files.
+
+A summary page containing links to all branches will be generated directly in
+the `my_dst_root` dir.
+====
+
+.Generate pdf from giblish git repo using the giblish pdf theme
+====
+Assuming you have cloned this git repo to `~/github/giblish` you can do:
+
+ giblish -f pdf -g "master" -r ~/github/giblish/resources ~/github/giblish my_dst_root
+
+The above will check-out all branches matching the regexp "master" and convert
+the .adoc or .ADOC files under the dir `my_src_root` to pdf and place the
+resulting files under the `my_dst_root/<branch_name>` dir.
+
+An index page named `index.pdf` is generated in each `my_dst_root/<branch_name`
+dir containing links and some info about the converted files.
+
+A summary page containing links to all branches will be generated directly in
+the `my_dst_root` dir.
+====
+////
 
-  * searchphrase -> the text string to search for
-  * ignorecase -> wether to ignore case or not
-  * useregexp -> wether the searchphrase above is treated as a regexp or
-    ordinary text
-  * css -> the css file name to use when styling the search result page
-  * topdir -> the absolute path to the root of the generated document tree
-  * reltop -> <clarify this>