giblish 0.8.2 → 2.0.0.pre.alpha1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/unit_tests.yml +30 -0
- data/.gitignore +7 -3
- data/.ruby-version +1 -1
- data/Changelog.adoc +59 -0
- data/README.adoc +261 -0
- data/docs/concepts/text_search.adoc +213 -0
- data/docs/concepts/text_search_im/cgi-search_request.puml +35 -0
- data/docs/concepts/text_search_im/cgi-search_request.svg +397 -0
- data/docs/concepts/text_search_im/search_request.puml +40 -0
- data/docs/concepts/text_search_im/search_request.svg +408 -0
- data/docs/howtos/trigger_generation.adoc +180 -0
- data/docs/{setup_server_assets → howtos/trigger_generation_im}/Render Documents.png +0 -0
- data/docs/{setup_server_assets → howtos/trigger_generation_im}/View Documents.png +0 -0
- data/docs/{setup_server_assets → howtos/trigger_generation_im}/deploy_with_hooks.graphml +0 -0
- data/docs/{setup_server_assets → howtos/trigger_generation_im}/deploy_with_hooks.svg +0 -0
- data/docs/{setup_server_assets → howtos/trigger_generation_im}/deploy_with_jenkins.graphml +0 -0
- data/docs/{setup_server_assets → howtos/trigger_generation_im}/deploy_with_jenkins.svg +0 -0
- data/docs/howtos/trigger_generation_im/docgen_github.puml +51 -0
- data/docs/{setup_server_assets → howtos/trigger_generation_im}/giblish_deployment.graphml +0 -0
- data/docs/howtos/trigger_generation_im/post-receive-example.sh +50 -0
- data/docs/reference/box_flow_spec.adoc +22 -0
- data/docs/reference/search_spec.adoc +185 -0
- data/giblish.gemspec +54 -32
- data/lib/giblish/adocsrc_providers.rb +23 -0
- data/lib/giblish/application.rb +214 -41
- data/lib/giblish/cmdline.rb +273 -259
- data/lib/giblish/config_utils.rb +41 -0
- data/lib/giblish/configurator.rb +163 -0
- data/lib/giblish/conversion_info.rb +120 -0
- data/lib/giblish/docattr_providers.rb +125 -0
- data/lib/giblish/docid/docid.rb +181 -0
- data/lib/giblish/github_trigger/webhook_manager.rb +64 -0
- data/lib/giblish/gitrepos/checkoutmanager.rb +124 -0
- data/lib/giblish/{gititf.rb → gitrepos/gititf.rb} +30 -4
- data/lib/giblish/gitrepos/gitsummary.erb +61 -0
- data/lib/giblish/gitrepos/gitsummaryprovider.rb +78 -0
- data/lib/giblish/gitrepos/history_pb.rb +41 -0
- data/lib/giblish/indexbuilders/d3treegraph.rb +88 -0
- data/lib/giblish/indexbuilders/depgraphbuilder.rb +109 -0
- data/lib/giblish/indexbuilders/dotdigraphadoc.rb +174 -0
- data/lib/giblish/indexbuilders/standard_index.erb +10 -0
- data/lib/giblish/indexbuilders/subtree_indices.rb +132 -0
- data/lib/giblish/indexbuilders/templates/circles.html.erb +111 -0
- data/lib/giblish/indexbuilders/templates/flame.html.erb +61 -0
- data/lib/giblish/indexbuilders/templates/tree.html.erb +366 -0
- data/lib/giblish/indexbuilders/templates/treemap.html.erb +127 -0
- data/lib/giblish/indexbuilders/verbatimtree.rb +94 -0
- data/lib/giblish/pathtree.rb +473 -74
- data/lib/giblish/resourcepaths.rb +150 -0
- data/lib/giblish/search/expand_adoc.rb +55 -0
- data/lib/giblish/search/headingindexer.rb +312 -0
- data/lib/giblish/search/request_manager.rb +110 -0
- data/lib/giblish/search/searchquery.rb +68 -0
- data/lib/giblish/search/textsearcher.rb +349 -0
- data/lib/giblish/subtreeinfobuilder.rb +77 -0
- data/lib/giblish/treeconverter.rb +272 -0
- data/lib/giblish/utils.rb +142 -294
- data/lib/giblish/version.rb +1 -1
- data/lib/giblish.rb +10 -7
- data/scripts/hooks/post-receive.example +66 -0
- data/{docgen/scripts/githook_examples → scripts/hooks}/post-update.example +0 -0
- data/{docgen → scripts}/resources/css/adoc-colony.css +0 -0
- data/scripts/resources/css/giblish-serif.css +419 -0
- data/scripts/resources/css/giblish.css +1979 -419
- data/{docgen → scripts}/resources/fonts/Ubuntu-B.ttf +0 -0
- data/{docgen → scripts}/resources/fonts/Ubuntu-BI.ttf +0 -0
- data/{docgen → scripts}/resources/fonts/Ubuntu-R.ttf +0 -0
- data/{docgen → scripts}/resources/fonts/Ubuntu-RI.ttf +0 -0
- data/{docgen → scripts}/resources/fonts/mplus1p-regular-fallback.ttf +0 -0
- data/{docgen → scripts}/resources/images/giblish_logo.png +0 -0
- data/{docgen → scripts}/resources/images/giblish_logo.svg +0 -0
- data/{docgen → scripts}/resources/themes/giblish.yml +0 -0
- data/scripts/wserv_development.rb +32 -0
- data/web_apps/cgi_search/gibsearch.rb +43 -0
- data/web_apps/gh_webhook_trigger/config.ru +2 -0
- data/web_apps/gh_webhook_trigger/gh_webhook_trigger.rb +73 -0
- data/web_apps/gh_webhook_trigger/public/dummy.txt +3 -0
- data/web_apps/sinatra_search/config.ru +2 -0
- data/web_apps/sinatra_search/public/dummy.txt +3 -0
- data/web_apps/sinatra_search/sinatra_search.rb +34 -0
- data/web_apps/sinatra_search/tmp/restart.txt +0 -0
- metadata +188 -85
- data/.rubocop.yml +0 -7
- data/.travis.yml +0 -3
- data/Changelog +0 -16
- data/Gemfile +0 -4
- data/README.adoc +0 -1
- data/Rakefile +0 -41
- data/bin/console +0 -14
- data/bin/setup +0 -8
- data/data/testdocs/malformed/no_header.adoc +0 -5
- data/data/testdocs/toplevel.adoc +0 -19
- data/data/testdocs/wellformed/adorned_purpose.adoc +0 -17
- data/data/testdocs/wellformed/docidtest/docid_1.adoc +0 -24
- data/data/testdocs/wellformed/docidtest/docid_2.adoc +0 -8
- data/data/testdocs/wellformed/simple.adoc +0 -14
- data/data/testdocs/wellformed/source_highlighting/highlight_source.adoc +0 -38
- data/docgen/resources/css/giblish.css +0 -1979
- data/docgen/scripts/Jenkinsfile +0 -18
- data/docgen/scripts/gen_adoc_org.sh +0 -58
- data/docs/README.adoc +0 -387
- data/docs/setup_server.adoc +0 -202
- data/lib/giblish/buildgraph.rb +0 -216
- data/lib/giblish/buildindex.rb +0 -459
- data/lib/giblish/core.rb +0 -451
- data/lib/giblish/docconverter.rb +0 -308
- data/lib/giblish/docid.rb +0 -180
- data/lib/giblish/docinfo.rb +0 -75
- data/lib/giblish/indexheadings.rb +0 -251
- data/lib/giblish-search.cgi +0 -459
- data/scripts/hooks/post-receive +0 -57
- data/scripts/publish_html.sh +0 -99
data/docgen/scripts/Jenkinsfile
DELETED
@@ -1,18 +0,0 @@
|
|
1
|
-
pipeline {
|
2
|
-
agent any
|
3
|
-
stages {
|
4
|
-
stage('Render html documentation') {
|
5
|
-
steps {
|
6
|
-
echo "workspace path: ${env.WORKSPACE}"
|
7
|
-
// render html versions of all docs found under the 'docs' subdir
|
8
|
-
// in the repo
|
9
|
-
sh "giblish -n -f html -w ${env.WORKSPACE} -s giblish -r docgen/resources docs gendocs"
|
10
|
-
}
|
11
|
-
}
|
12
|
-
// stage('Render pdf documentation') {
|
13
|
-
// steps {
|
14
|
-
// sh "giblish -n -f pdf -s vironova-theme -r scripts/adocgen/resources Documents/MiniTEM/third_party_software/ MiniTEM/Deployment/doc"
|
15
|
-
// }
|
16
|
-
// }
|
17
|
-
}
|
18
|
-
}
|
@@ -1,58 +0,0 @@
|
|
1
|
-
#!/bin/bash
|
2
|
-
# generates the asciidoctor.org docs to a web server
|
3
|
-
# see usage function for usage
|
4
|
-
|
5
|
-
### configuration variables
|
6
|
-
|
7
|
-
# the deployment paths needed by giblish when generating the docs
|
8
|
-
uri_path="/adocorg/with_search"
|
9
|
-
deployment_dir="/var/www/rillbert_se/adoc/examples/adocorg"
|
10
|
-
|
11
|
-
# the cgi bin paths on the deployment server
|
12
|
-
cgi_dir="/var/www/cgi-bin"
|
13
|
-
|
14
|
-
### end of config section
|
15
|
-
|
16
|
-
function usage() {
|
17
|
-
echo ""
|
18
|
-
echo "Generates the asciidoctor.org docs to html using giblish and"
|
19
|
-
echo "deploys it to the server given by <ssh_host>."
|
20
|
-
echo ""
|
21
|
-
echo "NOTE: that you most likely need to tweak the deployment paths within this script"
|
22
|
-
echo "to your specific situation. See the doc within the script."
|
23
|
-
echo ""
|
24
|
-
echo "Usage: "
|
25
|
-
echo " gen_adoc_org <ssh_host>"
|
26
|
-
echo ""
|
27
|
-
echo " ssh_host the ssh host including user name (eg user1@my.webserver.com)"
|
28
|
-
echo ""
|
29
|
-
}
|
30
|
-
|
31
|
-
# check input variables
|
32
|
-
if [ $# -ne 1 ]; then
|
33
|
-
echo ""
|
34
|
-
echo "ERROR: Wrong number of input arguments !!!"
|
35
|
-
usage
|
36
|
-
exit 1
|
37
|
-
fi
|
38
|
-
|
39
|
-
# ssh_host="${ssh_user}@${web_server}:"
|
40
|
-
ssh_host="$1:"
|
41
|
-
html_doc_target="${ssh_host}${deployment_dir}"
|
42
|
-
cgi_bin_target="${ssh_host}${cgi_dir}"
|
43
|
-
|
44
|
-
|
45
|
-
# clone the asciidoctor repo
|
46
|
-
git clone https://github.com/asciidoctor/asciidoctor.org.git
|
47
|
-
|
48
|
-
# generate the html docs
|
49
|
-
giblish -j '^.*_include.*' -m -mp "${deployment_dir}" -w ${uri_path} -g master --index-basename "myindex" asciidoctor.org/docs ./generated_docs
|
50
|
-
|
51
|
-
# copy the docs to the web server
|
52
|
-
scp -r ./generated_docs/* "${html_doc_target}"
|
53
|
-
|
54
|
-
# get the path to the search script
|
55
|
-
search_script="$(dirname $(gem which giblish))/giblish-search.rb"
|
56
|
-
|
57
|
-
# copy the script to the cgi-bin on the web server
|
58
|
-
scp "${search_script}" "${cgi_bin_target}/giblish-search.cgi"
|
data/docs/README.adoc
DELETED
@@ -1,387 +0,0 @@
|
|
1
|
-
= giblish
|
2
|
-
:idseparator:-
|
3
|
-
:idprefix:
|
4
|
-
:numbered:
|
5
|
-
|
6
|
-
image::https://travis-ci.org/rillbert/giblish.svg?branch=master["Build Status", link="https://travis-ci.org/rillbert/giblish"]
|
7
|
-
|
8
|
-
== Purpose
|
9
|
-
|
10
|
-
giblish is used to convert a source directory tree containing AsciiDoc files to
|
11
|
-
a destination directory tree containing the corresponding html or pdf files
|
12
|
-
and adds some handy tools for easier navigation of the resulting files.
|
13
|
-
|
14
|
-
An example of how giblish can generate the master branch of the official asciidoc.org
|
15
|
-
documentation git repo can be found at https://www.rillbert.se/asciidoctor_docs/master/myindex.html
|
16
|
-
|
17
|
-
The added tools include:
|
18
|
-
|
19
|
-
* An index page listing all rendered documents with clickable links.
|
20
|
-
* A (stripped-down but nonetheless useful) text-search of your (html) documents (requires
|
21
|
-
that you view your docs via a web-server.
|
22
|
-
* If the source directory tree is part of a git repository, giblish can generate
|
23
|
-
separate html/pdf trees for branches and/or tags that match a user specified
|
24
|
-
regexp (see examples below).
|
25
|
-
* Document ids - Note: the implementation of this is giblish-specific and thus
|
26
|
-
you need to render your adoc files using giblish to make this work as intended.
|
27
|
-
You can use document ids to:
|
28
|
-
** Reference one doc in the source tree from another doc without depending on file
|
29
|
-
names or relative paths. The referenced doc can thus be moved within the source
|
30
|
-
tree or change its file name and the reference will still be valid.
|
31
|
-
** Validate doc id references during document rendering and thus be alerted to
|
32
|
-
any invalid doc id references.
|
33
|
-
** Let giblish generate a clickable graph of all document references (requires
|
34
|
-
graphviz and the 'dot' tool).
|
35
|
-
|
36
|
-
NOTE: giblish shall be considered stable from a usage perspective, please report bugs to the issue tracker. It's API however, is to be considered alpha, meaning that future versions may break the current usage of cmd line flags, generated artifacts, styling or other functionality.
|
37
|
-
|
38
|
-
== Dependencies and credits
|
39
|
-
|
40
|
-
Giblish uses the awesome *asciidoctor* and *asciidoctor-pdf* projects under the hood.
|
41
|
-
Thank you @mojavelinux and others for making these brilliant tools available!!
|
42
|
-
|
43
|
-
== Installation
|
44
|
-
|
45
|
-
gem install giblish
|
46
|
-
|
47
|
-
Want to get started straight away? Go directly to the <<usage_examples>>.
|
48
|
-
|
49
|
-
=== Some caveats
|
50
|
-
|
51
|
-
When using giblish for generating docs the following applies:
|
52
|
-
|
53
|
-
* giblish *will overwrite* files with the same name in the destination directory.
|
54
|
-
* giblish requires that the git working tree and index of the repo containing source
|
55
|
-
documents are clean when generating documentation.
|
56
|
-
* giblish will make explicit check-outs of all the branches or tags that matches
|
57
|
-
the selection criteria. The working dir of the source git repo will thus have
|
58
|
-
the last branch that giblish checked-out as the current branch after doc
|
59
|
-
generation.
|
60
|
-
|
61
|
-
== Text search implementation
|
62
|
-
|
63
|
-
The text search enables a user to search for a text string and receive matching
|
64
|
-
sections in the documentation tree.
|
65
|
-
|
66
|
-
giblish ties together the following three pieces to enable the text search:
|
67
|
-
|
68
|
-
. the source text of all adoc files together with a JSON file that maps sections to
|
69
|
-
their line numbers.
|
70
|
-
** giblish collects this 'search data' when it generates the
|
71
|
-
html files to the destination directory. The JSON file and all adoc source files
|
72
|
-
are copied to a well-known place in the destination tree (see below).
|
73
|
-
. an html form somewhere on the rendered pages where the user can input search queries and
|
74
|
-
initiate a search.
|
75
|
-
** giblish injects such an html form in the generated index page when the user
|
76
|
-
specifies the '-m' switch.
|
77
|
-
. a server side script that handles a user request to search the documents for a specific
|
78
|
-
text string and presents the result to the user.
|
79
|
-
** this gem contains an implementation of such a server side script. It is intended to be
|
80
|
-
run as a cgi script and requires ruby and grep to be installed on the server where it runs.
|
81
|
-
|
82
|
-
=== Search data and html form parameters
|
83
|
-
|
84
|
-
giblish will copy all search data to a 'search_assets' dir just under the destination
|
85
|
-
root. This is illustrated below.
|
86
|
-
|
87
|
-
.When rendering documents from a git branch
|
88
|
-
dst_root_dir
|
89
|
-
|- branch_1_top_dir
|
90
|
-
| |- index.html
|
91
|
-
| |- file_1.html
|
92
|
-
| |- dir_1
|
93
|
-
| | |- file2.html
|
94
|
-
|- branch_2_top_dir
|
95
|
-
|- branch_x_...
|
96
|
-
|- web_assets
|
97
|
-
|- search_assets
|
98
|
-
| |- branch_1_top_dir
|
99
|
-
| |- heading_index.json
|
100
|
-
| |- file1.adoc
|
101
|
-
| |- dir_1
|
102
|
-
| | |- file2.adoc
|
103
|
-
| |- ...
|
104
|
-
| |- branch_2_top_dir
|
105
|
-
| | ...
|
106
|
-
|
107
|
-
.When rendering documents not in a git branch
|
108
|
-
dst_root_dir
|
109
|
-
|- index.html
|
110
|
-
|- file_1.html
|
111
|
-
|- dir_1
|
112
|
-
| |- file2.html
|
113
|
-
|...
|
114
|
-
|- web_assets (only if a custom stylesheet is used...)
|
115
|
-
|- search_assets
|
116
|
-
| |- heading_index.json
|
117
|
-
| |- file1.adoc
|
118
|
-
| |- dir_1
|
119
|
-
| | |- file2.adoc
|
120
|
-
| |- ...
|
121
|
-
|
122
|
-
== Document ids and the reference graph
|
123
|
-
|
124
|
-
NOTE: This is a non-standard extension of asciidoc. If you use this feature, you will
|
125
|
-
need to generate your documents using giblish to make this work as intended.
|
126
|
-
|
127
|
-
giblish extends the cross reference concept in asciidoc with a _document id_ mechanism.
|
128
|
-
To use this, you need to:
|
129
|
-
|
130
|
-
. Add a `:docid:` entry in your document's header section. The doc id can consist of
|
131
|
-
up to 10 characters and must be unique within the set of documents generated by
|
132
|
-
giblish.
|
133
|
-
. Refer to a document using the syntax pass:[<<:docid:#DOC_ID#>>].
|
134
|
-
. Run giblish with the -d switch when generating documents.
|
135
|
-
|
136
|
-
Using doc ids makes it possible for giblish to do two things:
|
137
|
-
|
138
|
-
. Make the reference from one document to another work even if one of the documents
|
139
|
-
have been moved within the source tree.
|
140
|
-
. Produce a clickable 'map' of the generated documents where the different references
|
141
|
-
are clearly seen (this feature require that the 'dot' tool, part of the graphwiz package
|
142
|
-
is installed on the machine where giblish is run).
|
143
|
-
|
144
|
-
The use of the -d switch makes giblish parse the document twice, once to map up the doc ids and
|
145
|
-
all references to them, once to actually generate the output documentation. Thus, you pay a
|
146
|
-
performance penalty but this should not be a big inconvenience since the generation is quite
|
147
|
-
fast in itself.
|
148
|
-
|
149
|
-
=== Example of using the docid feature
|
150
|
-
|
151
|
-
Consider that you have two documents located somewhere in the same folder tree, document one and
|
152
|
-
document two. You could then use the docid feature of giblish to refer to one document from the
|
153
|
-
other as in the example below.
|
154
|
-
|
155
|
-
Example document one::
|
156
|
-
|
157
|
-
[source,asciidoc]
|
158
|
-
----
|
159
|
-
= Document one
|
160
|
-
:toc:
|
161
|
-
:numbered:
|
162
|
-
:docid: D-001
|
163
|
-
|
164
|
-
== Purpose
|
165
|
-
|
166
|
-
To illustrate the use of doc id.
|
167
|
-
----
|
168
|
-
|
169
|
-
Example document two::
|
170
|
-
[source,asciidoc]
|
171
|
-
----
|
172
|
-
= Document two
|
173
|
-
:toc:
|
174
|
-
:numbered:
|
175
|
-
:docid: D-002
|
176
|
-
|
177
|
-
== Purpose
|
178
|
-
|
179
|
-
To illustrate the use of doc id. You can refer to document one as <<:docid:D-001>>.
|
180
|
-
This will display a clickable link with the doc id (D-001 in this case).
|
181
|
-
|
182
|
-
You can basically follow the same syntax as the normal asciidoc cross-ref, such as:
|
183
|
-
|
184
|
-
* <<:docid:D-002#purpose>> to refer to a specific section or anchor.
|
185
|
-
* <<:docid:D-002#purpose,The purpose section>> to refer to a specific section and
|
186
|
-
display a specific text for the link.
|
187
|
-
----
|
188
|
-
|
189
|
-
The above reference will work even if either document changes location or file name as long
|
190
|
-
as both documents are parsed by giblish in the same run.
|
191
|
-
|
192
|
-
[[usage_examples]]
|
193
|
-
== Usage Examples
|
194
|
-
|
195
|
-
Here follows a number of usages for giblish in increasing order of complexity.
|
196
|
-
|
197
|
-
=== Get available options
|
198
|
-
|
199
|
-
giblish -h
|
200
|
-
|
201
|
-
=== Giblish html 'hello world'
|
202
|
-
|
203
|
-
giblish my_src_root my_dst_root
|
204
|
-
|
205
|
-
* convert all .adoc or .ADOC files under the dir `my_src_root` to
|
206
|
-
html and place the resulting files under the `my_dst_root` dir.
|
207
|
-
* generate an index page named `index.html` that contains links and
|
208
|
-
some info about the converted files. The file is placed in the `my_dst_root` dir.
|
209
|
-
|
210
|
-
The default asciidoctor css will be used in the html conversion.
|
211
|
-
|
212
|
-
=== Giblish pdf 'hello world'
|
213
|
-
|
214
|
-
giblish -f pdf my_src_root my_dst_root
|
215
|
-
|
216
|
-
* convert all .adoc or .ADOC files under the dir `my_src_root` to
|
217
|
-
pdf and place the resulting files under the `my_dst_root` dir.
|
218
|
-
* generate an index page named `index.pdf` that contains links and
|
219
|
-
some info about the converted files. The file is placed in the `my_dst_root` dir.
|
220
|
-
|
221
|
-
The default asciidoctor pdf theme will be used in the pdf conversion.
|
222
|
-
|
223
|
-
=== Using a custom css for the generated html
|
224
|
-
|
225
|
-
Generate html that can be browsed locally from file:://<my_dst_root>.
|
226
|
-
|
227
|
-
giblish -r path/to/my/resources -s mylayout my_src_root my_dst_root
|
228
|
-
|
229
|
-
* convert all .adoc or .ADOC files under the dir `my_src_root` to
|
230
|
-
html and place the resulting files under the `my_dst_root` dir.
|
231
|
-
* generate an index page named `index.html` that contains links and
|
232
|
-
some info about the converted files. The file is placed in the `my_dst_root` dir.
|
233
|
-
* copy the `css`, `fonts` and `images` directories found under
|
234
|
-
`<working_dir>/path/to/my/resources` to `my_dst_root/web_assets`
|
235
|
-
* link all generated html files to the css found at
|
236
|
-
`/web_assets/css/mylayout.css`
|
237
|
-
|
238
|
-
|
239
|
-
=== Using a custom pdf theme for the generated pdfs
|
240
|
-
|
241
|
-
giblish -f pdf -r path/to/my/resources -s mylayout my_src_root my_dst_root
|
242
|
-
|
243
|
-
* convert all .adoc or .ADOC files under the dir `my_src_root` to
|
244
|
-
pdf and place the resulting files under the `my_dst_root` dir.
|
245
|
-
* generate an index page named `index.pdf` that contains links and
|
246
|
-
some info about the converted files. The file is placed in the `my_dst_root` dir.
|
247
|
-
* the generated pdf will use the theme found at
|
248
|
-
`<working_dir>/path/to/my/resources/themes/mylayout.yml`
|
249
|
-
|
250
|
-
=== Generate html from multiple git branches
|
251
|
-
|
252
|
-
giblish -g "feature" my_src_root my_dst_root
|
253
|
-
|
254
|
-
* check-out each branch matching the regexp "feature" in turn
|
255
|
-
* for each checked-out branch,
|
256
|
-
** convert the .adoc or .ADOC files under the dir `my_src_root` to html.
|
257
|
-
** place the resulting files under the `my_dst_root/<branch_name>` dir.
|
258
|
-
** generate an index page named `index.html` that contains links and
|
259
|
-
some info about the converted files. The file is placed in the
|
260
|
-
`my_dst_root/<branch_name` dir.
|
261
|
-
* generate a summary page containing links to a all branches and place it in
|
262
|
-
the `my_dst_root` dir.
|
263
|
-
|
264
|
-
=== Publish the asciidoctor.org documents with text search
|
265
|
-
|
266
|
-
giblish can be used to generate html docs suitable for serving via a web
|
267
|
-
server (e.g. Apache). You can use the cgi script included in the giblish
|
268
|
-
gem to provide text search capabilities.
|
269
|
-
|
270
|
-
Here is an example of how to publish the docs in the official asciidoctor.org git
|
271
|
-
repo to a web server, including index pages and text search.
|
272
|
-
|
273
|
-
NOTE: No consideration has been taken to how permissions are set up on the web server.
|
274
|
-
Just running the below commands as-is on e.g.a standard apache set-up will bail out
|
275
|
-
with 'permission denied' errors.
|
276
|
-
|
277
|
-
==== Generating the html documents
|
278
|
-
|
279
|
-
The example assumes that you have one machine where you generate the documents and another
|
280
|
-
machine that runs a web server to which you have ssh access.
|
281
|
-
|
282
|
-
The generated docs shall be accessible via _www.example.com/adocorg/with_search_
|
283
|
-
|
284
|
-
You need to provide two pieces of deployment info to giblish when generating the documents:
|
285
|
-
|
286
|
-
* the uri path where to access the deployed docs (/adocorg/with_search in this example)
|
287
|
-
* the path in the local file system on the web server where the search data can be accessed
|
288
|
-
(/var/www/html/site_1/adocorg/with_search in this example)
|
289
|
-
|
290
|
-
The following commands will generate the asciidoctor.org documentation and deploy the result
|
291
|
-
to the web server.
|
292
|
-
|
293
|
-
. Clone the asciidoctor doc repo to your development machine
|
294
|
-
+
|
295
|
-
git clone https://github.com/asciidoctor/asciidoctor.org.git
|
296
|
-
|
297
|
-
. Generate the html documentation with the correct deployment info
|
298
|
-
+
|
299
|
-
giblish -j '^.*_include.*' -m -mp /var/www/html/site_1/adocorg/with_search -w /adocorg/with_search -g master --index-basename "myindex" asciidoctor.org/docs ./generated_docs
|
300
|
-
+
|
301
|
-
Explanation of the parameters and arguments::
|
302
|
-
* *pass:[-j '^.*_include.*']* - exclude everything in the __include_ directory. (the
|
303
|
-
asciidoctor.org repo stores partial docs here).
|
304
|
-
* *-m* - assemble the necessary search data to support text search and include this data as part of
|
305
|
-
the generated documents.
|
306
|
-
* *-mp /var/www/html/site_1/adocorg/with_search* - the file system path on the deployment machine where
|
307
|
-
the search data is located
|
308
|
-
* *-w /adocorg/with_search* - the uri path to the deployed docs
|
309
|
-
* *-g master* - publish all git branches that matches the regexp 'master' (i.e. only the 'master'
|
310
|
-
branch).
|
311
|
-
* *--index-basename "myindex"* - change the default name (index) that giblish uses for the generated
|
312
|
-
index page. This is needed since asciidoctor.org contains an
|
313
|
-
"index.adoc" file that would otherwise be overwritten by giblish.
|
314
|
-
* *asciidoctor.org/docs* - the root of the source document tree.
|
315
|
-
* *./generated_docs* - a temporary storage for the generated html docs on the local system.
|
316
|
-
+
|
317
|
-
|
318
|
-
. Copy the generated files to the web server
|
319
|
-
|
320
|
-
scp -r ./generated_docs rillbert@my.web.server.org:/var/www/adocorg/with_search/.
|
321
|
-
+
|
322
|
-
|
323
|
-
|
324
|
-
==== Copy the text search script to the web server
|
325
|
-
|
326
|
-
This only needs to be done once (or if a new version of giblish breaks the currently used API).
|
327
|
-
|
328
|
-
IMPORTANT: The current version of giblish expects the script to be found at the URI path
|
329
|
-
`/cgi-bin/giblish-search.cgi`. This might be customizable in future versions but is currently
|
330
|
-
hard-coded. Thus, if your web-server serves pages at www.mywebsite.com, the search script must be
|
331
|
-
accessible at www.mywebsite.com/cgi-bin/giblish-search.cgi
|
332
|
-
|
333
|
-
. Find the server side script that implements text search that is included with
|
334
|
-
giblish
|
335
|
-
|
336
|
-
gem which giblish
|
337
|
-
+
|
338
|
-
|
339
|
-
In my case this returns `/var/lib/ruby/gems/2.4.0/gems/giblish-0.5.2/lib/giblish.rb`. This means that I will find the script in the same directory, i.e. `/var/lib/ruby/gems/2.4.0/gems/giblish-0.5.2/lib`.
|
340
|
-
|
341
|
-
. Copy the server side script to the /cgi-bin dir on the web server. In this example the cgi-bin dir is configured to be `/var/www/cgi-bin`
|
342
|
-
+
|
343
|
-
scp /var/lib/ruby/gems/2.4.0/gems/giblish-0.5.2/lib/giblish-search.cgi rillbert@my.web.server.org:/var/www/cgi-bin/giblish-search.cgi
|
344
|
-
|
345
|
-
////
|
346
|
-
|
347
|
-
If you
|
348
|
-
combine this with a server side git hook that invokes this script after push,
|
349
|
-
you will have a way of auto publish your latest documents and/or documents at
|
350
|
-
specific git tags. A document management system including nice index pages and
|
351
|
-
text search capabilities
|
352
|
-
|
353
|
-
=== Generate html from giblish git repo using giblish css
|
354
|
-
|
355
|
-
Assuming you have cloned this git repo to `~/github/giblish` you can do:
|
356
|
-
|
357
|
-
giblish -g "master" -r ~/github/giblish/resources ~/github/giblish my_dst_root
|
358
|
-
|
359
|
-
The above will check-out all branches matching the regexp "master" and convert
|
360
|
-
the .adoc or .ADOC files under the dir `my_src_root` to html and place the
|
361
|
-
resulting files under the `my_dst_root/<branch_name>` dir.
|
362
|
-
|
363
|
-
An index page named `index.html` is generated in each `my_dst_root/<branch_name`
|
364
|
-
dir containing links and some info about the converted files.
|
365
|
-
|
366
|
-
A summary page containing links to all branches will be generated directly in
|
367
|
-
the `my_dst_root` dir.
|
368
|
-
====
|
369
|
-
|
370
|
-
.Generate pdf from giblish git repo using the giblish pdf theme
|
371
|
-
====
|
372
|
-
Assuming you have cloned this git repo to `~/github/giblish` you can do:
|
373
|
-
|
374
|
-
giblish -f pdf -g "master" -r ~/github/giblish/resources ~/github/giblish my_dst_root
|
375
|
-
|
376
|
-
The above will check-out all branches matching the regexp "master" and convert
|
377
|
-
the .adoc or .ADOC files under the dir `my_src_root` to pdf and place the
|
378
|
-
resulting files under the `my_dst_root/<branch_name>` dir.
|
379
|
-
|
380
|
-
An index page named `index.pdf` is generated in each `my_dst_root/<branch_name`
|
381
|
-
dir containing links and some info about the converted files.
|
382
|
-
|
383
|
-
A summary page containing links to all branches will be generated directly in
|
384
|
-
the `my_dst_root` dir.
|
385
|
-
====
|
386
|
-
////
|
387
|
-
|
data/docs/setup_server.adoc
DELETED
@@ -1,202 +0,0 @@
|
|
1
|
-
= Publish asciidoc docs in git repos using giblish
|
2
|
-
:imagesdir: setup_server_assets
|
3
|
-
:numbered:
|
4
|
-
:toc:
|
5
|
-
|
6
|
-
== Purpose
|
7
|
-
|
8
|
-
To describe how to use giblish as a tool for creating a static web site with documents that are automatically generated each time a contributer push changes to a git repo.
|
9
|
-
|
10
|
-
If you want to dive straight in and setup one of the scenarios below, jump to <<setup_instructions>>, otherwise read on for some examples and considerations.
|
11
|
-
|
12
|
-
=== Example usages
|
13
|
-
|
14
|
-
TBD
|
15
|
-
|
16
|
-
== Two deployment scenarios
|
17
|
-
|
18
|
-
This text covers two deployment scenarios; one using a git hook together with a simple (well) shell script, and one using a git hook together with Jenkins.
|
19
|
-
|
20
|
-
They have both been tested on Linux servers (Ubuntu). Most of the tools and scripts should work on a Windows server as well but might need some tweaking and is, again, not tested.
|
21
|
-
|
22
|
-
=== Git hook and shell script
|
23
|
-
|
24
|
-
This requires the least number of installed components to work. An overview of the scenario is given below.
|
25
|
-
|
26
|
-
.Deploy using git hook and shell script
|
27
|
-
image::deploy_with_hooks.svg[]
|
28
|
-
|
29
|
-
The components needed on the server are:
|
30
|
-
|
31
|
-
Master Repo::
|
32
|
-
The common (bare) git repo used by all content writers to push updates to. This repo needs to be setup with a server-side hook (_post-receive_) that are executed by git after each push to the repository.
|
33
|
-
|
34
|
-
Staging Repo::
|
35
|
-
A git repo that mirrors the _Master Repo_ and fulfills two functions:
|
36
|
-
|
37
|
-
. provide a checked-out working tree with the source files (adoc files).
|
38
|
-
. provide a script that uses giblish to publish the adoc files in the working tree as HTML files that reside at a place in the file system where the web server can access them.
|
39
|
-
|
40
|
-
Web Server::
|
41
|
-
Apache, Ngnix or other web server that provides clients with HTML pages located somewhere on the file system on the server (eg under /var/www/... )
|
42
|
-
|
43
|
-
==== Considerations
|
44
|
-
|
45
|
-
Possible advantages::
|
46
|
-
* it has few dependencies on external tools, only git, giblish and a web server are needed for this to work.
|
47
|
-
* giblish provides templates for both the `post-receive` hook and the `publish_html` script that can be tailored to your specific setup quite easily.
|
48
|
-
|
49
|
-
Possible drawbacks::
|
50
|
-
* The hook and publish scripts provided with giblish runs syncronously at each push from a _Doc Writer_ to the _Master Repo_. The time it takes to generate the HTML docs from the adoc source will thus be added to each push to the _Master Repo_.
|
51
|
-
* You need to manually setup the _Staging Repo_ on the server and this is a bit more 'hackish' than letting a build orchestrator tool implement a proper 'build' of your documents. You might for example end up with race conditions if two pushes to the _Master Repo_ are done close in time.
|
52
|
-
* The `publish_html` script provided with giblish only uses the currently checked-out branch on the _Staging Repo_ (typically 'master') to generate the HTML docs. It is non-trivial to add multiple-branch generation to the provided script.
|
53
|
-
|
54
|
-
=== Using Jenkins / git hook combo
|
55
|
-
|
56
|
-
This setup adds a Jenkins (or similar build orchestrator) installation so it is more tools to setup but offer more flexibility and performance. It is also more robust and thus more 'production friendly'.
|
57
|
-
|
58
|
-
If you are already doing some kind of development, chances are that you already have this kind of setup as a CI pipeline.
|
59
|
-
|
60
|
-
.Deploy using Jenkins
|
61
|
-
image::deploy_with_jenkins.svg[]
|
62
|
-
|
63
|
-
The components needed on the server are:
|
64
|
-
|
65
|
-
Master Repo::
|
66
|
-
The common (bare) git repo used by all content writers to push updates to. This repo needs to be setup with a server-side hook (_post-receive_) that are executed by git after each push to the repository.
|
67
|
-
|
68
|
-
Jenkins instance::
|
69
|
-
A running instance of Jenkins and one or more defined build jobs that use giblish to build the HTML documents.
|
70
|
-
|
71
|
-
Web Server::
|
72
|
-
Apache, Ngnix or other web server that provides clients with HTML pages located somewhere on the file system on the server (eg under /var/www/... )
|
73
|
-
|
74
|
-
==== Considerations
|
75
|
-
|
76
|
-
Possible advantages::
|
77
|
-
* Using Jenkins enables a lot of flexibility and scaleability. You can setup multiple jenkins agents to increase performance, you can define many build jobs where each job builds either a particular branch from a particular git repo or many branches from one or many repos.
|
78
|
-
|
79
|
-
Possible drawbacks::
|
80
|
-
* You need to be familiar with, and maintain, the Jenkins instance.
|
81
|
-
|
82
|
-
[[setup_instructions]]
|
83
|
-
== Setup instructions
|
84
|
-
|
85
|
-
Follow the instructions below to get this running on your server.
|
86
|
-
|
87
|
-
=== Some preliminary notes
|
88
|
-
|
89
|
-
Permissions and security::
|
90
|
-
These instructions does not give any advice on setting up permissions in a way that is suitable for your use case. You should however, give this an appropriate amount of consideration. You need to set the permissions so that the relevant tools and scripts can perform their expected tasks without constituting security vulnerabilities on your server. You might want to consider setting up specific user accounts/groups to achieve this but this is outside the scope of this text.
|
91
|
-
|
92
|
-
Server side git hooks::
|
93
|
-
git supports several hooks that are executed at different points in the sequence of committing and pushing changes to a repository. This feature is used in both scenarios below to provide the trigger for generating the HTML document. For details on git hooks see https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks[this doc].
|
94
|
-
|
95
|
-
[[common_setup]]
|
96
|
-
=== Initial setup
|
97
|
-
|
98
|
-
These steps are common to both deployment scenarios.
|
99
|
-
|
100
|
-
giblish setup::
|
101
|
-
. Install ruby on the _Server_ (a version no more than 18 months old)
|
102
|
-
. Install giblish on the _Server_ using `gem install giblish`
|
103
|
-
|
104
|
-
git and gitrepo setup::
|
105
|
-
. Install git on the _Server_.
|
106
|
-
. Setup the *bare* _Master Repo_ on the _Server_ by either.
|
107
|
-
.. Copy a bare repo you already use to the _Server_ file system.
|
108
|
-
.. Init a new repo using `git init --bare <your_repo_name>` somewhere on the _Server_ file system.
|
109
|
-
+
|
110
|
-
NOTE: If you start with an empty, bare, _Master Repo_ on the _Server_, it is a good idea to directly clone it to your local machine, commit some content to the 'master' branch and push the result back to the _Master Repo_. An empty, bare repo does not even contain a 'master' branch from the start and this can lead to some edge cases that complicate things.
|
111
|
-
|
112
|
-
. Setup the _Staging Repo_ on the _Server_ by cloning the _Master Repo_ to a suitable folder in the _Server_ file system, ex:
|
113
|
-
+
|
114
|
-
.Setup a staging repo in your home folder on the _Server_
|
115
|
-
====
|
116
|
-
cd ~
|
117
|
-
git clone file:///usr/local/master_repo.git
|
118
|
-
====
|
119
|
-
|
120
|
-
=== Setup the git hook and shell script scenario
|
121
|
-
|
122
|
-
First, follow the steps described in <<common_setup>>. Then proceed with the steps below.
|
123
|
-
|
124
|
-
script setup::
|
125
|
-
. Find out where the template scripts for the `post-receive` and the `publish_html.sh` scripts are located by running `gem which giblish`. Strip of the `lib/giblish.rb` from the end of the returned path and `cd` to the resulting folder. There should be a subfolder named `scripts` under which you can find the template scripts somewhere.
|
126
|
-
+
|
127
|
-
.Installation on Ubuntu 16.04
|
128
|
-
====
|
129
|
-
On a server running Ubuntu 16.04, `gem which giblish` returns:
|
130
|
-
|
131
|
-
/var/lib/gems/2.6.0/gems/giblish-0.7.0/lib/giblish.rb
|
132
|
-
|
133
|
-
and the template scripts can thus be found somewhere under
|
134
|
-
|
135
|
-
/var/lib/gems/2.6.0/gems/giblish-0.7.0/scripts
|
136
|
-
====
|
137
|
-
|
138
|
-
. Copy the `publish_html.sh` script to a suitable folder in your _local repo_ on your _local machine_ (e.g. under `<your_local_repo_root>/scripts`).
|
139
|
-
. Tweak the configuration variables in the script to fit your use case.
|
140
|
-
. Commit and push script to the _Master Repo_ on the _Server_.
|
141
|
-
. Copy the `post-receive` script to the `hooks` folder in your bare _Master Repo_.
|
142
|
-
. Tweak the configuration variables of the copy to suite your use case.
|
143
|
-
. Set the execute permission using `chmod +x post-receive`
|
144
|
-
|
145
|
-
stylesheet setup::
|
146
|
-
The template scripts assume that the the css used to style the generated HTML is located under `<repo_root>/scripts/resources/css` and is called `giblish.css`. To enable a styling of your liking, either create this folder/css file in your repo at that exact place or tweak the `html_publish.sh` script to use another location/css file name for the styling.
|
147
|
-
|
148
|
-
other tweaking::
|
149
|
-
You can tweak the `publish_html.sh` script to use other features of giblish/asciidoctor, such as docid's, docinfo support and others. To do this, change the flags and arguments used to invoke giblish from the `publish_html.sh` script accordingly.
|
150
|
-
|
151
|
-
=== Setup the git hook and Jenkins scenario
|
152
|
-
|
153
|
-
First, follow the steps described in <<common_setup>>. Then proceed with the steps below.
|
154
|
-
|
155
|
-
TBD
|
156
|
-
|
157
|
-
==== Sequence for generating documents
|
158
|
-
|
159
|
-
The following image shows how the sequence from user commit to generated documents.
|
160
|
-
|
161
|
-
.Sequence diagram for generating docs from adoc
|
162
|
-
image::Render Documents.png[]
|
163
|
-
|
164
|
-
[appendix]
|
165
|
-
== Scripts
|
166
|
-
|
167
|
-
.The giblish `publish_html.sh` template script
|
168
|
-
[%collapsible]
|
169
|
-
====
|
170
|
-
[source,bash]
|
171
|
-
----
|
172
|
-
include::../scripts/publish_html.sh[]
|
173
|
-
----
|
174
|
-
====
|
175
|
-
|
176
|
-
.The giblish `post-receive` template git hook
|
177
|
-
[%collapsible]
|
178
|
-
====
|
179
|
-
[source,bash]
|
180
|
-
----
|
181
|
-
include::../scripts/hooks/post-receive[]
|
182
|
-
----
|
183
|
-
====
|
184
|
-
|
185
|
-
=== post-update hook
|
186
|
-
|
187
|
-
Below is an example of a `post-update` hook that triggers Jenkins jobs after a push to a git repo. This hook should be installed on the server side git repository to trigger Jenkins builds
|
188
|
-
|
189
|
-
.Example of a git hook triggering Jenkins builds
|
190
|
-
[%collapsible]
|
191
|
-
====
|
192
|
-
[source,bash]
|
193
|
-
----
|
194
|
-
include::../docgen/scripts/githook_examples/post-update.example[]
|
195
|
-
----
|
196
|
-
====
|
197
|
-
|
198
|
-
=== Jenkins pipeline script
|
199
|
-
|
200
|
-
Below is a very basic example of a Jenkins pipeline that triggers giblish to render html and pdf documents located in a specific directory in a git repository.
|
201
|
-
|
202
|
-
TBD
|