ki-repo 0.1.0 → 0.1.1
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +26 -12
- data/VERSION +1 -1
- data/bin/ki +21 -0
- data/docs/backlog.md +35 -0
- data/docs/development.md +45 -0
- data/docs/development_setup.md +49 -0
- data/{lib/ki-repo.rb → docs/images/for_git.txt} +0 -0
- data/docs/ki_commands.md +202 -0
- data/docs/repository_basics.md +171 -0
- data/docs/writing_extensions.md +50 -0
- data/lib/cmd/cmd.rb +224 -0
- data/lib/cmd/user_pref_cmd.rb +122 -0
- data/lib/cmd/version_cmd.rb +483 -0
- data/lib/data_access/repository_finder.rb +200 -0
- data/lib/data_access/repository_info.rb +153 -0
- data/lib/data_access/version_helpers.rb +242 -0
- data/lib/data_access/version_iterators.rb +145 -0
- data/lib/data_access/version_operations.rb +80 -0
- data/lib/data_storage/dir_base.rb +106 -0
- data/lib/data_storage/ki_home.rb +44 -0
- data/lib/data_storage/ki_json.rb +153 -0
- data/lib/data_storage/repository.rb +91 -0
- data/lib/data_storage/version_metadata.rb +141 -0
- data/lib/ki_repo_all.rb +42 -0
- data/lib/util/attr_chain.rb +258 -0
- data/lib/util/exception_catcher.rb +118 -0
- data/lib/util/hash.rb +46 -0
- data/lib/util/hash_cache.rb +31 -0
- data/lib/util/ruby_extensions.rb +137 -0
- data/lib/util/service_registry.rb +88 -0
- data/lib/util/simple_optparse.rb +103 -0
- data/lib/util/test.rb +323 -0
- metadata +69 -13
- data/.document +0 -5
- data/.rspec +0 -1
- data/Gemfile +0 -14
- data/Gemfile.lock +0 -38
- data/Rakefile +0 -42
- data/spec/ki-repo_spec.rb +0 -6
- data/spec/spec_helper.rb +0 -12
data/README.md
CHANGED
@@ -1,18 +1,32 @@
|
|
1
1
|
# ki-repo
|
2
2
|
|
3
|
-
Repository for storing
|
3
|
+
Repository for storing packages and metadata.
|
4
4
|
|
5
|
-
|
6
|
-
|
7
|
-
* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
|
8
|
-
* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
|
9
|
-
* Fork the project.
|
10
|
-
* Start a feature/bugfix branch.
|
11
|
-
* Commit and push until you are happy with your contribution.
|
12
|
-
* Make sure to add tests for it.
|
13
|
-
* Please make your commit tidy.
|
5
|
+
note: Currently Ki is not ready for any kind of use.
|
14
6
|
|
15
|
-
|
7
|
+
# Links
|
16
8
|
|
17
|
-
|
9
|
+
* Rubygem: https://rubygems.org/gems/ki-repo
|
10
|
+
* Documentation for latest released version is available at http://rubydoc.info/gems/ki-repo
|
11
|
+
|
12
|
+
# Documentation
|
13
|
+
|
14
|
+
* {file:docs/repository_basics.md Repository basics} includes a simple tutorial and explains basic concepts.
|
15
|
+
* Ki command line utilies are documented in {file:docs/ki_commands.md}.
|
16
|
+
* {file:docs/development.md Development} provides additional development related information.
|
17
|
+
|
18
|
+
# Plan
|
19
|
+
|
20
|
+
1. Local repository features: create package, import, export, list, test, dependencies
|
21
|
+
2. Metadata support: statuses, file tagging
|
22
|
+
3. Repository cleanup and removal
|
23
|
+
4. Use script files in repository to add additional commands to ki
|
24
|
+
5. Documentation
|
25
|
+
|
26
|
+
Once these features are implemented Ki is ready for use on local server. In the future, the goal is to provide tools
|
27
|
+
to manage distributed repositories: downloads and replication.
|
28
|
+
|
29
|
+
# Copyright
|
30
|
+
|
31
|
+
Copyright (c) 2012 Mikko Apo. See {file:LICENSE.txt} for further details.
|
18
32
|
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
0.1.
|
1
|
+
0.1.1
|
data/bin/ki
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
# encoding: UTF-8
|
3
|
+
|
4
|
+
# Copyright 2012 Mikko Apo
|
5
|
+
#
|
6
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
7
|
+
# you may not use this file except in compliance with the License.
|
8
|
+
# You may obtain a copy of the License at
|
9
|
+
#
|
10
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
11
|
+
#
|
12
|
+
# Unless required by applicable law or agreed to in writing, software
|
13
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
14
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
15
|
+
# See the License for the specific language governing permissions and
|
16
|
+
# limitations under the License.
|
17
|
+
|
18
|
+
require_relative '../lib/ki_repo_all'
|
19
|
+
|
20
|
+
Ki::KiCommand.new.execute(ARGV)
|
21
|
+
|
data/docs/backlog.md
ADDED
@@ -0,0 +1,35 @@
|
|
1
|
+
# @title Ki: Backlog
|
2
|
+
|
3
|
+
# Backlog for 0.2
|
4
|
+
* fix defined? checks
|
5
|
+
* fix double repository entries in finder.all_repositories "should show imported version"
|
6
|
+
* cleanup gem
|
7
|
+
* create release notes
|
8
|
+
|
9
|
+
# 0.3
|
10
|
+
* cleanup and removal operations
|
11
|
+
* version() returns sometime nil, sometimes Version
|
12
|
+
* version!()
|
13
|
+
* fix Version.exists? - there should be a better way to check if version exists
|
14
|
+
* VersionTester should support test_version(metadata, source)
|
15
|
+
* Download & replication
|
16
|
+
* website command
|
17
|
+
* daemon: long running processes, web site monitoring
|
18
|
+
* multiple local repositories: command line tools and helpers
|
19
|
+
* support for separate binaries directory
|
20
|
+
* pref: define repository lookup order
|
21
|
+
|
22
|
+
# Future releases
|
23
|
+
* Digital signing
|
24
|
+
* Encrypted/packed packages
|
25
|
+
* Support for using files from other repositories
|
26
|
+
* when building version create file operations based on user's changes
|
27
|
+
|
28
|
+
# Maybe at some point...
|
29
|
+
* replace popen4.spawn with Kernel.spawn
|
30
|
+
* alias
|
31
|
+
* package dep operations: dep-rm, dep-mv, dep-cp, dep-switch
|
32
|
+
* named version lists for component my/component#released:Smoke=green
|
33
|
+
|
34
|
+
# Future backwards compatability issues
|
35
|
+
* how to store version directories so that per directory limits can be bypassed
|
data/docs/development.md
ADDED
@@ -0,0 +1,45 @@
|
|
1
|
+
Warning! Ki is currently not ready for public use. APIs, classes and functionality will change.
|
2
|
+
|
3
|
+
# Setting up development environment
|
4
|
+
|
5
|
+
Follow the instructions here: {file:docs/development_setup.md}
|
6
|
+
|
7
|
+
# Writing extensions
|
8
|
+
|
9
|
+
{file:docs/writing_extensions.md} describes how to extend Ki by writing extension scripts.
|
10
|
+
|
11
|
+
# Important Ruby classes
|
12
|
+
|
13
|
+
## Command line utilities
|
14
|
+
|
15
|
+
* {Ki::KiCommand} - Main ki entry point
|
16
|
+
* {Ki::KiCommandHelp}, {Ki::KiInfoCommand} - help, ki-info
|
17
|
+
* {Ki::UserPrefCommand} - Stored preferences
|
18
|
+
* {Ki::BuildVersionMetadataFile}, {Ki::TestVersion}, {Ki::ImportVersion}, {Ki::ExportVersion}, {Ki::VersionStatus}, {Ki::ShowVersion}, {Ki::VersionSearch} - repository management
|
19
|
+
|
20
|
+
## Data storage - files and directories
|
21
|
+
|
22
|
+
* {Ki::KiHome}, {Ki::Repository::Repository}, {Ki::Repository::Component}, {Ki::Repository::Version} - Repository directory objects
|
23
|
+
* {Ki::VersionMetadataFile}, {Ki::Dependency}, {Ki::VersionStatusFile} - Version metadata and statuses
|
24
|
+
* {Ki::DirectoryBaseModule}, {Ki::DirectoryBase} - Base classes for file and directory management
|
25
|
+
* {Ki::KiJSONFile}, {Ki::KiJSONListFile}, {Ki::KiJSONHashFile}, {Ki::KiJSONHashFile::CachedMapDataAccessor} - Base classes for JSON files
|
26
|
+
* {Ki::DirectoryWithChildrenInListFile} - Helper to generate list file class and related methods for a repository object
|
27
|
+
* {Ki::RepositoryMethods}, {Ki::RepositoryMethods::RepositoryListFile} - Repository helper methods
|
28
|
+
* {Ki::UserPrefFile} - Stored preferences
|
29
|
+
|
30
|
+
## Data access - finders, helpers, iterators
|
31
|
+
|
32
|
+
* {Ki::VersionTester}, {Ki::VersionImporter}, {Ki::VersionExporter} - Utility classes for managing version
|
33
|
+
* {Ki::RepositoryFinder} - Loads components from all available repositories and finds components and versions
|
34
|
+
* {Ki::VersionIterator} - Provides method to iterate through all component's matching versions
|
35
|
+
* {Ki::FileFinder} - Finds matchins files from a Version
|
36
|
+
* {Ki::Component}, {Ki::Version} - Classes that manage combined information for versions and components from all repositories
|
37
|
+
* {Ki::VersionFileOperations} - Helper class to process Version's file operations
|
38
|
+
|
39
|
+
## Utilities
|
40
|
+
|
41
|
+
* {Ki::Tester} - Helper class for tests, makes it easy to clear resources
|
42
|
+
* {AttrChain} - Chained accessor methods
|
43
|
+
* {ExceptionCatcher} - Execute multiple blocks and process exceptions in the end
|
44
|
+
* {Ki::ServiceRegistry} - Class for storing all Ki extensions
|
45
|
+
* {Ki::SimpleOptionParser} - Simplified OptionParser
|
@@ -0,0 +1,49 @@
|
|
1
|
+
# @title Ki: Setting up development environment
|
2
|
+
# Setting up development environment
|
3
|
+
|
4
|
+
## Install Ruby 1.9
|
5
|
+
|
6
|
+
Rvm makes it easy to use multiple ruby installations. By default it installs rubies to users home directory.
|
7
|
+
Rvm is available at http://beginrescueend.com/
|
8
|
+
|
9
|
+
Follow the rvm installation instructions, once the rvm command line tool works execute:
|
10
|
+
|
11
|
+
rvm install 1.9.2
|
12
|
+
rvm use 1.9.2
|
13
|
+
rvm gemset create ki
|
14
|
+
rvm use 1.9.2@ki
|
15
|
+
|
16
|
+
Now ruby 1.9.2 should be available and all gems will be installed for Ki use only.
|
17
|
+
|
18
|
+
## Check out the source
|
19
|
+
|
20
|
+
git clone git@github.com:mikko-apo/ki-repo.git
|
21
|
+
cd ki-repo
|
22
|
+
|
23
|
+
## Install gems
|
24
|
+
|
25
|
+
Bundler is used to download and install the required gems to the ruby environment in use. if you are using rvm, gems are installed to rvm's directories.
|
26
|
+
|
27
|
+
gem install bundler
|
28
|
+
bundle install
|
29
|
+
|
30
|
+
## Run tests
|
31
|
+
|
32
|
+
Tests are run with
|
33
|
+
|
34
|
+
rake spec
|
35
|
+
|
36
|
+
All tests should pass. Code coverage report is generated to coverage/index.html
|
37
|
+
|
38
|
+
## Generate documentation
|
39
|
+
|
40
|
+
Documentation is generated to doc/index.html with
|
41
|
+
|
42
|
+
rake yard
|
43
|
+
|
44
|
+
You can also start up the yard in server mode, which makes it easier to update the documentation. When server starts
|
45
|
+
up, the documentation will be available at http://localhost:8808/
|
46
|
+
|
47
|
+
yard server --reload
|
48
|
+
|
49
|
+
|
File without changes
|
data/docs/ki_commands.md
ADDED
@@ -0,0 +1,202 @@
|
|
1
|
+
# @title Ki: Command line utilities
|
2
|
+
# Command line utilities for Ki Repository v0.1.0
|
3
|
+
"ki" is the main command line tool that starts all other Ki processes. Whenever ki command line tools
|
4
|
+
are executed, ki goes through the following startup process
|
5
|
+
|
6
|
+
1. Common command line parameters are parsed. These can used to set execution parameters for this invocation.
|
7
|
+
2. Extension scripts are loaded from repository. Version configuration is from either -u or user preferences
|
8
|
+
3. Find command by name
|
9
|
+
4. Execute the command and pass rest of the command line parameters
|
10
|
+
|
11
|
+
Examples
|
12
|
+
|
13
|
+
ki build-version *.txt
|
14
|
+
ki -u my/tools compile
|
15
|
+
ki -u my/tools:scripts,tools compile
|
16
|
+
|
17
|
+
note: By default only files with tag "ki-cmd" are used. Use the 'my/tools:scripts,tools' to define additional tags.
|
18
|
+
|
19
|
+
Common parameters:
|
20
|
+
|
21
|
+
-h, --home Path to Ki root directory
|
22
|
+
-u, --use Use defined scripts
|
23
|
+
|
24
|
+
## help: Displays help for given Ki command
|
25
|
+
|
26
|
+
"ki help" shows information Ki and its commands.
|
27
|
+
|
28
|
+
### Examples
|
29
|
+
|
30
|
+
ki help
|
31
|
+
ki help version-build
|
32
|
+
|
33
|
+
|
34
|
+
## ki-info: Show information about Ki
|
35
|
+
|
36
|
+
"ki ki-info" shows information about Ki.
|
37
|
+
|
38
|
+
### Examples
|
39
|
+
|
40
|
+
ki ki-info -c
|
41
|
+
ki ki-info -r
|
42
|
+
|
43
|
+
### Parameters
|
44
|
+
-c, --commands List commands
|
45
|
+
-r, --registered List all registered extensions
|
46
|
+
|
47
|
+
## version-build: Create version metadata file
|
48
|
+
|
49
|
+
"ki version-build" can be used to generate version metadata files. Version metadata files
|
50
|
+
contain information about files (size, permission bits, hash checksums), version origins
|
51
|
+
and dependencies.
|
52
|
+
|
53
|
+
After version metadata file is ready, it can be imported to repository using version-import.
|
54
|
+
|
55
|
+
### Usage
|
56
|
+
|
57
|
+
ki version-build <parameters> file_pattern1*.* file_pattern2*.*
|
58
|
+
|
59
|
+
### Examples
|
60
|
+
|
61
|
+
ki version-build test.sh
|
62
|
+
ki version-build readme* -t doc
|
63
|
+
ki version-build -d my/component/1,name=comp,path=doc,internal -O "mv doc/test.sh helloworld.sh"
|
64
|
+
ki version-import
|
65
|
+
|
66
|
+
### Parameters
|
67
|
+
|
68
|
+
-f, --file FILE Version file target
|
69
|
+
-i, --input-directory INPUT-DIR Input directory
|
70
|
+
-v, --version-id VERSION-ID Version's id
|
71
|
+
--source-url URL Build source parameter url
|
72
|
+
--source-tag-url TAG-URL Build source parameter tag-url
|
73
|
+
--source-author AUTHOR Build source parameter author
|
74
|
+
--source-repotype REPOTYPE Build source parameter repotype
|
75
|
+
-t, --tags TAGS Tag files with keywords
|
76
|
+
--hashes HASHES Calculate checksums using defined hash algos. Default: sha1. Available: sha1, sha2, md5
|
77
|
+
-d, --dependency DEPENDENCY Dependency definition my/component/123[,name=AA][,path=aa][,internal]
|
78
|
+
-o, --operation OP Add operation to previous dependency
|
79
|
+
-O, --version-operation OP Add operation to version
|
80
|
+
|
81
|
+
|
82
|
+
## version-test: Tests versions and their dependencies
|
83
|
+
|
84
|
+
"ki version-test" tests versions, their files and their dependencies. Can also test version that has not been imported yet.
|
85
|
+
|
86
|
+
### Examples
|
87
|
+
|
88
|
+
ki version-test -r my/product other/product
|
89
|
+
ki version-test -f ki-version.json -i file-directory
|
90
|
+
|
91
|
+
### Parameters
|
92
|
+
|
93
|
+
-f, --file FILE Version source file. By default uses file's directory as source for binary files.'
|
94
|
+
-i, --input-directory INPUT-DIR Binary file input directory
|
95
|
+
-r, --recursive Tests version's dependencies also.'
|
96
|
+
|
97
|
+
|
98
|
+
## version-import: Imports version metadata and files to repository
|
99
|
+
|
100
|
+
"ki version-import" imports version and its files to repository.
|
101
|
+
|
102
|
+
Version name can be defined either during "version-build",
|
103
|
+
or generated automatically for component at import (with -c my/component) or defined to be a specific version (-v).
|
104
|
+
Can also move files (-m), test dependencies before import (-t).
|
105
|
+
|
106
|
+
### Examples
|
107
|
+
|
108
|
+
ki version-import -m -t -c my/product
|
109
|
+
ki version-import -f ki-version.json -i file-directory
|
110
|
+
|
111
|
+
### Parameters
|
112
|
+
|
113
|
+
-f, --file FILE Version source file. By default uses file's directory as source for binary files.'
|
114
|
+
-i, --input-directory INPUT-DIR Input directory
|
115
|
+
-t, --test-recursive Tests version's dependencies before importing.'
|
116
|
+
-m, --move Moves files to repository'
|
117
|
+
-c COMPONENT, Creates new version number for defined component'
|
118
|
+
--create-new-version
|
119
|
+
-v, --version-id VERSION Imports version with defined version id'
|
120
|
+
|
121
|
+
|
122
|
+
## version-export: Export version to a directory
|
123
|
+
|
124
|
+
"ki version-export" exports version and its dependencies to target directory.
|
125
|
+
|
126
|
+
### Usage
|
127
|
+
|
128
|
+
ki version-export <parameters> <file_export_pattern*.*>
|
129
|
+
|
130
|
+
### Examples
|
131
|
+
|
132
|
+
ki version-export -o export-dir --tags -c bin my/product
|
133
|
+
ki version-export -o scripts -c -t my/admin-tools '*.sh'
|
134
|
+
|
135
|
+
### Parameters
|
136
|
+
|
137
|
+
-o, --output-directory INPUT-DIR Input directory
|
138
|
+
--tags TAGS Select files with matching tag
|
139
|
+
-t, --test Test version before export
|
140
|
+
-c, --copy Exported files are copied instead of linked
|
141
|
+
|
142
|
+
|
143
|
+
## version-status: Add status values to version
|
144
|
+
|
145
|
+
"ki version-status" sets status values to versions and sets status value order to component.
|
146
|
+
|
147
|
+
Status value order is used to determine which statuses match version queries:
|
148
|
+
|
149
|
+
my/component:maturity>alpha
|
150
|
+
|
151
|
+
### Examples
|
152
|
+
|
153
|
+
ki version-status add my/component/1.2.3 Smoke=Green action=path/123
|
154
|
+
ki version-status order my/component maturity alpha,beta,gamma
|
155
|
+
|
156
|
+
## version-show: Prints information about version or versions
|
157
|
+
|
158
|
+
"ki version-show" prints information about version or versions and their dependencies
|
159
|
+
|
160
|
+
### Examples
|
161
|
+
|
162
|
+
ki version-show -r -d my/component/23 my/product/127
|
163
|
+
ki version-show -f ki-version.json -i binary-dir
|
164
|
+
|
165
|
+
## version-search: Searches for versions and components
|
166
|
+
|
167
|
+
"ki version-search" searches for versions and components.
|
168
|
+
|
169
|
+
### Examples
|
170
|
+
|
171
|
+
ki version-search my/component
|
172
|
+
ki version-search my/*
|
173
|
+
|
174
|
+
## pref: Sets user preferences
|
175
|
+
|
176
|
+
Sets user preferences
|
177
|
+
Syntax: ki pref prefix|use parameters...
|
178
|
+
|
179
|
+
### Examples for command prefixes:
|
180
|
+
ki pref prefix
|
181
|
+
- shows command prefixes, when a "ki command" is executed ki looks for the command with all prefix combinations
|
182
|
+
ki pref prefix version package
|
183
|
+
- sets two command prefixes, looks for "command", "version-command" and "package-command"
|
184
|
+
ki pref prefix + foo
|
185
|
+
- adds one command prefix to existing ones, looks for "command", "version-command", "package-command", "foo-command"
|
186
|
+
ki pref prefix - package foo
|
187
|
+
- removes two command prefixes from list
|
188
|
+
ki pref prefix -c
|
189
|
+
- clears command prefix list
|
190
|
+
|
191
|
+
### Examples for automatic script loading:
|
192
|
+
ki pref use
|
193
|
+
- shows list of automatically loading scripts. when ki starts up, it looks for all defined versions and loads all files tagged with ki-cmd
|
194
|
+
ki pref use ki/http ki/ftp/123:ki-extra
|
195
|
+
- scripts are loaded from two different version. ki/http uses latest available version and files tagged with "ki-cmd", ki/ftp uses specific version and files tagged with "ki-extra"
|
196
|
+
ki pref use + ki/scp
|
197
|
+
- adds one more script package version
|
198
|
+
ki pref use - ki/scp ki/ftp/123:ki-extra
|
199
|
+
- removes two configurations
|
200
|
+
ki pref use -c
|
201
|
+
- clear use list
|
202
|
+
|
@@ -0,0 +1,171 @@
|
|
1
|
+
# @title Ki: Repository basics
|
2
|
+
|
3
|
+
# Getting started
|
4
|
+
|
5
|
+
1. Create a version and import it to repository
|
6
|
+
|
7
|
+
echo 'Hello World!' > test.sh
|
8
|
+
echo "Simple demo" > "readme.txt"
|
9
|
+
chmod u+x test.sh
|
10
|
+
ki version-build test.sh
|
11
|
+
ki version-build readme* -t doc
|
12
|
+
ki version-import -m -c my/component
|
13
|
+
ki version-show my/component
|
14
|
+
|
15
|
+
* `version-import -c` creates a new version under `"my/component`` and `-m` moves the files to version's repository directory
|
16
|
+
* `version-show` looks for the latest available version and shows information about that
|
17
|
+
|
18
|
+
2. Create another version, with dependency to `my/component/1` and import that too
|
19
|
+
|
20
|
+
ki pref prefix version
|
21
|
+
ki build -d my/component/1,name=comp,path=doc,internal -O "mv doc/test.sh helloworld.sh"
|
22
|
+
ki import -m -c my/product
|
23
|
+
ki show -r my/product
|
24
|
+
ki export my/product -o export
|
25
|
+
find export
|
26
|
+
|
27
|
+
* `ki pref prefix version` configures a shortcut to call version commands with shorter syntax
|
28
|
+
* `version-build` generates a dependency to `"my/component/1"` and puts the files from `"my/component/1"` to doc directory
|
29
|
+
* `version-build -O` operations are executed when the product version is exported, `"doc/test.sh"` is moved to helloworld.sh
|
30
|
+
* the contents of "export" directory should be:
|
31
|
+
|
32
|
+
export
|
33
|
+
export/doc
|
34
|
+
export/doc/readme.txt
|
35
|
+
export/helloworld.sh
|
36
|
+
|
37
|
+
# Repository structure: versions, components
|
38
|
+
|
39
|
+
Ki-Repo is a repository for storing file packages and metadata about those packages.
|
40
|
+
|
41
|
+
An example repository has following structure:
|
42
|
+
|
43
|
+
* Repository has three components: `"my/componentA"`, `"my/componentB"`, `"my/product"`.
|
44
|
+
* Each component maintains a chronological list of versions. For example, `"my/component"` has versions `"3"`, `"2"` and `"1"`.
|
45
|
+
* Component's name should be a unique identifier and it can include any number of identifiers. Valid component names include `"ki/repo"`, `"ki-repo"` or `"my/test/builds/ki/repo/"`.
|
46
|
+
* Version contains a set of files. Version can also define dependencies and other metadata.
|
47
|
+
|
48
|
+
The repository and its directories and files would look like this:
|
49
|
+
|
50
|
+
repository/
|
51
|
+
my/
|
52
|
+
component/
|
53
|
+
23/
|
54
|
+
jar/lib.jar
|
55
|
+
readme.txt
|
56
|
+
start.sh
|
57
|
+
22/
|
58
|
+
jar/lib.jar
|
59
|
+
readme.txt
|
60
|
+
start.sh
|
61
|
+
21/
|
62
|
+
jar/lib.jar
|
63
|
+
readme.txt
|
64
|
+
start.sh
|
65
|
+
componentB/
|
66
|
+
build-211/
|
67
|
+
lib/utils.rb
|
68
|
+
build-210/
|
69
|
+
...
|
70
|
+
build-209/
|
71
|
+
...
|
72
|
+
product/
|
73
|
+
98/
|
74
|
+
97/
|
75
|
+
...
|
76
|
+
|
77
|
+
## Version
|
78
|
+
|
79
|
+
Each version identifies a unique combination of files and dependencies. After version has been built and imported to
|
80
|
+
repository it does not change. This ensures that when ever the version is used, the contents stay the same.
|
81
|
+
|
82
|
+
In addition to having files, version can define metadata about its files (size, permission bits, hash checksums), origins and dependencies.
|
83
|
+
Files can also be tagged with identifiers to make it easier to identify files of different types. Versions can also have status
|
84
|
+
information (for example "IntegrationTest=green"), which makes it easier to search for versions.
|
85
|
+
|
86
|
+
Version's full name is the name of the component and the version name: `"my/component/23"`
|
87
|
+
|
88
|
+
### File metadata
|
89
|
+
|
90
|
+
{ "path": "test.sh", "size": 2, "executable": true, "tags": [ "test-start" ], "sha1": "9a900f538965a426994e1e90600920aff0b4e8d2" }
|
91
|
+
|
92
|
+
* path identifies the actual file (stored in the version)
|
93
|
+
* size and sha1 are calculated when version is built
|
94
|
+
* executable is stored when version is built and ki ensures that when the file is in repository or exported is has its executable flag set on
|
95
|
+
* file tags can be used to tag files with different identifiers
|
96
|
+
|
97
|
+
### Source information
|
98
|
+
|
99
|
+
{ "url": "http://test.repo/component@21331", "tag-url": "http://test.repo/component/tags/23", "repotype": "git", "author": "john" }
|
100
|
+
|
101
|
+
* source information is used to store reference to the original source of the package
|
102
|
+
|
103
|
+
### Dependencies
|
104
|
+
|
105
|
+
Version can have dependencies, that include additional versions in to the main version. Dependencies refer to other
|
106
|
+
versions with full version name.
|
107
|
+
|
108
|
+
{ "version_id": "my/component/23", "name": "comp", "path": "comp", "internal": true,
|
109
|
+
"operations": [
|
110
|
+
[
|
111
|
+
"cp", "comp/test.sh", "test.bat"
|
112
|
+
]
|
113
|
+
]
|
114
|
+
}
|
115
|
+
|
116
|
+
* version_id defines the full name of the required version
|
117
|
+
* path defines a subdirectory where files from this dependency are placed
|
118
|
+
* dependency can have a name, which makes it possible to compare version's dependencies and also navigate version hierarchies
|
119
|
+
* navigation syntax: my/product/1>comp -> my/component/1
|
120
|
+
* my/product/1>comp = my/component/23, my/product/2>comp -> my/component/24
|
121
|
+
* internal dependencies are visible in the version hierarchy only for top level version. comp is visible from my/product,
|
122
|
+
but if other version has a dependency on product, files from comp are not visible
|
123
|
+
|
124
|
+
### File Operations
|
125
|
+
|
126
|
+
Version can define file operations to modify the exported file structure. Operations can be defined at version level, so
|
127
|
+
that they affect version's all files (the ones defined in the version and brought by the dependencies). File operations
|
128
|
+
can also be defined per dependency.
|
129
|
+
|
130
|
+
Available file operations:
|
131
|
+
|
132
|
+
* cp - copy pattern1 pattern2 ... dest
|
133
|
+
* mv - move pattern1 pattern2 ... dest
|
134
|
+
* rm - remove.
|
135
|
+
|
136
|
+
Examples, command line format - json storage format
|
137
|
+
|
138
|
+
"rm *.txt" - ["rm", "*.txt"]
|
139
|
+
"cp *.txt sub-directory" - ["cp", "*.txt", "sub-directory"]
|
140
|
+
"mv *.sh scripts" - ["mv", "*.sh", "scripts"]
|
141
|
+
|
142
|
+
File operations are defined for "version-build" with -o and -O parameters
|
143
|
+
|
144
|
+
ki version-build -d my/tests/a/123,name=tests -o "rm *.txt"
|
145
|
+
ki version-build -O "cp scripts/start.sh start.sh"
|
146
|
+
|
147
|
+
## Component
|
148
|
+
|
149
|
+
Component has only a two responsibilities:
|
150
|
+
|
151
|
+
1. Keep a chronological list of versions
|
152
|
+
2. Store status value order for different status fields
|
153
|
+
|
154
|
+
Status value order is used to determine which statuses match version queries:
|
155
|
+
|
156
|
+
my/component:maturity>alpha
|
157
|
+
|
158
|
+
## Repository
|
159
|
+
|
160
|
+
Repository directory keeps a list of all components that have data in the repository.
|
161
|
+
|
162
|
+
## KiHome
|
163
|
+
|
164
|
+
KiHome is the root directory for Ki. It contains repositories.
|
165
|
+
|
166
|
+
# Multiple repositories (TODO)
|
167
|
+
|
168
|
+
Ki supports storing information to multiple repositories. It will be possible to use one repository for shared
|
169
|
+
artifacts like binaries and store statuses to a different repository. This way it's easy to support for example
|
170
|
+
team specific status values for artifacts produced by other team.
|
171
|
+
|
@@ -0,0 +1,50 @@
|
|
1
|
+
# @title Ki: Writing extensions
|
2
|
+
|
3
|
+
# Writing extensions
|
4
|
+
|
5
|
+
Ki can be extended by packaging extension code in to version packages and using the `ki -u version/id` command line
|
6
|
+
parameter to load scripts in to the Ruby VM. Extension versions can also be configured to be automatically loaded
|
7
|
+
with the `ki pref prefix version/id` command.
|
8
|
+
|
9
|
+
"ki" is the main command line tool that starts all other Ki processes. Whenever ki command line tools
|
10
|
+
are executed, ki goes through the following startup process
|
11
|
+
|
12
|
+
1. Common command line parameters are parsed. These can used to set execution parameters for this invocation.
|
13
|
+
2. Extension scripts are loaded from repository. Version configuration is from either -u or user preferences
|
14
|
+
3. Find command by name
|
15
|
+
4. Execute the command and pass rest of the command line parameters
|
16
|
+
|
17
|
+
Examples
|
18
|
+
|
19
|
+
ki build-version *.txt
|
20
|
+
ki -u my/tools compile
|
21
|
+
ki -u my/tools:scripts,tools compile
|
22
|
+
|
23
|
+
note: By default only files with tag "ki-cmd" are used. Use the 'my/tools:scripts,tools' to define additional tags.
|
24
|
+
|
25
|
+
Ki's extension mechanism makes it easy to manage different scenarios:
|
26
|
+
* write and distribute command line utilities
|
27
|
+
* use different versions of those utilities at the same time on the same machine (backwards compatability)
|
28
|
+
* add new features to existing utilities: hashing algorithms, integrations to different tools like git, mercurial and svn
|
29
|
+
|
30
|
+
# Command line utility
|
31
|
+
|
32
|
+
Command classes are registered with KiCommand.register_cmd
|
33
|
+
|
34
|
+
KiCommand.register_cmd("version-build", BuildVersionMetadataFile)
|
35
|
+
|
36
|
+
They should implement following methods:
|
37
|
+
* execute(ctx, args)
|
38
|
+
* help, summary
|
39
|
+
* attr_chain :shell_command, :require is optional but help method can use it to generate more useful help texts
|
40
|
+
|
41
|
+
For more information, see {Ki::ImportVersion}
|
42
|
+
|
43
|
+
# Extension points
|
44
|
+
|
45
|
+
{Ki::KiCommand} stores all registered extensions to its class variable {Ki::KiCommand::KiExtensions} ({Ki::ServiceRegistry}).
|
46
|
+
|
47
|
+
Currently used extension points are:
|
48
|
+
|
49
|
+
* /commands/
|
50
|
+
* /hashing/
|