hologit 0.43.2 → 0.45.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. package/README.md +69 -286
  2. package/index.d.ts +113 -76
  3. package/lib/Lens.js +218 -31
  4. package/lib/Projection.js +2 -2
  5. package/lib/Studio.js +150 -126
  6. package/package.json +3 -4
package/README.md CHANGED
@@ -1,328 +1,111 @@
1
1
  # hologit
2
2
 
3
- Hologit lets you declaratively define virtual sub-branches (called holobranches) within any Git branch that mix together content from their host branch, content from other repositories/branches, and executable-driven transformations.
3
+ A Git-native framework for declarative code automation that makes it simple to combine code from multiple sources and apply transformations efficiently.
4
4
 
5
- ## Features
5
+ ## Overview
6
6
 
7
- - Track and merge remote code from multiple sources
8
- - Advanced merge, filter, and sourcing strategies
9
- - Apply arbitrary executable steps efficiently and consistently via Chef Habitat packages
10
- - Content-based git-distributed caching of build steps
11
- - GitHub Action for materializing holobranches to real branches
12
- - `--watch` command to produce live updates (currently lazy/slow, theoretically can be made near-instant)
7
+ Hologit enables you to define virtual "holobranches" within your Git repository that can:
13
8
 
14
- ## Introduction
9
+ 1. Mix together content from:
10
+ - The host branch
11
+ - Other repositories/branches
12
+ - Generated/transformed content
15
13
 
16
- **Hologit** is a [free and open](https://www.fsf.org/about/what-is-free-software) framework for code and content automation inside your local git repository. It makes it simple, fast, and reliable for projects to automate complex editing and publishing workflows that can involve multiple source repositories, languages, and build tools. Aiming to make working on software easier for everyone—pro and beginner alike—hologit gets rid of the need to think about or even know what needs to happen after you change files. There should just be content, and it goes places when you change it.
14
+ 2. Apply transformations through "hololenses" using:
15
+ - Docker containers
16
+ - Chef Habitat packages
17
17
 
18
- This works by enabling a project's git repository to define virtual "holobranches" that can be continuously and efficiently "projected" from any source branch. The projection process handles combining code from remote sources ("compositing") and executing build tools on the result ("lensing") to produce an output file tree and optionally commit it to a branch/ref.
18
+ 3. Project changes efficiently by:
19
+ - Computing new git trees in memory
20
+ - Caching results based on content
21
+ - ~~Watching for live updates~~ *Coming Soon*
19
22
 
20
- **Compositing** offers deeper control over which files are pulled from a remote repository and where they are integrated than [git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules) alone, while being more dependable and tracable than language-specific package managers like [npm](https://www.npmjs.com/) and [composer](https://getcomposer.org/). Instead of copying and moving files around on disk, hologit takes a git-native approach to minimize disk activity by computing new git trees in memory. Computed trees may be written to disk later or used as input to another process without the overhead.
23
+ ## Key Concepts
21
24
 
22
- **Lensing** can execute any existing code or build tool consistently by leveraging [habitat](https://www.habitat.sh/) and using containers where necessary. However, it also opens the door to a new generation of git-native build tools that do as much of their work as possible in memory, reading and writing to git's object database instead of a working tree on disk.
25
+ ### Holobranches
23
26
 
24
- ## Quickstart
27
+ A holobranch is a virtual branch defined in `.holo/branches/` that specifies:
25
28
 
26
- The guide will walk you through an illustrative minimal use of hologit to publish a GitHub Pages branch.
29
+ - What content to include from which sources
30
+ - How to transform that content through lenses
31
+ - Where the content should be placed in the output tree
27
32
 
28
- Each heading links to [branches in the hologit/examples repository](https://github.com/hologit/examples/branches/all?query=basic%2F) showing the final state of the example project at the end of the section.
33
+ Unlike regular Git branches, holobranches are computed on-demand and can mix content from multiple sources while maintaining clean history.
29
34
 
30
- ### Create a repository with some minimal code [\[example branch\]](https://github.com/hologit/examples/tree/basic/01-init-repo)
35
+ ### Holosources
31
36
 
32
- To start this example, we'll use [the starter template from Bootstrap's *Getting Started* guide](https://getbootstrap.com/docs/4.2/getting-started/introduction/#starter-template) to create a website:
37
+ Sources let you pull in code from:
33
38
 
34
- ```console
35
- $ git init holo-example
36
- Initialized empty Git repository in /Users/chris/holo-example/.git/
37
- $ cd holo-example/
38
- $ curl -s https://raw.githubusercontent.com/hologit/examples/basic/01-init-repo/index.html > index.html
39
- $ git add index.html
40
- $ git commit -m "Add Bootstrap's starter template as index.html"
41
- [master (root-commit) 9fe77ec] Add Bootstrap's starter template as index.html
42
- 1 file changed, 22 insertions(+)
43
- create mode 100644 index.html
44
- ```
45
-
46
- ### Install hologit
47
-
48
- See [docs/grand-tour/installation.md](docs/grand-tour/installation.md)
39
+ - Other repositories via Git submodules
40
+ - Other branches in the same repository
41
+ - Remote Git repositories
42
+ - Output from local or remote holobranches
49
43
 
50
- ### Initialize .holo/ configuration [\[example branch\]](https://github.com/hologit/examples/tree/basic/02-init-holo)
44
+ Sources are configured in `.holo/sources/` and can be referenced by holobranches to include specific files or directories.
51
45
 
52
- Hologit configuration is stored under the `.holo/` tree at the root of a repository. Initialize it in each branch that will generate projections:
46
+ ### Hololenses
53
47
 
54
- ```console
55
- $ git holo init
56
- name=holo-example
57
- initialized .holo/config.toml
58
- $ cat .holo/config.toml
59
- [holo]
60
- name = "holo-example"
61
- $ git commit -m "Initialize .holo/ configuration"
62
- [master 881b0b6] Initialize .holo/ configuration
63
- 1 file changed, 2 insertions(+)
64
- create mode 100644 .holo/config.toml
65
- ```
48
+ Lenses are transformations that can be applied to source content through:
66
49
 
67
- To start, this configuration file only assigns a name for the code in the current source branch, which can be used later as an alternative to remote sources. The name `holo-example` was detected from the name of the repository's working tree, but could have been chosen by passing `--name ${my_project_name}` for the `init` command or just by editing the `./holo/config.toml` file later.
50
+ - Docker containers that process input trees
51
+ - Habitat packages that provide build tools
68
52
 
69
- ### Define a holobranch [\[example branch\]](https://github.com/hologit/examples/tree/basic/03-create-holobranch)
53
+ Lenses are configured in `.holo/lenses/` and can be chained together to form complex build pipelines.
70
54
 
71
- A holobranch can be defined by creating a holobranch config file at `.holo/branches/${my_holobranch_name}.toml` or any number of holomapping config files within `.holo/branches/${my_holobranch_name}/**.toml`. Generate a minimal "passthrough" holobranch that will copy all files from the current source branch:
55
+ ## Getting Started
72
56
 
73
- ```console
74
- $ git holo branch create --template=passthrough gh-pages
75
- initialized .holo/branches/gh-pages/_holo-example.toml
76
- $ cat .holo/branches/gh-pages/_holo-example.toml
77
- [holomapping]
78
- files = "**"
79
- $ git commit -m "Initialize .holo/branches/gh-pages configuration"
80
- [master 4b9aa68] Initialize .holo/branches/gh-pages configuration
81
- 1 file changed, 2 insertions(+)
82
- create mode 100644 .holo/branches/gh-pages/_holo-example.toml
83
- ```
57
+ 1. Initialize hologit configuration:
84
58
 
85
- This defines a holobranch named `gh-pages` with all files from holosource `holo-example` matching the [glob pattern](https://github.com/isaacs/minimatch) `**` populating its root directory. There are several elements of convention on display here:
86
-
87
- - The underscore prefixing the filename of`/_holo-example.toml` indicates that any files produced by the holomapping should be merged into the root directory of the projected holobranch.
88
- - If the filename were just `/holo-example.toml`, a subdirectory name `/holo-example/` would be created to contain all the files produced by the holomapping.
89
- - A holomapping config prefixed with an underscore could be named anything, all such holomappings at the same path will have their files merged to populate the directory.
90
- - There are only two required configuration options for each holomapping:
91
- - `holosource`: The name of a configured holosource referencing a repository to pull files from
92
- - Ommitted in the generated holomapping config
93
- - Defaults to the name of the file with the `.toml` extension and any `_` prefix stripped
94
- - `files`: A string or array for strings containing [glob patterns](https://github.com/isaacs/minimatch) for matching or excluding files
95
- - A value of just `'**'`, as in the generated config, matches all files in the source
96
-
97
- ### Project holobranch for first time
98
-
99
- With a holobranch defined with at least one holomapping, we have enough for our first tree projection:
100
-
101
- ```console
102
- $ git holo project gh-pages
103
- info: reading mappings from holobranch: gitDir=/Users/chris/holo-example/.git, ref=HEAD, workTree=false, name=gh-pages
104
- info: compositing tree...
105
- info: merging holo-example:{**} -> /
106
- info: stripping .holo/ tree from output tree...
107
- info: writing final output tree...
108
- info: projection ready:
109
- ff954bb0a1e4878db424cb1033a0c356dac8d350
110
- $ git cat-file -t ff954bb0a1e4878db424cb1033a0c356dac8d350
111
- tree
112
- $ git ls-tree -r ff954bb0a1e4878db424cb1033a0c356dac8d350
113
- 100644 blob 8092fa2adb4a9a395ac291fbdc9717b68be669aa index.html
59
+ ```bash
60
+ git holo init
114
61
  ```
115
62
 
116
- The output of the `project` command seen above is the git hash of a [**tree** object](https://git-scm.com/book/en/v2/Git-Internals-Git-Objects) that has been generated, if needed, within your git repository's object database. This hash *does not* reference a commit object like most git hashes most commonly seen. A tree object is the main ingrediant of a commit obect: the tree represents a complete unique state of all the files and a commit attaches the tree to a point in your chain of commits with timestamp and authorship information.
117
-
118
- A tree can be used directly:
119
-
120
- ```console
121
- $ git archive --format=zip $(git holo project gh-pages) > website.zip
122
- info: reading mappings from holobranch: gitDir=/Users/chris/Repositories/holo-example/.git, ref=HEAD, workTree=false, name=gh-pages
123
- info: compositing tree...
124
- info: merging holo-example:{**} -> /
125
- info: stripping .holo/ tree from output tree...
126
- info: writing final output tree...
127
- info: projection ready:
128
- $ unzip -l website.zip
129
- Archive: website.zip
130
- Length Date Time Name
131
- --------- ---------- ----- ----
132
- 1230 12-23-2018 20:32 index.html
133
- --------- -------
134
- 1230 1 file
135
- ```
63
+ 2. Create a holobranch:
136
64
 
137
- or wrapped in a commit:
138
-
139
- ```console
140
- $ git commit-tree -m "Update gh-pages" $(git holo project gh-pages)
141
- info: reading mappings from holobranch: gitDir=/Users/chris/Repositories/holo-example/.git, ref=HEAD, workTree=false, name=gh-pages
142
- info: compositing tree...
143
- info: merging holo-example:{**} -> /
144
- info: stripping .holo/ tree from output tree...
145
- info: writing final output tree...
146
- info: projection ready:
147
- 846a551ce356d5fa4088e58b3ad0f0d05aa6d389
148
- $ git cat-file -t 846a551ce356d5fa4088e58b3ad0f0d05aa6d389
149
- commit
150
- $ git cat-file -p 846a551ce356d5fa4088e58b3ad0f0d05aa6d389
151
- tree ff954bb0a1e4878db424cb1033a0c356dac8d350
152
- author Chris Alfano <chris@jarv.us> 1545615571 -0500
153
- committer Chris Alfano <chris@jarv.us> 1545615571 -0500
154
-
155
- Update gh-pages
65
+ ```bash
66
+ git holo branch create my-branch
156
67
  ```
157
68
 
158
- With the `--commit-branch` option, you can commit the generated tree to a give branch and output the new commit's hash instead:
159
-
160
- ```console
161
- $ git cat-file -p $(git holo project gh-pages --commit-branch=gh-pages)
162
- info: reading mappings from holobranch: gitDir=/Users/chris/Repositories/holo-example/.git, ref=HEAD, workTree=false, name=gh-pages
163
- info: compositing tree...
164
- info: merging holo-example:{**} -> /
165
- info: stripping .holo/ tree from output tree...
166
- info: writing final output tree...
167
- info: committed new tree to "gh-pages": 734f7dc034868af4e2bd23daf23e119faca1e0b8
168
- info: projection ready:
169
- tree ff954bb0a1e4878db424cb1033a0c356dac8d350
170
- author Chris Alfano <chris@jarv.us> 1545616786 -0500
171
- committer Chris Alfano <chris@jarv.us> 1545616786 -0500
172
-
173
- Projected gh-pages from 4b9aa68
174
- ```
69
+ 3. Add a source:
175
70
 
176
- ### Merge external code via a holosource [\[example branch\]](https://github.com/hologit/examples/tree/basic/04-create-holosource)
177
-
178
- The first step to using external code in your projections is defining a holosource:
179
-
180
- ```console
181
- $ git holo source create https://github.com/twbs/bootstrap --ref=v4.2.1
182
- info: listing https://github.com/twbs/bootstrap#v4.2.1
183
- info: fetching https://github.com/twbs/bootstrap#refs/tags/v4.2.1@9e4e94747bd698f4f61d48ed54c9c6d4d199bd32
184
- fetched https://github.com/twbs/bootstrap#refs/tags/v4.2.1@9e4e94747bd698f4f61d48ed54c9c6d4d199bd32
185
- initialized .holo/sources/bootstrap.toml
186
- $ cat .holo/sources/bootstrap.toml
187
- [holosource]
188
- url = "https://github.com/twbs/bootstrap"
189
- ref = "refs/tags/v4.2.1"
190
- $ git commit -m "Initialize .holo/sources/bootstrap configuration"
191
- [master 64ef9fc] Initialize .holo/sources/bootstrap configuration
192
- 1 file changed, 3 insertions(+)
193
- create mode 100644 .holo/sources/bootstrap.toml
71
+ ```bash
72
+ git holo source create https://github.com/example/repo
194
73
  ```
195
74
 
196
- Now this source can be referenced in holobranch mappings, this example takes advantage of the holosource being automatically set from the mapping filename:
197
-
198
- ```console
199
- $ mkdir .holo/branches/gh-pages/{js,css}
200
- $ cat > .holo/branches/gh-pages/css/_bootstrap.toml <<- END_OF_TOML
201
- [holomapping]
202
- root = "dist/css"
203
- files = "*.min.css"
204
- END_OF_TOML
205
- $ cat > .holo/branches/gh-pages/js/_bootstrap.toml <<- END_OF_TOML
206
- [holomapping]
207
- root = "dist/js"
208
- files = "*.min.js"
209
- END_OF_TOML
210
- $ git add --all
211
- $ git commit -am "Add css and js mappings for bootstrap to gh-pages holobranch"
212
- [master 4180e45] Add css and js mappings for bootstrap to gh-pages holobranch
213
- 2 files changed, 6 insertions(+)
214
- create mode 100644 .holo/branches/gh-pages/css/_bootstrap.toml
215
- create mode 100644 .holo/branches/gh-pages/js/_bootstrap.toml
216
- ```
75
+ 4. Project your holobranch:
217
76
 
218
- Projecting the `gh-pages` tree now shows the files merged from bootstrap:
219
-
220
- ```console
221
- $ git ls-tree -r $(git holo project gh-pages)
222
- info: reading mappings from holobranch: gitDir=/Users/chris/Repositories/holo-example/.git, ref=HEAD, workTree=false, name=gh-pages
223
- info: compositing tree...
224
- info: merging holo-example:{**} -> /
225
- info: merging bootstrap:dist/css/{*.min.css} -> /css/
226
- info: merging bootstrap:dist/js/{*.min.js} -> /js/
227
- info: stripping .holo/ tree from output tree...
228
- info: writing final output tree...
229
- info: projection ready:
230
- 100644 blob b3e6881a586c99b55e2d1878839eede6fb3fa9d7 css/bootstrap-grid.min.css
231
- 100644 blob 0668a8cd93bba140c00bc0c410ad54c61af71d9e css/bootstrap-reboot.min.css
232
- 100644 blob e6b4977799e3a3a377e475ee765eb4a9961c6c71 css/bootstrap.min.css
233
- 100644 blob 8092fa2adb4a9a395ac291fbdc9717b68be669aa index.html
234
- 100644 blob 97f14c05c3d5960129caf3e4666f661dfdb8228a js/bootstrap.bundle.min.js
235
- 100644 blob 9df6b6c2ced14a60259171e1fdacc2534ddee183 js/bootstrap.min.js
77
+ ```bash
78
+ git holo project my-branch
236
79
  ```
237
80
 
238
- For reference, here is what the holobranch definition that projected this tree looks like at this point:
239
-
240
- ```console
241
- $ tree .holo/branches/gh-pages
242
- .holo/branches/gh-pages
243
- ├── _holo-example.toml
244
- ├── css
245
- │ └── _bootstrap.toml
246
- └── js
247
- └── _bootstrap.toml
248
- ```
81
+ See the [Installation Guide](docs/grand-tour/installation.md) and [Grand Tour](docs/grand-tour/README.md) for detailed setup and usage instructions.
249
82
 
250
- Before projecting again, you might want to update all remote sources to their latest commits:
83
+ ## Key Features
251
84
 
252
- ```console
253
- $ git holo source fetch --all
254
- fetched bootstrap https://github.com/twbs/bootstrap#refs/heads/v4-dev@dc17c924e86948ae514d72f8ccc67f9d77657f6b
255
- ```
85
+ - **Git-native**: Works directly with Git's object database for maximum efficiency
86
+ - **Content-based caching**: Automatically caches build results based on input content, optionally sharing with other users and CI/CD via the same Git server hosting your project
87
+ - **Declarative configuration**: Define complex automation workflows in TOML files
88
+ - ~~**Live updates**: Watch mode for continuous projection of changes~~ *Coming Soon*
89
+ - **GitHub Action**: Materialize holobranches to real branches in CI/CD
90
+ - **Flexible transformations**: Use any build tool through containers or packages
256
91
 
257
- ### Assemble the complete source code via a holo lens
92
+ ## Use Cases
258
93
 
259
- - Apply sass compilation and compression via generic lenses
94
+ - **Monorepo Management**: Combine code from multiple repositories while maintaining clean history
95
+ - **Build Automation**: Create efficient, reproducible build pipelines
96
+ - **Documentation**: Generate and publish documentation from multiple sources
97
+ - **Deployment**: Prepare deployment artifacts with consistent transformations
98
+ - **Code Generation**: Automate code generation and transformation workflows
260
99
 
261
- ### Work upstream by checking out a holosource
100
+ ## Documentation
262
101
 
263
- To work on changes to code being pulled in from remote repositories, any or all sources can be checkout out as a [git submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules):
102
+ - [Installation Guide](docs/grand-tour/installation.md)
103
+ - [Repository Setup](docs/grand-tour/repository-setup.md)
104
+ - [Working with Sources](docs/workflows/work-on-sources.md)
105
+ - [Holobranches Guide](docs/grand-tour/holobranches.md)
106
+ - [Hololenses Guide](docs/grand-tour/hololenses.md)
107
+ - [Holoreactors Guide](docs/grand-tour/holoreactors.md)
264
108
 
265
- ```console
266
- $ git holo source checkout --all
267
- checked out .holo/sources/bootstrap from https://github.com/twbs/bootstrap#refs/tags/v4.2.1@9e4e9474
268
- $ git commit -m "Initialize .holo/sources/bootstrap submodule"
269
- [basic/05-checkout-holosource ee39b88] Initialize .holo/sources/bootstrap submodule
270
- 2 files changed, 5 insertions(+)
271
- create mode 100644 .gitmodules
272
- create mode 160000 .holo/sources/bootstrap
273
- ```
109
+ ## License
274
110
 
275
- - Make commits inside submodule, project with --working
276
- - Commit gitlink outside submodule, change all projections
277
-
278
- ### Make use of a projected tree
279
-
280
- - Archive tree-ish
281
- - Write to a real branch
282
- - Push to github gh-pages
283
-
284
- ## Advanced Usage
285
-
286
- ### Overlay a project
287
-
288
- ### Build new holo lenses
289
-
290
- ## Roadmap
291
-
292
- - [~] (in progress) Complete getting started tour, break into sections in another doc format
293
- - [ ] Develop shorter quickstart
294
- - [~] `* --ref` (in progress) option to use a specific ref instead of HEAD
295
- - [~] `* ---no-working` (in progress) option to ignore working directory and only use ref
296
- - [ ] `project --watch` option to keep running and automatically update projection with changes to input
297
- - [ ] `project --audit` option to produce audit commits chain
298
- - [ ] Implement `holoreactor` objects: defined like lenses within the projected branches, the handle piping result subtrees into single-run or persistent processes running in the studio via `hab exec` or `hab svc load`
299
- - [ ] Implement a `holoreactor` for serving static websites
300
- - [ ] Implement a `holoreactor` for running/restarting a node app
301
- - [ ] Implement a `holoreactor` for running an emergence app locally or on a remote cluster
302
- - [ ] Expose from the studio's HTTP interface a *virtual* holospace that can be mutated via git push or WebDAV
303
- - [ ] Add option to `[holosource]` config to override submodule checkout path
304
- - [ ] Leverage lower-overhead chroot environments instead of Docker containers on Linux systems
305
- - [ ] Enable running and connecting to a persistent background studio for quick on-demand projection
306
- - Build a studio docker image that extends Habitat's studio image with hologit pre-installed
307
- - Wrap around opening interactive shells
308
- - Wrap around entrypoint to start holostudio process
309
- - [ ] Implement in-memory tree hashing to avoid calls to `git mktree` for tree hashes that are known in the tree cache to already exist in the repo
310
- - [ ] Visual Studio Code extension
311
- - Top-level hologit section with views of sources and branches
312
- - Commands via context menu and command palette
313
- - Ability to graphically toggle watch mode for each source
314
- - Open holobranches in workspace via filesystem provider for read-only browsing of either composited or lensed content
315
- - Enable writing to mounted holobranches by routing writes to estimated source via reverse-compositing, checking out submodules on-the-fly
316
- - [ ] Isolate lens environments further from source
317
- - Currently, `.git` directory is mounted into lensing environment, so working tree is safe but repo is exposed to damage by lens code
318
- - Lensing only needs to exchange object hashes, so objects tree could be mounted read-write and the rest could be empty
319
- - More robust options might include using git's fetch/push mechanism and/or mounting objects tree as read-only alternates
320
- - A git SmartHTTP server could be exposed via the studio socket, but could objects be exchanged as efficiently as just bind-mounting the objects database directly?
321
- - [ ] Enable running web-based code editor as a holoreactor to your code base
322
-
323
- ## Reference
324
-
325
- ## TODO
326
-
327
- - [ ] Have `project` fetch and read source HEAD if no submodule commit is found
328
- - [ ] Refactor `source add` and `source fetch` to use common code, leave things in same state
111
+ This project is [free and open source](https://www.fsf.org/about/what-is-free-software) software.