archsight 0.3.3 → 0.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b601281c3dc26c95647876457b96e590146aee23b762d14034ed2ae5f0953710
-  data.tar.gz: 7c271d26e8def3f7e85d82210ebda0d45628e280c1bb58832b81f2900c3e1097
+  metadata.gz: 0312d7e0c0790b8b2a117ad6277e49bfb18c61545134b27510fa3209e951697d
+  data.tar.gz: 94915564bbbab8c59e139422c1a99317ec7fd5a87370e9905b16ed330cd56951
 SHA512:
-  metadata.gz: 416c904c7b8ecc1c44527b226fe7e6c7697c03d905d855db532bb80f1bd5ff0ff1277c770947eed411a6ca7cae7dafd8e32085690d72a4235c73bd4c1f35950b
-  data.tar.gz: 9c22b78187060cdc3606b4f423a38b79f84edcee6cb1d6f1446348a9da46dc088bedb4f537c809aad28c2db33f875d490637c4dcc13b57b4ee0ccc76822a0193
+  metadata.gz: 7b15fc5fb4130fe0783efe8e20cf7aa2ce3962e6566d54ae5c1c85834692e5363cc3a9667eb2f7cfbf17832badc0461cbf684c1d5a5c04711e5e529d1fb11bb2
+  data.tar.gz: 36e154fb3fefd3dd0de59ced6d6f404e4238242ec182d78f5ced051fec01002a3af8dbe11c8eced3ff16fc978791301627fe3f06641e0ea859e9a529326ea814

data/README.md

@@ -48,13 +48,19 @@ Also available as [Docker](docs/docker.md) image and [Helm chart](docs/kubernete
 archsight web [OPTIONS]      # Start web server
 archsight lint               # Validate YAML and relations
 archsight import             # Execute pending imports
-archsight analyze            # Execute analysis scripts
+archsight analyze            # Execute analysis scripts (--format typst -o out.typ for Typst source)
 archsight template KIND      # Generate YAML template for a resource type
 archsight diagram FILE.asd   # Render a diagram DSL file to SVG
 archsight console            # Interactive Ruby console
 archsight version            # Show version
 ```
 
+### Analysis as Typst
+
+`archsight analyze --format typst [-o report.typ] [-f REGEX] [-v]` writes the results of all (or the filtered) analyses as one [Typst](https://typst.app) document instead of the terminal rendering: a summary, a section per analysis with its tables, lists, text and messages, and `asd` code sections as inline SVG diagrams. Without `-o` the source goes to stdout; paste it into typst.app or run `typst compile report.typ`. `-v` keeps all rows (otherwise tables and lists are cut after 10) and the error backtraces.
+
+`--template layout.typ` replaces the built-in layout with your own. The document then imports the template and calls two functions it must define: `archsight-report` as a show rule (`count`, `passed`, `failed`, `findings`), and `analysis-result(name:, description:, state:, duration:, data:)[body]` once per analysis, where `body` is the rendered sections and `data` is a dictionary of what the script passed to `data(key, value)`. `data` sections appear only here (not in the web UI or terminal), so a script can hand structured values (for example a matrix) to a template while keeping its tables for the web. With a template, tables and lists are never cut.
+
 ### Web Server Options
 
 ```bash

data/docs/archimate.md

@@ -39,6 +39,15 @@ ArchiMate is an open and independent enterprise architecture modeling language d
 |--------------|-------------|
 | **Specialization** | Represents that an element is a particular kind of another element. |
 
+### Support in archsight
+
+| Relationship | Supported as |
+|--------------|--------------|
+| **Specialization** | `specializes` between technology elements of one kind (node, system software, service), see "Concept and Implementation" in the modeling guide |
+| **Composition** | `composedOf` between nodes and between system software |
+| **Aggregation** | `aggregates` between technology artifacts (an image contains packages), see "Deployment" in the modeling guide |
+| **Assignment** | `assignedTo` from a node or system software to the technology artifacts deployed on it |
+
 ### Relationship Connectors
 
 | Connector | Description |
@@ -100,6 +109,9 @@ ArchiMate is an open and independent enterprise architecture modeling language d
 | **Representation** | A perceptible form of the information carried by a business object. |
 | **Product** | A coherent collection of services and/or passive structure elements, accompanied by a contract, offered to customers. |
 
+Archsight models Contract as `BusinessContract` (aggregated by a `BusinessProduct`, `agreedBy` its parties, `composedOf` its
+call-offs, SLAs and amendments) and Product as `BusinessProduct`. Business Object and Representation are not modelled.
+
 ## Application Layer Elements
 
 ### Active Structure Elements
@@ -140,6 +152,8 @@ ArchiMate is an open and independent enterprise architecture modeling language d
 | **Path** | A link between two or more technology internal active structure elements, through which these elements can exchange data, energy, or material. |
 | **Communication Network** | A set of structures and behaviors that connects devices or system software for transmission, routing, and reception of data. |
 
+**Supported in archsight:** Device (`TechnologyDevice`) and Communication Network (`TechnologyCommunicationNetwork`) are modelled in the Technology layer. A device is one physical machine; the cable between two devices is the ArchiMate association `connectedTo`, a network is `realizedBy` its switches and routers and `connects` the devices attached to it. Path is not modelled: a link between facilities is `TechnologyEquipment`. See [Logical Networks](/doc/modeling) and, for which kind to pick, [Choosing the Kind: Infrastructure](modeling.md#choosing-the-kind-infrastructure).
+
 ### Behavior Elements
 
 | Element | Definition |
@@ -165,6 +179,8 @@ ArchiMate is an open and independent enterprise architecture modeling language d
 | **Distribution Network** | A physical network used to transport materials or energy. |
 | **Material** | Tangible physical matter or energy. |
 
+**Supported in archsight:** Equipment (`TechnologyEquipment`) and Facility (`TechnologyFacility`) are modelled, as kinds of the Technology layer (ArchiMate defines the physical elements as an extension of it). Distribution Network and Material are not. See [Where Technology Runs](/doc/modeling) and, for which kind to pick, [Choosing the Kind: Infrastructure](modeling.md#choosing-the-kind-infrastructure).
+
 ## Implementation and Migration Layer Elements
 
 | Element | Definition |
@@ -182,6 +198,8 @@ ArchiMate is an open and independent enterprise architecture modeling language d
 | **Grouping** | Aggregates or composes concepts that belong together based on some common characteristic. |
 | **Location** | A conceptual or physical place or position where concepts are located (structure elements) or performed (behavior elements). |
 
+**Supported in archsight:** Location (`Location`) is modelled and shown under Other. Grouping is not.
+
 ## ArchiMate Layers
 
 ArchiMate organizes elements into distinct layers:
@@ -191,9 +209,11 @@ ArchiMate organizes elements into distinct layers:
 3. **Business Layer** - What: business processes, actors, services, objects
 4. **Application Layer** - How: application components, services, data objects
 5. **Technology Layer** - How: infrastructure, nodes, devices, networks
-6. **Physical Layer** - Where: facilities, equipment, materials, distribution networks
+6. **Physical Layer** - Where: facilities, equipment, materials, distribution networks (archsight: `TechnologyFacility` and `TechnologyEquipment` in the Technology layer, `Location` under Other)
 7. **Implementation & Migration Layer** - When: work packages, plateaus, gaps
 
+Capability, Product, Application Service and Application Component are easily mixed up in practice; [Choosing the Kind](modeling.md#choosing-the-kind) gives the question, test and naming for each; [Choosing the Kind: Infrastructure](modeling.md#choosing-the-kind-infrastructure) does the same for locations, facilities, equipment, networks and compute.
+
 ## Key Concepts
 
 ### Active vs Passive Structure

data/docs/configuration.md

@@ -37,6 +37,7 @@ that is set but empty counts as not set.
 | `jira.server` | `ARCHSIGHT_JIRA_SERVER` | Name of the Jira server as Confluence knows it (the `server` parameter of its Jira macro). Needed by the Confluence export when Confluence has more than one Jira server: without it the exported `{jira:KEY}` macros have no server and Confluence shows an error |
 | `jira.server_id` | `ARCHSIGHT_JIRA_SERVER_ID` | Id of that server (the `serverId` parameter). Copy both from the storage format of a Jira macro that already works in Confluence (page "Edit in storage format") |
 | `jira.token` | `ARCHSIGHT_JIRA_TOKEN` | Reserved, nothing uses it yet |
+| `import.cache_root` | `ARCHSIGHT_IMPORT_CACHE_ROOT` | Where the GitHub, GitLab and repository imports clone repositories, and what importers of the resources repository get as `cache_root` / `ARCHSIGHT_CACHE_DIR`. Default `~/.cache/archsight/git`; an import's own `import/config/cacheRoot` wins, see [Importers of the Resources Repository](/doc/import#importers-of-the-resources-repository). Every import makes sure the clones are not indexed by Spotlight (macOS): an empty `.metadata_never_index` in `~/.cache/archsight`, or in the cache root when it is elsewhere |
 
 ## Command line
 

data/docs/diagram.md

@@ -40,11 +40,20 @@ Ready-made sources live in `examples/diagrams/` (`archsight.asd` models archsigh
 archsight diagram examples/diagrams/aws_vpc.asd -o diagram.svg
 archsight diagram aws_vpc.asd --watch              # re-renders on every save
 archsight diagram three_tier.asd --relation=all    # also show control/data-flow edges
-archsight diagram overview.asd --theme=compact     # denser spacing + smaller fonts (or cozy)
+archsight diagram overview.asd --theme=compact     # denser spacing + smaller fonts (or cozy, tight)
 archsight diagram overview.asd --legend=bottom     # legend below (or right/left/top/none; default auto)
 archsight diagram *.asd                            # render every file to its own .svg
+archsight diagram --format *.asd                   # re-indent the sources in place
+archsight diagram --format --check *.asd           # list unformatted sources, exit 1 if any (CI)
 ```
 
+`--format` tidies the sources instead of rendering them: every line is
+indented by its brace depth (two spaces per level), trailing whitespace
+and runs of blank lines go, the file ends in one newline. Comments, line
+breaks and one-line blocks stay as written. A file that does not parse is
+reported and left alone, and the result is checked to parse to the same
+diagram before it is written. With `-o` a single input is written there.
+
 `-o`/`--output` only makes sense with a single input file; with multiple
 inputs (including a shell glob like `*.asd`) each one is rendered next to
 itself with a `.svg` extension. `--watch` works with multiple inputs too,
@@ -177,7 +186,7 @@ level, where `OP` is:
 - `<->` — bidirectional (arrowheads on both ends).
 - `--` — undirected (no arrowheads).
 
-Edges between boxes that sit one above the other (or side by side) and overlap on the other axis are drawn as a single straight axis-aligned line when nothing is in the way, e.g. many components dropping straight down onto one wide database. An explicit `style "orthogonal"` keeps its right-angle turns.
+Edges between boxes that sit one above the other (or side by side) and overlap on the other axis are drawn as a single straight axis-aligned line when nothing is in the way, e.g. many components dropping straight down onto one wide database. An explicit `style "orthogonal"` takes that straight line too (it has no bend at all), and is never drawn diagonally.
 
 Edge attributes:
 - `label "..."` — text drawn along the edge.
@@ -231,10 +240,13 @@ From roomiest to densest:
 | `default` | small diagrams; used when there's no `theme` statement | 13px |
 | `cozy` | mid-sized diagrams: roughly halfway between the two | 12px |
 | `compact` | large diagrams that would otherwise sprawl | 11px |
+| `tight` | very large diagrams: `compact` with boxes that hug their labels | 11px |
 
 `cozy` and `compact` shrink container padding, sibling gaps and node
 sizes along with the fonts, and size each node from its label's real
-Helvetica glyph widths so boxes hug their text. `--theme=NAME` on the CLI (or `theme:` in
+Helvetica glyph widths so boxes hug their text. `tight` is `compact` with
+less than half the space above and below a component's label and half the
+space beside it. `--theme=NAME` on the CLI (or `theme:` in
 `Archsight::Diagram.render`) overrides the file's own choice. The legend
 isn't affected.
 

data/docs/docker.md

@@ -32,7 +32,10 @@ Access web interface at: <http://localhost:4567>
 
 - Volume mount `-v "/path/to/resources:/resources"` mounts your resources directory
 - Default command starts the web server on port 4567
-- Pass subcommands directly (lint, version, console, template)
+- Pass subcommands directly (lint, version, console, template, import)
+- `import` runs the imports of the mounted repository, including its own handlers (`import/handlerPath` must point
+  inside `/resources`). The image has no git, go or python, so imports that clone repositories or run such programs
+  need a derived image
 
 ## Environment Variables
 

data/docs/icons.md

@@ -12,6 +12,7 @@ Current icon assignments for each resource type:
 | ApplicationService | `cube` | <i class="iconoir-cube"></i> |
 | BusinessActor | `community` | <i class="iconoir-community"></i> |
 | ApplicationEvent | `bell-notification` | <i class="iconoir-bell-notification"></i> |
+| BusinessContract | `journal-page` | <i class="iconoir-journal-page"></i> |
 | BusinessControl | `shield-search` | <i class="iconoir-shield-search"></i> |
 | BusinessEvent | `flash` | <i class="iconoir-flash"></i> |
 | BusinessRole | `user-badge-check` | <i class="iconoir-user-badge-check"></i> |
@@ -34,6 +35,11 @@ Current icon assignments for each resource type:
 | MotivationRequirement | `task-list` | <i class="iconoir-task-list"></i> |
 | MotivationConstraint | `prohibition` | <i class="iconoir-prohibition"></i> |
 | MotivationStakeholder | `user-crown` | <i class="iconoir-user-crown"></i> |
+| TechnologyCommunicationNetwork | `network` | <i class="iconoir-network"></i> |
+| TechnologyEquipment | `cpu` | <i class="iconoir-cpu"></i> |
+| TechnologyDevice | `server` | <i class="iconoir-server"></i> |
+| TechnologyFacility | `building` | <i class="iconoir-building"></i> |
+| Location | `map-pin` | <i class="iconoir-map-pin"></i> |
 | TechnologyNode | `server-connection` | <i class="iconoir-server-connection"></i> |
 | StrategyCapability | `strategy` | <i class="iconoir-strategy"></i> |
 | TechnologyArtifact | `puzzle` | <i class="iconoir-puzzle"></i> |
@@ -48,12 +54,12 @@ Icons and graph nodes take the colour of their layer.
 
 | Layer | Colour | Graph (light) | Icon |
 |-------|--------|---------------|------|
-| Strategy | orange | `#F4A261` | `#D4824A` |
 | Motivation | purple | `#CE93D8` | `#AB47BC` |
+| Strategy | orange | `#F4A261` | `#D4824A` |
+| Implementation & Migration | rose | `#F48FB1` | `#E0527F` |
 | Business | yellow | `#F9DC5C` | `#D4B83C` |
 | Application | blue | `#6CBEED` | `#4A9ECD` |
 | Technology | green | `#8AC926` | `#7AB316` |
-| Implementation & Migration | rose | `#F48FB1` | `#E0527F` |
 | Other | gray | `#CCCCCC` | `#999999` |
 
 ## Icon Categories

data/docs/import.md

@@ -10,7 +10,9 @@ Imports are defined as YAML resources with kind `Import`. Each import specifies:
 - **Configuration** specific to that handler
 - Optional **caching** to avoid re-running unchanged imports
 
-Dependencies between imports are automatically derived from the `generates` relation. When a parent import generates child imports, those children depend on the parent and will run after it completes.
+Dependencies between imports are automatically derived from the `generates` relation. When a parent import generates child imports, those children depend on the parent and will run after it completes. An import can also declare further dependencies with `import/dependsOn` (see [Dependencies](#dependencies)).
+
+Besides the built-in handlers, a resources repository can bring its own (see [Importers of the Resources Repository](#importers-of-the-resources-repository)).
 
 The import executor runs imports concurrently in dependency order, with a visual progress display showing completion percentage and ETA.
 
@@ -28,8 +30,21 @@ archsight import --dry-run
 
 # Force re-run all imports (ignore cache)
 archsight import --force
+
+# Run one import and the imports it depends on
+archsight import -f Import:Deployments
+
+# Run only that import, not its dependencies (with --force to ignore its cache)
+archsight import -f Import:Deployments --no-deps --force
+
+# Validate the resources after the run (like archsight lint)
+archsight import --lint
 ```
 
+With `--lint`, an error in a resource that an import wrote names that import (`(generated by Import:X)`): the import
+in the resource's `generated/script`, else the import whose output file holds it. So a dangling reference between
+generated files shows up with the import that caused it.
+
 ### Progress Display
 
 In TTY mode, imports show a live progress display:
@@ -62,12 +77,22 @@ spec: {}
 
 | Annotation | Description |
 |------------|-------------|
-| `import/handler` | Handler to execute (required) |
+| `import/handler` | Handler to execute (required): a built-in one or one registered by `import/handlerPath` |
+| `import/handlerPath` | Ruby file of a handler of the resources repository, relative to the resources directory |
+| `import/dependsOn` | Imports that must run first: names or glob patterns, comma separated |
 | `import/enabled` | Set to "false" to disable |
 | `import/priority` | Execution order (lower runs first, default: 0) |
-| `import/cacheTime` | Cache duration: "30m", "1h", "24h", "7d", or "never" |
+| `import/cacheTime` | Cache duration: "30s", "30m", "1h", "24h", "7d", or "never" |
 | `import/outputPath` | Output file path relative to resources directory |
 
+Settings every handler understands:
+
+| Annotation | Description |
+|------------|-------------|
+| `import/config/inputs` | Files the import reads (glob patterns relative to the resources directory, comma separated); a change invalidates the cache |
+| `import/config/requiredEnv` | Environment variables the import needs, comma separated; a missing one fails the import |
+| `import/config/optional` | `"true"`: skip the import with a warning when a required variable is missing (it stays stale; its dependents run) |
+
 ### Caching
 
 Imports can be cached to avoid re-running when data hasn't changed:
@@ -77,6 +102,7 @@ import/cacheTime: "24h"  # Re-run after 24 hours
 ```
 
 Supported duration formats:
+- `30s` - 30 seconds
 - `30m` - 30 minutes
 - `1h` - 1 hour
 - `24h` - 24 hours
@@ -85,6 +111,32 @@ Supported duration formats:
 
 The cache uses the `generated/at` annotation written by handlers. When an import completes, it writes a marker with the current timestamp. On subsequent runs, the executor checks if `generated/at + cacheTime > now` to skip cached imports.
 
+The cache is also invalid, and the import runs again, when:
+- its output file (`import/outputPath`) or a file it recorded as its own (`generated/outputs`) is missing
+- its handler or `import/config/*` settings changed (`generated/configHash`)
+- a file matched by `import/config/inputs` changed (`generated/inputsHash`: paths, sizes, modification times)
+- an import it declares in `import/dependsOn` executed in the same run (one served from the cache does not count).
+  Child imports are not affected when their parent runs; they keep their own cache time.
+
+### Dependencies
+
+An import runs after the import that generated it. `import/dependsOn` adds more: import names or glob patterns,
+comma separated.
+
+```yaml
+import/dependsOn: "Import:GitHub:acme, Import:Repo:*"
+```
+
+A pattern also covers imports that do not exist yet: the import waits for every import whose last run generated a
+match, and on a first run for every import whose handler generates imports. So `Import:Repo:*` waits for the GitHub
+import and then for the repository imports it creates. Declared dependencies are part of the execution plan (`-n`,
+which also lists patterns without a match), of `-f` (a filtered import brings its dependencies along) and of the
+cycle check. `archsight lint` reports a name that matches no import.
+
+`--no-deps` runs only the imports `-f` matches: their dependencies neither run nor are waited for, so the import works
+on the model as it is (re-importing thousands of repositories is not needed to try a change to one importer). The
+cache still applies; add `--force` to run it anyway.
+
 ### Configuration Pattern
 
 Handler-specific configuration uses `import/config/*` annotations:
@@ -394,73 +446,144 @@ If an import isn't re-running when expected:
 3. Use `archsight import --force` to bypass cache for all imports
 4. Alternatively, use `import/cacheTime: never` to always re-run a specific import
 
-**Note:** The cache is automatically invalidated when:
-- The output file (specified by `import/outputPath`) is deleted or missing
-- The import source YAML file has been modified since the last run
+**Note:** The cache is automatically invalidated when the output file is missing, the configuration or the inputs
+changed, or a declared dependency executed (see [Caching](#caching)). Editing the Import definition invalidates it
+only through its handler and `import/config/*` settings.
 
-## Creating Custom Handlers
+## Importers of the Resources Repository
 
-To create a new handler:
+A resources repository often needs importers that only make sense for it: deployments read from the repositories
+archsight already cloned, a wiki space, team membership from a directory service. They live in the repository and
+run with `archsight import` like the built-in handlers: scheduled with their dependencies, cached, filtered with
+`-f`, shown by `-n` and checked by `archsight lint`.
 
-1. Create a Ruby file in `lib/archsight/import/handlers/`
-2. Inherit from `Archsight::Import::Handler`
-3. Implement the `execute` method
-4. Register with `Registry.register("name", YourHandler)`
+### A Ruby handler
 
-```ruby
-# Repository handler - clones/syncs and analyzes a git repository
-class Archsight::Import::Handlers::Custom < Archsight::Import::Handler
-  def execute
-    # Read configuration
-    url = config("url")
-
-    # Update progress
-    progress.update("Fetching data")
+The Import names the handler and the file that registers it (`import/handlerPath`, relative to the resources
+directory, like `import/outputPath`):
 
-    # Fetch and process data
-    data = fetch_data(url)
+```yaml
+# imports/deployments.yaml
+apiVersion: architecture/v1alpha1
+kind: Import
+metadata:
+  name: Import:Deployments
+  annotations:
+    import/handler: deployments
+    import/handlerPath: importers/deployments.rb
+    import/dependsOn: "Import:Repo:*"
+    import/outputPath: generated/deployments.yaml
+    import/cacheTime: "24h"
+    import/config/inputs: "deployments/*.yaml"
+spec: {}
+```
 
-    # Generate resources
-    resources = data.map { |item| build_resource(item) }
+```ruby
+# importers/deployments.rb
+require_relative "lib/charts" # helpers next to the handler
 
-    # Write output with self-marker for caching
-    yaml_content = resources_to_yaml(resources) + YAML.dump(self_marker)
-    write_yaml(yaml_content)
+class DeploymentsImporter < Archsight::Import::Handler
+  def execute
+    progress.update("Reading charts")
+    # the model as loaded after the dependencies ran
+    components = database.instances_by_kind("ApplicationComponent")
+    # the clones of the repository imports
+    charts = Charts.find(File.join(cache_root, "github"))
+
+    resources = charts.filter_map do |chart|
+      next unless components.key?(chart.component)
+
+      resource_yaml(kind: "TechnologyArtifact", name: "Chart:#{chart.name}",
+                    annotations: { "artifact/type" => "chart" })
+    end
+
+    write_file("pages/deployments.md", Charts.page(charts)) # a page next to the generated YAML
+    write_yaml(resources_to_yaml(resources) + YAML.dump(self_marker))
+    write_generates_meta
   end
+end
 
-  # Marker for cache timestamp
-  def self_marker
-    {
-      "apiVersion" => "architecture/v1alpha1",
-      "kind" => "Import",
-      "metadata" => {
-        "name" => import_resource.name,
-        "annotations" => { "generated/at" => Time.now.utc.iso8601 }
-      },
-      "spec" => {}
-    }
-  end
+Archsight::Import::Registry.register("deployments", DeploymentsImporter)
+```
 
-  private
+- `import`, `lint` and `web` load the handler files the Imports declare. The resources repository is trusted like the
+  gem: its handler code runs in all three commands.
+- A name registered twice, a file that does not load or does not register the name in `import/handler`, and an
+  unknown handler name are lint errors.
+- Only the Ruby standard library and the gems archsight ships are available to a handler.
+- Run it with what it depends on: `archsight import -f Import:Deployments`, or alone with `--no-deps`; see the plan
+  with `-n`.
+- A web server loads a handler file once; restart it after changing the file.
 
-  def build_resource(item)
-    resource_yaml(
-      kind: "TechnologyArtifact",
-      name: item["name"],
-      annotations: { "custom/field" => item["value"] }
-    )
-  end
-end
+### A command in any language
+
+`import/handler: command` runs a program instead:
 
-Archsight::Import::Registry.register("custom", Archsight::Import::Handlers::Custom)
+```yaml
+apiVersion: architecture/v1alpha1
+kind: Import
+metadata:
+  name: Import:Wiki
+  annotations:
+    import/handler: command
+    import/config/command: "python3 importers/wiki.py"
+    import/config/env: CONFLUENCE_TOKEN
+    import/config/requiredEnv: CONFLUENCE_TOKEN
+    import/config/optional: "true"
+    import/config/outputs: "pages/wiki/*.md"
+    import/outputPath: generated/wiki.yaml
+    import/cacheTime: "7d"
+spec: {}
 ```
 
+- The command line is split like a shell does, but runs without a shell (`sh -c '...'` for pipes). It runs in the
+  root of the git repository that holds the resources directory (`import/config/workdir` relative to the resources
+  directory overrides it).
+- The environment is clean: `PATH`, `HOME`, `LANG`, `LC_ALL`, `TMPDIR`, the variables named in `import/config/env`, and
+  - `ARCHSIGHT_RESOURCES_DIR` - the resources directory
+  - `ARCHSIGHT_CACHE_DIR` - where the git-based imports clone repositories
+  - `ARCHSIGHT_IMPORT_NAME` - the name of the Import
+  - `ARCHSIGHT_OUTPUT_PATH` - a file for the YAML resources the program generates
+- archsight writes what the program put in `ARCHSIGHT_OUTPUT_PATH` unchanged to `import/outputPath` and adds the self
+  marker, so caching and dependencies work. The documents are not marked as generated: a document that only adds
+  relations to a hand-written resource must not look generated, and many kinds (Location, TechnologyNode, ...) do not
+  accept the `generated/*` annotations. A program that wants a resource marked sets `generated/script: <import name>`
+  itself. Files matched by `import/config/outputs` are recorded as the import's own (see below).
+- The output file belongs to the import: it is replaced on every run and removed when `import/outputPath` changes.
+  `archsight lint` reports another import that writes to the same file.
+- A non-zero exit fails the import with the last lines of the error output.
+
+### Shared state and outputs
+
+- `cache_root` (Ruby) and `ARCHSIGHT_CACHE_DIR` (command) are the directory the GitHub, GitLab and repository
+  imports clone into: `import/config/cacheRoot`, else `import.cache_root` in the
+  [configuration](configuration.md) (`ARCHSIGHT_IMPORT_CACHE_ROOT`), else `~/.cache/archsight/git`.
+- `database` holds the model as loaded after the dependencies ran (the import command loads the kinds the importers
+  use: Import, BusinessActor, ApplicationComponent, ApplicationInterface).
+- `write_yaml` may be called several times with `filename:` for more YAML files next to the output path.
+  `write_file(path, content)` writes a whole file of the import's own, such as a page. Its files are recorded in the
+  self marker (`generated/outputs`): the cache is invalid while one is missing, and a file of an earlier run that is
+  not written again is deleted. Call `self_marker` after writing them.
+
+### Docker
+
+The image runs `archsight import` with the repository mounted (`-v /path/to/repository:/resources`). Handler files
+must be inside the mounted resources directory. The image has no git, go or python; a command import can only run
+programs available in the image.
+
 ### Handler Helper Methods
 
 | Method | Description |
 |--------|-------------|
-| `config(key, default:)` | Get configuration value |
-| `progress.update(msg)` | Update progress display |
-| `write_yaml(content)` | Write YAML to output path |
-| `resource_yaml(kind:, name:, ...)` | Build resource hash |
-| `import_yaml(name:, handler:, ...)` | Build child import hash |
+| `config(key, default:)` | Get a configuration value (`import/config/<key>`) |
+| `config_path(key)` | Get a path setting with `~` expanded |
+| `cache_root` | Directory of the cloned repositories |
+| `database` | The model loaded before this batch of imports |
+| `progress.update(msg)` / `progress.warn(msg)` | Update the progress display |
+| `resource_yaml(kind:, name:, ...)` | Build a resource hash marked as generated by this import |
+| `import_yaml(name:, handler:, ...)` | Build a child import hash (override `self.generates_imports?` to return true) |
+| `resources_to_yaml(resources)` | Join resource hashes into YAML documents |
+| `write_yaml(content, filename:, sort_key:)` | Write YAML to the output path (or a file next to it) |
+| `write_file(path, content)` | Write a file of the import's own below the resources directory |
+| `self_marker` | Marker with `generated/at` and the cache hashes; write it last |
+| `write_generates_meta` | Record what the import generated (`spec.generates`) |

data/docs/index.md.erb

@@ -35,23 +35,34 @@ Resources representing stakeholders, goals, outcomes, requirements, and constrai
 - [MotivationAssessment](/doc/resources/motivation_assessment) - Risks, vulnerabilities and findings
 - [MotivationPrinciple](/doc/resources/motivation_principle) - Policies and principles
 
+### Strategy Layer
+
+Resources representing strategic capabilities and planning.
+
+- [StrategyCapability](/doc/resources/strategy_capability) - Strategic abilities and functions
+
+### Implementation & Migration Layer
+
+Resources representing change: work, results, milestones and the states of the architecture between them.
+
+- [ImplementationWorkPackage](/doc/resources/implementation_work_package) - Projects, measures and remediation
+- [ImplementationDeliverable](/doc/resources/implementation_deliverable) - Results of work packages
+- [ImplementationEvent](/doc/resources/implementation_event) - Milestones, go-lives and deadlines
+- [ImplementationPlateau](/doc/resources/implementation_plateau) - Baseline, transition and target states
+- [ImplementationGap](/doc/resources/implementation_gap) - Differences between plateaus
+
 ### Business Layer
 
-Resources representing business processes, actors, products, and the controls that guide processes.
+Resources representing business processes, actors, products, contracts, and the controls that guide processes.
 
 - [BusinessActor](/doc/resources/business_actor) - Teams and organizations
 - [BusinessRole](/doc/resources/business_role) - Responsibilities that actors take on
 - [BusinessProcess](/doc/resources/business_process) - Structured business workflows
 - [BusinessProduct](/doc/resources/business_product) - Products offered to customers
+- [BusinessContract](/doc/resources/business_contract) - Agreements with customers: frameworks, call-offs, SLAs
 - [BusinessControl](/doc/resources/business_control) - Controls that guide business processes
 - [BusinessEvent](/doc/resources/business_event) - Threat events, loss events and triggers
 
-### Strategy Layer
-
-Resources representing strategic capabilities and planning.
-
-- [StrategyCapability](/doc/resources/strategy_capability) - Strategic abilities and functions
-
 ### Application Layer
 
 Resources representing application services, components, and interfaces.
@@ -71,21 +82,16 @@ Resources representing technology infrastructure and artifacts.
 - [TechnologyInterface](/doc/resources/technology_interface) - Technical interfaces
 - [TechnologyEvent](/doc/resources/technology_event) - Infrastructure events, scan findings and outages
 - [TechnologySystemSoftware](/doc/resources/technology_system_software) - Logical infrastructure components
-- [TechnologyNode](/doc/resources/technology_node) - Physical infrastructure
-
-### Implementation & Migration Layer
-
-Resources representing change: work, results, milestones and the states of the architecture between them.
-
-- [ImplementationWorkPackage](/doc/resources/implementation_work_package) - Projects, measures and remediation
-- [ImplementationDeliverable](/doc/resources/implementation_deliverable) - Results of work packages
-- [ImplementationEvent](/doc/resources/implementation_event) - Milestones, go-lives and deadlines
-- [ImplementationPlateau](/doc/resources/implementation_plateau) - Baseline, transition and target states
-- [ImplementationGap](/doc/resources/implementation_gap) - Differences between plateaus
+- [TechnologyNode](/doc/resources/technology_node) - Hosts, VMs and host roles
+- [TechnologyDevice](/doc/resources/technology_device) - Physical machines: servers, switches, routers, firewalls and appliances
+- [TechnologyFacility](/doc/resources/technology_facility) - Data centers, colocation sites, offices and edge sites
+- [TechnologyEquipment](/doc/resources/technology_equipment) - Racks, fabrics, links and physical clusters
+- [TechnologyCommunicationNetwork](/doc/resources/technology_communication_network) - Logical networks: underlay, overlay, management, public, VPN
 
 ### Other
 
 - [ComplianceEvidence](/doc/resources/compliance_evidence) - Evidence for compliance requirements
+- [Location](/doc/resources/location) - Countries, metros, regions and other places
 - [View](/doc/resources/view) - Saved queries and custom views
 
 ## References