@shortlink-org/portolan 0.3.0 → 0.4.0

package/README.md

@@ -87,8 +87,9 @@ reference).
 - **Diagrams** — LikeC4 C4 views (estate landscape, every container in the
   estate with its technology and the protocol on each edge, one per context,
   two per service, one dynamic view per flow), an ELK-routed dependency graph,
-  and a context map. The app never draws these itself; `npm run likec4:gen`
-  writes the model from the catalog.
+  and a context map. The app never draws these itself; `npm run gen` writes
+  the model from the catalog as its last step, and holds it to the catalog in
+  `gen:check`, and `npm run likec4:gen` writes it on its own before `dev`.
 - **ER canvases** — per store: tables, views, keys and crow's feet, plus column
   lineage (`from`) drawn dashed; hovering a column lights the whole chain back
   to where the value came from.
@@ -116,7 +117,43 @@ first:
   with a second writer, a table that no longer holds the aggregate it claims,
   a column whose type has drifted from its field's, an outbox with no payload;
 - a channel with a second publisher, an event on a channel its service does not
-  declare, a declared channel no event names, a subscription nothing publishes.
+  declare, a declared channel no event names, a subscription nothing publishes;
+- what the deployer runs from a place no service lives at, and where the GitOps
+  tree and the deployer disagree.
+
+Each check is a rule in `rules/builtin.json`: a passport - id, severity, what
+it looks for and what to do about a row - and the rule itself, in CEL over one
+subject. A subject is one row with every fact a rule would otherwise have to
+look up already on it - a table knows who writes it, a channel knows who else
+publishes there, a call knows whether its peer is in the estate - so a rule is
+one line a reader can read, copy and tighten. **Settings → Rules** lists them
+with the rows each produces now. A shipped rule can be switched off or
+re-graded in `portolan.json`, with a reason, and a rule of your own is written
+there the same way - over a service, a call, a copy, an event, a consumer, a
+channel, a subscription, a table, a column, a deployment, a flow or an
+aggregate - and runs in the page the moment it is saved:
+
+```json
+{
+  "problemRules": [
+    { "id": "shared-store", "enabled": false, "reason": "the estate shares one database by design" },
+    {
+      "id": "team.quiet-event",
+      "over": "event",
+      "severity": "warning",
+      "title": "Event nobody consumes",
+      "when": "size(event.consumers) == 0 && !event.name.endsWith('Audit')",
+      "message": "'nothing consumes ' + event.id",
+      "peer": "event.service"
+    }
+  ]
+}
+```
+
+Expressions are type-checked against the subject's fields when the manifest is
+read - `event.nme` is refused, not shown as an empty page - and again in the
+page, by the same module. The Rules page writes the entry after a preview of
+the rows it would add (portolan.0016, portolan.0017).
 
 ## Where the facts come from
 
@@ -125,17 +162,24 @@ Plugins, one JSON message in and one out (`plugins/README.md`), declared in
 
 | phase | plugins |
 | --- | --- |
-| extract | `extract-project`, `extract-go`, `extract-ts`, `extract-rust`, `extract-java`, `extract-django`, `extract-celery`, `extract-python-kafka`, `extract-openapi`, `extract-wsdl`, `extract-http-clients`, `extract-redis`, `extract-asyncapi`, `extract-graphql`, `extract-proto`, `extract-river`, `extract-watermill`, `extract-go-nats`, `extract-csr`, `extract-sql`, `extract-flows`, `extract-adr`, `extract-glossary`, `extract-commands` |
+| extract | `extract-project`, `extract-go`, `extract-ts`, `extract-rust`, `extract-java`, `extract-django`, `extract-laravel`, `extract-php-ddd`, `extract-csharp-ddd`, `extract-celery`, `extract-python-kafka`, `extract-openapi`, `extract-wsdl`, `extract-http-clients`, `extract-redis`, `extract-asyncapi`, `extract-graphql`, `extract-proto`, `extract-river`, `extract-watermill`, `extract-go-nats`, `extract-go-sqs`, `extract-csr`, `extract-sql`, `extract-flows`, `extract-adr`, `extract-glossary`, `extract-commands`, `extract-k8s`, `extract-argocd` |
 | verify | `verify-otel` — reads traces, marks the hops they show as `verified`; `verify-codeowners` — reads CODEOWNERS, says who to ask about each service |
 | generate | `gen-markdown` — `docs/`, `gen-mermaid` — standalone flow diagrams, `gen-backstage` — Backstage entities |
 
 `fetch-git`, `fetch-bsr` and `fetch-csr` bring in sources from other
 repositories, the Buf Schema Registry and a Confluent Schema Registry, against a
-lock, so a later build can reproduce them without a socket.
+lock, so a later build can reproduce them without a socket. `fetch-k8s` reads a
+live cluster through `kubectl` into the names each service answers on and
+dials, and replays the committed fragment when there is no cluster to ask.
+`fetch-argocd` reads the applications an Argo CD server manages into a snapshot
+of where each service runs - environment, cluster, namespace, revision, images -
+against the same kind of lock; the service page shows it under "Where it runs".
 
 Each plugin describes its own options; `npm run schema` asks all of them and
 composes `schema/portolan.schema.json`, which editors complete against and `gen`
-checks before running anything.
+checks before running anything. The same answers are rendered as the site's
+plugin reference - `/plugins` on any generated site, grouped by what each reads
+or makes, with the options it takes - so the list above is the short form.
 
 Non-fatal extractor diagnostics remain attached to their pipeline step. The
 Settings page groups repetitions by plugin, stable rule and severity, and shows
@@ -257,6 +301,107 @@ live in a dedicated architecture repository: `fetch-git` pins the service
 repositories at immutable commits and the normal merge, check, diff, and build
 commands operate on the combined estate.
 
+### A real one: Bagisto
+
+The services under `examples/` are small on purpose. For what the catalog
+looks like on a real application, the site ships a third profile: pick
+**Bagisto** in the catalog selector. [Bagisto](https://github.com/bagisto/bagisto)
+is an open-source e-commerce platform on Laravel - 41 packages, one per
+module, talking to each other through named events - read by
+`extract-laravel` alone, with nothing annotated for the catalog.
+
+What comes out of Bagisto 2.4, pinned in `vendor/repos/bagisto/bagisto`:
+
+| | |
+| --- | --- |
+| model groups | 28, one per package with models, 125 Eloquent models and 22 enums between them |
+| events | 309, most of them named - `Event::dispatch('sales.order.cancel.after', …)` - owned by the package their name says, 50 with a listener |
+| HTTP | 520 operations in 10 interfaces, one per route file, with an inferred OpenAPI document |
+| database | 138 tables and 1300 columns replayed from 189 migrations, 185 foreign keys, 120 places the code reads or writes them |
+| queues | 17 jobs on one queue, each a hop in the flow that dispatches it and a worker flow of its own |
+| flows | 610 |
+
+Go to the service and read it the way the tabs are ordered: **bus** for the
+queue, **data** for the schema laid out by model group - the picker beside
+the arrangement toggle switches groups off, the search brings a table up
+close - and **flows** for `shop-checkout-onepage-orders-store`, the checkout
+followed from the controller through the repository it holds to the events
+and the job it ends in. Every "view source" link opens the file on GitHub at
+the pinned commit.
+
+The warnings the extractor left are findings about Bagisto, not about the
+tool: 218 events under the admin package are dispatched by name and declare
+no payload; `Settings\CurrencyController` answers admin routes and does not
+exist; the sitemap job reads a `pages` table no migration creates; a
+foreign key points at a `theme_customizations` table no migration creates;
+one listener waits for `bagisto.shop.products.price.after`, which nothing
+dispatches, and one for the repository events of a vendored package. How the
+fragments were made, and how to refresh them, is in
+`vendor/repos/bagisto/bagisto/README.md`; the reading rules are in
+`plugins/extract-laravel/README.md`.
+
+### The small one, by the book: CodelyTV php-ddd-example
+
+The fourth profile, **CodelyTV php-ddd-example**, is the opposite kind of
+real: [php-ddd-example](https://github.com/CodelyTV/php-ddd-example) is
+CodelyTV's reference for DDD, hexagonal architecture and CQRS in PHP, a
+Symfony monorepository laid out as `src/<Context>/<Module>/{Domain,Application,Infrastructure}`
+with the deployables under `apps/`, read by `extract-php-ddd` alone - the
+layout is the claim, and nothing is annotated. What comes out, pinned in
+`vendor/repos/CodelyTV/php-ddd-example`: four contexts (Mooc, Backoffice,
+Analytics, Retention), six services, five aggregates with their value
+objects and three domain events on one RabbitMQ exchange, one queue per
+subscriber, the MySQL tables from the Doctrine mappings including a JOINED
+`steps` family, and sixteen flows. The one to read is
+`backoffice-frontend-courses-post`: the back office's form dispatches
+Mooc's `CreateCourseCommand` over an in-memory bus, so the flow crosses to
+`mooc.backend` with a `call` step, records `CourseCreated`, saves the
+course, and the subscribers in Mooc, Backoffice and Analytics pick the event
+up from their own queues.
+
+The warnings are findings about the project: Analytics and Retention have no
+application under `apps/`, so their code deploys nowhere and Analytics'
+subscriber to every event runs nowhere; `TrimVideoCommandHandler` is named
+like a handler and implements neither bus interface; Retention's four modules
+and Mooc's Notifications are empty scaffolding; the front end routes
+`/api/courses` to a controller that does not exist. How the fragments were
+made is in `vendor/repos/CodelyTV/php-ddd-example/README.md`; the reading
+rules are in `plugins/extract-php-ddd/README.md`.
+
+### The same lesson in C#: Modular Monolith with DDD
+
+The fifth profile, **Modular Monolith with DDD**, is Kamil Grzybek's
+[modular-monolith-with-ddd](https://github.com/kgrzybek/modular-monolith-with-ddd),
+the reference for the pattern in .NET: five modules laid out as
+`src/Modules/<Module>/{Domain,Application,Infrastructure,IntegrationEvents}`,
+one API host, one SQL Server database with a schema per module, read by
+`extract-csharp-ddd` alone - through Roslyn, the compiler's own parser, over
+the tree as one compilation and without restoring a single package. What
+comes out, pinned in `vendor/repos/kgrzybek/modular-monolith-with-ddd`: five
+contexts with one service each, nineteen aggregates and five model groups,
+sixty domain events dispatched in process and seven integration events on
+the in-memory bus, sixty-one commands and twenty-nine queries, an internal command queue
+per module, the forty-three tables and seventeen views of the database
+project with the root tables mapped to their aggregates through the EF
+configurations, fifty-one HTTP operations across four documents, the
+project's seventeen decision records, and a hundred and two flows. The one
+to read is `register-new-user`: the User Access controller hands
+`RegisterNewUserCommand` to the Registrations module's facade in process,
+the flow crosses with a `call` step, `NewUserRegisteredDomainEvent` is
+raised and saved, and from there the outbox republishes it to the bus as an
+integration event that Meetings, Administration and Payments each hear and
+turn into a job on their own internal queue.
+
+The warnings are findings about the project: `MemberCreatedIntegrationEvent`
+is declared and published nowhere while User Access subscribes to it;
+Payments subscribes to `MeetingGroupProposalAcceptedIntegrationEvent` and
+has no handler; the `app` schema belongs to no module; the view is created
+as `v_Countriess` and the query reads `v_Countries`; and two modules keep a
+`Domain/Users` directory with a typed id and nothing to hold it. How the
+fragments were made is in
+`vendor/repos/kgrzybek/modular-monolith-with-ddd/README.md`; the reading
+rules are in `plugins/extract-csharp-ddd/README.md`.
+
 ## Develop Portolan itself
 
 ```bash

package/catalog/model.go

@@ -44,9 +44,59 @@ type Catalog struct {
 	Modules     []ProtoModule      `json:"modules,omitempty"`
 	Terms       []Term             `json:"terms,omitempty"`
 	Repos       []RepoPin          `json:"repos,omitempty"`
+	Deployments []Deployment       `json:"deployments,omitempty"`
 	Externals   []External         `json:"externals,omitempty"`
 }
 
+// Deployment is one place a service runs: an Argo CD Application as the
+// deployer listed it, reduced to what a deploy changes. A list on the catalog
+// rather than a field on Service: the fetcher that writes it reads a control
+// plane and does not know which service an Application is, and one service
+// stands in several places.
+type Deployment struct {
+	// ID is "<argocd namespace>/<application name>".
+	ID          string `json:"id"`
+	Name        string `json:"name"`
+	Project     string `json:"project"`
+	Environment string `json:"environment"`
+	Cluster     string `json:"cluster"`
+	Namespace   string `json:"namespace"`
+	// Repo is spelled the way Service.Repo spells it; empty for a chart
+	// from a registry.
+	Repo string `json:"repo"`
+	Path string `json:"path"`
+	// Chart is the Helm chart name when the source is a registry.
+	Chart          string `json:"chart,omitempty"`
+	TargetRevision string `json:"targetRevision"`
+	Revision       string `json:"revision"`
+	// Tool is helm, kustomize, directory or plugin, as Argo CD says it.
+	Tool string `json:"tool"`
+	URL  string `json:"url"`
+	// Service is the id of the service the Application's labels name;
+	// empty when they name none, and the path decides.
+	Service string   `json:"service,omitempty"`
+	Images  []string `json:"images,omitempty"`
+	// Basis says who said so: "manifest" for a row read out of a GitOps
+	// tree, "api" for one read off the deployer, "both" for one the merge
+	// folded from the two. Empty reads as "api", which is what every row
+	// was before there were trees to read.
+	Basis string `json:"basis,omitempty"`
+	// Drift is what the tree says where it and the deployer disagree, set
+	// by the merge when both spoke (portolan.0013).
+	Drift *DeploymentDrift `json:"drift,omitempty"`
+}
+
+// DeploymentDrift is the GitOps tree's word on the fields where the
+// deployer says otherwise. A field is set only when the two differ.
+type DeploymentDrift struct {
+	Project        string   `json:"project,omitempty"`
+	Cluster        string   `json:"cluster,omitempty"`
+	Namespace      string   `json:"namespace,omitempty"`
+	Path           string   `json:"path,omitempty"`
+	TargetRevision string   `json:"targetRevision,omitempty"`
+	Images         []string `json:"images,omitempty"`
+}
+
 // External is a system outside the estate with a contract: what it answers on,
 // read from the copy of its document vendored beside the adapter that calls it,
 // and nothing else. It sits at the root beside the contexts, so its id is its
@@ -160,6 +210,23 @@ type Service struct {
 	// list is the one the runner would accept. Absent when nothing declares
 	// any, which is not the same as a service that cannot be built.
 	Commands []Command `json:"commands,omitempty"`
+
+	// Hosts are the names this service answers on, read from what deploys
+	// it: a Kubernetes Service's name in its short, namespaced, `svc` and
+	// fully qualified forms, and the hosts of the Ingress or HTTPRoute in
+	// front of it. Written so that a call another service is configured to
+	// make to `pricing.shop.svc` can find the service that answers. Absent
+	// when nothing in the tree says where the service is reachable.
+	Hosts []string `json:"hosts,omitempty"`
+
+	// Dials are the in-cluster names this service's workload is configured
+	// to reach, read out of its environment and config maps and reduced to
+	// the host alone. A value is never kept: not the variable it came from,
+	// not the scheme, port, path or credentials around the name. Only a name
+	// the cluster resolves qualifies - a Service in the same tree, or a
+	// `<name>.<namespace>.svc` form - so a password in an environment
+	// variable is not something this list can hold by shape.
+	Dials []string `json:"dials,omitempty"`
 }
 
 // Command is one entry of a task runner's file: a make target, an npm script,
@@ -477,6 +544,12 @@ type Operation struct {
 	// appear in RpcService.Methods. Empty means nothing outside the service can
 	// reach it, which is a fact worth having rather than a gap.
 	ExposedBy []string `json:"exposedBy,omitempty"`
+	// Fields is what the caller hands in: the command's or query's own shape
+	// as the message class declares it. Nil when the extractor does not read
+	// messages; an empty list is a message that carries nothing.
+	Fields []Field `json:"fields,omitempty"`
+	// Source is where the handler is, "path:line".
+	Source string `json:"source,omitempty"`
 }
 
 // Block is an entity or a value object. The two are told apart by the list
@@ -603,6 +676,30 @@ type Field struct {
 	// Number is the protobuf field number when the source carries one. Other
 	// schema and domain extractors leave it absent.
 	Number int `json:"number,omitempty"`
+	// Required is set when the source says the field must be sent: a
+	// Protovalidate `required`, a name in a JSON Schema `required` list.
+	// Absent means the source does not say, which in proto3 and OpenAPI alike
+	// means it may be left out.
+	Required bool `json:"required,omitempty"`
+	// Rules is what the source says a value must satisfy, in the order it
+	// said it.
+	Rules []FieldRule `json:"rules,omitempty"`
+}
+
+// FieldRule is one constraint on a field's value, in the catalog's own
+// vocabulary so a Protovalidate `min_len` and a JSON Schema `minLength` are
+// one rule: `min_len`, `max_len`, `len`, `pattern`, `prefix`, `suffix`,
+// `contains`, `not_contains`, `format`, `gt`, `gte`, `lt`, `lte`, `const`,
+// `in`, `not_in`, `multiple_of`, `min_items`, `max_items`, `unique`,
+// `min_pairs`, `max_pairs`, `defined_only`, `lt_now`, `gt_now`, `cel`. A rule
+// on what a list holds is prefixed `items.`; on a map's keys or values,
+// `keys.` or `values.`. A custom option the catalog has no word for keeps the
+// name the source gave it, `(acme.pii)`, so it is shown rather than lost.
+type FieldRule struct {
+	Name string `json:"name"`
+	// Value is the bound as the source wrote it, text so a 64-bit number
+	// survives; absent for a bare flag such as `unique`.
+	Value string `json:"value,omitempty"`
 }
 
 type TypeDef struct {
@@ -619,6 +716,7 @@ const (
 	StoreKindMongoDB    StoreKind = "mongodb"
 	StoreKindClickHouse StoreKind = "clickhouse"
 	StoreKindS3         StoreKind = "s3"
+	StoreKindDynamoDB   StoreKind = "dynamodb"
 	StoreKindOther      StoreKind = "other"
 )
 
@@ -788,6 +886,48 @@ type Flow struct {
 	// Participants order is significant: it is the lane order.
 	Participants []Participant `json:"participants"`
 	Steps        FlowNodes     `json:"steps"`
+	// Examples are recordings of this flow running: one per trace a verifier
+	// kept, each naming the steps it showed with what the spans said about
+	// them. They are examples of data, not evidence - the evidence is the
+	// status on the step.
+	Examples []FlowExample `json:"examples,omitempty"`
+}
+
+// FlowExample is one recorded run of a flow, read from one trace.
+type FlowExample struct {
+	// ID is "<recording>#<trace id>": unique within the flow, stable across
+	// runs of the same files.
+	ID string `json:"id"`
+	// Recording is the file the trace was read from, relative to the step's
+	// input root - where the verifier was pointed, which is where an uploaded
+	// recording is kept.
+	Recording string `json:"recording"`
+	TraceID   string `json:"traceId"`
+	// RecordedAt is when the root span started, RFC 3339 in UTC; absent when
+	// the recording carries no clock.
+	RecordedAt string `json:"recordedAt,omitempty"`
+	// DurationMs is root start to the last end the trace shows.
+	DurationMs float64 `json:"durationMs"`
+	// Steps are the steps this trace showed, in the order it showed them,
+	// with what an allowlist of span attributes said about each.
+	Steps []ExampleStep `json:"steps"`
+}
+
+// ExampleStep is what one span said about one step: how long it took and
+// the names it carried. Values that could be somebody's data - a query text,
+// a header, a path with an id in it - are not carried; see verify-otel.
+type ExampleStep struct {
+	Step       string            `json:"step"`
+	Label      string            `json:"label,omitempty"`
+	DurationMs float64           `json:"durationMs"`
+	Attributes map[string]string `json:"attributes,omitempty"`
+}
+
+// StepSeen says how many recordings showed a step. On a declared step it
+// accompanies the raised status; on a step the code does not declare it is
+// the reason the step is in the flow at all.
+type StepSeen struct {
+	Traces int `json:"traces"`
 }
 
 type FlowTrigger struct {
@@ -926,6 +1066,9 @@ type Step struct {
 	// enriched with its Redis operation and key family after store fragments
 	// have been merged into the catalog.
 	StoreAccess *FlowStoreAccess `json:"storeAccess,omitempty"`
+	// Seen counts the recordings that showed this hop. A verifier sets it;
+	// on a step no source declares, it is what put the step here.
+	Seen *StepSeen `json:"seen,omitempty"`
 }
 
 func (*Step) NodeType() string { return "step" }
@@ -985,6 +1128,10 @@ type AltBranch struct {
 	// Without it, the steps drawn after the alt read as if they follow this
 	// branch too.
 	Terminal bool `json:"terminal,omitempty"`
+	// Seen counts the recordings that went this way, on a frame a verifier
+	// wrote where recordings parted. A branch with no steps and a count is
+	// the recordings that went no further.
+	Seen *StepSeen `json:"seen,omitempty"`
 }
 
 type Loop struct {

package/catalog/roundtrip_test.go

@@ -27,6 +27,13 @@ func TestRoundTrip(t *testing.T) {
 	if len(files) == 0 {
 		t.Fatal("no catalogs under ../data - this test proves nothing without one")
 	}
+	// The snapshot a fetcher wrote is a catalog too, and the one place the
+	// deployments field is exercised by a real file.
+	snapshots, err := filepath.Glob("../examples/argocd/argocd.apps.json")
+	if err != nil {
+		t.Fatal(err)
+	}
+	files = append(files, snapshots...)
 
 	for _, path := range files {
 		t.Run(filepath.Base(path), func(t *testing.T) {

package/cli/portolan.mjs

@@ -213,7 +213,9 @@ export async function prepareSite(workspace) {
   rmSync(stage, { recursive: true, force: true });
   mkdirSync(stage, { recursive: true });
 
-  for (const directory of ["src", "public", "scripts"]) {
+  // rules/ rides along: scripts/manifest.mjs and src/lib/problem-rules.ts read
+  // ../rules/builtin.json beside themselves.
+  for (const directory of ["src", "public", "scripts", "rules"]) {
     cpSync(resolve(installRoot, directory), resolve(stage, directory), { recursive: true });
   }
   for (const file of ["index.html", "vite.config.ts", "tsconfig.json"]) {
@@ -382,6 +384,9 @@ function prepareHost(workspace) {
   mkdirSync(host, { recursive: true });
   cpSync(resolve(installRoot, "scripts"), resolve(host, "scripts"), { recursive: true });
   cpSync(resolve(installRoot, "src"), resolve(host, "src"), { recursive: true });
+  // scripts/manifest.mjs and src/lib/problem-rules.ts read ../rules/builtin.json
+  // beside themselves; the host is the package with the same shape.
+  cpSync(resolve(installRoot, "rules"), resolve(host, "rules"), { recursive: true });
   writeFileSync(resolve(host, "package.json"), `${JSON.stringify({ private: true, type: "module", version: VERSION })}\n`);
   symlinkSync(dependencyRoot(), resolve(host, "node_modules"), process.platform === "win32" ? "junction" : "dir");
   return host;

package/go.mod

@@ -3,12 +3,19 @@ module github.com/shortlink-org/portolan
 go 1.27.0
 
 require (
+	github.com/hashicorp/hcl/v2 v2.24.0
 	github.com/pgplex/pgparser v0.2.0
+	github.com/zclconf/go-cty v1.19.0
 	golang.org/x/tools v0.50.0
 	gopkg.in/yaml.v3 v3.0.1
 )
 
 require (
+	github.com/agext/levenshtein v1.2.1 // indirect
+	github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
+	github.com/apparentlymart/go-textseg/v17 v17.0.1 // indirect
+	github.com/mitchellh/go-wordwrap v1.0.1 // indirect
 	golang.org/x/mod v0.41.0 // indirect
 	golang.org/x/sync v0.23.0 // indirect
+	golang.org/x/text v0.25.0 // indirect
 )

package/go.sum

@@ -1,17 +1,31 @@
+github.com/agext/levenshtein v1.2.1 h1:QmvMAjj2aEICytGiWzmxoE0x2KZvE0fvmqMOfy2tjT8=
+github.com/agext/levenshtein v1.2.1/go.mod h1:JEDfjyjHDjOF/1e4FlBE/PkbqA9OfWu2ki2W0IB5558=
+github.com/apparentlymart/go-textseg/v15 v15.0.0 h1:uYvfpb3DyLSCGWnctWKGj857c6ew1u1fNQOlOtuGxQY=
+github.com/apparentlymart/go-textseg/v15 v15.0.0/go.mod h1:K8XmNZdhEBkdlyDdvbmmsvpAG721bKi0joRfFdHIWJ4=
+github.com/apparentlymart/go-textseg/v17 v17.0.1 h1:bpMXRgQ5cEoRNuQke1a80/Nl6w3G5eoIbWo9f3gXkAs=
+github.com/apparentlymart/go-textseg/v17 v17.0.1/go.mod h1:fa8X4jgGeevslICIY6LcdjkSecWnXmYd9Lk34z/VxZs=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/go-test/deep v1.0.3 h1:ZrJSEWsXzPOxaZnFteGEfooLba+ju3FYIbOrS+rQd68=
+github.com/go-test/deep v1.0.3/go.mod h1:wGDj63lr65AM2AQyKZd/NYHGb0R+1RLqB8NKt3aSFNA=
 github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
 github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/hashicorp/hcl/v2 v2.24.0 h1:2QJdZ454DSsYGoaE6QheQZjtKZSUs9Nh2izTWiwQxvE=
+github.com/hashicorp/hcl/v2 v2.24.0/go.mod h1:oGoO1FIQYfn/AgyOhlg9qLC6/nOJPX3qGbkZpYAcqfM=
+github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0=
+github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0=
 github.com/pgplex/pgparser v0.2.0 h1:BB7I/Mrwetiw07PTq/AlXcKfajlCwoJfwB7KZ7V9irI=
 github.com/pgplex/pgparser v0.2.0/go.mod h1:DWhK7oZJn/16KTFeOQ+mk78+sRnEYya1XPghIoP9NFU=
-golang.org/x/mod v0.39.0 h1:UF5zwQdCRRUpHfyPwr7d4UrGiVeldIsogtzWVnczL74=
-golang.org/x/mod v0.39.0/go.mod h1:bvIbwjQ0HUFFf5AKukeeYQG4ZBUG9yxQbR9aEweIwYY=
+github.com/zclconf/go-cty v1.19.0 h1:IV8WdqYZc2c5rLX9bEoLNXKojBAp0MZPBHMIrCoa/s4=
+github.com/zclconf/go-cty v1.19.0/go.mod h1:12W89jGn3JCOIQi7infWr9m80rOkb5RNYJqXMZcN4c8=
+github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940 h1:4r45xpDWB6ZMSMNJFMOjqrGHynW3DIBuR2H9j0ug+Mo=
+github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940/go.mod h1:CmBdvvj3nqzfzJ6nTCIwDTPZ56aVGvDrmztiO5g3qrM=
 golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
 golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
-golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
-golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
 golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
 golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
-golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
-golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
+golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4=
+golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA=
 golang.org/x/tools v0.50.0 h1:c2ifzfcuY7L90lZ2aKd8S4K2NpASF08SZx9ZuJkHmSU=
 golang.org/x/tools v0.50.0/go.mod h1:7ulVMw3831Mwi5EZD6RomGyffr4VFjuNYXf2BbCEAV0=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=

package/package.json

@@ -2,7 +2,7 @@
   "name": "@shortlink-org/portolan",
   "description": "Generate a navigable architecture catalog from code and specifications.",
   "private": false,
-  "version": "0.3.0",
+  "version": "0.4.0",
   "type": "module",
   "license": "MIT",
   "homepage": "https://github.com/shortlink-org/portolan#readme",
@@ -21,6 +21,7 @@
   },
   "files": [
     "cli",
+    "rules",
     "scripts",
     "src",
     "public",
@@ -61,7 +62,7 @@
     "build": "node cli/portolan.mjs build",
     "preview": "vite preview",
     "test": "vitest run",
-    "test:go": "go test ./...",
+    "test:go": "GOFLAGS=-mod=mod go test ./...",
     "pretest:package": "npm run plugins:build",
     "test:package": "node scripts/package-smoke.mjs",
     "test:watch": "vitest",
@@ -69,7 +70,7 @@
     "likec4:gen": "node scripts/gen-likec4.mjs && likec4 gen react likec4 -o src/likec4/generated.jsx --no-use-dot",
     "likec4:validate": "likec4 validate likec4",
     "predev": "npm run likec4:gen",
-    "plugins:build": "GOOS=wasip1 GOARCH=wasm go build -trimpath -ldflags=\"-s -w\" -o plugins/portolan-go.wasm ./plugins/cmd/portolan-go && javac --release 21 -d plugins/extract-java/build plugins/extract-java/src/org/portolan/extract/*.java",
+    "plugins:build": "GOFLAGS=-mod=mod GOOS=wasip1 GOARCH=wasm go build -trimpath -ldflags=\"-s -w\" -o plugins/portolan-go.wasm ./plugins/cmd/portolan-go && javac --release 21 -d plugins/extract-java/build plugins/extract-java/src/org/portolan/extract/*.java && dotnet build plugins/extract-csharp-ddd -c Release --nologo -v q",
     "schema": "npm run plugins:build && node scripts/schema.mjs",
     "schema:check": "npm run plugins:build && node scripts/schema.mjs --check",
     "gen": "npm run plugins:build && node scripts/gen.mjs",
@@ -78,8 +79,8 @@
     "prepack": "npm run plugins:build"
   },
   "dependencies": {
-    "@ai-sdk/openai-compatible": "3.0.47",
-    "@ai-sdk/react": "4.0.100",
+    "@ai-sdk/openai-compatible": "3.0.48",
+    "@ai-sdk/react": "4.0.102",
     "@asyncapi/react-component": "^3.1.8",
     "@clack/prompts": "^1.8.0",
     "@floating-ui/react": "^0.27.20",
@@ -89,16 +90,16 @@
     "@tailwindcss/vite": "^4.0.0",
     "@tanstack/react-query": "^5.102.8",
     "@tanstack/react-table": "^9.2.4",
-    "@tanstack/react-virtual": "^3.14.10",
+    "@tanstack/react-virtual": "^3.14.12",
     "@vitejs/plugin-react": "^6.1.1",
     "@xyflow/react": "^12.11.5",
-    "ai": "7.0.97",
+    "ai": "7.0.99",
     "ajv": "^8.20.0",
     "elkjs": "^0.12.0",
     "html-to-image": "^1.11.13",
     "likec4": "^1.59.2",
     "lowlight": "^3.3.0",
-    "lucide-react": "^1.38.0",
+    "lucide-react": "^1.45.0",
     "mermaid": "^12.0.0",
     "motion": "^13.2.0",
     "oxc-parser": "^0.149.0",
@@ -111,7 +112,7 @@
     "remark-gfm": "^4.0.0",
     "simple-icons": "^16.30.0",
     "tailwindcss": "^4.0.0",
-    "vite": "^8.2.2",
+    "vite": "^8.3.0",
     "vite-plugin-node-polyfills": "^0.28.0",
     "zustand": "^5.0.15"
   },

package/plugin/describe.go

@@ -32,6 +32,40 @@ const (
 	PhaseGenerate = "generate"
 )
 
+// Categories a plugin sorts itself under on the plugin index: what it reads,
+// or what it makes. A phase says where in a run a plugin goes; a category
+// says what kind of fact it is after, which is the question a reader choosing
+// extractors for an estate is asking.
+const (
+	// CategoryCode reads a language: the aggregates, events and use cases a
+	// service declares in its source.
+	CategoryCode = "code"
+	// CategoryContracts reads an interface description - OpenAPI, protobuf,
+	// GraphQL, WSDL, AsyncAPI, a registry's schemas.
+	CategoryContracts = "contracts"
+	// CategoryMessaging reads queues, subjects, topics and the jobs on them.
+	CategoryMessaging = "messaging"
+	// CategoryData reads the stores a service keeps and their shape.
+	CategoryData = "data"
+	// CategoryInfrastructure reads what the service is deployed on - the
+	// queues, topics, functions and stores a Terraform module declares, and
+	// the wiring between them that only the infrastructure knows.
+	CategoryInfrastructure = "infrastructure"
+	// CategoryRepository reads what the repository says about itself: its
+	// metadata, its task runners.
+	CategoryRepository = "repository"
+	// CategoryDocuments reads what people wrote by hand: decisions, glossaries,
+	// flows.
+	CategoryDocuments = "documents"
+	// CategoryEvidence checks the catalog against something outside the code:
+	// traces, ownership rules.
+	CategoryEvidence = "evidence"
+	// CategorySources brings in trees from elsewhere for the extractors to read.
+	CategorySources = "sources"
+	// CategoryExports turns the catalog into something else.
+	CategoryExports = "exports"
+)
+
 // Descriptor is a plugin's answer to KindDescribe.
 type Descriptor struct {
 	// Name is what the plugin calls itself. The manifest is free to declare it
@@ -42,6 +76,10 @@ type Descriptor struct {
 	// Summary is one line, and it is the line an editor shows beside the step.
 	Summary string `json:"summary,omitempty"`
 
+	// Category is one of the Category constants: what the plugin reads or
+	// makes, which is how the plugin index groups it.
+	Category string `json:"category"`
+
 	// Phases are the manifest sections this plugin belongs in.
 	Phases []string `json:"phases"`