@foss.global/codefeed 5.4.0 → 5.5.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@foss.global/codefeed",
-  "version": "5.4.0",
+  "version": "5.5.0",
   "private": false,
   "description": "The @foss.global/codefeed module generates commit/activity feeds from forge providers including Gitea and GitManager.",
   "exports": {
@@ -11,14 +11,14 @@
   "author": "Task Venture Capital GmbH",
   "license": "MIT",
   "devDependencies": {
-    "@foss.global/forgefixtures": "^1.1.0",
-    "@git.zone/tsrepo": "^2.1.0",
+    "@foss.global/forgefixtures": "^1.2.0",
+    "@git.zone/tsrepo": "^2.2.0",
     "@push.rocks/qenv": "^8.2.1",
     "@types/node": "^26.6.6"
   },
   "dependencies": {
     "@foss.global/gitmanager": "^2.4.0",
-    "@foss.global/interfaces": "^3.3.0",
+    "@foss.global/interfaces": "^3.5.1",
     "@push.rocks/lik": "^6.4.4",
     "@push.rocks/smartnpm": "^2.1.3",
     "@push.rocks/smarttime": "^5.0.1"

package/readme.md

@@ -101,13 +101,13 @@ const results = await reader.measureUnits(reader.units(discovery), {
 const inventory = composeGiteaSourceInventory(discovery, results, [{ package: '@foss.global/codefeed', version }]);
 ```
 
-A run is split into units: the instance, each owner and each repository. `measureUnit` never rejects; a unit whose primary record cannot be read becomes `failed` with a sanitized reason. A resumed run measures failed units, and units holding a partial measure, again and passes all results to `composeGiteaSourceInventory`, where the last result per unit wins. The composed document is deterministic: it carries no timestamps, sorts owners, repositories, packages and reasons, and does not depend on the order of the results. An aborted signal or a rejected `onResult` aborts the units in flight, hands over no further result, and rejects `measureUnits` once every worker has stopped.
+A run is split into units: the instance, each owner and each repository, and, when discovery selects them, each package and each repository's collaboration. `measureUnit` never rejects; a unit whose primary record cannot be read becomes `failed` with a sanitized reason. A resumed run measures failed units, and units holding a partial measure, again and passes all results to `composeGiteaSourceInventory`, where the last result per unit wins. The composed document is deterministic: it carries no timestamps, sorts owners, repositories, packages and reasons, and does not depend on the order of the results. An aborted signal or a rejected `onResult` aborts the units in flight, hands over no further result, and rejects `measureUnits` once every worker has stopped.
 
 Discovery records each repository's Gitea `id` as a canonical decimal string, and `units()` passes it into the repository unit. A unit with an `id` fails with `repository-identity-changed` when its name answers with another id, as after a rename and a new repository under the old name; a unit without one, from a discovery stored by an earlier version, is checked by name only. `composeGiteaSourceInventory` counts a repository result for another id than discovery recorded as a failed unit (`repository: unit failed: repository-identity-changed`), and adds `repository: duplicate-id` when two measured repositories share one id, which a discovery without ids cannot rule out.
 
 `lookupRepository(id)` reads one repository by its id through `GET /api/v1/repositories/{id}`, which keeps answering after a rename or transfer. It resolves `{ status: 'present', id, owner, name }` with the name Gitea gives the repository now, or `{ status: 'absent', id }` only when Gitea answers 404, which a site administrator gets only for a deleted repository. Every other failed read rejects with its code, so a timeout or a 500 never counts as a deletion; an id that is not a canonical decimal rejects with `invalid-repository-id` without a request.
 
-What is measured: users with their account flags, two-factor and SSH/GPG keys; organizations, members, teams, team memberships and repository grants; repositories with their flags, Git refs split into branches, tags, pull request refs and other refs, Git objects and their uncompressed bytes, LFS objects referenced anywhere in history with the bytes Gitea holds and the number it lacks, issues, pull requests, comments, reviews, review comments, labels, milestones, reactions and attachments; releases including drafts, assets and asset bytes; wiki pages; collaborators, team grants, SSH deploy keys, deploy tokens, webhooks, branch and tag protections and push mirrors; packages per type with versions, files and bytes; system webhooks, Actions runners, workflows, runs, artifacts and the number of Actions secrets and variables; unadopted repositories.
+What is measured: users with their account flags, two-factor and SSH/GPG keys; organizations, members, teams, team memberships and repository grants; repositories with their flags, Git refs split into branches, tags, pull request refs and other refs, Git objects and their uncompressed bytes, LFS objects referenced anywhere in history with the bytes Gitea holds and the number it lacks, issues, pull requests, comments, reviews, review comments, labels, milestones, reactions and attachments, with comments, reactions and attachments also split by kind (`collaboration.byKind`) and pull requests counted as open, closed and merged; project boards per repository and per owner; releases including drafts, assets and asset bytes; wiki pages; collaborators, team grants, SSH deploy keys, deploy tokens, webhooks, branch and tag protections and push mirrors; packages per type with versions, files and bytes; system webhooks, Actions runners, workflows, runs, artifacts and the number of Actions secrets and variables; unadopted repositories.
 
 Beyond counts, each repository keeps records of what the migration recreates. `description` is the repository's description exactly as Gitea's API reports it, without trimming or normalising; `''` means the source has none. A description that is not a string fails the repository unit with `invalid-string`, so the inventory stays incomplete; reports from readers before 4.4.0 carry no `description`, which a consumer treats as unread, never as `''`. `releaseRecords` lists every release, drafts and prereleases included, sorted by numeric id, as `TForgeSourceRelease` records (narrow on `draft` before reading `publishedAt` as a string): tag name, target commitish, name, body, the draft and prerelease flags, created and published times as Gitea reports them (a published release must carry a valid publication time; a draft's is recorded as reported, `null` on Gitea 28 and never replaced by its creation time, and a draft's tag does not exist as a Git ref yet; on Gitea 28 a new release's creation time is the date of the commit it points at), an author snapshot (account id and login; Gitea's placeholder for a deleted account has id `-1`) and its assets sorted by numeric id with id, storage `uuid`, name, size, download count and creation time. Gitea exposes no content digest, so a record carries none; a replaced file carries another id and uuid, and the download count moves with every download, so it never decides whether a release changed. `protections` holds the branch protection rules sorted by rule name (code-unit order) with every restriction Gitea reads back, allowlists included, and the tag protection rules sorted by numeric id with their name pattern and allowlist. `releaseRecords` is null while a release count is not complete, and `protections` is null while either protection count is not; the counts then say why.
 
@@ -124,9 +124,33 @@ Secrets are counted, never named: the report carries no tokens, Actions secret o
 Git-level measures come from a mirror clone of every ref Gitea advertises, `refs/pull/*` included, made with `@foss.global/gitmanager` in a fresh directory below `gitWorkDir` and removed afterwards. The reader's login and token form Basic credentials; the credential reaches Git only through GIT_CONFIG_* entries in the child process environment; it is never in the URL, the command line or any file. The private CA list applies to the clone as well. The mirror's HEAD becomes the repository's `head`: the ref it names as `target` and whether it `resolves`. In a repository that is not `empty`, a HEAD that does not resolve names a missing branch, such as a default branch deleted at the source, and a `target` other than `refs/heads/<defaultBranch>` means Gitea's setting and its Git HEAD disagree. `head` is null when the mirror or its refs could not be read. A detached source HEAD is one such case: GitManager's mirror import requires a symbolic HEAD, so the clone fails with `git-mirror-failed`, and this reader never reports a `target` of null. The same mirror records `refDigests`: for each of `refs/heads/`, `refs/tags/` and `refs/notes/`, GitManager's ref `inventoryDigest` (HEAD and the refs under that prefix) and the `refCount` of that selection, read as one single-ref `listRefs` page per prefix; codefeed computes no digest of its own. Any other GitManager repository with the same HEAD and refs under a prefix reports the same digest, so a migration compares them with its native copy. Pull request refs have no digest, and `git.otherRefs - refDigests.notes.refCount` refs lie outside every digested namespace. `refDigests` is null whenever `head` is, or when any of the three reads fails; it is never partial. The mirror's distinct LFS pointers, from every reachable revision, are checked against Gitea in batches of 100: an object with a download action counts as held, one answered with error 404 as missing, and any other answer makes the LFS measures `partial`. A clone that fails or exceeds `gitTimeoutMs` makes every Git and LFS measure of that repository `partial`; only the API-reported size stays complete. A clone starts only once its scratch reservation, twice the reported size plus 64 MiB, fits the free space of the file system that holds `gitWorkDir`, counting the reservations of the clones already in flight from every reader in the process. A clone that does not fit waits for them, in arrival order; one that does not fit even alone makes every Git and LFS measure of its repository `partial` with `git-workdir-too-small`, so the inventory stays incomplete. Size `gitWorkDir` for the largest repositories measured at once.
 
 Known gaps that keep an inventory incomplete:
-- A repository with its issues or pull requests unit disabled hides both through the API, so its issues, pull requests, comments, reviews, reactions and attachments are `notMeasured`. A disabled wiki hides its pages the same way. Labels, milestones, releases and Actions stay readable while their units are disabled.
+- Gitea's issue and repository comment listings leave out the kind whose unit is disabled (Gitea 28.2.0 `ListIssues`, `ListRepoIssueComments`) and answer 404 only when both are. Each kind is therefore measured on its own: with the pull request unit disabled, `issues` and every issue measure of `collaboration.byKind` stay measured, and only the pull request measures (`pullRequests`, `reviews`, `reviewComments`, the pull request kinds and states of `byKind`) and the mixed measures that include them (`comments`, `reactions`, `attachments`, `attachmentBytes`) are `notMeasured`, and the reverse. An issue unit configured as an external tracker keeps `features.issues` true but lists no issue, so its issue measures are `notMeasured` too, never a complete zero. A repository record whose `has_issues` disagrees with its trackers fails the repository unit with `inconsistent-issue-unit`, and one whose `projects_mode` is not `repo`, `owner` or `all` with `invalid-projects-mode` (`invalid-string` when it is missing); Gitea 28.2.0 cannot produce either state, so they mean a forge the reader was not built against (S8 U3 ruling, Q3). A disabled wiki hides its pages, and a projects unit that allows only owner boards (`projects_mode` `owner`) hides the repository's boards (`collaboration.projects`), which Gitea still serves while the projects unit is disabled. Labels, milestones, releases and Actions stay readable while their units are disabled.
 - Gitea 28 answers 404 to every pull request route of an archived repository, reads included. The reviews of each pull request are still requested; when an archived repository answers 404 there, `reviews` and `reviewComments` are `partial` with the reason `repository-archived` (in `incompleteReasons` as `repository.collaboration.reviews: repository-archived`). Its issues, pull requests, comments and `refs/pull/*` stay measured. A 404 in a repository that is not archived stays `http-404`.
 
+#### Collaboration records
+
+Issues are recorded only when discovery selects the collaboration class: `reader.discover(signal, { classes: ['collaboration'] })` sets `discovery.collaboration: true`, and `units()` then adds one `{ kind: 'collaboration', owner, name, id }` unit per repository, keyed by the repository's discovered id (S8 ruling D12). A discovery that selects the class while a repository has no id rejects in `units()` with `repository-id-missing`, and composing it names `collaboration: repository-id-missing`; it is never keyed by name. Without the class, discovery carries no `collaboration`, no collaboration unit is measured, and the composed inventory has neither `collaboration` nor any organization's `labelRecords`, so the records are unread. The classes combine: `{ classes: ['packages', 'collaboration'] }`.
+
+A collaboration unit returns an `IForgeSourceCollaborationInventory` (`@foss.global/interfaces` 3.4), read with GETs and the administrator's token, never through `Sudo`:
+
+- The repository must still answer its name with the discovered id, else the unit fails with `repository-identity-changed`.
+- `labels` is every label of the repository, used or not, and `milestones` every milestone (`state=all`), each sorted by numeric id. Organization labels are not repeated here: each organization owner records them as `labelRecords`, and an issue's `labelIds` may name either.
+- `highestNumber` is the highest number Gitea's issue listing holds, issues and pull requests together, since they share one sequence; `0` when it lists none, and not measured when both units are disabled. A deleted number, or one a disabled unit hides, is invisible to every read, so Gitea may have allocated higher ones (S8 ruling D1).
+- `issues` holds every issue, never a pull request, sorted by number, with its comments sorted by id. Title, body, comment bodies, logins and `ref` are verbatim, never trimmed; times are Gitea's RFC 3339 strings as served. Each issue records its poster, Gitea's original author (null when it reports none), assignees, label, milestone and project ids, `locked`, `pinOrder`, `dueDate`, `timeEstimate` and the counts of its tracked times (`/times`), dependencies (`/dependencies`), attachments and reactions; each comment its poster, original author, times and attachment and reaction counts. Gitea's `content_version`, a counter for concurrent edits, is not recorded.
+- `issues` is `notMeasured`, and no issue's comments, reactions, tracked times or dependencies are requested, while Gitea's API hides them: when the issue unit is disabled or an external tracker, when time tracking is switched off for the repository (`internal_tracker.enable_time_tracker`) or the instance (`/settings/repository` `time_tracking_disabled`), or when dependencies are switched off (`internal_tracker.enable_issue_dependencies`). Gitea 28.2.0 then answers 404 to every issue's `/times` or `/dependencies`, and a refused read must never count as none (S8 U1 ruling). Each such case leaves the repository's issues to the owner's per-repository decision (S8 ruling D13); switching the setting on at the source resolves it. Labels and milestones stay readable in every case.
+- `issues` is also `notMeasured`, again without a read of any issue, when Gitea imported the repository (its record's `original_url` is not empty) and the listing holds at least one issue, with the reason "Gitea imported this repository (original_url is set); its API names the importing account as poster and never reports the original author." Gitea's API names the account that ran the import as the poster of every imported issue and comment and never reports the original author (see the facts below), so recording them would credit the wrong account (S8 U3 ruling, Q1). Comments are part of their issues, so the rule covers them. An imported repository without issues keeps a complete empty list. The rule does not depend on `mirror`: a mirror converted to a normal repository keeps its `original_url`. `original_url` itself is never recorded, and a record without it as a string fails the unit with `invalid-string`. A forge whose API fills the original author has it recorded as `originalAuthor`.
+- A unit is read whole or fails, never partly: any listing that does not finish fails it with the listing's reason, and so do an issue naming a milestone the listing lacks (`issue-milestone-unlisted`), an issue number listed twice (`duplicate-number`), an original author id without a name (`invalid-original-author`) and a result over `forgeSourceInventoryUnitMaxBytes` (`unit-over-bound`). A resumed run measures a failed unit again.
+
+`composeGiteaSourceInventory` carries `collaboration`, sorted by owner and name, and every organization's `labelRecords` (null while its `labels` count is not complete) exactly when discovery selects collaboration. A unit that failed, was not measured or answers for another name adds `collaboration: unit failed: <code>` or `collaboration: unit not measured`, and a list that is `notMeasured` adds its path, such as `collaboration.issues: <reason>`. Owner and repository results stored by a reader before this version lack `labelRecords`, `projects` and `collaboration.byKind`; composing them names each as a reason, so the inventory stays incomplete until those units are measured again.
+
+Gitea 28.2.0 facts behind the records, verified with the forgefixtures Gitea:
+- Label and milestone listings are not gated by the issue unit: they answer while it is disabled (the routes carry no unit check, `routers/api/v1/api.go`).
+- `/issues/{index}/times` answers 404 while time tracking is off for the repository, and `/issues/{index}/dependencies` while dependencies are off.
+- `/orgs/{org}/projects` and `/users/{user}/projects` answer the site administrator's token for private organizations and users alike. A repository's `/projects` keeps listing its boards while its projects unit is disabled (Gitea then reports `projects_mode` `all`) and answers 404 while `projects_mode` is `owner`.
+- Gitea applies `internal_tracker` only beside `has_issues: true`, and `projects_mode` only beside `has_projects: true`, in a repository edit.
+- An issue's comments and a comment's reactions are served whole, without pages (`ListIssueComments` takes no page options); the reader requests them once and checks the rows against `X-Total-Count` when Gitea sends it.
+- Gitea's REST API never fills `original_author` or `original_author_id` of an issue or comment (Gitea 28.2.0 `services/convert/issue.go` `toIssue` and `services/convert/issue_comment.go` `ToAPIComment` do not set them), so `originalAuthor` is null for all content. Content Gitea itself imported from another service keeps its source authors only in Gitea's database: through the API every such issue and comment reads as posted by the account that ran the import (`services/migrations/gitea_uploader.go`), as the fixture test shows with an import from Gitea's own loopback listener. This is why an imported repository's issues are `notMeasured` rather than recorded.
+
 #### Packages
 
 Packages are measured only when discovery selects their class: `reader.discover(signal, { classes: ['packages'] })` lists every package from the administrator package listing (`/admin/packages`, user-owned packages included) as `discovery.packages`, and `units()` then adds one unit per package. Without the class, discovery carries no `packages`, no package unit is measured, and the composed inventory has neither `packages` nor `totals.packageContent`, so the package class is unread; a stored discovery from an earlier reader composes as before. An unknown class rejects with `invalid-options`.

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@foss.global/codefeed',
-  version: '5.4.0',
+  version: '5.5.0',
   description: 'The @foss.global/codefeed module generates commit/activity feeds from forge providers including Gitea and GitManager.'
 }