@www.hyperlinks.space/program-kit 18.18.18 → 123.123.123

package/research & docs/github-gitlab-bidirectional-mirroring.md

@@ -0,0 +1,154 @@
+# GitHub and GitLab Repository Mirroring
+
+This document describes how to keep a Git repository synchronized between GitHub and GitLab, including **bidirectional** setups, the problems that appear in practice, **automation guards** that reduce loops and wasted work, and **automatic escalation** when the automation must not guess (for example, divergent histories).
+
+It is guidance for operators and automation authors; it is not specific to application code in this repository.
+
+## Terminology
+
+- **Canonical remote:** The single place where humans are expected to push day-to-day changes (or where merge commits land after review). Everything else is a **mirror**.
+- **One-way mirror:** After a push to A, automation updates B. Humans do not push to B for the same branches (or B is protected so only the mirror bot can push).
+- **Bidirectional mirror:** Automation tries to propagate pushes from A→B and B→A. This is workable only with strict guards and a clear policy when histories **diverge**.
+- **Escalate:** Stop silent auto-fix; surface the condition to people or ticketing (failed job, alert, issue). See [Automatic escalation](#automatic-escalation).
+
+## Recommended default: one canonical remote + one-way mirror
+
+For most teams, the lowest-risk approach is:
+
+1. Choose **one** platform as the source of truth for each protected branch (usually `main`).
+2. Mirror to the other platform with **fast-forward-only** pushes and **short-circuit** when tips already match.
+
+Typical implementations (pick **one** row: canonical host is where humans push; the arrow is where commits are **copied** next):
+
+| Canonical | Replication direction | How |
+|---|---|---|
+| **GitHub** | GitHub → GitLab | GitHub Actions on `push`: fetch GitLab, compare SHAs, `git push` to GitLab using a [GitLab deploy key](https://docs.gitlab.com/ee/user/project/deploy_keys/) or [project access token](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html). |
+| **GitLab** | GitHub ← GitLab | [GitLab repository push mirroring](https://docs.gitlab.com/ee/user/project/repository/mirror/) so GitLab pushes to GitHub after receiving commits. |
+
+The two directions are **not** used together for the same branch in this pattern: they are **alternatives** depending on which platform is source of truth. (Using both at once without guards is how bidirectional mirroring starts.)
+
+Official overview: [GitLab repository mirroring](https://docs.gitlab.com/ee/user/project/repository/mirror/), [GitHub Actions push trigger](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#push).
+
+This gives **near-instant** propagation from the moment the canonical host accepts the push, without fighting bidirectional edge cases.
+
+## Bidirectional mirroring: when it is used and what breaks
+
+Bidirectional setups appear when both platforms must accept pushes (different teams, migration, or CI only on one side). They are **not** “set and forget.”
+
+| Risk | What goes wrong | Why it matters |
+|---|---|---|
+| **Loop / ping-pong** | A push to GitHub triggers sync to GitLab; GitLab triggers sync back to GitHub; repeat or duplicate work | Duplicate CI, wasted API calls, flaky pipelines |
+| **New objects from automation** | Mirror job merges, rebases, or force-pushes to “fix” state | New commits or rewritten history; harder audits; can feed loops |
+| **Divergence** | Two different commits on `main` at each host, neither is ancestor of the other | `git push` rejects (non-fast-forward) unless someone force-pushes |
+| **Duplicate CI** | Same tree built twice because both hosts run pipelines on the same logical change | Cost and noise; confusing status checks |
+| **Credential exposure** | Tokens in logs, overly broad PATs, shared user accounts | Security and compliance issues |
+
+## Guards in automation (layered)
+
+Use several of these together; one guard rarely covers every failure mode.
+
+### 1. Event and scheduling guards
+
+- **SHA equality short-circuit:** After fetching both remotes, if `github/main` and `gitlab/main` are the **same commit OID**, exit successfully without pushing.
+- **Ancestor / fast-forward check before push:** Only push from A to B if B’s tip is an **ancestor** of A’s tip (pure fast-forward). If not, do not force-push; treat as divergence and [escalate](#automatic-escalation).
+- **Mirror bot identity:** Perform mirror pushes with a **dedicated** bot user or deploy key. On the receiving side, webhook or pipeline logic can **skip enqueuing reverse sync** when the actor matches the mirror bot (where the platform exposes this).
+- **Idempotency:** Persist “last mirrored OID per branch” (CI cache, KV, issue comment, etc.) and skip if the incoming event’s `after` SHA was already mirrored.
+- **Debounce / concurrency:** One in-flight sync per branch (GitHub Actions `concurrency`, GitLab `resource_group`, or a lock) to avoid stacked identical jobs from webhook retries.
+
+### 2. Git protocol guards
+
+- **No `--force` on mirrored branches** unless you have an explicit, rare break-glass procedure. Non-fast-forward should **fail** and escalate.
+- **Narrow refspecs:** Mirror only agreed branches (for example `refs/heads/main`, `refs/heads/release/*`), not all refs.
+- **Atomic multi-ref push** when supported (`git push --atomic`) to avoid half-updated mirror state.
+
+### 3. Loop-specific logic
+
+- **Directional rule:** GitHub→GitLab runs only when GitHub is strictly ahead of GitLab (fast-forward). GitLab→GitHub runs only when GitLab is strictly ahead. If **both** are ahead of the common ancestor (diverged), **neither** direction should force-sync; escalate.
+- **Cooldown:** If the same `(branch, OID)` was mirrored within N seconds, exit (reduces double webhook storms).
+
+### 4. CI noise controls
+
+Mirroring duplicates pipelines if both hosts run on every push.
+
+- Prefer **one** platform for **required** checks; treat the other as informational or limit runs (for example scheduled smoke only).
+- Where policy allows, use host-specific **skip directives** for mirror-only pushes (conventions vary; confirm org rules).
+- **Path filters** help only when mirror commits change nothing meaningful; usually both sides see the same files.
+
+### 5. Security guards
+
+- **Least privilege:** Repo-scoped tokens or deploy keys; no org-wide admin tokens for mirroring.
+- **Branch protection** on both sides: block force-push to mirrored branches; restrict who may push to protected branches.
+- **Secrets hygiene:** Mask remotes in logs; disable interactive prompts (`GIT_TERMINAL_PROMPT=0`); store credentials only in CI secrets.
+
+## Automatic escalation
+
+**Escalate** means: the automation **detects a condition it must not resolve alone**, marks the run as failed or blocked, and **notifies** or **files a ticket** with enough context for a human to decide.
+
+This is the opposite of silently force-pushing or auto-merging without policy.
+
+### When to escalate automatically
+
+Typical conditions:
+
+- **Divergence:** `main` on GitHub and GitLab point to different commits and neither is an ancestor of the other (merge-base exists but both tips have unique commits).
+- **Non-fast-forward push rejected:** Mirror push fails with non-FF; do not retry with force.
+- **Repeated failures:** Same branch fails N times in a row (may indicate permission, quota, or hook problems).
+- **Unexpected ref state:** Missing branch, empty repo, or OID mismatch after push (verify step fails).
+
+### What “automatic escalation” can do (concrete patterns)
+
+Implement one or more of the following from the same job that detected the problem:
+
+1. **Fail the pipeline** with a clear title and non-zero exit code so the default branch protection and dashboards show breakage.
+2. **Post a structured comment** on a pinned tracking issue (GitHub Issues / GitLab issue) including: branch name, both OIDs, link to the failed pipeline, and a one-line `git` hint (`git merge-base`, `git log --left-right`) for the on-call engineer.
+3. **Send chat or email** via webhook (Slack incoming webhook, Microsoft Teams, email SMTP API) with the same summary fields.
+4. **Open a new issue** (if API token allows) with label `mirror-divergence` so work is tracked and deduplicated (search for open issues with the same branch before creating).
+5. **Set a repository flag** (for example GitHub Actions variable or GitLab CI variable via API) `MIRROR_HALTED=true` and have sync jobs check it at the start so you do not amplify divergence while humans fix it.
+
+### Content to include in escalation messages
+
+- Repository identifiers (both URLs or slugs).
+- Branch (or ref) name.
+- **OID on side A** and **OID on side B** (full hashes avoid ambiguity).
+- **Pipeline run URL** and job name.
+- Short **runbook pointer** (internal doc link) for “how we pick canonical history” and “who may force-push if ever.”
+
+### After humans resolve divergence
+
+Runbook should require:
+
+1. Agreement on which tip (or merged result) is correct.
+2. A single deliberate push to the **canonical** remote (or fast-forward merge on one side then sync).
+3. Clear `MIRROR_HALTED` reset and verification that both tips match before re-enabling bidirectional automation.
+
+## Implementation sketches (non-prescriptive)
+
+### GitHub Actions (push to GitLab)
+
+- Trigger: `on: push` for selected branches.
+- Steps: clone with depth sufficient for ancestry check (or full clone for simplicity), add GitLab remote with token in secret, `fetch` GitLab, compare OIDs / ancestry, `git push` only on fast-forward.
+- On divergence or push failure: exit 1 and call a small script that opens an issue or posts to Slack.
+
+### GitLab push mirror to GitHub
+
+- Configure in GitLab project settings; use a GitHub **fine-grained PAT** or deploy key with write access to the target repo.
+- Complement with **branch protection** on GitHub and monitoring for mirror errors in GitLab.
+
+### Webhook-driven bidirectional
+
+- Each side receives push webhooks; each handler runs the same **guarded** sync script (SHA short-circuit, bot skip, fast-forward only, escalate on divergence).
+- **Do not** run two independent scripts that both force-push without shared policy.
+
+## Summary
+
+- **Best default:** one canonical remote, **one-way** mirror with fast-forward-only pushes and OID short-circuit.
+- **Bidirectional** is possible with **layered guards** (ancestor checks, bot identity, concurrency, no force).
+- **Escalate automatically** on divergence and non-FF: fail visibly, notify, optionally open issues and halt further sync until humans reconcile.
+
+## References
+
+- [GitLab: Repository mirroring](https://docs.gitlab.com/ee/user/project/repository/mirror/)
+- [GitHub Actions: Workflow triggers](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows)
+- [GitLab: Deploy keys](https://docs.gitlab.com/ee/user/project/deploy_keys/)
+- [GitLab: Project access tokens](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html)
+- [GitHub: Fine-grained personal access tokens](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-fine-grained-personal-access-token)

package/research & docs/keys-retrieval-console-scripts.js

@@ -0,0 +1,131 @@
+/**
+ * Telegram Mini App key retrieval checker (paste into DevTools console).
+ *
+ * Purpose:
+ * - Inspect where wallet keys are currently stored after app flow runs.
+ * - Check these keys across available storages:
+ *   1) SecureStorage: wallet_master_key
+ *   2) DeviceStorage: wallet_master_key
+ *   3) CloudStorage:  wallet_seed_cipher
+ *
+ * What it prints:
+ * - Per-storage availability (API object + method presence).
+ * - Read result for each key (err/value present/length).
+ * - Final quick verdict about expected Desktop fallback model:
+ *   - secureOnly: master key in SecureStorage
+ *   - deviceFallback: master key in DeviceStorage
+ *   - none: master key not found in either local store
+ *
+ * Notes:
+ * - This script does not print full secret values; it shows only existence/length.
+ * - In some Telegram Desktop builds, SecureStorage methods exist but return UNSUPPORTED.
+ */
+(async () => {
+  const wa = window.Telegram?.WebApp;
+  if (!wa) {
+    const out = { ok: false, error: "Telegram.WebApp not found" };
+    console.log("[keys-retrieval]", out);
+    return out;
+  }
+
+  const readSecure = (key) =>
+    new Promise((resolve) => {
+      const storage = wa.SecureStorage;
+      if (!storage || typeof storage.getItem !== "function") {
+        resolve({ present: false, err: "NO_API", value: null, canRestore: null });
+        return;
+      }
+      storage.getItem(key, (err, value, canRestore) => {
+        resolve({
+          present: true,
+          err: err ?? null,
+          value: value ?? null,
+          canRestore: canRestore ?? null,
+        });
+      });
+    });
+
+  const readDevice = (key) =>
+    new Promise((resolve) => {
+      const storage = wa.DeviceStorage;
+      if (!storage || typeof storage.getItem !== "function") {
+        resolve({ present: false, err: "NO_API", value: null });
+        return;
+      }
+      storage.getItem(key, (err, value) => {
+        resolve({
+          present: true,
+          err: err ?? null,
+          value: value ?? null,
+        });
+      });
+    });
+
+  const readCloud = (key) =>
+    new Promise((resolve) => {
+      const storage = wa.CloudStorage;
+      if (!storage || typeof storage.getItem !== "function") {
+        resolve({ present: false, err: "NO_API", value: null });
+        return;
+      }
+      storage.getItem(key, (err, value) => {
+        resolve({
+          present: true,
+          err: err ?? null,
+          value: value ?? null,
+        });
+      });
+    });
+
+  const [secureMaster, deviceMaster, cloudSeedCipher] = await Promise.all([
+    readSecure("wallet_master_key"),
+    readDevice("wallet_master_key"),
+    readCloud("wallet_seed_cipher"),
+  ]);
+
+  const hasSecureMaster = secureMaster.value != null && secureMaster.err == null;
+  const hasDeviceMaster = deviceMaster.value != null && deviceMaster.err == null;
+  const hasCloudSeedCipher = cloudSeedCipher.value != null && cloudSeedCipher.err == null;
+
+  const localTier = hasSecureMaster ? "secureOnly" : hasDeviceMaster ? "deviceFallback" : "none";
+
+  const result = {
+    ok: true,
+    webApp: {
+      platform: wa.platform ?? null,
+      version: wa.version ?? null,
+    },
+    keys: {
+      wallet_master_key: {
+        secureStorage: {
+          available: secureMaster.present,
+          err: secureMaster.err,
+          exists: hasSecureMaster,
+          valueLength: typeof secureMaster.value === "string" ? secureMaster.value.length : 0,
+          canRestore: secureMaster.canRestore ?? null,
+        },
+        deviceStorage: {
+          available: deviceMaster.present,
+          err: deviceMaster.err,
+          exists: hasDeviceMaster,
+          valueLength: typeof deviceMaster.value === "string" ? deviceMaster.value.length : 0,
+        },
+      },
+      wallet_seed_cipher: {
+        cloudStorage: {
+          available: cloudSeedCipher.present,
+          err: cloudSeedCipher.err,
+          exists: hasCloudSeedCipher,
+          valueLength: typeof cloudSeedCipher.value === "string" ? cloudSeedCipher.value.length : 0,
+        },
+      },
+    },
+    verdict: {
+      localTier,
+      modelOkForDesktopFallback: localTier === "deviceFallback" && hasCloudSeedCipher,
+    },
+  };
+
+  console.log("[keys-retrieval]", result);
+  return result;
+})();

package/research & docs/npm-release.md

@@ -0,0 +1,46 @@
+# Milestone snapshot package (npm)
+
+This repository includes a publishable snapshot package for fast developer bootstrap:
+
+- package source: repository root (published directly)
+- **npmjs (public):** `@www.hyperlinks.space/program-kit` — manage org and tokens: [www.hyperlinks.space on npm](https://www.npmjs.com/settings/www.hyperlinks.space/packages)
+- **GitHub Packages:** `@hyperlinksspace/program-kit` (same version; GitHub requires the package scope to match this repo's owner)
+
+## Verify publish payload locally
+
+The npm package page uses `README.md` from the published tarball, not `npmReadMe.md`. The published package also includes **`fullREADME.md`**, a copy of the developer readme (saved before the npm readme replaces `README.md`). Match CI before `npm pack`, then restore:
+
+```bash
+cp README.md fullREADME.md
+cp npmReadMe.md README.md
+npm pack --dry-run
+git checkout -- README.md
+rm -f fullREADME.md
+```
+
+## Install snapshot as a developer
+
+```bash
+npx @www.hyperlinks.space/program-kit ./my-hsp-app
+```
+
+The CLI materializes the bundled package payload into your target folder, then you run:
+
+```bash
+cd my-hsp-app
+npm install
+```
+
+## Release channels
+
+- `latest`: immutable stable snapshots (tag workflow `snapshot-vX.Y.Z`)
+- `next`: rolling snapshots from manual workflow dispatch
+
+In the output, you'll find options to open the app in:
+
+- [a development build](https://docs.expo.dev/develop/development-builds/introduction/)
+- [an Android emulator](https://docs.expo.dev/workflow/android-studio-emulator/)
+- [an iOS simulator](https://docs.expo.dev/workflow/ios-simulator/)
+- [Expo Go](https://expo.dev/go), a limited sandbox for trying out app development with Expo
+
+You can start developing by editing the files inside the **app** directory. This project uses [file-based routing](https://docs.expo.dev/router/introduction).

package/research & docs/releases.md

@@ -0,0 +1,201 @@
+# GitHub Releases Automation Plan (Final)
+
+This document defines the production plan for release detection, deduped GitHub Release publishing, and near-instant app update signaling.
+
+## Goals
+
+- Build the Windows installer on GitHub Actions (`windows-latest`) with `npm run build:win` (no `.exe` committed to git).
+- Publish the installer to a GitHub Release only when that release tag does not already exist.
+- Notify an app-update service immediately after a new release is published.
+- Keep app-side update detection near-instant using push notification plus fallback polling.
+
+## Source of Truth
+
+- The **GitHub Release tag** is the release identity (`release_id`), for example `build_03252026_1929`.
+- Locally, `windows/cleanup.cjs` moves the built installer to `releases/builder/build_MMDDYYYY_HHMM/HyperlinksSpaceProgramInstaller_<stamp>.exe` (other artifacts under `dev/`). In CI you can set `RELEASE_BUILD_ID` to match a chosen tag, or leave it unset so the folder name comes from build time.
+
+## Workflow Trigger
+
+- **Manual only:** `workflow_dispatch` (Actions → “Windows release (CI build)”).
+- Optional input **release_id** (`build_MMDDYYYY_HHMM`). If empty, the tag is taken from the build output folder after `cleanup.cjs` runs.
+
+Example trigger:
+
+```yaml
+on:
+  workflow_dispatch:
+    inputs:
+      release_id:
+        description: "Optional tag, e.g. build_03252026_1929"
+        required: false
+        default: ""
+```
+
+## Dedupe Rules (No Duplicate Releases)
+
+1. Run `npm run build:win` (or use optional `RELEASE_BUILD_ID` so the output folder matches the intended tag).
+2. Resolve `release_id` from the optional input or from `releases/builder/build_*/HyperlinksSpaceProgramInstaller_*.exe`.
+3. Check whether GitHub Release/tag already exists for that `release_id`.
+4. If it exists:
+   - Exit successfully (`0`)
+   - Do not upload assets
+   - Do not send webhook notification
+5. If it does not exist:
+   - Create GitHub Release/tag
+   - Upload all files from that folder as release assets
+   - Continue to webhook notification
+
+Recommended extra safety:
+
+- Use workflow `concurrency` keyed by `release_id` to avoid race conditions.
+
+## Endpoint Contract
+
+- Webhook endpoint path: `api/releases.ts`
+- Route URL: `POST /api/releases`
+- Purpose: accept release-published events from GitHub Actions and fan out update notifications to app clients.
+
+## Security
+
+- GitHub Actions sends auth header:
+  - `x-release-token: <secret>`
+- Endpoint validates against:
+  - `process.env.RELEASE_WEBHOOK_TOKEN`
+- Optional hardening:
+  - Add HMAC signature verification for request body.
+
+If auth fails:
+
+- Return `401` and do not process payload.
+
+## Payload Shape
+
+`POST /api/releases` expects JSON:
+
+```json
+{
+  "release_id": "build_03252026_1929",
+  "version": "1.0.0",
+  "published_at": "2026-03-25T19:29:00Z",
+  "platform": "windows",
+  "assets": [
+    {
+      "name": "HyperlinksSpaceProgramInstaller.exe",
+      "url": "https://github.com/<org>/<repo>/releases/download/build_03252026_1929/HyperlinksSpaceProgramInstaller.exe",
+      "sha256": "<optional>"
+    }
+  ],
+  "github_release_url": "https://github.com/<org>/<repo>/releases/tag/build_03252026_1929"
+}
+```
+
+## Endpoint Behavior (`api/releases.ts`)
+
+1. Accept only `POST`.
+2. Validate auth token/signature.
+3. Parse and validate required fields:
+   - `release_id`, `published_at`, `assets`.
+4. Enforce idempotency by `release_id`:
+   - If already processed, return `200 { ok: true, duplicate: true }`.
+5. Store/update latest release metadata in persistent storage.
+6. Push update signal to clients (WebSocket/SSE/Firebase/Expo push).
+7. Return quickly with `200 { ok: true }`.
+
+## GitHub Actions Notification Step
+
+After successful release creation and asset upload:
+
+1. Read webhook URL and token from repository secrets:
+   - `RELEASE_WEBHOOK_URL`
+   - `RELEASE_WEBHOOK_TOKEN`
+2. Send `POST` to `/api/releases` with release payload.
+3. Retry webhook call with backoff on transient failures.
+
+Important:
+
+- Do not send webhook if release already existed (dedupe branch).
+
+## When You Must Make a New Installer Release
+
+Use this section as the decision rule between OTA update and installer release.
+
+### Native/runtime changes (installer required)
+
+Create a new installer release when a change affects native binaries or runtime compatibility, including:
+
+- Installing, removing, or updating any package that forces you to rebuild the app.
+- Changing app permissions or native capabilities (camera, notifications, background modes, deep links, etc.).
+- Changing package identifiers, signing, entitlements, or other platform build settings.
+- Changing Expo SDK or React Native version in a way that changes native runtime.
+- Changing `runtimeVersion` policy/behavior or bumping app version when runtime compatibility changes.
+- Any change that requires running a fresh native build to take effect.
+
+### Non-native changes (OTA only, no installer)
+
+Do not make a new installer release for:
+
+- JavaScript/TypeScript business logic changes.
+- UI/layout/style changes.
+- Text/copy changes and static asset updates that are OTA-compatible.
+- Server/API behavior changes that do not require new native modules in app.
+
+### Exact release checklist
+
+Make a new installer release if at least one statement is true:
+
+1. "This change cannot work without rebuilding native binaries."
+2. "This change modifies runtime compatibility between app binary and updates."
+3. "This change touches native permissions/capabilities/config."
+
+If all three are false, ship via OTA update instead of installer release.
+
+### Team policy
+
+- Prefer OTA by default for speed.
+- Use installer releases only for native/runtime boundaries.
+- When uncertain, treat as installer-required and verify in staging before production.
+
+## App Update Strategy
+
+Primary strategy:
+
+- Real-time push signal from backend triggered by `/api/releases`.
+
+Fallback strategy:
+
+- App checks latest release on:
+  - app foreground/resume
+  - periodic interval (for example every 5-15 minutes)
+
+Client behavior:
+
+1. Compare local build/version with latest server metadata.
+2. If newer exists:
+   - Show "Update available" prompt or trigger controlled update flow.
+
+## Reliability and Observability
+
+- Idempotent handling on `release_id`.
+- Structured logs for each stage:
+  - detected -> published/skipped -> webhook sent -> app signal broadcast
+- Alert on partial failure:
+  - release created but webhook failed
+- Keep webhook processing fast and non-blocking for external calls.
+
+## Rollout Plan
+
+1. Deploy `api/releases.ts` in staging.
+2. Run workflow in dry-run mode to validate folder parsing and dedupe checks.
+3. Enable real release creation for test folder.
+4. Enable webhook call to staging endpoint.
+5. Verify end-to-end with one client device.
+6. Promote to production after stable validation.
+
+## Acceptance Criteria
+
+- A new `releases/builder/build_.../` folder produces exactly one GitHub Release.
+- Re-running workflow for the same `release_id` does not create duplicates.
+- Webhook is called only for newly created releases.
+- `POST /api/releases` rejects unauthorized requests.
+- Connected app receives update signal near-instantly in normal conditions.
+- Fallback polling still discovers updates if push delivery fails.