@getvitops/cli 5.0.0 → 7.0.0

package/CHANGELOG.md

@@ -1,5 +1,126 @@
 # @getvitops/cli
 
+## 7.0.0
+
+### Minor Changes
+
+- c0c092b: `vitops search` now works from a `gcloud` login, and attributes API usage per site.
+
+  **Being logged in is enough.** With no `VITOPS_GOOGLE_*` set, an Application Default Credentials
+  login is used — the credential `gcloud auth application-default login` already wrote. It needs the
+  Search Console scopes, which are not in the default ADC set:
+
+  ```
+  gcloud auth application-default login \
+    --scopes=openid,https://www.googleapis.com/auth/siteverification,\
+  https://www.googleapis.com/auth/webmasters,https://www.googleapis.com/auth/cloud-platform
+  ```
+
+  Explicit `VITOPS_GOOGLE_*` vars still win. Ambient credentials are only a fallback, because a stale
+  local login silently overriding a deliberate CI secret is the worst available precedence.
+
+  Beyond convenience, this removes a documented footgun: an OAuth client you create yourself sits in
+  _Testing_ publishing status, where Google **expires the refresh token after 7 days**. The Cloud SDK
+  client is published, so an ADC credential doesn't rot between onboarding runs.
+
+  **New: `site.google.project`** — the Google Cloud project API usage is attributed to, sent as
+  `x-goog-user-project`. One project per site is what keeps each site's usage and billing separate,
+  and it is the same field Maps or anything else will read later, rather than a `quotaProject` here
+  and a `mapsProject` there.
+
+  It is **required for a user credential and wrong for a service account**, which is why it is not
+  sent unconditionally. A user credential authenticates through a shared OAuth client that owns no
+  project, so Google refuses the call outright — measured: `403 "requires a quota project, which is
+not set by default"`. A service account already belongs to a project, and pointing it at a
+  different one would newly require `serviceusage.services.use` there, turning a working CI
+  credential into a 403 for no benefit.
+
+  Using it needs both APIs (`searchconsole.googleapis.com`, `siteverification.googleapis.com`)
+  enabled on that project, and `serviceusage.services.use` for the identity — one grant per site,
+  which suits an agency administering many sites from one admin identity.
+
+  An ADC credential with no `site.google.project` is now **refused before any request is made**,
+  naming the file the credential came from and the project gcloud recorded with it:
+
+  ```
+  ✖ this Google credential comes from ~/.config/gcloud/application_default_credentials.json,
+    which authenticates through a shared OAuth client that owns no project — so it needs
+    "site.google.project" in site.json to say which project the API usage belongs to.
+    gcloud recorded "acme-web" as the quota project for that login, so that is probably the value.
+  ```
+
+  That is the mistake waiting for the _second_ site: the command gets copied from whatever
+  onboarded the first one, the field is left out, and Google's own answer names neither the config
+  field nor the credential.
+
+  **Fixed: an ADC file in `GOOGLE_APPLICATION_CREDENTIALS` no longer hard-exits.** That variable is
+  Google's own convention _and_ where gcloud writes ADC, so the same variable carries two kinds of
+  credential. Parsing an `authorized_user` file as a service account failed with `service account
+JSON is missing client_email or private_key` — while holding a credential the command already knew
+  how to use. It is now discriminated on `type`, the field Google's own libraries switch on.
+
+  **Also fixed — `search setup --dry` and `--check` are not offline.** The docs implied otherwise. The
+  state gather runs before the planner, so both read from Cloudflare and Google and both need
+  credentials. They still create and change nothing; `notify --dry` remains genuinely request-free.
+
+  New in `@getvitops/utils/indexing`: `parseAdcUser`, `adcQuotaProject`, `adcCredentialsPath` (pure,
+  so what is decidable is asserted without a filesystem), plus `googleHeaders` and the `GoogleAuth` /
+  `GoogleAuthLike` types. The seven exported Google request functions now take a token **or** a
+  `{ token, quotaProject }` — a bare string still works everywhere, so nothing needs changing.
+
+### Patch Changes
+
+- **Added — `utils`.** `itemListGraph()`, a pure schema.org `ItemList` JSON-LD builder — a sibling
+  of `breadcrumbGraph()`/`localBusinessGraph()`. Extracted from `@getvitops/astro`'s `<Carousel />`
+  so any consumer can build a valid `ItemList` (Google's "all-in-one page" carousel format:
+  `ListItem.item` carrying `@type`/`name`/`url`) for content that isn't one of the six Google
+  carousel rich-result types (`Article`/`Recipe`/`Movie`/`Course`/`Restaurant`/`Product`) — the
+  entity type is no longer constrained to that set.
+
+  **Changed — `astro`.** `<Carousel />`'s `CarouselItem['type']` now accepts any string, not just
+  the six carousel rich-result types — non-breaking, the union only widens. It's a thin wrapper
+  over `itemListGraph()` now; behaviour for existing callers is unchanged.
+
+  No consumer-facing change in this release outside `utils` and `astro` — `generator`, `cli` and
+  `vite` still ship in this release only because the six share one version.
+
+  **Fixed — `core` (`css`/`bricks`/`tailwind` build output, all formats).** The CSS and HTML
+  authored inside Lit's `css`/`html` tagged templates in `<wc-*>` component sources previously
+  survived JS minification verbatim — a JS minifier treats a template literal's cooked text as
+  opaque, so `elements.js` shipped ~690 lines of indented, unminified CSS (and the equivalent in
+  HTML) inside an otherwise-minified 886-line file. Two source-level build transforms now collapse
+  both before bundling: `postcss` + `postcss-lit` + `cssnano` for `css` templates, `minify-html-literals`
+  (`conservativeCollapse: true`, so whitespace between inline elements in light-DOM components never
+  collapses to zero) for `html` templates. `elements.js` drops from 886 lines / ~101 KB to 49 lines /
+  ~96 KB. A second build step deduplicates the 17 `@license` comment blocks (only 6 are textually
+  distinct) down to one verbatim copy of each — required by BSD-3/MIT/Apache-2.0's notice-retention
+  terms either way, just no longer repeated once per source file that carries it. No API change;
+  build-time only, and none of the added tooling (`postcss`, `postcss-lit`, `cssnano`,
+  `minify-html-literals`) is a runtime or published dependency.
+
+- Updated dependencies [c0c092b]
+- Updated dependencies
+- Updated dependencies [aad43b7]
+- Updated dependencies [2b50f9e]
+- Updated dependencies [a6a8b39]
+- Updated dependencies [7b391b3]
+- Updated dependencies [eb9ac79]
+  - @getvitops/utils@7.0.0
+  - @getvitops/generator@7.0.0
+
+## 6.0.0
+
+**No consumer-facing change in this package.** The major is the toolchain's shared version, bumped
+in lockstep for the carousel rework in `@getvitops/core` / `generator` / `astro`. Every command
+behaves as it did in 5.0.0; `vitops docs components` and `vitops docs css` will report the new
+carousel vocabulary, because they render live from the generator.
+
+### Patch Changes
+
+- Updated dependencies
+  - @getvitops/generator@6.0.0
+  - @getvitops/utils@6.0.0
+
 ## 5.0.0
 
 ### Major Changes

package/dist/cli.mjs

@@ -3,10 +3,11 @@ import { createRequire } from "node:module";
 import { parseArgs } from "node:util";
 import { existsSync, lstatSync, mkdirSync, readFileSync, readlinkSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
 import { dirname, extname, join, parse, relative, resolve } from "node:path";
+import { homedir } from "node:os";
 import { pathToFileURL } from "node:url";
 import { JURISDICTION_COUNTRIES, NUMERIC_STEPS, SCHEMA_LOCAL_PATH, buildIconSprite, defaultConfig, enabledDocs, expandPalette, functionalRole, generate, generateDocs, generateLegal, isConfig, movedTokens, processorsMissingLocation, resolveConfig, resolveInput, resolveTheme, roleColorUtilities, roleHue, roleKind, validate } from "@getvitops/generator";
 import { collectIconRefs, generateFavicons, generateIconInclude, resolveIcon, scanFiles } from "@getvitops/utils";
-import { SCOPES, SNAPSHOT_PATH, collectEntries, formatPlan, googleAccessToken, inspectUrl, keyFileContents, newKey, parseServiceAccount, plan, readSnapshot, resolveKeyLocation, resolveSitemapUrl, submitBatch, submitSitemap, toSnapshot, verifyKeyFile, writeSnapshot } from "@getvitops/utils/indexing";
+import { SCOPES, SNAPSHOT_PATH, adcCredentialsPath, adcQuotaProject, collectEntries, formatPlan, googleAccessToken, inspectUrl, keyFileContents, newKey, parseAdcUser, parseServiceAccount, plan, readSnapshot, resolveKeyLocation, resolveSitemapUrl, submitBatch, submitSitemap, toSnapshot, verifyKeyFile, writeSnapshot } from "@getvitops/utils/indexing";
 import { addSite, backoffSchedule, createApexTxt, findZoneId, formatPlan as formatPlan$1, formatSummary, getSite, getVerificationToken, getWebResource, hasDrift, listApexTxt, ownerUnion, plan as plan$1, refreshTokenGrant, siteUrlFor, updateOwners, verifyWebResource } from "@getvitops/utils/onboarding";
 import { AD_PLATFORMS, AD_PROVIDER_KEYS, formatPlan as formatPlan$2, formatSummary as formatSummary$1, hasDrift as hasDrift$1, isAdProvider, missingFields, plan as plan$2, renderTag, tagId } from "@getvitops/utils/ads";
 import { PLATFORM_PARAMS } from "@getvitops/utils/tracking";
@@ -278,6 +279,16 @@ Search setup options:
   scoped to siteverification + webmasters). Granting a Google Group Full-User
   access has no API and is surfaced as a reminder.
 
+  Or just be logged in: with no VITOPS_GOOGLE_* set, an Application Default
+  Credentials login is used —
+    gcloud auth application-default login \\
+      --scopes=openid,https://www.googleapis.com/auth/siteverification,\\
+  https://www.googleapis.com/auth/webmasters,https://www.googleapis.com/auth/cloud-platform
+  ADC authenticates through a shared OAuth client that owns no project, so it also
+  needs site.google.project in the config (sent as x-goog-user-project), both APIs
+  enabled on that project, and serviceusage.services.use on it. One project per site
+  is what keeps each site's API usage and billing separate.
+
 Search notify options:
   -i, --input <path>    Site config carrying seo.indexing (default: ./site.json)
       --site-env <env>  Environment to notify for (default: production). An
@@ -1963,6 +1974,7 @@ function loadServiceAccount() {
 	const path = process.env.GOOGLE_APPLICATION_CREDENTIALS;
 	const raw = inline ?? (path && existsSync(path) ? readFileSync(path, "utf8") : void 0);
 	if (!raw) return void 0;
+	if (parseAdcUser(raw)) return void 0;
 	try {
 		return parseServiceAccount(raw);
 	} catch (err) {
@@ -2038,7 +2050,8 @@ async function cmdSearchNotify(argv) {
 		console.log(`  Then serve it at /${key}.txt — \`vitops search notify --write-key public\`.`);
 		return;
 	}
-	const config = toIndexingConfig(await loadConfig(resolve(values.input), values["site-env"]), values["site-env"]);
+	const site = await loadConfig(resolve(values.input), values["site-env"]);
+	const config = toIndexingConfig(site, values["site-env"]);
 	if (values["write-key"] !== void 0) {
 		const key = config.indexNow?.key;
 		if (!key) fail("no site.seo.indexing.indexNow.key in the config — generate one with --new-key");
@@ -2077,10 +2090,10 @@ async function cmdSearchNotify(argv) {
 		if (!p.searchConsole.siteUrl) fail("--check needs seo.indexing.searchConsole.siteUrl (the Search Console property)");
 		if (!credential) fail(`--check needs a Search Console credential — ${CREDENTIAL_HINT}`);
 		if (p.check.length === 0) fail("--check needs seo.indexing.priorityUrls — the pages whose indexing matters");
-		const token = await googleAccessToken(credential);
+		const auth = searchAuth(await googleAccessToken(credential), credential, site, values.input);
 		console.log(`\nInspecting ${p.check.length} priority URL(s):`);
 		for (const url of p.check) {
-			const r = await inspectUrl(token, p.searchConsole.siteUrl, url);
+			const r = await inspectUrl(auth, p.searchConsole.siteUrl, url);
 			if (r.error) {
 				console.log(`  ? ${url} — ${r.error}`);
 				failed = true;
@@ -2119,7 +2132,7 @@ async function cmdSearchNotify(argv) {
 	}
 	if (p.searchConsole.enabled) if (!credential) console.log(`· Search Console: skipped — no credential (${CREDENTIAL_HINT})`);
 	else {
-		const r = await submitSitemap(await googleAccessToken(credential), p.searchConsole.siteUrl, sitemapUrl);
+		const r = await submitSitemap(searchAuth(await googleAccessToken(credential), credential, site, values.input), p.searchConsole.siteUrl, sitemapUrl);
 		if (r.ok) console.log(`✓ Search Console: sitemap resubmitted`);
 		else {
 			console.error(`✖ Search Console: ${r.message}`);
@@ -2169,11 +2182,53 @@ function loadCloudflareToken() {
 * the sites. All-or-nothing — a partial set is a misconfiguration worth naming
 * rather than a reason to run degraded.
 */
+/**
+* Set when the Google credential came from an ADC file rather than the env vars.
+*
+* Provenance matters for exactly one check: an ADC credential authenticates through
+* a shared OAuth client that owns no project, so it CANNOT work without
+* `site.google.project`, while a credential from your own OAuth client works fine
+* without one. Knowing which we hold is what turns Google's "requires a quota
+* project" 403 into an error that names the field and the likely value.
+*/
+let adcOrigin;
+/**
+* Refuse an ADC credential with no project, before spending a request on a 403.
+*
+* This is the mistake waiting for the *second* site: the command is copied from
+* whatever onboarded the first one, `site.google.project` is left out, and Google
+* answers with a message about quota projects that names neither the config field
+* nor the file the credential came from.
+*/
+function requireAdcProject(project, input) {
+	if (!adcOrigin || project) return;
+	fail(`this Google credential comes from ${adcOrigin.path}, which authenticates through a shared OAuth client that owns no project — so it needs "site.google.project" in ${input} to say which project the API usage belongs to.` + (adcOrigin.project ? `\n  gcloud recorded "${adcOrigin.project}" as the quota project for that login, so that is probably the value.` : "") + "\n  One project per site keeps each site's usage and billing separate. That project also needs searchconsole.googleapis.com and siteverification.googleapis.com enabled, and the identity needs serviceusage.services.use on it.\n  Or set VITOPS_GOOGLE_CLIENT_ID / _CLIENT_SECRET / _REFRESH_TOKEN to use your own OAuth client instead.");
+}
 function loadGoogleOAuth() {
 	const clientId = process.env.VITOPS_GOOGLE_CLIENT_ID;
 	const clientSecret = process.env.VITOPS_GOOGLE_CLIENT_SECRET;
 	const refreshToken = process.env.VITOPS_GOOGLE_REFRESH_TOKEN;
-	if (!clientId && !clientSecret && !refreshToken) return void 0;
+	if (!clientId && !clientSecret && !refreshToken) {
+		const path = process.env.GOOGLE_APPLICATION_CREDENTIALS ?? adcCredentialsPath({
+			env: process.env,
+			platform: process.platform,
+			home: homedir(),
+			join
+		});
+		if (path && existsSync(path)) {
+			const raw = readFileSync(path, "utf8");
+			const adc = parseAdcUser(raw);
+			if (adc) {
+				const recorded = adcQuotaProject(raw);
+				adcOrigin = recorded ? {
+					path,
+					project: recorded
+				} : { path };
+				return adc;
+			}
+		}
+		return;
+	}
 	const missing = [
 		["VITOPS_GOOGLE_CLIENT_ID", clientId],
 		["VITOPS_GOOGLE_CLIENT_SECRET", clientSecret],
@@ -2213,6 +2268,24 @@ function loadSearchCredential() {
 		oauth
 	} : void 0;
 }
+/**
+* Attach the site's Google project to a token — for a **user** credential only.
+*
+* A user credential (an ADC login, or `VITOPS_GOOGLE_*`) authenticates through a
+* shared OAuth client that owns no project, so Google refuses the call outright
+* without `x-goog-user-project`. A service account already belongs to a project, and
+* pointing it at a *different* one would newly require `serviceusage.services.use`
+* there — turning a working CI credential into a 403 for no benefit. So the header
+* is sent exactly when it is needed, never as a default.
+*/
+function searchAuth(token, credential, cfg, input) {
+	const project = cfg.site?.google?.project;
+	if (credential.kind === "oauth") requireAdcProject(project, input);
+	return project && credential.kind === "oauth" ? {
+		token,
+		quotaProject: project
+	} : token;
+}
 /** Both spellings, for an error that has to tell you what to actually set. */
 const CREDENTIAL_HINT = "a service account in VITOPS_GSC_SERVICE_ACCOUNT / GOOGLE_APPLICATION_CREDENTIALS, or a user OAuth credential in VITOPS_GOOGLE_CLIENT_ID / VITOPS_GOOGLE_CLIENT_SECRET / VITOPS_GOOGLE_REFRESH_TOKEN";
 const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
@@ -2243,6 +2316,8 @@ async function cmdSearch(argv) {
 * Idempotent by construction: the planner decides each step from the domain's live
 * state, so a re-run of a fully-onboarded domain is all skips. `--check` reports
 * drift and exits non-zero without mutating; `--dry` prints the plan and stops.
+* Neither is offline: the state gather runs before the planner either way, so both
+* read from Cloudflare and Google and both need credentials.
 * DNS is only ever *created* — the verification TXT — never edited or removed.
 *
 * Search Console has no user/permission API, so granting a Google Group Full-User
@@ -2273,11 +2348,14 @@ async function cmdSearchSetup(argv) {
 		},
 		allowPositionals: false
 	});
-	const domains = toSearchConsoleSetup(await loadConfig(resolve(values.input), values["site-env"]), values.domain);
+	const cfg = await loadConfig(resolve(values.input), values["site-env"]);
+	const domains = toSearchConsoleSetup(cfg, values.domain);
 	if (domains.length === 0) fail("no domains in site.searchConsole — add domains to onboard (keyed by bare hostname)");
 	const cfToken = loadCloudflareToken();
 	const oauth = loadGoogleOAuth();
-	if (values.dry && (!cfToken || !oauth)) {
+	const project = cfg.site?.google?.project;
+	const usableOauth = oauth && !(adcOrigin && !project);
+	if (values.dry && (!cfToken || !usableOauth)) {
 		const blank = {
 			zoneId: "",
 			txtPresent: false,
@@ -2286,28 +2364,33 @@ async function cmdSearchSetup(argv) {
 			propertyExists: false
 		};
 		console.log(formatPlan$1(plan$1({ domains }, new Map(domains.map((d) => [d.domain, blank])))));
-		const missing = [...cfToken ? [] : ["CLOUDFLARE_API_TOKEN (Zone:DNS:Edit + Zone:Read)"], ...oauth ? [] : ["VITOPS_GOOGLE_CLIENT_ID / VITOPS_GOOGLE_CLIENT_SECRET / VITOPS_GOOGLE_REFRESH_TOKEN"]];
+		const missing = [...cfToken ? [] : ["CLOUDFLARE_API_TOKEN (Zone:DNS:Edit + Zone:Read)"], ...usableOauth ? [] : oauth ? [`site.google.project in ${values.input} (this ADC login has no project)`] : ["VITOPS_GOOGLE_CLIENT_ID / VITOPS_GOOGLE_CLIENT_SECRET / VITOPS_GOOGLE_REFRESH_TOKEN"]];
 		console.log(`\n(--dry, and no credentials set: planned from scratch rather than from live state, so nothing here is reported as already done.\n A real run needs ${missing.join(" and ")}.)`);
 		return;
 	}
 	if (!cfToken) fail("set CLOUDFLARE_API_TOKEN (a Zone:DNS:Edit token — the standard \"Edit zone DNS\" template, which also carries the Zone:Read that the zone-by-name lookup needs)");
-	if (!oauth) fail("set VITOPS_GOOGLE_CLIENT_ID / VITOPS_GOOGLE_CLIENT_SECRET / VITOPS_GOOGLE_REFRESH_TOKEN — a user OAuth token scoped to siteverification + webmasters");
+	if (!oauth) fail("set VITOPS_GOOGLE_CLIENT_ID / VITOPS_GOOGLE_CLIENT_SECRET / VITOPS_GOOGLE_REFRESH_TOKEN — a user OAuth token scoped to siteverification + webmasters — or run `gcloud auth application-default login` with the siteverification + webmasters + cloud-platform scopes (see --help)");
+	requireAdcProject(project, values.input);
 	const token = await refreshTokenGrant(oauth);
+	const auth = project ? {
+		token,
+		quotaProject: project
+	} : token;
 	const states = /* @__PURE__ */ new Map();
 	const verifyTokens = /* @__PURE__ */ new Map();
 	const webIds = /* @__PURE__ */ new Map();
 	for (const d of domains) {
 		const zone = await findZoneId(cfToken, d.domain);
 		if (!zone.ok || !zone.zoneId) fail(`${d.domain}: ${zone.message}`);
-		const vt = await getVerificationToken(token, d.domain);
+		const vt = await getVerificationToken(auth, d.domain);
 		if (!vt.ok || !vt.token) fail(`${d.domain}: could not obtain a verification token — ${vt.message}`);
 		verifyTokens.set(d.domain, vt.token);
 		const txt = await listApexTxt(cfToken, zone.zoneId, d.domain);
 		if (!txt.ok) fail(`${d.domain}: ${txt.message}`);
-		const wr = await getWebResource(token, d.domain);
+		const wr = await getWebResource(auth, d.domain);
 		if (!wr.ok) fail(`${d.domain}: ${wr.message}`);
 		if (wr.id) webIds.set(d.domain, wr.id);
-		const site = await getSite(token, siteUrlFor(d.domain));
+		const site = await getSite(auth, siteUrlFor(d.domain));
 		if (!site.ok) fail(`${d.domain}: ${site.message}`);
 		states.set(d.domain, {
 			zoneId: zone.zoneId,
@@ -2360,7 +2443,7 @@ async function cmdSearchSetup(argv) {
 		if (verified) result.verified = "yes";
 		else {
 			for (let attempt = 0; attempt < delays.length; attempt++) {
-				const v = await verifyWebResource(token, d.domain);
+				const v = await verifyWebResource(auth, d.domain);
 				if (v.ok) {
 					verified = true;
 					if (v.id) webIds.set(d.domain, v.id);
@@ -2381,7 +2464,7 @@ async function cmdSearchSetup(argv) {
 		}
 		if (dp.property.action === "skip") result.property = "present";
 		else {
-			const r = await addSite(token, dp.siteUrl);
+			const r = await addSite(auth, dp.siteUrl);
 			if (!r.ok) {
 				console.error(`✖ ${d.domain} property: ${r.message}`);
 				result.property = "failed";