@commercelayer/agent-cli 0.8.0 → 0.8.1

package/README.md

@@ -144,7 +144,7 @@ products:
 A store has at most 6 languages, the base one included. Every country gets its own pages, `/<cc>-<ll>/` (`/it-it/`), in that country's language: the product texts in it when the product has them, the base language otherwise; `/` lets the buyer choose the country when there are several. Shopping agents get the same texts over UCP, in the language they ask for. `plan` warns about a language no country of the store speaks. A store without translations plans and applies as before.
 
 ### A catalog the human edits
-When the products live in a tool the human edits too, `store.yml` names it and `c11r catalog pull` copies it into `.commercelayer/products/catalog.csv`, sorted by sku: same source, same file. Pull applies the product rules above before it writes, with errors naming the row in the source, and reports the skus added, changed and removed. Removed products leave the store only through `apply --allow-delete`. Every source has the same columns: `sku`, `name`, `description`, `image_url`, `category`, and one price column per market currency, named by its ISO 4217 code in lowercase (`eur`, `usd`).
+When the products live in a tool the human edits too, `store.yml` names it and `c11r catalog pull` copies it into `.commercelayer/products/catalog.csv`, sorted by sku: same source, same file. Pull applies the product rules above before it writes, with errors naming the row in the source, and reports the skus added, changed and removed, and `categories`: the category paths, in the file's order, each once (named in `next` when there are at most 12). Removed products leave the store only through `apply --allow-delete`. Every source has the same columns: `sku`, `name`, `description`, `image_url`, `category`, and one price column per market currency, named by its ISO 4217 code in lowercase (`eur`, `usd`).
 ```yaml
 catalog:
   source: sheets     # sheets | supabase | merchant | website
@@ -155,10 +155,10 @@ catalog:
 - `sheets`: a Google Sheet, by its CSV link (File > Share > Publish to web > Comma-separated values) or by its own link when anyone with the link can view it. No key. Translations are `name_<ll>`, `description_<ll>` and `category_<ll>` columns, in sheets and Supabase tables alike.
 - `supabase`: a table (default `products`), read over the REST API with the project's anon key: `C11R_SUPABASE_URL=https://<project>.supabase.co` and `C11R_SUPABASE_KEY=<anon key>` in `.commercelayer/credentials`, or the environment. The key needs a row level security policy that allows select. Never use the `service_role` key. Columns `id`, `created_at` and `updated_at` are skipped.
 - `merchant`: a Google Merchant Center product feed URL, RSS 2.0 (XML, `g:` namespace) or TSV. `id` is the sku, `title` the name, `description`, `image_link` the image, `price` (`14.50 EUR`) the price in its currency, `product_type` the category. An id listed once per currency is one product with several prices. Other attributes, `sale_price` included, are ignored. Other languages: `catalog.translations: { it: <the Italian feed URL> }`, one feed per language, merged by `id` (`title`, `description`, `product_type`).
-- `website`: a brand's own website, for a demo store (the human's site, or a brand they are pitching to). `url` is the site, `limit` how many products (default 100, at most 500). Pull tries, in order, and the first that yields products wins: Shopify's public `products.json` (also `/collections/all/products.json`), WooCommerce's Store API (`/wp-json/wc/store/v1/products`), then the product pages its sitemap lists (`/products/`, `/product/`, `/p/`, `/shop/`, `/item/`) and their schema.org Product JSON-LD. One row per product: the first available variant's sku and price (or the handle in upper snake case), the title, the description as plain text (paragraphs kept, at most 2,000 characters), the first image, the product type or breadcrumb as category. Products without a price are left out, and a sku seen twice keeps the first. Prices are the site's, in the site's currency, never converted: the store's markets must sell in it (`currency_mismatch` otherwise), and when the site does not say it, `currency_unknown` asks for `--currency <code>`. Translations are not pulled yet; `source_site.languages` lists the site's languages. **Images and the logo stay on the brand's servers: `image_url` links them, nothing is copied.** Pull reads politely: `robots.txt` is honoured for every path (`robots_skipped` counts what it disallows), one request at a time and at most 2 a second, 15 s per request and 60 s in all (`budget_exhausted` when it stopped early, with what it read), as `c11r/<version> (+https://commercelayer.ai)`. It prints `strategy`, `source_site: { url, title, platform, product_count_seen, currency, languages }`, `pulled`, `variants_skipped`, `robots_skipped`, `no_price_skipped`, `duplicates_skipped`, `other_currency_skipped`, `tried: [{ strategy, result }]` and `requests`, so the agent can tell the human "I found 340 products, pulled 100".
+- `website`: a brand's own website, for a demo store (the human's site, or a brand they are pitching to). `url` is the site, `limit` how many products (default 100, at most 500). Pull tries, in order, and the first that yields products wins: Shopify's public `products.json` (also `/collections/all/products.json`), WooCommerce's Store API (`/wp-json/wc/store/v1/products`), then the product pages its sitemap lists (`/products/`, `/product/`, `/p/`, `/shop/`, `/item/`) and their schema.org Product JSON-LD. One row per product: the first available variant's sku and price (or the handle in upper snake case), the title, the description as plain text (paragraphs kept, at most 2,000 characters), the first image, the product type or breadcrumb as category. Products without a price are left out, and a sku seen twice keeps the first. Prices are the site's, in the site's currency, never converted: the store's markets must sell in it (`currency_mismatch` otherwise), and when the site does not say it, `currency_unknown` asks for `--currency <code>`. Translations are not pulled yet; `source_site.languages` lists the site's languages. **Images and the logo stay on the brand's servers: `image_url` links them, nothing is copied.** Pull reads politely: `robots.txt` is honoured for every path (`robots_skipped` counts what it disallows), one request at a time and at most 2 a second, 15 s per request and 60 s in all (`budget_exhausted` when it stopped early, with what it read), as `c11r/<version> (+https://commercelayer.ai)`. It prints `strategy`, `source_site: { url, title, platform, product_count_seen, currency, languages }`, `pulled`, `categories`, `variants_skipped`, `robots_skipped`, `no_price_skipped`, `duplicates_skipped`, `other_currency_skipped`, `tried: [{ strategy, result }]` and `requests`, so the agent can tell the human "I found 340 products, pulled 100".
 Error codes: `no_catalog_source`, `invalid_catalog`, `missing_credentials`, `catalog_unreachable`, `catalog_unauthorized`, `catalog_not_found`, `catalog_not_csv`, `catalog_error`, `empty_catalog`, plus `invalid_product` and `missing_price`; for a website `site_unreachable`, `no_products_found` (ask the human for a feed or a sheet instead), `currency_unknown` and `currency_mismatch`.
 
-`storefront inspect <url>` reads the same way, one page and up to 3 of its stylesheets, and writes nothing: `{ url, title, site_name, description, language, languages, theme_color, icons, og_image, logo_candidates: [{ url, source: json_ld|header_img|img|header_svg, alt, width, height }], colors: [{ hex, source: theme_color|css, count, neutral }], currencies_seen, countries_seen, platform, robots: { products_allowed, paths }, next }`. An inline SVG logo has `url: null`: there is nothing to link. Errors: `missing_argument`, `site_unreachable`, `site_error`, `robots_disallowed`.
+`storefront inspect <url>` reads the same way, one page and up to 3 of its stylesheets, and writes nothing: `{ url, title, site_name, description, language, languages, theme_color, icons, og_image, og_image_width, og_image_height, og_image_is_logo, logo_candidates: [{ url, source: json_ld|header_img|img|header_svg, alt, width, height }], colors: [{ hex, source: theme_color|css, count, neutral }], currencies_seen, countries_seen, platform, robots: { products_allowed, paths }, next }`. An inline SVG logo has `url: null`: there is nothing to link. `og_image_width` and `og_image_height` come from `og:image:width`/`og:image:height` (null without them); `og_image_is_logo` is true when the `og:image` is one of the logo candidates or has `logo` in its file name, so it is not a hero photo. Errors: `missing_argument`, `site_unreachable`, `site_error`, `robots_disallowed`.
 
 ### The website
 The theme renders the home page (logo, hero with name and tagline, products by top-level category), a page per category at `/categories/<slug>/...` and per product at `/products/<sku>`, with the market from `?market=<code>`, cart and checkout. Images written `img/<file>` (`theme.logo`, `theme.hero`, a product's `image_url`) are files in `.commercelayer/img/`, served at `<preview_url>/img/<file>`: paths of letters, digits, `.`, `_`, `-` and `/`, a lowercase `png`, `jpg`, `jpeg`, `webp`, `gif`, `avif` or `svg` extension, at most 5 MB each. `apply` uploads the new and changed ones (by SHA-256) before anything else, and the store host drops the ones nothing references any more.

package/dist/catalog.js

@@ -326,6 +326,8 @@ export async function pull(values = {}) {
         }
     }
     const after = new Map(products.map((p) => [p.sku, p]));
+    // The category paths in the order catalog.csv lists them (by sku), each once.
+    const categories = [...new Set(toCsvOrder(products).flatMap((p) => (p.category ? [p.category] : [])))];
     const csv = toCsv(products);
     const changed = !fs.existsSync(file) || fs.readFileSync(file, 'utf8') !== csv;
     if (changed) {
@@ -342,11 +344,12 @@ export async function pull(values = {}) {
         added: [...after.keys()].filter((sku) => !before.has(sku)),
         changed: [...after.keys()].filter((sku) => before.has(sku) && canonical(before.get(sku)) !== canonical(after.get(sku))),
         removed,
+        categories,
         ...(site ? websiteOutput(site, products.length) : {}),
         next: site
-            ? websiteNext(site, products.length, changed, removed.length, file)
+            ? websiteNext(site, products.length, changed, removed.length, file, categories)
             : changed
-                ? `Run c11r plan to see what apply will change in the store.${removed.length ? ' Removed products are deleted from the store only by c11r apply --allow-delete, once the human agrees.' : ''} ${file} is rewritten by every pull: change products in the ${config.source} source, not in the file.`
+                ? `${categoriesSentence(categories)}Run c11r plan to see what apply will change in the store.${removed.length ? ' Removed products are deleted from the store only by c11r apply --allow-delete, once the human agrees.' : ''} ${file} is rewritten by every pull: change products in the ${config.source} source, not in the file.`
                 : `${file} already matches the ${config.source} source.`,
     };
 }
@@ -354,12 +357,15 @@ function websiteOutput(r, pulled) {
     const { strategy: _, ...rest } = r;
     return { ...rest, pulled };
 }
-function websiteNext(r, pulled, changed, removed, file) {
+const toCsvOrder = (products) => [...products].sort((a, b) => (a.sku < b.sku ? -1 : a.sku > b.sku ? 1 : 0));
+/** "Categories: Coffee, Mugs." up to 12 of them, otherwise how many; nothing without categories. */
+const categoriesSentence = (c) => (c.length === 0 ? '' : c.length <= 12 ? `Categories: ${c.join(', ')}. ` : `${c.length} categories. `);
+function websiteNext(r, pulled, changed, removed, file, categories) {
     const seen = r.source_site.product_count_seen;
     const found = seen !== null && seen > pulled ? `I found ${seen} products on the site and pulled ${pulled} (catalog.limit is ${r.limit})` : `I pulled ${pulled} products from the site`;
     const left = [r.no_price_skipped && `${r.no_price_skipped} without a price`, r.duplicates_skipped && `${r.duplicates_skipped} with a sku already taken`, r.robots_skipped && `${r.robots_skipped} pages robots.txt disallows`, r.other_currency_skipped && `${r.other_currency_skipped} in another currency`].filter(Boolean);
     return [
-        `Tell the human: "${found}, in ${r.source_site.currency}${r.variants_skipped ? `, one variant each (${r.variants_skipped} other variants left out)` : ''}." ${left.length ? `Left out: ${left.join(', ')}. ` : ''}${r.budget_exhausted ? 'The site was slow: pull stopped at 60 s, so this is what it read by then. ' : ''}`,
+        `Tell the human: "${found}, in ${r.source_site.currency}${r.variants_skipped ? `, one variant each (${r.variants_skipped} other variants left out)` : ''}." ${left.length ? `Left out: ${left.join(', ')}. ` : ''}${r.budget_exhausted ? 'The site was slow: pull stopped at 60 s, so this is what it read by then. ' : ''}${categoriesSentence(categories)}`,
         'Images and the logo stay on the brand\'s servers: image_url links them, nothing is copied. ',
         changed ? `Run c11r plan to see what apply will change in the store.${removed ? ' Removed products are deleted from the store only by c11r apply --allow-delete, once the human agrees.' : ''} ${file} is rewritten by every pull: never edit it; products to add go in another file in ${paths.products}.` : `${file} already matches the site.`,
     ].join('');

package/dist/commands.js

@@ -31,10 +31,10 @@ export const HELP = {
     init: 'Create a sandbox store for this repo, with no login: c11r init --name "<business name>" (the folder name when --name is absent). The name becomes the organization\'s name, and the org slug and store hostname derive from it. Writes .commercelayer/store.yml and .commercelayer/credentials, and a section in AGENTS.md and CLAUDE.md (created, or appended once) that points later agents in this repo to the skills: commit them. Reuses the store if store.yml already exists.',
     plan: 'Show what apply would create, update or delete: markets, products, images, and the store\'s name, language, policies and theme; missing_images lists references with no file in .commercelayer/img/ (apply refuses them), unused_images files nothing references, from .commercelayer/store.yml and the product files under .commercelayer/products/ (*.yml lists, *.csv rows, merged by sku). Products may carry other languages: name, description and category as { <ll>: text } maps (the base language, store.yml language:, default en, among them), or name_<ll>, description_<ll>, category_<ll> CSV columns; at most 6 languages. warnings lists a language no country of the store speaks.',
     apply: 'Make the store match .commercelayer/store.yml and .commercelayer/products/: markets, products, the storefront mode (storefront.mode: hosted, the default, or agent-only: UCP and the pages it needs, no catalog pages), the theme (its settings, badge included, and the images in .commercelayer/img/ that the theme and products reference, uploaded when new or changed), name and policies. The store\'s website is live at preview_url after apply. Refuses deletes unless --allow-delete is given. Product changes go through a Commerce Layer import: apply waits for it (progress as JSON lines on stderr) and prints one result; if it stops with apply_pending the import keeps going, so run `c11r status` to follow it. While Commerce Layer is busy or unavailable it keeps reading, with a retrying line. An apply still running is finished first. --full re-imports every product and price, after edits in the Commerce Layer dashboard.',
-    'catalog pull': 'Read the products from the catalog source in store.yml (catalog: { source: supabase | sheets | merchant | website, url, table, limit }) and write them to .commercelayer/products/catalog.csv; then plan and apply as usual. Columns: sku, name, description, image_url, category, name_<ll>, description_<ll> and category_<ll> for other languages (name_it), and one price column per market currency, in lowercase (eur, usd). A Merchant feed adds other languages with catalog.translations: { it: <feed url> }, merged by id. Supabase reads C11R_SUPABASE_URL and C11R_SUPABASE_KEY (the anon key) from .commercelayer/credentials or the environment; a Google Sheet needs its CSV link or a link anyone can open; a Merchant Center feed is RSS/XML or TSV. source website reads a brand\'s site for a demo store (url: the site, limit: default 100, at most 500): Shopify products.json, then the WooCommerce Store API, then the sitemap\'s product pages and their JSON-LD, first that yields products wins (strategy). One row per product, first available variant (variants_skipped counts the others); prices in the site\'s currency, never converted: currency_mismatch when the markets sell in another, currency_unknown when the site does not say (ask the human, then --currency <code>). It honours robots.txt (robots_skipped), reads one request at a time, at most 2 a second, 60 s in all (budget_exhausted). Images stay on the brand\'s servers: image_url links them, nothing is copied. Prints strategy, source_site { url, title, platform, product_count_seen, currency, languages }, pulled and the skipped counts: tell the human how many products the site has and how many were pulled. no_products_found: ask the human for a feed or a sheet instead.',
+    'catalog pull': 'Read the products from the catalog source in store.yml (catalog: { source: supabase | sheets | merchant | website, url, table, limit }) and write them to .commercelayer/products/catalog.csv; then plan and apply as usual. Columns: sku, name, description, image_url, category, name_<ll>, description_<ll> and category_<ll> for other languages (name_it), and one price column per market currency, in lowercase (eur, usd). A Merchant feed adds other languages with catalog.translations: { it: <feed url> }, merged by id. Supabase reads C11R_SUPABASE_URL and C11R_SUPABASE_KEY (the anon key) from .commercelayer/credentials or the environment; a Google Sheet needs its CSV link or a link anyone can open; a Merchant Center feed is RSS/XML or TSV. source website reads a brand\'s site for a demo store (url: the site, limit: default 100, at most 500): Shopify products.json, then the WooCommerce Store API, then the sitemap\'s product pages and their JSON-LD, first that yields products wins (strategy). One row per product, first available variant (variants_skipped counts the others); prices in the site\'s currency, never converted: currency_mismatch when the markets sell in another, currency_unknown when the site does not say (ask the human, then --currency <code>). It honours robots.txt (robots_skipped), reads one request at a time, at most 2 a second, 60 s in all (budget_exhausted). Images stay on the brand\'s servers: image_url links them, nothing is copied. Prints strategy, source_site { url, title, platform, product_count_seen, currency, languages }, pulled, categories and the skipped counts: tell the human how many products the site has and how many were pulled. no_products_found: ask the human for a feed or a sheet instead.',
     'storefront init': 'Custom storefronts only: the hosted theme needs none of this (set theme: in store.yml and apply). Write a static storefront on drop-in.js, no build step: storefront/index.html (the products by category, a minicart that opens on add to cart, the hosted cart and checkout, and a product page at #/p/<sku>), styles.css (theme tokens first) and app.js. Product images are the store\'s, at /img/ (.commercelayer/img/, uploaded by apply); the site\'s own files go in storefront/assets/. Refuses to overwrite unless --force is given.',
     'storefront check': 'Custom storefronts only. Check the published storefront at the preview URL: drop-in.js and its config block wired to this store, the drop-in elements, an add to cart for every product, every image loading, alt text, stylesheets after drop-in.css, the theme tokens and drop-in overrides, title, lang, viewport, description, and whether the folder changed since publish. Prints { status, summary, checks: [{ id, kind, status: pass|warn|fail, message, fix, items }] }; exits 1 when a check fails.',
-    'storefront inspect': "Read a brand's website for a demo store that looks like it, read-only, nothing written: c11r storefront inspect <url>. Returns { url, title, site_name, description, language, languages, theme_color, icons, og_image, logo_candidates: [{ url, source: json_ld|header_img|img|header_svg, alt, width, height }], colors: [{ hex, source: theme_color|css, count, neutral }], currencies_seen, countries_seen, platform: shopify|woocommerce|unknown, robots: { products_allowed, paths } }. Pick theme.color (the first colour that is not neutral, usually) and theme.logo (a logo candidate's url: hot-linked, never downloaded; an inline SVG has url null) and show the human. robots.products_allowed false: catalog pull cannot read the products, ask for a feed or a sheet. Same polite reading as catalog pull (robots.txt, 2 requests a second). Errors: site_unreachable, site_error, robots_disallowed.",
+    'storefront inspect': "Read a brand's website for a demo store that looks like it, read-only, nothing written: c11r storefront inspect <url>. Returns { url, title, site_name, description, language, languages, theme_color, icons, og_image, og_image_width, og_image_height, og_image_is_logo (true: the og:image is the logo, not a hero photo), logo_candidates: [{ url, source: json_ld|header_img|img|header_svg, alt, width, height }], colors: [{ hex, source: theme_color|css, count, neutral }], currencies_seen, countries_seen, platform: shopify|woocommerce|unknown, robots: { products_allowed, paths } }. Pick theme.color (the first colour that is not neutral, usually) and theme.logo (a logo candidate's url: hot-linked, never downloaded; an inline SVG has url null) and show the human. robots.products_allowed false: catalog pull cannot read the products, ask for a feed or a sheet. Same polite reading as catalog pull (robots.txt, 2 requests a second). Errors: site_unreachable, site_error, robots_disallowed.",
     'storefront publish': 'Custom storefronts only: upload the storefront folder (store.yml storefront.path, default storefront) to the preview URL, where it replaces the hosted theme.',
     'promotions list': 'The store\'s promotions, live in Commerce Layer (not in store files): id, name, status (active, scheduled, expired, disabled, used_up), rules, actions, uses, starts, ends; managed: false marks one made in the dashboard (its id is Commerce Layer\'s), which can be enabled, disabled or deleted but not updated. warnings flags more than 10 active promotion actions without a coupon (Commerce Layer applies 10).',
     'promotions show': 'One promotion by id, as list shows it: c11r promotions show <id>.',

package/dist/inspect.js

@@ -203,6 +203,9 @@ export async function inspect(raw) {
             .sort((a, b) => b[1] - a[1])
             .map(([hex, count]) => ({ hex, source: 'css', count, neutral: neutral(hex) })),
     ].slice(0, MAX_COLORS);
+    const ogImage = absoluteUrl(meta['og:image:secure_url'] || meta['og:image'], base);
+    const logos = logoCandidates(html, base, siteName ?? title);
+    const ogFile = ogImage ? decodeURIComponent(new URL(ogImage).pathname.split('/').pop() ?? '') : '';
     const robots = await web.robotsFor(finalOrigin).catch(() => ({ rules: [], sitemaps: [] }));
     const paths = Object.fromEntries(Object.values(PRODUCT_PATHS).flat().map((p) => [p, robotsAllows(robots, p)]));
     const platform = platformOf(html, page.headers);
@@ -216,8 +219,12 @@ export async function inspect(raw) {
         languages,
         theme_color: themeColor,
         icons,
-        og_image: absoluteUrl(meta['og:image:secure_url'] || meta['og:image'], base),
-        logo_candidates: logoCandidates(html, base, siteName ?? title),
+        og_image: ogImage,
+        og_image_width: num(meta['og:image:width']) ?? null,
+        og_image_height: num(meta['og:image:height']) ?? null,
+        // The hero is a photo: an og:image that is the logo (a candidate, or logo in its file name) is not one.
+        og_image_is_logo: ogImage ? logos.some((l) => l.url === ogImage) || /logo/i.test(ogFile) : false,
+        logo_candidates: logos,
         colors,
         currencies_seen: currenciesSeen(html, meta),
         countries_seen: countriesSeen(html, base),

package/dist/ucp/check.js

@@ -6,6 +6,8 @@ import { send } from '../api/transport.js';
 import { sandboxInstrument, TEST_BUYER, testAddress } from './buy.js';
 import { discover, replay, request, shopping } from './client.js';
 import { EXTENSIONS, SHOPPING_CAPABILITIES, UCP_VERSION } from './platform.js';
+/** One nonsense word, no space or hyphen: the store's search splits a query into words, and "no" or "product" match real descriptions. */
+export const UNMATCHED_QUERY = 'qxzjvwkq';
 class Fail extends Error {
 }
 const must = (cond, detail) => {
@@ -95,7 +97,7 @@ export async function check(origin) {
         return finish(origin, checks);
     const v = variant;
     await run('dev.ucp.shopping.catalog.search', 'search filters by query and price', async () => {
-        const c = await shopping.search(s, { query: 'zzz-no-such-product-zzz' });
+        const c = await shopping.search(s, { query: UNMATCHED_QUERY });
         must(c.body.products.length === 0, 'an unmatched query must return no products');
         const p = await shopping.search(s, { filters: { price: { max: v.price.amount } }, context: { currency: v.price.currency } });
         must(p.body.products.some((x) => x.variants[0].id === v.id), 'a price filter including the variant must return it');

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@commercelayer/agent-cli",
-  "version": "0.8.0",
+  "version": "0.8.1",
   "description": "c11r: the Commerce Layer CLI for coding agents. Builds a sandbox store that people and shopping agents can buy from.",
   "homepage": "https://commercelayer.ai",
   "keywords": [