@comchatx/cx-sponsored-ads-playbook 0.0.0-stage → 0.1.0

package/research/method/change-measurement.md

@@ -0,0 +1,800 @@
+# Measuring an ad change
+
+Read 2026-11-09 by the research seat. Tags: [official] Amazon's own words (Ads API docs and OpenAPI models, Amazon Ads library and launch pages, Sell on Amazon pages, Seller Forums); [opinion] a named practitioner, agency or paid tool; [research] published research; [seat] the seat's own reasoning, allowed in this file and never a position. Figures that Amazon may change are quoted with their page and date, and are marked "for human readers only": a skill reads them where they live, at run time.
+
+## What this covers
+
+**Which part of a seller's ad work.** Every ad change a seller or cx makes is a bet: a bid raised, a budget cut, a keyword added, a negative added, a campaign split, a placement adjustment, a bidding strategy switched, a Sponsored Brands creative swapped, a campaign paused. This file is about the step after the bet: how to tell whether it paid. That means the change record (what was there before, what was sent, what Amazon answered, what Amazon's own history says, and what was expected), the windows (baseline, lag, observation, and the date the observation window's figures stop moving), the confounders that move ad and total figures on their own, the experiment tools Amazon offers for ads as of 2026-11-09 and what they do not cover, the on-off (pause) test and its research basis, the statistics of small ad samples, and the three verdicts (*held*, *not held*, *cannot tell*) that cx-listing-playbook's `effect-review` uses for listing changes.
+
+**Lanes.** What each ad metric means and how attribution windows work in depth is `beyond/metrics-and-attribution.md`. How reports are created, polled and read is `method/reporting.md`. The Ads API operations themselves are `method/ads-api-surface.md`. Why spend or sales moved without a planned change is `areas/diagnosis.md`. Ads against organic and total sales, including how much of an ad sale is incremental, is `areas/ads-organic-and-total.md`. Season and event calendars are `areas/seasons-and-deals.md`. Amazon's automatic bidding is `areas/bids-and-placements.md`; budget rules and pacing are `areas/budgets-and-pacing.md`; Amazon's own agents and tools are `beyond/automation-and-amazon-tools.md`. This file takes from them only what a verdict on one change needs.
+
+**Why a senior Amazon advertising lead cares.** Three things make ad changes harder to judge than listing changes, and an assistant that does not know them will report false wins and false losses:
+
+1. *The figures for the newest days are not final.* Conversions are dated on the day of the ad click and keep arriving for the length of the look-back window; Amazon restates conversions afterwards and invalidates clicks for up to 30 days. A bid raised on Monday and "judged" on Thursday is judged on clicks that have not had time to convert.
+2. *Amazon changes things inside the window.* Dynamic bidding moves bids on every auction, rule-based bidding moves base bids and can switch itself off, budget rules raise budgets on event days, Sponsored Brands can rewrite headlines, new paid placements are auto-enrolled, recommendations are applied in one click from the console, and an ad stops serving the moment the product loses the Featured Offer or goes out of stock. Amazon's change history does not say who made a change.
+3. *Ad sales are attributed, not incremental.* An ad-attributed sale on a brand term can be a sale that would have come through the organic result anyway (Blake, Nosko & Tadelis 2015). A change that "improves ACOS" can lower total sales. The only clean answer is a randomized holdout, and Amazon gives sellers none for Sponsored Products.
+
+The method in cx-listing-playbook's `research/method/effect-measurement.md` (read 2026-10-14, re-read for this file on 2026-11-09) carries over: one change per window where attribution is wanted, windows fixed before the change, whole weeks with matching weekdays, a control series where one exists, a confounder check, no verdict during a major sale, counts and not only percentages, and three verdicts. This file adds what is specific to ads.
+
+## Amazon's own rules and guidance
+
+### A. The change history API (RULE: the API contract)
+
+The Ads API has one operation for the history of entity changes. It is described in Amazon's OpenAPI model `Changehistory_prod_3p.json` (title "Change history", version "3.0"), served from the docs site's model host at `https://d1y2lf8k3vrkfu.cloudfront.net/openapi/en-us/dest/Changehistory_prod_3p.json` and read 2026-11-09. No guide page for it was found on the docs host (the paths tried are listed under Unreachable). [official]
+
+- **Operation.** `POST /history`, `operationId` `getHistory`, summary "History of entity changes." Permissions: `["advertiser_campaign_edit","advertiser_campaign_view"]`. [official]
+- **Scope, verbatim.** "Returns history of changes for provided event sources that match the filters and time ranges specified. Only events that belong to the authenticated Advertiser can be queried. All times will be in UTC Epoch format." … "This API only returns change history for Sponsored Products and Sponsored Brands campaigns. Changes for Sponsored Display campaigns are not returned. Also, it doesn't return information about who made the change." [official]
+- **IDs.** "This API accepts identifiers in either the alphamumeric format (default), or the numeric format. If numeric IDs are supplied, then numeric IDs will be returned otherwise, alphanumeric IDs are returned." (Amazon's spelling.) [official]
+- **Versioning.** "Use `Accept: application/vnd.historyresponse.v1.1+json` to access version 1.1 features. **THEME event type requires version 1.1 or higher.** Without the proper Accept header, THEME requests will return a 400 error." The model's 400 example: `"Invalid request: Error: Key in HistoryEventTypes: THEME is not supported."` [official]
+- **Request body** (required: `eventTypes`, `fromDate`, `toDate`):
+  - `fromDate`, `toDate`: integers, epoch milliseconds in the model's example (`"fromDate": "1704195640000"`). `fromDate`'s description: "Max 90 days of history." (figure for human readers only)
+  - `count`: "Requested number of results. Default 100. Minimum 50. Maximum 200." (figures for human readers only)
+  - `nextToken`, or `pageOffset`: "Mutually exclusive with 'nextToken'. Max results with pageOffset is 10000. Use nextToken instead for more results." `totalResults` is "Only provided for pageOffset queries and capped at 10000."
+  - `sort`: `{ "key": "DATE", "direction": "ASC" | "DESC" }`, default `DESC`.
+  - `eventTypes`: an object keyed by `AD`, `AD_GROUP`, `CAMPAIGN`, `KEYWORD`, `NEGATIVE_KEYWORD`, `PRODUCT_TARGETING`, `THEME`. Each holds `eventTypeIds` ("max of 10 event types. IDs here belong to the EventType. For example, if requesting CAMPAGIN as the eventType, these are campaignIds."), `filters`, and `parents` ("maximum of 10 parents", each `campaignId`, `adGroupId`, `useProfileIdAdvertiser`: "If true, retrieves events which belong to the associated advertiser").
+  - `filters` enum: `BID_AMOUNT`, `BUDGET_AMOUNT`, `END_DATE`, `IN_BUDGET`, `NAME`, `PLACEMENT_GROUP`, `SMART_BIDDING_STRATEGY`, `START_DATE`, `STATUS`; the description maps them: `BUDGET_AMOUNT`, `IN_BUDGET`, `END_DATE`, `START_DATE`, `PLACEMENT_GROUP`, `SMART_BIDDING_STRATEGY` to CAMPAIGN; `STATUS` to CAMPAIGN, AD_GROUP, AD, KEYWORD, PRODUCT_TARGET, NEGATIVE_KEYWORD; `BID_AMOUNT` to AD_GROUP; `NAME` to CAMPAIGN, AD_GROUP. [official]
+- **Response.** `events[]`, `nextToken`, `totalResults`. Each event: `entityType` (`AD`, `AD_GROUP`, `CAMPAIGN`, `KEYWORD`, `NEGATIVE_KEYWORD`, `PRODUCT_TARGETING`, `THEME`), `entityId`, `changeType`, `previousValue`, `newValue`, `timestamp` ("unix time"), and `metadata` (`campaignId`, `adGroupId`, `campaignBudgetType` `DAILY`/`LIFETIME_BUDGET`, `keyword`, `keywordType` `KEYWORD_BROAD`/`KEYWORD_EXACT`/`KEYWORD_PHRASE`, `negativeTargetingType` `NEGATIVE_KEYWORD_BROAD`/`_EXACT`/`_PHRASE`, `placementGroupPosition` `DETAIL_PAGE`/`TOP_OF_SEARCH`, `predefinedTarget` `CLOSE-MATCH`/`COMPLEMENTS`/`LOOSE-MATCH`/`SUBSTITUTES`, `productTargetingType` `EXPRESSION`/`PREDEFINED`, `targetingExpression`, `targetingType` and `theme` for THEME). [official]
+- **Change types by entity**, from the `changeType` description table: CAMPAIGN "STATUS, IN_BUDGET, PORTFOLIO, BUDGET_AMOUNT, NAME, START_DATE, END_DATE, SMART_BIDDING_STRATEGY, PLACEMENT_GROUP, CREATED, IS_PREMIUM_BID_ENABLED, PORTFOLIO_ID, BUDGET_DOCUMENT_VERSION, IS_PROCESSING, RULE_BASED_BUDGET_VALUE, APPLICABLE_RULE_NAME, APPLICABLE_RULE_ID, CREATION_DATE, AUTO_TARGETED"; AD_GROUP "STATUS, NAME, DEFAULT_BID_AMOUNT"; AD "STATUS"; KEYWORD "STATUS, BID_AMOUNT"; PRODUCT_TARGETING "STATUS, BID_AMOUNT"; NEGATIVE_KEYWORD "STATUS". [official]
+- **The model disagrees with itself.** The `changeType` enum lists only `BID_AMOUNT`, `BUDGET_AMOUNT`, `DEFAULT_BID_AMOUNT`, `END_DATE`, `IN_BUDGET`, `NAME`, `PLACEMENT_GROUP`, `PORTFOLIO`, `SMART_BIDDING_STRATEGY`, `START_DATE`, `STATUS`. The description table names more for CAMPAIGN (`RULE_BASED_BUDGET_VALUE`, `APPLICABLE_RULE_NAME`, `APPLICABLE_RULE_ID`, `IS_PREMIUM_BID_ENABLED`, `AUTO_TARGETED`, and others). Which of these a reply actually carries is read from the reply. [official; the reconciliation is seat's]
+- **Value forms**, from the `previousValue`/`newValue` table: STATUS "CREATED, ENABLED, PAUSED, ARCHIVED"; IN_BUDGET "true or false"; PORTFOLIO "portfolioId"; BUDGET_AMOUNT and BID_AMOUNT and DEFAULT_BID_AMOUNT "amount in marketplace currency"; START_DATE and END_DATE "unix time"; SMART_BIDDING_STRATEGY "OPTIMIZE_FOR_SALES, LEGACY, MANUAL"; PLACEMENT_GROUP "number as percent". Both fields are typed `string`. [official]
+- **Gaps against the campaign model** [seat, from comparing the two models read 2026-11-09]:
+  - The history's bidding-strategy values (`OPTIMIZE_FOR_SALES`, `LEGACY`, `MANUAL`) are not the Sponsored Products 3.0 names (`AUTO_FOR_SALES` "Dynamic bids - up and down", `LEGACY_FOR_SALES` "Dynamic bids - down only", `MANUAL` "Fixed bid", `RULE_BASED`, `OTHER`). No `RULE_BASED` value is listed in the history table.
+  - `placementGroupPosition` lists `DETAIL_PAGE` and `TOP_OF_SEARCH`; the SP 3.0 placement enum has four (`PLACEMENT_TOP`, `PLACEMENT_PRODUCT_PAGE`, `PLACEMENT_REST_OF_SEARCH`, `SITE_AMAZON_BUSINESS`). A rest-of-search or Amazon Business adjustment may not appear in the history.
+  - No event type for negative product targets, campaign-level negative targets, audience or shopper-cohort bid adjustments, ad creative edits (Sponsored Brands headline, image or video), budget-rule association, or optimization rules was found in the model.
+  - No Sponsored Display, Sponsored TV or DSP events.
+- **The console view.** A trade-press article reports a History tab for Sponsored Products and Sponsored Brands campaigns, and for Sponsored Products ad groups, giving "a summary of all changes made to campaign settings in the past 90 days", and says "The history feature is available in Amazon DSP." Amazon's own announcement URL (`advertising.amazon.com/blog/history-now-available-for-sponsored-products-and-sponsored-brands`) now lands on a general Sponsored ads page. [opinion, trade press: Luis Rijo, PPC Land, "Amazon rolls out a new history feature for Sponsored Products and Sponsored Brands campaigns", byline date 2020-01-20 as shown, read 2026-11-09]
+- **Other history-shaped reads.** The budget-rule history endpoints (`GET /sp|sb|sd/campaigns/{campaignId}/budgetRules/budgetHistory`) were announced for deprecation on August 31, 2023 [official] (All releases, "Awareness: Budget rules history endpoints to be deprecated on August 31, 2023", read 2026-11-09). The Sponsored Products 3.0 model read 2026-11-09 has no `budgetHistory` path.
+
+### B. The write's own receipt (RULE: the API contract)
+
+The change history is one witness. The other is the reply to the write itself, which only exists if it is kept when it arrives.
+
+- **Ads API v1 writes answer 207.** `POST /adsApi/v1/update/campaigns` (`operationId` `UpdateCampaign`) answers `207` with `CampaignMultiStatusResponseWithPartialErrors`: `success[]` (each `{ index, campaign }`, `index` 0–999), `partialSuccess[]` (`{ index, campaign, errors }`) and `error[]` (`{ index, errors }`, where the model's `ErrorsIndex.index` reads 0–19), each array capped at 1,000 entries. The returned `Campaign` carries `lastUpdatedDateTime` ("The date time that the campaign was last updated."), `creationDateTime`, `state`, `status` and `budgets`. Ad groups, ads and targets have the same multi-status shape (`AdGroupMultiStatusResponseWithPartialErrors`, `AdMultiStatusResponseWithPartialErrors`, `TargetMultiStatusResponseWithPartialErrors`). "Batch size limits are specific to each ad product." [official] (Amazon Ads API v1 merged OpenAPI, `AmazonAdsAPIALLMerged_prod_3p.json`, read 2026-11-09)
+- **Sponsored Products 3.0 objects carry their own timestamps.** Every SP 3.0 `extendedData` (campaign, ad group, product ad, keyword, negative keyword, targeting clause, negative targeting clause, campaign negatives) has `creationDateTime`, `lastUpdateDateTime` ("Last updated date in ISO 8601."), `servingStatus` and `servingStatusDetails`. [official] (Sponsored Products 3.0 OpenAPI, `SponsoredProducts_prod_3p.json`, read 2026-11-09). Note the spelling: `lastUpdateDateTime` in SP 3.0, `lastUpdatedDateTime` in v1.
+  - The object is returned when the list request sets `includeExtendedDataFields: true`. The model's description of that flag names the fields differently from the object it returns: "Whether to get entity with extended data fields such as creationDate, lastUpdateDate, servingStatus", while the returned schemas (for example `SponsoredProductsCampaignExtendedData`) hold `creationDateTime`, `lastUpdateDateTime`, `servingStatus` and `servingStatusDetails`. The field names a reply carries are the schema's. [official] (SP 3.0 OpenAPI, re-read 2026-11-09 for the gap fill; `method/ads-api-surface.md` §H owns the request shape and records the flag's own wording)
+- **Labels on objects.** v1 campaigns and targets have `tags`, "Open ended labels with a key value pair applied to the campaign" (and "…target"); the v1 Campaign entity guide marks `tags.Tag[].key` and `.value` as supported for Sponsored Products, Sponsored Brands and DSP. A labels API also exists (`/adsApi/v1/query/labels`, `/adsApi/v1/query/labelObjectAssociations`; `LabelObjectAssociation` with `labelObjectId` "The identifier of the advertising object this label is applied to (e.g., a deal ID, ad group ID)"). [official] (v1 merged OpenAPI; Campaign entity guide, `https://advertising.amazon.com/API/docs/en-us/guides/campaign-management/entities/campaign`, read 2026-11-09 from its markdown source)
+- **A lost reply.** cx-amazon-ads' door reads every `POST` as a change; a `POST` whose reply was lost reads `sent, outcome unknown`, and a redirect is followed nowhere. [door.md, ComChatX's own behaviour, read 2026-11-09] Whether the change landed is then read from the object (its `lastUpdatedDateTime` / `lastUpdateDateTime` and the field itself) or from `POST /history`. [seat]
+- **What no Amazon read holds.** Neither the history nor the write reply holds the expected effect, the windows, the reason, the metric to be judged, or who asked. Those exist only if they are written down before the change. [seat]
+
+### C. When a window's figures arrive, and when they stop moving (RULE about processing; GUIDANCE on waiting; figures for human readers only)
+
+Three Amazon pages give three different horizons for "final". All three were read 2026-11-09 from the docs site's markdown host.
+
+- **Sponsored ads reporting FAQ (version 3)** (`guides/reporting/v3/faq`):
+  - "Initial impression and click data is available using the API within 12 hours. Note that small changes to impression and click data may happen up to three days after the initial click date due to the traffic validation process. We recommend retrieving click and impression data again after three days to ensure you have the latest data."
+  - "Initial conversion data is available within 24 hours, but may change as part of the restatement process. Restatements of conversion data occur 1, 7, and 28 days after the conversion event. Because conversions are reported on the date of the ad interaction, restatement includes all dates in your attribution window. For example, if your attribution window is 14 days, restatement can update attribution up to 42 days (14 plus 28) after the report date."
+  - "We currently recommend 1-2 report requests, per advertiser, per day, for each report type. This is based on data refresh rates."
+  - The page opens: "**Reporting API v3 will be deprecated on June 30, 2027.**" [official]
+- **Amazon Ads reporting overview** (`guides/reporting/overview`):
+  - "The invalid traffic detection system monitors traffic data for up to 30 days after the fact to recognize invalid traffic patterns. We recommend waiting at least 5 days for the most accurate click data to be available before evaluating campaign performance."
+  - "Conversion data is refreshed and validated at daily, weekly, and monthly intervals and is subject to change for up to 60 days back from the current date. We recommend re-retrieving reports monthly to help ensure your conversion metrics are up to date."
+  - "Amazon Marketing Stream v2 delivers data updates and automated backfill up to 70 days on subscription creation, so your data self-corrects as attribution validation completes." [official]
+- **Amazon Marketing Stream v2** (datasets overview and `ads-performance` dataset page):
+  - "Records can be restated after first delivery as attribution data matures. A restated record arrives as a new version of the same key. Each dataset defines a restatement window — the period during which a record can still change — in its dataset specifications. Treat a record as final only after its window has passed."
+  - `ads-performance-v1`: grain "Hourly", "Restatement window | 70 days". [official] (Overview [Stream v2 datasets], `guides/amazon-marketing-stream/v2/datasets/overview`; Ads performance, `guides/amazon-marketing-stream/v2/datasets/ads-performance`; both read 2026-11-09)
+- **Amazon Ads help, Traffic validation** (GBYPYH79NGJJ5JPS, `lastModified` 2026-11-02; read 2026-11-09 for the gap fill from the article JSON embedded in the page, after the earlier pass found only a script shell): "Most post-impression IVT is invalidated in real time. Some takes longer to detect, so we monitor traffic for up to 30 days after it occurs. IVT detected in this window is automatically removed from invoices and advertiser reports". "If an investigation identifies previously undetected IVT, affected metrics and invoices may be restated." "IVT metrics are available in report center and in Sponsored Ads Gross and Invalid Traffic report", as click and impression invalidation rates per campaign per day. [official] (Traffic validation, `https://advertising.amazon.com/help/GBYPYH79NGJJ5JPS`). So a click count is not final for 30 days, and an investigation can restate it later still.
+- **The horizons do not agree.** 3 days (clicks, FAQ), 30 days (invalid traffic, overview), up to 42 days for a 14-day window (FAQ), 60 days (overview) and 70 days (Stream v2). The FAQ's 42 days is an example built on a 14-day window; with a 7-day window the same arithmetic gives 35 days [seat]. The Stream page is the only one that states the rule as "final only after its window has passed". [official, with the comparison seat's]
+- **Dating.** "All conversion metrics are now reported on 'traffic date,' meaning conversions are reported on the date of the ad interaction rather than the date the conversion occurred." Example: "If a shopper clicked an ad on Oct 1 and purchased on Oct 3, the purchase conversion is now reported on Oct 1". [official] (Overview of the Reporting v1 API, `guides/reporting/ads-v1/overview`, read 2026-11-09)
+- **Look-back.** "**Sponsored Products:** Base metrics automatically align with campaign defaults—7-day look-back for sellers and 14-day look-back for vendors—consistent with the advertising console and downloadable reports." "**Other campaign types:** The standard look-back is 14 days". v3 columns keep the window in the name: `purchases7d` "Number of attributed conversion events occurring within 7 days of an ad click"; `purchasesSameSku7d` "…where the purchased `SKU` was the same as the `SKU` advertised"; `salesOtherSku7d` "…where the purchased `SKU` was different from the `SKU` advertised". [official] (Overview of the Reporting v1 API; Reporting columns (version 3), `guides/reporting/v3/columns`; both read 2026-11-09)
+- **Days are campaign-timezone days.** "By default, performance is measured and data is aggregated based on each campaign timezone, which can also be included in the report via the `timeZone.value` dimension field." "Future releases will include support for UTC time zone normalization". The change history's timestamps are "UTC Epoch". [official] (Dates and historical data retention, `guides/reporting/ads-v1/dates`; Changehistory model; both read 2026-11-09)
+- **Retention** (figures for human readers only): v1 `hour.value` 14 days (max range 14 days), `date.value` 24 months (max range 120 days), `week.value` and `day.value` 24 months (15 months), `month.value` 6 years (25 months); benchmark dimensions "Available from 6/1/2025". v3 campaign reports: 95 days for SP, and 60, 65, 95, 95 days for the other columns of the page's table. The change history: 90 days. [official] (Dates and historical data retention; Campaign reports (v3), `guides/reporting/v3/report-types/campaign`; Changehistory model; all read 2026-11-09) A baseline older than 90 days has its performance data but no change history from `POST /history`. [seat]
+- **Search terms below a threshold disappear.** The v3 FAQ: "The API search term report only contains impressions that also had at least one click." Stream v2's dataset page: "In this dataset, `searchTerm.value` is subject to reporting thresholds"; the datasets overview says values under the threshold are "grouped into a row with value `(Other)`" and "Totals stay complete: the sum of all rows at a grain, including `(Other)`, matches the totals at coarser grains." [official] (Sponsored ads reporting FAQ (version 3); Ads performance [Stream v2]; Overview [Stream v2 datasets]; all read 2026-11-09)
+- **Ratios are aggregated, then divided.** "When you compute a derived metric such as ROAS (sales ÷ total cost), aggregate the numerator and the denominator across all rows at your grain before dividing, because the components can sit on different rows." "The dataset does not include calculated metrics such as ROAS or CTR." [official] (Ads performance [Stream v2], read 2026-11-09)
+- **Promoted and halo.** "All conversion metrics now include Promoted and Halo qualified versions"; base metrics such as `metric.purchases` "now include *all* conversions—both promoted and halo", and, in a warning addressed to DSP integrations, "To report attribution for promoted products only, **use metrics with the Promoted qualifier**". [official] (Overview of the Reporting v1 API, read 2026-11-09). Detail is `beyond/metrics-and-attribution.md`'s.
+- **Warnings on a v1 report.** Since 2026-10-05 a completed v1 report may carry `warnings[]`, each `{ field, warningCodes[] }`, for example `NO_PERMISSION` (values redacted) or `PENDING_DEPRECATION`; "Treat `warningCodes` as an open-ended list". [official] (All releases, "Warning signals now returned on new Reporting API reports"; New features [Reporting v1], `guides/reporting/ads-v1/new-features`; both read 2026-11-09)
+
+### D. What Amazon changes inside a window (RULE where it is a documented mechanism; GUIDANCE where it is advice)
+
+These are documented mechanisms by which a campaign's bids, budgets, placements, creative or serving change with no write from the seller or cx.
+
+- **Dynamic bidding.** SP 3.0 `SponsoredProductsBiddingStrategy`: `AUTO_FOR_SALES` "Dynamic bids - up and down": "Increases or decreases your bids in real time by a maximum of 100%."; `LEGACY_FOR_SALES` "Dynamic bids - down only": "Lowers your bids in real time when your ad may be less likely to convert to a sale."; `MANUAL` "Fixed bid": "Uses your exact bid and any placement adjustments you set, and is not subject to dynamic bidding."; `RULE_BASED`. [official] (Sponsored Products 3.0 OpenAPI, read 2026-11-09)
+  - The ceiling by placement. Amazon's current help page and library guide say 100% for all placements; one Amazon page, the author's guide, still says 50% outside top of search. The help page *Bidding strategies for Sponsored Products* (GCU2BUWJH2W3A8Z7, `lastModified` 2026-09-28): "We may increase or decrease your bids by up to 100% for all placements based on performance", with "Example: Amazon can adjust your bid of $1.00 up to $2.00 for all placements"; for down only, "We may decrease your bids by up to 100% for all placements based on performance." The up-and-down library guide: "We will increase or decrease your bids by up to 100% for all placements based on performance", and "Currently, we can increase this bid by up to 100%, so that would be a maximum of $2.00 for all placements opportunities". The author's guide (written for book authors): "Bids will be increased by up to 100% for placements at the top of the first page of shopping results and by up to 50% for all other placements." [official] (Bidding strategies for Sponsored Products, `https://advertising.amazon.com/help/GCU2BUWJH2W3A8Z7`; Guide to dynamic bidding - up and down with Sponsored Products, `https://advertising.amazon.com/library/guides/dynamic-bidding-sponsored-products`; The author's guide to Sponsored Products, `https://advertising.amazon.com/library/guides/authors-guide-to-sponsored-products`; all re-read 2026-11-09). `areas/bids-and-placements.md` owns this question. Its P-bids-and-placements-8 counts four current Amazon sources for "100% for all placements" (help, library guide, SP 3.0 and v1 models) against the author's guide, and notes that no Amazon page dates a change from 100/50. That file records practitioners still repeating 100/50: BidX and Perpetua, and Trellis "according to a third-party summary of Amazon's announcement". This file follows it. [seat]
+  - Placement adjustments stack: "a 50% Top of Search adjustment on a $1.00 bid would increase the bid to $1.50 … A further 100% Amazon Business adjustment would increase the bid to $3.00 for the Amazon Business Top of Search placement and to $2.00 for all other Amazon Business placements." [official] (SP 3.0 OpenAPI, `SponsoredProductsPlacement`)
+- **Rule-based bidding** moves base bids and can undo itself: "Amazon Ads may then adjust your base bids up and down to increase conversions while maintaining your guardrails." "if your campaign is not meeting the guardrail and the return on ad spend (ROAS) of the campaign for the last 20 days drops to less than 30% of your ROAS over a 20-to-40-day period, we will disable the rule on your behalf and will enable your previous bidding strategy and targeting level bids." Eligibility: "running for at least 10 days prior to setting a rule and have a minimum of 10 conversions in the last 30 days", plus a minimum daily budget per marketplace (US 10, JP 600, IN 300, … local currency; figures for human readers only). "the ROAS guardrail is based on the previous 21 days." [official] (Sponsored Products rule-based bidding overview, `guides/rules/bidding-rules/overview`, read 2026-11-09)
+- **Budget rules** raise budgets without a write on the day: "Schedule-based rules … temporarily increases campaign budgets by the percentage specified in the rule." "Performance-based rules … increase campaign budgets when your campaigns meet a certain performance threshold … calculated using data from the previous 7 days for sellers and 14 days for vendors." [official] (Sponsored ads budget rules overview, `guides/rules/budget-rules/overview`, read 2026-11-09). v3 `spCampaigns` carries `campaignRuleBasedBudgetAmount` ("The value of the rule-based budget for a campaign."), `campaignApplicableBudgetRuleId` and `campaignApplicableBudgetRuleName`. [official] (Reporting columns (version 3))
+- **Out of budget** is a serving state, not a write: SP 3.0 campaign `servingStatus` values include `CAMPAIGN_OUT_OF_BUDGET`, `PORTFOLIO_OUT_OF_BUDGET`, `ACCOUNT_OUT_OF_BUDGET`, `ADVERTISER_OUT_OF_BUDGET`; `POST /sp/campaigns/budget/usage` returns `budgetUsagePercent` ("Budget usage percentage (spend / available budget) for the given budget policy.") and `usageUpdatedTimestamp` ("Last evaluation time for budget usage"). The history records `IN_BUDGET` "true or false" on CAMPAIGN. [official] (SP 3.0 OpenAPI; Changehistory model)
+- **The Featured Offer and stock gate.** "Sponsored Products only appear when your advertised items are in stock"; "Your products need to be in stock to be the featured offer and eligible for advertising." [official] (Best practices for your Sponsored Products ads, `https://advertising.amazon.com/library/guides/sponsored-products-best-practices`, read 2026-11-09). The product ad's `servingStatus` names it: `AD_NOT_IN_BUYBOX`, `NOT_IN_BUYBOX`, `AD_OUT_OF_STOCK`, `OUT_OF_STOCK`, `NO_INVENTORY`, `AD_NOT_BUYABLE`, `AD_NO_PURCHASABLE_OFFER`, among others; v1 `Status.deliveryStatus` is `DELIVERING`, `LIMITED`, `NOT_DELIVERING` or `UNAVAILABLE`, with `deliveryReasons` (93 values in the model, including `NOT_IN_BUYBOX`, `OUT_OF_STOCK`, `CAMPAIGN_OUT_OF_BUDGET`, `NOT_IN_POLICY` and the `MODERATION_*` policy reasons). [official] (SP 3.0 OpenAPI `SponsoredProductsAdServingStatus`; v1 merged OpenAPI `DeliveryStatus`, `DeliveryReason`; both read 2026-11-09). `beyond/ad-products-and-eligibility.md` owns the eligibility detail.
+- **Recommendations applied from the console or an agent.** "AI-powered guidance for sponsored ads: Quickly identify opportunities to improve Sponsored Products campaigns with recommendations for targeting, bids, and budgets—and apply them in one click. All of these capabilities will roll out over the coming months." The same page: advertisers can "let Amazon's AI optimize on their behalf". [official] (Amazon Ads introduces Full-Funnel Campaigns, DVA+, and enhanced chat experience in AI-powered ad platform, Amazon Ads Agent (HTML title "Introducing Amazon Ads Agent, your AI-powered ad platform"), `https://advertising.amazon.com/library/news/amazon-ads-agent`, 2026-09-29, read 2026-11-09). The unBoxed Tokyo page repeats the one-click sentence for Japan, 2026-11-05 (unBoxed Tokyo 2026 AI-powered ad innovations news, read 2026-11-09). The v1 `Recommendation` object has `state` (`RecommendationState`: `ACTIVE`, `APPLIED`, `DEFERRED`, `DISMISSED`, `EXPIRED`, `INACTIVE`) and `potentialImpacts`, "Estimated customer benefits if recommendation is followed". [official] (v1 merged OpenAPI; Recommendations API overview, `guides/recommendations/recommendations-api/overview`, read 2026-11-09) Because the history "doesn't return information about who made the change", an applied recommendation and a seller's own edit look the same in it. [seat]
+- **New placements switched on by Amazon.** "campaigns will be automatically enrolled in prompts"; "we will begin to charge for these ads as part of your CPC bidding and billing parameters"; GA in the U.S. "on March 25, 2026"; prompts show in "shopping results and product detail pages on Amazon"; "If you'd like to pause a prompt, you can do so at any time in Ad Console." [official] (Sponsored Products prompts and Sponsored Brands prompts, `https://advertising.amazon.com/resources/whats-new/unboxed-2025-sponsored-products-and-sponsored-brands-prompts`, read 2026-11-09). cx-listing-playbook's `method/amazon-change-calendar.md` §C also lists Alexa+ serving (from 2025-07-15) and the "Off-Amazon ad serving" bulksheet column (2026-06-08); both release-note headings were re-read in All releases on 2026-11-09 ("Alexa+ tests product discovery through conversational AI", July 15, 2025; "New column header "Off-Amazon ad serving" for Sponsored Products campaigns in bulksheets", June 8, 2026) [official]. The SP 3.0 campaign carries `offAmazonSettings.offAmazonBudgetControlStrategy` (`MAXIMIZE_REACH`, `MINIMIZE_SPEND`) [official] (SP 3.0 OpenAPI, read 2026-11-09). A trade-press report says Amazon's help article "Understand Sponsored Products off-Amazon advertising" was revised on 2026-08-04 to add creators in the Amazon Influencer Program as an off-Amazon destination from 2026-08-10, with campaigns "enrolled by default at their existing bids and budgets" [opinion, trade press: Luis Rijo, PPC Land, "Amazon Sponsored Products campaigns gain creator placements on August 10", 2026-08-04, read 2026-11-09; the Amazon help article itself was not read].
+- **Creative Amazon rewrites.** Sponsored Brands headline DCO: "Advertisers can consent to leveraging DCO during a campaign to enhance or generate new headlines", set by `creativePropertiesToOptimize: ["HEADLINE"]`; "Based on internal testing, headline text optimization can drive up to 1.6% `CTR` lift vs. non-optimized campaigns." (2025-04-17). Auto collections: "Amazon dynamically curates the most relevant products for each shopper" (2026-02-18; worldwide 2026-09-23 per the release list). [official] (All releases, read 2026-11-09)
+- **Reserve prices move the price paid with no bid change.** Amazon's help page *Understand Amazon Ads auctions* (GEXGZNBBWPZZK7P8, `lastModified` 2026-04-27): "reserve pricing functions may affect the cost of your ad. Some reserves help allocate ad space by setting a bid threshold. Your bid must meet this amount to be allocated, and you'll be charged at least this amount if your ad wins. Other reserves influence your final CPC or vCPM based on factors including the likelihood that the ad will lead to a sale, predicted return on your ad spend, the value of any competing bids, placement, context, and predicted performance (e.g., the same ad placement may have a different value during a peak shopping period than a non-peak period). As a result, the price charged to you may exceed the runner-up bid but will never exceed the maximum bid you authorize." [official] (Understand Amazon Ads auctions, `https://advertising.amazon.com/help/GEXGZNBBWPZZK7P8`, read 2026-11-09). Amazon's response to the FTC: "we began to test a concept called "soft reserve prices," a real-time minimum value that seeks to better reflect what each placement is actually worth"; "When the winning advertiser's bid exceeds both the hard and soft reserve, they pay the soft reserve … When the winning advertiser's bid exceeds the hard reserve but doesn't meet the soft reserve, we still grant the placement to that advertiser and they pay their bid." [official] (Amazon's response to the FTC's lawsuit regarding Sponsored Ads, `https://www.aboutamazon.com/md/company-news/amazon-ftc-sponsored-ads-lawsuit-response`, read 2026-11-09). A reserve set in real time, and valued differently at peak, moves the CPC of a target whose bid and whose rivals' bids did not change. [seat] Detail and the dispute are `areas/bids-and-placements.md`'s and `beyond/brand-defense-and-competition.md`'s.
+  - The complaint's allegations about event periods (allegations, untested, disputed by Amazon): "during the Christmas holiday season and other peak shopping events (referred to as 'high velocity events' or 'HVEs'), Amazon often raises its maximum surcharge levels" (¶68); Amazon allegedly decided "to mask future surcharge increases by gradually implementing them in advance of peak shopping days" (¶72); "In 2021 and 2022, Amazon raised surcharge constraints... during Prime Day and after Thanksgiving" (¶226). [opinion: regulator and state plaintiffs; quoted from `areas/seasons-and-deals.md` §E3, which read the public filing, `https://www.illinoisattorneygeneral.gov/News-Room/Current-News/FILED%20PUBLIC%20Amazon%20Complaint%208.31.26.pdf`, on 2026-11-09; this seat did not re-open the filing]. If those allegations hold, a pre-event window's CPC rise can be a price change by Amazon, not a competitor's move. [seat]
+- **Amazon tests changes to its auction and ranking continually.** The auctions help page: "We are continually experimenting to refine and improve our ad auctions, customer experience, and the performance of the ads we show. This might include introducing, removing, or modifying the data points or factors used in the auction process, or introducing new formats and features that help make ads more useful to customers while delivering returns to advertisers." [official] (Understand Amazon Ads auctions). Amazon's FTC response: "In 2014, we started testing advanced machine learning-based relevance models … We intended to test this over a long period of time, and in fact did, before we rolled out broadly." [official] (Amazon's FTC response). Amazon-authored search papers report live A/B tests and launches on product search: "An online A/B test on 50 million queries" of empirical-Bayes cold-start priors (Han, Castells, Gupta, Xu & Salaka, CIKM 2022); "The online A/B test on 140MM queries" (Gupta et al., The Web Conference 2020, as read in `areas/launch.md`); and substitute-based behavioural boosting for new items: "The BFS-enhanced ranking model has been launched in production and has served customers since 2025" (Dong, Momma, Wang, Gao & Sun, arXiv 2602.14502). [research, Amazon-authored; see Published research] A seller is not told when a search shopper is in such a test. Amazon's experimentation platform is called Weblab in Amazon job listings found by search; no amazon.science or Amazon Ads page read describes it, so this file does not rely on it. [seat; Unreachable]
+- **Competitors respond inside the window.** Amazon's FTC response: "On active campaigns, from 2019 to 2024, eighty percent of bid changes on clicked Sponsored Products search ads occurred within one day of a prior change"; "in one study, when one advertiser stopped bidding on specific keywords, average bids for those keywords from competitors dropped about 83%, from $15.96 to $2.74." [official, Amazon's figures; for human readers only] (Amazon's FTC response, read 2026-11-09). A pause or a bid cut can change rivals' bids on the same term, so the observation window measures the change together with the market's response to it. [seat]
+- **Amazon-side incidents.** "Advertisers can see the current status of services like campaign creation, campaign management, ad delivery, and reporting on sponsored ads and Amazon DSP." Status history "spanning the past 6 months" is at `https://status.ads.amazon.com/history.json` (schema at `…/history.schema.json`): each item has `region`, `category` (`Sponsored ads`, `Amazon DSP`, `Measurement & Reporting`, `Status announcement`, `Brand content`, `Amazon Marketing Stream`), `subcategory` (`Ad delivery`, `Campaign creation`, `Campaign management`, `Creative management`, `Bulksheets`, `Reports`, `Stores`, `Posts`, `Streams`), `currentStatus`, `statusMessages[]` with `timestamp`, `startTime`, `endTime`. [official] (Amazon Ads Status, `https://advertising.amazon.com/API/docs/en-us/support/status`, read from its markdown source; history.schema.json; both read 2026-11-09). The live file on 2026-11-09 held 15 items; its `region` values read `EU`, `NA`, `FE`, not the schema's long names, and its items carry `id` where the schema names `hashedId`. [official, as observed]
+
+### E. Amazon's own experiment and lift tools for ads, as of 2026-11-09 (RULE on scope; GUIDANCE on use)
+
+- **Manage Your Experiments (MYE) tests listing content, not ads.** "test different versions of your product images, titles, bullet points, descriptions, and A+ Content"; eligibility needs a Professional account, a Brand Representative for a Brand Registry brand, and an ASIN that "has received enough traffic in recent weeks"; "we recommend running experiments for 8 to 10 weeks", "as soon as four weeks" with "to significance". Bids, budgets, targeting, ad creative, price and coupons are not on the list. [official] (Test listing content with Manage Your Experiments, `https://sell.amazon.com/tools/manage-your-experiments`, read 2026-11-09). An MYE experiment running on an advertised ASIN changes the page the ad lands on for half its shoppers, which makes it a confounder for an ad verdict on that ASIN [seat]; cx-listing-playbook's effect-measurement §A–I holds the MYE mechanics.
+- **Sponsored Brands: several ads in one ad group.** Amazon's v4 features guide: "you can test different versions of the same ad formats within the same ad group to get a better understanding of shopper behavior", and "You can test targeting against different ad formats or landing pages and measure the performance." The 2024-06-13 release note lists "Testing different creative and landing page combinations within ad groups." [official] (Sponsored Brands v4 features, `guides/sponsored-brands/v4-features`; All releases; both read 2026-11-09). **No page read says how impressions are split among the ads of one ad group** (random, even, or optimized toward the better ad). Without that, two ads in one ad group are a comparison, not a randomized test. [seat; Open question 2]
+- **No Amazon split-test tool for Sponsored Brands creative or for Stores was found (searched 2026-11-09 for the gap fill).** Several searches were run: Amazon Ads What's new, the library, the help centre, the Ads API docs' full-text search index (`searchIndex.json`, about 8,000 entries, searched for "holdout", "A/B", "experiment", "split test", "control group", "lift") and Sell on Amazon. None found an Amazon page offering a Sponsored Brands creative experiment or a Store A/B test with a shopper split and a result. What Amazon's pages do offer:
+  - Sponsored Brands v4: several ads in one ad group "to test", with no stated split (above). The SB v3-to-v4 migration guide repeats it: "Brands can test different creatives within the same ad group, enabling advertisers to compare the performance". [official] (Ads API docs search index, `reference/migration-guides/sb-v3-v4`, read 2026-11-09)
+  - Manage Your Experiments includes A+ Brand Story, the "From the Brand" module on the detail page: "Registered brands with eligible products can now run A/B tests on Brand Story content with Manage Your Experiments." It was posted by the forum account "News_Amazon", shown as "3 years ago". This tests a detail-page module, not the Store. [official, Amazon's news account] (Seller Forums EU, "Brands can now run A/B tests on Brand Story content with Manage Your Experiments", `https://sellercentral-europe.amazon.com/seller-forums/discussions/t/b3cc84fe-5747-48e3-a3d6-9ee5e27409a7`, read 2026-11-09)
+  - Amazon's own case-study page calls a two-campaign comparison an A/B test. A partner "A/B test[ed] this style of creative on campaigns that already had proven results", comparing an AI-image campaign with the original campaign "With comparable impression volumes" (2,368 clicks at 1.1% CTR against 1,292 at 0.6%; for human readers only). [official page, partner's account] (Trellis helps lift Dandy Blend's results with AI creative, `https://advertising.amazon.com/library/case-studies/dandy-blend-trellis-creative-success`, read 2026-11-09)
+  - In AMC, the Brand Store insights paid dataset's `ingress_type` lists store traffic sources as "0 - uncategorized / default 1 - search 2 - detail page byline 4 - ads 6 - store recommendations 7 through 11 - experimentation". The page does not say what "experimentation" traffic is. [official] (AMC data source *brand_store_insights_paid*, `https://d3a0d0y2hgofx6.cloudfront.net/en-us/guides/amazon-marketing-cloud/datasources/brand_store_insights_paid.md`, read 2026-11-09; Open question 14)
+  - So the documented answer to "can we A/B this SB creative" is: several ads in one SB ad group with no stated split, or two campaigns. Neither is a randomized split of shoppers on the pages read. Sagum's "built-in" claim (What practitioners add) names no tool. [seat]
+- **Amazon DSP has a randomized customer holdout; sponsored ads have none on the pages read.** "Amazon DSP provides 10 Customer holdout groups for A/B testing. Each group contains 10% of randomly assigned Amazon customers, allowing you to create controlled experiments." The setup: "Create two identical orders", "Split users 50/50 between control and test", "Randomly assign 5 holdout groups to control, 5 to test", and "Turn off **Automated budget optimization** to ensure your testing line items are not affecting each other". Also: "Randomize group assignments for each new test to avoid potential cross contamination between experiments or any long-term bias." And Amazon's own warning: "We do not recommend comparing performance before and after implementing bid adjustments as market conditions and other factors vary over time." [official] (How to test DSP bid adjustments, `https://d3a0d0y2hgofx6.cloudfront.net/en-us/guides/dsp/bid-adjustments/testing.md`, read 2026-11-09). Among the Ads API docs, this is the only page found in the search index whose text holds "holdout". It is a DSP audience. The v1 Target entity guide gives Sponsored Products and Sponsored Brands no audience target of this kind (§F). [seat, from the index search and §F]
+- **Creative Testing (Amazon DSP, survey).** "Creative Testing helps you choose the right image or video to include in their upcoming brand campaigns." It is on the Studies page of Amazon DSP, "Self-service", "North America: United States", launched "January 30, 2023". It surveys panelists before launch; it is not a live split of shoppers. [official] (Identify the right creative for an upcoming brand campaign with Creative Testing, `https://advertising.amazon.com/resources/whats-new/creative-testing`, read 2026-11-09)
+- **Amazon Brand Lift (survey, exposed vs control).** Measures "customer awareness, attitudes, preferences, favorability, intent, and ad recall" through "surveys, issued to both an ad-exposed audience and an unexposed control group"; results report "absolute lift, which is the difference between the ad-exposed group's and control group's response rate"; panelists from the Amazon Shopper Panel, "an opt-in program". "All ad campaigns run through Display, Video, and Audio campaigns and sponsored ads are eligible", with Sponsored Products excluded; "Amazon Brand Lift is free as long as campaigns meet spend and impression minimums" (minimums not stated); "results are ready in as few as 10 business days after submission"; "Amazon Brand Lift controls are available via the Amazon Ads API and to all Amazon Ads Agent advertisers in the United States, Canada, Germany, Spain, France, Mexico, Japan, and United Kingdom. They are also available to sponsored ads advertisers in the United States." [official] (Brand Lift: Study the impact of your advertising, `https://advertising.amazon.com/measurement-analytics/brand-lift`, read 2026-11-09). GA for "Sponsored Display, Sponsored Brands, and Sponsored TV campaigns" in the US on "June 30, 2024", eligible users "Brand owners", via the Studies page or the API. [official] (Measure brand marketing impact using Amazon Brand Lift, `https://advertising.amazon.com/resources/whats-new/measure-brand-marketing-impact-using-amazon-brand-lift`, read 2026-11-09). **API paths: three Amazon texts, and the model decides.** The OpenAPI model, `Measurement_prod_3p.json` (`info.title` "Measurement", version "3.0", re-read 2026-11-09 for the gap fill), defines `POST /dsp/measurement/eligibility/brandLift` (`CheckDSPBrandLiftEligibility`), `GET`, `POST` and `PUT /dsp/measurement/studies/brandLift` (`GetDSPBrandLiftStudies`, `CreateDSPBrandLiftStudies`, `UpdateDSPBrandLiftStudies`, the last "a full update"), and `GET /measurement/studies/brandLift/{studyId}/result` (`GetDSPBrandLiftStudyResult`). Each operation's description reads "Authorized resource type: DSP Rodeo Entity ID, DSP Advertiser Account ID", and each takes an `Amazon-Ads-AccountId` header. The result call "Returns a 307 Temporary Redirect response if any of the file types is requested". The 2023 release note "French advertisers can now measure brand marketing impact using Amazon Brand Lift" names the model's paths, except that it gives `PUT /measurement/studies/brandLift` where the model has `/dsp/…`. The 2024 launch page for sponsored ads names `POST /measurementStudies/brandLift`, `PUT /measurementStudies/brandLift/{studyId}` and `GET /measurementStudies/brandLift/{studyId}/result`. No `/measurementStudies/…` path is in the model. [official] (Measurement OpenAPI, `https://d1y2lf8k3vrkfu.cloudfront.net/openapi/en-us/dest/Measurement_prod_3p.json`; Measure brand marketing impact using Amazon Brand Lift; All releases; all read 2026-11-09). A call is checked against the model. The launch page's sponsored-ads paths have no model behind them on the pages read. [seat; `areas/ads-organic-and-total.md` §G2 agrees]
+- **Amazon Marketing Cloud (AMC): analysis, not randomization.** AMC is a clean room of "Pseudonymized inputs from Amazon Ads campaign events such as ad impressions, ad clicks, and ad-attributed conversions"; "raw event information cannot be exported"; a sponsored ads account can "automatically create an AMC instance for that account and start using AMC within minutes" (self-service access). [official] (How AMC works, `guides/amazon-marketing-cloud/how-amc-works`, read 2026-11-09). Outputs pass aggregation thresholds: "Medium" and "High" columns appear only in rows with "at least one hundred distinct users"; `user_id` never. [official] (Data aggregation thresholds in AMC, `guides/amazon-marketing-cloud/aggregation-threshold`, read 2026-11-09). Since 2026-10-20 the Amazon Retail Purchases dataset carries "13 promotional signal columns" so "advertisers can separate the effect of their media from the effect of their deals", through AMC Paid Features, with history from 2024-09-24. [official] (All releases, "Promotional signals now available in the Amazon Retail Purchases dataset", read 2026-11-09). **No Amazon page read documents a randomized holdout, a ghost-ad control, or a lift study for Sponsored Products inside AMC.** [seat, from the pages read; the search budget ran out before a wider search, see Open question 1]
+- **Long-term sales (LTS), and a way to check it.** LTS is "ad-attributed measures that estimate the incremental sales value your brand can expect to generate over the next 12 months" (2025-06-05); v3 `longTermSales` is "The 12 month potential value of new-to-brand customer-brand engagements … plus the 14d total sales of the campaign." Since 2026-09-25 (US) the v1 API carries horizons `metric.longTermSales30d`…`180d`, cohort estimates `metric.cohortLongTermSales*`, and actual cohort sales `metric.cohortAccumulatedSales30d`…`365d` by `longTermSalesCohort.value` (e.g. `2026M05`); "Compare LTS estimates against actual accumulated sales from new-to-brand customers to validate estimate accuracy." Cohort metrics "aren't summable across cohorts". [official] (All releases; Reporting columns (version 3); both read 2026-11-09). LTS is a model estimate, partly a forecast; it is not a measured effect of a change. [seat]
+- **Benchmarks (peer percentiles).** v1 benchmark metrics at P25/P50/P75 for CPC, CTR, CPM, ROAS, purchase rate, cost per purchase and more, against "similar brands in your category", via dimensions benchmark brand, browse category, category marketplace and campaign goal (from 2026-09-25 in Unified Reporting; Sponsored Ads report type `crossProgramBenchmarks` GA worldwide 2026-05-18). [official] (All releases, read 2026-11-09). A category's P50 CPC series is a series the seller's change did not cause (at the scale of one seller), so it can serve as a demand-side or auction-side control [seat].
+- **Other measurement surfaces.** Amazon Attribution (beta; tagged off-Amazon media; "clicks and click-attributed conversion metrics"), Brand Metrics (open beta; "Awareness and Consideration indices that compare your performance to peers using models predictive of consideration and sales"), and the Marketing Mix Modeling API (aggregated sales and media signals; NA region, 21 marketplaces). [official] (Amazon Attribution API overview; Brand Metrics API open beta overview; Marketing Mix Modeling overview; all read 2026-11-09). None is a randomized test of a Sponsored Products change.
+- **Multi-touch attribution (beta) divides credit; it does not test a change.** "Multi-touch attribution is new campaign measurement that divides credit for Amazon purchase conversions across Amazon Ads touchpoints in proportion to their value"; it "uses insights from experiments, machine learning, and historical shopping signals"; "Amazon Ads' traditional, standard attribution is "last touch"". Announced at unBoxed 2024 for US advertisers, shown as Orders, Sales and ROAS "(multi-touch)" beside last-touch metrics. [official] (Uncover ad value across the funnel with multi-touch attribution (beta), `https://advertising.amazon.com/resources/whats-new/uncover-ad-value-across-funnel-with-multi-touch-attribution`, read 2026-11-09). An Amazon-authored paper describes the RCTs behind it: "one group of shoppers is eligible to see ads from the focal campaign ("treatment") while another group of shoppers is not ("holdout")", forming "a database of hundreds of thousands of RCTs across our ad products"; the paper does not say that an advertiser can run or read one of these RCTs. [research, Amazon-authored: Lewis, Zettelmeyer, Gordon et al., "Amazon Ads Multi-Touch Attribution", arXiv:2508.08209 (2025), read 2026-11-09].
+  - **Where the fields are.** The launch page says "Multi-touch metrics can also be accessed in most report types for version 3 of the Amazon Ads API, as well as Amazon Marketing Stream conversion datasets for Amazon DSP, Sponsored Products , Sponsored Brands, and Sponsored Display." The *Reporting columns (version 3)* page has no column whose name or description contains "multi-touch" or "multitouch" (searched again 2026-11-09). Reporting v1's *Purchases* metric page defines 56 `metric.multiTouch*` fields. Examples are `metric.multiTouchPurchases` ("The number of purchases attributed to an ad interaction. This is measured using multi-touch attribution.", required field `metric.purchases`), `metric.multiTouchSales`, `metric.multiTouchRoas` and `metric.multiTouchSalesPromoted`. There are `FromClicks`, `FromViews`, `Promoted`, `Halo`, `Converted` and `Reconciled` variants. Each field gives "Version 3 Reporting Name (Sponsored Ads): N/A", and the compatible dimensions include `Target`, `Targeting` and `Search term`. [official] (Uncover ad value across the funnel with multi-touch attribution (beta); Reporting columns (version 3); Purchases (Amazon retail conversions) (v1), `https://d3a0d0y2hgofx6.cloudfront.net/en-us/guides/reporting/ads-v1/metrics/amazon-retail-conversions/purchases.md`; all read 2026-11-09). `beyond/metrics-and-attribution.md` M1–M2 owns these fields. Whether a seller's Sponsored Products rows carry values in them is read from a report. [seat]
+- **Third-party measurement studies are a DSP workflow.** "Amazon Ads launched a self-service workflow for third-party measurement studies in 18 countries", with results "within Amazon DSP" (2026-05-15). [official] (Self-Service Third Party Measurement Studies, `https://advertising.amazon.com/resources/whats-new/self-service-partner-measurement-studies`, read 2026-11-09)
+- **Amazon's own "lift" claims are not experiments a seller can repeat.** Full-Funnel Campaigns: "Advertisers using Full-Funnel Campaigns in beta observed 67% higher Long-Term ROAS* compared to the same formats configured independently, and acquired new-to-brand customers at 29% lower cost per purchase*", footnoted "Results based on Full-Funnel Campaigns against the blended average of other campaign performance across beta advertisers. Amazon internal (June – August 2026). Early beta advertiser data; individual results may vary." [official] (Grow total sales with Full-Funnel Campaigns, `https://advertising.amazon.com/resources/whats-new/full-funnel-campaigns-from-discovery-to-sales`, 2026-09-29, read 2026-11-09); Improvado's Ivo Petrovic reads the footnote as "not a holdout test on your account" [opinion: "Amazon unBoxed 2026: What Changed for Retail Media Metrics", 2026-10-06, read 2026-11-09]. The Ads Agent announcement: "advertisers using natural language to apply campaign targeting recommendations saw more than 25% additional unique customers while lowering costs by more than 10% for cost-per-impression on their campaigns on average", with no method stated [official] (Amazon Ads Agent announcement). SB DCO's "up to 1.6% CTR lift" is "Based on internal testing" [official]. These are vendor-reported figures, not priors for one seller's change. [seat]
+
+### F. What a seller cannot randomize on Amazon (RULE where the API says so)
+
+- **No geography for Sponsored Products or Sponsored Brands.** The v1 Target entity guide's support table marks `targetDetails.locationTarget` and `LocationTarget.locationId` for the common model and Amazon DSP only; the Sponsored Products and Sponsored Brands columns are empty. The same holds for `dayPartTarget`, `deviceTarget`, `audienceTarget` and the other DSP-only target types. Sponsored Products targets are `keywordTarget`, `productTarget`, `productCategoryTarget` and `themeTarget`. [official] (Target entity guide, `guides/campaign-management/entities/target`, read 2026-11-09 from its markdown source). A geo holdout of the Blake et al. kind (ads off in a random set of regions) cannot be run on Sponsored Products through Amazon's targeting. [seat, from the table]
+- **No shopper split.** No Ads API object read assigns shoppers to arms for a Sponsored Products bid, budget, target or negative. MYE splits shoppers, but only for listing content (§E). [official, by absence on the pages and models read; seat's reading]
+- **Time is the lever left, and the seller can randomize it.** A seller can switch a change on and off by date (an on-off or "pause" test) or run two campaigns side by side. Which days or weeks are "on" can be decided by a coin flip per period, which is what makes a switchback randomized (Bojinov, Simchi-Levi & Zhao 2023; Published research). A fixed alternation, or a before/after, is not randomized. [seat]
+- **Duplicate targets: what Amazon states.** The match-type help page: "If you set multiple match types with different bids for the same keyword, then the match type with the highest bid will be used for any matching shopping query." The targeting guide: "And don't worry, you won't be bidding against yourself by adding different match types on the same keyword." On several campaigns for different products: "if you have multiple campaigns where you're bidding on category keywords that are relevant to multiple advertised products, set bids based on your priority for each product; this will give your top priority product the best opportunity to get top ad placements, while still giving your lower priority product(s) a chance to also appear on the search results page." [official] (Understand keyword match types, `https://advertising.amazon.com/help/GHTRFDZRJPW6764R`, `lastModified` 2026-07-29; A guide to targeting with Sponsored Products; both re-read 2026-11-09). The first sentence covers one keyword in several match types. No page read says whether it reaches across ad groups or campaigns, which of two copies of one keyword in the same match type enters an auction, or whether the bid compared is the base bid or the placement-adjusted one. [seat; `areas/keyword-and-product-targeting.md` and `areas/campaign-structure.md` P-56 own the rule and record the same limits]
+- **Same-keyword arms are not a shopper split.** The A/B designs practitioners recommend put the same keyword in two ad groups (ClearAds) or two campaigns (Sagum, Marin) for the same product. Within the scope of Amazon's sentence, the arm whose match type carries the higher bid "will be used for any matching shopping query". Outside it, Amazon states no rule. Either way, Amazon, not a coin, decides which arm serves a given search, and the arms meet in the same auctions and, in ClearAds' design, under one campaign budget (Basse et al. 2016). Arms that differ only in a creative or a landing page, for different targets or different products, are not covered by the match-type sentence. [seat, from the quotes above and Published research]
+- **Budget-constrained arms interfere.** When a test arm and a control arm draw on budgets or share an auction, research shows the measured effect can be badly biased (Basse, Soufiani & Lambert 2016; Liu, Mao & Kang 2020; see Published research). [research]
+
+### G. What Amazon says about judging a change (GUIDANCE; figures for human readers only)
+
+- "We recommend that you limit your changes while testing strategies, so that you can attribute the difference in performance to a specific change." "You should review your bids often, about once every two weeks." "Take time to test and learn to find what the best approach is for you and your products." "Ensure you are prepared to potentially spend double the amount of the bid you input." [official] (Guide to dynamic bidding - up and down with Sponsored Products, read 2026-11-09)
+- "Wait about two weeks after the launch of your campaign" before adjusting the bidding strategy; "Make sure to check in every two weeks after that." "Every few weeks, make time to analyze how different keywords and strategies perform." [official] (The author's guide to Sponsored Products, read 2026-11-09; a guide written for book authors, whose examples are books and readers)
+- On negatives: "we recommend evaluating its performance after it has received at least 20 clicks"; "let your negative keywords run for two weeks or more, before making any changes to your strategy." On match types: "Test different match types for at least 1-2 weeks to gather enough data before pausing any keywords." On auto campaigns: "let your automatic campaign run for about two weeks before creating a manual campaign." On baselines: "You should establish benchmarks based on your goals and the historical performance of your campaigns." "Then, measure your performance against these to determine if your strategy is meeting your objectives." [official] (A guide to targeting with Sponsored Products, `https://advertising.amazon.com/library/guides/targeting-with-sponsored-products`, read 2026-11-09)
+- "In the first week, monitor your metrics at least twice a week to evaluate the effectiveness of your campaign" and "Adjust budget and bids based on performance." [official] (Best practices for your Sponsored Products ads, read 2026-11-09)
+- "We recommend waiting at least 5 days for the most accurate click data to be available before evaluating campaign performance." [official] (Amazon Ads reporting overview, read 2026-11-09)
+- Theme-based bid suggestions: "The bid suggestions are calculated from a group of recent winning bids for similar products"; the impact metrics "are based on historic performance of similar products to provide guidance on choosing a bid, and are not estimates of expected performance. They are also computed assuming your campaign does not run out of budget." [official] (Theme-based bid suggestions - Quick-start guide, `guides/sponsored-products/bid-suggestions/theme-based-bid-suggestions-quickstart-guide`, read 2026-11-09)
+- **None of these is an evaluation rule.** No Amazon page read states a statistical test, a minimum sample or a confidence level for judging an ad change. The 20 clicks and the two weeks are recommendations on those pages. [seat]
+
+### H. What a report row says about state, and what it does not (RULE: field definitions; gaps marked)
+
+- v3 `spCampaigns` can carry `campaignBudgetAmount` ("Total budget allocated to the campaign."), `campaignBiddingStrategy` ("The bidding strategy associated with a campaign."), `campaignStatus`, `campaignRuleBasedBudgetAmount`; `spTargeting` and `spSearchTerm` carry `keywordBid` ("Bid associated with a keyword or targeting expression."). v1 reports can carry `campaign.bidStrategy`, `campaign.budgetAmount`, `campaign.budgetType`, `campaign.deliveryStatus` (example query in the v1 New features page). [official] (Reporting columns (version 3); New features [Reporting v1]; both read 2026-11-09)
+- **Neither page says whether a daily row's `keywordBid` or `campaignBudgetAmount` is the value on that day or the value when the report ran.** [seat; Open question 4]. The change history is the documented record of when a value changed.
+- `topOfSearchImpressionShare`: "The percentage of top-of-search impressions earned out of all the top-of-search impressions that were eligible for a given date range. Various factors determine the eligibility for an impression including campaign status and targeting status." [official] (Reporting columns (version 3)). A bid change's first-order effect on exposure is visible here and in impressions before any sales effect shows. [seat]
+
+### I. The confounder series Amazon publishes outside the Ads API (RULE: field and file names; read by the sibling packages)
+
+- **SP-API Data Kiosk, seller analytics** (`analytics_sellerAnalytics_2025_03_31.graphql`, `amzn/selling-partner-api-models`, read 2026-11-09): `traffic.sessions`, `traffic.featureOfferPct`, `traffic.conversionRatePct` (the older `conversationRatePct` is `@deprecated(reason: "No longer supported. Use conversionRatePct instead.")`), `sales.unitsOrdered`, `sales.averageSellingPrice`, `promotion.promotionType`. Read through `cx-amazon-sp-api`. Readiness and null rules are recorded in cx-listing-playbook's effect-measurement §C. [official]
+- **SP-API Data Kiosk, economics** (`analytics_economics_2024_03_15.graphql`, read 2026-11-09): `AdSummary` with `adTypeName` ("for example: \"Sponsored Products charge\"") and `charge`: ad spend per product from the seller's ledger, with no impressions or clicks. [official]
+- **SP-API report schemas** present in the repository on 2026-11-09 include `sellerSalesAndTrafficReport.json`, `sellingPartnerSearchQueryPerformanceReport.json`, `sellingPartnerSearchCatalogPerformanceReport.json`, `sellerCouponReport.json`, `promotionReport.json`. [official] (GitHub contents listing of `schemas/reports`, read 2026-11-09)
+- **Keepa** (vendor, read through `cx-keepa`): the product object's `csv` types include `AMAZON`, `NEW`, `SALES` (sales rank), `LIGHTNING_DEAL`, `COUNT_NEW`, `RATING`, `COUNT_REVIEWS`, `BUY_BOX_SHIPPING`, with `couponHistory`, `buyBoxSellerIdHistory`, `monthlySold` and `monthlySoldHistory`. [opinion, vendor: Keepa `Product.java`, `github.com/keepacom/api_backend`, read 2026-11-09]
+
+## What practitioners add
+
+Each claim below is the named source's, not Amazon's. Where it conflicts with an Amazon page, the conflict is marked.
+
+- **Ad Badger ("The Badger"), "Attribution Window Blind Spots: How 14-Day Windows Quietly Wreck Your Decisions", 2026-08-13** [opinion] (`https://www.adbadger.com/blog/attribution-window-blind-spots-how-14-day-windows-quietly-wreck-your-decisions/`, read 2026-11-09):
+  - States Sponsored Products as "14-day click attribution", Sponsored Brands the same, Sponsored Display "14-day click attribution plus 14-day view-through attribution".
+  - States "Amazon's standard advertising reports are built by order date, not by click date".
+  - Recommends judging only a "closed cohort", "one where every click in the range is now more than 14 days old"; "A 3-5 day evaluation window is usually safe" for impulse products; for mid-consideration products "Evaluating at day 3 systematically understates true performance", with day 7–10 suggested.
+  - Worked example (illustrative, not a study): 100 clicks, 2 sales by day 3 (about 90% ACOS), 8 by day 14 (about 22%).
+  - **Conflicts with Amazon.** Amazon's v1 overview gives Sponsored Products a 7-day look-back for sellers (14 for vendors) and says conversions are reported on "traffic date", the date of the ad interaction (§C). The article's warning that recent days are immature agrees with Amazon; its window and dating claims do not.
+- **Ad Badger, "You're Optimizing Too Much: 3 Amazon PPC Habits to Quit Today", 2026-06-02, a write-up of a PPC Den podcast episode with Amazon PPC consultant Clément Hynaux** [opinion] (`https://www.adbadger.com/blog/amazon-ppc-optimization-mistakes/`, read 2026-11-09). In the blog's own voice: "every change you make resets Amazon's learning loop"; "Always use at least seven days of data before making major decisions"; "Even 14 days with only 30 clicks might not be enough to draw conclusions". Quoted from Hynaux: "Yesterday's data or today's data is just a rough draft. We're missing a lot of data, we're missing a lot of sales."; "Clément recommends removing the last 48 hours from your analysis as a baseline". The same post says a late sale "gets attributed back to the original click date", which contradicts Ad Badger's own August 2026 article (above) and agrees with Amazon. No data behind the "learning loop" claim is given, and no Amazon page read describes one for manual bids.
+- **Ad Badger help centre, article 196** [opinion, a tool's own behaviour] (`https://support.adbadger.com/hc/knowledge-base/articles/196-…`, read 2026-11-09): its automated bidder "typically allow[s] a 24-hour window, plus or minus 15 minutes, before making further changes." A tool that changes bids daily makes a multi-day verdict on any one of its changes impossible by construction. [seat]
+- **BidX, Selina Krüger, "Understanding the Amazon Attribution Window and its impact?", 2021-03-22** [opinion] (`https://bidx.io/blog/amazon-attribution-window`, read 2026-11-09): SP "7 days", SB "14 days"; "we recommend waiting until the seventh day after the observation period before making adjustments if the difference from the target ACoS on the day of observation is rather small" (the recommendation is conditional on a small gap to target); example: February's SP data reviewed no earlier than 7 March. This matches Amazon's seller look-back and traffic-date rule, but ignores the restatement Amazon describes after the look-back.
+- **AdLabs, Andrew Bailiff, "Amazon PPC Bid Optimization: How Long Should You Wait Before Adjusting Bids Again?", 2025-09-03** [opinion] (`https://adlabs.app/amazon-ppc-bid-optimization-how-long-should-you-wait-before-adjusting-bids-again/`, read 2026-11-09): "Most sellers optimize once a week or every other week"; "Because of the sales attribution delay, ACOS will always look a little high for the last couple of days"; "What's accurate immediately after a round of optimizations is your spend trend. Sales and ACOS take longer to reflect true performance". The author argues against a tool logic that waits for "X number of clicks" after a bid change, because a bid cut too far may never collect them. No data cited.
+- **Pare, George Reid, "Amazon PPC Governance: Cut Q4 Bid Changes by 82%", 2026-10-19** [opinion] (`https://pare.so/blog/amazon-ppc-governance-decision-matrix-q4-bid-changes-2026`, read 2026-11-09): "A minimum of 7–14 days of post-change data before the next change on the same target. Not 3 days. Not 5." Cites one account (Gabriel Caceros) cut from "11 interventions in 9 days down to 2 in 2 weeks", reported secondhand, single account, no primary source. Recommends auditing the change history and warns against "no documented rationale attached to each change".
+- **Seller Forums, "How often should we be updating bids in the PPC", two sellers, shown "8 years ago"** [opinion, sellers, no Amazon staff reply] (`https://sellercentral.amazon.com/seller-forums/discussions/t/2f0b934c4850ae38be6d08910effc64b`, read 2026-11-09): one watches "12 days" before judging a keyword; another lets a keyword "collect data for a while – month or longer", noting that "the difference between a sky-high ACoS and a manageable one is just a handful of sales".
+- **Marin Software, Andrew Todd, "Testing effectively with Amazon Sponsored Brands campaigns", 2019-06-20** [opinion] (`https://www.marinsoftware.com/blog/testing-effectively-with-amazon-sponsored-brands-campaigns`, read 2026-11-09): "you can choose to run campaigns concurrently (one test and one control), or iteratively (week one control, week two test)"; warns of "an apples to oranges scenario that may happen if volume is different for reasons beyond your control"; "change one variable, keep everything else consistent."
+- **ClearAds Agency, "Mastering A/B Testing in Amazon PPC", 2024-01-02** [opinion, no named author] (`https://clearadsagency.com/amazon-ppc/ab-testing/`, read 2026-11-09): two ad groups in one campaign with the budget split evenly, run "during the same periods"; "at least a week"; "limit the A/B tests to one variable at a time". Its own example changes imagery, copy, match type, bidding strategy and placement bids together.
+- **Sagum, Chase Sagum, "How do I use A/B testing for Amazon ad creatives?", 2026-04-19 (updated 2026-05-13)** [opinion] (`https://sagum.com/2026/04/19/how-do-i-use-a-b-testing-for-amazon-ad-creatives/`, read 2026-11-09): "Amazon's A/B testing tools for Sponsored Brands are built-in" (no tool named); "For Sponsored Products, you must manually run two separate campaigns"; "at least 1-2 weeks, accounting for the full weekly purchase cycle". The post opens "While our core expertise at Sagum lies in platforms like Facebook, Instagram, and TikTok". No Amazon page read names a built-in SB A/B tool beyond several ads per ad group (§E).
+- **SalesDuo, Srushti P. Borle, "Amazon Ad Creative Testing Framework", updated 2026-08-27** [opinion] (`https://salesduo.com/blog/amazon-ad-creative-testing-framework`, read 2026-11-09): describes Amazon's Creative Testing as using "Amazon DSP surveys to learn which image or video connects better with a selected audience before launch" — consistent with Amazon's page (§E).
+- **Perpetua, Apoorva Ashok Kumar, "How to identify incremental opportunities on Amazon", 2024-10-23** [opinion] (`https://perpetua.io/blog-incrementality-amazon-ads-optimization-strategies/`, read 2026-11-09): for branded terms "you can analyze the incremental sale as a result of changing your Branded Keyword strategy"; for category terms incremental sales "can be measured using the Total ACoS"; attributed sales can rise while total sales do not. No data cited.
+- **Eva Commerce, Hai Mag, "Amazon Ad Incrementality: Measurement Guide for 2026", 2026-04-17** [opinion] (`https://eva.guru/blog/amazon-ad-incrementality-measurement/`, read 2026-11-09): proposes "a well-documented geo-holdout test approach" for branded Sponsored Products ("Run branded search in the control market, pause it in the test market") over "2-3 weeks"; says AMC holdout studies "work best for DSP" because "suppressing individual users from search auctions is technically harder"; claims "In many cases, 80-90% of traffic and sales that would have been attributed to branded ads return through organic channels when the paid ad is removed" without a source. **Conflicts with Amazon's API:** Sponsored Products and Sponsored Brands have no location targeting (§F), so a geo holdout cannot be set inside one Amazon marketplace's Sponsored Products campaigns. Across marketplaces (for example US against CA) is possible, but those are different markets, not randomized regions. [seat]
+- **FiveX, Lisa van Broekhoven, "Amazon DSP + AMC incrementality profit test", 2026-07-23** [opinion; FiveX sells the platform described] (`https://fivex.com/blog/amazon-dsp-amc-incrementality-profit-test/`, read 2026-11-09): a DSP retargeting example with a "matched control group, suppressed from DSP" giving incremental ROAS 1.9 against 8.0 attributed. No group sizes, intervals or significance; the cases are labelled "Named example" (for example "Nora Home"), with no source for their figures.
+- **Seller Labs, "Denis D" (the byline as shown), "Sales Dropped? Split It Before You Raise Ad Spend", 2026-10-01** [opinion] (`https://www.sellerlabs.com/blog/amazon-sales-drop-traffic-conversion-price/`, read 2026-11-09): "Below a few hundred sessions a month, conversion swings on single orders"; "At 300 sessions, one point of conversion is 3 units. That is noise, not a trend."; low-traffic ASINs: "Use a 60- or 90-day window, or skip the ASIN"; where seasonality or impression share is the suspected cause, "Compare the same month last year before you act."; confounders named: coupons, Lightning Deals, Subscribe & Save mix, repricers, lost Featured Offer, stockouts, hijackers, traffic mix from broad or auto campaigns, new negative reviews, competitor undercuts; "Traffic data can run several days behind".
+
+**Where practitioners disagree, with each other or with Amazon:**
+1. *The Sponsored Products look-back.* Ad Badger (2026) says 14 days; BidX (2021) says 7 days; Amazon says 7 days for sellers and 14 for vendors in its v1 overview (read 2026-11-09). The look-back is read from Amazon's metric definition for the account type at run time.
+2. *How conversions are dated.* Ad Badger's August 2026 article says reports are "built by order date" (its June 2026 post says the opposite, "attributed back to the original click date"); Amazon says "traffic date" (date of the ad interaction) for all v1 conversion metrics and for v3 sponsored ads ("Because conversions are reported on the date of the ad interaction…", FAQ).
+3. *How long after a change.* 24 hours (Ad Badger's bidder), "at least seven days" (Ad Badger blog, after Clément Hynaux), "once a week or every other week" as what "Most sellers" do (AdLabs, Andrew Bailiff), "7–14 days" (Pare, George Reid), "12 days" or "a month or longer" (sellers), "seventh day after the observation period" when the gap to target ACoS is small (BidX, Selina Krüger), "about once every two weeks" (Amazon's dynamic-bidding guide), "at least 5 days" before evaluating click data (Amazon's reporting overview). None gives a basis in counts; the research section does.
+4. *Counts against days.* Ad Badger and Amazon's targeting guide (20 clicks for a negative) speak in clicks; AdLabs argues against click thresholds; Seller Labs speaks in sessions and units. These are compatible: a window is long enough when it holds enough events, and days are a proxy. [seat]
+5. *Sponsored Products incrementality tests.* Eva proposes geo holdouts; the Ads API offers no geography for SP (§F). FiveX's and Eva's holdouts are DSP or unspecified. No practitioner page read describes a randomized Sponsored Products holdout that Amazon runs for a seller.
+6. *Creative tests in Sponsored Brands.* Sagum says the A/B tool is built in; Amazon's pages read describe several ads per ad group "to test" without a split method; SalesDuo names Creative Testing, which is a DSP pre-launch survey.
+
+7. *What "A/B test" means.* ClearAds, Sagum, Marin and Pattern use the term for two ad groups or campaigns run at once. So does the partner case study on Amazon's own site (Dandy Blend). Amazon's DSP page uses it for randomized customer holdout groups. Only the second assigns shoppers at random. [seat]
+
+**Gap fill 2026-11-09: the tool vendors' own methods.** These were searched and read for the gap fill. Each claim is the vendor's, about its own product.
+
+- **Skai, Kevin Weiss (VP of Retail Media), interview "Retail media incrementality: Skai's innovative approach", 2023-07-27** [opinion, vendor] (`https://skai.io/blog/retail-media-incrementality-skais-innovative-approach/`, read 2026-11-09). Calls holdout testing the "gold standard": spend paused on a test group while the rest continues as control. It gives Skai's working formula as "[incrementality score] x ROAS = iROAS". From his audits of "hundreds" of accounts, 5–15% of Amazon ad sales come from non-branded campaigns triggered by branded queries (no method stated). He calls retail media "an imperfect environment for testing". The page describes no Skai test design for a bid or budget change.
+- **Skai, Kevin Weiss, "Unlocking Incrementality Insights in Retail Media: Six Innovative Approaches", 2024-06-25** [opinion, vendor] (`https://skai.io/blog/unlocking-incrementality-insights-in-retail-media-six-innovative-approaches/`, read 2026-11-09). Six client cases. One is a third-party iROAS "calculated daily for each campaign and keyword". One is an MMM. One is a "product-holdout-based incrementality testing model" in an alpha project with Skai that combines AMC and Selling Partner API data, with test and control product groups chosen with the brand teams; the setup was more complex than expected. No geo, switchback or synthetic-control design is described.
+- **Pacvue, "Pacvue Debuts New Incrementality Console…", GlobeNewswire press release, 2024-09-17** [opinion, vendor] (`https://www.globenewswire.com/news-release/2024/09/17/2947513/0/en/Pacvue-Debuts-New-Incrementality-Console-to-Help-Advertisers-Grow-Incremental-Revenue-with-Accurate-and-Actionable-Insights.html`, read 2026-11-09). The iROAS Dashboard "utilizes dozens of inputs, data sources and multiple advanced linear regression data models", described as "best-in-class data science and machine learning techniques". The release names no experiment or holdout validation and no granularity. Pacvue's own pages answered HTTP 403 (Unreachable).
+- **Teikametrics help centre** [opinion, vendor: the tool's stated behaviour] (`https://help.teikametrics.com/en/articles/8358916-how-frequently-are-bids-changed.md`; `https://help.teikametrics.com/en/articles/9306896-how-to-speed-up-bid-exploration-in-your-campaigns.md`; `https://help.teikametrics.com/en/articles/8308845-intro-to-the-teikametrics-bidder.md`; all read 2026-11-09). Amazon bids are recalculated "Hourly", fed by "Amazon's AMS Streaming data". "Bid exploration is designed to keep your campaigns efficient by limiting the rate at which bids can increase within a 24-hour period". One tip temporarily raises a min-bid to `min_bid = ACOS Limit * 0.10 * Average_Order_Value`, which "sets the min-bid at a 10% conversion rate". The bidder article says "Both Amazon and Walmart run a second-price auction". That conflicts with Amazon's own description of reserves under which a winner can pay its own bid (§D). It claims "up to 30% increase in net revenue and up to 15% increase in CVR" with no method stated. No Teikametrics page read describes a holdout or a measurement of its own bid changes. Under hourly re-bidding, no single bid value holds long enough for a before/after read; the bidder's policy is the unit that can be judged. [seat]
+- **Intentwise, Michael Waters, "Speed up your ad analysis with What Changed", 2024-03-17** [opinion, vendor] (`https://www.intentwise.com/blog/speed-up-your-ad-analysis-with-what-changed`, read 2026-11-09). Compares two chosen periods "through a mix of machine learning and AI technologies" and ranks entities by an Impact Score built from ad spend, ad revenue and ACOS. In the post's own words it is for "preliminary trend identification" and "isn't supposed to be the *end* of your analysis. It's the starting point." A search-result summary of an Intentwise help-centre article on Change History says it separates changes the platform applied (when auto-apply bid recommendations are on) from changes by users. The article itself was not read: its help-centre URL answered 404 on 2026-11-09. No Intentwise page found describes an AMC lift or holdout method. Its 2026-02-18 webinar "The brand guide to incrementality: Boost profits with AMC and Brand Analytics" was found as a title only.
+- **Helium 10, "Helium 10 Ads Change Log Page", updated 2025-02-24** [opinion, vendor] (`https://kb.helium10.com/hc/en-us/articles/27613163890715`, read 2026-11-09 through the help centre's public article API, because the HTML answered 403). The log covers budget, bid, keyword, negative keyword and campaign status changes. The Change Source filter separates "Automatically in Helium 10 Ads", "Semi-auto in Helium 10 Ads", "Manually in Helium 10 Ads" and "In Seller Central or 3rd party". Amazon's `POST /history` "doesn't return information about who made the change" (§A). A tool that labels a change "In Seller Central or 3rd party" is inferring it from the absence of its own write. [seat]
+- **Perpetua, Laura Conboy, "Redefining Success with Amazon Marketing Cloud (AMC): Measuring What Matters", 2025-04-22** [opinion, vendor] (`https://perpetua.io/blog-amazon-marketing-cloud-amc-incrementality-growth-strategies/`, read 2026-11-09). Uses new-to-brand share, cost per new-to-brand customer against lifetime value, and AMC path-to-purchase as its measures. It names no holdout or lift test, and no measurement of a bid or budget change. Perpetua's 2024 post (above) reads incrementality from total ACoS.
+- **AdExchanger, James Hercher, "Beware The Platform Incrementality Trap", 2025-04-25** [opinion, trade press] (`https://www.adexchanger.com/commerce/beware-the-platform-incrementality-trap/`, read 2026-11-09). "The Amazon ad platform only recently began enabling incrementality tests". Per three unnamed measurement and agency sources, advertisers "couldn't create large holdout or control groups city by city or market to market". Incrementality test groups can now be built through AMC, but that program is in beta. "Incrementality reporting by a platform may be useful and directionally correct, but it is not true." The article does not discuss sponsored ads. No Amazon page read documents the AMC test-group program (§E).
+- **Optmyzr, Frederick Vallaeys, "Amazon Ads Reporting Delay Is Killing Your Conversions", 2025-06-11** [opinion, vendor study] (`https://www.optmyzr.com/blog/amazon-ads-reporting-data-delay-study/`, read 2026-11-09). This is the one measured size of report maturation found. Campaigns running on 2025-02-10 were pulled repeatedly over 17 days, "assuming the snapshot on February 27 was the most complete", from 302 accounts, 14 marketplaces and 14,991 campaigns, with 14-day attributed conversions and sales. For the top 5% of campaigns, attributed sales grew "by at least 18.75%" and impressions changed "by at least 36.67%" between day 1 and day 17. No median is given. Clicks and spend are called "relatively quickly and are much more stable", with no figure in the text. It advises against ROAS decisions in the first 3 days. Lakshmi Padmanaban's 2026-04-08 Optmyzr post repeats the figures (`beyond/metrics-and-attribution.md` P-33). Day 17 sits inside Amazon's own 28-day restatement point and 60- and 70-day horizons (§C), so the study measures maturation to day 17, not to final. [seat]
+- **Jonathan Hershaff, "Estimating Amazon Ad Incrementality for Sellers: How to Combine Switchback Tests with MMM", Substack, 2026-02-13** [opinion, seller-analyst] (`https://whatstheimpact.substack.com/p/estimating-amazon-ad-incrementality`, read 2026-11-09; also cited in `areas/ads-organic-and-total.md`). This is the one Amazon seller switchback found. Ads were alternated "On" and "Off" daily "for about a month" after Christmas (late December 2025 to January 2026), by a seller with "$20k annual ad spend". The outcome was total sales: "I only care if my **Total Sales** actually went up when the ads were on". The spend difference was about $55 a day, the ad-attributed sales difference about $129 (platform ROAS about 2.35), and the total-sales difference about $68.56 (naive incremental ROAS 1.25). A Google Meridian MMM on the same data estimated adstock decay of about 0.35 ("~35% of ad pressure bleeds into the next day"). Dividing by the adstock difference ($74.72 against $30.29, so $44.43) gives 1.54. The MMM's own ROAS had median 1.42, mean 1.79 and a 90% credible interval of 0.27–4.57. He writes "A daily switchback assumes marketing has zero memory" and treats 1.25 as a "floor". The switch times were a fixed daily alternation. The page says nothing about randomizing them. [seat] Two further seat readings follow. The 1.54 uses a decay estimated on the same month's data, so the two estimates are not independent checks. And under Amazon's traffic-date reporting (§C), an "on"-day click that converts on an "off" day is credited to the "on" day in ad reports but counted on the "off" day in total sales.
+- **Pattern, Bailey Bezzant, "The Importance of Sponsored Brands A/B Testing", 2021-08-13** [opinion] (`https://www.pattern.com/blog/the-importance-of-sponsored-brands-a-b-testing`, read 2026-11-09). One variable per test, campaigns named "– Product Test A/B", and a two-month test period "in which a good or bad one-month period can be ruled out".
+- **Flywheel, Bernie Che, "How advertisers can use Amazon Ads solutions as a creative testing engine", 2026-06-12, a partner article on Amazon's site** [opinion, partner] (`https://advertising.amazon.com/library/expert-advice/flywheel-creative-learning`, read 2026-11-09). Measures creative tests in three layers: immediate response, then audience engagement, then funnel outcomes, through AMC "and proprietary measurement frameworks". No randomized split is described for sponsored ads.
+
+8. *Where the vendors disagree.* Skai and AdExchanger call holdouts the standard. Pacvue sells regression-modelled iROAS with no holdout stated. Intentwise ranks period-over-period differences and calls the result a starting point. Teikametrics changes bids hourly and judges no single change. Helium 10 logs and labels changes but states no effect estimate. None of the pages read describes a vendor-run randomized test of a Sponsored Products bid or budget change. That matches Amazon's documentation, which offers randomized customer holdouts only in DSP (§E). [seat]
+
+## Published research
+
+- **Blake, Nosko & Tadelis, "Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment", NBER Working Paper 20171 (May 2014); *Econometrica* 83 (2015) 155–174** [research] (abstract at `https://www.nber.org/papers/w20171`; full text `https://www.nber.org/system/files/working_papers/w20171/w20171.pdf`; both read 2026-11-09).
+  - *Brand-keyword on-off test.* "eBay conducted a test of brand keyword advertising (all queries that included the term eBay, e.g., "ebay shoes") by halting SEM queries for these keywords on both Yahoo! and Microsoft (MSN), while continuing to pay for these terms on Google, which we use as a control in our estimation routine." "The experiment revealed that almost all (99.5 percent) of the forgone click traffic from turning off brand keyword paid search was immediately captured by natural search traffic". The 99.5% is reached "once the seasonality is accounted for" and is "a lower bound of retention".
+  - *Non-brand geo test.* "We designate a random sample of 30 percent of eBay's U.S. traffic in which we stopped all bidding for all non-brand keywords for 60 days." Candidate DMAs were drawn at random and "divided into test and control DMAs using an algorithm that matched" them on history; the design "lends itself to a standard difference-in-differences estimation".
+  - *Finding.* "Because search clicks and purchase behavior are correlated, we show that returns from paid search are a fraction of conventional non-experimental estimates." "brand-keyword ads have no measurable short-term benefits." For non-brand terms, "new and infrequent users are positively influenced by ads" but "more frequent users whose purchasing behavior is not influenced by ads" carry most of the spend.
+  - *Counter-evidence on brand terms.* Coviello, Gneezy & Goette ran "a similar experiment with Edmunds.com, arguably a more representative company, and found starkly different results. More than half of the paid traffic is lost when we shut off paid-links search." [research] (Coviello, Gneezy & Goette, "A Large-Scale Field Experiment to Evaluate the Effectiveness of Paid Search Advertising", CESifo Working Paper 6684 (2017), abstract at `https://econpapers.repec.org/RePEc:ces:ceswps:_6684`, read 2026-11-09). Simonov, Nosko & Rao (below) find that the own-brand ad matters most when competitors bid on the name. Taken together, the eBay 99.5% is one site's answer, not a general rate. [seat; `beyond/brand-defense-and-competition.md` P-26 holds the same pair]
+  - *Transfer to Amazon.* The setting is a retailer buying ads on search engines, with its own site in the organic result. On Amazon the seller's own product is often in the organic results below its ad, so the substitution mechanism applies; but a paused brand-term ad on Amazon can be replaced by a competitor's ad in the same slot, which eBay's brand test did not face on its own name (see Simonov et al.). The on-off design transfers; the geo randomization does not, because Sponsored Products has no geography (§F). The control in the brand test was another platform (Google) running unchanged; on Amazon the analogues are another marketplace, another product line, or the same terms' organic and total figures. [seat]
+- **Simonov, Nosko & Rao, "Competition and Crowd-Out for Brand Keywords in Sponsored Search", *Marketing Science* 37(2) (2018) 200–215** [research] (abstract at `https://ideas.repec.org/a/inm/ormksc/v37y2018i2p200-215.html`, read 2026-11-09). "large-scale experiments on Bing with data from thousands of brands": "When no competitors are present, we find a positive, statistically significant impact of brand ads of 1%–4%, with larger brands having a smaller causal effect." "competitors in paid positions 2–4 can "steal" 1%–5% of the focal brand's clicks"; for brands that do not advertise on their own brand search, "competitors "steal" 18%–42% of clicks, suggesting a strong causal effect of position." *Transfer:* the incremental value of a brand-term ad depends on whether competitors bid on the term; a pause test on a brand term measures the effect under that week's competitor bids, which the seller does not see. [seat]
+- **Moshary, "Does Sponsored Search Advertising Augment Organic Search? Evidence from an E-Commerce Platform", *Management Science* 71(11) (2025) 9687–9709** [research] (abstract at `https://ideas.repec.org/a/inm/ormnsc/v71y2025i11p9687-9709.html`, read 2026-11-09; author from the IDEAS record). "I leverage a field experiment that blocks all sponsored search advertising for 3% of site visitors. The experiment shows that sponsored search advertising both steals business from organic listings and also reduces the total volume of transactions intermediated by the platform." *Transfer:* a platform-run shopper holdout is the design that can answer the question; only the platform can run it. [seat]
+- **Joo, Shi & Abhishek, "Do Sellers Benefit from Sponsored Product Listings? Evidence from an Online Marketplace", *Marketing Science* 43(4) (2024) 817–839** [research] (abstract at `https://ideas.repec.org/a/inm/ormksc/v43y2024i4p817-839.html`, read 2026-11-09; authors from the IDEAS record). "consumers prefer organic listings in the top-ranked positions to sponsored listings of the same product/position. Consumers become indifferent between sponsored and organic listings in the lower-ranked positions." *Transfer:* the value of a sponsored slot depends on where the product already ranks organically; a bid change's effect differs between a term where the product ranks top organically and one where it does not. [seat]
+- **Ju, Zhao & Aral, "Advertising Spillovers in Mobile Apps: Evidence from Ad Shutoffs and Store Rankings", arXiv:2504.16151 (2025)** [research] (read 2026-11-09). A global shutoff "decreased organic installs by 20-30%", through rankings: "paid installs boost app store category rankings, thereby increasing organic visibility." *Transfer:* where paid sales feed a ranking, a pause test's effect on total sales grows over weeks, so a short pause understates the loss through that channel. The setting is app stores, not Amazon. Amazon's own ranking inputs are `areas/ads-organic-and-total.md`'s question. Amazon's own Sponsored Products carry-over estimate (Qin 2023, below) is short, and Golden & Horton (below) found that a non-experimental read overstated a pause's loss. So this paper gives one direction of bias, through one channel, and is not a general rule for Amazon. [seat]
+- **Lewis & Rao, "The Unfavorable Economics of Measuring the Returns to Advertising", *Quarterly Journal of Economics* 130(4) (2015) 1941–1973** [research] (abstract at `https://ideas.repec.org/a/oup/qjecon/v130y2015i4p1941-1973.html`, read 2026-11-09). "Twenty-five large field experiments with major U.S. retailers and brokerages … The median confidence interval on return on investment is over 100 percentage points wide." "individual-level sales are very volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than 10 million person-weeks". "selection bias, due to the targeted nature of advertising, is a crippling concern for widely employed observational methods." *Transfer:* if randomized experiments with millions of people return ROI intervals wider than 100 points, a before/after on one seller's campaign will rarely give a precise ROAS effect; what it can show is large moves in volume metrics (impressions, clicks, spend) and in totals with enough orders. [seat]
+- **Gordon, Zettelmeyer, Bhargava & Chapsky, "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook", *Marketing Science* 38(2) (2019) 193–225** [research] (abstract at `https://ideas.repec.org/a/inm/ormksc/v38y2019i2p193-225.html`, read 2026-11-09; authors from the IDEAS record). "Using data from 15 U.S. advertising experiments at Facebook comprising 500 million user-experiment observations and 1.6 billion ad impressions … The observational methods often fail to produce the same effects as the randomized experiments, even after conditioning on extensive demographic and behavioral variables." *Transfer:* matching, regression and propensity methods on observational ad data are not a substitute for a holdout; a before/after with a control series is weaker still. [seat]
+- **Lewis & Reiley, "Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!", *Quantitative Marketing and Economics* 12(3) (2014) 235–266** [research] (abstract at `https://ideas.repec.org/a/kap/qmktec/v12y2014i3p235-266.html`, read 2026-11-09). "78% of the increase derives from consumers who never click the ads"; the experiment "provides a specification check for observational difference-in-differences and cross-sectional estimators; the latter exhibits a large negative bias three times the estimated experimental effect." *Transfer:* click-attributed sales miss effects on shoppers who saw and did not click (Sponsored Products reports no view-through conversions: "Campaigns that do not measure view-through attribution (e.g., Sponsored Products or Sponsored Brands CPC) will continue to report 0 for conversions from views"); total sales catch them. [seat, with Amazon's quote from the v1 overview]
+- **Johnson, Lewis & Reiley, "When Less Is More: Data and Power in Advertising Experiments", *Marketing Science* 36(1) (2017) 43–53** [research] (abstract at `https://ideas.repec.org/a/inm/ormksc/v36y2017i1p43-53.html`, read 2026-11-09). With control ads that identify who would have been exposed, "Less data give us 31% more precision", whereas covariate adjustment improved precision "by 5%". *Transfer:* precision comes from restricting the comparison to the shoppers or searches the change could affect (for a bid change on one keyword, that keyword's searches and the advertised ASIN's sales), not from adding covariates to an account-wide total. [seat]
+- **Narayanan & Kalyanam, "Position Effects in Search Advertising and their Moderators: A Regression Discontinuity Approach", *Marketing Science* 34(3) (2015) 388–407** [research] (abstract at `https://ideas.repec.org/a/inm/ormksc/v34y2015i3p388-407.html`, read 2026-11-09). "A simple mean comparison of outcomes at two positions is likely to be biased due to these selection issues. Additionally, experimentation is rendered difficult in this situation by competitors' bidding behavior, which induces selection biases that cannot be eliminated by randomizing the bids for the focal advertiser." RD estimates are "much smaller" than naive ones and "exist only in some of the positions". *Transfer:* a bid raise that wins top of search and shows a higher CTR or CVR there does not show that the position caused it; placement-level comparisons in Amazon's placement report (`campaignPlacement`) carry the same selection. [seat]
+- **Basse, Soufiani & Lambert, "Randomization and The Pernicious Effects of Limited Budgets on Auction Experiments", AISTATS 2016 (PMLR 51)** [research] (`https://proceedings.mlr.press/v51/basse16b.html`, read 2026-11-09). "if an A/B experiment affects only bids, then the observed treatment effect is an unbiased estimator when all the bidders in the same auction are randomly assigned to A or B but the observed treatment effect can be severely biased otherwise, even in the absence of throttling. Experiments that affect throttling algorithms can also be badly biased, but the bias can be much reduced if separate budgets are maintained for the A and B arms of the experiment." *Transfer:* two arms drawing on one Sponsored Products campaign budget, or a portfolio budget, interfere through pacing; arms in separate campaigns with separate budgets interfere less, but still meet in the same auctions when they target the same terms. [seat]
+- **Liu, Mao & Kang, "Trustworthy Online Marketplace Experimentation with Budget-split Design", arXiv:2012.08724 (2020); published as "Trustworthy and Powerful Online Marketplace Experimentation with Budget-split Design", KDD 2021, DOI 10.1145/3447548.3467193** [research] (arXiv abstract and Crossref record, read 2026-11-09). "The Bernoulli randomized experiments are biased because treatment units interfere with control units through market competition and violate the "stable unit treatment value assumption"(SUTVA)." The proposed design is "unbiased in any marketplace where buyers have a defined budget". *Transfer:* the platform-side remedy (splitting each budget between arms) is not available to a seller on Amazon. [seat]
+- **Johari, Li, Liskovich & Weintraub, "Experimental Design in Two-Sided Platforms: An Analysis of Bias", arXiv:2002.05670 (2020)** [research] (read 2026-11-09). "an intervention applied to one market participant influences the behavior of another participant. This interference leads to biased estimates". *Transfer:* a competitor's movement is not a clean control for a seller's ad change, because the change moves shoppers between them. [seat]
+- **Waisman, Nair & Carrion, "Online Causal Inference for Advertising in Real-Time Bidding Auctions", arXiv:1908.08600 (2019, rev. 2024)** [research] (read 2026-11-09). "Leveraging the economic structure of first- and second-price auctions, we first show that the effects of advertising are identified by the optimal bids." *Transfer:* limited; a platform-side design. Recorded because it ties ad effect to bid levels, which is the quantity a seller changes. [seat]
+- **Hohnhold, O'Brien & Tang, "Focusing on the Long-term: It's Good for Users and Business", KDD 2015** [research] (`https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43887.pdf`, read 2026-11-09). "the short-term effect is not always predictive of the long-term effect"; on Google search ads, "ads blindness and sightedness are driven by the quality of previously viewed or clicked ads"; "It may take months after a launch for the full effect … to develop." *Transfer:* a change that lifts this week's clicks with poorly matched traffic (for example a broad-match expansion) can lower later CTR or CVR; a two-week window sees only the first part. [seat]
+- **Johari, Pekelis & Walsh, "Always Valid Inference: Bringing Sequential Analysis to A/B Testing", arXiv:1512.04922** [research] (read 2026-11-09). Frequentist p-values and intervals "are wholly unreliable if users endogenously choose samples sizes by *continuously monitoring* their tests." *Transfer:* a cx that reads the campaign every day and calls a result when it looks good inflates false wins; either the window's end is fixed in advance or a sequential method is used. [seat]
+- **Kohavi, Longbotham, Sommerfield & Henne, "Controlled experiments on the web: survey and practical guide", *Data Mining and Knowledge Discovery* 18 (2009)** [research] (metadata and abstract via Semantic Scholar, DOI 10.1007/s10618-008-0114-1, read 2026-11-09; the Springer page answered with a bot challenge). The abstract: "Controlled experiments embody the best scientific design for establishing a causal relationship between changes and their influence on user-observable behavior", with sections on "statistical power, sample size, and techniques for variance reduction". The sample-size rule n = 16σ²/Δ² per variant (95% confidence, 80% power), the "run experiments for at least a week or two, then continuing by multiples of a week" rule, and the "agree on the OEC upfront" rule are quoted from the paper's text in cx-listing-playbook's `method/effect-measurement.md` (read there 2026-10-14); the full text was not re-opened here.
+- **Kohavi, Deng & Vermeer, "A/B Testing Intuition Busters: Common Misunderstandings in Online Controlled Experiments", KDD 2022** [research] (abstract via Semantic Scholar, DOI 10.1145/3534678.3539160, read 2026-11-09; the ACM page answered with a bot challenge). "we have observed some seemingly intuitive concepts being touted, including by A/B tool vendors and agencies, which are misleading, often badly so." Only the abstract was read; its specific misunderstandings are not stated here.
+- **Kohavi, Deng, Longbotham & Xu, "Seven rules of thumb for web site experimenters", KDD 2014; Kohavi, Deng, Frasca, Longbotham, Walker & Xu, "Trustworthy online controlled experiments: five puzzling outcomes explained", KDD 2012** [research] (metadata via Semantic Scholar, read 2026-11-09). Their findings on small true effects, Twyman's law and novelty are recorded, with quotes, in cx-listing-playbook's `method/effect-measurement.md`; not re-read here.
+- **Brodersen, Gallusser, Koehler, Remy & Scott, "Inferring causal impact using Bayesian structural time-series models", arXiv:1506.00356; *Annals of Applied Statistics* 9(1) (2015)** [research] (read 2026-11-09). Builds "a synthetic control that would have occurred had no intervention taken place", accommodating "local trends, seasonality and the time-varying influence of contemporaneous covariates", and illustrated "by estimating the causal effect of an online advertising campaign on search-related site visits". *Transfer:* the method needs control series the change did not touch; on Amazon these are, for example, the same product's sales in another marketplace, the brand's other products, the category's benchmark CPC, or a search query's volume. [seat]
+- **Deng, Knoblich & Lu, "Applying the Delta method in metric analytics: A practical guide with novel ideas", arXiv:1803.06336 (2018)** [research] (read 2026-11-09). On metrics whose "data generating processes and aggregation levels, pose numerous challenges for constructing trustworthy estimation and inference procedures". *Transfer:* CVR (orders over clicks), CPC (cost over clicks) and ACOS (cost over sales) are ratios of two random totals; their variance is not the binomial variance of a proportion of independent trials, and ACOS's variance is dominated by the few orders in its denominator. [seat]
+- **Brown, Cai & DasGupta, "Interval Estimation for a Binomial Proportion", *Statistical Science* 16(2) (2001)** [research] (metadata via Semantic Scholar, DOI 10.1214/ss/1009213286, read 2026-11-09; abstract not served). The standard reference for the poor coverage of the normal-approximation ("Wald") interval at small n. The Wilson score interval "doesn't suffer from problems of overshoot and zero-width intervals that afflict the normal interval"; the Jeffreys interval "is the Bayesian credible interval obtained when using the non-informative Jeffreys prior" with "coverage properties that are similar to those of the Wilson interval". The "rule of three": "if a certain event did not occur in a sample with n subjects, the interval from 0 to 3/n is a 95% confidence interval", and "When n is greater than 30, this is a good approximation of results from more sensitive tests" (so at n = 20 it is rougher). [reference text: Wikipedia, "Binomial proportion confidence interval" and "Rule of three (statistics)", read 2026-11-09, citing Brown et al. 2001, Wilson 1927, Newcombe 1998]
+- **Barnett, van der Pols & Dobson, "Regression to the mean: what it is and how to deal with it", *International Journal of Epidemiology* (2005; online 2004)** [research] (metadata via Semantic Scholar, DOI 10.1093/ije/dyh299, read 2026-11-09). A keyword or campaign picked for a change because its ACOS spiked last week will tend to look better next week with no change; its use in cx-listing-playbook's effect-measurement (P-16) carries over. [seat]
+- **Dash, Ghosh, Mukherjee, Chakraborty & Gummadi, "Sponsored is the New Organic: Implications of Sponsored Results on Quality of Search Results in the Amazon Marketplace", arXiv:2407.19099 (2024)** [research] (read 2026-11-09). In 4,800 simulated searches across four Amazon marketplaces, "items with poor organic ranks (beyond 100th position) appear as sponsored results even before the top organic results on the first page". *Transfer:* a bid raise can place a product on page one for a term where it has no organic presence, so the change's effect on exposure can be large while its effect on conversion is a different question. [seat]
+
+- **Lewis, Zettelmeyer, Gordon, Garib, Hermle, Perry, Romero & Schnaidt, "Amazon Ads Multi-Touch Attribution", arXiv:2508.08209 (2025)** [research, Amazon-authored] (abstract and HTML full text, read 2026-11-09). "Amazon MTA uses a combination of randomized controlled trials (RCTs) and machine learning (ML) models to allocate credit"; "ML models trained purely on observational data are easy to scale and can yield precise predictions, but the models might produce biased estimates of ad effects. RCTs yield unbiased ad effects but can be noisy." The RCT database spans "hundreds of thousands of RCTs across our ad products"; Sponsored Products is not named in the text, and the paper does not describe advertiser access to the RCTs. *Transfer:* Amazon holds shopper-level holdouts for its ad products and uses them to calibrate credit; a seller sees the calibrated credit (multi-touch metrics, §E), not the experiment. [seat]
+
+### Gap fill 2026-11-09: research added
+
+**Amazon-authored measurement work**
+
+- **Qin, "Lengthen your attribution window: Which digital ads have most long-term impact?", *Applied Marketing Analytics* 9(1) (2023) 73–83** [research, Amazon-authored: Vivian Qin, Senior Data Scientist, Amazon Ads] (PDF at `https://cdn.amazon.science/79/6e/915822bb4663a28c97c869a75931/lengthen-your-attribution-window-which-digital-ads-have-most-long-term-impact.pdf`, text extracted and read 2026-11-09; publication page on amazon.science read the same day).
+  - *Data and method.* "15 US brands that asked Amazon Ads to quantify ad carry-over effects"; weekly data "from January 2021 to June 2022"; KPIs detail page views and revenue; a VARMAX model with impulse responses, "the total effect on KPI is calculated up to seven weeks after the occurring shock". Effects in the appendix table are "statistically significant at the p < 0.10 level". Sponsored Products appears in the brand table for two brands.
+  - *Findings.* "within the two-week attribution window, upper / middle-funnel ad products only materialise 30–50 per cent of the total effects, compared to lower-funnel at 60–90 per cent". "SP does not show significant carry-over effects beyond three weeks for any brand." "some ad products, such as SPs, see all their effects dissipate after three weeks." DSP display had its largest revenue share "in week 5 and beyond"; Sponsored Display had "∼30 per cent" after five weeks. Carry-over beyond two weeks varies by vertical: "the smallest for CPG (30 per cent) and the largest for Softlines (70 per cent)", for DPV.
+  - *Limits the author states.* The findings "are based only on brands that asked for a long-term effects assessment"; the impulse response "represents the net effect of all underlying factors, which include consumer behaviour, but also company feedback effects and competitive response."
+  - *Transfer.* For a Sponsored Products change, Amazon's own estimate puts the direct effect inside three weeks, with a material share in weeks 2–3 for the two brands with a Sponsored Products row. That bounds a lag window for direct effects. It says nothing about a rank-mediated organic effect, which the paper's revenue KPI would include only in aggregate. [seat]
+- **Pauwels, Peran, Shah, Schnaidt & Vercamer, "Sponsored brands video rings up clicks and sales in the short and long run", *Applied Marketing Analytics* (2023)** [research, Amazon-authored] (PDF at `https://cdn.amazon.science/d1/66/f222c8524c4d87fc1b0b342345b7/sponsored-brands-video-rings-up-clicks-and-sales-in-the-short-and-long-run.pdf`, read 2026-11-09). The one-month estimate uses "a two-stage Gaussian process — our version of causal multi-task Gaussian processes" on "a matched sample of 25,364 advertisers in North America and Europe". It finds "an average 21.3% increase in sales the following month" for first-time SBv adopters. The one-year estimate uses "propensity score stratification": SBv adopters had "1.25 times higher CTR and 1.1 times higher year-over-year (YoY) sales growth". *Transfer:* these are estimates from advertisers' own adoption choices, not randomized. They show Amazon's researchers using matching on observational data where they did not randomize, which is the kind of method Gordon et al. (2019) found unreliable against RCTs. [seat]
+- **Jeunen, Murphy & Allison, "Learning to Bid with AuctionGym" (AdKDD 2022; KDD 2023 as "Off-Policy Learning-to-Bid with AuctionGym")** [research, Amazon-authored] (PDF at `https://cdn.amazon.science/4f/3c/7f9a5c6c4181894d5e64c684c0d7/learning-to-bid-with-auctiongym.pdf`, text extracted and read 2026-11-09). "an advertiser does not observe counterfactual outcomes of bid amounts that were not submitted, and successful advertisers will adapt their own strategies based on bids placed by competitors. These characteristics complicate effective learning and evaluation of bidding strategies based on logged data alone." Its simulator models unobserved context: "advertisers do not fully observe all the contextual information that influences user behaviour. This confounding effect is simulated by obfuscating the true contextual vector". *Transfer:* a seller's history of bids and outcomes was produced by the seller's own earlier choices. Fitting "clicks as a function of bid" to that history confounds the bid with whatever drove the choice. [seat; `areas/bids-and-placements.md` holds the same reading]
+- **Chen (Outreach Inc.), Nabi & Siniscalchi (Amazon Ads), "Advancing Ad Auction Realism: Practical Insights & Modeling Implications", arXiv 2307.11732 (AdKDD 2023)** [research, Amazon-affiliated] (abstract read 2026-11-09). Real auctions differ from textbook ones because "advertisers only receive partial, aggregated feedback, and (4) payment rules are only partially known to bidders". In simulation, "soft floors can improve revenues relative to standard auction formats" when values are query-dependent. *Transfer:* the seller's reports are the "partial, aggregated feedback". A change's measured CPC effect mixes the seller's own bid with a pricing rule the seller does not fully see (§D). [seat]
+- **Han, Castells, Gupta, Xu & Salaka, "Addressing Cold Start in Product Search via Empirical Bayes", CIKM 2022** [research, Amazon-authored] (amazon.science publication page, read 2026-11-09). "An online A/B test on 50 million queries shows a significant improvement in new product impressions by 13.53% and a significant increase in new product purchase by 11.14%, with overall purchases up by 0.08%" (figures for human readers only). **Dong, Momma, Wang, Gao & Sun, "Behavioral Feature Boosting via Substitute Relationships for E-commerce Search", arXiv 2602.14502 (2026)** [research, Amazon-authored] (abstract read 2026-11-09): new items get a warm start from their substitutes' "clicks, add-to-carts, purchases, and ratings"; "The BFS-enhanced ranking model has been launched in production and has served customers since 2025." *Transfer:* a new ASIN's organic exposure, which an ad change's total-sales verdict sits on, depends on a ranking that Amazon tests and changes. It also depends on how the ASIN's substitutes behave. Both move without any change by the seller. [seat; `areas/launch.md` P-29 and P-30 hold the detail]
+- **Cooprider & Nassiri (Amazon), "The science of price experiments in the Amazon Store", amazon.science blog, 2023-04-14** [research, Amazon-authored; Amazon's own account of its method] (`https://www.amazon.science/blog/the-science-of-price-experiments-in-the-amazon-store`, read 2026-11-09). Because all customers must see the same price, Amazon cannot split shoppers for a price test. It uses time-bound, triggered (staggered by an external event) and switchback designs. "The most straightforward switchback experiment is the random-days experiments, in which, each day, each product is randomly assigned to either the control group or the treatment group. Our analyses indicate that random days can reduce the standard error of our experimental results … by 60%." It names the risk: "One of the drawbacks with any switchback experiment, however, is the risk of carryover … if treatment increases a product's sales, recommendation algorithms may recommend that product more often. That could artificially boost the product's sales even during control periods." *Transfer:* a seller's ad change faces the same constraint (no shopper split) and the same carryover path (sales feeding recommendations and ranking). Amazon's own pricing teams answer it with randomized days across many products. [seat]
+
+**Time-alternating (switchback) designs**
+
+- **Bojinov, Simchi-Levi & Zhao, "Design and Analysis of Switchback Experiments", *Management Science* 69(7) (2023) 3759–3777, DOI 10.1287/mnsc.2022.4583** [research] (arXiv 2009.00148 abstract and full text, read 2026-11-09; journal record via Crossref).
+  - *Setting.* "a firm sequentially exposes an experimental unit to random treatments". The carryover effect is "the length of time a treatment persists in impacting the outcome", of order *m* periods.
+  - *Design.* Under the paper's assumptions, "any optimal design of experiment (T,Q) must satisfy q0 = q1 = ... = qK = 1/2" (fair coin flips at the randomization points). When *T* = *n·m* (n ≥ 4), the optimal randomization points are "T∗ = {1, 2m+1, 3m+1, ..., (n−2)m+1}". So the first and last blocks are 2*m* long and the rest *m*. The worked example is *T* = 12 and *m* = 2, with points {1, 5, 7, 9}.
+  - *Estimation.* A Horvitz-Thompson estimator compares periods that follow *p*+1 consecutive treatments with periods that follow *p*+1 consecutive controls, weighted by their known assignment probabilities. It is unbiased for "the average lag-p causal effect" when *p* ≥ *m*. "when p<m, the Horvitz-Thompson estimator (4) will be biased for the causal estimand", although exact randomization-based p-values stay valid.
+  - *Practical guidance, verbatim.* "we suggest that each period's length be smaller than the carryover effect duration"; "the firm should set p = m"; choosing *n* "resembles a typical power analysis". Identifying *m* from data needs "experimental resources at the scale of T/m>100 to distinguish two candidate values".
+  - *Transfer.* A seller's daily or weekly on/off test of a campaign is a switchback with one unit. Three things decide whether it estimates the effect: whether the switch times are randomized, whether the period is long enough against the carryover, and whether the analysis drops or models the periods right after a switch. Hershaff's daily alternation (What practitioners add) met a measured daily decay of about 0.35, so a one-day carryover is not negligible. Qin's Sponsored Products carry-over reached up to three weeks in weekly data. Those two figures differ by more than an order of magnitude, measured on different outcomes and data. [seat]
+
+**Difference-in-differences and staggered changes**
+
+- **Goodman-Bacon, "Difference-in-differences with variation in treatment timing", *Journal of Econometrics* 225(2) (2021) 254–277** [research] (abstract via IDEAS, read 2026-11-09). "Most DD applications, however, exploit variation across groups of units that receive treatment at different times. This paper shows that the two-way fixed effects estimator equals a weighted average of all possible two-group/two-period DD estimators in the data. A causal interpretation of two-way fixed effects DD estimates requires both a parallel trends assumption and treatment effects that are constant over time."
+- **de Chaisemartin & D'Haultfœuille, "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects", *American Economic Review* 110(9) (2020) 2964–96** [research] (abstract via IDEAS, read 2026-11-09). Such regressions "estimate weighted sums of the average treatment effects (ATE) in each group and period, with weights that may be negative. Due to the negative weights, the linear regression coefficient may for instance be negative while all the ATEs are positive."
+- **Callaway & Sant'Anna, "Difference-in-Differences with multiple time periods", *Journal of Econometrics* 225(2) (2021) 200–230** [research] (abstract via IDEAS, read 2026-11-09). It identifies group-time effects "with (i) multiple time periods, (ii) variation in treatment timing, and (iii) when the "parallel trends assumption" holds potentially only after conditioning on observed covariates". It proposes aggregations "to highlight treatment effect heterogeneity" and a bootstrap for "simultaneous (instead of pointwise) inference".
+- **Sun & Abraham, "Estimating dynamic treatment effects in event studies with heterogeneous treatment effects", *Journal of Econometrics* 225(2) (2021) 175–199** [research] (abstract via IDEAS, read 2026-11-09). "in settings with variation in treatment timing across units, the coefficient on a given lead or lag can be contaminated by effects from other periods, and apparent pretrends can arise solely from treatment effects heterogeneity."
+- *Transfer.* cx changes many targets and campaigns on different dates. Pooling their before/after differences in one regression with target and date effects is the two-way fixed-effects estimator these papers analyse. When effects differ across targets or grow over time, the pooled coefficient can carry negative weights and even the wrong sign, and a pre-trend test on such a regression can mislead. The papers' remedies compare each changed unit only with units not yet changed, or never changed, at the same dates, then aggregate. Parallel trends is an assumption about the untreated paths, checked, not proved, on the pre-period. [seat]
+
+**Synthetic control**
+
+- **Abadie, Diamond & Hainmueller, "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program", *JASA* 105(490) (2010) 493–505** [research] (NBER Working Paper 12831 abstract, read 2026-11-09). The method builds "a comparable synthetic control region" as a weighted combination of untreated units. It is meant for interventions that "affect a small number of aggregate units", and "produce[s] informative inference regardless of the number of available comparison units".
+- **Abadie, "Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects", *Journal of Economic Literature* 59(2) (2021) 391–425** [research] (NBER Summer Institute copy of the published article, `https://conference.nber.org/confer/2021/SI2021/Abadie_2021.pdf`, text extracted and read 2026-11-09). Its feasibility conditions, by heading and verbatim where quoted:
+  - *Size of the effect and volatility of the outcome.* "small effects will be indistinguishable from other shocks to the outcome of the affected unit, especially if the outcome variable of interest is highly volatile."
+  - *Availability of a comparison group.* "it is important that not all units adopt interventions similar to the one under investigation during the period of the study".
+  - *No anticipation*, *no interference* ("units' outcomes are invariant to other units' treatments").
+  - *Convex hull condition.* "it should be checked that the differences in the characteristics of the affected unit and the synthetic control are small".
+  - *Time horizon.* Use "surrogate outcomes or leading indicators" when effects take time.
+  - On donor pools: "each of the units in the donor pool have to be chosen judiciously". Including unsuitable units risks "large interpolation biases".
+  - Inference is by placebo runs: "iteratively reassigning the treatment to the units in the donor pool and estimating "placebo effects"". Abadie, Diamond & Hainmueller's ratio of post- to pre-intervention RMSPE serves as the test statistic.
+- **Arkhangelsky, Athey, Hirshberg, Imbens & Wager, "Synthetic Difference-in-Differences", *American Economic Review* 111(12) (2021) 4088–4118** [research] (abstract via IDEAS, read 2026-11-09). "a new estimator for causal effects with panel data that builds on insights behind the widely used difference-in-differences and synthetic control methods" with "desirable robustness properties".
+- *Transfer.* The control series in S5 (the brand's unadvertised ASINs, the same ASIN in another marketplace, the category benchmark CPC, untouched campaigns) are a donor pool. Abadie's conditions map onto ad work. The pre-period fit must be close. The donors must not have had a similar change in the window, which is Abadie's comparison-group condition applied to cx's other changes. The donors must not be moved by the change: a competitor's ASIN fails this (Johari et al.), and a sibling ASIN that shares the shopper may fail too. A single ASIN's daily orders are often volatile against the effect sought. [seat]
+
+**Variance reduction and trustworthy-experiment checks**
+
+- **Deng, Xu, Kohavi & Walker, "Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-Experiment Data" (CUPED), WSDM 2013** [research] (PDF on Microsoft's ExP site, `https://exp-platform.com/Documents/2013-02-CUPED-ImprovingSensitivityOfControlledExperiments.pdf`, read 2026-11-09). With a pre-period covariate *X* and θ = cov(*Y*,*X*)/var(*X*), "var(Ycv) = var(Y)(1−ρ²)". On Bing "we can reduce variance by about 50%, effectively achieving the same statistical power with only half of the users, or half the duration". "Using the metric measured in the pre-period as the covariate typically provides the best variance reduction." "Using a pre-experiment period of 1-2 weeks works well for variance reduction. Too short a period will lead to poor matching, whereas too long a period will reduce correlation with the outcome metric during the experiment period." "Never use covariates that could be affected by the treatment, as this could bias the results." *Transfer:* CUPED reduces variance in a comparison between randomized arms, for example targets assigned at random to a change against targets left alone. There each target's own pre-period clicks or orders are the covariate. In a single before/after with no control arm it has nothing to adjust. The gain depends on ρ, the correlation of a target's pre- and post-period values. That correlation is low for sparse targets whose weekly orders are mostly 0 or 1. [seat]
+- **Kohavi, Tang & Xu, *Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing*, Cambridge University Press (2020)** [research, book] (bibliographic record via Open Library, read 2026-11-09). The publisher's pages and excerpt PDFs answered HTTP 403 or 503, so the book's chapters were not read, and its content is cited here only through the authors' papers read below.
+- **Dmitriev, Gupta, Kim & Vaz, "A Dirty Dozen: Twelve Common Metric Interpretation Pitfalls in Online Controlled Experiments", KDD 2017** [research] (ExP PDF, `https://exp-platform.com/Documents/2017-08%20KDDMetricInterpretationPitfalls.pdf`, read 2026-11-09). If the ratio of units between arms "is different than the expected (a chi-square test can be used), it means something is wrong with the sampling process. We call this situation Sample Ratio Mismatch (SRM). Most of the time SRM implies a severe selection bias, enough to render the experiment results invalid". The paper also has a section on "Novelty and Primacy Effects".
+- **Kohavi, Deng, Frasca, Longbotham, Walker & Xu, "Trustworthy online controlled experiments: five puzzling outcomes explained", KDD 2012** [research] (ExP PDF, `https://exp-platform.com/Documents/puzzlingOutcomesInControlledExperiments.pdf`, read 2026-11-09). On carryover when units are reused across experiments: "A/A tests can be run to check for carryover effects". "What was surprising to us is the duration of the carryover effect". Their example monitored buckets for "more than three weeks" after a 47-day experiment ended. "We consider a week to be the minimum to look at day-of-week effects." An A/A test "can be used to (i) collect data and assess its variability for power calculations, and (ii) test the experimentation system".
+- *Transfer.* An ad analogue of SRM is a change set whose "treated" targets did not all receive the change, for example partial failures in a `207` reply (§B), or targets the history shows as changed again by someone else. The analysed arm then differs from the assigned one. An analogue of the A/A test is running the verdict procedure on a window with no change. How often it "finds" an effect is its false-alarm rate on that seller's data. [seat]
+
+**Measured interference in marketplaces**
+
+- **Blake & Coey, "Why marketplace experimentation is harder than it seems: the role of test-control interference", ACM EC 2014, pp. 567–582, DOI 10.1145/2600057.2602837** [research] (Crossref and Semantic Scholar records, read 2026-11-09; the ACM abstract is elided in both and the full text was not read). As summarized by Holtz & Aral (arXiv 2004.12162, read 2026-11-09), the paper studies an eBay email-marketing experiment in which test and control users bid in the same auctions. It limits the analysis "to auctions where a majority of bidders are in the treatment or control, and compare[s] outcomes across auctions"; secondary summaries say the naive estimate overstated the effect by about a factor of two. These are secondhand.
+- **Holtz, Lobel, Lobel, Liskovich & Aral, "Reducing Interference Bias in Online Marketplace Experiments Using Cluster Randomization: Evidence from a Pricing Meta-experiment on Airbnb", *Management Science* (online 2024-04-05), DOI 10.1287/mnsc.2020.01157** [research] (Crossref abstract, read 2026-11-09). "at least 20% of the TATE estimate produced by an individual-level randomized evaluation of the platform fee increase we study is attributable to interference bias and eliminated through the use of cluster randomization". There is "suggestive, nonstatistically significant evidence that interference bias in seller-side experiments is more severe in demand-constrained geographies".
+- **Holtz & Aral, "Limiting Bias from Test-Control Interference in Online Marketplace Experiments", arXiv 2004.12162 (2020)** [research] (abstract read 2026-11-09). In an Airbnb-based simulation, "blocked graph cluster randomization reduces the bias of the naive difference-in-means estimator by as much as 62%, it also significantly increases the variance of the estimator".
+- *Transfer.* When the arms of a seller's test compete for the same shoppers, for example two of the seller's own campaigns on the same terms or two of its ASINs that substitute for each other, the measured difference overstates what switching everything would do. Measured marketplace biases run from at least 20% (Airbnb, fees) to about double (eBay, secondhand). Amazon's match-type rule (§F) means some same-keyword arms do not both serve at all. [seat]
+
+**Identifying a target's response to its own bid**
+
+- **Wager & Xu, "Experimenting in Equilibrium", *Management Science* 67(11) (2021) 6694–6715, DOI 10.1287/mnsc.2020.3844** [research] (arXiv 1903.02124 abstract and full text, read 2026-11-09). "Our approach enables us to accurately estimate the effect of small changes to system parameters by combining unobstrusive randomization with lightweight modeling, all while remaining in equilibrium. We can then use these estimates to optimize the system by gradient descent." Each unit gets a small random shock "ζεi", with "εi = ±1 independently and uniformly at random". The regression on that shock "recovers a certain marginal response function, which captures the supplier's sensitivity to payment changes against a fixed ambient market equilibrium". In the platform's problem, the mean-field model is then needed because all suppliers would move together. The paper optimizes "in large systems using vanishingly small perturbations".
+- *Transfer.* The platform in the paper sets one parameter for many suppliers. A single seller is the opposite case: one small bidder whose own bid barely moves the market equilibrium. The paper's direct regression, the marginal response "against a fixed ambient market equilibrium", is then the quantity the seller wants: how clicks, cost and orders on a target respond to a small change in its own bid. Small ± bid perturbations assigned at random across targets or across days identify that local slope without relying on the seller's logged history, which AuctionGym (above) warns is confounded. Under dynamic bidding, Amazon's real-time adjustment sits on top of the perturbed base bid (§D). Effects on clicks and cost will be visible long before effects on orders, by the arithmetic in the worked statistics. [seat]
+
+**Many verdicts at once**
+
+- **Benjamini & Hochberg, "Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing", *JRSS B* 57(1) (1995) 289–300** [research] (Crossref abstract, read 2026-11-09). "It calls for controlling the expected proportion of falsely rejected hypotheses — the false discovery rate." "A simple sequential Bonferronitype procedure is proved to control the false discovery rate for independent test statistics". (Also R12 in `areas/search-terms-and-negatives.md`.)
+- **Gelman, Hill & Yajima, "Why We (Usually) Don't Have to Worry About Multiple Comparisons", *Journal of Research on Educational Effectiveness* 5(2) (2012) 189–211** [research] (arXiv 0907.2478 PDF from Columbia, read 2026-11-09). "Multilevel models perform partial pooling (shifting estimates toward each other), whereas classical procedures typically keep the centers of intervals stationary, adjusting for multiple comparisons by making the intervals wider". The paper also defines "Type S" (sign) and "Type M" (magnitude) errors: a noisier estimator "is much more likely to produce effect estimates that are larger in magnitude".
+- **Lee & Shen, "Winner's Curse: Bias Estimation for Total Effects of Features in Online Controlled Experiments", KDD 2018, DOI 10.1145/3219819.3219905** [research] (abstract via Semantic Scholar, read 2026-11-09). "When a group of experiments is conducted, usually the ones with significant successful results are chosen to be launched into the product. … we investigate a statistical selection bias in this process and propose a correction method of getting an unbiased estimator", implemented at Airbnb.
+- **Deng, "Objective Bayesian Two Sample Hypothesis Testing for Online Controlled Experiments", WWW 2015 workshop** [research] (abstract on Microsoft's ExP site, `https://exp-platform.com/objective-bayesian-ab/`, read 2026-11-09). "this method requires the existence of historical A/B test data to objectively learn a prior. We have successfully applied this method to Bing, using thousands of experiments to establish the priors."
+- **Azevedo, Deng, Montiel Olea, Rao & Weyl, "A/B Testing with Fat Tails", *Journal of Political Economy* 128(12) (2020)** [research] (abstract via IDEAS, read 2026-11-09). "the optimal strategy depends on whether most gains accrue from typical innovations or from rare, unpredictable large successes". With fat tails, "a "lean" strategy of trying more ideas, each with possibly smaller sample sizes, is preferred", from an analysis of "Microsoft Bing's EXP platform".
+- *Transfer.* A heartbeat that judges dozens of changes a week runs dozens of tests. At a 5% per-test threshold with no real effects, about one in twenty *held* verdicts is a false win. The changes kept because they "won" have effects overstated by selection (Lee & Shen). Noisy estimates exaggerate magnitude (Gelman's Type M). The published remedies are false-discovery control across the batch, and shrinkage of each change's estimate toward the average effect of similar past changes. Deng's and Azevedo et al.'s empirical priors are built from that seller's own history of verdicts. Azevedo et al. add that, when big wins are rare and large, many small tests can beat a few large ones. [seat]
+
+**Brand-term pauses: the two directions of bias**
+
+- **Golden & Horton, "The Effects of Search Advertising on Competitors: An Experiment Before a Merger", *Management Science* 67(1) (2021) 342–362** [research] (abstract via IDEAS, read 2026-11-09). "Firm Vary temporarily suspended its sponsored search advertising campaign on Google in randomly selected advertising markets in the United States. By shutting off its ads, Firm Vary lost customers, but only 63% as many as a nonexperimental estimate would have suggested." Spillovers to its rival "were surprisingly small, even in the market for Firm Vary's brand name as a keyword search term".
+- *Transfer.* The abstract does not name the non-experimental method. Its finding is that a non-randomized read of a pause overstated the loss by about 1/0.63 ≈ 1.6 times [seat arithmetic]. Ju, Zhao & Aral point the other way for a ranking channel (a short pause understates a loss that builds through rank). Coviello et al. found large losses at Edmunds, Blake et al. almost none at eBay. Which direction dominates on Amazon is not measured on the pages read. `beyond/brand-defense-and-competition.md` cites this file for "why a before/after read of a pause overstates the loss". The supported statement is Golden & Horton's: a non-experimental estimate overstated it in one randomized geo test. [seat]
+
+**What a holdout identifies**
+
+- **Johnson, "Inferno: A guide to field experiments in online display advertising", *Journal of Economics & Management Strategy* 32(3) (2023) 469–490, DOI 10.1111/jems.12513** [research] (Crossref abstract, read 2026-11-09; SSRN full text not read). "Display-ad effects are tiny and necessitate large-scale experiments. The experimenter has limited control because ad exposure is jointly determined by advertisers, users, algorithms, and market competition." The guide covers "statistical power and compliance as well as … identity fragmentation, experimental spillovers, and incrementality optimization".
+- **Johnson, Lewis & Nubbemeyer, "Ghost Ads: Improving the Economics of Measuring Ad Effectiveness" (2016 working paper; *Journal of Marketing Research* 54(6), 2017)** [research] (NBER conference PDF, `https://conference.nber.org/conf_papers/f84173.pdf`, read 2026-11-09; the earlier pass found only an SSRN block). "ITT is a valid experimental approach whose estimator differences the outcomes of the entire treatment group and the entire control group." "the ITT estimator is less precise than the TOT", because the experimental difference among the untreated "should be zero, but the empirical difference contributes noise". On PSA controls: "PSA experiments are rendered invalid when marketers use performance-optimizing computer algorithms to deliver ads", because the platform sends different users to the treatment ad and to the PSA.
+- *Transfer.* For a seller's change, the "assignment" is the change sent (a bid raised on a target) and "exposure" is what it produced (impressions won at the new bid). The intent-to-treat comparison is all assigned targets against all controls, including targets whose raise won nothing. Comparing only the targets that won more impressions conditions on an outcome of the change, as a PSA comparison under an optimizing platform does. Under Amazon's dynamic bidding and relevance-based allocation, exposure is chosen by Amazon's algorithms. [seat]
+
+**Weighing how long a Sponsored Products effect lasts** [seat, from the sources above]
+
+| Source | What was measured | Data | Duration finding |
+|---|---|---|---|
+| Qin 2023 (Amazon) | Carry-over of SP spend on brand revenue and DPV | 15 US brands that asked for the study (SP rows for two), weekly, 2021–22, VARMAX | No significant SP carry-over beyond 3 weeks; in the two SP rows, 31–63% of the DPV effect after week 1 |
+| Lily Tong (Amazon), via press | Share of brand sales value after the attribution window | "more than 2 million campaigns" in June, all Amazon ad products | "roughly 20%" after the window |
+| Hershaff 2026 | Daily adstock of all ads on total sales | One seller, about one month, Meridian MMM | Decay about 0.35 per day |
+| Ju, Zhao & Aral 2025 | Organic installs after a global ad shutoff | One mobile game, app-store rankings | Loss grows through rank over weeks |
+| Hohnhold et al. 2015 | Users' learning about ad quality | Google search | Months |
+
+The Amazon-side figures agree that Sponsored Products' direct effect is short: days to three weeks. It is not all in the first week. `beyond/metrics-and-attribution.md` read Qin's appendix table: in the two Sponsored Products rows, week 1 holds 37% and 69% of the DPV effect, and week 3 holds 36% and 9%. So a 7-day read misses part of a Sponsored Products effect even on Amazon's own estimate (that file's reading; this seat confirmed the table lists Sponsored Products for Brand A and Brand D only). Lily Tong's 20% is for all ad products and is not split out for Sponsored Products. The long-horizon results are measured off Amazon, through channels (store ranking, ad blindness) that no Amazon page read quantifies for Sponsored Products. A lag window of two to three weeks after a Sponsored Products change matches Amazon's own direct-effect evidence. Whether a rank-mediated effect adds to it on Amazon is open (Open question 15). `areas/ads-organic-and-total.md` and `beyond/metrics-and-attribution.md` P-30 and P-32 hold the same sources; this file owns the window question.
+
+### Worked statistics for small ad samples [seat; arithmetic by the seat on 2026-11-09, using the formulas above]
+
+These are illustrations of the size of chance in ad data, not thresholds.
+
+- **A conversion rate from 100 clicks.** 10 orders from 100 clicks is a CVR of 10%; its Wilson 95% interval is about 5.5% to 17.4% (Jeffreys: 5.3% to 17.0%). 3 orders from 30 clicks: 3.5% to 25.6%. 0 orders from 20 clicks: 0% to 16.1% (Wilson), 0% to 11.7% (Jeffreys), 15% by the rule of three. 40 from 400: 7.4% to 13.3%. 100 from 1,000: 8.3% to 12.0%.
+- **Telling 10% from 12%.** With the n = 16σ²/Δ² rule (σ² = p(1−p)), detecting a CVR move from 10% to 12% at 95% confidence and 80% power needs about 3,600 clicks per period; 10% to 11% needs about 14,400; 5% to 6% about 7,600. A keyword with 30 clicks a week would need years. Only large moves on high-volume terms, or totals pooled across many terms, can be told from chance in a few weeks.
+- **A Bayesian reading.** With uniform Beta(1,1) priors, 12/100 before and 18/100 after gives a posterior probability of about 0.88 that the after-rate is higher; 30/300 against 40/300 gives about 0.90. Both are "probably higher", neither is near certainty, and neither says by how much. A probability that one period beat the other is not the probability that the change caused it, because the periods differ in more than the change.
+- **ACOS from few orders.** If orders are roughly Poisson, the relative standard error of an order count k is about 1/√k: about 32% at 10 orders, 22% at 20, 10% at 100. ACOS (cost over sales) inherits at least that relative error from its denominator, plus the spread of order values. An ACOS that moves from 30% to 24% on 10 orders a week is inside chance.
+- **Zero conversions.** A keyword with 20 clicks and no order has a 95% upper bound on its CVR of about 11–16% by the intervals above; Amazon's targeting guide recommends judging a negative after "at least 20 clicks" (§G). The guide's 20 clicks rules out only very high conversion rates. [seat]
+- **Clicks are not independent trials.** One shopper can click several times; clicks cluster by day and by query. The binomial intervals above are therefore too narrow, not too wide (Deng, Knoblich & Lu 2018 on aggregation levels).
+
+## Stable enough for a skill
+
+Each item is stable in kind; any figure in it is read from its page at run time.
+
+1. Amazon's change history (`POST /history`) covers Sponsored Products and Sponsored Brands only, at most 90 days back, and does not say who made a change; events carry `previousValue`, `newValue`, `timestamp` (UTC epoch) and `metadata`. (Changehistory_prod_3p.json)
+2. The history model's change types and values are narrower than, and partly named differently from, the campaign models: three bidding-strategy values against SP 3.0's four plus `OTHER`, two placement positions against four, and no event types for negative product targets, audience or shopper bid adjustments, ad creative or budget-rule associations. (Changehistory_prod_3p.json; SponsoredProducts_prod_3p.json)
+3. A v1 write answers `207` with per-index `success`, `partialSuccess` and `error`, and returns the object with its `lastUpdatedDateTime`; SP 3.0 objects carry `extendedData.lastUpdateDateTime` and `servingStatus`. (AmazonAdsAPIALLMerged_prod_3p.json; SponsoredProducts_prod_3p.json)
+4. Ad conversions are dated on the day of the ad interaction, inside a look-back that depends on ad product and account type (Sponsored Products: 7 days for sellers, 14 for vendors; others 14). (Overview of the Reporting v1 API; Reporting columns (version 3))
+5. The newest days of an ad report are provisional: clicks are validated after the fact and conversions restated afterwards. Amazon's pages give different horizons for when a day is final, and the longest stated is Stream v2's restatement window. (Sponsored ads reporting FAQ (version 3); Amazon Ads reporting overview; Stream v2 datasets overview; Ads performance [Stream v2])
+6. Report days follow each campaign's time zone; the history's timestamps are UTC. (Dates and historical data retention; Changehistory_prod_3p.json)
+7. Ratios (ROAS, ACOS, CTR, CVR, CPC) are computed after summing numerator and denominator at the chosen grain. (Ads performance [Stream v2])
+8. Base v1 conversion metrics include halo sales; promoted-only figures carry the Promoted qualifier; SP and SB CPC campaigns report no view-through conversions. (Overview of the Reporting v1 API)
+9. With dynamic bidding on, Amazon moves the bid on each auction within a ceiling its pages state; the current help page, library guide and models say up to 100% for all placements under up and down, while the author's guide still says 50% outside top of search. With a fixed bid it does not. Rule-based bidding moves base bids and can switch itself off. Budget rules raise budgets on dates or on performance. (SP 3.0 OpenAPI; Bidding strategies for Sponsored Products; Guide to dynamic bidding - up and down; The author's guide to Sponsored Products; Sponsored Products rule-based bidding overview; Sponsored ads budget rules overview)
+10. A Sponsored Products ad stops serving when the advertised product is out of stock or not the Featured Offer, and the product ad's serving status names the reason. (Best practices for your Sponsored Products ads; SP 3.0 OpenAPI; v1 merged OpenAPI)
+11. Amazon can change where and how ads serve without a seller write: prompts auto-enrolment, off-Amazon serving, headline optimisation by consent, automatic collections, recommendations applied in one click (announced as rolling out), and incidents listed on Amazon Ads Status. (Sponsored Products prompts and Sponsored Brands prompts; All releases; Amazon Ads Agent announcement; Recommendations API overview; Amazon Ads Status)
+12. Amazon offers no randomized test of a Sponsored Products bid, budget, target or negative to sellers on the pages read; MYE tests listing content only; Brand Lift is a survey that excludes Sponsored Products; Creative Testing is a DSP pre-launch survey; third-party studies run in DSP; multi-touch attribution divides credit rather than testing a change; Sponsored Brands lets several ads share an ad group without a documented split. (Test listing content with Manage Your Experiments; Brand Lift: Study the impact of your advertising; Identify the right creative … with Creative Testing; Self-Service Third Party Measurement Studies; Uncover ad value across the funnel with multi-touch attribution (beta); Sponsored Brands v4 features)
+13. Sponsored Products and Sponsored Brands targets carry no geography, daypart, device or audience in the v1 model, so a seller cannot randomize those ads by region or time through targeting. (Target entity guide)
+14. Attributed ad sales are not incremental sales; randomized on-off tests on search ads found brand-term ads largely substituted by organic clicks, and observational methods overstate ad returns. (Blake, Nosko & Tadelis 2015; Simonov, Nosko & Rao 2018; Gordon et al. 2019; Lewis & Rao 2015)
+15. Arms that share a budget or an auction interfere; comparisons of positions won by bidding carry selection. (Basse, Soufiani & Lambert 2016; Liu, Mao & Kang 2020; Narayanan & Kalyanam 2015)
+16. Reading the series daily and stopping when it looks good invalidates ordinary p-values and intervals. (Johari, Pekelis & Walsh 2015)
+17. Short windows can miss effects that build over weeks or months through learning or ranking; Amazon's own carry-over estimate for Sponsored Products finds no significant direct effect beyond three weeks. (Hohnhold, O'Brien & Tang 2015; Ju, Zhao & Aral 2025; Qin 2023)
+18. At ad-sample sizes, proportions need Wilson or Jeffreys intervals rather than the normal approximation, zero counts need an upper bound (rule of three), and ratio metrics need the delta method. (Brown, Cai & DasGupta 2001; Deng, Knoblich & Lu 2018)
+19. A term or campaign picked because last week was bad will tend to look better next week without any change. (Barnett, van der Pols & Dobson 2005)
+20. The three verdicts and their conditions are those of cx-listing-playbook's `effect-review` (effect-measurement P-18): *held*, *not held*, *cannot tell*. (cx-listing-playbook `research/method/effect-measurement.md`)
+21. Invalid traffic is monitored for up to 30 days and removed from reports in that window; an investigation can restate metrics after it. (Traffic validation)
+22. Reserve prices, some set in real time and valued differently at peak, move the price a winning ad pays with no change to its bid; Amazon states it continually experiments with the auction's inputs and formats. (Understand Amazon Ads auctions; Amazon's FTC response)
+23. For one keyword set in several match types with different bids, the highest-bid match type is used for a matching query; no page read extends the rule across campaigns or to copies in the same match type. (Understand keyword match types; A guide to targeting with Sponsored Products)
+24. Amazon DSP offers 10 customer holdout groups of 10% each for A/B tests and advises against before/after comparison of bid adjustments; no sponsored-ads holdout is documented. (How to test DSP bid adjustments; Ads API docs search index)
+25. A switchback is unbiased for the lag-*m* effect when its randomization points and estimator respect the carryover length *m*, and biased when they do not. (Bojinov, Simchi-Levi & Zhao 2023)
+26. With staggered changes and effects that differ, a pooled two-way fixed-effects coefficient can weight some effects negatively and take the wrong sign. (Goodman-Bacon 2021; de Chaisemartin & D'Haultfœuille 2020; Callaway & Sant'Anna 2021; Sun & Abraham 2021)
+27. A synthetic or donor control needs close pre-period fit, donors untouched by similar changes, and no spillover from the treated unit. (Abadie 2021)
+28. A pre-period covariate reduces variance by the factor 1−ρ²; covariates affected by the treatment bias the result. (Deng et al. 2013)
+29. Selecting winners from many tests overstates their effects; false-discovery control and partial pooling are the published corrections. (Benjamini & Hochberg 1995; Gelman, Hill & Yajima 2012; Lee & Shen 2018)
+30. Comparing only exposed units, when an optimizing platform chooses exposure, conditions on the platform's choice; the intent-to-treat contrast over all assigned units stays valid. (Johnson, Lewis & Nubbemeyer 2016)
+
+## What cx must read at run time
+
+**Amazon Ads API, through `cx raw amazon-ads` (cx-amazon-ads door), at the profile's regional origin, with `Amazon-Advertising-API-Scope`:**
+- `POST /history`, with `Accept: application/vnd.historyresponse.v1.1+json` when `THEME` is asked for; body `eventTypes` (per entity: `eventTypeIds` ≤ 10, `filters`, `parents` ≤ 10), `fromDate`, `toDate`, `count`, `sort`, `nextToken` or `pageOffset`. The 90-day bound and the count limits are read from the model, not built in. (Changehistory_prod_3p.json)
+- The object before and after a write: v1 `POST /adsApi/v1/query/campaigns`, `/adGroups`, `/ads`, `/targets` (`lastUpdatedDateTime`, `state`, `status.deliveryStatus`, `status.deliveryReasons`, `budgets`, `optimizations.bidSettings`, `tags`); or SP 3.0 `POST /sp/campaigns/list` (`dynamicBidding.strategy`, `dynamicBidding.placementBidding[]`, `budget`, `extendedData.lastUpdateDateTime`, `extendedData.servingStatus`), `/sp/adGroups/list`, `/sp/keywords/list`, `/sp/targets/list`, `/sp/negativeKeywords/list`, `/sp/negativeTargets/list`, `/sp/campaignNegativeKeywords/list`, `/sp/productAds/list`. Sponsored Brands: `POST /sb/v4/campaigns/list` (the door's example).
+- The write itself and its reply: `POST /adsApi/v1/update/campaigns` (and `/adGroups`, `/ads`, `/targets`), whose `207` body is kept whole; `create` and `delete` likewise.
+- Budget state in the window: `POST /sp/campaigns/budget/usage` (`budgetUsagePercent`, `usageUpdatedTimestamp`); budget rules on a campaign `GET /sp/campaigns/{campaignId}/budgetRules`; rule-based bidding paths under `/sp/rules/campaignOptimization` (as listed in the SP 3.0 model).
+- Recommendations and their state: `POST /adsApi/v1/query/recommendations` (`QueryRecommendation`), or the list read, which the Recommendations API overview names `GET /adsApi/v1/list/recommendations` and the v1 merged OpenAPI names `GET /adsApi/v1/recommendations` (`ListRecommendation`); both read 2026-11-09, so the path is read from the model at run time (`state`, `potentialImpacts`).
+- Multi-touch metrics (beta, US): Orders, Sales and ROAS "(multi-touch)" beside last-touch in the console and downloadable reports per Amazon's launch page. In the API they are Reporting v1 fields, `metric.multiTouchPurchases`, `metric.multiTouchSales`, `metric.multiTouchRoas` and their `Promoted`/`FromClicks`/`FromViews` variants (the v1 *Purchases* page, whose fields give no v3 name); the v3 columns page has none. Whether a seller's rows hold values is read from the report.
+- Product eligibility: `POST /eligibility/product/list` (Eligibility 3.0 model), or the product ad's `servingStatus`.
+- Performance by day, both windows, same fields, same calendar:
+  - v3 `POST /reporting/reports` (until its 2027-06-30 deprecation): `spCampaigns` (groupBy `campaign`, `adGroup` or `campaignPlacement`; columns including `impressions`, `clicks`, `cost`, `purchases7d`/`14d`, `sales7d`/`14d`, `purchasesSameSku7d`, `topOfSearchImpressionShare`, `campaignBudgetAmount`, `campaignBiddingStrategy`, `campaignRuleBasedBudgetAmount`), `spTargeting` (`keywordBid`), `spSearchTerm`, `spAdvertisedProduct`, `spPurchasedProduct`; `timeUnit` `DAILY` with `date`.
+  - v1 `POST /adsApi/v1/create/reports` and `POST /adsApi/v1/retrieve/reports`: `date.value` (or `week.value`), `timeZone.value`, `campaign.id`, `adGroup.id`, `adGroupTarget.id` (the target's key), `targeting.value` ("The keyword, product or category you choose to reach shoppers.") with `targetingMatchType.value`, `searchTerm.value`, `metric.impressions`, `metric.clicks`, `metric.totalCost`, `metric.purchases`, `metric.purchasesPromoted`, `metric.sales`, `metric.salesPromoted`, `metric.unitsSold`, and the report's `warnings[]`. Amazon's own v1 examples write `target.value` and `target.matchType`, and the New features page's warning example flags `"field": "target.value"` as `PENDING_DEPRECATION`; the dimension library defines `targeting.value`, `targetingMatchType.value` and `adGroupTarget.id`, so a request is built from the library and its `warnings[]` are kept (`method/reporting.md` owns the field list).
+  - Peer series: v1 benchmark metrics (`metric.cpcP50`, `metric.ctrP50`, `metric.roasP50`, …) with the benchmark dimensions, or the v3 `crossProgramBenchmarks` report.
+- Where the figures live (read, not memorised): look-back by ad product and account type (Overview of the Reporting v1 API; Reporting columns (version 3)); freshness and restatement (Sponsored ads reporting FAQ (version 3); Amazon Ads reporting overview; the Stream v2 dataset page's "Restatement window"); retention (Dates and historical data retention; each v3 report-type page); dynamic-bid ceilings (SP 3.0 model; Bidding strategies help page; the two library guides); rule-based bidding eligibility and disable rule (rule-based bidding overview); MYE durations (Sell on Amazon MYE page); Brand Lift timing and markets (Amazon Brand Lift page).
+
+**Public Amazon pages, not Ads API hosts:**
+- Amazon Ads Status history: `https://status.ads.amazon.com/history.json` (schema `history.schema.json`), filtered by `region`, `category` `Sponsored ads` / `Measurement & Reporting`, `subcategory` `Ad delivery` / `Campaign management` / `Reports`, and the window's dates.
+
+**SP-API, through `cx-amazon-sp-api` (its door names the report types and queries):**
+- Data Kiosk `analytics_sellerAnalytics_2025_03_31`: `traffic.sessions`, `traffic.pageViews`, `traffic.featureOfferPct`, `traffic.conversionRatePct`, `sales.unitsOrdered`, `sales.averageSellingPrice`, `promotion.*` by ASIN and day, for the advertised ASINs and their siblings.
+- Data Kiosk `analytics_economics_2024_03_15`: `ads` (`AdSummary.adTypeName`, `charge`) by product and day, as a ledger check on spend.
+- The Sales and Traffic report (`sellerSalesAndTrafficReport.json`), Search Query Performance (`sellingPartnerSearchQueryPerformanceReport.json`) for the changed terms' query volume and the brand's share, Search Catalog Performance, coupon and promotion reports, and FBA inventory for stock-outs.
+
+**Keepa, through `cx-keepa`:** price (`AMAZON`, `NEW`, `BUY_BOX_SHIPPING`), sales rank (`SALES`), `LIGHTNING_DEAL`, `couponHistory`, `buyBoxSellerIdHistory`, `COUNT_NEW`, `RATING`, `COUNT_REVIEWS`, `monthlySold`, for the advertised ASIN and the competitors on the changed terms, across both windows.
+
+**Not reachable from the box as built (recorded for completeness):** Amazon Marketing Stream v2 delivers to an S3 bucket and SQS queue the subscriber owns in AWS; AMC queries run in an AMC instance; Brand Lift studies are created on the Studies page or through the Measurement model's DSP paths (`/dsp/measurement/studies/brandLift`, authorised by DSP advertiser account), which the cx-amazon-ads door's sponsored-ads scope does not cover; DSP's customer holdout groups are DSP audiences; MYE results are in Seller Central. [seat, from the pages in §E and the door]
+
+## Pages read
+
+| Page name | URL | Type | Date read | What it gave |
+|---|---|---|---|---|
+| Change history (OpenAPI model) | https://d1y2lf8k3vrkfu.cloudfront.net/openapi/en-us/dest/Changehistory_prod_3p.json | official model | 2026-11-09 | `POST /history`; scope SP+SB, no SD, no who; 90 days; event types, filters, change types, value forms; v1.1 header for THEME |
+| Amazon Ads API v1 merged OpenAPI | https://d1y2lf8k3vrkfu.cloudfront.net/openapi/en-us/dest/AmazonAdsAPIALLMerged_prod_3p.json | official model | 2026-11-09 | 106 paths; `UpdateCampaign` 207 shape; `Campaign.lastUpdatedDateTime`, `tags`; `TargetType`; `Status`, `DeliveryStatus`, 93 `DeliveryReason`s; `Recommendation.state`, `potentialImpacts`; labels; stream subscriptions |
+| Sponsored Products 3.0 OpenAPI | https://d1y2lf8k3vrkfu.cloudfront.net/openapi/en-us/dest/SponsoredProducts_prod_3p.json | official model | 2026-11-09 | Path list; `extendedData` timestamps and serving statuses; bidding strategy enum and definitions; placement enum and stacking example; budget usage schema; no `budgetHistory` path |
+| Eligibility 3.0 OpenAPI | https://d1y2lf8k3vrkfu.cloudfront.net/openapi/en-us/dest/Eligibility_prod_3p.json | official model | 2026-11-09 | Downloaded; product eligibility endpoint confirmed present (content recorded in `angles/paid-keywords.md`) |
+| Sponsored ads reporting FAQ (version 3) | https://advertising.amazon.com/API/docs/en-us/guides/reporting/v3/faq (markdown source on d3a0d0y2hgofx6.cloudfront.net) | official | 2026-11-09 | 12h/24h availability; 3-day traffic validation; restatement 1/7/28 days, 42-day example; traffic-date reporting; 1–2 requests per day; API search terms need a click; v3 deprecation 2027-06-30 |
+| Amazon Ads reporting overview | https://advertising.amazon.com/API/docs/en-us/guides/reporting/overview | official | 2026-11-09 | Surfaces and shutoffs; IVT 30 days; wait ≥5 days; conversions change up to 60 days; Stream v2 70-day backfill; one concurrent v1 report per advertiser |
+| Overview of the Reporting v1 API | https://advertising.amazon.com/API/docs/en-us/guides/reporting/ads-v1/overview | official | 2026-11-09 | Traffic date; promoted/halo; look-back 7d sellers/14d vendors; no view-through for SP/SB CPC; Stream recommended for automation; concurrency per client ID note |
+| Dates and historical data retention | https://advertising.amazon.com/API/docs/en-us/guides/reporting/ads-v1/dates | official | 2026-11-09 | Campaign-timezone days; retention table; benchmark dimensions from 2025-06-01 |
+| Limits and best practices [Reporting v1] | https://advertising.amazon.com/API/docs/en-us/guides/reporting/ads-v1/limits | official | 2026-11-09 | Limits "per client ID"; "Limits are not applied per advertiser account"; concurrency "as low as 1" |
+| New features [Reporting v1] | https://advertising.amazon.com/API/docs/en-us/guides/reporting/ads-v1/new-features | official | 2026-11-09 | Partitioned formats; currency conversion; `warnings[]`; example fields `campaign.bidStrategy`, `campaign.budgetAmount`, `campaign.deliveryStatus` |
+| Getting started with the Reporting API v1 (quickstart) | https://advertising.amazon.com/API/docs/en-us/guides/reporting/ads-v1/quickstart | official | 2026-11-09 | Downloaded for create/retrieve operation names |
+| Dimensions library [Reporting v1] | https://advertising.amazon.com/API/docs/en-us/guides/reporting/ads-v1/dimensions/library | official | 2026-11-09 | Dimension families incl. benchmarks and long-term-sales cohort |
+| Reporting columns (version 3) | https://advertising.amazon.com/API/docs/en-us/guides/reporting/v3/columns | official | 2026-11-09 | `purchases7d`, `purchasesSameSku7d`, `salesOtherSku7d`, `unitsSoldClicks7d`, `topOfSearchImpressionShare`, `longTermSales`, `campaignBudgetAmount`, `campaignBiddingStrategy`, `keywordBid`, budget-rule columns |
+| Campaign reports (v3) | https://advertising.amazon.com/API/docs/en-us/guides/reporting/v3/report-types/campaign | official | 2026-11-09 | Retention 95 days SP; groupBy `campaignPlacement`; budget columns |
+| Benchmarks reports (v3) | https://advertising.amazon.com/API/docs/en-us/guides/reporting/v3/report-types/benchmarks | official | 2026-11-09 | Downloaded; report types `crossProgramBenchmarks`, `dspBenchmarks` (as named in release note) |
+| Purchases; Long-term sales (v1 metrics) | https://advertising.amazon.com/API/docs/en-us/guides/reporting/ads-v1/metrics/amazon-retail-conversions/purchases ; …/long-term-sales | official | 2026-11-09 | Downloaded; metric families for purchases, LTS and benchmarks |
+| Overview [Stream v2] | https://advertising.amazon.com/API/docs/en-us/guides/amazon-marketing-stream/v2/overview | official | 2026-11-09 | S3/SQS delivery to the subscriber's AWS; total values; restated records re-delivered; history for the restatement period; reporting concurrency "(one concurrent report per advertiser, approximately 100 reports per day)" (re-read 2026-11-09 for the gap fill; the earlier row's "no concurrency statement" was wrong) |
+| Data delivery [Stream v2] | https://advertising.amazon.com/API/docs/en-us/guides/amazon-marketing-stream/v2/data-delivery | official | 2026-11-09 | At-least-once, unordered; keep highest `streamBatch.version`; silent no-data on missing permissions |
+| Overview [Stream v2 datasets] | https://advertising.amazon.com/API/docs/en-us/guides/amazon-marketing-stream/v2/datasets/overview | official | 2026-11-09 | Restatement rule "final only after its window has passed"; `(Other)` thresholds; "Totals stay complete" |
+| Ads performance [Stream v2 dataset] | https://advertising.amazon.com/API/docs/en-us/guides/amazon-marketing-stream/v2/datasets/ads-performance | official | 2026-11-09 | Hourly; restatement window 70 days; no calculated metrics; aggregate before dividing; `searchTerm.value` thresholds |
+| Overview [Stream v1] | https://advertising.amazon.com/API/docs/en-us/guides/amazon-marketing-stream/overview | official | 2026-11-09 | Downloaded; v1 hourly delta delivery, shut off after 2027-06-30 (per reporting overview) |
+| Get started [Stream v2]; Process your data [Stream v2] | https://advertising.amazon.com/API/docs/en-us/guides/amazon-marketing-stream/v2/get-started ; …/process-your-data | official | 2026-11-09 | Downloaded; setup through AWS CloudFormation/CDK |
+| All releases | https://advertising.amazon.com/API/docs/en-us/release-notes/index | official | 2026-11-09 | Warnings (2026-10-05); Stream v2 + v1 reporting GA (2026-09-30); benchmarks (2026-05-18, 2026-09-25); LTS (2025-06-05) and accumulated sales (2026-09-25); AMC promotional signals (2026-10-20); SB DCO (2025-04-17); recommendations API (2026-08-31) and apply (2023); budget usage (2023-04-05); Status history (2024-12-02); Brand Lift endpoints; budget-rule history deprecation; SB v4 testing note (2024-06-13) |
+| Amazon Ads API v1 release notes | https://advertising.amazon.com/API/docs/en-us/release-notes/ads-api-v1 | official | 2026-11-09 | Downloaded; cross-check of v1 launch items |
+| Amazon Ads API overview | https://advertising.amazon.com/API/docs/en-us/reference/api-overview | official | 2026-11-09 | Regional hosts; typical client loop (store entities, pull reports, change bids and budgets) |
+| Amazon Ads API v1 overview | https://advertising.amazon.com/API/docs/en-us/reference/amazon-ads/overview | official | 2026-11-09 | v1 common model across SP, SB, SD, STV, DSP |
+| Campaign management overview; Entities overview | https://advertising.amazon.com/API/docs/en-us/guides/campaign-management/overview ; …/entities/overview | official | 2026-11-09 | Entity guide links |
+| Target entity guide | https://advertising.amazon.com/API/docs/en-us/guides/campaign-management/entities/target | official | 2026-11-09 | Support table: no location, daypart, device or audience targets for SP and SB |
+| Campaign entity guide | https://advertising.amazon.com/API/docs/en-us/guides/campaign-management/entities/campaign | official | 2026-11-09 | `tags` supported for SP/SB; `optimizations`, `bidAdjustments`, `budgets` support by product |
+| Recommendations API overview | https://advertising.amazon.com/API/docs/en-us/guides/recommendations/recommendations-api/overview | official | 2026-11-09 | Endpoints (`GET /adsApi/v1/list/recommendations`, which the v1 model names `GET /adsApi/v1/recommendations`); universal fields incl. `state`; launched August 2026 |
+| Sponsored Products rule-based bidding overview | https://advertising.amazon.com/API/docs/en-us/guides/rules/bidding-rules/overview | official | 2026-11-09 | Base-bid adjustment; disable rule; eligibility; 21-day guardrail; minimum budgets |
+| Sponsored ads budget rules overview | https://advertising.amazon.com/API/docs/en-us/guides/rules/budget-rules/overview | official | 2026-11-09 | Schedule- and performance-based rules; 7/14-day metric basis |
+| Theme-based bid suggestions - Quick-start guide | https://advertising.amazon.com/API/docs/en-us/guides/sponsored-products/bid-suggestions/theme-based-bid-suggestions-quickstart-guide | official | 2026-11-09 | Suggestions from recent winning bids; impact metrics "not estimates of expected performance" |
+| Sponsored Brands v4 features | https://advertising.amazon.com/API/docs/en-us/guides/sponsored-brands/v4-features | official | 2026-11-09 | Several ads per ad group "to test"; no split method |
+| Sponsored Brands overview | https://advertising.amazon.com/API/docs/en-us/guides/sponsored-brands/overview | official | 2026-11-09 | Serving status mapping; `servingStatus` only via list |
+| How AMC works | https://advertising.amazon.com/API/docs/en-us/guides/amazon-marketing-cloud/how-amc-works | official | 2026-11-09 | Pseudonymized events; no raw export; self-service instance; Paid Features |
+| AMC overview; AMC data retention; Sponsored ads traffic; Conversions (AMC data sources) | https://advertising.amazon.com/API/docs/en-us/guides/amazon-marketing-cloud/overview ; …/datasources/data-retention ; …/datasources/sponsored_ads_traffic ; …/datasources/conversions | official | 2026-11-09 | Downloaded; retention up to 5 years for eligible traffic views (per 2026-10-27 release note) |
+| Data aggregation thresholds in AMC | https://advertising.amazon.com/API/docs/en-us/guides/amazon-marketing-cloud/aggregation-threshold | official | 2026-11-09 | Threshold classes; 100 distinct users for medium/high |
+| Marketing Mix Modeling overview | https://advertising.amazon.com/API/docs/en-us/guides/reporting/marketing-mix-modeling/overview | official | 2026-11-09 | Aggregated signals; NA region; marketplaces |
+| Brand Metrics API open beta overview | https://advertising.amazon.com/API/docs/en-us/guides/reporting/brand-metrics/overview | official | 2026-11-09 | Awareness and Consideration indices "using models predictive of consideration and sales"; eligibility |
+| Amazon Attribution API overview | https://advertising.amazon.com/API/docs/en-us/guides/amazon-attribution/overview | official | 2026-11-09 | Tagged off-Amazon media; beta; markets |
+| Overview [MCP] | https://advertising.amazon.com/API/docs/en-us/mcp/mcp-overview | official | 2026-11-09 | Standard vs expert tools; "Increase my campaign budget" example |
+| MCP campaign_management specification | https://d3a0d0y2hgofx6.cloudfront.net/en-us/mcp/campaign_management.json | official model | 2026-11-09 | 32 tools incl. `update_target_bid`, `update_campaign_budget`, `update_campaign_state`, `create_bid_adjustment_rule` |
+| Amazon Ads Status (Known issues) | https://advertising.amazon.com/API/docs/en-us/support/status | official | 2026-11-09 | Status and history JSON endpoints; 6 months |
+| Amazon Ads Status history schema; history.json | https://status.ads.amazon.com/history.schema.json ; https://status.ads.amazon.com/history.json | official | 2026-11-09 | Item fields and enums; 15 items live; region values `EU`/`NA`/`FE` |
+| Guide to dynamic bidding - up and down with Sponsored Products | https://advertising.amazon.com/library/guides/dynamic-bidding-sponsored-products | official | 2026-11-09 | Up to 100% all placements; "limit your changes while testing"; review every two weeks; spend double |
+| The author's guide to Sponsored Products | https://advertising.amazon.com/library/guides/authors-guide-to-sponsored-products | official | 2026-11-09 | 100% top / 50% other; wait two weeks; check every two weeks |
+| A guide to targeting with Sponsored Products | https://advertising.amazon.com/library/guides/targeting-with-sponsored-products | official | 2026-11-09 | Negatives after 20 clicks, two weeks; match types 1–2 weeks; auto two weeks; benchmarks from history |
+| Best practices for your Sponsored Products ads | https://advertising.amazon.com/library/guides/sponsored-products-best-practices | official | 2026-11-09 | In stock and Featured Offer; monitor twice a week in week one |
+| Amazon Ads Agent announcement (H1 "Amazon Ads introduces Full-Funnel Campaigns, DVA+, and enhanced chat experience in AI-powered ad platform, Amazon Ads Agent"; HTML title "Introducing Amazon Ads Agent, your AI-powered ad platform") | https://advertising.amazon.com/library/news/amazon-ads-agent | official | 2026-11-09 | One-click apply of recommendations, rolling out "over the coming months"; AI "on their behalf"; 25% / 10% claim with no method; no audit trail described; no 67% figure |
+| Get more done with natural conversation | https://advertising.amazon.com/resources/whats-new/conversational-experience-amazon-ads-agent | official | 2026-11-09 | Closed beta across North and South America, Europe, Middle East and Asia Pacific marketplaces; recommendations for "audience reach, bids, and budgets" |
+| unBoxed Tokyo 2026 AI-powered ad innovations news | https://advertising.amazon.com/library/news/unboxed-tokyo-2026-news | official | 2026-11-09 | 2026-11-05; AI guidance applies in one click; no experiment announcement |
+| Grow total sales with Full-Funnel Campaigns | https://advertising.amazon.com/resources/whats-new/full-funnel-campaigns-from-discovery-to-sales | official | 2026-11-09 | 2026-09-29 open beta (US); 67% higher Long-Term ROAS and 29% lower cost per NTB purchase "against the blended average", "Amazon internal (June – August 2026)" (added by verifier) |
+| Uncover ad value across the funnel with multi-touch attribution (beta) | https://advertising.amazon.com/resources/whats-new/uncover-ad-value-across-funnel-with-multi-touch-attribution | official | 2026-11-09 | MTA divides credit; experiments + ML; last-touch is standard; US (added by verifier) |
+| Self-Service Third Party Measurement Studies | https://advertising.amazon.com/resources/whats-new/self-service-partner-measurement-studies | official | 2026-11-09 | 2026-05-15; 18 countries; results within Amazon DSP (added by verifier) |
+| unBoxed 2026 news: Explore the latest advertising announcements | https://advertising.amazon.com/library/news/unboxed-2026-news-announcements | official | 2026-11-09 | No experiment, holdout or lift tool for sponsored ads in the page text (added by verifier) |
+| Sponsored Products prompts and Sponsored Brands prompts | https://advertising.amazon.com/resources/whats-new/unboxed-2025-sponsored-products-and-sponsored-brands-prompts | official | 2026-11-09 | Auto-enrolment; CPC charging; US GA 2026-03-25 |
+| Identify the right creative for an upcoming brand campaign with Creative Testing | https://advertising.amazon.com/resources/whats-new/creative-testing | official | 2026-11-09 | DSP survey; US; 2023-01-30 |
+| Brand Lift: Study the impact of your advertising | https://advertising.amazon.com/measurement-analytics/brand-lift | official | 2026-11-09 | Exposed vs control survey; SP excluded; free above minimums; 10 business days; API and Ads Agent markets; sponsored ads US |
+| Measure brand marketing impact using Amazon Brand Lift | https://advertising.amazon.com/resources/whats-new/measure-brand-marketing-impact-using-amazon-brand-lift | official | 2026-11-09 | GA for SB, SD, STV in US, 2024-06-30 |
+| History now available for Sponsored Products and Sponsored Brands (blog) | https://advertising.amazon.com/blog/history-now-available-for-sponsored-products-and-sponsored-brands | official | 2026-11-09 | Lands on a general Sponsored ads page; no history content |
+| Test listing content with Manage Your Experiments | https://sell.amazon.com/tools/manage-your-experiments | official | 2026-11-09 | Testable content; eligibility; 8–10 weeks |
+| Seller Forums: How often should we be updating bids in the PPC | https://sellercentral.amazon.com/seller-forums/discussions/t/2f0b934c4850ae38be6d08910effc64b | opinion (sellers) | 2026-11-09 | 12 days; a month or longer |
+| selling-partner-api-models: schemas/data-kiosk and schemas/reports listings | https://github.com/amzn/selling-partner-api-models/tree/main/schemas | official | 2026-11-09 | Dataset and report schema file names |
+| analytics_sellerAnalytics_2025_03_31.graphql | https://github.com/amzn/selling-partner-api-models/blob/main/schemas/data-kiosk/analytics_sellerAnalytics_2025_03_31.graphql | official | 2026-11-09 | `featureOfferPct`, `sessions`, `conversionRatePct` (and deprecated spelling), `unitsOrdered`, `averageSellingPrice`, `promotionType` |
+| analytics_economics_2024_03_15.graphql | https://github.com/amzn/selling-partner-api-models/blob/main/schemas/data-kiosk/analytics_economics_2024_03_15.graphql | official | 2026-11-09 | `AdSummary.adTypeName`, `charge` |
+| Keepa Product.java | https://github.com/keepacom/api_backend/blob/master/src/main/java/com/keepa/api/backend/structs/Product.java | vendor | 2026-11-09 | csv types; `couponHistory`; `buyBoxSellerIdHistory`; `monthlySold` |
+| PPC Land (Luis Rijo): Amazon rolls out a new history feature… | https://ppc.land/amazon-rolls-out-a-new-history-feature-for-sponsored-products-and-sponsored-brands-campaigns/ | trade press | 2026-11-09 | Console history 90 days; SP ad groups; DSP |
+| PPC Land (Luis Rijo): Amazon Sponsored Products campaigns gain creator placements on August 10 | https://ppc.land/amazon-sponsored-products-campaigns-gain-creator-placements-on-august-10/ | trade press | 2026-11-09 | Off-Amazon creator placements from 2026-08-10, enrolled by default (added by verifier) |
+| Improvado (Ivo Petrovic): Amazon unBoxed 2026: What Changed for Retail Media Metrics (HTML title "…for Measurement") | https://improvado.io/blog/amazon-unboxed-2026-retail-media-measurement | opinion | 2026-11-09 | Ads Agent, Full-Funnel, DVA+; reads Amazon's footnote as not a holdout |
+| Ad Badger: Attribution Window Blind Spots | https://www.adbadger.com/blog/attribution-window-blind-spots-how-14-day-windows-quietly-wreck-your-decisions/ | opinion | 2026-11-09 | 14-day and order-date claims (conflict); closed cohorts |
+| Ad Badger: You're Optimizing Too Much (2026-06-02; PPC Den podcast with Clément Hynaux) | https://www.adbadger.com/blog/amazon-ppc-optimization-mistakes/ | opinion | 2026-11-09 | ≥7 days; 30 clicks may not suffice; Hynaux: drop last 48h, "rough draft"; click-date attribution |
+| Ad Badger help 196 | https://support.adbadger.com/hc/knowledge-base/articles/196-how-many-clicks-how-long-does-bbb-wait-after-changing-a-bid-before-it-changes-it-again | opinion (tool) | 2026-11-09 | 24h ±15 min between automated changes |
+| BidX (Selina Krüger): Understanding the Amazon Attribution Window and its impact? | https://bidx.io/blog/amazon-attribution-window | opinion | 2026-11-09 | SP 7, SB 14; seventh day after period when the gap to target is small |
+| AdLabs (Andrew Bailiff, 2025-09-03): Amazon PPC Bid Optimization: How Long Should You Wait Before Adjusting Bids Again? | https://adlabs.app/amazon-ppc-bid-optimization-how-long-should-you-wait-before-adjusting-bids-again/ | opinion | 2026-11-09 | Weekly cadence; spend first after a change; against click-count waits |
+| Pare: Amazon PPC Governance | https://pare.so/blog/amazon-ppc-governance-decision-matrix-q4-bid-changes-2026 | opinion | 2026-11-09 | 7–14 days between changes on a target; one-account case |
+| Marin Software: Testing effectively with Amazon Sponsored Brands campaigns | https://www.marinsoftware.com/blog/testing-effectively-with-amazon-sponsored-brands-campaigns | opinion | 2026-11-09 | Concurrent vs iterative designs |
+| ClearAds: Mastering A/B Testing in Amazon PPC | https://clearadsagency.com/amazon-ppc/ab-testing/ | opinion | 2026-11-09 | Two ad groups, split budget |
+| Sagum: How do I use A/B testing for Amazon ad creatives? | https://sagum.com/2026/04/19/how-do-i-use-a-b-testing-for-amazon-ad-creatives/ | opinion | 2026-11-09 | Built-in SB testing claim; SP manual |
+| SalesDuo: Amazon Ad Creative Testing Framework | https://salesduo.com/blog/amazon-ad-creative-testing-framework | opinion | 2026-11-09 | Creative Testing is a DSP survey |
+| Perpetua: How to identify incremental opportunities on Amazon | https://perpetua.io/blog-incrementality-amazon-ads-optimization-strategies/ | opinion | 2026-11-09 | Branded strategy change; total ACoS |
+| Eva Commerce: Amazon Ad Incrementality: Measurement Guide for 2026 | https://eva.guru/blog/amazon-ad-incrementality-measurement/ | opinion | 2026-11-09 | Geo holdout claim (conflicts with SP targeting) |
+| FiveX: DSP + AMC incrementality profit test | https://fivex.com/blog/amazon-dsp-amc-incrementality-profit-test/ | opinion | 2026-11-09 | DSP holdout example, illustrative |
+| Seller Labs (Denis D): Sales Dropped? Split It Before You Raise Ad Spend | https://www.sellerlabs.com/blog/amazon-sales-drop-traffic-conversion-price/ | opinion | 2026-11-09 | Small-number rules; windows; confounders |
+| Consumer Heterogeneity and Paid Search Effectiveness (NBER abstract and full text) | https://www.nber.org/papers/w20171 ; https://www.nber.org/system/files/working_papers/w20171/w20171.pdf | research | 2026-11-09 | Brand on-off with Google control; 99.5%; 30% DMA non-brand 60 days |
+| Competition and Crowd-Out for Brand Keywords in Sponsored Search (IDEAS) | https://ideas.repec.org/a/inm/ormksc/v37y2018i2p200-215.html | research | 2026-11-09 | 1–4% brand-ad effect; 18–42% stolen |
+| Does Sponsored Search Advertising Augment Organic Search? (IDEAS) | https://ideas.repec.org/a/inm/ormnsc/v71y2025i11p9687-9709.html | research | 2026-11-09 | 3% visitor holdout |
+| Do Sellers Benefit from Sponsored Product Listings? (IDEAS) | https://ideas.repec.org/a/inm/ormksc/v43y2024i4p817-839.html | research | 2026-11-09 | Organic top preferred |
+| Advertising Spillovers in Mobile Apps (arXiv) | https://arxiv.org/abs/2504.16151 | research | 2026-11-09 | Shutoff −20–30% organic via ranking |
+| The Unfavorable Economics of Measuring the Returns to Advertising (IDEAS) | https://ideas.repec.org/a/oup/qjecon/v130y2015i4p1941-1973.html | research | 2026-11-09 | Median ROI CI >100 points; CV 10 |
+| A Comparison of Approaches to Advertising Measurement (IDEAS) | https://ideas.repec.org/a/inm/ormksc/v38y2019i2p193-225.html | research | 2026-11-09 | Observational methods fail against RCTs |
+| Online ads and offline sales (IDEAS) | https://ideas.repec.org/a/kap/qmktec/v12y2014i3p235-266.html | research | 2026-11-09 | 78% from non-clickers; observational bias |
+| When Less Is More (IDEAS) | https://ideas.repec.org/a/inm/ormksc/v36y2017i1p43-53.html | research | 2026-11-09 | Control ads +31% precision |
+| Position Effects in Search Advertising (IDEAS) | https://ideas.repec.org/a/inm/ormksc/v34y2015i3p388-407.html | research | 2026-11-09 | Selection in position comparisons |
+| Randomization and The Pernicious Effects of Limited Budgets on Auction Experiments (PMLR) | https://proceedings.mlr.press/v51/basse16b.html | research | 2026-11-09 | Budget interference; separate budgets |
+| Trustworthy Online Marketplace Experimentation with Budget-split Design (arXiv); Crossref record of the KDD 2021 version | https://arxiv.org/abs/2012.08724 ; https://api.crossref.org/works?query.bibliographic=… | research | 2026-11-09 | SUTVA violation; budget-split; KDD title and DOI |
+| Amazon Ads Multi-Touch Attribution (arXiv abstract and HTML) | https://arxiv.org/abs/2508.08209 ; https://arxiv.org/html/2508.08209v1 | research (Amazon-authored) | 2026-11-09 | Treatment/holdout RCTs; "hundreds of thousands of RCTs across our ad products"; no advertiser access described (added by verifier) |
+| Experimental Design in Two-Sided Platforms (arXiv) | https://arxiv.org/abs/2002.05670 | research | 2026-11-09 | Interference bias |
+| Online Causal Inference for Advertising in Real-Time Bidding Auctions (arXiv) | https://arxiv.org/abs/1908.08600 | research | 2026-11-09 | Effects identified by optimal bids |
+| Focusing on the Long-term (Google Research PDF) | https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43887.pdf | research | 2026-11-09 | Short vs long term; ads blindness |
+| Always Valid Inference (arXiv) | https://arxiv.org/abs/1512.04922 | research | 2026-11-09 | Peeking invalidates inference |
+| Inferring causal impact using Bayesian structural time-series models (arXiv) | https://arxiv.org/abs/1506.00356 | research | 2026-11-09 | Synthetic control from covariate series |
+| Applying the Delta method in metric analytics (arXiv) | https://arxiv.org/abs/1803.06336 | research | 2026-11-09 | Ratio-metric inference |
+| Designing and Deploying Online Field Experiments (arXiv) | https://arxiv.org/abs/1409.3174 | research | 2026-11-09 | Read; not used for a claim |
+| Sponsored is the New Organic (arXiv) | https://arxiv.org/abs/2407.19099 | research | 2026-11-09 | Sponsored items with poor organic rank on page one |
+| Semantic Scholar records (Kohavi et al. 2009, 2012, 2014; Kohavi, Deng & Vermeer 2022; Brown, Cai & DasGupta 2001; Barnett et al. 2005 [record year 2004]; Sahni 2016) | https://api.semanticscholar.org/graph/v1/paper/DOI:… | research metadata | 2026-11-09 | Titles, venues, years, abstracts where served |
+| Binomial proportion confidence interval; Rule of three (statistics) (Wikipedia) | https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval ; https://en.wikipedia.org/wiki/Rule_of_three_(statistics) | reference | 2026-11-09 | Wilson, Jeffreys, rule of three wording |
+| cx-amazon-ads SKILL.md and references/door.md | /tmp/adsread/cx-amazon-ads/skills/amazon-ads/ | ComChatX | 2026-11-09 | Door behaviour: POST as change, `sent, outcome unknown`, no retries |
+| cx-listing-playbook research: method/effect-measurement.md, angles/paid-keywords.md, method/amazon-change-calendar.md, method/report-lifecycle.md, method/help-page-access.md | /tmp/adsread/cx-listing-playbook/research/ | ComChatX research | 2026-11-09 | Verdicts and listing method; leads re-opened above; ads-side change calendar |
+| Traffic validation (help, GBYPYH79NGJJ5JPS; article JSON in the page) | https://advertising.amazon.com/help/GBYPYH79NGJJ5JPS | official | 2026-11-09 (gap fill) | IVT monitored up to 30 days and removed; later restatement after investigation; Gross and Invalid Traffic report |
+| Bidding strategies for Sponsored Products (help, GCU2BUWJH2W3A8Z7) | https://advertising.amazon.com/help/GCU2BUWJH2W3A8Z7 | official | 2026-11-09 (gap fill) | "up to 100% for all placements", $1.00 → $2.00 example; lastModified 2026-09-28 |
+| Understand keyword match types (help, GHTRFDZRJPW6764R) | https://advertising.amazon.com/help/GHTRFDZRJPW6764R | official | 2026-11-09 (gap fill) | Highest-bid match type used for a matching query; lastModified 2026-07-29 |
+| Understand Amazon Ads auctions (help, GEXGZNBBWPZZK7P8) | https://advertising.amazon.com/help/GEXGZNBBWPZZK7P8 | official | 2026-11-09 (gap fill) | Reserves; peak-period value; "continually experimenting"; lastModified 2026-04-27 |
+| Amazon's response to the FTC's lawsuit regarding Sponsored Ads | https://www.aboutamazon.com/md/company-news/amazon-ftc-sponsored-ads-lawsuit-response | official | 2026-11-09 (gap fill) | Soft and hard reserves; relevance-model testing from 2014; 80% of bid changes within a day; competitor bids −83% |
+| Measurement (OpenAPI model, 3.0) | https://d1y2lf8k3vrkfu.cloudfront.net/openapi/en-us/dest/Measurement_prod_3p.json | official model | 2026-11-09 (gap fill) | Brand Lift operations under `/dsp/measurement/…` and `/measurement/studies/brandLift/{studyId}/result`; DSP authorisation; no `/measurementStudies` path |
+| SP 3.0 OpenAPI (re-read) | https://d1y2lf8k3vrkfu.cloudfront.net/openapi/en-us/dest/SponsoredProducts_prod_3p.json | official model | 2026-11-09 (gap fill) | `includeExtendedDataFields` description against the `*ExtendedData` schemas |
+| Targeting; Targeting match type; Target (v1 dimension pages) | https://d3a0d0y2hgofx6.cloudfront.net/en-us/guides/reporting/ads-v1/dimensions/targeting/targeting.md ; …/targeting-match-type.md ; …/target.md | official | 2026-11-09 (gap fill) | `targeting.value`, `targetingMatchType.value`, `adGroupTarget.id` definitions |
+| New features [Reporting v1] (re-read); sp-targeting report type (v1) | https://d3a0d0y2hgofx6.cloudfront.net/en-us/guides/reporting/ads-v1/new-features.md ; https://d3a0d0y2hgofx6.cloudfront.net/en-us/guides/reporting/ads-v1/report-types/sp-targeting.md | official | 2026-11-09 (gap fill) | `target.value` flagged `PENDING_DEPRECATION` in the warning example; `target.value`, `target.matchType` in the example |
+| Purchases (v1 metrics, re-read) | https://d3a0d0y2hgofx6.cloudfront.net/en-us/guides/reporting/ads-v1/metrics/amazon-retail-conversions/purchases.md | official | 2026-11-09 (gap fill) | 56 `metric.multiTouch*` fields; v3 name N/A; compatible dimensions |
+| Overview [Stream v2] (re-read) | https://d3a0d0y2hgofx6.cloudfront.net/en-us/guides/amazon-marketing-stream/v2/overview.md | official | 2026-11-09 (gap fill) | "(one concurrent report per advertiser, approximately 100 reports per day)" |
+| How to test DSP bid adjustments | https://d3a0d0y2hgofx6.cloudfront.net/en-us/guides/dsp/bid-adjustments/testing.md | official | 2026-11-09 (gap fill) | 10 customer holdout groups of 10%; setup; advice against before/after |
+| Ads API docs table of contents and search index | https://d3a0d0y2hgofx6.cloudfront.net/en-us/toc2.json ; https://d3a0d0y2hgofx6.cloudfront.net/en-us/searchIndex.json ; https://d3a0d0y2hgofx6.cloudfront.net/llms.txt | official | 2026-11-09 (gap fill) | Full-text search for holdout/A/B/experiment; SB v3→v4 migration guide's test sentence |
+| AMC data source brand_store_insights_paid | https://d3a0d0y2hgofx6.cloudfront.net/en-us/guides/amazon-marketing-cloud/datasources/brand_store_insights_paid.md | official | 2026-11-09 (gap fill) | `ingress_type` "7 through 11 - experimentation" |
+| Seller Forums EU: Brand Story A/B tests with Manage Your Experiments | https://sellercentral-europe.amazon.com/seller-forums/discussions/t/b3cc84fe-5747-48e3-a3d6-9ee5e27409a7 | official (News_Amazon) | 2026-11-09 (gap fill) | MYE covers A+ Brand Story |
+| Trellis helps lift Dandy Blend's results with AI creative (case study) | https://advertising.amazon.com/library/case-studies/dandy-blend-trellis-creative-success | official page, partner account | 2026-11-09 (gap fill) | Two-campaign "A/B" at comparable impressions |
+| Flywheel: creative testing engine (expert advice) | https://advertising.amazon.com/library/expert-advice/flywheel-creative-learning | opinion (partner on Amazon's site) | 2026-11-09 (gap fill) | Layered creative measurement; no split tool |
+| A guide to targeting with Sponsored Products (re-read) | https://advertising.amazon.com/library/guides/targeting-with-sponsored-products | official | 2026-11-09 (gap fill) | "you won't be bidding against yourself"; multiple campaigns, priority bids |
+| Guide to dynamic bidding - up and down; The author's guide (re-read) | https://advertising.amazon.com/library/guides/dynamic-bidding-sponsored-products ; https://advertising.amazon.com/library/guides/authors-guide-to-sponsored-products | official | 2026-11-09 (gap fill) | 100% all placements against 100/50 |
+| Skai: Retail media incrementality: Skai's innovative approach | https://skai.io/blog/retail-media-incrementality-skais-innovative-approach/ | opinion (vendor) | 2026-11-09 (gap fill) | Holdout "gold standard"; iROAS formula; 5–15% branded-query claim |
+| Skai: Unlocking Incrementality Insights in Retail Media | https://skai.io/blog/unlocking-incrementality-insights-in-retail-media-six-innovative-approaches/ | opinion (vendor) | 2026-11-09 (gap fill) | Product-holdout alpha; daily third-party iROAS |
+| Pacvue Incrementality Console (GlobeNewswire) | https://www.globenewswire.com/news-release/2024/09/17/2947513/0/en/Pacvue-Debuts-New-Incrementality-Console-to-Help-Advertisers-Grow-Incremental-Revenue-with-Accurate-and-Actionable-Insights.html | opinion (vendor) | 2026-11-09 (gap fill) | Regression-modelled iROAS; no holdout stated |
+| Teikametrics help: bid frequency; bid exploration; Bidder intro | https://help.teikametrics.com/en/articles/8358916-how-frequently-are-bids-changed.md ; https://help.teikametrics.com/en/articles/9306896-how-to-speed-up-bid-exploration-in-your-campaigns.md ; https://help.teikametrics.com/en/articles/8308845-intro-to-the-teikametrics-bidder.md | opinion (vendor) | 2026-11-09 (gap fill) | Hourly bids; 24-hour increase limit; min-bid formula; outcome claims with no method |
+| Intentwise: Speed up your ad analysis with What Changed | https://www.intentwise.com/blog/speed-up-your-ad-analysis-with-what-changed | opinion (vendor) | 2026-11-09 (gap fill) | Two-period Impact Score; "the starting point" |
+| Helium 10 Ads Change Log Page (article API) | https://kb.helium10.com/api/v2/help_center/en-us/articles/27613163890715.json | opinion (vendor) | 2026-11-09 (gap fill) | Change types; change-source labels |
+| Perpetua: Redefining Success with AMC | https://perpetua.io/blog-amazon-marketing-cloud-amc-incrementality-growth-strategies/ | opinion (vendor) | 2026-11-09 (gap fill) | NTB and LTV measures; no holdout |
+| AdExchanger: Beware The Platform Incrementality Trap | https://www.adexchanger.com/commerce/beware-the-platform-incrementality-trap/ | opinion (trade press) | 2026-11-09 (gap fill) | Amazon tests recent; AMC test groups in beta |
+| Optmyzr: Amazon Ads Reporting Delay Is Killing Your Conversions | https://www.optmyzr.com/blog/amazon-ads-reporting-data-delay-study/ | opinion (vendor study) | 2026-11-09 (gap fill) | 17-day maturation of 14,991 campaigns; top-5% growth |
+| Hershaff: Estimating Amazon Ad Incrementality for Sellers | https://whatstheimpact.substack.com/p/estimating-amazon-ad-incrementality | opinion (seller-analyst) | 2026-11-09 (gap fill) | Daily switchback; 2.35 / 1.25 / 1.54; adstock 0.35 |
+| Pattern: The Importance of Sponsored Brands A/B Testing | https://www.pattern.com/blog/the-importance-of-sponsored-brands-a-b-testing | opinion | 2026-11-09 (gap fill) | Manual SB A/B; two-month period |
+| Relevant Audience: Amazon Ads 20% of sales after attribution window | https://www.relevantaudience.com/ecommerce-marketing/amazon-ads-20-percent-sales-after-attribution-window/ | press | 2026-11-09 (gap fill) | Lily Tong's 20% |
+| Qin 2023 (Amazon Science page and PDF) | https://www.amazon.science/publications/lengthen-your-attribution-window-which-digital-ads-have-most-long-term-impact ; https://cdn.amazon.science/79/6e/915822bb4663a28c97c869a75931/lengthen-your-attribution-window-which-digital-ads-have-most-long-term-impact.pdf | research (Amazon-authored) | 2026-11-09 (gap fill) | SP carry-over within three weeks; sample and limits |
+| Pauwels et al. 2023 (PDF) | https://cdn.amazon.science/d1/66/f222c8524c4d87fc1b0b342345b7/sponsored-brands-video-rings-up-clicks-and-sales-in-the-short-and-long-run.pdf | research (Amazon-authored) | 2026-11-09 (gap fill) | Matching and propensity methods; 21.3%; 1.25×/1.1× |
+| Learning to Bid with AuctionGym (Amazon Science page and PDF) | https://www.amazon.science/publications/learning-to-bid-with-auctiongym ; https://cdn.amazon.science/4f/3c/7f9a5c6c4181894d5e64c684c0d7/learning-to-bid-with-auctiongym.pdf | research (Amazon-authored) | 2026-11-09 (gap fill) | No counterfactual bids; logged-data confounding |
+| Advancing Ad Auction Realism (arXiv abstract) | https://arxiv.org/abs/2307.11732 | research (Amazon-affiliated) | 2026-11-09 (gap fill) | Partial feedback; soft floors |
+| Addressing Cold Start in Product Search via Empirical Bayes | https://www.amazon.science/publications/addressing-cold-start-in-product-search-via-empirical-bayes | research (Amazon-authored) | 2026-11-09 (gap fill) | A/B on 50 million queries |
+| Behavioral Feature Boosting via Substitute Relationships (arXiv abstract) | https://arxiv.org/abs/2602.14502 | research (Amazon-authored) | 2026-11-09 (gap fill) | In production since 2025 |
+| The science of price experiments in the Amazon Store | https://www.amazon.science/blog/the-science-of-price-experiments-in-the-amazon-store | research (Amazon-authored blog) | 2026-11-09 (gap fill) | Random-days switchback; −60% standard error; recommendation carryover |
+| Design and Analysis of Switchback Experiments (arXiv) | https://arxiv.org/abs/2009.00148 | research | 2026-11-09 (gap fill) | Theorems 1–2; Horvitz-Thompson bias when p<m; T/m>100 |
+| Goodman-Bacon 2021; Callaway & Sant'Anna 2021; Sun & Abraham 2021 (IDEAS) | https://ideas.repec.org/a/eee/econom/v225y2021i2p254-277.html ; https://ideas.repec.org/a/eee/econom/v225y2021i2p200-230.html ; https://ideas.repec.org/a/eee/econom/v225y2021i2p175-199.html | research | 2026-11-09 (gap fill) | Staggered DiD abstracts |
+| de Chaisemartin & D'Haultfœuille 2020; Arkhangelsky et al. 2021 (IDEAS) | https://ideas.repec.org/a/aea/aecrev/v110y2020i9p2964-96.html ; https://ideas.repec.org/a/aea/aecrev/v111y2021i12p4088-4118.html | research | 2026-11-09 (gap fill) | Negative weights; synthetic DiD |
+| Abadie, Diamond & Hainmueller (NBER w12831) | https://www.nber.org/papers/w12831 | research | 2026-11-09 (gap fill) | Synthetic control abstract |
+| Abadie 2021 (NBER SI copy; EconPapers record) | https://conference.nber.org/confer/2021/SI2021/Abadie_2021.pdf ; https://econpapers.repec.org/RePEc:aea:jeclit:v:59:y:2021:i:2:p:391-425 | research | 2026-11-09 (gap fill) | Feasibility conditions; placebo inference |
+| CUPED (ExP PDF) | https://exp-platform.com/Documents/2013-02-CUPED-ImprovingSensitivityOfControlledExperiments.pdf | research | 2026-11-09 (gap fill) | 1−ρ²; ~50% on Bing; pre-period 1–2 weeks |
+| A Dirty Dozen (ExP PDF) | https://exp-platform.com/Documents/2017-08%20KDDMetricInterpretationPitfalls.pdf | research | 2026-11-09 (gap fill) | SRM; novelty and primacy |
+| Five puzzling outcomes explained (ExP PDF) | https://exp-platform.com/Documents/puzzlingOutcomesInControlledExperiments.pdf | research | 2026-11-09 (gap fill) | Carryover; A/A tests; one-week minimum |
+| Objective Bayesian two-sample testing (ExP page) | https://exp-platform.com/objective-bayesian-ab/ | research | 2026-11-09 (gap fill) | Priors from thousands of experiments |
+| Trustworthy Online Controlled Experiments (Open Library record) | https://openlibrary.org/search.json?q=trustworthy+online+controlled+experiments | research (book, metadata only) | 2026-11-09 (gap fill) | Bibliographic record only |
+| Limiting Bias from Test-Control Interference (arXiv abstract; EconPapers) | https://arxiv.org/abs/2004.12162 ; https://econpapers.repec.org/paper/arxpapers/2004.12162.htm | research | 2026-11-09 (gap fill) | Up to 62% bias reduction; account of Blake & Coey |
+| Holtz et al., Airbnb pricing meta-experiment (Crossref) | https://api.crossref.org/works/10.1287/mnsc.2020.01157 | research | 2026-11-09 (gap fill) | At least 20% interference bias |
+| Experimenting in Equilibrium (arXiv) | https://arxiv.org/abs/1903.02124 | research | 2026-11-09 (gap fill) | Local perturbations; marginal response at fixed equilibrium |
+| Benjamini & Hochberg 1995 (Crossref) | https://api.crossref.org/works/10.1111/j.2517-6161.1995.tb02031.x | research | 2026-11-09 (gap fill) | FDR definition and procedure |
+| Gelman, Hill & Yajima 2012 (Columbia PDF) | https://stat.columbia.edu/~gelman/research/published/multiple2f.pdf | research | 2026-11-09 (gap fill) | Partial pooling; Type S and Type M |
+| Lee & Shen 2018 (DOI; abstract via Semantic Scholar) | https://doi.org/10.1145/3219819.3219905 | research | 2026-11-09 (gap fill) | Winner's curse correction |
+| A/B Testing with Fat Tails (IDEAS) | https://ideas.repec.org/a/ucp/jpolec/doi10.1086-710607.html | research | 2026-11-09 (gap fill) | Lean strategy under fat tails |
+| Golden & Horton 2021 (IDEAS) | https://ideas.repec.org/a/inm/ormnsc/v67y2021i1p342-362.html | research | 2026-11-09 (gap fill) | 63% of the nonexperimental loss |
+| Coviello, Gneezy & Goette 2017 (EconPapers) | https://econpapers.repec.org/RePEc:ces:ceswps:_6684 | research | 2026-11-09 (gap fill) | Edmunds: more than half of paid traffic lost |
+| Ghost Ads (NBER conference PDF) | https://conference.nber.org/conf_papers/f84173.pdf | research | 2026-11-09 (gap fill) | ITT, TOT, PSA invalidity |
+| Johnson, Inferno (Crossref) | https://doi.org/10.1111/jems.12513 | research | 2026-11-09 (gap fill) | Abstract: tiny effects; algorithmic exposure; spillovers |
+
+**Unreachable**
+- Amazon Ads help centre, Traffic validation (`https://advertising.amazon.com/help/GBYPYH79NGJJ5JPS`): the first pass found a script shell. Gap fill 2026-11-09: the article JSON embedded in the same page's HTML was read (§C), so it is no longer unreachable.
+- A change-history guide page on the docs markdown host: `guides/change-history/overview`, `guides/change-history/get-started`, `reference/change-history`, `change-history/overview`, `guides/history/overview` all answered 404. Only the OpenAPI model was read. The model host also answered 403 for `ChangeHistory_prod_3p.json`, `History_prod_3p.json`, `SponsoredBrands_v4_prod_3p.json`, `BrandLift_prod_3p.json`, `Experiments_prod_3p.json`.
+- Amazon's 2020 History blog post: its URL now lands on a general Sponsored ads page.
+- Springer (Kohavi et al. 2009), ACM Digital Library (Kohavi, Deng & Vermeer 2022; Deng et al. 2013 CUPED), SSRN (Lewis & Rao; Gordon et al.; Johnson, Lewis & Nubbemeyer "Ghost Ads"): bot challenge or HTTP 403. Abstracts were read through IDEAS or Semantic Scholar where served. Gap fill 2026-11-09: Ghost Ads was read from the NBER conference copy and CUPED from Microsoft's ExP site (Published research).
+- `ai.stanford.edu` PDFs of Kohavi's papers: no connection. Gap fill 2026-11-09: `exp-platform.com` answered, and CUPED, "A Dirty Dozen" and "Five puzzling outcomes" were read there; Kohavi et al. 2009 was still not found there.
+- Oxford Academic QJE page for Lewis & Rao: navigation only; the IDEAS record was used.
+- Project Euclid page for Brown, Cai & DasGupta: empty page; metadata via Semantic Scholar.
+- `research.google` publication page for Hohnhold et al.: 404; the static PDF was read.
+- SellerMetrics "Amazon Bid Optimization: How Often Should I Change Keyword Bids?": bot-verification page.
+- Search Engine Land "The data behind incrementality on Amazon" (redirected from martech.org): HTTP 403.
+- WebSearch: the first pass's search budget ran out mid-research. The gap fill 2026-11-09 searched again (What practitioners add).
+- Gap fill 2026-11-09:
+  - Pacvue's own pages (blog and product pages): HTTP 403. Its press release on GlobeNewswire was read instead.
+  - Helium 10 knowledge base HTML: HTTP 403. The same article was read through the help centre's public article API.
+  - Cambridge University Press pages and excerpts for Kohavi, Tang & Xu (2020): HTTP 403 or 503, or no connection. Only the bibliographic record was read.
+  - Search Engine Land, Teikametrics' Andrew Waber on Amazon incrementality: HTTP 403 (as in the first pass).
+  - amazon.jobs listings that describe Weblab: HTTP 404. Weblab is named only in job aggregators found by search, so it is not cited as Amazon's own description.
+  - Blake & Coey (EC 2014): the ACM abstract is elided in both Crossref and Semantic Scholar, and ACM answered a bot challenge. Only Holtz & Aral's account of it was read.
+  - Semantic Scholar API: HTTP 429 on several lookups. Crossref, IDEAS and authors' own copies were used.
+  - SSRN full text of Johnson's "Inferno": not read. The Crossref abstract was used.
+  - Intentwise help article on Change History: `https://help.intentwise.com/en/articles/7018074` and its slugged forms answered 404. Only its search-result summary was read.
+- Verifier, 2026-11-09: `ai.stanford.edu` still refused the TLS connection for Kohavi et al. 2009, and the `exp-platform.com` and `robotics.stanford.edu` copies answered 404, so the 16σ²/Δ², "at least a week or two" and OEC quotes remain secondhand from cx-listing-playbook's effect-measurement (read there 2026-10-14). The Amazon help article "Understand Sponsored Products off-Amazon advertising" was not opened (help centre pages are script shells); its 2026-08-04 revision is known only through PPC Land. The Semantic Scholar API rate-limited (HTTP 429) one lookup; Crossref was used instead.
+
+## Open questions
+
+1. **Does any Amazon tool give a seller a randomized holdout for Sponsored Products?** No page read documents one: MYE is listing content, Brand Lift excludes SP, Creative Testing is a DSP survey, third-party studies run in DSP, AMC documents analysis and audiences but no SP holdout on the pages read. Amazon itself runs "hundreds of thousands of RCTs across our ad products" to calibrate multi-touch attribution (arXiv:2508.08209), without saying whether any covers Sponsored Products or is visible to an advertiser. A search on 2026-11-09 found no unBoxed 2026 announcement of a Sponsored Products experiment tool. To settle: AMC's instructional-query and audience pages, and the Amazon Ads help centre's measurement pages (script-rendered here).
+2. **How are impressions split among several Sponsored Brands ads in one ad group?** Amazon says they can be used "to test" but not whether serving is random, even or optimized. To settle: the SB ads guide or help centre, or a first real two-ad ad group read by `adGroup`/`ad` impressions per day.
+3. **What happens when one advertiser's two campaigns target the same keyword for the same product?** Partly answered (gap fill 2026-11-09): for one keyword in several match types with different bids, "the match type with the highest bid will be used for any matching shopping query" (Understand keyword match types), and the targeting guide tells sellers with several campaigns on shared category keywords to "set bids based on your priority for each product". Still open: whether the rule reaches across ad groups and campaigns, which of two same-match-type copies serves, and whether the base bid or the placement-adjusted bid is compared. To settle: an Amazon page that states the scope. `areas/keyword-and-product-targeting.md` owns the rule.
+4. **Is a daily report row's `keywordBid`, `campaignBudgetAmount` or `campaignBiddingStrategy` the value on that day or at report time?** Not stated on the columns page. To settle: a report spanning a known change, read against `POST /history`.
+5. **Which change types does `POST /history` actually return?** The model's enum and its own description table differ (§A). To settle: a first real reply.
+6. **Does the history log Amazon-made changes**: base-bid moves under rule-based bidding, budget increases under budget rules, applied recommendations, the rule's own disabling? It does not say who made a change; it does not say whether Amazon's automatic changes appear. To settle: a campaign with a rule, read through `POST /history`.
+7. **When is a window final?** 3 days (clicks), 30 (IVT), 42 (FAQ example), 60 (overview), 70 (Stream v2). The pages disagree; whether the v1 Reporting API restates on Stream's 70-day schedule is not stated. To settle: Amazon's v1 reporting pages on restatement, or two pulls of the same window weeks apart.
+8. **What is the dynamic-bidding ceiling outside top of search?** Settled for this file (gap fill 2026-11-09) by `areas/bids-and-placements.md` P-bids-and-placements-8. Four current Amazon sources say up to 100% for all placements under up and down: the help page *Bidding strategies for Sponsored Products* (`lastModified` 2026-09-28), the up-and-down library guide, and the SP 3.0 and v1 models. The author's guide is the one Amazon page that still says 50% for placements other than top of search. No Amazon page dates a change. Still open: whether the author's guide is stale or describes a different programme. To settle: an Amazon release note on the ceiling.
+9. **Practitioners not reached.** Partly answered (gap fill 2026-11-09): Skai, Pacvue (press release only), Teikametrics, Intentwise, Helium 10, Perpetua, Optmyzr, Pattern, Flywheel and AdExchanger were read (What practitioners add). Still not reached: Quartile, Scale Insights, m19, Canopy, Pacvue's own pages, and Teikametrics' Search Engine Land piece. To settle: those vendors' public method pages.
+10. **Amazon Science.** Answered (gap fill 2026-11-09). amazon.science holds Qin (2023) on carry-over by ad product, Pauwels et al. (2023) on SB video, AuctionGym (2022/2023), Chen, Nabi & Siniscalchi (2023), Han et al. (2022) and Cooprider & Nassiri's blog on switchback price experiments (Published research). None describes a test a seller can run on Sponsored Products. The question that remains is Open question 1.
+11. **Brand Lift minimums.** Amazon says "spend and impression minimums" without figures. To settle: the Studies page in the console, or the Brand Lift API's eligibility reply.
+12. **Reporting v1 limits: per client ID or per advertiser?** The limits page says "Reporting API v1 applies its rate and concurrency limits per client ID" and "Limits are not applied per advertiser account", and the Reporting v1 overview says "one concurrent report per client ID, not per advertiser account"; the Amazon Ads reporting overview says "one concurrent report per advertiser" and so does the Stream v2 overview, "(one concurrent report per advertiser, approximately 100 reports per day)" (corrected in the gap fill 2026-11-09; the first pass wrote that it states neither). The pages split two (overview, Stream v2 overview) to two (v1 overview, limits page), and the two v1-specific pages say per client ID. Under cx's shared client, which applies changes how many reports a measurement can pull in a day. `method/reporting.md` owns the answer.
+13. **Do Sponsored Brands DCO headline changes, auto-collection product swaps or prompts appear in any change record?** None is an event type in the history model. To settle: SB creative and prompts reports over a window.
+14. **What is the "experimentation" store traffic in AMC?** The Brand Store insights paid dataset's `ingress_type` lists "7 through 11 - experimentation" with no definition. It may be Amazon's own Store experiments or a brand-facing tool; no page read says which. To settle: an AMC or Stores page defining the values.
+15. **Does a Sponsored Products change move organic rank on Amazon over weeks?** Qin (2023) finds no significant SP carry-over past three weeks in brand-level revenue. Ju, Zhao & Aral found a rank-mediated loss building over weeks in app stores. No Amazon page read quantifies the ad-to-organic-rank channel for Sponsored Products. To settle: `areas/ads-organic-and-total.md`'s sources, or a seller's switchback with a long enough period.
+16. **Is there an AMC incrementality test-group program for sponsored ads?** AdExchanger (2025-04-25) reports one in beta. No Amazon page read documents it. To settle: AMC's documentation pages on audiences and experiments.
+
+## Positions
+
+1. **P-change-measurement-1.** `POST /history` returns change events for Sponsored Products and Sponsored Brands campaigns only; "Changes for Sponsored Display campaigns are not returned", and "it doesn't return information about who made the change". (Changehistory_prod_3p.json [official])
+2. **P-change-measurement-2.** A history event carries `entityType`, `entityId`, `changeType`, `previousValue`, `newValue` (both strings), a UTC epoch `timestamp` and `metadata`; the request's `fromDate` reaches back at most the number of days the model states (90 on 2026-11-09), and `THEME` events need `Accept: application/vnd.historyresponse.v1.1+json` or the call answers 400. (Changehistory_prod_3p.json [official])
+3. **P-change-measurement-3.** The history model's bidding-strategy values (`OPTIMIZE_FOR_SALES`, `LEGACY`, `MANUAL`) and placement positions (`DETAIL_PAGE`, `TOP_OF_SEARCH`) differ from Sponsored Products 3.0's (`AUTO_FOR_SALES`, `LEGACY_FOR_SALES`, `MANUAL`, `RULE_BASED`, `OTHER`; `PLACEMENT_TOP`, `PLACEMENT_PRODUCT_PAGE`, `PLACEMENT_REST_OF_SEARCH`, `SITE_AMAZON_BUSINESS`), and the model has no event type for negative product targets, audience or shopper bid adjustments, ad creative or budget-rule associations. (Changehistory_prod_3p.json; SponsoredProducts_prod_3p.json [official])
+4. **P-change-measurement-4.** An Ads API v1 update answers `207` with `success`, `partialSuccess` and `error` entries keyed by the request's `index`, and a successful entry returns the object with its `lastUpdatedDateTime`; Sponsored Products 3.0 objects carry `extendedData.lastUpdateDateTime` and `servingStatus`. (AmazonAdsAPIALLMerged_prod_3p.json; SponsoredProducts_prod_3p.json [official])
+5. **P-change-measurement-5.** None of Amazon's change history, write replies or report rows has a field for the reason for a change, its expected effect, the metric it is judged on or its windows. (Changehistory_prod_3p.json; AmazonAdsAPIALLMerged_prod_3p.json; Reporting columns (version 3) [official, by the fields they define])
+6. **P-change-measurement-6.** Ad-attributed conversions are reported on the date of the ad interaction ("traffic date"), counted within a look-back that Amazon sets by ad product and account type (Sponsored Products: 7 days for sellers, 14 for vendors; other campaign types 14 on 2026-11-09). (Overview of the Reporting v1 API; Reporting columns (version 3) [official])
+7. **P-change-measurement-7.** Amazon states that impression and click data can change for up to three days through traffic validation, that invalid-traffic detection looks back up to 30 days, and recommends "waiting at least 5 days for the most accurate click data to be available before evaluating campaign performance". (Sponsored ads reporting FAQ (version 3); Amazon Ads reporting overview [official])
+8. **P-change-measurement-8.** Amazon's pages give different horizons for restated conversions: restatements "1, 7, and 28 days after the conversion event", up to 42 days for a 14-day window (FAQ); "subject to change for up to 60 days" (overview); a 70-day restatement window for Stream v2's `ads-performance-v1`, with "Treat a record as final only after its window has passed." (Sponsored ads reporting FAQ (version 3); Amazon Ads reporting overview; Stream v2 datasets overview; Ads performance [Stream v2] [official])
+9. **P-change-measurement-9.** Ad report days are cut in each campaign's time zone (`timeZone.value`), while history timestamps are UTC epoch. (Dates and historical data retention; Changehistory_prod_3p.json [official])
+10. **P-change-measurement-10.** Amazon's instruction for derived metrics is to "aggregate the numerator and the denominator across all rows at your grain before dividing". (Ads performance [Stream v2] [official])
+11. **P-change-measurement-11.** v1 base conversion metrics include promoted and halo conversions, promoted-only figures carry the `Promoted` qualifier, and Sponsored Products and Sponsored Brands CPC campaigns "will continue to report 0 for conversions from views". (Overview of the Reporting v1 API [official])
+12. **P-change-measurement-12.** Under `AUTO_FOR_SALES` Amazon raises or lowers the bid in real time "by a maximum of 100%" (SP 3.0 model). The current help page says "up to 100% for all placements", as does the up-and-down guide. The author's guide is the dissenting Amazon page: up to 100% at the top of the first page and up to 50% elsewhere. No Amazon page dates a change. Under `MANUAL` the bid "is not subject to dynamic bidding". (SponsoredProducts_prod_3p.json; Bidding strategies for Sponsored Products, GCU2BUWJH2W3A8Z7; Guide to dynamic bidding - up and down; The author's guide to Sponsored Products [official]; aligned with P-bids-and-placements-8)
+13. **P-change-measurement-13.** Under rule-based bidding Amazon adjusts base bids against a ROAS guardrail based on the previous 21 days, and disables the rule and restores the previous strategy and bids when the last 20 days' ROAS falls below 30% of the 20-to-40-day ROAS. (Sponsored Products rule-based bidding overview [official])
+14. **P-change-measurement-14.** Budget rules raise a campaign's daily budget on scheduled dates or when performance thresholds are met, measured on the previous 7 days for sellers and 14 for vendors; v3 `spCampaigns` records `campaignRuleBasedBudgetAmount` and `campaignApplicableBudgetRuleId`. (Sponsored ads budget rules overview; Reporting columns (version 3) [official])
+15. **P-change-measurement-15.** Sponsored Products ads appear only while the advertised product is in stock and the featured offer; the product ad's `servingStatus` names the cause (`AD_NOT_IN_BUYBOX`, `AD_OUT_OF_STOCK`, `NO_INVENTORY`, and others), and v1 objects carry `deliveryStatus` with `deliveryReasons`. (Best practices for your Sponsored Products ads; SponsoredProducts_prod_3p.json; AmazonAdsAPIALLMerged_prod_3p.json [official])
+16. **P-change-measurement-16.** Amazon changes where and how ads serve without a seller write: campaigns were "automatically enrolled in prompts" charged as CPC (US GA 2026-03-25), Sponsored Brands headlines can be rewritten under DCO consent, automatic collections choose products per shopper, and Amazon's Ads Agent announcement (2026-09-29) says Sponsored Products recommendations for targeting, bids and budgets can be applied "in one click", with "All of these capabilities" rolling out "over the coming months". (Sponsored Products prompts and Sponsored Brands prompts; All releases; Amazon Ads Agent announcement [official])
+17. **P-change-measurement-17.** A v1 recommendation carries `state` (`ACTIVE`, `APPLIED`, `DEFERRED`, `DISMISSED`, `EXPIRED`, `INACTIVE`) and `potentialImpacts`, "Estimated customer benefits if recommendation is followed"; Amazon's theme-based bid suggestion impacts "are not estimates of expected performance". (AmazonAdsAPIALLMerged_prod_3p.json; Recommendations API overview; Theme-based bid suggestions - Quick-start guide [official])
+18. **P-change-measurement-18.** Amazon Ads Status publishes six months of incidents by region and by category and subcategory (including `Ad delivery`, `Campaign management` and `Reports`), with start and end timestamps, at `status.ads.amazon.com/history.json`. (Amazon Ads Status; history.schema.json [official])
+19. **P-change-measurement-19.** Manage Your Experiments tests product images, titles, bullet points, descriptions and A+ Content; bids, budgets, targeting, ad creative, price and coupons are not among its experiment types. (Test listing content with Manage Your Experiments [official])
+20. **P-change-measurement-20.** Amazon Brand Lift surveys an ad-exposed group and an unexposed control group and reports absolute lift in awareness, favorability, intent and similar measures; Sponsored Products is excluded, studies are free above unstated spend and impression minimums, and results take "as few as 10 business days". (Brand Lift: Study the impact of your advertising; Measure brand marketing impact using Amazon Brand Lift [official])
+21. **P-change-measurement-21.** Amazon's Creative Testing is a pre-launch panel survey on the Amazon DSP Studies page in the United States, not a live split of shoppers. (Identify the right creative for an upcoming brand campaign with Creative Testing [official]; SalesDuo, Borle 2026 [opinion], agreeing)
+22. **P-change-measurement-22.** Sponsored Brands version 4 lets several ads share one ad group so that advertisers "can test different versions of the same ad formats within the same ad group"; no Amazon page read states how impressions are divided among them. (Sponsored Brands v4 features; All releases [official])
+23. **P-change-measurement-23.** AMC returns only aggregated results under per-column thresholds (at least 100 distinct users per row for medium- and high-threshold columns), is open to sponsored ads accounts by self-service, and since 2026-10-20 separates promotional purchases through 13 Paid Features columns; no AMC page read documents a randomized holdout for Sponsored Products. (How AMC works; Data aggregation thresholds in AMC; All releases [official])
+24. **P-change-measurement-24.** Long-term sales is a model estimate of 12-month new-to-brand value plus the campaign's 14-day sales; since 2026-09-25 (US) v1 reports carry `metric.cohortAccumulatedSales*` so estimates can be compared with actual cohort sales, and cohort metrics "aren't summable across cohorts". (All releases; Reporting columns (version 3) [official])
+25. **P-change-measurement-25.** Amazon's benchmark metrics give the 25th, 50th and 75th percentiles of peer brands in a category for CPC, CTR, CPM, ROAS, purchase rate and others, as series separate from the seller's own campaigns. (All releases, "Benchmark metrics and dimensions now available in Unified Reporting" and "Benchmarks reporting now generally available worldwide" [official])
+26. **P-change-measurement-26.** In the v1 model, Sponsored Products and Sponsored Brands support keyword, product, product-category and theme targets but no location, daypart, device or audience targets, which are DSP-only. (Target entity guide [official])
+27. **P-change-measurement-27.** Amazon recommends limiting changes while testing bidding strategies "so that you can attribute the difference in performance to a specific change", reviewing bids "about once every two weeks", and judging a negative keyword "after it has received at least 20 clicks" and after "two weeks or more"; no Amazon page read states a statistical test or confidence level for an ad change. (Guide to dynamic bidding - up and down; The author's guide to Sponsored Products; A guide to targeting with Sponsored Products [official])
+28. **P-change-measurement-28.** In eBay's field experiments, halting brand-keyword ads on Yahoo! and MSN while Google ran unchanged showed 99.5% of the forgone paid clicks captured by organic clicks, and switching off non-brand ads in a random 30% of US DMAs for 60 days showed returns that were "a fraction of conventional non-experimental estimates". (Blake, Nosko & Tadelis 2015 [research])
+29. **P-change-measurement-29.** In Bing experiments with data from thousands of brands, brand ads had "a positive, statistically significant impact" of 1%–4% when no competitor was present, while competitors took 18%–42% of clicks from brands that did not advertise on their own name. (Simonov, Nosko & Rao 2018 [research])
+30. **P-change-measurement-30.** A platform experiment that blocked sponsored search for 3% of an e-commerce site's visitors found that sponsored search "steals business from organic listings" and reduced total transactions; a marketplace field study found shoppers preferred top-ranked organic listings to sponsored listings of the same product and position. (Moshary 2025; Joo, Shi & Abhishek 2024 [research])
+31. **P-change-measurement-31.** Across 25 large retail ad experiments the median confidence interval on ROI was over 100 percentage points wide, and across 15 Facebook experiments observational methods "often fail to produce the same effects as the randomized experiments". (Lewis & Rao 2015; Gordon, Zettelmeyer, Bhargava & Chapsky 2019 [research])
+32. **P-change-measurement-32.** Outcome differences between ad positions won by bidding are biased by selection, and the bias "cannot be eliminated by randomizing the bids for the focal advertiser" because competitors bid too. (Narayanan & Kalyanam 2015 [research])
+33. **P-change-measurement-33.** When test and control arms share budgets or auctions, the measured effect of a bid change can be "severely biased" unless separate budgets are kept for each arm. (Basse, Soufiani & Lambert 2016; Liu, Mao & Kang 2020 [research])
+34. **P-change-measurement-34.** Ordinary p-values and confidence intervals "are wholly unreliable if users endogenously choose samples sizes by continuously monitoring their tests". (Johari, Pekelis & Walsh 2015 [research])
+35. **P-change-measurement-35.** Short-term ad effects are "not always predictive of the long-term effect" and user learning about ads took months to develop on Google search; an ad shutoff for a mobile game reduced organic installs by 20–30% through store rankings. (Hohnhold, O'Brien & Tang 2015; Ju, Zhao & Aral 2025 [research])
+36. **P-change-measurement-36.** At the sample sizes of single keywords, the normal-approximation interval for a conversion rate has poor coverage; the Wilson and Jeffreys intervals perform better, a zero count has a 95% upper bound of about 3/n (a good approximation when n is greater than 30), and ratio metrics such as CPC and ACOS need delta-method variance. (Brown, Cai & DasGupta 2001; Deng, Knoblich & Lu 2018 [research]; Wikipedia reference pages read 2026-11-09)
+37. **P-change-measurement-37.** Units picked because their last measurement was extreme tend to move back toward their mean with no intervention. (Barnett, van der Pols & Dobson 2005 [research])
+38. **P-change-measurement-38.** In cx-listing-playbook's `effect-review`, *held* means the targeted metric moved, after the control's movement is taken out, in the expected direction by at least the smallest move logged before the change, with orders not falling and no confounder or event explaining it; *not held* means the window held enough data to have seen that move and did not, or the metric moved the wrong way; *cannot tell* is every other case. (cx-listing-playbook `research/method/effect-measurement.md`, P-effect-measurement-18 [ComChatX research, definition])
+39. **P-change-measurement-39.** Practitioners disagree on how long to wait after a bid change: Ad Badger's automated bidder "typically allow[s] a 24-hour window", Ad Badger's blog (after Clément Hynaux on its PPC Den podcast) says "at least seven days of data", Pare's George Reid says "7–14 days", BidX's Selina Krüger waits until the seventh day after the observation period when the gap to target ACoS is small; Ad Badger's August 2026 attribution article states a 14-day Sponsored Products window and order-date reporting, which Amazon's v1 overview contradicts and Ad Badger's own June 2026 post does not share. (Overview of the Reporting v1 API [official]; Ad Badger 2026; Pare, Reid 2026; BidX, Krüger 2021 [opinion])
+40. **P-change-measurement-40.** Eva Commerce's Hai Mag proposes a geo holdout for branded Sponsored Products and says "In many cases, 80-90% of traffic and sales that would have been attributed to branded ads return through organic channels when the paid ad is removed", without a source; Perpetua's Apoorva Ashok Kumar measures category-term incrementality "using the Total ACoS". (Target entity guide [official], on SP geography; Eva Commerce, Mag 2026; Perpetua, Ashok Kumar 2024 [opinion])
+41. **P-change-measurement-41.** Seller Labs' author "Denis D" states that below a few hundred sessions a month conversion "swings on single orders", and uses 60- or 90-day windows or skips such ASINs; Amazon's targeting guide puts its own figure at 20 clicks for judging a negative. (A guide to targeting with Sponsored Products [official]; Seller Labs, Denis D 2026 [opinion])
+42. **P-change-measurement-42.** Amazon's published lift figures for its own automation come from internal testing or beta data: Sponsored Brands DCO's "up to 1.6% CTR lift" is "Based on internal testing", and Full-Funnel Campaigns' "67% higher Long-Term ROAS" is footnoted "against the blended average of other campaign performance across beta advertisers. Amazon internal (June – August 2026)"; Improvado's Ivo Petrovic reads the latter as "not a holdout test on your account". (All releases; Grow total sales with Full-Funnel Campaigns [official]; Improvado, Petrovic 2026 [opinion])
+43. **P-change-measurement-43.** Amazon's multi-touch attribution (beta, US) "divides credit for Amazon purchase conversions across Amazon Ads touchpoints in proportion to their value" using "insights from experiments, machine learning, and historical shopping signals", beside the standard last-touch metrics; Amazon's authors describe a database of "hundreds of thousands of RCTs across our ad products" behind it and do not describe advertiser access to those RCTs. (Uncover ad value across the funnel with multi-touch attribution (beta) [official]; Lewis, Zettelmeyer, Gordon et al. 2025 [research, Amazon-authored])
+44. **P-change-measurement-44.** Sponsored Products campaigns carry `offAmazonSettings.offAmazonBudgetControlStrategy` (`MAXIMIZE_REACH`, `MINIMIZE_SPEND`), bulksheets gained an "Off-Amazon ad serving" column on 2026-06-08, and PPC Land's Luis Rijo reports that from 2026-08-10 Sponsored Products ads may also serve through Amazon Influencer Program creators, enrolled by default at existing bids and budgets. (SponsoredProducts_prod_3p.json; All releases [official]; PPC Land, Rijo 2026 [opinion, trade press])
+45. **P-change-measurement-45.** Amazon monitors traffic "for up to 30 days after it occurs", and invalid traffic found in that window "is automatically removed from invoices and advertiser reports". If an investigation finds invalid traffic that was missed, "affected metrics and invoices may be restated". Click and impression invalidation rates are in the Sponsored Ads Gross and Invalid Traffic report. (Traffic validation, GBYPYH79NGJJ5JPS [official])
+46. **P-change-measurement-46.** SP 3.0 list calls return `extendedData` when `includeExtendedDataFields` is true. The flag's description names "creationDate, lastUpdateDate, servingStatus", while the returned schemas hold `creationDateTime`, `lastUpdateDateTime`, `servingStatus` and `servingStatusDetails`. (SponsoredProducts_prod_3p.json [official])
+47. **P-change-measurement-47.** The Reporting v1 dimension library defines `targeting.value` ("The keyword, product or category you choose to reach shoppers."), `targetingMatchType.value` ("The type of matching for keywords used in a bid.") and `adGroupTarget.id`, the target's primary key. Amazon's own v1 examples write `target.value` and `target.matchType`. The New features page's sample warning flags `"field": "target.value"` as `PENDING_DEPRECATION`. (Dimensions library [Reporting v1]; Targeting and Target dimension pages; New features [Reporting v1] [official])
+48. **P-change-measurement-48.** Amazon's launch page places multi-touch metrics in "most report types for version 3 of the Amazon Ads API". The *Reporting columns (version 3)* page has no multi-touch column. Reporting v1's *Purchases* page defines 56 `metric.multiTouch*` fields, each with "Version 3 Reporting Name (Sponsored Ads): N/A". (Uncover ad value across the funnel with multi-touch attribution (beta); Reporting columns (version 3); Purchases (v1) [official])
+49. **P-change-measurement-49.** The Measurement OpenAPI model defines Brand Lift as DSP operations: `POST /dsp/measurement/eligibility/brandLift`, `GET`, `POST` and `PUT /dsp/measurement/studies/brandLift`, and `GET /measurement/studies/brandLift/{studyId}/result`, authorised by "DSP Rodeo Entity ID, DSP Advertiser Account ID". The `/measurementStudies/brandLift` paths named on Amazon's 2024 sponsored-ads launch page are not in the model. (Measurement_prod_3p.json; Measure brand marketing impact using Amazon Brand Lift; All releases [official])
+50. **P-change-measurement-50.** "Amazon DSP provides 10 Customer holdout groups for A/B testing. Each group contains 10% of randomly assigned Amazon customers". The same page says: "We do not recommend comparing performance before and after implementing bid adjustments as market conditions and other factors vary over time." No Ads API docs page found by full-text search documents a holdout for sponsored ads. (How to test DSP bid adjustments [official]; Ads API docs search index, searched 2026-11-09)
+51. **P-change-measurement-51.** No Amazon page read offers a randomized shopper split for a Sponsored Brands creative or a Store. Sponsored Brands lets "Brands … test different creatives within the same ad group", with no stated split. Manage Your Experiments covers A+ Brand Story content. An Amazon-hosted partner case study calls a comparison of two campaigns "With comparable impression volumes" an A/B test. (SB v3-to-v4 migration guide; Seller Forums EU, News_Amazon; Trellis helps lift Dandy Blend's results with AI creative [official])
+52. **P-change-measurement-52.** "If you set multiple match types with different bids for the same keyword, then the match type with the highest bid will be used for any matching shopping query". Amazon's targeting guide says "you won't be bidding against yourself by adding different match types on the same keyword". No page read states whether the rule spans ad groups or campaigns, or how same-match-type copies are chosen. (Understand keyword match types, GHTRFDZRJPW6764R; A guide to targeting with Sponsored Products [official])
+53. **P-change-measurement-53.** Amazon's reserves "influence your final CPC or vCPM" by factors that include placement value, which "may have a different value during a peak shopping period than a non-peak period". Amazon describes soft reserve prices as "a real-time minimum value". A winner whose bid exceeds the hard reserve but not the soft reserve "pay[s] their bid". (Understand Amazon Ads auctions, GEXGZNBBWPZZK7P8; Amazon's response to the FTC's lawsuit regarding Sponsored Ads [official])
+54. **P-change-measurement-54.** Amazon states it is "continually experimenting to refine and improve our ad auctions", including "introducing, removing, or modifying the data points or factors used in the auction process". Amazon-authored product-search papers report online A/B tests on 50 million and 140 million queries. They also report a ranking change "launched in production" that "has served customers since 2025". (Understand Amazon Ads auctions [official]; Han et al. 2022; Gupta et al. 2020, as read in `areas/launch.md`; Dong et al. 2026 [research, Amazon-authored])
+55. **P-change-measurement-55.** Amazon reports two figures. "eighty percent of bid changes on clicked Sponsored Products search ads occurred within one day of a prior change" (2019–2024). In one study, after an advertiser stopped bidding on specific keywords, competitors' average bids on them "dropped about 83%, from $15.96 to $2.74". (Amazon's response to the FTC's lawsuit regarding Sponsored Ads [official, Amazon's figures; for human readers only])
+56. **P-change-measurement-56.** Amazon Ads' Qin (2023) studied 15 US brands that asked for a carry-over study, with weekly data from 2021–22 and a VARMAX impulse response over seven weeks. She found that "SP does not show significant carry-over effects beyond three weeks for any brand", and that lower-funnel products realise 60–90% of their total effect within two weeks. The paper states the sample is "only … brands that asked for a long-term effects assessment" and that the estimate nets out "company feedback effects and competitive response". (Qin 2023 [research, Amazon-authored])
+57. **P-change-measurement-57.** Lily Tong of Amazon Ads, as reported by Relevant Audience from Digiday, said that across "more than 2 million campaigns" in June, "roughly 20% of the sales value for a brand shows up over time and isn't captured in that immediate attribution window". The figure covers all ad products and is not split out for Sponsored Products. (Relevant Audience, Antonio Fernandez, 2026-10-02; Digiday, Seb Joseph, 2026-10-01, as read in `beyond/metrics-and-attribution.md` [opinion, trade press quoting Amazon])
+58. **P-change-measurement-58.** Jonathan Hershaff reports a daily on/off switchback for about a month, by a seller with $20k annual ad spend. Platform ROAS was about 2.35 and incremental ROAS from total sales 1.25, rising to 1.54 after an MMM-estimated adstock of about 0.35 per day. He calls the 1.25 a "floor" because "A daily switchback assumes marketing has zero memory". (Hershaff, Substack, 2026-02-13 [opinion, seller-analyst])
+59. **P-change-measurement-59.** In Bojinov, Simchi-Levi & Zhao's switchback design, under carryover of order *m*, optimal assignment probabilities are 1/2 at randomization points {1, 2*m*+1, 3*m*+1, …, (*n*−2)*m*+1}. The Horvitz-Thompson estimator is unbiased for the lag-*p* effect when *p* ≥ *m* and biased when *p* < *m*. The authors suggest "each period's length be smaller than the carryover effect duration". (Bojinov, Simchi-Levi & Zhao 2023 [research])
+60. **P-change-measurement-60.** Amazon's pricing scientists cannot split shoppers for a price test, so they use switchback experiments. Their "random-days" design "can reduce the standard error … by 60%". They name carryover through recommendations, which "could artificially boost the product's sales even during control periods". (Cooprider & Nassiri, amazon.science, 2023-04-14 [research, Amazon-authored])
+61. **P-change-measurement-61.** With treatment at different times, the two-way fixed-effects estimator "equals a weighted average of all possible two-group/two-period DD estimators". Its weights "may be negative", so "the linear regression coefficient may for instance be negative while all the ATEs are positive". In event studies, "apparent pretrends can arise solely from treatment effects heterogeneity". (Goodman-Bacon 2021; de Chaisemartin & D'Haultfœuille 2020; Sun & Abraham 2021 [research])
+62. **P-change-measurement-62.** Abadie's feasibility conditions for a synthetic control are these: an effect large against the outcome's volatility; a donor pool that did not adopt similar interventions; no anticipation; no interference; a close pre-period fit (the convex hull condition); and donors "chosen judiciously" to avoid "large interpolation biases". Inference is by placebo reassignment across donors. (Abadie 2021 [research])
+63. **P-change-measurement-63.** CUPED cuts the variance of a treatment-control difference to var(Y)(1−ρ²) using a pre-period covariate, about 50% on Bing. The same metric in a pre-period of 1–2 weeks "works well", and "Never use covariates that could be affected by the treatment". (Deng, Xu, Kohavi & Walker 2013 [research])
+64. **P-change-measurement-64.** A sample ratio mismatch between arms "implies a severe selection bias, enough to render the experiment results invalid" most of the time. A chi-square test detects it. Carryover from earlier experiments on the same units lasted more than three weeks in Bing's case and was detected with A/A tests. (Dmitriev, Gupta, Kim & Vaz 2017; Kohavi et al. 2012 [research])
+65. **P-change-measurement-65.** In an Airbnb pricing meta-experiment, "at least 20% of the TATE estimate" from individual-level randomization was interference bias, which cluster randomization removed. In simulation, blocked cluster randomization cut the naive estimator's bias "by as much as 62%" at the cost of higher variance. (Holtz, Lobel, Lobel, Liskovich & Aral 2024; Holtz & Aral 2020 [research])
+66. **P-change-measurement-66.** Wager & Xu estimate marginal responses to system parameters "by combining unobstrusive randomization with lightweight modeling, all while remaining in equilibrium". They use ±1 shocks per unit, and the regression on them recovers the response "against a fixed ambient market equilibrium". (Wager & Xu 2021 [research])
+67. **P-change-measurement-67.** Amazon's AuctionGym authors state that "an advertiser does not observe counterfactual outcomes of bid amounts that were not submitted", and that adaptation by competitors complicates "evaluation of bidding strategies based on logged data alone". (Jeunen, Murphy & Allison, AuctionGym [research, Amazon-authored])
+68. **P-change-measurement-68.** The false discovery rate is "the expected proportion of falsely rejected hypotheses", and a step-up procedure controls it for independent tests. Multilevel models instead shrink estimates toward each other. Winners chosen from a group of experiments carry a selection bias, for which Lee & Shen give a correction. (Benjamini & Hochberg 1995; Gelman, Hill & Yajima 2012; Lee & Shen 2018 [research])
+69. **P-change-measurement-69.** Deng's objective Bayesian test "requires the existence of historical A/B test data to objectively learn a prior", built at Bing from "thousands of experiments". Azevedo et al. find that with fat-tailed effects "a "lean" strategy of trying more ideas, each with possibly smaller sample sizes, is preferred". (Deng 2015; Azevedo, Deng, Montiel Olea, Rao & Weyl 2020 [research])
+70. **P-change-measurement-70.** Against the eBay brand-term result, Coviello, Gneezy & Goette found at Edmunds.com that "More than half of the paid traffic is lost when we shut off paid-links search". In Golden & Horton's randomized geo pause, the firm "lost customers, but only 63% as many as a nonexperimental estimate would have suggested". (Coviello, Gneezy & Goette 2017; Golden & Horton 2021 [research])
+71. **P-change-measurement-71.** Intent-to-treat "differences the outcomes of the entire treatment group and the entire control group" and is valid but less precise than treatment-on-the-treated. "PSA experiments are rendered invalid when marketers use performance-optimizing computer algorithms to deliver ads". In display ads, "ad exposure is jointly determined by advertisers, users, algorithms, and market competition". (Johnson, Lewis & Nubbemeyer 2016; Johnson 2023 [research])
+72. **P-change-measurement-72.** Optmyzr pulled 14,991 campaigns' 14-day attributed figures for one day over 17 days. In the top 5% of campaigns, attributed sales grew "by at least 18.75%" and impressions changed "by at least 36.67%". Clicks and spend are described as "much more stable". (Optmyzr, Vallaeys, 2025-06-11 [opinion, vendor study; for human readers only])
+73. **P-change-measurement-73.** The vendors' own pages describe their methods as follows. Skai calls holdout testing the "gold standard" and computes "[incrementality score] x ROAS = iROAS". Pacvue's iROAS uses "multiple advanced linear regression data models". Intentwise calls its What Changed comparison "the starting point". Teikametrics updates Amazon bids "Hourly". Helium 10's change log labels a change's source, including "In Seller Central or 3rd party". (Skai, Weiss 2023 and 2024; Pacvue press release 2024-09-17; Intentwise, Waters 2024; Teikametrics help centre; Helium 10 knowledge base [opinion, vendors])
+74. **P-change-measurement-74.** AdExchanger's James Hercher reports that "The Amazon ad platform only recently began enabling incrementality tests". He also reports, per unnamed sources, that AMC incrementality test groups are in beta. No Amazon page read documents that program. (AdExchanger, 2025-04-25 [opinion, trade press])
+75. **P-change-measurement-75.** Pauwels et al.'s Sponsored Brands video estimates rest on matching and causal Gaussian processes over 25,364 advertisers for one month, and on propensity score stratification for one year. The brands chose to adopt; nothing was randomized. (Pauwels, Peran, Shah, Schnaidt & Vercamer 2023 [research, Amazon-authored])
+
+### Seat's positions (enter no skill)
+
+- **S1. The change record.** A record that makes a later verdict a test rather than a story holds, written before the write: the entity ids and the profile; the field's value read just before (from the object, not memory); the request as sent (method, path, body); Amazon's reply whole (the `207` body, or the door's `sent, outcome unknown`); the object read just after; the metric the change is judged on, with its Amazon field name and look-back; the expected direction and the smallest move that would count; the baseline and observation windows by date in the campaign's time zone; the date the observation window's figures are expected to be final under the horizon chosen from §C; the control series; and every other change in the same campaign, ad group or ASIN during both windows, from `POST /history` and the seller's word. This follows cx-listing-playbook's P-effect-measurement-6 with the ad-specific fields added.
+- **S2. Windows for an ad change.** Whole weeks, matching weekdays. The observation window starts the day after the change lands (its `lastUpdatedDateTime`). A verdict on conversion metrics waits until the observation window's last day is past the chosen restatement horizon; a verdict on impressions, clicks and spend can come after Amazon's click-validation period. Exposure (impressions, top-of-search impression share, clicks, CPC) answers within days for a bid or budget change; conversion and ACOS answer later and more noisily.
+- **S3. Amazon's automatic responses are part of the effect.** Under dynamic or rule-based bidding, what is measured is the change plus Amazon's response to it. A manual change by anyone else in the window is a confounder; an automatic bid move by Amazon is not. A rule's switch-off, a budget rule's event day, an applied recommendation and an Amazon Ads Status incident are confounders, because they would have happened without the change or are separate changes.
+- **S4. The on-off test on Amazon.** The design Blake et al. used for brand terms (ads off on some platforms, on in another) has two Amazon analogues a seller can run: alternating whole weeks on and off for the same term or campaign, and switching off in one marketplace while another runs. As usually run, neither is randomized. The first meets seasonality and competitor moves between weeks, and the second compares different markets. The first becomes a randomized switchback when a coin decides each period's arm (Bojinov, Simchi-Levi & Zhao 2023; Amazon's own random-days price tests). Under their design, the period is set against the carryover (Qin's up to three weeks for Sponsored Products in weekly brand data, against Hershaff's daily decay of about 0.35), and the periods right after a switch are dropped or modelled. The second is a donor comparison and needs Abadie's conditions (pre-period fit, no similar change in the donor, no spillover). In either, Amazon's competitors respond within days (Amazon's FTC response: rival bids fell about 83% when one advertiser stopped), so the test measures the pause together with the market's answer to it. The quantity to read is total sales of the advertised ASIN and brand (SP-API), not attributed ad sales, and organic signals (Search Query Performance share, Keepa rank), because the question is substitution.
+- **S5. Control series an ad change does not touch.** The brand's unadvertised ASINs, the same ASIN in another marketplace, the category benchmark CPC/CTR series, Search Query Performance query volume for the changed terms, and the seller's own other campaigns not sharing the budget, auctions or ASIN. Competitors' series are partly caused by the change (Johari et al. 2020).
+- **S6. Verdicts adapted to ads.** *Held* needs the change visible in its first-order metric (bid → CPC or impression share; budget → hours or days out of budget; negative → spend on the negated term; new target → impressions on it) and the judged metric past the logged smallest move once restatement has settled. *Not held* needs a window large enough, by the counts in the worked statistics, to have seen the logged move. *Cannot tell* covers a change that did not land, a change Amazon overrode (rule switch-off, out-of-budget days, Featured Offer loss, stock-out), an incident, a sale event, another change in the same entity, or too few events. For most single keywords, a conversion-rate or ACOS verdict will be *cannot tell* for weeks; exposure and spend verdicts will not.
+- **S7. Several changes at once.** When a seller makes a change set (a restructure, a harvest of twenty terms), the verdict is about the set, measured on totals of the ASINs it touches, and the record says so.
+- **S8. Peeking.** A heartbeat that reads a campaign daily reports progress and gives no verdict before the logged end date and the restatement horizon.
+- **S9. Many verdicts.** A heartbeat that issues verdicts on many changes is running many tests. The published corrections apply to it: false-discovery control across the batch, shrinkage of each estimate toward the seller's own history of similar changes, and a discount on kept winners (Benjamini & Hochberg; Gelman, Hill & Yajima; Lee & Shen; Deng 2015). The procedure's false-alarm rate on a given seller's data can be read by running it on windows with no change, the A/A analogue (Kohavi et al. 2012).
+- **S10. The unit is the change sent.** The contrast that stays valid is between everything assigned the change and everything not assigned, counting targets whose raise won nothing and changes that partly failed (Johnson, Lewis & Nubbemeyer's ITT). Restricting to targets that gained impressions conditions on Amazon's allocation.
+- **S11. Pooled before/after across staggered changes.** One regression over many targets changed on different dates, with target and date effects, is the two-way fixed-effects estimator. When effects differ, its coefficient can carry negative weights. The published estimators compare each changed unit only with not-yet-changed or never-changed units at the same dates, then aggregate (Callaway & Sant'Anna; Sun & Abraham).
+- **S12. Small randomized bid perturbations.** For a seller whose bid barely moves the market, small ± changes to base bids assigned at random across targets or days identify the local response of clicks and cost to the target's own bid (Wager & Xu's marginal response at a fixed equilibrium). The seller's logged bid history cannot identify it cleanly, because the seller chose those bids (AuctionGym). Under dynamic bidding the perturbation is on the base bid, and Amazon's adjustment sits on top of it.
+
+## Verification
+
+Verified 2026-11-09 by an adversarial verifier seat.
+
+**Method.** Every page in *Pages read* was re-opened on 2026-11-09 by plain GET. Ads API docs pages were read from their markdown sources on `d3a0d0y2hgofx6.cloudfront.net`; the OpenAPI models (Changehistory, v1 merged, Sponsored Products 3.0, Eligibility) and the MCP `campaign_management.json` specification were parsed from the docs model hosts and every quoted field, enum, description, path and operation id was checked against them. Library, What's new, Sell on Amazon and Seller Forums pages were read as HTML text, with page titles taken from the HTML. Practitioner pages were checked for each quote, author and date. Research claims were checked against IDEAS abstracts, arXiv abstracts, PMLR, the NBER abstract and the NBER and Google PDFs (text extracted in memory), Semantic Scholar and Crossref records, and Wikipedia's raw text. The worked statistics (Wilson, Jeffreys, 16σ²/Δ² sample sizes, Beta-posterior probabilities, Poisson relative errors) were recomputed and match. Searches were run for 2025–2026 Amazon announcements that might overturn a claim (a Sponsored Products experiment or holdout tool, change-history changes, unBoxed 2026 measurement launches). The verifier called no Ads API host (`advertising-api*.amazon.com`), no Login with Amazon endpoint, no Amazon Ads MCP server endpoint and no SP-API host; `status.ads.amazon.com`, GitHub (`raw.githubusercontent.com`, `api.github.com`) and the public docs hosts were the only Amazon-owned hosts read.
+
+**Held as written.** §A (every Changehistory field, limit, enum and the enum/table mismatch), §B shapes, §C quotes and figures (FAQ, reporting overview, Stream v2, v1 overview, dates and retention tables, columns, warnings dated 2026-10-05), §D bidding, rule-based bidding, budget rules, serving statuses (93 `DeliveryReason` values), prompts and Status, §E MYE, Brand Lift GA, AMC thresholds, LTS cohort metrics and benchmarks, §F Target support table, §G library-guide quotes, §I schema names and Keepa fields; Blake et al., Simonov et al., Moshary, Joo et al., Lewis & Rao, Gordon et al., Lewis & Reiley, Johnson et al., Narayanan & Kalyanam, Basse et al., Johari et al. (both), Waisman et al., Hohnhold et al., Brodersen et al., Deng et al., Dash et al.; Ad Badger (attribution article and help 196), Pare, Marin, ClearAds, SalesDuo, Perpetua, FiveX figures, the Seller Forums thread, PPC Land (2020). No 2025–2026 Amazon page found overturns a position: no Sponsored Products experiment tool was announced, and the change history model still excludes Sponsored Display and the actor.
+
+**Corrections (was → now).**
+1. §D, P-16: Ads Agent quote "AI guidance for Sponsored Products offers recommendations for targeting, bids, and budgets—and apply them in one click" (not on the page) → the page's verbatim sentence "AI-powered guidance for sponsored ads: Quickly identify opportunities …—and apply them in one click. All of these capabilities will roll out over the coming months."; P-16 now carries the roll-out wording.
+2. §E, P-42: Full-Funnel "67% higher Long-Term ROAS" sourced to Improvado [opinion] and, in P-42, to the Ads Agent announcement (which does not carry it) → sourced to Amazon's own launch page "Grow total sales with Full-Funnel Campaigns" [official] with its footnote verbatim; Improvado kept as a reading of the footnote; the Ads Agent page's own 25% / 10% claim added.
+3. §E: Brand Metrics quote "not just ad-attributed engagements" (not on the overview page) → "Awareness and Consideration indices that compare your performance to peers using models predictive of consideration and sales".
+4. §E, P-20, P-21, Pages read: page names "Amazon Brand Lift" → "Brand Lift: Study the impact of your advertising"; "Creative Testing" → "Identify the right creative for an upcoming brand campaign with Creative Testing"; "Conversational experience in Amazon Ads Agent" → "Get more done with natural conversation", and its markets "Closed beta US/CA/MX" → closed beta across North and South America, Europe, Middle East and Asia Pacific; Improvado's title as shown on the page.
+5. §E: Brand Lift availability "available to sponsored ads advertisers in the United States" → the full sentence, adding API and Ads Agent availability in eight countries.
+6. Practitioners, P-39: Ad Badger "You're Optimizing Too Much" quotes "Yesterday's data or today's data is just a rough draft" and the 48-hour rule attributed to Ad Badger → attributed to Clément Hynaux on Ad Badger's PPC Den podcast (post dated 2026-06-02); added that the post says sales are "attributed back to the original click date", contradicting Ad Badger's August article.
+7. Practitioners, P-39: BidX's seventh-day rule stated unconditionally → conditional, as on the page ("if the difference from the target ACoS on the day of observation is rather small").
+8. Practitioners: AdLabs had no author or date → Andrew Bailiff, 2025-09-03; its spend claim now quoted verbatim.
+9. Practitioners, P-41: "Denis Dmitriev" (not on the page) → the byline as shown, "Denis D"; "Compare against the same month last year when the season is the likely cause" (paraphrase in quotes) → "Compare the same month last year before you act."
+10. Practitioners, P-40: Eva quote "80-90% of branded-ad sales return through organic channels when the ads are paused" (paraphrase in quotes) → the page's sentence verbatim.
+11. Practitioners: FiveX "presented as illustrations" → labelled "Named example" with no source for figures; Sagum's own statement that its core expertise is Facebook, Instagram and TikTok added; PPC Land author Luis Rijo added.
+12. §C: "Totals stay complete" attributed to the Ads performance page → the Stream v2 datasets overview, with the sentence in full; promoted-qualifier sentence marked as a DSP-addressed warning.
+13. Run-time, Pages read: `GET /adsApi/v1/list/recommendations` given as the path → the guide's path and the v1 model's `GET /adsApi/v1/recommendations` (`ListRecommendation`) both recorded, the model to be read at run time.
+14. Open question 12: "the reporting overview and the Stream v2 overview say one concurrent report per advertiser" and limits quote "Not applied per advertiser account" → limits page and v1 overview say per client ID (verbatim), reporting overview says per advertiser, Stream v2 overview says neither.
+15. Open question 8: the model was quoted as "for all placements" → model "by a maximum of 100%" with no placement named; the author's guide is marked as written for book authors (also in §D and §G).
+16. §B: added that the v1 model's `ErrorsIndex.index` reads 0–19 while success indexes read 0–999.
+17. §D: the Status quote is the Status page's (also in the 2023-05-10 release note); live items carry `id` where the schema names `hashedId`.
+18. Research: Liu, Mao & Kang "KDD 2021" unverified → the KDD 2021 version's different title and DOI from Crossref; Simonov "experiments … across thousands of brands" → "with data from thousands of brands", and P-29 now quotes "a positive, statistically significant impact"; rule of three's "When n is greater than 30" caveat added (P-36).
+
+**Additions found by the verifier's search.** Amazon's multi-touch attribution (beta) and the Amazon-authored arXiv:2508.08209 on its RCTs (§E, Published research, P-43, Open questions 1 and 10); Self-Service Third Party Measurement Studies (DSP, 2026-05-15); off-Amazon serving (`offAmazonSettings`, the 2026-06-08 bulksheet column, and PPC Land's report of creator placements from 2026-08-10; §D, P-44); the unBoxed 2026 news page holds no sponsored-ads experiment tool.
+
+**Removals.** None. No position was worded as behaviour (when to ask, a fixed sequence, a threshold given as the rule, a ceiling, how cx talks); none was unsourced, so none moved to the seat's list.
+
+**Stayed unreachable.** Amazon Ads help centre Traffic validation (script shell); change-history guide pages (404) and the other model filenames (403); the 2020 History blog post (redirects to a general Sponsored ads page); Kohavi et al. 2009 full text (TLS refused; other copies 404); Springer, ACM, SSRN pages (bot challenges, not retried); the Amazon help article on off-Amazon advertising (not opened; known through PPC Land only).
+
+### Gap fill 2026-11-09
+
+This seat filled the gaps and settled the contradictions that completeness critics raised. Every page was read by plain GET on 2026-11-09. No Ads API, Login with Amazon, MCP, SP-API or signed-in page was called, and no seller's data was read.
+
+**Contradictions settled (10).**
+1. *Dynamic bidding ceiling (§D, Stable 9, P-12, Open question 8).* The file had presented this as an open disagreement. It now follows P-bids-and-placements-8. The help page GCU2BUWJH2W3A8Z7 (`lastModified` 2026-09-28), the up-and-down guide and the models say 100% for all placements, the author's guide dissents with 100/50, and no Amazon page dates a change. Other files that state a single ceiling (`areas/launch.md`, `areas/diagnosis.md`, the automation file) should either name the author's guide as the dissenting Amazon page or defer to P-bids-and-placements-8. This seat did not edit them.
+2. *Dynamic bidding ceiling, the second critic's wording of the same point.* This is the same settlement as item 1, applied to §G's references and the run-time "where the figures live" line (help page added).
+3. *`target.value` (run-time).* Was `target.value`. Now `adGroupTarget.id`, `targeting.value` and `targetingMatchType.value`, as the v1 dimension library defines them. Amazon's examples use `target.value`/`target.matchType`, and the New features warning example flags `target.value` `PENDING_DEPRECATION` (P-47). Aligned with `method/reporting.md`.
+4. *Extended data (§B, P-46).* Added that `extendedData` comes only with `includeExtendedDataFields: true`, whose description names `creationDate`/`lastUpdateDate` while the schemas return `creationDateTime`/`lastUpdateDateTime`. Both readings are in the model, which agrees with `method/ads-api-surface.md` §H and `areas/diagnosis.md` P-78.
+5. *Multi-touch fields (§E, run-time, P-48).* Was "No multi-touch column was found" and "no API column name was found". Now: 56 v1 `metric.multiTouch*` fields on the v1 Purchases page, none on the v3 columns page, and the launch page's "most report types for version 3" recorded as a third, unsupported reading. Aligned with `beyond/metrics-and-attribution.md` M1–M2.
+6. *Brand Lift paths (§E, run-time, P-49).* Was the 2023 release note's paths only. Now the Measurement model's DSP paths, the release note's one `PUT` difference, and the 2024 launch page's `/measurementStudies/…` paths, which are not in the model. A call is checked against the model. This file owns the paths, and `areas/ads-organic-and-total.md` G2 agrees.
+7. *Duplicate targets (§F, Open question 3, P-52).* Was "no page read says how Amazon chooses". Now Amazon's highest-bid match-type rule and the targeting guide's two sentences, with the rule's scope across campaigns and same-match-type copies stated as unknown, as in `areas/campaign-structure.md` P-56. The ClearAds/Sagum/Marin same-keyword A/B designs are now judged against the rule (§F).
+8. *Concurrency (Pages read, Open question 12).* Was "the Stream v2 overview states neither". Now it states "(one concurrent report per advertiser, approximately 100 reports per day)", so the split is two pages to two, with the v1-specific pages saying per client ID. `method/reporting.md` P-reporting-15 owns it. The automation file should carry the conflict rather than one side.
+9. *How long a Sponsored Products effect lasts (Stable 17, the weighing table, Ju transfer, P-56 to P-58).* P-35's long-horizon sources are now weighed against Qin 2023 (Amazon data: no significant SP carry-over past three weeks), Lily Tong's 20% (all ad products) and Hershaff's daily decay of 0.35. This aligns with `beyond/metrics-and-attribution.md`'s weighing, including its reading of Qin's two SP rows.
+10. *Does a pause read overstate or understate the loss? (Published research, P-70).* Golden & Horton found that a non-experimental estimate overstated a pause's loss (the experiment showed 63% of it). Ju et al. found a short pause understates a loss that builds through rank. Coviello et al. found large losses at Edmunds against eBay's near zero. P-28 now has counter-evidence in P-70. `beyond/brand-defense-and-competition.md`'s citation of this file for "overstates" is supported only in Golden & Horton's form.
+
+**Gaps filled (13).**
+1. *Tool vendors' methods.* Skai (two posts), Pacvue (press release), Teikametrics (three help articles), Intentwise, Helium 10, Perpetua, AdExchanger, Optmyzr, Hershaff, Pattern and Flywheel were added to What practitioners add, with a disagreement note (8) and P-73 and P-74. The first pass's note that "the search budget ran out" was replaced. Open question 9 is partly answered.
+2. *Sponsored Brands and Stores test tools.* No Amazon split tool was found. Added the SB v4 and migration-guide test sentences, MYE on A+ Brand Story, the Dandy Blend two-campaign "A/B", and AMC's undefined "experimentation" store ingress (§E, P-51, Open question 14). Amazon DSP's customer holdout groups were also added (§E, P-50, Stable 24).
+3. *Switchback design.* Bojinov, Simchi-Levi & Zhao's theorems and estimator, and Amazon's own random-days price experiments (Published research, P-59, P-60, Stable 25, S4 revised).
+4. *Difference-in-differences with staggered changes.* Goodman-Bacon; de Chaisemartin & D'Haultfœuille; Callaway & Sant'Anna; Sun & Abraham (P-61, Stable 26, S11).
+5. *Synthetic control.* Abadie, Diamond & Hainmueller 2010; Abadie 2021's feasibility conditions; synthetic DiD (P-62, Stable 27).
+6. *CUPED and the trustworthy-experiment literature.* CUPED, "A Dirty Dozen" (SRM, novelty and primacy) and "Five puzzling outcomes" (carryover, A/A) were read from Microsoft's ExP site (P-63, P-64, Stable 28). The Kohavi, Tang & Xu book is cited by record only; its publisher answered 403/503.
+7. *Measured marketplace interference.* Holtz et al. 2024 (at least 20%), Holtz & Aral (up to 62%), and Blake & Coey through Holtz & Aral's account. Tied to Amazon's match-type rule (P-65).
+8. *A target's response to its own bid.* Wager & Xu, AuctionGym's logged-data warning, Chen–Nabi–Siniscalchi's partial feedback, and Amazon's FTC figures on rival responses (§D, P-55, P-66, P-67, S12).
+9. *Many verdicts.* Benjamini & Hochberg, Gelman–Hill–Yajima, Lee & Shen, Deng 2015, Azevedo et al. (P-68, P-69, Stable 29, S9).
+10. *Brand-term counter-evidence.* Coviello et al. (Edmunds) and Golden & Horton (P-70).
+11. *Amazon-authored measurement work.* Qin, Pauwels et al., AuctionGym, Chen et al., Han et al., Dong et al., Cooprider & Nassiri (Published research; P-54, P-56, P-60, P-67, P-75; Open question 10 answered). Report-maturation size from Optmyzr (P-72).
+12. *Amazon-side changes inside a window.* Reserve prices (help page and FTC response), the complaint's event-period surcharge allegations (as read in `areas/seasons-and-deals.md`), "continually experimenting", and product-search A/B tests and launches (§D, P-53, P-54, Stable 22). Invalid-traffic monitoring was added to §C from the Traffic validation page (P-45, Stable 21).
+13. *What a holdout identifies.* Johnson's "Inferno" and Ghost Ads (ITT against TOT; PSA invalid under optimizing delivery) (P-71, Stable 30, S10).
+
+**Declined in part.**
+- *Weblab as Amazon's documented experimentation platform (within gap 12).* It is named only in job aggregators; amazon.jobs answered 404, and no amazon.science or Amazon Ads page read describes it. It is recorded as not relied on (§D, Unreachable), not as a fact.
+- *Sagum's claim that Sponsored Brands has a "built-in" A/B tool (within gap 2).* No Amazon page read shows a shopper split for SB creative. The claim stays as Sagum's opinion and is not carried into a position.
+- *Kohavi, Tang & Xu's book content (within gap 6).* It is not quoted, because the book could not be read. Its authors' papers stand in.
+- *The complaint's allegations (within gap 12).* These are quoted from the seasons file's reading, labelled as allegations, and not put in a position.
+
+**Corrections to the first pass found while filling.** The Traffic validation help page is readable from the article JSON in its HTML (moved out of Unreachable). Ghost Ads and CUPED are now read and cited (removed from Unreachable). The Stream v2 overview row's "no concurrency statement" was wrong (item 8 above).
+
+**Positions added.** P-change-measurement-45 to P-change-measurement-75, each worded as a definition, Amazon's rule, a documented mechanism, a measured effect with its source, or a named practitioner's claim. P-12 was revised. No new position states when to ask the seller, a fixed sequence, a threshold as the rule, a ceiling on cx, or how cx talks. The seat's reasoning went to S4 (revised) and S9–S12.