@alexbosworth/fiat 2.0.0 → 3.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (105) hide show
  1. package/CHANGELOG.md +22 -0
  2. package/README.md +93 -1
  3. package/arrays/index.js +3 -0
  4. package/arrays/sort_by.js +44 -0
  5. package/chain/add_days.js +22 -0
  6. package/chain/blocks/block_header_hash.js +29 -0
  7. package/chain/blocks/block_header_time.js +25 -0
  8. package/chain/blocks/find_complete_day.js +90 -0
  9. package/chain/blocks/find_height_for_time.js +93 -0
  10. package/chain/blocks/get_block_hash.js +60 -0
  11. package/chain/blocks/get_block_header.js +45 -0
  12. package/chain/blocks/get_block_payments.js +100 -0
  13. package/chain/blocks/get_blocks_for_date.js +110 -0
  14. package/chain/blocks/get_median_time.js +92 -0
  15. package/chain/blocks/index.js +19 -0
  16. package/chain/blocks/payment_outputs_from_block.js +178 -0
  17. package/chain/blocks/sample_heights.js +33 -0
  18. package/chain/blocks/window_heights.js +34 -0
  19. package/chain/chain_rates.js +129 -0
  20. package/chain/constants.json +26 -0
  21. package/chain/estimates/check_known_estimates.js +119 -0
  22. package/chain/estimates/current_estimates.js +66 -0
  23. package/chain/estimates/estimate_and_remember.js +199 -0
  24. package/chain/estimates/estimate_day_price.js +361 -0
  25. package/chain/estimates/estimate_record.js +107 -0
  26. package/chain/estimates/estimate_windows.js +246 -0
  27. package/chain/estimates/find_starting_day.js +86 -0
  28. package/chain/estimates/forget_estimates.js +113 -0
  29. package/chain/estimates/get_chain_price.js +167 -0
  30. package/chain/estimates/get_price_for_date.js +607 -0
  31. package/chain/estimates/get_price_from_blocks.js +281 -0
  32. package/chain/estimates/get_provisional_price.js +163 -0
  33. package/chain/estimates/index.js +25 -0
  34. package/chain/estimates/is_alias_ratio.js +38 -0
  35. package/chain/estimates/is_supported_estimate.js +81 -0
  36. package/chain/estimates/nearest_estimate.js +49 -0
  37. package/chain/estimates/price_window.js +25 -0
  38. package/chain/estimates/remember_estimate.js +83 -0
  39. package/chain/estimates/store_estimate.js +198 -0
  40. package/chain/histogram/best_position.js +63 -0
  41. package/chain/histogram/histogram_excess.js +50 -0
  42. package/chain/histogram/histogram_peak.js +87 -0
  43. package/chain/histogram/implied_prices_histogram.js +53 -0
  44. package/chain/histogram/index.js +33 -0
  45. package/chain/histogram/is_round_tokens.js +26 -0
  46. package/chain/histogram/kernel_width.js +25 -0
  47. package/chain/histogram/lower_bound.js +28 -0
  48. package/chain/histogram/noise_of_neighbors.js +28 -0
  49. package/chain/histogram/noise_strength.js +35 -0
  50. package/chain/histogram/noise_strengths.js +29 -0
  51. package/chain/histogram/outputs_histogram.js +80 -0
  52. package/chain/histogram/parabola_peak.js +28 -0
  53. package/chain/histogram/payments_near_position.js +31 -0
  54. package/chain/histogram/peak_width.js +37 -0
  55. package/chain/histogram/position_score.js +38 -0
  56. package/chain/histogram/score_position.js +94 -0
  57. package/chain/histogram/smooth_bins.js +29 -0
  58. package/chain/histogram/sorted_neighbors.js +89 -0
  59. package/chain/histogram/spike_strengths.js +33 -0
  60. package/chain/histogram/template_agreement.js +27 -0
  61. package/chain/index.js +3 -0
  62. package/chain/known/earliest_supported_day.js +12 -0
  63. package/chain/known/index.js +11 -0
  64. package/chain/known/known_prices.js +43 -0
  65. package/chain/known/known_prices.json +1836 -0
  66. package/chain/known/method_version.js +26 -0
  67. package/chain/known/parse_known_prices.js +67 -0
  68. package/chain/outputs/alias_shifts.js +31 -0
  69. package/chain/outputs/cap_near_round_values.js +83 -0
  70. package/chain/outputs/cap_repeated_values.js +33 -0
  71. package/chain/outputs/dollar_bin_for_price.js +19 -0
  72. package/chain/outputs/estimate_from_outputs.js +321 -0
  73. package/chain/outputs/index.js +11 -0
  74. package/chain/outputs/peak_around_price.js +86 -0
  75. package/chain/outputs/price_for_dollar_bin.js +21 -0
  76. package/chain/outputs/price_from_outputs.js +98 -0
  77. package/chain/outputs/refine_price.js +73 -0
  78. package/chain/outputs/round_amounts.js +49 -0
  79. package/chain/outputs/template.json +22 -0
  80. package/coinbase/constants.json +5 -0
  81. package/coinbase/get_coinbase_day_candles.js +155 -0
  82. package/coinbase/get_coinbase_historic_price.js +135 -0
  83. package/coinbase/index.js +7 -1
  84. package/dates/constants.json +3 -0
  85. package/dates/day_of_date.js +26 -0
  86. package/dates/index.js +5 -0
  87. package/dates/is_calendar_day.js +16 -0
  88. package/dates/time_of_date.js +61 -0
  89. package/docs/METHODOLOGY.md +984 -0
  90. package/index.js +2 -1
  91. package/package.json +29 -8
  92. package/rates/get_prices.js +51 -8
  93. package/storage/constants.json +4 -0
  94. package/storage/get_prices_file.js +169 -0
  95. package/storage/index.js +4 -0
  96. package/storage/is_prices.js +31 -0
  97. package/storage/put_prices_file.js +252 -0
  98. package/storage/temp_path.js +22 -0
  99. package/.gitattributes +0 -1
  100. package/.tm_properties +0 -1
  101. package/test/coinbase/test_get_coinbase_current_price.js +0 -77
  102. package/test/coindesk/test_get_coindesk_current_price.js +0 -95
  103. package/test/coingecko/test_get_coingecko_rates.js +0 -93
  104. package/test/rates/test_get_current_price.js +0 -74
  105. package/test/rates/test_get_prices.js +0 -78
@@ -0,0 +1,984 @@
1
+ # Deriving Prices From Chain Data
2
+
3
+ This document describes how the `chain` methods derive a daily USD price of
4
+ BTC using nothing but the blocks of the Bitcoin chain, why each step is there,
5
+ how the method tells whether it is working, and what its limits are.
6
+
7
+ ## Purpose
8
+
9
+ Historic exchange rates are normally looked up from a third party. That means
10
+ a network request to a service that sees which dates are being asked about,
11
+ that may rate limit or remove access to older data, and whose answer cannot be
12
+ reproduced later by anyone else. The chain is the one data source every node
13
+ already has, so a price that is derived from it can be computed offline, gives
14
+ the same answer to everyone with the same blocks, and can be audited by
15
+ re-running the derivation over the recorded block range.
16
+
17
+ The derived price is not an exchange price. It is an estimate of the rate at
18
+ which people were paying for things in bitcoin on that day, read from the
19
+ amounts they paid. What exactly that number means is defined below.
20
+
21
+ ## The Signal
22
+
23
+ People pay in round amounts of their own currency. Someone sending $100 of
24
+ bitcoin produces an output whose value in satoshis is `100 / price * 1e8`. On
25
+ any given day there are thousands of such payments, at $10, $20, $50, $100,
26
+ $500, $1,000 and so on, and all of them are divided by the same price.
27
+
28
+ Take the logarithm of every output value. A payment of a round amount `U`
29
+ lands at `log10(U) + c`, where `c = log10(1e8 / price)` is the same unknown
30
+ for every payment that day. So the round payments form a fixed pattern of
31
+ spikes, at the log positions of the popular round amounts, that is translated
32
+ as a whole by `c`. Finding the translation finds the price.
33
+
34
+ Each spike is blurred because the rates behind the payments are not all the
35
+ same: the price moves within the day, and wallets quote slightly different
36
+ rates at the same moment. So a spike spans roughly one to a few percent of its
37
+ value. The spikes sit on top of a background of everything else: non-round
38
+ payments, change outputs, consolidations, and so on. The background is dense
39
+ and its shape is unknown, but it has no reason to form spikes at round dollar
40
+ positions.
41
+
42
+ ## Selecting Payments
43
+
44
+ Not every output is a payment, and the cleaner the input the sharper the
45
+ spikes. A transaction is treated as a payment when it is not a coinbase, has
46
+ at most 5 inputs, has exactly 2 outputs, has no OP_RETURN output, and has no
47
+ input carrying more than 500 bytes of witness data. Two outputs is the shape of
48
+ a regular payment with change. Many inputs, many outputs, and OP_RETURN mark
49
+ batching, consolidation and protocol traffic rather than purchases. Large
50
+ witness data marks inscriptions and other data embedding whose output values
51
+ are arbitrary.
52
+
53
+ Every transaction is judged on its own. An earlier version also dropped
54
+ transactions that spent outputs created earlier in the same set of blocks, to
55
+ remove chains of exchange plumbing, but whether a transaction is dropped by
56
+ that rule depends on which other blocks are being looked at, so a sample of a
57
+ day's blocks and the whole day would not agree on which outputs count. The
58
+ rule's benefit was never measured and what it removes is background, which
59
+ the matching below is indifferent to, so it is gone.
60
+
61
+ These filters are plausible rather than proven. Their effect on real blocks
62
+ should be measured by running the estimator with and without each one on days
63
+ with known prices, and the filters kept only where they sharpen the result.
64
+ Change outputs cannot be told from payments by their values, so about half
65
+ of the outputs that pass the filters are change. They have no reason to be
66
+ round dollar amounts, so they add to the background rather than forming false
67
+ spikes, which is what the background removal below is for.
68
+
69
+ Output values of 1,000 sats or less, or of 100,000 BTC or more, are ignored.
70
+
71
+ ### Round Bitcoin Amounts
72
+
73
+ Values that are round amounts of bitcoin, any multiple of 1,000 sats, form
74
+ spikes of their own that do not move with the price. Each round value is
75
+ compared with the values immediately around it, within a tenth of a percent
76
+ on either side, and the excess of its count over what those neighbors lead one
77
+ to expect is removed as a round bitcoin spike. A round value that sits inside
78
+ a spike of round dollar payments has elevated neighbors, so it keeps the
79
+ density of that spike and loses only what rises above it.
80
+
81
+ What cannot be told apart is payers of a round dollar amount who all used
82
+ exactly one rate that put them on a round value: their payments look like a
83
+ round bitcoin spike and are removed with it. On a day where every payer of an
84
+ amount used a rate like that, the amount is lost. Payers whose rates differ
85
+ even slightly spread over the neighboring values and are kept. A wallet that
86
+ rounds its payments to the nearest 1,000 sats loses most of its contribution
87
+ to the signal but does not bias it, since its payments spread over the round
88
+ values on both sides of the true position.
89
+
90
+ The price is also derived with round values counted in full, and the round
91
+ shift is how far that moves the price. On synthetic days it is zero to a
92
+ hundredth of a percent; a large round shift means the round values carry a
93
+ pattern of their own, which is either round bitcoin payments lining up with
94
+ the template at some price or round dollar payers all at one rate.
95
+
96
+ ### Repeated Amounts
97
+
98
+ Payments of round dollar amounts are made at rates that move through the day
99
+ and differ from one service to the next, so their values in sats spread over
100
+ many neighboring values and an exact value rarely repeats more than a few
101
+ times in a day. A flood of transactions from one sender is different: on
102
+ real days, chains of one input transactions repeated a handful of exact
103
+ amounts tens of thousands of times, forming spikes that the matching can take
104
+ for round dollar amounts. Each exact value that is not a round amount of
105
+ bitcoin is therefore counted at most ten times in an estimate, which keeps a
106
+ flood to a few outputs while leaving the spread of real payments as it was.
107
+ Round bitcoin values are left to the comparison with their neighbors above.
108
+
109
+ What this costs is payers who all paid one amount at exactly one rate, such as
110
+ a service that prices in dollars with a rate it holds for hours: the payments
111
+ of each amount beyond the tenth are not counted. They are a small part of
112
+ the signal on real days, where the cap changed the typical estimate by a
113
+ thousandth of a percent.
114
+
115
+ The cap is a count of exact values, not a share of a spike, so what it does
116
+ depends on how many values in sats an amount spreads over. At today's prices a
117
+ round amount spreads over hundreds of values and the cap leaves real spikes
118
+ alone, taking about 1% of the $10 spike at $100,000 in synthetic days of 20,000
119
+ round payments with rates spread by 1%. At a million dollars $10 is 1,000 sats
120
+ and the same spread covers a few dozen values, and the cap takes about two
121
+ thirds of the $10 spike, a third of the $20 spike and a seventh of the $50
122
+ spike. That bends the measured strengths towards the larger amounts, the
123
+ direction that favors the tenfold alias described under Aliases and Confidence.
124
+ A cap relative to the spread of each amount, as the comparison of round values
125
+ with their neighbors is relative, would not, and is left for the next version
126
+ of the method.
127
+
128
+ ## Estimating the Price
129
+
130
+ ### The Histogram
131
+
132
+ The output values are counted into a histogram on a log10 scale with 200 bins
133
+ per decade, so each bin is about 1.16% wide. That is a little narrower than
134
+ the blur of a spike on a calm day, which lets the spike be located precisely
135
+ without splitting it over too many bins. The histogram is normalized to sum to
136
+ one so that a day with many payments and a day with few are comparable.
137
+
138
+ ### Excess Over the Local Background
139
+
140
+ A spike is only a spike relative to what surrounds it. For each bin, the local
141
+ background is the median of the bins within 12 bins on either side, leaving
142
+ out the two bins immediately next to it, and the excess is how far the bin
143
+ rises above that background, floored at zero. The template is matched against
144
+ the excess, not the raw histogram.
145
+
146
+ This matters more than it might seem. Matching against the raw histogram
147
+ rewards any position where the background happens to be dense, spikes or not,
148
+ and compensating for that requires assumptions about where the background
149
+ sits in dollar terms, which is exactly the thing that drifts between years and
150
+ price regimes. Matching against the excess makes the estimator indifferent to
151
+ the shape and position of the background, so the same procedure works at $300
152
+ and at $300,000 without tuning.
153
+
154
+ The fixed neighborhood is a compromise. On a day where the price signal is
155
+ widely dispersed, a spike is wider than the two excluded bins and its own
156
+ shoulders enter the background estimate, which understates the spike. That
157
+ affects every spike the same way so it does not move the position, only the
158
+ measured strength and the margins, and the width of the peak is measured and
159
+ reported so that such days can be recognized.
160
+
161
+ ### The Template
162
+
163
+ The template is the set of round USD amounts people pay. It is not a list of
164
+ observations but is generated from the way people round numbers: the leading
165
+ digits 1, 1.5, 2, 2.5, 3, 4, 5, 6 and 7.5 at every power of ten from $1 to
166
+ $100,000, which gives 46 amounts. These positions are as stable as the habit
167
+ of rounding itself.
168
+
169
+ Each amount also has a prior weight, and this is the one place where the method
170
+ encodes an assumption about behavior rather than deriving it. The weight has
171
+ two parts. The digit part says that amounts led by 1, 2 and 5 are common and
172
+ amounts led by 1.5, 2.5, 3, 4, 6 and 7.5 are rare, which follows the
173
+ denominations money comes in. The envelope part says that round payments are
174
+ mostly between ten and a thousand dollars: it is a bell curve over powers of
175
+ ten, centered on a hundred dollars with a width of one decade, so an amount a
176
+ decade away from a hundred dollars keeps 61% of its weight, two decades away
177
+ 14%, and three decades away about 1%. The prior is a dozen numbers and both
178
+ parts can be checked against the measured spike strengths that every estimate
179
+ reports.
180
+
181
+ For a candidate price, the template says which bin each amount should land
182
+ in. An amount whose spike would fall below 1,000 sats at that price cannot be
183
+ seen there and takes no part, for or against; the share of the prior weight
184
+ that is observable at a position is its evidence. The strength of a spike at
185
+ an observable amount is the excess at its bin plus its neighbors weighted by a
186
+ small kernel spanning two bins either side, which absorbs the blur.
187
+
188
+ Excess is floored at zero, so even where there is no spike the bins around an
189
+ amount add up to something positive, and a position that can see more
190
+ amounts would collect more of that noise and win for that reason alone. So
191
+ each spike strength is taken above the strength that noise alone produces
192
+ there, the trimmed mean of the excess in the surrounding bins with the spike
193
+ itself and the highest fifth of the neighbors left out, which brings the
194
+ score close to zero wherever there is no pattern, however many amounts are
195
+ observable. It is not exactly zero: leaving out the highest neighbors keeps
196
+ other spikes out of the noise but also puts the noise a little low, so on
197
+ simulated payments with no round amounts at all a position scored about 1% to
198
+ 2% of what a real pattern of round payments scores at its price, slightly less
199
+ where fewer amounts were observable, a difference far smaller than the margin
200
+ a price needs over its aliases. The score of a position is the sum of these
201
+ strengths above
202
+ noise, weighted by the template weights, over the observable amounts, and
203
+ it is not scaled up for the amounts that cannot be seen: a position with less
204
+ to show for itself is not helped. The search scores every position in the
205
+ window, one bin at a time, and a parabola through the best bin and its two
206
+ neighbors places the peak between bins.
207
+
208
+ The search under the prior weights decides the alignment, that is, which
209
+ spike is which amount. The strength of every spike is then measured at that
210
+ alignment, and the measured strengths are used as the weights to place the
211
+ peak within the alignment, within two bins of the position the prior chose,
212
+ and to weight the refinement below. The measured strengths cannot be allowed
213
+ to choose between alignments: the alignment they were measured at always fits
214
+ them best, so using them for that would only confirm whatever the first pass
215
+ found. The alignment decision stays with the prior, and everything that
216
+ judges whether that decision was right is computed under the prior too.
217
+
218
+ ### Aliases and Confidence
219
+
220
+ The round dollar pattern looks a lot like itself when shifted. Shift it by a
221
+ factor of two and $50 lands where $100 was, $20 where $10 was, $500 where
222
+ $1,000 was. The same is true, to a lesser degree, for a factor of five, of
223
+ ten, of two and a half, and for every other ratio between two amounts in the
224
+ pattern. No estimator can escape this: a pattern that is nearly invariant
225
+ under scaling cannot by itself pin down the scale. The data alone cannot say
226
+ whether people paid $100 at one price or $200 at half that price.
227
+
228
+ What breaks the symmetry is that the amounts are not equally popular. A
229
+ doubling puts the $20 and $200 spikes onto $40 and $400, amounts hardly anyone
230
+ pays, and puts $50 onto $100, which should be the strongest spike but is then
231
+ weaker than $200. A tenfold shift keeps every digit in place and is only told
232
+ apart by the envelope, that payments of $1,000 are rarer than payments of
233
+ $100. That is what the prior weights are for, and it is the only thing they
234
+ are for.
235
+
236
+ Not every ratio is an equally serious alias. Under the prior, and under
237
+ realistic spike strengths, the pattern shifted by two scores about 0.8 of the
238
+ pattern in place, by five or ten about 0.75, by two and a half or four about
239
+ 0.4. The ratios below two, 1.2, 1.25, 1.33, 1.5, 1.6, 1.67 and 1.875, score
240
+ between 0.2 and 0.01 of the pattern in place, because few of the amounts land
241
+ on other amounts under those shifts. So the aliases at ratios of two and
242
+ above are the ones that can be mistaken for the price, and the ones below two
243
+ are weak but real.
244
+
245
+ Every estimate reports a confidence: the score of the best position divided by
246
+ the best score among its alias positions, which are the best position shifted
247
+ by the ratio between any two amounts in the template, in either direction, at
248
+ possible prices. An alias position only competes when it shows a
249
+ multi-denomination pattern of its own, several observable amounts with spikes
250
+ of at least a fifth of its strongest, so that a lone stray spike that happens
251
+ to land on one template amount is not treated as a rival price. Each position
252
+ is scored against the whole histogram through the amounts it can see there; two
253
+ candidate prices explain the same bins through different amounts, so this is
254
+ the like for like comparison, and the noise correction keeps a position from
255
+ winning merely by seeing more amounts. Both scores use the prior weights. The
256
+ confidence is reported as at most 100, which is what it is when no alias
257
+ position competes, because none shows a pattern that scores above noise, and
258
+ also when the best position scores more than a hundred times better than the
259
+ best alias. Real days are nowhere near that: every confidence logged on real
260
+ days while developing the method was below 1.3. A clear day gives a confidence
261
+ around 1.2 in synthetic data. A confidence near 1 means the data cannot
262
+ distinguish the price from a multiple of itself, and a confidence below 1 means
263
+ an alias outside the search window fits better than anything inside it, which
264
+ is a sign that the window was in the wrong place.
265
+
266
+ At high prices the small amounts fall under 1,000 sats and cannot be seen: $10
267
+ at a million dollars, $50 at five million. The floor is in sats while the
268
+ envelope is in dollars, so as the lower side of the envelope disappears, the
269
+ position a decade below the price sees amounts that the envelope favors about
270
+ as much as the price itself does. In synthetic days of 20,000 round payments
271
+ drawn from the prior, an estimate without a guide at a million dollars found
272
+ the price but was refused for a confidence of 1.09 to 1.10 against the 1.12 it
273
+ needed. At three and five million dollars it found a tenth of the price, with a
274
+ confidence of 1.09 to 1.20 against the 1.11 it needed and every other measure
275
+ healthy, and was supported in three of four runs. So far above today's prices a
276
+ starting price can be off by a factor of ten. A guided estimate cannot land on
277
+ a tenth of its guide, since neither its window nor the widened window reaches
278
+ that far, but the alias is still scored against it, since alias positions are
279
+ only limited to prices that are possible at all. In the same synthetic days,
280
+ guided by the true price, the estimate found the price every time, but its
281
+ confidence was 1.07 to 1.10 at a million dollars, about 1 at 1.8 million, 0.95
282
+ to 0.98 at two million and 0.88 to 0.92 at three million, so from about two
283
+ million dollars a guided day is refused. That is a regime the prior has not
284
+ been built for, and it is discussed under Limits.
285
+
286
+ The confidence says how well the prior separates the scale. It does not say
287
+ whether the price is right in any other sense, which is why the other health
288
+ measures below exist.
289
+
290
+ ### Refinement
291
+
292
+ The rough price is accurate to a fraction of a bin. To do better, every output
293
+ is asked which round amount it is close to at the rough price. An output
294
+ within nine percent of a template amount implies the exact price at which it
295
+ would have been exactly that amount, and those implied prices are counted into
296
+ a fine histogram with 1,000 bins per decade, each weighted by the measured
297
+ strength of its amount. Nine percent is less than the spacing between
298
+ neighboring template amounts, so an output can imply at most one price.
299
+
300
+ Round payments all imply the true price and pile into a peak. The other
301
+ outputs imply prices spread smoothly across the histogram, because their
302
+ values are spread smoothly. The background of each fine bin is the median of
303
+ the bins that are not near it and is subtracted, the excess is smoothed with a
304
+ small triangular kernel to gather the peak, and a parabola through the top of
305
+ the peak places its center. When the peak is wider than the kernel, the kernel
306
+ is widened to match, up to a limit. The histogram is then rebuilt centered on
307
+ the center that was found and the center taken again, until it moves by less
308
+ than half a fine bin, so that the result does not depend on where in the
309
+ histogram the peak first landed. The center of the settled peak is the price.
310
+ Recentering stops after five passes, and a peak that is still moving then is
311
+ reported as unsettled. That alone does not refuse the estimate: an unsettled
312
+ estimate that passes every rule under Supported, Weak and Undetermined is
313
+ accepted, which mostly means its refinement moved by less than one percent
314
+ when started three percent higher or lower, as when a broad peak wanders the
315
+ same way from every start. On synthetic days that trended 8% to 25%, about
316
+ half of the refinements of whole days did not settle, and a few of those
317
+ passed every rule, one of them 7.2% from the middle of a day that moved 25%.
318
+ A full day is normally priced from its windows of about two hours, which
319
+ rarely trend that much, so an unsettled estimate of a whole day only sets the
320
+ price of a day whose supported windows cover less than half of it, while an
321
+ unsettled bridging estimate can set the window of the next day. Refusing an
322
+ unsettled estimate, at least when its search was not confined to a window, is
323
+ left for the next version of the method, since it changes which days are
324
+ supported and so needs the shipped prices to be derived again.
325
+
326
+ The refinement is then repeated from starting prices three percent above and
327
+ below the rough price, and with a range of six percent instead of nine, and
328
+ the stability is the most the price changed. On synthetic days it is zero to
329
+ a hundredth of a percent on calm days and around half a percent on days with
330
+ a wide dispersion of rates.
331
+
332
+ This replaces an earlier refinement that rounded every output to the nearest
333
+ five dollars and took the median of the implied prices. That approach was
334
+ anchored to its own starting point: outputs that are not round payments imply
335
+ prices that are spread symmetrically around whatever rough price was used, so
336
+ the median was a compromise between the rough price and the truth, weighted by
337
+ how much of the data was signal, and it could return the rough price almost
338
+ unchanged while appearing to have refined it. Removing the background before
339
+ taking the center, and recentering until the center settles, means the rough
340
+ price only decides where to look and cannot pull the answer towards itself.
341
+
342
+ ### What the Daily Price Means
343
+
344
+ The price at the center of the peak of round payments is the rate at which
345
+ the typical round payment was made, on a log scale, after the background is
346
+ removed. For a calm day that is close to a payment weighted median of the
347
+ day's rates. On a day where the price trended it sits near the rate at which
348
+ the most payments happened, which need not be the middle of the range, since
349
+ payments crowd into the hours people are awake. Measured against the median
350
+ of the day's hourly prices, the peak of a whole day that moved 5% to 6% was up
351
+ to 3.8% away.
352
+
353
+ So a full day is priced from consecutive windows of about two hours, each with
354
+ a sharp peak of its own, and the price of the day is the time weighted median
355
+ of the windows: the price the day spent as long above as below, which is what
356
+ the reference used for validation measures too. Across the five years of prices
357
+ shipped with the method, the ten days that moved most, 10.4% to 15.4% from open
358
+ to close, were all within 1% of the reference, six of them within 0.1%. The
359
+ width of the peak in each window is reported as its dispersion, and the windows
360
+ themselves show how the price moved through the day.
361
+
362
+ ## Health Measures
363
+
364
+ Every estimate reports several measures of its own quality, each catching a
365
+ different way the method could fail.
366
+
367
+ The payments count is the number of outputs in the refinement peak above the
368
+ background. It estimates how many genuine round payments the price rests on,
369
+ unlike a count of outputs that merely fall in a window, which would include
370
+ the background. It is a count at the peak, not for the day: on a day whose
371
+ price moved several percent the round payments are spread along the path the
372
+ price took and only those near the mode are counted, so the count falls as
373
+ the move grows, which on real days is a strong relationship. The same count
374
+ is taken at every other position in the
375
+ refinement range away from the peak, and the largest of them is the noise:
376
+ the best that a peak made of nothing achieved across all the positions that
377
+ were searched. The significance is the payments beyond the noise, in standard
378
+ deviations of the counts in the peak region. It is calibrated against the
379
+ search rather than against the winning position alone, so that a peak that
380
+ only slightly beats the best noise peak counts for little however many
381
+ outputs there were.
382
+
383
+ The confidence, described above, is the margin over the patterned aliases
384
+ under the prior, and the agreement is the cosine similarity between the
385
+ measured spike strengths and the prior weights. In synthetic data the
386
+ agreement is about 0.9 at the true price and about 0.6 at an alias, so a
387
+ falling agreement over time is the signal that the prior is drifting away from
388
+ how people pay, and the reported spike strengths are the means to refit it.
389
+ The evidence is the share of the prior weight that was observable at the
390
+ price; it is 1 at today's prices and falls as the price rises and the small
391
+ amounts drop below 1,000 sats.
392
+
393
+ The coverage is the share of all spike-like excess, across the span of the
394
+ template, that sits at round dollar positions. Spikes that are not at round
395
+ dollar positions, whether from another currency or from a service paying a
396
+ fixed odd amount, lower it.
397
+
398
+ The rivalry is the best score the pattern achieves at any other position
399
+ within a factor of two of the price that shows a pattern of its own, beyond
400
+ the dispersion of the peak, divided by the score at the best position. That
401
+ includes the weak aliases at ratios below two, which no window wide enough for
402
+ a daily move can exclude, and it includes round payments in a second currency
403
+ at its exchange rate: in synthetic data a euro pattern with half the volume of
404
+ the dollar pattern gives a rivalry of 0.5 at exactly the exchange rate, and
405
+ one with equal volume gives a rivalry near 1, at which point the method cannot
406
+ say which currency it is pricing. On a clean day the rivalry is about 0.2,
407
+ from the 1.5 ratio alias.
408
+
409
+ The dispersion is the width of the refinement peak at half its height, as a
410
+ fraction of the price, beyond the width that the smoothing of the histogram
411
+ gives every peak. The smoothing turns even a perfectly sharp peak into one
412
+ about 2% wide, so the smoothed width is a floor, not a measurement, on a calm
413
+ day; the floor is removed in quadrature, so a peak as sharp as the histogram
414
+ can resolve has a dispersion of zero, and a calm real day reads about 1%. The
415
+ smoothed width itself is still what decides how far from the peak a rival
416
+ position must be. The dispersion is that of the observed price signal, not a
417
+ measure of market movement: it reflects both the price moving during the
418
+ blocks that were used and wallets quoting different rates at the same time.
419
+ A day whose price stepped from one level to another does not read as wide at
420
+ all, since each level forms its own sharp peak; it is the rivalry that reports
421
+ the second level, and the drift that reports the step.
422
+ To tell the two apart, when a day's worth of blocks is used the price is also
423
+ estimated from the first half of the blocks and from the second half, and the
424
+ drift is how much the second differs from the first. A price that moved shows
425
+ a drift of the same order as the dispersion; dispersion with little drift is
426
+ rate scatter. The spikes themselves are reported as the measured strength of
427
+ each template amount relative to the strongest, and the round shift and the
428
+ stability are described above.
429
+
430
+ ## Supported, Weak and Undetermined
431
+
432
+ An estimate is supported when at least three template amounts form spikes of at
433
+ least a fifth of the strongest, at least 100 round payments are in the peak,
434
+ the peak is at least four standard deviations beyond the best noise peak, the
435
+ refinement moves the price by less than one percent when its start or its range
436
+ is changed, every patterned rival position scores less than nine tenths of the
437
+ best, at least half of the prior weight was observable, and the price beats
438
+ every patterned alias. When the search was confined to a window around a known
439
+ price, beating the aliases means a confidence of at least 1. When it was not,
440
+ because there was no known price or because the window was widened, it means a
441
+ confidence of at least 1.1 with the whole pattern observable, and a margin that
442
+ grows as the observable evidence shrinks, one plus a tenth divided by the
443
+ evidence, since then the prior is the only thing excluding the aliases and less
444
+ of the pattern is there to support it.
445
+
446
+ A sample estimate that is not supported, or that rests on fewer than 500
447
+ payments, is weak, and is retried with twice as many blocks until it is
448
+ supported or all the blocks of the day have been used.
449
+
450
+ A full day is estimated both from all of its blocks at once and in consecutive
451
+ windows of twelve blocks, about two hours each, with a last window of fewer
452
+ than six blocks joined to the one before it. Each window is judged by the same
453
+ rules in the same price window, and a window whose search had to be widened is
454
+ not counted, even when the price it found is inside the price window after all,
455
+ as happens when its best position is within three bins of an edge. A window
456
+ covers the time from the block before it to its own last block, by their median
457
+ times past, described below. Timestamps can run backwards from one window to
458
+ the next, and the time they went back would then be counted again by the next
459
+ window, while the median time past never goes backwards, so every second of the
460
+ day is counted once. When the counted windows cover at least half of the day,
461
+ the day is supported, and its price is their time weighted median; otherwise
462
+ the estimate from all of the blocks at once is used when it is supported. On a
463
+ day that trends to an edge of the price window, the windows that were widened
464
+ can leave less than half of the day counted, and the day is then priced from
465
+ all of its blocks at once; counting such windows is left for the next version
466
+ of the method. The measures reported for a day priced from windows are those of
467
+ the median window, except for the outputs and payments, which are the totals of
468
+ the counted windows. A sample that is still not supported once all of the
469
+ blocks of its day are used is estimated in windows the same way.
470
+
471
+ A day that its windows do not support either is undetermined. It is not
472
+ remembered, not used to guide any other day, and the price lookup fails with
473
+ an error that names the day, so that a report built on it fails loudly rather
474
+ than carrying a guess.
475
+
476
+ The thresholds are set from synthetic data and are deliberately conservative.
477
+ They need to be calibrated on real blocks, as described under Validation.
478
+
479
+ ## Finding the Blocks of a Day
480
+
481
+ A day is a UTC calendar day, and a block belongs to the day its median time
482
+ past falls on. The median time past of a block is the median of the
483
+ timestamps of the block and the ten blocks before it. Consensus requires every
484
+ block's timestamp to be later than the median time past of the blocks before
485
+ it, so unlike timestamps, which miners can set a couple of hours off and which
486
+ can run backwards from one block to the next, the median time past never
487
+ decreases with height. That makes every day a contiguous range of heights that
488
+ a binary search finds exactly, and that can never change because a later block
489
+ carries an earlier time. The median time past lags the clock by about an hour,
490
+ which is immaterial for a daily price.
491
+
492
+ A day is complete when at least six blocks have been mined after its last
493
+ block. With the day defined by median time past, completion is purely about
494
+ the chance of a chain reorganization replacing the blocks of the day, which six
495
+ confirmations make negligible. Only complete days get a final price.
496
+
497
+ ## Estimating Day After Day
498
+
499
+ Because of the aliases, the estimator should never search blind when it does
500
+ not have to. Once one day's price is known, the next day's search is confined
501
+ to a window from 0.62 to 1.2 times that price. The window spans less than a
502
+ factor of two, so no price in the window has its double or its half in the
503
+ window too, which keeps out every alias at a ratio of two or more: the ones
504
+ that score close to the price itself. The weak aliases at ratios between 1.2
505
+ and 1.875 can be inside the window with the price; they are not excluded by
506
+ the window but are checked by the rivalry, which must stay below nine tenths
507
+ for the estimate to be supported. The window reaches further down than up
508
+ because falls have been sharper than rises.
509
+
510
+ The window is a guide, not a wall. When the best position is within three bins
511
+ of the edge of the window, or when an alias outside the window scores better
512
+ than anything inside it, the search is repeated in a window twice as wide on
513
+ each side, and the result is accepted only if it beats its aliases by the clear
514
+ margin, and is reported as widened, even when the price it finds turns out to
515
+ be inside the window. A real move of forty percent in a day is found that way.
516
+
517
+ A window placed on the wrong scale is not stable either. Suppose a day were
518
+ stored at double its true price. The next day's window would be centered on
519
+ that double, the true price would be outside it, and the best position inside
520
+ it would be the alias at double the truth. But the true price is an alias of
521
+ that position, so it is scored too, under the prior, and when the prior
522
+ prefers the truth, the confidence comes out below 1, the search widens, and
523
+ the true price is found with its clear margin. The chain of estimates heals
524
+ at the next day. What it cannot do is decide which of the two days was wrong:
525
+ the widened result then sits at an alias ratio of the day that guided it, and
526
+ one of the two is misplaced. That is treated as a scale conflict, the estimate
527
+ is not stored, and the lookup fails with an error naming both days and both
528
+ prices, so that the question is settled by checking one of them against the
529
+ world rather than by storing a file that disagrees with itself.
530
+
531
+ The conflict test takes a ratio within six percent of a ratio between two
532
+ amounts of the pattern, of two or more, as an alias ratio, and nearly every
533
+ ratio from 1.89 up is within six percent of one, apart from narrow bands such
534
+ as 2.12 to 2.26. So a real move from one day to the next of 47% or more down,
535
+ or 89% or more up, almost always ends as a scale conflict rather than a price,
536
+ and such a day is settled against the world like any other conflict.
537
+
538
+ The case that nothing inside the method can detect is a prior that prefers the
539
+ wrong scale on every day, which is discussed under Limits. It is the reason
540
+ that a price estimated without a guide is logged as a starting point and should
541
+ be checked against any exchange, and the reason the validation below exists.
542
+
543
+ When no price is known at all, the most recent complete day is estimated first
544
+ with an unconstrained search, which is where the clear margin applies. The
545
+ present is the regime the prior describes best, and the result can be checked
546
+ against any exchange immediately. A day that does not clear the margin on its
547
+ own, which on real days happens about once a month when the day's mix of
548
+ amounts happens to read well under a relabeling, is not a reason to fail: the
549
+ day before it is tried, and the one before that, for up to a week, and the
550
+ first day that identifies its price on its own is the starting point. The
551
+ same start is made when every known price is more than a week farther from
552
+ the requested day than the most recent complete day, since bridging back from
553
+ the present is then the shorter way.
554
+
555
+ Days between known estimates and the requested day are bridged one day at a
556
+ time. A bridging estimate uses an evenly spaced sample of 12 of the day's
557
+ blocks, one from the middle of each twelfth of the day's blocks, and doubles
558
+ the sample while the estimate is weak, keeping the blocks it has and adding the
559
+ blocks halfway between them, so that each doubling only fetches new blocks. A
560
+ day that is still not supported once the sample is the whole day is estimated
561
+ in windows, so that a day the price moved a lot in does not stop the bridge.
562
+ Each block is retrieved once however many estimates of the day use it. Bridging
563
+ estimates are kept as priors, used only to guide neighboring searches, and are
564
+ replaced when a day gets a full price.
565
+
566
+ A day that cannot be priced stops a lookup that has to bridge through it, with
567
+ an error that names the day, so that it can be checked against an exchange.
568
+
569
+ The day of the requested date itself is priced from all of its blocks, in
570
+ windows as described under What the Daily Price Means. A date on a day that is
571
+ not yet complete gets a provisional price from the confirmed blocks of the day
572
+ so far, padded with the blocks just before the day when the day has fewer than
573
+ 36 confirmed blocks, so that a provisional price measures the same thing as the
574
+ final price as far as the data allows. The latest complete day is bridged to
575
+ first when it has no estimate, so that the guide of a provisional price is
576
+ never further back than that day, which is the day before except in the hour or
577
+ so after midnight, when the day before is not complete yet. When the latest
578
+ complete day is undetermined, the provisional price is guided by the nearest
579
+ other estimate instead of failing. A provisional price
580
+ is kept apart from the full day prices with the blocks it was derived from and
581
+ reused until about six more blocks have been mined, about an hour, after which
582
+ it is derived again, so that looking up the current day again does not fetch
583
+ its blocks each time. Once the day is complete, its full price replaces it. A
584
+ day that starts more than six hours after the median time of the chain tip is
585
+ refused, since the chain has not reached it and a provisional price would only
586
+ come from older blocks, as when the node is still syncing; six hours is the
587
+ span of the padding, and well beyond how far the median time lags the clock,
588
+ even when blocks are slow.
589
+
590
+ ## Remembering Prices
591
+
592
+ Full day prices are kept as rates, bridging estimates as priors and the
593
+ provisional prices of the latest days apart from both in a JSON file. The file
594
+ carries a version of its own format, so that a reader can tell a file written
595
+ in another layout from one it understands. Each record carries the price, the
596
+ block range it was derived from, the hash of the last block in that range, and
597
+ a version made of the method version and a hash of the template, which is
598
+ separate from the file format version. Rates also carry the output and payment
599
+ counts and the main health measures, which show how well a price was supported
600
+ without its blocks, though not every input of the support rule, and they mark a
601
+ price that was estimated without a guide. Checking a price fully means deriving
602
+ it again from its block range, which the hash and the version make
603
+ reproducible.
604
+
605
+ Records whose method version does not match the running method are ignored, so
606
+ a change to the method or the template causes prices to be derived again rather
607
+ than mixing prices from different methods, without discarding the file. Records
608
+ whose last block is within about a week of the chain tip are checked against
609
+ the chain before use. A record whose block hash no longer
610
+ matches, because the chain was reorganized or the file came from another chain,
611
+ is removed from the file for good, as long as the file still holds that same
612
+ record rather than a newer price for the day, and the day is derived again when
613
+ it is needed. A record whose last block is beyond the chain tip, as when the
614
+ node is catching up after being down or another node shares the file, cannot be
615
+ checked yet, so it is left out of that lookup and kept in the file, to be
616
+ checked once the node reaches its block.
617
+
618
+ Every write merges with what is already in the file, so two processes deriving
619
+ prices at the same time add to each other's work instead of replacing it, and
620
+ each write goes to a temporary file of its own that is swapped into place so
621
+ that a reader never sees a partial file. Two writes that overlap by moments can
622
+ still lose one new entry, which is only derived again later. A full day rate
623
+ replaces the prior and the provisional price of its day only when they are of
624
+ its own version, so that a rate left by another version does not take the place
625
+ of current prices. A file that cannot be read, or whose format is not
626
+ understood, is an error rather than an empty file, and nothing is written over
627
+ it, so prices that could not be read are never replaced. An empty file is an
628
+ empty set of prices, and an entry that is not a price for a day of the calendar
629
+ is dropped rather than read. A lookup of a shipped day reads neither the node
630
+ nor the file, so a damaged file only fails the days it is needed for.
631
+
632
+ ## Prices Shipped With the Method
633
+
634
+ Deriving a price takes the blocks of its day, and reaching a day takes a bridge
635
+ from a known price, so a first lookup far from the present waits for a long
636
+ bridge. The prices of every day up to a release are therefore derived before
637
+ the release and shipped with the package, and a lookup on one of those days is
638
+ answered from them without asking the node for anything. A day after them is
639
+ reached from the last shipped day, so a lookup a month after a release bridges
640
+ a month rather than from the present, unless the present is more than a week
641
+ closer, as it is for a recent day looked up long after a release.
642
+
643
+ The shipped prices are derived exactly as a lookup derives a requested day,
644
+ from all of the blocks of each day, guided by the day before it, using a local
645
+ Bitcoin Core. Before they are written, every new day has to be priced, and its
646
+ price has to fall within the range the day traded in on Coinbase Exchange, from
647
+ its lowest to its highest trade, which catches a day placed on the wrong scale
648
+ or pulled away by a flood while leaving room for where the payments of a day
649
+ that moved a lot were made. A price a percent or two off would still pass that
650
+ check, so every new price is also compared with the median of the hourly prices
651
+ of its day, and a price more than 5% from it stops the run to be looked into,
652
+ since none of the 1,824 shipped days measured against that median was more than
653
+ 3.06% from it. On a version change each new price is also compared with the
654
+ price the day had before, and the run ends with the median, 95th percentile,
655
+ bias and largest of each comparison, to be looked over before the prices are
656
+ committed. The most recent shipped days are also derived again and have to come
657
+ out exactly as they were, which catches a change to the method that was not
658
+ given a new version. When the method version changes, every day is derived
659
+ again. When earlier days become supported, only they are derived, bridging back
660
+ from the first shipped day and then forward a day at a time, and the first
661
+ shipped day is derived again from its new neighbor and has to come out exactly
662
+ as it was.
663
+
664
+ The file is JSON holding the method version and the price in cents of each
665
+ day under its date, about 27 bytes a day, and it is only read the first time
666
+ a price is needed. Shipped prices are never written to a user's prices file,
667
+ and anyone with a node can derive them again with the same script.
668
+
669
+ ## Accuracy
670
+
671
+ On synthetic days built from a realistic mix of round payments over a dense
672
+ background, the estimate is within 0.05% of the true price across prices from
673
+ $250 to $1,000,000, including days where the background is centered far from
674
+ where payments usually are, days where only 1% of outputs are round payments,
675
+ and days where the rates behind the payments were dispersed by 1%. Bridging
676
+ through 30 synthetic days that include falls of 35% and rises of 27% tracks
677
+ every day within 0.12%, with the large moves found by widening the window.
678
+
679
+ Adversarial synthetic days behave as follows. A day where only 0.3% of outputs
680
+ are round payments is misread by a factor of two and is correctly refused for
681
+ a significance below 1. A day whose rates trend 15% across the day is placed
682
+ within 1.6% of its midpoint with a dispersion of 10% and a stability of 0.6%;
683
+ a day with a 30% trend is placed 9% from its midpoint and is refused for a
684
+ stability of 1.5%, which is the right outcome, since such a day has no single
685
+ price. Payers who round their payments to the nearest 1,000 sats cost signal
686
+ but no accuracy, even at 80% of payers, and move the round shift by two
687
+ hundredths of a percent. Round payments in euros at half the dollar volume
688
+ have no effect on the price and give a rivalry of 0.5; at equal volume the
689
+ method prices the euro pattern and is refused for a rivalry of 0.99 and a
690
+ significance of 3. A service paying one fixed odd amount in 10% of all payment
691
+ outputs would move the price by 20% but is refused because it leaves fewer
692
+ than three spikes in the pattern. A window placed on the wrong scale by a
693
+ factor of two or five is detected with a confidence well below 1. A day on
694
+ which every payer of an amount used one rate that put it on a round value, at
695
+ a price of exactly $50,000, loses the pattern and is refused, with a round
696
+ shift of over 200% showing that the round values carried it.
697
+
698
+ ## Validation
699
+
700
+ Synthetic data can only show that the mechanism works and that its failure
701
+ modes are the ones expected. The thresholds above, how well the prior
702
+ describes real payments, how large the margins really are, and what the health
703
+ measures look like on real blocks, have to be measured against known prices on
704
+ real blocks. `npm run check-prices` does that for a list of dates without
705
+ storing anything: it derives each price from a cold start, takes a reference
706
+ price for the day from an exchange, and reports every estimate's record
707
+ alongside its difference from the reference. It runs with a local Bitcoin
708
+ Core, like the script that updates the shipped prices.
709
+
710
+ The reference is the median of the typical prices of the day's hourly candles
711
+ on Coinbase Exchange, which go back to early 2015 and need no key: the price
712
+ the day traded above for half of its hours and below for the other half. A
713
+ time weighted measure is chosen on purpose. The chain price is where the most
714
+ round payments were made, and payments are spread through the day with the
715
+ hours people are awake, while exchange volume piles into the hours when the
716
+ price moves most violently, so a volume weighted average is dragged toward a
717
+ crash in a way the payments are not. The volume weighted average is reported
718
+ too, along with the day's first and last trades, and on a calm day they all
719
+ agree. When a day has no candles, CoinGecko's prices for the start of the day
720
+ and of the next day stand in, with their geometric mean as the reference, but
721
+ CoinGecko's public API only serves the past year.
722
+
723
+ The first measurements below were made against that pair of midnight prices
724
+ for every day, before the hourly reference existed, and it is a crude
725
+ instrument: two midnight prices say little about where a day that trended
726
+ spent its payments, so on a day that moved a few percent a difference of
727
+ around a percent was expected, with its sign following the trend.
728
+
729
+ The same comparison can be run unguided, with `--unguided`: each date
730
+ is estimated from the blocks of its own day with no price window from a
731
+ neighbor and no bridging, exactly as a cold start is, and the estimate is
732
+ reported whether or not the data support it, together with whether they do.
733
+ This is the direct test of the method's central claim, that one day's
734
+ payments identify the price on their own, and because it costs one day of
735
+ blocks per date rather than bridging from the present it is the practical
736
+ way to cover many dates across price regimes: the few hundred dollar days
737
+ with sparse payments, fee spikes and inscription waves that fill blocks with
738
+ dust, crash days, days when the price sat near a round number so that round
739
+ BTC and round USD values coincide, and days above $100,000 where the smallest
740
+ denominations fall below the minimum observable output. The estimates to study
741
+ are the ones that are wrong while supported and right while unsupported,
742
+ since those are what would change the support rule.
743
+
744
+ A cold start happens on the latest complete day. Every other day is reached by
745
+ bridging, with a neighboring day's price as its guide, which is the day after
746
+ it when bridging back from the present, so a day that cannot be priced on its
747
+ own blocks is not by itself a problem. What would be is a guided
748
+ estimate that is wrong and still supported, since bridging accepts it and uses
749
+ it to guide the day before. The comparison can be run reference-guided to look
750
+ for that, with `--reference-guided`: each date is estimated from the
751
+ blocks of its own day in the window that a correctly bridged neighbor would
752
+ give it, the window around the reference price of the next day, or around its
753
+ own reference price with `--own-reference`, and the search is widened and
754
+ checked for a scale conflict with its guide exactly as in a derivation. It also
755
+ costs one day of blocks per date. The guide stands in for a neighbor's chain
756
+ price, which on recent days is within a fraction of a percent of the reference.
757
+ The estimate is of the whole day, as for the requested day of a lookup, or with
758
+ `--sampled` of a sample of its blocks, the way bridging estimates every other
759
+ day: a dozen evenly spaced blocks, doubled until the estimate is supported and
760
+ rests on at least 500 round payments or the whole day is used.
761
+
762
+ The first real measurements, guided, were on the 30 consecutive days ending
763
+ October 16, 2026, with 18,000 to 36,000 round payments a day. All 30 were
764
+ derived, the median absolute difference from the reference was 0.25%, 22 were
765
+ within 0.5% and 27 within 1%, and the signed mean was +0.06%, so there was no
766
+ bias. The difference correlated 0.85 with the day's move from open to next
767
+ open, the chain price lay between the two opens on 21 of the 30 days and was
768
+ never more than 0.5% outside that range, and on the two days that moved 6% to
769
+ 7% the chain price sat near the next day's open, the moves having happened
770
+ early in the day, with differences of 2.4% and 3.0% that are the reference's
771
+ failing rather than the chain's. The payment count correlated -0.73 with the
772
+ size of the move, as expected of a count at the peak. The health measures were
773
+ where the synthetic data put them: confidence 1.05 to 1.21 with a median of
774
+ 1.17, rivalry 0.21 to 0.34 except for 0.55 on the day with the step change,
775
+ where the earlier level formed the rival, and stability at most 0.001. The two
776
+ lowest confidences, 1.05 and 1.11, were on the two days with the most
777
+ payments, both priced to within 0.2%. On the lower of the two the strongest
778
+ spike was at $2.50 instead of the usual $100 or $50, a burst of payments of
779
+ that size, and under a relabeling at twice the price $2.50 reads as $5, one of
780
+ the heaviest amounts in the prior, which narrowed the margin over that alias.
781
+ Derived on its own blocks with no guide, the day came to the same price to the
782
+ cent but was not supported, as it should not be when its payments alone
783
+ distinguish the price from its double less clearly than usual, and that is why
784
+ a cold start steps back from a day it cannot support rather than failing.
785
+ Those are the easy regime, recent days with abundant payments, and leave the
786
+ thresholds with wide margins there.
787
+
788
+ The first unguided measurements covered seven days from December 2023 to
789
+ October 2026, again against the pair of midnight prices. All seven were
790
+ supported on their own blocks. The day of the 2024 halving, when the fees paid
791
+ for a new token protocol left 5,341 round payments, about a quarter of the
792
+ count of a day a month before, was within 0.52%, and a day in the inscription
793
+ wave of December 2023 with 11,326 was within 0.42%. The three days above
794
+ $100,000, where the $1 amount falls below the smallest observable output, were
795
+ all supported; they were picked for large moves inside the day, and differed
796
+ from the midnight prices by 1.6% to 4.6%, which is what the hourly reference
797
+ is meant to settle, so they are to be measured again against it. October 13,
798
+ 2026 came to 83,269.28 on its own blocks, the same price to the cent as in two
799
+ runs that reached it by bridging from a neighbor, which is the window doing
800
+ what it is meant to do: excluding aliases without moving the price.
801
+
802
+ Five days of the earlier regime have been measured unguided, against the hourly
803
+ reference, and none was supported. Three were wrong: December 17, 2017 came to
804
+ $102.01 against $19,489.49, June 26, 2019 to $5,008.25 against $12,603.35, and
805
+ March 12, 2020 to $10,023.34 against $6,170.90. Each locked onto a price just
806
+ above $100, $5,000 or $10,000, where round amounts of bitcoin are also round
807
+ dollar amounts: at $10,000, 0.001 BTC is $10. Round amounts of bitcoin sent
808
+ with the fee taken out of the amount would do that, since they sit just below
809
+ the round values, where the comparison with neighboring values, which only
810
+ removes exact round values, leaves them in place, and they read as round
811
+ dollars at a price just above the round number. The first two fell short of the
812
+ margin over the aliases, with confidences of 1.03 and 1.01, but March 12, 2020
813
+ cleared it at 1.43 and was refused by another of the checks, so the margin
814
+ alone would not have caught it. The other two days, May 19, 2021 and November
815
+ 9, 2022, both of which fell 14%, were right, within 0.93% and 0.25%, and fell
816
+ short of the margin, with confidences of 0.94 and 1.05 and rivalries of 0.42
817
+ and 0.44, above the recent days' range. The coincidence prices, one, two or
818
+ five times a power of ten, do not move with the price, and no two are more than
819
+ two and a half times apart while a guided window spans almost two, so most
820
+ windows contain one; the window for June 26, 2019 contains $10,000. A guided
821
+ estimate only needs a confidence of 1 to be supported, so the next measurements
822
+ are of the same days, reference-guided.
823
+
824
+ A year of days, October 20, 2025 to October 18, 2026, was then checked
825
+ reference-guided from samples, the way bridging estimates them, against the
826
+ hourly reference. 358 of the 364 days were supported, with a median absolute
827
+ difference of 0.12%, 95% of them within 1.33% and none more than 3.88% off, and
828
+ no bias. All 161 days that moved less than 1% from open to close were within
829
+ 0.95%, and every difference over 2% was on a day that moved at least 2%, where
830
+ a single price for the day says little. Three of the six days that were not
831
+ supported were floods of repeated amounts. On May 9, 2026 eight exact amounts
832
+ from 1,046 to 2,146 sats, nearly all in one input transactions and most of
833
+ those spending a transaction of the same amount, made 22% of the day's payment
834
+ outputs and pulled the estimate onto a pattern of $1 and $1.50, and on March 22
835
+ and 23 a family of amounts from 3,451 to 3,553 sats, each repeated hundreds to
836
+ thousands of times, put March 23 at twice its price. Counting each exact amount
837
+ at most ten times priced all three within 0.8% from the first sample, and that
838
+ is now part of the method. Across the whole year it supported 361 of the 364
839
+ days, moved the typical price by a thousandth of a percent and 95% of days by
840
+ less than 0.04%, brought 95% of days within 1.2%, and improved nine of the ten
841
+ days it moved by more than 0.1%, among them the three floods and days next to
842
+ them that were partly flooded. The other three, December 2 and 15, 2025 and
843
+ February 25, 2026, moved 2% to 6% during the day and were right as a whole but
844
+ not supported, because the payments spread over the day's range read as noise.
845
+ Estimated in windows of twelve blocks, every window of those three days was
846
+ supported, and so was every window of two days that moved 5% to 6% and of a
847
+ calm day, 71 windows in all. The windows followed Coinbase over the same hours
848
+ with a median difference of 0.04% to 0.13% a day and at most 1.08%, in the
849
+ window a crash happened in, and the time weighted medians of the windows were
850
+ within 0.2% of the reference on all six days. On seven days that moved 4.8% to
851
+ 12.3%, the time weighted median of the windows was within 0.95% of the
852
+ reference where the peak of the whole day was up to 3.8% away, and since then
853
+ every full day is priced from its windows. With both changes in the method, a
854
+ lookup that bridged from October 19, 2026 back to October 20, 2025 reached
855
+ every day, and the six days it priced in full, among them two of the floods
856
+ and the three days that had needed windows, were within 0.78% of the
857
+ reference.
858
+
859
+ The four years before, October 20, 2021 to October 19, 2025, were checked the
860
+ same way with the method as it now stands. Two days, January 31 and February 1,
861
+ 2023, were left unchecked when Coinbase answered the request for their prices
862
+ with a server error, and every one of the other 1,459 days was supported, with
863
+ a median absolute difference of 0.12%, 95% of them within 0.93% and none more
864
+ than 3.79% off, and no bias. All 613 days that moved less than 1% from open to
865
+ close were within 1.93%, the eleven differences over 2% were on days that moved
866
+ more than 1% and were estimated from samples of about two hours, and the 121
867
+ days within 3% of $20,000, $50,000 or $100,000, where round amounts of bitcoin
868
+ are also round dollar amounts, were all supported and within 2.33%, among them
869
+ the summer of 2022 that the price spent near $20,000. November 9, 2022, which
870
+ fell 14% and was short of the margin on its own blocks, was within 0.68%
871
+ guided. Prices are therefore derived from October 20, 2021 on.
872
+
873
+ The shipped prices, every day from October 20, 2021 to October 22, 2026, were
874
+ then derived, each day from all of its blocks and guided by the day before, and
875
+ every day fell within the range it traded in on Coinbase. Against the hourly
876
+ reference, on the 1,824 days it covers, the median difference was 0.06%, 95% of
877
+ days were within 0.38%, 1,813 within 1% and all but two within 2%, with no
878
+ bias, and each year's median was between 0.04% and 0.07%. The largest
879
+ differences were 3.06% on September 13, 2022, a day that fell 9.9%, where the
880
+ price was within 0.08% of the day's volume weighted average price, and 2.12% on
881
+ March 4, 2024, a day that rose 8.2%. The 51 days within 3% of $20,000 had a
882
+ median difference of 0.07% and were all within 1.7%. Counting the time of each
883
+ window by the median time past of its blocks instead of their timestamps moved
884
+ 526 of the days from the previous table, by a median of 0.07% and at most
885
+ 2.15%, and brought August 8, 2024, a day that rose 11.9%, from 2.63% to 0.42%
886
+ away from the reference.
887
+
888
+ The calibration should be done on one set of historical periods and tested on
889
+ another, across price regimes, from the thousand dollar prices of early 2017,
890
+ the earliest days the method is meant to support, to the present, and it should
891
+ look hardest at the estimates that are wrong while passing every check, since
892
+ those are the ones a user would never notice. The expected pattern is that the
893
+ measures separate the wrong estimates from the right ones; where they do not,
894
+ the measures are incomplete, not the data.
895
+
896
+ ## Manipulation
897
+
898
+ Someone who wanted to skew the derived price would have to create payments
899
+ that look like round dollar payments at a different price, in enough volume to
900
+ outweigh the real ones. The estimator weighs outputs, not value, so the cost
901
+ is in transaction fees: a two output transaction is about 140 virtual bytes,
902
+ so at a fee rate of 1 to 5 sats per virtual byte each forged payment costs
903
+ about $0.10 to $0.50 at a $70,000 price. Real round payments are on the order
904
+ of tens of thousands a day, so matching them costs thousands of dollars a day
905
+ and takes a few percent of the day's block space, and doing so for a whole
906
+ month costs on the order of a hundred thousand dollars and is visible as a
907
+ sustained change in the payment count and the spike pattern. Forging a single
908
+ spike is cheaper but is refused as not being a pattern; forging the whole
909
+ pattern at a wrong scale is detected as a rival when the volumes are
910
+ comparable, and only succeeds silently when the forged volume clearly exceeds
911
+ the real one.
912
+
913
+ Weighting payments by the fees they paid would raise the cost of such an
914
+ attack, but fees are not visible in the block data available here, since they
915
+ require the values of the inputs being spent, and the proxies that are
916
+ visible, such as position in the block, are weak. The practical defense is
917
+ the one above: the measures that change when the data changes, and the record
918
+ of what every price was derived from.
919
+
920
+ ## Limits
921
+
922
+ The price is for the currency whose round payments dominate, which is the
923
+ dollar. Round payments in other currencies form their own pattern at their own
924
+ rate; while the dollar pattern is the larger, the others only lower the
925
+ coverage and raise the rivalry, and if another currency's round payments ever
926
+ matched the dollar's, the method would price that currency instead and the
927
+ rivalry near 1 would say so. The method is mainnet only, and dates before
928
+ October 20, 2021 are refused for now. Every day since then has been checked
929
+ against exchange prices, while earlier years have only been sampled, and of the
930
+ four days measured before then none was supported on its own and three locked
931
+ onto prices where round amounts of bitcoin are also round dollar amounts, which
932
+ needs work before those years are supported.
933
+
934
+ The prior weights are the one assumption about behavior, and the scale of
935
+ every price rests on them. If people's payment habits shifted so that every
936
+ amount was paid as often as twice that amount is paid today, the data would be
937
+ indistinguishable from the same habits at half the price and no method could
938
+ tell the two apart. If payments of a thousand dollars became as common as
939
+ payments of a hundred, the envelope would be wrong and the scale could be off
940
+ by ten. A prior that is wrong in one of these ways is wrong on every day, so
941
+ the chain of estimates would be coherent and no check inside the method would
942
+ fire; the agreement would drift and the measured strengths would stop
943
+ resembling the prior, but only a comparison with the world settles it. Such an
944
+ error can only enter at a cold start, since a day guided by a correct neighbor
945
+ searches a window where no strong alias can be, and a day guided by a wrong
946
+ neighbor is caught as a scale conflict when the prior prefers the truth.
947
+
948
+ A day whose round payments come in only one or two amounts cannot be placed
949
+ by the data, since one spike could be any amount, and the method refuses it
950
+ when the pattern has fewer than three spikes but can be misled when it has
951
+ exactly the wrong two. A day on which every payer of an amount used the same
952
+ rate, and that rate put the amount on a multiple of 1,000 sats, loses that
953
+ amount. A day on which the price signal is dispersed by more than ten percent
954
+ has no single price; the estimate is somewhere within the range, the
955
+ dispersion says how wide the range was, and the stability check refuses the
956
+ widest.
957
+
958
+ The estimate depends on there being many round payments in a day. Sampled
959
+ estimates of a few thousand outputs can be weak and are escalated; full days
960
+ of a hundred thousand or more outputs are not weak in testing. A large share
961
+ of non-round junk concentrated in a narrow band of values can create a spike
962
+ of its own and mislead the matcher if it is accompanied by enough other spikes
963
+ to pass as a pattern.
964
+
965
+ Far above today's prices two parts of the method stop working as intended,
966
+ because they count in sats rather than in dollars. The floor of 1,000 sats
967
+ hides the smallest amounts, and the cap of ten on each exact value trims real
968
+ spikes of small amounts once a round amount spreads over few values in sats,
969
+ about two thirds of the $10 spike at a million dollars with rates spread by 1%.
970
+ Both push an estimate without a guide towards a tenth of the price, which from
971
+ about three million dollars it can accept, as described under Aliases and
972
+ Confidence. A guided estimate still finds the price, but from about two
973
+ million dollars a tenth of the price outscores it, and since widening the
974
+ window cannot reach that alias, the day is refused and bridging stops there.
975
+ Neither matters at today's prices, but prices at such levels should be checked
976
+ against an exchange, and a floor and a cap that scale with the price are left
977
+ for the next version of the method.
978
+
979
+ Finally, this is slow compared to a lookup. A day is about 144 blocks, which
980
+ is a few hundred megabytes through the LND chain backend: tens of seconds per
981
+ day with a local full node, minutes per day when blocks have to be fetched
982
+ from peers, plus about a second of computation per estimate for the
983
+ refinement, its stability variants and the round value comparison. Prices are
984
+ remembered precisely so that this cost is only paid once per day.