spree-typesense 6.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +203 -0
- data/LICENSE +9 -0
- data/README.md +311 -0
- data/Rakefile +6 -0
- data/app/models/spree/search_provider/typesense/facets.rb +147 -0
- data/app/models/spree/search_provider/typesense/index_layout.rb +91 -0
- data/app/models/spree/search_provider/typesense/query_builder.rb +161 -0
- data/app/models/spree/search_provider/typesense/schema.rb +128 -0
- data/app/models/spree/search_provider/typesense.rb +901 -0
- data/app/presenters/spree/typesense/product_presenter.rb +252 -0
- data/lib/spree/typesense/circuit_breaker.rb +137 -0
- data/lib/spree/typesense/engine.rb +15 -0
- data/lib/spree/typesense/version.rb +6 -0
- data/lib/spree/typesense.rb +42 -0
- data/lib/spree-typesense.rb +6 -0
- metadata +115 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: d4511197f622f06c51312eb55ee9666bcb8b2576e8b99488685788ff1941f901
|
|
4
|
+
data.tar.gz: 507bde5c69e8803ea0b19560e654ae8e2e094f2f3cc224e3da91df0f0f0edd77
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 67cb48607bccdef46db48344acddce96347d69f739f541e9a83acb5638faf25823d9506d4e33b5aeef10c5e5b02ba8c007302406bc1a264dab2572b7e4966cb9
|
|
7
|
+
data.tar.gz: f682513ba927b3dca9e18ea0979b9259e86de789aa92912781fde7d9d3e9d644022ed4c5b2a8c82138434fae939f3ce9fe81484b94da2007cb46c1eb427fe47f
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,203 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 6.0.0 (2026-11-19)
|
|
4
|
+
|
|
5
|
+
The first release as **spree-typesense** (`gem 'spree-typesense', '~> 6.0'`),
|
|
6
|
+
renamed from spree_typesense as the other Aypex Spree gems are named. Its major
|
|
7
|
+
version now tracks Spree's. 0.3.0 was never released; its changes ship here.
|
|
8
|
+
|
|
9
|
+
- **Spree 6 only** (`spree_core ~> 6.0`, tested on 6.0.0 and 6.0.5). Spree 5.x
|
|
10
|
+
support and its code paths are gone; stay on spree_typesense 0.2.0 for 5.x.
|
|
11
|
+
(So #11, which concerned 5.x category pages, no longer applies.)
|
|
12
|
+
- **Renamed:** `require 'spree/typesense'` (Bundler's `require
|
|
13
|
+
'spree-typesense'` works too), namespace `Spree::Typesense`
|
|
14
|
+
(`Spree::Typesense::ProductPresenter`, `Spree::Typesense::VERSION`). The
|
|
15
|
+
provider is still `Spree::SearchProvider::Typesense`.
|
|
16
|
+
- **Published to RubyGems** (#7). MIT licensed; released from `vX.Y.Z` tags by
|
|
17
|
+
GitHub Actions through RubyGems trusted publishing.
|
|
18
|
+
- **Fix: blank names for a store not written in the application's locale.**
|
|
19
|
+
Indexing runs in a job or a rake task, where Spree reads translated base
|
|
20
|
+
columns (name, description, slug, category and option names) only in the
|
|
21
|
+
application's default locale. A store whose default market locale differs
|
|
22
|
+
(`nl` beside an `en` application) was indexed with blank names, so its
|
|
23
|
+
products could not be found by name. The presenter now reads in the store's
|
|
24
|
+
locale, through `Spree::Current.with_content_locale` on Spree 6.0.5+ and by
|
|
25
|
+
setting `Spree::Current.content_locale` itself on 6.0.0 to 6.0.4, the same
|
|
26
|
+
fix Spree 6.0.5 made to `spree_meilisearch` (spree/spree#15226).
|
|
27
|
+
- **A full reindex swaps in a fresh collection** (#4). `rake
|
|
28
|
+
spree:search:reindex` builds `products_<code>_v<epoch ms>`, then points the
|
|
29
|
+
alias `products_<code>` at it and drops the collection it replaced, so
|
|
30
|
+
documents of deleted or out-of-scope products no longer linger in totals
|
|
31
|
+
and facets. Searches answer from the old collection until the swap; products
|
|
32
|
+
saved during the reindex are written to both. Rejected documents, or any
|
|
33
|
+
error, leave the store on the index it had. A field whose type changed now
|
|
34
|
+
needs only a reindex. Only a reindex creates the index. `reindex(scope)`
|
|
35
|
+
makes the scope the whole index; to refresh some products, index them one
|
|
36
|
+
by one.
|
|
37
|
+
- **Category and collection listings in manual order are sorted by
|
|
38
|
+
Typesense** (#5). Documents carry `pos_<grouping prefixed id>`, the
|
|
39
|
+
product's position in each collection and category (an ancestor category
|
|
40
|
+
takes the lowest position below it, as core's `MIN(position)` does),
|
|
41
|
+
declared by the schema pattern `pos_.*`. A listing of one grouping in manual
|
|
42
|
+
order (`sort` omitted or `manual`, with a text query too, as core's Database
|
|
43
|
+
provider decides) sorts by it instead of being handed to the `Database`
|
|
44
|
+
provider. A grouping no indexed document has a position in yet still falls
|
|
45
|
+
back, quietly.
|
|
46
|
+
- **Facets on category and collection pages count only that grouping** (#6).
|
|
47
|
+
The filters endpoint passes the page's category and collection; its facets
|
|
48
|
+
and total now cover that grouping instead of the whole store. Buyer-catalog
|
|
49
|
+
narrowing is still not mirrored.
|
|
50
|
+
- **Searches skip Typesense for a while after it fails** (#8). After three
|
|
51
|
+
transient failures in a row (timeouts, 5xx, lost connections), a process
|
|
52
|
+
answers searches and facets from the `Database` provider at once for 30
|
|
53
|
+
seconds, logging and reporting the outage once
|
|
54
|
+
(`Spree::SearchProvider::Typesense::CircuitOpened`), then lets one search
|
|
55
|
+
probe Typesense. Set `TYPESENSE_BREAKER_THRESHOLD` and
|
|
56
|
+
`TYPESENSE_BREAKER_SECONDS` to change them. Errors that need someone to act
|
|
57
|
+
never open it. Indexing is unaffected.
|
|
58
|
+
- **Fix: overlapping index jobs for one product no longer drop the newer
|
|
59
|
+
job's documents** (#9). Each document is stamped `indexed_at`, the moment
|
|
60
|
+
its job read the product, and a job deletes only stale documents stamped
|
|
61
|
+
before its own read. Spree 6.0.5 runs one search job per store at a time,
|
|
62
|
+
but only under Solid Queue; this holds on any queue adapter (Sidekiq
|
|
63
|
+
included).
|
|
64
|
+
- **Multi-option filters match one variant exactly** (#10). Documents carry
|
|
65
|
+
`option_value_combination_ids`, one token per set of two to six option
|
|
66
|
+
values a variant has (as `spree_meilisearch` 6.0.5 writes them), and a
|
|
67
|
+
filter on values of several option types asks for those tokens. Pages are
|
|
68
|
+
full and totals exact; core's `with_option_value_ids` narrowing now applies
|
|
69
|
+
only beyond six types or 1000 combinations.
|
|
70
|
+
- **Fix: locale and store id are quoted in every search's filter** (#12).
|
|
71
|
+
A hyphenated locale (`en-US`) and a UUID store id now reach Typesense as one
|
|
72
|
+
backtick-quoted value, as channel ids already did.
|
|
73
|
+
- **Searches are sent as POST** (a one-search `multi_search`), so a long
|
|
74
|
+
filter is not refused by Typesense's 4000-character limit on a GET query
|
|
75
|
+
string. A failed search raises the error its status means (400, 401/403,
|
|
76
|
+
404, 422, 5xx), so a per-search 5xx counts as transient.
|
|
77
|
+
|
|
78
|
+
### Upgrading from spree_typesense 0.2.0
|
|
79
|
+
|
|
80
|
+
Upgrade Spree to 6.0 first, on 0.2.0. Then swap the Gemfile line for
|
|
81
|
+
`gem 'spree-typesense', '~> 6.0'`, rename any `SpreeTypesense` reference to
|
|
82
|
+
`Spree::Typesense`, and run `bin/rake spree:search:reindex` straight after
|
|
83
|
+
deploying: every store must, for the new document fields and the alias
|
|
84
|
+
layout.
|
|
85
|
+
|
|
86
|
+
1. The first reindex builds `products_<code>_v<epoch ms>`, then drops 0.2.0's
|
|
87
|
+
real collection `products_<code>` and creates the alias of that name in its
|
|
88
|
+
place (Typesense refuses an alias named like a live collection). Searches
|
|
89
|
+
in that moment answer from the `Database` provider.
|
|
90
|
+
2. Until it has run, multi-option filters and manual listings are answered by
|
|
91
|
+
the `Database` provider (correct, more slowly).
|
|
92
|
+
3. Rolling back to 0.2.0 works: its reindex recreates a real collection, which
|
|
93
|
+
Typesense serves ahead of the alias; searches return nothing until it has
|
|
94
|
+
filled it.
|
|
95
|
+
|
|
96
|
+
See [Upgrading](README.md#upgrading) for the details.
|
|
97
|
+
|
|
98
|
+
### Known limitations
|
|
99
|
+
|
|
100
|
+
See [Known limitations](README.md#known-limitations): buyer-catalog
|
|
101
|
+
narrowing is not mirrored in facets; option facet counts beside a selection
|
|
102
|
+
of another option type count products, not variants; manual sort adds a
|
|
103
|
+
schema field per category and collection; filters across more than six option
|
|
104
|
+
types fall back to core's narrowing; `price_between`, `cf_*` and other ransack
|
|
105
|
+
filters and sorts are not served by the index.
|
|
106
|
+
|
|
107
|
+
## spree_typesense 0.2.0 (2026-10-31)
|
|
108
|
+
|
|
109
|
+
### Spree 6.0 support
|
|
110
|
+
|
|
111
|
+
- Supports Spree 5.5 to 6.0 (`spree_core >= 5.5, < 6.1`; 0.1.x allowed
|
|
112
|
+
`< 6.0`).
|
|
113
|
+
- Ships its own product presenter, `SpreeTypesense::ProductPresenter`, and
|
|
114
|
+
indexes with it on every Spree version: Spree 6.0 core ships none. On 5.x
|
|
115
|
+
its documents are the same as core's, field for field. On 6.0 the engine
|
|
116
|
+
registers it as `Spree::Dependencies.search_product_presenter` when
|
|
117
|
+
Typesense is the selected search provider and nothing else is registered.
|
|
118
|
+
- Follows Spree 6.0's renames (categories, option `label`, custom fields,
|
|
119
|
+
plain-text descriptions, `variants`) without triggering its deprecations.
|
|
120
|
+
- `in_collection` filters on a new `collection_ids` field (Spree 6), so a
|
|
121
|
+
collection listing returns, counts and facets only that collection.
|
|
122
|
+
Existing collections gain the field in place on the next reindex.
|
|
123
|
+
|
|
124
|
+
### Fixes
|
|
125
|
+
|
|
126
|
+
- **Reindexing adds new fields in place.** `rake spree:search:reindex` now
|
|
127
|
+
adds any field a release declares that the existing collection lacks,
|
|
128
|
+
without dropping it, so searches keep answering. Before, an existing
|
|
129
|
+
collection was left as it was, and `in_collection` filters against it
|
|
130
|
+
failed. A field whose type changed is reported by name, with the remedy
|
|
131
|
+
(drop and reindex).
|
|
132
|
+
- **Rejected documents are reported.** Typesense reports a rejected document
|
|
133
|
+
in its import response rather than as an error, and these used to pass
|
|
134
|
+
unnoticed. Each is now logged with its id and the reason; indexing a
|
|
135
|
+
product raises, so the index job retries; and the reindex carries on, does
|
|
136
|
+
not count them, and fails at the end naming them.
|
|
137
|
+
- **Stale documents are removed.** Re-indexing a product deletes the
|
|
138
|
+
documents it no longer produces, such as those of a currency or locale it
|
|
139
|
+
lost, so it stops showing in that market. A product that produces no
|
|
140
|
+
documents loses them all.
|
|
141
|
+
- **Pre-orders show.** A product scheduled to publish later is listed while
|
|
142
|
+
it is on pre-order, as Spree's own storefront listing does.
|
|
143
|
+
- **Results follow the current channel.** With a current channel, searches
|
|
144
|
+
and facets count only the products published on it, so totals and pages
|
|
145
|
+
match what the channel sells.
|
|
146
|
+
- **Default and manual sort.** The filters endpoint offers `manual`, and on a
|
|
147
|
+
Spree 6 collection page reports the collection's own sort order as the
|
|
148
|
+
default. On Spree 6, a category or collection listing in manual order with
|
|
149
|
+
no text query is answered by the `Database` provider, which orders it by
|
|
150
|
+
position.
|
|
151
|
+
- **Filtering by two or more option types is exact** on Spree 6: only
|
|
152
|
+
products with one variant matching every selected type are listed.
|
|
153
|
+
- **Any Typesense failure falls back to the `Database` provider.** Before,
|
|
154
|
+
only a refused connection did: a timeout, a 5xx, a rejected query (such as
|
|
155
|
+
a filter on a field the collection lacks) or a missing collection returned
|
|
156
|
+
an empty page or empty facets, and an unset `TYPESENSE_URL` or
|
|
157
|
+
`TYPESENSE_API_KEY` raised a 500. Searches and facets now answer from the
|
|
158
|
+
`Database` provider in all these cases, with the failure logged (missing
|
|
159
|
+
configuration once per process) and reported through `Rails.error`: as a
|
|
160
|
+
warning when transient (a timeout, a 5xx, a lost connection), and as an
|
|
161
|
+
error when it needs someone to act (a rejected query, a missing collection,
|
|
162
|
+
a bad API key, missing configuration).
|
|
163
|
+
- **Searches give up sooner on a hung node.** They use their own client with
|
|
164
|
+
a 2 second timeout and no retry, rather than about 10 seconds; set
|
|
165
|
+
`TYPESENSE_SEARCH_TIMEOUT` to change it. Indexing keeps its 5 second timeout
|
|
166
|
+
and one retry.
|
|
167
|
+
|
|
168
|
+
### Known limitations
|
|
169
|
+
|
|
170
|
+
See [Known limitations](README.md#known-limitations) in the README:
|
|
171
|
+
|
|
172
|
+
- Manual (position) order comes from the `Database` provider, on Spree 6
|
|
173
|
+
only; the index carries no positions.
|
|
174
|
+
- A full reindex does not remove documents of deleted products.
|
|
175
|
+
- Facet counts don't apply narrowing made only to the product scope, such as
|
|
176
|
+
Spree 6's buyer catalogs.
|
|
177
|
+
- `price_between`, `in_taxon`, `search_by_name`, other ransack predicates and
|
|
178
|
+
`cf_*` filters are ignored. Sorts other than price, name, availability date
|
|
179
|
+
and best selling (`created_at`, `cf_*`, ...) get the default ranking, and
|
|
180
|
+
`cf_*` sorts are not offered.
|
|
181
|
+
- On Spree 6, filtering by values of two or more option types narrows the page
|
|
182
|
+
after Typesense has counted it, so a page can come back short and the total
|
|
183
|
+
can over-count.
|
|
184
|
+
|
|
185
|
+
### Upgrading
|
|
186
|
+
|
|
187
|
+
1. Bump the gem and run `bin/rake spree:search:reindex`. It adds
|
|
188
|
+
`collection_ids` to a collection a 0.1.x release created, in place. A
|
|
189
|
+
store on Spree 5.6 can do this before moving to Spree 6.
|
|
190
|
+
2. When moving to Spree 6.0: upgrade Spree, run
|
|
191
|
+
`bin/rake spree:migrate_taxons_to_categories_and_collections`, then
|
|
192
|
+
`bin/rake spree:search:reindex` straight after.
|
|
193
|
+
3. `rake spree:search:reindex` now fails, naming the documents, if Typesense
|
|
194
|
+
rejects any of them. On a multi-store install the stores after the failing
|
|
195
|
+
one are then not reindexed: fix the documents and rerun.
|
|
196
|
+
|
|
197
|
+
See [Upgrading](README.md#upgrading) for the details, including what to do
|
|
198
|
+
when a field's type changes.
|
|
199
|
+
|
|
200
|
+
## spree_typesense 0.1.2 (2026-08-02)
|
|
201
|
+
|
|
202
|
+
- Sort options carry `label: nil`, which Spree 5.6's filters serializer
|
|
203
|
+
requires; the filters endpoint no longer fails on Spree 5.6.
|
data/LICENSE
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Aypex
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
|
6
|
+
|
|
7
|
+
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
|
8
|
+
|
|
9
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,311 @@
|
|
|
1
|
+
# spree-typesense
|
|
2
|
+
|
|
3
|
+
A [Typesense](https://typesense.org) search provider for [Spree Commerce](https://spreecommerce.org) 6
|
|
4
|
+
(`spree_core ~> 6.0`; tested on 6.0.0 and 6.0.5). Its major version tracks
|
|
5
|
+
Spree's, as the other Aypex Spree gems' do.
|
|
6
|
+
|
|
7
|
+
Spree 6 ships a `Database` search provider, and `spree_meilisearch` adds a
|
|
8
|
+
Meilisearch one. This gem adds a Typesense one,
|
|
9
|
+
implementing the same `Spree::SearchProvider::Base` contract so
|
|
10
|
+
nothing downstream changes: it returns Spree's own `SearchResult` and
|
|
11
|
+
`FiltersResult` objects, and Spree's existing indexing jobs and
|
|
12
|
+
`rake spree:search:reindex` work unmodified.
|
|
13
|
+
|
|
14
|
+
Motivation: Meilisearch's open-source build is single-node, with HA reserved for
|
|
15
|
+
its Enterprise edition. Typesense ships Raft-based clustering in open source, so
|
|
16
|
+
a multi-node cluster survives losing a node.
|
|
17
|
+
|
|
18
|
+
## Install
|
|
19
|
+
|
|
20
|
+
```ruby
|
|
21
|
+
gem 'spree-typesense', '~> 6.0'
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Before 6.0.0 the gem was `spree_typesense`, published as git tags only, and
|
|
25
|
+
supported Spree 5.5 to 6.0; see [Upgrading](#upgrading).
|
|
26
|
+
|
|
27
|
+
Then select the provider by environment:
|
|
28
|
+
|
|
29
|
+
```ruby
|
|
30
|
+
# config/initializers/spree.rb
|
|
31
|
+
Spree.search_provider = 'Spree::SearchProvider::Typesense' if ENV['TYPESENSE_URL'].present?
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Requires `TYPESENSE_URL` and `TYPESENSE_API_KEY`. `TYPESENSE_SEARCH_TIMEOUT`,
|
|
35
|
+
`TYPESENSE_BREAKER_THRESHOLD` and `TYPESENSE_BREAKER_SECONDS` are optional; see
|
|
36
|
+
[When Typesense is unavailable](#when-typesense-is-unavailable).
|
|
37
|
+
|
|
38
|
+
Then build the index: `bin/rake spree:search:reindex`.
|
|
39
|
+
|
|
40
|
+
See [CHANGELOG.md](CHANGELOG.md) for what each release changes.
|
|
41
|
+
|
|
42
|
+
## When Typesense is unavailable
|
|
43
|
+
|
|
44
|
+
Searches and filter facets answer from Spree's `Database` provider whenever
|
|
45
|
+
Typesense cannot answer: the connection is refused, the request times out,
|
|
46
|
+
Typesense returns a 5xx, or it rejects the query (a missing collection, a
|
|
47
|
+
filter on a field the collection lacks). On Spree 6 the `Database` provider is
|
|
48
|
+
built with the same `audience:`, so a back-office search stays one. Correct
|
|
49
|
+
results, more slowly, beat an empty page. Each failure is logged at error level
|
|
50
|
+
and reported through `Rails.error` as handled: as a warning when it is
|
|
51
|
+
transient (a timeout, a 5xx, a lost connection), and as an error when it needs
|
|
52
|
+
someone to act (a rejected query, a missing collection, a bad API key, missing
|
|
53
|
+
configuration), so a broken index does not go unnoticed.
|
|
54
|
+
|
|
55
|
+
An unset `TYPESENSE_URL` or `TYPESENSE_API_KEY` counts as unavailable too:
|
|
56
|
+
searches answer from the `Database` provider, and the problem is logged once
|
|
57
|
+
per process rather than on every search. Indexing still raises, so the jobs
|
|
58
|
+
fail and retry.
|
|
59
|
+
|
|
60
|
+
Searches and facets go through their own client, with a 2 second timeout and
|
|
61
|
+
no retry, so a hung node costs a shopper about 2 seconds before the fallback
|
|
62
|
+
answers. Set `TYPESENSE_SEARCH_TIMEOUT` (in seconds, e.g. `1.5`) to change it.
|
|
63
|
+
Indexing keeps a 5 second timeout and one retry, and Spree's index jobs retry
|
|
64
|
+
on failure.
|
|
65
|
+
|
|
66
|
+
During an outage each process stops trying after a while. After three
|
|
67
|
+
transient failures in a row (timeouts, 5xx, lost connections) it skips
|
|
68
|
+
Typesense for 30 seconds: searches and facets answer from the `Database`
|
|
69
|
+
provider at once, with nothing logged per search. Opening is logged once at
|
|
70
|
+
error level and reported once through `Rails.error` as a warning
|
|
71
|
+
(`Spree::SearchProvider::Typesense::CircuitOpened`). Then one search probes
|
|
72
|
+
Typesense; if it answers, searches go back to it, and if not, it waits
|
|
73
|
+
another 30 seconds. Errors that need someone to act (a rejected query, a
|
|
74
|
+
missing collection, a bad key, missing configuration) never open it.
|
|
75
|
+
|
|
76
|
+
| Variable | Default | |
|
|
77
|
+
|---|---|---|
|
|
78
|
+
| `TYPESENSE_SEARCH_TIMEOUT` | `2` | seconds a search waits for Typesense |
|
|
79
|
+
| `TYPESENSE_BREAKER_THRESHOLD` | `3` | transient failures in a row that open the breaker |
|
|
80
|
+
| `TYPESENSE_BREAKER_SECONDS` | `30` | seconds searches skip Typesense once it opens |
|
|
81
|
+
|
|
82
|
+
## Reindexing
|
|
83
|
+
|
|
84
|
+
`bin/rake spree:search:reindex` builds each store's index afresh: a new
|
|
85
|
+
collection, `products_<store code>_v<epoch ms>`, filled from the store's
|
|
86
|
+
products, then swapped in behind the alias `products_<store code>`, which is
|
|
87
|
+
the name searches and index jobs use. The collection it replaced is dropped,
|
|
88
|
+
and with it the documents of products that were deleted or left the scope.
|
|
89
|
+
|
|
90
|
+
- Searches keep answering from the previous collection until the swap.
|
|
91
|
+
- While a reindex runs, the collection it is filling is marked by the alias
|
|
92
|
+
`products_<store code>__building`, and a product saved meanwhile is written
|
|
93
|
+
to both, so the change survives the swap.
|
|
94
|
+
- If Typesense rejects any document, or the reindex fails, the new collection
|
|
95
|
+
is dropped and the alias left where it was: the store keeps the index it
|
|
96
|
+
had, and the rake task fails naming the rejected documents.
|
|
97
|
+
- Only a reindex creates the index. Until the first one, index jobs fail and
|
|
98
|
+
retry, and searches answer from the `Database` provider.
|
|
99
|
+
- The scope a reindex is given becomes the whole index. `rake
|
|
100
|
+
spree:search:reindex` passes every product of the store; passing fewer
|
|
101
|
+
(`provider.reindex(Spree::Product.where(id: ids))`) would leave only those
|
|
102
|
+
searchable. To refresh some products, index them one by one
|
|
103
|
+
(`provider.index(product)`, or Spree's `product.add_to_search_index`).
|
|
104
|
+
|
|
105
|
+
## Known limitations
|
|
106
|
+
|
|
107
|
+
- **Manual (position) sort uses one field per grouping.** Each document
|
|
108
|
+
carries `pos_<prefixed id>`, the product's position in each
|
|
109
|
+
collection and category it belongs to (an ancestor category takes the
|
|
110
|
+
lowest position below it), and a single category or collection listing in
|
|
111
|
+
manual order (`sort` omitted or `manual`, with or without a text query, as
|
|
112
|
+
core decides) sorts by it inside Typesense. Each grouping adds a field to
|
|
113
|
+
the collection's schema, which suits catalogues with hundreds or a few
|
|
114
|
+
thousand groupings. A grouping no indexed document has a position in yet
|
|
115
|
+
is answered by the `Database` provider.
|
|
116
|
+
- **Facets don't apply buyer-catalog narrowing.** Facet counts come from the
|
|
117
|
+
index and the filters given. A category or collection page's own grouping is
|
|
118
|
+
applied (the filters endpoint passes it), but narrowing made only to the
|
|
119
|
+
product `scope`, such as Spree 6's buyer catalogs and audiences, is not:
|
|
120
|
+
the index cannot express it. Spree's Meilisearch provider has the same
|
|
121
|
+
limit. Search results are always intersected with the scope.
|
|
122
|
+
- **Some filters and sorts are ignored.** The index answers `price_gte`,
|
|
123
|
+
`price_lte`, `in_stock`, `out_of_stock`, `in_category`, `in_categories`,
|
|
124
|
+
`in_collection` (Spree 6) and `with_option_value_ids`. It ignores
|
|
125
|
+
`price_between`, `in_taxon`, `search_by_name`, other ransack predicates such
|
|
126
|
+
as `name_cont`, and `cf_*` custom-field filters. It sorts by `price`,
|
|
127
|
+
`-price`, `name`, `-name`, `available_on`, `-available_on` and
|
|
128
|
+
`best_selling`; any other sort, such as `created_at` or a `cf_*` sort, gets
|
|
129
|
+
the default ranking. `cf_*` sorts are not offered.
|
|
130
|
+
- **Option facet counts are per product.** With values of one option type
|
|
131
|
+
selected, the counts for another type's values count products that carry
|
|
132
|
+
the value on any variant: with Blue selected, XL counts a product selling a
|
|
133
|
+
Red XL beside a Blue S, though choosing XL then lists only products with a
|
|
134
|
+
Blue XL. The listing and its total are exact; only the counts beside the
|
|
135
|
+
values can run higher. Spree's Meilisearch provider counts the same way.
|
|
136
|
+
- **Values of more than six option types at once.** Values of
|
|
137
|
+
two to six option types must sit on one variant, and the index answers that
|
|
138
|
+
exactly through one token per set of a variant's values. Beyond six types
|
|
139
|
+
(or 1000 combinations of the selected values) the index only pre-filters by
|
|
140
|
+
each type and core's `Product.with_option_value_ids` narrows the scope, so a
|
|
141
|
+
page can come back shorter than `limit` and the total can over-count.
|
|
142
|
+
|
|
143
|
+
## Engine behaviours this gem depends on
|
|
144
|
+
|
|
145
|
+
Typesense and Meilisearch diverge in several ways that fail **silently** — no
|
|
146
|
+
error, just wrong answers. Those behaviours are pinned as executable specs in
|
|
147
|
+
`spec/typesense_behaviour_spec.rb`, which doubles as an upgrade canary: run it
|
|
148
|
+
first when bumping the Typesense version.
|
|
149
|
+
|
|
150
|
+
### Documents
|
|
151
|
+
|
|
152
|
+
The gem builds its documents with its own `Spree::Typesense::ProductPresenter`,
|
|
153
|
+
following `spree_meilisearch`'s presenter. Spree 6 core ships no search
|
|
154
|
+
presenter (`search_product_presenter: nil`); at boot the engine registers this
|
|
155
|
+
one as `Spree::Dependencies.search_product_presenter`, unless the application
|
|
156
|
+
or another search gem already set one, and only when Typesense is the selected
|
|
157
|
+
`Spree.search_provider`, so with `spree_meilisearch` installed alongside, each
|
|
158
|
+
gem's presenter goes with its own provider.
|
|
159
|
+
|
|
160
|
+
An application can set `Spree::Dependencies.search_product_presenter` to a
|
|
161
|
+
subclass of `Spree::Typesense::ProductPresenter`, and the provider indexes with
|
|
162
|
+
it. Any other presenter there is ignored and the gem's own is used: the setting
|
|
163
|
+
is shared by every installed search gem, and another gem's documents
|
|
164
|
+
(`spree_meilisearch`'s, say) do not fit this collection's schema.
|
|
165
|
+
|
|
166
|
+
- **Categories:** `category_ids` (with every ancestor) and `category_names`
|
|
167
|
+
come from `product.categories`; the category facet looks up
|
|
168
|
+
`Spree::Category`. The gem never references the deprecated `Spree::Taxon`.
|
|
169
|
+
- **Options:** `option_type_names` and `option_values` read `label`; option
|
|
170
|
+
values come from `product.variants`.
|
|
171
|
+
- **Custom fields:** `cf_*` keys come from `product.custom_fields`, for
|
|
172
|
+
definitions that are storefront-visible and marked searchable or sortable.
|
|
173
|
+
They are stored but not declared in the schema, so Typesense does not index
|
|
174
|
+
them: they cannot be searched, filtered or sorted on.
|
|
175
|
+
- **Collections:** documents carry `collection_ids`, and `in_collection`
|
|
176
|
+
filters on it, so a collection listing returns, counts and facets only that
|
|
177
|
+
collection.
|
|
178
|
+
- **Descriptions** are indexed as plain text
|
|
179
|
+
(`Spree::RichTextHelper.to_plain_text`), so tag names are not searchable.
|
|
180
|
+
- **Base columns are read in the store's content locale.** Spree 6 reads a
|
|
181
|
+
translated attribute's base column only in `Spree::Current.content_locale`,
|
|
182
|
+
which a job or rake task leaves at the application's default locale. The
|
|
183
|
+
presenter sets it to the store's default market locale while it builds, as
|
|
184
|
+
`spree_meilisearch` does from 6.0.5. Without this, a store whose locale is
|
|
185
|
+
not the application's (`nl` beside `en`) would index blank names.
|
|
186
|
+
- **Providers take an `audience:`** (`:store` by default, `nil` for the back
|
|
187
|
+
office). The gem inherits `Base#initialize`, so `custom_field_schema` is
|
|
188
|
+
limited to what that audience may see. When Typesense cannot answer, the
|
|
189
|
+
Database fallback is built with the same audience, so a back-office search
|
|
190
|
+
does not degrade into a storefront one.
|
|
191
|
+
|
|
192
|
+
Loading and using the gem triggers no Spree deprecation warning; the suite
|
|
193
|
+
raises on any.
|
|
194
|
+
|
|
195
|
+
## Upgrading
|
|
196
|
+
|
|
197
|
+
### From spree_typesense 0.2.0 to spree-typesense 6.0.0
|
|
198
|
+
|
|
199
|
+
The gem is renamed, as the other Aypex Spree gems are named, and supports
|
|
200
|
+
Spree 6 only. Upgrade Spree first (see [To Spree 6.0](#to-spree-60)) while on
|
|
201
|
+
spree_typesense 0.2.0, which supports both.
|
|
202
|
+
|
|
203
|
+
1. In the Gemfile, replace `gem 'spree_typesense', github: ...` with
|
|
204
|
+
`gem 'spree-typesense', '~> 6.0'`.
|
|
205
|
+
2. Rename any reference to the gem's own constants: `SpreeTypesense` is now
|
|
206
|
+
`Spree::Typesense` (`Spree::Typesense::ProductPresenter` for a presenter
|
|
207
|
+
subclass). The provider is still `Spree::SearchProvider::Typesense`. Inside
|
|
208
|
+
`module Spree`, a bare `Typesense` now names the gem's module: reach the
|
|
209
|
+
typesense-ruby client there as `::Typesense`.
|
|
210
|
+
3. Run `bin/rake spree:search:reindex` straight after deploying. It is
|
|
211
|
+
required: documents gain `indexed_at`, `option_value_combination_ids` and
|
|
212
|
+
the `pos_*` position fields, and the index moves behind an alias.
|
|
213
|
+
|
|
214
|
+
- **The first reindex replaces the 0.2.0 collection.** It builds
|
|
215
|
+
`products_<store code>_v<epoch ms>`, then drops the real collection
|
|
216
|
+
`products_<store code>` that 0.2.0 searched and creates the alias of that
|
|
217
|
+
name in its place. Typesense refuses an alias named like an existing
|
|
218
|
+
collection, so for the moment between the two, searches get a 404 and
|
|
219
|
+
answer from the `Database` provider.
|
|
220
|
+
- **Until it has run**, a filter on values of two or more option types asks
|
|
221
|
+
for a field the old collection lacks, and a manual listing sorts by
|
|
222
|
+
positions it lacks. Typesense refuses both, and the `Database` provider
|
|
223
|
+
answers them: correct, more slowly. The option filter is reported as an
|
|
224
|
+
error; the manual listing only logged. Index jobs keep writing to the old
|
|
225
|
+
collection.
|
|
226
|
+
- **If Typesense rejects a document**, the reindex fails naming it and the
|
|
227
|
+
store keeps the index it had. On a multi-store install
|
|
228
|
+
`spree:search:reindex` then stops at that store: fix the documents and
|
|
229
|
+
rerun it.
|
|
230
|
+
- **Rolling back to spree_typesense 0.2.0** works: its reindex creates a real
|
|
231
|
+
`products_<store code>` again, which Typesense serves ahead of the alias.
|
|
232
|
+
Searches return nothing until that reindex has filled it, so roll back at
|
|
233
|
+
a quiet time. A later spree-typesense reindex drops it and the collection
|
|
234
|
+
left behind.
|
|
235
|
+
|
|
236
|
+
A store whose default market locale differs from the application's
|
|
237
|
+
`config.i18n.default_locale` also gets its product names back in the index
|
|
238
|
+
(they were indexed blank).
|
|
239
|
+
|
|
240
|
+
### To Spree 6.0
|
|
241
|
+
|
|
242
|
+
On spree_typesense 0.2.0:
|
|
243
|
+
|
|
244
|
+
1. Upgrade Spree, run its migrations, then
|
|
245
|
+
`bin/rake spree:migrate_taxons_to_categories_and_collections`.
|
|
246
|
+
2. Then run `bin/rake spree:search:reindex`, straight after: documents gain
|
|
247
|
+
`collection_ids` and positions from the migrated collections. Category ids
|
|
248
|
+
and option names do not change.
|
|
249
|
+
|
|
250
|
+
Until the reindex has run, documents carry no `collection_ids`, so
|
|
251
|
+
collection listings miss products in the meantime.
|
|
252
|
+
|
|
253
|
+
### A field whose type changed
|
|
254
|
+
|
|
255
|
+
Typesense can add a field to a collection in place but cannot change a field's
|
|
256
|
+
type. A release that changes one needs nothing special: the reindex builds a
|
|
257
|
+
fresh collection with the declared types. Until then
|
|
258
|
+
`ensure_index_settings!` warns, naming the field.
|
|
259
|
+
|
|
260
|
+
## Local development
|
|
261
|
+
|
|
262
|
+
Tests run against a real Typesense server. The few that check only which
|
|
263
|
+
requests the provider sends use a recording client in its place
|
|
264
|
+
(`spec/support/recording_typesense_client.rb`); what Typesense does with those
|
|
265
|
+
requests is pinned against the server. Start one locally before running the
|
|
266
|
+
suite:
|
|
267
|
+
|
|
268
|
+
```bash
|
|
269
|
+
curl -O https://dl.typesense.org/releases/30.2/typesense-server-30.2-darwin-arm64.tar.gz
|
|
270
|
+
tar xzf typesense-server-30.2-darwin-arm64.tar.gz
|
|
271
|
+
./typesense-server --data-dir=/tmp/ts --api-key=test-key --listen-port=8108
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
The version **must match the pinned CI version (`30.2`, see `.github/workflows/ci.yml`)**
|
|
275
|
+
so local and CI behavior agree. Apple's `container` CLI is not required (and, as of
|
|
276
|
+
2026-08-02, its `apiserver` does not reliably start on all machines) — the native
|
|
277
|
+
binary above is the supported local path.
|
|
278
|
+
|
|
279
|
+
Then run the suite:
|
|
280
|
+
|
|
281
|
+
```bash
|
|
282
|
+
TYPESENSE_URL=http://localhost:8108 TYPESENSE_API_KEY=test-key bundle exec rspec
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
## Typesense behaviours this gem depends on
|
|
286
|
+
|
|
287
|
+
`spec/typesense_behaviour_spec.rb` asserts, directly against a live Typesense
|
|
288
|
+
server (not through our provider), five behaviours this gem's design leans on.
|
|
289
|
+
Each one is a real defect that hid behind a stub during the original build —
|
|
290
|
+
this suite is what would have caught it, and now exists so the *next* Typesense
|
|
291
|
+
upgrade fails loudly here instead of silently in production.
|
|
292
|
+
|
|
293
|
+
1. **Range operators on string fields silently match nothing** — no error,
|
|
294
|
+
just zero results. This is why the provider derives an int64
|
|
295
|
+
`available_on_i` at index time instead of filtering on an ISO8601 string.
|
|
296
|
+
2. **Facet values are capped at 10 by default and truncated silently** —
|
|
297
|
+
but numeric facets still expose exact `stats.min`/`stats.max` (and those
|
|
298
|
+
stats respect `filter_by`), while non-numeric facets get a stats block
|
|
299
|
+
with no min/max at all. This is the escape hatch the price facet uses
|
|
300
|
+
instead of raising `max_facet_values`.
|
|
301
|
+
3. **Zero-count facet values are omitted entirely, not reported as 0** — the
|
|
302
|
+
disjunctive facet merge has to treat a missing value as absent, not zero.
|
|
303
|
+
4. **`multi_search` returns per-result `{error, code}` bodies instead of
|
|
304
|
+
raising** — a failing sub-search degrades silently unless each result is
|
|
305
|
+
checked individually.
|
|
306
|
+
5. **`::Typesense::Error::Error` does not exist** — the base class is
|
|
307
|
+
`::Typesense::Error`. A `rescue` naming the former is invisible dead code.
|
|
308
|
+
|
|
309
|
+
**Run this suite first whenever the pinned Typesense version changes.** A
|
|
310
|
+
failure here means an assumption above no longer holds — investigate before
|
|
311
|
+
assuming the rest of the gem is safe to upgrade.
|