metka 2.3.4 → 3.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +273 -236
- data/lib/generators/metka/sql_identifier.rb +25 -0
- data/lib/generators/metka/strategies/index/index_generator.rb +84 -0
- data/lib/generators/metka/strategies/index/templates/migration.rb.erb +89 -0
- data/lib/generators/metka/strategies/table/table_generator.rb +91 -0
- data/lib/generators/metka/strategies/table/templates/migration.rb.erb +128 -0
- data/lib/generators/metka/strategies/table/templates/migration.sqlite.rb.erb +93 -0
- data/lib/metka/generic_parser.rb +28 -14
- data/lib/metka/model.rb +130 -51
- data/lib/metka/query_builder.rb +45 -49
- data/lib/metka/tag_list.rb +16 -8
- data/lib/metka/tags_query.rb +93 -0
- data/lib/metka/version.rb +1 -1
- data/lib/metka.rb +24 -10
- metadata +35 -154
- data/.github/ISSUE_TEMPLATE.md +0 -15
- data/.github/workflows/lint_code.yml +0 -21
- data/.github/workflows/lint_docs.yml +0 -57
- data/.github/workflows/specs.yml +0 -86
- data/.gitignore +0 -18
- data/.mdlrc +0 -1
- data/.rspec +0 -2
- data/.rubocop-md.yml +0 -20
- data/.rubocop.yml +0 -27
- data/.ruby-version +0 -1
- data/Gemfile +0 -12
- data/Gemfile.lock +0 -239
- data/Rakefile +0 -13
- data/bin/console +0 -14
- data/bin/setup +0 -8
- data/forspell.dict +0 -7
- data/gemfiles/rails52.gemfile +0 -6
- data/gemfiles/rails6.gemfile +0 -6
- data/gemfiles/rails61.gemfile +0 -6
- data/gemfiles/railsmain.gemfile +0 -5
- data/gemfiles/rubocop.gemfile +0 -4
- data/lib/generators/metka/strategies/materialized_view/materialized_view_generator.rb +0 -73
- data/lib/generators/metka/strategies/materialized_view/templates/migration.rb.erb +0 -54
- data/lib/generators/metka/strategies/view/templates/migration.rb.erb +0 -26
- data/lib/generators/metka/strategies/view/view_generator.rb +0 -70
- data/lib/metka/query_builder/all_tags_query.rb +0 -11
- data/lib/metka/query_builder/any_tags_query.rb +0 -11
- data/lib/metka/query_builder/base_query.rb +0 -48
- data/metka.gemspec +0 -42
data/README.md
CHANGED
|
@@ -1,15 +1,15 @@
|
|
|
1
|
-
|
|
2
|
-
[](https://github.com/jetrockets/metka/actions)
|
|
3
|
-
[](https://www.codetriage.com/jetrockets/metka)
|
|
1
|
+
# 
|
|
4
2
|
|
|
5
|
-
|
|
3
|
+
[](https://badge.fury.io/rb/metka)
|
|
4
|
+
[](https://github.com/metka-ruby/metka/actions)
|
|
5
|
+
[](https://www.codetriage.com/metka-ruby/metka)
|
|
6
6
|
|
|
7
|
-
Rails gem
|
|
7
|
+
A Rails tagging gem built on PostgreSQL array columns. Tags live in an indexed array column right on your table — no join tables, no extra models, no N+1 queries. SQLite is supported too: there the tags live in a JSON column and every query compiles to `json_each` probes (see [Database support](#database-support)).
|
|
8
8
|
|
|
9
9
|
:exclamation: Requirements:
|
|
10
10
|
|
|
11
|
-
* Ruby
|
|
12
|
-
* Rails >= 5.2
|
|
11
|
+
* Ruby >= 3.2
|
|
12
|
+
* Rails >= 7.1 (for Rails 5.2 to 6.1 use version ~> 2.3, for Rails 5.1 and 5.0 use version <2.1.0)
|
|
13
13
|
|
|
14
14
|
## Installation
|
|
15
15
|
|
|
@@ -31,6 +31,70 @@ Or install it yourself as:
|
|
|
31
31
|
gem install metka
|
|
32
32
|
```
|
|
33
33
|
|
|
34
|
+
## Database support
|
|
35
|
+
|
|
36
|
+
Metka works on PostgreSQL and SQLite, with the same API and the same matching
|
|
37
|
+
semantics on both. The adapter is detected at query time, so nothing needs to
|
|
38
|
+
be configured — only the migration differs:
|
|
39
|
+
|
|
40
|
+
```ruby
|
|
41
|
+
# PostgreSQL: an array column, indexed with GIN
|
|
42
|
+
t.string :tags, array: true, default: [], index: { using: :gin }
|
|
43
|
+
|
|
44
|
+
# SQLite: a JSON column holding an array of strings
|
|
45
|
+
t.json :tags, default: []
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
On PostgreSQL, queries use the array containment operators (`@>`, `&&`) and
|
|
49
|
+
are served by GIN indexes. On SQLite, tags are stored as a JSON array and
|
|
50
|
+
queries compile to `EXISTS` probes over the `json_each` table-valued function.
|
|
51
|
+
SQLite has no index type that can serve membership-in-array predicates, so by
|
|
52
|
+
default tag queries there are table scans — fast at embedded-database scale
|
|
53
|
+
because SQLite runs in-process, but growing linearly with the table. When
|
|
54
|
+
that starts to matter, the opt-in [index strategy](#index-strategy-sqlite)
|
|
55
|
+
turns tag queries into index seeks.
|
|
56
|
+
|
|
57
|
+
The [table strategy](#table-strategy-with-triggers-recommended) works on both
|
|
58
|
+
databases: the generator inspects the adapter and emits transition-table
|
|
59
|
+
triggers for PostgreSQL or per-row `json_each` triggers for SQLite. SQLite
|
|
60
|
+
3.35 or newer is required (the `sqlite3` gem bundles a current version).
|
|
61
|
+
|
|
62
|
+
### Index Strategy (SQLite)
|
|
63
|
+
|
|
64
|
+
The index strategy maintains one `(tag_name, record_id)` side table per
|
|
65
|
+
tagged column — a `WITHOUT ROWID` table whose primary key doubles as a
|
|
66
|
+
covering index — kept in step by per-row triggers, exactly like the table
|
|
67
|
+
strategy maintains its counters. With it declared, `tagged_with` and the
|
|
68
|
+
column scopes answer from index seeks (`INTERSECT` of per-tag seeks for
|
|
69
|
+
"all", one `IN` probe for "any") instead of scanning `json_each`, with
|
|
70
|
+
identical semantics: case-sensitive, any string is a valid tag, and rows
|
|
71
|
+
with a `NULL` tag column behave exactly as before.
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
rails g metka:strategies:index --source-table-name=songs --source-columns=tags genres
|
|
75
|
+
rails db:migrate
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
Then route queries through the tables by declaring them on the model:
|
|
79
|
+
|
|
80
|
+
```ruby
|
|
81
|
+
class Song < ActiveRecord::Base
|
|
82
|
+
include Metka::Model(
|
|
83
|
+
columns: %w[tags genres],
|
|
84
|
+
index_tables: {
|
|
85
|
+
"tags" => "songs_tags_index",
|
|
86
|
+
"genres" => "songs_genres_index"
|
|
87
|
+
}
|
|
88
|
+
)
|
|
89
|
+
end
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
On PostgreSQL the generator produces nothing (GIN indexes already serve tag
|
|
93
|
+
queries) and the `index_tables` declaration is inert, so the same model code
|
|
94
|
+
runs on both adapters. The same ownership caveat as the table strategy
|
|
95
|
+
applies: writes that bypass the triggers (restoring from a dump) require
|
|
96
|
+
reseeding the index tables the way the migration seeds them.
|
|
97
|
+
|
|
34
98
|
## Tag objects
|
|
35
99
|
|
|
36
100
|
```bash
|
|
@@ -38,7 +102,7 @@ rails g migration CreateSongs
|
|
|
38
102
|
```
|
|
39
103
|
|
|
40
104
|
```ruby
|
|
41
|
-
class CreateSongs < ActiveRecord::Migration[
|
|
105
|
+
class CreateSongs < ActiveRecord::Migration[7.1]
|
|
42
106
|
def change
|
|
43
107
|
create_table :songs do |t|
|
|
44
108
|
t.string :title
|
|
@@ -61,8 +125,35 @@ end
|
|
|
61
125
|
@song.save
|
|
62
126
|
```
|
|
63
127
|
|
|
128
|
+
### Writing tags: `tag_list=` vs the raw column
|
|
129
|
+
|
|
130
|
+
`tag_list=` (and its per-column siblings like `genre_list=`) is the Metka write
|
|
131
|
+
path. Input goes through the configured parser, which splits on the delimiter,
|
|
132
|
+
honors quoting, strips blanks, and de-duplicates:
|
|
133
|
+
|
|
134
|
+
```ruby
|
|
135
|
+
@song.tag_list = 'chill, chill, top'
|
|
136
|
+
@song.tags
|
|
137
|
+
#=> ["chill", "top"]
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
Assigning the array column directly is a plain ActiveRecord attribute write.
|
|
141
|
+
Metka does not see it, so nothing is parsed or de-duplicated — and duplicates
|
|
142
|
+
stored this way are counted twice by tag clouds, which aggregate the raw array
|
|
143
|
+
elements:
|
|
144
|
+
|
|
145
|
+
```ruby
|
|
146
|
+
@song.tags = [ 'chill', 'chill' ] # stored exactly as given
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
This is by design: the column belongs to your schema, and Metka only owns the
|
|
150
|
+
`*_list` API. If you write the column directly, normalizing the array is your
|
|
151
|
+
responsibility.
|
|
152
|
+
|
|
64
153
|
## Find tagged objects
|
|
65
154
|
|
|
155
|
+
Every scope below builds on PostgreSQL's array operators (`@>` for "all", `&&` for "any"), so queries can use the GIN indexes created in the migration above. On SQLite the same scopes compile to `EXISTS` probes over `json_each` with identical semantics. Passing an empty string or `nil` returns the unfiltered relation.
|
|
156
|
+
|
|
66
157
|
### .with_all_#{column_name}
|
|
67
158
|
|
|
68
159
|
```ruby
|
|
@@ -160,7 +251,7 @@ Song.tagged_with(nil)
|
|
|
160
251
|
Song.tagged_with('rock')
|
|
161
252
|
#=> [#<Song id: 1, title: 'Migrate tags in Rails to PostgreSQL', tags: ['top', 'chill'], genres: ['rock', 'jazz', 'pop']]
|
|
162
253
|
|
|
163
|
-
Song.tagged_with('rock', join_operator: Metka::
|
|
254
|
+
Song.tagged_with('rock', join_operator: Metka::AND)
|
|
164
255
|
#=> []
|
|
165
256
|
|
|
166
257
|
Song.tagged_with('chill', any: true)
|
|
@@ -172,12 +263,9 @@ Song.tagged_with('chill, 1980', any: true)
|
|
|
172
263
|
Song.tagged_with('', any: true)
|
|
173
264
|
#=> [#<Song id: 1, title: 'Migrate tags in Rails to PostgreSQL', tags: ['top', 'chill'], genres: ['rock', 'jazz', 'pop']]
|
|
174
265
|
|
|
175
|
-
Song.tagged_with('rock, rap', any: true, on: ['genres'])
|
|
266
|
+
Song.tagged_with('rock, rap', any: true, on: [ 'genres' ])
|
|
176
267
|
#=> [#<Song id: 1, title: 'Migrate tags in Rails to PostgreSQL', tags: ['top', 'chill'], genres: ['rock', 'jazz', 'pop']]
|
|
177
268
|
|
|
178
|
-
Song.without_all_tags('top')
|
|
179
|
-
#=> []
|
|
180
|
-
|
|
181
269
|
Song.tagged_with('top, 1990', exclude: true)
|
|
182
270
|
#=> [#<Song id: 1, title: 'Migrate tags in Rails to PostgreSQL', tags: ['top', 'chill'], genres: ['rock', 'jazz', 'pop']]
|
|
183
271
|
|
|
@@ -189,23 +277,31 @@ Song.tagged_with('top, 1990', any: true, exclude: true)
|
|
|
189
277
|
|
|
190
278
|
Song.tagged_with('1990, 1980', any: true, exclude: true)
|
|
191
279
|
#=> [#<Song id: 1, title: 'Migrate tags in Rails to PostgreSQL', tags: ['top', 'chill'], genres: ['rock', 'jazz', 'pop']]
|
|
192
|
-
|
|
193
|
-
Song.without_any_genres('rock, pop')
|
|
194
|
-
#=> []
|
|
195
280
|
```
|
|
196
281
|
|
|
282
|
+
`join_operator:` controls how multiple tagged columns combine: `Metka::OR`
|
|
283
|
+
(the default) matches when any column satisfies the tags, `Metka::AND` requires
|
|
284
|
+
every column to. The plain symbols `:or` and `:and` work too. Anything else
|
|
285
|
+
raises `ArgumentError`, as does any option other than `any:`, `exclude:`,
|
|
286
|
+
`join_operator:` and `on:`.
|
|
287
|
+
|
|
197
288
|
## Custom delimiter
|
|
198
289
|
|
|
199
|
-
By default, a comma is used
|
|
200
|
-
You can
|
|
290
|
+
By default, a comma is used to split a tag string into tags.
|
|
291
|
+
You can configure your own delimiter:
|
|
201
292
|
|
|
202
293
|
```ruby
|
|
203
|
-
Metka.
|
|
294
|
+
Metka.delimiter = '|'
|
|
295
|
+
|
|
204
296
|
parsed_data = Metka::GenericParser.instance.call('cool, data|I have')
|
|
205
297
|
parsed_data.to_a
|
|
206
|
-
#=>['cool, data', 'I have']
|
|
298
|
+
#=> ['cool, data', 'I have']
|
|
207
299
|
```
|
|
208
300
|
|
|
301
|
+
The setting is global and affects every model that uses the default parser.
|
|
302
|
+
`Metka.config.delimiter = '|'` and `Metka.configure { |config| config.delimiter = '|' }`
|
|
303
|
+
still work as well.
|
|
304
|
+
|
|
209
305
|
## Tags with quote
|
|
210
306
|
|
|
211
307
|
```ruby
|
|
@@ -216,8 +312,8 @@ parsed_data.to_a
|
|
|
216
312
|
|
|
217
313
|
## Custom parser
|
|
218
314
|
|
|
219
|
-
By default
|
|
220
|
-
|
|
315
|
+
By default tags are parsed with [Metka::GenericParser](lib/metka/generic_parser.rb "generic_parser").
|
|
316
|
+
To plug in your own parser for a specific model:
|
|
221
317
|
|
|
222
318
|
```ruby
|
|
223
319
|
class Song < ActiveRecord::Base
|
|
@@ -225,127 +321,121 @@ class Song < ActiveRecord::Base
|
|
|
225
321
|
end
|
|
226
322
|
```
|
|
227
323
|
|
|
228
|
-
|
|
324
|
+
A parser is any object that responds to `call`, accepts the raw tag value (a
|
|
325
|
+
string or an array) and returns a `Metka::TagList`. The simplest approach is to
|
|
326
|
+
subclass `Metka::GenericParser`. You can also replace the parser globally with
|
|
327
|
+
`Metka.parser = Your::Custom::Parser` — note that the global setting takes the
|
|
328
|
+
singleton class itself, and `.instance` is called on it at parse time.
|
|
229
329
|
|
|
230
330
|
## Tag Cloud Strategies
|
|
231
331
|
|
|
232
|
-
There are
|
|
332
|
+
There are two strategies to get tag statistics. The [Table Strategy](#table-strategy-with-triggers-default) is the default choice — it reads like a pre-aggregated table and writes within measurement noise of having no strategy at all (see the [benchmark](#benchmark-comparison)). The ActiveRecord Strategy needs zero setup and is fine for occasional clouds on small tables.
|
|
233
333
|
|
|
234
|
-
###
|
|
334
|
+
### Table Strategy with Triggers (Default)
|
|
235
335
|
|
|
236
|
-
Data about taggings
|
|
336
|
+
Data about taggings will be maintained in a real table with two columns, `tag_name` and `taggings_count`, kept up to date by statement-level triggers. Instead of recomputing the whole aggregation on every write, the triggers read the statement's transition tables and apply per-tag deltas, so a write statement only touches the counters of the tags it actually changed. That keeps writes within measurement noise of a table with no triggers at all while reads stay as fast as a plain indexed table — the trade-off is that it is an ordinary table, so anything that writes `NAME_OF_TABLE_WITH_TAGS` without firing the triggers (`TRUNCATE`, restoring from a dump) leaves the counters stale until you reseed the table by hand. The same ownership caveat as for raw column writes applies: keeping the counters honest is your responsibility the moment you go around the write path.
|
|
237
337
|
|
|
238
|
-
```
|
|
338
|
+
```bash
|
|
339
|
+
rails g metka:strategies:table --source-table-name=NAME_OF_TABLE_WITH_TAGS [--source-columns=NAME_OF_COLUMN_1 NAME_OF_COLUMN_2] [--table-name=NAME_OF_RESULTING_TABLE]
|
|
340
|
+
```
|
|
239
341
|
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
include Metka::Model(column: 'co_authors')
|
|
243
|
-
end
|
|
342
|
+
* If `--source-columns` is omitted, the `tags` column is used by default. When several columns are given, a tag found in more than one of them gets a single row in the summary table with the sum of its occurrences across all those columns.
|
|
343
|
+
* `--table-name` is optional too. Without it, the table is named `<source_table>_<columns>_cloud`, mirroring the index strategy's `<source_table>_<column>_index`: `songs_tags_cloud` for a `songs` table, or `books_authors_and_co_authors_cloud` when several columns are given.
|
|
244
344
|
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
#=> [["A.P. Chekhov", 5], ["N.V. Gogol", 8], ["L.N. Tolstoy", 2]]
|
|
249
|
-
summary_cloud = Book.metka_cloud('authors', 'co_authors')
|
|
250
|
-
#=> [["L.N. Tolstoy", 5], ["F.M. Dostoevsky", 6], ["A.P. Chekhov", 5], ["N.V. Gogol", 8]]
|
|
251
|
-
```
|
|
345
|
+
The generated migration creates the table, seeds it from the rows already present in `NAME_OF_TABLE_WITH_TAGS`, and installs one statement-level trigger per operation (`INSERT`, `UPDATE`, `DELETE`). The migration template can be seen [here](test/dummy/db/migrate/11_create_table_posts_tags_cloud_table.rb "here")
|
|
346
|
+
|
|
347
|
+
For a `notes` table with a `tags` column the resulting `notes_tags_cloud` table would look like this:
|
|
252
348
|
|
|
253
|
-
|
|
349
|
+
| tag_name | taggings_count |
|
|
350
|
+
|----------|----------------|
|
|
351
|
+
| Ruby | 124056 |
|
|
352
|
+
| React | 30632 |
|
|
353
|
+
| Rails | 28696 |
|
|
354
|
+
| Crystal | 6566 |
|
|
355
|
+
| Elixir | 3475 |
|
|
254
356
|
|
|
255
|
-
|
|
357
|
+
And you can also create a model to work with the table as with a Rails model — set the table name explicitly, since Rails would infer the plural `notes_tags_clouds` from the class name:
|
|
256
358
|
|
|
257
|
-
```
|
|
258
|
-
|
|
359
|
+
```ruby
|
|
360
|
+
class NotesTagsCloud < ApplicationRecord
|
|
361
|
+
self.table_name = "notes_tags_cloud"
|
|
362
|
+
end
|
|
259
363
|
```
|
|
260
364
|
|
|
261
|
-
|
|
262
|
-
If `source-columns` option is not provided, then `tags` column would be used as defaults. If array of multiple values would be provided to the option, then the aggregation would be made with the tags from multiple tagged columns, so if a single tag would be found within multiple tagged columns, the resulting aggregation inside the view would have a single row for that tag with a sum of it's occurrences across all stated tagged columns.
|
|
263
|
-
`view-name` option is also optional, it would just force the resulting view's name to the one of your choice. If it's not provided, then view name would be generated automatically, you could check it within generated migration.
|
|
365
|
+
#### Migrating from on-the-fly tag clouds
|
|
264
366
|
|
|
265
|
-
|
|
367
|
+
If you already tag rows with Metka and serve clouds straight off the model — `Song.tag_cloud`, `Book.author_cloud`, `Book.metka_cloud('authors', 'co_authors')` — the generated migration doubles as the migration path. It backfills the summary table from your existing rows and installs the triggers in the same transaction, locking the source table against writes (reads are not blocked) so no statement can slip between the backfill and the triggers; the counts are exact from the moment the migration commits, even under live traffic.
|
|
266
368
|
|
|
267
|
-
|
|
268
|
-
|--------|---------------------|-----------------------------------|
|
|
269
|
-
| id | integer | nextval('notes_id_seq'::regclass) |
|
|
270
|
-
| body | text | |
|
|
271
|
-
| tags | character varying[] | '{}'::character varying[] |
|
|
369
|
+
Generate and run the migration, listing every column your cloud aggregates:
|
|
272
370
|
|
|
273
|
-
|
|
371
|
+
```bash
|
|
372
|
+
rails g metka:strategies:table --source-table-name=songs
|
|
373
|
+
```
|
|
274
374
|
|
|
275
375
|
```bash
|
|
276
|
-
rails
|
|
376
|
+
rails db:migrate
|
|
277
377
|
```
|
|
278
378
|
|
|
279
|
-
|
|
379
|
+
For a multi-column cloud like `Book.metka_cloud('authors', 'co_authors')` pass `--source-columns=authors co_authors`, and the seeded counts sum both columns exactly like `metka_cloud` does.
|
|
280
380
|
|
|
281
|
-
|
|
282
|
-
# frozen_string_literal: true
|
|
283
|
-
|
|
284
|
-
class CreateTaggedNotesView < ActiveRecord::Migration[5.0]
|
|
285
|
-
def up
|
|
286
|
-
execute <<-SQL
|
|
287
|
-
CREATE OR REPLACE VIEW tagged_notes AS
|
|
288
|
-
SELECT
|
|
289
|
-
tag_name,
|
|
290
|
-
COUNT ( * ) AS taggings_count
|
|
291
|
-
FROM (
|
|
292
|
-
SELECT UNNEST
|
|
293
|
-
( tags ) AS tag_name
|
|
294
|
-
FROM
|
|
295
|
-
view_posts
|
|
296
|
-
) subquery
|
|
297
|
-
GROUP BY
|
|
298
|
-
tag_name;
|
|
299
|
-
SQL
|
|
300
|
-
end
|
|
381
|
+
Add a model for the summary table and swap the call sites — `tag_cloud` returns `[tag_name, count]` pairs, and the summary table stores the same data one row per tag:
|
|
301
382
|
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
SQL
|
|
306
|
-
end
|
|
383
|
+
```ruby
|
|
384
|
+
class SongsTagsCloud < ActiveRecord::Base
|
|
385
|
+
self.table_name = "songs_tags_cloud"
|
|
307
386
|
end
|
|
308
|
-
```
|
|
309
|
-
|
|
310
|
-
Now lets take a look at `tagged_notes` view.
|
|
311
|
-
|
|
312
|
-
| tag_name | taggings_count |
|
|
313
|
-
|----------|----------------|
|
|
314
|
-
| Ruby | 124056 |
|
|
315
|
-
| React | 30632 |
|
|
316
|
-
| Rails | 28696 |
|
|
317
|
-
| Crystal | 6566 |
|
|
318
|
-
| Elixir | 3475 |
|
|
319
387
|
|
|
320
|
-
|
|
388
|
+
Song.tag_cloud # before
|
|
389
|
+
SongsTagsCloud.pluck(:tag_name, :taggings_count) # after
|
|
390
|
+
```
|
|
321
391
|
|
|
322
|
-
|
|
392
|
+
Sorting and limiting that used to happen in Ruby becomes a normal query: `SongsTagsCloud.order(taggings_count: :desc).limit(50)`.
|
|
323
393
|
|
|
324
|
-
|
|
394
|
+
Nothing about how you write tags changes: `tag_list=` and friends keep working, and the triggers keep the counts in step with every `INSERT`, `UPDATE` and `DELETE`, including bulk statements like `insert_all`, `update_all` and `delete_all`. If you ever write around the triggers (`TRUNCATE`, restoring from a dump), rebuild the table the same way the migration seeded it:
|
|
325
395
|
|
|
326
|
-
```
|
|
327
|
-
|
|
396
|
+
```sql
|
|
397
|
+
BEGIN;
|
|
398
|
+
LOCK TABLE songs IN SHARE ROW EXCLUSIVE MODE;
|
|
399
|
+
DELETE FROM songs_tags_cloud;
|
|
400
|
+
INSERT INTO songs_tags_cloud (tag_name, taggings_count)
|
|
401
|
+
SELECT tag_name, COUNT(*)
|
|
402
|
+
FROM (SELECT UNNEST(tags) AS tag_name FROM songs) subquery
|
|
403
|
+
GROUP BY tag_name;
|
|
404
|
+
COMMIT;
|
|
328
405
|
```
|
|
329
406
|
|
|
330
|
-
|
|
407
|
+
On SQLite the same reseed reads the arrays through `json_each` (no lock is
|
|
408
|
+
needed — SQLite allows a single writer per database):
|
|
409
|
+
|
|
410
|
+
```sql
|
|
411
|
+
BEGIN;
|
|
412
|
+
DELETE FROM songs_tags_cloud;
|
|
413
|
+
INSERT INTO songs_tags_cloud (tag_name, taggings_count)
|
|
414
|
+
SELECT value, COUNT(*)
|
|
415
|
+
FROM songs, json_each(songs.tags)
|
|
416
|
+
GROUP BY value;
|
|
417
|
+
COMMIT;
|
|
418
|
+
```
|
|
331
419
|
|
|
332
|
-
|
|
420
|
+
### ActiveRecord Strategy (Zero Setup)
|
|
333
421
|
|
|
334
|
-
|
|
422
|
+
Tagging statistics are available via class methods on any model that includes `Metka::Model`. You can build a cloud for a single tagged column or for several at once — in the latter case each tag's count is summed across the given columns. The ActiveRecord strategy is the easiest to use since it requires no additional code, but it is the slowest one on SELECT.
|
|
335
423
|
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
| Rails | 28696 |
|
|
341
|
-
| Crystal | 6566 |
|
|
342
|
-
| Elixir | 3475 |
|
|
343
|
-
|
|
344
|
-
And you can also create `TaggedNote` model to work with the view as with a Rails model.
|
|
424
|
+
```ruby
|
|
425
|
+
class Book < ActiveRecord::Base
|
|
426
|
+
include Metka::Model(columns: %w[authors co_authors])
|
|
427
|
+
end
|
|
345
428
|
|
|
346
|
-
|
|
429
|
+
author_cloud = Book.author_cloud
|
|
430
|
+
#=> [["L.N. Tolstoy", 3], ["F.M. Dostoevsky", 6]]
|
|
431
|
+
co_author_cloud = Book.co_author_cloud
|
|
432
|
+
#=> [["A.P. Chekhov", 5], ["N.V. Gogol", 8], ["L.N. Tolstoy", 2]]
|
|
433
|
+
summary_cloud = Book.metka_cloud('authors', 'co_authors')
|
|
434
|
+
#=> [["L.N. Tolstoy", 5], ["F.M. Dostoevsky", 6], ["A.P. Chekhov", 5], ["N.V. Gogol", 8]]
|
|
435
|
+
```
|
|
347
436
|
|
|
348
|
-
|
|
437
|
+
`metka_cloud` accepts only columns declared in `Metka::Model`; anything else
|
|
438
|
+
raises `ArgumentError`.
|
|
349
439
|
|
|
350
440
|
## Inspired by
|
|
351
441
|
|
|
@@ -355,10 +445,10 @@ TBD
|
|
|
355
445
|
|
|
356
446
|
## Migration from ActsAsTaggable
|
|
357
447
|
|
|
358
|
-
|
|
448
|
+
Migrating your data from `ActsAsTaggable` can be done with a migration like the following.
|
|
359
449
|
|
|
360
450
|
```ruby
|
|
361
|
-
class AddTagsToYourTable < ActiveRecord::Migration[
|
|
451
|
+
class AddTagsToYourTable < ActiveRecord::Migration[7.1]
|
|
362
452
|
def change
|
|
363
453
|
add_column :your_table, :tags, :string, array: true
|
|
364
454
|
add_index :your_table, :tags, using: 'gin'
|
|
@@ -373,7 +463,7 @@ class AddTagsToYourTable < ActiveRecord::Migration[6.0]
|
|
|
373
463
|
INNER JOIN taggings
|
|
374
464
|
ON tags.id = taggings.tag_id
|
|
375
465
|
WHERE
|
|
376
|
-
taggings.taggable_type = '
|
|
466
|
+
taggings.taggable_type = 'YourTableType'
|
|
377
467
|
GROUP BY taggings.taggable_id
|
|
378
468
|
) as tags
|
|
379
469
|
WHERE your_table.id = tags.your_table_id
|
|
@@ -384,140 +474,87 @@ end
|
|
|
384
474
|
|
|
385
475
|
## Benchmark Comparison
|
|
386
476
|
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
Calculating -------------------------------------
|
|
456
|
-
Metka: 42.291M memsize ( 0.000 retained)
|
|
457
|
-
388.694k objects ( 0.000 retained)
|
|
458
|
-
50.000 strings ( 0.000 retained)
|
|
459
|
-
ActsAsTaggableOn: 178.664M memsize ( 0.000 retained)
|
|
460
|
-
1.812M objects ( 0.000 retained)
|
|
461
|
-
50.000 strings ( 0.000 retained)
|
|
462
|
-
ActsAsTaggableOnArray: 42.173M memsize ( 0.000 retained)
|
|
463
|
-
383.003k objects ( 0.000 retained)
|
|
464
|
-
50.000 strings ( 0.000 retained)
|
|
465
|
-
TagColumns: 41.948M memsize ( 0.000 retained)
|
|
466
|
-
383.003k objects ( 0.000 retained)
|
|
467
|
-
50.000 strings ( 0.000 retained)
|
|
468
|
-
|
|
469
|
-
###################################################################
|
|
470
|
-
|
|
471
|
-
bench:find_by_tag
|
|
472
|
-
|
|
473
|
-
Time measurements:
|
|
474
|
-
|
|
475
|
-
Rehearsal ----------------------------------------------------------
|
|
476
|
-
Metka: 0.029961 0.000059 0.030020 ( 0.030052)
|
|
477
|
-
ActsAsTaggableOn: 0.067095 0.000068 0.067163 ( 0.067205)
|
|
478
|
-
ActsAsTaggableOnArray: 0.043156 0.000133 0.043289 ( 0.043440)
|
|
479
|
-
TagColumns: 0.056475 0.000143 0.056618 ( 0.056697)
|
|
480
|
-
------------------------------------------------- total: 0.197090sec
|
|
481
|
-
|
|
482
|
-
user system total real
|
|
483
|
-
Metka: 0.028291 0.000019 0.028310 ( 0.028321)
|
|
484
|
-
ActsAsTaggableOn: 0.065925 0.000036 0.065961 ( 0.065989)
|
|
485
|
-
ActsAsTaggableOnArray: 0.043214 0.000079 0.043293 ( 0.043361)
|
|
486
|
-
TagColumns: 0.056390 0.000160 0.056550 ( 0.056666)
|
|
487
|
-
|
|
488
|
-
Memory measurements:
|
|
489
|
-
|
|
490
|
-
Calculating -------------------------------------
|
|
491
|
-
Metka: 4.752M memsize ( 0.000 retained)
|
|
492
|
-
43.000k objects ( 0.000 retained)
|
|
493
|
-
1.000 strings ( 0.000 retained)
|
|
494
|
-
ActsAsTaggableOn: 8.967M memsize ( 0.000 retained)
|
|
495
|
-
81.002k objects ( 0.000 retained)
|
|
496
|
-
9.000 strings ( 0.000 retained)
|
|
497
|
-
ActsAsTaggableOnArray: 5.211M memsize ( 0.000 retained)
|
|
498
|
-
57.003k objects ( 0.000 retained)
|
|
499
|
-
6.000 strings ( 0.000 retained)
|
|
500
|
-
TagColumns: 6.696M memsize ( 0.000 retained)
|
|
501
|
-
94.003k objects ( 0.000 retained)
|
|
502
|
-
8.000 strings ( 0.000 retained)
|
|
503
|
-
|
|
504
|
-
Finished all benchmarks
|
|
505
|
-
```
|
|
477
|
+
Metka ships a [benchmark suite](benchmark/) comparing it to
|
|
478
|
+
[acts-as-taggable-on](https://github.com/mbleigh/acts-as-taggable-on),
|
|
479
|
+
[acts-as-taggable-array-on](https://github.com/tmiyamon/acts-as-taggable-array-on),
|
|
480
|
+
[gutentag](https://github.com/pat/gutentag) and
|
|
481
|
+
[tag_columns](https://github.com/hopsoft/tag_columns) on a shared dataset:
|
|
482
|
+
10,000 posts per gem, 5 tags per post from a 100-tag vocabulary, identical
|
|
483
|
+
seeded tag assignments. Iterations per second, higher is better (Ruby 4.0,
|
|
484
|
+
Rails 8.1, PostgreSQL 18). The metka numbers are with the recommended
|
|
485
|
+
[Table Strategy](#table-strategy-with-triggers-recommended) aggregate in
|
|
486
|
+
place; tag queries never touch the aggregate, so the query rows apply with
|
|
487
|
+
or without it:
|
|
488
|
+
|
|
489
|
+
| Operation | metka | taggable-array | tag_columns | acts-as-taggable-on | gutentag |
|
|
490
|
+
| --- | --- | --- | --- | --- | --- |
|
|
491
|
+
| Query: ALL of 2 tags, load records | 6,003 | 6,725 | 986 | 2,292 | 1,299 |
|
|
492
|
+
| Query: ANY of 2 tags, count | 4,575 | 4,479 | 678 | 788 | 1,027 |
|
|
493
|
+
| Tag cloud over all posts | 9,025 | 204 | 198 | 127 | 161 |
|
|
494
|
+
| Create post with 5 tags | 1,495 | 1,634 | 1,599 | 204 | 196 |
|
|
495
|
+
| Replace tags of existing post | 8,131 | 7,766 | 7,396 | 190 | 183 |
|
|
496
|
+
| Bulk seed 10k posts | 0.17 s | 0.17 s | 0.16 s | 45.6 s | 50.2 s |
|
|
497
|
+
| Storage, tables + indexes | 2.75 MB | 2.68 MB | 2.68 MB | 18.17 MB | 10.97 MB |
|
|
498
|
+
|
|
499
|
+
The suite also measures what maintaining the aggregate costs on the same
|
|
500
|
+
dataset — reads served from the summary table against the write overhead of
|
|
501
|
+
keeping it fresh, which stays within measurement noise:
|
|
502
|
+
|
|
503
|
+
| Tag-cloud aggregate | Cloud read | Create post | Replace tags | Bulk seed | Storage |
|
|
504
|
+
| --- | --- | --- | --- | --- | --- |
|
|
505
|
+
| none (live aggregation) | 212 | 1,619 | 8,293 | 0.21 s | 2.68 MB |
|
|
506
|
+
| table | 9,025 | 1,495 | 8,131 | 0.17 s | 2.75 MB |
|
|
507
|
+
|
|
508
|
+
### SQLite results
|
|
509
|
+
|
|
510
|
+
The suite also runs on SQLite (`DB=sqlite`), against the gems that support it
|
|
511
|
+
— acts-as-taggable-on and gutentag; acts-as-taggable-array-on and tag_columns
|
|
512
|
+
are PostgreSQL-only and sit this one out. Same dataset and conventions as
|
|
513
|
+
above (Ruby 4.0, Rails 8.1, SQLite 3.53). The metka column has the table
|
|
514
|
+
aggregate in place; metka (index) adds the opt-in
|
|
515
|
+
[index strategy](#index-strategy-sqlite), which changes only how queries are
|
|
516
|
+
answered:
|
|
517
|
+
|
|
518
|
+
| Operation | metka | metka (index) | acts-as-taggable-on | gutentag |
|
|
519
|
+
| --- | --- | --- | --- | --- |
|
|
520
|
+
| Query: ALL of 2 tags, load records | 398 | 7,199 | 3,189 | 1,965 |
|
|
521
|
+
| Query: ANY of 2 tags, count | 379 | 4,113 | 410 | 1,900 |
|
|
522
|
+
| Tag cloud over all posts | 12,697 | — | 111 | 88 |
|
|
523
|
+
| Create post with 5 tags | 6,895 | 5,349 | 268 | 284 |
|
|
524
|
+
| Replace tags of existing post | 12,604 | 12,696 | 416 | 809 |
|
|
525
|
+
| Bulk seed 10k posts | 0.08 s | 0.10 s | 31.4 s | 35.3 s |
|
|
526
|
+
| Storage, tables + indexes | 0.63 MB | 1.39 MB | 12.53 MB | 7.20 MB |
|
|
527
|
+
|
|
528
|
+
By default the read rows flip against metka: SQLite has no GIN equivalent,
|
|
529
|
+
so tag queries are `json_each` table scans (~2.5 ms at 10k rows, growing
|
|
530
|
+
linearly) while the join-table gems keep their ordinary B-tree indexes. The
|
|
531
|
+
index strategy takes the reads back — 18x over the scan and 2.3x faster
|
|
532
|
+
than acts-as-taggable-on — for a modest price: creates run ~1.5x slower
|
|
533
|
+
than bare metka (still ~20x ahead of the join-table gems), tag replacement
|
|
534
|
+
stays within noise, and the side table adds ~0.8 MB per 10k posts. Writes,
|
|
535
|
+
seeding and storage stay with metka by a wide margin in every setup.
|
|
536
|
+
|
|
537
|
+
Keep in mind that these results alone can't prove one solution better than
|
|
538
|
+
the others — each gem has unique features. The join-table gems maintain a
|
|
539
|
+
normalized tag vocabulary (global renames, tag metadata, cross-model tags)
|
|
540
|
+
that array columns don't provide; their storage and write overhead buys those
|
|
541
|
+
features. See [benchmark/README.md](benchmark/README.md) for methodology,
|
|
542
|
+
analysis of the generated SQL and query plans, and instructions for running
|
|
543
|
+
the suite yourself.
|
|
506
544
|
|
|
507
545
|
## Development
|
|
508
546
|
|
|
509
|
-
After checking out the repo, run `bin/setup` to install dependencies. Then
|
|
547
|
+
After checking out the repo, run `bin/setup` to install dependencies and prepare the test databases (a running PostgreSQL server is required; the SQLite database is a file created automatically). Then run `rake test` to run the tests against PostgreSQL, or `DB=sqlite rake test` to run them against SQLite. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
|
510
548
|
|
|
511
549
|
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
|
512
550
|
|
|
513
551
|
## Contributing
|
|
514
552
|
|
|
515
|
-
Bug reports and pull requests are welcome on GitHub at [https://github.com/
|
|
553
|
+
Bug reports and pull requests are welcome on GitHub at [https://github.com/metka-ruby/metka](https://github.com/metka-ruby/metka). This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
|
|
516
554
|
|
|
517
555
|
## Credits
|
|
518
556
|
|
|
519
|
-
|
|
520
|
-
Metka is maintained by [JetRockets](http://www.jetrockets.ru).
|
|
557
|
+
Metka is maintained by [JetRockets](https://jetrockets.com).
|
|
521
558
|
|
|
522
559
|
## License
|
|
523
560
|
|
|
@@ -525,4 +562,4 @@ The gem is available as open source under the terms of the [MIT License](https:/
|
|
|
525
562
|
|
|
526
563
|
## Code of Conduct
|
|
527
564
|
|
|
528
|
-
Everyone interacting in the Metka project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/
|
|
565
|
+
Everyone interacting in the Metka project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/metka-ruby/metka/blob/master/CODE_OF_CONDUCT.md).
|