mensa 0.6.19 → 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +1 -1
- data/.shipkit.yml +1 -1
- data/CHANGELOG.md +32 -0
- data/Gemfile.lock +1 -1
- data/README.md +50 -11
- data/app/tables/mensa/base.rb +9 -11
- data/app/tables/mensa/filter.rb +21 -4
- data/docs/columns.png +0 -0
- data/docs/export.png +0 -0
- data/docs/filters1.png +0 -0
- data/docs/filters2.png +0 -0
- data/docs/table.png +0 -0
- data/lib/mensa/verifier/result.rb +5 -2
- data/lib/mensa/verifier.rb +159 -9
- data/lib/mensa/version.rb +1 -1
- data/lib/tasks/mensa.rake +7 -4
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: c9016fbd279d8168641f18891986045e07f26b124842610e22728268d3f7557e
|
|
4
|
+
data.tar.gz: 9ca2ab7decb8b7a5d901821a3d129c94ae3a43f20bd981c1d7cb1d0ab68cb402
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: dbcd2b404c1da8e152ac4ad555a3a95093a0a33bafb3f809f9f099c2b21b21028667b2dae708ffae57dcd9d0e9db2fd539de2baea1110e0628d536b01d9aab49
|
|
7
|
+
data.tar.gz: 49e1a2b30f5efd72d913d22981aad389116d3935dedd4245ffebbc87637222a1f3bca58850bbbd094461b59b29bce9400e9a94867608195d96fc2ecf4773313b
|
data/.gitignore
CHANGED
data/.shipkit.yml
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
1
|
---
|
|
2
|
-
previous_version: 0.
|
|
2
|
+
previous_version: 0.7.0
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,37 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## Unreleased
|
|
4
|
+
|
|
5
|
+
- The verifier's report only lists failures and warnings. `-v`/`--verbose` (or
|
|
6
|
+
`VERBOSE=1`) on the rake tasks, or `result.to_s(verbose: true)`, lists every
|
|
7
|
+
check.
|
|
8
|
+
|
|
9
|
+
## 0.7.0
|
|
10
|
+
|
|
11
|
+
- Filters only offer the operators that work for the column's type in SQL,
|
|
12
|
+
picking one no longer raises. `matches`/`does_not_match` (`LIKE`) are only
|
|
13
|
+
offered for string, text, citext and untyped attribute columns, so boolean,
|
|
14
|
+
float, decimal, uuid, time and other columns no longer get them.
|
|
15
|
+
`gt`/`gteq`/`lt`/`lteq` are only offered for number, date and time columns,
|
|
16
|
+
so boolean, uuid and text columns no longer get them. json/jsonb columns only
|
|
17
|
+
get `is_empty`/`isnt_empty` (and jsonb `is_duplicate`). Host apps no longer
|
|
18
|
+
need to patch `Mensa::Filter#operators` or list `operators` per column to
|
|
19
|
+
avoid these. An explicit `operators` list is used as given.
|
|
20
|
+
- Time columns use a time input, and a filter value that isn't a time matches
|
|
21
|
+
nothing instead of raising.
|
|
22
|
+
- The verifier warns when an `attribute` has no `type` but its SQL isn't text,
|
|
23
|
+
e.g. a boolean `EXISTS (...)`, and says which `type` to add.
|
|
24
|
+
- Columns a `column_order` doesn't list are kept and shown after the listed
|
|
25
|
+
ones, instead of being dropped. Browser state or views saved before a column
|
|
26
|
+
was added no longer hide that column (also from the column customizer),
|
|
27
|
+
ignore filters and sorting on it, or leave its attribute unselected.
|
|
28
|
+
- The verifier checks column order and visibility: a reversed order, an order
|
|
29
|
+
that lists only one column (like state saved before columns were added),
|
|
30
|
+
hiding each column and hiding all of them, each rendered and exported.
|
|
31
|
+
- The verifier creates a custom view like a user saving the table's state
|
|
32
|
+
would, without saving it, and validates, replays and exports it.
|
|
33
|
+
- The verifier confirms when it found no N+1 queries.
|
|
34
|
+
|
|
3
35
|
## 0.6.19
|
|
4
36
|
|
|
5
37
|
- `Mensa::Verifier` checks a table against the database: scope, column SQL,
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
|
@@ -4,7 +4,8 @@ Fast and awesome tables, with pagination, sorting, filtering, batch processing,
|
|
|
4
4
|
Due to search, it only works with postgresql at the moment.
|
|
5
5
|
|
|
6
6
|

|
|
7
|
-

|
|
8
|
+

|
|
8
9
|

|
|
9
10
|

|
|
10
11
|
|
|
@@ -175,7 +176,31 @@ end
|
|
|
175
176
|
```
|
|
176
177
|
|
|
177
178
|
The operators are `is`, `isnt`, `matches`, `does_not_match`, `gt`, `gteq`, `lt`,
|
|
178
|
-
`lteq`, `is_duplicate`, `is_current`, `is_empty` and `isnt_empty`.
|
|
179
|
+
`lteq`, `is_duplicate`, `is_current`, `is_empty` and `isnt_empty`. Without an
|
|
180
|
+
`operators` list, a column gets the ones that work for its type in SQL:
|
|
181
|
+
|
|
182
|
+
| Column type | Operators besides `is_duplicate`, `is_empty` and `isnt_empty` |
|
|
183
|
+
|---|---|
|
|
184
|
+
| string, text, citext, or an `attribute` without `type` | `is`, `isnt`, `matches`, `does_not_match` |
|
|
185
|
+
| integer, float, decimal, time | `is`, `isnt`, `gt`, `gteq`, `lt`, `lteq` |
|
|
186
|
+
| date, datetime | `gt`, `gteq`, `lt`, `lteq` |
|
|
187
|
+
| boolean, uuid, enums and other types | `is`, `isnt` |
|
|
188
|
+
| jsonb, json | none (json also has no `is_duplicate`) |
|
|
189
|
+
|
|
190
|
+
`matches` and `does_not_match` are left out when the filter has a `collection`, and
|
|
191
|
+
`is_current` is only offered when `Current` has a method named after the column.
|
|
192
|
+
|
|
193
|
+
Mensa reads the type from the database column. Give an `attribute` that isn't text a
|
|
194
|
+
`type`, otherwise it's treated as text and gets the text operators:
|
|
195
|
+
|
|
196
|
+
```ruby
|
|
197
|
+
column(:has_orders) do
|
|
198
|
+
attribute "EXISTS (SELECT 1 FROM orders WHERE orders.user_id = users.id)"
|
|
199
|
+
type :boolean
|
|
200
|
+
end
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
An `operators` list is used as given, also when it lists operators the type can't run.
|
|
179
204
|
|
|
180
205
|
#### Custom filter scopes
|
|
181
206
|
|
|
@@ -382,13 +407,20 @@ real database, using the same code paths as requests do:
|
|
|
382
407
|
- **Exports**: `Mensa::ExportJob` generates the first page of the export, also for
|
|
383
408
|
each view and as a password protected zip. It needs the expected headers
|
|
384
409
|
(skipped for `exportable false`).
|
|
410
|
+
- **Column order and visibility**: the columns shown, rendered and exported in a
|
|
411
|
+
reversed order, with an order that lists only one column (like browser state or a
|
|
412
|
+
view saved before the other columns were added, which must still show them all),
|
|
413
|
+
with each column hidden and with all of them hidden.
|
|
385
414
|
- **Combined**: search, a filter on every column, sorting on every column and
|
|
386
415
|
grouping with aggregates, all at once.
|
|
387
416
|
- **Grouping**: every groupable column and every aggregate, rendered with their
|
|
388
417
|
group headers. A `group_by` or `aggregates` naming a column that doesn't allow
|
|
389
418
|
it fails, because Mensa would silently ignore it.
|
|
390
419
|
- **Views**: system views load, render and only refer to existing columns. With
|
|
391
|
-
custom views enabled, saved views load too
|
|
420
|
+
custom views enabled, saved views load too, and a custom view is created like a
|
|
421
|
+
user saving the table's state would (search, filters, sorting, column order, a
|
|
422
|
+
hidden column and grouping), then validated, replayed and exported. It's never
|
|
423
|
+
saved, and needs a user to own it (`user:`).
|
|
392
424
|
|
|
393
425
|
It also warns, without failing:
|
|
394
426
|
|
|
@@ -396,7 +428,11 @@ It also warns, without failing:
|
|
|
396
428
|
each record. The warning shows the query and how to avoid it for the association
|
|
397
429
|
it loads, e.g. preload it with `includes` (and `internal :customer_id`, since Mensa
|
|
398
430
|
only selects the table's columns), select the value in SQL with `attribute`, or
|
|
399
|
-
count in SQL.
|
|
431
|
+
count in SQL. When none is found, the report confirms it under Rendering, e.g.
|
|
432
|
+
`✓ N+1 queries: none in 18 renders of 7 sampled records`. It needs at least two
|
|
433
|
+
records to tell.
|
|
434
|
+
- **Untyped attributes**: an `attribute` without `type` whose SQL isn't text, e.g. a
|
|
435
|
+
boolean `EXISTS (...)`. The warning names the `type` to add.
|
|
400
436
|
- **Slow queries**: queries that take `config.verifier_slow_query` seconds (1.0 by
|
|
401
437
|
default) or more.
|
|
402
438
|
|
|
@@ -413,9 +449,16 @@ bin/rails "mensa:verify[inventories]" # or InventoriesTable
|
|
|
413
449
|
bin/rails mensa:verify_all # every table in app/tables
|
|
414
450
|
```
|
|
415
451
|
|
|
416
|
-
The tasks print a report
|
|
417
|
-
exit non-zero when a check failed.
|
|
418
|
-
|
|
452
|
+
The tasks print a report of the failures (✗, with the error and SQL) and warnings (!)
|
|
453
|
+
and exit non-zero when a check failed. Add `-v`/`--verbose` (or `VERBOSE=1`) to list
|
|
454
|
+
every check (✓ passed, - skipped):
|
|
455
|
+
|
|
456
|
+
```sh
|
|
457
|
+
bin/rails "mensa:verify[inventories]" -v
|
|
458
|
+
```
|
|
459
|
+
|
|
460
|
+
Classes and modules that don't descend from `Mensa::Base` (e.g. helpers in
|
|
461
|
+
`app/tables`) are reported as skipped.
|
|
419
462
|
|
|
420
463
|
#### From tests
|
|
421
464
|
|
|
@@ -469,11 +512,7 @@ end
|
|
|
469
512
|
|
|
470
513
|
## Contributing
|
|
471
514
|
|
|
472
|
-
```
|
|
473
515
|
Contribution directions go here.
|
|
474
516
|
|
|
475
|
-
|
|
476
|
-
|
|
477
517
|
## License
|
|
478
518
|
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
|
479
|
-
```
|
data/app/tables/mensa/base.rb
CHANGED
|
@@ -38,18 +38,16 @@ module Mensa
|
|
|
38
38
|
order = order&.map(&:to_sym)
|
|
39
39
|
return order if order.nil?
|
|
40
40
|
|
|
41
|
-
# Internal columns are never shown in the column customizer UI, so they
|
|
42
|
-
# are absent from any URL-supplied column_order. Always append them so
|
|
43
|
-
# that columns and selected_scope include their attributes.
|
|
44
41
|
all_keys = (config[:columns]&.keys || []).map(&:to_sym)
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
42
|
+
return order if all_keys.empty?
|
|
43
|
+
|
|
44
|
+
# A column_order can come from browser state or a view saved for
|
|
45
|
+
# different params, or before columns were added, so drop names that
|
|
46
|
+
# aren't columns here and append the columns it doesn't list. Internal
|
|
47
|
+
# columns are never in it, as the column customizer doesn't show them.
|
|
48
|
+
# Appending keeps every column in columns and selected_scope, so no
|
|
49
|
+
# column disappears and their filters and attributes keep working.
|
|
50
|
+
(order & all_keys) | all_keys
|
|
53
51
|
end
|
|
54
52
|
|
|
55
53
|
# Returns all columns
|
data/app/tables/mensa/filter.rb
CHANGED
|
@@ -14,6 +14,15 @@ module Mensa
|
|
|
14
14
|
config_reader :multiple
|
|
15
15
|
config_reader :having?
|
|
16
16
|
|
|
17
|
+
# Types that LIKE works on, nil is an attribute without a type
|
|
18
|
+
TEXT_TYPES = [:string, :text, :citext, nil].freeze
|
|
19
|
+
# Types with a meaningful order, for gt, gteq, lt and lteq
|
|
20
|
+
RANGE_TYPES = %i[integer bigint float decimal money date datetime timestamp timestamptz time interval].freeze
|
|
21
|
+
# Types where an exact match on a value is of little use
|
|
22
|
+
POINT_IN_TIME_TYPES = %i[date datetime timestamp timestamptz].freeze
|
|
23
|
+
# Types that can't be compared with a value the user types
|
|
24
|
+
STRUCTURED_TYPES = %i[json jsonb].freeze
|
|
25
|
+
|
|
17
26
|
class << self
|
|
18
27
|
def OPERATORS
|
|
19
28
|
[
|
|
@@ -102,10 +111,15 @@ module Mensa
|
|
|
102
111
|
operators = operators.select { |op| config[:operators].include?(op[0]) }
|
|
103
112
|
else
|
|
104
113
|
operators.delete_if { |op| op[0] == :is_current } unless Current.method_defined?(column.name, false)
|
|
114
|
+
type = column.type
|
|
105
115
|
operators.delete_if { |op| [:matches, :does_not_match].include?(op[0]) } if collection.present?
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
operators.delete_if { |op| [:
|
|
116
|
+
# Only these operators work for the column's type in SQL, the others
|
|
117
|
+
# raise, e.g. LIKE on a boolean
|
|
118
|
+
operators.delete_if { |op| [:matches, :does_not_match].include?(op[0]) } unless TEXT_TYPES.include?(type)
|
|
119
|
+
operators.delete_if { |op| [:gt, :lt, :gteq, :lteq].include?(op[0]) } unless RANGE_TYPES.include?(type)
|
|
120
|
+
operators.delete_if { |op| [:is, :isnt].include?(op[0]) } if POINT_IN_TIME_TYPES.include?(type) || STRUCTURED_TYPES.include?(type)
|
|
121
|
+
# json has no equality, so duplicates can't be grouped
|
|
122
|
+
operators.delete_if { |op| op[0] == :is_duplicate } if type == :json
|
|
109
123
|
end
|
|
110
124
|
operators
|
|
111
125
|
end
|
|
@@ -162,6 +176,8 @@ module Mensa
|
|
|
162
176
|
"date"
|
|
163
177
|
when :datetime
|
|
164
178
|
"datetime-local"
|
|
179
|
+
when :time
|
|
180
|
+
"time"
|
|
165
181
|
else
|
|
166
182
|
"text"
|
|
167
183
|
end
|
|
@@ -242,7 +258,8 @@ module Mensa
|
|
|
242
258
|
when :integer, :bigint then entry.strip.match?(/\A[+-]?\d+\z/)
|
|
243
259
|
when :decimal, :float then entry.strip.match?(/\A[+-]?(\d+\.?\d*|\.\d+)\z/)
|
|
244
260
|
when :date then Date.parse(entry).present?
|
|
245
|
-
when :datetime, :timestamp then Time.zone.parse(entry).present?
|
|
261
|
+
when :datetime, :timestamp, :timestamptz then Time.zone.parse(entry).present?
|
|
262
|
+
when :time then entry.strip.match?(/\A\d{1,2}:\d{2}(:\d{2}(\.\d+)?)?\z/)
|
|
246
263
|
when :uuid then entry.strip.match?(UUID_FORMAT)
|
|
247
264
|
else true
|
|
248
265
|
end
|
data/docs/columns.png
CHANGED
|
Binary file
|
data/docs/export.png
CHANGED
|
Binary file
|
data/docs/filters1.png
ADDED
|
Binary file
|
data/docs/filters2.png
ADDED
|
Binary file
|
data/docs/table.png
CHANGED
|
Binary file
|
|
@@ -36,6 +36,7 @@ module Mensa
|
|
|
36
36
|
filters: "Filters",
|
|
37
37
|
search: "Search",
|
|
38
38
|
render: "Rendering",
|
|
39
|
+
layout: "Column order and visibility",
|
|
39
40
|
combined: "Combined",
|
|
40
41
|
exports: "Exports",
|
|
41
42
|
grouping: "Grouping",
|
|
@@ -91,12 +92,14 @@ module Mensa
|
|
|
91
92
|
[table_class.name || table_class.to_s, ("(#{label})" if label)].compact.join(" ")
|
|
92
93
|
end
|
|
93
94
|
|
|
94
|
-
|
|
95
|
+
# Lists the failures and warnings, +verbose+ lists every check
|
|
96
|
+
def to_s(verbose: false)
|
|
95
97
|
return "#{title}\n - skipped: #{checks.first&.message}" unless mensa_table?
|
|
96
98
|
|
|
97
99
|
lines = [title]
|
|
100
|
+
shown = verbose ? checks : checks.select { |check| check.failed? || check.warning? }
|
|
98
101
|
|
|
99
|
-
|
|
102
|
+
shown.group_by(&:section).each do |section, section_checks|
|
|
100
103
|
lines << " #{SECTIONS.fetch(section, section.to_s.humanize)}"
|
|
101
104
|
section_checks.each do |check|
|
|
102
105
|
lines << " #{SYMBOLS[check.status]} #{check.label}#{" (skipped: #{check.message})" if check.skipped?}"
|
data/lib/mensa/verifier.rb
CHANGED
|
@@ -94,12 +94,15 @@ module Mensa
|
|
|
94
94
|
# their reports to +io+ and returns true when all of them passed.
|
|
95
95
|
# Context comes from USER_ID, ACCOUNT_ID and PARAMS (JSON) in +env+, and
|
|
96
96
|
# STRICT=1 makes warnings fail, used by the mensa:verify rake tasks.
|
|
97
|
-
|
|
97
|
+
# The reports list failures and warnings, +verbose+ (or VERBOSE=1) lists
|
|
98
|
+
# every check.
|
|
99
|
+
def run(tables = nil, env: ENV, io: $stdout, verbose: false)
|
|
98
100
|
classes = tables.present? ? Array(tables).map { |table| resolve_table_class(table) } : table_classes
|
|
99
101
|
options = options_from_env(env)
|
|
102
|
+
verbose ||= %w[1 true yes].include?(env["VERBOSE"].to_s.downcase)
|
|
100
103
|
|
|
101
104
|
results = classes.flat_map do |klass|
|
|
102
|
-
verify(klass, **options).each { |result| io.puts(result, "") }
|
|
105
|
+
verify(klass, **options).each { |result| io.puts(result.to_s(verbose: verbose), "") }
|
|
103
106
|
end
|
|
104
107
|
|
|
105
108
|
by_table = results.select(&:mensa_table?).group_by(&:table_class)
|
|
@@ -149,6 +152,7 @@ module Mensa
|
|
|
149
152
|
|
|
150
153
|
@queries = []
|
|
151
154
|
@reported_n_plus_one = Set.new
|
|
155
|
+
@per_record_renders = 0
|
|
152
156
|
thread = Thread.current
|
|
153
157
|
ActiveSupport::Notifications.subscribed(->(event) { record_query(event) if Thread.current == thread }, "sql.active_record") do
|
|
154
158
|
read_only_transaction do
|
|
@@ -217,6 +221,8 @@ module Mensa
|
|
|
217
221
|
verify_filters
|
|
218
222
|
verify_search
|
|
219
223
|
verify_rendering
|
|
224
|
+
verify_n_plus_one
|
|
225
|
+
verify_layout
|
|
220
226
|
verify_combined
|
|
221
227
|
verify_exports
|
|
222
228
|
verify_grouping
|
|
@@ -373,6 +379,8 @@ module Mensa
|
|
|
373
379
|
def warn_n_plus_one(section, subject, what, queries_per_record)
|
|
374
380
|
return if queries_per_record.size < 2
|
|
375
381
|
|
|
382
|
+
@per_record_renders += 1
|
|
383
|
+
|
|
376
384
|
queries_per_record.flatten.tally.each do |sql, records|
|
|
377
385
|
next if records < 2
|
|
378
386
|
next unless @reported_n_plus_one.add?([subject, sql])
|
|
@@ -458,10 +466,24 @@ module Mensa
|
|
|
458
466
|
selects = (table.send(:id_column?) ? [:id] : []) + table.columns.select(&:internal?).filter_map(&:attribute)
|
|
459
467
|
relation = table.filtered_scope.select(*selects, Arel.sql(column.attribute)).limit(1)
|
|
460
468
|
c.sql = relation.to_sql
|
|
461
|
-
relation
|
|
469
|
+
result = table.model.connection.select_all(relation)
|
|
470
|
+
warn_untyped_attribute(column, result.column_types[column.name.to_s]&.type)
|
|
462
471
|
end
|
|
463
472
|
end
|
|
464
473
|
|
|
474
|
+
# The SQL type of an `attribute` decides which filter operators and values
|
|
475
|
+
# fit it. Without `type`, Mensa treats it as text: it offers LIKE and
|
|
476
|
+
# compares with text, which fails for e.g. a boolean expression.
|
|
477
|
+
def warn_untyped_attribute(column, sql_type)
|
|
478
|
+
return if column.config[:attribute].blank? || column.config[:type].present?
|
|
479
|
+
return if sql_type.nil? || Mensa::Filter::TEXT_TYPES.include?(sql_type)
|
|
480
|
+
|
|
481
|
+
type = {timestamptz: :datetime, timestamp: :datetime}.fetch(sql_type, sql_type)
|
|
482
|
+
warn!(:columns, column.name, "untyped attribute",
|
|
483
|
+
"its SQL is #{sql_type}, but without `type` Mensa treats it as text: filters offer text operators and compare with text.\n" \
|
|
484
|
+
"Fix: add `type :#{type}` to the column.")
|
|
485
|
+
end
|
|
486
|
+
|
|
465
487
|
def verify_column_sort(column)
|
|
466
488
|
check(:columns, column.name, "sort asc/desc") do
|
|
467
489
|
%i[asc desc].each { |direction| build_table(order: {column.name => direction}).ordered_scope.limit(1).load }
|
|
@@ -567,7 +589,8 @@ module Mensa
|
|
|
567
589
|
case column.type
|
|
568
590
|
when :integer, :bigint, :decimal, :float then "1"
|
|
569
591
|
when :date then Date.current.iso8601
|
|
570
|
-
when :datetime, :timestamp, :
|
|
592
|
+
when :datetime, :timestamp, :timestamptz then Time.current.strftime("%Y-%m-%dT%H:%M")
|
|
593
|
+
when :time then "12:00"
|
|
571
594
|
when :boolean then "true"
|
|
572
595
|
when :uuid then SecureRandom.uuid
|
|
573
596
|
else "a"
|
|
@@ -578,7 +601,8 @@ module Mensa
|
|
|
578
601
|
def invalid_value(column)
|
|
579
602
|
case column.type
|
|
580
603
|
when :integer, :bigint, :decimal, :float then "not a number"
|
|
581
|
-
when :date, :datetime, :timestamp then "not a date"
|
|
604
|
+
when :date, :datetime, :timestamp, :timestamptz then "not a date"
|
|
605
|
+
when :time then "not a time"
|
|
582
606
|
when :uuid then "not a uuid"
|
|
583
607
|
end
|
|
584
608
|
end
|
|
@@ -643,6 +667,21 @@ module Mensa
|
|
|
643
667
|
end
|
|
644
668
|
end
|
|
645
669
|
|
|
670
|
+
# Confirms no N+1 query was found, after everything rendered per record
|
|
671
|
+
# (column renders, the row link, row actions). Found ones are warnings.
|
|
672
|
+
def verify_n_plus_one
|
|
673
|
+
return if @reported_n_plus_one.any?
|
|
674
|
+
|
|
675
|
+
check(:render, nil, "N+1 queries") do |c|
|
|
676
|
+
skip!("records could not be loaded") if @samples.nil?
|
|
677
|
+
skip!("no records") if @samples.empty?
|
|
678
|
+
skip!("needs at least 2 sampled records, found 1") if @samples.one?
|
|
679
|
+
skip!("nothing is rendered per record") if @per_record_renders.zero?
|
|
680
|
+
|
|
681
|
+
c.name = "N+1 queries: none in #{@per_record_renders} #{"render".pluralize(@per_record_renders)} of #{@samples.size} sampled records"
|
|
682
|
+
end
|
|
683
|
+
end
|
|
684
|
+
|
|
646
685
|
# What Mensa::TablesController#show renders into the table's turbo frame
|
|
647
686
|
def render_page(table, view: nil)
|
|
648
687
|
table.table_view = view
|
|
@@ -650,6 +689,57 @@ module Mensa
|
|
|
650
689
|
view_context.render(Mensa::View::Component.new(table))
|
|
651
690
|
end
|
|
652
691
|
|
|
692
|
+
# Column order and visibility, as users change them with the column
|
|
693
|
+
# customizer, or as saved views and browser state replay them
|
|
694
|
+
|
|
695
|
+
def verify_layout
|
|
696
|
+
names = @table.display_columns.map(&:name)
|
|
697
|
+
|
|
698
|
+
check(:layout, nil, "reversed column order") do
|
|
699
|
+
skip!("no columns are shown") if names.empty?
|
|
700
|
+
|
|
701
|
+
verify_layout_config({column_order: names.reverse}, names.reverse)
|
|
702
|
+
end
|
|
703
|
+
|
|
704
|
+
check(:layout, nil, "column order listing only :#{names.last} (e.g. saved before the others were added)") do
|
|
705
|
+
skip!("fewer than 2 columns are shown") if names.size < 2
|
|
706
|
+
|
|
707
|
+
verify_layout_config({column_order: [names.last]}, [names.last] + names[0..-2],
|
|
708
|
+
hint: "columns a column_order doesn't list should follow the listed ones")
|
|
709
|
+
end
|
|
710
|
+
|
|
711
|
+
check(:layout, nil, "hide each column (#{names.size})") do
|
|
712
|
+
skip!("no columns are shown") if names.empty?
|
|
713
|
+
|
|
714
|
+
errors = names.filter_map do |name|
|
|
715
|
+
savepoint { verify_layout_config({hidden_columns: [name]}, names - [name]) }
|
|
716
|
+
nil
|
|
717
|
+
rescue StandardError, ScriptError => e
|
|
718
|
+
":#{name}: #{e.is_a?(Failure) ? e.message : "#{e.class}: #{e.message}"}"
|
|
719
|
+
end
|
|
720
|
+
raise Failure, errors.join("\n") if errors.any?
|
|
721
|
+
end
|
|
722
|
+
|
|
723
|
+
check(:layout, nil, "hide all columns") do
|
|
724
|
+
skip!("no columns are shown") if names.empty?
|
|
725
|
+
|
|
726
|
+
verify_layout_config({hidden_columns: names}, [])
|
|
727
|
+
end
|
|
728
|
+
end
|
|
729
|
+
|
|
730
|
+
# Builds the table with +config+, checks the columns it shows, renders it
|
|
731
|
+
# and exports it
|
|
732
|
+
def verify_layout_config(config, expected, hint: nil)
|
|
733
|
+
table = build_table(config)
|
|
734
|
+
shown = table.display_columns.map(&:name)
|
|
735
|
+
unless shown == expected
|
|
736
|
+
raise Failure, "shows columns #{shown.inspect}, expected #{expected.inspect}#{"\n#{hint}" if hint}"
|
|
737
|
+
end
|
|
738
|
+
|
|
739
|
+
render_page(table)
|
|
740
|
+
verify_export(config, columns_required: expected.any?) if @table.exportable?
|
|
741
|
+
end
|
|
742
|
+
|
|
653
743
|
# Everything a user can combine at once: search, a filter on every
|
|
654
744
|
# column, sorting on every column and grouping with aggregates
|
|
655
745
|
def verify_combined
|
|
@@ -711,7 +801,7 @@ module Mensa
|
|
|
711
801
|
@table.export_with_password? ? "csv (zip with password)" : "csv"
|
|
712
802
|
end
|
|
713
803
|
|
|
714
|
-
def verify_export(config, table_view: nil)
|
|
804
|
+
def verify_export(config, table_view: nil, columns_required: true)
|
|
715
805
|
config = JSON.parse({params: params.presence, page: 1}.compact.merge(config).to_json)
|
|
716
806
|
export = Mensa::Export.new(
|
|
717
807
|
table_name: table_name, table_view_id: table_view&.id, format: "plain_csv", scope: "current_page",
|
|
@@ -724,12 +814,14 @@ module Mensa
|
|
|
724
814
|
data = file.read
|
|
725
815
|
file.close!
|
|
726
816
|
csv = (content_type == "application/zip") ? unzip(data, export.password) : data
|
|
727
|
-
raise Failure, "#{filename} is empty" if csv.blank?
|
|
817
|
+
raise Failure, "#{filename} is empty" if csv.blank? && columns_required
|
|
728
818
|
|
|
729
819
|
expected = build_table((table_view&.config || {}).deep_symbolize_keys.merge(config.deep_symbolize_keys))
|
|
730
820
|
headers = expected.display_columns.map { |column| column.name.to_s }
|
|
731
821
|
lines = CSV.parse(csv.force_encoding(Encoding::UTF_8).delete_prefix(""))
|
|
732
|
-
raise Failure, "the export has no columns" if headers.empty?
|
|
822
|
+
raise Failure, "the export has no columns" if headers.empty? && columns_required
|
|
823
|
+
return if headers.empty?
|
|
824
|
+
|
|
733
825
|
raise Failure, "the export has headers #{lines.first.inspect}, expected #{headers.inspect}" unless lines.first == headers
|
|
734
826
|
|
|
735
827
|
records = expected.rows.size
|
|
@@ -812,13 +904,71 @@ module Mensa
|
|
|
812
904
|
end
|
|
813
905
|
end
|
|
814
906
|
|
|
815
|
-
|
|
907
|
+
unless @table.supports_custom_views?
|
|
908
|
+
check(:views, "custom view", "create") { skip!("supports_custom_views is off") }
|
|
909
|
+
return
|
|
910
|
+
end
|
|
911
|
+
|
|
912
|
+
verify_custom_view
|
|
816
913
|
|
|
817
914
|
saved_views.each do |view|
|
|
818
915
|
check(:views, "saved view #{view.id}", view.name.to_s) { render_page(build_table((view.config || {}).deep_symbolize_keys), view: view) }
|
|
819
916
|
end
|
|
820
917
|
end
|
|
821
918
|
|
|
919
|
+
# Creates a custom view like a user saving the table's state would, but
|
|
920
|
+
# doesn't save it: it's validated, then replayed and exported like a saved
|
|
921
|
+
# view.
|
|
922
|
+
def verify_custom_view
|
|
923
|
+
check(:views, "custom view", "create and replay (not saved)") do |c|
|
|
924
|
+
skip!("custom views are owned by Current.user, give the verifier a user:") if @table.current_user.blank?
|
|
925
|
+
|
|
926
|
+
view = custom_view
|
|
927
|
+
raise Failure, "the view is invalid: #{view.errors.full_messages.to_sentence}" unless view.valid?
|
|
928
|
+
|
|
929
|
+
c.name = "create and replay #{view.config.keys.join(", ")} (not saved)"
|
|
930
|
+
# Like Mensa::TablesController#show and the views controller
|
|
931
|
+
render_page(build_table(view.config.deep_transform_keys(&:to_sym)), view: view)
|
|
932
|
+
end
|
|
933
|
+
|
|
934
|
+
return unless @table.exportable? && @table.current_user.present?
|
|
935
|
+
|
|
936
|
+
check(:exports, "custom view", export_format) do
|
|
937
|
+
verify_export(custom_view.config.deep_symbolize_keys.slice(:filters, :column_order, :hidden_columns, :group_by, :order))
|
|
938
|
+
end
|
|
939
|
+
end
|
|
940
|
+
|
|
941
|
+
# A view as Mensa::Tables::ViewsController#create builds it from the
|
|
942
|
+
# table's state in the browser
|
|
943
|
+
def custom_view
|
|
944
|
+
Mensa::TableView.new(
|
|
945
|
+
table_name: table_name,
|
|
946
|
+
name: "Mensa::Verifier",
|
|
947
|
+
description: "Created by Mensa::Verifier, not saved",
|
|
948
|
+
config: JSON.parse(custom_view_config.to_json),
|
|
949
|
+
user: @table.current_user
|
|
950
|
+
)
|
|
951
|
+
end
|
|
952
|
+
|
|
953
|
+
def custom_view_config
|
|
954
|
+
@custom_view_config ||= begin
|
|
955
|
+
columns = @table.display_columns
|
|
956
|
+
sortable = columns.find(&:sortable?)
|
|
957
|
+
groupable = @table.columns.find(&:groupable?)
|
|
958
|
+
|
|
959
|
+
config = {query: search_term, filters: combined_filters.first(2).to_h}
|
|
960
|
+
config[:order] = {sortable.name => :desc} if sortable
|
|
961
|
+
config[:column_order] = columns.map(&:name).reverse
|
|
962
|
+
config[:hidden_columns] = [columns.last.name] if columns.size > 1
|
|
963
|
+
if groupable && @table.scope.is_a?(ActiveRecord::Relation)
|
|
964
|
+
config[:group_by] = groupable.name
|
|
965
|
+
aggregatable = @table.columns.reject(&:internal?).find(&:aggregatable?)
|
|
966
|
+
config[:aggregates] = {aggregatable.name => aggregatable.aggregates.first} if aggregatable
|
|
967
|
+
end
|
|
968
|
+
config
|
|
969
|
+
end
|
|
970
|
+
end
|
|
971
|
+
|
|
822
972
|
def saved_views
|
|
823
973
|
Mensa::TableView.where(table_name: table_name, user: [nil, @table.current_user].uniq).order(:created_at).limit(25).to_a
|
|
824
974
|
end
|
data/lib/mensa/version.rb
CHANGED
data/lib/tasks/mensa.rake
CHANGED
|
@@ -1,13 +1,16 @@
|
|
|
1
1
|
namespace :mensa do
|
|
2
|
-
|
|
2
|
+
# -v/--verbose sets Rake.verbose to true, it's a default object otherwise
|
|
3
|
+
verbose = -> { Rake.verbose == true }
|
|
4
|
+
|
|
5
|
+
desc "Verify a Mensa table against the database, e.g. mensa:verify[users] (context: USER_ID=, ACCOUNT_ID=, PARAMS='{...}', STRICT=1 fails on warnings, -v or VERBOSE=1 lists every check)"
|
|
3
6
|
task :verify, [:table] => :environment do |_task, args|
|
|
4
7
|
abort "Usage: bin/rails \"mensa:verify[table_name]\"" if args[:table].blank?
|
|
5
8
|
|
|
6
|
-
exit(1) unless Mensa::Verifier.run(args[:table])
|
|
9
|
+
exit(1) unless Mensa::Verifier.run(args[:table], verbose: verbose.call)
|
|
7
10
|
end
|
|
8
11
|
|
|
9
|
-
desc "Verify all Mensa tables in app/tables against the database (context: USER_ID=, ACCOUNT_ID=, PARAMS='{...}', STRICT=1 fails on warnings)"
|
|
12
|
+
desc "Verify all Mensa tables in app/tables against the database (context: USER_ID=, ACCOUNT_ID=, PARAMS='{...}', STRICT=1 fails on warnings, -v or VERBOSE=1 lists every check)"
|
|
10
13
|
task verify_all: :environment do
|
|
11
|
-
exit(1) unless Mensa::Verifier.run
|
|
14
|
+
exit(1) unless Mensa::Verifier.run(verbose: verbose.call)
|
|
12
15
|
end
|
|
13
16
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: mensa
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.7.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Tom de Grunt
|
|
@@ -314,6 +314,8 @@ files:
|
|
|
314
314
|
- docs/columns.png
|
|
315
315
|
- docs/export.png
|
|
316
316
|
- docs/filters.png
|
|
317
|
+
- docs/filters1.png
|
|
318
|
+
- docs/filters2.png
|
|
317
319
|
- docs/table.png
|
|
318
320
|
- lib/generators/mensa/install_generator.rb
|
|
319
321
|
- lib/generators/mensa/table_generator.rb
|