panko_serializer 0.3.2 → 0.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/.gitignore +4 -0
- data/.travis.yml +15 -2
- data/README.md +9 -86
- data/benchmarks/bm_active_model_serializers.rb +2 -2
- data/benchmarks/bm_panko_json.rb +1 -1
- data/benchmarks/bm_serializer_resolver.rb +21 -0
- data/docs/README.md +11 -0
- data/docs/associations.md +76 -0
- data/docs/attributes.md +104 -0
- data/docs/design-choices.md +122 -0
- data/docs/docpress.json +4 -0
- data/docs/getting-started.md +47 -0
- data/docs/performance.md +35 -0
- data/docs/response-bag.md +53 -0
- data/ext/panko_serializer/association.c +6 -0
- data/lib/panko/serialization_descriptor.rb +19 -24
- data/lib/panko/serializer.rb +12 -2
- data/lib/panko/serializer_resolver.rb +29 -0
- data/lib/panko/version.rb +1 -1
- data/lib/panko_serializer.rb +1 -0
- metadata +13 -4
- data/_config.yml +0 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: eef3c77d8a03018af85326b4c07890d6d543909925c664f6baf6f20f634033e2
|
4
|
+
data.tar.gz: 28c88dea3cd021f1ba3cb5e31b21e03f3ddb9cdbf4851cccafa78770d742a994
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: edbe1e16a0216d8cd9e8af32289e6b173565ad7d1b50c3177f90a2cc683fd00d26cf7248e0d563e4a76916e481d0723b23041fe6973c5df05b180227e1da82fe
|
7
|
+
data.tar.gz: d0883a7ba84ff19d57738d56846ca9fc9b4ed70d6343c6280225ccfa83abe6489d611b64cea4be294ce4ddccfc8e992be55545fd6ff18bf65936fae91db94ce3
|
data/.gitignore
CHANGED
data/.travis.yml
CHANGED
@@ -3,9 +3,22 @@ cache: bundler
|
|
3
3
|
language: ruby
|
4
4
|
rvm:
|
5
5
|
- 2.4.2
|
6
|
+
|
7
|
+
env:
|
8
|
+
global:
|
9
|
+
- GIT_NAME: Travis CI
|
10
|
+
- GIT_EMAIL: nobody@nobody.org
|
11
|
+
|
6
12
|
install: bundle install --path=vendor/bundle --retry=3 --jobs=3
|
7
|
-
|
8
|
-
|
13
|
+
|
14
|
+
before_install:
|
15
|
+
- gem install bundler
|
16
|
+
- nvm install 9
|
17
|
+
|
18
|
+
after_success:
|
19
|
+
- npm install docpress && $(npm bin)/docpress build
|
20
|
+
- if [ -n "$TRAVIS_TAG" -a "$TRAVIS_PULL_REQUEST" = "false" ]; then npm install git-update-ghpages && $(npm bin)/git-update-ghpages yosiat/panko_serializer _docpress; fi
|
21
|
+
- bundle exec rake benchmarks
|
9
22
|
|
10
23
|
env:
|
11
24
|
matrix:
|
data/README.md
CHANGED
@@ -4,101 +4,24 @@
|
|
4
4
|
|
5
5
|
Panko is library which is inspired by ActiveModelSerializers 0.9 for serializing ActiveRecord objects to JSON strings, fast.
|
6
6
|
|
7
|
-
To achieve it's performance:
|
7
|
+
To achieve it's [performance](https://yosiat.github.io/panko_serializer/performance.html):
|
8
8
|
|
9
9
|
* Oj - Panko relies Oj since it's fast and allow to to serialize incrementally using `Oj::StringWriter`
|
10
10
|
* Serialization Descriptor - Panko computes most of the metadata ahead of time, to save time later in serialization.
|
11
11
|
* Type casting — Panko does type casting by it's self, instead of relying ActiveRecord.
|
12
12
|
|
13
|
-
To dig deeper about the performance choices, read [Design Choices](https://github.
|
13
|
+
To dig deeper about the performance choices, read [Design Choices](https://yosiat.github.io/panko_serializer/design-choices.html).
|
14
14
|
|
15
|
-
### Status
|
16
15
|
|
17
|
-
|
18
|
-
|
16
|
+
Support
|
17
|
+
-------
|
19
18
|
|
19
|
+
- [Documentation](https://yosiat.github.io/panko_serializer)
|
20
|
+
- [Getting Started](https://yosiat.github.io/panko_serializer/getting-started.html)
|
21
|
+
- Join our [slack community](https://pankoserializer.herokuapp.com/)
|
20
22
|
|
21
|
-
|
23
|
+
License
|
24
|
+
-------
|
22
25
|
|
23
|
-
To install Panko, all you need is to add it to your Gemfile:
|
24
|
-
|
25
|
-
```ruby
|
26
|
-
gem "panko_serializer"
|
27
|
-
```
|
28
|
-
|
29
|
-
Then, install it on the command line:
|
30
|
-
|
31
|
-
```
|
32
|
-
> bundle install
|
33
|
-
```
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
## Usage
|
38
|
-
|
39
|
-
### Getting Started
|
40
|
-
|
41
|
-
Let's create serializer and use it inside Rails controller.
|
42
|
-
|
43
|
-
```ruby
|
44
|
-
class PostSerializer < Panko::Serializer
|
45
|
-
attributes :title
|
46
|
-
end
|
47
|
-
|
48
|
-
class UserSerializer < Panko::Serializer
|
49
|
-
attributes :id, :name, :age
|
50
|
-
|
51
|
-
has_many :posts, serializer: PostSerializer
|
52
|
-
end
|
53
|
-
```
|
54
|
-
|
55
|
-
As you can see, defining serializers is simple and resembles ActiveModelSerializers 0.9,
|
56
|
-
To utilize the `UserSerializer` inside a Rails controller and serialize some users, all we need to do is:
|
57
|
-
|
58
|
-
```ruby
|
59
|
-
class UsersController < ApplicationController
|
60
|
-
def index
|
61
|
-
users = User.includes(:posts).all
|
62
|
-
render json: Panko::ArraySerializer(users, each_serializer: UserSerializer).to_json
|
63
|
-
end
|
64
|
-
end
|
65
|
-
```
|
66
|
-
|
67
|
-
And voila, we have endpoint which serialize users using Panko!
|
68
|
-
|
69
|
-
|
70
|
-
## Features
|
71
|
-
|
72
|
-
### Attributes
|
73
|
-
|
74
|
-
Attributes allow you to specify which record attributes you want to serialize,
|
75
|
-
There are two types of attributes:
|
76
|
-
|
77
|
-
* Field - simple columns defined on the record it self.
|
78
|
-
* Virtual/Method - this allows to include properties beyond simple fields.
|
79
|
-
|
80
|
-
Example:
|
81
|
-
|
82
|
-
```ruby
|
83
|
-
class UserSerializer < Panko::Serializer
|
84
|
-
attributes :full_name
|
85
|
-
|
86
|
-
def full_name
|
87
|
-
"#{object.first_name} #{object.last_name}"
|
88
|
-
end
|
89
|
-
end
|
90
|
-
```
|
91
|
-
|
92
|
-
As you can see, in order to access the serialized record, you need to access `object`.
|
93
|
-
If you want to pass data to the serializer, beyond the serialized record, you can pass `context` to the serializer (both in single and array serializer).
|
94
|
-
|
95
|
-
#### TODO:
|
96
|
-
Finished feature, will add documentation sson:
|
97
|
-
- Realtionships - `has_one`, `has_many`
|
98
|
-
- Filters & Nested Filters
|
99
|
-
- Reponse bag
|
100
|
-
|
101
|
-
## License
|
102
26
|
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
|
103
27
|
|
104
|
-
|
@@ -8,7 +8,7 @@ class AmsAuthorFastSerializer < ActiveModel::Serializer
|
|
8
8
|
end
|
9
9
|
|
10
10
|
class AmsPostFastSerializer < ActiveModel::Serializer
|
11
|
-
attributes :id, :body, :title, :author_id
|
11
|
+
attributes :id, :body, :title, :author_id, :created_at
|
12
12
|
end
|
13
13
|
|
14
14
|
class AmsPostWithHasOneFastSerializer < ActiveModel::Serializer
|
@@ -39,7 +39,7 @@ def benchmark_ams(prefix, serializer, options = {})
|
|
39
39
|
end
|
40
40
|
|
41
41
|
|
42
|
-
benchmark_ams "HasOne", AmsPostWithHasOneFastSerializer
|
43
42
|
benchmark_ams "Simple", AmsPostFastSerializer
|
43
|
+
benchmark_ams "HasOne", AmsPostWithHasOneFastSerializer
|
44
44
|
benchmark_ams "Except", AmsPostWithHasOneFastSerializer, except: [:title]
|
45
45
|
benchmark_ams "Include", AmsPostWithHasOneFastSerializer, include: [:id, :body, :author_id, :author]
|
data/benchmarks/bm_panko_json.rb
CHANGED
@@ -0,0 +1,21 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
require_relative "./benchmarking_support"
|
3
|
+
require_relative "./app"
|
4
|
+
|
5
|
+
class NotSerializer
|
6
|
+
end
|
7
|
+
|
8
|
+
class RealSerializer < Panko::Serializer
|
9
|
+
end
|
10
|
+
|
11
|
+
Benchmark.run("CantFindConst") do
|
12
|
+
Panko::SerializerResolver.resolve("cant_find_const")
|
13
|
+
end
|
14
|
+
|
15
|
+
Benchmark.run("NotSerializer") do
|
16
|
+
Panko::SerializerResolver.resolve("not")
|
17
|
+
end
|
18
|
+
|
19
|
+
Benchmark.run("RealSerializer") do
|
20
|
+
Panko::SerializerResolver.resolve("real")
|
21
|
+
end
|
data/docs/README.md
ADDED
@@ -0,0 +1,11 @@
|
|
1
|
+
* [Panko](../README.md)
|
2
|
+
* [Getting Stated](getting-started.md)
|
3
|
+
* Reference
|
4
|
+
* [Attributes](attributes.md)
|
5
|
+
* [Associations](associations.md)
|
6
|
+
* [Response bag](response-bag.md)
|
7
|
+
* [Performance](performance.md)
|
8
|
+
* [Design Choices](design-choices.md)
|
9
|
+
|
10
|
+
|
11
|
+
|
@@ -0,0 +1,76 @@
|
|
1
|
+
# Associations
|
2
|
+
|
3
|
+
A serializer can define it's own associations - both `has_many` and `has_one` to serializer under the context of the object.
|
4
|
+
|
5
|
+
For example:
|
6
|
+
|
7
|
+
```ruby
|
8
|
+
class PostSerializer < Panko::Serailizer
|
9
|
+
attributes :title, :body
|
10
|
+
|
11
|
+
has_one :author, serializer: AuthorSerializer
|
12
|
+
has_many :comments, each_serializer: CommentSerializer
|
13
|
+
end
|
14
|
+
```
|
15
|
+
|
16
|
+
### Inference
|
17
|
+
|
18
|
+
Panko can find the type of the serializer by looking at the realtionship name, so instead specifying
|
19
|
+
the serializer at the above example, we can -
|
20
|
+
|
21
|
+
```ruby
|
22
|
+
class PostSerializer < Panko::Serailizer
|
23
|
+
attributes :title, :body
|
24
|
+
|
25
|
+
has_one :author
|
26
|
+
has_many :comments
|
27
|
+
end
|
28
|
+
```
|
29
|
+
|
30
|
+
The logic of inferencing is -
|
31
|
+
- Take the name of the relationship (for example - `:author` / `:comments`) singularize and camelize it
|
32
|
+
- Look for const defined with the name aboe and "Serializer" suffix (by using `Object.const_get`)
|
33
|
+
|
34
|
+
> If Panko can't find the serializer it will throw an error on startup time, for example: `Can't find serializer for PostSerializer.author has_one relationship`
|
35
|
+
|
36
|
+
## Nested Filters
|
37
|
+
|
38
|
+
As talked before, Panko allows you to filter the attributes of a serializer.
|
39
|
+
But Panko let you take that step further, and filters the attributes of you associations so you can re-use your serializers in your application.
|
40
|
+
|
41
|
+
For example, let's say one portion of the application needs to serializer list of posts and serializer their - `title`, `body`, author's id and comments id.
|
42
|
+
|
43
|
+
We can declare tailored serializer for this, or we can re-use the above defined serializer - `PostSerializer` and use nested filters.
|
44
|
+
|
45
|
+
```ruby
|
46
|
+
posts = Post.all
|
47
|
+
|
48
|
+
Panko::ArraySerializer.new(posts, only: {
|
49
|
+
instance: [:title, :body, :author, :comments],
|
50
|
+
author: [:id],
|
51
|
+
comments: [:id],
|
52
|
+
})
|
53
|
+
```
|
54
|
+
|
55
|
+
Let's disect `only` option we passed -
|
56
|
+
* `instance` - list of attributes (and associations) we want to serializer for current instance of the serializer, in this case - `PostSerializer`.
|
57
|
+
* `author`, `comments` - here we specify the list of attributes we want to serialize for each association.
|
58
|
+
|
59
|
+
It's important to note that Nested Filters, are recursive, in other words, we can filter the association's associations.
|
60
|
+
|
61
|
+
For example, `CommentSerializer` have has_one association `Author`, and for each `comments.author` we only it's name.
|
62
|
+
|
63
|
+
```ruby
|
64
|
+
posts = Post.all
|
65
|
+
|
66
|
+
Panko::ArraySerializer.new(posts, only: {
|
67
|
+
instance: [:title, :body, :author, :comments],
|
68
|
+
author: [:id],
|
69
|
+
comments: {
|
70
|
+
instance: [:id, :author],
|
71
|
+
author: [:name]
|
72
|
+
}
|
73
|
+
})
|
74
|
+
```
|
75
|
+
|
76
|
+
As you see now in `comments` the `instance` have differenet meaning, the `CommentSerializer`.
|
data/docs/attributes.md
ADDED
@@ -0,0 +1,104 @@
|
|
1
|
+
# Attributes
|
2
|
+
|
3
|
+
Attributes allow you to specify which record attributes you want to serialize,
|
4
|
+
There are two types of attributes:
|
5
|
+
|
6
|
+
* Field - simple columns defined on the record it self.
|
7
|
+
* Virtual/Method - this allows to include properties beyond simple fields.
|
8
|
+
|
9
|
+
|
10
|
+
```ruby
|
11
|
+
class UserSerializer < Panko::Serializer
|
12
|
+
attributes :full_name
|
13
|
+
|
14
|
+
def full_name
|
15
|
+
"#{object.first_name} #{object.last_name}"
|
16
|
+
end
|
17
|
+
end
|
18
|
+
```
|
19
|
+
|
20
|
+
## Field Attributes
|
21
|
+
|
22
|
+
Using field attributes you can control which columns of the given ActiveRecord object you want to serialize.
|
23
|
+
|
24
|
+
Instead of relying ActiveRecord to do it's type casting, Panko does on it's own for performance reasons (read more in [Design Choices](design-choices.md#type-casting)).
|
25
|
+
|
26
|
+
|
27
|
+
## Method Attributes
|
28
|
+
|
29
|
+
Method attributes are used when your serialized values can be derived from the object you are serializing.
|
30
|
+
|
31
|
+
The serializer's attribute methods can access the object being serialized as `object` -
|
32
|
+
|
33
|
+
```ruby
|
34
|
+
class PostSerializer < Panko::Serializer
|
35
|
+
def author_name
|
36
|
+
"#{object.author.first_name} #{object.author.last_name}"
|
37
|
+
end
|
38
|
+
end
|
39
|
+
```
|
40
|
+
|
41
|
+
Another useful, thing you can pass your serializer is `context`, a `context` is a bag of data whom your serializer may need.
|
42
|
+
|
43
|
+
For example, here we will pass the current user:
|
44
|
+
```ruby
|
45
|
+
class UserSerializer < Panko::Serializer
|
46
|
+
attributes :id, :email
|
47
|
+
|
48
|
+
def feature_flags
|
49
|
+
context[:feature_flags]
|
50
|
+
end
|
51
|
+
end
|
52
|
+
|
53
|
+
serializer = UserSerializer.new(context: {
|
54
|
+
feature_flags: FeatureFlags.all
|
55
|
+
})
|
56
|
+
|
57
|
+
serializer.serialize(User.first)
|
58
|
+
```
|
59
|
+
|
60
|
+
## Filters
|
61
|
+
|
62
|
+
Filters allows us to reduce the amount of attributes we can serialize, therefore reduce the data usage & performance of serializing.
|
63
|
+
|
64
|
+
There are two types of filters:
|
65
|
+
* only - use those attributes **only** and nothing else
|
66
|
+
* except - all attributes **except** those attributes
|
67
|
+
|
68
|
+
Usage example:
|
69
|
+
```ruby
|
70
|
+
class UserSerializer < Panko::Serializer
|
71
|
+
attributes :id, :name, :email
|
72
|
+
end
|
73
|
+
|
74
|
+
# this line will return { 'name': '..' }
|
75
|
+
UserSerializer.new(only: [:name]).serialize(User.first)
|
76
|
+
|
77
|
+
# this line will return { 'id': '..', 'email': ... }
|
78
|
+
UserSerializer.new(except: [:name]).serialize(User.first)
|
79
|
+
```
|
80
|
+
|
81
|
+
## Aliases
|
82
|
+
|
83
|
+
Let's say we have attribute name that we want to expose to client as different name, the current way of doing so is using method attribute, for example:
|
84
|
+
|
85
|
+
```ruby
|
86
|
+
class PostSerializer < Panko::Serializer
|
87
|
+
attributes :published_at
|
88
|
+
|
89
|
+
def published_at
|
90
|
+
object.created_at
|
91
|
+
end
|
92
|
+
end
|
93
|
+
```
|
94
|
+
|
95
|
+
The downside of this approach is that `created_at` skips Panko's type casting, therefore we get direct hit on performance.
|
96
|
+
|
97
|
+
To fix this, we can use aliases -
|
98
|
+
|
99
|
+
```ruby
|
100
|
+
class PostSerializer < Panko::Serializer
|
101
|
+
aliases created_at: :published_at
|
102
|
+
end
|
103
|
+
```
|
104
|
+
|
@@ -0,0 +1,122 @@
|
|
1
|
+
# Design Choices
|
2
|
+
|
3
|
+
In short, Panko, is a serializer for ActiveRecord objects (it can't serialize any other object), which strives for high performance & simple API (which is inspired by ActiveModelSerializers).
|
4
|
+
|
5
|
+
Its performance is achieved by:
|
6
|
+
|
7
|
+
* `Oj::StringWriter` - I will elaborate later.
|
8
|
+
* Type casting — instead of relying on ActiveRecord to do its type cast, Panko is doing it by itself.
|
9
|
+
* Figuring out the metadata, ahead of time — therefore, we ask less questions during the `serialization loop`.
|
10
|
+
|
11
|
+
|
12
|
+
## Serialization overview
|
13
|
+
|
14
|
+
First, let's start with overview. Let's say we want to serialize `User` object, which has
|
15
|
+
`first_name`, `last_name`, `age`, and `email` properties.
|
16
|
+
|
17
|
+
The serializer definition will be something like this:
|
18
|
+
|
19
|
+
```ruby
|
20
|
+
class UserSerializer < Panko::Serializer
|
21
|
+
attributes :name, :age, :email
|
22
|
+
|
23
|
+
def name
|
24
|
+
"#{object.first_name} #{object.last_name}"
|
25
|
+
end
|
26
|
+
end
|
27
|
+
```
|
28
|
+
|
29
|
+
And the usage of this serializer will be:
|
30
|
+
|
31
|
+
```ruby
|
32
|
+
# fetch user from database
|
33
|
+
user = User.first
|
34
|
+
|
35
|
+
# create serializer, with empty options
|
36
|
+
serializer = UserSerilizer.new
|
37
|
+
|
38
|
+
# serialize to JSON
|
39
|
+
serializer.serialize_to_json(user)
|
40
|
+
```
|
41
|
+
|
42
|
+
Let's go over the steps that Panko will execute behind the scenes for this flow.
|
43
|
+
_I will skip the serializer definition part, because it's fairly simple and straightforward (see `lib/panko/serializer.rb`)_
|
44
|
+
|
45
|
+
First step, while initializing the UserSerializer, we will create a **Serialization Descriptor** for this class.
|
46
|
+
Serialization Descriptor's goal is to answer those questions:
|
47
|
+
|
48
|
+
* Which fields do we have? In our case, `:age`, `:email`
|
49
|
+
* Which method fields do we have? In our case `:name`
|
50
|
+
* Which associations do we have (and their serialization descriptors)
|
51
|
+
|
52
|
+
The serialization description is also responsible for filtering the attributes (`only` \ `except`).
|
53
|
+
|
54
|
+
Now, that we have the serialization descriptor, we are finished with the Ruby part of Panko, and all we did here is done in *initialization time* and now we move to C code.
|
55
|
+
|
56
|
+
In C land, we take the `user` object and the serialization descriptor, and start the serialization process which is separated to 4 parts:
|
57
|
+
|
58
|
+
* Serializing Fields - looping through serialization descriptor's `fields` and read them from the ActiveRecord object (see `Type Casting`) and write them to the writer.
|
59
|
+
* Serializing Method Fields - creating (a cached) serializer instance, setting its `@object` and `@context`, calling all the method fields and writing them to the writer.
|
60
|
+
* Serializing associations — this is simple, once we have fields + method fields, we just repeat the process.
|
61
|
+
|
62
|
+
Once this is finished, we have nice JSON string.
|
63
|
+
Now let's dig deeper.
|
64
|
+
|
65
|
+
## Interesting parts
|
66
|
+
|
67
|
+
### Oj::StringWriter
|
68
|
+
|
69
|
+
If you read the code of ActiveRecord serialization code in Ruby, you will observe this flow:
|
70
|
+
|
71
|
+
1. Get an array of ActiveRecord objects (`User.all` for example)
|
72
|
+
2. Build new array of hashes where each hash is `User` with the attributes we selected
|
73
|
+
3. The JSON serializer, takes this array of hashes and loop them, and converts it to JSON string
|
74
|
+
|
75
|
+
This entire process is expensive in terms of Memory & CPU, and this where the combination of Panko and Oj::StringWriter really shines.
|
76
|
+
|
77
|
+
In Panko, the serialization process of the above is:
|
78
|
+
|
79
|
+
1. Get an array of ActiveRecord objects (`User.all` for example)
|
80
|
+
2. Create `Oj::StringWriter` and feed the values to it, via `push_value` / `push_object` / `push_object` and behind the scene, `Oj::StringWriter` will serialize the objects incrementally into a string.
|
81
|
+
3. Get from `Oj::StringWriter` the completed JSON string — which is a no-op, since `Oj::StringWriter` already built the string.
|
82
|
+
|
83
|
+
### Figuring out the metadata, ahead of time.
|
84
|
+
|
85
|
+
Another observation I noticed in the ruby serializers is that they ask and do a lot in a serialization loop:
|
86
|
+
|
87
|
+
* Is this field a method? is it a property?
|
88
|
+
* Which fields and associations do I need for the serializer to consider the `only` and `except` options
|
89
|
+
* What is the serializer of this has_one association?
|
90
|
+
|
91
|
+
Panko tries to ask the bare minimum in serialization by building `Serialization Descriptor` for each serialization and caching it.
|
92
|
+
|
93
|
+
The Serialization Descriptor will do the filtering of `only` and `except` and will check if a field is a method or not (therefore Panko doesn't have list of `attributes`)
|
94
|
+
|
95
|
+
|
96
|
+
### Type Casting
|
97
|
+
|
98
|
+
This is the final part, which helped yield most of the performance improvements.
|
99
|
+
In ActiveRecord, when we read a value of attribute, it does type casting of the DB value to its real ruby type.
|
100
|
+
|
101
|
+
For example, time strings are converted to Time objects, Strings are duplicated, and Integers are converts from their values to Number.
|
102
|
+
|
103
|
+
This type casting is really expensive, as it's responsible for most of the allocations in the serialization flow and most of them can be "relaxed".
|
104
|
+
|
105
|
+
If we think about it, we don't need to duplicate strings or convert time strings to time objects or even parse JSON strings for the JSON serialization process.
|
106
|
+
|
107
|
+
What Panko does is that if we have ActiveRecord type string, we won't duplicate it.
|
108
|
+
If we have an integer string value, we will convert it to an integer, and the same goes for other types.
|
109
|
+
|
110
|
+
All of these conversions are done in C, which of course yields a big performance improvement.
|
111
|
+
|
112
|
+
#### Time type casting
|
113
|
+
While you read Panko source code, you will encounter the time type casting and immediately you will have a "WTF?" moment.
|
114
|
+
|
115
|
+
The idea behind the time type casting code relies on the end result of JSON type casting — what we need in order to serialize Time to JSON? UTC ISO8601 time format representation.
|
116
|
+
|
117
|
+
The time type casting works as follows:
|
118
|
+
|
119
|
+
* If it's a string that ends with `Z`, and the strings matches the UTC ISO8601 regex, then we just return the string.
|
120
|
+
* If it's a string and it doesn't follow the rules above, we check if it's a timestamp in database format and convert it via regex + string concat to UTC ISO8601 - Yes, there is huge assumption here, that the database returns UTC timestamps — this will be configureable (before Panko official release).
|
121
|
+
* If it's none of the above, I will let ActiveRecord type casting do it's magic.
|
122
|
+
|
data/docs/docpress.json
ADDED
@@ -0,0 +1,47 @@
|
|
1
|
+
# Getting Started
|
2
|
+
|
3
|
+
## Installation
|
4
|
+
|
5
|
+
To install Panko, all you need is to add it to your Gemfile:
|
6
|
+
|
7
|
+
```ruby
|
8
|
+
gem "panko_serializer"
|
9
|
+
```
|
10
|
+
|
11
|
+
Then, install it on the command line:
|
12
|
+
|
13
|
+
```
|
14
|
+
> bundle install
|
15
|
+
```
|
16
|
+
|
17
|
+
|
18
|
+
## Creating your first serializer
|
19
|
+
|
20
|
+
Let's create serializer and use it inside Rails controller.
|
21
|
+
|
22
|
+
```ruby
|
23
|
+
class PostSerializer < Panko::Serializer
|
24
|
+
attributes :title
|
25
|
+
end
|
26
|
+
|
27
|
+
class UserSerializer < Panko::Serializer
|
28
|
+
attributes :id, :name, :age
|
29
|
+
|
30
|
+
has_many :posts, serializer: PostSerializer
|
31
|
+
end
|
32
|
+
```
|
33
|
+
|
34
|
+
As you can see, defining serializers is simple and resembles ActiveModelSerializers 0.9,
|
35
|
+
To utilize the `UserSerializer` inside a Rails controller and serialize some users, all we need to do is:
|
36
|
+
|
37
|
+
```ruby
|
38
|
+
class UsersController < ApplicationController
|
39
|
+
def index
|
40
|
+
users = User.includes(:posts).all
|
41
|
+
render json: Panko::ArraySerializer.new(users, each_serializer: UserSerializer).to_json
|
42
|
+
end
|
43
|
+
end
|
44
|
+
```
|
45
|
+
|
46
|
+
And voila, we have endpoint which serialize users using Panko!
|
47
|
+
|
data/docs/performance.md
ADDED
@@ -0,0 +1,35 @@
|
|
1
|
+
# Performance
|
2
|
+
|
3
|
+
The performance of Panko is measured using microbenchmarks and load testing.
|
4
|
+
|
5
|
+
## Microbenchmarks
|
6
|
+
|
7
|
+
The following microbenchmarks are run on MacBook Pro (Retina, 15-inch, Mid 2015), Ruby 2.4 with Rails 4.2
|
8
|
+
demonstrating the performance of ActiveModelSerializers 0.9 and Panko 0.3.3
|
9
|
+
|
10
|
+
|
11
|
+
Benchmark | AMS ip/s | Panko ip/s
|
12
|
+
---------------------------------------|----------|-----------------
|
13
|
+
| Simple_Posts_2300 | 25.81 | 135.29 |
|
14
|
+
| Simple_Posts_50 | 1,248.39 | 6,518.68 |
|
15
|
+
| HasOne_Posts_2300 | 11.33 | 73.42 |
|
16
|
+
| HasOne_Posts_50 | 523.14 | 4,985.41 |
|
17
|
+
|
18
|
+
> The corresponding benchmarks are `benchmarks/bm_active_model_serializers.rb` and `benchmarks/bm_panko_json.rb`
|
19
|
+
|
20
|
+
|
21
|
+
## Real-world benchmark
|
22
|
+
|
23
|
+
The real-world benchmark here is endpoint which serializes 7,884 entries with 48 attributes and no associations.
|
24
|
+
The benchmark took place in environment that simulates production environment and run using `wrk` from machine on the same cluster.
|
25
|
+
|
26
|
+
|
27
|
+
Metric | AMS | Panko
|
28
|
+
------------ |------------ | -------------
|
29
|
+
Avg Response Time| 4.89s| 1.48s|
|
30
|
+
Max Response Time| 5.42s| 1.83s|
|
31
|
+
99th Response Time| 5.42s| 1.74s|
|
32
|
+
Total Requests| 61| 202|
|
33
|
+
|
34
|
+
|
35
|
+
*Thanks to [Bringg](https://www.bringg.com) for providing the infrastructrue for the benchmarks*
|
@@ -0,0 +1,53 @@
|
|
1
|
+
# Response
|
2
|
+
|
3
|
+
Let's say you have some JSON payload which can is constructed using Panko serialization result,
|
4
|
+
like this:
|
5
|
+
|
6
|
+
```ruby
|
7
|
+
class PostsController < ApplicationController
|
8
|
+
def index
|
9
|
+
posts = Post.all
|
10
|
+
render json: {
|
11
|
+
success: true,
|
12
|
+
total_count: posts.count,
|
13
|
+
posts: Panko::ArraySerializer.new(posts, each_serializer: PostSerializer).to_json
|
14
|
+
}
|
15
|
+
end
|
16
|
+
end
|
17
|
+
```
|
18
|
+
|
19
|
+
The output of the above will be json string (for `posts`) inside json string and this were `Panko::Response` shines.
|
20
|
+
|
21
|
+
```ruby
|
22
|
+
class PostsController < ApplicationController
|
23
|
+
def index
|
24
|
+
posts = Post.all
|
25
|
+
render json: Panko::Response.new(
|
26
|
+
success: true,
|
27
|
+
total_count: posts.count,
|
28
|
+
posts: Panko::ArraySerializer.new(posts, each_serializer: PostSerializer)
|
29
|
+
)
|
30
|
+
end
|
31
|
+
end
|
32
|
+
```
|
33
|
+
|
34
|
+
And everything will work as expected!
|
35
|
+
|
36
|
+
## JsonValue
|
37
|
+
|
38
|
+
Let's take the above example further, we serialized the posts and cached it as JSON string in our Cache.
|
39
|
+
Now, you can wrap the cached value with `Panko::JsonValue`, like here -
|
40
|
+
|
41
|
+
```ruby
|
42
|
+
class PostsController < ApplicationController
|
43
|
+
def index
|
44
|
+
posts = Cache.get("/posts")
|
45
|
+
|
46
|
+
render json: Panko::Response.new(
|
47
|
+
success: true,
|
48
|
+
total_count: posts.count,
|
49
|
+
posts: Panko::JsonValue.from(posts)
|
50
|
+
)
|
51
|
+
end
|
52
|
+
end
|
53
|
+
```
|
@@ -56,6 +56,11 @@ VALUE association_name_sym_ref(VALUE self) {
|
|
56
56
|
return association->name_sym;
|
57
57
|
}
|
58
58
|
|
59
|
+
VALUE association_name_str_ref(VALUE self) {
|
60
|
+
Association association = (Association)DATA_PTR(self);
|
61
|
+
return association->name_str;
|
62
|
+
}
|
63
|
+
|
59
64
|
VALUE association_descriptor_ref(VALUE self) {
|
60
65
|
Association association = (Association)DATA_PTR(self);
|
61
66
|
return association->rb_descriptor;
|
@@ -76,6 +81,7 @@ void panko_init_association(VALUE mPanko) {
|
|
76
81
|
rb_define_module_function(cAssociation, "new", association_new, -1);
|
77
82
|
|
78
83
|
rb_define_method(cAssociation, "name_sym", association_name_sym_ref, 0);
|
84
|
+
rb_define_method(cAssociation, "name_str", association_name_str_ref, 0);
|
79
85
|
rb_define_method(cAssociation, "descriptor", association_descriptor_ref, 0);
|
80
86
|
rb_define_method(cAssociation, "descriptor=", association_decriptor_aset, 1);
|
81
87
|
}
|
@@ -20,9 +20,7 @@ module Panko
|
|
20
20
|
|
21
21
|
backend.type = descriptor.type
|
22
22
|
|
23
|
-
backend.attributes = descriptor.attributes.
|
24
|
-
Attribute.create(attr.name, alias_name: attr.alias_name)
|
25
|
-
end
|
23
|
+
backend.attributes = descriptor.attributes.dup
|
26
24
|
|
27
25
|
backend.method_fields = descriptor.method_fields.dup
|
28
26
|
|
@@ -30,21 +28,8 @@ module Panko
|
|
30
28
|
backend.serializer = descriptor.serializer.reset
|
31
29
|
end
|
32
30
|
|
33
|
-
backend.has_many_associations = descriptor.has_many_associations.
|
34
|
-
|
35
|
-
assoc.name_sym,
|
36
|
-
assoc.name_sym.to_s,
|
37
|
-
Panko::SerializationDescriptor.duplicate(assoc.descriptor)
|
38
|
-
)
|
39
|
-
end
|
40
|
-
|
41
|
-
backend.has_one_associations = descriptor.has_one_associations.map do |assoc|
|
42
|
-
Panko::Association.new(
|
43
|
-
assoc.name_sym,
|
44
|
-
assoc.name_sym.to_s,
|
45
|
-
Panko::SerializationDescriptor.duplicate(assoc.descriptor)
|
46
|
-
)
|
47
|
-
end
|
31
|
+
backend.has_many_associations = descriptor.has_many_associations.dup
|
32
|
+
backend.has_one_associations = descriptor.has_one_associations.dup
|
48
33
|
|
49
34
|
backend
|
50
35
|
end
|
@@ -58,7 +43,8 @@ module Panko
|
|
58
43
|
attributes_only_filters, associations_only_filters = resolve_filters(options, :only)
|
59
44
|
attributes_except_filters, associations_except_filters = resolve_filters(options, :except)
|
60
45
|
|
61
|
-
apply_attribute_filters
|
46
|
+
self.attributes = apply_attribute_filters(
|
47
|
+
self.attributes,
|
62
48
|
attributes_only_filters,
|
63
49
|
attributes_except_filters
|
64
50
|
)
|
@@ -120,10 +106,17 @@ module Panko
|
|
120
106
|
filters = {}
|
121
107
|
filters[:only] = only_filter unless only_filter.nil?
|
122
108
|
filters[:except] = except_filter unless except_filter.nil?
|
123
|
-
|
109
|
+
|
110
|
+
unless filters.empty?
|
111
|
+
next Panko::Association.new(
|
112
|
+
name,
|
113
|
+
association.name_str,
|
114
|
+
Panko::SerializationDescriptor.build(descriptor.type, filters)
|
115
|
+
)
|
116
|
+
end
|
124
117
|
|
125
118
|
association
|
126
|
-
end
|
119
|
+
end
|
127
120
|
end
|
128
121
|
|
129
122
|
def resolve_filters(options, filter)
|
@@ -152,9 +145,9 @@ module Panko
|
|
152
145
|
fields
|
153
146
|
end
|
154
147
|
|
155
|
-
def apply_attribute_filters
|
148
|
+
def apply_attribute_filters(attributes, only, except)
|
156
149
|
unless only.empty?
|
157
|
-
|
150
|
+
attributes = attributes.select do |attribute|
|
158
151
|
name_to_check = attribute.name
|
159
152
|
name_to_check = attribute.alias_name unless attribute.alias_name.nil?
|
160
153
|
|
@@ -163,13 +156,15 @@ module Panko
|
|
163
156
|
end
|
164
157
|
|
165
158
|
unless except.empty?
|
166
|
-
|
159
|
+
attributes = attributes.reject do |attribute|
|
167
160
|
name_to_check = attribute.name
|
168
161
|
name_to_check = attribute.alias_name unless attribute.alias_name.nil?
|
169
162
|
|
170
163
|
except.include?(name_to_check.to_sym)
|
171
164
|
end
|
172
165
|
end
|
166
|
+
|
167
|
+
attributes
|
173
168
|
end
|
174
169
|
end
|
175
170
|
end
|
data/lib/panko/serializer.rb
CHANGED
@@ -36,8 +36,13 @@ module Panko
|
|
36
36
|
@_descriptor.method_fields << method
|
37
37
|
end
|
38
38
|
|
39
|
-
def has_one(name, options)
|
39
|
+
def has_one(name, options = {})
|
40
40
|
serializer_const = options[:serializer]
|
41
|
+
serializer_const = Panko::SerializerResolver.resolve(name.to_s) if serializer_const.nil?
|
42
|
+
|
43
|
+
if serializer_const.nil?
|
44
|
+
raise "Can't find serializer for #{self.name}.#{name} has_one relationship."
|
45
|
+
end
|
41
46
|
|
42
47
|
@_descriptor.has_one_associations << Panko::Association.new(
|
43
48
|
name,
|
@@ -46,8 +51,13 @@ module Panko
|
|
46
51
|
)
|
47
52
|
end
|
48
53
|
|
49
|
-
def has_many(name, options)
|
54
|
+
def has_many(name, options = {})
|
50
55
|
serializer_const = options[:serializer] || options[:each_serializer]
|
56
|
+
serializer_const = Panko::SerializerResolver.resolve(name.to_s) if serializer_const.nil?
|
57
|
+
|
58
|
+
if serializer_const.nil?
|
59
|
+
raise "Can't find serializer for #{self.name}.#{name} has_many relationship."
|
60
|
+
end
|
51
61
|
|
52
62
|
@_descriptor.has_many_associations << Panko::Association.new(
|
53
63
|
name,
|
@@ -0,0 +1,29 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
require "byebug"
|
3
|
+
|
4
|
+
class Panko::SerializerResolver
|
5
|
+
|
6
|
+
def self.resolve(name)
|
7
|
+
serializer_name = "#{name.singularize.camelize}Serializer"
|
8
|
+
serializer_const = self.safe_const_get(serializer_name)
|
9
|
+
|
10
|
+
return nil if serializer_const.nil?
|
11
|
+
return nil unless self.is_serializer(serializer_const)
|
12
|
+
|
13
|
+
serializer_const
|
14
|
+
end
|
15
|
+
|
16
|
+
|
17
|
+
private
|
18
|
+
|
19
|
+
def self.is_serializer(const)
|
20
|
+
const < Panko::Serializer
|
21
|
+
end
|
22
|
+
|
23
|
+
def self.safe_const_get(name)
|
24
|
+
Object.const_get(name)
|
25
|
+
rescue NameError
|
26
|
+
nil
|
27
|
+
end
|
28
|
+
|
29
|
+
end
|
data/lib/panko/version.rb
CHANGED
data/lib/panko_serializer.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: panko_serializer
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.3.
|
4
|
+
version: 0.3.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Yosi Attias
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-12-
|
11
|
+
date: 2017-12-23 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
@@ -97,7 +97,6 @@ files:
|
|
97
97
|
- LICENSE.txt
|
98
98
|
- README.md
|
99
99
|
- Rakefile
|
100
|
-
- _config.yml
|
101
100
|
- benchmarks/BENCHMARKS.md
|
102
101
|
- benchmarks/allocs.rb
|
103
102
|
- benchmarks/app.rb
|
@@ -107,12 +106,21 @@ files:
|
|
107
106
|
- benchmarks/bm_panko_json.rb
|
108
107
|
- benchmarks/bm_panko_object.rb
|
109
108
|
- benchmarks/bm_serialization_descriptor.rb
|
109
|
+
- benchmarks/bm_serializer_resolver.rb
|
110
110
|
- benchmarks/profile.rb
|
111
111
|
- benchmarks/sanity.rb
|
112
112
|
- benchmarks/setup.rb
|
113
113
|
- benchmarks/type_casts/bm_active_record.rb
|
114
114
|
- benchmarks/type_casts/bm_panko.rb
|
115
115
|
- benchmarks/type_casts/support.rb
|
116
|
+
- docs/README.md
|
117
|
+
- docs/associations.md
|
118
|
+
- docs/attributes.md
|
119
|
+
- docs/design-choices.md
|
120
|
+
- docs/docpress.json
|
121
|
+
- docs/getting-started.md
|
122
|
+
- docs/performance.md
|
123
|
+
- docs/response-bag.md
|
116
124
|
- ext/panko_serializer/association.c
|
117
125
|
- ext/panko_serializer/association.h
|
118
126
|
- ext/panko_serializer/attribute.c
|
@@ -133,6 +141,7 @@ files:
|
|
133
141
|
- lib/panko/response.rb
|
134
142
|
- lib/panko/serialization_descriptor.rb
|
135
143
|
- lib/panko/serializer.rb
|
144
|
+
- lib/panko/serializer_resolver.rb
|
136
145
|
- lib/panko/version.rb
|
137
146
|
- lib/panko_serializer.rb
|
138
147
|
- panko_serializer.gemspec
|
@@ -156,7 +165,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
156
165
|
version: '0'
|
157
166
|
requirements: []
|
158
167
|
rubyforge_project:
|
159
|
-
rubygems_version: 2.
|
168
|
+
rubygems_version: 2.7.3
|
160
169
|
signing_key:
|
161
170
|
specification_version: 4
|
162
171
|
summary: Fast serialization for ActiveModel
|
data/_config.yml
DELETED
@@ -1 +0,0 @@
|
|
1
|
-
theme: jekyll-theme-cayman
|