panko_serializer 0.3.2 → 0.3.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/.gitignore +4 -0
- data/.travis.yml +15 -2
- data/README.md +9 -86
- data/benchmarks/bm_active_model_serializers.rb +2 -2
- data/benchmarks/bm_panko_json.rb +1 -1
- data/benchmarks/bm_serializer_resolver.rb +21 -0
- data/docs/README.md +11 -0
- data/docs/associations.md +76 -0
- data/docs/attributes.md +104 -0
- data/docs/design-choices.md +122 -0
- data/docs/docpress.json +4 -0
- data/docs/getting-started.md +47 -0
- data/docs/performance.md +35 -0
- data/docs/response-bag.md +53 -0
- data/ext/panko_serializer/association.c +6 -0
- data/lib/panko/serialization_descriptor.rb +19 -24
- data/lib/panko/serializer.rb +12 -2
- data/lib/panko/serializer_resolver.rb +29 -0
- data/lib/panko/version.rb +1 -1
- data/lib/panko_serializer.rb +1 -0
- metadata +13 -4
- data/_config.yml +0 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: eef3c77d8a03018af85326b4c07890d6d543909925c664f6baf6f20f634033e2
|
4
|
+
data.tar.gz: 28c88dea3cd021f1ba3cb5e31b21e03f3ddb9cdbf4851cccafa78770d742a994
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: edbe1e16a0216d8cd9e8af32289e6b173565ad7d1b50c3177f90a2cc683fd00d26cf7248e0d563e4a76916e481d0723b23041fe6973c5df05b180227e1da82fe
|
7
|
+
data.tar.gz: d0883a7ba84ff19d57738d56846ca9fc9b4ed70d6343c6280225ccfa83abe6489d611b64cea4be294ce4ddccfc8e992be55545fd6ff18bf65936fae91db94ce3
|
data/.gitignore
CHANGED
data/.travis.yml
CHANGED
@@ -3,9 +3,22 @@ cache: bundler
|
|
3
3
|
language: ruby
|
4
4
|
rvm:
|
5
5
|
- 2.4.2
|
6
|
+
|
7
|
+
env:
|
8
|
+
global:
|
9
|
+
- GIT_NAME: Travis CI
|
10
|
+
- GIT_EMAIL: nobody@nobody.org
|
11
|
+
|
6
12
|
install: bundle install --path=vendor/bundle --retry=3 --jobs=3
|
7
|
-
|
8
|
-
|
13
|
+
|
14
|
+
before_install:
|
15
|
+
- gem install bundler
|
16
|
+
- nvm install 9
|
17
|
+
|
18
|
+
after_success:
|
19
|
+
- npm install docpress && $(npm bin)/docpress build
|
20
|
+
- if [ -n "$TRAVIS_TAG" -a "$TRAVIS_PULL_REQUEST" = "false" ]; then npm install git-update-ghpages && $(npm bin)/git-update-ghpages yosiat/panko_serializer _docpress; fi
|
21
|
+
- bundle exec rake benchmarks
|
9
22
|
|
10
23
|
env:
|
11
24
|
matrix:
|
data/README.md
CHANGED
@@ -4,101 +4,24 @@
|
|
4
4
|
|
5
5
|
Panko is library which is inspired by ActiveModelSerializers 0.9 for serializing ActiveRecord objects to JSON strings, fast.
|
6
6
|
|
7
|
-
To achieve it's performance:
|
7
|
+
To achieve it's [performance](https://yosiat.github.io/panko_serializer/performance.html):
|
8
8
|
|
9
9
|
* Oj - Panko relies Oj since it's fast and allow to to serialize incrementally using `Oj::StringWriter`
|
10
10
|
* Serialization Descriptor - Panko computes most of the metadata ahead of time, to save time later in serialization.
|
11
11
|
* Type casting — Panko does type casting by it's self, instead of relying ActiveRecord.
|
12
12
|
|
13
|
-
To dig deeper about the performance choices, read [Design Choices](https://github.
|
13
|
+
To dig deeper about the performance choices, read [Design Choices](https://yosiat.github.io/panko_serializer/design-choices.html).
|
14
14
|
|
15
|
-
### Status
|
16
15
|
|
17
|
-
|
18
|
-
|
16
|
+
Support
|
17
|
+
-------
|
19
18
|
|
19
|
+
- [Documentation](https://yosiat.github.io/panko_serializer)
|
20
|
+
- [Getting Started](https://yosiat.github.io/panko_serializer/getting-started.html)
|
21
|
+
- Join our [slack community](https://pankoserializer.herokuapp.com/)
|
20
22
|
|
21
|
-
|
23
|
+
License
|
24
|
+
-------
|
22
25
|
|
23
|
-
To install Panko, all you need is to add it to your Gemfile:
|
24
|
-
|
25
|
-
```ruby
|
26
|
-
gem "panko_serializer"
|
27
|
-
```
|
28
|
-
|
29
|
-
Then, install it on the command line:
|
30
|
-
|
31
|
-
```
|
32
|
-
> bundle install
|
33
|
-
```
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
## Usage
|
38
|
-
|
39
|
-
### Getting Started
|
40
|
-
|
41
|
-
Let's create serializer and use it inside Rails controller.
|
42
|
-
|
43
|
-
```ruby
|
44
|
-
class PostSerializer < Panko::Serializer
|
45
|
-
attributes :title
|
46
|
-
end
|
47
|
-
|
48
|
-
class UserSerializer < Panko::Serializer
|
49
|
-
attributes :id, :name, :age
|
50
|
-
|
51
|
-
has_many :posts, serializer: PostSerializer
|
52
|
-
end
|
53
|
-
```
|
54
|
-
|
55
|
-
As you can see, defining serializers is simple and resembles ActiveModelSerializers 0.9,
|
56
|
-
To utilize the `UserSerializer` inside a Rails controller and serialize some users, all we need to do is:
|
57
|
-
|
58
|
-
```ruby
|
59
|
-
class UsersController < ApplicationController
|
60
|
-
def index
|
61
|
-
users = User.includes(:posts).all
|
62
|
-
render json: Panko::ArraySerializer(users, each_serializer: UserSerializer).to_json
|
63
|
-
end
|
64
|
-
end
|
65
|
-
```
|
66
|
-
|
67
|
-
And voila, we have endpoint which serialize users using Panko!
|
68
|
-
|
69
|
-
|
70
|
-
## Features
|
71
|
-
|
72
|
-
### Attributes
|
73
|
-
|
74
|
-
Attributes allow you to specify which record attributes you want to serialize,
|
75
|
-
There are two types of attributes:
|
76
|
-
|
77
|
-
* Field - simple columns defined on the record it self.
|
78
|
-
* Virtual/Method - this allows to include properties beyond simple fields.
|
79
|
-
|
80
|
-
Example:
|
81
|
-
|
82
|
-
```ruby
|
83
|
-
class UserSerializer < Panko::Serializer
|
84
|
-
attributes :full_name
|
85
|
-
|
86
|
-
def full_name
|
87
|
-
"#{object.first_name} #{object.last_name}"
|
88
|
-
end
|
89
|
-
end
|
90
|
-
```
|
91
|
-
|
92
|
-
As you can see, in order to access the serialized record, you need to access `object`.
|
93
|
-
If you want to pass data to the serializer, beyond the serialized record, you can pass `context` to the serializer (both in single and array serializer).
|
94
|
-
|
95
|
-
#### TODO:
|
96
|
-
Finished feature, will add documentation sson:
|
97
|
-
- Realtionships - `has_one`, `has_many`
|
98
|
-
- Filters & Nested Filters
|
99
|
-
- Reponse bag
|
100
|
-
|
101
|
-
## License
|
102
26
|
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
|
103
27
|
|
104
|
-
|
@@ -8,7 +8,7 @@ class AmsAuthorFastSerializer < ActiveModel::Serializer
|
|
8
8
|
end
|
9
9
|
|
10
10
|
class AmsPostFastSerializer < ActiveModel::Serializer
|
11
|
-
attributes :id, :body, :title, :author_id
|
11
|
+
attributes :id, :body, :title, :author_id, :created_at
|
12
12
|
end
|
13
13
|
|
14
14
|
class AmsPostWithHasOneFastSerializer < ActiveModel::Serializer
|
@@ -39,7 +39,7 @@ def benchmark_ams(prefix, serializer, options = {})
|
|
39
39
|
end
|
40
40
|
|
41
41
|
|
42
|
-
benchmark_ams "HasOne", AmsPostWithHasOneFastSerializer
|
43
42
|
benchmark_ams "Simple", AmsPostFastSerializer
|
43
|
+
benchmark_ams "HasOne", AmsPostWithHasOneFastSerializer
|
44
44
|
benchmark_ams "Except", AmsPostWithHasOneFastSerializer, except: [:title]
|
45
45
|
benchmark_ams "Include", AmsPostWithHasOneFastSerializer, include: [:id, :body, :author_id, :author]
|
data/benchmarks/bm_panko_json.rb
CHANGED
@@ -0,0 +1,21 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
require_relative "./benchmarking_support"
|
3
|
+
require_relative "./app"
|
4
|
+
|
5
|
+
class NotSerializer
|
6
|
+
end
|
7
|
+
|
8
|
+
class RealSerializer < Panko::Serializer
|
9
|
+
end
|
10
|
+
|
11
|
+
Benchmark.run("CantFindConst") do
|
12
|
+
Panko::SerializerResolver.resolve("cant_find_const")
|
13
|
+
end
|
14
|
+
|
15
|
+
Benchmark.run("NotSerializer") do
|
16
|
+
Panko::SerializerResolver.resolve("not")
|
17
|
+
end
|
18
|
+
|
19
|
+
Benchmark.run("RealSerializer") do
|
20
|
+
Panko::SerializerResolver.resolve("real")
|
21
|
+
end
|
data/docs/README.md
ADDED
@@ -0,0 +1,11 @@
|
|
1
|
+
* [Panko](../README.md)
|
2
|
+
* [Getting Stated](getting-started.md)
|
3
|
+
* Reference
|
4
|
+
* [Attributes](attributes.md)
|
5
|
+
* [Associations](associations.md)
|
6
|
+
* [Response bag](response-bag.md)
|
7
|
+
* [Performance](performance.md)
|
8
|
+
* [Design Choices](design-choices.md)
|
9
|
+
|
10
|
+
|
11
|
+
|
@@ -0,0 +1,76 @@
|
|
1
|
+
# Associations
|
2
|
+
|
3
|
+
A serializer can define it's own associations - both `has_many` and `has_one` to serializer under the context of the object.
|
4
|
+
|
5
|
+
For example:
|
6
|
+
|
7
|
+
```ruby
|
8
|
+
class PostSerializer < Panko::Serailizer
|
9
|
+
attributes :title, :body
|
10
|
+
|
11
|
+
has_one :author, serializer: AuthorSerializer
|
12
|
+
has_many :comments, each_serializer: CommentSerializer
|
13
|
+
end
|
14
|
+
```
|
15
|
+
|
16
|
+
### Inference
|
17
|
+
|
18
|
+
Panko can find the type of the serializer by looking at the realtionship name, so instead specifying
|
19
|
+
the serializer at the above example, we can -
|
20
|
+
|
21
|
+
```ruby
|
22
|
+
class PostSerializer < Panko::Serailizer
|
23
|
+
attributes :title, :body
|
24
|
+
|
25
|
+
has_one :author
|
26
|
+
has_many :comments
|
27
|
+
end
|
28
|
+
```
|
29
|
+
|
30
|
+
The logic of inferencing is -
|
31
|
+
- Take the name of the relationship (for example - `:author` / `:comments`) singularize and camelize it
|
32
|
+
- Look for const defined with the name aboe and "Serializer" suffix (by using `Object.const_get`)
|
33
|
+
|
34
|
+
> If Panko can't find the serializer it will throw an error on startup time, for example: `Can't find serializer for PostSerializer.author has_one relationship`
|
35
|
+
|
36
|
+
## Nested Filters
|
37
|
+
|
38
|
+
As talked before, Panko allows you to filter the attributes of a serializer.
|
39
|
+
But Panko let you take that step further, and filters the attributes of you associations so you can re-use your serializers in your application.
|
40
|
+
|
41
|
+
For example, let's say one portion of the application needs to serializer list of posts and serializer their - `title`, `body`, author's id and comments id.
|
42
|
+
|
43
|
+
We can declare tailored serializer for this, or we can re-use the above defined serializer - `PostSerializer` and use nested filters.
|
44
|
+
|
45
|
+
```ruby
|
46
|
+
posts = Post.all
|
47
|
+
|
48
|
+
Panko::ArraySerializer.new(posts, only: {
|
49
|
+
instance: [:title, :body, :author, :comments],
|
50
|
+
author: [:id],
|
51
|
+
comments: [:id],
|
52
|
+
})
|
53
|
+
```
|
54
|
+
|
55
|
+
Let's disect `only` option we passed -
|
56
|
+
* `instance` - list of attributes (and associations) we want to serializer for current instance of the serializer, in this case - `PostSerializer`.
|
57
|
+
* `author`, `comments` - here we specify the list of attributes we want to serialize for each association.
|
58
|
+
|
59
|
+
It's important to note that Nested Filters, are recursive, in other words, we can filter the association's associations.
|
60
|
+
|
61
|
+
For example, `CommentSerializer` have has_one association `Author`, and for each `comments.author` we only it's name.
|
62
|
+
|
63
|
+
```ruby
|
64
|
+
posts = Post.all
|
65
|
+
|
66
|
+
Panko::ArraySerializer.new(posts, only: {
|
67
|
+
instance: [:title, :body, :author, :comments],
|
68
|
+
author: [:id],
|
69
|
+
comments: {
|
70
|
+
instance: [:id, :author],
|
71
|
+
author: [:name]
|
72
|
+
}
|
73
|
+
})
|
74
|
+
```
|
75
|
+
|
76
|
+
As you see now in `comments` the `instance` have differenet meaning, the `CommentSerializer`.
|
data/docs/attributes.md
ADDED
@@ -0,0 +1,104 @@
|
|
1
|
+
# Attributes
|
2
|
+
|
3
|
+
Attributes allow you to specify which record attributes you want to serialize,
|
4
|
+
There are two types of attributes:
|
5
|
+
|
6
|
+
* Field - simple columns defined on the record it self.
|
7
|
+
* Virtual/Method - this allows to include properties beyond simple fields.
|
8
|
+
|
9
|
+
|
10
|
+
```ruby
|
11
|
+
class UserSerializer < Panko::Serializer
|
12
|
+
attributes :full_name
|
13
|
+
|
14
|
+
def full_name
|
15
|
+
"#{object.first_name} #{object.last_name}"
|
16
|
+
end
|
17
|
+
end
|
18
|
+
```
|
19
|
+
|
20
|
+
## Field Attributes
|
21
|
+
|
22
|
+
Using field attributes you can control which columns of the given ActiveRecord object you want to serialize.
|
23
|
+
|
24
|
+
Instead of relying ActiveRecord to do it's type casting, Panko does on it's own for performance reasons (read more in [Design Choices](design-choices.md#type-casting)).
|
25
|
+
|
26
|
+
|
27
|
+
## Method Attributes
|
28
|
+
|
29
|
+
Method attributes are used when your serialized values can be derived from the object you are serializing.
|
30
|
+
|
31
|
+
The serializer's attribute methods can access the object being serialized as `object` -
|
32
|
+
|
33
|
+
```ruby
|
34
|
+
class PostSerializer < Panko::Serializer
|
35
|
+
def author_name
|
36
|
+
"#{object.author.first_name} #{object.author.last_name}"
|
37
|
+
end
|
38
|
+
end
|
39
|
+
```
|
40
|
+
|
41
|
+
Another useful, thing you can pass your serializer is `context`, a `context` is a bag of data whom your serializer may need.
|
42
|
+
|
43
|
+
For example, here we will pass the current user:
|
44
|
+
```ruby
|
45
|
+
class UserSerializer < Panko::Serializer
|
46
|
+
attributes :id, :email
|
47
|
+
|
48
|
+
def feature_flags
|
49
|
+
context[:feature_flags]
|
50
|
+
end
|
51
|
+
end
|
52
|
+
|
53
|
+
serializer = UserSerializer.new(context: {
|
54
|
+
feature_flags: FeatureFlags.all
|
55
|
+
})
|
56
|
+
|
57
|
+
serializer.serialize(User.first)
|
58
|
+
```
|
59
|
+
|
60
|
+
## Filters
|
61
|
+
|
62
|
+
Filters allows us to reduce the amount of attributes we can serialize, therefore reduce the data usage & performance of serializing.
|
63
|
+
|
64
|
+
There are two types of filters:
|
65
|
+
* only - use those attributes **only** and nothing else
|
66
|
+
* except - all attributes **except** those attributes
|
67
|
+
|
68
|
+
Usage example:
|
69
|
+
```ruby
|
70
|
+
class UserSerializer < Panko::Serializer
|
71
|
+
attributes :id, :name, :email
|
72
|
+
end
|
73
|
+
|
74
|
+
# this line will return { 'name': '..' }
|
75
|
+
UserSerializer.new(only: [:name]).serialize(User.first)
|
76
|
+
|
77
|
+
# this line will return { 'id': '..', 'email': ... }
|
78
|
+
UserSerializer.new(except: [:name]).serialize(User.first)
|
79
|
+
```
|
80
|
+
|
81
|
+
## Aliases
|
82
|
+
|
83
|
+
Let's say we have attribute name that we want to expose to client as different name, the current way of doing so is using method attribute, for example:
|
84
|
+
|
85
|
+
```ruby
|
86
|
+
class PostSerializer < Panko::Serializer
|
87
|
+
attributes :published_at
|
88
|
+
|
89
|
+
def published_at
|
90
|
+
object.created_at
|
91
|
+
end
|
92
|
+
end
|
93
|
+
```
|
94
|
+
|
95
|
+
The downside of this approach is that `created_at` skips Panko's type casting, therefore we get direct hit on performance.
|
96
|
+
|
97
|
+
To fix this, we can use aliases -
|
98
|
+
|
99
|
+
```ruby
|
100
|
+
class PostSerializer < Panko::Serializer
|
101
|
+
aliases created_at: :published_at
|
102
|
+
end
|
103
|
+
```
|
104
|
+
|
@@ -0,0 +1,122 @@
|
|
1
|
+
# Design Choices
|
2
|
+
|
3
|
+
In short, Panko, is a serializer for ActiveRecord objects (it can't serialize any other object), which strives for high performance & simple API (which is inspired by ActiveModelSerializers).
|
4
|
+
|
5
|
+
Its performance is achieved by:
|
6
|
+
|
7
|
+
* `Oj::StringWriter` - I will elaborate later.
|
8
|
+
* Type casting — instead of relying on ActiveRecord to do its type cast, Panko is doing it by itself.
|
9
|
+
* Figuring out the metadata, ahead of time — therefore, we ask less questions during the `serialization loop`.
|
10
|
+
|
11
|
+
|
12
|
+
## Serialization overview
|
13
|
+
|
14
|
+
First, let's start with overview. Let's say we want to serialize `User` object, which has
|
15
|
+
`first_name`, `last_name`, `age`, and `email` properties.
|
16
|
+
|
17
|
+
The serializer definition will be something like this:
|
18
|
+
|
19
|
+
```ruby
|
20
|
+
class UserSerializer < Panko::Serializer
|
21
|
+
attributes :name, :age, :email
|
22
|
+
|
23
|
+
def name
|
24
|
+
"#{object.first_name} #{object.last_name}"
|
25
|
+
end
|
26
|
+
end
|
27
|
+
```
|
28
|
+
|
29
|
+
And the usage of this serializer will be:
|
30
|
+
|
31
|
+
```ruby
|
32
|
+
# fetch user from database
|
33
|
+
user = User.first
|
34
|
+
|
35
|
+
# create serializer, with empty options
|
36
|
+
serializer = UserSerilizer.new
|
37
|
+
|
38
|
+
# serialize to JSON
|
39
|
+
serializer.serialize_to_json(user)
|
40
|
+
```
|
41
|
+
|
42
|
+
Let's go over the steps that Panko will execute behind the scenes for this flow.
|
43
|
+
_I will skip the serializer definition part, because it's fairly simple and straightforward (see `lib/panko/serializer.rb`)_
|
44
|
+
|
45
|
+
First step, while initializing the UserSerializer, we will create a **Serialization Descriptor** for this class.
|
46
|
+
Serialization Descriptor's goal is to answer those questions:
|
47
|
+
|
48
|
+
* Which fields do we have? In our case, `:age`, `:email`
|
49
|
+
* Which method fields do we have? In our case `:name`
|
50
|
+
* Which associations do we have (and their serialization descriptors)
|
51
|
+
|
52
|
+
The serialization description is also responsible for filtering the attributes (`only` \ `except`).
|
53
|
+
|
54
|
+
Now, that we have the serialization descriptor, we are finished with the Ruby part of Panko, and all we did here is done in *initialization time* and now we move to C code.
|
55
|
+
|
56
|
+
In C land, we take the `user` object and the serialization descriptor, and start the serialization process which is separated to 4 parts:
|
57
|
+
|
58
|
+
* Serializing Fields - looping through serialization descriptor's `fields` and read them from the ActiveRecord object (see `Type Casting`) and write them to the writer.
|
59
|
+
* Serializing Method Fields - creating (a cached) serializer instance, setting its `@object` and `@context`, calling all the method fields and writing them to the writer.
|
60
|
+
* Serializing associations — this is simple, once we have fields + method fields, we just repeat the process.
|
61
|
+
|
62
|
+
Once this is finished, we have nice JSON string.
|
63
|
+
Now let's dig deeper.
|
64
|
+
|
65
|
+
## Interesting parts
|
66
|
+
|
67
|
+
### Oj::StringWriter
|
68
|
+
|
69
|
+
If you read the code of ActiveRecord serialization code in Ruby, you will observe this flow:
|
70
|
+
|
71
|
+
1. Get an array of ActiveRecord objects (`User.all` for example)
|
72
|
+
2. Build new array of hashes where each hash is `User` with the attributes we selected
|
73
|
+
3. The JSON serializer, takes this array of hashes and loop them, and converts it to JSON string
|
74
|
+
|
75
|
+
This entire process is expensive in terms of Memory & CPU, and this where the combination of Panko and Oj::StringWriter really shines.
|
76
|
+
|
77
|
+
In Panko, the serialization process of the above is:
|
78
|
+
|
79
|
+
1. Get an array of ActiveRecord objects (`User.all` for example)
|
80
|
+
2. Create `Oj::StringWriter` and feed the values to it, via `push_value` / `push_object` / `push_object` and behind the scene, `Oj::StringWriter` will serialize the objects incrementally into a string.
|
81
|
+
3. Get from `Oj::StringWriter` the completed JSON string — which is a no-op, since `Oj::StringWriter` already built the string.
|
82
|
+
|
83
|
+
### Figuring out the metadata, ahead of time.
|
84
|
+
|
85
|
+
Another observation I noticed in the ruby serializers is that they ask and do a lot in a serialization loop:
|
86
|
+
|
87
|
+
* Is this field a method? is it a property?
|
88
|
+
* Which fields and associations do I need for the serializer to consider the `only` and `except` options
|
89
|
+
* What is the serializer of this has_one association?
|
90
|
+
|
91
|
+
Panko tries to ask the bare minimum in serialization by building `Serialization Descriptor` for each serialization and caching it.
|
92
|
+
|
93
|
+
The Serialization Descriptor will do the filtering of `only` and `except` and will check if a field is a method or not (therefore Panko doesn't have list of `attributes`)
|
94
|
+
|
95
|
+
|
96
|
+
### Type Casting
|
97
|
+
|
98
|
+
This is the final part, which helped yield most of the performance improvements.
|
99
|
+
In ActiveRecord, when we read a value of attribute, it does type casting of the DB value to its real ruby type.
|
100
|
+
|
101
|
+
For example, time strings are converted to Time objects, Strings are duplicated, and Integers are converts from their values to Number.
|
102
|
+
|
103
|
+
This type casting is really expensive, as it's responsible for most of the allocations in the serialization flow and most of them can be "relaxed".
|
104
|
+
|
105
|
+
If we think about it, we don't need to duplicate strings or convert time strings to time objects or even parse JSON strings for the JSON serialization process.
|
106
|
+
|
107
|
+
What Panko does is that if we have ActiveRecord type string, we won't duplicate it.
|
108
|
+
If we have an integer string value, we will convert it to an integer, and the same goes for other types.
|
109
|
+
|
110
|
+
All of these conversions are done in C, which of course yields a big performance improvement.
|
111
|
+
|
112
|
+
#### Time type casting
|
113
|
+
While you read Panko source code, you will encounter the time type casting and immediately you will have a "WTF?" moment.
|
114
|
+
|
115
|
+
The idea behind the time type casting code relies on the end result of JSON type casting — what we need in order to serialize Time to JSON? UTC ISO8601 time format representation.
|
116
|
+
|
117
|
+
The time type casting works as follows:
|
118
|
+
|
119
|
+
* If it's a string that ends with `Z`, and the strings matches the UTC ISO8601 regex, then we just return the string.
|
120
|
+
* If it's a string and it doesn't follow the rules above, we check if it's a timestamp in database format and convert it via regex + string concat to UTC ISO8601 - Yes, there is huge assumption here, that the database returns UTC timestamps — this will be configureable (before Panko official release).
|
121
|
+
* If it's none of the above, I will let ActiveRecord type casting do it's magic.
|
122
|
+
|
data/docs/docpress.json
ADDED
@@ -0,0 +1,47 @@
|
|
1
|
+
# Getting Started
|
2
|
+
|
3
|
+
## Installation
|
4
|
+
|
5
|
+
To install Panko, all you need is to add it to your Gemfile:
|
6
|
+
|
7
|
+
```ruby
|
8
|
+
gem "panko_serializer"
|
9
|
+
```
|
10
|
+
|
11
|
+
Then, install it on the command line:
|
12
|
+
|
13
|
+
```
|
14
|
+
> bundle install
|
15
|
+
```
|
16
|
+
|
17
|
+
|
18
|
+
## Creating your first serializer
|
19
|
+
|
20
|
+
Let's create serializer and use it inside Rails controller.
|
21
|
+
|
22
|
+
```ruby
|
23
|
+
class PostSerializer < Panko::Serializer
|
24
|
+
attributes :title
|
25
|
+
end
|
26
|
+
|
27
|
+
class UserSerializer < Panko::Serializer
|
28
|
+
attributes :id, :name, :age
|
29
|
+
|
30
|
+
has_many :posts, serializer: PostSerializer
|
31
|
+
end
|
32
|
+
```
|
33
|
+
|
34
|
+
As you can see, defining serializers is simple and resembles ActiveModelSerializers 0.9,
|
35
|
+
To utilize the `UserSerializer` inside a Rails controller and serialize some users, all we need to do is:
|
36
|
+
|
37
|
+
```ruby
|
38
|
+
class UsersController < ApplicationController
|
39
|
+
def index
|
40
|
+
users = User.includes(:posts).all
|
41
|
+
render json: Panko::ArraySerializer.new(users, each_serializer: UserSerializer).to_json
|
42
|
+
end
|
43
|
+
end
|
44
|
+
```
|
45
|
+
|
46
|
+
And voila, we have endpoint which serialize users using Panko!
|
47
|
+
|
data/docs/performance.md
ADDED
@@ -0,0 +1,35 @@
|
|
1
|
+
# Performance
|
2
|
+
|
3
|
+
The performance of Panko is measured using microbenchmarks and load testing.
|
4
|
+
|
5
|
+
## Microbenchmarks
|
6
|
+
|
7
|
+
The following microbenchmarks are run on MacBook Pro (Retina, 15-inch, Mid 2015), Ruby 2.4 with Rails 4.2
|
8
|
+
demonstrating the performance of ActiveModelSerializers 0.9 and Panko 0.3.3
|
9
|
+
|
10
|
+
|
11
|
+
Benchmark | AMS ip/s | Panko ip/s
|
12
|
+
---------------------------------------|----------|-----------------
|
13
|
+
| Simple_Posts_2300 | 25.81 | 135.29 |
|
14
|
+
| Simple_Posts_50 | 1,248.39 | 6,518.68 |
|
15
|
+
| HasOne_Posts_2300 | 11.33 | 73.42 |
|
16
|
+
| HasOne_Posts_50 | 523.14 | 4,985.41 |
|
17
|
+
|
18
|
+
> The corresponding benchmarks are `benchmarks/bm_active_model_serializers.rb` and `benchmarks/bm_panko_json.rb`
|
19
|
+
|
20
|
+
|
21
|
+
## Real-world benchmark
|
22
|
+
|
23
|
+
The real-world benchmark here is endpoint which serializes 7,884 entries with 48 attributes and no associations.
|
24
|
+
The benchmark took place in environment that simulates production environment and run using `wrk` from machine on the same cluster.
|
25
|
+
|
26
|
+
|
27
|
+
Metric | AMS | Panko
|
28
|
+
------------ |------------ | -------------
|
29
|
+
Avg Response Time| 4.89s| 1.48s|
|
30
|
+
Max Response Time| 5.42s| 1.83s|
|
31
|
+
99th Response Time| 5.42s| 1.74s|
|
32
|
+
Total Requests| 61| 202|
|
33
|
+
|
34
|
+
|
35
|
+
*Thanks to [Bringg](https://www.bringg.com) for providing the infrastructrue for the benchmarks*
|
@@ -0,0 +1,53 @@
|
|
1
|
+
# Response
|
2
|
+
|
3
|
+
Let's say you have some JSON payload which can is constructed using Panko serialization result,
|
4
|
+
like this:
|
5
|
+
|
6
|
+
```ruby
|
7
|
+
class PostsController < ApplicationController
|
8
|
+
def index
|
9
|
+
posts = Post.all
|
10
|
+
render json: {
|
11
|
+
success: true,
|
12
|
+
total_count: posts.count,
|
13
|
+
posts: Panko::ArraySerializer.new(posts, each_serializer: PostSerializer).to_json
|
14
|
+
}
|
15
|
+
end
|
16
|
+
end
|
17
|
+
```
|
18
|
+
|
19
|
+
The output of the above will be json string (for `posts`) inside json string and this were `Panko::Response` shines.
|
20
|
+
|
21
|
+
```ruby
|
22
|
+
class PostsController < ApplicationController
|
23
|
+
def index
|
24
|
+
posts = Post.all
|
25
|
+
render json: Panko::Response.new(
|
26
|
+
success: true,
|
27
|
+
total_count: posts.count,
|
28
|
+
posts: Panko::ArraySerializer.new(posts, each_serializer: PostSerializer)
|
29
|
+
)
|
30
|
+
end
|
31
|
+
end
|
32
|
+
```
|
33
|
+
|
34
|
+
And everything will work as expected!
|
35
|
+
|
36
|
+
## JsonValue
|
37
|
+
|
38
|
+
Let's take the above example further, we serialized the posts and cached it as JSON string in our Cache.
|
39
|
+
Now, you can wrap the cached value with `Panko::JsonValue`, like here -
|
40
|
+
|
41
|
+
```ruby
|
42
|
+
class PostsController < ApplicationController
|
43
|
+
def index
|
44
|
+
posts = Cache.get("/posts")
|
45
|
+
|
46
|
+
render json: Panko::Response.new(
|
47
|
+
success: true,
|
48
|
+
total_count: posts.count,
|
49
|
+
posts: Panko::JsonValue.from(posts)
|
50
|
+
)
|
51
|
+
end
|
52
|
+
end
|
53
|
+
```
|
@@ -56,6 +56,11 @@ VALUE association_name_sym_ref(VALUE self) {
|
|
56
56
|
return association->name_sym;
|
57
57
|
}
|
58
58
|
|
59
|
+
VALUE association_name_str_ref(VALUE self) {
|
60
|
+
Association association = (Association)DATA_PTR(self);
|
61
|
+
return association->name_str;
|
62
|
+
}
|
63
|
+
|
59
64
|
VALUE association_descriptor_ref(VALUE self) {
|
60
65
|
Association association = (Association)DATA_PTR(self);
|
61
66
|
return association->rb_descriptor;
|
@@ -76,6 +81,7 @@ void panko_init_association(VALUE mPanko) {
|
|
76
81
|
rb_define_module_function(cAssociation, "new", association_new, -1);
|
77
82
|
|
78
83
|
rb_define_method(cAssociation, "name_sym", association_name_sym_ref, 0);
|
84
|
+
rb_define_method(cAssociation, "name_str", association_name_str_ref, 0);
|
79
85
|
rb_define_method(cAssociation, "descriptor", association_descriptor_ref, 0);
|
80
86
|
rb_define_method(cAssociation, "descriptor=", association_decriptor_aset, 1);
|
81
87
|
}
|
@@ -20,9 +20,7 @@ module Panko
|
|
20
20
|
|
21
21
|
backend.type = descriptor.type
|
22
22
|
|
23
|
-
backend.attributes = descriptor.attributes.
|
24
|
-
Attribute.create(attr.name, alias_name: attr.alias_name)
|
25
|
-
end
|
23
|
+
backend.attributes = descriptor.attributes.dup
|
26
24
|
|
27
25
|
backend.method_fields = descriptor.method_fields.dup
|
28
26
|
|
@@ -30,21 +28,8 @@ module Panko
|
|
30
28
|
backend.serializer = descriptor.serializer.reset
|
31
29
|
end
|
32
30
|
|
33
|
-
backend.has_many_associations = descriptor.has_many_associations.
|
34
|
-
|
35
|
-
assoc.name_sym,
|
36
|
-
assoc.name_sym.to_s,
|
37
|
-
Panko::SerializationDescriptor.duplicate(assoc.descriptor)
|
38
|
-
)
|
39
|
-
end
|
40
|
-
|
41
|
-
backend.has_one_associations = descriptor.has_one_associations.map do |assoc|
|
42
|
-
Panko::Association.new(
|
43
|
-
assoc.name_sym,
|
44
|
-
assoc.name_sym.to_s,
|
45
|
-
Panko::SerializationDescriptor.duplicate(assoc.descriptor)
|
46
|
-
)
|
47
|
-
end
|
31
|
+
backend.has_many_associations = descriptor.has_many_associations.dup
|
32
|
+
backend.has_one_associations = descriptor.has_one_associations.dup
|
48
33
|
|
49
34
|
backend
|
50
35
|
end
|
@@ -58,7 +43,8 @@ module Panko
|
|
58
43
|
attributes_only_filters, associations_only_filters = resolve_filters(options, :only)
|
59
44
|
attributes_except_filters, associations_except_filters = resolve_filters(options, :except)
|
60
45
|
|
61
|
-
apply_attribute_filters
|
46
|
+
self.attributes = apply_attribute_filters(
|
47
|
+
self.attributes,
|
62
48
|
attributes_only_filters,
|
63
49
|
attributes_except_filters
|
64
50
|
)
|
@@ -120,10 +106,17 @@ module Panko
|
|
120
106
|
filters = {}
|
121
107
|
filters[:only] = only_filter unless only_filter.nil?
|
122
108
|
filters[:except] = except_filter unless except_filter.nil?
|
123
|
-
|
109
|
+
|
110
|
+
unless filters.empty?
|
111
|
+
next Panko::Association.new(
|
112
|
+
name,
|
113
|
+
association.name_str,
|
114
|
+
Panko::SerializationDescriptor.build(descriptor.type, filters)
|
115
|
+
)
|
116
|
+
end
|
124
117
|
|
125
118
|
association
|
126
|
-
end
|
119
|
+
end
|
127
120
|
end
|
128
121
|
|
129
122
|
def resolve_filters(options, filter)
|
@@ -152,9 +145,9 @@ module Panko
|
|
152
145
|
fields
|
153
146
|
end
|
154
147
|
|
155
|
-
def apply_attribute_filters
|
148
|
+
def apply_attribute_filters(attributes, only, except)
|
156
149
|
unless only.empty?
|
157
|
-
|
150
|
+
attributes = attributes.select do |attribute|
|
158
151
|
name_to_check = attribute.name
|
159
152
|
name_to_check = attribute.alias_name unless attribute.alias_name.nil?
|
160
153
|
|
@@ -163,13 +156,15 @@ module Panko
|
|
163
156
|
end
|
164
157
|
|
165
158
|
unless except.empty?
|
166
|
-
|
159
|
+
attributes = attributes.reject do |attribute|
|
167
160
|
name_to_check = attribute.name
|
168
161
|
name_to_check = attribute.alias_name unless attribute.alias_name.nil?
|
169
162
|
|
170
163
|
except.include?(name_to_check.to_sym)
|
171
164
|
end
|
172
165
|
end
|
166
|
+
|
167
|
+
attributes
|
173
168
|
end
|
174
169
|
end
|
175
170
|
end
|
data/lib/panko/serializer.rb
CHANGED
@@ -36,8 +36,13 @@ module Panko
|
|
36
36
|
@_descriptor.method_fields << method
|
37
37
|
end
|
38
38
|
|
39
|
-
def has_one(name, options)
|
39
|
+
def has_one(name, options = {})
|
40
40
|
serializer_const = options[:serializer]
|
41
|
+
serializer_const = Panko::SerializerResolver.resolve(name.to_s) if serializer_const.nil?
|
42
|
+
|
43
|
+
if serializer_const.nil?
|
44
|
+
raise "Can't find serializer for #{self.name}.#{name} has_one relationship."
|
45
|
+
end
|
41
46
|
|
42
47
|
@_descriptor.has_one_associations << Panko::Association.new(
|
43
48
|
name,
|
@@ -46,8 +51,13 @@ module Panko
|
|
46
51
|
)
|
47
52
|
end
|
48
53
|
|
49
|
-
def has_many(name, options)
|
54
|
+
def has_many(name, options = {})
|
50
55
|
serializer_const = options[:serializer] || options[:each_serializer]
|
56
|
+
serializer_const = Panko::SerializerResolver.resolve(name.to_s) if serializer_const.nil?
|
57
|
+
|
58
|
+
if serializer_const.nil?
|
59
|
+
raise "Can't find serializer for #{self.name}.#{name} has_many relationship."
|
60
|
+
end
|
51
61
|
|
52
62
|
@_descriptor.has_many_associations << Panko::Association.new(
|
53
63
|
name,
|
@@ -0,0 +1,29 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
require "byebug"
|
3
|
+
|
4
|
+
class Panko::SerializerResolver
|
5
|
+
|
6
|
+
def self.resolve(name)
|
7
|
+
serializer_name = "#{name.singularize.camelize}Serializer"
|
8
|
+
serializer_const = self.safe_const_get(serializer_name)
|
9
|
+
|
10
|
+
return nil if serializer_const.nil?
|
11
|
+
return nil unless self.is_serializer(serializer_const)
|
12
|
+
|
13
|
+
serializer_const
|
14
|
+
end
|
15
|
+
|
16
|
+
|
17
|
+
private
|
18
|
+
|
19
|
+
def self.is_serializer(const)
|
20
|
+
const < Panko::Serializer
|
21
|
+
end
|
22
|
+
|
23
|
+
def self.safe_const_get(name)
|
24
|
+
Object.const_get(name)
|
25
|
+
rescue NameError
|
26
|
+
nil
|
27
|
+
end
|
28
|
+
|
29
|
+
end
|
data/lib/panko/version.rb
CHANGED
data/lib/panko_serializer.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: panko_serializer
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.3.
|
4
|
+
version: 0.3.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Yosi Attias
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-12-
|
11
|
+
date: 2017-12-23 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
@@ -97,7 +97,6 @@ files:
|
|
97
97
|
- LICENSE.txt
|
98
98
|
- README.md
|
99
99
|
- Rakefile
|
100
|
-
- _config.yml
|
101
100
|
- benchmarks/BENCHMARKS.md
|
102
101
|
- benchmarks/allocs.rb
|
103
102
|
- benchmarks/app.rb
|
@@ -107,12 +106,21 @@ files:
|
|
107
106
|
- benchmarks/bm_panko_json.rb
|
108
107
|
- benchmarks/bm_panko_object.rb
|
109
108
|
- benchmarks/bm_serialization_descriptor.rb
|
109
|
+
- benchmarks/bm_serializer_resolver.rb
|
110
110
|
- benchmarks/profile.rb
|
111
111
|
- benchmarks/sanity.rb
|
112
112
|
- benchmarks/setup.rb
|
113
113
|
- benchmarks/type_casts/bm_active_record.rb
|
114
114
|
- benchmarks/type_casts/bm_panko.rb
|
115
115
|
- benchmarks/type_casts/support.rb
|
116
|
+
- docs/README.md
|
117
|
+
- docs/associations.md
|
118
|
+
- docs/attributes.md
|
119
|
+
- docs/design-choices.md
|
120
|
+
- docs/docpress.json
|
121
|
+
- docs/getting-started.md
|
122
|
+
- docs/performance.md
|
123
|
+
- docs/response-bag.md
|
116
124
|
- ext/panko_serializer/association.c
|
117
125
|
- ext/panko_serializer/association.h
|
118
126
|
- ext/panko_serializer/attribute.c
|
@@ -133,6 +141,7 @@ files:
|
|
133
141
|
- lib/panko/response.rb
|
134
142
|
- lib/panko/serialization_descriptor.rb
|
135
143
|
- lib/panko/serializer.rb
|
144
|
+
- lib/panko/serializer_resolver.rb
|
136
145
|
- lib/panko/version.rb
|
137
146
|
- lib/panko_serializer.rb
|
138
147
|
- panko_serializer.gemspec
|
@@ -156,7 +165,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
156
165
|
version: '0'
|
157
166
|
requirements: []
|
158
167
|
rubyforge_project:
|
159
|
-
rubygems_version: 2.
|
168
|
+
rubygems_version: 2.7.3
|
160
169
|
signing_key:
|
161
170
|
specification_version: 4
|
162
171
|
summary: Fast serialization for ActiveModel
|
data/_config.yml
DELETED
@@ -1 +0,0 @@
|
|
1
|
-
theme: jekyll-theme-cayman
|