ruby_memprofiler_pprof 0.0.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/ext/ruby_memprofiler_pprof/backtrace.c +429 -0
- data/ext/ruby_memprofiler_pprof/collector.c +1055 -0
- data/ext/ruby_memprofiler_pprof/compat.c +182 -0
- data/ext/ruby_memprofiler_pprof/extconf.rb +72 -0
- data/ext/ruby_memprofiler_pprof/pprof.upb.c +170 -0
- data/ext/ruby_memprofiler_pprof/pprof.upb.h +848 -0
- data/ext/ruby_memprofiler_pprof/pprof_out.c +285 -0
- data/ext/ruby_memprofiler_pprof/ruby_memprofiler_pprof.c +11 -0
- data/ext/ruby_memprofiler_pprof/ruby_memprofiler_pprof.h +301 -0
- data/ext/ruby_memprofiler_pprof/strtab.c +391 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/BUILD +719 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/CONTRIBUTING.md +37 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/DESIGN.md +201 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/LICENSE +26 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/README.md +78 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/WORKSPACE +58 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/bazel/BUILD +53 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/bazel/amalgamate.py +129 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/bazel/build_defs.bzl +160 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/bazel/lua.BUILD +127 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/bazel/protobuf.patch +54 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/bazel/py_proto_library.bzl +137 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/bazel/python_downloads.bzl +84 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/bazel/system_python.bzl +101 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/bazel/upb_proto_library.bzl +388 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/bazel/workspace_deps.bzl +89 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/BUILD +252 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/BUILD.googleapis +54 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/benchmark.cc +333 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/build_defs.bzl +88 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/compare.py +118 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/descriptor.proto +888 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/descriptor_sv.proto +890 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/empty.proto +6 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/gen_protobuf_binary_cc.py +64 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/gen_synthetic_protos.py +118 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/benchmarks/gen_upb_binary_c.py +65 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/cmake/BUILD.bazel +102 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/cmake/README.md +23 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/cmake/build_defs.bzl +73 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/cmake/make_cmakelists.py +340 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/cmake/staleness_test.py +57 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/cmake/staleness_test_lib.py +186 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/docs/render.py +43 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/docs/style-guide.md +65 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/docs/vs-cpp-protos.md +255 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/docs/wrapping-upb.md +444 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/BUILD +216 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/convert.c +394 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/convert.h +63 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/descriptor.c +1694 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/descriptor.h +80 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/descriptor_containers.c +704 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/descriptor_containers.h +114 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/descriptor_pool.c +650 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/descriptor_pool.h +48 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/dist/BUILD.bazel +193 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/dist/dist.bzl +190 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/extension_dict.c +247 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/extension_dict.h +39 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/map.c +522 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/map.h +66 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/message.c +1909 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/message.h +101 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/minimal_test.py +183 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/BUILD +70 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/README.md +11 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/descriptor_database_test_wrapper.py +30 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/descriptor_pool_test_wrapper.py +45 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/descriptor_test_wrapper.py +46 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/generator_test_wrapper.py +30 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/json_format_test_wrapper.py +30 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/keywords_test_wrapper.py +30 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/message_factory_test_wrapper.py +37 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/message_test_wrapper.py +52 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/proto_builder_test_wrapper.py +32 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/pyproto_test_wrapper.bzl +36 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/reflection_test_wrapper.py +45 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/service_reflection_test_wrapper.py +30 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/symbol_database_test_wrapper.py +30 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/text_encoding_test_wrapper.py +30 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/text_format_test_wrapper.py +30 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/unknown_fields_test_wrapper.py +30 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/well_known_types_test_wrapper.py +36 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/pb_unit_tests/wire_format_test_wrapper.py +30 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/protobuf.c +350 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/protobuf.h +230 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/py_extension.bzl +55 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/python_api.h +61 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/repeated.c +828 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/repeated.h +69 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/unknown_fields.c +404 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/unknown_fields.h +39 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/python/version_script.lds +6 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/third_party/lunit/LICENSE +32 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/third_party/lunit/README.google +9 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/third_party/lunit/console.lua +156 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/third_party/lunit/lunit.lua +725 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/third_party/utf8_range/BUILD +19 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/third_party/utf8_range/LICENSE +21 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/third_party/utf8_range/naive.c +92 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/third_party/utf8_range/range2-neon.c +157 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/third_party/utf8_range/range2-sse.c +170 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/third_party/utf8_range/utf8_range.h +9 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/BUILD.bazel +129 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/README.md +8 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/def.c +939 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/lua_proto_library.bzl +138 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/main.c +83 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/msg.c +1118 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/test.proto +69 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/test_upb.lua +846 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/upb.c +258 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/upb.h +132 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/upb.lua +58 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/bindings/lua/upbc.cc +134 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/collections.c +192 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/collections.h +174 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/conformance_upb.c +346 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/conformance_upb_failures.txt +1 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/decode.c +1221 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/decode.h +94 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/decode_fast.c +1055 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/decode_fast.h +153 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/decode_internal.h +211 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/def.c +3262 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/def.h +414 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/def.hpp +438 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/empty.proto +1 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/encode.c +604 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/encode.h +71 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/fuzz/BUILD +13 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/fuzz/file_descriptor_parsenew_fuzzer.cc +43 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/json_decode.c +1509 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/json_decode.h +47 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/json_encode.c +776 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/json_encode.h +62 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/mini_table.c +1147 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/mini_table.h +189 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/mini_table.hpp +112 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/mini_table_accessors.c +363 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/mini_table_accessors.h +263 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/mini_table_accessors_internal.h +59 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/mini_table_accessors_test.cc +425 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/mini_table_test.cc +230 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/msg.c +428 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/msg.h +114 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/msg_internal.h +836 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/msg_test.cc +491 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/msg_test.proto +195 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/port_def.inc +261 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/port_undef.inc +62 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/reflection.c +323 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/reflection.h +109 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/reflection.hpp +37 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/table.c +926 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/table_internal.h +385 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/test.proto +74 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/test_cpp.cc +186 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/test_cpp.proto +12 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/test_generated_code.cc +977 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/test_table.cc +580 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/text_encode.c +472 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/text_encode.h +64 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/upb.c +362 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/upb.h +378 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/upb.hpp +115 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/upb_internal.h +68 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/BUILD +121 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/README.md +7 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/compare.c +300 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/compare.h +66 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/compare_test.cc +236 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/def_to_proto.c +572 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/def_to_proto.h +62 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/def_to_proto_public_import_test.proto +32 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/def_to_proto_regular_import_test.proto +36 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/def_to_proto_test.cc +143 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/def_to_proto_test.proto +119 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/def_to_proto_weak_import_test.proto +28 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/def_to_proto_wweak_import_test.proto +28 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/required_fields.c +311 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/required_fields.h +94 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/required_fields_test.cc +202 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upb/util/required_fields_test.proto +48 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upbc/BUILD +78 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upbc/common.cc +77 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upbc/common.h +112 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upbc/protoc-gen-upb.cc +1997 -0
- data/ext/ruby_memprofiler_pprof/vendor/upb/upbc/protoc-gen-upbdefs.cc +193 -0
- data/lib/ruby_memprofiler_pprof/atfork.rb +77 -0
- data/lib/ruby_memprofiler_pprof/block_flusher.rb +61 -0
- data/lib/ruby_memprofiler_pprof/file_flusher.rb +45 -0
- data/lib/ruby_memprofiler_pprof/profile_app.rb +30 -0
- data/lib/ruby_memprofiler_pprof/profile_data.rb +18 -0
- data/lib/ruby_memprofiler_pprof/version.rb +5 -0
- data/lib/ruby_memprofiler_pprof.rb +8 -0
- data/libexec/ruby_memprofiler_pprof_profile +16 -0
- metadata +257 -0
@@ -0,0 +1,255 @@
|
|
1
|
+
<?% config.freshness.reviewed = '2022-04-12' %?>
|
2
|
+
|
3
|
+
# upb vs. C++ Protobuf Design
|
4
|
+
|
5
|
+
[upb](https://github.com/protocolbuffers/upb) is a small C protobuf library.
|
6
|
+
While some of the design follows in the footsteps of the C++ Protobuf Library,
|
7
|
+
upb departs from C++'s design in several key ways. This document compares
|
8
|
+
and contrasts the two libraries on several design points.
|
9
|
+
|
10
|
+
## Design Goals
|
11
|
+
|
12
|
+
Before we begin, it is worth calling out that upb and C++ have different design
|
13
|
+
goals, and this motivates some of the differences we will see.
|
14
|
+
|
15
|
+
C++ protobuf is a user-level library: it is designed to be used directly by C++
|
16
|
+
applications. These applications will expect a full-featured C++ API surface
|
17
|
+
that uses C++ idioms. The C++ library is also willing to add features to
|
18
|
+
increase server performance, even if these features would add size or complexity
|
19
|
+
to the library. Because C++ protobuf is a user-level library, API stability is
|
20
|
+
of utmost importance: breaking API changes are rare and carefully managed when
|
21
|
+
they do occur. The focus on C++ also means that ABI compatibility with C is not
|
22
|
+
a priority.
|
23
|
+
|
24
|
+
upb, on the other hand, is designed primarily to be wrapped by other languages.
|
25
|
+
It is a C protobuf kernel that forms the basis on which a user-level protobuf
|
26
|
+
library can be built. This means we prefer to keep the API surface as small and
|
27
|
+
orthogonal as possible. While upb supports all protobuf features required for
|
28
|
+
full conformance, upb prioritizes simplicity and small code size, and avoids
|
29
|
+
adding features like lazy fields that can accelerate some use cases but at great
|
30
|
+
cost in terms of complexity. As upb is not aimed directly at users, there is
|
31
|
+
much more freedom to make API-breaking changes when necessary, which helps the
|
32
|
+
core to stay small and simple. We want to be compatible with all FFI
|
33
|
+
interfaces, so C ABI compatibility is a must.
|
34
|
+
|
35
|
+
Despite these differences, C++ protos and upb offer [roughly the same core set
|
36
|
+
of features](https://github.com/protocolbuffers/upb#features).
|
37
|
+
|
38
|
+
## Arenas
|
39
|
+
|
40
|
+
upb and C++ protos both offer arena allocation, but there are some key
|
41
|
+
differences.
|
42
|
+
|
43
|
+
### C++
|
44
|
+
|
45
|
+
As a matter of history, when C++ protos were open-sourced in 2008, they did not
|
46
|
+
support arenas. Originally there was only unique ownership, whereby each
|
47
|
+
message uniquely owns all child messages and will free them when the parent is
|
48
|
+
freed.
|
49
|
+
|
50
|
+
Arena allocation was added as a feature in 2014 as a way of dramatically
|
51
|
+
reducing allocation and (especially) deallocation costs. But the library was
|
52
|
+
not at liberty to remove the unique ownership model, because it would break far
|
53
|
+
too many users. As a result, C++ has supported a **hybrid allocation model**
|
54
|
+
ever since, allowing users to allocate messages either directly from the
|
55
|
+
stack/heap or from an arena. The library attempts to ensure that there are
|
56
|
+
no dangling pointers by performing automatic copies in some cases (for example
|
57
|
+
`a->set_allocated_b(b)`, where `a` and `b` are on different arenas).
|
58
|
+
|
59
|
+
C++'s arena object itself `google::protobuf::Arena` is **thread-safe** by
|
60
|
+
design, which allows users to allocate from multiple threads simultaneously
|
61
|
+
without external synchronization. The user can supply an initial block of
|
62
|
+
memory to the arena, and can choose some parameters to control the arena block
|
63
|
+
size. The user can also supply block alloc/dealloc functions, but the alloc
|
64
|
+
function is expected to always return some memory. The C++ library in general
|
65
|
+
does not attempt to handle out of memory conditions.
|
66
|
+
|
67
|
+
### upb
|
68
|
+
|
69
|
+
upb uses **arena allocation exclusively**. All messages must be allocated from
|
70
|
+
an arena, and can only be freed by freeing the arena. It is entirely the user's
|
71
|
+
responsibility to ensure that there are no dangling pointers: when a user sets a
|
72
|
+
message field, this will always trivially overwrite the pointer and will never
|
73
|
+
perform an implicit copy.
|
74
|
+
|
75
|
+
upb's `upb::Arena` is **thread-compatible**, which means it cannot be used
|
76
|
+
concurrently without synchronization. The arena can be seeded with an initial
|
77
|
+
block of memory, but it does not explicitly support any parameters for choosing
|
78
|
+
block size. It support a custom alloc/dealloc function, and this function is
|
79
|
+
allowed to return `NULL` if no dynamic memory is available. This allows upb
|
80
|
+
arenas to have a max/fixed size, and makes it possible in theory to write code
|
81
|
+
that is tolerant to out-of-memory errors.
|
82
|
+
|
83
|
+
upb's arena also supports a novel operation known as **fuse**, which joins two
|
84
|
+
arenas together into a single lifetime. Though both arenas must still be freed
|
85
|
+
separately, none of the memory will actually be freed until *both* arenas have
|
86
|
+
been freed. This is useful for avoiding dangling pointers when reparenting a
|
87
|
+
message with one that may be on a different arena.
|
88
|
+
|
89
|
+
### Comparison
|
90
|
+
|
91
|
+
**hybrid allocation vs. arena-only**:
|
92
|
+
* The C++ hybrid allocation model introduces a great deal of complexity and
|
93
|
+
unpredictability into the library. upb benefits from having a much simpler
|
94
|
+
and more predictable design.
|
95
|
+
* Some of the complexity in C++'s hybrid model arises from the fact that arenas
|
96
|
+
were added after the fact. Designing for a hybrid model from the outset
|
97
|
+
would likely yield a simpler result.
|
98
|
+
* Unique ownership does support some usage patterns that arenas cannot directly
|
99
|
+
accommodate. For example, you can reparent a message and the child will precisely
|
100
|
+
follow the lifetime of its new parent. An arena would require you to either
|
101
|
+
perform a deep copy or extend the lifetime.
|
102
|
+
|
103
|
+
**thread-compatible vs. thread-safe arena**
|
104
|
+
* A thread-safe arena (as in C++) is safer and easier to use. A thread-compatible
|
105
|
+
arena requires that the user prove that the arena cannot be used concurrently.
|
106
|
+
* [Thread Sanitizer](https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual)
|
107
|
+
is far more accessible than it was in 2014 (when C++ introduced a thread-safe
|
108
|
+
arena). We now have more tools at our disposal to ensure that we do not trigger
|
109
|
+
data races in a thread-compatible arena like upb.
|
110
|
+
* Thread-compatible arenas are more performant.
|
111
|
+
* Thread-compatible arenas have a far simpler implementation. The C++ thread-safe
|
112
|
+
arena relies on thread-local variables, which introduce complications on some
|
113
|
+
platforms. It also requires far more subtle reasoning for correctness and
|
114
|
+
performance.
|
115
|
+
|
116
|
+
**fuse vs. no fuse**
|
117
|
+
* The `upb_Arena_Fuse()` operation is a key part of how upb supports reparenting
|
118
|
+
of messages when the parent may be on a different arena. Without this, upb has
|
119
|
+
no way of supporting `foo.bar = bar` in dynamic languages without performing a
|
120
|
+
deep copy.
|
121
|
+
* A downside of `upb_Arena_Fuse()` is that passing an arena to a function can allow
|
122
|
+
that function to extend the lifetime of the arena in potentially
|
123
|
+
unpredictable ways. This can be prevented if necessary, as fuse can fail, eg. if
|
124
|
+
one arena has an initial block. But this adds some complexity by requiring callers
|
125
|
+
to handle the case where fuse fails.
|
126
|
+
|
127
|
+
## Code Generation vs. Tables
|
128
|
+
|
129
|
+
The C++ protobuf library has always been built around code generation, while upb
|
130
|
+
generates only tables. In other words, `foo.pb.cc` files contain functions,
|
131
|
+
whereas `foo.upb.c` files emit only data structures.
|
132
|
+
|
133
|
+
### C++
|
134
|
+
|
135
|
+
C++ generated code emits a large number of functions into `foo.pb.cc` files.
|
136
|
+
An incomplete list:
|
137
|
+
|
138
|
+
* `FooMsg::FooMsg()` (constructor): initializes all fields to their default value.
|
139
|
+
* `FooMsg::~FooMsg()` (destructor): frees any present child messages.
|
140
|
+
* `FooMsg::Clear()`: clears all fields back to their default/empty value.
|
141
|
+
* `FooMsg::_InternalParse()`: generated code for parsing a message.
|
142
|
+
* `FooMsg::_InternalSerialize()`: generated code for serializing a message.
|
143
|
+
* `FooMsg::ByteSizeLong()`: calculates serialized size, as a first pass before serializing.
|
144
|
+
* `FooMsg::MergeFrom()`: copies/appends present fields from another message.
|
145
|
+
* `FooMsg::IsInitialized()`: checks whether required fields are set.
|
146
|
+
|
147
|
+
This code lives in the `.text` section and contains function calls to the generated
|
148
|
+
classes for child messages.
|
149
|
+
|
150
|
+
### upb
|
151
|
+
|
152
|
+
upb does not generate any code into `foo.upb.c` files, only data structures. upb uses a
|
153
|
+
compact data table known as a *mini table* to represent the schema and all fields.
|
154
|
+
|
155
|
+
upb uses mini tables to perform all of the operations that would traditionally be done
|
156
|
+
with generated code. Revisiting the list from the previous section:
|
157
|
+
|
158
|
+
* `FooMsg::FooMsg()` (constructor): upb instead initializes all messages with `memset(msg, 0, size)`.
|
159
|
+
Non-zero defaults are injected in the accessors.
|
160
|
+
* `FooMsg::~FooMsg()` (destructor): upb messages are freed by freeing the arena.
|
161
|
+
* `FooMsg::Clear()`: can be performed with `memset(msg, 0, size)`.
|
162
|
+
* `FooMsg::_InternalParse()`: upb's parser uses mini tables as data, instead of generating code.
|
163
|
+
* `FooMsg::_InternalSerialize()`: upb's serializer also uses mini-tables instead of generated code.
|
164
|
+
* `FooMsg::ByteSizeLong()`: upb performs serialization in reverse so that an initial pass is not required.
|
165
|
+
* `FooMsg::MergeFrom()`: upb supports this via serialize+parse from the other message.
|
166
|
+
* `FooMsg::IsInitialized()`: upb's encoder and decoder have special flags to check for required fields.
|
167
|
+
A util library `upb/util/required_fields.h` handles the corner cases.
|
168
|
+
|
169
|
+
### Comparison
|
170
|
+
|
171
|
+
If we compare compiled code size, upb is far smaller. Here is a comparison of the code
|
172
|
+
size of a trivial binary that does nothing but a parse and serialize of `descriptor.proto`.
|
173
|
+
This means we are seeing both the overhead of the core library itself as well as the
|
174
|
+
generated code (or table) for `descriptor.proto`. (For extra clarity we should break this
|
175
|
+
down by generated code vs core library in the future).
|
176
|
+
|
177
|
+
|
178
|
+
| Library | `.text` | `.data` | `.bss` |
|
179
|
+
|------------ |---------|---------|--------|
|
180
|
+
| upb | 26Ki | 0.6Ki | 0.01Ki |
|
181
|
+
| C++ (lite) | 187Ki | 2.8Ki | 1.25Ki |
|
182
|
+
| C++ (code size) | 904Ki | 6.1Ki | 1.88Ki |
|
183
|
+
| C++ (full) | 983Ki | 6.1Ki | 1.88Ki |
|
184
|
+
|
185
|
+
"C++ (code size)" refers to protos compiled with `optimize_for = CODE_SIZE`, a mode
|
186
|
+
in which generated code contains reflection only, in an attempt to make the
|
187
|
+
generated code size smaller (however it requires the full runtime instead
|
188
|
+
of the lite runtime).
|
189
|
+
|
190
|
+
## Bifurcated vs. Optional Reflection
|
191
|
+
|
192
|
+
upb and C++ protos both offer reflection without making it mandatory. However
|
193
|
+
the models for enabling/disabling reflection are very different.
|
194
|
+
|
195
|
+
### C++
|
196
|
+
|
197
|
+
C++ messages offer full reflection by default. Messages in C++ generally
|
198
|
+
derive from `Message`, and the base class provides a member function
|
199
|
+
`Reflection* Message::GetReflection()` which returns the reflection object.
|
200
|
+
|
201
|
+
It follows that any message deriving from `Message` will always have reflection
|
202
|
+
linked into the binary, whether or not the reflection object is ever used.
|
203
|
+
Because `GetReflection()` is a function on the base class, it is not possible
|
204
|
+
to statically determine if a given message's reflection is used:
|
205
|
+
|
206
|
+
```c++
|
207
|
+
Reflection* GetReflection(const Message& message) {
|
208
|
+
// Can refer to any message in the whole binary.
|
209
|
+
return message.GetReflection();
|
210
|
+
}
|
211
|
+
```
|
212
|
+
|
213
|
+
The C++ library does provide a way of omitting reflection: `MessageLite`. We can
|
214
|
+
cause a message to be lite in two different ways:
|
215
|
+
|
216
|
+
* `optimize_for = LITE_RUNTIME` in a `.proto` file will cause all messages in that
|
217
|
+
file to be lite.
|
218
|
+
* `lite` as a codegen param: this will force all messages to lite, even if the
|
219
|
+
`.proto` file does not have `optimize_for = LITE_RUNTIME`.
|
220
|
+
|
221
|
+
A lite message will derive from `MessageLite` instead of `Message`. Since
|
222
|
+
`MessageLite` has no `GetReflection()` function, this means no reflection is
|
223
|
+
available, so we can avoid taking the code size hit.
|
224
|
+
|
225
|
+
### upb
|
226
|
+
|
227
|
+
upb does not have the `Message` vs. `MessageLite` bifurcation. There is only one
|
228
|
+
kind of message type `upb_Message`, which means there is no need to configure in
|
229
|
+
a `.proto` file which messages will need reflection and which will not.
|
230
|
+
Every message has the *option* to link in reflection from a separate `foo.upbdefs.o`
|
231
|
+
file, without needing to change the message itself in any way.
|
232
|
+
|
233
|
+
upb does not provide the equivalent of `Message::GetReflection()`: there is no
|
234
|
+
facility for retrieving the reflection of a message whose type is not known statically.
|
235
|
+
It would be possible to layer such a facility on top of the upb core, though this
|
236
|
+
would probably require some kind of code generation.
|
237
|
+
|
238
|
+
### Comparison
|
239
|
+
|
240
|
+
* Most messages in C++ will not bother to declare themselves as "lite". This means
|
241
|
+
that many C++ messages will link in reflection even when it is never used, bloating
|
242
|
+
binaries unnecessarily.
|
243
|
+
* `optimize_for = LITE_RUNTIME` is difficult to use in practice, because it prevents
|
244
|
+
any non-lite protos from `import`ing that file.
|
245
|
+
* Forcing all protos to lite via a codegen parameter (for example, when building for
|
246
|
+
mobile) is more practical than `optimize_for = LITE_RUNTIME`. But this will break
|
247
|
+
the compile for any code that tries to upcast to `Message`, or tries to use a
|
248
|
+
non-lite method.
|
249
|
+
* The one major advantage of the C++ model is that it can support `msg.DebugString()`
|
250
|
+
on a type-erased proto. For upb you have to explicitly pass the `upb_MessageDef*`
|
251
|
+
separately if you want to perform an operation like printing a proto to text format.
|
252
|
+
|
253
|
+
## Explicit Registration vs. Globals
|
254
|
+
|
255
|
+
TODO
|