network-infra-utility 0.2.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +21 -0
- data/CHANGELOG.md +31 -4
- data/Gemfile +2 -0
- data/Gemfile.lock +70 -0
- data/Rakefile +1 -1
- data/bin/dns-query +834 -0
- data/bin/geo-doc +135 -0
- data/bin/geo-get +1 -1
- data/document/ASNum/346/250/241/345/235/227/345/212/237/350/203/275/350/257/264/346/230/216.md +242 -0
- data/document/DNSQuery/345/267/245/345/205/267/344/275/277/347/224/250/346/226/271/346/263/225.md +248 -0
- data/document/Geo/345/221/275/344/273/244/345/267/245/345/205/267/344/275/277/347/224/250/346/226/271/346/263/225.md +441 -0
- data/document/IP/346/250/241/345/235/227/345/212/237/350/203/275/350/257/264/346/230/216.md +297 -0
- data/document/MAC/346/250/241/345/235/227/345/212/237/350/203/275/350/257/264/346/230/216.md +296 -0
- data/document/SSH/350/277/236/346/216/245/345/256/242/346/210/267/347/253/257/344/275/277/347/224/250/346/226/271/346/263/225.md +764 -0
- data/network-infra-utility.gemspec +4 -2
- data/network.rb +3 -1
- data/service/geodb/GeoAPI.md +1 -0
- data/service/geodb/geodb.rb +278 -1
- data/service/ssh/README.md +942 -0
- data/service/ssh/bin/ssh-client +198 -0
- data/service/ssh/design/SSH/350/277/236/346/216/245/345/256/242/346/210/267/347/253/257/345/212/237/350/203/275/351/234/200/346/261/202/346/226/207/346/241/243.md +292 -0
- data/service/ssh/design/SSH/350/277/236/346/216/245/345/256/242/346/210/267/347/253/257/350/257/246/347/273/206/350/256/276/350/256/241/346/226/207/346/241/243.md +1521 -0
- data/service/ssh/design/SSH/350/277/236/346/216/245/345/256/242/346/210/267/347/253/257/350/275/257/344/273/266/350/256/276/350/256/241/346/226/207/346/241/243.md +2493 -0
- data/service/ssh/ext/ssh_core/bin/ssh_core.cmd +28 -0
- data/service/ssh/ext/ssh_core/config/sys.config +0 -0
- data/service/ssh/ext/ssh_core/config/vm.args +0 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/CHECKSUM +1 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/LICENSE +21 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/README.md +696 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/VERSION +1 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/contents.tar.gz +0 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/metadata.config +15 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/rebar.config +17 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/rebar.lock +1 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx.app.src +10 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx.erl +506 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx_config.erl +393 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx_config.hrl +18 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx_consult.erl +81 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx_decoder.erl +1909 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx_encoder.erl +116 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx_parser.erl +1214 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx_to_json.erl +408 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx_to_term.erl +389 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/jsx_verify.erl +121 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/src/jsx.erl +506 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/src/jsx_config.erl +393 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/src/jsx_config.hrl +18 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/src/jsx_consult.erl +81 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/src/jsx_decoder.erl +1909 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/src/jsx_encoder.erl +116 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/src/jsx_parser.erl +1214 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/src/jsx_to_json.erl +408 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/src/jsx_to_term.erl +389 -0
- data/service/ssh/ext/ssh_core/local_deps/jsx/src/src/jsx_verify.erl +121 -0
- data/service/ssh/ext/ssh_core/rebar.config +24 -0
- data/service/ssh/ext/ssh_core/rebar.lock +1 -0
- data/service/ssh/ext/ssh_core/src/ssh_auth_engine.erl +156 -0
- data/service/ssh/ext/ssh_core/src/ssh_channel_stm.erl +232 -0
- data/service/ssh/ext/ssh_core/src/ssh_codec.erl +83 -0
- data/service/ssh/ext/ssh_core/src/ssh_conn_sup.erl +48 -0
- data/service/ssh/ext/ssh_core/src/ssh_conn_worker.erl +535 -0
- data/service/ssh/ext/ssh_core/src/ssh_core.app.src +36 -0
- data/service/ssh/ext/ssh_core/src/ssh_core_app.erl +11 -0
- data/service/ssh/ext/ssh_core/src/ssh_core_sup.erl +46 -0
- data/service/ssh/ext/ssh_core/src/ssh_infra_sup.erl +104 -0
- data/service/ssh/ext/ssh_core/src/ssh_ipc.hrl +80 -0
- data/service/ssh/ext/ssh_core/src/ssh_ipc_coalesce.erl +94 -0
- data/service/ssh/ext/ssh_core/src/ssh_ipc_gateway.erl +467 -0
- data/service/ssh/ext/ssh_core/src/ssh_ipc_proto.erl +95 -0
- data/service/ssh/ext/ssh_core/src/ssh_jump_chain.erl +101 -0
- data/service/ssh/ext/ssh_core/src/ssh_keepalive_mgr.erl +222 -0
- data/service/ssh/ext/ssh_core/src/ssh_known_hosts_proxy.erl +67 -0
- data/service/ssh/ext/ssh_core/src/ssh_port_fwd.erl +225 -0
- data/service/ssh/ext/ssh_core/src/ssh_sftp_session.erl +250 -0
- data/service/ssh/ext/ssh_core/src/ssh_sftp_sup.erl +62 -0
- data/service/ssh/ext/ssh_core_rs/Cargo.lock +2345 -0
- data/service/ssh/ext/ssh_core_rs/Cargo.toml +30 -0
- data/service/ssh/ext/ssh_core_rs/bin/ssh_core_rs +34 -0
- data/service/ssh/ext/ssh_core_rs/bin/ssh_core_rs.cmd +40 -0
- data/service/ssh/ext/ssh_core_rs/src/channel.rs +296 -0
- data/service/ssh/ext/ssh_core_rs/src/coalesce.rs +143 -0
- data/service/ssh/ext/ssh_core_rs/src/codec.rs +71 -0
- data/service/ssh/ext/ssh_core_rs/src/conn.rs +628 -0
- data/service/ssh/ext/ssh_core_rs/src/gateway.rs +389 -0
- data/service/ssh/ext/ssh_core_rs/src/handler.rs +293 -0
- data/service/ssh/ext/ssh_core_rs/src/keepalive.rs +194 -0
- data/service/ssh/ext/ssh_core_rs/src/main.rs +351 -0
- data/service/ssh/ext/ssh_core_rs/src/portfwd.rs +378 -0
- data/service/ssh/ext/ssh_core_rs/src/proto.rs +198 -0
- data/service/ssh/ext/ssh_core_rs/src/sftp.rs +294 -0
- data/service/ssh/lib/network_infra_utility/ssh/automation/macro_engine.rb +213 -0
- data/service/ssh/lib/network_infra_utility/ssh/client.rb +257 -0
- data/service/ssh/lib/network_infra_utility/ssh/config/schema.rb +90 -0
- data/service/ssh/lib/network_infra_utility/ssh/config/settings.rb +103 -0
- data/service/ssh/lib/network_infra_utility/ssh/config/store.rb +90 -0
- data/service/ssh/lib/network_infra_utility/ssh/ipc/coalesce.rb +83 -0
- data/service/ssh/lib/network_infra_utility/ssh/ipc/errors.rb +36 -0
- data/service/ssh/lib/network_infra_utility/ssh/ipc/router.rb +212 -0
- data/service/ssh/lib/network_infra_utility/ssh/ipc/transport.rb +81 -0
- data/service/ssh/lib/network_infra_utility/ssh/security/host_key.rb +211 -0
- data/service/ssh/lib/network_infra_utility/ssh/security/vault.rb +211 -0
- data/service/ssh/lib/network_infra_utility/ssh/session/history.rb +56 -0
- data/service/ssh/lib/network_infra_utility/ssh/session/manager.rb +92 -0
- data/service/ssh/lib/network_infra_utility/ssh/session/session.rb +109 -0
- data/service/ssh/lib/network_infra_utility/ssh/session/tree.rb +95 -0
- data/service/ssh/lib/network_infra_utility/ssh/terminal/ansi_parser.rb +435 -0
- data/service/ssh/lib/network_infra_utility/ssh/terminal/buffer.rb +78 -0
- data/service/ssh/lib/network_infra_utility/ssh/terminal/emulator.rb +159 -0
- data/service/ssh/lib/network_infra_utility/ssh/terminal/logger.rb +195 -0
- data/service/ssh/lib/network_infra_utility/ssh/terminal/screen.rb +212 -0
- data/service/ssh/lib/network_infra_utility/ssh/terminal/theme.rb +127 -0
- data/service/ssh/lib/network_infra_utility/ssh/version.rb +7 -0
- data/service/ssh/lib/network_infra_utility/ssh.rb +44 -0
- data/support/basic/as_num.rb +221 -0
- data/support/basic/mac_address.rb +281 -0
- data/version.rb +1 -1
- metadata +138 -1
|
@@ -0,0 +1,696 @@
|
|
|
1
|
+
# jsx (v3.0.0) #
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
an erlang application for consuming, producing and manipulating [json][json].
|
|
5
|
+
inspired by [yajl][yajl]
|
|
6
|
+
|
|
7
|
+
**jsx** is built via [rebar3][rebar3]
|
|
8
|
+
|
|
9
|
+
current status: 
|
|
10
|
+
|
|
11
|
+
**jsx** is released under the terms of the [MIT][MIT] license
|
|
12
|
+
|
|
13
|
+
copyright 2010-2016 alisdair sullivan
|
|
14
|
+
|
|
15
|
+
## index ##
|
|
16
|
+
|
|
17
|
+
* [quickstart](#quickstart)
|
|
18
|
+
* [description](#description)
|
|
19
|
+
- [migrating from 1.x](#migrating)
|
|
20
|
+
- [json <-> erlang mapping](#json---erlang-mapping)
|
|
21
|
+
- [incomplete input](#incomplete-input)
|
|
22
|
+
* [data types](#data-types)
|
|
23
|
+
- [`json_term()`](#json_term)
|
|
24
|
+
- [`json_text()`](#json_text)
|
|
25
|
+
- [`event()`](#event)
|
|
26
|
+
- [`option()`](#option)
|
|
27
|
+
* [exports](#exports)
|
|
28
|
+
- [`encoder/3`, `decoder/3` & `parser/3`](#encoder3-decoder3--parser3)
|
|
29
|
+
- [`decode/1,2`](#decode12)
|
|
30
|
+
- [`encode/1,2`](#encode12)
|
|
31
|
+
- [`format/1,2`](#format12)
|
|
32
|
+
- [`minify/1`](#minify1)
|
|
33
|
+
- [`prettify/1`](#prettify1)
|
|
34
|
+
- [`is_json/1,2`](#is_json12)
|
|
35
|
+
- [`is_term/1,2`](#is_term12)
|
|
36
|
+
* [callback exports](#callback_exports)
|
|
37
|
+
- [`Module:init/1`](#moduleinit1)
|
|
38
|
+
- [`Module:handle_event/2`](#modulehandle_event2)
|
|
39
|
+
* [acknowledgements](#acknowledgements)
|
|
40
|
+
|
|
41
|
+
|
|
42
|
+
## quickstart ##
|
|
43
|
+
|
|
44
|
+
#### to add to a rebar3 project ####
|
|
45
|
+
Add to `rebar.config`
|
|
46
|
+
```erlang
|
|
47
|
+
...
|
|
48
|
+
{erl_opts, [debug_info]}.
|
|
49
|
+
{deps, [
|
|
50
|
+
...
|
|
51
|
+
{jsx, "~> 3.0"}
|
|
52
|
+
]}.
|
|
53
|
+
...
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
#### to build the library and run tests ####
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
$ rebar3 compile
|
|
60
|
+
$ rebar3 eunit
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
#### to convert a utf8 binary containing a json string into an erlang term ####
|
|
64
|
+
|
|
65
|
+
```erlang
|
|
66
|
+
1> jsx:decode(<<"{\"library\": \"jsx\", \"awesome\": true}">>, []).
|
|
67
|
+
#{<<"awesome">> => true,<<"library">> => <<"jsx">>}
|
|
68
|
+
2> jsx:decode(<<"{\"library\": \"jsx\", \"awesome\": true}">>, [{return_maps, false}]).
|
|
69
|
+
[{<<"library">>,<<"jsx">>},{<<"awesome">>,true}]
|
|
70
|
+
3> jsx:decode(<<"[\"a\",\"list\",\"of\",\"words\"]">>).
|
|
71
|
+
[<<"a">>, <<"list">>, <<"of">>, <<"words">>]
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
#### to convert an erlang term into a utf8 binary containing a json string ####
|
|
75
|
+
|
|
76
|
+
```erlang
|
|
77
|
+
1> jsx:encode(#{<<"library">> => <<"jsx">>, <<"awesome">> => true}).
|
|
78
|
+
<<"{\"awesome\":true,\"library\":\"jsx\"}">>
|
|
79
|
+
2> jsx:encode([{<<"library">>,<<"jsx">>},{<<"awesome">>,true}]).
|
|
80
|
+
<<"{\"library\": \"jsx\", \"awesome\": true}">>
|
|
81
|
+
3> jsx:encode([<<"a">>, <<"list">>, <<"of">>, <<"words">>]).
|
|
82
|
+
<<"[\"a\",\"list\",\"of\",\"words\"]">>
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
#### to check if a binary or a term is valid json ####
|
|
86
|
+
|
|
87
|
+
```erlang
|
|
88
|
+
1> jsx:is_json(<<"[\"this is json\"]">>).
|
|
89
|
+
true
|
|
90
|
+
2> jsx:is_json("[\"this is not\"]").
|
|
91
|
+
false
|
|
92
|
+
3> jsx:is_term([<<"this is a term">>]).
|
|
93
|
+
true
|
|
94
|
+
4> jsx:is_term([this, is, not]).
|
|
95
|
+
false
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
#### to minify some json ####
|
|
99
|
+
|
|
100
|
+
```erlang
|
|
101
|
+
1> jsx:minify(<<"{
|
|
102
|
+
\"a list\": [
|
|
103
|
+
1,
|
|
104
|
+
2,
|
|
105
|
+
3
|
|
106
|
+
]
|
|
107
|
+
}">>).
|
|
108
|
+
<<"{\"a list\":[1,2,3]}">>
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
#### to prettify some json ####
|
|
112
|
+
|
|
113
|
+
```erlang
|
|
114
|
+
1> jsx:prettify(<<"{\"a list\":[1,2,3]}">>).
|
|
115
|
+
<<"{
|
|
116
|
+
\"a list\": [
|
|
117
|
+
1,
|
|
118
|
+
2,
|
|
119
|
+
3
|
|
120
|
+
]
|
|
121
|
+
}">>
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
## description ##
|
|
125
|
+
|
|
126
|
+
|
|
127
|
+
**jsx** is an erlang application for consuming, producing and manipulating
|
|
128
|
+
[json][json]
|
|
129
|
+
|
|
130
|
+
**jsx** follows the json [spec][rfc4627] as closely as possible with allowances for
|
|
131
|
+
real world usage
|
|
132
|
+
|
|
133
|
+
**jsx** is pragmatic. the json spec allows extensions so **jsx** extends the spec in a
|
|
134
|
+
number of ways. see the section on `strict` in [options](#option) below though
|
|
135
|
+
|
|
136
|
+
json has no official comments but this parser allows c/c++ style comments.
|
|
137
|
+
anywhere whitespace is allowed you can insert comments (both `// ...` and `/* ... */`)
|
|
138
|
+
|
|
139
|
+
some particularly irresponsible json emitters leave trailing commas at the end of
|
|
140
|
+
objects or arrays. **jsx** allows a single trailing comma in input. multiple commas
|
|
141
|
+
in any posistion or a preceding comma are still errors
|
|
142
|
+
|
|
143
|
+
all **jsx** decoder input should be `utf8` encoded binaries. sometimes you get binaries
|
|
144
|
+
that are almost but not quite valid utf8 whether due to improper escaping or poor
|
|
145
|
+
encoding. **jsx** replaces invalid codepoints and poorly formed sequences with the
|
|
146
|
+
unicode replacement character (`u+FFFD`) but does it's best to return something
|
|
147
|
+
comprehensible
|
|
148
|
+
|
|
149
|
+
json only allows keys and strings to be delimited by double quotes (`u+0022`) but
|
|
150
|
+
javascript allows them to be delimited by single quotes (`u+0027`) as well. **jsx**
|
|
151
|
+
follows javascript in this. strings that start with single quotes can contain double
|
|
152
|
+
quotes but must end with single quotes and must escape any single quotes they contain
|
|
153
|
+
|
|
154
|
+
json and **jsx** only recognize escape sequences as outlined in the json spec. it just
|
|
155
|
+
ignores bad escape sequences leaving them in strings unaltered
|
|
156
|
+
|
|
157
|
+
### json <-> erlang mapping ###
|
|
158
|
+
|
|
159
|
+
**json** | **erlang**
|
|
160
|
+
--------------------------------|--------------------------------
|
|
161
|
+
`number` | `integer()` and `float()`
|
|
162
|
+
`string` | `binary()` and `atom()`
|
|
163
|
+
`true`, `false` and `null` | `true`, `false` and `null`
|
|
164
|
+
`array` | `[]` and `[JSON]`
|
|
165
|
+
`object` | `#{}`, `[{}]` and `[{binary() OR atom() OR integer(), JSON}]`
|
|
166
|
+
see below | `datetime()`
|
|
167
|
+
|
|
168
|
+
* numbers
|
|
169
|
+
|
|
170
|
+
javascript and thus json represent all numeric values with floats. there's no
|
|
171
|
+
reason for erlang -- a language that supports arbitrarily large integers -- to
|
|
172
|
+
restrict all numbers to the ieee754 range
|
|
173
|
+
|
|
174
|
+
whenever possible, **jsx** will interpret json numbers that look like integers as
|
|
175
|
+
integers. other numbers will be converted to erlang's floating point type, which
|
|
176
|
+
is nearly but not quite iee754. negative zero is not representable in erlang (zero
|
|
177
|
+
is unsigned in erlang and `0` is equivalent to `-0`) and will be interpreted as
|
|
178
|
+
regular zero. numbers not representable are beyond the concern of this implementation,
|
|
179
|
+
and will result in parsing errors
|
|
180
|
+
|
|
181
|
+
when converting from erlang to json, floats are represented with their
|
|
182
|
+
shortest representation that will round trip without loss of precision. this
|
|
183
|
+
means that some floats may be superficially dissimilar (although
|
|
184
|
+
functionally equivalent). for example, `1.0000000000000001` will be
|
|
185
|
+
represented by `1.0`
|
|
186
|
+
|
|
187
|
+
* strings
|
|
188
|
+
|
|
189
|
+
json strings must be unicode encoded binaries or erlang atoms. in practice,
|
|
190
|
+
because **jsx** only accepts `utf8` binaries all binary strings must be `utf8`.
|
|
191
|
+
in addition to being unicode json strings restrict a number of codepoints and
|
|
192
|
+
define a number of escape sequences
|
|
193
|
+
|
|
194
|
+
json string escapes of the form `\uXXXX` will be converted to their
|
|
195
|
+
equivalent codepoints during parsing. this means control characters and
|
|
196
|
+
other codepoints disallowed by the json spec may be encountered in resulting
|
|
197
|
+
strings. the utf8 restriction means the surrogates are explicitly disallowed.
|
|
198
|
+
if a string contains escaped surrogates (`u+d800` to `u+dfff`) they are
|
|
199
|
+
interpreted but only when they form valid surrogate pairs. surrogates
|
|
200
|
+
encountered otherwise are replaced with the replacement codepoint (`u+fffd`)
|
|
201
|
+
|
|
202
|
+
all erlang strings are represented by **valid** `utf8` encoded binaries. the
|
|
203
|
+
encoder will check strings for conformance. badly formed `utf8` sequences may
|
|
204
|
+
be replaced with the replacement codepoint (`u+fffd`) according to the unicode
|
|
205
|
+
spec
|
|
206
|
+
|
|
207
|
+
this implementation performs no normalization on strings beyond that
|
|
208
|
+
detailed here. be careful when comparing strings as equivalent strings
|
|
209
|
+
may have different `utf8` encodings
|
|
210
|
+
|
|
211
|
+
* true, false and null
|
|
212
|
+
|
|
213
|
+
the json primitives `true`, `false` and `null` are represented by the
|
|
214
|
+
erlang atoms `true`, `false` and `null`. surprise
|
|
215
|
+
|
|
216
|
+
* arrays
|
|
217
|
+
|
|
218
|
+
json arrays are represented with erlang lists of json values as described
|
|
219
|
+
in this section
|
|
220
|
+
|
|
221
|
+
* objects
|
|
222
|
+
|
|
223
|
+
json objects are represented by erlang maps.
|
|
224
|
+
|
|
225
|
+
* datetime
|
|
226
|
+
|
|
227
|
+
erlang datetime tuples (`{{Year, Month, Day}, {Hour, Min, Sec}}`) as returned
|
|
228
|
+
from `erlang:localtime/0` are automatically encoded as [iso8601][iso8601]
|
|
229
|
+
strings and are assumed to be UTC time. no conversion is attempted of json [iso8601][iso8601] strings in decoded json
|
|
230
|
+
|
|
231
|
+
|
|
232
|
+
### incomplete input ###
|
|
233
|
+
|
|
234
|
+
**jsx** can handle incomplete json texts. if the option `stream` is passed to the decoder
|
|
235
|
+
or parser and if a partial json text is parsed, rather than returning a term from
|
|
236
|
+
your callback handler, **jsx** returns `{incomplete, F}` where `F` is a function with
|
|
237
|
+
an identical API to the anonymous fun returned from `decoder/3`, `encoder/3` or
|
|
238
|
+
`parser/3`. it retains the internal state of the parser at the point where input
|
|
239
|
+
was exhausted. this allows you to parse as you stream json over a socket or file
|
|
240
|
+
descriptor, or to parse large json texts without needing to keep them entirely in
|
|
241
|
+
memory
|
|
242
|
+
|
|
243
|
+
however, it is important to recognize that **jsx** is conservative by default. **jsx** will
|
|
244
|
+
not consider the parsing complete even when input is exhausted and the json text is
|
|
245
|
+
unambiguously incomplete. to end parsing call the `incomplete` function with the
|
|
246
|
+
argument `end_stream` (or `end_json`) like:
|
|
247
|
+
|
|
248
|
+
```erlang
|
|
249
|
+
1> {incomplete, F} = jsx:decode(<<"[">>, [stream]).
|
|
250
|
+
{incomplete,#Fun<jsx_decoder.1.122947756>}
|
|
251
|
+
2> F(end_stream). % can also be `F(end_json)`
|
|
252
|
+
** exception error: bad argument
|
|
253
|
+
3> {incomplete, G} = F(<<"]">>).
|
|
254
|
+
{incomplete,#Fun<jsx_decoder.1.122947756>}
|
|
255
|
+
4> G(end_stream). % can also be `G(end_json)`
|
|
256
|
+
[]
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
|
|
260
|
+
## data types ##
|
|
261
|
+
|
|
262
|
+
#### `json_term()` ####
|
|
263
|
+
|
|
264
|
+
```erlang
|
|
265
|
+
json_term() = [json_term()]
|
|
266
|
+
| [{binary() | atom() | integer(), json_term()}]
|
|
267
|
+
| #{} % map of any size, not just the empty map
|
|
268
|
+
| true
|
|
269
|
+
| false
|
|
270
|
+
| null
|
|
271
|
+
| integer()
|
|
272
|
+
| float()
|
|
273
|
+
| binary()
|
|
274
|
+
| atom()
|
|
275
|
+
| datetime()
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
the erlang representation of json. binaries should be `utf8` encoded, or close
|
|
279
|
+
at least
|
|
280
|
+
|
|
281
|
+
#### `json_text()` ####
|
|
282
|
+
|
|
283
|
+
```erlang
|
|
284
|
+
json_text() = binary()
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
a utf8 encoded binary containing a json string
|
|
288
|
+
|
|
289
|
+
#### `event()` ####
|
|
290
|
+
|
|
291
|
+
```erlang
|
|
292
|
+
event() = start_object
|
|
293
|
+
| end_object
|
|
294
|
+
| start_array
|
|
295
|
+
| end_array
|
|
296
|
+
| {key, binary()}
|
|
297
|
+
| {string, binary()}
|
|
298
|
+
| {integer, integer()}
|
|
299
|
+
| {float, float()}
|
|
300
|
+
| {literal, true}
|
|
301
|
+
| {literal, false}
|
|
302
|
+
| {literal, null}
|
|
303
|
+
| end_json
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
the subset of [`token()`](#token) emitted by the decoder and encoder to handlers
|
|
307
|
+
|
|
308
|
+
#### `option()` ####
|
|
309
|
+
|
|
310
|
+
```erlang
|
|
311
|
+
option() = dirty_strings
|
|
312
|
+
| escaped_forward_slashes
|
|
313
|
+
| escaped_strings
|
|
314
|
+
| repeat_keys
|
|
315
|
+
| stream
|
|
316
|
+
| strict
|
|
317
|
+
| {strict, [strict_option()]}
|
|
318
|
+
| return_tail
|
|
319
|
+
| uescape
|
|
320
|
+
| unescaped_jsonp
|
|
321
|
+
|
|
322
|
+
strict_option() = comments
|
|
323
|
+
| trailing_commas
|
|
324
|
+
| utf8
|
|
325
|
+
| single_quotes
|
|
326
|
+
| escapes
|
|
327
|
+
```
|
|
328
|
+
|
|
329
|
+
**jsx** functions all take a common set of options. not all flags have meaning
|
|
330
|
+
in all contexts, but they are always valid options. functions may have
|
|
331
|
+
additional options beyond these. see
|
|
332
|
+
[individual function documentation](#exports) for details
|
|
333
|
+
|
|
334
|
+
- `dirty_strings`
|
|
335
|
+
|
|
336
|
+
json escaping is lossy; it mutates the json string and repeated application
|
|
337
|
+
can result in unwanted behaviour. if your strings are already escaped (or
|
|
338
|
+
you'd like to force invalid strings into "json" you monster) use this flag
|
|
339
|
+
to bypass escaping. this can also be used to read in **really** invalid json
|
|
340
|
+
strings. everything between unescaped quotes are passed as is to the resulting
|
|
341
|
+
string term. note that this takes precedence over any other options
|
|
342
|
+
|
|
343
|
+
- `escaped_forward_slashes`
|
|
344
|
+
|
|
345
|
+
json strings are escaped according to the json spec. this means forward
|
|
346
|
+
slashes (solidus) are only escaped when this flag is present. otherwise they
|
|
347
|
+
are left unescaped. you may want to use this if you are embedding json
|
|
348
|
+
directly into a html or xml document
|
|
349
|
+
|
|
350
|
+
- `escaped_strings`
|
|
351
|
+
|
|
352
|
+
by default both the encoder and decoder return strings as utf8 binaries
|
|
353
|
+
appropriate for use in erlang. escape sequences that were present in decoded
|
|
354
|
+
terms are converted into the appropriate codepoint while encoded terms are
|
|
355
|
+
unaltered. this flag escapes strings as if for output in json, removing
|
|
356
|
+
control codes and problematic codepoints and replacing them with the
|
|
357
|
+
appropriate escapes
|
|
358
|
+
|
|
359
|
+
- `stream`
|
|
360
|
+
|
|
361
|
+
see [incomplete input](#incomplete-input)
|
|
362
|
+
|
|
363
|
+
- `strict`
|
|
364
|
+
|
|
365
|
+
as mentioned [earlier](#description), **jsx** is pragmatic. if you're more of a
|
|
366
|
+
json purist or you're really into bdsm stricter adherence to the spec is
|
|
367
|
+
possible. the following restrictions are available
|
|
368
|
+
|
|
369
|
+
* `comments`
|
|
370
|
+
|
|
371
|
+
comments are disabled and result in a `badarg` error
|
|
372
|
+
|
|
373
|
+
* `trailing_commas`
|
|
374
|
+
|
|
375
|
+
trailing commas in an object or list result in `badarg` errors
|
|
376
|
+
|
|
377
|
+
* `utf8`
|
|
378
|
+
|
|
379
|
+
invalid codepoints and malformed unicode result in `badarg` errors
|
|
380
|
+
|
|
381
|
+
* `single_quotes`
|
|
382
|
+
|
|
383
|
+
only keys and strings delimited by double quotes (`u+0022`) are allowed. the
|
|
384
|
+
single quote (`u+0027`) results in a `badarg` error
|
|
385
|
+
|
|
386
|
+
* `escapes`
|
|
387
|
+
|
|
388
|
+
escape sequences not adhering to the json spec result in a `badarg` error
|
|
389
|
+
|
|
390
|
+
* `control_codes`
|
|
391
|
+
|
|
392
|
+
control codes in strings result in `badarg` errors
|
|
393
|
+
|
|
394
|
+
any combination of these can be passed to **jsx** by using `{strict, [strict_option()]}`.
|
|
395
|
+
`strict` is equivalent to `{strict, [comments, trailing_commas, utf8, single_quotes, escapes, control_codes]}`
|
|
396
|
+
|
|
397
|
+
- `return_tail`
|
|
398
|
+
|
|
399
|
+
upon reaching the end of a valid json term in an input stream return the term and any
|
|
400
|
+
remaining bytes in the input stream as `{with_tail, term(), binary()}` where the second
|
|
401
|
+
member of the tuple is the json term and the third is any remaining bytes. note that
|
|
402
|
+
leading whitespace will be stripped from the tail
|
|
403
|
+
|
|
404
|
+
- `uescape`
|
|
405
|
+
|
|
406
|
+
escape all codepoints outside the ascii range for 7 bit clean output. note
|
|
407
|
+
this escaping takes place even if no other string escaping is requested (via
|
|
408
|
+
`escaped_strings`)
|
|
409
|
+
|
|
410
|
+
- `unescaped_jsonp`
|
|
411
|
+
|
|
412
|
+
javascript interpreters treat the codepoints `u+2028` and `u+2029` as
|
|
413
|
+
significant whitespace. json strings that contain either of these codepoints
|
|
414
|
+
will be parsed incorrectly by some javascript interpreters. by default,
|
|
415
|
+
these codepoints are escaped (to `\u2028` and `\u2029`, respectively) to
|
|
416
|
+
retain compatibility. this option simply removes that escaping
|
|
417
|
+
|
|
418
|
+
|
|
419
|
+
## exports ##
|
|
420
|
+
|
|
421
|
+
|
|
422
|
+
#### `encoder/3`, `decoder/3` & `parser/3` ####
|
|
423
|
+
|
|
424
|
+
```erlang
|
|
425
|
+
decoder(Module, Args, Opts) -> Fun((JSONText) -> any())
|
|
426
|
+
encoder(Module, Args, Opts) -> Fun((JSONTerm) -> any())
|
|
427
|
+
parser(Module, Args, Opts) -> Fun((Tokens) -> any())
|
|
428
|
+
|
|
429
|
+
Module = atom()
|
|
430
|
+
Args = any()
|
|
431
|
+
Opts = [option()]
|
|
432
|
+
JSONText = json_text()
|
|
433
|
+
JSONTerm = json_term()
|
|
434
|
+
Tokens = event() | [event()]
|
|
435
|
+
```
|
|
436
|
+
|
|
437
|
+
**jsx** is a json compiler with interleaved tokenizing, syntactic analysis and
|
|
438
|
+
semantic analysis stages. included are two tokenizers; one that handles json
|
|
439
|
+
texts (`decoder/3`) and one that handles erlang terms (`encoder/3`). there is
|
|
440
|
+
also an entry point to the syntactic analysis stage for use with user-defined
|
|
441
|
+
tokenizers (`parser/3`)
|
|
442
|
+
|
|
443
|
+
all three functions return an anonymous function that takes the appropriate type
|
|
444
|
+
of input and returns the result of performing semantic analysis, the tuple
|
|
445
|
+
`{incomplete, F}` where `F` is a new anonymous function (see
|
|
446
|
+
[incomplete input](#incomplete_input)) or a `badarg` error exception if
|
|
447
|
+
syntactic analysis fails
|
|
448
|
+
|
|
449
|
+
`Module` is the name of the callback module
|
|
450
|
+
|
|
451
|
+
`Args` is any term that will be passed to `Module:init/1` prior to syntactic
|
|
452
|
+
analysis to produce an initial state
|
|
453
|
+
|
|
454
|
+
`Opts` are detailed [here](#option)
|
|
455
|
+
|
|
456
|
+
check out [callback module documentation](#callback_exports) for details of
|
|
457
|
+
the callback module interface
|
|
458
|
+
|
|
459
|
+
#### `decode/1,2` ####
|
|
460
|
+
|
|
461
|
+
```erlang
|
|
462
|
+
decode(JSON) -> Term
|
|
463
|
+
decode(JSON, Opts) -> Term
|
|
464
|
+
|
|
465
|
+
JSON = json_text()
|
|
466
|
+
Term = json_term()
|
|
467
|
+
Opts = [option() | labels | {labels, Label} | return_maps]
|
|
468
|
+
Label = binary | atom | existing_atom | attempt_atom
|
|
469
|
+
F = fun((any()) -> any())
|
|
470
|
+
```
|
|
471
|
+
|
|
472
|
+
`decode` parses a json text (a `utf8` encoded binary) and produces an erlang
|
|
473
|
+
term
|
|
474
|
+
|
|
475
|
+
the option `labels` controls how keys are converted from json to
|
|
476
|
+
erlang terms. `binary` (the default behavior) does no conversion
|
|
477
|
+
beyond normal escaping. `atom` converts keys to erlang atoms and
|
|
478
|
+
results in a `badarg` error if the keys fall outside the range of erlang
|
|
479
|
+
atoms. `existing_atom` is identical to `atom` except it will not add
|
|
480
|
+
new atoms to the atom table and will result in a `badarg` error if the atom
|
|
481
|
+
does not exist. `attempt_atom` will convert keys to atoms when they exist,
|
|
482
|
+
and leave them as binary otherwise
|
|
483
|
+
|
|
484
|
+
the option `{return_maps, false}` will return objects as proplists instead
|
|
485
|
+
of maps.
|
|
486
|
+
|
|
487
|
+
raises a `badarg` error exception if input is not valid json
|
|
488
|
+
|
|
489
|
+
|
|
490
|
+
#### `encode/1,2` ####
|
|
491
|
+
|
|
492
|
+
```erlang
|
|
493
|
+
encode(Term) -> JSON
|
|
494
|
+
encode(Term, Opts) -> JSON
|
|
495
|
+
|
|
496
|
+
Term = json_term()
|
|
497
|
+
JSON = json_text()
|
|
498
|
+
Opts = [option() | space | {space, N} | indent | {indent, N}]
|
|
499
|
+
N = pos_integer()
|
|
500
|
+
```
|
|
501
|
+
|
|
502
|
+
`encode` converts an erlang term into json text (a `utf8` encoded binary)
|
|
503
|
+
|
|
504
|
+
the option `{space, N}` inserts `N` spaces after every comma and colon in your
|
|
505
|
+
json output. `space` is an alias for `{space, 1}`. the default is `{space, 0}`
|
|
506
|
+
|
|
507
|
+
the option `{indent, N}` inserts a newline and `N` spaces for each level of
|
|
508
|
+
indentation in your json output. note that this overrides spaces inserted after
|
|
509
|
+
a comma. `indent` is an alias for `{indent, 1}`. the default is `{indent, 0}`
|
|
510
|
+
|
|
511
|
+
raises a `badarg` error exception if input is not a valid
|
|
512
|
+
[erlang representation of json](#json---erlang-mapping)
|
|
513
|
+
|
|
514
|
+
|
|
515
|
+
#### `format/1,2` ####
|
|
516
|
+
|
|
517
|
+
```erlang
|
|
518
|
+
format(JSON) -> JSON
|
|
519
|
+
format(JSON, Opts) -> JSON
|
|
520
|
+
|
|
521
|
+
JSON = json_text()
|
|
522
|
+
Opts = [option() | space | {space, N} | indent | {indent, N} | {newline, LF}]
|
|
523
|
+
N = pos_integer()
|
|
524
|
+
LF = binary()
|
|
525
|
+
```
|
|
526
|
+
|
|
527
|
+
`format` parses a json text (a `utf8` encoded binary) and produces a new json
|
|
528
|
+
text according to the format rules specified by `Opts`
|
|
529
|
+
|
|
530
|
+
the option `{space, N}` inserts `N` spaces after every comma and colon in your
|
|
531
|
+
json output. `space` is an alias for `{space, 1}`. the default is `{space, 0}`
|
|
532
|
+
|
|
533
|
+
the option `{indent, N}` inserts a newline and `N` spaces for each level of
|
|
534
|
+
indentation in your json output. note that this overrides spaces inserted after
|
|
535
|
+
a comma. `indent` is an alias for `{indent, 1}`. the default is `{indent, 0}`
|
|
536
|
+
|
|
537
|
+
the option `{newline, LF}` defines a custom newline symbol(s).
|
|
538
|
+
the default is `{newline, <<$\n>>}`
|
|
539
|
+
|
|
540
|
+
raises a `badarg` error exception if input is not valid json
|
|
541
|
+
|
|
542
|
+
|
|
543
|
+
#### `minify/1` ####
|
|
544
|
+
|
|
545
|
+
```erlang
|
|
546
|
+
minify(JSON) -> JSON
|
|
547
|
+
|
|
548
|
+
JSON = json_text()
|
|
549
|
+
```
|
|
550
|
+
|
|
551
|
+
`minify` parses a json text (a `utf8` encoded binary) and produces a new json
|
|
552
|
+
text stripped of whitespace
|
|
553
|
+
|
|
554
|
+
raises a `badarg` error exception if input is not valid json
|
|
555
|
+
|
|
556
|
+
|
|
557
|
+
#### `prettify/1` ####
|
|
558
|
+
|
|
559
|
+
```erlang
|
|
560
|
+
prettify(JSON) -> JSON
|
|
561
|
+
|
|
562
|
+
JSON = json_text()
|
|
563
|
+
```
|
|
564
|
+
|
|
565
|
+
`prettify` parses a json text (a `utf8` encoded binary) and produces a new json
|
|
566
|
+
text equivalent to `format(JSON, [{space, 1}, {indent, 2}])`
|
|
567
|
+
|
|
568
|
+
raises a `badarg` error exception if input is not valid json
|
|
569
|
+
|
|
570
|
+
|
|
571
|
+
#### `is_json/1,2` ####
|
|
572
|
+
|
|
573
|
+
```erlang
|
|
574
|
+
is_json(MaybeJSON) -> true | false
|
|
575
|
+
is_json(MaybeJSON, Opts) -> true | false
|
|
576
|
+
|
|
577
|
+
MaybeJSON = any()
|
|
578
|
+
Opts = options()
|
|
579
|
+
```
|
|
580
|
+
|
|
581
|
+
returns true if input is a valid json text, false if not
|
|
582
|
+
|
|
583
|
+
what exactly constitutes valid json may be [altered](#option)
|
|
584
|
+
|
|
585
|
+
|
|
586
|
+
#### `is_term/1,2` ####
|
|
587
|
+
|
|
588
|
+
```erlang
|
|
589
|
+
is_term(MaybeJSON) -> true | false
|
|
590
|
+
is_term(MaybeJSON, Opts) -> true | false
|
|
591
|
+
|
|
592
|
+
MaybeJSON = any()
|
|
593
|
+
Opts = options()
|
|
594
|
+
```
|
|
595
|
+
|
|
596
|
+
returns true if input is a valid erlang representation of json, false if not
|
|
597
|
+
|
|
598
|
+
what exactly constitutes valid json may be altered via [options](#option)
|
|
599
|
+
|
|
600
|
+
## callback exports ##
|
|
601
|
+
|
|
602
|
+
the following functions should be exported from a **jsx** callback module
|
|
603
|
+
|
|
604
|
+
#### `Module:init/1` ####
|
|
605
|
+
|
|
606
|
+
```erlang
|
|
607
|
+
Module:init(Args) -> InitialState
|
|
608
|
+
|
|
609
|
+
Args = any()
|
|
610
|
+
InitialState = any()
|
|
611
|
+
```
|
|
612
|
+
|
|
613
|
+
whenever any of `encoder/3`, `decoder/3` or `parser/3` are called, this function
|
|
614
|
+
is called with the `Args` argument provided in the calling function to obtain
|
|
615
|
+
`InitialState`
|
|
616
|
+
|
|
617
|
+
#### `Module:handle_event/2` ####
|
|
618
|
+
|
|
619
|
+
```erlang
|
|
620
|
+
Module:handle_event(Event, State) -> NewState
|
|
621
|
+
|
|
622
|
+
Event = [event()]
|
|
623
|
+
State = any()
|
|
624
|
+
NewState = any()
|
|
625
|
+
```
|
|
626
|
+
|
|
627
|
+
semantic analysis is performed by repeatedly calling `handle_event/2` with a
|
|
628
|
+
stream of events emitted by the tokenizer and the current state. the new state
|
|
629
|
+
returned is used as the input to the next call to `handle_event/2`. the
|
|
630
|
+
following events must be handled:
|
|
631
|
+
|
|
632
|
+
- `start_object`
|
|
633
|
+
|
|
634
|
+
the start of a json object
|
|
635
|
+
|
|
636
|
+
- '{key, binary()}'
|
|
637
|
+
|
|
638
|
+
the key of an entry in a json object
|
|
639
|
+
|
|
640
|
+
- `end_object`
|
|
641
|
+
|
|
642
|
+
the end of a json object
|
|
643
|
+
|
|
644
|
+
- `start_array`
|
|
645
|
+
|
|
646
|
+
the start of a json array
|
|
647
|
+
|
|
648
|
+
- `end_array`
|
|
649
|
+
|
|
650
|
+
the end of a json array
|
|
651
|
+
|
|
652
|
+
- `{string, binary()}`
|
|
653
|
+
|
|
654
|
+
a json string. it will usually be a `utf8` encoded binary. see the
|
|
655
|
+
[options](#option) for possible exceptions. note that keys are also
|
|
656
|
+
json strings
|
|
657
|
+
|
|
658
|
+
- `{integer, integer()}`
|
|
659
|
+
|
|
660
|
+
an erlang integer (bignum)
|
|
661
|
+
|
|
662
|
+
- `{float, float()}`
|
|
663
|
+
|
|
664
|
+
an erlang float
|
|
665
|
+
|
|
666
|
+
- `{literal, true}`
|
|
667
|
+
|
|
668
|
+
the atom `true`
|
|
669
|
+
|
|
670
|
+
- `{literal, false}`
|
|
671
|
+
|
|
672
|
+
the atom `false`
|
|
673
|
+
|
|
674
|
+
- `{literal, null}`
|
|
675
|
+
|
|
676
|
+
the atom `null`
|
|
677
|
+
|
|
678
|
+
- `end_json`
|
|
679
|
+
|
|
680
|
+
this event is emitted when syntactic analysis is completed. you should
|
|
681
|
+
do any cleanup and return the result of your semantic analysis
|
|
682
|
+
|
|
683
|
+
|
|
684
|
+
## acknowledgements ##
|
|
685
|
+
|
|
686
|
+
jsx wouldn't be what it is without the contributions of [Paul J. Davis](https://github.com/davisp), [Lloyd Hilaiel](https://github.com/lloyd), [John Engelhart](https://github.com/johnezang), [Bob Ippolito](https://github.com/etrepum), [Brujo Benavides](https://github.com/elbrujohalcon), [Alex Kropivny](https://github.com/amtal), [Steve Strong](https://github.com/srstrong), [Michael Truog](https://github.com/okeuday), [Devin Torres](https://github.com/devinus), [fogfish](https://github.com/fogfish), [emptytea](https://github.com/emptytea), [John Daily](https://github.com/macintux), [Ola Bäckström](https://github.com/olabackstrom), [Joseph Crowe](https://github.com/JosephCrowe), [Patrick Gombert](https://github.com/patrickgombert), [Eshengazin S. Kuat](https://github.com/eskuat), [Max Lapshin](https://github.com/maxlapshin), [Bikram Chatterjee](https://github.com/c-bik), [Michael Uvarov](https://github.com/arcusfelis), [Led](https://github.com/Ledest) and [tvv](https://github.com/tvv)
|
|
687
|
+
|
|
688
|
+
[json]: http://json.org
|
|
689
|
+
[yajl]: http://lloyd.github.com/yajl
|
|
690
|
+
[MIT]: http://www.opensource.org/licenses/mit-license.html
|
|
691
|
+
[rebar3]: https://rebar3.org
|
|
692
|
+
[meck]: https://github.com/eproxus/meck
|
|
693
|
+
[rfc4627]: http://tools.ietf.org/html/rfc4627
|
|
694
|
+
[travis]: https://travis-ci.org/
|
|
695
|
+
[jsxn]: https://github.com/talentdeficit/jsxn
|
|
696
|
+
[iso8601]: http://www.iso.org/iso/iso8601
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
3
|
|
Binary file
|