ts-proto 1.87.0 → 1.90.0

package/CHANGELOG.md

@@ -1,3 +1,31 @@
+# [1.90.0](https://github.com/stephenh/ts-proto/compare/v1.89.0...v1.90.0) (2021-11-24)
+
+
+### Features
+
+*  Add support for 'json_name' annotation ([#408](https://github.com/stephenh/ts-proto/issues/408)) ([b519717](https://github.com/stephenh/ts-proto/commit/b5197174bcaacb8f163cd197d52ab9c645d21d4c))
+
+# [1.89.0](https://github.com/stephenh/ts-proto/compare/v1.88.0...v1.89.0) (2021-11-24)
+
+
+### Features
+
+* Improve map reading (fromJSON/fromPartial) ([#410](https://github.com/stephenh/ts-proto/issues/410)) ([057d438](https://github.com/stephenh/ts-proto/commit/057d438548d95c354331f7d2d767ccff952ad5c6))
+
+# [1.88.0](https://github.com/stephenh/ts-proto/compare/v1.87.1...v1.88.0) (2021-11-22)
+
+
+### Features
+
+* Support for Google.Protobuf.Value, ListValue and Struct ([#396](https://github.com/stephenh/ts-proto/issues/396)) ([7dd9c16](https://github.com/stephenh/ts-proto/commit/7dd9c16ffdec4d9ea296fbdc30d390fe44192c42))
+
+## [1.87.1](https://github.com/stephenh/ts-proto/compare/v1.87.0...v1.87.1) (2021-11-21)
+
+
+### Bug Fixes
+
+* code generation for int64 map values in fromPartial and fromJson ([#395](https://github.com/stephenh/ts-proto/issues/395)) ([d3ea8eb](https://github.com/stephenh/ts-proto/commit/d3ea8eb69e19a5e45fcc1766c4af1194b17e48fc))
+
 # [1.87.0](https://github.com/stephenh/ts-proto/compare/v1.86.0...v1.87.0) (2021-11-16)
 
 

package/README.markdown

@@ -22,10 +22,13 @@
 - [Assumptions](#assumptions)
 - [Todo](#todo)
 - [OneOf Handling](#oneof-handling)
-- [Primitive Types](#primitive-types)
+- [Default Values and Unset Fields](#default-values-and-unset-fields)
+- [Well-Known Types](#well-known-types)
+  - [Wrapper Types](#wrapper-types)
+  - [JSON Types (Struct Types)](#json-types-struct-types)
+  - [Timestamps](#timestamp)
 - [Wrapper Types](#wrapper-types)
 - [Number Types](#number-types)
-- [Timestamps](#timestamps)
 - [Current Status of Optional Values](#current-status-of-optional-values)
 
 # Overview
@@ -70,7 +73,7 @@ It will also generate client implementations of `PingService`; currently [Twirp]
 - `npm install ts-proto`
 - `protoc --plugin=./node_modules/.bin/protoc-gen-ts_proto --ts_proto_out=. ./simple.proto`
   - (Note that the output parameter name, `ts_proto_out`, is named based on the suffix of the plugin's name, i.e. "ts_proto" suffix in the `--plugin=./node_modules/.bin/protoc-gen-ts_proto` parameter becomes the `_out` prefix, per `protoc`'s CLI conventions.)
-  - On Windows, use `protoc --plugin=protoc-gen-ts_proto=.\node_modules\.bin\protoc-gen-ts_proto.cmd --ts_proto_out=. ./simple.proto`
+  - On Windows, use `protoc --plugin=protoc-gen-ts_proto=.\node_modules\.bin\protoc-gen-ts_proto.cmd --ts_proto_out=. ./simple.proto` (see [#93](https://github.com/stephenh/ts-proto/issues/93))
   - Ensure you're using a modern `protoc`, i.e. the original `protoc` `3.0.0` doesn't support the `_opt` flag
 
 This will generate `*.ts` source files for the given `*.proto` types.
@@ -376,7 +379,6 @@ The test suite's proto files (i.e. `simple.proto`, `batching.proto`, etc.) curre
 # Todo
 
 - Support the string-based encoding of duration in `fromJSON`/`toJSON`
-- Support the `json_name` annotation
 - Make `oneof=unions` the default behavior in 2.0
 - Probably change `forceLong` default in 2.0, should default to `forceLong=long`
 - Make `esModuleInterop=true` the default in 2.0
@@ -399,29 +401,201 @@ As this will automatically enforce only one of `field_a` or `field_b` "being set
 
 In ts-proto's currently-unscheduled 2.x release, `oneof=unions` will become the default behavior.
 
-# Primitive Types
+# Default values and unset fields
 
-Protobuf has the somewhat annoying behavior that primitives types cannot differentiate between set-to-defalut-value and unset.
+In core Protobuf, values that are _unset_ or equal to the default value are not sent over the wire.
+The default value of a message is `undefined`. Primitive types take their natural default value, i.e. `string` is `''`, `number` is `0`, etc.
+This behavior enables forward compatibility, as primitive fields will always have a value, even when omitted by outdated agents, but it also means _default_ and _unset_ values cannot be distinguished.
 
-I.e. if you have a `string name = 1`, and set `object.name = ''`, Protobuf will skip sending the tagged `name` field over the wire, because its understood that readers on the other end will, when they see `name` is not included in the payload, return empty string.
+If you need primitive fields where you can detect set/unset, see [Wrapper Types](#wrapper-types).
 
-`ts-proto` models this behavior, of "unset" values being the primitive's default. (Technically by setting up an object prototype that knows the default values of the message's primitive fields.)
+**Encode / Decode**
 
-If you want fields where you can model set/unset, see Wrapper Types.
+`ts-proto` follows the Protobuf rules, and always returns default values for unsets fields when decoding, while omitting them from the output when serialized in binary format.
 
-# Wrapper Types
+```protobuf
+syntax = "proto3";
+message Foo {
+  string bar = 1;
+}
+```
+
+```typescript
+protobufBytes; // assume this is an empty Foo object, in protobuf binary format
+Foo.decode(protobufBytes); // => { bar: '' }
+```
+
+```typescript
+Foo.encode({ bar: '' }); // => { }, writes an empty Foo object, in protobuf binary format
+```
+
+**fromJSON / toJSON**
+
+Reading JSON will also initialize the default values. Since senders may either omit unset fields, or set them to the default value, use `fromJSON` to normalize the input.
+
+```typescript
+Foo.fromJSON({ }); // => { bar: '' }
+Foo.fromJSON({ bar: '' }); // => { bar: '' }
+Foo.fromJSON({ bar: 'baz' }); // => { bar: 'baz' }
+```
+
+When writing JSON, `ts-proto` currently does **not** normalize message when converting to JSON, other than omitting unset fields, but it may do so in the future.
+
+```typescript
+// Current ts-proto behavior
+Foo.toJSON({ }); // => { }
+Foo.toJSON({ bar: undefined }); // => { }
+Foo.toJSON({ bar: '' }); // => { bar: '' } - note: this is the default value, but it's not omitted
+Foo.toJSON({ bar: 'baz' }); // => { bar: 'baz' }
+```
+
+```typescript
+// Possible future behavior, where ts-proto would normalize message
+Foo.toJSON({ }); // => { }
+Foo.toJSON({ bar: undefined }); // => { }
+Foo.toJSON({ bar: '' }); // => { } - note: omitting the default value, as expected
+Foo.toJSON({ bar: 'baz' }); // => { bar: 'baz' }
+```
 
-In core Protobuf, unset primitive fields become their respective default values (so you loose ability to distinguish "unset" from "default").
+-  Please open an issue if you need this behavior.
 
-However, unset message fields stay `null`.
+# Well-Known Types
 
-This allows a cute hack where you can model a logical `string | unset` by creating a field that is technically a message (i.e. so it can stay `null` for the unset case), but the message only has a single string field (i.e for storing the value in the set case).
+Protobuf comes with several predefined message definitions, called "[Well-Known Types](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf)".
+Their interpretation is defined by the Protobuf specification, and libraries are expected to convert these messages to corresponding native types in the target language.
 
-Protobuf has already "blessed" this pattern with several built-in types, i.e. `google.protobuf.StringValue`, `google.protobuf.Int32Value`, etc.
+`ts-proto` currently automatically converts these messages to their corresponding native types.
 
-`ts-proto` understands these wrapper types and "re-idiomizes" them by generating a `google.protobuf.StringValue name = 1` field as a `name: string | undefined`, and hides the `StringValue` implementation detail from your code (i.e. during `encode`/`decode` of the `name` field on the wire to external consumers, it's still read/written as a `StringValue` message field).
+- Wrapper Types:
+
+  * [google.protobuf.DoubleValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#DoubleValue) ⇆ `number | undefined`
+  * [google.protobuf.FloatValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#FloatValue) ⇆ `number | undefined`
+  * [google.protobuf.Int64Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Int64Value) ⇆ `number | undefined`
+  * [google.protobuf.UInt64Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#UInt64Value) ⇆ `number | undefined`
+  * [google.protobuf.Int32Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Int32Value) ⇆ `number | undefined`
+  * [google.protobuf.UInt32Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#UInt32Value) ⇆ `number | undefined`
+  * [google.protobuf.BoolValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#BoolValue) ⇆ `boolean | undefined`
+  * [google.protobuf.StringValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#StringValue) ⇆ `string | undefined`
+  * [google.protobuf.BytesValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.BytesValue) ⇆ `Uint8Array | undefined`
+
+- JSON Types (Struct Types):
+
+  * [google.protobuf.Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Value)  ⇆ `any | undefined` (i.e. `number | string | boolean | null | array | object`)
+  * [google.protobuf.ListValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#ListValue) ⇆ `any[]`
+  * [google.protobuf.Struct](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Struct) ⇆ `{ [key: string]: any } | undefined`
+
+## Wrapper Types
+
+Wrapper Types are messages containing a single primitive field, and can be imported in `.proto` files with `import "google/protobuf/wrappers.proto"`.
+
+
+Since these are _messages_, their default value is `undefined`, allowing you to distinguish unset primitives from their default values, when using Wrapper Types.
+`ts-proto` generates these fields as `<primitive> | undefined`.
+
+For example:
+
+```protobuf
+// Protobuf
+syntax = "proto3";
+
+import "google/protobuf/wrappers.proto";
+
+message ExampleMessage {
+  google.protobuf.StringValue name = 1;
+}
+```
+
+```typescript
+// TypeScript
+interface ExampleMessage {
+  name: string | undefined;
+}
+```
+
+When encoding a message the primitive value is converted back to its corresponding wrapper type:
+
+```typescript
+ExampleMessage.encode({ name: 'foo' }) // => { name: { value: 'foo' } }, in binary
+```
+
+When calling toJSON, the value is not converted, because wrapper types are idiomatic in JSON.
+
+```typescript
+ExampleMessage.toJSON({ name: 'foo' }) // => { name: 'foo' }
+```
+
+## JSON Types (Struct Types)
+
+Protobuf's language and types are not sufficient to represent all possible JSON values, since JSON may contain values whose type is unknown in advance.
+For this reason, Protobuf offers several additional types to represent arbitrary JSON values.
+
+These are called Struct Types, and can be imported in `.proto` files with `import "google/protobuf/struct.proto"`.
+
+- [google.protobuf.Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Value) &lrarr; `any`  
+  - This is the most general type, and can represent any JSON value (i.e. `number | string | boolean | null | array | object`).
+- [google.protobuf.ListValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#ListValue) &lrarr; `any[]`
+  - To represent a JSON array
+- [google.protobuf.Struct](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Struct) &lrarr; `{ [key: string]: any }`
+  - To represent a JSON object
+
+`ts-proto` automatically converts back and forth between these Struct Types and their corresponding JSON types.
+
+Example:
+
+```protobuf
+// Protobuf
+syntax = "proto3";
+
+import "google/protobuf/struct.proto";
+
+message ExampleMessage {
+  google.protobuf.Value anything = 1;
+}
+```
+
+```typescript
+// TypeScript
+interface ExampleMessage {
+  anything: any | undefined;
+}
+```
+
+Encoding a JSON value embedded in a message, converts it to a Struct Type:
+```typescript
+ExampleMessage.encode({ anything: { "name": "hello" } })
+/* Outputs the following structure, encoded in protobuf binary format:
+{
+  anything: Value {
+    structValue = Struct {
+      fields = [
+        MapEntry {
+          key = "name",
+          value = Value {
+            stringValue = "hello"
+          }
+        ]
+      }
+    }
+ }
+}*/
+
+ExampleMessage.encode({ anything: true })
+/* Outputs the following structure encoded in protobuf binary format:
+{
+  anything: Value {
+    boolValue = true
+  }
+}*/
+```
+
+## Timestamp
+
+The representation of `google.protobuf.Timestamp` is configurable by the `useDate` flag.
+
+| Protobuf well-known type    | Default/`useDate=true` | `useDate=false`                      | `useDate=string` |
+| --------------------------- | ---------------------- | ------------------------------------ | ---------------- |
+| `google.protobuf.Timestamp` | `Date`                 | `{ seconds: number, nanos: number }` | `string`         |
 
-This makes dealing with `string | unset` in your code much nicer, albeit it's unfortunate that, in Protobuf core, this is not as simple as marking a `string name = 1` field as `optional`, i.e. you have to "dirty" your proto files a bit by knowing to use the `StringValue` convention.
 
 # Number Types
 
@@ -452,14 +626,6 @@ The protobuf number types map to JavaScript types based on the `forceLong` confi
 
 Where (\*) indicates they might throw an error at runtime.
 
-# Timestamps
-
-The representation of `google.protobuf.Timestamp` is configurable by the `useDate` flag.
-
-| Protobuf well-known type    | Default/`useDate=true` | `useDate=false`                      | `useDate=string` |
-| --------------------------- | ---------------------- | ------------------------------------ | ---------------- |
-| `google.protobuf.Timestamp` | `Date`                 | `{ seconds: number, nanos: number }` | `string`         |
-
 # Current Status of Optional Values
 
 - Required primitives: use as-is, i.e. `string name = 1`.

package/build/encode.js

@@ -32,6 +32,8 @@ function generateEncoder(ctx, typeName) {
             return ts_poet_1.code `${TypeValue}.encode({value: value ?? false}).finish()`;
         case 'BytesValue':
             return ts_poet_1.code `${TypeValue}.encode({value: value ?? new Uint8Array()}).finish()`;
+        case 'ListValue':
+            return ts_poet_1.code `${TypeValue}.encode({value: value ?? []]}).finish()`;
     }
     throw new Error(`unknown wrapper type: ${name}`);
 }

package/build/main.js

@@ -184,6 +184,7 @@ function makeUtils(options) {
         ...makeDeepPartial(options, longs),
         ...makeTimestampMethods(options, longs),
         ...longs,
+        ...makeWrappingUtils(),
     };
 }
 exports.makeUtils = makeUtils;
@@ -240,6 +241,57 @@ function makeLongUtils(options, bytes) {
     `);
     return { numberToLong, longToNumber, longToString, longInit, Long };
 }
+function makeWrappingUtils() {
+    const wrapAnyValue = ts_poet_1.conditionalOutput('wrapAnyValue', ts_poet_1.code `function wrapAnyValue(value: any): Value {
+    if (value === null) {
+        return {nullValue: 0} as Value;
+      } else if (typeof value === 'boolean') {
+        return {boolValue: value} as Value;
+      } else if (typeof value === 'number') {
+        return {numberValue: value} as Value;
+      } else if (typeof value === 'string') {
+        return {stringValue: value} as Value;
+      } else if (Array.isArray(value)) {
+        return {listValue: value} as Value;
+      } else if (typeof value === 'object') {
+        return {structValue: value} as Value;
+      } else if (typeof value === 'undefined') {
+        return {} as Value;
+      } else {
+        throw new Error('Unsupported any value type: ' + typeof value);
+      }
+  }`);
+    const unwrapAnyValue = ts_poet_1.conditionalOutput('unwrapAnyValue', ts_poet_1.code `function unwrapAnyValue(value: Value): string | number | boolean | Object | null | Array<any> | undefined {
+    if (value.stringValue !== undefined) {
+      return value.stringValue;
+    } else if (value.numberValue !== undefined) {
+      return value.numberValue;
+    } else if (value.boolValue !== undefined) {
+      return value.boolValue;
+    } else if (value.structValue !== undefined) {
+      return value.structValue;
+    } else if (value.listValue !== undefined) {
+        return value.listValue;
+    } else if (value.nullValue !== undefined) {
+      return null;
+    }
+  }`);
+    const wrapStruct = ts_poet_1.conditionalOutput('wrapStruct', ts_poet_1.code `function wrapStruct(object: {[key: string]: any}): Struct {
+    const struct = Struct.fromPartial({});
+    Object.keys(object).forEach(key => {
+      struct.fields[key] = object[key];
+    });
+    return struct;
+  }`);
+    const unwrapStruct = ts_poet_1.conditionalOutput('unwrapStruct', ts_poet_1.code `function unwrapStruct(struct: Struct): {[key: string]: any} {
+    const object: { [key: string]: any } = {};
+    Object.keys(struct.fields).forEach(key => {
+      object[key] = struct.fields[key];
+    });
+    return object;
+  }`);
+    return { wrapAnyValue, unwrapAnyValue, wrapStruct, unwrapStruct };
+}
 function makeByteUtils() {
     const globalThis = ts_poet_1.conditionalOutput('globalThis', ts_poet_1.code `
       declare var self: any | undefined;
@@ -285,13 +337,15 @@ function makeDeepPartial(options, longs) {
     `;
     }
     const maybeExport = options.exportCommonSymbols ? 'export' : '';
-    const maybeLong = options.forceLong === options_1.LongOption.LONG ? ts_poet_1.code ` | ${longs.Long}` : '';
+    // Allow passing longs as numbers or strings, nad we'll convert them
+    const maybeLong = options.forceLong === options_1.LongOption.LONG ? ts_poet_1.code ` : T extends ${longs.Long} ? string | number | Long ` : '';
     const keys = options.outputTypeRegistry ? ts_poet_1.code `Exclude<keyof T, '$type'>` : ts_poet_1.code `keyof T`;
     // Based on the type from ts-essentials
     const DeepPartial = ts_poet_1.conditionalOutput('DeepPartial', ts_poet_1.code `
-      type Builtin = Date | Function | Uint8Array | string | number | boolean | undefined${maybeLong};
-      ${maybeExport} type DeepPartial<T> = T extends Builtin
+      type Builtin = Date | Function | Uint8Array | string | number | boolean | undefined;
+      ${maybeExport} type DeepPartial<T> =  T extends Builtin
         ? T
+        ${maybeLong}
         : T extends Array<infer U>
         ? Array<DeepPartial<U>>
         : T extends ReadonlyArray<infer U>
@@ -526,8 +580,18 @@ function generateDecode(ctx, fullName, messageDesc) {
             }
         }
         else if (types_1.isValueType(ctx, field)) {
+            const unwrap = (decodedValue) => {
+                if (types_1.isAnyValueType(field))
+                    return ts_poet_1.code `${ctx.utils.unwrapAnyValue}(${decodedValue})`;
+                if (types_1.isStructType(field))
+                    return ts_poet_1.code `${ctx.utils.unwrapStruct}(${decodedValue})`;
+                if (types_1.isListValueType(field))
+                    return ts_poet_1.code `${decodedValue}.values`;
+                return ts_poet_1.code `${decodedValue}.value`;
+            };
             const type = types_1.basicTypeName(ctx, field, { keepValueType: true });
-            readSnippet = ts_poet_1.code `${type}.decode(reader, reader.uint32()).value`;
+            const decoder = ts_poet_1.code `${type}.decode(reader, reader.uint32())`;
+            readSnippet = ts_poet_1.code `${unwrap(decoder)}`;
         }
         else if (types_1.isTimestamp(field) && (options.useDate === options_1.DateOption.DATE || options.useDate === options_1.DateOption.STRING)) {
             const type = types_1.basicTypeName(ctx, field, { keepValueType: true });
@@ -622,10 +686,19 @@ function generateEncode(ctx, fullName, messageDesc) {
             writeSnippet = (place) => ts_poet_1.code `${type}.encode(${utils.toTimestamp}(${place}), writer.uint32(${tag}).fork()).ldelim()`;
         }
         else if (types_1.isValueType(ctx, field)) {
+            const maybeTypeField = options.outputTypeRegistry ? `$type: '${field.typeName.slice(1)}',` : '';
+            const wrappedValue = (place) => {
+                if (types_1.isAnyValueType(field))
+                    return ts_poet_1.code `${ctx.utils.wrapAnyValue}(${place})`;
+                if (types_1.isListValueType(field))
+                    return ts_poet_1.code `{values: ${place}}`;
+                if (types_1.isStructType(field))
+                    return ts_poet_1.code `${ctx.utils.wrapStruct}(${place})`;
+                return ts_poet_1.code `{${maybeTypeField} value: ${place}!}`;
+            };
             const tag = ((field.number << 3) | 2) >>> 0;
             const type = types_1.basicTypeName(ctx, field, { keepValueType: true });
-            const maybeTypeField = options.outputTypeRegistry ? `$type: '${field.typeName.slice(1)}',` : '';
-            writeSnippet = (place) => ts_poet_1.code `${type}.encode({ ${maybeTypeField} value: ${place}! }, writer.uint32(${tag}).fork()).ldelim()`;
+            writeSnippet = (place) => ts_poet_1.code `${type}.encode(${wrappedValue(place)}, writer.uint32(${tag}).fork()).ldelim()`;
         }
         else if (types_1.isMessage(field)) {
             const tag = ((field.number << 3) | 2) >>> 0;
@@ -741,6 +814,7 @@ function generateFromJson(ctx, fullName, messageDesc) {
     // add a check for each incoming field
     messageDesc.field.forEach((field) => {
         const fieldName = case_1.maybeSnakeToCamel(field.name, options);
+        const jsonName = utils_1.determineFieldJsonName(field, options);
         // get a generic 'reader.doSomething' bit that is specific to the basic type
         const readSnippet = (from) => {
             if (types_1.isEnum(field)) {
@@ -773,6 +847,9 @@ function generateFromJson(ctx, fullName, messageDesc) {
                 (options.useDate === options_1.DateOption.DATE || options.useDate === options_1.DateOption.TIMESTAMP)) {
                 return ts_poet_1.code `${utils.fromJsonTimestamp}(${from})`;
             }
+            else if (types_1.isAnyValueType(field) || types_1.isStructType(field)) {
+                return ts_poet_1.code `${from}`;
+            }
             else if (types_1.isValueType(ctx, field)) {
                 const valueType = types_1.valueTypeName(ctx, field.typeName);
                 if (types_1.isLongValueType(field) && options.forceLong === options_1.LongOption.LONG) {
@@ -781,6 +858,9 @@ function generateFromJson(ctx, fullName, messageDesc) {
                 else if (types_1.isBytesValueType(field)) {
                     return ts_poet_1.code `new ${case_1.capitalize(valueType.toCodeString())}(${from})`;
                 }
+                else if (types_1.isListValueType(field)) {
+                    return ts_poet_1.code `[...${from}]`;
+                }
                 else {
                     return ts_poet_1.code `${case_1.capitalize(valueType.toCodeString())}(${from})`;
                 }
@@ -798,6 +878,9 @@ function generateFromJson(ctx, fullName, messageDesc) {
                                 return ts_poet_1.code `${utils.bytesFromBase64}(${from} as string)`;
                             }
                         }
+                        else if (types_1.isLong(valueType) && options.forceLong === options_1.LongOption.LONG) {
+                            return ts_poet_1.code `Long.fromString(${from} as string)`;
+                        }
                         else if (types_1.isEnum(valueType)) {
                             return ts_poet_1.code `${from} as number`;
                         }
@@ -817,6 +900,9 @@ function generateFromJson(ctx, fullName, messageDesc) {
                         const type = types_1.basicTypeName(ctx, valueType);
                         return ts_poet_1.code `${from} as ${type}`;
                     }
+                    else if (types_1.isAnyValueType(valueType)) {
+                        return ts_poet_1.code `${from}`;
+                    }
                     else {
                         const type = types_1.basicTypeName(ctx, valueType);
                         return ts_poet_1.code `${type}.fromJSON(${from})`;
@@ -834,36 +920,53 @@ function generateFromJson(ctx, fullName, messageDesc) {
         // and then use the snippet to handle repeated fields if necessary
         if (types_1.isRepeated(field)) {
             if (types_1.isMapType(ctx, messageDesc, field)) {
-                chunks.push(ts_poet_1.code `message.${fieldName} = {};`);
-                chunks.push(ts_poet_1.code `if (object.${fieldName} !== undefined && object.${fieldName} !== null) {`);
+                const fieldType = types_1.toTypeName(ctx, messageDesc, field);
                 const i = maybeCastToNumber(ctx, messageDesc, field, 'key');
                 chunks.push(ts_poet_1.code `
-          Object.entries(object.${fieldName}).forEach(([key, value]) => {
-            message.${fieldName}[${i}] = ${readSnippet('value')};
-          });
+          message.${fieldName} = Object.entries(object.${jsonName} ?? {}).reduce<${fieldType}>((acc, [key, value]) => {
+            acc[${i}] = ${readSnippet('value')};
+            return acc;
+          }, {});
+        `);
+            }
+            else if (types_1.isAnyValueType(field)) {
+                chunks.push(ts_poet_1.code `
+          message.${fieldName} = Array.isArray(object?.${jsonName}) ? [...object.${jsonName}] : [];
         `);
-                chunks.push(ts_poet_1.code `}`);
             }
             else {
                 // Explicit `any` type required to make TS with noImplicitAny happy. `object` is also `any` here.
                 chunks.push(ts_poet_1.code `
-          message.${fieldName} = (object.${fieldName} ?? []).map((e: any) => ${readSnippet('e')});
+          message.${fieldName} = (object.${jsonName} ?? []).map((e: any) => ${readSnippet('e')});
         `);
             }
         }
         else if (types_1.isWithinOneOfThatShouldBeUnion(options, field)) {
-            chunks.push(ts_poet_1.code `if (object.${fieldName} !== undefined && object.${fieldName} !== null) {`);
+            chunks.push(ts_poet_1.code `if (object.${jsonName} !== undefined && object.${jsonName} !== null) {`);
             const oneofName = case_1.maybeSnakeToCamel(messageDesc.oneofDecl[field.oneofIndex].name, options);
             chunks.push(ts_poet_1.code `
-        message.${oneofName} = { $case: '${fieldName}', ${fieldName}: ${readSnippet(`object.${fieldName}`)} }
+        message.${oneofName} = { $case: '${fieldName}', ${fieldName}: ${readSnippet(`object.${jsonName}`)} }
       `);
             chunks.push(ts_poet_1.code `}`);
         }
+        else if (types_1.isAnyValueType(field)) {
+            chunks.push(ts_poet_1.code `message.${fieldName} = object.${fieldName};`);
+        }
+        else if (types_1.isStructType(field)) {
+            chunks.push(ts_poet_1.code `message.${fieldName} = typeof(object.${fieldName}) === 'object' ? object.${fieldName} : undefined;`);
+        }
+        else if (types_1.isListValueType(field)) {
+            chunks.push(ts_poet_1.code `
+        message.${fieldName} = Array.isArray(object?.${fieldName})
+          ? ${readSnippet(`object.${fieldName}`)}
+          : ${'undefined'};
+      `);
+        }
         else {
             const fallback = types_1.isWithinOneOf(field) ? 'undefined' : types_1.defaultValue(ctx, field);
             chunks.push(ts_poet_1.code `
-        message.${fieldName} = (object.${fieldName} !== undefined && object.${fieldName} !== null)
-          ? ${readSnippet(`object.${fieldName}`)}
+        message.${fieldName} = (object.${jsonName} !== undefined && object.${jsonName} !== null)
+          ? ${readSnippet(`object.${jsonName}`)}
           : ${fallback};
       `);
         }
@@ -884,6 +987,7 @@ function generateToJson(ctx, fullName, messageDesc) {
     // then add a case for each field
     messageDesc.field.forEach((field) => {
         const fieldName = case_1.maybeSnakeToCamel(field.name, options);
+        const jsonName = utils_1.determineFieldJsonName(field, options);
         const readSnippet = (from) => {
             if (types_1.isEnum(field)) {
                 const toJson = types_1.getEnumMethod(typeMap, field.typeName, 'ToJSON');
@@ -919,14 +1023,23 @@ function generateToJson(ctx, fullName, messageDesc) {
                 else if (types_1.isTimestamp(valueType) && options.useDate === options_1.DateOption.TIMESTAMP) {
                     return ts_poet_1.code `${utils.fromTimestamp}(${from}).toISOString()`;
                 }
+                else if (types_1.isLong(valueType) && options.forceLong === options_1.LongOption.LONG) {
+                    return ts_poet_1.code `${from}.toString()`;
+                }
                 else if (types_1.isScalar(valueType) || types_1.isValueType(ctx, valueType)) {
                     return ts_poet_1.code `${from}`;
                 }
+                else if (types_1.isAnyValueType(valueType)) {
+                    return ts_poet_1.code `${from}`;
+                }
                 else {
                     const type = types_1.basicTypeName(ctx, valueType);
                     return ts_poet_1.code `${type}.toJSON(${from})`;
                 }
             }
+            else if (types_1.isAnyValueType(field)) {
+                return ts_poet_1.code `${from}`;
+            }
             else if (types_1.isMessage(field) && !types_1.isValueType(ctx, field) && !types_1.isMapType(ctx, messageDesc, field)) {
                 const type = types_1.basicTypeName(ctx, field, { keepValueType: true });
                 return ts_poet_1.code `${from} ? ${type}.toJSON(${from}) : ${types_1.defaultValue(ctx, field)}`;
@@ -950,10 +1063,10 @@ function generateToJson(ctx, fullName, messageDesc) {
         if (types_1.isMapType(ctx, messageDesc, field)) {
             // Maps might need their values transformed, i.e. bytes --> base64
             chunks.push(ts_poet_1.code `
-        obj.${fieldName} = {};
+        obj.${jsonName} = {};
         if (message.${fieldName}) {
           Object.entries(message.${fieldName}).forEach(([k, v]) => {
-            obj.${fieldName}[k] = ${readSnippet('v')};
+            obj.${jsonName}[k] = ${readSnippet('v')};
           });
         }
       `);
@@ -962,9 +1075,9 @@ function generateToJson(ctx, fullName, messageDesc) {
             // Arrays might need their elements transformed
             chunks.push(ts_poet_1.code `
         if (message.${fieldName}) {
-          obj.${fieldName} = message.${fieldName}.map(e => ${readSnippet('e')});
+          obj.${jsonName} = message.${fieldName}.map(e => ${readSnippet('e')});
         } else {
-          obj.${fieldName} = [];
+          obj.${jsonName} = [];
         }
       `);
         }
@@ -972,11 +1085,11 @@ function generateToJson(ctx, fullName, messageDesc) {
             // oneofs in a union are only output as `oneof name = ...`
             const oneofName = case_1.maybeSnakeToCamel(messageDesc.oneofDecl[field.oneofIndex].name, options);
             const v = readSnippet(`message.${oneofName}?.${fieldName}`);
-            chunks.push(ts_poet_1.code `message.${oneofName}?.$case === '${fieldName}' && (obj.${fieldName} = ${v});`);
+            chunks.push(ts_poet_1.code `message.${oneofName}?.$case === '${fieldName}' && (obj.${jsonName} = ${v});`);
         }
         else {
             const v = readSnippet(`message.${fieldName}`);
-            chunks.push(ts_poet_1.code `message.${fieldName} !== undefined && (obj.${fieldName} = ${v});`);
+            chunks.push(ts_poet_1.code `message.${fieldName} !== undefined && (obj.${jsonName} = ${v});`);
         }
     });
     chunks.push(ts_poet_1.code `return obj;`);
@@ -996,7 +1109,10 @@ function generateFromPartial(ctx, fullName, messageDesc) {
     messageDesc.field.forEach((field) => {
         const fieldName = case_1.maybeSnakeToCamel(field.name, options);
         const readSnippet = (from) => {
-            if (types_1.isPrimitive(field) ||
+            if ((types_1.isLong(field) || types_1.isLongValueType(field)) && options.forceLong === options_1.LongOption.LONG) {
+                return ts_poet_1.code `Long.fromValue(${from})`;
+            }
+            else if (types_1.isPrimitive(field) ||
                 (types_1.isTimestamp(field) && (options.useDate === options_1.DateOption.DATE || options.useDate === options_1.DateOption.STRING)) ||
                 types_1.isValueType(ctx, field)) {
                 return ts_poet_1.code `${from}`;
@@ -1011,11 +1127,17 @@ function generateFromPartial(ctx, fullName, messageDesc) {
                         else if (types_1.isEnum(valueType)) {
                             return ts_poet_1.code `${from} as number`;
                         }
+                        else if (types_1.isLong(valueType) && options.forceLong === options_1.LongOption.LONG) {
+                            return ts_poet_1.code `Long.fromValue(${from})`;
+                        }
                         else {
                             const cstr = case_1.capitalize(types_1.basicTypeName(ctx, valueType).toCodeString());
                             return ts_poet_1.code `${cstr}(${from})`;
                         }
                     }
+                    else if (types_1.isAnyValueType(valueType)) {
+                        return ts_poet_1.code `${from}`;
+                    }
                     else if (types_1.isTimestamp(valueType) &&
                         (options.useDate === options_1.DateOption.DATE || options.useDate === options_1.DateOption.STRING)) {
                         return ts_poet_1.code `${from}`;
@@ -1028,6 +1150,9 @@ function generateFromPartial(ctx, fullName, messageDesc) {
                         return ts_poet_1.code `${type}.fromPartial(${from})`;
                     }
                 }
+                else if (types_1.isAnyValueType(field)) {
+                    return ts_poet_1.code `${from}`;
+                }
                 else {
                     const type = types_1.basicTypeName(ctx, field);
                     return ts_poet_1.code `${type}.fromPartial(${from})`;
@@ -1040,17 +1165,16 @@ function generateFromPartial(ctx, fullName, messageDesc) {
         // and then use the snippet to handle repeated fields if necessary
         if (types_1.isRepeated(field)) {
             if (types_1.isMapType(ctx, messageDesc, field)) {
-                chunks.push(ts_poet_1.code `message.${fieldName} = {};`);
-                chunks.push(ts_poet_1.code `if (object.${fieldName} !== undefined && object.${fieldName} !== null) {`);
+                const fieldType = types_1.toTypeName(ctx, messageDesc, field);
                 const i = maybeCastToNumber(ctx, messageDesc, field, 'key');
                 chunks.push(ts_poet_1.code `
-          Object.entries(object.${fieldName}).forEach(([key, value]) => {
+          message.${fieldName} = Object.entries(object.${fieldName} ?? {}).reduce<${fieldType}>((acc, [key, value]) => {
             if (value !== undefined) {
-              message.${fieldName}[${i}] = ${readSnippet('value')};
+              acc[${i}] = ${readSnippet('value')};
             }
-          });
+            return acc;
+          }, {});
         `);
-                chunks.push(ts_poet_1.code `}`);
             }
             else {
                 chunks.push(ts_poet_1.code `
@@ -1071,16 +1195,6 @@ function generateFromPartial(ctx, fullName, messageDesc) {
         }  
       `);
         }
-        else if ((types_1.isLong(field) || types_1.isLongValueType(field)) && options.forceLong === options_1.LongOption.LONG) {
-            const v = readSnippet(`object.${fieldName}`);
-            const type = types_1.basicTypeName(ctx, field);
-            chunks.push(ts_poet_1.code `if (object.${fieldName} !== undefined && object.${fieldName} !== null) {`);
-            chunks.push(ts_poet_1.code `message.${fieldName} = ${v} as ${type};`);
-            chunks.push(ts_poet_1.code `} else {`);
-            const fallback = types_1.isWithinOneOf(field) ? 'undefined' : types_1.defaultValue(ctx, field);
-            chunks.push(ts_poet_1.code `message.${fieldName} = ${fallback}`);
-            chunks.push(ts_poet_1.code `}`);
-        }
         else if (readSnippet(`x`).toCodeString() == 'x') {
             // An optimized case of the else below that works when `readSnippet` returns the plain input
             const fallback = types_1.isWithinOneOf(field) ? 'undefined' : types_1.defaultValue(ctx, field);

package/build/types.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.detectBatchMethod = exports.responsePromiseOrObservable = exports.responseObservable = exports.responsePromise = exports.responseType = exports.requestType = exports.rawRequestType = exports.detectMapType = exports.toTypeName = exports.getEnumMethod = exports.messageToTypeName = exports.wrapperTypeName = exports.valueTypeName = exports.isEmptyType = exports.isLongValueType = exports.isBytesValueType = exports.isValueType = exports.isTimestamp = exports.isMapType = exports.isLong = exports.isRepeated = exports.isWithinOneOfThatShouldBeUnion = exports.isWithinOneOf = exports.isEnum = exports.isMessage = exports.isBytes = exports.isPrimitive = exports.isScalar = exports.createTypeMap = exports.notDefaultCheck = exports.defaultValue = exports.packedType = exports.toReaderCall = exports.basicTypeName = exports.basicLongWireType = exports.basicWireType = void 0;
+exports.detectBatchMethod = exports.responsePromiseOrObservable = exports.responseObservable = exports.responsePromise = exports.responseType = exports.requestType = exports.rawRequestType = exports.detectMapType = exports.toTypeName = exports.getEnumMethod = exports.messageToTypeName = exports.wrapperTypeName = exports.valueTypeName = exports.isEmptyType = exports.isLongValueType = exports.isStructType = exports.isListValueTypeName = exports.isListValueType = exports.isBytesValueType = exports.isAnyValueType = exports.isValueType = exports.isTimestamp = exports.isMapType = exports.isLong = exports.isRepeated = exports.isWithinOneOfThatShouldBeUnion = exports.isWithinOneOf = exports.isEnum = exports.isMessage = exports.isBytes = exports.isPrimitive = exports.isScalar = exports.createTypeMap = exports.notDefaultCheck = exports.defaultValue = exports.packedType = exports.toReaderCall = exports.basicTypeName = exports.basicLongWireType = exports.basicWireType = void 0;
 const ts_proto_descriptors_1 = require("ts-proto-descriptors");
 const ts_poet_1 = require("ts-poet");
 const options_1 = require("./options");
@@ -359,10 +359,26 @@ function isValueType(ctx, field) {
     return valueTypeName(ctx, field.typeName) !== undefined;
 }
 exports.isValueType = isValueType;
+function isAnyValueType(field) {
+    return field.typeName === '.google.protobuf.Value';
+}
+exports.isAnyValueType = isAnyValueType;
 function isBytesValueType(field) {
     return field.typeName === '.google.protobuf.BytesValue';
 }
 exports.isBytesValueType = isBytesValueType;
+function isListValueType(field) {
+    return isListValueTypeName(field.typeName);
+}
+exports.isListValueType = isListValueType;
+function isListValueTypeName(typeName) {
+    return typeName === 'google.protobuf.ListValue' || typeName === '.google.protobuf.ListValue';
+}
+exports.isListValueTypeName = isListValueTypeName;
+function isStructType(field) {
+    return field.typeName === '.google.protobuf.Struct';
+}
+exports.isStructType = isStructType;
 function isLongValueType(field) {
     return field.typeName === '.google.protobuf.Int64Value' || field.typeName === '.google.protobuf.UInt64Value';
 }
@@ -388,6 +404,12 @@ function valueTypeName(ctx, typeName) {
             return ts_poet_1.code `boolean`;
         case '.google.protobuf.BytesValue':
             return ts_poet_1.code `Uint8Array`;
+        case '.google.protobuf.ListValue':
+            return ts_poet_1.code `Array<any>`;
+        case '.google.protobuf.Value':
+            return ts_poet_1.code `any`;
+        case '.google.protobuf.Struct':
+            return ts_poet_1.code `{[key: string]: any}`;
         default:
             return undefined;
     }
@@ -404,6 +426,7 @@ function wrapperTypeName(typeName) {
         case '.google.protobuf.UInt64Value':
         case '.google.protobuf.BoolValue':
         case '.google.protobuf.BytesValue':
+        case '.google.protobuf.ListValue':
         case '.google.protobuf.Timestamp':
             return typeName.split('.')[3];
         default:

package/build/utils.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.FormattedMethodDescriptor = exports.assertInstanceOf = exports.maybePrefixPackage = exports.prefixDisableLinter = exports.maybeAddComment = exports.upperFirst = exports.lowerFirst = exports.singular = exports.fail = exports.readToBuffer = exports.protoFilesToGenerate = void 0;
+exports.determineFieldJsonName = exports.FormattedMethodDescriptor = exports.assertInstanceOf = exports.maybePrefixPackage = exports.prefixDisableLinter = exports.maybeAddComment = exports.upperFirst = exports.lowerFirst = exports.singular = exports.fail = exports.readToBuffer = exports.protoFilesToGenerate = void 0;
 const ts_poet_1 = require("ts-poet");
 const options_1 = require("./options");
 const case_1 = require("./case");
@@ -144,3 +144,12 @@ class FormattedMethodDescriptor {
     }
 }
 exports.FormattedMethodDescriptor = FormattedMethodDescriptor;
+function determineFieldJsonName(field, options) {
+    // By default jsonName is camelCased by the protocol compilier unless the user has
+    // set a "json_name" option on this field.
+    if (field.jsonName.length > 0) {
+        return field.jsonName;
+    }
+    return case_1.maybeSnakeToCamel(field.name, options);
+}
+exports.determineFieldJsonName = determineFieldJsonName;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ts-proto",
-  "version": "1.87.0",
+  "version": "1.90.0",
   "description": "",
   "main": "build/plugin.js",
   "repository": "github:stephenh/ts-proto",
@@ -25,9 +25,9 @@
     "@grpc/proto-loader": "^0.5.6",
     "@improbable-eng/grpc-web": "^0.14.0",
     "@improbable-eng/grpc-web-node-http-transport": "^0.14.0",
-    "@nestjs/common": "^7.6.15",
-    "@nestjs/core": "^7.6.15",
-    "@nestjs/microservices": "^7.6.15",
+    "@nestjs/common": "^8.2.2",
+    "@nestjs/core": "^8.2.2",
+    "@nestjs/microservices": "^8.2.2",
     "@semantic-release/changelog": "^5.0.1",
     "@semantic-release/commit-analyzer": "^8.0.1",
     "@semantic-release/git": "^9.0.0",
@@ -36,11 +36,10 @@
     "@semantic-release/release-notes-generator": "^9.0.2",
     "@types/jest": "^26.0.22",
     "@types/node": "^14.14.37",
-    "grpc": "^1.24.6",
     "jest": "^26.6.3",
     "prettier": "^2.2.1",
     "reflect-metadata": "^0.1.13",
-    "rxjs": "^6.6.7",
+    "rxjs": "^7.4.0",
     "semantic-release": "^17.4.2",
     "ts-jest": "^26.5.4",
     "ts-node": "^9.1.1",