ts-proto 1.87.1 → 1.90.1

package/CHANGELOG.md

@@ -1,3 +1,31 @@
+## [1.90.1](https://github.com/stephenh/ts-proto/compare/v1.90.0...v1.90.1) (2021-11-27)
+
+
+### Bug Fixes
+
+* code-generation for Services with Struct response types ([#407](https://github.com/stephenh/ts-proto/issues/407)) ([f041fa1](https://github.com/stephenh/ts-proto/commit/f041fa1047816748c366bcb81895b6b917eb328d))
+
+# [1.90.0](https://github.com/stephenh/ts-proto/compare/v1.89.0...v1.90.0) (2021-11-24)
+
+
+### Features
+
+*  Add support for 'json_name' annotation ([#408](https://github.com/stephenh/ts-proto/issues/408)) ([b519717](https://github.com/stephenh/ts-proto/commit/b5197174bcaacb8f163cd197d52ab9c645d21d4c))
+
+# [1.89.0](https://github.com/stephenh/ts-proto/compare/v1.88.0...v1.89.0) (2021-11-24)
+
+
+### Features
+
+* Improve map reading (fromJSON/fromPartial) ([#410](https://github.com/stephenh/ts-proto/issues/410)) ([057d438](https://github.com/stephenh/ts-proto/commit/057d438548d95c354331f7d2d767ccff952ad5c6))
+
+# [1.88.0](https://github.com/stephenh/ts-proto/compare/v1.87.1...v1.88.0) (2021-11-22)
+
+
+### Features
+
+* Support for Google.Protobuf.Value, ListValue and Struct ([#396](https://github.com/stephenh/ts-proto/issues/396)) ([7dd9c16](https://github.com/stephenh/ts-proto/commit/7dd9c16ffdec4d9ea296fbdc30d390fe44192c42))
+
 ## [1.87.1](https://github.com/stephenh/ts-proto/compare/v1.87.0...v1.87.1) (2021-11-21)
 
 

package/README.markdown

@@ -22,10 +22,13 @@
 - [Assumptions](#assumptions)
 - [Todo](#todo)
 - [OneOf Handling](#oneof-handling)
-- [Primitive Types](#primitive-types)
+- [Default Values and Unset Fields](#default-values-and-unset-fields)
+- [Well-Known Types](#well-known-types)
+  - [Wrapper Types](#wrapper-types)
+  - [JSON Types (Struct Types)](#json-types-struct-types)
+  - [Timestamps](#timestamp)
 - [Wrapper Types](#wrapper-types)
 - [Number Types](#number-types)
-- [Timestamps](#timestamps)
 - [Current Status of Optional Values](#current-status-of-optional-values)
 
 # Overview
@@ -376,7 +379,6 @@ The test suite's proto files (i.e. `simple.proto`, `batching.proto`, etc.) curre
 # Todo
 
 - Support the string-based encoding of duration in `fromJSON`/`toJSON`
-- Support the `json_name` annotation
 - Make `oneof=unions` the default behavior in 2.0
 - Probably change `forceLong` default in 2.0, should default to `forceLong=long`
 - Make `esModuleInterop=true` the default in 2.0
@@ -399,29 +401,201 @@ As this will automatically enforce only one of `field_a` or `field_b` "being set
 
 In ts-proto's currently-unscheduled 2.x release, `oneof=unions` will become the default behavior.
 
-# Primitive Types
+# Default values and unset fields
 
-Protobuf has the somewhat annoying behavior that primitives types cannot differentiate between set-to-defalut-value and unset.
+In core Protobuf, values that are _unset_ or equal to the default value are not sent over the wire.
+The default value of a message is `undefined`. Primitive types take their natural default value, i.e. `string` is `''`, `number` is `0`, etc.
+This behavior enables forward compatibility, as primitive fields will always have a value, even when omitted by outdated agents, but it also means _default_ and _unset_ values cannot be distinguished.
 
-I.e. if you have a `string name = 1`, and set `object.name = ''`, Protobuf will skip sending the tagged `name` field over the wire, because its understood that readers on the other end will, when they see `name` is not included in the payload, return empty string.
+If you need primitive fields where you can detect set/unset, see [Wrapper Types](#wrapper-types).
 
-`ts-proto` models this behavior, of "unset" values being the primitive's default. (Technically by setting up an object prototype that knows the default values of the message's primitive fields.)
+**Encode / Decode**
 
-If you want fields where you can model set/unset, see Wrapper Types.
+`ts-proto` follows the Protobuf rules, and always returns default values for unsets fields when decoding, while omitting them from the output when serialized in binary format.
 
-# Wrapper Types
+```protobuf
+syntax = "proto3";
+message Foo {
+  string bar = 1;
+}
+```
+
+```typescript
+protobufBytes; // assume this is an empty Foo object, in protobuf binary format
+Foo.decode(protobufBytes); // => { bar: '' }
+```
+
+```typescript
+Foo.encode({ bar: '' }); // => { }, writes an empty Foo object, in protobuf binary format
+```
+
+**fromJSON / toJSON**
+
+Reading JSON will also initialize the default values. Since senders may either omit unset fields, or set them to the default value, use `fromJSON` to normalize the input.
+
+```typescript
+Foo.fromJSON({ }); // => { bar: '' }
+Foo.fromJSON({ bar: '' }); // => { bar: '' }
+Foo.fromJSON({ bar: 'baz' }); // => { bar: 'baz' }
+```
+
+When writing JSON, `ts-proto` currently does **not** normalize message when converting to JSON, other than omitting unset fields, but it may do so in the future.
+
+```typescript
+// Current ts-proto behavior
+Foo.toJSON({ }); // => { }
+Foo.toJSON({ bar: undefined }); // => { }
+Foo.toJSON({ bar: '' }); // => { bar: '' } - note: this is the default value, but it's not omitted
+Foo.toJSON({ bar: 'baz' }); // => { bar: 'baz' }
+```
+
+```typescript
+// Possible future behavior, where ts-proto would normalize message
+Foo.toJSON({ }); // => { }
+Foo.toJSON({ bar: undefined }); // => { }
+Foo.toJSON({ bar: '' }); // => { } - note: omitting the default value, as expected
+Foo.toJSON({ bar: 'baz' }); // => { bar: 'baz' }
+```
 
-In core Protobuf, unset primitive fields become their respective default values (so you loose ability to distinguish "unset" from "default").
+-  Please open an issue if you need this behavior.
 
-However, unset message fields stay `null`.
+# Well-Known Types
 
-This allows a cute hack where you can model a logical `string | unset` by creating a field that is technically a message (i.e. so it can stay `null` for the unset case), but the message only has a single string field (i.e for storing the value in the set case).
+Protobuf comes with several predefined message definitions, called "[Well-Known Types](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf)".
+Their interpretation is defined by the Protobuf specification, and libraries are expected to convert these messages to corresponding native types in the target language.
 
-Protobuf has already "blessed" this pattern with several built-in types, i.e. `google.protobuf.StringValue`, `google.protobuf.Int32Value`, etc.
+`ts-proto` currently automatically converts these messages to their corresponding native types.
 
-`ts-proto` understands these wrapper types and "re-idiomizes" them by generating a `google.protobuf.StringValue name = 1` field as a `name: string | undefined`, and hides the `StringValue` implementation detail from your code (i.e. during `encode`/`decode` of the `name` field on the wire to external consumers, it's still read/written as a `StringValue` message field).
+- Wrapper Types:
+
+  * [google.protobuf.DoubleValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#DoubleValue) ⇆ `number | undefined`
+  * [google.protobuf.FloatValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#FloatValue) ⇆ `number | undefined`
+  * [google.protobuf.Int64Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Int64Value) ⇆ `number | undefined`
+  * [google.protobuf.UInt64Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#UInt64Value) ⇆ `number | undefined`
+  * [google.protobuf.Int32Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Int32Value) ⇆ `number | undefined`
+  * [google.protobuf.UInt32Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#UInt32Value) ⇆ `number | undefined`
+  * [google.protobuf.BoolValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#BoolValue) ⇆ `boolean | undefined`
+  * [google.protobuf.StringValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#StringValue) ⇆ `string | undefined`
+  * [google.protobuf.BytesValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.BytesValue) ⇆ `Uint8Array | undefined`
+
+- JSON Types (Struct Types):
+
+  * [google.protobuf.Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Value)  ⇆ `any | undefined` (i.e. `number | string | boolean | null | array | object`)
+  * [google.protobuf.ListValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#ListValue) ⇆ `any[]`
+  * [google.protobuf.Struct](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Struct) ⇆ `{ [key: string]: any } | undefined`
+
+## Wrapper Types
+
+Wrapper Types are messages containing a single primitive field, and can be imported in `.proto` files with `import "google/protobuf/wrappers.proto"`.
+
+
+Since these are _messages_, their default value is `undefined`, allowing you to distinguish unset primitives from their default values, when using Wrapper Types.
+`ts-proto` generates these fields as `<primitive> | undefined`.
+
+For example:
+
+```protobuf
+// Protobuf
+syntax = "proto3";
+
+import "google/protobuf/wrappers.proto";
+
+message ExampleMessage {
+  google.protobuf.StringValue name = 1;
+}
+```
+
+```typescript
+// TypeScript
+interface ExampleMessage {
+  name: string | undefined;
+}
+```
+
+When encoding a message the primitive value is converted back to its corresponding wrapper type:
+
+```typescript
+ExampleMessage.encode({ name: 'foo' }) // => { name: { value: 'foo' } }, in binary
+```
+
+When calling toJSON, the value is not converted, because wrapper types are idiomatic in JSON.
+
+```typescript
+ExampleMessage.toJSON({ name: 'foo' }) // => { name: 'foo' }
+```
+
+## JSON Types (Struct Types)
+
+Protobuf's language and types are not sufficient to represent all possible JSON values, since JSON may contain values whose type is unknown in advance.
+For this reason, Protobuf offers several additional types to represent arbitrary JSON values.
+
+These are called Struct Types, and can be imported in `.proto` files with `import "google/protobuf/struct.proto"`.
+
+- [google.protobuf.Value](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Value) &lrarr; `any`  
+  - This is the most general type, and can represent any JSON value (i.e. `number | string | boolean | null | array | object`).
+- [google.protobuf.ListValue](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#ListValue) &lrarr; `any[]`
+  - To represent a JSON array
+- [google.protobuf.Struct](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#Struct) &lrarr; `{ [key: string]: any }`
+  - To represent a JSON object
+
+`ts-proto` automatically converts back and forth between these Struct Types and their corresponding JSON types.
+
+Example:
+
+```protobuf
+// Protobuf
+syntax = "proto3";
+
+import "google/protobuf/struct.proto";
+
+message ExampleMessage {
+  google.protobuf.Value anything = 1;
+}
+```
+
+```typescript
+// TypeScript
+interface ExampleMessage {
+  anything: any | undefined;
+}
+```
+
+Encoding a JSON value embedded in a message, converts it to a Struct Type:
+```typescript
+ExampleMessage.encode({ anything: { "name": "hello" } })
+/* Outputs the following structure, encoded in protobuf binary format:
+{
+  anything: Value {
+    structValue = Struct {
+      fields = [
+        MapEntry {
+          key = "name",
+          value = Value {
+            stringValue = "hello"
+          }
+        ]
+      }
+    }
+ }
+}*/
+
+ExampleMessage.encode({ anything: true })
+/* Outputs the following structure encoded in protobuf binary format:
+{
+  anything: Value {
+    boolValue = true
+  }
+}*/
+```
+
+## Timestamp
+
+The representation of `google.protobuf.Timestamp` is configurable by the `useDate` flag.
+
+| Protobuf well-known type    | Default/`useDate=true` | `useDate=false`                      | `useDate=string` |
+| --------------------------- | ---------------------- | ------------------------------------ | ---------------- |
+| `google.protobuf.Timestamp` | `Date`                 | `{ seconds: number, nanos: number }` | `string`         |
 
-This makes dealing with `string | unset` in your code much nicer, albeit it's unfortunate that, in Protobuf core, this is not as simple as marking a `string name = 1` field as `optional`, i.e. you have to "dirty" your proto files a bit by knowing to use the `StringValue` convention.
 
 # Number Types
 
@@ -452,14 +626,6 @@ The protobuf number types map to JavaScript types based on the `forceLong` confi
 
 Where (\*) indicates they might throw an error at runtime.
 
-# Timestamps
-
-The representation of `google.protobuf.Timestamp` is configurable by the `useDate` flag.
-
-| Protobuf well-known type    | Default/`useDate=true` | `useDate=false`                      | `useDate=string` |
-| --------------------------- | ---------------------- | ------------------------------------ | ---------------- |
-| `google.protobuf.Timestamp` | `Date`                 | `{ seconds: number, nanos: number }` | `string`         |
-
 # Current Status of Optional Values
 
 - Required primitives: use as-is, i.e. `string name = 1`.

package/build/encode.js

@@ -7,12 +7,20 @@ const options_1 = require("./options");
 function generateEncoder(ctx, typeName) {
     const name = types_1.wrapperTypeName(typeName);
     if (!name) {
-        return ts_poet_1.code `${types_1.messageToTypeName(ctx, typeName)}.encode(value).finish()`;
+        return ts_poet_1.code `${types_1.messageToTypeName(ctx, typeName, { keepValueType: true })}.encode(value).finish()`;
     }
     if (name == 'Timestamp') {
         const TimestampValue = ts_poet_1.imp(`${name}@./google/protobuf/timestamp`);
         return ts_poet_1.code `${TimestampValue}.encode(${ctx.utils.toTimestamp}(value)).finish()`;
     }
+    if (name == 'Struct') {
+        const StructType = ts_poet_1.imp(`${name}@./google/protobuf/struct`);
+        return ts_poet_1.code `${StructType}.encode(${StructType}.wrap(value)).finish()`;
+    }
+    if (name == 'ListValue') {
+        const ListValueType = ts_poet_1.imp(`${name}@./google/protobuf/struct`);
+        return ts_poet_1.code `${ListValueType}.encode({values: value ?? []}).finish()`;
+    }
     const TypeValue = ts_poet_1.imp(`${name}@./google/protobuf/wrappers`);
     switch (name) {
         case 'StringValue':
@@ -39,13 +47,17 @@ exports.generateEncoder = generateEncoder;
 function generateDecoder(ctx, typeName) {
     let name = types_1.wrapperTypeName(typeName);
     if (!name) {
-        return ts_poet_1.code `${types_1.messageToTypeName(ctx, typeName)}.decode(value)`;
+        return ts_poet_1.code `${types_1.messageToTypeName(ctx, typeName, { keepValueType: true })}.decode(value)`;
     }
     let TypeValue;
     if (name == 'Timestamp') {
         TypeValue = ts_poet_1.imp(`${name}@./google/protobuf/timestamp`);
         return ts_poet_1.code `${TypeValue}.decode(value)`;
     }
+    if (name == 'Struct' || name == 'ListValue') {
+        TypeValue = ts_poet_1.imp(`${name}@./google/protobuf/struct`);
+        return ts_poet_1.code `${TypeValue}.unwrap(${TypeValue}.decode(value))`;
+    }
     TypeValue = ts_poet_1.imp(`${name}@./google/protobuf/wrappers`);
     return ts_poet_1.code `${TypeValue}.decode(value).value`;
 }

package/build/main.js

@@ -77,6 +77,8 @@ function generateFile(ctx, fileDesc) {
             if (options.outputPartialMethods) {
                 staticMembers.push(generateFromPartial(ctx, fullName, message));
             }
+            staticMembers.push(...generateWrap(fullTypeName));
+            staticMembers.push(...generateUnwrap(fullTypeName));
             chunks.push(ts_poet_1.code `
           export const ${ts_poet_1.def(fullName)} = {
             ${ts_poet_1.joinCode(staticMembers, { on: ',\n\n' })}
@@ -529,7 +531,14 @@ function generateDecode(ctx, fullName, messageDesc) {
         }
         else if (types_1.isValueType(ctx, field)) {
             const type = types_1.basicTypeName(ctx, field, { keepValueType: true });
-            readSnippet = ts_poet_1.code `${type}.decode(reader, reader.uint32()).value`;
+            const unwrap = (decodedValue) => {
+                if (types_1.isListValueType(field) || types_1.isStructType(field) || types_1.isAnyValueType(field)) {
+                    return ts_poet_1.code `${type}.unwrap(${decodedValue})`;
+                }
+                return ts_poet_1.code `${decodedValue}.value`;
+            };
+            const decoder = ts_poet_1.code `${type}.decode(reader, reader.uint32())`;
+            readSnippet = ts_poet_1.code `${unwrap(decoder)}`;
         }
         else if (types_1.isTimestamp(field) && (options.useDate === options_1.DateOption.DATE || options.useDate === options_1.DateOption.STRING)) {
             const type = types_1.basicTypeName(ctx, field, { keepValueType: true });
@@ -624,10 +633,16 @@ function generateEncode(ctx, fullName, messageDesc) {
             writeSnippet = (place) => ts_poet_1.code `${type}.encode(${utils.toTimestamp}(${place}), writer.uint32(${tag}).fork()).ldelim()`;
         }
         else if (types_1.isValueType(ctx, field)) {
-            const tag = ((field.number << 3) | 2) >>> 0;
-            const type = types_1.basicTypeName(ctx, field, { keepValueType: true });
             const maybeTypeField = options.outputTypeRegistry ? `$type: '${field.typeName.slice(1)}',` : '';
-            writeSnippet = (place) => ts_poet_1.code `${type}.encode({ ${maybeTypeField} value: ${place}! }, writer.uint32(${tag}).fork()).ldelim()`;
+            const type = types_1.basicTypeName(ctx, field, { keepValueType: true });
+            const wrappedValue = (place) => {
+                if (types_1.isAnyValueType(field) || types_1.isListValueType(field) || types_1.isStructType(field)) {
+                    return ts_poet_1.code `${type}.wrap(${place})`;
+                }
+                return ts_poet_1.code `{${maybeTypeField} value: ${place}!}`;
+            };
+            const tag = ((field.number << 3) | 2) >>> 0;
+            writeSnippet = (place) => ts_poet_1.code `${type}.encode(${wrappedValue(place)}, writer.uint32(${tag}).fork()).ldelim()`;
         }
         else if (types_1.isMessage(field)) {
             const tag = ((field.number << 3) | 2) >>> 0;
@@ -743,6 +758,7 @@ function generateFromJson(ctx, fullName, messageDesc) {
     // add a check for each incoming field
     messageDesc.field.forEach((field) => {
         const fieldName = case_1.maybeSnakeToCamel(field.name, options);
+        const jsonName = utils_1.determineFieldJsonName(field, options);
         // get a generic 'reader.doSomething' bit that is specific to the basic type
         const readSnippet = (from) => {
             if (types_1.isEnum(field)) {
@@ -775,6 +791,9 @@ function generateFromJson(ctx, fullName, messageDesc) {
                 (options.useDate === options_1.DateOption.DATE || options.useDate === options_1.DateOption.TIMESTAMP)) {
                 return ts_poet_1.code `${utils.fromJsonTimestamp}(${from})`;
             }
+            else if (types_1.isAnyValueType(field) || types_1.isStructType(field)) {
+                return ts_poet_1.code `${from}`;
+            }
             else if (types_1.isValueType(ctx, field)) {
                 const valueType = types_1.valueTypeName(ctx, field.typeName);
                 if (types_1.isLongValueType(field) && options.forceLong === options_1.LongOption.LONG) {
@@ -783,6 +802,9 @@ function generateFromJson(ctx, fullName, messageDesc) {
                 else if (types_1.isBytesValueType(field)) {
                     return ts_poet_1.code `new ${case_1.capitalize(valueType.toCodeString())}(${from})`;
                 }
+                else if (types_1.isListValueType(field)) {
+                    return ts_poet_1.code `[...${from}]`;
+                }
                 else {
                     return ts_poet_1.code `${case_1.capitalize(valueType.toCodeString())}(${from})`;
                 }
@@ -822,6 +844,9 @@ function generateFromJson(ctx, fullName, messageDesc) {
                         const type = types_1.basicTypeName(ctx, valueType);
                         return ts_poet_1.code `${from} as ${type}`;
                     }
+                    else if (types_1.isAnyValueType(valueType)) {
+                        return ts_poet_1.code `${from}`;
+                    }
                     else {
                         const type = types_1.basicTypeName(ctx, valueType);
                         return ts_poet_1.code `${type}.fromJSON(${from})`;
@@ -839,36 +864,53 @@ function generateFromJson(ctx, fullName, messageDesc) {
         // and then use the snippet to handle repeated fields if necessary
         if (types_1.isRepeated(field)) {
             if (types_1.isMapType(ctx, messageDesc, field)) {
-                chunks.push(ts_poet_1.code `message.${fieldName} = {};`);
-                chunks.push(ts_poet_1.code `if (object.${fieldName} !== undefined && object.${fieldName} !== null) {`);
+                const fieldType = types_1.toTypeName(ctx, messageDesc, field);
                 const i = maybeCastToNumber(ctx, messageDesc, field, 'key');
                 chunks.push(ts_poet_1.code `
-          Object.entries(object.${fieldName}).forEach(([key, value]) => {
-            message.${fieldName}[${i}] = ${readSnippet('value')};
-          });
+          message.${fieldName} = Object.entries(object.${jsonName} ?? {}).reduce<${fieldType}>((acc, [key, value]) => {
+            acc[${i}] = ${readSnippet('value')};
+            return acc;
+          }, {});
+        `);
+            }
+            else if (types_1.isAnyValueType(field)) {
+                chunks.push(ts_poet_1.code `
+          message.${fieldName} = Array.isArray(object?.${jsonName}) ? [...object.${jsonName}] : [];
         `);
-                chunks.push(ts_poet_1.code `}`);
             }
             else {
                 // Explicit `any` type required to make TS with noImplicitAny happy. `object` is also `any` here.
                 chunks.push(ts_poet_1.code `
-          message.${fieldName} = (object.${fieldName} ?? []).map((e: any) => ${readSnippet('e')});
+          message.${fieldName} = (object.${jsonName} ?? []).map((e: any) => ${readSnippet('e')});
         `);
             }
         }
         else if (types_1.isWithinOneOfThatShouldBeUnion(options, field)) {
-            chunks.push(ts_poet_1.code `if (object.${fieldName} !== undefined && object.${fieldName} !== null) {`);
+            chunks.push(ts_poet_1.code `if (object.${jsonName} !== undefined && object.${jsonName} !== null) {`);
             const oneofName = case_1.maybeSnakeToCamel(messageDesc.oneofDecl[field.oneofIndex].name, options);
             chunks.push(ts_poet_1.code `
-        message.${oneofName} = { $case: '${fieldName}', ${fieldName}: ${readSnippet(`object.${fieldName}`)} }
+        message.${oneofName} = { $case: '${fieldName}', ${fieldName}: ${readSnippet(`object.${jsonName}`)} }
       `);
             chunks.push(ts_poet_1.code `}`);
         }
+        else if (types_1.isAnyValueType(field)) {
+            chunks.push(ts_poet_1.code `message.${fieldName} = object.${fieldName};`);
+        }
+        else if (types_1.isStructType(field)) {
+            chunks.push(ts_poet_1.code `message.${fieldName} = typeof(object.${fieldName}) === 'object' ? object.${fieldName} : undefined;`);
+        }
+        else if (types_1.isListValueType(field)) {
+            chunks.push(ts_poet_1.code `
+        message.${fieldName} = Array.isArray(object?.${fieldName})
+          ? ${readSnippet(`object.${fieldName}`)}
+          : ${'undefined'};
+      `);
+        }
         else {
             const fallback = types_1.isWithinOneOf(field) ? 'undefined' : types_1.defaultValue(ctx, field);
             chunks.push(ts_poet_1.code `
-        message.${fieldName} = (object.${fieldName} !== undefined && object.${fieldName} !== null)
-          ? ${readSnippet(`object.${fieldName}`)}
+        message.${fieldName} = (object.${jsonName} !== undefined && object.${jsonName} !== null)
+          ? ${readSnippet(`object.${jsonName}`)}
           : ${fallback};
       `);
         }
@@ -889,6 +931,7 @@ function generateToJson(ctx, fullName, messageDesc) {
     // then add a case for each field
     messageDesc.field.forEach((field) => {
         const fieldName = case_1.maybeSnakeToCamel(field.name, options);
+        const jsonName = utils_1.determineFieldJsonName(field, options);
         const readSnippet = (from) => {
             if (types_1.isEnum(field)) {
                 const toJson = types_1.getEnumMethod(typeMap, field.typeName, 'ToJSON');
@@ -930,11 +973,17 @@ function generateToJson(ctx, fullName, messageDesc) {
                 else if (types_1.isScalar(valueType) || types_1.isValueType(ctx, valueType)) {
                     return ts_poet_1.code `${from}`;
                 }
+                else if (types_1.isAnyValueType(valueType)) {
+                    return ts_poet_1.code `${from}`;
+                }
                 else {
                     const type = types_1.basicTypeName(ctx, valueType);
                     return ts_poet_1.code `${type}.toJSON(${from})`;
                 }
             }
+            else if (types_1.isAnyValueType(field)) {
+                return ts_poet_1.code `${from}`;
+            }
             else if (types_1.isMessage(field) && !types_1.isValueType(ctx, field) && !types_1.isMapType(ctx, messageDesc, field)) {
                 const type = types_1.basicTypeName(ctx, field, { keepValueType: true });
                 return ts_poet_1.code `${from} ? ${type}.toJSON(${from}) : ${types_1.defaultValue(ctx, field)}`;
@@ -958,10 +1007,10 @@ function generateToJson(ctx, fullName, messageDesc) {
         if (types_1.isMapType(ctx, messageDesc, field)) {
             // Maps might need their values transformed, i.e. bytes --> base64
             chunks.push(ts_poet_1.code `
-        obj.${fieldName} = {};
+        obj.${jsonName} = {};
         if (message.${fieldName}) {
           Object.entries(message.${fieldName}).forEach(([k, v]) => {
-            obj.${fieldName}[k] = ${readSnippet('v')};
+            obj.${jsonName}[k] = ${readSnippet('v')};
           });
         }
       `);
@@ -970,9 +1019,9 @@ function generateToJson(ctx, fullName, messageDesc) {
             // Arrays might need their elements transformed
             chunks.push(ts_poet_1.code `
         if (message.${fieldName}) {
-          obj.${fieldName} = message.${fieldName}.map(e => ${readSnippet('e')});
+          obj.${jsonName} = message.${fieldName}.map(e => ${readSnippet('e')});
         } else {
-          obj.${fieldName} = [];
+          obj.${jsonName} = [];
         }
       `);
         }
@@ -980,11 +1029,11 @@ function generateToJson(ctx, fullName, messageDesc) {
             // oneofs in a union are only output as `oneof name = ...`
             const oneofName = case_1.maybeSnakeToCamel(messageDesc.oneofDecl[field.oneofIndex].name, options);
             const v = readSnippet(`message.${oneofName}?.${fieldName}`);
-            chunks.push(ts_poet_1.code `message.${oneofName}?.$case === '${fieldName}' && (obj.${fieldName} = ${v});`);
+            chunks.push(ts_poet_1.code `message.${oneofName}?.$case === '${fieldName}' && (obj.${jsonName} = ${v});`);
         }
         else {
             const v = readSnippet(`message.${fieldName}`);
-            chunks.push(ts_poet_1.code `message.${fieldName} !== undefined && (obj.${fieldName} = ${v});`);
+            chunks.push(ts_poet_1.code `message.${fieldName} !== undefined && (obj.${jsonName} = ${v});`);
         }
     });
     chunks.push(ts_poet_1.code `return obj;`);
@@ -1030,6 +1079,9 @@ function generateFromPartial(ctx, fullName, messageDesc) {
                             return ts_poet_1.code `${cstr}(${from})`;
                         }
                     }
+                    else if (types_1.isAnyValueType(valueType)) {
+                        return ts_poet_1.code `${from}`;
+                    }
                     else if (types_1.isTimestamp(valueType) &&
                         (options.useDate === options_1.DateOption.DATE || options.useDate === options_1.DateOption.STRING)) {
                         return ts_poet_1.code `${from}`;
@@ -1042,6 +1094,9 @@ function generateFromPartial(ctx, fullName, messageDesc) {
                         return ts_poet_1.code `${type}.fromPartial(${from})`;
                     }
                 }
+                else if (types_1.isAnyValueType(field)) {
+                    return ts_poet_1.code `${from}`;
+                }
                 else {
                     const type = types_1.basicTypeName(ctx, field);
                     return ts_poet_1.code `${type}.fromPartial(${from})`;
@@ -1054,17 +1109,16 @@ function generateFromPartial(ctx, fullName, messageDesc) {
         // and then use the snippet to handle repeated fields if necessary
         if (types_1.isRepeated(field)) {
             if (types_1.isMapType(ctx, messageDesc, field)) {
-                chunks.push(ts_poet_1.code `message.${fieldName} = {};`);
-                chunks.push(ts_poet_1.code `if (object.${fieldName} !== undefined && object.${fieldName} !== null) {`);
+                const fieldType = types_1.toTypeName(ctx, messageDesc, field);
                 const i = maybeCastToNumber(ctx, messageDesc, field, 'key');
                 chunks.push(ts_poet_1.code `
-          Object.entries(object.${fieldName}).forEach(([key, value]) => {
+          message.${fieldName} = Object.entries(object.${fieldName} ?? {}).reduce<${fieldType}>((acc, [key, value]) => {
             if (value !== undefined) {
-              message.${fieldName}[${i}] = ${readSnippet('value')};
+              acc[${i}] = ${readSnippet('value')};
             }
-          });
+            return acc;
+          }, {});
         `);
-                chunks.push(ts_poet_1.code `}`);
             }
             else {
                 chunks.push(ts_poet_1.code `
@@ -1104,6 +1158,82 @@ function generateFromPartial(ctx, fullName, messageDesc) {
     chunks.push(ts_poet_1.code `}`);
     return ts_poet_1.joinCode(chunks, { on: '\n' });
 }
+function generateWrap(fullProtoTypeName) {
+    const chunks = [];
+    if (types_1.isStructTypeName(fullProtoTypeName)) {
+        chunks.push(ts_poet_1.code `wrap(object: {[key: string]: any} | undefined): Struct {
+      const struct = Struct.fromPartial({});
+      if (object !== undefined) {
+        Object.keys(object).forEach(key => {
+          struct.fields[key] = object[key];
+        });
+      }
+      return struct;
+    }`);
+    }
+    if (types_1.isAnyValueTypeName(fullProtoTypeName)) {
+        chunks.push(ts_poet_1.code `wrap(value: any): Value {
+      if (value === null) {
+        return {nullValue: 0} as Value;
+      } else if (typeof value === 'boolean') {
+        return {boolValue: value} as Value;
+      } else if (typeof value === 'number') {
+        return {numberValue: value} as Value;
+      } else if (typeof value === 'string') {
+        return {stringValue: value} as Value;
+      } else if (Array.isArray(value)) {
+        return {listValue: value} as Value;
+      } else if (typeof value === 'object') {
+        return {structValue: value} as Value;
+      } else if (typeof value === 'undefined') {
+        return {} as Value;
+      } else {
+        throw new Error('Unsupported any value type: ' + typeof value);
+      }
+    }`);
+    }
+    if (types_1.isListValueTypeName(fullProtoTypeName)) {
+        chunks.push(ts_poet_1.code `wrap(value: Array<any>): ListValue {
+      return {values: value};
+    }`);
+    }
+    return chunks;
+}
+function generateUnwrap(fullProtoTypeName) {
+    const chunks = [];
+    if (types_1.isStructTypeName(fullProtoTypeName)) {
+        chunks.push(ts_poet_1.code `unwrap(message: Struct): {[key: string]: any} {
+      const object: { [key: string]: any } = {};
+      Object.keys(message.fields).forEach(key => {
+        object[key] = message.fields[key];
+      });
+      return object;
+    }`);
+    }
+    if (types_1.isAnyValueTypeName(fullProtoTypeName)) {
+        chunks.push(ts_poet_1.code `unwrap(message: Value): string | number | boolean | Object | null | Array<any> | undefined {
+      if (message.stringValue !== undefined) {
+        return message.stringValue;
+      } else if (message.numberValue !== undefined) {
+        return message.numberValue;
+      } else if (message.boolValue !== undefined) {
+        return message.boolValue;
+      } else if (message.structValue !== undefined) {
+        return message.structValue;
+      } else if (message.listValue !== undefined) {
+          return message.listValue;
+      } else if (message.nullValue !== undefined) {
+        return null;
+      }
+    }`);
+    }
+    if (types_1.isListValueTypeName(fullProtoTypeName)) {
+        chunks.push(ts_poet_1.code `unwrap(message: ListValue): Array<any> {
+      return message.values;
+    }`);
+    }
+    return chunks;
+}
 exports.contextTypeVar = 'Context extends DataLoaders';
 function maybeCastToNumber(ctx, messageDesc, field, variableName) {
     const { keyType } = types_1.detectMapType(ctx, messageDesc, field);

package/build/types.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.detectBatchMethod = exports.responsePromiseOrObservable = exports.responseObservable = exports.responsePromise = exports.responseType = exports.requestType = exports.rawRequestType = exports.detectMapType = exports.toTypeName = exports.getEnumMethod = exports.messageToTypeName = exports.wrapperTypeName = exports.valueTypeName = exports.isEmptyType = exports.isLongValueType = exports.isBytesValueType = exports.isValueType = exports.isTimestamp = exports.isMapType = exports.isLong = exports.isRepeated = exports.isWithinOneOfThatShouldBeUnion = exports.isWithinOneOf = exports.isEnum = exports.isMessage = exports.isBytes = exports.isPrimitive = exports.isScalar = exports.createTypeMap = exports.notDefaultCheck = exports.defaultValue = exports.packedType = exports.toReaderCall = exports.basicTypeName = exports.basicLongWireType = exports.basicWireType = void 0;
+exports.detectBatchMethod = exports.responsePromiseOrObservable = exports.responseObservable = exports.responsePromise = exports.responseType = exports.requestType = exports.rawRequestType = exports.detectMapType = exports.toTypeName = exports.getEnumMethod = exports.messageToTypeName = exports.wrapperTypeName = exports.valueTypeName = exports.isEmptyType = exports.isLongValueType = exports.isStructTypeName = exports.isStructType = exports.isListValueTypeName = exports.isListValueType = exports.isBytesValueType = exports.isAnyValueTypeName = exports.isAnyValueType = exports.isValueType = exports.isTimestamp = exports.isMapType = exports.isLong = exports.isRepeated = exports.isWithinOneOfThatShouldBeUnion = exports.isWithinOneOf = exports.isEnum = exports.isMessage = exports.isBytes = exports.isPrimitive = exports.isScalar = exports.createTypeMap = exports.notDefaultCheck = exports.defaultValue = exports.packedType = exports.toReaderCall = exports.basicTypeName = exports.basicLongWireType = exports.basicWireType = void 0;
 const ts_proto_descriptors_1 = require("ts-proto-descriptors");
 const ts_poet_1 = require("ts-poet");
 const options_1 = require("./options");
@@ -359,10 +359,34 @@ function isValueType(ctx, field) {
     return valueTypeName(ctx, field.typeName) !== undefined;
 }
 exports.isValueType = isValueType;
+function isAnyValueType(field) {
+    return isAnyValueTypeName(field.typeName);
+}
+exports.isAnyValueType = isAnyValueType;
+function isAnyValueTypeName(typeName) {
+    return typeName === 'google.protobuf.Value' || typeName === '.google.protobuf.Value';
+}
+exports.isAnyValueTypeName = isAnyValueTypeName;
 function isBytesValueType(field) {
     return field.typeName === '.google.protobuf.BytesValue';
 }
 exports.isBytesValueType = isBytesValueType;
+function isListValueType(field) {
+    return isListValueTypeName(field.typeName);
+}
+exports.isListValueType = isListValueType;
+function isListValueTypeName(typeName) {
+    return typeName === 'google.protobuf.ListValue' || typeName === '.google.protobuf.ListValue';
+}
+exports.isListValueTypeName = isListValueTypeName;
+function isStructType(field) {
+    return isStructTypeName(field.typeName);
+}
+exports.isStructType = isStructType;
+function isStructTypeName(typeName) {
+    return typeName === 'google.protobuf.Struct' || typeName === '.google.protobuf.Struct';
+}
+exports.isStructTypeName = isStructTypeName;
 function isLongValueType(field) {
     return field.typeName === '.google.protobuf.Int64Value' || field.typeName === '.google.protobuf.UInt64Value';
 }
@@ -388,6 +412,12 @@ function valueTypeName(ctx, typeName) {
             return ts_poet_1.code `boolean`;
         case '.google.protobuf.BytesValue':
             return ts_poet_1.code `Uint8Array`;
+        case '.google.protobuf.ListValue':
+            return ts_poet_1.code `Array<any>`;
+        case '.google.protobuf.Value':
+            return ts_poet_1.code `any`;
+        case '.google.protobuf.Struct':
+            return ts_poet_1.code `{[key: string]: any}`;
         default:
             return undefined;
     }
@@ -404,7 +434,9 @@ function wrapperTypeName(typeName) {
         case '.google.protobuf.UInt64Value':
         case '.google.protobuf.BoolValue':
         case '.google.protobuf.BytesValue':
+        case '.google.protobuf.ListValue':
         case '.google.protobuf.Timestamp':
+        case '.google.protobuf.Struct':
             return typeName.split('.')[3];
         default:
             return undefined;
@@ -525,7 +557,7 @@ function requestType(ctx, methodDesc) {
 }
 exports.requestType = requestType;
 function responseType(ctx, methodDesc) {
-    return messageToTypeName(ctx, methodDesc.outputType);
+    return messageToTypeName(ctx, methodDesc.outputType, { keepValueType: true });
 }
 exports.responseType = responseType;
 function responsePromise(ctx, methodDesc) {

package/build/utils.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.FormattedMethodDescriptor = exports.assertInstanceOf = exports.maybePrefixPackage = exports.prefixDisableLinter = exports.maybeAddComment = exports.upperFirst = exports.lowerFirst = exports.singular = exports.fail = exports.readToBuffer = exports.protoFilesToGenerate = void 0;
+exports.determineFieldJsonName = exports.FormattedMethodDescriptor = exports.assertInstanceOf = exports.maybePrefixPackage = exports.prefixDisableLinter = exports.maybeAddComment = exports.upperFirst = exports.lowerFirst = exports.singular = exports.fail = exports.readToBuffer = exports.protoFilesToGenerate = void 0;
 const ts_poet_1 = require("ts-poet");
 const options_1 = require("./options");
 const case_1 = require("./case");
@@ -144,3 +144,12 @@ class FormattedMethodDescriptor {
     }
 }
 exports.FormattedMethodDescriptor = FormattedMethodDescriptor;
+function determineFieldJsonName(field, options) {
+    // By default jsonName is camelCased by the protocol compilier unless the user has
+    // set a "json_name" option on this field.
+    if (field.jsonName.length > 0) {
+        return field.jsonName;
+    }
+    return case_1.maybeSnakeToCamel(field.name, options);
+}
+exports.determineFieldJsonName = determineFieldJsonName;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ts-proto",
-  "version": "1.87.1",
+  "version": "1.90.1",
   "description": "",
   "main": "build/plugin.js",
   "repository": "github:stephenh/ts-proto",
@@ -25,9 +25,9 @@
     "@grpc/proto-loader": "^0.5.6",
     "@improbable-eng/grpc-web": "^0.14.0",
     "@improbable-eng/grpc-web-node-http-transport": "^0.14.0",
-    "@nestjs/common": "^7.6.15",
-    "@nestjs/core": "^7.6.15",
-    "@nestjs/microservices": "^7.6.15",
+    "@nestjs/common": "^8.2.2",
+    "@nestjs/core": "^8.2.2",
+    "@nestjs/microservices": "^8.2.2",
     "@semantic-release/changelog": "^5.0.1",
     "@semantic-release/commit-analyzer": "^8.0.1",
     "@semantic-release/git": "^9.0.0",
@@ -36,11 +36,10 @@
     "@semantic-release/release-notes-generator": "^9.0.2",
     "@types/jest": "^26.0.22",
     "@types/node": "^14.14.37",
-    "grpc": "^1.24.6",
     "jest": "^26.6.3",
     "prettier": "^2.2.1",
     "reflect-metadata": "^0.1.13",
-    "rxjs": "^6.6.7",
+    "rxjs": "^7.4.0",
     "semantic-release": "^17.4.2",
     "ts-jest": "^26.5.4",
     "ts-node": "^9.1.1",