@jayesol/jayeson.lib.delivery 2.0.6 → 2.0.7

package/README.md

@@ -1,96 +1,117 @@
-v 1.4.1
- - Use StreamFinder 1.6.0: Supports advertisements with paths.
- - New message group for Name Feed: group 30.
- - Prevent streams from getting excluded more than once by having only one ReconnectClientTask.
- - Not to attempt to load truststore if truststores are not provided in the SSL config.
-
-v 1.3.9
- - Use StreamFinder 1.4.1 : Sends advertisements containing all unadvertised streams instead of one advertisement per stream.
- - Use StreamFinder 1.5.0 : Split streams inside an advertisement into chunks of 100.
- - Fix memory leak of event loop not garbage collected if connection fails by shutting down the event loop.
- - Fix guice circular dependency between AuthService and AuthGroupProcessor.
- - Fix reconnection missing streams issue #18763.
- - Fix relayer oom issue #19366.
- - Fix disconnection from stream by not sending deregistration request twice #19650
- - Created ConsumptionStopEvent class in Java. An instance will be sent upon successful deregistration from stream.
- - Fixed not able to reconnect to redis server as a result of a change in redis server ip.
- - Added keepalive sending from subscriber to publisher and made sending of keepalive from publisher to subscriber be configurable and default to false.
-
-v 1.2.0
- - Use StreamFinder 1.3.0 : Discovering with scope information
- - TypeScript Support for delivery
- - Improved keep-aliv sending. Will only send if there are no other messages sent
- - Bug fixes involving release of reference counted messages, AuthMessage time out. **Config change:** Move `authMsgTimeout` to `authServiceConfig`
- - StreamStopMessage implementation. PE can explictly tell subscriber that it is stopping a stream and subscriber can failover. Refer to wiki for more information
-
-v 1.1.2
- - Use StreamFinder 1.2.3
- - Use Netty 4.0.36.Final
- - Support multiple scopes in subscriber(**Update subscriber config**.Will not work with older config)
- - Fixed Reconnection bug in Subscriber where reconnection trying stops after some retries
- - Remove StreamRegistry Group Processing from Eventloop.(No change is needed in PE)
-    - To Describing PE callback behavoiur more explcitly:
-        - PE should expect messages from an endpoint AFTER call of actOnStreamRegistrationRequest()
-        - PE should initiate sending messages ONLY afte call of startPublishing()
-- Subscriber now dont make unnecessary connection to StreamFinder if application is not using subscriber module(i.e If no discovery streams in config).
-- Exposed default handlers in JsonBaseMessageClass. Applications can now extend JSonBaseMessageClass to do custom transformation of message within pipeline itself. E:g [here](https://gitlab.jayeson.com.sg/snippets/7) 
-- Added Abstrations for using with jruby
-- Minor bug fixes and improved logging  
-
-------------------------------------------------------------------------
-
-### Registered MessageGroups. Please register your message group before using on common infrastructure
-
-| Group                         | Byte |
-| ------------------------------- | ------ |
-| ObscureGroup(used internally) | 0    |
-| StreamRegistryGroup           | 2    |
-| ConnectionCheckingGroup(keep alive)  | 4    |  
-| Authentication Group          | 5    |
-| GenericMessageGroup           | 6    |
-| OrderFeed                     |11    |
-| MinMax Feed                   |12|
-| Vodds Config Feed              |13|
-| Soccer Results Feed            |14|
-| Sports Results Feed            |15|
-| Name Feed                      |30|
-| SoccerFeed                     |69|
-| Ping Pong Feed(Demo Example)   |71|
-| Sports Feed                    |70|
-
-
-### Develeoper Notes
-
-We follow google java style guide  
-https://google.github.io/styleguide/javaguide.html  
-
-Few Points to take note(Copied from style guide ):  
-
-4.1 Braces   
-
-4.1.1 Braces are used where optional   
-
-Braces are used with if, else, for, do and while statements, even when the body is empty or contains only a single statement.  
-
-4.1.2 Nonempty blocks: K & R style    
-
-Braces follow the Kernighan and Ritchie style ("Egyptian brackets") for nonempty blocks and block-like constructs:  
-
-No line break before the opening brace.  
-Line break after the opening brace.  
-Line break before the closing brace.  
-Line break after the closing brace if that brace terminates a statement or the body of a method, constructor or named class. For example, there is no line break after the brace if it is followed by else or a comma.
-
-Example:  
-
-```return new MyClass() {
-  @Override public void method() {
-    if (condition()) {
-      try {
-        something();
-      } catch (ProblemException e) {
-        recover();
-      }
-    }
-  }
-};```
+v 1.6.0
+ - Use netty 4.1 for library to be usable with Java 11.
+ - Added stop consumption feature for delivery client to be disconnected from server.
+
+v 1.4.1
+ - Use StreamFinder 1.6.0: Supports advertisements with paths.
+ - New message group for Name Feed: group 30.
+ - Prevent streams from getting excluded more than once by having only one ReconnectClientTask.
+ - Not to attempt to load truststore if truststores are not provided in the SSL config.
+
+v 1.3.9
+ - Use StreamFinder 1.4.1 : Sends advertisements containing all unadvertised streams instead of one advertisement per stream.
+ - Use StreamFinder 1.5.0 : Split streams inside an advertisement into chunks of 100.
+ - Fix memory leak of event loop not garbage collected if connection fails by shutting down the event loop.
+ - Fix guice circular dependency between AuthService and AuthGroupProcessor.
+ - Fix reconnection missing streams issue #18763.
+ - Fix relayer oom issue #19366.
+ - Fix disconnection from stream by not sending deregistration request twice #19650
+ - Created ConsumptionStopEvent class in Java. An instance will be sent upon successful deregistration from stream.
+ - Fixed not able to reconnect to redis server as a result of a change in redis server ip.
+ - Added keepalive sending from subscriber to publisher and made sending of keepalive from publisher to subscriber be configurable and default to false.
+
+v 1.2.0
+ - Use StreamFinder 1.3.0 : Discovering with scope information
+ - TypeScript Support for delivery
+ - Improved keep-aliv sending. Will only send if there are no other messages sent
+ - Bug fixes involving release of reference counted messages, AuthMessage time out. **Config change:** Move `authMsgTimeout` to `authServiceConfig`
+ - StreamStopMessage implementation. PE can explictly tell subscriber that it is stopping a stream and subscriber can failover. Refer to wiki for more information
+
+v 1.1.2
+ - Use StreamFinder 1.2.3
+ - Use Netty 4.0.36.Final
+ - Support multiple scopes in subscriber(**Update subscriber config**.Will not work with older config)
+ - Fixed Reconnection bug in Subscriber where reconnection trying stops after some retries
+ - Remove StreamRegistry Group Processing from Eventloop.(No change is needed in PE)
+    - To Describing PE callback behavoiur more explcitly:
+        - PE should expect messages from an endpoint AFTER call of actOnStreamRegistrationRequest()
+        - PE should initiate sending messages ONLY afte call of startPublishing()
+- Subscriber now dont make unnecessary connection to StreamFinder if application is not using subscriber module(i.e If no discovery streams in config).
+- Exposed default handlers in JsonBaseMessageClass. Applications can now extend JSonBaseMessageClass to do custom transformation of message within pipeline itself. E:g [here](https://gitlab.jayeson.com.sg/snippets/7) 
+- Added Abstrations for using with jruby
+- Minor bug fixes and improved logging  
+
+------------------------------------------------------------------------
+
+### Registered MessageGroups. Please register your message group before using on common infrastructure
+
+| Group                         | Byte |
+| ------------------------------- | ------ |
+| ObscureGroup(used internally) | 0    |
+| StreamRegistryGroup           | 2    |
+| ConnectionCheckingGroup(keep alive)  | 4    |  
+| Authentication Group          | 5    |
+| GenericMessageGroup           | 6    |
+| OrderFeed                     |11    |
+| MinMax Feed                   |12|
+| Vodds Config Feed              |13|
+| Soccer Results Feed            |14|
+| Sports Results Feed            |15|
+| Name Feed                      |30|
+| SoccerFeed                     |69|
+| Ping Pong Feed(Demo Example)   |71|
+| Sports Feed                    |70|
+
+
+### Develeoper Notes
+
+We follow google java style guide  
+https://google.github.io/styleguide/javaguide.html  
+
+Few Points to take note(Copied from style guide ):  
+
+4.1 Braces   
+
+4.1.1 Braces are used where optional   
+
+Braces are used with if, else, for, do and while statements, even when the body is empty or contains only a single statement.  
+
+4.1.2 Nonempty blocks: K & R style    
+
+Braces follow the Kernighan and Ritchie style ("Egyptian brackets") for nonempty blocks and block-like constructs:  
+
+No line break before the opening brace.  
+Line break after the opening brace.  
+Line break before the closing brace.  
+Line break after the closing brace if that brace terminates a statement or the body of a method, constructor or named class. For example, there is no line break after the brace if it is followed by else or a comma.
+
+Example:  
+
+```return new MyClass() {
+  @Override public void method() {
+    if (condition()) {
+      try {
+        something();
+      } catch (ProblemException e) {
+        recover();
+      }
+    }
+  }
+};```
+``````
+
+### CI/CD guideline 
+
+#### Publish
+
+Guideline: [CI/CD guideline](https://docs.jayeson.com.sg/ops/operations/gitlab-cicd/usage#how-to-use)
+
+- Publish for non prod repo:
+  ```
+     publish: snapshot 
+  ```
+
+- Publish for production repo:
+  ```
+     publish: prod
+  ```

package/lib/CoreMessages.js

@@ -1 +1,248 @@
-var util=require("./util"),_c=util._c,_=require("underscore"),DeliveryStack=require("./DeliveryStack"),IMessageGroup=DeliveryStack.IMessageGroup,IMessageClass=DeliveryStack.IMessageClass;function EmptyMessageClass(e,s){IMessageClass.call(this,e,s,String),this.inH={unpack:function(e){void 0!==e.message&&null!==e.message||(e.message="")}},this.outH={pack:function(e){e.message=[]}}}function StringMessageClass(e,s){EmptyMessageClass.call(this,e,s),this.inH={unpack:function(e){for(var s="",t=0;t<e.message.length;t++)s+=String.fromCharCode(e.message[t]);e.message=s}},this.outH={pack:function(e){for(var s=[],t=0;t<e.message.length;t++)s.push(e.message.charCodeAt(t));e.message=s}}}function JSonMessageClass(e,s,n){IMessageClass.call(this,e,s,String),this.inH={unpack:function(e){for(var s="",t=0;t<e.message.length;t++)s+=String.fromCharCode(e.message[t]);var a=JSON.parse(s),i=new n;_.extend(i,a),e.message=i}},this.outH={pack:function(e){for(var s=JSON.stringify(e.message),t=[],a=0;a<s.length;a++)t.push(s.charCodeAt(a));e.message=t}}}function GenericMessageGroup(){IMessageGroup.call(this,6),this._classes=[new StringMessageClass(this,0)]}function KeepAliveMessageGroup(){IMessageGroup.call(this,4),this._classes=[new EmptyMessageClass(this,0)]}function GeneralResponse(e,s){this.status=e,this.message=s}function TicketRenew(e,s){this.clientId=e,this.newTicket=s}function AuthGroup(){IMessageGroup.call(this,5),this.GENERAL_RESPONSE=new JSonMessageClass(this,0,GeneralResponse),this.HTTP_AUTH=new EmptyMessageClass(this,1),this.TCP_AUTH=new EmptyMessageClass(this,2),this.TICKET_RENEW=new JSonMessageClass(this,3,TicketRenew),this.STRING=new StringMessageClass(this,4),this._classes=[this.GENERAL_RESPONSE,this.HTTP_AUTH,this.TCP_AUTH,this.TICKET_RENEW,this.STRING]}function StreamRegistryRequest(e,s,t){this.messageGroup=e,this.streams=s,this.requestType=t}function StreamRegistryResponse(e,s,t){this.messageGroup=e,this.streams=s,this.responseType=t}function StreamRegistryMessageGroup(){IMessageGroup.call(this,2),this.REQUEST=new JSonMessageClass(this,0,StreamRegistryRequest),this.RESPONSE=new JSonMessageClass(this,1,StreamRegistryResponse),this._classes=[this.REQUEST,this.RESPONSE]}_c(EmptyMessageClass,{inHandlers:function(){return[this.inH]},outHandlers:function(){return[this.outH]}},IMessageClass),_c(StringMessageClass,{outHandlers:function(){return[this.outH]}},EmptyMessageClass),_c(JSonMessageClass,{inHandlers:function(){return[this.inH]},outHandlers:function(){return[this.outH]}},IMessageClass),_c(GenericMessageGroup,{},IMessageGroup),GenericMessageGroup=new GenericMessageGroup,_c(KeepAliveMessageGroup,{},IMessageGroup),KeepAliveMessageGroup=new KeepAliveMessageGroup,_c(GeneralResponse,{getStatus:function(){return this.status},getMessage:function(){return this.message}}),_c(GeneralResponse,{getClientId:function(){return this.clientId},getNewTicket:function(){return this.newTicket}}),_c(AuthGroup,{},IMessageGroup),AuthGroup=new AuthGroup,_.extend(StreamRegistryRequest,{REGISTRATION:0,CONSUMPTION:1,DEREGISTRATION:2}),_.extend(StreamRegistryResponse,{REGISTRATION:0,CONSUMPTION:1,STOP_CONSUMPTION:6,DEREGISTRATION:2,GENERAL:3,SUCCESS:4,FAILED:5,DEREGISTRATION_TIMEOUT:7}),_c(StreamRegistryMessageGroup,{},IMessageGroup),StreamRegistryMessageGroup=new StreamRegistryMessageGroup,module.exports=exports={StreamRegistryMessageGroup:StreamRegistryMessageGroup,EmptyMessageClass:EmptyMessageClass,StringMessageClass:StringMessageClass,JSonMessageClass:JSonMessageClass,StreamRegistryRequest:StreamRegistryRequest,StreamRegistryResponse:StreamRegistryResponse,AuthGroup:AuthGroup,GenericMessageGroup:GenericMessageGroup,KeepAliveMessageGroup:KeepAliveMessageGroup,TicketRenew:TicketRenew};
+var util = require("./util")
+var _c = util._c
+var _ = require("underscore")
+var DeliveryStack = require("./DeliveryStack")
+var IMessageGroup = DeliveryStack.IMessageGroup
+var IMessageClass = DeliveryStack.IMessageClass
+
+/**
+ * This javascript file define some default message group and classes
+ * That is commonly used
+ */
+
+/****************** MESSAGE CLASS DEFINITIONS *******************/
+/**
+ * Empty Message Class
+ */
+function EmptyMessageClass(group, id) {
+	IMessageClass.call(this, group, id, String);
+	this.inH = {
+		unpack: function (wrapper) {	// this unpack method convert undefined to "" for easy handling at processor / hook
+			if (wrapper.message === undefined || wrapper.message === null) wrapper.message = "";
+		}
+	};
+
+	this.outH = {
+		pack: function (wrapper) {
+			// in whatever case, the message sent in an empty message should 
+			// be encoded into an empty byte array, regardless of what the user set
+			wrapper.message = [];
+		}
+	}
+}
+_c(EmptyMessageClass, {
+	inHandlers: function () {	// in handlers of the empty message class does nothing
+		return [this.inH];
+	},
+
+	outHandlers: function () {
+		return [this.outH];
+	}
+}, IMessageClass);
+
+/**
+ * String message class, decode (encode) from  byte[] (string) to string (byte[])
+ * @param group
+ * @param id
+ */
+function StringMessageClass(group, id) {
+	EmptyMessageClass.call(this, group, id);
+	this.inH = {
+		unpack: function (wrapper) {
+			var msg = "";
+			for (var i = 0; i < wrapper.message.length; i++) {
+				msg += String.fromCharCode(wrapper.message[i]);
+			}
+			wrapper.message = msg;
+		}
+	}
+
+	this.outH = {
+		pack: function (wrapper) {
+			var result = [];
+			for (var i = 0; i < wrapper.message.length; i++) {
+				result.push(wrapper.message.charCodeAt(i));
+			}
+			wrapper.message = result;
+		}
+	}
+}
+_c(StringMessageClass, {
+
+	outHandlers: function () {
+		return [this.outH];
+	}
+
+}, EmptyMessageClass);
+
+function JSonMessageClass(group, id, instanceClass) {
+	IMessageClass.call(this, group, id, String);
+
+	this.inH = {
+		unpack: function (wrapper) {
+			// convert from byte[] to json object
+			var msg = "";
+			for (var i = 0; i < wrapper.message.length; i++) {
+				msg += String.fromCharCode(wrapper.message[i]);
+			}
+			var obj = JSON.parse(msg);	// convert into a message
+			var actual = new instanceClass();
+			_.extend(actual, obj);		// copy data into the object of the correct type
+
+			wrapper.message = actual;
+		}
+	};
+
+	this.outH = {
+		pack: function (wrapper) {
+			var msg = JSON.stringify(wrapper.message);	// convert original message to string
+			var result = [];
+			for (var i = 0; i < msg.length; i++) {
+				result.push(msg.charCodeAt(i));
+			}
+
+			wrapper.message = result;					// update wrapper
+		}
+	};
+}
+_c(JSonMessageClass, {
+	inHandlers: function () {
+		return [this.inH];
+	},
+
+	outHandlers: function () {
+		return [this.outH];
+	}
+}, IMessageClass);
+
+/******************** CORE MESSAGE GROUP DEFINITIONS ****************/
+
+function GenericMessageGroup() {
+	IMessageGroup.call(this, 6);
+	this._classes = [
+		new StringMessageClass(this, 0)
+	];
+};
+_c(GenericMessageGroup, {}, IMessageGroup);
+GenericMessageGroup = new GenericMessageGroup();		// singleton
+
+function KeepAliveMessageGroup() {
+	IMessageGroup.call(this, 4);
+	this._classes = [
+		new EmptyMessageClass(this, 0)
+	];
+};
+_c(KeepAliveMessageGroup, {}, IMessageGroup);
+KeepAliveMessageGroup = new KeepAliveMessageGroup();	// singleton
+
+/**************** AUTH MESSAGE GROUP DEFINITION **********************/
+
+// bean definition for the general response
+function GeneralResponse(status, message) {
+	this.status = status;
+	this.message = message;
+}
+_c(GeneralResponse, {
+	getStatus: function () { return this.status; },
+	getMessage: function () { return this.message; }
+});
+
+//bean definition for the ticket renew
+function TicketRenew(clientId, newTicket) {
+	this.clientId = clientId;
+	this.newTicket = newTicket;
+}
+_c(GeneralResponse, {
+	getClientId: function () { return this.clientId; },
+	getNewTicket: function () { return this.newTicket; }
+});
+
+function AuthGroup() {
+	IMessageGroup.call(this, 5);
+
+	this.GENERAL_RESPONSE = new JSonMessageClass(this, 0, GeneralResponse);
+	this.HTTP_AUTH = new EmptyMessageClass(this, 1);
+	this.TCP_AUTH = new EmptyMessageClass(this, 2);
+	this.TICKET_RENEW = new JSonMessageClass(this, 3, TicketRenew);
+	this.STRING = new StringMessageClass(this, 4);
+
+	this._classes = [
+		this.GENERAL_RESPONSE,
+		// this is corresponding HTTP_AUTH_CONTENT, since we are sending auth content
+		// using XHR we don't need to define any handlers for this message class
+		this.HTTP_AUTH,
+		// this correspond to tcp auth content. We are not sending auth content through tcp,
+		// so ignore this
+		this.TCP_AUTH,
+		// ticket renew
+		this.TICKET_RENEW,
+		// generic string
+		this.STRING
+
+
+	];
+}
+_c(AuthGroup, {}, IMessageGroup);
+AuthGroup = new AuthGroup();	// singleton
+
+/**************** STREAM REGISTRATION GROUP **********************/
+function StreamRegistryRequest(messageGroup, streams, requestType) {
+	this.messageGroup = messageGroup;
+	this.streams = streams;
+	this.requestType = requestType;
+}
+_.extend(StreamRegistryRequest, {
+	REGISTRATION: 0,
+	CONSUMPTION: 1,
+	DEREGISTRATION: 2
+});
+
+/**
+ * It is guaranteed by the stream registry service that if I sent a registration message
+ * for Group A Stream X, Y, Z, the response will contains the information for all the stream X, Y, Z.
+ * 
+ * @param: messageGroup - the message group i'm registered for
+ * @param: streams - a map from stream name to the registration status, can be SUCCESS (4) or FAILED (5)
+ * @param: responseType - type of the response, can be REGISTRATION, CONSUMPTION, DEREGISTRATION
+ *
+ */
+function StreamRegistryResponse(messageGroup, streams, responseType) {
+	this.messageGroup = messageGroup;
+	this.streams = streams;
+	this.responseType = responseType;
+}
+_.extend(StreamRegistryResponse, {
+	REGISTRATION: 0,
+	CONSUMPTION: 1,
+	STOP_CONSUMPTION: 6,
+	DEREGISTRATION: 2,
+	GENERAL: 3,
+
+	SUCCESS: 4,
+	FAILED: 5,
+	DEREGISTRATION_TIMEOUT: 7
+});
+
+function StreamRegistryMessageGroup() {
+	IMessageGroup.call(this, 2);
+
+	this.REQUEST = new JSonMessageClass(this, 0, StreamRegistryRequest);
+	this.RESPONSE = new JSonMessageClass(this, 1, StreamRegistryResponse);
+
+	this._classes = [this.REQUEST, this.RESPONSE];
+}
+_c(StreamRegistryMessageGroup, {}, IMessageGroup);					// inherit from IMessageGroup
+StreamRegistryMessageGroup = new StreamRegistryMessageGroup();		// singleton
+
+module.exports = exports = {
+	StreamRegistryMessageGroup: StreamRegistryMessageGroup,
+	EmptyMessageClass: EmptyMessageClass,
+	StringMessageClass: StringMessageClass,
+	JSonMessageClass: JSonMessageClass,
+	StreamRegistryRequest: StreamRegistryRequest,
+	StreamRegistryResponse: StreamRegistryResponse,
+	AuthGroup: AuthGroup,
+	GenericMessageGroup: GenericMessageGroup,
+	KeepAliveMessageGroup: KeepAliveMessageGroup,
+	TicketRenew: TicketRenew
+}