iotagent-node-lib 4.5.0 → 4.6.0

package/doc/deprecated.md

@@ -25,6 +25,9 @@ A list of deprecated features and the version in which they were deprecated foll
 -   Support to legacy expressions (finally removed in 3.2.0)
 -   Bidirectinal pluging (finally removed in 3.4.0)
 -   appendMode configuration (`IOTA_APPEND_MODE` env var) (finally removed in 3.4.0)
+-   `config.stats` section, and push-mode statistics.
+-   Services API routes (`/iot/services`) in favor of the `/iot/groups`. Both are still supported, but the former is
+    deprecated.
 
 The use of Node.js v14 is highly recommended.
 
@@ -57,3 +60,4 @@ The following table provides information about the last iotagent-node-lib versio
 | Support to Legacy Expressions                         | 3.1.0                                                 | April 25th, 2023              |
 | bidirectional plugin                                  | 3.3.0                                                 | August 24th, 2023             |
 | appendMode configuration (`IOTA_APPEND_MODE` env var) | 3.3.0                                                 | August 24th, 2023             |
+| push-mode stats                                       | 4.5.0                                                 | June 11th, 2024               |

package/doc/devel/architecture.md

@@ -4,31 +4,6 @@ The following section defines the architecture and message flow which is common
 
 ### Device to NGSI Mapping
 
-Each Device will be mapped as an Entity associated to a Context Provider: the Device ID will be mapped by default to the
-entity ID and the type of the entity will be selected by the IoT Agent in a protocol-dependent way (e.g: with different
-URLs for different types). Both the name and type will be configurable by the user, either by type configuration or with
-the device preprovisioning.
-
-Each of the measures obtained from the device should be mapped to a different attribute. The name and type of the
-attribute will be configured by the user (globally for all the types in the IoT Agent configuration or in a per device
-basis preprovisioning the devices). Device measures can have three different behaviors:
-
--   **Active attributes**: are measures that are pushed from the device to the IoT agent. This measure changes will be
-    sent to the Context Broker as updateContext requests over the device entity. NGSI queries to the context broker will
-    be resolved in the Broker database.
-
--   **Lazy attributes**: some sensors will be passive, and will wait for the IoT Agent to request for data. For those
-    measures, the IoT Agent will register itself in the Context Broker as a Context Provider (for all the lazy measures
-    of that device), so if any component asks the Context Broker for the value of that sensor, its request will be
-    redirected to the IoT Agent (that behaves as a NGSI10 Context Provider). This operation will be synchronous from the
-    customer perspective: the Context Broker won't return a response until the device has returned its response to the
-    IoT Agent.
-
--   **Commands**: in this case, the interaction will begin by setting an attribute in the device's entity, for which the
-    IoT Agent will be regitered as CP. The IoT Agent will return an immediate response to the Context Broker, and will
-    be held responsible of contacting the device to perform the command itself, updating special `status` and `info`
-    attributes in the entity as soon as it has any information of the command progress.
-
 The following sequence diagram shows the different NGSI interactions an IoT Agent makes with the Context Broker,
 explained in the following subsections (using the example of a OMA Lightweight M2M device).
 
@@ -38,7 +13,7 @@ Be aware that the IoT Agents are only required to support NGSI10 operations `upd
 standard formats (currently in JSON format; XML deprecated) but will not answer to NGSI9 operations (or NGSI convenience
 operations of any kind).
 
-#### Configurations and Device provisioning information
+#### Config groups and Device provisioning information
 
 In order for a device to connect to the IoT Agent, the device should be provisioned (although there may be occasions
 where this registration is not needed). The provision process is meant to provide the IoT Agent with the following
@@ -58,27 +33,15 @@ information:
 In order to provide this information, the IoT Agent Northbound API provides two resources: Device and Configuration
 provisioning.
 
-**Configurations** may be used when a set of similar devices will be connected to the IoT Agent, to avoid provisioning
-the same set of information for every device. Custom APIKeys can be only provided with the use of Configurations for
+**Config groups** may be used when a set of similar devices will be connected to the IoT Agent, to avoid provisioning
+the same set of information for every device. Custom APIKeys can be only provided with the use of config group for
 device groups. When a device is provisioned, it is assigned to a configuration _if there is one that matches its type,
 its service and its subservice_. In that case, all the default information in the Configuration is merged with the
 device information to create the definitive Device object that will be stored in the system.
 
-Particular IoT Agents _may_ support autoregistration of devices into configurations, if enough information is given from
+Particular IoT Agents _may_ support autoregistration of devices into config groups, if enough information is given from
 the Southbound.
 
-#### Configurations and subservices
-
-Configurations are meant to be a mean of simplifying the device provisioning for groups of very similar devices.
-Considering that different groups of devices may be created in the same subservice that may require different
-configurations, multiple configurations are allowed for each subservice. Considering the key in the association between
-Device and Configuration was the triplet (service, subservice, type), all of these elements are considered mandatory.
-
-This statement doesn't hold true for older IoT Agents, though. In older versions of the IoT Agents, each device
-configuration was assigned to a particular subservice and just one configuration was allowed per subservice, so the
-relation between a Device and a Configuration didn't need the type to discriminate between Configurations. That's why
-for those agents, type was not a mandatory parameter.
-
 #### Registration
 
 Whenever a device is registered, the IoT Agent reads the device's entity information from the request or, if that
@@ -103,7 +66,7 @@ response to the caller, transparently.
 
 **IMPORTANT NOTE:** at the present moment, commands (both push and poll) are supported only in the case of explicitly
 provisioned agents. For autoprovisioned agents commands are not currently supported, although
-[an issue](https://github.com/telefonicaid/iot-agent-node-lib/issues/572) has been created about this functionality.
+[an issue](https://github.com/telefonicaid/iotagent-node-lib/issues/572) has been created about this functionality.
 
 Commands are modelled as updates over a lazy attribute. As in the case of the lazy attributes, updates over a command
 will be forwarded by the Context Broker to the IoT Agent, that will in turn interact with the device to perform the
@@ -153,52 +116,6 @@ taking too much to pick them up. See the configuration section for details.
 The library does not deal with protocol transformation or South Bound communications for neither of the command types
 (that's the task for those specific IoT Agents using the library).
 
-##### NGSI-LD `datasetId` support
-
-Limited support for parsing the NGSI-LD `datasetId` attribute is included within the library. A series of sequential
-commands for a single attribute can be sent as an NGSI-LD PATCH payload as follows:
-
-```json
-{
-    "lampColor": [
-        {
-            "type": "Property",
-            "value": { "color": "green", "duration": "55 secs" },
-            "datasetId": "urn:ngsi-ld:Sequence:do-this"
-        },
-        {
-            "type": "Property",
-            "value": { "color": "red", "duration": "10 secs" },
-            "datasetId": "urn:ngsi-ld:Sequence:then-do-this"
-        }
-    ]
-}
-```
-
-This results in the following sequential array of attribute updates to be sent to the `UpdateHandler` of the IoT Agent
-itself:
-
-```json
-[
-    {
-        "name": "lampColor",
-        "type": "Property",
-        "datasetId": "urn:ngsi-ld:Sequence:do-this",
-        "metadata": {},
-        "value": { "color": "green", "duration": "55 secs" }
-    },
-    {
-        "name": "lampColor",
-        "type": "Property",
-        "datasetId": "urn:ngsi-ld:Sequence:then-do-this",
-        "metadata": {},
-        "value": { "color": "red", "duration": "10 secs" }
-    }
-]
-```
-
-The equivalent for NGSI-v2 is not required since `datasetId` syntax is not supported by NGSI-v2.
-
 #### Active attributes
 
 Whenever a device proactively sends a message to the IoT Agent, it should transform its data to the appropriate NGSI
@@ -243,50 +160,3 @@ The following figure offers a graphical example of how a COAP IoT Agent work, or
 device to a command update to the device.
 
 ![General ](../img/iotAgentLib.png 'Architecture Overview')
-
-### The `TimeInstant` element
-
-As part of the device to entity mapping process the IoT Agent creates and updates automatically a special timestamp.
-This timestamp is represented as two different properties of the mapped entity::
-
--   With NGSI-v2, an attribute metadata named `TimeInstant` (per dynamic attribute mapped, which captures as an ISO8601
-    timestamp when the associated measurement (represented as attribute value) was observed. With NGSI-LD, the Standard
-    `observedAt` property-of-a-property is used instead.
-
--   For NGSI-v2 only, an additional attribute `TimeInstant` is added to the entity which captures as an ISO8601
-    timestamp when the last measurement received from the device was observed.
-
-If no information about the measurement timestamp is received by the IoT Agent, the arrival time of the measurement will
-be used to generate a `TimeInstant` for both the entity attribute and the attribute metadata.
-
-Take into account that:
-
--   the timestamp of different attributes belonging to the same measurement record may not be equal.
--   the arrival time and the measurement timestamp will not be the same in the general case.
--   if `timezone` field is defined as part of the provisioning of the device or group, timestamp fields will be
-    generated using it. For instance, if `timezone` is set to `America/Los_Angeles`, a possible timestamp could be
-    `2025-08-05T00:35:01.468-07:00`. If `timezone` field is not defined, by default Zulu Time Zone (UTC +0) will be
-    used. Following the previous example, timestamp could be `2015-08-05T07:35:01.468Z`.
-
-E.g.: in the case of a device that can take measurements every hour of both temperature and humidity and sends the data
-once every day, at midnight, the `TimeInstant` reported for each measurement will be the hour when that measurement was
-observed (e.g. 4:00 PM), while all the measurements will have an arrival time around midnight. If no timestamps were
-reported with such measurements, the `TimeInstant` attribute would take those values around midnight.
-
-This functionality can be turned on and off through the use of the `timestamp` configuration flag (described in the
-configuration), as well as 'timestamp' flag in device or group provision.
-
-### Implementation decisions
-
-Given the aforementioned requirements, there are some aspects of the implementation that were chosen, and are
-particularly under consideration:
-
--   Aside from its configuration, the IoT Agent Lib is considered to be stateless. To be precise, the library mantains a
-    state (the list of entities/devices whose information the agent can provide) but that state is considered to be
-    transient. It's up to the particular implementation of the agent to consider whether it should have a persistent
-    storage to hold the device information (so the internal list of devices is read from a DB) or to register the
-    devices each time a device sends a measure. To this extent, two flavours of the Device Registry has been provided: a
-    transient one (In-memory Registry) and a persistent one (based in MongoDB).
--   The IoT Agent does not care about the origin of the data, its type or structure. The mapping from raw data to the
-    entity model, if there is any, is a responsibility of the particular IoT Agent implementation, or of another third
-    party library.

package/doc/devel/development.md

@@ -24,7 +24,7 @@
     -   [Function reference](#function-reference)
         -   [Generic middlewares](#generic-middlewares)
 -   [DB Models from API document](#db-models-from-api-document)
-    -   [Service group model](#service-group-model)
+    -   [Config group model](#config-group-model)
     -   [Device model](#device-model)
 -   [Developing a new IoT Agent](#developing-a-new-iot-agent)
     -   [Protocol](#protocol)
@@ -37,6 +37,7 @@
     -   [IoT Agent in multi-thread mode](#iot-agent-in-multi-thread-mode)
     -   [Configuration management](#configuration-management)
         -   [Provisioning handlers](#provisioning-handlers)
+-   [IoT Agent additional tools](#iot-agent-additional-tools)
 
 ## Preface
 
@@ -226,18 +227,20 @@ npm run prettier:text
 
 ### Stats Registry
 
-The library provides a mechanism for the periodic reporting of stats related to the library's work. In order to activate
-the use of the periodic stats, it must be configured in the config file, as described in the
-[Configuration](../admin.md#configuration) section.
-
-The Stats Registry holds two dictionaries, with the same set of stats. For each stat, one of the dictionaries holds the
-historical global value and the other one stores the value since the last value reporting (or current value).
+The library provides a mechanism for the collection of stats related to the library's work. The Stats Registry holds a
+dictionary with the historical global value of each stat.
 
 The stats library currently stores only the following values:
 
 -   **deviceCreationRequests**: number of Device Creation Requests that arrived to the API (no matter the result).
 -   **deviceRemovalRequests**: number of Removal Device Requests that arrived to the API (no matter the result).
 -   **measureRequests**: number of times the ngsiService.update() function has been invoked (no matter the result).
+-   **raiseAlarm**: number of times the alarmManagement.raise() function has been invoked.
+-   **releaseAlarm**: number of times the alarmManagement.release() function has been invoked.
+-   **updateEntityRequestsOk**: number of times the ngsiService.sendUpdateValue() function has been invoked
+    successfully.
+-   **updateEntityRequestsError**: number of times the ngsiService.sendUpdateValue() function has been invoked and
+    failed.
 
 More values will be added in the future to the library. The applications using the library can add values to the Stats
 Registry just by using the following function:
@@ -247,7 +250,7 @@ iotagentLib.statsRegistry.add('statName', statIncrementalValue, callback);
 ```
 
 The first time this function is invoked, it will add the new stat to the registry. Subsequent calls will add the value
-to the specified stat both to the current and global measures. The stat will be cleared in each interval as usual.
+to the specified stat.
 
 ### Alarm module
 
@@ -753,9 +756,9 @@ The `newConfiguration` parameter will contain the newly created configuration. T
 callback with no parameters (this handler should only be used for reconfiguration purposes of the IoT Agent).
 
 For the cases of multiple updates (a single Device Configuration POST that will create several device groups), the
-handler will be called once for each of the configurations (both in the case of the creations and the updates).
+handler will be called once for each of the config groups (both in the case of the creations and the updates).
 
-The handler will be also called in the case of updates related to configurations. In that situation, the
+The handler will be also called in the case of updates related to config groups. In that situation, the
 `newConfiguration` parameter contains also the fields needed to identify the configuration to be updated, i.e.,
 `service`, `subservice`, `resource` and `apikey`.
 
@@ -1330,7 +1333,7 @@ The IoT Agent is now ready to be used. Execute it with the following command:
 node index.js
 ```
 
-The North Port interface should now be fully functional, i.e.: management of device registrations and configurations.
+The North Port interface should now be fully functional, i.e.: management of device registrations and config groups.
 
 ### IoT Agent With Active attributes
 
@@ -1828,7 +1831,7 @@ iotAgentLib.startServer(config, iotAgent, function (error) {
 
 ### Configuration management
 
-For some IoT Agents, it will be useful to know what devices or configurations were registered in the Agent, or to do
+For some IoT Agents, it will be useful to know what devices or config groups were registered in the Agent, or to do
 some actions whenever a new device is registered. All this configuration and provisioning actions can be performed using
 two mechanisms: the provisioning handlers and the provisioning API.
 
@@ -1877,3 +1880,212 @@ iotAgentLib.setProvisioningHandler(provisioningHandler);
 Now we can test our implementation by sending provisioning requests to the North Port of the IoT Agent. If we provision
 a new device into the platform, and then we ask for the list of provisioned devices, we shall see the type of the
 provisioned device has changed to `CertifiedType`.
+
+## IoT Agent additional tools
+
+The IoT Agent Node Lib provides some additional tools that can be used to ease the development of IoT Agents and test
+their functionality.
+
+### Agent Console
+
+A command-line client to experiment with the library is packed with it. The command-line client can be started using the
+following command:
+
+```console
+bin/agentConsole.js
+```
+
+The client offers an API similar to the one offered by the library: it can start and stop an IoT agent, register and
+unregister devices, send measures mimicking the device and receive updates of the device data. Take into account that,
+by default, the console uses the same `config.js` file than the IoT Agent.
+
+The command-line client creates a console that offers the following options:
+
+```text
+stressInit
+
+	Start recording a stress batch.
+
+stressCommit <delay> <times> <threads> <initTime>
+
+	Executes the recorded batch as many times as requested, with delay (ms) between commands.
+	The "threads" parameter indicates how many agents will repeat that same sequence. The "initTime" (ms)
+	parameter indicates the mean of the random initial waiting times for each agent.
+
+exit
+
+	Exit from the command-line.
+
+start
+
+	Start the IoT Agent
+
+stop
+
+	Stop the IoT Agent
+
+register <id> <type>
+
+	Register a new device in the IoT Agent. The attributes to register will be extracted from the
+	type configuration
+
+unregister <id> <type>
+
+	Unregister the selected device
+
+showConfig
+
+	Show the current configuration file
+
+config <newConfig>
+
+	Change the configuration file to a new one
+
+updatevalue <deviceId> <deviceType> <attributes>
+
+	Update a device value in the Context Broker. The attributes should be triads with the following
+	format: "name/type/value" sepparated by commas.
+
+listdevices
+
+	List all the devices that have been registered in this IoT Agent session
+```
+
+### Agent tester
+
+#### Command-line testing
+
+The library also offers a Context Broker and IoT Agent client that can be used to:
+
+-   Simulate operations to the Context Broker used by the IoT Agent, triggering Context Provider forwardings for lazy
+    attributes and checking the appropriate values for active ones.
+-   Simulate operations to the Device Provisioning API and Configuration API of the IoT Agent.
+
+The tester can be started with the following command, from the root folder of the project:
+
+```console
+bin/iotAgentTester.js
+```
+
+From the command-line, the `help` command can be used to show a description of the currently supported features. These
+are the following:
+
+```text
+stressInit
+
+	Start recording a stress batch.
+
+stressCommit <delay> <times> <threads> <initTime>
+
+	Executes the recorded batch as many times as requested, with delay (ms) between commands.
+	The "threads" parameter indicates how many agents will repeat that same sequence. The "initTime" (ms)
+	parameter indicates the mean of the random initial waiting times for each agent.
+
+exit
+
+	Exit from the command-line.
+
+update <entity> <type> <attributes>
+
+	Update the values of the defined set of attributes, using the following format: name#type=value(|name#type=value)*
+
+append <entity> <type> <attributes>
+
+	Append a new Entity with the defined set of attributes, using the following format: name:type=value(,name:type=value)*
+
+query <entity> <type>
+
+	Get all the information on the selected object.
+
+queryAttr <entity> <type> <attributes>
+
+	Get information on the selected object for the selected attributes.
+
+discover <entity> <type>
+
+	Get all the context providers for a entity and type.
+
+configCb <host> <port> <service> <subservice>
+
+	Config a new host and port for the remote Context Broker.
+
+showConfigCb
+
+	Show the current configuration of the client for the Context Broker.
+
+configIot <host> <port> <service> <subservice>
+
+	Config a new host and port for the remote IoT Agent.
+
+showConfigIot
+
+	Show the current configuration of the client for the IoT Agent.
+
+provision <filename>
+
+	Provision a new device using the Device Provisioning API. The device configuration is
+	read from the file specified in the "filename" parameter.
+
+provisionGroup <template> <data> <type>
+
+	Provision a group of devices with the selected template, taking the information needed to
+	fill the template from a CSV with two columns, DEVICE_ID and DEVICE_NAME. The third parameter, type
+	will be used to replace the DEVICE_TYPE field in the template. All the devices will be provisioned
+	to the same IoT Agent, once the templates have been fulfilled.
+
+listProvisioned
+
+	List all the provisioned devices in an IoT Agent.
+
+removeProvisioned <deviceId>
+
+	Remove the selected provisioned device from the IoT Agent, specified by its Device ID.
+
+addGroup <filename>
+
+	Add a new device group to the specified IoT Agent through the Configuration API. The
+	body is taken from the file specified in the "filename" parameter.
+
+listGroups
+
+	List all the device groups created in the selected IoT Agent for the configured service
+
+removeGroup <apiKey> <resource>
+
+	Remove the device group corresponding to the current configured subservice.
+
+authenticate <host> <port> <user> <password> <service>
+
+	Authenticates to the given authentication server, and use the token in subsequent requests.
+
+setProtocol <protocol>
+
+	Sets the protocol to use in the requests (http or https). Defaults to http.
+
+configMigration <host> <port> <originDb>
+
+	Sets the configuration for a migration between a C++ IoTA and a Node.js one.
+
+showConfigMigration
+
+	Shows the current migration configuration.
+
+addProtocols <protocols>
+
+	Add a protocol translation table, in the following format:
+		protocolOrigin1=protocolTarget1;protocolOrigin2=protocolTarget2...
+
+
+migrate <targetDb> <service> <subservice>
+
+	Migrate all the devices and groups for the selected service and subservice into the
+	specified Mongo database. To perform the migration for all the services or all the
+	subservices, use the "*" value.
+```
+
+The agent session stores transient configuration data about the target Context Broker and the target IoT Agent. This
+configuration is independent, and can be checked with the `showConfigCb` and `showConfigIot` commands, respectively.
+Their values can be changed with the `configCb` and `configIot` commands respectively. The new config group will be
+deleted upon startup.
+
+---