iotagent-node-lib 4.4.0 → 4.6.0

package/README.md

@@ -1,4 +1,4 @@
-# FIWARE IoT Agent Node.js Library
+# FIWARE IoT Agent Node Library
 
 [![FIWARE IoT Agents](https://nexus.lab.fiware.org/static/badges/chapters/iot-agents.svg)](https://www.fiware.org/developers/catalogue/)
 [![License: APGL](https://img.shields.io/github/license/telefonicaid/iotagent-node-lib.svg)](https://opensource.org/licenses/AGPL-3.0)
@@ -10,300 +10,95 @@
 ![Status](https://nexus.lab.fiware.org/static/badges/statuses/iot-node-lib.svg)
 [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/4671/badge)](https://bestpractices.coreinfrastructure.org/projects/4671)
 
-This project aims to provide a Node.js module to enable IoT Agent developers to build custom agents for their devices
-that can easily connect to NGSI Context Brokers (such as [Orion](https://github.com/telefonicaid/fiware-orion)).
+An IoT Agent is a component that lets groups of IoT devices send their data to a NGSI Context Broker (such as
+[Orion Context Broker](https://github.com/telefonicaid/fiware-orion)) using their own native protocols and translating
+them into NGSI.
 
-An IoT Agent is a component that lets groups of devices send their data to and be managed from a FIWARE NGSI Context
-Broker using their own native protocols. IoT Agents should also be able to deal with security aspects of the FIWARE
-platform (authentication and authorization of the channel) and provide other common services to the device programmer.
+The **IoT Agent Node Lib** provides simple standardized REST API for registering, provisioning, discovering and managing
+devices and groups of devices. It also ensures that each IoT Agent can configure its device communications using a
+common vocabulary regardless of the payload, syntax or transport protocol used by the device itself
 
-This project is part of [FIWARE](https://www.fiware.org/). For more information check the FIWARE Catalogue entry for the
-[IoT Agents](https://github.com/Fiware/catalogue/tree/master/iot-agents).
-
-| :books: [Documentation](https://iotagent-node-lib.rtfd.io) | :mortar_board: [Academy](https://fiware-academy.readthedocs.io/en/latest/iot-agents/idas) | :dart: [Roadmap](https://github.com/telefonicaid/iotagent-node-lib/blob/master/doc/roadmap.md) |
-| ---------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
-
-## Index
-
--   [Background](#background)
--   [Install](#install)
--   [API](#api)
--   [Usage](#usage)
--   [Testing](#testing)
-    -   [Agent Console](#agent-console)
-    -   [Agent tester](#agent-tester)
--   [Licence](#licence)
+It also provides a common configuration framework driven by a `config.js` configuration file and a series of common ENV
+variables, configuring some flags for common functionality provided by the **IoT Agent node lib** (e.g. for contecting
+to a Conext Broker or for authenticating with an Identity Manager).Some of common utility functions provided by the
+**IoT Agent node lib** include:
 
-## Background
-
-The main concept of the **IoT Agent node library** is to provide a common framework for provisioning IoT devices,
-allowing each individual IoT Agent to access standardized mapping data for devices and to offer a series common utility
-functions.
-
--   For southbound communications, the library listens to changes in context entities and raises callbacks for the IoT
-    Agent to process.
--   For northbound communications, the library offers an interface which accepts structured input data so that all NGSI
-    communications are left to the library.
+-   For the device communication (southbound), the library listens to changes in context entities and raises callbacks
+    for the IoT Agent to process. It also handles the commands sent by the Context Broker to the devices.
+-   For the context broker communications (northbound), the library offers an interface which persists data from the
+    device in the Context Broker and accepts NGSI data from the Context Broker to be sent to the device.
 -   Standardized OAuth2-based security is available to enable each IoT Agent to connect to several common Identity
     Managers (e.g. Keystone and Keyrock) so that communications can be restricted to trusted components.
 
-Each individual IoT Agent offers is driven by a `config.js` configuration file contains explicit custom settings based
-on the protocol and payload the IoT Agent is translating. It will also contain some common flags for common
-functionality provided by the IoT Agent node lin (e.g. for contecting to a conext broker or for authentication). The
-**IoT Agent node library** offers a standard API for provisioning devices and ensures that each IoT Agent can configure
-its device communications using a common vocabulary regardless of the payload, syntax or transport protocol used by the
-device itself.
-
-## Install
-
-The **IoT Agent node library** is not a standalone product and should be added as a dependency to `package.json` of the
-IoT Agent
-
-```json
-...
-"dependencies": {
-    "iotagent-node-lib": "*",
-}
-```
-
-In order to use the library within your own IoT Agent, you must first you require it before use:
-
-```javascript
-const iotagentLib = require('iotagent-node-lib');
-```
-
-Information about how to configure the Library can be found at the corresponding section of the
-[Installation & Administration Guide](doc/admin.md).
-
-## Usage
-
-This library has no packaging or build processes. The [Getting Started](./doc/getting-started.md) is a good place to
-start. You can also review the [API documentation](./doc/api.md) for a full list of the available functions.
-
-If you plan to use the library in your own IoT Agent, you should read the [Developer Guide](./doc/devel/development.md).
-You can also review the [Architecture](./doc/devel/architecture.md) documentation and
-[Northbound API](./doc/devel/northbound-api.md).
-
-The following features are listed as [deprecated](doc/deprecated.md).
-
-## API
-
-The **IoT Agent node library** offers a simple REST API which provides common functionality to access, provision and
-decommission devices. [API](doc/api.md).
-
-## Testing
-
-Contributions to development can be found [here](doc/devel/development.md) - additional contributions are welcome.
-
-If you are whishing to test the library, or include new tests (either as part of a contribution or as a new feature or
-as a bug report), you can use the functional tests suite included in the project. The tests are described using a JSON
-file. You can find more information about the test suite in the [Functional Tests Guide](test/functional/README.md).
-
-### Agent Console
-
-A command-line client to experiment with the library is packed with it. The command-line client can be started using the
-following command:
-
-```console
-bin/agentConsole.js
-```
-
-The client offers an API similar to the one offered by the library: it can start and stop an IoT agent, register and
-unregister devices, send measures mimicking the device and receive updates of the device data. Take into account that,
-by default, the console uses the same `config.js` file than the IoT Agent.
-
-The command-line client creates a console that offers the following options:
-
-```text
-stressInit
-
-	Start recording a stress batch.
-
-stressCommit <delay> <times> <threads> <initTime>
-
-	Executes the recorded batch as many times as requested, with delay (ms) between commands.
-	The "threads" parameter indicates how many agents will repeat that same sequence. The "initTime" (ms)
-	parameter indicates the mean of the random initial waiting times for each agent.
-
-exit
-
-	Exit from the command-line.
-
-start
-
-	Start the IoT Agent
-
-stop
-
-	Stop the IoT Agent
-
-register <id> <type>
-
-	Register a new device in the IoT Agent. The attributes to register will be extracted from the
-	type configuration
-
-unregister <id> <type>
-
-	Unregister the selected device
-
-showConfig
-
-	Show the current configuration file
-
-config <newConfig>
-
-	Change the configuration file to a new one
-
-updatevalue <deviceId> <deviceType> <attributes>
-
-	Update a device value in the Context Broker. The attributes should be triads with the following
-	format: "name/type/value" sepparated by commas.
-
-listdevices
-
-	List all the devices that have been registered in this IoT Agent session
-```
-
-### Agent tester
-
-#### Command-line testing
-
-The library also offers a Context Broker and IoT Agent client that can be used to:
-
--   Simulate operations to the Context Broker used by the IoT Agent, triggering Context Provider forwardings for lazy
-    attributes and checking the appropriate values for active ones.
--   Simulate operations to the Device Provisioning API and Configuration API of the IoT Agent.
-
-The tester can be started with the following command, from the root folder of the project:
-
-```console
-bin/iotAgentTester.js
-```
-
-From the command-line, the `help` command can be used to show a description of the currently supported features. These
-are the following:
-
-```text
-stressInit
-
-	Start recording a stress batch.
-
-stressCommit <delay> <times> <threads> <initTime>
-
-	Executes the recorded batch as many times as requested, with delay (ms) between commands.
-	The "threads" parameter indicates how many agents will repeat that same sequence. The "initTime" (ms)
-	parameter indicates the mean of the random initial waiting times for each agent.
-
-exit
-
-	Exit from the command-line.
-
-update <entity> <type> <attributes>
-
-	Update the values of the defined set of attributes, using the following format: name#type=value(|name#type=value)*
-
-append <entity> <type> <attributes>
-
-	Append a new Entity with the defined set of attributes, using the following format: name:type=value(,name:type=value)*
-
-query <entity> <type>
-
-	Get all the information on the selected object.
-
-queryAttr <entity> <type> <attributes>
-
-	Get information on the selected object for the selected attributes.
-
-discover <entity> <type>
-
-	Get all the context providers for a entity and type.
-
-configCb <host> <port> <service> <subservice>
-
-	Config a new host and port for the remote Context Broker.
-
-showConfigCb
-
-	Show the current configuration of the client for the Context Broker.
-
-configIot <host> <port> <service> <subservice>
-
-	Config a new host and port for the remote IoT Agent.
-
-showConfigIot
-
-	Show the current configuration of the client for the IoT Agent.
-
-provision <filename>
-
-	Provision a new device using the Device Provisioning API. The device configuration is
-	read from the file specified in the "filename" parameter.
-
-provisionGroup <template> <data> <type>
-
-	Provision a group of devices with the selected template, taking the information needed to
-	fill the template from a CSV with two columns, DEVICE_ID and DEVICE_NAME. The third parameter, type
-	will be used to replace the DEVICE_TYPE field in the template. All the devices will be provisioned
-	to the same IoT Agent, once the templates have been fulfilled.
-
-listProvisioned
-
-	List all the provisioned devices in an IoT Agent.
-
-removeProvisioned <deviceId>
-
-	Remove the selected provisioned device from the IoT Agent, specified by its Device ID.
-
-addGroup <filename>
-
-	Add a new device group to the specified IoT Agent through the Configuration API. The
-	body is taken from the file specified in the "filename" parameter.
-
-listGroups
-
-	List all the device groups created in the selected IoT Agent for the configured service
-
-removeGroup <apiKey> <resource>
+This project is part of [FIWARE](https://www.fiware.org/). For more information check the FIWARE Catalogue entry for the
+[IoT Agents](https://github.com/Fiware/catalogue/tree/master/iot-agents).
 
-	Remove the device group corresponding to the current configured subservice.
+| :books: [Documentation](https://iotagent-node-lib.rtfd.io) | :mortar_board: [Academy](https://fiware-academy.readthedocs.io/en/latest/iot-agents/idas) | :dart: [Roadmap](https://github.com/telefonicaid/iotagent-node-lib/blob/master/doc/roadmap.md) |
+| ---------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
 
-authenticate <host> <port> <user> <password> <service>
+## Content
 
-	Authenticates to the given authentication server, and use the token in subsequent requests.
+-   [Documentation](#documentation)
+    -   [User documentation](#user-documentation)
+    -   [Development documentation](#development-documentation)
+-   [IoT Agents available](#iot-agents-available)
+-   [Install and usage](#install-and-usage)
+-   [License](#license)
 
-setProtocol <protocol>
+## Documentation
 
-	Sets the protocol to use in the requests (http or https). Defaults to http.
+This repository contains the common user documentation across all IoT Agents. For particular documentation, you can
+check the specific documentation for each IoT Agent. You can see the a list of available IoT Agents in the
+[IoT Agents available](#iot-agents-available) section.
 
-configMigration <host> <port> <originDb>
+### User documentation
 
-	Sets the configuration for a migration between a C++ IoTA and a Node.js one.
+-   [Getting started](doc/getting-started.md)
+-   [IoT Agent API](doc/api.md)
+-   [Administration manual](doc/admin.md)
+-   [Deprecated features](doc/deprecated.md)
+-   [Roadmap](doc/roadmap.md)
 
-showConfigMigration
+### Development documentation
 
-	Shows the current migration configuration.
+-   [Development manual](doc/devel/development.md)
+-   [Contributing guide](doc/devel/contribution-guidelines.md)
+-   [Architecture](doc/devel/architecture.md)
+-   [North Port - NGSI Interactions](doc/devel/northboundinteractions.md)
 
-addProtocols <protocols>
+## IoT Agents available
 
-	Add a protocol translation table, in the following format:
-		protocolOrigin1=protocolTarget1;protocolOrigin2=protocolTarget2...
+The following IoT Agents using the **IoT Agent Node Lib** are available:
 
+-   [IoT Agent JSON](https://github.com/telefonicaid/iotagent-json) - a bridge between HTTP/MQTT messaging (with a JSON
+    payload) and NGSI
+-   [IoT Agent Ultralight](https://github.com/telefonicaid/iotagent-ul) - a bridge between HTTP/MQTT messaging
+-   [IoT Agent LWM2M](https://github.com/telefonicaid/lightweightm2m-iotagent) - a bridge between the
+    [Lightweight M2M](https://www.omaspecworks.org/what-is-oma-specworks/iot/lightweight-m2m-lwm2m/) protocol and NGSI
+    (with an UltraLight2.0 payload) and NGSI
+-   [IoT Agent for Sigfox](https://github.com/telefonicaid/sigfox-iotagent) - a bridge between the
+    [Sigfox](https://www.sigfox.com/en) protocol and NGSI
+-   [IoT Agent for LoRaWAN](https://github.com/Atos-Research-and-Innovation/IoTagent-LoRaWAN) - a bridge between the
+    [LoRaWAN](https://www.thethingsnetwork.org/docs/lorawan/) protocol and NGSI
+-   [IoT Agent for OPC-UA](https://github.com/Engineering-Research-and-Development/iotagent-opcua) - a bridge between
+    the [OPC Unified Architecture](http://www.opcua.us/) protocol and NGSI
+-   [IoT Agent for ISOXML](https://github.com/FIWARE/iotagent-isoxml) - a bridge between the ISOXML/ADAPT protocol for
+    agricultural machinery and NGSI
 
-migrate <targetDb> <service> <subservice>
+## Install and usage
 
-	Migrate all the devices and services for the selected service and subservice into the
-	specified Mongo database. To perform the migration for all the services or all the
-	subservices, use the "*" value.
-```
+The **IoT Agent node library** is not a standalone product. If you plan to install and use any of the IoT Agents
+available, you should follow the installation instructions for each IoT Agent (find the link in the
+[previous section](#iot-agents-available)). You can find the common API provided by the **IoT Agent node library** under
+[API](doc/api.md) documentation.
 
-The agent session stores transient configuration data about the target Context Broker and the target IoT Agent. This
-configuration is independent, and can be checked with the `showConfigCb` and `showConfigIot` commands, respectively.
-Their values can be changed with the `configCb` and `configIot` commands respectively. The new configurations will be
-deleted upon startup.
+If you plan to use the IoT Agent node library in your own project or IoT Agent, you should follow the
+[Developer manual](doc/devel/development.md), which includes the installation instructions and the usage of the library.
 
----
+Information about how to configure the IoT agent or the library can be found at the corresponding section of the
+[Administration manual](doc/admin.md).
 
-## Licence
+## License
 
 The IoT Agent Node Library is licensed under [Affero General Public License (GPL) version 3](./LICENSE).
 

package/config.js

@@ -76,7 +76,8 @@ var config = {
     subservice: '/gardens',
     providerUrl: 'http://192.168.56.1:4041',
     deviceRegistrationDuration: 'P1M',
-    defaultType: 'Thing'
+    defaultType: 'Thing',
+    expressLimit: '1Mb'
 };
 
 module.exports = config;

package/doc/README.md

@@ -4,7 +4,7 @@
 
 -   [Getting started](getting-started.md)
 -   [IoT Agent API](api.md)
--   [Installation and administration manual](admin.md)
+-   [Administration manual](admin.md)
 -   [Deprecated features](deprecated.md)
 -   [Roadmap](roadmap.md)
 

package/doc/admin.md

@@ -5,7 +5,6 @@
         -   [loglevel](#loglevel)
         -   [contextBroker](#contextbroker)
         -   [server](#server)
-        -   [stats](#stats)
         -   [authentication](#authentication)
         -   [deviceRegistry](#deviceregistry)
         -   [mongodb](#mongodb)
@@ -126,7 +125,7 @@ allowing the computer to interpret the rest of the data with more clarity and de
 ```
 
 Under mixed mode, **NGSI v2** payloads are used for context broker communications by default, but this payload may also
-be switched to **NGSI LD** at service group or device provisioning time using the `ngsiVersion` field in the
+be switched to **NGSI LD** at group or device provisioning time using the `ngsiVersion` field in the
 provisioning API. The `ngsiVersion` field switch may be added at either group or device level, with the device level
 overriding the group setting.
 
@@ -159,17 +158,6 @@ support nulls or multi-attribute requests if they are encountered.
 }
 ```
 
-#### `stats`
-
-It configures the periodic collection of statistics. Use `interval` in milliseconds to set the time between stats
-writings.
-
-```javascript
-stats: {
-    interval: 100;
-}
-```
-
 #### `authentication`
 
 Stores the authentication data, for use in retrieving tokens for devices with a trust token (required in scenarios with
@@ -260,13 +248,13 @@ the `mongob` section (as described bellow). E.g.:
 
 It configures the MongoDB driver for those repositories with 'mongodb' type. If the `host` parameter is a list of
 comma-separated IPs, they will be considered to be part of a Replica Set. In that case, the optional property
-`replicaSet` should contain the Replica Set name. If the database requires authentication, username (`user`),
-password (`password`) and authSource (`authSource`) can be set. If the database requires TLS/SSL connection but any
-validation of the certificate chain is not mandatory, all you need is to set the ssl (`ssl`) option as `true` to connect
-the database. If you need to add more complex option(s) such as `retryWrites=true` or `w=majority` when connection
-database, extraArgs (`extraArgs`) can be used to perform it. For The MongoBD driver will retry the connection at startup
-time `retries` times, waiting `retryTime` seconds between attempts, if those attributes are present (default values are
-5 and 5 respectively). E.g.:
+`replicaSet` should contain the Replica Set name. If the database requires authentication, username (`user`), password
+(`password`) and authSource (`authSource`) can be set. If the database requires TLS/SSL connection but any validation of
+the certificate chain is not mandatory, all you need is to set the ssl (`ssl`) option as `true` to connect the database.
+If you need to add more complex option(s) such as `retryWrites=true` or `w=majority` when connection database, extraArgs
+(`extraArgs`) can be used to perform it. For The MongoBD driver will retry the connection at startup time `retries`
+times, waiting `retryTime` seconds between attempts, if those attributes are present (default values are 5 and 5
+respectively). E.g.:
 
 ```javascript
 {
@@ -318,7 +306,7 @@ added `agentPath`:
 
 #### `types`
 
-See **Type Configuration** in the [Configuration API](#configurationapi) section below.
+This parameter includes additional groups configuration as described into the [Config group API](api.md#config-group-api) section.
 
 #### `service`
 
@@ -356,7 +344,7 @@ if this flag is activated:
 
 #### `defaultResource`
 
-default string to use as resource for the registration of new Configurations (if no resource is provided).
+default string to use as resource for the registration of new config groups (if no resource is provided).
 
 #### `defaultKey`
 
@@ -421,6 +409,14 @@ characters (such as semi-colons) which are
 specification. When provisioning devices, it is necessary that the developer provides valid `objectId`-`name` mappings
 whenever relaxed mode is used, to prevent the consumption of forbidden characters.
 
+#### `expressLimit`
+
+IotAgents, as all Express applications that use the body-parser middleware, have a default limit to the request body
+size that the application will handle. This default limit for ioiotagnets are 1Mb. So, if your IotAgent receives a
+request with a body that exceeds this limit, the application will throw a “Error: Request entity too large”.
+
+The 1Mb default can be changed setting the `expressLimit` configuration parameter (or equivalente `IOTA_EXPRESS_LIMIT` environment variable).
+
 ### Configuration using environment variables
 
 Some of the configuration parameters can be overriden with environment variables, to ease the use of those parameters
@@ -482,6 +478,7 @@ overrides.
 | IOTA_EXPLICIT_ATTRS                  | `explicitAttrs`                 |
 | IOTA_DEFAULT_ENTITY_NAME_CONJUNCTION | `defaultEntityNameConjunction`  |
 | IOTA_RELAX_TEMPLATE_VALIDATION       | `relaxTemplateValidation`       |
+| IOTA_EXPRESS_LIMIT                   | `expressLimit`                  |
 
 Note: