generative-bayesian-network 0.1.0-beta.3 → 2.0.0-dev.0

package/{LICENSE → LICENSE.md}

@@ -178,7 +178,7 @@
    APPENDIX: How to apply the Apache License to your work.
 
       To apply the Apache License to your work, attach the following
-      boilerplate notice, with the fields enclosed by brackets "[]"
+      boilerplate notice, with the fields enclosed by brackets "{}"
       replaced with your own identifying information. (Don't include
       the brackets!)  The text should be enclosed in the appropriate
       comment syntax for the file format. We also recommend that a
@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright [yyyy] [name of copyright owner]
+   Copyright 2018 Apify Technologies s.r.o.
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

package/README.md

@@ -1,155 +1,3 @@
-# Generative bayesian network
-NodeJs package containing a bayesian network capable of randomly sampling from a distribution defined by a json object.
-
-<!-- toc -->
-
-- [Installation](#installation)
-- [Usage](#usage)
-- [API Reference](#api-reference)
-
-<!-- tocstop -->
-
-## Installation
-Run the `npm i generative-bayesian-network` command. No further setup is needed afterwards.
-## Usage
-To use the network, you need to create an instance of the `BayesianNetwork` class which is exported from this package. Constructor of this class accepts a JSON object containing the network definition.  This definition can either include the probability distributions for the nodes, or these can be calculated later using data. An example of such a definition saved in a JSON file could look like:
-```json
-{
-    "nodes": [
-        {
-            "name": "ParentNode",
-            "values": ["A", "B", "C"],
-            "parentNames": [],
-            "conditionalProbabilities": {
-                "A": 0.1,
-                "B": 0.8,
-                "C": 0.1
-            }
-        },
-        {
-            "name": "ChildNode",
-            "values": [".", ",", "!", "?"],
-            "parentNames": ["ParentNode"],
-            "conditionalProbabilities": {
-                "A": {
-                    ".": 0.7,
-                    "!": 0.3
-                },
-                "B": {
-                    ",": 0.3,
-                    "?": 0.7
-                },
-                "C": {
-                    ".": 0.5,
-                    "?": 0.5
-                }
-            }
-        }
-    ]
-}
-```
-Once you have the network definition ready, you can create an instance simply by executing:
-```js
-let generatorNetwork = new BayesianNetwork(networkDefinition);
-```
-If the network definition didn't include the probabilities, you also need to call the `setProbabilitiesAccordingToData` method and provide it with a [Danfo.js dataframe](https://danfo.jsdata.org/api-reference/dataframe) containing the dataset you want to be used to calculate the probabilities:
-```js
-generatorNetwork.setProbabilitiesAccordingToData(dataframe);
-```
-After the setup, you can save the current network's definition by doing:
-```js
-generatorNetwork.saveNetworkDefinition(networkDefinitionFilePath);
-```
-Once you have the network all set up, you can use two methods to actually generate the samples - `generateSample` and `generateConsistentSampleWhenPossible`. The first one generates a sample of all node values given (optionally) the values we already know in the form of an object. The second does much the same thing, but instead of just getting the known values of some of the attributes, the object you can give it as an argument can contain multiple possible values for each node, not just one. You could run them for example like this:
-```js
-let sample = generatorNetwork.generateSample({ "ParentNode": "A" });
-let consistentSample = generatorNetwork.generateSample({
-    "ParentNode": ["A","B"], "ChildNode": [",","!"]
-});
-```
-
-## API Reference
-All public classes, methods and their parameters can be inspected in this API reference.
-
-<a name="BayesianNetwork"></a>
-
-### BayesianNetwork
-BayesianNetwork is an implementation of a bayesian network capable of randomly sampling from the distribution
-represented by the network.
-
-
-* [BayesianNetwork](#BayesianNetwork)
-    * [`new BayesianNetwork(networkDefinition)`](#new_BayesianNetwork_new)
-    * [`.generateSample(inputValues)`](#BayesianNetwork+generateSample)
-    * [`.generateConsistentSampleWhenPossible(valuePossibilities)`](#BayesianNetwork+generateConsistentSampleWhenPossible)
-    * [`.setProbabilitiesAccordingToData(dataframe)`](#BayesianNetwork+setProbabilitiesAccordingToData)
-    * [`.saveNetworkDefinition(networkDefinitionFilePath)`](#BayesianNetwork+saveNetworkDefinition)
-
-
-* * *
-
-<a name="new_BayesianNetwork_new"></a>
-
-#### `new BayesianNetwork(networkDefinition)`
-
-| Param | Type | Description |
-| --- | --- | --- |
-| networkDefinition | <code>object</code> | object defining the network structure and distributions |
-
-
-* * *
-
-<a name="BayesianNetwork+generateSample"></a>
-
-#### `bayesianNetwork.generateSample(inputValues)`
-Randomly samples from the distribution represented by the bayesian network.
-
-
-| Param | Type | Description |
-| --- | --- | --- |
-| inputValues | <code>object</code> | node values that are known already |
-
-
-* * *
-
-<a name="BayesianNetwork+generateConsistentSampleWhenPossible"></a>
-
-#### `bayesianNetwork.generateConsistentSampleWhenPossible(valuePossibilities)`
-Randomly samples from the distribution represented by the bayesian network,
-making sure the sample is consistent with the provided restrictions on value possibilities.
-Returns false if no such sample can be generated.
-
-
-| Param | Type | Description |
-| --- | --- | --- |
-| valuePossibilities | <code>object</code> | a dictionary of lists of possible values for nodes                                      (if a node isn't present in the dictionary, all values are possible) |
-
-
-* * *
-
-<a name="BayesianNetwork+setProbabilitiesAccordingToData"></a>
-
-#### `bayesianNetwork.setProbabilitiesAccordingToData(dataframe)`
-Sets the conditional probability distributions of this network's nodes to match the given data.
-
-
-| Param | Type | Description |
-| --- | --- | --- |
-| dataframe | <code>object</code> | a Danfo.js dataframe containing the data |
-
-
-* * *
-
-<a name="BayesianNetwork+saveNetworkDefinition"></a>
-
-#### `bayesianNetwork.saveNetworkDefinition(networkDefinitionFilePath)`
-Saves the network definition to the specified file path to be used later.
-
-
-| Param | Type | Description |
-| --- | --- | --- |
-| networkDefinitionFilePath | <code>string</code> | a file path where the network definition should be saved |
-
-
-* * *
+# Fingerprint suite
 
+This repository contains a set of fingerprinting tools developed by Apify.

package/index.js

@@ -0,0 +1,6 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.BayesianNetwork = void 0;
+var bayesian_network_1 = require("./bayesian-network");
+Object.defineProperty(exports, "BayesianNetwork", { enumerable: true, get: function () { return bayesian_network_1.BayesianNetwork; } });
+//# sourceMappingURL=index.js.map

package/package.json

@@ -1,41 +1,40 @@
 {
-  "author": {
-    "name": "Apify"
-  },
-  "files": [
-    "src"
-  ],
-  "bugs": {
-    "url": "https://github.com/apify/generative-bayesian-network/issues"
-  },
-  "dependencies": {
-    "ow": "^0.23.0"
-  },
-  "description": "An implementation of a bayesian network usable for .",
-  "devDependencies": {
-    "@apify/eslint-config": "^0.1.3",
-    "eslint": "^7.19.0",
-    "fs-extra": "^9.1.0",
-    "jest": "^26.6.3",
-    "jsdoc-to-markdown": "^7.0.0",
-    "markdown-toc": "^1.2.0",
-    "jest-os-detection": "^1.3.1",
-    "danfojs-node": "^0.2.2"
-  },
-  "homepage": "https://github.com/apify/generative-bayesian-network#readme",
-  "license": "Apache-2.0",
-  "main": "src/main.js",
-  "name": "generative-bayesian-network",
-  "repository": {
-    "type": "git",
-    "url": "git+https://github.com/apify/generative-bayesian-network.git"
-  },
-  "scripts": {
-    "build-docs": "npm run build-toc && node docs/build-docs.js",
-    "build-toc": "markdown-toc docs/README.md -i",
-    "lint": "eslint ./src --ext .js,.jsx",
-    "lint:fix": "eslint ./src --ext .js,.jsx --fix",
-    "test": "jest --maxWorkers=3 --forceExit"
-  },
-  "version": "0.1.0-beta.3"
+    "name": "generative-bayesian-network",
+    "version": "2.0.0-dev.0",
+    "author": {
+        "name": "Apify"
+    },
+    "files": [
+        "dist"
+    ],
+    "main": "index.js",
+    "module": "index.mjs",
+    "types": "index.d.ts",
+    "exports": {
+        ".": {
+            "import": "./index.mjs",
+            "require": "./index.js"
+        }
+    },
+    "bugs": {
+        "url": "https://github.com/apify/generative-bayesian-network/issues"
+    },
+    "dependencies": {
+        "adm-zip": "^0.5.9",
+        "danfojs-node": "^1.1.1"
+    },
+    "description": "An implementation of a bayesian network usable for .",
+    "homepage": "https://github.com/apify/generative-bayesian-network#readme",
+    "license": "Apache-2.0",
+    "repository": {
+        "type": "git",
+        "url": "git+https://github.com/apify/generative-bayesian-network.git"
+    },
+    "scripts": {
+        "build": "npm run clean && npm run compile",
+        "clean": "rimraf ./dist",
+        "compile": "tsc -p tsconfig.build.json && gen-esm-wrapper ./index.js ./index.mjs",
+        "copy": "ts-node -T ../../scripts/copy.ts"
+    },
+    "gitHead": "2dd3d8c9ced1a14cb74ef16ebdd50b349bb5971b"
 }

package/src/bayesian-network.js

@@ -1,112 +0,0 @@
-const fs = require('fs');
-
-const { BayesianNode } = require('./bayesian-node');
-
-/**
- * BayesianNetwork is an implementation of a bayesian network capable of randomly sampling from the distribution
- * represented by the network.
- */
-class BayesianNetwork {
-    /**
-     * @param {object} networkDefinition - object defining the network structure and distributions
-     */
-    constructor(networkDefinition) {
-        this.nodesInSamplingOrder = networkDefinition.nodes.map((nodeDefinition) => new BayesianNode(nodeDefinition));
-        this.nodesByName = {};
-        for (const node of this.nodesInSamplingOrder) {
-            this.nodesByName[node.name] = node;
-        }
-    }
-
-    /**
-     * Randomly samples from the distribution represented by the bayesian network.
-     * @param {object} inputValues - node values that are known already
-     */
-    generateSample(inputValues = {}) {
-        const sample = inputValues;
-        for (const node of this.nodesInSamplingOrder) {
-            if (!(node.name in sample)) {
-                sample[node.name] = node.sample(sample);
-            }
-        }
-
-        return sample;
-    }
-
-    /**
-     * Randomly samples from the distribution represented by the bayesian network,
-     * making sure the sample is consistent with the provided restrictions on value possibilities.
-     * Returns false if no such sample can be generated.
-     * @param {object} valuePossibilities - a dictionary of lists of possible values for nodes
-     *                                      (if a node isn't present in the dictionary, all values are possible)
-     */
-    generateConsistentSampleWhenPossible(valuePossibilities) {
-        return this._recursivelyGenerateConsistentSampleWhenPossible({}, valuePossibilities, 0);
-    }
-
-    /**
-     * Recursively generates a random sample consistent with the given restrictions on possible values.
-     * @param {object} sampleSoFar - node values that are known already
-     * @param {object} valuePossibilities - a dictionary of lists of possible values for nodes
-     *                                      (if a node isn't present in the dictionary, all values are possible)
-     * @param {number} depth - in what depth of the recursion this function call is,
-     *                         specifies what node this function call is sampling
-     * @private
-     */
-    _recursivelyGenerateConsistentSampleWhenPossible(sampleSoFar, valuePossibilities, depth) {
-        const bannedValues = [];
-        const node = this.nodesInSamplingOrder[depth];
-
-        let sampleValue;
-        do {
-            sampleValue = node.sampleAccordingToRestrictions(sampleSoFar, valuePossibilities[node.name], bannedValues);
-
-            if (!sampleValue) break;
-
-            sampleSoFar[node.name] = sampleValue;
-
-            if (depth + 1 < this.nodesInSamplingOrder.length) {
-                const sample = this._recursivelyGenerateConsistentSampleWhenPossible(sampleSoFar, valuePossibilities, depth + 1);
-                if (sample) {
-                    return sample;
-                }
-            } else {
-                return sampleSoFar;
-            }
-
-            bannedValues.push(sampleValue);
-        } while (sampleValue);
-
-        return false;
-    }
-
-    /**
-     * Sets the conditional probability distributions of this network's nodes to match the given data.
-     * @param {object} dataframe - a Danfo.js dataframe containing the data
-     */
-    setProbabilitiesAccordingToData(dataframe) {
-        this.nodesInSamplingOrder.forEach((node) => {
-            const possibleParentValues = {};
-            for (const parentName of node.parentNames) {
-                possibleParentValues[parentName] = this.nodesByName[parentName].possibleValues;
-            }
-            node.setProbabilitiesAccordingToData(dataframe, possibleParentValues);
-        });
-    }
-
-    /**
-     * Saves the network definition to the specified file path to be used later.
-     * @param {string} networkDefinitionFilePath - a file path where the network definition should be saved
-     */
-    saveNetworkDefinition(networkDefinitionFilePath) {
-        const network = {
-            nodes: this.nodesInSamplingOrder.map((node) => node.nodeDefinition),
-        };
-
-        fs.writeFileSync(networkDefinitionFilePath, JSON.stringify(network));
-    }
-}
-
-module.exports = {
-    BayesianNetwork,
-};

package/src/bayesian-node.js

@@ -1,182 +0,0 @@
-/**
-* Calculates relative frequencies of values of specific attribute from the given data
-* @param {object} dataframe - a Danfo.js dataframe containing the data
-* @param {string} attributeName - name of the attribute
-* @private
-*/
-function getRelativeFrequencies(dataframe, attributeName) {
-    const frequencies = {};
-    const totalCount = dataframe.shape[0];
-    const valueCounts = dataframe[attributeName].value_counts();
-
-    for (let index = 0; index < valueCounts.index.length; index++) {
-        frequencies[valueCounts.index[index]] = valueCounts.values[index] / totalCount;
-    }
-
-    return frequencies;
-}
-
-/**
- * BayesianNode is an implementation of a single node in a bayesian network allowing
- * sampling from its conditional distribution.
- */
-class BayesianNode {
-    /**
-     * @param {object} nodeDefinition - node structure and distributions definition
-     *                                  taken from the network definition file
-     */
-    constructor(nodeDefinition) {
-        this.nodeDefinition = nodeDefinition;
-    }
-
-    /**
-     * Extracts unconditional probabilities of node values given the values of the parent nodes
-     * @param {object} parentValues - values of the parent nodes
-     * @private
-     */
-    _getProbabilitiesGivenKnownValues(parentValues = {}) {
-        let probabilities = this.nodeDefinition.conditionalProbabilities;
-
-        for (const parentName of this.parentNames) {
-            const parentValue = parentValues[parentName];
-            if (parentValue in probabilities.deeper) {
-                probabilities = probabilities.deeper[parentValue];
-            } else {
-                probabilities = probabilities.skip;
-            }
-        }
-        return probabilities;
-    }
-
-    /**
-     * Randomly samples from the given values using the given probabilities
-     * @param {array} possibleValues - a list of values to sample from
-     * @param {number} totalProbabilityOfPossibleValues - a sum of probabilities of possibleValues
-     *                                                    in the conditional distribution
-     * @param {object} probabilities - a dictionary of probabilities from the conditional distribution
-     *                                 indexed by the values
-     * @private
-     */
-    _sampleRandomValueFromPossibilities(possibleValues, totalProbabilityOfPossibleValues, probabilities) {
-        let chosenValue = possibleValues[0];
-        const anchor = Math.random() * totalProbabilityOfPossibleValues;
-        let cumulativeProbability = 0;
-        for (const possibleValue of possibleValues) {
-            cumulativeProbability += probabilities[possibleValue];
-            if (cumulativeProbability > anchor) {
-                chosenValue = possibleValue;
-                break;
-            }
-        }
-
-        return chosenValue;
-    }
-
-    /**
-     * Randomly samples from the conditional distribution of this node given values of parents
-     * @param {object} parentValues - values of the parent nodes
-     */
-    sample(parentValues = {}) {
-        const probabilities = this._getProbabilitiesGivenKnownValues(parentValues);
-        const possibleValues = Object.keys(probabilities);
-
-        return this._sampleRandomValueFromPossibilities(possibleValues, 1.0, probabilities);
-    }
-
-    /**
-     * Randomly samples from the conditional distribution of this node given restrictions on the possible
-     * values and the values of the parents.
-     * @param {object} parentValues - values of the parent nodes
-     * @param {object} valuePossibilities - a list of possible values for this node
-     * @param {object} bannedValues - what values of this node are banned
-     */
-    sampleAccordingToRestrictions(parentValues, valuePossibilities, bannedValues) {
-        const probabilities = this._getProbabilitiesGivenKnownValues(parentValues);
-        let totalProbability = 0.0;
-        const validValues = [];
-        const valuesInDistribution = Object.keys(probabilities);
-        const possibleValues = valuePossibilities || valuesInDistribution;
-        for (const value of possibleValues) {
-            if (!bannedValues.includes(value) && valuesInDistribution.includes(value)) {
-                validValues.push(value);
-                totalProbability += probabilities[value];
-            }
-        }
-
-        if (validValues.length === 0) return false;
-        return this._sampleRandomValueFromPossibilities(validValues, totalProbability, probabilities);
-    }
-
-    /**
-     * Sets the conditional probability distribution for this node to match the given data.
-     * @param {object} dataframe - a Danfo.js dataframe containing the data
-     * @param {object} possibleParentValues - a dictionary of lists of possible values for parent nodes
-     */
-    setProbabilitiesAccordingToData(dataframe, possibleParentValues = {}) {
-        this.nodeDefinition.possibleValues = dataframe[this.name].unique().values;
-        this.nodeDefinition.conditionalProbabilities = this._recursivelyCalculateConditionalProbabilitiesAccordingToData(
-            dataframe,
-            possibleParentValues,
-            0,
-        );
-    }
-
-    /**
-     * Recursively calculates the conditional probability distribution for this node from the data.
-     * @param {object} dataframe - a Danfo.js dataframe containing the data
-     * @param {object} possibleParentValues - a dictionary of lists of possible values for parent nodes
-     * @param {number} depth - depth of the recursive call
-     * @private
-     */
-    _recursivelyCalculateConditionalProbabilitiesAccordingToData(dataframe, possibleParentValues, depth) {
-        let probabilities = {
-            deeper: {},
-        };
-
-        if (depth < this.parentNames.length) {
-            const currentParentName = this.parentNames[depth];
-            for (const possibleValue of possibleParentValues[currentParentName]) {
-                const skip = !dataframe[currentParentName].unique().values.includes(possibleValue);
-                let filteredDataframe = dataframe;
-                if (!skip) {
-                    filteredDataframe = dataframe.query({
-                        column: currentParentName,
-                        is: '==',
-                        to: possibleValue,
-                    });
-                }
-                const nextLevel = this._recursivelyCalculateConditionalProbabilitiesAccordingToData(
-                    filteredDataframe,
-                    possibleParentValues,
-                    depth + 1,
-                );
-
-                if (!skip) {
-                    probabilities.deeper[possibleValue] = nextLevel;
-                } else {
-                    probabilities.skip = nextLevel;
-                }
-            }
-        } else {
-            probabilities = getRelativeFrequencies(dataframe, this.name);
-        }
-
-        return probabilities;
-    }
-
-    get name() {
-        return this.nodeDefinition.name;
-    }
-
-    get parentNames() {
-        return this.nodeDefinition.parentNames;
-    }
-
-    get possibleValues() {
-        return this.nodeDefinition.possibleValues;
-    }
-}
-
-module.exports = {
-    BayesianNode,
-};

package/src/main.js

@@ -1,3 +0,0 @@
-const BayesianNetwork = require('./bayesian-network');
-
-module.exports = BayesianNetwork;