@etherisc/gif-next 0.0.2-e875d61-189 → 0.0.2-e8a628e-326

package/README.md

@@ -1,4 +1,24 @@
-# gif-next (Generic Insurance Framework next version)
+# GIF-Next (Generic Insurance Framework, next version)
+
+
+## Add OpenZeppelin V5 Dependencies
+
+```shell
+forge install openzeppelin-contracts-500=OpenZeppelin/openzeppelin-contracts@v5.0.0
+cd cd lib/openzeppelin-contracts-500
+git checkout tags/v5.0.0
+cd ../..
+```
+
+See `remappings.txt` to see how to work with different OpenZeppelin versions in parallel
+
+```
+cat remappings.txt 
+@openzeppelin5/contracts/=lib/openzeppelin-contracts-500/contracts/
+@openzeppelin/contracts/=lib/openzeppelin-contracts/contracts/
+@openzeppelin/contracts-upgradeable/=lib/openzeppelin-contracts-upgradeable/contracts/
+```
+
 
 ## Submodules checkout
 
@@ -8,6 +28,16 @@ This repository uses submodules. To checkout or update to the latest submodules,
 git submodule update --recursive
 ```
 
+For initial checkout call
+
+```shell
+git submodule update --init --recursive
+```
+
+## Example components
+
+A (very early) example of a product and a pool can be found in the [gif-next-sandbox repository](https://github.com/etherisc/gif-next-sandbox). 
+
 ## Hardhat 
 
 ### NPM Commands
@@ -21,25 +51,94 @@ npm run test-with-gas
 npm run coverage
 ```
 
-### Deployment 
+### Docgen
+
+Produce `solidty-docgen` documentation using the command below.
+
+```shell
+hh docgen
+```
+
+The resulting markdown files are written to `docs`
+
+
+### Full protocol deployment 
 
-The deploy script will deploy all required contracts and create a test instance with a test product and a test pool. 
+Install the dependencies before running the script below for the first time.
+
+```bash
+npm install
+```
+
+The deploy script will deploy all required contracts for gif and create a test instance. 
 
 ```bash
 # run deployment on a locally created ganache instance
-hh run scripts/deploy_all.ts
+hh run scripts/deploy_gif.ts
+```
+
+```bash
+# set appropriate values vor env variables (see below)
 
 # run deployment on another network
-hh run --network <networkname> scripts/deploy_all.ts
+hh run --network <networkname> scripts/deploy_gif.ts
+```
+
+The deployment will persist deployment information into the files `deployments/<chainid>/deployment_state.json`, `deployments/<chainid>/libraries.json` and `deployments/<chainid>/verification_log.json`. 
+This data can then be used for verification of the deployed contracts on etherscan/polygonscan.
+
+For the verification of the contracts on etherscan/polygonscan the above files (created by previous deployment) are required and then the following command can be used:
+
+```bash
+hh run --network <networkname> scripts/verify_deployment.ts 
 ```
 
 Environment variables:
 
-- `SKIP_VERIFICATION` set to `true` to skip etherscan verification (required for ganacht and anvil)
-- `WEB3_INFURA_PROJECT_ID` set to infura project id (required for mumbai and mainnet)
+- `WRITE_ADDRESSES_TO_FILE` set to `true` to write the addresses of the deployed contracts to a file (default: `false`)
+- `RESUMEABLE_DEPLOYMENT` set to `true` to have all (deployment) transactions written to a state file so the script can be resumed after a failure (or after a manual stop) (default: `false`)
+- `GAS_PRICE` set to the gas price to use for deployment (default: `undefined`)
 - `WALLET_MNEMONIC` the mnemonic of the wallet to use for deployment (required for mumbai and mainnet)
+
+- `WEB3_INFURA_PROJECT_ID` set to infura project id (required for mumbai and mainnet)
 - `ETHERSCAN_API_KEY` `POLYGONSCAN_API_KEY` the api key for etherscan/polygonscan (required for mumbai and mainnet)
 
+### Deploy full protocol with fire example components
+
+```bash
+hh run scripts/deploy_all.ts
+```
+
+Like before, use the `--network` option to deploy on a different network.
+
+### Deploy only the fire example components
+
+```bash
+hh run scripts/deploy_fire_components.ts
+```
+
+Ensure that the deployment runs on a chain where a compatible version of the GIF is already deployed. Then ensure the correct environment variables are set. An up to date list of required environment variables can be found in the `deploy_fire_components.ts` script in the main method (just check the lines that contain a `process.env` statement). To be safe, set the environment variable `WRITE_ADDRESSES_TO_FILE` to `true` when deploying the gif and then copy all values from the generated `deployment.env` file to the `.env` file in the root directory of the repository before running the fire example deployment. 
+
+Like before, use the `--network` option to deploy on a different network.
+
+### Create a new instance
+
+Requires previous step to be completed. 
+
+```bash
+# set appropriate values vor env variables (see below)
+
+hh run --network <networkname> scripts/new_instance.ts
+```
+
+Currently an HD wallet is expected to be used for the deployment. The mnemonic of the wallet needs to be provided via the `WALLET_MNEMONIC` environment variable. 
+The instance owner will be the 11th address of the wallet.
+
+Environment variables:
+
+- `WEB3_INFURA_PROJECT_ID` set to infura project id (required for mumbai and mainnet)
+- `WALLET_MNEMONIC` the mnemonic of the wallet to use for deployment (required for mumbai and mainnet)
+- `REGISTRY_ADDRESS` the address of the registry that is already deployed and configured and has a valid master instance
 
 ### Console
 
@@ -56,6 +155,27 @@ await provider.getBalance(me)
 
 https://hardhat.org/hardhat-runner/docs/guides/compile-contracts
 
+### Scripts to find syntax bugs in code
+
+#### find methods missing the `restricted` modifier
+
+```bash
+hh run scripts/find_unrestricted_methods.ts
+```
+
+Checks all public/external methods in services and components for methods that are missing the `restricted` modifier but should have one.
+
+The script `find_unrestricted_methods.ts` is based on the ANTLR grammar `Solidity.g4` (from https://github.com/solidity-parser/antlr) and uses the antlr4ng runtime (https://github.com/mike-lischke/antlr4ng).
+To compile grammar to ts classes run `antlr4ng -Dlanguage=TypeScript -o antlr/generated/ -visitor -listener  antlr/Solidity.g4` (requires openjdk to be installed `sudo apt install openjdk-17-jre-headless`).
+
+#### find methods missing the `virtual` keyword
+
+```bash
+hh run scripts/find_missing_virtual_methods.ts.ts
+```
+
+Checks all public/external methods in services and components for methods that are not marked as `virtual` but should be.
+
 ## Forge 
 
 ### Commands
@@ -78,10 +198,34 @@ forge test -vvvv --mt test_decimals
 # provide gas report for a single test
 forge test --mt test_decimals --gas-report
 
+# provide code coverage report
 forge coverage
+forge coverage --report lcov 
 ```
 
-Chisel session
+### Aliases 
+
+
+| Alias | Command |
+| --- | --- |
+| fb | forge build |
+| fbw | forge build --watch |
+| fbc | forge build contracts/ |
+| fbcw | forge build --watch contracts/ |
+| ft | forge test |
+| ftf | forge test --nmt "_longRunning" |
+| ftt | forge test -vvvv --mt |
+| fttg | forge test -vvvv --gas-report --mt |
+| fttw | forge test -vvvv --watch --mt |
+| ftc | forge test -vvvv --mc |
+| ftcw | forge test -vvvv --watch --mc |
+| fcf | forge coverage --nmt "_longRunning" |
+| fcfr | forge coverage --nmt "_longRunning" --report lcov |
+
+**Important**: All profiles are run using the foundry `ci` profile which disables the optimizer (for speed).
+
+### Chisel session
+
 ```typescript
 import "./contracts/components/Component.sol";
 import "./contracts/components/IPool.sol";
@@ -155,17 +299,7 @@ https://book.getfoundry.sh/reference/
 
 ## Style Guide
 
-Solidity code is to be written according to the [Solidity Style Guide](https://docs.soliditylang.org/en/latest/style-guide.html).
-
-Documentation of the code should be written inline using [NatSpec](https://docs.soliditylang.org/en/latest/natspec-format.html).
-
-### Naming conventions
-
-Additionally, we use the following naming conventions:
-
-- Function arguments and return types: If using custom data types, make the name include the type by appending the Type to the argument name, e.g. `function getInfo(NftId bundleNftId)` instead of `function getInfo(NftId bundleId)`. Background: Custom data types are lost when using the ABI or Typescript binding classes (e.g. instead of `NftID` a `uint96` is used), so the type needs to be included in the name to make it clear what the argument is without having to look at the documentation or checking the solidity source code. 
-- When naming a field or an attribute `id` and the context is not clear, call it `nftId` instead so its clear what type if id it is as there will be multiple ids for different kind of objects. Example: if you the function has a bundle nft id and a policy nft id as arguments, call them `bundleNftId` and `policyNftId` instead of `id` and `policyId`. In case of doubt, be a bit more verbose for the sake of clarity. 
-- When naming things, remember that the code will likely be used in Javascript/Typescript as well, so avoid names that are reserved in Javascript/Typescript. A list of reserved words in Javascript can be found [here](https://www.w3schools.com/js/js_reserved.asp) and a list of reserved words in Typescript can be found [here](https://www.tektutorialshub.com/typescript/identifiers-keywords-in-typescript/). 
+Please see https://docs.etherisc.com/gif-next/3.x/ for style guide and general coding rules. 
 
 ### Automatic code formatting
 
@@ -180,7 +314,7 @@ We use solhint to lint the code.
 To execute linting run `npm run lint`.
 
 
-### Adding Brownie (Legacy)
+### Adding Brownie (Legacy - don't do that :wink: )
 
 python3 is already installed
 
@@ -190,6 +324,7 @@ sudo apt update
 sudo apt install python3-pip
 pip install eth-brownie
 brownie pm install OpenZeppelin/openzeppelin-contracts@4.9.3
+brownie pm install OpenZeppelin/openzeppelin-contracts@5.0.0
 ```
 
 ```bash
@@ -241,28 +376,232 @@ pool_id = pool.getNftId()
 product_id = product.getNftId()
 ```
 
-## migrate to kv store
-
-sizes before
-| Instance                       | 23.178    | 1.398       |
-| TestInstanceBase               | 3.305     | 21.271      |
-| TestInstanceModuleAccess       | 7.86      | 16.716      |
-| TestInstanceModuleBundle       | 7.586     | 16.99       |
-| TestInstanceModuleCompensation | 3.305     | 21.271      |
-| TestInstanceModuleComponent    | 5.123     | 19.453      |
-| TestInstanceModulePolicy       | 6.937     | 17.639      |
-| TestInstanceModulePool         | 4.809     | 19.767      |
-| TestInstanceModuleRisk         | 3.305     | 21.271      |
-| TestInstanceModuleTreasury     | 7.563     | 17.013      |
----
-sizes after
-| Instance                       | 21.776    | 2.8         |
-| TestInstanceBase               | 3.485     | 21.091      |
-| TestInstanceModuleAccess       | 8.036     | 16.54       |
-| TestInstanceModuleBundle       | 8.354     | 16.222      |
-| TestInstanceModuleCompensation | 3.485     | 21.091      |
-| TestInstanceModuleComponent    | 5.482     | 19.094      |
-| TestInstanceModulePolicy       | 7.335     | 17.241      |
-| TestInstanceModulePool         | 5.522     | 19.054      |
-| TestInstanceModuleRisk         | 3.485     | 21.091      |
-| TestInstanceModuleTreasury     | 8.945     | 15.631      |
+## Objects
+
+### Registry
+
+Mandatory properties
+
+* One registry per chain
+* Global registry: the registry on mainnet
+* One entry per protocol object
+* Once registered object properties are immutable
+* Globally unique NFT minted per protocol object (chain id is embedded in NFT id)
+* Object ownership defined by NFT ownernship
+* Typed objects (instance, product, policy, ...)
+* Each object has parent object (only one exception: protocol object does not have parent object)
+* Parent object type is defined by type of child object
+
+Object properties (for smart contracts)
+* Objects that represent a smart contract record its contract address
+* With one exception (see next point) smart contract addresses refer to the address on the same chain as the registry
+* The global registry also holds entries for all the registries on different chains than mainnet, in these cases the addresses refer to the chains of these chain specific registries (this is the only case where registered addresses do not refer to the same chain)
+
+Optional properties
+
+* Objects can have names
+* Object names are unique per chain
+* Objects may define an intercepting property, in which case a predefined smart contract is involved in NFT transfers of the objects NFT (transfer may trigger actions, transfer may be blocked etc)
+* 
+
+Ownership property
+
+* Object ownership defined by ownership of NFT representing the object
+* Protocol
+  * Fixed/predefined owner (address without private key)
+* Registry
+  * Allows registration of token and services (per major release)
+  * May transfer ownership
+* Token
+  * Fixed/predefined owner (address without private key)
+  * TODO add whitelisting for token per major release
+* Service
+  * Allows upgrades of services as long as major version is same
+  * Until further notice the same entity as the registry owner
+  * May transfer ownership
+* Instance
+  * Granting/revoking of roles (both default + custom)
+  * Management of custom roles and targets
+  * May lock instance and/or components
+  * May transfer ownership
+* Component (Product, Pool, Distribution, Oracle)
+  * Register component
+  * Set component wallet (which receives fees, holds funds)
+  * May lock component
+  * May transfer ownership
+  * Additional use case specific features
+* Policy
+  * Represents policy holder
+  * Receive payouts (GIF default behaviour)
+  * May transfer ownership
+* Bundle
+  * Represents funds owner
+  * May withdraw funds not locked by active policies
+  * May transfer ownership
+* Distributor
+  * Receives commissions from sales (GIF default behaviour)
+  * May transfer ownership
+
+Intercepting property
+  * Service owner is indirectly owner of service manager contract
+
+* Instance
+  * intercepts transfer of instance owner (new owner needs access manager admin rights for custom roles and targets)
+  * intercepts transfer of components, to do what? check that new owner has necessary roles?
+* Product
+  * intercepts transfer of policies (use case specific: eg. limit/disallow transfers)
+* Pool:
+  * intercepts transfer of bundles (use case specific: eg. limit/disallow transfers)
+* Distribution
+  * intercepts transfer of distributors (bookkeeping: only one distributor per address allowed)
+* Oracle
+  * likely meaningless
+
+
+### Instance
+
+### Product
+
+### Pool
+
+### Distribution
+
+### Oracle
+
+## Registry and Services
+
+### Principles
+
+- 1 service per object type and major version
+- registry service guards write access to registry
+- all other objects registered via registry service
+- root object for the complete tree is the protocol object
+- under the root object a single registry object is registered (= global registry/ethereum mainnet)
+
+
+### Service Responsibilities
+
+Registry Service
+
+- deployed and registered during bootstrapping of registry
+- used to register tokens and other services by registry owner
+- an object may only be registered by the service designated by the type of the object
+- to register an object the parent object needs already be registered
+- the type of the object to be registered needs to match a valid child type/parent type combination
+
+Instance Service
+
+- deploys master instance during its own bootstrapping (if allowed by contract size)
+- registered via registry service by registry owner
+- registeres master instance during its own registration by regsitry owner
+- deploys and registeres new instances (= instance factory) by instance owner (instance owner is a permissionless role, anybody may creates a new instance)
+- provides upgrade functionality to instance owners
+
+Product Service
+
+- registered via registry service by registry owner
+- registers products for registered instances via registry service by product owner (product owner role is permissend by the product's instance)
+- registers applications/policies for registered products via registry service
+
+Distribution Service
+
+- registered via registry service by registry owner
+- registers distribution components for registered products via registry service by distribution owner (distribution owner role is permissend by the product's instance)
+- registers distributors for registered distribution components via registry service
+
+## Contract Organisation
+
+now 
+
+contracts
+  components
+  instance
+  registry
+  shared
+  types
+
+
+contracts
+  component
+    pool
+      Pool.sol
+      PoolService.sol
+    oracle
+    distribution
+    product
+      Product.sol
+      ProductService.sol
+    Component.sol
+    ComponentService.sol
+  instance
+    Instance.sol
+    InstanceAdmin.sol
+    InstanceReader.sol
+    InstanceService.sol
+    InstanceServiceManager.sol
+  registry
+    ChainNft.sol
+    Registry.sol
+    RegistryAdmin.sol
+    RegistryService.sol
+    RegistryServiceManager.sol
+  shared
+    Registerable.sol
+    RegisterableUpgradable.sol
+  type
+    NftId.sol
+    ObjectType.sol
+
+## Storage layout
+
+### Analysis tool
+
+Command line call
+
+```bash
+hh run scripts/analyze_storage_layout.ts
+```
+
+Analyses contract `MockStorageLayout.sol` and writes the storage layout to `storage_layout.json` and `storage_layout.csv`.
+New structs must be added to `MockStorageLayout` to be included in analysis. 
+
+Storage layout details:
+- Items fill up the current slot if possible
+- If not enough space is left in the current slot, the item is placed in the next slot
+- Stucts and arrays always start a new slot
+- Items following a struct or array always start a new slot
+
+More at https://docs.soliditylang.org/en/latest/internals/layout_in_storage.html
+
+### Custom types sizes
+
+| Type | Size |
+| --- | --- |
+| Amount | 12 bytes |
+| Blocknumber | 4 bytes |
+| ClaimId | 2 bytes |
+| DistributorType | 8 bytes |
+| NftId | 12 bytes |
+| ObjectType | 1 byte |
+| ReferralId | 8 bytes |
+| RoleId | 1 byte |
+| RiskId | 8 bytes |
+| Seconds | 5 bytes |
+| Selector | 4 bytes |
+| StateId | 1 byte |
+| Str | 32 bytes |
+| Timestamp | 5 bytes |
+| Version | 3 byte |
+| VersionPart | 1 byte |
+| UFixed | 20 bytes |
+| address | 20 bytes |
+| uint256 | 32 bytes |
+| uint128 | 16 bytes |
+| uint160 | 20 bytes |
+| uint64 | 8 bytes |
+| uint32 | 4 bytes |
+| uint16  | 2 bytes  |
+| string  | 32 bytes |
+| bytes   | 32 bytes |
+| bytes8  | 8 byte  |
+| bool    | 1 byte  |

package/artifacts/contracts/accounting/AccountingService.sol/AccountingService.dbg.json

@@ -0,0 +1,4 @@
+{
+  "_format": "hh-sol-dbg-1",
+  "buildInfo": "../../../build-info/6e88a942f537d0031463aea6fcae4af5.json"
+}