c2cwsgiutils 5.2.1__tar.gz → 5.2.1.dev197__tar.gz

{c2cwsgiutils-5.2.1 → c2cwsgiutils-5.2.1.dev197}/PKG-INFO

@@ -1,13 +1,13 @@
 Metadata-Version: 2.1
 Name: c2cwsgiutils
-Version: 5.2.1
+Version: 5.2.1.dev197
 Summary: Common utilities for Camptocamp WSGI applications
 Home-page: https://github.com/camptocamp/c2cwsgiutils
 License: BSD-2-Clause
 Keywords: geo,gis,sqlalchemy,orm,wsgi
 Author: Camptocamp
 Author-email: info@camptocamp.com
-Requires-Python: >=3.8,<4.0
+Requires-Python: >=3.9,<4.0
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Environment :: Plugins
 Classifier: Framework :: Pyramid
@@ -17,45 +17,48 @@ Classifier: License :: OSI Approved :: BSD License
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Topic :: Internet :: WWW/HTTP :: WSGI :: Application
 Classifier: Typing :: Typed
+Provides-Extra: alembic
 Provides-Extra: all
 Provides-Extra: broadcast
+Provides-Extra: debug
 Provides-Extra: dev
 Provides-Extra: oauth2
+Provides-Extra: sentry
 Provides-Extra: standard
 Provides-Extra: test-images
-Requires-Dist: SQLAlchemy (>=1.4.0,<3.0.0) ; extra == "standard" or extra == "all"
-Requires-Dist: SQLAlchemy-Utils ; extra == "standard" or extra == "all"
-Requires-Dist: alembic ; extra == "standard" or extra == "all"
-Requires-Dist: boltons ; extra == "standard" or extra == "all"
-Requires-Dist: cee_syslog_handler ; extra == "standard" or extra == "all"
+Provides-Extra: tests
+Provides-Extra: webserver
+Requires-Dist: SQLAlchemy (>=1.4.0,<3.0.0)
+Requires-Dist: SQLAlchemy-Utils
+Requires-Dist: alembic ; extra == "standard" or extra == "alembic" or extra == "all"
+Requires-Dist: boltons ; extra == "tests" or extra == "all"
+Requires-Dist: cee_syslog_handler
 Requires-Dist: certifi
-Requires-Dist: cornice ; extra == "standard" or extra == "all"
-Requires-Dist: gunicorn ; extra == "standard" or extra == "all"
-Requires-Dist: lxml ; extra == "standard" or extra == "all"
-Requires-Dist: netifaces ; extra == "standard" or extra == "all"
-Requires-Dist: objgraph ; extra == "standard" or extra == "all"
-Requires-Dist: psycopg2 ; extra == "standard" or extra == "all"
-Requires-Dist: pyjwt ; extra == "oauth2" or extra == "all"
-Requires-Dist: pyramid ; extra == "standard" or extra == "all"
-Requires-Dist: pyramid-tm ; extra == "standard" or extra == "all"
+Requires-Dist: cornice
+Requires-Dist: gunicorn ; extra == "standard" or extra == "webserver" or extra == "all"
+Requires-Dist: lxml ; extra == "tests" or extra == "all"
+Requires-Dist: objgraph ; extra == "debug" or extra == "all"
+Requires-Dist: prometheus-client
+Requires-Dist: psycopg2
+Requires-Dist: pyjwt ; extra == "standard" or extra == "oauth2" or extra == "all"
+Requires-Dist: pyramid
+Requires-Dist: pyramid-tm
+Requires-Dist: pyramid_mako
 Requires-Dist: pyyaml
-Requires-Dist: redis ; extra == "standard" or extra == "broadcast" or extra == "all" or extra == "all"
+Requires-Dist: redis ; extra == "standard" or extra == "broadcast" or extra == "all"
 Requires-Dist: requests
-Requires-Dist: requests-oauthlib ; extra == "oauth2" or extra == "all"
+Requires-Dist: requests-oauthlib ; extra == "standard" or extra == "oauth2" or extra == "all"
 Requires-Dist: scikit-image ; extra == "test-images"
-Requires-Dist: sentry-sdk ; extra == "standard" or extra == "all"
-Requires-Dist: ujson ; extra == "standard" or extra == "all"
+Requires-Dist: sentry-sdk ; extra == "standard" or extra == "sentry" or extra == "all"
+Requires-Dist: ujson
 Requires-Dist: waitress ; extra == "dev" or extra == "all"
-Requires-Dist: zope.interface ; extra == "standard" or extra == "all"
-Requires-Dist: zope.sqlalchemy ; extra == "standard" or extra == "all"
+Requires-Dist: zope.interface
+Requires-Dist: zope.sqlalchemy
 Project-URL: Repository, https://github.com/camptocamp/c2cwsgiutils
 Description-Content-Type: text/markdown
 
@@ -64,8 +67,7 @@ Description-Content-Type: text/markdown
 This is a Python 3 library (>=3.5) providing common tools for Camptocamp WSGI
 applications:
 
-- Provide a small framework for gathering performance statistics about
-  a web application (statsd protocol)
+- Provide prometheus metrics
 - Allow to use a master/slave PostgresQL configuration
 - Logging handler for CEE/UDP logs
   - An optional view to change runtime the log levels
@@ -104,7 +106,7 @@ You should install `c2cwsgiutils` with the tool you use to manage your pip depen
 
 In the `Dockerfile` you should add the following lines:
 
-```
+```dockerfile
 # Generate the version file.
 RUN c2cwsgiutils-genversion $(git rev-parse HEAD)
 
@@ -112,20 +114,19 @@ CMD ["gunicorn", "--paste=/app/production.ini"]
 
 # Default values for the environment variables
 ENV \
-    DEVELOPMENT=0 \
-    SQLALCHEMY_POOL_RECYCLE=30 \
-    SQLALCHEMY_POOL_SIZE=5 \
-    SQLALCHEMY_MAX_OVERFLOW=25 \
-    SQLALCHEMY_SLAVE_POOL_RECYCLE=30 \
-    SQLALCHEMY_SLAVE_POOL_SIZE=5 \
-    SQLALCHEMY_SLAVE_MAX_OVERFLOW=25\
-    LOG_TYPE=console \
-    OTHER_LOG_LEVEL=WARNING \
-    GUNICORN_LOG_LEVEL=WARNING \
-    GUNICORN_ACCESS_LOG_LEVEL=INFO \
-    SQL_LOG_LEVEL=WARNING \
-    C2CWSGIUTILS_LOG_LEVEL=WARNING \
-    LOG_LEVEL=INFO
+  DEVELOPMENT=0 \
+  SQLALCHEMY_POOL_RECYCLE=30 \
+  SQLALCHEMY_POOL_SIZE=5 \
+  SQLALCHEMY_MAX_OVERFLOW=25 \
+  SQLALCHEMY_SLAVE_POOL_RECYCLE=30 \
+  SQLALCHEMY_SLAVE_POOL_SIZE=5 \
+  SQLALCHEMY_SLAVE_MAX_OVERFLOW=25\
+  LOG_TYPE=console \
+  OTHER_LOG_LEVEL=WARNING \
+  GUNICORN_LOG_LEVEL=WARNING \
+  SQL_LOG_LEVEL=WARNING \
+  C2CWSGIUTILS_LOG_LEVEL=WARNING \
+  LOG_LEVEL=INFO
 ```
 
 Add in your `main` function.
@@ -161,8 +162,6 @@ The related environment variables:
 - `SQL_LOG_LEVEL`: The SQL query log level, `WARNING`: no logs, `INFO`: logs the queries,
   `DEBUG` also logs the results, default is `WARNING`.
 - `GUNICORN_ERROR_LOG_LEVEL`: The Gunicorn error log level, default is `WARNING`.
-- `GUNICORN_ACCESS_LOG_LEVEL`: The Gunicorn access log level, the logs have the level `INFO`,
-  default is `WARNING`.
 - `C2CWSGIUTILS_CONFIG`: The fallback ini file to use by gunicorn, default is `production.ini`.
 - `C2CWSGIUTILS_LOG_LEVEL`: The c2c WSGI utils log level, default is `WARNING`.
 - `OTHER_LOG_LEVEL`: The log level for all the other logger, default is `WARNING`.
@@ -331,47 +330,6 @@ The requests module is also patched to monitor requests done without timeout. In
 configure a default timeout with the `C2C_REQUESTS_DEFAULT_TIMEOUT` environment variable
 (`c2c.requests_default_timeout`). If no timeout and no default is specified, a warning is issued.
 
-## Metrics
-
-To enable and configure the metrics framework, you can use:
-
-- STATS_VIEW (c2c.stats_view): if defined, will enable the stats view `{C2C_BASE_PATH}/stats.json`
-- STATSD_ADDRESS (c2c.statsd_address): if defined, send stats to the given statsd server
-- STATSD_PREFIX (c2c.statsd_prefix): prefix to add to every metric names
-- STATSD_USE_TAGS: If true, automatic metrics will use tags
-- STATSD*TAG*{tag_name}: To set a global tag for the service
-
-If enabled, some metrics are automatically generated:
-
-- {STATSD_PREFIX}.route.{verb}.{route_name}.{status}: The time to process a query (includes rendering)
-- {STATSD_PREFIX}.render.{verb}.{route_name}.{status}: The time to render a query
-- {STATSD_PREFIX}.sql.{query}: The time to execute the given SQL query (simplified and normalized)
-- {STATSD_PREFIX}.requests.{scheme}.{hostname}.{port}.{verb}.{status}: The time to execute HTTP requests to
-  outside services (only the time between the start of sending of the request and when the header is
-  back with a chunk of the body)
-- {STATSD_PREFIX}.redis.{command}: The time to execute the given Redis command
-
-You can manually measure the time spent on something like that:
-
-```python
-from c2cwsgiutils import stats
-with stats.timer_context(['toto', 'tutu']):
-    do_something()
-```
-
-It will only add a timer event in case of success. If you want to measure both success and failures, do that:
-
-```python
-from c2cwsgiutils import stats
-with stats.outcome_timer_context(['toto', 'tutu']):
-    do_something()
-```
-
-Other functions exists to generate metrics. Look at the `c2cwsgiutils.stats` module.
-
-Look at the `c2cwsgiutils-stats-db` utility if you want to generate statistics (gauges) about the
-row counts.
-
 ## SQL profiler
 
 The SQL profiler must be configured with the `C2C_SQL_PROFILER_ENABLED` environment variable. That enables a view
@@ -519,27 +477,100 @@ If the `/app/versions.json` exists, a view is added (`{C2C_BASE_PATH}/versions.j
 version of a app. This file is generated by calling the `c2cwsgiutils-genversion [$GIT_TAG] $GIT_HASH`
 command line. Usually done in the [Dockerfile](acceptance_tests/app/Dockerfile) of the WSGI application.
 
-## Metrics
+## Prometheus
 
-The path `/metrics` provide some metrics for Prometheus.
-By default we have the `smap` `pss`, but we can easily add the `rss`, `size` or your custom settings:
+[Prometheus client](https://github.com/prometheus/client_python) is integrated in c2cwsgiutils.
 
-Example:
+It will work in multi process mode with the limitation listed in the
+[`prometheus_client` documentation](https://github.com/prometheus/client_python#multiprocess-mode-eg-gunicorn).
 
+To enable it you should provide the `C2CWSGIUTILS_PROMETHEUS_PORT` environment variable.
+For security reason, this port should not be exposed.
+
+We can customize it with the following environment variables:
+
+- `C2C_PROMETHEUS_PREFIX`: to customize the prefix, default is `c2cwsggiutils-`.
+- `C2C_PROMETHEUS_PACKAGES` the packages that will be present in the version information, default is `c2cwsgiutils,pyramid,gunicorn,sqlalchemy`.
+- `C2C_PROMETHEUS_APPLICATION_PACKAGE` the packages that will be present in the version information as application.
+
+And you should add in your `gunicorn.conf.py`:
+
+```python
+from prometheus_client import multiprocess
+
+
+def on_starting(server):
+    from c2cwsgiutils import prometheus
+
+    del server
+
+    prometheus.start()
+
+
+def post_fork(server, worker):
+    from c2cwsgiutils import prometheus
+
+    del server, worker
+
+    prometheus.cleanup()
+
+
+def child_exit(server, worker):
+    del server
+
+    multiprocess.mark_process_dead(worker.pid)
 ```
-from import c2cwsgiutils.metrics import add_provider, Provider, MemoryMapProvider
 
-class CustomProvider(Provider):
-    def __init__(self):
-        super().__init__("my_metrics", "My Metric")
+In your `Dockerfile` you should add:
+
+```dockerfile
+RUN mkdir -p /prometheus-metrics \
+    && chmod a+rwx /prometheus-metrics
+ENV PROMETHEUS_MULTIPROC_DIR=/prometheus-metrics
+```
 
-    def get_data(self):
-        return [({'metadata_key': 'matadata_value'}, metrics_value)]
+### Add custom metric collector
 
-add_provider(MemoryMapProvider('rss'))
-add_provider(CustomProvider())
+See [official documentation](https://github.com/prometheus/client_python#custom-collectors).
+
+Related to the Unix process.
+
+```python
+from c2cwsgiutils import broadcast, prometheus
+
+prometheus.MULTI_PROCESS_COLLECTOR_BROADCAST_CHANNELS.append("prometheus_collector_custom")
+broadcast.subscribe("c2cwsgiutils_prometheus_collect_gc", _broadcast_collector_custom)
+my_custom_collector_instance = MyCustomCollector()
+
+
+def _broadcast_collector_custom() -> List[prometheus.SerializedGauge]:
+    """Get the collected GC gauges."""
+
+    return prometheus.serialize_collected_data(my_custom_collector_instance)
+```
+
+Related to the host, use that in the `gunicorn.conf.py`:
+
+```python
+def on_starting(server):
+    from c2cwsgiutils import prometheus
+
+    del server
+
+    registry = CollectorRegistry()
+    registry.register(MyCollector())
+    prometheus.start(registry)
 ```
 
+### Database metrics
+
+Look at the `c2cwsgiutils-stats-db` utility if you want to generate statistics (gauges) about the
+row counts.
+
+### Usage of metrics
+
+With c2cwsgiutils each instance (Pod) has its own metrics, so we need to aggregate them to have the metrics for the service you probably need to use `sum by (<fields>) (<metric>)` to get the metric (especially for counters).
+
 ## Custom scripts
 
 To have the application initialized in a script you should use the
@@ -592,7 +623,7 @@ have dumps of a few things:
 - memory usage: `{C2C_BASE_PATH}/debug/memory?secret={C2C_SECRET}&limit=30&analyze_type=builtins.dict&python_internals_map=false`
 - object ref: `{C2C_BASE_PATH}/debug/show_refs.dot?secret={C2C_SECRET}&analyze_type=gunicorn.app.wsgiapp.WSGIApplication&analyze_id=12345&max_depth=3&too_many=10&filter=1024&no_extra_info&backrefs`
   `analyze_type` and `analyze_id` should not ve used toogether, you can use it like:
-  ```
+  ```bash
   curl "<URL>" > /tmp/show_refs.dot
   dot -Lg -Tpng /tmp/show_refs.dot > /tmp/show_refs.png
   ```
@@ -653,7 +684,7 @@ client. In production mode, you can still get them by sending the secret defined
 
 If you want to use pyramid_debugtoolbar, you need to disable exception handling and configure it like that:
 
-```
+```ini
 pyramid.includes =
     pyramid_debugtoolbar
 debugtoolbar.enabled = true
@@ -727,3 +758,41 @@ To make a release:
 - Add the new branch name in the `.github/workflows/rebuild.yaml` and
   `.github/workflows/audit.yaml` files.
 
+## Testing
+
+### Screenshots
+
+To test the screenshots, you need to install `node` with `npm`, to do that add the following lines in your `Dockerfile`:
+
+```dockerfile
+RUN --mount=type=cache,target=/var/lib/apt/lists \
+    --mount=type=cache,target=/var/cache,sharing=locked \
+    . /etc/os-release \
+    && echo "deb https://deb.nodesource.com/node_18.x ${VERSION_CODENAME} main" > /etc/apt/sources.list.d/nodesource.list \
+    && curl --silent https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add - \
+    && apt-get update \
+    && apt-get install --assume-yes --no-install-recommends 'nodejs=18.*'
+```
+
+To do the image test call `check_screenshot` e.g.:
+
+```python
+def test_screenshot(app_connection):
+    image.check_screenshot(
+        app_connection.base_url + "my-path",
+        width=800,
+        height=600,
+        result_folder="results",
+        expected_filename=os.path.join(os.path.dirname(__file__), "my-check.expected.png"),
+    )
+```
+
+## Contributing
+
+Install the pre-commit hooks:
+
+```bash
+pip install pre-commit
+pre-commit install --allow-missing-config
+```
+

{c2cwsgiutils-5.2.1 → c2cwsgiutils-5.2.1.dev197}/README.md

@@ -3,8 +3,7 @@
 This is a Python 3 library (>=3.5) providing common tools for Camptocamp WSGI
 applications:
 
-- Provide a small framework for gathering performance statistics about
-  a web application (statsd protocol)
+- Provide prometheus metrics
 - Allow to use a master/slave PostgresQL configuration
 - Logging handler for CEE/UDP logs
   - An optional view to change runtime the log levels
@@ -43,7 +42,7 @@ You should install `c2cwsgiutils` with the tool you use to manage your pip depen
 
 In the `Dockerfile` you should add the following lines:
 
-```
+```dockerfile
 # Generate the version file.
 RUN c2cwsgiutils-genversion $(git rev-parse HEAD)
 
@@ -51,20 +50,19 @@ CMD ["gunicorn", "--paste=/app/production.ini"]
 
 # Default values for the environment variables
 ENV \
-    DEVELOPMENT=0 \
-    SQLALCHEMY_POOL_RECYCLE=30 \
-    SQLALCHEMY_POOL_SIZE=5 \
-    SQLALCHEMY_MAX_OVERFLOW=25 \
-    SQLALCHEMY_SLAVE_POOL_RECYCLE=30 \
-    SQLALCHEMY_SLAVE_POOL_SIZE=5 \
-    SQLALCHEMY_SLAVE_MAX_OVERFLOW=25\
-    LOG_TYPE=console \
-    OTHER_LOG_LEVEL=WARNING \
-    GUNICORN_LOG_LEVEL=WARNING \
-    GUNICORN_ACCESS_LOG_LEVEL=INFO \
-    SQL_LOG_LEVEL=WARNING \
-    C2CWSGIUTILS_LOG_LEVEL=WARNING \
-    LOG_LEVEL=INFO
+  DEVELOPMENT=0 \
+  SQLALCHEMY_POOL_RECYCLE=30 \
+  SQLALCHEMY_POOL_SIZE=5 \
+  SQLALCHEMY_MAX_OVERFLOW=25 \
+  SQLALCHEMY_SLAVE_POOL_RECYCLE=30 \
+  SQLALCHEMY_SLAVE_POOL_SIZE=5 \
+  SQLALCHEMY_SLAVE_MAX_OVERFLOW=25\
+  LOG_TYPE=console \
+  OTHER_LOG_LEVEL=WARNING \
+  GUNICORN_LOG_LEVEL=WARNING \
+  SQL_LOG_LEVEL=WARNING \
+  C2CWSGIUTILS_LOG_LEVEL=WARNING \
+  LOG_LEVEL=INFO
 ```
 
 Add in your `main` function.
@@ -100,8 +98,6 @@ The related environment variables:
 - `SQL_LOG_LEVEL`: The SQL query log level, `WARNING`: no logs, `INFO`: logs the queries,
   `DEBUG` also logs the results, default is `WARNING`.
 - `GUNICORN_ERROR_LOG_LEVEL`: The Gunicorn error log level, default is `WARNING`.
-- `GUNICORN_ACCESS_LOG_LEVEL`: The Gunicorn access log level, the logs have the level `INFO`,
-  default is `WARNING`.
 - `C2CWSGIUTILS_CONFIG`: The fallback ini file to use by gunicorn, default is `production.ini`.
 - `C2CWSGIUTILS_LOG_LEVEL`: The c2c WSGI utils log level, default is `WARNING`.
 - `OTHER_LOG_LEVEL`: The log level for all the other logger, default is `WARNING`.
@@ -270,47 +266,6 @@ The requests module is also patched to monitor requests done without timeout. In
 configure a default timeout with the `C2C_REQUESTS_DEFAULT_TIMEOUT` environment variable
 (`c2c.requests_default_timeout`). If no timeout and no default is specified, a warning is issued.
 
-## Metrics
-
-To enable and configure the metrics framework, you can use:
-
-- STATS_VIEW (c2c.stats_view): if defined, will enable the stats view `{C2C_BASE_PATH}/stats.json`
-- STATSD_ADDRESS (c2c.statsd_address): if defined, send stats to the given statsd server
-- STATSD_PREFIX (c2c.statsd_prefix): prefix to add to every metric names
-- STATSD_USE_TAGS: If true, automatic metrics will use tags
-- STATSD*TAG*{tag_name}: To set a global tag for the service
-
-If enabled, some metrics are automatically generated:
-
-- {STATSD_PREFIX}.route.{verb}.{route_name}.{status}: The time to process a query (includes rendering)
-- {STATSD_PREFIX}.render.{verb}.{route_name}.{status}: The time to render a query
-- {STATSD_PREFIX}.sql.{query}: The time to execute the given SQL query (simplified and normalized)
-- {STATSD_PREFIX}.requests.{scheme}.{hostname}.{port}.{verb}.{status}: The time to execute HTTP requests to
-  outside services (only the time between the start of sending of the request and when the header is
-  back with a chunk of the body)
-- {STATSD_PREFIX}.redis.{command}: The time to execute the given Redis command
-
-You can manually measure the time spent on something like that:
-
-```python
-from c2cwsgiutils import stats
-with stats.timer_context(['toto', 'tutu']):
-    do_something()
-```
-
-It will only add a timer event in case of success. If you want to measure both success and failures, do that:
-
-```python
-from c2cwsgiutils import stats
-with stats.outcome_timer_context(['toto', 'tutu']):
-    do_something()
-```
-
-Other functions exists to generate metrics. Look at the `c2cwsgiutils.stats` module.
-
-Look at the `c2cwsgiutils-stats-db` utility if you want to generate statistics (gauges) about the
-row counts.
-
 ## SQL profiler
 
 The SQL profiler must be configured with the `C2C_SQL_PROFILER_ENABLED` environment variable. That enables a view
@@ -458,27 +413,100 @@ If the `/app/versions.json` exists, a view is added (`{C2C_BASE_PATH}/versions.j
 version of a app. This file is generated by calling the `c2cwsgiutils-genversion [$GIT_TAG] $GIT_HASH`
 command line. Usually done in the [Dockerfile](acceptance_tests/app/Dockerfile) of the WSGI application.
 
-## Metrics
+## Prometheus
+
+[Prometheus client](https://github.com/prometheus/client_python) is integrated in c2cwsgiutils.
+
+It will work in multi process mode with the limitation listed in the
+[`prometheus_client` documentation](https://github.com/prometheus/client_python#multiprocess-mode-eg-gunicorn).
+
+To enable it you should provide the `C2CWSGIUTILS_PROMETHEUS_PORT` environment variable.
+For security reason, this port should not be exposed.
+
+We can customize it with the following environment variables:
+
+- `C2C_PROMETHEUS_PREFIX`: to customize the prefix, default is `c2cwsggiutils-`.
+- `C2C_PROMETHEUS_PACKAGES` the packages that will be present in the version information, default is `c2cwsgiutils,pyramid,gunicorn,sqlalchemy`.
+- `C2C_PROMETHEUS_APPLICATION_PACKAGE` the packages that will be present in the version information as application.
+
+And you should add in your `gunicorn.conf.py`:
+
+```python
+from prometheus_client import multiprocess
+
+
+def on_starting(server):
+    from c2cwsgiutils import prometheus
+
+    del server
+
+    prometheus.start()
+
+
+def post_fork(server, worker):
+    from c2cwsgiutils import prometheus
+
+    del server, worker
+
+    prometheus.cleanup()
+
+
+def child_exit(server, worker):
+    del server
 
-The path `/metrics` provide some metrics for Prometheus.
-By default we have the `smap` `pss`, but we can easily add the `rss`, `size` or your custom settings:
+    multiprocess.mark_process_dead(worker.pid)
+```
+
+In your `Dockerfile` you should add:
+
+```dockerfile
+RUN mkdir -p /prometheus-metrics \
+    && chmod a+rwx /prometheus-metrics
+ENV PROMETHEUS_MULTIPROC_DIR=/prometheus-metrics
+```
+
+### Add custom metric collector
+
+See [official documentation](https://github.com/prometheus/client_python#custom-collectors).
+
+Related to the Unix process.
+
+```python
+from c2cwsgiutils import broadcast, prometheus
 
-Example:
+prometheus.MULTI_PROCESS_COLLECTOR_BROADCAST_CHANNELS.append("prometheus_collector_custom")
+broadcast.subscribe("c2cwsgiutils_prometheus_collect_gc", _broadcast_collector_custom)
+my_custom_collector_instance = MyCustomCollector()
 
+
+def _broadcast_collector_custom() -> List[prometheus.SerializedGauge]:
+    """Get the collected GC gauges."""
+
+    return prometheus.serialize_collected_data(my_custom_collector_instance)
 ```
-from import c2cwsgiutils.metrics import add_provider, Provider, MemoryMapProvider
 
-class CustomProvider(Provider):
-    def __init__(self):
-        super().__init__("my_metrics", "My Metric")
+Related to the host, use that in the `gunicorn.conf.py`:
 
-    def get_data(self):
-        return [({'metadata_key': 'matadata_value'}, metrics_value)]
+```python
+def on_starting(server):
+    from c2cwsgiutils import prometheus
 
-add_provider(MemoryMapProvider('rss'))
-add_provider(CustomProvider())
+    del server
+
+    registry = CollectorRegistry()
+    registry.register(MyCollector())
+    prometheus.start(registry)
 ```
 
+### Database metrics
+
+Look at the `c2cwsgiutils-stats-db` utility if you want to generate statistics (gauges) about the
+row counts.
+
+### Usage of metrics
+
+With c2cwsgiutils each instance (Pod) has its own metrics, so we need to aggregate them to have the metrics for the service you probably need to use `sum by (<fields>) (<metric>)` to get the metric (especially for counters).
+
 ## Custom scripts
 
 To have the application initialized in a script you should use the
@@ -531,7 +559,7 @@ have dumps of a few things:
 - memory usage: `{C2C_BASE_PATH}/debug/memory?secret={C2C_SECRET}&limit=30&analyze_type=builtins.dict&python_internals_map=false`
 - object ref: `{C2C_BASE_PATH}/debug/show_refs.dot?secret={C2C_SECRET}&analyze_type=gunicorn.app.wsgiapp.WSGIApplication&analyze_id=12345&max_depth=3&too_many=10&filter=1024&no_extra_info&backrefs`
   `analyze_type` and `analyze_id` should not ve used toogether, you can use it like:
-  ```
+  ```bash
   curl "<URL>" > /tmp/show_refs.dot
   dot -Lg -Tpng /tmp/show_refs.dot > /tmp/show_refs.png
   ```
@@ -592,7 +620,7 @@ client. In production mode, you can still get them by sending the secret defined
 
 If you want to use pyramid_debugtoolbar, you need to disable exception handling and configure it like that:
 
-```
+```ini
 pyramid.includes =
     pyramid_debugtoolbar
 debugtoolbar.enabled = true
@@ -665,3 +693,41 @@ To make a release:
 - Tag the GIT commit.
 - Add the new branch name in the `.github/workflows/rebuild.yaml` and
   `.github/workflows/audit.yaml` files.
+
+## Testing
+
+### Screenshots
+
+To test the screenshots, you need to install `node` with `npm`, to do that add the following lines in your `Dockerfile`:
+
+```dockerfile
+RUN --mount=type=cache,target=/var/lib/apt/lists \
+    --mount=type=cache,target=/var/cache,sharing=locked \
+    . /etc/os-release \
+    && echo "deb https://deb.nodesource.com/node_18.x ${VERSION_CODENAME} main" > /etc/apt/sources.list.d/nodesource.list \
+    && curl --silent https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add - \
+    && apt-get update \
+    && apt-get install --assume-yes --no-install-recommends 'nodejs=18.*'
+```
+
+To do the image test call `check_screenshot` e.g.:
+
+```python
+def test_screenshot(app_connection):
+    image.check_screenshot(
+        app_connection.base_url + "my-path",
+        width=800,
+        height=600,
+        result_folder="results",
+        expected_filename=os.path.join(os.path.dirname(__file__), "my-check.expected.png"),
+    )
+```
+
+## Contributing
+
+Install the pre-commit hooks:
+
+```bash
+pip install pre-commit
+pre-commit install --allow-missing-config
+```