dnaStreaming 2.3.0__tar.gz → 2.4.0__tar.gz

dnastreaming-2.4.0/.github/workflows/publish.yaml

@@ -0,0 +1,32 @@
+name: Publish a new version to PyPI when a new release is published
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  pypi-publish:
+    name: Publish release to PyPI
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/p/dnaStreaming
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0 # retrieves git history with tag
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.14"
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install setuptools wheel build
+      - name: Build package
+        run: |
+          python -m build
+      - name: Publish package distributions to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

dnastreaming-2.4.0/.gitignore

@@ -0,0 +1,132 @@
+.DS_Store
+*.bak
+*.pyc
+.idea/*
+.vscode/*
+/src/tests/DowJonesDNA.json
+r.sh
+/build
+dnaStreaming/logs
+.cache
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Dow Jones DNA Data folders
+data*/
+dicts/
+
+# Dow Jones DNA local working files
+ZZ*
+
+# MacOS DS_Store
+.DS_Store
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+.envrc
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+# Visual Studio Code
+.vscode
+
+# Ruff files
+.ruff_cache

dnastreaming-2.4.0/CHANGELOG.md

@@ -0,0 +1,152 @@
+0.9.2 / 2017-04-14
+==================
+- [docs] Updated README.rst. 
+- [breaking] Removed subscriber module. Added 'listener'. This will simplify how customers run our code. No longer will they be creating
+the subscription *and* consuming it. Instead they will just consume an already existing subscription.
+- [Tests] Updated tests to use listener. Updated tests to use new name for sample credentials file. Deleted old subscriber test.
+
+0.9.3 / 2017-04-25
+==================
+- [docs] Updated README.rst
+- [breaking] Replaced references 'DOW_JONES_APPLICATION_CREDENTIALS' envinronmental variable for 'DOW_JONES_DNA_JSON_PATH'.
+- [breaking] Replaced system's filename expectation for 'dowJonesApplicationCredentials.json' to 'DowJonesDNA.json':
+- [Tests] Updated test to use the changes described above.
+
+1.0.0 / 2017-06-07
+==================
+- [docs] Updated README.rst
+- [breaking] - No longer using 'DowJonesDNA.json' file.
+- [changed] - Added customer_config.json file. See README.rst for usage explanation.
+- [changed] - Get the user's credentials information by making a REST call to Dow Jones extraction API service.
+- [Tests] Updated tests to use the changes described above.
+
+1.0.1 / 2017-06-09
+==================
+- [breaking] Removed ability to consume more than one stream at a time.
+- [tests] Updated tests.
+- [docs] Updated README.
+
+1.0.2 / 2017-06-29
+==================
+- [changed] - Reduced the listed dependencies in requirements.txt and setup.py.
+- [changed] - Corrected URI in setup.py.
+- [tests] - Fixed 2 broken tests in test_config.py.
+- [docs] - Updated README with more complete instructions.
+
+1.0.3 / 2017-06-29
+==================
+- [changed] - Changed default PROD URI.
+
+1.0.4 / 2017-06-29
+==================
+- [docs] - Updated README. Git project pip install should use 'https'.
+
+1.0.5 / 2017-08-02
+==================
+- [changed] - Added Google API Extension (GaxError) error handling. Adding a logging file that will catch logged information. Log name is 'logs/dj-dna-streaming-python.log'.
+- [docs] - Updated README with information about logs.  
+
+1.0.6 / 2017-08-04
+==================
+- [changed] - Fixed how we handle GaxErrors
+
+1.0.7 / 2017-08-04
+==================
+- [changed] - Added tox tests to test against python2 and python3.
+- [changed] - Made some python3 compatibility changes. 
+
+1.0.8 / 2017-08-04
+==================
+- [changed] - Refactored how we do pull and acknowledge messages to be more clear. 
+
+1.0.9 / 2017-08-17
+==================
+- [changed] - Created a Dockerfile for demo purposes.
+- [docs] - Updated README.
+
+1.0.10 / 2017-08-22
+==================
+- [changed] - Dependency now pulls in entire GCloud library due to compatibility concerns. Will revert back to more focused dependency list in the future.
+- [changed] - Upgraded Google Cloud dependency version to fix incompatibility with Google language module in a related project.
+
+1.1.0 / 2018-03-19
+==================
+- [changed] - Updated authentication to use new service account flow (while still supporting old account ID flow for backwards compatibility)
+- [changed] - Updated demo code to accept environment variable that, when used, reduces the output volume.
+- [changed] - Removed maximum messages parameter in listen call. It was unnecessary.
+- [changed] - Added test shell script './dnaStreaming/test/test_run_docker.sh' for testing Docker and streams; to be used only with DNA Engineering assistance.
+
+1.4.2 / 2019-12-13
+==================
+- [changed] - Removed support for Python 2, due to near end of life.
+- [changed] - Updated all dependencies to latest ones.
+- [docs] - Minor improvements.
+
+1.4.3 / 2020-01-28
+==================
+- [changed] - Message format provided by the listener.
+- [changed] - Default logging level (still hardcoded)
+- [changed] - Demo files output, to be more explicit on current actions.
+- [added] - Demo file that push messages to GCP Pub/Sub
+- [docs] - Minor improvements.
+
+2.0.0 / 2020-05-11
+==================
+- [changed] Phased out Python 2.7 support in preparation for PyPi publication
+- [changed] Messages from the listener are parsed and returned, instead of requiring the user to access the JSON
+- [docs] - Minor improvements.
+
+2.0.1 / 2020-11-03
+==================
+- [changed] Updated dependencies and required versions of pubsub
+
+2.0.2 / 2020-12-03
+==================
+- [changed] - Removed references to depreciated Extraction API service, now you may manually specify your API host by exporting API_HOST instead of EXTRACTION_API_HOST
+
+2.0.3 / 2022-09-15
+==================
+- [changed] - Updated all GCP dependencies in requirements and setup.
+- [changed] - Optimized for installation via PIP. Setup.py is deprecated.
+- [changed] - Adapted Pub/Sub operations as per new library requirements.
+- [docs] - Changed README removing legacy references and updated the running demo steps.
+
+2.0.4 / 2022-09-27
+==================
+- [changed] - Updated all dependencies in requirements and setup.
+
+2.0.5 / 2022-11-02
+==================
+- [changed] - Removed threaded article quota limit check.
+
+2.1.0 / 2023-03-15
+==================
+- [removed] - Fully removed legacy authentication method by password
+- [added] - Explicit argument to provide the path to the customer_config.json file to the Listener constructor (view README for more details)
+
+2.2.0 / 2024-05-21
+==================
+- [added] - Support for highly-available streams with a new method under the `Listener` class: `listen_async_ha`
+
+2.2.1 / 2024-07-01
+==================
+- [fixed] - Fixed the `subscription_id` parameter of the listener callback so it changes if the listener switches regions (for HA streams only)
+
+2.3.0 / 2025-05-08
+==================
+- [added] - Allow the use of temporal log path
+- [added] - Implement automatic builds via Github Actions
+- [added] - Adds a minimum version requirement for the setuptools package to ensure compatibility and access to necessary features or fixes.
+
+
+2.3.1 / 2025-05-27
+==================
+- [added] - Updated demo scripts to properly handle the new 'source_delete' event type as well as any future event types.
+- [added] - Improve demo scripts folder location to be easier to found it
+
+2.4.0 / 2026-04-09
+==================
+- [added] - Authentication via OAuth bearer token
+- [changed] - Upgraded to Python 3.14
+- [changed] - Upgraded dependencies
+- [changed] - Upgraded package to work with pyproject.toml

{dnastreaming-2.3.0 → dnastreaming-2.4.0}/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2016 Dow Jones Corporation
+Copyright (c) 2026 Dow Jones Corporation
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

dnastreaming-2.4.0/PKG-INFO

@@ -0,0 +1,279 @@
+Metadata-Version: 2.4
+Name: dnaStreaming
+Version: 2.4.0
+Summary: A Python 3 library for Dow Jones customers to consume data from a Factiva Analytics Stream.
+Author-email: Zachary Kagan <zachary.kagan@dowjones.com>, Daniel Morales <daniel.morales@dowjones.com>, Alan Fuentes <alan.fuentes@dowjones.com>, Carlos Blazquez <carlos.blazquez@dowjones.com>, Carlos Niño <carlos.ninoalarcon@dowjones.com>
+License: MIT
+Project-URL: Homepage, https://github.com/dowjones/dj-dna-streams-python
+Project-URL: Download, https://github.com/dowjones/dj-dna-streams-python/archive/master.tar.gz
+Keywords: DOWJONES,FACTIVA,STREAMS
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: google-cloud-pubsub>=2.36.0
+Requires-Dist: google-auth>=2.49.0
+Requires-Dist: google-api-core>=2.30.0
+Requires-Dist: requests>=2.32.5
+Provides-Extra: dev
+Requires-Dist: tox; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Dynamic: license-file
+
+# Dow Jones DNA Streaming Python Library
+
+A Python 3 library for Dow Jones customers to consume data from a Factiva Analytics Stream.
+
+## Installation
+
+### Without source code
+
+If you don't wish to work with any of the demo snippets provided by this repository,
+you can proceed with a clean installation of the package by running either of the
+following commands.
+
+```bash
+# Install the last published version
+pip3 install dnaStreaming
+
+# Install a specific published version
+pip3 install dnaStreaming==<VERSION>
+
+# Install from a specific Github branch
+pip3 install git+https://github.com/dowjones/dj-dna-streams-python.git@<BRANCH_NAME>
+```
+
+Alternatively, you can add add it as a dependency inside your `requirements.txt` file
+
+```txt
+...
+# Include ONE of the following lines:
+dnaStreaming
+dnaStreaming==<VERSION>
+git+https://github.com/dowjones/dj-dna-streams-python.git@<BRANCH_NAME>
+...
+```
+
+Or inside your `pyproject.toml` file
+
+```toml
+[project]
+...
+dependencies = [
+    # Include only ONE of the following lines:
+    "dnaStreaming",
+    "dnaStreaming==<VERSION>",
+    "git+https://github.com/dowjones/dj-dna-streams-python.git@<BRANCH_NAME>",
+]
+```
+
+### With source code
+
+On the other hand, if you wish to work with one of the code samples provided in this
+repository, or you want to work with a customized version of the listener by you,
+you can perform an installation from source code once you have cloned the
+repository.
+
+```bash
+cd dj-dna-streams-python
+git switch <BRANCH> # optional if you wish to work on a specific branch
+python3 install -m venv env
+source env/bin/activate
+pip3 install -e . # editable install, recommended
+```
+
+#### Run the demo files
+
+Once you're done installing from source code, make sure you set your credentials as environment variables as described by the [Authentication](#authentication) and [Configuration](#configuration) sections.
+
+Then, you can run the demo file that you prefer:
+
+```bash
+python3 demo/show_stream.py
+# or
+python3 demo/show_stream_async.py
+```
+
+---
+
+## Authentication
+
+You have two options to authenticate:
+- By using your **user key**
+- By using an **OAuth bearer token** (new)
+
+---
+
+## Configuration
+
+To run this code, you need to provide credentials from one of the authentication methods, as well as your subscriptions. There are three ways you can do this: 
+
+### 1. Environment Variables
+
+To set your credentials, set either `USER_KEY` or `OAUTH_TOKEN` as an environment variable:
+
+```bash
+export USER_KEY="<your_user_key>"
+# or
+export OAUTH_TOKEN="<your_oauth_bearer_token>"
+```
+
+To set your subscription ID, set an environment variable named `SUBSCRIPTION_ID`:
+
+```bash
+export SUBSCRIPTION_ID="ABC1234567889"
+```
+
+To set your log folder path, set a directory variable named `LOG_PATH`:
+
+```bash
+export LOG_PATH="/your/custom/log/path"
+```
+
+### 2. Configuration File
+
+In this codebase, you will find a file named `customer_config.json`. You are not required to use this file, but if you prefer to, fill the JSON object within by adding your user key and your subscription ID. Follow basic JSON formatting and syntax conventions.
+
+> **Note:** The listener will search for the `customer_config.json` file inside your `$HOME` directory by default.
+
+If you prefer using an explicit path to your configuration file, pass the absolute path to the `Listener` constructor:
+
+```python
+from dnaStreaming.listener import Listener
+
+# Config file authentication
+listener = Listener(config_file="<ABSOLUTE_PATH_TO_YOUR_CONFIG_FILE>")
+```
+
+### 3. Function Arguments
+
+You may pass your user key or OAuth token to the `Listener` constructor and your subscription ID directly to the `listen` method:
+
+```python
+from dnaStreaming.listener import Listener
+
+# Use the user_key argument to provide your credentials
+listener = Listener(user_key="<YOUR_USER_KEY>")
+
+# Alternatively, use the oauth_token argument
+listener = Listener(oauth_token="<YOUR_OAUTH_BEARER_TOKEN>")
+
+# Use the subscription_id argument to provide your subscription id to the listener
+listener.listen(callback, subscription_id="<YOUR_SUBSCRIPTION_ID>")
+
+# The same parameter applies for the async variation
+listener.listen_async(callback, subscription_id="<YOUR_SUBSCRIPTION_ID>")
+```
+
+*Note: Passing credentials and subscription ID(s) via function arguments will override both environment variables and config file settings.*
+
+If you choose to strictly rely on environment variables, you can simply initialize the listener empty:
+
+```python
+from dnaStreaming.listener import Listener
+
+listener = Listener()
+```
+
+---
+
+## Listening to Messages
+
+### Synchronous Listening
+
+If you want to listen to messages synchronously:
+
+```python
+def callback(message, subscription_id):
+    print(f'Subscription ID: {subscription_id}: Message: {message.data}')
+    return True  # Return False to stop the message flow and unblock the process.
+
+# Omitting maximum_messages means you will continue to get messages as they appear. 
+# This can be a firehose. Use with caution.
+listener.listen(callback, maximum_messages=10)  
+```
+
+### Asynchronous Listening
+
+If you want to listen to messages asynchronously:
+
+```python
+from time import sleep
+
+def callback(message, subscription_id):
+    print(f'Subscription ID: {subscription_id}: Message: {message.data}')
+
+future = listener.listen_async(callback)
+
+# After calling `listen_async`, you need to keep the main thread alive.
+for count in range(0, 5):
+    sleep(1)
+
+# Stop receiving messages after 5 seconds
+if future.running():
+    future.cancel()
+```
+
+---
+
+## Log Files
+
+Minimal logging is written to a file named `dj-dna-streaming-python.log`.
+
+By default, logs are written to the first available directory from the following list:
+1. A custom path set via the environment variable `LOG_PATH`.
+2. A `logs/` folder located within the package installation directory.
+3. A fallback directory: `~/.dj-dna-streaming-python/logs/`.
+
+The first writable location found is selected. A message like `Will log to: /your/custom/log/path` is printed to the console on startup.
+
+💡 **Note:** The log file is overwritten each time the application starts to keep maintenance simple.
+
+You can specify:
+- **Absolute paths**: For example, `/var/log/dna-streaming`.
+- **Relative paths**: For example, `./logs`, relative to the current working directory at runtime.
+
+The code verifies that the specified path is writable. If it isn’t, it automatically falls back to the next available option.
+
+---
+
+## Development, Testing & Linting (for developers)
+
+### Setup
+
+To prepare for local development and debugging, create a virtual environment and install
+both runtime and dev dependencies (make sure to run with a recent version of Python 3):
+
+```bash
+python3 install -m venv env
+source env/bin/activate
+# Install runtime dependencies from pyproject.toml (-e for recommended editable install)
+pip3 install -e .
+# Install dev dependencies from pyproject.toml
+pip3 install -e ".[dev]"
+```
+
+### Running Tests
+To test the library works over multiple Python versions (e.g., Python >= 3.10), use `tox`:
+
+```bash
+# Make sure to set USER_KEY (or OAUTH_TOKEN), API_HOST and SUBSCRIPTION_ID accordingly
+# Strong recommendation:
+#  - SUBSCRIPTION_ID should belong to an active stream with about 2000 queued messages
+#    for faster test execution times.
+tox
+```
+
+### Linting
+Before releasing, or ideally, before you commit, make sure to `ruff` your code 
+to align with minimal but recommended linting. At the project root, run:
+
+```bash
+ruff check . # checks code without changing it
+ruff check --fix . # checks code and applies changes
+ruff format . # formats code
+```