queutils 0.9.1__tar.gz → 0.9.4__tar.gz

{queutils-0.9.1 → queutils-0.9.4}/.github/workflows/python-publish.yml

@@ -12,28 +12,25 @@ on:
   release:
     types: [published]
 
-permissions:
-  contents: read
-
 jobs:
   deploy:
-
     runs-on: ubuntu-latest
+    environment: production
+    permissions:
+      id-token: write
+      contents: read
 
     steps:
-    - uses: actions/checkout@v4
-    - name: Set up Python
-      uses: actions/setup-python@v5
-      with:
-        python-version: '3.11'
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        pip install build
-    - name: Build package
-      run: python -m build
-    - name: Publish package
-      uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
-      with:
-        user: __token__
-        password: ${{ secrets.PYPI_API_TOKEN }}
+      - uses: actions/checkout@v4
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install build
+      - name: Build package
+        run: python -m build
+      - name: Publish package
+        uses: pypa/gh-action-pypi-publish@v1.12.4

{queutils-0.9.1 → queutils-0.9.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: queutils
-Version: 0.9.1
+Version: 0.9.4
 Summary: Handy Python Queue utilies
 Project-URL: Homepage, https://github.com/Jylpah/queutils
 Project-URL: Bug Tracker, https://github.com/Jylpah/queutils/issues
@@ -35,15 +35,16 @@ Description-Content-Type: text/markdown
 
 Queutils *[Queue Utils]* is a package of handy Python queue classes:
 
-- **[AsyncQueue](docs/asyncqueue.md)** - An `async` wrapper for non-async `queue.Queue`
-- **[IterableQueue](docs/iterablequeue.md)** - An `AsyncIterable` queue that terminates when finished
-- **[FileQueue](docs/filequeue.md)** - Builds an iterable queue of filenames from files/dirs given as input
+- **AsyncQueue** - An `async` wrapper for non-async `queue.Queue`
+- **IterableQueue** - An `AsyncIterable` queue that terminates when finished
+- **EventCounterQueue** - An `IterableQueue` for counting events in `async` threads
+- **FileQueue** - Builds an `IterableQueue[pathlib.Path]` of filenames from files/dirs given as input
 
 
 # AsyncQueue
 
-[`AsyncQueue`](docs/asyncqueue.md) is a async wrapper for non-async `queue.Queue`. It can be used to create 
-an `asyncio.Queue` compatible interface to a (non-async) managed `multiprocessing.Queue` and thus enable `async` code in parent/child processes to communicate over  `multiprocessing.Queue` as it were an `asyncio.Queue`. 
+`AsyncQueue` is a async wrapper for non-async `queue.Queue`. It can be used to create 
+an `asyncio.Queue` compatible interface to a (non-async) managed `multiprocessing.Queue` and thus enable `async` code in parent/child processes to communicate over  `multiprocessing.Queue` as it were an `asyncio.Queue`. Uses `sleep()` for `get()`/`put()` if the queue is empty/full.
 
 ## Features 
 
@@ -54,10 +55,10 @@ an `asyncio.Queue` compatible interface to a (non-async) managed `multiprocessin
 
 # IterableQueue
 
-[`IterableQueue`](docs/iterablequeue.md) is an `asyncio.Queue` subclass that is `AsyncIterable[T]` i.e. it can be 
+`IterableQueue` is an `asyncio.Queue` subclass that is `AsyncIterable[T]` i.e. it can be 
 iterated in `async for` loop. `IterableQueue` terminates automatically when the queue has been filled and emptied. 
 
-The `IterableQueue` requires "producers" (functions adding items to the queue) to register themselves and it 
+The `IterableQueue` requires "producers" (functions adding items to the queue) to register themselves with `add_producer()` call and it 
 keeps count of registered producers which are "finished" adding items to the queue. Once all the registered 
 producers are "finished", the queue enters into "filled" state and no new items can be added. Once an 
 "filled" queue is emptied, the queue becomes "done" and all new `get()` calls to the queue will 
@@ -74,9 +75,19 @@ producers are "finished", the queue enters into "filled" state and no new items
 - Countable property can be disabled with count_items=False. This is useful when you
     want to sum the count of multiple IterableQueues 
 
+# EventCounterQueue
+
+`EventCounterQueue` can be used to count named events (default event is `count`) between `async` threads. `async` worker threads call `queue.send(event="event_name", N=amount)`. The receving end can either `receive()` a single event or `listen()` all  events and return `collections.defaultdict[str, int]` as a result.
+
+## Features
+
+- Supports multiple producers and a single listener
+- Default event is `count`
+
+
 # FileQueue
 
-[`FileQueue`](docs/filequeue.md) builds a queue (`IterableQueue[pathlib.Path]`) of the matching 
+`FileQueue` builds a queue (`IterableQueue[pathlib.Path]`) of the matching 
 files found based on search parameters given. It can search both list of files or directories or 
 mixed. Async method `FileQueue.mk_queue()` searches subdirectories of given directories.  
 

{queutils-0.9.1 → queutils-0.9.4}/README.md

@@ -6,13 +6,14 @@ Queutils *[Queue Utils]* is a package of handy Python queue classes:
 
 - **[AsyncQueue](docs/asyncqueue.md)** - An `async` wrapper for non-async `queue.Queue`
 - **[IterableQueue](docs/iterablequeue.md)** - An `AsyncIterable` queue that terminates when finished
-- **[FileQueue](docs/filequeue.md)** - Builds an iterable queue of filenames from files/dirs given as input
+- **EventCounterQueue** - An `IterableQueue` for counting events in `async` threads
+- **[FileQueue](docs/filequeue.md)** - Builds an `IterableQueue[pathlib.Path]` of filenames from files/dirs given as input
 
 
 # AsyncQueue
 
 [`AsyncQueue`](docs/asyncqueue.md) is a async wrapper for non-async `queue.Queue`. It can be used to create 
-an `asyncio.Queue` compatible interface to a (non-async) managed `multiprocessing.Queue` and thus enable `async` code in parent/child processes to communicate over  `multiprocessing.Queue` as it were an `asyncio.Queue`. 
+an `asyncio.Queue` compatible interface to a (non-async) managed `multiprocessing.Queue` and thus enable `async` code in parent/child processes to communicate over  `multiprocessing.Queue` as it were an `asyncio.Queue`. Uses `sleep()` for `get()`/`put()` if the queue is empty/full.
 
 ## Features 
 
@@ -26,7 +27,7 @@ an `asyncio.Queue` compatible interface to a (non-async) managed `multiprocessin
 [`IterableQueue`](docs/iterablequeue.md) is an `asyncio.Queue` subclass that is `AsyncIterable[T]` i.e. it can be 
 iterated in `async for` loop. `IterableQueue` terminates automatically when the queue has been filled and emptied. 
 
-The `IterableQueue` requires "producers" (functions adding items to the queue) to register themselves and it 
+The `IterableQueue` requires "producers" (functions adding items to the queue) to register themselves with `add_producer()` call and it 
 keeps count of registered producers which are "finished" adding items to the queue. Once all the registered 
 producers are "finished", the queue enters into "filled" state and no new items can be added. Once an 
 "filled" queue is emptied, the queue becomes "done" and all new `get()` calls to the queue will 
@@ -43,6 +44,16 @@ producers are "finished", the queue enters into "filled" state and no new items
 - Countable property can be disabled with count_items=False. This is useful when you
     want to sum the count of multiple IterableQueues 
 
+# EventCounterQueue
+
+`EventCounterQueue` can be used to count named events (default event is `count`) between `async` threads. `async` worker threads call `queue.send(event="event_name", N=amount)`. The receving end can either `receive()` a single event or `listen()` all  events and return `collections.defaultdict[str, int]` as a result.
+
+## Features
+
+- Supports multiple producers and a single listener
+- Default event is `count`
+
+
 # FileQueue
 
 [`FileQueue`](docs/filequeue.md) builds a queue (`IterableQueue[pathlib.Path]`) of the matching 

{queutils-0.9.1 → queutils-0.9.4}/docs/filequeue.md

@@ -35,4 +35,18 @@ async def main() -> None:
 
 if __name__ == "__main__":
     run(main())
+```
+
+### Run
+
+```bash
+cd demos
+python -m filequeue_demo
+```
+Output
+```text
+found asyncqueue_demo.py
+found iterablequeue_demo.py
+found filequeue_demo.py
+finished, no need to use fileQ.join()
 ```

queutils-0.9.4/docs/rm_links

@@ -0,0 +1,14 @@
+#!/bin/bash
+
+# Usage: ./rmlinks.sh input.md > output.md
+
+if [ "$#" -ne 1 ]; then
+    echo "Usage: $0 <markdown-file>"
+    exit 1
+fi
+
+INPUT_FILE="$1"
+
+# Use sed to remove markdown links to /docs but keep the link text
+# Matches: [link text](/docs/whatever.md) and turns into: link text
+perl -pe 's/\[([^\]]+)\]\((\/?docs\/[^)]+)\)/\1/g' "$INPUT_FILE"

queutils-0.9.4/pypi.md

@@ -0,0 +1,69 @@
+[![Python package](https://github.com/Jylpah/queutils/actions/workflows/python-package.yml/badge.svg)](https://github.com/Jylpah/queutils/actions/workflows/python-package.yml)  [![codecov](https://codecov.io/gh/Jylpah/queutils/graph/badge.svg?token=rMKdbfHOFs)](https://codecov.io/gh/Jylpah/queutils)
+
+# Queutils
+
+Queutils *[Queue Utils]* is a package of handy Python queue classes:
+
+- **AsyncQueue** - An `async` wrapper for non-async `queue.Queue`
+- **IterableQueue** - An `AsyncIterable` queue that terminates when finished
+- **EventCounterQueue** - An `IterableQueue` for counting events in `async` threads
+- **FileQueue** - Builds an `IterableQueue[pathlib.Path]` of filenames from files/dirs given as input
+
+
+# AsyncQueue
+
+`AsyncQueue` is a async wrapper for non-async `queue.Queue`. It can be used to create 
+an `asyncio.Queue` compatible interface to a (non-async) managed `multiprocessing.Queue` and thus enable `async` code in parent/child processes to communicate over  `multiprocessing.Queue` as it were an `asyncio.Queue`. Uses `sleep()` for `get()`/`put()` if the queue is empty/full.
+
+## Features 
+
+- `asyncio.Queue` compatible
+- `queue.Queue` support
+- `multiprocessing.Queue` support
+
+
+# IterableQueue
+
+`IterableQueue` is an `asyncio.Queue` subclass that is `AsyncIterable[T]` i.e. it can be 
+iterated in `async for` loop. `IterableQueue` terminates automatically when the queue has been filled and emptied. 
+
+The `IterableQueue` requires "producers" (functions adding items to the queue) to register themselves with `add_producer()` call and it 
+keeps count of registered producers which are "finished" adding items to the queue. Once all the registered 
+producers are "finished", the queue enters into "filled" state and no new items can be added. Once an 
+"filled" queue is emptied, the queue becomes "done" and all new `get()` calls to the queue will 
+`raise QueueDone` exception. 
+    
+## Features
+
+- `asyncio.Queue` interface, `_nowait()` methods are experimental
+- `AsyncIterable` support: `async for item in queue:`
+- Automatic termination of the consumers with `QueueDone` exception when the queue has been emptied 
+- Producers must be registered with `add_producer()` and they must notify the queue
+  with `finish()` once they have finished adding items 
+- Countable interface to count number of items task_done() through `count` property
+- Countable property can be disabled with count_items=False. This is useful when you
+    want to sum the count of multiple IterableQueues 
+
+# EventCounterQueue
+
+`EventCounterQueue` can be used to count named events (default event is `count`) between `async` threads. `async` worker threads call `queue.send(event="event_name", N=amount)`. The receving end can either `receive()` a single event or `listen()` all  events and return `collections.defaultdict[str, int]` as a result.
+
+## Features
+
+- Supports multiple producers and a single listener
+- Default event is `count`
+
+
+# FileQueue
+
+`FileQueue` builds a queue (`IterableQueue[pathlib.Path]`) of the matching 
+files found based on search parameters given. It can search both list of files or directories or 
+mixed. Async method `FileQueue.mk_queue()` searches subdirectories of given directories.  
+
+## Features
+
+- Input can be given both as `str` and `pathlib.Path`
+- `exclude: bool` exclusive or  inclusive filtering. Default is `False`.
+- `case_sensitive: bool` case sensitive filtering (use of `fnmatch` or `fnmatchcase`). Default is `True`.
+- `follow_symlinks: bool` whether to follow symlinks. Default is `False`.
+

{queutils-0.9.1 → queutils-0.9.4}/pyproject.toml

@@ -1,9 +1,9 @@
 [project]
 name = "queutils"
-version = "0.9.1"
+version = "0.9.4"
 authors = [{ name = "Jylpah", email = "jylpah@gmail.com" }]
 description = "Handy Python Queue utilies"
-readme = "README.md"
+readme = { file = "pypi.md", content-type = "text/markdown" }
 requires-python = ">=3.11"
 classifiers = [
     "Programming Language :: Python :: 3",

{queutils-0.9.1 → queutils-0.9.4}/src/queutils/__init__.py

@@ -2,15 +2,15 @@ from .countable import Countable as Countable
 from .asyncqueue import AsyncQueue as AsyncQueue
 from .iterablequeue import IterableQueue as IterableQueue, QueueDone as QueueDone
 from .filequeue import FileQueue as FileQueue
-from .categorycounterqueue import (
+from .eventcounterqueue import (
     QCounter as QCounter,
-    CategoryCounterQueue as CategoryCounterQueue,
+    EventCounterQueue as EventCounterQueue,
 )
 
 __all__ = [
     "asyncqueue",
     "countable",
-    "categorycounterqueue",
+    "eventcounterqueue",
     "filequeue",
     "iterablequeue",
 ]

queutils-0.9.1/src/queutils/categorycounterqueue.py → queutils-0.9.4/src/queutils/eventcounterqueue.py

@@ -20,7 +20,7 @@ debug = logger.debug
 T = TypeVar("T")
 
 
-@deprecated(version="0.9.1", reason="Use CategoryCounterQueue instead")
+@deprecated(version="0.9.1", reason="Use EventCounterQueue instead")
 class CounterQueue(Queue[T], Countable):
     """
     CounterQueue is a asyncio.Queue for counting items
@@ -55,9 +55,9 @@ class CounterQueue(Queue[T], Countable):
         return self._count_items
 
 
-class CategoryCounterQueue(IterableQueue[tuple[str, int]]):
+class EventCounterQueue(IterableQueue[tuple[str, int]]):
     """
-    CategorySummerQueue is a asyncio.Queue for summing up values by category
+    EventCounterQueue is a asyncio.Queue for counting events by name
     """
 
     _counter: defaultdict[str, int]
@@ -69,29 +69,29 @@ class CategoryCounterQueue(IterableQueue[tuple[str, int]]):
         self._counter = defaultdict(int)
 
     async def receive(self) -> tuple[str, int]:
-        """Receive a category value from the queue and sum it"""
-        category: str
+        """Receive an event value from the queue and sum it"""
+        event: str
         value: int
-        category, value = await super().get()
-        self._counter[category] += value
+        event, value = await super().get()
+        self._counter[event] += value
         super().task_done()
-        return (category, value)
+        return (event, value)
 
-    async def send(self, category: str = "count", value: int = 1) -> None:
-        """Send count of a category"""
-        await super().put((category, value))
+    async def send(self, event: str = "count", value: int = 1) -> None:
+        """Send count of an event"""
+        await super().put((event, value))
         return None
 
-    def get_count(self, category: str = "count") -> int:
-        """Return count of a category"""
-        return self._counter[category]
+    def get_count(self, event: str = "count") -> int:
+        """Return count for an event"""
+        return self._counter[event]
 
     def get_counts(self) -> defaultdict[str, int]:
-        """Return counts of all categories"""
+        """Return counts of all events"""
         return self._counter
 
     async def listen(self) -> defaultdict[str, int]:
-        """Listen for category values"""
+        """Listen for event values"""
         try:
             while True:
                 await self.receive()

queutils-0.9.4/tests/test_eventcounterqueue.py

@@ -0,0 +1,80 @@
+import pytest  # type: ignore
+from asyncio import (
+    Task,
+    create_task,
+    gather,
+    TimeoutError,
+)
+from random import choice, randint
+import string
+from collections import defaultdict
+
+from queutils import EventCounterQueue
+
+
+def randomword(length: int) -> str:
+    """Generate a random word of fixed length"""
+    # https://stackoverflow.com/a/2030081/12946084
+    letters: str = string.ascii_lowercase
+    return "".join(choice(letters) for i in range(length))
+
+
+QSIZE: int = 10
+N: int = 100  # N >> QSIZE
+THREADS: int = 4
+# N : int = int(1e10)
+
+
+@pytest.mark.parametrize(
+    "events,N, producers",
+    [
+        ([randomword(5) for _ in range(10)], 1000, 1),
+        ([randomword(5) for _ in range(20)], 10000, 1),
+        ([randomword(5) for _ in range(5)], 1000, 3),
+    ],
+)
+@pytest.mark.timeout(10)
+@pytest.mark.asyncio
+async def test_1_category_counter_queue(
+    events: list[str], N: int, producers: int
+) -> None:
+    """Test EventCounterQueue"""
+    Q = EventCounterQueue(maxsize=QSIZE)
+
+    async def producer(
+        Q: EventCounterQueue, events: list[str], N: int = 100
+    ) -> defaultdict[str, int]:
+        """
+        Test Producer for EventCounterQueue
+        """
+        _counter: defaultdict[str, int] = defaultdict(int)
+        await Q.add_producer()
+        for _ in range(N):
+            cat: str = choice(events)
+            count: int = randint(1, 10)
+            await Q.send(cat, count)
+            _counter[cat] += count
+        await Q.finish()
+        return _counter
+
+    senders: list[Task] = list()
+
+    for _ in range(producers):
+        senders.append(create_task(producer(Q, events, N)))
+
+    try:
+        res_in: defaultdict[str, int] = await Q.listen()
+        res_out: defaultdict[str, int] = defaultdict(int)
+        for res in await gather(*senders):
+            for event, count in res.items():
+                res_out[event] += count
+
+        assert res_in == res_out, f"EventCounterQueue: {res_in} != {res_out}"
+        assert Q.qsize() == 0, "queue size is > 0 even it should be empty"
+        assert Q.empty(), "queue not empty"
+        assert Q.count == N * producers, (
+            f"count returned wrong value {Q.count}, should be {N * producers}"
+        )
+
+    except TimeoutError:
+        assert False, "await IterableQueue.join() failed with an empty queue finished"

queutils-0.9.1/tests/test_ccategorycounterqueue.py

@@ -1,333 +0,0 @@
-import pytest  # type: ignore
-from asyncio.queues import QueueEmpty, QueueFull
-from asyncio import (
-    Task,
-    create_task,
-    sleep,
-    gather,
-    timeout,
-    TimeoutError,
-    CancelledError,
-)
-from random import random, choice, randint
-import string
-from collections import defaultdict
-
-from queutils import IterableQueue, CategoryCounterQueue, QueueDone
-
-
-def randomword(length: int) -> str:
-    """Generate a random word of fixed length"""
-    # https://stackoverflow.com/a/2030081/12946084
-    letters: str = string.ascii_lowercase
-    return "".join(choice(letters) for i in range(length))
-
-
-QSIZE: int = 10
-N: int = 100  # N >> QSIZE
-THREADS: int = 4
-# N : int = int(1e10)
-
-
-@pytest.fixture
-def test_interablequeue_int() -> IterableQueue[int]:
-    return IterableQueue[int](maxsize=QSIZE)
-
-
-async def _producer_int(
-    Q: IterableQueue[int], n: int, finish: bool = False, wait: float = 0
-) -> None:
-    await Q.add_producer(N=1)
-    await sleep(wait)
-    try:
-        for i in range(n):
-            await sleep(wait * random())
-            await Q.put(i)
-    except QueueDone:
-        pass
-    if finish:
-        await Q.finish()
-    return None
-
-
-async def _consumer_int(Q: IterableQueue[int], n: int = -1, wait: float = 0) -> bool:
-    try:
-        while n != 0:
-            _ = await Q.get()
-            await sleep(wait * random())
-            Q.task_done()
-            n -= 1
-    except QueueDone:
-        pass
-    except CancelledError:
-        raise
-    return True
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.asyncio
-async def test_1_put_get_async(test_interablequeue_int: IterableQueue[int]):
-    """Test: put(), get(), join(), qsize(), empty() == True"""
-    Q = test_interablequeue_int
-    try:
-        async with timeout(5):
-            await _producer_int(Q, QSIZE - 1, finish=True)
-    except TimeoutError:
-        assert False, "IterableQueue got stuck"
-    assert Q.qsize() == QSIZE - 1, (
-        f"qsize() returned {Q.qsize()}, should be {QSIZE - 1}"
-    )
-    try:
-        await Q.put(1)
-        assert False, "Queue is filled and put() should raise an exception"
-    except QueueDone:
-        pass  # Queue is done and put() should raise an exception
-    assert not Q.is_done, "is_done returned True even queue is not finished"
-    consumer: Task = create_task(_consumer_int(Q))
-    try:
-        async with timeout(5):
-            await Q.join()
-        await Q.get()
-        assert False, "Queue is done and put() should raise an exception"
-    except TimeoutError:
-        assert False, "IterableQueue.join() took too long"
-    except QueueDone:
-        pass  # should be raised
-    assert Q.qsize() == 0, "queue not empty"
-    assert Q.empty(), "queue not empty"
-    consumer.cancel()
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.asyncio
-async def test_2_put_get_nowait(test_interablequeue_int: IterableQueue[int]):
-    """Test put_nowait() and get_nowait() methods"""
-    Q = test_interablequeue_int
-    producer: Task = create_task(_producer_int(Q, N))
-    await sleep(1)
-    # In theory this could fail without a real error
-    # if QSIZE is huge and/or system is slow
-    assert Q.qsize() == Q.maxsize, "Queue was supposed to be at maxsize"
-    assert Q.full(), "Queue should be full"
-    assert not Q.empty(), "Queue should not be empty"
-
-    try:
-        Q.put_nowait(1)
-        assert False, "Queue was supposed to be full, but was not"
-    except QueueFull:
-        pass  # OK, Queue was supposed to be full
-
-    try:
-        while True:
-            _ = Q.get_nowait()
-            Q.task_done()
-            await sleep(0.01)
-    except QueueEmpty:
-        assert Q.qsize() == 0, "Queue size should be zero"
-
-    try:
-        async with timeout(5):
-            await Q.finish()
-            await Q.join()
-    except TimeoutError:
-        assert False, "Queue.join() took longer than it should"
-    assert Q.qsize() == 0, "queue size is > 0 even it should be empty"
-    assert Q.empty(), "queue not empty()"
-    producer.cancel()
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.asyncio
-async def test_3_multiple_producers(test_interablequeue_int: IterableQueue[int]):
-    Q = test_interablequeue_int
-    workers: list[Task] = list()
-    for _ in range(THREADS):
-        workers.append(create_task(_producer_int(Q, N, finish=True, wait=0.05)))
-    try:
-        assert not Q.is_done, "is_done returned True even queue is not finished"
-        async with timeout(10):
-            async for _ in Q:
-                pass
-    except TimeoutError:
-        assert False, "IterableQueue.join() took too long"
-    except QueueDone:
-        pass  # Queue is done
-
-    assert Q.qsize() == 0, f"queue size is {Q.qsize()} even it should be empty"
-    assert Q.empty(), "queue not empty"
-    for w in workers:
-        w.cancel()
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.asyncio
-async def test_4_multiple_producers_consumers(
-    test_interablequeue_int: IterableQueue[int],
-):
-    Q = test_interablequeue_int
-    producers: list[Task] = list()
-    consumers: list[Task] = list()
-
-    for _ in range(THREADS):
-        producers.append(create_task(_producer_int(Q, N, finish=False, wait=0.05)))
-        consumers.append(create_task(_consumer_int(Q, 2 * N, wait=0.06)))
-    try:
-        async with timeout(10):
-            await gather(*producers)
-            await Q.finish(all=True)
-            await Q.join()
-            assert not Q.has_wip, "Queue should not have any items WIP"
-    except TimeoutError:
-        assert False, "IterableQueue.join() took too long"
-    assert Q.count == THREADS * N, (
-        f"count returned wrong value {Q.count}, should be {THREADS * N}"
-    )
-    assert Q.qsize() == 0, "queue size is > 0 even it should be empty"
-    assert Q.empty(), "queue not empty"
-    for p in consumers:
-        p.cancel()
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.asyncio
-async def test_5_empty_join(test_interablequeue_int: IterableQueue[int]):
-    """Test for await join when an empty queue is finished"""
-    Q = test_interablequeue_int
-    producer: Task = create_task(_producer_int(Q, n=0, finish=True, wait=2))
-    assert not Q.is_done, "is_done returned True even queue is not finished"
-    consumer: Task = create_task(_consumer_int(Q))
-    try:
-        async with timeout(3):
-            await Q.join()
-        assert Q.empty(), (
-            "Queue is done after 3 secs and the join() should finish before timeout(5)"
-        )
-    except TimeoutError:
-        assert False, "await IterableQueue.join() failed with an empty queue finished"
-    await sleep(0.1)
-
-    try:
-        consumer.cancel()
-        assert not consumer.cancelled(), (
-            "consumer task was cancelled and did not complete even it should have"
-        )
-    except Exception as err:
-        assert False, f"Unknown Exception caught: {err}"
-    assert producer.done(), "producer has not finished"
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.asyncio
-async def test_6_finish_full_queue(test_interablequeue_int: IterableQueue[int]):
-    """Test for await join when an empty queue is finished"""
-    Q = test_interablequeue_int
-    producer: Task = create_task(_producer_int(Q, n=QSIZE * 2))
-    try:
-        await sleep(0.5)
-        async with timeout(3):
-            await Q.finish(all=True, empty=True)
-        assert Q.empty(), (
-            f"Queue should be empty: qsize={Q._Q.qsize()}: {Q._Q.get_nowait()}, {Q._Q.get_nowait()}"
-        )
-        assert Q.is_done, "Queue is not done"
-    except TimeoutError:
-        assert False, "await IterableQueue.join() failed with an empty queue finished"
-    await sleep(0.1)
-    assert Q.is_done, "Queue is not done"
-    producer.cancel()
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.asyncio
-async def test_7_aiter(test_interablequeue_int: IterableQueue[int]):
-    """Test for await join when an empty queue is finished"""
-    Q = test_interablequeue_int
-    await _producer_int(Q, n=QSIZE - 1, finish=True)
-
-    try:
-        await sleep(0.5)
-        async for i in Q:
-            assert i >= 0, "Did not receive an int"
-        assert Q.is_done, "Queue is not done"
-        # assert (
-        #     True
-        # ), "Queue is done after 3 secs and the join() should finish before timeout(5)"
-    except TimeoutError:
-        assert False, "await IterableQueue.join() failed with an empty queue finished"
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.asyncio
-async def test_8_aiter_1_item(test_interablequeue_int: IterableQueue[int]):
-    """Test for await join when an empty queue is finished"""
-    Q = test_interablequeue_int
-    await _producer_int(Q, n=1, finish=True)
-
-    try:
-        assert Q.qsize() == 1, f"incorrect queue length {Q.qsize()} != 1"
-        await sleep(0.5)
-        count: int = 0
-        async for i in Q:
-            count += 1
-            assert i >= 0, "Did not receive an int"
-        assert count == 1, f"Did not receive correct number of elements {count} != 1"
-        assert True, (
-            "Queue is done after 3 secs and the join() should finish before timeout(5)"
-        )
-    except TimeoutError:
-        assert False, "await IterableQueue.join() failed with an empty queue finished"
-
-
-@pytest.mark.parametrize(
-    "cats,N, producers",
-    [
-        ([randomword(5) for _ in range(10)], 1000, 1),
-        ([randomword(5) for _ in range(20)], 10000, 1),
-        ([randomword(5) for _ in range(5)], 1000, 3),
-    ],
-)
-@pytest.mark.timeout(10)
-@pytest.mark.asyncio
-async def test_9_category_counter_queue(
-    cats: list[str], N: int, producers: int
-) -> None:
-    """Test CategoryCounterQueue"""
-    Q = CategoryCounterQueue(maxsize=QSIZE)
-
-    async def producer(
-        Q: CategoryCounterQueue, cats: list[str], N: int = 100
-    ) -> defaultdict[str, int]:
-        """
-        Test Producer for CategoryCounterQueue
-        """
-        _counter: defaultdict[str, int] = defaultdict(int)
-        await Q.add_producer()
-        for _ in range(N):
-            cat: str = choice(cats)
-            count: int = randint(1, 10)
-            await Q.send(cat, count)
-            _counter[cat] += count
-        await Q.finish()
-        return _counter
-
-    senders: list[Task] = list()
-
-    for _ in range(producers):
-        senders.append(create_task(producer(Q, cats, N)))
-
-    try:
-        res_in: defaultdict[str, int] = await Q.listen()
-        res_out: defaultdict[str, int] = defaultdict(int)
-        for res in await gather(*senders):
-            for cat, count in res.items():
-                res_out[cat] += count
-
-        assert res_in == res_out, f"CategoryCounterQueue: {res_in} != {res_out}"
-        assert Q.qsize() == 0, "queue size is > 0 even it should be empty"
-        assert Q.empty(), "queue not empty"
-        assert Q.count == N * producers, (
-            f"count returned wrong value {Q.count}, should be {N * producers}"
-        )
-
-    except TimeoutError:
-        assert False, "await IterableQueue.join() failed with an empty queue finished"