bulk-chain 1.1.0__tar.gz → 1.2.1__tar.gz

bulk_chain-1.2.1/PKG-INFO

@@ -0,0 +1,134 @@
+Metadata-Version: 2.1
+Name: bulk_chain
+Version: 1.2.1
+Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
+Home-page: https://github.com/nicolay-r/bulk-chain
+Author: Nicolay Rusnachenko
+Author-email: rusnicolay@gmail.com
+License: MIT License
+Description: # bulk-chain 1.2.1
+        ![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
+        [![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
+        [![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
+        [![PyPI downloads](https://img.shields.io/pypi/dm/bulk-chain.svg)](https://pypistats.org/packages/bulk-chain)
+        
+        <p align="center">
+            <img src="logo.png"/>
+        </p>
+        
+        <p align="center">
+          <a href="https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm"><b>Third-party providers hosting</b>↗️</a>
+          <br>
+          <a href="https://github.com/nicolay-r/bulk-chain-shell">👉<b>demo</b>👈</a>
+        </p>
+        
+        A no-strings-attached **framework**  for your LLM that allows applying Chain-of-Thought-alike [prompt `schema`](#chain-of-thought-schema) towards a massive textual collections using custom **[third-party providers ↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm)**.
+        
+        ### Main Features
+        * ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
+        * ✅ **Support schemas descriptions** for Chain-of-Thought concept.
+        * ✅ **Provides iterator over infinite amount of input contexts**
+        
+        # Installation
+        
+        From PyPI: 
+        
+        ```bash
+        pip install --no-deps bulk-chain
+        ```
+        
+        or latest version from here:
+        
+        ```bash
+        pip install git+https://github.com/nicolay-r/bulk-chain@master
+        ```
+        
+        ## Chain-of-Thought Schema
+        
+        To declare Chain-of-Though (CoT) schema we use `JSON` format. 
+        The field `schema` is a list of CoT instructions for the Large Language Model.
+        Each item of the list represent a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
+        All the variable names should be mentioned in `{}`.
+        
+        **Example**:
+        ```python
+        [
+          {"prompt": "extract topic: {text}", "out": "topic"},
+          {"prompt": "extract subject: {text}", "out": "subject"},
+        ]
+        ```
+        
+        # Usage
+        
+        ## 🤖 Prepare 
+        
+        1. [schema](#chain-of-thought-schema)
+            * [Example for Sentiment Analysis](test/schema/thor_cot_schema.json)
+        2. **LLM model** from the [<b>Third-party providers hosting</b>↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
+        3. Data (iter of dictionaries)
+        
+        ## 🚀 Launch
+        
+        > **API**: For more details see the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
+        
+        ```python
+        from bulk_chain.core.utils import dynamic_init
+        from bulk_chain.api import iter_content
+        
+        content_it = iter_content(
+            # 1. Your schema.              
+            schema=[
+              {"prompt": "extract topic: {text}", "out": "topic" },
+              {"prompt": "extract subject: {text}", "out": "subject"},
+            ],
+            # 2. Your third-party model implementation.
+            llm=dynamic_init(class_filepath="replicate_104.py")(
+               api_token="<API-KEY>",
+               model_name="meta/meta-llama-3-70b-instruct"),
+            # 3. Toggle streaming if needed
+            stream=False,
+            # 4. Toggle Async API mode usage.
+            async_mode=True,
+            # 5. Batch size.
+            batch_size=10,
+            # 6. Your iterator of dictionaries
+            input_dicts_it=[
+                # Example of data ...
+                { "text": "Rocks are hard" },
+                { "text": "Water is wet" },
+                { "text": "Fire is hot" }
+            ],
+        )
+            
+        for batch in content_it:
+           for entry in batch:
+              print(entry)
+        ```
+        
+        Outputs entries represent texts augmented with `topic` and `subject`:
+        ```jsonl
+        {'text': 'Rocks are hard', 'topic': 'The topic is: Geology/Rocks', 'subject': 'The subject is: "Rocks"'}
+        {'text': 'Water is wet', 'topic': 'The topic is: Properties of Water', 'subject': 'The subject is: Water'}
+        {'text': 'Fire is hot', 'topic': 'The topic is: Temperature/Properties of Fire', 'subject': 'The subject is: "Fire"'}
+        ```
+        
+        # API
+        
+        | Method               | Mode       | Description                                                         |
+        |----------------------|------------|---------------------------------------------------------------------|
+        | `ask(prompt)`        | Sync       | Infers the model with a single prompt.                              |
+        | `ask_stream(prompt)` | Sync       | Returns a generator that yields chunks of the inferred result.      |
+        | `ask_async(prompt)`  | Async      | Asynchronously infers the model with a single prompt.               |
+        | `ask_stream_async(prompt)` | Async | Asynchronously returns a generator of result chunks of the inferred result.          |
+        
+        See examples with models [at nlp-thirdgate 🌌](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
+        
+Keywords: natural language processing,chain-of-thought,reasoning
+Platform: UNKNOWN
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/README.md

@@ -1,4 +1,4 @@
-# bulk-chain 1.1.0
+# bulk-chain 1.2.1
 ![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
 [![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
 [![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
@@ -37,24 +37,17 @@ pip install git+https://github.com/nicolay-r/bulk-chain@master
 
 ## Chain-of-Thought Schema
 
-To declare Chain-of-Though (CoT) schema, this project exploits `JSON` format.
-This format adopts `name` field for declaring a name and `schema` is a list of CoT instructions for the Large Language Model.
-
-Each step represents a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
-All the variable names are expected to be mentioned in `{}`.
-
-Below, is an example on how to declare your own schema:
+To declare Chain-of-Though (CoT) schema we use `JSON` format. 
+The field `schema` is a list of CoT instructions for the Large Language Model.
+Each item of the list represent a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
+All the variable names should be mentioned in `{}`.
 
+**Example**:
 ```python
-{
-"name": "schema-name",
-"schema": [
-    {"prompt": "Given the question '{text}', let's think step-by-step.", 
-     "out": "steps"},
-    {"prompt": "For the question '{text}' the reasoining steps are '{steps}'. what would be an answer?", 
-     "out":  "answer"},
+[
+  {"prompt": "extract topic: {text}", "out": "topic"},
+  {"prompt": "extract subject: {text}", "out": "subject"},
 ]
-}
 ```
 
 # Usage
@@ -76,25 +69,48 @@ from bulk_chain.api import iter_content
 
 content_it = iter_content(
     # 1. Your schema.              
-    schema="YOUR_SCHEMA.json",
+    schema=[
+      {"prompt": "extract topic: {text}", "out": "topic" },
+      {"prompt": "extract subject: {text}", "out": "subject"},
+    ],
     # 2. Your third-party model implementation.
-    llm=dynamic_init(class_filepath="replicate_104.py", class_name="Replicate")(api_token="<API-KEY>"),
-    # 3. Customize your inference and result providing modes: 
-    infer_mode="batch_async", 
-    return_mode="batch",
-    # 4. Your iterator of dictionaries
-    input_dicts_it=YOUR_DATA_IT,
+    llm=dynamic_init(class_filepath="replicate_104.py")(
+       api_token="<API-KEY>",
+       model_name="meta/meta-llama-3-70b-instruct"),
+    # 3. Toggle streaming if needed
+    stream=False,
+    # 4. Toggle Async API mode usage.
+    async_mode=True,
+    # 5. Batch size.
+    batch_size=10,
+    # 6. Your iterator of dictionaries
+    input_dicts_it=[
+        # Example of data ...
+        { "text": "Rocks are hard" },
+        { "text": "Water is wet" },
+        { "text": "Fire is hot" }
+    ],
 )
     
-for content in content_it:
-    # Handle your LLM responses here ...
+for batch in content_it:
+   for entry in batch:
+      print(entry)
 ```
 
+Outputs entries represent texts augmented with `topic` and `subject`:
+```jsonl
+{'text': 'Rocks are hard', 'topic': 'The topic is: Geology/Rocks', 'subject': 'The subject is: "Rocks"'}
+{'text': 'Water is wet', 'topic': 'The topic is: Properties of Water', 'subject': 'The subject is: Water'}
+{'text': 'Fire is hot', 'topic': 'The topic is: Temperature/Properties of Fire', 'subject': 'The subject is: "Fire"'}
+```
 
-# Embed your LLM
+# API
 
-All you have to do is to implement `BaseLM` class, that includes:
-* `__init__` -- for setting up *batching mode support* and (optional) *model name*;
-* `ask(prompt)` -- infer your model with the given `prompt`.
+| Method               | Mode       | Description                                                         |
+|----------------------|------------|---------------------------------------------------------------------|
+| `ask(prompt)`        | Sync       | Infers the model with a single prompt.                              |
+| `ask_stream(prompt)` | Sync       | Returns a generator that yields chunks of the inferred result.      |
+| `ask_async(prompt)`  | Async      | Asynchronously infers the model with a single prompt.               |
+| `ask_stream_async(prompt)` | Async | Asynchronously returns a generator of result chunks of the inferred result.          |
 
 See examples with models [at nlp-thirdgate 🌌](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/bulk_chain/api.py

@@ -3,6 +3,7 @@ import collections
 import logging
 import os
 from itertools import chain
+from types import AsyncGeneratorType
 
 from bulk_chain.core.llm_base import BaseLM
 from bulk_chain.core.service_asyncio import AsyncioService
@@ -16,8 +17,8 @@ from bulk_chain.core.utils import attempt_wrapper
 
 INFER_MODES = {
     "single": lambda llm, batch, **kwargs: [llm.ask(prompt) for prompt in batch],
+    "batch": lambda llm, batch, **kwargs: llm.ask_batch(batch),
     "single_stream": lambda llm, batch, **kwargs: [llm.ask_stream(prompt) for prompt in batch],
-    "batch": lambda llm, batch, **kwargs: llm.ask(batch),
     "batch_async": lambda llm, batch, **kwargs: AsyncioService.run_tasks(
         batch=batch, async_handler=llm.ask_async, event_loop=kwargs.get("event_loop")
     ),
@@ -69,6 +70,9 @@ def __handle_gen(handle, batch, event_loop):
         elif isinstance(entry, collections.abc.Iterable):
             for chunk in map(lambda item: str(item), entry):
                 yield chunk
+        elif isinstance(entry, AsyncGeneratorType):
+            for chunk in AsyncioService.async_gen_to_iter(entry, loop=event_loop):
+                yield str(chunk)
         else:
             raise Exception(f"Non supported type `{type(entry)}` for handling output from batch")
 
@@ -85,11 +89,14 @@ def _iter_chunks(p_column, batch_content_it, **kwargs):
         yield ind_in_batch, chunk
 
 
-def _infer_batch(batch, batch_ind, schema, return_mode, cols=None, **kwargs):
+def _column_ordered_chunks_iter(batch, schema, cols=None, keep_prompts=True, **kwargs):
+    """
+    NOTE: we populate `batch` content automatically
+    """
     assert (isinstance(batch, list))
 
     if len(batch) == 0:
-        return batch
+        return
 
     if cols is None:
         first_item = batch[0]
@@ -112,33 +119,66 @@ def _infer_batch(batch, batch_ind, schema, return_mode, cols=None, **kwargs):
             for ind_in_batch, chunk in content_it:
                 # Append batch.
                 batch[ind_in_batch][c].append(chunk)
-                # Returning (optional).
-                if return_mode == "chunk":
-                    global_ind = batch_ind * len(batch) + ind_in_batch
-                    yield [global_ind, c, chunk]
+                yield [ind_in_batch, c, chunk]
 
             # Convert content to string.
             for item in batch:
                 item[c] = "".join(item[c])
 
-    if return_mode == "record":
+    if not keep_prompts:
+        for batch_item in batch:
+            for key in list(batch_item.keys()):
+                prompt_col = SchemaService.col_to_prompt(col_name=key, prompt_data=batch_item)
+                if prompt_col in batch_item:
+                    del batch_item[prompt_col]
+
+
+def _infer_batch(return_type, batch, batch_ind, **kwargs):
+    assert (return_type in ["batch", "chunk", "record"])
+
+    # Filling batch with inference content.
+    for ind_in_batch, column, chunk in _column_ordered_chunks_iter(batch=batch, **kwargs):
+        if return_type == "chunk":
+            global_ind = batch_ind * len(batch) + ind_in_batch
+            yield [global_ind, column, chunk]
+
+    if return_type == "record":
         for record in batch:
             yield record
 
-    if return_mode == "batch":
+    if return_type == "batch":
         yield batch
 
 
+def get_infer_mode(stream, batch_size, async_mode):
+    if not stream and batch_size == 1:
+        return 'single', 'record'
+    elif not stream and batch_size > 1:
+        if async_mode:
+            return 'batch_async', 'batch'
+        else:
+            return 'batch', 'batch'
+    elif stream and batch_size == 1:
+        return 'single_stream', 'chunk'
+    elif stream and batch_size > 1:
+        return 'batch_stream_async', 'chunk'
+
+    raise ValueError(f"Invalid combination of stream and batch_size: {stream}, {batch_size}")
+
+
 def iter_content(input_dicts_it, llm, schema, batch_size=1, limit_prompt=None,
-                 infer_mode="batch", return_mode="batch", attempts=1, event_loop=None,
-                 **kwargs):
+                 stream=False, async_mode=False, attempts=1, event_loop=None,
+                 handle_missed_value_func=lambda *_: None, **kwargs):
     """ This method represent Python API aimed at application of `llm` towards
         iterator of input_dicts via cache_target that refers to the SQLite using
         the given `schema`
     """
-    assert (infer_mode in INFER_MODES.keys())
-    assert (return_mode in ["batch", "chunk", "record"])
     assert (isinstance(llm, BaseLM))
+    assert (isinstance(batch_size, int) and batch_size > 0)
+    assert (isinstance(async_mode, bool)) 
+
+    infer_type, return_type = get_infer_mode(stream=stream, batch_size=batch_size, async_mode=async_mode)
+    infer_mode = INFER_MODES[infer_type]
 
     # Setup event loop.
     event_loop = asyncio.get_event_loop_policy().get_event_loop() \
@@ -149,13 +189,15 @@ def iter_content(input_dicts_it, llm, schema, batch_size=1, limit_prompt=None,
         schema = JsonService.read(schema)
     if isinstance(schema, dict):
         schema = SchemaService(json_data=schema)
+    if isinstance(schema, list):
+        schema = SchemaService(json_data={"schema": schema})
 
     prompts_it = map(
         lambda data: DictionaryService.custom_update(src_dict=dict(data), other_dict=schema.cot_args),
         input_dicts_it
     )
 
-    handle_batch_func = lambda batch, **handle_kwargs: INFER_MODES[infer_mode](
+    handle_batch_func = lambda batch, **handle_kwargs: infer_mode(
         llm,
         DataService.limit_prompts(batch, limit=limit_prompt),
         **handle_kwargs
@@ -172,12 +214,13 @@ def iter_content(input_dicts_it, llm, schema, batch_size=1, limit_prompt=None,
                                       logger=logger)
         handle_batch_func = attempt_dec(handle_batch_func)
 
-    content_it = (_infer_batch(batch=batch,
+    kwargs["handle_missed_value_func"] = handle_missed_value_func
+
+    content_it = (_infer_batch(return_type=return_type,
+                               batch=batch,
                                batch_ind=batch_ind,
                                infer_mode=infer_mode,
                                handle_batch_func=handle_batch_func,
-                               handle_missed_value_func=lambda *_: None,
-                               return_mode=return_mode,
                                schema=schema,
                                event_loop=event_loop,
                                **kwargs)

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/bulk_chain/core/llm_base.py

@@ -3,12 +3,17 @@ class BaseLM(object):
     def __init__(self, **kwargs):
         pass
 
-    def ask(self, content):
+    def ask(self, prompt):
         """ Assumes to return str.
         """
         raise NotImplemented()
 
-    def ask_stream(self, content):
+    def ask_batch(self, batch):
+        """ Assumes to return generator.
+        """
+        raise NotImplemented()
+
+    def ask_stream(self, prompt):
         """ Assumes to return generator.
         """
         raise NotImplemented()
@@ -18,7 +23,7 @@ class BaseLM(object):
         """
         raise NotImplemented()
 
-    async def ask_stream_async(self, batch):
+    async def ask_stream_async(self, prompt):
         """ Assumes to return AsyncGenerator.
         """
         raise NotImplemented()

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/bulk_chain/core/service_schema.py

@@ -9,6 +9,10 @@ class SchemaService(object):
         prompt_schema = {"schema": [{"prompt": prompt, "out": "response", "in": "prompt"}]}
         return cls(prompt_schema)
 
+    @staticmethod
+    def col_to_prompt(col_name, prompt_data):
+        return col_name + "_prompt" if "in" not in prompt_data else prompt_data["in"]
+
     @staticmethod
     def __init_schema(prompts):
 
@@ -19,7 +23,7 @@ class SchemaService(object):
 
         for prompt in prompts:
             r_col_name = prompt["out"]
-            p_col_name = r_col_name + "_prompt" if "in" not in prompt else prompt["in"]
+            p_col_name = SchemaService.col_to_prompt(col_name=r_col_name, prompt_data=prompt)
 
             assert r_col_name not in schema_r2p, f"`{r_col_name}` has been already declared!"
             assert p_col_name not in schema_p2r, f"`{p_col_name}` has been already declared!"

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/bulk_chain/core/utils.py

@@ -1,3 +1,4 @@
+import ast
 import importlib
 import logging
 import sys
@@ -35,18 +36,30 @@ def find_by_prefix(d, key):
     return d[matches[0]]
 
 
+def check_is_param_name(param_name):
+    return param_name.replace("_", "").isalpha()
+
+
 def iter_params(text):
     assert(isinstance(text, str))
     beg = 0
     while beg < len(text):
+        print(beg)
         try:
             pb = text.index('{', beg)
         except ValueError:
             break
-        pe = text.index('}', beg+1)
-        # Yield argument.
-        yield text[pb+1:pe]
-        beg = pe+1
+        pe = text.index('}', pb+1)
+        param_name = text[pb + 1:pe]
+
+        # Check parameter validity.
+        if not check_is_param_name(param_name):
+            beg = pb + 1
+            continue
+
+        # Passing.
+        yield param_name
+        beg = pe + 1
 
 
 def auto_import(name, is_class=False):
@@ -61,6 +74,17 @@ def auto_import(name, is_class=False):
     return m() if is_class else m
 
 
+def get_class_name(file_path):
+    with open(file_path, 'r') as f:
+        tree = ast.parse(f.read(), filename=file_path)
+
+    for node in ast.walk(tree):
+        if isinstance(node, ast.ClassDef):
+            return node.name
+
+    return None
+
+
 def dynamic_init(class_filepath, class_name=None):
 
     # Registering path.
@@ -75,7 +99,7 @@ def dynamic_init(class_filepath, class_name=None):
         class_filename = class_filename[:-len(".py")]
 
     # Loading library.
-    class_name = class_path_list[-1].title() if class_name is None else class_name
+    class_name = get_class_name(class_filepath) if class_name is None else class_name
     class_path = ".".join([class_filename, class_name])
     logger.info(f"Dynamic loading for the file and class `{class_path}`")
     cls = auto_import(class_path, is_class=False)

bulk_chain-1.2.1/bulk_chain.egg-info/PKG-INFO

@@ -0,0 +1,134 @@
+Metadata-Version: 2.1
+Name: bulk-chain
+Version: 1.2.1
+Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
+Home-page: https://github.com/nicolay-r/bulk-chain
+Author: Nicolay Rusnachenko
+Author-email: rusnicolay@gmail.com
+License: MIT License
+Description: # bulk-chain 1.2.1
+        ![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
+        [![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
+        [![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
+        [![PyPI downloads](https://img.shields.io/pypi/dm/bulk-chain.svg)](https://pypistats.org/packages/bulk-chain)
+        
+        <p align="center">
+            <img src="logo.png"/>
+        </p>
+        
+        <p align="center">
+          <a href="https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm"><b>Third-party providers hosting</b>↗️</a>
+          <br>
+          <a href="https://github.com/nicolay-r/bulk-chain-shell">👉<b>demo</b>👈</a>
+        </p>
+        
+        A no-strings-attached **framework**  for your LLM that allows applying Chain-of-Thought-alike [prompt `schema`](#chain-of-thought-schema) towards a massive textual collections using custom **[third-party providers ↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm)**.
+        
+        ### Main Features
+        * ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
+        * ✅ **Support schemas descriptions** for Chain-of-Thought concept.
+        * ✅ **Provides iterator over infinite amount of input contexts**
+        
+        # Installation
+        
+        From PyPI: 
+        
+        ```bash
+        pip install --no-deps bulk-chain
+        ```
+        
+        or latest version from here:
+        
+        ```bash
+        pip install git+https://github.com/nicolay-r/bulk-chain@master
+        ```
+        
+        ## Chain-of-Thought Schema
+        
+        To declare Chain-of-Though (CoT) schema we use `JSON` format. 
+        The field `schema` is a list of CoT instructions for the Large Language Model.
+        Each item of the list represent a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
+        All the variable names should be mentioned in `{}`.
+        
+        **Example**:
+        ```python
+        [
+          {"prompt": "extract topic: {text}", "out": "topic"},
+          {"prompt": "extract subject: {text}", "out": "subject"},
+        ]
+        ```
+        
+        # Usage
+        
+        ## 🤖 Prepare 
+        
+        1. [schema](#chain-of-thought-schema)
+            * [Example for Sentiment Analysis](test/schema/thor_cot_schema.json)
+        2. **LLM model** from the [<b>Third-party providers hosting</b>↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
+        3. Data (iter of dictionaries)
+        
+        ## 🚀 Launch
+        
+        > **API**: For more details see the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
+        
+        ```python
+        from bulk_chain.core.utils import dynamic_init
+        from bulk_chain.api import iter_content
+        
+        content_it = iter_content(
+            # 1. Your schema.              
+            schema=[
+              {"prompt": "extract topic: {text}", "out": "topic" },
+              {"prompt": "extract subject: {text}", "out": "subject"},
+            ],
+            # 2. Your third-party model implementation.
+            llm=dynamic_init(class_filepath="replicate_104.py")(
+               api_token="<API-KEY>",
+               model_name="meta/meta-llama-3-70b-instruct"),
+            # 3. Toggle streaming if needed
+            stream=False,
+            # 4. Toggle Async API mode usage.
+            async_mode=True,
+            # 5. Batch size.
+            batch_size=10,
+            # 6. Your iterator of dictionaries
+            input_dicts_it=[
+                # Example of data ...
+                { "text": "Rocks are hard" },
+                { "text": "Water is wet" },
+                { "text": "Fire is hot" }
+            ],
+        )
+            
+        for batch in content_it:
+           for entry in batch:
+              print(entry)
+        ```
+        
+        Outputs entries represent texts augmented with `topic` and `subject`:
+        ```jsonl
+        {'text': 'Rocks are hard', 'topic': 'The topic is: Geology/Rocks', 'subject': 'The subject is: "Rocks"'}
+        {'text': 'Water is wet', 'topic': 'The topic is: Properties of Water', 'subject': 'The subject is: Water'}
+        {'text': 'Fire is hot', 'topic': 'The topic is: Temperature/Properties of Fire', 'subject': 'The subject is: "Fire"'}
+        ```
+        
+        # API
+        
+        | Method               | Mode       | Description                                                         |
+        |----------------------|------------|---------------------------------------------------------------------|
+        | `ask(prompt)`        | Sync       | Infers the model with a single prompt.                              |
+        | `ask_stream(prompt)` | Sync       | Returns a generator that yields chunks of the inferred result.      |
+        | `ask_async(prompt)`  | Async      | Asynchronously infers the model with a single prompt.               |
+        | `ask_stream_async(prompt)` | Async | Asynchronously returns a generator of result chunks of the inferred result.          |
+        
+        See examples with models [at nlp-thirdgate 🌌](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
+        
+Keywords: natural language processing,chain-of-thought,reasoning
+Platform: UNKNOWN
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/bulk_chain.egg-info/SOURCES.txt

@@ -1,4 +1,3 @@
-LICENSE
 README.md
 setup.py
 bulk_chain/__init__.py
@@ -18,6 +17,7 @@ bulk_chain/core/service_schema.py
 bulk_chain/core/utils.py
 test/test_api.py
 test/test_api_batching.py
+test/test_api_novita.py
 test/test_api_streaming.py
 test/test_replicate_async_baseline.py
 test/test_replicate_async_batch_async.py

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/setup.py

@@ -15,7 +15,7 @@ def get_requirements(filenames):
 
 setup(
     name='bulk_chain',
-    version='1.1.0',
+    version='1.2.1',
     python_requires=">=3.6",
     description='A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, '
                 'ensuring reliable results for bulk input requests.',

bulk_chain-1.2.1/test/test_api.py

@@ -0,0 +1,64 @@
+import unittest
+from os.path import join
+
+from bulk_chain.api import iter_content
+from utils import current_dir, default_remote_llm
+
+
+class TestAPI(unittest.TestCase):
+
+    llm = default_remote_llm()
+
+    @staticmethod
+    def it_data(n):
+        for i in range(n):
+            yield {"ind": i, "text": "X invent sanctions against Y", "entity": "X"}
+
+    def test_single(self):
+        data_it = iter_content(input_dicts_it=self.it_data(20),
+                               llm=self.llm,
+                               batch_size=1,
+                               schema=join(current_dir, "schema/default.json"))
+
+        for data in data_it:
+            print(data)
+
+    def test_single_stream(self):
+        """ Returns individual chunks.
+        """
+        chunk_it = iter_content(input_dicts_it=self.it_data(20),
+                                llm=self.llm,
+                                batch_size=1,
+                                stream=True,
+                                schema=join(current_dir, "schema/default.json"))
+
+        for data in chunk_it:
+            print(data)
+
+    def test_batch_async(self):
+        """ Return batches that passed async at the Replicate.
+        """
+        batch_it = iter_content(input_dicts_it=self.it_data(20),
+                                llm=self.llm,
+                                batch_size=5,
+                                async_mode=True,
+                                schema=join(current_dir, "schema/default.json"))
+
+        for batch in batch_it:
+            for item in batch:
+                print(item)
+
+    def test_batch_stream_async(self):
+        chunk_it = iter_content(input_dicts_it=self.it_data(20),
+                                llm=self.llm,
+                                batch_size=5,
+                                stream=True,
+                                async_mode=True,
+                                schema=join(current_dir, "schema/default.json"))
+
+        for chunk_info in chunk_it:
+            print(chunk_info)
+
+
+if __name__ == '__main__':
+    unittest.main()

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/test/test_api_batching.py

@@ -9,15 +9,14 @@ from utils import iter_test_jsonl_samples
 
 class TestProviderBatching(unittest.TestCase):
 
-    llm = dynamic_init(class_filepath="providers/transformers_flan_t5.py",
-                       class_name="FlanT5")(model_name="nicolay-r/flan-t5-tsa-thor-base",
-                                            max_new_tokens=128)
+    llm = dynamic_init(class_filepath="providers/transformers_flan_t5.py")(
+        model_name="nicolay-r/flan-t5-tsa-thor-base",
+        max_new_tokens=128)
 
     def test_iter(self):
         input_dicts_it = iter_test_jsonl_samples()
         data_it = iter_content(input_dicts_it=input_dicts_it,
                                llm=self.llm,
-                               infer_mode="batch",
                                batch_size=10,
                                return_batch=False,
                                schema="schema/thor_cot_schema.json")

bulk_chain-1.2.1/test/test_api_novita.py

@@ -0,0 +1,34 @@
+from bulk_chain.api import iter_content
+from bulk_chain.core.utils import dynamic_init
+
+
+# Tested under: https://github.com/nicolay-r/nlp-thirdgate/blob/master/llm/openai_156.py
+provider = dynamic_init(class_filepath="providers/openai_156.py")(
+    base_url="https://api.novita.ai/openai",
+    api_token="sk_qpuGSSuz4sF3PgWXg72wstp6-ojtYt1rX3pdXTVocPU",
+    model_name="meta-llama/llama-3.3-70b-instruct")
+
+content_it = iter_content(
+    # 1. Your schema.              
+    schema=[
+      {"prompt": "extract topic: {text}", "out": "topic" },
+      {"prompt": "extract subject: {text}", "out": "subject"},
+    ],
+    # 2. Your third-party model implementation.
+    llm=provider,
+    batch_size=10,
+    # 3. Toggle streaming if needed
+    stream=False,
+    # 4. Toggle asynchronous mode if needed
+    async_mode=True,
+    # 5. Your iterator of dictionaries
+    input_dicts_it=[
+        { "text": "Rocks are hard" },
+        { "text": "Water is wet" },
+        { "text": "Fire is hot" }
+    ],
+)
+
+for batch in content_it:
+    for entry in batch:
+        print(entry)

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/test/test_api_streaming.py

@@ -1,7 +1,7 @@
 import unittest
 
 from bulk_chain.api import iter_content
-from utils import iter_test_jsonl_samples, DEFAULT_REMOTE_LLM
+from utils import iter_test_jsonl_samples, default_remote_llm
 
 
 class TestAPI_Streaming(unittest.TestCase):
@@ -10,10 +10,9 @@ class TestAPI_Streaming(unittest.TestCase):
 
         input_dicts_it = iter_test_jsonl_samples()
         data_it = iter_content(input_dicts_it=input_dicts_it,
-                               llm=DEFAULT_REMOTE_LLM,
+                               llm=default_remote_llm(),
                                batch_size=1,
-                               infer_mode="single_stream",
-                               return_mode="chunk",
+                               stream=True,
                                attempts=2,
                                schema="schema/thor_cot_schema.json")
 

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/test/test_replicate_async_baseline.py

@@ -1,8 +1,11 @@
 from timeit import default_timer as timer
-from utils import DEFAULT_REMOTE_LLM
+from utils import default_remote_llm
+
+
+llm = default_remote_llm()
 
 start = timer()
-r = ["".join([str(s) for s in DEFAULT_REMOTE_LLM.ask(f"what's the color of the {p}")])
+r = ["".join([str(s) for s in llm.ask(f"what's the color of the {p}")])
      for p in ["sky", "ground", "water"]]
 end = timer()
 

{bulk_chain-1.1.0 → bulk_chain-1.2.1}/test/test_replicate_async_batch_async.py

@@ -1,12 +1,15 @@
 from timeit import default_timer as timer
 import asyncio
 
-from utils import DEFAULT_REMOTE_LLM
+from utils import default_remote_llm
+
+
+llm = default_remote_llm()
 
 
 async def infer_item(prompt):
     content = []
-    for chunk in DEFAULT_REMOTE_LLM.ask(prompt):
+    for chunk in llm.ask(prompt):
         content.append(str(chunk))
     return content
 

bulk_chain-1.1.0/LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2024 Nicolay Rusnachenko
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

bulk_chain-1.1.0/PKG-INFO

@@ -1,118 +0,0 @@
-Metadata-Version: 2.1
-Name: bulk_chain
-Version: 1.1.0
-Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
-Home-page: https://github.com/nicolay-r/bulk-chain
-Author: Nicolay Rusnachenko
-Author-email: rusnicolay@gmail.com
-License: MIT License
-Keywords: natural language processing,chain-of-thought,reasoning
-Classifier: Programming Language :: Python
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Classifier: Topic :: Scientific/Engineering :: Information Analysis
-Classifier: Topic :: Text Processing :: Linguistic
-Requires-Python: >=3.6
-Description-Content-Type: text/markdown
-License-File: LICENSE
-
-# bulk-chain 1.1.0
-![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
-[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
-[![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
-[![PyPI downloads](https://img.shields.io/pypi/dm/bulk-chain.svg)](https://pypistats.org/packages/bulk-chain)
-
-<p align="center">
-    <img src="logo.png"/>
-</p>
-
-<p align="center">
-  <a href="https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm"><b>Third-party providers hosting</b>↗️</a>
-  <br>
-  <a href="https://github.com/nicolay-r/bulk-chain-shell">👉<b>demo</b>👈</a>
-</p>
-
-A no-strings-attached **framework**  for your LLM that allows applying Chain-of-Thought-alike [prompt `schema`](#chain-of-thought-schema) towards a massive textual collections using custom **[third-party providers ↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm)**.
-
-### Main Features
-* ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
-* ✅ **Support schemas descriptions** for Chain-of-Thought concept.
-* ✅ **Provides iterator over infinite amount of input contexts**
-
-# Installation
-
-From PyPI: 
-
-```bash
-pip install --no-deps bulk-chain
-```
-
-or latest version from here:
-
-```bash
-pip install git+https://github.com/nicolay-r/bulk-chain@master
-```
-
-## Chain-of-Thought Schema
-
-To declare Chain-of-Though (CoT) schema, this project exploits `JSON` format.
-This format adopts `name` field for declaring a name and `schema` is a list of CoT instructions for the Large Language Model.
-
-Each step represents a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
-All the variable names are expected to be mentioned in `{}`.
-
-Below, is an example on how to declare your own schema:
-
-```python
-{
-"name": "schema-name",
-"schema": [
-    {"prompt": "Given the question '{text}', let's think step-by-step.", 
-     "out": "steps"},
-    {"prompt": "For the question '{text}' the reasoining steps are '{steps}'. what would be an answer?", 
-     "out":  "answer"},
-]
-}
-```
-
-# Usage
-
-## 🤖 Prepare 
-
-1. [schema](#chain-of-thought-schema)
-    * [Example for Sentiment Analysis](test/schema/thor_cot_schema.json)
-2. **LLM model** from the [<b>Third-party providers hosting</b>↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
-3. Data (iter of dictionaries)
-
-## 🚀 Launch
-
-> **API**: For more details see the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
-
-```python
-from bulk_chain.core.utils import dynamic_init
-from bulk_chain.api import iter_content
-
-content_it = iter_content(
-    # 1. Your schema.              
-    schema="YOUR_SCHEMA.json",
-    # 2. Your third-party model implementation.
-    llm=dynamic_init(class_filepath="replicate_104.py", class_name="Replicate")(api_token="<API-KEY>"),
-    # 3. Customize your inference and result providing modes: 
-    infer_mode="batch_async", 
-    return_mode="batch",
-    # 4. Your iterator of dictionaries
-    input_dicts_it=YOUR_DATA_IT,
-)
-    
-for content in content_it:
-    # Handle your LLM responses here ...
-```
-
-
-# Embed your LLM
-
-All you have to do is to implement `BaseLM` class, that includes:
-* `__init__` -- for setting up *batching mode support* and (optional) *model name*;
-* `ask(prompt)` -- infer your model with the given `prompt`.
-
-See examples with models [at nlp-thirdgate 🌌](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).

bulk_chain-1.1.0/bulk_chain.egg-info/PKG-INFO

@@ -1,118 +0,0 @@
-Metadata-Version: 2.1
-Name: bulk_chain
-Version: 1.1.0
-Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
-Home-page: https://github.com/nicolay-r/bulk-chain
-Author: Nicolay Rusnachenko
-Author-email: rusnicolay@gmail.com
-License: MIT License
-Keywords: natural language processing,chain-of-thought,reasoning
-Classifier: Programming Language :: Python
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Classifier: Topic :: Scientific/Engineering :: Information Analysis
-Classifier: Topic :: Text Processing :: Linguistic
-Requires-Python: >=3.6
-Description-Content-Type: text/markdown
-License-File: LICENSE
-
-# bulk-chain 1.1.0
-![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
-[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
-[![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
-[![PyPI downloads](https://img.shields.io/pypi/dm/bulk-chain.svg)](https://pypistats.org/packages/bulk-chain)
-
-<p align="center">
-    <img src="logo.png"/>
-</p>
-
-<p align="center">
-  <a href="https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm"><b>Third-party providers hosting</b>↗️</a>
-  <br>
-  <a href="https://github.com/nicolay-r/bulk-chain-shell">👉<b>demo</b>👈</a>
-</p>
-
-A no-strings-attached **framework**  for your LLM that allows applying Chain-of-Thought-alike [prompt `schema`](#chain-of-thought-schema) towards a massive textual collections using custom **[third-party providers ↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm)**.
-
-### Main Features
-* ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
-* ✅ **Support schemas descriptions** for Chain-of-Thought concept.
-* ✅ **Provides iterator over infinite amount of input contexts**
-
-# Installation
-
-From PyPI: 
-
-```bash
-pip install --no-deps bulk-chain
-```
-
-or latest version from here:
-
-```bash
-pip install git+https://github.com/nicolay-r/bulk-chain@master
-```
-
-## Chain-of-Thought Schema
-
-To declare Chain-of-Though (CoT) schema, this project exploits `JSON` format.
-This format adopts `name` field for declaring a name and `schema` is a list of CoT instructions for the Large Language Model.
-
-Each step represents a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
-All the variable names are expected to be mentioned in `{}`.
-
-Below, is an example on how to declare your own schema:
-
-```python
-{
-"name": "schema-name",
-"schema": [
-    {"prompt": "Given the question '{text}', let's think step-by-step.", 
-     "out": "steps"},
-    {"prompt": "For the question '{text}' the reasoining steps are '{steps}'. what would be an answer?", 
-     "out":  "answer"},
-]
-}
-```
-
-# Usage
-
-## 🤖 Prepare 
-
-1. [schema](#chain-of-thought-schema)
-    * [Example for Sentiment Analysis](test/schema/thor_cot_schema.json)
-2. **LLM model** from the [<b>Third-party providers hosting</b>↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
-3. Data (iter of dictionaries)
-
-## 🚀 Launch
-
-> **API**: For more details see the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
-
-```python
-from bulk_chain.core.utils import dynamic_init
-from bulk_chain.api import iter_content
-
-content_it = iter_content(
-    # 1. Your schema.              
-    schema="YOUR_SCHEMA.json",
-    # 2. Your third-party model implementation.
-    llm=dynamic_init(class_filepath="replicate_104.py", class_name="Replicate")(api_token="<API-KEY>"),
-    # 3. Customize your inference and result providing modes: 
-    infer_mode="batch_async", 
-    return_mode="batch",
-    # 4. Your iterator of dictionaries
-    input_dicts_it=YOUR_DATA_IT,
-)
-    
-for content in content_it:
-    # Handle your LLM responses here ...
-```
-
-
-# Embed your LLM
-
-All you have to do is to implement `BaseLM` class, that includes:
-* `__init__` -- for setting up *batching mode support* and (optional) *model name*;
-* `ask(prompt)` -- infer your model with the given `prompt`.
-
-See examples with models [at nlp-thirdgate 🌌](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).

bulk_chain-1.1.0/test/test_api.py

@@ -1,67 +0,0 @@
-import unittest
-from os.path import join
-
-from bulk_chain.api import iter_content
-from utils import current_dir, DEFAULT_REMOTE_LLM
-
-
-class TestAPI(unittest.TestCase):
-
-
-    @staticmethod
-    def it_data(n):
-        for i in range(n):
-            yield {"ind": i, "text": "X invent sanctions against Y", "entity": "X"}
-
-    def test_single(self):
-        data_it = iter_content(input_dicts_it=self.it_data(20),
-                               llm=DEFAULT_REMOTE_LLM,
-                               batch_size=1,
-                               infer_mode="single",
-                               return_mode="batch",
-                               schema=join(current_dir, "schema/default.json"))
-
-        for data in data_it:
-            print(data)
-
-    def test_single_stream(self):
-        """ Returns individual chunks.
-        """
-        data_it = iter_content(input_dicts_it=self.it_data(20),
-                               llm=DEFAULT_REMOTE_LLM,
-                               batch_size=1,
-                               infer_mode="single_stream",
-                               return_mode="chunk",
-                               schema=join(current_dir, "schema/default.json"))
-
-        for data in data_it:
-            print(data)
-
-    def test_batch_async(self):
-        """ Return batches that passed async at the Replicate.
-        """
-        data_it = iter_content(input_dicts_it=self.it_data(20),
-                               llm=DEFAULT_REMOTE_LLM,
-                               batch_size=5,
-                               infer_mode="batch_async",
-                               return_mode="batch",
-                               schema=join(current_dir, "schema/default.json"))
-
-        for batch in data_it:
-            for item in batch:
-                print(item)
-
-    def test_batch_stream_async(self):
-        data_it = iter_content(input_dicts_it=self.it_data(20),
-                               llm=DEFAULT_REMOTE_LLM,
-                               batch_size=5,
-                               infer_mode="batch_stream_async",
-                               return_mode="chunk",
-                               schema=join(current_dir, "schema/default.json"))
-
-        for chunk_info in data_it:
-            print(chunk_info)
-
-
-if __name__ == '__main__':
-    unittest.main()