bulk-chain 0.25.0__tar.gz → 0.25.2__tar.gz

{bulk_chain-0.25.0 → bulk_chain-0.25.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: bulk_chain
-Version: 0.25.0
+Version: 0.25.2
 Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
 Home-page: https://github.com/nicolay-r/bulk-chain
 Author: Nicolay Rusnachenko
@@ -15,8 +15,10 @@ Classifier: Topic :: Text Processing :: Linguistic
 Requires-Python: >=3.6
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: tqdm
+Requires-Dist: source-iter==0.24.3
 
-# bulk-chain 0.25.0
+# bulk-chain 0.25.2
 ![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
 [![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
 [![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
@@ -26,7 +28,13 @@ License-File: LICENSE
     <img src="logo.png"/>
 </p>
 
-A lightweight, no-strings-attached **framework**  for your LLM that allows applying [Chain-of-Thought](https://arxiv.org/abs/2201.11903) prompt `schema` (See [related section](#chain-of-thought-schema)) towards a massive textual collections.
+<p align="center">
+  <a href="https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm"><b>Third-party providers hosting</b>↗️</a>
+  <br>
+  <a href="https://github.com/nicolay-r/bulk-chain/blob/master/README.md#demo-mode">👉<b>demo</b>👈</a>
+</p>
+
+A no-strings-attached **framework**  for your LLM that allows applying Chain-of-Thought-alike [prompt `schema`](#chain-of-thought-schema) towards a massive textual collections using custom **[third-party providers ↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm)**.
 
 ### Main Features
 * ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
@@ -42,7 +50,7 @@ A lightweight, no-strings-attached **framework**  for your LLM that allows apply
 From PyPI: 
 
 ```bash
-pip install bulk-chain
+pip install --no-deps bulk-chain
 ```
 
 or latest version from here:
@@ -73,37 +81,67 @@ Below, is an example on how to declare your own schema:
 }
 ```
 
-Another templates are available [here](/ext/schema/).
-
 # Usage
 
 Preliminary steps:
 
 1. Define your [schema](#chain-of-thought-schema) ([Example for Sentiment Analysis](/ext/schema/thor_cot_schema.json)))
-2. Wrap or pick **LLM model** from the [list of presets](/ext/).
+2. Wrap or pick **LLM model** from the [<b>Third-party providers hosting</b>↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
 
-## API
+## Shell
 
-Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
+### Demo Mode
+
+**demo mode** to interact with LLM via command line with LLM output streaming support. 
+The video below illustrates an example of application for sentiment analysis on author opinion extraction towards mentioned object in text.
+
+Quck start with launching demo:
+1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
+2. 📜 Setup your reasoning `thor_cot_schema.json` according to the [following example ↗️](test/schema/thor_cot_schema.json)
+3. 🚀 Launch `demo.py` as follows:
+```bash
+python3 -m bulk_chain.demo \
+    --schema "test/schema/thor_cot_schema.json" \
+    --adapter "dynamic:replicate_104.py:Replicate" \
+    %%m \
+    --model_name "meta/meta-llama-3-70b-instruct" \
+    --api_token "<REPLICATE-API-TOKEN>" \
+    --stream
+```
+
+📺 This video showcase application of the [↗️ Sentiment Analysis Schema](https://github.com/nicolay-r/bulk-chain/blob/master/test/schema/thor_cot_schema.json) towards [LLaMA-3-70B-Instruct](https://replicate.com/meta/meta-llama-3-70b-instruct) hosted by Replicate for reasoning over submitted texts
+![sa-bulk-chain-cot-final](https://github.com/user-attachments/assets/0cc8fdcb-6ddb-44a3-8f05-d76250ae6423)
 
-## Shell
 
-> **NOTE:** You have to install `source-iter` package
+### Inference Mode
 
+> **NOTE:** You have to install `source-iter` and `tqdm` packages that actual [dependencies](dependencies.txt) of this project
+
+1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
+```bash
+wget https://raw.githubusercontent.com/nicolay-r/nlp-thirdgate/refs/heads/master/llm/replicate_104.py
+```
+2. 📜 Setup your reasoning `schema.json` according to the [following example ↗️](test/schema/default.json)
+3. 🚀 Launch inference using `DeepSeek-R1`:
 ```bash
 python3 -m bulk_chain.infer \
     --src "<PATH-TO-YOUR-CSV-or-JSONL>" \
-    --schema "ext/schema/default.json" \
-    --adapter "dynamic:ext/replicate.py:Replicate" \
+    --schema "test/schema/default.json" \
+    --adapter "replicate_104.py:Replicate" \
     %%m \
-    --api_token "<REPLICATE-API-TOKEN>" \
-    --temp 0.1
+    --model_name "deepseek-ai/deepseek-r1" \
+    --api_token "<REPLICATE-API-TOKEN>"
 ```
 
+## API
+
+Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
+
+
 # Embed your LLM
 
 All you have to do is to implement `BaseLM` class, that includes:
 * `__init__` -- for setting up *batching mode support* and (optional) *model name*;
 * `ask(prompt)` -- infer your model with the given `prompt`.
 
-See examples with models [here](/ext).
+See examples with models [at nlp-thirdgate 🌌](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).

bulk_chain-0.25.2/README.md

@@ -0,0 +1,127 @@
+# bulk-chain 0.25.2
+![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
+[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
+[![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
+[![PyPI downloads](https://img.shields.io/pypi/dm/bulk-chain.svg)](https://pypistats.org/packages/bulk-chain)
+
+<p align="center">
+    <img src="logo.png"/>
+</p>
+
+<p align="center">
+  <a href="https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm"><b>Third-party providers hosting</b>↗️</a>
+  <br>
+  <a href="https://github.com/nicolay-r/bulk-chain/blob/master/README.md#demo-mode">👉<b>demo</b>👈</a>
+</p>
+
+A no-strings-attached **framework**  for your LLM that allows applying Chain-of-Thought-alike [prompt `schema`](#chain-of-thought-schema) towards a massive textual collections using custom **[third-party providers ↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm)**.
+
+### Main Features
+* ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
+* ✅ **Support schemas descriptions** for Chain-of-Thought concept.
+* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
+
+### Extra Features
+* ✅ **Progress caching [for remote LLMs]**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
+
+
+# Installation
+
+From PyPI: 
+
+```bash
+pip install --no-deps bulk-chain
+```
+
+or latest version from here:
+
+```bash
+pip install git+https://github.com/nicolay-r/bulk-chain@master
+```
+
+## Chain-of-Thought Schema
+
+To declare Chain-of-Though (CoT) schema, this project exploits `JSON` format.
+This format adopts `name` field for declaring a name and `schema` is a list of CoT instructions for the Large Language Model.
+
+Each step represents a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
+All the variable names are expected to be mentioned in `{}`.
+
+Below, is an example on how to declare your own schema:
+
+```python
+{
+"name": "schema-name",
+"schema": [
+    {"prompt": "Given the question '{text}', let's think step-by-step.", 
+     "out": "steps"},
+    {"prompt": "For the question '{text}' the reasoining steps are '{steps}'. what would be an answer?", 
+     "out":  "answer"},
+]
+}
+```
+
+# Usage
+
+Preliminary steps:
+
+1. Define your [schema](#chain-of-thought-schema) ([Example for Sentiment Analysis](/ext/schema/thor_cot_schema.json)))
+2. Wrap or pick **LLM model** from the [<b>Third-party providers hosting</b>↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
+
+## Shell
+
+### Demo Mode
+
+**demo mode** to interact with LLM via command line with LLM output streaming support. 
+The video below illustrates an example of application for sentiment analysis on author opinion extraction towards mentioned object in text.
+
+Quck start with launching demo:
+1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
+2. 📜 Setup your reasoning `thor_cot_schema.json` according to the [following example ↗️](test/schema/thor_cot_schema.json)
+3. 🚀 Launch `demo.py` as follows:
+```bash
+python3 -m bulk_chain.demo \
+    --schema "test/schema/thor_cot_schema.json" \
+    --adapter "dynamic:replicate_104.py:Replicate" \
+    %%m \
+    --model_name "meta/meta-llama-3-70b-instruct" \
+    --api_token "<REPLICATE-API-TOKEN>" \
+    --stream
+```
+
+📺 This video showcase application of the [↗️ Sentiment Analysis Schema](https://github.com/nicolay-r/bulk-chain/blob/master/test/schema/thor_cot_schema.json) towards [LLaMA-3-70B-Instruct](https://replicate.com/meta/meta-llama-3-70b-instruct) hosted by Replicate for reasoning over submitted texts
+![sa-bulk-chain-cot-final](https://github.com/user-attachments/assets/0cc8fdcb-6ddb-44a3-8f05-d76250ae6423)
+
+
+### Inference Mode
+
+> **NOTE:** You have to install `source-iter` and `tqdm` packages that actual [dependencies](dependencies.txt) of this project
+
+1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
+```bash
+wget https://raw.githubusercontent.com/nicolay-r/nlp-thirdgate/refs/heads/master/llm/replicate_104.py
+```
+2. 📜 Setup your reasoning `schema.json` according to the [following example ↗️](test/schema/default.json)
+3. 🚀 Launch inference using `DeepSeek-R1`:
+```bash
+python3 -m bulk_chain.infer \
+    --src "<PATH-TO-YOUR-CSV-or-JSONL>" \
+    --schema "test/schema/default.json" \
+    --adapter "replicate_104.py:Replicate" \
+    %%m \
+    --model_name "deepseek-ai/deepseek-r1" \
+    --api_token "<REPLICATE-API-TOKEN>"
+```
+
+## API
+
+Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
+
+
+# Embed your LLM
+
+All you have to do is to implement `BaseLM` class, that includes:
+* `__init__` -- for setting up *batching mode support* and (optional) *model name*;
+* `ask(prompt)` -- infer your model with the given `prompt`.
+
+See examples with models [at nlp-thirdgate 🌌](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).

{bulk_chain-0.25.0 → bulk_chain-0.25.2}/bulk_chain/api.py

@@ -7,7 +7,7 @@ from bulk_chain.core.service_data import DataService
 from bulk_chain.core.service_dict import DictionaryService
 from bulk_chain.core.service_json import JsonService
 from bulk_chain.core.service_schema import SchemaService
-
+from bulk_chain.core.utils import dynamic_init, find_by_prefix
 
 INFER_MODES = {
     "default": lambda llm, prompt, limit_prompt=None: llm.ask_core(
@@ -76,4 +76,24 @@ def iter_content(input_dicts_it, llm, schema, batch_size=1, return_batch=True, l
                                schema=schema)
                   for batch in BatchIterator(prompts_it, batch_size=batch_size))
 
-    yield from content_it if return_batch else chain.from_iterable(content_it)
+    yield from content_it if return_batch else chain.from_iterable(content_it)
+
+
+def init_llm(adapter, **model_kwargs):
+    """ This method perform dynamic initialization of LLM from third-party resource.
+    """
+
+    # List of the Supported models and their API wrappers.
+    models_preset = {
+        "dynamic": lambda: dynamic_init(class_dir=CWD, class_filepath=llm_model_name,
+                                        class_name=llm_model_params)(**model_kwargs)
+    }
+
+    # Initialize LLM model.
+    params = adapter.split(':')
+    llm_model_type = params[0]
+    llm_model_name = params[1] if len(params) > 1 else params[-1]
+    llm_model_params = ':'.join(params[2:]) if len(params) > 2 else None
+    llm = find_by_prefix(d=models_preset, key=llm_model_type)()
+
+    return llm, llm_model_name

{bulk_chain-0.25.0 → bulk_chain-0.25.2}/bulk_chain/core/llm_base.py

@@ -24,7 +24,7 @@ class BaseLM(object):
             try:
                 if self.__support_batching:
                     # Launch in batch mode.
-                    content = self.ask(batch)
+                    content = batch
                 else:
                     # Launch in non-batch mode.
                     assert len(batch) == 1, "The LM does not support batching," \

{bulk_chain-0.25.0 → bulk_chain-0.25.2}/bulk_chain/core/service_args.py

@@ -12,6 +12,11 @@ class CmdArgsService:
     def iter_arguments(lst):
 
         def __release():
+
+            # We use the True value by default to treat the related parameter as flag.
+            if len(buf) == 0:
+                buf.append(True)
+
             return key, buf if len(buf) > 1 else buf[0]
 
         key = None
@@ -29,7 +34,7 @@ class CmdArgsService:
                 buf.append(a)
 
         # Sharing the remaining params.
-        if len(buf) > 0:
+        if key is not None:
             yield __release()
 
     @staticmethod

bulk_chain-0.25.2/bulk_chain/core/service_llm.py

@@ -0,0 +1,94 @@
+from bulk_chain.core.llm_base import BaseLM
+from bulk_chain.core.service_data import DataService
+from bulk_chain.core.utils import iter_params
+from bulk_chain.core.utils_logger import StreamedLogger
+
+
+def pad_str(text, pad):
+    return text.rjust(len(text) + pad, ' ')
+
+
+def nice_output(text, remove_new_line=False):
+    short_text = text.replace("\n", "") if remove_new_line else text
+    return short_text
+
+
+def chat_with_lm(lm, preset_dict=None, chain=None, model_name=None, pad=0):
+    assert (isinstance(lm, BaseLM))
+    assert (isinstance(chain, list))
+    assert (isinstance(model_name, str) or model_name is None)
+
+    preset_dict = {} if preset_dict is None else preset_dict
+
+    streamed_logger = StreamedLogger(__name__)
+
+    do_exit = False
+    model_name = model_name if model_name is not None else "agent"
+
+    while not do_exit:
+
+        streamed_logger.info("----------------")
+        streamed_logger.info("\n")
+
+        # Launching the CoT engine loop.
+        data_dict = {} | preset_dict
+        for chain_ind, prompt_args in enumerate(chain):
+
+            # Processing the prompt.
+            prompt = prompt_args["prompt"]
+
+            # Filling necessary parameters.
+            user_informed = False
+            field_names = list(iter_params(prompt))
+            for ind, f_name in enumerate(field_names):
+
+                if f_name in data_dict:
+                    continue
+
+                user_input = input(f"Enter your prompt for `{f_name}` ({ind+1}/{len(field_names)}) "
+                                   f"(or 'exit' to quit): ")
+                user_informed = True
+
+                if user_input.lower() == 'exit':
+                    do_exit = True
+                    break
+
+                data_dict[f_name] = user_input
+
+            if do_exit:
+                break
+
+            # In the case of the initial interaction with the chain.
+            # we make sure that aware user for starting interaction.
+            if chain_ind == 0 and not user_informed:
+                user_input = input(f"Enter to continue (or 'exit' to quit) ...")
+                if user_input.lower() == 'exit':
+                    do_exit = True
+
+            # Finally asking LLM.
+            DataService.compose_prompt_text(prompt=prompt, data_dict=data_dict, field_names=field_names)
+            actual_prompt = DataService.get_prompt_text(prompt=prompt, data_dict=data_dict)
+
+            # Returning meta information, passed to LLM.
+            streamed_logger.info(pad_str(f"{model_name} (ask [{chain_ind+1}/{len(chain)}]) ->", pad=pad))
+            streamed_logger.info("\n")
+            streamed_logger.info(nice_output(actual_prompt, remove_new_line=True))
+            streamed_logger.info("\n\n")
+
+            # Response.
+            response = lm.ask_core(batch=[actual_prompt])[0]
+            streamed_logger.info(pad_str(f"{model_name} (resp [{chain_ind+1}/{len(chain)}])->", pad=pad))
+            streamed_logger.info("\n")
+            if isinstance(response, str):
+                streamed_logger.info(nice_output(response, remove_new_line=False))
+                buffer = [response]
+            else:
+                buffer = []
+                for chunk in response:
+                    streamed_logger.info(chunk)
+                    buffer.append(str(chunk))
+
+            streamed_logger.info("\n\n")
+
+            # Collecting the answer for the next turn.
+            data_dict[prompt_args["out"]] = "".join(buffer)

bulk_chain-0.25.2/bulk_chain/core/utils_logger.py

@@ -0,0 +1,41 @@
+import logging
+
+
+def StreamedLogger(name: str) -> logging.Logger:
+    """ https://medium.com/@r.das699/optimizing-logging-practices-for-streaming-data-in-python-521798e1ed82
+    """
+    root_handlers = logging.getLogger().handlers
+    current_logger = logging.getLogger(name)
+    if not root_handlers:
+        new_handler = logging.StreamHandler()
+        new_handler.terminator = ""
+        new_handler.setFormatter(logging.Formatter("%(message)s"))
+        current_logger.addHandler(new_handler)
+        current_logger.propagate = False
+        current_logger.setLevel(logging.INFO)
+        return current_logger
+
+    for handler in current_logger.handlers[:]:
+        current_logger.removeHandler(handler)
+
+    for handler_r in root_handlers:
+        if type(handler_r) is logging.StreamHandler:
+            new_handler = logging.StreamHandler()
+            new_handler.terminator = ""
+            new_handler.setFormatter(logging.Formatter("%(message)s"))
+            current_logger.addHandler(new_handler)
+        elif type(handler_r) is logging.FileHandler:
+            new_handler = logging.FileHandler(
+                handler_r.baseFilename,
+                handler_r.mode,
+                handler_r.encoding,
+                handler_r.delay,
+                handler_r.errors,
+            )
+            new_handler.terminator = ""  # This will stop the printing in new line
+            new_handler.setFormatter(logging.Formatter("%(message)s"))
+            current_logger.addHandler(new_handler)
+        else:
+            continue
+    current_logger.propagate = False  # Don't propagate to root logger
+    return current_logger

bulk_chain-0.25.2/bulk_chain/demo.py

@@ -0,0 +1,84 @@
+import json
+
+import argparse
+import logging
+import sys
+
+from source_iter.service_jsonl import JsonlService
+
+from bulk_chain.api import init_llm
+from bulk_chain.core.service_args import CmdArgsService
+from bulk_chain.core.service_json import JsonService
+from bulk_chain.core.service_llm import chat_with_lm
+from bulk_chain.core.service_schema import SchemaService
+from bulk_chain.core.utils import parse_filepath
+
+logger = logging.getLogger(__name__)
+logging.basicConfig(level=logging.INFO)
+
+
+def iter_from_json(filepath):
+    with open(filepath, "r") as f:
+        content = json.load(f)
+        for key, value in content.items():
+            yield key, value
+
+
+def iter_from_text_file(filepath):
+    with open(filepath, "r") as f:
+        yield filepath.split('.')[0], f.read()
+
+
+if __name__ == '__main__':
+
+    parser = argparse.ArgumentParser(description="LLM demo usage based on CoT schema")
+    parser.add_argument('--adapter', dest='adapter', type=str, default=None)
+    parser.add_argument('--attempts', dest='attempts', type=int, default=None)
+    parser.add_argument('--src', dest='src', type=str, nargs="*", default=None)
+    parser.add_argument('--schema', dest='schema', type=str, default=None,
+                        help="Path to the JSON file that describes schema")
+    parser.add_argument('--limit-prompt', dest="limit_prompt", type=int, default=None,
+                        help="Optional trimming prompt by the specified amount of characters.")
+
+    # Extract native arguments.
+    native_args = CmdArgsService.extract_native_args(sys.argv, end_prefix="%%")
+    args = parser.parse_args(args=native_args[1:])
+
+    # Extract model-related arguments and Initialize Large Language Model.
+    model_args = CmdArgsService.find_grouped_args(lst=sys.argv, starts_with="%%m", end_prefix="%%")
+    model_args_dict = CmdArgsService.args_to_dict(model_args) | {"attempts": args.attempts}
+    llm, llm_model_name = init_llm(adapter=args.adapter, **model_args_dict)
+
+    # Setup schema.
+    schema = SchemaService(json_data=JsonService.read(args.schema))
+    schema_name = schema.src.get("name", None)
+    if schema is not None:
+        logger.info(f"Using schema: {schema_name}")
+
+    output_providers = {
+        "jsonl": lambda filepath, data_it, header:
+            JsonlService.write(target=filepath,
+                               data_it=map(lambda item: {key: item[i] for i, key in enumerate(header)}, data_it))
+    }
+
+    input_file_handlers = {
+        "json": lambda filepath: iter_from_json(filepath),
+        "txt": lambda filepath: iter_from_text_file(filepath)
+    }
+
+    # Input extension type defines the provider.
+    if args.src is None:
+        args.src = []
+    if isinstance(args.src, str):
+        args.src = [args.src]
+    sources = [parse_filepath(s) for s in args.src]
+
+    preset_dict = {}
+    for fp, ext, _ in sources:
+        for key, value in input_file_handlers[ext](fp):
+            if key in preset_dict:
+                raise Exception(f"While at handling {fp}: Key {key} is already registered!")
+            preset_dict[key] = value
+
+    # Launch Demo.
+    chat_with_lm(llm, preset_dict=preset_dict, chain=schema.chain, model_name=llm_model_name)

{bulk_chain-0.25.0 → bulk_chain-0.25.2}/bulk_chain/infer.py

@@ -9,51 +9,28 @@ from source_iter.service_jsonl import JsonlService
 from source_iter.service_sqlite import SQLite3Service
 from tqdm import tqdm
 
-from bulk_chain.api import INFER_MODES, _infer_batch, CWD
+from bulk_chain.api import INFER_MODES, _infer_batch, CWD, init_llm
 from bulk_chain.core.llm_base import BaseLM
 from bulk_chain.core.service_args import CmdArgsService
 from bulk_chain.core.service_dict import DictionaryService
 from bulk_chain.core.service_json import JsonService
-from bulk_chain.core.service_llm import chat_with_lm
 from bulk_chain.core.service_schema import SchemaService
-from bulk_chain.core.utils import dynamic_init, find_by_prefix, handle_table_name, optional_limit_iter, parse_filepath
+from bulk_chain.core.utils import handle_table_name, optional_limit_iter, parse_filepath
 
 logger = logging.getLogger(__name__)
 logging.basicConfig(level=logging.INFO)
 
-
 WRITER_PROVIDERS = {
     "sqlite": lambda filepath, table_name, data_it, infer_data_func, **kwargs: SQLite3Service.write(
         data_it=data_it, target=filepath, table_name=table_name, data2col_func=infer_data_func,
         skip_existed=True, **kwargs)
 }
 
-
 READER_PROVIDERS = {
     "sqlite": lambda filepath, table_name: SQLite3Service.read(filepath, table=table_name)
 }
 
 
-def init_llm(**model_kwargs):
-    """ This method perform dynamic initialization of LLM from third-party resource.
-    """
-
-    # List of the Supported models and their API wrappers.
-    models_preset = {
-        "dynamic": lambda: dynamic_init(class_dir=CWD, class_filepath=llm_model_name,
-                                        class_name=llm_model_params)(**model_kwargs)
-    }
-
-    # Initialize LLM model.
-    params = args.adapter.split(':')
-    llm_model_type = params[0]
-    llm_model_name = params[1] if len(params) > 1 else params[-1]
-    llm_model_params = ':'.join(params[2:]) if len(params) > 2 else None
-    llm = find_by_prefix(d=models_preset, key=llm_model_type)()
-
-    return llm, llm_model_name
-
-
 def iter_content_cached(input_dicts_it, llm, schema, cache_target, limit_prompt=None, **cache_kwargs):
     assert (isinstance(llm, BaseLM))
     assert (isinstance(cache_target, str))
@@ -91,9 +68,8 @@ if __name__ == '__main__':
 
     parser = argparse.ArgumentParser(description="Infer Instruct LLM inference based on CoT schema")
     parser.add_argument('--adapter', dest='adapter', type=str, default=None)
-    parser.add_argument('--attempts', dest='attempts', type=int, default=None)
     parser.add_argument('--id-col', dest='id_col', type=str, default="uid")
-    parser.add_argument('--src', dest='src', type=str, default=None)
+    parser.add_argument('--src', dest='src', type=str, nargs="?", default=None)
     parser.add_argument('--schema', dest='schema', type=str, default=None,
                         help="Path to the JSON file that describes schema")
     parser.add_argument('--to', dest='to', type=str, default=None, choices=["csv", "sqlite"])
@@ -114,7 +90,7 @@ if __name__ == '__main__':
     # Extract model-related arguments and Initialize Large Language Model.
     model_args = CmdArgsService.find_grouped_args(lst=sys.argv, starts_with="%%m", end_prefix="%%")
     model_args_dict = CmdArgsService.args_to_dict(model_args) | {"attempts": args.attempts}
-    llm, llm_model_name = init_llm(**model_args_dict)
+    llm, llm_model_name = init_llm(adapter=args.adapter, **model_args_dict)
 
     # Setup schema.
     schema = SchemaService(json_data=JsonService.read(args.schema))
@@ -123,7 +99,6 @@ if __name__ == '__main__':
         logger.info(f"Using schema: {schema_name}")
 
     input_providers = {
-        None: lambda _: chat_with_lm(llm, chain=schema.chain, model_name=llm_model_name),
         "csv": lambda filepath: CsvService.read(src=filepath, row_id_key=args.id_col,
                                                 as_dict=True, skip_header=True,
                                                 delimiter=csv_args_dict.get("delimiter", ","),
@@ -155,14 +130,9 @@ if __name__ == '__main__':
     args.output = args.output.format(model=llm.name()) if args.output is not None else args.output
     tgt_filepath, tgt_ext, tgt_meta = parse_filepath(args.output, default_ext=args.to)
 
-    # Input extension type defines the provider.
+    # We do not support multiple files for other modes.
     src_filepath, src_ext, src_meta = parse_filepath(args.src)
 
-    # Check whether we are in chat mode.
-    if src_ext is None:
-        input_providers[src_ext](None)
-        exit(0)
-
     def default_output_file_template(ext):
         # This is a default template for output files to be generated.
         return "".join(["_".join([join(CWD, basename(src_filepath)), llm.name(), schema_name]), ext])

{bulk_chain-0.25.0 → bulk_chain-0.25.2}/bulk_chain.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: bulk_chain
-Version: 0.25.0
+Version: 0.25.2
 Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
 Home-page: https://github.com/nicolay-r/bulk-chain
 Author: Nicolay Rusnachenko
@@ -15,8 +15,10 @@ Classifier: Topic :: Text Processing :: Linguistic
 Requires-Python: >=3.6
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: tqdm
+Requires-Dist: source-iter==0.24.3
 
-# bulk-chain 0.25.0
+# bulk-chain 0.25.2
 ![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
 [![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
 [![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
@@ -26,7 +28,13 @@ License-File: LICENSE
     <img src="logo.png"/>
 </p>
 
-A lightweight, no-strings-attached **framework**  for your LLM that allows applying [Chain-of-Thought](https://arxiv.org/abs/2201.11903) prompt `schema` (See [related section](#chain-of-thought-schema)) towards a massive textual collections.
+<p align="center">
+  <a href="https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm"><b>Third-party providers hosting</b>↗️</a>
+  <br>
+  <a href="https://github.com/nicolay-r/bulk-chain/blob/master/README.md#demo-mode">👉<b>demo</b>👈</a>
+</p>
+
+A no-strings-attached **framework**  for your LLM that allows applying Chain-of-Thought-alike [prompt `schema`](#chain-of-thought-schema) towards a massive textual collections using custom **[third-party providers ↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm)**.
 
 ### Main Features
 * ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
@@ -42,7 +50,7 @@ A lightweight, no-strings-attached **framework**  for your LLM that allows apply
 From PyPI: 
 
 ```bash
-pip install bulk-chain
+pip install --no-deps bulk-chain
 ```
 
 or latest version from here:
@@ -73,37 +81,67 @@ Below, is an example on how to declare your own schema:
 }
 ```
 
-Another templates are available [here](/ext/schema/).
-
 # Usage
 
 Preliminary steps:
 
 1. Define your [schema](#chain-of-thought-schema) ([Example for Sentiment Analysis](/ext/schema/thor_cot_schema.json)))
-2. Wrap or pick **LLM model** from the [list of presets](/ext/).
+2. Wrap or pick **LLM model** from the [<b>Third-party providers hosting</b>↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
 
-## API
+## Shell
 
-Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
+### Demo Mode
+
+**demo mode** to interact with LLM via command line with LLM output streaming support. 
+The video below illustrates an example of application for sentiment analysis on author opinion extraction towards mentioned object in text.
+
+Quck start with launching demo:
+1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
+2. 📜 Setup your reasoning `thor_cot_schema.json` according to the [following example ↗️](test/schema/thor_cot_schema.json)
+3. 🚀 Launch `demo.py` as follows:
+```bash
+python3 -m bulk_chain.demo \
+    --schema "test/schema/thor_cot_schema.json" \
+    --adapter "dynamic:replicate_104.py:Replicate" \
+    %%m \
+    --model_name "meta/meta-llama-3-70b-instruct" \
+    --api_token "<REPLICATE-API-TOKEN>" \
+    --stream
+```
+
+📺 This video showcase application of the [↗️ Sentiment Analysis Schema](https://github.com/nicolay-r/bulk-chain/blob/master/test/schema/thor_cot_schema.json) towards [LLaMA-3-70B-Instruct](https://replicate.com/meta/meta-llama-3-70b-instruct) hosted by Replicate for reasoning over submitted texts
+![sa-bulk-chain-cot-final](https://github.com/user-attachments/assets/0cc8fdcb-6ddb-44a3-8f05-d76250ae6423)
 
-## Shell
 
-> **NOTE:** You have to install `source-iter` package
+### Inference Mode
 
+> **NOTE:** You have to install `source-iter` and `tqdm` packages that actual [dependencies](dependencies.txt) of this project
+
+1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
+```bash
+wget https://raw.githubusercontent.com/nicolay-r/nlp-thirdgate/refs/heads/master/llm/replicate_104.py
+```
+2. 📜 Setup your reasoning `schema.json` according to the [following example ↗️](test/schema/default.json)
+3. 🚀 Launch inference using `DeepSeek-R1`:
 ```bash
 python3 -m bulk_chain.infer \
     --src "<PATH-TO-YOUR-CSV-or-JSONL>" \
-    --schema "ext/schema/default.json" \
-    --adapter "dynamic:ext/replicate.py:Replicate" \
+    --schema "test/schema/default.json" \
+    --adapter "replicate_104.py:Replicate" \
     %%m \
-    --api_token "<REPLICATE-API-TOKEN>" \
-    --temp 0.1
+    --model_name "deepseek-ai/deepseek-r1" \
+    --api_token "<REPLICATE-API-TOKEN>"
 ```
 
+## API
+
+Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
+
+
 # Embed your LLM
 
 All you have to do is to implement `BaseLM` class, that includes:
 * `__init__` -- for setting up *batching mode support* and (optional) *model name*;
 * `ask(prompt)` -- infer your model with the given `prompt`.
 
-See examples with models [here](/ext).
+See examples with models [at nlp-thirdgate 🌌](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).

{bulk_chain-0.25.0 → bulk_chain-0.25.2}/bulk_chain.egg-info/SOURCES.txt

@@ -3,10 +3,12 @@ README.md
 setup.py
 bulk_chain/__init__.py
 bulk_chain/api.py
+bulk_chain/demo.py
 bulk_chain/infer.py
 bulk_chain.egg-info/PKG-INFO
 bulk_chain.egg-info/SOURCES.txt
 bulk_chain.egg-info/dependency_links.txt
+bulk_chain.egg-info/requires.txt
 bulk_chain.egg-info/top_level.txt
 bulk_chain/core/__init__.py
 bulk_chain/core/llm_base.py
@@ -18,6 +20,9 @@ bulk_chain/core/service_json.py
 bulk_chain/core/service_llm.py
 bulk_chain/core/service_schema.py
 bulk_chain/core/utils.py
+bulk_chain/core/utils_logger.py
+test/test.py
 test/test_api.py
 test/test_args_seeking.py
-test/test_cmdargs.py
+test/test_cmdargs.py
+test/test_provider_batching.py

bulk_chain-0.25.2/bulk_chain.egg-info/requires.txt

@@ -0,0 +1,2 @@
+tqdm
+source-iter==0.24.3

{bulk_chain-0.25.0 → bulk_chain-0.25.2}/setup.py

@@ -15,7 +15,7 @@ def get_requirements(filenames):
 
 setup(
     name='bulk_chain',
-    version='0.25.0',
+    version='0.25.2',
     python_requires=">=3.6",
     description='A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, '
                 'ensuring reliable results for bulk input requests.',

bulk_chain-0.25.2/test/test.py

@@ -0,0 +1,62 @@
+import random
+import time
+import logging
+
+
+def setup_logger_behaviour(name: str) -> logging.Logger:
+    root_handlers = logging.getLogger().handlers # gets root logger
+    current_logger = logging.getLogger(name)   # gets current logger
+    if not root_handlers:  # if root logger has no handlers then create streaming handeler only
+        new_handler = logging.StreamHandler()
+        new_handler.terminator = ""
+        new_handler.setFormatter(logging.Formatter("%(message)s"))
+        current_logger.addHandler(new_handler)
+        current_logger.propagate = False
+        current_logger.setLevel(logging.INFO)
+        return current_logger
+
+    # Remove  exixting Handlers from the current logger
+    for handler in current_logger.handlers[:]:
+        current_logger.removeHandler(handler)
+
+    for handler_r in root_handlers:  # if root logger has handlers
+        if type(handler_r) is logging.StreamHandler:  # if root logger has streaming handler
+            new_handler = logging.StreamHandler() 
+            new_handler.terminator = ""   # This will stop the printing in new line
+            new_handler.setFormatter(logging.Formatter("%(message)s")) # This will set the format
+            current_logger.addHandler(new_handler)
+        elif type(handler_r) is logging.FileHandler:  # if root logger has file handler
+            new_handler = logging.FileHandler( # create new file handler
+                handler_r.baseFilename,        # with same filename and other properties
+                handler_r.mode,
+                handler_r.encoding,
+                handler_r.delay,
+                handler_r.errors,
+            )
+            new_handler.terminator = ""  # This will stop the printing in new line
+            new_handler.setFormatter(logging.Formatter("%(message)s")) # This will set the format
+            current_logger.addHandler(new_handler)
+        else:
+            continue
+    current_logger.propagate = False  # Don't propagate to root logger
+    return current_logger
+
+# Configure the logger
+logger =logging.getLogger(__name__)
+class FakeStreamingDataGenerator:
+
+    def stream_data(self):
+        while True:
+            data = random.randint(0, 100)
+            yield data
+            time.sleep(0.5)
+
+# Example usage:
+generator = FakeStreamingDataGenerator()
+stream = generator.stream_data()
+
+logger = setup_logger_behaviour(__name__) # call you set up function here
+while True:
+    chunk = next(stream)
+    # Replacing print with logger
+    logger.info(chunk)  # Best practice now

{bulk_chain-0.25.0 → bulk_chain-0.25.2}/test/test_api.py

@@ -9,8 +9,9 @@ from bulk_chain.infer import iter_content_cached
 class TestAPI(unittest.TestCase):
 
     llm = dynamic_init(class_dir=join(CWD, ".."),
-                       class_filepath="ext/replicate.py",
-                       class_name="Replicate")(api_token="<API-KEY>")
+                       class_filepath="providers/replicate_104.py",
+                       class_name="Replicate")(api_token="<API-KEY>",
+                                               model_name="deepseek-ai/deepseek-r1")
 
     def it_data(self, n):
         for i in range(n):
@@ -19,7 +20,7 @@ class TestAPI(unittest.TestCase):
     def test_iter_cached(self):
         data_it = iter_content_cached(input_dicts_it=self.it_data(20),
                                       llm=self.llm,
-                                      schema="../ext/schema/default.json",
+                                      schema="../schema/default.json",
                                       # Cache-related extra parameters.
                                       cache_target="out.sqlite:content",
                                       id_column_name="ind")
@@ -32,7 +33,7 @@ class TestAPI(unittest.TestCase):
                                llm=self.llm,
                                batch_size=1,
                                return_batch=True,
-                               schema="../ext/schema/default.json")
+                               schema="../schema/default.json")
 
         for data in data_it:
             print(data)

bulk_chain-0.25.2/test/test_cmdargs.py

@@ -0,0 +1,29 @@
+import sys
+import unittest
+
+from bulk_chain.core.service_args import CmdArgsService
+
+
+class TestCmdArgs(unittest.TestCase):
+
+    def test(self):
+
+        # Csv-related.
+        csv_args = CmdArgsService.find_grouped_args(sys.argv, starts_with="%%csv", end_prefix="%%")
+        print(csv_args)
+        csv_args = CmdArgsService.args_to_dict(csv_args)
+        print("csv\t", csv_args)
+
+        # Model-related.
+        m_args = CmdArgsService.find_grouped_args(sys.argv, starts_with="%%m", end_prefix="%%")
+        m_args = CmdArgsService.args_to_dict(m_args)
+        print("mod\t", m_args)
+
+        # native.
+        n_args = CmdArgsService.extract_native_args(sys.argv, end_prefix="%%")
+        n_args = CmdArgsService.args_to_dict(n_args)
+        print("nat\t", n_args)
+
+
+if __name__ == '__main__':
+    unittest.main()

bulk_chain-0.25.2/test/test_provider_batching.py

@@ -0,0 +1,31 @@
+import unittest
+from os.path import join
+
+from tqdm import tqdm
+
+from bulk_chain.api import CWD, iter_content
+from bulk_chain.core.utils import dynamic_init
+from utils import iter_test_jsonl_samples
+
+
+class TestProviderBatching(unittest.TestCase):
+
+    llm = dynamic_init(class_dir=join(CWD, ".."),
+                       class_filepath="providers/transformers_flan_t5.py",
+                       class_name="FlanT5")(model_name="nicolay-r/flan-t5-tsa-thor-base",
+                                            max_new_tokens=128)
+
+    def test_iter(self):
+        input_dicts_it = iter_test_jsonl_samples()
+        data_it = iter_content(input_dicts_it=input_dicts_it,
+                               llm=self.llm,
+                               batch_size=20,
+                               return_batch=False,
+                               schema="schema/default.json")
+
+        for item in tqdm(data_it):
+            print(item)
+
+
+if __name__ == '__main__':
+    unittest.main()

bulk_chain-0.25.0/README.md

@@ -1,91 +0,0 @@
-# bulk-chain 0.25.0
-![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
-[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
-[![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
-[![PyPI downloads](https://img.shields.io/pypi/dm/bulk-chain.svg)](https://pypistats.org/packages/bulk-chain)
-
-<p align="center">
-    <img src="logo.png"/>
-</p>
-
-A lightweight, no-strings-attached **framework**  for your LLM that allows applying [Chain-of-Thought](https://arxiv.org/abs/2201.11903) prompt `schema` (See [related section](#chain-of-thought-schema)) towards a massive textual collections.
-
-### Main Features
-* ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
-* ✅ **Support schemas descriptions** for Chain-of-Thought concept.
-* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
-
-### Extra Features
-* ✅ **Progress caching [for remote LLMs]**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
-
-
-# Installation
-
-From PyPI: 
-
-```bash
-pip install bulk-chain
-```
-
-or latest version from here:
-
-```bash
-pip install git+https://github.com/nicolay-r/bulk-chain@master
-```
-
-## Chain-of-Thought Schema
-
-To declare Chain-of-Though (CoT) schema, this project exploits `JSON` format.
-This format adopts `name` field for declaring a name and `schema` is a list of CoT instructions for the Large Language Model.
-
-Each step represents a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
-All the variable names are expected to be mentioned in `{}`.
-
-Below, is an example on how to declare your own schema:
-
-```python
-{
-"name": "schema-name",
-"schema": [
-    {"prompt": "Given the question '{text}', let's think step-by-step.", 
-     "out": "steps"},
-    {"prompt": "For the question '{text}' the reasoining steps are '{steps}'. what would be an answer?", 
-     "out":  "answer"},
-]
-}
-```
-
-Another templates are available [here](/ext/schema/).
-
-# Usage
-
-Preliminary steps:
-
-1. Define your [schema](#chain-of-thought-schema) ([Example for Sentiment Analysis](/ext/schema/thor_cot_schema.json)))
-2. Wrap or pick **LLM model** from the [list of presets](/ext/).
-
-## API
-
-Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
-
-## Shell
-
-> **NOTE:** You have to install `source-iter` package
-
-```bash
-python3 -m bulk_chain.infer \
-    --src "<PATH-TO-YOUR-CSV-or-JSONL>" \
-    --schema "ext/schema/default.json" \
-    --adapter "dynamic:ext/replicate.py:Replicate" \
-    %%m \
-    --api_token "<REPLICATE-API-TOKEN>" \
-    --temp 0.1
-```
-
-# Embed your LLM
-
-All you have to do is to implement `BaseLM` class, that includes:
-* `__init__` -- for setting up *batching mode support* and (optional) *model name*;
-* `ask(prompt)` -- infer your model with the given `prompt`.
-
-See examples with models [here](/ext).

bulk_chain-0.25.0/bulk_chain/core/service_llm.py

@@ -1,82 +0,0 @@
-import logging
-
-from bulk_chain.core.llm_base import BaseLM
-from bulk_chain.core.service_data import DataService
-from bulk_chain.core.utils import iter_params
-
-
-def pad_str(text, pad):
-    return text.rjust(len(text) + pad, ' ')
-
-
-def text_wrap(content, width, handle_line=lambda l: l):
-    lines = []
-    for text in content.split('\n'):
-        for i in range(0, len(text), width):
-            line = handle_line(text[i:i + width])
-            lines.append(line)
-    return '\n'.join(lines)
-
-
-def nice_output(text, width, pad=4, remove_new_line=False):
-    short_text = text.replace("\n", "") if remove_new_line else text
-    return text_wrap(content=short_text, width=width, handle_line=lambda line: pad_str(line, pad=pad))
-
-
-def chat_with_lm(lm, chain=None, model_name=None):
-    assert (isinstance(lm, BaseLM))
-    assert (isinstance(chain, list))
-    assert (isinstance(model_name, str) or model_name is None)
-
-    logger = logging.getLogger(__name__)
-    logging.basicConfig(level=logging.INFO)
-
-    do_exit = False
-    model_name = model_name if model_name is not None else "agent"
-
-    while not do_exit:
-
-        logger.info("----------------")
-
-        # Launching the CoT engine loop.
-        data_dict = {}
-        for prompt_args in chain:
-
-            # Processing the prompt.
-            prompt = prompt_args["prompt"]
-
-            # Filling necessary parameters.
-            field_names = list(iter_params(prompt))
-            for ind, f_name in enumerate(field_names):
-
-                if f_name in data_dict:
-                    continue
-
-                user_input = input(f"Enter your prompt for `{f_name}` ({ind+1}/{len(field_names)}) "
-                                   f"(or 'exit' to quit): ")
-
-                if user_input.lower() == 'exit':
-                    do_exit = True
-                    break
-
-                data_dict[f_name] = user_input
-
-            if do_exit:
-                break
-
-            # Finally asking LLM.
-            DataService.compose_prompt_text(prompt=prompt, data_dict=data_dict, field_names=field_names)
-            actual_prompt = DataService.get_prompt_text(prompt=prompt, data_dict=data_dict)
-
-            # Returning meta information, passed to LLM.
-            pad = 4
-            logger.info(pad_str(f"{model_name} (ask) ->", pad=pad))
-            logger.info(nice_output(actual_prompt, pad=pad*2, remove_new_line=True, width=80))
-
-            # Response.
-            response_batch = lm.ask_core(batch=[actual_prompt])
-            logger.info(pad_str(f"{model_name} (resp)->", pad=pad))
-            logger.info(nice_output(response_batch[0], pad=pad * 2, remove_new_line=False, width=80))
-
-            # Collecting the answer for the next turn.
-            data_dict[prompt_args["out"]] = response_batch[0]

bulk_chain-0.25.0/test/test_cmdargs.py

@@ -1,20 +0,0 @@
-import sys
-
-from bulk_chain.core.service_args import CmdArgsService
-
-
-# Csv-related.
-csv_args = CmdArgsService.find_grouped_args(sys.argv, starts_with="%%csv", end_prefix="%%")
-print(csv_args)
-csv_args = CmdArgsService.args_to_dict(csv_args)
-print("csv\t", csv_args)
-
-# Model-related.
-m_args = CmdArgsService.find_grouped_args(sys.argv, starts_with="%%m", end_prefix="%%")
-m_args = CmdArgsService.args_to_dict(m_args)
-print("mod\t", m_args)
-
-# native.
-n_args = CmdArgsService.extract_native_args(sys.argv, end_prefix="%%")
-n_args = CmdArgsService.args_to_dict(n_args)
-print("nat\t", n_args)