bulk-chain 0.24.1__tar.gz → 0.25.0__tar.gz

{bulk_chain-0.24.1 → bulk_chain-0.25.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: bulk_chain
-Version: 0.24.1
+Version: 0.25.0
 Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
 Home-page: https://github.com/nicolay-r/bulk-chain
 Author: Nicolay Rusnachenko
@@ -15,32 +15,42 @@ Classifier: Topic :: Text Processing :: Linguistic
 Requires-Python: >=3.6
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: tqdm
 
-# bulk-chain 0.24.1
+# bulk-chain 0.25.0
 ![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
 [![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
 [![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
+[![PyPI downloads](https://img.shields.io/pypi/dm/bulk-chain.svg)](https://pypistats.org/packages/bulk-chain)
 
 <p align="center">
     <img src="logo.png"/>
 </p>
 
-A lightweight, no-strings-attached **[Chain-of-Thought](https://arxiv.org/abs/2201.11903) framework** for your LLM, ensuring reliable results for bulk input requests stored in `CSV` / `JSONL` / `sqlite`.
-It allows applying series of prompts formed into `schema` (See [related section](#chain-of-thought-schema))
+A lightweight, no-strings-attached **framework**  for your LLM that allows applying [Chain-of-Thought](https://arxiv.org/abs/2201.11903) prompt `schema` (See [related section](#chain-of-thought-schema)) towards a massive textual collections.
 
-### Features
+### Main Features
 * ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
-* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
-* ✅ **Progress caching**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
 * ✅ **Support schemas descriptions** for Chain-of-Thought concept.
+* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
+
+### Extra Features
+* ✅ **Progress caching [for remote LLMs]**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
+
 
 # Installation
 
+From PyPI: 
+
 ```bash
 pip install bulk-chain
 ```
 
+or latest version from here:
+
+```bash
+pip install git+https://github.com/nicolay-r/bulk-chain@master
+```
+
 ## Chain-of-Thought Schema
 
 To declare Chain-of-Though (CoT) schema, this project exploits `JSON` format.
@@ -63,35 +73,37 @@ Below, is an example on how to declare your own schema:
 }
 ```
 
-Another templates are available [here](/ext/schema/thor_cot_schema.json).
+Another templates are available [here](/ext/schema/).
 
 # Usage
 
-Just **three** simple steps:
+Preliminary steps:
 
-1. Define your [CoT Schema](#chain-of-thought-schema), or fetch it as shown below:
-```bash
-!wget https://raw.githubusercontent.com/nicolay-r/bulk-chain/refs/heads/master/ext/schema/default.json
-```
-2. Fetch or write your own **model** or pick the one [preset here](/ext/):
-```bash
-!wget https://raw.githubusercontent.com/nicolay-r/bulk-chain/refs/heads/master/ext/flan_t5.py
-```
+1. Define your [schema](#chain-of-thought-schema) ([Example for Sentiment Analysis](/ext/schema/thor_cot_schema.json)))
+2. Wrap or pick **LLM model** from the [list of presets](/ext/).
+
+## API
+
+Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
+
+## Shell
+
+> **NOTE:** You have to install `source-iter` package
 
-3. Launch inference in (chat mode):
 ```bash
-!python -m bulk_chain.infer \
-    --schema "default.json" \
-    --adapter "dynamic:flan_t5.py:FlanT5" \
-    %% \
-    --device "cpu" \
+python3 -m bulk_chain.infer \
+    --src "<PATH-TO-YOUR-CSV-or-JSONL>" \
+    --schema "ext/schema/default.json" \
+    --adapter "dynamic:ext/replicate.py:Replicate" \
+    %%m \
+    --api_token "<REPLICATE-API-TOKEN>" \
     --temp 0.1
 ```
 
 # Embed your LLM
 
 All you have to do is to implement `BaseLM` class, that includes:
-* `__init__` -- for initialization;
+* `__init__` -- for setting up *batching mode support* and (optional) *model name*;
 * `ask(prompt)` -- infer your model with the given `prompt`.
 
 See examples with models [here](/ext).

{bulk_chain-0.24.1 → bulk_chain-0.25.0}/README.md

@@ -1,27 +1,38 @@
-# bulk-chain 0.24.1
+# bulk-chain 0.25.0
 ![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
 [![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
 [![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
+[![PyPI downloads](https://img.shields.io/pypi/dm/bulk-chain.svg)](https://pypistats.org/packages/bulk-chain)
 
 <p align="center">
     <img src="logo.png"/>
 </p>
 
-A lightweight, no-strings-attached **[Chain-of-Thought](https://arxiv.org/abs/2201.11903) framework** for your LLM, ensuring reliable results for bulk input requests stored in `CSV` / `JSONL` / `sqlite`.
-It allows applying series of prompts formed into `schema` (See [related section](#chain-of-thought-schema))
+A lightweight, no-strings-attached **framework**  for your LLM that allows applying [Chain-of-Thought](https://arxiv.org/abs/2201.11903) prompt `schema` (See [related section](#chain-of-thought-schema)) towards a massive textual collections.
 
-### Features
+### Main Features
 * ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
-* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
-* ✅ **Progress caching**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
 * ✅ **Support schemas descriptions** for Chain-of-Thought concept.
+* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
+
+### Extra Features
+* ✅ **Progress caching [for remote LLMs]**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
+
 
 # Installation
 
+From PyPI: 
+
 ```bash
 pip install bulk-chain
 ```
 
+or latest version from here:
+
+```bash
+pip install git+https://github.com/nicolay-r/bulk-chain@master
+```
+
 ## Chain-of-Thought Schema
 
 To declare Chain-of-Though (CoT) schema, this project exploits `JSON` format.
@@ -44,35 +55,37 @@ Below, is an example on how to declare your own schema:
 }
 ```
 
-Another templates are available [here](/ext/schema/thor_cot_schema.json).
+Another templates are available [here](/ext/schema/).
 
 # Usage
 
-Just **three** simple steps:
+Preliminary steps:
 
-1. Define your [CoT Schema](#chain-of-thought-schema), or fetch it as shown below:
-```bash
-!wget https://raw.githubusercontent.com/nicolay-r/bulk-chain/refs/heads/master/ext/schema/default.json
-```
-2. Fetch or write your own **model** or pick the one [preset here](/ext/):
-```bash
-!wget https://raw.githubusercontent.com/nicolay-r/bulk-chain/refs/heads/master/ext/flan_t5.py
-```
+1. Define your [schema](#chain-of-thought-schema) ([Example for Sentiment Analysis](/ext/schema/thor_cot_schema.json)))
+2. Wrap or pick **LLM model** from the [list of presets](/ext/).
+
+## API
+
+Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)
+
+## Shell
+
+> **NOTE:** You have to install `source-iter` package
 
-3. Launch inference in (chat mode):
 ```bash
-!python -m bulk_chain.infer \
-    --schema "default.json" \
-    --adapter "dynamic:flan_t5.py:FlanT5" \
-    %% \
-    --device "cpu" \
+python3 -m bulk_chain.infer \
+    --src "<PATH-TO-YOUR-CSV-or-JSONL>" \
+    --schema "ext/schema/default.json" \
+    --adapter "dynamic:ext/replicate.py:Replicate" \
+    %%m \
+    --api_token "<REPLICATE-API-TOKEN>" \
     --temp 0.1
 ```
 
 # Embed your LLM
 
 All you have to do is to implement `BaseLM` class, that includes:
-* `__init__` -- for initialization;
+* `__init__` -- for setting up *batching mode support* and (optional) *model name*;
 * `ask(prompt)` -- infer your model with the given `prompt`.
 
 See examples with models [here](/ext).

bulk_chain-0.25.0/bulk_chain/api.py

@@ -0,0 +1,79 @@
+import os
+from itertools import chain
+
+from bulk_chain.core.llm_base import BaseLM
+from bulk_chain.core.service_batch import BatchIterator, BatchService
+from bulk_chain.core.service_data import DataService
+from bulk_chain.core.service_dict import DictionaryService
+from bulk_chain.core.service_json import JsonService
+from bulk_chain.core.service_schema import SchemaService
+
+
+INFER_MODES = {
+    "default": lambda llm, prompt, limit_prompt=None: llm.ask_core(
+        prompt[:limit_prompt] if limit_prompt is not None else prompt),
+    "batch": lambda llm, batch, limit_prompt=None: llm.ask_core(
+        DataService.limit_prompts(batch, limit=limit_prompt))
+}
+
+
+CWD = os.getcwd()
+
+
+def _update_batch_content(c, batch, schema, infer_func):
+    assert (isinstance(batch, list))
+    assert (isinstance(c, str))
+
+    if c in schema.p2r:
+        for batch_item in batch:
+            batch_item[c] = DataService.get_prompt_text(prompt=batch_item[c]["prompt"], data_dict=batch_item)
+    if c in schema.r2p:
+        p_column = schema.r2p[c]
+        # This instruction takes a lot of time in a non-batching mode.
+        BatchService.handle_param_as_batch(batch=batch,
+                                           src_param=p_column,
+                                           tgt_param=c,
+                                           handle_func=lambda b: infer_func(b))
+
+
+def _infer_batch(batch, schema, infer_func, cols=None):
+    assert (isinstance(batch, list))
+    assert (callable(infer_func))
+
+    if len(batch) == 0:
+        return batch
+
+    if cols is None:
+        first_item = batch[0]
+        cols = first_item.keys() if cols is None else cols
+
+    for c in cols:
+        _update_batch_content(c=c, batch=batch, schema=schema, infer_func=infer_func)
+
+    return batch
+
+
+def iter_content(input_dicts_it, llm, schema, batch_size=1, return_batch=True, limit_prompt=None):
+    """ This method represent Python API aimed at application of `llm` towards
+        iterator of input_dicts via cache_target that refers to the SQLite using
+        the given `schema`
+    """
+    assert (isinstance(llm, BaseLM))
+
+    # Quick initialization of the schema.
+    if isinstance(schema, str):
+        schema = JsonService.read(schema)
+    if isinstance(schema, dict):
+        schema = SchemaService(json_data=schema)
+
+    prompts_it = map(
+        lambda data: DictionaryService.custom_update(src_dict=data, other_dict=schema.cot_args),
+        input_dicts_it
+    )
+
+    content_it = (_infer_batch(batch=batch,
+                               infer_func=lambda batch: INFER_MODES["batch"](llm, batch, limit_prompt),
+                               schema=schema)
+                  for batch in BatchIterator(prompts_it, batch_size=batch_size))
+
+    yield from content_it if return_batch else chain.from_iterable(content_it)

bulk_chain-0.25.0/bulk_chain/core/llm_base.py

@@ -0,0 +1,52 @@
+import logging
+import time
+
+from bulk_chain.core.utils import format_model_name
+
+
+class BaseLM(object):
+
+    def __init__(self, name=None, attempts=None, delay_sec=1, enable_log=True,
+                 support_batching=False, **kwargs):
+
+        self.__name = name
+        self.__attempts = 1 if attempts is None else attempts
+        self.__delay_sec = delay_sec
+        self.__support_batching = support_batching
+
+        if enable_log:
+            self.__logger = logging.getLogger(__name__)
+            logging.basicConfig(level=logging.INFO)
+
+    def ask_core(self, batch):
+
+        for i in range(self.__attempts):
+            try:
+                if self.__support_batching:
+                    # Launch in batch mode.
+                    content = self.ask(batch)
+                else:
+                    # Launch in non-batch mode.
+                    assert len(batch) == 1, "The LM does not support batching," \
+                                            f" while size of the content is {len(batch)} which is not equal 1. " \
+                                            f"Please enable batch-supporting or set required inference settings."
+                    content = batch[0]
+
+                response = self.ask(content)
+
+                # Wrapping into batch the response in the case of non-batching mode.
+                return response if self.__support_batching else [response]
+
+            except Exception as e:
+                if self.__logger is not None:
+                    self.__logger.info("Unable to infer the result. Try {} out of {}.".format(i, self.__attempts))
+                    self.__logger.info(e)
+                time.sleep(self.__delay_sec)
+
+        raise Exception("Can't infer")
+
+    def ask(self, content):
+        raise NotImplemented()
+
+    def name(self):
+        return format_model_name(self.__name)

{bulk_chain-0.24.1 → bulk_chain-0.25.0}/bulk_chain/core/service_args.py

@@ -33,14 +33,33 @@ class CmdArgsService:
             yield __release()
 
     @staticmethod
-    def partition_list(lst, sep):
+    def __find_suffix_ind(lst, idx_from, end_prefix):
+        for i in range(idx_from, len(lst)):
+            if lst[i].startswith(end_prefix):
+                return i
+        return len(lst)
+
+    @staticmethod
+    def extract_native_args(lst, end_prefix):
+        return lst[:CmdArgsService.__find_suffix_ind(lst, idx_from=0, end_prefix=end_prefix)]
+
+    @staticmethod
+    def find_grouped_args(lst, starts_with, end_prefix):
         """Slices a list in two, cutting on index matching "sep"
         """
-        if sep in lst:
-            idx = lst.index(sep)
-            return (lst[:idx], lst[idx+1:])
-        else:
-            return (lst[:], None)
+
+        # Checking the presence of starts_with.
+        # We have to return empty content in the case of absence starts_with in the lst.
+        if starts_with not in lst:
+            return []
+
+        # Assigning start index.
+        idx_from = lst.index(starts_with) + 1
+
+        # Assigning end index.
+        idx_to = CmdArgsService.__find_suffix_ind(lst, idx_from=idx_from, end_prefix=end_prefix)
+
+        return lst[idx_from:idx_to]
 
     @staticmethod
     def args_to_dict(args):

bulk_chain-0.25.0/bulk_chain/core/service_batch.py

@@ -0,0 +1,51 @@
+class BatchService(object):
+
+    @staticmethod
+    def handle_param_as_batch(batch, src_param, tgt_param, handle_func):
+        assert (isinstance(batch, list))
+        assert (isinstance(src_param, str))
+        assert (callable(handle_func))
+
+        _batch = [item[src_param] for item in batch]
+
+        # Do handling for the batch.
+        _handled_batch = handle_func(_batch)
+        assert (isinstance(_handled_batch, list))
+
+        # Apply changes.
+        for i, item in enumerate(batch):
+            item[tgt_param] = _handled_batch[i]
+
+
+class BatchIterator:
+
+    def __init__(self, data_iter, batch_size, end_value=None):
+        assert(isinstance(batch_size, int) and batch_size > 0)
+        assert(callable(end_value) or end_value is None)
+        self.__data_iter = data_iter
+        self.__index = 0
+        self.__batch_size = batch_size
+        self.__end_value = end_value
+
+    def __iter__(self):
+        return self
+
+    def __next__(self):
+        buffer = []
+        while True:
+            try:
+                data = next(self.__data_iter)
+            except StopIteration:
+                break
+            buffer.append(data)
+            if len(buffer) == self.__batch_size:
+                break
+
+        if len(buffer) > 0:
+            self.__index += 1
+            return buffer
+
+        if self.__end_value is None:
+            raise StopIteration
+        else:
+            return self.__end_value()

{bulk_chain-0.24.1 → bulk_chain-0.25.0}/bulk_chain/core/service_data.py

@@ -20,3 +20,7 @@ class DataService(object):
         field_names = list(parse_fields_func(prompt))
         return DataService.compose_prompt_text(
             prompt=prompt, data_dict=data_dict, field_names=field_names)
+
+    @staticmethod
+    def limit_prompts(prompts_list, limit=None):
+        return [p[:limit] if limit is not None else p for p in prompts_list]

bulk_chain-0.25.0/bulk_chain/core/service_dict.py

@@ -0,0 +1,10 @@
+class DictionaryService:
+
+    @staticmethod
+    def custom_update(src_dict, other_dict):
+        for k, v in other_dict.items():
+            if k in src_dict:
+                raise Exception(f"The key `{k}` is already defined in both dicts with values: "
+                                f"`{src_dict[k]}` (src) and `{v}` (other)")
+            src_dict[k] = v
+        return src_dict

bulk_chain-0.25.0/bulk_chain/core/service_json.py

@@ -0,0 +1,10 @@
+import json
+
+
+class JsonService(object):
+
+    @staticmethod
+    def read(src):
+        assert (isinstance(src, str))
+        with open(src, "r") as f:
+            return json.load(f)

{bulk_chain-0.24.1 → bulk_chain-0.25.0}/bulk_chain/core/service_llm.py

@@ -4,9 +4,6 @@ from bulk_chain.core.llm_base import BaseLM
 from bulk_chain.core.service_data import DataService
 from bulk_chain.core.utils import iter_params
 
-logger = logging.getLogger(__name__)
-logging.basicConfig(level=logging.INFO)
-
 
 def pad_str(text, pad):
     return text.rjust(len(text) + pad, ' ')
@@ -27,9 +24,12 @@ def nice_output(text, width, pad=4, remove_new_line=False):
 
 
 def chat_with_lm(lm, chain=None, model_name=None):
-    assert(isinstance(lm, BaseLM))
-    assert(isinstance(chain, list))
-    assert(isinstance(model_name, str) or model_name is None)
+    assert (isinstance(lm, BaseLM))
+    assert (isinstance(chain, list))
+    assert (isinstance(model_name, str) or model_name is None)
+
+    logger = logging.getLogger(__name__)
+    logging.basicConfig(level=logging.INFO)
 
     do_exit = False
     model_name = model_name if model_name is not None else "agent"
@@ -74,9 +74,9 @@ def chat_with_lm(lm, chain=None, model_name=None):
             logger.info(nice_output(actual_prompt, pad=pad*2, remove_new_line=True, width=80))
 
             # Response.
-            response = lm.ask(actual_prompt)
+            response_batch = lm.ask_core(batch=[actual_prompt])
             logger.info(pad_str(f"{model_name} (resp)->", pad=pad))
-            logger.info(nice_output(response, pad=pad*2, remove_new_line=False, width=80))
+            logger.info(nice_output(response_batch[0], pad=pad * 2, remove_new_line=False, width=80))
 
             # Collecting the answer for the next turn.
-            data_dict[prompt_args["out"]] = response
+            data_dict[prompt_args["out"]] = response_batch[0]

{bulk_chain-0.24.1 → bulk_chain-0.25.0}/bulk_chain/core/service_schema.py

@@ -2,12 +2,11 @@ class SchemaService(object):
 
     def __init__(self, json_data):
         self.src = json_data
-        self.name = self.src["name"]
         self.r2p, self.p2r, self.cot_args, self.chain = SchemaService.__init_schema(prompts=json_data["schema"])
 
     @classmethod
     def from_prompt(cls, prompt):
-        prompt_schema = {"name": "prompt", "schema": [{"prompt": prompt, "out": "response", "in": "prompt"}]}
+        prompt_schema = {"schema": [{"prompt": prompt, "out": "response", "in": "prompt"}]}
         return cls(prompt_schema)
 
     @staticmethod