bulk-chain 0.25.2__tar.gz → 1.0.0__tar.gz

{bulk_chain-0.25.2 → bulk_chain-1.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: bulk_chain
-Version: 0.25.2
+Version: 1.0.0
 Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
 Home-page: https://github.com/nicolay-r/bulk-chain
 Author: Nicolay Rusnachenko
@@ -16,9 +16,8 @@ Requires-Python: >=3.6
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: tqdm
-Requires-Dist: source-iter==0.24.3
 
-# bulk-chain 0.25.2
+# bulk-chain 1.0.0
 ![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
 [![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
 [![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
@@ -31,7 +30,7 @@ Requires-Dist: source-iter==0.24.3
 <p align="center">
   <a href="https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm"><b>Third-party providers hosting</b>↗️</a>
   <br>
-  <a href="https://github.com/nicolay-r/bulk-chain/blob/master/README.md#demo-mode">👉<b>demo</b>👈</a>
+  <a href="https://github.com/nicolay-r/bulk-chain-shell">👉<b>demo</b>👈</a>
 </p>
 
 A no-strings-attached **framework**  for your LLM that allows applying Chain-of-Thought-alike [prompt `schema`](#chain-of-thought-schema) towards a massive textual collections using custom **[third-party providers ↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm)**.
@@ -39,11 +38,7 @@ A no-strings-attached **framework**  for your LLM that allows applying Chain-of-
 ### Main Features
 * ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
 * ✅ **Support schemas descriptions** for Chain-of-Thought concept.
-* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
-
-### Extra Features
-* ✅ **Progress caching [for remote LLMs]**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
-
+* ✅ **Provides iterator over infinite amount of input contexts**
 
 # Installation
 
@@ -88,51 +83,8 @@ Preliminary steps:
 1. Define your [schema](#chain-of-thought-schema) ([Example for Sentiment Analysis](/ext/schema/thor_cot_schema.json)))
 2. Wrap or pick **LLM model** from the [<b>Third-party providers hosting</b>↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
 
-## Shell
-
-### Demo Mode
-
-**demo mode** to interact with LLM via command line with LLM output streaming support. 
-The video below illustrates an example of application for sentiment analysis on author opinion extraction towards mentioned object in text.
-
-Quck start with launching demo:
-1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
-2. 📜 Setup your reasoning `thor_cot_schema.json` according to the [following example ↗️](test/schema/thor_cot_schema.json)
-3. 🚀 Launch `demo.py` as follows:
-```bash
-python3 -m bulk_chain.demo \
-    --schema "test/schema/thor_cot_schema.json" \
-    --adapter "dynamic:replicate_104.py:Replicate" \
-    %%m \
-    --model_name "meta/meta-llama-3-70b-instruct" \
-    --api_token "<REPLICATE-API-TOKEN>" \
-    --stream
-```
-
-📺 This video showcase application of the [↗️ Sentiment Analysis Schema](https://github.com/nicolay-r/bulk-chain/blob/master/test/schema/thor_cot_schema.json) towards [LLaMA-3-70B-Instruct](https://replicate.com/meta/meta-llama-3-70b-instruct) hosted by Replicate for reasoning over submitted texts
-![sa-bulk-chain-cot-final](https://github.com/user-attachments/assets/0cc8fdcb-6ddb-44a3-8f05-d76250ae6423)
 
 
-### Inference Mode
-
-> **NOTE:** You have to install `source-iter` and `tqdm` packages that actual [dependencies](dependencies.txt) of this project
-
-1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
-```bash
-wget https://raw.githubusercontent.com/nicolay-r/nlp-thirdgate/refs/heads/master/llm/replicate_104.py
-```
-2. 📜 Setup your reasoning `schema.json` according to the [following example ↗️](test/schema/default.json)
-3. 🚀 Launch inference using `DeepSeek-R1`:
-```bash
-python3 -m bulk_chain.infer \
-    --src "<PATH-TO-YOUR-CSV-or-JSONL>" \
-    --schema "test/schema/default.json" \
-    --adapter "replicate_104.py:Replicate" \
-    %%m \
-    --model_name "deepseek-ai/deepseek-r1" \
-    --api_token "<REPLICATE-API-TOKEN>"
-```
-
 ## API
 
 Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)

{bulk_chain-0.25.2 → bulk_chain-1.0.0}/README.md

@@ -1,4 +1,4 @@
-# bulk-chain 0.25.2
+# bulk-chain 1.0.0
 ![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
 [![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
 [![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
@@ -11,7 +11,7 @@
 <p align="center">
   <a href="https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm"><b>Third-party providers hosting</b>↗️</a>
   <br>
-  <a href="https://github.com/nicolay-r/bulk-chain/blob/master/README.md#demo-mode">👉<b>demo</b>👈</a>
+  <a href="https://github.com/nicolay-r/bulk-chain-shell">👉<b>demo</b>👈</a>
 </p>
 
 A no-strings-attached **framework**  for your LLM that allows applying Chain-of-Thought-alike [prompt `schema`](#chain-of-thought-schema) towards a massive textual collections using custom **[third-party providers ↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm)**.
@@ -19,11 +19,7 @@ A no-strings-attached **framework**  for your LLM that allows applying Chain-of-
 ### Main Features
 * ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
 * ✅ **Support schemas descriptions** for Chain-of-Thought concept.
-* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
-
-### Extra Features
-* ✅ **Progress caching [for remote LLMs]**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
-
+* ✅ **Provides iterator over infinite amount of input contexts**
 
 # Installation
 
@@ -68,51 +64,8 @@ Preliminary steps:
 1. Define your [schema](#chain-of-thought-schema) ([Example for Sentiment Analysis](/ext/schema/thor_cot_schema.json)))
 2. Wrap or pick **LLM model** from the [<b>Third-party providers hosting</b>↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
 
-## Shell
-
-### Demo Mode
-
-**demo mode** to interact with LLM via command line with LLM output streaming support. 
-The video below illustrates an example of application for sentiment analysis on author opinion extraction towards mentioned object in text.
-
-Quck start with launching demo:
-1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
-2. 📜 Setup your reasoning `thor_cot_schema.json` according to the [following example ↗️](test/schema/thor_cot_schema.json)
-3. 🚀 Launch `demo.py` as follows:
-```bash
-python3 -m bulk_chain.demo \
-    --schema "test/schema/thor_cot_schema.json" \
-    --adapter "dynamic:replicate_104.py:Replicate" \
-    %%m \
-    --model_name "meta/meta-llama-3-70b-instruct" \
-    --api_token "<REPLICATE-API-TOKEN>" \
-    --stream
-```
-
-📺 This video showcase application of the [↗️ Sentiment Analysis Schema](https://github.com/nicolay-r/bulk-chain/blob/master/test/schema/thor_cot_schema.json) towards [LLaMA-3-70B-Instruct](https://replicate.com/meta/meta-llama-3-70b-instruct) hosted by Replicate for reasoning over submitted texts
-![sa-bulk-chain-cot-final](https://github.com/user-attachments/assets/0cc8fdcb-6ddb-44a3-8f05-d76250ae6423)
 
 
-### Inference Mode
-
-> **NOTE:** You have to install `source-iter` and `tqdm` packages that actual [dependencies](dependencies.txt) of this project
-
-1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
-```bash
-wget https://raw.githubusercontent.com/nicolay-r/nlp-thirdgate/refs/heads/master/llm/replicate_104.py
-```
-2. 📜 Setup your reasoning `schema.json` according to the [following example ↗️](test/schema/default.json)
-3. 🚀 Launch inference using `DeepSeek-R1`:
-```bash
-python3 -m bulk_chain.infer \
-    --src "<PATH-TO-YOUR-CSV-or-JSONL>" \
-    --schema "test/schema/default.json" \
-    --adapter "replicate_104.py:Replicate" \
-    %%m \
-    --model_name "deepseek-ai/deepseek-r1" \
-    --api_token "<REPLICATE-API-TOKEN>"
-```
-
 ## API
 
 Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)

bulk_chain-1.0.0/bulk_chain/api.py

@@ -0,0 +1,143 @@
+import collections
+import os
+from itertools import chain
+
+from bulk_chain.core.llm_base import BaseLM
+from bulk_chain.core.service_batch import BatchIterator
+from bulk_chain.core.service_data import DataService
+from bulk_chain.core.service_dict import DictionaryService
+from bulk_chain.core.service_json import JsonService
+from bulk_chain.core.service_schema import SchemaService
+from bulk_chain.core.utils import dynamic_init, find_by_prefix
+
+
+INFER_MODES = {
+    "batch": lambda llm, batch, limit_prompt=None: llm.ask_core(
+        DataService.limit_prompts(batch, limit=limit_prompt))
+}
+
+
+CWD = os.getcwd()
+
+
+def _iter_entry_content(entry, entry_info=None, **kwargs):
+
+    if isinstance(entry, str):
+        kwargs.get("callback_str_func", lambda *_: None)(entry, entry_info)
+        yield entry
+    elif isinstance(entry, collections.abc.Iterable):
+        h = kwargs.get("callback_stream_func", lambda *_: None)
+        h(None, entry_info | {"action": "start"})
+        for chunk in map(lambda item: str(item), entry):
+            yield chunk
+            h(chunk, entry_info)
+        h(None, entry_info | {"action": "end"})
+    else:
+        raise Exception(f"Non supported type `{type(entry)}` for handling output from batch")
+
+
+def _iter_batch_prompts(c, batch_content_it, **kwargs):
+    for ind_in_batch, entry in enumerate(batch_content_it):
+        content = DataService.get_prompt_text(
+            prompt=entry[c]["prompt"],
+            data_dict=entry,
+            handle_missed_func=kwargs["handle_missed_value_func"])
+        yield ind_in_batch, content
+
+
+def _iter_batch_responses(p_column, c, batch_content_it, **kwargs):
+    p_batch = [item[p_column] for item in batch_content_it]
+    # TODO. This part could be async.
+    # TODO. ind_in_batch might be a part of the async return.
+    for ind_in_batch, entry in enumerate(kwargs["handle_batch_func"](p_batch)):
+        yield ind_in_batch, _iter_entry_content(entry=entry, entry_info={"ind": ind_in_batch, "param": c}, **kwargs)
+
+
+def _infer_batch(batch, schema, return_mode, cols=None, **kwargs):
+    assert (isinstance(batch, list))
+
+    if len(batch) == 0:
+        return batch
+
+    if cols is None:
+        first_item = batch[0]
+        cols = list(first_item.keys()) if cols is None else cols
+
+    for c in cols:
+
+        # Handling prompt column.
+        if c in schema.p2r:
+            content_it = _iter_batch_prompts(c=c, batch_content_it=iter(batch), **kwargs)
+            for ind_in_batch, prompt in content_it:
+                batch[ind_in_batch][c] = prompt
+
+        # Handling column for inference.
+        if c in schema.r2p:
+            content_it = _iter_batch_responses(c=c, p_column=schema.r2p[c], batch_content_it=iter(batch), **kwargs)
+            for ind_in_batch, chunk_it in content_it:
+
+                chunks = []
+                for chunk in chunk_it:
+                    chunks.append(chunk)
+
+                    if return_mode == "chunk":
+                        yield [ind_in_batch, c, chunk]
+
+                batch[ind_in_batch][c] = "".join(chunks)
+
+    if return_mode == "record":
+        for record in batch:
+            yield record
+
+    if return_mode == "batch":
+        yield batch
+
+
+def iter_content(input_dicts_it, llm, schema, batch_size=1, limit_prompt=None, return_mode="batch", **kwargs):
+    """ This method represent Python API aimed at application of `llm` towards
+        iterator of input_dicts via cache_target that refers to the SQLite using
+        the given `schema`
+    """
+    assert (return_mode in ["batch", "chunk"])
+    assert (isinstance(llm, BaseLM))
+
+    # Quick initialization of the schema.
+    if isinstance(schema, str):
+        schema = JsonService.read(schema)
+    if isinstance(schema, dict):
+        schema = SchemaService(json_data=schema)
+
+    prompts_it = map(
+        lambda data: DictionaryService.custom_update(src_dict=dict(data), other_dict=schema.cot_args),
+        input_dicts_it
+    )
+
+    content_it = (_infer_batch(batch=batch,
+                               handle_batch_func=lambda batch: INFER_MODES["batch"](llm, batch, limit_prompt),
+                               return_mode=return_mode,
+                               schema=schema,
+                               **kwargs)
+                  for batch in BatchIterator(prompts_it, batch_size=batch_size))
+
+    yield from chain.from_iterable(content_it)
+
+
+def init_llm(adapter, **model_kwargs):
+    """ This method perform dynamic initialization of LLM from third-party resource.
+    """
+    assert (isinstance(adapter, str))
+
+    # List of the Supported models and their API wrappers.
+    models_preset = {
+        "dynamic": lambda: dynamic_init(class_dir=CWD, class_filepath=llm_model_name,
+                                        class_name=llm_model_params)(**model_kwargs)
+    }
+
+    # Initialize LLM model.
+    params = adapter.split(':')
+    llm_model_type = params[0]
+    llm_model_name = params[1] if len(params) > 1 else params[-1]
+    llm_model_params = ':'.join(params[2:]) if len(params) > 2 else None
+    llm = find_by_prefix(d=models_preset, key=llm_model_type)()
+
+    return llm, llm_model_name

{bulk_chain-0.25.2 → bulk_chain-1.0.0}/bulk_chain/core/llm_base.py

@@ -1,8 +1,6 @@
 import logging
 import time
 
-from bulk_chain.core.utils import format_model_name
-
 
 class BaseLM(object):
 
@@ -49,4 +47,4 @@ class BaseLM(object):
         raise NotImplemented()
 
     def name(self):
-        return format_model_name(self.__name)
+        return self.__name.replace("/", "_")

{bulk_chain-0.25.2 → bulk_chain-1.0.0}/bulk_chain/core/service_batch.py

@@ -1,31 +1,13 @@
-class BatchService(object):
-
-    @staticmethod
-    def handle_param_as_batch(batch, src_param, tgt_param, handle_func):
-        assert (isinstance(batch, list))
-        assert (isinstance(src_param, str))
-        assert (callable(handle_func))
-
-        _batch = [item[src_param] for item in batch]
-
-        # Do handling for the batch.
-        _handled_batch = handle_func(_batch)
-        assert (isinstance(_handled_batch, list))
-
-        # Apply changes.
-        for i, item in enumerate(batch):
-            item[tgt_param] = _handled_batch[i]
-
-
 class BatchIterator:
 
-    def __init__(self, data_iter, batch_size, end_value=None):
+    def __init__(self, data_iter, batch_size, end_value=None, filter_func=None):
         assert(isinstance(batch_size, int) and batch_size > 0)
         assert(callable(end_value) or end_value is None)
         self.__data_iter = data_iter
         self.__index = 0
         self.__batch_size = batch_size
         self.__end_value = end_value
+        self.__filter_func = (lambda _: True) if filter_func is None else filter_func
 
     def __iter__(self):
         return self
@@ -37,7 +19,8 @@ class BatchIterator:
                 data = next(self.__data_iter)
             except StopIteration:
                 break
-            buffer.append(data)
+            if self.__filter_func(data):
+                buffer.append(data)
             if len(buffer) == self.__batch_size:
                 break
 

{bulk_chain-0.25.2 → bulk_chain-1.0.0}/bulk_chain/core/service_data.py

@@ -4,8 +4,8 @@ from bulk_chain.core.utils import iter_params
 class DataService(object):
 
     @staticmethod
-    def compose_prompt_text(prompt, data_dict, field_names):
-        assert(isinstance(data_dict, dict))
+    def __compose_prompt_text(prompt, data_dict, field_names):
+        assert (isinstance(data_dict, dict))
         fmt_d = {col_name: data_dict[col_name] for col_name in field_names}
 
         # Guarantee that items has correct type.
@@ -16,10 +16,14 @@ class DataService(object):
         return prompt.format(**fmt_d)
 
     @staticmethod
-    def get_prompt_text(prompt, data_dict, parse_fields_func=iter_params):
+    def get_prompt_text(prompt, data_dict, parse_fields_func=iter_params, handle_missed_func=None):
         field_names = list(parse_fields_func(prompt))
-        return DataService.compose_prompt_text(
-            prompt=prompt, data_dict=data_dict, field_names=field_names)
+
+        for col_name in field_names:
+            if col_name not in data_dict:
+                data_dict[col_name] = handle_missed_func(col_name)
+
+        return DataService.__compose_prompt_text(prompt=prompt, data_dict=data_dict, field_names=field_names)
 
     @staticmethod
     def limit_prompts(prompts_list, limit=None):

{bulk_chain-0.25.2 → bulk_chain-1.0.0}/bulk_chain/core/utils.py

@@ -2,6 +2,7 @@ import importlib
 import logging
 import sys
 from collections import Counter
+from os.path import dirname, join, basename
 
 logger = logging.getLogger(__name__)
 logging.basicConfig(level=logging.INFO)
@@ -47,28 +48,6 @@ def iter_params(text):
         beg = pe+1
 
 
-def format_model_name(name):
-    return name.replace("/", "_")
-
-
-def parse_filepath(filepath, default_filepath=None, default_ext=None):
-    """ This is an auxiliary function for handling sources and targets from cmd string.
-    """
-    if filepath is None:
-        return default_filepath, default_ext, None
-    info = filepath.split(":")
-    filepath = info[0]
-    meta = info[1] if len(info) > 1 else None
-    ext = filepath.split('.')[-1] if default_ext is None else default_ext
-    return filepath, ext, meta
-
-
-def handle_table_name(name):
-    return name.\
-        replace('-', '_').\
-        replace('.', "_")
-
-
 def auto_import(name, is_class=False):
     """ Import from the external python packages.
     """
@@ -82,13 +61,24 @@ def auto_import(name, is_class=False):
 
 
 def dynamic_init(class_dir, class_filepath, class_name=None):
-    sys.path.append(class_dir)
+
+    # Registering path.
+    target = join(class_dir, dirname(class_filepath))
+    logger.info(f"Adding sys path for `{target}`")
+    sys.path.insert(1, target)
     class_path_list = class_filepath.split('/')
-    class_path_list[-1] = '.'.join(class_path_list[-1].split('.')[:-1])
+
+    # Composing proper class name.
+    class_filename = basename(class_path_list[-1])
+    if class_filename.endswith(".py"):
+        class_filename = class_filename[:-len(".py")]
+
+    # Loading library.
     class_name = class_path_list[-1].title() if class_name is None else class_name
-    class_path = ".".join(class_path_list + [class_name])
+    class_path = ".".join([class_filename, class_name])
     logger.info(f"Dynamic loading for the file and class `{class_path}`")
     cls = auto_import(class_path, is_class=False)
+
     return cls
 
 

{bulk_chain-0.25.2 → bulk_chain-1.0.0}/bulk_chain.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: bulk_chain
-Version: 0.25.2
+Version: 1.0.0
 Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
 Home-page: https://github.com/nicolay-r/bulk-chain
 Author: Nicolay Rusnachenko
@@ -16,9 +16,8 @@ Requires-Python: >=3.6
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: tqdm
-Requires-Dist: source-iter==0.24.3
 
-# bulk-chain 0.25.2
+# bulk-chain 1.0.0
 ![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
 [![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
 [![twitter](https://img.shields.io/twitter/url/https/shields.io.svg?style=social)](https://x.com/nicolayr_/status/1847969224636961033)
@@ -31,7 +30,7 @@ Requires-Dist: source-iter==0.24.3
 <p align="center">
   <a href="https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm"><b>Third-party providers hosting</b>↗️</a>
   <br>
-  <a href="https://github.com/nicolay-r/bulk-chain/blob/master/README.md#demo-mode">👉<b>demo</b>👈</a>
+  <a href="https://github.com/nicolay-r/bulk-chain-shell">👉<b>demo</b>👈</a>
 </p>
 
 A no-strings-attached **framework**  for your LLM that allows applying Chain-of-Thought-alike [prompt `schema`](#chain-of-thought-schema) towards a massive textual collections using custom **[third-party providers ↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm)**.
@@ -39,11 +38,7 @@ A no-strings-attached **framework**  for your LLM that allows applying Chain-of-
 ### Main Features
 * ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
 * ✅ **Support schemas descriptions** for Chain-of-Thought concept.
-* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
-
-### Extra Features
-* ✅ **Progress caching [for remote LLMs]**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
-
+* ✅ **Provides iterator over infinite amount of input contexts**
 
 # Installation
 
@@ -88,51 +83,8 @@ Preliminary steps:
 1. Define your [schema](#chain-of-thought-schema) ([Example for Sentiment Analysis](/ext/schema/thor_cot_schema.json)))
 2. Wrap or pick **LLM model** from the [<b>Third-party providers hosting</b>↗️](https://github.com/nicolay-r/nlp-thirdgate?tab=readme-ov-file#llm).
 
-## Shell
-
-### Demo Mode
-
-**demo mode** to interact with LLM via command line with LLM output streaming support. 
-The video below illustrates an example of application for sentiment analysis on author opinion extraction towards mentioned object in text.
-
-Quck start with launching demo:
-1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
-2. 📜 Setup your reasoning `thor_cot_schema.json` according to the [following example ↗️](test/schema/thor_cot_schema.json)
-3. 🚀 Launch `demo.py` as follows:
-```bash
-python3 -m bulk_chain.demo \
-    --schema "test/schema/thor_cot_schema.json" \
-    --adapter "dynamic:replicate_104.py:Replicate" \
-    %%m \
-    --model_name "meta/meta-llama-3-70b-instruct" \
-    --api_token "<REPLICATE-API-TOKEN>" \
-    --stream
-```
-
-📺 This video showcase application of the [↗️ Sentiment Analysis Schema](https://github.com/nicolay-r/bulk-chain/blob/master/test/schema/thor_cot_schema.json) towards [LLaMA-3-70B-Instruct](https://replicate.com/meta/meta-llama-3-70b-instruct) hosted by Replicate for reasoning over submitted texts
-![sa-bulk-chain-cot-final](https://github.com/user-attachments/assets/0cc8fdcb-6ddb-44a3-8f05-d76250ae6423)
 
 
-### Inference Mode
-
-> **NOTE:** You have to install `source-iter` and `tqdm` packages that actual [dependencies](dependencies.txt) of this project
-
-1. ⬇️ Download [replicate](https://replicate.com/) provider for `bulk-chain`:
-```bash
-wget https://raw.githubusercontent.com/nicolay-r/nlp-thirdgate/refs/heads/master/llm/replicate_104.py
-```
-2. 📜 Setup your reasoning `schema.json` according to the [following example ↗️](test/schema/default.json)
-3. 🚀 Launch inference using `DeepSeek-R1`:
-```bash
-python3 -m bulk_chain.infer \
-    --src "<PATH-TO-YOUR-CSV-or-JSONL>" \
-    --schema "test/schema/default.json" \
-    --adapter "replicate_104.py:Replicate" \
-    %%m \
-    --model_name "deepseek-ai/deepseek-r1" \
-    --api_token "<REPLICATE-API-TOKEN>"
-```
-
 ## API
 
 Please take a look at the [**related Wiki page**](https://github.com/nicolay-r/bulk-chain/wiki)

{bulk_chain-0.25.2 → bulk_chain-1.0.0}/bulk_chain.egg-info/SOURCES.txt

@@ -3,8 +3,6 @@ README.md
 setup.py
 bulk_chain/__init__.py
 bulk_chain/api.py
-bulk_chain/demo.py
-bulk_chain/infer.py
 bulk_chain.egg-info/PKG-INFO
 bulk_chain.egg-info/SOURCES.txt
 bulk_chain.egg-info/dependency_links.txt
@@ -12,17 +10,14 @@ bulk_chain.egg-info/requires.txt
 bulk_chain.egg-info/top_level.txt
 bulk_chain/core/__init__.py
 bulk_chain/core/llm_base.py
-bulk_chain/core/service_args.py
 bulk_chain/core/service_batch.py
 bulk_chain/core/service_data.py
 bulk_chain/core/service_dict.py
 bulk_chain/core/service_json.py
-bulk_chain/core/service_llm.py
 bulk_chain/core/service_schema.py
 bulk_chain/core/utils.py
-bulk_chain/core/utils_logger.py
 test/test.py
 test/test_api.py
+test/test_api_streaming.py
 test/test_args_seeking.py
-test/test_cmdargs.py
 test/test_provider_batching.py

bulk_chain-1.0.0/bulk_chain.egg-info/requires.txt

@@ -0,0 +1 @@
+tqdm

{bulk_chain-0.25.2 → bulk_chain-1.0.0}/setup.py

@@ -15,7 +15,7 @@ def get_requirements(filenames):
 
 setup(
     name='bulk_chain',
-    version='0.25.2',
+    version='1.0.0',
     python_requires=">=3.6",
     description='A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, '
                 'ensuring reliable results for bulk input requests.',

{bulk_chain-0.25.2 → bulk_chain-1.0.0}/test/test_api.py

@@ -3,37 +3,28 @@ from os.path import join
 
 from bulk_chain.api import iter_content, CWD
 from bulk_chain.core.utils import dynamic_init
-from bulk_chain.infer import iter_content_cached
+from utils import current_dir, API_TOKEN
 
 
 class TestAPI(unittest.TestCase):
 
     llm = dynamic_init(class_dir=join(CWD, ".."),
                        class_filepath="providers/replicate_104.py",
-                       class_name="Replicate")(api_token="<API-KEY>",
+                       class_name="Replicate")(api_token=API_TOKEN,
                                                model_name="deepseek-ai/deepseek-r1")
 
-    def it_data(self, n):
+    @staticmethod
+    def it_data(n):
         for i in range(n):
             yield {"ind": i, "text": "X invent sanctions against Y", "entity": "X"}
 
-    def test_iter_cached(self):
-        data_it = iter_content_cached(input_dicts_it=self.it_data(20),
-                                      llm=self.llm,
-                                      schema="../schema/default.json",
-                                      # Cache-related extra parameters.
-                                      cache_target="out.sqlite:content",
-                                      id_column_name="ind")
-
-        for data in data_it:
-            print(data)
-
     def test_iter(self):
         data_it = iter_content(input_dicts_it=self.it_data(20),
                                llm=self.llm,
                                batch_size=1,
-                               return_batch=True,
-                               schema="../schema/default.json")
+                               handle_missed_value_func=lambda *_: None,
+                               return_mode="batch",
+                               schema=join(current_dir, "schema/default.json"))
 
         for data in data_it:
             print(data)