bulk-chain 0.24.0__tar.gz

bulk_chain-0.24.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Nicolay Rusnachenko
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

bulk_chain-0.24.0/PKG-INFO

@@ -0,0 +1,92 @@
+Metadata-Version: 2.1
+Name: bulk_chain
+Version: 0.24.0
+Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
+Home-page: https://github.com/nicolay-r/bulk-chain
+Author: Nicolay Rusnachenko
+Author-email: rusnicolay@gmail.com
+License: MIT License
+Keywords: natural language processing,chain-of-thought,reasoning
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: tqdm
+
+# bulk-chain
+![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
+[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
+
+A lightweight, no-strings-attached **[Chain-of-Thought](https://arxiv.org/abs/2201.11903) framework** for your LLM, ensuring reliable results for bulk input requests stored in `CSV` / `JSONL` / `sqlite`.
+It allows applying series of prompts formed into `schema` (See [related section](#chain-of-thought-schema))
+
+### Features
+* ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
+* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
+* ✅ **Progress caching**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
+* ✅ **Support schemas descriptions** for Chain-of-Thought concept.
+
+# Installation
+
+```bash
+pip install git+https://github.com/nicolay-r/bulk-chain@master
+```
+
+## Chain-of-Thought Schema
+
+To declare Chain-of-Though (CoT) schema, this project exploits `JSON` format.
+This format adopts `name` field for declaring a name and `schema` is a list of CoT instructions for the Large Language Model.
+
+Each step represents a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
+All the variable names are expected to be mentioned in `{}`.
+
+Below, is an example on how to declare your own schema:
+
+```python
+{
+"name": "schema-name",
+"schema": [
+    {"prompt": "Given the question '{text}', let's think step-by-step.", 
+     "out": "steps"},
+    {"prompt": "For the question '{text}' the reasoining steps are '{steps}'. what would be an answer?", 
+     "out":  "answer"},
+]
+}
+```
+
+Another templates are available [here](/ext/schema/thor_cot_schema.json).
+
+# Usage
+
+Just **three** simple steps:
+
+1. Define your [CoT Schema](#chain-of-thought-schema), or fetch it as shown below:
+```bash
+!wget https://raw.githubusercontent.com/nicolay-r/bulk-chain/refs/heads/master/ext/schema/default.json
+```
+2. Fetch or write your own **model** or pick the one [preset here](/ext/):
+```bash
+!wget https://raw.githubusercontent.com/nicolay-r/bulk-chain/refs/heads/master/ext/flan_t5.py
+```
+
+3. Launch inference in (chat mode):
+```bash
+!python -m bulk_chain.infer \
+    --schema "default.json" \
+    --adapter "dynamic:flan_t5.py:FlanT5" \
+    %% \
+    --device "cpu" \
+    --temp 0.1
+```
+
+# Embed your LLM
+
+All you have to do is to implement `BaseLM` class, that includes:
+* `__init__` -- for initialization;
+* `ask(prompt)` -- infer your model with the given `prompt`.
+
+See examples with models [here](/ext).

bulk_chain-0.24.0/README.md

@@ -0,0 +1,73 @@
+# bulk-chain
+![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
+[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
+
+A lightweight, no-strings-attached **[Chain-of-Thought](https://arxiv.org/abs/2201.11903) framework** for your LLM, ensuring reliable results for bulk input requests stored in `CSV` / `JSONL` / `sqlite`.
+It allows applying series of prompts formed into `schema` (See [related section](#chain-of-thought-schema))
+
+### Features
+* ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
+* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
+* ✅ **Progress caching**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
+* ✅ **Support schemas descriptions** for Chain-of-Thought concept.
+
+# Installation
+
+```bash
+pip install git+https://github.com/nicolay-r/bulk-chain@master
+```
+
+## Chain-of-Thought Schema
+
+To declare Chain-of-Though (CoT) schema, this project exploits `JSON` format.
+This format adopts `name` field for declaring a name and `schema` is a list of CoT instructions for the Large Language Model.
+
+Each step represents a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
+All the variable names are expected to be mentioned in `{}`.
+
+Below, is an example on how to declare your own schema:
+
+```python
+{
+"name": "schema-name",
+"schema": [
+    {"prompt": "Given the question '{text}', let's think step-by-step.", 
+     "out": "steps"},
+    {"prompt": "For the question '{text}' the reasoining steps are '{steps}'. what would be an answer?", 
+     "out":  "answer"},
+]
+}
+```
+
+Another templates are available [here](/ext/schema/thor_cot_schema.json).
+
+# Usage
+
+Just **three** simple steps:
+
+1. Define your [CoT Schema](#chain-of-thought-schema), or fetch it as shown below:
+```bash
+!wget https://raw.githubusercontent.com/nicolay-r/bulk-chain/refs/heads/master/ext/schema/default.json
+```
+2. Fetch or write your own **model** or pick the one [preset here](/ext/):
+```bash
+!wget https://raw.githubusercontent.com/nicolay-r/bulk-chain/refs/heads/master/ext/flan_t5.py
+```
+
+3. Launch inference in (chat mode):
+```bash
+!python -m bulk_chain.infer \
+    --schema "default.json" \
+    --adapter "dynamic:flan_t5.py:FlanT5" \
+    %% \
+    --device "cpu" \
+    --temp 0.1
+```
+
+# Embed your LLM
+
+All you have to do is to implement `BaseLM` class, that includes:
+* `__init__` -- for initialization;
+* `ask(prompt)` -- infer your model with the given `prompt`.
+
+See examples with models [here](/ext).

bulk_chain-0.24.0/bulk_chain/core/llm_base.py

@@ -0,0 +1,13 @@
+from bulk_chain.core.utils import format_model_name
+
+
+class BaseLM(object):
+
+    def __init__(self, name):
+        self.__name = name
+
+    def ask(self, prompt):
+        raise NotImplemented()
+
+    def name(self):
+        return format_model_name(self.__name)

bulk_chain-0.24.0/bulk_chain/core/provider_sqlite.py

@@ -0,0 +1,79 @@
+import sqlite3
+
+
+class SQLiteProvider(object):
+
+    @staticmethod
+    def __create_table(table_name, columns, id_column_name,
+                       id_column_type, sqlite3_column_types, cur):
+
+        # Provide the ID column.
+        sqlite3_column_types = [id_column_type] + sqlite3_column_types
+
+        # Compose the whole columns list.
+        content = ", ".join([" ".join(item) for item in zip(columns, sqlite3_column_types)])
+        cur.execute(f"CREATE TABLE IF NOT EXISTS {table_name}({content})")
+        cur.execute(f"CREATE INDEX IF NOT EXISTS i_id ON {table_name}({id_column_name})")
+
+    @staticmethod
+    def write_auto(data_it, target, data2col_func, table_name, id_column_name="id",
+                   id_column_type="INTEGER"):
+        """ NOTE: data_it is an iterator of dictionaries.
+            This implementation automatically creates the table and
+        """
+        with sqlite3.connect(target) as con:
+            cur = con.cursor()
+
+            columns = None
+            for data in data_it:
+                assert(isinstance(data, dict))
+
+                # Extracting columns from data.
+                row_columns = list(data.keys())
+                assert(id_column_name in row_columns)
+
+                # Optionally create table.
+                if columns is None:
+
+                    # Setup list of columns.
+                    columns = row_columns
+                    # Place ID column first.
+                    columns.insert(0, columns.pop(columns.index(id_column_name)))
+
+                    SQLiteProvider.__create_table(
+                        columns=columns, table_name=table_name, cur=cur,
+                        id_column_name=id_column_name, id_column_type=id_column_type,
+                        sqlite3_column_types=["TEXT"] * len(columns))
+
+                # Check that each rows satisfies criteria of the first row.
+                [Exception(f"{column} is expected to be in row!") for column in row_columns if column not in columns]
+
+                uid = data[id_column_name]
+                r = cur.execute(f"SELECT EXISTS(SELECT 1 FROM {table_name} WHERE {id_column_name}='{uid}');")
+                ans = r.fetchone()[0]
+                if ans == 1:
+                    continue
+
+                params = ", ".join(tuple(['?'] * (len(columns))))
+                row_columns_str = ", ".join(row_columns)
+                cur.execute(f"INSERT INTO {table_name}({row_columns_str}) VALUES ({params})",
+                            [data2col_func(c, data) for c in row_columns])
+                con.commit()
+
+            cur.close()
+
+    @staticmethod
+    def read(target, column_names=None, table="content"):
+        with sqlite3.connect(target) as conn:
+            cursor = conn.cursor()
+            cols = "*" if column_names is None else ",".join(column_names)
+            cursor.execute(f"SELECT {cols} FROM {table}")
+            for row in cursor:
+                yield row
+                
+    @staticmethod
+    def get_columns(target, table="content"):
+        with sqlite3.connect(target) as conn:
+            cursor = conn.cursor()
+            cursor.execute(f"PRAGMA table_info({table})")
+            return [row[1] for row in cursor.fetchall()]

bulk_chain-0.24.0/bulk_chain/core/service_args.py

@@ -0,0 +1,48 @@
+class CmdArgsService:
+
+    @staticmethod
+    def autocast(v):
+        for t in [int, float, str]:
+            try:
+                return t(v)
+            except:
+                pass
+
+    @staticmethod
+    def iter_arguments(lst):
+
+        def __release():
+            return key, buf if len(buf) > 1 else buf[0]
+
+        key = None
+        buf = []
+        for a in lst:
+            if a.startswith('--'):
+                # release
+                if key is not None:
+                    yield __release()
+                # set new key and empty buf
+                key = a[2:]
+                buf = []
+            else:
+                # append argument into buffer.
+                buf.append(a)
+
+        # Sharing the remaining params.
+        if len(buf) > 0:
+            yield __release()
+
+    @staticmethod
+    def partition_list(lst, sep):
+        """Slices a list in two, cutting on index matching "sep"
+        """
+        if sep in lst:
+            idx = lst.index(sep)
+            return (lst[:idx], lst[idx+1:])
+        else:
+            return (lst[:], None)
+
+    @staticmethod
+    def args_to_dict(args):
+        return {k: CmdArgsService.autocast(v) if not isinstance(v, list) else v
+                for k, v in CmdArgsService.iter_arguments(args)} if args is not None else {}

bulk_chain-0.24.0/bulk_chain/core/service_csv.py

@@ -0,0 +1,57 @@
+import csv
+import logging
+
+logger = logging.getLogger(__name__)
+logging.basicConfig(level=logging.INFO)
+
+
+class CsvService:
+
+    @staticmethod
+    def write(target, lines_it):
+        f = open(target, "w")
+        logger.info(f"Saving: {target}")
+        w = csv.writer(f, delimiter="\t", quotechar='"', quoting=csv.QUOTE_MINIMAL)
+        for content in lines_it:
+            w.writerow(content)
+
+    @staticmethod
+    def write_handled(target, data_it, data2col_func, header):
+
+        def __it():
+            yield header
+            for data in data_it:
+                content = data2col_func(data)
+                assert(len(content) == len(header))
+                yield content
+
+        CsvService.write(target, lines_it=__it())
+
+    @staticmethod
+    def read(target, skip_header=False, cols=None, as_dict=False, row_id_key=None, **csv_kwargs):
+        assert (isinstance(row_id_key, str) or row_id_key is None)
+        assert (isinstance(cols, list) or cols is None)
+
+        header = None
+        with open(target, newline='\n') as f:
+            for row_id, row in enumerate(csv.reader(f, **csv_kwargs)):
+                if skip_header and row_id == 0:
+                    header = ([row_id_key] if row_id_key is not None else []) + row
+                    continue
+
+                # Determine the content we wish to return.
+                if cols is None:
+                    content = row
+                else:
+                    row_d = {header[col_ind]: value for col_ind, value in enumerate(row)}
+                    content = [row_d[col_name] for col_name in cols]
+
+                content = ([row_id-1] if row_id_key is not None else []) + content
+
+                # Optionally attach row_id to the content.
+                if as_dict:
+                    assert (header is not None)
+                    assert (len(content) == len(header))
+                    yield {k: v for k, v in zip(header, content)}
+                else:
+                    yield content

bulk_chain-0.24.0/bulk_chain/core/service_data.py

@@ -0,0 +1,22 @@
+from bulk_chain.core.utils import iter_params
+
+
+class DataService(object):
+
+    @staticmethod
+    def compose_prompt_text(prompt, data_dict, field_names):
+        assert(isinstance(data_dict, dict))
+        fmt_d = {col_name: data_dict[col_name] for col_name in field_names}
+
+        # Guarantee that items has correct type.
+        for k, v in fmt_d.items():
+            if not isinstance(v, str):
+                Exception("'{k}' parameter is expected to be string, but received '{v}'")
+
+        return prompt.format(**fmt_d)
+
+    @staticmethod
+    def get_prompt_text(prompt, data_dict, parse_fields_func=iter_params):
+        field_names = list(parse_fields_func(prompt))
+        return DataService.compose_prompt_text(
+            prompt=prompt, data_dict=data_dict, field_names=field_names)

bulk_chain-0.24.0/bulk_chain/core/service_json.py

@@ -0,0 +1,26 @@
+import json
+
+
+class JsonService(object):
+
+    @staticmethod
+    def read_data(src):
+        assert (isinstance(src, str))
+        with open(src, "r") as f:
+            return json.load(f)
+
+    @staticmethod
+    def read_lines(src, row_id_key=None):
+        assert (isinstance(src, str))
+        with open(src, "r") as f:
+            for line_ind, line in enumerate(f.readlines()):
+                content = json.loads(line)
+                if row_id_key is not None:
+                    content[row_id_key] = line_ind
+                yield content
+
+    @staticmethod
+    def write_lines(target, data_it):
+        with open(target, "w") as f:
+            for item in data_it:
+                f.write(f"{json.dumps(item, ensure_ascii=False)}\n")

bulk_chain-0.24.0/bulk_chain/core/service_llm.py

@@ -0,0 +1,82 @@
+import logging
+
+from bulk_chain.core.llm_base import BaseLM
+from bulk_chain.core.service_data import DataService
+from bulk_chain.core.utils import iter_params
+
+logger = logging.getLogger(__name__)
+logging.basicConfig(level=logging.INFO)
+
+
+def pad_str(text, pad):
+    return text.rjust(len(text) + pad, ' ')
+
+
+def text_wrap(content, width, handle_line=lambda l: l):
+    lines = []
+    for text in content.split('\n'):
+        for i in range(0, len(text), width):
+            line = handle_line(text[i:i + width])
+            lines.append(line)
+    return '\n'.join(lines)
+
+
+def nice_output(text, width, pad=4, remove_new_line=False):
+    short_text = text.replace("\n", "") if remove_new_line else text
+    return text_wrap(content=short_text, width=width, handle_line=lambda line: pad_str(line, pad=pad))
+
+
+def chat_with_lm(lm, chain=None, model_name=None):
+    assert(isinstance(lm, BaseLM))
+    assert(isinstance(chain, list))
+    assert(isinstance(model_name, str) or model_name is None)
+
+    do_exit = False
+    model_name = model_name if model_name is not None else "agent"
+
+    while not do_exit:
+
+        logger.info("----------------")
+
+        # Launching the CoT engine loop.
+        data_dict = {}
+        for prompt_args in chain:
+
+            # Processing the prompt.
+            prompt = prompt_args["prompt"]
+
+            # Filling necessary parameters.
+            field_names = list(iter_params(prompt))
+            for ind, f_name in enumerate(field_names):
+
+                if f_name in data_dict:
+                    continue
+
+                user_input = input(f"Enter your prompt for `{f_name}` ({ind+1}/{len(field_names)}) "
+                                   f"(or 'exit' to quit): ")
+
+                if user_input.lower() == 'exit':
+                    do_exit = True
+                    break
+
+                data_dict[f_name] = user_input
+
+            if do_exit:
+                break
+
+            # Finally asking LLM.
+            DataService.compose_prompt_text(prompt=prompt, data_dict=data_dict, field_names=field_names)
+            actual_prompt = DataService.get_prompt_text(prompt=prompt, data_dict=data_dict)
+
+            # Returning meta information, passed to LLM.
+            pad = 4
+            logger.info(pad_str(f"{model_name} (ask) ->", pad=pad))
+            logger.info(nice_output(actual_prompt, pad=pad*2, remove_new_line=True, width=80))
+
+            # Response.
+            response = lm.ask(actual_prompt)
+            logger.info(pad_str(f"{model_name} (resp)->", pad=pad))
+            logger.info(nice_output(response, pad=pad*2, remove_new_line=False, width=80))
+
+            # Collecting the answer for the next turn.
+            data_dict[prompt_args["out"]] = response

bulk_chain-0.24.0/bulk_chain/core/service_schema.py

@@ -0,0 +1,34 @@
+class SchemaService(object):
+
+    def __init__(self, json_data):
+        self.src = json_data
+        self.name = self.src["name"]
+        self.r2p, self.p2r, self.cot_args, self.chain = SchemaService.__init_schema(prompts=json_data["schema"])
+
+    @classmethod
+    def from_prompt(cls, prompt):
+        prompt_schema = {"name": "prompt", "schema": [{"prompt": prompt, "out": "response", "in": "prompt"}]}
+        return cls(prompt_schema)
+
+    @staticmethod
+    def __init_schema(prompts):
+
+        schema_args = {}
+        schema_r2p = {}
+        schema_p2r = {}
+        chain = []
+
+        for prompt in prompts:
+            r_col_name = prompt["out"]
+            p_col_name = r_col_name + "_prompt" if "in" not in prompt else prompt["in"]
+
+            assert r_col_name not in schema_r2p, f"`{r_col_name}` has been already declared!"
+            assert p_col_name not in schema_p2r, f"`{p_col_name}` has been already declared!"
+
+            schema_r2p[r_col_name] = p_col_name
+            schema_p2r[p_col_name] = r_col_name
+            schema_args[p_col_name] = prompt
+            schema_args[r_col_name] = None
+            chain.append(prompt)
+
+        return schema_r2p, schema_p2r, schema_args, chain

bulk_chain-0.24.0/bulk_chain/core/utils.py

@@ -0,0 +1,101 @@
+import importlib
+import logging
+import sys
+from collections import Counter
+
+logger = logging.getLogger(__name__)
+logging.basicConfig(level=logging.INFO)
+
+
+def find_by_prefix(d, key):
+    """
+        d: dict (str, val).
+    """
+    assert(isinstance(d, dict))
+    assert(isinstance(key, str))
+
+    # We first check the full match.
+    for k, value in d.items():
+        if k == key:
+            return value
+
+    # If we can't establish full match, then we seek by prefix.
+    matches = []
+    for k, value in d.items():
+        if key.startswith(k):
+            matches.append(k)
+
+    if len(matches) > 1:
+        raise Exception(f"There are multiple entries that are related to `{key}`: {matches}")
+    if len(matches) == 0:
+        raise Exception(f"No entries were found for {key}!")
+
+    return d[matches[0]]
+
+
+def iter_params(text):
+    assert(isinstance(text, str))
+    beg = 0
+    while beg < len(text):
+        try:
+            pb = text.index('{', beg)
+        except ValueError:
+            break
+        pe = text.index('}', beg+1)
+        # Yield argument.
+        yield text[pb+1:pe]
+        beg = pe+1
+
+
+def format_model_name(name):
+    return name.replace("/", "_")
+
+
+def parse_filepath(filepath, default_filepath=None, default_ext=None):
+    """ This is an auxiliary function for handling sources and targets from cmd string.
+    """
+    if filepath is None:
+        return default_filepath, default_ext, None
+    info = filepath.split(":")
+    filepath = info[0]
+    meta = info[1] if len(info) > 1 else None
+    ext = filepath.split('.')[-1] if default_ext is None else default_ext
+    return filepath, ext, meta
+
+
+def handle_table_name(name):
+    return name.\
+        replace('-', '_').\
+        replace('.', "_")
+
+
+def auto_import(name, is_class=False):
+    """ Import from the external python packages.
+    """
+    def __get_module(comps_list):
+        return importlib.import_module(".".join(comps_list))
+
+    components = name.split('.')
+    m = getattr(__get_module(components[:-1]), components[-1])
+
+    return m() if is_class else m
+
+
+def dynamic_init(class_dir, class_filepath, class_name=None):
+    sys.path.append(class_dir)
+    class_path_list = class_filepath.split('/')
+    class_path_list[-1] = '.'.join(class_path_list[-1].split('.')[:-1])
+    class_name = class_path_list[-1].title() if class_name is None else class_name
+    class_path = ".".join(class_path_list + [class_name])
+    logger.info(f"Dynamic loading for the file and class `{class_path}`")
+    cls = auto_import(class_path, is_class=False)
+    return cls
+
+
+def optional_limit_iter(it_data, limit=None):
+    counter = Counter()
+    for data in it_data:
+        counter["returned"] += 1
+        if limit is not None and counter["returned"] > limit:
+            break
+        yield data

bulk_chain-0.24.0/bulk_chain/infer.py

@@ -0,0 +1,170 @@
+import argparse
+import logging
+import os
+import sys
+
+from tqdm import tqdm
+
+from os.path import join, basename
+
+from bulk_chain.core.llm_base import BaseLM
+from bulk_chain.core.provider_sqlite import SQLiteProvider
+from bulk_chain.core.service_args import CmdArgsService
+from bulk_chain.core.service_csv import CsvService
+from bulk_chain.core.service_data import DataService
+from bulk_chain.core.service_json import JsonService
+from bulk_chain.core.service_llm import chat_with_lm
+from bulk_chain.core.service_schema import SchemaService
+from bulk_chain.core.utils import dynamic_init, find_by_prefix, handle_table_name, optional_limit_iter, parse_filepath
+
+logger = logging.getLogger(__name__)
+logging.basicConfig(level=logging.INFO)
+
+
+CWD = os.getcwd()
+
+
+def init_llm(**model_kwargs):
+    """ This method perform dynamic initialization of LLM from third-party resource.
+    """
+
+    # List of the Supported models and their API wrappers.
+    models_preset = {
+        "dynamic": lambda: dynamic_init(class_dir=CWD, class_filepath=llm_model_name,
+                                        class_name=llm_model_params)(**model_kwargs)
+    }
+
+    # Initialize LLM model.
+    params = args.adapter.split(':')
+    llm_model_type = params[0]
+    llm_model_name = params[1] if len(params) > 1 else params[-1]
+    llm_model_params = ':'.join(params[2:]) if len(params) > 2 else None
+    llm = find_by_prefix(d=models_preset, key=llm_model_type)()
+
+    return llm, llm_model_name
+
+
+def init_schema(json_filepath):
+    return SchemaService(json_data=JsonService.read_data(json_filepath))
+
+
+def iter_content(input_dicts_iter, llm, schema, cache_target, cache_table):
+    """ This method represent Python API aimed at application of `llm` towards
+        iterator of input_dicts via cache_target that refers to the SQLite using
+        the given `schema`
+    """
+    assert (isinstance(llm, BaseLM))
+    assert (isinstance(schema, SchemaService))
+    assert (isinstance(cache_target, str))
+    assert (isinstance(cache_table, str))
+
+    infer_modes = {
+        "default": lambda prompt: llm.ask(prompt[:args.limit_prompt] if args.limit_prompt is not None else prompt)
+    }
+
+    def optional_update_data_records(c, data):
+        assert (isinstance(c, str))
+
+        if c in schema.p2r:
+            data[c] = DataService.get_prompt_text(prompt=data[c]["prompt"], data_dict=data)
+        if c in schema.r2p:
+            p_column = schema.r2p[c]
+            # This instruction takes a lot of time in a non-batching mode.
+            data[c] = infer_modes["default"](data[p_column])
+
+        return data[c]
+
+    cache_providers = {
+        "sqlite": lambda filepath, table_name, data_it: SQLiteProvider.write_auto(
+            data_it=data_it, target=filepath,
+            data2col_func=optional_update_data_records,
+            table_name=handle_table_name(table_name if table_name is not None else "contents"),
+            id_column_name="uid")
+    }
+
+    # We optionally wrap into limiter.
+    queries_it = optional_limit_iter(
+        it_data=map(lambda data: data.update(schema.cot_args) or data, input_dicts_iter),
+        limit=args.limit)
+
+    # Provide data caching.
+    cache_providers["sqlite"](cache_target, table_name=tgt_meta, data_it=tqdm(queries_it, desc="Iter content"))
+
+    return SQLiteProvider.read(cache_target, table=cache_table)
+
+
+if __name__ == '__main__':
+
+    parser = argparse.ArgumentParser(description="Infer Instruct LLM inference based on CoT schema")
+    parser.add_argument('--adapter', dest='adapter', type=str, default=None)
+    parser.add_argument('--src', dest='src', type=str, default=None)
+    parser.add_argument('--schema', dest='schema', type=str, default=None,
+                        help="Path to the JSON file that describes schema")
+    parser.add_argument('--csv-sep', dest='csv_sep', type=str, default='\t')
+    parser.add_argument('--csv-escape-char', dest='csv_escape_char', type=str, default=None)
+    parser.add_argument('--to', dest='to', type=str, default=None, choices=["csv", "sqlite"])
+    parser.add_argument('--output', dest='output', type=str, default=None)
+    parser.add_argument('--limit', dest='limit', type=int, default=None,
+                        help="Limit amount of source texts for prompting.")
+    parser.add_argument('--limit-prompt', dest="limit_prompt", type=int, default=None,
+                        help="Optional trimming prompt by the specified amount of characters.")
+
+    native_args, model_args = CmdArgsService.partition_list(lst=sys.argv, sep="%%")
+
+    args = parser.parse_args(args=native_args[1:])
+
+    # Initialize Large Language Model.
+    llm, llm_model_name = init_llm(**CmdArgsService.args_to_dict(model_args))
+
+    # Setup schema.
+    schema = init_schema(args.schema)
+    if schema is not None:
+        logger.info(f"Using schema: {schema.name}")
+
+    input_providers = {
+        None: lambda _: chat_with_lm(llm, chain=schema.chain, model_name=llm_model_name),
+        "csv": lambda filepath: CsvService.read(target=filepath, row_id_key="uid", delimiter=args.csv_sep,
+                                                as_dict=True, skip_header=True, escapechar=args.csv_escape_char),
+        "jsonl": lambda filepath: JsonService.read_lines(src=filepath, row_id_key="uid")
+    }
+
+    output_providers = {
+        "csv": lambda filepath, data_it, header:
+        CsvService.write_handled(target=filepath, data_it=data_it, header=header, data2col_func=lambda v: list(v)),
+        "jsonl": lambda filepath, data_it, header:
+        JsonService.write_lines(target=filepath,
+                                data_it=map(lambda item: {key:item[i] for i, key in enumerate(header)}, data_it))
+    }
+
+    # Setup output.
+    args.output = args.output.format(model=llm.name()) if args.output is not None else args.output
+    tgt_filepath, tgt_ext, tgt_meta = parse_filepath(args.output, default_ext=args.to)
+
+    # Input extension type defines the provider.
+    src_filepath, src_ext, src_meta = parse_filepath(args.src)
+
+    # Check whether we are in chat mode.
+    if src_ext is None:
+        input_providers[src_ext](None)
+        exit(0)
+
+    # Setup cache target as well as the related table.
+    cache_target = "".join(["_".join([join(CWD, basename(src_filepath)), llm.name(), schema.name]), f".sqlite"]) \
+        if tgt_filepath is None else tgt_filepath
+    cache_table = handle_table_name(tgt_meta if tgt_meta is not None else "contents")
+
+    data_it = iter_content(input_dicts_iter=input_providers[src_ext](src_filepath),
+                           schema=schema,
+                           llm=llm,
+                           cache_target=cache_target,
+                           cache_table=cache_table)
+
+    # Setup output target
+    tgt_ext = src_ext if tgt_ext is None else tgt_ext
+    output_target = "".join(["_".join([join(CWD, basename(src_filepath)), llm.name(), schema.name]), f".{tgt_ext}"]) \
+        if tgt_filepath is None else tgt_filepath
+
+    # Perform output writing process.
+    output_providers[tgt_ext](filepath=output_target,
+                              data_it=data_it,
+                              header=SQLiteProvider.get_columns(target=cache_target, table=cache_table))

bulk_chain-0.24.0/bulk_chain.egg-info/PKG-INFO

@@ -0,0 +1,92 @@
+Metadata-Version: 2.1
+Name: bulk_chain
+Version: 0.24.0
+Summary: A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.
+Home-page: https://github.com/nicolay-r/bulk-chain
+Author: Nicolay Rusnachenko
+Author-email: rusnicolay@gmail.com
+License: MIT License
+Keywords: natural language processing,chain-of-thought,reasoning
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: tqdm
+
+# bulk-chain
+![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)
+[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb)
+
+A lightweight, no-strings-attached **[Chain-of-Thought](https://arxiv.org/abs/2201.11903) framework** for your LLM, ensuring reliable results for bulk input requests stored in `CSV` / `JSONL` / `sqlite`.
+It allows applying series of prompts formed into `schema` (See [related section](#chain-of-thought-schema))
+
+### Features
+* ✅ **No-strings**: you're free to LLM dependencies and flexible `venv` customization.
+* ✅ **Provides iterator over infinite amount of input contexts** served in `CSV`/`JSONL`.
+* ✅ **Progress caching**: withstanding exception during LLM calls by using `sqlite3` engine for caching LLM answers;
+* ✅ **Support schemas descriptions** for Chain-of-Thought concept.
+
+# Installation
+
+```bash
+pip install git+https://github.com/nicolay-r/bulk-chain@master
+```
+
+## Chain-of-Thought Schema
+
+To declare Chain-of-Though (CoT) schema, this project exploits `JSON` format.
+This format adopts `name` field for declaring a name and `schema` is a list of CoT instructions for the Large Language Model.
+
+Each step represents a dictionary with `prompt` and `out` keys that corresponds to the input prompt and output variable name respectively.
+All the variable names are expected to be mentioned in `{}`.
+
+Below, is an example on how to declare your own schema:
+
+```python
+{
+"name": "schema-name",
+"schema": [
+    {"prompt": "Given the question '{text}', let's think step-by-step.", 
+     "out": "steps"},
+    {"prompt": "For the question '{text}' the reasoining steps are '{steps}'. what would be an answer?", 
+     "out":  "answer"},
+]
+}
+```
+
+Another templates are available [here](/ext/schema/thor_cot_schema.json).
+
+# Usage
+
+Just **three** simple steps:
+
+1. Define your [CoT Schema](#chain-of-thought-schema), or fetch it as shown below:
+```bash
+!wget https://raw.githubusercontent.com/nicolay-r/bulk-chain/refs/heads/master/ext/schema/default.json
+```
+2. Fetch or write your own **model** or pick the one [preset here](/ext/):
+```bash
+!wget https://raw.githubusercontent.com/nicolay-r/bulk-chain/refs/heads/master/ext/flan_t5.py
+```
+
+3. Launch inference in (chat mode):
+```bash
+!python -m bulk_chain.infer \
+    --schema "default.json" \
+    --adapter "dynamic:flan_t5.py:FlanT5" \
+    %% \
+    --device "cpu" \
+    --temp 0.1
+```
+
+# Embed your LLM
+
+All you have to do is to implement `BaseLM` class, that includes:
+* `__init__` -- for initialization;
+* `ask(prompt)` -- infer your model with the given `prompt`.
+
+See examples with models [here](/ext).

bulk_chain-0.24.0/bulk_chain.egg-info/SOURCES.txt

@@ -0,0 +1,22 @@
+LICENSE
+README.md
+setup.py
+bulk_chain/__init__.py
+bulk_chain/infer.py
+bulk_chain.egg-info/PKG-INFO
+bulk_chain.egg-info/SOURCES.txt
+bulk_chain.egg-info/dependency_links.txt
+bulk_chain.egg-info/requires.txt
+bulk_chain.egg-info/top_level.txt
+bulk_chain/core/__init__.py
+bulk_chain/core/llm_base.py
+bulk_chain/core/provider_sqlite.py
+bulk_chain/core/service_args.py
+bulk_chain/core/service_csv.py
+bulk_chain/core/service_data.py
+bulk_chain/core/service_json.py
+bulk_chain/core/service_llm.py
+bulk_chain/core/service_schema.py
+bulk_chain/core/utils.py
+test/test_args_seeking.py
+test/test_cmdargs.py

bulk_chain-0.24.0/bulk_chain.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

bulk_chain-0.24.0/bulk_chain.egg-info/requires.txt

@@ -0,0 +1 @@
+tqdm

bulk_chain-0.24.0/bulk_chain.egg-info/top_level.txt

@@ -0,0 +1 @@
+bulk_chain

bulk_chain-0.24.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

bulk_chain-0.24.0/setup.py

@@ -0,0 +1,40 @@
+from setuptools import (
+    setup,
+    find_packages,
+)
+
+
+def get_requirements(filenames):
+    r_total = []
+    for filename in filenames:
+        with open(filename) as f:
+            r_local = f.read().splitlines()
+            r_total.extend(r_local)
+    return r_total
+
+
+setup(
+    name='bulk_chain',
+    version='0.24.0',
+    python_requires=">=3.6",
+    description='A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, '
+                'ensuring reliable results for bulk input requests.',
+    long_description=open('README.md').read(),
+    long_description_content_type='text/markdown',
+    url='https://github.com/nicolay-r/bulk-chain',
+    author='Nicolay Rusnachenko',
+    author_email='rusnicolay@gmail.com',
+    license='MIT License',
+    classifiers=[
+        'Programming Language :: Python',
+        'Programming Language :: Python :: 3.9',
+        'Topic :: Software Development :: Libraries :: Python Modules',
+        'Topic :: Scientific/Engineering :: Information Analysis',
+        'Topic :: Text Processing :: Linguistic',
+    ],
+    keywords='natural language processing, '
+             'chain-of-thought, '
+             'reasoning',
+    packages=find_packages(),
+    install_requires=get_requirements(['dependencies.txt'])
+)

bulk_chain-0.24.0/test/test_args_seeking.py

@@ -0,0 +1,26 @@
+import unittest
+
+from bulk_chain.core.utils import iter_params
+
+
+class TestArgumentsSeeking(unittest.TestCase):
+
+    def test(self):
+        params = list(iter_params("X is a {x} and p is {text} and for {k}"))
+
+        line = ",".join(["{{{x}}}".format(x=x) for x in params])
+        print(line)
+        d_params = {}
+        for param in params:
+            d_params[param] = 2
+        print(d_params)
+
+        z = line.format(**d_params)
+        print(z)
+
+        b = list(iter_params("X"))
+        print(b)
+
+
+if __name__ == '__main__':
+    unittest.main()

bulk_chain-0.24.0/test/test_cmdargs.py

@@ -0,0 +1,9 @@
+import sys
+from bulk_chain.core.service_args import CmdArgsService
+
+
+print(sys.argv)
+string, args = CmdArgsService.partition_list(sys.argv[1:], "%%")
+d = CmdArgsService.args_to_dict(args)
+
+print(d)