edsl 0.1.14__tar.gz → 0.1.40__tar.gz

edsl-0.1.40/PKG-INFO

@@ -0,0 +1,111 @@
+Metadata-Version: 2.1
+Name: edsl
+Version: 0.1.40
+Summary: Create and analyze LLM-based surveys
+Home-page: https://www.expectedparrot.com/
+License: MIT
+Keywords: LLM,social science,surveys,user research
+Author: Apostolos Filippas
+Author-email: apostolos@expectedparrot.com
+Requires-Python: >=3.9.1,<3.13
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Dist: aiohttp (>=3.9.1,<4.0.0)
+Requires-Dist: anthropic (>=0.23.1,<0.24.0)
+Requires-Dist: azure-ai-inference (>=1.0.0b3,<2.0.0)
+Requires-Dist: black[jupyter] (>=24.4.2,<25.0.0)
+Requires-Dist: boto3 (>=1.34.161,<2.0.0)
+Requires-Dist: google-generativeai (>=0.8.2,<0.9.0)
+Requires-Dist: groq (>=0.9.0,<0.10.0)
+Requires-Dist: jinja2 (>=3.1.2,<4.0.0)
+Requires-Dist: json-repair (>=0.28.4,<0.29.0)
+Requires-Dist: jupyter (>=1.0.0,<2.0.0)
+Requires-Dist: markdown2 (>=2.4.11,<3.0.0)
+Requires-Dist: matplotlib (>=3.8,<3.9)
+Requires-Dist: mistralai (>=1.0.2,<2.0.0)
+Requires-Dist: nest-asyncio (>=1.5.9,<2.0.0)
+Requires-Dist: numpy (>=1.22,<2.0)
+Requires-Dist: openai (>=1.4.0,<2.0.0)
+Requires-Dist: openpyxl (>=3.1.5,<4.0.0)
+Requires-Dist: pandas (>=2.1.4,<3.0.0)
+Requires-Dist: platformdirs (>=4.3.6,<5.0.0)
+Requires-Dist: pydot (>=2.0.0,<3.0.0)
+Requires-Dist: pygments (>=2.17.2,<3.0.0)
+Requires-Dist: pymupdf (>=1.24.4,<2.0.0)
+Requires-Dist: pypdf2 (>=3.0.1,<4.0.0)
+Requires-Dist: pyreadstat (>=1.2.7,<2.0.0)
+Requires-Dist: python-docx (>=1.1.0,<2.0.0)
+Requires-Dist: python-dotenv (>=1.0.0,<2.0.0)
+Requires-Dist: python-pptx (>=1.0.2,<2.0.0)
+Requires-Dist: restrictedpython (>=7.1,<8.0)
+Requires-Dist: rich (>=13.7.0,<14.0.0)
+Requires-Dist: setuptools (<72.0)
+Requires-Dist: simpleeval (>=0.9.13,<0.10.0)
+Requires-Dist: sqlalchemy (>=2.0.23,<3.0.0)
+Requires-Dist: tabulate (>=0.9.0,<0.10.0)
+Requires-Dist: tenacity (>=8.2.3,<9.0.0)
+Requires-Dist: urllib3 (>=1.25.4,<1.27)
+Project-URL: Documentation, https://docs.expectedparrot.com
+Description-Content-Type: text/markdown
+
+# Expected Parrot Domain-Specific Language 
+<p align="center">
+  <img src="https://github.com/expectedparrot/edsl/blob/main/static/logo.png?raw=true" alt="edsl.png" width="100"/>
+</p>
+
+The Expected Parrot Domain-Specific Language (EDSL) package lets you conduct computational social science and market research with AI. Use it to design surveys and experiments, simulate responses with large language models, and perform data labeling and other research tasks. Results are formatted as specified datasets and come with built-in methods for analyzing, visualizing, and sharing. 
+
+## 🔗 Links
+- [PyPI](https://pypi.org/project/edsl/)
+- [Documentation](https://docs.expectedparrot.com)
+- [Getting started](https://docs.expectedparrot.com/en/latest/starter_tutorial.html) 
+- [Discord](https://discord.com/invite/mxAYkjfy9m)
+- [Twitter](https://x.com/ExpectedParrot)
+- [LinkedIn](https://www.linkedin.com/company/expectedparrot/)
+- [Blog](https://blog.expectedparrot.com)
+
+## 🌎 Hello, World!
+A quick example:
+
+```python
+# Import a question type
+from edsl import QuestionMultipleChoice
+
+# Construct a question using the question type template
+q = QuestionMultipleChoice(
+    question_name="example_question",
+    question_text="How do you feel today?",
+    question_options=["Bad", "OK", "Good"]
+)
+
+# Run it with the default language model
+results = q.run()
+
+# Inspect the results 
+results.select("example_question")
+```
+
+Output:
+```python
+┏━━━━━━━━━━━━━━━━━━━━━━━━━┓
+┃ answer.example_question ┃
+┡━━━━━━━━━━━━━━━━━━━━━━━━━┩
+│ Good                    │
+└─────────────────────────┘
+```
+
+## 💻 Requirements
+* EDSL is compatible with Python 3.9 - 3.12.
+* API keys for large language models that you want to use, stored in a `.env` file.
+See instructions on [storing API keys](https://docs.expectedparrot.com/en/latest/api_keys.html) and [activating remote inference](https://docs.expectedparrot.com/en/latest/remote_inference.html).
+
+## 💡 Contributions, feature requests & bugs
+Interested in contributing? Want us to add a new feature? Found a bug for us to squash? 
+Please send us an email at [info@expectedparrot.com](mailto:info@expectedparrot.com) or message us at our [Discord channel](https://discord.com/invite/mxAYkjfy9m).
+

edsl-0.1.40/README.md

@@ -0,0 +1,54 @@
+# Expected Parrot Domain-Specific Language 
+<p align="center">
+  <img src="https://github.com/expectedparrot/edsl/blob/main/static/logo.png?raw=true" alt="edsl.png" width="100"/>
+</p>
+
+The Expected Parrot Domain-Specific Language (EDSL) package lets you conduct computational social science and market research with AI. Use it to design surveys and experiments, simulate responses with large language models, and perform data labeling and other research tasks. Results are formatted as specified datasets and come with built-in methods for analyzing, visualizing, and sharing. 
+
+## 🔗 Links
+- [PyPI](https://pypi.org/project/edsl/)
+- [Documentation](https://docs.expectedparrot.com)
+- [Getting started](https://docs.expectedparrot.com/en/latest/starter_tutorial.html) 
+- [Discord](https://discord.com/invite/mxAYkjfy9m)
+- [Twitter](https://x.com/ExpectedParrot)
+- [LinkedIn](https://www.linkedin.com/company/expectedparrot/)
+- [Blog](https://blog.expectedparrot.com)
+
+## 🌎 Hello, World!
+A quick example:
+
+```python
+# Import a question type
+from edsl import QuestionMultipleChoice
+
+# Construct a question using the question type template
+q = QuestionMultipleChoice(
+    question_name="example_question",
+    question_text="How do you feel today?",
+    question_options=["Bad", "OK", "Good"]
+)
+
+# Run it with the default language model
+results = q.run()
+
+# Inspect the results 
+results.select("example_question")
+```
+
+Output:
+```python
+┏━━━━━━━━━━━━━━━━━━━━━━━━━┓
+┃ answer.example_question ┃
+┡━━━━━━━━━━━━━━━━━━━━━━━━━┩
+│ Good                    │
+└─────────────────────────┘
+```
+
+## 💻 Requirements
+* EDSL is compatible with Python 3.9 - 3.12.
+* API keys for large language models that you want to use, stored in a `.env` file.
+See instructions on [storing API keys](https://docs.expectedparrot.com/en/latest/api_keys.html) and [activating remote inference](https://docs.expectedparrot.com/en/latest/remote_inference.html).
+
+## 💡 Contributions, feature requests & bugs
+Interested in contributing? Want us to add a new feature? Found a bug for us to squash? 
+Please send us an email at [info@expectedparrot.com](mailto:info@expectedparrot.com) or message us at our [Discord channel](https://discord.com/invite/mxAYkjfy9m).

edsl-0.1.40/edsl/Base.py

@@ -0,0 +1,413 @@
+"""Base class for all classes in the package. It provides rich printing and persistence of objects."""
+
+from abc import ABC, abstractmethod, ABCMeta
+import gzip
+import json
+from typing import Any, Optional, Union
+from uuid import UUID
+
+
+class PersistenceMixin:
+    """Mixin for saving and loading objects to and from files."""
+
+    def duplicate(self, add_edsl_version=False):
+        """Return a duplicate of the object."""
+        return self.from_dict(self.to_dict(add_edsl_version=False))
+
+    def push(
+        self,
+        description: Optional[str] = None,
+        alias: Optional[str] = None,
+        visibility: Optional[str] = "unlisted",
+        expected_parrot_url: Optional[str] = None,
+    ):
+        """Post the object to coop."""
+        from edsl.coop import Coop
+
+        c = Coop(url=expected_parrot_url)
+        return c.create(self, description, alias, visibility)
+
+    def to_yaml(self, add_edsl_version=False, filename: str = None) -> Union[str, None]:
+        import yaml
+
+        output = yaml.dump(self.to_dict(add_edsl_version=add_edsl_version))
+        if not filename:
+            return output
+
+        with open(filename, "w") as f:
+            f.write(output)
+
+    @classmethod
+    def from_yaml(cls, yaml_str: Optional[str] = None, filename: Optional[str] = None):
+        if yaml_str is None and filename is not None:
+            with open(filename, "r") as f:
+                yaml_str = f.read()
+                return cls.from_yaml(yaml_str=yaml_str)
+        elif yaml_str and filename is None:
+            import yaml
+
+            d = yaml.load(yaml_str, Loader=yaml.FullLoader)
+            return cls.from_dict(d)
+        else:
+            raise ValueError("Either yaml_str or filename must be provided.")
+
+    def create_download_link(self):
+        from tempfile import NamedTemporaryFile
+        from edsl.scenarios.FileStore import FileStore
+
+        with NamedTemporaryFile(suffix=".json.gz") as f:
+            self.save(f.name)
+            print(f.name)
+            fs = FileStore(path=f.name)
+        return fs.create_link()
+
+    @classmethod
+    def pull(
+        cls,
+        uuid: Optional[Union[str, UUID]] = None,
+        url: Optional[str] = None,
+        expected_parrot_url: Optional[str] = None,
+    ):
+        """Pull the object from coop."""
+        from edsl.coop import Coop
+        from edsl.coop.utils import ObjectRegistry
+
+        object_type = ObjectRegistry.get_object_type_by_edsl_class(cls)
+        coop = Coop(url=expected_parrot_url)
+        return coop.get(uuid, url, object_type)
+
+    @classmethod
+    def delete(cls, uuid: Optional[Union[str, UUID]] = None, url: Optional[str] = None):
+        """Delete the object from coop."""
+        from edsl.coop import Coop
+
+        coop = Coop()
+        return coop.delete(uuid, url)
+
+    @classmethod
+    def patch(
+        cls,
+        uuid: Optional[Union[str, UUID]] = None,
+        url: Optional[str] = None,
+        description: Optional[str] = None,
+        alias: Optional[str] = None,
+        value: Optional[Any] = None,
+        visibility: Optional[str] = None,
+    ):
+        """
+        Patch an uploaded objects attributes.
+        - `description` changes the description of the object on Coop
+        - `value` changes the value of the object on Coop. **has to be an EDSL object**
+        - `visibility` changes the visibility of the object on Coop
+        """
+        from edsl.coop import Coop
+
+        coop = Coop()
+        return coop.patch(uuid, url, description, alias, value, visibility)
+
+    @classmethod
+    def search(cls, query):
+        """Search for objects on coop."""
+        from edsl.coop import Coop
+
+        c = Coop()
+        return c.search(cls, query)
+
+    def store(self, d: dict, key_name: Optional[str] = None):
+        if key_name is None:
+            index = len(d)
+        else:
+            index = key_name
+        d[index] = self
+
+    def save(self, filename, compress=True):
+        """Save the object to a file as zippped JSON.
+
+        >>> obj.save("obj.json.gz")
+
+        """
+        if filename.endswith("json.gz"):
+            import warnings
+
+            filename = filename[:-8]
+        if filename.endswith("json"):
+            filename = filename[:-5]
+
+        if compress:
+            full_file_name = filename + ".json.gz"
+            with gzip.open(full_file_name, "wb") as f:
+                f.write(json.dumps(self.to_dict()).encode("utf-8"))
+        else:
+            full_file_name = filename + ".json"
+            with open(filename + ".json", "w") as f:
+                f.write(json.dumps(self.to_dict()))
+
+        print("Saved to", full_file_name)
+
+    @staticmethod
+    def open_compressed_file(filename):
+        with gzip.open(filename, "rb") as f:
+            file_contents = f.read()
+            file_contents_decoded = file_contents.decode("utf-8")
+            d = json.loads(file_contents_decoded)
+        return d
+
+    @staticmethod
+    def open_regular_file(filename):
+        with open(filename, "r") as f:
+            d = json.loads(f.read())
+        return d
+
+    @classmethod
+    def load(cls, filename):
+        """Load the object from a file.
+
+        >>> obj = cls.load("obj.json.gz")
+
+        """
+
+        if filename.endswith("json.gz"):
+            d = cls.open_compressed_file(filename)
+        elif filename.endswith("json"):
+            d = cls.open_regular_file(filename)
+        else:
+            try:
+                d = cls.open_compressed_file(filename + ".json.gz")
+            except:
+                d = cls.open_regular_file(filename + ".json")
+            # finally:
+            #    raise ValueError("File must be a json or json.gz file")
+
+        return cls.from_dict(d)
+
+
+class RegisterSubclassesMeta(ABCMeta):
+    """Metaclass for registering subclasses."""
+
+    _registry = {}
+
+    def __init__(cls, name, bases, nmspc):
+        """Register the class in the registry upon creation."""
+        super(RegisterSubclassesMeta, cls).__init__(name, bases, nmspc)
+        if cls.__name__ != "Base":
+            RegisterSubclassesMeta._registry[cls.__name__] = cls
+
+    @staticmethod
+    def get_registry(exclude_classes: Optional[list] = None):
+        """Return the registry of subclasses."""
+        if exclude_classes is None:
+            exclude_classes = []
+        return {
+            k: v
+            for k, v in dict(RegisterSubclassesMeta._registry).items()
+            if k not in exclude_classes
+        }
+
+
+class DiffMethodsMixin:
+    def __sub__(self, other):
+        """Return the difference between two objects."""
+        from edsl.BaseDiff import BaseDiff
+
+        return BaseDiff(self, other)
+
+
+def is_iterable(obj):
+    try:
+        iter(obj)
+    except TypeError:
+        return False
+    return True
+
+
+class RepresentationMixin:
+    def json(self):
+        return json.loads(json.dumps(self.to_dict(add_edsl_version=False)))
+
+    def to_dataset(self):
+        from edsl.results.Dataset import Dataset
+
+        return Dataset.from_edsl_object(self)
+
+    def view(self):
+        "Displays an interactive / perspective view of the object"
+        return self.to_dataset().view()
+
+    # def print(self, format="rich"):
+    #     return self.to_dataset().table()
+
+    def display_dict(self):
+        display_dict = {}
+        d = self.to_dict(add_edsl_version=False)
+        for key, value in d.items():
+            if isinstance(value, dict):
+                for k, v in value.items():
+                    display_dict[f"{key}:{k}"] = v
+            elif isinstance(value, list):
+                for i, v in enumerate(value):
+                    display_dict[f"{key}:{i}"] = v
+            else:
+                display_dict[key] = value
+        return display_dict
+
+    def print(self, format="rich"):
+        from rich.table import Table
+        from rich.console import Console
+
+        table = Table(title=self.__class__.__name__)
+        table.add_column("Key", style="bold")
+        table.add_column("Value", style="bold")
+
+        for key, value in self.display_dict().items():
+            table.add_row(key, str(value))
+
+        console = Console(record=True)
+        console.print(table)
+
+    def help(obj):
+        """
+        Extract all public instance methods and their docstrings from a class instance.
+
+        Args:
+            obj: The instance to inspect
+
+        Returns:
+            dict: A dictionary where keys are method names and values are their docstrings
+        """
+        import inspect
+
+        if inspect.isclass(obj):
+            raise TypeError("Please provide a class instance, not a class")
+
+        methods = {}
+
+        # Get all members of the instance
+        for name, member in inspect.getmembers(obj):
+            # Skip private and special methods (those starting with underscore)
+            if name.startswith("_"):
+                continue
+
+            # Check if it's specifically an instance method
+            if inspect.ismethod(member):
+                # Get the docstring (or empty string if none exists)
+                docstring = inspect.getdoc(member) or ""
+                methods[name] = docstring
+
+        from edsl.results.Dataset import Dataset
+
+        d = Dataset(
+            [
+                {"method": list(methods.keys())},
+                {"documentation": list(methods.values())},
+            ]
+        )
+        return d
+
+    def _repr_html_(self):
+        from edsl.results.TableDisplay import TableDisplay
+
+        if hasattr(self, "_summary"):
+            summary_dict = self._summary()
+            summary_line = "".join([f" {k}: {v};" for k, v in summary_dict.items()])
+            class_name = self.__class__.__name__
+            docs = getattr(self, "__documentation__", "")
+            return (
+                "<p>"
+                + f"<a href='{docs}'>{class_name}</a>"
+                + summary_line
+                + "</p>"
+                + self.table()._repr_html_()
+            )
+        else:
+            class_name = self.__class__.__name__
+            documenation = getattr(self, "__documentation__", "")
+            summary_line = "<p>" + f"<a href='{documenation}'>{class_name}</a>" + "</p>"
+            display_dict = self.display_dict()
+            return (
+                summary_line
+                + TableDisplay.from_dictionary_wide(display_dict)._repr_html_()
+            )
+
+    def __str__(self):
+        return self.__repr__()
+
+
+class HashingMixin:
+    def __hash__(self) -> int:
+        """Return a hash of the question."""
+        from edsl.utilities.utilities import dict_hash
+
+        return dict_hash(self.to_dict(add_edsl_version=False))
+
+    def __eq__(self, other):
+        """Return whether two objects are equal."""
+        return hash(self) == hash(other)
+
+
+class Base(
+    RepresentationMixin,
+    PersistenceMixin,
+    DiffMethodsMixin,
+    HashingMixin,
+    ABC,
+    metaclass=RegisterSubclassesMeta,
+):
+    """Base class for all classes in the package."""
+
+    def keys(self):
+        """Return the keys of the object."""
+        _keys = list(self.to_dict().keys())
+        if "edsl_version" in _keys:
+            _keys.remove("edsl_version")
+        if "edsl_class_name" in _keys:
+            _keys.remove("edsl_class_name")
+        return _keys
+
+    def values(self):
+        """Return the values of the object."""
+        data = self.to_dict()
+        keys = self.keys()
+        return {data[key] for key in keys}
+
+    @abstractmethod
+    def example():
+        """This method should be implemented by subclasses."""
+        raise NotImplementedError("This method is not implemented yet.")
+
+    @abstractmethod
+    def to_dict():
+        """This method should be implemented by subclasses."""
+        raise NotImplementedError("This method is not implemented yet.")
+
+    def to_json(self):
+        return json.dumps(self.to_dict())
+
+    def store(self, d: dict, key_name: Optional[str] = None):
+        if key_name is None:
+            index = len(d)
+        else:
+            index = key_name
+        d[index] = self
+
+    @abstractmethod
+    def from_dict():
+        """This method should be implemented by subclasses."""
+        raise NotImplementedError("This method is not implemented yet.")
+
+    @abstractmethod
+    def code():
+        """This method should be implemented by subclasses."""
+        raise NotImplementedError("This method is not implemented yet.")
+
+    def show_methods(self, show_docstrings=True):
+        """Show the methods of the object."""
+        public_methods_with_docstrings = [
+            (method, getattr(self, method).__doc__)
+            for method in dir(self)
+            if callable(getattr(self, method)) and not method.startswith("_")
+        ]
+        if show_docstrings:
+            for method, documentation in public_methods_with_docstrings:
+                print(f"{method}: {documentation}")
+        else:
+            return [x[0] for x in public_methods_with_docstrings]