clear-skies-doc-builder 2.0.0__tar.gz

clear_skies_doc_builder-2.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 clearskies-py
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

clear_skies_doc_builder-2.0.0/PKG-INFO

@@ -0,0 +1,40 @@
+Metadata-Version: 2.3
+Name: clear-skies-doc-builder
+Version: 2.0.0
+Summary: The docbuilder for all 'official' clearskies plugins (as well as the main clearskies docs)
+License: MIT
+Author: Conor Mancone
+Author-email: cmancone@gmail.com
+Requires-Python: >=3.11,<4.0
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Dist: clear-skies (>=2.0.0,<3.0.0)
+Project-URL: Repository, https://github.com/clearskies-py/docs
+Description-Content-Type: text/markdown
+
+# docs
+
+The documentation builder for clearskies and related plugins
+
+## Overview
+
+Each "official" clearskies module (including the core clearskies library itself) has the documentation primarily written in the codebase as docblocks.  The documentation site is then built by extracting these docblocks and stitching them together.  To be clear, this isn't about the low-level "API" documentation that describes every single class/method in the framework.  Rather, this is about the primary documentation site itself (clearskies.info) which is focused on high-level use cases and examples of the primary configuration options.  As a result, it's not a simple matter of just iterating over the classes/methods and building documentation.  To build a coherent documentation site, each plugin has a configuration file that basically outlines the final "structure" or organization of the resulting documentation, as well as the name of a builder class that will combine that configuration information with the codebase itself to create the actual docs.
+
+The docs themselves (in the source code) are all written with markdown.  This documentation builder then takes that markdwon and adds the necessary headers/etc so to make them valid files for [Jekyll](https://jekyllrb.com/), the builder for the current documentation site.  The site itself is hosted in S3, so building an actual documentation site means:
+
+ 1. Properly documenting everything inside of the source code via markdown.
+ 2. Creating a config file (`docs/python/config.json`) to map source code docs to Jekyll files.
+ 3. Creating a skeleton of a Jekyll site in the `doc/jekyll` folder of the plugin.
+ 4. Installing this doc builder via `poetry add clear-skies-doc-builder`.
+ 5. Run the doc builder.
+ 6. Build with Jekyll.
+ 7. Push to the appropriate subfolder via S3.
+ 8. (Only once) Update the main clearskies doc site to know about the new subfolder for this plugin.
+
+Of course, we want the Jekyll sites to be consistent with eachother in terms of style/look.  In the long run we'll probably have this doc builder also bootstrap the Jekyll site, but for now you just have to manually setup the Jekyll build using the main clearskies repo as a template.
+

clear_skies_doc_builder-2.0.0/README.md

@@ -0,0 +1,20 @@
+# docs
+
+The documentation builder for clearskies and related plugins
+
+## Overview
+
+Each "official" clearskies module (including the core clearskies library itself) has the documentation primarily written in the codebase as docblocks.  The documentation site is then built by extracting these docblocks and stitching them together.  To be clear, this isn't about the low-level "API" documentation that describes every single class/method in the framework.  Rather, this is about the primary documentation site itself (clearskies.info) which is focused on high-level use cases and examples of the primary configuration options.  As a result, it's not a simple matter of just iterating over the classes/methods and building documentation.  To build a coherent documentation site, each plugin has a configuration file that basically outlines the final "structure" or organization of the resulting documentation, as well as the name of a builder class that will combine that configuration information with the codebase itself to create the actual docs.
+
+The docs themselves (in the source code) are all written with markdown.  This documentation builder then takes that markdwon and adds the necessary headers/etc so to make them valid files for [Jekyll](https://jekyllrb.com/), the builder for the current documentation site.  The site itself is hosted in S3, so building an actual documentation site means:
+
+ 1. Properly documenting everything inside of the source code via markdown.
+ 2. Creating a config file (`docs/python/config.json`) to map source code docs to Jekyll files.
+ 3. Creating a skeleton of a Jekyll site in the `doc/jekyll` folder of the plugin.
+ 4. Installing this doc builder via `poetry add clear-skies-doc-builder`.
+ 5. Run the doc builder.
+ 6. Build with Jekyll.
+ 7. Push to the appropriate subfolder via S3.
+ 8. (Only once) Update the main clearskies doc site to know about the new subfolder for this plugin.
+
+Of course, we want the Jekyll sites to be consistent with eachother in terms of style/look.  In the long run we'll probably have this doc builder also bootstrap the Jekyll site, but for now you just have to manually setup the Jekyll build using the main clearskies repo as a template.

clear_skies_doc_builder-2.0.0/pyproject.toml

@@ -0,0 +1,70 @@
+[project]
+name = "clear-skies-doc-builder"
+description = "The docbuilder for all 'official' clearskies plugins (as well as the main clearskies docs)"
+version = "v2.0.0"
+license = "MIT"
+dynamic = ["classifiers"]
+readme = "./README.md"
+authors = [{name = "Conor Mancone", email = "cmancone@gmail.com"}]
+requires-python = '>=3.11,<4.0'
+dependencies = ['clear-skies (>=2.0.0,<3.0.0)']
+
+[project.urls]
+repository = "https://github.com/clearskies-py/docs"
+
+[tool.poetry]
+packages = [
+    { include = "clearskies_doc_builder", from = "src" }
+]
+exclude = [
+    "src/clearskies_doc_builder/*_test.py",
+    "src/clearskies_doc_builder/**/*_test.py",
+    "src/clearskies_doc_builder/integration_tests/*"
+]
+classifiers = [
+    "Development Status :: 5 - Production/Stable",
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Intended Audience :: Developers"
+]
+requires-poetry = '>=2.0,<3.0'
+
+[[tool.poetry.source]]
+name = "PyPI"
+priority = "primary"
+
+[tool.poetry.group.dev.dependencies]
+black = "^25.1.0"
+mypy = "^1.16.1"
+pre-commit = "^3.8.0"
+pytest = "^8.4.1"
+ruff = "^0.12.1"
+types-dateparser = "^1.2.2.20250627"
+types-jwcrypto = "^1.5.0.20250516"
+types-pymysql = "^1.1.0.20250516"
+types-requests = "^2.32.4.20250611"
+
+[tool.pytest]
+addopts = "--ignore=src/clearskies/contexts/test.py --cache-clear"
+
+[build-system]
+requires = ['poetry-core (>=2.0,<3.0)']
+build-backend = "poetry.core.masonry.api"
+
+[tool.black]
+line-length = 120
+# The following is Black's default, but it's good to be explicit
+# to match your Ruff config.
+skip-magic-trailing-comma = false
+preview = true
+
+[tool.mypy]
+python_version = "3.11"
+
+exclude = [
+ ".*_test\\.py$",
+ "docs/.*"
+]
+
+[tool.poetry-git-version-plugin]
+release_type = "tag"

clear_skies_doc_builder-2.0.0/src/clearskies_doc_builder/__init__.py

@@ -0,0 +1,34 @@
+import pathlib
+import sys
+import json
+
+import clearskies
+from clearskies_doc_builder import models, backends
+from clearskies_doc_builder.build_callable import build_callable
+
+
+def build(build_file_string: str) -> None:
+    # We assume a folder structure here where the repo root contains a `src/` folder and a `docs/python` folder.
+    # `build_file_string` should contain the absolute path to the file that kicked this off, which should
+    # live in the `docs/python` folder.  This comes in as a string, which we convert to a path.
+    # We then also need to calculate the path to the `src/` folder and add that to our
+    # python path.  We do this because we want to import the clearskies module in question, since the python
+    # code is where all of our documentation lives.
+
+    doc_python_path = pathlib.Path(build_file_string).parents[0]
+    project_root = doc_python_path.parents[1]
+    sys.path.append(str(project_root / "src"))
+
+    config_file = open(str(doc_python_path / "config.json"), "r")
+    config = json.loads(config_file.read())
+    config_file.close()
+
+    cli = clearskies.contexts.Cli(
+        build_callable,
+        modules=[models, backends],
+        bindings={
+            "config": config,
+            "project_root": project_root / "docs",
+        },
+    )
+    cli()

clear_skies_doc_builder-2.0.0/src/clearskies_doc_builder/backends/__init__.py

@@ -0,0 +1,9 @@
+from clearskies_doc_builder.backends.attribute_backend import AttributeBackend
+from clearskies_doc_builder.backends.class_backend import ClassBackend
+from clearskies_doc_builder.backends.module_backend import ModuleBackend
+
+__all__ = [
+    "AttributeBackend",
+    "ClassBackend",
+    "ModuleBackend",
+]

clear_skies_doc_builder-2.0.0/src/clearskies_doc_builder/backends/attribute_backend.py

@@ -0,0 +1,90 @@
+import inspect
+from typing import Any
+
+import clearskies
+import clearskies.model
+import clearskies.column
+import clearskies.query
+from clearskies_doc_builder.backends.module_backend import ModuleBackend
+
+
+class AttributeBackend(ModuleBackend):
+    _search_functions = {
+        "id": lambda attribute, name, value: id(attribute) == int(value),  # type: ignore
+        "name": lambda attribute, name, value: name == value,  # type: ignore
+        "type": lambda attribute, name, value: attribute.__class__ == value,  # type: ignore
+    }
+
+    def records(
+        self, query: clearskies.query.Query, next_page_data: dict[str, str | int] | None = None
+    ) -> list[dict[str, Any]]:
+        """
+        Returns a list of records that match the given query configuration
+
+        next_page_data is used to return data to the caller.  Pass in an empty dictionary, and it will be populated
+        with the data needed to return the next page of results.  If it is still an empty dictionary when returned,
+        then there is no additional data.
+        """
+        disallowed = ["joins", "selects", "group_by"]
+        for attribute_name in disallowed:
+            if getattr(query, attribute_name):
+                raise ValueError(f"The AttributeBackend received {attribute_name} in a query but doesn't support this.")
+
+        for condition in query.conditions:
+            if condition.operator != "=":
+                raise ValueError("The AttributeBackend only supports searching with the equals operator")
+
+        if "parent_class" not in query.conditions_by_column:
+            raise ValueError("When searching for attributes you must include a condition on 'parent_class'")
+
+        parent_class = query.conditions_by_column["parent_class"][0].values[0]
+        matching_attributes = []
+        for name in dir(parent_class):
+            attribute = getattr(parent_class, name)
+            matches = True
+            for condition in query.conditions:
+                if condition.column_name not in self._search_functions:
+                    continue
+                if not self._search_functions[condition.column_name](attribute, name, condition.values[0]):  # type: ignore
+                    matches = False
+
+            if not matches:
+                continue
+            matching_attributes.append(self.unpack(attribute, name, parent_class))
+
+        return self.paginate(matching_attributes, query)
+
+    def unpack(self, attribute: Any, name: str, parent_class: type) -> dict[str, Any]:  # type: ignore
+        all_args = []
+        args = []
+        kwargs = []
+        defaults: dict[str, Any] = {}
+        argdata = None
+        try:
+            argdata = inspect.getfullargspec(attribute)
+        except:
+            pass
+
+        if argdata:
+            nargs = len(argdata.args)
+            nkwargs = len(argdata.defaults) if argdata.defaults else 0
+            npargs = nargs - nkwargs
+            all_args = argdata.args
+            kwargs = all_args[nargs - nkwargs :]
+            args = all_args[:nkwargs]
+            defaults = {}
+            if argdata.defaults:
+                defaults = {argdata.args[index + npargs]: default for (index, default) in enumerate(argdata.defaults)}
+
+        return {
+            "id": id(attribute),
+            "name": name,
+            "type": attribute.__class__,
+            "doc": attribute.__doc__,
+            "parent_class": parent_class,
+            "attribute": attribute,
+            "all_args": all_args,
+            "args": args,
+            "kwargs": kwargs,
+            "defaults": defaults,
+        }

clear_skies_doc_builder-2.0.0/src/clearskies_doc_builder/backends/class_backend.py

@@ -0,0 +1,98 @@
+from typing import Any
+from types import ModuleType
+import importlib
+import inspect
+
+import clearskies
+import clearskies.model
+import clearskies.column
+import clearskies.query
+from clearskies_doc_builder.backends.module_backend import ModuleBackend
+
+
+class ClassBackend(ModuleBackend):
+    _search_functions = {
+        "id": lambda module, value: id(module) == int(value),
+        "source_file": lambda module, value: (module.__file__ if hasattr(module, "__file__") else "") == value,
+    }
+
+    def records(
+        self, query: clearskies.query.Query, next_page_data: dict[str, str | int] | None = None
+    ) -> list[dict[str, Any]]:
+        """
+        Returns a list of records that match the given query configuration
+
+        next_page_data is used to return data to the caller.  Pass in an empty dictionary, and it will be populated
+        with the data needed to return the next page of results.  If it is still an empty dictionary when returned,
+        then there is no additional data.
+        """
+        disallowed = ["joins", "selects", "group_by"]
+        for attribute_name in disallowed:
+            if getattr(query, attribute_name):
+                raise ValueError(f"The ClassBackend received {attribute_name} in a query but doesn't support this.")
+
+        for condition in query.conditions:
+            if condition.operator != "=":
+                raise ValueError("The ClassBackend only supports searching with the equals operator")
+
+        if "import_path" in query.conditions_by_column:
+            import_path = query.conditions_by_column["import_path"][0].values[0]
+            path_parts = import_path.split(".")
+            if len(path_parts) < 2:
+                raise ValueError(
+                    'In order to search for classes by import path you must provide the module and class name, e.g. `classes.find("import_path=clearskies.Endpoint")`'
+                )
+            class_name = path_parts[-1]
+            module_path = ".".join(path_parts[0:-1])
+            module = importlib.import_module(module_path)
+            if not hasattr(module, class_name):
+                raise ValueError(f"Module {import_path} has no class named {class_name}")
+            Class = getattr(module, class_name)
+            if not inspect.isclass(Class):
+                raise ValueError(
+                    f"I was asked to import the class named '{import_path}' but this doesn't actually reference a class"
+                )
+            return [self.unpack(Class, module)]
+
+        if "module" not in query.conditions_by_column:
+            raise ValueError(
+                "When searching for classes you must include a condition on either 'module' or 'import_path'"
+            )
+
+        parent_module = query.conditions_by_column["module"][0].values[0]
+        matching_classes = []
+        for name in dir(parent_module):
+            attribute = getattr(parent_module, name)
+            if not inspect.isclass(attribute):
+                continue
+
+            matches = True
+            for condition in query.conditions:
+                if condition.column_name not in self._search_functions:
+                    continue
+                if not self._search_functions[condition.column_name](attribute, condition.values[0]):
+                    matches = False
+
+            if not matches:
+                continue
+            matching_classes.append(self.unpack(attribute, parent_module))
+
+        return self.paginate(matching_classes, query)
+
+    def unpack(self, Class: type, module: ModuleType) -> dict[str, Any]:  # type: ignore
+        source_file = ""
+        try:
+            # this fails for built ins
+            source_file = inspect.getfile(Class)
+        except TypeError:
+            pass
+
+        return {
+            "id": id(Class),
+            "import_path": Class.__module__ + "." + Class.__name__,
+            "name": Class.__name__,
+            "source_file": source_file,
+            "doc": Class.__doc__,
+            "module": module,
+            "type": Class,
+        }

clear_skies_doc_builder-2.0.0/src/clearskies_doc_builder/backends/module_backend.py

@@ -0,0 +1,139 @@
+from typing import Any, Callable
+from types import ModuleType
+import sys
+import importlib
+
+import clearskies
+import clearskies.model
+import clearskies.column
+import clearskies.query
+from clearskies.autodoc.schema import Schema as AutoDocSchema
+from clearskies.autodoc.schema import Integer as AutoDocInteger
+
+
+class ModuleBackend(clearskies.backends.Backend):
+    _search_functions = {
+        "id": lambda module, value: id(module) == int(value),
+        "is_builtin": lambda module, value: (0 if hasattr(module, "__file__") else 1) == int(value),
+        "source_file": lambda module, value: (module.__file__ if hasattr(module, "__file__") else "") == value,
+        "module": lambda module, value: id(module) == id(value),
+    }
+
+    def update(self, id: int | str, data: dict[str, Any], model: clearskies.model.Model) -> dict[str, Any]:
+        """
+        Updates the record with the given id with the information from the data dictionary
+        """
+        raise Exception(f"The {self.__class__.__name__} only supports read operations: update is not allowed")
+
+    def create(self, data: dict[str, Any], model: clearskies.model.Model) -> dict[str, Any]:
+        """
+        Creates a record with the information from the data dictionary
+        """
+        raise Exception(f"The {self.__class__.__name__} only supports read operations: create is not allowed")
+
+    def delete(self, id: int | str, model: clearskies.model.Model) -> bool:
+        """
+        Deletes the record with the given id
+        """
+        raise Exception(f"The {self.__class__.__name__} only supports read operations: delete is not allowed")
+
+    def count(self, query: clearskies.query.Query) -> int:
+        """
+        Returns the number of records which match the given query configuration
+        """
+        return len(self.records(query))
+
+    def records(
+        self, query: clearskies.query.Query, next_page_data: dict[str, str | int] | None = None
+    ) -> list[dict[str, Any]]:
+        """
+        Returns a list of records that match the given query configuration
+
+        next_page_data is used to return data to the caller.  Pass in an empty dictionary, and it will be populated
+        with the data needed to return the next page of results.  If it is still an empty dictionary when returned,
+        then there is no additional data.
+        """
+        disallowed = ["joins", "selects", "group_by"]
+        for attribute_name in disallowed:
+            if getattr(query, attribute_name):
+                raise ValueError(f"The ModuleBackend received {attribute_name} in a query but doesn't support this.")
+
+        for condition in query.conditions:
+            if condition.operator != "=":
+                raise ValueError("The ModuleBackend only supports searching with the equals operator")
+
+        module_name_condition = query.conditions_by_column.get("import_path", query.conditions_by_column.get("name"))
+        if module_name_condition:
+            module_name = module_name_condition[0].values[0]
+            module = importlib.import_module(module_name)
+            return [self.unpack(module)]
+
+        matching_modules = []
+        module_names = set(sys.modules) & set(globals())
+        for module_name in module_names:
+            if module_name not in sys.modules:
+                continue
+
+            module = sys.modules[module_name]
+            matches = True
+            for condition in query.conditions:
+                if condition.column_name not in self._search_functions:
+                    continue
+                if not self._search_functions[condition.column_name](module, condition.values[0]):
+                    matches = False
+
+            if not matches:
+                continue
+
+            matching_modules.append(self.unpack(module))
+
+        return self.paginate(matching_modules, query)
+
+    def unpack(self, module: ModuleType) -> dict[str, Any]:
+        return {
+            "id": id(module),
+            "import_path": module.__name__,
+            "name": module.__name__,
+            "is_builtin": not hasattr(module, "__file__"),
+            "source_file": module.__file__ if hasattr(module, "__file__") else "",
+            "doc": module.__doc__,
+            "module": module,
+        }
+
+    def paginate(self, records, query):
+        return records
+
+    def validate_pagination_data(self, data: dict[str, Any], case_mapping: Callable) -> str:
+        extra_keys = set(data.keys()) - set(self.allowed_pagination_keys())
+        if len(extra_keys):
+            key_name = case_mapping("start")
+            return "Invalid pagination key(s): '" + "','".join(extra_keys) + f"'.  Only '{key_name}' is allowed"
+        if "start" not in data:
+            key_name = case_mapping("start")
+            return f"You must specify '{key_name}' when setting pagination"
+        start = data["start"]
+        try:
+            start = int(start)
+        except:
+            key_name = case_mapping("start")
+            return f"Invalid pagination data: '{key_name}' must be a number"
+        return ""
+
+    def allowed_pagination_keys(self) -> list[str]:
+        return ["start"]
+
+    def documentation_pagination_next_page_response(self, case_mapping: Callable[[str], str]) -> list[Any]:
+        return [AutoDocInteger(case_mapping("start"), example=0)]
+
+    def documentation_pagination_next_page_example(self, case_mapping: Callable[[str], str]) -> dict[str, Any]:
+        return {case_mapping("start"): 0}
+
+    def documentation_pagination_parameters(
+        self, case_mapping: Callable[[str], str]
+    ) -> list[tuple[AutoDocSchema, str]]:
+        return [
+            (
+                AutoDocInteger(case_mapping("start"), example=0),
+                "The zero-indexed record number to start listing results from",
+            )
+        ]

clear_skies_doc_builder-2.0.0/src/clearskies_doc_builder/build_callable.py

@@ -0,0 +1,24 @@
+from typing import Any
+
+from clearskies_doc_builder import models
+from clearskies_doc_builder.prepare_doc_space import prepare_doc_space
+
+
+def build_callable(modules: models.Module, classes: models.Class, config: dict[str, Any], project_root: str):
+    doc_root = prepare_doc_space(project_root)
+    nav_order_parent_count = {}
+
+    for index, branch in enumerate(config["tree"]):
+        nav_order_title_tracker = branch.get("parent", branch["title"])
+        if nav_order_title_tracker not in nav_order_parent_count:
+            nav_order_parent_count[nav_order_title_tracker] = 0
+        nav_order_parent_count[nav_order_title_tracker] += 1
+        builder_class = classes.find("import_path=" + branch["builder"]).type
+        builder = builder_class(
+            branch,
+            modules,
+            classes,
+            doc_root,
+            nav_order=nav_order_parent_count[nav_order_title_tracker] if branch.get("parent") else index + 2,
+        )
+        builder.build()

clear_skies_doc_builder-2.0.0/src/clearskies_doc_builder/builders/__init__.py

@@ -0,0 +1,9 @@
+from clearskies_doc_builder.builders.module import Module
+from clearskies_doc_builder.builders.single_class import SingleClass
+from clearskies_doc_builder.builders.single_class_to_section import SingleClassToSection
+
+__all__ = [
+    "Module",
+    "SingleClass",
+    "SingleClassToSection",
+]