PyProd 0.2.0__tar.gz → 0.3.0__tar.gz

{pyprod-0.2.0 → pyprod-0.3.0}/.github/workflows/publish.yml

@@ -5,6 +5,7 @@ on:
   push:
     tags:
       - "[0-9]+.[0-9]+.[0-9]+"
+      - "[0-9]+.[0-9]+.[0-9]+.post[0-9]+"
 
 concurrency:
   group: pyprod-deploy

{pyprod-0.2.0 → pyprod-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: PyProd
-Version: 0.2.0
+Version: 0.3.0
 Summary: PyProd: More Makeable than Make
 Project-URL: Homepage, https://github.com/atsuoishimoto/pyprod
 Project-URL: Documentation, https://pyprod.readthedocs.io/en/latest/
@@ -17,7 +17,7 @@ Description-Content-Type: text/x-rst
 PyProd - More Makeable than Make
 =================================
 
-PyProd is a Python script that can be used as an alternative to Makefile. By leveraging Python's versatility, it enables you to define build rules and dependencies programmatically, allowing for dynamic configurations, integration with existing Python libraries, and custom build logic not easily achievable with traditional Makefiles. For detailed documentation, please refer to the `official documentation <https://example.com/pyprod-docs>`_.
+PyProd is a Python script that can be used as an alternative to Makefile. By leveraging Python's versatility, it enables you to define build rules and dependencies programmatically, allowing for dynamic configurations, integration with existing Python libraries, and custom build logic not easily achievable with traditional Makefiles. For detailed documentation, please refer to the `official documentation <https://pyprod.readthedocs.io/en/latest/>`_.
 
 
 Features

{pyprod-0.2.0 → pyprod-0.3.0}/README.rst

@@ -1,7 +1,7 @@
 PyProd - More Makeable than Make
 =================================
 
-PyProd is a Python script that can be used as an alternative to Makefile. By leveraging Python's versatility, it enables you to define build rules and dependencies programmatically, allowing for dynamic configurations, integration with existing Python libraries, and custom build logic not easily achievable with traditional Makefiles. For detailed documentation, please refer to the `official documentation <https://example.com/pyprod-docs>`_.
+PyProd is a Python script that can be used as an alternative to Makefile. By leveraging Python's versatility, it enables you to define build rules and dependencies programmatically, allowing for dynamic configurations, integration with existing Python libraries, and custom build logic not easily achievable with traditional Makefiles. For detailed documentation, please refer to the `official documentation <https://pyprod.readthedocs.io/en/latest/>`_.
 
 
 Features

{pyprod-0.2.0 → pyprod-0.3.0}/docs/index.rst

@@ -35,6 +35,8 @@ Table of Contents
    quickstart
    prodfile
    commandline
+   releasenotes
+   
 
 
 

{pyprod-0.2.0 → pyprod-0.3.0}/docs/prodfile.rst

@@ -29,7 +29,13 @@ A rule is defined using the ``@rule`` decorator, which takes the target file as
 
 Build function
 ~~~~~~~~~~~~~~~~~~~
-The first argument of the decorated function specifies the target file to be generated. The target can be a string or a Path object, depending on the value provided in the target parameter. Subsequent arguments correspond to the filenames listed in the depends parameter. The rule function must accept the target and the same number of arguments as those specified in depends.
+
+The function following the ``@rule`` decorator is the build function that generates the target file. 
+
+- The first argument of the build function specifies the target file to be generated.
+- Subsequent arguments correspond to the filenames listed in the depends parameter. The rule function must accept the target and the same number of arguments as those specified in depends.
+- ``uses`` dependencies are not passed to the build function.
+- Even if Path objects are specified in the ``rule``, all arguments passed to the builder function will be of type str.
 
 For example, the following code prints ``file1 ['file2', 'file3']`` when the target file ``file1`` is built:
 
@@ -229,6 +235,36 @@ Example:
       msg = capture("echo Hello, World!")
    
 
+.. py:function:: read(filename):
+   
+   Read the contents of a file.
+
+   :param filename: The file to read.
+   :type filename: str | Path
+
+   :return: The contents of the file.
+   :rtype: str
+
+.. py:function:: write(filename, txt, append=False):
+   
+   Write text to a file.
+
+   :param filename: The file to write to.
+   :type filename: str | Path
+
+   :param txt: The text to write.
+   :type txt: str
+
+   :param append: Append to the file instead of overwriting it (default ``False``).
+   :type append: bool
+
+.. py:function:: makedirs(path):
+   
+   Create a directory along with any necessary parent directories if they do not already exist. This function wraps `os.makedirs() <https://docs.python.org/3/library/os.html#os.makedirs>`_ with the ``exists_ok`` parameter set to ``True``.
+
+   :param path: The directory to create.
+   :type path: str | Path
+
 .. py:function::  glob(path, dir=".")
 
    Glob the given relative pattern in the directory represented by this path. This function is a wrapper around `pathlib.Path.glob() <https://docs.python.org/3/library/pathlib.html#pathlib.Path.glob>`_. Unlike ``pathlib.Path.glob()``, this function ignores files and directlies that start with a dot. Also, this function returns a list of Path objects.
@@ -250,8 +286,9 @@ Example:
       SRCFILES = glob("**/*.c")
    
 .. py:function::  quote(s)
+.. py:function::  q(s)
 
-   Quote a string for use as a shell command argument. This function is a wrapper around `shlex.quote() <https://docs.python.org/3/library/shlex.html#shlex.quote>`_.
+   Convert ``s`` to string and quote for use as a shell command argument. This function is a wrapper around `shlex.quote() <https://docs.python.org/3/library/shlex.html#shlex.quote>`_.
 
    :param s: The string to quote.
    :type s: str
@@ -259,6 +296,17 @@ Example:
    :return: The quoted string.
    :rtype: str
 
+.. py:function::  squote(*s)
+.. py:function::  sq(*s)
+
+   Quote strings in ``s``. Each ``s`` is flattend.
+
+   :param s: The string to quote.
+   :type s: str | list
+
+   :return: The list of quoted strings.
+   :rtype: list[str]
+
 .. py:class::  Path
    
    A class representing file paths. This function is an alias for `pathlib.Path <https://docs.python.org/3/library/pathlib.html#pathlib.Path>`_.

{pyprod-0.2.0 → pyprod-0.3.0}/docs/quickstart.rst

@@ -52,16 +52,16 @@ Next, let's modify the ``Prodfile.py`` to output the file into an ``output`` dir
 
 .. code-block:: python
 
-    output = Path("output") # We can use pathlib.Path without importing it
-    
-    @rule(output / "hello.txt", depends=output) # hello now depends on output directory
-    def hello(target):
-        with open(target, "w") as f:
-            f.write("Hello, world!")
-
-    @rule(output)
-    def makedir(target):
-        target.mkdir()
+   output = Path("output") # We can use pathlib.Path without importing it
+   
+   @rule(output / "hello.txt", depends=output) # hello now depends on output directory
+   def hello(target):
+       with open(target, "w") as f:
+           f.write("Hello, world!")
+
+   @rule(output)
+   def makedir(target):
+       os.makedirs(target)
 
 In the modified ``Prodfile.py``, we have defined a rule to create the ``output`` directory and added a rule that makes the ``output/hello.txt`` file dependent on the ``output`` directory.
 

pyprod-0.3.0/docs/releasenotes.rst

@@ -0,0 +1,9 @@
+Release Notes
+================
+
+0.3.0 (2025-01-03)
+------------------
+
+- Arguments for build and check function are converted to string.
+- Add built-in functions.
+- Validate rule dependencies.

{pyprod-0.2.0 → pyprod-0.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "PyProd"
-version = "0.2.0"
+version = "0.3.0"
 description = "PyProd: More Makeable than Make"
 readme = "README.rst"
 requires-python = ">=3.10"
@@ -40,4 +40,4 @@ asyncio_default_fixture_loop_scope="function"
 
 [tool.ruff.lint]
 select = ["E","F","W","B","N","T10","I"]
-ignore = ["F401"]
+ignore = ["F401", "B9"]

{pyprod-0.2.0 → pyprod-0.3.0}/sample/md-to-pdf/Prodfile.py

@@ -31,16 +31,15 @@ def make_pdf(target, src):
 # Rebuilds when Python modules change
 @rule(BUILD / "%.html", depends=(Path("%.md"), TEMPLATE, MODULES), uses=BUILD)
 def make_html(target, src, template, *_):
-    src = src.read_text()
-    body = md_to_html(src)
-    html = template.read_text().format(body=body)
-    target.write_text(html)
+    body = md_to_html(open(src).read())
+    html = open(template).read().format(body=body)
+    open(target, "w").write(html)
 
 
 # create outputs directory
 @rule(BUILD)
 def builds(target):
-    target.mkdir(parents=True)
+    os.makedirs(target)
 
 
 def clean():

pyprod-0.2.0/sample/s3files/PRODFILE.py → pyprod-0.3.0/sample/s3files/Prodfile.py

@@ -7,7 +7,8 @@ import boto3, botocore
 from urllib.parse import urlparse
 
 s3 = boto3.client("s3")
-TARGET = f"s3://TESTBUCKET/S3TEST.txt"
+BUCKET = params.BUCKET or "TESTBUCKET"  # Run pyprod with BUCKET=bucket-name
+TARGET = f"s3://{BUCKET}/S3TEST.txt"
 
 
 def parse_s3url(s3url):

pyprod-0.3.0/sample/tutorial-1/Prodfile.py

@@ -0,0 +1,4 @@
+@rule("hello.txt")
+def hello(target):
+    with open(target, "w") as f:
+        f.write("Hello, world!")

pyprod-0.3.0/sample/tutorial-2/Prodfile.py

@@ -0,0 +1,13 @@
+output = Path("output")  # We can use pathlib.Path without importing it
+
+
+@rule(output / "hello.txt", depends=output)  # hello now depends on output directory
+def hello(target, *args):
+    # output_dir is not used
+    with open(target, "w") as f:
+        f.write("Hello, world!")
+
+
+@rule(output)
+def makedir(target):
+    os.makedirs(target)

{pyprod-0.2.0 → pyprod-0.3.0}/src/pyprod/prod.py

@@ -33,7 +33,7 @@ class NoRuleToMakeTargetError(Exception):
     pass
 
 
-class InvalidRuleError(Exception):
+class RuleError(Exception):
     pass
 
 
@@ -109,13 +109,11 @@ def glob(path, dir="."):
 
 def rule_to_re(rule):
     if not isinstance(rule, (str, Path)):
-        raise InvalidRuleError(rule)
+        raise RuleError(rule)
 
     srule = str(rule)
     srule = translate(srule)
     srule = replace_pattern(srule, "(?P<stem>.*)", maxreplace=1)
-    if isinstance(rule, Path):
-        return Path(srule)
     return srule
 
 
@@ -132,38 +130,97 @@ def replace_pattern(rule, replaceto, *, maxreplace=None):
             if maxreplace is not None:
                 if n > maxreplace:
                     # contains multiple '%'
-                    raise InvalidRuleError(s_rule)
+                    raise RuleError(f"{s_rule} contains multiple '%'")
+
             return replaceto
 
     s_rule = re.sub("%%|%", f, s_rule)
-    if isinstance(rule, Path):
-        return Path(s_rule)
     return s_rule
 
 
-@dataclass
+def _check_pattern_count(pattern):
+    """Counts number of '%' in the pattern"""
+    matches = re.finditer(r"%%|%", pattern)
+    num = len([m for m in matches if len(m[0]) == 1])
+    if num > 1:
+        raise RuleError(f"{pattern}: Multiple '%' is not allowed")
+    return num
+
+
+def _check_pattern(pattern):
+    matches = re.finditer(r"%%|%", pattern)
+    singles = [m for m in matches if len(m[0]) == 1]
+    if len(singles) > 1:
+        raise RuleError(f"{pattern}: Multiple '%' is not allowed")
+    if not len(singles):
+        raise RuleError(f"{pattern}: Pattern should contain a '%'.")
+
+
+def _strip_dot(path):
+    if not path:
+        return path
+    path = Path(path)  # ./aaa/ -> aaa
+    parts = path.parts
+    if ".." in parts:
+        raise RuleError(f"{path}: '..' directory is not allowed")
+    return str(path)
+
+
+def _check_wildcard(path):
+    if "*" in path:
+        raise RuleError(f"{path}: '*' directory is not allowed")
+
+
 class Rule:
-    targets: list
-    pattern: str | None
-    depends: list
-    uses: list | None
-    builder: str | None
-
-    orig_targets: list = field(init=False)
-    orig_depends: list = field(init=False)
-    orig_uses: list = field(init=False)
-
-    def __post_init__(self):
-        self.orig_targets = self.targets
-        self.orig_depends = self.depends
-        self.orig_uses = self.uses
-
-        self.targets = [rule_to_re(r) for r in flatten(self.targets)]
-        self.depends = list(flatten(self.depends))
-        if self.pattern:
-            self.pattern = rule_to_re(self.pattern)
-        if self.uses:
-            self.uses = list(flatten(self.uses))
+    def __init__(self, targets, pattern, depends, uses, builder=None):
+        self.targets = []
+        for target in flatten(targets or ()):
+            target = str(target)
+            _check_pattern_count(target)
+            target = _strip_dot(target)
+            target = rule_to_re(target)
+            self.targets.append(target)
+
+        self.first_target = None
+        for target in flatten(targets or ()):
+            target = str(target)
+            if not target:
+                continue
+
+            if "*" in target:
+                continue
+
+            if _check_pattern_count(target) == 0:
+                # not contain one %
+                self.first_target = target
+                break
+
+        if pattern:
+            pattern = _strip_dot(pattern)
+            if _check_pattern_count(pattern) != 1:
+                raise RuleError(f"{pattern}: Pattern should contain a '%'")
+
+            self.pattern = rule_to_re(pattern)
+        else:
+            self.pattern = None
+
+        self.depends = []
+        for depend in flatten(depends or ()):
+            depend = str(depend)
+            _check_pattern_count(depend)
+            _check_wildcard(depend)
+            depend = _strip_dot(depend)
+            self.depends.append(depend)
+
+        self.uses = []
+        for use in flatten(uses or ()):
+            use = str(use)
+            _check_pattern_count(use)
+            _check_wildcard(use)
+            use = _strip_dot(use)
+            self.uses.append(use)
+
+        self.builder = builder
 
     def __call__(self, f):
         self.builder = f
@@ -211,12 +268,15 @@ class Rules:
                         stem = d
                     elif dep.pattern:
                         m = re.fullmatch(str(dep.pattern), name)
-                        d = m.groupdict().get("stem", None)
-                        if d is not None:
-                            stem = d
+                        if m:
+                            stem = m.groupdict().get("stem", None)
 
-                    depends = [replace_pattern(r, stem) for r in dep.depends]
-                    uses = [replace_pattern(r, stem) for r in dep.uses]
+                    if stem is not None:
+                        depends = [replace_pattern(r, stem) for r in dep.depends]
+                        uses = [replace_pattern(r, stem) for r in dep.uses]
+                    else:
+                        depends = dep.depends[:]
+                        uses = dep.uses[:]
 
                     yield depends, uses, dep
                     break
@@ -237,14 +297,8 @@ class Rules:
 
     def select_first_target(self):
         for dep in self.rules:
-            for target in dep.orig_targets:
-                s_target = str(target)
-                # pattern?
-                matches = re.finditer(r"%%|%", s_target)
-                if any((len(m[0]) == 1) for m in matches):
-                    # has %
-                    continue
-                return target
+            if dep.first_target:
+                return dep.first_target
 
     def select_builder(self, name):
         for depends, uses, dep in self.iter_rule(name):
@@ -356,6 +410,30 @@ class Envs:
         return os.environ.get(name, default=default)
 
 
+def read(filename):
+    with open(filename, "r") as f:
+        return f.read()
+
+
+def write(filename, s, append=False):
+    mode = "a" if append else "w"
+    with open(filename, mode) as f:
+        f.write(s)
+
+
+def quote(s):
+    return shlex.quote(str(s))
+
+
+def squote(*s):
+    ret = [shlex.quote(str(x)) for x in flatten(s)]
+    return ret
+
+
+def makedirs(path):
+    os.makedirs(path, exist_ok=True)
+
+
 class Prod:
     def __init__(self, modulefile, njobs=1, params=None):
         self.modulefile = Path(modulefile)
@@ -372,18 +450,25 @@ class Prod:
 
     def get_module_globals(self):
         globals = {
+            "capture": capture,
+            "check": self.checkers.check,
+            "environ": Envs(),
+            "glob": glob,
+            "makedirs": makedirs,
+            "os": os,
+            "params": self.params,
             "pip": pip,
+            "quote": quote,
+            "q": quote,
+            "squote": squote,
+            "sq": squote,
+            "read": read,
             "rule": self.rules.rule,
-            "check": self.checkers.check,
-            "Path": Path,
             "run": run,
-            "capture": capture,
-            "glob": glob,
-            "MAX_TS": MAX_TS,
-            "environ": Envs(),
             "shutil": shutil,
-            "quote": shlex.quote,
-            "params": self.params,
+            "write": write,
+            "MAX_TS": MAX_TS,
+            "Path": Path,
         }
         return globals
 
@@ -432,13 +517,13 @@ class Prod:
             if isinstance(obj, str | Path):
                 builds.append(obj)
             elif isinstance(obj, Rule):
-                raise InvalidRuleError(obj)
+                raise RuleError(obj)
             elif callable(obj):
                 self.built += 1
                 task = asyncio.create_task(self.run_in_executor(obj))
                 waitings.append(task)
             else:
-                raise InvalidRuleError(obj)
+                raise RuleError(obj)
 
         await self.build(builds)
         await asyncio.gather(*waitings)
@@ -497,11 +582,11 @@ class Prod:
         return Exists(name, True, ret)
 
     async def run(self, name):  # -> Any | int:
-        s_name = str(name)
+        name = str(name)
 
-        self.rules.build_tree(s_name)
-        deps, uses = self.rules.get_dep_names(s_name)
-        selected = self.rules.select_builder(s_name)
+        self.rules.build_tree(name)
+        deps, uses = self.rules.get_dep_names(name)
+        selected = self.rules.select_builder(name)
         if selected:
             build_deps, build_uses, builder = selected
             deps = deps + build_deps
@@ -513,19 +598,19 @@ class Prod:
         if uses:
             await self.build(uses)
 
-        exists = await self.is_exists(s_name)
+        exists = await self.is_exists(name)
 
         if not exists.exists:
-            logger.debug("%r does not exists", str(s_name))
+            logger.debug("%r does not exists", name)
         elif (ts >= MAX_TS) or (exists.ts < ts):
-            logger.debug("%r should be updated", str(s_name))
+            logger.debug("%r should be updated", name)
         else:
-            logger.debug("%r already exists", str(s_name))
+            logger.debug("%r already exists", name)
 
         if not exists.exists and not selected:
-            raise NoRuleToMakeTargetError(s_name)
+            raise NoRuleToMakeTargetError(name)
         elif selected and ((not exists.exists) or (ts >= MAX_TS) or (exists.ts < ts)):
-            logger.warning("building: %r", s_name)
+            logger.warning("building: %r", name)
             await self.run_in_executor(builder.builder, name, *build_deps)
             self.built += 1
             return MAX_TS

{pyprod-0.2.0 → pyprod-0.3.0}/tests/test_prod.py

@@ -88,19 +88,19 @@ async def test_pattern(tmp_path):
     src = """
 @rule(target=("a.o", "b.o"), pattern=Path("%.o"), depends=Path("%.c"))
 def build(target, src):
-    assert isinstance(target, Path)
+    assert isinstance(target, str)
     Path(target).write_text(str(target))
 
 @rule(target=Path("%.c"))
 def build_c(target):
-    assert isinstance(target, Path)
+    assert isinstance(target, str)
     Path(target).write_text(str(target))
 
 @rule(Path("app.exe"), depends=(Path("a.o"), Path("b.o")))
 def build_app(target, a, b):
-    assert isinstance(target, Path)
-    assert isinstance(a, Path)
-    assert isinstance(b, Path)
+    assert isinstance(target, str)
+    assert isinstance(a, str)
+    assert isinstance(b, str)
     Path(target).write_text(f"{target}, {a}, {b}")
 
 all = Path("app.exe")
@@ -124,19 +124,19 @@ async def test_preserve_pathobj(tmp_path):
     src = """
 @rule(target=Path("%.o"), depends=Path("%.c"))
 def build(target, src):
-    assert isinstance(target, Path)
+    assert isinstance(target, str)
     Path(target).write_text("a")
 
 @rule(target=Path("%.c"))
 def build_c(target):
-    assert isinstance(target, Path)
+    assert isinstance(target, str)
     Path(target).write_text(str(target))
 
 @rule(Path("app.exe"), depends=Path("a.o"))
 def build_app(target, src):
-    assert isinstance(target, Path)
-    assert isinstance(src, Path)
-    target.write_text("app.exe")
+    assert isinstance(target, str)
+    assert isinstance(src, str)
+    Path(target).write_text("app.exe")
 
 all = Path("app.exe")
 """

{pyprod-0.2.0 → pyprod-0.3.0}/tests/test_prodfuncs.py

@@ -47,3 +47,7 @@ def test_glob(tmp_path):
         d1 / "b.txt",
         d2 / "a.txt",
     }
+
+
+def test_squote():
+    assert prod.squote("abc", ["12 3"]) == ["abc", "'12 3'"]

{pyprod-0.2.0 → pyprod-0.3.0}/tests/test_rule.py

@@ -1,3 +1,5 @@
+import re
+
 import pytest
 
 from pyprod import prod
@@ -91,20 +93,74 @@ def test_stem_escape():
 
 
 def test_stem_error():
-    with pytest.raises(ValueError):
-        raise ValueError("s;dlf,")
-
     rules = prod.Rules()
 
-    with pytest.raises(prod.InvalidRuleError):
+    with pytest.raises(prod.RuleError):
 
         @rules.rule(target="%.%", depends="%.c")
         def f():
             pass
 
-    @rules.rule(target="%.xxx", depends="%.%")
+    @rules.rule(target="%.xxx", depends="%")
     def f():
         pass
 
     deps, _, _ = rules.select_builder("abc.xxx")
-    assert deps == ["abc.abc"]
+    assert deps == ["abc"]
+
+
+def test_validate_target():
+    with pytest.raises(prod.RuleError):
+        prod.Rule("%.%", "", "", "")
+
+    with pytest.raises(prod.RuleError):
+        prod.Rule("../aaa", "", "", "")
+
+    with pytest.raises(prod.RuleError):
+        rule = prod.Rule("../aaa/", "", "", "")
+
+    rule = prod.Rule("./aaa/", None, None, None)
+    assert re.fullmatch(rule.targets[0], "aaa")
+
+
+def test_validate_pattern():
+    with pytest.raises(prod.RuleError):
+        prod.Rule("a.b", "%.%", "", "")
+
+    with pytest.raises(prod.RuleError):
+        prod.Rule("a.b", "../%.b", "", "")
+
+    with pytest.raises(prod.RuleError):
+        rule = prod.Rule("a.b", "../a.%", "", "")
+
+    with pytest.raises(prod.RuleError):
+        rule = prod.Rule("a.b", "a.b", "", "")
+
+    rule = prod.Rule("a.b", "./a.%", None, None)
+    assert re.fullmatch(rule.pattern, "a.b")
+
+
+def test_validate_depends():
+    with pytest.raises(prod.RuleError):
+        prod.Rule("a.%", None, "%.%", "")
+
+    with pytest.raises(prod.RuleError):
+        prod.Rule("a.%", None, "*/x.y", "")
+
+    with pytest.raises(prod.RuleError):
+        prod.Rule("a.%", None, "../x.y", "")
+
+    prod.Rule("a.b", None, "x.y", "")
+
+
+def test_validate_uses():
+    with pytest.raises(prod.RuleError):
+        prod.Rule("a.%", None, "", "%.%")
+
+    with pytest.raises(prod.RuleError):
+        prod.Rule("a.%", None, "", "*/x.y")
+
+    with pytest.raises(prod.RuleError):
+        prod.Rule("a.%", None, "", "../x.y")
+
+    prod.Rule("a.b", None, "x.y", "")

{pyprod-0.2.0 → pyprod-0.3.0}/uv.lock

@@ -278,7 +278,7 @@ wheels = [
 
 [[package]]
 name = "pyprod"
-version = "0.1.0"
+version = "0.2.0.post1"
 source = { editable = "." }
 
 [package.dev-dependencies]