yaml-reference 2.2.1__tar.gz → 2.3.0__tar.gz

{yaml_reference-2.2.1 → yaml_reference-2.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: yaml-reference
-Version: 2.2.1
+Version: 2.3.0
 Summary: Extension package built on top of  `ruamel.yaml` to support cross-file references in YAML files using tags `!reference` and `!reference-all`.
 Project-URL: Repository, https://github.com/dsillman2000/yaml-reference.git
 Author-email: David Sillman <dsillman2000@gmail.com>
@@ -155,15 +155,15 @@ version: 3.1
 $ yaml-reference-cli root.yaml | yq -P > .compiled/root.yaml
 ```
 
-## Safety note
+## Circular reference protection
 
-As of now, the specification does not require any explicit protection against circular references. This package does not check for circular references and will result in an infinite loop (max recursion depth exceeded) if a circular reference is encountered. Onus is on the users of this package to ensure that circular references are not present in their referential YAML files.
+As required by the yaml-reference-specs specification, this package includes circular reference detection to prevent infinite recursion. If a circular reference is detected (e.g., A references B, B references C, C references A), a `ValueError` will be raised with a descriptive error message. This protects against self-references and circular chains in both `!reference` and `!reference-all` tags.
 
 ## Security considerations
 
-### Path restriction with `allow_paths`
+### Path restriction and `allow_paths`
 
-By default, `!reference` and `!reference-all` tags can only reference files within the same directory as the source YAML file. To allow references to files in other directories, you must explicitly specify allowed paths using the `allow_paths` parameter:
+By default, `!reference` and `!reference-all` tags can only reference files within the same directory as the source YAML file (or child subdirectories). To allow references to files in other disparate directory trees, you must explicitly specify allowed paths using the `allow_paths` parameter:
 
 ```python
 from yaml_reference import load_yaml_with_references
@@ -181,15 +181,11 @@ In the CLI, use the `--allow` flag:
 yaml-reference compile input.yml --allow /allowed/path1 --allow /allowed/path2
 ```
 
-### Absolute path restrictions
-
-References using absolute paths (e.g., `/tmp/file.yml`) are explicitly rejected with a `ValueError`. All reference paths must be relative to the source file's directory.
-
-### Permission errors
+Whether or not `allow_paths` is specified, the default behavior is to allow references to files in the same directory as the source YAML file (or subdirectories). "Back-navigating" out of a the root directory is not allowed (".." local references in a root YAML file). This provides a secure baseline to prevent unsafe access which is not explicitly allowed.
 
-If a reference attempts to access a file outside the allowed paths, a `PermissionError` is raised. This prevents unauthorized file access through YAML references.
+### Absolute path restrictions
 
-Whether or not `allow_paths` is specified, the default behavior is to allow references to files in the same directory as the source YAML file (or subdirectories). "Back-navigating" out of a the root directory is not allowed (".." local references in a root YAML file). This provides a secure baseline to prevent unsafe access which is not explicitly allowed.
+References using absolute paths (e.g., `/tmp/file.yml`) are explicitly rejected with a `ValueError`. All reference paths must be relative to the source file's directory. If you absolutely must reference an absolute path, relative paths to symlinks can be used. Note that their target directories must be explicitly allowed to avoid permission errors (see the above section about "Path restriction and `allow_paths`").
 
 ## Acknowledgements
 

{yaml_reference-2.2.1 → yaml_reference-2.3.0}/README.md

@@ -129,15 +129,15 @@ version: 3.1
 $ yaml-reference-cli root.yaml | yq -P > .compiled/root.yaml
 ```
 
-## Safety note
+## Circular reference protection
 
-As of now, the specification does not require any explicit protection against circular references. This package does not check for circular references and will result in an infinite loop (max recursion depth exceeded) if a circular reference is encountered. Onus is on the users of this package to ensure that circular references are not present in their referential YAML files.
+As required by the yaml-reference-specs specification, this package includes circular reference detection to prevent infinite recursion. If a circular reference is detected (e.g., A references B, B references C, C references A), a `ValueError` will be raised with a descriptive error message. This protects against self-references and circular chains in both `!reference` and `!reference-all` tags.
 
 ## Security considerations
 
-### Path restriction with `allow_paths`
+### Path restriction and `allow_paths`
 
-By default, `!reference` and `!reference-all` tags can only reference files within the same directory as the source YAML file. To allow references to files in other directories, you must explicitly specify allowed paths using the `allow_paths` parameter:
+By default, `!reference` and `!reference-all` tags can only reference files within the same directory as the source YAML file (or child subdirectories). To allow references to files in other disparate directory trees, you must explicitly specify allowed paths using the `allow_paths` parameter:
 
 ```python
 from yaml_reference import load_yaml_with_references
@@ -155,15 +155,11 @@ In the CLI, use the `--allow` flag:
 yaml-reference compile input.yml --allow /allowed/path1 --allow /allowed/path2
 ```
 
-### Absolute path restrictions
-
-References using absolute paths (e.g., `/tmp/file.yml`) are explicitly rejected with a `ValueError`. All reference paths must be relative to the source file's directory.
-
-### Permission errors
+Whether or not `allow_paths` is specified, the default behavior is to allow references to files in the same directory as the source YAML file (or subdirectories). "Back-navigating" out of a the root directory is not allowed (".." local references in a root YAML file). This provides a secure baseline to prevent unsafe access which is not explicitly allowed.
 
-If a reference attempts to access a file outside the allowed paths, a `PermissionError` is raised. This prevents unauthorized file access through YAML references.
+### Absolute path restrictions
 
-Whether or not `allow_paths` is specified, the default behavior is to allow references to files in the same directory as the source YAML file (or subdirectories). "Back-navigating" out of a the root directory is not allowed (".." local references in a root YAML file). This provides a secure baseline to prevent unsafe access which is not explicitly allowed.
+References using absolute paths (e.g., `/tmp/file.yml`) are explicitly rejected with a `ValueError`. All reference paths must be relative to the source file's directory. If you absolutely must reference an absolute path, relative paths to symlinks can be used. Note that their target directories must be explicitly allowed to avoid permission errors (see the above section about "Path restriction and `allow_paths`").
 
 ## Acknowledgements
 

{yaml_reference-2.2.1 → yaml_reference-2.3.0}/tests/unit/test_reference.py

@@ -160,3 +160,50 @@ def test_allow_paths_load_yaml_with_references(stage_files):
         load_yaml_with_references(
             stg / "inner/with_all.yml", allow_paths=[stg / "some"]
         )
+
+
+@pytest.mark.parametrize(
+    "test_name, files, entry_point",
+    [
+        (
+            "self_reference",
+            {
+                "self_ref.yml": "data: !reference { path: ./self_ref.yml }",
+            },
+            "self_ref.yml",
+        ),
+        (
+            "triangle_reference",
+            {
+                "file1.yml": "name: File 1\nref: !reference { path: ./file2.yml }",
+                "file2.yml": "name: File 2\nref: !reference { path: ./file3.yml }",
+                "file3.yml": "name: File 3\nref: !reference { path: ./file1.yml }",
+            },
+            "file1.yml",
+        ),
+        (
+            "reference_all_circular",
+            {
+                "main.yml": "data: !reference-all { glob: ./refs/*.yml }",
+                "refs/file1.yml": "name: File 1\nref: !reference { path: ../main.yml }",
+                "refs/file2.yml": "name: File 2",
+            },
+            "main.yml",
+        ),
+        (
+            "reference_all_self",
+            {
+                "main.yml": "data: !reference-all { glob: '*.yml' }",
+                "doc-a.yml": "name: Doc A",
+                "doc-b.yml": "name: Doc B",
+            },
+            "main.yml",
+        ),
+    ],
+)
+def test_circular_reference_detection(stage_files, test_name, files, entry_point):
+    """Test that circular references are detected and disallowed."""
+    stg = stage_files(files)
+
+    with pytest.raises(ValueError, match="Circular reference detected"):
+        load_yaml_with_references(stg / entry_point)

{yaml_reference-2.2.1 → yaml_reference-2.3.0}/yaml_reference/__init__.py

@@ -1,6 +1,6 @@
 import os
 from pathlib import Path
-from typing import Any, Sequence, Union
+from typing import Any, Optional, Sequence, Union
 
 from ruamel.yaml import YAML
 
@@ -148,11 +148,62 @@ def _recursively_attribute_location_to_references(data: Any, base_path: Path):
     return data
 
 
-def _recursively_resolve_references(data: Any, allow_paths: Sequence[Path]) -> Any:
+def _check_and_track_path(path: Path, visited_paths: set[Path]) -> None:
+    """
+    Check for circular reference and add path to visited set.
+
+    Args:
+        path: The file path to check and track.
+        visited_paths: Set of visited file paths.
+
+    Raises:
+        ValueError: If a circular reference is detected.
+    """
+    if path in visited_paths:
+        raise ValueError(
+            f"Circular reference detected: {path} has already been visited. "
+            f"Visited path chain: {visited_paths}"
+        )
+    visited_paths.add(path)
+
+
+def _recursively_resolve_references(
+    data: Any, allow_paths: Sequence[Path], visited_paths: Optional[set[Path]] = None
+) -> Any:
+    """
+    Recursively resolve references in YAML data.
+
+    Args:
+        data: The YAML data to resolve references in.
+        allow_paths: List of allowed paths for file access.
+        visited_paths: Set of file paths that have been visited during resolution.
+                      Used to detect circular references.
+
+    Returns:
+        The resolved YAML data with all references expanded.
+
+    Raises:
+        ValueError: If a circular reference is detected.
+    """
+    if visited_paths is None:
+        visited_paths = set()
+
     if isinstance(data, Reference):
         abs_path = (Path(data.location).parent / data.path).resolve()
+
+        # Check for circular reference and track path
+        _check_and_track_path(abs_path, visited_paths)
+
         parsed = parse_yaml_with_references(abs_path, allow_paths=allow_paths)
-        return _recursively_resolve_references(parsed, allow_paths=allow_paths)
+        resolved = _recursively_resolve_references(
+            parsed, allow_paths=allow_paths, visited_paths=visited_paths
+        )
+
+        # Remove current path from visited set after processing
+        visited_paths.remove(abs_path)
+
+        return resolved
+
     elif isinstance(data, ReferenceAll):
         glob_results = Path(data.location).parent.glob(data.glob)
         abs_paths = [path.resolve() for path in glob_results]
@@ -161,22 +212,35 @@ def _recursively_resolve_references(data: Any, allow_paths: Sequence[Path]) -> A
                 f'No files found matching glob pattern "{data.glob}" in directory "{Path(data.location).parent}"'
             )
         abs_paths = sorted(abs_paths, key=lambda x: str(x))
-        parsed = [
-            parse_yaml_with_references(path, allow_paths=allow_paths)
-            for path in abs_paths
-        ]
-        return [
-            _recursively_resolve_references(item, allow_paths=allow_paths)
-            for item in parsed
-        ]
+
+        resolved_items = []
+        for path in abs_paths:
+            # Check for circular reference and track path
+            _check_and_track_path(path, visited_paths)
+
+            parsed = parse_yaml_with_references(path, allow_paths=allow_paths)
+            resolved = _recursively_resolve_references(
+                parsed, allow_paths=allow_paths, visited_paths=visited_paths
+            )
+            resolved_items.append(resolved)
+
+            # Remove current path from visited set after processing
+            visited_paths.remove(path)
+
+        return resolved_items
+
     elif isinstance(data, list):
         return [
-            _recursively_resolve_references(item, allow_paths=allow_paths)
+            _recursively_resolve_references(
+                item, allow_paths=allow_paths, visited_paths=visited_paths
+            )
             for item in data
         ]
     elif isinstance(data, dict):
         return {
-            key: _recursively_resolve_references(value, allow_paths=allow_paths)
+            key: _recursively_resolve_references(
+                value, allow_paths=allow_paths, visited_paths=visited_paths
+            )
             for key, value in data.items()
         }
     else:
@@ -211,7 +275,15 @@ def load_yaml_with_references(
     allow_paths += [Path(file_path).parent.absolute()]
     path = _check_file_path(file_path, allow_paths=allow_paths)
     parsed = parse_yaml_with_references(path, allow_paths=allow_paths)
-    return _recursively_resolve_references(parsed, allow_paths=allow_paths)  # type: ignore
+
+    # Initialize visited paths with the root file to detect self-references
+    visited_paths = {path.resolve()}
+
+    return _recursively_resolve_references(
+        parsed,
+        allow_paths=allow_paths,  # type: ignore
+        visited_paths=visited_paths,
+    )
 
 
 __all__ = ["parse_yaml_with_references", "load_yaml_with_references"]