liquidrandom 0.3.2__tar.gz → 0.3.4__tar.gz

liquidrandom-0.3.4/IMAGE_GEN.md

@@ -0,0 +1,189 @@
+# liquidrandom - Image as seed data type project
+
+In this project we are using Replicate AI API to generate images as additional seed data type for liquidrandom.
+eg. each sample has an image and text attributes that we can use for the prompting/data generation part, 
+eg. an image and text attributes such as "number_of_people": int, "setting": str,
+
+Instead of having just one big "image" type we want to actually differentiate between different images groups or categories.
+For instance, "humans" as a category of images that include people in natural everyday situations and settings.
+Another category would be areal images of landscapes, cities, etc, eg. from an airplane, drone or satellite.
+
+Your task is to come up with about 5 different categories of images that could be useful for typical on-device vision-language tasks, eg. "animals" is probably not a good category.
+Use the ask-user tool to give me a few options that you think are suitable.
+
+
+Your second task is then to use these categoreis to generate test images using the Replicate API (REPLICATE_API_KEY is defined, not "TOKEN" but "KEY") with the different models shown below.
+This is important since we want to manually check "human-in-the-loop" is the quality for each pair of category X model is acceptable for our use case. We want to make sure that the images generated are of good quality and relevant to the category they belong to.
+
+## Feedback from user about task 2:
+
+- Image generation models product per default "too sterile" images with too much focus on aesthetics, which is not ideal for our use case. We want to have more "natural" or "real" images that are more representative of real-world scenarios. Thus, we need to adjust the prompting to include more instructions about this. Two things: it should be "randomized" such that each "real grounding" applied to the image prompt is not the same for each image, and 2, specific imperfections and other types of making it real. Eg. each image has a "depth of field" with blur for aesthetics but we don't want this, i.e. less cinematic.
+- The dashboards and screenshot images are not include enough instructions to make the image an actual screenshot, not a photo of a dashbaord. Make sure in each prompt in this category we include insturction that this is a screenshot.
+- for the "documents" category, the example is good but the images are too "zoomed in". Include explicit instructions in the prompt to make sure the generated images are more "zoomed out" and include the surrounding environment.
+- The warehouse category looks very "bland" (probably since these models are not trained on a lot of back warehouse images). Thus include explictly the instructions on what should be there, eg. objects or people.
+
+## Feedback from user about task 2 - Part 2:
+We will keep the following categories:
+- Aerial images
+- Indoor (but also cover partially outdoor and outdoor scenes, eg. entrance, etc)
+- People scenes
+- Warehouse
+- Street, but make it not only "urban" but the theme should be "navigation path", eg. also suburban, rual, even a hiking path in the woods or bike path in a park, etc.), indoor navigation as well, eg. a hallway with a navigation path, etc. In essence, it should "Navigation"
+
+We will drop the "documents" and the "dashboards/screenshots" due to high halluciation rate
+
+Your third task is then to adjust the seed data generation process to produce around 10k images in each of the categories using the taxonomy tree approach we used in the text-only seed data generation process.
+
+Your fourth tasks is to visualize these data, again for human-in-the-loop checking.
+
+
+Your fifth tasks is then to adjust the `liquidrandom` public package to support the image seed data types.
+
+```
+import replicate
+
+input = {
+    "prompt": "A striking fashion editorial photograph of a model wearing an elaborate haute couture gown made entirely of living flowers — white peonies, pale pink roses, and trailing jasmine. She stands in a misty English garden at dawn. The dress cascades to the ground and merges with the garden floor. Dewdrops on every petal catch the first light. Her hair is pulled back severely, face serene, skin porcelain. A single monarch butterfly rests on her shoulder. The background is a soft impressionist blur of green hedgerows and pale sky. Shot on Phase One IQ4 150MP, every stamen and pistil razor sharp.",
+    "aspect_ratio": "2:3"
+}
+
+output = replicate.run(
+    "bytedance/seedream-5-lite",
+    input=input
+)
+
+# To access the file URLs:
+print(output[0].url)
+#=> "https://replicate.delivery/.../output_0.png"
+
+# To write the files to disk:
+for index, item in enumerate(output):
+    with open(f"output_{index}.png", "wb") as file:
+        file.write(item.read())
+#=> output_0.png written to disk
+```
+
+```
+import replicate
+
+input = {
+    "prompt": "Bookstore window display. A sign displays “New Arrivals This Week”. Below, a shelf tag with the text “Best-Selling Novels Here”. To the side, a colorful poster advertises “Author Meet And Greet on Saturday” with a central portrait of the author. There are four books on the bookshelf, namely “The light between worlds” “When stars are scattered” “The slient patient” “The night circus”",
+    "guidance": 4,
+    "num_inference_steps": 50
+}
+
+output = replicate.run(
+    "qwen/qwen-image",
+    input=input
+)
+
+# To access the file URLs:
+print(output[0].url)
+#=> "https://replicate.delivery/.../output_0.webp"
+
+# To write the files to disk:
+for index, item in enumerate(output):
+    with open(f"output_{index}.webp", "wb") as file:
+        file.write(item.read())
+#=> output_0.webp written to disk
+```
+
+
+```
+import replicate
+
+input = {
+    "size": "4K",
+    "prompt": "A warm, nostalgic film-style interior of a cozy café, shot on 35mm-inspired digital photography with soft afternoon sunlight filtering through the front windows. Wooden shelves display neatly arranged ceramics, pastries, and coffee beans. Hand-painted signage on the main interior window reads ‘Seedream 4.5’ in clean, classic lettering, similar to boutique branding. A vintage bicycle with a wicker basket is visible outside the entrance, casting soft shadows on the floor. Rich textures, natural light, warm tones, subtle grain, and calm neighborhood-café ambiance.",
+    "aspect_ratio": "16:9"
+}
+
+output = replicate.run(
+    "bytedance/seedream-4.5",
+    input=input
+)
+
+# To access the file URLs:
+print(output[0].url)
+#=> "https://replicate.delivery/.../output_0.jpg"
+
+# To write the files to disk:
+for index, item in enumerate(output):
+    with open(f"output_{index}.jpg", "wb") as file:
+        file.write(item.read())
+#=> output_0.jpg written to disk
+```
+
+```
+import replicate
+
+input = {
+    "prompt": "A close-up portrait of a leopard with distinctive spotted pattern and intense eyes",
+    "aspect_ratio": "3:4"
+}
+
+output = replicate.run(
+    "minimax/image-01",
+    input=input
+)
+
+# To access the file URLs:
+print(output[0].url)
+#=> "https://replicate.delivery/.../output_0.jpeg"
+
+# To write the files to disk:
+for index, item in enumerate(output):
+    with open(f"output_{index}.jpeg", "wb") as file:
+        file.write(item.read())
+#=> output_0.jpeg written to disk
+```
+
+
+```
+import replicate
+
+input = {
+    "prompt": "An illustration of a pink and brown butterfly on a pink background, with the black text “Hunyuan Image 3 on Replicate” below"
+}
+
+output = replicate.run(
+    "tencent/hunyuan-image-3",
+    input=input
+)
+
+# To access the file URLs:
+print(output[0].url)
+#=> "https://replicate.delivery/.../output_0.webp"
+
+# To write the files to disk:
+for index, item in enumerate(output):
+    with open(f"output_{index}.webp", "wb") as file:
+        file.write(item.read())
+#=> output_0.webp written to disk
+```
+
+
+```
+import replicate
+
+input = {
+    "prompt": "Replace the color of the car to blue",
+    "aspect_ratio": "match_input_image",
+    "input_images": ["https://replicate.delivery/pbxt/O7kbtH7wgLIItlCyeWjm0fgPpx7OpGsT9VbYyxBXnfieVxQe/woman-by-car.jpg"],
+    "output_format": "jpg"
+}
+
+output = replicate.run(
+    "black-forest-labs/flux-2-dev",
+    input=input
+)
+
+# To access the file URL:
+print(output.url)
+#=> "https://replicate.delivery/.../output.jpeg"
+
+# To write the file to disk:
+with open("output.jpeg", "wb") as file:
+    file.write(output.read())
+#=> output.jpeg written to disk
+```

{liquidrandom-0.3.2 → liquidrandom-0.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: liquidrandom
-Version: 0.3.2
+Version: 0.3.4
 Summary: Pseudo-random seed data generation for ML/LLM training diversity
 License-Expression: MIT
 Requires-Python: >=3.11

{liquidrandom-0.3.2 → liquidrandom-0.3.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "liquidrandom"
-version = "0.3.2"
+version = "0.3.4"
 description = "Pseudo-random seed data generation for ML/LLM training diversity"
 readme = "README.md"
 requires-python = ">=3.11"

{liquidrandom-0.3.2 → liquidrandom-0.3.4}/seed_generation/categories.py

@@ -503,4 +503,44 @@ CATEGORY_CONFIGS: dict[str, CategoryConfig] = {
         generation_prompt_template="",  # Not used; tool_sampler has its own prompts
         specificity_guidance="Each tool group should represent a coherent set of related API functions with inter-tool dependencies.",
     ),
+    "physical_tool_group": CategoryConfig(
+        name="physical_tool_group",
+        display_name="Physical-AI Tool Groups",
+        fields=[
+            FieldSpec("domain", "string", "Tool group domain (e.g. 'Smart Home Lighting Control')"),
+            FieldSpec("description", "string", "What this tool group does"),
+            FieldSpec("taxonomy_path", "string", "Taxonomy path this group belongs to"),
+            FieldSpec("tools_json", "string", "JSON-serialized list of ToolFunction dicts"),
+        ],
+        taxonomy_seed_prompt=(
+            "Generate a taxonomy of tool/function calling domains for PHYSICAL and EMBODIED "
+            "AI systems (not SaaS or cloud APIs). These represent groups of related tools that "
+            "an embodied agent — a voice assistant, robot, vehicle, or wearable AI — would use "
+            "together to sense, act on, and reason about the physical world.\n\n"
+            "Root categories:\n"
+            "- Smart Home Voice Assistants (lighting, climate, security, appliances, media & entertainment)\n"
+            "- Mobile Service Robots (indoor delivery, floor cleaning, facility inspection)\n"
+            "- Manipulator Robots (arm control, grasping, assembly, pick-and-place, sorting)\n"
+            "- Humanoid Robots (bipedal locomotion, whole-body manipulation, human interaction)\n"
+            "- Autonomous Vehicles (perception, motion planning, vehicle control, V2X communication)\n"
+            "- Sidewalk & Last-Mile Delivery Robots (navigation, obstacle handling, handoff)\n"
+            "- Aerial Drones (mission planning, inspection, cinematography, precision agriculture)\n"
+            "- Underwater & Marine Robots (ROV/AUV operations, bathymetric mapping)\n"
+            "- Industrial & Factory Automation (robotic cells, AMRs, conveyor & PLC control)\n"
+            "- Agricultural Robotics (field navigation, harvesting, targeted spraying)\n"
+            "- Medical & Surgical Robots (teleoperation, imaging-guided intervention)\n"
+            "- Wearable & Personal AI (AR glasses assistants, smart earbuds, fitness devices)\n\n"
+            "Each leaf should be a specific embodied tool domain (e.g. 'Quadruped locomotion "
+            "gait switching on uneven terrain', not just 'robotics'). Focus on real physical "
+            "capabilities: sensing, actuation, state estimation, planning, safety — NOT cloud "
+            "services, databases, or web APIs."
+        ),
+        generation_prompt_template="",  # Not used; tool_sampler has its own prompts
+        specificity_guidance=(
+            "Each tool group should represent a coherent set of physical-world capabilities "
+            "(sensing + actuation + state) that an embodied AI agent would use together, with "
+            "inter-tool state/data dependencies (e.g. a pose returned by a localization tool is "
+            "consumed by a navigation tool, or a detected object handle flows into a grasp planner)."
+        ),
+    ),
 }

{liquidrandom-0.3.2 → liquidrandom-0.3.4}/seed_generation/generate.py

@@ -26,6 +26,16 @@ from config import (
     DEFAULT_TOOL_TAXONOMY_DEPTH,
     STATE_FILE,
 )
+from image_categories import IMAGE_CATEGORY_CONFIGS
+from image_config import (
+    DEFAULT_IMAGE_BATCH_SIZE,
+    DEFAULT_IMAGE_CONCURRENCY,
+    DEFAULT_IMAGE_DEDUP_THRESHOLD,
+    DEFAULT_IMAGE_K,
+    DEFAULT_IMAGE_N,
+    DEFAULT_IMAGE_SAMPLES_PER_LEAF,
+    DEFAULT_IMAGE_TAXONOMY_DEPTH,
+)
 from llm import create_client
 from sampler import generate_samples
 from state import RunState
@@ -349,7 +359,7 @@ def generate_tools(
     output_dir: str = typer.Option("output", "--output-dir", help="Output directory"),
 ) -> None:
     """Generate tool group seed data with parameter variations."""
-    resolved = categories if categories else ["tool_group"]
+    resolved = categories if categories else ["tool_group", "physical_tool_group"]
     for cat in resolved:
         if cat not in CATEGORY_CONFIGS:
             console.print(f"[red]Unknown category: {cat}[/red]")
@@ -370,15 +380,218 @@ def generate_tools(
     )
 
 
+async def _run_generate_images(
+    categories: list[str],
+    n: int,
+    k: int,
+    batch_size: int,
+    image_concurrency: int,
+    taxonomy_depth: int,
+    samples_per_leaf: int,
+    dedup_threshold: float,
+    output_dir: str,
+    resume: bool,
+) -> None:
+    from image_sampler import generate_image_samples
+
+    client = create_client()
+    state_path = str(Path(output_dir) / "state.json")
+    # Always load existing state to preserve progress from previous runs
+    state = RunState.load(state_path)
+
+    results: dict[str, dict[str, int | float]] = {}
+    total_start = time.time()
+
+    console.print(f"\n[bold]Generating image seed data for {len(categories)} categories[/bold]")
+    console.print(f"  Target: {n} images per category, {k} prompts per LLM call")
+    console.print(f"  Taxonomy: depth {taxonomy_depth}, {samples_per_leaf} images/leaf")
+    console.print(f"  Image concurrency: {image_concurrency}, dedup threshold: {dedup_threshold}")
+    console.print()
+
+    for i, cat_name in enumerate(categories, 1):
+        cat_config = IMAGE_CATEGORY_CONFIGS[cat_name]
+        cat_state = state.get_category(cat_name)
+        cat_start = time.time()
+
+        console.print(
+            f"[bold cyan][{i}/{len(categories)}] {cat_config.display_name}[/bold cyan]"
+        )
+
+        # Phase 1: Taxonomy (reuses existing taxonomy.py)
+        if not cat_state.taxonomy_done or not resume:
+            try:
+                root = await generate_taxonomy(
+                    client,
+                    cat_config,
+                    target_samples=n,
+                    samples_per_leaf=samples_per_leaf,
+                    max_depth=taxonomy_depth,
+                    batch_size=batch_size,
+                    output_dir=output_dir,
+                )
+                cat_state.taxonomy_done = True
+                state.save(state_path)
+            except Exception as e:
+                console.print(f"  [red]Taxonomy generation failed: {e}[/red]")
+                continue
+        else:
+            from taxonomy import load_taxonomy
+            root = load_taxonomy(output_dir, cat_name)
+            if root is None:
+                console.print(f"  [red]No saved taxonomy found, regenerating[/red]")
+                cat_state.taxonomy_done = False
+                continue
+
+        # Phase 2+3: Image prompt generation + Replicate image generation
+        if not cat_state.generation_done or not resume:
+            try:
+                total = await generate_image_samples(
+                    client,
+                    root,
+                    cat_config,
+                    target_samples=n,
+                    k=k,
+                    batch_size=batch_size,
+                    image_concurrency=image_concurrency,
+                    dedup_threshold=dedup_threshold,
+                    output_dir=output_dir,
+                )
+                # Only mark done if we reached the target
+                cat_state.generation_done = total >= n
+                cat_state.total_samples = total
+                state.save(state_path)
+            except KeyboardInterrupt:
+                console.print("\n  [yellow]Interrupted! Saving state...[/yellow]")
+                state.save(state_path)
+                console.print(
+                    f"  [yellow]Resume with: python generate.py generate-images --resume "
+                    f"--output-dir {output_dir}[/yellow]"
+                )
+                raise
+            except Exception as e:
+                console.print(f"  [red]Image generation failed: {e}[/red]")
+                state.save(state_path)
+                continue
+        else:
+            total = cat_state.total_samples
+            console.print(
+                f"  [green]Already complete ({total} images)[/green]"
+            )
+
+        elapsed = time.time() - cat_start
+        results[cat_name] = {"total": total, "time": elapsed}
+        console.print()
+
+    # Summary table
+    total_elapsed = time.time() - total_start
+    table = Table(title="Image Generation Summary")
+    table.add_column("Category", style="cyan")
+    table.add_column("Target", justify="right")
+    table.add_column("Generated", justify="right")
+    table.add_column("Time", justify="right")
+
+    for cat_name in categories:
+        if cat_name in results:
+            r = results[cat_name]
+            table.add_row(
+                IMAGE_CATEGORY_CONFIGS[cat_name].display_name,
+                str(n),
+                str(r["total"]),
+                f"{r['time']:.1f}s",
+            )
+
+    console.print(table)
+    console.print(f"\n[bold]Total time: {total_elapsed:.1f}s[/bold]")
+
+
+@app.command(name="generate-images")
+def generate_images(
+    n: int = typer.Option(DEFAULT_IMAGE_N, "--n", help="Target images per category"),
+    k: int = typer.Option(DEFAULT_IMAGE_K, "--k", help="Prompts per LLM call"),
+    batch_size: int = typer.Option(
+        DEFAULT_IMAGE_BATCH_SIZE, "--batch-size", help="Concurrent LLM calls"
+    ),
+    image_concurrency: int = typer.Option(
+        DEFAULT_IMAGE_CONCURRENCY, "--image-concurrency", help="Concurrent Replicate API calls"
+    ),
+    taxonomy_depth: int = typer.Option(
+        DEFAULT_IMAGE_TAXONOMY_DEPTH, "--taxonomy-depth", help="Max taxonomy tree depth"
+    ),
+    samples_per_leaf: int = typer.Option(
+        DEFAULT_IMAGE_SAMPLES_PER_LEAF, "--samples-per-leaf", help="Target images per leaf"
+    ),
+    dedup_threshold: float = typer.Option(
+        DEFAULT_IMAGE_DEDUP_THRESHOLD, "--dedup-threshold", help="Jaccard similarity threshold"
+    ),
+    categories: Optional[list[str]] = typer.Option(
+        None, "--categories", help="Image categories to generate (default: all)"
+    ),
+    resume: bool = typer.Option(False, "--resume", help="Resume from checkpoint"),
+    output_dir: str = typer.Option("output", "--output-dir", help="Output directory"),
+) -> None:
+    """Generate image seed data for all or selected image categories."""
+    if not categories:
+        resolved = list(IMAGE_CATEGORY_CONFIGS.keys())
+    else:
+        for cat in categories:
+            if cat not in IMAGE_CATEGORY_CONFIGS:
+                console.print(f"[red]Unknown image category: {cat}[/red]")
+                console.print(
+                    f"Available: {', '.join(sorted(IMAGE_CATEGORY_CONFIGS.keys()))}"
+                )
+                raise typer.Exit(1)
+        resolved = categories
+    asyncio.run(
+        _run_generate_images(
+            categories=resolved,
+            n=n,
+            k=k,
+            batch_size=batch_size,
+            image_concurrency=image_concurrency,
+            taxonomy_depth=taxonomy_depth,
+            samples_per_leaf=samples_per_leaf,
+            dedup_threshold=dedup_threshold,
+            output_dir=output_dir,
+            resume=resume,
+        )
+    )
+
+
+@app.command(name="review-images")
+def review_images(
+    category: str = typer.Argument(help="Image category to review"),
+    max_images: int = typer.Option(500, "--max-images", help="Max images to include in gallery"),
+    output_dir: str = typer.Option("output", "--output-dir", help="Output directory"),
+) -> None:
+    """Generate an HTML gallery page for reviewing generated images."""
+    from image_viewer import generate_review_html
+
+    if category not in IMAGE_CATEGORY_CONFIGS:
+        console.print(f"[red]Unknown image category: {category}[/red]")
+        console.print(
+            f"Available: {', '.join(sorted(IMAGE_CATEGORY_CONFIGS.keys()))}"
+        )
+        raise typer.Exit(1)
+    path = generate_review_html(output_dir, category, max_images=max_images)
+    console.print(f"\n[bold]Open in browser: file://{path.resolve()}[/bold]")
+
+
 @app.command(name="upload-only")
 def upload_only(
     output_dir: str = typer.Option("output", "--output-dir", help="Output directory"),
     repo_id: str = typer.Option(
         "mlech26l/liquidrandom-data", "--repo-id", help="HuggingFace repo ID"
     ),
+    skip_images: bool = typer.Option(
+        False,
+        "--skip-images",
+        help="Skip image category consolidation/upload (any existing remote image parquets stay untouched; README will only list text categories)",
+    ),
 ) -> None:
     """Consolidate samples and upload to HuggingFace."""
-    consolidate_and_upload(output_dir=output_dir, repo_id=repo_id)
+    consolidate_and_upload(
+        output_dir=output_dir, repo_id=repo_id, skip_images=skip_images
+    )
 
 
 if __name__ == "__main__":