nextrec 0.4.14__tar.gz → 0.4.16__tar.gz

{nextrec-0.4.14 → nextrec-0.4.16}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nextrec
-Version: 0.4.14
+Version: 0.4.16
 Summary: A comprehensive recommendation library with match, ranking, and multi-task learning models
 Project-URL: Homepage, https://github.com/zerolovesea/NextRec
 Project-URL: Repository, https://github.com/zerolovesea/NextRec
@@ -63,10 +63,13 @@ Description-Content-Type: text/markdown
 
 <div align="center">
 
+[![PyPI Downloads](https://static.pepy.tech/personalized-badge/nextrec?period=total&units=NONE&left_color=BLACK&right_color=GREEN&left_text=PyPI-downloads)](https://pepy.tech/projects/nextrec)
 ![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)
 ![PyTorch](https://img.shields.io/badge/PyTorch-1.10+-ee4c2c.svg)
+
 ![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)
-![Version](https://img.shields.io/badge/Version-0.4.14-orange.svg)
+![Version](https://img.shields.io/badge/Version-0.4.16-orange.svg)
+
 
 中文文档 | [English Version](README_en.md)
 
@@ -99,7 +102,7 @@ NextRec是一个基于PyTorch的现代推荐系统框架，旨在为研究工程
 
 ## NextRec近期进展
 
-- **21/12/2025** 在v0.4.14中加入了对[GradNorm](/nextrec/loss/grad_norm.py)的支持，通过compile的`loss_weight='grad_norm'`进行配置
+- **21/12/2025** 在v0.4.16中加入了对[GradNorm](/nextrec/loss/grad_norm.py)的支持，通过compile的`loss_weight='grad_norm'`进行配置
 - **12/12/2025** 在v0.4.9中加入了[RQ-VAE](/nextrec/models/representation/rqvae.py)模块。配套的[数据集](/dataset/ecommerce_task.csv)和[代码](tutorials/notebooks/zh/使用RQ-VAE构建语义ID.ipynb)已经同步在仓库中
 - **07/12/2025** 发布了NextRec CLI命令行工具，它允许用户根据配置文件进行一键训练和推理，我们提供了相关的[教程](/nextrec_cli_preset/NextRec-CLI_zh.md)和[教学代码](/nextrec_cli_preset)
 - **03/12/2025** NextRec获得了100颗🌟！感谢大家的支持
@@ -241,11 +244,11 @@ nextrec --mode=train --train_config=path/to/train_config.yaml
 nextrec --mode=predict --predict_config=path/to/predict_config.yaml
 ```
 
-> 截止当前版本0.4.14，NextRec CLI支持单机训练，分布式训练相关功能尚在开发中。
+> 截止当前版本0.4.16，NextRec CLI支持单机训练，分布式训练相关功能尚在开发中。
 
 ## 兼容平台
 
-当前最新版本为0.4.14，所有模型和测试代码均已在以下平台通过验证，如果开发者在使用中遇到兼容问题，请在issue区提出错误报告及系统版本：
+当前最新版本为0.4.16，所有模型和测试代码均已在以下平台通过验证，如果开发者在使用中遇到兼容问题，请在issue区提出错误报告及系统版本：
 
 | 平台 | 配置 | 
 |------|------|

{nextrec-0.4.14 → nextrec-0.4.16}/README.md

@@ -4,10 +4,13 @@
 
 <div align="center">
 
+[![PyPI Downloads](https://static.pepy.tech/personalized-badge/nextrec?period=total&units=NONE&left_color=BLACK&right_color=GREEN&left_text=PyPI-downloads)](https://pepy.tech/projects/nextrec)
 ![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)
 ![PyTorch](https://img.shields.io/badge/PyTorch-1.10+-ee4c2c.svg)
+
 ![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)
-![Version](https://img.shields.io/badge/Version-0.4.14-orange.svg)
+![Version](https://img.shields.io/badge/Version-0.4.16-orange.svg)
+
 
 中文文档 | [English Version](README_en.md)
 
@@ -40,7 +43,7 @@ NextRec是一个基于PyTorch的现代推荐系统框架，旨在为研究工程
 
 ## NextRec近期进展
 
-- **21/12/2025** 在v0.4.14中加入了对[GradNorm](/nextrec/loss/grad_norm.py)的支持，通过compile的`loss_weight='grad_norm'`进行配置
+- **21/12/2025** 在v0.4.16中加入了对[GradNorm](/nextrec/loss/grad_norm.py)的支持，通过compile的`loss_weight='grad_norm'`进行配置
 - **12/12/2025** 在v0.4.9中加入了[RQ-VAE](/nextrec/models/representation/rqvae.py)模块。配套的[数据集](/dataset/ecommerce_task.csv)和[代码](tutorials/notebooks/zh/使用RQ-VAE构建语义ID.ipynb)已经同步在仓库中
 - **07/12/2025** 发布了NextRec CLI命令行工具，它允许用户根据配置文件进行一键训练和推理，我们提供了相关的[教程](/nextrec_cli_preset/NextRec-CLI_zh.md)和[教学代码](/nextrec_cli_preset)
 - **03/12/2025** NextRec获得了100颗🌟！感谢大家的支持
@@ -182,11 +185,11 @@ nextrec --mode=train --train_config=path/to/train_config.yaml
 nextrec --mode=predict --predict_config=path/to/predict_config.yaml
 ```
 
-> 截止当前版本0.4.14，NextRec CLI支持单机训练，分布式训练相关功能尚在开发中。
+> 截止当前版本0.4.16，NextRec CLI支持单机训练，分布式训练相关功能尚在开发中。
 
 ## 兼容平台
 
-当前最新版本为0.4.14，所有模型和测试代码均已在以下平台通过验证，如果开发者在使用中遇到兼容问题，请在issue区提出错误报告及系统版本：
+当前最新版本为0.4.16，所有模型和测试代码均已在以下平台通过验证，如果开发者在使用中遇到兼容问题，请在issue区提出错误报告及系统版本：
 
 | 平台 | 配置 | 
 |------|------|

{nextrec-0.4.14 → nextrec-0.4.16}/README_en.md

@@ -4,10 +4,12 @@
 
 <div align="center">
 
+[![PyPI Downloads](https://static.pepy.tech/personalized-badge/nextrec?period=total&units=NONE&left_color=BLACK&right_color=GREEN&left_text=PyPI-downloads)](https://pepy.tech/projects/nextrec)
 ![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)
 ![PyTorch](https://img.shields.io/badge/PyTorch-1.10+-ee4c2c.svg)
+
 ![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)
-![Version](https://img.shields.io/badge/Version-0.4.14-orange.svg)
+![Version](https://img.shields.io/badge/Version-0.4.16-orange.svg)
 
 English | [中文文档](README.md)
 
@@ -42,7 +44,7 @@ NextRec is a modern recommendation framework built on PyTorch, delivering a unif
 
 ## NextRec Progress
 
-- **21/12/2025** Added support for [GradNorm](/nextrec/loss/grad_norm.py) in v0.4.14, configurable via `loss_weight='grad_norm'` in the compile method
+- **21/12/2025** Added support for [GradNorm](/nextrec/loss/grad_norm.py) in v0.4.16, configurable via `loss_weight='grad_norm'` in the compile method
 - **12/12/2025** Added [RQ-VAE](/nextrec/models/representation/rqvae.py), a common module for generative retrieval in v0.4.9. Paired [dataset](/dataset/ecommerce_task.csv) and [notebook code](tutorials/notebooks/en/Build%20semantic%20ID%20with%20RQ-VAE.ipynb) are available.
 - **07/12/2025** Released the NextRec CLI tool to run training/inference from configs. See the [guide](/nextrec_cli_preset/NextRec-CLI.md) and [reference code](/nextrec_cli_preset).
 - **03/12/2025** NextRec reached 100 ⭐—thanks for the support!
@@ -186,11 +188,11 @@ nextrec --mode=train --train_config=path/to/train_config.yaml
 nextrec --mode=predict --predict_config=path/to/predict_config.yaml
 ```
 
-> As of version 0.4.14, NextRec CLI supports single-machine training; distributed training features are currently under development.
+> As of version 0.4.16, NextRec CLI supports single-machine training; distributed training features are currently under development.
 
 ## Platform Compatibility
 
-The current version is 0.4.14. All models and test code have been validated on the following platforms. If you encounter compatibility issues, please report them in the issue tracker with your system version:
+The current version is 0.4.16. All models and test code have been validated on the following platforms. If you encounter compatibility issues, please report them in the issue tracker with your system version:
 
 | Platform | Configuration | 
 |----------|---------------|

{nextrec-0.4.14 → nextrec-0.4.16}/docs/rtd/conf.py

@@ -11,7 +11,7 @@ sys.path.insert(0, str(PROJECT_ROOT / "nextrec"))
 project = "NextRec"
 copyright = "2025, Yang Zhou"
 author = "Yang Zhou"
-release = "0.4.14"
+release = "0.4.16"
 
 extensions = [
     "myst_parser",

nextrec-0.4.16/nextrec/__version__.py

@@ -0,0 +1 @@
+__version__ = "0.4.16"

{nextrec-0.4.14 → nextrec-0.4.16}/nextrec/basic/metrics.py

@@ -77,6 +77,8 @@ def check_user_id(*metric_sources: Any) -> bool:
 
 def compute_ks(y_true: np.ndarray, y_pred: np.ndarray) -> float:
     """Compute Kolmogorov-Smirnov statistic."""
+    y_true = np.asarray(y_true).reshape(-1)
+    y_pred = np.asarray(y_pred).reshape(-1)
     sorted_indices = np.argsort(y_pred)[::-1]
     y_true_sorted = y_true[sorted_indices]
 

{nextrec-0.4.14 → nextrec-0.4.16}/nextrec/basic/model.py

@@ -481,7 +481,7 @@ class BaseModel(FeatureSet, nn.Module):
                         "[BaseModel-compile Error] loss_weights list must have exactly one element for single-task setup."
                     )
                 loss_weights = loss_weights[0]
-            self.loss_weights = [float(loss_weights)] # type: ignore
+            self.loss_weights = [float(loss_weights)]  # type: ignore
         else:
             if isinstance(loss_weights, (int, float)):
                 weights = [float(loss_weights)] * self.nums_task
@@ -591,8 +591,8 @@ class BaseModel(FeatureSet, nn.Module):
 
     def fit(
         self,
-        train_data = None,
-        valid_data = None,
+        train_data=None,
+        valid_data=None,
         metrics: (
             list[str] | dict[str, list[str]] | None
         ) = None,  # ['auc', 'logloss'] or {'target1': ['auc', 'logloss'], 'target2': ['mse']}
@@ -1420,6 +1420,11 @@ class BaseModel(FeatureSet, nn.Module):
         # Create DataLoader based on data type
         if isinstance(data, DataLoader):
             data_loader = data
+            if num_workers != 0:
+                logging.warning(
+                    "[Predict Warning] num_workers parameter is ignored when data is already a DataLoader. "
+                    "The DataLoader's existing num_workers configuration will be used."
+                )
         elif isinstance(data, (str, os.PathLike)):
             rec_loader = RecDataLoader(
                 dense_features=self.dense_features,
@@ -1578,6 +1583,17 @@ class BaseModel(FeatureSet, nn.Module):
         else:
             data_loader = data
 
+            if hasattr(data_loader, "num_workers") and data_loader.num_workers > 0:
+                if (
+                    hasattr(data_loader.dataset, "__class__")
+                    and "Streaming" in data_loader.dataset.__class__.__name__
+                ):
+                    logging.warning(
+                        f"[Predict Streaming Warning] Detected DataLoader with num_workers={data_loader.num_workers} "
+                        "and streaming dataset. This may cause data duplication! "
+                        "When using streaming mode, set num_workers=0 to avoid reading data multiple times."
+                    )
+
         suffix = ".csv" if save_format == "csv" else ".parquet"
         target_path = resolve_save_path(
             path=save_path,

{nextrec-0.4.14 → nextrec-0.4.16}/nextrec/cli.py

@@ -21,6 +21,7 @@ Author: Yang Zhou, zyaztec@gmail.com
 import argparse
 import logging
 import pickle
+import resource
 import sys
 import time
 from pathlib import Path
@@ -625,6 +626,17 @@ def predict_model(predict_config_path: str) -> None:
 def main() -> None:
     """Parse CLI arguments and dispatch to train or predict mode."""
 
+    # Increase file descriptor limit to avoid "Too many open files" error
+    # when using DataLoader with multiple workers
+    try:
+        soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
+        target_limit = 65535
+        if soft < target_limit:
+            resource.setrlimit(resource.RLIMIT_NOFILE, (min(target_limit, hard), hard))
+    except (ValueError, OSError):
+        # If we can't set the limit, continue anyway
+        pass
+
     root = logging.getLogger()
     if not root.handlers:
         handler = logging.StreamHandler(sys.stdout)

{nextrec-0.4.14 → nextrec-0.4.16}/nextrec/data/dataloader.py

@@ -210,6 +210,15 @@ class RecDataLoader(FeatureSet):
             DataLoader instance.
         """
 
+        # Enforce num_workers=0 for streaming mode to prevent data duplication
+        if streaming and num_workers > 0:
+            logging.warning(
+                f"[RecDataLoader Warning] num_workers={num_workers} is not compatible with streaming=True. "
+                "Each worker would create its own data stream, causing data duplication. "
+                "Forcing num_workers=0."
+            )
+            num_workers = 0
+
         if isinstance(data, DataLoader):
             return data
         elif isinstance(data, (str, os.PathLike)):
@@ -363,6 +372,13 @@ class RecDataLoader(FeatureSet):
             logging.info(
                 "[RecDataLoader Info] Streaming mode enforces batch_size=1; tune chunk_size to control memory/throughput."
             )
+        if num_workers > 0:
+            logging.warning(
+                f"[RecDataLoader Warning] num_workers={num_workers} is not compatible with streaming mode. "
+                "Each worker would create its own data stream, causing data duplication. "
+                "Forcing num_workers=0."
+            )
+            num_workers = 0
         dataset = FileDataset(
             file_paths=file_paths,
             dense_features=self.dense_features,

{nextrec-0.4.14 → nextrec-0.4.16}/nextrec/utils/config.py

@@ -28,7 +28,9 @@ if TYPE_CHECKING:
     from nextrec.data.preprocessor import DataProcessor
 
 
-def resolve_path(path_str: str | Path | None = None, base_dir: Path | None = None) -> Path:
+def resolve_path(
+    path_str: str | Path | None = None, base_dir: Path | None = None
+) -> Path:
     if path_str is None:
         return Path.cwd()
     path = Path(path_str).expanduser()
@@ -36,9 +38,16 @@ def resolve_path(path_str: str | Path | None = None, base_dir: Path | None = Non
         return path
     # Prefer resolving relative to current working directory when the path (or its parent)
     # already exists there; otherwise fall back to the config file's directory.
-    candidates = ((Path.cwd() / path).resolve(), ((base_dir or Path.cwd()) / path).resolve())
+    candidates = (
+        (Path.cwd() / path).resolve(),
+        ((base_dir or Path.cwd()) / path).resolve(),
+    )
     return next(
-        (candidate for candidate in candidates if candidate.exists() or candidate.parent.exists()),
+        (
+            candidate
+            for candidate in candidates
+            if candidate.exists() or candidate.parent.exists()
+        ),
         candidates[0],
     )
 

{nextrec-0.4.14 → nextrec-0.4.16}/nextrec/utils/console.py

@@ -16,8 +16,9 @@ import numbers
 import os
 import platform
 import sys
+import time
 from datetime import datetime, timedelta
-from typing import Any, Callable, Iterable, Mapping, TypeVar
+from typing import Any, Callable, Mapping, TypeVar
 
 import numpy as np
 from rich import box
@@ -128,45 +129,85 @@ class BlackMofNCompleteColumn(MofNCompleteColumn):
         )
 
 
-def progress(
-    iterable: Iterable[T],
-    *,
-    description: str | None = None,
-    total: int | None = None,
-    disable: bool = False,
-) -> Iterable[T]:
+def progress(iterable, *, description=None, total=None, disable=False):
     if disable:
-        for item in iterable:
-            yield item
+        yield from iterable
         return
+
     resolved_total = total
     if resolved_total is None:
         try:
-            resolved_total = len(iterable)  # type: ignore[arg-type]
+            resolved_total = len(iterable)
         except TypeError:
             resolved_total = None
 
+    stream = sys.stderr
+
+    if not stream.isatty():
+        start_time = time.monotonic()
+        last_tick = start_time
+        min_interval_seconds = 10.0
+        max_interval_seconds = 300.0
+        target_steps = (
+            max(1, resolved_total // 20) if resolved_total is not None else 500
+        )
+        interval_seconds = min_interval_seconds
+        completed = 0
+
+        def emit(now: float):
+            elapsed = max(0.0, now - start_time)
+            speed = completed / elapsed if elapsed > 0 else 0.0
+            if resolved_total is not None and speed > 0:
+                remaining = max(0.0, resolved_total - completed)
+                eta_seconds = remaining / speed
+                eta_text = str(timedelta(seconds=int(eta_seconds)))
+            else:
+                eta_text = "--:--:--"
+            total_text = str(resolved_total) if resolved_total is not None else "?"
+            stream.write(
+                f"{description or 'Working'}: {completed}/{total_text} "
+                f"elapsed={timedelta(seconds=int(elapsed))} "
+                f"speed={speed:.2f}/s ETA={eta_text}\n"
+            )
+            stream.flush()
+            return speed
+
+        for item in iterable:
+            yield item
+            completed += 1
+            now = time.monotonic()
+            if now - last_tick >= interval_seconds:
+                speed = emit(now)
+                last_tick = now
+                if speed > 0:
+                    interval_seconds = min(
+                        max_interval_seconds,
+                        max(min_interval_seconds, target_steps / speed),
+                    )
+        end_now = time.monotonic()
+        if end_now - last_tick >= 1e-6:
+            emit(end_now)
+        return
+
+    # TTY: rich
+    console = Console(file=stream, force_terminal=True)
     progress_bar = Progress(
-        SpinnerColumn(style="black"),
-        TextColumn("{task.description}", style="black"),
-        BarColumn(
-            bar_width=36, style="black", complete_style="black", finished_style="black"
-        ),
-        TaskProgressColumn(style="black"),
-        BlackMofNCompleteColumn(),
-        BlackTimeElapsedColumn(),
-        BlackTimeRemainingColumn(),
+        SpinnerColumn(),
+        TextColumn("{task.description}"),
+        BarColumn(bar_width=36),
+        TaskProgressColumn(),
+        MofNCompleteColumn(),
+        TimeElapsedColumn(),
+        TimeRemainingColumn(),
         refresh_per_second=12,
+        console=console,
     )
 
-    task_id = progress_bar.add_task(description or "Working", total=resolved_total)
-    progress_bar.start()
-    try:
+    with progress_bar:
+        task_id = progress_bar.add_task(description or "Working", total=resolved_total)
         for item in iterable:
             yield item
             progress_bar.advance(task_id, 1)
-    finally:
-        progress_bar.stop()
 
 
 def group_metrics_by_task(

{nextrec-0.4.14 → nextrec-0.4.16}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "nextrec"
-version = "0.4.14"
+version = "0.4.16"
 description = "A comprehensive recommendation library with match, ranking, and multi-task learning models"
 readme = "README.md"
 requires-python = ">=3.10"

nextrec-0.4.14/nextrec/__version__.py

@@ -1 +0,0 @@
-__version__ = "0.4.14"