rdkit-cli 0.1.0__tar.gz → 0.2.0__tar.gz

{rdkit_cli-0.1.0 → rdkit_cli-0.2.0}/CHANGELOG.md

@@ -5,6 +5,21 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.0] - 2026-01-06
+
+### Added
+
+- **stats**: Calculate dataset statistics (MolWt, LogP, TPSA, etc. with min/max/mean/median/stdev)
+- **split**: Split files into smaller chunks (by number of chunks or chunk size)
+- **sample**: Randomly sample molecules (by count or fraction, with reservoir sampling for large files)
+- **deduplicate**: Remove duplicate molecules (by SMILES, InChI, InChIKey, or scaffold)
+- **validate**: Validate molecular structures (valence, kekulization, stereo, element constraints)
+
+### Changed
+
+- Commands are now displayed in alphabetical order in help output
+- Total command count increased from 14 to 19
+
 ## [0.1.0] - 2026-01-06
 
 ### Added

{rdkit_cli-0.1.0 → rdkit_cli-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rdkit-cli
-Version: 0.1.0
+Version: 0.2.0
 Summary: A comprehensive CLI tool for RDKit cheminformatics operations
 Project-URL: Homepage, https://github.com/vitruves/rdkit-cli
 Project-URL: Repository, https://github.com/vitruves/rdkit-cli
@@ -38,7 +38,7 @@ A comprehensive, high-performance CLI tool wrapping RDKit functionality for chem
 
 ## Features
 
-- **14 Command Categories**: descriptors, fingerprints, filter, convert, standardize, similarity, conformers, reactions, scaffold, enumerate, fragment, diversity, mcs, depict
+- **19 Command Categories**: descriptors, fingerprints, filter, convert, standardize, similarity, conformers, reactions, scaffold, enumerate, fragment, diversity, mcs, depict, stats, split, sample, deduplicate, validate
 - **Multiple Input/Output Formats**: CSV, TSV, SMI, SDF, Parquet
 - **Parallel Processing**: Efficient multi-core support via ProcessPoolExecutor
 - **Ninja-style Progress**: Real-time progress display with speed and ETA
@@ -290,6 +290,93 @@ rdkit-cli depict batch -i molecules.csv -o images/ -f svg
 rdkit-cli depict grid -i molecules.csv -o grid.svg --mols-per-row 4
 ```
 
+### stats
+
+Calculate dataset statistics.
+
+```bash
+# Basic statistics
+rdkit-cli stats -i molecules.csv -o stats.json --format json
+
+# Specific properties
+rdkit-cli stats -i molecules.csv -p MolWt,LogP,TPSA
+
+# List available properties
+rdkit-cli stats -i molecules.csv --list-properties
+```
+
+### split
+
+Split files into smaller chunks.
+
+```bash
+# Split into N files
+rdkit-cli split -i large.csv -o chunks/ -c 10
+
+# Split by chunk size
+rdkit-cli split -i large.csv -o chunks/ -s 1000
+
+# With custom prefix
+rdkit-cli split -i large.csv -o chunks/ -c 5 --prefix molecules
+```
+
+### sample
+
+Randomly sample molecules.
+
+```bash
+# Sample by count
+rdkit-cli sample -i molecules.csv -o sample.csv -k 100 --seed 42
+
+# Sample by fraction
+rdkit-cli sample -i molecules.csv -o sample.csv -f 0.1
+
+# Memory-efficient streaming (reservoir sampling)
+rdkit-cli sample -i huge.csv -o sample.csv -k 1000 --stream
+```
+
+### deduplicate
+
+Remove duplicate molecules.
+
+```bash
+# Deduplicate by canonical SMILES (default)
+rdkit-cli deduplicate -i molecules.csv -o unique.csv
+
+# Deduplicate by InChIKey
+rdkit-cli deduplicate -i molecules.csv -o unique.csv -b inchikey
+
+# Deduplicate by scaffold
+rdkit-cli deduplicate -i molecules.csv -o unique.csv -b scaffold
+
+# Keep last occurrence instead of first
+rdkit-cli deduplicate -i molecules.csv -o unique.csv --keep last
+```
+
+### validate
+
+Validate molecular structures.
+
+```bash
+# Basic validation
+rdkit-cli validate -i molecules.csv -o validated.csv
+
+# Output only valid molecules
+rdkit-cli validate -i molecules.csv -o valid.csv --valid-only
+
+# With constraints
+rdkit-cli validate -i molecules.csv -o validated.csv \
+    --max-atoms 100 --max-rings 8
+
+# Check allowed elements
+rdkit-cli validate -i molecules.csv -o validated.csv \
+    --allowed-elements C,H,N,O,S,F,Cl
+
+# Check stereo and show summary
+rdkit-cli validate -i molecules.csv -o validated.csv \
+    --check-stereo --summary
+```
+
 ## Global Options
 
 | Option | Description |
@@ -319,19 +406,28 @@ rdkit-cli depict grid -i molecules.csv -o grid.svg --mols-per-row 4
 ### Cheminformatics Pipeline
 
 ```bash
-# 1. Standardize input molecules
-rdkit-cli standardize -i raw.csv -o std.csv --cleanup --neutralize
+# 1. Validate and filter input
+rdkit-cli validate -i raw.csv -o validated.csv --valid-only
+
+# 2. Deduplicate
+rdkit-cli deduplicate -i validated.csv -o unique.csv -b inchikey
 
-# 2. Filter by drug-likeness
+# 3. Standardize molecules
+rdkit-cli standardize -i unique.csv -o std.csv --cleanup --neutralize
+
+# 4. Filter by drug-likeness
 rdkit-cli filter druglike -i std.csv -o druglike.csv --rule lipinski
 
-# 3. Compute descriptors
+# 5. Compute descriptors
 rdkit-cli descriptors compute -i druglike.csv -o desc.csv -d MolWt,MolLogP,TPSA,HBD,HBA
 
-# 4. Select diverse subset
+# 6. Get dataset statistics
+rdkit-cli stats -i druglike.csv -o stats.json --format json
+
+# 7. Select diverse subset
 rdkit-cli diversity pick -i druglike.csv -o diverse.csv -k 500
 
-# 5. Generate depictions
+# 8. Generate depictions
 rdkit-cli depict grid -i diverse.csv -o library.svg --mols-per-row 10
 ```
 
@@ -358,6 +454,19 @@ rdkit-cli scaffold murcko -i library.csv -o scaffolds.csv
 rdkit-cli diversity analyze -i scaffolds.csv --smiles-column scaffold
 ```
 
+### Large Dataset Processing
+
+```bash
+# Sample from a huge dataset
+rdkit-cli sample -i huge_library.csv -o sample.csv -k 10000 --stream
+
+# Split for parallel processing
+rdkit-cli split -i library.csv -o batches/ -c 10
+
+# Process batches in parallel (using xargs)
+ls batches/*.csv | xargs -P 4 -I {} rdkit-cli descriptors compute -i {} -o {}.desc.csv -d MolWt,LogP
+```
+
 ## Development
 
 ```bash

{rdkit_cli-0.1.0 → rdkit_cli-0.2.0}/README.md

@@ -4,7 +4,7 @@ A comprehensive, high-performance CLI tool wrapping RDKit functionality for chem
 
 ## Features
 
-- **14 Command Categories**: descriptors, fingerprints, filter, convert, standardize, similarity, conformers, reactions, scaffold, enumerate, fragment, diversity, mcs, depict
+- **19 Command Categories**: descriptors, fingerprints, filter, convert, standardize, similarity, conformers, reactions, scaffold, enumerate, fragment, diversity, mcs, depict, stats, split, sample, deduplicate, validate
 - **Multiple Input/Output Formats**: CSV, TSV, SMI, SDF, Parquet
 - **Parallel Processing**: Efficient multi-core support via ProcessPoolExecutor
 - **Ninja-style Progress**: Real-time progress display with speed and ETA
@@ -256,6 +256,93 @@ rdkit-cli depict batch -i molecules.csv -o images/ -f svg
 rdkit-cli depict grid -i molecules.csv -o grid.svg --mols-per-row 4
 ```
 
+### stats
+
+Calculate dataset statistics.
+
+```bash
+# Basic statistics
+rdkit-cli stats -i molecules.csv -o stats.json --format json
+
+# Specific properties
+rdkit-cli stats -i molecules.csv -p MolWt,LogP,TPSA
+
+# List available properties
+rdkit-cli stats -i molecules.csv --list-properties
+```
+
+### split
+
+Split files into smaller chunks.
+
+```bash
+# Split into N files
+rdkit-cli split -i large.csv -o chunks/ -c 10
+
+# Split by chunk size
+rdkit-cli split -i large.csv -o chunks/ -s 1000
+
+# With custom prefix
+rdkit-cli split -i large.csv -o chunks/ -c 5 --prefix molecules
+```
+
+### sample
+
+Randomly sample molecules.
+
+```bash
+# Sample by count
+rdkit-cli sample -i molecules.csv -o sample.csv -k 100 --seed 42
+
+# Sample by fraction
+rdkit-cli sample -i molecules.csv -o sample.csv -f 0.1
+
+# Memory-efficient streaming (reservoir sampling)
+rdkit-cli sample -i huge.csv -o sample.csv -k 1000 --stream
+```
+
+### deduplicate
+
+Remove duplicate molecules.
+
+```bash
+# Deduplicate by canonical SMILES (default)
+rdkit-cli deduplicate -i molecules.csv -o unique.csv
+
+# Deduplicate by InChIKey
+rdkit-cli deduplicate -i molecules.csv -o unique.csv -b inchikey
+
+# Deduplicate by scaffold
+rdkit-cli deduplicate -i molecules.csv -o unique.csv -b scaffold
+
+# Keep last occurrence instead of first
+rdkit-cli deduplicate -i molecules.csv -o unique.csv --keep last
+```
+
+### validate
+
+Validate molecular structures.
+
+```bash
+# Basic validation
+rdkit-cli validate -i molecules.csv -o validated.csv
+
+# Output only valid molecules
+rdkit-cli validate -i molecules.csv -o valid.csv --valid-only
+
+# With constraints
+rdkit-cli validate -i molecules.csv -o validated.csv \
+    --max-atoms 100 --max-rings 8
+
+# Check allowed elements
+rdkit-cli validate -i molecules.csv -o validated.csv \
+    --allowed-elements C,H,N,O,S,F,Cl
+
+# Check stereo and show summary
+rdkit-cli validate -i molecules.csv -o validated.csv \
+    --check-stereo --summary
+```
+
 ## Global Options
 
 | Option | Description |
@@ -285,19 +372,28 @@ rdkit-cli depict grid -i molecules.csv -o grid.svg --mols-per-row 4
 ### Cheminformatics Pipeline
 
 ```bash
-# 1. Standardize input molecules
-rdkit-cli standardize -i raw.csv -o std.csv --cleanup --neutralize
+# 1. Validate and filter input
+rdkit-cli validate -i raw.csv -o validated.csv --valid-only
+
+# 2. Deduplicate
+rdkit-cli deduplicate -i validated.csv -o unique.csv -b inchikey
 
-# 2. Filter by drug-likeness
+# 3. Standardize molecules
+rdkit-cli standardize -i unique.csv -o std.csv --cleanup --neutralize
+
+# 4. Filter by drug-likeness
 rdkit-cli filter druglike -i std.csv -o druglike.csv --rule lipinski
 
-# 3. Compute descriptors
+# 5. Compute descriptors
 rdkit-cli descriptors compute -i druglike.csv -o desc.csv -d MolWt,MolLogP,TPSA,HBD,HBA
 
-# 4. Select diverse subset
+# 6. Get dataset statistics
+rdkit-cli stats -i druglike.csv -o stats.json --format json
+
+# 7. Select diverse subset
 rdkit-cli diversity pick -i druglike.csv -o diverse.csv -k 500
 
-# 5. Generate depictions
+# 8. Generate depictions
 rdkit-cli depict grid -i diverse.csv -o library.svg --mols-per-row 10
 ```
 
@@ -324,6 +420,19 @@ rdkit-cli scaffold murcko -i library.csv -o scaffolds.csv
 rdkit-cli diversity analyze -i scaffolds.csv --smiles-column scaffold
 ```
 
+### Large Dataset Processing
+
+```bash
+# Sample from a huge dataset
+rdkit-cli sample -i huge_library.csv -o sample.csv -k 10000 --stream
+
+# Split for parallel processing
+rdkit-cli split -i library.csv -o batches/ -c 10
+
+# Process batches in parallel (using xargs)
+ls batches/*.csv | xargs -P 4 -I {} rdkit-cli descriptors compute -i {} -o {}.desc.csv -d MolWt,LogP
+```
+
 ## Development
 
 ```bash

{rdkit_cli-0.1.0 → rdkit_cli-0.2.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "rdkit-cli"
-version = "0.1.0"
+version = "0.2.0"
 description = "A comprehensive CLI tool for RDKit cheminformatics operations"
 readme = "README.md"
 license = "Apache-2.0"

{rdkit_cli-0.1.0 → rdkit_cli-0.2.0}/src/rdkit_cli/__init__.py

@@ -1,4 +1,4 @@
 """rdkit-cli: A comprehensive CLI tool for RDKit cheminformatics operations."""
 
-__version__ = "0.1.0"
+__version__ = "0.2.0"
 __author__ = "Vitruves"

{rdkit_cli-0.1.0 → rdkit_cli-0.2.0}/src/rdkit_cli/cli.py

@@ -68,6 +68,18 @@ def add_common_processing_options(parser: argparse.ArgumentParser):
         action="store_true",
         help="Suppress progress output",
     )
+    parser.add_argument(
+        "--no-warnings",
+        action="store_true",
+        help="Suppress RDKit warnings (kekulization errors, etc.)",
+    )
+    parser.add_argument(
+        "--log-level",
+        choices=["debug", "info", "warning", "error", "critical"],
+        default=None,
+        metavar="LEVEL",
+        help="RDKit log level (default: warning, use 'error' to suppress warnings)",
+    )
 
 
 def create_parser() -> argparse.ArgumentParser:
@@ -100,39 +112,49 @@ def create_parser() -> argparse.ArgumentParser:
 
 
 def _register_commands(subparsers):
-    """Register all command subparsers."""
+    """Register all command subparsers (alphabetical order)."""
     from rdkit_cli.commands import (
-        descriptors,
-        fingerprints,
-        filter,
-        convert,
-        standardize,
-        similarity,
         conformers,
-        reactions,
-        scaffold,
+        convert,
+        deduplicate,
+        depict,
+        descriptors,
+        diversity,
         enumerate,
+        filter,
+        fingerprints,
         fragment,
-        diversity,
         mcs,
-        depict,
+        reactions,
+        sample,
+        scaffold,
+        similarity,
+        split,
+        standardize,
+        stats,
+        validate,
     )
 
     # Each module has a register_parser(subparsers) function
-    descriptors.register_parser(subparsers)
-    fingerprints.register_parser(subparsers)
-    filter.register_parser(subparsers)
-    convert.register_parser(subparsers)
-    standardize.register_parser(subparsers)
-    similarity.register_parser(subparsers)
     conformers.register_parser(subparsers)
-    reactions.register_parser(subparsers)
-    scaffold.register_parser(subparsers)
+    convert.register_parser(subparsers)
+    deduplicate.register_parser(subparsers)
+    depict.register_parser(subparsers)
+    descriptors.register_parser(subparsers)
+    diversity.register_parser(subparsers)
     enumerate.register_parser(subparsers)
+    filter.register_parser(subparsers)
+    fingerprints.register_parser(subparsers)
     fragment.register_parser(subparsers)
-    diversity.register_parser(subparsers)
     mcs.register_parser(subparsers)
-    depict.register_parser(subparsers)
+    reactions.register_parser(subparsers)
+    sample.register_parser(subparsers)
+    scaffold.register_parser(subparsers)
+    similarity.register_parser(subparsers)
+    split.register_parser(subparsers)
+    standardize.register_parser(subparsers)
+    stats.register_parser(subparsers)
+    validate.register_parser(subparsers)
 
 
 def main(args: Optional[list[str]] = None) -> int:
@@ -144,6 +166,18 @@ def main(args: Optional[list[str]] = None) -> int:
         parser.print_help()
         return 1
 
+    # Configure logging based on --no-warnings or --log-level
+    from rdkit_cli.utils import configure_all_warnings, set_rdkit_log_level
+    no_warnings = getattr(parsed_args, "no_warnings", False)
+    log_level = getattr(parsed_args, "log_level", None)
+
+    if no_warnings:
+        # Suppress both RDKit and application warnings
+        configure_all_warnings(suppress=True)
+    elif log_level is not None:
+        # Only control RDKit log level
+        set_rdkit_log_level(log_level)
+
     # Each command has a run(args) function via set_defaults(func=...)
     try:
         return parsed_args.func(parsed_args)

rdkit_cli-0.2.0/src/rdkit_cli/commands/deduplicate.py

@@ -0,0 +1,123 @@
+"""Deduplicate command implementation."""
+
+import sys
+from pathlib import Path
+
+from rdkit_cli.cli import RdkitHelpFormatter, add_common_io_options, add_common_processing_options
+
+
+def register_parser(subparsers):
+    """Register the deduplicate command."""
+    parser = subparsers.add_parser(
+        "deduplicate",
+        help="Remove duplicate molecules",
+        description="Remove duplicate molecules from a dataset based on various molecular identifiers.",
+        formatter_class=RdkitHelpFormatter,
+    )
+
+    add_common_io_options(parser)
+    add_common_processing_options(parser)
+
+    parser.add_argument(
+        "-b", "--by",
+        choices=["smiles", "inchi", "inchikey", "scaffold"],
+        default="smiles",
+        help="Deduplication key type (default: smiles)",
+    )
+    parser.add_argument(
+        "--keep",
+        choices=["first", "last"],
+        default="first",
+        help="Which duplicate to keep (default: first)",
+    )
+    parser.add_argument(
+        "--list-keys",
+        action="store_true",
+        help="List available key types and exit",
+    )
+
+    parser.set_defaults(func=run_deduplicate)
+
+
+def run_deduplicate(args) -> int:
+    """Run the deduplicate command."""
+    from rdkit_cli.core.deduplicate import Deduplicator
+    from rdkit_cli.io import create_reader, create_writer
+    from rdkit_cli.progress.ninja import NinjaProgress
+
+    # Handle --list-keys
+    if args.list_keys:
+        print("Available deduplication keys:")
+        print("  smiles    - Canonical SMILES (default)")
+        print("  inchi     - InChI string")
+        print("  inchikey  - InChIKey (27 character hash)")
+        print("  scaffold  - Murcko scaffold SMILES")
+        return 0
+
+    input_path = Path(args.input)
+    if not input_path.exists():
+        print(f"Error: Input file not found: {input_path}", file=sys.stderr)
+        return 1
+
+    # Create reader
+    reader = create_reader(
+        input_path,
+        smiles_column=args.smiles_column,
+        name_column=args.name_column,
+        has_header=not args.no_header,
+    )
+
+    # Read all records with progress
+    if not args.quiet:
+        print("Reading molecules...", file=sys.stderr)
+
+    records = []
+    with reader:
+        total = len(reader)
+        progress = NinjaProgress(total=total, quiet=args.quiet)
+        progress.start()
+
+        for record in reader:
+            records.append(record)
+            progress.update(1)
+
+        progress.finish()
+
+    if not records:
+        print("Error: No molecules found in input file", file=sys.stderr)
+        return 1
+
+    if not args.quiet:
+        print(f"Deduplicating {len(records)} molecules by {args.by}...", file=sys.stderr)
+
+    # Create deduplicator
+    deduplicator = Deduplicator(
+        key_type=args.by,
+        keep=args.keep,
+    )
+
+    # Deduplicate
+    unique_records, n_duplicates = deduplicator.deduplicate(records)
+
+    # Write output
+    output_path = Path(args.output)
+    writer = create_writer(output_path)
+
+    with writer:
+        for record in unique_records:
+            row = {"smiles": record.smiles}
+            if record.name:
+                row["name"] = record.name
+            for key, value in record.metadata.items():
+                if key not in row and key != "smiles":
+                    row[key] = value
+            writer.write_row(row)
+
+    if not args.quiet:
+        print(
+            f"Removed {n_duplicates} duplicates. "
+            f"Wrote {len(unique_records)} unique molecules to {output_path}",
+            file=sys.stderr,
+        )
+
+    return 0