src2purl 1.3.4__tar.gz → 1.4.0__tar.gz

src2purl-1.4.0/AUTHORS.md

@@ -0,0 +1,9 @@
+# Authors
+
+## Project Lead
+
+* Oscar Valenzuela B. - Project creator and maintainer
+
+---
+
+For a complete list of all contributors, please see the [GitHub contributors page](https://github.com/SemClone/src2purl/graphs/contributors).

{src2purl-1.3.4/src2purl.egg-info → src2purl-1.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: src2purl
-Version: 1.3.4
+Version: 1.4.0
 Summary: Source to PURL - Identify package coordinates and PURLs from source code using UPMEX and multiple discovery strategies
 Author-email: "Oscar Valenzuela B." <oscar.valenzuela.b@gmail.com>
 License: AGPL-3.0
@@ -20,6 +20,7 @@ Classifier: Topic :: Software Development :: Version Control
 Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
+License-File: AUTHORS.md
 Requires-Dist: aiohttp>=3.9.0
 Requires-Dist: click>=8.1.0
 Requires-Dist: rich>=13.0.0
@@ -43,84 +44,48 @@ Provides-Extra: enhanced
 Requires-Dist: osslili>=1.3.3; extra == "enhanced"
 Dynamic: license-file
 
-# SRC2PURL - Source Code to Package URL
+# SRC2PURL - Source Code to Package URL Discovery
 
-A Python tool that identifies package coordinates (name, version, license, PURL) from source code directories using an hybrid discovery strategy with manifest parsing, code fingerprinting, repository search, and Software Heritage archive.
+[![License](https://img.shields.io/badge/License-AGPL--3.0-blue.svg)](https://opensource.org/licenses/AGPL-3.0)
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+[![PyPI version](https://img.shields.io/pypi/v/src2purl.svg)](https://pypi.org/project/src2purl/)
 
-## Overview
-
-src2purl uses a **progressive 2-phase discovery strategy** to identify packages:
-
-### **Phase 1: Hash-based Package Discovery** (5-15 seconds)
-**Primary identification using content fingerprinting and repository search:**
-
-1. **Directory & Content Scanning** - Generate SWHIDs for directories and files
-   - ✅ **Complete file tree analysis** with configurable depth
-   - ✅ **SWHID generation** for precise content identification
-   - ✅ **Smart filtering** of binary files and hidden directories
-
-2. **Software Heritage Archive** (Optional) - Deep provenance discovery
-   - ✅ **Most comprehensive** - finds exact source code matches
-   - ✅ **Historical accuracy** - can identify older versions
-   - ⚠️ **Requires opt-in** with `--use-swh` due to longer processing time
-
-3. **Keyword Search** (Fallback) - Multi-platform repository discovery
-   - **GitHub API Search** - Repository identification by keywords
-   - **SCANOSS Fingerprinting** - Code similarity detection via file fingerprints
-   - ✅ **Universal coverage** - finds repositories for any project
-   - ✅ **Fast execution** (~10 seconds total)
-
-### **Phase 2: Manifest-based Validation & Enhancement** (1-3 seconds)
-**Authoritative package metadata extraction and result enhancement:**
-
-4. **UPMEX Manifest Parsing** - Universal Package Metadata Extractor
-   - ✅ **Perfect metadata extraction** from package files (package.json, setup.py, pom.xml, go.mod, Cargo.toml, etc.)
-   - ✅ **Multi-ecosystem support** (PyPI, NPM, Maven, Go, Ruby Gems, PHP, .NET)
-   - ✅ **Complete package info** (name, version, license, PURL)
-   - ✅ **Cross-validation** with Phase 1 results for enhanced accuracy
+A Python tool that identifies package coordinates (name, version, license, PURL) from source code directories using a hybrid discovery strategy with manifest parsing, code fingerprinting, repository search, and Software Heritage archive integration.
 
 ## Features
 
-### **Core Capabilities**
 - **2-Phase Discovery Strategy**: Hash-based identification enhanced by manifest parsing
 - **Multi-Ecosystem Support**: PyPI, NPM, Maven, Go, Ruby Gems, PHP, .NET, and more
-- **Cross-Phase Validation**: UPMEX manifest data validates and enhances hash-based findings
-- **Confidence Scoring**: Multi-factor scoring (85-100% for exact matches)
-- **Package Coordinate Extraction**: Complete metadata (name, version, license, PURL)
-
-### **Performance & Reliability**
-- **Fast by Default**: 5-15 seconds for typical projects (vs 90+ seconds with SWH)
-- **No API Keys Required**: Works well without authentication (SCANOSS, GitHub search)
-- **Optional API Keys**: Enhanced rate limits and accuracy with GitHub/SCANOSS tokens
-- **Persistent Caching**: File-based cache with smart TTL to avoid API rate limits
-- **Rate Limit Handling**: Automatic backoff and retry logic
-
-### **Discovery Methods**
-- **UPMEX Manifest Parsing**: Universal Package Metadata Extractor for all major package ecosystems
-- **SCANOSS Fingerprinting**: 100% accuracy code similarity with detailed license detection
-- **GitHub Repository Search**: Universal coverage repository identification
-- **Software Heritage Archive**: Comprehensive source inventory (opt-in with `--use-swh`)
-
-### **Output & Integration**
-- **Multiple Output Formats**: JSON and table output formats
-- **PURL Generation**: Standard Package URLs for identified packages
-- **Enhanced License Detection**: Integration with oslili for improved license detection
-- **Subcomponent Detection**: Identifies multiple packages within monorepos and complex projects
+- **Fast Performance**: 5-15 seconds for typical projects (vs 90+ seconds with SWH)
+- **SEMCL.ONE Integration**: Works seamlessly with upmex, osslili, and other ecosystem tools
 
 ## Installation
 
-### From Source
+```bash
+pip install src2purl
+```
 
+For development:
 ```bash
-git clone https://github.com/oscarvalenzuelab/src2purl.git
+git clone https://github.com/SemClone/src2purl.git
 cd src2purl
 pip install -e .
 ```
 
+## Quick Start
+
+```bash
+# Identify package from source code
+src2purl /path/to/source/code
+
+# With Software Heritage archive (comprehensive but slower)
+src2purl /path/to/source --use-swh
+```
+
 
 ## Usage
 
-### Basic Usage
+### CLI Usage
 
 ```bash
 # Fast discovery (default) - Uses manifest parsing + SCANOSS + GitHub (5-15 seconds)
@@ -137,15 +102,24 @@ src2purl /path/to/source --output-format json
 
 # Detect subcomponents in monorepos
 src2purl /path/to/source --detect-subcomponents
+```
 
-# Skip license detection (faster)
-src2purl /path/to/source --no-license-detection
+### Python API
 
-# Verbose output for debugging
-src2purl /path/to/source --verbose
+```python
+from src2purl import identify_package
 
-# Clear cache and exit
-src2purl --clear-cache
+# Basic identification
+result = identify_package("/path/to/source")
+print(f"Package: {result.name}@{result.version}")
+print(f"PURL: {result.purl}")
+
+# With options
+result = identify_package(
+    path="/path/to/source",
+    use_swh=True,
+    confidence_threshold=0.85
+)
 ```
 
 ### Discovery Strategy Examples
@@ -169,7 +143,7 @@ src2purl /path/to/project --confidence-threshold 0.85
 
 ### API Authentication
 
-**⚠️ No API keys required!** The tool works with the free public APIs. API keys only provide enhanced rate limits and additional features.
+**No API keys required!** The tool works with the free public APIs. API keys only provide enhanced rate limits and additional features.
 
 #### Recommended API Keys (Optional)
 
@@ -179,9 +153,9 @@ export GITHUB_TOKEN=your_github_personal_access_token
 ```
 - **Get token**: https://github.com/settings/tokens (no special permissions needed)
 - **Benefits**:
-  - ✅ **Rate limit**: 10 → 5000 requests/hour
-  - ✅ **Better search**: More accurate repository identification
-  - ✅ **No cost**: Completely free
+  - **Rate limit**: 10 → 5000 requests/hour
+  - **Better search**: More accurate repository identification
+  - **No cost**: Completely free
 - **Impact**: Significant improvement for repository discovery
 
 **2. SCANOSS API** - **Nice to Have** (Free, optional)
@@ -190,9 +164,9 @@ export SCANOSS_API_KEY=your_scanoss_key
 ```
 - **Get token**: Register at https://www.scanoss.com
 - **Benefits**:
-  - ✅ **No cost**: Free tier available
-  - ✅ **Enhanced rate limits**: Premium API endpoint
-  - ✅ **Additional features**: Possible extra metadata
+  - **No cost**: Free tier available
+  - **Enhanced rate limits**: Premium API endpoint
+  - **Additional features**: Possible extra metadata
 - **Impact**: Minor improvement (SCANOSS works great without key)
 
 **3. Software Heritage API** - **For Heavy Usage** (Free, only if using `--use-swh`)
@@ -201,8 +175,8 @@ export SWH_API_TOKEN=your_swh_token
 ```
 - **Get token**: Register at https://archive.softwareheritage.org/api/
 - **Benefits**:
-  - ✅ **Bypass rate limits**: No 60-second waits
-  - ✅ **Faster comprehensive scans**: When using `--use-swh`
+  - **Bypass rate limits**: No 60-second waits
+  - **Faster comprehensive scans**: When using `--use-swh`
 - **Impact**: Essential for `--use-swh` flag, not needed for default fast mode
 
 #### Performance Comparison
@@ -270,10 +244,61 @@ src2purl /path/to/project --no-license-detection  # Skip license enhancement
 src2purl /path/to/project --use-swh --verbose     # Full discovery with details
 ```
 
+## Integration with SEMCL.ONE
+
+SRC2PURL is a core component of the SEMCL.ONE ecosystem:
+
+- Uses **upmex** for manifest parsing and package metadata extraction
+- Integrates with **osslili** for enhanced license detection
+- Provides PURLs for **purl2src** to download source packages
+- Supports **purl2notices** for generating attribution documentation
+- Enables **ospac** policy evaluation with identified packages
+
+## Configuration
+
+Configuration via environment variables:
+
+```bash
+# API tokens (optional but recommended)
+export GITHUB_TOKEN=your_github_token
+export SCANOSS_API_KEY=your_scanoss_key
+export SWH_API_TOKEN=your_swh_token
+
+# Performance settings
+export SRC2PURL_CACHE_DIR=~/.cache/src2purl
+export SRC2PURL_MAX_DEPTH=2
+```
+
+## Documentation
+
+- [User Guide](docs/user-guide.md) - Comprehensive usage examples
+- [API Reference](docs/api.md) - Python API documentation
+- [Discovery Methods](docs/discovery-methods.md) - Detailed explanation of identification strategies
+- [Examples](docs/examples.md) - Common use cases and workflows
+
+## Contributing
+
+We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on:
+- Code of conduct
+- Development setup
+- Submitting pull requests
+- Reporting issues
+
+## Support
+
+For support and questions:
+- [GitHub Issues](https://github.com/SemClone/src2purl/issues) - Bug reports and feature requests
+- [Documentation](https://github.com/SemClone/src2purl) - Complete project documentation
+- [SEMCL.ONE Community](https://semcl.one) - Ecosystem support and discussions
+
 ## License
 
-This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0) - see the LICENSE file for details.
+GNU Affero General Public License v3.0 (AGPL-3.0) - see [LICENSE](LICENSE) file for details.
+
+## Authors
+
+See [AUTHORS.md](AUTHORS.md) for a list of contributors.
 
-## Status
+---
 
-This project is currently in active development. See the [Issues](https://github.com/oscarvalenzuelab/src2purl/issues) page for planned features and known issues.
+*Part of the [SEMCL.ONE](https://semcl.one) ecosystem for comprehensive OSS compliance and code analysis.*

{src2purl-1.3.4 → src2purl-1.4.0}/README.md

@@ -1,81 +1,45 @@
-# SRC2PURL - Source Code to Package URL
+# SRC2PURL - Source Code to Package URL Discovery
 
-A Python tool that identifies package coordinates (name, version, license, PURL) from source code directories using an hybrid discovery strategy with manifest parsing, code fingerprinting, repository search, and Software Heritage archive.
+[![License](https://img.shields.io/badge/License-AGPL--3.0-blue.svg)](https://opensource.org/licenses/AGPL-3.0)
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+[![PyPI version](https://img.shields.io/pypi/v/src2purl.svg)](https://pypi.org/project/src2purl/)
 
-## Overview
-
-src2purl uses a **progressive 2-phase discovery strategy** to identify packages:
-
-### **Phase 1: Hash-based Package Discovery** (5-15 seconds)
-**Primary identification using content fingerprinting and repository search:**
-
-1. **Directory & Content Scanning** - Generate SWHIDs for directories and files
-   - ✅ **Complete file tree analysis** with configurable depth
-   - ✅ **SWHID generation** for precise content identification
-   - ✅ **Smart filtering** of binary files and hidden directories
-
-2. **Software Heritage Archive** (Optional) - Deep provenance discovery
-   - ✅ **Most comprehensive** - finds exact source code matches
-   - ✅ **Historical accuracy** - can identify older versions
-   - ⚠️ **Requires opt-in** with `--use-swh` due to longer processing time
-
-3. **Keyword Search** (Fallback) - Multi-platform repository discovery
-   - **GitHub API Search** - Repository identification by keywords
-   - **SCANOSS Fingerprinting** - Code similarity detection via file fingerprints
-   - ✅ **Universal coverage** - finds repositories for any project
-   - ✅ **Fast execution** (~10 seconds total)
-
-### **Phase 2: Manifest-based Validation & Enhancement** (1-3 seconds)
-**Authoritative package metadata extraction and result enhancement:**
-
-4. **UPMEX Manifest Parsing** - Universal Package Metadata Extractor
-   - ✅ **Perfect metadata extraction** from package files (package.json, setup.py, pom.xml, go.mod, Cargo.toml, etc.)
-   - ✅ **Multi-ecosystem support** (PyPI, NPM, Maven, Go, Ruby Gems, PHP, .NET)
-   - ✅ **Complete package info** (name, version, license, PURL)
-   - ✅ **Cross-validation** with Phase 1 results for enhanced accuracy
+A Python tool that identifies package coordinates (name, version, license, PURL) from source code directories using a hybrid discovery strategy with manifest parsing, code fingerprinting, repository search, and Software Heritage archive integration.
 
 ## Features
 
-### **Core Capabilities**
 - **2-Phase Discovery Strategy**: Hash-based identification enhanced by manifest parsing
 - **Multi-Ecosystem Support**: PyPI, NPM, Maven, Go, Ruby Gems, PHP, .NET, and more
-- **Cross-Phase Validation**: UPMEX manifest data validates and enhances hash-based findings
-- **Confidence Scoring**: Multi-factor scoring (85-100% for exact matches)
-- **Package Coordinate Extraction**: Complete metadata (name, version, license, PURL)
-
-### **Performance & Reliability**
-- **Fast by Default**: 5-15 seconds for typical projects (vs 90+ seconds with SWH)
-- **No API Keys Required**: Works well without authentication (SCANOSS, GitHub search)
-- **Optional API Keys**: Enhanced rate limits and accuracy with GitHub/SCANOSS tokens
-- **Persistent Caching**: File-based cache with smart TTL to avoid API rate limits
-- **Rate Limit Handling**: Automatic backoff and retry logic
-
-### **Discovery Methods**
-- **UPMEX Manifest Parsing**: Universal Package Metadata Extractor for all major package ecosystems
-- **SCANOSS Fingerprinting**: 100% accuracy code similarity with detailed license detection
-- **GitHub Repository Search**: Universal coverage repository identification
-- **Software Heritage Archive**: Comprehensive source inventory (opt-in with `--use-swh`)
-
-### **Output & Integration**
-- **Multiple Output Formats**: JSON and table output formats
-- **PURL Generation**: Standard Package URLs for identified packages
-- **Enhanced License Detection**: Integration with oslili for improved license detection
-- **Subcomponent Detection**: Identifies multiple packages within monorepos and complex projects
+- **Fast Performance**: 5-15 seconds for typical projects (vs 90+ seconds with SWH)
+- **SEMCL.ONE Integration**: Works seamlessly with upmex, osslili, and other ecosystem tools
 
 ## Installation
 
-### From Source
+```bash
+pip install src2purl
+```
 
+For development:
 ```bash
-git clone https://github.com/oscarvalenzuelab/src2purl.git
+git clone https://github.com/SemClone/src2purl.git
 cd src2purl
 pip install -e .
 ```
 
+## Quick Start
+
+```bash
+# Identify package from source code
+src2purl /path/to/source/code
+
+# With Software Heritage archive (comprehensive but slower)
+src2purl /path/to/source --use-swh
+```
+
 
 ## Usage
 
-### Basic Usage
+### CLI Usage
 
 ```bash
 # Fast discovery (default) - Uses manifest parsing + SCANOSS + GitHub (5-15 seconds)
@@ -92,15 +56,24 @@ src2purl /path/to/source --output-format json
 
 # Detect subcomponents in monorepos
 src2purl /path/to/source --detect-subcomponents
+```
 
-# Skip license detection (faster)
-src2purl /path/to/source --no-license-detection
+### Python API
 
-# Verbose output for debugging
-src2purl /path/to/source --verbose
+```python
+from src2purl import identify_package
 
-# Clear cache and exit
-src2purl --clear-cache
+# Basic identification
+result = identify_package("/path/to/source")
+print(f"Package: {result.name}@{result.version}")
+print(f"PURL: {result.purl}")
+
+# With options
+result = identify_package(
+    path="/path/to/source",
+    use_swh=True,
+    confidence_threshold=0.85
+)
 ```
 
 ### Discovery Strategy Examples
@@ -124,7 +97,7 @@ src2purl /path/to/project --confidence-threshold 0.85
 
 ### API Authentication
 
-**⚠️ No API keys required!** The tool works with the free public APIs. API keys only provide enhanced rate limits and additional features.
+**No API keys required!** The tool works with the free public APIs. API keys only provide enhanced rate limits and additional features.
 
 #### Recommended API Keys (Optional)
 
@@ -134,9 +107,9 @@ export GITHUB_TOKEN=your_github_personal_access_token
 ```
 - **Get token**: https://github.com/settings/tokens (no special permissions needed)
 - **Benefits**:
-  - ✅ **Rate limit**: 10 → 5000 requests/hour
-  - ✅ **Better search**: More accurate repository identification
-  - ✅ **No cost**: Completely free
+  - **Rate limit**: 10 → 5000 requests/hour
+  - **Better search**: More accurate repository identification
+  - **No cost**: Completely free
 - **Impact**: Significant improvement for repository discovery
 
 **2. SCANOSS API** - **Nice to Have** (Free, optional)
@@ -145,9 +118,9 @@ export SCANOSS_API_KEY=your_scanoss_key
 ```
 - **Get token**: Register at https://www.scanoss.com
 - **Benefits**:
-  - ✅ **No cost**: Free tier available
-  - ✅ **Enhanced rate limits**: Premium API endpoint
-  - ✅ **Additional features**: Possible extra metadata
+  - **No cost**: Free tier available
+  - **Enhanced rate limits**: Premium API endpoint
+  - **Additional features**: Possible extra metadata
 - **Impact**: Minor improvement (SCANOSS works great without key)
 
 **3. Software Heritage API** - **For Heavy Usage** (Free, only if using `--use-swh`)
@@ -156,8 +129,8 @@ export SWH_API_TOKEN=your_swh_token
 ```
 - **Get token**: Register at https://archive.softwareheritage.org/api/
 - **Benefits**:
-  - ✅ **Bypass rate limits**: No 60-second waits
-  - ✅ **Faster comprehensive scans**: When using `--use-swh`
+  - **Bypass rate limits**: No 60-second waits
+  - **Faster comprehensive scans**: When using `--use-swh`
 - **Impact**: Essential for `--use-swh` flag, not needed for default fast mode
 
 #### Performance Comparison
@@ -225,10 +198,61 @@ src2purl /path/to/project --no-license-detection  # Skip license enhancement
 src2purl /path/to/project --use-swh --verbose     # Full discovery with details
 ```
 
+## Integration with SEMCL.ONE
+
+SRC2PURL is a core component of the SEMCL.ONE ecosystem:
+
+- Uses **upmex** for manifest parsing and package metadata extraction
+- Integrates with **osslili** for enhanced license detection
+- Provides PURLs for **purl2src** to download source packages
+- Supports **purl2notices** for generating attribution documentation
+- Enables **ospac** policy evaluation with identified packages
+
+## Configuration
+
+Configuration via environment variables:
+
+```bash
+# API tokens (optional but recommended)
+export GITHUB_TOKEN=your_github_token
+export SCANOSS_API_KEY=your_scanoss_key
+export SWH_API_TOKEN=your_swh_token
+
+# Performance settings
+export SRC2PURL_CACHE_DIR=~/.cache/src2purl
+export SRC2PURL_MAX_DEPTH=2
+```
+
+## Documentation
+
+- [User Guide](docs/user-guide.md) - Comprehensive usage examples
+- [API Reference](docs/api.md) - Python API documentation
+- [Discovery Methods](docs/discovery-methods.md) - Detailed explanation of identification strategies
+- [Examples](docs/examples.md) - Common use cases and workflows
+
+## Contributing
+
+We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on:
+- Code of conduct
+- Development setup
+- Submitting pull requests
+- Reporting issues
+
+## Support
+
+For support and questions:
+- [GitHub Issues](https://github.com/SemClone/src2purl/issues) - Bug reports and feature requests
+- [Documentation](https://github.com/SemClone/src2purl) - Complete project documentation
+- [SEMCL.ONE Community](https://semcl.one) - Ecosystem support and discussions
+
 ## License
 
-This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0) - see the LICENSE file for details.
+GNU Affero General Public License v3.0 (AGPL-3.0) - see [LICENSE](LICENSE) file for details.
+
+## Authors
+
+See [AUTHORS.md](AUTHORS.md) for a list of contributors.
 
-## Status
+---
 
-This project is currently in active development. See the [Issues](https://github.com/oscarvalenzuelab/src2purl/issues) page for planned features and known issues.
+*Part of the [SEMCL.ONE](https://semcl.one) ecosystem for comprehensive OSS compliance and code analysis.*

{src2purl-1.3.4 → src2purl-1.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "src2purl"
-version = "1.3.4"
+version = "1.4.0"
 description = "Source to PURL - Identify package coordinates and PURLs from source code using UPMEX and multiple discovery strategies"
 readme = "README.md"
 requires-python = ">=3.9"

{src2purl-1.3.4 → src2purl-1.4.0}/src2id/__init__.py

@@ -6,7 +6,7 @@ using multiple strategies including SCANOSS, hash search, web search, and
 optionally Software Heritage archive.
 """
 
-__version__ = "1.3.3"
+__version__ = "1.3.5"
 __author__ = "Oscar Valenzuela B."
 
 from src2id.core.config import SWHPIConfig

{src2purl-1.3.4 → src2purl-1.4.0}/src2id/core/orchestrator.py

@@ -141,16 +141,6 @@ class SHPackageIdentifier:
                     if self.config.verbose:
                         console.print("[dim]Skipping Software Heritage (use --use-swh to enable)[/dim]")
 
-                if not all_matches:
-                    # Try keyword search with GitHub and SCANOSS
-                    if self.config.verbose:
-                        console.print("[yellow]Trying web search (GitHub, SCANOSS)[/yellow]")
-
-                    # Try keyword search
-                    keyword_matches = await self._find_keyword_matches(path)
-                    if keyword_matches:
-                        all_matches = keyword_matches
-
                 # Step 1b: Process hash-based matches
                 if all_matches:
                     hash_based_matches = await self._process_matches(all_matches)
@@ -178,6 +168,27 @@ class SHPackageIdentifier:
 
             # Final deduplication and sorting
             final_matches = self._prioritize_and_deduplicate(enhanced_matches)
+
+            # Keyword search is the last resort, and only when it was asked for.
+            # It searches on the project name, so it answers with whatever else
+            # shares that name: running it for express returned ten npm packages
+            # scoring 0.83 against the real one at 0.85, close enough that a
+            # confidence threshold cannot separate them, and carrying no PURL so
+            # nothing downstream can act on them either.
+            #
+            # It also belongs here rather than in phase 1. The flag says fuzzy
+            # runs when exact matches fail, and whether they failed is not known
+            # until the manifest parsing in phase 2 has had its turn.
+            if not final_matches and self.config.enable_fuzzy_matching:
+                if self.config.verbose:
+                    console.print(
+                        "[yellow]Nothing identified - trying keyword search[/yellow]"
+                    )
+                keyword_matches = await self._find_keyword_matches(path)
+                if keyword_matches:
+                    final_matches = self._prioritize_and_deduplicate(
+                        await self._process_matches(keyword_matches)
+                    )
             
             # Step 5: Optionally enhance with oslili license detection
             if enhance_licenses:
@@ -588,14 +599,12 @@ class SHPackageIdentifier:
             for candidate in non_existing_dirs:
                 console.print(f"[yellow]✗ No match for {candidate.path.name} ({candidate.swhid[:12]}...)[/yellow]")
         
-        # Try keyword search fallback if no exact matches and fuzzy is enabled
-        if not all_matches and self.config.enable_fuzzy_matching:
-            if self.config.verbose:
-                console.print("[yellow]No exact matches found - trying keyword search[/yellow]")
-            # Only try keyword search if fuzzy matching is enabled
-            keyword_matches = await self._find_keyword_matches(dir_candidates[0].path if dir_candidates else Path("."))
-            all_matches.extend(keyword_matches)
-        
+        # Keyword search is not run here. It used to be, and that put it before
+        # the manifest parsing in phase 2, so "when exact matches fail", which
+        # is what the flag promises, was decided before the step that finds the
+        # exact match had run. identify_packages runs it once, after everything
+        # else has had its turn.
+
         # Report match summary
         if self.config.verbose and (child_matches > 0 or parent_matches > 0):
             console.print("\n[bold]Match Summary:[/bold]")

{src2purl-1.3.4 → src2purl-1.4.0}/src2id/core/package_identifier.py

@@ -82,8 +82,12 @@ class PackageIdentifier:
                         
                         if integration.available:
                             license_info = integration.detect_licenses(path)
-                            if license_info["licenses"]:
-                                match.license = ", ".join(license_info["licenses"][:3])
+                            # Only the project's own licenses identify this package;
+                            # licenses from bundled third-party notice files belong
+                            # to its dependencies
+                            own_licenses = license_info.get("own_licenses") or []
+                            if own_licenses:
+                                match.license = ", ".join(own_licenses[:3])
                     except ImportError:
                         pass
                 

{src2purl-1.3.4 → src2purl-1.4.0}/src2id/integrations/oslili.py

@@ -59,14 +59,20 @@ class OsliliIntegration:
         Returns:
             Dictionary with license information:
             - licenses: List of detected license identifiers
-            - confidence: Confidence score (0-1)
+            - own_licenses: Identifiers eligible to be the project's own license
+            - third_party_licenses: Identifiers from bundled third-party notice files
+            - confidence: Confidence score (0-1) across all detections
+            - own_confidence: Confidence score (0-1) across own licenses only
             - files: Dict mapping files to their licenses
             - summary: Human-readable summary
         """
         if not self.available or not self.detector:
             return {
                 "licenses": [],
+                "own_licenses": [],
+                "third_party_licenses": [],
                 "confidence": 0.0,
+                "own_confidence": 0.0,
                 "files": {},
                 "summary": "oslili not available",
                 "error": "oslili integration not available"
@@ -83,13 +89,19 @@ class OsliliIntegration:
                 files = {}
                 
                 # Group licenses by category for better understanding
-                declared = [l for l in result.licenses if l.category == "declared"]
-                detected = [l for l in result.licenses if l.category == "detected"]
-                referenced = [l for l in result.licenses if l.category == "referenced"]
-                
-                # Prioritize declared licenses, then detected, then referenced
-                all_licenses = declared + detected + referenced
-                
+                declared = [lic for lic in result.licenses if lic.category == "declared"]
+                detected = [lic for lic in result.licenses if lic.category == "detected"]
+                referenced = [lic for lic in result.licenses if lic.category == "referenced"]
+                # Licenses of bundled dependencies (THIRD_PARTY_NOTICES.txt and
+                # friends), not the project's own license. Kept, but ranked last so
+                # they never win the primary-license selection.
+                third_party = [lic for lic in result.licenses if lic.category == "third-party"]
+
+                # Prioritize declared licenses, then detected, then referenced,
+                # with bundled third-party licenses last
+                own = declared + detected + referenced
+                all_licenses = own + third_party
+
                 for license_info in all_licenses:
                     licenses.append(license_info.spdx_id)
                     confidence_scores.append(license_info.confidence)
@@ -107,6 +119,12 @@ class OsliliIntegration:
                 
                 # Calculate average confidence
                 avg_confidence = sum(confidence_scores) / len(confidence_scores) if confidence_scores else 0.8
+
+                # Confidence in the project's own licenses, kept separate so a pile of
+                # high-confidence bundled notice files cannot inflate the score used to
+                # decide the package's own license
+                own_scores = [lic.confidence for lic in own]
+                own_confidence = sum(own_scores) / len(own_scores) if own_scores else 0.0
                 
                 # Generate summary
                 if licenses:
@@ -126,7 +144,15 @@ class OsliliIntegration:
                 
                 return {
                     "licenses": licenses,
+                    # Split view of the same licenses: "own" are candidates for the
+                    # project's license, "third_party" come from bundled notice files
+                    # and must not be attributed to the project itself.
+                    "own_licenses": self._deduplicate_licenses([lic.spdx_id for lic in own]),
+                    "third_party_licenses": self._deduplicate_licenses(
+                        [lic.spdx_id for lic in third_party]
+                    ),
                     "confidence": avg_confidence,
+                    "own_confidence": own_confidence,
                     "files": files,
                     "summary": summary,
                     "copyrights": [c.to_dict() for c in result.copyrights] if result.copyrights else []
@@ -134,7 +160,10 @@ class OsliliIntegration:
             else:
                 return {
                     "licenses": [],
+                    "own_licenses": [],
+                    "third_party_licenses": [],
                     "confidence": 0.0,
+                    "own_confidence": 0.0,
                     "files": {},
                     "summary": "No licenses detected",
                     "copyrights": []
@@ -143,7 +172,10 @@ class OsliliIntegration:
         except Exception as e:
             return {
                 "licenses": [],
+                "own_licenses": [],
+                "third_party_licenses": [],
                 "confidence": 0.0,
+                "own_confidence": 0.0,
                 "files": {},
                 "summary": f"Error during detection: {e}",
                 "error": str(e)
@@ -170,17 +202,28 @@ class OsliliIntegration:
         # Detect licenses
         license_info = self.detect_licenses(path)
         
+        # Only the project's own licenses are eligible to become this package's
+        # license: one found in a bundled third-party notice file belongs to a
+        # dependency, so attributing it here would be wrong. Bundled notices must
+        # therefore neither raise nor lower the bar for the own license, which is
+        # why the decision below uses own_confidence rather than the aggregate.
+        own_licenses = self._deduplicate_licenses(license_info.get("own_licenses") or [])
+        own_confidence = license_info.get("own_confidence") or 0.0
+
         # Update match if we found licenses with good confidence
-        if license_info["licenses"] and license_info["confidence"] > 0.7:
+        if license_info["licenses"] and (
+            license_info["confidence"] > 0.7 or own_confidence > 0.7
+        ):
             # Remove duplicates while preserving order
             unique_licenses = self._deduplicate_licenses(license_info["licenses"])
             
-            # Use the most confident license as primary
-            primary_license = unique_licenses[0]
-            
-            # Update match license if not already set or if ours is more confident
-            if not match.license or license_info["confidence"] > 0.85:
-                match.license = primary_license
+            # Update match license if not already set or if ours is more confident.
+            # When nothing but bundled notices was found there is no own license, so
+            # match.license is left alone rather than guessed at.
+            if own_licenses and own_confidence > 0.7 and (
+                not match.license or own_confidence > 0.85
+            ):
+                match.license = own_licenses[0]
             
             # Add metadata about additional licenses and copyrights
             if not hasattr(match, "metadata"):
@@ -188,6 +231,11 @@ class OsliliIntegration:
             
             if len(unique_licenses) > 1:
                 match.metadata["additional_licenses"] = unique_licenses[1:]
+
+            # Surface bundled third-party licenses separately so callers can report
+            # them without mistaking them for the package's own license
+            if license_info.get("third_party_licenses"):
+                match.metadata["third_party_licenses"] = license_info["third_party_licenses"]
             
             match.metadata["license_confidence"] = license_info["confidence"]
             

{src2purl-1.3.4 → src2purl-1.4.0/src2purl.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: src2purl
-Version: 1.3.4
+Version: 1.4.0
 Summary: Source to PURL - Identify package coordinates and PURLs from source code using UPMEX and multiple discovery strategies
 Author-email: "Oscar Valenzuela B." <oscar.valenzuela.b@gmail.com>
 License: AGPL-3.0
@@ -20,6 +20,7 @@ Classifier: Topic :: Software Development :: Version Control
 Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
+License-File: AUTHORS.md
 Requires-Dist: aiohttp>=3.9.0
 Requires-Dist: click>=8.1.0
 Requires-Dist: rich>=13.0.0
@@ -43,84 +44,48 @@ Provides-Extra: enhanced
 Requires-Dist: osslili>=1.3.3; extra == "enhanced"
 Dynamic: license-file
 
-# SRC2PURL - Source Code to Package URL
+# SRC2PURL - Source Code to Package URL Discovery
 
-A Python tool that identifies package coordinates (name, version, license, PURL) from source code directories using an hybrid discovery strategy with manifest parsing, code fingerprinting, repository search, and Software Heritage archive.
+[![License](https://img.shields.io/badge/License-AGPL--3.0-blue.svg)](https://opensource.org/licenses/AGPL-3.0)
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+[![PyPI version](https://img.shields.io/pypi/v/src2purl.svg)](https://pypi.org/project/src2purl/)
 
-## Overview
-
-src2purl uses a **progressive 2-phase discovery strategy** to identify packages:
-
-### **Phase 1: Hash-based Package Discovery** (5-15 seconds)
-**Primary identification using content fingerprinting and repository search:**
-
-1. **Directory & Content Scanning** - Generate SWHIDs for directories and files
-   - ✅ **Complete file tree analysis** with configurable depth
-   - ✅ **SWHID generation** for precise content identification
-   - ✅ **Smart filtering** of binary files and hidden directories
-
-2. **Software Heritage Archive** (Optional) - Deep provenance discovery
-   - ✅ **Most comprehensive** - finds exact source code matches
-   - ✅ **Historical accuracy** - can identify older versions
-   - ⚠️ **Requires opt-in** with `--use-swh` due to longer processing time
-
-3. **Keyword Search** (Fallback) - Multi-platform repository discovery
-   - **GitHub API Search** - Repository identification by keywords
-   - **SCANOSS Fingerprinting** - Code similarity detection via file fingerprints
-   - ✅ **Universal coverage** - finds repositories for any project
-   - ✅ **Fast execution** (~10 seconds total)
-
-### **Phase 2: Manifest-based Validation & Enhancement** (1-3 seconds)
-**Authoritative package metadata extraction and result enhancement:**
-
-4. **UPMEX Manifest Parsing** - Universal Package Metadata Extractor
-   - ✅ **Perfect metadata extraction** from package files (package.json, setup.py, pom.xml, go.mod, Cargo.toml, etc.)
-   - ✅ **Multi-ecosystem support** (PyPI, NPM, Maven, Go, Ruby Gems, PHP, .NET)
-   - ✅ **Complete package info** (name, version, license, PURL)
-   - ✅ **Cross-validation** with Phase 1 results for enhanced accuracy
+A Python tool that identifies package coordinates (name, version, license, PURL) from source code directories using a hybrid discovery strategy with manifest parsing, code fingerprinting, repository search, and Software Heritage archive integration.
 
 ## Features
 
-### **Core Capabilities**
 - **2-Phase Discovery Strategy**: Hash-based identification enhanced by manifest parsing
 - **Multi-Ecosystem Support**: PyPI, NPM, Maven, Go, Ruby Gems, PHP, .NET, and more
-- **Cross-Phase Validation**: UPMEX manifest data validates and enhances hash-based findings
-- **Confidence Scoring**: Multi-factor scoring (85-100% for exact matches)
-- **Package Coordinate Extraction**: Complete metadata (name, version, license, PURL)
-
-### **Performance & Reliability**
-- **Fast by Default**: 5-15 seconds for typical projects (vs 90+ seconds with SWH)
-- **No API Keys Required**: Works well without authentication (SCANOSS, GitHub search)
-- **Optional API Keys**: Enhanced rate limits and accuracy with GitHub/SCANOSS tokens
-- **Persistent Caching**: File-based cache with smart TTL to avoid API rate limits
-- **Rate Limit Handling**: Automatic backoff and retry logic
-
-### **Discovery Methods**
-- **UPMEX Manifest Parsing**: Universal Package Metadata Extractor for all major package ecosystems
-- **SCANOSS Fingerprinting**: 100% accuracy code similarity with detailed license detection
-- **GitHub Repository Search**: Universal coverage repository identification
-- **Software Heritage Archive**: Comprehensive source inventory (opt-in with `--use-swh`)
-
-### **Output & Integration**
-- **Multiple Output Formats**: JSON and table output formats
-- **PURL Generation**: Standard Package URLs for identified packages
-- **Enhanced License Detection**: Integration with oslili for improved license detection
-- **Subcomponent Detection**: Identifies multiple packages within monorepos and complex projects
+- **Fast Performance**: 5-15 seconds for typical projects (vs 90+ seconds with SWH)
+- **SEMCL.ONE Integration**: Works seamlessly with upmex, osslili, and other ecosystem tools
 
 ## Installation
 
-### From Source
+```bash
+pip install src2purl
+```
 
+For development:
 ```bash
-git clone https://github.com/oscarvalenzuelab/src2purl.git
+git clone https://github.com/SemClone/src2purl.git
 cd src2purl
 pip install -e .
 ```
 
+## Quick Start
+
+```bash
+# Identify package from source code
+src2purl /path/to/source/code
+
+# With Software Heritage archive (comprehensive but slower)
+src2purl /path/to/source --use-swh
+```
+
 
 ## Usage
 
-### Basic Usage
+### CLI Usage
 
 ```bash
 # Fast discovery (default) - Uses manifest parsing + SCANOSS + GitHub (5-15 seconds)
@@ -137,15 +102,24 @@ src2purl /path/to/source --output-format json
 
 # Detect subcomponents in monorepos
 src2purl /path/to/source --detect-subcomponents
+```
 
-# Skip license detection (faster)
-src2purl /path/to/source --no-license-detection
+### Python API
 
-# Verbose output for debugging
-src2purl /path/to/source --verbose
+```python
+from src2purl import identify_package
 
-# Clear cache and exit
-src2purl --clear-cache
+# Basic identification
+result = identify_package("/path/to/source")
+print(f"Package: {result.name}@{result.version}")
+print(f"PURL: {result.purl}")
+
+# With options
+result = identify_package(
+    path="/path/to/source",
+    use_swh=True,
+    confidence_threshold=0.85
+)
 ```
 
 ### Discovery Strategy Examples
@@ -169,7 +143,7 @@ src2purl /path/to/project --confidence-threshold 0.85
 
 ### API Authentication
 
-**⚠️ No API keys required!** The tool works with the free public APIs. API keys only provide enhanced rate limits and additional features.
+**No API keys required!** The tool works with the free public APIs. API keys only provide enhanced rate limits and additional features.
 
 #### Recommended API Keys (Optional)
 
@@ -179,9 +153,9 @@ export GITHUB_TOKEN=your_github_personal_access_token
 ```
 - **Get token**: https://github.com/settings/tokens (no special permissions needed)
 - **Benefits**:
-  - ✅ **Rate limit**: 10 → 5000 requests/hour
-  - ✅ **Better search**: More accurate repository identification
-  - ✅ **No cost**: Completely free
+  - **Rate limit**: 10 → 5000 requests/hour
+  - **Better search**: More accurate repository identification
+  - **No cost**: Completely free
 - **Impact**: Significant improvement for repository discovery
 
 **2. SCANOSS API** - **Nice to Have** (Free, optional)
@@ -190,9 +164,9 @@ export SCANOSS_API_KEY=your_scanoss_key
 ```
 - **Get token**: Register at https://www.scanoss.com
 - **Benefits**:
-  - ✅ **No cost**: Free tier available
-  - ✅ **Enhanced rate limits**: Premium API endpoint
-  - ✅ **Additional features**: Possible extra metadata
+  - **No cost**: Free tier available
+  - **Enhanced rate limits**: Premium API endpoint
+  - **Additional features**: Possible extra metadata
 - **Impact**: Minor improvement (SCANOSS works great without key)
 
 **3. Software Heritage API** - **For Heavy Usage** (Free, only if using `--use-swh`)
@@ -201,8 +175,8 @@ export SWH_API_TOKEN=your_swh_token
 ```
 - **Get token**: Register at https://archive.softwareheritage.org/api/
 - **Benefits**:
-  - ✅ **Bypass rate limits**: No 60-second waits
-  - ✅ **Faster comprehensive scans**: When using `--use-swh`
+  - **Bypass rate limits**: No 60-second waits
+  - **Faster comprehensive scans**: When using `--use-swh`
 - **Impact**: Essential for `--use-swh` flag, not needed for default fast mode
 
 #### Performance Comparison
@@ -270,10 +244,61 @@ src2purl /path/to/project --no-license-detection  # Skip license enhancement
 src2purl /path/to/project --use-swh --verbose     # Full discovery with details
 ```
 
+## Integration with SEMCL.ONE
+
+SRC2PURL is a core component of the SEMCL.ONE ecosystem:
+
+- Uses **upmex** for manifest parsing and package metadata extraction
+- Integrates with **osslili** for enhanced license detection
+- Provides PURLs for **purl2src** to download source packages
+- Supports **purl2notices** for generating attribution documentation
+- Enables **ospac** policy evaluation with identified packages
+
+## Configuration
+
+Configuration via environment variables:
+
+```bash
+# API tokens (optional but recommended)
+export GITHUB_TOKEN=your_github_token
+export SCANOSS_API_KEY=your_scanoss_key
+export SWH_API_TOKEN=your_swh_token
+
+# Performance settings
+export SRC2PURL_CACHE_DIR=~/.cache/src2purl
+export SRC2PURL_MAX_DEPTH=2
+```
+
+## Documentation
+
+- [User Guide](docs/user-guide.md) - Comprehensive usage examples
+- [API Reference](docs/api.md) - Python API documentation
+- [Discovery Methods](docs/discovery-methods.md) - Detailed explanation of identification strategies
+- [Examples](docs/examples.md) - Common use cases and workflows
+
+## Contributing
+
+We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on:
+- Code of conduct
+- Development setup
+- Submitting pull requests
+- Reporting issues
+
+## Support
+
+For support and questions:
+- [GitHub Issues](https://github.com/SemClone/src2purl/issues) - Bug reports and feature requests
+- [Documentation](https://github.com/SemClone/src2purl) - Complete project documentation
+- [SEMCL.ONE Community](https://semcl.one) - Ecosystem support and discussions
+
 ## License
 
-This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0) - see the LICENSE file for details.
+GNU Affero General Public License v3.0 (AGPL-3.0) - see [LICENSE](LICENSE) file for details.
+
+## Authors
+
+See [AUTHORS.md](AUTHORS.md) for a list of contributors.
 
-## Status
+---
 
-This project is currently in active development. See the [Issues](https://github.com/oscarvalenzuelab/src2purl/issues) page for planned features and known issues.
+*Part of the [SEMCL.ONE](https://semcl.one) ecosystem for comprehensive OSS compliance and code analysis.*

{src2purl-1.3.4 → src2purl-1.4.0}/src2purl.egg-info/SOURCES.txt

@@ -1,3 +1,4 @@
+AUTHORS.md
 LICENSE
 README.md
 pyproject.toml
@@ -33,4 +34,6 @@ src2purl.egg-info/SOURCES.txt
 src2purl.egg-info/dependency_links.txt
 src2purl.egg-info/entry_points.txt
 src2purl.egg-info/requires.txt
-src2purl.egg-info/top_level.txt
+src2purl.egg-info/top_level.txt
+tests/test_integration.py
+tests/test_src2purl.py

src2purl-1.4.0/tests/test_integration.py

@@ -0,0 +1,35 @@
+"""Integration tests for src2purl."""
+
+import pytest
+from pathlib import Path
+
+
+def test_project_imports():
+    """Test that project modules can be imported."""
+    # This will be customized per project
+    assert True
+
+
+def test_documentation_exists():
+    """Test that documentation files exist."""
+    project_root = Path(__file__).parent.parent
+
+    docs = [
+        "README.md",
+        "CONTRIBUTING.md",
+        "AUTHORS.md",
+    ]
+
+    for doc in docs:
+        doc_path = project_root / doc
+        assert doc_path.exists(), f"Documentation {doc} is missing"
+
+
+def test_workflow_files_exist():
+    """Test that GitHub workflow files exist."""
+    project_root = Path(__file__).parent.parent
+    workflows_dir = project_root / ".github" / "workflows"
+
+    if workflows_dir.exists():
+        workflow_files = list(workflows_dir.glob("*.yml"))
+        assert len(workflow_files) > 0, "No workflow files found"

src2purl-1.4.0/tests/test_src2purl.py

@@ -0,0 +1,83 @@
+"""Tests for src2purl package."""
+
+import pytest
+import sys
+from pathlib import Path
+
+
+def test_package_import():
+    """Test that the package can be imported."""
+    try:
+        import src2purl
+        assert True
+    except ImportError:
+        # Package might have different structure
+        assert True
+
+
+def test_basic_functionality():
+    """Basic test to ensure pytest works."""
+    assert True
+
+
+def test_python_version():
+    """Test Python version compatibility."""
+    assert sys.version_info >= (3, 8)
+
+
+class TestPackageStructure:
+    """Test package structure and configuration."""
+
+    def test_project_root_exists(self):
+        """Test that project root exists."""
+        project_root = Path(__file__).parent.parent
+        assert project_root.exists()
+
+    def test_package_directory_exists(self):
+        """Test that package directory exists."""
+        project_root = Path(__file__).parent.parent
+        package_dir = project_root / "src2purl"
+        # Some projects might have different structure
+        assert project_root.exists()
+
+    def test_pyproject_toml_exists(self):
+        """Test that pyproject.toml exists."""
+        project_root = Path(__file__).parent.parent
+        pyproject = project_root / "pyproject.toml"
+        assert pyproject.exists()
+
+
+@pytest.mark.parametrize("required_file", [
+    "README.md",
+    "LICENSE",
+    "pyproject.toml",
+])
+def test_required_files_exist(required_file):
+    """Test that required project files exist."""
+    project_root = Path(__file__).parent.parent
+    file_path = project_root / required_file
+    assert file_path.exists(), f"{required_file} not found"
+
+
+def test_no_syntax_errors():
+    """Test that the package has no syntax errors."""
+    import ast
+    import os
+
+    project_root = Path(__file__).parent.parent
+    package_dir = project_root / "src2purl"
+
+    if package_dir.exists():
+        for root, dirs, files in os.walk(package_dir):
+            # Skip __pycache__ directories
+            dirs[:] = [d for d in dirs if d != '__pycache__']
+
+            for file in files:
+                if file.endswith('.py'):
+                    file_path = Path(root) / file
+                    try:
+                        with open(file_path, 'r', encoding='utf-8') as f:
+                            source = f.read()
+                        ast.parse(source)
+                    except SyntaxError as e:
+                        pytest.fail(f"Syntax error in {file_path}: {e}")