label-studio-sdk 1.0.14__tar.gz → 1.0.16__tar.gz

{label_studio_sdk-1.0.14 → label_studio_sdk-1.0.16}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: label-studio-sdk
-Version: 1.0.14
+Version: 1.0.16
 Summary: 
 Requires-Python: >=3.9,<4
 Classifier: Intended Audience :: Developers

{label_studio_sdk-1.0.14 → label_studio_sdk-1.0.16}/pyproject.toml

@@ -3,7 +3,7 @@ name = "label-studio-sdk"
 
 [tool.poetry]
 name = "label-studio-sdk"
-version = "1.0.14"
+version = "1.0.16"
 description = ""
 readme = "README.md"
 authors = []

label_studio_sdk-1.0.16/src/label_studio_sdk/converter/README.md

@@ -0,0 +1,441 @@
+# Note
+
+The old repository `label-studio-converter` has been archived and merged into the Label Studio SDK at the current folder:
+https://github.com/HumanSignal/label-studio-sdk/tree/master/src/label_studio_sdk/converter
+
+# Label Studio Converter
+
+[Website](https://labelstud.io/) • [Docs](https://labelstud.io/guide) • [Twitter](https://twitter.com/heartexlabs) • [Join Slack Community <img src="https://app.heartex.ai/docs/images/slack-mini.png" width="18px"/>](https://slack.labelstud.io)
+
+## Table of Contents
+
+- [Introduction](#introduction)
+- [Examples](#examples)
+    - [JSON](#json)
+    - [CSV](#csv)
+    - [CoNLL 2003](#conll-2003)
+    - [COCO](#coco)
+    - [Pascal VOC XML](#pascal-voc-xml)
+- [YOLO to Label Studio Converter](#yolo-to-label-studio-converter)
+  - [Usage](#usage)
+  - [Tutorial: Importing YOLO Pre-Annotated Images to Label Studio using Local Storage](#tutorial-importing-yolo-pre-annotated-images-to-label-studio-using-local-storage)
+- [Contributing](#contributing)
+- [License](#license)
+
+## Introduction
+
+Label Studio Format Converter helps you to encode labels into the format of your favorite machine learning library.
+
+## Examples
+
+#### JSON
+**Running from the command line:**
+
+```bash
+pip install -U label-studio-converter
+python label-studio-converter export -i exported_tasks.json -c examples/sentiment_analysis/config.xml -o output_dir -f CSV
+```
+
+**Running from python:**
+```python
+from label_studio_converter import Converter
+
+c = Converter('examples/sentiment_analysis/config.xml')
+c.convert_to_json('examples/sentiment_analysis/completions/', 'tmp/output.json')
+```
+
+Getting output file: `tmp/output.json`
+```json
+[
+  {
+    "reviewText": "Good case, Excellent value.",
+    "sentiment": "Positive"
+  },
+  {
+    "reviewText": "What a waste of money and time!",
+    "sentiment": "Negative"
+  },
+  {
+    "reviewText": "The goose neck needs a little coaxing",
+    "sentiment": "Neutral"
+  }
+]
+```
+
+Use cases: any tasks
+
+
+#### CSV
+Running from the command line:
+```bash
+python label_studio_converter/cli.py --input examples/sentiment_analysis/completions/ --config examples/sentiment_analysis/config.xml --output output_dir --format CSV --csv-separator $'\t'
+```
+
+Running from python:
+```python
+from label_studio_converter import Converter
+
+c = Converter('examples/sentiment_analysis/config.xml')
+c.convert_to_csv('examples/sentiment_analysis/completions/', 'output_dir', sep='\t', header=True)
+```
+
+Getting output file `tmp/output.tsv`:
+```tsv
+reviewText	sentiment
+Good case, Excellent value.	Positive
+What a waste of money and time!	Negative
+The goose neck needs a little coaxing	Neutral
+```
+
+Use cases: any tasks
+
+#### CoNLL 2003
+
+Running from the command line:
+```bash
+python label_studio_converter/cli.py --input examples/named_entity/completions/ --config examples/named_entity/config.xml --output tmp/output.conll --format CONLL2003
+```
+
+Running from python:
+```python
+from label_studio_converter import Converter
+
+c = Converter('examples/named_entity/config.xml')
+c.convert_to_conll2003('examples/named_entity/completions/', 'tmp/output.conll')
+```
+
+Getting output file `tmp/output.conll`
+```text
+-DOCSTART- -X- O
+Showers -X- _ O
+continued -X- _ O
+throughout -X- _ O
+the -X- _ O
+week -X- _ O
+in -X- _ O
+the -X- _ O
+Bahia -X- _ B-Location
+cocoa -X- _ O
+zone, -X- _ O
+...
+```
+
+Use cases: text tagging
+
+
+#### COCO
+Running from the command line:
+```bash
+python label_studio_converter/cli.py --input examples/image_bbox/completions/ --config examples/image_bbox/config.xml --output tmp/output.json --format COCO --image-dir tmp/images
+```
+
+Running from python:
+```python
+from label_studio_converter import Converter
+
+c = Converter('examples/image_bbox/config.xml')
+c.convert_to_coco('examples/image_bbox/completions/', 'tmp/output.conll', output_image_dir='tmp/images')
+```
+
+Output images could be found in `tmp/images`
+
+Getting output file `tmp/output.json`
+```json
+{
+  "images": [
+    {
+      "width": 800,
+      "height": 501,
+      "id": 0,
+      "file_name": "tmp/images/62a623a0d3cef27a51d3689865e7b08a"
+    }
+  ],
+  "categories": [
+    {
+      "id": 0,
+      "name": "Planet"
+    },
+    {
+      "id": 1,
+      "name": "Moonwalker"
+    }
+  ],
+  "annotations": [
+    {
+      "id": 0,
+      "image_id": 0,
+      "category_id": 0,
+      "segmentation": [],
+      "bbox": [
+        299,
+        6,
+        377,
+        260
+      ],
+      "ignore": 0,
+      "iscrowd": 0,
+      "area": 98020
+    },
+    {
+      "id": 1,
+      "image_id": 0,
+      "category_id": 1,
+      "segmentation": [],
+      "bbox": [
+        288,
+        300,
+        132,
+        90
+      ],
+      "ignore": 0,
+      "iscrowd": 0,
+      "area": 11880
+    }
+  ],
+  "info": {
+    "year": 2019,
+    "version": "1.0",
+    "contributor": "Label Studio"
+  }
+}
+```
+
+Use cases: image object detection
+
+#### Pascal VOC XML
+Running from the command line:
+```bash
+python label_studio_converter/cli.py --input examples/image_bbox/completions/ --config examples/image_bbox/config.xml --output tmp/voc-annotations --format VOC --image-dir tmp/images
+```
+
+Running from python:
+```python
+from label_studio_converter import Converter
+
+c = Converter('examples/image_bbox/config.xml')
+c.convert_to_voc('examples/image_bbox/completions/', 'tmp/output.conll', output_image_dir='tmp/images')
+```
+
+Output images can be found in `tmp/images`
+
+Corresponding annotations could be found in `tmp/voc-annotations/*.xml`:
+```xml
+<?xml version="1.0" encoding="utf-8"?>
+<annotation>
+<folder>tmp/images</folder>
+<filename>62a623a0d3cef27a51d3689865e7b08a</filename>
+<source>
+<database>MyDatabase</database>
+<annotation>COCO2017</annotation>
+<image>flickr</image>
+<flickrid>NULL</flickrid>
+</source>
+<owner>
+<flickrid>NULL</flickrid>
+<name>Label Studio</name>
+</owner>
+<size>
+<width>800</width>
+<height>501</height>
+<depth>3</depth>
+</size>
+<segmented>0</segmented>
+<object>
+<name>Planet</name>
+<pose>Unspecified</pose>
+<truncated>0</truncated>
+<difficult>0</difficult>
+<bndbox>
+<xmin>299</xmin>
+<ymin>6</ymin>
+<xmax>676</xmax>
+<ymax>266</ymax>
+</bndbox>
+</object>
+<object>
+<name>Moonwalker</name>
+<pose>Unspecified</pose>
+<truncated>0</truncated>
+<difficult>0</difficult>
+<bndbox>
+<xmin>288</xmin>
+<ymin>300</ymin>
+<xmax>420</xmax>
+<ymax>390</ymax>
+</bndbox>
+</object>
+</annotation>
+```
+
+Use cases: image object detection
+
+--------
+
+# YOLO to Label Studio Converter 
+
+### YOLO directory structure
+
+Check the structure of YOLO folder first, keep in mind that the root is `/yolo/datasets/one`. 
+
+```
+/yolo/datasets/one
+  images
+   - 1.jpg
+   - 2.jpg
+   - ...
+  labels
+   - 1.txt
+   - 2.txt
+
+  classes.txt
+```
+
+*classes.txt example*
+
+```
+Airplane
+Car
+```
+
+### Usage
+
+```
+label-studio-converter import yolo -i /yolo/datasets/one -o ls-tasks.json --image-root-url "/data/local-files/?d=one/images"
+```
+Where the URL path from `?d=` is relative to the path you set in `LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT`.
+
+**Note for Local Storages** 
+  * It's very important to set `LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=/yolo/datasets` (**not** to `/yolo/datasets/one`, but **`/yolo/datasets`**) for Label Studio to run.
+  * [Add a new Local Storage](https://labelstud.io/guide/storage#Local-storage) in the project settings and set **Absolute local path** to `/yolo/datasets/one/images` (or `c:\yolo\datasets\one\images` for Windows). 
+
+**Note for Cloud Storages**
+  * Use `--image-root-url` to make correct prefixes for task URLs, e.g. `--image-root-url s3://my-bucket/yolo/datasets/one`.
+  * [Add a new Cloud Storage](https://labelstud.io/guide/storage) in the project settings with the corresponding bucket and prefix.
+
+**Help command**
+
+```
+label-studio-converter import yolo -h
+
+usage: label-studio-converter import yolo [-h] -i INPUT [-o OUTPUT]
+                                          [--to-name TO_NAME]
+                                          [--from-name FROM_NAME]
+                                          [--out-type OUT_TYPE]
+                                          [--image-root-url IMAGE_ROOT_URL]
+                                          [--image-ext IMAGE_EXT]
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -i INPUT, --input INPUT
+                        directory with YOLO where images, labels, notes.json
+                        are located
+  -o OUTPUT, --output OUTPUT
+                        output file with Label Studio JSON tasks
+  --to-name TO_NAME     object name from Label Studio labeling config
+  --from-name FROM_NAME
+                        control tag name from Label Studio labeling config
+  --out-type OUT_TYPE   annotation type - "annotations" or "predictions"
+  --image-root-url IMAGE_ROOT_URL
+                        root URL path where images will be hosted, e.g.:
+                        http://example.com/images or s3://my-bucket
+  --image-ext IMAGE_EXT
+                        image extension to search: .jpg, .png
+```
+
+
+## Tutorial: Importing YOLO Pre-Annotated Images to Label Studio using Local Storage
+
+This tutorial will guide you through the process of importing a folder with YOLO annotations into Label Studio for further annotation. 
+We'll cover setting up your environment, converting YOLO annotations to Label Studio's format, and importing them into your project.
+
+
+### Prerequisites
+- Label Studio installed locally
+- YOLO annotated images and corresponding .txt label files in the directory `/yolo/datasets/one`.
+- label-studio-converter installed (available via `pip install label-studio-converter`)
+
+### Step 1: Set Up Your Environment and Run Label Studio
+Before starting Label Studio, set the following environment variables to enable Local Storage file serving:
+
+Unix systems: 
+```
+export LABEL_STUDIO_LOCAL_FILES_SERVING_ENABLED=true
+export LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=/yolo/datasets
+label-studio
+```
+
+Windows: 
+```
+set LABEL_STUDIO_LOCAL_FILES_SERVING_ENABLED=true
+set LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=C:\\yolo\\datasets
+label-studio
+```
+
+Replace `/yolo/datasets` with the actual path to your YOLO datasets directory.
+
+### Step 2: Setup Local Storage
+1. Create a new project.
+2. Go to the project settings and select **Cloud Storage**. 
+3. Click **Add Source Storage** and select **Local files** from the **Storage Type** options.  
+3. Set the **Absolute local path** to `/yolo/datasets/one/images` or `c:\yolo\datasets\one\images` on Windows.
+4. Click `Add storage`.
+
+Check more details about Local Storages [in the documentation](https://labelstud.io/guide/storage.html#Local-storage).
+
+### Step 3: Verify Image Access
+Before importing the converted annotations from YOLO, verify that you can access an image from your Local storage via Label Studio. Open a new browser tab and enter the following URL:
+
+```
+http://localhost:8080/data/local-files/?d=one/images/<your_image>.jpg
+```
+
+Replace `one/images/<your_image>.jpg` with the path to one of your images. The image should display **in the new tab of the browser**.
+If you can't open an image, the Local Storage configuration is incorrect. The most likely reason is that you made a mistake when specifying your `Path` in Local Storage settings or in `LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT`. 
+
+**Note:** The URL path from `?d=` should be relative to `LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=/yolo/datasets`, 
+it means that the real path will be `/yolo/datasets/one/images/<your_image>.jpg` and this image should exist on your hard drive.
+
+### Step 4: Convert YOLO Annotations
+Use the label-studio-converter to convert your YOLO annotations to a format that Label Studio can understand:
+
+```
+label-studio-converter import yolo -i /yolo/datasets/one -o output.json --image-root-url "/data/local-files/?d=one/images"
+```
+
+### Step 5: Import Converted Annotations
+Now import the `output.json` file into Label Studio:
+1. Go to your Label Studio project.
+2. From the Data Manager, click **Import**.
+3. Select the `output.json` file and import it.
+
+### Step 6: Verify Annotations
+After importing, you should see your images with the pre-annotated bounding boxes in Label Studio. Verify that the annotations are correct and make any necessary adjustments.
+
+### Troubleshooting
+If you encounter issues with paths or image access, ensure that:
+- The LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT is set correctly.
+- The `--image-root-url` in the conversion command matches the relative path:
+```
+`Absolute local path from Local Storage Settings` - `LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT` = `path for --image_root_url`
+```
+e.g.:
+```
+/yolo/datasets/one/images - /yolo/datasets/ = one/images
+```
+- The Local Storage in Label Studio is set up correctly with the Absolute local path to your images (`/yolo/datasets/one/images`)
+- For more details, refer to the documentation on [importing pre-annotated data](https://labelstud.io/guide/predictions.html) and [setting up Cloud Storages](https://labelstud.io/guide/storage).
+
+------------
+
+# Contributing
+
+We would love to get your help for creating converters to other models. Please feel free to create pull requests.
+
+- [Contributing Guideline](https://github.com/heartexlabs/label-studio/blob/develop/CONTRIBUTING.md)
+- [Code Of Conduct](https://github.com/heartexlabs/label-studio/blob/develop/CODE_OF_CONDUCT.md)
+
+# License
+
+This software is licensed under the [Apache 2.0 LICENSE](/LICENSE) © [Heartex](https://www.heartex.com/). 2020
+
+<img src="https://github.com/heartexlabs/label-studio/blob/master/images/opossum_looking.png?raw=true" title="Hey everyone!" height="140" width="140" />

{label_studio_sdk-1.0.14 → label_studio_sdk-1.0.16}/src/label_studio_sdk/converter/converter.py

@@ -363,6 +363,23 @@ class Converter(object):
                 input_tag_types.add(input_tag["type"])
 
         all_formats = [f.name for f in Format]
+
+        # Check if KeyPointLabels exists without RectangleLabels
+        has_keypoint_labels = "KeyPointLabels" in output_tag_types
+        has_rectangle_labels = "RectangleLabels" in output_tag_types
+
+        # If config has KeyPointLabels but no RectangleLabels, exclude COCO and YOLO formats
+        # RectangleLabels are required for COCO and YOLO formats for category mapping
+        if has_keypoint_labels and not has_rectangle_labels:
+            if Format.COCO.name in all_formats:
+                all_formats.remove(Format.COCO.name)
+            if Format.COCO_WITH_IMAGES.name in all_formats:
+                all_formats.remove(Format.COCO_WITH_IMAGES.name)
+            if Format.YOLO.name in all_formats:
+                all_formats.remove(Format.YOLO.name)
+            if Format.YOLO_WITH_IMAGES.name in all_formats:
+                all_formats.remove(Format.YOLO_WITH_IMAGES.name)
+
         if not ("Text" in input_tag_types and "Labels" in output_tag_types):
             all_formats.remove(Format.CONLL2003.name)
         if is_mig or not (

{label_studio_sdk-1.0.14 → label_studio_sdk-1.0.16}/src/label_studio_sdk/converter/keypoints.py

@@ -89,6 +89,16 @@ def get_bbox_coco(keypoints, kp_order):
 
 
 def process_keypoints_for_coco(keypoint_labels, kp_order, annotation_id, image_id, category_name_to_id):
+    if not kp_order:
+        raise ValueError(
+            "No keypoint order found. Please ensure that KeyPointLabels in your label config "
+            "have 'model_index' attributes defined. For example:\n"
+            "<KeyPointLabels name=\"keypoints\" toName=\"image\">\n"
+            "  <Label value=\"eye\" model_index=\"0\"/>\n"
+            "  <Label value=\"nose\" model_index=\"1\"/>\n"
+            "</KeyPointLabels>"
+        )
+
     keypoints = [0] * (len(kp_order) * 3)
 
     for kp in keypoint_labels:

{label_studio_sdk-1.0.14 → label_studio_sdk-1.0.16}/src/label_studio_sdk/tokens/client_ext.py

@@ -1,7 +1,7 @@
 import threading
 import typing
 from datetime import datetime, timezone
-import inspect
+import ssl
 
 import httpx
 import jwt
@@ -78,34 +78,63 @@ class TokensClientExt:
         
         return self._access_token
 
+    def _get_client_params(self, existing_client: httpx.AsyncClient) -> dict:
+        """Extract parameters from an existing client to create a new one.
+
+        Args:
+            existing_client: The existing client to extract parameters from.
+
+        Returns:
+            dict: Parameters for creating a new client.
+        """
+        return {
+            'auth': existing_client.auth,
+            'params': existing_client.params,
+            'headers': existing_client.headers,
+            'cookies': existing_client.cookies,
+            'timeout': existing_client.timeout,
+            'follow_redirects': existing_client.follow_redirects,
+            'max_redirects': existing_client.max_redirects,
+            'event_hooks': existing_client.event_hooks,
+            'base_url': existing_client.base_url,
+            'trust_env': existing_client.trust_env,
+            'default_encoding': existing_client._default_encoding,
+            'verify': existing_client._transport._pool._ssl_context.verify_mode != ssl.CERT_NONE,
+            'http1': existing_client._transport._pool._http1,
+            'http2': existing_client._transport._pool._http2,
+            'limits': httpx.Limits(
+                max_connections=existing_client._transport._pool._max_connections,
+                max_keepalive_connections=existing_client._transport._pool._max_keepalive_connections,
+                keepalive_expiry=existing_client._transport._pool._keepalive_expiry
+            )
+        }
+
     def refresh(self) -> AccessTokenResponse:
         """Refresh the access token and return the token response."""
-        # We don't do this often, just use a separate sync httpx client for simplicity here
-        # (avoids complicated state management and sync vs async handling)
-        # Create a new client with the same parameters as the existing one
         existing_client = self._client_wrapper.httpx_client.httpx_client
 
-        # Get all parameters from httpx.Client.__init__
-        client_params = {}
-        sig = inspect.signature(httpx.Client.__init__)
-        for param_name in sig.parameters:
-            if param_name != 'self':  # Skip 'self' parameter
-                try:
-                    value = getattr(existing_client, param_name, None)
-                    if value is not None:
-                        client_params[param_name] = value
-                except AttributeError:
-                    continue
-
-        with httpx.Client(**client_params) as sync_client:
-            response = sync_client.request(
+        # For sync client, use it directly
+        if isinstance(existing_client, httpx.Client):
+            print(f"\nverify:{existing_client._transport._pool._ssl_context.verify_mode != ssl.CERT_NONE}")
+            response = existing_client.request(
                 method="POST",
                 url=f"{self._base_url}/api/token/refresh/",
                 json={"refresh": self._api_key},
                 headers={"Content-Type": "application/json"},
             )
+        else:
+            # If an async client was used, get all parameters from the client to init a new sync client
+            client_params = self._get_client_params(existing_client)
+
+            with httpx.Client(**client_params) as sync_client:
+                response = sync_client.request(
+                    method="POST",
+                    url=f"{self._base_url}/api/token/refresh/",
+                    json={"refresh": self._api_key},
+                    headers={"Content-Type": "application/json"},
+                )
             
-            if response.status_code == 200:
-                return AccessTokenResponse.parse_obj(response.json())
-            else:
-                raise ApiError(status_code=response.status_code, body=response.json())
+        if response.status_code == 200:
+            return AccessTokenResponse.parse_obj(response.json())
+        else:
+            raise ApiError(status_code=response.status_code, body=response.json())