caltechdata-api 1.5.0__tar.gz → 1.6.0__tar.gz

{caltechdata_api-1.5.0 → caltechdata_api-1.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: caltechdata_api
-Version: 1.5.0
+Version: 1.6.0
 Summary: Python wrapper for CaltechDATA API.
 Home-page: https://github.com/caltechlibrary/caltechdata_api
 Author: Thomas E Morrell
@@ -18,38 +18,56 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 
 
-# caltechdata_api
+# CaltechDATA API Python Library
 
-Python library for using the CaltechDATA API
+The `caltechdata_api` Python library provides a convenient interface for interacting with the CaltechDATA API. It allows users to write files, create DataCite 4 standard JSON records, edit existing records, and retrieve metadata from the CaltechDATA repository.
 
-- caltechdata_write write files and a DataCite 4 standard json record to CaltechDATA repository
-- caltechdata_edit edits records in CaltechDATA
-- get_metadata gets metadata from CaltechDATA records
+## Features
 
-Requires Python 3 (Recommended via Anaconda https://www.anaconda.com/download) with reqests library.
+### Writing and Editing Records
+- `caltechdata_write`: Writes files and a DataCite 4 standard JSON record to the CaltechDATA repository.
+- `caltechdata_edit`: Edits existing records in CaltechDATA.
+
+### Metadata Operations
+- `get_metadata`: Retrieves metadata from CaltechDATA records.
+
+## Requirements
+
+- Python 3 (Anaconda is recommended)
+- `requests` library
+
+## Installation
+
+Install the library via pip:
+
+```shell
+pip install caltechdata_api
+```
 
 ## Examples
 
 There are some example python scripts in the GitHub repository.
 
-Create a record:
+###Create a record:
 
 ```shell
 python write.py example.json -fnames logo.gif
-pbkn6-m9y63
+# Output: pbkn6-m9y63 (unique identifier)
 ```
-The response will be the unique identifier for the record. You can put this at
+> The response will be the unique identifier for the record. You can put this at
 the end of a url to visit the record (e.g.
 https://data.caltechlibrary.dev/records/pbkn6-m9y63)
 
-Edit a record (make changes to the example.json file to see a change)
+###Edit a record 
+Make changes to the example.json file to see a change)
 ```
 python edit.py example.json -id pbkn6-m9y63
 10.33569/pbkn6-m9y63
 ```
-The response is the DOI for the record, which includes the unique identifier
+> The response is the DOI for the record, which includes the unique identifier
 for the record in the default configuration.
 
+## Using Custom DOIs 
 Some groups have worked with the library to create custom DOIs. These can be
 passed in the metadata like:
 
@@ -66,18 +84,16 @@ python edit.py example_custom.json -id m6zxz-p4j22
 
 This returns the custom DOI of the record if it is successful.
 
-## Setup 
 
-Install by typing 'pip install caltechdata_api'
+## Setup and Authentication
 
-## Usage
+1. Acquire a personal access token from your CaltechDATA account (found under "Applications" at the top right of your screen).
+2. Copy the token to a file named token.bash.
+3. Load the token in the command line using source token.bash.
 
-You need to acquire a personal access token from your CaltechDATA account
-(find it at the top right of your screen under "Applications").
-Then copy the token to token.bash.  Type `source token.bash` in 
-the command line to load the token.  
+## Note on Testing
 
-Only test your application on the test repository (data.caltechlibrary.dev).  Testing the API on the public 
+Only test your application on the test repository (`data.caltechlibrary.dev`).  Testing the API on the public 
 repository will generate junk records that are annoying to delete.
 
 

caltechdata_api-1.6.0/README.md

@@ -0,0 +1,78 @@
+# CaltechDATA API Python Library
+
+The `caltechdata_api` Python library provides a convenient interface for interacting with the CaltechDATA API. It allows users to write files, create DataCite 4 standard JSON records, edit existing records, and retrieve metadata from the CaltechDATA repository.
+
+## Features
+
+### Writing and Editing Records
+- `caltechdata_write`: Writes files and a DataCite 4 standard JSON record to the CaltechDATA repository.
+- `caltechdata_edit`: Edits existing records in CaltechDATA.
+
+### Metadata Operations
+- `get_metadata`: Retrieves metadata from CaltechDATA records.
+
+## Requirements
+
+- Python 3 (Anaconda is recommended)
+- `requests` library
+
+## Installation
+
+Install the library via pip:
+
+```shell
+pip install caltechdata_api
+```
+
+## Examples
+
+There are some example python scripts in the GitHub repository.
+
+###Create a record:
+
+```shell
+python write.py example.json -fnames logo.gif
+# Output: pbkn6-m9y63 (unique identifier)
+```
+> The response will be the unique identifier for the record. You can put this at
+the end of a url to visit the record (e.g.
+https://data.caltechlibrary.dev/records/pbkn6-m9y63)
+
+###Edit a record 
+Make changes to the example.json file to see a change)
+```
+python edit.py example.json -id pbkn6-m9y63
+10.33569/pbkn6-m9y63
+```
+> The response is the DOI for the record, which includes the unique identifier
+for the record in the default configuration.
+
+## Using Custom DOIs 
+Some groups have worked with the library to create custom DOIs. These can be
+passed in the metadata like:
+
+```shell
+python write.py example_custom.json -fnames logo.gif
+m6zxz-p4j22
+```
+
+And then you can edit with
+```
+python edit.py example_custom.json -id m6zxz-p4j22
+10.5281/inveniordm.1234
+```
+
+This returns the custom DOI of the record if it is successful.
+
+
+## Setup and Authentication
+
+1. Acquire a personal access token from your CaltechDATA account (found under "Applications" at the top right of your screen).
+2. Copy the token to a file named token.bash.
+3. Load the token in the command line using source token.bash.
+
+## Note on Testing
+
+Only test your application on the test repository (`data.caltechlibrary.dev`).  Testing the API on the public 
+repository will generate junk records that are annoying to delete.
+

{caltechdata_api-1.5.0 → caltechdata_api-1.6.0}/caltechdata_api/caltechdata_edit.py

@@ -33,7 +33,6 @@ def caltechdata_accept(ids, token=None, production=False):
     }
 
     for idv in ids:
-
         result = requests.get(
             url + "/api/records/" + idv + "/draft/review", headers=headers
         )

{caltechdata_api-1.5.0 → caltechdata_api-1.6.0}/caltechdata_api/caltechdata_write.py

@@ -67,9 +67,9 @@ def write_files_rdm(files, file_link, headers, f_headers, s3=None):
 def add_file_links(
     metadata, file_links, file_descriptions=[], additional_descriptions="", s3_link=None
 ):
-    # Currently configured for OSN S3 links
+    # Currently configured for S3 links, assuming all are at same endpoint
     link_string = ""
-    endpoint = "https://renc.osn.xsede.org/"
+    endpoint = "https://" + file_links[0].split("/")[2]
     s3 = s3fs.S3FileSystem(anon=True, client_kwargs={"endpoint_url": endpoint})
     index = 0
     for link in file_links:
@@ -102,7 +102,6 @@ def add_file_links(
 
 
 def send_to_community(review_link, data, headers, publish, community, message=None):
-
     if not message:
         message = "This record is submitted automatically with the CaltechDATA API"
 

caltechdata_api-1.6.0/caltechdata_api/cli.py

@@ -0,0 +1,608 @@
+import argparse
+import requests
+import s3fs
+from caltechdata_api import caltechdata_write, caltechdata_edit
+from .md_to_json import parse_readme_to_json
+import json
+import os
+import configparser
+
+CALTECHDATA_API = "https://data.caltech.edu/api/names?q=identifiers.identifier:{}"
+ORCID_API = "https://orcid.org/"
+HEADERS = {"Accept": "application/json"}
+
+name = ""
+affiliationIdentifierScheme = ""
+affiliation_identifier = ""
+
+awardNumber = ""
+awardTitle = ""
+funderIdentifier = ""
+funderIdentifierType = ""
+funderName = ""
+
+
+CONFIG_FILE = "caltechdata_config.ini"
+
+
+def get_or_set_token():
+    config = configparser.ConfigParser()
+
+    if os.path.isfile(CONFIG_FILE):
+        config.read(CONFIG_FILE)
+        if "CaltechDATA" in config and "token" in config["CaltechDATA"]:
+            return config["CaltechDATA"]["token"]
+    else:
+        while True:
+            token = get_user_input("Enter your CaltechDATA token: ")
+            confirm_token = get_user_input("Confirm your CaltechDATA token: ")
+            if token == confirm_token:
+                config.add_section("CaltechDATA")
+                config.set("CaltechDATA", "token", token)
+                with open(CONFIG_FILE, "w") as configfile:
+                    config.write(configfile)
+                return token
+            else:
+                print("Tokens do not match. Please try again.")
+
+
+def welcome_message():
+    print("Welcome to CaltechDATA CLI")
+
+
+def get_user_input(prompt, required=True):
+    while True:
+        user_input = input(prompt)
+        if required and not user_input:
+            print("This field is required. Please provide a value.")
+        else:
+            return user_input
+
+
+def confirm_upload():
+    while True:
+        user_input = input("Do you want to send this record to CaltechDATA? (y/n): ")
+        if user_input.lower() == "y":
+            return True
+        elif user_input.lower() == "n":
+            print("Upload canceled.")
+            return False
+        else:
+            print("Invalid input. Please enter 'y' or 'n'.")
+
+
+def check_award_number(award_number):
+    response = requests.get(
+        f"https://data.caltech.edu/api/awards?q=number:{award_number}"
+    )
+    data = response.json()
+    total_hits = data.get("hits", {}).get("total", 0)
+    return total_hits > 0
+
+
+def get_funding_entries():
+    while True:
+        try:
+            num_entries = int(
+                input("How many funding entries do you want to provide? ")
+            )
+            if num_entries >= 0:
+                return num_entries
+            else:
+                print("Please enter a non-negative integer.")
+        except ValueError:
+            print("Please enter a valid integer.")
+
+
+def validate_funder_identifier(funder_identifier):
+    response = requests.get(f"https://api.ror.org/organizations/{funder_identifier}")
+    if response.status_code == 200:
+        return True
+    else:
+        return False
+
+
+def get_funding_details():
+    award_number = get_user_input("Enter the award number for funding: ")
+    award_exists = check_award_number(award_number)
+    if not award_exists:
+        print(
+            f"""Error: No award with number '{award_number}' found in
+              CaltechDATA. You will need to provide more details about the
+              funding."""
+        )
+    award_title = get_user_input("Enter the award title for funding: ")
+    while True:
+        funder_identifier = get_user_input("Enter the funder ROR (https://ror.org): ")
+        if validate_funder_identifier(funder_identifier):
+            break
+        else:
+            print(
+                """This funder identifier is not a ROR. Please enter a valid
+                  ROR identifier (without the url). For example the ROR for the
+                  NSF is 021nxhr62."""
+            )
+    print("-" * 10)
+    return {
+        "awardNumber": award_number,
+        "awardTitle": award_title,
+        "funderIdentifier": funder_identifier,
+        "funderIdentifierType": "ROR",
+    }
+
+
+def parse_arguments():
+    welcome_message()
+    args = {}
+    args["title"] = get_user_input("Enter the title of the dataset: ")
+    args["description"] = get_user_input(
+        "Enter the abstract or description of the dataset: "
+    )
+    print("License options:")
+    print("1. Creative Commons Zero Waiver (cc-zero)")
+    print("2. Creative Commons Attribution (cc-by)")
+    print("3. Creative Commons Attribution Non Commercial (cc-by-nc)")
+
+    # Prompt user to select a license
+    while True:
+        license_number = input(
+            "Enter the number corresponding to the desired license: "
+        )
+        if license_number.isdigit() and 1 <= int(license_number) <= 8:
+            # Valid license number selected
+            args["license"] = {
+                "1": "cc0-1.0",
+                "2": "cc-by-4.0",
+                "3": "cc-by-nc-4.0",
+            }[license_number]
+            break
+        else:
+            print("Invalid input. Please enter a number between 1 and 8.")
+
+    while True:
+        orcid = get_user_input("Enter your ORCID identifier: ")
+        family_name, given_name = get_names(orcid)
+        if family_name is not None and given_name is not None:
+            args["orcid"] = orcid
+            break  # Break out of the loop if names are successfully retrieved
+        retry = input("Do you want to try again? (y/n): ")
+        if retry.lower() != "y":
+            print("Exiting program.")
+            return
+    # Optional arguments
+    num_funding_entries = get_funding_entries()
+    funding_references = []
+    for _ in range(num_funding_entries):
+        funding_references.append(get_funding_details())
+    args["fundingReferences"] = funding_references
+    return args
+
+
+def query_caltechdata_api(orcid):
+    response = requests.get(CALTECHDATA_API.format(orcid), headers=HEADERS)
+    return response.json()
+
+
+def query_orcid_api(orcid):
+    response = requests.get(ORCID_API + orcid, headers=HEADERS)
+    return response.json()
+
+
+def get_names(orcid):
+    caltechdata_response = query_caltechdata_api(orcid)
+    global affiliationIdentifierScheme, affiliation_identifier, name
+    if caltechdata_response.get("hits", {}).get("hits"):
+        hit = caltechdata_response["hits"]["hits"][0]
+        family_name = hit.get("family_name", "")
+        given_name = hit.get("given_name", "")
+        affiliation_identifier = "05dxps055"
+        affiliationIdentifierScheme = "ROR"
+        name = "California Institute of Technology"
+
+    else:
+        orcid_link = "https://orcid.org/"
+        headers = {"Accept": "application/json"}
+        orcid_response = requests.get(orcid_link + orcid, headers=headers)
+        try:
+            orcid_data = orcid_response.json()
+            name_info = orcid_data.get("person", {}).get("name", {})
+            family_name = name_info.get("family-name", {}).get("value", "")
+            given_name = name_info.get("given-names", {}).get("value", "")
+        except json.decoder.JSONDecodeError:
+            print(
+                f"Error: ORCID identifier not found or invalid. Please check the ORCID identifier and try again."
+            )
+            return None, None
+    return family_name, given_name
+
+
+def upload_supporting_file(record_id=None):
+    filepath = ""
+    file_link = ""
+    while True:
+        choice = get_user_input(
+            "Do you want to upload or link data files? (upload/link/n): "
+        ).lower()
+        if choice == "link":
+            endpoint = "https://sdsc.osn.xsede.org/"
+            path = "ini230004-bucket01/"
+
+            if not record_id:
+                record_id = get_user_input("Folder where OSN files are uploaded")
+
+            s3 = s3fs.S3FileSystem(anon=True, client_kwargs={"endpoint_url": endpoint})
+            # Find the files
+            files = s3.glob(path + record_id + "/*")
+
+            file_links = []
+
+            for link in files:
+                fname = link.split("/")[-1]
+                if "." not in fname:
+                    # If there is a directory, get files
+                    folder_files = s3.glob(link + "/*")
+                    for file in folder_files:
+                        name = file.split("/")[-1]
+                        if "." not in name:
+                            level_2_files = s3.glob(file + "/*")
+                            for f in level_2_files:
+                                name = f.split("/")[-1]
+                                if "." not in name:
+                                    level_3_files = s3.glob(f + "/*")
+                                    for l3 in level_3_files:
+                                        file_links.append(endpoint + l3)
+                                else:
+                                    file_links.append(endpoint + f)
+                        else:
+                            file_links.append(endpoint + file)
+                else:
+                    file_links.append(endpoint + link)
+            return filepath, file_links
+        elif choice == "upload":
+            print("Current files in the directory:")
+            files = [
+                f for f in os.listdir() if not f.endswith(".json") and os.path.isfile(f)
+            ]
+            print("\n".join(files))
+            filename = get_user_input(
+                "Enter the filename to upload as a supporting file: "
+            )
+            if filename in files:
+                file_size = os.path.getsize(filename)
+                if file_size > 1024 * 1024 * 1024:
+                    file_link = get_user_input(
+                        "Enter the S3 link to the file (File size is more than 1GB): "
+                    )
+                    if file_link:
+                        return filepath, file_link
+                    else:
+                        print("Link is required for files larger than 1GB.")
+                        continue
+                else:
+                    filepath = os.path.abspath(filename)
+                    break
+            else:
+                print(
+                    f"Error: File '{filename}' not found. Please enter a valid filename."
+                )
+        elif choice == "n":
+            break
+        else:
+            print("Invalid input. Please enter 'link' or 'upload' or 'n'.")
+
+    return filepath, file_link
+
+
+def upload_data_from_file():
+    while True:
+        print("Current JSON files in the directory:")
+        files = [f for f in os.listdir() if f.endswith(".json") and os.path.isfile(f)]
+        print("\n".join(files))
+
+        filename = get_user_input(
+            "Enter a README.md or JSON filename to upload to CaltechDATA (or type 'exit' to go back): "
+        )
+
+        if filename.lower() == "exit":
+            return None
+
+        if filename == "README.md":
+            data = parse_readme_to_json(filename)
+            return data
+        else:
+            try:
+                with open(filename, "r") as file:
+                    data = json.load(file)
+                return data
+
+            except json.JSONDecodeError as e:
+                print(f"Error: Invalid JSON format in the file '{filename}'. {str(e)}")
+
+
+def main():
+    choice = get_user_input(
+        "Do you want to create or edit a CaltechDATA record? (create/edit): "
+    ).lower()
+    if choice == "create":
+        create_record()
+    elif choice == "edit":
+        edit_record()
+    else:
+        print("Invalid choice. Please enter 'create' or 'edit'.")
+
+
+def create_record():
+    token = get_or_set_token()
+    print("Using CaltechDATA token:", token)
+    while True:
+        choice = get_user_input(
+            "Do you want to use metadata from an existing file or create new metadata? (existing/create): "
+        ).lower()
+        if choice == "existing":
+            existing_data = upload_data_from_file()
+            filepath, file_link = upload_supporting_file()
+            if existing_data:
+                if filepath != "":
+                    response = caltechdata_write(
+                        existing_data, token, filepath, production=False, publish=False
+                    )
+                elif file_link != "":
+                    response = caltechdata_write(
+                        existing_data,
+                        token,
+                        file_links=[file_link],
+                        s3_link=file_link,
+                        production=False,
+                        publish=False,
+                    )
+                else:
+                    response = caltechdata_write(
+                        existing_data, token, production=False, publish=False
+                    )
+                rec_id = response
+                print(
+                    f"You can view and publish this record at https://data.caltechlibrary.dev/uploads/{rec_id}"
+                )
+                break
+            else:
+                print("Going back to the main menu.")
+        elif choice == "create":
+            args = parse_arguments()
+            family_name, given_name = get_names(args["orcid"])
+            metadata = {
+                "titles": [{"title": args["title"]}],
+                "descriptions": [
+                    {"description": args["description"], "descriptionType": "Abstract"}
+                ],
+                "creators": [
+                    {
+                        "affiliation": [
+                            {
+                                "affiliationIdentifier": affiliation_identifier,
+                                "affiliationIdentifierScheme": affiliationIdentifierScheme,
+                                "name": name,
+                            }
+                        ],
+                        "familyName": family_name,
+                        "givenName": given_name,
+                        "name": f"{family_name}, {given_name}",
+                        "nameIdentifiers": [
+                            {
+                                "nameIdentifier": args["orcid"],
+                                "nameIdentifierScheme": "ORCID",
+                            }
+                        ],
+                        "nameType": "Personal",
+                    }
+                ],
+                "types": {"resourceType": "", "resourceTypeGeneral": "Dataset"},
+                "rightsList": [
+                    {
+                        "rightsIdentifier": args["license"],
+                    }
+                ],
+                "fundingReferences": args["fundingReferences"],
+                "schemaVersion": "http://datacite.org/schema/kernel-4",
+            }
+            filepath, file_link = upload_supporting_file()
+            if confirm_upload():
+                if filepath != "":
+                    response = caltechdata_write(
+                        metadata, token, filepath, production=False, publish=False
+                    )
+                elif file_link != "":
+                    response = caltechdata_write(
+                        metadata,
+                        token,
+                        file_links=[file_link],
+                        production=False,
+                        publish=False,
+                    )
+                else:
+                    response = caltechdata_write(
+                        metadata, token, production=False, publish=False
+                    )
+                rec_id = response
+                print(
+                    f"You can view and publish this record at https://data.caltechlibrary.dev/uploads/{rec_id}"
+                )
+                with open(response + ".json", "w") as file:
+                    json.dump(metadata, file, indent=2)
+                break
+            else:
+                break
+        else:
+            print("Invalid choice. Please enter 'existing' or 'create'.")
+
+
+def edit_record():
+    record_id = input("Enter the CaltechDATA record ID: ")
+    token = get_or_set_token()
+    file_name = download_file_by_id(record_id, token)
+    if file_name:
+        try:
+            # Read the edited metadata file
+            with open(file_name, "r") as file:
+                metadata = json.load(file)
+            response = caltechdata_edit(
+                record_id, metadata, token, production=False, publish=False
+            )
+            if response:
+                print("Metadata edited successfully.")
+            else:
+                print("Failed to edit metadata.")
+        except Exception as e:
+            print(f"An error occurred during metadata editing: {e}")
+    else:
+        print("No metadata file found.")
+    choice = get_user_input("Do you want to add files? (y/n): ").lower()
+    if choice == "y":
+        filepath, file_link = upload_supporting_file(record_id)
+        print(file_link)
+        if filepath != "":
+            response = caltechdata_edit(
+                record_id, token=token, files=filepath, production=False, publish=False
+            )
+        elif file_link != "":
+            response = caltechdata_edit(
+                record_id,
+                metadata,
+                token=token,
+                file_links=file_link,
+                production=False,
+                publish=False,
+            )
+        rec_id = response
+        print(
+            f"You can view and publish this record at https://data.caltechlibrary.dev/uploads/{rec_id}"
+        )
+
+
+def download_file_by_id(record_id, token=None):
+    url = f"https://data.caltechlibrary.dev/api/records/{record_id}"
+
+    headers = {
+        "accept": "application/vnd.datacite.datacite+json",
+    }
+
+    if token:
+        headers["Authorization"] = "Bearer %s" % token
+
+    try:
+        response = requests.get(url, headers=headers)
+        if response.status_code != 200:
+            # Might have a draft
+            response = requests.get(
+                url + "/draft",
+                headers=headers,
+            )
+            if response.status_code != 200:
+                raise Exception(f"Record {record_id} does not exist, cannot edit")
+        file_content = response.content
+        file_name = f"downloaded_data_{record_id}.json"
+        with open(file_name, "wb") as file:
+            file.write(file_content)
+        print(f"Metadata downloaded successfully: {file_name}")
+        with open(file_name, "r") as file:
+            metadata = json.load(file)
+            while True:
+                print("Fields:")
+                for i, field in enumerate(metadata.keys()):
+                    print(f"{i + 1}. {field}")
+
+                field_choice = int(
+                    input(
+                        "Enter the number of the field you want to edit (or 0 to skip, 'exit' to exit): "
+                    )
+                )
+
+                if field_choice == 0:
+                    break
+
+                selected_field = list(metadata.keys())[field_choice - 1]
+
+                if isinstance(metadata[selected_field], list):
+                    while True:
+                        print(f"Items in {selected_field}:")
+                        for i, item in enumerate(metadata[selected_field]):
+                            print(f"{i + 1}. {item}")
+
+                        item_choice = int(
+                            input(
+                                "Enter the number of the item you want to edit (or 0 to go back): "
+                            )
+                        )
+
+                        if item_choice == 0:
+                            break
+
+                        selected_item = metadata[selected_field][item_choice - 1]
+
+                        while True:
+                            print(f"Subfields for {selected_field}:")
+                            for i, subfield in enumerate(selected_item.keys()):
+                                print(f"{i + 1}. {subfield}")
+
+                            subfield_choice = int(
+                                input(
+                                    "Enter the number of the subfield you want to edit (or 0 to go back): "
+                                )
+                            )
+
+                            if subfield_choice == 0:
+                                break
+
+                            selected_subfield = list(selected_item.keys())[
+                                subfield_choice - 1
+                            ]
+
+                            new_value = input(
+                                f"Enter the new value for {selected_subfield}: "
+                            )
+
+                            metadata[selected_field][item_choice - 1][
+                                selected_subfield
+                            ] = new_value
+
+                            with open(file_name, "w") as file:
+                                json.dump(metadata, file, indent=2)
+
+                            print(f"File updated successfully.")
+
+                else:
+                    while True:
+                        print(f"Subfields for {selected_field}:")
+                        for i, subfield in enumerate(metadata[selected_field].keys()):
+                            print(f"{i + 1}. {subfield}")
+
+                        subfield_choice = int(
+                            input(
+                                "Enter the number of the subfield you want to edit (or 0 to go back): "
+                            )
+                        )
+
+                        if subfield_choice == 0:
+                            break
+
+                        selected_subfield = list(metadata[selected_field].keys())[
+                            subfield_choice - 1
+                        ]
+
+                        new_value = input(
+                            f"Enter the new value for {selected_subfield}: "
+                        )
+
+                        metadata[selected_field][selected_subfield] = new_value
+
+                        with open(file_name, "w") as file:
+                            json.dump(metadata, file, indent=2)
+
+                        print(f"File updated successfully.")
+
+    except Exception as e:
+        print(f"An error occurred: {e}")
+    return file_name
+
+
+if __name__ == "__main__":
+    main()

{caltechdata_api-1.5.0 → caltechdata_api-1.6.0}/caltechdata_api/customize_schema.py

@@ -57,7 +57,6 @@ def get_vocabularies():
 
 
 def customize_schema(json_record, schema="43"):
-
     if schema == "43":
         return customize_schema_rdm(json_record)
     else:
@@ -134,7 +133,6 @@ def rdm_creators_contributors(person_list, peopleroles):
 
 
 def customize_schema_rdm(json_record):
-
     # Get vocabularies used in InvenioRDM
     vocabularies = get_vocabularies()
 

caltechdata_api-1.6.0/caltechdata_api/md_to_json.py

@@ -0,0 +1,139 @@
+import re
+import json
+import requests
+
+
+class ReadmeFormatException(Exception):
+    """Custom exception for errors in the README format."""
+
+
+def camel_case(s):
+    """Converts a string to camelCase."""
+    s = re.sub(r"(\s|_|-)+", " ", s).title().replace(" ", "")
+    return s[0].lower() + s[1:] if s else ""
+
+
+def expand_special_keys(key, value):
+    """Expand special keys into their structured format (affiliation, nameIdentifiers)."""
+    if key == "affiliation":
+        if "ror.org" not in value:
+            raise ValueError("Affiliation Identifier is not a ROR")
+        ror = value.split("ror.org/")[1].split("]")[0]
+        response = requests.get(f"https://api.ror.org/organizations/{ror}").json()
+        return [
+            {
+                "affiliationIdentifier": ror,
+                "affiliationIdentifierScheme": "ROR",
+                "name": response["name"],
+            }
+        ]
+    elif key == "nameIdentifiers":
+        orcid = value.split("orcid.org/")[1].split("]")[0]
+        return [
+            {
+                "nameIdentifier": orcid,
+                "nameIdentifierScheme": "ORCID",
+            }
+        ]
+    return value
+
+
+def parse_readme_to_json(readme_path):
+    try:
+        with open(readme_path, "r") as file:
+            lines = file.read().split("\n")
+    except IOError as e:
+        raise ReadmeFormatException(f"Failed to open or read the file: {e}")
+
+    json_data = {}
+    current_section = None
+    current_object = {}
+
+    title_line = lines.pop(0)
+    if title_line.startswith("#") == False:
+        raise ValueError('README.md needs to start with "# Title"')
+    else:
+        json_data["titles"] = [{"title": title_line.replace("# ", "")}]
+
+    section_pattern = re.compile(r"^##\s+(.*)$")
+    key_value_pattern = re.compile(r"^-\s+(.*?):\s+(.*)$")
+    link_pattern = re.compile(r"\[.*?\]\((.*?)\)")
+
+    for line_number, line in enumerate(lines, 1):
+        if not line.strip():
+            if current_object and current_section:
+                if current_section == "types":
+                    json_data[current_section] = current_object
+                elif len(current_object) == 1:
+                    key, value = next(iter(current_object.items()))
+                    if key in ["language", "publicationYear", "publisher", "version"]:
+                        json_data[current_section] = value
+                    else:
+                        json_data[current_section].append(current_object)
+                else:
+                    json_data[current_section].append(current_object)
+                current_object = {}
+            continue
+
+        section_match = section_pattern.match(line)
+        if section_match:
+            if current_section and current_object:
+                if current_section == "types":
+                    json_data[current_section] = current_object
+                elif len(current_object) == 1:
+                    key, value = next(iter(current_object.items()))
+                    if key in ["language", "publicationYear", "publisher", "version"]:
+                        json_data[current_section].append(value)
+                    else:
+                        json_data[current_section].append(current_object)
+                else:
+                    json_data[current_section].append(current_object)
+                current_object = {}
+            current_section = camel_case(section_match.group(1))
+            json_data[current_section] = [] if current_section != "types" else {}
+            continue
+
+        key_value_match = key_value_pattern.match(line)
+        if key_value_match and current_section:
+            key, value = key_value_match.groups()
+            key = camel_case(key)
+
+            if key in ["affiliation", "nameIdentifiers"]:
+                value = expand_special_keys(key, value)
+            else:
+                link_match = link_pattern.search(value)
+                if link_match:
+                    value = link_match.group(1)
+
+            current_object[key] = value
+
+        elif line.strip() and not section_match:
+            raise ReadmeFormatException(
+                f"Incorrect format detected at line {line_number}: {line}"
+            )
+
+    if current_section and current_object:
+        if current_section == "types":
+            json_data[current_section] = current_object
+        elif len(current_object) == 1:
+            key, value = next(iter(current_object.items()))
+            if key in ["language", "publicationYear", "publisher", "version"]:
+                json_data[current_section].append(value)
+            else:
+                json_data[current_section].append(current_object)
+        else:
+            json_data[current_section].append(current_object)
+
+    return json_data
+
+
+if __name__ == "__main__":
+    readme_path = "exampleREADME.md"
+    try:
+        json_data = parse_readme_to_json(readme_path)
+        output_json_path = "output1.json"
+        with open(output_json_path, "w") as json_file:
+            json.dump(json_data, json_file, indent=4)
+        print(f"Converted JSON saved to {output_json_path}")
+    except ReadmeFormatException as e:
+        print(f"Error parsing README file: {e}")

{caltechdata_api-1.5.0 → caltechdata_api-1.6.0}/caltechdata_api/utils.py

@@ -54,7 +54,6 @@ def humanbytes(num: Union[int, float], metric: bool = True, precision: int = 1)
 
 
 if __name__ == "__main__":
-
     print(humanbytes(2251799813685247))  # 2 pebibytes
     print(humanbytes(2000000000000000, True))  # 2 petabytes
     print(humanbytes(1099511627776))  # 1 tebibyte

{caltechdata_api-1.5.0 → caltechdata_api-1.6.0}/caltechdata_api.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: caltechdata-api
-Version: 1.5.0
+Version: 1.6.0
 Summary: Python wrapper for CaltechDATA API.
 Home-page: https://github.com/caltechlibrary/caltechdata_api
 Author: Thomas E Morrell
@@ -18,38 +18,56 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 
 
-# caltechdata_api
+# CaltechDATA API Python Library
 
-Python library for using the CaltechDATA API
+The `caltechdata_api` Python library provides a convenient interface for interacting with the CaltechDATA API. It allows users to write files, create DataCite 4 standard JSON records, edit existing records, and retrieve metadata from the CaltechDATA repository.
 
-- caltechdata_write write files and a DataCite 4 standard json record to CaltechDATA repository
-- caltechdata_edit edits records in CaltechDATA
-- get_metadata gets metadata from CaltechDATA records
+## Features
 
-Requires Python 3 (Recommended via Anaconda https://www.anaconda.com/download) with reqests library.
+### Writing and Editing Records
+- `caltechdata_write`: Writes files and a DataCite 4 standard JSON record to the CaltechDATA repository.
+- `caltechdata_edit`: Edits existing records in CaltechDATA.
+
+### Metadata Operations
+- `get_metadata`: Retrieves metadata from CaltechDATA records.
+
+## Requirements
+
+- Python 3 (Anaconda is recommended)
+- `requests` library
+
+## Installation
+
+Install the library via pip:
+
+```shell
+pip install caltechdata_api
+```
 
 ## Examples
 
 There are some example python scripts in the GitHub repository.
 
-Create a record:
+###Create a record:
 
 ```shell
 python write.py example.json -fnames logo.gif
-pbkn6-m9y63
+# Output: pbkn6-m9y63 (unique identifier)
 ```
-The response will be the unique identifier for the record. You can put this at
+> The response will be the unique identifier for the record. You can put this at
 the end of a url to visit the record (e.g.
 https://data.caltechlibrary.dev/records/pbkn6-m9y63)
 
-Edit a record (make changes to the example.json file to see a change)
+###Edit a record 
+Make changes to the example.json file to see a change)
 ```
 python edit.py example.json -id pbkn6-m9y63
 10.33569/pbkn6-m9y63
 ```
-The response is the DOI for the record, which includes the unique identifier
+> The response is the DOI for the record, which includes the unique identifier
 for the record in the default configuration.
 
+## Using Custom DOIs 
 Some groups have worked with the library to create custom DOIs. These can be
 passed in the metadata like:
 
@@ -66,18 +84,16 @@ python edit.py example_custom.json -id m6zxz-p4j22
 
 This returns the custom DOI of the record if it is successful.
 
-## Setup 
 
-Install by typing 'pip install caltechdata_api'
+## Setup and Authentication
 
-## Usage
+1. Acquire a personal access token from your CaltechDATA account (found under "Applications" at the top right of your screen).
+2. Copy the token to a file named token.bash.
+3. Load the token in the command line using source token.bash.
 
-You need to acquire a personal access token from your CaltechDATA account
-(find it at the top right of your screen under "Applications").
-Then copy the token to token.bash.  Type `source token.bash` in 
-the command line to load the token.  
+## Note on Testing
 
-Only test your application on the test repository (data.caltechlibrary.dev).  Testing the API on the public 
+Only test your application on the test repository (`data.caltechlibrary.dev`).  Testing the API on the public 
 repository will generate junk records that are annoying to delete.
 
 

{caltechdata_api-1.5.0 → caltechdata_api-1.6.0}/caltechdata_api.egg-info/SOURCES.txt

@@ -4,13 +4,16 @@ setup.py
 caltechdata_api/__init__.py
 caltechdata_api/caltechdata_edit.py
 caltechdata_api/caltechdata_write.py
+caltechdata_api/cli.py
 caltechdata_api/customize_schema.py
 caltechdata_api/download_file.py
 caltechdata_api/get_files.py
 caltechdata_api/get_metadata.py
+caltechdata_api/md_to_json.py
 caltechdata_api/utils.py
 caltechdata_api.egg-info/PKG-INFO
 caltechdata_api.egg-info/SOURCES.txt
 caltechdata_api.egg-info/dependency_links.txt
+caltechdata_api.egg-info/entry_points.txt
 caltechdata_api.egg-info/requires.txt
 caltechdata_api.egg-info/top_level.txt

caltechdata_api-1.6.0/caltechdata_api.egg-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+caltechdata_api = caltechdata_api.cli:main
+

{caltechdata_api-1.5.0 → caltechdata_api-1.6.0}/caltechdata_api.egg-info/requires.txt

@@ -3,3 +3,5 @@ datacite>1.1.0
 tqdm>=4.62.3
 pyyaml
 s3fs
+configparser
+awscli

{caltechdata_api-1.5.0 → caltechdata_api-1.6.0}/setup.py

@@ -60,7 +60,15 @@ name = meta["name"]
 REQUIRES_PYTHON = ">=3.6.0"
 
 # What packages are required for this module to be executed?
-REQUIRED = ["requests", "datacite>1.1.0", "tqdm>=4.62.3", "pyyaml", "s3fs"]
+REQUIRED = [
+    "requests",
+    "datacite>1.1.0",
+    "tqdm>=4.62.3",
+    "pyyaml",
+    "s3fs",
+    "configparser",
+    "awscli",
+]
 
 # What packages are optional?
 EXTRAS = {
@@ -163,4 +171,9 @@ setup(
     cmdclass={
         "upload": UploadCommand,
     },
+    entry_points={
+        "console_scripts": [
+            "caltechdata_api=caltechdata_api.cli:main",
+        ],
+    },
 )

caltechdata_api-1.5.0/README.md

@@ -1,62 +0,0 @@
-# caltechdata_api
-
-Python library for using the CaltechDATA API
-
-- caltechdata_write write files and a DataCite 4 standard json record to CaltechDATA repository
-- caltechdata_edit edits records in CaltechDATA
-- get_metadata gets metadata from CaltechDATA records
-
-Requires Python 3 (Recommended via Anaconda https://www.anaconda.com/download) with reqests library.
-
-## Examples
-
-There are some example python scripts in the GitHub repository.
-
-Create a record:
-
-```shell
-python write.py example.json -fnames logo.gif
-pbkn6-m9y63
-```
-The response will be the unique identifier for the record. You can put this at
-the end of a url to visit the record (e.g.
-https://data.caltechlibrary.dev/records/pbkn6-m9y63)
-
-Edit a record (make changes to the example.json file to see a change)
-```
-python edit.py example.json -id pbkn6-m9y63
-10.33569/pbkn6-m9y63
-```
-The response is the DOI for the record, which includes the unique identifier
-for the record in the default configuration.
-
-Some groups have worked with the library to create custom DOIs. These can be
-passed in the metadata like:
-
-```shell
-python write.py example_custom.json -fnames logo.gif
-m6zxz-p4j22
-```
-
-And then you can edit with
-```
-python edit.py example_custom.json -id m6zxz-p4j22
-10.5281/inveniordm.1234
-```
-
-This returns the custom DOI of the record if it is successful.
-
-## Setup 
-
-Install by typing 'pip install caltechdata_api'
-
-## Usage
-
-You need to acquire a personal access token from your CaltechDATA account
-(find it at the top right of your screen under "Applications").
-Then copy the token to token.bash.  Type `source token.bash` in 
-the command line to load the token.  
-
-Only test your application on the test repository (data.caltechlibrary.dev).  Testing the API on the public 
-repository will generate junk records that are annoying to delete.
-