nexaai 0.0.9.1__tar.gz → 0.0.9.4__tar.gz

nexaai-0.0.9.4/CLI.md

@@ -0,0 +1,410 @@
+## CLI Reference
+
+### Overview
+
+```
+usage: nexa [-h] [-V] {run,onnx,server,pull,remove,clean,list,login,whoami,logout} ...
+
+Nexa CLI tool for handling various model operations.
+
+positional arguments:
+  {run,onnx,server,pull,remove,clean,list,login,whoami,logout}
+                        sub-command help
+    run                 Run inference for various tasks using GGUF models.
+    onnx                Run inference for various tasks using ONNX models.
+    embed               Generate embeddings for text.
+    convert             Convert and quantize a Hugging Face model to GGUF format.
+    server              Run the Nexa AI Text Generation Service.
+    eval                Run the Nexa AI Evaluation Tasks.
+    pull                Pull a model from official or hub.
+    remove              Remove a model from local machine.
+    clean               Clean up all model files.
+    list                List all models in the local machine.
+    login               Login to Nexa API.
+    whoami              Show current user information.
+    logout              Logout from Nexa API.
+
+options:
+  -h, --help            show this help message and exit
+  -V, --version         Show the version of the Nexa SDK.
+```
+
+### List Local Models
+
+List all models on your local computer. You can use `nexa run <model_name>` to run any model shown in the list.
+
+```
+nexa list
+```
+
+### Download a Model
+
+Download a model file to your local computer from Nexa Model Hub.
+
+```
+nexa pull MODEL_PATH
+usage: nexa pull [-h] model_path
+
+positional arguments:
+  model_path  Path or identifier for the model in Nexa Model Hub, Hugging Face repo ID when using -hf flag, or ModelScope model ID when using -ms flag
+
+options:
+  -h, --help            show this help message and exit
+  -hf, --huggingface    Pull model from Hugging Face Hub
+  -ms, --modelscope     Pull model from ModelScope Hub
+  -o, --output_path OUTPUT_PATH
+                        Custom output path for the pulled model
+```
+
+#### Example
+
+```
+nexa pull llama2
+```
+
+### Remove a Model
+
+Remove a model from your local computer.
+
+```
+nexa remove MODEL_PATH
+usage: nexa remove [-h] model_path
+
+positional arguments:
+  model_path  Path or identifier for the model in Nexa Model Hub
+
+options:
+  -h, --help  show this help message and exit
+```
+
+#### Example
+
+```
+nexa remove llama2
+```
+
+### Remove All Downloaded Models
+
+Remove all downloaded models on your local computer.
+
+```
+nexa clean
+```
+
+### Run a Model
+
+Run a model on your local computer. If the model file is not yet downloaded, it will be automatically fetched first.
+
+By default, `nexa` will run gguf models. To run onnx models, use `nexa onnx MODEL_PATH`
+
+You can run any model shown in `nexa list` command.
+
+#### Run Text-Generation Model
+
+```
+nexa run MODEL_PATH
+usage: nexa run [-h] [-t TEMPERATURE] [-m MAX_NEW_TOKENS] [-k TOP_K] [-p TOP_P] [-sw [STOP_WORDS ...]] [-pf] [-st] [-lp] [-mt {NLP, COMPUTER_VISION, MULTIMODAL, AUDIO}] [-hf] [-ms] model_path
+
+positional arguments:
+  model_path            Path or identifier for the model in Nexa Model Hub
+
+options:
+  -h, --help            show this help message and exit
+  -pf, --profiling      Enable profiling logs for the inference process
+  -st, --streamlit      Run the inference in Streamlit UI, can be used with -lp or -hf
+  -lp, --local_path     Indicate that the model path provided is the local path
+  -mt, --model_type     Indicate the model running type, must be used with -lp or -hf or -ms, choose from [NLP, COMPUTER_VISION, MULTIMODAL, AUDIO]
+  -hf, --huggingface    Load model from Hugging Face Hub
+  -ms, --modelscope     Load model from ModelScope Hub
+
+Text generation options:
+  -t, --temperature TEMPERATURE
+                        Temperature for sampling
+  -m, --max_new_tokens MAX_NEW_TOKENS
+                        Maximum number of new tokens to generate
+  -k, --top_k TOP_K     Top-k sampling parameter
+  -p, --top_p TOP_P     Top-p sampling parameter
+  -sw, --stop_words [STOP_WORDS ...]
+                        List of stop words for early stopping
+  --nctx TEXT_CONTEXT   Length of context window
+```
+
+##### Example
+
+```
+nexa run llama2
+```
+
+#### Run Image-Generation Model
+
+```
+nexa run MODEL_PATH
+usage: nexa run [-h] [-i2i] [-ns NUM_INFERENCE_STEPS] [-np NUM_IMAGES_PER_PROMPT] [-H HEIGHT] [-W WIDTH] [-g GUIDANCE_SCALE] [-o OUTPUT] [-s RANDOM_SEED] [-st] [-lp] [-mt {NLP, COMPUTER_VISION, MULTIMODAL, AUDIO}] [-hf] [-ms] model_path
+
+positional arguments:
+  model_path            Path or identifier for the model in Nexa Model Hub
+
+options:
+  -h, --help            show this help message and exit
+  -st, --streamlit      Run the inference in Streamlit UI, can be used with -lp or -hf
+  -lp, --local_path     Indicate that the model path provided is the local path
+  -mt, --model_type     Indicate the model running type, must be used with -lp or -hf or -ms, choose from [NLP, COMPUTER_VISION, MULTIMODAL, AUDIO]
+  -hf, --huggingface    Load model from Hugging Face Hub
+  -ms, --modelscope     Load model from ModelScope Hub
+
+Image generation options:
+  -i2i, --img2img       Whether to run image-to-image generation
+  -ns, --num_inference_steps NUM_INFERENCE_STEPS
+                        Number of inference steps
+  -np, --num_images_per_prompt NUM_IMAGES_PER_PROMPT
+                        Number of images to generate per prompt
+  -H, --height HEIGHT   Height of the output image
+  -W, --width WIDTH     Width of the output image
+  -g, --guidance_scale GUIDANCE_SCALE
+                        Guidance scale for diffusion
+  -o, --output OUTPUT   Output path for the generated image
+  -s, --random_seed RANDOM_SEED
+                        Random seed for image generation
+  --lora_dir LORA_DIR   Path to directory containing LoRA files
+  --wtype WTYPE         Weight type (f32, f16, q4_0, q4_1, q5_0, q5_1, q8_0)
+  --control_net_path CONTROL_NET_PATH
+                        Path to the control net model
+  --control_image_path CONTROL_IMAGE_PATH
+                        Path to the image condition for Control Net
+  --control_strength CONTROL_STRENGTH
+                        Strength to apply Control Net
+```
+
+##### Example
+
+```
+nexa run sd1-4
+```
+
+#### Run Vision-Language Model
+
+```
+nexa run MODEL_PATH
+usage: nexa run [-h] [-t TEMPERATURE] [-m MAX_NEW_TOKENS] [-k TOP_K] [-p TOP_P] [-sw [STOP_WORDS ...]] [-pf] [-st] [-lp] [-mt {NLP, COMPUTER_VISION, MULTIMODAL, AUDIO}] [-hf] [-ms] model_path
+
+positional arguments:
+  model_path            Path or identifier for the model in Nexa Model Hub
+
+options:
+  -h, --help            show this help message and exit
+  -pf, --profiling      Enable profiling logs for the inference process
+  -st, --streamlit      Run the inference in Streamlit UI, can be used with -lp or -hf
+  -lp, --local_path     Indicate that the model path provided is the local path
+  -mt, --model_type     Indicate the model running type, must be used with -lp or -hf or -ms, choose from [NLP, COMPUTER_VISION, MULTIMODAL, AUDIO]
+  -hf, --huggingface    Load model from Hugging Face Hub
+  -ms, --modelscope     Load model from ModelScope Hub
+
+VLM generation options:
+  -t, --temperature TEMPERATURE
+                        Temperature for sampling
+  -m, --max_new_tokens MAX_NEW_TOKENS
+                        Maximum number of new tokens to generate
+  -k, --top_k TOP_K     Top-k sampling parameter
+  -p, --top_p TOP_P     Top-p sampling parameter
+  -sw, --stop_words [STOP_WORDS ...]
+                        List of stop words for early stopping
+  --nctx TEXT_CONTEXT   Length of context window
+```
+
+##### Example
+
+```
+nexa run nanollava
+```
+
+#### Run Audio Model
+
+```
+nexa run MODEL_PATH
+usage: nexa run [-h] [-o OUTPUT_DIR] [-b BEAM_SIZE] [-l LANGUAGE] [--task TASK] [-t TEMPERATURE] [-c COMPUTE_TYPE] [-st] [-lp] [-mt {NLP, COMPUTER_VISION, MULTIMODAL, AUDIO}] [-hf] [-ms] model_path
+
+positional arguments:
+  model_path            Path or identifier for the model in Nexa Model Hub
+
+options:
+  -h, --help            show this help message and exit
+  -st, --streamlit      Run the inference in Streamlit UI, can be used with -lp or -hf
+  -lp, --local_path     Indicate that the model path provided is the local path
+  -mt, --model_type     Indicate the model running type, must be used with -lp or -hf or -ms, choose from [NLP, COMPUTER_VISION, MULTIMODAL, AUDIO]
+  -hf, --huggingface    Load model from Hugging Face Hub
+  -ms, --modelscope     Load model from ModelScope Hub
+
+Automatic Speech Recognition options:
+  -b, --beam_size BEAM_SIZE
+                        Beam size to use for transcription
+  -l, --language LANGUAGE
+                        The language spoken in the audio. It should be a language code such as 'en' or 'fr'.
+  --task TASK           Task to execute (transcribe or translate)
+  -c, --compute_type COMPUTE_TYPE
+                        Type to use for computation (e.g., float16, int8, int8_float16)
+```
+
+##### Example
+
+```
+nexa run faster-whisper-tiny
+```
+
+### Generate Embeddings
+
+#### Generate Text Embeddings
+
+```
+nexa embed MODEL_PATH
+usage: nexa embed [-h] [-lp] [-hf] [-ms] [-n] [-nt] model_path prompt
+
+positional arguments:
+  model_path            Path or identifier for the model in Nexa Model Hub
+  prompt                Prompt to generate embeddings
+
+options:
+  -h, --help            show this help message and exit
+  -lp, --local_path     Indicate that the model path provided is the local path
+  -hf, --huggingface    Load model from Hugging Face Hub
+  -ms, --modelscope     Load model from ModelScope Hub
+  -n, --normalize       Normalize the embeddings
+  -nt, --no_truncate    Not truncate the embeddings
+```
+
+#### Example
+
+```
+nexa embed mxbai "I love Nexa AI."
+nexa embed nomic "I love Nexa AI." >> generated_embeddings.txt
+nexa embed nomic-embed-text-v1.5:fp16 "I love Nexa AI."
+nexa embed sentence-transformers/all-MiniLM-L6-v2:gguf-fp16 "I love Nexa AI." >> generated_embeddings.txt
+```
+
+### Convert and quantize a Hugging Face Model to GGUF
+
+Additional package `nexa-gguf` is required to run this command.
+
+You can install it by `pip install "nexaai[convert]"` or `pip install nexa-gguf`.
+
+```
+nexa convert HF_MODEL_PATH [ftype] [output_file]
+usage: nexa convert [-h] [-t NTHREAD] [--convert_type CONVERT_TYPE] [--bigendian] [--use_temp_file] [--no_lazy]
+                    [--metadata METADATA] [--split_max_tensors SPLIT_MAX_TENSORS] [--split_max_size SPLIT_MAX_SIZE]
+                    [--no_tensor_first_split] [--vocab_only] [--dry_run] [--output_tensor_type OUTPUT_TENSOR_TYPE]
+                    [--token_embedding_type TOKEN_EMBEDDING_TYPE] [--allow_requantize] [--quantize_output_tensor]
+                    [--only_copy] [--pure] [--keep_split] input_path [ftype] [output_file]
+
+positional arguments:
+  input_path            Path to the input Hugging Face model directory or GGUF file
+  ftype                 Quantization type (default: q4_0)
+  output_file           Path to the output quantized GGUF file
+
+options:
+  -h, --help            show this help message and exit
+  -t, --nthread NTHREAD Number of threads to use (default: 4)
+  --convert_type CONVERT_TYPE
+                        Conversion type for safetensors to GGUF (default: f16)
+  --bigendian           Use big endian format
+  --use_temp_file       Use a temporary file during conversion
+  --no_lazy             Disable lazy loading
+  --metadata METADATA   Additional metadata as JSON string
+  --split_max_tensors SPLIT_MAX_TENSORS
+                        Maximum number of tensors per split
+  --split_max_size SPLIT_MAX_SIZE
+  --no_tensor_first_split
+                        Disable tensor-first splitting
+  --vocab_only          Only process vocabulary
+  --dry_run             Perform a dry run without actual conversion
+  --output_tensor_type  Output tensor type
+  --token_embedding_type
+                        Token embedding type
+  --allow_requantize    Allow quantizing non-f32/f16 tensors
+  --quantize_output_tensor
+                        Quantize output.weight
+  --only_copy           Only copy tensors (ignores ftype, allow_requantize, and quantize_output_tensor)
+  --pure                Quantize all tensors to the default type
+  --keep_split          Quantize to the same number of shards
+  -ms --modelscope      Load model from ModelScope Hub
+```
+
+#### Example
+
+```
+# Default quantization type (q4_0) and output file in current directory
+nexa convert meta-llama/Llama-3.2-1B-Instruct
+
+# Equivalent to:
+# nexa convert meta-llama/Llama-3.2-1B-Instruct q4_0 ./Llama-3.2-1B-Instruct-q4_0.gguf
+
+# Specifying quantization type and output file
+nexa convert meta-llama/Llama-3.2-1B-Instruct q6_k llama3.2-1b-instruct-q6_k.gguf
+```
+
+Note: When not specified, the default quantization type is set to q4_0, and the output file will be created in the current directory with the name format: `<model_name>-q4_0.gguf`.
+
+### Start Local Server
+
+Start a local server using models on your local computer.
+
+```
+nexa server MODEL_PATH
+usage: nexa server [-h] [--host HOST] [--port PORT] [--reload] [-lp] [-mt {NLP, COMPUTER_VISION, MULTIMODAL, AUDIO}] [-hf] [-ms] model_path
+
+positional arguments:
+  model_path   Path or identifier for the model in S3
+
+options:
+  -h, --help   show this help message and exit
+  -lp, --local_path     Indicate that the model path provided is the local path
+  -mt, --model_type     Indicate the model running type, must be used with -lp or -hf or -ms, choose from [NLP, COMPUTER_VISION, MULTIMODAL, AUDIO]
+  -hf, --huggingface    Load model from Hugging Face Hub
+  -ms, --modelscope     Load model from ModelScope Hub
+  --host HOST  Host to bind the server to
+  --port PORT  Port to bind the server to
+  --reload     Enable automatic reloading on code changes
+```
+
+#### Example
+
+```
+nexa server llama2
+```
+
+### Run Model Evaluation
+
+Run evaluation using models on your local computer.
+
+```
+usage: nexa eval model_path [-h] [--tasks TASKS] [--limit LIMIT]
+
+positional arguments:
+  model_path            Path or identifier for the model in Nexa Model Hub
+
+options:
+  -h, --help            show this help message and exit
+  --tasks TASKS         Tasks to evaluate, comma-separated
+  --limit LIMIT         Limit the number of examples per task. If <1, limit is a percentage of the total number of examples.
+```
+
+#### Examples
+
+```
+nexa eval phi3 --tasks ifeval --limit 0.5
+```
+
+### Model Path Format
+
+For `model_path` in nexa commands, it's better to follow the standard format to ensure correct model loading and execution. The standard format for `model_path` is:
+
+- `[user_name]/[repo_name]:[tag_name]` (user's model)
+- `[repo_name]:[tag_name]` (official model)
+
+#### Examples:
+
+- `gemma-2b:q4_0`
+- `Meta-Llama-3-8B-Instruct:onnx-cpu-int8`
+- `liuhaotian/llava-v1.6-vicuna-7b:gguf-q4_0`
+
+```
+</rewritten_chunk>
+```

nexaai-0.0.9.4/CMakeLists.txt

@@ -0,0 +1,266 @@
+cmake_minimum_required(VERSION 3.16)
+project(nexa_gguf)
+
+include(ExternalProject)
+
+# Platform-specific settings
+if(WIN32)
+    # Windows-specific settings
+    add_definitions(-D_CRT_SECURE_NO_WARNINGS)
+    # OpenMP is optional on Windows
+    find_package(OpenMP QUIET)
+    if(NOT OpenMP_FOUND)
+        message(STATUS "OpenMP not found - OpenMP support will be disabled")
+        set(OpenMP_C_FLAGS "")
+        set(OpenMP_CXX_FLAGS "")
+        set(OpenMP_EXE_LINKER_FLAGS "")
+    endif()
+elseif(APPLE)
+    # macOS-specific settings
+    find_package(OpenMP QUIET)
+    if(NOT OpenMP_FOUND)
+        message(STATUS "OpenMP not found - OpenMP support will be disabled")
+        set(OpenMP_C_FLAGS "")
+        set(OpenMP_CXX_FLAGS "")
+        set(OpenMP_EXE_LINKER_FLAGS "")
+    endif()
+else()
+    # Linux and other Unix systems
+    find_package(OpenMP REQUIRED)
+endif()
+
+set(CMAKE_POSITION_INDEPENDENT_CODE ON)
+set(CMAKE_CXX_STANDARD 17)
+
+# Windows-specific configurations
+if(WIN32)
+    add_definitions(-D_CRT_SECURE_NO_WARNINGS)
+    add_definitions(-DNOMINMAX)
+    add_definitions(-D_WIN32_WINNT=0x0A00)  # Target Windows 10 or later
+    set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
+endif()
+
+# Function to collect all user-defined options
+function(get_all_options output_var)
+    get_cmake_property(variables VARIABLES)
+    set(options)
+    foreach(var ${variables})
+        if(var MATCHES "^[A-Z]" AND NOT var MATCHES "^CMAKE_" AND NOT var MATCHES "^_")
+            list(APPEND options "-D${var}=${${var}}")
+        endif()
+    endforeach()
+    set(${output_var} ${options} PARENT_SCOPE)
+endfunction()
+
+# Create empty file if GGML_CUDA or GGML_METAL is ON
+if (GGML_CUDA OR GGML_METAL OR GGML_HIPBLAS OR GGML_VULKAN)
+    set(SOURCE_EMPTY_FILE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/nexa/gguf/lib/empty_file.txt")
+    add_custom_command(
+        OUTPUT ${SOURCE_EMPTY_FILE_PATH}
+        COMMAND ${CMAKE_COMMAND} -E touch ${SOURCE_EMPTY_FILE_PATH}
+        COMMENT "Creating an empty file to source folder because gpu option is ON"
+    )
+    set(WHEEL_EMPTY_FILE_PATH "${SKBUILD_PLATLIB_DIR}/nexa/gguf/lib/empty_file.txt")
+    add_custom_command(
+        OUTPUT ${WHEEL_EMPTY_FILE_PATH}
+        COMMAND ${CMAKE_COMMAND} -E touch ${WHEEL_EMPTY_FILE_PATH}
+        COMMENT "Creating an empty file to lib folder because gpu option is ON"
+    )    
+    add_custom_target(create_empty_file ALL DEPENDS ${SOURCE_EMPTY_FILE_PATH} ${WHEEL_EMPTY_FILE_PATH})
+endif()
+
+if(WIN32 AND GGML_CUDA)
+    set(BUILD_PARALLEL_LEVEL "2" CACHE STRING "Number of parallel jobs for MSBuild in CUDA compilation")
+    set(MSBUILD_ARGS "/m:${BUILD_PARALLEL_LEVEL}")
+else()
+    set(MSBUILD_ARGS "")
+endif()
+
+
+# Function to set up common installation paths
+function(setup_install_paths target install_dir)
+    install(
+        TARGETS ${target}
+        LIBRARY DESTINATION ${install_dir}
+        RUNTIME DESTINATION ${install_dir}
+        ARCHIVE DESTINATION ${install_dir}
+    )
+endfunction()
+
+# Collect all user-defined options
+get_all_options(USER_DEFINED_OPTIONS)
+
+if(APPLE)
+    set(CMAKE_INSTALL_RPATH "@loader_path")
+else()
+    set(CMAKE_INSTALL_RPATH "$ORIGIN")
+endif()
+
+# Define common CMake options
+set(COMMON_CMAKE_OPTIONS
+    -DCMAKE_BUILD_WITH_INSTALL_RPATH=TRUE
+    -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=TRUE
+    -DCMAKE_SKIP_BUILD_RPATH=FALSE
+    -DCMAKE_SKIP_RPATH=FALSE
+    -DCMAKE_INSTALL_RPATH=${CMAKE_INSTALL_RPATH}
+)
+
+if(WIN32)
+    if(CMAKE_SYSTEM_VERSION)
+        list(APPEND COMMON_CMAKE_OPTIONS
+            -DCMAKE_SYSTEM_VERSION=${CMAKE_SYSTEM_VERSION}
+        )
+    endif()
+    
+    if(CMAKE_VS_WINDOWS_TARGET_PLATFORM_VERSION)
+        list(APPEND COMMON_CMAKE_OPTIONS
+            -DCMAKE_VS_WINDOWS_TARGET_PLATFORM_VERSION=${CMAKE_VS_WINDOWS_TARGET_PLATFORM_VERSION}
+        )
+    endif()
+endif()
+
+# stable_diffusion_cpp project
+option(STABLE_DIFFUSION_BUILD "Build stable-diffusion.cpp" ON)
+if(STABLE_DIFFUSION_BUILD)
+    ExternalProject_Add(stable_diffusion_project
+        SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/dependency/stable-diffusion.cpp
+        BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/stable_diffusion_build
+        CMAKE_ARGS
+            ${USER_DEFINED_OPTIONS}
+            ${COMMON_CMAKE_OPTIONS}
+            -DCMAKE_INSTALL_PREFIX=${CMAKE_CURRENT_BINARY_DIR}/stable_diffusion_install
+            -DCMAKE_POSITION_INDEPENDENT_CODE=ON
+            -DCMAKE_CXX_STANDARD=17
+            -DSD_BUILD_SHARED_LIBS=ON
+            -DBUILD_SHARED_LIBS=ON
+            -DSD_METAL=${GGML_METAL}
+            -DSD_CUBLAS=${GGML_CUDA}
+            -DSD_HIPBLAS=${GGML_HIPBLAS}
+            -DSD_VULKAN=${GGML_VULKAN}
+        BUILD_ALWAYS 1
+        BUILD_COMMAND ${CMAKE_COMMAND} --build . --config Release -- ${MSBUILD_ARGS}
+        INSTALL_COMMAND ${CMAKE_COMMAND} --build . --config Release --target install
+    )
+endif()
+
+# llama_cpp project
+option(LLAMA_BUILD "Build llama.cpp" ON)
+if(LLAMA_BUILD)
+    set(LLAMA_CUDA ${GGML_CUDA})
+    set(LLAMA_METAL ${GGML_METAL})
+
+    if(WIN32)
+        # Add Windows-specific definitions and flags for llama.cpp
+        list(APPEND COMMON_CMAKE_OPTIONS
+            -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=ON
+            -DLLAMA_NATIVE=OFF           # Disable native CPU optimizations on Windows
+            -DLLAMA_DISABLE_CXXABI=ON    # Disable cxxabi.h dependency
+        )
+
+        # Add compile definition for all targets
+        add_compile_definitions(LLAMA_DISABLE_CXXABI)
+    endif()
+
+    ExternalProject_Add(llama_project
+        SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/dependency/llama.cpp
+        BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/llama_build
+        CMAKE_ARGS
+            ${USER_DEFINED_OPTIONS}
+            ${COMMON_CMAKE_OPTIONS}
+            -DCMAKE_INSTALL_PREFIX=${CMAKE_CURRENT_BINARY_DIR}/llama_install
+            -DCMAKE_POSITION_INDEPENDENT_CODE=ON
+            -DCMAKE_CXX_STANDARD=17
+            -DBUILD_SHARED_LIBS=ON
+            -DLLAMA_CUDA=${LLAMA_CUDA}
+            -DLLAMA_METAL=${LLAMA_METAL}
+            -DCMAKE_C_FLAGS=${OpenMP_C_FLAGS}
+            -DCMAKE_CXX_FLAGS=${OpenMP_CXX_FLAGS}
+            -DCMAKE_EXE_LINKER_FLAGS=${OpenMP_EXE_LINKER_FLAGS}
+            -DGGML_AVX=$<IF:$<AND:$<PLATFORM_ID:Darwin>,$<NOT:$<STREQUAL:${CMAKE_SYSTEM_PROCESSOR},arm64>>>,OFF,ON>
+            -DGGML_AVX2=$<IF:$<AND:$<PLATFORM_ID:Darwin>,$<NOT:$<STREQUAL:${CMAKE_SYSTEM_PROCESSOR},arm64>>>,OFF,ON>
+            -DGGML_FMA=$<IF:$<AND:$<PLATFORM_ID:Darwin>,$<NOT:$<STREQUAL:${CMAKE_SYSTEM_PROCESSOR},arm64>>>,OFF,ON>
+            -DGGML_F16C=$<IF:$<AND:$<PLATFORM_ID:Darwin>,$<NOT:$<STREQUAL:${CMAKE_SYSTEM_PROCESSOR},arm64>>>,OFF,ON>
+            -DGGML_METAL_EMBED_LIBRARY=$<IF:$<PLATFORM_ID:Darwin>,ON,OFF>
+        BUILD_ALWAYS 1
+        BUILD_COMMAND ${CMAKE_COMMAND} --build . --config Release -- ${MSBUILD_ARGS}
+        INSTALL_COMMAND ${CMAKE_COMMAND} --build . --config Release --target install
+    )
+endif()
+
+# bark_cpp project
+# Temporarily disabled since version v0.0.9.3
+option(BARK_BUILD "Build bark.cpp" OFF)
+if(BARK_BUILD)
+    # Filter out HIPBLAS and Vulkan options for bark.cpp since it doesn't support them
+    set(BARK_CMAKE_OPTIONS ${USER_DEFINED_OPTIONS})
+    list(FILTER BARK_CMAKE_OPTIONS EXCLUDE REGEX "GGML_HIPBLAS|GGML_VULKAN")
+    
+    ExternalProject_Add(bark_project
+        SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/dependency/bark.cpp
+        BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/bark_build
+        CMAKE_ARGS
+            ${USER_DEFINED_OPTIONS}
+            ${COMMON_CMAKE_OPTIONS}
+            -DCMAKE_INSTALL_PREFIX=${CMAKE_CURRENT_BINARY_DIR}/bark_install
+            -DCMAKE_POSITION_INDEPENDENT_CODE=ON
+            -DCMAKE_CXX_STANDARD=17
+            -DGGML_CUDA=${GGML_CUDA}
+            -DGGML_METAL=OFF
+            -DBUILD_SHARED_LIBS=ON
+            -DBARK_BUILD_EXAMPLES=OFF
+        BUILD_ALWAYS 1
+        BUILD_COMMAND ${CMAKE_COMMAND} --build . --config Release -- ${MSBUILD_ARGS}
+        INSTALL_COMMAND ${CMAKE_COMMAND} --build . --config Release --target install
+    )
+endif()
+
+# Install the built libraries to the final destination
+if(WIN32)
+    install(
+        DIRECTORY
+            ${CMAKE_CURRENT_BINARY_DIR}/stable_diffusion_build/bin/Release/
+            ${CMAKE_CURRENT_BINARY_DIR}/llama_build/bin/Release/
+            ${CMAKE_CURRENT_BINARY_DIR}/bark_build/bin/Release/
+            ${CMAKE_CURRENT_BINARY_DIR}/bark_build/Release/
+        DESTINATION ${SKBUILD_PLATLIB_DIR}/nexa/gguf/lib
+        USE_SOURCE_PERMISSIONS
+        FILES_MATCHING
+            PATTERN "*.dll"
+    )
+
+    install(
+        DIRECTORY
+            ${CMAKE_CURRENT_BINARY_DIR}/stable_diffusion_build/bin/Release/
+            ${CMAKE_CURRENT_BINARY_DIR}/llama_build/bin/Release/
+            ${CMAKE_CURRENT_BINARY_DIR}/bark_build/bin/Release/
+            ${CMAKE_CURRENT_BINARY_DIR}/bark_build/Release/
+        DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}/nexa/gguf/lib
+        USE_SOURCE_PERMISSIONS
+        FILES_MATCHING
+            PATTERN "*.dll"
+    )
+else()
+    install(
+        DIRECTORY
+            ${CMAKE_CURRENT_BINARY_DIR}/stable_diffusion_build/bin/
+            ${CMAKE_CURRENT_BINARY_DIR}/llama_install/lib/
+            ${CMAKE_CURRENT_BINARY_DIR}/bark_install/lib/
+        DESTINATION ${SKBUILD_PLATLIB_DIR}/nexa/gguf/lib
+        USE_SOURCE_PERMISSIONS
+        FILES_MATCHING
+            PATTERN "*.so"
+            PATTERN "*.dylib"
+    )
+
+    install(
+        DIRECTORY
+            ${CMAKE_CURRENT_BINARY_DIR}/stable_diffusion_build/bin/
+            ${CMAKE_CURRENT_BINARY_DIR}/llama_install/lib/
+            ${CMAKE_CURRENT_BINARY_DIR}/bark_install/lib/
+        DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}/nexa/gguf/lib
+        USE_SOURCE_PERMISSIONS
+        FILES_MATCHING
+            PATTERN "*.so"
+            PATTERN "*.dylib"
+    )
+endif()