spark-nlp 6.1.0__py2.py3-none-any.whl → 6.1.2__py2.py3-none-any.whl

{spark_nlp-6.1.0.dist-info → spark_nlp-6.1.2.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: spark-nlp
-Version: 6.1.0
+Version: 6.1.2
 Summary: John Snow Labs Spark NLP is a natural language processing library built on top of Apache Spark ML. It provides simple, performant & accurate NLP annotations for machine learning pipelines, that scale easily in a distributed environment.
 Home-page: https://github.com/JohnSnowLabs/spark-nlp
 Author: John Snow Labs
@@ -58,7 +58,7 @@ Dynamic: summary
 
 Spark NLP is a state-of-the-art Natural Language Processing library built on top of Apache Spark. It provides **simple**, **performant** & **accurate** NLP annotations for machine learning pipelines that **scale** easily in a distributed environment.
 
-Spark NLP comes with **83000+** pretrained **pipelines** and **models** in more than **200+** languages.
+Spark NLP comes with **100000+** pretrained **pipelines** and **models** in more than **200+** languages.
 It also offers tasks such as **Tokenization**, **Word Segmentation**, **Part-of-Speech Tagging**, Word and Sentence **Embeddings**, **Named Entity Recognition**, **Dependency Parsing**, **Spell Checking**, **Text Classification**, **Sentiment Analysis**, **Token Classification**, **Machine Translation** (+180 languages), **Summarization**, **Question Answering**, **Table Question Answering**, **Text Generation**, **Image Classification**, **Image to Text (captioning)**, **Automatic Speech Recognition**, **Zero-Shot Learning**, and many more [NLP tasks](#features).
 
 **Spark NLP** is the only open-source NLP library in **production** that offers state-of-the-art transformers such as **BERT**, **CamemBERT**, **ALBERT**, **ELECTRA**, **XLNet**, **DistilBERT**, **RoBERTa**, **DeBERTa**, **XLM-RoBERTa**, **Longformer**, **ELMO**, **Universal Sentence Encoder**, **Llama-2**, **M2M100**, **BART**, **Instructor**, **E5**, **Google T5**, **MarianMT**, **OpenAI GPT2**, **Vision Transformers (ViT)**, **OpenAI Whisper**, **Llama**, **Mistral**, **Phi**, **Qwen2**, and many more not only to **Python** and **R**, but also to **JVM** ecosystem (**Java**, **Scala**, and **Kotlin**) at **scale** by extending **Apache Spark** natively.
@@ -102,7 +102,7 @@ $ java -version
 $ conda create -n sparknlp python=3.7 -y
 $ conda activate sparknlp
 # spark-nlp by default is based on pyspark 3.x
-$ pip install spark-nlp==6.1.0 pyspark==3.3.1
+$ pip install spark-nlp==6.1.2 pyspark==3.3.1
 ```
 
 In Python console or Jupyter `Python3` kernel:
@@ -168,11 +168,11 @@ For a quick example of using pipelines and models take a look at our official [d
 
 ### Apache Spark Support
 
-Spark NLP *6.1.0* has been built on top of Apache Spark 3.4 while fully supports Apache Spark 3.0.x, 3.1.x, 3.2.x, 3.3.x, 3.4.x, and 3.5.x
+Spark NLP *6.1.2* has been built on top of Apache Spark 3.4 while fully supports Apache Spark 3.0.x, 3.1.x, 3.2.x, 3.3.x, 3.4.x, and 3.5.x
 
 | Spark NLP | Apache Spark 3.5.x | Apache Spark 3.4.x | Apache Spark 3.3.x | Apache Spark 3.2.x | Apache Spark 3.1.x | Apache Spark 3.0.x | Apache Spark 2.4.x | Apache Spark 2.3.x |
 |-----------|--------------------|--------------------|--------------------|--------------------|--------------------|--------------------|--------------------|--------------------|
-| 6.0.x     | YES                | YES                | YES                | YES                | YES                | YES                | NO                 | NO                 |
+| 6.x.x and up     | YES                | YES                | YES                | YES                | YES                | YES                | NO                 | NO                 |
 | 5.5.x     | YES                | YES                | YES                | YES                | YES                | YES                | NO                 | NO                 |
 | 5.4.x     | YES                | YES                | YES                | YES                | YES                | YES                | NO                 | NO                 |
 | 5.3.x     | YES                | YES                | YES                | YES                | YES                | YES                | NO                 | NO                 |
@@ -198,7 +198,7 @@ Find out more about 4.x `SparkNLP` versions in our official [documentation](http
 
 ### Databricks Support
 
-Spark NLP 6.1.0 has been tested and is compatible with the following runtimes:
+Spark NLP 6.1.2 has been tested and is compatible with the following runtimes:
 
 | **CPU**            | **GPU**            |
 |--------------------|--------------------|
@@ -206,16 +206,17 @@ Spark NLP 6.1.0 has been tested and is compatible with the following runtimes:
 | 14.2 / 14.2 ML     | 14.2 ML & GPU      |
 | 14.3 / 14.3 ML     | 14.3 ML & GPU      |
 | 15.0 / 15.0 ML     | 15.0 ML & GPU      |
-| 15.1 / 15.0 ML     | 15.1 ML & GPU      |
-| 15.2 / 15.0 ML     | 15.2 ML & GPU      |
-| 15.3 / 15.0 ML     | 15.3 ML & GPU      |
-| 15.4 / 15.0 ML     | 15.4 ML & GPU      |
+| 15.1 / 15.1 ML     | 15.1 ML & GPU      |
+| 15.2 / 15.2 ML     | 15.2 ML & GPU      |
+| 15.3 / 15.3 ML     | 15.3 ML & GPU      |
+| 15.4 / 15.4 ML     | 15.4 ML & GPU      |
+| 16.4 / 16.4 ML     | 16.4 ML & GPU      |
 
 We are compatible with older runtimes. For a full list check databricks support in our official [documentation](https://sparknlp.org/docs/en/install#databricks-support)
 
 ### EMR Support
 
-Spark NLP 6.1.0 has been tested and is compatible with the following EMR releases:
+Spark NLP 6.1.2 has been tested and is compatible with the following EMR releases:
 
 | **EMR Release**    |
 |--------------------|

{spark_nlp-6.1.0.dist-info → spark_nlp-6.1.2.dist-info}/RECORD

@@ -3,7 +3,7 @@ com/johnsnowlabs/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,
 com/johnsnowlabs/ml/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 com/johnsnowlabs/ml/ai/__init__.py,sha256=YQiK2M7U4d8y5irPy_HB8ae0mSpqS9583MH44pnKJXc,295
 com/johnsnowlabs/nlp/__init__.py,sha256=DPIVXtONO5xXyOk-HB0-sNiHAcco17NN13zPS_6Uw8c,294
-sparknlp/__init__.py,sha256=wxPbTrab8A3tELe8XRaGCfuZ-T8Dc8szbOHXH9ZgLIU,13814
+sparknlp/__init__.py,sha256=beylcD_JfS6wohhs_UyT6WE2IBwRT4_C75_xPyG1_BE,13814
 sparknlp/annotation.py,sha256=I5zOxG5vV2RfPZfqN9enT1i4mo6oBcn3Lrzs37QiOiA,5635
 sparknlp/annotation_audio.py,sha256=iRV_InSVhgvAwSRe9NTbUH9v6OGvTM-FPCpSAKVu0mE,1917
 sparknlp/annotation_image.py,sha256=xhCe8Ko-77XqWVuuYHFrjKqF6zPd8Z-RY_rmZXNwCXU,2547
@@ -105,7 +105,7 @@ sparknlp/annotator/dependency/dependency_parser.py,sha256=SxyvHPp8Hs1Xnm5X1nLTMi
 sparknlp/annotator/dependency/typed_dependency_parser.py,sha256=60vPdYkbFk9MPGegg3m9Uik9cMXpMZd8tBvXG39gNww,12456
 sparknlp/annotator/embeddings/__init__.py,sha256=Aw1oaP5DI0OS6259c0TEZZ6j3VFSvYFEerah5a-udVw,2528
 sparknlp/annotator/embeddings/albert_embeddings.py,sha256=6Rd1LIn8oFIpq_ALcJh-RUjPEO7Ht8wsHY6JHSFyMkw,9995
-sparknlp/annotator/embeddings/auto_gguf_embeddings.py,sha256=IlqkPGOH2lmZvxEyDSGX-G90DtTFOe2Rvujfbg5zvlU,20185
+sparknlp/annotator/embeddings/auto_gguf_embeddings.py,sha256=TRAYbhGS4K8uSpsScvDr6uD3lYdxMpCUjwDMhV_74rM,19977
 sparknlp/annotator/embeddings/bert_embeddings.py,sha256=HVUjkg56kBcpGZCo-fmPG5uatMDF3swW_lnbpy1SgSI,8463
 sparknlp/annotator/embeddings/bert_sentence_embeddings.py,sha256=NQy9KuXT9aKsTpYCR5RAeoFWI2YqEGorbdYrf_0KKmw,9148
 sparknlp/annotator/embeddings/bge_embeddings.py,sha256=ZGbxssjJFaSfbcgqAPV5hsu81SnC0obgCVNOoJkArDA,8105
@@ -167,9 +167,10 @@ sparknlp/annotator/sentence/sentence_detector_dl.py,sha256=-Osj9Bm9KyZRTAWkOsK9c
 sparknlp/annotator/sentiment/__init__.py,sha256=Lq3vKaZS1YATLMg0VNXSVtkWL5q5G9taGBvdrvSwnfg,766
 sparknlp/annotator/sentiment/sentiment_detector.py,sha256=m545NGU0Xzg_PO6_qIfpli1uZj7JQcyFgqe9R6wAPFI,8154
 sparknlp/annotator/sentiment/vivekn_sentiment.py,sha256=4rpXWDgzU6ddnbrSCp9VdLb2epCc9oZ3c6XcqxEw8nk,9655
-sparknlp/annotator/seq2seq/__init__.py,sha256=Aj43G1MuQE0mW7LakCWPjiTkIGl7iHPAnKIwT_DfdIM,1781
-sparknlp/annotator/seq2seq/auto_gguf_model.py,sha256=Oah_RvOy9YrvfnnMRMKOGJHnAMYxo0SeczBZsndM3kY,11638
-sparknlp/annotator/seq2seq/auto_gguf_vision_model.py,sha256=EYrm8EW7AMq3AoIKPe7Gp6ayBlFpWeg76AsAr4nanqU,15346
+sparknlp/annotator/seq2seq/__init__.py,sha256=aDiph00Hyq7L8uDY0frtyuHtqFodBqTMbixx_nq4z1I,1841
+sparknlp/annotator/seq2seq/auto_gguf_model.py,sha256=yhZQHMHfp88rQvLHTWyS-8imZrwqp-8RQQwnw6PmHfc,11749
+sparknlp/annotator/seq2seq/auto_gguf_reranker.py,sha256=QpGpyO1_epWzMospTFrfVVLj2KZ_n3gbHN269vo9fbU,12667
+sparknlp/annotator/seq2seq/auto_gguf_vision_model.py,sha256=swBek2026dW6BOX5O9P8Uq41X2GC71VGW0ADFeUIvs0,15299
 sparknlp/annotator/seq2seq/bart_transformer.py,sha256=I1flM4yeCzEAKOdQllBC30XuedxVJ7ferkFhZ6gwEbE,18481
 sparknlp/annotator/seq2seq/cohere_transformer.py,sha256=43LZBVazZMgJRCsN7HaYjVYfJ5hRMV95QZyxMtXq-m4,13496
 sparknlp/annotator/seq2seq/cpm_transformer.py,sha256=0CnBFMlxMu0pD2QZMHyoGtIYgXqfUQm68vr6zEAa6Eg,13290
@@ -223,12 +224,12 @@ sparknlp/common/annotator_properties.py,sha256=7B1os7pBUfHo6b7IPQAXQ-nir0u3tQLzD
 sparknlp/common/annotator_type.py,sha256=ash2Ip1IOOiJamPVyy_XQj8Ja_DRHm0b9Vj4Ni75oKM,1225
 sparknlp/common/coverage_result.py,sha256=No4PSh1HSs3PyRI1zC47x65tWgfirqPI290icHQoXEI,823
 sparknlp/common/match_strategy.py,sha256=kt1MUPqU1wCwk5qCdYk6jubHbU-5yfAYxb9jjAOrdnY,1678
-sparknlp/common/properties.py,sha256=4jDyxr2IGWEuNlGtOoPzqdCF7oLAKGy1z6MtqxUVMug,52704
+sparknlp/common/properties.py,sha256=7eBxODxKmFQAgOtrxUH9ly4LugUlkNRVXNQcM60AUK4,53025
 sparknlp/common/read_as.py,sha256=imxPGwV7jr4Li_acbo0OAHHRGCBbYv-akzEGaBWEfcY,1226
 sparknlp/common/recursive_annotator_approach.py,sha256=vqugBw22cE3Ff7PIpRlnYFuOlchgL0nM26D8j-NdpqU,1449
 sparknlp/common/storage.py,sha256=D91H3p8EIjNspjqAYu6ephRpCUtdcAir4_PrAbkIQWE,4842
 sparknlp/common/utils.py,sha256=Yne6yYcwKxhOZC-U4qfYoDhWUP_6BIaAjI5X_P_df1E,1306
-sparknlp/internal/__init__.py,sha256=f5kxLz6bEX8FCMo6lek30MXbn_lDC-dBTI9s_fARblI,40748
+sparknlp/internal/__init__.py,sha256=m7Y7y-IPkB6aJuGUCM54eOueGOEt65C3ujAzN16hegQ,40995
 sparknlp/internal/annotator_java_ml.py,sha256=UGPoThG0rGXUOXGSQnDzEDW81Mu1s5RPF29v7DFyE3c,1187
 sparknlp/internal/annotator_transformer.py,sha256=fXmc2IWXGybqZpbEU9obmbdBYPc798y42zvSB4tqV9U,1448
 sparknlp/internal/extended_java_wrapper.py,sha256=hwP0133-hDiDf5sBF-P3MtUsuuDj1PpQbtGZQIRwzfk,2240
@@ -247,7 +248,8 @@ sparknlp/pretrained/utils.py,sha256=T1MrvW_DaWk_jcOjVLOea0NMFE9w8fe0ZT_5urZ_nEY,
 sparknlp/reader/__init__.py,sha256=-Toj3AIBki-zXPpV8ezFTI2LX1yP_rK2bhpoa8nBkTw,685
 sparknlp/reader/enums.py,sha256=MNGug9oJ1BBLM1Pbske13kAabalDzHa2kucF5xzFpHs,770
 sparknlp/reader/pdf_to_text.py,sha256=eWw-cwjosmcSZ9eHso0F5QQoeGBBnwsOhzhCXXvMjZA,7169
-sparknlp/reader/reader2doc.py,sha256=xahxkEuNM21mb0-MHQoYLtDF1cbAYrMTRpN1-u5K3ec,6587
+sparknlp/reader/reader2doc.py,sha256=LRqfaL9nidhlPkJIwTJo7SnGYmNNfOqwEdrsWYGEdnI,7146
+sparknlp/reader/reader2table.py,sha256=GC6Yz0gQ83S6XKOi329TUNQuAvLrBxysqDkDRZPvcYA,4759
 sparknlp/reader/sparknlp_reader.py,sha256=MJs8v_ECYaV1SOabI1L_2MkVYEDVImtwgbYypO7DJSY,20623
 sparknlp/training/__init__.py,sha256=qREi9u-5Vc2VjpL6-XZsyvu5jSEIdIhowW7_kKaqMqo,852
 sparknlp/training/conll.py,sha256=wKBiSTrjc6mjsl7Nyt6B8f4yXsDJkZb-sn8iOjix9cE,6961
@@ -279,7 +281,7 @@ sparknlp/training/_tf_graph_builders_1x/ner_dl/dataset_encoder.py,sha256=R4yHFN3
 sparknlp/training/_tf_graph_builders_1x/ner_dl/ner_model.py,sha256=EoCSdcIjqQ3wv13MAuuWrKV8wyVBP0SbOEW41omHlR0,23189
 sparknlp/training/_tf_graph_builders_1x/ner_dl/ner_model_saver.py,sha256=k5CQ7gKV6HZbZMB8cKLUJuZxoZWlP_DFWdZ--aIDwsc,2356
 sparknlp/training/_tf_graph_builders_1x/ner_dl/sentence_grouper.py,sha256=pAxjWhjazSX8Vg0MFqJiuRVw1IbnQNSs-8Xp26L4nko,870
-spark_nlp-6.1.0.dist-info/METADATA,sha256=MDLwobOveRxQL45CWF-NY26iHa3a7PijF9wntBXpeZE,19722
-spark_nlp-6.1.0.dist-info/WHEEL,sha256=JNWh1Fm1UdwIQV075glCn4MVuCRs0sotJIq-J6rbxCU,109
-spark_nlp-6.1.0.dist-info/top_level.txt,sha256=uuytur4pyMRw2H_txNY2ZkaucZHUs22QF8-R03ch_-E,13
-spark_nlp-6.1.0.dist-info/RECORD,,
+spark_nlp-6.1.2.dist-info/METADATA,sha256=l6za09CF7uliVRGYEFRi02vualYyzRt_kPuuVa8MnWg,19774
+spark_nlp-6.1.2.dist-info/WHEEL,sha256=JNWh1Fm1UdwIQV075glCn4MVuCRs0sotJIq-J6rbxCU,109
+spark_nlp-6.1.2.dist-info/top_level.txt,sha256=uuytur4pyMRw2H_txNY2ZkaucZHUs22QF8-R03ch_-E,13
+spark_nlp-6.1.2.dist-info/RECORD,,

sparknlp/__init__.py

@@ -66,7 +66,7 @@ sys.modules['com.johnsnowlabs.ml.ai'] = annotator
 annotators = annotator
 embeddings = annotator
 
-__version__ = "6.1.0"
+__version__ = "6.1.2"
 
 
 def start(gpu=False,

sparknlp/annotator/embeddings/auto_gguf_embeddings.py

@@ -12,8 +12,6 @@
 #  See the License for the specific language governing permissions and
 #  limitations under the License.
 """Contains classes for the AutoGGUFEmbeddings."""
-from typing import List
-
 from sparknlp.common import *
 
 
@@ -32,7 +30,7 @@ class AutoGGUFEmbeddings(AnnotatorModel, HasBatchedAnnotate):
     ...     .setInputCols(["document"]) \\
     ...     .setOutputCol("embeddings")
 
-    The default model is ``"Nomic_Embed_Text_v1.5.Q8_0.gguf"``, if no name is provided.
+    The default model is ``"Qwen3_Embedding_0.6B_Q8_0_gguf"``, if no name is provided.
 
     For extended examples of usage, see the
     `AutoGGUFEmbeddingsTest <https://github.com/JohnSnowLabs/spark-nlp/tree/master/src/test/scala/com/johnsnowlabs/nlp/embeddings/AutoGGUFEmbeddingsTest.scala>`__
@@ -313,12 +311,6 @@ class AutoGGUFEmbeddings(AnnotatorModel, HasBatchedAnnotate):
         "Set the pooling type for embeddings, use model default if unspecified",
         typeConverter=TypeConverters.toString,
     )
-    embedding = Param(
-        Params._dummy(),
-        "embedding",
-        "Whether to load model with embedding support",
-        typeConverter=TypeConverters.toBoolean,
-    )
     flashAttention = Param(
         Params._dummy(),
         "flashAttention",
@@ -489,10 +481,10 @@ class AutoGGUFEmbeddings(AnnotatorModel, HasBatchedAnnotate):
             classname=classname, java_model=java_model
         )
         self._setDefault(
-            embedding=True,
             nCtx=4096,
             nBatch=512,
             poolingType="MEAN",
+            nGpuLayers=99,
         )
 
     @staticmethod
@@ -517,13 +509,13 @@ class AutoGGUFEmbeddings(AnnotatorModel, HasBatchedAnnotate):
         return AutoGGUFEmbeddings(java_model=jModel)
 
     @staticmethod
-    def pretrained(name="Nomic_Embed_Text_v1.5.Q8_0.gguf", lang="en", remote_loc=None):
+    def pretrained(name="Qwen3_Embedding_0.6B_Q8_0_gguf", lang="en", remote_loc=None):
         """Downloads and loads a pretrained model.
 
         Parameters
         ----------
         name : str, optional
-            Name of the pretrained model, by default "Nomic_Embed_Text_v1.5.Q8_0.gguf"
+            Name of the pretrained model, by default "Qwen3_Embedding_0.6B_Q8_0_gguf"
         lang : str, optional
             Language of the pretrained model, by default "en"
         remote_loc : str, optional

sparknlp/annotator/seq2seq/__init__.py

@@ -32,3 +32,4 @@ from sparknlp.annotator.seq2seq.llama3_transformer import *
 from sparknlp.annotator.seq2seq.cohere_transformer import *
 from sparknlp.annotator.seq2seq.olmo_transformer import *
 from sparknlp.annotator.seq2seq.phi4_transformer import *
+from sparknlp.annotator.seq2seq.auto_gguf_reranker import *

sparknlp/annotator/seq2seq/auto_gguf_model.py

@@ -37,7 +37,11 @@ class AutoGGUFModel(AnnotatorModel, HasBatchedAnnotate, HasLlamaCppProperties):
     ...     .setInputCols(["document"]) \\
     ...     .setOutputCol("completions")
 
-    The default model is ``"phi3.5_mini_4k_instruct_q4_gguf"``, if no name is provided.
+    The default model is ``"Phi_4_mini_instruct_Q4_K_M_gguf"``, if no name is provided.
+
+    AutoGGUFModel is also able to load pretrained models from AutoGGUFVisionModel. Just
+    specify the same name for the pretrained method, and it will load the text-part of the
+    multimodal model automatically.
 
     For extended examples of usage, see the
     `AutoGGUFModelTest <https://github.com/JohnSnowLabs/spark-nlp/tree/master/src/test/scala/com/johnsnowlabs/nlp/annotators/seq2seq/AutoGGUFModelTest.scala>`__
@@ -120,8 +124,6 @@ class AutoGGUFModel(AnnotatorModel, HasBatchedAnnotate, HasLlamaCppProperties):
         Set path to static lookup cache to use for lookup decoding (not updated by generation)
     lookupCacheDynamicFilePath
         Set path to dynamic lookup cache to use for lookup decoding (updated by generation)
-    embedding
-        Whether to load model with embedding support
     flashAttention
         Whether to enable Flash Attention
     inputPrefixBos
@@ -252,20 +254,19 @@ class AutoGGUFModel(AnnotatorModel, HasBatchedAnnotate, HasLlamaCppProperties):
             useChatTemplate=True,
             nCtx=4096,
             nBatch=512,
-            embedding=False,
             nPredict=100,
             nGpuLayers=99,
             systemPrompt="You are a helpful assistant."
         )
 
     @staticmethod
-    def loadSavedModel(folder, spark_session):
+    def loadSavedModel(path, spark_session):
         """Loads a locally saved model.
 
         Parameters
         ----------
-        folder : str
-            Folder of the saved model
+        path : str
+            Path to the gguf model
         spark_session : pyspark.sql.SparkSession
             The current SparkSession
 
@@ -275,17 +276,17 @@ class AutoGGUFModel(AnnotatorModel, HasBatchedAnnotate, HasLlamaCppProperties):
             The restored model
         """
         from sparknlp.internal import _AutoGGUFLoader
-        jModel = _AutoGGUFLoader(folder, spark_session._jsparkSession)._java_obj
+        jModel = _AutoGGUFLoader(path, spark_session._jsparkSession)._java_obj
         return AutoGGUFModel(java_model=jModel)
 
     @staticmethod
-    def pretrained(name="phi3.5_mini_4k_instruct_q4_gguf", lang="en", remote_loc=None):
+    def pretrained(name="Phi_4_mini_instruct_Q4_K_M_gguf", lang="en", remote_loc=None):
         """Downloads and loads a pretrained model.
 
         Parameters
         ----------
         name : str, optional
-            Name of the pretrained model, by default "phi3.5_mini_4k_instruct_q4_gguf"
+            Name of the pretrained model, by default "Phi_4_mini_instruct_Q4_K_M_gguf"
         lang : str, optional
             Language of the pretrained model, by default "en"
         remote_loc : str, optional

sparknlp/annotator/seq2seq/auto_gguf_reranker.py

@@ -0,0 +1,329 @@
+#  Copyright 2017-2023 John Snow Labs
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+"""Contains classes for the AutoGGUFReranker."""
+from typing import List, Dict
+
+from sparknlp.common import *
+
+
+class AutoGGUFReranker(AnnotatorModel, HasBatchedAnnotate, HasLlamaCppProperties):
+    """
+    Annotator that uses the llama.cpp library to rerank text documents based on their relevance
+    to a given query using GGUF-format reranking models.
+
+    This annotator is specifically designed for text reranking tasks, where multiple documents
+    or text passages are ranked according to their relevance to a query. It uses specialized
+    reranking models in GGUF format that output relevance scores for each input document.
+
+    The reranker takes a query (set via :meth:`.setQuery`) and a list of documents, then returns the
+    same documents with added metadata containing relevance scores. The documents are processed
+    in batches and each receives a ``relevance_score`` in its metadata indicating how relevant
+    it is to the provided query.
+
+    For settable parameters, and their explanations, see the parameters of this class and refer to
+    the llama.cpp documentation of
+    `server.cpp <https://github.com/ggerganov/llama.cpp/tree/7d5e8777ae1d21af99d4f95be10db4870720da91/examples/server>`__
+    for more information.
+
+    If the parameters are not set, the annotator will default to use the parameters provided by
+    the model.
+
+    Pretrained models can be loaded with :meth:`.pretrained` of the companion
+    object:
+
+    >>> reranker = AutoGGUFReranker.pretrained() \\
+    ...     .setInputCols(["document"]) \\
+    ...     .setOutputCol("reranked_documents") \\
+    ...     .setQuery("A man is eating pasta.")
+
+    The default model is ``"bge-reranker-v2-m3-Q4_K_M"``, if no name is provided.
+
+    For extended examples of usage, see the
+    `AutoGGUFRerankerTest <https://github.com/JohnSnowLabs/spark-nlp/tree/master/src/test/scala/com/johnsnowlabs/nlp/annotators/seq2seq/AutoGGUFRerankerTest.scala>`__
+    and the
+    `example notebook <https://github.com/JohnSnowLabs/spark-nlp/tree/master/examples/python/llama.cpp/llama.cpp_in_Spark_NLP_AutoGGUFReranker.ipynb>`__.
+
+    For available pretrained models please see the `Models Hub <https://sparknlp.org/models>`__.
+
+    ====================== ======================
+    Input Annotation types Output Annotation type
+    ====================== ======================
+    ``DOCUMENT``           ``DOCUMENT``
+    ====================== ======================
+
+    Parameters
+    ----------
+    query
+        The query to be used for reranking. If not set, the input text will be used as the query.
+    nThreads
+        Set the number of threads to use during generation
+    nThreadsDraft
+        Set the number of threads to use during draft generation
+    nThreadsBatch
+        Set the number of threads to use during batch and prompt processing
+    nThreadsBatchDraft
+        Set the number of threads to use during batch and prompt processing
+    nCtx
+        Set the size of the prompt context
+    nBatch
+        Set the logical batch size for prompt processing (must be >=32 to use BLAS)
+    nUbatch
+        Set the physical batch size for prompt processing (must be >=32 to use BLAS)
+    nGpuLayers
+        Set the number of layers to store in VRAM (-1 - use default)
+    nGpuLayersDraft
+        Set the number of layers to store in VRAM for the draft model (-1 - use default)
+    gpuSplitMode
+        Set how to split the model across GPUs
+    mainGpu
+        Set the main GPU that is used for scratch and small tensors.
+    tensorSplit
+        Set how split tensors should be distributed across GPUs
+    grpAttnN
+        Set the group-attention factor
+    grpAttnW
+        Set the group-attention width
+    ropeFreqBase
+        Set the RoPE base frequency, used by NTK-aware scaling
+    ropeFreqScale
+        Set the RoPE frequency scaling factor, expands context by a factor of 1/N
+    yarnExtFactor
+        Set the YaRN extrapolation mix factor
+    yarnAttnFactor
+        Set the YaRN scale sqrt(t) or attention magnitude
+    yarnBetaFast
+        Set the YaRN low correction dim or beta
+    yarnBetaSlow
+        Set the YaRN high correction dim or alpha
+    yarnOrigCtx
+        Set the YaRN original context size of model
+    defragmentationThreshold
+        Set the KV cache defragmentation threshold
+    numaStrategy
+        Set optimization strategies that help on some NUMA systems (if available)
+    ropeScalingType
+        Set the RoPE frequency scaling method, defaults to linear unless specified by the model
+    poolingType
+        Set the pooling type for embeddings, use model default if unspecified
+    modelDraft
+        Set the draft model for speculative decoding
+    modelAlias
+        Set a model alias
+    lookupCacheStaticFilePath
+        Set path to static lookup cache to use for lookup decoding (not updated by generation)
+    lookupCacheDynamicFilePath
+        Set path to dynamic lookup cache to use for lookup decoding (updated by generation)
+    flashAttention
+        Whether to enable Flash Attention
+    inputPrefixBos
+        Whether to add prefix BOS to user inputs, preceding the `--in-prefix` string
+    useMmap
+        Whether to use memory-map model (faster load but may increase pageouts if not using mlock)
+    useMlock
+        Whether to force the system to keep model in RAM rather than swapping or compressing
+    noKvOffload
+        Whether to disable KV offload
+    systemPrompt
+        Set a system prompt to use
+    chatTemplate
+        The chat template to use
+    inputPrefix
+        Set the prompt to start generation with
+    inputSuffix
+        Set a suffix for infilling
+    cachePrompt
+        Whether to remember the prompt to avoid reprocessing it
+    nPredict
+        Set the number of tokens to predict
+    topK
+        Set top-k sampling
+    topP
+        Set top-p sampling
+    minP
+        Set min-p sampling
+    tfsZ
+        Set tail free sampling, parameter z
+    typicalP
+        Set locally typical sampling, parameter p
+    temperature
+        Set the temperature
+    dynatempRange
+        Set the dynamic temperature range
+    dynatempExponent
+        Set the dynamic temperature exponent
+    repeatLastN
+        Set the last n tokens to consider for penalties
+    repeatPenalty
+        Set the penalty of repeated sequences of tokens
+    frequencyPenalty
+        Set the repetition alpha frequency penalty
+    presencePenalty
+        Set the repetition alpha presence penalty
+    miroStat
+        Set MiroStat sampling strategies.
+    mirostatTau
+        Set the MiroStat target entropy, parameter tau
+    mirostatEta
+        Set the MiroStat learning rate, parameter eta
+    penalizeNl
+        Whether to penalize newline tokens
+    nKeep
+        Set the number of tokens to keep from the initial prompt
+    seed
+        Set the RNG seed
+    nProbs
+        Set the amount top tokens probabilities to output if greater than 0.
+    minKeep
+        Set the amount of tokens the samplers should return at least (0 = disabled)
+    grammar
+        Set BNF-like grammar to constrain generations
+    penaltyPrompt
+        Override which part of the prompt is penalized for repetition.
+    ignoreEos
+        Set whether to ignore end of stream token and continue generating (implies --logit-bias 2-inf)
+    disableTokenIds
+        Set the token ids to disable in the completion
+    stopStrings
+        Set strings upon seeing which token generation is stopped
+    samplers
+        Set which samplers to use for token generation in the given order
+    useChatTemplate
+        Set whether or not generate should apply a chat template
+
+    Notes
+    -----
+    This annotator is designed for reranking tasks and requires setting a query using ``setQuery``.
+    The query represents the search intent against which documents will be ranked. Each input
+    document receives a relevance score in the output metadata.
+
+    To use GPU inference with this annotator, make sure to use the Spark NLP GPU package and set
+    the number of GPU layers with the `setNGpuLayers` method.
+
+    When using larger models, we recommend adjusting GPU usage with `setNCtx` and `setNGpuLayers`
+    according to your hardware to avoid out-of-memory errors.
+
+    Examples
+    --------
+    >>> import sparknlp
+    >>> from sparknlp.base import *
+    >>> from sparknlp.annotator import *
+    >>> from pyspark.ml import Pipeline
+    >>> document = DocumentAssembler() \\
+    ...     .setInputCol("text") \\
+    ...     .setOutputCol("document")
+    >>> reranker = AutoGGUFReranker.pretrained("bge-reranker-v2-m3-Q4_K_M") \\
+    ...     .setInputCols(["document"]) \\
+    ...     .setOutputCol("reranked_documents") \\
+    ...     .setBatchSize(4) \\
+    ...     .setQuery("A man is eating pasta.")
+    >>> pipeline = Pipeline().setStages([document, reranker])
+    >>> data = spark.createDataFrame([
+    ...     ["A man is eating food."],
+    ...     ["A man is eating a piece of bread."],
+    ...     ["The girl is carrying a baby."],
+    ...     ["A man is riding a horse."]
+    ... ]).toDF("text")
+    >>> result = pipeline.fit(data).transform(data)
+    >>> result.select("reranked_documents").show(truncate = False)
+    # Each document will have a relevance_score in metadata showing how relevant it is to the query
+    """
+
+    name = "AutoGGUFReranker"
+    inputAnnotatorTypes = [AnnotatorType.DOCUMENT]
+    outputAnnotatorType = AnnotatorType.DOCUMENT
+
+    query = Param(Params._dummy(), "query",
+                  "The query to be used for reranking. If not set, the input text will be used as the query.",
+                  typeConverter=TypeConverters.toString)
+    @keyword_only
+    def __init__(self, classname="com.johnsnowlabs.nlp.annotators.seq2seq.AutoGGUFReranker", java_model=None):
+        super(AutoGGUFReranker, self).__init__(
+            classname=classname,
+            java_model=java_model
+        )
+        self._setDefault(
+            useChatTemplate=True,
+            nCtx=4096,
+            nBatch=512,
+            nGpuLayers=99,
+            systemPrompt="You are a helpful assistant.",
+            query=""
+        )
+
+    def setQuery(self, value: str):
+        """Set the query to be used for reranking.
+        
+        Parameters
+        ----------
+        value : str
+            The query text that documents will be ranked against.
+            
+        Returns
+        -------
+        AutoGGUFReranker
+            This instance for method chaining.
+        """
+        return self._set(query=value)
+
+    def getQuery(self):
+        """Get the current query used for reranking.
+        
+        Returns
+        -------
+        str
+            The current query string.
+        """
+        return self._call_java("getQuery")
+
+    @staticmethod
+    def loadSavedModel(folder, spark_session):
+        """Loads a locally saved model.
+
+        Parameters
+        ----------
+        folder : str
+            Folder of the saved model
+        spark_session : pyspark.sql.SparkSession
+            The current SparkSession
+
+        Returns
+        -------
+        AutoGGUFReranker
+            The restored model
+        """
+        from sparknlp.internal import _AutoGGUFRerankerLoader
+        jModel = _AutoGGUFRerankerLoader(folder, spark_session._jsparkSession)._java_obj
+        return AutoGGUFReranker(java_model=jModel)
+
+    @staticmethod
+    def pretrained(name="bge-reranker-v2-m3-Q4_K_M", lang="en", remote_loc=None):
+        """Downloads and loads a pretrained model.
+
+        Parameters
+        ----------
+        name : str, optional
+            Name of the pretrained model, by default "bge-reranker-v2-m3-Q4_K_M"
+        lang : str, optional
+            Language of the pretrained model, by default "en"
+        remote_loc : str, optional
+            Optional remote address of the resource, by default None. Will use
+            Spark NLPs repositories otherwise.
+
+        Returns
+        -------
+        AutoGGUFReranker
+            The restored model
+        """
+        from sparknlp.pretrained import ResourceDownloader
+        return ResourceDownloader.downloadModel(AutoGGUFReranker, name, lang, remote_loc)

sparknlp/annotator/seq2seq/auto_gguf_vision_model.py

@@ -43,7 +43,7 @@ class AutoGGUFVisionModel(AnnotatorModel, HasBatchedAnnotate, HasLlamaCppPropert
             .setOutputCol("completions")
 
 
-    The default model is ``"llava_v1.5_7b_Q4_0_gguf"``, if no name is provided.
+    The default model is ``"Qwen2.5_VL_3B_Instruct_Q4_K_M_gguf"``, if no name is provided.
 
     For available pretrained models please see the `Models Hub <https://sparknlp.org/models>`__.
 
@@ -116,8 +116,6 @@ class AutoGGUFVisionModel(AnnotatorModel, HasBatchedAnnotate, HasLlamaCppPropert
         Set optimization strategies that help on some NUMA systems (if available)
     ropeScalingType
         Set the RoPE frequency scaling method, defaults to linear unless specified by the model
-    poolingType
-        Set the pooling type for embeddings, use model default if unspecified
     modelDraft
         Set the draft model for speculative decoding
     modelAlias
@@ -126,8 +124,6 @@ class AutoGGUFVisionModel(AnnotatorModel, HasBatchedAnnotate, HasLlamaCppPropert
         Set path to static lookup cache to use for lookup decoding (not updated by generation)
     lookupCacheDynamicFilePath
         Set path to dynamic lookup cache to use for lookup decoding (updated by generation)
-    embedding
-        Whether to load model with embedding support
     flashAttention
         Whether to enable Flash Attention
     inputPrefixBos
@@ -284,8 +280,10 @@ class AutoGGUFVisionModel(AnnotatorModel, HasBatchedAnnotate, HasLlamaCppPropert
             useChatTemplate=True,
             nCtx=4096,
             nBatch=512,
-            embedding=False,
-            nPredict=100
+            nPredict=100,
+            nGpuLayers=99,
+            systemPrompt="You are a helpful assistant.",
+            batchSize=2,
         )
 
     @staticmethod
@@ -311,13 +309,13 @@ class AutoGGUFVisionModel(AnnotatorModel, HasBatchedAnnotate, HasLlamaCppPropert
         return AutoGGUFVisionModel(java_model=jModel)
 
     @staticmethod
-    def pretrained(name="llava_v1.5_7b_Q4_0_gguf", lang="en", remote_loc=None):
+    def pretrained(name="Qwen2.5_VL_3B_Instruct_Q4_K_M_gguf", lang="en", remote_loc=None):
         """Downloads and loads a pretrained model.
 
         Parameters
         ----------
         name : str, optional
-            Name of the pretrained model, by default "llava_v1.5_7b_Q4_0_gguf"
+            Name of the pretrained model, by default "Qwen2.5_VL_3B_Instruct_Q4_K_M_gguf"
         lang : str, optional
             Language of the pretrained model, by default "en"
         remote_loc : str, optional

sparknlp/common/properties.py

@@ -628,7 +628,6 @@ class HasGeneratorProperties:
                              "The number of sequences to return from the beam search.",
                              typeConverter=TypeConverters.toInt)
 
-
     def setTask(self, value):
         """Sets the transformer's task, e.g. ``summarize:``.
 
@@ -639,7 +638,6 @@ class HasGeneratorProperties:
         """
         return self._set(task=value)
 
-
     def setMinOutputLength(self, value):
         """Sets minimum length of the sequence to be generated.
 
@@ -650,7 +648,6 @@ class HasGeneratorProperties:
         """
         return self._set(minOutputLength=value)
 
-
     def setMaxOutputLength(self, value):
         """Sets maximum length of output text.
 
@@ -661,7 +658,6 @@ class HasGeneratorProperties:
         """
         return self._set(maxOutputLength=value)
 
-
     def setDoSample(self, value):
         """Sets whether or not to use sampling, use greedy decoding otherwise.
 
@@ -672,7 +668,6 @@ class HasGeneratorProperties:
         """
         return self._set(doSample=value)
 
-
     def setTemperature(self, value):
         """Sets the value used to module the next token probabilities.
 
@@ -683,7 +678,6 @@ class HasGeneratorProperties:
         """
         return self._set(temperature=value)
 
-
     def setTopK(self, value):
         """Sets the number of highest probability vocabulary tokens to keep for
         top-k-filtering.
@@ -695,7 +689,6 @@ class HasGeneratorProperties:
         """
         return self._set(topK=value)
 
-
     def setTopP(self, value):
         """Sets the top cumulative probability for vocabulary tokens.
 
@@ -709,7 +702,6 @@ class HasGeneratorProperties:
         """
         return self._set(topP=value)
 
-
     def setRepetitionPenalty(self, value):
         """Sets the parameter for repetition penalty. 1.0 means no penalty.
 
@@ -725,7 +717,6 @@ class HasGeneratorProperties:
         """
         return self._set(repetitionPenalty=value)
 
-
     def setNoRepeatNgramSize(self, value):
         """Sets size of n-grams that can only occur once.
 
@@ -738,7 +729,6 @@ class HasGeneratorProperties:
         """
         return self._set(noRepeatNgramSize=value)
 
-
     def setBeamSize(self, value):
         """Sets the number of beam size for beam search.
 
@@ -749,7 +739,6 @@ class HasGeneratorProperties:
         """
         return self._set(beamSize=value)
 
-
     def setNReturnSequences(self, value):
         """Sets the number of sequences to return from the beam search.
 
@@ -845,11 +834,10 @@ class HasLlamaCppProperties:
                             typeConverter=TypeConverters.toString)
     # Set the pooling type for embeddings, use model default if unspecified
     #
-    #   - 0 NONE: Don't use any pooling
-    #   - 1 MEAN: Mean Pooling
-    #   - 2 CLS: CLS Pooling
-    #   - 3 LAST: Last token pooling
-    #   - 4 RANK: For reranked models
+    #   - MEAN: Mean Pooling
+    #   - CLS: CLS Pooling
+    #   - LAST: Last token pooling
+    #   - RANK: For reranked models
     poolingType = Param(Params._dummy(), "poolingType",
                         "Set the pooling type for embeddings, use model default if unspecified",
                         typeConverter=TypeConverters.toString)
@@ -882,6 +870,10 @@ class HasLlamaCppProperties:
                          typeConverter=TypeConverters.toString)
     chatTemplate = Param(Params._dummy(), "chatTemplate", "The chat template to use",
                          typeConverter=TypeConverters.toString)
+    logVerbosity = Param(Params._dummy(), "logVerbosity", "Set the log verbosity level",
+                         typeConverter=TypeConverters.toInt)
+    disableLog = Param(Params._dummy(), "disableLog", "Whether to disable logging",
+                       typeConverter=TypeConverters.toBoolean)
 
     # -------- INFERENCE PARAMETERS --------
     inputPrefix = Param(Params._dummy(), "inputPrefix", "Set the prompt to start generation with",
@@ -1082,10 +1074,10 @@ class HasLlamaCppProperties:
         ropeScalingTypeUpper = ropeScalingType.upper()
         ropeScalingTypes = ["NONE", "LINEAR", "YARN"]
         if ropeScalingTypeUpper not in ropeScalingTypes:
-           raise ValueError(
-               f"Invalid RoPE scaling type: {ropeScalingType}. "
-               + f"Valid values are: {ropeScalingTypes}"
-           )
+            raise ValueError(
+                f"Invalid RoPE scaling type: {ropeScalingType}. "
+                + f"Valid values are: {ropeScalingTypes}"
+            )
         return self._set(ropeScalingType=ropeScalingTypeUpper)
 
     def setPoolingType(self, poolingType: str):
@@ -1093,11 +1085,10 @@ class HasLlamaCppProperties:
 
         Possible values:
 
-        - 0 NONE: Don't use any pooling
-        - 1 MEAN: Mean Pooling
-        - 2 CLS: CLS Pooling
-        - 3 LAST: Last token pooling
-        - 4 RANK: For reranked models
+        - MEAN: Mean Pooling
+        - CLS: CLS Pooling
+        - LAST: Last token pooling
+        - RANK: For reranked models
         """
         poolingTypeUpper = poolingType.upper()
         poolingTypes = ["NONE", "MEAN", "CLS", "LAST", "RANK"]
@@ -1124,10 +1115,6 @@ class HasLlamaCppProperties:
     #     """Set path to dynamic lookup cache to use for lookup decoding (updated by generation)"""
     #     return self._set(lookupCacheDynamicFilePath=lookupCacheDynamicFilePath)
 
-    def setEmbedding(self, embedding: bool):
-        """Whether to load model with embedding support"""
-        return self._set(embedding=embedding)
-
     def setFlashAttention(self, flashAttention: bool):
         """Whether to enable Flash Attention"""
         return self._set(flashAttention=flashAttention)
@@ -1280,11 +1267,19 @@ class HasLlamaCppProperties:
     def setUseChatTemplate(self, useChatTemplate: bool):
         """Set whether generate should apply a chat template"""
         return self._set(useChatTemplate=useChatTemplate)
-    
+
     def setNParallel(self, nParallel: int):
         """Sets the number of parallel processes for decoding. This is an alias for `setBatchSize`."""
         return self.setBatchSize(nParallel)
 
+    def setLogVerbosity(self, logVerbosity: int):
+        """Set the log verbosity level"""
+        return self._set(logVerbosity=logVerbosity)
+
+    def setDisableLog(self, disableLog: bool):
+        """Whether to disable logging"""
+        return self._set(disableLog=disableLog)
+
     # -------- JAVA SETTERS --------
     def setTokenIdBias(self, tokenIdBias: Dict[int, float]):
         """Set token id bias"""

sparknlp/internal/__init__.py

@@ -1191,4 +1191,9 @@ class _Phi4Loader(ExtendedJavaWrapper):
             path,
             jspark,
             use_openvino,
-        )
+        )
+
+class _AutoGGUFRerankerLoader(ExtendedJavaWrapper):
+    def __init__(self, path, jspark):
+        super(_AutoGGUFRerankerLoader, self).__init__(
+            "com.johnsnowlabs.nlp.annotators.seq2seq.AutoGGUFReranker.loadSavedModel", path, jspark)

sparknlp/reader/reader2doc.py

@@ -25,7 +25,7 @@ class Reader2Doc(
     HasExcelReaderProperties,
     HasHTMLReaderProperties,
     HasPowerPointProperties,
-    HasTextReaderProperties,
+    HasTextReaderProperties
 ):
     """
     The Reader2Doc annotator allows you to use reading files more smoothly within existing
@@ -36,7 +36,7 @@ class Reader2Doc(
     output as a structured Spark DataFrame.
 
     Supported formats include:
-    
+
     - Plain text
     - HTML
     - Word (.doc/.docx)
@@ -77,42 +77,49 @@ class Reader2Doc(
         Params._dummy(),
         "contentPath",
         "contentPath path to files to read",
-        typeConverter=TypeConverters.toString,
+        typeConverter=TypeConverters.toString
     )
 
     outputCol = Param(
         Params._dummy(),
         "outputCol",
         "output column name",
-        typeConverter=TypeConverters.toString,
+        typeConverter=TypeConverters.toString
     )
 
     contentType = Param(
         Params._dummy(),
         "contentType",
         "Set the content type to load following MIME specification",
-        typeConverter=TypeConverters.toString,
+        typeConverter=TypeConverters.toString
     )
 
     explodeDocs = Param(
         Params._dummy(),
         "explodeDocs",
         "whether to explode the documents into separate rows",
-        typeConverter=TypeConverters.toBoolean,
+        typeConverter=TypeConverters.toBoolean
     )
 
     flattenOutput = Param(
         Params._dummy(),
         "flattenOutput",
         "If true, output is flattened to plain text with minimal metadata",
-        typeConverter=TypeConverters.toBoolean,
+        typeConverter=TypeConverters.toBoolean
     )
 
     titleThreshold = Param(
         Params._dummy(),
         "titleThreshold",
         "Minimum font size threshold for title detection in PDF docs",
-        typeConverter=TypeConverters.toFloat,
+        typeConverter=TypeConverters.toFloat
+    )
+
+    outputFormat = Param(
+        Params._dummy(),
+        "outputFormat",
+        "Output format for the table content. Options are 'plain-text' or 'html-table'. Default is 'json-table'.",
+        typeConverter=TypeConverters.toString
     )
 
     @keyword_only
@@ -126,7 +133,6 @@ class Reader2Doc(
             titleThreshold=18
         )
     @keyword_only
-
     def setParams(self):
         kwargs = self._input_kwargs
         return self._set(**kwargs)
@@ -192,3 +198,13 @@ class Reader2Doc(
             Minimum font size threshold for title detection in PDF docs
         """
         return self._set(titleThreshold=value)
+
+    def setOutputFormat(self, value):
+        """Sets the output format for the table content.
+
+        Parameters
+        ----------
+        value : str
+            Output format for the table content. Options are 'plain-text' or 'html-table'. Default is 'json-table'.
+        """
+        return self._set(outputFormat=value)

sparknlp/reader/reader2table.py

@@ -0,0 +1,163 @@
+#  Copyright 2017-2025 John Snow Labs
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+from pyspark import keyword_only
+from pyspark.ml.param import TypeConverters, Params, Param
+
+from sparknlp.common import AnnotatorType
+from sparknlp.internal import AnnotatorTransformer
+from sparknlp.partition.partition_properties import *
+
+class Reader2Table(
+    AnnotatorTransformer,
+    HasEmailReaderProperties,
+    HasExcelReaderProperties,
+    HasHTMLReaderProperties,
+    HasPowerPointProperties,
+    HasTextReaderProperties
+):
+    name = 'Reader2Table'
+
+    outputAnnotatorType = AnnotatorType.DOCUMENT
+
+    contentPath = Param(
+        Params._dummy(),
+        "contentPath",
+        "contentPath path to files to read",
+        typeConverter=TypeConverters.toString
+    )
+
+    outputCol = Param(
+        Params._dummy(),
+        "outputCol",
+        "output column name",
+        typeConverter=TypeConverters.toString
+    )
+
+    contentType = Param(
+        Params._dummy(),
+        "contentType",
+        "Set the content type to load following MIME specification",
+        typeConverter=TypeConverters.toString
+    )
+
+    explodeDocs = Param(
+        Params._dummy(),
+        "explodeDocs",
+        "whether to explode the documents into separate rows",
+        typeConverter=TypeConverters.toBoolean
+    )
+
+    flattenOutput = Param(
+        Params._dummy(),
+        "flattenOutput",
+        "If true, output is flattened to plain text with minimal metadata",
+        typeConverter=TypeConverters.toBoolean
+    )
+
+    titleThreshold = Param(
+        Params._dummy(),
+        "titleThreshold",
+        "Minimum font size threshold for title detection in PDF docs",
+        typeConverter=TypeConverters.toFloat
+    )
+
+    outputFormat = Param(
+        Params._dummy(),
+        "outputFormat",
+        "Output format for the table content. Options are 'plain-text' or 'html-table'. Default is 'json-table'.",
+        typeConverter=TypeConverters.toString
+    )
+
+    @keyword_only
+    def __init__(self):
+        super(Reader2Table, self).__init__(classname="com.johnsnowlabs.reader.Reader2Table")
+        self._setDefault(outputCol="document")
+
+    @keyword_only
+    def setParams(self):
+        kwargs = self._input_kwargs
+        return self._set(**kwargs)
+
+    def setContentPath(self, value):
+        """Sets content path.
+
+        Parameters
+        ----------
+        value : str
+            contentPath path to files to read
+        """
+        return self._set(contentPath=value)
+
+    def setContentType(self, value):
+        """
+        Set the content type to load following MIME specification
+
+        Parameters
+        ----------
+        value : str
+            content type to load following MIME specification
+        """
+        return self._set(contentType=value)
+
+    def setExplodeDocs(self, value):
+        """Sets whether to explode the documents into separate rows.
+
+
+        Parameters
+        ----------
+        value : boolean
+        Whether to explode the documents into separate rows
+        """
+        return self._set(explodeDocs=value)
+
+    def setOutputCol(self, value):
+        """Sets output column name.
+
+        Parameters
+        ----------
+        value : str
+            Name of the Output Column
+        """
+        return self._set(outputCol=value)
+
+    def setFlattenOutput(self, value):
+        """Sets whether to flatten the output to plain text with minimal metadata.
+
+        Parameters
+        ----------
+        value : bool
+            If true, output is flattened to plain text with minimal metadata
+        """
+        return self._set(flattenOutput=value)
+
+    def setTitleThreshold(self, value):
+        """Sets the minimum font size threshold for title detection in PDF documents.
+
+        Parameters
+        ----------
+        value : float
+            Minimum font size threshold for title detection in PDF docs
+        """
+        return self._set(titleThreshold=value)
+
+    def setOutputFormat(self, value):
+        """Sets the output format for the table content.
+
+        Parameters
+        ----------
+        value : str
+            Output format for the table content. Options are 'plain-text' or 'html-table'. Default is 'json-table'.
+        """
+        return self._set(outputFormat=value)