spark-nlp 5.3.1__py2.py3-none-any.whl → 5.3.2__py2.py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of spark-nlp might be problematic. Click here for more details.
- spark_nlp-5.3.2.dist-info/.uuid +1 -0
- {spark_nlp-5.3.1.dist-info → spark_nlp-5.3.2.dist-info}/METADATA +45 -45
- {spark_nlp-5.3.1.dist-info → spark_nlp-5.3.2.dist-info}/RECORD +6 -5
- sparknlp/__init__.py +2 -2
- {spark_nlp-5.3.1.dist-info → spark_nlp-5.3.2.dist-info}/WHEEL +0 -0
- {spark_nlp-5.3.1.dist-info → spark_nlp-5.3.2.dist-info}/top_level.txt +0 -0
|
@@ -0,0 +1 @@
|
|
|
1
|
+
90f78083-0ee0-43e9-8240-7263731b6707
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: spark-nlp
|
|
3
|
-
Version: 5.3.
|
|
3
|
+
Version: 5.3.2
|
|
4
4
|
Summary: John Snow Labs Spark NLP is a natural language processing library built on top of Apache Spark ML. It provides simple, performant & accurate NLP annotations for machine learning pipelines, that scale easily in a distributed environment.
|
|
5
5
|
Home-page: https://github.com/JohnSnowLabs/spark-nlp
|
|
6
6
|
Author: John Snow Labs
|
|
@@ -197,7 +197,7 @@ To use Spark NLP you need the following requirements:
|
|
|
197
197
|
|
|
198
198
|
**GPU (optional):**
|
|
199
199
|
|
|
200
|
-
Spark NLP 5.3.
|
|
200
|
+
Spark NLP 5.3.2 is built with ONNX 1.17.0 and TensorFlow 2.7.1 deep learning engines. The minimum following NVIDIA® software are only required for GPU support:
|
|
201
201
|
|
|
202
202
|
- NVIDIA® GPU drivers version 450.80.02 or higher
|
|
203
203
|
- CUDA® Toolkit 11.2
|
|
@@ -213,7 +213,7 @@ $ java -version
|
|
|
213
213
|
$ conda create -n sparknlp python=3.7 -y
|
|
214
214
|
$ conda activate sparknlp
|
|
215
215
|
# spark-nlp by default is based on pyspark 3.x
|
|
216
|
-
$ pip install spark-nlp==5.3.
|
|
216
|
+
$ pip install spark-nlp==5.3.2 pyspark==3.3.1
|
|
217
217
|
```
|
|
218
218
|
|
|
219
219
|
In Python console or Jupyter `Python3` kernel:
|
|
@@ -258,7 +258,7 @@ For more examples, you can visit our dedicated [examples](https://github.com/Joh
|
|
|
258
258
|
|
|
259
259
|
## Apache Spark Support
|
|
260
260
|
|
|
261
|
-
Spark NLP *5.3.
|
|
261
|
+
Spark NLP *5.3.2* has been built on top of Apache Spark 3.4 while fully supports Apache Spark 3.0.x, 3.1.x, 3.2.x, 3.3.x, 3.4.x, and 3.5.x
|
|
262
262
|
|
|
263
263
|
| Spark NLP | Apache Spark 3.5.x | Apache Spark 3.4.x | Apache Spark 3.3.x | Apache Spark 3.2.x | Apache Spark 3.1.x | Apache Spark 3.0.x | Apache Spark 2.4.x | Apache Spark 2.3.x |
|
|
264
264
|
|-----------|--------------------|--------------------|--------------------|--------------------|--------------------|--------------------|--------------------|--------------------|
|
|
@@ -302,7 +302,7 @@ Find out more about `Spark NLP` versions from our [release notes](https://github
|
|
|
302
302
|
|
|
303
303
|
## Databricks Support
|
|
304
304
|
|
|
305
|
-
Spark NLP 5.3.
|
|
305
|
+
Spark NLP 5.3.2 has been tested and is compatible with the following runtimes:
|
|
306
306
|
|
|
307
307
|
**CPU:**
|
|
308
308
|
|
|
@@ -375,7 +375,7 @@ Spark NLP 5.3.1 has been tested and is compatible with the following runtimes:
|
|
|
375
375
|
|
|
376
376
|
## EMR Support
|
|
377
377
|
|
|
378
|
-
Spark NLP 5.3.
|
|
378
|
+
Spark NLP 5.3.2 has been tested and is compatible with the following EMR releases:
|
|
379
379
|
|
|
380
380
|
- emr-6.2.0
|
|
381
381
|
- emr-6.3.0
|
|
@@ -425,11 +425,11 @@ Spark NLP supports all major releases of Apache Spark 3.0.x, Apache Spark 3.1.x,
|
|
|
425
425
|
```sh
|
|
426
426
|
# CPU
|
|
427
427
|
|
|
428
|
-
spark-shell --packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
428
|
+
spark-shell --packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2
|
|
429
429
|
|
|
430
|
-
pyspark --packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
430
|
+
pyspark --packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2
|
|
431
431
|
|
|
432
|
-
spark-submit --packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
432
|
+
spark-submit --packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2
|
|
433
433
|
```
|
|
434
434
|
|
|
435
435
|
The `spark-nlp` has been published to
|
|
@@ -438,11 +438,11 @@ the [Maven Repository](https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/s
|
|
|
438
438
|
```sh
|
|
439
439
|
# GPU
|
|
440
440
|
|
|
441
|
-
spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:5.3.
|
|
441
|
+
spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:5.3.2
|
|
442
442
|
|
|
443
|
-
pyspark --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:5.3.
|
|
443
|
+
pyspark --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:5.3.2
|
|
444
444
|
|
|
445
|
-
spark-submit --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:5.3.
|
|
445
|
+
spark-submit --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:5.3.2
|
|
446
446
|
|
|
447
447
|
```
|
|
448
448
|
|
|
@@ -452,11 +452,11 @@ the [Maven Repository](https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/s
|
|
|
452
452
|
```sh
|
|
453
453
|
# AArch64
|
|
454
454
|
|
|
455
|
-
spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-aarch64_2.12:5.3.
|
|
455
|
+
spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-aarch64_2.12:5.3.2
|
|
456
456
|
|
|
457
|
-
pyspark --packages com.johnsnowlabs.nlp:spark-nlp-aarch64_2.12:5.3.
|
|
457
|
+
pyspark --packages com.johnsnowlabs.nlp:spark-nlp-aarch64_2.12:5.3.2
|
|
458
458
|
|
|
459
|
-
spark-submit --packages com.johnsnowlabs.nlp:spark-nlp-aarch64_2.12:5.3.
|
|
459
|
+
spark-submit --packages com.johnsnowlabs.nlp:spark-nlp-aarch64_2.12:5.3.2
|
|
460
460
|
|
|
461
461
|
```
|
|
462
462
|
|
|
@@ -466,11 +466,11 @@ the [Maven Repository](https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/s
|
|
|
466
466
|
```sh
|
|
467
467
|
# M1/M2 (Apple Silicon)
|
|
468
468
|
|
|
469
|
-
spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-silicon_2.12:5.3.
|
|
469
|
+
spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-silicon_2.12:5.3.2
|
|
470
470
|
|
|
471
|
-
pyspark --packages com.johnsnowlabs.nlp:spark-nlp-silicon_2.12:5.3.
|
|
471
|
+
pyspark --packages com.johnsnowlabs.nlp:spark-nlp-silicon_2.12:5.3.2
|
|
472
472
|
|
|
473
|
-
spark-submit --packages com.johnsnowlabs.nlp:spark-nlp-silicon_2.12:5.3.
|
|
473
|
+
spark-submit --packages com.johnsnowlabs.nlp:spark-nlp-silicon_2.12:5.3.2
|
|
474
474
|
|
|
475
475
|
```
|
|
476
476
|
|
|
@@ -484,7 +484,7 @@ set in your SparkSession:
|
|
|
484
484
|
spark-shell \
|
|
485
485
|
--driver-memory 16g \
|
|
486
486
|
--conf spark.kryoserializer.buffer.max=2000M \
|
|
487
|
-
--packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
487
|
+
--packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2
|
|
488
488
|
```
|
|
489
489
|
|
|
490
490
|
## Scala
|
|
@@ -502,7 +502,7 @@ coordinates:
|
|
|
502
502
|
<dependency>
|
|
503
503
|
<groupId>com.johnsnowlabs.nlp</groupId>
|
|
504
504
|
<artifactId>spark-nlp_2.12</artifactId>
|
|
505
|
-
<version>5.3.
|
|
505
|
+
<version>5.3.2</version>
|
|
506
506
|
</dependency>
|
|
507
507
|
```
|
|
508
508
|
|
|
@@ -513,7 +513,7 @@ coordinates:
|
|
|
513
513
|
<dependency>
|
|
514
514
|
<groupId>com.johnsnowlabs.nlp</groupId>
|
|
515
515
|
<artifactId>spark-nlp-gpu_2.12</artifactId>
|
|
516
|
-
<version>5.3.
|
|
516
|
+
<version>5.3.2</version>
|
|
517
517
|
</dependency>
|
|
518
518
|
```
|
|
519
519
|
|
|
@@ -524,7 +524,7 @@ coordinates:
|
|
|
524
524
|
<dependency>
|
|
525
525
|
<groupId>com.johnsnowlabs.nlp</groupId>
|
|
526
526
|
<artifactId>spark-nlp-aarch64_2.12</artifactId>
|
|
527
|
-
<version>5.3.
|
|
527
|
+
<version>5.3.2</version>
|
|
528
528
|
</dependency>
|
|
529
529
|
```
|
|
530
530
|
|
|
@@ -535,7 +535,7 @@ coordinates:
|
|
|
535
535
|
<dependency>
|
|
536
536
|
<groupId>com.johnsnowlabs.nlp</groupId>
|
|
537
537
|
<artifactId>spark-nlp-silicon_2.12</artifactId>
|
|
538
|
-
<version>5.3.
|
|
538
|
+
<version>5.3.2</version>
|
|
539
539
|
</dependency>
|
|
540
540
|
```
|
|
541
541
|
|
|
@@ -545,28 +545,28 @@ coordinates:
|
|
|
545
545
|
|
|
546
546
|
```sbtshell
|
|
547
547
|
// https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/spark-nlp
|
|
548
|
-
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp" % "5.3.
|
|
548
|
+
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp" % "5.3.2"
|
|
549
549
|
```
|
|
550
550
|
|
|
551
551
|
**spark-nlp-gpu:**
|
|
552
552
|
|
|
553
553
|
```sbtshell
|
|
554
554
|
// https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/spark-nlp-gpu
|
|
555
|
-
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp-gpu" % "5.3.
|
|
555
|
+
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp-gpu" % "5.3.2"
|
|
556
556
|
```
|
|
557
557
|
|
|
558
558
|
**spark-nlp-aarch64:**
|
|
559
559
|
|
|
560
560
|
```sbtshell
|
|
561
561
|
// https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/spark-nlp-aarch64
|
|
562
|
-
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp-aarch64" % "5.3.
|
|
562
|
+
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp-aarch64" % "5.3.2"
|
|
563
563
|
```
|
|
564
564
|
|
|
565
565
|
**spark-nlp-silicon:**
|
|
566
566
|
|
|
567
567
|
```sbtshell
|
|
568
568
|
// https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/spark-nlp-silicon
|
|
569
|
-
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp-silicon" % "5.3.
|
|
569
|
+
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp-silicon" % "5.3.2"
|
|
570
570
|
```
|
|
571
571
|
|
|
572
572
|
Maven
|
|
@@ -588,7 +588,7 @@ If you installed pyspark through pip/conda, you can install `spark-nlp` through
|
|
|
588
588
|
Pip:
|
|
589
589
|
|
|
590
590
|
```bash
|
|
591
|
-
pip install spark-nlp==5.3.
|
|
591
|
+
pip install spark-nlp==5.3.2
|
|
592
592
|
```
|
|
593
593
|
|
|
594
594
|
Conda:
|
|
@@ -617,7 +617,7 @@ spark = SparkSession.builder
|
|
|
617
617
|
.config("spark.driver.memory", "16G")
|
|
618
618
|
.config("spark.driver.maxResultSize", "0")
|
|
619
619
|
.config("spark.kryoserializer.buffer.max", "2000M")
|
|
620
|
-
.config("spark.jars.packages", "com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
620
|
+
.config("spark.jars.packages", "com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2")
|
|
621
621
|
.getOrCreate()
|
|
622
622
|
```
|
|
623
623
|
|
|
@@ -688,7 +688,7 @@ Use either one of the following options
|
|
|
688
688
|
- Add the following Maven Coordinates to the interpreter's library list
|
|
689
689
|
|
|
690
690
|
```bash
|
|
691
|
-
com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
691
|
+
com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2
|
|
692
692
|
```
|
|
693
693
|
|
|
694
694
|
- Add a path to pre-built jar from [here](#compiled-jars) in the interpreter's library list making sure the jar is
|
|
@@ -699,7 +699,7 @@ com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.1
|
|
|
699
699
|
Apart from the previous step, install the python module through pip
|
|
700
700
|
|
|
701
701
|
```bash
|
|
702
|
-
pip install spark-nlp==5.3.
|
|
702
|
+
pip install spark-nlp==5.3.2
|
|
703
703
|
```
|
|
704
704
|
|
|
705
705
|
Or you can install `spark-nlp` from inside Zeppelin by using Conda:
|
|
@@ -727,7 +727,7 @@ launch the Jupyter from the same Python environment:
|
|
|
727
727
|
$ conda create -n sparknlp python=3.8 -y
|
|
728
728
|
$ conda activate sparknlp
|
|
729
729
|
# spark-nlp by default is based on pyspark 3.x
|
|
730
|
-
$ pip install spark-nlp==5.3.
|
|
730
|
+
$ pip install spark-nlp==5.3.2 pyspark==3.3.1 jupyter
|
|
731
731
|
$ jupyter notebook
|
|
732
732
|
```
|
|
733
733
|
|
|
@@ -744,7 +744,7 @@ export PYSPARK_PYTHON=python3
|
|
|
744
744
|
export PYSPARK_DRIVER_PYTHON=jupyter
|
|
745
745
|
export PYSPARK_DRIVER_PYTHON_OPTS=notebook
|
|
746
746
|
|
|
747
|
-
pyspark --packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
747
|
+
pyspark --packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2
|
|
748
748
|
```
|
|
749
749
|
|
|
750
750
|
Alternatively, you can mix in using `--jars` option for pyspark + `pip install spark-nlp`
|
|
@@ -771,7 +771,7 @@ This script comes with the two options to define `pyspark` and `spark-nlp` versi
|
|
|
771
771
|
# -s is for spark-nlp
|
|
772
772
|
# -g will enable upgrading libcudnn8 to 8.1.0 on Google Colab for GPU usage
|
|
773
773
|
# by default they are set to the latest
|
|
774
|
-
!wget https://setup.johnsnowlabs.com/colab.sh -O - | bash /dev/stdin -p 3.2.3 -s 5.3.
|
|
774
|
+
!wget https://setup.johnsnowlabs.com/colab.sh -O - | bash /dev/stdin -p 3.2.3 -s 5.3.2
|
|
775
775
|
```
|
|
776
776
|
|
|
777
777
|
[Spark NLP quick start on Google Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp/blob/master/examples/python/quick_start_google_colab.ipynb)
|
|
@@ -794,7 +794,7 @@ This script comes with the two options to define `pyspark` and `spark-nlp` versi
|
|
|
794
794
|
# -s is for spark-nlp
|
|
795
795
|
# -g will enable upgrading libcudnn8 to 8.1.0 on Kaggle for GPU usage
|
|
796
796
|
# by default they are set to the latest
|
|
797
|
-
!wget https://setup.johnsnowlabs.com/colab.sh -O - | bash /dev/stdin -p 3.2.3 -s 5.3.
|
|
797
|
+
!wget https://setup.johnsnowlabs.com/colab.sh -O - | bash /dev/stdin -p 3.2.3 -s 5.3.2
|
|
798
798
|
```
|
|
799
799
|
|
|
800
800
|
[Spark NLP quick start on Kaggle Kernel](https://www.kaggle.com/mozzie/spark-nlp-named-entity-recognition) is a live
|
|
@@ -813,9 +813,9 @@ demo on Kaggle Kernel that performs named entity recognitions by using Spark NLP
|
|
|
813
813
|
|
|
814
814
|
3. In `Libraries` tab inside your cluster you need to follow these steps:
|
|
815
815
|
|
|
816
|
-
3.1. Install New -> PyPI -> `spark-nlp==5.3.
|
|
816
|
+
3.1. Install New -> PyPI -> `spark-nlp==5.3.2` -> Install
|
|
817
817
|
|
|
818
|
-
3.2. Install New -> Maven -> Coordinates -> `com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
818
|
+
3.2. Install New -> Maven -> Coordinates -> `com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2` -> Install
|
|
819
819
|
|
|
820
820
|
4. Now you can attach your notebook to the cluster and use Spark NLP!
|
|
821
821
|
|
|
@@ -866,7 +866,7 @@ A sample of your software configuration in JSON on S3 (must be public access):
|
|
|
866
866
|
"spark.kryoserializer.buffer.max": "2000M",
|
|
867
867
|
"spark.serializer": "org.apache.spark.serializer.KryoSerializer",
|
|
868
868
|
"spark.driver.maxResultSize": "0",
|
|
869
|
-
"spark.jars.packages": "com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
869
|
+
"spark.jars.packages": "com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2"
|
|
870
870
|
}
|
|
871
871
|
}]
|
|
872
872
|
```
|
|
@@ -875,7 +875,7 @@ A sample of AWS CLI to launch EMR cluster:
|
|
|
875
875
|
|
|
876
876
|
```.sh
|
|
877
877
|
aws emr create-cluster \
|
|
878
|
-
--name "Spark NLP 5.3.
|
|
878
|
+
--name "Spark NLP 5.3.2" \
|
|
879
879
|
--release-label emr-6.2.0 \
|
|
880
880
|
--applications Name=Hadoop Name=Spark Name=Hive \
|
|
881
881
|
--instance-type m4.4xlarge \
|
|
@@ -939,7 +939,7 @@ gcloud dataproc clusters create ${CLUSTER_NAME} \
|
|
|
939
939
|
--enable-component-gateway \
|
|
940
940
|
--metadata 'PIP_PACKAGES=spark-nlp spark-nlp-display google-cloud-bigquery google-cloud-storage' \
|
|
941
941
|
--initialization-actions gs://goog-dataproc-initialization-actions-${REGION}/python/pip-install.sh \
|
|
942
|
-
--properties spark:spark.serializer=org.apache.spark.serializer.KryoSerializer,spark:spark.driver.maxResultSize=0,spark:spark.kryoserializer.buffer.max=2000M,spark:spark.jars.packages=com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
942
|
+
--properties spark:spark.serializer=org.apache.spark.serializer.KryoSerializer,spark:spark.driver.maxResultSize=0,spark:spark.kryoserializer.buffer.max=2000M,spark:spark.jars.packages=com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2
|
|
943
943
|
```
|
|
944
944
|
|
|
945
945
|
2. On an existing one, you need to install spark-nlp and spark-nlp-display packages from PyPI.
|
|
@@ -982,7 +982,7 @@ spark = SparkSession.builder
|
|
|
982
982
|
.config("spark.kryoserializer.buffer.max", "2000m")
|
|
983
983
|
.config("spark.jsl.settings.pretrained.cache_folder", "sample_data/pretrained")
|
|
984
984
|
.config("spark.jsl.settings.storage.cluster_tmp_dir", "sample_data/storage")
|
|
985
|
-
.config("spark.jars.packages", "com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
985
|
+
.config("spark.jars.packages", "com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2")
|
|
986
986
|
.getOrCreate()
|
|
987
987
|
```
|
|
988
988
|
|
|
@@ -996,7 +996,7 @@ spark-shell \
|
|
|
996
996
|
--conf spark.kryoserializer.buffer.max=2000M \
|
|
997
997
|
--conf spark.jsl.settings.pretrained.cache_folder="sample_data/pretrained" \
|
|
998
998
|
--conf spark.jsl.settings.storage.cluster_tmp_dir="sample_data/storage" \
|
|
999
|
-
--packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
999
|
+
--packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2
|
|
1000
1000
|
```
|
|
1001
1001
|
|
|
1002
1002
|
**pyspark:**
|
|
@@ -1009,7 +1009,7 @@ pyspark \
|
|
|
1009
1009
|
--conf spark.kryoserializer.buffer.max=2000M \
|
|
1010
1010
|
--conf spark.jsl.settings.pretrained.cache_folder="sample_data/pretrained" \
|
|
1011
1011
|
--conf spark.jsl.settings.storage.cluster_tmp_dir="sample_data/storage" \
|
|
1012
|
-
--packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.
|
|
1012
|
+
--packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.3.2
|
|
1013
1013
|
```
|
|
1014
1014
|
|
|
1015
1015
|
**Databricks:**
|
|
@@ -1281,7 +1281,7 @@ spark = SparkSession.builder
|
|
|
1281
1281
|
.config("spark.driver.memory", "16G")
|
|
1282
1282
|
.config("spark.driver.maxResultSize", "0")
|
|
1283
1283
|
.config("spark.kryoserializer.buffer.max", "2000M")
|
|
1284
|
-
.config("spark.jars", "/tmp/spark-nlp-assembly-5.3.
|
|
1284
|
+
.config("spark.jars", "/tmp/spark-nlp-assembly-5.3.2.jar")
|
|
1285
1285
|
.getOrCreate()
|
|
1286
1286
|
```
|
|
1287
1287
|
|
|
@@ -1290,7 +1290,7 @@ spark = SparkSession.builder
|
|
|
1290
1290
|
version (3.0.x, 3.1.x, 3.2.x, 3.3.x, 3.4.x, and 3.5.x)
|
|
1291
1291
|
- If you are local, you can load the Fat JAR from your local FileSystem, however, if you are in a cluster setup you need
|
|
1292
1292
|
to put the Fat JAR on a distributed FileSystem such as HDFS, DBFS, S3, etc. (
|
|
1293
|
-
i.e., `hdfs:///tmp/spark-nlp-assembly-5.3.
|
|
1293
|
+
i.e., `hdfs:///tmp/spark-nlp-assembly-5.3.2.jar`)
|
|
1294
1294
|
|
|
1295
1295
|
Example of using pretrained Models and Pipelines in offline:
|
|
1296
1296
|
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
com/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
2
2
|
com/johnsnowlabs/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
3
3
|
com/johnsnowlabs/nlp/__init__.py,sha256=DPIVXtONO5xXyOk-HB0-sNiHAcco17NN13zPS_6Uw8c,294
|
|
4
|
-
sparknlp/__init__.py,sha256=
|
|
4
|
+
sparknlp/__init__.py,sha256=N74jXyWIf6cZnJADb-aBShj-r4eCjcLGDPmXcRNEBN0,13588
|
|
5
5
|
sparknlp/annotation.py,sha256=I5zOxG5vV2RfPZfqN9enT1i4mo6oBcn3Lrzs37QiOiA,5635
|
|
6
6
|
sparknlp/annotation_audio.py,sha256=iRV_InSVhgvAwSRe9NTbUH9v6OGvTM-FPCpSAKVu0mE,1917
|
|
7
7
|
sparknlp/annotation_image.py,sha256=xhCe8Ko-77XqWVuuYHFrjKqF6zPd8Z-RY_rmZXNwCXU,2547
|
|
@@ -224,7 +224,8 @@ sparknlp/training/_tf_graph_builders_1x/ner_dl/dataset_encoder.py,sha256=R4yHFN3
|
|
|
224
224
|
sparknlp/training/_tf_graph_builders_1x/ner_dl/ner_model.py,sha256=EoCSdcIjqQ3wv13MAuuWrKV8wyVBP0SbOEW41omHlR0,23189
|
|
225
225
|
sparknlp/training/_tf_graph_builders_1x/ner_dl/ner_model_saver.py,sha256=k5CQ7gKV6HZbZMB8cKLUJuZxoZWlP_DFWdZ--aIDwsc,2356
|
|
226
226
|
sparknlp/training/_tf_graph_builders_1x/ner_dl/sentence_grouper.py,sha256=pAxjWhjazSX8Vg0MFqJiuRVw1IbnQNSs-8Xp26L4nko,870
|
|
227
|
-
spark_nlp-5.3.
|
|
228
|
-
spark_nlp-5.3.
|
|
229
|
-
spark_nlp-5.3.
|
|
230
|
-
spark_nlp-5.3.
|
|
227
|
+
spark_nlp-5.3.2.dist-info/.uuid,sha256=1f6hF51aIuv9yCvh31NU9lOpS34NE-h3a0Et7R9yR6A,36
|
|
228
|
+
spark_nlp-5.3.2.dist-info/METADATA,sha256=lvuYdEbmUUpC9QYY4YIfuMhBUIhy5axZrrX3XgPc0uQ,57087
|
|
229
|
+
spark_nlp-5.3.2.dist-info/WHEEL,sha256=bb2Ot9scclHKMOLDEHY6B2sicWOgugjFKaJsT7vwMQo,110
|
|
230
|
+
spark_nlp-5.3.2.dist-info/top_level.txt,sha256=uuytur4pyMRw2H_txNY2ZkaucZHUs22QF8-R03ch_-E,13
|
|
231
|
+
spark_nlp-5.3.2.dist-info/RECORD,,
|
sparknlp/__init__.py
CHANGED
|
@@ -128,7 +128,7 @@ def start(gpu=False,
|
|
|
128
128
|
The initiated Spark session.
|
|
129
129
|
|
|
130
130
|
"""
|
|
131
|
-
current_version = "5.3.
|
|
131
|
+
current_version = "5.3.2"
|
|
132
132
|
|
|
133
133
|
if params is None:
|
|
134
134
|
params = {}
|
|
@@ -309,4 +309,4 @@ def version():
|
|
|
309
309
|
str
|
|
310
310
|
The current Spark NLP version.
|
|
311
311
|
"""
|
|
312
|
-
return '5.3.
|
|
312
|
+
return '5.3.2'
|
|
File without changes
|
|
File without changes
|