pyneoinstance 1.0.0__tar.gz → 1.1.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. {pyneoinstance-1.0.0/pyneoinstance.egg-info → pyneoinstance-1.1.2}/PKG-INFO +71 -37
  2. pyneoinstance-1.1.2/README.md +236 -0
  3. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance/__init__.py +1 -1
  4. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance/database/neo4jdbms.py +26 -21
  5. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2/pyneoinstance.egg-info}/PKG-INFO +71 -37
  6. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance.egg-info/requires.txt +1 -1
  7. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyproject.toml +3 -3
  8. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/test/cypher_queries.yaml +11 -8
  9. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/test/test_neo4jdb.py +12 -2
  10. pyneoinstance-1.0.0/README.md +0 -205
  11. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/LICENSE +0 -0
  12. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/MANIFEST.in +0 -0
  13. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance/database/__init__.py +0 -0
  14. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance/database/context.py +0 -0
  15. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance/fileload/__init__.py +0 -0
  16. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance/fileload/context.py +0 -0
  17. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance/fileload/helpers.py +0 -0
  18. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance/fileload/loadyaml.py +0 -0
  19. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance/util/__init__.py +0 -0
  20. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance/util/functions.py +0 -0
  21. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance.egg-info/SOURCES.txt +0 -0
  22. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance.egg-info/dependency_links.txt +0 -0
  23. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/pyneoinstance.egg-info/top_level.txt +0 -0
  24. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/setup.cfg +0 -0
  25. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/setup.py +0 -0
  26. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/test/load_files.yaml +0 -0
  27. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/test/movies.csv +0 -0
  28. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/test/people.csv +0 -0
  29. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/test/roles.csv +0 -0
  30. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/test/test.py +0 -0
  31. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/test/test_fileload.py +0 -0
  32. {pyneoinstance-1.0.0 → pyneoinstance-1.1.2}/test/test_util_functions.py +0 -0
@@ -1,8 +1,8 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: pyneoinstance
3
- Version: 1.0.0
3
+ Version: 1.1.2
4
4
  Summary: Handle Neo4j read and write transactions
5
- Author-email: Jose A Alvarado-Guzma <jalvarado-guzman@neo4j.com>
5
+ Author-email: "Jose A. Alvarado-Guzman" <jose.alvarado-guzman@neo4j.com>
6
6
  License: GNU GENERAL PUBLIC LICENSE
7
7
  Version 3, 29 June 2007
8
8
 
@@ -685,16 +685,22 @@ Classifier: Development Status :: 4 - Beta
685
685
  Requires-Python: >=3.8
686
686
  Description-Content-Type: text/markdown
687
687
  License-File: LICENSE
688
+ Requires-Dist: neo4j>=5.0.0
689
+ Requires-Dist: pandas
690
+ Requires-Dist: PyYAML
688
691
 
689
692
  # PyNeoInstance Package
690
693
 
691
694
  ## What is it?
692
- Is a Python package that provides a user friendly interface for submitting Cypher queries to Neo4j 4.x.
695
+
696
+ PyNeoInstance is a Python package that provides a user friendly API for submitting Cypher queries to Neo4j.
693
697
  It supports the following features:
694
698
 
695
- - __Read query__: Submit a Cypher query and get the result of it as a Pandas DataFrame.
696
- - __Write query__: Sumbmit a Cypher query to update an existing database.
697
- - __Write query with data__: Update a database based on a Pandas DataFrame and one or more Cypher queries.
699
+ - __Read query__: Submit a Cypher read query, with or without parameters, and get a Pandas Data Frame with the results.
700
+ - __Write query__: Submit a write Cypher query, with or without parameters, to update an existing database.
701
+ - __Write queries__: Submit a list of Cypher queries to update an existing database.
702
+ - __Write query with data__: Update a database based on a Pandas Data Frame and a Cypher query.
703
+ - __Write queries with data__: Update a database based on a Pandas Data Frame and a list of Cypher queries.
698
704
 
699
705
  ## How to install it?
700
706
 
@@ -706,16 +712,16 @@ pip install pyneoinstance
706
712
 
707
713
  ### Instantiating the Neo4jInstance class
708
714
 
709
- In order to use this package you first need to import the Neo4jInstance class.
715
+ In order to use this package, you first need to import the Neo4jInstance class.
710
716
 
711
717
  ```python
712
- from pyneoingest import Neo4jInstance
718
+ from pyneoinstance import Neo4jInstance
713
719
  ```
714
720
 
715
721
  Once imported, you need to instantiate this class by providing the following arguments:
716
722
 
717
723
  - _uri_: The Neo4j instance unique resource identifier.
718
- - _user_: The user name use to authenticate.
724
+ - _user_: The user name used to authenticate.
719
725
  - _password_: The user name password.
720
726
 
721
727
  ```python
@@ -736,7 +742,7 @@ user = os.environ['NEO4J_USER']
736
742
  password = os.environ['NEO4J_PASSWORD']
737
743
  graph = Neo4jInstance(uri,user,password)
738
744
  ```
739
- Another alternative is to create a YAML configutation file to store this infromation.
745
+ Another alternative is to create a YAML configuration file to store this information.
740
746
 
741
747
  ```yaml
742
748
  # configuration.yaml
@@ -744,9 +750,10 @@ db_info:
744
750
  uri: neo4j_uri
745
751
  user: neo4j_user
746
752
  password: user_password
753
+ database: neo4j
747
754
  ```
748
755
 
749
- This YAML file can then be loaded in Python and these information can be retrieve.
756
+ This YAML file can then be loaded in Python and this information can be retrieve.
750
757
 
751
758
  ```python
752
759
  from pyneoinstance import load_yaml_file
@@ -757,7 +764,7 @@ graph = Neo4jInstance(db_info['uri'], db_info['user'], db_info['password'])
757
764
  ```
758
765
  ### Submitting a Cypher read query
759
766
 
760
- Once you instantiated the Neo4jInstance class, you can submit a Cypher read query by using the class method execute\_read\_query. This method receive the Cypher query string as an argument and optionally the name of the database on which the query should be executed. If the database name is not provided the query will be execute using your default Neo4j database. The results of the query is return as a a Pandas DataFrame. To avoid unwanted column names in the resulting DataFrame, always provide aliases for all return properties.
767
+ Once you instantiated the Neo4jInstance class, you can submit a Cypher read query by using the class method execute\_read\_query. This method receives the Cypher query string as an argument and optionally the name of the database on which the query should be executed. If the database name is not provided the query will be execute using your default Neo4j database. The results of the query are return as a Pandas Data Frame. To avoid unwanted column names in the resulting Data Frame, always provide aliases for all returned properties.
761
768
 
762
769
  ```python
763
770
  query = """
@@ -767,35 +774,56 @@ query = """
767
774
  m.tagline AS tagline;
768
775
  """
769
776
 
770
- data = graph.execute_read_query(query)
777
+ data = graph.execute_read_query(query,db_info['database'])
771
778
  ```
779
+
780
+ ### Submitting a Cypher read query with parameters
781
+
782
+ The method execute\_read\_query can optionally received a Python Dictionary with the parameters required as part of you Cypher query.
783
+
784
+ ```python
785
+ query = """
786
+ MATCH (m:Movie {title: $movie_title})
787
+ RETURN m.tile AS title,
788
+ m.released AS releasedYear,
789
+ m.tagline AS tagline;
790
+ """
791
+
792
+ data = graph.execute_read_query(query, db_info['database'], db_info['database'], parameters={'movie_title':'The Matrix'})
793
+ ```
794
+
772
795
  ### Submitting a Cypher write query
773
796
 
774
- We submit a Cypher write query using the method execute\_write\_query. The signature of this method is identical to the execute\_read\_query. This method returns a dictionary containing the updates made to the database.
797
+ You can submit a Cypher write query using the method execute_write_query. The signature of this method is identical to the execute_read_query. This method returns a dictionary containing the updates made to the database. As with read queries you can also specify a Python dictionary containing Cypher parameters.
775
798
 
776
799
  ```python
777
800
  query = """
778
- MERGE(m:Movie {title:'Wakanda Forever'})
801
+ MERGE(m:Movie {title:$movie_title})
779
802
  ON CREATE SET
780
- m.release=2022,
781
- m.tagline = 'Wakanda Forever! Long Live Wakanda! For Honor, For Legacy, For Wakanda!'
803
+ m.release = $year,
804
+ m.tagline = $tagline
782
805
  """
783
806
 
784
- results = graph.execute_write_query(query)
807
+ results = graph.execute_write_query(query, db_info['database'],
808
+ parameters={
809
+ 'movie_title':'Wakanda Forever',
810
+ 'year'=2022,
811
+ tagline=''Wakanda Forever! Long Live Wakanda! For Honor, For Legacy, For Wakanda!''
812
+ })
785
813
  ```
786
814
 
787
815
  ### Updating a database with data in a DataFrame: Sequentially
788
816
 
789
- We can update a Neo4j database with data contained in a Pandas DataFrame using the methods execute\_write\_query\_with\_data or execute\_write\_queries\_with\_data. These methos return a dictionary containing the numbers and types of updates made to the database. Besides taking the query or list of queries as an argument, we can pass to these methods the following arguments:
817
+ We can update a Neo4j database with data contained in a Pandas Data Frame using the methods execute\_write\_query\_with\_data or execute\_write\_queries\_with\_data. These methods return a dictionary containing the numbers and types of updates made to the database. Besides taking the query or list of queries as an argument, we can pass to these methods the following arguments:
790
818
 
791
- - _data_: Pandas DataFrame containing data to process.
792
- - _database_: Optional, database on which to execute the quieries.
793
- - _partitions_: Optional, number of partitions use to break the DataFrame in smaller chucks.
794
- - _parallel_: Wheather to execute the query using multiple Python process.
819
+ - _data_: Pandas Data Frame containing data to process.
820
+ - _database_: Optional, database on which to execute the queries.
821
+ - _partitions_: Optional, number of partitions use to break the Data Frame in smaller chucks.
822
+ - _parallel_: Whether to execute the query using multiple Python process.
795
823
  - _workers_: The number of processes to spawn to execute the queries in parallel.
796
824
  - _parameters_: Extra arguments containing optional cypher parameters.
797
825
 
798
- To illustrates how to use this methods lets imagine we have a csv file that looks like this:
826
+ To illustrates how to use these methods lets imagine we have a csv file that looks like this:
799
827
 
800
828
  ```python
801
829
  # people.csv
@@ -811,11 +839,11 @@ personId,name,birthYear,deathYear
811
839
  3482,Chus Lampreave,1930,
812
840
  ```
813
841
 
814
- We want to update our database by loading this data as a node with the label Person containing the properties id, name, birthYear and deathYear. To accomplish this, we need to performe the following steps:
842
+ We want to update our database by loading this data as a node with the label Person containing the properties id, name, birthYear and deathYear. To accomplish this, we need to performed the following steps:
815
843
 
816
- 1. Load the csv to Python as a DataFrame.
817
- 2. Develop the Cypher query that is going to be use to load the DataFrame into Neo4j.
818
- 3. Call the execute\_write\_query\_with\_data of an existing instance of the Neo4jInstance class passing the DataFrame, Cypher query and any other optinal arguments to this method.
844
+ 1. Load the csv to Python as a Data Frame.
845
+ 2. Develop the Cypher query that is going to be use to load the Data Frame into Neo4j.
846
+ 3. Call the execute\_write\_query\_with\_data of an existing instance of the Neo4jInstance class passing the Data Frame, Cypher query and any other optional arguments to this method.
819
847
 
820
848
  ```python
821
849
  import pandas as pd
@@ -833,10 +861,10 @@ person_load_query = """
833
861
  p.deathYear = toInteger(row.deathYear)
834
862
  """
835
863
 
836
- person_load_result = graph.execute_write_query_with_data(person_data, person_load_query)
864
+ person_load_result = graph.execute_write_query_with_data(person_load_query, person_data, db_info['database'])
837
865
  ```
838
866
 
839
- Is not a good practice to store your Cypher queries as strings within your Python modules. To solve this we can extend our YAML configuration file to store all our Cypher queries.
867
+ It is not a good practice to store your Cypher queries as strings within your Python modules. To solve this, we can extend our YAML configuration file to store all our Cypher queries.
840
868
 
841
869
  ```yaml
842
870
  # configuration.yaml
@@ -844,9 +872,10 @@ db_info:
844
872
  uri: neo4j_uri
845
873
  user: neo4j_user
846
874
  password: user_password
875
+ database: neo4j
847
876
  loading_queries:
848
877
  nodes:
849
- person: |
878
+ Person: |
850
879
  WITH $rows AS rows
851
880
  UNWIND rows AS row
852
881
  MERGE(p:Person {id:row.personId})
@@ -866,28 +895,33 @@ person_data = pd.read_csv('people.csv')
866
895
 
867
896
  node_load_queries = config['loading_queries']['nodes']
868
897
 
869
- person_load_result = graph.execute_write_query_with_data(person_data, node_load_queries['person'])
898
+ person_load_result = graph.execute_write_query_with_data(node_load_queries['Person'], person_data, db_info['database'])
870
899
  ```
871
900
 
872
- If the number of rows in your DataFrame is 100K or more it is reccomended to break the DataFrame in smaller chunks so that they can be process in different transactions. This will avoid running out of heap error. To do this, just indicate in the _partitions_ argument how many partitions would you like to break this DataFrame into. This partitions will be loaded in different sequential transactions unless you specified loading them in parallel by setting the _parallel_ argument to true. We will discuss this process in the next section.
901
+ If the number of rows in your Data Frame is 100K or more it is recommended to break the Data Frame in smaller chunks so that they can be process in different transactions. This will avoid running out of heap error. To do this, just indicate in the _partitions_ argument how many partitions would you like to break this Data Frame into. These partitions will be loaded in different sequential transactions unless you specified loading them in parallel by setting the _parallel_ argument to true. We will discuss this process in the next section.
873
902
 
874
903
  ```python
875
904
  person_load_result = graph.execute_write_query_with_data(
876
- person_data,
877
905
  node_load_queries['person'],
906
+ person_data,
907
+ db_info['database'],
878
908
  partitions = 5
879
909
  )
880
910
  ```
881
- ## Updating a database with data in a DataFrame: Parallel
911
+ ## Updating a database with data in a Data Frame: Parallel
882
912
 
883
- To load the data in the DataFrame in parallel, we need to partition the DataFrame by incrementing the _partitions_ argument and set the _parallel_ argument to true. By default this will use all the available virtual CPUs in the machine. If you need to use less, you can specified how many to use by setting the _workers_ argument.
913
+ To load the data in the Data Frame in parallel, we need to partition the Data Frame by incrementing the _partitions_ argument and set the _parallel_ argument to true. By default, this will use all the available virtual CPUs in the machine. If you need to use less, you can specify how many to use by setting the _workers_ argument.
884
914
 
885
915
  ```python
886
916
  person_load_result = graph.execute_write_query_with_data(
887
- person_data,
888
917
  node_load_queries['person'],
918
+ person_data,
889
919
  partitions = 5,
890
920
  parallel = True,
891
921
  workers = 5
892
922
  )
893
923
  ```
924
+
925
+ ## More detailed examples
926
+
927
+ For more detailed examples please referred to this [article](https://medium.com/@a-gilmore/pyneoinstance-a-user-friendly-python-library-for-neo4j-dbefa3117bb2).
@@ -0,0 +1,236 @@
1
+ # PyNeoInstance Package
2
+
3
+ ## What is it?
4
+
5
+ PyNeoInstance is a Python package that provides a user friendly API for submitting Cypher queries to Neo4j.
6
+ It supports the following features:
7
+
8
+ - __Read query__: Submit a Cypher read query, with or without parameters, and get a Pandas Data Frame with the results.
9
+ - __Write query__: Submit a write Cypher query, with or without parameters, to update an existing database.
10
+ - __Write queries__: Submit a list of Cypher queries to update an existing database.
11
+ - __Write query with data__: Update a database based on a Pandas Data Frame and a Cypher query.
12
+ - __Write queries with data__: Update a database based on a Pandas Data Frame and a list of Cypher queries.
13
+
14
+ ## How to install it?
15
+
16
+ ```python
17
+ pip install pyneoinstance
18
+ ```
19
+
20
+ ## How to use it?
21
+
22
+ ### Instantiating the Neo4jInstance class
23
+
24
+ In order to use this package, you first need to import the Neo4jInstance class.
25
+
26
+ ```python
27
+ from pyneoinstance import Neo4jInstance
28
+ ```
29
+
30
+ Once imported, you need to instantiate this class by providing the following arguments:
31
+
32
+ - _uri_: The Neo4j instance unique resource identifier.
33
+ - _user_: The user name used to authenticate.
34
+ - _password_: The user name password.
35
+
36
+ ```python
37
+ uri = 'bolt://localhost:7687'
38
+ user = 'neo4j'
39
+ password = 'password'
40
+
41
+ graph = Neo4jInstance(uri, user, password)
42
+ ```
43
+
44
+ Although the previous code is valid, it is not recommended to hard code your user credentials in your application. As an alternative you can store this information as part of your OS user environment variables and then retrieve them in Python.
45
+
46
+ ```python
47
+ import os
48
+
49
+ uri = os.environ['NEO4J_URI']
50
+ user = os.environ['NEO4J_USER']
51
+ password = os.environ['NEO4J_PASSWORD']
52
+ graph = Neo4jInstance(uri,user,password)
53
+ ```
54
+ Another alternative is to create a YAML configuration file to store this information.
55
+
56
+ ```yaml
57
+ # configuration.yaml
58
+ db_info:
59
+ uri: neo4j_uri
60
+ user: neo4j_user
61
+ password: user_password
62
+ database: neo4j
63
+ ```
64
+
65
+ This YAML file can then be loaded in Python and this information can be retrieve.
66
+
67
+ ```python
68
+ from pyneoinstance import load_yaml_file
69
+
70
+ config = load_yaml_file(configuration.yaml)
71
+ db_info = config['db_info']
72
+ graph = Neo4jInstance(db_info['uri'], db_info['user'], db_info['password'])
73
+ ```
74
+ ### Submitting a Cypher read query
75
+
76
+ Once you instantiated the Neo4jInstance class, you can submit a Cypher read query by using the class method execute\_read\_query. This method receives the Cypher query string as an argument and optionally the name of the database on which the query should be executed. If the database name is not provided the query will be execute using your default Neo4j database. The results of the query are return as a Pandas Data Frame. To avoid unwanted column names in the resulting Data Frame, always provide aliases for all returned properties.
77
+
78
+ ```python
79
+ query = """
80
+ MATCH (m:Movie {title: "The Matrix"})
81
+ RETURN m.tile AS title,
82
+ m.released AS releasedYear,
83
+ m.tagline AS tagline;
84
+ """
85
+
86
+ data = graph.execute_read_query(query,db_info['database'])
87
+ ```
88
+
89
+ ### Submitting a Cypher read query with parameters
90
+
91
+ The method execute\_read\_query can optionally received a Python Dictionary with the parameters required as part of you Cypher query.
92
+
93
+ ```python
94
+ query = """
95
+ MATCH (m:Movie {title: $movie_title})
96
+ RETURN m.tile AS title,
97
+ m.released AS releasedYear,
98
+ m.tagline AS tagline;
99
+ """
100
+
101
+ data = graph.execute_read_query(query, db_info['database'], db_info['database'], parameters={'movie_title':'The Matrix'})
102
+ ```
103
+
104
+ ### Submitting a Cypher write query
105
+
106
+ You can submit a Cypher write query using the method execute_write_query. The signature of this method is identical to the execute_read_query. This method returns a dictionary containing the updates made to the database. As with read queries you can also specify a Python dictionary containing Cypher parameters.
107
+
108
+ ```python
109
+ query = """
110
+ MERGE(m:Movie {title:$movie_title})
111
+ ON CREATE SET
112
+ m.release = $year,
113
+ m.tagline = $tagline
114
+ """
115
+
116
+ results = graph.execute_write_query(query, db_info['database'],
117
+ parameters={
118
+ 'movie_title':'Wakanda Forever',
119
+ 'year'=2022,
120
+ tagline=''Wakanda Forever! Long Live Wakanda! For Honor, For Legacy, For Wakanda!''
121
+ })
122
+ ```
123
+
124
+ ### Updating a database with data in a DataFrame: Sequentially
125
+
126
+ We can update a Neo4j database with data contained in a Pandas Data Frame using the methods execute\_write\_query\_with\_data or execute\_write\_queries\_with\_data. These methods return a dictionary containing the numbers and types of updates made to the database. Besides taking the query or list of queries as an argument, we can pass to these methods the following arguments:
127
+
128
+ - _data_: Pandas Data Frame containing data to process.
129
+ - _database_: Optional, database on which to execute the queries.
130
+ - _partitions_: Optional, number of partitions use to break the Data Frame in smaller chucks.
131
+ - _parallel_: Whether to execute the query using multiple Python process.
132
+ - _workers_: The number of processes to spawn to execute the queries in parallel.
133
+ - _parameters_: Extra arguments containing optional cypher parameters.
134
+
135
+ To illustrates how to use these methods lets imagine we have a csv file that looks like this:
136
+
137
+ ```python
138
+ # people.csv
139
+ personId,name,birthYear,deathYear
140
+ 23945,Gérard Pirès,1942,
141
+ 553509,Helen Reddy,1941,
142
+ 113934,Susan Flannery,1939,
143
+ 26706,David Rintoul,1948,
144
+ 237537,Rita Marley,1946,
145
+ 11502,Harry Davenport,1866,1949
146
+ 11792,George Dickerson,1933,
147
+ 7242,Temuera Morrison,1960,
148
+ 3482,Chus Lampreave,1930,
149
+ ```
150
+
151
+ We want to update our database by loading this data as a node with the label Person containing the properties id, name, birthYear and deathYear. To accomplish this, we need to performed the following steps:
152
+
153
+ 1. Load the csv to Python as a Data Frame.
154
+ 2. Develop the Cypher query that is going to be use to load the Data Frame into Neo4j.
155
+ 3. Call the execute\_write\_query\_with\_data of an existing instance of the Neo4jInstance class passing the Data Frame, Cypher query and any other optional arguments to this method.
156
+
157
+ ```python
158
+ import pandas as pd
159
+
160
+ person_data = pd.read_csv('people.csv')
161
+
162
+ person_load_query = """
163
+ WITH $rows AS rows
164
+ UNWIND rows AS row
165
+ MERGE(p:Person {id:row.personId})
166
+ ON CREATE
167
+ SET p.id = row.personId,
168
+ p.name = row.name,
169
+ p.birthYear=toInteger(row.birthYear),
170
+ p.deathYear = toInteger(row.deathYear)
171
+ """
172
+
173
+ person_load_result = graph.execute_write_query_with_data(person_load_query, person_data, db_info['database'])
174
+ ```
175
+
176
+ It is not a good practice to store your Cypher queries as strings within your Python modules. To solve this, we can extend our YAML configuration file to store all our Cypher queries.
177
+
178
+ ```yaml
179
+ # configuration.yaml
180
+ db_info:
181
+ uri: neo4j_uri
182
+ user: neo4j_user
183
+ password: user_password
184
+ database: neo4j
185
+ loading_queries:
186
+ nodes:
187
+ Person: |
188
+ WITH $rows AS rows
189
+ UNWIND rows AS row
190
+ MERGE(p:Person {id:row.personId})
191
+ ON CREATE
192
+ SET p.id = row.personId,
193
+ p.name = row.name,
194
+ p.birthYear=toInteger(row.birthYear),
195
+ p.deathYear = toInteger(row.deathYear)
196
+ ```
197
+
198
+ Now that we added the loading query to our configuration file, lets re-write our Python script to get this query from the configuration file.
199
+
200
+ ```python
201
+ import pandas as pd
202
+
203
+ person_data = pd.read_csv('people.csv')
204
+
205
+ node_load_queries = config['loading_queries']['nodes']
206
+
207
+ person_load_result = graph.execute_write_query_with_data(node_load_queries['Person'], person_data, db_info['database'])
208
+ ```
209
+
210
+ If the number of rows in your Data Frame is 100K or more it is recommended to break the Data Frame in smaller chunks so that they can be process in different transactions. This will avoid running out of heap error. To do this, just indicate in the _partitions_ argument how many partitions would you like to break this Data Frame into. These partitions will be loaded in different sequential transactions unless you specified loading them in parallel by setting the _parallel_ argument to true. We will discuss this process in the next section.
211
+
212
+ ```python
213
+ person_load_result = graph.execute_write_query_with_data(
214
+ node_load_queries['person'],
215
+ person_data,
216
+ db_info['database'],
217
+ partitions = 5
218
+ )
219
+ ```
220
+ ## Updating a database with data in a Data Frame: Parallel
221
+
222
+ To load the data in the Data Frame in parallel, we need to partition the Data Frame by incrementing the _partitions_ argument and set the _parallel_ argument to true. By default, this will use all the available virtual CPUs in the machine. If you need to use less, you can specify how many to use by setting the _workers_ argument.
223
+
224
+ ```python
225
+ person_load_result = graph.execute_write_query_with_data(
226
+ node_load_queries['person'],
227
+ person_data,
228
+ partitions = 5,
229
+ parallel = True,
230
+ workers = 5
231
+ )
232
+ ```
233
+
234
+ ## More detailed examples
235
+
236
+ For more detailed examples please referred to this [article](https://medium.com/@a-gilmore/pyneoinstance-a-user-friendly-python-library-for-neo4j-dbefa3117bb2).
@@ -1,4 +1,4 @@
1
1
  from .database.neo4jdbms import Neo4jInstance
2
2
  from .fileload.loadyaml import load_yaml_file
3
3
 
4
- __version__ = '1.0.0'
4
+ __version__ = '1.1.2'
@@ -20,7 +20,6 @@ from neo4j.exceptions import ServiceUnavailable
20
20
  from neo4j.exceptions import AuthError
21
21
  from neo4j.exceptions import ClientError
22
22
  from neo4j.exceptions import ConfigurationError
23
- from .context import get_logger
24
23
 
25
24
  def _get_driver(neo_info : Dict[str, str]):
26
25
  try:
@@ -34,7 +33,8 @@ def _get_driver(neo_info : Dict[str, str]):
34
33
  neo_info['uri'],
35
34
  auth=(neo_info['user'], neo_info['password']))
36
35
  except ServiceUnavailable as exception:
37
- raise ServiceUnavailable() from exception
36
+ raise ServiceUnavailable()
37
+ sys.exit()
38
38
  except AuthError as exception:
39
39
  raise AuthError() from exception
40
40
  except ConfigurationError as exception:
@@ -67,14 +67,14 @@ def _execute_write(session, query,
67
67
  params = {}
68
68
  try:
69
69
  if rows:
70
- results = session.write_transaction(
70
+ results = session.execute_write(
71
71
  _write_transaction_function, query,
72
72
  rows = rows, **params).counters.__dict__
73
73
  else:
74
- results = session.write_transaction(
74
+ results = session.execute_write(
75
75
  _write_transaction_function, query,
76
76
  **params).counters.__dict__
77
- except ServiceUnavailable() as exception:
77
+ except ServiceUnavailable as exception:
78
78
  raise ServiceUnavailable() from exception
79
79
  except ClientError as exception:
80
80
  raise ClientError() from exception
@@ -92,14 +92,14 @@ def _execute_write_parallel(neo_info, database, query,
92
92
  with _get_session(driver, database) as session:
93
93
  try:
94
94
  if rows:
95
- results = session.write_transaction(
95
+ results = session.execute_write(
96
96
  _write_transaction_function, query,
97
97
  rows = rows, **params).counters.__dict__
98
98
  else:
99
- results = session.write_transaction(
99
+ results = session.execute_write(
100
100
  _write_transaction_function, query,
101
101
  **params).counters.__dict__
102
- except ServiceUnavailable() as exception:
102
+ except ServiceUnavailable as exception:
103
103
  raise ServiceUnavailable() from exception
104
104
  except ClientError as exception:
105
105
  raise ClientError() from exception
@@ -167,12 +167,11 @@ class Neo4jInstance:
167
167
  self.neo_info['password'] = password
168
168
  self.neo_info['encrypted'] = kwargs.get('encrypted') or ''
169
169
  self.__results = defaultdict(int)
170
- self.__logger = get_logger('pyneoingest', logging.INFO)
171
170
 
172
171
 
173
172
  def execute_read_query(self, query: str,
174
173
  database: Optional[str] = None,
175
- **kwargs: Optional[Dict[str, Any]]
174
+ parameters: Optional[Dict[str, Any]] = None
176
175
  ) -> DataFrame:
177
176
  """Execute a read transaction to a specific database.
178
177
 
@@ -184,7 +183,7 @@ class Neo4jInstance:
184
183
  Name of the Neo4j database of which to execute the
185
184
  transaction. If not provided the default database is going to
186
185
  be use.
187
- kwargs : Dict[str, Any], optional
186
+ parameters: Dict[str, Any], optional
188
187
  Extra arguments containing optional cypher parameters.
189
188
 
190
189
  Return
@@ -199,11 +198,15 @@ class Neo4jInstance:
199
198
  ClientError
200
199
  When there is a Cypher syntax error or datatype error.
201
200
  """
201
+ if parameters:
202
+ params = parameters
203
+ else:
204
+ params = {}
202
205
  with _get_driver(self.neo_info) as driver:
203
206
  with _get_session(driver, database) as session:
204
207
  try:
205
- result = session.read_transaction(
206
- _read_transaction_function,query=query,**kwargs)
208
+ result = session.execute_read(
209
+ _read_transaction_function,query=query,**params)
207
210
  except ServiceUnavailable as exception:
208
211
  raise ServiceUnavailable() from exception
209
212
  except ClientError as exception:
@@ -212,7 +215,7 @@ class Neo4jInstance:
212
215
 
213
216
  def execute_write_queries(self, queries: List[str],
214
217
  database: Optional[str] = None,
215
- **kwargs: Optional[Dict[str, Any]]
218
+ parameters: Optional[Dict[str, Any]] = None
216
219
  ) -> Dict[str, int]:
217
220
  """Execute a write query to a specific database.
218
221
 
@@ -223,7 +226,7 @@ class Neo4jInstance:
223
226
  database : str, optional
224
227
  Name of the Neo4j database of which to execute the transaction.
225
228
  If not provided the default database is going to be use.
226
- kwargs : Dict[str, Any], optional
229
+ parameters : Dict[str, Any], optional
227
230
  Extra arguments containing optional cypher parameters.
228
231
 
229
232
  Returns
@@ -239,20 +242,23 @@ class Neo4jInstance:
239
242
  ClientError
240
243
  When their is a Cypher syntax error or datatype error.
241
244
  """
245
+ if parameters:
246
+ params = parameters
247
+ else:
248
+ params = {}
242
249
  results = defaultdict(int)
243
250
  with _get_driver(self.neo_info) as driver:
244
251
  with _get_session(driver, database) as session:
245
252
  for query in queries:
246
- result = _execute_write(session, query, **kwargs)
253
+ result = _execute_write(session, query, parameters=params)
247
254
  for key, value in result.items():
248
255
  if key != '_contains_updates':
249
256
  results[key] += value
250
- self.__logger.info(f'Loading stats: {dict(results)}')
251
257
  return dict(results)
252
258
 
253
259
  def execute_write_query(self, query: str,
254
260
  database: Optional[str] = None,
255
- **kwargs: Optional[Dict['str',Any]]
261
+ parameters: Optional[Dict['str',Any]] = None
256
262
  ) -> Dict[str, Any]:
257
263
  """Execute a write query to a specific database.
258
264
 
@@ -263,7 +269,7 @@ class Neo4jInstance:
263
269
  database : str, optional
264
270
  Name of the Neo4j database of which to execute the transaction.
265
271
  If not provided the default database is going to be use.
266
- kwargs : Dict[str, Any], optional
272
+ parameters : Dict[str, Any], optional
267
273
  Extra arguments containing optional cypher parameters.
268
274
 
269
275
  Returns
@@ -279,7 +285,7 @@ class Neo4jInstance:
279
285
  ClientError
280
286
  When their is a Cypher syntax error or datatype error.
281
287
  """
282
- result = self.execute_write_queries([query], database, **kwargs)
288
+ result = self.execute_write_queries([query], database, parameters)
283
289
  return result
284
290
 
285
291
  def execute_write_queries_with_data(self, queries: List[str], data: DataFrame,
@@ -362,7 +368,6 @@ class Neo4jInstance:
362
368
  self.__results[key] += value
363
369
  results = dict(self.__results.copy())
364
370
  self.__results.clear()
365
- self.__logger.info(f'Loading stats: {results}')
366
371
  return results
367
372
 
368
373
  def execute_write_query_with_data(self,
@@ -1,8 +1,8 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: pyneoinstance
3
- Version: 1.0.0
3
+ Version: 1.1.2
4
4
  Summary: Handle Neo4j read and write transactions
5
- Author-email: Jose A Alvarado-Guzma <jalvarado-guzman@neo4j.com>
5
+ Author-email: "Jose A. Alvarado-Guzman" <jose.alvarado-guzman@neo4j.com>
6
6
  License: GNU GENERAL PUBLIC LICENSE
7
7
  Version 3, 29 June 2007
8
8
 
@@ -685,16 +685,22 @@ Classifier: Development Status :: 4 - Beta
685
685
  Requires-Python: >=3.8
686
686
  Description-Content-Type: text/markdown
687
687
  License-File: LICENSE
688
+ Requires-Dist: neo4j>=5.0.0
689
+ Requires-Dist: pandas
690
+ Requires-Dist: PyYAML
688
691
 
689
692
  # PyNeoInstance Package
690
693
 
691
694
  ## What is it?
692
- Is a Python package that provides a user friendly interface for submitting Cypher queries to Neo4j 4.x.
695
+
696
+ PyNeoInstance is a Python package that provides a user friendly API for submitting Cypher queries to Neo4j.
693
697
  It supports the following features:
694
698
 
695
- - __Read query__: Submit a Cypher query and get the result of it as a Pandas DataFrame.
696
- - __Write query__: Sumbmit a Cypher query to update an existing database.
697
- - __Write query with data__: Update a database based on a Pandas DataFrame and one or more Cypher queries.
699
+ - __Read query__: Submit a Cypher read query, with or without parameters, and get a Pandas Data Frame with the results.
700
+ - __Write query__: Submit a write Cypher query, with or without parameters, to update an existing database.
701
+ - __Write queries__: Submit a list of Cypher queries to update an existing database.
702
+ - __Write query with data__: Update a database based on a Pandas Data Frame and a Cypher query.
703
+ - __Write queries with data__: Update a database based on a Pandas Data Frame and a list of Cypher queries.
698
704
 
699
705
  ## How to install it?
700
706
 
@@ -706,16 +712,16 @@ pip install pyneoinstance
706
712
 
707
713
  ### Instantiating the Neo4jInstance class
708
714
 
709
- In order to use this package you first need to import the Neo4jInstance class.
715
+ In order to use this package, you first need to import the Neo4jInstance class.
710
716
 
711
717
  ```python
712
- from pyneoingest import Neo4jInstance
718
+ from pyneoinstance import Neo4jInstance
713
719
  ```
714
720
 
715
721
  Once imported, you need to instantiate this class by providing the following arguments:
716
722
 
717
723
  - _uri_: The Neo4j instance unique resource identifier.
718
- - _user_: The user name use to authenticate.
724
+ - _user_: The user name used to authenticate.
719
725
  - _password_: The user name password.
720
726
 
721
727
  ```python
@@ -736,7 +742,7 @@ user = os.environ['NEO4J_USER']
736
742
  password = os.environ['NEO4J_PASSWORD']
737
743
  graph = Neo4jInstance(uri,user,password)
738
744
  ```
739
- Another alternative is to create a YAML configutation file to store this infromation.
745
+ Another alternative is to create a YAML configuration file to store this information.
740
746
 
741
747
  ```yaml
742
748
  # configuration.yaml
@@ -744,9 +750,10 @@ db_info:
744
750
  uri: neo4j_uri
745
751
  user: neo4j_user
746
752
  password: user_password
753
+ database: neo4j
747
754
  ```
748
755
 
749
- This YAML file can then be loaded in Python and these information can be retrieve.
756
+ This YAML file can then be loaded in Python and this information can be retrieve.
750
757
 
751
758
  ```python
752
759
  from pyneoinstance import load_yaml_file
@@ -757,7 +764,7 @@ graph = Neo4jInstance(db_info['uri'], db_info['user'], db_info['password'])
757
764
  ```
758
765
  ### Submitting a Cypher read query
759
766
 
760
- Once you instantiated the Neo4jInstance class, you can submit a Cypher read query by using the class method execute\_read\_query. This method receive the Cypher query string as an argument and optionally the name of the database on which the query should be executed. If the database name is not provided the query will be execute using your default Neo4j database. The results of the query is return as a a Pandas DataFrame. To avoid unwanted column names in the resulting DataFrame, always provide aliases for all return properties.
767
+ Once you instantiated the Neo4jInstance class, you can submit a Cypher read query by using the class method execute\_read\_query. This method receives the Cypher query string as an argument and optionally the name of the database on which the query should be executed. If the database name is not provided the query will be execute using your default Neo4j database. The results of the query are return as a Pandas Data Frame. To avoid unwanted column names in the resulting Data Frame, always provide aliases for all returned properties.
761
768
 
762
769
  ```python
763
770
  query = """
@@ -767,35 +774,56 @@ query = """
767
774
  m.tagline AS tagline;
768
775
  """
769
776
 
770
- data = graph.execute_read_query(query)
777
+ data = graph.execute_read_query(query,db_info['database'])
771
778
  ```
779
+
780
+ ### Submitting a Cypher read query with parameters
781
+
782
+ The method execute\_read\_query can optionally received a Python Dictionary with the parameters required as part of you Cypher query.
783
+
784
+ ```python
785
+ query = """
786
+ MATCH (m:Movie {title: $movie_title})
787
+ RETURN m.tile AS title,
788
+ m.released AS releasedYear,
789
+ m.tagline AS tagline;
790
+ """
791
+
792
+ data = graph.execute_read_query(query, db_info['database'], db_info['database'], parameters={'movie_title':'The Matrix'})
793
+ ```
794
+
772
795
  ### Submitting a Cypher write query
773
796
 
774
- We submit a Cypher write query using the method execute\_write\_query. The signature of this method is identical to the execute\_read\_query. This method returns a dictionary containing the updates made to the database.
797
+ You can submit a Cypher write query using the method execute_write_query. The signature of this method is identical to the execute_read_query. This method returns a dictionary containing the updates made to the database. As with read queries you can also specify a Python dictionary containing Cypher parameters.
775
798
 
776
799
  ```python
777
800
  query = """
778
- MERGE(m:Movie {title:'Wakanda Forever'})
801
+ MERGE(m:Movie {title:$movie_title})
779
802
  ON CREATE SET
780
- m.release=2022,
781
- m.tagline = 'Wakanda Forever! Long Live Wakanda! For Honor, For Legacy, For Wakanda!'
803
+ m.release = $year,
804
+ m.tagline = $tagline
782
805
  """
783
806
 
784
- results = graph.execute_write_query(query)
807
+ results = graph.execute_write_query(query, db_info['database'],
808
+ parameters={
809
+ 'movie_title':'Wakanda Forever',
810
+ 'year'=2022,
811
+ tagline=''Wakanda Forever! Long Live Wakanda! For Honor, For Legacy, For Wakanda!''
812
+ })
785
813
  ```
786
814
 
787
815
  ### Updating a database with data in a DataFrame: Sequentially
788
816
 
789
- We can update a Neo4j database with data contained in a Pandas DataFrame using the methods execute\_write\_query\_with\_data or execute\_write\_queries\_with\_data. These methos return a dictionary containing the numbers and types of updates made to the database. Besides taking the query or list of queries as an argument, we can pass to these methods the following arguments:
817
+ We can update a Neo4j database with data contained in a Pandas Data Frame using the methods execute\_write\_query\_with\_data or execute\_write\_queries\_with\_data. These methods return a dictionary containing the numbers and types of updates made to the database. Besides taking the query or list of queries as an argument, we can pass to these methods the following arguments:
790
818
 
791
- - _data_: Pandas DataFrame containing data to process.
792
- - _database_: Optional, database on which to execute the quieries.
793
- - _partitions_: Optional, number of partitions use to break the DataFrame in smaller chucks.
794
- - _parallel_: Wheather to execute the query using multiple Python process.
819
+ - _data_: Pandas Data Frame containing data to process.
820
+ - _database_: Optional, database on which to execute the queries.
821
+ - _partitions_: Optional, number of partitions use to break the Data Frame in smaller chucks.
822
+ - _parallel_: Whether to execute the query using multiple Python process.
795
823
  - _workers_: The number of processes to spawn to execute the queries in parallel.
796
824
  - _parameters_: Extra arguments containing optional cypher parameters.
797
825
 
798
- To illustrates how to use this methods lets imagine we have a csv file that looks like this:
826
+ To illustrates how to use these methods lets imagine we have a csv file that looks like this:
799
827
 
800
828
  ```python
801
829
  # people.csv
@@ -811,11 +839,11 @@ personId,name,birthYear,deathYear
811
839
  3482,Chus Lampreave,1930,
812
840
  ```
813
841
 
814
- We want to update our database by loading this data as a node with the label Person containing the properties id, name, birthYear and deathYear. To accomplish this, we need to performe the following steps:
842
+ We want to update our database by loading this data as a node with the label Person containing the properties id, name, birthYear and deathYear. To accomplish this, we need to performed the following steps:
815
843
 
816
- 1. Load the csv to Python as a DataFrame.
817
- 2. Develop the Cypher query that is going to be use to load the DataFrame into Neo4j.
818
- 3. Call the execute\_write\_query\_with\_data of an existing instance of the Neo4jInstance class passing the DataFrame, Cypher query and any other optinal arguments to this method.
844
+ 1. Load the csv to Python as a Data Frame.
845
+ 2. Develop the Cypher query that is going to be use to load the Data Frame into Neo4j.
846
+ 3. Call the execute\_write\_query\_with\_data of an existing instance of the Neo4jInstance class passing the Data Frame, Cypher query and any other optional arguments to this method.
819
847
 
820
848
  ```python
821
849
  import pandas as pd
@@ -833,10 +861,10 @@ person_load_query = """
833
861
  p.deathYear = toInteger(row.deathYear)
834
862
  """
835
863
 
836
- person_load_result = graph.execute_write_query_with_data(person_data, person_load_query)
864
+ person_load_result = graph.execute_write_query_with_data(person_load_query, person_data, db_info['database'])
837
865
  ```
838
866
 
839
- Is not a good practice to store your Cypher queries as strings within your Python modules. To solve this we can extend our YAML configuration file to store all our Cypher queries.
867
+ It is not a good practice to store your Cypher queries as strings within your Python modules. To solve this, we can extend our YAML configuration file to store all our Cypher queries.
840
868
 
841
869
  ```yaml
842
870
  # configuration.yaml
@@ -844,9 +872,10 @@ db_info:
844
872
  uri: neo4j_uri
845
873
  user: neo4j_user
846
874
  password: user_password
875
+ database: neo4j
847
876
  loading_queries:
848
877
  nodes:
849
- person: |
878
+ Person: |
850
879
  WITH $rows AS rows
851
880
  UNWIND rows AS row
852
881
  MERGE(p:Person {id:row.personId})
@@ -866,28 +895,33 @@ person_data = pd.read_csv('people.csv')
866
895
 
867
896
  node_load_queries = config['loading_queries']['nodes']
868
897
 
869
- person_load_result = graph.execute_write_query_with_data(person_data, node_load_queries['person'])
898
+ person_load_result = graph.execute_write_query_with_data(node_load_queries['Person'], person_data, db_info['database'])
870
899
  ```
871
900
 
872
- If the number of rows in your DataFrame is 100K or more it is reccomended to break the DataFrame in smaller chunks so that they can be process in different transactions. This will avoid running out of heap error. To do this, just indicate in the _partitions_ argument how many partitions would you like to break this DataFrame into. This partitions will be loaded in different sequential transactions unless you specified loading them in parallel by setting the _parallel_ argument to true. We will discuss this process in the next section.
901
+ If the number of rows in your Data Frame is 100K or more it is recommended to break the Data Frame in smaller chunks so that they can be process in different transactions. This will avoid running out of heap error. To do this, just indicate in the _partitions_ argument how many partitions would you like to break this Data Frame into. These partitions will be loaded in different sequential transactions unless you specified loading them in parallel by setting the _parallel_ argument to true. We will discuss this process in the next section.
873
902
 
874
903
  ```python
875
904
  person_load_result = graph.execute_write_query_with_data(
876
- person_data,
877
905
  node_load_queries['person'],
906
+ person_data,
907
+ db_info['database'],
878
908
  partitions = 5
879
909
  )
880
910
  ```
881
- ## Updating a database with data in a DataFrame: Parallel
911
+ ## Updating a database with data in a Data Frame: Parallel
882
912
 
883
- To load the data in the DataFrame in parallel, we need to partition the DataFrame by incrementing the _partitions_ argument and set the _parallel_ argument to true. By default this will use all the available virtual CPUs in the machine. If you need to use less, you can specified how many to use by setting the _workers_ argument.
913
+ To load the data in the Data Frame in parallel, we need to partition the Data Frame by incrementing the _partitions_ argument and set the _parallel_ argument to true. By default, this will use all the available virtual CPUs in the machine. If you need to use less, you can specify how many to use by setting the _workers_ argument.
884
914
 
885
915
  ```python
886
916
  person_load_result = graph.execute_write_query_with_data(
887
- person_data,
888
917
  node_load_queries['person'],
918
+ person_data,
889
919
  partitions = 5,
890
920
  parallel = True,
891
921
  workers = 5
892
922
  )
893
923
  ```
924
+
925
+ ## More detailed examples
926
+
927
+ For more detailed examples please referred to this [article](https://medium.com/@a-gilmore/pyneoinstance-a-user-friendly-python-library-for-neo4j-dbefa3117bb2).
@@ -1,3 +1,3 @@
1
- neo4j<=4.4.0
1
+ neo4j>=5.0.0
2
2
  pandas
3
3
  PyYAML
@@ -6,10 +6,10 @@ build-backend = "setuptools.build_meta"
6
6
 
7
7
  [project]
8
8
  name = "pyneoinstance"
9
- version = "1.0.0"
9
+ version = "1.1.2"
10
10
  description = "Handle Neo4j read and write transactions"
11
11
  readme = "README.md"
12
- authors = [{ name = "Jose A Alvarado-Guzma",email = "jalvarado-guzman@neo4j.com" }]
12
+ authors = [{ name = "Jose A. Alvarado-Guzman",email = "jose.alvarado-guzman@neo4j.com" }]
13
13
  license = { file = "LICENSE" }
14
14
  classifiers = [
15
15
  "License :: OSI Approved :: GNU General Public License (GPL)",
@@ -18,7 +18,7 @@ classifiers = [
18
18
  ]
19
19
  keywords = ["neo4j", "graph", "cypher"]
20
20
  dependencies = [
21
- "neo4j<=4.4.0",
21
+ "neo4j>=5.0.0",
22
22
  "pandas",
23
23
  "PyYAML"
24
24
  ]
@@ -1,11 +1,10 @@
1
1
  database: test
2
2
  pre_ingest:
3
- - CREATE CONSTRAINT nkPerson IF NOT EXISTS ON(n:Person) ASSERT (n.personId) IS NODE KEY;
4
- - CREATE CONSTRAINT nkMovie IF NOT EXISTS ON(n:Movie) ASSERT (n.movieId) IS NODE KEY;
3
+ - CREATE CONSTRAINT nkPerson IF NOT EXISTS FOR(n:Person) REQUIRE (n.personId) IS NODE KEY;
4
+ - CREATE CONSTRAINT nkMovie IF NOT EXISTS FOR(n:Movie) REQUIRE (n.movieId) IS NODE KEY;
5
5
 
6
6
  queries:
7
- load_person:
8
- |
7
+ load_person: |
9
8
  WITH $rows AS rows
10
9
  UNWIND rows AS row
11
10
  MERGE(p:Person {personId:row.personId})
@@ -13,8 +12,7 @@ queries:
13
12
  SET p.personId=row.personId,
14
13
  p.name=row.name,
15
14
  p.birthYear=toInteger(row.birthYear)
16
- load_movie:
17
- |
15
+ load_movie: |
18
16
  WITH $rows AS rows
19
17
  UNWIND rows AS row
20
18
  MERGE(m:Movie {movieId:row.movieId})
@@ -24,11 +22,16 @@ queries:
24
22
  m.averageVote=toFloat(row.avgVote),
25
23
  m.releaseYear=toInteger(row.releaseYear),
26
24
  m.genres=split(row.genres,':')
27
- load_role:
28
- |
25
+ load_role: |
29
26
  WITH $rows AS rows
30
27
  UNWIND rows AS row
31
28
  MATCH(p:Person {personId:row.personId}),(m:Movie {movieId:row.movieId})
32
29
  MERGE(p)-[r:ACTED_IN]->(m)
33
30
  ON CREATE
34
31
  SET r.character=split(row.character,':')
32
+ create_actor: |
33
+ MERGE(p:Person:Actor {personId:$id})
34
+ ON CREATE
35
+ SET p.personId=$id,
36
+ p.name=$name,
37
+ p.birthYear=$year
@@ -93,11 +93,21 @@ class TestNeo4jInstance(unittest.TestCase):
93
93
  # Test the execute_read_query_using cypher parameters
94
94
  movie = "The Matrix"
95
95
  query = "MATCH(m:Movie) WHERE m.title=$movie_name RETURN m.title AS movie"
96
- print('Parameters')
96
+ print('Read Parameters')
97
97
  result = self.graph.execute_read_query(query,
98
98
  self.queries['database'],
99
- movie_name=movie)
99
+ {'movie_name':movie})
100
100
  self.assertEqual(movie,result['movie'][0])
101
101
 
102
+ # Test the execute_write_query_using cypher parameters
103
+ print('Write Parameters')
104
+ result = self.graph.execute_write_query(
105
+ self.queries['queries']['create_actor'],
106
+ self.queries['database'],
107
+ parameters={'id':-1,'name':'Darth Vader','year':'1977'})
108
+ solution = {'nodes_created':1,'labels_added':2,
109
+ 'properties_set':4}
110
+ self.assertEqual(solution, result)
111
+
102
112
  if __name__ == '__main__':
103
113
  unittest.main()
@@ -1,205 +0,0 @@
1
- # PyNeoInstance Package
2
-
3
- ## What is it?
4
- Is a Python package that provides a user friendly interface for submitting Cypher queries to Neo4j 4.x.
5
- It supports the following features:
6
-
7
- - __Read query__: Submit a Cypher query and get the result of it as a Pandas DataFrame.
8
- - __Write query__: Sumbmit a Cypher query to update an existing database.
9
- - __Write query with data__: Update a database based on a Pandas DataFrame and one or more Cypher queries.
10
-
11
- ## How to install it?
12
-
13
- ```python
14
- pip install pyneoinstance
15
- ```
16
-
17
- ## How to use it?
18
-
19
- ### Instantiating the Neo4jInstance class
20
-
21
- In order to use this package you first need to import the Neo4jInstance class.
22
-
23
- ```python
24
- from pyneoingest import Neo4jInstance
25
- ```
26
-
27
- Once imported, you need to instantiate this class by providing the following arguments:
28
-
29
- - _uri_: The Neo4j instance unique resource identifier.
30
- - _user_: The user name use to authenticate.
31
- - _password_: The user name password.
32
-
33
- ```python
34
- uri = 'bolt://localhost:7687'
35
- user = 'neo4j'
36
- password = 'password'
37
-
38
- graph = Neo4jInstance(uri, user, password)
39
- ```
40
-
41
- Although the previous code is valid, it is not recommended to hard code your user credentials in your application. As an alternative you can store this information as part of your OS user environment variables and then retrieve them in Python.
42
-
43
- ```python
44
- import os
45
-
46
- uri = os.environ['NEO4J_URI']
47
- user = os.environ['NEO4J_USER']
48
- password = os.environ['NEO4J_PASSWORD']
49
- graph = Neo4jInstance(uri,user,password)
50
- ```
51
- Another alternative is to create a YAML configutation file to store this infromation.
52
-
53
- ```yaml
54
- # configuration.yaml
55
- db_info:
56
- uri: neo4j_uri
57
- user: neo4j_user
58
- password: user_password
59
- ```
60
-
61
- This YAML file can then be loaded in Python and these information can be retrieve.
62
-
63
- ```python
64
- from pyneoinstance import load_yaml_file
65
-
66
- config = load_yaml_file(configuration.yaml)
67
- db_info = config['db_info']
68
- graph = Neo4jInstance(db_info['uri'], db_info['user'], db_info['password'])
69
- ```
70
- ### Submitting a Cypher read query
71
-
72
- Once you instantiated the Neo4jInstance class, you can submit a Cypher read query by using the class method execute\_read\_query. This method receive the Cypher query string as an argument and optionally the name of the database on which the query should be executed. If the database name is not provided the query will be execute using your default Neo4j database. The results of the query is return as a a Pandas DataFrame. To avoid unwanted column names in the resulting DataFrame, always provide aliases for all return properties.
73
-
74
- ```python
75
- query = """
76
- MATCH (m:Movie {title: "The Matrix"})
77
- RETURN m.tile AS title,
78
- m.released AS releasedYear,
79
- m.tagline AS tagline;
80
- """
81
-
82
- data = graph.execute_read_query(query)
83
- ```
84
- ### Submitting a Cypher write query
85
-
86
- We submit a Cypher write query using the method execute\_write\_query. The signature of this method is identical to the execute\_read\_query. This method returns a dictionary containing the updates made to the database.
87
-
88
- ```python
89
- query = """
90
- MERGE(m:Movie {title:'Wakanda Forever'})
91
- ON CREATE SET
92
- m.release=2022,
93
- m.tagline = 'Wakanda Forever! Long Live Wakanda! For Honor, For Legacy, For Wakanda!'
94
- """
95
-
96
- results = graph.execute_write_query(query)
97
- ```
98
-
99
- ### Updating a database with data in a DataFrame: Sequentially
100
-
101
- We can update a Neo4j database with data contained in a Pandas DataFrame using the methods execute\_write\_query\_with\_data or execute\_write\_queries\_with\_data. These methos return a dictionary containing the numbers and types of updates made to the database. Besides taking the query or list of queries as an argument, we can pass to these methods the following arguments:
102
-
103
- - _data_: Pandas DataFrame containing data to process.
104
- - _database_: Optional, database on which to execute the quieries.
105
- - _partitions_: Optional, number of partitions use to break the DataFrame in smaller chucks.
106
- - _parallel_: Wheather to execute the query using multiple Python process.
107
- - _workers_: The number of processes to spawn to execute the queries in parallel.
108
- - _parameters_: Extra arguments containing optional cypher parameters.
109
-
110
- To illustrates how to use this methods lets imagine we have a csv file that looks like this:
111
-
112
- ```python
113
- # people.csv
114
- personId,name,birthYear,deathYear
115
- 23945,Gérard Pirès,1942,
116
- 553509,Helen Reddy,1941,
117
- 113934,Susan Flannery,1939,
118
- 26706,David Rintoul,1948,
119
- 237537,Rita Marley,1946,
120
- 11502,Harry Davenport,1866,1949
121
- 11792,George Dickerson,1933,
122
- 7242,Temuera Morrison,1960,
123
- 3482,Chus Lampreave,1930,
124
- ```
125
-
126
- We want to update our database by loading this data as a node with the label Person containing the properties id, name, birthYear and deathYear. To accomplish this, we need to performe the following steps:
127
-
128
- 1. Load the csv to Python as a DataFrame.
129
- 2. Develop the Cypher query that is going to be use to load the DataFrame into Neo4j.
130
- 3. Call the execute\_write\_query\_with\_data of an existing instance of the Neo4jInstance class passing the DataFrame, Cypher query and any other optinal arguments to this method.
131
-
132
- ```python
133
- import pandas as pd
134
-
135
- person_data = pd.read_csv('people.csv')
136
-
137
- person_load_query = """
138
- WITH $rows AS rows
139
- UNWIND rows AS row
140
- MERGE(p:Person {id:row.personId})
141
- ON CREATE
142
- SET p.id = row.personId,
143
- p.name = row.name,
144
- p.birthYear=toInteger(row.birthYear),
145
- p.deathYear = toInteger(row.deathYear)
146
- """
147
-
148
- person_load_result = graph.execute_write_query_with_data(person_data, person_load_query)
149
- ```
150
-
151
- Is not a good practice to store your Cypher queries as strings within your Python modules. To solve this we can extend our YAML configuration file to store all our Cypher queries.
152
-
153
- ```yaml
154
- # configuration.yaml
155
- db_info:
156
- uri: neo4j_uri
157
- user: neo4j_user
158
- password: user_password
159
- loading_queries:
160
- nodes:
161
- person: |
162
- WITH $rows AS rows
163
- UNWIND rows AS row
164
- MERGE(p:Person {id:row.personId})
165
- ON CREATE
166
- SET p.id = row.personId,
167
- p.name = row.name,
168
- p.birthYear=toInteger(row.birthYear),
169
- p.deathYear = toInteger(row.deathYear)
170
- ```
171
-
172
- Now that we added the loading query to our configuration file, lets re-write our Python script to get this query from the configuration file.
173
-
174
- ```python
175
- import pandas as pd
176
-
177
- person_data = pd.read_csv('people.csv')
178
-
179
- node_load_queries = config['loading_queries']['nodes']
180
-
181
- person_load_result = graph.execute_write_query_with_data(person_data, node_load_queries['person'])
182
- ```
183
-
184
- If the number of rows in your DataFrame is 100K or more it is reccomended to break the DataFrame in smaller chunks so that they can be process in different transactions. This will avoid running out of heap error. To do this, just indicate in the _partitions_ argument how many partitions would you like to break this DataFrame into. This partitions will be loaded in different sequential transactions unless you specified loading them in parallel by setting the _parallel_ argument to true. We will discuss this process in the next section.
185
-
186
- ```python
187
- person_load_result = graph.execute_write_query_with_data(
188
- person_data,
189
- node_load_queries['person'],
190
- partitions = 5
191
- )
192
- ```
193
- ## Updating a database with data in a DataFrame: Parallel
194
-
195
- To load the data in the DataFrame in parallel, we need to partition the DataFrame by incrementing the _partitions_ argument and set the _parallel_ argument to true. By default this will use all the available virtual CPUs in the machine. If you need to use less, you can specified how many to use by setting the _workers_ argument.
196
-
197
- ```python
198
- person_load_result = graph.execute_write_query_with_data(
199
- person_data,
200
- node_load_queries['person'],
201
- partitions = 5,
202
- parallel = True,
203
- workers = 5
204
- )
205
- ```
File without changes
File without changes
File without changes
File without changes