citation-map 4.4__tar.gz → 4.6__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {citation-map-4.4/citation_map.egg-info → citation_map-4.6}/PKG-INFO +57 -21
- citation-map-4.4/PKG-INFO → citation_map-4.6/README.md +36 -34
- {citation-map-4.4 → citation_map-4.6}/citation_map/citation_map.py +68 -49
- {citation-map-4.4 → citation_map-4.6}/citation_map/scholarly_support.py +3 -0
- citation-map-4.4/README.md → citation_map-4.6/citation_map.egg-info/PKG-INFO +70 -18
- {citation-map-4.4 → citation_map-4.6}/setup.py +1 -1
- {citation-map-4.4 → citation_map-4.6}/LICENSE.md +0 -0
- {citation-map-4.4 → citation_map-4.6}/citation_map/__init__.py +0 -0
- {citation-map-4.4 → citation_map-4.6}/citation_map.egg-info/SOURCES.txt +0 -0
- {citation-map-4.4 → citation_map-4.6}/citation_map.egg-info/dependency_links.txt +0 -0
- {citation-map-4.4 → citation_map-4.6}/citation_map.egg-info/requires.txt +0 -0
- {citation-map-4.4 → citation_map-4.6}/citation_map.egg-info/top_level.txt +0 -0
- {citation-map-4.4 → citation_map-4.6}/setup.cfg +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
Metadata-Version: 2.
|
|
1
|
+
Metadata-Version: 2.2
|
|
2
2
|
Name: citation-map
|
|
3
|
-
Version: 4.
|
|
3
|
+
Version: 4.6
|
|
4
4
|
Summary: A simple tool to generate your Google Scholar citation world map.
|
|
5
5
|
Home-page: https://github.com/ChenLiu-1996/CitationMap
|
|
6
6
|
Author: Chen Liu
|
|
@@ -13,45 +13,69 @@ Classifier: License :: Other/Proprietary License
|
|
|
13
13
|
Classifier: Programming Language :: Python :: 3
|
|
14
14
|
Description-Content-Type: text/markdown
|
|
15
15
|
License-File: LICENSE.md
|
|
16
|
+
Requires-Dist: scholarly
|
|
17
|
+
Requires-Dist: geopy
|
|
18
|
+
Requires-Dist: folium
|
|
19
|
+
Requires-Dist: tqdm
|
|
20
|
+
Requires-Dist: requests
|
|
21
|
+
Requires-Dist: bs4
|
|
22
|
+
Requires-Dist: pycountry
|
|
23
|
+
Requires-Dist: pandas
|
|
24
|
+
Dynamic: author
|
|
25
|
+
Dynamic: author-email
|
|
26
|
+
Dynamic: classifier
|
|
27
|
+
Dynamic: description
|
|
28
|
+
Dynamic: description-content-type
|
|
29
|
+
Dynamic: home-page
|
|
30
|
+
Dynamic: keywords
|
|
31
|
+
Dynamic: license
|
|
32
|
+
Dynamic: requires-dist
|
|
33
|
+
Dynamic: summary
|
|
34
|
+
|
|
35
|
+
<h1 align="center">
|
|
36
|
+
Google Scholar Citation World Map
|
|
37
|
+
</h1>
|
|
38
|
+
|
|
39
|
+
<p align="center">
|
|
40
|
+
<strong>CitationMap: A Python Tool to Identify and Visualize Your Google Scholar Citations Around the World</strong> [<a href="https://openreview.net/pdf?id=BqJgCgl1IA">PDF</a>]
|
|
41
|
+
</p>
|
|
42
|
+
|
|
43
|
+
<div align="center">
|
|
16
44
|
|
|
17
|
-
# Google Scholar Citation World Map
|
|
18
|
-
|
|
19
|
-
[](https://www.techrxiv.org/users/809001/articles/1213717-citationmap-a-python-tool-to-identify-and-visualize-your-google-scholar-citations-around-the-world)
|
|
20
45
|
[](https://openreview.net/pdf?id=BqJgCgl1IA)
|
|
46
|
+
[](https://www.techrxiv.org/users/809001/articles/1213717-citationmap-a-python-tool-to-identify-and-visualize-your-google-scholar-citations-around-the-world)
|
|
47
|
+
[](https://twitter.com/ChenLiu_1996)
|
|
48
|
+
[](https://www.linkedin.com/in/chenliu1996/)
|
|
49
|
+
[![CC BY-NC-SA 4.0][cc-by-nc-sa-shield]][cc-by-nc-sa]
|
|
50
|
+
[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]
|
|
51
|
+
<br>
|
|
21
52
|
[](https://pypi.org/project/citation-map/)
|
|
53
|
+
[](https://pepy.tech/projects/citation-map)
|
|
22
54
|
[](https://pypistats.org/packages/citation-map)
|
|
23
55
|
[](https://pypistats.org/packages/citation-map)
|
|
24
|
-
[![CC BY-NC-SA 4.0][cc-by-nc-sa-shield]][cc-by-nc-sa]
|
|
25
|
-
[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]
|
|
26
56
|
|
|
27
57
|
[cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/
|
|
28
58
|
[cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png
|
|
29
59
|
[cc-by-nc-sa-shield]: https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey.svg
|
|
30
60
|
|
|
31
|
-
|
|
61
|
+
</div>
|
|
32
62
|
|
|
33
|
-
|
|
63
|
+
⚠ There is **no need to fork** this repo unless you want to make custom changes.
|
|
34
64
|
|
|
65
|
+
⚠ It only takes **one line to install** and **almost one line to run**!
|
|
35
66
|
|
|
36
|
-
|
|
37
|
-
**[Asking for advice]**
|
|
38
|
-
1. This is my first time dealing with webscraping/crawling. Users have reported stability issues, and I suspect the major problems are (1) being caught by CAPTCHA or robot check, and (2) being flagged for blacklist by Google Scholar. If you are experienced in these areas and have good advice, I would highly appreciate a **GitHub issue or a pull request**.
|
|
39
|
-
|
|
40
|
-
[Aug 2, 2024] Version 4.0 released >>> Logic update. A new input argument `affiliation_conservative`. If set to True, we will use **a very conservative approach to identify affiliations** which leads to **much higher precision and lower recall**. Many thanks to [Zhijian Liu](https://github.com/zhijian-liu) for the [helpful discussion](https://github.com/ChenLiu-1996/CitationMap/issues/8).
|
|
67
|
+
<br>
|
|
41
68
|
|
|
42
|
-
[
|
|
69
|
+
I am [Chen Liu](https://chenliu-1996.github.io/), a CS PhD Candidate at Yale University.
|
|
43
70
|
|
|
44
|
-
|
|
71
|
+
Research areas: Machine Learning (Manifold Learning, Spatial-Temporal Dynamics, Medical Vision, AI4Science).
|
|
45
72
|
|
|
46
|
-
[Jul 26, 2024] Version 1.0 released >>> First working version for my profile with 100 citations.
|
|
47
73
|
|
|
48
74
|
## Purpose
|
|
49
75
|
This is a simple Python tool to generate an HTML citation world map from your Google Scholar ID.
|
|
50
76
|
|
|
51
77
|
It is easy to install (`pip install citation-map`, available on [PyPI](https://pypi.org/project/citation-map/)) and easy to use (see the [User Guide](https://github.com/ChenLiu-1996/CitationMap?tab=readme-ov-file#user-guide)).
|
|
52
78
|
|
|
53
|
-
**You don't need to fork this repo unless you want to make custom changes.**
|
|
54
|
-
|
|
55
79
|
## Expected Outcome
|
|
56
80
|
You will be given an **HTML file** as the output of the script.
|
|
57
81
|
|
|
@@ -89,6 +113,18 @@ Chicago
|
|
|
89
113
|
Liu, Chen. "CitationMap: A Python Tool to Identify and Visualize Your Google Scholar Citations Around the World." Authorea Preprints (2024).
|
|
90
114
|
```
|
|
91
115
|
|
|
116
|
+
## News
|
|
117
|
+
**[Asking for advice]**
|
|
118
|
+
1. This is my first time dealing with webscraping/crawling. Users have reported stability issues, and I suspect the major problems are (1) being caught by CAPTCHA or robot check, and (2) being flagged for blacklist by Google Scholar. If you are experienced in these areas and have good advice, I would highly appreciate a **GitHub issue or a pull request**.
|
|
119
|
+
|
|
120
|
+
[Aug 2, 2024] Version 4.0 released >>> Logic update. A new input argument `affiliation_conservative`. If set to True, we will use **a very conservative approach to identify affiliations** which leads to **much higher precision and lower recall**. Many thanks to [Zhijian Liu](https://github.com/zhijian-liu) for the [helpful discussion](https://github.com/ChenLiu-1996/CitationMap/issues/8).
|
|
121
|
+
|
|
122
|
+
[Jul 28, 2024] Version 3.10 released >>> Logic update. Tested on a professor's profile **with 10,000 citations**!
|
|
123
|
+
|
|
124
|
+
[Jul 27, 2024] Version 2.0 released >>> 10x speedup with multiprocessing (1 hour to 5 minutes for my profile).
|
|
125
|
+
|
|
126
|
+
[Jul 26, 2024] Version 1.0 released >>> First working version for my profile with 100 citations.
|
|
127
|
+
|
|
92
128
|
## User Guide
|
|
93
129
|
0. If you are new to Python, you probably want to start with a distribution of Python that also helps with environment management (such as [anaconda](https://www.anaconda.com/)). Once you have set up your environment (for example, when you reach the stage of `conda activate env39` in [this tutorial](https://www.youtube.com/watch?v=MUZtVEDKXsk&t=242s)), you can move on to the next step.
|
|
94
130
|
1. Install this tool by running the following line in your conda-accessible command line.
|
|
@@ -115,7 +151,7 @@ Liu, Chen. "CitationMap: A Python Tool to Identify and Visualize Your Google Sch
|
|
|
115
151
|
generate_citation_map(scholar_id)
|
|
116
152
|
```
|
|
117
153
|
|
|
118
|
-
Note that in Version 4.
|
|
154
|
+
Note that in Version 4.5, we support manual edit of the csv. You can run `generate_citation_map` first, and get the csv and HTML. Then, you can manually modify the csv content, and generate an updated HTML by running `generate_citation_map` again with `parse_csv=True`.
|
|
119
155
|
|
|
120
156
|
Note that in Version 4.0, we will cache the results before identifying affiliations. So if you want to rerun the same author from scratch, you need to delete the cache (default location is 'cache').
|
|
121
157
|
|
|
@@ -206,7 +242,7 @@ Liu, Chen. "CitationMap: A Python Tool to Identify and Visualize Your Google Sch
|
|
|
206
242
|
|
|
207
243
|
## Changelog
|
|
208
244
|
<details>
|
|
209
|
-
<summary>Version 4.
|
|
245
|
+
<summary>Version 4.5 (Dec 6, 2024)</summary>
|
|
210
246
|
<br>
|
|
211
247
|
|
|
212
248
|
1. **Support manual edit of the csv.** Note you do need to manually correct the coordinates too if you choose to use this functionality. You can run `generate_citation_map` first, and get the csv and HTML. Then, you can manually modify the csv content, and generate an updated HTML by running `generate_citation_map` again with `parse_csv=True`.
|
|
@@ -1,57 +1,47 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
Classifier: Development Status :: 3 - Alpha
|
|
11
|
-
Classifier: Intended Audience :: Developers
|
|
12
|
-
Classifier: License :: Other/Proprietary License
|
|
13
|
-
Classifier: Programming Language :: Python :: 3
|
|
14
|
-
Description-Content-Type: text/markdown
|
|
15
|
-
License-File: LICENSE.md
|
|
16
|
-
|
|
17
|
-
# Google Scholar Citation World Map
|
|
1
|
+
<h1 align="center">
|
|
2
|
+
Google Scholar Citation World Map
|
|
3
|
+
</h1>
|
|
4
|
+
|
|
5
|
+
<p align="center">
|
|
6
|
+
<strong>CitationMap: A Python Tool to Identify and Visualize Your Google Scholar Citations Around the World</strong> [<a href="https://openreview.net/pdf?id=BqJgCgl1IA">PDF</a>]
|
|
7
|
+
</p>
|
|
8
|
+
|
|
9
|
+
<div align="center">
|
|
18
10
|
|
|
19
|
-
[](https://www.techrxiv.org/users/809001/articles/1213717-citationmap-a-python-tool-to-identify-and-visualize-your-google-scholar-citations-around-the-world)
|
|
20
11
|
[](https://openreview.net/pdf?id=BqJgCgl1IA)
|
|
12
|
+
[](https://www.techrxiv.org/users/809001/articles/1213717-citationmap-a-python-tool-to-identify-and-visualize-your-google-scholar-citations-around-the-world)
|
|
13
|
+
[](https://twitter.com/ChenLiu_1996)
|
|
14
|
+
[](https://www.linkedin.com/in/chenliu1996/)
|
|
15
|
+
[![CC BY-NC-SA 4.0][cc-by-nc-sa-shield]][cc-by-nc-sa]
|
|
16
|
+
[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]
|
|
17
|
+
<br>
|
|
21
18
|
[](https://pypi.org/project/citation-map/)
|
|
19
|
+
[](https://pepy.tech/projects/citation-map)
|
|
22
20
|
[](https://pypistats.org/packages/citation-map)
|
|
23
21
|
[](https://pypistats.org/packages/citation-map)
|
|
24
|
-
[![CC BY-NC-SA 4.0][cc-by-nc-sa-shield]][cc-by-nc-sa]
|
|
25
|
-
[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]
|
|
26
22
|
|
|
27
23
|
[cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/
|
|
28
24
|
[cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png
|
|
29
25
|
[cc-by-nc-sa-shield]: https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey.svg
|
|
30
26
|
|
|
31
|
-
|
|
27
|
+
</div>
|
|
32
28
|
|
|
33
|
-
|
|
29
|
+
⚠ There is **no need to fork** this repo unless you want to make custom changes.
|
|
34
30
|
|
|
31
|
+
⚠ It only takes **one line to install** and **almost one line to run**!
|
|
35
32
|
|
|
36
|
-
|
|
37
|
-
**[Asking for advice]**
|
|
38
|
-
1. This is my first time dealing with webscraping/crawling. Users have reported stability issues, and I suspect the major problems are (1) being caught by CAPTCHA or robot check, and (2) being flagged for blacklist by Google Scholar. If you are experienced in these areas and have good advice, I would highly appreciate a **GitHub issue or a pull request**.
|
|
33
|
+
<br>
|
|
39
34
|
|
|
40
|
-
|
|
35
|
+
I am [Chen Liu](https://chenliu-1996.github.io/), a CS PhD Candidate at Yale University.
|
|
41
36
|
|
|
42
|
-
|
|
37
|
+
Research areas: Machine Learning (Manifold Learning, Spatial-Temporal Dynamics, Medical Vision, AI4Science).
|
|
43
38
|
|
|
44
|
-
[Jul 27, 2024] Version 2.0 released >>> 10x speedup with multiprocessing (1 hour to 5 minutes for my profile).
|
|
45
|
-
|
|
46
|
-
[Jul 26, 2024] Version 1.0 released >>> First working version for my profile with 100 citations.
|
|
47
39
|
|
|
48
40
|
## Purpose
|
|
49
41
|
This is a simple Python tool to generate an HTML citation world map from your Google Scholar ID.
|
|
50
42
|
|
|
51
43
|
It is easy to install (`pip install citation-map`, available on [PyPI](https://pypi.org/project/citation-map/)) and easy to use (see the [User Guide](https://github.com/ChenLiu-1996/CitationMap?tab=readme-ov-file#user-guide)).
|
|
52
44
|
|
|
53
|
-
**You don't need to fork this repo unless you want to make custom changes.**
|
|
54
|
-
|
|
55
45
|
## Expected Outcome
|
|
56
46
|
You will be given an **HTML file** as the output of the script.
|
|
57
47
|
|
|
@@ -89,6 +79,18 @@ Chicago
|
|
|
89
79
|
Liu, Chen. "CitationMap: A Python Tool to Identify and Visualize Your Google Scholar Citations Around the World." Authorea Preprints (2024).
|
|
90
80
|
```
|
|
91
81
|
|
|
82
|
+
## News
|
|
83
|
+
**[Asking for advice]**
|
|
84
|
+
1. This is my first time dealing with webscraping/crawling. Users have reported stability issues, and I suspect the major problems are (1) being caught by CAPTCHA or robot check, and (2) being flagged for blacklist by Google Scholar. If you are experienced in these areas and have good advice, I would highly appreciate a **GitHub issue or a pull request**.
|
|
85
|
+
|
|
86
|
+
[Aug 2, 2024] Version 4.0 released >>> Logic update. A new input argument `affiliation_conservative`. If set to True, we will use **a very conservative approach to identify affiliations** which leads to **much higher precision and lower recall**. Many thanks to [Zhijian Liu](https://github.com/zhijian-liu) for the [helpful discussion](https://github.com/ChenLiu-1996/CitationMap/issues/8).
|
|
87
|
+
|
|
88
|
+
[Jul 28, 2024] Version 3.10 released >>> Logic update. Tested on a professor's profile **with 10,000 citations**!
|
|
89
|
+
|
|
90
|
+
[Jul 27, 2024] Version 2.0 released >>> 10x speedup with multiprocessing (1 hour to 5 minutes for my profile).
|
|
91
|
+
|
|
92
|
+
[Jul 26, 2024] Version 1.0 released >>> First working version for my profile with 100 citations.
|
|
93
|
+
|
|
92
94
|
## User Guide
|
|
93
95
|
0. If you are new to Python, you probably want to start with a distribution of Python that also helps with environment management (such as [anaconda](https://www.anaconda.com/)). Once you have set up your environment (for example, when you reach the stage of `conda activate env39` in [this tutorial](https://www.youtube.com/watch?v=MUZtVEDKXsk&t=242s)), you can move on to the next step.
|
|
94
96
|
1. Install this tool by running the following line in your conda-accessible command line.
|
|
@@ -115,7 +117,7 @@ Liu, Chen. "CitationMap: A Python Tool to Identify and Visualize Your Google Sch
|
|
|
115
117
|
generate_citation_map(scholar_id)
|
|
116
118
|
```
|
|
117
119
|
|
|
118
|
-
Note that in Version 4.
|
|
120
|
+
Note that in Version 4.5, we support manual edit of the csv. You can run `generate_citation_map` first, and get the csv and HTML. Then, you can manually modify the csv content, and generate an updated HTML by running `generate_citation_map` again with `parse_csv=True`.
|
|
119
121
|
|
|
120
122
|
Note that in Version 4.0, we will cache the results before identifying affiliations. So if you want to rerun the same author from scratch, you need to delete the cache (default location is 'cache').
|
|
121
123
|
|
|
@@ -206,7 +208,7 @@ Liu, Chen. "CitationMap: A Python Tool to Identify and Visualize Your Google Sch
|
|
|
206
208
|
|
|
207
209
|
## Changelog
|
|
208
210
|
<details>
|
|
209
|
-
<summary>Version 4.
|
|
211
|
+
<summary>Version 4.5 (Dec 6, 2024)</summary>
|
|
210
212
|
<br>
|
|
211
213
|
|
|
212
214
|
1. **Support manual edit of the csv.** Note you do need to manually correct the coordinates too if you choose to use this functionality. You can run `generate_citation_map` first, and get the csv and HTML. Then, you can manually modify the csv content, and generate an updated HTML by running `generate_citation_map` again with `parse_csv=True`.
|
|
@@ -16,7 +16,7 @@ from scholarly import scholarly, ProxyGenerator
|
|
|
16
16
|
from tqdm import tqdm
|
|
17
17
|
from typing import Any, List, Tuple
|
|
18
18
|
|
|
19
|
-
from .scholarly_support import get_citing_author_ids_and_citing_papers, get_organization_name
|
|
19
|
+
from .scholarly_support import get_citing_author_ids_and_citing_papers, get_organization_name, NO_AUTHOR_FOUND_STR
|
|
20
20
|
|
|
21
21
|
|
|
22
22
|
def find_all_citing_authors(scholar_id: str, num_processes: int = 16) -> List[Tuple[str]]:
|
|
@@ -103,23 +103,26 @@ def clean_affiliation_names(author_paper_affiliation_tuple_list: List[Tuple[str]
|
|
|
103
103
|
'''
|
|
104
104
|
cleaned_author_paper_affiliation_tuple_list = []
|
|
105
105
|
for author_name, citing_paper_title, cited_paper_title, affiliation_string in author_paper_affiliation_tuple_list:
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
106
|
+
if author_name == NO_AUTHOR_FOUND_STR:
|
|
107
|
+
cleaned_author_paper_affiliation_tuple_list.append((NO_AUTHOR_FOUND_STR, citing_paper_title, cited_paper_title, NO_AUTHOR_FOUND_STR))
|
|
108
|
+
else:
|
|
109
|
+
# Use a regular expression to split the string by ';' or 'and'.
|
|
110
|
+
substring_list = [part.strip() for part in re.split(r'[;]|\band\b', affiliation_string)]
|
|
111
|
+
# Further split the substrings by ',' if the latter component is not a country.
|
|
112
|
+
substring_list = __country_aware_comma_split(substring_list)
|
|
113
|
+
|
|
114
|
+
for substring in substring_list:
|
|
115
|
+
# Use a regular expression to remove anything before 'at', or '@'.
|
|
116
|
+
cleaned_affiliation = re.sub(r'.*?\bat\b|.*?@', '', substring, flags=re.IGNORECASE).strip()
|
|
117
|
+
# Use a regular expression to filter out strings that represent
|
|
118
|
+
# a person's identity rather than affiliation.
|
|
119
|
+
is_common_identity_string = re.search(
|
|
120
|
+
re.compile(
|
|
121
|
+
r'\b(director|manager|chair|engineer|programmer|scientist|professor|lecturer|phd|ph\.d|postdoc|doctor|student|department of)\b',
|
|
122
|
+
re.IGNORECASE),
|
|
123
|
+
cleaned_affiliation)
|
|
124
|
+
if not is_common_identity_string:
|
|
125
|
+
cleaned_author_paper_affiliation_tuple_list.append((author_name, citing_paper_title, cited_paper_title, cleaned_affiliation))
|
|
123
126
|
return cleaned_author_paper_affiliation_tuple_list
|
|
124
127
|
|
|
125
128
|
def affiliation_text_to_geocode(author_paper_affiliation_tuple_list: List[Tuple[str]], max_attempts: int = 3) -> List[Tuple[str]]:
|
|
@@ -146,34 +149,41 @@ def affiliation_text_to_geocode(author_paper_affiliation_tuple_list: List[Tuple[
|
|
|
146
149
|
desc='Finding geographic coordinates from %d unique citing affiliations in %d entries' % (
|
|
147
150
|
len(affiliation_map), len(author_paper_affiliation_tuple_list)),
|
|
148
151
|
total=len(affiliation_map)):
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
if
|
|
160
|
-
city
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
152
|
+
if affiliation_name == NO_AUTHOR_FOUND_STR:
|
|
153
|
+
corresponding_entries = affiliation_map[affiliation_name]
|
|
154
|
+
for entry_idx in corresponding_entries:
|
|
155
|
+
author_name, citing_paper_title, cited_paper_title, affiliation_name = author_paper_affiliation_tuple_list[entry_idx]
|
|
156
|
+
coordinates_and_info.append((NO_AUTHOR_FOUND_STR, citing_paper_title, cited_paper_title, NO_AUTHOR_FOUND_STR,
|
|
157
|
+
'', '', '', '', '', ''))
|
|
158
|
+
else:
|
|
159
|
+
for _ in range(max_attempts):
|
|
160
|
+
try:
|
|
161
|
+
geo_location = geolocator.geocode(affiliation_name)
|
|
162
|
+
if geo_location:
|
|
163
|
+
# Get the full location metadata that includes county, city, state, country, etc.
|
|
164
|
+
location_metadata = geolocator.reverse(str(geo_location.latitude) + ',' + str(geo_location.longitude), language='en')
|
|
165
|
+
address = location_metadata.raw['address']
|
|
166
|
+
county, city, state, country = None, None, None, None
|
|
167
|
+
if 'county' in address:
|
|
168
|
+
county = address['county']
|
|
169
|
+
if 'city' in address:
|
|
170
|
+
city = address['city']
|
|
171
|
+
if 'state' in address:
|
|
172
|
+
state = address['state']
|
|
173
|
+
if 'country' in address:
|
|
174
|
+
country = address['country']
|
|
175
|
+
|
|
176
|
+
corresponding_entries = affiliation_map[affiliation_name]
|
|
177
|
+
for entry_idx in corresponding_entries:
|
|
178
|
+
author_name, citing_paper_title, cited_paper_title, affiliation_name = author_paper_affiliation_tuple_list[entry_idx]
|
|
179
|
+
coordinates_and_info.append((author_name, citing_paper_title, cited_paper_title, affiliation_name,
|
|
180
|
+
geo_location.latitude, geo_location.longitude,
|
|
181
|
+
county, city, state, country))
|
|
182
|
+
# This location is successfully recorded.
|
|
183
|
+
num_located_affiliations += 1
|
|
184
|
+
break
|
|
185
|
+
except:
|
|
186
|
+
continue
|
|
177
187
|
print('\nConverted %d/%d affiliations to Geocodes.' % (num_located_affiliations, num_total_affiliations))
|
|
178
188
|
coordinates_and_info = [item for item in coordinates_and_info if item is not None] # Filter out empty entries.
|
|
179
189
|
return coordinates_and_info
|
|
@@ -212,7 +222,9 @@ def create_map(coordinates_and_info: List[Tuple[str]], pin_colorful: bool = True
|
|
|
212
222
|
# Find unique affiliations and record their corresponding entries.
|
|
213
223
|
affiliation_map = {}
|
|
214
224
|
for entry_idx, (_, _, _, affiliation_name, _, _, _, _, _, _) in enumerate(coordinates_and_info):
|
|
215
|
-
if affiliation_name
|
|
225
|
+
if affiliation_name == NO_AUTHOR_FOUND_STR:
|
|
226
|
+
continue
|
|
227
|
+
elif affiliation_name not in affiliation_map.keys():
|
|
216
228
|
affiliation_map[affiliation_name] = [entry_idx]
|
|
217
229
|
else:
|
|
218
230
|
affiliation_map[affiliation_name].append(entry_idx)
|
|
@@ -261,14 +273,16 @@ def __affiliations_from_authors_conservative(citing_author_paper_info: str):
|
|
|
261
273
|
This will have higher precision and lower recall.
|
|
262
274
|
'''
|
|
263
275
|
citing_author_id, citing_paper_title, cited_paper_title = citing_author_paper_info
|
|
276
|
+
if citing_author_id == NO_AUTHOR_FOUND_STR:
|
|
277
|
+
return (NO_AUTHOR_FOUND_STR, citing_paper_title, cited_paper_title, NO_AUTHOR_FOUND_STR)
|
|
278
|
+
|
|
264
279
|
time.sleep(random.uniform(1, 5)) # Random delay to reduce risk of being blocked.
|
|
265
280
|
citing_author = scholarly.search_author_id(citing_author_id)
|
|
266
281
|
|
|
267
282
|
if 'organization' in citing_author:
|
|
268
|
-
author_name = citing_author['name']
|
|
269
283
|
try:
|
|
270
284
|
author_organization = get_organization_name(citing_author['organization'])
|
|
271
|
-
return (
|
|
285
|
+
return (citing_author['name'], citing_paper_title, cited_paper_title, author_organization)
|
|
272
286
|
except Exception as e:
|
|
273
287
|
print('[Warning!]', e)
|
|
274
288
|
return None
|
|
@@ -280,6 +294,9 @@ def __affiliations_from_authors_aggressive(citing_author_paper_info: str):
|
|
|
280
294
|
This will have lower precision and higher recall.
|
|
281
295
|
'''
|
|
282
296
|
citing_author_id, citing_paper_title, cited_paper_title = citing_author_paper_info
|
|
297
|
+
if citing_author_id == NO_AUTHOR_FOUND_STR:
|
|
298
|
+
return (NO_AUTHOR_FOUND_STR, citing_paper_title, cited_paper_title, NO_AUTHOR_FOUND_STR)
|
|
299
|
+
|
|
283
300
|
time.sleep(random.uniform(1, 5)) # Random delay to reduce risk of being blocked.
|
|
284
301
|
citing_author = scholarly.search_author_id(citing_author_id)
|
|
285
302
|
if 'affiliation' in citing_author:
|
|
@@ -318,6 +335,8 @@ def __iscountry(string: str) -> bool:
|
|
|
318
335
|
def __print_author_and_affiliation(author_paper_affiliation_tuple_list: List[Tuple[str]]) -> None:
|
|
319
336
|
__author_affiliation_tuple_list = []
|
|
320
337
|
for author_name, _, _, affiliation_name in sorted(author_paper_affiliation_tuple_list):
|
|
338
|
+
if author_name == NO_AUTHOR_FOUND_STR:
|
|
339
|
+
continue
|
|
321
340
|
__author_affiliation_tuple_list.append((author_name, affiliation_name))
|
|
322
341
|
|
|
323
342
|
# Take unique tuples.
|
|
@@ -6,6 +6,8 @@ import time
|
|
|
6
6
|
from bs4 import BeautifulSoup
|
|
7
7
|
from typing import List
|
|
8
8
|
|
|
9
|
+
NO_AUTHOR_FOUND_STR = 'No_author_found'
|
|
10
|
+
|
|
9
11
|
|
|
10
12
|
def get_html_per_citation_page(soup) -> List[str]:
|
|
11
13
|
'''
|
|
@@ -31,6 +33,7 @@ def get_html_per_citation_page(soup) -> List[str]:
|
|
|
31
33
|
paper_parsed = True
|
|
32
34
|
if not paper_parsed:
|
|
33
35
|
print("[WARNING!] Could not find author links for ", title)
|
|
36
|
+
citing_authors_and_citing_papers.append((NO_AUTHOR_FOUND_STR, title))
|
|
34
37
|
else:
|
|
35
38
|
continue
|
|
36
39
|
return citing_authors_and_citing_papers
|
|
@@ -1,41 +1,81 @@
|
|
|
1
|
-
|
|
1
|
+
Metadata-Version: 2.2
|
|
2
|
+
Name: citation-map
|
|
3
|
+
Version: 4.6
|
|
4
|
+
Summary: A simple tool to generate your Google Scholar citation world map.
|
|
5
|
+
Home-page: https://github.com/ChenLiu-1996/CitationMap
|
|
6
|
+
Author: Chen Liu
|
|
7
|
+
Author-email: chen.liu.cl2482@yale.edu
|
|
8
|
+
License: CC BY-NC-SA
|
|
9
|
+
Keywords: citation map,citation world map,google scholar
|
|
10
|
+
Classifier: Development Status :: 3 - Alpha
|
|
11
|
+
Classifier: Intended Audience :: Developers
|
|
12
|
+
Classifier: License :: Other/Proprietary License
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Description-Content-Type: text/markdown
|
|
15
|
+
License-File: LICENSE.md
|
|
16
|
+
Requires-Dist: scholarly
|
|
17
|
+
Requires-Dist: geopy
|
|
18
|
+
Requires-Dist: folium
|
|
19
|
+
Requires-Dist: tqdm
|
|
20
|
+
Requires-Dist: requests
|
|
21
|
+
Requires-Dist: bs4
|
|
22
|
+
Requires-Dist: pycountry
|
|
23
|
+
Requires-Dist: pandas
|
|
24
|
+
Dynamic: author
|
|
25
|
+
Dynamic: author-email
|
|
26
|
+
Dynamic: classifier
|
|
27
|
+
Dynamic: description
|
|
28
|
+
Dynamic: description-content-type
|
|
29
|
+
Dynamic: home-page
|
|
30
|
+
Dynamic: keywords
|
|
31
|
+
Dynamic: license
|
|
32
|
+
Dynamic: requires-dist
|
|
33
|
+
Dynamic: summary
|
|
34
|
+
|
|
35
|
+
<h1 align="center">
|
|
36
|
+
Google Scholar Citation World Map
|
|
37
|
+
</h1>
|
|
38
|
+
|
|
39
|
+
<p align="center">
|
|
40
|
+
<strong>CitationMap: A Python Tool to Identify and Visualize Your Google Scholar Citations Around the World</strong> [<a href="https://openreview.net/pdf?id=BqJgCgl1IA">PDF</a>]
|
|
41
|
+
</p>
|
|
42
|
+
|
|
43
|
+
<div align="center">
|
|
2
44
|
|
|
3
|
-
[](https://www.techrxiv.org/users/809001/articles/1213717-citationmap-a-python-tool-to-identify-and-visualize-your-google-scholar-citations-around-the-world)
|
|
4
45
|
[](https://openreview.net/pdf?id=BqJgCgl1IA)
|
|
46
|
+
[](https://www.techrxiv.org/users/809001/articles/1213717-citationmap-a-python-tool-to-identify-and-visualize-your-google-scholar-citations-around-the-world)
|
|
47
|
+
[](https://twitter.com/ChenLiu_1996)
|
|
48
|
+
[](https://www.linkedin.com/in/chenliu1996/)
|
|
49
|
+
[![CC BY-NC-SA 4.0][cc-by-nc-sa-shield]][cc-by-nc-sa]
|
|
50
|
+
[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]
|
|
51
|
+
<br>
|
|
5
52
|
[](https://pypi.org/project/citation-map/)
|
|
53
|
+
[](https://pepy.tech/projects/citation-map)
|
|
6
54
|
[](https://pypistats.org/packages/citation-map)
|
|
7
55
|
[](https://pypistats.org/packages/citation-map)
|
|
8
|
-
[![CC BY-NC-SA 4.0][cc-by-nc-sa-shield]][cc-by-nc-sa]
|
|
9
|
-
[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]
|
|
10
56
|
|
|
11
57
|
[cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/
|
|
12
58
|
[cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png
|
|
13
59
|
[cc-by-nc-sa-shield]: https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey.svg
|
|
14
60
|
|
|
15
|
-
|
|
61
|
+
</div>
|
|
16
62
|
|
|
17
|
-
|
|
63
|
+
⚠ There is **no need to fork** this repo unless you want to make custom changes.
|
|
18
64
|
|
|
65
|
+
⚠ It only takes **one line to install** and **almost one line to run**!
|
|
19
66
|
|
|
20
|
-
|
|
21
|
-
**[Asking for advice]**
|
|
22
|
-
1. This is my first time dealing with webscraping/crawling. Users have reported stability issues, and I suspect the major problems are (1) being caught by CAPTCHA or robot check, and (2) being flagged for blacklist by Google Scholar. If you are experienced in these areas and have good advice, I would highly appreciate a **GitHub issue or a pull request**.
|
|
67
|
+
<br>
|
|
23
68
|
|
|
24
|
-
|
|
69
|
+
I am [Chen Liu](https://chenliu-1996.github.io/), a CS PhD Candidate at Yale University.
|
|
25
70
|
|
|
26
|
-
|
|
71
|
+
Research areas: Machine Learning (Manifold Learning, Spatial-Temporal Dynamics, Medical Vision, AI4Science).
|
|
27
72
|
|
|
28
|
-
[Jul 27, 2024] Version 2.0 released >>> 10x speedup with multiprocessing (1 hour to 5 minutes for my profile).
|
|
29
|
-
|
|
30
|
-
[Jul 26, 2024] Version 1.0 released >>> First working version for my profile with 100 citations.
|
|
31
73
|
|
|
32
74
|
## Purpose
|
|
33
75
|
This is a simple Python tool to generate an HTML citation world map from your Google Scholar ID.
|
|
34
76
|
|
|
35
77
|
It is easy to install (`pip install citation-map`, available on [PyPI](https://pypi.org/project/citation-map/)) and easy to use (see the [User Guide](https://github.com/ChenLiu-1996/CitationMap?tab=readme-ov-file#user-guide)).
|
|
36
78
|
|
|
37
|
-
**You don't need to fork this repo unless you want to make custom changes.**
|
|
38
|
-
|
|
39
79
|
## Expected Outcome
|
|
40
80
|
You will be given an **HTML file** as the output of the script.
|
|
41
81
|
|
|
@@ -73,6 +113,18 @@ Chicago
|
|
|
73
113
|
Liu, Chen. "CitationMap: A Python Tool to Identify and Visualize Your Google Scholar Citations Around the World." Authorea Preprints (2024).
|
|
74
114
|
```
|
|
75
115
|
|
|
116
|
+
## News
|
|
117
|
+
**[Asking for advice]**
|
|
118
|
+
1. This is my first time dealing with webscraping/crawling. Users have reported stability issues, and I suspect the major problems are (1) being caught by CAPTCHA or robot check, and (2) being flagged for blacklist by Google Scholar. If you are experienced in these areas and have good advice, I would highly appreciate a **GitHub issue or a pull request**.
|
|
119
|
+
|
|
120
|
+
[Aug 2, 2024] Version 4.0 released >>> Logic update. A new input argument `affiliation_conservative`. If set to True, we will use **a very conservative approach to identify affiliations** which leads to **much higher precision and lower recall**. Many thanks to [Zhijian Liu](https://github.com/zhijian-liu) for the [helpful discussion](https://github.com/ChenLiu-1996/CitationMap/issues/8).
|
|
121
|
+
|
|
122
|
+
[Jul 28, 2024] Version 3.10 released >>> Logic update. Tested on a professor's profile **with 10,000 citations**!
|
|
123
|
+
|
|
124
|
+
[Jul 27, 2024] Version 2.0 released >>> 10x speedup with multiprocessing (1 hour to 5 minutes for my profile).
|
|
125
|
+
|
|
126
|
+
[Jul 26, 2024] Version 1.0 released >>> First working version for my profile with 100 citations.
|
|
127
|
+
|
|
76
128
|
## User Guide
|
|
77
129
|
0. If you are new to Python, you probably want to start with a distribution of Python that also helps with environment management (such as [anaconda](https://www.anaconda.com/)). Once you have set up your environment (for example, when you reach the stage of `conda activate env39` in [this tutorial](https://www.youtube.com/watch?v=MUZtVEDKXsk&t=242s)), you can move on to the next step.
|
|
78
130
|
1. Install this tool by running the following line in your conda-accessible command line.
|
|
@@ -99,7 +151,7 @@ Liu, Chen. "CitationMap: A Python Tool to Identify and Visualize Your Google Sch
|
|
|
99
151
|
generate_citation_map(scholar_id)
|
|
100
152
|
```
|
|
101
153
|
|
|
102
|
-
Note that in Version 4.
|
|
154
|
+
Note that in Version 4.5, we support manual edit of the csv. You can run `generate_citation_map` first, and get the csv and HTML. Then, you can manually modify the csv content, and generate an updated HTML by running `generate_citation_map` again with `parse_csv=True`.
|
|
103
155
|
|
|
104
156
|
Note that in Version 4.0, we will cache the results before identifying affiliations. So if you want to rerun the same author from scratch, you need to delete the cache (default location is 'cache').
|
|
105
157
|
|
|
@@ -190,7 +242,7 @@ Liu, Chen. "CitationMap: A Python Tool to Identify and Visualize Your Google Sch
|
|
|
190
242
|
|
|
191
243
|
## Changelog
|
|
192
244
|
<details>
|
|
193
|
-
<summary>Version 4.
|
|
245
|
+
<summary>Version 4.5 (Dec 6, 2024)</summary>
|
|
194
246
|
<br>
|
|
195
247
|
|
|
196
248
|
1. **Support manual edit of the csv.** Note you do need to manually correct the coordinates too if you choose to use this functionality. You can run `generate_citation_map` first, and get the csv and HTML. Then, you can manually modify the csv content, and generate an updated HTML by running `generate_citation_map` again with `parse_csv=True`.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|