segindex 0.1.3__tar.gz

segindex-0.1.3/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Acheul
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

segindex-0.1.3/PKG-INFO

@@ -0,0 +1,69 @@
+Metadata-Version: 2.1
+Name: segindex
+Version: 0.1.3
+Summary: Segregation Index
+Author-email: Jang Hyomin <seoul605.21@gmail.com>
+Project-URL: Homepage, https://github.com/acheul/reardon-segregation-index
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Sociology
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE.txt
+Requires-Dist: statsmodels>=0.14.1
+Requires-Dist: numpy>=1.18.5
+Requires-Dist: pandas>=2.1.3
+Requires-Dist: scipy>=1.7.0
+
+# Reardon's Segregation Index for Continuous Variables
+
+* [Repository](""https://github.com/acheul/reardon-segregation-index")
+
+## Use
+  ```python
+  ! pip install segindex==0.1.3
+  from segindex import estimate_Hp
+
+  # Say, each variables are income.
+  # Inner lists of each area stand for sectors.
+  # Each area has three sectors in this case.
+  # How much is each area segregated by sectors in terms of income?
+  area1 = [[80, 80, 70, 70], [50, 45, 40],[20, 20, 20, 10]]
+  area2 = [[80, 70, 50], [80, 70, 45, 20, 20], [40, 20, 10]]
+
+  print(estimate_Hp(area1))
+  print(estimate_Hp(area2))
+
+  >> 0.7182    
+  >> 0.3191
+  # area1 is more income-way segregated than area2. In other words, area2 is more mixed.
+  ```
+
+## Description
+
+* Many kinds of segregation index are used for various purposes like from policies to studies. While there are a wide range of categorical variables like race group to meausre an amount of segregation, continuous values like income are also important but do not fit very well with categorical segregation index.
+
+* [Reardon(2011)](https://cepa.stanford.edu/sites/default/files/reardon%20&%20bischoff%20income%20inequality%20segregation%20AJS%20final.pdf), [Reardon and Bischoff(2011)](https://cepa.stanford.edu/sites/default/files/reardon%20&%20bischoff%20income%20inequality%20segregation%20AJS%20final.pdf) propsed a rank-order segregation index based on Theil index which is based on the concept of Entropy. This index is widely accepted for practical and academic uses to calculate continuous value based segregation index like in [Chetty et al. 2014](https://www.nber.org/system/files/working_papers/w19843/w19843.pdf).
+
+* The proposed method of them is a bit intricate however and there seems to be no good online library or code that implements it. Therefore, here is one. Python codes inside [```src/segdex/segregation_index.py```](./src/segdex/segregation_index.py) implements Rank-Order Information Theory Index of Reardon(2011).
+
+* Essentials of the Index
+
+  * The inequality index *H* is an average of each value from a total of K sectors, which is total region's entropy(*E*) minus each sector's entropy(*E_K*). It is weighted by each sector's relative popultaion size(*t_k/T*). Here the entropy stands for how equally variables(ex. income) are distributed over sectors.
+
+    <img src="./assets/eq3-5.png">
+  
+  * Below is an equation to calculate entropy when there is two groups. *p* is a ratio of a group. As the variable here is continuous not categorical, one needs to integrate the below equation over *p* with a range of 0≤p≤1. Thus transformation of raw values into rank ordered values is required.
+  
+    <img src="./assets/eq3-6.png">
+
+  * Combining above equations, we can calculate below one to get a Rank-Order Information Theory Index, which is the segregation index for continuous variables. 0 means perfect equality. 1 means perfect segregation.
+  
+    <img src="./assets/eq3-7.png">

segindex-0.1.3/README.md

@@ -0,0 +1,45 @@
+# Reardon's Segregation Index for Continuous Variables
+
+* [Repository](""https://github.com/acheul/reardon-segregation-index")
+
+## Use
+  ```python
+  ! pip install segindex==0.1.3
+  from segindex import estimate_Hp
+
+  # Say, each variables are income.
+  # Inner lists of each area stand for sectors.
+  # Each area has three sectors in this case.
+  # How much is each area segregated by sectors in terms of income?
+  area1 = [[80, 80, 70, 70], [50, 45, 40],[20, 20, 20, 10]]
+  area2 = [[80, 70, 50], [80, 70, 45, 20, 20], [40, 20, 10]]
+
+  print(estimate_Hp(area1))
+  print(estimate_Hp(area2))
+
+  >> 0.7182    
+  >> 0.3191
+  # area1 is more income-way segregated than area2. In other words, area2 is more mixed.
+  ```
+
+## Description
+
+* Many kinds of segregation index are used for various purposes like from policies to studies. While there are a wide range of categorical variables like race group to meausre an amount of segregation, continuous values like income are also important but do not fit very well with categorical segregation index.
+
+* [Reardon(2011)](https://cepa.stanford.edu/sites/default/files/reardon%20&%20bischoff%20income%20inequality%20segregation%20AJS%20final.pdf), [Reardon and Bischoff(2011)](https://cepa.stanford.edu/sites/default/files/reardon%20&%20bischoff%20income%20inequality%20segregation%20AJS%20final.pdf) propsed a rank-order segregation index based on Theil index which is based on the concept of Entropy. This index is widely accepted for practical and academic uses to calculate continuous value based segregation index like in [Chetty et al. 2014](https://www.nber.org/system/files/working_papers/w19843/w19843.pdf).
+
+* The proposed method of them is a bit intricate however and there seems to be no good online library or code that implements it. Therefore, here is one. Python codes inside [```src/segdex/segregation_index.py```](./src/segdex/segregation_index.py) implements Rank-Order Information Theory Index of Reardon(2011).
+
+* Essentials of the Index
+
+  * The inequality index *H* is an average of each value from a total of K sectors, which is total region's entropy(*E*) minus each sector's entropy(*E_K*). It is weighted by each sector's relative popultaion size(*t_k/T*). Here the entropy stands for how equally variables(ex. income) are distributed over sectors.
+
+    <img src="./assets/eq3-5.png">
+  
+  * Below is an equation to calculate entropy when there is two groups. *p* is a ratio of a group. As the variable here is continuous not categorical, one needs to integrate the below equation over *p* with a range of 0≤p≤1. Thus transformation of raw values into rank ordered values is required.
+  
+    <img src="./assets/eq3-6.png">
+
+  * Combining above equations, we can calculate below one to get a Rank-Order Information Theory Index, which is the segregation index for continuous variables. 0 means perfect equality. 1 means perfect segregation.
+  
+    <img src="./assets/eq3-7.png">

segindex-0.1.3/pyproject.toml

@@ -0,0 +1,36 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "segindex"
+version = "0.1.3"
+authors = [
+  { name="Jang Hyomin", email="seoul605.21@gmail.com" },
+]
+description = "Segregation Index"
+readme = "README.md"
+requires-python = ">=3.8"
+
+dependencies = [
+  'statsmodels>=0.14.1',
+  'numpy >= 1.18.5',
+  'pandas>=2.1.3',
+  'scipy >= 1.7.0',
+]
+
+classifiers = [
+  'Development Status :: 4 - Beta',
+  'Environment :: Console',
+  'Intended Audience :: Science/Research',
+  'Operating System :: OS Independent',
+  'Programming Language :: Python :: 3.8',
+  'Programming Language :: Python :: 3.9',
+  'Programming Language :: Python :: 3.10',
+  'Programming Language :: Python :: 3.11',
+  'Programming Language :: Python :: 3 :: Only',
+  'Topic :: Sociology'
+]
+
+[project.urls]
+Homepage = "https://github.com/acheul/reardon-segregation-index"

segindex-0.1.3/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

segindex-0.1.3/src/segindex/__init__.py

@@ -0,0 +1 @@
+from .segregation_index import estimate_Hp

segindex-0.1.3/src/segindex/segregation_index.py

@@ -0,0 +1,114 @@
+# Implementing Segregation Index based on Theil Index
+# References: Reardon(2011), Reardon&Bischoff(2011)(Appendix A)
+
+import statsmodels.api as sm
+import statsmodels.formula.api as smf
+from scipy.stats import rankdata
+import math
+import numpy as np
+import pandas as pd
+from typing import List, Any
+
+
+def delta_m(m):
+    S = 0
+    for n in range(m+1):
+        s = ((-1)**(m-n))*math.comb(m,n)
+        s = s/(m-n+2)**2
+        S += s
+    d = (2/(m+2)**2) + 2*S
+    return d
+
+def entropy_p(p):
+    if p==0: p+=1e-6
+    elif p==1: p-=1e-6
+    e = -(p*np.log2(p) + (1-p)*np.log2(1-p))
+    return e
+
+def Theil_p(tt,pp):
+    tt,pp = map(np.array, [tt,pp])
+    T = np.sum(tt)
+    P = np.sum(tt*pp)/T
+    e_P = entropy_p(P)
+    e_pp = np.array([entropy_p(p) for p in pp])
+
+    H = 1 - np.sum(tt*e_pp)/(T*e_P)
+    return H
+
+def get_weighted_rank(ss,ww):
+    ss,ww = map(np.array, [ss,ww])
+    ss_ = ss*ww
+    rr = rankdata(ss_)
+    # ~1 normalization (X)
+    # rr = rr/len(rr)
+    return rr
+
+def from_vvs_to_rrs(vv_s, ww_s=None):
+    if ww_s==None:
+        tt = [len(e) for e in vv_s]
+        vvs = []
+        for vv in vv_s:
+            vvs.extend(vv)
+        rrs = rankdata(vvs)
+        ttc = np.cumsum(tt)
+        rr_s = [rrs[:ttc[0]]]
+        for t1,t2 in zip(ttc[:-1],ttc[1:]):
+            rr_s.append(rrs[t1:t2])
+        rr_s = np.array(rr_s, dtype=object)
+        rr_s = rr_s/len(rrs)
+        return rr_s
+    else:
+        tt = [len(e) for e in vv_s]
+        vvs, wws = [],[]
+        for vv,ww in zip(vv_s,ww_s):
+            vvs.extend(vv)
+            wws.extend(ww)
+        rrs = get_weighted_rank(vvs,wws)
+        ttc = np.cumsum(tt)
+        rr_s = [rrs[:ttc[0]]]
+        for t1,t2 in zip(ttc[:-1],ttc[1:]):
+            rr_s.append(rrs[t1:t2])
+        rr_s = np.array(rr_s, dtype=object)
+        rr_s = rr_s/len(rrs)
+        return rr_s
+    
+def estimate_Hp(vv_s: List[List[Any]], ww_s=None, K=14, m=4):
+    '''
+    Arguments:
+      vv_s (List[List[values]]): list of lists of values
+      ww_s: weights over vv_s
+      K: counts of thresholds
+      m: degree of regression equation used for the estimation
+    '''
+    vv_s = from_vvs_to_rrs(vv_s, ww_s=ww_s) # rank-order transformation
+    kk = np.array([e/K for e in range(1,K)]) # 1/K ~ (K-1)/K
+    e_kk = np.array([entropy_p(k) for k in kk])
+    w_kk = e_kk**2
+    #
+    tt = [len(vv) for vv in vv_s]
+    Hp_kk = []
+    for k in kk:
+        pp = []
+        for vv in vv_s:
+            vv_k = [e for e in vv if e<=k]
+            pp.append(len(vv_k)/len(vv))
+        Hp_k = Theil_p(tt,pp)
+        Hp_kk.append(Hp_k)
+    #
+    d = pd.DataFrame({'Hp':Hp_kk, 'p':kk, 'weight': w_kk})
+    for n in range(2,m+1):
+        d[f"p_{n}"] = d['p']**n
+    exog = d[[c for c in d.columns if 'p' in c and 'Hp' not in c]]
+    exog = sm.add_constant(exog)
+    global f
+    f = sm.WLS(endog=d['Hp'], exog=exog, weights=d['weight']).fit()
+    #
+    betas = [f.params['const'], f.params['p']]
+    for n in range(2,m+1):
+        betas.append(f.params[f"p_{n}"])
+    #
+    deltas = [delta_m(e) for e in range(m+1)]
+    betas, deltas = map(np.array, [betas, deltas])
+    #
+    H_r = np.sum(betas*deltas)
+    return H_r

segindex-0.1.3/src/segindex.egg-info/PKG-INFO

@@ -0,0 +1,69 @@
+Metadata-Version: 2.1
+Name: segindex
+Version: 0.1.3
+Summary: Segregation Index
+Author-email: Jang Hyomin <seoul605.21@gmail.com>
+Project-URL: Homepage, https://github.com/acheul/reardon-segregation-index
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Sociology
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE.txt
+Requires-Dist: statsmodels>=0.14.1
+Requires-Dist: numpy>=1.18.5
+Requires-Dist: pandas>=2.1.3
+Requires-Dist: scipy>=1.7.0
+
+# Reardon's Segregation Index for Continuous Variables
+
+* [Repository](""https://github.com/acheul/reardon-segregation-index")
+
+## Use
+  ```python
+  ! pip install segindex==0.1.3
+  from segindex import estimate_Hp
+
+  # Say, each variables are income.
+  # Inner lists of each area stand for sectors.
+  # Each area has three sectors in this case.
+  # How much is each area segregated by sectors in terms of income?
+  area1 = [[80, 80, 70, 70], [50, 45, 40],[20, 20, 20, 10]]
+  area2 = [[80, 70, 50], [80, 70, 45, 20, 20], [40, 20, 10]]
+
+  print(estimate_Hp(area1))
+  print(estimate_Hp(area2))
+
+  >> 0.7182    
+  >> 0.3191
+  # area1 is more income-way segregated than area2. In other words, area2 is more mixed.
+  ```
+
+## Description
+
+* Many kinds of segregation index are used for various purposes like from policies to studies. While there are a wide range of categorical variables like race group to meausre an amount of segregation, continuous values like income are also important but do not fit very well with categorical segregation index.
+
+* [Reardon(2011)](https://cepa.stanford.edu/sites/default/files/reardon%20&%20bischoff%20income%20inequality%20segregation%20AJS%20final.pdf), [Reardon and Bischoff(2011)](https://cepa.stanford.edu/sites/default/files/reardon%20&%20bischoff%20income%20inequality%20segregation%20AJS%20final.pdf) propsed a rank-order segregation index based on Theil index which is based on the concept of Entropy. This index is widely accepted for practical and academic uses to calculate continuous value based segregation index like in [Chetty et al. 2014](https://www.nber.org/system/files/working_papers/w19843/w19843.pdf).
+
+* The proposed method of them is a bit intricate however and there seems to be no good online library or code that implements it. Therefore, here is one. Python codes inside [```src/segdex/segregation_index.py```](./src/segdex/segregation_index.py) implements Rank-Order Information Theory Index of Reardon(2011).
+
+* Essentials of the Index
+
+  * The inequality index *H* is an average of each value from a total of K sectors, which is total region's entropy(*E*) minus each sector's entropy(*E_K*). It is weighted by each sector's relative popultaion size(*t_k/T*). Here the entropy stands for how equally variables(ex. income) are distributed over sectors.
+
+    <img src="./assets/eq3-5.png">
+  
+  * Below is an equation to calculate entropy when there is two groups. *p* is a ratio of a group. As the variable here is continuous not categorical, one needs to integrate the below equation over *p* with a range of 0≤p≤1. Thus transformation of raw values into rank ordered values is required.
+  
+    <img src="./assets/eq3-6.png">
+
+  * Combining above equations, we can calculate below one to get a Rank-Order Information Theory Index, which is the segregation index for continuous variables. 0 means perfect equality. 1 means perfect segregation.
+  
+    <img src="./assets/eq3-7.png">

segindex-0.1.3/src/segindex.egg-info/SOURCES.txt

@@ -0,0 +1,11 @@
+LICENSE.txt
+README.md
+pyproject.toml
+src/segindex/__init__.py
+src/segindex/segregation_index.py
+src/segindex.egg-info/PKG-INFO
+src/segindex.egg-info/SOURCES.txt
+src/segindex.egg-info/dependency_links.txt
+src/segindex.egg-info/requires.txt
+src/segindex.egg-info/top_level.txt
+src/tests/test.py

segindex-0.1.3/src/segindex.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

segindex-0.1.3/src/segindex.egg-info/requires.txt

@@ -0,0 +1,4 @@
+statsmodels>=0.14.1
+numpy>=1.18.5
+pandas>=2.1.3
+scipy>=1.7.0

segindex-0.1.3/src/segindex.egg-info/top_level.txt

@@ -0,0 +1,2 @@
+segindex
+tests

segindex-0.1.3/src/tests/test.py

@@ -0,0 +1,12 @@
+import sys, os
+sys.path.append(os.path.dirname("../src"))
+
+from segindex import estimate_Hp
+
+if __name__ == "__main__":
+
+  areas1 = [[80, 80, 70, 70], [50, 45, 40], [20, 20, 20, 10]]
+  areas2 = [[80, 70, 50], [80, 70, 45, 20, 20], [40, 20, 10]]
+
+  print(estimate_Hp(areas1))
+  print(estimate_Hp(areas2))