metacountregressor 0.1.76__tar.gz → 0.1.91__tar.gz

{metacountregressor-0.1.76 → metacountregressor-0.1.91}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: metacountregressor
-Version: 0.1.76
+Version: 0.1.91
 Summary: Extensions for a Python package for estimation of count models.
 Home-page: https://github.com/zahern/CountDataEstimation
 Author: Zeke Ahern
@@ -11,12 +11,18 @@ Description-Content-Type: text/markdown
 License-File: LICENSE.txt
 Requires-Dist: numpy>=1.13.1
 Requires-Dist: scipy>=1.0.0
+Requires-Dist: requests
 
 <div style="display: flex; align-items: center;">
-    <img src="https://github.com/zahern/data/raw/main/m.png" alt="My Image" style="width: 200px; margin-right: 20px;">
+    <img src="https://github.com/zahern/data/raw/main/m.png" alt="My Image" style="width: 100px; margin-right: 20px;">
     <p><span style="font-size: 60px;"><strong>MetaCountRegressor</strong></span></p>
 </div>
 
+# Tutorial also available as a jupyter notebook
+[Download Example Notebook](https://github.com/zahern/CountDataEstimation/blob/main/Tutorial.ipynb)
+
+The tutorial provides more extensive examples on how to run the code and perform experiments. Further documentation is currently in development.
+
 ##### Quick Setup
 The Below code demonstrates how to set up automatic optimization assisted by the harmony search algorithm. References to the Differential Evolution and Simulated Annealing has been mentioned (change accordingly)
 
@@ -35,8 +41,15 @@ from metacountregressor.solution import ObjectiveFunction
 from metacountregressor.metaheuristics import (harmony_search,
                                             differential_evolution,
                                             simulated_annealing)
+
+
 ```
 
+    loaded standard packages
+    loaded helper
+    testing
+    
+
 #### Basic setup. 
 The initial setup involves reading in the data and selecting an optimization algorithm. As the runtime progresses, new solutions will be continually evaluated. Finally, at the end of the runtime, the best solution will be identified and printed out. In the case of multiple objectives all of the best solutions will be printed out that belong to the Pareto frontier.
 
@@ -53,7 +66,7 @@ X = df.drop(columns=['FREQ', 'ID', 'AADT'])
 
 #some example argument, these are defualt so the following line is just for claritity. See the later agruments section for detials.
 arguments = {'algorithm': 'hs', 'test_percentage': 0.15, 'test_complexity': 6, 'instance_number':1,
-             'val_percentage':0.15, 'obj_1': 'bic', '_obj_2': 'RMSE_TEST', "MAX_TIME": 6}
+             'val_percentage':0.15, 'obj_1': 'bic', '_obj_2': 'RMSE_TEST', "_max_time": 6}
 # Fit the model with metacountregressor
 obj_fun = ObjectiveFunction(X, y, **arguments)
 #replace with other metaheuristics if desired
@@ -71,7 +84,7 @@ Note: Please Consider the main arguments to change.
 - `val_percentage`: This parameter represents the percentage of data used to validate the model. The value 0.15 corresponds to 15% of the data.
 - `test_complexity`: This parameter defines the complexity level for testing. The value 6 tests all complexities. Alternatively, you can provide a list of numbers to consider different complexities. The complexities are further explained later in this document.
 - `instance_number`: This parameter is used to give a name to the outputs.
-- `obj_1`: This parameter has multiple choices for obj_1, such as 'bic', 'aic', and 'hqic'. Only one choice should be defined as a string value.
+- `_obj_1`: This parameter has multiple choices for obj_1, such as 'bic', 'aic', and 'hqic'. Only one choice should be defined as a string value.
 - `_obj_2`: This parameter has multiple choices for objective 2, such as 'RMSE_TEST', 'MSE_TEST', and 'MAE_TEST'.
 - `_max_time`: This parameter specifies the maximum number of seconds for the total estimation before stopping.
 - `distribution`: This parameter is a list of distributions to consider. Please select all of the available options and put them into a list of valid options if you want to to consider the distribution type for use when modellign with random parameters. The valid options include: 'Normal', 'LnNormal', 'Triangular', and 'Uniform'.
@@ -80,7 +93,7 @@ Note: Please Consider the main arguments to change.
 
 
 
-### An Example of changing the arguments.
+### Example of changing the arguments:
 Modify the arguments according to your preferences using the commented code as a guide.
 
 
@@ -108,16 +121,18 @@ Listed below is an example of how to specify an initial solution within the fram
 
 
 ```python
- #Model Decisions, Specify for Intial Optimization
+ #Model Decisions, Specify for initial solution that will be optimised.
 manual_fit_spec = {
     'fixed_terms': ['SINGLE', 'LENGTH'],
     'rdm_terms': ['AADT:normal'],
-    'rdm_cor_terms': ['GRADEBR:uniform', 'CURVES:triangular'],
+    'rdm_cor_terms': ['GRADEBR:normal', 'CURVES:normal'],
     'grouped_terms': [],
     'hetro_in_means': ['ACCESS:normal', 'MINRAD:normal'],
     'transformations': ['no', 'no', 'log', 'no', 'no', 'no', 'no'],
-    'dispersion': 1
+    'dispersion': 0
 }
+
+
 #Search Arguments
 arguments = {
     'algorithm': 'hs',
@@ -129,7 +144,47 @@ arguments = {
 obj_fun = ObjectiveFunction(X, y, **arguments)
 ```
 
- simarly to return the results feed the objective function into a metaheuristic solution algorithm. An example of this is provided below:
+    Setup Complete...
+    Benchmaking test with Seed 42
+    --------------------------------------------------------------------------------
+    Log-Likelihood:  -1339.1862434675106
+    --------------------------------------------------------------------------------
+    bic: 2732.31
+    --------------------------------------------------------------------------------
+    MSE: 650856.32
+    +--------------------------+--------+-------+----------+----------+------------+
+    |          Effect          | $\tau$ | Coeff | Std. Err | z-values | Prob |z|>Z |
+    +==========================+========+=======+==========+==========+============+
+    | LENGTH                   | no     | -0.15 |   0.01   |  -12.98  | 0.00***    |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | SINGLE                   | no     | -2.46 |   0.04   |  -50.00  | 0.00***    |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | GRADEBR                  | log    | 4.23  |   0.10   |  42.17   | 0.00***    |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | CURVES                   | no     | 0.51  |   0.01   |  34.78   | 0.00***    |
+    +--------------------------+--------+-------+----------+----------+------------+
+    |  Chol: GRADEBR (Std.     |        | 2.21  |   0.00   |  50.00   | 0.00***    |
+    | Dev. normal) )           |        |       |          |          |            |
+    +--------------------------+--------+-------+----------+----------+------------+
+    |  Chol: CURVES (Std. Dev. |        | -0.51 |   0.00   |  -50.00  | 0.00***    |
+    | normal) )                |        |       |          |          |            |
+    +--------------------------+--------+-------+----------+----------+------------+
+    |  Chol: CURVES (Std. Dev. | no     | 0.55  |   0.00   |  50.00   | 0.00***    |
+    | normal) . GRADEBR (Std.  |        |       |          |          |            |
+    | Dev.   normal )          |        |       |          |          |            |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | main: MINRAD: hetro      | no     | -0.00 |   0.00   |  -44.36  | 0.00***    |
+    | group 0                  |        |       |          |          |            |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | ACCESS: hetro group 0    |        | 0.68  |   0.09   |   7.68   | 0.00***    |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | main: MINRAD: hetro      |        | -0.00 |   0.00   |  -44.86  | 0.00***    |
+    | group 0:normal:sd  hetro |        |       |          |          |            |
+    | group 0                  |        |       |          |          |            |
+    +--------------------------+--------+-------+----------+----------+------------+
+    
+
+ Simarly to return the results feed the objective function into a metaheuristic solution algorithm. An example of this is provided below:
 
 
 ```python
@@ -137,7 +192,7 @@ results = harmony_search(obj_fun)
 print(results)
 ```
 
-## Notes:
+# Notes:
 ### Capabilities of the software include:
 * Handling of Panel Data
 * Support for Data Transformations
@@ -155,11 +210,11 @@ Capability to handle heterogeneity in the means of the random parameters
 * Customization of Hyper-parameters to solve problems tailored to your dataset
 * Out-of-the-box optimization capability using default metaheuristics
 
-### Intreting the output of the model:
+### Intepreting the output of the model:
 A regression table is produced. The following text elements are explained:
 - Std. Dev.: This column appears for effects that are related to random paramters and displays the assument distributional assumption next to it
 - Chol: This term refers to Cholesky decomposition element, to show the correlation between two random paramaters. The combination of the cholesky element on iyself is equivalent to a normal random parameter.
-- hetro group #: This term represents the heterogeneity group number, which refers all of the contributing factors that share hetrogentiy in the means to each other under the same numbered value.
+- hetro group: This term represents the heterogeneity group number, which refers all of the contributing factors that share hetrogentiy in the means to each other under the same numbered value.
 - $\tau$: This column, displays the type of transformation that was applied to the specific contributing factor in the data.
 
 
@@ -211,10 +266,10 @@ The following list describes the arguments available in this function. By defaul
 
 8. **`_max_time`**: This argument is used to add a termination time in the algorithm. It takes values as seconds. Note the time is only dependenant on the time after intial population of solutions are generated.
 
-# Example
+## Example: Assistance by Harmony Search
 
 
-Let's start by fitting very simple models, use those model sto help and define the objectives, then perform more of an extensive search on the variables that are identified more commonly
+Let's begin by fitting very simple models and use the structure of these models to define our objectives. Then, we can conduct a more extensive search on the variables that are more frequently identified. For instance, in the case below, the complexity is level 3, indicating that we will consider, at most randomly correlated parameters. This approach is useful for initially identifying a suitable set of contributing factors for our search.
 
 
 
@@ -241,27 +296,30 @@ arguments = {
         '_max_time': 10000
     }
 obj_fun = ObjectiveFunction(X, y, **arguments)
-
 results = harmony_search(obj_fun)
 print(results)
 ```
 
+## Paper
+
+The following tutorial is in conjunction with our latest paper. A link the current paper can be found here [MetaCountRegressor](https://www.overleaf.com/read/mszwpwzcxsng#c5eb0c)
+
 ## Contact
 If you have any questions, ideas to improve MetaCountRegressor, or want to report a bug, just open a new issue in [GitHub repository](https://github.com/zahern/CountDataEstimation).
 
 ## Citing MetaCountRegressor
 Please cite MetaCountRegressor as follows:
 
-Ahern, Z., Corry P., Paz A. (2023). MetaCountRegressor [Computer software]. [https://pypi.org/project/metacounregressor/](https://pypi.org/project/metacounregressor/)
+Ahern, Z., Corry P., Paz A. (2024). MetaCountRegressor [Computer software]. [https://pypi.org/project/metacounregressor/](https://pypi.org/project/metacounregressor/)
 
 Or using BibTex as follows:
 
 ```bibtex
-@misc{Ahern2023,
-   author = {Zeke Ahern and Paul Corry and Alexander Paz},
+@misc{Ahern2024Meta,
+   author = {Zeke Ahern, Paul Corry and Alexander Paz},
    journal = {PyPi},
    title = {metacountregressor · PyPI},
-   url = {https://pypi.org/project/metacountregressor/0.1.47/},
-   year = {2023},
+   url = {https://pypi.org/project/metacountregressor/0.1.80/},
+   year = {2024},
 }
 

{metacountregressor-0.1.76 → metacountregressor-0.1.91}/README.rst

@@ -2,9 +2,18 @@
 
    ::
 
-      <img src="https://github.com/zahern/data/raw/main/m.png" alt="My Image" style="width: 200px; margin-right: 20px;">
+      <img src="https://github.com/zahern/data/raw/main/m.png" alt="My Image" style="width: 100px; margin-right: 20px;">
       <p><span style="font-size: 60px;"><strong>MetaCountRegressor</strong></span></p>
 
+Tutorial also available as a jupyter notebook
+=============================================
+
+`Download Example
+Notebook <https://github.com/zahern/CountDataEstimation/blob/main/README.ipynb>`__
+
+The tutorial provides more extensive examples on how to run the code and
+perform experiments. Further documentation is currently in development.
+
 Quick Setup
 '''''''''''
 
@@ -28,6 +37,16 @@ Install ``metacountregressor`` using pip as follows:
     from metacountregressor.metaheuristics import (harmony_search,
                                                 differential_evolution,
                                                 simulated_annealing)
+    
+    
+
+
+.. parsed-literal::
+
+    loaded standard packages
+    loaded helper
+    testing
+    
 
 Basic setup.
 ^^^^^^^^^^^^
@@ -52,7 +71,7 @@ the Pareto frontier.
     
     #some example argument, these are defualt so the following line is just for claritity. See the later agruments section for detials.
     arguments = {'algorithm': 'hs', 'test_percentage': 0.15, 'test_complexity': 6, 'instance_number':1,
-                 'val_percentage':0.15, 'obj_1': 'bic', '_obj_2': 'RMSE_TEST', "MAX_TIME": 6}
+                 'val_percentage':0.15, 'obj_1': 'bic', '_obj_2': 'RMSE_TEST', "_max_time": 6}
     # Fit the model with metacountregressor
     obj_fun = ObjectiveFunction(X, y, **arguments)
     #replace with other metaheuristics if desired
@@ -80,7 +99,7 @@ Note: Please Consider the main arguments to change.
    complexities are further explained later in this document.
 -  ``instance_number``: This parameter is used to give a name to the
    outputs.
--  ``obj_1``: This parameter has multiple choices for obj_1, such as
+-  ``_obj_1``: This parameter has multiple choices for obj_1, such as
    �bic�, �aic�, and �hqic�. Only one choice should be defined as a
    string value.
 -  ``_obj_2``: This parameter has multiple choices for objective 2, such
@@ -103,8 +122,8 @@ Note: Please Consider the main arguments to change.
    valid options include: �Normal�, �LnNormal�, �Triangular�, and
    �Uniform�.
 
-An Example of changing the arguments.
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Example of changing the arguments:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Modify the arguments according to your preferences using the commented
 code as a guide.
@@ -139,16 +158,18 @@ modeling components may completely replace the initial solution.
 
 .. code:: ipython3
 
-     #Model Decisions, Specify for Intial Optimization
+     #Model Decisions, Specify for initial solution that will be optimised.
     manual_fit_spec = {
         'fixed_terms': ['SINGLE', 'LENGTH'],
         'rdm_terms': ['AADT:normal'],
-        'rdm_cor_terms': ['GRADEBR:uniform', 'CURVES:triangular'],
+        'rdm_cor_terms': ['GRADEBR:normal', 'CURVES:normal'],
         'grouped_terms': [],
         'hetro_in_means': ['ACCESS:normal', 'MINRAD:normal'],
         'transformations': ['no', 'no', 'log', 'no', 'no', 'no', 'no'],
-        'dispersion': 1
+        'dispersion': 0
     }
+    
+    
     #Search Arguments
     arguments = {
         'algorithm': 'hs',
@@ -159,7 +180,50 @@ modeling components may completely replace the initial solution.
     }
     obj_fun = ObjectiveFunction(X, y, **arguments)
 
-simarly to return the results feed the objective function into a
+
+.. parsed-literal::
+
+    Setup Complete...
+    Benchmaking test with Seed 42
+    --------------------------------------------------------------------------------
+    Log-Likelihood:  -1339.1862434675106
+    --------------------------------------------------------------------------------
+    bic: 2732.31
+    --------------------------------------------------------------------------------
+    MSE: 650856.32
+    +--------------------------+--------+-------+----------+----------+------------+
+    |          Effect          | $\tau$ | Coeff | Std. Err | z-values | Prob |z|>Z |
+    +==========================+========+=======+==========+==========+============+
+    | LENGTH                   | no     | -0.15 |   0.01   |  -12.98  | 0.00***    |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | SINGLE                   | no     | -2.46 |   0.04   |  -50.00  | 0.00***    |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | GRADEBR                  | log    | 4.23  |   0.10   |  42.17   | 0.00***    |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | CURVES                   | no     | 0.51  |   0.01   |  34.78   | 0.00***    |
+    +--------------------------+--------+-------+----------+----------+------------+
+    |  Chol: GRADEBR (Std.     |        | 2.21  |   0.00   |  50.00   | 0.00***    |
+    | Dev. normal) )           |        |       |          |          |            |
+    +--------------------------+--------+-------+----------+----------+------------+
+    |  Chol: CURVES (Std. Dev. |        | -0.51 |   0.00   |  -50.00  | 0.00***    |
+    | normal) )                |        |       |          |          |            |
+    +--------------------------+--------+-------+----------+----------+------------+
+    |  Chol: CURVES (Std. Dev. | no     | 0.55  |   0.00   |  50.00   | 0.00***    |
+    | normal) . GRADEBR (Std.  |        |       |          |          |            |
+    | Dev.   normal )          |        |       |          |          |            |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | main: MINRAD: hetro      | no     | -0.00 |   0.00   |  -44.36  | 0.00***    |
+    | group 0                  |        |       |          |          |            |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | ACCESS: hetro group 0    |        | 0.68  |   0.09   |   7.68   | 0.00***    |
+    +--------------------------+--------+-------+----------+----------+------------+
+    | main: MINRAD: hetro      |        | -0.00 |   0.00   |  -44.86  | 0.00***    |
+    | group 0:normal:sd  hetro |        |       |          |          |            |
+    | group 0                  |        |       |          |          |            |
+    +--------------------------+--------+-------+----------+----------+------------+
+    
+
+Simarly to return the results feed the objective function into a
 metaheuristic solution algorithm. An example of this is provided below:
 
 .. code:: ipython3
@@ -168,7 +232,7 @@ metaheuristic solution algorithm. An example of this is provided below:
     print(results)
 
 Notes:
-------
+======
 
 Capabilities of the software include:
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -199,8 +263,8 @@ Capabilities of the software include:
    dataset
 -  Out-of-the-box optimization capability using default metaheuristics
 
-Intreting the output of the model:
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Intepreting the output of the model:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 A regression table is produced. The following text elements are
 explained: - Std. Dev.: This column appears for effects that are related
@@ -208,7 +272,7 @@ to random paramters and displays the assument distributional assumption
 next to it - Chol: This term refers to Cholesky decomposition element,
 to show the correlation between two random paramaters. The combination
 of the cholesky element on iyself is equivalent to a normal random
-parameter. - hetro group #: This term represents the heterogeneity group
+parameter. - hetro group: This term represents the heterogeneity group
 number, which refers all of the contributing factors that share
 hetrogentiy in the means to each other under the same numbered value. -
 :math:`\tau`: This column, displays the type of transformation that was
@@ -299,12 +363,16 @@ considered. Example code will be provided later in this guide.
    dependenant on the time after intial population of solutions are
    generated.
 
-Example
-=======
+Example: Assistance by Harmony Search
+-------------------------------------
 
-Let�s start by fitting very simple models, use those model sto help and
-define the objectives, then perform more of an extensive search on the
-variables that are identified more commonly
+Let�s begin by fitting very simple models and use the structure of these
+models to define our objectives. Then, we can conduct a more extensive
+search on the variables that are more frequently identified. For
+instance, in the case below, the complexity is level 3, indicating that
+we will consider, at most randomly correlated parameters. This approach
+is useful for initially identifying a suitable set of contributing
+factors for our search.
 
 .. code:: ipython3
 
@@ -330,10 +398,16 @@ variables that are identified more commonly
             '_max_time': 10000
         }
     obj_fun = ObjectiveFunction(X, y, **arguments)
-    
     results = harmony_search(obj_fun)
     print(results)
 
+Paper
+-----
+
+The following tutorial is in conjunction with our latest paper. A link
+the current paper can be found here
+`MetaCountRegressor <https://www.overleaf.com/read/mszwpwzcxsng#c5eb0c>`__
+
 Contact
 -------
 
@@ -346,12 +420,12 @@ Citing MetaCountRegressor
 
 Please cite MetaCountRegressor as follows:
 
-Ahern, Z., Corry P., Paz A. (2023). MetaCountRegressor [Computer
+Ahern, Z., Corry P., Paz A. (2024). MetaCountRegressor [Computer
 software]. https://pypi.org/project/metacounregressor/
 
 Or using BibTex as follows:
 
-\```bibtex @misc{Ahern2023, author = {Zeke Ahern and Paul Corry and
+\```bibtex @misc{Ahern2024Meta, author = {Zeke Ahern, Paul Corry and
 Alexander Paz}, journal = {PyPi}, title = {metacountregressor · PyPI},
-url = {https://pypi.org/project/metacountregressor/0.1.47/}, year =
-{2023}, }
+url = {https://pypi.org/project/metacountregressor/0.1.80/}, year =
+{2024}, }

{metacountregressor-0.1.76 → metacountregressor-0.1.91}/metacountregressor/main.py

@@ -29,6 +29,64 @@ def convert_df_columns_to_binary_and_wide(df):
 
 
 def main(args, **kwargs):
+    '''METACOUNT REGRESSOR TESTING ENVIRONMENT'''
+    import statsmodels.api as sm
+
+    data = sm.datasets.sunspots.load_pandas().data
+    # print(data.exog)
+    data_exog = data['YEAR']
+    data_exog = sm.add_constant(data_exog)
+    data_endog = data['SUNACTIVITY']
+
+    # Instantiate a gamma family model with the default link function.
+    import numpy as np
+
+    gamma_model = sm.NegativeBinomial(data_endog, data_exog)
+    gamma_results = gamma_model.fit()
+
+    print(gamma_results.summary())
+
+    # NOW LET's COMPARE THIS TO METACOUNT REGRESSOR
+    import metacountregressor
+    from importlib.metadata import version
+    print(version('metacountregressor'))
+    import pandas as pd
+    import numpy as np
+    from metacountregressor.solution import ObjectiveFunction
+    from metacountregressor.metaheuristics import (harmony_search,
+                                                   differential_evolution,
+                                                   simulated_annealing)
+
+    # Model Decisions,
+    manual_fit_spec = {
+
+        'fixed_terms': ['const', 'YEAR'],
+        'rdm_terms': [],
+        'rdm_cor_terms': [],
+        'grouped_terms': [],
+        'hetro_in_means': [],
+        'transformations': ['no', 'no'],
+        'dispersion': 1  # Negative Binomial
+    }
+
+    # Arguments
+    arguments = {
+        'algorithm': 'hs',
+        'test_percentage': 0,
+        'test_complexity': 6,
+        'instance_number': 'name',
+        'Manual_Fit': manual_fit_spec
+    }
+    obj_fun = ObjectiveFunction(data_exog, data_endog, **arguments)
+    #exit()
+
+
+
+
+
+
+
+
     print('the args is:', args)
     print('the kwargs is', kwargs)
 
@@ -109,6 +167,16 @@ def main(args, **kwargs):
             'transformations': ['no', 'no', 'no', 'no', 'no', 'no', 'no', 'no'],
             'dispersion': 1
         }
+        print('overriding this delete, just want to test the NB')
+        manual_fit_spec = {
+            'fixed_terms': ['const'],
+            'rdm_terms': [],
+            'rdm_cor_terms': [],
+            'grouped_terms': [],
+            'hetro_in_means': [],
+            'transformations': ['no'],
+            'dispersion': 1
+        }
 
         df = pd.read_csv('./data/Ex-16-3.csv')  # read in the data
         y_df = df[['FREQ']].copy()  # only consider crashes
@@ -118,7 +186,7 @@ def main(args, **kwargs):
         x_df['Offset'] = np.log(1 + x_df['AADT'] * x_df['LENGTH'] * 365 / 100000000)
         x_df = x_df.drop(columns=['AADT', 'LENGTH'])
 
-        if args['separate_out_factors']:
+        if args.get('seperate_out_factors', 0):
 
             x_df = helperprocess.as_wide_factor(x_df, keep_original=0,
                                                 exclude=['INTECHAG', 'CURVES', 'MIMEDSH', 'MXMEDSH', 'SPEED'])
@@ -173,8 +241,8 @@ def main(args, **kwargs):
             x_df = x_df.drop(columns = [ 'zonal_ID', 'ln_AADT', 'ln_seg'])
             x_df['rumble_install_year'] = x_df['rumble_install_year'].astype('category').cat.codes
             x_df.rename(columns={"rumble_install_year": "has_rumble"}, inplace=True)
-
-        except:
+        except Exception as e:
+            print(e)
             x_df = df.drop(columns=['Y'])  # was dropped postcode
 
         group_grab = x_df['county']
@@ -215,7 +283,6 @@ def main(args, **kwargs):
         else:
             print('fitting manually')
             args['Manual_Fit'] = manual_fit_spec
-
     if args['problem_number'] == str(8) or args['problem_number'] == 8:
         print('Maine County Dataset.')
         args['group'] = 'county'
@@ -346,10 +413,8 @@ if __name__ == '__main__':
         override = True
         if override:
             print('todo turn off, in testing phase')
-            parser.add_argument('-problem_number', default='8')
+            parser.add_argument('-problem_number', default='4')
             print('did it make it')
-
-
         if 'algorithm' not in args:
             parser.add_argument('-algorithm', type=str, default='hs',
                                 help='optimization algorithm')
@@ -370,7 +435,7 @@ if __name__ == '__main__':
     parser.print_help()
     args = vars(parser.parse_args())
     print(type(args))
-    # TODO add in chi 2 and df in estimation and compare degrees of freedom
+    # TODO add in chi 2 and df in estimation and compare degrees of freedom this needs to be done in solution
 
     # Print the args.
     profiler = cProfile.Profile()