noshot 1.0.0__py3-none-any.whl → 2.0.0__py3-none-any.whl

noshot/data/ML TS XAI/XAI/XAI 2/FeatureImportance_SensitivityAnalysis.ipynb

@@ -1,708 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "8TKef6Jkajof"
-   },
-   "source": [
-    "### Loading the modules"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    },
-    "id": "7KYFQmlqajof",
-    "outputId": "1a71ad6f-3091-4c5c-ec82-03cf1e1aab21"
-   },
-   "outputs": [],
-   "source": [
-    "!pip install --upgrade scipy\n",
-    "!pip install seaborn\n",
-    "import pandas as pd\n",
-    "import numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "\n",
-    "import warnings\n",
-    "warnings.filterwarnings(\"ignore\")\n",
-    "np.random.seed(5)\n",
-    "from sklearn import preprocessing\n",
-    "from sklearn.model_selection import train_test_split\n",
-    "from sklearn.ensemble import RandomForestClassifier as RFC\n",
-    "import tensorflow as tf\n",
-    "tf.get_logger().setLevel(40) # suppress deprecation messages\n",
-    "from tensorflow.keras.layers import Dense, Input, Embedding, Concatenate, Reshape, Dropout, Lambda\n",
-    "from tensorflow.keras.models import Model\n",
-    "from tensorflow.keras.utils import to_categorical"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "Wb_u2GSOajog"
-   },
-   "source": [
-    "### Loading the data"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 503
-    },
-    "id": "1CjwRgzkajog",
-    "outputId": "72d70ee8-aacc-45b3-e675-282d2181beca"
-   },
-   "outputs": [],
-   "source": [
-    "data  = pd.read_csv('/content/Titanic-Dataset.csv')\n",
-    "data.head()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    },
-    "id": "rLs-UAb8ajoh",
-    "outputId": "3a8ed330-414b-44c5-dd96-9bab35313207"
-   },
-   "outputs": [],
-   "source": [
-    "data.shape"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 387
-    },
-    "id": "whspZ3Reajoh",
-    "outputId": "7b7deede-2214-4107-c49e-3aceaf8a82cc"
-   },
-   "outputs": [],
-   "source": [
-    "data.hist(layout = (2,5), figsize=(15,8), color = 'r')\n",
-    "print('Data Distribution')"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 466
-    },
-    "id": "jI55ZqFMajoh",
-    "outputId": "c3e366a4-efcc-4d4c-e192-151bc70381bc"
-   },
-   "outputs": [],
-   "source": [
-    "import seaborn as sns\n",
-    "\n",
-    "print('This looks like a fairly imbalanced dataset')\n",
-    "sns.countplot(x=\"Survived\", data=data, palette=\"bwr\")\n",
-    "plt.show()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 178
-    },
-    "id": "sxB-fTkYajoh",
-    "outputId": "5f15942e-2b8e-4ba0-95ae-4e0850e2aecd"
-   },
-   "outputs": [],
-   "source": [
-    "data['Survived'].value_counts()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    },
-    "id": "7vlvwRCRajoh",
-    "outputId": "524e4854-4866-40e7-fb9e-06e1b25253ab"
-   },
-   "outputs": [],
-   "source": [
-    "print('Percentage of data belonging to class 1 is',int((268/768)*100))\n",
-    "print('Percentage of data belonging to class 0 is',int((500/768)*100))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "uY6hXRMyajoi"
-   },
-   "source": [
-    "### Null Check"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 460
-    },
-    "id": "Hxz-qLufajoi",
-    "outputId": "9ff24398-48d8-422a-87dd-da5f2a430b8f"
-   },
-   "outputs": [],
-   "source": [
-    "data.isnull().sum()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "HpavXNQeajoi"
-   },
-   "source": [
-    "### Duplication Check"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    },
-    "id": "xxoDvfGGajoi",
-    "outputId": "c2461cac-f676-47d5-e41f-742a64243869"
-   },
-   "outputs": [],
-   "source": [
-    "data.duplicated().any()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "rvyds736ajoi"
-   },
-   "source": [
-    "### Data Description"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 320
-    },
-    "id": "Lpco8HJTajoj",
-    "outputId": "22c7389c-0614-4c10-bdc3-2e32e3c721a8"
-   },
-   "outputs": [],
-   "source": [
-    "data.describe()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "hzy-s1qYajoj"
-   },
-   "source": [
-    "### Data Correlation"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 505
-    },
-    "id": "RCdfuJhjajoj",
-    "outputId": "86716729-98d5-4183-c4f6-00194fb98287"
-   },
-   "outputs": [],
-   "source": [
-    "numerical_data = data.select_dtypes(include=np.number)\n",
-    "correlation_matrix = numerical_data.corr()\n",
-    "\n",
-    "    # Now you can visualize the correlation matrix using a heatmap if you want:\n",
-    "sns.heatmap(correlation_matrix, annot=True, cmap='coolwarm')\n",
-    "plt.show()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "pevqOcfVajoj"
-   },
-   "source": [
-    "### Outlier Check"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 73
-    },
-    "id": "9ETrhB9iajok",
-    "outputId": "84cd8c89-6315-4437-9477-7009d08bfbbe"
-   },
-   "outputs": [],
-   "source": [
-    "data[(data['PassengerId'] == 0) & (data['Survived'] == 0) & (data['Pclass'] == 0)]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 877
-    },
-    "id": "WQR6YylPajok",
-    "outputId": "6114eb59-63fc-4fcc-830d-b50bb70dd21b"
-   },
-   "outputs": [],
-   "source": [
-    "data[(data['Survived'] == 0)]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "NBQF8fhuajok"
-   },
-   "source": [
-    "### Noise removal"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    },
-    "id": "aZ9yq8w8ajok",
-    "outputId": "d4111ad5-7a99-4a94-af64-b3f64809dbef"
-   },
-   "outputs": [],
-   "source": [
-    "cleaned_data = data[(data['Survived'] != 0)]\n",
-    "cleaned_data.shape"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "MOJp9AuIajok"
-   },
-   "source": [
-    "### Feature Engineering"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    },
-    "id": "wV0BlJJ1ajol",
-    "outputId": "9a23ce1e-efda-4f32-fe34-f490ad3c437f"
-   },
-   "outputs": [],
-   "source": [
-    "feature_engg_data = cleaned_data.copy()\n",
-    "outlier_data = cleaned_data.copy()\n",
-    "factor = 3\n",
-    "\n",
-    "\n",
-    "columns_to_include = ['PassengerId'\t,'Survived',\t'Pclass','Age'\t,'SibSp',\t'Parch'\t,'Fare']\n",
-    "for column in columns_to_include:\n",
-    "    upper_lim = feature_engg_data[column].mean () + feature_engg_data[column].std () * factor\n",
-    "    lower_lim = feature_engg_data[column].mean () - feature_engg_data[column].std () * factor\n",
-    "    feature_engg_data = feature_engg_data[(feature_engg_data[column] < upper_lim) & (feature_engg_data[column] > lower_lim)]\n",
-    "\n",
-    "outlier_data = pd.concat([outlier_data, feature_engg_data]).drop_duplicates(keep=False)\n",
-    "\n",
-    "print(feature_engg_data.shape)\n",
-    "print(outlier_data.shape)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "03ebxkHzajol"
-   },
-   "source": [
-    "### Normalization"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 425
-    },
-    "id": "u6o2xRP3ajol",
-    "outputId": "14ede3fb-2e3d-4338-b782-a5f5d98aac6a"
-   },
-   "outputs": [],
-   "source": [
-    "\n",
-    "\n",
-    "factor = 2  # Reduced factor\n",
-    "def normalize_data(df):\n",
-    "    val = df.values\n",
-    "    min_max_normalizer = preprocessing.MinMaxScaler()\n",
-    "    norm_val = min_max_normalizer.fit_transform(val)\n",
-    "    df2 = pd.DataFrame(norm_val, columns=df.columns)\n",
-    "\n",
-    "    if df.empty:\n",
-    "        print(\"DataFrame is empty. Skipping normalization.\")\n",
-    "        return df\n",
-    "    else:\n",
-    "      print('not right ')\n",
-    "\n",
-    "norm_feature_engg_data = normalize_data(feature_engg_data)\n",
-    "norm_outlier_data = normalize_data(outlier_data)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "6PbwZdYFajom"
-   },
-   "source": [
-    "### Train-Test split"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 245
-    },
-    "id": "xF_ng_jUajom",
-    "outputId": "9004a6d9-76ff-4bfc-99a8-33f7cad9fc95"
-   },
-   "outputs": [],
-   "source": [
-    "input_data = norm_feature_engg_data.drop(['Survived'],axis='columns')\n",
-    "targets =norm_feature_engg_data.filter(['Survived'],axis='columns')\n",
-    "\n",
-    "x, x_test, y, y_test = train_test_split(input_data,targets,test_size=0.1,train_size=0.9, random_state=5)\n",
-    "x_train, x_valid, y_train, y_valid = train_test_split(x,y,test_size = 0.22,train_size =0.78, random_state=5)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "W72xPVnTajom"
-   },
-   "outputs": [],
-   "source": [
-    "def apply_RFC(X,y,columns):\n",
-    "    rfc = RFC(n_estimators=500,min_samples_leaf=round(len(X)*.01),random_state=5,n_jobs=-1)\n",
-    "    imp_features = rfc.fit(X,y).feature_importances_\n",
-    "    imp_features = pd.DataFrame(imp_features,columns=['Feature Importance'],index=columns)\n",
-    "    imp_features.sort_values(by=['Feature Importance'],inplace=True,ascending=False)\n",
-    "    imp_features['Moving Sum'] = imp_features['Feature Importance'].cumsum()\n",
-    "    imp_features = imp_features[imp_features['Moving Sum']<=0.95]\n",
-    "    top_features = imp_features.index.tolist()\n",
-    "    return imp_features, top_features"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 228
-    },
-    "id": "3v7w3rA0ajon",
-    "outputId": "6800cd0b-ef07-4e39-bd61-e1cc79fab7f1"
-   },
-   "outputs": [],
-   "source": [
-    "important_features, top_features = apply_RFC(x,y, data.columns.drop('Survived'))\n",
-    "sns.barplot(important_features['Feature Importance'], important_features.index, palette = 'tab10')\n",
-    "plt.title('Random Forest Feature Importance for: '+\"Titanic Dataset\")\n",
-    "plt.show()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 332
-    },
-    "id": "yCoEW-SHajon",
-    "outputId": "165910d8-df92-4d74-e467-d55b1e4a2f35"
-   },
-   "outputs": [],
-   "source": [
-    "# For this we need a trained model. So, let's train a model first, may be with a neural network architecture.\n",
-    "\n",
-    "def model():\n",
-    "    '''\n",
-    "    Simple 3 layered Neural Network model for binary classification\n",
-    "    '''\n",
-    "    inp = Input(shape=(x_train.shape[1],))\n",
-    "    x = Dense(40, activation='relu')(inp)\n",
-    "    x = Dense(40, activation='relu')(x)\n",
-    "    op = Dense(2, activation='softmax')(x)\n",
-    "    model = Model(inputs=inp, outputs=op)\n",
-    "    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
-    "    return model\n",
-    "\n",
-    "model = model()\n",
-    "model.fit(x_train, to_categorical(y_train), batch_size=64, epochs=300, verbose=0)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "fh231xMWajon",
-    "outputId": "e7a32bce-74d8-48f1-cdf0-b95cef2fdceb"
-   },
-   "outputs": [],
-   "source": [
-    "# Evaluate the trained model\n",
-    "model.evaluate(x_test, to_categorical(y_test))[1]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "ta_aHQFtajoo"
-   },
-   "source": [
-    "Although we are not concerned about the final model accuracy, but we do have a decent model to try sensitivity analysis on. Next, we will take a query instance to perform the 6-σ (six sigma) variation rule for Sensitivity analysis on the query instance."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "7gQXI06jajoo",
-    "outputId": "c4f28a3e-dc44-4ec9-de0b-be5ce552225e"
-   },
-   "outputs": [],
-   "source": [
-    "query_instance = x_test.iloc[5].values.reshape((1,) + x_test.iloc[5].shape)\n",
-    "print(\"Let's take a look at the normalized query data instance in which all the features are in the range of (0.0 - 1.0):\" )\n",
-    "df_query = pd.DataFrame(query_instance, columns = input_data.columns)\n",
-    "df_query"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "jqwBNsIyajop",
-    "outputId": "63d294b2-03ad-4495-b294-d6ccf2dd45f6"
-   },
-   "outputs": [],
-   "source": [
-    "predicted_outcome = np.argmax(model.predict(query_instance))\n",
-    "true_label = int(y_test.iloc[5][0])\n",
-    "print(f\" The true label is : {true_label}\")\n",
-    "print(f\" The predicted outcome is : {predicted_outcome}\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "QwdpQRCIajop"
-   },
-   "source": [
-    "We can clearly see the model is correctly predicting the presence of diabetes. Now, let's see if it changes when we are doing sensitivity analysis, one by one for all the features."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "urfPmdM0ajop"
-   },
-   "source": [
-    "The measure for standard deviation(σ) can be calculated on the nomalized training data as we will be using the normalized data for the prediction part."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "CriOKzCIajoq"
-   },
-   "outputs": [],
-   "source": [
-    "sigma_glucose = np.std(x['Glucose'])\n",
-    "sigma_bmi = np.std(x['BMI'])\n",
-    "sigma_age = np.std(x['Age'])\n",
-    "sigma_dpf = np.std(x['DiabetesPedigreeFunction'])\n",
-    "sigma_pregnancies = np.std(x['Pregnancies'])\n",
-    "sigma_insulin = np.std(x['Insulin'])\n",
-    "sigma_bp = np.std(x['BloodPressure'])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "XHm6I6SPajoq",
-    "outputId": "c72b595c-3094-462d-899a-7ffbfec95a9a"
-   },
-   "outputs": [],
-   "source": [
-    "# Let's see the sensitivity analysis plots now\n",
-    "def sensitivity_analysis_plot(measure_tuple): #the function takes one argument measure_tuple,which has features (glucose,BMI),and std deviation\n",
-    "    '''\n",
-    "    Sensitivity Analysis plot using the 6-σ variation method\n",
-    "    '''\n",
-    "    (measure, sigma) = measure_tuple\n",
-    "\n",
-    "    sensitivity_output = [] #intialize a empty list\n",
-    "    original_value = df_query[measure].copy() #the original value is copied here\n",
-    "    for k in [-3, -2, -1, 1, 2, 3]:\n",
-    "        df_query[measure] = original_value.copy()\n",
-    "        df_query[measure] = np.clip(df_query[measure] + k * sigma, 0.0, 1.0)\n",
-    "        sensitivity_output.append(np.argmax(model.predict(df_query.values)))#the most likely class\n",
-    "    plt.plot(['-3σ', '-2σ', '-σ', 'σ', '2σ', '3σ'], sensitivity_output, 'r.-', label = 'Sensitivity output')\n",
-    "    plt.axhline(y = predicted_outcome, color = 'b', linestyle = '--', label = 'Original Prediction')\n",
-    "    plt.title(f'6-σ variation sensitity plot for the feature: {measure}')\n",
-    "    plt.legend()\n",
-    "    plt.show()\n",
-    "\n",
-    "measure_tuple_list = [('Glucose', sigma_glucose),\n",
-    "                     ('BMI', sigma_bmi),\n",
-    "                     ('Age', sigma_age),\n",
-    "                     ('DiabetesPedigreeFunction', sigma_dpf),\n",
-    "                     ('Pregnancies', sigma_pregnancies),\n",
-    "                     ('Insulin', sigma_insulin),\n",
-    "                     ('BloodPressure', sigma_bp)]\n",
-    "\n",
-    "for measure_tuple in measure_tuple_list:\n",
-    "    sensitivity_analysis_plot(measure_tuple)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "6UOYwVPIajoq"
-   },
-   "source": [
-    "From the above plots, we observe how each of the features are sensitive towards positive or negative changes and how each feature contributes towards influencing the model outcome.The features about Insulin, Diabetes Pedigree Function and Number of Preganancies doesn't seem to be sensitive towards any changes. The features giving information about Glucose, BMI, Blood Pressure seems to positive influence towards the outcome. That means, if the values for these features are increased, it may lead to the presence of diabetes according to the model. SUrprisingly, the feature Age shows a negative influence, which means if the age is increased, the model is less sensitive towards predicting the outcome as diabetes. This is contradicting our prior knowledge and hence is quite an interesting observation and needs to be inspected further."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "Tkw0-qLIajoq"
-   },
-   "source": [
-    "### Final Thoughts"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "6l9KPGqDajor"
-   },
-   "source": [
-    "We have seen how influence based methods like feature importance and sensitivity analysis can be applied to explain the influence of features towards the model's decision making process. But I have only show examples related to classification problem. I would strongly recommend you to try out these methods for explaining models used for regression based problems as well."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "F-NW-O3Eajor"
-   },
-   "source": [
-    "### Reference"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "sihXlZQhajor"
-   },
-   "source": [
-    "1. Kaggle | Pima Indians Diabetes Database - https://www.kaggle.com/uciml/pima-indians-diabetes-database?select=diabetes.csv\n",
-    "2.  How to Calculate Feature Importance With Python | Machine Learning Mastery - https://machinelearningmastery.com/calculate-feature-importance-with-python/\n",
-    "3. Some of the utility functions and code are taken from the GitHub Repository of the author - Aditya Bhattacharya https://github.com/adib0073"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "4IIeXhSIajor"
-   },
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "colab": {
-   "provenance": []
-  },
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.12.4"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}