@synsci/cli-darwin-x64-baseline 1.1.76 → 1.1.78

package/bin/skills/scikit-survival/references/competing-risks.md

@@ -0,0 +1,397 @@
+# Competing Risks Analysis
+
+## Overview
+
+Competing risks occur when subjects can experience one of several mutually exclusive events (event types). When one event occurs, it prevents ("competes with") the occurrence of other events.
+
+### Examples of Competing Risks
+
+**Medical Research:**
+- Death from cancer vs. death from cardiovascular disease vs. death from other causes
+- Relapse vs. death without relapse in cancer studies
+- Different types of infections in transplant patients
+
+**Other Applications:**
+- Job termination: retirement vs. resignation vs. termination for cause
+- Equipment failure: different failure modes
+- Customer churn: different reasons for leaving
+
+### Key Concept: Cumulative Incidence Function (CIF)
+
+The **Cumulative Incidence Function (CIF)** represents the probability of experiencing a specific event type by time *t*, accounting for the presence of competing risks.
+
+**CIF_k(t) = P(T ≤ t, event type = k)**
+
+This differs from the Kaplan-Meier estimator, which would overestimate event probabilities when competing risks are present.
+
+## When to Use Competing Risks Analysis
+
+**Use competing risks when:**
+- Multiple mutually exclusive event types exist
+- Occurrence of one event prevents others
+- Need to estimate probability of specific event types
+- Want to understand how covariates affect different event types
+
+**Don't use when:**
+- Only one event type of interest (standard survival analysis)
+- Events are not mutually exclusive (use recurrent events methods)
+- Competing events are extremely rare (can treat as censoring)
+
+## Cumulative Incidence with Competing Risks
+
+### cumulative_incidence_competing_risks Function
+
+Estimates the cumulative incidence function for each event type.
+
+```python
+from sksurv.nonparametric import cumulative_incidence_competing_risks
+from sksurv.datasets import load_leukemia
+
+# Load data with competing risks
+X, y = load_leukemia()
+# y has event types: 0=censored, 1=relapse, 2=death
+
+# Compute cumulative incidence for each event type
+# Returns: time points, CIF for event 1, CIF for event 2, ...
+time_points, cif_1, cif_2 = cumulative_incidence_competing_risks(y)
+
+# Plot cumulative incidence functions
+import matplotlib.pyplot as plt
+
+plt.figure(figsize=(10, 6))
+plt.step(time_points, cif_1, where='post', label='Relapse', linewidth=2)
+plt.step(time_points, cif_2, where='post', label='Death in remission', linewidth=2)
+plt.xlabel('Time (weeks)')
+plt.ylabel('Cumulative Incidence')
+plt.title('Competing Risks: Relapse vs Death')
+plt.legend()
+plt.grid(True, alpha=0.3)
+plt.show()
+```
+
+### Interpretation
+
+- **CIF at time t**: Probability of experiencing that specific event by time t
+- **Sum of all CIFs**: Total probability of experiencing any event (all cause)
+- **1 - sum of CIFs**: Probability of being event-free and uncensored
+
+## Data Format for Competing Risks
+
+### Creating Structured Array with Event Types
+
+```python
+import numpy as np
+from sksurv.util import Surv
+
+# Event types: 0 = censored, 1 = event type 1, 2 = event type 2
+event_types = np.array([0, 1, 2, 1, 0, 2, 1])
+times = np.array([10.2, 5.3, 8.1, 3.7, 12.5, 6.8, 4.2])
+
+# Create survival array
+# For competing risks: event=True if any event occurred
+# Store event type separately or encode in the event field
+y = Surv.from_arrays(
+    event=(event_types > 0),  # True if any event
+    time=times
+)
+
+# Keep event_types for distinguishing between event types
+```
+
+### Converting Data with Event Types
+
+```python
+import pandas as pd
+from sksurv.util import Surv
+
+# Assume data has: time, event_type columns
+# event_type: 0=censored, 1=type1, 2=type2, etc.
+
+df = pd.read_csv('competing_risks_data.csv')
+
+# Create survival outcome
+y = Surv.from_arrays(
+    event=(df['event_type'] > 0),
+    time=df['time']
+)
+
+# Store event types
+event_types = df['event_type'].values
+```
+
+## Comparing Cumulative Incidence Between Groups
+
+### Stratified Analysis
+
+```python
+from sksurv.nonparametric import cumulative_incidence_competing_risks
+import matplotlib.pyplot as plt
+
+# Split by treatment group
+mask_treatment = X['treatment'] == 'A'
+mask_control = X['treatment'] == 'B'
+
+y_treatment = y[mask_treatment]
+y_control = y[mask_control]
+
+# Compute CIF for each group
+time_trt, cif1_trt, cif2_trt = cumulative_incidence_competing_risks(y_treatment)
+time_ctl, cif1_ctl, cif2_ctl = cumulative_incidence_competing_risks(y_control)
+
+# Plot comparison
+fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 5))
+
+# Event type 1
+ax1.step(time_trt, cif1_trt, where='post', label='Treatment', linewidth=2)
+ax1.step(time_ctl, cif1_ctl, where='post', label='Control', linewidth=2)
+ax1.set_xlabel('Time')
+ax1.set_ylabel('Cumulative Incidence')
+ax1.set_title('Event Type 1')
+ax1.legend()
+ax1.grid(True, alpha=0.3)
+
+# Event type 2
+ax2.step(time_trt, cif2_trt, where='post', label='Treatment', linewidth=2)
+ax2.step(time_ctl, cif2_ctl, where='post', label='Control', linewidth=2)
+ax2.set_xlabel('Time')
+ax2.set_ylabel('Cumulative Incidence')
+ax2.set_title('Event Type 2')
+ax2.legend()
+ax2.grid(True, alpha=0.3)
+
+plt.tight_layout()
+plt.show()
+```
+
+## Statistical Testing with Competing Risks
+
+### Gray's Test
+
+Compare cumulative incidence functions between groups using Gray's test (available in other packages like lifelines).
+
+```python
+# Note: Gray's test not directly available in scikit-survival
+# Consider using lifelines or other packages
+
+# from lifelines.statistics import multivariate_logrank_test
+# result = multivariate_logrank_test(times, groups, events, event_of_interest=1)
+```
+
+## Modeling with Competing Risks
+
+### Approach 1: Cause-Specific Hazard Models
+
+Fit separate Cox models for each event type, treating other event types as censored.
+
+```python
+from sksurv.linear_model import CoxPHSurvivalAnalysis
+from sksurv.util import Surv
+
+# Separate outcome for each event type
+# Event type 1: treat type 2 as censored
+y_event1 = Surv.from_arrays(
+    event=(event_types == 1),
+    time=times
+)
+
+# Event type 2: treat type 1 as censored
+y_event2 = Surv.from_arrays(
+    event=(event_types == 2),
+    time=times
+)
+
+# Fit cause-specific models
+cox_event1 = CoxPHSurvivalAnalysis()
+cox_event1.fit(X, y_event1)
+
+cox_event2 = CoxPHSurvivalAnalysis()
+cox_event2.fit(X, y_event2)
+
+# Interpret coefficients for each event type
+print("Event Type 1 (e.g., Relapse):")
+print(cox_event1.coef_)
+
+print("\nEvent Type 2 (e.g., Death):")
+print(cox_event2.coef_)
+```
+
+**Interpretation:**
+- Separate model for each competing event
+- Coefficients show effect on cause-specific hazard for that event type
+- A covariate may increase risk for one event type but decrease for another
+
+### Approach 2: Fine-Gray Sub-distribution Hazard Model
+
+Models the cumulative incidence directly (not available directly in scikit-survival, but can use other packages).
+
+```python
+# Note: Fine-Gray model not directly in scikit-survival
+# Consider using lifelines or rpy2 to access R's cmprsk package
+
+# from lifelines import CRCSplineFitter
+# crc = CRCSplineFitter()
+# crc.fit(df, event_col='event', duration_col='time')
+```
+
+## Practical Example: Complete Competing Risks Analysis
+
+```python
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+from sksurv.nonparametric import cumulative_incidence_competing_risks
+from sksurv.linear_model import CoxPHSurvivalAnalysis
+from sksurv.util import Surv
+
+# Simulate competing risks data
+np.random.seed(42)
+n = 200
+
+# Create features
+age = np.random.normal(60, 10, n)
+treatment = np.random.choice(['A', 'B'], n)
+
+# Simulate event times and types
+# Event types: 0=censored, 1=relapse, 2=death
+times = np.random.exponential(100, n)
+event_types = np.zeros(n, dtype=int)
+
+# Higher age increases both events, treatment A reduces relapse
+for i in range(n):
+    if times[i] < 150:  # Event occurred
+        # Probability of each event type
+        p_relapse = 0.6 if treatment[i] == 'B' else 0.4
+        event_types[i] = 1 if np.random.rand() < p_relapse else 2
+    else:
+        times[i] = 150  # Censored at study end
+
+# Create DataFrame
+df = pd.DataFrame({
+    'time': times,
+    'event_type': event_types,
+    'age': age,
+    'treatment': treatment
+})
+
+# Encode treatment
+df['treatment_A'] = (df['treatment'] == 'A').astype(int)
+
+# 1. OVERALL CUMULATIVE INCIDENCE
+print("=" * 60)
+print("OVERALL CUMULATIVE INCIDENCE")
+print("=" * 60)
+
+y_all = Surv.from_arrays(event=(df['event_type'] > 0), time=df['time'])
+time_points, cif_relapse, cif_death = cumulative_incidence_competing_risks(y_all)
+
+plt.figure(figsize=(10, 6))
+plt.step(time_points, cif_relapse, where='post', label='Relapse', linewidth=2)
+plt.step(time_points, cif_death, where='post', label='Death', linewidth=2)
+plt.xlabel('Time (days)')
+plt.ylabel('Cumulative Incidence')
+plt.title('Competing Risks: Relapse vs Death')
+plt.legend()
+plt.grid(True, alpha=0.3)
+plt.show()
+
+print(f"5-year relapse incidence: {cif_relapse[-1]:.2%}")
+print(f"5-year death incidence: {cif_death[-1]:.2%}")
+
+# 2. STRATIFIED BY TREATMENT
+print("\n" + "=" * 60)
+print("CUMULATIVE INCIDENCE BY TREATMENT")
+print("=" * 60)
+
+for trt in ['A', 'B']:
+    mask = df['treatment'] == trt
+    y_trt = Surv.from_arrays(
+        event=(df.loc[mask, 'event_type'] > 0),
+        time=df.loc[mask, 'time']
+    )
+    time_trt, cif1_trt, cif2_trt = cumulative_incidence_competing_risks(y_trt)
+    print(f"\nTreatment {trt}:")
+    print(f"  5-year relapse: {cif1_trt[-1]:.2%}")
+    print(f"  5-year death: {cif2_trt[-1]:.2%}")
+
+# 3. CAUSE-SPECIFIC MODELS
+print("\n" + "=" * 60)
+print("CAUSE-SPECIFIC HAZARD MODELS")
+print("=" * 60)
+
+X = df[['age', 'treatment_A']]
+
+# Model for relapse (event type 1)
+y_relapse = Surv.from_arrays(
+    event=(df['event_type'] == 1),
+    time=df['time']
+)
+cox_relapse = CoxPHSurvivalAnalysis()
+cox_relapse.fit(X, y_relapse)
+
+print("\nRelapse Model:")
+print(f"  Age:        HR = {np.exp(cox_relapse.coef_[0]):.3f}")
+print(f"  Treatment A: HR = {np.exp(cox_relapse.coef_[1]):.3f}")
+
+# Model for death (event type 2)
+y_death = Surv.from_arrays(
+    event=(df['event_type'] == 2),
+    time=df['time']
+)
+cox_death = CoxPHSurvivalAnalysis()
+cox_death.fit(X, y_death)
+
+print("\nDeath Model:")
+print(f"  Age:        HR = {np.exp(cox_death.coef_[0]):.3f}")
+print(f"  Treatment A: HR = {np.exp(cox_death.coef_[1]):.3f}")
+
+print("\n" + "=" * 60)
+```
+
+## Important Considerations
+
+### Censoring in Competing Risks
+
+- **Administrative censoring**: Subject still at risk at end of study
+- **Loss to follow-up**: Subject leaves study before event
+- **Competing event**: Other event occurred - NOT censored for CIF, but censored for cause-specific models
+
+### Choosing Between Cause-Specific and Sub-distribution Models
+
+**Cause-Specific Hazard Models:**
+- Easier to interpret
+- Direct effect on hazard rate
+- Better for understanding etiology
+- Can fit with scikit-survival
+
+**Fine-Gray Sub-distribution Models:**
+- Models cumulative incidence directly
+- Better for prediction and risk stratification
+- More appropriate for clinical decision-making
+- Requires other packages
+
+### Common Mistakes
+
+**Mistake 1**: Using Kaplan-Meier to estimate event-specific probabilities
+- **Wrong**: Kaplan-Meier for event type 1, treating type 2 as censored
+- **Correct**: Cumulative incidence function accounting for competing risks
+
+**Mistake 2**: Ignoring competing risks when they're substantial
+- If competing event rate > 10-20%, should use competing risks methods
+
+**Mistake 3**: Confusing cause-specific and sub-distribution hazards
+- They answer different questions
+- Use appropriate model for your research question
+
+## Summary
+
+**Key Functions:**
+- `cumulative_incidence_competing_risks`: Estimate CIF for each event type
+- Fit separate Cox models for cause-specific hazards
+- Use stratified analysis to compare groups
+
+**Best Practices:**
+1. Always plot cumulative incidence functions
+2. Report both event-specific and overall incidence
+3. Use cause-specific models in scikit-survival
+4. Consider Fine-Gray models (other packages) for prediction
+5. Be explicit about which events are competing vs censored

package/bin/skills/scikit-survival/references/cox-models.md

@@ -0,0 +1,182 @@
+# Cox Proportional Hazards Models
+
+## Overview
+
+Cox proportional hazards models are semi-parametric models that relate covariates to the time of an event. The hazard function for individual *i* is expressed as:
+
+**h_i(t) = h_0(t) × exp(β^T x_i)**
+
+where:
+- h_0(t) is the baseline hazard function (unspecified)
+- β is the vector of coefficients
+- x_i is the covariate vector for individual *i*
+
+The key assumption is that the hazard ratio between two individuals is constant over time (proportional hazards).
+
+## CoxPHSurvivalAnalysis
+
+Basic Cox proportional hazards model for survival analysis.
+
+### When to Use
+- Standard survival analysis with censored data
+- Need interpretable coefficients (log hazard ratios)
+- Proportional hazards assumption holds
+- Dataset has relatively few features
+
+### Key Parameters
+- `alpha`: Regularization parameter (default: 0, no regularization)
+- `ties`: Method for handling tied event times ('breslow' or 'efron')
+- `n_iter`: Maximum number of iterations for optimization
+
+### Example Usage
+```python
+from sksurv.linear_model import CoxPHSurvivalAnalysis
+from sksurv.datasets import load_gbsg2
+
+# Load data
+X, y = load_gbsg2()
+
+# Fit Cox model
+estimator = CoxPHSurvivalAnalysis()
+estimator.fit(X, y)
+
+# Get coefficients (log hazard ratios)
+coefficients = estimator.coef_
+
+# Predict risk scores
+risk_scores = estimator.predict(X)
+```
+
+## CoxnetSurvivalAnalysis
+
+Cox model with elastic net penalty for feature selection and regularization.
+
+### When to Use
+- High-dimensional data (many features)
+- Need automatic feature selection
+- Want to handle multicollinearity
+- Require sparse models
+
+### Penalty Types
+- **Ridge (L2)**: alpha_min_ratio=1.0, l1_ratio=0
+  - Shrinks all coefficients
+  - Good when all features are relevant
+
+- **Lasso (L1)**: l1_ratio=1.0
+  - Performs feature selection (sets coefficients to zero)
+  - Good for sparse models
+
+- **Elastic Net**: 0 < l1_ratio < 1
+  - Combination of L1 and L2
+  - Balances feature selection and grouping
+
+### Key Parameters
+- `l1_ratio`: Balance between L1 and L2 penalty (0=Ridge, 1=Lasso)
+- `alpha_min_ratio`: Ratio of smallest to largest penalty in regularization path
+- `n_alphas`: Number of alphas along regularization path
+- `fit_baseline_model`: Whether to fit unpenalized baseline model
+
+### Example Usage
+```python
+from sksurv.linear_model import CoxnetSurvivalAnalysis
+
+# Fit with elastic net penalty
+estimator = CoxnetSurvivalAnalysis(l1_ratio=0.5, alpha_min_ratio=0.01)
+estimator.fit(X, y)
+
+# Access regularization path
+alphas = estimator.alphas_
+coefficients_path = estimator.coef_path_
+
+# Predict with specific alpha
+risk_scores = estimator.predict(X, alpha=0.1)
+```
+
+### Cross-Validation for Alpha Selection
+```python
+from sklearn.model_selection import GridSearchCV
+from sksurv.metrics import concordance_index_censored
+
+# Define parameter grid
+param_grid = {'l1_ratio': [0.1, 0.5, 0.9],
+              'alpha_min_ratio': [0.01, 0.001]}
+
+# Grid search with C-index
+cv = GridSearchCV(CoxnetSurvivalAnalysis(),
+                  param_grid,
+                  scoring='concordance_index_ipcw',
+                  cv=5)
+cv.fit(X, y)
+
+# Best parameters
+best_params = cv.best_params_
+```
+
+## IPCRidge
+
+Inverse probability of censoring weighted Ridge regression for accelerated failure time models.
+
+### When to Use
+- Prefer accelerated failure time (AFT) framework over proportional hazards
+- Need to model how features accelerate/decelerate survival time
+- High censoring rates
+- Want regularization with Ridge penalty
+
+### Key Difference from Cox Models
+AFT models assume features multiply survival time by a constant factor, rather than multiplying the hazard rate. The model predicts log survival time directly.
+
+### Example Usage
+```python
+from sksurv.linear_model import IPCRidge
+
+# Fit IPCRidge model
+estimator = IPCRidge(alpha=1.0)
+estimator.fit(X, y)
+
+# Predict log survival time
+log_time = estimator.predict(X)
+```
+
+## Model Comparison and Selection
+
+### Choosing Between Models
+
+**Use CoxPHSurvivalAnalysis when:**
+- Small to moderate number of features
+- Want interpretable hazard ratios
+- Standard survival analysis setting
+
+**Use CoxnetSurvivalAnalysis when:**
+- High-dimensional data (p >> n)
+- Need feature selection
+- Want to identify important predictors
+- Presence of multicollinearity
+
+**Use IPCRidge when:**
+- AFT framework is more appropriate
+- High censoring rates
+- Want to model time directly rather than hazard
+
+### Checking Proportional Hazards Assumption
+
+The proportional hazards assumption should be verified using:
+- Schoenfeld residuals
+- Log-log survival plots
+- Statistical tests (available in other packages like lifelines)
+
+If violated, consider:
+- Stratification by violating covariates
+- Time-varying coefficients
+- Alternative models (AFT, parametric models)
+
+## Interpretation
+
+### Cox Model Coefficients
+- Positive coefficient: increased hazard (shorter survival)
+- Negative coefficient: decreased hazard (longer survival)
+- Hazard ratio = exp(β) for one-unit increase in covariate
+- Example: β=0.693 → HR=2.0 (doubles the hazard)
+
+### Risk Scores
+- Higher risk score = higher risk of event = shorter expected survival
+- Risk scores are relative; use survival functions for absolute predictions