adaptivetesting 1.2b1__tar.gz → 1.2.1__tar.gz

adaptivetesting-1.2.1/PKG-INFO

@@ -0,0 +1,103 @@
+Metadata-Version: 2.4
+Name: adaptivetesting
+Version: 1.2.1
+Summary: adaptivetesting is a Python package that can be used to simulate and evaluate custom CAT scenarios as well as implement them in real-world testing scenarios from a single codebase
+Author-email: Jonas Engicht <dev@condecon.de>
+Project-URL: Homepage, https://github.com/condecon/adaptivetesting
+Project-URL: Documentation, https://github.com/condecon/adaptivetesting/wiki
+Project-URL: Repository, https://github.com/condecon/adaptivetesting.git
+Project-URL: Issues, https://github.com/condecon/adaptivetesting/issues
+Keywords: statistics,psychology,item-response-theory,computerized-adaptive-testing
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=2.1.0
+Requires-Dist: pandas>=2.3.0
+Requires-Dist: scipy>=1.15.0
+Requires-Dist: tqdm>=4.66.0
+Requires-Dist: matplotlib>=3.10.0
+Requires-Dist: numdifftools>=0.9.42
+Dynamic: license-file
+
+# adaptivetesting
+
+<img src="/docs/_static/logo.svg" style="width: 100%">
+</img>
+
+**An open-source Python package for simplified, customizable Computerized Adaptive Testing (CAT) using Bayesian methods.**
+
+
+## Key Features
+
+- **Bayesian Methods**: Built-in support for Bayesian ability estimation with customizable priors
+- **Flexible Architecture**: Object-oriented design with abstract classes for easy extension
+- **Item Response Theory**: Full support for 1PL, 2PL, 3PL, and 4PL models, GRM, GPCM
+- **Multiple Estimators**:
+  - Maximum Likelihood Estimation (MLE)
+  - Bayesian Modal Estimation (BM)
+  - Expected A Posteriori (EAP)
+- **Item Selection Strategies**: Maximum information criterion
+- **Content Balancing**: Maximum Priority Index, Weighted Penalty Model
+- **Exposure Control**: Randomesque Item Selection, Maximum Priority Index 
+- **Simulation Framework**: Comprehensive tools for CAT simulation and evaluation
+- **Real-world Application**: Direct transition from simulation to production testing
+- **Stopping Criteria**: Support for standard error and test length criteria
+- **Data Management**: Built-in support for CSV and pickle data formats
+
+## Installation
+
+Install from PyPI using pip:
+
+```bash
+pip install adaptivetesting
+```
+
+For the latest development version:
+
+```bash
+pip install git+https://github.com/condecon/adaptivetesting
+```
+
+## Documentation
+You can find our documentation in the [GitHub wiki](https://github.com/condecon/adaptivetesting/wiki).
+
+## Contributing
+
+We welcome contributions! Please see our [GitHub repository](https://github.com/condecon/adaptivetesting) for:
+
+- Issue tracking
+- Feature requests
+- Pull request guidelines
+- Development setup
+
+## Research and Applications
+
+This package is designed for researchers and practitioners in:
+
+- Educational assessment
+- Psychological testing
+- Cognitive ability measurement
+- Adaptive learning systems
+- Psychometric research
+
+The package facilitates the transition from research simulation to real-world testing applications without requiring major code modifications.
+
+## Citation
+If you use this package for your academic work, please provide the following reference:
+Engicht, J., Bee, R. M., & Koch, T. (2025). Customizable Bayesian Adaptive Testing with Python – The adaptivetesting Package. Open Science Framework. https://doi.org/10.31219/osf.io/d2xge_v1
+
+```
+@online{engichtCustomizableBayesianAdaptive2025,
+  title = {Customizable {{Bayesian Adaptive Testing}} with {{Python}} – {{The}} Adaptivetesting {{Package}}},
+  author = {Engicht, Jonas and Bee, R. Maximilian and Koch, Tobias},
+  date = {2025-08-06},
+  eprinttype = {Open Science Framework},
+  doi = {10.31219/osf.io/d2xge_v1},
+  url = {https://osf.io/d2xge_v1},
+  pubstate = {prepublished}
+}
+```
+
+## License
+
+This project is licensed under the terms specified in the [LICENSE](LICENSE) file.

adaptivetesting-1.2.1/README.md

@@ -0,0 +1,82 @@
+# adaptivetesting
+
+<img src="/docs/_static/logo.svg" style="width: 100%">
+</img>
+
+**An open-source Python package for simplified, customizable Computerized Adaptive Testing (CAT) using Bayesian methods.**
+
+
+## Key Features
+
+- **Bayesian Methods**: Built-in support for Bayesian ability estimation with customizable priors
+- **Flexible Architecture**: Object-oriented design with abstract classes for easy extension
+- **Item Response Theory**: Full support for 1PL, 2PL, 3PL, and 4PL models, GRM, GPCM
+- **Multiple Estimators**:
+  - Maximum Likelihood Estimation (MLE)
+  - Bayesian Modal Estimation (BM)
+  - Expected A Posteriori (EAP)
+- **Item Selection Strategies**: Maximum information criterion
+- **Content Balancing**: Maximum Priority Index, Weighted Penalty Model
+- **Exposure Control**: Randomesque Item Selection, Maximum Priority Index 
+- **Simulation Framework**: Comprehensive tools for CAT simulation and evaluation
+- **Real-world Application**: Direct transition from simulation to production testing
+- **Stopping Criteria**: Support for standard error and test length criteria
+- **Data Management**: Built-in support for CSV and pickle data formats
+
+## Installation
+
+Install from PyPI using pip:
+
+```bash
+pip install adaptivetesting
+```
+
+For the latest development version:
+
+```bash
+pip install git+https://github.com/condecon/adaptivetesting
+```
+
+## Documentation
+You can find our documentation in the [GitHub wiki](https://github.com/condecon/adaptivetesting/wiki).
+
+## Contributing
+
+We welcome contributions! Please see our [GitHub repository](https://github.com/condecon/adaptivetesting) for:
+
+- Issue tracking
+- Feature requests
+- Pull request guidelines
+- Development setup
+
+## Research and Applications
+
+This package is designed for researchers and practitioners in:
+
+- Educational assessment
+- Psychological testing
+- Cognitive ability measurement
+- Adaptive learning systems
+- Psychometric research
+
+The package facilitates the transition from research simulation to real-world testing applications without requiring major code modifications.
+
+## Citation
+If you use this package for your academic work, please provide the following reference:
+Engicht, J., Bee, R. M., & Koch, T. (2025). Customizable Bayesian Adaptive Testing with Python – The adaptivetesting Package. Open Science Framework. https://doi.org/10.31219/osf.io/d2xge_v1
+
+```
+@online{engichtCustomizableBayesianAdaptive2025,
+  title = {Customizable {{Bayesian Adaptive Testing}} with {{Python}} – {{The}} Adaptivetesting {{Package}}},
+  author = {Engicht, Jonas and Bee, R. Maximilian and Koch, Tobias},
+  date = {2025-08-06},
+  eprinttype = {Open Science Framework},
+  doi = {10.31219/osf.io/d2xge_v1},
+  url = {https://osf.io/d2xge_v1},
+  pubstate = {prepublished}
+}
+```
+
+## License
+
+This project is licensed under the terms specified in the [LICENSE](LICENSE) file.

{adaptivetesting-1.2b1 → adaptivetesting-1.2.1}/adaptivetesting/__init__.py

@@ -42,4 +42,14 @@ from .services.__item_selection_protocol import ItemSelectionStrategy
 
 from .simulation.__simulation import Simulation, SimulationPool, setup_simulation_and_start
 
-from .utils.__descriptives import bias, average_absolute_deviation, rmse
+from .utils.__descriptives import bias, average_absolute_deviation, rmse
+from .utils.__funcs import load_final_test_results, load_test_results_single_participant
+from .utils.__plots import (
+    plot_exposure_rate,
+    plot_final_ability_estimates,
+    plot_icc,
+    plot_iif,
+    plot_exposure_rate,
+    plot_test_information,
+    plot_theta_estimation_trace
+)

{adaptivetesting-1.2b1 → adaptivetesting-1.2.1}/adaptivetesting/data/__read_prev_items_exp_cont.py

@@ -9,8 +9,16 @@ from .__pickle_context import PickleContext
 def read_prev_items(format: Literal["CSV", "PICKLE"],
                     test_id: str,
                     participant_ids: list[str]) -> list[TestItem]:
-    """Read previously administered items
-    to perform exposure control
+    """Read previously administered items.
+    This function is internally used to perform exposure control.
+
+    Args:
+        format (Literal["CSV", "PICKLE"]): The format of the results: CSV or PICKLE.
+        test_id (str): The test ID / simulation ID.
+        participant_ids (list[str]): The participant IDs.
+
+    Returns:
+        list[TestItem]: A list of TestItem objects.
     """
     all_shown_test_items = []
     
@@ -23,9 +31,21 @@ def read_prev_items(format: Literal["CSV", "PICKLE"],
     return all_shown_test_items
 
 
-@deprecated("This function should be replaced after merging"
-            " with revision branch")
+@deprecated("This function will be removed in future releases.")
 def read_single_participant(test_id: str, participant_id: str, format: Literal["CSV", "PICKLE"]) -> list[TestResult]:
+    """
+    Reads the test results of a particular participant.
+    Args:
+        test_id (str): The test ID / simulation ID.
+        participant_id (str): The participant ID.
+        format (Literal["CSV", "PICKLE"]): The format of the results: CSV or PICKLE.
+
+    Returns:
+        list[TestResult]: A list of TestResult objects.
+
+    Raises:
+        ValueError: If the format is not supported.
+    """
     context: CSVContext | PickleContext
     if format == "CSV":
         context = CSVContext(test_id, participant_id)
@@ -33,3 +53,4 @@ def read_single_participant(test_id: str, participant_id: str, format: Literal["
     if format == "PICKLE":
         context = PickleContext(test_id, participant_id)
         return context.load()
+    raise ValueError(f"Unknown format: {format}")

{adaptivetesting-1.2b1 → adaptivetesting-1.2.1}/adaptivetesting/implementations/__test_assembler.py

@@ -133,6 +133,15 @@ class TestAssembler(AdaptiveTest):
             content_balancing (CONTENT_BALANCING, optional): Selected content balancing strategy. Defaults to None.
                 If a content balancing strategy is specified, the item selection strategy will be ignored.
 
+            content_balancing_args (ContentBalancingArgs, optional): Arguments for the content balancing strategy.
+                Defaults to None.
+
+            exposure_control (EXPOSURE_CONTROL, optional): Exposure control strategy. Defaults to None.
+                If an exposure control strategy is specified, the item selection strategy will be ignored.
+
+            exposure_control_args (ExposureControlArgs, optional): Arguments for the exposure control strategy.
+                Defaults to None.
+
             pretest (bool, optional): Whether to run a pretest phase before the main test. Defaults to False.
 
             pretest_seed (int | None, optional): Random seed for pretest item selection. Defaults to None.
@@ -166,6 +175,7 @@ class TestAssembler(AdaptiveTest):
                          initial_ability_level,
                          simulation,
                          debug,
+                         model_type=model_type,
                          **kwargs)
 
     def estimate_ability_level(self):
@@ -345,7 +355,7 @@ class TestAssembler(AdaptiveTest):
             The result of the superclass's run_test_once() method.
         """
         # check if to run pretest
-        if self.__pretest is True:
+        if self.__pretest:
             pretest = PreTest(
                 self.item_pool.test_items,
                 self.__pretest_seed
@@ -353,7 +363,7 @@ class TestAssembler(AdaptiveTest):
             # get selected items
             random_items = pretest.select_random_item_quantile()
             for item in random_items:
-                if self.simulation is True:
+                if self.simulation:
                     response = self.item_pool.get_item_response(item)
                 else:
                     # not simulation

{adaptivetesting-1.2b1 → adaptivetesting-1.2.1}/adaptivetesting/math/__gen_response_pattern.py

@@ -13,7 +13,7 @@ def generate_response_pattern(ability: float,
                               model: Literal["GRM", "GPCM"] | None = None,
                               seed: int | None = None) -> list[int]:
     """Generates a response pattern for a given ability level
-    and item difficulties. Also, a seed can be set.
+    and item difficulties. A seed may be set for reproducibility.
 
     Args:
         ability (float): participants ability
@@ -48,7 +48,19 @@ def generate_response_pattern(ability: float,
 
 def gen_pattern_dichotomous(ability: float,
                             items: list[TestItem],
-                            rng: np.random.RandomState):
+                            rng: np.random.RandomState) -> list[int]:
+    """
+    Generates a response pattern for a given ability level for an IRT
+    model with dichotomous test items (e.g., 4PL, 3PL).
+
+    Args:
+        ability (float): participants ability
+        items (list[TestItem]): test items
+        rng (np.random.RandomState): random state (numpy object)
+
+    Returns:
+        list[int]: generated response pattern
+    """
     responses: list[int] = []
 
     for item in items:
@@ -82,6 +94,18 @@ def gen_pattern_poly(
     model: Literal["GRM", "GPCM"],
     rng: np.random.RandomState
 ) -> list[int]:
+    """
+    Generates a response pattern for a given ability level for polytomous items.
+
+    Args:
+        ability (float): participants ability
+        items (list[TestItem]): test items
+        model (literal, optional): model type (GRM, or GPCM)
+        rng (np.random.RandomState, optional): random state (numpy object)
+
+    Returns:
+        list[int]: response pattern
+    """
     responses: list[int] = []
     # loop through all items
     for item in items:

{adaptivetesting-1.2b1 → adaptivetesting-1.2.1}/adaptivetesting/math/content_balancing/__constraint.py

@@ -3,7 +3,7 @@ from dataclasses import dataclass
 
 @dataclass
 class Constraint:
-    """Constraint for Exposure Constrol and Content Balancing
+    """Constraint for Exposure Control and Content Balancing
     """
     name: str
     """Constraint name. This string has to be the same

{adaptivetesting-1.2b1 → adaptivetesting-1.2.1}/adaptivetesting/math/content_balancing/__functions.py

@@ -21,6 +21,7 @@ def compute_priority_index(item: TestItem,
         required_items (int): number of items required to be shown per constraint
         shown_items (int): number of items already shown per constraint
         current_ability (float): currently estimated ability level
+        model (Literal['GRM', 'GPCM'] | None): model type. Required for polytomous items.
 
     Returns:
         float: priority index of an item
@@ -103,7 +104,7 @@ def compute_expected_difference(proportion: float,
         constraint_target (float): constraint target
 
     Returns:
-        float: _description_
+        float: expected difference
     """
     expected_difference = proportion - constraint_target
     return expected_difference
@@ -154,9 +155,9 @@ def compute_total_content_penalty_value_for_item(item: TestItem,
 
     Args:
         item (TestItem): given test item
-        shown_items:
-        available_items:
-        constraints (list[Constraint]): YYYYYY
+        shown_items: shown item
+        available_items: available items in the test
+        constraints (list[Constraint]): constraints assigned to the test and its items
 
     Returns:
         float: total content penalty value

{adaptivetesting-1.2b1 → adaptivetesting-1.2.1}/adaptivetesting/math/content_balancing/__weighted_penalty_model.py

@@ -21,7 +21,7 @@ ITEM_GROUP = Literal["green", "orange", "yellow", "red", None]
 class WeightedPenaltyModel(ContentBalancing):
     """This content balancing method follows Shin et al. (2009)
     and allows to apply constraints to the item selection.
-    The users can define a custom weight for the item information and the constriant.
+    The users can define a custom weight for the item information and the constraint.
 
     References
     ------------
@@ -98,7 +98,7 @@ class WeightedPenaltyModel(ContentBalancing):
         max_content_penalty = max(content_penalties)
         min_content_penalty = min(content_penalties)
 
-        self.calcualte_weighted_penalty_for_all_items(
+        self.calculate_weighted_penalty_for_all_items(
             item_information_list=item_information_list,
             max_item=max_item,
             content_penalties=content_penalties,
@@ -114,7 +114,16 @@ class WeightedPenaltyModel(ContentBalancing):
         # order items
         self.order_candidate_items()
 
-    def calculate_information(self, model: Literal['GRM', 'GPCM'] | None = None) -> list[float]:
+    def calculate_information(self,
+                              model: Literal['GRM', 'GPCM'] | None = None) -> list[float]:
+        """
+        Calculates the item information for every item
+        Args:
+            model: model type. Required for polytomous models.
+
+        Returns:
+            list of item information
+        """
         information_list = [
             float(item_information_function(
                 ability=self.ability,
@@ -126,6 +135,7 @@ class WeightedPenaltyModel(ContentBalancing):
         return information_list
 
     def calculate_content_penalties(self) -> list[float]:
+        """Calculate content penalty values for every item"""
         content_penalties = [
             compute_total_content_penalty_value_for_item(
                 item=item,
@@ -137,12 +147,13 @@ class WeightedPenaltyModel(ContentBalancing):
         ]
         return content_penalties
 
-    def calcualte_weighted_penalty_for_all_items(self,
+    def calculate_weighted_penalty_for_all_items(self,
                                                  item_information_list: list[float],
                                                  max_item: float,
                                                  content_penalties: list[float],
                                                  max_content_penalty: float,
                                                  min_content_penalty: float):
+        """Calculate the weighted penalty value for every item"""
         for i, item in enumerate(self.items):
             weighted_penalty_value = self.calculate_weighted_penalty_value(
                 item_information=item_information_list[i],
@@ -157,6 +168,8 @@ class WeightedPenaltyModel(ContentBalancing):
             self.eligible_items.append((item, weighted_penalty_value, None))
 
     def get_constraint_group_assignments(self) -> list[tuple[Constraint, CONSTRAINT_GROUP]]:
+        """Assign every constraint to a constraint group depending on the number
+        of items shown from each specific constraint."""
         group_assignment: list[tuple[Constraint, CONSTRAINT_GROUP]] = []
         for constraint in self.constraints:
             # calculate proportion of the constraint
@@ -183,6 +196,15 @@ class WeightedPenaltyModel(ContentBalancing):
     @staticmethod
     def assign_color_group_per_proportion(constraint: Constraint,
                                           prop: float):
+        """
+        Implements constraint assignment logic.
+        Args:
+            constraint (Constraint): constraint
+            prop (float): proportion
+
+        Returns:
+            tuple[Constraint, str]: constraint and assigned group
+        """
         if constraint.lower is not None and constraint.upper is not None:
             if prop <= constraint.lower:
                 return (constraint, "A")
@@ -195,6 +217,12 @@ class WeightedPenaltyModel(ContentBalancing):
 
     def form_list_of_candidate_items(self,
                                      group_assignment: list[tuple[Constraint, CONSTRAINT_GROUP]]) -> None:
+        """
+        Form a list of candidate items according to the constraint group assignment.
+        Args:
+            group_assignment (list[tuple[Constraint, CONSTRAINT_GROUP]): list of constraints and assigned groups
+
+        """
         for i, item_entry in enumerate(self.eligible_items):
             item, weighted_penalty_value, _ = item_entry
             # find associated constraint
@@ -214,6 +242,17 @@ class WeightedPenaltyModel(ContentBalancing):
         associated_constraints: list[tuple[Constraint, Literal['A', 'B', 'C']]],
         weighted_penalty_value: float
     ):
+        """
+        Assign items according to the constraint group assignment to an item group (color group)
+        Args:
+            item (TestItem): item
+            associated_constraints (list[tuple[Constraint, Literal['A', 'B', 'C']]]):
+                constraints relevant for this item
+            weighted_penalty_value (float): calculated weighted penalty value of this item
+
+        Returns:
+            tuple[TestItem, float, str]: item, weighted penalty value and assigned color
+        """
         group_set = set([group for _, group in associated_constraints])
         
         # if all associated constraints A or B -> green group
@@ -232,6 +271,8 @@ class WeightedPenaltyModel(ContentBalancing):
             return (item, weighted_penalty_value, None)
 
     def order_candidate_items(self):
+        """Order candidate items according to the color group assignment.
+        """
         # between group ordering: green, orange, yellow, red
         # within group ordering: ascending order of weighted penalty value
         self.eligible_items = sorted(
@@ -251,6 +292,20 @@ class WeightedPenaltyModel(ContentBalancing):
                                          constraint_weight: float,
                                          information_weight: float
                                          ) -> float:
+        """
+        Calculate the weighted penalty value of an item.
+        Args:
+            content_penalty: content penalty value
+            minimum_total_content_penalty: minimum total content penalty value
+            maximum_total_content_penalty: maximum total content penalty value
+            item_information: item information
+            max_information: maximum item information
+            constraint_weight: constraint weight
+            information_weight: information weight
+
+        Returns:
+            weighted penalty value
+        """
         # reference content penalty
         total_content_penalty_value = content_penalty
 

{adaptivetesting-1.2b1 → adaptivetesting-1.2.1}/adaptivetesting/math/estimators/__bayes_modal_estimation.py

@@ -31,6 +31,8 @@ class BayesModal(IEstimator):
                 prior (Prior): prior distribution
 
                 optimization_interval (Tuple[float, float]): interval used for the optimization function
+
+                model (Literal["GRM", "GPCM"], optional): model type (required for polytomous models)
             """
         super().__init__(response_pattern, items, optimization_interval)
 

{adaptivetesting-1.2b1 → adaptivetesting-1.2.1}/adaptivetesting/math/estimators/__expect_a_posteriori.py

@@ -31,6 +31,8 @@ class ExpectedAPosteriori(BayesModal):
                 prior (Prior): prior distribution
 
                 optimization_interval (Tuple[float, float]): interval used for the optimization function
+
+                model (Literal["GRM", "GPCM"], optional): model type (required for polytomous models)
         """
         super().__init__(response_pattern, items, prior, optimization_interval)
 

adaptivetesting-1.2.1/adaptivetesting/math/estimators/__functions/__bayes.py

@@ -0,0 +1,74 @@
+import numpy as np
+from scipy.optimize import minimize_scalar, OptimizeResult # type: ignore
+from ..__prior import Prior
+from ....models.__algorithm_exception import AlgorithmException
+from .__estimators import log_likelihood
+from .__estimators import probability_y0, probability_y1
+
+
+def maximize_posterior(
+    a: np.ndarray,
+    b: np.ndarray,
+    c: np.ndarray,
+    d: np.ndarray,
+    response_pattern: np.ndarray,
+    prior: Prior,
+    optimization_interval: tuple[float, float] = (-10, 10)
+) -> float:
+    """Get the maximum of the posterior distribution
+
+    Args:
+        a (np.ndarray): item parameter a
+        b (np.ndarray): item parameter b
+        c (np.ndarray): item parameter c
+        d (np.ndarray): item parameter d
+        response_pattern (np.ndarray): response pattern (simulated or user generated)
+        prior (Prior): prior distribution
+        optimization_interval (Tuple[float, float]): interval used for the optimization function
+
+    Returns:
+        float: Bayes Modal estimator for the given parameters
+    """
+    def log_posterior(mu):
+        log_likelihood_res = log_likelihood(mu, a, b, c, d, response_pattern)
+
+        if hasattr(prior, "logpdf"):
+            log_prior = prior.logpdf(mu)
+        else:
+            log_prior = np.log(np.clip(prior.pdf(mu), 1e-300, None))
+    
+        log_post = log_likelihood_res + log_prior
+
+        if not np.isfinite(log_post):
+            return -1e300
+        else:
+            return float(log_post.ravel()[0])
+    
+    result: OptimizeResult = minimize_scalar(lambda mu: -log_posterior(mu),
+                                             bounds=optimization_interval,
+                                             method="bounded") # type: ignore
+
+    if not result.success:
+        raise AlgorithmException(f"Optimization failed: {result.message}")
+
+    else:
+        return float(result.x)
+
+
+def likelihood(mu: np.ndarray,
+               a: np.ndarray,
+               b: np.ndarray,
+               c: np.ndarray,
+               d: np.ndarray,
+               response_pattern: np.ndarray
+               ):
+    
+    p1 = probability_y1(mu, a, b, c, d)
+    p0 = probability_y0(mu, a, b, c, d)
+
+    log_likelihood = np.sum(
+        (response_pattern * np.log(p1 + 1e-300)) + (
+            (1 - response_pattern) * np.log(p0 + 1e-300)
+        ))
+
+    return -np.exp(log_likelihood)