TY - JOUR
T1 - Factors Affecting the Classification Accuracy and Average Length of a Variable-Length Cognitive Diagnostic Computerized Test
JF - Journal of Computerized Adaptive Testing
Y1 - 2018
A1 - Huebner, Alan
A1 - Finkelman, Matthew D.
A1 - Weissman, Alexander
VL - 6
UR - http://iacat.org/jcat/index.php/jcat/article/view/55/30
IS - 1
ER -
TY - JOUR
T1 - On Computing the Key Probability in the Stochastically Curtailed Sequential Probability Ratio Test
JF - Applied Psychological Measurement
Y1 - 2016
A1 - Huebner, Alan R.
A1 - Finkelman, Matthew D.
AB - The Stochastically Curtailed Sequential Probability Ratio Test (SCSPRT) is a termination criterion for computerized classification tests (CCTs) that has been shown to be more efficient than the well-known Sequential Probability Ratio Test (SPRT). The performance of the SCSPRT depends on computing the probability that at a given stage in the test, an examinee’s current interim classification status will not change before the end of the test. Previous work discusses two methods of computing this probability, an exact method in which all potential responses to remaining items are considered and an approximation based on the central limit theorem (CLT) requiring less computation. Generally, the CLT method should be used early in the test when the number of remaining items is large, and the exact method is more appropriate at later stages of the test when few items remain. However, there is currently a dearth of information as to the performance of the SCSPRT when using the two methods. For the first time, the exact and CLT methods of computing the crucial probability are compared in a simulation study to explore whether there is any effect on the accuracy or efficiency of the CCT. The article is focused toward practitioners and researchers interested in using the SCSPRT as a termination criterion in an operational CCT.
VL - 40
UR - http://apm.sagepub.com/content/40/2/142.abstract
ER -
TY - JOUR
T1 - Stochastic Curtailment of Questionnaires for Three-Level Classification: Shortening the CES-D for Assessing Low, Moderate, and High Risk of Depression
JF - Applied Psychological Measurement
Y1 - 2016
A1 - Smits, Niels
A1 - Finkelman, Matthew D.
A1 - Kelderman, Henk
AB - In clinical assessment, efficient screeners are needed to ensure low respondent burden. In this article, Stochastic Curtailment (SC), a method for efficient computerized testing for classification into two classes for observable outcomes, was extended to three classes. In a post hoc simulation study using the item scores on the Center for Epidemiologic Studies–Depression Scale (CES-D) of a large sample, three versions of SC, SC via Empirical Proportions (SC-EP), SC via Simple Ordinal Regression (SC-SOR), and SC via Multiple Ordinal Regression (SC-MOR) were compared at both respondent burden and classification accuracy. All methods were applied under the regular item order of the CES-D and under an ordering that was optimal in terms of the predictive power of the items. Under the regular item ordering, the three methods were equally accurate, but SC-SOR and SC-MOR needed less items. Under the optimal ordering, additional gains in efficiency were found, but SC-MOR suffered from capitalization on chance substantially. It was concluded that SC-SOR is an efficient and accurate method for clinical screening. Strengths and weaknesses of the methods are discussed.
VL - 40
UR - http://apm.sagepub.com/content/40/1/22.abstract
ER -
TY - JOUR
T1 - Stochastic Curtailment in Adaptive Mastery Testing: Improving the Efficiency of Confidence Interval–Based Stopping Rules
JF - Applied Psychological Measurement
Y1 - 2015
A1 - Sie, Haskell
A1 - Finkelman, Matthew D.
A1 - Bartroff, Jay
A1 - Thompson, Nathan A.
AB - A well-known stopping rule in adaptive mastery testing is to terminate the assessment once the examinee’s ability confidence interval lies entirely above or below the cut-off score. This article proposes new procedures that seek to improve such a variable-length stopping rule by coupling it with curtailment and stochastic curtailment. Under the new procedures, test termination can occur earlier if the probability is high enough that the current classification decision remains the same should the test continue. Computation of this probability utilizes normality of an asymptotically equivalent version of the maximum likelihood ability estimate. In two simulation sets, the new procedures showed a substantial reduction in average test length while maintaining similar classification accuracy to the original method.
VL - 39
UR - http://apm.sagepub.com/content/39/4/278.abstract
ER -
TY - JOUR
T1 - Utilizing Response Times in Computerized Classification Testing
JF - Applied Psychological Measurement
Y1 - 2015
A1 - Sie, Haskell
A1 - Finkelman, Matthew D.
A1 - Riley, Barth
A1 - Smits, Niels
AB - A well-known approach in computerized mastery testing is to combine the Sequential Probability Ratio Test (SPRT) stopping rule with item selection to maximize Fisher information at the mastery threshold. This article proposes a new approach in which a time limit is defined for the test and examinees’ response times are considered in both item selection and test termination. Item selection is performed by maximizing Fisher information per time unit, rather than Fisher information itself. The test is terminated once the SPRT makes a classification decision, the time limit is exceeded, or there is no remaining item that has a high enough probability of being answered before the time limit. In a simulation study, the new procedure showed a substantial reduction in average testing time while slightly improving classification accuracy compared with the original method. In addition, the new procedure reduced the percentage of examinees who exceeded the time limit.
VL - 39
UR - http://apm.sagepub.com/content/39/5/389.abstract
ER -