%0 Journal Article
%J Journal of Computerized Adaptive Testing
%D 2018
%T Factors Affecting the Classification Accuracy and Average Length of a Variable-Length Cognitive Diagnostic Computerized Test
%A Huebner, Alan
%A Finkelman, Matthew D.
%A Weissman, Alexander
%B Journal of Computerized Adaptive Testing
%V 6
%P 1-14
%U http://iacat.org/jcat/index.php/jcat/article/view/55/30
%N 1
%R 10.7333/1802-060101
%0 Journal Article
%J Applied Psychological Measurement
%D 2016
%T On Computing the Key Probability in the Stochastically Curtailed Sequential Probability Ratio Test
%A Huebner, Alan R.
%A Finkelman, Matthew D.
%X The Stochastically Curtailed Sequential Probability Ratio Test (SCSPRT) is a termination criterion for computerized classification tests (CCTs) that has been shown to be more efficient than the well-known Sequential Probability Ratio Test (SPRT). The performance of the SCSPRT depends on computing the probability that at a given stage in the test, an examinee’s current interim classification status will not change before the end of the test. Previous work discusses two methods of computing this probability, an exact method in which all potential responses to remaining items are considered and an approximation based on the central limit theorem (CLT) requiring less computation. Generally, the CLT method should be used early in the test when the number of remaining items is large, and the exact method is more appropriate at later stages of the test when few items remain. However, there is currently a dearth of information as to the performance of the SCSPRT when using the two methods. For the first time, the exact and CLT methods of computing the crucial probability are compared in a simulation study to explore whether there is any effect on the accuracy or efficiency of the CCT. The article is focused toward practitioners and researchers interested in using the SCSPRT as a termination criterion in an operational CCT.
%B Applied Psychological Measurement
%V 40
%P 142-156
%U http://apm.sagepub.com/content/40/2/142.abstract
%R 10.1177/0146621615611633
%0 Journal Article
%J Applied Psychological Measurement
%D 2016
%T Stochastic Curtailment of Questionnaires for Three-Level Classification: Shortening the CES-D for Assessing Low, Moderate, and High Risk of Depression
%A Smits, Niels
%A Finkelman, Matthew D.
%A Kelderman, Henk
%X In clinical assessment, efficient screeners are needed to ensure low respondent burden. In this article, Stochastic Curtailment (SC), a method for efficient computerized testing for classification into two classes for observable outcomes, was extended to three classes. In a post hoc simulation study using the item scores on the Center for Epidemiologic Studies–Depression Scale (CES-D) of a large sample, three versions of SC, SC via Empirical Proportions (SC-EP), SC via Simple Ordinal Regression (SC-SOR), and SC via Multiple Ordinal Regression (SC-MOR) were compared at both respondent burden and classification accuracy. All methods were applied under the regular item order of the CES-D and under an ordering that was optimal in terms of the predictive power of the items. Under the regular item ordering, the three methods were equally accurate, but SC-SOR and SC-MOR needed less items. Under the optimal ordering, additional gains in efficiency were found, but SC-MOR suffered from capitalization on chance substantially. It was concluded that SC-SOR is an efficient and accurate method for clinical screening. Strengths and weaknesses of the methods are discussed.
%B Applied Psychological Measurement
%V 40
%P 22-36
%U http://apm.sagepub.com/content/40/1/22.abstract
%R 10.1177/0146621615592294
%0 Journal Article
%J Applied Psychological Measurement
%D 2015
%T Stochastic Curtailment in Adaptive Mastery Testing: Improving the Efficiency of Confidence Interval–Based Stopping Rules
%A Sie, Haskell
%A Finkelman, Matthew D.
%A Bartroff, Jay
%A Thompson, Nathan A.
%X A well-known stopping rule in adaptive mastery testing is to terminate the assessment once the examinee’s ability confidence interval lies entirely above or below the cut-off score. This article proposes new procedures that seek to improve such a variable-length stopping rule by coupling it with curtailment and stochastic curtailment. Under the new procedures, test termination can occur earlier if the probability is high enough that the current classification decision remains the same should the test continue. Computation of this probability utilizes normality of an asymptotically equivalent version of the maximum likelihood ability estimate. In two simulation sets, the new procedures showed a substantial reduction in average test length while maintaining similar classification accuracy to the original method.
%B Applied Psychological Measurement
%V 39
%P 278-292
%U http://apm.sagepub.com/content/39/4/278.abstract
%R 10.1177/0146621614561314
%0 Journal Article
%J Applied Psychological Measurement
%D 2015
%T Utilizing Response Times in Computerized Classification Testing
%A Sie, Haskell
%A Finkelman, Matthew D.
%A Riley, Barth
%A Smits, Niels
%X A well-known approach in computerized mastery testing is to combine the Sequential Probability Ratio Test (SPRT) stopping rule with item selection to maximize Fisher information at the mastery threshold. This article proposes a new approach in which a time limit is defined for the test and examinees’ response times are considered in both item selection and test termination. Item selection is performed by maximizing Fisher information per time unit, rather than Fisher information itself. The test is terminated once the SPRT makes a classification decision, the time limit is exceeded, or there is no remaining item that has a high enough probability of being answered before the time limit. In a simulation study, the new procedure showed a substantial reduction in average testing time while slightly improving classification accuracy compared with the original method. In addition, the new procedure reduced the percentage of examinees who exceeded the time limit.
%B Applied Psychological Measurement
%V 39
%P 389-405
%U http://apm.sagepub.com/content/39/5/389.abstract
%R 10.1177/0146621615569504