TY - JOUR T1 - Severity of Organized Item Theft in Computerized Adaptive Testing: A Simulation Study JF - Applied Psychological Measurement Y1 - 2008 A1 - Qing Yi, A1 - Jinming Zhang, A1 - Chang, Hua-Hua AB -

Criteria had been proposed for assessing the severity of possible test security violations for computerized tests with high-stakes outcomes. However, these criteria resulted from theoretical derivations that assumed uniformly randomized item selection. This study investigated potential damage caused by organized item theft in computerized adaptive testing (CAT) for two realistic item selection methods, maximum item information and a-stratified with content blocking, using the randomized method as a baseline for comparison. Damage caused by organized item theft was evaluated by the number of compromised items each examinee could encounter and the impact of the compromised items on examinees' ability estimates. Severity of test security violation was assessed under self-organized and organized item theft simulation scenarios. Results indicated that though item theft could cause severe damage to CAT with either item selection method, the maximum item information method was more vulnerable to the organized item theft simulation than was the a-stratified method.

VL - 32 UR - http://apm.sagepub.com/content/32/7/543.abstract ER - TY - JOUR T1 - Two-Phase Item Selection Procedure for Flexible Content Balancing in CAT JF - Applied Psychological Measurement Y1 - 2007 A1 - Ying Cheng, A1 - Chang, Hua-Hua A1 - Qing Yi, AB -

Content balancing is an important issue in the design and implementation of computerized adaptive testing (CAT). Content-balancing techniques that have been applied in fixed content balancing, where the number of items from each content area is fixed, include constrained CAT (CCAT), the modified multinomial model (MMM), modified constrained CAT (MCCAT), and others. In this article, four methods are proposed to address the flexible content-balancing issue with the a-stratification design, named STR_C. The four methods are MMM+, an extension of MMM; MCCAT+, an extension of MCCAT; the TPM method, a two-phase content-balancing method using MMM in both phases; and the TPF method, a two-phase content-balancing method using MMM in the first phase and MCCAT in the second. Simulation results show that all of the methods work well in content balancing, and TPF performs the best in item exposure control and item pool utilization while maintaining measurement precision.

VL - 31 UR - http://apm.sagepub.com/content/31/6/467.abstract ER -