01210nas a2200133 4500008003900000245008600039210006900125300001200194490000700206520076400213100002100977700002500998856005301023 2009 d00aDirect and Inverse Problems of Item Pool Design for Computerized Adaptive Testing0 aDirect and Inverse Problems of Item Pool Design for Computerized a533-5470 v693 a
The recent literature on computerized adaptive testing (CAT) has developed methods for creating CAT item pools from a large master pool. Each CAT pool is designed as a set of nonoverlapping forms reflecting the skill levels of an assumed population of test takers. This article presents a Monte Carlo method to obtain these CAT pools and discusses its advantages over existing methods. Also, a new problem is considered that finds a population ability density function best matching the master pool. An analysis of the solution to this new problem provides testing organizations with effective guidance for maintaining their master pools. Computer experiments with a pool of Law School Admission Test items and its assembly constraints are presented.
1 aBelov, Dmitry, I1 aArmstrong, Ronald, D uhttp://epm.sagepub.com/content/69/4/533.abstract01234nas a2200145 4500008003900000245007300039210006900112300001200181490000700193520076500200100002100965700002500986700002401011856005301035 2008 d00aA Monte Carlo Approach for Adaptive Testing With Content Constraints0 aMonte Carlo Approach for Adaptive Testing With Content Constrain a431-4460 v323 aThis article presents a new algorithm for computerized adaptive testing (CAT) when content constraints are present. The algorithm is based on shadow CAT methodology to meet content constraints but applies Monte Carlo methods and provides the following advantages over shadow CAT: (a) lower maximum item exposure rates, (b) higher utilization of the item pool, and (c) more robust ability estimates. Computer simulations with Law School Admission Test items demonstrated that the new algorithm (a) produces similar ability estimates as shadow CAT but with half the maximum item exposure rate and 100% pool utilization and (b) produces more robust estimates when a high- (or low-) ability examinee performs poorly (or well) at the beginning of the test.
1 aBelov, Dmitry, I1 aArmstrong, Ronald, D1 aWeissman, Alexander uhttp://apm.sagepub.com/content/32/6/431.abstract01426nas a2200133 4500008003900000245009600039210006900135300001200204490000700216520097000223100002101193700002501214856005301239 2008 d00aA Monte Carlo Approach to the Design, Assembly, and Evaluation of Multistage Adaptive Tests0 aMonte Carlo Approach to the Design Assembly and Evaluation of Mu a119-1370 v323 aThis article presents an application of Monte Carlo methods for developing and assembling multistage adaptive tests (MSTs). A major advantage of the Monte Carlo assembly over other approaches (e.g., integer programming or enumerative heuristics) is that it provides a uniform sampling from all MSTs (or MST paths) available from a given item pool. The uniform sampling allows a statistically valid analysis for MST design and evaluation. Given an item pool, MST model, and content constraints for test assembly, three problems are addressed in this study. They are (a) the construction of item response theory (IRT) targets for each MST path, (b) the assembly of an MST such that each path satisfies content constraints and IRT constraints, and (c) an analysis of the pool and constraints to increase the number of nonoverlapping MSTs that can be assembled from the pool. The primary intent is to produce reliable measurements and enhance pool utilization.
1 aBelov, Dmitry, I1 aArmstrong, Ronald, D uhttp://apm.sagepub.com/content/32/2/119.abstract01198nas a2200133 4500008003900000245006700039210006700106300001200173490000700185520077300192100002100965700002500986856005301011 2005 d00aMonte Carlo Test Assembly for Item Pool Analysis and Extension0 aMonte Carlo Test Assembly for Item Pool Analysis and Extension a239-2610 v293 aA new test assembly algorithm based on a Monte Carlo random search is presented in this article. A major advantage of the Monte Carlo test assembly over other approaches (integer programming or enumerative heuristics) is that it performs a uniform sampling from the item pool, which provides every feasible item combination (test) with an equal chance of being built during an assembly. This allows the authors to address the following issues of pool analysis and extension: compare the strengths and weaknesses of different pools, identify the most restrictive constraint(s) for test assembly, and identify properties of the items that should be added to a pool to achieve greater usability of the pool. Computer experiments with operational pools are given.
1 aBelov, Dmitry, I1 aArmstrong, Ronald, D uhttp://apm.sagepub.com/content/29/4/239.abstract01251nas a2200157 4500008003900000245006400039210006300103300001200166490000700178520076400185100002500949700002200974700002200996700002201018856005301040 2004 d00aComputerized Adaptive Testing With Multiple-Form Structures0 aComputerized Adaptive Testing With MultipleForm Structures a147-1640 v283 aA multiple-form structure (MFS) is an orderedcollection or network of testlets (i.e., sets of items).An examinee’s progression through the networkof testlets is dictated by the correctness of anexaminee’s answers, thereby adapting the test tohis or her trait level. The collection of pathsthrough the network yields the set of all possibletest forms, allowing test specialists the opportunityto review them before they are administered. Also,limiting the exposure of an individual MFS to aspecific period of time can enhance test security.This article provides an overview of methods thathave been developed to generate parallel MFSs.The approach is applied to the assembly of anexperimental computerized Law School Admission Test (LSAT).
1 aArmstrong, Ronald, D1 aJones, Douglas, H1 aKoppel, Nicole, B1 aPashley, Peter, J uhttp://apm.sagepub.com/content/28/3/147.abstract