%0 Journal Article %J Psicologica %D 2010 %T Bayesian item selection in constrained adaptive testing %A Veldkamp, B. P. %K computerized adaptive testing %X Application of Bayesian item selection criteria in computerized adaptive testing might result in improvement of bias and MSE of the ability estimates. The question remains how to apply Bayesian item selection criteria in the context of constrained adaptive testing, where large numbers of specifications have to be taken into account in the item selection process. The Shadow Test Approach is a general purpose algorithm for administering constrained CAT. In this paper it is shown how the approach can be slightly modified to handle Bayesian item selection criteria. No differences in performance were found between the shadow test approach and the modifiedapproach. In a simulation study of the LSAT, the effects of Bayesian item selection criteria are illustrated. The results are compared to item selection based on Fisher Information. General recommendations about the use of Bayesian item selection criteria are provided. %B Psicologica %V 31 %P 149-169 %G eng %0 Book Section %B Elements of Adaptive Testing %D 2010 %T Designing Item Pools for Adaptive Testing %A Veldkamp, B. P. %A van der Linden, W. J. %B Elements of Adaptive Testing %P 231-245 %G eng %& 12 %R 10.1007/978-0-387-85461-8 %0 Journal Article %J Personality and Individual Differences %D 2010 %T Detection of aberrant item score patterns in computerized adaptive testing: An empirical example using the CUSUM %A Egberink, I. J. L. %A Meijer, R. R. %A Veldkamp, B. P. %A Schakel, L. %A Smid, N. G. %K CAT %K computerized adaptive testing %K CUSUM approach %K person Fit %X The scalability of individual trait scores on a computerized adaptive test (CAT) was assessed through investigating the consistency of individual item score patterns. A sample of N = 428 persons completed a personality CAT as part of a career development procedure. To detect inconsistent item score patterns, we used a cumulative sum (CUSUM) procedure. Combined information from the CUSUM, other personality measures, and interviews showed that similar estimated trait values may have a different interpretation.Implications for computer-based assessment are discussed. %B Personality and Individual Differences %V 48 %P 921-925 %@ 01918869 %G eng %0 Journal Article %J Psicothema %D 2009 %T Comparison of methods for controlling maximum exposure rates in computerized adaptive testing %A Barrada, J %A Abad, F. J. %A Veldkamp, B. P. %K *Numerical Analysis, Computer-Assisted %K Psychological Tests/*standards/*statistics & numerical data %X This paper has two objectives: (a) to provide a clear description of three methods for controlling the maximum exposure rate in computerized adaptive testing —the Symson-Hetter method, the restricted method, and the item-eligibility method— showing how all three can be interpreted as methods for constructing the variable sub-bank of items from which each examinee receives the items in his or her test; (b) to indicate the theoretical and empirical limitations of each method and to compare their performance. With the three methods, we obtained basically indistinguishable results in overlap rate and RMSE (differences in the third decimal place). The restricted method is the best method for controlling exposure rate, followed by the item-eligibility method. The worst method is the Sympson-Hetter method. The restricted method presents problems of sequential overlap rate. Our advice is to use the item-eligibility method, as it saves time and satisfies the goals of restricting maximum exposure. Comparación de métodos para el control de tasa máxima en tests adaptativos informatizados. Este artículo tiene dos objetivos: (a) ofrecer una descripción clara de tres métodos para el control de la tasa máxima en tests adaptativos informatizados, el método Symson-Hetter, el método restringido y el métodode elegibilidad del ítem, mostrando cómo todos ellos pueden interpretarse como métodos para la construcción del subbanco de ítems variable, del cual cada examinado recibe los ítems de su test; (b) señalar las limitaciones teóricas y empíricas de cada método y comparar sus resultados. Se obtienen resultados básicamente indistinguibles en tasa de solapamiento y RMSE con los tres métodos (diferencias en la tercera posición decimal). El método restringido es el mejor en el control de la tasa de exposición,seguido por el método de elegibilidad del ítem. El peor es el método Sympson-Hetter. El método restringido presenta un problema de solapamiento secuencial. Nuestra recomendación sería utilizar el método de elegibilidad del ítem, puesto que ahorra tiempo y satisface los objetivos de limitar la tasa máxima de exposición. %B Psicothema %7 2009/05/01 %V 21 %P 313-320 %8 May %@ 0214-9915 (Print)0214-9915 (Linking) %G eng %M 19403088 %0 Book Section %D 2007 %T The development of a computerized adaptive test for integrity %A Egberink, I. J. L. %A Veldkamp, B. P. %C D. J. Weiss (Ed.), Proceedings of the 2007 GMAC Conference on Computerized Adaptive Testing. %G eng %0 Report %D 2007 %T A multiple objective test assembly approach for exposure control problems in computerized adaptive testing %A Veldkamp, B. P. %A Verschoor, Angela J. %A Theo Eggen %B Measurement and Research Department Reports %I Cito %C Arnhem, The Netherlands %G eng %0 Journal Article %J Journal of Educational and Behavioral Statistics %D 2006 %T Assembling a computerized adaptive testing item pool as a set of linear tests %A van der Linden, W. J. %A Ariel, A. %A Veldkamp, B. P. %K Algorithms %K computerized adaptive testing %K item pool %K linear tests %K mathematical models %K statistics %K Test Construction %K Test Items %X Test-item writing efforts typically results in item pools with an undesirable correlational structure between the content attributes of the items and their statistical information. If such pools are used in computerized adaptive testing (CAT), the algorithm may be forced to select items with less than optimal information, that violate the content constraints, and/or have unfavorable exposure rates. Although at first sight somewhat counterintuitive, it is shown that if the CAT pool is assembled as a set of linear test forms, undesirable correlations can be broken down effectively. It is proposed to assemble such pools using a mixed integer programming model with constraints that guarantee that each test meets all content specifications and an objective function that requires them to have maximal information at a well-chosen set of ability values. An empirical example with a previous master pool from the Law School Admission Test (LSAT) yielded a CAT with nearly uniform bias and mean-squared error functions for the ability estimator and item-exposure rates that satisfied the target for all items in the pool. %B Journal of Educational and Behavioral Statistics %I Sage Publications: US %V 31 %P 81-99 %@ 1076-9986 (Print) %G eng %M 2007-08137-004 %0 Conference Paper %B Paper presented at the SMABS-EAM Conference %D 2006 %T Multiple maximum exposure rates in computerized adaptive testing %A Barrada, J %A Veldkamp, B. P. %A Olea, J. %B Paper presented at the SMABS-EAM Conference %C Budapest, Hungary %G eng %0 Generic %D 2005 %T Constraining item exposure in computerized adaptive testing with shadow tests %A van der Linden, W. J. %A Veldkamp, B. P. %C Law School Admission Council Computerized Testing Report 02-03 %G eng %0 Journal Article %J Journal of Educational Measurement %D 2005 %T Infeasibility in automated test assembly models: A comparison study of different methods %A Huitzing, H. A. %A Veldkamp, B. P. %A Verschoor, A. J. %K Algorithms %K Item Content (Test) %K Models %K Test Construction %X Several techniques exist to automatically put together a test meeting a number of specifications. In an item bank, the items are stored with their characteristics. A test is constructed by selecting a set of items that fulfills the specifications set by the test assembler. Test assembly problems are often formulated in terms of a model consisting of restrictions and an objective to be maximized or minimized. A problem arises when it is impossible to construct a test from the item pool that meets all specifications, that is, when the model is not feasible. Several methods exist to handle these infeasibility problems. In this article, test assembly models resulting from two practical testing programs were reconstructed to be infeasible. These models were analyzed using methods that forced a solution (Goal Programming, Multiple-Goal Programming, Greedy Heuristic), that analyzed the causes (Relaxed and Ordered Deletion Algorithm (RODA), Integer Randomized Deletion Algorithm (IRDA), Set Covering (SC), and Item Sampling), or that analyzed the causes and used this information to force a solution (Irreducible Infeasible Set-Solver). Specialized methods such as the IRDA and the Irreducible Infeasible Set-Solver performed best. Recommendations about the use of different methods are given. (PsycINFO Database Record (c) 2005 APA ) (journal abstract) %B Journal of Educational Measurement %V 42 %P 223-243 %G eng %0 Journal Article %J Journal of Educational and Behavioral Statistics %D 2004 %T Constraining item exposure in computerized adaptive testing with shadow tests %A van der Linden, W. J. %A Veldkamp, B. P. %K computerized adaptive testing %K item exposure control %K item ineligibility constraints %K Probability %K shadow tests %X Item-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter’s (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. The method does not, however, require time-consuming simulation studies to set values for control parameters before the operational use of the test. Instead, it can set the probabilities of item ineligibility adaptively during the test using the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test showed that application of the method yielded perfect control of the item-exposure rates and had negligible impact on the bias and mean-squared error functions of the ability estimator. %B Journal of Educational and Behavioral Statistics %I American Educational Research Assn: US %V 29 %P 273-291 %@ 1076-9986 (Print) %G eng %M 2006-01687-001 %0 Journal Article %J Journal of Educational Measurement %D 2004 %T Constructing rotating item pools for constrained adaptive testing %A Ariel, A. %A Veldkamp, B. P. %A van der Linden, W. J. %K computerized adaptive tests %K constrained adaptive testing %K item exposure %K rotating item pools %X Preventing items in adaptive testing from being over- or underexposed is one of the main problems in computerized adaptive testing. Though the problem of overexposed items can be solved using a probabilistic item-exposure control method, such methods are unable to deal with the problem of underexposed items. Using a system of rotating item pools, on the other hand, is a method that potentially solves both problems. In this method, a master pool is divided into (possibly overlapping) smaller item pools, which are required to have similar distributions of content and statistical attributes. These pools are rotated among the testing sites to realize desirable exposure rates for the items. A test assembly model, motivated by Gulliksen's matched random subtests method, was explored to help solve the problem of dividing a master pool into a set of smaller pools. Different methods to solve the model are proposed. An item pool from the Law School Admission Test was used to evaluate the performances of computerized adaptive tests from systems of rotating item pools constructed using these methods. (PsycINFO Database Record (c) 2007 APA, all rights reserved) %B Journal of Educational Measurement %I Blackwell Publishing: United Kingdom %V 41 %P 345-359 %@ 0022-0655 (Print) %G eng %M 2004-21596-004 %0 Conference Paper %B Paper presented at the Annual meeting of the National Council on Measurement in Education %D 2003 %T Constraining item exposure in computerized adaptive testing with shadow tests %A van der Linden, W. J. %A Veldkamp, B. P. %B Paper presented at the Annual meeting of the National Council on Measurement in Education %C Chicago IL %G eng %0 Conference Paper %D 2003 %T Controlling item exposure and item eligibility in computerized adaptive testing %A van der Linden, W. J. %A Veldkamp, B. P. %G eng %0 Conference Paper %B Paper presented at the Annual meeting of the National Council on Measurement in Education %D 2003 %T Implementing an alternative to Sympson-Hetter item-exposure control in constrained adaptive testing %A Veldkamp, B. P. %A van der Linden, W. J. %B Paper presented at the Annual meeting of the National Council on Measurement in Education %C Chicago IL %G eng %0 Book Section %B New developments in psychometrics %D 2003 %T Item selection in polytomous CAT %A Veldkamp, B. P. %E A. Okada %E K. Shigenasu %E Y. Kano %E J. Meulman %K computerized adaptive testing %B New developments in psychometrics %I Psychometric Society, Springer %C Tokyo, Japan %P 207–214 %G eng %0 Book Section %D 2003 %T Item selection in polytomous CAT %A Veldkamp, B. P. %C H. Yanai, A. Okada, K. Shigemasu, Y Kano, and J. J. Meulman (eds.), New developments in psychometrics (pp. 207-214). Tokyo, Japan: Springer-Verlag. %G eng %0 Generic %D 2002 %T Constraining item exposure in computerized adaptive testing with shadow tests (Research Report No. 02-06) %A van der Linden, W. J. %A Veldkamp, B. P. %C University of Twente, The Netherlands %G eng %0 Report %D 2002 %T Mathematical-programming approaches to test item pool design %A Veldkamp, B. P. %A van der Linden, W. J. %A Ariel, A. %K Adaptive Testing %K Computer Assisted %K Computer Programming %K Educational Measurement %K Item Response Theory %K Mathematics %K Psychometrics %K Statistical Rotation computerized adaptive testing %K Test Items %K Testing %X (From the chapter) This paper presents an approach to item pool design that has the potential to improve on the quality of current item pools in educational and psychological testing and hence to increase both measurement precision and validity. The approach consists of the application of mathematical programming techniques to calculate optimal blueprints for item pools. These blueprints can be used to guide the item-writing process. Three different types of design problems are discussed, namely for item pools for linear tests, item pools computerized adaptive testing (CAT), and systems of rotating item pools for CAT. The paper concludes with an empirical example of the problem of designing a system of rotating item pools for CAT. %I University of Twente, Faculty of Educational Science and Technology %C Twente, The Netherlands %P 93-108 %@ 02-09 %G eng %0 Journal Article %J Psychometrika %D 2002 %T Multidimensional adaptive testing with constraints on test content %A Veldkamp, B. P. %A van der Linden, W. J. %X The case of adaptive testing under a multidimensional response model with large numbers of constraints on the content of the test is addressed. The items in the test are selected using a shadow test approach. The 0–1 linear programming model that assembles the shadow tests maximizes posterior expected Kullback-Leibler information in the test. The procedure is illustrated for five different cases of multidimensionality. These cases differ in (a) the numbers of ability dimensions that are intentional or should be considered as ldquonuisance dimensionsrdquo and (b) whether the test should or should not display a simple structure with respect to the intentional ability dimensions. %B Psychometrika %V 67 %P 575-588 %G eng %0 Generic %D 2001 %T Implementing constrained CAT with shadow tests for large item pools %A Veldkamp, B. P. %C Submitted for publication %G eng %0 Book Section %B Computerized adaptive testing: Theory and practice %D 2000 %T Designing item pools for computerized adaptive testing %A Veldkamp, B. P. %A van der Linden, W. J. %B Computerized adaptive testing: Theory and practice %I Kluwer Academic Publishers %C Dendrecht, The Netherlands %P 149–162 %G eng %0 Journal Article %J Applied Psychological Measurement %D 2000 %T An integer programming approach to item bank design %A van der Linden, W. J. %A Veldkamp, B. P. %A Reese, L. M. %K Aptitude Measures %K Item Analysis (Test) %K Item Response Theory %K Test Construction %K Test Items %X An integer programming approach to item bank design is presented that can be used to calculate an optimal blueprint for an item bank, in order to support an existing testing program. The results are optimal in that they minimize the effort involved in producing the items as revealed by current item writing patterns. Also presented is an adaptation of the models, which can be used as a set of monitoring tools in item bank management. The approach is demonstrated empirically for an item bank that was designed for the Law School Admission Test. %B Applied Psychological Measurement %V 24 %P 139-150 %G eng %0 Generic %D 2000 %T Modifications of the branch-and-bound algorithm for application in constrained adaptive testing (Research Report 00-05) %A Veldkamp, B. P. %C Enschede, The Netherlands: University of Twente, Faculty of Educational Science and Technology, Department of Measurement and Data Analysis %G eng %0 Generic %D 2000 %T Multidimensional adaptive testing with constraints on test content (Research Report 00-11) %A Veldkamp, B. P. %A van der Linden, W. J. %C Enschede, The Netherlands: University of Twente, Faculty of Educational Science and Technology, Department of Measurement and Data Analysis %G eng %0 Generic %D 1999 %T Designing item pools for computerized adaptive testing (Research Report 99-03 ) %A Veldkamp, B. P. %A van der Linden, W. J. %C Enschede, The Netherlands: University of Twente, Faculty of Educational Science and Technology, Department of Measurement and Data Analysis %G eng