00564nas a2200145 4500008003900000245008100039210006900120100002400189700002200213700001600235700001900251700002300270700002500293856010000318 2018 d00aMeasuring patient-reported outcomes adaptively: Multidimensionality matters!0 aMeasuring patientreported outcomes adaptively Multidimensionalit1 aPaap, Muirne, C. S.1 aKroeze, Karel, A.1 aGlas, C A W1 aTerwee, C., B.1 avan der Palen, Job1 aVeldkamp, Bernard, P uhttp://www.iacat.org/measuring-patient-reported-outcomes-adaptively-multidimensionality-matters00592nas a2200169 4500008003900000022001400039245011200053210006900165300001600234490000700250100002400257700002200281700002500303700002300328700002500351856004600376 2017 d a1573-264900aItem usage in a multidimensional computerized adaptive test (MCAT) measuring health-related quality of life0 aItem usage in a multidimensional computerized adaptive test MCAT a2909–29180 v261 aPaap, Muirne, C. S.1 aKroeze, Karel, A.1 aTerwee, Caroline, B.1 avan der Palen, Job1 aVeldkamp, Bernard, P uhttps://doi.org/10.1007/s11136-017-1624-300491nas a2200133 4500008003900000022001400039245010800053210006900161300000700230490000600237100002500243700002400268856006500292 2017 d a2504-284X00aRobust Automated Test Assembly for Testlet-Based Tests: An Illustration with Analytical Reasoning Items0 aRobust Automated Test Assembly for TestletBased Tests An Illustr a630 v21 aVeldkamp, Bernard, P1 aPaap, Muirne, C. S. uhttps://www.frontiersin.org/article/10.3389/feduc.2017.0006301586nas a2200133 4500008003900000022001400039245008800053210006900141300001400210490000700224520115500231100002501386856004101411 2016 d a1745-398400aOn the Issue of Item Selection in Computerized Adaptive Testing With Response Times0 aIssue of Item Selection in Computerized Adaptive Testing With Re a212–2280 v533 aMany standardized tests are now administered via computer rather than paper-and-pencil format. The computer-based delivery mode brings with it certain advantages. One advantage is the ability to adapt the difficulty level of the test to the ability level of the test taker in what has been termed computerized adaptive testing (CAT). A second advantage is the ability to record not only the test taker's response to each item (i.e., question), but also the amount of time the test taker spends considering and answering each item. Combining these two advantages, various methods were explored for utilizing response time data in selecting appropriate items for an individual test taker.Four strategies for incorporating response time data were evaluated, and the precision of the final test-taker score was assessed by comparing it to a benchmark value that did not take response time information into account. While differences in measurement precision and testing times were expected, results showed that the strategies did not differ much with respect to measurement precision but that there were differences with regard to the total testing time.1 aVeldkamp, Bernard, P uhttp://dx.doi.org/10.1111/jedm.1211001549nas a2200145 4500008003900000245010400039210006900143300001200212490000700224520104200231100002601273700002601299700002501325856005301350 2016 d00aMultidimensional Computerized Adaptive Testing for Classifying Examinees With Within-Dimensionality0 aMultidimensional Computerized Adaptive Testing for Classifying E a387-4040 v403 aA classification method is presented for adaptive classification testing with a multidimensional item response theory (IRT) model in which items are intended to measure multiple traits, that is, within-dimensionality. The reference composite is used with the sequential probability ratio test (SPRT) to make decisions and decide whether testing can be stopped before reaching the maximum test length. Item-selection methods are provided that maximize the determinant of the information matrix at the cutoff point or at the projected ability estimate. A simulation study illustrates the efficiency and effectiveness of the classification method. Simulations were run with the new item-selection methods, random item selection, and maximization of the determinant of the information matrix at the ability estimate. The study also showed that the SPRT with multidimensional IRT has the same characteristics as the SPRT with unidimensional IRT and results in more accurate classifications than the latter when used for multidimensional data.1 avan Groen, Maaike, M.1 aEggen, Theo, J. H. M.1 aVeldkamp, Bernard, P uhttp://apm.sagepub.com/content/40/6/387.abstract01545nas a2200145 4500008003900000245011500039210006900154300001200223490000700235520102700242100002601269700002601295700002501321856005301346 2014 d00aItem Selection Methods Based on Multiple Objective Approaches for Classifying Respondents Into Multiple Levels0 aItem Selection Methods Based on Multiple Objective Approaches fo a187-2000 v383 a
Computerized classification tests classify examinees into two or more levels while maximizing accuracy and minimizing test length. The majority of currently available item selection methods maximize information at one point on the ability scale, but in a test with multiple cutting points selection methods could take all these points simultaneously into account. If for each cutting point one objective is specified, the objectives can be combined into one optimization function using multiple objective approaches. Simulation studies were used to compare the efficiency and accuracy of eight selection methods in a test based on the sequential probability ratio test. Small differences were found in accuracy and efficiency between different methods depending on the item pool and settings of the classification method. The size of the indifference region had little influence on accuracy but considerable influence on efficiency. Content and exposure control had little influence on accuracy and efficiency.
1 avan Groen, Maaike, M.1 aEggen, Theo, J. H. M.1 aVeldkamp, Bernard, P uhttp://apm.sagepub.com/content/38/3/187.abstract01575nas a2200145 4500008003900000245008500039210006900124300001200193490000700205520109100212100002501303700002701328700002101355856005301376 2013 d00aUncertainties in the Item Parameter Estimates and Robust Automated Test Assembly0 aUncertainties in the Item Parameter Estimates and Robust Automat a123-1390 v373 aItem response theory parameters have to be estimated, and because of the estimation process, they do have uncertainty in them. In most large-scale testing programs, the parameters are stored in item banks, and automated test assembly algorithms are applied to assemble operational test forms. These algorithms treat item parameters as fixed values, and uncertainty is not taken into account. As a consequence, resulting tests might be off target or less informative than expected. In this article, the process of parameter estimation is described to provide insight into the causes of uncertainty in the item parameters. The consequences of uncertainty are studied. Besides, an alternative automated test assembly algorithm is presented that is robust against uncertainties in the data. Several numerical examples demonstrate the performance of the robust test assembly algorithm, and illustrate the consequences of not taking this uncertainty into account. Finally, some recommendations about the use of robust test assembly and some directions for further research are given.
1 aVeldkamp, Bernard, P1 aMatteucci, Mariagiulia1 aJong, Martijn, G uhttp://apm.sagepub.com/content/37/2/123.abstract01773nas a2200145 4500008003900000245006900039210006900108300001000177490000700187520131500194100002501509700002501534700001601559856005201575 2009 d00aMultiple Maximum Exposure Rates in Computerized Adaptive Testing0 aMultiple Maximum Exposure Rates in Computerized Adaptive Testing a58-730 v333 aComputerized adaptive testing is subject to security problems, as the item bank content remains operative over long periods and administration time is flexible for examinees. Spreading the content of a part of the item bank could lead to an overestimation of the examinees' trait level. The most common way of reducing this risk is to impose a maximum exposure rate (rmax) that no item should exceed. Several methods have been proposed with this aim. All of these methods establish a single value of rmax throughout the test. This study presents a new method, the multiple-rmax method, that defines as many values of rmax as the number of items presented in the test. In this way, it is possible to impose a high degree of randomness in item selection at the beginning of the test, leaving the administration of items with the best psychometric properties to the moment when the trait level estimation is most accurate. The implementation of the multiple-r max method is described and is tested in simulated item banks and in an operative bank. Compared with a single maximum exposure method, the new method has a more balanced usage of the item bank and delays the possible distortion of trait estimation due to security problems, with either no or only slight decrements of measurement accuracy.
1 aBarrada, Juan Ramón1 aVeldkamp, Bernard, P1 aOlea, Julio uhttp://apm.sagepub.com/content/33/1/58.abstract00469nas a2200121 4500008003900000245011200039210006900151300001200220490000600232100002500238700001900263856006500282 2008 d00aImplementing Sympson-Hetter Item-Exposure Control in a Shadow-Test Approach to Constrained Adaptive Testing0 aImplementing SympsonHetter ItemExposure Control in a ShadowTest a272-2890 v81 aVeldkamp, Bernard, P1 aLinden, Wim, J uhttp://www.tandfonline.com/doi/abs/10.1080/1530505080226223301361nas a2200133 4500008003900000245009700039210006900136300001200205490000700217520090200224100001901126700002501145856005701170 2007 d00aConditional Item-Exposure Control in Adaptive Testing Using Item-Ineligibility Probabilities0 aConditional ItemExposure Control in Adaptive Testing Using ItemI a398-4180 v323 aTwo conditional versions of the exposure-control method with item-ineligibility constraints for adaptive testing in van der Linden and Veldkamp (2004) are presented. The first version is for unconstrained item selection, the second for item selection with content constraints imposed by the shadow-test approach. In both versions, the exposure rates of the items are controlled using probabilities of item ineligibility given θ that adapt the exposure rates automatically to a goal value for the items in the pool. In an extensive empirical study with an adaptive version of the Law School Admission Test, the authors show how the method can be used to drive conditional exposure rates below goal values as low as 0.025. Obviously, the price to be paid for minimal exposure rates is a decrease in the accuracy of the ability estimates. This trend is illustrated with empirical data.
1 aLinden, Wim, J1 aVeldkamp, Bernard, P uhttp://jeb.sagepub.com/cgi/content/abstract/32/4/39801482nas a2200145 4500008003900000245006500039210006500104300001200169490000700181520102700188100002001215700002501235700002301260856005301283 2006 d00aOptimal Testlet Pool Assembly for Multistage Testing Designs0 aOptimal Testlet Pool Assembly for Multistage Testing Designs a204-2150 v303 aComputerized multistage testing (MST) designs require sets of test questions (testlets) to be assembled to meet strict, often competing criteria. Rules that govern testlet assembly may dictate the number of questions on a particular subject or may describe desirable statistical properties for the test, such as measurement precision. In an MST design, testlets of differing difficulty levels must be created. Statistical properties for assembly of the testlets can be expressed using item response theory (IRT) parameters. The testlet test information function (TIF) value can be maximized at a specific point on the IRT ability scale. In practical MST designs, parallel versions of testlets are needed, so sets of testlets with equivalent properties are built according to equivalent specifications. In this project, the authors study the use of a mathematical programming technique to simultaneously assemble testlets to ensure equivalence and fairness to candidates who may be administered different testlets.
1 aAriel, Adelaide1 aVeldkamp, Bernard, P1 aBreithaupt, Krista uhttp://apm.sagepub.com/content/30/3/204.abstract00443nas a2200133 4500008003900000245005900039210005900098300001200157490000600169100002300175700002000198700002500218856006600243 2005 d00aAutomated Simultaneous Assembly for Multistage Testing0 aAutomated Simultaneous Assembly for Multistage Testing a319-3300 v51 aBreithaupt, Krista1 aAriel, Adelaide1 aVeldkamp, Bernard, P uhttp://www.tandfonline.com/doi/abs/10.1207/s15327574ijt0503_801302nas a2200133 4500008003900000245008200039210006900121300001200190490000700202520085800209100001901067700002501086856005701111 2004 d00aConstraining Item Exposure in Computerized Adaptive Testing With Shadow Tests0 aConstraining Item Exposure in Computerized Adaptive Testing With a273-2910 v293 aItem-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter’s (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. The method does not, however, require time-consuming simulation studies to set values for control parameters before the operational use of the test. Instead, it can set the probabilities of item ineligibility adaptively during the test using the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test showed that application of the method yielded perfect control of the item-exposure rates and had negligible impact on the bias and mean-squared error functions of the ability estimator.
1 aLinden, Wim, J1 aVeldkamp, Bernard, P uhttp://jeb.sagepub.com/cgi/content/abstract/29/3/273