01430nas a2200133 4500008003900000245009900039210006900138300000900207490000700216520097800223100002301201700002101224856005101245 2007 d00aAutomated Simultaneous Assembly of Multistage Testlets for a High-Stakes Licensing Examination0 aAutomated Simultaneous Assembly of Multistage Testlets for a Hig a5-200 v673 a

Many challenges exist for high-stakes testing programs offering continuous computerized administration. The automated assembly of test questions to exactly meet content and other requirements, provide uniformity, and control item exposure can be modeled and solved by mixed-integer programming (MIP) methods. A case study of the computerized licensing examination of the American Institute of Certified Public Accountants is offered as one application of MIP techniques for test assembly. The solution illustrates assembly for a computer-adaptive multistage testing design. However, the general form of the constraint-based solution can be modified to generate optimal test designs for paper-based or computerized administrations, regardless of the specific psychometric model. An extension of this methodology allows for long-term planning for the production and use of test content on the basis of an exact psychometric test designs and administration schedules.

1 aBreithaupt, Krista1 aHare, Donovan, R uhttp://epm.sagepub.com/content/67/1/5.abstract01549nas a2200169 4500008003900000022001400039245006100053210006100114300001400175490000700189520104600196100001901242700002301261700002201284700001801306856005501324 2007 d a1745-398400aDetecting Differential Speededness in Multistage Testing0 aDetecting Differential Speededness in Multistage Testing a117–1300 v443 a

A potential undesirable effect of multistage testing is differential speededness, which happens if some of the test takers run out of time because they receive subtests with items that are more time intensive than others. This article shows how a probabilistic response-time model can be used for estimating differences in time intensities and speed between subtests and test takers and detecting differential speededness. An empirical data set for a multistage test in the computerized CPA Exam was used to demonstrate the procedures. Although the more difficult subtests appeared to have items that were more time intensive than the easier subtests, an analysis of the residual response times did not reveal any significant differential speededness because the time limit appeared to be appropriate. In a separate analysis, within each of the subtests, we found minor but consistent patterns of residual times that are believed to be due to a warm-up effect, that is, use of more time on the initial items than they actually need.

1 aLinden, Wim, J1 aBreithaupt, Krista1 aChuah, Siang Chee1 aZhang, Yanwei uhttp://dx.doi.org/10.1111/j.1745-3984.2007.00030.x01482nas a2200145 4500008003900000245006500039210006500104300001200169490000700181520102700188100002001215700002501235700002301260856005301283 2006 d00aOptimal Testlet Pool Assembly for Multistage Testing Designs0 aOptimal Testlet Pool Assembly for Multistage Testing Designs a204-2150 v303 a

Computerized multistage testing (MST) designs require sets of test questions (testlets) to be assembled to meet strict, often competing criteria. Rules that govern testlet assembly may dictate the number of questions on a particular subject or may describe desirable statistical properties for the test, such as measurement precision. In an MST design, testlets of differing difficulty levels must be created. Statistical properties for assembly of the testlets can be expressed using item response theory (IRT) parameters. The testlet test information function (TIF) value can be maximized at a specific point on the IRT ability scale. In practical MST designs, parallel versions of testlets are needed, so sets of testlets with equivalent properties are built according to equivalent specifications. In this project, the authors study the use of a mathematical programming technique to simultaneously assemble testlets to ensure equivalence and fairness to candidates who may be administered different testlets.

1 aAriel, Adelaide1 aVeldkamp, Bernard, P1 aBreithaupt, Krista uhttp://apm.sagepub.com/content/30/3/204.abstract00444nas a2200133 4500008003900000245006200039210006000101300001200161490000700173100002000180700002100200700002300221856006600244 2006 d00aA testlet assembly design for the uniform CPA Examination0 atestlet assembly design for the uniform CPA Examination a189-2020 v191 aLuecht, Richard1 aBrumfield, Terry1 aBreithaupt, Krista uhttp://www.tandfonline.com/doi/abs/10.1207/s15324818ame1903_200443nas a2200133 4500008003900000245005900039210005900098300001200157490000600169100002300175700002000198700002500218856006600243 2005 d00aAutomated Simultaneous Assembly for Multistage Testing0 aAutomated Simultaneous Assembly for Multistage Testing a319-3300 v51 aBreithaupt, Krista1 aAriel, Adelaide1 aVeldkamp, Bernard, P uhttp://www.tandfonline.com/doi/abs/10.1207/s15327574ijt0503_8