03049nas a2200241 4500008004100000020004100041245017000082210007100252250001500323300001000338490000700348520214300355653001402498653006602512653001102578653001302589100001402602700001202616700001402628700001502642700001702657856013302674 2010 spa d a0214-9915 (Print)0214-9915 (Linking)00aDeterioro de parámetros de los ítems en tests adaptativos informatizados: estudio con eCAT [Item parameter drift in computerized adaptive testing: Study with eCAT]0 aDeterioro de parámetros de los ítems en tests adaptativos inform a2010/04/29 a340-70 v223 a
En el presente trabajo se muestra el análisis realizado sobre un Test Adaptativo Informatizado (TAI) diseñado para la evaluación del nivel de inglés, denominado eCAT, con el objetivo de estudiar el deterioro de parámetros (parameter drift) producido desde la calibración inicial del banco de ítems. Se ha comparado la calibración original desarrollada para la puesta en servicio del TAI (N= 3224) y la calibración actual obtenida con las aplicaciones reales del TAI (N= 7254). Se ha analizado el Funcionamiento Diferencial de los Ítems (FDI) en función de los parámetros utilizados y se ha simulado el impacto que sobre el nivel de rasgo estimado tiene la variación en los parámetros. Los resultados muestran que se produce especialmente un deterioro de los parámetros a y c, que hay unimportante número de ítems del banco para los que existe FDI y que la variación de los parámetros produce un impacto moderado en la estimación de θ de los evaluados con nivel de inglés alto. Se concluye que los parámetros de los ítems se han deteriorado y deben ser actualizados.Item parameter drift in computerized adaptive testing: Study with eCAT. This study describes the parameter drift analysis conducted on eCAT (a Computerized Adaptive Test to assess the written English level of Spanish speakers). The original calibration of the item bank (N = 3224) was compared to a new calibration obtained from the data provided by most eCAT operative administrations (N =7254). A Differential Item Functioning (DIF) study was conducted between the original and the new calibrations. The impact that the new parameters have on the trait level estimates was obtained by simulation. Results show that parameter drift is found especially for a and c parameters, an important number of bank items show DIF, and the parameter change has a moderate impact on high-level-English θ estimates. It is then recommended to replace the original estimates by the new set. by the new set.
10a*Software10aEducational Measurement/*methods/*statistics & numerical data10aHumans10aLanguage1 aAbad, F J1 aOlea, J1 aAguado, D1 aPonsoda, V1 aBarrada, J R uhttp://www.iacat.org/content/deterioro-de-par%C3%A1metros-de-los-%C3%ADtems-en-tests-adaptativos-informatizados-estudio-con-ecat02418nas a2200325 4500008004100000020002200041245008700063210006900150250001500219300001100234490000700245520140100252653001901653653003001672653001901702653003801721653002101759653002001780653001401800653001501814653003301829653001101862653002401873653001801897100001701915700001501932700001501947700001501962856011501977 2007 eng d a0962-9343 (Print)00aDeveloping tailored instruments: item banking and computerized adaptive assessment0 aDeveloping tailored instruments item banking and computerized ad a2007/05/29 a95-1080 v163 aItem banks and Computerized Adaptive Testing (CAT) have the potential to greatly improve the assessment of health outcomes. This review describes the unique features of item banks and CAT and discusses how to develop item banks. In CAT, a computer selects the items from an item bank that are most relevant for and informative about the particular respondent; thus optimizing test relevance and precision. Item response theory (IRT) provides the foundation for selecting the items that are most informative for the particular respondent and for scoring responses on a common metric. The development of an item bank is a multi-stage process that requires a clear definition of the construct to be measured, good items, a careful psychometric analysis of the items, and a clear specification of the final CAT. The psychometric analysis needs to evaluate the assumptions of the IRT model such as unidimensionality and local independence; that the items function the same way in different subgroups of the population; and that there is an adequate fit between the data and the chosen item response models. Also, interpretation guidelines need to be established to help the clinical application of the assessment. Although medical research can draw upon expertise from educational testing in the development of item banks and CAT, the medical field also encounters unique opportunities and challenges.10a*Health Status10a*Health Status Indicators10a*Mental Health10a*Outcome Assessment (Health Care)10a*Quality of Life10a*Questionnaires10a*Software10aAlgorithms10aFactor Analysis, Statistical10aHumans10aModels, Statistical10aPsychometrics1 aBjorner, J B1 aChang, C-H1 aThissen, D1 aReeve, B B uhttp://www.iacat.org/content/developing-tailored-instruments-item-banking-and-computerized-adaptive-assessment03104nas a2200445 4500008004100000020002200041245007100063210006900134250001500203300001200218490000700230520183100237653003802068653001902106653002102125653002002146653001402166653001102180653003002191653001102221653000902232653002502241653004602266653001802312653002602330100001302356700001402369700001702383700001302400700001502413700001502428700001702443700001402460700001802474700002302492700001602515700001602531700001502547856009602562 2007 eng d a0962-9343 (Print)00aIRT health outcomes data analysis project: an overview and summary0 aIRT health outcomes data analysis project an overview and summar a2007/03/14 a121-1320 v163 aBACKGROUND: In June 2004, the National Cancer Institute and the Drug Information Association co-sponsored the conference, "Improving the Measurement of Health Outcomes through the Applications of Item Response Theory (IRT) Modeling: Exploration of Item Banks and Computer-Adaptive Assessment." A component of the conference was presentation of a psychometric and content analysis of a secondary dataset. OBJECTIVES: A thorough psychometric and content analysis was conducted of two primary domains within a cancer health-related quality of life (HRQOL) dataset. RESEARCH DESIGN: HRQOL scales were evaluated using factor analysis for categorical data, IRT modeling, and differential item functioning analyses. In addition, computerized adaptive administration of HRQOL item banks was simulated, and various IRT models were applied and compared. SUBJECTS: The original data were collected as part of the NCI-funded Quality of Life Evaluation in Oncology (Q-Score) Project. A total of 1,714 patients with cancer or HIV/AIDS were recruited from 5 clinical sites. MEASURES: Items from 4 HRQOL instruments were evaluated: Cancer Rehabilitation Evaluation System-Short Form, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire, Functional Assessment of Cancer Therapy and Medical Outcomes Study Short-Form Health Survey. RESULTS AND CONCLUSIONS: Four lessons learned from the project are discussed: the importance of good developmental item banks, the ambiguity of model fit results, the limits of our knowledge regarding the practical implications of model misfit, and the importance in the measurement of HRQOL of construct definition. With respect to these lessons, areas for future research are suggested. The feasibility of developing item banks for broad definitions of health is discussed.10a*Data Interpretation, Statistical10a*Health Status10a*Quality of Life10a*Questionnaires10a*Software10aFemale10aHIV Infections/psychology10aHumans10aMale10aNeoplasms/psychology10aOutcome Assessment (Health Care)/*methods10aPsychometrics10aStress, Psychological1 aCook, KF1 aTeal, C R1 aBjorner, J B1 aCella, D1 aChang, C-H1 aCrane, P K1 aGibbons, L E1 aHays, R D1 aMcHorney, C A1 aOcepek-Welikson, K1 aRaczek, A E1 aTeresi, J A1 aReeve, B B uhttp://www.iacat.org/content/irt-health-outcomes-data-analysis-project-overview-and-summary01799nas a2200265 4500008004100000020004100041245010400082210006900186250001500255300001100270490001500281520085700296653001901153653003801172653002101210653001401231653002901245653005601274653001101330653002501341653001901366653001801385100001501403856011501418 2007 eng d a0962-9343 (Print)0962-9343 (Linking)00aPatient-reported outcomes measurement and management with innovative methodologies and technologies0 aPatientreported outcomes measurement and management with innovat a2007/05/29 a157-660 v16 Suppl 13 aSuccessful integration of modern psychometrics and advanced informatics in patient-reported outcomes (PRO) measurement and management can potentially maximize the value of health outcomes research and optimize the delivery of quality patient care. Unlike the traditional labor-intensive paper-and-pencil data collection method, item response theory-based computerized adaptive testing methodologies coupled with novel technologies provide an integrated environment to collect, analyze and present ready-to-use PRO data for informed and shared decision-making. This article describes the needs, challenges and solutions for accurate, efficient and cost-effective PRO data acquisition and dissemination means in order to provide critical and timely PRO information necessary to actively support and enhance routine patient care in busy clinical settings.10a*Health Status10a*Outcome Assessment (Health Care)10a*Quality of Life10a*Software10aComputer Systems/*trends10aHealth Insurance Portability and Accountability Act10aHumans10aPatient Satisfaction10aQuestionnaires10aUnited States1 aChang, C-H uhttp://www.iacat.org/content/patient-reported-outcomes-measurement-and-management-innovative-methodologies-and01587nas a2200241 4500008004100000245005800041210005800099300001200157490000600169520081000175653001400985653001400999653004801013653005701061653001101118653001801129653003101147653003701178100001301215700001701228700001601245856008401261 2000 eng d00aCAT administration of language placement examinations0 aCAT administration of language placement examinations a292-3020 v13 aThis article describes the development of a computerized adaptive test for Cegep de Jonquiere, a community college located in Quebec, Canada. Computerized language proficiency testing allows the simultaneous presentation of sound stimuli as the question is being presented to the test-taker. With a properly calibrated bank of items, the language proficiency test can be offered in an adaptive framework. By adapting the test to the test-taker's level of ability, an assessment can be made with significantly fewer items. We also describe our initial attempt to detect instances in which "cheating low" is occurring. In the "cheating low" situation, test-takers deliberately answer questions incorrectly, questions that they are fully capable of answering correctly had they been taking the test honestly.10a*Language10a*Software10aAptitude Tests/*statistics & numerical data10aEducational Measurement/*statistics & numerical data10aHumans10aPsychometrics10aReproducibility of Results10aResearch Support, Non-U.S. Gov't1 aStahl, J1 aBergstrom, B1 aGershon, RC uhttp://www.iacat.org/content/cat-administration-language-placement-examinations01318nas a2200265 4500008004100000245005500041210005400096300001200150490000600162520053000168653003800698653002000736653001400756653003500770653003100805653001100836653001900847653001800866653002800884100001300912700001500925700001700940700001400957856008100971 1997 eng d00aOn-line performance assessment using rating scales0 aOnline performance assessment using rating scales a173-1910 v13 aThe purpose of this paper is to report on the development of the on-line performance assessment instrument--the Assessment of Motor and Process Skills (AMPS). Issues that will be addressed in the paper include: (a) the establishment of the scoring rubric and its implementation in an extended Rasch model, (b) training of raters, (c) validation of the scoring rubric and procedures for monitoring the internal consistency of raters, and (d) technological implementation of the assessment instrument in a computerized program.10a*Outcome Assessment (Health Care)10a*Rehabilitation10a*Software10a*Task Performance and Analysis10aActivities of Daily Living10aHumans10aMicrocomputers10aPsychometrics10aPsychomotor Performance1 aStahl, J1 aShumway, R1 aBergstrom, B1 aFisher, A uhttp://www.iacat.org/content/line-performance-assessment-using-rating-scales