ERIC Number: EJ863231
Record Type: Journal
Publication Date: 2009
Pages: 26
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0013-1644
EISSN: N/A
Available Date: N/A
Magnitude of Task-Sampling Variability in Performance Assessment: A Meta-Analysis
Huang, Chiungjung
Educational and Psychological Measurement, v69 n6 p887-912 2009
This study examined the percentage of task-sampling variability in performance assessment via a meta-analysis. In total, 50 studies containing 130 independent data sets were analyzed. Overall results indicate that the percentage of variance for (a) differential difficulty of task was roughly 12% and (b) examinee's differential performance of the same task was approximately 26%. Based on moderator analysis, research design and subject area were significant predictors of variance component for tasks ([sigma][superscript 2][subscript t]) and variance component for person-task interaction ([sigma][superscript 2][subscript pt]). Mathematics and listening were associated with relatively high proportion of task-sampling variability whereas foreign language was least affected by this variability. To reduce task variation, some techniques are recommended: (a) incorporating more facets when possible, (b) using a crossed rather than a nested design, and (c) incorporating occasion as a facet. (Contains 3 tables.)
Descriptors: Test Bias, Research Design, Performance Based Assessment, Performance Tests, Sampling, Test Items, Meta Analysis, Task Analysis, Difficulty Level, Second Language Learning, Mathematics Skills, Listening Skills, Effect Size, Interrater Reliability, Scoring, Coding
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A