In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of…

An equating procedure for a testing program with evolving distribution of examinee profiles is developed. No anchor is available because the original scoring scheme was based on expert judgment of the item difficulties. Pairs of examinees from two administrations are formed by matching on coarsened propensity scores derived from a set of…

Between March and June of 2014, the Smarter Balanced Assessment Consortium conducted a field test of its new online assessment system. Thirteen participating states provided the results of surveys given to students and adults involved in the Field Test. Overall, more than 70% of test coordinators in each of seven states indicated that the Field…

In this technical report, we present the results of a reliability study of the seventh-grade multiple choice reading comprehension measures available on the easyCBM learning system conducted in the spring of 2011. Analyses include split-half reliability, alternate form reliability, person and item reliability as derived from Rasch analysis,…

Standard setting is a method used to set cut scores on large-scale assessments. One of the most popular standard setting methods is the Bookmark method. In the Bookmark method, panelists are asked to envision a response probability (RP) criterion and move through a booklet of ordered items based on a RP criterion. This study investigates whether…

The Assessment of Language Competence (ALC) certificates is an annual, international testing program developed by the Australian Council for Educational Research to test the listening and reading comprehension skills of lower to middle year levels of secondary school. The tests are developed for three levels in French, German, Italian and…

The most critical feature of a common-item nonequivalent groups equating design is that the average score difference between the new and old groups can be accurately decomposed into a group ability difference and a form difficulty difference. Two widely used observed-score linear equating methods, the Tucker and the Levine observed-score methods,…

Operational testing programs employing item response theory (IRT) applications benefit from of the property of item parameter invariance whereby item parameter estimates obtained from one sample can be applied to other samples (when the underlying assumptions are satisfied). In theory, this feature allows for applications such as computer-adaptive…

In operational testing programs using item response theory (IRT), item parameter invariance is threatened when an item appears in a different location on the live test than it did when it was field tested. This study utilizes data from a large state's assessments to model change in Rasch item difficulty (RID) as a function of item position change,…

We exploit an exogenous increase in General Educational Development (GED) testing requirements to determine whether raising the difficulty of the test causes students to finish high school rather than drop out and GED certify. We find that a six point decrease in GED pass rates induces a 1.3 point decline in overall dropout rates. The effect size…

"Measuring College and Career Readiness" represents a snapshot of the ACT-tested graduates in the class of 2009 and focuses on their readiness for college and careers. Designed to inform policymakers and practitioners about selected indicators of effectiveness, it is not intended to be comprehensive but instead is designed to help…

The "Measuring College and Career Readiness" report for each state represents a snapshot of the ACT-tested graduates in the class of 2009 and focuses on their readiness for college and careers. Designed to inform policymakers and practitioners about selected indicators of effectiveness, the report is not intended to be comprehensive but…

The "Measuring College and Career Readiness" report for each state represents a snapshot of the ACT-tested graduates in the class of 2009 and focuses on their readiness for college and careers. Designed to inform policymakers and practitioners about selected indicators of effectiveness, the report is not intended to be comprehensive but…

The "Measuring College and Career Readiness" report for each state represents a snapshot of the ACT-tested graduates in the class of 2009 and focuses on their readiness for college and careers. Designed to inform policymakers and practitioners about selected indicators of effectiveness, the report is not intended to be comprehensive but…

The "Measuring College and Career Readiness" report for each state represents a snapshot of the ACT-tested graduates in the class of 2009 and focuses on their readiness for college and careers. Designed to inform policymakers and practitioners about selected indicators of effectiveness, the report is not intended to be comprehensive but…