Evaluators are often called upon to assess the effects of programs. To assess a program effect, evaluators need a clear understanding of how a program effect is defined. Arguably, the most widely used definition of a program effect is the counterfactual one. According to the counterfactual definition, a program effect is the difference between…

Several programs of research have sought to assess the replicability of scientific findings in different fields, including economics and psychology. These programs attempt to replicate several findings and use the results to say something about large-scale patterns of replicability in a field. However, little work has been done to understand the…

This chapter analyzes the various ways social class identity has been measured among undergraduate students. Implications of social class definitions and measurement for student affairs practitioners also are discussed.

This paper aims at highlighting the overlapping between the qualification mismatch and the skill mismatch, using a specific Italian data set drawn from PIAAC survey (2012). The main concern, regarding skills and competences, is that they may be gathered and accumulated, but also potentially become obsolete, thus, qualifications (formally…

As a result of this research, a quantitative model and a procedure have been developed to create an online mentoring effectiveness index (EI). To develop the model, mentoring and teaching effectiveness are defined, and then the constructs and factors of effectiveness are identified. The model's construction is based on the theory that…

Much of the argument about comparability of examination standards is at cross-purposes; contradictory positions are in fact often both defensible, but they are using the same words to mean different things. To clarify this, two broad conceptualisations of standards can be identified. One sees the standard in the observed phenomena of performance…

Robert Coe has claimed that three broad conceptions of comparability can be identified from the literature: performance, statistical and conventional. Each of these he rejected, in favour of a single, integrated conception which relies upon the notion of a "linking construct" and which he termed "construct comparability".…

"Diagnostic classification models" (DCM) are frequently promoted by psychometricians as important modelling alternatives for analyzing response data in situations where multivariate classifications of respondents are made on the basis of multiple postulated latent skills. In this review paper, a definitional boundary of the space of DCM…

We propose an integration of aspects of several developmental and systems of beliefs models of personal epistemology. Qualitatively different positions, including realism, dogmatism, skepticism, and rationalism, are characterized according to individuals' beliefs across three dimensions in a model of epistemic and ontological cognition. This model…

Presented are common words and phrases which are encountered in the statistical analysis of test results. Included are analysis of the mid-point, distribution of scores, calculation of scaled scores, test reliability, item analysis, and coefficients of reliability. (CS)

Matching, randomization, random sampling, and other methods of controlling extraneous variation are discussed. The purpose was to specify the benefits of randomization in estimating causal effects of treatments. It is concluded that randomization should be employed whenever possible, but the use of carefully controlled nonrandomized data to…

This paper gives an overview of the methods of moderation, or quality assurance and quality control, as they may be more widely known, that are used to enhance the quality of student assessment. The discussion is based on the educational systems of the United Kingdom but is applicable to assessment in other countries. Quality in assessment is seen…

This document presents the key principles underpinning effective evaluations as applied to policy initiatives in the field of lifelong learning (LL) and the various methods available to implement them. The following are among the topics discussed in Chapters 1-6: (1) the distinction between monitoring, modeling, and evaluation and considerations…

This manual, which is intended for practitioners involved in the Asia and the Pacific Programme of Education for All, explains the basic concepts and procedures involved in monitoring and evaluating literacy and continuing education programs. The following are among the topics discussed in the manual's eight chapters: (1) understanding monitoring…

This volume of the 27th International Group for the Psychology of Mathematics Education Conference includes the following research reports: (1) Improving Decimal Number Conception by Transfer from Fractions to Decimals (Irita Peled and Juhaina Awawdy Shahbari); (2) The Development of Student Teachers' Efficacy Beliefs in Mathematics during…