Evaluators are often called upon to assess the effects of programs. To assess a program effect, evaluators need a clear understanding of how a program effect is defined. Arguably, the most widely used definition of a program effect is the counterfactual one. According to the counterfactual definition, a program effect is the difference between…

In carrying out surveys involving sensitive characteristics, randomized response models have been considered among the best techniques since they provide the maximum privacy protection to the respondents and procure honest responses. Over the years, researchers have carried out studies on the estimation of proportions of the population possessing…

Policymakers and institutional leaders in higher education too often make decisions based on descriptive data analyses or even anecdote when better analysis options could produce more nuanced and more valuable results. Employing the setting of higher education program evaluation at a midwestern regional public university, for this study we…

Examples of the impact of statistical theory on assessment practice are provided from the perspective of a statistician trained in theoretical statistics who began to work on assessments. Goodness of fit of item-response models is examined in terms of restricted likelihood-ratio tests and generalized residuals. Minimum discriminant information…

We propose cross-disciplinary learning as a construct that can guide instruction and assessment in programs that feature sequential learning across multiple science disciplines. Crossdisciplinary learning combines insights from interdisciplinary learning, transfer, and resources frameworks and highlights the processes of resource activation,…

In systematic reviews, meta-analyses are routinely applied to summarize the results of the relevant studies for a specific research question. If one can assume that in all studies the same true effect is estimated, the application of a meta-analysis with common effect (commonly referred to as fixed-effect meta-analysis) is adequate. If…

Summary Employing a Wald confidence interval to test hypotheses about population proportions could lead to an increase in Type I or Type II errors unless the hypothesized value, p0, is used in computing its standard error rather than the sample proportion. Whereas the Wald confidence interval to estimate a population proportion uses the sample…

Nonlinear effects are both underreported and underrepresented in counseling research. We provide a rationale for evaluating nonlinear effects and steps to evaluate nonlinear relationships in counseling research. Two heuristic examples are provided along with discussion of the results and advantages to evaluating nonlinear effects.

Context is at the core of any statistical investigation, yet many statistics tasks barely require students to go beyond superficial consideration of the contexts the tasks are situated in. In this article, I discuss a framework for evaluating the level of interaction with context a task requires of students and how to modify tasks to increase the…

As states consider non-academic measures of school quality for their accountability plans under the Every Student Succeeds Act, few are choosing to focus on the school climate in which students learn. That's not a surprise, given concerns about the validity and usefulness of existing climate surveys. However, the authors argue, a recent study…

This article provides insight into how an adequate approach to selecting methods can establish credible and actionable evidence. The authors offer strategies to effectively support Extension professionals, including program developers and evaluators, in being more deliberate when selecting appropriate qualitative and quantitative methods. In…

This article addresses the problem of estimating the proportion Pi[subscript S] of the population belonging to a sensitive group using optional randomized response technique in stratified sampling based on Mangat model that has proportional and Neyman allocation and larger gain in efficiency. Numerically, it is found that the suggested model is…

The article serves as a guide for researchers in developing evidence of validity using bivariate correlations, specifically construct validity. The authors outline the steps for calculating and interpreting bivariate correlations. Additionally, they provide an illustrative example and discuss the implications.

This chapter asserts that data are more likely to improve learning when assessment focuses on sensemaking conversations among students, faculty, and student affairs administrators, rather than on advanced statistical techniques.

Current issues in the use of peer assessment techniques and sociometric methods are discussed. Attention is paid to the contributions of the four articles in this volume. Together these contributions point to the continual level of change and progress in these techniques. They also show that the paradigm underlying these methods has been unchanged…