Studies of group differences in intelligence often invite conclusions about groups in general from studies of group differences in selected populations. The same design is used in the study of group differences in other traits as well. Investigators observe samples from two groups (e.g. men and women) in some accessible population, but seek to…

Experimental research involves comparing two or more conditions. One of those conditions in single subject research is often the baseline. Adequate description of the baseline conditions is necessary for evaluating the effects of independent variables, drawing generalizations from studies, and providing information for subsequent replication…

Methodological and political issues arise during the designing, conducting, and reporting of campus-climate studies for LGBT students. These issues interact; making a decision about a methodological issue (e.g., sample size) has an impact on a political issue (e.g., how well the findings will be received). Ten key questions that must be addressed…

This article highlights the problem of omitted variable bias in research on the causal effect of financial aid on college-going. I first describe the problem of self-selection and the resulting bias from omitted variables. I then assess and explore the strengths and weaknesses of random assignment, multivariate regression, proxy variables, fixed…

Multiple matrix sampling is a sampling design in which both test items and examinees are randomly sampled from their respective populations. This study was designed to develop and assess a method for computing an estimate of a correlation coefficient when a multiple matrix sampling design is used. The examinee populations included 212 third-grade…

It is a false, but common, belief that statistical significance testing evaluates result replicability. In truth, statistical significance testing reveals nothing about results replicability. Since science is based on replication of results, methods that assess replicability are important. This is particularly true when multivariate methods, which…

This paper provides practical guidance for researchers who are designing studies that randomize groups to measure the impacts of interventions on children. To do so, the paper: (1) provides new empirical information about the values of parameters that influence the precision of impact estimates (intra-class correlations and R-squares); (2)…

This paper examines several ways in which teaching effectiveness and student learning in an undergraduate Business Statistics course can be enhanced. First, we review some key concepts in Business Statistics that are often challenging to teach and show how using real data sets assist students in developing deeper understanding of the concepts.…

The popularity of sample surveys in evaluation and research makes it necessary for consumers to tell a good survey from a poor one. Several sources were identified that gave advice on how to evaluate a sample design used in a survey study. The sources are either too limited or too extensive to be useful practically. The purpose of this paper is to…

Tests constructed through item-sampling (where several tests are constructed and randomly assigned to testees) were compared to a traditional constant-item procedure for evaluating student and class progress in a community college freshman psychology class. The item-sampling approach did not appear to be superior. (JKS)

Tests constructed through item-sampling (where several tests are constructed and randomly assigned to testees) were compared to a traditional constant-item procedure for evaluating class progress in a community college intermediate algebra class. The item-sampling format appeared to be slightly less valid than the traditional format. (JKS)

High quality assessment practice is expected to yield valid and useful score-based interpretations about what the examinees know and are able to do with respect to a defined target domain. Given this assertion, the article presents a framework based on the "unified view of validity," advanced by Cronbach and Messick over two decades ago, to assist…

At present, although the percentages of students with disabilities (SDs) and/or students who are English language learners (ELL) excluded from a NAEP administration are reported, no statistical adjustment is made for these excluded students in the calculation of NAEP results. However, the exclusion rates for both SD and ELL students vary…

Five branch libraries measured the quantity of books, sound recordings, and periodicals in their catalogs in comparison with the number available for loan. A sampling method similar to those used for political pools and market research was employed. (JAB)

Format dependence implies that assessment of the same subjective probability distribution produces different conclusions about over- or underconfidence depending on the assessment format. In 2 experiments, the authors demonstrate that the overconfidence bias that occurs when participants produce intervals for an uncertain quantity is almost…