Randomized control trials (RCTs) have long been considered the "gold standard" for evaluating the impacts of interventions. However, in most education RCTs, the sample of schools included is recruited based on convenience, potentially compromising a study's ability to generalize to an intended population. An alternative approach is to…

In institution research, surveys of students or faculty can be a helpful tool to gather data. Surveying a sample of students or faculty and computing weights to be able to make inferences to your student or faculty population are important. In this chapter, we introduce the connected topics of sampling and weighting. We begin with a discussion on…

Two recent publications in "Educational and Psychological Measurement" advocated that researchers consider using the a priori procedure. According to this procedure, the researcher specifies, prior to data collection, how close she wishes her sample mean(s) to be to the corresponding population mean(s), and the desired probability of…

Randomized control trials (RCTs) have long been considered the "gold standard" for evaluating the impacts of interventions. However, in most education RCTs, the sample of schools included is recruited based on convenience, potentially compromising a study's ability to generalize to an intended population. An alternative approach is to…

Sample representativeness is an important piece to consider when evaluating the quality of a screening assessment. If you are trying to determine whether or not the screening tool accurately measures children's skills, you want to ensure that the sample that is used to validate the tool is representative of your population of interest.

Autism research tends to exclude racially and ethnically minoritized autistic individuals, limiting the quality of the evidence base for characterizing and diagnosing language impairment in racially and ethnically minoritized autistic individuals. This systematic review examines reporting practices for participant socio-demographics in studies…

Reform efforts in statistics education emphasize the need for students to develop statistical thinking. Critical to this goal is a solid understanding of design in the process of collecting data, evaluating evidence, and drawing conclusions. We collected survey responses from over 700 college students at the start of an introductory statistics…

Time-sampling techniques are popular among school practitioners and researchers alike when conducting observations of student behavior. Despite their popularity, the psychometric properties of such observation data have not received much attention, likely given that both procedures and targets vary widely across individual coding systems. A…

Minimum discriminant information adjustment (MDIA), an approach to weighting samples to conform to known population information, provides a generalization of raking and poststratification. In the case of simple random sampling with replacement with uniform sampling weights, large-sample properties are available for MDIA estimates of population…

This article discusses the estimation of a population proportion, using the auxiliary information available, which is incorporated into the estimation procedure by a probit model fit. Three probit regression estimators are considered, using model-based and model-assisted approaches. The theoretical properties of the proposed estimators are derived…

Testing whether items fit the assumptions of an item response theory model is an important step in evaluating a test. In the literature, numerous item fit statistics exist, many of which show severe limitations. The current study investigates the root mean squared deviation (RMSD) item fit statistic, which is used for evaluating item fit in…

Factorial surveys use a population of vignettes to elicit respondents' attitudes or beliefs about different hypothetical scenarios. However, the vignette population is frequently too large to be assessed by each respondent. Experimental designs such as randomized block confounded factorial (RBCF) designs, D-optimal designs, or random sampling…

Background: Several statistical applications including Mplus, STATA, and R are available to conduct analyses such as structural equation modeling and multi-level modeling using large-scale assessment data that employ complex sampling and assessment designs and that provide associated information such as sampling weights, replicate weights, and…

Although recent studies have used the Classroom Observation Protocol for Undergraduate STEM (COPUS) to make claims about faculty reform, important questions remain: How should COPUS measures be situated within existing reform frameworks? Is there a universal sampling intensity that allows for valid inferences about the frequency of…

Risky decisions are often characterized by (a) imprecision about consequences and their likelihoods that can be reduced by information collection, and by (b) unavoidable background risk. This article addresses both aspects by eliciting risk attitude, prudence, and temperance in decisions from description and decisions from experience. The results…