The aim of this research is to evaluate the constructivist learning environments of physical education and sport teacher candidates. For this purpose, 928 students (523 male, 405 female) selected by the appropriate sampling method from the Physical Education and Sport Teaching Department of 17 universities consisted the sample of the research. In…

This paper intends to construct a survey data quality strategy for institutional researchers in higher education in light of total survey error theory. It starts with describing the characteristics of institutional research and identifying the gaps in literature regarding survey data quality issues in institutional research. Then it is followed by…

This study examined the reliability of three methods for detecting differential item functioning (DIF) (i.e., the Mantel-Haenszel method, the standardization method, and the logistic regression method) applied to achievement test data. In addition, the study examined the influences of different sources of error variance, including examinee,…

The literature on measurement reliability shows the consensus that group heterogeneity with regard to the trait being measured is a factor that affects the sample measurement reliability, but the degree of such effect is not entirely clear. Sample performance also has the potential to affect measurement reliability because of its effect on the…

Illustrates that sampling variance can be very large when a three-wave quasi simplex model is used to obtain reliability estimates. Also shows that, for the reliability parameter to be identified, the model assumes a Markov process. These problems are evaluated with both real and Monte Carlo data. (SLD)

The purpose of this paper is to highlight some psychometric cautions that should be observed when seeking to develop short form versions of tests. Several points are made: (1) score reliability is impacted directly by the characteristics of the sample and testing conditions; (2) sampling error has a direct influence on reliability and factor…

Basic to many psychological investigations is the question of agreement between observers who independently categorize people. Several recent studies have proposed measures of agreement when a set of nominal scale categories have been pre-defined and imposed on both observers. This study, in contrast, developes a measure of agreement for settings…

Statistical significance is often misinterpreted to mean replicability or generalizability of results, although a statistically significant difference does not equal a reliable difference. Sample splitting procedures may be a more accurate way of estimating research result generalizability. This type of cross-validation involves randomly dividing…

Some years ago, B. Efron and his colleagues developed bootstrap resampling methods as a way of estimating the degree to which statistical results will replicate across variations in sample. A basic problem in the multivariate use of bootstrap procedures involves the requirement that the results across resamplings must be rotated to best fit in a…

Time sampled observations of the cooperative behavior of two samples of 31 preschool children were analyzed for stability (that is, short term reliability of behavior) over a 2-month period using Cronbach's generalizability coefficient. Observations were made during free play periods on nursery school settings. The observation schedule required…

Although videotape is the preferred method for recording behavior, there are many natural settings in which it is impractical or too intrusive. The ability to collect data in such settings and in real time has been enhanced by the advent of light, powerful laptop computers. This poster (with paper) describes a set of eight programs (developed in…

The author conducted a cross national analysis of sociological research reported in leading journals at two points in time over a ten year period to determine if sociologists' ability to produce valid social generalizations had improved significantly over the recent past. The official journals of the United States (American Sociological Review),…

Arguments have recently been put forth that standard textbook procedures for determining the sample size necessary to achieve a certain level of power in a completely randomized design are incorrect when the dependent variable is fallible because they ignore measurement error. In fact, however, there are several correct procedures, one of which is…

The purpose of this study is to help define the precise nature and limits of the tolerable range in which a researcher may be relatively confident about the statistical validity of his or her research findings, focusing specifically on the statistical validity of results when violating the assumptions associated with the one-way, fixed-effects…

In order to assess the methodological merit of the published research in educational administration based on surveys, the authors analyzed data from a random sample of 24 survey studies published in the "Education Administration Quarterly" and the "Journal of Educational Administration." Each article was evaluated according to…