Effect sizes are important because they are an accessible way to indicate the practical importance of observed associations or differences. Standardized mean difference (SMD) effect sizes, such as Cohen's d, are widely used in education and the social sciences -- in part because they are relatively easy to calculate. However, SMD effect sizes…

The fit of an item response model is typically conceptualized as whether a given model could have generated the data. We advocate for an alternative view of fit, "predictive fit", based on the model's ability to predict new data. We derive two predictive fit metrics for item response models that assess how well an estimated item response…

Effect direction (evidence to indicate improvement, deterioration, or no change in an outcome) can be used as a standardized metric which enables the synthesis of diverse effect measures in systematic reviews. The effect direction (ED) plot was developed to support the synthesis and visualization of effect direction data. Methods for the ED plot…

Model fit indices are being increasingly recommended and used to select the number of factors in an exploratory factor analysis. Growing evidence suggests that the recommended cutoff values for common model fit indices are not appropriate for use in an exploratory factor analysis context. A particularly prominent problem in scale evaluation is the…

Purpose: This paper aims to introduce the extended qualitative content analysis (EQCA) method to integrate data-reducing and data-complicating research steps when conducting qualitative research on the United Nations and other international institutions. Design/methodology/approach: EQCA supplements the method of qualitative content analysis,…

Purpose: The paper aims to present a new two stage local causal learning algorithm -- HEISA. In the first stage, the algorithm discoveries the subset of features that better explains a target variable. During the second stage, computes the causal effect, using partial correlation, of each feature of the selected subset. Using this new algorithm,…

In secure web application development, the role of web services will not continue if it is not trustworthy. Retaining customers with applications is one of the major challenges if the services are not reliable and trustworthy. This article proposes a trust evaluation and decision model where the authors have defined indirect attribute, trust,…

This study builds on a 2019 Fordham Institute report that examined the relationship between charter school enrollment share--the share of students in a community who enroll in a charter school--and the average achievement of all the students in that community, including those in traditional public schools. Like its predecessor, this report seeks…

Background: This paper considers a new policy and statistical framework for evaluating K-12 schools and policies that aligns with diversity, equity, and inclusion values. The new approach broadens the standard approach to accountability and evaluation by combining features of evaluation and multi-level growth models with approaches used in systems…

This paper aims to illustrate how data visualization could be utilized to identify errors prior to modeling, using an example with multi-dimensional item response theory (MIRT). MIRT combines item response theory and factor analysis to identify a psychometric model that investigates two or more latent traits. While it may seem convenient to…

This paper provides a tutorial on epistemic network analysis (ENA), a novel method for identifying and quantifying connections among elements in coded data and representing them in dynamic network models. Such models illustrate the structure of connections and measure the strength of association among elements in a network, and they quantify…

This study investigates the relationships among factor correlations, inter-item correlations, and the reliability estimates of subscores, providing a guideline with respect to psychometric properties of useful subscores. In addition, it compares subscore estimation methods with respect to reliability and distinctness. The subscore estimation…

Educational systems typically contain a large pool of items (questions, problems). Using data mining techniques we can group these items into knowledge components, detect duplicated items and outliers, and identify missing items. To these ends, it is useful to analyze item similarities, which can be used as input to clustering or visualization…

Identifying schools that are beating the odds--schools that are exceeding academic performance expectations, given the composition of their student body--can lead to promising practices that other schools serving similar populations can implement. Regional Educational Laboratory Northeast & Islands used data on high-poverty high schools in…

Over the last 10 years faculty members in the Department of Mathematical Sciences at the United States Air Force Academy have incorporated individual oral exams into mathematics courses. We have experimented with various approaches, shared results and ideas with other department members, and refined our techniques. We have found that this…