This guide offers a comprehensive handbook to scientific research methodology and experimental design, specifically for novice MA and PhD researchers in Education and Language Learning (TESOL/TEFL). It establishes scientific research as a systematic, objective inquiry focused on identifying cause-and-effect relationships through empirical data.…

Experimental psychology is witnessing an increase in research on individual differences, which requires the development of new tasks that can reliably assess variations among participants. To do this, cognitive researchers need statistical methods that many researchers have not learned during their training. The lack of expertise can pose…

Attribute structure is an explicit way of presenting the relationship between attributes in diagnostic measurement. The specification of attribute structures directly affects the classification accuracy resulted from psychometric modeling. This study provides a conceptual framework for understanding misspecifications of attribute structures. Under…

Because the disciplines of mathematics and statistics are naturally intertwined, designing assessment questions that disentangle mathematical and statistical reasoning can be challenging. We explore the writing statistics assessment tasks that take into consideration potential mathematical reasoning they may inadvertently activate.

The main points of Sijtsma and Green and Yang in Educational Measurement: Issues and Practice (34, 4) are that reliability, internal consistency, and unidimensionality are distinct and that Cronbach's alpha may be problematic. Neither of these assertions are at odds with Davenport, Davison, Liou, and Love in the same issue. However, many authors…

Data collected from surveys can result in hundreds of variables and thousands of respondents. This implies that time and energy must be devoted to (a) carefully entering the data into a database, (b) running preliminary analyses to identify any problems (e.g., missing data, potential outliers), (c) checking the reliability and validity of the…

A latent variable modeling procedure is discussed that can be used to test if two or more homogeneous multicomponent instruments with distinct components are measuring the same underlying construct. The method is widely applicable in scale construction and development research and can also be of special interest in construct validation studies.…

Item response theory parameters have to be estimated, and because of the estimation process, they do have uncertainty in them. In most large-scale testing programs, the parameters are stored in item banks, and automated test assembly algorithms are applied to assemble operational test forms. These algorithms treat item parameters as fixed values,…

This paper aims to develop a valid method to assess the key competencies of the exercise physiology profession acquired through work-integrated learning (WIL). In order to develop a competency-based assessment, the key professional tasks needed to be identified and the test designed so students' competency in different tasks and settings could be…

This paper presents a project which was conducted in a capstone course in Information Security. The project focused on conducting research concerning the various aspects of phishing, such as why phishing works and who is more likely to be deceived by phishing. Students were guided through the process of conducting research: finding background and…

This paper presents a new index to quantify, measure and monitor the progress towards the objectives of the Europe 2020 strategy. This index is based on a set of relevant, accepted, credible, easy to monitor and robust indicators presented by the European Commission at the time the strategy was launched. The internal analysis of the index shows…

Undergraduate research is important for preparing the next generation of scholars in family and consumer sciences (FCS). The Research Resource Guide (RRG) was developed in response to that need; it provides comprehensive information on the research process for faculty to incorporate into undergraduate courses. The RRG comprises three levels: (a)…

Automatic item generation represents a relatively new but rapidly evolving research area where cognitive and psychometric theories are used to produce tests that include items generated using computer technology. Automatic item generation requires two steps. First, test development specialists create item models, which are comparable to templates…

Understanding students' prior beliefs in nuclear physics is a first step towards improving nuclear physics instruction. This paper describes the development of a diagnostic survey in nuclear physics covering the areas of radioactive decay, binding energy, properties of the nuclear force and nuclear reactions, that was administered to students at…

Males have significantly higher average scores than females on the SAT I quantitative section, which is designed to predict first-year college success in mathematics. However, it has been shown that gender gaps in performance on the SAT I have little to do with college readiness; rather they are due to the misaligned content of the instrument, as…