This paper examines representative research in information science and compares its designs to an abstract ideal of the experimental research design. Threats to validity of conclusions are highlighted, including history, maturation, testing, instrumentation, statistical regression, selection biases, experimental mortality, and selection-maturation…

To rule out threats to internal validity, proposes a control group design in which control group subjects are given an alternate treatment that they perceive as equally desirable to and that is similar in duration and procedures to the experimental treatment but that is concerned with dependent variables unrelated to the experimental treatment.…

Given the centrality of subject matter to teaching, the design, implementation, and effects of policy related to teaching are unlikely to be understood?by analysts and policymakers alike?unless they explicitly examine the interaction of subject matter with these policies. In this article, we offer a way of anticipating where and how subject matter…

This paper describes the design rationale for a prototype of an innovative assessment product, and the process that led to the design. The goals of the Biomass project were to demonstrate: (1) an assessment product designed to serve two new purposes in the transition from high school to college; and (2) the capability needed to produce this kind…

This document provides some examples of questions that might be asked about research design and analysis, or that might be used in test construction. The accompanying answers form a basic discussion of research design and analysis techniques. Short-answer constructed response answers are provided for questions about: (1) control in an experiment;…

The fervent debate over the last 100 years about quantitative and qualitative research paradigms has resulted in a great divide between quantitative and qualitative researchers, who often view themselves as in competition with each other. This polarization has promoted "uni-researchers," researchers who restrict themselves exclusively to either…

The popularity of sample surveys in evaluation and research makes it necessary for consumers to tell a good survey from a poor one. Several sources were identified that gave advice on how to evaluate a sample design used in a survey study. The sources are either too limited or too extensive to be useful practically. The purpose of this paper is to…

This report provides a summary of progress to date on the study of academic success in high-needs schools and is intended for use by the research team and IES staff. The report begins with a history of the project?s origins and a brief review of pilot study findings. The Introduction concludes with an overview of the conceptual framework and the…

Author, addressing potential Ph.D. candidates, outlines a route to a respectable problem for research,'' involving extensive reading. (Author/SP)

Before a decision is made, relevant questions, both statistical and judgmental, should be asked. If decisions are not the main purpose of the research investigation, (1) a conditional probability might be reported, (2) confidence intervals could be given as summaries of the data, or (3) a Bayesian approach to inference might be used. (Author)