In undergraduate research settings, students are likely to encounter anomalous data, that is, data that do not meet their expectations. Most of the research that directly or indirectly captures the role of anomalous data in research settings uses post-hoc reflective interviews or surveys. These data collection approaches focus on recall of past…

The development in information science has enabled an explosive growth of data, which attracts more and more researchers to engage in the field of big data analytics. Noticeably, in many real-world applications, large amounts of data are imbalanced data since the events of interests occur infrequently. Classification of imbalanced data is an…

This document deals with a common problem: the fact that "needs assessment" is often required by law and regulation, but there is little agreement on what the term means or what the process entails. The uses and definitions of the term "needs assessment" are examined and categorized along with the various methodologies which are ascribed to it.…

The selection of appropriate data sources for institutional research is discussed. It is suggested that the provision of timely, accurate information for administrative decision-making depends on the researcher's ability to determine which data are relevant to the problem. This is of particular concern for institutional researchers because they…

Conducted a content analysis of research articles (N=80) published in professional journals to assess the reliance on one family member for data to be generalized to the conjugal or family unit. Results showed that over time (1965-1978) there has been a continued reliance on individual family members for data. (JAC)

One problem faced by researchers is the selection of the most appropriate research method or methods, given the context of the research setting. This dilemma does not concern the choice of the best research method given the program objectives or research questions, nor the selection of the best state-of-the-art methods. The issue concerns the…

This chapter presents a number of problems related to designing local-level summative evaluation studies and suggests solutions to these problems. Three general areas are covered. First, many of the motivational, organizational, and attitudinal constraints likely to face the local-level evaluator are discussed. Second, various threats to the…

The preparation and use of data needed to make objective, useful comparisons in higher education are considered. The needs for comparative data in higher education, including demands for accountability and strategic planning, are reviewed. In addition, data-related problems are considered: problems of narrowness, misrepresentation, sameness,…

Decision making in the public sector encompasses many topics of interest to the academic researcher--environmental issues, health and human services, budget planning, and so on. Expertise in data collection and analysis is critical to the policy-making process and can be provided by academic researchers. But the "real world" policymaker…

The paper discusses problems faced by policy makers in utilizing social science research in decision making on educational issues and problems. The term social science research is interpreted as including both educational evaluation and educational research. The history of social science research and critical issues associated with social science…

Severe methodological problems were encountered in an attempt to investigate the impact of Education Division programs on girls and women--and, in particular, to learn whether these programs have a differential impact by sex or by race/ethnicity. (The "Education Division" comprises the Office of Education, the National Institute of…

Deliberations about social policy often center on estimates of harm or benefit generated by different interest groups. Problems in what is measured and how it is measured are illustrated by a discussion of research into sexual abuse and rape. Advocacy research is an unreliable foundation for social policy formation. (SLD)

Four evaluation methods for determining educational policy are objectivism; subjectivism; emotive-imperative; and instrumentalism. For the objectivist, the rationale for undertaking an evaluation is based on empirical verification or non-verification of claims of existing policies or activities. Secondly, the objectivist is interested in…

Directors of educational projects need to be aware of the consequences their decisions may have for evaluation and appreciate the need for working closely with their evaluators from the earliest planning stage. Attempting to address the needs of project directors and evaluators, this guidebook deals with one central aspect of project…

Measuring cognitive achievement gains in project evaluation is dealt with. This guide provides those concerned with project evaluation with the basic tools for conducting technically sound, interpretable evaluation studies. Exotic designs are avoided and five basic models which appear feasible to implement in real-world settings are focused on.…