Introduction

While many professionals in education, psychology, management, and other social science fields perform research and use statistics to analyze results, many more read the results of research and apply it to the real world. Therefore it is vitally important to be able to critically analyze a research report to determine if the methods and results are valid and if they apply to you as a professional. This chapter will look at each of the major sections of the research report and will provide ideas for what to look for, how to apply the information, and how to determine if a specific study is worth incorporating into your work.

Abstract

The abstract is almost always the first section we read in a research report. As such, it should provide valuable information about what is included in the remainder of the report. According to the APA manual, the abstract should be no more than 150 words in order to provide enough information about the paper without replicating large sections. In other words, a good abstract will include, in general, one or two sentences for each of the following:

(1) Statement of the problem;
(2) Brief summary of the literature;
(3) Brief Summary of the methods used in the present research;
(4) Brief summary of the results found in the present research; and
(5) The significance of the present study and/or need for further research

Introduction (Literature Review)

The purpose of the introduction is to give the reader a solid background of the phenomena being studied. It should provide the reader with information that will lead to the statement of the problem. This information can include a need to replicate previous studies due to shortcomings or to apply previous research to a different population. It can also discuss a new theory and the qualitative and quantitative data that has lead to this new idea about a relationship. In general, by the time the introduction section is read, the reader should be able to understand the need for the present study and have a solid understanding of the researcher’s theory.

Methods

The methods section is often the most precisely written part of a research report. Since replication and analyzing methods is so important, a good deal of time should be spent analyzing this section. As a consumer of research, it is imperative for you to understand the foundation of each study and be able to critically analyze how the data that will lead to the results section was derived.

When reading the methods section you should look for information regarding the subjects and the manner in which the subjects were selected. You should be able to discuss the pitfalls of not using randomization, or of various types of randomization. You should be able to understand the strengths and weaknesses of the type of design used and how the researchers used control groups or groups that were not equivalent. The use of standardized procedures is also important, as we ideally want every group to experience the same environment except for the variable(s) being measured. If confounding variables are not controlled for, you should be able to discuss how this lack of control might impact the results of the study.

Issues related to internal and external validity should be carefully addressed as well. Look for how the researchers addressed subject maturation in longitudinal or longer term studies, how they handled extreme scores and the tendency for scores to regress toward the mean, and how they dealt with differences in mortality or drop out rates between groups. And finally, you should be aware of what assessment procedures were used, whether these instruments are valid and reliable, and whether or not they were used correctly

Results

The results section will likely require at least some basic understanding of statistics as this section is often the most technical. The main ideas that should be analyzed in a results section include the statistical procedures used, the reporting of the numerical findings, and the determination of significance. The procedures used should correspond with the data they are working with. For instance, if the data is nominal or ordinal and the procedure used was parametric (See Summary of Statistics), then the results will be skewed at best and completely invalid at worst.

The numerical results of the statistics should also be reported so if the study shows a significant t test, the actual t-test score, the degrees of freedom, and the probability of error (p) as compared with the acceptable error (a) should all be reported. Finally, the decision for how much error was acceptable for the researchers should be addressed. Typically .05 or .01 are used, representing an acceptable error level of 5% or 1%, respectively. If the level is set different from these two, the rationale should be addressed.

Discussion

The discussion section allows the researchers to qualify their results and to discuss areas of concern regarding their research and areas of further study that may be needed. A good researcher will want to critique his or her own study before others do. This provides the reader with a more qualitative understanding of the findings. Shortcomings such as a lack of randomization, failure to use a control group, as well as many other issues, should be addressed and explained. Not only does this show that the researcher didn’t just arbitrarily omit steps, but it may help future researchers improve upon the present study.

The results of the research and the relevance to the topic at large should also be addressed. This is mainly to provide insight into what research might be completed in the future to either fix problems in the current study or advance the original theory. The researcher is typically well versed in the topic of study, or at least should be, and can help lead the profession to further advancements and knowledge through his discussion of the current study. The discussion section is the most qualitative part of a research report and therefore may be the most important section of the entire paper.

References

References allow the reader to look up information and read more about particular phenomena or research that was discussed in the current study. While reading the study, the quality of the references should be considered. If, for instance, a study uses research from unprofessional or questionable sources, the validity of his arguments will also be questionable. Look for journals that are known and respected in the profession as this will help to determine the strength of the supporting material that led to the development of the original theory.

Appendices

A research study can have many appendices or it can have none. When addressing this section, look at the need for the data in the first place. Did the author place information here just to take up space? Is the information confusing and difficult to understand? Was it addressed in he paper or just included arbitrarily? Does it provide information that helps the reader understand the methods or results of the study? And finally, does it provide assistance in analyzing the study as a whole and the need for replication or further research?

Chapter Conclusion

While many professionals in education and the social sciences perform research, the majority of us use this information in real life application. Teachers use research regarding teaching methods and learning style to help improve the education of their students. Therapists use research to provide better treatment for specific mental illnesses. Managers use research to help them improve retention rates, worker satisfaction, or communication. The purpose of research, then, is not merely to gather information, but to communicate this information to the research consumers.

Through this text, you should have a solid understanding of the importance of research, the methods of developing a hypothesis, and the specific designs used for particular types of research. You should understand the importance of standardization, randomization, controlling for confounds, and assuring internal and external validity. You should have a basic understanding of descriptive and inferential statistics, and be aware that the foundation for your study and the discussion of results are often more important than the results themselves.

Perhaps most importantly though, you should understand how to critically analyze a research report from start to finish. You should be able to address the strengths and weaknesses of particular designs and the specific methods of a study. You should be able to discuss results and combine studies in order to develop your own theories and to apply this information to your professional life. Research is not an end in itself, nor is it a means to an end. Research is merely a continuing and vital part of our need to understand and grow.