Detecting Bias
Bias, whether done intentionally or not, occurs when a statement reflects a partiality, preference, or prejudice for or against an object, person, place, or idea. Listed below are problems to look for when determining if the source is biased.
- Availability Bias -- this is a tendency for people to overestimate probabilities of events related to memorable or dramatic occurrences [e.g., after 9/11, people took vacation by traveling by car rather than airplane even though, statistically, car travel is much more dangerous]. This form of bias in a research study can take the form of an example used to support author’s argument or the design a case study focused around a particular event. Unless the purpose of the study is to illuminate new understanding around a memorable or dramatic occurrence, be critical of studies that use this type of measurement to examine a research problem. A seemingly mundane or uneventful occurrence can be just as valid in developing solutions to a problem or advancing new knowledge.
- Distortion or Stretching of the Facts -- this refers to the act of making issues, problems, or arguments appear more extreme by using misinformation or exaggerated and/or imprecise language to describe research outcomes [e.g., “Everyone agreed the policy was a complete disaster.” Who's everyone? How was data gathered to come to this conclusion? And, how does one specifically define something as a "disaster"? Is there sufficient evidence to support such a broad statement?]. Look for declarative statements that lack appropriate reference to supporting evidence or are follow up with detailed analysis.
- Flawed Research Design -- bias can enter the narrative as a result of a poorly designed study; this may include a claim or generalization about the findings based upon too small a sample, manipulating statistics, omitting contrary conclusions from other studies, or failing to recognize negative results [results that do not support the hypothesis].
- Lack of Citations -- it is acceptable to issue a broad declarative statement if it is clearly supported and linked to evidence from your study [e.g., "Testimony during Congressional hearings shows that the Department of Education is reluctant to act so teachers must do so"]. This problem refers to statements or information presented as fact that does not include proper citation to a source or to sources that support the researcher's position, or that are not statements explicitly framed as the author's opinion.
- Misquoting a Source -- this is when an author rewords, paraphrases, or manipulates a statement, the information about a source is incomplete, or a quote is presented in such a way that it misleads or conveys a false impression. This is important when paraphrasing another author. If you cannot adequate summarize a specific statement, finding, or recommendation, use a direct quote to avoid any ambiguity.
- Persuasive or Inflammatory Language -- using words and phrases intended to elicit a positive or negative response from the reader or that leads the reader to arrive at a specific conclusion [e.g., referring to one group in an armed conflict as “terrorists” and the other group as “peace-loving”].
- Selective Facts -- taking information out of context or selectively choosing information that only supports the argument while omitting the overall context or vital supporting evidence.
- Statistical Survey Bias -- this can take several forms so, if data is presented in a study that was gathered by the author(s), examine it critically for the following possible biases:
- Measurement Error: this results from problems with the process by which data was gathered, such as, the use of leading questions that influence the response rate or that are biased toward what respondents believe is socially desirable because most people want to present themselves favorably. The only way to assess bias in these cases is to have access to the survey instrument used to gather data.
- Sample Size: increasing the number of a sample, for example the number of people interviewed, does not necessarily decrease bias, but look to see if the sample used is representative of the population under study to ensure that any generalizations or conclusions from the interpretation of the data is valid.
- Undercoverage: this refers to the method of data gathering that is a result of non-response to a survey because some subjects do not have the opportunity to participate. In looking at data, be sure to understand the percentage of non-responses to a survey or groups of people who were not included.
- Voluntary Response: this bias occurs when respondents to a survey are self-selected, resulting in an overrepresentation of individuals who have strong opinions [e.g., data from a radio call-in show]. Be an especially critical reader of web-based surveys about controversial topics if the author(s) have not indicated how they interpreted thew data from voluntary surveys.
NOTE: The act of determining bias in scholarly research is also an act of constant self-reflection. Everyone has biases. Therefore, it is important that you minimize the influence of your own biases by approaching the assessment of another person's research introspectively and with a degree of self-awareness.
"Availability Bias, Source Bias, and Publication Bias in Meta-Analysis." In Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. 3rd Edition. (London: SAGE Publications, 2015), pp. 513-551; "Bias." In Key Concepts in Social Research. Geoff Payne and Judy Payne. (London: SAGE Publications, 2004), pp. 28-31; Evaluating Sources. Lakeland Library Research Guides. Lakeland Community College; Podsakoff, Philip M. et al. “Common Method Biases in Behavioral Research: A Critical Review of the Literature and Recommended Remedies.” Journal of Applied Psychology 88 (October 2003): 879-903; Stereotypes and Biased Language. The Writing Lab and The OWL. Purdue University; Bias in Survey Sampling. Stat Trek Online Tutorials; What is Availability Bias? InnovateUs.net.