Skip to Main Content

FYS 1038- Identities, Dwellings and Off the Grid Living (Austin)

Spotting Suspicious Science Graphic

A graphic with multiple images and text is on the screen. The top header text states “Something Smells” and in a larger font below it, the text states “Identifying suspicious science.” To the left of this header is a cartoon graphic of a dead fish with stink lines coming from the top of it. There is a line before the next chunk of text denoting a new section. Section two has a header that states “Not all studies are created equal” Below this header, in smaller font, the text states “ in a perfect world, all published scientific studies would be well sourced and credible. However, no field is immune to mistakes or fraud and science is no exception. When looking for scientific studies for your paper, keep an eye out for these signs that could mean its not the best source.” Section three has a header that states “signs you should be skeptical of your scientific source.” Section four has a header on the left side above a graphic of dinosaur bones. The header states “Isn’t this a bit old?” To the right of the header, there is a block of text that states “in some sciences, like medical science, older information might be out of date and no longer considered standard. Always check the publication date of your study if timeliness is a factor in your field.” Section four has a header to the left side above a graphic of dollar bills. The header states “follow the money.” To the right of the graphic and header, a text box reads “sometimes companies pay for favorable studies so they can use them for advertising. Always check who funded a study and if you don’t recognize an organization, look them up to insure there’s no conflict of interest. Section 5 has a header to the left side above a graphic of three stick figures. The header states “sample size shenanigans.” To the right of the graphic and header a text box reads “scientific studies that involve small sample sizes like a dozen people or one tree are useful indicators that further studies should be conducted but are not useful studies for proving a point. Always check the sample size to understand the scope of a study.” Section 6 has a header to the left side above a graphic of two dramatic masks, one laughing and one frowning. The frowning mask is partially hidden behind the smiling one. The header states “what, who are you?” To the right of the graphic and header a text box reads “sometimes a scientist may have a reputation for writing fraudulent studies or hiding conflicts of interest. Look up the authors of the study and check their track record in their respective field to catch potential bad actors.

The second page of the graphic has the same design as the first. Section 1 on page 2 has a header to the left side above a graphic of two pieces of paper with an arrow going from one to another to denote a copy. The header states “run that by me again?” To the right of the graphic and header a text box reads “one of the most crucial aspects of the scientific method is reproducibility: has the study been conducted more than once and produced the same results? If a study haven’t been reproduced, that may indicate something is wrong. “ Section 2 on page 2 has a header to the left side above a graphic of a shark fin poking out of water. The header states “pay to publish.” To the right of the graphic and header a text box reads “some journals will publish any paper as long as the writer pays a large fee. These journals are known as predatory journals and the studies in them usually cannot be trusted as scientifically sound.” Section 3 on page 2 has a header to the left side above a graphic of two people speaking to one another, with one speaking with speech bubbles featuring conflicting arrows. Meanwhile the other person has a question mark over their head. The header states “The lone scientist.” To the right of the graphic and header a text box reads “scientists will sometimes disagree in their respective fields: this is a normal part of the process. But when one scientist is the only one saying something to which all their peers loudly disagree, it’s worthwhile to be skeptical.” Section 4 on page 2 has a header to the left side above a graphic of a clipboard with a math problem on it. The header states “the data doesn’t say that.” To the right of the graphic and header a text box reads “Does the study’s conclusion section match the data section? Or does the conclusions section make sweeping claims about proving something with little backing? If the paper makes a big claim, the data better back it up.” Section 5 on pag2 2 has a header to the left side above a graphic of a person smiling and pointing at themselves with two hands. The header states “According to me, myself and I.” To the right of the graphic and header a text box reads “who is being cited in this paper? If the citations mostly link back to the author’s previous work, that might be a sign that their positions are in opposition to the positions of their respective field.” Section 6 on page 2 has a header to the left side above a graphic of a pot of gold. The header states “too good to be true.” To the right of the graphic and header a text box reads “sometimes scientific papers will propose their study has found simple solutions to huge complicated problems. Sometimes, they have, but as a rule of thumb, if someone claims to have solved an issue that has stumped thousands in their field with a simple solution, it’s worth being skeptical if that conclusion is actually true.” Section 7 on page 2 has a header in the center that reads “Not Sure About A Source?” Below the header a text box reads “If you see one of the above signs of suspicious science, that doesn’t mean your source isn’t trustworthy: only that you need to investigate it first. Look up the author and journal online. See if anyone has written anything like this before. Check if the data seems to match the conclusions. A little extra research into a source can go a long way to preventing spreading suspicious science.” At the bottom of the second page, in small text is text reading: “Source: Popular Mechanics, "6 Warning Signs that a Scientific Study is Bogus", Sarah Fecht, April 8th 2014”

Alt text: 

 

Alt Text Suspicious Science:

A graphic with multiple images and text is on the screen. The top header text states “Something Smells” and in a larger font below it, the text states “Identifying suspicious science.” To the left of this header is a cartoon graphic of a dead fish with stink lines coming from the top of it. There is a line before the next chunk of text denoting a new section.

Section two has a header that states “Not all studies are created equal” Below this header, in smaller font, the text states “ in a perfect world, all published scientific studies would be well sourced and credible. However, no field is immune to mistakes or fraud and science is no exception. When looking for scientific studies for your paper, keep an eye out for these signs that could mean its not the best source.”

Section three has a header that states “signs you should be skeptical of your scientific source.”

Section four has a header on the left side above a graphic of dinosaur bones. The header states “Isn’t this a bit old?” To the right of the header, there is a block of text that states “in some sciences, like medical science, older information might be out of date and no longer considered standard. Always check the publication date of your study if timeliness is a factor in your field.”

Section four has a header to the left side above a graphic of dollar bills. The header states “follow the money.” To the right of the graphic and header, a text box reads “sometimes companies pay for favorable studies so they can use them for advertising. Always check who funded a study and if you don’t recognize an organization, look them up to insure there’s no conflict of interest.

Section 5 has a header to the left side above a graphic of three stick figures. The header states “sample size shenanigans.” To the right of the graphic and header a text box reads “scientific studies that involve small sample sizes like a dozen people or one tree are useful indicators that further studies should be conducted but are not useful studies for proving a point. Always check the sample size to understand the scope of a study.”

Section 6 has a header to the left side above a graphic of two dramatic masks, one laughing and one frowning. The frowning mask is partially hidden behind the smiling one.  The header states “what, who are you?” To the right of the graphic and header a text box reads “sometimes a scientist may have a reputation for writing fraudulent studies or hiding conflicts of interest. Look up the authors of the study and check their track record in their respective field to catch potential bad actors.

The second page of the graphic has the same design as the first.

Section 1 on page 2   has a header to the left side above a graphic of two pieces of paper with an arrow going from one to another to denote a copy. The header states “run that by me again?” To the right of the graphic and header a text box reads “one of the most crucial aspects of the scientific method is reproducibility: has the study been conducted more than once and produced the same results? If a study haven’t been reproduced, that may indicate something is wrong. “

Section 2 on page 2 has a header to the left side above a graphic of a shark fin poking out of water. The header states “pay to publish.” To the right of the graphic and header a text box reads “some journals will publish any paper as long as the writer pays a large fee. These journals are known as predatory journals and the studies in them usually cannot be trusted as scientifically sound.”

Section 3 on page 2 has a header to the left side above a graphic of two people speaking to one another, with one speaking with speech bubbles featuring conflicting arrows. Meanwhile the other person has a question mark over their head. The header states “The lone scientist.” To the right of the graphic and header a text box reads “scientists will sometimes disagree in their respective fields: this is a normal part of the process. But when one scientist is the only one saying something to which all their peers loudly disagree, it’s worthwhile to be skeptical.”

Section 4 on page 2 has a header to the left side above a graphic of a clipboard with a math problem on it. The header states “the data doesn’t say that.” To the right of the graphic and header a text box reads “Does the study’s conclusion section match the data section? Or does the conclusions section make sweeping claims about proving something with little backing? If the paper makes a big claim, the data better back it up.”

Section 5 on pag2 2 has a header to the left side above a graphic of a person smiling and pointing at themselves with two hands. The header states “According to me, myself and I.”  To the right of the graphic and header a text box reads “who is being cited in this paper? If the citations mostly link back to the author’s previous work, that might be a sign that their positions are in opposition to the positions of their respective field.”

Section 6 on page 2 has a header to the left side above a graphic of a pot of gold. The header states “too good to be true.” To the right of the graphic and header a text box reads “sometimes scientific papers will propose their study has found simple solutions to huge complicated problems. Sometimes, they have, but as a rule of thumb, if someone claims to have solved an issue that has stumped thousands in their field with a simple solution, it’s worth being skeptical if that conclusion is actually true.”

Section 7 on page 2 has a header in the center that reads “Not Sure About A Source?” Below the header a text box reads “If you see one of the above signs of suspicious science, that doesn’t mean your source isn’t trustworthy: only that you need to investigate it first. Look up the author and journal online. See if anyone has written anything like this before. Check if the data seems to match the conclusions. A little extra research into a source can go a long way to preventing spreading suspicious science.”

At the bottom of the second page, in small text is text reading: “Source: Popular Mechanics, "6 Warning Signs that a Scientific Study is Bogus", Sarah Fecht, April 8th 2014”

Red Flags of Sources

A graphic listing red flags of information sources. Next to each warning sign is a graphic to symbolize that sign. The title reads "Information source red flags" with two red flag graphics next to it. Below that, the subtitle says "watch out for these potential signs of an untrustworthy source". The warning sides continue as follows: First box: an image of a dollar sign in a hand is next to the text "the source could profit from the information being shared. Example: the source sells a product and claims the product is also shown to work." The second box features a graphic of a briefcase with the following text "the source is a lobbying group, a company or an partisan organization. Example: using statistics provided by the NRA on gun violence." Box three contains a graphic of quotation marks with text "the source contains no citations to back up their claims and only cites their own work. Example: all citations lead back to papers published by the same author." The forth box is that of a thought bubble and has text "the source makes broad sweeping claims that seem too good to be true. Example: the source claims a medical treatment can cure multiple unrelated diseases and disorders" The fifth box contains the image of a man in a hoodie who's face is obscured holding a laptop: the image is meant to depict a hacker. The text says "the source main cites organizations and individuals who are discredited or untrustworthy. Example: the source only cites conspiracy websites and internet forums." Box six contains a graphic of scientific text tubes and has text "the source cites studies with small sample sizes, studies with no peer review, or studies that are retracted. Example: the source claims a study of five people is scientific proof." The seventh box contains a graphic of a hand puppeting a man with the text "the source makes it difficult to find who funds them or who runs their organization OR the source is funded by a clear conflict of interest. Example: a pro fracking site is funded by an oil company." The last box contains a graphic of a microphone and the text "the source makes claims you cannot find reported in other reputable news outlets or journals. Example: the source claims someone can reverse gravity but only they are reporting on it."

Alt Text: 

A graphic listing red flags of information sources. Next to each warning sign is a graphic to symbolize that sign. The title reads "Information Source Red Flags" with two red flag graphics on each side of the title. Below that, the subtitle says "watch out for these potential signs of an untrustworthy source". The warning signs continue as follows: First box: an image of a dollar sign in a hand is next to the text "the source could profit from the information being shared. Example: the source sells a product and claims the product is also shown to work." The second box features a graphic of a briefcase with the following text "the source is a lobbying group, a company or an partisan organization. Example: using statistics provided by the NRA on gun violence." Box three contains a graphic of quotation marks with text "the source contains no citations to back up their claims and only cites their own work. Example: all citations lead back to papers published by the same author." The forth box is that of a thought bubble and has text "the source makes broad sweeping claims that seem too good to be true. Example: the source claims a medical treatment can cure multiple unrelated diseases and disorders" The fifth box contains the image of a man in a hoodie who's face is obscured holding a laptop: the image is meant to depict a hacker. The text says "the source main cites organizations and individuals who are discredited or untrustworthy. Example: the source only cites conspiracy websites and internet forums." Box six contains a graphic of scientific text tubes and has text "the source cites studies with small sample sizes, studies with no peer review, or studies that are retracted. Example: the source claims a study of five people is scientific proof." The seventh box contains a graphic of a hand puppeting a man with the text "the source makes it difficult to find who funds them or who runs their organization OR the source is funded by a clear conflict of interest. Example: a pro fracking site is funded by an oil company." The last box contains a graphic of a microphone and the text "the source makes claims you cannot find reported in other reputable news outlets or journals. Example: the source claims someone can reverse gravity but only they are reporting on it."

Places to Fact Check

  • Snopes is one of the biggest fact checking websites on the internet and covers a very wide range of topics from the newsworthy (political claims, health treatments) to the more niche (small clickbait stories you might see on Twitter). They have a search bar, which can be invaluable to tracking down specific content. While the ratings of (true, false, misleading, ect) on each article are a quick shorthand to gage accuracy of a story, looking at the full reporting is often valuable as ratings can be subjective and the full reporting allows you to make your own judgement calls. 
  • Another general fact checking website. Like Snopes they do have a search feature and they also have a dropdown menu that allows one to browse fact checks by popular topics around the world. They also run Sci Check which fact checks claims about science in the media and Players Guide 2022 which follows groups that seek to influence the 2022 election. The Players Guide is a great resource for recognizing astroturf, which is when lobbying organizations or corporate interest groups impersonate concerned citizens to influence political change. 
  • Another fact checking website that focuses on politics. They have a database that can be used to find claims by verdict, author and type of bias. 
  • Retraction Watch is a database that contains lists of scientific articles that have been retracted. Scientific articles can be retracted for a variety of reasons, but when an article is retracted it is considered not a reliable source of information. One of the great things about Retraction Watch is that the database search includes the reasons an article was retracted in their results which is crucial information that can sometimes be more difficult to find. They also have a blog where they cover retractions on a weekly basis for those interested in scientific fraud. 
  • All Sides is a website that reports on media bias. On their website, they have a media bias chart that labels news sources by their partisan slant. They also have a similar chart for fact checking websites. For those interested in types of media bias, their list of different kinds of bias may be informative.