violen t m et aph or s to read and...“be sk ept i cal . bu t w hen you get proof , accept proof...

13
8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 1/13 V iolent metaphors Thoughts from the intersection of science, pseudoscience, and conflict. How to read and understand a scientific paper: a guide for non-scientists August 25, 2013 January 31, 2015 Jennifer Raff Update (8/30/14): I’ve wrien a shorter version of this guide for teachers to hand out to their classes. If you’d like a PDF, shoot me an email: jenniferraff (at) utexas (dot) edu. Last week’s post (The truth about vaccinations: Your physician knows more than the University of Google (hps://violentmetaphors.com/2013/08/14/the-truth-about-vaccinations-your-physician-knows-more-than-the- university-of-google/) ) sparked a very lively discussion, with comments from several people trying to persuade me (and the other readers) that their paper disproved everything that I’d been saying. While I encourage you to go read the comments and contribute your own, here I want to focus on the much larger issue that this debate raised: what constitutes scientific authority? It’s not just a fun academic problem. Geing the science wrong has very real consequences. For example, when a community doesn’t vaccinate children because they’re afraid of “toxins” and think that prayer (or diet, exercise, and “clean living”) is enough to prevent infection, outbreaks happen (hp://blogs.dallasobserver.com/unfairpark/2013/08/theres_a_measles_outbreak_at_v.php) . “Be skeptical. But when you get proof, accept proof.” –Michael Specter What constitutes enough proof? Obviously everyone has a different answer to that question. But to form a truly educated opinion on a scientific subject, you need to become familiar with current research in that field. And to do that, you have to read the “primary research literature” (often just called “the literature”). You might have tried to read scientific papers before and been frustrated by the dense, stilted writing and the unfamiliar jargon. I remember feeling this way! Reading and understanding research papers is a skill which every single doctor and scientist has had to learn during graduate school. You can learn it too, but like any skill it takes patience and practice. I want to help people become more scientifically literate, so I wrote this guide for how a layperson can approach reading and understanding a scientific research paper. It’s appropriate for someone who has no background whatsoever in science or medicine, and based on the assumption that he or she is doing this for the purpose of geing a basic understanding of a paper and deciding whether or not it’s a reputable study. The type of scientific paper I’m discussing here is referred to as a primary research article. It’s a peer-

Upload: others

Post on 20-Mar-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 1/13

Violent metaphors

Thoughts from the intersection of science, pseudoscience,and conflict.

How to read and understand a scientific paper: a guidefor non-scientists

August 25, 2013January 31, 2015 Jennifer RaffUpdate (8/30/14): I’ve wri�en a shorter version of this guide for teachers to hand out to their classes. If you’d like aPDF, shoot me an email: jenniferraff (at) utexas (dot) edu.

Last week’s post (The truth about vaccinations: Your physician knows more than the University of Google(h�ps://violentmetaphors.com/2013/08/14/the-truth-about-vaccinations-your-physician-knows-more-than-the-university-of-google/)) sparked a very lively discussion, with comments from several people trying to persuademe (and the other readers) that their paper disproved everything that I’d been saying. While I encourage youto go read the comments and contribute your own, here I want to focus on the much larger issue that thisdebate raised: what constitutes scientific authority?

It’s not just a fun academic problem. Ge�ing the science wrong has very real consequences. For example,when a community doesn’t vaccinate children because they’re afraid of “toxins” and think that prayer (ordiet, exercise, and “clean living”) is enough to prevent infection, outbreaks happen(h�p://blogs.dallasobserver.com/unfairpark/2013/08/theres_a_measles_outbreak_at_v.php).

“Be skeptical. But when you get proof, accept proof.” –Michael Specter

What constitutes enough proof? Obviously everyone has a different answer to that question. But to form atruly educated opinion on a scientific subject, you need to become familiar with current research in that field. And to do that, you have to read the “primary research literature” (often just called “the literature”). Youmight have tried to read scientific papers before and been frustrated by the dense, stilted writing and theunfamiliar jargon. I remember feeling this way! Reading and understanding research papers is a skill whichevery single doctor and scientist has had to learn during graduate school. You can learn it too, but like anyskill it takes patience and practice.

I want to help people become more scientifically literate, so I wrote this guide for how a layperson canapproach reading and understanding a scientific research paper. It’s appropriate for someone who has nobackground whatsoever in science or medicine, and based on the assumption that he or she is doing this forthe purpose of ge�ing a basic understanding of a paper and deciding whether or not it’s a reputable study.

The type of scientific paper I’m discussing here is referred to as a primary research article. It’s a peer-

Page 2: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 2/13

The type of scientific paper I’m discussing here is referred to as a primary research article. It’s a peer-reviewed report of new research on a specific question (or questions). Another useful type of publication is areview article. Review articles are also peer-reviewed, and don’t present new information, but summarizemultiple primary research articles, to give a sense of the consensus, debates, and unanswered questions withina field. (I’m not going to say much more about them here, but be cautious about which review articles youread. Remember that they are only a snapshot of the research at the time they are published. A review articleon, say, genome-wide association studies from 2001 is not going to be very informative in 2013. So muchresearch has been done in the intervening years that the field has changed considerably).

Before you begin: some general adviceReading a scientific paper is a completely different process than reading an article about science in a blog ornewspaper. Not only do you read the sections in a different order than they’re presented, but you also have totake notes, read it multiple times, and probably go look up other papers for some of the details. Reading asingle paper may take you a very long time at first. Be patient with yourself. The process will go much fasteras you gain experience.

Most primary research papers will be divided into the following sections: Abstract, Introduction, Methods,Results, and Conclusions/Interpretations/Discussion. The order will depend on which journal it’s published in.Some journals have additional files (called Supplementary Online Information) which contain importantdetails of the research, but are published online instead of in the article itself (make sure you don’t skip thesefiles).

Before you begin reading, take note of the authors and their institutional affiliations. Some institutions (e.g.University of Texas) are well-respected; others (e.g. the Discovery Institute (h�p://www.discovery.org/)) mayappear to be legitimate research institutions but are actually agenda-driven. Tip: google “Discovery Institute”to see why you don’t want to use it as a scientific authority on evolutionary theory.

Also take note of the journal in which it’s published. Reputable (biomedical) journals will be indexed byPubmed (h�p://www.ncbi.nlm.nih.gov/pubmed). [EDIT: Several people have reminded me that non-biomedical journals won’t be on Pubmed, and they’re absolutely correct! (thanks for catching that, Iapologize for being sloppy here). Check out Web of Science (h�p://thomsonreuters.com/web-of-science/) for a more complete index of science journals. And please feel free to share other resourcesin the comments!] Beware of questionable journals (h�p://scholarlyoa.com/individual-journals/).

As you read, write down every single word that you don’t understand. You’re going to have to look themall up (yes, every one. I know it’s a total pain. But you won’t understand the paper if you don’t understand thevocabulary. Scientific words have extremely precise meanings).

Step-by-step instructions for reading a primary research article

1. Begin by reading the introduction, not the abstract.

The abstract is that dense first paragraph at the very beginning of a paper. In fact, that’s often the only part of apaper that many non-scientists read when they’re trying to build a scientific argument. (This is a terrible practice—don’t do it.). When I’m choosing papers to read, I decide what’s relevant to my interests based on a combinationof the title and abstract. But when I’ve got a collection of papers assembled for deep reading, I always read theabstract last. I do this because abstracts contain a succinct summary of the entire paper, and I’m concerned aboutinadvertently becoming biased by the authors’ interpretation of the results.

2. Identify the BIG QUESTION.

Not “What is this paper about”, but “What problem is this entire field trying to solve?”

Page 3: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 3/13

Not “What is this paper about”, but “What problem is this entire field trying to solve?”

This helps you focus on why this research is being done. Look closely for evidence of agenda-motivated research.

3. Summarize the background in five sentences or less.

Here are some questions to guide you:

What work has been done before in this field to answer the BIG QUESTION? What are the limitations of thatwork? What, according to the authors, needs to be done next?

The five sentences part is a li�le arbitrary, but it forces you to be concise and really think about the context of thisresearch. You need to be able to explain why this research has been done in order to understand it.

4. Identify the SPECIFIC QUESTION(S)

What exactly are the authors trying to answer with their research? There may be multiple questions, or justone. Write them down. If it’s the kind of research that tests one or more null hypotheses, identify it/them.

Not sure what a null hypothesis is? Go read this(h�p://psychology.ucdavis.edu/faculty_sites/sommerb/sommerdemo/stat_inf/null.htm), then go back to my lastpost and read one of the papers that I linked to (like this one(h�p://pediatrics.aappublications.org/content/125/6/1134.full.pdf+html)) and try to identify the null hypothesesin it. Keep in mind that not every paper will test a null hypothesis.

5. Identify the approach

What are the authors going to do to answer the SPECIFIC QUESTION(S)?

6. Now read the methods section. Draw a diagram for each experiment, showing exactly what theauthors did.

I mean literally draw it. Include as much detail as you need to fully understand the work. As an example,here is what I drew to sort out the methods for a paper I read today (Ba�aglia et al. 2013: “The first peoplingof South America: New evidence from Y-chromosome haplogroup Q”(h�p://www.plosone.org/article/info:doi/10.1371/journal.pone.0071390)). This is much less detail than you’dprobably need, because it’s a paper in my specialty and I use these methods all the time. But if you werereading this, and didn’t happen to know what “process data with reduced-median method using Network”means, you’d need to look that up.

Page 4: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 4/13

(h�ps://violentmetaphors.files.wordpress.com/2013/08/ba�aglia-et-al-methods.jpg)

You don’t need to understand the methods in enough detail to replicate the experiment—that’s somethingreviewers have to do—but you’re not ready to move on to the results until you can explain the basics of themethods to someone else.

7. Read the results section. Write one or more paragraphs to summarize the results for eachexperiment, each figure, and each table. Don’t yet try to decide what the results mean, just writedown what they are.

You’ll find that, particularly in good papers, the majority of the results are summarized in the figures and

Page 5: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 5/13

You’ll find that, particularly in good papers, the majority of the results are summarized in the figures andtables. Pay careful a�ention to them! You may also need to go to the Supplementary Online Information fileto find some of the results.

It is at this point where difficulties can arise if statistical tests are employed in the paper and you don’t haveenough of a background to understand them. I can’t teach you stats in this post, but here(h�p://explorable.com/statistics-tutorial), here (h�p://owl.english.purdue.edu/owl/resource/672/05/), and here(h�p://bobhall.tamu.edu/FiniteMath/Module8/Introduction.html)are some basic resources to help you. ISTRONGLY advise you to become familiar with them.

THINGS TO PAY ATTENTION TO IN THE RESULTS SECTION:

-Any time the words “significant” or “non-significant” are used. These have precise statistical meanings. Readmore about this here (h�p://www.dcscience.net/?p=6518).

-If there are graphs, do they have error bars (h�p://en.wikipedia.org/wiki/Error_bar)on them? For certaintypes of studies, a lack of confidence intervals is a major red flag.

-The sample size. Has the study been conducted on 10, or 10,000 people? (For some research purposes, asample size of 10 is sufficient, but for most studies larger is be�er).

8. Do the results answer the SPECIFIC QUESTION(S)? What do you think they mean?

Don’t move on until you have thought about this. It’s okay to change your mind in light of the authors’interpretation—in fact you probably will if you’re still a beginner at this kind of analysis—but it’s a really goodhabit to start forming your own interpretations before you read those of others.

9. Read the conclusion/discussion/Interpretation section.

What do the authors think the results mean? Do you agree with them? Can you come up with any alternativeway of interpreting them? Do the authors identify any weaknesses in their own study? Do you see any thatthe authors missed? (Don’t assume they’re infallible!) What do they propose to do as a next step? Do youagree with that?

10. Now, go back to the beginning and read the abstract.

Does it match what the authors said in the paper? Does it fit with your interpretation of the paper?

11. FINAL STEP: (Don’t neglect doing this) What do other researchers say about this paper?

Who are the (acknowledged or self-proclaimed) experts in this particular field? Do they have criticisms of thestudy that you haven’t thought of, or do they generally support it?

Here’s a place where I do recommend you use google! But do it last, so you are be�er prepared to think criticallyabout what other people say.

(12. This step may be optional for you, depending on why you’re reading a particular paper. But for me, it’scritical! I go through the “Literature cited” section to see what other papers the authors cited. This allows meto be�er identify the important papers in a particular field, see if the authors cited my own papers(KIDDING!….mostly), and find sources of useful ideas or techniques.)

Now brace for more conflict– next week we’re going to use this method to go through a paper on a

Page 6: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 6/13

Now brace for more conflict– next week we’re going to use this method to go through a paper on acontroversial subject! Which one would you like to do? Shall we critique one of the papers I posted last week?

UPDATE: If you would like to see an example, you can find one here(h�ps://violentmetaphors.com/2013/09/08/an-example-of-how-to-read-a-vaccine-safety-study/) ———————————————————————————————————

I gratefully acknowledge Professors José Bonner and Bill Saxton for teaching me how to critically read andanalyze scientific papers using this method. I’m honored to have the chance to pass along what they taughtme.

Do you have anything to add to this guide? A completely different approach that you think is be�er? Additionalquestions? Links to other resources? Please share in the comments!

Science journal club, science

488 thoughts on “How to read and understand a scientific paper: aguide for non-scientists”

1. SARAH RUGHEIMER

grammarly.com

Free Grammar CheckerGrammarly makes sure your writingis effective and mistake-free.

grammarly.com

Free Grammar CheckerGrammarly makes sure your writingis effective and mistake-free.

Advertisements

Page 7: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 7/13

1. SARAH RUGHEIMERApril 21, 2016 / 1:54 pmOne other great resource to get research articles besides pubmed is arXiv.org. They have particularlyarticles in the natural sciences, all open free of charge (the author uploads their own generated copy evenfor articles which eventually appear in embargoed journals). You can see if the article is accepted,published, or simply submi�ed in the meta description which is useful to tell if it has been peer-reviewedand which journal it is submi�ed to.

Another resource of course is Sci-hub to get nearly any primary research article you want instantly.Fantastic for particularly for citizens or scientists not at large research Universities with wide journalaccess, but perhaps less legal at this time.

Reply2. JCANTOR2013

April 29, 2016 / 9:06 amI sent you an email looking for the PDF version of this information to relay to my students. If your emailaddress at Utexas is no longer active, please provide another one that I can reach you. Thanks in advance.

ReplyJENNIFER RAFFApril 29, 2016 / 1:09 pmAlready sent to you.

ReplyCARRIE GRABOWICZJuly 5, 2016 / 12:24 pmI was interested in a .pdf of this article, as well. I teach an undergraduate research methods courseand I would like to make this part of their supplemental reading. My email [email protected]

ReplyANONYMOUSDecember 22, 2016 / 3:31 amYour article is quite helpful. Please send me the pdf file. My email: [email protected] Thank you!

ReplyALIFebruary 9, 2017 / 12:01 pmI would also like the PDF file for my students.

my email address is [email protected]

ReplyANONYMOUSMay 28, 2017 / 7:39 pmAloha! Great article. Could I also have permission/pdf to share with my nursing students?Thank you!

Reply

3. Pingback: Very useful (for scientists too): How to read and understand a scientific paper: a guide for non-

Page 8: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 8/13

3. Pingback: Very useful (for scientists too): How to read and understand a scientific paper: a guide for non-scientists – microBEnet: the microbiology of the Built Environment network.

4. Pingback: How to read and understand a scientific paper: a guide for non-scientists | Viral BioinformaticsResource Center

5. MICHAELMay 11, 2016 / 1:33 pmHello,

Thank you very much for posting this. I have been looking for something like this for ages. Great blog too!Keep up the good work.

Reply6. JOSH

May 16, 2016 / 1:18 pmHello thanks for writing this, as a biology undergrad student I hope that it helps me be�er understandhow to read science papers! Because at the moment I find it difficult. My question is: How do I go aboutdoing number 11? Like where can I find critiques of papers? Especially recent ones!

ReplyKARI YOUNGJune 24, 2016 / 2:00 pmThere isn’t really a clearinghouse of critiques of papers. Instead, you find this either in review articles orin other primary articles that cite the article you’re reading. Often the commentary is subtle, so it takespractice to learn to read between the lines.

Reply7. Pingback: How to read a paper | Machine Learning Blog of Xiang8. Pingback: Почему нужно делать прививки детям? Железные аргументы за вакцинацию. - Сибирячка

мама9. Pingback: Dear parents, you are being lied to. - Spielzeug Online Shop

10. Pingback: Papers: Where To Find Them And How To Read Them – The Nevil Lab Blog11. Pingback: How to flunk out of the University of Google – Violent metaphors12. Pingback: Дорогие родители, вам лгут! | Лампа13. Pingback: How to read and understand a scientific paper: a guide for non-scientists – guo zhiling14. Pingback: DEAR PARENTS, YOU ARE BEING LIED TO. – Health Portal15. Pingback: How to read and understand a scientific paper: a guide for non-scientists – Peyton16. SAM

August 2, 2016 / 1:05 amThanks a lot it really helped me

Reply17. Pingback: Research teaching | Pearltrees18. Pingback: Cách đọc và phân tích một bài báo khoa học – Young Anthropologists19. Pingback: AP Bio Project: Lab Lyf – Science with Mr. Le20. PAT MARSTELLER

September 18, 2016 / 11:28 amThought you’d like to know that I assign this to my Intro to Research class and also to our IMSDundergrads and grad students.

ReplyJENNIFER RAFF

Page 9: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 9/13

JENNIFER RAFFSeptember 18, 2016 / 4:38 pmthank you so much!

Reply21. YOOO00O

October 10, 2016 / 4:20 amThanks you a lot! It helps me to read my papers in computer science. But I am a li�le bit confused how todistinguish SPECIFIC QUESTION from BIG QUESTION. Sometimes they look the same to me.

Reply22. KARI AZEVEDO

October 10, 2016 / 11:25 amHello, Jennifer,With your permission, I would love a pdf version of this article to share with my undergraduateresearchers in the McNair Scholars Program at the University of Central Missouri. Thanks, Kari Azevedo

Reply23. Pingback: Lire et comprendre un article scientifique, le guide pour débutants – Jennifer Raff | Associations

Libres24. Pingback: ReBlog: How to read and understand a scientific paper: a guide for non-scientists – The

Philosophical Fish25. Pingback: autisticagainstantivaxxers26. ALEX VERKADE

December 15, 2016 / 10:28 amHi Jennifer, I think your pieces are great. Since you say “If you like it, share it” I took the liberty oftranslating thiis useful piece into Dutch and sharing it through LinkedIn, here:h�ps://www.linkedin.com/pulse/hoe-lees-en-begrijp-je-een-wetenschappelijk-artikel-voor-alex-verkade.Please let me know whether you are satisfied with my mention of your authorship. Cheers! Alex

ReplyCHRISDecember 17, 2016 / 10:44 pmWow, this is just a silly off topic comment:

I saw your name, and then started to yearn for yummy yummy Cafe Noir cookies. Our family reallyenjoyed our visit to h�p://zaansmuseum.nl/?lang=en which included a functioning Verkade cookiemanufacturing display (my husband’s father was from near Amsterdam, and this is one place hubbyliked to go to as a child). It is a museum of windmills and Dutch history, especially industrial history.

Though, on further review is not quite off topic. I followed your links and discovered you are veryactive in science communication. Most excellent work, and I hope the children of my husband’s cousinget to learn and participate in what you have worked on (they should be in secondary school now).

ReplyALEX VERKADEDecember 19, 2016 / 4:44 am

Thanks Chris! There is a good probability that your husband’s cousin’s kids have seen at least some

Page 10: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 10/13

Thanks Chris! There is a good probability that your husband’s cousin’s kids have seen at least someof our projects in school. And yes, Verkade is a last name famously associated with cookies andchocolate in the Netherlands. Unfortunately, no giant chocolate family inheritances for me. I’m notrelated to the cookie branch :).

Reply27. Pingback: Academic Papers – DiscVentionsTech28. Pingback: Highlight: There’s so much research out there but how can I understand a research paper if I’m

not a scientist? – My Macular Degeneration Journey/Journal29. TAMMY BEHRENS

January 8, 2017 / 3:39 pmCan you please send me a pdf of the article or even the shorter article I saw on Facebook? I teachbiomedical science.

Reply30. Pingback: Selviytymisopas tieteellisiin julkaisuihin – Jan Ignatius31. DANIEL ANSONG

February 5, 2017 / 4:53 amDear Jennifer, i have read your article title How to read and understand scientific paper and would be mostgrateful if you could send to me the pdf version. Am a lecture in Ghana. Thank you in advance

Reply32. MARIA MOS, TILBURG UNIVERSITY (THE NETHERLANDS

February 7, 2017 / 11:03 amThank you for this post! I will distribute it to my intro to research methodology in Communication andInformation Sciences class – they had to read their first academic journal article this week and found itpre�y tough, so thanks in advance from them, too

Reply33. NYREE

February 8, 2017 / 6:48 amHi Jennifer,This is a great piece. I would really appreciate a copy of the PDF to distribute to a first year course I amteaching at Australian Catholic University. I think this will really help to give students some tools to dealwith and be�er understand the scientific literature.Thanks, Nyree

Reply34. PAUL K. STRODE

February 8, 2017 / 4:55 pmHi Jennifer: A colleague sent me this link to the guide for reading a scientific paper. I remember readingthrough it when it first came out and really appreciate what you’ve done. However, a couple of heads-up.

1) Your link to “questionable journals” takes us to Jeffrey Beall’s lists of predatory journals and publishers.Jeffrey is a friend of mine and I’ve had him speak to my Science Research Seminar class the last two years.CU Denver made Jeffrey take down the site because it was becoming a legal nightmare for the University;they felt they could no longer support him. When the FTC started coming after OMICS, OMICS in turncame after Jeffrey. It’s really unfortunate that we no longer have access to this great resource.

2) Your link to information about the null hypothesis has some really misleading information in it. From

Page 11: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 11/13

2) Your link to information about the null hypothesis has some really misleading information in it. Fromthe get-go: “[T]he purpose of research is to answer a question or test a prediction, generally stated in theform of hypotheses.”

Hypotheses are not predictions. It’s a common misunderstanding and is taught incorrectly throughout astudent’s science education. I published on this in the American Biology Teacher’s September 2015 issue.I’ve also blogged about it (h�ps://mrdrscienceteacher.wordpress.com/2014/11/02/teaching-the-hypothesis/).

For example, the very first example of a hypothesis presented at the site is: “People who take a driversafety course will have a lower accident rate than those who do not take the course.”

This is a prediction based on the hypothesis that driver safety courses produce more cautious drivers.

The author goes on to say: “The research hypothesis (or hypotheses — there may be more than one) is ourworking hypothesis — our prediction, or what we expect to happen.”

No.

The author confuses experimental hypotheses with statistical null hypotheses. Indeed, the concept of thenull comes from statistics, not the other way around.

Here is how I see it:

Hey, there are fruit flies on my bananas. Is it that fruit flies just like bananas, or that they are a�racted tothem because they are ripe?

I bet that the fruit flies are a�racted to the bananas because they are ripe.

Hypothesis: Fruit flies are a�racted to bananas when they are ripe.

Now at this point, yes, you could come up with a direct opposite statement and call it the null hypothesis,but I think that’s a waste of time. I also NEVER see this done in scientific publications, but maybe I haven’tlooked closely enough. If a tested hypothesis is provided (and not wri�en as a prediction, it is alwayswri�en as a description of a possible pa�ern (generalizing hypothesis) or as an explanation (explanatoryhypothesis). There is never an null statement paired with it that I have ever seen.

My argument is that the null statistical hypothesis is a mathematical expression borne from inferentialstatistics (e.g. O = E, u1 = u2, r = 0). The null statistical hypothesis states a current condition (not aprediction).

For Chi-square: The observed distribution and expected distributions ARE equal and any differences wesee between them are just chance differences.

For the t-Test: The two populations from which the samples were taken HAVE equal means and thedifference we observe between the sample means is merely a result of sampling error.

For Regression: There IS no relationship between the X and Y variables and the appearance of arelationship is accidental.

I disagree that a null hypothesis can be a prediction of what should happen as the result of an experiment

Page 12: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 12/13

I disagree that a null hypothesis can be a prediction of what should happen as the result of an experimentif the experimental hypothesis is false. That just mucks it all up and adds to the confusion that at leastsome kinds of hypotheses can be wri�en as predictions.

We never accept the statistical null hypothesis unless it is actually true and, for example, the observeddistribution does in fact equal the expected distribution. If this is the case, then there is no need for doing aChi-square Test. If two sample means turn out equal, then there is no reason to do a t-Test. However, if theerror bars are huge, I’d be skeptical of concluding that the sample means are at all representative of thepopulations from which they were taken.

If the probability of ge�ing the test statistic (e.g. Sum Chi-square) by accident is greater than 0.05, thenthe null statistical hypothesis cannot be confidently rejected. What I would say is this:

“I cannot reject the null statistical hypothesis given the greater than 0.05 probability that the differencesbetween the observed distribution and the expected distribution occurred by accident.”

Reply35. Pingback: How to read and understand a scientific paper: a guide for non-scientists – FatimahAljawi36. ANONYMOUS

May 17, 2017 / 10:53 amThanks for sharing!

Reply37. SARAH ANDRONICUS

May 23, 2017 / 3:30 pmI feel that we all need this! Not only would it vastly improve vaccination rates, it will also shed some muchneeded light to the alarming fall in prevailing climate change denier/skepticism!

Reply38. ANONYMOUS

June 30, 2017 / 10:13 amThis is excellent – I plan on posting it for my philosophy students to read. One comment and question: youmention that scientific vocabulary is unfamiliar to non-scientists, and therefore it is important to look upevery unfamiliar word. I agree – in my teaching I very much emphasize the understanding of basicdefinitions of concepts. However, as in science, much of philosophy vocabulary is specialized, meaningthat you will not find the correct definition in regular dictionaries and encyclopedias. For philosophy, Irecommend to my students the Stanford Online Encyclopedia of Philosophy (h�ps://plato.stanford.edu),the Internet Encyclopedia of Philosophy (h�p://www.iep.utm.edu/eds/), and the hard-bound Encyclopediaof Philosophy available in most good research libraries. What science-specific dictionaries andencyclopedias would you recommend?

Reply39. GEOFF HART

July 2, 2017 / 10:16 amExcellent. I’ve added a link to this post in the online errata and additions section of my book (“Writing forScience Journals”; h�p://www.geoff-hart.com/books/journals/journal-book.htm). In addition to being anexcellent primer for nonscientists, it’s a great article for teaching young scientists how to read — and write!— journal manuscripts. When I finally get around to writing the 2nd edition, I plan to integrate several ofyour points into the revised text.

Reply

Page 13: Violen t m et aph or s to read and...“Be sk ept i cal . Bu t w hen you get proof , accept proof .” –Mi chael S pect er What constitutes enough proof? Obviously everyone has a

8/3/2017 How to read and understand a scientific paper: a guide for non-scientists – Violent metaphors

https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/ 13/13

Reply40. IANCHANNING

July 5, 2017 / 8:20 amI’d say that the title isn’t quite correct, it’s not just for non-scientists, it’s for non-academics. There are manypeople who would regard themselves as scientists but don’t read papers.

Reply41. Pingback: FAKE NEWS! Or rather, how to determine what constitutes good research. – OTObiography

Blog at WordPress.com.