what to do about peer review-1

8/22/2019 What to Do About Peer Review-1

1/59

What to do about peerreview

by

Jeremy Horne, Ph.D.The Inventors Assistance League

[email protected]
http://www.inventions.org/mailto:[email protected]:[email protected]://www.inventions.org/


2/59

Presented to:

The 16th World Multi-Conference on

Systemics, Cybernetics and Informatics:WMSCI 2012

17-20 July 2012 Orlando, Florida, USA


3/59

Whom are we talking about?

Editors

Reviewers

Writers Readers

Each has an ethical duty to monitor the integrity ofwhat is being presented.


4/59

Carl Zimmer


5/59

In the fall of 2010, Dr. Ferric C. Fang made an unsettling

discovery. Dr. Fang, who is editor in chief of the journalInfection and Immunity, found that one of his authors haddoctored several papers.

It was a new experience for him. Prior to that time, he said inan interview, Infection and Immunity had only retracted nine

articles over a 40-year period.The journal wound up retracting six of the papers from theauthor, Naoki Mori of the University of the Ryukyus in Japan.And it soon became clear that Infection and Immunity washardly the only victim of Dr. Moris misconduct. Since then,

other scientific journals have retracted two dozen of hispapers, according to the watchdog blog Retraction Watch.http://www.nytimes.com/2012/04/17/science/rise-in-scientific-journal-re
http://www.nytimes.com/2012/04/17/science/rise-in-scientific-journal-retractions-prompts-calls-for-reform.html?_r=1&src=dayphttp://www.nytimes.com/2012/04/17/science/rise-in-scientific-journal-retractions-prompts-calls-for-reform.html?_r=1&src=dayp


6/59

What we think is peer review?

Ethical environment reviewers, reviewed, and audience

Standards of qualityJudgment of qualityQuality reviewersContribution to the body of knowledge


7/59

What peer review often is

Editorial bias

Destructive competition

Fraud, data manipulation, etc.

No uniform standards

Unqualified reviewers


8/59

Richard Horton, editor of the Britishmedical journal The Lancet -

The mistake, of course, is to have thought that peer reviewwas any more than a crude means of discovering theacceptabilitynot the validityof a new finding. Editorsand scientists alike insist on the pivotal importance of peer

review. We portray peer review to the public as a quasi-sacred process that helps to make science our mostobjective truth teller. But we know that the system of peerreview is biased, unjust, unaccountable, incomplete, easilyfixed, often insulting, usually ignorant, occasionally foolish,

and frequently wrong [Horton, Richard (2000). "Geneticallymodified food: consternation, confusion, and crack-up".MJA 172 (4): 1489. PMID 10772580.].


9/59

"Only 8% agreed that 'Peer Review workswell as it is'," according to a survey made tothe members of the Scientific Research

Society. " [Chubin, D. R. and Hackett E. J.,1990, Peerless Science, Peer Review andU.S. Science Policy; New York, State

University of New York Press, p. 192.]


10/59

`the practice of peer review is based

on faith in its effects, rather than onfacts'.

[Jefferson T, Alderson P, Wager E, Davidoff F. Effects ofeditorial peer review: a systematic review.

JAMA2002;287:2784 -6 -http://jrsm.rsmjournals.com/content/99/4/178.full#ref-2 ]
http://jrsm.rsmjournals.com/content/99/4/178.full#ref-2http://jrsm.rsmjournals.com/content/99/4/178.full#ref-2


11/59

Peer review is not only aboutacademics.

Faulty software Poor documentation

Defective products counterbalanced against

the claims Business scandals Bankster bailout, Wall

Street debacle

Politics Sales

The whole legal system

Advertising


12/59

90% of home computers areMicrosoft Windows-based


13/59

Interesting source note toillustrate lack of accountability -

Even today, Microsoft has more than 90percent market share of desktops, comparedwith 6 percent for Apple.

http://news.cnet.com/8301-1035_3-57455261-94/wh

No sources given for this ubiquitous figure.
http://news.cnet.com/8301-1035_3-57455261-94/why-microsoft-built-its-own-tablet-think-apple-and-xbox/http://news.cnet.com/8301-1035_3-57455261-94/why-microsoft-built-its-own-tablet-think-apple-and-xbox/


14/59

Other areas -

GSK [Glaxo-Smith-Klein] targeted the antidepressantPaxil to patients under age 18 when it was approved foradults only, and it pushed the drug Wellbutrin for uses itwas not approved for, including weight loss and

treatment of sexual dysfunction, according to aninvestigation led by the US Department of Justice.

The company went to extreme lengths to promote thedrugs, including distributing a misleading medical journal

article, providing doctors with expensive resort vacations,European hunting trips, high-paid speaking tours andeven tickets to a Madonna concert.

[http://www.aljazeera.com/news/americas/2012/07/201272195753767494.html]
http://www.aljazeera.com/news/americas/2012/07/201272195753767494.htmlhttp://www.aljazeera.com/news/americas/2012/07/201272195753767494.html


15/59

Sources of problems

Dishonesty

Lack of knowledge

Ideology

Wishful thinking

Peer pressure

Sloppy methods Ego


16/59

Sources of dishonesty

Willful altering of truth due to:

Genetics

Ego

Peer pressure

Loss of sense of virtue


17/59

ConsiderHow organisms manage to venture their egos to

meet peer pressure and competition -

It's natural!


18/59

They are not what they appear to be.

In animals, plumage, camouflage, and a cat ruffling itsfur are forms of puffery, or, to us a form of the lie saying something to the outside world when the truth issomething different. We seem to be stuck with this basicconstitution.


19/59

The Emperor's New Clothes

[Illustration by , Andersen's first illustrator]


20/59

We ask, who are the wizards, orexperts?

What is our knowledge base?


21/59

The Black Lie 2007-2012 by Zindy -http://zindy.deviantart.com/art/The-Black-Lie-58075950

The White Lie
http://zindy.deviantart.com/art/The-Black-Lie-58075950http://zindy.deviantart.com/art/The-Black-Lie-58075950


22/59

Who are we?

Society of lies lies to ourselves, our spouses,the products we sell the whole ecosystem is

based on lies advertising


23/59

EGO


24/59

Lack and questionable integrityof knowledge

This extends all across the spectrum fromgeneral public to the scientists.


25/59

June 1, 2012

In U.S., 46% Hold Creationist View of Human

Origins

by Frank Newport

PRINCETON, NJ -- Forty-six percent of Americans believein the creationist view that God created humans in theirpresent form at one time within the last 10,000 years. Theprevalence of this creationist view of the origin ofhumans is essentially unchanged from 30 years ago,

when Gallup first asked the question.

http://www.gallup.com/poll/155003/hold-creationist-view-human-origins.aspx
http://www.gallup.com/poll/155003/hold-creationist-view-human-origins.aspxhttp://www.gallup.com/poll/155003/hold-creationist-view-human-origins.aspx


26/59


27/59

In Academically Adrift, sociologists Richard Arum andJosipa Roksa report that after two years of college,45% of American students havent learned anything,and after four years, 36% havent. Most students, theydiscovered, define college as a social, not an academicor intellectual, experience; half the students in their

study said they hadnt taken a single course in theprevious semester that required more than 20 pages ofwriting, and a third said they hadnt taken a courserequiring more than 40 pages of reading. A Marist pollreleased on 4 July 2011 (appropriately enough) showedthat 69% of Americans in the under-30 age group areunaware that the U.S. declared its independence in1776.http://morrisberman.blogspot.mx/

We come to the borderlands of
http://morrisberman.blogspot.mx/http://morrisberman.blogspot.mx/


28/59

We come to the borderlands ofscience when good science

goes badNorm Shealy

Richard C. HoaglandStanley McDanielWilliam Shockley

Linus PaulingRhine Institute

Journal of Frontier ScienceJournal of Scientific Discovery


29/59

Then, there are

Bernard Haisch - Solar & Astrophysics Laboratory at LockheedMartin in Palo Alto, California, deputy director - Center forExtreme Ultraviolet Astrophysics Laboratory at the University ofCalifornia, Berkeley, visiting scientist - Max Planck Institute forExtraterrestrial Physics in Garching, Germany and at theUniversity of Utrecht in the Netherlands. More than 100 papers -

Nature, Science, Physical Review, Astrophysical Journal, andAnnalen der Physik. Website - UFO Skeptic, which promotes theinvestigation of the UFO phenomenon by professionalscientists.

Brian Josephson 1973 Nobel Laureate Josephson Junction,

parapsychological phenomena may be real, Eastern mysticismmay have relevance to scientific understanding Criticism byseveral fellow physicists including David Deutsch, a quantumphysicist at Oxford University who stated: "It [telepathyexplained by quantum processes] is utter rubbish. -

http://en.wikipedia.org/wiki/Brian_Josephson
http://en.wikipedia.org/wiki/Brian_Josephsonhttp://en.wikipedia.org/wiki/Brian_Josephson


30/59

Stanley McDaniel

Instructor in Philosophy, Brooklyn College, CUNY, 1964-66 Teacher of Philosophy at Sonoma State University, 1966-

1992:(Pragmatism, Kant, Logic, Eastern Thought, Philosophy ofPsychology,Aesthetics of Music, Philosophy of Esoteric Systems, etc.)

Chair, Department of Philosophy, Sonoma State University

Member, American Philosophical Association. Member, Society for the Advancement of American

Philosophy. Member, American Association of Philosophy Teachers. Vice-President of the Board of Directors,

The Foundation for Critical Thinking;Rohnert Park,California.


31/59

The McDaniel ReportInstead of carrying out legitimate scientific enquiry , NASA has regularly sent false andmisleading statements regarding the landforms to members of congress and theirconstituents. [http://www.bibliotecapleyades.net/marte/esp_marte_41.htm]

8 April 2001

1976 Viking 1photographof the Face on Mars

Pareidolia (play /pr do li/ parr-i-DOH-lee-) is a psychological phenomenon involving a vague and random stimulus (often an image or sound) being perceived as significant.Common examples include seeing images of animals or faces in clouds, the man in themoon or the Moon rabbit, and hearing hidden messages on records played in reverse.

http://en.wikipedia.org/wiki/Pareidolia
http://www.bibliotecapleyades.net/marte/esp_marte_41.htmhttp://www.bibliotecapleyades.net/marte/esp_marte_41.htmhttp://en.wikipedia.org/wiki/Pareidoliahttp://en.wikipedia.org/wiki/Pareidoliahttp://www.bibliotecapleyades.net/marte/esp_marte_41.htm


32/59

Norman Shealy, M.D., Ph.D


33/59

Gigatens


34/59

Gigatens circuit diagram


35/59

Signs of crossing the boundary Faulty methods

Will to believe

Abandonment of dispassionate search for truth

Straying into area outside of expertise

Not qualifying conclusions Psychotic break

Commercializing your findings

Release of results first through popular press Generalizing from a few legitimate examples

Religiosity and ideology control research direction


36/59

If this weren't bad enough


37/59

[ h t t p : / / w w w . c h r i s t o p h e r m c g i l l . c o m / 2 0 1 0 / 0 5 / 2 4 / t h e - t o p - n i n e - c v - l i e s - t o - l o o k - o u t - f o r - s a n -a n t o n i o / ]


38/59

A small sample

The dean of admissions at MIT was forced to resign in 2007 forfalsifying herresume.[ ]The vice dean of the Graduate School of Education at the

University of Pennsylvania resigned Thursday, one day after hewas placed on administrative leave pending an investigationinto his false claim to have a doctoral degree.

[http://www.academicjobstoday.com/news-1658-Penn_education_graduate_school_vice_dean_Doug_Lynch_resigns_over_questions_about_his_doctoral_credentials.html 27April 2012 ]

On Monday morning, President Pamela Gann of ClaremontMcKenna College announced via email to members of the CMCcommunity that a senior administrator at the Office ofAdmission had been reporting inaccurate SAT statistics since2005. - January 30, 2012 at 12:54 pm[http://cmcforum.com/news/01302012-cmc-office-of-admission-falsely-reported-sat-scores ]


39/59

Plagiarism

data are sorely lacking on the true extent of plagiarism, whether its prevalence isgrowing substantially and what differences might exist between disciplines. [Plagiarismpinioned Nature 466,159160 (08 July 2010) doi:10.1038/466159b Published online 07July 2010 - ]

The University of Virginia recently stripped Fred D. Smith of his doctorate in education.Last year, iThenticate was utilized by a reporter from to break a story on Smith, the ex-

Pocantico Hills superintendent who plagiarized much of his UVA dissertation.[http://www.ithenticate.com/plagiarism-checker-blog/bid/52964/Dissertation-Plagiarism ]

Boston University, where King received his in , conducted an investigation that found heplagiarized major portions of his doctoral thesis from various other authors who wroteabout the topic. Boston University decided not to revoke his doctorate, saying thatalthough King acted improperly, his dissertation still "makes an intelligent contribution to

scholarship." However, a letter is now attached to King's dissertation in the universitylibrary, noting that numerous passages were included without the appropriate quotationsand citations of sources.

[http://en.wikipedia.org/wiki/Martin_Luther_King,_Jr._authorship_issues ]


40/59

...more

Study by Don McCade, a University of Rutgers

Professornearly 40% of college students have plagiarizedpapers by the cut-and-paste function on their computers to lifttext off the internet (Heyboer). At the University of Toronto,more than 200 cases of plagiarism are reported each year, andthe university's administration notes that this is probably onlya fraction of those who actually commit academic fraud . Anarticle in London's Daily Mailclaims that in the UnitedKingdom as many as 30 to 35 percent of student assignmentsare cribbed at least in part, from essay banks and fellowstudents' work (Grant). An August 2002 report by LarissaDubecki observes that of 1000 university students surveyed

from four Australian universities, 41% admitted cheating ontests, while 86% of males and 77% of females admitted tocommitting some form of plagiarism ( ). -


41/59

Inflating test scores


42/59


43/59


44/59

E x c e r p t s -

T h e y a r e a m o n g a t l e a s t 2 2 0 C a n a d i a n s w i t h b o g u s a c a d e m i c c r e d e n t i a l s

u n c o v e r e d i n a r e c e n t p r o b e . W o r l d w i d e , f a k e d e g r e e s a r e a b i l l i o n d o l l a r

i n d u s t r y , e v e n t h r e a t e n i n g g o v e r n m e n t s e c u r i t y , i n v e s t i g a t o r s s a y .

L a s t w e e k , a n u n d e r c o v e r S t a r i n v e s t i g a t i o n e x p o s e d P e n g S u n , a Y o r k U n i v e r s i t y

g r a d w h o f o r g e s u n i v e r s i t y d e g r e e s f r o m r e a l C a n a d i a n u n i v e r s i t i e s f o r $ 4 , 0 0 0 .

S u n ' s c l i e n t l i s t w a s n o t a v a i l a b l e , b u t t h e S t a r o b t a i n e d a l i s t o f C a n a d i a n s w h o

b o u g h t f a k e d e g r e e s f r o m a n A m e r i c a n d i p l o m a m i l l b u s t e d t h r e e y e a r s a g o b y t h e

U . S . S e c r e t S e r v i c e a n d H o m e l a n d S e c u r i t y .

T h e g a n g r a k e d i n m o r e t h a n $ 7 m i l l i o n i n s a l e s t o 1 3 1 c o u n t r i e s . I t s o l d

e v e r y t h i n g f r o m h i g h s c h o o l d i p l o m a s t o P h D s a n d m e d i c a l d e g r e e s . D o z e n s o f

U . S . g o v e r n m e n t e m p l o y e e s a r e o n t h e l i s t , i n c l u d i n g a W h i t e H o u s e s t a f f m e m b e r , N a t i o n a l S e c u r i t y A g e n c y e m p l o y e e s , a s e n i o r S t a t e D e p a r t m e n t o f f i c i a l ,

a n d a D e p a r t m e n t o f J u s t i c e e m p l o y e e .

T e n s o f t h o u s a n d s o f p e o p l e a r e w a l k i n g a r o u n d w i t h " t i c k i n g t i m e b o m b s i n t h e i r

r e s u m e s , " s a y s A l l e n E z e l l , a f o r m e r F B I a g e n t w h o h a s s p e n t a b i g c h u n k o f h i s

c a r e e r i n v e s t i g a t i n g d i p l o m a m i l l s .

A t h i r d - y e a r O s g o o d e H a l l L a w S c h o o l s t u d e n t , F r e d e r i c k , 2 8 , i s o n t h e l i s t a sh a v i n g p a i d $ 1 , 1 0 9 f o r a " B . A . " i n B u s i n e s s A d m i n i s t r a t i o n , p l u s a t r a n s c r i p t o f

m a r k s . U s i n g t h e d e g r e e t r a n s c r i p t , F r e d e r i c k g o t i n t o O s g o o d e a s o n e o f 2 9 0

s t u d e n t s s e l e c t e d f r o m 2 , 5 0 0 a p p l i c a n t s i n 2 0 0 6 . [ Q u e s t i o n W h e r e w a s t h e

s c r e e n i n g , h e r e ? ]

[ h t t p : / / w w w . t h e s t a r . c o m / n e w s / i n v e s t i g a t i o n s / p r i v a t e c a r e e r c o l l e g e s / a r t i c l e / 5 5 3 3 3 0

- - p h o n y - d e g r e e s - c a t c h - u p - t o - b u y e r s ]


45/59

The dean of admissions at MIT was forced to resign in 2007 for falsifyingherresume.[ ]The vice dean of the Graduate School of Education at the University ofPennsylvania resigned Thursday, one day after he was placed onadministrative leave pending an investigation into his false claim tohave a doctoral degree. [http://www.academicjobstoday.com/news-1658-Penn_education_graduate_school_vice_dean_Doug_Lynch_resigns_over_questions_about_his_doctoral_credentials.html 27 April 2012 ]On Monday morning, President Pamela Gann of Claremont McKennaCollege announced via email to members of the CMC community that asenior administrator at the Office of Admission had been reportinginaccurate SAT statistics since 2005. - January 30, 2012 at 12:54 pm [

http://cmcforum.com/news/01302012-cmc-office-of-admission-falsely-repor]
http://cmcforum.com/news/01302012-cmc-office-of-admission-falsely-reported-sat-scoreshttp://cmcforum.com/news/01302012-cmc-office-of-admission-falsely-reported-sat-scores


46/59

Spam sent out

Buy Custom Made University Diplomas, FakeCollege Degrees and Transcripts of World

Famous Universities. recommends this siteYour friend Buy Custom Made University Diplomas, Fake College Degrees and Transcriptsof World Famous Universities. (.) has recommended this site to you, and sends you thefollowing message:*******************************************************************************Buy fake/novelty university diplomas and degrees online. We design our Degree or DiplomaCertificates and Transcripts to look 99.99% identical to world famous originals includingidentical security grade transcript paper, identical water markings, raised-ink crests,

embossed seals, correct card stock weight (60-100lb depending upon institutionspecifications). Most post-secondary institutions available. ---> http://realisticdiploma.cu.cc


47/59

Better controls over degrees andtranscripts

Biometric identification one calling for itcould require one submitting credential tovalidate it by fingerprint, etc.

Hologram and/or manufacture in samemanner as paper money

Verify issuance with university by internetand in person

Transcript serial number printed on degree


48/59

Question of information integrity

Problem: Source of most peoples information -Google, blogs, opinions

Solution: Require epistemology in all curriculums

integrated with subjects in all grades summarycourse in both high school and college on infrequency as the teaching of country's language(in the case of the U.S., English)

Require critical thinking, philosophy, and logic inboth high school and college, as well as corecurriculum

Wh ?


49/59

Where are we?

Alaric only sacked Rome for its gold the society was

already gutted.

A hi


50/59

Are we approaching a newMedievalism?

Bhaskaranaidu -http://commons.wikimedia.org/wiki/File:Clear_picture_of_moat_at_rayavellore_fort.JPG
http://commons.wikimedia.org/wiki/File:Clear_picture_of_moat_at_rayavellore_fort.JPGhttp://commons.wikimedia.org/wiki/File:Clear_picture_of_moat_at_rayavellore_fort.JPG


51/59

We need to avoid this!

Wait too fast we need to resolve one issue -Are we willing to internalize a life and knowledgesustaining value system?

What are our values?Once we have affirmed them, THEN we canproceed.


52/59

Before anything - 'Know Thyself'


53/59

Time for instrumentation


54/59

Basic standards

Who is the author? What is her/his background what qualifications does s/he have towrite about the subject? With what organizations or individuals is/are the author(s) associated

that might suggest a bias? Under what circumstances is the author writing that might contribute

to a bias? How current is the information? In what context is the information framed? What method was to obtain the information? In what location was the information prepared or formulated?

How plausible is the information? How skeptical should you be? How tight is the argumentation? How well is the information documented?


55/59

Beliefand knowledge

Epistemology accounting for HOW do weknow

Tradition

History Logic

Empiricism

Science

Faith

P t l i


56/59

Programme to solve peer reviewproblem must include

Need WILL to do something

Plan and means to carry it out

Action actual participation

Knowledge of standards

Ethics

Screening of reviewers (education, area ofcompetence, reviewing history, etc.)

Kno ledge Generation and


57/59

Knowledge Generation andCommunication Management

Knowledge from information from data

Communication how to convey it interdisciplinary means


58/59

What can KGCM do?

Act is arbitrator

Help establish common standards

Act as pre-reviewer for writers, and

reviewers Be a forum for writers, editors, readers, and

peer reviewers

Link together those in charge of peerreviewing venues editors, conferencechairs, etc.


59/59

What can YOU do?JOIN US in the ongoing workshop at this

conference and work with us in the future inadvancing our programme. The room forimproving the process is endless.

what to do about peer review-1

Documents