Research for NHSClinicians

What is Possible?

Alex Mitchell, Consultant in Liaison Psychiatry

ContentsContents

• Part I: Getting Published

• Part II: Being Measured

• Part III: Real World Outputs from Liaison Psychiatry

Part I: Getting PublishedPart I: Getting Published

Why Publish?

Three Bottlenecks

Successes

Your Research

The fate of medical research (DeBakey, 1974)

NHS Power -2004 StatsNHS Power -2004 StatsAcademics• 494 clinical lecturers (down 17% on 2003)• 1500 Readers / SnR Lecturers (down 11% on 2000)• 1146 Professors (up 10% on 2000)

• 3113 FTE clinical academics in all specialties

• 302 SnR academics in Psychiatry • 40 lecturers in Psychiatry

NHS Power -2004 StatsNHS Power -2004 StatsAcademics• 494 clinical lecturers (down 17% on 2003)• 1500 Readers / SnR Lecturers (down 11% on 2000)• 1146 Professors (up 10% on 2000)• 302 SnR academics in Psychiatry • 40 lecturers in Psychiatry • 3113 FTE clinical academics in all specialtiesNHS Staff• 117,000 NHS doctors (up 5% on 2000)• 34,000 GPs (up 2%)• 30,600 Consultants (up 5%)• 397,000 Nurses (up 2.5%)• 129,000 Allied health professionals (up 4%)• 1,071,462 FTE qualified staff• 3,500 NHS Consultant Psychiatrists vs 350 academic psychiatrists

38,411

30,650

2,294

5,467

41,697

16,823

20,601

4,273

6,888

1,048

3,4762,364

34,085

0

5000

10000

15000

20000

25000

30000

35000

40000

45000Care

er grad

es C

onsult

ant

Ass

ociate

Specia

list

Staf

f grad

eDocto

rs in tra

ining

Reg

istrar

group 2

Senio

r House

Offic

er

Hous

e Offic

er an

d FY1Other

grades

Hosp

ital P

ractiti

oner

Clin

ical A

ssist

ant O

ther Staf

f

GPs

Source: DOH Workforce Stats

618

4,355

392670

6,726

1,413

1,902

2,411

926

3,555

1,928

5,754

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

Acciden

t & em

ergenc

y

Anaesth

etics

Clinica

l onco

logyDen

tal Gro

up

General

med

icine g

roup 2

Obstet

rics &

gynae

cology

Paedia

tric gr

oup

Patholog

y gro

up

Public H

ealth

Psych

iatry gr

oup

Radiolog

y gro

upSurg

ical g

roup

Source: DOH Workforce Stats

Consultants by Specialty

ProductivityProductivity

• Academics– 3000 x 5 papers per year– N= 15,000

• NHS Medical Staff– 120,000 x 1 paper every 5 years!– N= 24,000

• NHS AHP– 500,000 x 1 paper every 100 years– N= 5,000

• Ratio of Outputs = 15,000 to 29,000 (1:2)

ProductivityProductivity

• Academics– 3000 x 5 papers per year– N= 15,000

• NHS Medical Staff– 120,000 x 1 paper every 5 years!– N= 24,000

• NHS AHP– 500,000 x 1 paper every 100 years– N= 5,000

• Ratio of Outputs = 15,000 to 29,000 (1:2)

Bottleneck 1: JournalsBottleneck 1: Journals

=> BMJ

Typical Journal Statistics (eg BMJ)Typical Journal Statistics (eg BMJ)• 7000 research papers received• 1000 rejected by one editor immediately

– further 4500 rejected by editors after reading

– Only 25% (1500) sent to reviewers

– Only 10% reach weekly manuscript meeting • (with Editor, a clinicians and a statistician)

– 350 research articles after revisions (and up to 6 peer reviews)

• Only 3% (1990-2000) to 6% (2000+) accepted by BMJ=> time!

Conventional Publication Cycle – 24moConventional Publication Cycle – 24mo

Step1

Paper Completed

Step 2

Present to colleagues (informal PR)

Step 3

Present at conference

Step 4

Submit to journal

Step 6

Peer

Review 1

Step 7

Revision 1

Step 8

Peer Review 2

Step 10

Proof

T0 T+1mo

Step 12

Readers letters (public PR)

T+3mo T+6mo

Step 11

Restricted Publication

Step 9

Revision 2

Step 5

Editor Agrees to Send for Review

T+7mo T+10mo T+12mo T+14mo

T+15mo T+18mo T+24mo

First received 26 Sep 2007, final revision 30 Sep 2008, accepted 20 Nov 2008; 540 days

Discussion Forum (Pub. Stage 1) + Journal (Pub. Stage 2)

Open Access Publishing

Open Publication Cycle – 12moOpen Publication Cycle – 12mo

Step1

Paper Completed

Step 2

Present to colleagues (informal PR)

Step 3

Present at conference

Step 4

Submit to online journal

Step 7 Peer Review 1

Step 6 Public comments

Step 8

Revision

T0 T+1mo

Step 11

Readers letters (post-publication)

T+3mo T+5mo

Step 10

Open Access Publication

Step 9

Online proof

Step 5

Editor Publishes “discussion paper”

T+6mo T+7mo T+9mo

T+11mo T+12mo

=> Good news2

How Many?How Many?• Hospital Specialist

• 75 journals in psychiatry • + 50 in neurology• + 50 in medicine• + 25 in misc. fields

– 2400 issues per year– 24,000 specialty articles per year

– 6.5 journals per day

– 65 articles per day

The Publishing LandscapeThe Publishing Landscape

10,000s+Not generally peer-reviewed but highly targetted readership

PulseMagazine titles

Direct ReadershipScopeExample

100s+Not peer-reviewed, not widely read, never cites

NewsletterInternal titles

1000sPrimary care focus, can be low scientific impact if jnl is new

PCPPrimary care titles

Sub-specialists on disease1000s

Specific disease area/therapy class, low impact

J of ECTSub-specialty titles

Specialists (mainly secondary care)10,000s-00,000s

Covering specialty as whole, high impact (IF3+)

BJPsychSpecialty titles

Generalist1000,000s?

Generalist, very high impact (IF5+), closely peer reviewed

The LancetGeneralist titles

Some Obscure Psychiatry TitlesSome Obscure Psychiatry Titles

STRESS AND HEALTH0.721

INT JJOURNAL PSYCHIATRY IN MEDICINE0.733

ARCH PSYCHIAT NURS0.734

NORDIC J PSYCHIAT0.752

J PSYCHOSOMATIC OBST & GYN0.951

BEHAVIORAL MEDICINE1

NEUROPSYCHIATRIE1.051

PERSPECT PSYCHIATR CARE1.062

JOURNAL OF ECT1.176

PSYCHIATRY CLINICAL NEUROSCIENCE1.181

ACTA NEUROPSYCHIATRY1.206

SUBST USE MISUSE1.229

AGING MENT HEALTH1.264

PSYCHOPATHOLOGY1.441

NEUROCASE1.505

WORLD J BIOL PSYCHIATRY1.691

Bottleneck 2: Your ColleaguesBottleneck 2: Your Colleagues

Overview of Peer ReviewOverview of Peer Review

Grading Accept

Revise

Reject

Peer Review Filter

Published Article

Send elsewhere

Reject

Qualitative Grade

Editorial Filter

Quantitative Grade

Outcome

Comments to Author

Author Filter

Article Appraisal

Where to Submit

Bias from Hidden ReviewersBias from Hidden Reviewers

AuthorAuthor

Friend

Unknown

Unknown

Enemy

Revision

Rejection

Enemy

+

+

--

-

Acceptance

See Maddox J. Conflicts of interest declared [news]. Nature 1992; 360: 205; Locke S. Fraud in medicine [editorial]. Br Med J 1988; 296: 376–7.

What Makes a Good Reviewer?What Makes a Good Reviewer?• A reviewer was less than 40 years old

• From a top academic institution

• Well known to the editor choosing the reviewer

• Author blinded to the identity of the manuscript's authors

• Then the probability of good review was 87% vs 7%

Evans AT, McNutt RA, Fletcher SW, Fletcher RH. The characteristics of peer reviewers who produce good-quality reviews. J Gen Intern Med. 1993 Aug;8(8):422-8

Open & Blind Review & SubmissionOpen & Blind Review & Submission

• Does revealing reviewers’ identity influence outcome? – Open vs blind peer review

• Does revealing Authors’ identity influence outcome?– Open vs blind submission

• http://www.slideshare.net/ajmitchell

Bottleneck 3: YourselfBottleneck 3: Yourself

Problems with AuthorsProblems with Authors• Multiple publication

• Ghost authorship

• Gift authorship

• Conflicts of Interest

• Fraud / Plagarism

Ghost AuthorshipGhost Authorship• An Inquiry by the UK House of Commons Health Select

Committee into the Influence of the Pharmaceutical Industry October 2004– 50% of therapeutics articles may be ghost written

• Flanagin A et al (1998) Prevalence of Articles With Honorary Authors and Ghost Authors in Peer-Reviewed Medical Journals JAMA 280:222-224

• Of 809 articles– 19% had evidence of honorary (gift) authors– 11% had evidence of ghost authors– 2% had evidence of both

How Many Authors?How Many Authors?• Deep Impact: Observations from a Worldwide Earth-Based CampaignK. J. Meech, N. Ageorges, M. F. A'Hearn, C. Arpigny, A. Ates, J. Aycock, S. Bagnulo, J. Bailey, R. Barber,

L. Barrera, R. Barrena, J. M. Bauer, M. J. S. Belton, F. Bensch, B. Bhattacharya, N. Biver, G. Blake, D. Bockelée-Morvan, H. Boehnhardt, B. P. Bonev, T. Bonev, M. W. Buie, M. G. Burton, H. M. Butner, R. Cabanac, R. Campbell, H. Campins, M. T. Capria, T. Carroll, F. Chaffee, S. B. Charnley, R. Cleis, A. Coates, A. Cochran, P. Colom, A. Conrad, I. M. Coulson, J. Crovisier, J. deBuizer, R. Dekany, J. de Léon, N. Dello Russo, A. Delsanti, M. DiSanti, J. Drummond, L. Dundon, P. B. Etzel, T. L. Farnham, P. Feldman, Y. R. Fernández, M. D. Filipovic, S. Fisher, A. Fitzsimmons, D. Fong, R. Fugate, H. Fujiwara, T. Fujiyoshi, R. Furusho, T. Fuse, E. Gibb, O. Groussin, S. Gulkis, M. Gurwell, E. Hadamcik, O. Hainaut, D. Harker, D. Harrington, M. Harwit, S. Hasegawa, C. W. Hergenrother, P. Hirst, K. Hodapp, M. Honda, E. S. Howell, D. Hutsemékers, D. Iono, W.-H. Ip, W. Jackson, E. Jehin, Z. J. Jiang, G. H. Jones, P. A. Jones, T. Kadono, U. W. Kamath, H. U. Käufl, T. Kasuga, H. Kawakita, M. S. Kelley, F. Kerber, M. Kidger, D. Kinoshita, M. Knight, L. Lara, S. M. Larson, S. Lederer, C.-F. Lee, A. C. Levasseur-Regourd, J. Y. Li, Q.-S. Li, J. Licandro, Z.-Y. Lin, C. M. Lisse, G. LoCurto, A. J. Lovell, S. C. Lowry, J. Lyke, D. Lynch, J. Ma, K. Magee-Sauer, G. Maheswar, J. Manfroid, O. Marco, P. Martin, G. Melnick, S. Miller, T. Miyata, G. H. Moriarty-Schieven, N. Moskovitz, B. E. A. Mueller, M. J. Mumma, S. Muneer, D. A. Neufeld, T. Ootsubo, D. Osip, S. K. Pandea, E. Pantin, R. Paterno-Mahler, B. Patten, B. E. Penprase, A. Peck, G. Petitas, N. Pinilla-Alonso, J. Pittichova, E. Pompei, T. P. Prabhu, C. Qi, R. Rao, H. Rauer, H. Reitsema, S. D. Rodgers, P. Rodriguez, R. Ruane, G. Ruch, W. Rujopakarn, D. K. Sahu, S. Sako, I. Sakon, N. Samarasinha, J. M. Sarkissian, I. Saviane, M. Schirmer, P. Schultz, R. Schulz, P. Seitzer, T. Sekiguchi, F. Selman, M. Serra-Ricart, R. Sharp, R. L. Snell, C. Snodgrass, T. Stallard, G. Stecklein, C. Sterken, J. A. Stüwe, S. Sugita, M. Sumner, N. Suntzeff, R. Swaters, S. Takakuwa, N. Takato, J. Thomas-Osip, E. Thompson, A. T. Tokunaga, G. P. Tozzi, H. Tran, M. Troy, C. Trujillo, J. Van Cleve, R. Vasundhara, R. Vazquez, F. Vilas, G. Villanueva, K. von Braun, P. Vora, R. J. Wainscoat, K. Walsh, J. Watanabe, H. A. Weaver, W. Weaver, M. Weiler, P. R. Weissman, W. F. Welsh, D. Wilner, S. Wolk, M. Womack, D. Wooden, L. M. Woodney, C. Woodward, Z.-Y. Wu, J.-H. Wu, T. Yamashita, B. Yang, Y.-B. Yang, S. Yokogawa, A. C. Zook, A. Zauderer, X. Zhao, X. Zhou, and J.-M. ZucconiScience 14 October 2005: 265-269.

Part II: Being MeasuredPart II: Being Measured

Impact Factors

Journals vs papers vs

individuals

H-Index

Dissemination

How to Grade Research…of an AuthorHow to Grade Research…of an Author

10% top 10 journals

70% peer reviewed journals

20% other material (eg chapters)

100% Listed on Pubmed

75% Listed on Pubmed90% Listed on Scopus/WoK

5% Listed on Pubmed15% Listed on Scopus/WoK

How to Grade Research…of an AuthorHow to Grade Research…of an Author

10% top 10 journals

70% peer reviewed journals

20% other material

Impact(citations)

PubMed Hits for March 2005 (millions)PubMed Hits for March 2005 (millions)

The Dreaded “Impact Factor”The Dreaded “Impact Factor”

Q. How do you know what published research is important?

Q. How do you decide what research to read?

2400 experts537 citations

What is the Impact factor?What is the Impact factor?

• Journal IF• A = citations in the last 2 years in journal• B = number of articles Impact Factor = A/B

• Article IF• A = Total citations of article• B = Age of article Impact Factor = A/B

• Personal IF• A = citations in the last 2 years by individual• B = number of articles Impact Factor = A/B

What is the Impact factor?What is the Impact factor?

• Journal IF• A = citations in the last 2 years in journal• B = number of articles Impact Factor = A/B

• Article IF• A = Total citations of article• B = Age of article Impact Factor = A/B

• Personal IF• A = citations in the last 2 years by individual• B = number of articles Impact Factor = A/B

Eugene GarfieldEugene Garfield

• Citation Databases– Science citation index (CD)– BIDS– ISI Science Direct– Web of Science– Web of Knowledge– Scopus

Medical Journals Impact FactorsMedical Journals Impact Factors

Credit: Panayiota Polydoratou and Martin Moyle

0

0.5

1

1.5

2

2.5

3

1988 1990 1993 1994 1995 1996 1997 1998 1999 2000 2001

CM

AJ'

s im

pact

fact

or

Hoey J, Todkill AM. CMAJ 2003;168:287-8 [online appendix]

One Journal’s Impact Factor Trend

http://www.americanscientist.org/my_amsci/restricted.aspx?act=pdf&id=3263000957901

Increase in Scientific Output

Type of Article and ImpactType of Article and Impact

Good Paper

Importance of ReviewsImportance of Reviews• WHITTINGTON, C. J., KENDALL,T., FONAGY, P., COTTRELL, D.,

COTGROVE, A. & BODDINGTON, E. (2004) Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet, 363, 1341-1345.

• Lancet Paper of the Year 2004

• BACKGROUND: Questions concerning the safety of selective serotonin reuptake inhibitors (SSRIs) in the treatment of depression in children led us to compare and contrast published and unpublished data on the risks and benefits of these drugs. METHODS: We did a meta-analysis of data from randomised controlled trials that evaluated an SSRI versus placebo in participants aged 5-18 years and that were published in a peer-reviewed journal or were unpublished and included in a review by the Committee on Safety of Medicines. The following outcomes were included: remission, response to treatment, depressive symptom scores, serious adverse events, suicide-related behaviours, and discontinuation of treatment because of adverse events. FINDINGS: Data for two published trials suggest that fluoxetinehas a favourable risk-benefit profile, and unpublished data lend support to this finding. Published results from one trial of paroxetine and two trials of sertraline suggest equivocal or weak positive risk-benefit profiles. However, in both cases, addition of unpublished data indicates that risks outweigh benefits. Data from unpublished trials of citalopram and venlafaxine show unfavourable risk-benefit profiles. INTERPRETATION: Published data suggest a favourable risk-benefit profile for some SSRIs; however, addition of unpublished data indicates that risks could outweigh benefits of these drugs (except fluoxetine) to treat depression in children and young people. Clinical guideline development and clinical decisions about treatment are largely dependent on an evidence base published in peer-reviewed journals. Non-publication of trials, for whatever reason, or the omission of important data from published trials, can lead to erroneous recommendations for treatment. Greater openness and transparency with respect to all intervention studies is needed.

IF Psychiatry JournalsIF Psychiatry Journals

Part III: Real World Research OutputsPart III: Real World Research Outputs

Journal outputs

Top 10 Psychiatric papers

Top 10 Authors

Journal OutputsJournal Outputs

Q. What is the effect of a high IF on individual authors?

IF and Submission RatioIF and Submission Ratio

• The IF can be considered inversely proportional to the chances of acceptance

BMJ Manuscript Processing

5000 scanned by 1 editor

2500 sentfor review

1000 Immediately

rejected

6000 received per year

Poorly writtenObvious flawsToo Obscure

Hand writtenSilly mistakesWrong journal

100 accepted

2500Rejected by

editor

1000 Rejected by reviewer /

editor

Methodological concerns

Not interesting

400 rejected

1500 sent to hanging

committees

500 discussed

100 accepted400 rejected500 discussed

100 accepted400 rejected500 discussed

300 accepted per year(6 per week + 2 short

reports)

Acceptance Rate = 1 in 20

1 in 10+

1 in 3+

Liaison JournalsLiaison Journals

>10.0300.733823INT J PSYCHIAT MED0.733

9.2131479BEHAV MED1

3.4451.206201ACTA NEUROPSYCHIATR1.206

7.91571.8596203J PSYCHOSOM RES1.859

7.8552.0532396J NEUROPSYCH CLIN N2.053

3.31782.0951937EPILEPSY BEHAV2.095

7832.1171952GEN HOSP PSYCHIAT2.117

7.5802.1992259PSYCHOSOMATICS2.199

Cited Half LifeArticlesImpact FactorTotal CitesAbbreviated Journal TitleImpact Factor

Individual PapersIndividual Papers

Q. What are the “most important” papers in psychiatry?

Q. How many times do you think the top papers are cited?

Top 10 Cited Papers in PsychiatryTop 10 Cited Papers in Psychiatry

Top 10 Cited Papers in Psychiatry_YearTop 10 Cited Papers in Psychiatry_Year

Top Non-Scale/Tool Papers?Top Non-Scale/Tool Papers?

Personal “Impact Factor”Personal “Impact Factor”

Q. Who are the most influential authors in psychiatry?

Q Can influence by measured?

GoldGeneral PsychiatryASZIGMOND

GoldOld Age PsychiatryKMCWILSON

GoldGeneral PsychiatryJLTREASURE

GoldGeneral PsychiatryGSTRATHDEE

GoldGeneral PsychiatryTESENSKY

GoldGeneral PsychiatrySPRIEBE

GoldGeneral PsychiatrySO'CONNOR

GoldGeneral PsychiatryASLEE

GoldGeneral PsychiatryPBJONES

GoldGeneral PsychiatrySHUNTER

GoldC & A PsychiatryJWHILL

GoldForensic PsychiatryDHGRUBIN

GoldC & A PsychiatryRGOODMAN

GoldGeneral PsychiatryEAGILVARRY

GoldGeneral PsychiatryASDAVID

GoldC & A PsychiatryPFBOLTON

GoldGeneral PsychiatryDBHUGRA

GoldGeneral PsychiatryLJAPPLEBY

PlatinumC & A PsychiatryRJWWILLIAMS

PlatinumGeneral PsychiatryGJTHORNICROFT

PlatinumGeneral PsychiatryMJOWEN

PlatinumOld Age PsychiatryIGMCKEITH

PlatinumGeneral PsychiatryJHLACEY

PlatinumGeneral PsychiatryGMGOODWIN

PlatinumGeneral PsychiatryCJAFAIRBURN

PlatinumC & A PsychiatrySMBAILEY

LevelSPECIALTYINITIALSSURNAME

Measuring Individuals ResearchMeasuring Individuals ResearchOld Methods (counts)• Subjective top 4 publications (RAE)• Subjective top 10 publications (CEA form D)• Total publications count• Total Peer reviewed publications

New Methods (IFs)• Total citations• Citations per publication• Citations per year (last 2 years)• No. publications with same number of citations (H-index)• Citations per year since qualification

Output MeasuredOutput Measured

12h-index

4.17Average Citations per Item

325Sum of the Times Cited

78Results found:

“Exceptional” Output“Exceptional” Output

Exceptional Citation GrowthExceptional Citation Growth

Publication Count in Liaison (Anonymous)Publication Count in Liaison (Anonymous)

464

292

243

203183 183

134114

87 78 77 69 67 6142 40

26 21 18 17 10

0

50

100

150

200

250

300

350

400

450

500

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icNHS

Academ

icAca

demic

Academ

icNHS

Academ

icAca

demic

Academ

icNHSNHSNHSNHS

Citation Count by Liaison (anonymous)Citation Count by Liaison (anonymous)

0

2,000

4,000

6,000

8,000

10,000

12,000

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

NHSNHS

Academ

icNHSNHSNHSNHS

2 Yr Citation Count by Liaison (anonymous)2 Yr Citation Count by Liaison (anonymous)

2040

1600

1020960

830 820

610 580520

390

250195 180 175 160

85 85 30 12 5 30

500

1000

1500

2000

2500

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icNHS

Academ

icNHS

Academ

icNHSNHSNHSNHS

H-Index by Liaison Authors (anonymous)H-Index by Liaison Authors (anonymous)

54

50

4038

34

28 2725

22

18

14 13 13 12 12 12

7 6 5 4 3

0

10

20

30

40

50

60

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

Academ

icAca

demic

NHSAca

demic

Academ

icNHS

Academ

icNHSNHSNHSNHS

End: 5 TipsEnd: 5 Tips

Predictors of Journal AcceptancePredictors of Journal Acceptance• 1. Submitting the paper!• 2. Proofing and high quality submission• 3. Co-authorship• 4. Choosing a low impact journal• 5. Improving and re-sending after rejection• 6. Selling in the Cover letter• 7. Speaking with the editor• 8. Getting an invited paper

Predictors of High Impact CitationPredictors of High Impact Citation• 1. Publishing in a high impact journal

“There was some delay in submission……. due to a backlog in the to do tray”

Lawrence S. Online or invisible. Nature 2001;411:521http://external.nj.nec.com/~lawrence/papers/online-nature01/

Citations vs probability article is online

A comment from Rennie Drummond deputy editor JAMAA comment from Rennie Drummond deputy editor JAMA

There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive…..for a paper to end up in print.

What is the Cost of Research?What is the Cost of Research?

• CATIE (Clinical Antipsychotic Trials of Intervention Effectiveness)– National Institute of Mental Health (NIMH) clinical trial cost $67 million, with 1460

participants enrolled = $45,000 per patient for one year or $123 per day

• Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT) – The National Heart, Lung, and Blood Institute (NHLBI) funding for Fiscal Years

(FY) 1993-2004 was $83,170,059, and 42,418 participants enrolled => $2,000 per patient

• Diabetes Prevention Program (DPP) Clinical Trial – Funding: FY 1994-2002 was $176 million and 3,234 participants enrolled (45% were

minorities). => $54,000 per participant

Pubmed vs Wok vs ScopusPubmed vs Wok vs ScopusPubmed• 5,500 journals• 18 million articles

Scopus • 16,000• 36 million articles

WOK• 23,000 journals • 40 million articles

EndEnd