WHO Global Benchmarking Tool

WHO Regulatory Systems Strengthening Programme

2

Overall regulatory systems’ maturity level of WHO Member States

ML 1100 countries

51%ML 2

44 countries23%

ML 3 and 450 countries

26%

(Updated 8 Feb 2019)WHO MVP/RSS/CRS

3

To continue to support Member States upon their request in the area of regulatory system strengthening, including, as appropriate, by

continuing to:

Evaluate national regulatory systems

Apply WHO evaluation tools

Generate and analyze evidence of regulatory system performance

Facilitate the formulation and implementation of institutional development plans

Provide technical support to national regulatory authorities and governments

WHA Resolution 67.20 What WHO should do

4

WHA Resolution 67.20 What WHO should do

Norms and standards

Medical devices including

Diagnostics

Health systemstrengthening

Capacity building in developing

countries

ICDRA

International and regional

collaboration

Networks

WHO PQ program

Regulation of complex

biological

5

WHO NRA 5 step capacity building

Development of NRA

benchmarking tool

Benchmarking of NRA

Formulation of Institutional

Development Plan (IDP)

Providing technical support,

Training/Learning, networking,

Monitoring progress and

impact

•M

inim

al c

ap

acit

y m

et•

Eli

gib

ilit

y fo

r va

ccin

e P

Q•

WH

O li

sted

NR

As

1 2 3 4 5

Global context: different assessment tools collecting information from Regulatory Authorities and affiliated institutions

Unification of WHO tools

Convergence with non WHO

tools

WHO Global Benchmarking

Tool

Development of the WHO Global Benchmarking Tool (GBT)

7

Strategic direction

Vaccine 2011

Medicine 2007

PAHO 2012

MayJun 2013 Dec

Harmonized tool revision I Discussion

Jul

Istanbul workshop to implement IC

recommendation

Refining the tool and develop

revision III

Informal consultation

meeting, Geneva,

June 2013

Nov

1st IC 2nd IC

Jan Dec

Revision IV

Mar

Development of fact sheets

Development of the Global Tool (1/4)

WHO-PAHOmeeting

May

Harmonized tool revision I

Oct

2015

Global tool revision II

20162014

ECBS and BRN

Continueon next page

Aug

Development of the Global Tool (2/4)

Revision V

Development of fact sheets

WHO-PAHOmeeting

WHO-PAHOmeeting

WHO-PAHOmeeting

May Nov Feb Sep OctJune

20172016

Aug

WHO-PAHO WebEx

meetings

BRN-TSN-RSS workshop for integration of

blood products into GBT

Aug

ECBS and BRN

WHO-PAHO F-to-F

meeting

Continueon next page

WGs experts share written responses

Update TOR for WGs

WHO-PAHOmeeting

8-10 25

2018

11

MayApr

Compile comments

WGs WebEx discussions

Consolidate experts

comments

1 8

Jun

WGs F2F meeting

Reflection of agreed

amendments

6 23-25

Jul

9

Field testing of revision VI

Jan-Feb

Publish on the WHO

website for public

consultation

Continueon next page

Development of the Global Tool (3/4)

Development of the Global Tool (4/4)

Pilot the GBT draft revision

VI

2018

OctSep Nov

Final adjustment, editing and

publication of GBT revision VI

Translate and publish GBT revision VI in Spanish and

French

Publish summary of

comments and responses

2019

Phasing in of GBT

revision VI

Q1

Edit the GBT draft revision

VI

Aug

WGs F2F meeting

Jul

12

WHO RECOMMENDED REGULATORY FUNCTIONS FOR MEDICINES, VACCINES BASED ON PRODUCT LIFECYCLE

PRE MARKETING

Pre-clinical Clinical

Production& Quality Control

Marketing and sales

Post-Marketing

Non Common RegulatoryFunction for vaccines and

blood products

Common Regulatory Functions

for: medicines & vaccines and blood

products

Next steps: Medical devices and

blood products in 2019

POST MARKETING PRODUCT

LIFECYLCE

National Regulatory System (RS)

Regulatory Inspection (RI)

Laboratory access and Testing (LA)

Clinical Trial’s Oversight (CT)

Vigilance (PV)

Licensing premises (LI)

Registration & marketing authorization (MA)

Market surveillance and Control (MS)

NRA Lot release (LR)

14

WHO Global Benchmarking ToolStructure/Hierarchy

SUB-INDICATORS

INDICATORS

SYSTEM FUNCTION

THE FACT SHEET

QUESTIONNAIRE FOR OTHER PRODUCTS /ACTIVITIES

15

WHO Global Benchmarking ToolStructure/Hierarchy

SUB-INDICATORS

INDICATORS

SYSTEM FUNCTION

THE FACT SHEET

QUESTIONNAIRE FOR OTHER PRODUCTS /ACTIVITIES

INDICATORS

1. Legal provisions, regulations and guidelines2. Organization and governance3. Policy and strategic planning4. Leadership and crisis management5. Transparency, accountability and communication6. Quality and risk management system7. Regulatory process8. Resources (HR, FR, Experts, Infrastructure, Equipment

and IMS)9. Monitoring progress and assessing impact

Indicators Categorization (cross cutting subjects)

Categories enable assessment of cross sectional subjects (across some and/or all functions)

Current Status of Regulatory Systems WHO Global Benchmarking (for medicines and vaccines: as of September 2019)

2

Advanced/referenceRegulatory Authorities

Can ensure the quality of products if rely on ML3 / ML4 regulatory systems

Target of WHA Resolution 67.2

100COUNTRIES

44COUNTRIES

50COUNTRIES 17

No formal approach Reactive approach Stable formal

system approachContinual

improvement emphasized

1 2 3 4

ISO

900

4

Regulatory system operating at advanced level of

performance and continuous improvement

Evolving national regulatory system

that partially performs essential regulatory functions

Stable, well-functioning and integrated regulatory system

Some elements of regulatory system exist

WH

O G

BT

18

Updated Figures of the WHO GBT revision VI

Item Function RS MA VL MC LI RI LA CT LRGrand Total

Number of Sub-Indicators

60 35 26 27 19 26 28 30 17 268

Sub-Indicators measuring maturity level 1

4 6 5 3 2 3 2 2 1 28

Sub-Indicators measuring maturity level 2

7 2 3 4 1 2 2 8 3 32

Sub-Indicators measuring maturity level 3

27 23 14 15 13 13 18 17 11 152

Sub-Indicators measuring maturity level 4

22 4 4 5 3 8 6 3 2 56

Minimal capacity

Advanced/reference

NRAs

• Comprehensive system-based benchmarking• Integration of different medical product streams• Enhanced ability for customization of the tool• Adoption of maturity concept based on ISO

standard• Integration of SF related indicators• Expanded benchmarking of QMS for NRAs• Categorization of the indicators enabling

transverse benchmarking• Link with predecessor tools• Updating and expansion of regulatory guidelines• Comprehensive guidance for benchmarking• Maintenance of functionality concept as part of

eligibility criteria for WHO prequalification

WHO Global Benchmarking

Tool

computerized Global Benchmarking Tool (cGBT)

21

Recommended activities to be included in Institutional Development (IDP)

Regulatory functions Total number of Recommendations

No. ofrecommendations required to reach

ML3

01-NATIONAL REGULATORY SYSTEM (RS) 29 1002-REGISTRATION AND MARKETING AUTHORIZATION (MA) 28 1203-VIGILANCE (VG) 16 904-MARKET SURVEILLANCE AND CONTROL (MC) 10 105-LICENSING PREMISES (LI) 5 1

06-REGULATORY INSPECTION (RI) 13 407-LABORATORY ACCESS AND TESTING (LA) 3 108-CLINICAL TRIAL’S OVERSIGHT (CT) 15 1009-NRA LOT RELEASE (LR) 5 2

Total 123 50

China: The Success Story

2005 Mar/2011 Oct/2013 Q2/2014

NRA Benchmarking

Functional NRA, after benchmarking

in Dec 2010

NRA Re-Assessment

IDPIDP

More vaccine

submitted for PQ

1st PQ vaccine

23

Tanzania: The Success Story

WHO RECOMMENDED REGULATORY FUNCTIONS FOR MEDICINES, VACCINES AND BLOOD PRODUCTS BASED ON PRODUCT LIFECYCLE

PRE MARKETING

Pre-clinical Clinical

Production& Quality Control

Marketing and sales

Post-Marketing

Non Common RegulatoryFunction for vaccines and

blood products

POST MARKETING PRODUCT

LIFECYLCE

National Regulatory System (RS)

Regulatory Inspection (RI)

Laboratory access and Testing (LA)

Clinical Trial’s Oversight (CT)

Vigilance (PV)

Licensing premises (LI)

Registration & marketing

authorization (MA)

Market surveillance and Control

(MS)NRA Lot release (LR)

Non Common RegulatoryFunctions for blood

products

Approval of blood and blood components (product and/or process approval)

Regulatory oversight of associated substances and medical devices including IVD

Common Regulatory Functions

for: medicines & vaccines and blood

products

Next steps: Medical devices

Countries benchmarked against GBTJan 2016 - Sep 2019

* in accordance with United Nations Security Council resolution 1244 (1999)The boundaries and names shown and the designations used on this map do not imply the expression of any opinion whatsoever on the part of the World Health Organization (WHO) concerning the legal status of any country, territory, city or area of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on map represent approximate border lines for which there

may be not yet be full agreement.

(Updated Sep 2019)

• India

• Papua new guinea

• Timor-Leste

• Tanzania

• Burundi

• Mozambique

• Ethiopia

• Kenya

• Djibouti

• Eritrea

• Sudan

• Somalia

• Uganda

• South Sudan

• Afghanistan

• Pakistan

• Malaysia

• Japan

• Iraq

• Jordan

• Lebanon

• Mongolia

• Kyrgyzstan

• Syria

• Korea

• Bangladesh

• Iran

• Egypt

• Saudi Arabia

Self BenchmarkingFormal Benchmarking

• Maldives

• Nepal

• Sri Lanka

• Bhutan

• Guinea Bissau

• Liberia

• Niger

• Cape Verde

• Gambia• Serbia

• Burkina Faso

• Benin

• Guinea

• Togo

• Mali

• Cote d’ivoire

• Senegal

• Sierra Leone

• Cambodia

• Lao PDR

• Thailand

• Indonesia• Montenegro

• Macedonia

• Bosnia and Herzegovina

• Albania

• Kosovo area*

• Kazakhstan

• Vietnam

• Rwanda

• Turkey• China

• Eswatini

• Comoros

• Madagascar

• Malawi

• South Africa

• Zambia

• Botswana• Lesotho

• Namibia

• Seychelles

• Zimbabwe

• Congo DR • Angola

• Mauritius

• Ghana

• El Salvador• Nigeria

26

Go to our Regulatory system strengthening website

Go to SharePoint site

Go to the Global Benchmarking Tool

Go to GBT online training

Important Links

Thank you

Dr Alireza Khadem

Email: khadembroojerdia@who.int

Regulatory Systems Strengthening (RSS)

Regulation of Medicines and Other Health Technologies (RHT)

World Health Organization (Geneva, Switzerland)