diploma in cosmetic science module 4 unit 25 …...iso 5496:2006 sensory analysis iso 11132:2012...
TRANSCRIPT
DIPLOMA IN COSMETIC SCIENCE
MODULE 4 UNIT 25 PRODUCT EVALUATION
PRODUCT EVALUATION – WITH EXAMPLES
FROM SKIN CARE
Paul Cornwell, SCS - Summer School 5th July 2020
Key points
Definition of product evaluation
Objective vs subjective
To design a good test
Have a clear objective and hypothesis
Define study outline
Have a tested & validated protocol
Get the operational details right
Define your statistics and success criteria
Understand the range of tests used for skin and
hair
To write a good protocol
Always follow the five steps above!
What is product evaluation? (section 1)
The term product evaluation is used to describe the process of
assessing one or more characteristics or attributes of a
product
The unit focuses on evaluation tests that assess how well
products perform when they come in contact with their target
body sites (skin, hair, teeth etc)
Includes instrumental and sensory tests
Formulation tests and product safety tests are also classed as
product evaluation but are the focus of other modules
Contact ATR-FTIR being used at TRI to test
scalp condition
Who uses skin product evaluation (section 1.3)
Discovery
Post-launch
evaluation
LaunchTesting and
validationDevelopment
Build Business
CaseScoping Gate 2
Idea
Screen
Second screen
Go to development Go to testing Go to launch
Gate 5Gate 4Gate 3
Gate 1
Gate 1
Gate 3
Gate 4
Gate 5
Gate 2
Skin
clinical Consumer
test
Expert
panel
Who uses skin product evaluation (section 1.3)
Quiz time! Match the evaluation test with the business need
Test comparing consumer preference for
liquid soap and synthetic detergent
based body wash products in India
Who uses skin product evaluation (section 1.3)
Question 1. Which type of test is
recommended in this case:
(a) Expert sensory panel
(b) Clinical test
(c) Consumer test
Performance test on a new cream formulation
for sensitive skin using TEWL (improved barrier
function) and SLS occlusive patch test (reduced
sensitivity to cosmetic products)
Who uses skin product evaluation (section 1.3)
Question 2. Which type of test is
recommended in this case:
(a) Expert sensory panel
(b) Clinical test
(c) Consumer test
Who uses skin product evaluation (section 1.3)
Rapid testing of multiple laboratory
prototype formulations for a new, fast
absorbing body lotion
Question 3. Which type of test is
recommended in this case:
(a) Expert sensory panel
(b) Clinical test
(c) Consumer test
Different categories of evaluation methods (section 2)
Product evaluation
Objective* Subjective+
Instrumental
e.g. skin clinical
Sensory
e.g. expert
sensory panel or
skin clinical with
expert grading
Sensory
e.g. consumer test
Adapted from: IFSCC Monograph Number 1. Principles of Product Evaluation: Objective Sensory Methods
* Procedures carried out by
trained personnel using either
instrumental or sensory methods
which produce reproducible and
quantifiable results
+ Procedures involving sensory
test methods carried out by
untrained people ..(and which
are)..influenced by personal
preference
Designing a product evaluation test (section 3)
1. Have a clear objective and technical hypothesis
2. Consider the study outline
3. Have a robust and validated protocol
4. Get the operational details right
5. Define your statistical tests and success criteria
1. Have a clear objective (section 3.2.2)
In product testing you should always ask: “why am I doing this?”, and “what, exactly, will the information I
collect be used for?”
A crystal clear objective will help you build a strong experimental plan
In businesses this usually means working closely with your team and key stakeholders to define what the
project needs really are
1. Have a clear objective (section 3.2.2)
Having a Hypothesis
Definition of hypothesis: A supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation (Oxford Dictionary)
In product testing you should always ask: “why should this work?”, ”is there any technical reason why this product should do what we want it to do?”
Tests based on strong hypotheses are more likely to succeed
Be ready not to perform a test if there is no hypothesis as to why it should succeed (You will be surprised how often this happens!)
2. Study outline – (a) Safety & Regulations (section
3.2.3)
International Ethical Guidelines (2002)1 based on 1964
Declaration of Helsinki
Ethical and justifiable experiment
Reviewed by an ethical review body
Informed consent
Suitable reimbursement or payment
Safety and wellbeing of panellists protected – safety
review
Protection of vulnerable panellists
Adverse reaction processes in place
Confidentiality
1. International Ethical Guidelines for Biomedical Research Involving Human Subjects. Prepared by the Council for International
Organizations of Medical Sciences (CIOMS) in collaboration with the World Health Organization (WHO), Geneva (2002)
2. Study Outline – (a) Safety & Regulations (section
3.2.3)
Guideline for Good Clinical Practice (GCP) (1996) ICH
– covers European Union, US and Japan
In the US you also follow either US FDA regulations or
‘Common Rule’
2. Study Outline – (b) Panellist recruitment (section
3.2.3)
Target population - right distribution of sex, age, ethnicity, socio-economic status etc
Skin health - various expert grading systems
Exclusion criteria – medical/skin conditions, pregnancy, breast feeding etc
(time of year)
2. Study Outline – (b) Panellist recruitment – dryness
(section 3.2.3)
1. Simion F.A., Abrutyn E.S., Draelos Z.D. Ability of moisturizers to reduce dry skin and irritation and to prevent their return. J Cosmet Sci.
56(6):427-44 (2005)
2. Yoshida-Amano Y., Nomura T., Sugiyama Y., Iwata K., Higaki Y., Tanahashi M. Dry skin conditions are related to the recovery rate of
skin temperature after cold stress rather than to blood flow Int J Dermatol. 56(2):176-183 (2017)
Images illustrating dryness grading system2
Words used for a dryness grading system1
Dryness
score
Description of skin
0 No dryness
1 Slight flaking
2 Moderate flaking/scaling
3 Marked scaling/slight fissuring
4 Severe scaling/fissuring
2. Study Outline – (b) Panellist recruitment – wrinkling
(section 3.2.3)
1. Dayan, N. Selecting the Population for Human Clinical Studies Part 1. HAPPI Magazine, 31st January (2018)
Glogau Scale for severity of photoaging and especially wrinkling1
Group Classification Typical age Description Skin Characteristics
1 Mild Late 20s to
mid 30s
No wrinkles Early photo-ageing; mild
pigment changes; no
keratosis, minimal wrinkles
II Moderate Mid 30s to
early 50s
Wrinkles in facial
expression
Early to moderate
photoageing; early brown
spots; visible keratosis; smile
lines begin to appear
III Advanced Early 50s to
mid 60s
Wrinkles at rest Advanced photo-ageing;
obvious discolorations; visible
capillaries; visible keratosis
2. Study Outline – (b) Panellist recruitment – UV
response (section 3.2.3)
1. Dayan, N. Selecting the Population for Human Clinical Studies Part 1. HAPPI Magazine, 31st January (2018)
Fitzpatrick scale for skin colour and response to UV light1
Skin type Skin response Skin Characteristics
I Skin that always burns, never tans Pale white; blond or red hair; blue eyes; freckles
II Usually burns, tans minimally White; fair; blond or red hair; blue, green or hazel eyes
III Sometimes mild burn, tans uniformly Cream white; fair with any hair or eye colour
IV Burns minimally, always tans well Moderate brown
V Very rarely burns, tans very easily Moderate to darker brown
VI Never burns, never tans Deep pigmented dark brown to darkest brown
2. Study Outline - (c) Method of assessment (section
3.2.3)
Skin hydration
Skin colour
Skin topography
In vivo tensile properties of the skin
Trans-epidermal water loss
In vivo skin greasiness
Skin surface pH
etc
Modified from – Nobile, V. Guidelines on Cosmetic Efficacy Testing on Humans. Ethical, Technical, and Regulatory Requirements in the Main
Cosmetics Markets J. Cosmo. Trichol. 2:1 (2016)
EEMCO guidance notes for skin measurements
2. Study Outline - (c) Method of assessment (section
3.2.3)
ISO 5496:2006 sensory analysis
ISO 11132:2012 expert sensory panels
ASTM E-1958, 2018 edition. sensory claim substantiation
ISO/TR 26369:2009 sun protection test methods
ISO 24444:2010 in vivo SPF
ISO 24442:2011 UVA protection
Final Rule 29: sunscreen drug products. FDA regulations
Modified from – Nobile, V. Guidelines on Cosmetic Efficacy Testing on Humans. Ethical, Technical, and Regulatory Requirements in the Main
Cosmetics Markets J. Cosmo. Trichol. 2:1 (2016)
Useful standards and regulations
2. Study Outline – (c) Method of assessment – sensory
(section 3.2.3)
1. Classification
2. Grading
3. Ranking
4. Scaling (e.g. category scaling, line scaling)
Chapter 5. Measuring Responses Meilgaard, M.C., Civille, G.V., Carr, B.T. Sensory Evaluation Techniques, Fifth Edition, CRC Press, pp 51-
70 (2016)
2. Study Outline – (c) Method of assessment – sensory
(section 3.2.3)
1. (Objective) Difference test – Does a sensory difference exist between
samples? e.g. triangle test, duo-trio test
2. (Objective) Attribute difference test – How does an attribute X differ
between samples? e.g. ranking test, rating test
3. (Subjective) Affective test – Which sample do people prefer? e.g. rank
preference, hedonic rating
Chapter 6. Guidelines for Choice of Technique Meilgaard, M.C., Civille, G.V., Carr, B.T. Sensory Evaluation Techniques, Fifth Edition, CRC
Press, pp 71-78 (2016)
2. Study Outline – (c) Method of assessment – sensory
(section 3.2.3)
Quantitative Descriptive Analysis (QDA) – expert panel
Use carefully selected panellists + use panel to define sensory
attributes + use line scales to score the products
Data limited to each test
SpectrumTM Descriptive Analysis – expert panel
Use carefully selected panellists + train panel to score reproducibly
+ use category scaling
Time consuming to perform, but data can be compared across studies
and time
2. Study Outline – (c) Method of assessment – sensory
(section 3.2.3)
Quiz time! Question 4. The on-line triangle test. Perhaps one
picture is slightly different to the other two? Which one could it
be? You have to pick one, even if you think they are all the
same.
2. Study Outline – (c) Method of assessment – sensory
(section 3.2.3)
A B C
2. Study Outline – (c) Method of assessment – sensory
(section 3.2.3)
A B C
2. Study Outline – (c) Method of assessment – sensory
(section 3.2.3)
n a 0.04 0.30 0.20 0.10 0.05 0.01 0.001
3 2 2 3 3 3 - -
4 3 3 3 4 4 - -
5 3 3 4 4 4 5 -
6 3 4 4 5 5 6 -
7 4 4 4 5 5 6 7
8 4 4 5 5 6 7 8
9 4 5 5 6 6 7 8
10 5 5 6 6 7 8 9
11 5 5 6 7 7 8 10
12 5 6 6 7 8 9 10
13 6 6 7 8 8 9 11
14 6 7 7 8 9 10 11
15 6 7 8 8 9 10 12
16 7 7 8 9 9 11 12
17 7 8 8 9 10 11 13
18 7 8 9 10 10 12 13
19 8 8 9 10 11 12 14
20 8 9 9 10 11 13 14
21 8 9 10 11 12 13 15
22 9 9 10 11 12 14 15
23 9 10 11 12 12 14 16
24 10 10 11 12 13 15 16
25 10 11 11 12 13 15 17
26 10 11 12 13 14 15 17
27 11 11 12 13 14 16 18
28 11 12 12 14 15 16 18
29 11 12 13 14 15 17 19
30 12 12 13 14 15 17 19
Critical number of correct responses in a triangle test. Taken from: Statistical tables in
Meilgaard, M.C., Civille G.V. and Carr, B.T Sensory Evaluation Techniques, 5th Edition,
CRC Press, page 554 (2016)
2. Study Outline (d) – other considerations (section 3.2.3
- 3.2.8)
Study duration. How much time would be needed to study
product performance?
Application regime. Realistic or exaggerated?
Avoiding bias. Single or double blind
Controls. Are the right controls in place?
3. Have a tested & validated protocol (section 3.2.4)
For both objective and subjective sensory tests it is advisable to do 1-2 dry-runs of the protocol to ensure
the instructions are clear and the questionnaires work properly
For expert sensory panels and skin clinicals with expert grading it will be necessary to train the expert
evaluators to perform the scoring, and then to check for accuracy, precision, discrimination and
reproducibility. Ineffective evaluators will need to be removed from the test
For objective instrumental tests equipment needs to be calibrated and validated
4. Get the operational details right (section 3.2.11)
Right study location. Humidity & temperature control?
Right amounts of product, packaged and labelled/coded in the correct way
Good GLP - all formulation and ingredient details recorded & available
Avoid unnecessary biases, e.g. colour, fragrance, packaging, knowledge etc
Right consumables available for evaluators
Panellists recruited and booked-in
Safety assessments done, and consent forms completed
5. Define your statistical tests and success criteria
(section 3.2.12)
Define your key measurements; and, if possible, have several measurements for robustness and
secondary analysis
Select the appropriate statistical tests and success criteria ahead of testing
Do power calculations before starting to get the right sample size. Type I and Type II errors are
usually set at 0.05 (5% chance of a “false positive”) and ≤0.2 (20% chance of a “false
negative”; at least 80% statistical power), respectively. Pilot studies might be necessary to
determine the expected differences and standard deviations
Understand the difference between statistically significant and consumer meaningful data
Back in the real world: SOPs and protocols
SOP – Detailed instructions for performing a task (usually a routine, validated method)
Protocol – Experimental plan specific to your study
NOTE- sometimes both are combined into one and just called a ‘protocol’
SOP
Scope
Roles &Responsibilities
Safety & ethics
Panellist recruitment
Location, materials & equipment
Experimental procedure
Data analysis
Appendices
Consent form
Protocol sheet
Assessment form
SOP – Assessment Sheet
SOP - Protocol sheet
Test Number: XXXXX
Project Name: XXXXX
Test objective: To compare the performance of lotion 345262 and 755465 using the lotion
testing SOP. Follow instructions for lotion applications.
Recruitment
criteria:
Females, 18-45 yrs, dry skin
Minimum replicates:
N=25
Success criteria: Paired t-tests should show p<0.05. Key attributes are absorbance and stickiness
Product details
Product code: 118
Treatment: Current lotion batch 345262
Product code 345
Treatment: Prototype lotion batch 755465
Panellist Name Arrangement Left Right
1 Mamma Mia AB 118 345
2 BA 345 118
3 BA 345 118
4 AB 118 345
5 BA 345 118
6 AB 118 345
7 AB 118 345
8 AB 118 345
9 BA 345 118
10 BA 345 118
11 AB 118 345
12 BA 345 118
13 BA 345 118
14 AB 118 345
15 BA 345 118
16 AB 118 345
17 AB 118 345
18 AB 118 345
19 BA 345 118
20 BA 345 118
21 AB 118 345
22 BA 345 118
23 BA 345 118
24 AB 118 345
25 BA 345 118
Key points
Definition of product evaluation
Objective vs subjective
To design a good test
Have a clear objective and hypothesis
Define study outline
Have a tested & validated protocol
Get the operational details right
Define your statistics and success criteria
Understand the range of tests used for skin and
hair
To write a good protocol
Always follow the five steps above!
Book list
IFSCC Monograph Number 1. Principles of Product Evaluation: Objective Sensory Methods
Kemp, S.E., Hollowood, T. and Hort, J. Sensory Evaluation: A Practical Handbook. Wiley-Blackwell,
2009
Meilgaard, M.C., Civille, G.V., Carr, B.T. Sensory Evaluation Techniques, Fifth Edition, CRC Press, 2016
Fluhr, J. (Ed.) Practical Aspects of Cosmetic Testing: How to Set up a Scientific Study in Skin
Physiology, Springer 2011
Glantz, S.A. Primer of Biostatistics, Seventh Edition, McGraw-Hill, 2012