google partners hangout: lexical analysis part 2
TRANSCRIPT
Part 2: Lexical Analysis October 2015
Confidential and Proprietary. © Boost Media Inc. All Rights Reserved
We’ll Cover
I. What is Lexical Analysis?
II. Process
III. Examples & Best Practices
IV.Q&A
1. SEM is a great place to test 2. Create a culture of testing3. Follow a well-defined process
Here’s What You Learned in Part 1: Understanding Hypothesis Testing
Defining Lexical Analysis
● Text analytics is the process of analyzing unstructured text, extracting relevant information, and transforming it into useful business intelligence.
Defining Lexical Analysis
● Text analytics is the process of analyzing unstructured text, extracting relevant information, and transforming it into useful business intelligence.
The Benefits
1. Find universal truths2. Identify concepts that are working in a specific section of your account3. Provide directional information that leads to hypothesis generation
Example: A Flower Company
How Do I Do This?
1. Generate ad comparisons
2. Statistically test
3. Tokenize ads
4. Assign value
5. Aggregate
6. Interpret results
1. Generate Relevant Ad Comparisons
1. Generate Relevant Ad Comparisons
Account
1. Generate Relevant Ad Comparisons
Account
Campaign 2Campaign 1 Campaign 3
1. Generate Relevant Ad Comparisons
Account
Campaign 2Campaign 1 Campaign 3
Ad Group 1
Ad 1
Ad 2
Ad 3
Ad 4
1. Generate Relevant Ad Comparisons
Account
Campaign 2Campaign 1 Campaign 3
Ad Group 1Ad Group 2
Ad 1
Ad 2
Ad 3
Ad 5
Ad 6
Ad 7
Ad 4
1. Generate Relevant Ad Comparisons
Account
Campaign 2Campaign 1 Campaign 3
Ad Group 1Ad Group 2 Ad Group 3
Ad 1
Ad 2
Ad 3
Ad 5
Ad 6
Ad 7
Ad 8
Ad 9
Ad 10
Ad 4
1. Generate Relevant Ad Comparisons
Account
Campaign 2Campaign 1 Campaign 3
Ad Group 1Ad Group 2 Ad Group 3 Ad Group 4
Ad 1
Ad 2
Ad 3
Ad 5
Ad 6
Ad 7
Ad 8
Ad 9
Ad 10
Ad 11
Ad 12
Ad 13
Ad 4
1. Generate Relevant Ad Comparisons
Account
Campaign 2Campaign 1 Campaign 3
Ad Group 1Ad Group 2 Ad Group 3 Ad Group 4 Ad Group 5
Ad 1
Ad 2
Ad 3
Ad 5
Ad 6
Ad 7
Ad 8
Ad 9
Ad 10
Ad 11
Ad 12
Ad 13
Ad 14
Ad 15
Ad 16
Ad 4
1. Generate Relevant Ad Comparisons
Account
Campaign 2Campaign 1 Campaign 3
Ad Group 1Ad Group 2 Ad Group 3 Ad Group 4 Ad Group 5
Ad 1
Ad 2
Ad 3
Ad 5
Ad 6
Ad 7
Ad 8
Ad 9
Ad 10
Ad 11
Ad 12
Ad 13
Ad 14
Ad 15
Ad 16
Ad 4
Rough estimate of data set: n C 2 * # of Ad Groups where n = # Ads / # Ad Groups
1. Generate Relevant Ad Comparisons
1. Generate Relevant Ad ComparisonsAccount
Campaign 1
Ad Group 1
Ad 1
Ad 2
Ad 3
Ad 4
1. Generate Relevant Ad ComparisonsAccount
Campaign 1
Ad Group 1
Ad 1
Ad 2
Ad 3
Ad 4
TIME
1. Generate Relevant Ad ComparisonsAccount
Campaign 1
Ad Group 1
Ad 1
Ad 2
Ad 3
Ad 4
TIME
1. Generate Relevant Ad ComparisonsAccount
Campaign 1
Ad Group 1
Ad 1
Ad 2
Ad 3
Ad 4
TIME
1. Generate Relevant Ad ComparisonsAccount
Campaign 1
Ad Group 1
Ad 1
Ad 2
Ad 3
Ad 4
TIME
2. Statistically Test All Comparisons
● Decide what metric you want to gain insight on● Choose an appropriate statistical test● For CTR, we use a Chi-Squared test with Yates correction● Choose a confidence threshold
2. Statistically Test All Comparisons
● Decide what metric you want to gain insight on● Choose an appropriate statistical test● For CTR, we use a Chi-Squared test with Yates correction● Choose a confidence threshold
2. Statistically Test All Comparisons
● Decide what metric you want to gain insight on● Choose an appropriate statistical test● For CTR, we use a Chi-Squared test with Yates correction● Choose a confidence threshold
Impressions Clicks CTR Difference Confidence
Ad 1 9,000 144 1.6% 22.22%
91%Ad 2 8,500 109 1.28% -22.22%
3. Tokenize Each Ad● Decide what to do with:
- Punctuation - Special Characters- Numbers - Keyword Insertion
3. Tokenize Each Ad● Decide what to do with:
- Punctuation - Special Characters- Numbers - Keyword Insertion
Great Mens Dress ShoesFind Super Deals On A WideSelection Of Shoes! Shop Now!
['great', 'mens', 'dress', 'shoes', 'find', 'super', 'deals', 'on', 'a', 'wide', 'selection', 'of', 'shoes', 'shoes', 'now', 'great mens', 'mens dress', 'dress shoes', 'find super', 'super deals', 'deals on', 'on a', 'a wide', 'wide selection', 'selection of', 'of shoes', 'shoes now', 'great mens dress', 'mens dress shoes', 'find super deals', 'super deals on', 'deals on a', 'on a wide', 'a wide selection', 'wide selection of', 'selection of shoes']
3. Tokenize Each Ad● Decide what to do with:
- Punctuation - Special Characters- Numbers - Keyword Insertion
Great Mens Dress ShoesFind Super Deals On A WideSelection Of Shoes! Shop Now!
Dress Shoes at AcmeExplore The Selection Of ShoesAt Acme Shoes! Shop Now!
['great', 'mens', 'dress', 'shoes', 'find', 'super', 'deals', 'on', 'a', 'wide', 'selection', 'of', 'shoes', 'shoes', 'now', 'great mens', 'mens dress', 'dress shoes', 'find super', 'super deals', 'deals on', 'on a', 'a wide', 'wide selection', 'selection of', 'of shoes', 'shoes now', 'great mens dress', 'mens dress shoes', 'find super deals', 'super deals on', 'deals on a', 'on a wide', 'a wide selection', 'wide selection of', 'selection of shoes']
['dress', 'shoes', 'at', 'acme', 'explore', 'the', 'selection', 'of', 'shoes', 'at', 'acme', 'shoes', 'show', 'now', 'dress shoes', 'shoes at', 'at acme', 'explore the', 'the selection', 'selection of', 'of shoes', 'shoes at', 'at acme', 'acme shoes', 'show now', 'dress shoes at', 'shoes at acme', 'explore the selection', 'the selection of', 'selection of shoes', 'of shoes at', 'shoes at acme', 'at acme shoes']
3. Tokenize Each Ad (cont)
● Tag each word with part of speech● Allows for insight exploration by word type
○ Descriptive words (adjectives, adverbs)○ Features (nouns)○ Action/CTA words (verbs)
3. Tokenize Each Ad (cont)
● Tag each word with part of speech● Allows for insight exploration by word type
○ Descriptive words (adjectives, adverbs)○ Features (nouns)○ Action/CTA words (verbs)
Great Mens Dress ShoesFind Super Deals On A WideSelection Of Shoes! Shop Now!
[('great', 'JJ'), ('mens', 'NNS'), ('dress', 'NN'), ('shoes', 'NNS'), ('find', ‘VB’), ('super', 'ADJ'), ('deals', 'NNS'), ('on', 'IN'), ('a', 'DT'), ('wide', 'JJ'), ('selection', 'NN'), ('of', 'IN'), ('shoes', 'NNS'), ('shop', 'VBP'), ('now', 'RB')]
4. Assign Value To Each Token
Great Mens Dress ShoesFind Super Deals On A WideSelection Of Shoes! Shop Now!
Dress Shoes at AcmeExplore The Selection Of ShoesAt Acme Shoes! Shop Now!
vs
22% DIFFERENCE22% -22%
4. Assign Value To Each Token
Great Mens Dress ShoesFind Super Deals On A WideSelection Of Shoes! Shop Now!
Dress Shoes at AcmeExplore The Selection Of ShoesAt Acme Shoes! Shop Now!
vs
22% DIFFERENCE
Token Value
find 0.22
explore -0.22
shoes 0 = .22+ (-.22)
22% -22%
5. Aggregate To Show Total Value Creation● Aggregate to token level● Sum values assigned for positive, neutral, and negative impact● Count positive, negative, and total occurrences
5. Aggregate To Show Total Value Creation● Aggregate to token level● Sum values assigned for positive, neutral, and negative impact● Count positive, negative, and total occurrences
Token Value
find 0.22
explore -0.22
shoes 0.22
shoes -0.22
great 0.22
find 0.16
explore -.16
find -.05
great -.025
5. Aggregate To Show Total Value Creation● Aggregate to token level● Sum values assigned for positive, neutral, and negative impact● Count positive, negative, and total occurrences
Token Value
find 0.22
explore -0.22
shoes 0.22
shoes -0.22
great 0.22
find 0.16
explore -.16
find -.05
great -.025
Σ
Token Value Count
find 0.33 3
great 0.09 2
shoes 0 2
great -.03 2
explore -0.38 2
6. Interpret the Results Carefully
1. Universal: Find tokens that have many occurrences and large impacts○ Negative - consider alternatives for replacement ○ Positive - find ways to increase usage
6. Interpret the Results Carefully
1. Universal: Find tokens that have many occurrences and large impacts○ Negative - consider alternatives for replacement ○ Positive - find ways to increase usage
2. Segments: Drill into tokens with large portions of negative and positive occurrences ○ Account segments○ Phrase segments○ These can be the MOST insightful
6. Interpret the Results Carefully
1. Universal: Find tokens that have many occurrences and large impacts○ Negative - consider alternatives for replacement ○ Positive - find ways to increase usage
2. Segments: Drill into tokens with large portions of negative and positive occurrences ○ Account segments○ Phrase segments○ These can be the MOST insightful
3. Leading Indicator: Positive tokens with small occurrences should be explored further
This is the “Art” of the analysis. These are guidelines but not rules. These datasets open the door for great exploration.
Example: A Flower Company
Example: A Flower Company
Example: A Flower Company
Example: A Flower Company
“Quality” Doesn’t Instill Confidence
“Fresh” is a descriptive explanation of why the product is high quality
Flowers Are “In The Moment”
Same day shipping is important to flower buyers as they are consistently an “in the moment” purchase for someone special or to react to an event
Example: A Flower Company
Use Emotional Action Statements
“Show” is an emotional action statement highlighting the benefit to the flower purchaser
“Choose” is a process oriented action, describing the buying action and not the recipient's reaction
What Can We Do With These Insights?
● Use these insights to drive other business strategy decisions
from product marketing to operations
● Write some pretty darn good ads to increase performance of
your paid search program:
What Can We Do With These Insights?
● Use these insights to drive other business strategy decisions
from product marketing to operations
● Write some pretty darn good ads to increase performance of
your paid search program:
Gorgeous Fresh FlowersShow you care with one of our fieldfresh varieties. Ship the same day!
{Specific Event} FlowersShow you care with one of our fieldfresh varieties. Ship the same day!
Summary - What Did We Learn?
1. Focus on expanding the use of your most positive words
2. Look to minimize negative word usage
3. Look for themes in positive and negative words
4. Quality of analysis depends on the diversity of language that has been
used
Boost Media150 Columbus Ave. San Francisco, CA 94133BoostMedia.com1.800.777.9415
@boostctr
Thank You! Questions?
Rob Lenderman [email protected]
Bill [email protected]
Ask your questions in the Q&A Hangout app or post them on the event page.