google partners hangout: lexical analysis part 2

Part 2: Lexical Analysis October 2015

Confidential and Proprietary. © Boost Media Inc. All Rights Reserved

We’ll Cover

I. What is Lexical Analysis?

II. Process

III. Examples & Best Practices

IV.Q&A

1. SEM is a great place to test 2. Create a culture of testing3. Follow a well-defined process

Here’s What You Learned in Part 1: Understanding Hypothesis Testing

Defining Lexical Analysis

● Text analytics is the process of analyzing unstructured text, extracting relevant information, and transforming it into useful business intelligence.

Defining Lexical Analysis

● Text analytics is the process of analyzing unstructured text, extracting relevant information, and transforming it into useful business intelligence.

The Benefits

1. Find universal truths2. Identify concepts that are working in a specific section of your account3. Provide directional information that leads to hypothesis generation

Example: A Flower Company

How Do I Do This?

1. Generate ad comparisons

2. Statistically test

3. Tokenize ads

4. Assign value

5. Aggregate

6. Interpret results

1. Generate Relevant Ad Comparisons

1. Generate Relevant Ad Comparisons

Account

1. Generate Relevant Ad Comparisons

Account

Campaign 2Campaign 1 Campaign 3

1. Generate Relevant Ad Comparisons

Account

Campaign 2Campaign 1 Campaign 3

Ad Group 1

Ad 1

Ad 2

Ad 3

Ad 4

1. Generate Relevant Ad Comparisons

Account

Campaign 2Campaign 1 Campaign 3

Ad Group 1Ad Group 2

Ad 1

Ad 2

Ad 3

Ad 5

Ad 6

Ad 7

Ad 4

1. Generate Relevant Ad Comparisons

Account

Campaign 2Campaign 1 Campaign 3

Ad Group 1Ad Group 2 Ad Group 3

Ad 1

Ad 2

Ad 3

Ad 5

Ad 6

Ad 7

Ad 8

Ad 9

Ad 10

Ad 4

1. Generate Relevant Ad Comparisons

Account

Campaign 2Campaign 1 Campaign 3

Ad Group 1Ad Group 2 Ad Group 3 Ad Group 4

Ad 1

Ad 2

Ad 3

Ad 5

Ad 6

Ad 7

Ad 8

Ad 9

Ad 10

Ad 11

Ad 12

Ad 13

Ad 4

1. Generate Relevant Ad Comparisons

Account

Campaign 2Campaign 1 Campaign 3

Ad Group 1Ad Group 2 Ad Group 3 Ad Group 4 Ad Group 5

Ad 1

Ad 2

Ad 3

Ad 5

Ad 6

Ad 7

Ad 8

Ad 9

Ad 10

Ad 11

Ad 12

Ad 13

Ad 14

Ad 15

Ad 16

Ad 4

1. Generate Relevant Ad Comparisons

Account

Campaign 2Campaign 1 Campaign 3

Ad Group 1Ad Group 2 Ad Group 3 Ad Group 4 Ad Group 5

Ad 1

Ad 2

Ad 3

Ad 5

Ad 6

Ad 7

Ad 8

Ad 9

Ad 10

Ad 11

Ad 12

Ad 13

Ad 14

Ad 15

Ad 16

Ad 4

Rough estimate of data set: n C 2 * # of Ad Groups where n = # Ads / # Ad Groups

1. Generate Relevant Ad Comparisons

1. Generate Relevant Ad ComparisonsAccount

Campaign 1

Ad Group 1

Ad 1

Ad 2

Ad 3

Ad 4

1. Generate Relevant Ad ComparisonsAccount

Campaign 1

Ad Group 1

Ad 1

Ad 2

Ad 3

Ad 4

TIME

1. Generate Relevant Ad ComparisonsAccount

Campaign 1

Ad Group 1

Ad 1

Ad 2

Ad 3

Ad 4

TIME

1. Generate Relevant Ad ComparisonsAccount

Campaign 1

Ad Group 1

Ad 1

Ad 2

Ad 3

Ad 4

TIME

1. Generate Relevant Ad ComparisonsAccount

Campaign 1

Ad Group 1

Ad 1

Ad 2

Ad 3

Ad 4

TIME

2. Statistically Test All Comparisons

● Decide what metric you want to gain insight on● Choose an appropriate statistical test● For CTR, we use a Chi-Squared test with Yates correction● Choose a confidence threshold

2. Statistically Test All Comparisons

● Decide what metric you want to gain insight on● Choose an appropriate statistical test● For CTR, we use a Chi-Squared test with Yates correction● Choose a confidence threshold

2. Statistically Test All Comparisons

● Decide what metric you want to gain insight on● Choose an appropriate statistical test● For CTR, we use a Chi-Squared test with Yates correction● Choose a confidence threshold

Impressions Clicks CTR Difference Confidence

Ad 1 9,000 144 1.6% 22.22%

91%Ad 2 8,500 109 1.28% -22.22%

3. Tokenize Each Ad● Decide what to do with:

- Punctuation - Special Characters- Numbers - Keyword Insertion

3. Tokenize Each Ad● Decide what to do with:

- Punctuation - Special Characters- Numbers - Keyword Insertion

Great Mens Dress ShoesFind Super Deals On A WideSelection Of Shoes! Shop Now!

['great', 'mens', 'dress', 'shoes', 'find', 'super', 'deals', 'on', 'a', 'wide', 'selection', 'of', 'shoes', 'shoes', 'now', 'great mens', 'mens dress', 'dress shoes', 'find super', 'super deals', 'deals on', 'on a', 'a wide', 'wide selection', 'selection of', 'of shoes', 'shoes now', 'great mens dress', 'mens dress shoes', 'find super deals', 'super deals on', 'deals on a', 'on a wide', 'a wide selection', 'wide selection of', 'selection of shoes']

3. Tokenize Each Ad● Decide what to do with:

- Punctuation - Special Characters- Numbers - Keyword Insertion

Great Mens Dress ShoesFind Super Deals On A WideSelection Of Shoes! Shop Now!

Dress Shoes at AcmeExplore The Selection Of ShoesAt Acme Shoes! Shop Now!

['great', 'mens', 'dress', 'shoes', 'find', 'super', 'deals', 'on', 'a', 'wide', 'selection', 'of', 'shoes', 'shoes', 'now', 'great mens', 'mens dress', 'dress shoes', 'find super', 'super deals', 'deals on', 'on a', 'a wide', 'wide selection', 'selection of', 'of shoes', 'shoes now', 'great mens dress', 'mens dress shoes', 'find super deals', 'super deals on', 'deals on a', 'on a wide', 'a wide selection', 'wide selection of', 'selection of shoes']

['dress', 'shoes', 'at', 'acme', 'explore', 'the', 'selection', 'of', 'shoes', 'at', 'acme', 'shoes', 'show', 'now', 'dress shoes', 'shoes at', 'at acme', 'explore the', 'the selection', 'selection of', 'of shoes', 'shoes at', 'at acme', 'acme shoes', 'show now', 'dress shoes at', 'shoes at acme', 'explore the selection', 'the selection of', 'selection of shoes', 'of shoes at', 'shoes at acme', 'at acme shoes']

3. Tokenize Each Ad (cont)

● Tag each word with part of speech● Allows for insight exploration by word type

○ Descriptive words (adjectives, adverbs)○ Features (nouns)○ Action/CTA words (verbs)

3. Tokenize Each Ad (cont)

● Tag each word with part of speech● Allows for insight exploration by word type

○ Descriptive words (adjectives, adverbs)○ Features (nouns)○ Action/CTA words (verbs)

Great Mens Dress ShoesFind Super Deals On A WideSelection Of Shoes! Shop Now!

[('great', 'JJ'), ('mens', 'NNS'), ('dress', 'NN'), ('shoes', 'NNS'), ('find', ‘VB’), ('super', 'ADJ'), ('deals', 'NNS'), ('on', 'IN'), ('a', 'DT'), ('wide', 'JJ'), ('selection', 'NN'), ('of', 'IN'), ('shoes', 'NNS'), ('shop', 'VBP'), ('now', 'RB')]

4. Assign Value To Each Token

Great Mens Dress ShoesFind Super Deals On A WideSelection Of Shoes! Shop Now!

Dress Shoes at AcmeExplore The Selection Of ShoesAt Acme Shoes! Shop Now!

vs

22% DIFFERENCE22% -22%

4. Assign Value To Each Token

Great Mens Dress ShoesFind Super Deals On A WideSelection Of Shoes! Shop Now!

Dress Shoes at AcmeExplore The Selection Of ShoesAt Acme Shoes! Shop Now!

vs

22% DIFFERENCE

Token Value

find 0.22

explore -0.22

shoes 0 = .22+ (-.22)

22% -22%

5. Aggregate To Show Total Value Creation● Aggregate to token level● Sum values assigned for positive, neutral, and negative impact● Count positive, negative, and total occurrences

5. Aggregate To Show Total Value Creation● Aggregate to token level● Sum values assigned for positive, neutral, and negative impact● Count positive, negative, and total occurrences

Token Value

find 0.22

explore -0.22

shoes 0.22

shoes -0.22

great 0.22

find 0.16

explore -.16

find -.05

great -.025

5. Aggregate To Show Total Value Creation● Aggregate to token level● Sum values assigned for positive, neutral, and negative impact● Count positive, negative, and total occurrences

Token Value

find 0.22

explore -0.22

shoes 0.22

shoes -0.22

great 0.22

find 0.16

explore -.16

find -.05

great -.025

Σ

Token Value Count

find 0.33 3

great 0.09 2

shoes 0 2

great -.03 2

explore -0.38 2

6. Interpret the Results Carefully

1. Universal: Find tokens that have many occurrences and large impacts○ Negative - consider alternatives for replacement ○ Positive - find ways to increase usage

6. Interpret the Results Carefully

1. Universal: Find tokens that have many occurrences and large impacts○ Negative - consider alternatives for replacement ○ Positive - find ways to increase usage

2. Segments: Drill into tokens with large portions of negative and positive occurrences ○ Account segments○ Phrase segments○ These can be the MOST insightful

6. Interpret the Results Carefully

1. Universal: Find tokens that have many occurrences and large impacts○ Negative - consider alternatives for replacement ○ Positive - find ways to increase usage

2. Segments: Drill into tokens with large portions of negative and positive occurrences ○ Account segments○ Phrase segments○ These can be the MOST insightful

3. Leading Indicator: Positive tokens with small occurrences should be explored further

This is the “Art” of the analysis. These are guidelines but not rules. These datasets open the door for great exploration.

Example: A Flower Company

Example: A Flower Company

Example: A Flower Company

Example: A Flower Company

“Quality” Doesn’t Instill Confidence

“Fresh” is a descriptive explanation of why the product is high quality

Flowers Are “In The Moment”

Same day shipping is important to flower buyers as they are consistently an “in the moment” purchase for someone special or to react to an event

Example: A Flower Company

Use Emotional Action Statements

“Show” is an emotional action statement highlighting the benefit to the flower purchaser

“Choose” is a process oriented action, describing the buying action and not the recipient's reaction

What Can We Do With These Insights?

● Use these insights to drive other business strategy decisions

from product marketing to operations

● Write some pretty darn good ads to increase performance of

your paid search program:

What Can We Do With These Insights?

● Use these insights to drive other business strategy decisions

from product marketing to operations

● Write some pretty darn good ads to increase performance of

your paid search program:

Gorgeous Fresh FlowersShow you care with one of our fieldfresh varieties. Ship the same day!

{Specific Event} FlowersShow you care with one of our fieldfresh varieties. Ship the same day!

Summary - What Did We Learn?

1. Focus on expanding the use of your most positive words

2. Look to minimize negative word usage

3. Look for themes in positive and negative words

4. Quality of analysis depends on the diversity of language that has been

used

Boost Media150 Columbus Ave. San Francisco, CA 94133BoostMedia.com1.800.777.9415

@boostctr

Thank You! Questions?

Rob Lenderman rob.lenderman@boostmedia.com

Bill Sherbybill.sherby@boostmedia.com

Ask your questions in the Q&A Hangout app or post them on the event page.