APPLYING THE SCIENTIFIC METHOD IN

SEO EXPERIMENTING

Branko Rihtman, June 2010

About me – Branko Rihtman SEO Scientist Warzone Experience Becks

http://www.seo-scientist.com

@neyne

http://www.whiteweb.com

1.Define the question

2.Gather information and resources (observe)

3.Form hypothesis

4.Perform experiment and collect data

5.Analyze and interpret data

6.Publish results

7.Retest (frequently done by other scientists)

The Scientific Method

@neyne

Define the question

Usually done after observation of a phenomena

• Sudden change in locations (over-optimization penalty, brand algo, etc.)• Anomalies in SERPs• Someone’s blog post (footer links, second link doesn’t count, etc.)• Social media

• Things to remember:

• You don’t have time to test everything• You don’t have the ability to test everything• What is the ROI of your test ?• Every change you do on your website is

a potential experiment!

"The most exciting phrase to hear in science -the one that heralds new discoveries- is not "Eureka!" but "That's funny...".

(I. Asimov)

@neyne

Gather information and resources • Related blog posts

• Problem: non-standardized terminology• -950 Penalty• Over-optimization penalty• OOP• Unbalanced link equation• Footer link abuse• Sidebar link abuse• Boilerplate linking

• Forums

• Problem: misinterpretation

• Social Media•

@neyne

Perform experiment and collect data

Problems: Lack of control over the system Limited output signal (ranking, title,

snippet) Non-conformity of results No reliable controls

Things to remember:

• Monitor your testing grounds• Testing keyword choice:

• [sdkfsdjwekrwle] – 0 results• [payday loans] – 5.5M results• [translational remedy] – 340K results

• Document everything!

@neyne

SolutionMulti-directional experiments

State A State B Reaction X- Reaction X

change

- change

@neyne

Interpret data and draw conclusions

Does the conclusion agree with expectations ? Does the conclusion have an alternative

explanation ? Does the conclusion agree with other existing

data ? (if not, what was different ?) Bounce the findings off someone

Important: No definite conclusions!!! Only an increasing level of certainty that yourconclusion supports (or defies) the hypothesis.

@neyne

Data Analysis Statistical analysis

Sample sizeSignificance (one and two-tailed tests)Standard DeviationCorrelation

Average is not an end-all measureGet help! Talk to a professional!

People will call you out on crap! (especially if it is self-serving)

.

@neyne

Jean Louis Théodore GéricaultThe Epsom Derby

Avoiding the personal bias

“We observe what we expect to observe, until shown otherwise”Ludwick Fleck

@neyne

Go Social!

Define the question

ForumTwitter

Gather information and resources

Social Media

Publish results

Find an independent blogger you can trust with dataMarket your findings

Interpret data and draw conclusions

Bounce the findings off someone who will not steal themMarket your findings

@neyne

The SEO Testing Secret Ingredient

@neyne

Ethanol

Thank You!

@neyne