email split testing the spooky connection between icecream and conversion rates

Email split testingThe spooky connection between icecream

and conversion rates

Some history• First banana split made in

Pennsylvania in 1904• First split test run a few

years later by Claude Hopkins in Michigan

• Coincidence???

Must read:

“Scientific Advertising” by Claude Hopkins

…now out of copyright and freely available!

Email recipients

50%

50%

A (Control email)

B (Experimental email)

1.9%

Successful form fills

3.8%

WINNER

BenefitsContinual improvement for your emails =Continual improvement for your Click-Through-Rate=Increasing amounts of money for free!

Additional bonus side benefits

Resolving awkward disputes in the office+

FUN!

Spot the difference

Spot the difference

What you already dovs

“Best practice”

Some things to tryName of senderImage / no imageHTML / plain textLong / short messageFormat of subject line

Type of greetingCall to action textButton / linkhttp://link.com/ or Click this link?

…and more advancedTime of day Day of weekPersonalise subject lineFraming of an issueNewsletter format / single subject letter

Type of imageUnsubscribe textHumour / guiltCalls to action: 1 or 2?1 or 2 columns?

And now some examples!

Logo redactedLogo redacted

Control Test

Email recipients 3,153 3,687

Donations 10 20

Total amount £455 £920

Average donation £45.5 £46

Conversion rate 0.32% 0.54%

Email sent to announce

letterLetter arrives Email reminder

Direct mail segment

Email sent with donation

ask

Email reminder

Email only segment

DM segment Sent Open rate CTR

Subject: I just sent you something

506 32.08% 9.29%

Subject: Cure a child [redacted]

474 27.64% 4.85%Statistically

significant!Statistically

significant!

Email segment Sent Open rate CTR

Subject: Warning: disturbing image inside…

362 23.81% 6.08%

Subject: Cure a child [redacted]

363 27.64% 7.71%Statistically

insignificant

Day Sent Open rate CTR

Thursday 19,415 24.9% 1.67%

Saturday 23,442 25.1% 2.34%

Statistically

significant!Statistically

significant!Statistically

insignificant

Link type Sent CTRBare URL 4,250 5.22%

Text link Doh! -

Big button 4,260 5.37%

Statistically

insignificant

Link type Sent CTRText link 20,148 1.10%

Big button 20,097 0.74%Statistically

significant!Statistically

significant!

Format Sent CTR

Plain text 4,246 3.44%

HTML 23,442 3.83% Statistically

significant!Statistically

significant!

Donors Sent Open rate CTR

Subject (long): Transcript of our webinar now available

46 45.65% 10.87%

Subject (short): Transcript

47 61.70% 23.40%

Statistically

significant!Statistically

significant!

Non-donors Sent Open rate CTR

Subject (long): Transcript of our webinar now available

4,138 18.37% 3.00%

Subject (short): Transcript

37,529 25.06% 3.35%

Statistically

significant!Statistically

significant!