Academy › Email A/B Testing and Analytics Certification › Lesson 1 of 3
Designing a fair test
An A/B test sends two versions to similar groups and compares what happens.
Rules for a fair test
- Change one thing: the subject line, or the button, or the send time. Not all at once.
- Pick the goal first: clicks, replies or sign-ups. Decide before you send.
- Split at random: both groups must be alike. Most email tools do this for you.
- Use enough people: tiny groups give results that are mostly luck.
Good things to test
- Subject lines and preview text.
- One clear call to action against two.
- Short against longer copy.
- Morning against afternoon sends.
Key idea: one change, one goal, random groups and enough people.