Make great design decisions that are backed by user research. Tools such as usability sessions, desirability studies, and A/B testing provide clarity.
The founders of Optimizely started at Facebook and left due to their passion for the power of A/B testing. Optimizely helps non-tech savvy users progress.
- How should we lay out the home page?
- Where should we place the call to action?
- Does this image drive more page value?
- Do the users seem engaged?
These questions can all be answered through well-constructed A/B tests.
Well-Constructed A/B Tests
- Identify the project scope
- Isolate macro and micro metrics
- Assess key page elements
- Randomly show two different screens to users
- Carefully evaluate your findings
After thousands of A/B studies that led to millions in revenue, the Optimizely founders had some valuable information to share for optimizing designs.
I. Identify the Project Scope
Are you focused on a home page? What is the project time span? What do we already know about the current product? Who are our users?
Figure out what you need and want to focus in on. Pro tip: start with blue sky goals for your A/B test, and then slowly narrow in on the most important.
II. Isolate Macro and Micro Metrics
Do not be distracted by the first relevant metrics that you identify. If you are trying to reach the tallest mountains, climbing the first hill you see will not help you to get there much faster.
For example, in Clinton’s Haiti fundraiser campaign, the team played with measuring rate of completion or value of each donation. They decided that influencing one positively could negatively influence the other metric. Instead, they focused on average dollars per view.
Vanity metrics can be misleading. For example, you may be excited about the number of users who visit your page, but unless they are moving through key click throughs, these metrics are not helpful or telling.
III. Assess Key Page Elements
Call to action (CTA) and click-throughs (CTT) are crucial for conversions. In order to best guide users to desired actions, consider what may be distracting. Look at the colors, sizes, and position of each information.
Consider what elements should be in a place of prominence and which elements may be the most appealing to your audience.
For example, in the Clinton Haiti fundraiser, the scope was on a single-donation form. They played with whether or not to include an image of the Haiti children and workers.
IV. Randomly show two different screens to users
Once you have identified the elements to play with, create two versions to test with users. Set up the test to randomly show one or the other to each user.
V. Carefully evaluate your findings
Do not run away with immediate findings before carefully evaluating. For example, the Clinton team was surprised when the form with an image on top made the dollar per page metric decrease. But, they realized the form was not viewable “above the fold,” and may be deflecting donators from filling it out.
On the rerun, a form with the image on the side instead of on top, rated better than both of the other designs, as expected.
Overall, A/B tests are simple to use and easy to evaluate. Once you start making more informed decisions with this tool, you won’t want to stop!
Give 50 CLAPS if you enjoy my work so that I will know to write more! Also, PLEASE share any feedback or ideas about impact UX methods and studies 🙂