10 powerful ways to find ideas worth A/B testing

“The discovery of what matters is important. If you figure it out, you know WHAT to optimize, and WHERE. One huge step closer to more winning tests with bigger impact.” — Peep Laja, Conversion XL

Are you ready to take your testing efforts to the next level?

When I made my first foray into A/B testing, I wasted a lot of time throwing ideas at the wall, hoping they would stick. The end result? A lot of failed tests and zero value driven for the organization I was running tests for. Even worse, I had a hard time drawing insights and learnings from these tests.

In talking with a number of conversion optimizers, this seems to be a pretty common occurrence. I’d be a rich man if I had a dime for every “What should I test?” or “How can I come up with good A/B test ideas?” I’ve heard. I attended CXL Live the past two years and each time, this was a very popular question asked by fellow attendees.

The good news is, you can make a course correction quickly and start cranking out ideas that will have a greater impact on your business. But first, let’s dive into what we’re trying to accomplish with A/B testing.

Where should you start?

Before you even entertain the idea of running A/B tests, let’s start by reframing the question, “How can I find ideas worth A/B testing?”

There is a better way to ask this question:

“What are my prospects’ most pressing problems and what is the best way I can solve them?”

In order to answer these questions, we have to dig into our customers heads and observe the obstacles that stand in their way, as opposed to making assumptions. While it is true that you’ll have to form a hypothesis, which is an assumption of sorts, you’ll have something more concrete to serve as the foundation to your hypothesis.

At the end of the day, observations > assumptions.

How to Hone in on Real Problems to Fuel Test Ideas

There are are number of methods and frameworks for formulating A/B test ideas. No single method is best — you’ll end up having to use a combination of these to get the ideas flowing. Here are 10 ways I’ve gathered ideas to launch hundreds of A/B tests over the last few years, some of which have generated hundreds of thousands of dollars in top-line revenue.

1. Perform a Thorough Website Review (AKA Heuristic Analysis).

When you conduct a website review, you’ll want to comb through every page and every step of your website’s funnel to identify the key factors that influence how well each of your pages convert. This includes examining the relevance, clarity, value, distractions, and friction that your users face.

Here’s each of those factors expanded:

Relevance: How relevant is the page to your target market? Are you meeting your user’s expectations?

Clarity: Is the interface (both content + design) straightforward and easy to use?

Value: Does the page offer value to your users? Is the value being expressed in a way that the user can understand?

Friction: Are there any glaring distractions, elements, or copy which might cause confusion, doubt, or uncertainty? Are there complexities to the buying process which could be simpler?

Distractions: Are you handling any potential objections your users may have? Are there potential turn-offs that would make a user reluctant to move forward through the funnel?

2. Conduct a Technical Audit.

If you don’t have a developer handy, you’ll need one for this step. Technical analysis entails diving into the inner-workings of your website for bugs and user experience issues, by way of cross-browser testing, cross-device testing, etc.

Believe it or not, this phase can yield significant returns. Bugs are true conversion killers and addressing these “show stoppers” can steer your ship in the right direction.

Now, your website doesn’t have to work across every single browser and device. In fact, doing so would be cost-prohibitive. Instead, opt for using your analytics platform to identify the most popular browsers that visitors are using to access your website. Focus on these.

If you see sizable differences in conversion rate across different browsers and/or browser versions, you’ll want to dig deeper to see if browser issues are the root case.

Another important area of focus during your technical audit is site speed. A website that loads quickly for visitors, regardless of the device being used, will generate significantly more returns than the same site with abysmal load times.

Significant strides can be made if you simply focus on these things:

• Compress your images
• Compress or minify CSS/JS
• Utilize a CDN for DNS, hosting and serving images, and CSS/JS assets
• Remove unused CSS/jQuery libraries
• Remove unused third-party marketing pixels
• Make use of browser caching

3. Look to Your Analytics Platform and Your Conversion Funnel.

While this isn’t a primer on mastering Google Analytics, it’s worth mentioning that you can identify conversion issues and formulate test ideas by paying attention to key metrics periodically.

Use your analytics software to slice and dice conversion rates by browser, device type, etc. If there are large differences, chances are that you may have stumbled upon a few areas where usability issues are present.

Here are some key metrics you want to pay attention to:

• Number of visits
• Number of views
• Average time on the website
• Average time on the page
• Bounce rate
• Exit rate
• Conversion rate
• Total number of conversions
• Revenue per visitor
• Average order value
• Revenue (ROI)

Keep an eye on your entire conversion funnel too. If you observe users dropping out of a certain step in the funnel, you should look into what might be broken, confusing, or missing at this step.

4. Leverage Mouse Tracking Software.

Mouse tracking software is abundant today and makes it much easier to see your website through your users’ own eyes. Two tools that come to mind are Hotjar and FullStory — both of which I’ve used extensively… and they have returned multiples of what I’ve paid for them.

Session Recordings (AKA Session Replays)

Think of session recordings like a DVR for your website, where you can collect and record batches of live browsing sessions from your users. When they hit your site, their mouse movement and interactions are rendered into a video, so you can see how they browse through each page of your website.

Both Hotjar and FullStory do this well, although FullStory focuses solely on session replays and has a deeper feature set in this area. Hotjar is more of an all-in-one solution for analyzing user behavior, which includes heatmaps, scroll maps, on-site polls, feedback surveys, funnel analysis, etc.

Session recordings can help you to:

• Identify potential UX/technical issues
• Find areas in your funnel which may be confusing or distracting
• Pinpoint the greatest areas of focus for your users

Heatmaps

Heatmaps provide a visual way to track and analyze user clicks and interactions on a particular page. These heatmaps can be used to draw insights on potential user motivators and desire on each page.

Scroll Maps

Scroll maps are very similar to heatmaps in that they show a rendering of user behavior on a page, but instead of showing clicks, it shows scroll distance and the “average fold” across device types.

The true value of scroll maps lies in the fact that you can assess what your users see first and how far they are scrolling.

If you have important content or your call-to-action is buried further down the page, then you’ll be able to use this data to identify where you need to adjust motivators and copy that carries users further towards the end goal.

5. Take Advantage of User Testing.

User testing allows you to observe real people using your website. You lay out tasks for them to accomplish and while they attempt to complete these tasks, they talk through their experience.

Here’s what makes user testing so valuable:

• You can see where test participants are getting stuck
• You can gauge what they are feeling as they carry out each task
• You can see and hear what frustrates or confuses them

User testing allows you to “get out of your own head” and dig into what your users are thinking. And while user testing is great for identifying usability issues, it doesn’t just stop there… it‘s also a powerful way to test a new product, service, or feature too.

6. Run On-Site User Feedback Polls.

An on-site user feedback poll is a quick and easy way to gather feedback on certain aspects of your website, your products, your services, and your selling process.

Remember to choose the number of questions and answer choices wisely. Too few choices and you may not get anything meaningful from the poll, but too many and you’ll find it difficult to collect responses from your users. It’s a balancing act between the two!

7. Send Out Post-Purchase Customer Surveys.

What better way to identify issues with your website, straight from the mouths of your customers? Ask a new customer how they felt during the purchase process, what went well, and what didn’t go so well. Use the responses you receive from these customers to formulate new A/B test ideas.

8. Talk to Customer-Facing Employees Across Your Organization.

Does your organization have a call center, live chat team, or sales department? Talk with these folks as quickly as you can!

Employees that are on the front lines are generally among the first to hear about initial product or service impressions, expectations, and frustrations. Talking with these fellow coworkers is a powerful way to discover real problems that you can use A/B testing to solve.

9. Express the Importance of A/B Testing and Encourage the “Testing Mindset” Across the Organization.

This is a big one! If you can build in a culture of testing and experimentation across your organization, you’ll accomplish a number of wonderful things:

• You’ll move away from “gut feels” and move closer towards making data-driven decisions
• You’ll empower other coworkers to think about solving problems in new ways
• You’ll save money on costly projects or initiatives that generate little value for the organization
• You’ll start to separate yourself from competitors who aren’t doing this

Creating a testing culture isn’t easy, especially if your company is very traditional and siloed. It takes time, but if you explain the value behind testing and how powerful it can be, you’ll be better for it. Remind the big wigs that it isn’t above proving who’s right and who’s wrong… it’s about getting into your customers’ heads, so that they’ll open their wallets more!

10. Study What Your Competitors Are Doing…

…but tread lightly when it comes to following their lead. You can use what your competitors are doing (in terms of UX, UI, and copy) as a seed for A/B tests, but understand that your mileage may vary. You likely won’t know if it’s performing well or impacting their business negatively, so don’t draw conclusions and make decisions without testing properly first!

I once experimented with a countdown ticker for an e-commerce store to gain a better understanding on how creating urgency would impact conversions during a few big sales. Since the ticker was being served via our testing platform, Adobe Target, I didn’t bother to add a fallback once the ticker ran out of time. Instead, I simply shut down the experiment just before the end of the promotion. Crisis avoided!

Fast forward a few months… and you’ll never believe what I found on a competitor’s website… that very same countdown ticker that I wrote from scratch. Only… they had forgotten to accommodate for what happened after the time was up. The countdown ticker showed -8 hours remaining… and it was still live on their website, meaning it had been expired for 8 hours. Can you imagine what a headache this might have been for customer service or sales? I’ll bet they don’t make that same mistake again!

The same can be said for following “best practices.” Best practices carry no guarantee when it comes to effectiveness … they simply serve as a starting point on what works for some businesses, but they may not work for yours.

Conclusion

Coming up with A/B test ideas is easy… but finding compelling a/b test ideas will take time and a lot of prep work on your behalf. Rest assured, this isn’t time wasted. By setting aside time to execute on these 10 methods, you’ll ensure your efforts are channeled towards creating, launching, and analyzing tests with meaningful impact.