Lately, I’ve been experimenting and learning about AB testing!
In this post, I’ll walk through how simple it is to do AB testing with Google Analytics and a bit of custom code. I'll show you an example of a successful AB test and some things I learned.
I don’t use Google Optimize, Mixpanel, etc as I found them to be way too clunky and hard to figure out.
The original feature
The AB test here has to do with email collection.
Here’s the original feature, or the "A" in the AB test:
The new thing we want to try
Instead of just asking to join the newsletter, I wanted to test out a "content upgrade" where the user can get access to 1,000+ business ideas:
The new test, or bucket B
These changes were made to a few other pages/places, as well as some UI changes, which you can see here.
Go to Admin -> Custom Dimensions and create a dimension for every AB test you want to try:
Admin -> Custom Dimensions
Server-side code
For every request made to our server, I run a simple method that puts the user into the random AB test bucket.
Only two buckets for this ab test: 'control' (A) and 'upsell_widget' (B)
Pick a random bucket and assign a cookie (as long as it's not already there)
Client-side code
Now, on the client-side, you need to send this data over to Google Analytics for every pageview or event.
Keep in mind, you must send the mapping as well as the values. Took me forever to figure this out.
For me, it looks like something like this this:
Submitting pageview data to Google Analytics (this is for 5 different AB tests)
The last thing is to change your code based on the test that each user is in. This is as simple as looking at what cookie they have:
This is pseudocode
And that’s it!
Now, set an event in your calendar to review the results a few weeks later.
The Results
Now, let’s dive into the results of the AB test.
In Google Analytics, go to Acquisition -> All Traffic -> Channels.
Right above the grid, look for the dropdown of Secondary Dimension, and search for your AB test:
Now, you can see how your AB test performs and cut this data however you’d like.
For this AB test, I'll use Google Analytics goals because I'm tracking email collection conversion rate, but you might be testing something else such as time on site, pageviews, etc.
Since the AB test is a custom dimension inside GA, you can do pretty much anything!
I added all of these up inside Google Sheets and here are the results:
