A/B Testing the Instapage Homepage Part 2: Images vs. MP4 Animations

At Instapage we are big advocates of A/B testing; our team frequently runs tests on our PPC ads, post-click landing pages, emails, and even our homepage. In fact, you can review the details of a previous homepage A/B test in this article.

We recently performed another homepage A/B test to see whether using mp4 animations instead of static images would have a significant impact on conversions. Our initial hypothesis was that using animations versus static images would not increase conversions.

Before we get into the results, let’s set the stage for the experiment.

What was the purpose of A/B testing our homepage?

The main purpose was to measure if animations have an impact on our sign up rate, session duration, page views, and bounce rate. Or, do they increase the page’s visual appeal in conveying our marketing message to potential customers.

The Instapage design team worked hard on creating MP4s from static images for the A/B test, because even though we have a dedicated design team they have a finite number of working hours available.

Our team went through a lengthy review session to run system checks to ensure that the traffic was split evenly between the variations, and to reduce flicker and page load times for the visitors who viewed the animations page variation.

Here’s a point-by-point breakdown of the test:

• Purpose: To understand whether animations would positively impact our homepage visual story and engage more visitors.
• Goal: Increase user signups
• Total sessions: 100,000+
• Traffic split: 50/50

What was the difference between the variations?

Starting from the top of the page, let’s compare the original (static images) versus the variation (animations).

Average conversion rate section

The average conversion rate section of the homepage showcases the difference between the average PPC conversion rate for brands who use other solutions and the average conversion rate for Instapage users.

While the static image shows the final conversion rate value, the animation helps visitors visualize how the conversion rate for Instapage users goes up from 0-22%.

Original image:

MP4 animation:

Own Your Post-click landing page

Instapage is the only post-click landing page platform built to work with Google. You can easily integrate with Google Ads and Analytics and then pass the data collected to your marketing automation or CRM system. Combine that with heat maps and A/B testing, and you have an enhanced post-click landing page. Plus, we recently announced post-click landing page variation tracking in Google Analytics.

The static image here shows the Analytics options that Instapage users can choose from, including Google Ads, Analytics, Google Tag Manager, and the Meta Pixel:

The variation shows how easy it is to integrate Instapage with Google Ads in just a few clicks:

Build Personalized post-click landing pages at Scale

The section highlights how easy it is to create message matched post-click landing pages at scale from the Instapage builder.

The original version shows a snapshot of the Instapage builder with a preview of all the options available to users:

The animation shows the creation of scalable post-click landing pages made possible with Instapage:

Speed Up Your Review Process

The original image in this section highlights a conversation hot spot in the Collaboration Solution. Here, team members can communicate with each other in real-time during the design review process:

Compare that to the animation variation where it mimics how the hot spot feature works:

Integrate with Your Marketing Stack

The original image shows a variety of marketing software that Instapage integrates with:

The variation depicts the same integration marketing platforms but with a revolving orbit:

Optimize Your Pages for Even More Conversions

The original image depicts how easy it is to run A/B tests and heat maps on Instapage post-click landing pages:

The variation showcases the increasing conversions for one page variation versus the other:

Make Attribution Easy

Instapage provides users an easy attribution solution, by allowing you to connect ad networks with drop-in pixel tracking and accessing all the metrics from within the Instapage platform.

The original image shows some of the metrics you can view with Instapage analytics:

Meanwhile, the animation gives a more detailed picture of the analytics options available to Instapage users:

The final data: Which variation won?

Conversions

A difference of 100 conversions is all that separates the static image from the mp4 variation:

Session duration

Both variations followed the same general trend for homepage session duration:

Conversion rate

Same with the conversion rate for both variations:

There was no clear winner, as the total conversions, session duration, and conversion rate all indicate. Our original hypothesis proved to be correct in that using animations in place of static images did not have a significant impact on conversion rates.

While making design decisions, it is important to take into account the impact the design has on time and resources as they directly relate to revenue and conversions. Go for design trends that don’t just add visual appeal but also positively affect conversions, and provide a good user experience.

What you can learn from our homepage A/B test

The purpose of A/B testing is not to see one variation overpower the other, so not every test will conclude a clear winner. Rather, the purpose of testing is to determine whether your original hypothesis is correct or incorrect based on the resulting data.

Brainstorm some ideas you feel would produce a noticeable effect on your campaigns and analyze the results. Only then can you decide how to proceed further, sign up for an Instapage Enterprise demo today.