A/B Testing

A/B Testing

Starting with version 2.0.29, MailWizz supports A/B Testing for email campaigns.
In simple terms, A/B Tests will allow your email campaign to get slight variations for some of its properties, then while the campaign is running, see which of the variation perform better, and send the best performing variation to the rest of your subscribers after a certain condition is met.

Since it takes quite some time to figure out how a campaign performs, it makes sense to implement A/B Testing for long running campaigns only, like Autoresponder Campaigns. For Regular Campaigns, it doesn't make much sense to use A/B Testing because most likely, those campaigns will finish sending before we have a chance to find a winner for the A/B Test.

That being said, when creating a Autoresponder Campaign, in the Setup step, you will find a new area, specific to A/B Testing where you can create your A/B Test. When creating a A/B Test, you can decide how the test winner will be determined, be it by the number of opens and/or after a certain number of days. This is a very powerful feature which allow for simple use case scenarios but also for complex ones.
Of course, only deciding how the test winner will be determined doesn't make a valid test, we also need to specify the properties we test against and also the variations for those properties.

A very simple example, and one of the most common, if not the most common A/B Test type, would be a campaign where we want to test a few subject lines and see which one performs better after x number of opens, then when we find which one performs better, use that subject line for the rest of our list subscribers for that particular campaign.
So let's say we have a campaign targeting 10k subscribers, and we have 3 subject variations and we want to see which one performs better after 100 opens, and then use that subject line for the rest of the emails in the campaign.
In MailWizz, we can define this A/B Test like shown below:

Once either of the campaign variation for these 3 subject lines will reach 100 opens, we will decide our winner and then we will continue using the subject that won for the rest of the emails in that campaign.

In your dashboard, you can see the results, in the campaign overview, as follows:

Additionally, you can see all the opens / clicks / Unsubscribes / Complaints and Bounces that have happened in the campaign after the test has been completed(after the winner has been decided).