If you have ever scheduled an email in Mailchimp, you may have noticed the option ‘let MailChimp optimise send time for maximum engagement’. This is the Chimp’s ‘send time optimisation’ function. Seems like a great idea. But does it work? I ran a couple of tests to find out. Here are the results.
The tests
Two separate A/B split tests were run on a monthly newsletter with over 5,000 subscribers.
Send Time Optimisation did not make a significant difference to engagement rate, measured by unique clicks as a percent of emails delivered. Actually, for both tests, engagement at Mailchimp’s optimal time was lower than the alternative. The results are in the table below.
There are reasons to be cautious about reading too much into these results without further testing. The list is small, and the times being compared differ (because Mailchimp’s optimal time was different for each test). Some more testing is needed.
But, for this list of subscribers at least, the signs are not good: Mailchimp’s send time optimisation does not appear to have resulted in higher engagement.
If you use this Mailchimp function, you should test to see if send time optimisation improves your email engagement. If you find optimisation underperforms, what can you do? Here are three options.
1. Move to another provider
Not all send time optimisations are the same. Other providers offer a better type of optimisation. The key difference is in how the emails are sent:
1) Single ‘blast’ send: Mailchimp’s optimisation delivers your email to all of your subscribers at once. The ‘optimal’ time is calculated by taking a weighted average of your subscribers’ past engagement.
2) Progressive individualised send: This type of optimisation, used by some other email providers, delivers your emails into each subscriber’s inbox at an optimal time for each of them. Your campaign is sent progressively over a period of time, delivering to each subscriber at a time they have tended to engage in the past.
(More detail on how different send time optimisations work is at the end of this article)*
If most of your subscribers engage with your emails at, say, 9am, Mailchimp’s optimisation will send your email to them all at 9am. But 9am will not be the best time for all your subscribers. There will be ‘outliers’ on your list – people who tend to open at 3pm, 5pm, or 10pm. Mailchimp’s single optimal send time will be sub-optimal for these ‘outlying’ subscribers.
Progressive, individualised optimisation delivers your email at 9am for subscribers who prefer 9am, 3pm for subscribers who prefer 3pm, 5pm for subscribers who prefer 5pm, and so on.
Progressive, individualised send time optimisation is clearly preferable. I have run an A/B test on Informz’s progressive send time optimisation: the result was a 24 percent higher engagement rate.
2. Make the most of your past data
If moving from Mailchimp is not feasible, take a look at your past data to see the times of day your subscribers are more likely to be engaged.
Analysing past data can be a bit convoluted for those with a standard Mailchimp account. To analyse engagement by time of day, you’ll need to download all reports as a CSV file and use pivot tables.
Your past data might reflect your audience’s preferences more accurately than Mailchimp’s. That’s because, to calculate its optimal send time, Mailchimp uses data from all emailers, not just you: Mailchimp’s optimal time is not based on engagement with your emails alone.
The 3pm send time in test 2 above was calculated using past data, and appears to have been better than Mailchimp’s recommended time to send.
Needless to say, test, test and don’t stop testing!
3. Demand better from Mailchimp
Why not call on Mailchimp to improve its send time optimisation? They would have a much better product, would be keeping up with their competitors, and would have much happier customers.
I have lodged a request to Mailchimp via their Application Feedback form. So far I haven’t received a response. Why not lodge your own request?
Or maybe blast Mailchimp on their Facebook page to ask them to get their send time optimisation act together.
Come on Mailchimp. You can do it!
———————————————-
* What is Send Time Optimisation?
Mailchimp’s definition: “Send time optimisation looks at the send and click data for your specific list of subscribers and makes a recommendation based on their past engagement times. We use [our] data to pinpoint an ideal time within 24 hours of your selected send date, and send your campaign at that time.”
I see two main problems with the Mailchimp method:
1) It sends to all your list at one time. People who open your emails at an ‘unusual’ time compared to your other subscribers will not get your email delivered at a time that is optimised for them. (Under the alternatives described below, the ‘unusual’ subscriber would receive your email at their ‘unusual’ optimal time)
2) It uses engagement data from all Mailchimp senders, not just you. So the optimal time is not calculated on engagement with your emails alone. Subscribers may engage with different senders’ emails in very different ways and at very different times. For example, subscribers may be more comfortable engaging with nonprofit emails during work hours than they are engaging with commercial emails. An optimal time arrived at by averaging both types may be sub-optimal for each.
Alternative providers
Informz: “Basically, [send time optimisation] means your members will get your mailing based [on their] past opens and engagements. Your message is…sent over a 24 hour period based [on each subscriber’s] engagement history.”
Campaign monitor: “Sends at the time that is best for each subscriber.”
artspolicies.org Christopher Madden 