Imagine you are a data scientist in a global e-commence company and the marketing team is exploring ways to improve email engagement. One of the ideas is to revamp the email design with picture and text instead of existing text only email. However, senior management concerns that a revamped design may lose customers. What should we do ?
Existing email design - text only
New email design - image with text
<aside> 💡 The answer is quite apparent… A/B testing
</aside>
A/B testing is a way to create two versions of a product and compare the performance against each other. After that, we can decide to deploy which version to entire audience group and ultimately drive the business. For a full definition, please reference to this link. “What is A/B testing?” - Optimizely
In this blog, we are going to demonstrate how to deliver A/B testing on email campaign and explore this through answering questions from both business and scientific perspective. A basic understanding of Python programming and statistics would be good but it is not essential.
Please note that all data are fictional in this page
For the first question, we need to come up the definition of “better” from a business perspective. We further break down the question into 3 components.
The other question focuses on how to deliver A/B testing in a scientific way. The question can be broken down into 3 components.