Email Measurement, Part 1: Valuable Email Metrics Tutorial

1.2 Introduction

This is Matt Bailey. In this chapter, we'll be looking at email analytics and measurement. Probably, the most important factor of your email marketing campaign is the ability to immediately measure the success or failure of that campaign. You can immediately compare it to other campaigns to ensure that you are giving the right message to the right people at the right time, and find out immediately what parts of the process may have failed you, or have shown a clear connection to your market. Now the importance of analyzing your campaigns is the importance of understanding your business success.

1.3 Case Study

An example of a business that was not tracking their return on investment through different channels shows a very sad story. In one year, they had revenues of over $500,000. In the second year, their revenues decreased to well below $300,000, which was a total loss of over $200,000 in revenue. Now, what happen? Well, we started looking at the numbers, we found out that they lost significant amounts of visitors, but that didn't tell the whole story. Now this company was doing search engine optimization. When we look at the numbers that were a result of their search engine optimization, we find that there was an increase in visitors dramatically, a 35% increase in visitors. However, that 35% increase in visitors only yielded an additional $1,000 in sales. And so, it wasn't enough to make up for the difference in the marketing and what we found out is that the company had taken money from different budgets and put it into their search engine optimization campaign. Unfortunately, when they took money from other budgets, we realized that they had taken money from their email budget. When we looked at the difference between what they did in email the year prior and the year they decided to pool money out of email put it into SEO, this showed us what had happened. You see their visitors and the amount of purchases went down dramatically. In fact, their email marketing transactions fell almost half. The revenue fell dramatically. This was traced to nearly $80,000 of revenue from one year to the next. It decreased almost half the amount of revenue, because they pooled funds away from their email campaigns and put it into a new marketing initiative. Had they been doing analytics, they would have known that email was one of their primary marketing methods and they would have realized that it was one of their highest return on investment channels and they would not have pulled money out of their email marketing and put it into a new channel development. If anything, their email marketing numbers showed that they should've increased the amount of email marketing that they were doing and they would have received a substantial return because of that. But this is the danger of not analyzing all of your marketing campaigns or analyzing all of your email marketing or specific channels and looking at the entire process, not just the little parts that make up the process.

1.4 Measure the Process

And so when we look at the full picture of email marketing, we need to measure everything. From the quality of the names on your list, how many of those get in to the inbox, how many of those emails are opened from the inbox. How many of those are clicked. How many people go to a conversion. How many of those conversions ultimately result in a sale. And then what is the long term value of receiving that new acquisition, that new customer? And you see, most times in email marketing, people are consumed with looking at how many opens and how many clicks. The main reason is a lot of email service provider data. That's the immediate measurement that you're shown when you log into your interface and the types of reporting that you'll see. It's one of those things that look good on a report. Unfortunately, all it does is it only reports on two steps of a much larger picture of determining what is success. So lets look at the entire funnel from top, down, and all of the measurements that help you understand each part in the process. And how you can measure them and what you should expect to learn from each measurement.

1.5 Delivery Rate

So the first measurement is the quality of the list that you have. And this is the delivery rate measurement. You take the amount of emails sent, subtract the amount of bounce backs, and divide by the number of emails sent, and that gives you the delivery rate. Now, if you want to take an advanced step, Avinash Kaushik suggests that you look specifically at your delivery rate by email recipients. Such as Gmail, Yahoo Mail, or any other types of email programs. This will help you determine if you are getting your message across to subscribers, depending upon how they receive your emails. But this measurement is specifically looking at list quality. This is the first step in the process of making sure we get into the inbox.

1.6 Subscriber Retention Rate

Now a second method of measuring your list quality is your retention rate. You see the first measurement was all about getting into the inbox, and we're subtracting the amount of invalid email addresses. This next measurement, you subtract the amount of invalid email addresses, these are your bounce backs, but you also subtract the number of people that unsubscribe from that mailing. This lets you know your retention rate. So if you start out with 100 emails, and you have three bounce backs, and two unsubscribes, you're now down to a 95% subscriber retention rate. And again, this focuses on the quality of the lists.

1.7 Open Rate

The next step after measuring your quality of the list is measuring the engagement of the visitor from the email. And so, when we start looking at engagement, we want to go from the list to the click. And so you take the number of emails opened in the inbox divided by the number of emails delivered. This would be after your bounce backs and that tells you your Open Rate. The number of emails delivered are the number of emails that are successfully delivered into the inbox. And then from there you take the number of emails opened. This lets you know how many people received your message and took the step of opening it. Now a lot of times this measurement is not going to be accurate. And the main reason is that a lot of email programs offer a preview pane. And people can see the email in the preview pane and never open it. And so as a result this number is going to be very highly skewed, and you can't trust it. It's only a guide to help you understand how many people are looking at it. But not all delivery mechanisms will report the same information, and many people may delete the email simply based off the preview. And so you can't depend on the open rate as a particularly accurate measurement.

1.8 Click to Deliver Rate

>> Now, the second method of engagement that will help you understand a bit more of an accurate measurement, are the number of emails clicked divided by the number of emails delivered. Now, this is a much harder type of measurement than simply the view or the open rate. Because this is dependent upon the recipient to click the email. This means they're taking an action which is trackable. And so, this number tells us the amount of people that engage directly with the email and take the next step. Divide that by the number of emails delivered. And that is your click to deliver rate. Now, one thing that you can do that will additionally give you insights as to how well this email resonated with certain groups is that you can segment demographics. You can look at your click rates, and maybe in different time zones. Maybe different regions. Or even by customer groups. So, if you've got additional data that you can apply to this, this may help you understand which groups clicked on this email more than other groups. And this helps you understand if you're giving the right message to the right people, or at the right time. And so, adding more and more context by time zone, region, a demographic factor, or a specific association within your customers, you add more and more context. The more context you provide to these numbers, the better story you will be able to tell, and the better understanding you will have about this specific group of people that have this in common. And so, that's the second level of an engagement.

1.9 ESP Report

Now, in looking at how an ESP would report this type of information, we can look specifically at a campaign. And this campaign has 192 recipients for this list. Now, we can break down this report and understand a little bit better of how we're going from quality of list and into the inbox and the initial engagement analytics. The open rate for this email is 25%. The list average open rate is about 17%. And the industry average is 12%. Most of your ESPs will include industry rate averages, as well as the average for that specific list. But, as we said open rate's not really one of those things that you can depend on because there are many factors in how emails are displayed for each user depending upon their email program. And so we want to look at the click rate. Now for this email, the click rate was substantially lower than the list average. The list average is about 3% which is better than the industry average but this specific mailing only got 1.1%. And so we know that this information just really didn't seem to resonate compared to previous mailings to this list. Now we can start to get some better information here about the quality of the list. That we have a 95% subscriber retention rate, that has subtracted out the number of bounces and un-subscribes, which you can see in the next column. So we subtract out the bounces, the unsubscribes and that gives us a 95% subscriber retention rate. We can also look here at the amount of abuse reports that have been initiated. These are people that mark the email as spam. Keep an eye on that based on what type of information you're sending out because if you're getting a lot of abuse reports then people don't know who you are or they don't remember signing up for your email. Or they may feel as though they're getting too much from you, and it may need to be investigated, or your ESP will contact you directly because of the amount of abuse reports you're receiving. And so that's how your ESP is going to show the top level of measurement of list quality, getting into the inbox, getting the preview, and then getting the click. These have to do with the top level of the funnel of going from your list quality to the inbox to the open and to the click. So we're about halfway through the funnel.

1.10 Depth of Visit

Now we start moving more into our engagement analytics. And what we want to look at is from the list to the conversion. We want to know how well we are looking at engaging our visitors, even those who may not have taken the conversion action. We want to be sure that we're communicating to them in a way that they are responding to, that we're presenting relevant, valid information that they're interested in, and we can measure the amount of interest. An example of measuring the interest based off clicking the email is the depth of visit. And we can measure the percentage of visits from that email that result in visits of more than a certain number of pages. One way you can start is, look at your average session length, of how many pages people visit. And then target that for your email campaign. Maybe you'll find that your email campaigns are generating more than the average depth of visit. And so based on that, maybe you want to target a higher number or also target some additional actions along with depth of visit. But this lets you know how many people that have clicked on the email, how deep are they going on the site, how long are they staying and engaging with the information you have. Looking at an analytics report for something like this we can see that in number 3 here, the highlighted information, that for this newsletter we had a substantial number of visits. And the average number of pages per visit based off the newsletter was 8.6. Now the overall site average of pages per visit is 2.9. So right away this lets us know that the depth of visit for this specific newsletter presented itself as a very interesting piece of information, which generated a lot of attention. And as a result, people stayed on the site for a long time and they viewed a significantly higher amount of pages than the average visitor. We can also look at the final column of Bounce Rate. The bounce rate of the people that clicked on this newsletter email is only 37%. Compare that to the bounce rate of the site overall, which is 68%, and we can know that we have engaged the visitors that clicked through that newsletter with the content of the site. And so that adds into our overall measurement of that specific email. For that newsletter campaign, we can measure the quality of the list, the depth of the engagement, the number of clicks on the website.

1.11 Engagement Actions

Now then, we can go beyond the depth of visit and start to measure some more specific actions. And here is where you create little mini goals. Mini goals such as watching a video, looking at a store locator, looking at specific content. This could also be called non-revenue behavioral goals. Things that show specific engagement, people looking for specific information, because that shows intent. Intent to learn more about the information you have. Or what they need to know on your website. And so beyond just looking at, well, they looked at eight pages, what specific pages did they do? Did they download something? Did they view something? And here is where you can start to add in much more context of the information people we're interested in based on the hook that you presented in the email and you measure that from the click-through all the way through to what are they doing. Even if it doesn't produce revenue, what actions are showing that this was interesting to them? So much so that they spent time, they did certain actions. And that becomes part of your overall report of success for the campaign.

1.12 Conversion Rate

And then we move into the specific conversion. This is the revenue generating big overall what did we want them to do and how many did it. How many people clicked through the email. And how many people bought a product, became a subscriber, became a lead, whatever it is that is the primary conversion action, and that is your conversion rate. As you can see, as you are going through the funnel, you are presenting a picture of that specific email campaign and the success of reaching people with the information that is important to them. And at every stage you can measure the amount of success from one to the other. And pinpoint areas where you need to improve or where you're doing very well and can leverage that success into other areas. But by measuring the sum total of the process, you can more clearly identify issues with the campaign or opportunities. The key is looking at the entirety of the process, and not just little bits of information from one step to the next.

1.13 Revenue per Email

We can look at the final measurement tier and that's of the entire campaign that you just sent. And the ideal measurement is the revenue per email. How much money did you make from that email and divide that by the number of emails sent, and that becomes the amount of revenue per email. And when you come up with this number, this provides a significant substantial way of measuring this specific message against other similar messages. And so you can test different types of messages, and ultimately have a revenue based measurement to use as a comparison.

1.14 Measuring Campaign Types

And so here's an example of how that happens because you're not just coming up with a revenue per email of all your campaigns. You want to break it down by all campaign types. And so you want to measure all of your Welcome e-mails to new subscribers, or new customers measure all of those together, so that you know what's my revenue per e-mail for that type of campaign. If you run a welcome series then what you want to do is measure your welcome series against other welcome series e-mails. So that you have a specific revenue per email as well as specific targets to hit for each type of campaign. If you're doing a cart abandonment you should see a higher revenue per email, and that's because it's a specific message to a specific person based on a specific action that they have performed on your site, and so it should speak to them differently, and it should produce a higher level of revenue. So you don't want to lump all of these different types of campaigns into one measurement, you want to keep these separate. Measure them accordingly, and use that information to improve each individual campaign type.

1.15 Thank You

This is Matt Bailey presenting email analytics and measurement.

  • Disclaimer
  • PMP, PMI, PMBOK, CAPM, PgMP, PfMP, ACP, PBA, RMP, SP, and OPM3 are registered marks of the Project Management Institute, Inc.

We use cookies on this site for functional and analytical purposes. By using the site, you agree to be cookied and to our Terms of Use. Find out more

Request more information

For individuals
For business
Name*
Email*
Phone Number*
Your Message (Optional)

By proceeding, you agree to our Terms of Use and Privacy Policy

We are looking into your query.
Our consultants will get in touch with you soon.

A Simplilearn representative will get back to you in one business day.

First Name*
Last Name*
Email*
Phone Number*
Company*
Job Title*

By proceeding, you agree to our Terms of Use and Privacy Policy