- 31 Oct
- Cynthia Butler
What is Performance: How to Measure, How to Compare
Hi everyone! Let me formally introduce myself. I’m Cynthia, Director of Business Intelligence for Mediavine. I work behind the scenes with Eric, Nicole, Heather and Brad (as well as the rest of the amazing Mediavine team).
My role is to analyze new opportunities, and offer performance suggestions and troubleshooting guidance to our product, revenue and publisher support teams. So when I was approached to write something for our publishers, I shockingly decided to talk about performance and analyzing it.
What is Performance?
What do you consider performance? Is it how you’re tracking compared to a goal you set? How you’re performing compared to another website? Week over week? Year over year?
There are so many different ways to monitor and analyze performance. Some can be complicated, others pretty straightforward.
Here’s a simple step-by-step process, and some tips to help analyze your performance for the more complicated questions.
Step 1: What is Your Ultimate Goal?
Do you want to increase revenue? Do you want to hit 1 million sessions? Do you want to increase your traffic from social media? Focus your analysis on what will help you achieve your goal.
Most likely, there will be more than one step to take, and more than one action needed to reach your ultimate goal. This is very typical. Just keep your goal in mind as you monitor results.
Don’t make the ultimate goal unreachable. It’s also easier if you build a number into your goal, e.g. “Increase revenue 10%” vs. “Make more money.”
For the sake of this blog post, we’re setting an ultimate goal for you to follow along with:
Reach 1 million sessions per month.
Growing site traffic is a great goal. (Go you!) We have a number of resources to guide you on the road to achieving your goals.
In this scenario, let’s say we worked on increasing pagespeed and it went up 20%! (Here’s a post on how to increase pagespeed.)
We’ll use this example throughout the steps to help guide you through your own analysis. Now we’ll look at how increasing pagespeed helped us attain our ultimate goal of 1 million sessions per month.
NOTE: You can follow these steps for any goal, so use our example as a jumping off point as you follow the rest of the steps.
Step 2: What Question Do You Want to Answer?
Next, determine the question you want an answer to.
It could be really simple, such as “How many sessions did I have yesterday?” It could be more complex, such as “Has changing pagespeed impacted my session growth?”
Figure out the question before looking at the numbers. This may sound backwards, especially if you only have a specific data set, but there are times that seeing the data could influence your question.
Be specific. Asking something like “Did my revenue go up?” is generic and may not lead to results that can help guide you.
Our Example Question:
“How much have my sessions grown since my pagespeed went up 20%?”
Step 3: Do I Have the Data I Need?
The next step is figuring out if you have the data you need. In certain cases, you might not. If that’s true, there are still ways to infer answers (we’ll go over some of these later).
Focusing on 2-3 things helps you zero in on the question. Too much data can create noise and not provide actionable insights.
Our Example Data:
- Date pagespeed changed (10/1/2019)
- Pagespeed before 10/1/2019
- Pagespeed after 10/1/2019
- Sessions by Day
Step 4: What Do I Want to Compare?
If you’re looking at performance, you probably want to compare it with something, most likely a date range.
Keep in mind that multiple factors might impact the comparison. For example, Sunday performance differs from Monday, January performs differently than June, Google Search traffic performs better than Pinterest, and so on.
Focus on one or two comparisons. Also make sure you give the data time to adjust to any changes you have made.
Our Example Metrics:
- 7 Day period before change (9/22/2019-9/28/2019)
- 7 Day period after change, plus a few weeks (10/13/2019-10/19/2019)
Step 5: What Metrics Will Help Answer My Question?
Again, there are a lot of metrics. Always focus on the question you want to answer. Obvious as it sounds, it’s easy to spiral down into a rabbit hole without that focus.
Make sure the metrics link exactly to your question. CPMs may be affected by session growth, but session growth won’t be impacted by CPMs.
Our Example Metrics:
Step 6: How to I Analyze the Data?
There are many different ways to analyze data. One that is usually very effective is % change. For example, if your RPM changes from $25 to $30, the change would be 20% growth.
To calculate % change, take the number you want to compare ($30 in the above example), subtract the number you’re comparing to ($25), and divide the answer ($5) by the number you’re comparing to ($25):
($30-25) / ($25) = .2 = 20%
Perhaps you had an article go viral and this could account for your session growth. Always look to see if there are outlying factors that could have influenced results.
- Sessions between 9/22 – 9/28: 105,000
- Sessions between 10/13 – 10/19 – 125,000
(125,000 – 105,000) / (105,000) = .19 = 19% session growth
Try to piece out if data is correlated.
Just because a bird flies by your window doesn’t mean your hair is turning gray (that might just be my hair … I blame my kids). Two things that have no relationship can occur simultaneously.
Keep a spreadsheet of any changes you make with plugins, layouts, styles, SEO, etc. with dates. This gives you historical reference points that you may not remember a month from now.
There are times when you don’t have all the data in front of you. For example, if you’ve increased your Google traffic due to SEO improvements, you might also see an increase in RPM using the above tools.
Even if you don’t have Google session RPM broken out, you can use the data in this article to help infer that RPM has increased because Google sessions increased, illustrating the value of improving SEO.
Here are some basic data points to keep in mind when looking through data:
Referral Source Value
Most publishers know that not all traffic is created equal.
You may be surprised at just how much more some referral sources are worth to your site than others, though. Keep that in mind when you look to increase your users.
Below is a graph showing the value of sources compared to one another.
For example, Google traffic is worth 10.1% more than average. If you grow Google traffic, your RPM would be impacted positively, especially compared to increasing your traffic from Instagram.
Country of origin is one of the most important factors that determines the value of a user, as ads perform far better in the U.S. than anywhere else. Here are our top countries and their values compared to each other.
Not unlike geography, device values also lead to a wide variance. Here are value breakouts by device compared to each other:
Try it Yourself
We put together a simple spreadsheet to help you try these new tools yourself:
Using your dashboard data, pull in data from last Monday (New) and compare it to data from the Sunday (Original) before. See if you can figure out which of those two days performs better for your site.
You can use this information to help inform you on when to post updates on social media, add blog posts, etc., to maximize revenue.
Good luck! As always, feel free to leave a comment or email email@example.com with any questions you have along the way.