My philosophy towards dashboards is to display a minimal amount of data on the page, allowing users to apply segments and interact with charts to drill into the data. I have a lot of data hidden behind the scenes with just a small amount displayed at any time. The dashboard should be visually appealing and contain only relevant, useful information.
Given this philosophy, I apologise every time I show the Performance Diagnostic report as it breaks all those rules. It is, to be quite honest, a data puke of a report. And it is incredibly valuable because of that.
The Performance Diagnostic report contains all your key data in one place. It allows you to spot discrepancies, outliers and exceptions based on internal benchmarks. These need to be investigated further. The purpose of the investigations is to identify actions, informed by data, which will improve your business performance. Take those actions, improve performance, let the website run to collect fresh data and start the process again.
Like the Retail Merchandise report, this approach was taken from my days with Logan Tod. Back then it was just referred to as a data matrix and used sporadically. I thought there was more potential and, after renaming, have used as the core tool for every website analysis since. Having all the data in one place and being able to use internal benchmarks to identify outliers allows me to quickly highlight potential issues and opportunities for further investigation.
Designing the Performance Diagnostic report
The structure of the Performance Diagnostic report is quite simple. Every row is a different metric and every column is a different dimension. All useful metrics and dimensions for your business should be included. It is a lot of numbers on a spreadsheet, truly a data puke. It is also all your relevant/useful data in one place for you to review and see what stands out.
Rows/Metrics
There are a standard set of metrics to include within the report with this then added to by sector/business, depending on the importance and availability of these additional metrics. The report will likely always include:
- Traffic metrics: Users, Sessions, Page Views, % Sessions
- Behavioural metrics: % New Session, Bounce Rate, Page Views per Engaged Session, Duration per Engaged Session
(Engaged session means excluding Bounce sessions that have one page view and zero time recorded on site)
Then you must include your macro conversion metrics and a funnel if one exists. A retailer would always have:
- Key conversion rates: Website Conversion Rate, Ecommerce Conversion Rate, % Create Basket
- Retail Funnel*: % Ecommerce Sessions, Ecommerce => Product, Product => Basket, Basket => Checkout, Checkout => Order
- Sales metrics: Transactions, Revenue, Items, Average Transaction Value, Items per Transaction
Stop here if this is your first Performance Diagnostic report. You can add many more metrics but review this initial version first.
On the other hand, if you are ready to expand your Performance Diagnostic report, list out micro conversion actions or other useful visitor website interactions for your business. Of course, before the data can be included, it will need to be tracked.
For a retailer, these micro conversion & useful visitor interactions could include metrics such as:
- % visitors that view the homepage
- % visitors that interact with product page features e.g. product images, product information
- % visitors that interact with product list page features e.g. apply a filter, change sort order, change number of items displayed
- % visitors that apply a coupon code
- % transactions for each payment method
- % transactions with coupon code applied or that includes discount transactions
- % new/existing customers
- % visitors with a bad experience e.g. failed form validation, failed coupon code
The list is fairly endless so start simple and keep focused on actionable behaviour.
Columns/Dimensions
The dimensions are typically more standard for any business. They include:
- New vs Returning Users
- User Location – Domestic vs International*
- Desktop vs Mobile vs Tablet
- Browser Type for Desktop devices
- Operating System for Mobile & Tablet devices
- Traffic Source
- Entry Point
* The value of User Location depends on the nature of the website. It is powerful when most users are from the same country (potentially up to three countries). You do not want to list out and view performance by every country within this report. If you have a global presence, don’t include these dimensions.
If you capture whether a visitor is a pre-existing customer or a prospect in your analytics, that is a critical dimension to include. With Google Analytics, you can include Age Range and Gender (less accurately). Traffic Sources can be reported on more granularly if that is valuable.
Any other session dimensions that you capture can be used as well. It can also be a segment for a pivoted metric e.g. visitors who create a basket, visitors with session duration of < 1min, 1min to 5min, 5+min. It depends on what is available and what you believe will be useful.
Using the Performance Diagnostic report
What you have now is a massive spreadsheet. My recommended approach is to print this off on multiple A3 pages (it will still be small print but readable). The Digital team should then be locked in a room with this report (and pizza) for likely half a day. The task is to review all numbers, highlighting those that are “unusual” for further investigation.
The website average for each metric can be used as an internal benchmark, allowing you to easily identify dimensions where that metric is high or low. Think through what the cause could be, is it something easily explainable as the nature of people grouped into that dimension (e.g. Desktop Users are more likely to purchase than Mobile Users) or not? If not, what would be the value of improving this metric (if low) or of this metric becoming the new standard (if high)? Does this value justify further investigations?
The desired output from this review session is a list of 10 or more data points for further investigation.
The desired output from the further investigations is a prioritised set of actions that will improve the performance of your business.
In the screenshot provided, the conversion rate for international visitors is half that of domestic visitors. This may not be unexpected, you might expect international visitors to complete the checkout process at a lower rate due to delivery address and payment method issues. However with the Performance Diagnostic, you can see that international visitors are able to complete the checkout process at a rate of 69% vs 56% for domestic visitors, this stage of the funnel is fine.
The biggest issue is instead with international visitors viewing a product page, having indicated they were shopping, with 45% completing this stage, compared to 83% for domestic visitors. Is that worth investigating further? Well, just halving that gap, so the 45% improves to 64% would, everything else staying constant, deliver an extra 848 transactions or £62.5k (for whatever time period this is).
That’s a 5.8% improvement in total revenue. That’s worth some time to research further.
Operational Points
Populating the Performance Diagnostic report can be a challenge. Tracking needs to be in place for all data points you want to include, that is challenge #1. If you are using the free version of Google Analytics, you may then have sampling affecting your ability to extract the data, as segments may be required. Using standard metrics or having them available as goals or custom metrics can eliminate this issue.
Ideally updating the Performance Diagnostic report is an automated process. For the first version this is less important. I even recommend extracting data points one at a time to fill the report in manually the first time. If you find value in this approach, the report can be automated for the future.
A key point is that the Performance Diagnostic is not a daily or weekly report. I recommend reviewing it quarterly, potentially only once every six to twelve months. The purpose is not to look at trends or changes but to identify actions to take. The next view of the report should be after those actions have been implemented and a new set of data has been reviewed. At most, the report is looked at monthly but I feel even that is excessive.
When populating this report, I prefer to use a four week period for data. This should not include any extreme events that can skew the data e.g. Black Friday.
Extending the Performance Diagnostic report
The Performance Diagnostic report can be used for more than a straight review of current performance. It is a great tool when you want to evaluate the impact of a change in circumstance, whether that is a new website, new product feature, marketing campaign or special event e.g. Black Friday, Christmas. Use one copy as a baseline and then a second version containing data after the change or for the period of the special event. The two reports can be compared side by side to see what the impact of the change is across all aspects of how people use the website.
There are times when the behaviour of opposing dimensions within the Performance Diagnostic appear quite different. In these circumstances, it can make sense to create a segmented Performance Diagnostic, so that all data within the report has that segment applied. Good use cases here would be for Customers vs Prospects, Desktop vs Mobile or Domestic vs International visitors.
This approach relies on the ability of users to spot the “interesting” numbers & data points. Isn’t that a job done better by machines and artificial intelligence these days? This is one case where, due to the quantity of data points, the machines may do better. However, they would need to learn the rules of thumb that a human might apply for different dimension comparisons, where you see two different numbers and (given your experience) that makes sense. So, maybe…
In Conclusion
I learnt an early lesson when I started producing these Performance Diagnostic reports, don’t bring them out at the start of meetings with Directors of Ecommerce. I unfortunately lost her attention for most of the meeting as one number after another caught their eye. It did at least demonstrate that these reports had value.
That was even more the case when this report showed a very low conversion rate for a retailer on mobile devices. The client put it down to a bad design of a mobile website but the numbers seemed off more than that would account for. Further investigations led to the discovery of a bug preventing purchases on the mobile website.
The Performance Diagnostic report is ugly and overwhelming. It is also the most powerful tool I use for identifying website issues and opportunities to improve business performance. A template for a detailed Retail Performance Diagnostic report can be downloaded below and I am happy to add more templates for different sectors upon request.
Please get in touch if your company needs a Performance Diagnostic report created and reviewed. It is a key tool used within my Business Performance Audit which, if you need to know the smartest actions to take to improve performance, is the project you need.
