The Digital Analytics weekly workshop on the 4th June 2021 was on the topic of the Performance Diagnostic report. This is an ugly data puke of a report and all attendees needed to confirm they had been given fair warning that looking at the report may harm their eyes…
The hidden beauty of the Performance Diagnostic is it contains all your relevant data in one place for review. You create your own internal benchmarks to identify outliers, where the website is under/overperforming. These need to be investigated further to determine actions your can take that will improve performance.
The video below is the recording from this workshop. During the talk, I:
- covered the concept o f the Performance Diagnostic
- gave an example of how identify interesting numbers within the report
- talked through how to design and build your own Performance Diagnostic
- shared template to help in the design and build process
- talked through the process for using a Performance Diagnostic
- answered a range of questions from the audience
Beyond the video, there are various resources being shared in this post. My answers to audience questions are provided below the video. And there is a button at the bottom of the screen to download a PDF of the deck. Here are links to
- the original blog post on the Performance Diagnostic
- a Google Sheets template and sample partial Performance Diagnostic for the Google Merchandise store
- a GA custom report as an example of how data can be extracted
To get notifications of future workshop topics, please connect with me on LinkedIn or follow me on Twitter.
Audience Questions
Can an API populate this spreadsheet?
Definitely and is the recommended approach, especially if you are going to use the report more than once. I have used Next Analytics a lot myself but this will equally work with Supermetrics, Analytics Edge, the Google Sheets add-in or anything else similar. It could even pull the data in from BigQuery (if using the new GA4 for instance).
Who is the target audience for the performance diagnostic review meeting?
This should ideally be a team of people, not just the analyst/s. The people who understand the marketing channels or the product features the best need to be in the room to identify if metrics are weird or not.
Keep in mind though that this meeting is intended to highlight all the weird numbers, not to provide the answers/actions. Don’t get bogged down in the detail, keep the focus on reviewing the entire report.
How do you stop people from saying – how does that compare to last month?
That’s a difficult one. It basically comes back to using this report to compare numbers against internal benchmarks, not previous periods. So it is a bit of discipline being enforced by whoever leads the review.
For anything flagged up as unusual, one of the first points of investigation will be to trend that metric – so that month on month comparison will still happen.
Anomaly detection should be used to identify metrics that changes week on week or month on month, not the Performance Diagnostic.
Would you use this for benchmarking a site migration?
Definitely. My original use case for this report was an overall performance review but, when creating a course for Product Managers once, I realised it could be used for evaluating any significant website change. It simply means you are using a whole range of data points to evaluate performance rather than a limited set of metrics.
This can be used for a site migration, the launch of a new website feature, the launch of a new marketing campaign or to understand visitor behaviour around a specific event like Black Friday. Create an original or baseline Performance Diagnostic and then one for post change – then simply compare the two.
Regarding the last comparisons, is this again the same process of exporting the data before and after the change, right?
Yes it is – and really important to note that the first Performance Diagnostic is intended as the baseline, what would the data be if nothing had changed. So it is important to match the days of the week if the reviewing a change with less than 7 days of data, to not include skewed data in the baseline and to include a reasonable date range e.g. four weeks.
What’s the biggest impact you’ve been able to make for a client with this report?
I once used this report to identify that mobile conversion rates were too low for a UK retailer (they should be ~ half of desktop). Digging further identified one bug early in the customer journey but then, that visitors could not purchase on the mobile website. The quick fix was to remove this website version, allow only the desktop version and revenue immediately jumped.
How do you build intuition for numbers that look “off”? I assume that just takes time?
Unfortunately yes, this one does just take time. The more websites and data you look at, the more you develop a sense that a data point feels wrong and should be looked at in more detail.
There would be statistics you could use (more than I do) which may help shortcut the process but the intuition builds over time.
Any More Questions?
Get in touch in the comments with any questions you have yourself.
This is a tool I recommend for every business. Reach out if you would like my help to create your own Performance Diagnostic report or to perform the analysis of the data within the report. This report is the core of my Business Performance Audit product offering, designed for companies with a lot of data who aren’t sure where the value is hidden.
Don’t forget to follow along for future Digital Analytics weekly workshops – held in your lunch hour if you are based in Europe (1pm CEST).
