Today I am going to show you how to use Adobe’s Calculated Metric functions in order to set up a dashboard and monitor when your critical metrics & KPI’s perform outside normal/predicted behaviour.
Currently Adobe Sitecatalyst gives a feature called “anomaly detection”, know to most Google analytic users as “Intelligent alerts”.
Anomaly Detection in Adobe Sitecatalyst and why you don’t really need it
Anomaly detection will spot unpredicted behavior – whether this is positive or negative – based on previous data you hold (you can read more about this here). They way it does this is by calculating the daily total for the selected metric and compare it with the training period using each of the following algorithms:
- Holt Winters Multiplicative (Triple Exponential Smoothing)
- Holt Winters Additive (Triple Exponential Smoothing)
- Holts Trend Corrected (Double Exponential Smoothing)
Each algorithm is applied to determine the algorithm with the smallest Sum of Squared Errors (SSE). The Mean Absolute Percent Error (MAPE) and the current Standard Error are then calculated to make sure that the model is statistically valid.
Theoretically speaking, Triple Exponential Smoothing is ideal for including “seasonality” trends (unlike Double exponential), however Anomaly Detection biggest weakness is it’s 90 day window.
90 days is pretty much insignificant for retail web sites where seasonality and trends plays a big role in terms of performance, hopefully they will add a YoY feature in future releases but in case they don;t that is ok because as long as you have the data available you can do it yourself as shown below.