How to be Careful with Using Forecast Accuracy as a Job Performance Metric
In Demand and Supply Planning, many KPIs hinge upon the idea of accuracy – particularly forecast accuracy. You may spend a lot of time trying to determine the degree of accuracy of your forecast, using various statistical measurements such as MAPE, WMAPE, MAE, RMSE, and others. You may measure accuracy over time and through various parts of your process using a technique such as Forecast Value Add. Consequently, as a forecaster or demand planner, part of your job performance may be judged by the accuracy of your forecast and the improvement in accuracy over time.
In order to set the appropriate metrics you need to know the forecast-ability of your products and the best techniques to use for forecasting. Otherwise, you may be placing your future career prospects in the hands of random fluctuations. Do you know how volatile your product sales are? How much variance can you explain by demand factors versus random vacillations? Will you be judged differently for different products, such as new products or markets versus established ones? You should have a basic understanding of these factors before you embark on personal KPIs based on these metrics.
How do you achieve this understanding? Forecast-ability AnalysisTM.
Forecast-ability Analysis segments your business’ products in various ways to uncover the variability and value in your mix. It is similar to an ABC/XYZ analysis. Forecast-ability Analysis tells you which products or product groups are the most valuable to your company, and how forecast-able they are. By segmenting your product mix in this way, you are better able to understand where you should spend your time and get the most bang for the buck.
Your company management should agree. You shouldn’t waste time trying to improve the statistical forecast for products that do not forecast well. This is where you should apply different techniques such as collaboration for the high value ones and various inventory strategies such as re-order point (ROP) on the low value ones. How that value is determined is up to you and management, but you should be looking at things like revenue and margin.
The process also tells you at what level you should be initiating your forecast. Rather than forecasting initially at the SKU level, perhaps you will get a more accurate forecast and thereby a better supply plan when you forecast at the product family level, and then blow that forecast down to the SKUs. You want to start your forecast at the right level and do it consistently.
In addition, you need to determine how much volatility is forecastable and what is random. Some companies and industries have inherently high volatility. You should create a measurement to determine the difference between randomness and forecast error. One way to do this, as Larry Lapide illustrated in his most recent post, is to first calculate a product’s Mean Absolute Percent Variation (MAPV), which is equal to the average absolute deviation from the historical mean demand divided by the historical mean demand and expressed as a percent. Then compare the MAPE with the MAPV to get an indication of how well one is forecasting compared to random variability. For example, if a product’s MAPV is 40% and its MAPE is 20%, then the forecasting process is explaining half of the inherent variations. Meanwhile if the MAPE is 60%, then the process is making things worse by adding more variation and uncertainty into the forecast.
Moreover, your product’s lifecycle has a large effect on forecast-ability. You need to identify what stage your products are at so you can apply the right forecasting method. New products are inherently more difficult to forecast and therefore need much more attention compared to established products. This is all part of Forecast-ability Analysis.
With Forecast-ability Analysis, you can go to your boss armed with a more accurate understanding of the dynamics of your business. From this point you will be able to create meaningful KPIs and goals for you and your forecasting team. He or she should laude you for this effort in making the company more efficient, and you will have a solid ground for launching a better career going forward.
