Forecasting with Temporal Hierarchies
Forecasts are essential for decisions. Decision makers need both forecasts and an understanding of their associated uncertainties. Uncertain forecasts result in more expensive decisions, as one needs to plan for the additional risk. For example, if forecasts are used for inventory decisions, then more accurate forecasts require less safety stock to cover the uncertain demand. This reduces the capital tied to stock and reduces potential waste due to goods perishing or becoming obsolete. This has motivated extensive research in forecasting.
Temporal hierarchies is a new approach to forecasting that is very competitive with existing methods. Its key characteristic is that it models data from multiple perspectives and then combines them into a single holistic model. These different perspectives are produced by temporally aggregating the data, for instance from daily to weekly. This helps reveal additional information that can be modelled.
This research improves on temporal hierarchies. First, it looks at the estimation of forecast uncertainty. Currently, it cannot provide this, limiting its applications. Second, it investigates the impact of improved forecasts on the decision metrics, like reduction of waste, rather than proxies such as forecast accuracy. Although there is an expectation that accurate forecasts lead to better decisions, in practice many other factors influence this. Revealing these can help the design of better forecasts and decisions.
Temporal hierarchies is a new approach to forecasting that is very competitive with existing methods. Its key characteristic is that it models data from multiple perspectives and then combines them into a single holistic model. These different perspectives are produced by temporally aggregating the data, for instance from daily to weekly. This helps reveal additional information that can be modelled.
This research improves on temporal hierarchies. First, it looks at the estimation of forecast uncertainty. Currently, it cannot provide this, limiting its applications. Second, it investigates the impact of improved forecasts on the decision metrics, like reduction of waste, rather than proxies such as forecast accuracy. Although there is an expectation that accurate forecasts lead to better decisions, in practice many other factors influence this. Revealing these can help the design of better forecasts and decisions.
Final report
The project aimed to explore three related research questions in the research area of predictive analytics and specifically the use of Temporal Hierarchy Forecasts (THieF):
(RQ1) obtain a predictive distribution from THieF and benchmark against conventional models,
(RQ2) best translate THieF into (i) single independent, and (ii) aligned joint decisions,
(RQ3) evaluate improvements in forecasting directly on supported decisions, rather than proxy metrics that are common in the literature, such as accuracy.
The sabbatical research has been productive in all three areas. Below we provide a summary of the findings and their usefulness. Details can be found in the associated working papers.
First, we proposed a reformulation of temporal hierarchies that is much more flexible than the standard approach, showing fundamental connections with other streams of the literature. From a theoretical point of view this allows more flexibility in the modelling, shows a direct connection with more fundamental statistical and modelling questions (namely model combinations), and allowed us to provide mathematical proofs and insights into the properties of the revised temporal hierarchy models: (1) we show that forecasts generated in this way will always be more congruent (which we show in a different paper done within the sabbatical that is beneficial for the supported decisions); (2) show that temporal hierarchy forecasts are most likely more accurate than conventionally built forecasts. The likelihood increases under model misspecification, which is the usual case, as the data generating process of real time series is unknown. In practice, we demonstrate gains of about 15% in terms of forecast accuracy over standard temporal hierarchies, and substantive improvements in estimation efficiency of over 90% in typical cases. The latter makes the proposed modelling easily applicable to small datasets that can often be the case in business applications. This work also directly contributes to (RQ1), as the reformulated model can directly benefit from the innovations in the forecast combination literature, where predictive distributions have been extensively investigated. (See paper A.)
Second, we show the value of temporal hierarchies to align decisions and improve them across the lifecycle of products. To support resource allocation decisions for new products, firms need effective short-, medium- and long-term forecasts. Product life cycle curves, traditionally applied on aggregated, lower-frequency data (e.g., monthly), can effectively model adoption patterns. While higher-frequency data (e.g., daily or weekly) naturally provides more data points, and has the potential to improve performance by capturing local stochastic trends, the introduction of high-frequency details can cause estimation issues and harm the long-term predictive performance of the life cycle curves. To address these issues, we employ temporal hierarchies that use optimally-suited models at each aggregation level to extract model structure and combine it to increase predictive accuracy. For example, one could fit a diffusion model at the quarterly level, with a long-term focus, and a seasonal exponential smoothing model at the weekly level, with a short-term focus. In addition, we describe how demand history from previous analogous products could be incorporated to further improve forecast accuracy. We demonstrate the usefulness of the approach on a data set from a large computer manufacturer, and provide insights on how to put these type of models into practice. Overall we find the forecasting performance to improve by 8-23% as compared to a non-hierarchical approach, with even greater improvement when analogous product history is available. Moreover, the proposed methodology does not require changing existing forecasting processes and models, which can be seamlessly incorporated. Being model agnostic, it can take advantage of predictions from models, or experts, and accommodate new forecasting models that the firm may adopt, facilitating ongoing improvements of a firm's forecasting process. This work contributes to both RQ2 and RQ3. (See paper B.)
Given the mathematical similarities between temporal hierarchies and cross-section hierarchical methods, and our previous work in joining the two in a cross-temporal formulation, we explore the usefulness of aligning decision making in the cross-sectional domain as well. We explore that in paper C. Inventory management relies on accurate demand forecasts. Typically, these are univariate forecasts extrapolating patterns from past demand. The disaggregate nature of demand at the Stock Keeping Unit (SKU) level makes the incorporation of external information challenging. Nonetheless, such leading information can be critical to identifying disruptions and changes in the demand dynamics. To address the inventory planning needs of a global manufacturer we propose a methodology that identifies predicatively useful leading indicators at an aggregate demand level, and translates that information to SKU-demand by leveraging on the hierarchical structure of the problem. Therefore, the proposed methodology provides probabilistic forecasts enriched by leading indicator information at SKU-level, as inputs for inventory management. The methodology automatically adjusts the choice of indicators for different required lead times, with some being more informative about the short-term demand dynamics and others for the long-term. We demonstrate the benefits both in the case of backorders and lost-sales, for a variety of lead times. We further benchmark the solution against solely using leading indicators or hierarchical forecasts, demonstrating that the benefits appear primarily by the proposed blending of the modelling approaches. The outcome is demonstratively better forecasts and inventory management for the case company. Additionally, management gains insights into the main drivers of their short and long-term demand, and the ability to adjust inventory replenishment accordingly. The ability to account for diverse macro and market information in operations is paramount for firms with a global reach that face different market conditions across countries. Finally, the transparency of which leading indicators are influencing forecasts of different lead times is conducive to increased forecast trustworthiness. This work contributes to RQ2 and RQ3.
In the first part of the research, we identified a property of temporal hierarchy forecasts, namely that they provide congruent forecasts. Forecasting is a step undertaken in many business decisions. In inventory management, demand forecasts are the cornerstone of inventory decisions, namely how much and when to order. Although it is assumed that accurate demand forecasts lead to effective and efficient inventory decisions, accuracy does not account for important aspects of inventory management. Although managers recognise this, we lack appropriate measures of forecast quality that are connected to inventory decisions, leading to ad-hoc heuristics and practices that lack theoretical or empirical support. We address this limitation by introducing the quantity of forecast congruence, which measures the `jitteriness' of forecasts over the decision period. We investigate its characteristics and connection to accuracy and demonstrate with simulations and a real case on a Fast-Moving Consumer Goods manufacturer that congruence is connected with the volatility of inventory decisions. We show that accounting for congruence in forecast design and selection can achieve favourable inventory performance, without necessitating evaluations that rely on complex inventory simulations. Therefore, the investigation of the effect of forecast congruence, together with the proof of congruency of THieF forecasts (paper A) demonstrates the potential that these forecasts carry for supporting decision-makers in a non-myopic way. (See Paper D).
Finally, given the work already described and the accumulated evidence of the benefit of temporal hierarchical modelling on decision-making, the last piece of work is providing a conceptual framework for managing organisations. The contribution here is not new mathematical innovations or additional empirical evidence, but rather guidelines on how to implement these innovations in practice, accounting for corporate reality, so as to obtain these benefits in practice. (This paper is still being authored, see paper E).
The work carried out during the sabbatical research has direct applications to Sweden’s economy. We have provided evidence of the benefits with various case study companies. Furthermore, the concept of temporal hierarchies has been used by the International Monetary Fund to develop models for long-term liquidity forecasting, demonstrating the potential usefulness for the public sector and regulators. To leverage this the plan is to seek additional funding, both to progress on leading the way in theoretical developments and in applied work to bring this know-how to Sweden’s organisations.
(RQ1) obtain a predictive distribution from THieF and benchmark against conventional models,
(RQ2) best translate THieF into (i) single independent, and (ii) aligned joint decisions,
(RQ3) evaluate improvements in forecasting directly on supported decisions, rather than proxy metrics that are common in the literature, such as accuracy.
The sabbatical research has been productive in all three areas. Below we provide a summary of the findings and their usefulness. Details can be found in the associated working papers.
First, we proposed a reformulation of temporal hierarchies that is much more flexible than the standard approach, showing fundamental connections with other streams of the literature. From a theoretical point of view this allows more flexibility in the modelling, shows a direct connection with more fundamental statistical and modelling questions (namely model combinations), and allowed us to provide mathematical proofs and insights into the properties of the revised temporal hierarchy models: (1) we show that forecasts generated in this way will always be more congruent (which we show in a different paper done within the sabbatical that is beneficial for the supported decisions); (2) show that temporal hierarchy forecasts are most likely more accurate than conventionally built forecasts. The likelihood increases under model misspecification, which is the usual case, as the data generating process of real time series is unknown. In practice, we demonstrate gains of about 15% in terms of forecast accuracy over standard temporal hierarchies, and substantive improvements in estimation efficiency of over 90% in typical cases. The latter makes the proposed modelling easily applicable to small datasets that can often be the case in business applications. This work also directly contributes to (RQ1), as the reformulated model can directly benefit from the innovations in the forecast combination literature, where predictive distributions have been extensively investigated. (See paper A.)
Second, we show the value of temporal hierarchies to align decisions and improve them across the lifecycle of products. To support resource allocation decisions for new products, firms need effective short-, medium- and long-term forecasts. Product life cycle curves, traditionally applied on aggregated, lower-frequency data (e.g., monthly), can effectively model adoption patterns. While higher-frequency data (e.g., daily or weekly) naturally provides more data points, and has the potential to improve performance by capturing local stochastic trends, the introduction of high-frequency details can cause estimation issues and harm the long-term predictive performance of the life cycle curves. To address these issues, we employ temporal hierarchies that use optimally-suited models at each aggregation level to extract model structure and combine it to increase predictive accuracy. For example, one could fit a diffusion model at the quarterly level, with a long-term focus, and a seasonal exponential smoothing model at the weekly level, with a short-term focus. In addition, we describe how demand history from previous analogous products could be incorporated to further improve forecast accuracy. We demonstrate the usefulness of the approach on a data set from a large computer manufacturer, and provide insights on how to put these type of models into practice. Overall we find the forecasting performance to improve by 8-23% as compared to a non-hierarchical approach, with even greater improvement when analogous product history is available. Moreover, the proposed methodology does not require changing existing forecasting processes and models, which can be seamlessly incorporated. Being model agnostic, it can take advantage of predictions from models, or experts, and accommodate new forecasting models that the firm may adopt, facilitating ongoing improvements of a firm's forecasting process. This work contributes to both RQ2 and RQ3. (See paper B.)
Given the mathematical similarities between temporal hierarchies and cross-section hierarchical methods, and our previous work in joining the two in a cross-temporal formulation, we explore the usefulness of aligning decision making in the cross-sectional domain as well. We explore that in paper C. Inventory management relies on accurate demand forecasts. Typically, these are univariate forecasts extrapolating patterns from past demand. The disaggregate nature of demand at the Stock Keeping Unit (SKU) level makes the incorporation of external information challenging. Nonetheless, such leading information can be critical to identifying disruptions and changes in the demand dynamics. To address the inventory planning needs of a global manufacturer we propose a methodology that identifies predicatively useful leading indicators at an aggregate demand level, and translates that information to SKU-demand by leveraging on the hierarchical structure of the problem. Therefore, the proposed methodology provides probabilistic forecasts enriched by leading indicator information at SKU-level, as inputs for inventory management. The methodology automatically adjusts the choice of indicators for different required lead times, with some being more informative about the short-term demand dynamics and others for the long-term. We demonstrate the benefits both in the case of backorders and lost-sales, for a variety of lead times. We further benchmark the solution against solely using leading indicators or hierarchical forecasts, demonstrating that the benefits appear primarily by the proposed blending of the modelling approaches. The outcome is demonstratively better forecasts and inventory management for the case company. Additionally, management gains insights into the main drivers of their short and long-term demand, and the ability to adjust inventory replenishment accordingly. The ability to account for diverse macro and market information in operations is paramount for firms with a global reach that face different market conditions across countries. Finally, the transparency of which leading indicators are influencing forecasts of different lead times is conducive to increased forecast trustworthiness. This work contributes to RQ2 and RQ3.
In the first part of the research, we identified a property of temporal hierarchy forecasts, namely that they provide congruent forecasts. Forecasting is a step undertaken in many business decisions. In inventory management, demand forecasts are the cornerstone of inventory decisions, namely how much and when to order. Although it is assumed that accurate demand forecasts lead to effective and efficient inventory decisions, accuracy does not account for important aspects of inventory management. Although managers recognise this, we lack appropriate measures of forecast quality that are connected to inventory decisions, leading to ad-hoc heuristics and practices that lack theoretical or empirical support. We address this limitation by introducing the quantity of forecast congruence, which measures the `jitteriness' of forecasts over the decision period. We investigate its characteristics and connection to accuracy and demonstrate with simulations and a real case on a Fast-Moving Consumer Goods manufacturer that congruence is connected with the volatility of inventory decisions. We show that accounting for congruence in forecast design and selection can achieve favourable inventory performance, without necessitating evaluations that rely on complex inventory simulations. Therefore, the investigation of the effect of forecast congruence, together with the proof of congruency of THieF forecasts (paper A) demonstrates the potential that these forecasts carry for supporting decision-makers in a non-myopic way. (See Paper D).
Finally, given the work already described and the accumulated evidence of the benefit of temporal hierarchical modelling on decision-making, the last piece of work is providing a conceptual framework for managing organisations. The contribution here is not new mathematical innovations or additional empirical evidence, but rather guidelines on how to implement these innovations in practice, accounting for corporate reality, so as to obtain these benefits in practice. (This paper is still being authored, see paper E).
The work carried out during the sabbatical research has direct applications to Sweden’s economy. We have provided evidence of the benefits with various case study companies. Furthermore, the concept of temporal hierarchies has been used by the International Monetary Fund to develop models for long-term liquidity forecasting, demonstrating the potential usefulness for the public sector and regulators. To leverage this the plan is to seek additional funding, both to progress on leading the way in theoretical developments and in applied work to bring this know-how to Sweden’s organisations.