Economic Forecasts: Themenheft Heft 1/Bd. 231 (2011) Jahrbücher für Nationalökonomie und Statistik 9783110510843, 9783828205352

Citation preview

Economic Forecasts

Herausgegeben von Ralf Brüggemann Winfried Pohlmeier Werner Smolny

Mit Beiträgen von Carstensen, Kai, Munich D o h m , Roland, Essen Halbleib, Roxana, Bruxelles Hofer, Helmut, Wien Krüger, Fabian, Konstanz Lenza, Michele, Frankfurt a. M . Lütkepohl, Helmut, Firenze Mokinski, Frieder, Konstanz Pohlmeier, Winfried, Konstanz

Lucius &c Lucius • Stuttgart 2011

Schmidt, Christoph M., Essen Schmidt, Torsten, Essen Schumacher, Christian, Frankfurt a . M . Voev, Valeri, Aarhus Warmedinger, Thomas, Frankfurt a . M . Weyerstrass, Klaus, Wien Wohlrabe, Klaus, Munich Ziegler, Christina, Leipzig

Anschriften der Herausgeber des Themenheftes Professor Dr. Ralf Brüggemann Universität Konstanz 7 8 4 5 7 Konstanz, Germany [email protected] Professor Dr. Winfried Pohlmeier Universität Konstanz 7 8 4 5 7 Konstanz, Germany [email protected] Professor Dr. Werner Smolny Universität Ulm Inst, für Wirtschaftspolitik Helmholtzstr. 2 0 8 9 0 8 1 Ulm, Germany [email protected]

Bibliografische Information der Deutschen Nationalbibliothek Die Deutsche Nationalbibliothek verzeichnet diese Publikation in der Deutschen Nationalbibliografie; detaillierte bibliografische Daten sind im Internet über http://dnb.d-nb.de abrufbar ISBN 9 7 8 - 3 - 8 2 8 2 - 0 5 3 5 - 2

© Lucius Sc Lucius Verlagsgesellschaft mbH • Stuttgart - 2 0 1 1 Gerokstraße 5 1 , 7 9 1 8 4 Stuttgart, Germany Das Werk einschließlich aller seiner Teile ist urheberrechtlich geschützt. Jede Verwertung außerhalb der engen Grenzen des Urheberrechtsgesetzes ist ohne Zustimmung des Verlags unzulässig und strafbar. Das gilt insbesondere für Vervielfältigungen, Übersetzungen und Mikroverfilmungen sowie die Einspeicherung und Verarbeitung in elektronischen Systemen.

Satz: Mitterweger & Partner Kommunikationsgesellschaft mbH, Plankstadt Druck und Bindung: Neumann Druck, Heidelberg Printed in Germany

Jahrbücher f. Nationalökonomie u. Statistik (Lucius & Lucius, Stuttgart 2011) Bd. (Vol.) 231/1

Inhalt / Contents Guest Editorial

5-8

Abhandlungen / Original Papers Dohm, Roland, Christoph M. Schmidt, Information or Institution? O n the Determinants of Forecast Accuracy Schumacher, Christian, Forecasting with Factor Models Estimated on Large Datasets: A Review of the Recent Literature and Evidence for German GDP Lenza, Michele, Thomas Warmedinger, A Factor Model for Euro-area Short-term Inflation Analysis Krüger, Fabian, Frieder Mokinski, Winfried Pohlmeier, Combining Survey Forecasts and Time Series Models: The Case of the Euribor Carstensen, Kai, Klaus Wohlrabe, Christina Ziegler, Predictive Ability of Business Cycle Indicators under Test Lütkepohl, Helmut, Forecasting Nonlinear Aggregates and Aggregates with Time-varying Weights Halbleib, Roxana, Valeri Voev, Forecasting Multivariate Volatility using the VARFIMA Model on Realized Covariance Cholesky Factors Hofer, Helmut, Torsten Schmidt, Klaus Weyerstrass, Practice and Prospects of Medium-term Economic Forecasting

9-27

28-49 50-62 63-81 82-106 107-133 134-152 153-171

Buchbesprechungen / Book Reviews Skedinger, Per, Employment Protection Legislation - Evolution, Effects, Winners and Losers Vogel, Harold L., Financial Market Bubbles and Crashes Wickstrem, Bengt-Arne (Hrsg.), Finanzpolitik und Unternehmensentscheidung

172 173 175

Jahrbücher f. Nationalökonomie u. Statistik (Lucius & Lucius, Stuttgart 2011) Bd. (Vol.) 231/1

Special Issue on Economic Forecasts: Guest Editorial Forecasts guide decisions in all areas of economics and finance. Economic policy makers base their decisions on business cycle forecasts, investment decisions of firms are based on demand forecasts, and portfolio managers try to outperform the market based on financial market forecasts. Forecasts extract relevant information from the past and help to reduce the inherent uncertainty of the future. The recent years have witnessed a large increase in the use and publication of forecasts in different fields of economics and finance. The general progress in information and communication technology has increased the availability and ease of use of data and econometrical software packages, and the methodological progress has provided us with sophisticated forecasting procedures. The topic of this special issue of the Journal of Economics and Statistics is the theory and practise of forecasting and forecast evaluation. The purpose is to provide an overview of the state of the art of forecasting; a specific focus is on business cycle forecasts and forecasting in finance. The papers included in this volume deal with both methodological issues and empirical applications. The paper "Information or Institution? On the Determinants of Forecast Accuracy" by Roland Dohrn and Christopher M . Schmidt uses a broad sample of forecasts of German GDP and its components to analyze the impact of institutions and information on forecast accuracy. The accuracy of macroeconomic forecasts depends on various factors, most importantly the mix of analytical methods used by the individual forecasters, the way that their personal experience is shaping their identification strategies, but also their efficiency in translating new information into revised forecasts. The central empirical result of the paper reveals the importance of the forecast horizon for forecast accuracy. By contrast, institutional factors are small and statistically insignificant. As better information seems to be the key to achieving better forecasts, approaches for acquiring reliable information early seem to be a good investment. Factor models that efficiently summarize the information contained in a large cross-section of time series by a few factors are by now routinely used in forecasting. The paper "Forecasting with Factor Models Estimated on Large Datasets: A Review of the Recent Literature and Evidence for German G D P " by Christian Schumacher provides a survey of recent developments in the area of factor based forecasts. In particular, the focus of this paper is on techniques that handle practically relevant data irregularities, as e.g. missing observations for some time series at the end of the sample ("ragged edge") or issues that arise when time series are observed at different frequencies ("mixed-frequency data"). Schumacher reviews first extensions of the standard single-frequency two-step procedure for balanced data that modify both, the factor estimation and the forecasting step. Then large state-space models that jointly estimate factors and forecasts are described and the respective merits and drawbacks of both alternative approaches are compared. The paper also provides an overview of factor-based forecasting results for German GDP. Two interesting results emerge: Factor-based forecasts for German GDP using mixed-frequency data tend to outperform those based on single-frequency (timeaggregated) data and for forecasting horizons longer than one quarter the factor models are not very informative.

6 • Special Issue on Economic Forecasts: Guest Editorial

In a related paper "A Factor Model for Euro-area Short-term Inflation Analysis" Michele Lenza and Thomas Warmedinger provide a forecasting application using an approximate dynamic factor model that accounts for a data structure with different publication lags, different historical sample periods for different variables and different observational frequencies. Using an evaluation period from 1999Q1-2008Q3, the forecasting accuracy of the factor model for year-on-year consumer price inflation is first compared to a naive random walk model, which in the last years has been a hard to beat benchmark for Euro area inflation. Interestingly, the authors find that their factor model outperforms the benchmark for forecasting horizons larger than nine month ahead. They also include an illustrative comparison of their results to those obtained from the Eurosystem projection exercise prepared by the European national central banks and the European Central Bank (ECB) using a wide range of analytical tools. The forecasts from both approaches are quite similar, possibly indicating that the large set of information in the factor based forecasts are exploited in a similar way as in institutional forecasts. Finally, the authors also illustrate that the timely use of newly released data and the inclusion raw materials and commodity prices typically improves forecast accuracy of the factor model at short horizons. The key feature of the forecasting method suggested in the paper "Combining Survey Forecasts and Time Series Models: The Case of the Euribor" by Fabian Krüger, Frieder Mokinski and Winfried Pohlmeier is very similar to the one of forecasting models with a large number of factors: Large information sets are to be used to improve forecasts. Krüger et al., however, apply a dynamic method where cross-sectional information from survey forecasts as well as conventional time series information is optimally combined. Their combination approach is attractive since it allows for the use of very different information sets. This is particularly useful in the presence of structural breaks if one takes into account that survey responses do not rely on historical information but can be regarded as being generated from a very short data filter. The approach by Krüger et al. reinterprets Maganelli's (2009) idea of "Forecasting with Judgment" by considering expert forecasts obtained from survey data as prior information. The authors use this information in order to estimate the parameter vector of a time series model. Thus their approach, which they apply to forecast the EURIBOR, can be seen as a special type of shrinkage estimator. Business cycle indicators provide useful information for forecasting macroeconomic aggregates. The paper "Predictive ability of Business Cycle Indicators under Test: A Case Study for the Euro Area Industrial Production" by Kai Carstensen, Klaus Wohlrabe and Christiane Ziegler assesses the information content of several widely cited early indicators for the euro area with respect to forecasting area-wide industrial production. It employs various tests to compare competing forecast models, i.e. it pays attention to nested model structures, alleviates the problem of data snooping arising from multiple pairwise testing, and analyzes the structural stability in relative forecast performance of one indicator compared to a benchmark model. The results show that there is not one best indicator that uniformly dominates all its competitors. The optimal choice rather depends on the specific forecast situation and the loss function of the user. Many economic time series variables are obtained from contemporaneous aggregation over cross-sectional units such as regions or countries. Prominent examples are areawide aggregates for the Euro-area, which are obtained by aggregating data from Euro member countries. While the theoretical literature has so far primarily focused

Special Issue on Economic Forecasts: Guest Editorial • 7

on forecasting linear aggregates obtained with fixed, time-invariant aggregation weights, there are many practical examples, where aggregation weights are in fact time-varying and/or the aggregate is based on non-linear aggregation. The paper "Forecasting Nonlinear Aggregates and Aggregates with Time-varying Weights" by Helmut Lütkepohl contributes to the literature by introducing a theoretical framework for comparing alternative predictors for nonlinear aggregates including aggregates that are based on stochastic and time-varying weights. The theoretical properties of specific aggregates are derived under simple assumptions on the stochastic process for the aggregation weights and theoretical properties of different predictors are compared. One predictor is based on a univariate model of the aggregate, while the alternative one also considers (parts of) the information on the disaggregate variables. The usefulness of different approaches is investigated using Monte Carlo experiments. The results indicate that the additional use of a small number of disaggregate components may indeed improve forecast accuracy of the aggregate relative to univariate forecasts of the aggregate. The main results are also confirmed in an empirical forecasting exercise for Euro-area unemployment and inflation rates. Over the last couple of years much research effort in financial econometrics has been devoted to the concept of realized volatilities as an alternative to the traditional GARCH approaches. Based on the assumption of a quadratic co-variation process, realized volatilities and realized co-volatilities are simply nonparametric estimates of volatilities and co-volatilities for a given frequency (typically daily) where the information on the variation in the returns series is taken from higher frequencies (intraday returns). The paper "Forecasting Multivariate Volatility using the VARFIMA Model on Realized Covariance Choleski Factors" by Roxana Halbleib and Valerie Voev uses realized volatilities and co-volatilities for portfolio selection problem as an alternative to MGARCH. Using a parsimonious fractionally integrated vector-autoregressive model they forecast the Cholesky factors of the realized covariance matrix. This guarantees positive definite forecasts of the variance-covariance matrix of the returns and accounts for the long memory property of volatilities, an empirical property which is difficult to account for in M G A R C H models. However, the Cholesky transformation induces a bias in the final covariance matrix forecasts. Halbleib and Voev show empirically that correcting for the bias does not necessarily improve the statistical quality of the multivariate volatility forecasts. Nevertheless, especially in finance, the superiority of new econometric forecasting techniques should be shown in terms of economic criteria rather that statistical goodness fit measures. Therefore Halbleib and Voev use in their application to the portfolio selection problem second order stochastic dominance tests to show that their VARIFIMA Cholesky approach also outperforms standard volatility approaches also in terms of an important economic measure. In addition to business cycle forecasts government agencies and other national and international institutions are asked to perform forecasts over the medium term. In particular, the EU Stability and Growth Pact contains the obligation to formulate stability programmes over four years, covering a general economic outlook as well as the projected development of public finances. However, the current practice of performing medium-term economic projections is unsatisfactory from a methodological point of view as the applied methodology has been developed for short-run forecasting and it is questionable whether these methods are useful for the medium term. The paper "Practice and Prospects of Medium-term Economic Forecasting" by Helmut Hofer, Torsten Schmidt

8 • Special Issue on Economic Forecasts: Guest Editorial

and Klaus Weyerstrass gives an overview of currently used methods for medium-term macroeconomic projections. In particular, it analyses the performance of medium-term forecasts for Austria to illustrate the strengths and weaknesses of the typical approach and describes approaches to improve medium-run projections. Ralf Briiggemann Winfried Pohlmeier Werner Smolny

J a h r b ü c h e r ! Nationalökonomie u. Statistik (Lucius & Lucius, Stuttgart 2 0 1 1 ) Bd. (Vol.) 2 3 1 / 1

Abhandlungen / Original Papers Information or Institution? O n the Determinants of Forecast Accuracy By Roland Dohrn and Christoph M . Schmidt, Essen* JEL C53; E27; E01 Forecast accuracy; forecast revisions; forecast horizon; economic activity.

Summary The accuracy of macroeconomic forecast depends on various factors, most importantly the mix of analytical methods used by the individual forecasters, the way that their personal experience is shaping their identification strategies, but also their efficiency in translating new information into revised forecasts. In this paper we use a broad sample of forecasts of German GDP and its components to analyze the impact of institutions and information on forecast accuracy. We find that forecast errors are a linear function of the forecast horizon, which serves as an indicator of the information available at the time a forecast is produced. This result is robust over a variety of different specifications. As better information seems to be the key to achieving better forecasts, approaches for acquiring reliable information early seem to be a good investment. By contrast, the institutional factors tend to be small and statistically insignificant. It has to remain open, whether this is the consequence of the efficiency-enhancing competition among German research institutions or rather the reflection of an abundance of forecast suppliers.

1

Introduction

Rankings of forecasters have gained popularity in Germany, at least since the Financial Times Deutschland has started to elect the „Forecaster of the Year" each December. However, a systematic analysis of the quality of forecasts and its determinants is still missing. Already the definition of forecast quality raises some important questions: Is it really sufficient to look only at ex post-forecast accuracy, i. e. the difference between outcome and prediction? More concretely, should we focus on GDP only or should we take a broader view and look also at the main components of GDP such as private consumption, investment etc.? Certainly, we would be hesitant to award a price to an accurate forecast of GDP which combined large, yet offsetting forecast errors for its components. And how should we appreciate the analytical and methodological foundations of a forecast? Is it sufficient to pick the right figure just by chance or should it be based on a solid background? A forecast that reflected the available information poorly at the time of its construction might be vindicated by surprises afterwards, but this does not make its ori-

* We are grateful for comments by Thomas Bauer, Wim Kösters, Joachim Schmidt, and by two anonymous referees.

10 • Roland Döhrn and Christoph M . Schmidt

ginator a reliable forecaster. These questions demonstrate that one may not do justice to forecasters when reducing their work to one figure and evaluating them based on it. Against this background this paper takes a broader view on forecast accuracy and its determinants. We do not only include GDP but also other relevant macroeconomic variables in our analysis, and focus on the role of the forecasting horizon and differences in institutional performance. Specifically, we use a large sample of real-time forecasts to find out whether some forecasting institutions tend to produce better forecasts than others. Our sample covers the years 1991 to 2008 which is a moderate time span, comprising two major recessions 1992/93 and 2001. Identifying institutional aspects is difficult, because forecast accuracy is heavily influenced by forecasting horizon, and forecasters systematically differ regarding the week when a forecast is completed. It is obvious that new information about the state of the economy and its important components, such as interest rates, exchange rates or commodity prices, keeps arriving day after day, making forecasting easier for those who hold back their publication a while longer. Yet, early forecasts are valuable for decision makers in business and for policy makers alike, and the different suppliers of forecasts therefore choose the date of publication in a trade-off between accuracy and novelty. But timing is not the only aspect that sets different forecasting institutions apart. True, all professional forecasters use the same data and quite similar techniques. Therefore, the institutional factor can be expected to be small. At the same time, the personal experience of forecasters may enable them to combine the bits of information they collected more or less efficiently. And sometimes, also the bureaucratic procedures inside institutions may have an influence, e. g. when a forecast has to be approved by a governing council, as is typically the case for international institutions. Moreover, an important feature of all relevant professional forecasting institutions is that they do not rely simply on a single method but combine various techniques either in the sense that they employ different methods for different questions, or that the outcome of one method is assessed and potentially modified in the light of the results of another one. 1 The use of different methods is particularly evident for different time horizons. The forecast process mostly starts with an estimate, a „nowcast", for the current quarter, and sometimes also for the quarter which just has ended, in the case official figures are not available yet. For these quarters, many indicators are already available which allow a more or less accurate estimate of GDP and its components. This part of the forecasting process which is sometimes extended to one quarter ahead provides the starting point for predicting the following quarters. For the remaining forecast horizon various techniques are combined, including time series as well as structural models, and also comparisons with reference cycles. All these different predictions are made consistent by linking them through the definitions of the system of national accounts. As creating a forecast is a rather complex endeavor, it is difficult if not impossible to evaluate forecasts analytically, i.e. to assess the performance of the various techniques employed and the efficiency of information processing. To capture the empirical manifestation of the second aspect, we make use of the fact that forecasters receive a continuous stream of data influencing their prediction. As the forecast horizon is shrinking, the information set is unequivocally growing. Therefore, we would expect a rather strong 1

An example for the interaction of different forecasting methods is provided by Nierhaus and Sturm (2003).

Information or Institution? • 11

negative relation between forecast horizon and accuracy, which we, indeed, can find in our data. The institution-specific deviations from this trend line are small which on the one hand confirms that no forecaster commands a superior technique, and on the other hand allows some more general conclusions about the art of forecasting. The paper is organized as follows. Section 2 offers a short review of the literature. To separate between institutional factors and the impact of information on forecast accuracy, our database contains, besides the forecasts of German GDP and its main demand side components, also the dates when the forecasts were constructed. A more detailed description of the database is given in section 3. Section 4 presents our analyses of forecast accuracy. After an inspection of the data, we at first assess the accuracy of GDP forecasts and, thereafter, we also investigate predictions of the main GDP components. In section 5, we draw some conclusions for improving forecast accuracy. 2

Review of the literature

Rational forecasts are constructed by processing all available information about the state of the economy and all the time paths of its elements leading up to this state. While this general principle is universal, the concrete way how this information is processed is decisive for the forecast's accuracy. Quite obviously, it is undesirable to construct forecasts which could evidently be improved upon by using pieces of the available information in a different manner. One strand of the forecast evaluation literature addresses this question of information efficiency by devising concrete testing strategies (for an overview: Stekler 2002). One test is provided by Isiklar et al. (2006) who use the autocorrelation of forecast revisions as a measure of „weak" information efficiency. If revisions are autocorrelated, they argue, it seems evident that forecasters have not used the full set of information available in their original forecast. Applying this approach to the Consensus Forecasts for 18 countries - each presenting an average of quite a large number of forecasts - they conclude that forecasts are not efficient, and that Germany belongs to the countries with the least efficient forecasts. Yet, the Consensus Forecasts collate predictions of quite different nature. Some forecasters adjust their predictions every month, whereas others only don't replace their forecasts for several months at a time. The German economic research institutes, e.g., which are also included in the Consensus, present new forecasts every three months, reflecting the publication rhythm of the quarterly national accounts. Hence, the inefficiency Isiklar et al. (2006) found may be in part an aggregation problem. Moreover Isiklar et al. (2006) acknowledge that the published data that form the information set for the forecasts are typically revised thereafter and, thus, the inefficiency they describe may be a result of bad data quality. Incidentally, Oiler and Teterukovsky (2007) even turn the perspective around and use the accuracy of forecasts as a measure of data quality. They argue that a statistical variable can hardly be forecasted accurately if the available information does not reflect previous values of the variable well. Similarly, Isiklar and Lahiri (2007) take the improvement of forecast accuracy as a measure of the information content of new data. By contrast, we shift our focus to the individual providers of the forecasts, recording in particular the dates of the finalization of individual forecasts. The role of the forecaster on the accuracy of the forecast has only received little attention in the literature. Heilemann and Klinger (2005) analyzed in an international comparison the impact of the structure of the market for forecasts, measured by the number of fore-

12 • Roland Döhrn and Christoph M . Schmidt

casters, on the „market result" in terms of forecast accuracy. However, they could not find clear evidence that the forecast errors are smaller in countries with a more intense competition. Dopke and Fritsche (2006a, 2006b) analyzed the accuracy of growth and of inflation forecasts in a panel data set which comprised forecasts by some 14 German institutions. They illustrated a negative correlation of accuracy and the length of the period that had to be forecasted. However, they did not explicitly separate forecast horizon and institutional performance. Our analysis, by contrast, aims at gauging institutional performance net of any differences in forecast horizon. Forecasts are constructed by researchers, not by lifeless automata. Consequently, they might be influenced by individual motives and personal or institutional characteristics. Most prominently, Lamont (2002) evaluated the accuracy of predictions provided by a large number of individual and institutional forecasters for the U. S., finding that the probability to make an outlier forecast increases with the age of the forecaster. He explained this behavior arguing that only experienced and well established forecasters dare presenting outlier forecasts which will increase their reputation, if they turn out to be right, but may do harm to the career if they are wrong. However, for the institutions included in his sample he did not find a bias in this direction. Hence, in our analysis one should expect the institutional factors to be small. 3

The data base

Our analysis comprises nearly two decades of data for the major professional institutions concerned with forecasting German economic activity. During a course of slightly more than two years all these institutions have published several forecasts for any given year, respectively. For the year 2008, e.g., the 'forecast season' was opened in the autumn of 2006, when the EU published its first forecast for this year. Subsequently the EU provided another two forecasts in the course of 2007, one in spring and one in autumn, and two further forecasts in 2008. Thus, the first forecast had a horizon of 10 quarters, i.e. the last quarter for which national accounts data were available at that time was the second quarter of 2006. Subsequent forecasts moved ever more closely to the end of 2008. Finally, the fifth and last forecast constructed in autumn 2008 had a forecast horizon of only two quarters. Other forecasters provide a different number of forecasts. In particular RWI, Essen, and the Institute for World Economics, Kiel, synchronized their forecasts in the recent years with the releases of new GDP figures, resulting in four publications per year, which makes up eight forecasts for a given year. However, publication rhythms and forecast horizons have altered in the course of time, and they are sometimes changed occasionally. 2 The database only contains forecasts which were available for all years between 1991 and 2008. Due to this restriction, some forecasts had to be skipped. For instance, the spring forecasts of the Gemeinschaftsdiagnose and of RWI for the subsequent year were excluded from the analysis, as they are available only since the mid-1990s. For the same reason, institutions are missing which only entered the forecasting business during the 1990s, i.e. the Institut fur Wirtschaftsforschung, Halle, as well as those who went out of business, i.e. HWWA. Furthermore, only institutions were included 2

In such turbulent times as the recent recession, many international institutions publish interim forecasts to get a more up-to-date picture of the economic situation.

Information or Institution? • 13

Table 1 Forecasts Included in the Analysis Abbreviation Forecast (first quarter to be predicted) German Institute for Economic Research (Deutsches Institut für Wirtschaftsforschung) DIW-7 DIW-5 DIW-3 DIW-1

Summer forecast year t-1 (q2,t-1) January forecast year t (q4, t-1) Summer forecast year t (q2, t) January forecast year t+1 (q4, t) European Commission

EU-10 EU-8 EU-6 EU-4 EU-2

Autumn forecast year t-2 (q3, t-2) Spring forecast year t-1 (q1, t-1) Autumn forecast year t-1 (q3, t-1) Spring forecast year t (q1, t) Autumn forecast year t (q3, t) Joint forecast of the German Institutes (Gemeinschaftsdiagnose)

GD-6 GD-4 GD-2

Autumn forecast year t-1 (q3, t-1) Spring forecast year t (q1, t) Autumn forecast year t (q3, t) ifo Institute for Economic Research (Ifo Institut für Wirtschaftsforschung)

lfo-7 lfo-5 lfo-3 Ifo-1

June Forecast year t-1 (q2, t-1) December forecast year t-1 (q4, t-1) June Forecast year t (q2, t) December forecast year t (q4, t)

lfW-7 lfW-5 lfW-3 lfW-1

June Forecast year t-1 (q2, t-1) December forecast year t-1 (q4, t-1) June Forecast year t (q2, t) December forecast year t (q4, t)

Kiel Institute of World Economics (Institut für Weltwirtschaft)

Organization for Economic Co-operation and Development OECD-9 OECD-7 OECD-5 OECD-3 OECD-1

Economic Economic Economic Economic Economic

Outlook Outlook Outlook Outlook Outlook

December year t-2 (q4, t-2) June year t-1 (q2, t-1) December year t-1 (q4, t-1) June year t (q2, t) December year t (q4, t)

Rheinisch-Westfälisches Institut für Wirtschaftsforschung RWI-7 RWI-5 RWI-4 RWI-3 RWI-1

Summer forecast year t-1 (q2, t-1) December forecast year t-1 (q4, t-1) February/March forecast year t (q1, t) Summer forecast year t (q2, t) December forecast year t (q4, t) The German Council of Economic Experts (Sachverständigenrat)

SVR-6 SVR-2

Annual Report year t-1 (q3, t-1) Annual Report year t (q3, t)

The index t marks the year the forecast was made for. The forecasts are named after the acronym of the forecasting institution and the forecast horizon expressed in quarters.

14 • Roland Döhrn and Christoph M . Schmidt

which provide information on all positions of the national accounts which are considered here. This is the reason why the International Monetary Fund forecasts have been skipped. In the end we analyze 32 forecasts from eight institutions, each providing 18 observations (Table 1) which consequently form a balanced sample. In the database the forecasts of the year-over-year growth of GDP are collected as well as those of its demand side components: private consumption, government consumption, gross fixed capital formation, investment in equipment and machinery, investment in buildings, exports, and imports. Some forecasts also contain data on the contributions of net-exports to growth. As far as these data were not provided directly in the publications, they were derived from the export and import forecasts. 3 As we focus in our paper on the efficiency of information processing, a very important part of the database is precise information on the dates when the forecasts were finalized. In quite the majority of forecasts in our database, the day when the forecast was completed is mentioned explicitly. In others, the editorials of the forecast are dated, and it seems logical that they were finalized close to this date. As far as no explicit reference is made, the publication date of the volume which contains the forecast provides some indication. If only the month of publication is known, which is the fact in rare cases, the middle of this month is regarded as the publication date. During the period under study, German unification took place. This created a considerable challenge to all forecasters, given the tremendous uncertainty about the true value of East German GDP and its growth potential. Until 1994, most institutions provided separate projections for West and for East Germany. To compare the comparable, our database contains West German figures for the years 1991 to 1994 and data for unified Germany thereafter. However, due to unification, the data on export and import forecasts may be contaminated to some extent for the years leading up to 1994, as West German figures include intra-trade with Eastern Germany to be consistent with the System of National Accounts. The intensity of this trade was difficult to measure because of the lack of any statistics on that issue. The results of forecast evaluations, of course, depend a lot on the choice of the figures that are considered as realizations. National accounts data are revised several times, and the revisions may be considerable, in particular when they reflect conceptual changes in the System of National Accounts. If these changes had already been known when the predictions were made, they would have altered the relevant information sets and, thus, have led to different forecasts. For the sake of a fair comparison, only those national accounts figures should be used as references which were released as close as possible to the forecast, to make sure that prediction and realization follow the same conceptions. 4 Therefore, we will always compare the forecasts under real time conditions using the figures as a yardstick that were published by the German Statistical Office in 3

The contribution of real net exports (ne) to growth was calculated as net = EXPt-i/GDPt_i*g(expt) - I M P M / G D P m

4

* g(impt).

with EXP, IMP and GDP being nominal exports, imports and GDP (in bn Euro resp. DM) and g(exp) resp. g(imp) being the growth rates of real exports and imports. The nominal values were taken from the Gemeinschaftsdiagnose published in spring. This procedure may lead to wrong results for the second year of a two-step forecast, as in this case not the actual values of EXP, IMP and GDP should be taken, but the forecasted values. However, as these data are not at hand for many forecasts, we have to rely on these calculated figures. Our experience from cases in which figures on net exports were published shows, that the error implied by our procedure tends not to be very large. In Germany, e.g., revisions tend to make recessions look less severe with every revision of the system of national accounts (Rath 2009).

Information or Institution? • 15

February of the year which followed the year the forecast w a s m a d e for. These data were taken f r o m the Gemeinschaftsdiagnose published in the spring of each year.

4

Accuracy of GDP forecasts over time

4.1 Inspection of the data We start our analyses by an inspection of our data. In C h a r t 1 the errors of the forecasts included are plotted against the length of the forecast horizon expressed in calendar days. It becomes evident that there is a strong correlation between production date and forecast accuracy in some years, whereas n o clear trend can be discovered in m a n y others. T h e relationship is particularly strong in years that m a r k turning points in the business cycle (1993, 1994, 2001). But also in 2 0 0 2 and 2 0 0 3 , w h e n an upswing w a s forecasted t h a t did not occur, a n d in 2 0 0 6 and 2 0 0 7 , w h e n the upswing turned out to be stronger t h a n expected, forecast errors become significantly smaller over time. For the remaining years, forecast errors often are generally low, and additional i n f o r m a t i o n apparently confirmed the forecasts which were m a d e earlier. Retaining our focus on forecast horizons and individual institutions, Table 2 presents s u m m a r y statistics on forecast accuracy. Across all institutions, mean absolute errors (MAE), m e a n squared errors (MSE) a n d r o o t m e a n squared errors (RMSE) all decline visibly as the forecasting horizon, here expressed in average days, becomes smaller. For instance, the M A E tends t o fall f r o m approximately 1.2 percentage points for a forecast horizon of 7 quarters to 0.5 percentage points or less for a horizon of just 3 quarters. T h e c o m p a r i s o n of M A E a n d RMSE which can serve as a rough measure for the contribution of outliers to the forecast errors, exhibits large differences in particular for long forecast horizons. This makes evident that a good deal of the errors results f r o m large deviations in a small n u m b e r of years. For shorter forecast horizons there are small differences between M A E and R M S E . Finally, the m e a n errors (ME) suggest that, if at all, then most forecasts tend to be biased u p w a r d s . Numerically, this bias is more substantial for large horizons, but in most cases, the M E is not significantly different f r o m zero. To illustrate the intimate relation between forecast p e r f o r m a n c e and horizon further, C h a r t 2 plots M A E against the forecast horizon, generating a n almost perfect straight line. T h e deviations of individual institutions f r o m this straight line are small, with t w o exceptions: The t w o early forecasts by the European Commission and the O E C D with forecast horizons above t w o years. They exhibit errors t h a t are nearly the same as those published a b o u t 2 0 0 days later by the same institutions. T h e adherence of all other average forecasts to this line indicates that institutional factors - if there are any - have to be considered as being r a t h e r small. All three forecasts of the Joint Forecast (Gemeinschaftsdiagnose) can be f o u n d below the average. This seems t o confirm t h a t pooling the knowledge of various forecasters m a y lead to better predictions. By contrast, the average errors of all four O E C D forecasts lie above trend. This indicates that the publication of O E C D forecasts is delayed by internal coordination requirements. But it remains t o be shown w h e t h e r these differences are statistically significant.

16 • Roland Dohrn and Christoph M. Schmidt

1991

1.50

1992

1.50

I

1.00

1.00 0.50

0.50

o.oo

V

« 1i

• T

*

••

-0.50

-1.00

J i •• -«•—Oft--*tfcr

-

v

•

-0.50

*

-1.00

i

-1.50

-1.50

~~I -2.00

-2.00

7 750

550

350

___

-1.00

550

mI

350

-3.00

4

2.50

m

2.00

1.50

• ••

%

1.00 0.50 -

—

•

—

0.00 -0.50

550

350

150

-50 750

550

350

150

-50

1996

1.50

i

1.00

«

0.50

0.00 V

* f

•

*

\

-1.50 -2.00

750

3.00

* •

•

1995

-1.00

-50

«

T

1.50

-0.50

150

1994

mmL

-2.00

7 750

-50 750

•

0.00

-5.00

150

1993

1.00

-4.00

0.00

550

350

150

n r -50 750

A

• • r#

1.00 0.50

•

0.00

• r

-0.50

-1.00

f •

550

350

-1.50 -2.00 150

Chart 1 Forecast Error versus Forecast Horizon1 for Individual Years

-50

Author's calculations - 1 Difference between the publication date of the forecast and December, 31 rd of the year, the forecast was made for. Values on the horizontal axis.

Information or Institution? • 17

1997

2.00

1998

i

1.50

!

]

1.00

\

i n»

m

-0.50

1.00 0.50

*

> -•

0.00

1.50

\

*

•

0.50

2.00

\

# « r

* •

0.00

•

m *

-0.50

-1.00

-1.00

-1.50

-1.50 1750

550

150

-50 750

0.50

350

150

-0.50

• A

-1.00

-«bM

2000

2.00 1.50

1.00

*

r

>

-I

••

350

150

-50 750

550

2001

0.50

- w -

-1.00 -1.50 350

150

-50

2002

0.50

i

0.00 -0.50

-mSL

A

-1.00 -1.50


" Fi.q.')2

RMSE,, = ^J MSE, q

with F denoting the forecast and R the realization, subscript t indicating the year for which the prediction was constructed, and subscripts i and q indicating the institution and the number of quarters to be predicted, respectively. »/**/*** denote significance at 90/95/99 % level respectively. - 1see Table 1.

20 • Roland Döhrn and Christoph M. Schmidt

1.6

1.6

1.4

1.4

O

1.2

1.2

¡•-¿vi-

1.0 0.8

1.0 0.8

P ü

0.6

ÖGD

BRWI

A SVR

0.4

• OECD

®EU

O ifo

0.2

ODIW

0.0

800

—r~ 700

OifW

600

o~

0.6

f'è

0.4 0.2

!

!

,

500

400

300

r

200

0.0 100

-100

Forecast horizon (days)

Chart 2 Mean Absolute Forecast Error1 and Forecast Horizon2. GDP forecasts for Germany by eight institutions

Author's calculations - 11991 to 2008. - difference between the publication date of the forecast and December, 31rd of the year, the forecast was made for. Average 1991 to 2008.

4.2 Accuracy of GDP forecasts Therefore, we now proceed from the analysis of summary measures by institution to the investigation of individual forecasts. To this end, we consider the variation of absolute forecast errors (AE) across institutions and individual years, and relate them to their precise forecast horizon (h). This variable is measured in days and it is defined as the distance between the day the forecast was released and the 31 s t December of the year the forecast was made for 5 . Here, we utilize 3 0 forecasts for 18 years starting from 1 9 9 1 and ending 2 0 0 8 , generating a sample of 5 4 0 observations 6 . Thus, the basic equation of our analyses can be written as follows: A E , i i i t = a 0 + a-yhi^t + £,• q f

(1)

In eq. (1), i denotes the institution, q the forecast horizon (in quarters, as defined in Table 1) and t the year the forecast is constructed for. The variable £ is the disturbance term. The parameter a 0 captures that forecast error which remains at the end o f the forecast year, whereas the parameter ai reflects the (negative of the) accuracy gain per day. To control for institutional factors, we additionally include a set of indicator variables D'|. These indicators take the value of 1 whenever a forecast was published by institution j, and 0 in all other cases. As these variables allow for institution-specific intercepts, their variation captures any permanent performance differences across institutions 7 : 5

6

7

Negative values indicate that the forecast for year t was published in year t+1, but before the official figures were released. In our regressions the two early forecasts of O E C D and E U are omitted, since they are three-step forecasts. In our concrete application, the RWI forecasts were used as a numeraire, retaining an overall constant. This leads to numerically identical results for the corresponding sum of squared residuals and the slope coefficients. Expressing institutional intercepts as deviations from a hypothetical mean (Haisken-DeNew/Schmidt 1 9 9 7 ) has not become an issue, due to the apparent irrelevance of performance differences across institutions.

Information or Institution? • 21

Table 3 Absolute Forecast Errors of GDP and Forecast Horizon 1991-2008 Regression

(1)

(2)

(3)

(4)

(5)

(6)

Explanatory variable Constant

0.097 0.062 0.069 0.068 0.095 0.063 (2.6) (3.0) (1.8) (7.3) (2.5) (1.5) 0.00197 0.00196 0.00198 0.00196 0.00196 0.00197 Forecast horizon (days) (13.3) (13.2) (18.2) (17.9) (13.2) (18.1) 0.661 0.661 0.662 Working-day effect (4.1) (absolute) (4.1) (4.1) Working-day "horizon -0.00784 -0.00784 -0.00785 (10.2) (10.2) (10.3) Carry-over1 0.138 0.138 0.138 (4.1) (absolute) (4.1) (4.1) Carry-over * horizon -0.00136 -0.00136 -0.00136 (10.0) (9.9) (10.0) 0.032 0.032 Forecast of International (0.5) (0.7) Institution Institutional effects Significance of institutional effects2 Number of observations R 2 (adj.)

NO 540 0.289

YES 0.21 (98.3 %) 540 0.281

NO 540 0.548

YES 0.37 (92.0%) 540 0.543

NO

NO

540 0.288

540 0.547

Authors' calculations. In parentheses: t-statistics. - White heteroskedasticity-consistent standard errors. - 1 End of previous year.- 2 F-Statistic, in parentheses: probability in % . AEi^t

a

0,/Di

=

+ t is the error of the forecast published by institution i for the current year t, looking q quarters ahead, and e^q+^t+i is the error of the coincidently published forecast for year t+1, looking q+4 quarters ahead. As the sign of the forecast error matters in this calculation, controlling for the influence of the length of the forecast horizon h requires Table 5 Two-Step-Forecast Errors: One-Step-Forecast Errors as Predictors 1992-2008 Constant Error of the one step forecast Forecast horizon (days) * sign of the forecast error Number of observations R 2 (adj.) F-Statistic

0.080 (1.4) 0.323 (2.9) 0.002 (17.4) 238 0.579 164.0

Authors' calculations. In parentheses: t-statistics. - White heteroskedasticity-consistent standard errors

24 • Roland Döhrn and Christoph M. Schmidt

taking it into account in the specification, via an interaction term. Table 5 documents that the estimate of bi is positive, i. e. that errors of coincidently produced forecasts tend to display the same sign. Hence it does not seem to be the case that a bad timing of events is a major culprit for forecast errors. To the contrary, it seems that forecasters frequently err on the general tendencies instead. 4.3 Demand side components of GDP While evaluating the forecasts of the demand side components of GDP we restrict ourselves to estimating equation (1). According to our results (Table 6), the correlation between forecast accuracy and forecast horizon is weaker in most cases compared to the results found for GDP (which are replicated from Table 3 for convenience). They are especially low for government consumption and net exports. The information-gain per day is particularly high for investment, exports and imports. This is consistent with the fact that these components of GDP are more volatile than the other components, so that each additional piece of information may reduce uncertainty more than for less Table 6 MAE and Forecast Horizon for GDP Components Regressions 1991-2008

GDP

Private Consumption

Government Consumption

Gross Investment

Exports

Imports

Net-exports2

Constant

Forecast horizon (days)

0.069 2.6) 0.097 (1.5) 0.251 (8.9) 0.285 (4.9) 0.491 (15.1) 0.440 (7.6) 0.672 (6.6) 0.575 (2.6) 1.140 (9.4) 1.083 (5.2) 1.377 (9.5) 1.206 (5.2) 0.476 (10.4) 0.506 (7.4)

0.00197 (13.3) 0.00196 (13.1) 0.00177 (14.1) 0.00177 (13.9) 0.00086 (7.4) 0.00085 (7.2) 0.00507 (10.6) 0.00502 (10.3) 0.00442 (10.5) 0.00445 (10.6) 0.00402 (8.4) 0.00403 (8.4) 0.00041 (3.4) 0.00041 (3.4)

Institutional Test on effects Institutional effects1 NO YES NO YES NO YES NO YES NO YES NO YES NO YES

R2 (adj) 0.289

0.21 (98.3)

0.281 0.293

0.29 (95.8)

0.286

0.58 (78.8)

0.079

0.32 (94.7)

0.180

0.46 (86.2)

0.183

0.62 (73.9)

0.123

0.20 (98.5)

0.011

0.086

0.187

0.188

0.127

0.022

Authors' calculations. 540 observations. In brackets: t-statistlcs- - White heteroskedasticlty-consistent standard errors. - 1F-Statistics, in parentheses: probability In %. 2Contributlon to growth in percentage points.

Information or Institution? • 25

volatile categories. Furthermore, the constants of the regressions which indicate the forecast errors remaining at the end of each year are higher t h a n those for G D P for all individual c o m p o n e n t s . Partly this indicates that c o m p o n e n t errors tend to offset one another at the aggregate level. But the result is also consistent with the fact t h a t the first national accounts data published for the G D P c o m p o n e n t s are not very reliable in some cases and subject to m a j o r revisions. This particularly holds for the data in foreign trade. As exports and imports are often revised in the same direction new data often provide little i n f o r m a t i o n on net exports. Hence, the accuracy of projections of net exports improves little over time. Here, the correlation between accuracy a n d forecast horizon is very low, and consequently the i n f o r m a t i o n gain per day is quite modest. Also for the G D P c o m p o n e n t s we additionally included d u m m y variables t o control for institution-specific effects, as it is described in equation (2). Again, the estimated coefficients of the institutional variables were small a n d statistically insignificant. A joint test did not reject the null hypothesis that there are n o institutional effects in the forecast errors.

5

Conclusions

Although the accuracy of forecasts provided by various institutions differs markedly in concrete situations, the differences become small when a longer-term perspective is taken. T h e variation can be explained to a large extent by differences in the date when the forecasts were completed. Taking into account the i n f o r m a t i o n that can be collected by delaying the finalization of the forecast by one day, the absolute error of the G D P forecast can be reduced, on average, by 0 . 0 0 2 percentage points. This a m o u n t s to an average reduction 0.06 percentage points per m o n t h , with a remaining absolute error of 0.07 percentage points as the horizon a p p r o a c h e s 0. This result is quite robust with respect to different specifications of the model as well as to the inclusion of control variables capturing time-specific effects such as the working-day effect and the carry-over effect. T h e institutional factors, captured in our analyses by institution-specific intercepts of the regressions exploring the correlates of absolute forecast errors, are extremely small and statistically insignificant. Similar conclusions hold for a more restrictive specification which merely distinguishes national and international institutions. As forecast accuracy varies substantially more across time t h a n across institutions, it is tempting to d r a w the conclusion that one could d o w i t h o u t one or the other of the forecast suppliers. Yet, our results m a y also indicate that it is precisely the presence of these c o m p e t e n t competitors which keeps providers on their toes, exploiting the available inf o r m a t i o n to the fullest of their abilities. Of course, it is completely unclear, w h a t w o u l d happen t o overall forecast p e r f o r m a n c e , if the n u m b e r of competitors were reduced. In the extreme, it seems very probable, that a forecast m o n o p o l y w o u l d p e r f o r m worse. But w e c a n n o t really say anything a b o u t interim cases. W h a t we can conclude confidently, t h o u g h , is that our results did not single out an obvious u n d e r p e r f o r m e r which could definitely be kicked out of the m a r k e t w i t h o u t a loss in overall p e r f o r m a n c e . Moreover, our results d e m o n s t r a t e t h a t it w o u l d be very difficult to find evidence for the prejudice that m a n y forecasters are engaged in herding behavior. This p o p u l a r view has its origin in the observation that m a n y forecasters publish predictions at almost the same time and that their forecasts very m u c h look alike. However, in the light of our results a

26 • Roland Döhrn and Christoph M. Schmidt

more convincing explanation for this effect is the - actually quite comforting - fact that forecasters base their publications on similar data, methods and, most importantly, on similar information sets. When important new information arrives about the state of the economy, competent forecasters will revise their forecasts accordingly and, consequently, in a similar fashion. Of course, it cannot be ruled out that herding plays some role either. But this would require more than just an eyeballing of correlations. Until shown otherwise, the null hypothesis of no herding has not been rejected yet in a careful quantitative analysis. Finally, our analyses demonstrate that even delaying forecasts for a few days may pay off in terms of a somewhat more accurate result, albeit progress is small. To governments demanding accurate macroeconomic forecasts as a basis for their decisions, this insight suggests a promising avenue for improving forecast performance: As timely information seems to be the key to achieving accurate forecasts, investing into approaches which provide information more timely might be a very valuable option. By contrast, one might not gain too much from attempts to improve further the way in which the currently available information is translated quickly and accurately into forecast revisions: In general, macroeconomic forecasts provided by the major professional suppliers tend to be information efficient. This notion is not only confirmed by our results, but also holds for more sophisticated tests regarding strong information efficiency (Döhrn 2006). Simply acquiring the data from official sources earlier, however, would threaten to reduce the quality of the data which in turn would depress forecast quality. Another approach which seems to be quite promising, though, could be the exploitation of new data sources from the internet. They may provide information on the decision making e. g. of consumers in advance to the market outcome (Schmidt/Vosen 2009). However, even the progress made by using these new data sources will be limited. If they were able to „buy" half a month of information gain via an earlier revelation of relevant information, according to our estimates this would be worth, on average, a reduction of the absolute forecast error by 0.03 percentage points. Only a careful assessment of the costs involved will be able to show whether this investment is really worthwhile.

References Döhrn, R. (2006), Improving Business Cycle Forecasts' Accuracy - What Can We Learn from Past Errors? RWI: Discussion Paper 51. RWI, Essen. Döpke, J., U. Fritsche (2006a), Growth and inflation forecasts for Germany: A panel-based assessment of accuracy and efficiency. Empirical Economics 31: 777-798. Döpke, J., U. Fritsche (2006b), When do forecasters disagree? An assessment of German growth and inflation forecast dispersion. International Journal of Forecasting 22: 125-135. Haisken-DeNew, J., C.M. Schmidt (1997), Inter-Industry and Inter-Region Differentials: Mechanics and Interpretation. The Review of Economics and Statistics 79: 516-521. Heilemann, U., S. Klinger (2005), Zu wenig Wettbewerb? Zu Stand und Entwicklung der Genauigkeit von makroökonomischen Prognosen. Technical Report/Universität Dortmund, SFB 475 Komplexitätsreduktion in Multivariaten Datenstrukturen 2005,16. Isiklar, G., K. Lahiri (2007), How far ahead can we forecast? Evidence from cross-country surveys. International Journal of Forecasting 23: 167-187. Isiklar, G., K. Lahiri, P. Loungani (2006), How quickly do forecasters incorporate News? Evidence from Cross-country Surveys. Journal of Applied Econometrics 21: 703-725. Lamont, O.A. (2002), Macroeconomic forecasts and microeconomic forecasters. Journal of Economic Behavior and Organization 48: 265-280.

Information or Institution? • 2 7

Nierhaus, W., J.-E. Sturm (2003), Methoden der Konjunkturprognose. Ifo-schnelldienst 56 (4): 7-23. Oller, L.-E., A. Teterukowsky (2007), Quantifying the quality of macroeconomic variables. International Journal of Forecasting 23: 204-217. Rath, N. (2009), Rezessionen in historischer Betrachtung. Wirtschaft und Statistik 2009: 203208. Schmidt, T., S. Vosen (2009), Forecasting Private Consumption - Survey-based Indicators vs. Google Trends. Ruhr Economic Paper 155. Stekler, H. O. (2002), The Rationality and Efficiency of Individuals Forecasts. Pp. 222-240 in: M.P. Clements, D. Hendry (2002), A Companion to Economic Forecasting. Blackwell. Prof. Dr. Roland Dohm, RWI, Hohenzollernstr. 1-3, 45128 Essen, Germany, und Universität Duisburg-Essen, Germany. [email protected] Prof. Dr. Christoph M. Schmidt, RWI, Hohenzollernstr. 1-3, 45128 Essen, Germany, und RuhrUniversität Bochum, Germany. [email protected]

Jahrbücher f. Nationalökonomie u. Statistik (Lucius & Lucius, Stuttgart 2011) Bd. (Vol.) 231/1

Forecasting with Factor Models Estimated on Large Datasets: A Review of the Recent Literature and Evidence for German GDP By Christian Schumacher, Frankfurt a.M.* JEL E37; C53 Factor models; forecasting; large datasets; mixed-frequency data; missing observations; ragged edge; time aggregation.

Summary This paper provides a review of the recent literature concerned with large factor models as forecast devices. We focus on factor models that account for mixed-frequency data and missing observations at the end of the sample. These are data irregularities applied forecasters have to cope with in real time. To extract the factors from the irregular data, special factor estimation techniques are necessary, expanding on the standard approaches for balanced data such as principal components (PC). The estimation methods include variants of the Expectation-Maximisation (EM) algorithm together with PC and factor estimation using state-space models. Given the estimated factors, forecasts can be obtained from bridge equations, mixed-data sampling (MIDAS) regressions and the Kalman smoother applied to fully-fledged factor models in state-space form. Empirical applications for German GDP growth often find that forecasts based on factor models are informative only a few months ahead compared to naive benchmarks. Thus, these models can be regarded as short-term forecast tools only. However, the factor models estimated on mixed-frequency data with missing observations tend to outperform factor models based on balanced data time-aggregated from high-frequency data.

1

Introduction

Nowadays, economists working in macroeconomic policy environments can rely on a wealth of economic time series for forecasting. In the recent forecast literature, the class of factor models that can account for this increase in data availability has received much attention. Recent surveys documenting these developments are provided in, for example, Boivin and N g (2005), Stock and Watson (2006), and Eickmeier and Ziegler (2008). In these papers, factor forecasting is a two-step procedure: Factors are extracted from a large set of data and the estimated factors are plugged into a dynamic equation or VAR model that explains the variable to be predicted (Boivin/Ng 2005). The factor estimation techniques are typically principal components (PC) following Stock and Watson (2002), dynamic PC estimated using frequency-domain methods proposed by Forni et al. (2005), or subspace estimators (Kapetanios/Marcellino 2009). A comprehensive * This paper represents the author's personal opinions and does not necessarily reflect the views of the Deutsche Bundesbank. I am grateful to Marta Banbura, Ralf Briiggemann, Sandra Eickmeier, Heinz Herrmann, Karl-Heinz Todter and two referees for helpful comments and discussions.

Forecasting with Factor Models Estimated on Large Datasets • 29

overview of the empirical forecast studies using these techniques is provided in Eickmeier and Ziegler (2008). A c o m m o n feature of the models in this strand of the literature is the use of balanced a n d single-frequency data for both estimating the factors and for forecasting. For example, studies concerned with forecasting quarterly G D P g r o w t h are typically based on time-aggregated quarterly data for estimating the factors, see Banerjee et al. (2005) for the Euro area, and Schumacher a n d Dreger (2004) as well as Schumacher (2007) for Germany. This paper surveys a strand of the recent literature on large factor models that goes beyond the papers cited above by taking into account data irregularities. We focus on papers containing models that are capable of forecasting quarterly G D P g r o w t h using large datasets in real time and review existing empirical results for Germany. W h e n producing forecasts for G D P in real time, applied forecasters have to cope with d a t a irregularities for various reasons: O n e reason is t h a t different groups of statistical d a t a are typically not available at the same time for a certain reference period. They are rather published with different delays. For example, financial indicators are almost immediately available, whereas survey data are reported with a small delay. H a r d data such as industrial production takes almost t w o m o n t h s to be collected and published. At a certain point in time, w h e n a forecaster d o w n l o a d s a sample of multivariate data, these differences in publication lags lead t o a complicated pattern of missing values at the end of the sample, the so-called 'ragged edge' of multivariate data (Wallis 1986). Another data irregularity stems f r o m the presence of different statistical sampling frequencies. G D P is typically sampled at quarterly frequency only, whereas a large n u m b e r of business cycle indicators are sampled at m o n t h l y (e.g. industrial production) or even higher frequencies (e.g. financial data). Finally, G D P is also subject to a considerable publication lag. In Germany, for example, the first release of G D P is published a b o u t six week after the end of the reference quarter. Thus, m a n y business cycle indicators are available earlier than GDP. In general, forecast models should be able to account for these data irregularities in order t o exploit all time series i n f o r m a t i o n currently available. In the survey below, we discuss factor models and forecast procedures that take into account the data irregularities in different ways. Some of the procedures can be regarded as extensions t o the single-frequency two-step procedures f r o m the literature based on balanced data cited above. This review includes the factor model estimated using the Expectation-Maximization (EM) algorithm together with PC as in Schumacher and Breitung (2008), the state-space factor estimator by Giannone et al. (2008), and the FactorM I D A S (Mixed-Data Sampling) a p p r o a c h as in Marcellino a n d Schumacher (2010). T h e modifications of the techniques proposed in these papers relate n o t only to the factor estimation step but also to the forecast step. In addition to these extensions of the two-step procedures, we discuss factor models in state-space f o r m that jointly explain G D P and the factors, namely Banbura and Riinstler (2010), Proietti (2008), as well as Banbura et al. (2010). Overall, this survey is an extension of reviews in Boivin and N g (2005) and Eickmeier and Ziegler (2008) which takes into account mixed-frequency data a n d missing observations at the end of the sample. T h e paper proceeds as follows. Section 2 briefly reviews the forecast approaches for balanced d a t a . Section 3 provides an overview of the a p p r o a c h e s t h a t tackle mixed-frequency data with missing observations. It also contains a discussion of the relative advantages and disadvantages of the models. Section 4 reviews the existing empirical results for forecasting G e r m a n GDP. Finally, section 5 concludes.

30 • Christian Schumacher

2

Forecasting with factor models for balanced and single-frequency data

Factor forecasting with large, single-frequency datasets is often carried out in two steps (Boivin/Ng 2005). The two steps can be described as follows: First, the factors are estimated from a large dataset and, second, a small dynamic model for the variable to be predicted is augmented with the estimated factors. For both estimation and forecasting, many possible ways of implementation exist, for example different methods for estimating factors with large data sets and different dynamic models for forecasting. Below, we present the forecast procedure of Stock and Watson (2002) as implemented for forecasting quarterly GDP in Banerjee et al. (2005) and Schumacher (2007). We discuss how alternative approaches differ from that model. Assume that we have N quarterly indicators for estimating the factors, comprised in the N-dimensional vector Xtq for quarters tq = 1 , . . . ,Tq. The time series are assumed to be stationary (Bai/Ng 2002; Stock/Watson 2002). The data admits a factor representation such as X(,=AFt,+iifI

(1)

where the r-dimensional factor vector is denoted by Fi?. The factors times the (N x r) loadings matrix A represent the common components of each variable. The idiosyncratic components are that part of X i? not explained by the factors. Factor estimation step Bai and Ng (2002) show that even under weak time-series and cross-section dependence of the idiosyncratic components as well as a limited degree of idiosyncratic heteroscedasticity and additional regularity conditions, the factors and loadings can be estimated consistently by principal components (PC). Prior to the PC estimation, the time series are standardized to have mean zero and unit variance in the sample to eliminate scale effects. Let V denote the (N x r) matrix of the r eigenvectors corresponding to the r largest eigenvalues of the sample covariance matrix of X, ? . The PC estimator of the factors and the factor loadings are obtained as F = X V / \ / N and A = VVN, where X = (X'j,..., X'T )'. As PC estimation in the end amounts to an eigenvalue decomposition of the sample covariance matrix of Xf , we see that balancing of the data sample is a necessary requirement for the estimation. Alternative factor estimators are: ^ • The dynamic estimator by Forni et al. (2005): The PC estimator F above is essentially static as it neglects dynamic relationships between the variables. The dynamic factor model takes into account that there might be q primitive, dynamic shocks that drive the r static factors F t such that q < r. Forni et al. (2005) propose a dynamic PC analysis in the frequency domain to estimate the factors. As the filters derived by inverse Fourier transform are two-sided, a modification is necessary for forecasting purposes. Estimation proceeds in two steps: Estimation of the covariances of the common components and the idiosyncratic components in a first step, and estimation of the static factors in a second step by solving a generalized eigenvalue problem taking into account different degrees of commonality between the variables. • The subspace estimator by Kapetanios and Marcellino (2009): In general, subspace algorithms are able to estimate factors without specifying and identifying a full dynamic model. The factor estimation is essentially a singular value decomposition (SVD) applied to a coefficient matrix derived from the dynamic solution of a state-space model, see Kapetanios and Marcellino (2009) for details.

Forecasting with Factor Models Estimated on Large Datasets • 31

The factor models described above differ primarily with respect to the dynamics underlying the factors, whereas the dynamic PC estimator and the subspace estimator consider dynamics to a certain extent, the PC estimator does not. However, Bai and Ng (2005) show how dynamic and static factors can be related in a VAR companion form (Stock/ Watson 2005). Here, the existence of dynamic factors simply implies a reduced rank of the residual covariance matrix of the VAR for the static factors. Therefore, some authors argue that little is to be gained from a clear distinction between the static factors and the dynamic factors for forecasting purposes (Bai/Ng 2 0 0 5 : 54). Furthermore, Boivin and Ng (2005) show that more complicated dynamics might be subject to misspecification of the auxiliary parameters. Therefore, it is an empirical question whether and under what conditions the more sophisticated dynamic approaches outperform the static PC estimator. Comparisons of the three estimation techniques can be found in Boivin and Ng (2005), Stock and Watson (2006: section 4), D'Agostino and Giannone (2006), Schumacher (2007), and Eickmeier and Ziegler (2008). These papers also provide more details on the estimation techniques. Forecast step Following Stock and Watson (2002), the forecast step proceeds as follows. Quarterly GDP growth is defined as^ (i) = \n{YtJYtq-i), and we want to forecast h q quarters ahead. The estimated factors Fti) augment a quarterly AR distributed-lag model of GDP growth according to

yhtUbq = A> + Pi (Lq)yu +

r(Lq)ftg + eu+hq,

(2)

where Lq is the quarterly lag operator defined as Lqytq = yt i, P\{Lq) and r(Lq)are finite-order lag polynomials. On the left-hand side, GDP growth is defined as y,\h = \n(Yt[i+hJYtii). Thus, the the regression is horizon-dependent and belongs to the class of direct multi-step forecast equations (Chevillon/Hendry^2005). After estimating the coefficients by OLS, the forecast becomes y^ + h iT = fio + (Lq)yTq + T(L q )¥j q • Alternatives to the forecast step emerge from dynamic factor models (Boivin/Ng 2005). In particular, the dynamic PC approach by Forni et al. (2005) includes projections that account for the theoretical restrictions in the model. Apart from the direct estimation approach taken above from Stock and Watson (2002), it is also possible to implement iterative multi-step forecasting (Boivin/Ng 2 0 0 5 : 124). A discussion on the differences between iterative and direct forecasting is provided in Marcellino et al. (2006) and Chevillon and Hendry (2005) for the single-frequency case. In general, the direct approach as in 2) is preferable in case of misspecifications that lead to deviations from the true dynamic model. 3

Forecasting with factor models for mixed-frequency data with missing observations

The following review of models capable of factor estimation and forecasting in the presence mixed-frequency data with missing observations considers broadly six models and variants of them. When presenting the models, the survey concentrates on mainly two characteristics of the alternative approaches: • the method of factor and parameter estimation and • to what extent data irregularities such as mixed-frequency data and missing observations are dealt with. To structure the review, we can separate the considered methods of factor and parameter estimation into two groups: Two-step approaches separating models for factor estima-

32 • Christian Schumacher

tion and forecasting on the one hand and joint models of GDP and the factors that can tackle factor estimation and forecasting in a single system framework on the other. The two-step approaches in the presence of data irregularities follow a similar logic as balanced data methods discussed in section 2, and have been developed earlier than the system approaches. As the review will show, the joint models are more general than two-step approaches, as they can account for more complex model features. The survey first describes the two-step approaches in section 3.1, and thereafter the joint models for GDP and the factors in section 3.2. Whereas the next two subsections 3.1 and 3.2 present the models and discuss their relative advantages, section 3.3 contains a comparison of the two groups of models. Model preliminaries and notation Models based on mixed-frequency data with missing observations require a more complicated notation than the balanced-data models. To focus the presentation, we consider models that can account for different sampling frequencies between GDP, which is assumed to be quarterly as above, and the large sample of indicators, which are assumed to be monthly for simplicity. Furthermore, we allow for missing observations of the indicators at the end of the sample due to different statistical publication lags as discussed in the introduction. Quarterly GDP growth is denoted as ytq where i^is the quarterly time index tq = 1,2,3,... ,Tq with Tq as the final quarter for which GDP is available. GDP growth can also be expressed at the monthly frequency by setting ytm = ytytm = 3tq with tm as the monthly time index. Thus, GDP growth ytm is observed only at months tm = 3,6,9,... ,Tm with Tm = 3 T q . We are interested in forecasting hq quarters ahead or hm = 3hq months ahead, based on all the available monthly information. As the final GDP observation is available in period Tq, our aim is t o f o r e c a s t t h e G D P v a l u e YRQ+HQ = VT„ 0 monthly values of the indicators that are earlier available than GDP, for example from surveys or financial indicators. However, due to publication lags, some observations for certain time series at the end of the sample may be missing. For forecasting, we want to exploit all the information available, and thus condition on information up to period Tm + w. The GDP growth forecast is the conditional expectation yTm+hm\T„+w equivalent to YTQ+HQ\T„, t w All the models discussed below are based on a factor representation Xtm = AFtm + $tm,

(3)

which explains the variables at monthly intervals tm, in contrast to the quarterly representation (1). The factors Ftmas well as idiosyncratic components £,tm are monthly variables. Differences between models arise, because GDP is interpolated to monthly frequency in alternative ways. B.1 Two-step approaches separating factor estimation and forecasting

In this section, forecasting with factor models is carried out in two steps: In the estimation step, ther factors are estimated from the large set of indicators. In a second step, the factors {Fim}(T"Lt1"' are used as predictors for GDP growth by specific single-equation models.

Forecasting with Factor Models Estimated on Large Datasets • 33

3.1.1

Schumacher and Breitung (2008): PC factors, the E M algorithm, and G D P interpolation

To consider missing values in the data for estimating factors, Stock and Watson (2002) propose an E M algorithm combined with the standard PC approach. 1 Schumacher and Breitung (2008) discuss how the E M algorithm can be used in the presence of raggededge data in real time and how forecasts can be derived. Factor estimation step The estimation procedure considers quarterly G D P as an additional variable in the dataset. As the factor model operates at monthly frequency and monthly GDP observations are not available, they have to be interpolated. Furthermore, the missing values due to publication delays or duetto the lower sampling frequency of GDP are interpolated and collected in the matrix {X ( m }Jm=-l" together with the observed values (Angelini et al. 2006). This partly estimated dataset is balanced and can then used to estimate factors by PC as in section 2. To estimate factors with the E M algorithm, observed and unobserved values have to be related in a formal way. Consider a particular variable i from X, m as a full data column vector X, = (x, i , . . . ,Xjjm+w)'. Assume that not all the observations are available. The vector X°h$ contains the available observations, which is only a subset of X , due to the ragged-edge problem and lower sampling frequencies of some of the variables. In general, we can formulate the relationship between observed and partly observed data by

X ohs — A Y

(4)

where A, is a matrix that accounts for missing values or mixed frequencies depending on the stock-flow properties of the variable (Stock/Watson 2 0 0 2 : appendix). In case no observations are missing, A, is just the identity matrix. In case one observation is missing at the end of the sample, the corresponding final row of the identity matrix is removed to ensure (4). Following M a r i a n o and M u r a s a w a (2003), G D P interpolation can be done by using the relationship for quarter-on-quarter GDP growth ytq = ( 1 + 2 L m + ZL]n + 2L]n + LAm)y?ytm = ?>tq with unobserved month-on-month G D P growth y™. This relationship implies a particular A, matrix. Given the A, for all variables X,-, Vi = 1 , . . . , N , the E M algorithm proceeds as follows: 1. Provide an initial (naive) guess of missing observations in X , and collect them together with the observations in variable x j 0 ' for i = 1 , . . . , N. These valuesjdeld a balanced dataset X ' 0 ' . Standard PC analysis provides initial monthly factors F ( 0 ' and loadings A(0). 2. E-step: An updated estimate of the missing observations for variable / is provided by

1

The E M algorithm is a general iterative solution for maximum likelihood estimation in the presence of missing or latent data and can generally be applied to different types of models. One of the first contributions in the context of dynamic factor models is Watson and Engle (1983).

34 • Christian Schumacher

The update consists of two components: The common component from the previous iteration pO-DAf" 1 ' plus the idiosyncratic component , distributed by the projection coefficient Aj(A,A-) _1 . Repeat the E-step for all i yielding again a dataset X"'. 3. M-step: Reestimate the factors and loadings, FW and A w by PC, and go to step 2 until convergence. After convergence, the EM algorithm provides monthly factor estimates Ftm, loadings A, as well as estimates of the missing values of the time series yielding a matrix of observations and interpolated values X. Forecast step The static nature of the model and PC estimation implies that forecasts cannot directly be derived in the estimation step and require a separate forecast step. Given the high-frequency factor estimates, Schumacher and Breitung (2008) provide various ways to forecast GDP. All the dynamic models chosen make use of the fact that the EM algorithm provides a monthly series of GDP with missing values interpolated in accordance with the factor model. The estimated monthly values of GDP, can be directly taken from the particular element of X tm . A direct forecast equation is y7m+h„ =

p

+

+

e tm+hm ,

(6)

which contains only monthly values on the left- and right-hand side of the equation. It provides forecasts for hm horizons. Note that L m is now a monthly lag operator, Lm¥lm = F (m _i. After computing the forecasts, time aggregation as in (4) is applied to the monthly forecast sequence in order to obtain quarterly forecast values. 3.1.2

Ciannone, Reichlin and Small (2008): Two-step estimation of a large factor model in state-space form and bridge equation

The factor estimation approach in Giannone et al. (2008) is based on a complete representation of a large factor model in state-space form. In the factor estimation step, the dataset Xtm includes only the N monthly indicators, but not GDP. The model contains the following two vector equations X(m=AFim+£m,

(7)

4»(L m )F ( „=B V t „.

(8)

Equation (7) is the static factor representation of X,m, and equation (8) specifies the monthly factor VAR with lag polynomial *P(Lm) = £)? =0 with ^ o = h- The ^-dimensional vector qtm contains the orthogonal dynamic shocks that drive the r factors, implying q and plugging them into the bridge equation.

3.1.3

Marcel lino and Schumacher (2010): Factor-MIDAS

The mixed-data sampling (MIDAS) regression approach (Ghysels et al. 2007; Ghysels/ Wright 2009; Clements/Galvao 2008, 2009), has received considerable attention in the recent literature as a forecast tool. The MIDAS framework consists of a regression of a low-frequency variable like GDP on a set of higher-frequency observed indicators, where distributed lag functions are employed to specify the dynamic relationship in a parsimonious way. The Factor-MIDAS approach in Marcellino and Schumacher (2010) exploits estimated factors rather than observed economic indicators as regressors. Factor estimation step The factor estimation methods used are based on a large set of monthly indicators. Three factor estimation methods are compared: First, the EM algorithm together with PC as discussed in section 3.1.1, but without GDP interpolation.

36 • Christian Schumacher

Second, the state-space estimator from Giannone et al. (2008) as discussed in section 3.1.2. Third, another convenient way of estimating factors from ragged-edge data is adopted from the construction of the CEPR 'New Eurocoin' composite indicator (Altissimo et al. 2 0 0 6 ) . This estimation technique is based on a vertical realignment of timeseries observations and dynamic PC. In detail, assume that variable i is released with a publication lag of kj months. Thus, given a dataset downloaded in period Tm + w, the final observation available of this time series is for period Tm + w — kj. The realignment proposed by Altissimo et al. (2006) is then simply a redefinition like xutm

= x

i f m

(10)

_ki,

for tin — , Tm + w. Applying this procedure to all the series z — 1 , . . . and harmonizing at the beginning of the sample, yields a balanced data set for X/m and tm = max({fc,-}j^j) + 1 , . . . , Tm + w. Then, Altissimo et al. (2006) propose dynamic PC to estimate the factors from X t m . As this dataset is balanced by construction, the estimation techniques by Forni et al. ( 2 0 0 5 ) directly apply. Forecast step Let us assume for simplicity that we have only one factor ftm for forecasting (r = 1). The Factor-MIDAS regression for forecast horizon bq quarters is = fio + P i b { L m , 0 i f £ l o + etm+hm, where the polynomial

b(Lm,8)

=

b(Lm,0)

± c ( k , 0 ) L l

(11)

is the exponential Almon lag with

c(k,0)

=

+

Eexp

(0ik

, +

( 1 2 )

02k2)

k=o

with 0 = {6\, 62}• In the Factor-MIDAS approach, the quarterly variable yt^hq is directly related to the monthly factor values f^ + u / and its monthly lags, where superscript (3) indicates that we have three times more factor values than observations for ytq- Note that no monthlyinformation of the factors is discarded in the regression, since the regressor b(Lm,0)f^\w is a function of lagged monthly factor values given that the lag operator Lm operates at the monthly frequency (Ghysels et al. 2 0 0 7 ; Andreou et al. 2 0 0 9 a ) for details. Given 61 and 62, the exponential lag function b(Lm,0) provides a parsimonious way to consider monthly lags of the factors as we can allow for large K to approximate the impulse response function of GDP growth from the factors. Nonlinear least squares (NLS) can be employed to estimate 0. The Factor-MIDAS forecast is given by yiq+bqi.T„,+w = Po + b(Lm,0)fjm+w. Therefore, the projection is based on the final values of the estimated factors from periods Tm + w, Tm + w — 1 , . . . . For the case of a larger number of factors r > 1, the MIDAS regression can easily be generalized by defining separate polynomials for each additional factor. 3.1.4

Comparison of two-step approaches

The factor estimation techniques presented above differ with respect to the factor dynamics and the treatment of data irregularities. The E M algorithm is based on static PC, whereas the new Eurocoin is based on dynamic PC. The parametric state-space factor model by Giannone et al. (2008) contains a factor VAR. Thus, the state-space factor model and the estimator based on dynamic PC contain a richer dynamic structure than

Forecasting with Factor Models Estimated on Large Datasets • 37

the E M algorithm together with PC for estimating the factors. However, as the discussion for balanced data models has shown, there is no clear empirical evidence dynamic approaches in general have a better forecast performance (Boivin/Ng 2 0 0 5 ) . T h e reason is that a more detailed specification of factor dynamics might increase potential misspecifications. T h e presence of missing values at the end of the sample can be tackled by all the factor estimation approaches mentioned above. However, interpolation of missing values within the sample can be done by using the state-space approach and the E M algorithm with PC, but not using vertical realignment of the time series as in Altissimo et al. ( 2 0 0 6 ) . Factor estimation and monthly interpolation o f G D P can be done by the E M algorithm together with PC only. If we assume that G D P is strongly related to the driving factors of the economy, a joint factor specification might be regarded as fruitful. In general, temporal disaggregation of various kinds can be considered by the E M algorithm together with PC. The frequency conversions are feasible if they can be cast in the form (4), which restricts time disaggregation to be linear as in M a r i a n o and Murasawa ( 2 0 0 3 ) , where the relationship for quarter-on-quarter G D P growth yt¡l = (1 + 2 L m + 3 + 2 Lm)y?Vtm = 3 i ? holds. However, note that this aggregation constraint is just an approximation to the exact relationship between quarter-on-quarter and month-on-month growth rates. In particular, if we denote the level of G D P as Y,q and ytq = \n(YtJYtq-\) holds, M a r i a n o and Murasawa ( 2 0 0 3 ) implicitly use lnY ( < = In Ytm + In Y i m _i + In Ytm-i, with multiplicative time aggregation of the levels, whereas the true relationship is additive time aggregation of the levels, i.e. l n Y t i = ln(Y í m + Y í j n _i + Y t m - i ) . A general solution to this non-linearity problem is provided by Proietti and M o a u r o ( 2 0 0 6 ) . However, apart from the E M algorithm with PC, none of the other estimation techniques can tackle the interpolation based on (4). T h e forecast equations presented above all can be used to forecast quarterly G D P with monthly factors. T h e factor estimation step using Factor-MIDAS from Marcellino and Schumacher ( 2 0 1 0 ) is very general and may involve different estimation techniques feasible for unbalanced ragged-edge data. T h e forecast models chosen by Schumacher and Breitung ( 2 0 0 8 ) , however, depend heavily on the interpolation o f monthly G D P using E M together with PC, and cannot be applied to the other factor estimation methods discussed above. The bridge equation employed by Giannone et al. ( 2 0 0 8 ) is based on time-aggregated quarterly data. This implies an information loss at the end of the sample, because the most recent observations of the factors are not used for estimating the equations. Furthermore, Andreou et al. ( 2 0 0 9 a ) have shown that M I D A S regressions actually nest regressions based on time-aggregated data. 3.2 Large state-space models jointly describing G D P and the factors In contrast to the two-step procedure discussed above, the state-space approach allows to estimate the factors and to forecast G D P in a single, coherent model framework. In general, state-space models can be estimated by iterative M L (Nunez 2 0 0 5 ; Aruoba et al. 2 0 0 9 ; Camacho/Perez-Quiros 2 0 1 0 a , b), but for a very large number of observables N it becomes practically infeasible and other estimation methods are called for. 3.2.1

Banbura and Rünstler (2010)

Banbura and Rünstler ( 2 0 1 0 ) follow the approach by Giannone et al. ( 2 0 0 8 ) and augment the state-space model consisting of the factor representation X t m = AF, m + and

38 • Christian Schumacher

the factor VAR *P(L„,)F,m = Brj, m , as stated in equations (7) and (8), by relationships that allow for temporal disaggregation of GDP. In particular, the two equations 1 Vt, = ^ + L

,m

m

+ L2m)y\

y™ = A,F t m + etm

(13) (14)

are added. Equation (13) relates quarterly G D P to estimated monthly GDP, implying a certain pattern of time aggregation. In particular, as G D P in the data is a quarter-onquarter growth rate and G D P is a flow variable, interpolation in the model yields a monthly growth rate o f G D P between the current month and three months before. As G D P is observed only once a quarter, this equation only holds for tm = 3 , 6 , . . . , Tm. Finally, (14) is the factor representation o f G D P at monthly frequency, introducing an idiosyncratic component for G D P etm, which is normally distributed with mean zero and constant variance. In line with the estimation procedure by Giannone et al. ( 2 0 0 8 ) as discussed in section 3 . 1 . 2 above, the model coefficients are obtained from outside the state-space model by estimating a equation consisting o f (13) and (14), which implies a regression for quarterly G D P growth dependent on time-aggregated quarterly factors. As in Giannone et al. ( 2 0 0 8 ) , initial factors are obtained by using PC applied to the balanced part of the monthly data. T h e estimates of the model coefficients are plugged into the complete state-space model and the Kalman smoother is applied to the dataset containing missing values and quarterly GDP. As ytq is part o f the observation vector, the smoother provides forecasts for GDP. As a by-product, interpolated values of 3-month G D P growth rates are also provided. Angelini et al. ( 2 0 0 8 ) discuss a model extension that uses time disaggregation as in the seminal work by M a r i a n o and M u r a sawa ( 2 0 0 3 ) with interpolation to month-on-month G D P growth. 3.2.2

Banbura, Giannone, and Reichlin (2010)

T h e state-space model proposed by Banbura et al. ( 2 0 1 0 ) consists of the factor representation (7) and the factor VAR from (8) as discussed before. Additionally, the idiosyncratic components are assumed to follow A R ( 1 ) processes. Quarterly G D P growth is an element of the observations vector, and its temporal disaggregation follows M a r i a n o and Murasawa ( 2 0 0 3 ) by using the relationship for quarter-on-quarter G D P growth ytq = (1 + 2 L m 4- 3 L ^ + 2L?m + Vi m = 3 t q as discussed in section 3 . 1 . 4 . Concerning factor estimation, Banbura et al. ( 2 0 1 0 ) avoid the estimation of the coefficients outside the state-space model as in Banbura and Runstler ( 2 0 1 0 ) by making use of the E M algorithm applied to the dynamic factor model. As in Banbura and M o d u g n o ( 2 0 1 0 ) or Raknerud et al. ( 2 0 1 0 ) , the E M algorithm can be used to estimate the factors and parameters in the model jointly in the presense o f missing observations and mixed-frequency data. In practice, the E M algorithm iterates between the Kalman smoother for estimating the factors conditional on estimates of the model parameters, and updating the parameters given the factor estimates. A formal derivation of the steps o f the E M algorithm can be found in Banbura et al. ( 2 0 1 0 : appendix). Because the idiosyncratic components are assumed to follow A R ( 1 ) processes, the E M algorithm becomes very time-consuming. Solutions to increase the computational efficiency in this case are discussed in Jungbacker et al. ( 2 0 0 9 ) following Jungbacker and K o o p m a n ( 2 0 0 8 ) .

Forecasting with Factor Models Estimated on Large Datasets • 39

3.2.3

Proietti (2008)

Proietti (2008) proposes to estimate a large, dynamic factor model with non-linear time aggregation of the quarterly variables. The model consists of the factor representation for the variables and a VAR for the factors. Quarterly GDP as well as other quarterly indicators are elements of the observations vector. Temporal disaggregation considers that the exact relationship between the quarterly and monthly GDP level is additive according to Yi = Ytm + + Ytm-2, as GDP is typically regarded as a flow variable. This implies that the quarterly growth rates are a non-linear function of monthly growth rates. Proietti (2008) provides a modified Kalman smoother and an EM algorithm with nested loops to estimate the factors and model parameters under the non-linear aggregation constraint. 3.2.4

Comparison of state-space models jointly describing G D P and the factors

As all the approaches belong to the class of state-space models, the Kalman smoother is the estimation method of the factors given the model parameters. Concerning parameter estimation, both Banbura et al. (2010) and Proietti (2008) provide a solution for ML estimation of large factor models by using the EM algorithm. In that respect, these two approaches directly expand on the two-step procedure in Banbura and Rünstler (2010). In terms of factor model structure, all the models include a VAR for the factors. Banbura et al. (2010) are most general, as they also include AR(1) idiosyncratic components. All the state-space models operate at monthly frequency and account for mixed-frequency data in different ways. The temporal disaggregation by Proietti (2008) fulfills the non-linear aggregation constraint of the flow variable GDP exactly, whereas Banbura et al. (2010) rely on the approximation by Mariano and Murasawa (2003). The model in Banbura and Rünstler (2010) is based on 3-month growth rates interpolated from quarterly GDP and implies a smoother interpolation of GDP. All three approaches can tackle missing values at the end of the sample as well as in-sample. 3.3 Comparison of two-step approaches to state-space models jointly describing G D P and the factors

The two-step approaches differ quite substantially from the state-space models jointly describing GDP and the factors. Comparing the two groups, the state-space models can handle a richer model structure than the two-step approaches. For example, exact temporal interpolation as in Proietti (2008) or AR(1) idiosyncratic components as in Banbura et al. (2010) cannot easily be implemented in the two-step approaches. Furthermore, a fully specified state-space model allows for statistical inference of the parameters, which might help to respecify the forecast model in a proper way. On the other hand, more complicated model structures might be subject to misspecification, an argument that has also been raised in favour of simple PC estimators versus dynamic PC estimators for balanced data (Boivin/Ng, 2005). Furthermore, it has been shown in Marcellino and Schumacher (2010: section III), how Factor-MIDAS regressions can be regarded as approximations to the optimal forecast equation, if the true structure of the factor model is unknown. Thus, it is in the end an empirical question to what extent a rich structure of a state-space model can represent the true DGP.

40 • Christian Schumacher

Another argument in favour of system approaches based on state-space models is the capability to produce uncertainty measures and forecast contributions. Many institutions, in particular central banks, regularly publish forecast confidence intervals in addition to point forecasts. Whereas the properties of factor forecasts with balanced data are well known (Bai/Ng 2006), this does not hold for the two-step approaches applied to ragged-edge and mixed-frequency data. A related interesting application of factor models estimated on ragged-edge data is the estimation of the informational content of asynchronous data releases. Following the work in Giannone et al. (2008), the state-space models can be used to identify those indicators in the large dataset that contribute most to the forecast performance (Banbura/Riinstler 2010; Aastveit/Trovik 2007; Camacho/ Perez-Quiros 2010a, b; Matheson 2010; Siliverstovs/Kholodilin 2010). A related relevant issue is the analysis of data news, which explores the evolution of forecast updates due to data releases over time (Banbura et al. 2010). An advantage of the two-step approach Factor-MIDAS is its ability to forecast GDP with indicator data sampled at a much higher frequency than GDP. For example, Andreou et al. (2009b) provide an example of how MIDAS regressions can be used to exploit financial data sampled at daily frequencies for macroeconomic forecasting. Within the MIDAS approach, it is possible to exploit daily data in parallel to monthly data by using a multiple MIDAS regression, where factors are estimated from daily and monthly data Table 1 Features of factor models estimated on balanced data Model

Factor model specifics and estimation estimation method VAR for factors AR for idiosyncratic components Forecast equation/system LF ADL model LF model with theoretical restrictions ADL model of interpolated HF GDP LF bridge equation MIDAS regression joint model of GDP and factors Consideration of data irregularities single-frequency data only GDP LF, factors HF approximate temporal disaggregation exact non-linear temporal disaggregation missing data at the end of sample missing data within the sample applicable to highest-frequency data

Stock/Watson (2002)

Forni et al. (2005)

Kapetanios/ Marcellino (2009)

PC

2S-DPC

subspace

x

x x

x

x

x

Note: The abbreviations used in the table are chosen as follows. LF denotes low frequency, HF denotes high frequency. As regards factor estimation techniques, 2S-DPC refers to the dynamic PC used in Forni etal. (2006), PC is the principal components estimator in the static model following Stock and Watson (2002) from the main text. The model features in the second column refer to the empirical application in the paper by Stock and Watson (2002), which is essentially based on balanced data. The EM algorithm from the appendix in Stock and Watson (2002) can take into account unbalanced data and is presented in the next table as adapted for forecasting GDP in real time by Schumacher and Breitung (2008). The entries not marked in this table are explained in the note of next table.

Forecasting with Factor Models Estimated on Large Datasets • 41 u ' v a> o , TJ O »

£ 00 «1 o a o O «« ¿ C I QQ "I ^ O "ft O c-Q 1810oIT < */>0 0 co Ci TJ o

S g f i j -5

l/l u_

Hfic ^ i; « L. O p C5 nJujI- «a)S ^ •