Comparison of macroeconomic simulation models, particularly agent-based models (ABM), with more traditional approaches such as VAR and DSGE models has long been identified as a problematic issue in the literature. First of all, this is because many such simulations have been developed following the great recession with a clear aim to inform policy, and secondly because the methodological tools required for validating these models on empirical data are still in their infancy.
The paper aims to address this issue by developing and testing a comparison framework for macroeconomic simulation models based on a multivariate extension of the univariate Markov Information Criterion (MIC) originally developed in Barde (2017). The MIC is designed to measure the informational distance between a set of models and some empirical data by mapping the simulated data to the markov transition matrix of the underlying data generating process, and is proven to perform optimally (i.e. the measurement error has a zero expectation) for all models reducible to a markov process. As a result, not only can the MIC provide a measure of distance solely on the basis of simulated data, but it can do it for a very wide class of data generating processes.
The paper first presents the strategies adopted to address the computational challenges that arise from extending the methodology to multivariate settings and validates the extension on VAR and DGSE models. In a second part, the paper carries out a comparison of the benchmark ABM of Caiani et al. (2016) and the DGSE framework of Smets and Wouter (2007), which to our knowledge, this is the first direct comparison between a macroeconomic ABM and a DGSE model.