ABMs leverage the flexibility offered by computational simulations in order to avoid a number of problematic assumptions required by traditional modelling approaches. As a result, recent years have seen the emergence of a growing class of realistic ABMs capable of more accurately replicating the heterogeneity and non-linearity that characterise real economic systems.
Unfortunately, this increased flexibility has also come at a cost: many ABMs, particularly large-scale macroeconomic variants, are not formulated in terms of smooth functions. For this reason, fitting ABMs to empirical data and estimating appropriate values for their parameters and latent variables remains an open problem and many otherwise excellent ABM contributions often lack a strong empirical component. Our work has therefore aimed to improve the statistical inference toolbox available for economic ABMs in a number of key areas:
- Comparative studies
- Parameter estimation method development
- Initialisation
- Developing statistical inference techniques for complex simulation models
Comparative studies
Increased emphasis has recently been placed on the development of parameter estimation methodologies, but many of these approaches have not been benchmarked against existing alternatives. This often leaves the modeller with a wide array of methods to choose from and difficulty in choosing the most effective for the task at hand. This led us to conduct one of the first comprehensive comparative studies of economic ABM calibration methods.
Parameter estimation method development
Most traditional ABM estimation techniques rely heavily on condensing model outputs and empirical data into sets of arbitrarily-chosen summary statistics. More often than not, this leads to much of the information contained in the data being discarded. We focused on the development of new methods that either avoid the need for such a transformation or provide automatic and effective ways to do so.
Initialisation
ABMs often incorporate large sets of latent variables that play a crucial role in defining the overall model dynamics. Much like free parameters, initial values for these latent variables must be estimated from empirical data. Importantly, however, the number of latent variables present in state-of-the-art ABMs may be several orders of magnitude larger than the number of free parameters, introducing additional complications. We therefore focused on the adoption of recent innovations in the general data assimilation literature to more effectively tackle this challenging problem.
Developing statistical inference techniques for complex simulation models
One of the key modelling tools in the Complexity Economics group is agent-based models: simulations that model the interactions between many heterogenous and autonomous “agents”, which are simplified representations of humans, firms, banks, etc. The goal with such models is to reproduce some aspect of empirically observed behaviour, and their ability to do so typically requires careful tuning of their parameters. Traditional approaches to parameter inference are infeasible, however, due to the complexity of these models. As such, the development of methods that are able to handle their complexity is a pertinent and active field of research in computational statistics.
Key findings:
- Simple Bayesian estimation procedures are able to outperform a number of sophisticated, frequentist simulated minimum distance (SMD) methods.
- Through a series of benchmarking experiments, our work indicates that neural networks and recent developments in stochastic analysis provide useful tools for generating accurate parameter inferences across a range of simulation models.
Key publications:
- Dyer, J., Cannon, P., Farmer, J. D., & Schmon, S. M. (2024). Black-box Bayesian inference for economic agent-based models. Journal of Economic Dynamics and Control, Volume 161.
- Dyer, J., Quera-Bofarull, A., Chopra, A., Farmer, J.D., Calinescu, A. and Wooldridge. M. (2023). Gradient-Assisted Calibration for Financial Agent-Based Models. In Proceedings of the Fourth ACM International Conference on AI in Finance (ICAIF '23).
- Quera-Bofarull, A.,Chopra, A., Calinescu, A., Wooldridge, M., Dyer, J. (2023). Bayesian calibration of differentiable agent-based models, ICLR Workshop AI for Agent-Based Modelling.
- Dyer, J., Cannon, P., Farmer, J.D. & Schmon, S.M. (2022). Calibrating agent-based models to microdata with graph neural networks, ICML Workshop for AI in Agent-based Modelling
- Dyer, J., Cannon, P., & Schmon, S. M. (2022). Amortised Likelihood-free Inference for Expensive Time-series Simulators with Signatured Ratio Estimation. 25th International Conference on Artificial Intelligence and Statistics (AISTATS).
- Dyer, J., Cannon, P., & Schmon, S. M. (2021). Approximate Bayesian Computation with Path Signatures. arXiv preprint.
- Platt, D. (2021). Bayesian Estimation of Economic Simulation Models Using Neural Networks. Computational Economics.
- Platt, D. (2020). A comparison of economic agent-based model calibration methods. Journal of Economic Dynamics and Control, Volume 113, 103859, ISSN 0165-1889.
Impact:
- Joel Dyer's research on Calibrating agent-based models to microdata with graph neural networks named Best Proposal Paper at ICML AI for Agent-Based Modelling (AI4ABM) Workshop 2022
- Donovan Platt wins Alan Turing Institute Prize for Best Student Methodology Paper at the 24th Annual Workshop on Economic Science with Heterogeneous Interacting Agents (WEHIA).
Funders include:
Baillie Gifford, Conacyt-SENER (Sustentabilidad Energética scholarship), EPSRC CDT in Industrially Focused Mathematical Modelling (EP/L015803/1) in collaboration with Improbable
External collaborators:
Sebastion M. Schmon (Improbable), Patrick Cannon (Improbable).
Researchers involved:
Doyne Farmer, Joel Dyer, Blas Kolic, Donovan Platt