ABMs leverage the flexibility offered by computational simulations in order to avoid a number of problematic assumptions required by traditional modelling approaches. As a result, recent years have seen the emergence of a growing class of realistic ABMs capable of more accurately replicating the heterogeneity and non-linearity that characterise real economic systems.
Unfortunately, this increased flexibility has also come at a cost: many ABMs, particularly large-scale macroeconomic variants, are not formulated in terms of smooth functions. For this reason, fitting ABMs to empirical data and estimating appropriate values for their parameters and latent variables remains an open problem and many otherwise excellent ABM contributions often lack a strong empirical component. Our work has therefore aimed to improve the statistical inference toolbox available for economic ABMs in a number of key areas:
- Comparative studies
- Parameter estimation method development
- Initialisation
- Developing statistical inference techniques for complex simulation models
Comparative studies
Increased emphasis has recently been placed on the development of parameter estimation methodologies, but many of these approaches have not been benchmarked against existing alternatives. This often leaves the modeller with a wide array of methods to choose from and difficulty in choosing the most effective for the task at hand. This led us to conduct one of the first comprehensive comparative studies of economic ABM calibration methods.
Parameter estimation method development
Most traditional ABM estimation techniques rely heavily on condensing model outputs and empirical data into sets of arbitrarily-chosen summary statistics. More often than not, this leads to much of the information contained in the data being discarded. We focused on the development of new methods that either avoid the need for such a transformation or provide automatic and effective ways to do so.
Initialisation
ABMs often incorporate large sets of latent variables that play a crucial role in defining the overall model dynamics. Much like free parameters, initial values for these latent variables must be estimated from empirical data. Importantly, however, the number of latent variables present in state-of-the-art ABMs may be several orders of magnitude larger than the number of free parameters, introducing additional complications. We therefore focused on the adoption of recent innovations in the general data assimilation literature to more effectively tackle this challenging problem.
Developing statistical inference techniques for complex simulation models
One of the key modelling tools in the Complexity Economics group is agent-based models: simulations that model the interactions between many heterogenous and autonomous “agents”, which are simplified representations of humans, firms, banks, etc. The goal with such models is to reproduce some aspect of empirically observed behaviour, and their ability to do so typically requires careful tuning of their parameters. Traditional approaches to parameter inference are infeasible, however, due to the complexity of these models. As such, the development of methods that are able to handle their complexity is a pertinent and active field of research in computational statistics.
Key findings:
- Simple Bayesian estimation procedures are able to outperform a number of sophisticated, frequentist simulated minimum distance (SMD) methods.
- Through a series of benchmarking experiments, our work indicates that neural networks and recent developments in stochastic analysis provide useful tools for generating accurate parameter inferences across a range of simulation models.
Impact:
- Joel Dyer's research on Calibrating agent-based models to microdata with graph neural networks named Best Proposal Paper at ICML AI for Agent-Based Modelling (AI4ABM) Workshop 2022
- Donovan Platt wins Alan Turing Institute Prize for Best Student Methodology Paper at the 24th Annual Workshop on Economic Science with Heterogeneous Interacting Agents (WEHIA).
Funders include:
Baillie Gifford, Conacyt-SENER (Sustentabilidad Energética scholarship), EPSRC CDT in Industrially Focused Mathematical Modelling (EP/L015803/1) in collaboration with Improbable
External collaborators:
Sebastion M. Schmon (Improbable), Patrick Cannon (Improbable).