Cosma Shalizi is an assistant professor of statistics at Carnegie Mellon University, where his research focuses on aspects of the statistical analysis of complex systems: nonlinear prediction algorithms, heavy-tailed distributions, contagion in networks, and self-organizing processes. Previously, he was a post-dcotoral fellow at the University of Michigan's Center for the Study of Complex Systems and at the Santa Fe Institute, where he is now an external faculty member. He got is Ph.D. in theoretical physics from the University of Wisconsin-Madison in 2001.
We derive generalization error bounds -- bounds on the expected inaccuracy of the predictions -- for traditional time series forecasting models. Our results hold for many standard forecasting tools including autoregressive models, moving average models, and, more generally, linear state-space models. These bounds allow forecasters to select among competing models and to guarantee that with high probability, their chosen model will perform well without making strong assumptions about the data generating process or appealing to asymptotic theory.
Cosma Shalizi urges economists to stop doing what they are doing: Fitting large complex models to a small set of highly correlated time series data. Once you add enough variables, parameters, bells and whistles, your model can fit past data very well, and yet fail miserably in the future. Shalizi tells us how to separate the wheat from the chaff, how to compensate for overfitting and prevent models from memorizing noise. He introduces techniques from data mining and machine learning to economics -- this is new economic thinking.
This project will bring new mathematical tools and ideas from high-dimensional statistics to bear on the problem of creating reliable macroeconomic forecasting models, ones which not only predict well out of sample but also support the sort of counterfactual reasoning needed for scientific explanation and policy evaluation. The proposed work draws extensively on, but goes beyond, the results of the researchers’ previous Institute grant on application of statistical learning methods to macroeconomics.
Cosma Shalizi, Mark Schervish, and Daniel McDonald of Carnegie Mellon University were awarded a grant by the Institute for New Economic Thinking to extend proven techniques in statistical learning theory so that they cover the kind of models and data of most interest to macroeconomic forecasting. The dominant modeling traditions among academic economists, namely dynamic stochastic general equilibrium (DSGE) and vector autoregression (VAR) models, both spectacularly failed to forecast the financial collapse and recession which began in 2007, or even to make sense of its course after the fact.