State Space Models and the Kalman Filter

Erik Cheever, a UPenn graduate and a professor of engineering at Swathmore, has very attractive elementary tutorial on the Kalman filter. This is the first place that I have found that makes clear those "systems diagrams" that one finds in so many papers on the Kalman filter. Systems diagrams are ubiquitous, but I always found them to be totally obscure until I read Cheever's page.

Another benefit of Cheever's presentation is that it focuses on the one-dimensional case. This lets one minimize the matrix algebra and sharpens the focus on two important ideas: (1) the notion of a "gain" and (2) the basic trick of a "predictor/corrector" algorithm. These ideas combine nicely with systems diagrams to suggest many variations on Kalman's basic theme.

Max Welling has a collection of notes on machine learning that are at about the level of our class, and one of these notes gives a detailed drivation of the Kalman Filter equations. A certain amount of (ugly?) algebra is unavoidable, but this note gives as clear a derivation of the vector-valued KF recurrsions as any source that I have found.

The exposition of Meinhold and Singpurwalla (1983) is brief and focuses on a statistical audience. It is definitely worth a look, but, in the prusuit of brevity, it leaves behing some useful engineering intuition.

Greg Welch and Gary Bishop maintain a useful site on the Kalman Filter which provides tutorials, software links, and links to original sources, such as Rudolph E. Kalman's seminal article. The introductory tutorial by Welsh and Bishop is widely used, and it will provide one resource for our discussion.

For a broader perspective, you can consult the Wikipedia article on the Kalman filter. The article may at first seem a little intimidating, but skimming it provides good sense of the big picture.

Application Articles

If you are a football fan or a person who likes to think about the sorts of things that happen in Las Vegas, you might enjoy

M. E. Glickman and H. S. Stern, "A State-Space Model for National Football League Scores," J. Amer. Statistist. Soc.(1998), 93, 25--36.

but if your tastes run to larger markets, such as soy beans or fish meal, you may prefer

T. Vukina and J. L. Anderson, "A State-Space Forecasting Approach to Optimal Intertemporal Cross-Hedging," Amer. J. Agr. Econ. 75 (1993), 416--424.

or perhaps something that deals with interest rates

S. H. Babbs and K. B. Nowman (1999), "Kalman Filtering of Generalized Vasicek Term Structure Models," J. Fin. and Quant. Anal. 34 (1), 115--130.

 

General or Methodological Articles

R. Kohn and C. G. Ansley, "A note on Kalman filtering for the seasonal moving average model," Biometrika (1984), 77, 2, 648--650.

G. W. Morrision and D. H. Pike, "Kalman Filtering Applied to Statistical Forecasting," Management Science (1977), 23, 7, 768--774.