My (rough) notes of the first part of the Theory and Algorithms for Forecasting Non-Stationary Time Series NIPS tutorial on 12/15/2016.

  • Time series prediction appears in many real-world applications
    • stocks
    • econ variables
    • weather
    • sensors
    • earthquakes
    • energy demand
    • signal processing
    • sales forecasting
  • Timeseries are challenging - different than what we typically see in machine learning.
  • classic framework
    • postulate generative model
    • use given sample to estimate unknown parameters
    • use estimated models to make predictions
  • Autoregressive models: next observation is weighted linear combination of past values.
  • Moving average model: observation is a weighted linear combination of uncertainty at previous times
  • ARMA model: combines autoregressive and moving average models.
  • Stationarity: a sequence of random variables is stationary if their distribution is invariant wrt time
  • Weak Stationarity: only the first two moments (mean and variance) must be invariant wrt time.
  • Lag operator: \(L(Y_t) = Y_T-1\)
  • ARIMA: ARMA models of a process that applies \((1-L)^D\) to \(Y_T\).
  • Different withods for estimating model params
    • Maximum likelihood
    • method of moments
    • requires additional parametric assumptions
    • conditional / unconditional least squares
    • requires additional assumptions
  • Summary:
    • Many generative models
    • Learning guarantees are asymptotic
    • Model needs to be correctly specified
    • non-stationarity needs to be modeled explicitly