Time Series

1. Purpose & Motivation

1.1 Why time series matters

  • Many real-world phenomena evolve over time (stock prices, weather, sensor readings)
  • Temporal ordering carries critical information that standard methods ignore
  • Forecasting future values is one of the most demanded capabilities in industry

1.2 The core problem it addresses

Model and predict data where observations are ordered in time, and the sequential structure contains meaningful patterns such as trends, seasonality, and autocorrelation.

2. Definition & Core Idea

2.1 Formal definition

A time series is a sequence of data points {x_t} indexed by time t = 1, 2, ..., T, where the ordering is meaningful and adjacent observations are typically correlated. The goal is often to forecast future values x_{T+1}, x_{T+2}, ... based on past observations.

2.2 Intuitive mental model

"A diary of measurements -- temperature every hour, sales every day -- where yesterday's value gives you a strong hint about today's."

3. Key Components

3.1 Decomposition

Component Description Example
Trend Long-term increase or decrease Growing GDP over decades
Seasonality Regular repeating patterns Ice cream sales peak in summer
Cyclical Non-fixed-period fluctuations Business cycles
Residual Random noise after removing other components Unpredictable variation

3.2 Stationarity

  • A time series is stationary if its statistical properties (mean, variance) do not change over time
  • Most models assume stationarity; non-stationary data must be transformed (differencing, detrending)

3.3 Autocorrelation

  • Correlation of the series with its own lagged values
  • ACF (Autocorrelation Function) and PACF (Partial ACF) guide model selection

4. Classical Models

Model Approach
AR Predict from weighted past values
MA Predict from weighted past errors
ARIMA Combines AR, differencing, and MA
SARIMA ARIMA with seasonal components
Exponential Smoothing Weighted average with exponential decay

5. Modern Approaches

5.1 Machine learning on time series

  • Feature engineering: Lag features, rolling statistics, calendar features
  • Tree-based models: XGBoost, LightGBM on engineered features
  • Requires careful handling of temporal ordering to avoid data leakage

5.2 Deep learning

  • RNNs / LSTMs: Process sequences step by step, maintain memory
  • Temporal CNNs: Apply convolutions along the time axis
  • Transformers: Attention-based models for long-range dependencies

6. Evaluation Considerations

6.1 Time-aware validation

  • Never shuffle time series data; validation must come after training chronologically
  • Use walk-forward or expanding window cross-validation
  • Standard k-fold CV causes data leakage in temporal settings

6.2 Common metrics

  • MAE (Mean Absolute Error), RMSE (Root Mean Squared Error)
  • MAPE (Mean Absolute Percentage Error)
  • sMAPE (Symmetric MAPE)

7. Strengths & Limitations

7.1 Strengths of time series modeling

  • Captures temporal dynamics that cross-sectional models miss
  • Enables forecasting, anomaly detection, and trend analysis
  • Rich toolkit from both statistics and ML

7.2 Limitations

  • Stationarity assumptions often violated in practice
  • Long-range dependencies are hard to capture
  • Sensitive to missing values and irregular sampling

8. One-Sentence Summary

Time series data consists of temporally ordered observations where sequential patterns like trends and seasonality are modeled to forecast future values.

9. Related Knowledge Pages