When working with time series data random variation can sometimes drown out the trends that exist in our data. With the use of smoothing methods we lower this random variation in order to locate the trends more easily. With our smoothed values we can also form predictions from our data.
Simple moving average is the most basic smoothing method. It works by averaging the past k observations in order to create a smoothed estimate. K is chosen before hand and can take values from 1 to n, where n is the number of observations in our time series.
If k=1:
No smoothing occurs, it simply takes the average of each individual observation which is the same as a number divided by 1.
If k=n:
All observations will be averaged together simply outputting a smoothed series with 1 observation, which would be the mean of the original series.
K should be set just high enough in order to try to lower the variation to see the true trends. If k is set too high, the real trends of the data can be lost.
Simple Moving Average:
\(\hat{s}_{t}=\frac{y_{t}+y_{t-1}+...+y_{t-k+1}}{k}\)
or
\(\hat{s}_{t}=\hat{s}_{t-1}+\frac{y_{t}-y_{t-k}}{k}\)
Simple moving average can also be used for predictions. In order to make a prediction, we simply predict the last available smoothed observation for all future values.
\(\hat{y}_{t}=b_{0}\)
\(b_{0}=\hat{s}_{n}\)
With simple moving average being such a simple method, it makes naive predictions in the sense that it expects no trends over time. It simply predicts the last recorded smoothed observation. Because of this, simple moving average can not be used when trends exist in the data. (James et al.)
Double smoothing average adds another layer to simple moving average by smoothing our already smoothed estimates. Our predictions will now include another parameter. This allows us to use smoothing to predict a series when a trend is present.
Double Smoothing Average:
\(\hat{s}_{t}^{2}=\frac{\hat{s}_{t}+\hat{s}_{t-1}+...+\hat{s}_{t-k+1}}{k}\)
k - Number of observations being averaged
For predictions:
\(\hat{y}_{t}=b_{0}+b_{1}l\)
\(b_{0}=\hat{s}_{n}\)
\(b_{1}=\frac{2(\hat{s}_{n}-\hat{s}_{n}^{(2)})}{k-1}\)
In time series forecasting often the most recent of observations are the most important in predicting future values. Exponential smoothing adds a tuning parameter, w, to our simple moving average which allows us to put more weight on the most recent observations than observations farther in the past. Smaller values of w put more weight on the most recent observations creating a smaller smoothing effect. Larger values of w put less weight on most recent observations creating a smoother series. (James et al.)
Exponential Smoothing:
\(\hat{s}_{t}=(1-w)(y_{t}+wy_{t-1}+...+w^{t}y_{0})\)
or
\(\hat{s}_{t}=(1-w)y_{t}+w\hat{s}_{t-1}\)
w - Value between 0-1
For predictions with exponential smoothing it is performed just like simple moving average, where the same prediction is made for each future time interval.
Predictions:
\(\hat{y}_{t}=b_{0}\)
\(b_{0}=\hat{s}_{n}\)
Just like simple moving average it should not be used when trends are present in the data. If there are trends present in the data, double exponential smoothing can be used.
The value of w can be chosen by choosing the value that minimized the sum of squared one step prediction errors.
sum of squared one step prediction errors:
\(SS(w)=\sum_{t=1}^{T} (y_{t}-\hat{s}_{t-1})\)
Double exponential smoothing applies exponential smoothing to the already exponentially smoothed series. It can be used when there is a trend present in the data.
Double Exponential Smoothing:
\(\hat{s}_{t}^{(2)}=(1-w)(\hat{s}_{t}+w\hat{s}_{t-1}+...+w^{t}\hat{s}_{0})\)
or
\(\hat{s}_{t}^{(2)}=(1-w)\hat{s}_{t}+w\hat{s}_{t-1}^{(2)}\)
In order to make predictions:
\(\hat{y}_{t}=b_{0}+b_{1}l\)
\(b_{0}=2\hat{s}_{n}-\hat{s}_{n}^{(2)}\)
\(b_{1}=\frac{1-w}{w}(\hat{s}_{n}-\hat{s}_{n}^{(2)})\)
James, G., Witten, D., Hastie, T., & Tibshirani, R. (n.d.). An introduction to statistical learning: With applications in R.
National Institute of Standards and Technology. (n.d.). Engineering Statistics Handbook. 6.4.2.1. single moving average. Retrieved October 4, 2022, from https://www.itl.nist.gov/div898/handbook/pmc/section4/pmc421.htm