---
title: "STAT 436 / 536 - Lecture 12"
output: pdf_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
knitr::opts_chunk$set(warning = FALSE)
knitr::opts_chunk$set(message = FALSE)
knitr::opts_chunk$set(fig.align= 'center')
knitr::opts_chunk$set(fig.height= 3)
knitr::opts_chunk$set(fig.width = 5)
library(tidyverse)
library(gridExtra)
library(readr)
library(ggfortify)
```

## Stationary Models

Time series data sets often have components that can be identified and modeled in a deterministic fashion. In particular, using regression we can fit models with:
\vfill

\vfill

\vfill

\vfill
\vfill
- As a result, with a well fit regression model, there should not be remaining deterministic features 
\vfill
- Two examples referenced in the textbook would be:

\vfill
\vfill

\vfill
- We have seen problems related to inference when serial correlation is present in the data; hence, our goal is fit a model to address the correlation in the residuals for valid inference.
\vfill
\newpage

- One approach that we have seen is to use an autoregressive process, where AR(p) denotes an autoregressive process of order p
$$x_t = \alpha_1 x_{t-1} + \dots + \alpha_p x_{t-p}$$
\vfill
- AR processes can be used for modeling the time series itself, with covariates, or for the residuals.
\vfill

#### Stationary Processes
We have talked about stationarity, but now we focus on more details.
\vfill
- A time series model $\{x_t\}$ is *strictly stationary* 
\vfill
\vfill
- Strict stationarity implies that 
\vfill
- Strict stationarity also implies that 
\vfill
- It is possible for a series to have constant mean and variance, and also have an autocorrelation function that depends only on the lag $k$, but not be strictly stationary. 
\vfill
- Stationary is a desirable property of modeling time series data. Fitting a model with stationarity assumptions, implies that the time series (could be a residual one) is indeed a realization from a stationary process.
\vfill
- Therefore, the series should be checked to determine if evidence of a trend or seasonal effects persist. Linear modeling (regression) can be used fit the trend or seasonal patterns. 
\vfill
- Thus, after checking residual diagnostics, it is reasonable 
\newpage

### Moving Average Models

- We have seen AR processes, but another way to handle serial correlation is with a moving average (MA) process.
\vfill
- Formally, a MA process of order q is a linear combination of the current white noise term and the q most recent white noise terms


\vfill
- Recall and AR process of order $p$ (with white noise) can be written as

\vfill 

##### Moving Average Computation

1. Simulate a moving average process of order 2, finish code below.
\vfill
```{r}
## SETUP PARAMETERS
q <- 2
beta <- c(.6, .1)
sigma <- 1
num.time <- 100

# INITILIZATION
x <-  rep(0, num.time)
w <- rnorm(num.time , mean = 0, sd = sigma )

# SIMULATE TIME PTS 1 AND 2
x[1] <- w[1]
x[2] <- w[2] + beta[1] * w[1]

# NOW SIMULATE TIME PTS 3 - NUM.TIME
for (time.pt in 3:num.time){
  # x[time.pt] <- 
}
```

2. Simulate several realizations from this MA process and track the mean and variances. Can you determine what the mean and variance of the process should be? (Either mathematically or computationally (hint $\beta$ matters for one of these))
\vfill

3. Select several different values of beta and plot the resultant process. Can you summarize the impacts of the $\beta$ values?
\vfill
4. Use the ACF and PACF to look the serial correlation in the series.
\newpage
- The moving average process can also be written using the backshift operator __B__.
\vfill
\vfill
- The MA process consists of a linear combination of a finite sum of stationary white noise terms, hence the MA process is stationary with time-invariant mean and autocorrelation.
\vfill
- The mean can be derived as:

\vfill
- Similarly the variance can be computed as:

\vfill
- The autocorrelation function for lag $k$ has three possible values:
\vfill
1. when $k=0$, $\rho_k =$
 \vfill
2. when $k > q$, $\rho_k =$
\vfill
3. for $k = 1, \dots, q$, $\rho_k =$
where $\beta_0$ is defined as 1.
\vfill
- Consider two MA(1) processes with coefficients equal to $\beta = .5$ and $\beta = 2$ and calculate the correlation $\rho_1$. 
\newpage
- The idea of 
\vfill
- An MA process in *invertible* if it can be expressed as a stationary AR process of infinite order without an error term. In other words, using
the MA process $x_t = (1 + \beta B) w_t$, then $x_t = w_t + \beta w_{t-1}$ and $w_{t-1} = x_{t-1} - \beta w_{t-2}.$
Using this idea in a recursive manner
$$x_t = w_t + \beta x_{t-1}  - \beta^2 x_{t-2} + \beta^3 x_{t-3} - \beta^4 x_{t-4} + \dots$$
with $|\beta| < 1$.
\vfill
- Recall the characteristic equation was used to determine stationarity of an AR process. 
\vfill
- Most importantly, a MA(q) process is unique only if invertibility is imposed.
\vfill

#### Fitting MA processes

- The `arima` function in r can be used to fit an MA process.
\vfill
- The `arima` function has an order argument that is : "A specification of the non-seasonal part of the ARIMA model: the three integer components (p, d, q) are the AR order, the degree of differencing, and the MA order."
\vfill
- Furthermore, "The exact likelihood is computed via a state-space representation of the ARIMA process, and the innovations and their variance found by a Kalman filter. The initialization of the differenced ARMA process uses stationarity and is based on Gardner et al (1980)."
\vfill
- Finally return to your simulation and fit the simulated MA models.