This site is an archived version of Voteview.com archived from University of Georgia on May 23, 2017. This point-in-time capture includes all files publicly linked on Voteview.com at that time. We provide access to this content as a service to ensure that past users of Voteview.com have access to historical files. This content will remain online until at least January 1st, 2018. UCLA provides no warranty or guarantee of access to these files.

### 45-734 PROBABILITY AND STATISTICS II (4th Mini AY1997-98)ARIMA Examples (Part 1)

I have created several ARIMA series for us to study. These examples are all stationary series so we do not need to do any differencing [ARIMA(p,0,q) examples]. Unfortunately, with ARIMA interpreting the ACFs and PACFs is much more tricky. Indeed, we have to proceed mostly by trial and error until we get a reasonable looking model. The best way to proceed is to start with the IDENT command and make a guess about the model, estimate the model, look at the residuals, and add more terms if the pattern of the spikes suggests it. This process is continued until the residuals are white noise.
1. First Example

IDENT(5) Y1
```                      Correlogram of Y1
==============================================================
Date: 04/21/98   Time: 16:42
Sample: 1 100
Included observations: 100
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
. |***    |      . |***    |  1 0.444 0.444 20.306 0.000
***| .     |  ******| .     |  2-0.431-0.782 39.652 0.000
*****| .     |      . |**     |  3-0.603 0.275 77.871 0.000
.*| .     |     ***| .     |  4-0.153-0.345 80.352 0.000
. |**     |      . |**     |  5 0.302 0.319 90.148 0.000
==============================================================
```
This pattern is somewhat ambiguous but clearly we have multiple terms present. Lets be conservative and start off by estimating up to a MA(2).
```============================================================
LS // Dependent Variable is Y1
Date: 04/21/98   Time: 16:44
Sample: 1 100
Included observations: 100
Convergence achieved after 23 iterations
============================================================
Variable      CoefficienStd. Errort-Statistic  Prob.
============================================================
C           0.067147   0.239710   0.280117   0.7800
MA(1)         0.835517   0.032102   26.02710   0.0000
MA(2)        -0.152868   0.002619  -58.37944   0.0000
============================================================
R-squared            0.574891    Mean dependent var 0.150504
Adjusted R-squared   0.566126    S.D. dependent var 2.163285
S.E. of regression   1.424937    Akaike info criter 0.737796
Sum squared resid    196.9532    Schwarz criterion  0.815951
Log likelihood      -175.7836    F-statistic        65.58840
Durbin-Watson stat   1.284884    Prob(F-statistic)  0.000000
============================================================
Inverted MA Roots          .1      -.99
============================================================

Correlogram of Residuals
==============================================================
Date: 04/21/98   Time: 16:45
Sample: 1 100
Included observations: 100
Q-statistic probabilities adjusted for 2 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
. |***    |      . |***    |  1 0.356 0.356 13.034
***| .     |   *****| .     |  2-0.430-0.637 32.265
****| .     |      .*| .     |  3-0.551-0.143 64.209 0.000
.*| .     |      . | .     |  4-0.100-0.032 65.265 0.000
. |**     |      .*| .     |  5 0.269-0.080 73.037 0.000
==============================================================
```
Clearly we are not done yet. The two spikes in the PACF graph suggest--now that we have MA(1) and MA(2) terms--that we should add AR(1) and AR(2) terms to the model.
```============================================================
LS // Dependent Variable is Y1
Date: 04/21/98   Time: 16:46
Included observations: 98 after adjusting endpoints
Convergence achieved after 11 iterations
============================================================
Variable      CoefficienStd. Errort-Statistic  Prob.
============================================================
C           0.162320   0.138851   1.169027   0.2454
AR(1)         0.702614   0.111219   6.317390   0.0000
AR(2)        -0.687793   0.077228  -8.905977   0.0000
MA(1)         0.603438   0.149147   4.045922   0.0001
MA(2)        -0.294895   0.146821  -2.008535   0.0475
============================================================
R-squared            0.784079    Mean dependent var 0.156682
Adjusted R-squared   0.774793    S.D. dependent var 2.182691
S.E. of regression   1.035819    Akaike info criter 0.120057
Sum squared resid    99.78159    Schwarz criterion  0.251943
Log likelihood      -139.9388    F-statistic        84.42848
Durbin-Watson stat   2.014543    Prob(F-statistic)  0.000000
============================================================
Inverted AR Roots      .35+.7   .35 -.75i
Inverted MA Roots          .3      -.92
============================================================

Correlogram of Residuals
==============================================================
Date: 04/21/98   Time: 16:47
Sample: 3 100
Included observations: 98
Q-statistic probabilities adjusted for 4 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
. | .     |      . | .     |  1-0.009-0.009 0.0075
. | .     |      . | .     |  2-0.017-0.017 0.0365
. | .     |      . | .     |  3-0.014-0.015 0.0577
. |*.     |      . |*.     |  4 0.121 0.121 1.5919
.*| .     |      .*| .     |  5-0.136-0.136 3.5300 0.060
. | .     |      . | .     |  6 0.040 0.046 3.7045 0.157
. | .     |      . | .     |  7-0.018-0.021 3.7402 0.291
. | .     |      . | .     |  8 0.038 0.022 3.8937 0.421
.*| .     |      . | .     |  9-0.085-0.054 4.6849 0.456
.*| .     |      .*| .     | 10-0.137-0.170 6.7816 0.342
==============================================================
```
This looks pretty good. All the p-values (except for the 5-lag Q-Statistic which has only 1 degree of freedom) are reasonable.

2. Second Example:

IDENT(5) Y2
```                      Correlogram of Y2
==============================================================
Date: 04/21/98   Time: 16:51
Sample: 1 100
Included observations: 100
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
. |****** |      . |****** |  1 0.728 0.728 54.619 0.000
. |*.     | *******| .     |  2 0.093-0.931 55.510 0.000
****| .     |      **| .     |  3-0.563-0.247 88.888 0.000
*******| .     |      . |*.     |  4-0.891 0.093 173.25 0.000
******| .     |      .*| .     |  5-0.745-0.167 232.86 0.000
==============================================================
```
The pattern is once again ambiguous. I am inclined to start once again by estimating MA terms. Given the absence of a spike at MA(2), I am going to first estimate MA(1) and MA(3) terms.
```============================================================
LS // Dependent Variable is Y2
Date: 04/21/98   Time: 16:52
Sample: 1 100
Included observations: 100
Convergence achieved after 14 iterations
============================================================
Variable      CoefficienStd. Errort-Statistic  Prob.
============================================================
C          -0.428425   0.387856  -1.104597   0.2721
MA(1)         0.958552   0.023630   40.56466   0.0000
MA(3)        -0.608096   0.000476  -1277.001   0.0000
============================================================
R-squared            0.749535    Mean dependent var-0.466657
Adjusted R-squared   0.744370    S.D. dependent var 5.677772
S.E. of regression   2.870672    Akaike info criter 2.138633
Sum squared resid    799.3532    Schwarz criterion  2.216788
Log likelihood      -245.8255    F-statistic        145.1396
Durbin-Watson stat   0.664071    Prob(F-statistic)  0.000000
============================================================
Inverted MA Roots          .6  -.79+.60i  -.79 -.60i
============================================================

Correlogram of Residuals
==============================================================
Date: 04/21/98   Time: 16:53
Sample: 1 100
Included observations: 100
Q-statistic probabilities adjusted for 2 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
. |*****  |      . |*****  |  1 0.655 0.655 44.146
. |*.     |    ****| .     |  2 0.128-0.525 45.860
****| .     |  ******| .     |  3-0.514-0.721 73.657 0.000
*******| .     |     ***| .     |  4-0.843-0.391 149.21 0.000
*****| .     |      . |***    |  5-0.680 0.365 198.85 0.000
==============================================================
```
Given that we have estimated MA(1) and MA(3) terms, the PACFs indicate that we should try at least AR(1) and AR(3). Given the spike at the 2nd PACF lets add that in too.
```============================================================
LS // Dependent Variable is Y2
Date: 04/21/98   Time: 16:54
Included observations: 97 after adjusting endpoints
Convergence achieved after 10 iterations
============================================================
Variable      CoefficienStd. Errort-Statistic  Prob.
============================================================
C          -0.354521   0.157481  -2.251192   0.0268
AR(1)         0.543147   0.063835   8.508590   0.0000
AR(2)         0.358374   0.087511   4.095176   0.0001
AR(3)        -0.919271   0.063529  -14.47018   0.0000
MA(1)         0.868996   0.089440   9.715997   0.0000
MA(3)        -0.123682   0.083465  -1.481841   0.1418
============================================================
R-squared            0.976675    Mean dependent var-0.483771
Adjusted R-squared   0.975394    S.D. dependent var 5.763586
S.E. of regression   0.904100    Akaike info criter-0.141770
Sum squared resid    74.38316    Schwarz criterion  0.017490
Log likelihood      -124.7612    F-statistic        762.0856
Durbin-Watson stat   2.051030    Prob(F-statistic)  0.000000
============================================================
Inverted AR Roots      .73+.6   .73 -.68      -.92
Inverted MA Roots          .3  -.60 -.17  -.60+.17i
============================================================

Correlogram of Residuals
==============================================================
Date: 04/21/98   Time: 16:55
Sample: 4 100
Included observations: 97
Q-statistic probabilities adjusted for 5 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
. | .     |      . | .     |  1-0.040-0.040 0.1581
. | .     |      . | .     |  2 0.036 0.034 0.2874
.*| .     |      .*| .     |  3-0.064-0.062 0.7108
. | .     |      . | .     |  4 0.057 0.051 1.0463
. |*.     |      . |*.     |  5 0.090 0.099 1.8878
. | .     |      . | .     |  6-0.043-0.044 2.0815 0.149
. |*.     |      . |*.     |  7 0.142 0.142 4.2181 0.121
. |*.     |      . |*.     |  8 0.101 0.127 5.3290 0.149
.*| .     |      .*| .     |  9-0.060-0.082 5.7267 0.221
. | .     |      . | .     | 10 0.002 0.006 5.7271 0.334
==============================================================
```
We have white noise and the MA(3) term is not significant. This suggests that we try dropping it and see if a simplier model works just as well.
```============================================================
LS // Dependent Variable is Y2
Date: 04/21/98   Time: 16:57
Included observations: 97 after adjusting endpoints
Convergence achieved after 11 iterations
============================================================
Variable      CoefficienStd. Errort-Statistic  Prob.
============================================================
C          -0.358680   0.162995  -2.200559   0.0303
AR(1)         0.523056   0.054485   9.600060   0.0000
AR(2)         0.384275   0.073637   5.218478   0.0000
AR(3)        -0.940735   0.052948  -17.76712   0.0000
MA(1)         0.818634   0.089641   9.132368   0.0000
============================================================
R-squared            0.975970    Mean dependent var-0.483771
Adjusted R-squared   0.974925    S.D. dependent var 5.763586
S.E. of regression   0.912671    Akaike info criter-0.132588
Sum squared resid    76.63316    Schwarz criterion  0.000129
Log likelihood      -126.2065    F-statistic        934.1233
Durbin-Watson stat   1.924115    Prob(F-statistic)  0.000000
============================================================
Inverted AR Roots      .73 -.   .73+.68i      -.94
Inverted MA Roots         -.82
============================================================

Correlogram of Residuals
==============================================================
Date: 04/21/98   Time: 16:57
Sample: 4 100
Included observations: 97
Q-statistic probabilities adjusted for 4 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
. | .     |      . | .     |  1 0.021 0.021 0.0461
. | .     |      . | .     |  2-0.029-0.029 0.1310
.*| .     |      .*| .     |  3-0.135-0.134 1.9965
. |*.     |      . |*.     |  4 0.122 0.129 3.5240
. | .     |      . | .     |  5 0.024 0.011 3.5859 0.058
. | .     |      . | .     |  6 0.000-0.013 3.5859 0.166
. |*.     |      . |*.     |  7 0.101 0.142 4.6840 0.196
. |*.     |      . |*.     |  8 0.133 0.120 6.6018 0.158
.*| .     |      .*| .     |  9-0.096-0.112 7.6122 0.179
. | .     |      . | .     | 10-0.018 0.031 7.6495 0.265
==============================================================
```
This looks pretty good. Only the Q-Statistic at 5 lags and 1 degree of freedom has a p-value below .1. The adjusted r-square is essentially unchanged when we drop the MA(3) term so this is our preferred model.

3. Third Example

IDENT(5) Y3
```                      Correlogram of Y3
==============================================================
Date: 04/21/98   Time: 17:09
Sample: 1 100
Included observations: 100
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
******| .     |  ******| .     |  1-0.723-0.723 53.804 0.000
. |*******|      . |****** |  2 0.897 0.785 137.60 0.000
*****| .     |      . |*.     |  3-0.672 0.190 185.08 0.000
. |****** |      . |**     |  4 0.835 0.209 259.19 0.000
*****| .     |      . | .     |  5-0.635 0.051 302.54 0.000
==============================================================
```
Given the big spikes in the first two positions of both plots lets try starting again with MA(1) and MA(2) and see where it leads us.
```============================================================
LS // Dependent Variable is Y3
Date: 04/21/98   Time: 17:10
Sample: 1 100
Included observations: 100
Convergence achieved after 20 iterations
============================================================
Variable      CoefficienStd. Errort-Statistic  Prob.
============================================================
C          -0.174127   0.154511  -1.126956   0.2625
MA(1)        -0.645419   0.055408  -11.64849   0.0000
MA(2)         0.720077   0.066320   10.85755   0.0000
============================================================
R-squared            0.640556    Mean dependent var-0.191290
Adjusted R-squared   0.633144    S.D. dependent var 2.389969
S.E. of regression   1.447571    Akaike info criter 0.769315
Sum squared resid    203.2598    Schwarz criterion  0.847470
Log likelihood      -177.3596    F-statistic        86.43050
Durbin-Watson stat   2.123049    Prob(F-statistic)  0.000000
============================================================
Inverted MA Roots      .32+.7   .32 -.78i
============================================================

Correlogram of Residuals
==============================================================
Date: 04/21/98   Time: 17:10
Sample: 1 100
Included observations: 100
Q-statistic probabilities adjusted for 2 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
.*| .     |      .*| .     |  1-0.062-0.062 0.4013
. |****   |      . |****   |  2 0.486 0.484 25.018
**| .     |     ***| .     |  3-0.290-0.320 33.867 0.000
. |*****  |      . |*****  |  4 0.645 0.629 78.089 0.000
. | .     |      . |*.     |  5-0.029 0.110 78.178 0.000
==============================================================
```
Given the big spike at the 2nd lag, lets try adding an AR(2) term.
```============================================================
LS // Dependent Variable is Y3
Date: 04/21/98   Time: 17:12
Included observations: 98 after adjusting endpoints
Convergence achieved after 11 iterations
============================================================
Variable      CoefficienStd. Errort-Statistic  Prob.
============================================================
C          -0.174180   0.792690  -0.219734   0.8266
AR(2)         0.929372   0.045696   20.33793   0.0000
MA(1)        -0.307308   0.103296  -2.975027   0.0037
MA(2)        -0.097869   0.113819  -0.859866   0.3921
============================================================
R-squared            0.857380    Mean dependent var-0.216506
Adjusted R-squared   0.852829    S.D. dependent var 2.404179
S.E. of regression   0.922313    Akaike info criter-0.121781
Sum squared resid    79.96218    Schwarz criterion -0.016272
Log likelihood      -129.0887    F-statistic        188.3652
Durbin-Watson stat   1.954877    Prob(F-statistic)  0.000000
============================================================
Inverted AR Roots          .9      -.96
Inverted MA Roots          .5      -.19
============================================================

Correlogram of Residuals
==============================================================
Date: 04/21/98   Time: 17:12
Sample: 3 100
Included observations: 98
Q-statistic probabilities adjusted for 3 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
. | .     |      . | .     |  1 0.006 0.006 0.0039
. | .     |      . | .     |  2-0.044-0.044 0.2028
. | .     |      . | .     |  3 0.054 0.054 0.5012
. |*.     |      . |*.     |  4 0.110 0.107 1.7560 0.185
. |*.     |      . |*.     |  5 0.144 0.149 3.9264 0.140
. | .     |      . | .     |  6-0.015-0.007 3.9495 0.267
. | .     |      . | .     |  7-0.034-0.034 4.0709 0.396
. |*.     |      . |*.     |  8 0.128 0.102 5.8653 0.320
. | .     |      . | .     |  9-0.012-0.045 5.8800 0.437
. |*.     |      . |*.     | 10 0.073 0.071 6.4753 0.485
==============================================================
```
This gives us white noise. However, the MA(2) term is now insignificant. Lets try dropping it:
```============================================================
LS // Dependent Variable is Y3
Date: 04/21/98   Time: 17:13
Included observations: 98 after adjusting endpoints
Convergence achieved after 7 iterations
============================================================
Variable      CoefficienStd. Errort-Statistic  Prob.
============================================================
C          -0.183495   0.659115  -0.278396   0.7813
AR(2)         0.910669   0.045020   20.22829   0.0000
MA(1)        -0.371488   0.095248  -3.900233   0.0002
============================================================
R-squared            0.856361    Mean dependent var-0.216506
Adjusted R-squared   0.853337    S.D. dependent var 2.404179
S.E. of regression   0.920720    Akaike info criter-0.135064
Sum squared resid    80.53393    Schwarz criterion -0.055933
Log likelihood      -129.4378    F-statistic        283.1891
Durbin-Watson stat   1.871770    Prob(F-statistic)  0.000000
============================================================
Inverted AR Roots          .9      -.95
Inverted MA Roots          .37
============================================================

Correlogram of Residuals
==============================================================
Date: 04/21/98   Time: 17:13
Sample: 3 100
Included observations: 98
Q-statistic probabilities adjusted for 2 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
. | .     |      . | .     |  1 0.050 0.050 0.2477
.*| .     |      .*| .     |  2-0.107-0.109 1.4086
. | .     |      . | .     |  3 0.014 0.025 1.4281 0.232
. |*.     |      . |*.     |  4 0.130 0.118 3.1917 0.203
. |*.     |      . |*.     |  5 0.135 0.130 5.1263 0.163
. | .     |      . | .     |  6-0.016-0.003 5.1524 0.272
. | .     |      . | .     |  7-0.044-0.023 5.3645 0.373
. |*.     |      . |*.     |  8 0.130 0.115 7.1965 0.303
. | .     |      . | .     |  9 0.005-0.045 7.1993 0.408
. | .     |      . |*.     | 10 0.065 0.082 7.6734 0.466
==============================================================
```
Very often more than one model will fit a time series equally well. After considerable experimentation with various models I found that ARIMA(1,0,3) model worked very well for this series.
```============================================================
LS // Dependent Variable is Y3
Date: 04/21/98   Time: 17:18
Included observations: 96 after adjusting endpoints
Convergence achieved after 12 iterations
============================================================
Variable      CoefficienStd. Errort-Statistic  Prob.
============================================================
C          -0.188639   0.595493  -0.316778   0.7521
AR(4)         0.804694   0.072112   11.15901   0.0000
MA(1)        -0.292846   0.086185  -3.397900   0.0010
MA(2)         0.791667   0.072370   10.93920   0.0000
MA(3)        -0.270966   0.085920  -3.153713   0.0022
============================================================
R-squared            0.859192    Mean dependent var-0.226304
Adjusted R-squared   0.853003    S.D. dependent var 2.428292
S.E. of regression   0.931011    Akaike info criter-0.092289
Sum squared resid    78.87720    Schwarz criterion  0.041270
Log likelihood      -126.7882    F-statistic        138.8180
Durbin-Watson stat   2.069957    Prob(F-statistic)  0.000000
============================================================
Inverted AR Roots          .9   .00 -.95   .00+.95i      -.95
Inverted MA Roots          .3  -.02 -.90  -.02+.90i
============================================================

Correlogram of Residuals
==============================================================
Date: 04/21/98   Time: 17:19
Sample: 5 100
Included observations: 96
Q-statistic probabilities adjusted for 4 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob
==============================================================
. | .     |      . | .     |  1-0.039-0.039 0.1513
. | .     |      . | .     |  2-0.011-0.013 0.1634
. | .     |      . | .     |  3 0.029 0.028 0.2505
. |*.     |      . |*.     |  4 0.072 0.074 0.7827
. |*.     |      . |*.     |  5 0.090 0.097 1.6197 0.203
. | .     |      . | .     |  6 0.000 0.009 1.6197 0.445
. | .     |      . | .     |  7-0.051-0.054 1.8963 0.594
. |*.     |      . |*.     |  8 0.087 0.072 2.7091 0.608
. | .     |      . | .     |  9-0.040-0.049 2.8817 0.718
. |*.     |      . |*.     | 10 0.145 0.141 5.1813 0.521
==============================================================
```