ARIMA Examples (Part 1)
Correlogram of Y1
==============================================================
Date: 04/21/98 Time: 16:42
Sample: 1 100
Included observations: 100
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
. |*** | . |*** | 1 0.444 0.444 20.306 0.000
***| . | ******| . | 2-0.431-0.782 39.652 0.000
*****| . | . |** | 3-0.603 0.275 77.871 0.000
.*| . | ***| . | 4-0.153-0.345 80.352 0.000
. |** | . |** | 5 0.302 0.319 90.148 0.000
==============================================================
This pattern is somewhat ambiguous but clearly we have multiple terms
present. Lets be conservative and start off by estimating up to a MA(2).
============================================================
LS // Dependent Variable is Y1
Date: 04/21/98 Time: 16:44
Sample: 1 100
Included observations: 100
Convergence achieved after 23 iterations
============================================================
Variable CoefficienStd. Errort-Statistic Prob.
============================================================
C 0.067147 0.239710 0.280117 0.7800
MA(1) 0.835517 0.032102 26.02710 0.0000
MA(2) -0.152868 0.002619 -58.37944 0.0000
============================================================
R-squared 0.574891 Mean dependent var 0.150504
Adjusted R-squared 0.566126 S.D. dependent var 2.163285
S.E. of regression 1.424937 Akaike info criter 0.737796
Sum squared resid 196.9532 Schwarz criterion 0.815951
Log likelihood -175.7836 F-statistic 65.58840
Durbin-Watson stat 1.284884 Prob(F-statistic) 0.000000
============================================================
Inverted MA Roots .1 -.99
============================================================
Correlogram of Residuals
==============================================================
Date: 04/21/98 Time: 16:45
Sample: 1 100
Included observations: 100
Q-statistic probabilities adjusted for 2 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
. |*** | . |*** | 1 0.356 0.356 13.034
***| . | *****| . | 2-0.430-0.637 32.265
****| . | .*| . | 3-0.551-0.143 64.209 0.000
.*| . | . | . | 4-0.100-0.032 65.265 0.000
. |** | .*| . | 5 0.269-0.080 73.037 0.000
==============================================================
Clearly we are not done yet. The two spikes in the PACF graph
suggest--now that we have MA(1) and MA(2) terms--that we should add AR(1) and
AR(2) terms to the model.
============================================================
LS // Dependent Variable is Y1
Date: 04/21/98 Time: 16:46
Sample(adjusted): 3 100
Included observations: 98 after adjusting endpoints
Convergence achieved after 11 iterations
============================================================
Variable CoefficienStd. Errort-Statistic Prob.
============================================================
C 0.162320 0.138851 1.169027 0.2454
AR(1) 0.702614 0.111219 6.317390 0.0000
AR(2) -0.687793 0.077228 -8.905977 0.0000
MA(1) 0.603438 0.149147 4.045922 0.0001
MA(2) -0.294895 0.146821 -2.008535 0.0475
============================================================
R-squared 0.784079 Mean dependent var 0.156682
Adjusted R-squared 0.774793 S.D. dependent var 2.182691
S.E. of regression 1.035819 Akaike info criter 0.120057
Sum squared resid 99.78159 Schwarz criterion 0.251943
Log likelihood -139.9388 F-statistic 84.42848
Durbin-Watson stat 2.014543 Prob(F-statistic) 0.000000
============================================================
Inverted AR Roots .35+.7 .35 -.75i
Inverted MA Roots .3 -.92
============================================================
Correlogram of Residuals
==============================================================
Date: 04/21/98 Time: 16:47
Sample: 3 100
Included observations: 98
Q-statistic probabilities adjusted for 4 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
. | . | . | . | 1-0.009-0.009 0.0075
. | . | . | . | 2-0.017-0.017 0.0365
. | . | . | . | 3-0.014-0.015 0.0577
. |*. | . |*. | 4 0.121 0.121 1.5919
.*| . | .*| . | 5-0.136-0.136 3.5300 0.060
. | . | . | . | 6 0.040 0.046 3.7045 0.157
. | . | . | . | 7-0.018-0.021 3.7402 0.291
. | . | . | . | 8 0.038 0.022 3.8937 0.421
.*| . | . | . | 9-0.085-0.054 4.6849 0.456
.*| . | .*| . | 10-0.137-0.170 6.7816 0.342
==============================================================
This looks pretty good. All the p-values (except for the 5-lag
Q-Statistic which has only 1 degree of freedom) are reasonable.
Correlogram of Y2
==============================================================
Date: 04/21/98 Time: 16:51
Sample: 1 100
Included observations: 100
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
. |****** | . |****** | 1 0.728 0.728 54.619 0.000
. |*. | *******| . | 2 0.093-0.931 55.510 0.000
****| . | **| . | 3-0.563-0.247 88.888 0.000
*******| . | . |*. | 4-0.891 0.093 173.25 0.000
******| . | .*| . | 5-0.745-0.167 232.86 0.000
==============================================================
The pattern is once again ambiguous. I am inclined to start once again
by estimating MA terms. Given the absence of a spike at MA(2), I am going to
first estimate MA(1) and MA(3) terms.
============================================================
LS // Dependent Variable is Y2
Date: 04/21/98 Time: 16:52
Sample: 1 100
Included observations: 100
Convergence achieved after 14 iterations
============================================================
Variable CoefficienStd. Errort-Statistic Prob.
============================================================
C -0.428425 0.387856 -1.104597 0.2721
MA(1) 0.958552 0.023630 40.56466 0.0000
MA(3) -0.608096 0.000476 -1277.001 0.0000
============================================================
R-squared 0.749535 Mean dependent var-0.466657
Adjusted R-squared 0.744370 S.D. dependent var 5.677772
S.E. of regression 2.870672 Akaike info criter 2.138633
Sum squared resid 799.3532 Schwarz criterion 2.216788
Log likelihood -245.8255 F-statistic 145.1396
Durbin-Watson stat 0.664071 Prob(F-statistic) 0.000000
============================================================
Inverted MA Roots .6 -.79+.60i -.79 -.60i
============================================================
Correlogram of Residuals
==============================================================
Date: 04/21/98 Time: 16:53
Sample: 1 100
Included observations: 100
Q-statistic probabilities adjusted for 2 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
. |***** | . |***** | 1 0.655 0.655 44.146
. |*. | ****| . | 2 0.128-0.525 45.860
****| . | ******| . | 3-0.514-0.721 73.657 0.000
*******| . | ***| . | 4-0.843-0.391 149.21 0.000
*****| . | . |*** | 5-0.680 0.365 198.85 0.000
==============================================================
Given that we have estimated MA(1) and MA(3) terms, the PACFs indicate
that we should try at least AR(1) and AR(3). Given the spike at the 2nd PACF
lets add that in too.
============================================================
LS // Dependent Variable is Y2
Date: 04/21/98 Time: 16:54
Sample(adjusted): 4 100
Included observations: 97 after adjusting endpoints
Convergence achieved after 10 iterations
============================================================
Variable CoefficienStd. Errort-Statistic Prob.
============================================================
C -0.354521 0.157481 -2.251192 0.0268
AR(1) 0.543147 0.063835 8.508590 0.0000
AR(2) 0.358374 0.087511 4.095176 0.0001
AR(3) -0.919271 0.063529 -14.47018 0.0000
MA(1) 0.868996 0.089440 9.715997 0.0000
MA(3) -0.123682 0.083465 -1.481841 0.1418
============================================================
R-squared 0.976675 Mean dependent var-0.483771
Adjusted R-squared 0.975394 S.D. dependent var 5.763586
S.E. of regression 0.904100 Akaike info criter-0.141770
Sum squared resid 74.38316 Schwarz criterion 0.017490
Log likelihood -124.7612 F-statistic 762.0856
Durbin-Watson stat 2.051030 Prob(F-statistic) 0.000000
============================================================
Inverted AR Roots .73+.6 .73 -.68 -.92
Inverted MA Roots .3 -.60 -.17 -.60+.17i
============================================================
Correlogram of Residuals
==============================================================
Date: 04/21/98 Time: 16:55
Sample: 4 100
Included observations: 97
Q-statistic probabilities adjusted for 5 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
. | . | . | . | 1-0.040-0.040 0.1581
. | . | . | . | 2 0.036 0.034 0.2874
.*| . | .*| . | 3-0.064-0.062 0.7108
. | . | . | . | 4 0.057 0.051 1.0463
. |*. | . |*. | 5 0.090 0.099 1.8878
. | . | . | . | 6-0.043-0.044 2.0815 0.149
. |*. | . |*. | 7 0.142 0.142 4.2181 0.121
. |*. | . |*. | 8 0.101 0.127 5.3290 0.149
.*| . | .*| . | 9-0.060-0.082 5.7267 0.221
. | . | . | . | 10 0.002 0.006 5.7271 0.334
==============================================================
We have white noise and the MA(3) term is not significant. This
suggests that we try dropping it and see if a simplier model works
just as well.
============================================================
LS // Dependent Variable is Y2
Date: 04/21/98 Time: 16:57
Sample(adjusted): 4 100
Included observations: 97 after adjusting endpoints
Convergence achieved after 11 iterations
============================================================
Variable CoefficienStd. Errort-Statistic Prob.
============================================================
C -0.358680 0.162995 -2.200559 0.0303
AR(1) 0.523056 0.054485 9.600060 0.0000
AR(2) 0.384275 0.073637 5.218478 0.0000
AR(3) -0.940735 0.052948 -17.76712 0.0000
MA(1) 0.818634 0.089641 9.132368 0.0000
============================================================
R-squared 0.975970 Mean dependent var-0.483771
Adjusted R-squared 0.974925 S.D. dependent var 5.763586
S.E. of regression 0.912671 Akaike info criter-0.132588
Sum squared resid 76.63316 Schwarz criterion 0.000129
Log likelihood -126.2065 F-statistic 934.1233
Durbin-Watson stat 1.924115 Prob(F-statistic) 0.000000
============================================================
Inverted AR Roots .73 -. .73+.68i -.94
Inverted MA Roots -.82
============================================================
Correlogram of Residuals
==============================================================
Date: 04/21/98 Time: 16:57
Sample: 4 100
Included observations: 97
Q-statistic probabilities adjusted for 4 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
. | . | . | . | 1 0.021 0.021 0.0461
. | . | . | . | 2-0.029-0.029 0.1310
.*| . | .*| . | 3-0.135-0.134 1.9965
. |*. | . |*. | 4 0.122 0.129 3.5240
. | . | . | . | 5 0.024 0.011 3.5859 0.058
. | . | . | . | 6 0.000-0.013 3.5859 0.166
. |*. | . |*. | 7 0.101 0.142 4.6840 0.196
. |*. | . |*. | 8 0.133 0.120 6.6018 0.158
.*| . | .*| . | 9-0.096-0.112 7.6122 0.179
. | . | . | . | 10-0.018 0.031 7.6495 0.265
==============================================================
This looks pretty good. Only the Q-Statistic at 5 lags and 1 degree
of freedom has a p-value below .1. The adjusted r-square is essentially
unchanged when we drop the MA(3) term so this is our preferred model.
Correlogram of Y3
==============================================================
Date: 04/21/98 Time: 17:09
Sample: 1 100
Included observations: 100
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
******| . | ******| . | 1-0.723-0.723 53.804 0.000
. |*******| . |****** | 2 0.897 0.785 137.60 0.000
*****| . | . |*. | 3-0.672 0.190 185.08 0.000
. |****** | . |** | 4 0.835 0.209 259.19 0.000
*****| . | . | . | 5-0.635 0.051 302.54 0.000
==============================================================
Given the big spikes in the first two positions of both plots lets try
starting again with MA(1) and MA(2) and see where it leads us.
============================================================
LS // Dependent Variable is Y3
Date: 04/21/98 Time: 17:10
Sample: 1 100
Included observations: 100
Convergence achieved after 20 iterations
============================================================
Variable CoefficienStd. Errort-Statistic Prob.
============================================================
C -0.174127 0.154511 -1.126956 0.2625
MA(1) -0.645419 0.055408 -11.64849 0.0000
MA(2) 0.720077 0.066320 10.85755 0.0000
============================================================
R-squared 0.640556 Mean dependent var-0.191290
Adjusted R-squared 0.633144 S.D. dependent var 2.389969
S.E. of regression 1.447571 Akaike info criter 0.769315
Sum squared resid 203.2598 Schwarz criterion 0.847470
Log likelihood -177.3596 F-statistic 86.43050
Durbin-Watson stat 2.123049 Prob(F-statistic) 0.000000
============================================================
Inverted MA Roots .32+.7 .32 -.78i
============================================================
Correlogram of Residuals
==============================================================
Date: 04/21/98 Time: 17:10
Sample: 1 100
Included observations: 100
Q-statistic probabilities adjusted for 2 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
.*| . | .*| . | 1-0.062-0.062 0.4013
. |**** | . |**** | 2 0.486 0.484 25.018
**| . | ***| . | 3-0.290-0.320 33.867 0.000
. |***** | . |***** | 4 0.645 0.629 78.089 0.000
. | . | . |*. | 5-0.029 0.110 78.178 0.000
==============================================================
Given the big spike at the 2nd lag, lets try adding an
AR(2) term.
============================================================
LS // Dependent Variable is Y3
Date: 04/21/98 Time: 17:12
Sample(adjusted): 3 100
Included observations: 98 after adjusting endpoints
Convergence achieved after 11 iterations
============================================================
Variable CoefficienStd. Errort-Statistic Prob.
============================================================
C -0.174180 0.792690 -0.219734 0.8266
AR(2) 0.929372 0.045696 20.33793 0.0000
MA(1) -0.307308 0.103296 -2.975027 0.0037
MA(2) -0.097869 0.113819 -0.859866 0.3921
============================================================
R-squared 0.857380 Mean dependent var-0.216506
Adjusted R-squared 0.852829 S.D. dependent var 2.404179
S.E. of regression 0.922313 Akaike info criter-0.121781
Sum squared resid 79.96218 Schwarz criterion -0.016272
Log likelihood -129.0887 F-statistic 188.3652
Durbin-Watson stat 1.954877 Prob(F-statistic) 0.000000
============================================================
Inverted AR Roots .9 -.96
Inverted MA Roots .5 -.19
============================================================
Correlogram of Residuals
==============================================================
Date: 04/21/98 Time: 17:12
Sample: 3 100
Included observations: 98
Q-statistic probabilities adjusted for 3 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
. | . | . | . | 1 0.006 0.006 0.0039
. | . | . | . | 2-0.044-0.044 0.2028
. | . | . | . | 3 0.054 0.054 0.5012
. |*. | . |*. | 4 0.110 0.107 1.7560 0.185
. |*. | . |*. | 5 0.144 0.149 3.9264 0.140
. | . | . | . | 6-0.015-0.007 3.9495 0.267
. | . | . | . | 7-0.034-0.034 4.0709 0.396
. |*. | . |*. | 8 0.128 0.102 5.8653 0.320
. | . | . | . | 9-0.012-0.045 5.8800 0.437
. |*. | . |*. | 10 0.073 0.071 6.4753 0.485
==============================================================
This gives us white noise. However, the MA(2) term is now
insignificant. Lets try dropping it:
============================================================
LS // Dependent Variable is Y3
Date: 04/21/98 Time: 17:13
Sample(adjusted): 3 100
Included observations: 98 after adjusting endpoints
Convergence achieved after 7 iterations
============================================================
Variable CoefficienStd. Errort-Statistic Prob.
============================================================
C -0.183495 0.659115 -0.278396 0.7813
AR(2) 0.910669 0.045020 20.22829 0.0000
MA(1) -0.371488 0.095248 -3.900233 0.0002
============================================================
R-squared 0.856361 Mean dependent var-0.216506
Adjusted R-squared 0.853337 S.D. dependent var 2.404179
S.E. of regression 0.920720 Akaike info criter-0.135064
Sum squared resid 80.53393 Schwarz criterion -0.055933
Log likelihood -129.4378 F-statistic 283.1891
Durbin-Watson stat 1.871770 Prob(F-statistic) 0.000000
============================================================
Inverted AR Roots .9 -.95
Inverted MA Roots .37
============================================================
Correlogram of Residuals
==============================================================
Date: 04/21/98 Time: 17:13
Sample: 3 100
Included observations: 98
Q-statistic probabilities adjusted for 2 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
. | . | . | . | 1 0.050 0.050 0.2477
.*| . | .*| . | 2-0.107-0.109 1.4086
. | . | . | . | 3 0.014 0.025 1.4281 0.232
. |*. | . |*. | 4 0.130 0.118 3.1917 0.203
. |*. | . |*. | 5 0.135 0.130 5.1263 0.163
. | . | . | . | 6-0.016-0.003 5.1524 0.272
. | . | . | . | 7-0.044-0.023 5.3645 0.373
. |*. | . |*. | 8 0.130 0.115 7.1965 0.303
. | . | . | . | 9 0.005-0.045 7.1993 0.408
. | . | . |*. | 10 0.065 0.082 7.6734 0.466
==============================================================
Very often more than one model will fit a time series equally well.
After considerable experimentation with various models I found that
ARIMA(1,0,3) model worked very well for this series.
============================================================
LS // Dependent Variable is Y3
Date: 04/21/98 Time: 17:18
Sample(adjusted): 5 100
Included observations: 96 after adjusting endpoints
Convergence achieved after 12 iterations
============================================================
Variable CoefficienStd. Errort-Statistic Prob.
============================================================
C -0.188639 0.595493 -0.316778 0.7521
AR(4) 0.804694 0.072112 11.15901 0.0000
MA(1) -0.292846 0.086185 -3.397900 0.0010
MA(2) 0.791667 0.072370 10.93920 0.0000
MA(3) -0.270966 0.085920 -3.153713 0.0022
============================================================
R-squared 0.859192 Mean dependent var-0.226304
Adjusted R-squared 0.853003 S.D. dependent var 2.428292
S.E. of regression 0.931011 Akaike info criter-0.092289
Sum squared resid 78.87720 Schwarz criterion 0.041270
Log likelihood -126.7882 F-statistic 138.8180
Durbin-Watson stat 2.069957 Prob(F-statistic) 0.000000
============================================================
Inverted AR Roots .9 .00 -.95 .00+.95i -.95
Inverted MA Roots .3 -.02 -.90 -.02+.90i
============================================================
Correlogram of Residuals
==============================================================
Date: 04/21/98 Time: 17:19
Sample: 5 100
Included observations: 96
Q-statistic probabilities adjusted for 4 ARMA term(s)
==============================================================
Autocorrelation Partial Correlation AC PAC Q-Stat Prob
==============================================================
. | . | . | . | 1-0.039-0.039 0.1513
. | . | . | . | 2-0.011-0.013 0.1634
. | . | . | . | 3 0.029 0.028 0.2505
. |*. | . |*. | 4 0.072 0.074 0.7827
. |*. | . |*. | 5 0.090 0.097 1.6197 0.203
. | . | . | . | 6 0.000 0.009 1.6197 0.445
. | . | . | . | 7-0.051-0.054 1.8963 0.594
. |*. | . |*. | 8 0.087 0.072 2.7091 0.608
. | . | . | . | 9-0.040-0.049 2.8817 0.718
. |*. | . |*. | 10 0.145 0.141 5.1813 0.521
==============================================================