double_center_drive_data.r -- R Program to
Perform Double-Centering of U.S. Driving Distances Data
Drive2.TXT -- U. S. Driving Distances Dataand place the files in the same directory.
Start R in the usual WINDOZE fashion by double-clicking on the R icon. You should see something like this:
R IS NOT USER FRIENDLY!!!!. However, it and Stata are rapidly establishing themselves as standards within Political Science -- especially with the "pocket-protector" and "propeller-head" types (like myself) -- and it is definitely worth your time to learn R!
Our program double_center_drive_data.r uses three plug-in libraries -- MASS, pcurve, and stats -- and the first thing you need to do is check to see if they are installed on your computer. To do this, go to the Packages menu and select Load package:
You will now see the packages that are already on your computer:
In my case I already have MASS, pcurve, and stats, so they will appear in this menu. MASS and stats should be in this menu as they are default packages in R. If pcurve is not on the list you can download it from the Comprehensive R Archive Network (CRAN) website by selecting Install package(s) from CRAN under the Packages menu. R will go to the website and a select menu will appear showing all the packages that you can download:
Scroll down until you see pcurve:
Click "OK" and it downloads and installs itself automatically. It then asks you if you want to delete the ZIP files:
Always answer N because the zip files do not take up much disk space and they are always around if you need them!
R has a fairly good help menu with the PDF of the manual and a useful HTML help that has the complete manual in the form of cross-linked webpages:
When you select HTML help you will get this:
Note that the HTML help page comes up in MICROSHAFT IE. This can be a problem if you have installed the latest service pack from MICROSHAFT that includes a version of IE that has a "pop-up" blocker. When you use the search engine MICROSHAFT may screw it up (I have found that this varies from machine to machine)!!! Note that you can always simply type in the link
C:\Program Files\R\rw1091\doc\html\rwin.html
in a non-MICROSHAFT browser and it will work just fine.
To run the double_center_drive_data.r R program, go to the file menu and select Source R Code:
You will get the usual WINDOZE directory menu. Go to where you placed double_center_drive_data.r and select it:
When you click Open R runs the program and you should get this:
If you Right-Click on the image it brings up this very handy menu:
If you copy it as a bitmap then you can paste it directly into MICROSHAFT WORD or paste it directly into a graphics package like Photoshop or Paint Shop Pro. If you paste it into a graphics package you can then save it as a JPEG or GIF file.
E-Mail me this graph in either form for a PASS on this problem.
Below is a listing of the double_center_drive_data.r program. R is an Interpreter -- that is, it compiles and executes one line of code at a time! In this regard R is a lot like the original BASIC programming language for microcomputers -- the computer quite literally runs the program one-line-at-a-time. You could type the program below into R one line at a time if you wanted to. In this regard, R is a very powerful calculator with many built-in commands like mean(x) and sd(x) where x is a vector of numbers.
# The cross-hatch is used as a comment marker -- R ignores the line
# double_center_drive_data.r -- Double-Center Program Always put the name of the program at the top
#
# Data Must Be Transformed to Squared Distances Below
#
# ATLANTA 0000 2340 1084 715 481 826 1519 2252 662 641 2450
# BOISE 2340 0000 2797 1789 2018 1661 891 908 2974 2480 680
# BOSTON 1084 2797 0000 976 853 1868 2008 3130 1547 443 3160
# CHICAGO 715 1789 976 0000 301 936 1017 2189 1386 696 2200 I embed the data for
# CINCINNATI 481 2018 853 301 0000 988 1245 2292 1143 498 2330 convenience
# DALLAS 826 1661 1868 936 988 0000 797 1431 1394 1414 1720
# DENVER 1519 891 2008 1017 1245 797 0000 1189 2126 1707 1290
# LOS ANGELES 2252 908 3130 2189 2292 1431 1189 0000 2885 2754 370
# MIAMI 662 2974 1547 1386 1143 1394 2126 2885 0000 1096 3110
# WASHINGTON 641 2480 443 696 498 1414 1707 2754 1096 0000 2870
# CASBS 2450 680 3160 2200 2330 1720 1290 370 3110 2870 0000
#
library(MASS) These are the three libraries
library(pcurve) These commands tell R to load them
library(stats)
# The command below reads the driving distances into the matrix T
# Note that R uses the syntax "<-" for "="
T <- matrix(scan("C:/ucsd_course/drive2.txt",0),ncol=11,byrow=TRUE)
# The c() command creates a vector
names <- c("Atlanta ","Boise ","Boston ","Chicago ","Cincinnati ","Dallas ","Denver ",
"Los Angeles ","Miami ","Washington ","CASBS ")
#
nrow <- length(T[,1]) The length command tells you the length of a vector
ncol <- length(T[1,]) The "," acts as a wildcard
TT <- rep(0,nrow*ncol) rep stands for repeat -- this creates a vector of zeroes
dim(TT) <- c(nrow,ncol) dim stands for dimension -- this creates an nrow by ncol matrix of zeroes
TTT <- rep(0,nrow*ncol)
dim(TTT) <- c(nrow,ncol)
#
xrow <- NULL The NULL is a convenient way to initialize a vector
xcol <- NULL
xcount <- NULL
matrixmean <- 0 Simple initialization to zero
matrixmean2 <- 0
#
# Transform the Matrix This is a simple double DO-Loop that divides the Distances
# by 1000 and squares them so that we have a set of squared
i <- 0 distances roughly in the range of 0 to 9
while (i < nrow) {
i <- i + 1 The programming language is essentially the same as C/C++ and
xcount[i] <- i Visual Basic
j <- 0
while (j < ncol) {
j <- j + 1
#
# Square the Driving Distances
#
TT[i,j] <- (T[i,j]/1000)**2
#
}
}
#
# Put it Back in T
#
T <- TT
#
#
# Below is the old Long Method of Double-Centering
#
# Compute Row and Column Means
#
i <- 0
while (i < nrow) {
i <- i + 1
xrow[i] <- mean(T[i,]) Row Means of the Squared Distance Matrix
}
i <- 0
while (i < ncol) {
i <- i + 1
xcol[i] <- mean(T[,i]) Column Means of the Squared Distance Matrix
}
matrixmean <- mean(xcol) Matrix Mean
matrixmean2 <- mean(xrow) This is a safety check
#
# Double-Center the Matrix Using old Long Method
# Compute comparison as safety check
#
i <- 0
while (i < nrow) {
i <- i + 1
j <- 0
while (j < ncol) {
j <- j + 1
TT[i,j] <- (T[i,j]-xrow[i]-xcol[j]+matrixmean)/(-2) Basic Formula for Double-Centering
TTT[i,j] <- (T[i,j]-xrow[i]-xcol[j]+matrixmean)/(-2) a Squared Distance Matrix
}
}
#
# Perform Eigenvalue-Eigenvector Decomposition of Double-Centered Matrix
# eigen performs an eigenvalue-eigenvector decomposition of the input matrix
ev <- eigen(TT) The n by n matrix of eigenvectors is returned in ev$vec[*,*]
# The n length vector of eigenvalues is returned in ev$val[*]
# Find Point furthest from Center of Space
# The max(x) command finds the maximum value in the vector x
aaa <- sqrt(max((abs(ev$vec[,1]))**2 + (abs(ev$vec[,2]))**2)) Note that aaa and bbb should be exactly the same!
bbb <- sqrt(max(((ev$vec[,1]))**2 + ((ev$vec[,2]))**2)) This is overkill, but it is a handy check on your math!
#
# Weight the Eigenvectors to Scale Space to Unit Circle
#
torgerson1 <- ev$vec[,1]*(1/aaa)*sqrt(ev$val[1]) This is the classical Torgerson Solution
#torgerson2 <- ev$vec[,2]*(1/aaa)
#
#torgerson1 <- -ev$vec[,1]*(1/aaa)
torgerson2 <- -ev$vec[,2]*(1/aaa)*sqrt(ev$val[2]) This is the classical Torgerson Solution
# The basic plot command. The first two arguments are the x-axis and y-axis coordinates.
plot(torgerson1,torgerson2,type="n",asp=1, The type="n" suppresses all plotting of points; asp=1 maintains the aspect ratio
main="Double-Centered Driving Distance Matrix \n Torgerson Coordinates", Title -- the \n is a "new-line" command
xlab="West East", Label for the x-axis
ylab="South North", Label for the y-axis
xlim=c(-3.0,3.0),ylim=c(-3.0,3.0)) Sets range for the axes -- these must be the same if asp=1
points(torgerson1,torgerson2,pch=16,col="red") Places points in the plot -- pch=16 is a solid circle
text(torgerson1,torgerson2,names,col="blue",adj=1) Places the City names -- adj=1 puts names on left
#
Go back to the R prompt by clicking on the R Console Window
to bring it to the front. You can now see the values for any of the variables in the program
by simply typing the variable name and hitting Enter. For
example, to check to see if matrixmean and matrixmean2 produce the same number,
simply type each in turn:
To see the eigenvalues type ev$val:
Note that some of the eigenvalues are negative!
We are now going to fix the plotting of the city names. Bring double_center_drive_data.r up in Epsilon. We are going to modify the file but for safety's sake, lets do that in a copy of the file that we will call double_center_drive_data_2.r. To do this, use the write-file command:
C-X C-W
A banner will appear in the bottom of the window giving the current path:
Now just type in the filename -- double_center_drive_data_2.r -- and hit Enter
Now, go to the bottom of the file and enter the text below using Epsilon. Note that you are commenting out the text(...) command and then adding lines followed by a new text(...) command.
#text(torgerson1,torgerson2,names,col="blue",adj=1) # # pos -- a position specifier for the text. Values of 1, 2, 3 and 4, # respectively indicate positions below, to the left of, above and # to the right of the specified coordinates # namepos <- NULL namepos[1] <- 2 # Atlanta namepos[2] <- 2 # Boise namepos[3] <- 2 # Boston namepos[4] <- 2 # Chicago namepos[5] <- 2 # Cininnati namepos[6] <- 2 # Dallas namepos[7] <- 2 # Denver namepos[8] <- 2 # Los Angeles namepos[9] <- 2 # Miami namepos[10] <- 2 # Washington namepos[11] <- 2 # CASBS # text(torgerson1,torgerson2,names,pos=namepos,offset=0.0,col="blue") #The vector namepos controls where the city names in the names vector are placed vis a vis the points. Run double_center_drive_data_2.R in R and you should get:
The parameter offset controls how close the text is to the points. If you set it to 0.5 you will get this:
Now, with offset=0.0 or offset=0.25 you can easily adjust the positions of the names of the cities by simply toggling back and forth between Epsilon and R. (Note that in R you can get the previous command by simply using the Up Arrow.) For example, with offset=0.25 and with namepos[8] <- 4 (Los Angeles) and namepos[4] <- 3 (Chicago) you will get:
Note that I put a space after the city names in the vector names. Take these out and you get:
We really do not need those spaces because we can achieve the same effect using the offset command.
For a PASS on this problem 1) produce a city plot with all the names clearly visible (no overlapping of any names); and 2) send me the final version of your double_center_drive_2.r program.
SEN90KH.ORD -- 90th (1967-68) Senate Roll Call Dataand the Optimal Classification program and its "Control File":
Optimal Classification (OC) Scaling Program Executable
(Compiled for WINTEL Machines) -- PERFL.EXE
Control File -- PERFSTRT.DATand put all the files in the same directory.
Bring SEN90KH.Ord up in Epsilon, split the window using the C-X 2 command and use the find-file C-X C-F command to bring up Perfstrt.dat. You should be here:
Now, just as we did in part 3 of the second homework assignment, we need to change Perfstrt.dat so that it tells OC the correct file to read, the title of the data, the correct number of roll calls, the number of dimensions to estimate, and so on.
To get the correct file name and title just change 107 to 90 in the first two lines. This U. S. Senate occurred during the period of history when two basic dimensions were required to account for voting in Congress. Accordingly, change the number of dimensions from "1" to "2". Use the method described in part 3 of the second homework assignment to find the number of roll call votes. You should get 596. You should be here:
If you counted the number of roll calls correctly using the Show-Point Command -- C-X =, you should have found:
Column 632, char 632 of 64566 is '^J'=10 decimal=0A hex
and
Column 36, char 36 of 64566 is '9'=57 decimal=39 hex
respectively. Therefore, the format statements are already set up correctly.
All that remains is to select what Senators to control the polarity of the two dimensions. In this Senate we will use President Johnson and Senator Sparkman, respectively. That is, change the 0001000018 (recall that the spacing of the numbers must be maintained!) to 0000100002. You should now be here:
Make sure that you save Perfstrt.dat -- C-X C-S or the usual WINDOZE file-save icon -- and exit Epsilon -- C-X C-C. At the WINDOZE command line type
PERFL
The program should only take about one minute to run and it produces three output files -- PERF21.DAT, PERF23.DAT, and PERF25.DAT. PERF21.DAT should look something like this:
21 NOVEMBER 2004 14.17.13.10.
RANDOM NUMBER SEED 33700
SEN90KH.ORD
NON-PARAMETRIC MULTIDIMENSIONAL UNFOLDING OF 90TH SENATE
2 596 20 36 1 2 10 0.005
(36A1,3900I1)
(I5,1X,36A1,2I5,50F8.3)
******************************************************************************
1 ROLL CALLS 2 6244 48878 0.12775 0.87225 0.55333 0.00000
LEGISLATORS 2 6210 48878 0.12705 0.87295 0.55576 0.00000
2 ROLL CALLS 2 6143 48878 0.12568 0.87432 0.56056 0.99832
LEGISLATORS 2 6129 48878 0.12539 0.87461 0.56156 0.99434
3 ROLL CALLS 2 6079 48878 0.12437 0.87563 0.56513 0.99853
LEGISLATORS 2 6072 48878 0.12423 0.87577 0.56563 0.99982
4 ROLL CALLS 2 6067 48878 0.12413 0.87587 0.56599 0.99935
LEGISLATORS 2 6063 48878 0.12404 0.87596 0.56628 0.99922
5 ROLL CALLS 2 6053 48878 0.12384 0.87616 0.56699 0.99761
LEGISLATORS 2 6053 48878 0.12384 0.87616 0.56699 0.99999
6 ROLL CALLS 2 6050 48878 0.12378 0.87622 0.56721 0.99981
LEGISLATORS 2 6049 48878 0.12376 0.87624 0.56728 0.99982
7 ROLL CALLS 2 6048 48878 0.12374 0.87626 0.56735 0.99976
LEGISLATORS 2 6048 48878 0.12374 0.87626 0.56735 1.00000
8 ROLL CALLS 2 6048 48878 0.12374 0.87626 0.56735 0.99983
LEGISLATORS 2 6047 48878 0.12372 0.87628 0.56742 0.99999
9 ROLL CALLS 2 6045 48878 0.12368 0.87632 0.56757 0.99987
LEGISLATORS 2 6045 48878 0.12368 0.87632 0.56757 1.00000
10 ROLL CALLS 2 6045 48878 0.12368 0.87632 0.56757 0.99986
LEGISLATORS 2 6044 48878 0.12365 0.87635 0.56764 0.99999
11 ROLL CALLS 2 6042 48878 0.12361 0.87639 0.56778 0.99986
LEGISLATORS 2 6041 48878 0.12359 0.87641 0.56785 0.99996
12 ROLL CALLS 2 6039 48878 0.12355 0.87645 0.56799 0.99985
LEGISLATORS 2 6035 48878 0.12347 0.87653 0.56828 0.99986
13 ROLL CALLS 2 6034 48878 0.12345 0.87655 0.56835 0.99974
LEGISLATORS 2 6034 48878 0.12345 0.87655 0.56835 1.00000
14 ROLL CALLS 2 6033 48878 0.12343 0.87657 0.56842 0.99986
LEGISLATORS 2 6033 48878 0.12343 0.87657 0.56842 1.00000
15 ROLL CALLS 2 6029 48878 0.12335 0.87665 0.56871 0.99991
LEGISLATORS 2 6029 48878 0.12335 0.87665 0.56871 1.00000
16 ROLL CALLS 2 6028 48878 0.12333 0.87667 0.56878 0.99987
LEGISLATORS 2 6028 48878 0.12333 0.87667 0.56878 1.00000
17 ROLL CALLS 2 6027 48878 0.12331 0.87669 0.56885 0.99986
LEGISLATORS 2 6027 48878 0.12331 0.87669 0.56885 1.00000
18 ROLL CALLS 2 6027 48878 0.12331 0.87669 0.56885 0.99980
LEGISLATORS 2 6027 48878 0.12331 0.87669 0.56885 1.00000
19 ROLL CALLS 2 6026 48878 0.12329 0.87671 0.56892 0.99988
LEGISLATORS 2 6026 48878 0.12329 0.87671 0.56892 1.00000
20 ROLL CALLS 2 5882 48878 0.12034 0.87966 0.57923 0.99358
LEGISLATORS 2 5882 48878 0.12034 0.87966 0.57923 0.99998
MEAN VOLUME LEG. 0.0057 0.0163
MACHINE PREC. 2 5882 48878 0.12034 0.87966 0.57923
MACHINE PREC. 2 5882 48878 0.12034 0.87966 0.57923
14.17.13.10.
ELAPSED TIME OF JOB 14.18.07.24.
You should reproduce the above almost exactly subject to very small variations between
CPU types and the random number draw that is used to do a final search on the legislator
polytopes. Note that the correct classification is 87.966 percent with an APRE of 0.57923
(the output files are explained in detail on the
Optimal Classification
(OC) Page).To place the legislator OC coordinates into the roll call file, bring SEN90KH.Ord up in Epsilon, split the window using the C-X 2 command and use the find-file C-X C-F command to bring up PERF25.DAT. You should be here:
Place the cursor at the beginning of the line for President Johnson and start a keyboard macro with the start-keyboard-macro command:
C-X (
where "(" is the left parenthesis. We are going to move 68 spaces to the right in PERF25.DAT, put the coordinates in the kill buffer, yank them back out, move to the beginning of the line, and move down one line so the cursor is now in front of the record for Sparkman. To do this, enter:
C-U68C-FC-KC-YC-AC-N
and you will now be at this point:
Now we are going to move to the upper window with the C-X P command (see selecting-windows commands in the Epsilon manual), move 36 spaces to the right in SEN90KH.ORD, yank the coordinates from the buffer, hit the space-bar once to open up one space between the coordinates and the first roll call, move to the beginning of the line, and move down one line so the cursor is now in front of the record for Sparkman. To finish the macro we move down to the lower window with the C-X N command (see selecting-windows commands in the Epsilon manual), and close the macro with C-X ). To do the above, enter
C-XPC-U36C-FC-YSPACEC-AC-NC-XNC-X)
where SPACE means hit the space-bar one time. You should now be here:
Execute the macro 100 times (C-U100C-XE) and this should produce:
Execute the macro 1 more time, save PERF25.DAT, close the lower window with C-X 0 (Control-X "Zero"), and then write out SEN90KH.ORD as SEN90KH_STATA.TXT using the write-file command:
C-X C-W
We are now here:
Go to the beginning of SEN90KH_STATA.TXT with the command:
Alt-< (hold Alt Key down and type <)
This will put you at the top of the file. What we need to do now is: 1) remove all the commas that exist in the name fields (you will see why in a moment); 2) run a macro that inserts commas between the variable fields (this allows us to paste into Stata); and 3) paste the file directly into the Stata data editor spreadsheet.
To begin simply use the replace-string command to replace every comma with a space (that way the columns of the file will remain in alignment). The replace-string command is:
Alt-X Replace-String
The Alt-X brings up the Command: banner and then you simply type replace-string:
Now hit Enter:
Type , and hit Enter:
At this point Epsilon remembers the last string you used in this situation and displays it in blue. Just hit Backspace if this occurs.
Now hit the Space-Bar one time -- this tells Epsilon that you are replacing every occurrence of a "," with " " -- and hit Enter:
Epsilon tells you how many replacements that it made.
To finish off our project download these two macro files
Homework-3-3.TXT -- Keyboard Macro as Text File
Comma.TXT -- Keyboard Macro to Insert a CommaSplit the window using the C-X 2 command, use the find-file C-X C-F command to bring up Homework-3-3.TXT, split the window again, and bring up Comma.TXT. You should see this:
Because homework-3-3 calls commax (defined in Comma.txt), you must load the Comma.txt file first as we did in homework 2 problem 2. With the cursor in the bottom window type Alt-X (Hold Alt Key down and type X) and after the Command: banner comes up, type load-buffer and hit the Enter key. You should get 0 errors detected.
Move the cursor up to homework-3-3.txt using C-X P and load this file as well using Alt-X and load-buffer. Finally, move the cursor up to SEN90KH_STATA.TXT using C-X P. To run the macro type Alt-X and at the Command: banner type homework-3-3 (the name in the quotes) and hit Enter. You should get:
Run the macro 100 more times using C-U100Alt-X and at the 100 Command: banner type homework-3-3
Execute the macro one more time and save SEN90KH_STATA.TXT, close the other two windows and go to the beginning of SEN90KH_STATA.TXT. Start Stata, set the memory to about 500 meg -- set mem 500m, and bring up the data editor spreadsheet:
Finally, at the top of SEN90KH_STATA.TXT type
C-@ (hold Control key down and type @)
This sets a mark in Epsilon. Now go to the bottom of the file with Alt-> and it will highlight the whole file:
Put the file on the clipboard:
Go to Stata and paste the file into the data editor using the Edit drop-down menu:
Save the file as ucsd_homework_3.dta. Turn on the logfile function in Stata (use .log file type!!!), issue the summ command, close the log file, and e-mail me the log file for a PASS on this problem.