ADDENDUM TO PROGRAMMING PRINCIPLES

Subject 999

All programs should have the feature that when the user enters the subject number 999, the program does not write any data to the disk. That way, we can check the program over and over without having to worry about mixing up test data with real data, and without finding a new subject number that the program will accept. As a further convenience, if the program requires a lot of additional data to be entered by the user, when you detect subject 999 fill the remaining items in with test data (“John Smith”, etc.). Always allow the user to test different order conditions, however.

Checking Programs for Accuracy

1. As you work on the program, keep the specs files current and accurate (any changes we make in the programs should be reflected in the specs file).

2. Always save everything about the events on a trial unless we specifically agree not to. This even includes information that is redundant (inferable from other information already saved). When stimuli are randomly selected on a given trial, save them. When stimuli have multiple attributes like color, save those. When the subject makes a response, save the response and the time – don’t just save whether it was correct even if the experiment is focused on accuracy rather than reaction time.

3. Before “releasing” program, please test it for accuracy in the following ways:

a. Check that whenever the program is supposed to equalize numbers of trial types, they really are equalized (you can do this by running yourself for a block of trials and then looking in the raw data file and counting up the numbers in a given column).

b. You can test that both the experiment program and the analysis program work properly in the following “trust-nothing” way: (1) make a miniature version of the experiment (e.g., instead of 10 blocks of 40 trials, have 10 blocks of 8 trials). Run yourself right through the mini-experiment but decide in advance deliberately to respond very slowly in one condition (e.g., in Folk3 you could deliberately take an extra two seconds to respond in the blocks where the letters move). Analyze the raw data and make an S-file. Now check that the reaction time that you deliberately retarded is indeed slow. This simultaneously checks all your programs and the labeling of rows and columns in your S-file. In my years of experiment programming I found that from time to time this turned up a horrible error that would otherwise not have been noticed!