Attention and Perception Lab
General Programming
Principles
Overall Concepts and Terms
An experiment is generated by a single VB program. Usually from 5 – 100 people (subjects) participate in each experiment.
When a subject comes to participate in an experiment, they first sign some consent forms. Sometimes they also read a page of instructions describing the experiment. Sometimes the instructions are provided by the computer program. In general, it’s best if the computer program provides the instructions.
The experimenter (research assistant) starts the VB executable program. The program asks for some information on the subject, which the experimenter types in. Normally this consists of the subject number for the subject and the subject’s initials. The subject numbers are consecutively applied to subjects 1 – n, where n is the number of subjects in the experiment. The subject numbers also become the labels on the output files containing the raw data for that subject.
Most experiments last about 40 minutes and consists of a number of blocks. Each block contains a number of trials (individual events in which some stimuli are presented and some response(s) are made). A typical experiment might consist of 10 blocks of 40 trials per block, for a total of 400 trials in the experiment.
Every experiment has some independent variables (IVs) and some dependent variables (DVs). IVs are what are manipulated; DVs are what are observed. An experiment basically looks at the DVs as a function of the IVs.
In most of our experiments, there are two DVs relating to subjects’ performance: reaction time and proportion of errors. Reaction time (abbreviated RT) is the time taken by the subject to respond. Proportion of errors is the proportion of trials on which the subject makes the correct response.
Normally we look at average correct RT in milliseconds. That is, when we average the RTs we do not count RTs on error trials.
Independent variables are of two types: IVs which are varied between blocks, and IVs which are varied between trials (but within the block). We speak of an independent variable having different levels. The levels are the particular values that the variable may take.
The following example should help make this clearer.
A Concrete Example: Visual Search
Folk3 had two independent variables:
Special Terms and Concepts
Suppose we have a block composed of 40 trials, and we have an independent variable with two levels manipulated between trials (like target presence/absence in Folk3). One thing we could do is choose the target presence/absence at random independently on every trial – like flipping a coin. Another thing we could do is guarantee that on precisely 20 of the 40 trials, the IV will have one level, and on the other 20 trials it would have the other level.
We do the second of these by specifying that there are two trialtypes. Trialtype is a variable which is set to one of two levels subject to the constraint that the number of trials with each value is equalized within a block.
We also have independent variables that are varied between blocks. For this, we use a construct called blocktype. The blocktype refers to a variable that determines which level of this IV we use.
A confound refers to a problem in an experiment where one tries to manipulate one IV and accidentally manipulates something else. We must make sure our experiments do not have confounds in them!
With IVs that are manipulated between block the trick is to make sure we do not confound the IV with the amount of practice the subject has in the task. Subjects get better and better at these tasks (their RTs and error rates get smaller over trials).
Suppose we have an IV manipulated between blocks with five levels – level 1, …, level 5. Suppose we had every subject do five blocks of trials. Block 1 was always done with level 1; block 2 with level 2;…., block 5 with level 5. Then suppose we find that subject do best in level 5; next best in level 4; and so on. The problem of course would be that we would have confounded level with practice.
The solution is to use counterbalancing. Counterbalancing refers to having different subjects experience different combinations of the IV and the “nuisance variable” (practice).
The table below shows the normal way we solve this. Subjects are assigned to an order condition. The entry in the table is the level of the IV, which is a joint function of the block and the order condition. If we run 5*k subjects, where k is an integer, then over subjects we have eliminated the confound.
Order condition of subject
Block 1 2 3 4 5
1 1 2 3 4 5
2 2 3 4 5 1
3 3 4 5 1 2
4 4 5 1 2 3
5 5 1 2 3 4
6 1 2 3 4 5
7 2 3 4 5 1
8 3 4 5 1 2
9 4 5 1 2 3
10 5 1 2 3 4
File
Structure
The executable VB file creates n data files, normally called R-1, …, R-n, where n is the number of subjects. These are called rawfiles, and are saved on the hard disks of the experiment running machines (in the booths) and backed up immediately on zipdisk.
The contents of the raw files normally consist of x rows, where x is the number of trials in the experiment. Each row of the rawfile contains at a minimum the block number, the trial number, the levels of the independent variables, the subject’s response (which key was pressed) and the response time. Very often it also contains other details about the stimuli presented, and perhaps about the responses made.
It is very important that when in doubt, the experiment program should save every detail of what happened on a given trial in the rawfile! Disk storage is cheap, but data is not. If we have run 50 subjects and find we should have saved something, this is a serious error. If we save thousands of pieces of information we don’t need, it doesn’t matter at all.
After an experiment has been run, you will get all the rawfiles. At this point, you need to make a program that goes through the rawfiles and creates what we call an sfile (short for “summary file”). This contains averages of the values of the dependent variables for each subject. In an sfile, there is normally one line per subject. We analyze this with a statistical package called Unix|stat.