How to do your own experiments in psychology and education
7.3 "Run" participants
You’ve already done all the planning; you’ve reserved and tested the equipment; you’ve copied all the materials; you’ve got people signed up to come. It’s show time!
Salkind’s (2006, p. 150) fifth Commandment of data collection contains a very relevant warning here: “Do not rely on other people to collect or transfer your data unless you personally have trained them and are confident that they understand the data collection process as well as you do.” That’s why you have done so much planning and have a script to follow: you can use these as training materials for your assistants. There’s much too much at stake for you to let any untrained person collect data for you: don’t ever simply turn over your materials for someone else to “just” run your experiment.
Trials and sessions. Each time a participant responds to a stimulus or group of stimuli it’s called a trial. If a participant reads one text and answers questions about it, that’s one trial. If a participant sees 20 words and for each one has to give a response, that’s 20 trials.
You will probably run more than one group of participants – each time you run a group of participants, it’s called a session. Sometimes you can run one session for two of your experimental conditions; sometimes you run one session for each condition. Make sure that you know how many sessions you will need to run. That will help you figure out how long your data collection will take.
You want to focus on three things when you’re running your participants: Ethics, Consistency, and Accuracy.
Ethics. It is your responsibility to treat all participants extremely ethically. This means, among other things, that it is your job as an experimenter to ensure that:
For anonymity, assign each participant a unique code like “12” for the 12th participant in the experiment. Get participants’ names only when you really, really, really need them (which is very rare). Make sure that the participants’ names are not visible during data coding – start using the participant codes as soon as possible, so that what you know about the participants doesn’t affect how you code their responses.
Read the APA’s Ethical Principles of Psychologists and Code of Conduct at http://www.apa.org/ethics/code2002.html for details. Michell & Jolley’s (2007) Appendix C has a clear and helpful discussion of ethical considerations.
Consistency. Your main goal when running the participants is to ensure that ALL of them participate under the same conditions, except for the differences that are planned parts of the experiment. But things happen -- there may be a flash of lightning, a car crash, a telephone rings, someone starts singing an opera, etc. – that may affect the outcome.
If this happens, don’t panic. Continue the experiment, if possible. What most experimenters do is just take notes during each run of the experiment. They observe anything that was non-routine: papers fell on the floor so it took longer to collect the stimuli; a door slammed during the instructions; there was a short brown-out; etc. These notes may be helpful in explaining the results later on.
Random assignment. Part of consistency is random assignment. Don’t forget to assign participants to the different experimental conditions randomly. Use a list of random numbers: the first 15 random numbers are the participants in the first experimental condition; the second 15 random numbers are the participants in the second experimental condition, etc. Then sort the numbers numerically: the first number tells you which condition is for the first participant who shows up, the second number the condition for the second participant, etc.
In practice, researchers emphasize random assignment (of participants to experimental conditions) instead of random selection because random assignment is more practical to carry out. Perhaps the most common technique for random assignment is to use a table of random numbers like the one here.
Say that you need to assign 40 participants to 4 experimental conditions. You can follow these steps:
Why is this random? For one thing, the assignment does not depend on any characteristics of the participants or of the experimenter. If you did the sampling yourself, you might unconsciously assign the smart (or male, or white, or …) participants to one condition or other. For another, the random number table is designed to provide a sequence that’s very close to pure randomness.
Accuracy. Accuracy in this case refers to being extra, super sure that participant 7 (for example) gave these responses to the background questionnaire, participated in this experimental condition with those materials, gave these other responses to the main task, etc. If any of this information gets mixed up, then the data goes to the trash! You have to be totally, absolutely sure. This is why researchers are usually required to keep their data for at least five years after the results have been published – so that their accuracy can be verified later on. This is Salkind’s (2006) ninth Commandment of data collection: don’t throw out the original data.
Don't rely on the participants to write their special number on the background questionnaire and each of the response sheets. They forget, ignore you, get distracted -- and their data becomes useless! Do it yourself and make absolutely, totally sure that each participant gets the same number each time you distribute something!
Collect your data. Go for it! Have the participants sit down and then follow your script as precisely as possible. Repeat until you have enough participants.
| Please plan ahead before you actually collect your data. At the very least, read to the end of this Task. |
Salkind’s (2006, p. 150) fifth Commandment of data collection contains a very relevant warning here: “Do not rely on other people to collect or transfer your data unless you personally have trained them and are confident that they understand the data collection process as well as you do.” That’s why you have done so much planning and have a script to follow: you can use these as training materials for your assistants. There’s much too much at stake for you to let any untrained person collect data for you: don’t ever simply turn over your materials for someone else to “just” run your experiment.
Trials and sessions. Each time a participant responds to a stimulus or group of stimuli it’s called a trial. If a participant reads one text and answers questions about it, that’s one trial. If a participant sees 20 words and for each one has to give a response, that’s 20 trials.
You will probably run more than one group of participants – each time you run a group of participants, it’s called a session. Sometimes you can run one session for two of your experimental conditions; sometimes you run one session for each condition. Make sure that you know how many sessions you will need to run. That will help you figure out how long your data collection will take.
You want to focus on three things when you’re running your participants: Ethics, Consistency, and Accuracy.
Ethics. It is your responsibility to treat all participants extremely ethically. This means, among other things, that it is your job as an experimenter to ensure that:
| a) they participate with informed consent – they have to know what they’re getting into and agree to it – and they can leave the experiment at any time; | |
| b) they are protected as much as possible from physical, social, and psychological risks; | |
| c) their participation and responses remain anonymous; | |
| d) it’s also a good idea to be nice to your participants – they’re doing you a favor; |
For anonymity, assign each participant a unique code like “12” for the 12th participant in the experiment. Get participants’ names only when you really, really, really need them (which is very rare). Make sure that the participants’ names are not visible during data coding – start using the participant codes as soon as possible, so that what you know about the participants doesn’t affect how you code their responses.
Read the APA’s Ethical Principles of Psychologists and Code of Conduct at http://www.apa.org/ethics/code2002.html for details. Michell & Jolley’s (2007) Appendix C has a clear and helpful discussion of ethical considerations.
Consistency. Your main goal when running the participants is to ensure that ALL of them participate under the same conditions, except for the differences that are planned parts of the experiment. But things happen -- there may be a flash of lightning, a car crash, a telephone rings, someone starts singing an opera, etc. – that may affect the outcome.
If this happens, don’t panic. Continue the experiment, if possible. What most experimenters do is just take notes during each run of the experiment. They observe anything that was non-routine: papers fell on the floor so it took longer to collect the stimuli; a door slammed during the instructions; there was a short brown-out; etc. These notes may be helpful in explaining the results later on.
Random assignment. Part of consistency is random assignment. Don’t forget to assign participants to the different experimental conditions randomly. Use a list of random numbers: the first 15 random numbers are the participants in the first experimental condition; the second 15 random numbers are the participants in the second experimental condition, etc. Then sort the numbers numerically: the first number tells you which condition is for the first participant who shows up, the second number the condition for the second participant, etc.
In practice, researchers emphasize random assignment (of participants to experimental conditions) instead of random selection because random assignment is more practical to carry out. Perhaps the most common technique for random assignment is to use a table of random numbers like the one here.
Say that you need to assign 40 participants to 4 experimental conditions. You can follow these steps:
| • Choose a two-digit number arbitrarily (from the serial number of a dollar bill, a license plate, the page of a book opened at random, etc.). | |
| • Use the first digit and the column number and the second digit as the row number to find where to start in your table of random numbers. | |
| • Go down the column one number at a time. If it is less than 41, then write that number on the list for experimental condition #1 until you have enough participants for condition 1. If the number is more than 40 (your total) or was already seen, then skip it. If you get to the bottom of a column, go to the next column. | |
| • Repeat until you fill all of the experimental conditions. |
Why is this random? For one thing, the assignment does not depend on any characteristics of the participants or of the experimenter. If you did the sampling yourself, you might unconsciously assign the smart (or male, or white, or …) participants to one condition or other. For another, the random number table is designed to provide a sequence that’s very close to pure randomness.
Accuracy. Accuracy in this case refers to being extra, super sure that participant 7 (for example) gave these responses to the background questionnaire, participated in this experimental condition with those materials, gave these other responses to the main task, etc. If any of this information gets mixed up, then the data goes to the trash! You have to be totally, absolutely sure. This is why researchers are usually required to keep their data for at least five years after the results have been published – so that their accuracy can be verified later on. This is Salkind’s (2006) ninth Commandment of data collection: don’t throw out the original data.
Don't rely on the participants to write their special number on the background questionnaire and each of the response sheets. They forget, ignore you, get distracted -- and their data becomes useless! Do it yourself and make absolutely, totally sure that each participant gets the same number each time you distribute something!
Collect your data. Go for it! Have the participants sit down and then follow your script as precisely as possible. Repeat until you have enough participants.
| Read this topic next: | 7.4 Organize and Code your data |
About
© 2009 Created by Mike Dillinger, PhD on Ning. Create a Ning Network!
