How to do your own experiments in psychology and education
4.2 Choose the levels of your factors
For every factor that you investigate, you will be contrasting two or more values of that factor. For example, if you’re studying the effects of Gender, you my compare people with a value of “male” for Gender and people with a value of “female” for Gender. In other words, you’ll focus on male and female as levels of the factor Gender.
However, you may have noticed that simply choosing “male” and “female” for Gender is really quite a simplification: unusual genetic combinations and lifestyle choices sometimes lead to people who cannot easily be classified as clearly male or clearly female. Male vs. female, then, is a simplification of reality, and this simplification adds some error or distortion to your experiment.
Simplification adds some error or distortion to your experiment, but it is an absolute necessity.
Simplification is an absolute necessity: you’ll have to simplify different things at many points in research. Researchers have to be aware of the distortions that simplification can add and try to minimize them where the distortions can have important effects. In other situations, researchers just have to accept this consequence of simplification and keep in mind that research (by definition) can’t be perfect. This is one of the reasons why researchers don’t actually “prove” anything: they know that every experiment involves many simplifications and some margin of error.
Researchers don’t actually “prove” anything.
To choose levels for your factors, you need to think about two issues:
a) Simplicity. You want to maximize the simplicity of your experiment so that you can execute it reliably. In most experiments, you’ll limit yourself to two or three levels for each factor. One common notation for this is the form [factor] ([value1], [value2], etc.), where you use the names for your own factors and values. For example:
In the clearest case, the two levels of the factor are present and absent. For example, some participants might read a text with a picture, while others read it without the picture. In this case, you’d be studying the effects of the factor Picture, which has two levels: with Picture and without Picture. You might also want to study the same factor with three (or more) levels, for example: with Picture, with partial Picture, and without Picture, for the case where you also show only a part of the picture.
b) Contrast. You want to choose levels of a factor so that the difference or contrast between them is very clear. Your experiment will focus on the effects of the difference between the levels of each factor. The contrast between these levels is very important. If the levels are too similar, then it will be harder to measure the differences in their effects.
One way of emphasizing the contrast between levels is to ignore some possible intermediate values when you define your levels. For example, Age might be defined with levels Young (<30 years old) and Old (>30 years old). Notice, however, that many of the participants in the two groups will be quite similar: the 28-, 29-, 31- and 32-year-olds. A more effective way of defining the contrast would be to skip some of the ages and define Young (<20 years old) and Old (>40 years old), leaving a gap in the middle of ages that you will ignore. This way the contrast between the two groups becomes much clearer.
Control groups are the simplest way of establishing a contrast: one group gets no experimental treatment (the control group) and another gets the experimental treatment (the experimental or treatment group). Talking about control groups usually doesn’t work for the more sophisticated experiments that you will do in this course: what’s the “control” group if you compare males and females? There is no control group in this case. When you investigate the effects of two or more factors, the notion of control group gets even more confusing: if you have males and females reading a story with or without music, which is the control group? It’s difficult to say. Avoid talking about control groups for this course.
“Baptize” your factors and levels
You will have to talk about your factors and levels more and more as you develop your experiment and write it up. It’s best to give your factors and levels specific names right now and use these names consistently. You don’t want to have to say “the differences between having a time limit and not having a time limit” each time you mention your factor; you can define this as one of your factors and call it Time.
Do not ever use general, uninformative names like Group A, Factor 1, etc. for your factors or levels.
If you do, then no one will understand anything that you’re saying!
Do you understand a sentence like “Group A recalled significantly more information than Group B”? Neither does anyone else. And experimenters who use these names end up confusing themselves, too. “The no-time-limit group recalled significantly more than the time-limit group” is much more informative. “No time limit” and “time limit” are the names that one experimenter chose for the levels of his factor, which he called Time.
Notice that factor names are capitalized and level names are usually not capitalized.
“The Time factor had two levels: time limit and no time limit.”
“The first factor was Time (time limit, no time limit); the second factor was Gender (male, female).”
When you have a name that has more than one word, like “no time limit”, remember to put in hyphens when you use the name as an adjective.
“In the no-time-limit condition, participants…”
“The no-time-limit group recalled…”
Read this topic Next: Identify your experimental conditions
However, you may have noticed that simply choosing “male” and “female” for Gender is really quite a simplification: unusual genetic combinations and lifestyle choices sometimes lead to people who cannot easily be classified as clearly male or clearly female. Male vs. female, then, is a simplification of reality, and this simplification adds some error or distortion to your experiment.
Simplification adds some error or distortion to your experiment, but it is an absolute necessity.
Simplification is an absolute necessity: you’ll have to simplify different things at many points in research. Researchers have to be aware of the distortions that simplification can add and try to minimize them where the distortions can have important effects. In other situations, researchers just have to accept this consequence of simplification and keep in mind that research (by definition) can’t be perfect. This is one of the reasons why researchers don’t actually “prove” anything: they know that every experiment involves many simplifications and some margin of error.
Researchers don’t actually “prove” anything.
To choose levels for your factors, you need to think about two issues:
a) Simplicity. You want to maximize the simplicity of your experiment so that you can execute it reliably. In most experiments, you’ll limit yourself to two or three levels for each factor. One common notation for this is the form [factor] ([value1], [value2], etc.), where you use the names for your own factors and values. For example:
- Gender (male, female);
- Music (presence, absence);
- Amount of sleep (less than 6 hours, more than 8 hours)
In the clearest case, the two levels of the factor are present and absent. For example, some participants might read a text with a picture, while others read it without the picture. In this case, you’d be studying the effects of the factor Picture, which has two levels: with Picture and without Picture. You might also want to study the same factor with three (or more) levels, for example: with Picture, with partial Picture, and without Picture, for the case where you also show only a part of the picture.
b) Contrast. You want to choose levels of a factor so that the difference or contrast between them is very clear. Your experiment will focus on the effects of the difference between the levels of each factor. The contrast between these levels is very important. If the levels are too similar, then it will be harder to measure the differences in their effects.
One way of emphasizing the contrast between levels is to ignore some possible intermediate values when you define your levels. For example, Age might be defined with levels Young (<30 years old) and Old (>30 years old). Notice, however, that many of the participants in the two groups will be quite similar: the 28-, 29-, 31- and 32-year-olds. A more effective way of defining the contrast would be to skip some of the ages and define Young (<20 years old) and Old (>40 years old), leaving a gap in the middle of ages that you will ignore. This way the contrast between the two groups becomes much clearer.
Control groups are the simplest way of establishing a contrast: one group gets no experimental treatment (the control group) and another gets the experimental treatment (the experimental or treatment group). Talking about control groups usually doesn’t work for the more sophisticated experiments that you will do in this course: what’s the “control” group if you compare males and females? There is no control group in this case. When you investigate the effects of two or more factors, the notion of control group gets even more confusing: if you have males and females reading a story with or without music, which is the control group? It’s difficult to say. Avoid talking about control groups for this course.
“Baptize” your factors and levels
You will have to talk about your factors and levels more and more as you develop your experiment and write it up. It’s best to give your factors and levels specific names right now and use these names consistently. You don’t want to have to say “the differences between having a time limit and not having a time limit” each time you mention your factor; you can define this as one of your factors and call it Time.
Do not ever use general, uninformative names like Group A, Factor 1, etc. for your factors or levels.
If you do, then no one will understand anything that you’re saying!
Do you understand a sentence like “Group A recalled significantly more information than Group B”? Neither does anyone else. And experimenters who use these names end up confusing themselves, too. “The no-time-limit group recalled significantly more than the time-limit group” is much more informative. “No time limit” and “time limit” are the names that one experimenter chose for the levels of his factor, which he called Time.
Notice that factor names are capitalized and level names are usually not capitalized.
“The Time factor had two levels: time limit and no time limit.”
“The first factor was Time (time limit, no time limit); the second factor was Gender (male, female).”
When you have a name that has more than one word, like “no time limit”, remember to put in hyphens when you use the name as an adjective.
“In the no-time-limit condition, participants…”
“The no-time-limit group recalled…”
Read this topic Next: Identify your experimental conditions
About
© 2009 Created by Mike Dillinger, PhD on Ning. Create a Ning Network!
