Controlled Experiments
Scurvy is a disease that causes a problems including fatigue, sore joints, and swollen and bleeding gums. Beginning with the start of the Age of Exploration in the mid-fifteenth century, scurvy was a major cause of disability and death among European sailors. In 1753, a Scottish naval surgeon named James Lind showed that scurvy could be cured by eating citrus fruits. Though his work was not immediately acted upon, by the end of the eighteenth century British ships travelled well stocked with citrus and the problem of scurvy (vitamin C deficiency) was under control.
Lind reached his conclusions about scurvy by conducting a simple experiment. He took 12 men who all showed symptoms of scurvy and divided them into six pairs. Each pair was assigned to receive one of these treatments daily:
- 1 quart cider
- 25 drops 3 times, elixir of vitriol (a mixture of alcohol and sulfuric acid)
- 1 cup sea water
- A small paste of garlic, mustard seed, horse-radish, balsam of Peru, and gum mhyrr
- Two spoonfuls 3 times, vinegar
- Two oranges and a lemon
Each of these treatments had, at some point, been used to treat scurvy. The pair that received the oranges and lemon treatment quickly recovered sufficiently to be able to help nurse the others. Lind's work was an early controlled experiment and established that foods containing vitamin C could be used to treat scurvy.
In a controlled experiment, a researcher imposes a treatment or intervention on the subjects (experimental units) and measures the response. Explanatory variables are often called factors.
A treatment is an intervention applied to the individuals in an experiment.
In a controlled experiment, a researcher imposes a treatment on the subjects and measures their responses.
The experimental unit is the smallest collection of individual (animals, people, plants) to which the treatment is applied. When the experimental units are people, they are commonly referred to as subjects.
Principles of effective experimental design
Not all controlled experiments are equally effective. However, when a controlled experiment is well designed, it can show that the treatment causes a specific response as Linds' study showed that citrus fruits caused scurvy sufferers to improve. Experiments can be used to show cause and effect; there is no other fully convincing way to do this. A well-designed experiment incorporates the following elements:
- Comparison
- Randomization
- Blinding
- Replication
Comparison
In an effective experimental design, two or more treatments are compared. In many experiments, some subjects are assigned to a control group; a group that does not receive the treatment and can therefore be used for baseline comparison. Other subjects are assigned to a treatment group; a group that receives a treatment or intervention. In other experiments, an experimental unit is assigned to one of two more possible treatments. The comparison groups are treated the same except for the treatment itself.
Using comparison was a strength of Lind's study. If he had given the treatment to all the sailors, it would be difficult to rule out other factors that affected all the sailors (like the weather) as possible reasons for their improvement.
Treatment group: a group that receives the experimental treatment or intervention.
Control group: a group that does not receive an experimental treatment.
Blinding
A blind study is one in which the subjects do not know which treatment (if any) they received. Sometimes the subjects in the control group receive a dummy treatment called a placebo. This prevents subjects from showing a response simply because they think they've been given the treatment (the placebo effect). If neither the subjects nor the evaluators know who has received which treatment, the study is called double-blind.
Blind: a study is blind when subjects do not know whether they are receiving the treatment.
Double-blind: a study is double-blind when neither evaluators nor subjects know who receives the treatment.
Placebo: a dummy treatment (such as a sugar pill) used to prevent subjects from knowing whether they have received the real treatment.
Randomization
A chance mechanism should be used to assign experimental units to treatment groups. Randomization eliminates human influence from the choice of who receives which treatment.
Replication
Each treatment should be administered to many experimental units to reduce chance variation in the results. In Lind's study, each of the treatments was given to only two sailors. There is stronger evidence for the efficacy of the treatment when the experimental groups are large.
When an experiment is well designed, a substantive difference observed between the control and treatment groups is likely due to the treatment rather than to chance. If an observed result is so large that it would rarely occur by chance it is call statistically significant. Despite these principles, many experiments still have some flaws. Evidence is most convincing when multiple experiments have shown the same thing.