Design Elements
Experimental or Observational StudiesIn experimental studies the intervention is under the control of the researcher. For example, the research team may determine (by random allocation) (1) which subjects receive a novel treatment and which ones get traditional (or no) treatment, (2) when an intervention is carried out in a community, or (3) how much of a new drug each patient is given. The aim is to determine how changes in the independent variable (the one under the researcher’s control) affect some outcome (the dependent variable). By controlling the timing or amount of the intervention or which subjects get it and which ones do not, the chances are minimized that other factors outside of the researcher’s control could have affected the results.
By contrast, the researcher does not control the intervention in observational studies but rather observes the effects of an experiment in nature. It would be both unethical and impractical, for example, to expose some people to cigarette smoke or putative occupational carcinogens deliberately for 20 years to determine their effects. However, by choice or chance, some people have been exposed so it is possible to draw some tentative conclusions based on observation of these subjects and, if possible, control subjects.
Most well-designed studies of a new treatment are experimental in that the research team determines which subjects receive the new drug or intervention and which ones receive traditional treatment or a placebo. Almost all studies that involve exposure to harmful agents or that try to trace the natural history of a disorder are observational. However, these general rules naturally have exceptions. For example, if VDTs were being introduced gradually into the workplace so that there were fewer terminals than eligible workers and there was no hard evidence of any adverse effects, women could be randomly assigned to work with them or continue to use typewriters. However, this may be difficult to do because of practical considerations, and an observational type of study may be more realistic. (Needless to say, the researcher cannot control which women become pregnant. The last one who tried was hauled up on morality charges.)
Number of ObservationsThe simplest research design would involve looking at or measuring the outcome only once. In many cases, such as when the outcome is either present or absent or when the timing of the outcome is of minor interest, one observation may be all that is necessary. For example, if the question is whether working at a VDT results in a higher incidence of stillbirths, miscarriages, or congenital abnormalities, then we could record these outcomes 9 months after conception for this group of women and for an appropriate control group. The outcome is recorded only on a single occasion.
However, if we were interested in the time course of an outcome, one observation is not sufficient. To use a different example, Bagby and his colleagues looked at the effects of a new mental health act introduced toward the end of 1978 on the proportion of psychiatric patients who were involuntarily admitted to the hospital ( Fig.1213). The graph shows a dramatic decline in this type of admission after the new, more restrictive legislation. If the analysis had stopped at this point, it’s wisely that people would have come to the erroneous conclusion that the new act resulted in a reduction in the proportion of people being admitted to psychiatric wards on an involuntary basis. Multiple observations over time, however, show a different picture, that is, a gradual return to a level even higher than those of the 7 years preceding the new law. So not only do multiple observations tell us something different than a single look, they also reveal something about the “natural history” of the legislation; there was a gradual return to the previous mode of practice as psychiatrists learned to live with the new law.
Figure 1213 – (Figure 3-1) The proportion of psychiatric patients involuntarily admitted to hospital before and after the new mental health act of 1978
From Bagby RM, Silverman I, Ryan DP: Effects of mental health legislative reforms in Ontario. Can Psycho1 28:21-29, 1987.
Some figures may not display clearly when rendered as a PDF or printed.
Direction of Data GatheringData can be gathered in one of two ways: (1) looking forward and getting new data after the start of the study or (2) looking backward and using data that have already been collected. Specific names are used for each of these strategies. Studies that involve gathering data after the study has begun are called prospective; in retrospective studies the data have already been recorded for other reasons at some time in the past. The advantage of prospective data collection is that the nature of the data, the definitions of symptoms, the method by which the data are recorded, and other factors can be worked out ahead of time and are constant throughout the course of the trial. In retrospective studies, definitions of symptoms or diseases may have been modified over time, units of measurement may have changed, and old methods for diagnosis may have been replaced, thereby resulting in more variability in the data. Perhaps the greatest advantage of prospective studies is that they allow us to determine the directionality of events (i.e., what occurred first and what happened later). As we’ll see in Chapter 5, directionality is necessary (hut not sufficient) if we want to be able to say anything about causation. Information of this sort is far more difficult (some would say impossible) to obtain accurately in retrospective studies.
Conducting the study retrospectively would involve identifying all women who were pregnant and worked with VDTs at least 9 months ago and then either interviewing them or reviewing their hospital charts to determine the outcome of the pregnancy. This is advantageous because the study could he done relatively quickly, but it suffers from a few risks: the type of terminals may have been changed, it may be difficult to establish how much time the women spent in front of the VDTs, and hospital documentation of all possible birth defects may he difficult to acquire (e.g., miscarriages may not have been recorded in hospital records). A prospective study would enter women into the trial only if they became pregnant after the start date. Although the researcher could now record all the relevant information with greater accuracy, the study might have to continue for a few years until enough women became pregnant to allow analysis of the results.
The term “prospective” should not be used to describe trials in which historical data are gathered after a diagnosis or exposure that occurred some time in the past. For example, if we gather hospital utilization data from 1945 to the present on people who witnessed the A-bomb tests in Nevada, the data would still be retrospective, although the hospitalizations occurred after the exposure. Even though the subjects were followed forward in time, the data involve events that happened before now; therefore the study would be called retrospective ( Fig.1215).
A few authors have tried to clarify this confusion in nomenclature by introducing terms such as “retrolective,” “prolective,” or “retrospective- prospective.” Laudable as this goal is, we feel that these neologisms have only further obfuscated the sufficiently murky picture.
Comparison GroupsKeeping with our study of women who worked with VDTs, we could easily derive prevalence figures for each of the outcomes of interest (stillbirths, miscarriages, and congenital abnormalities), but the meaning of these numbers would be unclear. The major reason is that women who do not work in front of VDTs also experience these adverse effects.
So now the question has become somewhat more complicated: Do women who work with VDTs have these outcomes at a higher rate than women who do not work in front of terminals? This means that we now need a group against which we can compare our prevalence results to deter- mine if the rate is higher.
There are two major types of comparison or control groups: historical and concurrent. In the former case, we would compare our results with data that already exist from previous studies (e.g., a large survey of the prevalence of miscarriages, stillbirths, and congenital abnormalities in the general population). If such data do not exist or if they are suspect for one reason or another, the researchers must gather information from a control group concurrently; in essence, the researchers should have at least two groups in the study.
Figure 1215 – (Figure 3-2) Prospective versus retrospective studies
Streiner DL, Norman GR. PDQ Epidemiology-Second Edition, 1996, BC Decker Inc., Hamilton, Ontario.
Some figures may not display clearly when rendered as a PDF or printed.
When good historical control groups exist, they can save a considerable amount of time, effort, and expense. Unfortunately, most historical control groups are compromised for some reason. The primary reason is that factors in the environment, such as clinical policies, may have changed since the data were originally gathered. For example, not too long ago few infants weighing less than 2500 g survived, whereas now it is not uncommon for neonatologists to save kids who weigh less than 1 kg. So, if infant mortality were one of our endpoints, it may appear that women who work with VDTs have a lower infant mortality rate than the historical controls. Conversely, it may be expected that infants who are born weighing 800 g or less may have more abnormalities than kids who were born weighing 2500 g or more. So the overall prevalence of birth defects may be increasing throughout time. The result is that this outcome may look poor when compared with a historical control, irrespective of any effect VDTs may have. The lesson is that when a historical control is used, we have to be certain that nothing has changed in the interim that could affect its comparability with the group we are looking at now.
On rare occasions a control group may not be necessary at all. To quote Bradford Hill, “If we survey the deaths of infants in the first month of life and find that so many are caused by dropping the baby on its head on the kitchen floor I am not myself convinced that we need controls to satisfy us that this is a bad habit.” The classic case of a study where a control group was unnecessary was the use of streptomycin for tuberculous meningitis; without treatment the disease was universally fatal so any improvement in survival was significant. Fortunately or unfortunately, such examples are rare.
Content on this page was last changed on March 19, 2009.
© 2002 BC Decker Inc. Show Disclaimer
| 5476. | Streiner DL, Norman GR. PDQ Epidemiology. 2nd ed. Hamilton, Ontario: BC Decker Inc.; 1996. |