NUR390: The Class: Research Design & Sampling: Research Design: Lesson

Lesson: Research Design

Experimental Designs

There are two types of experimental research designs: true experimental and quasi-experimental. These designs are generally used to determine cause and effect. They allow us to make inferences. According to David Hume, there are three criteria to be met to infer causality:

1. Contiguity between the presumed cause and effect. In other words, the causal variable (independent) and the effect variable (dependent) must be associated.

2. Temporal Precedence. In other words, the cause must precede effect.

3. Constant conjunction. In other words, the cause is present whenever the effect is obtained.

All experiments involve

1. treatment

2. outcome

3. units of assignment

4. comparison from which change can be inferred and attributed to the treatment.

Characteristics of True Experiments

1. manipulation

2. control

3. randomization



Manipulation

The experimenter does something to at least some of the subjects in the experiment. The introduction of that something is often referred to as the experimental treatment or experimental intervention. This is also known as the independent variable. The researcher manipulates this independent variable by administering it to some subjects and withholding it from other subjects. In other words, the experimenter varies the independent variable and observes the effect that the manipulation has on the dependent variable of interest.

Control

The notion of control summarizes all of the major experimental activities. They are manipulation, use of comparison (control) groups, and randomization. We talked a bout manipulation. We will now discuss control groups and then randomization.

Campbell and Stanley (1963) observed that obtaining scientific evidence requires at least one comparison. Control groups are used for this purpose. The term control group refers to the subjects that do not receive the experimental treatment and their performance on the dependent variable serves as a basis for evaluating the performance of the experimental group (the group who received the experimental treatment) on the same dependent variable.

Randomization

The assignment of subjects to groups on a random basis. The term random essentially means that every subject has an equal chance of being assigned to any group. The intent is to equalize the groups however, there is no guarantee that the groups will be equal. The chance of obtaining unequal groups diminishes as the sample size increases. Researchers generally use a table of random numbers to facilitate the randomization process.

There are four types of true experimental design

Post-test only design

Pre-test-Post-test design

Solomon four group design

Factorial design

The post-test only experimental design is a simple design. It is called post-test only because the data is collected after the experimental treatment is complete.

Example: Hypothesis is that the color of a pediatric nurses uniform affects the degree to which children display positive and negative affective behavior (laughing, crying). The causative or independent variable is the uniform color and the effect variable or dependent variable is the child’s behavior. The independent variable is manipulated by assigning some nurses white uniforms and some nurses colored uniforms. Thus, in this study we could compare the affective behaviors of children cared for by nurses in white uniforms and those cared for nurses in colored uniforms.

The pre-test-post-test experimental design is more complex. It is called pre-test-post-test because there are two points of measurement, one before the experimental treatment and one after the experimental treatment.

Example: Suppose we wanted to examine the effect on the heart rate of being restrained. The design would involve imposing a posey belt on the experimental group and no posey belt on the control group. The dependent variable which is the heart rate would be measured at two points in time. Before the posey belt and after the posey belt. This allows us to examine if heart rate changes were produced as a result of being restrained.

The Solomon four-group design is a version of the pre-test-post-test design. It adds two addition groups. The purpose of adding the two groups is to separate the effects of the pre-test and to segregate it from the intervention.. In other words, a pre-test may be a sensitizing treatment that may affect the results of an actual treatment.

1 is an experimental groups without the pretest

1 is the control group without the pre test

Example: If our intervention was a workshop to improve nurses attitudes toward alcoholic patients, the pre-test may sensitize the nurses and affect their attitudes at that point and obscure the analysis of the workshop’s effect.

The factorial designs allows the researcher to manipulate more than one independent variable at a time. Usually these designs contain two to four variables. Using more than four variables becomes too complex. These designs allow us to examine main effects and interaction effects. Main effects are obtained from examining each treatment variable and interaction effects are obtained by combining the treatment variables.

Example: We are interested in examining the effects of two therapeutic strategies for premature infants. The first is auditory stimuli and the second is tactile stimuli. The dependent variable is infant development.

Strengths of the Experimental Design

Some researchers believe that this is the most powerful research design as it gives us cause and effect relationships. If we do this then we can expect that. This if-then relationship is important to those health care practitioners who want to predict and control. Therefore the strength lies in the fact that causal relationships can be inferred. This is not without controversy. Some scholars believe that the notion of causality among phenomenon is untenable.

Weakness of the Experimental Design

There are a number of variable of interest that are not amenable to manipulation. For instance we cannot randomly confer upon infants their weight at birth to observe the effect of birth weight on subsequent morbidity.

Ethical considerations may prevent the manipulation of the independent variable. You would not inflict pain for the sake of an experiment.

Artificial circumstances may affect the results. Laboratory designs take place in an artificial setting. Easier to control for external variables and is not as generalizable because it constrains the human experience. Field designs take place in the actual setting and may be better but there are more problems with controls.

The Hawthorne Effect: effect of being in the study group may be sufficient to cause people to change their behaviors. This is the reason that double blind studies are conducted. In which neither subject nor those who administer the treatment knows which is the experimental or control group.

Experimental Designs in Nursing

1. experimental designs assumes that all variables have been described

2. many of the variables are not amenable to manipulation

3. experimental design may interrupt the normal health care practice

4. when several nurses are caring for a patient, it is hard to be assured that all interventions are carried out exactly the same

5. It is very difficult to disguise nursing interventions

Quasi Experimental Designs

This type of design involves a treatment (manipulation ) and an outcome but lacks one of the other two properties that characterize a true experiment: randomization or a control group.

Example: if you want to study the effects of smoking on a variable, you cannot randomly assign people to smoking vs nonsmoking group.

Types of Quasi-experimental Designs

Non equivalent control groups - other than the absence of randomly assigned groups, these designs are similar to experimental designs . However, lack of random assignment to control and experimental groups, can not assure that the groups are equal. The researcher must do everything possible to show that there are no differences. For example, a pretest may show that there is no difference. If the study is done on "after only data", this control is not present

Time series designs - tests for changes over time. There is no randomization and there is no control group. However, the researcher can use each person as his own control. If possible, test the group several times prior to introducing the intervention. This method gives more reliability to the study

Validity of Research Designs

Internal Validity refers to the extent to which it is possible to make an inference that a relationship is causal (the experimental manipulation resulted in the observed differences). According to Cook and Campbell (1979), there are 13 threats to internal validity:

1. History (events take place between the pre-test and the post-test that are not the treatment of research interest)

2. Selection (difference between kinds of people in one experimental group as opposed to another)

3. Maturation (observed effect is due to respondent growing older and wiser between the pre-test and the post-test when this maturation is not of research interest)

4. Testing (familiarity with a test where items and error responses can be remembered at a later testing)

5. Mortality (different kinds of people drop out and the experimental group are composed of different kinds of persons at the post-test)

6. Instrumentation (when the effect might be a change in the measuring instrument between pre-test and post-test and not to the treatment’s differential impact at each time interval.

7. Statistical Regression (movement of extreme scores toward the mean in pre-test/post-test designs and the treatment may have not been the cause)

8. Interactions with selection
   Selection-history
   Selection-maturation
   Selection-instrumentation

9. Ambiguity about the Direction of Causal Inference (not sure if A cause B or B caused A or if A and B interacted in a non-causal way)

10. Diffusion or Imitation of Treatments (the control group gains access to the treatment)

11. Compensatory Equalization of Treatments (may insist that control group receive the same treatment)

12. Compensatory Rivalry of Respondents Receiving Less Desirable Treatments (attempt to reduce or reverse the expected treatment effect)

13. Resentful Demoralization of Respondents Receiving Less Desirable Treatments (effects may be due to reactions rather then the treatment)

External Validity refers to the ability to generalize to particular target populations, settings, times and generalizing across particular target populations, settings, times. There are 3 threats to external validity.

1. Interaction of Selection and Treatment (those who volunteer and decline)

2. Interaction of Setting and Treatment (bias in settings or organizations who participate)

3. Interaction of History and Treatment (circumstances under which study conducted)

Statistical Conclusion Validity (SCV) refers to whether the conclusions about the relationships or difference in the study reflect the real world. SCV has to do with examining the probability or making a Type I or Type II error. There are 7 threats to statistical conclusion validity.

1. Low statistical power
   

2. Violating the assumptions of a statistical test

3. Fishing and the error rate problem
   increases when there are multiple comparisons of mean differences and a certain proportion of the differences will be significant by chance alone.

4. The reliability of measures

5. The reliability of treatment implementation

6. Random irrelevances in the experimental setting

7. Random heterogeneity of respondents

Construct Validity refers to the fit between the conceptual definition and the operational definition of the variables. There are 10 threats to construct validity.

1. Inadequate preoperation definitions

2. Mono-operation bias (only one instrument)

3. Mono-method bias (only one method of recording)

4. Hypothesis guessing (respondents try to guess what the researcher wants to learn)

5. Evaluation apprehension

6. Experimenter expectancies (the experimenter’s bias)

7. Confounding constructs and levels of constructs (manipulation of discrete levels of IV)

8. Interactions with different treatments

9. Interactions of testing and treatments

10. Restricted generalizability across constructs

Non Experimental Designs

I.   Historical  (investigation of events developments or experiences of the past)

II.  Evaluative  (how well a policy, program, or practice works)

1. Formative (monitors the program while in progress)
2. Summative (evaluates the program after it is over)

III.  Ex Post Facto  (systematic inquiry in which the researcher does not have control over IV)

1. Retrospective (the DV is known. Searching for IV)

2. Prospective(know the IV and look for the DV)

3. Prediction( uses retrospective data from one group to make predictions about similar groups data to predict)

4. Descriptive(survey /self report that looks at many cases across a few variables, relates one variable to another, does not demonstrate causality)

   1. Developmental(looks at change over time)

      1. Cross sectional(several groups at different stages all at the same time)

      2. Longitudinal(one group observed over time)
         1. trend(same population with different samples observed at different points in time)
         2. cohort(specific sub populations are examined over time, a type of trend study)
         3. panel(same subjects are examined over time)

   2. Comparative(compare two or more groups on the DV)

   3. Field Study(study of relationships among characteristics of existing groups in day to day life)

IV.  Secondary Analysis  (data collected for one purpose can be retrieved and analyzed for another purpose)

V.   Meta Analysis  (uses data on studies on the same topic looking for patterns or trends)

VI.  Methodological  (development and refinement of research tools)

Qualitative Designs
Qualitative Designs are concerned with human experiences, studied through sustained contact with people in their natural environment. The researcher is attempting to make sense of or interpret phenomenon in terms of the meaning people bring to them using a variety of empirical methods. In general, quantitative methods tests or explores relationships or differences while qualitative explores a human experience.

Types of qualitative studies

Phenomenology

Definition:	learning about the meaning of an experience through dialogue with a person going through the experience
Research Question:	meaning questions eliciting the essence of an experience. For example, what is it like to live with a chronic illness?
Sample:	will consist of people living that experience such as those living with a chronic illness
Data Collection:	audio taped conversations, interviews, ask for subject to write about experience
Data Analysis:	look for central meanings in the data

Grounded Theory

Definition:	aim to uncover the social theory underlying basic human social processes by determining what symbolic meaning things have for groups as they interact with one another.
Research Question:	Process questions that elicit experience over time or change. For example, what factors contribute to nurses care from the patient dying with cancer?
Sample:	people who fit the category of the question
Data collection:	interview, observation of the social situation
Data analysis:	data collection and data analysis occur simultaneously researcher develops hunches about what is happening, then follows up to explore these more deeply Researcher codes data for similarities, cluster ideas into categories Uses literature to help understand the categories

Ethnography

Definition:	develop descriptions of a cultural group as the people under study see it Emic: groups' world view Etic: outsider's view of the culture
Research Question:	descriptive questions of values, beliefs, and practices by looking for patterns of behavior within the social context of the culture - the culture can be based on an ethnic group, a work group, or any subculture of the population
Sample:	cultural group of interest
Data collection:	participant observation, informant interviews, films
Data analysis:	data collected and analyzed simultaneously looks for domains or categories

Once you have read this lesson, you should go to Assignment 1.