The best method — indeed the only fully
compelling method —of establishing causation is
to conduct a carefully designed experiment in
which the effects of possible lurking variables are
controlled. To experiment means to actively
change x and to observe the response in y.
اسلاید 153
Two general criteria of research designs
(Campbell and Stanley (1966):
Internal Validity
External Validity
اسلاید 154
Definition: refers to the extent to which the
changes observed in the DV are caused by
the IV.
Anything that contributes to the control of a
research design contributes to its internal
validity.
اسلاید 155
History: specific events or conditions, other
than the treatment, may occur between the
1st and 2nd measurements of the
participants to produce changes in the DV.
اسلاید 156
Maturation: processes that operate
within the participants simply as a
function of the passage of time.
اسلاید 157
Pre-testing: exposure to a pretest may affect
participants’ performance on a 2nd test,
regardless of the IV.
Measuring instruments: changes in the
measuring instruments, in the scorers, or in
the observers used may produce changes in
the obtained measures.
اسلاید 158
Statistical regression (toward to mean):
Groups having extreme scores on the pretest
(or selected on the basis of extreme scores)
will tend to have scores closer to the mean
on the posttest.
اسلاید 159
Example: In an experiment involving reading
instruction, subjects grouped because of poor pretest
reading scores show considerably greater gain
than do the groups who scored average and high
on the pre-test.
If a sample is selected on the basis of very low or
high scores, it is possible that at least part of the
DV scores are due to chance.
اسلاید 160
Differential selection of participants:
important differences may exist between the
groups before the IV is applied.
Are the groups equivalent at the beginning
of the study?
اسلاید161
Experimental mortality اثر افت آزمودنیها : occurs when
there is a differential loss of respondents from the
comparison groups.
احتمال كم شدن و از بین رفتن آزمودنی ها بخصوص در تحقیقات تداومی، سبب بروز اشكال در
ارزشیابی دقیق و صحیح از روایی داخلی تحقیق است.
Did some participants drop out? Did this affect the results?
Did about the same number of participants make it through
the entire study in both experimental and comparison
groups?
Is a threat for any design with more than one group.
اسلاید 162
Selection-maturation interaction:
كنش متقابل بین گزینش و بلوغ، گزینش و تاریخ و مانند اینها
Some of these internal validity threats may
interact. Frequently arises when volunteers
are compared with nonvolunteers.
If already formed groups are used, one group may profit
more (or less) from the IV (or treatment) because of
maturation, history, or testing factors.
اسلاید 163
Refers to generalizability or representativeness of
the findings.
Question addressed here is:
To what groups, settings, experimental variables, and
measurement variables can these findings be
generalized?
اسلاید 164
Interaction effect of testing: Pre-testing interacts
with the experimental treatment and causes some
effect such that the results will not generalize to an
untested population.
Example: In a physical performance experiment,
the pre-test clues the subjects to respond in a
certain way to the experimental treatment that
would not be the case if there were no pre-test.
اسلاید 165
Reactive effects of experimental
arrangements: اثر روشهای تجربی بر متغیر مستقل
An effect that is due simply to the fact that
subjects know that they are participating in an
experiment and experiencing the novelty of it
—the Hawthorne effect.
اسلاید 166
Multiple-treatment interference: تداخل چند متغیر مستقل
When the same subjects receive two or more
treatments as in a repeated measures design,
there may be a carry over effect between
treatments such the results cannot be
generalized to single treatments.
اسلاید 167
Interaction effects of selection biases and the
experimental treatment: An effect of some
selection factor of intact groups interacting with
the experimental treatment that would not be the
case if the groups were randomly selected.
اسلاید168
Types of designs (Campbell & Stanley, 1963)
1. Pre-experimental (descriptive)
2. True experimental
3. Quasi-experimental
اسلاید 169
Weak experimental designs in terms of
control
No random sampling
Threats to internal and external validity are
significant problems
Many definite weaknesses
Example: One-group pretest/posttest design
One-shot case studies,
اسلاید 171
A. One shot case study (One group posttest-only design) (afteronly
design)
(طرح آزمون نهایی و یك گروه آزمودنی)
For example, a training program is implemented and
participants are given a posttest at the conclusion of the
training.
اسلاید 172
B. One group pre-test-post-test design (before-and after- design)
(طرح آزمون مقدماتی و نهایی و یك گروه آزمودنی)
- One group, not randomly selected nor randomly assigned, is given a
pretest, followed by a treatment/intervention, and finally a posttest.
There is no comparison group.
- دراین طرح متغیر وابسته قبل و بعد ازاجرای متغیر مستقل اندازه گیری می شود.
- زمان استفاده : موقعی كه امكان گزینش تصادفی نباشد
- For example, a classroom teacher gives her students a pretest then
implements an instructional strategy followed by a posttest.
اسلاید 173
C. Static group comparison ( (طرح آزمون نهایی و گروه شاهد، مقایسه گروههای ایستا
One group which has experienced a treatment/intervention (X) is
compared to another group that has not had the intervention. The
groups are not randomly selected nor randomly assigned and are
generally pre-existing groups. There is no pre-observation/pretest.
No way to insure equivalence of groups.
For example, comparison of GRE scores for students who attended a
rural high school versus those who attended an urban high school.
اسلاید 174
Best type of research design because of their
ability to control threats to internal validity
Utilizes random selection of participants and
random assignment to groups
Types:
– Pretest-Post-test control group design
– Post-test only control group design
– Solomon Four-Group design
اسلاید 176
Steps in Pre-Test / Post-test Controlled Exp
■ Random sampling of subjects.
■ Random assignment of subjects to comparison or
experimental groups.
■ Administer pre-test to all subjects in both groups.
■ Ensure both groups have same conditions, except that
experimental group receives the treatment.
■ Administer post-test to all subjects in both groups.
■ Assess change in dependent variable from pre-test to
post-test for each group.
اسلاید 1
Non-randomized, control group, pre-test, post-test
They lack some of the control of true experimental
designs, but are generally considered to be fine
Extensively used in the social sciences
– A useful method for measuring social variables
Types
– Non-equivalent (Nonrandomized) control group pretest-posttest design
طرح آزمون مقدماتی و نهایی با گروه شاهد و بدون استفاده از گزینش تصادفی
– Time series
• One group time series design طرح چند آزمون مقدماتی و نهایی منظم زمانی و یك گروه آزمودنی
• Control group time series design طرح چند آزمون مقدماتی و نهایی منظم زمانی با گروه شاهد
اسلاید 3
The Nonequivalent Groups Design
The most frequently used in social research
Try to select groups that are as similar as
possible to compare the treated one with the
comparison one
e.g. two comparable classrooms or schools
Cannot be sure whether the groups are comparable
The groups may be different prior to the study
Any prior differences between the groups may affect the
outcome of the study
Require a pretest and posttest
اسلاید 6
Example of Quasi-Experimental Design in Geography
Baker and White (2003)
– The Effects of GIS on Students’ Attitudes, Self-efficacy, and
Achievement in Middle School Science Classrooms
– Conducted the Nonequivalent quasi-experimental design
• Two eighth grade teachers, across ten classrooms
• Total 192 eighth grade students participated
• Treatment group: used a Web-based GIS application
• Control group: used paper maps
Treatment
Group
Control
Group
Instructor 1 51 36
Instructor 2 42 63
STEP II: Conceptualizing a research design
Advanced Research Methods
اسلاید 9
Factorial Designs
To compare the effects of two or more independent
variables in the same experiment, cross them.
Factorial designs enable the researcher to determine the
independent influence of each independent variable, and
the interactive influence of the independent variables.
Each combination of independent variables is called a
cell.”
Subjects are randomly assigned to as many groups as there
are cells.
اسلاید 194
What criteria will we use to select a
tradition?
Intent or focus
Audience
Personal training/skills
Personal comfort level with structure
اسلاید 195
– A case can be something relatively concrete such as an organization, a group or an
individual, or something more abstract such as an event, a management decision or
a change program.
– A single case or multiple cases?
– Data collection strategies include direct observation, interviews,
documents, archival records, participant observation, physical
artifacts and audiovisual materials.
اسلاید 197
The purpose of this study was to take a look into education
through the eyes of three teachers who are facing their final year
as professional educators. The overarching goal was to determine
how they have seen children, teachers, administration, policy,
and testing change across the thirty year span of their work as
teachers in Texas’ public schools. Through their comments they
give a considerable amount of insight into the transformation
education has experienced in the last three decades. But
unexpectedly, they reveal as much about our changing society
than they do education itself
اسلاید 198
– Describes the meaning of the lived experience about a
concept or a phenomenon for several individuals.
– Moustakas, 1994, p. 13: to determine what an experience means for
the persons who have had the experience and are able to provide a
comprehensive description of it. From the individual descriptions,
general or universal meanings are derived, in other words, the essences
of structures of the experience.”
اسلاید 199
Phenomonological research
Describes the meaning for several
individuals of their lived experience of a
certain phenomena.
Can center around basic broad questions:
what have you experienced in terms of the
phenomena” and what contexts have
influenced your experience of the
phenomena
درباره این سایت