Research Methods Assignment 2 Experiment
6namenoufalkharashifcs 681 Research Methodsassignment 2experimental
A researcher wants to investigate whether college students’ knowledge of the negative consequences of the overuse of credit will affect their attitudes about credit use. He plans to recruit one family studies class of 30 students from a local college campus (already enrolled in one class) and measure their attitudes toward credit with an attitude scale. Then, on four successive days, the researcher will teach them four lessons on the consequences of the overuse of credit: wage garnishment, repossession, foreclosure, and bankruptcy. On the fifth day, the researcher will administer an alternate form of the credit attitude scale (one intended to measure the same attitude). Then, he will try to infer whether college students’ knowledge of the negative consequences of the overuse of credit affects their attitudes about credit use.
What is the independent variable and what is the treatment that the researcher will use to manipulate that variable?
The independent variable is the knowledge of the negative consequences of credit overuse, and the treatment involves teaching students four lessons about these consequences (wage garnishment, repossession, foreclosure, bankruptcy).
What is the dependent variable in this study?
The students’ attitudes toward credit, measured by an attitude scale.
What type of research design is this researcher planning to use? Diagram it.
This is an experimental research design, where the independent variable (education about credit consequences) influences the dependent variable (credit attitude). Diagram: The teaching of lessons (independent variable) influences students’ attitude (dependent variable).
State the most likely alternative hypothesis of this researcher.
Students’ knowledge of the negative consequences of credit overuse will significantly affect their attitudes toward credit use.
What are the most important threats to the internal validity of the study? (Give an example of each threat in the language of the research problem.)
- Measurement validity: The credit attitude scale may not accurately measure students’ attitudes.
- Sampling bias: Using a single class of 30 students may not represent the broader student population.
What two threats to internal validity are involved in this researcher’s plan to use an alternate form of the credit attitude scale? What is the trade-off?
- Inconsistent measurement: The alternate forms may differ in clarity or difficulty, affecting the results.
- Timing effects: Variations in administration time may influence students’ responses. The trade-off is between ensuring measurement consistency (using the same scale) versus reducing testing effects by using alternate forms.
Briefly describe the analysis this researcher would use to answer his research question (i.e., what would he compare to what?)
He would compare students’ attitudes before and after the lessons, and also compare scores on the original and alternate attitude scales to assess the impact of knowledge on attitudes. If a group shows significant attitude change after lessons, it suggests the education impacted their perceptions.
Suppose that instead of the design described, the researcher randomly assigned each of the 30 students to two groups. Suppose he did everything the same as before, except a teacher took one group of students on a field trip each of the four “treatment” days during their family studies class to a local historical site.
What type of research design is he now using? Diagram it.
This is a randomized blocked (stratified random) experimental design, where students are divided into groups randomly, and each group receives different treatments (field trip vs. lessons). Diagram: Random assignment to two groups: one receiving lessons, the other going on field trips; then measuring their attitudes afterward.
Does this design now have a classic “placebo” control group?
Yes, one group acts as a control by receiving no additional intervention (field trip versus lessons), serving as a comparison for the treatment effect.
Compare and contrast these two research designs’ internal and external validity.
- Internal validity: The original experiment has higher internal validity because it controls the intervention's influence directly. The stratified design introduces additional variables but allows for better control of confounding factors.
- External validity: Both have limitations; however, the stratified design may offer slightly better generalizability because it includes different treatment types, reflecting real-world variability.
- Control: The original experiment has fewer confounding variables but less ecological validity; the stratified design incorporates more real-life scenarios, enhancing external validity but risking internal validity due to complex confounding factors.
---
A research team wants to study the effect of handbag advertising on women’s choice of handbags. They are concerned about the potential effects of the women having any clue before the ads are shown about the content of the study (particularly the treatment). They randomly assign 100 women (19-22 years old) to four groups. They ask two groups of women to choose a handbag from a group of four handbags before the treatment. Two other groups will not be asked initially about their handbag choices. Of the first two groups, one group will be shown a television program containing 40-second handbag ads, and the other will see the same program without ads. The last two groups will see the program with or without ads. Then, each woman will choose a handbag from a set of four, including the one in the ads. The researchers aim to infer whether advertising influences women’s handbag choices.
What is the substantive independent variable in this study?
Presence or absence of handbag advertisements and the program content.
What additional (methodological) independent variable is planned?
Women’s prior knowledge or awareness of the study content (clue versus no clue).
What is the dependent variable in this study?
The choice of handbag by each woman.
What research design is this research team planning? Diagram it.
A 2x2 factorial randomized experiment: one factor is advertisement presence (yes/no), and the other is prior knowledge (clue/no clue). Diagram: Random assignment into four groups based on these factors, then measuring handbag choice after the intervention.
What are the threats to internal validity of this research design?
- Testing effects: The act of choosing a handbag might be influenced by prior choices or awareness.
- Interaction effects: Prior knowledge might interact with advertisement exposure, confounding results.
What is the third (methodological) null hypothesis in this study?
Women’s prior knowledge about the study’s content does not affect their handbag choices.
The team will analyze their data by comparing handbag choices across the different groups (with/without ads, with/without prior knowledge). They will test for differences in handbag selection attributable to ad exposure and prior information, possibly including interaction effects.
---
A researcher wants to investigate whether college students’ knowledge of the negative consequences of the overuse of credit will affect their attitudes about credit use. He plans to recruit two professors teaching related but distinct topics. Each professor's class has 30 students. The researcher will measure students’ credit attitudes before and after the lessons on credit and Australia, respectively. After instruction, an alternate-form attitude scale will be administered. The goal is to determine if increased knowledge about credit consequences influences attitudes.
What type of research design is this? Diagram it.
A quasi-experimental design, with pretest-posttest measures in non-randomly assigned groups (classes taught by different professors). Diagram: Two groups, pre- and post-measure of attitudes, with lessons on different topics.
What is the most significant threat to internal validity?
Selection bias, since the two classes might differ in characteristics unrelated to the intervention, affecting the outcomes.
What could the researcher do to address this threat?
Standardize the lesson content and ensure both classes cover equivalent material, and if possible, match classes on key demographic variables to enhance comparability.
Compare and contrast this design with the previous random assignment studies and identify which is stronger.
- Compared to a randomized experiment, the quasi-experimental design has lower internal validity because groups are not randomly assigned, risking confounding variables.
- It provides more ecological validity since it reflects real classroom settings.
- Randomized experiments are typically stronger in internal validity due to random assignment, but quasi-experiments are more practical in natural settings.
Among the designs reviewed, randomized controlled experiments generally offer stronger internal validity, but quasi-experimental designs may better reflect real-world conditions.
---
Another study involves a random sample of 100 college students at CSUN. Students’ knowledge of credit consequences is measured via a self-report test; attitudes are measured via a scale. The researcher aims to infer causality.
What type of research design is this? Diagram it.
A simple random design; each student has an equal chance to be selected, with pre- and post-measurements as relevant for causal inference. Diagram: Random sampling, pre-test, intervention (if any), post-test.
What are the advantages over other ex post facto designs?
- Random sampling enhances external validity and representativeness.
- Pre- and post-measurements facilitate causal inference by observing changes over time.
Is history a threat?
No, because all participants experience the same external events during the study.
Is maturation a threat?
No, because the time frame is short, and students are unlikely to undergo significant development affecting results.
Is testing a threat?
Yes, because repeated testing may influence students’ responses (testing familiarity or bias).
Is instrumentation a threat?
No, since the measurement tools remain consistent and calibrated, ensuring reliability.
Is mortality a threat?
No, assuming no participant dropout occurred.
What is the most significant threat, and how to address it?
Testing effects are most significant. To address this, the researcher could include a control group that is not subjected to repeated testing or implement measures to mitigate testing bias, such as using alternative forms or ensuring test anonymity to reduce response bias.
Compare with previous designs:
- The simple random design enhances external validity and causal inference but is vulnerable to testing effects.
- The internal validity is high due to random selection, but careful control of testing procedures is essential.
In conclusion, the design's strength rests on the random selection and standardized measures, with testing effects being a critical threat to manage.
---
References
- Cohen, L., Manion, L., & Morrison, K. (2018). Research Methods in Education (8th ed.). Routledge.
- Flick, U. (2018). An Introduction to Qualitative Research. Sage Publications.
- Gravetter, F. J., & Forzano, L. B. (2018). Research Methods for Behavioral Sciences (6th ed.). Cengage Learning.
- Kirk, J., & Miller, M. L. (1986). Reliability and Validity in Qualitative Research. Sage Publications.
- Leedy, P. D., & Ormrod, J. E. (2018). Practical Research: Planning and Design (12th ed.). Pearson.
- McMillan, J. H., & Schumacher, S. (2014). Research in Education: Evidence-Based Inquiry (7th ed.). Pearson.
- Robson, C. (2011). Real World Research (3rd ed.). Wiley.
- Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs. Houghton Mifflin.
- Yin, R. K. (2018). Case Study Research and Applications: Design and Methods. Sage Publications.
- Cook, T. D., & Campbell, D. T. (1979). Quasi-Experimentation: Design & Analysis Issues for Field Settings. Houghton Mifflin.