Week 12: Quantitative Research Methods - Surveys and Quantitative Data Analysis

Slide 1
Location: CBC Campus - Tuesday T-336 & SWL-220 Time: Tuesdays and Thursdays from 5:30-8:15 Week 12: 11/04/19 — 11/10/19 Reading Assignment: DeCarlo (2018) chapter 11 and 12 Topic and Content Area: Quantitative Research Methods Assignments Due: Assignment 02: reading quiz for chapters 11 and 12 are due at 5:30 PM prior to class via My Heritage Other Important Information: N/A

Location: CBC Campus - Tuesday T-336 & SWL-220
Time: Tuesdays and Thursdays from 5:30-8:15
Week 12: 11/04/19 — 11/10/19
Reading Assignment: DeCarlo (2018) chapter 11 and 12
Topic and Content Area: Quantitative Research Methods
Assignments Due: Assignment 02: reading quiz for chapters 11 and 12 are due at 5:30 PM prior to class via My Heritage
Other Important Information: N/A

Slide 2
Quantitative Designs Experiment Survey Program evaluation* Secondary data* Qualitative Designs Interview Focus group

Quantitative Designs

  • Experiment
  • Survey
  • Program evaluation*
  • Secondary data*

Qualitative Designs

  • Interview
  • Focus group
Slide 3
Strengths Cost-effectiveness Generalizability Reliability Versatility Longitudinal designs are great Limitations Inflexibility Lack of depth For cross-sectional designs, difficulty with time order Phone, mail, internet, and in-person surveys all have issues

Strengths

  • Cost-effectiveness
  • Generalizability
  • Reliability
  • Versatility
  • Longitudinal designs are great

Limitations

  • Inflexibility
  • Lack of depth
  • For cross-sectional designs, difficulty with time order
  • Phone, mail, internet, and in-person surveys all have issues
Slide 4
Questions are based on operational definitions (But also include other variables and characteristics) Concise, easy to understand Address the “most knowledgeable people” about a topicThink back to sampling Clear wording (Double negatives, double-barreled questions and answers, jargon, slang) Neutral wording (Leading language, social desirability) Pretesting is key
  • Questions are based on operational definitions (But also include other variables and characteristics)
  • Concise, easy to understand
  • Address the “most knowledgeable people” about a topicThink back to sampling
  • Clear wording (Double negatives, double-barreled questions and answers, jargon, slang)
  • Neutral wording (Leading language, social desirability)
  • Pretesting is key
Slide 5
Closed-ended questions (Mutually exclusive and exhaustive response options) Fence-sitters and floaters Filter questions, Matrix questions Group your questions by theme Ordering is important, though tricky Think about the time needed to complete the questionnaire Look professional
  • Closed-ended questions (Mutually exclusive and exhaustive response options)
  • Fence-sitters and floaters
  • Filter questions, Matrix questions
  • Group your questions by theme
  • Ordering is important, though tricky
  • Think about the time needed to complete the questionnaire
  • Look professional
Slide 6
Show examples of… Questionnaire Data Collection Report

Show examples of…

  • Questionnaire
  • Data Collection
  • Report
Slide 7
Create a 5-10 question survey which addresses the topic of “study skills,” though the specific research question is up to you… Include questions and answers Follow best practices listed in the book Don’t ask anything sensitive

Create a 5-10 question survey which addresses the topic of “study skills,” though the specific research question is up to you…

  • Include questions and answers
  • Follow best practices listed in the book
  • Don’t ask anything sensitive
Slide 8
Gather data from all of the class

Gather data from all of the class

Slide 9
What questions worked well? Didn’t work so well? What research questions could we answer? What would univariate analysis look like here? What bivariate relationships could we explore? Multivariate? What would you do differently, if you could? Setting? Format?
  • What questions worked well? Didn’t work so well?
  • What research questions could we answer?
  • What would univariate analysis look like here?
  • What bivariate relationships could we explore? Multivariate?
  • What would you do differently, if you could?
  • Setting? Format?
Slide 10
Classic Experimental Design Experimental and control groups Random assignment Pretest and posttest Variations Posttest only (testing effects) Solomon four group design Using a comparison group

Classic Experimental Design

  • Experimental and control groups
  • Random assignment
  • Pretest and posttest

Variations

  • Posttest only (testing effects)
  • Solomon four group design
  • Using a comparison group
Slide 11
Nonequivalent comparison group design (true experiment, without random assignment) Natural experiments Ex post facto control group Time series Matching (Individual and Aggregate)
  • Nonequivalent comparison group design (true experiment, without random assignment)
  • Natural experiments
  • Ex post facto control group
  • Time series
  • Matching (Individual and Aggregate)
Slide 12
Static group comparison One-shot case study One-group, pre/posttest Severe limitations
  • Static group comparison
  • One-shot case study
  • One-group, pre/posttest

Severe limitations

Slide 13
Internal vs. external validity Replication Threats to validity Noncomparable groups Selection bias Placebo effect Researcher effects
  • Internal vs. external validity
  • Replication
  • Threats to validity
    • Noncomparable groups
    • Selection bias
    • Placebo effect
    • Researcher effects
Slide 14
Response rates and nonresponse bias Importance of creating a codebook
  • Response rates and nonresponse bias
  • Importance of creating a codebook
Slide 15
Univariate analysis Measures of central tendency Frequencies Bivariate analysis Chi-square, t-test, ANOVA, and correlation Multivariate analysis Regression, MANOVA
  • Univariate analysis
    • Measures of central tendency
    • Frequencies
  • Bivariate analysis
    • Chi-square, t-test, ANOVA, and correlation
  • Multivariate analysis
    • Regression, MANOVA