04. Psychologist’s Toolbox: Sampling, Measurement, Reliability, & Validity

Activity: Identifying Potential Threats to Internal Validity Using MRS SMITH

To help students remember potential sources of threats to internal validity, I use the acronym “MRS SMITH.” This resource provides a handout describing these threats as well as an activity where students need to identify these possible threats. Click here for a link to the activity.

Activity: Identifying Potential Threats to Internal Validity Using MRS SMITH Read More »

Article: Self-Report Vs. Behavioral Logs in the Measurement of Digital Media Use

A meta-analysis looking at the correlation (or lack thereof) of self-report measures of media use and measures that track actual behavior. This would be useful to add to a discussion of types of measurement (contrasting the strengths and weaknesses of self-report and behavioral measures. Parry, D.A., Davidson, B.I., Sewall, C.J.R. et al. (2021). A systematic review and

Article: Self-Report Vs. Behavioral Logs in the Measurement of Digital Media Use Read More »

Article: Misunderstandings about Short Scales and Ways to Address Them

Measures that use fewer than 10 items are considered “short” and pose some challenges, including concerns about validity. This special issue addresses several misconceptions about these scales and offers ways to counteract potential problems. Ziegler, M., Kemper, C. J., & Kruyen, P. (2014). Short scales – Five misunderstandings and ways to overcome them [Editorial]. Journal of

Article: Misunderstandings about Short Scales and Ways to Address Them Read More »

NOBA: Research Designs

An article by Christie Napa Scollon that outlines the basics of research designs in psychology, including information about experimental vs. correlational design. This reading would be a good introduction to the topic, or a good review for students who previously took a methods course. You can access the full chapter here.

NOBA: Research Designs Read More »

NOBA: Conducting Research in the Real World

An article by Matthias Mehl that advocates for taking our research out of the lab and exploring phenomena in the real world. The discussion touches on topics like internal/external validity, ecological validity, sampling, as well as several specific/specialized methods related to capturing daily information from participants. The chapter also includes information about conducting research online

NOBA: Conducting Research in the Real World Read More »

Examples of Sampling Bias and Demand Characteristics through a Daily Show Clip

A clip (5:30) from the The Daily Show with Jon Stewart that demonstrates how sampling bias and demand characteristics can lead to misleading conclusions. “Poll Bearers: Cable news polls might not reflect public opinion so much as the ability of viewers to repeat the ideas they just heard.” View the episode here.

Examples of Sampling Bias and Demand Characteristics through a Daily Show Clip Read More »

Video: Random Sampling

A brief (2:25) video from Pew Research Center in which, “The first video in Pew Research Center’s Methods 101 series helps explain random sampling – a concept that lies at the heart of all probability-based survey research – and why it’s important.”

Video: Random Sampling Read More »

Practice: Measuring Dependent Variables: Types and Scales of Measurement

This is a brief in-class practice activity (created by Raechel Soicher) to help students review different types of measurement for dependent variables as well as identify scales of measurement. Raechel recommends teaching about the different ways to measure a dependent variable before giving students the opportunity to review using this brief worksheet. She provides definitions

Practice: Measuring Dependent Variables: Types and Scales of Measurement Read More »

Practice: Having Fun with Operational Definitions

One of the challenges instructors face when discussing conceptual and operational definitions of variables is helping students to understand that the operational definition of a variable involves measuring the variable in a concrete, quantitative manner. The practice activity asks students to identify the relevant variables of interest and then suggest ways that these variables can

Practice: Having Fun with Operational Definitions Read More »

Lecture Slides: Measuring Dependent Variables

This  lecture introduces students to several different ways of measuring   behavior as the dependent variable.  Several types of behavioral measures are shown, including: behavioral trace, behavioral observation, and behavioral choice. The goal of this lecture is to get students to see other types of measuring DVs, aside from self-report. This starts with a quick overview

Lecture Slides: Measuring Dependent Variables Read More »

Lecture Slides: Manipulating Independent Variables

This  lecture introduces students to several different ways of manipulating independent variables. Some techniques covered include: mock Facebook profiles, priming, having participants count money, and viewing cute pictures. The goal of this lecture is to get students to see a variety of ways that researchers can manipulate IVs. Each example provides the citation for the

Lecture Slides: Manipulating Independent Variables Read More »

Design & Data: How Do Parents Respond to Their Child’s Public Tantrums? (Observational Research/Descriptive Statistics)

Key Topics and Links to Files Data Analysis Guide The Many Forms of Discipline in Parents’ Bag of Tricks Analyses Included: Descriptive Statistics (Frequencies; Central Tendency); Inter-observer Reliability (Cohen’s Kappa); Inter-observer Reliability (Pearson’s r); Creating a Mean; Creating a Median Split; Selecting Cases Dataset Syntax Output BONUS: Dyads at Diners (How often and how much

Design & Data: How Do Parents Respond to Their Child’s Public Tantrums? (Observational Research/Descriptive Statistics) Read More »

Open Stats Lab: Identifying Questionnable Research Practices

Students read “Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling“ Student respond to a series of questions asking them to define questionable research practices, how they relate to Type 1 Error, consequences, and remedies. A lab summary includes: learning objectives, background, article citation, and the response questions. You can follow Open

Open Stats Lab: Identifying Questionnable Research Practices Read More »

Open Stats Lab: Additional Activities

Activities focus on the following topics: (RELIABILITY ANALYSIS) Do People Feel Immoral After Behaving Inauthentically? Here students get to compute new variable, calculate descriptive statistics, find Cronbach’s alpha, and create a table. (CHI-SQUARE TEST OF INDEPENDENCE) Why do People Fail to Follow Through on Their Good Intentions? Here students get to conduct a t-test for

Open Stats Lab: Additional Activities Read More »

Lecture Slides: Samples Representing the Population with the Use of M&Ms

This PowerPoint leads students through an activity that stresses the need to have a relatively large sample when trying to accurately represent the population.Instructors will need to supply students with fun size bags of plain M&Ms®. Students first estimate the population frequency of each color of candy using their small sample (fun size bag). Then

Lecture Slides: Samples Representing the Population with the Use of M&Ms Read More »

Online Activity: Simulation of Sampling Distributions

This Java applet (created by Dr. David Lane) estimates and plots the sampling distributions of various statistics based upon the user’s specified population distribution, sample size, and statistic. The simulation and its associated questions can be used as either homework assignment or as an in-class activity for those classes with access to computers with Internet

Online Activity: Simulation of Sampling Distributions Read More »

Activity: Threats to Internal Validity: Smarties vs. Dum Dums

This activity helps students remember potential sources of threats to internal validity, through an in-class activity where students engage in an “experiment” with several intentional threats to internal validity. Students identify the threats and get candy. Easily one of our students favorite activities. Please click here for the file.

Activity: Threats to Internal Validity: Smarties vs. Dum Dums Read More »