Experimental Psychology -- Summary

Reasoning in experimentation

Inductive and Deductive Reasoning

Data --> Induction --> Theory
Theory --> Deduction --> Prediction/confirmation

Logical Reasoning

If P, then Q
- Appropriate Logical deductions
  - P --> Q
    - Affirm the Antecedent
  - Not Q --> Not P
    - Deny the Consequent
- Incorrect logical deductions
  - Not P -> Not Q
    - deny the antecedent
  - Q --> P
    - affirming the consequent
Popper and the Testability of Theory
- the hallmark of science is testability of theory
- non-testable theories are not science
Platt and Strong Inference
- need to pit theories against each other
- theory testing as a series of pruning alternatives

Examples of scientific inference

Theory development and testing

Theories as organizations of observables

Data = Model + Error
Theory as simplification of data

Constructs, Latent Variables, and Observed Variables

Observables
- Multiple levels of description and abstraction of observables
- Multiple levels of inference about observables
Latent Variables as abstractions of observables
- latent variables as the common theme of a set of observables
- central tendency across time, space, people, situations
Constructs as organizations of latent variables and observed variables
- observed measures
- manipulated measures

Theory as metaphors and analogies

Physics
Business competition - business and evolutionary theory
Learning, memory and cognitive psychology - the mind as a computer
psychodynamc theory and the hydraulics of motivation
motivation theory and the dynamics of action

Models and theory

Formal models
- mathematical models
- dynamic models --simulations and virtual reality
Conceptual models
- as guides to new research
- as ways of telling a story
  - organizational devices
  - shared set of assumptions

Examples of theory

Biological Psychology
Clinical Psychology
Cognitive psychology
Personality theory
Social Psychology

Testing theory -- the role of design

measurement or manipulations of constructs
eliminating or reducing threats to validity of construct

Issues in measurement

What do numbers ³mean²

does order at observed level mean order at latent level?
do equal differences at observed level mean equal differences at latent level?

Latent Variables and Measured Variables

Attributes as Latent Variables
Observed Scales as transformations of Latent Variables
- observed scores are indicators of latent variables
- what is the degree of association
- what is the shape of the association
Scales and levels of measurement
- scales defined in terms of their sensitivity to transformations
  - ordinal transformations preserve order
  - partial orders have some distance information
  - interval scales preserve relative distance
  - ratio scales preserve distance from 0
- the meaning of transformations
  - statistics based upon observed data
  - inferences about latent variables
  - transformations for parsimony

Example of measurement-Theory of Signal Detection

Observed score
- the number of signals correctly detected
- the number of non-signals incorrectly identified
Latent scores
- Sensitivity to signal (d¹)
  - ability to detect signals in the presence of noise
  - d¹ = Signal + Noise distribution versus Noise distribution
- Bias of responding (ß)
  - willingness to make false alarms

Sources of data

non-intrusive
- archival
- observational
intrusive
- self reports
- peer reports
- experimental
  - time/accuracy
- psychophysiological

Statistical tests and the interpretation of effects

Descriptive Statistics

Distributions
- shape
  - normality
  - skewness
Central tendency
- mode
- median
- mean
Variability
- range
- variance
Correlation and regression
- relationship between X and Y
- regression is slope of the best fit line predicting Y from X
- correlation is standardized slope (Y from X and X from Y)

Inferential Statistics

Hypothesis of null difference
- samples from same population will differ
- Are observed differences likely to have occurred by chance?
Statistical tests -- overview
- how likely is it to observe a particular parameter value
- parameter/standard error of parameter
one Independent Variable -- One Dependent Variable
- 2 levels of the IV
  - t-test of group differences
  - F-test of group differences
- multiple levels of the IV
  - F-test of group differences
  - test of trends (linear, quadratic, etc)
Multiple Independent Variables -- One Dependent Variable
- Analysis of variance with muliple factors
  - main effects of IV 1 on DV
  - main effects of IV 2 on DV
  - interaction effects of IV 1 and IV 2 on the DV
    - does the relationship of IV1 with the DV depend upon IV2?

Issues in Design

Questions to ask about any research

Identification of the variables
- what are the theoretical variables
- what are the observed operationalizations of the theoretical concepts
- what are the independent variables
- what are the dependent variables
Identification of possible threats to validity

Types of validity

statistical
- can we detect real differences and reject non-differences?
- multiple tests and the increase of type I errors
- power and the reduction of type II errors
  - power varies as effect size and sample size
- estimations of effect sizes rather than statistical significance
internal
- subject effects
- experimenter effects
- subject x experimenter effects
construct
- multiple measures
- multiple methods
- issues in measurement
- paradigm limitations
external
- replicability across subjects, time, materials

Experimental-Correlational and Quasi-Experimental designs

Experimental
- Manipulation of (at least) one variable -- Independent Variable
- Effect on (at least) one other variable -- Dependent Variable
Correlational
- Observation of the relationship between two variables
- Inability to determine causality
Quasi-Experimental -- Field studies
- direct relevance
- difficult to have appropriate controls

Standard Experimental Designs and the control of extraneous influences

Within subjects
- controls for subject variability
- confounds condition with practice and order effects
  - order or practice
    - fatigue
    - learning
    - differential transfer
  - counterbalancing as a control
    - all orders
    - some orders
    - two orders
  - counterbalancing controls main effects of order, not interactions
Between subjects
- eliminates order and practice effects
- confounds subject characteristics with condition
  - subject characteristics as a confound
    - ability
    - motivation
  - randomization as a control
    - matching
    - randomization
    - block randomization
Mixed designs -- Within and Between
- multiple conditions within subject
- counterbalancing across subjects

Alternatives to Experimentation and threats to validity

Internal/External validity
- Internal validity assures that observed effects are associated with manipulations of independent variables
- external validity refers to generalizability of results
- is there a tradeoff between internal/external validity?
  - rigor vs. importance
  - lab vs. field
  - theoretical versus applied
³True² Experiments, ³Quasi-Experiments² and case studies
- ³true² experiments
  - randomized assignment of subjects to conditions
  - eliminates threats to internal validity
    - observed effects are causally linked to manipulation of independent variables
    - requires basic ³experimental hygiene²
    - controlling for possible confounds
  - randomization does not guarantee equivalency of groups, but introduces no systematic bias.
- ³Quasi² experiments resemble true experiments but lack random assignment
  - analogous to correlational studies of individual differences
  - need to consider multiple threats to the internal validity of the study
- Case studies
  - examples of HM, Phineas Gage, NA
Quasi experimental designs and threats to validity
Field studies and applied research
- Human factors and simple and complex systems
- Engineering psychology
- Environmental psychology
- Evaluation research
- ³Nothing so useful as a good theory²

Writing a scientific paper

Purpose

to add to scientific knowledge
- as it relates to prior knowledge
- as it provides new knowledge
to convey information clearly and concisely
- clarity of presentation
- conciseness of presentation

Types of articles and representative journals

Theory development
- Psychological Review
Reviews of prior findings
- Psychological Bulletin
- American Psychologist
³Archival² data
- studies that are worth publishing
- broad band
  - Journal of Experimental Psychology: General
- narrow field
  - Journal of Personality and Social Psychology
  - Journal of Abnormal Psychology
  - Cognitive Psychology
  - Developmental Psychology
Methodology
- Psychological Bulletin
- Psychometrika
- Psychological Assessment

Conventional Styles

Science Magazine + Psychological Science
American Psychological Association journals-- APA style
- Title page
- Abstract
- Introduction
- Method
  - Participants
  - Apparatus
  - Procedure
- Results
- Discussion and Conclusion
- References
- Tables
- Figure Captions
- Figures