Friday, May 27, 2016

Childhood Abuse, Reliability, and Measurement Error: More from Sara's Week in Psychological Science

This morning, I attended a great session on methodological issues in studying trauma. One of the presentations was about studying childhood abuse. Many measures of childhood abuse are done retrospectively, among college students/adults, asking about past experiences. Researchers have noted a variety of issues with this approach, including the issue that reports of childhood abuse seem to vary over time - that is, a person may report that they were abused as a child at one timepoint, but not at another.

This leads some to conclude that reports of childhood abuse may be influenced by current levels of distress - people may misremember or misreport childhood abuse depending on how distressed they are feeling as adults. We know that current experiences can color past experiences, which we see in any kind of memory research; memory is highly malleable. However, any time we measure something in people, we also have to worry about measurement error. Poorly worded questions, respondent fatigue, and other factors may affect how well the measure "works."

The great thing about structural equation modeling (SEM) is that it separates measurement error out, so we can get a more pure read of how much particular constructs relate to each other. Some people have used this as mark against SEM, that it shows a "perfect world" relationship that we would rarely see in practice. But in many cases, SEM is a great technique to use as a way to answer questions about reliability due to measurement error versus systematic sources of variation.

The researchers measured past experiences of childhood abuse at the same time as they measured symptoms of distress, using a post-traumatic stress disorder (PTSD) checklist. Two weeks later, they measured these two variables again. They built a model where time 1 PTSD predicted time 2 PTSD, time 1 reports of childhood abuse predicted time 2 reports, and time 2 PTSD predicted time 2 reports. What they found was that time 2 PTSD predicted less than 2% of the variance in time 2 reports of childhood abuse. Time 1 abuse reports was highly predictive of time 2 abuse reports, and once measurement error was factored out, they found reliability in abuse reports above 0.80 (which in measurement world is considered excellent reliability).

What this means is that, when it looks like people are saying different things at different times, measurement error is a much more likely culprit than how a person is currently feeling. Obviously, the study was done over a short timeline (2 weeks), so results may be different if that time period is longer. This was also a college sample, and reports of abuse, especially physical and sexual abuse, were low. But this study gives some guidance for studying reports of childhood abuse in other samples, and highlights a time when separating measurement error from systematic variation (i.e., actual differences in reports of childhood abuse) is the optimal approach.

No comments:

Post a Comment