Wednesday, October 26, 2016

Polling the Polls: Meta-Analysis in Action

I blogged recently about a new course I'm taking on the topic of meta-analysis, which is a set of techniques for aggregating results from multiple studies. I realized this morning that you've probably encountered meta-analysis recently, in the form of election polls - specifically, sites that aggregate data from multiple polls. A couple examples are Talking Points Memo's Poll Tracker and FiveThirtyEight's Election Forecast.

Since we can't poll everyone in the population, we use samples as a stand-in. (Terminology side note: When we poll a sample, we call it a survey; when we poll an entire population, we call it a census.) There are techniques to use in surveying, to ensure the sample is representative, but of course, weird things can happen. We may get bias in who responds, or bias because of how a question was worded, or any number of issues. These polls of polls are great examples of the usefulness of meta-analysis. While an individual study, even when done very well, has limitations, aggregating across studies and weighting each study's contribution by its sample size allows us to uncover relationships that may be too small to detect in a single group. And as sample sizes get bigger, we can get a closer and closer estimate to the true population value. In the case of presidential elections, that population value is of course the proportion of voters who will vote a particular way.

See? You've been watching meta-analysis in action all along!

No comments:

Post a Comment