A fairly large part of my statistics lecture today was spent, directly and indirectly, on just how bad postal surveys are. Earnest letters in the mail imploring the recipient to take some time out of their day to fill in some boxes and then post them back (free of charge in an already-addressed envelope provided) get poor enough response rates when they decide national referendums or local elections—surveys from random organisations addressed “To the Householder” can only be expected to perform worse.
Stopping as you do in the toilet after class I was then confronted with a very common sight in university campus bathrooms: an A4 poster searching for volunteers for a student’s psychology project, or in this case criminology. The subject turned out to be experiences of pornography, which caused me to raise my eybrows a little bit, although not quite as much as the list of requirements, which so far as I recall were:
- To identify as male or female
- To have an opinion on whether pornography has positively or negatively affected them
- To be heterosexual
I’m not sure if they meant the first to be “binary people only” (because they don’t think they will have enough enbys to properly analyse) or “everyone including both men and women but I think that this is the progressive way to say it” (in which case, more eyebrows, and that’s all I’m saying). The last one though seemed a bit strange: do you really want to exclude everyone not strictly heterosexual? The remainder seems important, especially in the generation likely to show up to a study advertised in a uni loo.
Though that being said, after other discussions in stats class lately the thing that really struck out at me was the middle one. Middle things are important, particularly people. Explicitly requesting people to show up who have “an opinion” is not a great way to get representation of people in between, and those people are, again, important to getting a proper picture to what people in general feel and have experienced. If this study winds up with only people who are strongly for or against its topic then it may have great difficulty figuring out why those people would have come to those positions.
So while I wish the student all the best in their endeavours you may have to excuse me if I am a little sceptical that their efforts—which will doubtless be quite significant in planning and carrying out this study—will actually produce anything reliable.