If you are doing research in the social sciences, or even if you’re not but like a bit of geek-based Indie music then this post is for you. If neither of those apply then I will forgive you for not reading the rest of this post (though it is worth checking out the very bad but catchy song at the end!), but in an effort to maintain my love of blogging along with my Masters Degree I am going to try and post a bit more psychology related stuff. Unfortunately for all my current module is statistics and research methods, so that is today’s topic.
A quick summary for those who don’t know, when we perform research in psychology we use a method called hypothesis testing, where we set a null and alternate hypotheses. The alternative hypothesis is always our prediction that there will be an effect of what we are measuring. The null hypothesis is always that there is no effect, and basically we are testing the assumption that there is no effect or difference in what we are measuring. Let’s give an example; I think doing crochet is more relaxing than watching football. I could design an experiment where I had a group of people watch football for an hour and a group of people do crochet for an hour, and then I could give them a test that measures relaxation and see if there is a difference. Now, this is an experiment at it’s simplest level, and there are many potential problems with it, feel free to comment on what they are, think of it as a crash course in Research Methods! Anyway, for that experiment our null hypothesis is “there is no difference in the level of relaxation attained by watching football or by doing crochet”, and the alternative would be “doing crochet makes you more relaxed than watching football”.
The reason we have the null is that we can never prove anything with statistics, we can only reject the null which supports our alternative hypothesis. We calculate the probability of our observation occurring if the null hypothesis is true, that is, what are the chances of getting this effect if there is really no difference in the things we are measure. This method is actually often criticised, as in real life if we take two measures of anything there is almost never no difference between them. We still do it this way though, but it highlights the importance in understanding the mechanisms underlying statistics and not just blindly accepting numbers that a computer spits out.
So, in psychology we determine the probability of obtaining a result at least as big as the one we obtained if the null hypothesis is true and use that to decide if we have a significant effect of not. We use the figure p=.05 as our cut off. Basically, if the statistics say there is a less than 5% chance of getting our observation if there is no effect we are comfortable enough to say “yep, guys, we have a significant effect here”. So if in our experiment above our p value for the differences between our groups (as calculated by a delightful programme called SPSS) is .02 it is basically saying “Look, if there really was no difference between the two groups you’ve got 2% chance of getting this result; that’s pretty low so probably there is a significant difference – reject the null hypothesis, reject I say!”. Anything up to 5% and we are comfortable that the difference is significant (yes, that figure is pretty arbitrary, and yes, there are many things wrong with it. What is the p value was 0.056? Well, it would be classed as non-significant for most academic journals).
However, even with our 5% p value there is still obviously 5% chance we could get our observation that makes it look like crochet is more relaxing than football, when really it isn’t. Maybe we just managed to find for our study the few people in the world who find crochet really relaxing, but the majority don’t. This idea that we might falsely reject the null hypothesis is called a Type I error. Or course, we may have got results that exceed our hallowed %5 probability thus causing us to accept the null hypothesis as true, there is no difference between the relaxing properties of crochet and football, when in fact there is a difference, we just didn’t pick it up in our study (maybe we didn’t look at enough people, or we didn’t make them do enough crochet…). This failure to reject the null hypothesis is known as a Type II error.
Now, why are these things important? Well mainly because I have an exam on such notions in a couple of weeks…but really because this is the basis of all social science research. Why am I writing a blog post on this? Well, those of you who have read and understood this probably already know it anyway from a basic research methods course. If you didn’t already know it then you can’t possibly have any reason to need to know it so I am impressed you persevered this far!m(Or maybe you are a student who does need to know but hasn’t understood it from your course, nor discovered a decent stats book like Discovering Statistics Using SPSS -seriously, this is a cracking stats book). Anyway, typing this all up has been great revision for me, which is mainly why I did it, and, well it’s my blog and I can write what I like! But what I really wanted to share was an “oh so bad it’s really good” song which someone, in the crusade to remember which way round Type I and Type II errors are, has written. Trying to remember which is why is a real pain, even after all these years (I first learned this stuff at A level) and is clearly an issue that plagues students the world over. For those who can’t make out the lyrics they are as follows:
If the null is zero
And it’s really zero
But you think it’s bull
And reject the null
If the null is zero
And it’s really not
And you accept the null
That’s off the spot
Isn’t it ace? I’m going to be singing this in my exam next month!