A study into fruit fly memories reminds us of the importance of using correct statistical methods.
© 2016 A*STAR Institute of Molecular and Cell Biology
Where do fruit flies keep their memories? This question has vexed neuroscientists for decades, with different studies reaching opposing conclusions. A*STAR research now shows this discrepancy does not lie in the raw data but rather in the statistical methods used to interpret it. Applying an alternative statistical treatment finally settles the debate, argue the researchers.1
Neuroscientists measure fruit fly memory using smell. They expose the flies to an odor and use mild electric shocks to condition fruit flies to be repelled by that odor. The flies are then sent down a T-shaped maze, one arm of which contains the odor, to test whether the flies remember and avoid the smell.
By testing flies in which particular genes have been knocked out, researchers have pinpointed as crucial for memory, a three-lobed brain structure called the mushroom body. For short-term memory, are all three lobes involved, or is just a single lobe required? That is where researchers disagree.
Adam Claridge-Chang from the A*STAR Institute of Molecular and Cell Biology, and his team have now shown that the reason for the disagreement is a statistical method called significance testing.
If the wild-type fruit fly has a memory of 100 per cent, he says, the knockout fly might fall to 20 per cent memory. “When you start to add genes back into different lobes, you start to get progressive values of rescue back to 100 per cent.”
On to this graduated series of data points, significance testing essentially draws an arbitrary line. It turns a continuum of data into a black or white, yes or no question, he says. “Significance testing creates a false dichotomy.”
So Claridge-Chang gathered all the raw data from earlier studies, and applied a statistical method known as a meta-analysis. Pioneered in clinical research, meta-analysis is “basically, a fancy way of averaging,” Claridge-Chang says. The meta-analysis showed all three lobes played a role, contradicting the mainstream view.
In fact, he found a direct correlation between the importance of each lobe for short-term memory with the number of neuronal cells each lobe contains. “The strength of correlation shows the data integrity is very good in the fly field,” Claridge-Chang says.
This is welcome news in an age where the reproducibility of many fields of research has been called into question. “It’s a great example of a situation where scientists have done everything right, but then used the wrong statistical method.” Claridge-Chang says he hopes scientists in other fields will take note and stop using significance tests2.
The A*STAR-affiliated researchers contributing to this research are from the Institute of Molecular and Cell Biology.
