I love it when negative results and outliers win

Recently, we at the University of Oregon were treated to a terrific seminar from Tom Reimchen. His has spent a career looking at the evolutionary traits of tiny stickleback fish in lakes on Haida Gwaii, off the coast of British Columbia. His results were sweeping and impressive, as results often are after decades of work – he has found over and over that a few morphological aspects of stickleback in lakes near each other varied based on their small lake environment and resulting selective pressures. Basically, long spines and tough protective plates protect from attack, but at a fitness (speed and evasion) cost. Murkier lakes held sneak attacks from predators and thus tough protection was key; clear water allows stickleback to see attacks coming, so speed and evasion win the day. A beautiful story.

But studying many different lakes with many different chemistries, conditions and myriad other covariates makes for nuanced conclusions – most skeptical scientists hearing these results immediately want to know if there is anything more simple that might explain. Which is why Reimchen really struck me with an answer to a good question. An audience member was curious about fitness costs of building tough plates, and if these structures appeared even when resources like Ca and P were limiting – these are necessary to build bony plates. Over the years Reimchen and colleagues of course collected many different environmental parameters, including pH and Ca concentration. Almost all of these lakes were strongly Ca-deficient, meaning that it would be really hard to build bony plates, but if the water was murky and if the right predators were present, the stickleback always seemed to be well protected by these chemically-expensive structures. Closer to the ocean, however, splash and surge from salty water makes for high Ca conditions, and the researchers found that these lakes near the ocean were no more likely to hold heavily-plated stickleback than those inland, and that visibility and predation explained most of the variability.

To a researcher trying to find out if predation is driving prey morphology, this negative result (bony structures are not explained by resource availability) likely comes as a relief – it supports your alternative hypothesis. But stoichiometry wants to explain so much in the natural world that it is really cool to see results that defy such a unifying principle.

Maybe Josh Schimel’s book, Writing Science is the reason I latched onto this result. I absolutely love this book and it is changing the way I write in a big way – but also the way I think about science. Schimel goes to great length early in the book to argue for paying close attention to outliers and ugly data points, since outliers are often “…more novel, exciting, and important science.” This chapter hit me hard – I’m a data analyst for the most part and I almost always try to ignore outliers. It comes easy for me to think that samples were mishandled, poorly sequenced, or in other ways discountable. Outliers tend to look ugly, and negative results might mean that my data are not clean enough to see the relationship I assume is there. But Schimel’s book, along with Reimchen’s talk, have helped me to take a step back when data are not explained cleanly – to look even deeper when a few samples are uncooperative. As a graduate student studying statistics, I committed to memory the accepted rules for ignoring outliers in a dataset. The rules generally revolve around an assumption that points represent replicates from the same population; discount them if you have reason to believe they don’t. Now I am rethinking this approach, realizing that instead of discounting, I should more thoroughly investigate why a point is so far away from expectation.