Dear Statistics,
I apologize for not having taken you seriously until this year. I said that you aren’t Mathematics; that’s true, but not in the (derisive) way I meant it. You aren’t Math in the same way that Physics isn’t Math—you sure use a whole lot of Math to do some very interesting and important things. Really, you are the science of data;1 you enable us to learn and make decisions in a noisy, uncertain world.
I changed my mind when my eyes were opened to all the useful things you can do, of which there is a whole enormous, gigantic, humongous, impressive list.2 In my ignorance and arrogance, I had no idea. Fortunately, I had the honor of taking classes with some incredible professors in graduate school that opened my eyes to what your buddy Probability had in store; once I stopped hating on Probability, it was only a matter of time until I gave you a second look.
By and large, though, one of the biggest things that helped me appreciate you better is my developing programming abilities. Rather than working by hand, I can now use things like R to, say, flip a billion unbiased virtual coins in the blink of an eye. Let’s be honest, you involve a lot of computation; what better tool could I use than a computer?
That last thought reminds me: arguably one of the reasons I never took you seriously is the poor way in which you’re taught in schools. I think you are subject to the problems mentioned in Lockhart’s Lament even more than Math is. A lot of teachers seem to think you consist solely of blinding applying formulas without understanding; as a Mathematician who loves learning and understanding things, I too lament this deplorable idea. As Richard Hamming once said, the purpose of computing is insight, not numbers. I now know it makes sense to use squared distance from the mean for variance instead of absolute distance, because doing calculus with absolute value is a pain. I now know that your best friend Probability is actually Analysis in disguise. Of course, there isn’t a need to know graduate level Analysis to successfully apply you, but I sure feel better knowing why I’m doing what I’m doing. There has to be a happy medium somewhere closer to developing understanding than what passes for Math & Stats education in most3 schools nowadays.
To sum up, Statistics: I apologize. Let’s be friends.
Best,
Graham
-
I’m avoiding calling you “Data Science,” since that’s too trendy, but you really are. ↩︎
-
I claim that if Machine Learning isn’t a proper sub-discipline of Statistics, it at least uses so much statistical knowledge that it might as well be counted as a win. ↩︎
-
Most American schools, perhaps? My perspective is limited to my own country. ↩︎