Truth or Truthiness by Howard Wainer

Truth or Truthiness by Howard Wainer ISBN-13: 9781107130579 Hardcover: 232 pages Publisher: Cambridge University Press Released: Dec. 1, 2015

Source: Review copy from the publisher.

Book Description, Modified from Back Cover:
Happy kids do better on tests. Kids who do well on tests are happier. Our kids are over-tested. Our kids are not tested enough. Escaping from the clutches of untested 'it feels true' statements begins with one simple question: 'what is the evidence?' Howard Wainer shows how using the tools of causal inference he evaluates the evidence, or lack thereof, supporting claims in many fields, with special emphasis in education.


My Review:
Truth or Truthiness is about how to design better causal studies and better graphs. It's mainly targeted at people working in education who can influence policies about testing, tenure, and such. It's written in a very formal way and uses technical language. The author assumed the reader already knew what a "longitudinal study" and "cross-sectional study" are, for example, and that you understand words like "ancillary information," "covariates," and "legerdemain." Some words were defined, but often pages after the author first used them.

While the author was inspired by real claims or studies, many of his Case Studies used made up data to illustrate his point. He explained how to set up a random-assignment controlled experiment, which is the gold standard when possible. He then explained ways to increase the accuracy of observational studies, like gathering additional information when randomization is impossible and how to interpret the results while including missing data (from people dropping out of the study, dying, etc.). He showed how to use extrapolation, ways to deal with unexpected events, and how to create effective graphs to clearly present the information discovered in a study. He also did some ranting about current education policies (removing tenure, detecting cheating, measuring school performance, changes in the SAT, and the accuracy of subscores).

While the information about creating better studies seemed useful, I did not care for his mocking, dismissive tone. For example, he acknowledges that there may be a missing "third variable" in regards to fracking apparently causing increased earthquakes. However, since he can't think of one, there must not be one. People who have pointed out (in their own way) that it's not a certain cause-effect get mocked by the author. I happen to agree with him about fracking, but he mocks people for assuming things because "it makes sense to them." Yet when he does it, his conclusions are based solely on "logic and evidence" and everyone else is either stupid or corrupt.


If you've read this book, what do you think about it? I'd be honored if you wrote your own opinion of the book in the comments.


Excerpt: Read an excerpt using Google Preview.