Economist Paul Krugman has occasionally cast a skeptical eye on FiveThirtyEight.com, Freakonomics, and other sites that attempt to draw large conclusions from smallish data sets. In most cases, the writers of these articles aren't experts in the field they are trying to analyze.
And what do you know, at the Washington Post's WonkBlog, we've got a guy making exactly that mistake. Based on just one of composer Suby Raman's excellent graphs, Christopher Ingraham has reached the conclusion that Opera is Dead, in One Graph.
For crying out loud. It is no secret that the Met is not in any way, shape, or form a hotbed of new music performance. The operatic behemoth has commissioned a tiny number of new operas in the last 25 years and picks up works by living composers only rarely, usually if your name happens to be John Adams or Philip Glass.
Raman's graph would look a lot different if the data set were Houston Grand Opera or even San Francisco Opera, both of which have commissioned a notable number of new works in the last 35 years (and not just under David Gockley, either, at SFO).
It's also not exactly a secret that an awful lot of new and recent operas are first performed, or revived, at the local or college or conservatory level, where smaller-budget organizations with loyal followings can take more chances than the Met or another big-budget US company. And European houses, many of which have government support, are also in a better position to produce new work.
Christopher Ingraham shouldn't be basing broad conclusions about an art form on one graph, but I'm willing to go out on a limb and say that he doesn't follow the business of opera very carefully, and very likely not at all.
In most cases, the writers of these articles aren't experts in the field they are trying to analyze.
ReplyDeleteNate Silver was doing baseball statistics before he did election predictions. Baseball statistician Bill James went from being an outsider and a complete unknown to one of the most influential people in the sport, maybe even the history of the sport. I think that's part of the ideal for Silver and other statisticians inspired by James - the notion that if you can do the numbers, you can find out things even experts in the field did not know (which was certainly true of James).
On the other hand, you can also make blunders like equating the Met with all of opera.
The key phrase is "if you can do the numbers." Ingraham can't, given this particular blunder.
ReplyDeleteSilver is an extremely accurate predictor of elections because he can. :)
"Outsider and complete unknown" does not necessarily equal "not expert in the field he's trying to analyze." It's obvious from Bill James's writings that he's expert in baseball. What he lacked, to begin with, was credentials. Not expertise.
ReplyDeleteBut someone who was expert in opera would already know, as Lisa has clearly pointed out, that the Met is not a good barometer for the health of modern opera. It doesn't matter how thorough your statistical analysis is if you don't know the significance of the statistics you're analyzing.
It's obvious from Bill James's writings that he's expert in baseball.
ReplyDeleteThat's a good point - I was conflating "expert" with "insider".
That's what's remarkable about James - he made himself the expert from first principles (by finding ways of measuring player value better than those widely used by the insider experts).