Every time I read an article about some company analyzing the scary end of the Twitter fire hose, I am convinced that Big Data is the equivalent of Dark Matter – it’s important, but not to me on a daily basis. Working in Information Services for a small company, it doesn’t take too long for ‘big’ to become absurd, but it seems that whenever I think big data doesn’t matter, I am reminded of why it does. When I wrote about this earlier, some of my friends took the time to correct me. This past Saturday, I was once again reminded that, like dark matter, big data can change our world even if we choose to ignore it.
I was listening to my favorite radio show, Only a Game (WBUR Boston) when host Bill Littlefield started talking about how Miguel Cabrera winning baseball’s triple crown (most home runs, hits and RBI’s) might not make him worthy of baseball’s MVP award. What would cause us to question his accomplishment? Well, a kind of big baseball data. Recently, a new crop of statistics has grown up around the sport that we were able to manage in our heads as kids, and two of these, on-base-plus-slugging (OPS) and Wins Above Replacement (WAR) might suggest that a different player should be the American League MVP. On-base-slugging? At first, I want to shake my head and join the legion of baseball fans who complain about these new statistics that require calculators and lots of RAM. Then I realize that all you need is an iPhone app, something as likely to be in the pocket of a little league player today as a pack of baseball cards was in the 60’s.
Big or at least bigger data is all around us, as are the tools to analyze it and digest its meaning. Here are two very simple (and small by big data standards) examples that have recently shown up in my small world:
We need to save that – As we work to determine what documents stay and what documents go, the advocates of keeping things always suggest the absolute value of a document by pointing out how often someone refers to it, as in: “oh. We use that all the time…” Guess what, I can know exactly how often people refer to documents, how often they open them, copy them and if I want, I can collect what they think about them. Just having the basic statistics of how often anyone even visits a library is starting to make people think harder about document value.
Future Project – We have thousands of documents that we have wanted to comb through to identify the few gems that will likely have value to our future selves and those who follow us. These might be expert opinions solicited during a court case, or interesting findings uncovered during an inspection, or a presentation that nailed the description of a complex bit of science. I demonstrated last week how this is and isn’t really a future project any more. First, it is a future project because it’s an ongoing effort, but it really isn’t the effort we feared. People can tag documents as they encounter them, and we can sort and filter and find these documents in the future using those tags. I should point out that I’m not talking about SharePoint’s social feature, but the use of Site Column metadata. The second thing I pointed out was search. The addition of BA Insight’s Longitude Search product gives our employees the ability to search and then refine and re-refine those search parameters to dig through big piled of documents.
Like the data behind Wins Above Replacement, these attributes have always been in existence. Some are new data elements, but some are the result of our newfound ability to consider them in a meaningful and repeatable process. I can’t imagine employees voting for Most-Valuable-Document anytime soon, but who knows, there’s probably an app for that.
Note: Even if you aren’t a huge sports fan, you might enjoy Only a Game. You can read the transcript and hear the podcast of the article behind this story here, and you can hear a story leading up to this, before Cabrera hit his triple crown winning homer over here. The comments tell the story of audience reaction to baseball’s new big data.#baseball #metadata #BigData #smallbusiness #onlyagame #bainsight #wbur