Blogs

I found myself in Vancouver BC last night, having dinner with my good friends Sherman Woo (@spsherm), Michal Pisarek (@michalpisarek) and Paul Culmsee (@paulculmsee) talking about food, travel, SharePoint, and everything else.  After eating far too much, we disbanded with the group, and Paul and I made our way to a local pub where we ended up talking for another couple hours about past projects and customers. Interestingly, we found ourselves circling back again and again to the problems of search within every collaboration platform, and the fact that it is most important aspect of knowledge management.

I had a 2.5 hour drive back to Seattle to think more on the topic, and thought I'd share a few ideas.

One point that Paul made was that even within the definition of 'contextual search,' there is a vastly different result set between a two-dimensional search versus a three-dimensional search. In other words, a two-dimensional search would use two points of reference, such as 1) an index or taxonomy to allow searching within a subset of the overall data, followed by 2) a keyword, which would further refine the results

Now compare this with a three-dimensional search using 1) an index or taxonomy to provide a data subset to which within, followed by 2) a keyword within those results, and then 3) application of social relevance for further refinement of the results. Logically, finding a specific point in space is quicker and more accurate through triangulation. Why would this not also be true for data? As we discussed, the quality of the result would be substantially greater. Of course, we also recognized that, in this example, the quality of the result depends greatly on the quality and depth of your network. Limit your network to 10 people, and that data dimension loses its value. I've written on this topic. However, the value of expanding your social network is a conversation for another day.

I refer to this data triangulation generically as contextual search, but my definition may be different than yours. Google, Yahoo and Microsoft have spent the bulk of their advertising technology investments on building out contextual search, and yet the technology falls short, arguably, of what is needed within knowledge management systems. Their approaches are largely to crawl, scan and analyze content using keyword recognition, sometimes adding human evaluation. But search from a simple crawl is slow and inefficient. Building massive indexes improves context, but again relies on a strong taxonomy and appropriate application of metadata to artifacts. This works well for targeted advertising, but falls far short of how the human brain seeks for content, makes connections.

And let's be honest, this limited contextual experience is better than nothing. But the future of search requires some degree of intelligence, some way of linking otherwise disparate content much like the human brain has the ability to create and destroy neural pathways at it grows and adapts.

I usually state the problem as: traditional search provides results based on what you ask for, but what I really want is what I am not asking for. In other words, if I knew the exact keyword combination, I'd use it. But I don't always know how to ask for the data I am really looking for. Today's contextual search would continue to provide suggestions based on what I know is the wrong direction….which often makes for a frustrating search experience. What we have not yet been able to automate is the contextualization of our queries, because it relies on the use of language and discourse (bouncing ideas off of your network) to refine both the questions we ask and the value of the responses we receive. Millions of lines of code have not been able to duplicate this so far.

Enter the social layer, which provides an often ad hoc contextual learning platform, allowing us to share, review, and discuss ideas, applying personal, religious, cultural, professional, and anecdotal filters to our content, thereby contextualizing our content.  While the technologies are not yet fully mature, it is your best chance of reaching a truly contextual search experience. It's why both Microsoft and Google are working to integrate social networking components into their search results - Microsoft through their Facebook relationship, and Google through their YouTube and Google+ investments.

If you've ever been through one of my sessions on social computing, you've heard me say "social is just another layer of search."  Now you have a little better perspective of where I'm coming from.