Evaluating Enterprise 2.0 Solutions with a Usability Scorecard

By Bert Sandie posted 05-26-2010 15:22


During the evaluation, marketing and selling of Enterprise 2.0 (E2.0) solutions there is significant focus aimed at the features provided with little or no mention of the usability of the solutions. Typically, vendors provide product sheets that list their solutions features and, in many cases, provide a competitor comparison table to differentiate their offerings. Perhaps there needs to be a major shift on how we evaluate and select E2.0 solutions based on a “user-centric” approach.

So the question is “How do Enterprises evaluate a solution with usability as a major part of the criteria?”

The creation and use of a usability scorecard as part of the evaluation criteria is a first step.  Here is a set of criteria to use as the start of a usability scorecard:

1.    Speed – In many usability studies and surveys the highest rated feature is the speed of the application.  E2.0 solutions need to be fast out-of-the-box as speed is a key attribute in users’ perception of an application.  It is critical to evaluate the application speed at different office locations, time of day and under different loads of usage during a pilot and beta rollout.  A few items to evaluate include login, page navigation, submitting/adding content, and searching.
2.    Intuitive – Solutions need be intuitive to gain adoption. A simple test for intuitiveness is if a user manual is required to understand commonly used features and functionality. Alternatively, the desired behavior is if users can just pick it up and use it (i.e., learnability).
3.    Ease of Use – The KISS principle is important to consider, solutions need to be as simple as possible. Criteria to consider include the number of clicks/steps to complete an action (efficiency) and examining the number and types of errors made.
4.    Enjoyable – Users need to have a user experience which is enjoyable or satisfying as opposed to frustrating or confusing.  
5.    Aesthetically Pleasing – A solution that is aesthetically pleasing appeals to us from a sensory and visual perspective. This aids in initial adoption as it draws us in and in continued usage as it appeals to us.

Another factor to consider is what applications employees use outside of the work environment, as these will help set benchmarks and usage behaviors to consider during your usability evaluation.

It is highly recommended that during initial evaluations, pilots and beta programs, observe employees undertaking a set of common use cases that they would do routinely with the solution. Listed below are a number of use cases to consider as a starting point:

  • Create/Edit a User Profile

About Me
Skills and Interests
Add colleagues/friends

  • Add content; i.e., an article, blog, forum, video

o    Editing content – inline and adding attachments
o    Tagging content
o    Adding authors
o    Publishing content

  • Comment on existing content
  • Join a Community
  • Find content via navigating a tag cloud
  • Search for a:

o    Person
o    Content; i.e., Article, Blog or Video

Capturing the user actions is best achieved through videotaping or screen capturing sessions. Watching the sessions will help complete the usability scorecard. Finally, interviewing the users will help provide the final insight to help complete the scorecard.

To learn more about usability check out Jakob Nielsen’s site at:

Good luck in finding the right solution for your employees needs!

#usability #user-centric