Developers need formative evaluation, quick feedback for making mid-project corrections or for improving the next version. Formative evaluation answers vital questions, like does this part work? Is this process too complicated?
On the other hand, both buyers and developers are interested in summative evaluation, an assessment of a system’s overall effectiveness, which could help predict how a system would fit into a specific clinic’s unique workflow.
An evaluation should answer questions and reduce uncertainty. It’s easy to answer questions about the concrete items: feature set, system compatibilities, cost and so forth - hard facts that should be readily available. But how do you evaluate something so important, yet seemingly so hard to define, as usability?
NCCD has put a great deal of time and effort into analyzing usability and specifying and defining its components. These attributes - useful, usable and satisfying – might seem intangible at first glance, but they can be defined and more importantly, they can be measured. Here’s how:
All of these qualities are essential for success. For example, a wonderfully usable system might not be useful, in which case it will be difficult to get the job done. And vice versa – if a useful system isn’t usable, then people will eventually get frustrated trying to learn it or feel like using it is a waste of time. Finally, if working with the EHR isn’t satisfying, how long before users start to avoid using it and eventually find excuses for not using it?
NCCD uses numerous methods for evaluating usability, and more importantly, we understand when each method is appropriate. In general, testing falls into two categories: evaluation by usability experts, and testing that involves real users. Experts will evaluate a system according to proven usability principles, by analyzing the environment in which it’s used, and by the application of special software tools such as Cogtool. With user-based testing, on the other hand, trained observers watch users work with an EHR, or ask users for feedback after or while they are using the system (using surveys, focus groups, talk-aloud techniques, etc.).
You already know the answer to this one: as early as possible. In the design phase, certainly before prototyping, engineers should be thinking usability and be familiar with the guidelines. Usability is a state of mind, an attitude.
It’s not too late, and in fact NCCD has developed a system to get rapid, accurate feedback on your product before you go to market. And if the product is already out there, usability engineering will help you improve the next release.
As the old adage says, you never get a second chance to make a first impression, so it’s best to get things right the first time. Early investment in usability engineering will make that first impression a good one.
