Defining a framework for health information technology evaluation

Governments and providers are investing in health information technologies with little evidence as to their ultimate value. We present a conceptual framework that can be used by hospitals, clinics, and health care systems to evaluate their health information technologies. The framework contains three dimensions that collectively define generic evaluation types. When these types are combined with contextual considerations, they define specific evaluation problems. The first dimension, domain, determines whether the evaluation will address the information intervention or its outcomes. The second dimension, mechanism, identifies the specific components of the new information technology and/or its health care system that will be the subject of the evaluation study. And, the third dimension, timing, determines whether the evaluation occurs before or after the health information technology is implemented. Answers to these questions define a set of evaluation types each with generic sets of evaluation questions, study designs, data collection requirements, and analytic methods. When these types are combined with details of the evaluation context, they define specific evaluation problems.