Sign up

Benefit realization.
Subject:
Adoption (Laws, regulations and rules)
Physicians (Laws, regulations and rules)
Authors:
Fickenscher, Kevin
Bakerman, Michael
Pub Date:
11/01/2011
Publication:
Name: Physician Executive Publisher: American College of Physician Executives Audience: Professional Format: Magazine/Journal Subject: Business; Health care industry Copyright: COPYRIGHT 2011 American College of Physician Executives ISSN: 0898-2759
Issue:
Date: Nov-Dec, 2011 Source Volume: 37 Source Issue: 6
Topic:
Event Code: 930 Government regulation; 940 Government regulation (cont); 980 Legal issues & crime Advertising Code: 94 Legal/Government Regulation Computer Subject: Government regulation
Product:
Product Code: 8011000 Physicians & Surgeons NAICS Code: 621111 Offices of Physicians (except Mental Health Specialists)
Geographic:
Geographic Scope: United States Geographic Code: 1USA United States
Legal:
Statute: American Recovery and Reinvestment Act

Accession Number:
273715896
Full Text:
In January, we began a series of articles on health information technology by noting that clinician adoption of new technology requires a rigorous look at systems, people, processes and capabilities. To date, we have discussed leadership and governance, change management and process redesign and clinical adoption principles.

Now, let's take a look at the importance of capturing often elusive clinical benefits measurements.

Measuring ROI

When the chief financial or operations officer asks, "What is our return on investment from all of the health information technology deployments we are making?" The answer is usually, "that depends. ..."

While some authors have documented a financial return on investment related to the "benefits" derived from deploying clinical systems initiatives, most of the literature remains silent on the issue.

There is a sense that if a return on investment cannot be expressed in terms of dollars and cents, hospital executives and boards of directors will stop funding these initiatives, even though quality and patient safety returns are frequently self-evident, especially to physicians and other clinicians.

In fact, it was this very issue that prompted Congress to pass legislation as part of the American Recovery and Reinvestment Act (ARRA) in support of clinical information systems deployments that ultimately resulted in the meaningful use guidelines.

Historically, the non-monetary returns on investment derived from enhanced quality and patient safety have been insufficient to justify the expense of a true benefits realization program. However, the external pressures, such as the Institute of Medicines (IOM) "call to action" (Too Err Is Human and Crossing the Quality Chasm) and subsequent the federal legislation have created an imperative for moving ahead with clinical information systems initiatives. In fact, the investment incentives are moving the health care industry in a new direction that will ultimately reduce costs, improve outcomes and enhance service.

When it comes to measuring a return on investment related to the benefits of clinical systems initiatives, the indicator most commonly used is expressed in terms of financial gains or cost savings--as are most expenditures, initiatives, or investments in the health care arena.

While valuable, measuring gains in terms of dollars and cents alone fails to capture the value of increased quality or patient-safety. It is these outcomes that are becoming increasingly important priorities in today's health care arena and will gain momentum as we move from volume to value in the new era of health reform.

Already, organizations are scrambling to meet the criteria for patient-centered medical homes through the CMS and NCQA initiatives--all of which focus on patient access, safety and quality of care. Meeting core measurement requirements, decreased or zero reimbursement for certain readmissions, and transparency all need to be taken into consideration when determining a return on health information technology investment.

Additionally, reporting meaningful use measures for eligible providers and eligible hospitals is a new dynamic in measurement and reporting. These elements will need to be incorporated into any benefit realization program

Developing a measurement framework

Adaptations from measurement science (i.e., the use of Shewhart's control charts and regression analysis) provide a framework for implementation of a benefits realization solution in a clinical environment. The use of a clear framework provides decision makers with methods for more accurately creating and disseminating statistical reports.

In most health care organizations, however, the limitations in measurement practices where multiple operational variables play a factor in results often obscures the ability of management to draw cause-and-effect relationships from the reported results.

The result is that good leaders and providers arc left in a quandary as to how best to achieve clinical and financial benefits realization.

The answer is that benefits realization can be best accomplished through the use of e-technology measurements and leadership-driven process improvements. We have worked with a number of health systems to solve this elusive benefits realization issue. Executive buy-in and the delivery of timely, metric-driven data points are crucial elements for fostering efforts to improve patient care. The integration clinical technology infrastructure with clinical transformation is essential.

The benefits realization pyramid (see Figure 1) offers an underlying framework for health care delivery organizations to better understand the linkages between technology, clinical transformation, and the benefits achieved. The deeper the breadth of technology and the level of transformational effort, the more benefits can be realized.

There is only one way to optimize the impact of information technology and transformational efforts within an organization and that is through the use of health care measurement science. To be effective, a benefits realization program must hardwire reliable measurement processes into the ongoing operation of the health care system, including efforts to:

[FIGURE 1 OMITTED]

* Link strategic goals and objectives to operational outcomes

* Create synergy between existing measurement programs

* Promote measurements that support course correction

* Enhance technology system design for clinicians

* Promote buy-in to the selection and deployment of measures

* Operationalize the assessment of value and opportunities for improvement using a sustainable, reliable approach

The traditional viewpoint for measuring the clinical systems benefits is to implement the system and then measure data to determine the impact of the system on operational processes. Such an approach, however, is shortsighted. Such a retrospective approach promotes a "tail-wagging-the-dog" effect, where the benefits measured are often selected because the data are readily available post go-live.

A second issue with the traditional retrospective approach is that as post-go-live performance issues become evident, the attention of health care executives is heightened. When this occurs, the implementation team frequently moves into a firefighter mode by focusing time and energy on resolving hot topic issues.

While resolving the hot topics is important, it frequently derails efforts to truly measure benefits due to the lack of a formal benefits realization program. Most often, the issues faced by the implementation team will influence the benefits achieved. The issues represent variables that should be proactively considered as part of a more comprehensive benefits realization program.

For example, if one of the benefits to be measured is clinician adoption, then the physician login issues must be resolved or adoption will be unduly impacted.

An alternative approach to establishing a benefits realization program is to adopt a measurement framework that has its origin in the strategic and operational plans of the organization. Using such an approach requires that organizational leadership consider the primary attributes of enabled technology during the formative stages of the decision making process related to investments in clinical information systems (CIS).

Elements that can provide the approach toward evaluating the investment include consideration of CIS capabilities related to:

* Information access and data use for all stakeholders

* Horizontal--across departments/business units

* Vertical--within departments/ business units

* Historical--data trending and actual results reporting

* Predictive--forecasting and future episode of care events evaluations

* Information timeliness

* Efficiency of operations

* Process/decision support (clinical and operational)

* Evidence/protocols management

* Automation/alerts and decision support

Consideration of these information system attributes and how they will impact the operations and processes within the health care system should be incorporated into the CIS selection process. During the CIS selection process, the selection team should also consider what strategic and operational problems they want to solve by implementing corresponding clinical technologies. These issues should drive the benefits realization indicators selected for measurement.

For instance, if a hospital is purchasing a pharmacy system to improve patient safety in the medication administration process, then indicators that measure patient safety should be included in the framework for the benefits realization program. For hospital systems, it is important to ensure a common framework horizontally across the system. This will enable intra-hospital comparisons that will be more valid and reliable.

The authors participated in and helped lead the development of a useful framework at Perot Systems that we believe provides a valuable approach in implementing a benefits realization program. The SCORE framework promotes an organized approach to benefits realization indicator management. SCORE is an acronym that reflects five key strategic or operational goals related to the deployment of clinical technologies, including:

* S = Safety/quality

* C = Clinical automation

* O = Operational efficiency

* R = Return on investment

* E = Evidence-based utilization

Within each of these five domains, an unlimited numbers of indicators can be developed to measure the outcomes of the systems implementation and the impact they have on organizational efficiency and effectiveness.

While the possibilities are "unlimited," our experience suggests strongly that the health care organization select one or two domains as the primary focus for benefits realization. By using such an approach, key attributes or drivers can be monitored that create synergy for the benefits expected from the investment based on the particular needs of the health care organization.

As an example, some organizations for strategic purposes may be focused on safety and quality while another organization really must foster operational efficiency before moving toward a focus on safety and quality. The use of the SCORE framework allows the health care organization the flexibility for managing across the spectrum of opportunities that have the potential for supporting greater efficiency and effectiveness.

It is critical to define each domain within a set of SCORE domains so there is a clear organizational understanding of what is intended to be measured. Ensuring that the constituencies involved in the benefits realization program have a clear understanding of the framework and the areas to be measured will dramatically improve data consistency and, ultimately, data integrity.

It is important to note that satisfaction is not included as a domain in the SCORE framework, even though there is some interest in measuring this area post-implementation of a clinical information system. Both patient satisfaction and employee/ physician satisfaction are measured using survey tools to gather data on an annual or semi-annual basis. These survey tools do not usually include questions that will help answer the question "What impact does the implementation of clinical systems have on your work environment?" (or, " ... your care?").

While these may be questions that should be asked, most patients cannot relate the use of technologies as a tool for improving their care. Proxies are often used, such as "Was the registration process timely and efficient?"

The SCORE framework uses similar assumptions. If patients are safe and have been given quality care, then their satisfaction will be high. If clinicians and physicians are satisfied with the system, they will embrace automated systems for clinical care management and will use the system as it is designed to be used. Inclusion of patient, physician, and clinician satisfaction should be performed as a balancing indicator when those results become available.

Indicator development and selection

In an infinite universe of possible variables to measure. It is helpful to define the scope of a benefits realization program. This will ensure that the program is "doable" and that the organization is not "collecting data for data's sake."

Every indicator included in the framework for benefits realization should be a deliberate decision on the part of the organization's leaders. Data collection and management take time and effort that is then not able to be spent on other initiatives or patient care. Clear, actionable indicators, within each of the SCORE domains, helps promote improved measurement and comparability, both internally and externally to an organization.

Accountability for consistency in data collection and reporting is a critical step in ensuring comparability, especially within distributed health care systems. Finally, it is important to recognize that the specific domains used by the health care organization can change or evolve over time.

For example, an organization may start with a focus on operational efficiency and move toward one of the other domains as success is realized. In our experience, use of the framework is an important adjunct in the process because the pareto principle or the "80-20 rule" applies whereby 80 percent of the value is derived from 20 percent of the activity.

The same principles applies in benefit realization. Eighty percent of the value derived from clinical information systems will come from the 20 percent of domain areas targeted by the organization under the SCORE framework.

The SCORE cards (see Figure 2) provide examples of tools that can be used to report meaningful indicators. The first example shows a scorecard that can be used for reporting results, while the second is a tool to publish the criteria upon which the indicator level of performance is scored.

Good information and good management

Understanding and using measurement terms and definitions consistently across the organization will improve data management, thereby enhancing the integrity of the results reported. This can be accomplished, in part, by having clearly defined indicator statement for each reported measure. Indicator statements describe in narrative form what is to be measured, how it will be measured, and how it will be reported.

The first step in writing an indicator statement is to clearly identify a question you want to answer by conducting data analysis and measurement. In a strong benefits realization program, the questions asked are always linked back to the organization's strategic and operational goals for clinical system implementation. Consider the following guidelines when defining indicator statements:

* Specifically define the activity (or a piece of any given process/activity) that you want to measure in order to answer the question posed. This promotes the ability to identify what impact a change in the process will have related to opportunities for improvement. In addition, clarity on the purpose of collecting the data will reduce the downstream challenges from colleagues and other constituencies, such as "Why are we measuring that?"

* The process should include the generation of clear definitions of the populations "or data sample" to be assessed and may also include a list of those parts of the process or population that you need to exclude (such as including only inpatients in length of stay calculations or excluding those patients who are coming into the emergency department for a 24-hour follow-up visit).

* Establishing data statement parameters very early in the process helps to clearly define the scope of the indicator that is being assessed and facilitates data analysis and interpretation.

Too frequently, health care organizations require too many metrics in compiling evidence for a benefits realization program, which can cause conflict between stakeholders and the organization. Leadership must endeavor to ensure that a limited number of meaningful indicators are included in the benefits realization program.

Data collection, analysis, and reporting Is a time-consuming process, even with the advantage of automated systems. Also, it is imperative that the same indicators be used across multi-system organizations if comparisons over time arc to be included in the reporting process. Selecting only three to five indicators for each of the SCORE domains promotes a balanced analysis of the major impacts of enabled technology and clinical systems in the health care environment.

And, finally, under the notion that repetition breeds recognition--we highly recommend that only one or two domains serve as the primary focus for the organization's benefits realization program.

Establishing collaborative relationships between the clinicians, physicians, information technologists, and quality management teams very early in the process is critical to program management success. These teams each possess unique viewpoints of the same system and together can provide a balanced view of implementation challenges, data management issues, and acceptance of results reporting.

Pushback from physicians, clinicians, and department leaders often occurs when initial data reports are received. Leaders must establish a process for dialogue to educate all interested parties about the indicator definitions, (as outlined in the written indicator statements) prior to publication of data.

Once the data have been disseminated, ongoing forums for discussion should be encouraged, and legitimate reasons for trends and findings must be explored and understood. Successful programs involve the interested parties early in the process so that issues and concerns related to potential indicators on the panel can be revealed and discussed during the developmental phase of the program.

References

1. Thompson DI, Henry S, Lockwood L, Anderson B, and Atkinson S. "Benefits Planning for Advanced Clinical Information Systems Implementation at Allina Hospitals and Clinics." Journal of Healthcare Information Management. Winter 2005. Vol. 19, No. 1.

2. Institute of Medicine- Too Err Is Human: Building a Safer Health System. L. T. Kohn, J.M. Corrigan, and M.S. Donaldson, eds. Washington, DC; National Academy Press, 2000.

3. Institute of Medicine - Crossing the Quality Chasm: A New Health System for the 21st Century. Washington. DC; National Academy Press, 2001.

NOTE: The authors wish to acknowledge the contributions of Patricia Bush, RN, and other members of the Perot Systems clinical transformation team who assisted in developing many of the concepts and ideas outlined in this paper.

[ILLUSTRATION OMITTED]

Kevin Fickensche, MD, CPE, FACPE. FAAFP, president and founder of CREO Strategic Systems. kevin.fickenscher@ps.net

[ILLUSTRATION OMITTED]

Michael Bakerman, MD, MMM, CPE, FACPE, is CMIO of UMASS Memorial Health System. Michael.bakerman@umassmemoriaL.org
Figure 2

Performance Scoring Levels

Meets/     At Risk,      Requires      New          Sept  Oct.  Nov.
Exceeds    Optimization  Performance   Initiative,
Standards  Strategies    Improvement   Not Scored,
           Recommended   Strategies    Not
                                       Applicable
                         Medication
                         Verification
                         Nursing

                         Medication
                         Verification
                         Pharmacist

                         Lab Time to
                         Results
                         Routing

                         Lab Time to
                         Results Stat

                         Radiology
                         Time to
                         Results
                         Routine

                         Radiology
                         Time to
                         Results Stat

Meets/     Target
Exceeds
Standards


           60
           Minutes

           60
           Minutes

           60
           Minutes

           30
           Minutes
           60
           Minutes


           30
           Minutes


Figure 2

Indicator        Performance   Performance       Requires
                 Level within    Level at      Performance
                 Reach/Meets       Risk/         Improvement
                    Target     Optimization  Plan/Strategies
                                Strategies
                               Recommended
Operational
Efficiency

  Median Coding        3 days      4-5 days       More than 5
Completion Time                                          days
 from Discharge

 Cross Match to           2.0           3-5       More than 5
    Transfusion
          Ratio

 Discharged Not     3 days or      4-5 days       More than 5
   Final Billed          less                            days

  Lab Duplicate    2% or less         3-40%      More than 5%
           Test
Cancellations as
   a percent of
   all canceled
          tests

      Radiology    2% or less          3-4%      More than 5%
 Duplicate Exam
Cancellations as
   a percent of
   all canceled
          tests

  ED Efficiency    210 min or   211-240 min     More than 241
Time into ED to          less                             min
      Admission

  ED Efficiency       120 min   121-251 min     More than 252
Time into ED to                                           min
      Discharge

 Productivity-%   increase of   Increase of  Increase of less
     Productive    5% or more          4-3%           than 3%
        Hrs/UOS

        Time of     30-40 min     41-60 min      More than 60
     Medication                                           min
Verification by
  Nursing Order
to Nursing Sign
            Off

Indicator            New
                 Initiative/
                     Not
                 Scored/Not
                 Applicable

Operational
Efficiency

  Median Coding
Completion Time
 from Discharge

 Cross Match to
    Transfusion
          Ratio

 Discharged Not
   Final Billed

  Lab Duplicate
           Test
Cancellations as
   a percent of
   all canceled
          tests

      Radiology
 Duplicate Exam
Cancellations as
   a percent of
   all canceled
          tests

  ED Efficiency
Time into ED to
      Admission

  ED Efficiency
Time into ED to
      Discharge

 Productivity-%
     Productive
        Hrs/UOS

        Time of
     Medication
Verification by
  Nursing Order
to Nursing Sign
            Off
Gale Copyright:
Copyright 2011 Gale, Cengage Learning. All rights reserved.