Note: The following article (reprinted as a two-part series) is an example of an IQI Insights — a brief quarterly newsletter published by the AAAHC Institute for Quality Improvement (AAAHC Institute). The article is copyrighted and reprinted here with the express permission of the AAAHC Institute. Please note that 14 IQI Insights are being combined into one volume, entitled Quality Improvement Insights, which will be available for purchase from the AAAHC Institute (www.aaahc.org/institute) in 2012.
External benchmarking has these clear benefits but it seems like a daunting task to many organizations. Here are some of the reasons why organizations may find external benchmarking difficult and some ways to conquer these potential problems.
Problem: Who are "peers"? It is very important that external benchmark comparisons be among peers; otherwise benchmarks may be unrealistic and/or meaningless ("comparing apples to oranges").
Answer: Peers are organizations that:
- See patients for similar issues
- Provide similar services
- Have similar providers
- Are often competitors
Problem: It seems that benchmarking and quality improvement are the same. What's the difference?
Answer: There are several levels of involvement.
Level 1 — Internal Performance Measurement
- Often an organization will monitor key issues (such as documentation of patient allergies, number of falls at the facility, etc.) over time.
- Through peer review, organizations also assess their own individual practitioners' performance and establish what is the best practitioner performance (or "internal benchmark").
- Risk assessment may also be used to detect changes in level of organizational risk.
These internal activities may reveal a noteworthy, undesirable change (such as a 10 percent decrease in the proportion of immunizations of eligible patients, or a 300 percent increase in proportion of falls), or a great deal of variation in the performance of the organization's practitioners (some have a 70 percent immunization rate others have a 63 percent immunization rate, or some have a fall rate of about 4 percent and others' fall rates approach 12 percent).
Sign up for our FREE E-Weekly for more coverage like this sent to your inbox!
Level 2 — External Performance Measurement
External benchmarking allows comparison with other similar organizations and may provide more appropriate information about whether or how much of a performance issue your organization has.
- National/professional targets/goals might "set the bar" (provide a goal) of 100 percent immunization of eligible patients, or no (0) falls in the facility.
- The organization and its peer organizations are involved in an external benchmark study that indicates the organization's peer organizations' immunization rates are hovering at approximately 90+ percent, or fall rates are 2 percent. These results indicate that organization should not be satisfied with pre "blip" performance or the best performance by one of its practitioners (70 percent documentation of allergies or 4 percent fall rates). The organization should be working toward 90 percent immunization rates and 2 percent falls, with the eventual goal of 100 percent immunization rates and no falls.
- But what if the organization's peer organizations are immunizing only 40 percent (or fewer) of their eligible patients, or peer organizations' fall rates are closer to 5 percent or more? This doesn't mean the organization should not continue to strive for 100 percent immunization rates and no falls; it may mean that there are other areas that should be studied.
Level 3 — Quality Improvement
Neither Level 1 or Level 2 include designing and implementing an intervention (such as a reminder in the chart for the provider to ensure that eligible patients' immunizations are up to date, or an in-service training on transferring patients from recovery beds to wheel chairs) or re-measuring to see if your organization has sustained and demonstrable performance improvement. These are necessary steps in a complete QI study.
Problem: We want to conduct an external benchmarking study. How can we overcome external benchmarking study logistics such as:
- How will peer organizations be recruited to participate?
- Who will decide on the measures?
- Who will do the work (including analyzing data, gathering information on best practices, and reporting)?
- How will patient and provider specific information be kept confidential?
Answer: Organizations often perceive that there are no relevant external benchmark opportunities available to them — they are going to have to develop and conduct the study themselves. However, organizations don't necessarily have to do it alone; they may be able to join programs where the answers to these questions have already been given.
There are several places to look for the right benchmark opportunity:
- Your professional society or organization is a great place to look for external benchmarking opportunities because benchmarking can be one of the many professional services offered.
- Commercial organizations may offer external benchmark services appropriate for your organization.
- Regulatory organizations, such as states, have begun to collect data on health care performance, and some offer external benchmark data back to those in their programs.
- The AAAHC Institute (www.aaahc.org/institute) focuses on external benchmark studies for key ambulatory services and issues.
- The peer review literature can offer external benchmark ideas and goals. To search abstracts from the collection of the National Library of Medicine (MEDLINE), you can go to www.pubmed.gov.
Problem: I have more than one of these options available to me. How do I judge which is best?
Answer: There are pros and cons to each option. Although you don't have a choice about responding to regulatory requirements, when you examine external benchmark options here are some of the questions you need to answer to help you decide whether they really work for your organization:
- Is the topic important enough for your organization to invest time and resources in collecting data and considering change? As with any quality related activity, if you are doing it to "go through the motions" but it is not an integral, important issue to your organization, do not bother wasting your resources. Surveyors can see through this!
- Will this activity offer your organization the opportunity to compare its performance with that of its peers? It really doesn't make sense to become involved in an external benchmark activity with those who are not your peers because their performance and yours may vary due to your organizational differences (things you cannot change easily, such as patient population and services provided). For example, most external benchmarks that can be found in the peer review literature are derived from hospital inpatient data.
- Will the activity provide educational feedback from "best performers" so that your organization can see if their strategies work for you? Knowing what a benchmark is, and having an idea of how to achieve it, are two different issues. Educational feedback can form the bridge between where you could be and where you are now.
- How much will the activity cost? Consider direct costs, such as fees, as well as indirect costs, such as investment in computer software or hardware, staff time and training for data collection, etc.
- When in doubt, if there is an opportunity to chat with staff at other organizations (like yours) that have participated in an activity, do so. They can tell you the pros and cons of their experiences first hand.
Additional resources
- Institute of Medicine. Performance Measurement: Accelerating Improvement. Washington, DC: National Academies Press. 2006.
- Kiefe CI, et al. Identifying achievable benchmarks of care: concepts and methodology. Int J Qual Health Care. 1998. 10:443-447.
- Pantall J. Benchmarking in healthcare. Nursing Times Research. 2001. 6:568-580.
Editor's note: View "ASC Benchmarking (Part I): Why Does the AAAHC Require Participation in External Benchmarking?" by clicking here.
More Articles Featuring the AAAHC Institute:
AAAHC Announces New Quality Improvement Studies for 2012
Choosing a Focus for a Quality Improvement Activity
Study Abstract of AAAHC Institute QI Award Winner