The following article is an example of an IQI Insights — a brief quarterly newsletter published by the AAAHC Institute for Quality Improvement (AAAHC Institute). The article is copyrighted and reprinted here with the express permission of the AAAHC Institute. Please note that 14 IQI Insights are being combined into one volume, entitled Quality Improvement Insights, which will be available for purchase from the AAAHC Institute (www.aaahc.org/institute) in 2012.
Introduction
This articlefocuses on setting a performance goal for a quality improvement activity (please see AAAHC Standard 5.II.B. 2).
What purposes do performance goals serve?
Performance goals provide:
1) A performance "target" for your QI activity
2) Information that will let you know whether the issueyou have chosen for your QI activity is frequent or severe enough to warrant corrective action or if you need to consider another issue.
Using certain processes for developing performance goals can also provide ideas about the source(s) of the issues you are examining, so that more effective corrective action(s) can be planned.
Sign up for our FREE E-Weekly for more coverage like this sent to your inbox!
Why do I have to set a performance goal so early in the QI process?
In the QI process, once you identify a potential important problem or issue (AAAHC Standard 5.II.B.1), you are supposed to set a performance goal (5.II.B.2) — even before deciding how to measure (5.II.B.3), actually collect information (5.II.B.4), etc. This timing may seem too early in the process to know what a performance goal should be. On the other hand, if you wait until you have the initial data on your performance, will that influence the goal you set? Will this lead to setting a goal you know you can attain with minimal corrective action?
Given that you don't know what your performance is when you set a performance goal, what sort of processes are there to set the performance goal in a realistic and appropriate manner? Let's consider some alternatives to use to set performance goals:
- Guessing
- Significant sustained improvement from measurement results
- Clinical practice guidelines and benchmarking [1]
Guessing
Without any information other than common sense and experience, the first alternative some may consider for setting a performance goal is guessing. Let's use an example from the game or sport of darts to illustrate the problem with guessing; guessing may not only give you a "target" that is not near the "bull's eye" but isn't even on the same wall as the dartboard!!! Further, guessing doesn't provide you with much information about what could be causing a problem or how to correct the problem. Although this may seem to be the only alternative, that is not the case.
Significant sustained improvement from measurement results
A slightly better option (than guessing) for setting a performance goal is aiming for "significant sustained improvement."
Via a monitoring project, internal benchmarking or pilot study, you may have already collected data on your issue. Unless your performance is perfect (100 percent influenza immunization of all appropriate patients or administering prophylactic antibiotics within 60 minutes of incision for all appropriate patients) — and therefore you need to consider another issue for your QI activity — your goal will be 5-10 percent improvement over current performance over a sustained (several month or longer) period of time. Yes, this is setting your performance goal from your measurement results, but it does not allow you to set a goal that you know you can attain with minimal corrective action. What are some of the problems with this method of setting a goal? They include the following:
- If the issue isn't flu shots or antibiotic timing, we may not know what "perfect" is — for example, what is "perfect" for patient wait time?
- Where we do know what "perfect" is supposed to be, is it attainable in "real life?"
- Continuing to use the dartboard analogy from above, 5-10 percent improvement (or more) may still be pretty far from the bull's eye (at the edge of the dartboard or on the wall nearby).
- You may not have gathered much information about the cause(s) or solution(s) to your problem.
Since it is mentioned, let's look at how we could set a goal for wait time: Organization A has completed an internal benchmarking [1] study and practitioners have already reduced the wait time (the time the patient checks in to the time the patient is brought back to the exam room, prep, or operating room) from almost an hour for some practitioners to 45 minutes, on average, for each of its practitioners. That's much more than a 5-10 percent improvement! Patients haven't complained, but sometimes the waiting area can still become a "little" crowded.
So, Organization A might assume that 45 minutes is the "gold standard" (or bull's eye).
Clinical practice guidelines, research literature and involvement in benchmarking activities
When you read the "significant sustained improvement" examples of flu shots and antibiotic timing on earlier, you may have asked, "Who says what 'appropriate' is and sets the bar at 100 percent?" The flu shots and antibiotic timing examples arefrom clinical practice guidelines. National medical specialty societies and others develop guidelines from evidence and expert opinion in order to provide recommendations for improving healthcare delivery. An important part of developing the guideline recommendations is defining the "appropriate" patient populations and once these are defined, some guidelines may suggest 100 percent compliance with recommendations is an appropriate (short-term) performance goal.
When you see 100 percent compliance recommendations in guidelines, you must consider what barriers (and possible solutions) there are to 100 percent compliance. This information may be contained within guidelines themselves or research literature developed from the measurement of real world compliance with guidelines. Here are a couple of examples.
1. Compliance with antibiotic timing guidelines: If a prophylactic antibiotic is recommended (depending on the type of procedure being performed and the patient) in CDC guidelines [2], the recommendations also include very specific recommendations for timing the administration of most antibiotic prophylaxes within 60 minutes of first incisions.
Think rationally about your organization's ability to comply with this guideline. For example, what happens to antibiotic timing when the a case runs long and the next patient has already received the recommended antibiotic prophylaxis because you don't want to "push" the antibiotic too fast but you want to have the patient ready when the surgeon is ready?
A search (5/10/2010) of the research literature, using the U.S. National Library of Medicine Medline (via www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed), and the search terms "antibiotic timing compliance," yields a wealth of information about issues associated with compliance and what a realistic goal (benchmark) may be. Now, you should look in the search results for research that appears to most closely resemble your setting and that can give you ideas that may help you improve [3,4].
2. Compliance with immunization guidelines: CDC also has recommendations for annual influenza immunization for adults with certain risk (medical, occupational, lifestyle, etc) factors or who are over 50 years of age. [5]
Consider what sort of issues may interfere with your organization's compliance with this guideline. For example, what if you have a very transient patient population (example: student health services) or issues of "medical record scattering" (example: immunization information from the Indian Health Services (IHS) RPMS system versus state registries, prior to 2005 [6])?
A search (5/10/2010) of the research literature, using the U.S. National Library of Medicine Medline (via www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed), and the search terms "influenza immunization compliance," provides several ideas about issues associated with annual vaccination compliance (such as the importance of having a regular provider [7]) and what a realistic goal (benchmark) may be. Now, you should look in the search results for those items that most closely resemble your setting (for example, replace "compliance" with "Native American" in your search terms for IHS populations or add "adolescent" to your search terms for information more relevant to student health services]) and ones that can give you ideas that may help you improve [7].
Value of external benchmarking
Now let's go back to our "wait time" example, where there aren't any clinical practice guidelines to indicate what an appropriate wait time goal is and peer-reviewed research literature offers little to nothing in the way of relevant benchmark information. Here is where external benchmarking [1] can provide guidance.
When we left Organization A earlier, 45 minutes met and exceeded their patient wait time performance goal. However, Organization A has now become involved in an external benchmarking study and sees that some peer organizations (Organizations C, G and N, with similar provider bases and similar patient loads and services, etc.) have average wait times of 20-25 minutes. Organization A would not know whether a shorter average wait time than 45 minutes, or how much shorter a wait time goal, is realistic without the information received from Organizations C, G and N. Organizations C, G and N's average wait times of 20-25 minutes suggest that 20-25 minutes is a more realistic/appropriate goal for Organization A to try to accomplish than 45 minutes. Further, if information is gathered from Organizations C, G and N regarding the processes they use to move patients from check in to the exam, prep or procedure room, Organization A can try these to help shorten its own patients' wait times.
Summary
Appropriately framed performance goals are important to QI activities because they give organizations targets (information on what they are striving for in their QI activity) that can be used to judge if an organization has a problem and how big the problem is. Also, the process of setting these goals may provide information about potential reasons for the problem and ways to correct the problem.
Although guessing and setting significant sustained improvement goals are options when setting goals, they are not optimal. By doing the "legwork" to find out whether there are relevant clinical practice guidelines or research that can provide information on actual performance from organizations like yours (benchmarks), you are more likely to develop not only more realistic/appropriate goals, but also find information about possible barriers to optimal performance and ideas for corrective action. When a search for relevant guidelines and research literature leaves you empty-handed, by becoming involved in a benchmark study you can obtain information on realistic goals and processes that have been used successfully to reach performance goals.
More Articles Featuring the AAAHC Institute:
ASC Benchmarking (Part I): Why Does the AAAHC Require Participation in External Benchmarking?
ASC Benchmarking (Part II): Why is Benchmarking So Hard and What Can You Do to Overcome Obstacles?
8 Clinical Findings on Low Back Injection From AAAHC Institute
References and endnotes (Note: References to websites or products are not endorsements. The National Guideline Clearinghouse (www.guidelines.gov) is a source of extensive guideline information available via the internet.
[1] See the Spring 2009 IQI Insights on benchmarking and the Fall 2009 IQI Insights on clinical practice guidelines. From the Accreditation Handbook for Ambulatory Health Care,benchmarking is defined as "a systematic comparison of products, services or work processes of similar organizations, departments or practitioners to identify best practices known to date for the purpose of continuous quality improvement. Internal benchmarking compares performance within an organization, such as by physician or department, or over time."
[2] Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR. Guideline for Prevention of Surgical Site Infection, 1999. Centers for Disease Control and Prevention (CDC) Hospital Infection Control Practices Advisory Committee. Am J Infect Control. 1999. 27:97-132.
[3] Braxton CC, Gerstenberger PA, Cox GG. Improving antibiotic stewardship: order set implementation to improve prophylactic antimicrobial prescribing in the outpatient surgical setting. J Ambul Care Manage. 2010. 33:131-140.
[4] Wax DB, Beilin Y, Levin M, Chadha N, Krol M, Reich DL. The effect of an interactive visual reminder in an anesthesia information management system on timeliness of prophylactic antibiotic administration. Anesth Analg. 2007. 104:1462-1466.
[5] Advisory Committee on Immunization Practices. Recommended adult immunization schedule: United States, 2010. Ann Intern Med. 2010. 152:36-39.
[6] www.ihs.gov/epi/index.cfm?module=epi_vaccine_projects
[7] Hueston WJ. Does having a personal physician improve quality of care in diabetes? J Am Board Fam Med. 2010. 23:82-87.