Validating a popular outpatient antibiotic database to reliably identify high prescribing physicians for patients 65 years of age and older

Summary

Objective

Many jurisdictions lack comprehensive population-based antibiotic use data and rely on third party companies, most commonly IQVIA. Our objective was to validate the accuracy of the IQVIA Xponent antibiotic database in identifying high prescribing physicians compared to the reference standard of a highly accurate population-wide database of outpatient antimicrobial dispensing for patients ≥65 years.

Methods

We conducted this study between 1 March 2016 and 28 February 2017 in Ontario, Canada. We evaluated the agreement and correlation between the databases using kappa statistics and Bland-Altman plots. We also assessed performance characteristics for Xponent to accurately identify high prescribing physicians with sensitivity, specificity, positive predictive value (PPV), and negative predictive value.

Results

We included 9,272 physicians. The Xponent database has a specificity of 92.4% (95%CI 92.0%-92.8%) and PPV of 77.2% (95%CI 76.0%-78.4%) for correctly identifying the top 25^th percentile of physicians by antibiotic volume. In the sensitivity analysis, 94% of the top 25^th percentile physicians in Xponent were within the top 40^th percentile in the reference database. The mean number of antibiotic prescriptions per physician were similar with a relative difference of -0.4% and 2.7% for female and male patients, respectively. The error was greater in rural areas with a relative difference of -8.4% and -5.6% per physician for female and male patients, respectively. The weighted kappa for quartile agreement was 0.68 (95%CI 0.67–0.69).

Conclusion

We validated the IQVIA Xponent antibiotic database to identify high prescribing physicians for patients ≥65 years, and identified some important limitations. Collecting accurate population-based antibiotic use data will remain vital to global antimicrobial stewardship efforts.

Keywords:

Physicians – Antimicrobials – antibiotics – Outpatients – Rural areas – Primary care – Ontario – Canada

Introduction

Rising antimicrobial resistance is a global public health threat jeopardizing multiple advances in modern medicine.[1] Antibiotic overuse in humans is associated with resistance at both the individual [2] and ecological [3] levels, and is the most important modifiable driver of antimicrobial resistance. An essential component of global, national, and regional antimicrobial stewardship strategies is improved surveillance of antibiotic use.[4–6]

Various forms of antibiotic use data are available including dispensing data compiled by pharmacies (e.g. National Prescription Audit^® CompuScript^® or GPM^®)[7] [8] or funders (e.g. Ontario Drug Benefit) [9] which can be population based. Other databases include dispensed data at the prescriber level (e.g. Xponent^®) [10] or by prescription from electronic medical records.[11] These data sources have important differences and may not correlate over time. [11,12] Many jurisdictions do not have access to valid population-based community antibiotic use data and rely on third party proprietary data. The most common source of antibiotic use data globally is IQVIA, formerly IMS Health. IQVIA maintains a variety of prescription drug databases. In the United States (U.S.) and Canada, the IQVIA Xponent database is derived from 65–70% of the populations’ dispensed prescriptions [7,13] IQVIA then applies a proprietary geospatial extrapolation algorithm. The data are then sold to third parties representing 100% complete prescription data. The projection methodology is internally validated by the company.[14] Despite some uncertainty in the data validity multiple organizations in Europe, Asia, and North America, such as the U.S. Centers for Disease Control and Prevention and Health Canada, rely on these data for both research and surveillance of antibiotic use patterns.[7,13–23] In Canada we have used IQVIA data to describe overall antibiotic use,[7,24] to identify predictors of prolonged antibiotic durations,[10] and we are currently studying the impact of providing audit and feedback letters from these data to primary care physicians (https://www.clinicaltrials.gov/ct2/show/NCT03776383).

The Ontario Drug Benefit (ODB) database has population-based drug dispensing data for medications funded by the ODB program, for all persons in Ontario, Canada over 65 years of age and has been previously validated compared to chart abstraction to be over 99% accurate.[25] Our objective in this study was to validate the accuracy of the IQVIA antibiotic Xponent database to identify high prescribing physicians compared to ODB as the gold standard for patients 65 years of age and older.

Methods

Setting

We conducted this study at ICES (formerly the Institute for Clinical Evaluative Sciences) in Ontario, Canada. ICES is a not-for-profit research institute and a prescribed entity under Ontario’s Personal Health Information Protection Act with permission to securely collect and store personal health information. Ontario has universal health insurance which includes virtually all residents (excluding recent migrants within the previous 3 months, those residing on an indigenous reserve, and military personnel). All persons 65 years of age or older, as well as select low income individuals or those on disability, have publically funded drug insurance through the ODB which includes most commonly used antibiotics.[26] The study population consisted of physicians prescribing antibiotics to patients 65 years of age or older, between 1 March 2016 and 28 February 2017. We have previously observed large disparities in antibiotic prescribing to male and female patients 65 years of age and older,[7] therefore we sought to validate physician prescribing to male and female patients separately.

Data sources

Xponent is an IQVIA database with dispensed antibiotic prescription counts aggregated at the physician prescriber-level. Non-physician prescribers (i.e. dentists, nurse practitioners, optometrists, etc.) are not included in this database. Antibiotic prescription counts were provided for total and 13 antibiotic class specific groupings. Antibiotics were defined as World Health Organization Anatomical Therapeutic Chemical Classification J01 antibacterials for systemic use limited to those taken orally and dispensed from an outpatient pharmacy (S1 Table). Counts of antibiotic prescriptions were limited to outpatient new prescriptions, excluding refills, topical, and intravenous medications. The data also includes antibiotic prescribing rates (number of antibiotic prescriptions/number of total prescriptions by that physician) and proportion of antibiotic prescriptions that were prolonged in duration which we defined as >8 days.[10] The data was broken down by patient age and sex strata; all patients, males <18 years, females <18 years, males 18–64 years, females 18–64 years, males >64 years, and females >64 years. We limited this study to males and females >64 years for which we had complete population data in ODB to act as the reference standard.

IQVIA creates the Xponent database by obtaining prescription data directly from 2,187 (49.8%) of Ontario’s 4391 pharmacies. As of 2017 this amounted to 61.3% of all Ontario prescriptions. IQVIA then incorporates sales and insurance data, as well as the geographical location of pharmacies not captured, into a patented geospatial projection algorithm and extrapolates to estimate all physician prescribed antibiotics. The methodology is proprietary, but according to the company is routinely internally validated.[15,27] Xponent was linked to ICES using unique College of Physicians and Surgeons of Ontario numbers.

We used ODB, held at ICES, from persons ≥65 years of age as the reference standard. The ODB database contains all dispensed medication claims prescribed by physicians from a formulary of 4400 prescription drugs funded through the ODB program. The ODB database has been previously validated against pharmacy chart abstraction and determined to be >99% accurate.[25] To exclude refilled prescriptions from both Xponent and ODB, we defined refills in ODB as the same drug being dispensed for the same patient, prescribed by the same physician for the identical dose and duration within 14 days of the expected end date of the previous prescription. The only antibiotic expected to be potentially poorly captured in ODB was doxycycline as was not funded by the ODB at the time of this study. We used the Ontario Health Insurance Plan database to ascertain the number of outpatient visits.

We excluded physicians who prescribed <9 antibiotics to patients ≥65 years of age during the study year in either Xponent or ODB databases. We used this cut off, which removes 5% of all antibiotic prescriptions, in order to exclude infrequent antibiotic prescribers. We excluded physicians with <200 total outpatient visits in the year based on billing claims in the Ontario Health Insurance Plan database. We also excluded the few physicians who had illogical antibiotic prescribing rates of ≤0% or ≥100%, or if there were more antibiotic prescriptions than patient visits.

Statistical analysis

We performed multiple analyses to evaluate the agreement and correlation between Xponent and ODB antibiotic data in patients ≥65 years of age. Our primary interest with the data was to be able to reliably identify high antibiotic prescribing physicians. We organized both Xponent and ODB in quartiles based on the volume of antibiotic prescriptions per physician. We first dichotomized the quartiles as high (top quartile) and low (bottom three quartiles) prescribers to calculate the performance characteristics of accurately identifying a high prescriber in Xponent. We first did this for all physicians and then restricted to primary care physicians only since this specialty prescribes majority of the outpatient antibiotics.[24] We reported sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), with 95% confidence intervals (CIs). As a sensitivity analysis, we adjusted the definition of high prescribers in ODB from the top 25% to 30% and to 40% in order to evaluate the degree of Xponent error. For instance, if 20% of high prescribers were misclassified in Xponent, but 90% of these were in the top 40% of ODB, this provides more reassurance related to the magnitude of error in the data. If low prescribing physicians in the bottom 60% were misclassified as high prescribers, this would represent a more clinically significant misclassification.

We descriptively compared the mean number of antibiotics prescribed per physician. Then we used the quartiles and evaluated agreement using weighted kappa statistics with 95% CIs for total antibiotics as well as subclasses of antibiotics. We did this for male and female patients separately as well as by urban and rural locations of practice. Rural areas were defined by the physician location of practice residing in a town with a population of <10,000. Kappa statistics were considered poor if <0, slight if 0–0.20, fair if 0.21–0.40, moderate if 0.41–0.60, substantial if 0.61–0.80, and almost perfect if 0.81–1.00.[28] To quantify the magnitude of the error we calculated the relative difference in the number of antibiotics per physician as (Xp-ODB)/ODB x100; where Xp is the number of antibiotics per physician in Xponent and ODB is the number of antibiotics per physician in the ODB database.

We evaluated the total antibiotic prescribing rate (number of antibiotic prescriptions per 100 total prescriptions) using Bland-Altman plots.[29] Bland-Altman plots allow one to visualize the error between two data sources by plotting the difference of the values on the Y-axis (Xponent-ODB) against the mean of the values on the X-axis ((Xponent+ODB)/2), along with plots of the two standard deviation (SD) limits above and below the mean difference. We performed this separately for male and female patients. IQVIA data does not contain numbers of patient visits, therefore we assessed the correlation of the antibiotic prescribing rate (number of antibiotic prescriptions per 100 total prescriptions) to number of antibiotic prescriptions per 100 patient visits using Spearman correlation coefficients.

To validate the antibiotic duration data, we defined antibiotic prescription durations that were >8 days as prolonged.[10] Similar to the above analysis we organized physicians into quartiles of proportion of prolonged durations and calculated performance characteristics, weighted kappa statistics, and Bland-Altman plots as described above for all physicians and primary care physicians only. Results were reported in accordance with guidelines for validation studies with administrative data.[30]

Ethics

This project has Ethics Research Board approval from Public Health Ontario 2017–032.02.

Results

The Xponent database had 25,678 unique physicians and 24,878 (97%) were successfully linked to ODB at ICES using College of Physicians and Surgeons identifiers. After applying the study exclusion criteria 14,671 were excluded for prescribing <9 antibiotics, 919 for <200 patient visits, and 16 for illogical prescribing rates, leaving 9,272 physicians included in this validation analysis (Fig 1).

Physician characteristics

These physicians were mostly male (68%) and 83% were primary care physicians. The majority of these physicians saw ≥20 patients daily, 90% worked in an urban area, and 34% were not Canadian trained (Table 1).

Performance characteristics

The Xponent database has a specificity of 92.4% (95%CI 92.0%-92.8%) and PPV of 77.2% (95%CI 76.0%-78.4%) for correctly identifying the top 25^th percentile of physicians by antibiotic prescribing volume. The performance measures of correctly identifying the top 25^th percentile of prolonged duration prescribers was slightly better with a specificity of 94.0% (95%CI 93.6%-94.4%) and PPV of 82.0% (95%CI 80.8%-83.0%). There was little difference when the cohort was limited to only primary care physicians (Table 2). The sensitivity analysis demonstrated that the magnitude of the misclassification was relatively small. In the primary analysis 77% of physicians were correctly classified in Xponent as top 25^th percentile prescribers, however 85% were at least within the top 30^th percentile, and 94% in the top 40^th percentile of ODB prescribers (Fig 2).

Agreement

We descriptively compared the mean number of antibiotic prescriptions per physician and found them to be similar with a relative difference of -0.4% and 2.7% for female and male patients, respectively. The error was greater in rural areas with a relative difference of -8.4% and -5.6% per physician for female and male patients, respectively. There were some notable differences between antibiotic subclasses, particularly for trimethoprim and/or sulfamethoxazole (Table 3). Overall, the weighted kappa for quartile agreement between Xponent and ODB was substantial at 0.68 (95%CI 0.67–0.69) for both female and male patients.

We constructed Bland-Altman plots to visualize and quantify the agreement in antibiotic prescribing rate (number of antibiotic prescriptions per 100 total prescriptions) between Xponent and ODB (Fig 3). These databases had high agreement for this variable with a mean difference (Xponent rate minus ODB rate) of -0.5 antibiotics per 100 total prescriptions for female patients. Meaning, that on average the antibiotic rate was 0.5 antibiotics per 100 total prescriptions higher in ODB than Xponent. However there were substantial outliers (mean-2SD = -16.4 to mean+2SD 15.4). Results were similar for male patients (S1 Fig).

The proportion of prolonged duration antibiotic prescriptions had overall substantial agreement as well between the data sources with more error observed in rural areas (Table 4). On the Bland-Altman plot the mean difference (Xponent-ODB) was -0.3 prolonged antibiotic prescriptions per 100 antibiotic prescriptions. Meaning, that on average the proportion of prolonged antibiotic duration prescriptions was 0.3% higher in ODB than Xponent. Similarly, substantial outliers were observed (mean-2SD -17.8 to mean+2SD 17.1) (Fig 4 and S2 Fig).

Physician ranking according to antibiotic prescriptions per 100 total prescriptions in Xponent was strongly correlated to the number of antibiotic prescriptions per 100 patient visits in ODB data for primary care physicians treating female (Spearman coefficient = 0.93 p<0.001, Fig 5) and male (Spearman coefficient = 0.93, p<0.001, S3 Fig) patients.

Discussion

Accurately monitoring population antibiotic use is a critical component of antimicrobial stewardship. We have validated that the IQVIA Xponent antibiotic database reliably identified high physician prescribers for patients 65 years of age and older. We identified that 23% of high antibiotic prescribing physicians were misclassified as being in the highest quartile of antibiotic prescribing volume, however only 6% of these physicians fell outside of the top 40% of prescribers. Furthermore, Xponent accurately captured the average number of antibiotic prescriptions per physician and the proportion of prolonged duration prescriptions, defined as >8 days. However, we identified an important limitation in the data with larger errors noted for physicians practicing in rural locations. This is most likely a reflection of a greater reliance on IQVIA’s projection algorithm as they collect less prescription data directly from pharmacies in rural locations. This error was slightly more pronounced in female patients. Further exploration regarding differences in data validity between male and female patients may be warranted.

Tan et al demonstrated the validity of hospital antibiotic purchasing data from IQVIA as a reliable metric of antibiotic utilization compared to internal pharmacy records.[31] Abstracts validating IQVIA antibiotic databases in other Canadian jurisdictions, performed with patients of all ages, have been previously presented.[32,33] Lin et al demonstrated substantial variability between different databases in the U.S. for estimating antibiotic use, including a database from IMS (renamed as IQVIA), highlighting the importance of reliable data sources for antibiotic use.[12] Both Canada and the U.S. rely on IQVIA for monitoring antibiotic use trends as well as assessing variability between regions and physicians.[7,13,15,24]

This study provides reassurance that IQVIA measures of total antibiotic prescribing in Xponent by physicians is valid. Furthermore, these results can assist public health departments, researchers, and policy makers towards appropriate uses of the data. We have demonstrated reasonable reliability of Xponent to identify high antibiotic prescribing physicians as well as reliably identifying physicians who prescribe prolonged antibiotic durations. Xponent is also quite reliable in estimating the number of antibiotic prescriptions per physician for both male and female patients. However, we caution the reliance on this database in rural areas or geographical locations where the data relies more heavily on the IQVIA projection algorithm. We demonstrated that antibiotic prescriptions per 100 total prescriptions is imperfect, but is highly correlated with antibiotic prescriptions per 100 patient visits for primary care physicians in patients 65 years of age or older. This is a potentially useful metric from pharmacy data to indirectly account for differences in patient volume, when patient visits are unavailable.

This study has some limitations. While we demonstrated that the misclassification of prescribers in the top 25^th percentile was small, this error may be important to clinicians receiving feedback on their prescribing. Certain subclasses of antibiotics were less well correlated between IQVIA and ODB databases, particularly trimethoprim and/or sulfamethoxazole and nitrofurantoin. One possible explanation for this could be the differences between the databases in defining repeats as these drugs are frequently used for prophylaxis for urinary tract infections in seniors. The IQVIA database utilized an explicit field from pharmacy data to denote refill prescriptions. No such field exists in ODB, and our study definition of repeats may have resulted in some differences in counting new antibiotic prescriptions. In addition, tetracyclines appeared to be overestimated in Xponent, however it is more likely that doxycycline was underestimated in ODB given that it was one of the only antibiotic treatments not funded by ODB during the study period. ODB does not fund First Nations populations with Non-insured Health Benefits, however this represents a small proportion of the included patient population. In Ontario, we only have complete population pharmacy data for patients 65 years of age or older. As a result validation studies may need to be conducted in other jurisdictions to determine whether our findings are generalizable to younger patient age groups. Non-physician prescribers are not included in either ODB or Xponent, and other databases are needed to study these important antibiotic prescribing populations.

In conclusion, we have validated the IQVIA Xponent antibiotic database to identify high prescribing physicians for patients 65 years of age and older, and identified some important limitations. Collecting accurate population-based antibiotic use data will remain vital to global efforts to combat rising rates of antimicrobial resistance. Governments and public health organizations should prioritize the need for accurate, population-based antimicrobial use datasets. An understanding of the uses and limitations of available databases are crucial for sound research and public policy decisions.

Supporting information

S1 Table [docx]
Oral outpatient antibiotic drugs included in the various antibiotic classes.

S1 Fig [docx]
Bland-Altman plot from 9,272 physicians comparing the antibiotic rates (number of antibiotics prescribed per 100 total medications prescribed) between Xponent and Ontario Drug Benefit (ODB) for male patients only.

S2 Fig [docx]
Bland-Altman plot from 9,272 physicians comparing the proportion of prolonged antibiotic duration (defined as >8 days) between Xponent and ODB for male patients only.

S3 Fig [docx]
Correlation of antibiotic prescriptions per 100 total prescriptions in Xponent compared to antibiotic prescriptions per 100 patient visits in the Ontario Drug Benefit (ODB) database for male patients by primary care physicians.