Epocrates announced this week that they were removing Bugs + Drugs, an app we found major flaws with in our review in 2013, from the App Store.
The app was initially released in 2013 and was generally greeted enthusiastically. In a response to the iMedicalApps review, Athena Health reported that 90,000 users had downloaded the app within two months release. And it made sense – the idea was innovative and had a lot of potential.
Athena Health had access to 40 million patient records in the cloud-based network, including a ton of data on microbial sensitivity patterns from bacterial cultures of all kinds. As they described it,
Bugs + Drugs geo-locates your Apple® iOS 7 device and delivers a rank-ordered list of bacteria found in urine, blood, or skin specimens predominant in your selected area. Depending upon the location you’ve selected, the radius dynamically expands to deliver meaningful results for that area. Tap on a bacterium to reveal sensitivity data by drug…Thousands of bacteria-to-drug data points tied to patient-level ZIP codes are available each day, enabling automatic continual app updates.
Unfortunately, there were serious problems with implementation as highlighted by a former iMedicalApps Editor & Assistant Professor at the Massachusetts College of Pharmacy and Health Sciences (MCPHS). For example, there were numerous outright errors in the initial release, such as listing pseudomonas as 100% sensitive to ertapenem or saying candida albicans is 100% sensitive to vancomyicin.
And when we looked at antibiograms from two hospitals, the sensitivities being listed for the same bugs was very different than what Bugs & Drugs gave. That appears, in part, to be related to the larger number of isolates used for most bugs by the hospital antibiograms vs. Bugs & Drugs.
There were other issues with the app that we highlighted, some related to content accuracy and others to design, that I won’t rehash here. And the Athena Health team gave a detailed response to our review soon after it was published. This week, they announced that they are removing the app from the market for unspecified reasons.
In retrospect, this entire exercise should remind us of an important fact – developing apps for healthcare is not a scenario where being first is better than being best. In their post announcing the app, the team at Athena Health went out of their way to highlight the simplicity of the app’s design. In fact, they noted they went from the idea to working prototype in two weeks.
What is now striking in its absence are comments regarding any sort of accuracy testing, peer review for quality assurance, or post-marketing monitoring before bringing this app to market. And while there were improvements made early on, it appears that enthusiasm & support for further development waned at Athena Health – the last update to the app was in January 2014 according to the cached iTunes page).
As they pointed out, the tool they sought to build was in many ways the first of its kind. That also means that it wasn’t entirely possible to double check the information being provided unless it was egregiously wrong; while you could look at general references like the Hopkins Guide, the entire pitch of this app was that it was more personalized than generic references. And with a trusted label like Epocrates on the cover, it’s entirely possible that “less wrong” information could be acted on. This was an app where it was crucial to get it mostly right from out of the gate.
Nonetheless, I hope this isn’t the end of this endeavor, whether by Athena Health or someone else. Despite its flaws, it was a great idea of how to leverage an enormous dataset to deliver actionable information to clinicians at the point of care. With some more thoughtful premarket testing and evaluation, it could be a really powerful tool.