A new comer on the market that is creating some buzz is MedSnap. The company brings a novel product that is capable of identifying medications with the camera equipped on the iPhone. As a pharmacist, I have readily been amazed at this product and the ramifications it could pose in care and its uses by both patients and health care professionals. One of the co-founders, Dr. Patrick Hymel answered a few questions we posed to him about MedSnap. Dr. Hymel background is that he was trained as an emergency physician who went on to work in medical technologies. His previous company was MedMined, Inc., which was sold to Cardinal Health in 2006.

Could you give a little introduction about how you came up with MedSnap? Did you past experience as a practitioner or family experience play any role?

Patrick Hymel: My experience as an Emergency Medicine physician made me familiar with the “unknown bag of patient pills” problem for ED staff. In the past I spent time with patients going through a set of unknown pills and asking them what each one treats. Then two years ago my grandfather took his cancer medications incorrectly for several weeks due to a prescribing error. It hit home how harmful and hard to diagnose errors in prescribing or dispensing could be. Shortly after this, over lunch with MedSnap’s co-founder, Dr. Stephen Brossette, I shared my experience and he relayed a similar personal story. Before the end of lunch MedSnap was born.


What work went into the development of this product?

Patrick Hymel: Identifying pills with great accuracy using computer vision is a very difficult problem – and despite our past start-up experience we underestimated just how difficult. We succeeded because we were able to build a world-class development team from members of our previous ventures who are as stubborn as we are. The three main problems we solved were pill segmentation, identification, and reference data management. Just solving segmentation – which required the development of the MedSnap Snap Surface – took twice as much development time as the core technology for our first successful start up. Identification of pills required performing color correction for ambient lighting and creating new methods for imprint analysis. Our internal Visual Pill Library management tools are highly sophisticated and allow an internal group of experts to review submissions of new pills from the external “crowd”. We use nearly 1 million pill images to build the statistical model that the algorithms use to identify pills. Our crowdsourcing model separates the external crowd from the reference database – two different “Snap Lab” technicians review all submissions before they are allowed into the Visual Pill Library.

What would you say have been the greatest challenges to making MedSnap work?

Patrick Hymel:The FDA and drug manufacturers do not make our life easy. FDA mandates that U.S. prescription pills have a unique side (by size, shape, color and/or imprint), but pills can vary in one or more dimension by batch – so our system has to track what we refer to as “variants” of a particular pill. An NDC code does not uniquely identify a pill’s appearance. Also, manufacturers will re-use components of other pills – the same logo imprint, pill shape/size/color or even capsule to make an entirely different medication. In the image attached, the first capsule treats depression and the second treats ADHD – yet they are in identical blue/green capsules. The only difference is the imprint. So MedSnap ID must read the characters to distinguish these, and if the capsule is positioned so it cannot read them, it will declare the appearance “ambiguous”, inform the user that it is non-unique and allow them to select the correct one from a “visually similar” listing. The second example is even more dangerous – these yellow pills vary only by 1mm in diameter – very difficult for a human clinician or patient to differentiate, but fortunately a chip shot for MedSnap ID.

Medication Comparison

What environments do you want to expand MedSnap into? How will clinicians and patients utilize MedSnap?

Patrick Hymel:We’ve had tremendous interest from community pharmacies, home health, transitional care, pre-op screening, and the Emergency Department. Medication therapy management (MTM) can be beneficial to patients, but difficult to provide efficiently at a retail pharmacy counter – MedSnap ID can save pharmacists time and make MTM much more efficient. On each home health visit the nurse must review and document the patient’s current medications – often this may have changed if a doctors visit has occurred since the last assessment. MedSnap ID can shorten the time required for medication review plus securely transmit the data to the home health office for billing or QA purposes. In the Emergency Department many clients find that by asking the patient to place the pills they actually take on the Snap Surface when the patient is placed in the exam room they get a better, faster medication history – plus a “mini mental status exam” of the patient’s ability to manage their home medications.

How will MedSnap adjust for future mobile devices? Will it come to Android or other operating systems? Will it incorporate eventually with a platform such as Google Glass?

Patrick Hymel:The iOS platform has been excellent to develop for – we’re asking a lot of the camera, processor and OS. Our identification algorithms run entirely on the phone – not in a cloud. We’re looking at other platforms but do not have immediate plans for another OS. Google Glass would present problems due to the angle of the camera relative to the pills that would cause a number of problems – you would have to look straight down at the pills. But yes, that would be cool.