The Sport Concussion Assessment Tool 2 (SCAT2) app provides a detailed, standardized method for evaluating injured individuals for possible concussions. It was developed based on the deliberations at the 3rd International Conference on Concussion in Sport held in Zurich, 2008.
It contains all the essentials you would want in a concussion app: a graded symptoms checklist, cognitive testing, balance testing, Glasgow coma scale, Maddocks score, baseline score ability, serial evaluation, and password protected information-sharing via email. The SCAT2 was originally in paper format but was later converted to an app in order to provide health care professionals with a paperless way of managing and tracking their potentially concussed patients.
The SCAT2 app was developed by Inovapp, which also has a NHL SCAT2 app — an app that describes itself as being modified specifically for use by members of the National Hockey League. The SCAT2 app specifically states how it is meant to be used by “medical and health professionals”.
So how does a concussion app that was seemingly developed in conjunction with the Zurich Consensus Working Group–a collection of the leading experts on anything-and-everything concussion-related–go from being widely used among healthcare professionals to getting a rating of 1 out of 5 stars (22 reviews) in the iTunes App Store?
Well it’s simple, you make your app non-operational, without any warning or replacement.
Here are some of the reviews you can find on the iTunes App Store:
“…I went in with a patient 30 minutes ago to do a follow up eval on the patient who received a concussion last week and the app will not work. If you were going to turn it off, you should have notified every person ahead of time so all of us could have enough time to plan accordingly. All the reviews on iTunes states that you guys turned it off. I know scat 3 just came out, but for patients that have current concussions, I need to be able to see the changes in the testing. Until you have a scat 3 app in place, this scat 2 app MUST be turned back. Dr. C D.C, CCSP “
“…As a healthcare professional working in high school this tool was amazing for communicating with other high schools that had healthcare professionals that needed to communicate about athletes with head injuries. The update should of been to SCAT3 rather than taking away a vital tool. Trying to administer paper-based test on the football sideline in the rain is not practical. Doing it on a phone with a waterproof case is very practical.”
“As an athletic trainer working with men’s ice hockey, this app was useful on the road. I could easily administer the test and have it emailed to me so I could print it when we got home. I wish I was warned that the app was going to be useless after the update because I really needed it during our last road game and didn’t have a paper copy of the test. Please update it to the scat3 test!!!”
“I needed to use this last week when a visiting baseball player had concussion signs. I had no idea it was no longer working. I had no paper forms with me at the time. Please update ASAP!!”
In November 2012, the 4th International Conference on Concussion in Sport was held and with it a new consensus statement was published that was meant to supercede the 2008 recommendations. Since SCAT2 was based on the 2008 recommendations, an updated assessment tool, SCAT3, was developed.
A paper version of SCAT3 exists; however, there is currently no mobile app version. So now users of the SCAT2 app are greeted with the following message when they open the app – “Please note: This app has been replaced by the SCAT3 paper form”. The SCAT2 app is deactivated with no SCAT3 replacement in sight. As many of the comments mentioned, one of the most appealing aspects of SCAT2 was having the paperless option. If you are wondering what the differences are between SCAT2 & SCAT3, SCAT3 has added the following to its assessment survey:
- Indications for emergency management
- Slightly more extensive background section
- neck exam
- more detailed return-to-play instructions
Some tremendous takeaways from this blunder:
1) Mobile apps are definitely being used for concussion management. The comments alone show why they are so useful for concussion management. Multiple people have commented how having concussion apps enable them to forgo paper charting that is significantly more cumbersome when assessing athletes on the sidelines. Some of the examples cite how using a paper chart system to assess an athlete in the rain is difficult and that mobile concussion apps offer significantly more ease of use when traveling to away games.
2) There were multiple avenues the app developers could have taken in place of immediately deactivating the SCAT2 app. Since the differences between SCAT2 and SCAT3 are minimal, they could have left SCAT2 operational and then added the SCAT3 modifications as an update once they were ready. They also could have left SCAT2 operational until they had created a newly developed SCAT3 app. The least they could have done was to keep SCAT2 functional and to add a disclaimer on the home screen telling users the guidelines had been updated via the paper SCAT3. If and when the SCAT3 app will be developed remains to be speculated.
3) When certain healthcare and medical apps fail, it’s a bigger deal than others. When you’re in the hospital setting, and your Epocrates app (drug reference) isn’t working, it’s not a patient altering or treatment changing problem. There are several alternatives for drug reference apps — my personal preference being Micromedex. Further, you easily have access to online resources as well.
However, when certain apps fail, such as this popular concussion management app, which is based on a consensus statement from a group of international experts — it can have disastrous consequences. The multiple comments show how the management of concussions was stymied due to this app’s failure.
There are some obvious things that can be done to prevent this, such as always backing up an app’s information, whether it be in the cloud or keeping constant paper or e-mail copies — but that’s what mobile apps were supposed to help us with.
In general, the failure of this app should be disappointing to the mobile health community at large. It shows how a healthcare app based on evidence based guidelines developed by reputable health care professionals isn’t impervious to a catastrophic failure. Indeed, the phrase to describe what happened truly is “catastrophic failure”.
