mHealth Journal Club
Article: The relationship between commercial website ratings and traditional hospital performance measures in the USA
Authors: Naomi Bardach, Renee Asteria-Penaloza, W John Boscardin, R Adams Dudley
Journal: BMJ Quality and Safety
Date Published: November 23, 2012
Bardach NS, Asteria-peñaloza R, Boscardin WJ, Adams dudley R. The relationship between commercial website ratings and traditional hospital performance measures in the USA. BMJ Qual Saf. 2012.
The rise of social media has allowed users to have a greater impact through social review services on businesses. While often critical of restaurants and retail establishments, healthcare practitioners and networks are not immune. The interesting component of social review services is they are more well known compared to more official ranking sites of hospitals provided by CMS.
While many organizations may receive a rating, how well do they stand up to the official scores conducted via industry standards such as the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS)? For those unfamiliar with HCAHPS, it is a standardized survey measurement of patient satisfaction at a hospital based upon 18 patient perspectives on care across 8 topics.
The authors sought to determine the quality of information available to consumers as a correlation to social review service (i.e. Yelp.com) when compared to HCAHPS.
- Data sources
- Quality scores of hospitals was utilized based upon information available from HCAHPS related to mortality and readmission data. Scores are based on a scale of 0 to 10.
- Yelp.com data was utilized due to its free access (comparable websites require a membership or subscription which the authors felt may cause bias). Scores are based on 1 to 5 stars.
- Data was collected over a one week period in 2011.
- Hospital characteristics were collected from the American Hospital Association survey.
- Outcomes measured
- Correlation of rating of hospitals based upon HCAHPS versus Yelp.com
- Relationship of correlated scores on mortality and readmission rates
- 25% of the 3796 hospitals with HCAHPS data had a Yelp rating and were included in analysis.
- Of the 962 Hospitals with a Yelp score, only 270 had more than 5 Yelp rating.
- These institutions with >5 ratings were frequently correlated with large, urban, non-profit academic hospitals
- High Yelp scores correlated with hospitals that had lower readmission and mortality rates
- High Yelp and HCAHPS scores had a correlation of 0.49 (p<0.001)
The authors concluded that social reviews through Yelp may be similar to ratings conducted through HCAHPS.
This is a very interesting piece. On one side, there have been multiple posts and articles written on how a review on Yelp or similar sites can damage the reputation of a business. Even providers have noted poor reviews can hurt their patient volume. That being the case, should hospitals be concerned about their scores via social review websites?
On one hand, it would make sense to always be concerned about how the public perceives the hospital, but then it invites the nature of how these review services work and the barriers to drawing a clear conclusion that there is a direct relationship between Yelp Scores and their HCAHPS rating. For instance, there were very few scores overall given through Yelp, and even those were relegated towards urban hospitals. The population posting reviews on Yelp tend to be younger, while those conducting the HCAHPS ratings are older.
Additionally, the question becomes as to what happens if you throw in the other review websites, such as Angies List?
Lastly, while very novel, this study method of analysis was very exploratory and others may have had different cutoffs or standards that may have seen a completely different outcome.
Nonetheless, this is a great way to invite further research to determine if social media reviews should be taken seriously by hospitals in the future and if patients should rely on these scores more than they realized. However, several issues need to be addressed, namely, ensuring that scores are not hyperinflated or reduced due to false manipulation of reviews.
BMJ Quality and Safety