Case Studies: Uses and LimitationsSeptember 14, 2008
Earlier in the month, the health informaticist drew attention  to a post by marketeer Mark Earls on the dangers of reliance on case studies to make a point .
Earls argues that “case studies are bad at establishing truth” because:
1. Case study thinking excludes failures, the grey mush of moderate success or indifferent performance …
2. Case studies make it seem as if the success was inevitable …
3. Case studies force things into an oversimplified narrative arc … 
Personally, I’m not convinced by either of the first two points – or at least I’d see them as subsets of point three – in describing a single case study, it is easy to include in our narrative only the highlights (corresponding to Earls’ first point, perhaps) and misattributions of success to one or more factors we wish to highlight without discussing the wider circumstances (Earls’ point 2?).
And, as Lovell points out in his brief post, point three can be either a positive or a negative, depending on circumstance:
[Case studies] are nice for telling stories (invites the listener to place themselves in the story and imagine what they might do …), though while they add flavour and realism they don’t prove that the story is true. And secondly they can generate hypotheses worthy of further consideration: “Oh gosh, look at that, hmmm… interesting, I wonder if…” – that sort of thing. 
Earls’ ends his post by asking “What’s the data say?”  In an area as fraught with discussion as the hierarchy of evidence, we can’t reduce it down to simplistic data analysis, but a quick survey of what’s out there did turn up a clean summary of features of the different levels of the hierarchy in a paper by Daly et al :
Single case study (level IV) – Features: Provides rich data on the views or experiences of one person. Can provide insights in unexplained contexts. Limitations: Does not analyze applicability to other contexts. Evidence for practice: Alerts practitioners to the existence of an unusual phenomenon.
Descriptive studies (level III) – Features: Sample selected to illustrate practical rather than theoretical issues. Record a range of illustrative quotes including themes from the accounts of “many”, “most” or “some” study participants. Limitations: Do not report full range of responses. Sample not diversified to analyze how or why differences occur. Evidence for practice: Demonstrate that a phenomenon exists in a defined group. Identify practice issues for further consideration. 
After discussing the top two levels in the hierarchy (“conceptual studies (level II)” and “generalizable studies (level III)”), the authors conclude that “not all research can reach this [ideal] standard” and go on to assert that “Understanding and evaluating claims to knowledge made by qualitative research is important in meeting the policy and practice needs of an increasing[ly] complex health environment.” 
So, perhaps don’t throw the baby out with the bathwater by abandoning the case study altogether … but use it as a springboard to further research (or an argument for further research funding) and, whether assessing the authority of a medical paper or sitting on the sofa watching a TV ad, be aware of the level of evidence with which we are presented before making our decisions …
 Ibid: 46.
 Ibid: 48.