Skip to content

Tell us the Scores on the Doors

April 15, 2014

Last week was both a good one and a bad one for evidence-based medicine. It was good because thanks to determined and dogged scientific sleuthing, we finally found out that, to quote Ben Goldacre in The Guardian, ‘Tamiflu doesn’t work so well after all’. It was bad because it was now apparent that for years, medical decisions about the effectiveness of Tamiflu (and millions of pounds of government expenditure) were based on data that was seriously flawed. Scandalous? Indeed. But the Tamiflu saga also raises wider questions. How do we know what we know? And how much of what we think we know is really just stuff that we take on trust?

The revelations about Tamiflu were brought to light by global not-for-profit ‘independent network of health practitioners, researchers, and patient advocates’ the Cochrane Collaboration. It’s a name already familiar to nurses: in February the Collaboration published an intervention review of risk assessment tools for the prevention of pressure ulcers. Their conclusion? ‘There is no Randomised Control Trial evidence to suggest that undertaking structured pressure ulcer risk assessment reduces the risk of pressure ulcers’. So all those Waterlow scores you’ve spent hours of your life totting up? They might have prevented a few pressure sores; but there’s bugger all to prove it.

A different kind of nursing evidence has also been in the spotlight recently. On 1st April, NHS England and the Care Quality Commission jointly published guidance on ‘the delivery of the Hard Truths* commitments associated with publishing staffing data [about] nursing, midwifery and care staff levels’. It didn’t propose any new initiatives, but it did impose a timetable: by June at the latest Trusts should ‘clearly display information about the nurses, midwives and care staff present and planned in each clinical setting on each shift. This should be visible, clear and accurate, and it should include the full range of patient care support staff (HCA and band 4 staff) available in the area during each shift’ (my italics). Action on staffing! That’s good isn’t it?

Of course, any focus on staffing is to be welcomed. But there is a problem, and it’s a big one: data showing numbers of staff present on any given shift will only have meaning if we can be confident that we are comparing them with a planned number of staff that really is safe. But can we? Not everyone thinks so. In last week’s Nursing Times Jane Ball, deputy director of the National Nursing Research Unit, was quoted as saying that there is ‘a lack of systematic research into the effectiveness of tools used to work out optimum staffing levels’. Ominously, this is the same Jane Ball who, together with Professor Peter Griffiths of Southampton University, was last year awarded a contract by the National Institute of Clinical Excellence (NICE) to conduct a literature review on exactly this subject. We are talking about somebody who is exceptionally well informed.

The fact that there are – as last November’s How to Ensure the Right People…guidance on NHS staffing capacity and capability – airily concedes ‘a range and variety of tools [to calculate optimum staffing levels] available for use at present’ is not itself cause for alarm. But it is concerning that ‘in the longer term, NICE will be reviewing the evidence base and accrediting tools in this area’. How many of these things does it envisage ending up with for each nursing environment?  And how will comparability be achieved across the NHS if Trusts are free to choose whichever one they fancy? It’s also unclear how NICE will ‘review the evidence’ if there is no evidence to review. To illustrate this, let’s take a look at one of the current tools in more detail.

Anointed with the seal of approval in How to Ensure the Right People…(where it was deployed as a case study), the Safer Nursing Care Tool ( SNCT) was formulated by the elite Shelford Group. This is an organisation that in its own words, ‘comprises ten NHS multi-specialty (sic) academic healthcare organisations’ whose aim is in part ‘to benchmark and share best practice…and constructively engage with Government, Parliament and industry to represent the interests of large tertiary centres and the wider National Health Service’.

According to the Safer Nursing Care Tool Implementation Resource Pack, SNCT ‘can be used to used to assist Chief Nurses to determine optimal nurse staffing’ for ‘the outcomes we aim to achieve’. Trusts can use it for free (a plus point – no public money lining the pockets of private software developers) and its approach is common-sense: it divides patients into of five levels of dependency and then applies a multiplier to calculate how many staff are needed for any given mix of dependency levels. If you don’t understand this, I strongly advise you take a look at the online version of the SNCT Implementation Resource Pack, which explains it much better than I can.

But the problems are obvious. Firstly, leaving aside the fact that the dependency categories are based on a Department of Health classification from fourteen years ago – a long time in health care – the whole approach is heavily task-orientated. There’s no attempt to unpack the complexity of nursing work. There are several places where this stands out. Take patients at Level 0 – the least demanding level whose ‘needs are met by the provision of normal ward cares’ (whatever that means). These patients may require fluid management – but what if the cannula has come out,  they’ve got poor veins, no nurses have been cannulation assessed and doctor can’t be located despite repeated phone calls over several hours…it happens. And why is ‘Patient and/or carers requiring enhanced psychological support owing to poor disease prognosis or clinical outcome’ only included as a potential interaction at one of the dependency levels (1b)? I could go on.

Secondly, it’s vague. Although ‘the multipliers account for normal patient-flow levels’ and the Resource Pack accepts that ‘when there is a high throughput of patients…additional staffing uplift may be considered appropriate’, there is no definition of what constitutes ‘normal’. Does the tool recognise how much paperwork is involved in each and every admission? On the information available here – we don’t know. It’s also noticeable that the tool deals in crude numbers of staff only; not skill mix. Presumably that’s left to individual employers – and budgets – to decide.

Thirdly, the Resource Pack offers no peer-reviewed evidence that the using the tool results in safer care. Neither does a Google search for ‘Safer Nursing Care Tool Evidence’. I’m not saying the evidence doesn’t exist – but if it does, why not make more accessible? For optimum results, the Resource Pack advocates triangulating the patient dependency scores with ‘quality indicators that can be linked to nurse staffing issues’ – suggested indicators include numbers of falls and hospital-acquired pressure sores. It’s perfectly sensible,  but how do we separate out the effects of staffing from other variables? To give just one example, supporters of now widely-implemented Intentional Rounding have made very grandiose claims about the positive effects of their particular hobby-horse on these self-same indicators.

So is it all just one big waste of time? Actually I don’t think so. We have to start somewhere, and any information about staffing is better than none. But we also have to acknowledge the limitations of what we currently have and try to do better. Fast. If not, all the data collection and ‘scores on the doors’ will simply be yet another burdensome exercise in futility. And for every relative who meekly returns to the bedside to wait when it’s pointed out that the ward is ‘officially’ short-staffed, there’ll plenty more who clench their fists, stand their ground and say “Listen love, I don’t care if every nurse in the entire bloody hospital has just died of bubonic plague. My mother needs the toilet NOW”.

* The government’s response to the Francis Report

For the Ben Goldacre article on Tamiflu, see:

For the SNCT Implementation Guide, see:

For How to Ensure the Right Staff are in the Right Place at the Right Time, see:

For a summary of the UK Cochrane Collaboration’s evaluation of Risk Assessment tools for the prevention of pressure ulcers, see:

  1. Elaine Maxwell permalink

    Great blog about the challenges. In my opinion the difficulty is that we are barking up the wrong tree looking for the magic number.
    We know that context plays a huge part in any intervention and therefore we should be modelling real time data about the number and skills of staff and the patient outcomes, mortality, 30 readmissions, harms and satisfaction on a given clinical environment. We have the technology and the data to do this now but not the imagination!
    This would both assure the quality and provide a predictive model for that clinical environment. As we know from Quality Improvement; measuring for improvement requires you to measure yourself against yourself rather than benchmark against others.
    Until we accept that healthcare environments are complex and we can not control from the variation thus comparative benchmarking is pointless, we are competing wasting our time and will continue to get inappropriate staffing and poor quality care

  2. Thank you for your comments and for putting forward a very interesting alternative paradigm. It’s salutary to be reminded of the extent to which we have unquestioningly absorbed the rhetoric about benchmarking and competition in health care and failed to consider other possibilities.
    The tools that we have at the moment all seem very ‘clunky’ compared to what should be possible with modern technology. I was very surprised that the ‘How to…’ document mentioned the GRASP system as a possible contender. I remember that from the late 1980s. Surely things have moved on since then?
    There seems to have been surprisingly little examination, at the granular level, of what nurses really spend their time doing, and what prevents them from doing the things they feel they should be doing. It may well be something other than what managers think it is. My example about going round in circles trying to find the right doctor to cannulate/write up discharge medication/correct incorrectly prescribed medication is not uncommon – but is largely unrecognised.
    My feeling is that patient acuity – though obviously relevant – is only part of the story, and staffing calculators that have no mechanism to capture the complexity of nursing activity are doomed to failure. On the other hand, one is tempted to wonder how much NHS England really wants to hear about ‘complexity’. A culture that is focussed on targets and league-tables is of necessity reductionist. The ‘scores on the doors’ will tell us something – but whether they tell us anything really useful is another matter entirely. If they foster a false sense of security, they could be positively dangerous.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: