My mission statement

The times we are working in now need a great deal of accelerated change and there must be no negotiating that down. So my mission statement for this part of my consultancy career is to be clear that there needs to be and will be a lot of change from the work that I do with individuals and organisations and if organisations don’t want that, then it is probably best to go somewhere else.

Read my statement in full »

If we can’t measure it, we can’t know for certain how we’re doing

Filed Under (Health Improvement, Public Health) by Paul on 28-03-2011

If you want to measure the improvement of the NHS against improvements in health outcomes it will be necessary to collect the data that show what those outcomes are.

One of the few consistencies of approach to which the new Government has adhered in its NHS reforms has been an emphasis on encouraging health services to judge their success or failure against the health outcomes that different services produce.

The current NHS system has some elements that think in terms of health outcomes – such as the value for money analyses of drugs carried out by NICE – but mostly it does not judge its economic effectiveness through an understanding of the specific health outcomes that are gained from the specific services it provides.

And this is not an easy task. It’s relatively simple to move from inputs (what we are going to do and how often) to outputs (did we succeed in making anything happen), but to move to outcomes requires understanding how the public live their lives differently as a result of the services provided.

As an example of a health related area, school dinners provide the best example I have known for understanding the success or failure of public services in these three different ways.

Inputs: Back in the mid 1990s a local authority would judge how good their school dinners were by the amount of money that was spent on each dinner. Local Authority x would boast that they spent three time as much on the dinner as their neighbouring authority. This was obviously a judgement based on input.

Outputs: Some local authorities recognised that they could spend a lot of money on a meal but the nutritional value might be low. So those interested in outputs put a lot of effort in to understanding the nutritional content of school dinners. This enabled the judgment to be made that meals with more nutritional content were a better output, whatever the cost of the meal.

Outcomes Some then, rather cruelly, pointed out that you could put a really high nutritional value meal in front of a child but if they did not eat it because it was the sort of food they didn’t like, then there was a problem. This might mean that a meal with a very high nutritional output value could have a zero outcome value in terms of the nutrition that the child actually consumed.

The difficult thing about outcomes is that the only way a public service gets to an outcome is by involving the public in its delivery. The meal may be nutritious in its own right, but the pupil has to eat it for an outcome to occur. The smoking cessation service may be delivered at the right time and place, and with the right people in the room, but the people themselves have to do the hard work of giving up smoking.

Measuring outcomes can therefore leave public services feeling that being judged by them is very unfair. Public servants can feel that they do everything they should but are wrongly judged by something they can’t control. But that brings into question why the public should pay for this service.

If they are paying for a service to develop intrinsically good outputs that have no impact on the public then it is good value for money. If on the other hand they are paying for this service to change the public in some way, the outcomes are vital.

  • So the amount of the NHS drugs budget is an input.
  • The outputs are the millions of prescriptions and consequent drugs that end up in people’s bathroom cabinets.
  • The outcomes are getting those drugs from the bathroom cabinets into the people who are meant to be taking them.

We can improve the efficiency of the first by hard bargaining with the drugs companies.

We can improve the efficiency of the second by better medicines management.

We can improve the efficiency of the third by involving the public much more in the whole process the prescribing of their drugs.

I won’t give a prize for guessing which of these three is the most inefficient at the moment.

And I won’t give a prize for guessing on which of those three insufficient work is being carried out.

This is what makes any public service very brave to judge itself by an outcome – not only do people get better but do they lead fuller and healthier lives as a result of the health care intervention.

But, correctly, this is how the Government wants to have its NHS reforms judged. That leaves us the question of determining how we will know if the health outcomes have improved? And the answer has to be to ask the public.

This makes last week’s announcement from the NHS Information Centre – that they will not be funding the General Lifestyle Survey any more – a real political problem for the Government. This survey is carried out every year by the Office for National Statistics(ONS) on behalf of a number of government departments. It provides information on a wide variety of topics, including smoking and drinking habits across the UK.

Each year, around 15,000 households are contacted and face-to-face interviews are carried out, questioning respondents on their smoking and drinking habits and their use of health services, as well as looking at the issues of housing, employment, education and income.

Once collated, the survey’s results provide government departments with a valuable insight into the lives of UK residents.  It has been running almost continuously since 1971, enabling experts to spot emerging trends and cyclical patterns.

But now the NHS Information Centre has decided to cut its contribution of £300,000 that funds part of the work of the ONS.

Those involved in understanding health outcomes will find this decision puzzling and will be pretty angry. Martin McKee from the London School of Hygiene and Tropical Medicine was clear:

This decision exposes the hypocrisy of a government that claims to promote public health yet enters into agreements with the food and alcohol industry that ignore the evidence on what really works – and now makes it impossible to know what the results of its misguided policies actually are.”

As with so many other aspects of Government health policy, because the Government has not been able to explain what is going on here, commentators have provided their own explanations. In this case the narrative that has attached to Government reasons for cutting the survey is to stop the public from finding out that health service outcomes may be getting worse.

Such an explanation results from the Government’s own commitment to outcomes as a measure of success or failure.

It was, after all, they who said this was important.

Comments:

2 Responses to “If we can’t measure it, we can’t know for certain how we’re doing”


  1. Surely the ‘outcome’ of drug prescription isn’t getting the medications from the bathroom cabinet into people, but is getting those medications into people that give them a better quality of life?


  2. Both the opposition and the government have a problem caused by their anti-management (or bureaucrat) rhetoric. While it is clear that you can please the public by promising to cull the bureaucrats (even Stalin did this regularly!), if you believe your own rhetoric you make bad decisions on where to spend the budget.

    We need some people to count up the numbers, organise the staff and to measure outcomes or we can’t spend any of the budget well. The NHS was undermanaged before the government set ridiculous management cost targets for the GP commissioners. It is probably part of the same cost cutting drive that the IC has decided to cut back on the lifestyle survey.

    It is not an easy sell to a skeptical public, but the NHS might actually save money overall if it spent more on managers, especially the ones who analyse data about outcomes.

Leave a Reply