Speakers:
Prof. Jake Chapman - DEMOS
John Roberts - Research Fellow, ODI
Chair:

Simon Maxwell, Director ODI

1. The third meeting of the series on Targets, Voice and Choice was held in the ODI Meeting Room on Wednesday 3 November. The speakers were Prof. Jake Chapman from DEMOS, author of "Systems Failure", and John Roberts from ODI. The meeting was chaired by Simon Maxwell, Director of ODI

2. Jake Chapman presented first, speaking to a PowerPoint presentation. He started by stating some of the possible advantages of using targets in public sector management. These include (a) a shift of focus from inputs to outputs, (b) the need to collect useful performance data, (c) guidance on policy priorities, and (d) improved accountability.

3. Targets have been widely used in public sector management in the UK despite some of the problems that they cause. In fact, often there is a proliferation of targets due to the need to cover many important indicators that would be neglected if a target was not set for them. The first NHS Plan, for example, had over 400 targets.

4. Prof. Chapman went on to describe the contradictions inherent in the use of targets, by illustrating some striking examples. Not only is the collection of performance information burdensome and causes frustration among professionals, but also, when targets are not met, a sense of failure further de-motivates them. Distortions caused by targets are also very common. Prof. Chapman quoted cases from the education, health and crime sectors, highlighting how the narrow focus and universality of targets often caused unexpected negative consequences, distorting the incentives faced by service providers.

5. The main problem, according to Prof. Chapman, is not about targets themselves, but about the existing "scientific approach" to performance management, which regards organisations as machines, and people as cogs. The "machine metaphor", often used when talking about bureaucracies, is misleading and dangerous. A much better approach views organisations as complex and adaptive systems, with a much lower degree of predictability and control. Multiple perspectives need to be taken into account.

6. If targets become the focus of managing an organisation's activities, this will inevitably lead to a focus on operations rather than on the overall objectives and activities of the organisation, to gaming and cheating in order to give the appearance of having achieved targets, and will take attention away from sources of variation and qualitative aspects which are so important in public service provision. Issues of equity, fairness, sustainability and trust will also be overlooked.

7. A mechanistic view of public services and organisations is therefore inadequate and leads to negative consequences. Systems theory and public value approaches are much more suited to a debate on public services and public sector management. Organisations should declare their overall aims and objectives, and then enter into dialogue with providers and customers to discuss ways of achieving their stated results.

8. John Roberts also spoke to a PowerPoint presentation, summarising the results of a cross-country study on results-oriented budgeting in developing countries with Poverty Reduction Strategy Papers in place.

9. He said that setting targets is a useful way of focusing attention in the short-term. We can distinguish among global long-term targets (MDGs), national medium-term targets enshrined in development strategies and PRSPs, and short-term targets linked to donor accountability (managing aid for development results).

10. Results-oriented budgeting is about linking rhetoric to practice, and policies to programmes to expenditures. A performance management system has to establish a relationship between inputs to activities, activities and programme outputs, and between those and outcomes and policy objectives. What governments in developing countries need to do is state their overall policy objectives first, then set targets and performance indicators, and finally specify the existing relationships between inputs and outputs and between aid and public expenditure.

11. John Roberts indicated different ways of doing this: either through public sector reform measures which introduce agency performance plans, although the track record of such initiatives is not too encouraging, or through a system which links budget allocations to performance as stated in contracts between the Ministries of Finance and implementing agencies.

12. There is widespread experience on results-oriented budgeting in various developing countries, including Uganda, Tanzania, Ghana, and Bolivia, and some other very interesting examples in countries such as Thailand, Malaysia and South Africa.

13. Some of the main conclusions from the research are that Results-Oriented Expenditure Management (ROEM): (a) works better with strong Ministries of Finance, (b) is more effective where there is government ownership, (c) helps focus resources on policy priorities, (d) needs to have links to decentralisation policies, and (e) discourages corruption and promotes accountability.

14. To conclude, John Roberts said that it is indeed possible pragmatically to make progress in ROEM, also by linking it to PRS processes and PFM reform. Donors should promote lesson learning, participation and awareness, and help strengthen planning and evaluation function.

15. There was a brief discussion after the two presentations. Participants noted the basic difference in the conclusions of the two speakers. Jake Chapman argued against the use of targets, while John Roberts made a case for using them to focus on results in developing countries expenditure management systems. The discussion that ensued focused on two main issues.

16. The first related to the need to set context-specific indicators, both quantitative and qualitative, which allow for monitoring progress towards policy objectives flexibly, while moving away from setting inflexible quantitative targets which can distort incentives. In the case of international development, the chairman quoted Adrian Wood, DFID Chief Economist, as saying that "the MDGs should be taken seriously, but not literally". Some of the main challenges in this area include the need for an inclusive and participatory process to define the indicators to be used, and the use of all available evidence to inform policy-making. The main difficulty in applying this principle could be, for example, that of reconciling the management of a national health system with local negotiation of priorities and strategies.

17. The second main topic in the discussion was the comparison between the use and usefulness of targets in a developed country such as the UK and in developing countries. In some cases the issues at stake are quite different, with policies in developing countries focused on universality and reach of public service provision, rather than on more specific quality issues which tend to be more prominent in developed countries. However, there was a growing consensus on the need to use targets sensibly in both environments, taking into account some of their possible distortionary effects. Again, the issue of using different instruments alongside targets, such as voice and choice mechanisms, was seen as an avenue to address the identified shortcomings of a target-centred approach.

Paolo de Renzio

3 November 2004