Speakers

Thomas O'Brian - Lead Economist, Independant Evaluation Group, World Bank
Discussants
Andrew Steer - Director General, Policy and Research, DFID

Alison Evans - Director of Programmes, Poverty and Public Policy, ODI

Chair:

Vinod Thomas - Director General, Independent Evaluation Group, World Bank

Thomas O'Brien
Lead Economist, Independent Evaluation Group, World Bank

1. O’Brien presented the Annual Review of Development Effectiveness 2008: Shared Global Challenges. Two key purposes of this review are; first, to hold managers of the World Bank accountable; and second, to provide constructive criticism on how to improve the Bank’s contribution to development outcomes.

2. O’Brien organised his introduction around the two part presentation of the report.

• Part I tracked World Bank performance, notably trends in outcomes of its projects and country programmes, the evolution of monitoring and evaluation, and the role of evaluation in the results agenda.
• Part II examined the Bank’s work in fostering global public goods. He defined global public good as something that every one is affected by, but too few actors address it. For example protecting the earth’s climate and preventing the spread of dangerous communicable diseases.

3. Part 1: Tracking Bank performance towards achieving its goals of growth, poverty reduction, and environmental sustainability. IEG reviews all projects/programmes that have come to a close in the previous fiscal year, typically around 250. It asks whether these operations  have delivered in accordance with development objectives. The scale for assessment ranges from highly satisfactory to highly unsatisfactory. The Bank’s own target is that 75 per cent of its projects are moderately satisfactory or better. Over the three years to end-fiscal 2007, IEG’s evaluations confirm the Bank has exceeded this target and is at 80 per cent.

4. The share of the Bank’s lending that is meeting its development objectives has improved over the medium term. However, performance across sectors varies. Some sectors experienced a significant turnaround, for instance water supply and sanitation. Others, such as public sector governance have declined. All regions have exhibited improved performance, in particular sub-Saharan Africa where the improvement has been most significant - albeit from a lower base.

5. Despite a strong medium term trend, project performance has dropped significantly this past year.  This is not a cause for alarm, rather a reason to be vigilant. Five factors are significant in influencing the recent deterioration in project outcomes, these are: overly complex projects, overly ambitious projects, delays in implementation as circumstances changed, weak results frameworks, and poor Bank performance (including weak supervision).

6. A notable finding in this years ARDE is the widening ‘disconnect’ between the World Bank’s own ratings of project performance during implementation and ex-post ratings found in self-evaluations and independent evaluations. The disconnect, which suggests a tendency towards over-optimism during project implementation, impairs real-time managing for results.

7. Part I concluded with two key lessons; first, practical steps are needed to “close the disconnect” and improve monitoring and evaluation. Second, the Bank and IEG should strengthen the evaluation knowledge base for the Bank’s corporate results. That is, provide a more comprehensive results framework for the institution as a whole.

8. Part II: A topic relevant to the results described in the first part is the Bank’s work in fostering global public goods. The World Bank focuses on five main strands, these include; environmental commons, communicable diseases, international trading regime, international financial architecture and global knowledge for development.

9. Two key reasons for examining these shared global challenges in 2008 are; first, the fight against poverty is increasingly intertwined with shared global challenges, and second, the Bank has promoted ‘fostering’ global public goods in its new strategy.

10. But when it comes to supporting the delivery of global public goods, the Bank’s country-based model is a double edged sword.  It enhances country ownership and focus, but detracts from a global focus. Usually, country governments’ attention to global public goods increases when a financing instrument is relevant for them, or if there is shared interest among national governments (as is the case in the current financial crisis). Otherwise country partners tend to view GPGs as involving local costs but few local benefits.

11. The Bank’s attention to global public goods is emphasized at the corporate level, but when tracked down to the country level it becomes diluted. Three levers to move from strategy to action at the country-level include resource allocation, financing instruments and global programmes.

12. Beyond the country-level support, the Bank is engaged in advocacy. For example in international trade, it has the capacity to argue on behalf of developing countries. The Bank has also been influential on avian influenza and the environmental commons agenda. That said, developing countries voices remain under-represented in the way in which the Bank engages on GPG issues.  With regards to the new climate change program, the Bank must ensure sound and equitable governance.  

13. Several practical recommendations are proposed on how the World Bank can improve its support for shared global challenges. These include developing new approaches to setting budgets and recognising performance; establishing clearer organisational arrangements to link together country, regional, and global initiatives, and; giving greater voice to developing countries in the governance of significant global programs.

14. To conclude, O’Brien emphasized that meeting the shared global challenges of our time will be crucial to reducing poverty, and the Bank’s role - particularly in supporting investments where global and local benefits diverge - is potentially path breaking.

Andrew Steer
(Director General, Policy and Research, DFID)

15. Steer commended the report and made four key points on Part II of the report:

1. The fundamental point the report makes (i.e. addressing global public goods is important) is itself a public good.  Although a country-level agenda exists (i.e. Paris Declaration and the Accra Agenda for Action) an equivalent model for global public goods is less apparent - hence, the value of raising the issue.
2. Redefining global public goods. The five main global public goods described by the Bank are at best regional public goods. Everything has both a global and national public good component, and a useful schematic representation of the tensions between global and national public goods is presented in the annexes of the report. Through understanding the differences between global, regional and national public goods the Bank can develop useful instruments to address them. On the other hand, there is value in conflating public goods to ensure that national government to see the value in addressing them. The Henley Centers framework for trust and engagement of "my world, our world, the world" was referred to as a useful framework. Essentially, Steer called on development practitioners to define the differences between various types of public goods, but also to show the links.
3. The importance of influence. Steer commended the report's chapter on influence. Although the World Bank has tremendous influence in economic domains, it does not fully understand the sophistication of influence.  The British government, alternatively, understands that decisions are made by politicians and several of paths of influence are needed to affect change. This approach is expensive and time consuming but has tremendous impact. Steer recommended that the World Bank adopt a more sophisticated approach to influence. The climate change debate was used as an example; to build the World Bank’s legitimacy in the climate change sector, it should draw on its social network of established climate change experts.
4. Knowledge is key. Steer argued that knowledge is a critical component to effectively addressing challenges related to public goods.  A reason why low-carbon development in middle income countries has not been effective is largely due to the lack of knowledge.  Although the available analysis on global public goods is world class, there is little knowledge on designing and implementing low carbon policies and projects in middle income countries.  Thus the British government is setting up a climate change network.

Alison Evans
(Director of Programmes, Poverty and Public Policy, ODI)

16. Evans commended the IEG team for a report that displays a high level of sophistication in analysis. The annexes of the report are very informative and relevant.  

17. Key comments on Part I of the report included:

• The Bank's good performance was put in perspective, when she noted that other organisations may look at similar results (80% performance rating) with enthusiasm.
• She cautioned the Bank to be watchful for the growing disconnect between ongoing portfolio ratings and ex-post evaluation ratings, as it may signal loss of management focus.
• More emphasis could have been placed in disaggregating the information between country categories (i.e. fragile states, middle income countries or low income countries) as this would inform the debate, and offer relevant lessons for development practitioners.

18. Remarks relating to Part II included:

• Is addressing global public goods a comparative advantage of the World Bank?  Rather than dominate the agenda on financing global public goods, the Bank could prioritise knowledge building on these public goods. There is an opportunity in the report to outline priorities.
• There are emerging tensions between global public goods and country based processes. Country based processes have become too introverted, rather they should be extroverted. The challenge is to translate excellent pieces of analysis, like the IEG report, to thinking about regional and global issues through the national perspective. 
• What incentives exist to encourage Bank employees to engage with global public goods? Bank incentives structures are country-focused. Role of country directors is very important.  Incentives to think beyond the country are not necessarily in place.
• Managing proliferation. The World Bank is providing or hosting numerous financing mechanisms for global public goods initiatives.  They are also proponent of the country-based approach. To avoid fragmentation, proliferation, and duplication of financing a clear vision is essential. There are needs for new financial instruments to engage with global public goods.

Comments and Questions Raised in Discussion

• Country level mapping of global partnerships for development may be useful.
• Why does the ARDE not mention/engage with the commitment to global partnerships set out in MDG8?
• Existing institutions and mechanisms intended to address the challenges of global public goods, and especially the role of the UN system, were not discussed, why not?
• The report is weak on how good a partner the Bank is in the search for solutions to the GPGs problem. What is its comparative advantage relative to other global actors?
• Does it make sense for bilaterals such as DFID to be routing funding for GPGs through the WB which, in turn, has ways of doing business at the country level that are not compatible with the Paris commitment to build-on country systems nor with DFID’s corporate views on conditionality?
• Building trust with public and private institutions is essential to address the challenges of global public goods effectively.
• Is there a tension between providing public goods and being responsive to clients needs?
• Country voice and capacity: The extent to which the World Bank has such a critical advantage on knowledge compared to national capacity is overwhelming. Is the development sector really investing in building capacity for national level analysis?
• Does value for money come into assessment criteria?