Speakers:

Liz Turner – Saana Consulting and co-author of Traidcraft/CAFOD report

Discussants:

Yurendra Basnett– Research Fellow, ODI

William Hynes - Policy Analyst, Statistics and Development Finance, Development Co-operation Directorate, OECD

Adaeze Igboemeka - Head of Aid for Trade, DFID

Chair:

Andrew Norton– Director of Research, ODI

Liz Turner, Principal Consultant (Saana Consulting) and co-author of Traidcraft-CAFOD report (Aid for Trade: Reviewing EC and DFID Monitoring and Evaluation Practices) began the discussion by outlining the scope of the study. The study centres on how DFID and the EC evaluate the impact of Aid for Trade (AfT) on poverty. The analysis explored a number of AfT projects and programmes and analysed whether reviews and evaluations examined the effect of AfT on the poor. The key findings are as follows:

The key findings are as follows:

1. There is very little publicly available information on whether AfT projects and programmes are impacting the poor. In many cases the evidence of the impact on trade is also limited.
2. There is often a gap between strategic ambitions and statements on poverty reduction (for example, in AfT strategies and other high-level political statements) and project and programme design, implementation and M&E.
3. The pathway from projects and programmes to the impact on the poor is often long, complex, and indirect, and therefore not assessed.
4. In many cases, the casual linkages are based on a series of assumptions – and in some cases a leap in logic – unless the poor are direct beneficiaries of a project.

Liz concluded with recommendations for donors including: greater accountability through conducting more regular and independent evaluations that go beyond monitoring information; exploring the possibility of conducting impact assessments to better understand the poverty impacts as well as trade outcomes; providing greater transparency of monitoring information; and, developing more realistic assumptions, theories of change and intervention logics behind projects and programmes. In addition, recommendations for non-state actors include: lobbying for greater transparency on data and accountability on results; and, undertaking more in-depth research to strengthen the evidence base of the impact of AfT on poverty to better inform advocacy efforts on AfT. The findings, conclusions and recommendations can be found in greater detail in the Traidcraft-CAFOD report (Aid for Trade: Reviewing EC and DFID Monitoring and Evaluation Practices).

Yurendra Basnett, Research Fellow in the ODI’s International Economic Development Group (IEDG) and co-author of ODI Briefing Paper: Focusing on what matters in Aid for Trade,continued the discussion by stating that ODI’s ongoing project on AfT had found very similar challenges in monitoring and evaluating the impact of AfT on poverty. These include problems of attribution and the difficulties in creating a results chain. Drawing upon his own experience on customs modernisation in South Sudan, he discussed the potential effects of improving customs efficiency on the poor. Without readily available data at the micro level, he stated that in many cases AfT projects and programmes are having to make difficult assumptions on poverty impact. He emphasised that AfT by helping countries expand trade capacity contributes to economic growth and development. He argued that such outcomes are likely to have noticeable impacts on poverty when AfT strategies and programmes are aligned to the national development plans and poverty reductions strategies.

William Hynes, Policy Analyst in the Development Co-operation Directorate of the OECD, argued that the report seemed dated in its emphasis on the poverty impact of donor programming. He said that discussions have moved beyond mere donor accountability and evaluating individual aid projects towards country-owned and country-based management for results frameworks in which developing countries measure progress towards trade-related targets and assess the contribution of policies and aid interventions. He also highlighted that different instruments, with implications for objectives and evaluation approaches, are required to tackle the development challenges of low and middle-income countries.

With regard to evaluation of aid for trade, while there are a range of conceptual and methodological challenges, many approaches ranging from basic qualitative assessment to Randomised Control Trials (RCTs) and Impact Evaluation have emerged and are deployed based on the relative resources and capacities donors have at their disposal. While RCTs in trade overcome a number of the methological problems associated with attribution and causality, they are costly, burdensome and sometimes against the incentives of programme managers. They are also inappropriate for evaluating aid for trade projects which are small or provide public goods which have no obvious treatment effect.

As for understanding how AfT impacts poverty, Hynes agreed with the first discussant, Yurendra Basnett, that aid for trade contributes to poverty reduction indirectly by helping to strengthen the contribution of trade to inclusive growth which when combined with a number of complementary policies can in turn lead to poverty reduction.

He concluded that while the Global debate is moving towards an emphasis on investment for trade, private sector partnerships, and value chain promotion, it was important that NGO's such as Tradecraft continue to highlight the importance of poverty reduction in the aid-for-trade agenda.

Adaeze Igboemeka, Head of Aid for Trade, DFID, followed the discussion with responses to the report including:

1. With regards to the need for greater transparency of monitoring information and better quality data, DFID has made a public commitment to increase its transparency and data availability by the 1st May 2013.
2. With respect to the need for realistic theories of change and intervention logics, DFID has focused on taking a very rigorous approach to articulating the evidence underpinning the assumptions underlying all programmes.
3. While many DFID AFT programmes directly target the poor, our focus is on supporting national development plans and government trade priorities; ie major programmes, such as TradeMark, that take a systemic approach to building the national/regional systems and institutions required to expand trade, create growth and reduce poverty.

She highlighted what is next for DFID in terms of AfT: Embedding evaluation better into programme design; Country level evaluations to assess the   results indicators we’ve committed to in our Operation Plans; along with a cross-organisational agenda to use more rigorous evaluation methods such as impact evaluation. Challenges remain for DFID: two thirds of DFID’s £1 billion of AfT spend is through multilaterals; this means we are prioritising working with multilaterals on the AFT results/evaluation agenda; comparing and aggregating results across a range of AfT efforts in different countries and regions, and getting better at attributing impact to DFID spending.

The discussion was geared towards the social impacts of AfT. For instance, even if the net effects of a project are positive, it is important to understand whether there may be losers. However, there is insufficient information, data and analysis to understand impact (good/bad/neutral) on particular groups. Also, the importance of ensuring M&E is embedded throughout the project cycle was emphasised.

The panel concluded that AfT has important impact on poverty through expansion of trade and economic growth. The evidence on when this happens and under what circumstances are context and AfT strategy specific as has been highlighted by an ODI working paper. Aft is more likely to contribute to poverty alleviation when it is mainstreamed into the national development plans and poverty reduction strategies.