PSIAs are studies into the social and poverty related impacts of policy change. Many PSIAs are undertaken ex-ante, but they can be undertaken while a policy is being implemented. Their aim is to increase the use of information in policy formation, and allow for the identification of winners and losers as a result of the proposed policy change and to understand or predict the likely distribution of benefit and loss. The intention is that this information feeds into policy design and redesign, its sequencing and the development of mitigating or complementary policies. However, PSIAs have been criticised as being donor led and insufficiently embedded in domestic policy making processes.
So, why use PSIAs rather than any other mechanism to investigate the impact of policy on the very poorest? We compared a number of investigative tools commonly used to examine the distribution of costs and benefits generated by policy change and found that PSIAs are not necessarily the best instrument for generating informed policy change and improvements in implementation by either donor organisations or in southern governments. They suffer from having four main end users (donors, domestic policy makers, domestic civil society and the international academic community); they commonly focus on issues that are of importance for the major donors or the country government but not necessarily on the topics of greatest interest to the very poorest; and they are not necessarily well embedded in government policy processes. In addition, most commentators feel that PSIAs are already methodologically complex. If they are to be adapted to fully differentiate amongst income and well-being groups to represent the impact of policy change on the very poorest they will become even more complex. This may make them more difficult to replicate successfully without use of expensive international technical inputs. In contrast household surveys and Participatory Poverty Assessments tend to have substantial domestic buy in. They are also regularly repeated (although PPAs are only beginning to be repeated now), producing time series data which allows for an analysis of change, often down to the District and sometimes sub-district level. PRSP Annual Reviews are part of the PRSP implementation process and therefore are both embedded and regular (or at least as embedded and regular as domestic buy-in of M&E processes allows). Beneficiary Assessments and the use of Report Cards are seen as being ‘civil society’ friendly and there tends to be the capacity in-country to facilitate and manage such participatory approaches to data collection. Both can be easily adapted in order to capture the opinions of the very poorest.
Even if PSIAs or another method provide governments and other decision makers with improved information about the very poorest, will that result in improved policies targeted at this group and in their improved well-being? We suggest that there are methodological, administrative and political difficulties in differentiating amongst the poor and in designing, funding and implementing policies aimed at this group which need to be addressed with vigour if the very poorest are to be assisted.
Focusing on twelve pilot PSIA studies conducted during 2002 and 2003, analysis shows that the majority of the studies present information on impact on the poor but do not differentiate meaningfully amongst the poor. They contain very little discussion of how policies or reform programmes may affect different sub-groups of poor people and the very poorest are rarely identified. This is partly due to the methodological difficulties in providing differentiated information but it is also a result of the policies that many of the pilot PSIAs focus on, which tended to be issues which are of less immediate relevance to the very poorest in terms of first round or direct impacts.
The paper suggests that improving PSIAs methodologically so that they can provide differentiated information about the very poorest is not the biggest hurdle. There are a number of complementary activities are necessary if such information is to be successful in mainstream reaching the very poorest within donor and southern country government policy and practice. DFID needs to decide whether PSIAs are the best information generation mechanism to achieve this. Methods for differentiated analysis within PSIA should be developed and pilot tested, this could be complemented by commissioning a desk study to examine the effectiveness of a range of existing methods used in such impact assessments, and the quality of the evidence that they have been able to produce analysis of policy impacts on differentiated groups.
The technical focus on improving data collection and analysis methods must be coupled with complementary activities in capacity building and dissemination. These must be seen as long term activities and supported through long term change management processes in government and in the donor community based on good institutional and political analysis. There is also a need to increase use of political analysis of decision-making processes within DFID. The domestic political economy must be understood. Why is it that the very poorest are excluded from political processes? Why is it politically unappealing to focus resources and attention on marginal and vulnerable groups? Until these two areas of weakness are resolved, PSIA will have limited impact on the very poorest. We suggest that a study is commissioned to examine these political processes in a small number of carefully selected countries.
A number of changes in donor practice are suggested if a focus in the very poorest is to have a chance of success. These include changing professional incentives to result in improved relationships with donor recipient governments, investing in long-term change management processes and embedding country strategies within a solid knowledge of the culture and political economy of that particular country.
DFID has an opportunity to take a lead amongst donors in focusing attention on the very poorest. However, if this effort is to be successful, clarity will be needed on what southern governments and donors should do if PSIAs or other investigative tools identify that policy change is likely to damage very poor people. First of all, policy makers may wish to know how many people will be affected and how badly, but then policy makers will need to make a calculation of predicted gain against predicted loss. The literature on social cost benefit analysis provides some guidance on this, but in the end a political decision must be made. If a decision is made to go ahead with the policy change, there may not be enough information to devise effective mitigating measures – even if there is the political will. DFID will need to develop clear policy on how to approach these difficult issues, including policy on social protection and compensation measures.
