This expert workshop brought together those who have been working on innovative value for money (VfM) approaches within adap­tive programmes, alongside VfM experts, researchers and DFID staff. The workshop was broken into three sections: problem setting and definition; case study presentations; and action planning.  

Problem setting and definition

DFID is increasingly supportive of adaptive management - an approach that takes complexity as its starting point. But, according to a recent review by the International Commission for Aid Impact (ICAI), its approach to VfM appears to have lagged behind, with no detailed attempts to conceptualise, measure or manage VfM differently for adaptive programmes.

DFID uses a 4E framework, where VfM represents a balance between economy (whether inputs of the appropriate quality are being bought at the right price), efficiency (how well inputs are being converted into outputs), effectiveness (how well the outputs from an intervention are achieving the intended effect) and equity, (the extent to which aid programmes reach the poorest and most marginalised). One participant noted that, while the 4E framework offers a helpful guide to VfM criteria, the simple picture of a results chain on which it’s based may not reflect the realities of adaptive programmes in complex settings, where change is often messy and unpredictable.

Approach to the 4Es

Donors have tended to demonstrate VfM by focussing on cost drivers and calculating the savings to be had from operational choices. But, as several participants stated, the key to the 4E framework is a balance of economy, efficiency, effectiveness and equity. Therefore, donors and their partner organisations need to be wary of pursuing cost-savings and efficient delivery without due regard for outcomes or impacts. In other words, it doesn’t matter how cheaply or efficiently a programme is delivered if it doesn’t deliver on its ultimate objectives.

It was pointed out that different departments within donor agencies are likely to have different ways of balancing the 4Es - procurement colleagues tend to emphasise economy; economists will likely prioritise efficiency; whilst social development advisors are generally most concerned with equity.

Measuring VfM

Adaptive programmes start from a position of uncertainty about which outputs are the right ones for getting to outcome-level change and so engage in an iterative process of testing and learning. For these situations, it was suggested that donors need tools that enable programme teams to (re)assess their VfM in ‘real-time’ as the programme tests and responds, and where the value of some activities consists in the learning they generate.   

It was also noted that different approaches and measures of VfM are likely to be more or less suitable depending on the type of adaptive programme at hand. For example, efficiency for a small, experimental WASH programme that is working in short sprints to test key assumptions in its theory of change will likely need to be measured differently in comparison to a large, technical assistance programme that is aiming to take a problem-driven approach to building state capacity. It was stated that, whilst both are examples of adaptive programming, measuring and managing their VfM will likely need different tools and measures.   

One participant noted that VfM is a donor-led agenda and asked whether the sector is missing the point if it fails to push the learning potential of VfM analysis. Another participant said that VfM raises questions about the purpose of development and whose perspective counts for the sake of appraising development initiatives. 

Managing and using VfM data

There was a discussion of where VfM should sit in the programme cycle of an adaptive programme. This brought into focus some issues around the utility of VfM data and the potential value of on-going VfM analysis for interrogating a programme’s theory of change. Questions were raised about how to bring VfM more centrally into on-going programme management, instead of seeing it as an accountability requirement at the approval stage. It was felt by some that there was a need to bring sustainability into the definition of effectiveness; something that can be overlooked by applying VfM analysis only to individual stages of projects.

Case study presentations  

The second part of the workshop consisted of presentations on VfM frameworks which were developed for Oxfam, Oxford Policy Management (OPM) and ActionAid, each of which had been specifically designed by consultants to suit a more adaptive programming approach. 

Oxfam

Oxfam’s definition of VfM is ‘the best use of resources to contribute to positive significant change in the most vulnerable people’s lives’.

Oxfam delivers rights-based programmes in complex settings where the pathways to change are non-linear and context-specific. Through VfM analysis, it aims to articulate the links between resources invested and outputs and outcomes achieved, and examines if and how greater change could have been achieved given the level and type of investment and operating context.

This is an evaluation approach which anchors VfM in the programme logic and theory of change. It is designed to support adaptive programming by generating information about the link between the use of resources and achievement of outcomes, and can be used during strategy testing sessions.

OPM

As with Oxfam, the approach developed for OPM is based on the idea that VfM is an evaluation  of how well resources are being used, rather than something that can be precisely measured using a formula. As such, it requires a judgement about whether the results of the programme’s interventions justify the costs, based on the strength of evidence and a transparent set of assumptions.

OPM’s approach was piloted through the Ligada female economic empowerment programme in Mozambique (subsequently renamed MUVA). In order to judge whether the programme delivers VfM, they have developed programme-specific definitions of economy, efficiency, effectiveness and equity, along with standards describing different levels of performance for each criterion.

For OPM, this evaluation approach enhances accountability, learning and adaptation, by providing a transparent basis for making judgements about VfM.

ActionAid

ActionAid’s approach to VfM is guided by two principles: 1) the value of a programme should be judged on how much social change it has generated and 2) that communities themselves can best assess this value. To this end, the organisation has developed a set of participatory tools to enable participants to identify the most valuable changes for them. These tools have been designed to be used easily by a variety of stakeholder, but still require good facilitators.

The organisation regards this approach as a way to fulfil its accountability commitments both to donors and programme partners. VfM also enables them to adapt programmes and ultimately improve their impact. VfM is seen as an opportunity to reflect not only on whether the organisation has the right systems and processes in place, but also if its investments are generating the kind of changes that are most valued by the communities with which it works. Hence, VfM is a way of informing overarching strategy as well as day-to-day operational choices, rather than just focussing on upwards accountability and reporting.

Session 3: Propositions for action

In the final section of the workshop, participants were divided into groups and tasked with identifying action points under three themes that reflected different views amongst the workshop as a whole on the value of the 4Es for adaptive management.  

Better use of the 4Es

One group identified the following ways of improving our understanding and application of the 4E framework in the context of adaptive programming:

• Remember that good VfM starts with good programme design (whilst taking note that this doesn’t mean programme design where everything is specified upfront). If you want better VfM you need to start with well-designed programmes, rather than well-designed VfM analysis.
• Consider ways of enabling and incentivising programme teams to define their own definitions of and metrics for assessing VfM.
• Encourage closer working relationships and mutual understanding between finance and programme teams.

Moving beyond the 4Es

A second group put forward the following reasons for moving beyond the 4Es and using – for example – a more explicitly participatory method, to measure and manage VfM for adaptive programmes:

• The common way of framing and talking about VfM has the potential to be corrosive for adaptive management. Ift organisations are pressured to articulate all of their activities in terms of value and returns on investment, they may be less inclined to document and share their reflections about processes of trial, error and subsequent learning.
• Given that the 4Es tend to lead to a monetary and donor-centred understanding of value, they can close the space for proper consideration of local partners’ perspective.
• Intangible factors like ‘quality of engagement’ are often key drivers of success in adaptive programmes, but these are hard to capture through standard measures, or in quantitative terms.

Using VfM for ongoing decision-making

Finally, a third group discussed the following more general factors to consider when using VfM data and analysis as a tool to support adaptive management:

• Remember the overarching donor politics in this space. While it’s important to bring a learning lens to questions of VfM, fulfilling the potential for using this kind of analysis to inform adaptive management also requires tackling some of the organisational constraints that can restrict flexibility, testing and learning.  
• Think about VfM at different stages in the programme cycle. The tools and measures for VfM analysis at the business case stage may be different from those that are suitable later down the line.
• Consider staff time and resource implications. Tools for measuring and managing VfM in programmes need to be simple enough for programme staff to incorporate into their work and the data that is generated needs to be in a useable format.

  This workshop is part of a broader project on adaptive development practice. Visit the #AdaptDev project page for more information.