Understanding impact – FFI’s approach to monitoring, evaluation & learning

Every year, when Fauna & Flora International (FFI) publishes its annual Conservation Report, I love seeing the impact we’ve had as an organisation. This year, I was delighted to see that most of the 25 offshore Caribbean islands now free of invasive alien species are showing a more than fivefold increase in land birds and lizards; community-based marine area management in Aceh, Indonesia is driving up fish biomass and diversity; and awareness-raising efforts by FFI and others drove an increase in government rangers patrolling Niassa National Reserve, Mozambique, culminating in a dramatic decline in elephant poaching.

We wouldn’t know that we were making these collective differences each year were it not for our organisational annual reporting process, from which the Conservation Report is derived. Effective and locally appropriate project monitoring, evaluation and learning underpins this process, providing the evidence from which to draw out our impacts and lessons learned.

The main purpose of monitoring and evaluation is to inform project strategies and decision making by applying knowledge gained from evidence to maximise the impact of our work. It also enables sharing of lessons within and outside FFI and allows us to confidently communicate our findings and learning with local stakeholders, peers and the donors that invest in our work. Thus, it also helps ensure our accountability.

Finding a way to assess impact and draw out learning at both project and organisational levels across a diverse and global organisation isn’t easy, though. For FFI, this is a particular challenge as we often operate through partners, on whom it is not appropriate to impose a specific system or process. Within this reality, FFI has, over more than a decade, given a lot of thought to how we understand conservation success and what constitutes ‘good’ monitoring and evaluation. We have made significant progress in tackling some of the challenges within this and are proud to share the key elements of our current thinking.

Our monitoring, evaluation and learning process helps us recognise that community-based marine area management in Aceh, Indonesia is driving up fish biomass and diversity. Credit: Rakhmat Dirgantara/FFI

Key to successful monitoring and evaluation is to start by asking the right questions. This may sound simple, but conservation professionals sometimes report that a wealth of information has been collected but not used – so why was it collected? Asking the right questions from the outset helps avoid this, and it starts with good project design.

At FFI, we advocate the Theory of Change method in project design. This involves working with conservation practitioners, technical experts, partners and local stakeholders to define long-term conservation goals and map the logical sequence of changes that are needed to reach those goals. Not only does this help identify the most appropriate strategies in any given context, but it also enables us to pose those questions that will give us the answers we need to show progress in the right direction. In the early 2000s, FFI worked with the Cambridge Conservation Forum’s ‘Harmonising measures of conservation success’ group to test this approach’s effectiveness for monitoring progress towards impact and found it to be an effective predictor of success.

Whilst defining monitoring and evaluation questions through design processes is now relatively common across the conservation sector, a wide range of approaches – with differing perspectives around what constitutes appropriate evidence and how much evidence is enough – are used. These methods range from strongly evidence-based (using experimental design and controls) with high burdens of proof on individual projects to lighter-touch and participatory approaches, often adapted from the development sector.

Recognising FFI’s practical approach to conservation, its partnership model and its focus on community-led conservation, we prioritise monitoring, evaluation and learning approaches that are appropriate to the questions, context, skills and resources available. Therefore, a plurality of approaches are maintained across our portfolio, encompassing qualitative and quantitative evidence and local or indigenous knowledge. In addition, given the time it can take to see biodiversity results, we aim to use approaches that are, wherever possible, relatively simple and efficient and thus easily sustained by the partners and communities with whom we work.

What constitutes “just enough” evidence to enable robust decision making and inform our organisational annual reporting process differs for each project. We are exploring how best we guide our FFI teams and partners to determine what is just enough for them.

Most of the 25 offshore Caribbean islands now free of invasive alien species are showing a more than fivefold increase in land birds and lizards. Credit: Edward Marshall/FFI

FFI’s annual reporting system has been designed to balance the need for project-specific monitoring and evaluation as described above with an organisational ability to evidence collective impact and learning. Project teams provide evidence against their key monitoring questions, which are then mapped against generic pathways to change, providing an indication of impact across common conservation approaches without a need for centrally defined indicators. Through this process we learn about common factors of success and difficulty, enabling us to adapt as an organisation and strengthen our collective impact.

For monitoring, evaluation and learning to become well-embedded within projects in order to support adaptive management, promote learning from successes and failures and strengthen the path to impact, it must be seen as a priority by those delivering conservation action and those funding it. FFI has made significant progress to bring about this culture shift to date and it is our continued focus as we look to deliver greater demonstrable and lasting impact in the future. We welcome funders who support us through enabling adequate resourcing of projects, not just for the collection and analysis of evidence – but also for making time for reflection to apply learning.