Impact Reporting

My big project for the next 6 months is to develop the Concern Universal 2014 Impact Report. The purpose of most aid programs is to ‘make an impact’, and CU Malawi is one of the few NGO’s aiming to publish an Impact Report on a regular basis. The impact referred to being a change in the broader system, not a large hole in the ground as from a meteorite.

A new (to me) concept being used in the aid industry is the idea of having a ‘Theory of Change’. My impression is that this has superseded LogFrames to describe the link between what can be physically done (outputs) and the intended outcomes (or impact) in many programs. Thus my personal ‘Theory of Change’ is that positive effects in the broader system occur when individuals work together to solve tangible problems (1). This is augmented by the many others who have researched and written on the technicalities of how to help people work better together when finding solutions to common problems (2).

Putting in a bore & pump is the easy bit. What is harder is ensuring that people can access it, that there is a management committee of those that use it, that they can raise money for spare parts, that spare parts are available etc. etc.

Putting in a bore & pump is the easy bit. What is harder is ensuring that people can access it, that there is a management committee of those that use it, that they can raise money for spare parts, that spare parts are available etc. etc.

 

A  ‘Theory of Change’ that seems prevalent with many aid donors in Malawi is that WASH (Water Sanitation & Hygiene) programs have a positive impact on the broader system. This is based on considerable evidence that improved water, sanitation & hygiene practice (not surprisingly) contributes to improved community health (3). Unfortunately ‘impact’ is dependent on the many variables of a complex system.

Some donors require a large range of ‘indicators’ of impact, which might be as many as 70 questions that have to be reported on monthly throughout the length of the project. An example being “Reduction  in  reported  cases  on waterborne  diseases (cholera, diarrhoea  and dysentery)’ measured by “#  of  waterborne  diseases  reported cases”. Another being  “students with access to an adequate number of school hand washing facilities with soap” measured monthly by “#  of  additional students with access to an adequate number of school hand washing facilities with soap”. Others take a different approach, maintaining that the linkage is a given and that due to the high number of confounding factors within each individual case of improved health, monitoring this aspect is not cost effective. Some donors require nothing more than a record of the number of water points created, whilst others focus on measuring sustainability indicators, such as water point management structures, financial management capabilities, maintenance, access to repair services, and spare parts.

To report on the ‘impact’ of the work by Concern Universal (who currently implement seven different donors WASH projects, all with different indicators) is in itself complex. Thus my intended approach is to engage the CU WASH teams in a workshop process that draws on their collective data, the processes they use in implementing the projects, their understanding of ‘what works’ and document stories of success and failure. We will then use this data to compile a broad picture of CU’s impact in WASH. Stay tuned!

1: Rittell, H & Webber, M 1973, ‘Dilemmas in a general theory of planning’, Policy Sciences, vol. 4.

2: Butcher, M 2001, ‘LogFrames Made Easy’, PLA Notes, no. 41.

Department of Sustainability and Environment 2005, Effective Engagement: Building Relationships with Community and Other Stakeholders, 3 vols., The Community Engagement Project, Resources and Regional Services Division, Victorian Government Department of Sustainability and Environment, Melbourne.

Dick, B 1991, Helping Groups to be Effective, 2 edn, Interchange, Chapel Hill.

Kaner, S 1996, Facilitator’s Guide to Participatory Decision-Making, 11 edn, New Society Publishers, Gabriola Island.

NSW Premiers Department 2006, Strengthening Rural Communities Resource Kit, pdf version edn, NSW Government, Sydney.

Pretty, J, Guijt, I, Thompson, J & Scoones, I 1995, Participatory Learning and Action : A Trainers Guide, IIED Participatory Methodology Series, International Institute for Environment and Development, London.

Rosenberg, M 2003, Nonviolent Communication: A Language of Life, 2 edn, Puddledancer Press

Schwartz, R 2002, The skilled facilitator, 2 edn, Jossey-Bass.

Southern Cross University 2003, Action Research Resources, Southern Cross University, viewed 04 03 2006 2006, <http://www.scu.edu.au/schools/gcm/ar/arp/arphome.html>.

Stanfield, B 1997, The Art of Focused Conversation, The Canadian Institute of Cultural Affairs (ICA Canada).

—- 2002, The Workshop Book: from Individual Creativity to Group Action, New Society Publishers and The Canadian Institute of Cultural Affairs, Gabriola Island.

United Nations Centre for Human Settlements (Habitat) 2001, Tools to Support Participatory Urban Decision Making.

Williams, RB 1996, More Than 50 Ways to Build Team Consensus, Hawker Brownlow.

World Bank 1996, The World Bank Participation Source Book, Adobe Acrobat PDF Version edn, World Bank, Washington, D.C.

3: Esrey,S 1985,1991; Fewtrell 2005; Clasen et al. 2009,2010, Waddington 2009; Norman et al. 2010

 

Newsletters from the Future

Mona Lisa Bandawe and I have recently run the Mid Term Learning Review workshop for the DFAT funded Mphuka Shared Futures Project. This Participants were from Concern Universal, the Govt. of Malawi and Community leaders.

Card storm of 'underlying blockages to the vision'.

Card storm of ‘underlying blockages to the vision’.

After looking at the state of the project to date, we ran a guided visualisation session, followed by (integrated with) a ‘newsletter from the future’ activity. With about 22 participants there were five groups that developed the newsletters (3 in English, 2 in Chichewe*). We gave a start to the newsletter that read:

“After hearing about the UN award for being the most liveable area in Malawi, I visited Thyolo last week to see for myself what it was all about. Honestly, I was amazed at the transformation since I worked there, I saw….”

………a well dressed, well nourished elderly woman walking majestically long the better road. Far behind her were her grand children coming from a well built secondary school and they were all walking towards their iron roofed house surrounded by a granary full of harvested maize, improved sanitary facilities, goat and pig kraals full of livestock and just 200m there was a borehole. All the hills of Mphuka were covered by trees and the ecology and been reverted to it’s natural state with happy people benefiting from forest products e.g.: honey.

……..aged people and youngsters who were looking healthy and well nourished and their dressing sent a message about people who are now outside the poverty trap. New school buildings and other structures were seen from a distance appearing as a busy trading centre with electricity. As I was passing through the village I saw a lot of iron and thatched houses with satellite dishes here and there.

…….TA Mphuka transformed into a a well developed area. There is 100% water supply coverage and the economic wellbeing of the people. There was a good rural road network for easy travel.

I heard stories about……

………how there was high self dependency & good partnerships happening in T/A (Traditional Authority) Mphuka

  • A CBO (Community Based Organisation) revolving fund accessed by the community
  • A livestock pass-on program working efficiently
  • Established advocacy and GBV (Gender Based Violence) committees.

……..a lot of pupils going to school and that there is compulsory primary education as there are now by-laws put in place. I also heard stories that where I saw a trading centre was a place that used to be Mylenga CBO but it has developed to become a local NGO and other rooms built by them are used as shops and local saloons.  I heard that most families are living peacefully and gender based violence is a thing of the past.

I felt proud that what used to be a poor Mphuka area is now a developed area where people are food secure, practicing all year round agriculture due to irrigation.

…… the CBO’s having grown into local NGO’s and the network coordinator is now a councillor, representing the people of his area and has recently been elected chairman of Thyolo council.  I also heard of 30 families formerly involved in gender based violence had bought motorcycles and were working to help other families change. They are role models of T/A (Traditional Authority) Mphuka. Almost 50% of the CBO’s have qualified and have Diplomas in Accounting. Every household in T/A Mphuka is using energy efficient stoves and the neighbouring T/A’s have emulated the good practice in T/A Mphuka.

I have been told that the two written in Chichewe were along similar lines to the above.

When in Vietnam last year I was impressed by the well built and maintained road network designed for bikes & pedestrians.

When in Vietnam last year I was impressed by the well built and maintained road network designed for bikes & pedestrians.

*English is a second language for all participants.

 

 

 

 

Bottom Up Logic

One of the great inventions of The New Development Paradigm (TNDP) is the LogFrame. It provides a link between a concrete activity (which we can control) and a desired effect within the broader system (which we can influence at best). As a framework based on logic it overcomes the limitations of acting out of ideology or belief, but does not negate the role of culture.

think

My understanding is that it was created by USAID in 1969, the same year as the landing on the moon. It provides a format that sets out the logical relationship between an ‘Activity’ (Space program) an ‘Output’ (such as ‘Moon Landing’) and an ‘Outcome’ (such as ‘Americans feel good about themselves’). The ‘Outcome’ being the important part of TNDP. In between the ‘Output’ and ‘Outcome’ is the ‘Strategy’, which in simple terms is the thinking behind the logic. In this instance it might have been ‘Demonstration of US technical capability’.

In it’s original format, each layer of logic contained both a method of measurement and also the assumptions behind each.

Level of Logic   Assumptions
Outcome American Sense of Pride That people would feel good about such a feat
Strategy Demonstrate US technical capability That there are no glitches
Output  Man on Moon That it is technically possible
Activity NASA space program That there would be the $$ available

Over the years the LogFrame has become both a core element of International Development Programs, and also highly reviled. I’ve been wondering if the derision is because a common (but unstated) ‘assumption’ is that other peoples values are the same as ours. In other words, there is an assumption that no longer having to go to the river to collect water, but having a water point within 300m will be a cause of great happiness. It does not allow that perhaps something more important would be having the opportunity to talk with friends whilst walking to the river.

Some examples of how the logic stacks up, but only if the assumption is that the values of those providing the service or describing the logic and the beneficiary are the same.

Consider five scenarios each starting with the same goal or outcome: ‘Having a happy family”. In each case the logic is a reflection of the ideology or values of the person developing the plan.

  • Water Engineer: Achieve your Goal of a happy family, with having easy access to fresh water (Strategy). This requires a reticulated water system (Output) and you should volunteer to dig the trenches this afternoon.
  • Health Advocate:To achieve your Goal of a happy family, requires that they have access to a good health service (Strategy). This means having a doctor in town (Output) and you should sign this petition for one right now. (Action)
  • Politician (elected representative):To achieve your Goal of having a happy family, requires them to be well represented (Strategy). Thus you should elect me to represent you, (Output) and an action you could take is to vote for me today.

In each case the logic works well. The problem is that none of them are ours.  I was advised by a colleague the other day that ‘being happy’ was such a Western concept. For Africans, she told me, survival is what is important.

By structuring the log frame in response to a different reality to ours this then helps explain the logic behind something that we might find illogical. An example being how a beneficiary gives away half the free seeds and fertiliser provided by an Aid Agency that was to ensure that the family would have enough food for the whole year.

Level of Logic Description Assumptions
Outcome Better chance of survival Outsiders (Aid agencies) come and go. 
Strategy Enhance and build on my social capital That if a child gets sick, it will be the neighbours that will help. 
Output  Insufficient seeds to grow food for whole of year.  But more than before when I had nothing
Activity Give half of allocated free seed and fertiliser to friends and neighbours They will be happy with what I am offering, and will help me in the future

Facilitator: The logic of the facilitative approach is based on the assumption that your values are different to mine. The Technology of  Participation process, (which evolved out of reconstruction activities in the aftermath of the Chicago Race riots of the early 1970’s), enables the logframe to be developed by the one who owns the problem. The role of the facilitator is to ask the questions in an order that generates the answers by participants.

Level of Logic Question Notes
Outcome What is your goal? To have a happy family/To survive etc.
Strategy What are the underlying blockages to that ? A simpler way of answering than ‘what is your strategy’?
Output What concrete, actionable, time bound things could you do to overcome this blockage?  The output, which is the limit to what we can actually do
Activity What Actions do you need to do to achieve the output?  The ‘step by step’ actions to complete the task

If there is a need to put the LogFrame developed this way into a funding proposal or report, simply change the answer provided to the Strategy Question from a negative to a positive and call it ‘Strategy’ e.g.: ‘Enhance and build on my social capital’ could have come from the answer ‘I owe a lot to my friends and neighbours’.

 

 

 

 

The new paradigm

Tea break at the Sugar Project review meeting

Tea break at the Sugar Project review meeting

A characteristic of the Opera House design was that at the time of the award, nobody knew how to build it, not even the architect. In a similar vein, when JFK said that he wanted an American on the moon, nobody knew how to do that either. In both cases though there was sufficient understanding of science and technology for society to be confident enough that such ambitious goals could be achieved.

Project vision developed at the previous workshop

Project vision developed at the previous workshop

Concern Universal’s goal of “A world where justice, dignity and respect prevail for all.” is a different kind of problem, or challenge, to putting a man on the moon, and also requires a

Discussion at the Sugar Project review meeting

Discussion at the Sugar Project review meeting

different kind of thinking to achieve it. Whilst it requires a different kind of thinking, I believe the process of incremental learning developed in the science and technology world, is still applicable.

Break out group at the LDSP  planning meeting

Break out group at the LDSP planning meeting

An aspect of all contemporary Donor Aid programs is that the donors demand regular Monitoring and Evaluation reports of projects and programs, both to ensure accountability but also to facilitate learning and change. I recently attended the Sugar and LDSP (Local Development Support Program) projects quarterly review meetings. I was impressed by the desire to explore the ‘so what?’ question in response to a list of outputs achieved, the real discussions on gender issues, and the use of facilitators in the meetings. The next step is to build on these capabilities, become even more inclusive, get a step closer to the desired goal.

Ideas for change

Change happens in two ways: It either happens to you or you make it happen. Predicting the future is trying to guess what might happen to you, design is purposefully achieving a desired goal or future. 
The competition winning drawing for the Opera House design

The competition winning drawing for the Opera House design.

There is always a gap between the image and the reality, but you need to start with an idea of what you are designing to achieve. The Sydney Opera house didn’t just appear as a finished design, it started as a description of an idea with considerable work and adjustment before the final reality.

On a visit to Maputo a few years ago I asked my friend Muchimba what her vision for Maputo would be if it achieved it’s goal of being ‘developed’ in 20yrs time. Her answer was:

“Well built houses, calm, not the noise and stress of ‘developed’ cities. People with dignity and good health.”

 Interestingly there is considerable overlap between Muchimba’s vision and Malawian’s criteria for a high quality of life:

 Enough food, harmonious relationships, good health and housing.

Morning commuting traffic past our house. Note the ‘Walk & Talk’ facilitation technique plus lots of smiling and laughing.

Morning commuting traffic past our house. Note the ‘Walk & Talk’ facilitation technique plus lots of smiling and laughing.

Neither mention freeways or self managed super schemes, which may or may not be successful strategies to reach the vision.  What I do find really strange is how whilst far from the ideal in many ways, contemporary Blantyre also illustrates some aspects of what the developed cities of the West might be missing out on.

Buying veg from the urban agriculture producer retailer.

Buying veg from the urban agriculture producer retailer.