Evaluating networks
As both a coordinator and an evaluator of networks I have been frustrated at the lack of understanding about this organisational form or way of working despite the growing popularity of networks in the world of development policy and practice. The proposal for a roundtable was presented to the 3ie Conference on "Perspectives of Impact Evaluation" because of this frustration. (For more information about the conference check their website) The roundtable was conceptualised in collaboration with three colleagues: Kate Czuczman from the International Forum for Rural Transport and Development (www.ifrtd.org), Sheila Oparaocha from ENERGIA (www.energia.org ) and Paul Starkey, author of Networking for development (P.H.Starkey@reading.ac.uk), but unfortunately none of them could come to Cairo. We decided, in the true spirit of networking, that we would draw the 'expertise' for the roundtable from among the participants at the Conference. Finally, there were seventeen people at the roundtable in Cairo, including the four people who helped stimulate the discussions: Ricardo Wilson-Grau an independent evaluator of networks, Katsuji Imata from CIVICUS (www.civicus.org), John Young from ODI (www.odi.org.uk) and Nancy MacPherson from the Rockefeller Foundation (www.rockfound.org)\.
Ricardo began the session by presenting two realities of networks and three dilemmas that face network evaluation. The two realities are: that networks are highly complex (so relationships of cause and effect between what a network does and what it achieves are substantially unknown), open and diverse with members operating in very different contexts, and, characterised by constant and discontinuous change; that they are structurally unique with a very diverse membership and expectations of democratic decision making. The three dilemmas: (i) that despite their uniqueness, stakeholders expect them to be evaluated against conventional criteria; (ii) that these criteria, especially the ability to measure and attribute change are particularly difficult; and (iii) that networks are both an end in themselves as well as a means to an end.
Katsuji and John reinforced Ricardo’s characterisation, while Nancy approached networking from a more structured perspective, focusing on the management of networked organisations. John drew from the work of a colleague, Enrique Mendizabal (see http://www.odi.org.uk/resources/odi-publications/opinions/105-enrique-mendizabal-supporting-networks.pdf or http://www.odi.org.uk/resources/odi-publications/background-notes/2008/humanitarian-network-functions-approach.pdf)
In retrospect, we may have done more justice to the wealth of information and experience around the room if we had moved more quickly to exploring the challenges of evaluating networks.
We had some good pointers from the discussions. We recognised the unhelpfulness of logical framework analysis as a tool, but agreed on the importance of developing a theory of change for a network and accepting that in such a fluid relationship structure, this theory of change can be subject to frequent revision, and that it requires an approach to evaluation that calls for iterative methodologies. Some of us felt that the space to work in this manner was somewhat constrained, but there were also several encouraging examples of the evaluation of networked relationships. Ricardo, for instance has worked extensively on this, and you can learn more about his work on http://mande.co.uk/blog/wp-content/uploads/2008/12/evaluating-international-social-change-networks-ricardo-wilson-grau-and-martha-nu1.pdf OR http://mande.co.uk/blog/wp-content/uploads/2008/12/evaluating-the-effects-of-international-advocacy-networks-ricardo-wilson-grau-20071.pdf
We also discussed the issue of the need to evaluate networks both as a means to an end as well as an end in themselves, since it was the act of effective 'networking' that often resulted in effective development outcomes.
It was encouraging to know that there could be (and are) different ways of approaching network evaluation that allow more appropriate frameworks to be used. The roundtable was probably a good introduction to what could have been a full day’s workshop. Let’s hope the donors are listening, and that we can continue this discussion elsewhere.
Ricardo began the session by presenting two realities of networks and three dilemmas that face network evaluation. The two realities are: that networks are highly complex (so relationships of cause and effect between what a network does and what it achieves are substantially unknown), open and diverse with members operating in very different contexts, and, characterised by constant and discontinuous change; that they are structurally unique with a very diverse membership and expectations of democratic decision making. The three dilemmas: (i) that despite their uniqueness, stakeholders expect them to be evaluated against conventional criteria; (ii) that these criteria, especially the ability to measure and attribute change are particularly difficult; and (iii) that networks are both an end in themselves as well as a means to an end.
Katsuji and John reinforced Ricardo’s characterisation, while Nancy approached networking from a more structured perspective, focusing on the management of networked organisations. John drew from the work of a colleague, Enrique Mendizabal (see http://www.odi.org.uk/resources/odi-publications/opinions/105-enrique-mendizabal-supporting-networks.pdf or http://www.odi.org.uk/resources/odi-publications/background-notes/2008/humanitarian-network-functions-approach.pdf)
In retrospect, we may have done more justice to the wealth of information and experience around the room if we had moved more quickly to exploring the challenges of evaluating networks.
We had some good pointers from the discussions. We recognised the unhelpfulness of logical framework analysis as a tool, but agreed on the importance of developing a theory of change for a network and accepting that in such a fluid relationship structure, this theory of change can be subject to frequent revision, and that it requires an approach to evaluation that calls for iterative methodologies. Some of us felt that the space to work in this manner was somewhat constrained, but there were also several encouraging examples of the evaluation of networked relationships. Ricardo, for instance has worked extensively on this, and you can learn more about his work on http://mande.co.uk/blog/wp-content/uploads/2008/12/evaluating-international-social-change-networks-ricardo-wilson-grau-and-martha-nu1.pdf OR http://mande.co.uk/blog/wp-content/uploads/2008/12/evaluating-the-effects-of-international-advocacy-networks-ricardo-wilson-grau-20071.pdf
We also discussed the issue of the need to evaluate networks both as a means to an end as well as an end in themselves, since it was the act of effective 'networking' that often resulted in effective development outcomes.
It was encouraging to know that there could be (and are) different ways of approaching network evaluation that allow more appropriate frameworks to be used. The roundtable was probably a good introduction to what could have been a full day’s workshop. Let’s hope the donors are listening, and that we can continue this discussion elsewhere.
.jpg)
Comments
Post a Comment