Evaluation of ComPart – The real question

As if there wasn’t enough going on in ICCO in the last quarter of 2010, the ComPart evaluation came to a head in December. Marc Coenders and Nancy White, the lead evaluators, held a validation workshop on 9 December 2010 to discuss their findings and the final report was presented to management on 12 December 2010. It was a deep investigation and the findings described in the report are rich, balanced between positive and negative, subtle and far-reaching so it is hard to summarise the main points in a blog post. This is therefore a personal take on the findings, from my own point of view, as both a ComPart admin team member for three years and an independent consultant with experience in KS and the use of digital technology to support the work of development organisations.

A storm tossed cork

In terms of context the report acknowledges that ICCO has been through a period of turbulence unrivalled in recent times, one that will unfortunately continue for at least the early part of 2011, and not simply because of the current financial pressures. The longer term change illustrated opposite is about

continuing to move to a more networked organisation at a time when public money is reducing sharply. In that context staff and partners are much less likely to be able to – or want to – prioritise learning about a ‘change project’ such as ComPart, especially one that is associated with a ‘way of working’ that is different to the status quo. ComPart, by definition, is focused on ICCO’s partners and alliance members, which, the report notes, has meant that the ComPart way of working has become associated with the wider changes taking place across ICCO during this period, which in many cases produces negative connotations. This provides a context for the finding that internationally based people generally had more positive views of ComPart and its role than Utrecht based staff. This is despite the fact that, as the report notes, international offices in their set-up phase were generally unable to engage with ComPart as they would have liked to and so the ambition to use ComPart with partners has not yet been achieved.

The ComPart Lab

The report notes early on that, "ComPart turned out to be a laboratory with a series of experiments rather than a project with defined, measurable goals." In terms of evaluating the results and impact of ComPart, the report acknowledges and speaks positively about the range and scale of the work that has been done; the quality of support provided; the continual experimentation and the consequent well documented lessons learnt; and the resultant growth in awareness across ICCO of how digital tools can support business and learning processes. However, the number of active users is still small. This, suggest the evaluators, is a product of the complexity of the ComPart toolset, the difficulty of choosing between or integrating ComPart with corporate IT and Communications tools and the fact that its usage is not seen as an essential part day to day working but as an extra. The one occasion where ComPart was integrated into a core Business Process – the development of the Business Plan – did raise ComPart’s profile and usage of the wiki but also generated a lot of extra work and some confusion as it was introduced rapidly and with inadequate preparation. That example also highlighted one of the key weaknesses raised in the report, that wikis are often difficult to introduce into organisations because a lot of people do not like their structure – or lack of it – and are not comfortable with the way of working they encourage. The ComPart wiki has become a rich source of information and learning but many people are less aware of the other tools in the ComPart toolkit.

Questions to do with ways of working as well as attitudes to technology and information storage or retrieval are central to the report, mainly because the evaluators note that there isn’t a consensus about these issues within ICCO. For example, to some people a wiki is a place to store information and should become as reliable, searchable and up-to-date as other online resources. Other people see a wiki as a dynamic and flexible tool to support collaborative processes and are not rigorous in how information is organised and stored. Wikis, of course, can be used in both ways but the result is an inconsistent collection of material not always accessible to new staff or others not so engaged in the ComPart experiments. At the same time, to many people ComPart was simply the toolkit and the ‘way of working’ which it promoted – and indeed depended on for its success – was not conceptually well understood, perhaps too open and flexible for the current ICCO way of working. This leads to one of the most fundamental findings which is that the questions which are raised in the review are not essentially to do with ComPart but to do with ICCO itself, how the organisation wants to go forward in terms of learning and Knowledge Management. The evaluators suggest the most important questions are about what ICCO wants to do with its learning and knowledge sharing, about what is the ICCO way of working - rather than the ComPart way of working; about how Departments work with each other and how ICCO is going to adapt to its evolution to a networked organisation. They suggest that ICCO should concentrate on thinking about those issues, and then look at what can be learnt from the ComPart programme to support those new ways of working.

A rose by any other name…

Names are important. Marc has some interesting things to say about learning and naming/theory (see blip below). The recognition that ‘ComPart’ has become associated with problematic issues for ICCO staff suggested to the evaluators that, “the ComPart brand carries enough misunderstanding that it could become a liability. We suggest this is a turning point where ComPart becomes the historical name of a period of experimentation and learning”. We need to talk about the “ICCO-Alliance way of working” rather than the “ComPart way of working”.

From pioneering to deploying

In the new decentralised (and down sized) ICCO social media and other digital tools definitely have a role to play. So the pioneering phase is over. The time has come to really deploy and integrate working with digital tools in the organisation its processes as it regionalises. Therefore it is time to launch a new phase, taking into account the experience gained and all lessons learnt from the “ComPart phase”. As well as the ongoing regional office development programmes there are other new initiatives, such as the learning communities programme, which could benefit from that experience. However, the report also suggests that two other key areas of attention. Firstly, ensure that the next phase is integrated within the whole organisational way of working. Secondly, as part of that, there must be smoother and more cooperative relations between directly involved departments (P&D, ICT, Communications) and thereby with all staff within the Alliance.

Pete Cranston

(If you are interested you can ask for a copy of the full report at compartadmin@gmail.com)