A story about collaborating on an approach to program evaluation that accommodated divergent agendas, and also allowed unanticipated outcomes to emerge organically.

 

Evaluating projects doesn’t immediately seem like an activity which requires collaboration. Perhaps for projects with a known process and set measures it does not need to be. For unique or innovative projects, however, a collaborative evaluation process can allow for a more complete picture of project outcomes - one which considers and gives equal weight to stakeholders with competing perspectives. This independent evaluation sought to find a collaborative way for key stakeholders to have a share in, co-evaluate and ultimately agree how to communicate their project’s outcome.

 

In 2013, the Young and Well CRC and their partner Victoria University invited Project Synthesis to lead the evaluation of an innovative program called “The Lab” which provided a learning space for tech-enthused young people with Asperger's syndrome.  The Lab had been developed by the team at Victoria University and their partner Oztron. Due to the nature of the project (an innovative idea which had not been tried before), and the diversity and passion of the stakeholders (from funding partners to the young people involved and their parents), an integrated and inclusive approach to evaluating the project was requested. Project Synthesis was chosen to coordinate the project evaluation due to their experience and strong existing relationships with the key stakeholders. They had confidence that because of the existing relationships, drawing together the insights and information needed to produce the report would be a fairly straightforward process.

 

The collaboration was initiated via a series of meetings which convened stakeholders to build a shared understanding of the project’s outcome from the varying perspectives. At these meetings, stakeholders discussed the results of the project in relation to their perspective and feedback collected by Project Synthesis from project participants. At the conclusion of these meetings, Project Synthesis drew together the outcomes discussed and communicated this back as a draft evaluation report to the participants. 

 

When presented with a document which concretely discussed the project outcomes, stakeholders were quick to communicate perceived gaps, biases and misalignments for the compiling team to address. Shifting focus to a central document demonstrated how far from aligned stakeholders really were - and how much their role in the project influenced which outcomes were valued and opinion of how these should be communicated. The level of disagreement was unanticipated, and Project Synthesis quickly realised that this had eroded confidence in the process being undertaken.

 

Realising that the relationships and understanding needed to be rebuilt, strengthened and leveraged to get the document where it needed to be, Project Synthesis continued to work with the group to facilitate the co-creation of the document over the following months. This required regular engagement of project stakeholders face-to-face to work through a process of converging competing demands and building confidence in the outcomes. The progress was slow, but Project Synthesis held strong to their collaborative process and coaxed the group away from slipping into standard modes of working such as trying to circumvent the group by requesting document changes directly with the editors. Issues were brought to the table and worked through as a group.

 

The level of misalignment amongst project stakeholders and the amount of time required to find a shared understanding were unexpected, but collaborations seldom follow the path of least resistance. Reflecting on his learnings from the project, Dan Donahoo from Project Synthesis commented:

 

“You can’t rely on having a good relationship to translate into a good collaboration. You need to have a frank conversation at the outset about expectations, about process and that the outcome will be unexpected.”

 

A sign of a good collaboration is that no-one can predict the outcome. This is uncomfortable for collaborators used to working with others to facilitate having their outcomes realised - instead of true collaboration. Dan commented, if he could say one thing to everyone at the outset, it would have been: “What will come out of this may be very different to what you expect… you may be surprised”

 

The final evaluation report took six months for final sign-off by the project stakeholders. It was a collaboration which, at moments, was uncomfortable, hard, and strained relationships. However, taking the time needed to permit tensions to reveal themselves, be worked through, and allow final outcomes to emerge was necessary, ultimately valued, and ensured all voices were heard. When the final document was released, all stakeholders were in alignment and agreed that the outcome was the right outcome and could not have been achieved by rushing or without the full inclusion of everyone in the group.

 

You can learn more about the work of Project Synthesis here.