Evaluating collaboration with POETQ

I have written on this blog a number of times about the very good fortune that I have to work with some great people and one of my most recent publications was co-written with a fantastic former colleague Stephen Jeffares (Jeff to most who know him).  I can’t talk enough about how much I enjoy working with Jeff who is a font of amazing and wonderful ideas and also a cracking musician.

Jeff and I did PhDs at the University of Birmingham around the same sort of time on broadly similar sorts of areas although in different departments and didn’t really know one another until we both took up academic roles at Birmingham.  When we got chatting about our work we discovered that both of us had created approaches to evaluating partnerships, albeit from slightly different perspectives.

We both had identified limitations to our work and had a desire to keep certain aspects and improve on others.  We both agreed that some sort of online tool that was able to explore certain process aspects of collaborative working was helpful (a feature of my POET tool) and that a process that would facilitate an exploration of the range of different perspectives relating to the aspirations of collaborative working was also needed.  In relation to the latter, Jeff had worked with a Q methodology approach in his PhD and found this helpful in exploring different perspectives on collaborative working.  I had struggled in my PhD to get professionals to articulate their aspirations for collaborative working in a satisfactory sense and could see a lot of value in the process of the Q methodology approach in overcoming some of the limitations of my approach.

We agreed to combine the insights gained in our respective PhD projects and apply these to a new tool for evaluating collaborative working.  We had the good fortune of being successful in a funding application to evaluate the outcomes of joint commissioning approaches in England which afforded us a small amount of money to do some development work on the new tool.

I learned an important lesson at this point that getting some money to support the development of an online tool is the easy bit!  Yes it may have taken 3 months to craft a 30 page research funding submission but this was nothing compared to the challenge of trying to give some money to someone in return for what we wanted.

We met with a whole range of different people who do development work but were not really happy with any of the ideas.  Most of the conversations seem to revolve around why it was too difficult to do what we wanted to do within our budgetary constraints.  To be clear we weren’t really asking for that much, just not to have an automated form – which is what most people wanted to develop for us and we could have just done ourselves.

Jeff had the great idea of advertising in the computer science building for students and this is how we met Greg.  Greg said yes to all of our stupid ideas, was bright and really knew his stuff (not that we did, but he was headhunted to Apple straight out of his undergrad degree and moved off to California which proved it for me).  Greg did some great work and between us we eventually got to the POETQ tool that exists today.

We used POETQ in the joint commissioning work and also in a number of other evaluations of collaborative working arrangements.  Jeff does a lot of work with graduate students and academics interested in Q methodology and we make this online application (which is considerably quicker than a manual sorting process) available to pretty much anyone who will talk to us (nicely) and will use it for good.  So although originally designed to evaluation collaborative working, POETQ can be customised through the statements it contains and the questions it asks to be used on a whole host of different topic areas.  Before this I don’t think a fully automated Q sorting system existed online in such an accessible way so has garnered attention for this community (both positive and negative).

Anyway, the paper that Jeff and I just published in Evaluation is, I guess, an academic version of the story of how we got from our separate PhD projects in broadly a similar area to developing POETQ, what different ideas underpin the tool and a little about our application in the joint commissioning project and some of the challenges with evidence and complex policy initiatives.  You can read the full version here and we’d be keen to hear from anyone who has used this type of approach in evaluating collaborative working or who might want to use this tool in their research.