One of my favourite Monty Python gags is the recurring theme of …” and now, for something completely different…”:
and now for something completely different.. a man with three buttocks.
or
and now for something completely different… a man in an evening dress, sitting in a cage at the zoo.
I love this kind of humour because it is so twisted and non sequitur. It tickles my brain and that makes me happy.
One of the most completely different –and recurring– ideas we have encountered at CAPTURE is the concept that people want to hear about interventions that didn’t work in public health practice. The thinking goes that we often learn more from mistakes than from successes. I completely agree. Learning from our mistakes is a fundamentally human endeavour (think learning to walk, learning to talk). Admitting our mistakes to our funders and peers, however, is not something that comes naturally to most people.
In the spirit of being completely different, I would like to share a fairly big “mistake” (a.k.a. “learning opportunity”) that we made during the development of the CAPTURE Platform.
Let me set the context because, as they say, context is everything. In January of 2010, CAPTURE found itself one year into its three year funding cycle and we had made minimal progress in developing our web based evaluation platform. We had done truckloads of consultations –which was very useful — but we hadn’t written a single line of code. Further, we had just decruited our first web developer and hired a new one. We also had a surplus of unspent cash because, as with most start ups, we hadn’t been able to ramp up our spending as quickly as we thought we would (another lesson!).
Given the context, it seemed perfectly logical to hire an outside web development company to help us move the project along. It was our thinking that our in house web developer and I would direct the external developers and, in the end, they would build us a product that we could extend later.
It turned out to be a bit of a disaster. It took us a month to get the developer to understand what we were trying to do and, in the end, I still don’t think they got it. To be fair, our thinking on the project evolved over time as new ideas came to the surface and this, understandably, frustrated the developers because they felt they were building code on shifting sand. We had intentionally tried to avoid writing a detailed business requirement specification because we knew we would be figuring it out as we went along but this only lead to the inevitable question of “how will we know when we are done”. In retrospect, these problems are painfully obvious and seem completely predictable but, at the time, both the external developer and the CAPTURE team went into the arrangement as seasoned developers who believed we could make it work.
So, what did we learn from our little “pilot project”? Well, we learned that we couldn’t develop an innovative application like CAPTURE using only outside developers. We hired 2 additional staff and now everything is being done in-house because we know that the sand will continue to shift and we will need to be able to respond. Further, we have adopted an agile development process that minimizes the time lag between idea conception and feedback collection from users. We have also realized that requirements specifications are valuable as long as we don’t try to design the whole system in one go. Specifying features for the next month’s development cycle seems to be the right level of formal planning for our team.
From a strict performance measurement perspective, CAPTURE might not fare too well: we spent 6 months of time building a system that never made it into production. From a learning perspective, however, those 6 months were incredibly valuable because we learned a great deal about what works and what doesn’t work in our situation. We tested an outsourcing model and rejected it. We tested out some coding techniques and web frameworks and rejected them. We experimented with a minimal documentation model in our feature design process and rejected that. These lessons have contributed directly to the well-oiled, productive development team that we currently enjoy today. Sometimes you have to “throw one away” to improve your practice. The trick is to try to embrace these situations and to grow from them.
December 7, 2011 at 7:36 pm
I really appreciate your honesty in talking about mistakes or ‘learning opportunities.’ I am currently working on a web based knowledge translation tool, and it is taking forever and it seems like we are going down different rabbit trails, and yes, not spending as fast as we should. I am definitely learning, and can see where I would do things differently next time. Interesting points about in-house development vs outside development–will have to see if this applies to our situation.
December 8, 2011 at 8:47 am
Thanks for your comments Asheya. I have really come to appreciate that almost nothing goes exactly to plan and that success is partly a function of how well you are able to pivot your idea as new data comes in.
One thing worth noting and that probably doesn’t come through in the text is that I still value planning and proper project management. The purpose of this blog was to say that sometimes your plans go up in flames despite your best efforts and intentions.
December 11, 2011 at 1:54 pm
“sometimes your plans go up in flames despite your best efforts and intentions.” — I believe that’s called ‘research’ :)
December 13, 2011 at 11:36 am
Exactly! I find it fascinating to see how certain domains embrace their learning opportunities and others shy away from them.
I like the way that the business literature uses the term “pivot” to describe that things aren’t working out as anticipated and that a change direction is needed. It is a positive and constructive concept.