A Geek’s Lens

There are many times when repeating things is necessary and helpful. That is certainly the case with a strong QA/testing/feedback process in software development. The key is getting feedback early and then often. In my most recent project at Fog Creek, we took a product from concept to release in 6 weeks. It was a substantial amount of work. It adds some great new functionality, but my best guess is that it took us fifteen to twenty percent longer then it “should have.”

The problem was simply that our rock star QA analyst was away on vacation at [what we failed to recognize as] a pivot point in the development cycle. It happened just as we finished the first pass of Fog Creek Copilot OneClick’s functionality, before any of the chrome was applied. The people around to give us feedback had a strong technical understanding of the product and the problem that it was solving. As a result, we spent a full week without serious first-time user feedback.

It is a distinct advantage, that QA analysts aren’t programmers (usually). It’s awesome that FC attracted a few that put themselves in the shoes of lead users in a way that is very hard once you are deeply involved in the architecture of the software. Without their feedback, we made decisions about features and interaction design, as well as assumptions about the readiness of the software for delivery to market, that were wrong.

We could have solved this problem without Alison (QA analyst extraordinaire) if we had recognized it. Steve Krug has a great methodology for dealing with this exact issue, through quick and cheap usability testing, but we didn’t think it was dangerous to delay the collection of feedback just a few days. Its not that we had to undo all of the decisions that were made in that intervening week, but some made it harder to incorporate the necessary changes that came out of Ali’s testing. Even hallway usability testing failed us, to some extent, because we had explained too much about the product and the process to our colleagues. Without credible non-tech-centric feedback, our blind spots grew, but our awareness of them receded. More importantly, we lost momentum because we started to feel like it was done even though we were only really at the halfway point, in terms of time.

The day Ali returned from vacation and started filing bugs, our productivity went right back up, our sense of urgency returned, and our blind spots started shrinking. 2 weeks later, we were cleaning the dust out of the corners and putting the final touches on the product. It’s comforting to know that this is a problem that can be hedged by proactively improving our process. Next time around, we will be sure to have either QA or user feedback built-in to the “initial pass complete” phase of our development cycle.

It seems to be the single word that describes the last few weeks. On Wednesday morning, I looked out over the Hudson River on my way to work and felt like nature was acting as a mirror.

Here’s to hoping that the fog lifts.

I returned late last week from the Business of Software conference. The speaker list was a veritable who’s who of experts on the process(es) of creating and managing software/technology companies.

Seth Godin’s presentation touched on many of his classic themes (being remarkable, making ethical use of the permission that your customers give you to talk to them, get product and marketing to match the culture of your company and customers), but I was taken with the sense that it was a massive therapy session for the audience. Much like a good psycho therapist, Seth stood up there and gave the audience permission to forget all of the standard ways in which businesses have made a name for themselves.

Forget traditional marketing.

Forget making a mass market product.

Forget competing through parity.

Having recently taken on the responsibility of “managing” the business of Copilot for Fog Creek, I have found myself falling into the trap of trying to do things the way that other companies or my competitors are doing them. We even tried a traditional marketing campaign this summer. The campaign failed in the sense of spending more money that we got back, but it put me in the right frame of mind to hear what Seth, and others at BoS, had to say about growing a software business. More on adwords later, but for now, let it suffice for me to say that there is a right size for things that you plan to sell through search or content network adverts.

I was shocked at finding myself wanting, or perhaps even needing, permission from someone else to act on my conclusions based on years of experience and my own observations. But there I was, sitting on the proverbial couch with a few hundred other people and nodding my head as Seth said that I should act on those instincts and conclusions, and that my Mom probably did love me after all. It wasn’t the answer to my questions, but it was the invitation I was looking for to throw common wisdom in the trash and try something different, and preferably remarkable.