Testing is one of those things that I would not only be bad at, but I would rather loath to do it. The infinite detail of what the task entails along with the endless permutations that are possible makes testing seem like an impossible task to me. I really just can’t focus for as long as it takes. Thankfully, I’ve always had people who could actually execute the tests for me instead of actually being forced to do more than a few myself.
In a recent round of user testing, I was astounded by the differences in participation rates based on the total preparation before the testing sessions. For one group of testers, blind initiations were sent out having not contacted the testers beforehand in any way. Obviously, the participation rate for these invitations was far less than 50% – I actually think it was closer to 25%.
For a second group of testers (in a different country), testers were initially contacted to briefly explain the request before invitations were sent to those who had time. After the invitations were sent, follow-up calls were done to revalidate who was showing up and why we needed them. The day before testing, we sent out emails with the materials they would need. Obviously, the participation rate for these testers was far higher. Given that we asked testers before sending calendar invites, you can imagine that we had a 100% acceptance rate. (ok, there were 3 tentative). What is surprising though is that we actually had a 100% show rate on over 30 testers, which I think is unheard of.
All I’m really saying here is that when most of us are doing testing for a new implementation, we go through a lot of effort to execute it and get feedback. UAT is especially important because it’s your last opportunity to create engagement with a small population of end users, and get change management feedback to help target messaging. There would seem to be a fairly minimal amount of effort to get great feedback versus doing slightly less work but getting pretty mediocre results.
