Usability testing of CPAN modules
Al Newkirk asked me to take a look at his Validation::Class module, and I did.
But instead of asking for the directions or watching the screencast, I tried to figure it out from docs.
And I took notes of my thought process.
The result is this gist: https://gist.github.com/2791412
And here's Al's answer: https://gist.github.com/2791581
So I think it was an interesting experience, and I wish more people did this kind of thing.
Because author can't enter the same river twice; once you're familiar with your project's concepts, you tend to lose the perspective on what will confuse a new user.
You don't have to be a good writer or an experienced developer to provide this kind of feedback. On the contrary, seeing the docs for the first time is what's important to make this process work.
And the author probably won't blame you for being unreasonable -- how can he if you're admitting that you don't understand what's going on and you're just dumping your thought process?
I think if more people did this, opensource community would be a better place, and software/documentation quality would improve significantly.
PS: And if nobody is willing to do this for your project, take your friend who's not familiar with your software, sit right next to him and watch (silently!) how he suffers :)
I wrote Ubic. I worked at Yandex for many years, and now i'm building my own startup questhub.io (formerly PlayPerl). I'm also working on Flux, streaming data processing framework.
CPAN ID: MMCLERIC.
Really great article and message, what we've basically done here is stumbled into our very own open-source quality assurance process, I hope others follow this lead.
Also, for those that look beyond this article, Vyacheslav's critique was of Validation::Class version 7.25 which prompted me to tune and release Validation::Class 7.36.
:)
It's a great idea, but you have to really care about your cpan modules to ask someone to do that.
People who don't care at all won't upload anything on CPAN, I think :)
Hi
Another point to mention when reviewing is: Does the POD have a 'See Also'? For data validation, I think these 3 modules, at least, should be mentioned:
Cheers
Ron
Indeed, this kind of feedback can be really valuable - as an author you're going to be pretty familiar with your own module, so much less likely to spot shortcomings in the documentation.
A query from a new user can often prompt you to think "oh, come on, that's obvious isn't it?", then skim the docs, and realise that, no, it isn't obvious at all other than to you.
Seeing the process people go through when first picking up your module, and which bits they didn't find clearly documented, is valuable indeed.