Software Test Podcast
I've recently been listening to Software Test Podcast. And I was particularly struck by episode 41, in which the presenters interview James Whittaker. He used to be Head Testing Honcho at Google until moving to Microsoft a year and a bit ago.
Almost all of what he says is good news for the perl community. The way we value testing and make it part of the community and how in our day jobs with perl most of us have "testing as an activity not a role", as he puts it, is not common elsewhere, but elsewhere is catching up to what we've been doing.
There is some bad news for us though. Almost all perl code has terrible diagnostics, so if we happen to ship something buggy, it can be awful hard to figure out what's going wrong even if you've got a shiny new test case written for the bug.
Of course, this is where breaking your code into small units with well-defined interfaces, interfaces that you can easily test, comes in handy. We rarely do proper unit testing, and if we wrote our code with unit testing in mind, it would make the sort of exploratory testing that makes it easier to find where a bug is a darned sight easier.
I've been thinking a lot recently about quality control versus quality assurance. And I think that we're still mostly in the world of quality control: we look for bugs, react to bug reports, and we fix 'em. Quality assurance is a much bigger concept. It includes quality control, but that's only a small part of it. Quality assurance is mainly about designing systems and working in such a way that it becomes harder for a bug to escape from your development environment in the first place, and that if it does it is easy to find. And it incorporates feedback. You learn from your mistakes and try to figure out how you can improve your practices so that that class of bug escapes less often. This is real software engineering, whereas most of us still work as artisans.
I'm convinced that unit testing is a big part of quality assurance. Another big part that I'm wrestling with is logging and diagnostics. Logging is, of course, easy to do - we have Log4perl. But doing it is only half - no, only a tenth - of the problem. Figuring out what to log so that the logs can help find bugs is something that we haven't figured out yet. It's something that we could learn from outside our community though, if only we could figure out where to ask for help.