3Party - cpan modules third party testing

Last time I have been deeply involved into test automation development. So some projects was born like swat, outthentic and finally sparrowhub.

After some mediation on the future of my projects I am still committed to the idea of sharing such a tests with the rest of perl/cpan community.

An issues arise on my way though:

  • people used to stick to convenient unit tests kept under their cpan distributions and are half-heartedly for third party tests alternative which of course is respected by me.

  • (IMHO) integration tests are hard to write and maintain, this is probably why people prefer to write unit tests, and as swat is kind of integration test tool, it could be "neglected" for the reason above. I believe still that integration tests are essential part of development.

Taking into account all of this I am introducing an idea of third party tests service - 3Party ( working name, just can't invent something better right now ) with the following features:

  • tests are written on swat or outthentic frameworks

  • cpan modules to test are randomly chosen and tests are provided

  • if one want to force your cpan module to get tested via 3Party - welcome! we could discuss it

  • a cpan modules author receive test reports via email or through 3Party web interface

  • tests triggered at every cpan module release so they could be considered as regression base

  • no warranty for full coverage of cpan module API, but test base could be increased eventually as far as tests contributors have a free time on it ( or new tests contributors join the team )

  • now tests contributors team is me only but new developers are welcome

  • tests relies on cpan module public API provided by it's documentation, in case false test issue a cpan module author is encouraged to help test author to correct tests behavior, but it is only guaranteed that test author will follow cpan module documentation and try to implement things based on his understanding of such a documentation

This only an informal draft of the idea.

I need a community feedback before to dive into this project.


Alexey Melezhik


Parts of this sound very similar to the CPAN Testers project. What makes this different? Could it be added to CPAN Testers?

Leave a comment

About melezhik

user-pic Dev & Devops --- Then I beheld all the work of God, that a man cannot find out the work that is done under the sun: because though a man labour to seek it out, yet he shall not find it; yea further; though a wise man think to know it, yet shall he not be able to find it. (Ecclesiastes 8:17)