How should I write a test suite which depends on an external server?
I'm in the process of writing a test suite for ElasticSearch.pm, but in order for it to run any tests, it requires access to an ElasticSearch cluster.
Currently, I just skip all tests unless $ENV{ES_SERVER} is set, but this requires manual installation / testing.
Alternatively, I could (if $ENV{ES_SERVER} isn't set) try to download and compile a test version, which requires git and java v 1.6 or higher. It doesn't take long to compile, but long enough so that user may not want to do it by default.
So I could ask them if they want the script to build a test server, but again, this requires manual installation.
It'd be nice to use the cpan testers to run the test suite on multiple platforms, which implies building a test cluster by default.
What would you do?
The doctor will see you now...
IMO, your current option is the most sane. It's the same approach taken in DBIx::Class for any of its DB backends -- you need to set DSN, username and password env vars for them to run. I've done the same thing for live.t in WebService::Solr.
I think the download + compile option is far to heavy for practical usage.
A third option would be to create a "mock" server. It would return static, pre-defined responses without the need to have a real ES cluster.
Thanks for the comment, Brian
Lars Dɪᴇᴄᴋᴏᴡ emailed me with this suggestion, which sounds pretty reasonable to me:
---------------------------------
Yes, do try to download and build ElasticSearch if missing; put that
functionality in a seperate distribution Alien::ElasticSearch. In
ElasticSearch's Build.PL, prompt (see thread
for the correct way) to do
depend on Alien::ElasticSearch if an ES server is missing. The prompt default
must be 'yes'. That way automated testers try to cover the greatest extent
possible. People doing manual installations can choose to skip it easily.
---------------------------------