The Great Re-write of 2010: CGI::Session/Test::Database/DBIx::NamedDSN

Hi Folks

I have just finished a re-write of CGI::Session, called Data::Session, and started wondering how best to test it.

I've always disliked setting all the env vars to make its tests work for MySQL, PG, etc, so I hit CPAN looking for A Better Way.

I found Test::Database, which seemed to be a good idea. But examining the code was discouraging, so I decided to re-write that too.

I soon realized that one thing (among many - see below) which bugged me about Test::Database was its handing of DSNs.

So I decided to split that out into a separate distro, using an INI-style file, so I could add extra attributes to the DSN, such as active and use_for_testing. I might add use_for_production too.

That's why I've just released to CPAN DBIx::Admin::DSNManager V 1.00.

Now I'll use that in the re-write of Test::Database, which I'm going to call Test::Admin::Database.

I don't want to reuse the namespace Test::Database, since the aims of the modules are a bit different.

Then finally the latter can be used to help test Data::Session which I'll release as V 0.90 at first.

Pain with Test::Database:

1) Test::Database does not install with cpanm. First you have to install DBD::CSV V 0.31 twice. Then it installs.

2) Also, the docs for Test::Database are very hard to fillow. For instance, under Description,
it talks about install leading you through configuration, and writing ~/.test-database. That does not happen. Perhaps it did once upon a time.

3) Also, in Test::Database::Tutorial, there is a talk about the difference between a dsn and a driver_dsn, but the code does not follow the docs.

For example, when I used a dsn (in ~/.test-database) for Postgres, with a user who has createdb rights, that dsn is ignored. But if I use dsn_driver, it is used.

4) Also, if I put a dsn for SQLite in ~/.test-databas, to see what happens, I get 2 handles to SQLite. Perhaps that's a good idea, but it should be documented.

5) Also, Test::Database does not use File::HomeDir - it should.

6) Also, when I remove dsns from ~/.test-database, the module does not rewrite /tmp/Test-Database-ron//Pg/mapping.yml, which is then left with dsns I don't want to use.

6 Comments

Now I have not seen your new code yes, but the old one in my experience was rather unfortunate in the sense that it did not cater for the case that two instances were accessing the session concurrently and updated different values in the session ...

A - open session
B - open session
A - write prop A
B - write prop B
A - close session
B - close session

would result in prop A being lost ...

All of those problems in Test::Database sound like bugs, not fundamental design problems. Can’t they just be fixed?

The main value of something like Test::Database is if (close to) every distribution uses it, because that lets users configure their test database credentials once and have all their module installation attempts use that configuration. If there are 15 different modules for supplying database credentials to tests, then the situation is no better for users than if every distribution were looking for some kind of environment variable.

So please consider whether you can somehow fix T::D instead of going off in an entirely new direction.

Have you talked with BooK about any of this?

Before replying to your points, I must say Test::Database has been looking for users for a long time, and I'm sad to say that I have had very little return on the topic. I have tried very hard to make the module pass all tests on a number of platforms/databases, and have talked a lot with a number of people to try and provide something useful to most.

The design goal of Test::Database is to allow people who try to write code that is portable across database drivers to use the power of CPAN Testers. It is not a DSN configuration manager.

Now, quick replies to your points:

1) Test::Database does not install with cpanm. First you have to install DBD::CSV V 0.31 twice. Then it installs.

Yes, there have been issues with DBD::CSV, which I tried to fix. I don't think I found time to fix all the issues, even thought DBD::CSV's author fixed a few things because of Test::Database.

2) Also, the docs for Test::Database are very hard to fillow. For instance, under Description, it talks about install leading you through configuration, and writing ~/.test-database. That does not happen. Perhaps it did once upon a time.

Yeah, that is actually a quote, back from early discussions on the topic. A bit of name-dropping too. Anyway, even though I planned to have some way to automatically create the file, this will probably never happen. I should fix the Description to match reality, indeed.

3) Also, in Test::Database::Tutorial, there is a talk about the difference between a dsn and a driver_dsn, but the code does not follow the docs.

For example, when I used a dsn (in ~/.test-database) for Postgres, with a user who has createdb rights, that dsn is ignored. But if I use dsn_driver, it is used.

Actually, the doc is incorrect. What it meant, is that Test::Database will only create databases with a driver_dsn.

In Test::Database, a dsn is a place where you're implicitly allowed to do anything up to CREATE TABLE and DROP TABLE. A dsn_driver assumes you can also CREATE DATABASE and DROP DATABASE.

4) Also, if I put a dsn for SQLite in ~/.test-databas, to see what happens, I get 2 handles to SQLite. Perhaps that's a good idea, but it should be documented.

File-based handles are automatically provided if the DBD is available.

5) Also, Test::Database does not use File::HomeDir - it should.

Actually, it does. Since March 2009.

6) Also, when I remove dsns from ~/.test-database, the module does not rewrite /tmp/Test-Database-ron//Pg/mapping.yml, which is then left with dsns I don't want to use.

This looks partly as a bug. Of course, rm -rf /tmp/Test-Database-ron/ won't hurt.

3) Also, in Test::Database::Tutorial, there is a talk about the difference between a dsn and a driver_dsn, but the code does not follow the docs.

For example, when I used a dsn (in ~/.test-database) for Postgres, with a user who has createdb rights, that dsn is ignored. But if I use dsn_driver, it is used.

After re-reading the Tutorial, I think the documentation explains exactly what the code does.

Basically, Test::Database is meant to provide database handles, not configured in any way, to let your test scripts do their thing.

It will return the same dbh to several successive test scripts running in the same cwd, thus allowing a test script to setup the database that the following tests scripts will use. The ability to create a database is only needed because Test::Database would prefer to create a database per distribution being tested.

Providing a dsn in the configuration is allowing any test script to do anything there, but has the drawback that several test script (possibly running in parallel) will access the same database handle. Not really recommended for a CPAN Tester setup.

4) Also, if I put a dsn for SQLite in ~/.test-databas, to see what happens, I get 2 handles to SQLite. Perhaps that's a good idea, but it should be documented.

That behaviour is actually documented in Test::Database::Tutorial:

So, without any configuration, Test::Database will only be able to provide file-based databases. It is also recommended to not put DSN or driver information for the file-based database engines that have a corresponding Test::Database::Driver class, since it will cause handles() to return several handles for the same database engine.

Leave a comment

About Ron Savage

user-pic I try to write all code in Perl, but find I end up writing in bash, CSS, HTML, JS, and SQL, and doing database design, just to get anything done...