DBIx::Class populate Gotcha

Today, we watched a series of test cases failing. Our tests started with a blank database and a series of populate method-calls to create their own test records.

Everything looked obviously right, but after a while of debugging, we found out that the populate method of DBIx::Class::ResultSet behaves differently depending on the context used. In Scalar or List context, the records are inserted using the INSERT statement, while in void context, a bulk insert method is used.

From a performance standpoint, this absolutely makes sense. The documentation does explain this very well.

However, some optimizations have been made under the hood that are not obvious unless you know about -- I did not know that before.

My code looked like this:

    MyApp->model('DB::Element')->populate([
        {element_id => 113, name => 'f4'},
        {element_id => 114, name => 'f5', usage_starts => '2000-04-01'},
    ]);

And the statement sent to the database was:

INSERT INTO element( element_id, name ) 
  VALUES( ?, ? ) : '__BULK_INSERT__'

Every column which was not mentioned in the first record got lost and simply was ignored.

Conclusion? A bulk insert can only operate on a well known set of columns. The logic behind DBIx::Class retrieves the set of rows from the keys of the first hashref. This is important to keep in mind when using this syntax.

Solutions? TIMTOWTDI :-) If speed is not important, a populate method call could easily put into list context forcing the usage of single insert statements that do what you expect.

    # force list context, ugly but usable :-)
    () = MyApp->model('DB::Element')->populate([
        # ...
    ]);

5 Comments

To be honest, this smells like a bit of a premature optimization. IMHO DBIx::Class should, by default, play it safe and only do the bulk insert if

a) You told it to use the first record to grok all.

b) It scanned the records and knows it's safe.

Absolutely. This is one of the more retarded design decisions (which tend to range from mildly to very retarded :) that happened during the early years. Unfortunately it's not something that can be easily killed, so is there to stay.

@Wolfgang - this smells like a bug to me. Which version of DBIC are you using?

The fact that you can live with it doesn't mean it is not an actual bug :) Should be fixed in next rel. Cheers!

Leave a comment

About Wolfgang Kinkeldei

user-pic I blog about Perl.