DBD::mysql - all your UTF-8 bugs are belong to us!!□□

After a couple of years of more or less "maintenance mode" on DBD::mysql - we had a hand full of people contributing occasional fixes and a whole slew of drive-by contributors - we now have a prolific contributor again: Pali Rohár.

It's great to see some more long-standing issues taken care of!

This time around, in the new development release 4.041_01 that is on CPAN now, there are some important fixes for some Unicode-related issues that I would like to point out. The sections below I have distilled based on the descriptions made by Pali.

Automatically converting to UTF-8 for bind parameters

Before this release perl scalars (statements or bind parameters) without UTF8 status flag were not encoded to UTF-8 even if mysql_enable_utf8 was enabled. This caused perl scalars with internal Latin1 encoding to be sent to the mysql server as Latin1 even if mysql_enable_utf8 was enabled.

Now all statements and bind parameters which are not a DBI binary type (SQL_BIT, SQL_BLOB, SQL_BINARY, SQL_VARBINARY or SQL_LONGVARBINARY) are automatically encoded to UTF-8 when mysql_enable_utf8 is enabled.

If mysql_enable_utf8 is not enabled and your statement or bind parameter contains a wide Unicode character then DBD::mysql shows a warning. If a binary parameter contains a wide Unicode character then DBD::mysql shows a warning too, similar like function print without using a :utf8 perlio layer. ("Wide character in...")

Perl's SvPV() returns char* from a perl scalar and the following SvUTF8() call for that scalar returns true if SvPV returned the data in UTF-8 or Latin1.

Decoding of UTF-8 fields when mysqlenableutf8 is enabled

For each fetched field mysql server tells us its charset id. Before this release when mysqlenableutf8 was enabled DBD::mysql UTF-8 decoded all fields with a charset id different than 63 (which means binary).

Now DBD::mysql UTF-8 decodes only those fields which have their charset set to utf8 or utf8mb4. By default mysql server sends data in encoding specified by the SET NAMES command, which is by default Latin1. So any received Latin1 data is not UTF-8 decoded anymore.

The mysql server sends a charset id, not a charset name. Each combination of charset name and collation pairs has its own charset id. A new function charsetnr_is_utf8() has hardcoded all utf8 and utf8mb4 charset ids from mysql (up to 8.0.0) and mariadb (up to 10.2.2) from their source code. So far it looks like those ids are not changing since old mysql 5.0, only new ones are added.

Conclusion

We hope these changes make DBD::mysql a lot more consistent for you. Since the changes are rather big, we'd urge you to test the development release 4.041_01 which is on CPAN and give feedback NOW; this allows us to make changes if needed before we create an actual stable release with these features.

And of course, if you test it with your software and all is good, we'd like to hear that as well!

You can leave your feedback via the DBI-users mailing list, or using our GitHub page.

Security release - use after free in DBD::mysql when using prepared statements

DBD::mysql is the perl DBI driver for MySQL and the primary way Perl applications and scripts access MySQL and MariaDB databases. The source repository is at https://github.com/perl5-dbi/DBD-mysql.

A vulnerability was discovered that can lead to a use after free when using prepared statements. This vulnerability is present in all releases at least back to versions 3.0 of the driver, which were released in 2005.

The CVE identifier for this vulnerability is CVE-2016-1251.

Version 4.041, including the fix for this vulnerability, is available on CPAN at https://metacpan.org/pod/DBD::mysql

Users of DBD::mysql with prepared statements are advised to patch their installations as soon as possible.

Many thanks to Pali Rohár for discovering and fixing the vulnerability.

The DBD::mysql maintainers, Patrick Galbraith Michiel Beijen

SECURITY RELEASE - Buffer overflow in DBD::mysql perl library

DBD::mysql is the perl DBI driver for MySQL and the primary way Perl applications and scripts access MySQL and MariaDB databases. The source repository is at https://github.com/perl5-dbi/DBD-mysql.

A vulnerability was discovered that can lead to a buffer overflow, possibly triggered by user supplied data. This vulnerability is present in all releases at least back to versions 3.0 of the driver, which were released in 2005.

The CVE identifier for this vulnerability is CVE-2016-1246.

Version 4.037, including the fix for this vulnerability, is available on CPAN at https://metacpan.org/pod/DBD::mysql

Users of DBD::mysql are advised to patch their installations as soon as possible.

We have already made a pre-announcement for this security release at the distros security mailing list. People using DBD::mysql installed from their (linux) distributions can expect to receive an updated version soon.

Many thanks to Pali Rohár for discovering and fixing the vulnerability.

The DBD::mysql maintainers, Patrick Galbraith Michiel Beijen

Test release for DBD::mysql available - ssl by default

Dear Perl and MySQL community,

We're pleased to announce the release of DBD::mysql 4.033_01, the Perl DBI driver for MySQL and MariaDB databases. This is not a 'stable' release but merely for testing and feedback. We'll put out a stable 4.034 release soon; probably before christmas.

Linking against SSL by default?

Apart from that, I'd like to announce that we might want to link to SSL by default. MySQL 5.7 makes SSL connections to databases more common; right now in DBD::mysql you'd need to pass an option to Makefile.PL (--ssl) in order to enable linking to libssl. Of course, many people (and linux distributions!) don't do this by default. On the expense of the added dependency to libssl we'd want to default to compiling against libssl. We'd introduce a --nossl flag for the cases where you'd explicitly NOT want to link to SSL. When DBD::mysql is compiled against libssl you can still make connections to not-sslified servers.

Any feedback (+1's, remarks or objections) would be appreciated!

Changelog for this development release

2015-12-15 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.033_01)

  • Raise minimum DBI version to 1.609 (from 2009!) in order to make tests pass on RHEL5.
  • Add explicit documentation for how to enable SSL at build time, provided by genio.
  • Improve test suite to make it more robust using mixed versions of client and server.
  • Fix use after free error in my_login, provided by hannob.
  • Add explicit instructions stating brackets are required around ipv6-addresses in connection strings, provided by Kenny Gryp.

Release on CPAN: https://metacpan.org/release/MICHIELB/DBD-mysql-4.033_01

UPDATE, UPDATE, read all about it!

After feedback we've decided to make the jump, which was probably long overdue, and link to SSL by default as proposed above. There's a new development release with these changes on CPAN.

2015-12-18 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.033_02)

  • Compile against libssl by default. This allows to connect against remote MySQL servers using SSL. Previously this was only achieved with an explicit switch provided to Makefile.PL - if for some reason you can't or don't wantto link against libssl, you can use the new --nossl switch to Makefile.PL.
  • Made tests more robust after CPAN Testers failures.

Release on CPAN: https://metacpan.org/release/MICHIELB/DBD-mysql-4.033_02

Github repo: https://github.com/perl5-dbi/DBD-mysql

Regards,

Patrick and Michiel

A little nicer way to use smartmatch on perl 5.18

5998623964_f1a4023855_n.jpg

Of course as Perl developers we all love new features, don't we?

So the moment we could work with perl 5.10 we all started using smartmatch, right? If not for the only reason it allows us to write elegant code like this:

use v5.10.1;
@array = qw ( Thom Jonny Colin Ed Phil );
say "I found Phil!" if 'Phil' ~~ @array;

But now we have perl 5.18 and some of the ideas of smartmatch turned out to be a little too smart, and so we now consider it an experimental feature. So even code like this, when executed on a 5.18 perl, gives warnings:

Smartmatch is experimental at smart.pl line 3.
I found Phil!

brian d foy wrote about how to stop these warnings, but it's not pretty:

no warnings 'experimental::smartmatch';

This works under perl 5.18 but gives nasty error messages under older perls:

Unknown warnings category 'experimental::smartmatch' at smart.pl line 3.
BEGIN failed--compilation aborted at smart.pl line 3.

so the 'best' way to do it is like below:

no if $] >= 5.017011, warnings => 'experimental::smartmatch';

Ugly, right? That is why Leon Timmermans created experimental, a CPAN module that allows you to simply write:

use experimental 'smartmatch';

That's much better! I can remember that, and it's readable. I hope you like it as much as I do!

Of course this does not alleviate the problem that smartmatch is now considered experimental, which means that its implementation is probably going to change in upcoming perls.