Given that this requires one step that cpan/cpanm won't deal with out of the box, in the interests of documenting for the interwebs ...

Dependencies ... at least on Ubuntu 12.04:

aptitude install xvfb libgirepository1.0-dev pkg-config libgtk-3-dev libglib2.0-dev libglib2.0-0 gir1.2-webkit-3.0

Gtk3::WebKit isn't intended to run "headless", which might be one response to the comments on Web Scraping with Perl & PhantomJS about why you'd want to use PhantomJS instead.

However, having been enlighted by potyl, you CAN run it sorta pseudo headless, by using framebuffer display. And thus, no X server is required.

But this is the bit that's going to throw cpanm and naive installs off. Here's the voodoo that you do:

xvfb-run --server-args="-screen 0 1024x768x24" cpanm Gtk3::WebKit

or, if you're doing it old school

perl Makefile.PL
make
xvfb-run --server-args="-screen 0 1024x768x24" make test

For the benefit of Google et al:

*** Cannot use generic signal marshallers for signal insert-text of Gtk3::Editable unless gobject-introspection >= 1.33.10; any handlers connected to the signal might thus be invoked incorrectly

This one (appears) mostly harmless. You can ignore it. At least as far as installation goes, I've not gotten any further as yet.

(webkit.t:17298): Gtk-WARNING **: cannot open display:
t/webkit.t ..

That's what indicates there is no X Display and hence that you need to "wrap" things with xvfb-run if you want a "psuedo headless" frame buffer. (or run it under an X environment).

Any advice on what the right way to test for this is? Is it just checking that $ENV{DISPLAY} is set? Should that be a test, or in the module code? (I'm outta my depth on this point, so suggestions welcomed & I'll marshal them into a pull request if someone can point me in the right direction).

# Failed test 'use Gtk3::WebKit;'
# at t/webkit.t line 14.
# Tried to use 'Gtk3::WebKit'.
# Error: Typelib file for namespace 'WebKit', version '3.0' not found at /home/minty/perl5/perlbrew/perls/perl-5.16.2/lib/site_perl/5.16.3/i686-linux/Glib/Object/Introspection.pm line 92.
# BEGIN failed--compilation aborted at (eval 4) line 2.
Can't locate object method "new" via package "Gtk3::WebKit::WebView" (perhaps you forgot to load "Gtk3::WebKit::WebView"?) at t/webkit.t line 19.

This was the last hurdle I tripped up on, and is fixed by installing gir1.2-webkit-3.0 debian/ubuntu apt package.

cpan support rocks! potyl is awesome :) I <3 WebKit

Now, back to coding ....

ps. Gtk3::WebKit is, in turn, a dependency for WWW::WebKit, which is ultimately what I'm wanting to play with.

perlbrew install --switch stable

Which expands as:

• Install the "latest stable" Perl - don't make me think about version numbers ;)
• When you're done, switch to it

It is just sugar, but it makes a one-liner out of what used to be:

• Figure out what the latest stable version of Perl is.
  • Think about here: "those outside the echo chamber"
• perlbrew install perl-5.16.2
• perlbrew switch perl-5.16.2

More usefully for my (planned) purposes, I can write documentation targetted to outside the echo chamber that ensures both:

• those reading it will always get the latest stable version of Perl, and
• I won't have to bump the docs every time a new release is rolled.

Go lazy!

:)

    [%-
        SET foo = 0;
        foo = 1 IF 0;
        '--'; foo; '--';
    %]

(If you don't see the answer, click through to the page ;)

----

Fix?

[% IF 0; foo = 1; END %]

Wanna play?

#!/usr/bin/env perl

use Template;

my $tt = Template->new();

my $tmpl = q{

    [%-

        SET foo = 0;

        foo = 1 IF 0;

        '--'; foo; '--';

    %]

};

$tt->process(\$tmpl);

I wonder if this is related to http://davesource.com/Bugs/perl.5.html

Tried on TT 2.22 on Perl 5.10.1, Debian Squeeze stock packages, and TT 2.24 on v5.16.2 built via perlbrew.

(Updated: to use q{} rather than q[] to quote the template code)

• Server A : is the old "Master"
• Server B : is the new "Master" (a rebuild of A) and for a transition period is replicating from A until it is promoted to the main Master and A can be switched off.
• Server C : currently replicates from A, however we want to instead move it to replicate from B in advance of replacing A with B.

Instead, we can stop everything on A, stop A replicating to B, and bring up B as the new master. Faster & less to go wrong & less downtime.

Our problem was that we had a lot of slaves chained off Server C, and relatively large databases that could take ~1 hour to dump/reload. Thus the more normal route of:

mysqldump --master-data master_db > some_dump.sql
rsync some_dump.sql slave:
ssh slave "mysql -u root < some_dump.sql"

would work, but we would need to duplicate this on each of the (many) servers down the replication chain. This seemed ugly.

Some background

MySQL binary log (if enabled) logs all the changes to the database, typically by logging the SQL queries that executed and changed data.

A MySQL Slave has two replication processes. The IO process which connects to the master and copies down data (aka SQL statements) from the masters binary log.

The slave stores these SQL statements in it's local Relay log. The second SQL replication process then applies these statements to the local/slave database.

Aside: recent versions of MySQL can now replicate the SQL statements (the older approach) or the row data.

If you want a single server (C) to act as both a Slave (from A), and then also act as a Master to other downstream slaves (A => C => other slaves), then in addition to

-- tell C to replicate "db_name" from it's master
replicate-do-db = db_name
-- tell C to binary log "db_name" so others can slave from C
binlog-do-db = db_name

and the various other options to tell C what/where it's Master is (which you can either put directly into my.cnf or instead use the CHANGE MASTER sql syntax

... you also need:

log-slave-updates

Without this, Server C won't binary log the statements it receives from an up-stream master. Namely, it will only log queries executed on C. Not queries executed on A and replicated to C. Without this binary logging, C cannot function as a Master to other slaves.

What to do

Stop C (slave) replicating from A (old master). B (new master) is still replicating from A, and thus B is guaranteed to be at the same point, or ahead, of C.

ssh C
mysql -u root
> STOP SLAVE;

Find the last statement that C has executed by looking in C's relay logs.

mysql -u root
> show slave status\G

and look for the lines:

Relay_Log_File: host-relay-bin.000002

that file will by default appear in your MySQL data dir (/var/lib/mysql on debian based OSs) or can be configured with the relay-log option.

cd /var/lib/mysql
mysqlbinlog host-relay-bin.000002

Find the equivalent statement in the new Master's (B) binary logs, the location of which is set by the log-bin option in your my.cnf

The most recent binary log is likely the one in use, but running "SHOW SLAVE STATUS\G" on the server will confirm this. Note that we're looking in B's binary logs, as these are what will be sent to our slave C.

mysqlbinlog mysql-bin.000022 | tail -100

should show something like:

#111115 13:02:48 server id 3102  end_log_pos 46316299   Query   thread_id=94257 exec_time=0     error_code=0
use db_name/*!*/;
SET TIMESTAMP=1321362168/*!*/;
UPDATE `some_table` SET `some_col` = 'some_value' WHERE `some_other_col` = 'some_other_value'
/*!*/;
# at 46316299
#111115 13:02:48 server id 3102  end_log_pos 46316326   Xid = 5346061
COMMIT/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;

In this case, the position we're after is 46316326 (assuming that the last statement to execute on C was the "UPDATE some_table SET some_col = 'some_value'..." one).

Finally, re-configure the Master details for C

ssh C
mysql -u root
> CHANGE MASTER TO
      MASTER_HOST = 'ip.or.hostname.for.B.com',
      MASTER_LOG_FILE = 'mysql-bin.000057',      -- binary log on B
      MASTER_LOG_POS = 46316326;                    -- position in B's binary log
> START SLAVE;
> SHOW SLAVE STATUS\G

Conclusion

This might seem insanely complex, but when you have many active production servers all replicating in non trivial patterns and relatively long "chains" of Master -> Slave/Master -> Slave/Master -> Slave and you want to minimise the downtime ... it does at least appear to work.

It's not really all that complicated -- just a lot of concepts and details (plus a few weird MySQL-isms) to wrap your head around.


my $html = '<p><iframe src="..."></iframe></p>';
my $doc = XML::LibXML->new->parse_html_string($html);
# do stuff with $doc
$doc->toString();


You would end up with:

<p><iframe src="..."/></p>';

Namely, the "empty" iframe tag is going to get output as a single, self-closing tag.

But that's not valid HTML. Not even valid HTML5.

XML::LibXML::Parser has the setTagCompression option but this is no good here.

<hr/>, <br/> and <img src="..." alt="..."/>

cannot be written as:

<hr></hr>, <br></br> and <img src="..." alt="..."></img>

at least, not if you want valid HTML.

I hit upon the idea of appending a single space:

for ($root->findnodes('//iframe')) {

    $_->appendChild(XML::LibXML::Text->new(' '))

        if !$_->hasChildNodes;

}

which works, because an extra space inside a previously "empty" iframe, script or canvas tag would be harmless. However this approach will cause problems when you come across "empty" <textarea> tags.

<textarea></textarea>

is not the same as:

<textarea> </textarea>

The solution that seems to work is:

for ($root->findnodes('//iframe')) {

    $_->appendChild(XML::LibXML::Text->new(''))

        if !$_->hasChildNodes;

}

Namely, append a child node, that represents the empty string. It is enough to convince XML::LibXML->toString() that the node has a child and so it should not try and "compress" the node down to a self-closing tag.

When it comes to emitting that child via toString(), it's an empty textstring, so nothing is output.

ps. yes, I know there are other ways to parse HTML and that HTML != XML.

# http://search.cpan.org/perldoc?DBIx::Class::ResultSet#cache

my $rs = $schema->resultset('Artist')->search(

    {},

    { cache => 1 },

);

my @artists = $rs->all;

pop (@artists);

$rs->set_cache(\@artists);

my $count1 = $rs->count;

my $count2 = scalar $rs->all;

my $count3 = scalar $rs->get_column('id')->all;

Q1: What are the values of $count1, $count2 and $count3?

A1: 9, 9, 10.

Q2: Why?

A2: $rs is a cached result set. Some methods on $rs return the cached results. Some don't and query the underlying db table(s).

Q3: Which ones do & which ones don't?