A plan for revamping and replacing the Perl debugger.
There is a little bit of overlap here with the first blog on Devel::Trepan. I apologize for that, but I don’t see how this can be avoided. The focus here though is more on the existing Perl debugger rather than Devel::Trepan which is given as one concrete example from which to draw lessons from.
A number of people, myself included, have talked about modernizing or replacing the venerable Perl debugger, perl5db.pl. Here I’d like to suggest a plan of action.
First the situation. It is in many ways similar to the challenges faced say in going from Perl5 to Perl6 — and in other languages Ruby 1.8 to 1.9 or Python 2.7 to Python 3K. What is there is:
- well known
- battle tested
- has lots of features, and is therefore…
- useful …
- used by a large number of people …
- and comes with Perl
That’s the good side. On the bad side, what is there is doesn’t represent current modern programming practice. In particular it is one large 10K file. It isn’t modular in two aspects. First, it doesn’t use Perl Modules or namespaces very effectively. Second, it doesn’t have an API that facilitates extension of its debugger commands or provide an API that allows front-ends to easily hook into it. This is not to say that, people haven’t extended tor written front ends for it. But these all feel like uncoordinated, custom, and heroic efforts.
Related to perl5db being large and not modular, there aren’t many tests for it, especially not the smaller “glass-box” unit tests. Even with the recent activity of adding tests to perl5db.pl, those tests and coverage pale in comparison to those that comes with Devel::Trepan. And to be honest, the coverage in Devel::Trepan leaves something to be desired. But comparing the number of tests does give a feel for how much is probably missing in terms of testing for perl5db.pl.
Finally, perl5db feels to me one-off in its command set. There is no other debugger that I know of that is similar to it to any great extent. Sure, all debuggers allow some sort of stopping, stepping and evaluation of expressions, but that is about where the similarity ends.
There have been some inquiry about using Devel::Trepan or something else as a replacement for perl5db.pl. It appears to me that in the past there have been efforts, and understanding why those have not succeeded might be instructive.
The challenges in replacing perl5db.pl are suggested by looking at perl5db.pl’s advantages. One of the harder challenges that I see is, again, similar to revamping Perl itself. How does one migrate programmers from perl5db given that there are already a number of people using perl5db.pl, either directly or via a front-end and are happy with it and the features it provides no matter how arcane or fragile they may be?
The overall approach I suggest is really two fold and again not unlike what I see for example in Perl5 versus Perl6. One approach is to work on a replacement such as Perl6 or Devel::Trepan. The replacement debugger aspect is what I have been doing. The other is approach to rework the existing debugger to modernize it. (For Perl5 there is perl5-porters which is trying to modernize Perl5). One hopes that the two disparate efforts meet in the middle, hopefully more towards Devel::Trepan or Perl6 side since both ends of the efforts are moving presumably forward.
But how does that specifically apply here to Perl debugger(s)? I’ve offered some specific suggestions on writing tests on the perl5db.pl project proposal.
At some point I should probably write more about how I test Devel::Trepan. I find it relatively easy, although it really needs more explanation as it might not be as self-evident until after the fact.
The other area I think very important is defining more of a debugger API as opposed to a command-line interface. The DB API effort was from a while ago, but it looks like that was also abandoned long ago.
In Devel::Trepan I am pretty convinced I have laid the ground work for using parts of the DB API but equally important isolating and breaking out other modules. There is something called an “interface” that glues together I/O layers so output could very easily be XML (or XML over a socket) rather than less-structured text. And there is a “Command Processor” which has command-oriented part but there are more API-ish things available separated from the specific command-line commands.
An interesting exercise is that someone should be able to replace Devel::Trepan::CmdProcessor so that it handles exactly the perl5db.pl commands using the other parts of Devel::Trepan — the Interface part with its separate I/O routines and the lower-level DB part which sort of subsumes the DB API. To date though, all of the people I’ve mentioned this to have felt that the perl5db.pl is ugly and are therefore are not interested in. However, I suspect there are people out there who don’t want to switch from perl5db.pl.
Right now in Devel::Trepan there is a DB layer that ultimately should be pulled out of Devel::Trepan and turned into a separate package that Devel::Trepan uses. This similar to what perl5db and the DB API both use. This is another area where I feel there should be a “meeting in the middle”. The two reasons I have not separated this out right now is that the DB namespace is special in its powers and that namespace is taken. So I can’t register that. Secondly, right now Devel::Trepan is the only user of this. So breaking this out adds gratuitous complexity for Devel::Trepan users. For example, if I change that lower-level API in an incompatible way, Devel::Trepan will also have to change. And users of Devel::Trepan will have to update two packages where they would have had to update only one when they are bundled together.
Still, if Devel::Trepan becomes more popular or there is a desire in other packages to use some of the lower level routines, this can be changed. For example, DB::LineCache caches Perl source code lines. When there is another package that wants to use that, I’ll make the split.
Let me close this off by listing things that a new debugger should have, some of which has been alluded to above:
- It needs to be modular and represent current programming practice. However it probably needs to handle lots of older Perls as well.
- It needs to work in minimal and hostile environments. And that’s why in Devel::Trepan I don’t use Moose.
- On the other hand it should be extensible so that environments it can use things that many environments might have. For example terminal coloring, nice data structure formatters, disassemblers, interactive shells like Devel::REPL.
- It should facilitate hooking into front-ends like Padre or Eclipse and so on, and out-of-process or out-of-host remote debugging (which for example is useful in debugging web server modules).
- It should facilitate different styles of debugging and debugger entry: non-interactive tracing, or pulling in the debugger in via a signal handler or Josh Ben-Jore’s Enbugger.
I blog about Perl.
Why the need or urge to replace the existing debugger? A better one is great, but there are surely people who use the old one to great effect and know it like the back of their hands. Build a better debugger and people will use it. Or not. Personally I'm happy with inspection, thought, and print statements (as a last resort) but I'm old fashioned.
First, let me say that if you are happy with the existing debugger, inspection, or just using print statements or whatnot, great!
But as for why others want to improve perl5db, see for example this perl foundation project which was approved. So the foundation people must have thought it worthwhile.
As for the pain trying to improve perl5db inflicts on the existing Perl maintainers, see http://www.nntp.perl.org/group/perl.perl5.porters/2012/09/msg191507.html
As for why I wanted a new debugger see my first my first perl blog.
Part of this and the other blog was about why. But another aspect acknowledges the observation that there have been efforts. So given that, what might be good possibilities to accomplish this?
I would like to see a debugger api. I've been working on Vim::Debug and an api would make integrating the debugger with Vim a lot more fun.
I suppose the way to do that these days would be to embed a Plack app in trepan?
Fix:
perl5db.plis not a 10K file. This is a 10K lines file.I'm really pleased to see progress on replacing the debugger with something less monolithic and more flexible.
As for how it's done, I think packaging up the old debugger as a dual life module is a good step so that those who still want it will eventually be able to install it from CPAN.
This is similar to what happened with Devel::DProf when it was superceded by Devel::NYTProf (though that isn't itself in the core).
Let's work together on this. When I browsed the github repository there seems to be a layer of dealing with sockets and parsing text output that should its own separate Module. In fact it would be the same thing you'd use if you were doing a Padre or GNU Emacs front end. In other words, there is a fair bit of code there that has nothing to do with VIM per se.
In the past, I've been promoting DBGp as opposed to the various custom interfaces that are in use in Java, gdb, among others. And in truth right now Devel::Trepan has a custom protocol for remote debugging. But everyone, including myself, who has looked at DBGp feels that the barrier to entry is a little high. And there isn't a simple reference platform to start out with.
That said, there is no reason to not start out with something DBGp-like with an eye towards completing it the way DBGp does. In fact in a sense that's what I've been doing with Devel::Trepan with respect to the gdb command set.
One of the ideas of DBGp is that commands to the remote debugger are text while results that come back are in XML. XML is used (and DBGp was set up in the days before JSON or Data::Dumper) presumably because it is easier to parse fields. However since they didn't want to require on the debugger side an XML library, commands to the debugger were plain text. At any rate, that idea seems useful.
You'll have to educate me though on how to "embed a Plack app in trepan" or what is a Plack app.
Finally, I'll say somewhat off the cuff, that Devel::Trepan prints the message: "That's all, Folks..." when it teminates precisely to make it easy for the front-end to know that it has finished.
Thanks for the correction. perl5db.pl in Perl 5.16.1 has only 9,493 lines; so 10K lines is a little a slight overstatement. But the point I was trying to make (and 10K bytes doesn't do it) still holds: that this is unusually large and to what extent.
Also in fairness to perl5db.pl, I should say that in total lines of code Devel::Trepan is most certainly larger. But the code is split over more than 160 files. And even with this, there are some features that perl5db.pl has that are not in Devel::Trepan yet.
But again, thanks for that correction. I make mistakes which is why I work on debuggers.
You may check Devel::ebug ; it's a module that provide a clean API over the perl debugger.