Visualizing a CPAN install plan
At the patch -p1 Perl hackathon, I'm working on bits related to my idea for a new sort of CPAN client. One of the features I want is a graphical display of the work to do and the work being done.
Creating the data structure in Perl is easy. Start with a module, figure out it's distribution, then grab that META.{json|yml} to get dependencies (for now, with more discovery later). Put that stuff into a data structure.
But I want to display that in a web browser (so I don't have to write a GUI that won't work everywhere). I've played with the D³ Javascript plotting tools, so I started with that today, and after some head banging, I got it to work. I have a demonstration for HTTP::Size. It's not that fancy. It is a directed graph, but I haven't done the work to use arrows. I haven't done any work to add individual styling and other fancy things. I haven't integrated any knowledge of the local installation and which dependencies are locally satisfied. I'm just happy something shows up.
I should be much further along, but I wasted a lot of time in the JavaScript. I'm so used to Perl that I started with a hard-coded structure that had the relationships:
That's mostly what I have in Perl, and in Perl it's easy to play with that. In JavaScript, not so much. I have to ensure that each of those relationships create nodes in the graph, and that if I've seen a module name before (many things depend on Test::More, for instance), that they that it reuses the existing node. The words "before" and "existing" go with a hash that I can use to track what I've done and look at a key to get a node reference. That's not easy in JavaScript, which has objects but not native hashes. There are various ways to get that, and most involve loading heavy libraries. StackOverflow has many questions and answers on this subject.
I fooled around with that for too long, then Sawyer X, sitting across from me, said "Just make the data structure you need in JSON in Perl!". Of course! That's what I do for things like Template already. Instead of doing a lot of work in the View to massage a data structure, give the View something ready for it to use in the way it wants to use it. The new data structure has a list for all the names, and the links pre-computed as their list indices.
This means that I end up writing much less JavaScript, not only because I'm not massaging the data in the client, but also because it's easier for me to change the data structure in Perl if I change plotting libraries (a likely scenario).
My next step is to play with annotating that data structure with module info and exploring various ways to mutate it on the fly, as well as styling the lines and circles to mean various things.
I'm the author of Mastering Perl, and the co-author of Learning Perl (6th Edition), Intermediate Perl, Programming Perl (4th Edition) and Effective Perl Programming (2nd Edition).
I've always wondered why CPAN does not at least show a full dependency list (what will be pulled in, not necessarily the full dependency list) similar to the way that yum does prior to launching into what can turn into a bit of nightmare of a dependency chain. Several especially highly dependent modules come to mind...
I'm guessing it was designed for an era when network connections were slower, and modules had fewer dependencies and/or smaller, faster test suites. Figuring out the deps requires a bunch of sequential network requests for META.yml or similar, which would take a long time over a modem.
CPANdeps works for now, but having an option for the CPAN client to do this would be nice.
> I've always wondered why CPAN does not at least show a full dependency list (what will be pulled in, not necessarily the full dependency list) similar to the way that yum does prior to launching into what can turn into a bit of nightmare of a dependency chain.
Meta/mymeta file support in CPAN clients is a fairly recent innovation. Before those times, the system just didn't know at all what the dependencies are until in ran «perl Makefile.PL», which made planning more than one step ahead difficult. Nowadays it can do that for modules that have dynamic_config=0, but that's not the default for ExtUtils::MakeMaker or Module::Build.
Also, configure_requires can make this even more complicated than it was previously.
Please just make it part of MetaCPAN instead of creating just another Perl-module-overview related portal or client.