Visually graphing module dependencies
I've just released the first version of prereq-grapher, a script which generates the dependency graph for a module or script in any of the formats supported by Graph::Easy. The functionality lives in App::PrereqGrapher, which in turn uses Perl::PrereqScanner to extract dependencies.
For example, to generate the dependency graph for Module::Path, on Unix I'd run:
% prereq-grapher Module::Path 6 modules processed. Graph written to dependencies.dot
And then you can visualise this graph using graphviz:
While writing my review of CPAN modules for getting module dependency information, I knocked up a script to build the prerequisite graph using Perl::PrereqScanner. Last week someone emailed me asking if the script was available, so I put together a release...
Updates (for version 0.04)
The -nc and -nrc control whether and how core modules are included in the graph. The -nc (or -no-core) option says not to include core modules:
% prereq-grapher -nc HTTP::Client 2 modules processed. Graph written to dependencies.dot
Here's the output from that:
The -nrc (or -no-recurse-core) option says to include core modules, but don't include their dependencies):
% prereq-grapher -nrc HTTP::Client 17 modules processed. Graph written to dependencies.dot
Here's the output from that:
By default it's now silent when dependencies aren't found locally; if you include the -verbose option you'll get a warning for each dependency not found.
Update 2: limiting depth
Paul Crovella told me he was using this to look at a rather large dependency graph, so I thought it might help to be able to create a subset of the graph, by limiting the depth recursed to. I release 0.05 today, which supports a -depth option.
For example, graphing App::PrereqGrapher produces a huge graph, after processing 305 modules. Here's just a small extract of the full graph:
So, to get your head around dependencies you might then try:
% prereq-grapher -d 2 App::PrereqGrapher 10 modules processed. Graph written to dependencies.dot
This produces a much more useful graph:
Perl hacker since 1992.
Hi Neil, Good to see this. I started a project to do this, using graphviz etc sometime ago and it's now abandonware: https://sourceforge.net/projects/codewalker/ If there's anything useful/interesting in there, please purloin..
Best regards Hugh Barnard
Nice. It would be good to add two options:
1. Stop recursion when you hit a core module; include the core module in the graph as a "leaf node". (Module::CoreList can tell you which modules are part of core for different versions of Perl.)
2. Stop recursion when you hit a core module; do not include the core module in the graph at all.
Also maybe hard-code some known mistakes Perl::PrereqScanner makes, and exclude them from the graph. e.g. it claims Moo requires Moose, and it's clear to see why it makes that mistake; but probably unhelpful to include Moose in the graph of a lightweight Moo-based module. Though maybe that should be done by Perl::PrereqScanner itself?
Good ideas, thanks. I'll have a look at Perl::PrereqScanner, and see if I can address the problem there. If not I'll add fudge rules in my module :-)
Cool!
I just tried it and got some warnings...
No clue why this is but it seems to be your intention?
The picture rendered nicely anyways. Just thought I'll report that since it is uncommon to have warnings for a general user... why not include this libs as dependencies?
$ prereq-grapher EBook::MOBI
can't find VMS::Stdio - keeping calm and carrying on.
at /home/bo/perl5/perlbrew/perls/perl-5.14.2/bin/prereq-grapher line 8.
can't find VMS::Filespec - keeping calm and carrying on.
at /home/bo/perl5/perlbrew/perls/perl-5.14.2/bin/prereq-grapher line 8.
can't find VMS::Feature - keeping calm and carrying on.
at /home/bo/perl5/perlbrew/perls/perl-5.14.2/bin/prereq-grapher line 8.
can't find HTML::FormatText - keeping calm and carrying on.
at /home/bo/perl5/perlbrew/perls/perl-5.14.2/bin/prereq-grapher line 8.
72 modules processed. Graph written to dependencies.dot
Hi Boris. This is an issue caused by the way Perl::PrereqScanner works, as noted in the documentation. One of the modules in the dependency graph is using something which checks to see if you're on VMS, and if you are, it require's some VMS modules. When parsing the source, Perl::PrereqScanner sees those require statements and assumes they're dependencies.
In the todo list I noted that maybe I shouldn't show these warnings by default. And you've convinced me of that, so I'll suppress them by default in tonight's release.
My next dependency graphing project will be for runtime dependencies, which might be more what you're after. Read my review for more of the gory details :-)
Toby, I just released 0.04, which added your suggestions for controlling inclusion of core modules.
Boris, if you grab 0.04, these warnings aren't shown by default now. The -verbose option will display them.
Something else you may want to consider as a feature:
Classifying dependencies by type, and augmenting the graph to suit, ie:
Build/Configure/Runtime/Test/Develop dependencies
And "Suggested/Recommended/Required" subsets of those dependencies.
You could opt to include these in the depgraph, and colour/pattern the link based on the dependency type, and changing the link "Weight" based on the "Hardness" of the dependency so that it controls the shape of the generated graph.
Of course, to add these dependency classes, you'll need more than just "intra-module" dependency calculation, and need to reach further for "intra-dist" dependencies, and this will of course need processing of dist-specific metadata resources such as the metacpan api, or the cpanmetadb ( http://cpanmetadb.com/ ) , and the performance negatives of using those datasources will be notable, so its probably "opt in" behaviour, not "opt out".
There is existing modules on CPAN that do something like that, ... but they need the /very/ heavyweight CPANDB ( http://ali.as/top100/index.html ) to work.
I've hacked in this table of dependencies to ignore...
my %FAKERY = ( 'Moo::HandleMoose|Moose' => 1, 'Moo::HandleMoose|Class::MOP' => 1, 'Moo::HandleMoose|Moose::Meta::Method::Constructor' => 1, 'Sub::Exporter::Progressive|Sub::Exporter' => 1, );I use that in conjunction with:
if ($FAKERY{ $module . '|' . $dep } or $self->no_core && is_core($dep)) {Wanted to try it out, but when running it on one of my projects I get the following error: layout did not finish in time. This happened with -svg and -html.
Whose time? I've got all the time in the world for it to generate. It's a project with a big dependency chain (https://github.com/maximos/maximus-web).
Digging into it it seems Graph::Easy has a default timeout of 5 seconds. Perhaps allow setting a modified timeout with a -timeout flag? e.g. -timeout 60 for 60 seconds. You can change the timeout with $graph->timeout($val);
I've modified mine to use a timeout of 1 hour and for Maximus-web it's still running. Wonder when it'll be done :-).
Did your graph every finish laying out? If a big enough timeout does produce results, I'll add a timeout option in the next release.
Now someone should create a browser user script to display this graph (or add a link to such graph) in search.cpan or MetaCPAN.
I've released 0.06, which has a timeout option.