Visually graphing module dependencies

I've just released the first version of prereq-grapher, a script which generates the dependency graph for a module or script in any of the formats supported by Graph::Easy. The functionality lives in App::PrereqGrapher, which in turn uses Perl::PrereqScanner to extract dependencies.

For example, to generate the dependency graph for Module::Path, on Unix I'd run:

  % prereq-grapher Module::Path
  6 modules processed. Graph written to dependencies.dot

And then you can visualise this graph using graphviz:

dependency graph for Module::Path

While writing my review of CPAN modules for getting module dependency information, I knocked up a script to build the prerequisite graph using Perl::PrereqScanner. Last week someone emailed me asking if the script was available, so I put together a release...

Updates (for version 0.04)

The -nc and -nrc control whether and how core modules are included in the graph. The -nc (or -no-core) option says not to include core modules:

  % prereq-grapher -nc HTTP::Client
  2 modules processed. Graph written to dependencies.dot

Here's the output from that:

no-core.png

The -nrc (or -no-recurse-core) option says to include core modules, but don't include their dependencies):

  % prereq-grapher -nrc HTTP::Client
  17 modules processed. Graph written to dependencies.dot

Here's the output from that:

no-recurse-core.png

By default it's now silent when dependencies aren't found locally; if you include the -verbose option you'll get a warning for each dependency not found.

Update 2: limiting depth

Paul Crovella told me he was using this to look at a rather large dependency graph, so I thought it might help to be able to create a subset of the graph, by limiting the depth recursed to. I release 0.05 today, which supports a -depth option.

For example, graphing App::PrereqGrapher produces a huge graph, after processing 305 modules. Here's just a small extract of the full graph:

depth-all.png

So, to get your head around dependencies you might then try:

  % prereq-grapher -d 2 App::PrereqGrapher
  10 modules processed. Graph written to dependencies.dot

This produces a much more useful graph:

depth-2.png

13 Comments

Hi Neil, Good to see this. I started a project to do this, using graphviz etc sometime ago and it's now abandonware: https://sourceforge.net/projects/codewalker/ If there's anything useful/interesting in there, please purloin..
Best regards Hugh Barnard

Nice. It would be good to add two options:

1. Stop recursion when you hit a core module; include the core module in the graph as a "leaf node". (Module::CoreList can tell you which modules are part of core for different versions of Perl.)

2. Stop recursion when you hit a core module; do not include the core module in the graph at all.

Also maybe hard-code some known mistakes Perl::PrereqScanner makes, and exclude them from the graph. e.g. it claims Moo requires Moose, and it's clear to see why it makes that mistake; but probably unhelpful to include Moose in the graph of a lightweight Moo-based module. Though maybe that should be done by Perl::PrereqScanner itself?

Cool!
I just tried it and got some warnings...
No clue why this is but it seems to be your intention?
The picture rendered nicely anyways. Just thought I'll report that since it is uncommon to have warnings for a general user... why not include this libs as dependencies?

$ prereq-grapher EBook::MOBI
can't find VMS::Stdio - keeping calm and carrying on.
at /home/bo/perl5/perlbrew/perls/perl-5.14.2/bin/prereq-grapher line 8.
can't find VMS::Filespec - keeping calm and carrying on.
at /home/bo/perl5/perlbrew/perls/perl-5.14.2/bin/prereq-grapher line 8.
can't find VMS::Feature - keeping calm and carrying on.
at /home/bo/perl5/perlbrew/perls/perl-5.14.2/bin/prereq-grapher line 8.
can't find HTML::FormatText - keeping calm and carrying on.
at /home/bo/perl5/perlbrew/perls/perl-5.14.2/bin/prereq-grapher line 8.
72 modules processed. Graph written to dependencies.dot

Something else you may want to consider as a feature:

Classifying dependencies by type, and augmenting the graph to suit, ie:

Build/Configure/Runtime/Test/Develop dependencies

And "Suggested/Recommended/Required" subsets of those dependencies.

You could opt to include these in the depgraph, and colour/pattern the link based on the dependency type, and changing the link "Weight" based on the "Hardness" of the dependency so that it controls the shape of the generated graph.

Of course, to add these dependency classes, you'll need more than just "intra-module" dependency calculation, and need to reach further for "intra-dist" dependencies, and this will of course need processing of dist-specific metadata resources such as the metacpan api, or the cpanmetadb ( http://cpanmetadb.com/ ) , and the performance negatives of using those datasources will be notable, so its probably "opt in" behaviour, not "opt out".

There is existing modules on CPAN that do something like that, ... but they need the /very/ heavyweight CPANDB ( http://ali.as/top100/index.html ) to work.

I've hacked in this table of dependencies to ignore...

my %FAKERY = (
    'Moo::HandleMoose|Moose' => 1,
    'Moo::HandleMoose|Class::MOP' => 1,
    'Moo::HandleMoose|Moose::Meta::Method::Constructor' => 1,
    'Sub::Exporter::Progressive|Sub::Exporter' => 1,
);

I use that in conjunction with:

if ($FAKERY{ $module . '|' . $dep } or $self->no_core && is_core($dep)) {

Wanted to try it out, but when running it on one of my projects I get the following error: layout did not finish in time. This happened with -svg and -html.

Whose time? I've got all the time in the world for it to generate. It's a project with a big dependency chain (https://github.com/maximos/maximus-web).

Digging into it it seems Graph::Easy has a default timeout of 5 seconds. Perhaps allow setting a modified timeout with a -timeout flag? e.g. -timeout 60 for 60 seconds. You can change the timeout with $graph->timeout($val);

I've modified mine to use a timeout of 1 hour and for Maximus-web it's still running. Wonder when it'll be done :-).

Now someone should create a browser user script to display this graph (or add a link to such graph) in search.cpan or MetaCPAN.

Leave a comment

About Neil Bowers

user-pic Perl hacker since 1992.