Perl in a Monorepo
Without weighing in to the pros and cons of using a Monorepo approach to your organizations codebase, I am interested in hearing about tools and approaches that have been used with Perl.
For example, I have found that Bazel has Perl support which seem fairly actively. I wonder if there is anything that can integrate with Dist::Zilla? Or any way of managing pulling third party code?
Experiences with CI/CD in the normal Git hosting platforms are also of interest - although it does seem like Github and Gitlab are designed around death-by-repo - and I have seen some features to vary the "actions" behavior based on whats changed. I am however just as interested in if you have had experiences with other platforms please chime in!
Fwiw I realize that perhaps Git isn't the best for Monorepos (although you could argue that the Linux Kernel is in a monorepo) and I have no issue with current alternatives and upcoming ones.
Any plugins that can help? For anything mentioned or not.
Totally open ended question. Please comment!
I blog about Perl. I am now in California
The one big issue with monorepos is the
lib/directory. On an active and long-lived repository, it can become pretty unwieldy when you manage thousands of Perl modules in your repository.One side-effect of using a tool like Bazel is that it introduces a build step in your workflow, and forces you to think about dependencies. You can't just copy the your repository checkout on your production machines (and all internal dependencies are available by virtue of having been copied there) any more: you build an artifact, and you deploy that.
If you have multiple independent services sharing libraries, you only want to deploy their actual dependencies from the repository. The unfortunate consequence of deploying your repository checkout is that everything in the repository may depend on anything else in it, leading to an absolute dependency mess.
That's a fascinating insight.
The thought hadn't occurred that someone might dump everything in a single lib/ (although thinking back I have seen that before on a shared nfs which shared a similar function).
Having come from an organization that uses Buck (like Bazel) the layout is sort of like below and my assumption was to follow a similar pattern.
company/lib
org/common/lib
org/team/common/lib
org/team/thing/lib
third-party/thing/lib
Or perhaps something like looks more like what metacpan-cpan-extracted (https://github.com/metacpan/metacpan-cpan-extracted) looks like.
Could you recommend a path layout that works well?
Yeah, you can basically map Bazel packages one-to-one with CPAN-like "distributions".
The layout of a package/distribution setup could be something like:
Each distribution is self-contained, and the Bazel BUILD file lists its dependencies.
This is also where a slight impedance mismatch starts to become visible.
On the one hand, since Bazel is a build tool, the graph of package dependencies must be a DAG (i.e. a tree).
Perl, on the other hand, has no such constraints: modules can depend on other modules that depend on them, and the cyclic dependency is resolved at load time, by loading one before the other and hoping all goes for the best. It's likely the same for distributions.
That's a significant insight, thanks. Could you point to any other open source tools that have been useful?