Al Newkirk

True story.

Once upon a time there was a web application stack built atop of a particular CI (continue integration) pattern with particular security constraints which I had the pleasure of configuring. The application needed certain secrets (third-party API keys, passwords, trade-secrets, etc) in order to run properly but that data needed to be very well guarded (and definitely not shipped with the codebase). The following is how I tackled the assignment, I think it works well but I could be mistaken. Please feel free to comment with any questions, suggestions, or other feedback.

First, I decide to put all configuration information in environment-based YAML files except for the sensitive info. I decided to store sensitive data in shell environment variables in-memory having and referencing them in the codebase where needed. This particular application uses Puppet for CM (configuration management), the provisioning and continuous deployment process is similar to the following...

A fresh server is booted-up using an image which executes a bootstrapping script which installs a few essentials and prompts for some user input. One of the expected inputs is a password which is used to access the contents of a password-protected 7zip archive which gets downloaded from a private file hosting server. The 7zip archive contains a GPG (GNU Privacy Guard) public and private keyring which is used to decrypt the environment config file which gets downloaded from a private GitHub repository which only hosts the various app configurations. The bash-based environment configuration file is then sourced in-place (e.g. eval gpg -d /tmp/envfile) having the commands within the file executed and certain secrets exposed as environment variables. Certain bits are also copied to disk in-case of failure and reboot (each location is secured).

Having the secrets exposed and a process and workflow which seems to be somewhat secure, I just needed a better way of getting at the complex environment variable names which held the sensitive data. So I decided to write Config::Environment, a module which give you access to app-specific environment variables (with some additional goodies as well).

Thoughts?

Every so often I have a need to debug static web content which is usually within a random directory on my development system. I've created an alias in Linux which uses Python to start an HTTP daemon which serves static HTML content from the CWD (current working directory).

This alias uses Python and a core library to start an http daemon with directory browsing enabled. Very handy.

alias httpd='python -m SimpleHTTPServer'

Being a Perlist, I figured there is probably a way to do this in Perl, but after a few searches and lame attempts at it myself, I haven't been able to figure it out.

The closest I've gotten, after more research than this is worth, was starting the Mojolicious daemon and setting the MOJO_HOME envvar, which doesn't work.

MOJO_HOME=$(pwd) mojo daemon

Having bounced around various web frameworks and documentation, I've finally landed on Plack::App::Directory which does exactly what I'm after, with a little arm twisting.

plackup -MPlack::App::Directory -e 'Plack::App::Directory->new->to_app'

use Port::Authority;

Port::Authority would be a module which assigns and tracks ports for use with multiple application servers running on the same box. It would need an algorithm, capable of generating reproducible results, to generate port numbers in which are available, maybe it could be based on a namespace or random string + current user (etc), or maybe you could be able to specify the type of algorithm (e.g. UUID).

I need this yesterday for a variety of reason, what do you guys think?

#!/usr/bin/env perl

=head1 NAME

App-sourcery

=head1 USAGE

# loads all @assets within $path and reports any not referenced in at-least one of the @sources
./sourcery --path=/var/www --asset=*.css --asset=*.js --asset=*.png --source=*.html

=head1 DESCRIPTION

Often times I'll find myself mucking around with an old webapp codebase that has had assets (css, js, images, etc) added to it over and over throughout its life, and being the neat-freak that I am I like to keep my codebase clean.

This (potentially) is a program that f…

Over the weekend I worked on a new Perl library, MongoDB::QueryBuilder, a tool which is designed to simplify composing complex and dynamic queries for MongoDB in a chainable and object-oriented fashion.

Seldomly do we see examples of querying a MongoDB datastore of significant size and complexity, however, as your application grows and your data evolves, the questions you’ll want to ask your datastore will become more intricate.

For example. Let’s suppose we have an inventory management application that tracks equipment used in our business. Over time the business grows as does the database and (for whatever reason) we want to ask the database to “give us the (serial number, asset type, and the location manager’s name and phone number) closest to one-of-the-three latitude/longitude coordinates we will provide where the asset has been inspected and placed in a bin-location”. Oh, and limit that result-set by 100 documents.

Using MongoDB::QueryBuilder, the abstraction might resemble the following:

use MongoDB::QueryBuilder;

my $query = MongoDB::QueryBuilder->new(
    only => [
        'serial_number',
        'asset.type.name',
        'asset.location.phone',
        'asset.location.manager.name'
    ],
    and_where => [
        'asset.location.bin$ne' => undef,
        '$or' => [ # this condition will short-circuit
            {'asset.location.latlng$near' => [$a, $b]},
            {'asset.location.latlng$near' => [$c, $d]},
            {'asset.location.latlng$near' => [$e, $f]}
        ]
    ],
    limit => 100
);

Needless to say, manually producing the code to query the database would be a bit more involved. Another noteworthy feature is that the query-builder object is chainable which allows you to create classes that will return a result-set.

For example. Considering the aforementioned scenario.

package MyApp::Inventory::Assets;

use MongoDB::QueryBuilder;

has 'query';
sub _build_query { MongoDB::QueryBuilder->new }

sub has_been_inspected {
    my $self = shift;
    $self->query->and_where('asset.location.bin$ne' => undef);
    return $self;
}

sub is_located_near {
    my $self = shift;
    $self->query->and_where('asset.location.latlng$near' => [@_]);
    return $self;
}

sub limit {
    my $self = shift;
    $self->query->limit($_[0] || 100);
    return $self;
}

package main;

my $assets = MyApp::Inventory::Assets->new;

my $cursor = $assets->limit(25)->has_been_inspected;

while (my $asset = $cursor->next) {
    say $asset->{serial_number};
}

.. I rushed through the demonstration, sorry. I’m extremely busy these days. I hope this module will be of as-much use to others as it is to me (at-the-moment).

The end.