August 2011 Archives

#Catalyst on @dot_cloud: Adding a #PostgreSQL data service. (#Perl in the cloud, Part IIII)

Following up on my previous post that demonstrated how to get a basic Catalyst application up-and-running on dotCloud in under ten minutes, let’s explore how to take things a step further by adding a database service.

For convenience sake, I’m just going to walk you through the Catalyst::Manual::Tutorial.

However, unlike the tutorial (or most Catalyst tutorials for that matter), we’re going to use PostgreSQL instead of SQLite — and we’re going to deploy the app into the cloud vs. just developing locally (thanks to the magic of dotCloud, which makes it so easy).

Luckily, it looks like the Catalyst::Manual::Tutorial Chapter 3 has been updated with a PostgreSQL-specific appendix, which makes things a lot easier (and means that I can spare you from my terrible SQLite-to-PosgreSQL conversion skills).

Here we go:

  • Following along with the tutorial, we go ahead and add Catalyst::Plugin::StackTrace to the base application module and the Makefile.PL, which ensures it will get auto-installed and built by dotCloud when we push our app. Here’s the commit on Github.

  • Next, we use the Catalyst::Helper script to create a controller for ‘books’ (and a simple test), and update the controller per the tutorial. Commit

  • Then, using the Catalyst::Helper script again, we create a simple view called HTML that will use Template Toolkit as its rendering engine. Finally, we set the component path to let the application know where to find the templates. Commit

  • Last but not least, we create the TT2 template to accompany the /books/list action. Commit

Now we diverge a little bit and head over to the PostgreSQL appendix and create our application’s database for managing books. This assumes that you’re familiar with PostgreSQL, have installed the PostgreSQL server and client and the Perl DBD::Pg module.

  • So, working locally for now, let’s create a user for this application and then a database per the instructions.

  • The data file provided by the appendix had a couple of typos, so I fixed this up here. Use that data file and load up your PostgreSQL database and check that everything loaded properly.

  • Next create the some DBIC schema models with the assistance of Catalyst::Helper::Model::DBIC::Schema

This creates the application’s database model files automatically from the database tables and relationships; see this commit.

  • Now, with the models auto-generated and some data in the database, we need to enable our model in our ‘books’ controller. Commit

At this point, you can check out your application locally to ensure that everything is running. In fact, this is a good point to mention a Catalyst development trick: If you run the development server with script/ -r option, the server reloads when you update an application file. Thus, if there’s an error, you can see it right away. I usually leave this window with the server output visible next to my editing window. Good for caching typos right away.

  • Okay, so finishing up, we configure the HTML view to use a ‘wrapper’ (think header, footer, etc.) for our action-specific views, and we add a CSS file, etc. Commit

  • Even though we’re not going to use them yet, to stay consistent with the tutorial, we update generated DBIx::Class result class files for many-to-many relationships. Commit

  • Update the books/list view template to include authors. Commit

Great. That was all pretty straightforward, so let’s deploy this on dotCloud:

  • Add the additional requirement DBIx::Class to the make file. (In fact, I forgot a few requirements along the way — typical! — so let’s also add: Catalyst::Model::DBIC::Schema, DBD::Pg, Catalyst::View::TT, and MooseX::NonMoose. Curiously, I thought that MooseX::NonMoose should have been built as a dependency of Catalyst::Model::DBIC::Schema, but wasn’t … so I had to add it manually to the Makefile.)

Okay, now for the fun part, let’s add a PostgreSQL data service to our dotCloud instance by adding a couple lines to the dotcloud.yml file (Commit) as described in their documentation on PostgreSQL. Pretty simple, eh?

Now, let’s deploy these new files to dotCloud (note that our Catalyst application and the new data service are not connected yet) with dotcloud push catalyst . and watch dotCloud do it’s incredible magic of installing all of the CPAN modules that your Catalyst app needs. It really is magic.

If all goes well, you should see:

Deployment finished. Your application is available at the following URLs www:

Run dotcloud info and you should see something like:

Now, you just need to connect up your new data service with your app (well, almost, we’ll still need to create the remote database and load it with data). To do that, you can either put the database connection info directly into your lib/MyApp/Name/Model/ file, or read it from the dotCloud environment.json file.

However, at this point, if you put your dotCloud database connection info into your app your local development version is going to complain loudly and will stop being useful as a way to see what you’re doing before you push the app to the cloud. So, this becomes a good opportunity to get our local environment set-up to be as similar as possible as our cloud environment.

On dotCloud, the database connection information is automatically put into a handy environment.json file at the root of our dotCloud environment (/home/dotcloud/). So, to make things easy, let’s also create a environment.json file at the root of our application directory. So, my application root now looks like this:

And I set my local version of environment.json to match the variable names that dotCloud provides, but with my local connection information, like so:

Okay, we’re in the home stretch now! So, to finish things off:

  • To read these environment.json files, we can just add the handy JSON and IO:All modules to the Makefile. Commit

  • Now we can update our Model::DB file to read the environment.json on dotCloud if it exists, or to fall back to our local version if not.

  • We’re all set now to actually create the database (earlier, we simply created the data service). We’ll do that by running dotcloud run -- createdb default-catalyst. Note that this is using version 0.4 of the dotCloud command-line client — future versions might change this format. The .data targets the command to run for the data service that we set-up (vs. the www service running the app). If that all worked, you should see: # createdb default-catalyst

  • Last but not least, we load the data from our local development environment to the cloud database. There are probably other (possibly better!) ways to do this, but I found this approach straightforward: su - postgres and then ./bin/pg_dump default-catalyst | ./bin/psql -h -p XXXXX -U root default-catalyst. Obviously, replace the Xs with the sub-domain and port of your data service.

With all of that done — phew! — we can run one last dotcloud push catalyst . to push the latest changes to into the cloud, install any remaining dependencies, and restart nginx. If all went well, you should see:

Hopefully your PostgreSQL-backed app is now running in the cloud. Hurray!

If you find an error in this post, or have improvement suggestions, please let me know in the comments.

P.S. If your app is not running, the one thing to check that tripped me us is how dotCloud integrates with git. They key take-away is: be sure to commit your changes to git, or dotCloud won’t pick them up! Personally, I found this a bit confusing, and — in the future — I’ll probably use the dotcloud ssh catalyst.www command to do my dotCloud-specific debugging on dotCloud and then manually bring those changes back into the local version and then commit the changes. Without doing that, I ended up with a lot of unecessary commits in the repository as I futzed about with a connection issue.

@dotCloud loves Catalyst apps: Up-and-running in 10-minutes (#Perl in the cloud, Part III)

Lots is happening in the Perl Web framework world these days. The three main frameworks are getting better at a faster-and-faster rate, great screencasts are starting to appear, and — finally — Perl is moving into the cloud, thanks to support from new Platform-as-a-Service vendors like dotCloud.

Now, I’ve been known to kvetch a bit about “Perl in the cloud” once or twice before. But this is not a kvetch. No, no, my friend: this is a “Forget the ode, show me the code” post.

I cracked open an old project this weekend with the intention of getting back to work on it. I thought I’d give dotCloud another try, as I wanted a quick way to have this application interact with OAuth providers (hard to do on my local machine). But I quickly noticed that many of the existing Perl Web framework on dotCloud posts — of which there are several — were a bit out of date, and didn’t reference dotCloud’s new command-line tool or way of deploying services. (Though, I should note, that they’re excellent posts and I steal from them liberally here.)

Anyway, after a bit of poking around, I had my app deployed on dotCloud and wanted to share the newer process for getting a bare-bones Catalyst application up-and-running in the cloud in 10-minutes:

  • I’m going to start by assuming that you have Catalyst running properly on your local development computer. If you’re not at that stage yet, you should probably read the Catalyst::Manual or grab a copy of the rather excellent The Definitive Guide to Catalyst.

  • Once you’ve got that sorted, run App::Name to create the scaffolding for your application. I used Catalyst::Default for this example application, and I’ll use that app name throughout. Catalyst will create the scaffolding in a directory called App-Name — so, in my case, that’s Catalyst-Default.

  • Change your working directory to the app that you’ve just created, in my case that’s cd Catalyst-Defualt. Let’s call this your app’s root directory for convenience to this walk through.

  • Next, you’ll want to link your root/static directory from the app’s root directory, because that’s where dotCloud will look for static files. You can do this with a simple ln -s root/static static.

  • Then you’ll want to set-up support for PSGI. To do that, you can simply:

  • Now edit the ‘app.psgi’ file and add the line use lib 'lib' after the use warnings line. Here’s mine:
  • At this point, if you’re aiming to deploy a real application, you should update your Makefile.PL with the modules, plugins, and so on that are required by the application. By doing this, dotCloud is able to install all the required Perl modules from CPAN automatically. This is the most impressive part of dotCloud to me — it’s simply amazing to watch how it handles complex dependency chains without breaking a sweat.

  • If you’re just going to deploy the default Catalyst application that is built by the scaffolding to follow along, you’ll want to add requires 'Catalyst::Engine::PSGI'; to your Makefile.PL. Add that after the other lines that start with requires.

  • Now you’re ready to run your Makefile.PL (perl Makefile.PL). Without this step, dotCloud won’t be able to parse your Makefile.PL and you’ll be stuck updating the dotcloud.yml with your dependencies. Not the end of the world, but create unnecessary redundancy.

  • Now we’re ready to let dotCloud know about our application. First, you’ll need to download and install the dotCloud command-line client.

  • Use the dotcloud create appname command to create your app. In my case, I just used dotcloud create catalyst. (dotCloud doens’t seem to like names with any special characters, so you’ll need to choose something like ‘catalystdefault’ or ‘catalystappname’.)

  • You’ll need a dotcloud.yml file to tell dotCloud about the service your app requires — e.g., Perl, Python, Ruby, etc. — so fire up your favourite editor and open up dotcloud.yml and add these lines:

  • Save that file in the root directory of your app. At this point, your app’s root directory should look like so:
  • Now you’re ready to deploy your application into the dotCloud cloud. Drum roll please…. So, to do that, simply run dotcloud push appname ..

At this point, you’ll want to go grab a coffee or beer or something because, if you’ve done everything right, the dotCloud build system will review your Mekefile.PL for dependencies and start installing Catalyst in your cloud instance so that your app can run properly. If that’s happening, you should see something like:

At the end of the process, you should see this line:

Visit the URL that dotCloud provided in your browser and, if you’re lucky, you should see this.

If you get a 404 or some other page, there was a problem along the way. To troubleshoot, just run dotcloud logs catalyst.www (replacing ‘catalyst’ with the name you gave your app on dotCloud) — the ‘www’ indicates the name of the service you created in your dotcloud.yml. (We’ll dig a bit more into the different services in the next post.)

Let me know in the comments if you have any questions, if anything above is unclear (or could be clearer), or if you run into any problems. I’m not a dotCloud expert by any means, but I am starting to get my head around it.

Next up: Setting up a PostgreSQL data service.

About Phillip Smith

user-pic Phillip Smith is a digital publishing consultant, online advocacy specialist, and strategic convener. And I blog about Perl.