Sparrowdo - a simple configuration management tool written on Perl6 and consuming sparrow plugins

Hi!

This is the very first ( to take it for a spin ) release of sparrowdo - a simple configuration management tool written on Perl6 and consuming sparrow plugins.

This is where Perl5, Perl6 could make a synergy. Consider a simple example of sparrowdo scenario:

$ cat sparrowfile

use v6;

use Sparrowdo;

task_run  %(
  task => 'install my packages',
  plugin => 'package-generic',
  parameters => %( list => 'git-core carton cpanminus' )
);

task_run  %(
  task => 'install psgi app',
  plugin => 'perl-app',
  parameters => %( 
    'app_source_url' => 'https://github.com/melezhik/web-app.git',
    'git_branch' => 'master',
    'http_port' => 3030
  )
);

All it does is:

  • installing some packages required for further deployment ( git, cpanm, carton )
  • installing perl5 psgi application by fetching source code from remote repository, installing dependencies with cpanfile/carton and running service using Ubic and Starman.

Here we go. On my vagrant Ubuntu 14.04 Trusty box I will have:

$ sparrowdo --ssh_user=vagrant  --ssh_port=2200 --host=127.0.0.1  --verbose 
running sparrow tasks on 127.0.0.1 ... 
running task <install my packages> plg <package-generic> 
parameters:
{list => git-core carton cpanminus}
ssh -q -tt -p 2200 vagrant@127.0.0.1 ' sudo bash -c "export LC_ALL=en_US.UTF-8 ; sparrow index update"'
get index updates from SparrowHub ... OK
ssh -q -tt -p 2200 vagrant@127.0.0.1 ' sudo bash -c "export LC_ALL=en_US.UTF-8 ; sparrow project remove sparrowdo"'
project sparrowdo successfully removed

ssh -q -tt -p 2200 vagrant@127.0.0.1 ' sudo bash -c "export LC_ALL=en_US.UTF-8 ; sparrow plg install package-generic"'
public@package-generic is uptodate (0.1.1)
ssh -q -tt -p 2200 vagrant@127.0.0.1 ' sudo bash -c "export LC_ALL=en_US.UTF-8 ; sparrow project create sparrowdo"'
project sparrowdo successfully created

ssh -q -tt -p 2200 vagrant@127.0.0.1 ' sudo bash -c "export LC_ALL=en_US.UTF-8 ; sparrow task add sparrowdo install_my_packages package-generic"'
task - set plugin to public@package-generic

task sparrowdo/install_my_packages successfully created

install_my_packages.json                                                                                                                              100%   41     0.0KB/s   00:00    
ssh -q -tt -p 2200 vagrant@127.0.0.1 ' sudo bash -c "export LC_ALL=en_US.UTF-8 ; sparrow task run sparrowdo install_my_packages --json /tmp/install_my_packages.json"'
# cd /home/vagrant/sparrow/plugins/public/package-generic 
# export PATH=$PATH:local/bin 
# export PERL5LIB=local/lib/perl5:$PERL5LIB 
# strun --root ./ --json /tmp/install_my_packages.json --json /tmp/install_my_packages.json

/tmp/.outthentic/31112/home/vagrant/sparrow/plugins/public/package-generic/story.t .. 
# [/home/vagrant/sparrow/plugins/public/package-generic/modules/apt-get]
# Package: git-core
# Version: 1:2.1.4-2.1
# Status: install ok installed
ok 1 - output match 'Status: install ok installed'
# [/home/vagrant/sparrow/plugins/public/package-generic/modules/apt-get]
# Package: carton
# Version: 1.0.12-1
# Status: install ok installed
ok 2 - output match 'Status: install ok installed'
# [/home/vagrant/sparrow/plugins/public/package-generic/modules/apt-get]
# Package: cpanminus
# Version: 1.7014-1
# Status: install ok installed
ok 3 - output match 'Status: install ok installed'
# [/home/vagrant/sparrow/plugins/public/package-generic]
# done
ok 4 - output match 'done'
1..4
ok
All tests successful.
Files=1, Tests=4,  1 wallclock secs ( 0.01 usr  0.01 sys +  1.28 cusr  0.07 csys =  1.37 CPU)
Result: PASS
running task <install-psgi-app> plg <perl-app> 
parameters:
{app_source_url => https://github.com/melezhik/web-app.git, git_branch => master, http_port => 3030}
ssh -q -tt -p 2200 vagrant@127.0.0.1 ' sudo bash -c "export LC_ALL=en_US.UTF-8 ; sparrow plg install perl-app"'
public@perl-app is uptodate (0.1.5)
ssh -q -tt -p 2200 vagrant@127.0.0.1 ' sudo bash -c "export LC_ALL=en_US.UTF-8 ; sparrow project create sparrowdo"'
project sparrowdo already exists - nothing to do here ... 

ssh -q -tt -p 2200 vagrant@127.0.0.1 ' sudo bash -c "export LC_ALL=en_US.UTF-8 ; sparrow task add sparrowdo install-psgi-app perl-app"'
task - set plugin to public@perl-app

task sparrowdo/install-psgi-app successfully created

install-psgi-app.json                                                                                                                                 100%  110     0.1KB/s   00:00    
ssh -q -tt -p 2200 vagrant@127.0.0.1 ' sudo bash -c "export LC_ALL=en_US.UTF-8 ; sparrow task run sparrowdo install-psgi-app --json /tmp/install-psgi-app.json"'
# cd /home/vagrant/sparrow/plugins/public/perl-app 
# export PATH=$PATH:local/bin 
# export PERL5LIB=local/lib/perl5:$PERL5LIB 
# strun --root ./ --json /tmp/install-psgi-app.json --json /tmp/install-psgi-app.json

/tmp/.outthentic/31292/home/vagrant/sparrow/plugins/public/perl-app/story.t .. 
# [/home/vagrant/sparrow/plugins/public/perl-app]
# useradd: user 'perl-app' already exists
# Ubic is up to date. (1.59)
# Ubic::Service::Plack is up to date. (1.18)
# Starman is up to date. (0.4014)
# Stopping perl-app... stopped
# Already on 'master'
# Your branch is up-to-date with 'origin/master'.
# Already up-to-date.
# Installing modules using /opt/perl-app/cpanfile (deployment mode)
# Complete! Modules were installed into /opt/perl-app/local
# Starting perl-app... started
# perl-app  running
# install-ok
ok 1 - output match 'Complete! Modules were installed into'
ok 2 - output match 'Starting perl-app... started'
ok 3 - output match 'install-ok'
1..3
ok
All tests successful.
Files=1, Tests=3,  3 wallclock secs ( 0.01 usr  0.02 sys +  0.74 cusr  0.15 csys =  0.92 CPU)
Result: PASS

Asciinema video

A visual example of the sparrowdo scenario could be found here -

asciicast

Volunteers/Contributors needed!!!

Interested in Perl 6 automation for devops? Sparrowdo could be a good greenfield project to start with. Not involved in Perl 6 and use Perl 5 ? - that is fine, sparrowdo USES sparrow plugins which are to be written on Perl5|Bash|Ruby! So why not write a useful plugin to share with? Here is a short list of plugins now - use it as examples and create your own one! And finally sparrow itself is written on Perl5! Just contribute somehow into sparrow/sparrowhub/sparrowdo echo system! Merge requests are welcome!

-- Regards

Alexey

How to write a simple nginx installer with sparrow in 4 minutes - nice asciinema video!

A simple nginx installer with #sparrow - nice asciinema video!

https://asciinema.org/a/06h3ezswbyidu0pdl993rl30v

#nginx #perl #devops #automation

And ... yeah I forgot to say cpanm Sparrow at the very beginning .... ))

https://sparrowhub.org updated

HI ! https://sparrowhub.org/ site updated! A various changes of index page:

  • welcome text distilled to make it sparrow usage more clear and understandable
  • a look-and-feel screenshots added
  • a links to tutorials, documentation pages and papers added
  • a new motto proposed ))) - "Sparrow - Reusable automation scripts" !

Alexey

Building docker images with sparrow

This is a paper written in Russian on building docker images with sparrow - https://habrahabr.ru/post/302278/

Playing with Docker and Sparrow

Docker is quite popular solution to rapidly spin up developers environments. I have been playing with it and it seems fun for me. The other fun thing I found that Sparrow could be a good solution to build up new docker images.

Here is short example of how it could be. A few lines in Dockerfile and you have a GitPrep server up and running as docker container. Whew!

Here is my Dockerfile:

FROM perl:5.20
RUN apt-get update
RUN apt-get install sudo
RUN apt-get install git-core
RUN cpanm Sparrow
RUN sparrow index update
RUN sparrow index summary
RUN sparrow plg install gitprep
RUN sparrow plg run gitprep
CMD sparrow plg run gitprep --param action=start --param start_mode=foreground

I base on official perl docker image and then let all the job to be done by sparrow gitprep plugin!

This is how one can use it:

$ git clone https://github.com/melezhik/docker-projects.git
$ cd docker-projects/gitprep

Build an image with target melezhik/gitprep or whatever you want. You probably will need to run this command more than once if meet "Installing the dependencies failed: Installed version (1.636) of DBI is not in range '== 1.634'" error.

$ sudo docker build -t melezhik/gitprep .

Run gitprep server

$ sudo docker run -p 10020:10020 -d -i melezhik/gitprep

Test it!

curl 127.0.0.1:10020