Monitoring minion workers/tasks using Sparrow

Hi! This is small post concerning sparrow plugin to monitor minion workers. I had some post on this recently, but now I have made some improvements at existed plugin, so I am writing a new one ...

On my production server I use Minion to send emails to my clients. For some reasons there are faults on executing minion tasks or even minion workers get stopped sometimes for reasons unknown. As sending emails is a vital part of the service registration system I need to know if everything goes bad with my email system. Here is minon-check sparrow plugin to risqué.

Install sparrow

$ cpanm Sparrow

Install sparrow plugin

$ sparrow index update # get latest plugins index from Sparrowhub
$ sparrow plg install minion-check # install plugin

Set plugin up

$ sparrow project create webapp # create a projects to hold web applications tasks
$ sparrow task add webapp minion-health minion-check # this is 
# minion health task which is "tied" to minion-check plugin

$ EDITOR=nano  sparrow task ini webapp minion-health # set task parameters. 
# on my server this will be:

# I use carton install, so:
command = cd /path/to/your/mojo/app && carton exec ./app.pl minion

# sets worker footprint to lookup at processes list
worker_cmd = minion worker

A few comments on plugin setup here:

  • A command parameter define a system command to handle your minion jobs/tasks , An app.pl is Mojolicious application to run my web application, so I set my minion command as mojolicious command.

  • A worker_cmd parameter is just a string to look up inside a process list, to identify that minion worker is alive. A simple stings minion worker works for me.

Let's run a task and see the output:

$ sparrow task run webapp minion-health


<minion-tasks>

/modules/check-worker/ started

look up {{{minion worker}}} at proccess list ...
web-app       2748  0.3  3.4 202704 35216 pts/0    S    23:17   0:00 perl ./app.pl minion worker -m production -I 15 -j 2
done
ok      scenario succeeded
ok      output match 'done'

/modules/last-task/ started

Q=0
ok      scenario succeeded
ok      output match /Q=(1|0)/

/ started

no failed tasks found
ok      0 failed jobs found
STATUS  SUCCEED

So what sparrow plugin minion-check does?

  • check if your minion worker(s) is running.

  • check if you have no tasks marked as FAILED for the given period of time. ( I do not set it up explicitly here, but for default it uses 10 minutes interval , for details follow minion-check documentation ).

Finally I setup cron job to run a sparrow plugin every 10 minutes:

$ sparrow task run webapp minion-health --cron

A --cron flag makes sparrow client silent unless any errors happen during execution.


Regards.

Alexey Melezhik

Leave a comment

About melezhik

user-pic Dev & Devops --- Then I beheld all the work of God, that a man cannot find out the work that is done under the sun: because though a man labour to seek it out, yet he shall not find it; yea further; though a wise man think to know it, yet shall he not be able to find it. (Ecclesiastes 8:17)