tinita

This week we had the SUSE Hackweek 18. It was my first Hackweek since I started working for SUSE. It happens about once or twice a year. People can contribute to any open source project they want to in that week.

So, thanks to SUSE for making this happen!

This blog post is about what I hacked on Tuesday, when Hackweek started for me. Expect more posts for the other days ;-)

• Day 2 - YAML::PP !include
• Day 3 - (Not) Loading Objects in YAML::PP
• Day 4 - Fighting with XS and C

Writing shell scripts

First I'll write about shell scripts. Later we'll see how Perl comes into play.

Many of you probably create or maintain shell scripts sometimes. For certain tasks it can be a better choice than Perl, for example.

But there are problems, and not unique to shell scripts.

Maybe you think that these problems are not that important for small, simple scripts. But scripts grow, and I find it annoying, if I have to use a tool which has outdated (or not helpful) usage output and no tab completion.

Here are the problems I see, and then I will show you a tool that could be able to fix all that.

Command line parsing

Let's say you have a bash script with a single command line switch -v. Extracting such a single switch from the commandline is comparable easy.

Now you want to add --verbose also, and maybe add another option --task, which gets an argument.

And maybe you want to support levels of verbosity, so the -v switch works incremental (e.g. -vvv).

There are tools out there that let you automatically parse that, but it's limited.

Before a script gets too big, it can be a good idea to add subcommands. (Imagine git without subcommands. Or think about gpg...)

Some options are global, some only for a certain subcommand.

Help output matches actual behaviour

You probably all know the problem that, when options are added to a script, or changed, the usage output / man page doesn't get updated. Because it's forgotten.

Writing usage output for several subcommands isn't fun.

Tab completion

Nobody (well, almost) likes to manually write tab completion for their scripts.

Sometimes scripts come with completion, but it's very basic, and maybe only for bash.

Zsh, for example, offers much more features, and if people can only use the bash completion in zsh, they are missing several features.

Checking values

Some of your commnd line options may take a static list of values, for example ssh-keygen -t {rsa,dsa,ecdsa,ed25519}.

You need to check if a value is in the list. (And, additionally, these values should appear when hitting <TAB>.)

Introducing App::Spec::Bash

Maybe you already heard about my perl5 App::Spec command line framework.

It does all the argument parsing for you, can generate completion, pod/man and help output. You only have to write a YAML file and the actual code for your script's commands.

How to port this to Bash?

The cool thing is, for pod/man and completion, I can simply reuse the existing code. Half of the work is already done!

Now for writing the bash command line parser using the YAML spec file, I had the idea to not do all the work in bash, but generate bash code, which saves me from working around the problem that bash doesn't really have rich data structures.

It can already do most of what I mentioned above, except argument value checking.

It supports flags and options with values, incremental flags (-vvv), options with multiple values, stacking of short options (-a -b -t val == -abtval) and subcommands.

The documentation is minimal right now. The following example is taken from the command's pod.

For your bash script, the following files are needed:

share/mytool.yaml                   # spec
bin/mytool                          # simple script calling the framework
lib/mytool                          # your code for each subcommand
lib/appspec                         # generated
share/completion/zsh/_mytool        # generated
share/completion/bash/mytool.bash   # generated
pod/mytool.pod                      # generated
lib/help                            # generated

To generate the files, you need the appspec tool and appspec-bash. The latter one is not yet on CPAN, only on github.

This is the YAML specification file:

# share/mytool.yaml
---
name: mytool           # commandname
appspec: { version: 0.001 }
title: My cool tool    # Will be shown in help
class: MyTool          # "Class name" (means function prefix here)

subcommands:
  command1:
    op: command1       # The function name
    summary: cmd one   # Will be shown in help and completion
    options:
    - foo|f=s --Foo    # --foo or -f; '=s' means string
    - bar|b   --Bar    # --bar or -b; a flag

Maybe you recognize that the syntax for options looks like the one from Getopt::Long. I thought it would be good to reuse existing syntax, but for more features I added other syntax elements. There's not much documentation so far for this except App::Spec::Argument. For examples, you can also have a look at my collection of shell completions.

The script:

# bin/mytool
#!/bin/bash
DIR="$( dirname $BASH_SOURCE )"
source "$DIR/../lib/appspec"
source "$DIR/../lib/mytool"
APPSPEC.run $@

The actual app:

# lib/mytool
#!/bin/bash
MyTool.command1() {
    echo "=== OPTION foo: $OPT_FOO"
    echo "=== OPTION bar: $OPT_BAR"
}

Output example:

$ ./bin/mytool command1 --foo x --bar
# or
$ mytool command1 -f x -b
=== OPTION foo: x
=== OPTION bar: true

Please see the documentation to learn how to generate the other files.

So far I have only a few tests, so there might be bugs.