perlancar

My adventure in shell tab completion continues. (A bit of background: I got interested in tab completion around two years ago when I realized that it is a significant UX element, if not one of the most important ones, in a CLI environment. Since then I've created, among other things, a couple of command-line frameworks that make it easy to do custom tab completion in Perl, as well as a bunch of modules in Complete::* namespace for generic completion which can be used in other environments like GUI and web).

This week, I added tab completion support to my frameworks for several other shells other than bash: tcsh, zsh, and fish. This blog is an observation of the different ways of doing programmable tab completion in those shells, especially from the viewpoint of a programmer who wants to do it using Perl (or other programming languages) instead of using built-in shell functions.

Aside from the abovementioned shells, I also took a look at ksh but it has, let's say very primitive support for programmable completion via keybinding, so I skip it for now. I also glanced at various other non-Unix shells like Clink, Windows Power Shell, and even CMD.EXE. But as I don't use those (as a matter of fact I virtually never used shell other than bash for the past decade and a half) so I didn't bother to delve deeply into any of these.

bash

bash turns out to be relatively very friendly and convenient when it comes to letting you complete using an external command. All you have to do is:

% complete -C mycompleter someprog

This means that if user press Tab here:

% someprog --foo [Tab]

bash will invoke mycompleter program/script with two environment variables set: COMP_LINE and COMP_POINT. COMP_LINE is the command line string ("someprog --foo ") and COMP_POINT is an integer indicating cursor position (15). This means that if you type something, move cursor to the left several times like this:

% someprog --ba[Tab] --foo --baz

COMP_LINE will still contain the full "someprog --ba --foo --baz" and COMP_POINT set to 13 and your completer program is fed with enough information to do things like providing completion of unmentioned command-line options. For example, in the above case your completer program can just provide a sole completion of --bar since --baz is already mentioned.

Your completer program usually must parse COMP_LINE by breaking it into words, taking into account things like single/double quotes and backslash escapes. This is different than when you use a shell function to provide completion, where in that case bash will provide COMP_WORDS and COMP_CWORD instead. COMP_WORDS is an array containing the command-line already broken/parsed by bash into words, and COMP_CWORD is an integer pointing to the element in the array where the cursor is in, definitely more convenient.

Your completer program can then return completion result by printing the entries to STDOUT, one on each line. Special characters can be escaped with backslash.

tcsh

tcsh also has a builtin command complete. Compared to bash's version, tcsh's complete is slightly more complex (or richer, depending on how you look at it). It allows you to use completion for any ambiguous command, and allows you to specify some conditions on which completion should be applied. For example, only apply completion for words at certain numerical positions, or when the previous word matches a certain pattern.

Since I want to support multiple shells, I don't find these features worth using. All I care about is this command:

% complete someprog 'p/*/`mycompleter`/'

which means that tcsh should invoke mycompleter program/script when a user wants to get a completion for someprog command, at any position (the p/*/.../ part).

Compared to bash, tcsh currently only provides a single environment variable COMMAND_LINE containing the entire command line string. There is no equivalent for bash's COMP_POINT, although there are unofficial patches to do so. I wish tcsh would just follow bash and switch to COMP_LINE/COMP_POINT.

Similar to bash, completer program only needs to print completion answer one line for each entry. However, tcsh also does not yet clearly define escaping mechanism for the entries so you currently can't provide completion answer entry which contains a whitespace, for example, because tcsh will consider the entry as two entries.

Also of note is that tcsh doesn't have the concept of shell functions, although that is not relevant for this blog post's topic.

zsh

zsh is supposedly the first shell that offered programmable completion back in the 1990's when other shells like bash did not yet have the feature. Currently zsh has two "flavors" or "systems" for doing completion, respectively described in its two manpages zshcompsys (the new one) and zshcompctl (the old one). Unfortunately those two manpages are very confusing to read and feel very abstract to me. I had to, and I suspect you probably will have too, search elsewhere for overview articles and tutorials.

The new system (or framework, if you will) is basically a suite of shell functions to make creating completion for command easier. Similar to using Getopt::Long::Complete, with this framework you can just use the _arguments shell function to describe what options a command accepts and how should the value of each option be completed (e.g. either with a list of files/directories, or usernames, or from an external command, and so on).

The old system, using the builtin command compctl, is more akin to what you would find in other shells. zsh allows delegating completion to a shell function using:

% compctl -K funcname someprog

It does not offer delegating to an external command like bash's complete -C. However, we can call external commands from a function. So I ended up with something like this to emulate bash's way:

_% mycompleter() { read -l; local cl="$REPLY"; read -ln; local cp="$REPLY"; reply=(`COMP_SHELL=zsh COMP_LINE="$cl" COMP_POINT="$cp" mycompleter`) }
% compctl -K _mycompleter someprog

The above command creates a function which will locally set COMP_LINE and COMP_POINT like what bash would do. I added COMP_SHELL=zsh so the completer program can know that it is dealing with zsh and do proper escaping, for instance.

fish

For a shell that boasts interactivity and user-friendly features, fish has a surprisingly simple mechanism for completion (that feels a bit restrictive or limited in some ways). It also has a builtin command complete. However, it requires us to specify completion rule for each command-line option for each command. But, one feature other shells do not yet have is to provide a description string for each command-line option as well as each completion answer entry which the user will see at the right of each completion entry. For example:

% complete -c someprog -l force -d 'Force deletion'
% complete -c someprog -s u -d 'Username' -r -f -a "(cat /etc/passwd|cut -d : -f 1)"

The above command says define two options for someprog: --opt1 and -z. The first option does not require an argument, while the second option requires (-r) an argument, the completion for which is delegated to the subshell.

Since fish does not provide any special environment variable for the external completer program, if we want to do it bash-style we must emulate it like in the case with zsh. Here's what I ended up with:

% complete -c someprog -s u -d 'Username' -r -f -a '(begin; set -lx COMP_SHELL fish; set -lx COMP_LINE (commandline); set -lx COMP_POINT (commandline -C); mycompleter; end)'

Like in other shells, your completer program needs to print completion entries to STDOUT, each entry per line. A description for each entry can be added at the end of each line, prefixed by a tab character.