Replacing the File Manager in Zsh

Replacing the File Manager in Zsh

Peter Stephenson

Summary

One of the things we emphasise in From Bash to Z Shell is how, despite the apparent ease of use of graphical file managers such as Windows Explorer or the Unix/Linux system KDE’s Konqueror, the shell can often be a more powerful alternative that helps you do your tasks more quickly and efficiently. In this essay we’ll see how you can make zsh do something usually associated exclusively with file managers, handling a file based on its suffix. Bash doesn’t currently have the key piece of support for this; people as old as the author may remember the DOS Command Prompt replacement 4DOS had a similar facility. We’ll also talk a bit about the MIME system that allows you to add this ability without doing too much work; unlike in 4DOS, you shouldn’t have to set up every suffix yourself.

The Shell and Graphical File Managers

If you use both the shell and a graphical file manager, you will notice there’s one big difference in the way files are managed. In the shell, to run a command on a file you type the name of the command, then the name of the file. When you want to view or execute a file in the file manager, you will often simply click on the file name, and the appropriate command to handle it will be started; you may not even know the name of the program that’s executed. (On Windows you’ll need to click twice in quick succession unless you’ve set the appropriate display option—I‘ll just say ‘click’ for short.)

For example, to view a PDF file with the suffix (extension) pdf, you click on the file and the file manager starts up Adobe Acrobat Reader, or one of a range of open source alternatives, to show you the contents. It’s often said the file is associated with the program that executes or otherwise handles it. We’ll call the program the handler; it’s sometimes called a dispatcher (or despatcher). Here the handler is Acrobat Reader. The key part of the file name for the association is after the last dot: we’ll call this the suffix, but it’s sometimes called the file’s extension. Here it’s pdf.

You may be surprised to hear that zsh can do this, too. It’s not a question of clicking, however; instead, you type or the shell completes the file name. When you type the Return key the shell realises that there is a handler for it. It’s based on a facility called suffix aliases. Simple aliases are probably familiar; they’re present in most shells and are introduced in the book on page 11. You’ll probably recognize commands like the following

  alias ll='ls -l'

This allows you to type ll to make the shell execute ls -l.

Suffix Aliases

A ‘suffix alias’ is a little different. Instead of the complete word being replaced by other text, only the word after the final dot is examined; also, the command word isn’t replaced by the full text, instead the text is inserted in front of what’s already present. Here’s a simple example:

  alias -s pl=perl
  script.pl

The first line defines pl as a suffix alias. When the shell sees the word script.pl in command position, it notices the suffix pl and finds the suffix alias with the replacement word perl. It inserts this in front to produce the following command line:

  perl script.pl

So the shell knows that it needs to run the Perl interpreter on files with the pl suffix. The only change is the insertion of the new command word at the start of the line; the rest of the line is left unchanged.

There’s a bonus if you use the new completion system: it already knows about suffix aliases, in particular that they can occur in command position. This means that in the example just above the file script.pl will be offered for completion as if it were a command. If you use the format completion style (described on page 238 of the book), so that the completion system tells you what it’s completing, you will see a message like Completing suffix alias in completion listings for such files. You might find it useful to develop the habit of typing ./ in front of a file with a suffix alias: this stops the completion system from looking for normal commands, and makes it focus on the current directory, cutting down the number of possible completions right from the start. (Of course you can use a path to another directory, too; you don’t need to change directory first.)

The shell’s MIME system

You can already see how this can help you run commands in a similar way to the file manager. This command:

  alias -s pdf=acroread

would allow the effect described at the top: you just give the shell the name of a PDF file, and the shell starts up Acrobat Reader, assuming it’s in your path. (Under Windows, the program is probably AcroRd32.exe, though you don’t need to specify the .exe.)

However, a lot of the work has already been done for you. Versions of zsh starting with 4.2 come with functions which can use the information already present on most modern systems to associate suffixes with handlers for their files. What’s more, you can adapt and configure this system for your own use.

To turn on the shell’s side of this, you simply need to add the following to your .zshrc:

  autoload -U zsh-mime-setup
  zsh-mime-setup

That’s it! The function zsh-mime-setup will go away and look at the associations defined by the system, and set up the suffix aliases accordingly. However, you might find it doesn’t set up the alias you need (to see, run the commands above, then try ‘executing’ a file with a well-known suffix). It will help to know a little bit about what’s going on so that you can add your own.

You can list all suffix aliases by typing alias -s with no arguments. After running zsh-mime-setup, you’ll probably see that a number of aliases have been added to be handled by the command zsh-mime-handler. This is a function that decides which command to run for each suffix.

Aside: why is it done that way, instead of simply aliasing the handler command directly, as we did before? As we’ll see, there are various ‘flags’ you can supply to modify the behaviour of the handler. Since most of the commands that will be executed aren’t aware of these, a little additional processing is done by zsh-mime-handler.

A Brief Introduction to MIME

Let’s look at how you can configure the way suffixes are handled when you use zsh-mime-setup. First, we’ll introduce the standard system that lies behind the shell’s own features.

The mime part of the function names refers to the Multimedia Internet Mail Extensions (MIME). This is the system used for encoding parts of an email message and associating them with certain types. Those types are in two parts, a major and a minor one, separated by a slash. For example, the most basic text file has the type text/plain, while files that use HTML markup (such as this web page) have the type text/html. Files associated with applications often have the major type application, so a PDF file has the type application/pdf.

For email you can always specify the type explicitly, and it’s sensible to do so, since you can’t be sure the receiver has the same associations; nonetheless, some mailers unfortunately rely a bit too much on suffixes. For many other uses, though, where there’s no easy way of specifying the type directly—you certainly don’t want to type application/pdf every time you want your file manager to run Acrobat Reader on a file—it’s very convenient to associate the type of a file with the file’s suffix. MIME allows you to do that, too. The configuration is actually in two parts: associate the types with suffixes, and also associate them with handlers. When you use the shell’s own system, you’re not concerned with the type name; but to change the configuration you will need to know the type.

There’s nothing to stop you inventing your own MIME type, and following the instructions below to configure suffixes to use it. You’ll then have the bonus that any other utility that uses these standard files will also be able to handle your newly defined associations; Mozilla and Firefox, for example, examine the files we’ll talk about. Some of the Unix desktops don’t, however; KDE has its own system, although it’s still based around the same MIME types. If you are going to define your own type, there’s one convention to remember: non-standard types have an x- in front of one of the type names, usually the minor part. This doesn’t prevent other people picking the same name for a type, but it does guarantee it will never clash with a standard type.

There’s a tendency for new applications to use a slightly different way of trying to avoid clashes between types. The minor part starts with vnd. and continues with a series of elements separated by dots. The first element indicates the organisation that defined the minor type; then as long as the organisation is unique, the whole type will also be unique. For example, OpenOffice.org 2.0 uses the type application/vnd.oasis.opendocument.text to represent its native Open Document Format for word processing documents.

Let’s suppose, then, that your application is called is called fimble and it runs files with the suffix fim. Then you’d probably pick the type name application/x-fimble. So how do you tell the shell and other utilities about it?

MIME configuration files

The two key files are the mailcap (mail capababilities) file, which associates types with handlers, and mime.types, which associates types with suffixes. Let’s discuss these in order.

The mailcap File

Typically, the system has a file /etc/mailcap, although it may be in other places. (If it’s not there, try using locate mailcap to find it. We talked about locate in the book on page 177.) There’s also often a file .mailcap in your home directory, which has exactly the same format and augments or overrides the definitions in /etc/mailcap. If that second file doesn’t exist, you can create it; most utilities that use the standard MIME system will find it. However, you may find it has already been created by one of the more heavyweight utilities such as Mozilla or OpenOffice.org that helpfully added definitions for types it handles. These existing definitions would be picked up automatically when you run zsh-mime-setup. Alas, Windows don’t use this system, so for Cygwin you’ll have to make or copy your own files and update them appropriately.

A simple mailcap entry consits of a type name, a semicolon, and a handler. The format of the handler is of a command line where the file to be handled is represented by %s. Here’s an entry to indicate that PDF files should be handled by Acrobat Reader:

  application/pdf;/usr/local/bin/acroread %s

This version gives the complete directory path to the handler. Many utilities add the full path to be sure the right program is edited. However, if you know the command is in your command search path (the shell variables path or PATH), you can just use the base name of the command. It’s best to keep just to simple command lines to make sure the various different programs that can interpret the handler (such as zsh-mime-handler itself) don’t get confused by the syntax.

The vast majority of mailcap entries have just these two parts. However, it’s possible to have a second semicolon (remember that’s a normal way of ending a shell command) and a set of flags. Each flag is separated by a further semicolon. Each flag may contain an equals sign and an argument. Otherwise the flag is Boolean; the feature is on if the flag is present (true), else it’s off (false). Note that entries may contain continuation lines, indicated, as in shell syntax, by putting a backslash before the newline at the end of the previous line.

Many utilities use flags for their own purposes; these are ignored by other MIME-aware utilities. Two flags are standard and important enough for zsh-mime-handler to be aware of them. The first is needsterminal, which, as its name suggests, indicates that the handler must run with a terminal. This tells zsh-mime-handler that it shouldn’t run the handler listed for the type in the background, since it expects to control the terminal directly.

The second flags is copiousoutput, which is also fairly self-explanatory. (In case English isn’t your first language, ‘copious’ simply means ‘a lot of’.) Hence this handler can produce a lot of output. If this flag is set, the shell will pipe the output of a handler to a pager such as more or less so that you can see the output one screenful at a time.

Let’s assume your utility fimble runs on a file and produces a large amount of output based on the file you give it as an argument. Then you could add the following entry to your .mailcap file:

  application/x-fimble;fimble %s;copiousoutput

The mime.types file

The other file gives a list of suffixes associated with each type. It can usually be found in the same directories as the mailcap file, typically /etc/mime.types or .mime.types in your home directory. Again, you can create the file if it doesn’t exist. Note that you don’t necessarily need to change both mailcap and mime.types: if the suffix in /etc/mime.types is correct, but you want to change the handler, you just need to edit ~/.mailcap.

The usual format of .mime.types is even simpler that that of mailcap. Each line consists of a type name, a space, and then a list of suffixes (no dot in front, consistent with the way we’ve given them above) separated by spaces. So here are the lines for both PDF and fimble’s fim files:

  application/pdf pdf
  application/x-fimble fim

If fimble also handle files ending in fum, you could simply add ‘ fum’ to the end of the line. The combination of the application/x-fimble entries in the two files is enough to trigger zsh and other utilities into handling files ending with .fim.

There’s a more complicated format that’s sometimes used, in particular by Netscape. This uses key/value pairs separated by spaces; values that contain spaces must be quoted. Continuation lines may again be used, so here’s how Netscape indicates the same information as the line for PDF above, but with a description:

  type=application/pdf  \
  desc="Portable Document Format"  \
  exts="pdf" 

If you prefer, you can use this form, since the shell’s system understands it, too. In this case multiple suffixes (extensions) are separated by commas with no space. The description is simply ignored.

Trying Out Your Modifications

If you’ve already run zsh-mime-setup and subsequently modified one of the setup files, you need to run zsh-mime-setup -f to force the shell to reexamine the files. You can then run zsh-mime-setup -l to list the information stored (note that you can’t combine the options). If you got it right, you should see a line like the following:

  fim       fimble %s

You don’t see the MIME type; the shell doesn’t need that again, so doesn’t remember it. Now you should find that when you use a fim file in command position it executes your application with the file as argument.

Troubleshooting

The fact that there are two different types of configuration file which might be found in different directories means that it’s not uncommon to find even common file types aren’t set up properly. If you find that you can’t use a suffix you would expect to be handled, here are some tips for finding the problems.

List the handlers

As we’ve already seen, running zsh-mime-setup -l will tell you what handler the shell will use. If a handler is listed but it’s wrong, edit your ~/.mailcap file or create a new one. If no handler is listed in ~/.mailcap, just add one; it’ll replace any set by standard system files.

Locate the system files

If there’s no handler at all, it means the type either doesn’t appear in any of the mailcap files, or doesn’t appear in any of the mime.types files. It’s perfectly safe to add appropriate lines to both ~/.mailcap and ~/.mime.types, although you may just have to guess the name of the MIME type.

However, it may be that the shell hasn’t found your system’s configuration files at all. If you ran zsh-mime-setup -l and saw very little output this is quite likely. In that case you will have to tell the shell where the files are. Use the locate or find commands or inspired guesswork to locate them: /usr/etc, /usr/local/etc, /usr/local/share/etc are possible directories. The base names of the files are unlikely to be different from mailcap and mime.types. In addition, some utilities provide additional versions files; you can tell the shell about these, too, although you might first want to check whether the contents of those files have been appended to a file in a more standard location.

You tell the shell about these locations using styles. We met this highly flexibly way of configuring the shell when discussing the completion system in the book, chapter 10 (see page 237). Recall that styles are recognized using two components: the name of the style itself, and also the context, which is typically a string consisting of items separated by colons. The context is useful because it may take different values in different circumstances when the style is used, and you may select those circumstances by using a pattern. However, in this case the context is simply :mime: to specify the MIME function system. Configuring the files is very little different from setting an array of values.

The two styles are mailcap and mime-types (note there’s a hyphen, which is normal in style names, not a dot, which isn’t). You can supply as many files as you like to each, but note that earlier files are used in preference to later files, so make sure that your own personal file come first. Here’s an example:

  zstyle :mime: mailcap ~/.mailcap /usr/local/etc/mailcap /etc/mailcap
  zstyle :mime: mime-types ~/.mime.types /usr/local/etc/mime.types \
    /etc/mime.types

You should put those lines in ~/.zshrc before the line that executes zsh-mime-setup.

Check the Program

If you’re still having problems, check that the handler program is being run in the correct way. For example, if the handler needs a terminal make sure the needsterminal flag is present: if you were handling HTML files with the program lynx you would need a terminal, whereas with firefox you wouldn’t. (There’s another shell function to help you start a web browser, pick-web-browser, that we might talk about another time.)

Configuration specific to the shell

There are a few things you can tweak that only apply to zsh’s handler. These are also set by styles. Unlike the paths to the standard files, however, the styles are tested separately for each suffix. To use this extra information the context is extended to include the suffix preceeded by a dot. As usual with styles, the separators are colons, so for example when the suffix is pdf the context is :mime:.pdf:.

The styles handler and flags can be set to a handler and a set of flags, respectively, in just the formats they would be in the mailcap file. Effectively this provides the shell with a private handler. For example, the following tells the shell that it should handle HTML files with one of the suffixes html or htm using the shell function pick-web-browser we described above, and that this has no flags:

  autoload -U pick-web-browser
  zstyle ':mime:.htm(|l):' handler 'pick-web-browser %s'
  zstyle ':mime:.htm(|l):' flags ''

This is a good way of configuring the system in this particular case, since other programs can’t run the shell function pick-web-browser directly. However, they could still execute that file as a script if you made it executable. It’s important, however, that a standard handler does appear in mailcap and mime.types, otherwise zsh-mime-setup won’t know that it should associate zsh-mime-handler with the suffix. You can change these styles without running zsh-mime-setup -f.

Note that unfortunately zsh-mime-setup -l doesn’t know about the styles, so it will list the standard handler; this should be fixed in version 4.3.1 (I spotted the problem just in time when writing this essay!) However, zsh-mime-handler will run pick-web-browser. As this is a shell function, we needed to mark it for autoloading.

For now, there’s just one other style, though version 4.3 of the shell will provide a little extra configuration. The style is pager and is used any time the shell needs to pass output through a pager because of the flag copiousoutput. Usually the pager to use is taken from the the environment variable PAGER, but you can use the power of the style system to pick a different pager for each suffix.

Conclusion

That’s the end of our tour of zsh’s MIME features. Together with the completion system we hope you’ll find it removes yet another reason for preferring a graphical file manager over the shell. If there are any other things you can do quickly with a file manager where you think there’s scope for improvement in the shell, drop us a line at authors(at)bash2zsh(dot)com (replace (at) with @ and (dot) with .); we’ll think about it.