Client Application Management

Prerequisites: Client Configuration Management

Everything up to this point has been substrate for applications to run on -- and we need to remember that applications are the only reason the infrastructure exists in the first place. This is where we make or break our infrastructure's perception in the eyes of our users.

We wanted location transparency for every application running on any host in our enterprise cluster. We wanted the apparent location and directory structure to be identical whether the application was installed on the local disk or on a remote file server. To accomplish this, we used SUP to maintain identical installations of selected applications on local disks, automounted application directories for NFS-served apps, and Perl-managed symbolic link farms to glue it all together. [mott]

A heterogeneous and readily available caching filesystem would have been much simpler to understand, and as mentioned before we originally considered AFS.

We made all applications available for execution on all hosts, regardless of where the application binaries physically resided. At first, it may seem strange that a secretary might have the ability to run a CAD program, but an ASIC engineer will certainly appreciate the fact that, when their own workstation fails, the secretary's machine can do the job (see Disaster Recovery ) .

We executed our apps from /apps/application_name. We had the automounter deliver these binaries, not to /apps, but to /remote/apps/application_name. We then created a symbolic link farm in /apps. The link farm simply pointed to the /remote/apps directories of the same name.

To support the extra speed we needed for some applications, we used SUP to replicate the application from the NFS server into the /local/apps/application_name directory on the client hard disk. The Perl code which drove SUP referred to a flat file (/etc/autosup.map) which listed applications to be replicated on particular machines. We inspiringly dubbed this code 'autosup' [autosup] . The autosup.map file looked something like:

scotty: elm wingz escapade metrics
luna: elm wingz 
[...]

After 'autosup' updated the local copies of applications, possibly adding or deleting entire apps, another Perl script, 'autolink', updated the symbolic link farm to select the "best" destination for each /apps symlink. The selection of the best destination was made by simply ordering the autolink targets (in /etc/autolink.map) so that more preferential locations overrode less preferential locations. The autolink.map file usually looked something like this:

# create         from 
#
/apps          /remote/apps
/apps          /local/apps
/apps/pub      /remote/apps/pub
#
/prd/sw         /net/${HOMESERVER}/export/apps${HOE}/prd/sw
/prd/sw         /local/apps1/prd/sw
[...]

The trivial example below shows how the symbolic links in /apps would look with a CAD package installed locally, and TeX installed on a file server.

The 'autosup' script was usually triggered by a nightly crontab which SUPed down the new autosup.map, and 'autolink' was usually triggered by 'autosup'.

It is important to note that part of application management is developer management. At first, many of our application developers loved to have their programs write files in the directory tree that contained their program, and they tended to hardcode pathnames to other binaries. We consider this a bad thing. For our in-house developers we managed to convince them to refer to environment variables for where data and binaries lived. For external applications we had to do tricks with symlinks.