Final Testing Done!

Mozilla issued an update to Thunderbird - version 45.1.1 today. I've downloaded and installed it with the new version of mozupdate (after correcting a small syntax error where I used the arithmetic expression '<=' in a conditional expression ([[...]]) rather than '-le').

Here's the output from the update...

root@wideboy:~# newmozupdate ~steve/Dow*/thunderbird-45.1.1.tar.bz2
Install thunderbird-45.1.1 (y/n): y

installing thunderbird-45.1.1
creating directory for thunderbird-45.1.1
extracting files from tar archive
Updating thunderbird symlink
Archiving installation file /home/steve/Downloads/thunderbird-45.1.1.tar.bz2 to /other/archive

Remove unused old versions of thunderbird (y/n): y
removing thunderbird-45.0

Remove older versions of thunderbird from archive directory (y/n): y
removed /other/archive/thunderbird-38.6.0.tar.bz2
removed /other/archive/thunderbird-38.5.1.tar.bz2
removed /other/archive/thunderbird-38.5.0.tar.bz2
removed /other/archive/thunderbird-38.4.0.tar.bz2
removed /other/archive/thunderbird-38.3.0.tar.bz2
removed /other/archive/thunderbird-38.2.0.tar.bz2
removed /other/archive/thunderbird-38.1.0.tar.bz2
removed /other/archive/thunderbird-38.0.1.tar.bz2
removed /other/archive/thunderbird-31.7.0.tar.bz2
removed /other/archive/thunderbird-31.6.0.tar.bz2
removed /other/archive/thunderbird-31.5.0.tar.bz2
removed /other/archive/thunderbird-31.4.0.tar.bz2
removed /other/archive/thunderbird-31.3.0.tar.bz2
removed /other/archive/thunderbird-31.2.0.tar.bz2
removed /other/archive/thunderbird-31.1.2.tar.bz2
removed /other/archive/thunderbird-31.1.1.tar.bz2
removed /other/archive/thunderbird-31.1.0.tar.bz2
removed /other/archive/thunderbird-31.0.tar.bz2
removed /other/archive/thunderbird-24.6.0.tar.bz2
removed /other/archive/thunderbird-24.5.0.tar.bz2
removed /other/archive/thunderbird-24.4.0.tar.bz2
thunderbird updated to thunderbird-45.1.1
root@wideboy:~#

That's freed up a considerable amount of disk space, more than 800 Mbytes by a rough calculation.  I will now start using the revised script for future updates.

As a small aside, I don't remember mentioning that one of my design criteria was that this update script should not compromise any running copies of firefox or thunderbird.  The new installation won't be used until the application is exited and then started again, and I still have the previous version ready to go should there be an issue with the update.


Steve Austin, 1 June 2016
 

Tidying archived install kit files.

I've finally (two years later!) got round to creating the final pieces of the mozupdate script and am now awaiting new releases from Mozilla so that I can complete UAT on the changes.

Changes made are...

1. Add an exit trap to remove a work file the script now creates in the /tmp directory.
2. Add a simple check to ensure that the user provided the required parameter when invoking the script.
3. Add a new function to action the archive clean up.
4. Add a small piece of code to ask the user if they want to remove older archived files.

Exit trap

On exit, the shell executing the script will be sent an EXIT signal.  This is your last chance to perform any required actions on termination of your script (although only non-blocking actions should be performed as there is not an infinite amount of time available).
Setting a signal trap is straightforward - the command syntax is basically
trap command SIGNAL_NAME.  Always quote command and use ';' to separate multiple commands.
Here's the exit trap I've added to the script...

# set an exit trap to clean up any work files
trap 'rm /tmp/*.$$ 2>/dev/null' EXIT

Note the '.$$' appended to the file matching pattern:  $$ is substituted with the shell's PID number by the shell which provides a convenient way of identifying files that belong to this invocation of the script.

Check parameter present

Just a trivial check to ensure that a parameter (which needs to be the absolute pathname of the file containing the Mozilla install kit) has been given

# check some sort of parameter given
if [ "$1" = "" ]
then
        ABORT absolute pathname of mozilla install kit is mandatory
fi

Enough said.

Archive Clean-up Function

This is the nuts and bolts of the archive clean-up. The first rough cut of this had control structures (if, for, while, until, case) nested 4 deep and needed some simplification.

I decided to use a count of the number of install kits to retain in the archive rather than attempt to discard by age, which is tricky when releases happen at arbitrary intervals.  This count is set in a global variable at the start of script execution. A sensible minimum is 2, so that you can reinstall the current and previous versions if the installation directories get damaged.

arcmax=6                #max number of install kits to keep

The main function performs it's actions in two stages: firstly generate a list of candidate files to remove in a temporary file (/tmp is a good place to put these) sorted newest first and then attempt to remove all these files except the first arcmax listed.

# define function to delete all but the most recent arcmax installation kits
# from the archive directory.
function CLEAN_ARCHIVE {
typeset -i i                #create local variables i and arcfile
typeset arcfile         #used in this function only.

# get list of candidate files for removal
if [ -d "$kitarc" ]
then
        #one filename per line sorted newest first
        #file created here will be removed by exit trap defined earlier.
        ls -1 -t $kitarc/$mozprod* >/tmp/arclist.$$
else
        echo Can\'t access archive directory $kitarc
        return 1
fi

# Remove all but the first arcmax files
i=0
while read arcfile
do
        i=$((i+1))
        [[ i <= arcmax ]] && continue   #skip first arcmax entries in list
        rm "$arcfile" && echo removed "$arcfile"
done </tmp/arclist.$$
return 0
}

Note there are now no nested control structures and the function is comparatively simple.  The only slight oddness is the use of a free-standing test followed by the conditional execution operator &&.  This works because the test operator has to return it's result in $? (the return code of any executable) or it wouldn't work with an if statement.  In any loop body delimited by 'do' and 'done' continue means 'scrap any further instructions in this iteration and go round again' and break means 'scrap any further instructions in this iteration and continue execution from the next instruction after 'done' thus scrapping any further iterations of the loop.

User Interaction

Asking users questions keeps them awake and stops them getting bored. Here's the little scrap of code that asks the user if they would like to tidy the archived collection of install kits.

# Remove older archived install kits if required.
if ASKYN Remove older versions of $mozprod from archive directory
then
        CLEAN_ARCHIVE
else
        echo Not removing older archives of $mozprod
fi

Nice and simple.  This happens right at the end of the mainline code, so the only thing that happens afterwards is a goodbye message and a return to the command prompt.
ASKYN is listed in the text of the script in an earlier post.

That's it for this post.  All I need now is a firefox or thunderbird update so I can complete final testing of the changes.

At last! a script to perform the update installations.

This script is intended to be run interactively - it asks the operator to answer a few questions and is quite gobby to prevent the operator from getting bored.
Also, I once had the joy of administering a system which was creakily slow and old, and provided many hundreds of NFS shares to the users, and used Veritas Storage Foundation to manage the disk storage.  The command "vxdctl enable" (used during startup) never says anything and took about 15 minutes to execute - ample time to work yourself into a panic in case it hasn't worked.  Gobby progress messages have some advantages!

Read the earlier posts to see the evolution of this update process.  If you're wondering why i don't allow the Mozilla automated updates to happen, it's because I can't:  I never run a privileged browser - doing this is like handing a loaded gun to a mugger - or X-server which would be like handing a loaded gun to any remote X-clients allowed to use it.  You can be caned enough by malevolent web sites or remote X-clients without giving them carte blanche to hijack or destroy your entire system, as many Windows users find out to their cost.  (root) Privilege is needed to perform the updates, and I try quite hard not to hand it to anybody.

Residency and invocation

The script is called mozupdate and resides in a private directory inside root's home directory /root/bin (/root as root's home directory is a linux convention that prevents the root user's scrappy files clogging up system directories).  I've added /root/bin to the end of root's PATH variable at login time.  Only the root user can execute this script.

Invocation is simple and and independent of the current working directory - it will cd into /usr/local/lib for you.  Only one parameter is required: the absolute pathname of the file containing the update you wish to install (which must have the systematised name given to it by Mozilla, so no renaming the downloaded file).

Here's what happens at runtime:

root@wideboy:~# ls -lrt ~steve/Dow* | tail -1
-rw-r--r-- 1 steve steve  36639715 Apr  3 10:33 thunderbird-31.6.0.tar.bz2

root@wideboy:~# mozupdate ~steve/Dow*/thunderbird-31.6.0.tar.bz2
Install thunderbird-31.6.0 ? : y

installing thunderbird-31.6.0
creating directory for thunderbird-31.6.0
extracting files from tar archive
Updating thunderbird symlink
Archiving installation file /home/steve/Downloads/thunderbird-31.6.0.tar.bz2 to /other/archive

Remove older versions of thunderbird ? : y
removing thunderbird-31.4.0
thunderbird updated to thunderbird-31.6.0

root@wideboy:~#

User typed entries are in bold in the above.

The Script

The script itself is listed here.  I confess openly to using some legacy syntax and commands - you never know when the urge to run ksh or sh for some perverse reason will strike.

Additional commentary I've added here is italicised.

#!/bin/bash
#=================================================
# mozupdate - a script to install new versions of firefox or thunderbird in a
# linux environment.  This script expects the installation kit file to have the
# standard mozilla name (i.e. firefox-version.tar.bz2 or
# thunderbird-version.tar.bz2) and that the directory that will contain the
# installation exists and contains a symlink called firefox or thunderbird
# pointing to the executable file in the current installation -
#       e.g. firefox -> ./firefox-37.0.1/firefox
# I expect that /usr/bin/firefox and/or /usr/bin/thunderbird will be symlink(s)
# pointing to the appropriate symlink in the installation directory.
#
# If the defined archive directory (see below) exists the install kit file will
# be moved to it after installation.
#
# Invoke the script as user root as
#
# mozupdate file
# where file is the firefox or thunderbird install kit file.
#
# sja 7oct14 + later amendments.
#================================================

# Static definitions of the installation and archive directories.
insdir=/usr/local/lib
kitarc=/other/archive

# define ABORT function for emergency exit
function ABORT {
echo
echo 'Aborting -' "$*"
exit 1
}

# Define ASKYN function for asking the user a question.
# note the function assumes "no" is the default answer.
function ASKYN {
typeset reply
echo
echo -n $* "(y/n): "
read reply
case "$reply" in
[Yy]*)  return 0
           ;;
*)        return 1
           ;;
esac
}

# define RM_OLD_VERS function for removing old installations of firefox or
# thunderbird.
# This function exists solely to remove the action code it contains from a
# control structure later in the script.
function RM_OLD_VERS {
typeset oldmoz          #should force oldmoz to be local to this function.
for oldmoz in ${mozprod}-*
do
        [[ "$oldmoz" = "$mozver" ]] && continue #retain current (new) version
        [[ "$oldmoz" = "$oldver" ]] && continue #retain last version
        echo removing $oldmoz
        rm -rf $oldmoz
done
return 0
}

# check we are running as user "root" (UID 0) before we go any further
if [ $(id -u) -ne 0 ]
then
        ABORT You must be user root to successfully run this script
fi

# check "file" exists
if ! [ -f "$1" ]
then
        ABORT "$1" not found
fi

# Get absolute pathname of install kit file
inskit=$(realpath "$1")

# extract product and version names from filename
mozver=$(basename -s .tar.bz2 "$inskit")
mozprod=$(echo "$mozver" | cut -d- -f1)

# change into installation directory
cd "$insdir"

# extract current (old) version from symlink
oldver=$(ls -l $mozprod | cut -d/ -f2)

The variables inskit, mozver, mozprod and oldver are used widely in the script.
NO changing them!

# abort if old version same as new version
if [[ "$oldver" = "$mozver" ]]
then
        ABORT $mozver has already been installed
fi

# Confirm installation
ASKYN Install $mozver || ABORT not installing $mozver

# proceed with update
echo
echo installing $mozver

# create new product directory in installation directory
echo creating directory for $mozver
mkdir $mozver || ABORT can\'t create directory $insdir/$mozver

# extract archive into the new directory
echo extracting files from tar archive
cd $mozver
tar --extract --bzip2 --preserve-permissions \
--strip-components=1 --file="$inskit"
if [ $? -ne 0 ]
then
        ABORT extract failed
fi
cd ..


I added the line wrap ("\") in the tar command; It suits the blogger page width better. Note the use of the GNU tar option --strip-components to get rid of the unwanted top level directory in the tar archive and the conditional execution operators && and || in the above.
 
# Update Symlink
echo Updating $mozprod symlink
rm $mozprod
ln -s ./$mozver/$mozprod $mozprod || ABORT symlink update failed
sync

# Archive install kit to archive directory
if [ -d "$kitarc" ]
then
        echo Archiving installation file $inskit to $kitarc
        mv "$inskit" "$kitarc"
        sync
fi

# Remove older versions (not new or previous version) if required
if ASKYN Remove unused old versions of $mozprod
then
        RM_OLD_VERS
else
        echo Not removing older versions of $mozprod
fi

# Finished!
echo $mozprod updated to $mozver
exit 0

That's it!  I've managed to avoid nesting ifs in the script - makes life easier for the maintenance crew.  Note the use of the syntax "if <command>": this can bite you if <command> doesn't return 0 for true (OK) or non-zero for false (gone horribly wrong).  As an aside, I never use the "elif" operator - I regard it as ugly as it unbalances the if command and can lead to seriously horrible things - the worst I've seen was an if...elif tree that went on for 54 printed pages and in my opinion needed to be edited using "rm".  And never forget the case statement!

This script does almost everything and should give you some idea of what's gone wrong in the event of a failure.  It doesn't yet do anything to prune the growing heap of install kits in the /other/archive directory.  I can't decide whether to do this by a maximum count or by age; both are likely to be ugly to implement.

If you use the conditional execution operators (&& and ||) take care if you use them in combination - which commands will be executed  in a command of the form "a && b || c" is not as simple as you might think (c will be executed if a or b return a false (non-zero) exit code, and the operators can also be derailed by commands which return arbitrary or unset return values.
 

Steve Austin
16 May 2016.

Still languishing, this blog.  Time to get on and finish it.

Let's look at the manual process I used to use to update Firefox and Thunderbird.  It's a bit clunky, and doesn't do a lot of the tedious tidying up that needs to be done to stop old installations and install kits downloaded from Mozilla clogging up lots of disk space.  It does allow you to leave the applications running during the process and switch to the new version on the next restart of the application, which is a good thing.

Here we go....

First (having downloaded the update from Mozilla [not a third party redistributor!]),  create a directory to contain the new version
Note: by chance "Dow*" matches just one entry in my home directory, "Downloads", which means I can save typing 5 whole characters.

# cd /usr/local/lib
# ls -lrt ~steve/Dow* | tail -1
-rw-r--r-- 1 steve steve  41957229 Sep 21 08:54 firefox-32.0.2.tar.bz2
# mkdir firefox-32.0.2

Now move into the new directory and extract the file from the install kit.
Sorry about the "sync" commands - I'm old enough to remember when they were a good idea after doing anything disc-related that you didn't want to lose.

# cd firefox-32.0.2
# bunzip2 -c ~steve/Dow*/firefox-32.0.2.tar.bz2 | tar xpf -
# sync

The tar archive contains a single directory (firefox or thunderbird depending which you're installing) which contains all the files which comprise the application.  I was brought up to avoid creating skinny dangling directory trees like this, so the next few commands shuffle everything round to get rid of it.  GNU tar can also do this for you, as per the script I'll describe later.
The fiddly renames to "sja" prevent a collision during the execution of "mv" which would cause it to fail. [Mozilla have not yet seen any advantage in storing any of the application components in a file graced with my initials as it's name]

# mv * sja
# cd *
# mv * ..
# cd ..
# rmdir sja





Now change up a directory into /usr/local/lib and remake the symlink there which points to the main executable (firefox or thunderbird).  The absolute path of the directory they were loaded from is one of the items passed to them in their environment at runtime, so they can find the rest of the installation components.

# cd ..
# ls -ld fire*
lrwxrwxrwx 1 root root   24 Sep 14 10:15 firefox -> ./firefox-32.0.1/firefox*
drwxr-xr-x 7 root root 4096 Sep 14 10:00 firefox-32.0.1/
drwxr-xr-x 7 root root 4096 Sep 21 08:59 firefox-32.0.2/
# rm firefox;ln -s ./firefox-32.0.2/firefox firefox;sync

All done, apart from tidying up, and the newly installed version will be used from the next restart of firefox of thunderbird.

[It isn't really.  Haven't cleaned up at all, and this ought to be done.  this needs a script, as there is quite a lot to do and I don't want to wear out my typing fingers.]

Scheme Outline

Oh dear!  Like all documentation projects, this one has languished.  There's always something better to do.

My only excuse is that I've been very busy having a hard time adapting my (defective) diabetes management strategy to cope with new medication and to be less defective in order to avoid hyperglycaemia and the peripheral circulatory and neuropathy problems that come with it.  I've also discovered that if you can keep your blood sugar in a narrow band around the target value of 6 mmol/L you come alive and function properly - for the first time in a few years for me.

What I really wanted to do with the Mozilla apps was move away from the ESR versions (which you would normally update via the update distribution process favoured by the producers of the distribution you are using) to using the versions distributed directly by Mozilla.  These are updated more often than the ESR versions, which means you keep up with developments and have to live for less time with any bugs that have crept in and bite you.

I evolved a simple scheme to do this based upon use of the systematised filenames used by Mozilla for the bzipped tar files containing each release of the firefox and thunderbird apps - these have (to date) always been of the form app-version.update[.patch].tar.bz2 where "app" is "firefox" or "thunderbird", version is the release number, update is the lower level release number and patch is an [optional] even lower level release number but not always present.
As with additional software I add to my systems I placed the folders in the /usr/local tree to avoid cluttering up the base system files in the main /usr tree, in this case in /usr/local/lib, and named them app-version.update[.patch] with root ownership and no write access for other users to protect the product files from accidental scribbling over.
To access the binaries, I replaced the symlinks in /usr/bin with these...

/usr/bin/firefox -> /usr/local/lib/firefox
/usr/bin/thunderbird -> /usr/local/lib/thunderbird

... which point to symlinks in /usr/local/lib which redirect forwards to the actual binaries in the release I'm currently using...

firefox -> ./firefox-43.0.3/firefox
thunderbird -> ./thunderbird-38.5.0/thunderbird

... so at update installation time everything that needs to be changed is in one place, /usr/local/lib, and in the event of a problem, rollback simply requires the /usr/local/lib symlinks to be changed to point to the executables in the previous version directories.

What existed before I started applying direct updates

The system I'm currently running on was built from scratch using Slackware a little over a year ago.

The ESR versions of both Firefox and Thunderbird are installed as part of the Slackware installation, in /usr/lib/firefox-24.1.0 and /usr/lib/thunderbird-24.1.0 respectively.  The base executables (/usr/lib/firefox-24.1.0/firefox and /usr/lib/thunderbird-24.1.0/thunderbird) are accessed via symlinks in /usr/bin...

/usr/bin/firefox -> /usr/lib/firefox-24.1.0/firefox
/usr/bin/thunderbird -> /usr/lib/thunderbird-24.1.0/thunderbird

The remainder of the firefox and thunderbird packages (additional executables, private libraries and other files required at runtime) reside in the /usr/lib/firefox and /usr/lib/thunderbird directories and various directories inside them, exactly the way you would get them if you unpacked an install/update kit tar archive from Mozilla.

This arrangement of private files in a containing directory around a main executable works well, especially as the main executable can obtain it's location from it's environment, and needs to be left the way Mozilla have arranged it.  Of course, Mozilla may also change the layout and actual files from time to time.

Updating Mozilla Apps in a Linux Environment

This blog is about installing updated versions of Firefox and Thunderbird obtained direct from Mozilla in a personalised Linux environment. The methods described probably aren't suitable for organisational environments containing many systems and where updates are taken from the maintainers of your Linux distribution.


I've performed these updates manually for quite some time, and only got around to scripting the process recently.  In some ways this is a good thing because it allows the process you use to be designed, tested and corrected as necessary before you start implementing an automation script.

To save boring you, I'll divide this blog into multiple posts covering these areas...

• What existed on the system before I started applying updates to it
• What I wanted the update process to achieve
• What you get in an update pack downloaded from Mozilla
• Rolling back an update if necessary
• Automating the process - what I wanted to achieve
• Automating the process - additional actions
• Automating the process - using the script
• Current version of the script
• Future development

Steve
10 June 2015