The mountcheck script is an administrative script the systems administrator
can configure to run checks just before each frame renders, verifying the
machine is usable for rendering, i.e. file system mounts are in place,
licenses are available, etc. before renders are started.
Normally one would do such checks in the render script, but if the
render script exists on a file server that isn't mounted, it therefore
cannot be accessed, and will be unavailable to do such checks.
Use this script if you're having network instabilities where, for
example, render nodes are intermittently losing access to the file
server and have become unusable for rendering.
This is where the mountcheck script is useful; since it's a script
located locally on each machine (in the rush/etc directory), it is
always available to be used to test file system mounts and try to
either fix them, or it can offline the local machine if need be
and tell the job to requeue the frame elsewhere by returning exit(1).
Common goals this script might want to achieve:
- Check if the file server's mounts are accessible.
If unavailable, try to either fix the problem and return an
exit code of '0' if successful, or disable the machine
from rendering jobs either with
to prevent further rendering. You can optionally notify the user with
and return an exit code of '1' to retry the frame elsewhere.
- Check for missing infrastructure that all renders depend on.
Things like missing licensing, plug-ins, fonts, etc. Anything that you
know all your renders depend on for correct operation that
might be missing for one reason or another.
This script is enabled by uncommenting these lines in the rush.conf file:
Mountcheck Configuration in rush.conf
# MOUNT CHECK COMMAND
# Command run before each frame to verify mounts.
# Uncomment + modify example scripts as needed.
#os=windows mountcheck_cmd "perl c:/rush/etc/mountcheck"
#os=unix mountcheck_cmd "perl /usr/local/rush/etc/mountcheck"
NOTE: You can rewrite this script in python, or any other language you prefer,
as long as the language is installed on your machines. Just change 'perl'
in the above commands to 'python', 'sh', or whatever you prefer,
and rewrite the script in that scripting language.
If you are only concerned with running mountcheck on a single machine,
you can enable this script to be executed only on that particular
machine using a host= prefix to target a single machine
(or +hostgroup of machines), e.g.
host=tahoe mountcheck_cmd "perl /usr/local/rush/etc/mountcheck"
When enabled, this script will be run just before each render is started,
and should at minimum return one of the following exit codes indicating if the
machine is OK to render frames:
You should modify the logic of this script to do whatever checks
you want. Just be sure to keep it simple for the common case where
there's no errors detected, as any overhead will affect every frame rendered.
Mountcheck Exit Codes
Machine is OK, the render is started.
Machine is NOT OK, the render is NOT started:
the frame is requeued, and its TRY count increments by 1.
Other exit codes are reserved for future use.
For reasons that should be obvious, this script must exist LOCALLY on each
machine in order to be available when file servers might be down or unavailable.
Since the script lives in rush/etc, it can be pushed around the network with
'rush -push mountcheck +any'.
When enabled as shown above, this script will be executed before the render script
starts, and before the logfile for the render is created. Any output this script
generates is appended to the machine's local rushd.log. So only print messages
if there's an error, and keep messages terse. Include time stamps for clarity,
as shown in the example script.
This script runs as the user the renders run as, and will inherit all RUSH
environment variables the user's render script would have (e.g. RUSH_JOBID, RUSH_LOGFILE, etc).
If you need the script to do anything as root, you'll need to involve su(1),
or setuid scripts, or some other technique to escalate permissions.
WARNING: Keep the contents of this script simple. See 'Caveats' below to prevent
creating worse problems for your network.