Rush Logo Rush Render Queue - Hosts File
(C) Copyright 1995,2000 Greg Ercolano. All rights reserved.
V 102.43 10/10/07
Strikeout text indicates features not yet implemented




   Example Hosts File  

The following shows an example rush/etc/hosts file for a network of workstations (+work), render farm machines (+farm), and some miscellaneous machines being evaled (+eval). Some machines have maya (+maya), After Effects (+ae) and 'tahoe' is for administration use only (+admin), who's 'Cpus' field is set to '0' to prevent rendering.

Note the use of '+hostgroups' (+work,+farm,+maya,+ae..) to reflect groups of machines users commonly want to submit to, and the use of 'criteria' (linux,irix,sgi,win2k..) for platform names users might want to avoid using as a group.

    Example Hosts File 
    
    # HOST CACHING
    daemon-hostcache full
    app-hostcache    demand
    negcachesecs     900

    #
    # The 'Host' field should contain short names for hosts (aliases are ok),
    # and must be unique.
    #
    # The 'Criteria/Hostgroups' field must *NOT* contain white space, and words are
    # comma delimited. All hosts must contain '+any' in the Criteria/Hostgroups field.  
    #
    #Hostname   Cpus  Ram   MinPri  Criteria/Hostgroups
    #--------   ----  ----  ------  ------------------------------------
    tahoe       0     256   0       +any,+admin,linux
    ontario     1     2048  0       +any,+work,+maya,linux,linux6.0,intel
    superior    2     2048  0       +any,+work,+maya,sgi,irix,irix6.2
    erie        1     2048  0       +any,+work,+maya,+ae,win2k
    rf1         2     1024  0       +any,+farm,+maya,+ae,win2k
    rf2         2     1024  0       +any,+farm,+maya,+ae,win2k
    rf3         2     1024  0       +any,+farm,+maya,+ae,win2k
    rf4         2     1024  0       +any,+farm,+maya,+ae,win2k
    rf5         2     1024  0       +any,+farm,+maya,+ae,win2k
    eval1       2     1024  0       +any,+eval,mac
    eval2       2     1024  0       +any,+eval,linux
    eval3       2     1024  0       +any,+eval,windows

   File Format  

The $RUSH_DIR/etc/hosts file must contain the names of all hosts that participate in rendering.

The hosts file can be updated on the fly. Simply edit the file, make changes, then use 'rush -push hosts +any' to copy the local changes to all the machines, and the daemons will pick up your modifications within a minute.

The rushadmin tool makes this easy to do; just run rushadmin, click on 'hosts' to edit the file, save when done, and hit 'Send' to send the file to the network. Or you can use these commands.

Rush won't talk to any machine not in the rush hosts file, as a security measure. So be sure all machines (including license servers) are in the rush hosts file on all machines.

The format of the hosts file is single lines of 5 white space separated fields, one line per host:

Blank lines and lines starting with '#' are ignored.

   Daemon-HostCache  

Configures caching hostname-to-ip lookups in the rushd daemon.

Synopsis

    daemon-hostcache [options]

Options

    daemon-hostcache Options
    none No caching. Use the OS for all lookups.
    demand   Cache on demand, or whenever new hostlist is reloaded.
    full Cache on boot, or whenever hostlist reloaded.

Examples

    daemon-hostcache Examples
    daemon-hostcache full Caches hostname lookups on boot
    daemon-hostcache demand   Caches whenever requested
    daemon-hostcache none Caching disabled

Description

Configures caching hostname-to-ip lookups in the rush daemon (rushd). Normally this is set to 'full', so the daemon caches all the hostname lookups in the rush/etc/hosts file as soon as it starts up. This prevents rushd from having to make further hostname queries from the operating system.

If set to 'none', all hostname-to-ip lookups will be done by the operating system on a per-request basis. Useful if IPs are expected to change, but this is not recommended. Also, depending on the operating system, disabling caching will pass all name lookup load onto the operating system, especially if each request involves a network transaction to do the lookup (NIS, DNS).

Since rush can potentially make dozens of hostname lookups per second when jobs are active, it is highly recommended to leave this setting at the default; 'full'.

   App-HostCache  

Configures caching hostname-to-ip lookups in the rush client application.

Synopsis

    app-hostcache [options]

Options

    daemon-hostcache Options
    none No caching. Use the OS for all lookups.
    demand   Cache on demand (default)
    full Cache all hostnames first

Examples

    daemon-hostcache Examples
    app-hostcache full Caches all lookups whenever 'rush' is executed
    app-hostcache demand   Caches lookups on demand
    app-hostcache none No caching; OS does all hostname lookups

Description

Configures caching hostname-to-ip lookups in the rush client (rush). Normally this is set to 'demand', so rush(1) cache the lookups on demand, for the duration of its execution (which is usually short).

There are few situations where any setting other than 'demand' would be useful.

   NegCacheSecs  
(New in 102.42)

Sets the amount of time a bad hostname stays in the negative cache.

Synopsis 

    negcachesecs [seconds]
Examples 
    negcachesecs 900    # default 900 seconds (15 minutes)
    negcachesecs 0      # disable all negative caching
Description
Negative caching enables the system to remember bad hostname lookups after the first failed attempt, since bad hostname lookups can cause the rush daemons to be unresponsive while waiting for the OS to determine a hostname is unknown. This is especially a problem on Windows systems using WINS for hostname lookups, instead of DNS. (DNS is better)

With negative caching enabled to e.g. 900 seconds, a bad hostname lookup will be remembered, causing similar lookups for the same bad hostname to immediately fail during the 900 second period.

After the 900 seconds, the negative cache for that hostname is cleared, so that future lookups for the host will again be passed to the OS, in case the host comes back online.

900 is the default. 0 will completely disable negative caching.

   Hostname  
This is the name of the host, and should be the shortest name possible (e.g., host aliases can be used here).

This is the name that will be used in jobids and other cpu reports, so it is best if short names are used (10 chars or less). Longer names are ok, but will misalign columnar reports. Avoid using FQDN hostnames (e.g., foo.domain.com).

You can optionally specify an alternate network interface other than the default. Just append to the hostname a ':' followed by the name of the interface, e.g.:

    tahoe:tahoe-eth

This says 'tahoe' is the actual name of the machine (ie. hostname(1)), but rush should use tahoe's 'tahoe-eth' network interface for all Rush communications.

You can also specify an IP address after the ':' (for instance, if no hostname has been assigned to the other IP address) e.g.:

    tahoe:192.168.1.20

This says 'tahoe' should use the 192.168.1.20 interface for all Rush communications.

Such rush/etc/hosts file modifications need to be on all machines for communications to work properly.. don't just make the change to the rush/etc/hosts file on a single machine.

   Cpus  
This should be the number of cpus the host has. This is how many processes the host will run at the same time. This value can be larger or smaller than the actual number of physical cpus the machine has.

'0' is an acceptable value that effectively disables the machine from participating in rendering, while allowing the host to remain in the hosts file.

   Ram  
This is the amount of ram the machine has. This value can be less or more than the actual ram the machine has; usually this value takes into account some percentage of the host's swap space as well. This value is used when accepting frames to render; a frame that asks for more ram than the machine has will be turned away.

On multiprocessor machines, this value is a total from which rendering frames subtract their estimated ram use. For instance, if a 4 cpu machine is configured with a Ram value of 512, and 2 frames are currently rendering each with ram values 200, then only 112 will be left for rendering on the other two processors (112 = 512 - ( 200 x 2 ) ).

   MinimumPriority  
Use this value to set a limit on the minimum priority a job must have to render on this machine.

Useful where you want to prevent people from rendering on workstations unless they are of at least a certain priority, or if you want to allow only the local workstation user to submit to their own workstation using a policy enforced priority value.

A value of '0' allows all jobs. A value of '900' will only allow renders with a priority of 900 or above; renders with less than that will be turned away.

   Criteria/Hostgroups  
This is a list of comma separated strings that define platform or operating system specific features for the host. These can be arbitrary alpha-numeric strings that may also contain dashes, underbars and periods, but must not contain any whitespace. '+' characters have the special purpose of leading off a Host Group specification.

The <Criteria/Hostgroups> field might be set to:

+any,linux,linux6.1,prman3.7
These strings can then be used in TD's submit scripts to limit which hosts will render their frames. See the Criteria Submit Script command for more info. All hosts should have a criteria entry that at least contains +any.

Changing the '+any' to '+offline' has the useful effect of temporarily removing a machine from showing up in 'rushtop', 'All Cpus' and 'All Jobs' reports, without removing it from the rush/etc/hosts file. This is useful if a machine will be taken down for service for a short time.

Host Group names are configured in this field, too. To add a hostgroup called +servers to the above example:

+any,linux,linux6.1,prman3.7,+servers

   Negative Cache Description  
What is a 'negative cache'? (New in Rush 102.42)

Hostname lookups are a big part of rush's operation, and it's important that hostname lookups occur quickly.

When a bad hostname is passed to rush, it quickly determines its not in the hosts file, and passes the name on to DNS for resolution. DNS lookups are expensive; it can take a while for the DNS server to determine a hostname is unknown. This will cause rush to spend a few seconds waiting for DNS to say it's an unknown hostname.

During this time, the daemon may be unresponsive.

In situations where a misconfigured script or user typo is repeated many times, these repeat bad lookups can effectively end up causing a 'denial of service'.

By using a "negative cache", Rush keeps track of unknown hostnames, so when a bad hostname is requested a second time, the daemon will immediately indicate the hostname is bad, based on its 'negative cache', avoiding the expensive DNS lookups and unresponsiveness. The reasoning being, if DNS just said the hostname was unknown 5 seconds ago, it is likely that will still be the case now, so just save DNS the trouble, and indicate it's unknown immediately.

The only problem with this is if the sysadmin recently added a new host to DNS.. if the bad name is now a good name, the negative cache will prevent the name from being passed to DNS and being resolved.

The solution is to make the negative cache timeout automatically after a given amount of time. By default, the first request for a bad hostname (ie. a hostname that fails a DNS lookup) will be 'negative cached' in the daemon for up to 15 minutes. The 15 minute value is configurable via the rush hosts file) via the negcachesecs value.

The 'negative cache' for a daemon can be inspected via 'rush -lah <hostname>' command; the current negative cache will be shown at the bottom of the 'rush -lah' report. (See link for examples)

When the 15 minutes are up, the daemon clears that entry from the cache, so the next request will be checked through DNS again, just in case it's now a valid hostname. (if it's not, it will be negative cached again, as above)

Whenever the rush/etc/hosts file is reloaded, rush will automatically clear the negative cache. This will happen within 60 seconds of the hosts file's date stamp changing, or if the sysadmin invokes the 'rush -reload' command. To force the entire network to clear the negative cache, and reload the rush/etc/hosts files without sending out a new hosts file, the sysadmin can use 'rush -reload hosts +any -t 4'.