Antoine Durr wrote:
> Is there a way to have the set of hostgroups to which a host belongs
> be configured from a program, rather than entered statically into the
> Rush hosts file?
No, rush only gets its hostgroup info from the rush/etc/hosts file.
Rush parses and caches this file, and only reloads it when it
sees the datestamp change, or when 'rush -reload hosts' is used
to force a reload immediately.
There are no hooks where rush can 'run a program' to get this
info. That could maybe be arranged, just for the hostgroup info,
but I didn't want to go there, because future modifications might
involve accessing the hostgroup information fairly often, if it
were to ever be changed to being dynamic.
So it will probably always be a file that is 'cached', updated
only when changed or 'poked' with a reload, to keep things from
bottlenecking.
In a very old render queue design, I tried to implement scheduling
and host grouping as a script, and I very quickly found that
the script was too much overhead (too slow) to manage the high
turnaround needed in high volume environments, so I stopped
researching that idea pretty quickly.
> I have a standalone program (called 'hostgroups', appropriately
> enought!) that if I give it 'fgs2' as a parameter, returns a list that
> looks similar to '+any,+linux,+server'. How do I wedge my program into
> this list here or onto what Rush things are the host groups?
If you're trying to make jobs be able to change memberships
on the fly, I would attack it differently, where you have
a 'watcher script' that changes jobs on the fly, bumping
around the cpu cap values.
eg. if you know your job will need to be in the +a, +b and +c
groups, submit the job with:
cpus +a=0 +b=0 +c=0
..and then let the 'watcher script' bump the =0 values (cpu caps)
up and down with the 'rush -cp' command, eg:
rush -cp +a=0 +a=3 somejob.123
Rush is designed so you can do that on the fly without causing
too much effort on the scheduler. It also allows you to decrease
cpu caps without killing rendering frames, as opposed to actually
REMOVING cpus (with 'rush -rc') which /will/ kill frames. I would
not advise using -rc/-ac on the fly either; it creates a lot of
effort to the daemon on large networks. -cp is better for batting
priorities and cpu limits around.
|