> We depend heavily on environment variables for our production pipeline.
> It seems that when I submit a job the environment from the submitting
> shell isn't available at rendertime (I'm just doing print(%ENV, "\n")
> in place of the actual rendering in the submit-renderman.pl script).
Correct.
Rush runs the renders with a 'bare' environment; it is up
to the render script to set or gather up whatever environment
it needs to run the renders.
There are two common techniques:
1) Set the variables in the render script.
This way the script can automatically determine the local
OS, and set the variables appropriate for that platform, eg:
if ( -d "/windows" ) {
$ENV{PATH} = "C:/foo/bar;$ENV{PATH}"; ### WINDOWS
} elsif ( -d "/Applications" ) {
$ENV{PATH} = "/Applications/foo:$ENV{PATH}"; ### OSX
} else {
$ENV{PATH} = "/usr/local/bin:$ENV{PATH}"; ### LINUX
}
2) Source the environment from centrally maintained rc files.
For instance, a render script that uses MAYA and MTOR:
#!/bin/csh -f
#
# RENDER SCRIPT
#
source /usr/sohovfx/adm/.maya # maintained by sohovfx admins
source /usr/sohovfx/adm/.mtor # maintained by sohovfx admins
maya -render ..
The reason Rush does not send the local environment by default
is because a) it would pollute the remote environment with variables
that are often not relevant on the remote machines (esp in cross
platform environments), and b) it makes handoffs between users
buggy; eg. a TD hands off the job to a render watcher, where
the jobs submit fine if submitted by the TD, but not by the
render watcher because of differences in environments. Or, a TD
hands off their rendering to another TD with a different environment.
Also, most companies invoke the example submit script GUIs from
their desktop, which has no useful environment context.
Rush definitely does not do /anything/ with .cshrc or .login
files. The only way those could come into play is if you write
your own CSH render scripts, and leave out the -f flag
from the #!/bin/csh line, causing the user's .cshrc to be sourced.
I DONT advise this; definitely specify the -f flag for csh scripts.
> Is there an easy way to get the environment into the render context?
Yes, as you mention, you can write the environment out from your
submit script to eg. a ".env" file in the "log directory" (unique
to each render), then source it into the render script using
$RUSH_LOGFILE to 'find' the .env file.
Another way is to pass simple settings (such as show/shot/sequence info)
from the submit script as arguments to the render script, eg:
#!/bin/csh -f
#
# SUBMIT SCRIPT
#
echo SHOW is $SHOW
echo SHOT is $SHOT
rush -submit << EOF
:
command /server/scripts/my-render-script $SHOW $SHOT
: ^^^^^ ^^^^^
EOF $1 $2
..then in the render script, pick up the arguments:
#!/bin/csh -f
#
# RENDER SCRIPT
#
setenv SHOW $1; echo SHOW is $1
setenv SHOT $2; echo SHOT is $2
:
> This is a very useful feature of [other render queue systems]
It is not something I have Rush do by default, because it
makes problems for handoffs between users (eg. for render
watchers or people or handoff before going on vacation),
and makes cross platform rendering impossible.
Also, most company's user environments are messy, and inconsistent..
they don't port well even across similar platforms.
My feeling was, any company that has their user environment
together is very aware of technical issues, and has their own
custom way of dealing with things, in which case they can deal
with passing around specific environment settings, and would
probably want full control over that logic.
So, to keep things simple, rush's environment is very bare
bones, and it's up to the render script to set up whatever
is needed (eg. maya specific variables like PATH, PLUG_IN_PATH,
LD_LIBRARY_PATH, etc etc)
> suspect that I could write the environment out to a file in the
> .submit file and then just read it back in in the .render file
> (am I correct about that?) but I'd hope for an automatic solution.
Correct, though I don't advise doing it wholesale; some variables
don't port well, even across machines of the same platform. ($DISPLAY
would be one)
Usually, either you want to pass a little data, like SHOW/SHOT info,
in which case pass these as arguments to the render script.
Or, if you want to pass large amounts of data, use a file written
out the the render's log directory.. a safe place for data that
needs to be passed to all the machines.
eg. in a csh script, you could save/load the environment this way:
# SUBMIT SCRIPT
printenv | sed 's/^/setenv /;s/=/ "/;s/$/"/' > $logdir/.env
# RENDER SCRIPT
source $logdir/.env
> Note that simply sourcing .cshrc files when the render starts is
> not an option because as the user works on different shots the
> environment changes (ie the environment at submit time is most
> certainly different from the environment at login time).
If you have logic based on show/shot variables, then pass the
show/shot variables to the render script, and have the render script
then run the logic that sets the rest of the variables appropriately
at render time.
Assuming you have things modularized, where 'show' and 'shot' are
possibly aliases that set variables, then sources a script that
sets other environment variables based on some logic, put this
into your render script, eg.
#!/bin/csh -f
#
# RENDER SCRIPT
# Expects $1 to be the show name, and $2 to be the shot
#
source /usr/sohovfs/adm/.stdenv # configures 'show' and 'shot' aliases
show $1 # invokes 'show' alias
shot $2 # invokes 'shot' alias
# At this point, environment is 'configured'
maya -render ..
I've worked at a lot of companies, both well organized and disorganized,
and I designed rush to work for both.
|