On 12/16/11 18:31, Mr. Daniel Browne wrote:
> I've come upon a stumbling block; making a safety copy that the render is run from. Though you can add the %s wildcard into the log directory name for the JobID, that would mean having to make the copy into the job sub-directory in the Render portion of the script which could lead to race conditions. Is there another mechanism I can use? Is it possible for me to have submit get passed back to itself so that the completion happens and then the copy can take place in a single action before any batches start?
If you want to copy data into the job's log directory
for storage, sounds like what the jobstartcommand is for.
That's the cleanest way, I'd think.
1. JOBSTARTCOMMAND
------------------
The jobstartcommand runs just before the first frame
starts.
The jobstartcommand will be passed the logdir (via
the RUSH_LOGFILE and/or RUSH_LOGDIR env variables)
and the command's own output will in fact be sent
to the jobstartcommand.log in the logdir.
You can pass arguments to the script to tell it e.g
where the source material is, or you can point to a
file that is a manifest of data.
The jobstartcommand can be the submit script itself,
passed a special argument to tell it what to do.
For instance I often have the submit scripts
understand these arguments:
-submit -- handle submitting the job
-render -- handle rendering a frame
-jobstartcommand -- handle the jobstartcommand (if any)
-jobdonecommand -- handle the jobdonecommand (if any)
[..etc..]
This way the submit script can set up the render queue
to use itself to do all the different operations.
2. PAUSE APPROACH
-----------------
The other way would be to again submit the job
in either the paused state, or with 0 cpus assigned
to it, so you can then do your work, then kick the
job into gear once you're done.
Or an even different route: don't use the jobid at all..
3. CREATE A UNIQUE DIRNAME
--------------------------
Use some combo of time of day, username, hostname,
and even the pid of the script itself if need be.
That way you always end up with a unique pathname
for the job, so you don't end up with any races
by depending on attributes of the job. (ie. define
your own attributes, then use those)
* * *
If you still have trouble, give me the whole picture
of what you're doing, as choosing the best route to take
often involves knowing the big picture first.
--
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)ext.23
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)
|