Mr. Daniel Browne wrote:
> I'm writing a basic submit script to perform a unix cp copy =
> operation on the farm.
It sounds like you're submitting a job that copies frames,
but want it to be done with one command, as opposed to one
copy command per frame..?
Since copying is pure I/O, I would think the best place to
run a large frame copy command is only on the file server,
to avoid network in/network out, so that the copy operation
is entirely local.
For this reason I'm not sure I understand the question;
normally one wants to run things through the render queue
to harness the power of parallel cpu use, where each machine
works on a frame or batch of frames. Either that, or you want
to throw a single operation at some available machine, and have
it do the entire operation as a 'single frame rush job'.
> Is there a routine already available in Rush to =
> perform shell substitution of the digits in a frame range, or would I =
> have to do that myself?
Hmm, I might need a few more details.
If you're making a submit script to do a copy, the scripting
language you're using should work well for doing shell substitution.
Do not try to submit to rush a simple 'cp' command, but rather,
have the script submit itself, so that the script runs the cp command
on each machine, so that you'll have all the facilities of the
scripting language you're working with at your finger tips.
(ie. shell wildcard expansion, sed/perl/awk access, etc)
For some simple examples of writing scripts that
"submit themselves", see:
http://www.seriss.com/rush-current/rush/rush-submit.html
For digits in a frame range, there are two variables rush supplies
at execution time on all the machines; RUSH_FRAME and RUSH_PADFRAME,
the latter being 0000 format, and the former having no padding at all.
If you need some different padding, you can use printf() to change
RUSH_FRAME into whatever padding you want, eg:
perl -e 'printf("%06d", $ENV{RUSH_FRAME});'
If you're trying to do shell substitution, I'm not sure, do you
mean wild card expansions, like:
cp /some/directory/foo.[0-9]*.tif /some/other/directory/
..or do you really mean substitution, like sed/perl regex
to turn e.g. foo.0001.tif into foo.0002.tif?
> The alternative of course is to do a cp =
> operation for each frame, but I was hoping to avoid the additional =
> overhead.
I wouldn't think the cp command would be much overhead,
but I might be missing something.
If you're trying to distribute the cp commands to a bunch
of machines to parallelize the I/O, I'd think you'd want
to do them either as single frames or in batches.
Or better yet, batch several commands on the file server
itself, eg:
cp foo.0*.tif /some/other/dir & # copies 1000 frames (0000 - 0999) in bg
cp foo.1*.tif /some/other/dir & # copies 1000 frames (1000 - 1999) in bg
cp foo.2*.tif /some/other/dir & # etc..
..which if the box has multiple procs and the network is idle,
should go really fast.
If you can follow up with more specifics, I can probably
help you narrow down a specific technique.
--
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)
|