*** ***************************************************** *** Message #1927 ***************************************************** *** ***************************************************** Subject: Re: [SUBMIT] auto confirm/submit Form From: Stephan Kosinski Date: Fri, 05 Mar 2010 14:41:54 -0500 Thank you so much Greg. I'll try that. Stephan. Greg Ercolano wrote: > Stephan Kosinski wrote: >> [posted to rush.general] >> >> Thank you very much Greg, it works perfectly. >> >> That will even allows me to generate one file per layer to render. >> >> One last thing, the "Be very Happy" window is still popping-up, which is >> really nice :) , but annoying when I have several to close after I >> launched several forms at a time. Any way to prevent this? > > Ah, that. > > Yes, I recently added the ability to turn that off, which will > be in the next release. (The new code takes a flexible argument > list that lets one disable that dialog) > > The code responsible for opening that dialog is in the RushSubmit() > function in .common.pl, specifically: > > ---- > open(STARTIRUSH, "|$inputcmd"); > print STARTIRUSH "-----------------\n" . > "--- SUBMIT OK ---\n" . > "-----------------\n\n" . > $msg; > unless ( close(STARTIRUSH) ) { return(0); } # picked "No Irush" > ---- > > What I'd suggest as a workaround is putting a conditional around that code > that checks an environment variable (which you'd set in your application > to turn that off), ie: > > ---- > if ( ! defined($ENV{RUSH_NO_SUCCESS_DIALOG}) ) ### ADD THIS LINE > { ### ADD THIS LINE > open(STARTIRUSH, "|$inputcmd"); > print STARTIRUSH "-----------------\n" . > "--- SUBMIT OK ---\n" . > "-----------------\n\n" . > $msg; > unless ( close(STARTIRUSH) ) { return(0); } # picked "No Irush" > } ### ADD THIS LINE > ---- > > ..then in your script, just before you invoke the submit script to > submit the job, set that environment variable RUSH_NO_SUCCESS_DIALOG to 1. > That will prevent the dialog from opening. > > You can set environment variables from all scripting languages, > including 'mel', which I think would be with: > > putenv("RUSH_NO_SUCCESS_DIALOG", "1"); > > Meanwhile other users invoking the submit script interactively > will continue to have the normal behavior, because that variable > would not be set. > > Let me know if that doesn't work.. I'd think it should. > *** ***************************************************** *** Message #1926 ***************************************************** *** ***************************************************** Subject: Re: [SUBMIT] auto confirm/submit Form From: Greg Ercolano Date: Thu, 04 Mar 2010 23:45:20 -0500 Stephan Kosinski wrote: > [posted to rush.general] > > Thank you very much Greg, it works perfectly. > > That will even allows me to generate one file per layer to render. > > One last thing, the "Be very Happy" window is still popping-up, which is > really nice :) , but annoying when I have several to close after I > launched several forms at a time. Any way to prevent this? Ah, that. Yes, I recently added the ability to turn that off, which will be in the next release. (The new code takes a flexible argument list that lets one disable that dialog) The code responsible for opening that dialog is in the RushSubmit() function in .common.pl, specifically: ---- open(STARTIRUSH, "|$inputcmd"); print STARTIRUSH "-----------------\n" . "--- SUBMIT OK ---\n" . "-----------------\n\n" . $msg; unless ( close(STARTIRUSH) ) { return(0); } # picked "No Irush" ---- What I'd suggest as a workaround is putting a conditional around that code that checks an environment variable (which you'd set in your application to turn that off), ie: ---- if ( ! defined($ENV{RUSH_NO_SUCCESS_DIALOG}) ) ### ADD THIS LINE { ### ADD THIS LINE open(STARTIRUSH, "|$inputcmd"); print STARTIRUSH "-----------------\n" . "--- SUBMIT OK ---\n" . "-----------------\n\n" . $msg; unless ( close(STARTIRUSH) ) { return(0); } # picked "No Irush" } ### ADD THIS LINE ---- ..then in your script, just before you invoke the submit script to submit the job, set that environment variable RUSH_NO_SUCCESS_DIALOG to 1. That will prevent the dialog from opening. You can set environment variables from all scripting languages, including 'mel', which I think would be with: putenv("RUSH_NO_SUCCESS_DIALOG", "1"); Meanwhile other users invoking the submit script interactively will continue to have the normal behavior, because that variable would not be set. Let me know if that doesn't work.. I'd think it should. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1925 ***************************************************** *** ***************************************************** Subject: Re: [SUBMIT] auto confirm/submit Form From: Stephan Kosinski Date: Thu, 04 Mar 2010 21:41:44 -0500 Thank you very much Greg, it works perfectly. That will even allows me to generate one file per layer to render. One last thing, the "Be very Happy" window is still popping-up, which is really nice :) , but annoying when I have several to close after I launched several forms at a time. Any way to prevent this? Thank you again for your precious help. Stephan. Greg Ercolano wrote: > Stephan Kosinski wrote: >> I am using this kind of command to generate and submit maya render form: >> >> submit-maya.pl -field JobTitle jobname -field ScenePath myScenePath >> etc..... >> >> It is really convenient since I can launch renders within maya but >> would be even more convenient I didn't have to click on the "Submit" >> button to launch the render on the farm. >> >> Is there a way to auto-confirm the form so it goes to the farm as soon >> as hit "enter" in my shell? > > Yes. It should be as easy as: > > perl //path/to/submit-maya.pl -submit /var/tmp/myfile.txt > > ..where the /var/tmp/myfile.txt contains the key/value pairs > of the submit form, which you can generate from within maya, > or from another script. > > When you run that command, the job will go straight to the farm. > > To do that, you can simulate what the submit script does when the > user hits the 'Submit' button, basically these two steps: > > 1) Save out a simple text file that contains the key/value pairs > for all the submit form's fields (see below) > > 2) Invoke the submit script with the arguments "-submit file" > where "file" is the pathname of the above text file. eg: > > perl //path/to/submit-maya.pl -submit /var/tmp/myfile.txt > > ..where /var/tmp/myfile.txt is a file that contains key/value pairs > of all the submit fields. > > When you submit a maya job, a copy of this key/value file > is saved in the user's home directory as: > > UNIX: ~/.rush/.submit-maya-last > WINDOWS: c:\temp\.rush-\.submit-maya-last > > So for instance, to resubmit the last maya job you submitted, > under unix you might do: > > perl //path/to/submit-maya.pl -submit ~/.rush/.submit-maya-last > > To generate the key/value file yourself, you can nab a copy of the > one the rush submit form generates, and use that as a template in your > own script, changing the fields as you want (Frames:, Cpus:, etc). > > It's important that the key/value file have the same fields the submit > script expects. Be aware that newer versions of the submit-maya script > sometimes have new fields, so your app would need to keep in sync with these. > > What follows is a unix bash script that does this. > If you need an example in some other language, let me know. > > Note you may need to modify the key/value data with one of your own > submit-maya-last files, and you'll need to change the pathname of > to the submit script for the location on your file server. > > * * * > > #!/bin/sh > > ### CREATE A KEY VALUE PAIR FILE > cat << EOF > /var/tmp/myfile.txt > AutoDump: off > BatchClip: yes > BatchFrames: 1 > Cpus: +any=5 > Debug: off > DoneMail: > DumpMail: > Frames: 1-10 > ImageCommand: > ImageDirectory: > JobDoneCommand: > JobDumpCommand: > JobStartCommand: > JobTitle: > LicenseBehavior: Licpause+Retry > LogDirectory: > LogFlags: Overwrite > MaxLogSize: 0 > MaxTime: 00:00:00 > MaxTimeState: Que > MayaFlags: > MrayCommand: ray > MrayFlags: > MrayVerbosity: 5 > NeverUseCpus: > PrintEnvironment: off > Project: > Ram: 1 > Renderer: maya(sw) > Retries: 3 > RetryBehavior: Fail > SaveMIDirectory: > ScenePath: //meade/net/tmp/scenes/foo.ma > StartIrush: yes > SubmitHosts: > SubmitOptions: > Threads: all > WaitFor: > WaitForState: Dump > _SAVE_ID: Maya-Submit > EOF > > ### NOW RUN THE SUBMIT-MAYA SCRIPT TO SUBMIT IT AS A JOB > perl //yourserver/rushscripts/submit-maya.pl -submit /var/tmp/myfile.txt > *** ***************************************************** *** Message #1924 ***************************************************** *** ***************************************************** Subject: Re: [SUBMIT] auto confirm/submit Form From: Greg Ercolano Date: Thu, 04 Mar 2010 21:14:34 -0500 Stephan Kosinski wrote: > I am using this kind of command to generate and submit maya render form: > > submit-maya.pl -field JobTitle jobname -field ScenePath myScenePath > etc..... > > It is really convenient since I can launch renders within maya but > would be even more convenient I didn't have to click on the "Submit" > button to launch the render on the farm. > > Is there a way to auto-confirm the form so it goes to the farm as soon > as hit "enter" in my shell? Yes. It should be as easy as: perl //path/to/submit-maya.pl -submit /var/tmp/myfile.txt ..where the /var/tmp/myfile.txt contains the key/value pairs of the submit form, which you can generate from within maya, or from another script. When you run that command, the job will go straight to the farm. To do that, you can simulate what the submit script does when the user hits the 'Submit' button, basically these two steps: 1) Save out a simple text file that contains the key/value pairs for all the submit form's fields (see below) 2) Invoke the submit script with the arguments "-submit file" where "file" is the pathname of the above text file. eg: perl //path/to/submit-maya.pl -submit /var/tmp/myfile.txt ..where /var/tmp/myfile.txt is a file that contains key/value pairs of all the submit fields. When you submit a maya job, a copy of this key/value file is saved in the user's home directory as: UNIX: ~/.rush/.submit-maya-last WINDOWS: c:\temp\.rush-\.submit-maya-last So for instance, to resubmit the last maya job you submitted, under unix you might do: perl //path/to/submit-maya.pl -submit ~/.rush/.submit-maya-last To generate the key/value file yourself, you can nab a copy of the one the rush submit form generates, and use that as a template in your own script, changing the fields as you want (Frames:, Cpus:, etc). It's important that the key/value file have the same fields the submit script expects. Be aware that newer versions of the submit-maya script sometimes have new fields, so your app would need to keep in sync with these. What follows is a unix bash script that does this. If you need an example in some other language, let me know. Note you may need to modify the key/value data with one of your own submit-maya-last files, and you'll need to change the pathname of to the submit script for the location on your file server. * * * #!/bin/sh ### CREATE A KEY VALUE PAIR FILE cat << EOF > /var/tmp/myfile.txt AutoDump: off BatchClip: yes BatchFrames: 1 Cpus: +any=5 Debug: off DoneMail: DumpMail: Frames: 1-10 ImageCommand: ImageDirectory: JobDoneCommand: JobDumpCommand: JobStartCommand: JobTitle: LicenseBehavior: Licpause+Retry LogDirectory: LogFlags: Overwrite MaxLogSize: 0 MaxTime: 00:00:00 MaxTimeState: Que MayaFlags: MrayCommand: ray MrayFlags: MrayVerbosity: 5 NeverUseCpus: PrintEnvironment: off Project: Ram: 1 Renderer: maya(sw) Retries: 3 RetryBehavior: Fail SaveMIDirectory: ScenePath: //meade/net/tmp/scenes/foo.ma StartIrush: yes SubmitHosts: SubmitOptions: Threads: all WaitFor: WaitForState: Dump _SAVE_ID: Maya-Submit EOF ### NOW RUN THE SUBMIT-MAYA SCRIPT TO SUBMIT IT AS A JOB perl //yourserver/rushscripts/submit-maya.pl -submit /var/tmp/myfile.txt -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1923 ***************************************************** *** ***************************************************** Subject: [SUBMIT] auto confirm/submit Form From: Stephan Kosinski Date: Thu, 04 Mar 2010 20:49:21 -0500 Hello, I am using this kind of command to generate and submit maya render form: submit-maya.pl -field JobTitle jobname -field ScenePath myScenePath etc..... It is really convenient since I can launch renders within maya but would be even more convenient I didn't have to click on the "Submit" button to launch the render on the farm. Is there a way to auto-confirm the form so it goes to the farm as soon as hit "enter" in my shell? Thank you very much for your help. Stephan. *** ***************************************************** *** Message #1922 ***************************************************** *** ***************************************************** Subject: [Q+A] Maya + Vray From: Greg Ercolano Date: Tue, 02 Mar 2010 04:49:38 -0500 > We'd like to render with Vray for Maya. > > But our submit-maya script doesn't have a 'vray' option > in the submit form for "Renderer".. what should we do to render with Vray? Just set the "Renderer:" field to "default" in the submit form, and it will use the renderer the scene file was set to use. So if the scene was set to use vray, that's what Maya will use to render the scene if the job was submitted with "Renderer: default". That goes for just about all the Maya plugin renderers. That's all you should need. If, however, you want to be able to /force/ Maya renders to use vray even if the scene isn't set up to use it, and you have a submit-maya script that doesn't have 'vray' in the Renderer: pulldown menu, it's easy to add it; just add "vray(vray)" to the end of this line in the submit-maya script: Renderer: "default,maya(sw),mentalray(mr),mentalray-standalone(mi),mi-gen(mi),hardware(hw),vector(vr),renderman(rman),ribgen(rib),3Delight(3delight),maxwell(maxwell)" ? For example: BEFORE: Renderer: "default,maya(sw),mentalray(mr),mentalray-standalone(mi),mi-gen(mi),hardware(hw),vector(vr),renderman(rman),ribgen(rib),3Delight(3delight),maxwell(maxwell)" ? AFTER: Renderer: "default,maya(sw),mentalray(mr),mentalray-standalone(mi),mi-gen(mi),hardware(hw),vector(vr),renderman(rman),ribgen(rib),3Delight(3delight),maxwell(maxwell),vray(vray)" ? ..that way 'vray' will be in the Renderer: pulldown so you can select it to force the render to run with vray. But it's much easier to just leave the submit form set to "default". *** ***************************************************** *** Message #1921 ***************************************************** *** ***************************************************** Subject: Re: nuke path From: Greg Ercolano Date: Tue, 26 Jan 2010 15:16:52 -0500 I believe the problem here was one of permissions on the license file or directory, where the user the Rush render was running as didn't have read permission to the license file, so nuke couldn't load it. The way to debug this sort of thing is to login to the machine in question as the user the rush render is running as, which you can see from the "Host:" and "RunningAs:" lines in the rush log. To determine the problem, you would login as that user on that machine, and then try to 'more' the license file to see if you have permission to read it, eg: more "/Applications/Nuke5.0v2/Nuke5.0v2.app/Contents/MacOS/*.lic" more "/usr/local/foundry/FLEXlm/foundry.lic" more "/Library/Application Support/TheFoundry/FLEXlm/*.lic" Note that quotes are needed if the pathnames contain spaces. The above are unix commands, though similar commands should work in DOS as well, only you'd need to avoid the '*' wildcards, as the DOS shell doesn't handle wildcard expansions the same way as unix shells. (In DOS, wild cards are handled by the command, not the shell, and only certain DOS commands, like "DIR", can handle wildcards) > [..] now I get: > > [..] > Nuke 5.0v2, built May 28 2008, revision 6380. > Copyright (C) 2008 The Foundry Visionmongers Ltd. All Rights Reserved. > License failure: License server machine is down or not responding. > See the system adminstrator about starting the license server system, or > make sure you're referring to the right host (see LM_LICENSE_FILE). > Feature: nuke_i > Hostname: lserver > License path: /Applications/Nuke5.0v2/Nuke5.0v2.app/Contents/MacOS/*.lic: [..] > /Library/Application Support/TheFoundry/FLEXlm/*.lic:/usr/local/foundry/FLEXlm/foundry.lic > FLEXnet Licensing error:-96,7. System Error: 2 "No such file or directory" > For further information, refer to the FLEXnet Licensing End User Guide, > available at "www.macrovision.com". > --- NUKE LICENSE ERROR: EXITCODE=100 *** ***************************************************** *** Message #1920 ***************************************************** *** ***************************************************** Subject: Re: Batch Copy Script From: Greg Ercolano Date: Wed, 16 Dec 2009 14:27:09 -0500 Mr. Daniel Browne wrote: >>> The alternative of course is to do a cp operation for each frame, >>> but I was hoping to avoid the additional overhead. >> >> I wouldn't think the cp command would be much overhead, >> but I might be missing something. >> >> If you're trying to distribute the cp commands to a bunch >> of machines to parallelize the I/O, I'd think you'd want >> to do them either as single frames or in batches. >> >> Or better yet, batch several commands on the file server >> itself, eg: >> >> cp foo.0*.tif /some/other/dir & # copies 1000 = frames (0000 - 0999) in bg >> cp foo.1*.tif /some/other/dir & # copies 1000 = frames (1000 - 1999) in bg >> cp foo.2*.tif /some/other/dir & # etc.. >> >> ..which if the box has multiple procs and the network is idle, >> should go really fast. > Yes, this is what I wanted to go for. The problem obviously is that you > can't wildcard a frame range as a whole; you have to wildcard each digit > with its potential range of values. Tedious, but possible. I was just > hoping you might have a library routine already in place to do this. If you're using perl, you can use perl's built in copy command and make a loop, eg: use File::Copy; [..] for ( $t=$sfrm; $t<$efrm; $t++ ) { my $src = sprintf("/some/src/path/foo.%04d.tif", $t); my $dst = sprintf("/some/dst/path/foo.%04d.tif", $t); copy($src,$dst); } ..however you might find the operating system's own copy command to be faster (these might be optimized to take advantage of threading) in which case: for ( $t=$sfrm; $t<$efrm; $t++ ) { my $src = sprintf("/some/src/path/foo.%04d.tif", $t); my $dst = sprintf("/some/dst/path/foo.%04d.tif", $t); system("cp $src $dst"); } ..and you could probably get 3 or 4 of those going at a time with use of fork()/wait() or threads. (I'd suggest throttling this to limit starting too many at a time, and using wait(), you can start the next one as soon as one gets done, so that there are always 3 or 4 going at a time, until you work your way through the entire loop) For instance, I have a script that I wrote a while ago: http://seriss.com/people/erco/unixtools/prun ..the blurb description: ********************************************************************************* Runs unix commands in parallel Supply a list of commands to run, one per line, and it runs them all in parallel. You can limit the number of parallel processes with the -M flag. Even keeps track of all the stdout/stderr logs, serializing them for easy reading. Hit ^C to stop all processes. Great for multiple rsh, rcp or rdists to run on a whole network quickly. Very small, simple program no one has time to write, but everyone always needs when managing multiple hosts. ********************************************************************************* So it takes a file of one liner unix commands, and runs them in parallel with a throttle, backgrounding several at a time, each time one command gets done it moves further onto the next in the list (like rush's frame queue) keeping the throttle maxed. It was handy for running commands many copy commands in parallel and keeping the output syncrhonized. Might be helpful with or without Rush..! I hope it still works.. it looks like I wrote it last century, as it still uses pre-perl5 syntax. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1919 ***************************************************** *** ***************************************************** Subject: Re: Batch Copy Script From: "Mr. Daniel Browne" Date: Wed, 16 Dec 2009 13:24:03 -0500 >=20 > On Dec 15, 2009, at 9:22 PM, Greg Ercolano wrote: >=20 > [posted to rush.general] >=20 > Mr. Daniel Browne wrote: >> I'm writing a basic submit script to perform a unix cp copy =3D >> operation on the farm. >=20 > It sounds like you're submitting a job that copies frames, > but want it to be done with one command, as opposed to one > copy command per frame..? Correct >=20 > Since copying is pure I/O, I would think the best place to > run a large frame copy command is only on the file server, > to avoid network in/network out, so that the copy operation > is entirely local. Unfortunately the BlueArc architecture currently doesn't allow you do to = operations like that within the NAS head. In fact, most of the head's = shell commands do not support recursive operations. >=20 > For this reason I'm not sure I understand the question; > normally one wants to run things through the render queue > to harness the power of parallel cpu use, where each machine > works on a frame or batch of frames. Either that, or you want > to throw a single operation at some available machine, and have > it do the entire operation as a 'single frame rush job'. >=20 We had thought since we perform other operations on frames to and from = volumes on the NAS head that we could do the same with copy operations. = I admit, it sounds a bit zany. At the very least I do have a script that = would allow me to hand off the copy operation to a single farm machine = as opposed to splitting it into pieces. Theoretically this would be = faster as the head is executing these operations in parallel with the = help of its cache memory. >=20 > If you're trying to do shell substitution, I'm not sure, do you > mean wild card expansions, like: >=20 > cp /some/directory/foo.[0-9]*.tif /some/other/directory/ >=20 >> The alternative of course is to do a cp =3D >> operation for each frame, but I was hoping to avoid the additional =3D >> overhead. >=20 > I wouldn't think the cp command would be much overhead, > but I might be missing something. >=20 > If you're trying to distribute the cp commands to a bunch > of machines to parallelize the I/O, I'd think you'd want > to do them either as single frames or in batches. >=20 > Or better yet, batch several commands on the file server > itself, eg: >=20 > cp foo.0*.tif /some/other/dir & # copies 1000 = frames (0000 - 0999) in bg > cp foo.1*.tif /some/other/dir & # copies 1000 = frames (1000 - 1999) in bg > cp foo.2*.tif /some/other/dir & # etc.. >=20 > ..which if the box has multiple procs and the network is idle, > should go really fast. >=20 > If you can follow up with more specifics, I can probably > help you narrow down a specific technique. Yes, this is what I wanted to go for. The problem obviously is that you = can't wildcard a frame range as a whole; you have to wildcard each digit = with its potential range of values. Tedious, but possible. I was just = hoping you might have a library routine already in place to do this. I think I know what to do now; thanks Greg. Happy holidays. -Dan >=20 > --=20 > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) >=20 >=20 > ---------- > Dan "Doc" Browne > System Administrator >=20 > Evil Eye Pictures > d.list at evileyepictures.com > Office: (415) 777-0666 >=20 *** ***************************************************** *** Message #1918 ***************************************************** *** ***************************************************** Subject: Re: Batch Copy Script From: Greg Ercolano Date: Wed, 16 Dec 2009 00:22:30 -0500 Mr. Daniel Browne wrote: > I'm writing a basic submit script to perform a unix cp copy = > operation on the farm. It sounds like you're submitting a job that copies frames, but want it to be done with one command, as opposed to one copy command per frame..? Since copying is pure I/O, I would think the best place to run a large frame copy command is only on the file server, to avoid network in/network out, so that the copy operation is entirely local. For this reason I'm not sure I understand the question; normally one wants to run things through the render queue to harness the power of parallel cpu use, where each machine works on a frame or batch of frames. Either that, or you want to throw a single operation at some available machine, and have it do the entire operation as a 'single frame rush job'. > Is there a routine already available in Rush to = > perform shell substitution of the digits in a frame range, or would I = > have to do that myself? Hmm, I might need a few more details. If you're making a submit script to do a copy, the scripting language you're using should work well for doing shell substitution. Do not try to submit to rush a simple 'cp' command, but rather, have the script submit itself, so that the script runs the cp command on each machine, so that you'll have all the facilities of the scripting language you're working with at your finger tips. (ie. shell wildcard expansion, sed/perl/awk access, etc) For some simple examples of writing scripts that "submit themselves", see: http://www.seriss.com/rush-current/rush/rush-submit.html For digits in a frame range, there are two variables rush supplies at execution time on all the machines; RUSH_FRAME and RUSH_PADFRAME, the latter being 0000 format, and the former having no padding at all. If you need some different padding, you can use printf() to change RUSH_FRAME into whatever padding you want, eg: perl -e 'printf("%06d", $ENV{RUSH_FRAME});' If you're trying to do shell substitution, I'm not sure, do you mean wild card expansions, like: cp /some/directory/foo.[0-9]*.tif /some/other/directory/ ..or do you really mean substitution, like sed/perl regex to turn e.g. foo.0001.tif into foo.0002.tif? > The alternative of course is to do a cp = > operation for each frame, but I was hoping to avoid the additional = > overhead. I wouldn't think the cp command would be much overhead, but I might be missing something. If you're trying to distribute the cp commands to a bunch of machines to parallelize the I/O, I'd think you'd want to do them either as single frames or in batches. Or better yet, batch several commands on the file server itself, eg: cp foo.0*.tif /some/other/dir & # copies 1000 frames (0000 - 0999) in bg cp foo.1*.tif /some/other/dir & # copies 1000 frames (1000 - 1999) in bg cp foo.2*.tif /some/other/dir & # etc.. ..which if the box has multiple procs and the network is idle, should go really fast. If you can follow up with more specifics, I can probably help you narrow down a specific technique. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1917 ***************************************************** *** ***************************************************** Subject: Batch Copy Script From: "Mr. Daniel Browne" Date: Tue, 15 Dec 2009 21:07:31 -0500 Hi Greg, I'm writing a basic submit script to perform a unix cp copy = operation on the farm. Is there a routine already available in Rush to = perform shell substitution of the digits in a frame range, or would I = have to do that myself? The alternative of course is to do a cp = operation for each frame, but I was hoping to avoid the additional = overhead. Thanks, -Dan ---------- Dan "Doc" Browne System Administrator Evil Eye Pictures d.list at evileyepictures.com Office: (415) 777-0666 *** ***************************************************** *** Message #1916 ***************************************************** *** ***************************************************** Subject: Seriss Corp. offices moved From: Greg Ercolano Date: Tue, 15 Dec 2009 13:41:44 -0500 Hi All, Just FYI, our offices have moved to a new location; the new address and phone info (as an image to avoid ascii harvesting): http://seriss.com/images/seriss-corp-address.png Old phones are automatically forwarding to the new office. *** ***************************************************** *** Message #1915 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Fri, 11 Dec 2009 23:57:14 -0500 Looks like Daniel has confirmed the fix, so I bumped the OSX version number to 102.42a9d (emphasis on "d") which has that fix, and modified the 102.42a9c release page so that downloading the mac version gets you the 'd' release binary with the fix. So if you want the 'd' fix release, go to the public upgrade page and download the Mac version. (ie. go to http://seriss.com/rush/ and click on the red upgrade monkeys) Greg Ercolano wrote: >> Greg Ercolano wrote: >>> Mr. Daniel Browne wrote: >>>> The machine stops responding after the last line, which corresponds to = >>>> the knew hosts file being acknowledged: >> OK, after a phone conversation, it seems on Snow Leopard machines >> when the rush/etc/hosts file reloads, the UDP listener is getting closed, >> and replaced with a unix domain socket. >> >> Since rush doesn't use unix domain sockets, this seems a bit odd. >> >> Also, I can't seem to replicate this on other platforms, only snow leopard. > > > Indeed, the same code running on Tiger and Leopard is fine, > so this seems to be operating system specific only to Snow Leopard. > > I think I've found the solution now -- running some tests. > > Dan, I'll have a new binary for you Tuesday to try. > > Seems to have to do with Snow Leopard's changes to hostname resolution > that use using unix domain sockets. *** ***************************************************** *** Message #1914 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Tue, 08 Dec 2009 01:39:08 -0500 > Greg Ercolano wrote: >> Mr. Daniel Browne wrote: >>> The machine stops responding after the last line, which corresponds to = >>> the knew hosts file being acknowledged: > > OK, after a phone conversation, it seems on Snow Leopard machines > when the rush/etc/hosts file reloads, the UDP listener is getting closed, > and replaced with a unix domain socket. > > Since rush doesn't use unix domain sockets, this seems a bit odd. > > Also, I can't seem to replicate this on other platforms, only snow leopard. Indeed, the same code running on Tiger and Leopard is fine, so this seems to be operating system specific only to Snow Leopard. I think I've found the solution now -- running some tests. Dan, I'll have a new binary for you Tuesday to try. Seems to have to do with Snow Leopard's changes to hostname resolution that use using unix domain sockets. *** ***************************************************** *** Message #1913 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Mon, 07 Dec 2009 19:31:21 -0500 Greg Ercolano wrote: > Mr. Daniel Browne wrote: >> The machine stops responding after the last line, which corresponds to = >> the knew hosts file being acknowledged: >> >>> 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts > > Hmm, we'll have to work offline on this; I can't seem to replicate here. > Will contact each other by phone. OK, after a phone conversation, it seems on Snow Leopard machines when the rush/etc/hosts file reloads, the UDP listener is getting closed, and replaced with a unix domain socket. Since rush doesn't use unix domain sockets, this seems a bit odd. Also, I can't seem to replicate this on other platforms, only snow leopard. Whatever the case, this is causing the daemon to become unresponsive to UDP transactions, but still responds to TCP (eg. 'rush -ping' still works). This can be seen pretty clearly with 'lsof' before and after: BEFORE: $ lsof | grep rushd rushd 1447 root cwd DIR 14,5 374 600231 /usr/local/rush/var rushd 1447 root txt REG 14,5 3086040 599807 /usr/local/rush/bin/rushd rushd 1447 root txt REG 14,5 1054960 24705 /usr/lib/dyld rushd 1447 root txt REG 14,5 199118848 526079 /private/var/db/dyld/dyld_shared_cache_i386 rushd 1447 root 0r CHR 3,2 0t0 303 /dev/null rushd 1447 root 1w REG 14,5 3169 600232 /usr/local/rush/var/rushd.log rushd 1447 root 2w REG 14,5 3169 600232 /usr/local/rush/var/rushd.log rushd 1447 root 3w REG 14,5 5 600323 /usr/local/rush/var/.rushd.LCK rushd 1447 root 4u IPv4 0x040f1ef8 0t0 TCP *:rushd (LISTEN) rushd 1447 root 5u IPv4 0x047cc9c8 0t0 UDP snow.erco.x:rushd ^^ ^^^^^^^^^^^^^^^^^^^^^ AFTER: lsof | grep rushd tail 1100 root 3r REG 14,5 3231 600232 /usr/local/rush/var/rushd.log rushd 1447 root cwd DIR 14,5 374 600231 /usr/local/rush/var rushd 1447 root txt REG 14,5 3086040 599807 /usr/local/rush/bin/rushd rushd 1447 root txt REG 14,5 1054960 24705 /usr/lib/dyld rushd 1447 root txt REG 14,5 199118848 526079 /private/var/db/dyld/dyld_shared_cache_i386 rushd 1447 root 0r CHR 3,2 0t0 303 /dev/null rushd 1447 root 1w REG 14,5 3231 600232 /usr/local/rush/var/rushd.log rushd 1447 root 2w REG 14,5 3231 600232 /usr/local/rush/var/rushd.log rushd 1447 root 3w REG 14,5 5 600323 /usr/local/rush/var/.rushd.LCK rushd 1447 root 4u IPv4 0x040f1ef8 0t0 TCP *:rushd (LISTEN) rushd 1447 root 5u unix 0x037c8910 0t0 ->0x03fbf1c0 ^^ ^^^^ ^^^^^^^^^^^^ Unix socket What the heck? I'll follow up when I've done more tracing to see what's causing that. *** ***************************************************** *** Message #1912 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Mon, 07 Dec 2009 18:45:25 -0500 Mr. Daniel Browne wrote: > The machine stops responding after the last line, which corresponds to = > the knew hosts file being acknowledged: > >> 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts Hmm, we'll have to work offline on this; I can't seem to replicate here. Will contact each other by phone. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1911 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: "Mr. Daniel Browne" Date: Mon, 07 Dec 2009 17:32:23 -0500 The machine stops responding after the last line, which corresponds to = the knew hosts file being acknowledged: > 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts On Dec 7, 2009, at 2:30 PM, Greg Ercolano wrote: [posted to rush.general] Hmm, at what point in the rushd.log did the daemon stop? Every START entry seems to have a matching EXIT entry. Is it possible the problem happened on a different day from today? (eg. if yesterday, paste the Orushd.log (O=3Dold)) It would appear there was a temporary problem with the machine's own = hostname (wayne) not being resolvable during startup, where it says: Local hostname wayne won't resolve (ping: cannot resolve wayne: = Unknown host): 10 sec retries The daemon won't start the rush daemon until 'ping' can resolve the = local machine's own hostname, to make sure hostname resolution is working before rush is = started. In pseudocode, basically: while ( ping `hostname` doesn't work ) { sleep 10 try again } Mr. Daniel Browne wrote: > Here you go (IP addresses removed): >=20 > 12/07,00:00:04 ROTATE rushd.log rotated. pid=3D3D203, 0/1 busy, = ONLINE > 12/07,00:00:04 ROTATE wayne RUSHD 102.42a9c PID=3D3D203 = Boot=3D3D12/04/09,14:07:03 > 12/07,09:47:26 SECURITY Daemon changed to Offline by root@wayne[], = Remark:offline by dbrowne via login script state (offline) by = root@wayne[] > 12/07,12:20:35 SECURITY Daemon changed to Online by root@wayne[], = Remark:online by dbrowne via logout script state (online) by = root@wayne[] > 12/07,12:20:36 SECURITY Daemon changed to Getoff by root@wayne[] = state (getoff) by root@wayne[] > 12/07,12:20:40 INIT.D Shutdown script '/usr/local/rush/etc/S99rush = stop' > 12/07,12:20:40 EXIT Shutdown script '/usr/local/rush/etc/S99rush = stop' > --------- > 12/07,12:21:34 INIT.D Startup script '/usr/local/rush/etc/S99rush = start' > 12/07,12:21:34 INIT.D Executing: ( /usr/local/rush/etc/S99rush = waitlocal && cd /usr/local/rush/var && /usr/libexec/StartupItemContext = /usr/local/rush/bin/rushd ) & > 12/07,12:21:34 INIT.D Local hostname wayne won't resolve (ping: = cannot resolve wayne: Unknown host): 10 sec retries > 12/07,12:21:45 LICENSE validated with server vegas > 12/07,12:21:45 LICENSE expires 03/15/2036 > 12/07,12:21:45 START wayne RUSHD 102.42a9c PID=3D3D189 = Boot=3D3D12/07/09,12:21:45 Online > 12/07,12:21:45 INFO TCP listening on port 696, service 'rushd', = sockfd=3D3D4 > 12/07,12:21:45 INFO UDP listening on port 696, service 'rushd', = sockfd=3D3D5 > 12/07,12:21:45 CHECKPOINT START: Loading = /usr/local/rush/var/jobs-checkpoint > 12/07,12:21:45 CHECKPOINT DONE > 12/07,12:21:51 SECURITY Daemon changed to Offline by root@wayne[], = Remark:offline by dbrowne via login script state (offline) by = root@wayne[172.21.7.9] > 12/07,13:57:54 PUSH /usr/local/rush/etc/hosts pushed from = ?@vegas:52291[] > 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts --=20 Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) ---------- Dan "Doc" Browne System Administrator Evil Eye Pictures d.list at evileyepictures.com Office: (415) 777-0666 *** ***************************************************** *** Message #1910 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Mon, 07 Dec 2009 17:30:20 -0500 Hmm, at what point in the rushd.log did the daemon stop? Every START entry seems to have a matching EXIT entry. Is it possible the problem happened on a different day from today? (eg. if yesterday, paste the Orushd.log (O=old)) It would appear there was a temporary problem with the machine's own hostname (wayne) not being resolvable during startup, where it says: Local hostname wayne won't resolve (ping: cannot resolve wayne: Unknown host): 10 sec retries The daemon won't start the rush daemon until 'ping' can resolve the local machine's own hostname, to make sure hostname resolution is working before rush is started. In pseudocode, basically: while ( ping `hostname` doesn't work ) { sleep 10 try again } Mr. Daniel Browne wrote: > Here you go (IP addresses removed): > > 12/07,00:00:04 ROTATE rushd.log rotated. pid=3D203, 0/1 busy, ONLINE > 12/07,00:00:04 ROTATE wayne RUSHD 102.42a9c PID=3D203 Boot=3D12/04/09,14:07:03 > 12/07,09:47:26 SECURITY Daemon changed to Offline by root@wayne[], Remark:offline by dbrowne via login script state (offline) by root@wayne[] > 12/07,12:20:35 SECURITY Daemon changed to Online by root@wayne[], Remark:online by dbrowne via logout script state (online) by root@wayne[] > 12/07,12:20:36 SECURITY Daemon changed to Getoff by root@wayne[] state (getoff) by root@wayne[] > 12/07,12:20:40 INIT.D Shutdown script '/usr/local/rush/etc/S99rush stop' > 12/07,12:20:40 EXIT Shutdown script '/usr/local/rush/etc/S99rush stop' > --------- > 12/07,12:21:34 INIT.D Startup script '/usr/local/rush/etc/S99rush start' > 12/07,12:21:34 INIT.D Executing: ( /usr/local/rush/etc/S99rush waitlocal && cd /usr/local/rush/var && /usr/libexec/StartupItemContext /usr/local/rush/bin/rushd ) & > 12/07,12:21:34 INIT.D Local hostname wayne won't resolve (ping: cannot resolve wayne: Unknown host): 10 sec retries > 12/07,12:21:45 LICENSE validated with server vegas > 12/07,12:21:45 LICENSE expires 03/15/2036 > 12/07,12:21:45 START wayne RUSHD 102.42a9c PID=3D189 Boot=3D12/07/09,12:21:45 Online > 12/07,12:21:45 INFO TCP listening on port 696, service 'rushd', sockfd=3D4 > 12/07,12:21:45 INFO UDP listening on port 696, service 'rushd', sockfd=3D5 > 12/07,12:21:45 CHECKPOINT START: Loading /usr/local/rush/var/jobs-checkpoint > 12/07,12:21:45 CHECKPOINT DONE > 12/07,12:21:51 SECURITY Daemon changed to Offline by root@wayne[], Remark:offline by dbrowne via login script state (offline) by root@wayne[172.21.7.9] > 12/07,13:57:54 PUSH /usr/local/rush/etc/hosts pushed from ?@vegas:52291[] > 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1909 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: "Mr. Daniel Browne" Date: Mon, 07 Dec 2009 17:01:26 -0500 Here you go (IP addresses removed): 12/07,00:00:04 ROTATE rushd.log rotated. pid=3D203, 0/1 busy, ONLINE 12/07,00:00:04 ROTATE wayne RUSHD 102.42a9c PID=3D203 = Boot=3D12/04/09,14:07:03 12/07,09:47:26 SECURITY Daemon changed to Offline by root@wayne[], = Remark:offline by dbrowne via login script state (offline) by = root@wayne[] 12/07,12:20:35 SECURITY Daemon changed to Online by root@wayne[], = Remark:online by dbrowne via logout script state (online) by = root@wayne[] 12/07,12:20:36 SECURITY Daemon changed to Getoff by root@wayne[] state = (getoff) by root@wayne[] 12/07,12:20:40 INIT.D Shutdown script '/usr/local/rush/etc/S99rush = stop' 12/07,12:20:40 EXIT Shutdown script '/usr/local/rush/etc/S99rush = stop' --------- 12/07,12:21:34 INIT.D Startup script '/usr/local/rush/etc/S99rush = start' 12/07,12:21:34 INIT.D Executing: ( /usr/local/rush/etc/S99rush = waitlocal && cd /usr/local/rush/var && /usr/libexec/StartupItemContext = /usr/local/rush/bin/rushd ) & 12/07,12:21:34 INIT.D Local hostname wayne won't resolve (ping: = cannot resolve wayne: Unknown host): 10 sec retries 12/07,12:21:45 LICENSE validated with server vegas 12/07,12:21:45 LICENSE expires 03/15/2036 12/07,12:21:45 START wayne RUSHD 102.42a9c PID=3D189 = Boot=3D12/07/09,12:21:45 Online 12/07,12:21:45 INFO TCP listening on port 696, service 'rushd', = sockfd=3D4 12/07,12:21:45 INFO UDP listening on port 696, service 'rushd', = sockfd=3D5 12/07,12:21:45 CHECKPOINT START: Loading = /usr/local/rush/var/jobs-checkpoint 12/07,12:21:45 CHECKPOINT DONE 12/07,12:21:51 SECURITY Daemon changed to Offline by root@wayne[], = Remark:offline by dbrowne via login script state (offline) by = root@wayne[172.21.7.9] 12/07,13:57:54 PUSH /usr/local/rush/etc/hosts pushed from = ?@vegas:52291[] 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts On Dec 7, 2009, at 12:52 PM, Greg Ercolano wrote: [posted to rush.general] Mr. Daniel Browne wrote: > [posted to rush.general] >=20 > Hi Greg, >=20 > The 102.42a9c update seems to have fixed most of our operation =3D= > problems with Rush on SL (we love the new GUI look too, by the way), = but =3D > there is one thing we've noticed; rush on SL machines seems to stop =3D > responding after pushing a hosts file. The failure happens even if the = =3D > file hasn't been changed. This requires you to restart the rush = daemon, =3D > much like how before this release we had to do whenever rush on an SL = =3D > machine stopped responding. Hmm, can you send me the contents of the rushd.log from that = machine where the daemon stopped responding? --=20 Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) ---------- Dan "Doc" Browne System Administrator Evil Eye Pictures d.list at evileyepictures.com Office: (415) 777-0666 *** ***************************************************** *** Message #1908 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Mon, 07 Dec 2009 15:52:47 -0500 Mr. Daniel Browne wrote: > [posted to rush.general] > > Hi Greg, > > The 102.42a9c update seems to have fixed most of our operation = > problems with Rush on SL (we love the new GUI look too, by the way), but = > there is one thing we've noticed; rush on SL machines seems to stop = > responding after pushing a hosts file. The failure happens even if the = > file hasn't been changed. This requires you to restart the rush daemon, = > much like how before this release we had to do whenever rush on an SL = > machine stopped responding. Hmm, can you send me the contents of the rushd.log from that machine where the daemon stopped responding? -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1907 ***************************************************** *** ***************************************************** Subject: Rush 102.42a9c and Snow Leopard From: "Mr. Daniel Browne" Date: Mon, 07 Dec 2009 15:03:23 -0500 Hi Greg, The 102.42a9c update seems to have fixed most of our operation = problems with Rush on SL (we love the new GUI look too, by the way), but = there is one thing we've noticed; rush on SL machines seems to stop = responding after pushing a hosts file. The failure happens even if the = file hasn't been changed. This requires you to restart the rush daemon, = much like how before this release we had to do whenever rush on an SL = machine stopped responding. Thanks, -Dan ---------- Dan "Doc" Browne System Administrator Evil Eye Pictures d.list at evileyepictures.com Office: (415) 777-0666 *** ***************************************************** *** Message #1906 ***************************************************** *** ***************************************************** Subject: OSX + After Effects CS4: orphaned processes From: Greg Ercolano Date: Wed, 14 Oct 2009 05:21:38 -0400 This problem is Mac OSX specific, and is happening with CS4. It might be happening with CS3 as well, I haven't checked. It has come to my attention that After Effects CS4's 'aerender' program is now invoking AfterFx in a way where AfterFx no longer remains part of the process hierarchy, making it immune to receiving kill signals from normal process group management. So when you requeue busy frames, or dump a running After Effects job, the After Effects process remains running. In the case of rush, the frames show in the Die state, and don't immediately transition to Que until the AfterFx process finishes rendering the frame it's working on. Other render queues have trouble with this as well. Even scripts that are interrupted with ^C can leave AfterFx running in the background, taking up CPU. Apparently when aerender starts AfterFx, the AfterFx process becomes parented to launchd, and is no longer parented to the aerender process. I've notified Adobe, and they're working on a solution. *** ***************************************************** *** Message #1905 ***************************************************** *** ***************************************************** Subject: Re: Multicore workstations and clever farm use From: Greg Ercolano Date: Thu, 01 Oct 2009 17:15:27 -0400 Greg Ercolano wrote: > [..] you can go into the Task Manager, right click on the > 'rushd.exe' process, and assign it an affinity for cpus #0 and #1. > > Then all renders rush starts will be forced to /only/ use those > two processors; the other two processors will be idle, and > available for interactive desktop use. I played around with this a bit yesterday, and it seems to work quite well for restricting processors, and this is something you all can do with the current release of rush you have. In fact, the above should be able to work for all render managing software, or any processes, including interactive renders. I decided to add an option into the current alpha release of Rush (that will go into beta very soon) where you can include a "processor affinity" setting in the Rush hosts file, eg: #Host Cpus Ram MinPri Criteria/Hostgroups Affinity #------------- ---- ----- ------ ------------------- -------- ta 4 100 0 +any,+erco,+linux,linux,linux6.0,intel,dante,+farm - geneva 1 100 0 +any,+w2k,+work affinity=1 meade 8 100 0 +any,+erco,+linux,linux - superior 2 100 0 +any,+erco,+winxp,winxp affinity=3 [..] ..the new 'Affinity' field at the far right being the new option. Here it shows host 'geneva' that has 2 physical processors, but rush is set up to start only one instance of a render, and will restrict rendering only to cpu#0 (due to affinity=1), leaving cpu#1 available for interactive use. Also, 'superior' which has 4 physical processors, but will only start two instances of renders, and those renders will be restricted to only using physical cpus #0 and #1 (due to affinity=3). This way the new release of rush will be able to let the admin force it to only use certain physical processors. In my tests with the above alpha release, the task manager showed multi-threaded renders started by rushd were indeed restricted precisely to the cpus the 'affinity=' setting defined. However, practically, I'm not sure how useful this is. Despite the restriction leaving cpus completely idle, while renders were running, I still could "feel" slowness from the machine during interactive use, even though the 'render' was only doing a sin() computation in a tight loop, not using any I/O, ram, or networking. So even when you get the control you want, it may not get you the results you want. So forcing cpus to be idle won't necessarily get you responsiveness. It probably depends on the mother board I/O config and physical cpus (cores vs cpus) configurations. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1904 ***************************************************** *** ***************************************************** Subject: Re: unable to grab license over site-site vpn From: Greg Ercolano Date: Wed, 30 Sep 2009 19:58:40 -0400 Followup: We hooked up offline; think we found the problem was a hostname lookup issue. Apparently a notebook was used in the test which lived at site #1 and had a site#1 hostname/IP. When moved to site #2, the site#2 IP was assigned, but the hostname stayed the same. This caused the LA office license server to fail the license checkout because the hostname in the license checkout no longer matched the IP address. Michael's going to try a follow up making sure the hostname/IP match up at site #2. *** ***************************************************** *** Message #1903 ***************************************************** *** ***************************************************** Subject: Re: unable to grab license over site-site vpn From: Greg Ercolano Date: Wed, 30 Sep 2009 16:54:23 -0400 Hi Michael, Just tried calling you in NY, but I guess you already left. Was hoping I could help you out by phone.. were you able to get it to work, or would you like me to talk to one of the guys still at the NYC facility? BTW, I know the names of good freelance guys in NYC who are familiar with Rush and film/video systems administration. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1902 ***************************************************** *** ***************************************************** Subject: Re: unable to grab license over site-site vpn From: Greg Ercolano Date: Wed, 30 Sep 2009 14:36:18 -0400 Michael Oliver wrote: > I have two networks connected via a site to site vpn. The two sites are > on different private subnets but can both see each other. Other > services work perfectly (ie. DNS (forward and reverse), http, nfs, ldap, > etc...) However although computers on Site #2 can rush -ping computers > on Site#1, computers on site #2 cannot retrieve a license. Here is the > error I'm getting: > > 09/30,13:02:00 LICENSE Read from ns2[10.1.1.1] failed: read error: > BinaryRead(8192): recv() 0 bytes from ns2[10.1.1.1] > > 09/30,13:02:00 LICENSE no servers could validate license (30 sec retries) Is there a NAT'ing firewall in the mix? If so, that could be the problem; Rush looks at the IP packets to determine the return address in order to make replies, so if there's a NAT'ing firewall rewriting the return address to be the firewall's own, that could be the issue. Is 10.1.1.1 the correct IP address for ns2? Also, what are the hostnames and IP addresses of the machines at either end (license server and client), and can you show the 'rush -lah' report from both of these machines. If you prefer, you may want to reply to me via private email with this info if you don't want it to be public. Definitely you should be able to make this work, as there are several customers doing this (VPN WAN setups sharing licenses). -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1901 ***************************************************** *** ***************************************************** Subject: unable to grab license over site-site vpn From: Michael Oliver Date: Wed, 30 Sep 2009 13:03:50 -0400 I have two networks connected via a site to site vpn. The two sites are on different private subnets but can both see each other. Other services work perfectly (ie. DNS (forward and reverse), http, nfs, ldap, etc...) However although computers on Site #2 can rush -ping computers on Site#1, computers on site #2 cannot retrieve a license. Here is the error I'm getting: 09/30,13:02:00 LICENSE Read from ns2[10.1.1.1] failed: read error: BinaryRead(8192): recv() 0 bytes from ns2[10.1.1.1] 09/30,13:02:00 LICENSE no servers could validate license (30 sec retries) Any ideas? *** ***************************************************** *** Message #1900 ***************************************************** *** ***************************************************** Subject: Re: Multicore workstations and clever farm use From: Greg Ercolano Date: Wed, 30 Sep 2009 10:23:25 -0400 BTW, I added some screenshots below that show how assigning affinity really does force renders to use specific processors. See "SCREENSHOTS" below. > PROCESSOR AFFINITY > ------------------ > [..] > > Under Windows there's "Processor Affinity" which lets one ensure > a program only uses eg. 1 processor. You can also force a program/render to use only 2 processors, or even 4. And, you can specify precisely /which/ processors to use. "Processor Affinity" is an attribute of the process, and is also passed down to child processes. So an affinity assigned to a process, it will affect all child processes it creates. This means if you tell rushd to have an affinity, then all renders rushd starts will also have that same affinity. So say you have a 4 processor machine, and you want to force rush jobs on that machine to only use physical processors 0 and 1. This would ensure processors 2 and 3 would be completely available for interactive use. To do this, you can go into the Task Manager, right click on the 'rushd.exe' process, and assign it an affinity for cpus #0 and #1. Then all renders rush starts will be forced to /only/ use those two processors; the other two processors will be idle, and available for interactive desktop use. CAVEAT: Keep in mind that if the render makes heavy use of the hard disk, ram, or network, your "interactive use" may suffer at the limitations of those resources. So in other words, you may still 'feel' slowness if your interactive use involves any of those resources. Remember that a program's responsiveness is not /only/ due to cpu availability. I/O bandwidth is also at play. > One way to do this is with > the Windows 'start' command, eg. "start /AFFINITY ", > where indicates which processors to lock a command to.. The DOS 'START' command is nice if you want the render script to control processor use. Note that the START command's 'AFFINITY' flag is only available on machines with multiple processors. SCREENSHOTS ----------- Here are some screenshots that show how affinity really forces a process to use the processors you specify, no matter how many threads that process starts. In this case I'm running a program called 'createthread' which expects two parameters; a number of threads to start, and a number of seconds to run. In the following three screenshots, I'm running 'createthread 2 60' which starts two threads running for 60 seconds. In these 3 examples, I'm changing /only/ the affinity. Each screenshot includes the Task Manager's cpu graph, so you can see how the program makes use of the processors. Here are the three examples: 1) No affinity (use any cpus): http://seriss.com/rush/misc-docs/affinity-windows/affinity-none-threads-2.png 2) Affinity of 1 (use cpu#1): http://seriss.com/rush/misc-docs/affinity-windows/affinity-1-threads-2.png 3) Affinity of 2 (use cpu#2) http://seriss.com/rush/misc-docs/affinity-windows/affinity-2-threads-2.png Note that for "no affinity" both cpus are pegged 100%, total cpu usage is 100%. For "affinity 1", only cpu #1 is pegged, total cpu usage is only 50%. For "affinity 2", only cpu #2 is pegged, total cpu usage is only 50%. This makes it clear we can control precisely which processors get assigned to the process, and that regardless of the number of threads the process starts, only the processors with and affinity assigned are actually used. > for instance: > > START /AFFINITY 3 shake -exec /path/to/your.shk -t 1-69 > > ..which will 'lock' the shake process to cpus #0 and #1. And I should add to prevent START from opening another window, you can use: START /AFFINITY 3 /B /WAIT shake -exec /path/to/your.shk -t 1-69 -------- CAVEATS: one general problem with 'START'; it has a bug where it does not pass the exit code of the process back to the shell. The exit code returned is always 0, even if the process (in this case, shake) returns a non-zero exit code. So if you tried to use this in a script, it would not be able to tell if shake failed or succeeded. Kinda annoying. > The /AFFINITY value is a "bit field" which breaks down this way: > > Affinity > Value Cpu# Binary Value > -------- ---- ------------ > 1 0 0001 > 2 1 0010 > 3 0,1 0011 > 4 2 0100 > 5 0,2 0101 > 6 1,2 0110 > 7 0,1,2 0111 > : > 16 3 1000 > 17 3,0 1001 > : > etc. Correction -- a few mistakes in the above table the entries for 16 and 17 are wrong in two ways: First, my binary digits are wrong, and second, apparently the /AFFINITY parameter's value is *hex*. So a corrected table would be: Affinity Value Cpu# Binary Value -------- ---- ------------ 1 0 00001 2 1 00010 3 0,1 00011 4 2 00100 5 0,2 00101 6 1,2 00110 7 0,1,2 00111 8 3 01000 : : : f 0,1,2,3 01111 10 4 10000 11 4,0 10001 : etc. So if you have an 8 processor machine, and want a render to only use the first 4 processors (0,1,2,3), then use an affinity of 'f'. (Hex counts 0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f,10,11,12..) *** ***************************************************** *** Message #1899 ***************************************************** *** ***************************************************** Subject: Re: Multicore workstations and clever farm use From: Greg Ercolano Date: Tue, 29 Sep 2009 22:21:07 -0400 Abraham Schneider wrote: > Sometimes it would be really nice to use 4 of the cores to work in the > GUI and the other 4 cores to render in with rush. 'rush -reserve' is made for this purpose: http://www.seriss.com/rush-current/rush/rush-command-line-options.html#-reserve USING RESERVE JOBS ------------------ The CPUS number in the rush/etc/hosts file is really to be thought of as the number of instances of the renderer you want to run, so if you have '4' in the rush/etc/hosts file, and want to reserve 2 for GUI use, you could say: rush -reserve localhost=2@200 ..which says reserve 2 of rush's 'cpus' at 200 priority on the local machine. This means no job under 200 priority can use those two procs, even if its a 'killer' job. If you want to make it so no job can run, use 999. 'rush -reserve' creates a 'dummy sleep job' that holds the processors to prevent other renders from using them. Just dump the job to get rid of the reservation. PROCESSOR AFFINITY ------------------ However, if you've got multithreaded jobs and are specifying fixed numbers on the command line (eg. "shake -cpus 4") then threads might still leak into using processors you want "reserved". In such cases it can get tricky if you're trying to split hairs. The desire here is to really lock physical processors to a particular user. eg. if 'fred' is logged in, he wants to make sure that of the 8 processors on his machine, 4 can ONLY be used by him, so that even if a multithreaded shake job jumps on the machine, there'd be no chance that his 4 procs get used by the renders, no matter how many threads are running. Under Windows there's "Processor Affinity" which lets one ensure a program only uses eg. 1 processor. One way to do this is with the Windows 'start' command, eg. "start /AFFINITY ", where indicates which processors to lock a command to, for instance: START /AFFINITY 3 shake -exec /path/to/your.shk -t 1-69 ..which will 'lock' the shake process to cpus #0 and #1. The /AFFINITY value is a "bit field" which breaks down this way: Affinity Value Cpu# Binary Value -------- ---- ------------ 1 0 0001 2 1 0010 3 0,1 0011 4 2 0100 5 0,2 0101 6 1,2 0110 7 0,1,2 0111 : 16 3 1000 17 3,0 1001 : etc. There's a chart here: http://technet.microsoft.com/en-us/library/cc778499%28WS.10%29.aspx There are some other techniques you can do too. To simplify things, usually all people want to do is use their own machine for rendering, and not allow others to use it. WORKSTATION SPECIFIC RENDERING ------------------------------ To do this, a common technique is to use 'rush -reserve' to reserve the entire machine at some high priority, say @899. Then when you want to render on your machine, submit a job that asks for your machine (eg. 'tahoe') at a priority of 900k, eg: Cpus: tahoe=2@900k ..and setting e.g. the -cpus for shake to the number of threads you want, so that you don't use the entire machine. This will let your render bump the reservation job so that it can run, and when the render's done, the reserve job will maintain a hold on the processor again. If you're trying to set things up so that jobs can run on the farm AND on your workstation, but on the workstation, jobs never use more threads than you want, you'd have to modify the logic of the submit scripts to clamp eg. shake's "-cpus #" so that if the render is running on that machine, the -cpus number gets limited down to the value you want. Also, you can use MINPRI column of the hosts file is something you can use as well to put a 'wall' on the jobs the workstations accept. The 'MINPRI' value sets a minimum priority a job needs to render on that machine. This way the machine's processors will remain idle unless a job asks for this machine with a high enough priority. You can establish policies that allow users to only submit to their own machines with a priority sufficient to render on their own workstation, while keeping other renders away. If you need more specifics on any of these, let me know. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1898 ***************************************************** *** ***************************************************** Subject: Multicore workstations and clever farm use From: "Abraham Schneider" Date: Tue, 29 Sep 2009 10:54:03 -0400 Hi there! With all the 'new' multicore workstations we have now with 4, 8 or 16=20 cores, I'm thinking about using them in the renderfarm as well. But I=20 can't find a good solution to do this at the moment. Problem is: I defined all the machines in the hosts file to be a 'cluster' of 4=20 cores. So a machine with 4 real cores is listed as 1 Cpu in the hosts=20 file, a machine with 8 cores is listed as 2 Cpus. And in Shake I start=20 my jobs with '-cpus 4' to use 4 cores to render on one frame. That way of working is fine when I don't use the workstation at all and=20 want to use it completely in the farm. But for machines with 8 or more=20 cores there would be another nice scenario: Sometimes it would be really nice to use 4 of the cores to work in the=20 GUI and the other 4 cores to render in with rush. Is there any clever way to handle this in rush? At the moment the only=20 way I see is to edit the hosts file and change the 'cpus' value. But I=20 have to do this every time an operator wants to change from 'all cores=20 to the farm' to 'only some cores to the farm' and back. Really perfect would be an advanced 'onrush' where I have more than two=20 options: not online, partially online, fully online. Or something = similar. How do you all handle this? Thanks, Abraham -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Dr. Martin Prillmann; Thomas Till *** ***************************************************** *** Message #1897 ***************************************************** *** ***************************************************** Subject: Re: Snow Leopard -- Fixes for Rush 102.42a9b and older From: Greg Ercolano Date: Fri, 25 Sep 2009 05:03:45 -0400 > Once we removed .local from the search suffix all is well again. Sounds like you fixed it. Yes, I recommend avoiding the .local domain name suffix because that enables all that ZeroConf/Bonjours broadcast-based discovery stuff, including multicast DNS, which creates unreliable behavior that messes up network stability. ie: http://en.wikipedia.org/wiki/Zeroconf If you need zeroconf based stuff on your machines also running rush, you may be able to specify it without causing trouble IF you specify your internal domain first, and then .local after it. This way (maybe) regular DNS will be interrogated first, and then falls back to Bonjours only if that fails. Dylan Penhale wrote: > Yes I thought DNS at first but we thought we had it sorted as we where > using the same settings as other non snow leopard machines. I assume > DNS is used to reverse lookup the IP addresses of the hostnames but > that this isn't cached anywhere. Both OSX and Windows traditionally have DNS caching turned on as a default. "lookupd' used to be responsible for hostname caching in OSX, but now I think it's DirectoryServices(8) since lookupd was dropped. According to the docs, dscacheutil can be used to display the DNS cache (supposedly), ie. as root: dscacheutil -cachedump -entries Host However, when I do this on my snow leopard box, I get zippo. Even after I do things that /should/ trigger DNS cache hits, like pinging hosts by name, or running 'rush -lah'. Because of this empty output, I'm not entirely sure it IS caching namelookups by default in Snow Leopard. Just a casual observation though.. I haven't dug into SL deeply yet. > It turns out that snow leopard treats .local in a different way to > previous apple operating systems, as far as I can tell. The bad behavior with .local has always been in there, it just creeps in whenever you start with a fresh machine, and forget to configure all the DNS stuff statically. When Bonjours creeps in, you can get flakey name lookups due to the broadcast based discovery stuff. I refer to this in the prerequisite docs as something to avoid: http://www.seriss.com/rush-current/rush/rush-admin-faq.html#ADMINFAQ-PREREQ (Search for hostname.local in that text) > Usually this > wouldn't be a problem but as some of our domain uses .local this > appears to clash with the apple .local. which I believe is used for > zeroconf type broadcast traffic. Yes, correct. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1896 ***************************************************** *** ***************************************************** Subject: Re: Snow Leopard -- Fixes for Rush 102.42a9b and older From: Dylan Penhale Date: Fri, 25 Sep 2009 03:34:17 -0400 Yes I thought DNS at first but we thought we had it sorted as we where using the same settings as other non snow leopard machines. I assume DNS is used to reverse lookup the IP addresses of the hostnames but that this isn't cached anywhere. It turns out that snow leopard treats .local in a different way to previous apple operating systems, as far as I can tell. Usually this wouldn't be a problem but as some of our domain uses .local this appears to clash with the apple .local. which I believe is used for zeroconf type broadcast traffic. Once we removed .local from the search suffix all is well again. Thanks On 25/09/2009, at 5:10 PM, Greg Ercolano wrote: > [posted to rush.general] > > Dylan Penhale wrote: >> We are experiencing slow performance issuing a rush -lah on snow >> leopard. > >> Will the 102.42a9c maintenance release address this? > > I haven't seen a slowness problem on Snow Leopard > with 'rush -lah'. > > Slowness with 'rush -lah' usually means hostname lookups > aren't working correctly because it does a hostname<->IP > lookup for each hostname in the rush/etc/hosts file, in > order to show the IP addresses for each host. > > A very slow output of 'rush -lah' usually means the primary > DNS server is being unresponsive. > > Check your DNS settings, to make sure the primary DNS server > is responsive, and the search domain is set to your internal > domain. > > Also; are there any hosts coming up with ???'s for IP's? > If so, that's surely the cause, as it means DNS is failing > the lookup for those hosts, and that can slow that report > down quite a bit. > *** ***************************************************** *** Message #1895 ***************************************************** *** ***************************************************** Subject: Re: Snow Leopard -- Fixes for Rush 102.42a9b and older From: Greg Ercolano Date: Fri, 25 Sep 2009 03:10:55 -0400 Dylan Penhale wrote: > We are experiencing slow performance issuing a rush -lah on snow > leopard. > Will the 102.42a9c maintenance release address this? I haven't seen a slowness problem on Snow Leopard with 'rush -lah'. Slowness with 'rush -lah' usually means hostname lookups aren't working correctly because it does a hostname<->IP lookup for each hostname in the rush/etc/hosts file, in order to show the IP addresses for each host. A very slow output of 'rush -lah' usually means the primary DNS server is being unresponsive. Check your DNS settings, to make sure the primary DNS server is responsive, and the search domain is set to your internal domain. Also; are there any hosts coming up with ???'s for IP's? If so, that's surely the cause, as it means DNS is failing the lookup for those hosts, and that can slow that report down quite a bit. *** ***************************************************** *** Message #1894 ***************************************************** *** ***************************************************** Subject: Re: Snow Leopard -- Fixes for Rush 102.42a9b and older From: Dylan Penhale Date: Fri, 25 Sep 2009 01:19:31 -0400 Hi Greg We are experiencing slow performance issuing a rush -lah on snow leopard. Will the 102.42a9c maintenance release address this? Dylan Penhale Fuel VFX On 10/09/2009, at 8:45 AM, Greg Ercolano wrote: > [posted to rush.general] > > RUSH + SNOW LEOPARD > > There will be a new maintenance release of Rush that addresses > Snow Leopard: Rush 102.42a9c, which will beta in about a week. > (Still doing internal testing) > > It will have other things as well; watch this space for details. > > * * * > > If you have any questions, let me know. *** ***************************************************** *** Message #1892 ***************************************************** *** ***************************************************** Subject: Next release: Rush 102.42a9c beta testing From: Greg Ercolano Date: Mon, 21 Sep 2009 20:25:15 -0400 Hi All, I'm collecting names for those folks wanting to beta test the next release. I'm looking for no more than 5 companies to beta test. We're still doing alpha testing internally, but we're closing in on a beta release. Alpha revealed some things I'm fixing, and unearthed some good feature requests which I want to fit into the release as soon as those all go back through alpha. You'll find irush has several features in this release, including easy ways to view the frame logs in a text editor, popup contextual menus for the Jobs and Frames reports, easy ways to view job start/done/dump command logs interactively, as well as some other features. [..] Please contact me via email directly if you're interested in testing: erco@(email surpressed) *** ***************************************************** *** Message #1891 ***************************************************** *** ***************************************************** Subject: Re: OSX hidden process using 50% sys. From: Dylan Penhale Date: Fri, 18 Sep 2009 02:54:39 -0400 > From: Greg Ercolano > Reply-To: > Date: 17 Sep 2009 07:56:27 -0000 > To: > Subject: Re: OSX hidden process using 50% sys. > > [posted to rush.general] > > Dylan Penhale wrote: >>>> We are unable to read the log file from the problem render node, >>>> but the file is readable from all other hosts. >>> When you try to read the log, what happens, and with what >>> command/technique are you trying to look at the file? >>> (rush -log, more, type, cat, text editor, etc) >> >> Cat, more, tail - all return no output. >> >> As I say, the log file is ok because we can view it from other machines. I >> am curios why this box is even trying to write these files as this node >> isn't even supposed to be rendering this frame. I think this might be the >> clue to the issue. > > This sounds messed up at the file system level; > I'd suspect the network file system as the problem, > probably at the client side. > > Hopefully other folks can chime in here if they've seen > this too. > > What kind of machine is the file server at the other end > of the NFS mount? Possibly there's an NFS incompatibility > there.. only thing I can think of other than the client OS. > > I haven't heard of such a problem in OSX's nfs; in my > experience NFS has been pretty much problem free mounting > the linux NFS server here. > > BTW, you mention you're unable to umount the file server, > is this true even after you kill the Render process, and > lsof verifies there are no other mounts? Or is the Render > process unkillable because it's 'hung accessing the mount', > which would all the more point to an NFS client fault. > > Did you try 'umount -f'? Yes, this doesn't work though, it just hangs the shell. Certainly looks like a kernel error. Once in this state you can't cancel the umount command, you have to create a new ssh session or power cycle the machine. The mount options are "intr,bg,resvport,vers=3" which I would think would be pretty safe. The log server in this case is an OSX server (it's pretty much only used for rush logs). The server doesn't give anything away in it's logs. > >>>> We think that the kernel of this render node is stuck trying to access >>>> this file, which in turn is causing the high cpu sys load. > >>> Yes, most likely, if the cpu is not attributed to a process. > >> The render log looks the same as the others, nothing to suggest an error. >> Log size is 46K. > > Try yanking the network cable to see if it affects the 50% cpu use. > My guess is it won't, which means the kernel is spinning doing > 'something bad'.. hard to say what. > > Is the whole mount in a bad state (ie can you view other files OK?), > or is it just this file and all else works fine? Yes we can view other files, it's just this one file we have problems with. > > All the behavior sounds quite odd from an OS point of view. > I'd say the client OS is at fault here, with only a slim > possibility the NFS server might be triggering it. > > Once rush starts the Render process, its up to the OS and Render > to do its thing; rush has no part of the rendering process once > it starts until the process exits; then rush steps in and returns > the exit code. > > I think if you want to move forward, get out the nfs debugging > tools; tcpdump, possibly even ktrace (or whatever it's called > on Leopard). If you suspect maybe the server is at fault, > contact the vendor if you're on support. > >> RUSHD 102.42a9 > > That sounds fine. > > That release is a little over a year old (June 08), > and there's only been 2 minor maintenance releases since then.. > but nothing you'd need. > > -- > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) *** ***************************************************** *** Message #1890 ***************************************************** *** ***************************************************** Subject: Re: OSX hidden process using 50% sys. From: Greg Ercolano Date: Thu, 17 Sep 2009 03:56:27 -0400 Dylan Penhale wrote: >>> We are unable to read the log file from the problem render node, >>> but the file is readable from all other hosts. >> When you try to read the log, what happens, and with what >> command/technique are you trying to look at the file? >> (rush -log, more, type, cat, text editor, etc) > > Cat, more, tail - all return no output. > > As I say, the log file is ok because we can view it from other machines. I > am curios why this box is even trying to write these files as this node > isn't even supposed to be rendering this frame. I think this might be the > clue to the issue. This sounds messed up at the file system level; I'd suspect the network file system as the problem, probably at the client side. Hopefully other folks can chime in here if they've seen this too. What kind of machine is the file server at the other end of the NFS mount? Possibly there's an NFS incompatibility there.. only thing I can think of other than the client OS. I haven't heard of such a problem in OSX's nfs; in my experience NFS has been pretty much problem free mounting the linux NFS server here. BTW, you mention you're unable to umount the file server, is this true even after you kill the Render process, and lsof verifies there are no other mounts? Or is the Render process unkillable because it's 'hung accessing the mount', which would all the more point to an NFS client fault. Did you try 'umount -f'? >>> We think that the kernel of this render node is stuck trying to access >>> this file, which in turn is causing the high cpu sys load. >> Yes, most likely, if the cpu is not attributed to a process. > The render log looks the same as the others, nothing to suggest an error. > Log size is 46K. Try yanking the network cable to see if it affects the 50% cpu use. My guess is it won't, which means the kernel is spinning doing 'something bad'.. hard to say what. Is the whole mount in a bad state (ie can you view other files OK?), or is it just this file and all else works fine? All the behavior sounds quite odd from an OS point of view. I'd say the client OS is at fault here, with only a slim possibility the NFS server might be triggering it. Once rush starts the Render process, its up to the OS and Render to do its thing; rush has no part of the rendering process once it starts until the process exits; then rush steps in and returns the exit code. I think if you want to move forward, get out the nfs debugging tools; tcpdump, possibly even ktrace (or whatever it's called on Leopard). If you suspect maybe the server is at fault, contact the vendor if you're on support. > RUSHD 102.42a9 That sounds fine. That release is a little over a year old (June 08), and there's only been 2 minor maintenance releases since then.. but nothing you'd need. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1889 ***************************************************** *** ***************************************************** Subject: Re: OSX hidden process using 50% sys. From: Dylan Penhale Date: Thu, 17 Sep 2009 02:10:51 -0400 > From: Greg Ercolano > Reply-To: > Date: 17 Sep 2009 04:31:30 -0000 > To: > Subject: Re: OSX hidden process using 50% sys. > > [posted to rush.general] > > Dylan Penhale wrote: >> We are submitting a Maya mentalray render from a PC to a bunch of PC=92= > s >> and Macs. The job server is running XP. On one Mac render node, >> sometimes more than one, we seemingly randomly see the CPU pegged at 50= > % >> system. Top does not show what process is using that 50%, it appears to >> be hidden. > > The kernel can take CPU of its own to process IO, and that > won't show up associated with any process. For instance, in > rushtop this would show up in red. Yes, rushtop shows lots of red :) > >> A little digging reveals that there are two processes that >> appear to be =93holding=94 log files open on the rush log server. > >> Interestingly these commands are both trying to access the same log fil= > e >> and they are NOT the log file that the machine has rendered, nor been >> assigned to render (from what we can tell). > >> We are unable to read the log file from the problem render node, but th= > e >> file is readable from all other hosts. > > When you try to read the log, what happens, and with what > command/technique are you trying to look at the file? > (rush -log, more, type, cat, text editor, etc) Cat, more, tail - all return no output. As I say, the log file is ok because we can view it from other machines. I am curios why this box is even trying to write these files as this node isn't even supposed to be rendering this frame. I think this might be the clue to the issue. > >> We think that the kernel of this render node is stuck trying to access >> this file, which in turn is causing the high cpu sys load. > > Yes, most likely, if the cpu is not attributed to a process. > >> When we lsof we can see that the command that is accessing the log file >> is Render. >> =20 >> Render 19841 netrender 1w REG 26,10 47549 >> 37261653 >> /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_mat= > teSealA.log/0008 >> Render 19841 netrender 2w REG 26,10 47549 >> 37261653 >> /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_mat= > teSealA.log/0008 > > The process ids are the same (19841), so it's one process. > > There are two entries in lsof, one for the stdout file descriptor (1w) > and one for the stderr file descriptor (2w) which are both 'write only' = > (w). > > This is normal behavior during rendering, because the processes > are generating output that is being written to the file server. > > You might check to see what's going on in the log, as maybe it's generat= > ing > a large amount of output due to a high verbosity setting. > The render log looks the same as the others, nothing to suggest an error. Log size is 46K. > Often you can mitigate the cpu/network overhead of render's verbose outp= > ut > by setting the 'maxlogsize' to some very high number (like 10000000, ie.= > 10M) > so that the render output is piped through 'logtrim', which will line bu= > ffer > the render's output so that verbose output is buffered. > > BTW, what version of rush are you running? > I don't think it matters in this case, as the io has to do with the rend= > er > and likely its verbosity setting. But it may help to know on this end. RUSHD 102.42a9 > > > --=20 > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) *** ***************************************************** *** Message #1888 ***************************************************** *** ***************************************************** Subject: Re: OSX hidden process using 50% sys. From: Greg Ercolano Date: Thu, 17 Sep 2009 00:31:30 -0400 Dylan Penhale wrote: > We are submitting a Maya mentalray render from a PC to a bunch of PC=92= s > and Macs. The job server is running XP. On one Mac render node, > sometimes more than one, we seemingly randomly see the CPU pegged at 50= % > system. Top does not show what process is using that 50%, it appears to > be hidden. The kernel can take CPU of its own to process IO, and that won't show up associated with any process. For instance, in rushtop this would show up in red. > A little digging reveals that there are two processes that > appear to be =93holding=94 log files open on the rush log server. > Interestingly these commands are both trying to access the same log fil= e > and they are NOT the log file that the machine has rendered, nor been > assigned to render (from what we can tell). > We are unable to read the log file from the problem render node, but th= e > file is readable from all other hosts. When you try to read the log, what happens, and with what command/technique are you trying to look at the file? (rush -log, more, type, cat, text editor, etc) > We think that the kernel of this render node is stuck trying to access > this file, which in turn is causing the high cpu sys load. Yes, most likely, if the cpu is not attributed to a process. > When we lsof we can see that the command that is accessing the log file > is Render. >=20 > Render 19841 netrender 1w REG 26,10 47549 > 37261653 > /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_mat= teSealA.log/0008 > Render 19841 netrender 2w REG 26,10 47549 > 37261653 > /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_mat= teSealA.log/0008 The process ids are the same (19841), so it's one process. There are two entries in lsof, one for the stdout file descriptor (1w) and one for the stderr file descriptor (2w) which are both 'write only' = (w). This is normal behavior during rendering, because the processes are generating output that is being written to the file server. You might check to see what's going on in the log, as maybe it's generat= ing a large amount of output due to a high verbosity setting. Often you can mitigate the cpu/network overhead of render's verbose outp= ut by setting the 'maxlogsize' to some very high number (like 10000000, ie.= 10M) so that the render output is piped through 'logtrim', which will line bu= ffer the render's output so that verbose output is buffered. BTW, what version of rush are you running? I don't think it matters in this case, as the io has to do with the rend= er and likely its verbosity setting. But it may help to know on this end. --=20 Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1887 ***************************************************** *** ***************************************************** Subject: OSX hidden process using 50% sys. From: Dylan Penhale Date: Wed, 16 Sep 2009 23:04:54 -0400 --_000_C6D7E47110986dylanfuelvfxcom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi We have a tricky little problem here, I wonder if you have seen anything li= ke it. We are submitting a Maya mentalray render from a PC to a bunch of PC's and = Macs. The job server is running XP. On one Mac render node, sometimes more = than one, we seemingly randomly see the CPU pegged at 50% system. Top does = not show what process is using that 50%, it appears to be hidden. A little = digging reveals that there are two processes that appear to be "holding" lo= g files open on the rush log server. Interestingly these commands are both = trying to access the same log file and they are NOT the log file that the m= achine has rendered, nor been assigned to render (from what we can tell). We are unable to read the log file from the problem render node, but the fi= le is readable from all other hosts. We think that the kernel of this render node is stuck trying to access this= file, which in turn is causing the high cpu sys load. When we lsof we can see that the command that is accessing the log file is = Render. Render 19841 netrender 1w REG 26,10 47549 37261653 = /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_matteSe= alA.log/0008 Render 19841 netrender 2w REG 26,10 47549 37261653 = /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_matteSe= alA.log/0008 Note: This not did not try to render frame 0008 from what we can tell. Should Render be writing the log file directly, or should the STDERR from R= ender be passed to Rush to do the writing? The PID's listed do not show up in the process list (top or ps) so we assum= e the kernel is experiencing some sort of bug. We also thought that perhaps this render node was assigned to render this f= rame, and at the last minute informed that another node was already doing i= t so told to abort it's render. It's hard to tell from the cpu.acct file wh= at is happening as I think this only gets updated at the end of each succes= sful frame. We are not able to umount the NFS share that the logs are written to, which= again suggests the logs are somehow the cause. In fact the only way to get= the CPU to behave normally is to restart the box, which can only be done b= y physically holding the power button in. In this case the render node is running OSX 10.5.4, and is an Intel Core 2 = Duo. Any ideas? DYLAN PENHALE IT MANAGER FUEL 65 KING STREET NEWTOWN SYDNEY NSW 2042 AUSTRALIA T. 61 2 9557 7799 F. 61 2 9557 7882 M. 0424 655 320 WWW.FUELVFX.COM --_000_C6D7E47110986dylanfuelvfxcom_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable OSX hidden process using 50% sys. Hi

We have a tricky little problem here, I wonder if you have seen anything li= ke it.  

We are submitting a Maya mentalray render from a PC to a bunch of PC’= s and Macs. The job server is running XP. On one Mac render node, sometimes= more than one, we seemingly randomly see the CPU pegged at 50% system. Top= does not show what process is using that 50%, it appears to be hidden. A l= ittle digging reveals that there are two processes that appear to be “= ;holding” log files open on the rush log server. Interestingly these = commands are both trying to access the same log file and they are NOT the l= og file that the machine has rendered, nor been assigned to render (from wh= at we can tell).

We are unable to read the log file from the problem render node, but the fi= le is readable from all other hosts.
We think that the kernel of this render node is stuck trying to access this= file, which in turn is causing the high cpu sys load.

When we lsof we can see that the command that is accessing the log file is = Render.

Render    19841      netrender &nbs= p;  1w      REG     = 26,10     47549 37261653 /Volumes/atlantic/rushlogs/3d/= tjn_se_0210_c004rs_a033as_l014hh_seal_matteSealA.log/0008
Render    19841      netrender &nbs= p;  2w      REG     = 26,10     47549 37261653 /Volumes/atlantic/rushlogs/3d/= tjn_se_0210_c004rs_a033as_l014hh_seal_matteSealA.log/0008

Note: This not did not try to render frame 0008 from what we can tell.

Should Render be writing the log file directly, or should the STDERR from R= ender be passed to Rush to do the writing?

The PID’s listed do not show up in the process list (top or ps) so we= assume the kernel is experiencing some sort of bug.

We also thought that perhaps this render node was assigned to render this f= rame, and at the last minute informed that another node was already doing i= t so told to abort it’s render. It’s hard to tell from the cpu.= acct file what is happening as I think this only gets updated at the end of= each successful frame.

We are not able to umount the NFS share that the logs are written to, which= again suggests the logs are somehow the cause. In fact the only way to get= the CPU to behave normally is to restart the box, which can only be done b= y physically holding the power button in.

In this case the render node is running OSX 10.5.4, and is an Intel Core 2 = Duo.

Any ideas?

DYLAN PENHALE
IT MANAGER

FUEL
65 KING STREET
NEWTOWN  SYDNEY
NSW  2042  AUSTRALIA

T. 61 2 9557 7799
F. 61 2 9557 = 7882
M. 0424 655 3= 20

WWW.FUELVFX.COM
 --_000_C6D7E47110986dylanfuelvfxcom_-- *** ***************************************************** *** Message #1886 ***************************************************** *** ***************************************************** Subject: Re: NFS Permissions & Snow Leopard From: Greg Ercolano Date: Thu, 10 Sep 2009 15:21:55 -0400 Greg Ercolano wrote: > Craig Allison wrote: >> Just to let everyone know I've just done an install of Rush version >> 102.42a9 on Snow Leopard and all works well as long as you install Rosetta. > > Actually, no, you /shouldn't/ need Rosetta > if you comment out this section of the install script: > [snip] > > The rush binaries will all run fine (including the GUIs) > without the above 'Rez' code. This is only needed for > older versions of OSX that depended on resource/data forks. Actually, I stand corrected on that. During testing last week, I noticed that all the Mac Universal releases of Rush to date (102.42a9b and older) had one binary that was *still* PPC that I didn't know about; the little executable that is in all the submit-*.app dirs. To address that, I've posted a patch in this newsgroup article: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1885 So that will fix the problem with the existing versions of Rush for OSX. There will also be a maintenance release of Rush (102.42a9c) that will beta next week that has a fix for that, as well as fixed for other obscure issues that came up over the last month or two. *** ***************************************************** *** Message #1885 ***************************************************** *** ***************************************************** Subject: Snow Leopard -- Fixes for Rush 102.42a9b and older From: Greg Ercolano Date: Wed, 09 Sep 2009 18:45:07 -0400 RUSH + SNOW LEOPARD There will be a new maintenance release of Rush that addresses Snow Leopard: Rush 102.42a9c, which will beta in about a week. (Still doing internal testing) It will have other things as well; watch this space for details. * * * Meanwhile, those of you running Rush 102.42a9b or older, I've put together a 'fix script' for you which fixes your existing Rush OSX installations to run on Snow Leopard: http://www.seriss.com/rush/releases/patches/rush-snow-leopard-fix-scripts--for-102-42a9b-and-older.tar.gz (Use your downloads login/password to install it. If you forgot your downloads password, please contact me.) See the README file for details. It's very simple to use; just run the appropriate script as root, and it will apply the fixes automagically. Can be run on older OSX releases as well, so that you can then just copy the mac installs to snow leopard machines. You do /not/ need to stop the queue to run these scripts; they can be run while renders are running and guis are open. The fix script repairs these things: o The rush/etc/bin/install.sh script o The rush/etc/S99rush boot script (small cosmetic fixes.. nothing critical) o The wrapper binaries in examples/Applications/submit-*.app/Contents/MacOS which handle bringing up the submit scripts from the Finder. This is somewhat important; it turns out these binaries were PPC and I didn't know it. The fix installs universal bins for these, since Snow Leopard no longer runs PPC binaries by default. So if you're running any release of Rush 102.42a8 thru 102.42a9b, this fix script should help you for running Rush on Snow Leopard. These fixes will be in all Rush releases 102.42a9c and up, so if you upgrade to that release, you won't need to run the fix script. If you have any questions, let me know. *** ***************************************************** *** Message #1884 ***************************************************** *** ***************************************************** Subject: Re: [SYSADMIN/OSX] Changing the global umask (eg. for the Finder,etc) From: Greg Ercolano Date: Thu, 03 Sep 2009 14:59:13 -0400 The 'defaults write' method is probably a good one. If you like editing files by hand or by script, here's a macosxhints article that covers a difference in Leopard: http://www.macosxhints.com/article.php?story=20071207091554360 Quoting the relevant bits of the article, in case the link goes stale: "Setting NSUmask in Leopard is done differently than in previous OS X releases (older hints on NSUmask). In 10.5, NSUmask is gone. To set a default umask (for both shell and GUI apps), edit /etc/launchd.conf and add this line: umask 077 ..where 077 is the new default umask. If nothing is there, the default is 022. Note, the /etc/launchd.conf file umask "trick" should work in Tiger too, but I didn't test it." Most of you will probably want 'umask 0' for wide open perms (rw-rw-rw-) or maybe 'umask 2' for wide open perms to just the user + group, and read only for 'other' (rw-rw--r) Mathieu Xavier Mauser wrote: > On 2006-04-18 15:23:48 -0700, Greg Ercolano said: > >> Greg Ercolano wrote: >>> OSX: HOW TO CHANGE THE GLOBAL UMASK FOR ALL USERS TO 002 >>> -------------------------------------------------------- > > Hi > > This is what works for my set up. > > 1. For group read-write perms, in the Finder. Use Terminal to set > globally (all users/same box): > > sudo defaults write /Library/Preferences/.GlobalPreferences NSUmask 2 > > 2. And, in Terminal, to set this for apps launched from the Bash shell: > > Put "umask 002" in /etc/profile (with no quotation marks) *** ***************************************************** *** Message #1883 ***************************************************** *** ***************************************************** Subject: Re: NFS Permissions & Snow Leopard From: Greg Ercolano Date: Wed, 02 Sep 2009 14:10:35 -0400 Craig Allison wrote: > Just to let everyone know I've just done an install of Rush version > 102.42a9 on Snow Leopard and all works well as long as you install Rosetta. Actually, no, you /shouldn't/ need Rosetta if you comment out this section of the install script: # rez=$RUSH_DIR/etc/bin/Rez # for i in $RUSH_DIR/examples/bin/input.app/Contents/MacOS/input \ # $RUSH_DIR/examples/bin/input; do # echo -n "-- Creating resource forks for 'input': " # i_dir=`dirname $i` # if ( cd $i_dir && $rez -t APPL $RUSH_DIR/etc/bin/mac.r -o input ); then # echo OK # else # echo "FAILED while working on ${i}" # let errors++ # fi # done The rush binaries will all run fine (including the GUIs) without the above 'Rez' code. This is only needed for older versions of OSX that depended on resource/data forks. Next release will have a fix for that, but you can just comment out the section by hand. If you're installing rush Universal build (which has the intel and ppc binaries) you won't need Rosetta. Rosetta would only be needed if you had an old version of Rush that was PPC only, which would be about 3 years old or more. > Slightly off topic as far as rush is concerned, but I wondered if anyone > out there is using Leopard or Snow Leopard over NFS to an XSan share and > is getting the same issue as me with permissions. > > If I open a tif file from a Leopard box client, for example, edit it > then try to save over the file I get a "file locked" permission error, > this is using both Preview and Photoshop. What are your NFS mount flags? Also, while the file is open, does ls(1) show any interesting flags on the file? (eg. 'ls -la@" (extended attributes), 'ls -lae' (ACLS) ) I often disable file locking over NFS because I've never trusted it; see 'man mount_nfs' the 'nolocks' option. Not sure though if this will affect your situation or not. It sounds like somehow the lock is remaining active. I'll try your test with Preview in just a bit. Could be an actual problem with how > And just another gripe with Snow/Leopard, why does the Finder often show > a bunch of identical 2k dpx files as different file sizes, even though > Get Info shows the correct, and identical, byte count???? i.e. A byte > count of 12,754,744 displays in the Finder as 15.4Mb, with the next file > in the sequence showing as 17.3Mb even though it has the same byte count!! Wow, sounds like a Finder bug. IIRC, in Snow Leopard the Finder is a complete rewrite, so there could easily be hiccups in this new code. For a long time back in the 10.0/10.1 days, there were bad bugs in the Finder's computation of disk size. Snow Leopard is only 10.5.0 at the moment (".0" releases are always the most unstable) -- there will probably be updates that fix it. But you might want to report it, giving details of your Xsan/NFS mount flags and file system details (HFS, HFS+). I'll try a few experiments later today with some large files just to check this, as I'm curious. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1882 ***************************************************** *** ***************************************************** Subject: NFS Permissions & Snow Leopard From: Craig Allison Date: Wed, 02 Sep 2009 12:17:28 -0400 --Apple-Mail-5-870611914 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Just to let everyone know I've just done an install of Rush version 102.42a9 on Snow Leopard and all works well as long as you install Rosetta. Slightly off topic as far as rush is concerned, but I wondered if anyone out there is using Leopard or Snow Leopard over NFS to an XSan share and is getting the same issue as me with permissions. If I open a tif file from a Leopard box client, for example, edit it then try to save over the file I get a "file locked" permission error, this is using both Preview and Photoshop. This is to a share coming from a Leopard server box, if I do the same to a share from a Tiger box it works fine. If I use a Tiger client I can overwrite files from both the Leopard share and the Tiger share. I have checked that the correct user with rwx is being exported from the servers and all is well. And just another gripe with Snow/Leopard, why does the Finder often show a bunch of identical 2k dpx files as different file sizes, even though Get Info shows the correct, and identical, byte count???? i.e. A byte count of 12,754,744 displays in the Finder as 15.4Mb, with the next file in the sequence showing as 17.3Mb even though it has the same byte count!! Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com --Apple-Mail-5-870611914 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Just to let everyone know I've just done an install of Rush version = 102.42a9 on Snow Leopard and all works well as long as you install = Rosetta.

Slightly = off topic as far as rush is concerned, but I wondered if anyone out = there is using Leopard or Snow Leopard over NFS to an XSan share and is = getting the same issue as me with permissions.

If I open a tif file from = a Leopard box client, for example, edit it then try to save over the = file I get a "file locked" permission error, this is using both Preview = and Photoshop.

This is to a share coming = from a Leopard server box, if I do the same to a share from a Tiger box = it works fine.

If I use a Tiger client I = can overwrite files from both the Leopard share and the Tiger = share.

I = have checked that the correct user with rwx is being exported from the = servers and all is well.

And just another gripe = with Snow/Leopard, why does the Finder often show a bunch of identical = 2k dpx files as different file sizes, even though Get Info shows the = correct, and identical, byte count???? i.e. A byte count of 12,754,744 = displays in the Finder as 15.4Mb, with the next file in the sequence = showing as 17.3Mb even though it has the same byte count!!

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


= --Apple-Mail-5-870611914-- *** ***************************************************** *** Message #1881 ***************************************************** *** ***************************************************** Subject: Re: Adding extra variables to logs From: Craig Allison Date: Thu, 13 Aug 2009 04:40:44 -0400 --Apple-Mail-49--884753689 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Thanks Greg Some great tips there if I have to change the way this works or add any extra reporting. Much appreciated once again! Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 12 Aug 2009, at 19:46, Greg Ercolano wrote: > [posted to rush.general] > > Hi Craig, > > Glad you found a solution. > > Here's what I was going to reply yesterday, but forgot > to relply, which may give you some other ideas as well: > >>> I have added custom fields to the rush submit and would like to be >>> able to include the slate notes on the jobdone email to give >>> production a better idea of what's been done. > > Your jobdonecommand can invoke a script that gathers the > info from the files left behind in the log directory, or > from rush itself. > > When the jobdonecommand runs, environment variables such > as RUSH_JOBID will be set, which you can use to gather > info about the job with regular rush commands (like 'rush -ljf', > 'rush -lf', etc). > > Any info you add to the submit form you can save into a > data file in the log directory at submit time, eg: > > # SAVE '$in{MySlateInfo}' TO A FILE IN THE RUSH LOG DIRECTORY > # This will be picked up by the jobdonecommand when the job is > done. > # > if ( $in{LogDirectory} ne "-" ) { > my $slatefile = "$in{LogDirectory}/slate-info.dat"; > open(SLATEFILE, ">$slatefile"); > print SLATEFILE "$in{MySlateInfo}\n"; > close(SLATEFILE); > } > > ..then, when the job finishes and your jobdonecommand is running, > it can use the RUSH_JOBID or RUSH_LOGFILE variables to determine > the location of the log directory, load the 'slate-info.dat' file > (if it exists) and use that for the emails. > >>> Does anyone know a good way of adding new variables to the >>> jobdonecommand log or the job log? > > To add info to the 'jobdonecommand log', just have your > script print the messages you want to either stdout or stderr > and they'll appear in the jobdonecommand log. > >>> I have used the jobdonecommand.log to pull in the artist's name >>> to the >>> email from the "Owner" field. > > To send mail from the jobdonecommand script, you can use > the various scripting techniques for sending email, such as > perl's Net::SMTP module, eg: > > ---- snip > #!/usr/bin/perl -w > # > # Example of how to send emails from perl on all platforms (win/ > unix/osx) > # > use strict; > use Net::SMTP; > my $mailto = "foo\@bar.com"; > my $relay = "relay.bar.com"; > my $smtp = Net::SMTP->new($relay); > if ( ! defined($smtp) ) { > # ERROR OCCURRED -- TRY AGAIN W/DEBUG ENABLED > $smtp = Net::SMTP->new($relay, Debug => 1); > if ( !defined($smtp) ) { > print "ERROR: Could not connect to $relay (see above)\n"; > exit(1); > } > } > my ($errs, $expect) = (0,0); > $expect++; $errs += $smtp->mail($mailto); > $expect++; $errs += $smtp->to($mailto); > $expect++; $errs += $smtp->data(); > $expect++; $errs += $smtp->datasend("From: Foo <$mailto>\n"); > $expect++; $errs += $smtp->datasend("To: $mailto\n"); > $expect++; $errs += $smtp->datasend("Subject: testing from SMTP/perl > \n"); > $expect++; $errs += $smtp->datasend("\n"); > $expect++; $errs += $smtp->datasend("This is a test\n"); > $expect++; $errs += $smtp->datasend("\n"); > $expect++; $errs += $smtp->dataend(); > $expect++; $errs += $smtp->quit; > print "EXPECT=$expect, ERRORS=$errs\n"; > if ( $expect != $errs ) { > print "SMTP_ERROR=".$smtp->cgi_error()."\n"; > } > exit(0); > ---- snip > > Craig Allison wrote: >> I've solved the issue by getting the submit script to send an >> email to >> the production team containing Shot/Artist/Frames/Notes and then the >> jobdonecommand lets them know when it's complete and ready for >> review. >> >> Cheers >> Craig >> >> On 10 Aug 2009, at 15:01, Craig Allison wrote: >> >>> Hello there >>> >>> Does anyone know a good way of adding new variables to the >>> jobdonecommand log or the job log? >>> >>> I have added custom fields to the rush submit and would like to be >>> able to include the slate notes on the jobdone email to give >>> production a better idea of what's been done. >>> >>> I have used the jobdonecommand.log to pull in the artist's name >>> to the >>> email from the "Owner" field. > > > -- > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) > --Apple-Mail-49--884753689 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Thanks Greg

Some = great tips there if I have to change the way this works or add any extra = reporting.

Much appreciated once = again!

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


On 12 Aug 2009, at 19:46, Greg Ercolano = wrote:

[posted to = rush.general]

Hi Craig,

Glad you found a = solution.

Here's what I was going to reply = yesterday, but forgot
to = relply, which may give you some other ideas as well:

I have added = custom fields to the rush submit and would like to be
able to include the slate notes on the jobdone email = to give
production a better idea of = what's been done.

Your = jobdonecommand can invoke a script that gathers the
info from the files left behind = in the log directory, or
from rush = itself.

When the jobdonecommand runs, = environment variables such
as = RUSH_JOBID will be set, which you can use to gather
info about the job with regular = rush commands (like 'rush -ljf',
'rush = -lf', etc).

Any info you add to the submit = form you can save into a
data file = in the log directory at submit time, eg:

# SAVE = '$in{MySlateInfo}' TO A FILE IN THE RUSH LOG DIRECTORY
# =A0 =A0 = This will be picked up by the jobdonecommand when the job is = done.
#
if ( = $in{LogDirectory} ne "-" ) {
=A0 =A0 my $slatefile =3D = "$in{LogDirectory}/slate-info.dat";
=A0 =A0 open(SLATEFILE, = ">$slatefile");
=A0 =A0 print SLATEFILE = "$in{MySlateInfo}\n";
=A0 =A0 = close(SLATEFILE);
}

..then, = when the job finishes and your jobdonecommand is running,
it can use the RUSH_JOBID or = RUSH_LOGFILE variables to determine
the = location of the log directory, load the 'slate-info.dat' file
(if it exists) and use that for = the emails.

=
Does anyone know a good way of adding new variables = to the
jobdonecommand log or the job = log?

To add info to the = 'jobdonecommand log', just have your
script = print the messages you want to either stdout or stderr
and they'll appear in the = jobdonecommand log.
I have used the jobdonecommand.log to pull in the = artist's name to the
email from the "Owner" = field.

To send mail from the = jobdonecommand script, you can use
the = various scripting techniques for sending email, such as
perl's Net::SMTP module, = eg:

---- snip
#
# Example of how to send emails from perl on all = platforms (win/unix/osx)
#
use strict;
use = Net::SMTP;
my $mailto =3D = "foo\@bar.com";
my $relay=A0 =3D = "relay.bar.com";
my $smtp =3D = Net::SMTP->new($relay);
if ( ! = defined($smtp) ) {
=A0 =A0 # ERROR OCCURRED -- TRY = AGAIN W/DEBUG ENABLED
=A0 =A0 $smtp =3D = Net::SMTP->new($relay, Debug =3D> 1);
=A0 =A0 if ( !defined($smtp) ) = {
=A0 =A0 =A0= =A0 print "ERROR: Could not connect to $relay (see = above)\n";
=A0 =A0 =A0 =A0 = exit(1);
=A0 =A0 }
}
my ($errs, $expect) =3D = (0,0);
$expect++; $errs +=3D = $smtp->mail($mailto);
$expect++; = $errs +=3D $smtp->to($mailto);
$expect++; = $errs +=3D $smtp->data();
$expect++; = $errs +=3D $smtp->datasend("From: Foo <$mailto>\n");
$expect++; $errs +=3D $smtp->datasend("To: = $mailto\n");
$expect++; $errs +=3D = $smtp->datasend("Subject: testing from SMTP/perl\n");
$expect++; $errs +=3D = $smtp->datasend("\n");
$expect++; = $errs +=3D $smtp->datasend("This is a test\n");
$expect++; $errs +=3D = $smtp->datasend("\n");
$expect++; = $errs +=3D $smtp->dataend();
$expect++; = $errs +=3D $smtp->quit;
print = "EXPECT=3D$expect, ERRORS=3D$errs\n";
if ( = $expect !=3D $errs ) {
=A0 =A0 print = "SMTP_ERROR=3D".$smtp->cgi_error()."\n";
exit(0);
---- snip

Craig Allison wrote:
=
I've solved the issue by = getting the submit script to send an email to
the production team containing = Shot/Artist/Frames/Notes and then the

Cheers

On 10 Aug 2009, at 15:01, Craig Allison = wrote:

=
Hello there

Does = anyone know a good way of adding new variables to the
jobdonecommand log or the job log?

I have = added custom fields to the rush submit and would like to be
able to include the slate notes on the jobdone email = to give
production a better idea of = what's been done.

I have used the jobdonecommand.log to pull in the = artist's name to the
email from the "Owner" = field.


--=A0
Greg = Ercolano, erco@(email surpressed)
Seriss Corporation
Rush = Render Queue, http://seriss.com/rush/
Tel: 626-795-5922x23
Fax: = 626-795-5947
Cel: 310-266-8906

=

= --Apple-Mail-49--884753689-- *** ***************************************************** *** Message #1880 ***************************************************** *** ***************************************************** Subject: Re: Adding extra variables to logs From: Greg Ercolano Date: Wed, 12 Aug 2009 14:46:33 -0400 Hi Craig, Glad you found a solution. Here's what I was going to reply yesterday, but forgot to relply, which may give you some other ideas as well: >> I have added custom fields to the rush submit and would like to be >> able to include the slate notes on the jobdone email to give >> production a better idea of what's been done. Your jobdonecommand can invoke a script that gathers the info from the files left behind in the log directory, or from rush itself. When the jobdonecommand runs, environment variables such as RUSH_JOBID will be set, which you can use to gather info about the job with regular rush commands (like 'rush -ljf', 'rush -lf', etc). Any info you add to the submit form you can save into a data file in the log directory at submit time, eg: # SAVE '$in{MySlateInfo}' TO A FILE IN THE RUSH LOG DIRECTORY # This will be picked up by the jobdonecommand when the job is done. # if ( $in{LogDirectory} ne "-" ) { my $slatefile = "$in{LogDirectory}/slate-info.dat"; open(SLATEFILE, ">$slatefile"); print SLATEFILE "$in{MySlateInfo}\n"; close(SLATEFILE); } ..then, when the job finishes and your jobdonecommand is running, it can use the RUSH_JOBID or RUSH_LOGFILE variables to determine the location of the log directory, load the 'slate-info.dat' file (if it exists) and use that for the emails. >> Does anyone know a good way of adding new variables to the >> jobdonecommand log or the job log? To add info to the 'jobdonecommand log', just have your script print the messages you want to either stdout or stderr and they'll appear in the jobdonecommand log. >> I have used the jobdonecommand.log to pull in the artist's name to the >> email from the "Owner" field. To send mail from the jobdonecommand script, you can use the various scripting techniques for sending email, such as perl's Net::SMTP module, eg: ---- snip #!/usr/bin/perl -w # # Example of how to send emails from perl on all platforms (win/unix/osx) # use strict; use Net::SMTP; my $mailto = "foo\@bar.com"; my $relay = "relay.bar.com"; my $smtp = Net::SMTP->new($relay); if ( ! defined($smtp) ) { # ERROR OCCURRED -- TRY AGAIN W/DEBUG ENABLED $smtp = Net::SMTP->new($relay, Debug => 1); if ( !defined($smtp) ) { print "ERROR: Could not connect to $relay (see above)\n"; exit(1); } } my ($errs, $expect) = (0,0); $expect++; $errs += $smtp->mail($mailto); $expect++; $errs += $smtp->to($mailto); $expect++; $errs += $smtp->data(); $expect++; $errs += $smtp->datasend("From: Foo <$mailto>\n"); $expect++; $errs += $smtp->datasend("To: $mailto\n"); $expect++; $errs += $smtp->datasend("Subject: testing from SMTP/perl\n"); $expect++; $errs += $smtp->datasend("\n"); $expect++; $errs += $smtp->datasend("This is a test\n"); $expect++; $errs += $smtp->datasend("\n"); $expect++; $errs += $smtp->dataend(); $expect++; $errs += $smtp->quit; print "EXPECT=$expect, ERRORS=$errs\n"; if ( $expect != $errs ) { print "SMTP_ERROR=".$smtp->cgi_error()."\n"; } exit(0); ---- snip Craig Allison wrote: > I've solved the issue by getting the submit script to send an email to > the production team containing Shot/Artist/Frames/Notes and then the > jobdonecommand lets them know when it's complete and ready for review. > > Cheers > Craig > > On 10 Aug 2009, at 15:01, Craig Allison wrote: > >> Hello there >> >> Does anyone know a good way of adding new variables to the >> jobdonecommand log or the job log? >> >> I have added custom fields to the rush submit and would like to be >> able to include the slate notes on the jobdone email to give >> production a better idea of what's been done. >> >> I have used the jobdonecommand.log to pull in the artist's name to the >> email from the "Owner" field. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1879 ***************************************************** *** ***************************************************** Subject: Re: Adding extra variables to logs From: Craig Allison Date: Wed, 12 Aug 2009 07:46:36 -0400 --Apple-Mail-37--960002535 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed I've solved the issue by getting the submit script to send an email to the production team containing Shot/Artist/Frames/Notes and then the jobdonecommand lets them know when it's complete and ready for review. Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 10 Aug 2009, at 15:01, Craig Allison wrote: > Hello there > > Does anyone know a good way of adding new variables to the > jobdonecommand log or the job log? > > I have added custom fields to the rush submit and would like to be > able to include the slate notes on the jobdone email to give > production a better idea of what's been done. > > I have used the jobdonecommand.log to pull in the artist's name to > the email from the "Owner" field. > > Cheers > > Craig > > Craig Allison > Digital Systems & Data Manager > The Senate Visual Effects > Twickenham Film Studios > St.Margarets > Twickenham > TW1 2AW > > t: 0208 607 8866 > skype: craig_9000 > www.senatevfx.com > > > --Apple-Mail-37--960002535 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 I've solved the issue by getting the submit script to send an email to = the production team containing Shot/Artist/Frames/Notes and then the = jobdonecommand lets them know when it's complete and ready for = review.

Cheers

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


On 10 Aug 2009, at 15:01, Craig Allison = wrote:

Hello there

Does anyone know a good = way of adding new variables to the jobdonecommand log or the job = log?

I have = added custom fields to the rush submit and would like to be able to = include the slate notes on the jobdone email to give production a better = idea of what's been done.

I have used the = jobdonecommand.log to pull in the artist's name to the email from the = "Owner" field.

Cheers

Craig

The Senate Visual = Effects
St.Margarets

t: 0208 607 8866=A0
skype: craig_9000

=


= --Apple-Mail-37--960002535-- *** ***************************************************** *** Message #1878 ***************************************************** *** ***************************************************** Subject: Adding extra variables to logs From: Craig Allison Date: Mon, 10 Aug 2009 10:01:36 -0400 --Apple-Mail-19-1022777502 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello there Does anyone know a good way of adding new variables to the jobdonecommand log or the job log? I have added custom fields to the rush submit and would like to be able to include the slate notes on the jobdone email to give production a better idea of what's been done. I have used the jobdonecommand.log to pull in the artist's name to the email from the "Owner" field. Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com --Apple-Mail-19-1022777502 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hello there

Does = anyone know a good way of adding new variables to the jobdonecommand log = or the job log?

I have added custom fields = to the rush submit and would like to be able to include the slate notes = on the jobdone email to give production a better idea of what's been = done.

I have = used the jobdonecommand.log to pull in the artist's name to the email = from the "Owner" field.

Cheers

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


= --Apple-Mail-19-1022777502-- *** ***************************************************** *** Message #1877 ***************************************************** *** ***************************************************** Subject: Re: Rush FIFO fix From: Greg Ercolano Date: Thu, 30 Jul 2009 01:54:06 -0400 Greg Ercolano wrote: > (If you don't use FIFO scheduling, you can ignore this) > > The folks at Deluxe just helped me identify a problem with FIFO scheduling > when multiple job servers are in use. > [..] > Watch this space for release info. I've uploaded zip/tar files for Deluxe to try out; basically a new rushd daemon solves the problem. Once they've confirmed things are working well, I'll make the new fixes generally available. *** ***************************************************** *** Message #1876 ***************************************************** *** ***************************************************** Subject: Rush FIFO fix From: Greg Ercolano Date: Wed, 29 Jul 2009 20:50:07 -0400 (If you don't use FIFO scheduling, you can ignore this) The folks at Deluxe just helped me identify a problem with FIFO scheduling when multiple job servers are in use. We just nailed down the problem minutes ago, and I've found the problem and I think I've fixed it.. will continue to do some testing, and should have a fix release by tonight. The problem shows up in Rush 102.42a9 thru 102.42a9b, and the problem shows up where jobs simply don't always schedule in FIFO order when multiple job servers are involved. 102.42a9c will have the fix. You won't likely see the problem if all your jobs are being submitted to a single job server (I know some of you prefer to run this way, so you may not have noticed it). I suppose that is something of a "workaround" if you don't have the ability to take the update. Watch this space for release info. *** ***************************************************** *** Message #1875 ***************************************************** *** ***************************************************** Subject: Rush 102.42a9 runs OK on Windows 7 RC From: Greg Ercolano Date: Fri, 24 Jul 2009 15:18:22 -0400 Rush 102.42a9 (and up) tested to run fine on Windows 7 64bit RC. Rush and Perl both installed normally with no caveats. Submitted and rendered network jobs with UNC paths to a linux/samba file server, bouncing cpu bars in rushtop. As always, be sure to run the install as the real Administrator. The UAC (introduced in Vista) causes there to be a distinction between running a DOS window as an administrative user vs. "running as Administator". An easy way toopen a DOS window as Administrator: 1) Start->All Programs->Accessories 2) Right click on 'Command Prompt' and choose "Run as administrator". This ensures the install script has sufficient privileges to make changes to the system environment variables (eg. for making changes to the PATH) * * * Some general comments about Windows 7 that are not Rush related: o 'edit' is gone, apparently. It was a nice DOS based text editor. o 'wordpad' is still not in the PATH, causing a 'command not found' in the DOS shell. o 'notepad' is still as dumb as a sack of hammers o Start -> Run, where have you gone? o To enable the real administrator user so that you can actually login as administrator from the login screen: 1) Start->All Programs->Accessories, right click "Command Prompt" and choose "Run as administrator". 2) In the DOS shell, run: net user administrator /active:yes 3) Restart or log off *** ***************************************************** *** Message #1874 ***************************************************** *** ***************************************************** Subject: Re: First In-First Out Tip From: Marco Recuay Date: Mon, 20 Jul 2009 19:37:49 -0400 Thanks for the correction Greg.. I guess I'll have to do some more testing on this. The clocks all appear synchronized on all the machines, and they all have the same rush version and updated rush.conf file. Using the same job server is a workaround for now, but when we have a free moment I'll see how they react to the tests you outlined to pinpoint where the problem lies. On 2009-07-20 08:40:37 -0700, Greg Ercolano said: > BTW, Marco, if you're having strange behavior with different > job servers, I'll be happy to help. > > Make sure you've got 'sched fifo' in ALL the rush.conf files, > and are running the same version of rush that supports FIFO > (102.42a9 or higher) on all the machines, since the older > releases won't understand FIFO scheduling. > > There are some easy tests you can do from the command line > to submit a couple of jobs and watch them compete. > > For instance, from a Unix machine (linux or OSX), you can > submit two 100 frame jobs as a test: > > (echo title AAA; echo frames 1-100; echo cpus +any=500@10; echo command > rush -sleep 10) | rush -submit > > (echo title BBB; echo frames 1-100; echo cpus +any=500@10; echo command > rush -sleep 10) | rush -submit > > Each line will submit a job that does nothing but sleep 10 seconds > per frame. > > Those should be two separate lines; make sure your newsreader > is wide enough to show those as complete lines before copy/pasting. > > Run the lines one at a time, with at least 1 or 2 seconds between > each submit, so the FIFO system can tell which job was first. > > When running, the AAA job should get all the cpus, and the BBB > job should wait around until the AAA job starts finishing. > > You should also be able to run similar command lines on other job > servers, or tack a hostname on the end, after the 'rush -submit', > to tell it to use some other machine as the job server, eg. to use > 'tahoe' as the job server: > > (echo title CCC; echo frames 1-100; echo cpus +any=500@10; echo command > rush -sleep 10) | rush -submit tahoe *** ***************************************************** *** Message #1873 ***************************************************** *** ***************************************************** Subject: Re: First In-First Out Tip From: Greg Ercolano Date: Mon, 20 Jul 2009 11:40:37 -0400 BTW, Marco, if you're having strange behavior with different job servers, I'll be happy to help. Make sure you've got 'sched fifo' in ALL the rush.conf files, and are running the same version of rush that supports FIFO (102.42a9 or higher) on all the machines, since the older releases won't understand FIFO scheduling. There are some easy tests you can do from the command line to submit a couple of jobs and watch them compete. For instance, from a Unix machine (linux or OSX), you can submit two 100 frame jobs as a test: (echo title AAA; echo frames 1-100; echo cpus +any=500@10; echo command rush -sleep 10) | rush -submit (echo title BBB; echo frames 1-100; echo cpus +any=500@10; echo command rush -sleep 10) | rush -submit Each line will submit a job that does nothing but sleep 10 seconds per frame. Those should be two separate lines; make sure your newsreader is wide enough to show those as complete lines before copy/pasting. Run the lines one at a time, with at least 1 or 2 seconds between each submit, so the FIFO system can tell which job was first. When running, the AAA job should get all the cpus, and the BBB job should wait around until the AAA job starts finishing. You should also be able to run similar command lines on other job servers, or tack a hostname on the end, after the 'rush -submit', to tell it to use some other machine as the job server, eg. to use 'tahoe' as the job server: (echo title CCC; echo frames 1-100; echo cpus +any=500@10; echo command rush -sleep 10) | rush -submit tahoe -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1872 ***************************************************** *** ***************************************************** Subject: Re: First In-First Out Tip From: Greg Ercolano Date: Mon, 20 Jul 2009 11:18:56 -0400 Marco Recuay wrote: > Just wanted to pass on my experience with the new FIFO scheduling, > since it wasn't working as I expected when first enabled. > > It appears that FIFO only schedules the jobs on the same job server. If > you want all renders across the network to schedule FIFO, you need to > make sure that jobs are sent to one submit host only, which then orders > them based on the job age. Hmm, an interesting theory, but no -- FIFO works off time stamps from when the job was submitted, regardless which machine is the job server. Assuming the machines acting as job servers are time synched, it shouldn't matter which machine is the job server. You can see the raw FIFO numbers used for scheduling in the "Jobs Full" report: "FifoSchedOrder" -- the raw unix time() value (eg. 3542120794) "FifoSchedOrderDate" -- the same value as a date/time (eg. 06/26/09,09:28:23.41) The 'Elapsed' time in the normal "Jobs" all "All Jobs" report can usually be used as the indicator of the FIFO scheduling value, assuming the clocks are synched well. The way FIFO works: when a proc becomes available, and the priority of the jobs asking to use it are equal, the unix time value is used to figure out which job gets to run; the oldest job wins the proc. So if you have 100 machines and everyone's submitting with +any=100@10, then no matter which machine is the job server, the oldest job gets the next available proc. If you have 100 machines and people are asking for less cpus than are on the farm, eg. +any=5@10, then the oldest job will get the first 5 procs, and the next oldest job(s) will get the rest. Note that if people are submitting with different priorities, then FIFO breaks up into tiers of priority, ie. priority takes precedence over FIFO. For instance if everyone is in the FIFO queue @10 priority (examples above), and NEW GUY submits @11 priority, his job will be in front of all the @10's. And if other people submit @11 (same as him), then FIFO rules will determine which of the @11 jobs get the next cpu. Anything "leftover" goes to the other jobs in the @10 priority. So if these jobs are submitted, the next cpu to come available will try to give itself to one of the jobs in this order, starting at the top: OWNER PRIORITY SUBMIT TIME ----- ---------- ----------- _ Fred +any=100@11 14:00 Today | 11 priority "tier" Gina +any=100@11 15:00 Today _| _ Jane +any=100@10 9:00 Today | Tim +any=100@10 13:00 Today | 10 priority "tier" Sandy +any=100@10 14:40 Today _| So the next available cpu will go to Fred first, and if his job is maxed out on cpus or has no more frames to render, Gina's job is next, then Jane, Tim, and Sandy last. Note that even though Jane submitted before everyone (9am), she was still @10 priority, so she still comes "after" all the higher @11 priority jobs. > One trick would also be to have more than one designated submit host, > so you could have separate FIFO queues for different tasks. You can get different FIFO queues via priority, as shown above where there are two FIFO queues; @11 and @10. You can also get different FIFO queues by requesting different hostgroups, ie. splitting the farm into two or more groups. So if there's a job that needs to be in front of everything in the FIFO queue, but only needs 5 procs, then that job should submit at a higher priority (eg. @11 in the examples above) but should just ask for =5, eg. +any=5@11 Let me know if anyone has questions about the above, or if you'd like more details. Pretty much the easiest way to make use of FIFO is to just have everyone submit at the SAME PRIORITY, and FIFO will be very predictable. It's only when you introduce priority differences that you get the multiple FIFO queues happening. *** ***************************************************** *** Message #1871 ***************************************************** *** ***************************************************** Subject: First In-First Out Tip From: Marco Recuay Date: Sun, 19 Jul 2009 21:08:19 -0400 Just wanted to pass on my experience with the new FIFO scheduling, since it wasn't working as I expected when first enabled. It appears that FIFO only schedules the jobs on the same job server. If you want all renders across the network to schedule FIFO, you need to make sure that jobs are sent to one submit host only, which then orders them based on the job age. One trick would also be to have more than one designated submit host, so you could have separate FIFO queues for different tasks. That's it... something that might have been obvious to other admins, but I think it's worth documenting here. -Marco *** ***************************************************** *** Message #1870 ***************************************************** *** ***************************************************** Subject: Re: Job Done Command Not Working On Some Boxes From: Greg Ercolano Date: Fri, 17 Jul 2009 23:20:37 -0400 Newsgroup follow up. OK, after working with Craig earlier this week, I was able to determine the exact combo of events it took to replicate this. Turned out these specifics were needed to make this problem show up: o Rush version 102.42a9 on both machines o Submit from a Tiger workstation with "Submit Host:" pointing at a leopard jobserver o Submitting user has no account on the jobserver, just their workstation o Rush configured to force all renders to run as "render" user o "render" user has a valid account on all machines (workstation and jobserver) o Job submitted with a jobdonecommand With this combo, the renders run ok, but the /jobdonecommand/ fails to run on the leopard machine with this series of errors in the log: >07/17,20:05:03 SECURITY RUSH: uid '0' outside configured range 100-65000 >07/17,20:05:03 INFO jobdonecommand 'ls' failed for jobid=leopard.2: RUSH: uid '0' outside configured range 100-65000 Now that I've got the problem canned, and replicated here at my office, I should have it solved soon -- pretty sure now it's a bug in Rush. *** ***************************************************** *** Message #1869 ***************************************************** *** ***************************************************** Subject: [Q+A] Submit script GUI forms From: Greg Ercolano Date: Thu, 16 Jul 2009 13:09:08 -0400 > We're playing around with customizing the input forms in the > submit scripts. It looks like the submit scripts use a compact > shorthand for defining the forms that is not the same as the format > described in the docs for the 'input.exe' program: > http://www.seriss.com/rush-current/input/ I'm guessing you mean the shorthand 'ascii art' representation of the submit forms that you see in all the modern Rush submit scripts (last 5 years) e.g.: -------------------------------------------------------------------------- snip <> "Rush" = Log Directory: ___________________________________________ ? Browse"" = Log Flags: "Keep Last,Keep All,Overwrite" ? = Max Log Size: ______________ ? AutoDump: "off,fail,done,donefail" ? Done Mail: ______________ ? Dump Mail: ______________ ? -------------------------------------------------------------------------- snip Right.. this format you see in the submit scripts is not actually passed directly to input.exe.. there's a function in .common.pl called CreateInputForm($$$) which converts the "ascii art" into the "long format" that input.exe understands. The reason for doing this: In ye olde days (2003 or so), each submit script had the "long form" input.exe commands, which made the scripts huge and hard to read. Also, everyone edits those scripts to customize, and all that redundant code seemed unnecessary bulk. So I thought, "what's the easiest way to represent these forms?" to reduce the size of the scripts and make them easier to read. 'ascii art' seemed like the best solution; it visually looks just like what you see in the submit forms; easy to read, easy to change. It avoids redundant blocks of code, and I took a few shortcuts to represent common things to all the submit forms (submit/help/default buttons, etc) So I inserted the CreateInputForm() function in .common.pl, and slowly modified all the submit scripts to use the ascii art representation, and pass it to CreateInputForm() to open the GUI. input.exe still understands the "long form", so old scripts with the long form content still work normally. So anyway, the intermediate "long form" GUI commands are generated on the fly by that CreateInputForm() function, and these files are left behind (for info purposes) in the usual place: UNIX: ~/.rush/submit-xxx.in WINDOWS: c:/temp/.rush-/submit-xxx.in What's interesting is that in the next release, input.exe will be able to parse the ascii art format directly. This was done so that submit scripts can be written in other languages (eg. python) without having to port all the logic in CreateInputForm() to each scripting language. *** ***************************************************** *** Message #1868 ***************************************************** *** ***************************************************** Subject: Re: Job Done Command Not Working On Some Boxes From: Greg Ercolano Date: Mon, 13 Jul 2009 16:31:44 -0400 Craig Allison wrote: > Hey Greg > > I'm working late tonight so we're kind of on the same time for now : ) > > Checked the log on the host machine and for the fail jobs I'm getting > the following: > > jobdonecommand 'perl /mnt/vfxxserve3/Scripts/Senate_DoneMail.pl' failed > for jobid=senatemac89.326: RUSH: uid '0' outside configured range 100-65000 > > I've got forceuid/gid set to 501 on the user's machine in rush.conf, > where is it picking this one up from? Hi Craig, Hmm.. apparently it thinks it should be trying to run the command as root on that machine (senatemac89). From that machine, can you send me (via private email): a) The contents of: /usr/local/rush/etc/rush.conf b) The output of: rush -ljf senatemac89.326 It sounds like maybe there's a forceuid of zero on that machine, or the job's owner is root, or there's a bug I need to look into.. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1867 ***************************************************** *** ***************************************************** Subject: Re: Job Done Command Not Working On Some Boxes From: Craig Allison Date: Mon, 13 Jul 2009 16:22:52 -0400 --Apple-Mail-1-773902012 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hey Greg I'm working late tonight so we're kind of on the same time for now : ) Checked the log on the host machine and for the fail jobs I'm getting the following: jobdonecommand 'perl /mnt/vfxxserve3/Scripts/Senate_DoneMail.pl' failed for jobid=senatemac89.326: RUSH: uid '0' outside configured range 100-65000 I've got forceuid/gid set to 501 on the user's machine in rush.conf, where is it picking this one up from? Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 13 Jul 2009, at 20:19, Greg Ercolano wrote: > [posted to rush.general] > > Craig Allison wrote: >> I'm having a problem with certain boxes not running the JobDone >> Command >> on complete of a render. I'm not getting any logs from the faulty >> boxes >> to even suggest the command is being initiated. > > Hi Craig, > > I'd suggest looking in the rushd.log first for the machine acting > as the job server for the job in question. Ie. login to the machine > whose hostname is in the jobid, and look at the rush/var/rushd.log > file for errors about the jobdonecommand. > >> Running Rush 102.42a9 on Leopard and Tiger, running a perl script on >> JobDone. >> >> perl /mnt/vfxxserve3/Scripts/Senate_Donemail.pl >> >> Any ideas? > > Try sending me via private email: > > 1) the 'rush -ljf' info for this job > 2) the contents of your rushd.log > > ..Be sure the job got 'done' since midnight, so that any errors > would likely be in "today's" rushd.log. > > I'll follow up to the group on what we determine is the problem. > > Certainly you should be seeing a 'jobdonecommand.log' in the job's > log directory. Only reason you wouldn't is if a) logs are disabled, > b) the -nolog flag was specified for the jobdonecommand, c) the log > directory was unwritable for some reason by the job server, d) a bug > I don't know about, as 102.42a9 is very recent. > > -- > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) > --Apple-Mail-1-773902012 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hey Greg

I'm = working late tonight so we're kind of on the same time for now : = )

Checked = the log on the host machine and for the fail jobs I'm getting the = following:

jobdonecommand 'perl = /mnt/vfxxserve3/Scripts/Senate_DoneMail.pl' failed for = jobid=3Dsenatemac89.326: RUSH: uid '0' outside configured range = 100-65000

I've got forceuid/gid set to 501 on the user's machine=A0in = rush.conf, where is it picking this one up from?


Cheers


Craig



Craig Allison

Digital Systems & Data=A0Manager

The Senate Visual = Effects

St.Margarets

Twickenham

TW1 2AW


t: 0208 607 8866=A0

skype: craig_9000


=


On 13 Jul 2009, at 20:19, Greg Ercolano = wrote:

[posted to = rush.general]

Craig Allison wrote:
I'm having a problem with = certain boxes not running the JobDone Command
on complete of a render.=A0 I'm not getting any logs from = the faulty boxes
to even suggest the command is = being initiated.

Hi Craig,

I'd = suggest looking in the rushd.log first for the machine acting
as the job server for the job in = question. Ie. login to the machine
whose = hostname is in the jobid, and look at the rush/var/rushd.log
file for errors about the = jobdonecommand.

=
Running Rush 102.42a9 on = Leopard and Tiger, running a perl script on

perl = /mnt/vfxxserve3/Scripts/Senate_Donemail.pl

Any = ideas?
Try sending me via private = email:

1) the 'rush -ljf' info for this = job
2) the contents of your = rushd.log

..Be sure the job got 'done' = since midnight, so that any errors
would = likely be in "today's" rushd.log.

I'll follow up to the group on = what we determine is the problem.

Certainly you should be seeing a = 'jobdonecommand.log' in the job's
log = directory. Only reason you wouldn't is if a) logs are = disabled,
b) the -nolog flag was specified = for the jobdonecommand, c) the log
directory = was unwritable for some reason by the job server, d) a bug
I don't know about, as 102.42a9 = is very recent.

--=A0
Greg = Ercolano, erco@(email surpressed)
Seriss Corporation
Rush = Render Queue, http://seriss.com/rush/
Tel: 626-795-5922x23
Fax: = 626-795-5947
Cel: 310-266-8906

=

= --Apple-Mail-1-773902012-- *** ***************************************************** *** Message #1866 ***************************************************** *** ***************************************************** Subject: Re: Job Done Command Not Working On Some Boxes From: Greg Ercolano Date: Mon, 13 Jul 2009 15:19:35 -0400 Craig Allison wrote: > I'm having a problem with certain boxes not running the JobDone Command > on complete of a render. I'm not getting any logs from the faulty boxes > to even suggest the command is being initiated. Hi Craig, I'd suggest looking in the rushd.log first for the machine acting as the job server for the job in question. Ie. login to the machine whose hostname is in the jobid, and look at the rush/var/rushd.log file for errors about the jobdonecommand. > Running Rush 102.42a9 on Leopard and Tiger, running a perl script on > JobDone. > > perl /mnt/vfxxserve3/Scripts/Senate_Donemail.pl > > Any ideas? Try sending me via private email: 1) the 'rush -ljf' info for this job 2) the contents of your rushd.log ..Be sure the job got 'done' since midnight, so that any errors would likely be in "today's" rushd.log. I'll follow up to the group on what we determine is the problem. Certainly you should be seeing a 'jobdonecommand.log' in the job's log directory. Only reason you wouldn't is if a) logs are disabled, b) the -nolog flag was specified for the jobdonecommand, c) the log directory was unwritable for some reason by the job server, d) a bug I don't know about, as 102.42a9 is very recent. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1865 ***************************************************** *** ***************************************************** Subject: Job Done Command Not Working On Some Boxes From: Craig Allison Date: Mon, 13 Jul 2009 12:17:23 -0400 --Apple-Mail-19-759179278 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello there I'm having a problem with certain boxes not running the JobDone Command on complete of a render. I'm not getting any logs from the faulty boxes to even suggest the command is being initiated. Most of my boxes execute the perl script no problem but a few don't and I can't see anything different between the ones that do and the ones that don't. Running Rush 102.42a9 on Leopard and Tiger, running a perl script on JobDone. perl /mnt/vfxxserve3/Scripts/Senate_Donemail.pl Any ideas? Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com --Apple-Mail-19-759179278 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hello there

I'm = having a problem with certain boxes not running the JobDone Command on = complete of a render. =A0I'm not getting any logs from the faulty boxes = to even suggest the command is being initiated.

Most of my boxes execute = the perl script no problem but a few don't and I can't see anything = different between the ones that do and the ones that = don't.

Running= Rush 102.42a9 on Leopard and Tiger, running a perl script on = JobDone.

perl = /mnt/vfxxserve3/Scripts/Senate_Donemail.pl

Any ideas?

Cheers

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


= --Apple-Mail-19-759179278-- *** ***************************************************** *** Message #1864 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: autostarting in KDE From: Greg Ercolano Date: Fri, 10 Jul 2009 14:42:44 -0400 Ahh, you got it then -- great. Stephan Kosinski wrote: > Greg, > > I found out. hanks for the Autostart clue. > > I actually placed there a script a long time ago that declare the > station offline as soon as a user log in KDE. > > A line was lauching rushtop but it wasn't working and with time I > forgot this line was existing. > > The new version of rushtop seems to be have became launchable by KDE > autostart script. > > Case closed. > > Thank you. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1863 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: autostarting in KDE From: Stephan Kosinski Date: Fri, 10 Jul 2009 14:24:33 -0400 Greg, I found out. hanks for the Autostart clue. I actually placed there a script a long time ago that declare the station offline as soon as a user log in KDE. A line was lauching rushtop but it wasn't working and with time I forgot this line was existing. The new version of rushtop seems to be have became launchable by KDE autostart script. Case closed. Thank you. Stephan. Stephan Kosinski wrote: > Greg Ercolano wrote: >> Starting a new thread for this.. >> >> Stephan Kosinski wrote: >>> A quick question though since I propagated the new binary on my >>> linux >>> stations rushtop is auto-starting at user login. >> >> Hmm, nothing in rushtop or the rush installers does any autostart >> type stuff.. at least not in my code. >> >> Possibly the GUI toolkit rushtop is using has changed some internals >> that makes it more compatible with window manager autostart stuff >> that it didn't have before. >> >> Do other tools autostart if you logout while they're running? >> ie. firefox, konqueror, clocks, etc? > > Yes but only when the session has been saved with them running. That > means at logout or when the user save is session manually > >> >> Sounds like a feature of the window manager remembering what >> was running when you logout, and starting them again when you >> login. >> >> This stuff is likely saved in an ascii file in your ~/.kde directory, >> but by what mechanism I don't know. >> > > I'll check there > > >>> What is reaaly weird is that I was never able to do that with >>> previous >>> binary even by saving the KDE session with rushtop running. >> >> Hmm, I don't know what could have changed in rushtop to do that. >> Nothing in the code to rushtop itself.. I don't know of any way >> to control it from the GUI toolkit's API. >> >>> Do you have any idea how to prevent rushtop to auto-start? Is it >>> related to my system or to the new binary? >> >> Try closing rushtop *then* logging out, then back in. >> I would assume it won't start then. > > > First thing I tried. Close rushtop, save my session, logout and > login again... Rushtop is stubbornly coming back . :) > >> >> I stopped using KDE a while ago; too many memory hungry daemons and >> menus kept moving around, and same to be said for gnome, so I >> standardized on a custom 'tiny' window manager that has no daemons >> and one menu that I defined. This way I can upgrade my workstation, >> but have the interface stay the same. >> >> However, I'll try bringing up KDE on one of my other systems to see >> if I can at least replicate. >> >> But I figure KDE's own docs on autostart stuff will give reason >> to how to control autostarting, and possibly even specifying >> exclusion terms. >> > > Thank you for your help. I will keep on searching. It is not big > deal to have rushtop autostarts, just I'd like to understand and know > how to control it. > > Have a nice weekend. > > Stephan. *** ***************************************************** *** Message #1862 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: autostarting in KDE From: Stephan Kosinski Date: Fri, 10 Jul 2009 14:15:35 -0400 Greg Ercolano wrote: > Starting a new thread for this.. > > Stephan Kosinski wrote: >> A quick question though since I propagated the new binary on my linux >> stations rushtop is auto-starting at user login. > > Hmm, nothing in rushtop or the rush installers does any autostart > type stuff.. at least not in my code. > > Possibly the GUI toolkit rushtop is using has changed some internals > that makes it more compatible with window manager autostart stuff > that it didn't have before. > > Do other tools autostart if you logout while they're running? > ie. firefox, konqueror, clocks, etc? Yes but only when the session has been saved with them running. That means at logout or when the user save is session manually > > Sounds like a feature of the window manager remembering what > was running when you logout, and starting them again when you > login. > > This stuff is likely saved in an ascii file in your ~/.kde directory, > but by what mechanism I don't know. > I'll check there >> What is reaaly weird is that I was never able to do that with previous >> binary even by saving the KDE session with rushtop running. > > Hmm, I don't know what could have changed in rushtop to do that. > Nothing in the code to rushtop itself.. I don't know of any way > to control it from the GUI toolkit's API. > >> Do you have any idea how to prevent rushtop to auto-start? Is it >> related to my system or to the new binary? > > Try closing rushtop *then* logging out, then back in. > I would assume it won't start then. First thing I tried. Close rushtop, save my session, logout and login again... Rushtop is stubbornly coming back . :) > > I stopped using KDE a while ago; too many memory hungry daemons and > menus kept moving around, and same to be said for gnome, so I > standardized on a custom 'tiny' window manager that has no daemons > and one menu that I defined. This way I can upgrade my workstation, > but have the interface stay the same. > > However, I'll try bringing up KDE on one of my other systems to see > if I can at least replicate. > > But I figure KDE's own docs on autostart stuff will give reason > to how to control autostarting, and possibly even specifying > exclusion terms. > Thank you for your help. I will keep on searching. It is not big deal to have rushtop autostarts, just I'd like to understand and know how to control it. Have a nice weekend. Stephan. *** ***************************************************** *** Message #1861 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Fri, 10 Jul 2009 14:04:45 -0400 Abraham Schneider wrote: > Works fine for me, thanks alot! > Abraham Great, thanks for the feedback + debugging Abraham + Stephan. If anyone has anything else to report with the new 07/08/2009 build date release of rushtop, let me know! -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1860 ***************************************************** *** ***************************************************** Subject: Rushtop: autostarting in KDE From: Greg Ercolano Date: Fri, 10 Jul 2009 13:41:30 -0400 Starting a new thread for this.. Stephan Kosinski wrote: > A quick question though since I propagated the new binary on my linux > stations rushtop is auto-starting at user login. Hmm, nothing in rushtop or the rush installers does any autostart type stuff.. at least not in my code. Possibly the GUI toolkit rushtop is using has changed some internals that makes it more compatible with window manager autostart stuff that it didn't have before. Do other tools autostart if you logout while they're running? ie. firefox, konqueror, clocks, etc? Sounds like a feature of the window manager remembering what was running when you logout, and starting them again when you login. This stuff is likely saved in an ascii file in your ~/.kde directory, but by what mechanism I don't know. > What is reaaly weird is that I was never able to do that with previous > binary even by saving the KDE session with rushtop running. Hmm, I don't know what could have changed in rushtop to do that. Nothing in the code to rushtop itself.. I don't know of any way to control it from the GUI toolkit's API. > Do you have any idea how to prevent rushtop to auto-start? Is it > related to my system or to the new binary? Try closing rushtop *then* logging out, then back in. I would assume it won't start then. I stopped using KDE a while ago; too many memory hungry daemons and menus kept moving around, and same to be said for gnome, so I standardized on a custom 'tiny' window manager that has no daemons and one menu that I defined. This way I can upgrade my workstation, but have the interface stay the same. However, I'll try bringing up KDE on one of my other systems to see if I can at least replicate. But I figure KDE's own docs on autostart stuff will give reason to how to control autostarting, and possibly even specifying exclusion terms. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1859 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Stephan Kosinski Date: Fri, 10 Jul 2009 13:10:39 -0400 Hello Greg, Thank you very much for the new version. It really is convenient!.. A quick question though since I propagated the new binary on my linux stations rushtop is auto-starting at user login. What is reaaly weird is that I was never able to do that with previous binary even by saving the KDE session with rushtop running. Do you have any idea how to prevent rushtop to auto-start? Is it related to my system or to the new binary? Thank you for your help and all your work. Stephan. Greg Ercolano wrote: > Abraham Schneider wrote: >> [..describes 'gray gap' problem and parenthesis disappearing..] > > OK, I've believe I've fixed both of those issues, and have posted > the new binaries here: > > http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort-07-08-2009.zip > > (Same download password as before) > > This version advertises itself with a build date of 07/08/2009 > in the 'rushtop -help' screen. > > Let me know if you encounter any further trouble. *** ***************************************************** *** Message #1858 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: "Abraham Schneider" Date: Fri, 10 Jul 2009 11:26:11 -0400 Works fine for me, thanks alot! Abraham Greg Ercolano schrieb: > [posted to rush.general] >=20 > Abraham Schneider wrote: >> [..describes 'gray gap' problem and parenthesis disappearing..] >=20 > OK, I've believe I've fixed both of those issues, and have posted > the new binaries here: >=20 > = http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort-07-= 08-2009.zip >=20 > (Same download password as before) >=20 > This version advertises itself with a build date of 07/08/2009 > in the 'rushtop -help' screen. >=20 > Let me know if you encounter any further trouble. -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Dr. Martin Prillmann; Thomas Till *** ***************************************************** *** Message #1857 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Wed, 08 Jul 2009 15:32:42 -0400 Abraham Schneider wrote: > [..describes 'gray gap' problem and parenthesis disappearing..] OK, I've believe I've fixed both of those issues, and have posted the new binaries here: http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort-07-08-2009.zip (Same download password as before) This version advertises itself with a build date of 07/08/2009 in the 'rushtop -help' screen. Let me know if you encounter any further trouble. *** ***************************************************** *** Message #1856 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Tue, 07 Jul 2009 12:55:24 -0400 Abraham Schneider wrote: > Sorry for not giving feedback sooner, it was a hard week here. Thanks > for doing this so fast and nice! Seems to work quite well. > > Only thing that isn't working for me with the new build is that after > restarting the app the number in parentheses which signals the core > count isn't showing anymore. With the old build it worked just fine. > (Talking about rushtop on a Mac Pro with 10.4). Strange thing is: if I > switch on the option the paranthesis appear, but the layout of the lines > is strange because I have a big grey gap after the first 5 lines. If I > quit and restart the app the gap is gone, but the parantheses as well. > Tell me if I should send you screenshots or test something. Thanks Abraham -- I'll look into those and will follow up here with fix binaries. > BTW: seems you have read my mind there, because after sending you the > original post I thought it would be nice to still see the amount of > cores somewhere. Then you have send the first build and it was already > in there :) Yes; I felt you had to see that info somehow.. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1855 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: "Abraham Schneider" Date: Tue, 07 Jul 2009 07:10:24 -0400 Hi Greg! Sorry for not giving feedback sooner, it was a hard week here. Thanks=20 for doing this so fast and nice! Seems to work quite well. Only thing that isn't working for me with the new build is that after=20 restarting the app the number in parentheses which signals the core=20 count isn't showing anymore. With the old build it worked just fine.=20 (Talking about rushtop on a Mac Pro with 10.4). Strange thing is: if I=20 switch on the option the paranthesis appear, but the layout of the lines = is strange because I have a big grey gap after the first 5 lines. If I=20 quit and restart the app the gap is gone, but the parantheses as well.=20 Tell me if I should send you screenshots or test something. BTW: seems you have read my mind there, because after sending you the=20 original post I thought it would be nice to still see the amount of=20 cores somewhere. Then you have send the first build and it was already=20 in there :) Thanks again and best regards, Abraham Greg Ercolano schrieb: > [posted to rush.general] >=20 > I have an update for the rushtop with the single cpu bar mode; >=20 > Dylan noticed that if "Use Decay" mode is on, all the bars = 'pulsate' > instead of bouncing smoothly. >=20 > I hadn't noticed this, because I had been testing with "Use Decay" > mode turned off (right click, choose Properties -> Use Decay to = toggle) > When disabled, the bars act normally. >=20 > Anyway, I just uploaded a new zip file with a fix for this: > = http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort-07-= 01-2009.zip >=20 > This new version advertises its build date as being '07/01/09' > when rushtop is invoked with the '-help' flag, eg: >=20 > rushtop -help > [..] > COPYRIGHT > Rush(R) Network Render Queue, V102.42b-prerelease, Build Date: = 07/01/09 > = ^^^^^^^^ >=20 >> Greg Ercolano wrote: >> Done -- give this a try: >> = http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort.zip= >> (Use your downloads password. If you don't know what it is, = contact me.) -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Dr. Martin Prillmann; Thomas Till *** ***************************************************** *** Message #1854 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Wed, 01 Jul 2009 04:23:23 -0400 I have an update for the rushtop with the single cpu bar mode; Dylan noticed that if "Use Decay" mode is on, all the bars 'pulsate' instead of bouncing smoothly. I hadn't noticed this, because I had been testing with "Use Decay" mode turned off (right click, choose Properties -> Use Decay to toggle) When disabled, the bars act normally. Anyway, I just uploaded a new zip file with a fix for this: http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort-07-01-2009.zip This new version advertises its build date as being '07/01/09' when rushtop is invoked with the '-help' flag, eg: rushtop -help [..] COPYRIGHT Rush(R) Network Render Queue, V102.42b-prerelease, Build Date: 07/01/09 ^^^^^^^^ > Greg Ercolano wrote: > Done -- give this a try: > http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort.zip > (Use your downloads password. If you don't know what it is, contact me.) *** ***************************************************** *** Message #1853 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Fri, 26 Jun 2009 20:44:43 -0400 Oh, and I should add how it works ;) Just right click on the new rushtop interface, and you should see: Preferences/Single Cpu Bar Per Host Also, if you run 'rushtop -help', you'll see the new "-single" and "-multi" command line flags to enable/disable this feature. eg: rushtop -single rushtop -multi Note that once turned on, rushtop remembers the setting for the next time rushtop is invoked (the setting is saved in the ~/.rushtoprc file). So if no flags are specified, the last setting is used. Note that when single mode is enabled, multiproc machines show the actual number of cpus in parens next to the hostname. There's also a new -nosort flag (which you didn't ask for, but another customer did) that disables sorting the hostnames in alphabetical order, so the hosts appear in rushtop in the order they appear in the rush/etc/hosts file, or in the order they appear on the command line, eg: rushtop -nosort zenith sony toshiba rca +farm If you have any trouble with the above, let me know. This version of rushtop SHOULD be compatible with recent versions of rush. If you have trouble, revert to the old rushtop (you did make a copy of the original rushtop, right? ;) > Greg Ercolano wrote: > Done -- give this a try: > http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort.zip > > 2009/6/26 Abraham Schneider : >>>> Is there any way to have Rushtop to only show one line per machine with >>>> a summary value for the CPU load instead of showing a bar for each individual core? -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1852 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Fri, 26 Jun 2009 20:36:33 -0400 Greg Ercolano wrote: > Sure, I can add that. Done -- give this a try: http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort.zip (Use your downloads password. If you don't know what it is, contact me.) That zip contains new rushtop executables for mac/linux/windows: mac/rushtop -- for all versions of OSX (intel/ppc) windows/rushtop.exe -- for all versions of windows fedora3/rushtop -- for most 32bit and 64bit versions of Linux ubuntu7-64/rushtop -- Use this if you've got a 64bit linux the fedora3 binary doesn't work with You can either use the install scripts in the zip file to install, or just copy the appropriate binary for your platform into the appropriate rush directory (rename out the old one first): WIN: c:\rush\bin\rushtop.exe MAC: /usr/local/rush/Applications/rushtop.app/Contents/MacOS/rushtop LINUX: /usr/local/rush/bin/rushtop Note that on the Mac the rushtop binary is NOT in rush/bin, it's in the .app directory shown above. The 'rushtop' you'll find in the rush/bin directory is a *script*, so don't replace that. Greg Ercolano wrote: > Sure, I can add that. > > I'll see if I can pull together some new 'rushtop' executables > that do this for you, and make it available on the website. 2009/6/26 Abraham Schneider : > >> Hi there! > >> > >> Is there any way to have Rushtop to only show one line per machine with > >> a summary value for the CPU load instead of showing a bar for each > >> individual core? -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1851 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Fri, 26 Jun 2009 10:28:22 -0400 Sure, I can add that. I'll see if I can pull together some new 'rushtop' executables that do this for you, and make it available on the website. Dylan Penhale wrote: > [posted to rush.general] > I would second this request. > > Dylan Penhale > > 2009/6/26 Abraham Schneider : >> Hi there! >> >> Is there any way to have Rushtop to only show one line per machine with >> a summary value for the CPU load instead of showing a bar for each >> individual core? -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1850 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Dylan Penhale Date: Fri, 26 Jun 2009 07:17:50 -0400 I would second this request. Dylan Penhale 2009/6/26 Abraham Schneider : > [posted to rush.general] > > Hi there! > > Is there any way to have Rushtop to only show one line per machine with > a summary value for the CPU load instead of showing a bar for each > individual core? > > I'd really like to see my whole renderfarm on a screen without scrolling > like hell. > > In earlier times with only one or two cores per machine it was fine. But > now with Nehalem-based Mac Pros with 16 virtual cores, it's hard to keep > track of the status of the whole farm. > > Thanks for any help and best regards, > > Abraham > > -- > Abraham Schneider Dylan Penhale 6 Bennabra Place Frenchs Forest NSW 2086 Australia Mobile: 0424655320 Email: dylan@(email surpressed) Web: www.his-best.com *** ***************************************************** *** Message #1849 ***************************************************** *** ***************************************************** Subject: Rushtop: view with cpu summary instead of each single core? From: "Abraham Schneider" Date: Fri, 26 Jun 2009 04:37:53 -0400 Hi there! Is there any way to have Rushtop to only show one line per machine with a summary value for the CPU load instead of showing a bar for each individual core? I'd really like to see my whole renderfarm on a screen without scrolling like hell. In earlier times with only one or two cores per machine it was fine. But now with Nehalem-based Mac Pros with 16 virtual cores, it's hard to keep track of the status of the whole farm. Thanks for any help and best regards, Abraham -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Dr. Martin Prillmann; Thomas Till *** ***************************************************** *** Message #1848 ***************************************************** *** ***************************************************** Subject: Re: -b maya flag [MAYA 2009 UPDATE: FLOATING POINT RENDERING WITH From: Greg Ercolano Date: Tue, 09 Jun 2009 03:38:01 -0400 Replying to this old thread to update regarding rendering floating point frame ranges with Maya 2009 and up. In Maya 2009 they dropped the -se/-be flags that the submit-maya script used to handle floating point frame numbers. So if you try to render floating point frame numbers with Maya 2009, you'll get an error about the '-se' flag being uknown. So if you're using Maya 2009 with floating point frames and getting that error, find this line in the submit-maya.pl script: $opt{MayaFlags} .= "-se $se -be $be "; ..and change it to read: $opt{MayaFlags} .= "-rfs $se -rfb $be "; ..and that should solve it. [THIS FIX IS IN RUSH 102.42a9b AND UP] Greg Ercolano wrote: > Gary Jaeger wrote: >> Is there any way using rush to use a step rate with maya renders? I >> commonly render out slow versions of animations as tests using -b .5, >> etc. That doesn't work with rush. > > The new submit-maya6 script in 102.42a6 supports both > floating point frame ranges, and stepped frame ranges > when batching. > > Did you upgrade to 102.42a6 already? > If not, contact me directly and I'll supply you the upgrade info, > which is free to all existing customers. > > If so, open the submit-maya6 interface, and click the "Help" > button at the bottom, and scroll to the section on > "Maya Floating Point Rendering", eg: > > Maya Floating Point Rendering > ----------------------------- > Here's an example of rendering with floating point frames. > Maya has the capability of rendering the frames 'between frames', > sometimes useful for motion blurring, or slowing the animation > of a sequence down by rendering images between frames: > > Job Title: -- (Blank uses scene name) > Scene Path: //path/scenes/sc17a.ma -- Scene file to render > Renderer: maya(sw) -- Use Maya's software renderer > Threads: 1 -- One thread for each frame > Frames: 10.0-50.0,0.2 -- Render 10.0 - 50.0 stepping at 0.2 > Batch Frames: 1 -- Renders 1 frame per machine > Cpus: +any=5 -- Use any 5 available cpus > > > This will create a job that generates a floating point frame list, > where Maya will write out 'integerized' frame numbers (with the > decimal points removed) that looks like eg: > > Rush Frame Output Image > ---------- ------------ > 0010.0 foo.0100.iff > 0010.2 foo.0102.iff > 0010.4 foo.0104.iff > : > 0049.6 foo.0496.iff > 0049.8 foo.0498.iff > 0050.0 foo.0500.iff > > > When 'batching' is used, several floating point frames will be > rendered at a time, eg: > > Frames: 10.0-50.0,0.2 -- Render 10.0 - 50.0 stepping at 0.2 > Batch Frames: 5 -- Renders 5 frame per machine > > > .this will create a "batched frames" job that generates a frame list > for every 5th floating point frame, each of which will render 5 frames > at a time on each machine: > > Rush Scene > Frame Frame Output Image > -------- _ ------- ------------- > | 10.0 foo.0100.iff > | 10.2 foo.0102.iff > 0010.0 --| 10.4 foo.0104.iff > | 10.6 foo.0106.iff > |_ 10.8 foo.0108.iff > _ > | 11.0 foo.0110.iff > | 11.2 foo.0112.iff > 0011.0 --| 11.4 foo.0114.iff > | 11.6 foo.0116.iff > |_ 11.8 foo.0118.iff > : > : _ > | 49.0 foo.0490.iff > | 49.2 foo.0492.iff > 0049.0 --| 49.4 foo.0494.iff > | 49.6 foo.0496.iff > |_ 49.8 foo.0498.iff > _ > 0050.0 --|_ 50.0 foo.0500.iff > > *** ***************************************************** *** Message #1847 ***************************************************** *** ***************************************************** Subject: [OSX/ADMIN] Boot script to mount multiple file servers From: Greg Ercolano Date: Wed, 03 Jun 2009 00:53:06 -0400 Sometimes companies have multiple file servers/shares they want to have mounted up on boot of their OSX machines. The following shows a boot script for OSX that uses subroutines to handle the details of mounting, to make it easier to manage multiple mounts, so that there's just one mount command per line, eg: Mount 192.168.1.2:/jobs /net/jobs "intr,bg,resvport,vers=3" Mount 192.168.1.3:/Volumes/Prod1 /net/prod1 "intr,bg,tcp,vers=3" Mount 192.168.1.3:/Volumes/Prod2 /net/prod2 "intr,bg,tcp,vers=3" Mount 192.168.1.3:/Volumes/Prod3 /net/prod3 "intr,bg,tcp,vers=3" This is better for multiple mounts than the previous examples I posted last year, where to mount multiple servers you needed to have a paragraph of code for each mount. (Use of IP addresses is recommended in place of hostnames, as DNS is often unstable under OSX during boot.) The following describes how to create and install this boot script which, once working, can be copied to all your OSX machines to ensure each boots up with the correct mounts. * * * 1) Login as root, create a boot script directory called "MountFileServers": mkdir -m 755 /Library/StartupItems/MountFileServers chown 0:0 /Library/StartupItems/MountFileServers It's important the perms are 755 and owned by root/wheel, as shown. 2) 'cd' into that directory and create a "StartupParameters.plist" file: with the perms carefully set to 644 and owned by root/wheel: cd /Library/StartupItems/MountFileServers touch StartupParameters.plist chown 0:0 StartupParameters.plist chmod 644 StartupParameters.plist vi StartupParameters.plist Here's what to paste into that file (leave out the 'snip' lines!). --------------------------------------------------------- snip { Description = "Mounts the File Servers"; Provides = ("MountFileServers"); Requires = ("Network", "System Log", "Resolver"); OrderPreference = "Last"; Messages = { start = "Mounting local file server"; stop = "Unmounting local file server"; }; } --------------------------------------------------------- snip 3) While you're in that same directory, create an executable script called "MountFileServers", with the perms 755, root/wheel: cd /Library/StartupItems/MountFileServers touch MountFileServers chown 0:0 MountFileServers chmod 755 MountFileServers vi MountFileServers What follows is the contents of the script. Be sure to customize the "Mount" and "Umount" lines to suit your environment. When specifying mounts, use IP addresses, not hostnames. eg: Mount helium:/jobs /net/jobs "intr,bg" # BAD ^^^^^^ Mount 192.168.0.2:/jobs /net/jobs "intr,bg" # GOOD ^^^^^^^^^^^ Here's the script: --------------------------------------------------------- snip #!/bin/sh ### ### Mount File Servers -- Set up local mounts for our file server ### ### Save this file as /Library/StartupItems/MountFileServers/MountFileServers ### and make sure the files/dirs are owned by root/wheel and modes are 755 ### (rwx-r-xr-x). ### ### 05/27/2009 erco@(email surpressed) ### # SUBROUTINE: MOUNT A REMOTE DRIVE # # NOTE: Use ip addresses in the remote device name, do NOT use hostnames. # This avoids assuming DNS fully operational during boot. # Often DNS is unstable during boot, even with boot dependencies. # Mount() { remdrive="$1" # $1 -- the remote drive, eg. "192.168.1.10:/jobs" localdir="$2" # $2 -- local mount point, eg. "/net/jobs" nfsflags="$3" # $3 -- nfs flags, eg. "rw,intr,bg,vers=3" # ENSURE MOUNT POINT EXISTS if [ ! -d "$localdir" ]; then $LOGGER "Creating mount point $localdir" mkdir -p -m 755 "$localdir" fi # ALREADY MOUNTED? DONT MOUNT TWICE if mount | grep -q "$localdir"; then $LOGGER "$localdir already mounted" else $LOGGER "Mounting '$remdrive' on '$localdir'" ( mount -t nfs -o "$nfsflags" "$remdrive" "$localdir" ) 2>&1 | $LOGGER fi } # SUBROUTINE: UNMOUNT THE SPECIFIED DRIVE Umount() { localdir="$1" # $1 -- local mount point, eg. "/net/jobs" $LOGGER "Un-mounting '$localdir'" umount "$localdir" 2>&1 | $LOGGER } ### ### MAIN ### if [ -x /etc/rc.common ]; then . /etc/rc.common; fi export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/libexec if [ "`tty`" = "not a tty" ]; then LOGGER="logger -t MountFileServers" else LOGGER="logger -s -t MountFileServers"; fi # Handle stop/start/restart ARG1=${1:-start} # assume 'start' if unset case "$ARG1" in start) ### MOUNT ALL SERVERS ON BOOT Mount 192.168.1.2:/jobs /net/jobs "intr,bg,resvport,vers=3" # LINUX Mount 192.168.1.3:/Volumes/Prod1 /net/prod1 "intr,bg,tcp,vers=3" # OSX/TCP Mount 192.168.1.3:/Volumes/Prod2 /net/prod2 "intr,bg,tcp,vers=3" # OSX/TCP Mount 192.168.1.3:/Volumes/Prod3 /net/prod3 "intr,bg,tcp,vers=3" # OSX/TCP Mount 192.168.1.3:/Volumes/STOCK /net/stock "intr,bg,tcp,vers=3" # OSX/TCP Mount 192.168.1.4:/Volumes/admin /net/admin "intr,bg,vers=3" # OSX Mount 192.168.1.4:/Volumes/personal /net/personal "intr,bg,vers=3" # OSX ;; stop) ### UN-MOUNT ALL SERVERS ON SHUTDOWN Umount /net/jobs Umount /net/prod1 Umount /net/prod2 Umount /net/prod3 Umount /net/stock Umount /net/admin Umount /net/personal ;; restart) $0 stop $0 start ;; *) echo "usage: $0 {start|stop|restart}" exit 1 ;; esac exit 0 --------------------------------------------------------- snip 4) That's it. You can test if the script works by running it as root from your terminal with the start/stop options. To test the script to see that it mounts properly: /Library/StartupItems/MountFileServers/MountFileServers start ..and to test that it unmounts properly, run: /Library/StartupItems/MountFileServers/MountFileServers stop After each command, run 'mount' to verify the mounts appear and disappear correctly. Once working, test by actually rebooting the machine, and when you get a login, check the /var/log/system.log to make sure there are no errors, eg: grep MountFileServers /var/log/system.log This is where any errors will be from your mount commands would appear, in case there's errors. Then you can install the boot script directory on the other OSX machines with a simple scp command, eg: scp -rp /Library/StartupItems/MountFileServers HOST1:/Library/StartupItems/MountFileServers scp -rp /Library/StartupItems/MountFileServers HOST2:/Library/StartupItems/MountFileServers ...etc.. Replace "HOST1" and "HOST2" with the remote hostnames. The above has been tested to work equally well on both Tiger and Leopard. *** ***************************************************** *** Message #1846 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Tue, 02 Jun 2009 14:47:29 -0400 Craig Allison wrote: > Ok, the retiming of the slate didn't work, Shake still insisted on > rendering the slate/grey over and over again as it rendered the quicktime. > > Have used to following to submit the two scripts from one submit: > > my $submit = <<"EOF"; > title $in{JobTitle} > frames 1 > command perl $G::self -render $QT_script_to_render > $in{Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} > $in{PrintEnvironment} $in{Frames} $in{Production} $in{OutputQuicktime} > $in{ShakeFlags} > $submitoptions > EOF > > my $submit2 = <<"EOF"; > title $in{JobTitle} > frames 1 > command perl $G::self -render $Slate_script_to_render > $in{Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} > $in{PrintEnvironment} $in{ShakeFlags} > $submitoptions > EOF > > # ErrorWindow("$submit\n"); # DEBUGGING > RushSubmit($submithost, $submit2, $in{JobTitle}, $in{StartIrush}); > exit(RushSubmit($submithost, $submit, $in{JobTitle}, $in{StartIrush})); > > This works fine, only that when the second job starts, the log for > $submit2 is overwritten as both jobs share the same directory, not such > a problem, but may try to iron this out over the coming days. This is because both submits are using the same $submitoptions: my $submit = <<"EOF"; .. $submitoptions .. EOF my $submit2 = <<"EOF"; .. $submitoptions .. EOF If you look at the contents of the $submitoptions variable, I think you'll understand why you're getting so many settings common between the Shake and QT jobs. In this case, the $submitoptions contains the 'logdir' and 'frames' command from the shake job, and passing that to both jobs is what's causing the overlap. ie. thats why the frame range and logdir are coming up the same for both. What you'll want to do is /not/ include the $submitoptions variable in the second job, and just set the frames/logdir commands you want, eg: ### CREATE LOGDIR FOR QT JOB my $qtlogdir = "${Slate_script_to_render}.log"; if ( ! -d $qtlogdir ) { mkdir($qtlogdir, 0777); if ( $G::iswindows ) { my $dir = $qtlogdir; $dir =~ s%/%\\%g; system("cacls $dirname /e /c /g everyone:f"); } } ### SUBMIT THE QT JOB my $submit2 = <<"EOF"; title $in{JobTitle}_QT frames 1 command perl $G::self -render $Slate_script_to_render .. logdir $qtlogdir EOF BTW, if you want to see what the first job's $submitoptions is set to, you can stick an 'ErrorWindow();' command in there just above the first submit, eg: ErrorWindow("SUBMITOPTIONS IS SET TO:\n$submitoptions\n"); my $submit = <<"EOF"; .. This way when you submit a test job, you'll see what the variable is set to, so you can then determine what you'll need to pass in the second job, in case there's more than just 'frames' and 'logdir'. > As an aside does anyone know a quick way of always resetting to defaults > for the QT render instead of Rush picking up the last settings every > time? I assume it's a deletion of a certain local file, but maybe I > could just reset certain fields which would work much better? I'm thinking my answers above should help with that. Basically its that same "$submitoptions" variable being passed to both jobs. Let us know if you have any other questions. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1845 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Craig Allison Date: Tue, 02 Jun 2009 06:24:41 -0400 --Apple-Mail-23--656878284 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Ok, the retiming of the slate didn't work, Shake still insisted on rendering the slate/grey over and over again as it rendered the quicktime. Have used to following to submit the two scripts from one submit: my $submit = <<"EOF"; title $in{JobTitle} frames 1 command perl $G::self -render $QT_script_to_render $in {Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} $in {PrintEnvironment} $in{Frames} $in{Production} $in{OutputQuicktime} $in{ShakeFlags} $submitoptions EOF my $submit2 = <<"EOF"; title $in{JobTitle} frames 1 command perl $G::self -render $Slate_script_to_render $in {Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} $in {PrintEnvironment} $in{ShakeFlags} $submitoptions EOF # ErrorWindow("$submit\n"); # DEBUGGING RushSubmit($submithost, $submit2, $in{JobTitle}, $in{StartIrush}); exit(RushSubmit($submithost, $submit, $in{JobTitle}, $in {StartIrush})); This works fine, only that when the second job starts, the log for $submit2 is overwritten as both jobs share the same directory, not such a problem, but may try to iron this out over the coming days. As an aside does anyone know a quick way of always resetting to defaults for the QT render instead of Rush picking up the last settings every time? I assume it's a deletion of a certain local file, but maybe I could just reset certain fields which would work much better? I can see that I'm going to run in to trouble with artists not choosing the correct production and frame range etc so would like to force them into a choice each time. I will get them to start using "forms" too, I hadn't noticed that nice little feature until I was developing this, will come in useful for artists who are always working on the same shots. C Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 27 May 2009, at 18:11, Victor DiMichina wrote: > You can try something like this, suited to your needs of course. > Note the eval command (thanks go to Greg again and again, of course) > > ################################ > #!/bin/csh -f > if ( ! $?RUSH_DIR ) setenv RUSH_DIR /usr/local/rush > source $RUSH_DIR/etc/.submit > eval `/usr/local/rush/bin/rush -submit` << EOF > title (your title) > ram 1 > frames (your frame range) > logdir (your log dir) > command perl //server/raid/2D_Apps/RushSubmits/Applications/ > submit-shake.app/Contents/MacOS/submit-shake-autolog.pl -render 1 1 > Fail /server/raid/path/to/someshakeproject.shk 0 -v -proxyscale 1 -v > cpus +farm=10@10 > notes (optional) > EOF > if ( $status ) exit 1 > # (eval eats the setenv command, so we duplicate it here) > echo "setenv RUSH_JOBID $RUSH_JOBID" > echo $RUSH_JOBID > /tmp/curr_jobid.txt > # Job #2 - > /usr/local/rush/bin/rush -submit << EOF > title (your 2nd title) > ram 1 > frames (frame range) > logdir (your log dir) > command (another command) > cpus +farm=11@980 > waitfor $RUSH_JOBID > waitforstate done > notes (optional) > EOF > #################################################### > > > HTH > Vic > > > On Wed, May 27, 2009 at 7:54 AM, Craig Allison > wrote: > Ok, so I've finally got around to looking into this and have > decided to change my original way of thinking and give the artists > the opportunity to render a quicktime on the farm once they are > happy with the results previewed in Framecycler. > > I have taken the submit-shake-quicktime script and amended it to > allow users to select which film they are working on, select the > first frame in the sequence to render and dialogue for slate notes > etc - the combination of all this information chooses the correct > shake setup script on the network, replaces the variables, saves a > copy and submits it to the farm. > > The script renders a 2K slate & greyscale (named and numbered > correctly) and also a QT with the appropriate settings for the > production. I have it working fine but now want to speed it up by > splitting the job into two, i.e. one script to render the two 2k > frames and one script to render the QT. > > I've got rush creating the 2 distinct shake scripts to render, but > don't know how to get 2 scripts to render with a single rush > submit, is there an easy way? > > Cheers > > Craig > > Craig Allison > Digital Systems & Data Manager > The Senate Visual Effects > Twickenham Film Studios > St.Margarets > Twickenham > TW1 2AW > > t: 0208 607 8866 > skype: craig_9000 > www.senatevfx.com > > > > On 19 Dec 2008, at 09:20, Abraham Schneider wrote: > >> [posted to rush.general] >> >> Greg Ercolano schrieb: >>> Sounds interesting, but unless they have a non-gui interface, >>> it's not something I think Rush can control properly. >> >> yes, that's right. I'm doing it in two steps at the moment: rush=20 >> executes a jobdonecommand script which creates an entry with the >> path to = >> >> the image sequence in a queue folder. A small python script on my >> QT=20 >> creation Mac is watching this folder and does the actual 'feeding' >> of=20 >> the Silverstack rendering app (called Cinemator). >> >> That's not a perfect solution but it works quite well for us at >> the=20 >> moment. And with the template files it is easy to do different >> burnins,=20 >> codecs, framerates, etc. depending on project or user choose. >> >> Abraham >> >> -- >> Abraham Schneider >> Senior VFX Compositor, 2 D Artist >> ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = >> Germany >> Phone (Tel# suppressed) EMail aschneider@arri.de >> www.arri.de/filmtv >> ARRI Film & TV Services GmbH >> Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = >> Handelsregisternummer: HRB 69396 >> Gesch=E4ftsf=FChrer: Franz Kraus; Thomas Till >> > > > > > -- > Victor DiMichina > Pixel Magic > 818.760.0862 --Apple-Mail-23--656878284 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Ok, the retiming of the slate didn't work, Shake still insisted on = rendering the slate/grey over and over again as it rendered the = quicktime.

Have = used to following to submit the two scripts from one = submit:

my $submit =3D = <<"EOF";
title =A0 =A0 =A0 = =A0$in{JobTitle}
frames =A0 =A0 =A0 =A0 =A01
command = =A0 =A0 =A0 =A0 perl $G::self -render $QT_script_to_render = $in{Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} = $in{PrintEnvironment} $in{Frames} $in{Production} $in{OutputQuicktime} = $in{ShakeFlags}=A0
$submitoptions
EOF

my = $submit2 =3D <<"EOF";
title =A0 =A0 =A0 = =A0$in{JobTitle}
frames =A0 =A0 =A0 =A0 =A01
command = =A0 =A0 =A0 =A0 perl $G::self -render $Slate_script_to_render = $in{Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} = $in{PrintEnvironment} = $in{ShakeFlags}=A0
$submitoptions
EOF
=
=A0# = ErrorWindow("$submit\n"); # DEBUGGING
=A0=A0 = =A0RushSubmit($submithost, $submit2, $in{JobTitle}, = $in{StartIrush});
=A0=A0 =A0exit(RushSubmit($submithost, = $submit, $in{JobTitle}, $in{StartIrush}));



This works fine, only that when = the second job starts, the log for $submit2 is overwritten as both jobs = share the same directory, not such a problem, but may try to iron this = out over the coming days.


As an aside does anyone know a quick way of always resetting to = defaults for the QT render instead of Rush picking up the last settings = every time? I assume it's a deletion of a certain local file, but maybe = I could just reset certain fields which would work much better? =A0I can = see that I'm going to run in to trouble with artists not choosing the = correct production and frame range etc so would like to force them into = a choice each time. =A0I will get them to start using "forms" too, I = hadn't noticed that nice little feature until I was developing this, = will come in useful for artists who are always working on the same = shots.


C


Craig Allison

Digital Systems & Data=A0Manager

The Senate Visual = Effects

St.Margarets

Twickenham

TW1 2AW


t: 0208 607 8866=A0

skype: craig_9000


=


On 27 May 2009, at 18:11, Victor DiMichina = wrote:

You can try something like this,=A0 suited to your needs = of course.=A0=A0 Note the eval command (thanks go=A0 to Greg again and = again,=A0 of = course)

################################
#!/bin/csh -f
if ( = ! $?RUSH_DIR ) setenv RUSH_DIR /usr/local/rush
source = $RUSH_DIR/etc/.submit
eval `/usr/local/rush/bin/rush -submit` = << EOF
title=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your = title)
ram=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1
frames=A0=A0=A0=A0=A0= =A0=A0=A0=A0 (your frame range)
logdir=A0=A0=A0=A0=A0=A0=A0=A0=A0 = (your log dir)
command=A0=A0=A0=A0=A0=A0=A0 perl = //server/raid/2D_Apps/RushSubmits/Applications/submit-shake.app/Contents/M= acOS/submit-shake-autolog.pl -render 1 1 Fail=A0 = /server/raid/path/to/someshakeproject.shk=A0 0 -v -proxyscale 1 -v
= cpus=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 +farm=3D10@10
notes=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 (optional)
EOF
if ( $status ) exit 1
# (eval = eats the setenv command, so we duplicate it here)
echo "setenv = RUSH_JOBID $RUSH_JOBID"
echo $RUSH_JOBID > /tmp/curr_jobid.txt
= # Job #2 -
/usr/local/rush/bin/rush -submit << = EOF
title=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your 2nd = title)
ram=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1
frames=A0=A0=A0=A0=A0= =A0=A0=A0=A0 (frame range)
logdir=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your = log dir)
command=A0=A0=A0=A0=A0=A0=A0=A0 (another command)
= cpus=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 +farm=3D11@980
waitfor=A0=A0=A0=A0= =A0=A0=A0=A0 $RUSH_JOBID
waitforstate=A0=A0=A0 = done
notes=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = (optional)
EOF
####################################################<= br>

HTH
Vic


On Wed, May = 27, 2009 at 7:54 AM, Craig Allison <craig@(email surpressed)> = wrote:
Ok, so I've finally got around to looking into = this and have decided to change my original way of thinking and give the = artists the opportunity to render a quicktime on the farm once they are = happy with the results previewed in Framecycler.

I = have taken the submit-shake-quicktime script and amended it to allow = users to select which film they are working on, select the first frame = in the sequence to render and dialogue for slate notes etc - the = combination of all this information chooses the correct shake setup = script on the network, replaces the variables, saves a copy and submits = it to the farm.

The script renders a 2K slate = & greyscale (named and numbered correctly) and also a QT with the = appropriate settings for the production. =A0I have it working fine but = now want to speed it up by splitting the job into two, i.e. one script = to render the two 2k frames and one script to render the QT.
=

I've got rush creating the 2 distinct shake scripts = to render, but don't know how to get 2 scripts to render with a single = rush submit, is there an easy way?

Cheers
=

Craig

Craig Allison
Digital Systems = & Data=A0Manager
The Senate Visual Effects
Twickenham Film = Studios
St.Margarets
Twickenham
TW1 = 2AW

t: 0208 607 = 8866=A0
skype: = craig_9000


=

On 19 Dec 2008, at 09:20, Abraham Schneider = wrote:

[posted to rush.general]

Greg Ercolano = schrieb:
Sounds = interesting, but unless they have a non-gui interface,
= it's not something I think Rush can control properly.
=

yes, that's right. I'm = doing it in two steps at the moment: rush=3D20
executes a jobdonecommand script which creates an entry with the = path to =3D

the image sequence in a = queue folder. A small python script on my QT=3D20
creation Mac is watching this folder and does the = actual 'feeding' of=3D20
the = Silverstack rendering app (called Cinemator).

That's not = a perfect solution but it works quite well for us at the=3D20
moment. And with the template files it is easy to = do different burnins,=3D20
codecs, = framerates, etc. depending on project or user choose.

Abraham

--
Abraham Schneider
Senior VFX = Compositor, 2 D Artist
ARRI Film & = TV Services GmbH =A0 Tuerkenstr. 89 =A0 = D-80799 Muenchen / =3D
Germany
Phone +49 89 3809-1269 = =A0 =A0 EMail aschneider@arri.de
ARRI Film & TV Services GmbH
Sitz: M=3DFCnchen=A0 -=A0 = Registergericht: Amtsgericht M=3DFCnchen=A0 -=A0= =3D
Handelsregisternummer: HRB = 69396
Gesch=3DE4ftsf=3DFChrer: Franz = Kraus; Thomas Till


=



-- =
Victor DiMichina
Pixel = Magic
818.760.0862

= --Apple-Mail-23--656878284-- *** ***************************************************** *** Message #1844 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Craig Allison Date: Thu, 28 May 2009 04:53:53 -0400 --Apple-Mail-13-1053132520 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hey Greg The way my Shake script works is that it renders the QT without the need for the slate and grey to be generated as 2k, it files them in from a set location and adds the relevant text and overlays a mask with counter and shot details, plus it files in the same "empty" slate, adds the same text and has another fileout to render the single 2k frame. The greyscale that is added to the QT has a branched fileout to also render a single 2k grey at the end. The problem I have observed is that the 2k slate renders itself over and over again until the script reaches the end of it's frame count, i.e. for a 200 frame QT, the slate will render 200 times, hence the need for splitting up the job so that I can give the slate and grey globals of 1 and the QT 1-200 for example. I'm going to look into using an offset for the slate/grey to see if this addresses this issue, otherwise I'll look at Victor's suggestion. Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 27 May 2009, at 22:21, Greg Ercolano wrote: > [posted to rush.general] > > Craig Allison wrote: >> Ok, so I've finally got around to looking into this and have >> decided to >> change my original way of thinking and give the artists the >> opportunity >> to render a quicktime on the farm once they are happy with the >> results >> previewed in Framecycler. >> >> I have taken the submit-shake-quicktime script and amended it to >> allow >> users to select which film they are working on, select the first >> frame >> in the sequence to render and dialogue for slate notes etc - the >> combination of all this information chooses the correct shake setup >> script on the network, replaces the variables, saves a copy and >> submits >> it to the farm. >> >> The script renders a 2K slate & greyscale (named and numbered >> correctly) >> and also a QT with the appropriate settings for the production. I >> have >> it working fine but now want to speed it up by splitting the job into >> two, i.e. one script to render the two 2k frames and one script to >> render the QT. > > Sounds cool > > But does it really take that long to generate the 2K slate and > greyscale to make it worth while to have those happen in separate > executions? > > I would assume that the quicktime gen can't happen until the > 2K slate/grayscale are done anyway, so even if another box > were working on the slate, the qt part would still have to > wait for it to get done. > > But if there's something like 50 frames of slate, then I guess > it pays to have each frame render on a separate machine, > and then have the qt gen run as a 'jobdonecommand', so that > it doesn't run until all the slate frames are finished. > > Thing is, if what you're doing is generating just one slate > image and one grayscale, but need those to hold for eg. 25x each, > then just have the render create one image each, then make symbolic > links for all the others; that'll save on disk space and render time, > so that the qt gen can refer to the links. > > For instance, the script could render these two images: > > slate-myshow-myshot.dpx > grayscale-myshow-myshot.dbx > > and then create symlinks for the frame range, eg: > > # GRAYSCALE > ln grayscale-myshow-myshot.dpx myshow-shot-frame-0001.dpx > ln grayscale-myshow-myshot.dpx myshow-shot-frame-0002.dpx > ln grayscale-myshow-myshot.dpx myshow-shot-frame-0003.dpx > : > ln grayscale-myshow-myshot.dpx myshow-shot-frame-0025.dpx > # SLATE > ln slate-myshow-myshot.dpx myshow-shot-frame-0026.dpx > ln slate-myshow-myshot.dpx myshow-shot-frame-0027.dpx > ln slate-myshow-myshot.dpx myshow-shot-frame-0028.dpx > : > ln slate-myshow-myshot.dpx myshow-shot-frame-0050.dpx > > That way you can 'instantly' make all the frames from just > the single 2k image, so that the quicktime can insert them > into the shot as needed. > > HTH. > > -- > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) > --Apple-Mail-13-1053132520 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hey Greg

The way = my Shake script works is that it renders the QT without the need for the = slate and grey to be generated as 2k, it files them in from a set = location and adds the relevant text and overlays a mask with counter and = shot details, plus it files in the same "empty" slate, adds the same = text and has another fileout to render the single 2k frame. =A0The = greyscale that is added to the QT has a branched fileout to also render = a single 2k grey at the end.

The problem I have = observed is that the 2k slate renders itself over and over again until = the script reaches the end of it's frame count, i.e. for a 200 frame QT, = the slate will render 200 times, hence the need for splitting up the job = so that I can give the slate and grey globals of 1 and the QT 1-200 for = example.

I'm = going to look into using an offset for the slate/grey to see if this = addresses this issue, otherwise I'll look at Victor's = suggestion.

Cheers

Craig



Craig Allison

Digital Systems & = Data=A0Manager

Twickenham Film Studios

St.Margarets

Twickenham

TW1 2AW


=A0

skype: craig_9000


=


On 27 May 2009, at 22:21, Greg Ercolano = wrote:

[posted to = rush.general]

Craig Allison wrote:
Ok, so I've finally got around = to looking into this and have decided to
change = my original way of thinking and give the artists the = opportunity
to render a quicktime on the = farm once they are happy with the results

I have taken the = submit-shake-quicktime script and amended it to allow
users to select which film they are working on, = select the first frame
in the sequence to render = and dialogue for slate notes etc - the
script on the network, replaces = the variables, saves a copy and submits
it to = the farm.

The script renders a 2K slate & greyscale (named = and numbered correctly)
and also a QT = with the appropriate settings for the production.=A0 I have
it working fine but now want to speed it up by = splitting the job into
two, i.e. one script to = render the two 2k frames and one script to
render = the QT.
Sounds cool

But does = it really take that long to generate the 2K slate and
greyscale to make it worth while = to have those happen in separate
= executions?
I would assume that the quicktime = gen can't happen until the
2K = slate/grayscale are done anyway, so even if another box
were working on the slate, the qt = part would still have to
wait for = it to get done.

But if there's something like 50 = frames of slate, then I guess
it pays = to have each frame render on a separate machine,
and then have the qt gen run as a = 'jobdonecommand', so that
it = doesn't run until all the slate frames are finished.

Thing is, = if what you're doing is generating just one slate
image and one grayscale, but need = those to hold for eg. 25x each,
then just = have the render create one image each, then make symbolic
links for all the others; that'll = save on disk space and render time,
so that = the qt gen can refer to the links.

For instance, the script could = render these two images:


and then create symlinks for the = frame range, eg:

# GRAYSCALE
ln = grayscale-myshow-myshot.dpx myshow-shot-frame-0001.dpx
ln grayscale-myshow-myshot.dpx = myshow-shot-frame-0002.dpx
ln = grayscale-myshow-myshot.dpx myshow-shot-frame-0003.dpx
:
ln = grayscale-myshow-myshot.dpx myshow-shot-frame-0025.dpx
# SLATE
ln = slate-myshow-myshot.dpx =A0 =A0 = myshow-shot-frame-0026.dpx
ln = slate-myshow-myshot.dpx =A0 =A0 = myshow-shot-frame-0027.dpx
ln = slate-myshow-myshot.dpx =A0 =A0 = myshow-shot-frame-0028.dpx
:
ln slate-myshow-myshot.dpx =A0 =A0 = myshow-shot-frame-0050.dpx

That way you can 'instantly' make = all the frames from just
the = single 2k image, so that the quicktime can insert them
into the shot as = needed.

HTH.

--=A0
Greg = Ercolano, erco@(email surpressed)
Seriss Corporation
Rush = Render Queue, http://seriss.com/rush/
Tel: 626-795-5922x23
Fax: = 626-795-5947
Cel: 310-266-8906

=

= --Apple-Mail-13-1053132520-- *** ***************************************************** *** Message #1843 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Wed, 27 May 2009 17:20:59 -0400 Craig Allison wrote: > Ok, so I've finally got around to looking into this and have decided to > change my original way of thinking and give the artists the opportunity > to render a quicktime on the farm once they are happy with the results > previewed in Framecycler. > > I have taken the submit-shake-quicktime script and amended it to allow > users to select which film they are working on, select the first frame > in the sequence to render and dialogue for slate notes etc - the > combination of all this information chooses the correct shake setup > script on the network, replaces the variables, saves a copy and submits > it to the farm. > > The script renders a 2K slate & greyscale (named and numbered correctly) > and also a QT with the appropriate settings for the production. I have > it working fine but now want to speed it up by splitting the job into > two, i.e. one script to render the two 2k frames and one script to > render the QT. Sounds cool But does it really take that long to generate the 2K slate and greyscale to make it worth while to have those happen in separate executions? I would assume that the quicktime gen can't happen until the 2K slate/grayscale are done anyway, so even if another box were working on the slate, the qt part would still have to wait for it to get done. But if there's something like 50 frames of slate, then I guess it pays to have each frame render on a separate machine, and then have the qt gen run as a 'jobdonecommand', so that it doesn't run until all the slate frames are finished. Thing is, if what you're doing is generating just one slate image and one grayscale, but need those to hold for eg. 25x each, then just have the render create one image each, then make symbolic links for all the others; that'll save on disk space and render time, so that the qt gen can refer to the links. For instance, the script could render these two images: slate-myshow-myshot.dpx grayscale-myshow-myshot.dbx and then create symlinks for the frame range, eg: # GRAYSCALE ln grayscale-myshow-myshot.dpx myshow-shot-frame-0001.dpx ln grayscale-myshow-myshot.dpx myshow-shot-frame-0002.dpx ln grayscale-myshow-myshot.dpx myshow-shot-frame-0003.dpx : ln grayscale-myshow-myshot.dpx myshow-shot-frame-0025.dpx # SLATE ln slate-myshow-myshot.dpx myshow-shot-frame-0026.dpx ln slate-myshow-myshot.dpx myshow-shot-frame-0027.dpx ln slate-myshow-myshot.dpx myshow-shot-frame-0028.dpx : ln slate-myshow-myshot.dpx myshow-shot-frame-0050.dpx That way you can 'instantly' make all the frames from just the single 2k image, so that the quicktime can insert them into the shot as needed. HTH. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1842 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Victor DiMichina Date: Wed, 27 May 2009 13:11:57 -0400 --001636e0aedbbae7e5046ae7f049 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit You can try something like this, suited to your needs of course. Note the eval command (thanks go to Greg again and again, of course) ################################ #!/bin/csh -f if ( ! $?RUSH_DIR ) setenv RUSH_DIR /usr/local/rush source $RUSH_DIR/etc/.submit eval `/usr/local/rush/bin/rush -submit` << EOF title (your title) ram 1 frames (your frame range) logdir (your log dir) command perl //server/raid/2D_Apps/RushSubmits/Applications/submit-shake.app/Contents/MacOS/submit-shake-autolog.pl -render 1 1 Fail /server/raid/path/to/someshakeproject.shk 0 -v -proxyscale 1 -v cpus +farm=10@10 notes (optional) EOF if ( $status ) exit 1 # (eval eats the setenv command, so we duplicate it here) echo "setenv RUSH_JOBID $RUSH_JOBID" echo $RUSH_JOBID > /tmp/curr_jobid.txt # Job #2 - /usr/local/rush/bin/rush -submit << EOF title (your 2nd title) ram 1 frames (frame range) logdir (your log dir) command (another command) cpus +farm=11@980 waitfor $RUSH_JOBID waitforstate done notes (optional) EOF #################################################### HTH Vic On Wed, May 27, 2009 at 7:54 AM, Craig Allison wrote: > Ok, so I've finally got around to looking into this and have decided to > change my original way of thinking and give the artists the opportunity to > render a quicktime on the farm once they are happy with the results > previewed in Framecycler. > I have taken the submit-shake-quicktime script and amended it to allow > users to select which film they are working on, select the first frame in > the sequence to render and dialogue for slate notes etc - the combination of > all this information chooses the correct shake setup script on the network, > replaces the variables, saves a copy and submits it to the farm. > > The script renders a 2K slate & greyscale (named and numbered correctly) > and also a QT with the appropriate settings for the production. I have it > working fine but now want to speed it up by splitting the job into two, i.e. > one script to render the two 2k frames and one script to render the QT. > > I've got rush creating the 2 distinct shake scripts to render, but don't > know how to get 2 scripts to render with a single rush submit, is there an > easy way? > > Cheers > > Craig > > > Craig Allison > > Digital Systems & Data Manager > > The Senate Visual Effects > > Twickenham Film Studios > > St.Margarets > > Twickenham > > TW1 2AW > > > t: 0208 607 8866 > > skype: craig_9000 > > www.senatevfx.com > > > > On 19 Dec 2008, at 09:20, Abraham Schneider wrote: > > [posted to rush.general] > > Greg Ercolano schrieb: > > Sounds interesting, but unless they have a non-gui interface, > it's not something I think Rush can control properly. > > > yes, that's right. I'm doing it in two steps at the moment: rush=20 > executes a jobdonecommand script which creates an entry with the path to = > > the image sequence in a queue folder. A small python script on my QT=20 > creation Mac is watching this folder and does the actual 'feeding' of=20 > the Silverstack rendering app (called Cinemator). > > That's not a perfect solution but it works quite well for us at the=20 > moment. And with the template files it is easy to do different burnins,=20 > codecs, framerates, etc. depending on project or user choose. > > Abraham > > -- > Abraham Schneider > Senior VFX Compositor, 2 D Artist > ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = > Germany > Phone (Tel# suppressed) EMail aschneider@arri.de > www.arri.de/filmtv > ARRI Film & TV Services GmbH > Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = > Handelsregisternummer: HRB 69396 > Gesch=E4ftsf=FChrer: Franz Kraus; Thomas Till > > > -- Victor DiMichina Pixel Magic (Tel# suppressed) --001636e0aedbbae7e5046ae7f049 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable You can try something like this,=A0 suited to your needs of course.=A0=A0 N= ote the eval command (thanks go=A0 to Greg again and again,=A0 of course)
################################
#!/bin/csh -f
if ( ! $?RUSH_DI= R ) setenv RUSH_DIR /usr/local/rush
source $RUSH_DIR/etc/.submit
eval `/usr/local/rush/bin/rush -submit` <= ;< EOF
title=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your title)
ram=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1
frames=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your= frame range)
logdir=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your log dir)
comman= d=A0=A0=A0=A0=A0=A0=A0 perl //server/raid/2D_Apps/RushSubmits/Applications/= submit-shake.app/Contents/MacOS/submit-shake-autolog.pl -render 1 1 Fail=A0= /server/raid/path/to/someshakeproject.shk=A0 0 -v -proxyscale 1 -v
cpus=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 +farm=3D10@10
notes=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 (optional)
EOF
if ( $status ) exit 1
# (eval eats = the setenv command, so we duplicate it here)
echo "setenv RUSH_JOBI= D $RUSH_JOBID"
echo $RUSH_JOBID > /tmp/curr_jobid.txt
# Job #2 -
/usr/local/rush/bin/rush -submit << EOF
title=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 (your 2nd title)
ram=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 1
frames=A0=A0=A0=A0=A0=A0=A0=A0=A0 (frame range)
logdir=A0= =A0=A0=A0=A0=A0=A0=A0=A0 (your log dir)
command=A0=A0=A0=A0=A0=A0=A0=A0 = (another command)
cpus=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 +farm=3D11@980
waitfor=A0=A0=A0=A0= =A0=A0=A0=A0 $RUSH_JOBID
waitforstate=A0=A0=A0 done
notes=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 (optional)
EOF
###################################= #################


HTH
Vic


On Wed, May 27, 2009 at 7:54 AM, Craig Allison <craig@(email surpressed)> wrote:<= br>
Ok, so I've finally got around to looking into this and have decided to= change my original way of thinking and give the artists the opportunity to= render a quicktime on the farm once they are happy with the results previe= wed in Framecycler.

I have taken the submit-shake-quicktime script and amended i= t to allow users to select which film they are working on, select the first= frame in the sequence to render and dialogue for slate notes etc - the com= bination of all this information chooses the correct shake setup script on = the network, replaces the variables, saves a copy and submits it to the far= m.

The script renders a 2K slate & greyscale (named an= d numbered correctly) and also a QT with the appropriate settings for the p= roduction. =A0I have it working fine but now want to speed it up by splitti= ng the job into two, i.e. one script to render the two 2k frames and one sc= ript to render the QT.

I've got rush creating the 2 distinct shake scripts= to render, but don't know how to get 2 scripts to render with a single= rush submit, is there an easy way?

Cheers

Craig


Craig= Allison

Digital Systems & Data=A0Manager

The Senate Visual Effects

Twickenham Fi= lm Studios

St.Margarets

Twickenham

TW1 2AW


t: 0208 607 8866=A0

skype: craig_9000

www.senatevfx.com<= /p>


On 19 Dec 2008, at 09:20, Abraham Schneider wro= te:

[posted = to rush.general]

Greg Ercolano schrieb:
Sounds interesting, but u= nless they have a non-gui interface,
it's not something I think Rush ca= n control properly.

yes, that's right. I'm doing it in two steps= at the moment: rush=3D20
executes a jobdo= necommand script which creates an entry with the path to =3D

the image sequence in a queue folder. A small python script on my = QT=3D20
creation Mac is watching this fold= er and does the actual 'feeding' of=3D20
the Silverstack rendering app (called Cinemator= ).

That's not a perfect solution but it works quite well= for us at the=3D20
moment. And with the template files it is easy = to do different burnins,=3D20
codecs, fram= erates, etc. depending on project or user choose.

Abraham

--
Abraham Schneider
Sen= ior VFX Compositor, 2 D Artist
ARRI Film & TV Services GmbH =A0 Tuerkenstr. 89 =A0 D-80799 Muenchen / =3D
Germany
Phone +49 89 380= 9-1269 =A0 =A0 EMail aschneider@arri.de
ARRI Film = & TV Services GmbH
Sitz: M=3DFCnchen=A0 -=A0 Registergericht: Amtsgericht M=3DFCnchen=A0 -=A0 =3D
Handelsregisternummer: HRB 69396
Gesch=3DE4ftsf=3DFChrer: Franz Kraus; Thomas Till
<= div style=3D"margin: 0px; min-height: 14px;">
=



--
Victo= r DiMichina
Pixel Magic
818.760.0862
--001636e0aedbbae7e5046ae7f049-- *** ***************************************************** *** Message #1841 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Craig Allison Date: Wed, 27 May 2009 10:54:49 -0400 --Apple-Mail-3-988392269 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Ok, so I've finally got around to looking into this and have decided to change my original way of thinking and give the artists the opportunity to render a quicktime on the farm once they are happy with the results previewed in Framecycler. I have taken the submit-shake-quicktime script and amended it to allow users to select which film they are working on, select the first frame in the sequence to render and dialogue for slate notes etc - the combination of all this information chooses the correct shake setup script on the network, replaces the variables, saves a copy and submits it to the farm. The script renders a 2K slate & greyscale (named and numbered correctly) and also a QT with the appropriate settings for the production. I have it working fine but now want to speed it up by splitting the job into two, i.e. one script to render the two 2k frames and one script to render the QT. I've got rush creating the 2 distinct shake scripts to render, but don't know how to get 2 scripts to render with a single rush submit, is there an easy way? Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 19 Dec 2008, at 09:20, Abraham Schneider wrote: > [posted to rush.general] > > Greg Ercolano schrieb: >> Sounds interesting, but unless they have a non-gui interface, >> it's not something I think Rush can control properly. > > yes, that's right. I'm doing it in two steps at the moment: rush=20 > executes a jobdonecommand script which creates an entry with the > path to = > > the image sequence in a queue folder. A small python script on my > QT=20 > creation Mac is watching this folder and does the actual 'feeding' > of=20 > the Silverstack rendering app (called Cinemator). > > That's not a perfect solution but it works quite well for us at the=20 > moment. And with the template files it is easy to do different > burnins,=20 > codecs, framerates, etc. depending on project or user choose. > > Abraham > > -- > Abraham Schneider > Senior VFX Compositor, 2 D Artist > ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = > Germany > Phone (Tel# suppressed) EMail aschneider@arri.de > www.arri.de/filmtv > ARRI Film & TV Services GmbH > Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = > Handelsregisternummer: HRB 69396 > Gesch=E4ftsf=FChrer: Franz Kraus; Thomas Till > --Apple-Mail-3-988392269 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Ok, so I've finally got around to looking into this and have decided to = change my original way of thinking and give the artists the opportunity = to render a quicktime on the farm once they are happy with the results = previewed in Framecycler.

I have taken the = submit-shake-quicktime script and amended it to allow users to select = which film they are working on, select the first frame in the sequence = to render and dialogue for slate notes etc - the combination of all this = information chooses the correct shake setup script on the network, = replaces the variables, saves a copy and submits it to the = farm.

The = script renders a 2K slate & greyscale (named and numbered correctly) = and also a QT with the appropriate settings for the production. =A0I = have it working fine but now want to speed it up by splitting the job = into two, i.e. one script to render the two 2k frames and one script to = render the QT.

I've got rush creating the = 2 distinct shake scripts to render, but don't know how to get 2 scripts = to render with a single rush submit, is there an easy way?

Cheers

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


On 19 Dec 2008, at 09:20, Abraham Schneider = wrote:

[posted to = rush.general]

Greg Ercolano schrieb:
Sounds interesting, but unless = they have a non-gui interface,
it's not = something I think Rush can control properly.

yes, = that's right. I'm doing it in two steps at the moment: rush=3D20
executes a jobdonecommand script which creates an = entry with the path to =3D

the image sequence in a queue = folder. A small python script on my QT=3D20
the Silverstack rendering app = (called Cinemator).
That's not a perfect solution = but it works quite well for us at the=3D20
moment. = And with the template files it is easy to do different = burnins,=3D20
codecs, framerates, etc. = depending on project or user choose.

Abraham

Abraham Schneider
Senior VFX Compositor, 2 D Artist
ARRI Film & TV Services GmbH =A0 Tuerkenstr. 89 =A0 D-80799 Muenchen / = =3D
Germany
Phone +49 89 3809-1269 =A0 =A0 EMail aschneider@arri.de
www.arri.de/filmtv
ARRI = Film & TV Services GmbH
Sitz: = M=3DFCnchen=A0 -=A0 Registergericht: Amtsgericht = M=3DFCnchen=A0 -=A0 =3D
Handelsregisternummer: HRB 69396
Gesch=3DE4ftsf=3DFChrer: Franz Kraus; Thomas = Till

=

= --Apple-Mail-3-988392269-- *** ***************************************************** *** Message #1840 ***************************************************** *** ***************************************************** Subject: Freelance Systems Admins and Render pipeline scripting, plug-ins From: Greg Ercolano Date: Wed, 13 May 2009 20:56:38 -0400 If you're looking for an available render pipeline scripting programmer, talk to Jorg Mohnen: 310-437-3614, jorg(at)mohnen(dot)de I've added him to my 'list' of freelance sysadmins/pipeline programmers (which I can send Rush customers via private mail) He's based here in LA, has worked for a lot of shops, and knows Rush quite well, and wants to help other shops looking to stream lining their Rush render pipelines. In the past he's been working fulltime for different companies, but has wanted to put more attention to doing freelance scripting and admin. His resume is here: http://www.wavgen.com/mohnen-TD.pdf -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1839 ***************************************************** *** ***************************************************** Subject: Re: [OSX/SYSADMIN] Example NFS setup for Rush on a small 'pure OSX' From: Greg Ercolano Date: Wed, 13 May 2009 20:37:16 -0400 Just a followup to this thread. When creating the 'render' user from the command line, Apple changed everything around in Leopard (10.5), so that now the 'niload' command is no longer available. They now recommend using their new 'dscl' command instead. So the Leopard version of the script to create the new 'render' user would be: #!/bin/sh # # make-render-user - Run this script to create the 'render' user on a Leopard # 1.0 05/10/06 - erco@(email surpressed) (original Tiger version) # 1.1 05/13/09 - erco@(email surpressed) (changed for Leopard) # username=render # user's name uid=555 # must be an unused uid gid=20 # 20=staff passwd=render # the actual *cleartext* password for the user # CREATE THE USER dscl . -create /Users/$username dscl . -create /Users/$username UserShell /bin/bash dscl . -create /Users/$username RealName $username dscl . -create /Users/$username UniqueID $uid dscl . -create /Users/$username PrimaryGroupID $gid dscl . -create /Users/$username NFSHomeDirectory /Users/$username dscl . -passwd /Users/$username $passwd # CREATE THE USER'S HOME DIRECTORY cp -R '/System/Library/User Template/English.lproj' /Users/$username # ENSURE OWNERSHIPS ASSIGNED TO USER'S ENTIRE HOME DIRECTORY chown -R ${uid}:${gid} /Users/$username So that should be it. The older Tiger script is below for reference. BTW, if you want the new user to be an "administrator" user, you can add this line to the list of dscl commands: dscl . -append /Groups/admin GroupMembership $username > STEPS TO SETUP > -------------- > > 1) On all machines, create the "render" user with a fixed UID/GID that > doesn't conflict with any uid/gid on the network. > > This can be done easily by running the following script on each machine > as root to create the 'render' account on each machine with the uid=555, > gid=20, and the password set to 'render'. (Tested under OSX 10.4.x) > > --- snip > #!/bin/csh -f > # > # make-render-user - Run this script to create the 'render' user on a machine > # 1.0 05/10/06 - erco@(email surpressed) > # > set username = render # user's name > set uid = 555 # must be an unused uid > set gid = 20 # 20=staff > set passwd = 'oFxbR2cAnG902' # passwd is 'render' > > # CREATE PASSWD ENTRY > echo ${username}:${passwd}:${uid}:${gid}::0:0:${username}:/Users/${username}:/bin/bash | niload passwd / > > # CREATE USER'S HOME DIRECTORY > if ( ! -d /Users/${username} ) then > cp -R '/System/Library/User Template/English.lproj' /Users/${username} > endif > > # ENSURE OWNERSHIPS ASSIGNED TO USER'S ENTIRE HOME DIRECTORY > /usr/sbin/chown -R ${uid}:${gid} /Users/${username} > --- snip *** ***************************************************** *** Message #1838 ***************************************************** *** ***************************************************** Subject: Re: Log scrolling From: Greg Ercolano Date: Fri, 17 Apr 2009 17:39:51 -0400 Marco Recuay wrote: > [posted to rush.general] > > We're using version 102.42a8. > > I had already turned off log tailing like you'd mentioned, but reading > in the logs still happens progressively. The larger the log, the more > it slows down it seems. Granted, this was a 10mb log file (!), but the > way it was loading the log line by line seemed odd to me. It's trying to display it in a browser that's line oriented, but it should be reading the data in blocks of 4096. If its not, might be a bug in that version.. I'll check. You might try installing the more recent 102.42a9 (July 2008 release) on a single machine to see if it makes a difference. Which OS platform is this on? (Some platforms behave differently than others) -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1837 ***************************************************** *** ***************************************************** Subject: Re: Log scrolling From: Marco Recuay Date: Fri, 17 Apr 2009 16:53:28 -0400 We're using version 102.42a8. I had already turned off log tailing like you'd mentioned, but reading in the logs still happens progressively. The larger the log, the more it slows down it seems. Granted, this was a 10mb log file (!), but the way it was loading the log line by line seemed odd to me. Marco On 2009-04-13 21:00:37 -0700, Greg Ercolano said: > Hmm, which version of Rush are you running, and on which platform? > > You can find out inside irush via "Help -> About Irush", and include > the version number here, which will be something like 102.42aXX > > There were a few versions where the log tailing feature was slower > than other releases, so upgrading may help solve that. > > Given the version info, I can probably tell you if you can upgrade > just the irush tool, without having to upgrade the rest. *** ***************************************************** *** Message #1836 ***************************************************** *** ***************************************************** Subject: Re: Log scrolling From: Greg Ercolano Date: Tue, 14 Apr 2009 00:00:37 -0400 Marco Recuay wrote: > I was wondering if it is possible to disable, or speed up, IRush log > playback. Our Maya renders spit out an ungodly amount of info, which is > good for problem-solving, but it takes a long time to reach the end of > the file. Yes; if by playback you mean the auto-tailing feature, you can disable that via "Edit -> Preferences -> Tail Logs" > If we just navigate to the log location we can open it in a text editor > immediately and read it there, but that is a cumbersome process to > repeat often. Next release of irush will actually have a popup menu for alternately viewing the log in a separate editor (that you can set, eg. gvim/wordpad/TextEditor, etc) > Disabling log tailing will stop the automatic scroll, but the log file > is not loaded immediately, you still have to wait for the log to chug > along to the bottom. > > Any ideas? Hmm, which version of Rush are you running, and on which platform? You can find out inside irush via "Help -> About Irush", and include the version number here, which will be something like 102.42aXX There were a few versions where the log tailing feature was slower than other releases, so upgrading may help solve that. Given the version info, I can probably tell you if you can upgrade just the irush tool, without having to upgrade the rest. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1835 ***************************************************** *** ***************************************************** Subject: Log scrolling From: Marco Recuay Date: Mon, 13 Apr 2009 22:42:49 -0400 I was wondering if it is possible to disable, or speed up, IRush log playback. Our Maya renders spit out an ungodly amount of info, which is good for problem-solving, but it takes a long time to reach the end of the file. If we just navigate to the log location we can open it in a text editor immediately and read it there, but that is a cumbersome process to repeat often. Disabling log tailing will stop the automatic scroll, but the log file is not loaded immediately, you still have to wait for the log to chug along to the bottom. Any ideas? Marco *** ***************************************************** *** Message #1834 ***************************************************** *** ***************************************************** Subject: Maya 2009 not using all processors From: Greg Ercolano Date: Wed, 08 Apr 2009 17:27:21 -0400 If you're having trouble with the new Maya 2009 not using all processors on eg. quad core machines, check out this article: http://mattgadient.com/2009/02/22/make-maya-2009-use-a-multi-core-cpu/ I'm quoting the highlights of the article here, just in case the article "disappears" for some reason: Make Maya 2009 use a multi core cpu ----------------------------------- [February 22nd, 2009] Article by: Matthew Gadien "A small shocker using Maya on a Quad-Core Intel processor was that CPU usage by default was about 50%. Yes that’s right FIFTY PERCENT. It looked similar to this: [greg's ascii-art of their Task Manager showing 50% usage] ____________________________________________________________ | Windows Task Manager | |____________________________________________________________| | | | Cpu Usage _________ _________ _________ _________ | | .... .... | || || || || | || | | | .... .... | || || ||__|| | || | | | .... .... | ||_||__| ||___|__|__|| | | | XXXX XXXX |___ ___|| || ||_____ _| | | XXXX XXXX | |_| || || || |_| | | | XXXX XXXX |_________||_________||_________||_________| | | 50 % Cpu 1 Cpu 2 Cpu 3 Cpu 4 | |____________________________________________________________| "This was a little disappointing - Maya 2009 basically caused another dual-core machine to grind to a halt, and was only using about half the potential of the Quad-Core." He goes on to say (skipping some content): "If you’re using the default Maya Software renderer, there’s an option in the Render Settings menu, but I wont go into that here. The individual I was helping needed MENTAL RAY to be used (instead of Maya Software Renderer), and there wasn’t an equivilent option in that menu. "If you’re using Mental Ray, this is where you make the change [inside maya]: [Greg's summary of the article's screenshots] -------------------------------------------------------------- 1) Switch to the 'Rendering' submenu 2) Under the 'Render' menu, click on "Batch Render" 3) In the "Mental Ray Batch Render Option" dialog, in the "Parallelism" section: 3a) Set Render Threads: 8 3b) Disable the 'Auto Render Threads' checkbox -------------------------------------------------------------- [For a Quad-Core I needed to use 8 to max out the processors.] "This is the trick! "By default, Maya chooses the threads on it’s own. It actually appears to select 4 on a quad-core by default, but it doesn’t max out the cpu usage on each. Changing that number to 8 queues up threads, which pegs the processor to full usage, and speeds up the rendering considerably. "Of course, with all 4 cores maxed, everything else on the computer will be VERY SLOW - even typing out an email, I was able to type an entire sentance out before it appeared on the screen. You will not even want to try doing other stuff while it renders in the background - trust me. On the plus side, more renders happen. The stuff being rendered in my case was doing about 3 renders per minute for the section it was in (a simple section), and it jumped to between 5-6 per minute - not *quite* double, but still quite the increase. "Just for completeness, this is what Task Manager looked like in the end: [greg's ascii-art of their Task Manager showing 100% usage] ____________________________________________________________ | Windows Task Manager | |____________________________________________________________| | | | Cpu Usage _________ _________ _________ _________ | | xxxx xxxx |---------||---------||---------||---------| | | xxxx xxxx | | | || | | | || | || | | | | xxxx xxxx | | || | || || | | | | xxxx xxxx | || || || | | | xxxx xxxx |_________||_________||_________||_________| | | 100 % Cpu 1 Cpu 2 Cpu 3 Cpu 4 | |____________________________________________________________| "Finally, with 8 threads selected, the CPU usage is pegged at 100%." The article goes on to note memory usage stays low, at about 1/4 use in his case, and said he manually set the memory but it made little difference in usage. Then he talks about THREADS vs CORES, which you may find useful; quoting again: "This change talks about THREADS vs CORES. There is a difference, although for our purposes it doesn’t really matter. THREADS are basically like a process, and the operating system sends out threads to cores based on what core is the least busy at the time. We set up 8 threads in the above, so assuming they distribute evenly, each core should be getting fed 2 threads consistantly. We can’t specify CORES with Mental Ray (as far as I can tell), but it doesn’t matter - there wouldn’t be an advantage anyway, since enough threads will fill all the cores up regardless. The only real benefit to actually being able to specify cores would be that if you wanted to you could choose 3 cores, filling them up and leaving the 4th core alone (the system would probably remain snappy with 1 core untouched by Maya, although you’d be rendering at around 3/4 of the potential)." And he then goes on to speculate /why/ 4 threads doesn't max out the cpu cores, thinking it's one of two things [quoting again]: * "Each thread opens, does something, then closes, and the processor isn’t used much between the time a thread closes and a new one opens. * "Each thread ends up waiting on other things (memory reads/writes) which leave it idle." My guess (erco here) is more the latter; rendering can be IO intensive at times, so while IO is happening, the cpu is free to render, so having that extra thread keeps the proc slammed while IO buffers are waiting to clear. *** ***************************************************** *** Message #1833 ***************************************************** *** ***************************************************** Subject: Re: Rush submit menu in Shake From: Greg Ercolano Date: Thu, 12 Mar 2009 15:47:35 -0400 johan aberg wrote: > The following Shake code will put a menu in Shake that will call Rush's > submitscript for Shake. It will pass the name of the current Shake > script as a argument to submit-shake.pl. If you modify submit-shake so > that it picks up the new argument, all you have to do is to press the > update button in submit-shake.pl to set the framerange and fire off the > render. Since this original newsgroup posting was made (2004), all the rush submit scripts now support setting any of the fields in the submit form by using the -field command option to the submit script. For example, to preset the 'Shake Path:' field in the submit form to "/some/path/foo.shk", one can invoke the script with: perl /path/to/RUSH/submit-shake.pl -field ShakePath /some/path/foo.shk So to make the example Johan posted work, you don't need to change these new submit scripts at all, and would only need to change this one line in Johan's shake code from BEFORE to AFTER: BEFORE: const char* command = stringf("/path/to/RUSH/submit-shake.pl %s &", NRiMainWin1.scriptName); AFTER: const char* command = stringf("/path/to/RUSH/submit-shake.pl -field ShakePath %s &", NRiMainWin1.scriptName); ^^^^^^^^^^^^^^^^^^^ [Thanks to Vincent at Eclair Laboratories for verifying this] You can initialize other fields in the submit form this way as well. Multiple '-field' options can be specified to initialize several fields all at once, eg: perl /path/to/RUSH/submit-shake.pl \ -field ShakePath /some/path/foo.shk \ -field Frames 1-10 \ -field JobTitle TESTING_ONE_TWO Note that when specifying fields this way, leave out any spaces in the field's name. Example: "Shake Path" in the submit form becomes "ShakePath" on the command line. > Put the code in a file.h in Shake's include/startup/ui directory. > > ps. Mind the linewraps > > johan > > //////////// > > extern "C" { > int system(const char*); > } > > void CallSystemMenu() > { > const char* command = stringf("/path/to/RUSH/submit-shake.pl %s &", > NRiMainWin1.scriptName); > system(command); > } > > nuiPushMenu("RushRender"); > nuiMenuItem("Submit Rush Job", CallSystemMenu()); > nuiPopMenu(); > > //////////// *** ***************************************************** *** Message #1832 ***************************************************** *** ***************************************************** Subject: [MAC/ADMIN] FAILED TO GET ASN FROM CORESERVICES so aborting From: Greg Ercolano Date: Fri, 06 Feb 2009 18:39:32 -0500 If you're seeing either of these two errors on Leopard systems when running either REDline or After Effects renders: 1) FAILED TO GET ASN FROM CORESERVICES so aborting 2) CFMessagePort: bootstrap_register(): failed 1100 (0x44c) ...let me know via direct email, as I have a workaround that involves a small one line modification to the S99rush boot script. The above errors you'll see on applications that try to access the window manager (whether they actually open windows or not), and run into trouble with Apple's technote TN2083, which is new in Leopard. *** ***************************************************** *** Message #1831 ***************************************************** *** ***************************************************** Subject: [WINDOWS/ADMIN] How to force drive mapping within submit scripts, From: Greg Ercolano Date: Fri, 23 Jan 2009 16:23:03 -0500 This subject has been covered before, but just to tie it all up into a single article.. * * * You can have the submit scripts try to detect and fix broken drive maps on the fly by adding logic to the 'render' section of the submit script just before the render is invoked. Drives can be mapped with the DOS 'net use' command, eg: NET USE Z: \\YOURSERVER\YOURSHARE3 /PERSISTENT:YES You can add commands like this to the MAIN_Render() section of the script. Since in some cases the maps can time out, you may have to first delete the broken map before re-creating it. Some example perl code to do all this: if ( ! -d "z:/" ) { my $cmd = "net use z: /delete < nul"; print "Z: NOT MAPPED -- FORCE UNMAP FIRST: $cmd\n"; system($cmd); $cmd = "net use z: \\\\yourserver\\yourshare1 /PERSISTENT:YES < nul"; print "Z: MAPPING WITH: $cmd\n"; system($cmd); } Note that *backslashes* have to be used for the pathname of this DOS command, so be sure to 'double up' your slashes, since backslashes are an escape character in perl (and most all scripting languages). One problem with the above is a catch-22 situation, where the script itself may be specified with a drive letter; so if the drive isn't mapped, the machine can't even run the script to fix the problem, because the path to the script can't be reached. In this case, you have to ensure the pathname to the script *and* the log directory are both UNC paths. Fortunately, you can do this by modifying the FixPath() function in .common.pl on your fileserver, so that the pathname is fixed just before the job is submitted to Rush; this will handle translating the drive letters into a UNC equivalent. See the lines commented with "# ADD THIS LINE" in the following to handle changing several different drive mappings: sub FixPath($) { my ($path) = @_; if ( $G::iswindows ) { .. } else { .. } $path =~ s%/\./%/%g; # /foo/./bar -> /foo/bar $path =~ s%^z:%//yourserver/share1%gi; # ADD THIS LINE: z: -> //yourserver/share1 $path =~ s%^y:%//yourserver/share2%gi; # ADD THIS LINE: y: -> //yourserver/share2 $path =~ s%^x:%//yourserver/share3%gi; # ADD THIS LINE: x: -> //yourserver/share3 return($path); } *** ***************************************************** *** Message #1830 ***************************************************** *** ***************************************************** Subject: [ADMIN/OT] Environment variables in Windows' native file browsers.. From: Greg Ercolano Date: Thu, 22 Jan 2009 18:20:17 -0500 Just an FYI. I didn't know this, but apparently one can specify environment variables in Microsoft Window's file browsers? For example, if you have the environment variable called SHOW that is set to an existing path like \\server\jobs\honda, you can type at the Filename: prompt: %SHOW%\ ...and see a directory listing of \\server\jobs\honda. Since eg. Maya uses Microsoft's native file browsers, this means companies can come up with shortcut ways for users to access long pathnames easily, saving a bit of typing and manual surfing. Similarly, under Linux, with konqueror you can use $SHOW in the filename bar. Unfortunately, I don't think the Mac's file browsers work similarly. Pretty interesting, though. *** ***************************************************** *** Message #1829 ***************************************************** *** ***************************************************** Subject: Re: Maxwell Standalone Renders on OSX From: Luke Cole Date: Thu, 22 Jan 2009 18:12:01 -0500 Thanks for the extra info Robert, I'll try contacting Next Limit too and see what kind of response we can get. On 23/01/2009, at 9:49 AM, Robert Minsk wrote: > I contacted NextLimit over year ago about this issue and nothing > has changed. > Maybe if a few more users add some weight behind the issue they may do > something. > > Be careful with making the executable setuid. Maxwell will write > things all > over the drive given the chance. --- Luke Cole Research & Development FUEL International 65 King St., Newtown, Sydney NSW, Australia 2042 *** ***************************************************** *** Message #1828 ***************************************************** *** ***************************************************** Subject: Re: Maxwell Standalone Renders on OSX From: Robert Minsk Date: Thu, 22 Jan 2009 17:49:36 -0500 On Wednesday 21 January 2009 22:59, Greg Ercolano wrote: > I'd suggest opening a dialog with the Maxwell guys right away. > GUIs make unattended/distributed rendering almost impossible. > > Best thing I can advise trying is making the executable > setuid root, eg: > > chown 0.0 /path/to/maxwell/executable > chmod 4755 /path/to/maxwell/executable > > ..and see how that flies. I contacted NextLimit over year ago about this issue and nothing has changed. Maybe if a few more users add some weight behind the issue they may do something. Be careful with making the executable setuid. Maxwell will write things all over the drive given the chance. *** ***************************************************** *** Message #1827 ***************************************************** *** ***************************************************** Subject: Re: Maxwell Standalone Renders on OSX From: Luke Cole Date: Thu, 22 Jan 2009 02:04:55 -0500 Thanks Greg, We'll give it a try and see how it goes! On 22/01/2009, at 5:59 PM, Greg Ercolano wrote: > > I'd suggest opening a dialog with the Maxwell guys right away. > GUIs make unattended/distributed rendering almost impossible. > > Best thing I can advise trying is making the executable > setuid root, eg: > > chown 0.0 /path/to/maxwell/executable > chmod 4755 /path/to/maxwell/executable > > ..and see how that flies. --- Luke Cole Research & Development FUEL International 65 King St., Newtown, Sydney NSW, Australia 2042 *** ***************************************************** *** Message #1826 ***************************************************** *** ***************************************************** Subject: Re: Maxwell Standalone Renders on OSX From: Greg Ercolano Date: Thu, 22 Jan 2009 01:59:34 -0500 Luke Cole wrote: > We are experiencing a problem getting maxwell standalone renders > working successfully on osx. The problem appears to be due to the > maxwell standalone renderer needing to open a graphical console > window to output logging information rather than writing to stdout. > > Rush is configured to run apps on the osx machines under a dedicated > render user account, and if this user is not logged in to the osx > window server, the maxwell standalone "command line" client (mxcl) > fails with the following error, because the dedicated render user > does not have permission to access the window server: > > _RegisterApplication(), FAILED TO establish the default connection to > the WindowServer, _CGSDefaultConnection() is NULL. > > Annoyingly, there appears to be no means of forcing mxcl to run in a > silent mode - it will always attempt to open a graphical console > window. On our dedicated osx render nodes, this is not really an > issue, as we are able to leave these machines logged in under the > dedicated render user, and all is well. > > We would like to be able to render on some of our high-end osx > workstations in the evenings however, and as these are usually left > logged in under artist user accounts, the jobs end up failing due to > the above error. > > I was wondering if anyone might have any suggestions on how we could > work around this issue? I'd suggest opening a dialog with the Maxwell guys right away. GUIs make unattended/distributed rendering almost impossible. Best thing I can advise trying is making the executable setuid root, eg: chown 0.0 /path/to/maxwell/executable chmod 4755 /path/to/maxwell/executable ..and see how that flies. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1825 ***************************************************** *** ***************************************************** Subject: Maxwell Standalone Renders on OSX From: Luke Cole Date: Thu, 22 Jan 2009 01:41:14 -0500 Hi, We are experiencing a problem getting maxwell standalone renders working successfully on osx. The problem appears to be due to the maxwell standalone renderer needing to open a graphical console window to output logging information rather than writing to stdout. Rush is configured to run apps on the osx machines under a dedicated render user account, and if this user is not logged in to the osx window server, the maxwell standalone "command line" client (mxcl) fails with the following error, because the dedicated render user does not have permission to access the window server: _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. Annoyingly, there appears to be no means of forcing mxcl to run in a silent mode - it will always attempt to open a graphical console window. On our dedicated osx render nodes, this is not really an issue, as we are able to leave these machines logged in under the dedicated render user, and all is well. We would like to be able to render on some of our high-end osx workstations in the evenings however, and as these are usually left logged in under artist user accounts, the jobs end up failing due to the above error. I was wondering if anyone might have any suggestions on how we could work around this issue? Thanks in advance, --- Luke Cole Research & Development FUEL International 65 King St., Newtown, Sydney NSW, Australia 2042 *** ***************************************************** *** Message #1824 ***************************************************** *** ***************************************************** Subject: Re: Toxic 2009 From: Greg Ercolano Date: Wed, 14 Jan 2009 20:21:36 -0500 matx wrote: > [posted to rush.general] > > On 2009-01-14 14:39:08 -0800, Greg Ercolano said: >> Shouldn't be a problem to have a submit script for it, though. >> At very least, you can run it through submit-generic, eg: > > I haven't tried rendering with it in Terminal, but it does indeed > appear to be possible. I'll try the submit generic script. What I would do first is try to get it to work from the command line. If you can get it to work in the terminal, then toss the command into submit-generic, replacing the 'startframe' and 'endframe' values with the following, depending on your OS: UNIX ---- If you're using unix/osx, then in submit-generic, set "Shell: sh", and in place of the startframe and endframe, use: $BATCH_SFRM <-- use this in place of the start frame $BATCH_EFRM <-- use this in place of the end frame eg: source /opt/Autodesk/Autodesk_Toxik-2009/bin/toxik-env.sh txrender .. -s $BATCH_SFRM -e $BATCH_EFRM .. (Note the 'source' command is recommended by Toxik's docs) WINDOWS ------- If you using Windows, then in submit-generic set the "Shell: DOS-Batch", and in place of the startframe and endframe, use: %BATCH_SFRM% <-- use this in place of the start frame %BATCH_EFRM% <-- use this in place of the end frame eg: txrender .. -s %BATCH_SFRM% -e %BATCH_EFRM% .. If you have any trouble, email me privately, and when we iron out the specifics, I'll follow up here. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1823 ***************************************************** *** ***************************************************** Subject: Re: Toxic 2009 From: matx Date: Wed, 14 Jan 2009 19:30:09 -0500 On 2009-01-14 14:39:08 -0800, Greg Ercolano said: > > Shouldn't be a problem to have a submit script for it, though. > At very least, you can run it through submit-generic, eg: I haven't tried rendering with it in Terminal, but it does indeed appear to be possible. I'll try the submit generic script. $ txrender A project path must be given Usage: -userfilepath : User file path. (default: /Users/admin/Library/Preferences/Autodesk/Autodesk Toxik 2009b1//.txuser) -p,-project : Project path. -c,-composition : Composition path. -v,-version : Version name. -o,-outputs : List of output names ('/' separated). -s,-startframe : Start index of the interval that constrains the frames to render for every output. Can be omitted meaning there's no start limit. -e,-endframe : End index (exclusive) of the interval that constrains the frames to render for every output. Can be omitted meaning there's no end limit. -f,-frameindex : Renders only the frame at the given index in each output where possible. -frameoffset : Renders only the frame at the given offset from the start index (--startframe parameter required) in each output where possible (offset is 1-based => start frame is at offset 1). > Is this a serious contender to Nuke/AfterEffects/etc? Some people are looking at it seriously, as it integrates well with Maya. :) Mat X *** ***************************************************** *** Message #1822 ***************************************************** *** ***************************************************** Subject: Re: Toxic 2009 From: Greg Ercolano Date: Wed, 14 Jan 2009 17:59:21 -0500 Greg Ercolano wrote: > matx wrote: >> Any support currently for Toxic 2009? > > Autodesk "Toxik", the compositor? BTW, if you're seriously starting to use it, I'll make a submit script for it, and I'll work with you to test it out. But if it's just an inquiry, I won't write a script for it. Someone has to be willing to test it. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1821 ***************************************************** *** ***************************************************** Subject: Re: Toxic 2009 From: Greg Ercolano Date: Wed, 14 Jan 2009 17:39:08 -0500 matx wrote: > Any support currently for Toxic 2009? Autodesk "Toxik", the compositor? I've never tried it. No one's really asked much about it. I think I've had one previous inquiry just seeing if people were using it. Shouldn't be a problem to have a submit script for it, though. At very least, you can run it through submit-generic, eg: Shell: DOS-Batch Job Title: TOXIK_TEST Frames: 1-100 Batch: 5 Log Directory: //yourserver/some/empty/dir Command: txrender –d DB_MYDATABASE –u User1 –p Project1 –c CompA –r “CompA_06-06-29_174526” –f “CompA-” –D “\\yourserver\renders” -s %BATCH_START% –e %BATCH_END% –i EXR Cpus: +any=5 Their docs on the command line are pretty thin (their PDF docs refer to a screenshot of someone running the command line help) Is this a serious contender to Nuke/AfterEffects/etc? -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1820 ***************************************************** *** ***************************************************** Subject: Toxic 2009 From: matx Date: Wed, 14 Jan 2009 16:49:35 -0500 Any support currently for Toxic 2009? -- :) Mat X *** ***************************************************** *** Message #1819 ***************************************************** *** ***************************************************** Subject: [Q+A/WINDOWS ADMIN] No more connections can be made to this remote From: Greg Ercolano Date: Thu, 08 Jan 2009 20:01:36 -0500 > We have a problem where we're getting this error > from some of the machines on our farm: > ******************************************************* > ERROR: can't create log: //server001/XSI/Scenes/bw12.scn.log: > No more connections can be made to this remote computer at > this time because there are already as many connections as the > computer can accept. > ******************************************************* > > All the systems used are Windows2000 Professional. > And the number of machines used for a rendering is 10. This is a standard Microsoft error when a machine tries to open a file on a file server that has maxed out its CAL licenses. In this case, there are too many file sharing connections open to file server //server001/XSI/. Microsoft devotes several "Knowledge Base" (KB) articles to this: 1) http://support.microsoft.com/kb/314882 2) http://support.microsoft.com/kb/328459 3) http://support.microsoft.com/kb/122920 The error is also covered pretty well on Usenet, eg: http://groups.google.com/group/microsoft.public.windowsxp.network_web/browse_frm/thread/1a5c4a7ed0f4d954?hl=en&lr=&ie=UTF-8&oe=UTF-8&rnum=7&prev=/groups%3Fhl%3Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26q%3DNo%2Bmore%2Bconnections%2Bcan%2Bbe%2Bmade%2Bto%2Bthis%2Bremote%2Bcomputer%2Bat%2B%26btnG%3DGoogle%2BSearch&fwc=1 If "server001" is running Windows 2000 Professional, there can only be 10 NTFS connections open at a time. Any more than that you would need to get Windows 2000 Server. On host 'server001', go into 'My Computer' and right click on the 'XSI' share, and check what the 'User Limit' is set to. You will find that, for Win2K professional, there's a 10 user/remote host connection limit. Microsoft says this cannot be changed for their "Professional" series of products; you /have/ to upgrade to one of the Microsoft "Server" products which support more than 10 connections, and you can purchase more "CAL" licenses to increase the connection count. (CAL=Client Access Licenses). Microsoft is unusual in that the license their file server's connections. A connection is "per-user", "per-share". This means even if there's one share, a single workstation might still use up more than one connection for that share, if more than one user account is actively accessing the share. This problem will appear to be intermittent, as connections will timeout after 15 minutes of inactivity, freeing them up. Then different machines will have the problem. Even users may encounter the error while opening files interactively. Administrators of Microsoft file servers have to make sure file servers have enough Microsoft's CAL licenses to support their entire render farm and workstation users all working concurrently. This is why many companies do not use Microsoft machines for file servers, to avoid the CAL licensing issues. The solution is to either: 1) Add more Microsoft CAL licenses to your "Server" 2) If your server is running a workstation version of Microsoft's OS, (XP Pro, 2K Pro, etc), you will need to install Microsoft "Server" instead. The workstation OS's are limited to 10 connections, period. 3) Use a non-Microsoft file server, eg. Mac, Linux, or some third party file server, like NetApp, BlueArc, Isilon, etc.. *** ***************************************************** *** Message #1818 ***************************************************** *** ***************************************************** Subject: [FYI] Freelance Perl/Python programmer available From: Greg Ercolano Date: Fri, 19 Dec 2008 15:50:23 -0500 CGI pipeline scripting programmers are about as rare as hens teeth, so thought you all would be interested to know.. Ammon Riley is apparently available for doing freelance perl/python/pyQT pipeline scripting via remote access from Portland. His resume: http://www.linkedin.com/in/ammonriley He used to work at R&H and Laika, and I guess he got caught up in the massive Laika layoffs due to the closing of the feature. Caught wind of him from one of the CGI mailing lists.. FYI. *** ***************************************************** *** Message #1817 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: "Abraham Schneider" Date: Fri, 19 Dec 2008 04:20:37 -0500 Greg Ercolano schrieb: > Sounds interesting, but unless they have a non-gui interface, > it's not something I think Rush can control properly. yes, that's right. I'm doing it in two steps at the moment: rush=20 executes a jobdonecommand script which creates an entry with the path to = the image sequence in a queue folder. A small python script on my QT=20 creation Mac is watching this folder and does the actual 'feeding' of=20 the Silverstack rendering app (called Cinemator). That's not a perfect solution but it works quite well for us at the=20 moment. And with the template files it is easy to do different burnins,=20 codecs, framerates, etc. depending on project or user choose. Abraham -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Thomas Till *** ***************************************************** *** Message #1816 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Thu, 18 Dec 2008 13:30:57 -0500 Craig Allison wrote: > I'm currently working on the Applescript suggestion to find/replace the > contents of a Shake "setup" script and then submitting to command line > render. Note that shake files are simple ascii, so you can use tools like sed or perl to take a template shake script and convert it on the fly to what you need. IIRC, you can also just edit the .shk file, and replace the strings you want to use with environment variables.. then all you have to do is be sure to set those env vars before running shake, and the replacing will be handled by shake. I think there may also be a way to pass variables from the shake command line. > Any QT additions to the Rush submit would be helpful for the artists, as > they could make use of the farm for quick preview QT's for long shots > instead of Framecycler/DJView/FlipBook. Right.. See also the submit-shake-quicktime submit script, which artists can use to do QT conversions. Use the one from 102.42a9, which has many codecs, and several fixes from previous versions. These tweaks are mentioned in the release notes for A9: http://www.seriss.com/rush-current/rush/release-notes.html#102.42a9 ..specifically: o submit-shake-quicktime -- numerous fixes to support codecs > I'll let you know how I get on with the Applescript as I'm just > unravelling how it all works and am seriously under the cosh with > deadlines!!! Yes, feel free to send pastes. I've done a few scripts with AppleScript to descan the slitscan sequences from 2001; I used applescript to hit the frame advance button in iDVD so that I could grab scanlines from the film to unwrap them. Fun pictures here: http://seriss.com/people/erco/2001/wip/ ..with the scripts at the bottom. See 'shoot.pl' for a perl script that sends applescript text strings to osascript on a remote machine. eg: system("( echo 'tell app \"DVD Player\"'; echo step dvd; echo end tell ) | rsh tow osascript"); Must admit, applescript is quite neat. But the fact it works through GUIs is a problem for automation in contexts where there is no window manager, or no one logged in. In the above case, I had to make sure the GUI was running before I could run the script. > Aye, aye, aye bring on Christmas! Amen to that..! -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1815 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Craig Allison Date: Thu, 18 Dec 2008 13:12:19 -0500 --Apple-Mail-30-61134651 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed I was looking at Silverstack recently and DJV-convert but they don't quite meet my requirements as we have movie specific masking/crops applied to the full 2k frames when they are converted to QT... I'm currently working on the Applescript suggestion to find/replace the contents of a Shake "setup" script and then submitting to command line render. Any QT additions to the Rush submit would be helpful for the artists, as they could make use of the farm for quick preview QT's for long shots instead of Framecycler/DJView/FlipBook. I'll let you know how I get on with the Applescript as I'm just unravelling how it all works and am seriously under the cosh with deadlines!!! Aye, aye, aye bring on Christmas! Cheers everyone - Have a good one! Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 18 Dec 2008, at 17:27, Greg Ercolano wrote: > [posted to rush.general] > >> Our pipeline at the moment is: we render downsized, LUT-converted >> JPEGs >> in the same setup as the main DPX files as a second output. After the >> job is done I submit a job to the renderqueue of Silverstack (called >> Cinemator) on a small Mac mini to generate quicktimes from these >> JPEGs. >> >> Silverstack can create templates for burnin, LUT conversion, >> codec, etc. >> which you can use while submitting the job to Cinemator on the >> commandline. >> >> At the moment, Cinemator is a GUI app which can be fed by a >> commandline >> command, but what I heard is that they are working on a more >> sophisticated solution. >> >> Maybe you want to have a look on it. > > Sounds interesting, but unless they have a non-gui interface, > it's not something I think Rush can control properly. > > What I'd like to do is have a way to make it easy to add any > kind of command line tool to handle QT conversions to the > submit scripts, and have a pulldown that lets one select which > converter to use, eg. ffmpeg, MetaRender, SequencePublisher, > qt-tools, etc. > > Probably what I'd do is add a "QT" tab to all the submit scripts > that has prompts for the input images and output movie, as well > as being able to select which codecs to use. All the rush submit > scripts would then be able to call a single, central script > to handle the actual QT conversion as a 'jobdonecommand'. > --Apple-Mail-30-61134651 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 I was looking at Silverstack recently and DJV-convert but they don't = quite meet my requirements as we have movie specific masking/crops = applied to the full 2k frames when they are converted to QT...

I'm currently working on = the Applescript suggestion to find/replace the contents of a Shake = "setup" script and then submitting to command line render.

Any QT additions to the = Rush submit would be helpful for the artists, as they could make use of = the farm for quick preview QT's for long shots instead of = Framecycler/DJView/FlipBook.

I'll let you know how I = get on with the Applescript as I'm just unravelling how it all works and = am seriously under the cosh with deadlines!!!

Aye, aye, aye bring on = Christmas!

Cheers everyone - Have a = good one!


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


On 18 Dec 2008, at 17:27, Greg Ercolano = wrote:

[posted to = rush.general]

=
Our pipeline at the moment = is: we render downsized, LUT-converted JPEGs=A0
in the = same setup as the main DPX files as a second output. After the=A0
job is = done I submit a job to the renderqueue of Silverstack (called=A0

Silverstack can create templates for burnin, LUT = conversion, codec, etc.=A0
which = you can use while submitting the job to Cinemator on the = commandline.

At the moment, Cinemator is a GUI app which can be = fed by a commandline=A0
command, = but what I heard is that they are working on a more=A0

Maybe you want to have a look on = it.

Sounds interesting, but unless = they have a non-gui interface,
it's not = something I think Rush can control properly.

What I'd = like to do is have a way to make it easy to add any
kind of command line tool to = handle QT conversions to the
submit = scripts, and have a pulldown that lets one select which
converter to use, eg. ffmpeg, = MetaRender, SequencePublisher,
qt-tools, = etc.

Probably what I'd do is add a = "QT" tab to all the submit scripts
that has = prompts for the input images and output movie, as well
as being able to select which = codecs to use. All the rush submit
scripts = would then be able to call a single, central script
to handle the actual QT = conversion as a 'jobdonecommand'.


= --Apple-Mail-30-61134651-- *** ***************************************************** *** Message #1814 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Thu, 18 Dec 2008 12:27:26 -0500 > Our pipeline at the moment is: we render downsized, LUT-converted JPEGs > in the same setup as the main DPX files as a second output. After the > job is done I submit a job to the renderqueue of Silverstack (called > Cinemator) on a small Mac mini to generate quicktimes from these JPEGs. > > Silverstack can create templates for burnin, LUT conversion, codec, etc. > which you can use while submitting the job to Cinemator on the commandline. > > At the moment, Cinemator is a GUI app which can be fed by a commandline > command, but what I heard is that they are working on a more > sophisticated solution. > > Maybe you want to have a look on it. Sounds interesting, but unless they have a non-gui interface, it's not something I think Rush can control properly. What I'd like to do is have a way to make it easy to add any kind of command line tool to handle QT conversions to the submit scripts, and have a pulldown that lets one select which converter to use, eg. ffmpeg, MetaRender, SequencePublisher, qt-tools, etc. Probably what I'd do is add a "QT" tab to all the submit scripts that has prompts for the input images and output movie, as well as being able to select which codecs to use. All the rush submit scripts would then be able to call a single, central script to handle the actual QT conversion as a 'jobdonecommand'. *** ***************************************************** *** Message #1813 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: "Abraham Schneider" Date: Wed, 17 Dec 2008 03:43:39 -0500 Greg Ercolano schrieb: > I've been trying to look for a nice cross-platform QT converter > solution. There are definitely a few commercial apps out there > (MetaRender looks like one, part of the iridas tools), > and some freeware apps as well (qt-tools, libquicktime, ffmpeg), > but many of the freeware apps are not "cross platform". >=20 > Some suggestions for command line quicktime converters I've seen: >=20 > Framecycler (SequencePublisher/MetaRender) > Nuke > Shake > Fusion > http://avidemux.org/admWiki/index.php?title=3DCommand_line_usage > djv_convert > ImageMagick+OpenQuicktime (alternative to ffmpeg) > MEncoder (http://www.mplayerhq.hu) > libquicktime + custom code > qt-tools (qt_export tool) >=20 One you haven't mentioned: Silverstack from Pomfort: www.pomfort.com Our pipeline at the moment is: we render downsized, LUT-converted JPEGs=20 in the same setup as the main DPX files as a second output. After the=20 job is done I submit a job to the renderqueue of Silverstack (called=20 Cinemator) on a small Mac mini to generate quicktimes from these JPEGs. Silverstack can create templates for burnin, LUT conversion, codec, etc. = which you can use while submitting the job to Cinemator on the = commandline. At the moment, Cinemator is a GUI app which can be fed by a commandline=20 command, but what I heard is that they are working on a more=20 sophisticated solution. Maybe you want to have a look on it. Abraham -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Thomas Till *** ***************************************************** *** Message #1812 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Tue, 16 Dec 2008 17:34:10 -0500 matx wrote: > [posted to rush.general] > > On 2008-12-16 11:16:35 -0800, Craig Allison said: >> I'll have a look at AppleScript as a possible interim solution, > > We use an AppleScript to interface between a template Shake script and > the image sequence selected (drag drop folder onto script). After the > images are rendered by an artist they make a QT using this tools which > basically substitutes new fileins and fileouts, etc. If this is something you can call as a command, then you can make it the jobdonecommand for the job. > I'd love to see how you could incorporate a Shake script in a > shake-submit for Rush as a followup job using the variables of the > previous job. Can RUSH pass variables to another job? You can pass anything you want, but you'd need to add the operation to do it. The manual way is: submit your shake job, and lets say its jobid is "tahoe.1". While it's rendering, you can fill out the submit-shake-quicktime.pl form, and set these fields: WaitFor: tahoe.1 Wait For State: Done ..and submit the job. This way when your shake job finishes the last frame, the shake quicktime script runs to do the QT conversion. Or, if you're clever with scripting, you can modify the submit-shake script to automatically fire off submit-shake-quicktime to do the QT conversion as a "jobdonecommand". It's currently a two step process use submit-shake-quicktime.pl, After you submit the shake job, you can use submit-shake-quicktime.pl to submit a separate job to handle the conversion. It needs to know the input images and output quicktime file. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1811 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: matx Date: Tue, 16 Dec 2008 17:23:27 -0500 On 2008-12-16 11:16:35 -0800, Craig Allison said: > > I'll have a look at AppleScript as a possible interim solution, We use an AppleScript to interface between a template Shake script and the image sequence selected (drag drop folder onto script). After the images are rendered by an artist they make a QT using this tools which basically substitutes new fileins and fileouts, etc. I'd love to see how you could incorporate a Shake script in a shake-submit for Rush as a followup job using the variables of the previous job. Can RUSH pass variables to another job? :) Mat X *** ***************************************************** *** Message #1810 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Tue, 16 Dec 2008 14:29:15 -0500 Craig Allison wrote: > I'll have a look at AppleScript as a possible interim solution, but I > really wanted to have it bundled into the Rush submit with a dialogue to > add text to variables set up in my Shake script for slate notes etc. I > could give them a custom slate node to avoid this though, hmmm... > > We're doing 50-100 qts a day here, not sure if that's classed as a lot, > but it sure would be better than doing them manually! It should be possible to have the submit script create the .shk script on the fly, and then run it as a jobdonecommand. This is more or less what the submit-shake-quicktime.pl script does already. Maybe just take an existing shake script, and replace all the parts you want changeable with environment variables, then set these variables before invoking the shake script to convert the frames. I've been trying to look for a nice cross-platform QT converter solution. There are definitely a few commercial apps out there (MetaRender looks like one, part of the iridas tools), and some freeware apps as well (qt-tools, libquicktime, ffmpeg), but many of the freeware apps are not "cross platform". Some suggestions for command line quicktime converters I've seen: Framecycler (SequencePublisher/MetaRender) Nuke Shake Fusion http://avidemux.org/admWiki/index.php?title=Command_line_usage djv_convert ImageMagick+OpenQuicktime (alternative to ffmpeg) MEncoder (http://www.mplayerhq.hu) libquicktime + custom code qt-tools (qt_export tool) Nuke: n = nuke.createNode('Write') n.knob('file').setValue('C:/quicktime.mov') n.knob('codec').setValue(12) # H.264 n.knob('settings').setValue('0000000000000000000000000000019a7365616e0000000100000001000000000000018676696465000000010000000e00000000000000227370746c0000000100000000000000006176633100000000001800000300000000207470726c000000010000000000000000000003000018000000000018000000246472617400000001000000000000000000177000000000530000010000000100000000156d70736f00000001000000000000000000000000186d66726100000001000000000000000000000000000000187073667200000001000000000000000000000000000000156266726100000001000000000000000001000000166d70657300000001000000000000000000000000002868617264000000010000000000000000000000000000000000000000000000000000000000000016656e647300000001000000000000000000000000001663666c67000000010000000000000000004400000018636d66720000000100000000000000006170706c00000014636c75740000000100000000000000000000001c766572730000000100000000000000000003001a00010000') Shake has a similar technique, where a magic blob of text is needed for the QT codec settings (see submit-shake-quicktime.pl) -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1809 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Craig Allison Date: Tue, 16 Dec 2008 14:16:35 -0500 --Apple-Mail-11--107803867 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed I'll have a look at AppleScript as a possible interim solution, but I really wanted to have it bundled into the Rush submit with a dialogue to add text to variables set up in my Shake script for slate notes etc. I could give them a custom slate node to avoid this though, hmmm... We're doing 50-100 qts a day here, not sure if that's classed as a lot, but it sure would be better than doing them manually! Will keep working on it Thanks Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 16 Dec 2008, at 18:01, matx wrote: > [posted to rush.general] > > On 2008-12-16 07:43:55 -0800, Craig Allison > said: >> I'm looking at setting up an autorender of QT's when a set of 2k >> frames have been rendered in Shake using Rush > > Since RUSH can only allocate one machine to render a QuickTime from > an image sequence, you could submit a follow up Rush job that waits > for the first to finish. The QuickTime will be made at the end of > the frame rendering. > > We wrap our custom Shake scripts in AppleScript and drag/drop the > finished frames folder onto the appropriate tool to make our > QuickTimes. That works for most cases. However, if you're > processing a lot of QTs something more automated may be appropriate. > > Mat X > > --Apple-Mail-11--107803867 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 I'll have a look at AppleScript = as a possible interim solution, but I really wanted to have it bundled = into the Rush submit with a dialogue to add text to variables set up in = my Shake script for slate notes etc. =A0I could give them a custom slate = node to avoid this though, hmmm...

We're doing 50-100 qts a = day here, not sure if that's classed as a lot, but it sure would be = better than doing them manually!

Will keep working on = it

Thanks



Craig Allison

Digital Systems & = Data=A0Manager

Twickenham Film Studios

St.Margarets

Twickenham

TW1 2AW


=A0

skype: craig_9000


=


On 16 Dec 2008, at 18:01, matx wrote:

[posted to rush.general]

On = 2008-12-16 07:43:55 -0800, Craig Allison <craig@(email surpressed)> = said:
I'm looking = at setting up an autorender of QT's when a set of 2k
frames have been rendered in Shake using Rush
=

Since RUSH can only allocate one machine to render a = QuickTime from an image sequence, you could submit a follow up Rush job = that waits for the first to finish. The QuickTime will be made at the = end of the frame rendering.

We wrap our custom Shake scripts = in AppleScript and drag/drop the finished frames folder onto the = appropriate tool to make our QuickTimes. That works for most cases. = However, if you're processing a lot of QTs something more automated may = be appropriate.

Mat X


=

= --Apple-Mail-11--107803867-- *** ***************************************************** *** Message #1808 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: matx Date: Tue, 16 Dec 2008 13:01:34 -0500 On 2008-12-16 07:43:55 -0800, Craig Allison said: > > I'm looking at setting up an autorender of QT's when a set of 2k > frames have been rendered in Shake using Rush Since RUSH can only allocate one machine to render a QuickTime from an image sequence, you could submit a follow up Rush job that waits for the first to finish. The QuickTime will be made at the end of the frame rendering. We wrap our custom Shake scripts in AppleScript and drag/drop the finished frames folder onto the appropriate tool to make our QuickTimes. That works for most cases. However, if you're processing a lot of QTs something more automated may be appropriate. Mat X *** ***************************************************** *** Message #1807 ***************************************************** *** ***************************************************** Subject: Re: RenderMan for Maya taking too many licenses From: matx Date: Tue, 16 Dec 2008 12:57:39 -0500 On 2008-12-16 06:07:23 -0800, Greg Ercolano said: > > For instance, if your nodes each have 4000 of ram > (in the rush/etc/hosts file), then submit with > "Ram: 4000" to ensure no more than one instance > of that job is started on each machine, to prevent > using more licenses than you have. I'll try setting the RAM explicitly. Good idea, I haven't tried that. > > Try telling rush to render one of your scenes on just > one machine, and then query the renderman license server > to see how many licenses it thinks that machine has checked > out for rendering, and then look at how many processes > are actually running on the machine (with ps(1) or TaskManager) I'm using the Pixar licensing tools to check how many licenses stations have, and they'll often take two or three licenses. I don't know if its because a frame is failing then checking out a new frame before it lets go (or doens't let go) of the previous license, or because the license server does not use a persistent connection to the client to determine license status. More poking required. Thanks. -x *** ***************************************************** *** Message #1806 ***************************************************** *** ***************************************************** Subject: Auto Rendering of QT's on jobcomplete From: Craig Allison Date: Tue, 16 Dec 2008 10:43:55 -0500 --Apple-Mail-8--120565124 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello out there I'm looking at setting up an autorender of QT's when a set of 2k frames have been rendered in Shake using Rush, as Shake scripts are effectively text files so I was looking to send variables from the Rush submit to a Shake script I have setup on our server. Is this possible? Has anyone done it already? Any advice would be much appreciated! Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com --Apple-Mail-8--120565124 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hello out there

I'm = looking at setting up an autorender of QT's when a set of 2k frames have = been rendered in Shake using Rush, as Shake scripts are effectively text = files so I was looking to send variables from the Rush submit to a Shake = script I have setup on our server.

Is this possible? Has = anyone done it already?

Any advice would be much = appreciated!

Cheers

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


= --Apple-Mail-8--120565124-- *** ***************************************************** *** Message #1805 ***************************************************** *** ***************************************************** Subject: Re: RenderMan for Maya taking too many licenses From: Greg Ercolano Date: Tue, 16 Dec 2008 09:07:23 -0500 matx wrote: > We're seeing an odd issue with RenderMan for Maya taking too many > licenses when we submit through RUSH. Even when specifying "-n 4" > manually in the submit script, which normally works fine. In the rush hosts file, do you have the CPUS column set to values >1? If so, rush will try to start up to that many instances of your render on the machine, unless your job is set to prevent that. When you submit the job with '-n 4', try submitting the job with 'Ram:' set to the amount of ram rush has been configured to think each machine has. For instance, if your nodes each have 4000 of ram (in the rush/etc/hosts file), then submit with "Ram: 4000" to ensure no more than one instance of that job is started on each machine, to prevent using more licenses than you have. > restarted the Pixar RenderMan for Maya license server (v.5.0.2) and it > seemed to have no affect on licenses being taken in excess by the > clients. You can probably use the licensing tools to determine which machines have which licenses checked out, to see why there are so many licenses being used. Try telling rush to render one of your scenes on just one machine, and then query the renderman license server to see how many licenses it thinks that machine has checked out for rendering, and then look at how many processes are actually running on the machine (with ps(1) or TaskManager) -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1804 ***************************************************** *** ***************************************************** Subject: RenderMan for Maya taking too many licenses From: matx Date: Tue, 16 Dec 2008 03:32:01 -0500 Hi We're seeing an odd issue with RenderMan for Maya taking too many licenses when we submit through RUSH. Even when specifying "-n 4" manually in the submit script, which normally works fine. These are 8-proc Xeon Xserves running Mac OS X Server 10.5.5, and Rush 102.42a9. This may not be a Rush problem, as it usually render fine. But we restarted the Pixar RenderMan for Maya license server (v.5.0.2) and it seemed to have no affect on licenses being taken in excess by the clients. A frame would be rendering, reporting in rush, then suddenly the frame failed, and the time in reported rush to have been taken was less than earlier reported, and then multiple licenses will have been checked out until no licenses remain... Sounds like the renderman license server is to blame, and Rush is only showing the symptom, but I'd thought I'd bouce off some peeps here. I'll be sure to post on Pixar's forums, as well. Anyone using RfMv.2 with Maya 2008 in Mac OS X? Any ideas? Of course, we're up against a tight deadline... When are we not? :) Mat X *** ***************************************************** *** Message #1803 ***************************************************** *** ***************************************************** Subject: [Q+A/ADMIN] Is there a way to use Rush to check for installed software? From: Greg Ercolano Date: Wed, 03 Dec 2008 16:09:18 -0500 > We'd like to audit all our machines to see what versions > of software is installed. Is there a way to use Rush to do this? Yes, there's a simple example here: http://seriss.com/rush-current/examples/software-audit/ You supply the multi-platform pathnames in the script, and the script will check for their existence on all the machines it can, building a single report. An example of the .txt file report it generates: Hostname AE-6.5 AE-7.0 AE-CS3 maya2008 maya2008x64 maya8.0 maya8.5 maya8.5x64 ---------------- ------ ------ ------ -------- ----------- ------- ------- ---------- superior - - - X - - - - ontario - - - X - - - - meade X - X X - X X - erie - X X X - X X - waccabuc - X X X - X X - geneva - - - X - - - - Modify the script to taste. The script has no GUI.. it's assumed to be something only a sysadmin would run, and you hand modify the script to supply the pathnames to check for. (See the "MODIFIABLE VARIABLES" section of the script) *** ***************************************************** *** Message #1802 ***************************************************** *** ***************************************************** Subject: Re: environment variables in win xp 64bit From: murphy Date: Wed, 19 Nov 2008 03:36:13 -0500 Greg Ercolano napsal(a): > murphy wrote: >> [posted to rush.general] >> > > My guess is this is not environment oriented, but rather user > permission oriented. Probably the local Administrator does not > have access to the //skserver/oscar volume because the local > administrator's password doesn't match the one on skserver, > causing file access not to be available. > Thank you so much for your valuable help Greg! mur *** ***************************************************** *** Message #1801 ***************************************************** *** ***************************************************** Subject: Re: environment variables in win xp 64bit From: Greg Ercolano Date: Fri, 14 Nov 2008 09:03:04 -0500 murphy wrote: > [posted to rush.general] > > we did last test and rerun the rushd under normal system account. > previously it was run under Administrator account and it works now! If it's running as the system account, that means rushd will run as whomever your window manager is logged in as. If that user is "soft", that's probably why it works (see my last message). If that's the case, try configuring the rushd service to run as 'soft', so that the daemon remains running as that user, regardless of who's logged into the machine. I do not recommend you run rushd as the 'normal system account', because then rushd will not operate unless someone is logged in, and it will stop short if someone logs out/in as someone else. I'd suggest seeing my last message for a few tests. My guess is this is not environment oriented, but rather user permission oriented. Probably the local Administrator does not have access to the //skserver/oscar volume because the local administrator's password doesn't match the one on skserver, causing file access not to be available. Possible Solutions: If you want rushd to run as "administrator", then make sure the password on "snap" is the same as the local admin's password on "skserver", so that the permissions match up. *Or*, if matching the administrator's passwords are not possible, then consider configuring Rushd to run as the user 'soft' instead, since that user probably has matching passwords with skserver, and might be why it works from your DOS window tests as 'soft', (and when you have rushd set up to run as the 'system' account) -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1800 ***************************************************** *** ***************************************************** Subject: Re: environment variables in win xp 64bit From: Greg Ercolano Date: Fri, 14 Nov 2008 08:55:40 -0500 A few things catch my eye: 1) There's a subtle difference in the variable names: MAYA_PLUGIN_PATH -- variable set in both DOS and RUSH MAYA_PLUG_IN_PATH -- variable maya is complaining about Note the difference between these names "PLUGIN" vs. "PLUG_IN"; Possibly maya honors both, but in my experience, MAYA_PLUG_IN_PATH is the correct variable name. 2) The plugin path is pointing to a file server: MAYA_PLUGIN_PATH=//skserver/oscar/Maya_Custom/2008-x64/plugins Since the problem seems user oriented, I can't help but wonder if permissions are causing the trouble. If you login as the same user rushd is running as (local Administrator), can you then open a DOS window and run the render? Does this user have permission to access the plugins directory, eg. does this command work? dir \\skserver\oscar\Maya_Custom\2008-x64\plugins\ If that command fails when logged in as Administrator, but works when logged in as 'soft', the problem is likely the server 'skserver' has a different Administrator password than host 'snap', which means the local Administrator won't have access permission to that server unless the passwords match up. 3) I notice the rushd service is running as "Administrator" whereas your DOS tests are being run as "soft". If you login to the window manager as Administrator, are you able to replicate the same error rushd does? Also, if the render runs OK for soft, then try running the rushd service as "soft"; does the render then run normally? *** ***************************************************** *** Message #1799 ***************************************************** *** ***************************************************** Subject: Re: environment variables in win xp 64bit From: murphy Date: Fri, 14 Nov 2008 07:33:45 -0500 we did last test and rerun the rushd under normal system account. previously it was run under Administrator account and it works now! But I still do not understand why it does not work with Administrator. How is it possible that Maya can not see the variables under these conditions? murphy *** ***************************************************** *** Message #1798 ***************************************************** *** ***************************************************** Subject: Re: environment variables in win xp 64bit From: murphy Date: Fri, 14 Nov 2008 05:10:42 -0500 Greg Ercolano napsal(a): > ..and that will dump out the environment in the log, so you can compare > it to the dos-env.txt (above). If I compare both environments they are basically the same. All the necessary environments are fine. I'm confused what could be wrong than . . . :( You can find bot log in bottom of my mail. through rushd it is not rendered correctly because of "missing" plugin, but again, when I run the same batch from normal cmd line it renders without problem. the only difference is in users. rushd is run under Administrator and cmd line is run under user soft. But this should not be a problem. Both users have all privileges to access the plugin. thank you very much for your help murphy this is output from rushd: ========================================================================================================== rushd log ========================================================================================================== ALLUSERSPROFILE=C:\Documents and Settings\All Users CLASSPATH=.;C:\Program Files (x86)\QuickTime\QTSystem\QTJava.zip CommonProgramFiles=C:\Program Files (x86)\Common Files CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files CommonProgramW6432=C:\Program Files\Common Files COMPUTERNAME=SNAP ComSpec=C:\WINDOWS\system32\cmd.exe FP_NO_HOST_CHECK=NO HOME=/temp LOGNAME=soft MAYA_APP_DIR=//skserver/oscar/Maya_Custom MAYA_PLUGIN_PATH=//skserver/oscar/Maya_Custom/2008-x64/plugins NUMBER_OF_PROCESSORS=2 OS=Windows_NT Path=C:\Program Files\Autodesk\Maya2009\bin;C:\Program Files\Autodesk\Maya8.5\bin;C:\Program Files\Autodesk\Maya2008\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;c:\rush\bin;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Cinetools\Phantom PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH PROCESSOR_ARCHITECTURE=x86 PROCESSOR_ARCHITEW6432=AMD64 PROCESSOR_IDENTIFIER=EM64T Family 6 Model 23 Stepping 10, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=170a ProgramFiles=C:\Program Files (x86) ProgramFiles(x86)=C:\Program Files (x86) ProgramW6432=C:\Program Files PROMPT=$P$G QTJAVA=C:\Program Files (x86)\QuickTime\QTSystem\QTJava.zip RUSH_DIR=c:\rush RUSH_FIRST_FRAME=1 RUSH_FRAME=20 RUSH_HOSTNAME=snap RUSH_ISDAEMON=2024 RUSH_JOBID=snap.554 RUSH_LAST_FRAME=147 RUSH_LOGFILE=//matrix/lima/HighSpeed_SHOWREEL/Bowl/logs/15176_na/0020 RUSH_PADFRAME=0020 RUSH_RAM=1 RUSH_TMPDIR=C:/TEMP/.RUSH_TMP.17 RUSH_TRY=0 SystemDrive=C: SystemRoot=C:\WINDOWS TEMP=C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp TERM=unknown TMP=C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp USER=soft USERDOMAIN=SNAP USERNAME=Administrator USERPROFILE=C:\Documents and Settings\Administrator windir=C:\WINDOWS Starting "\\skserver\oscar\Maya2008-x64\bin\mayabatch.exe" Command Line Process Viewer/Killer/Suspender for Windows NT/2000/XP V2.03 Copyright(C) 2002-2003 Craig.Peacock@(email surpressed) Setting PriorityClass on PID 4272 'mayabatch.exe' to 64 mental ray for Maya 9.0.1 mental ray: version 3.6.51.0, 4 Dec 2007, revision 23078 Error: line 1: Plug-in, "RealflowMesher.mll", was not found on MAYA_PLUG_IN_PATH. <<<<<---------- Warning: file: //matrix/lima/HighSpeed_SHOWREEL/Bowl/scenes/Bowl_Final_oranzova_Honza.ma line 43306: Unrecognized node type 'RealflowMesh'; preserving node information during this session. File read in 0 seconds. Warning: line 1: Errors have occurred while reading this scene that may result in data loss. ================================================================================================== dos command line log ================================================================================================== ALLUSERSPROFILE=C:\Documents and Settings\All Users APPDATA=C:\Documents and Settings\soft\Application Data CLASSPATH=.;C:\Program Files (x86)\QuickTime\QTSystem\QTJava.zip CLIENTNAME=Console com.adobe.versioncue.client.applocale=cs_CZ com.adobe.versioncue.client.appname=AdobeDrive com.adobe.versioncue.client.appversion=1.0.0 CommonProgramFiles=C:\Program Files\Common Files CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files COMPUTERNAME=SNAP ComSpec=C:\WINDOWS\system32\cmd.exe FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Documents and Settings\soft LOGONSERVER=\\SNAP MAYA_APP_DIR=//skserver/oscar/Maya_Custom MAYA_PLUGIN_PATH=//skserver/oscar/Maya_Custom/2008-x64/plugins NUMBER_OF_PROCESSORS=2 OS=Windows_NT Path=C:\Program Files\Autodesk\Maya2009\bin;C:\Program Files\Autodesk\Maya8.5\bin;C:\Program Files\Autodesk\Maya2008\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;c:\rus Files (x86)\QuickTime\QTSystem\;C:\Cinetools\Phantom PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH PROCESSOR_ARCHITECTURE=AMD64 PROCESSOR_IDENTIFIER=EM64T Family 6 Model 23 Stepping 10, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=170a ProgramFiles=C:\Program Files ProgramFiles(x86)=C:\Program Files (x86) PROMPT=$P$G QTJAVA=C:\Program Files (x86)\QuickTime\QTSystem\QTJava.zip RUSH_DIR=c:\rush SESSIONNAME=Console SystemDrive=C: SystemRoot=C:\WINDOWS TEMP=C:\DOCUME~1\soft\LOCALS~1\Temp TMP=C:\DOCUME~1\soft\LOCALS~1\Temp USERDOMAIN=SNAP USERNAME=soft USERPROFILE=C:\Documents and Settings\soft windir=C:\WINDOWS Starting "\\skserver\oscar\Maya2008-x64\bin\mayabatch.exe" mental ray for Maya 9.0.1 mental ray: version 3.6.51.0, 4 Dec 2007, revision 23078 File read in 0 seconds. Result: //matrix/lima/HighSpeed_SHOWREEL/Bowl/scenes/Bowl_Final_oranzova_Honza.ma mental ray: got 8 satellite CPUs. Prog: (Mayatomr.Scene) : optimize animation detection: done (15 animated nodes) Info: (mental ray) : number of render thread(s): 2 MEM 0.0 info : heap size limit set to 3084 MB JOB 0.0 info : slave rendering OFF Info: (Mayatomr.Nodes) : custom string options found Info: (Mayatomr.Nodes) : derive mental ray global approximation settings from Maya Info: (Mayatomr.Nodes) : derive mental ray global displacement settings from Maya Info: (Mayatomr.Scene) : update lights and related shaders Prog: (Mayatomr.Scene) : DAG node: spotLight1 Prog: (Mayatomr.Scene) : DAG node: spotLight2 Prog: (Mayatomr.Scene) : DAG node: directionalLight1specular Info: (Mayatomr.Scene) : update cameras and related shaders Info: (Mayatomr.Scene) : update geometry and materials Prog: (Mayatomr.Scene) : DAG node: pPlane1 Prog: (Mayatomr.Scene) : DAG node: pPlane5 Prog: (Mayatomr.Scene) : DAG node: RealflowMesh1 Prog: (Mayatomr.Scene) : DAG node: Miska_Poly2 Info: (Mayatomr.Scene) : render camera: camera1 SCEN 0.2 progr: begin scene preprocessing for frame 22 =========================================================================================================================== *** ***************************************************** *** Message #1797 ***************************************************** *** ***************************************************** Subject: Re: environment variables in win xp 64bit From: Greg Ercolano Date: Thu, 13 Nov 2008 17:51:43 -0500 murphy wrote: > Hello, > > we have some strange problems rendering maya scene with realflow mesh > inside. > > we have a network installation of different maya versions and different > platforms. As a central point for preferences, plugins and modules we > use MAYA_APP_DIR environment variable pointing to a network disk. > > during rush render maya cries it can not find Reaflowmesh.mll plugin. > But when I run rushrender$$$.bat used by rush to render my frames and > execute it manually from cmd line it works perfectly and all plugins are > successfully loaded. Are you running that in a DOS window that's logged in as the same user the Rushd service is running as? Sometimes permissions on eg. dll files or the directories they're in are restrictive enough that only certain users can access them. > I suspect there is a huge difference running the > bat file manually and running the same file from rushd. It seems to me > system environments are not visible to mayarender when invoked from rush. > We have modified our submit to include set MAYA_APP_DIR just before > render is executed in our bat file but it did not helped. Actually, rush under windows does inherit the combined System Environment and User Environment (for the user the rushd service is running as), so assuming you've rebooted the rendering box since installing the plugin, rush should pick up all those environment variables. Check to see if the script is changing them around in ways where it might loose the values. eg: $ENV{MAYA_PLUG_IN_PATH} = "/some/path"; ..would overwrite the value of that variable if it were correctly in the system environment, whereas: $ENV{MAYA_PLUG_IN_PATH} = "/some/path;$ENV{MAYA_PLUG_IN_PATH}"; ..would *augment* the path, without stomping on it. > Now I really do not know what else could be wrong . . . . If you think it's an environment issue, try running the following from your working DOS window: set > dos-env.txt ..and then load that file up, and compare it to the environment variable dump in the rush log which you can enable by either changing the 'Print Environment:' pulldown from 'No' to 'yes' (In the submit form, click on the 'Rush' tab, and scroll down to the bottom to find that setting) Or, if you've written your own script for submitting, just stick in a command just before the render that prints out the environment, eg. in the script just before it invokes the renderer, add: if ( $G::iswindows ) { print "--- ENVIRONMENT IS:\n"; system("set"); } ..and that will dump out the environment in the log, so you can compare it to the dos-env.txt (above). *** ***************************************************** *** Message #1796 ***************************************************** *** ***************************************************** Subject: Re: Job Posting for Windows Help Desk Position From: Greg Ercolano Date: Thu, 13 Nov 2008 17:41:41 -0500 Jeff Yana wrote: > Hello Render Wizards- > > RTT USA has an opening for a Windows Help Desk Position with a background in > 3D/VFX and, of course, supporting Rush or some other application Render > Queue application. > > Does anyone have a recommendation on where to post. So far I have posted on > VFX Pro. > > Any suggestions would be welcome. You might try searching on Linked-In. Scott B. tells me he gets a lot of pings from people who find him via linkedin. *** ***************************************************** *** Message #1795 ***************************************************** *** ***************************************************** Subject: environment variables in win xp 64bit From: murphy Date: Thu, 13 Nov 2008 08:14:34 -0500 Hello, we have some strange problems rendering maya scene with realflow mesh inside. we have a network installation of different maya versions and different platforms. As a central point for preferences, plugins and modules we use MAYA_APP_DIR environment variable pointing to a network disk. during rush render maya cries it can not find Reaflowmesh.mll plugin. But when I run rushrender$$$.bat used by rush to render my frames and execute it manually from cmd line it works perfectly and all plugins are successfully loaded. I suspect there is a huge difference running the bat file manually and running the same file from rushd. It seems to me system environments are not visible to mayarender when invoked from rush. We have modified our submit to include set MAYA_APP_DIR just before render is executed in our bat file but it did not helped. Now I really do not know what else could be wrong . . . . thanks murphy *** ***************************************************** *** Message #1794 ***************************************************** *** ***************************************************** Subject: Re: Job Posting for Windows Help Desk Position From: Jeff Yana Date: Tue, 11 Nov 2008 14:39:30 -0500 > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3309248369_55485 Content-type: text/plain; charset="ISO-8859-1" Content-transfer-encoding: 8bit Thanks Craig! Jeff On 11/11/08 2:23 AM, in article 1793-rush decimal general at seriss decimal com, "Craig Allison" wrote: > This might be a good one... > > http://www.vfx-recruit.com/  > > Good luck > > Craig > > > Craig Allison > > Digital Systems & Data Manager > > The Senate Visual Effects > > Twickenham Film Studios > > St.Margarets > > Twickenham > > TW1 2AW > > > > t: 0208 607 8866  > > skype: craig_9000 > > www.senatevfx.com > > > > > On 11 Nov 2008, at 07:42, Jeff Yana wrote: > >> [posted to rush.general] >> >> Hello Render Wizards- >> >> RTT USA has an opening for a Windows Help Desk Position with a background in >> 3D/VFX and, of course, supporting Rush or some other application Render >> Queue application. >> >> Does anyone have a recommendation on where to post. So far I have posted on >> VFX Pro. >> >> Any suggestions would be welcome. >> >> Muchas Gracias. >> >> Cheers! >> >> Jeff  >> >> >> > > --B_3309248369_55485 Content-type: text/html; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable Re: Job Posting for Windows Help Desk Position Thanks Craig!

Jeff


On 11/11/08 2:23 AM, in article 1793= -rush decimal general at seriss decimal com, "Craig Allison" <craig@(email surpressed)> wrote:

<= SPAN STYLE=3D'font-size:11pt'>This might be a good one...

http://www.vfx-recruit.com/=A0

Good luck

Craig


= Craig Allison

Digital Systems &= amp; Data=A0Manager

The Senate Visual= Effects

Twickenham Film S= tudios

St.Margarets

Twickenham

TW1 2AW



t: 0208 607 8866=A0=

skype: craig_9000

www.senatevfx.com

On 11 Nov 2008, at 07:42, Jeff Yana wrote:

<= SPAN STYLE=3D'font-size:11pt'>[posted to rush.general]

Hello Render Wizards-

RTT USA has an opening for a Windows Help Desk Position with a background i= n
3D/VFX and, of course, supporting Rush or some other application Render
Queue application.

Does anyone have a recommendation on where to post. So far I have posted on=
VFX Pro.

Any suggestions would be welcome.

Muchas Gracias.

Cheers!

Jeff=A0


 
=

--B_3309248369_55485-- *** ***************************************************** *** Message #1793 ***************************************************** *** ***************************************************** Subject: Re: Job Posting for Windows Help Desk Position From: Craig Allison Date: Tue, 11 Nov 2008 05:23:34 -0500 --Apple-Mail-17--1016301247 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed This might be a good one... http://www.vfx-recruit.com/ Good luck Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 11 Nov 2008, at 07:42, Jeff Yana wrote: > [posted to rush.general] > > Hello Render Wizards- > > RTT USA has an opening for a Windows Help Desk Position with a > background in > 3D/VFX and, of course, supporting Rush or some other application > Render > Queue application. > > Does anyone have a recommendation on where to post. So far I have > posted on > VFX Pro. > > Any suggestions would be welcome. > > Muchas Gracias. > > Cheers! > > Jeff > > --Apple-Mail-17--1016301247 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1
This might be a good one...

http://www.vfx-recruit.com/=A0
Good = luck

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000



On 11 = Nov 2008, at 07:42, Jeff Yana wrote:

[posted to rush.general]

Hello = Render Wizards-

RTT USA has an opening for a Windows Help Desk = Position with a background in
3D/VFX and, = of course, supporting Rush or some other application Render
Queue application.

Does anyone = have a recommendation on where to post. So far I have posted = on
VFX Pro.

Any suggestions would be = welcome.

Muchas Gracias.

Cheers!

=A0


=

= --Apple-Mail-17--1016301247-- *** ***************************************************** *** Message #1792 ***************************************************** *** ***************************************************** Subject: Job Posting for Windows Help Desk Position From: Jeff Yana Date: Tue, 11 Nov 2008 02:42:00 -0500 Hello Render Wizards- RTT USA has an opening for a Windows Help Desk Position with a background in 3D/VFX and, of course, supporting Rush or some other application Render Queue application. Does anyone have a recommendation on where to post. So far I have posted on VFX Pro. Any suggestions would be welcome. Muchas Gracias. Cheers! Jeff *** ***************************************************** *** Message #1791 ***************************************************** *** ***************************************************** Subject: [MAC OSX] Rush still booting on macs even if RUSHSERVER=-NO- From: Greg Ercolano Date: Wed, 08 Oct 2008 20:06:35 -0400 > We wanted to disable rush from starting on some of our macs. > > We changed the /etc/hostconfig line from RUSHSERVER=-YES- > to instead read RUSHSERVER=-NO- but the daemon still boots. Mmm, looks like maybe the perms have changed on the /etc/rc.common file so that its 'read only' now, instead of executable. Currently the Mac rush boot scripts (102.42a9 and older) test if /etc/rc.common is executable, which is preventing it from working right. Simple fix; just make this small change to the following line in the Rush boot script "/Library/StartupItems/Rush/Rush" BEFORE: if [ -x /etc/rc.common ]; then . /etc/rc.common; fi AFTER: if [ -r /etc/rc.common ]; then . /etc/rc.common; fi ^^ Then it should work correctly. *** ***************************************************** *** Message #1790 ***************************************************** *** ***************************************************** Subject: [Q+A] How can I run commands when a render has exceeded its 'maxtime'? From: Greg Ercolano Date: Mon, 29 Sep 2008 19:23:56 -0400 > We've been using rush's 'maxtime' setting to kill renders that > take too long to run. > > But we have a situation where one of our render scripts interacts > with a database at the start of a render, and again at the end. > > We'd like to be able to run special database commands if the render > took too long. > > Is there a way to do that? You really wouldn't want to use the rush 'maxtime' to do this. To do what you want, I'd advise modifying the submit script so that if someone does specify the 'Max Time:' field in the submit form, the script handles it itself, instead of to rush's 'maxtime' command. To do this, pass the maxtime value as a command line argument to your render script, and have the render script parse it, and run the process in the background while the render script keeps an eye on the wallclock time in a while() loop, so when the time gets above the maxtime value, the render script can stop the child, and do the database tweak as a post-operation. This, instead of using rush's own builtin 'maxtime' submit command which is not trappable; when rush's maxtime is triggered, the render process is simply killed. This is because there is no clean 'cross platform' mechanism rush can use to reliably notify a process it is about to be killed. Windows and unix vary widely in their techniques, and some work better than others, depending on the job being run. This is fairly easy to do in your script, and this way your script can do all kinds of complex handling when the timer expires. The following working Unix perl example should give you an idea of how to approach the render script side; this way you'll have full control over what to do when the child process takes too long, what kill signal to use, etc: ---------------------------------------------------------------------- #!/usr/bin/perl -w use strict; use POSIX ":sys_wait_h"; # unix only ### ### UNIX EXAMPLE: HOW TO LIMIT A CHILD PROCESS'S EXECUTION TIME ### erco 09/29/08 ### # SUBROUTINE: RUN CHILD, KILL IF TAKES LONGER THAN 'maxsecs' # $1 - command to run # $2 - max time in seconds # sub RunChild($$) { my ($command, $maxsecs) = @_; my $pid = fork(); if ( $pid == 0 ) { # CHILD exec($command) || die "Could not exec($command)"; } else { # PARENT -- if child takes longer than maxsecs, kill it print "--- CHILD STARTED: PID=$pid, COMMAND='$command'\n"; while ( 1 ) { # CHECK IF CHILD DONE my $ret = waitpid($pid, WNOHANG); if ( $ret == -1 ) { print "waitpid() failed: $?\n"; next; } if ( $ret == $pid ) { # CHILD DONE printf("--- CHILD EXITED: STATUS=%d, EXITCODE=%d\n", $?, ($?>>8)); return($?); } # MAX TIME EXPIRED? KILL CHILD, DONE if ( $maxsecs-- == 0 ) { print "--- CHILD TOOK TOO LONG: KILLING PID $pid\n"; kill(-9, $pid); return(-1); } # KEEP WAITING sleep(1); print "--- Waiting for child.. $maxsecs\n"; } } #NOTREACHED# } ### MAIN ### { # TWEAK THESE AS NEEDED my $command = "(sleep 30 && exit 33)"; # command to run and exit code my $maxsecs = 5; # max time to wait # RUN CHILD PROCESS my $ret = RunChild($command, $maxsecs); # DONE print "--- DONE: RET=$ret\n"; exit(0); } ---------------------------------------------------------------------- The above example is written to assume the child takes too long so as to show what happens in that case. If you change the 'sleep 30' to 'sleep 3', then the child will execute normally, and the process's exit code of 33 will be printed. Note the use of ($? >> 8) to get the exit code, as opposed to the raw 'status' returned by $?. Read the unix docs on wait() and friends wait4, waitpid, etc. for more info on this. These extra bits can tell you if the process core dumped or was signaled, instead of exiting on its own, should you need to make a distinction. WINDOWS CAVEAT If you're on Windows, I think you can probably do a timer technique similar to the above using perl's Win32::Process::Create() stuff; see .common.pl for an example, and activestate perl's docs on the WIN32 module for more info. *** ***************************************************** *** Message #1789 ***************************************************** *** ***************************************************** Subject: Re: [OsX 10.5.5] After Effect CS3 render issue. From: Stephan Kosinski Date: Mon, 29 Sep 2008 17:12:48 -0400 Thank you Greg, I think I will go like that for now, ignoring the warning. I was just trying to understand but maybe I'm too curious :) I'm not yet advanced enough to parse and use the log file inforamtion in order to change the ownership of the file but I'll keep everyone up-to-date if a solution comes out. Have a nice day. Stephan. Greg Ercolano wrote: > Stephan Kosinski wrote: >> Hello Greg, hello rush folks! >> >> Thank you for the tip. Actaully it now renders the images >> properly 2 of the 3 machines I'm testing AErender on. >> >> On the two rendering machines I still get the message: >> >> Mon Sep 29 10:52:51 loganM005 aerender[5254] : 3891612: (connectAndCheck) Untrusted apps are not allowed to connect to or launch Window Server before login. >> _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. >> >> .... But it renders the current image properly. > > Sounds like no one is logged into the window manager on that machine. > I'd ignore it as just a warning. > > Definitely lobby for Adobe to fix aerender so that it does not need > to interact with the window manager. They really need to fix that > so that aerender can be a true command-line-only app. All this > is because the app is really still interacting with the window manager. > >> As mentionned in the link you sent the rendered images belong to root. >> On the one of the rendering machine (loganM014)I get this message: >> >> Executing: aerender -project >> "/Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep" >> -comp "test_ShaveAndHairCut" -s 3 -e 3 >> _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. >> PROGRESS: Launching After Effects... >> aerender ERROR -600: GetProcessForPID failed at line 847 >> >> .... and the image does not render. > > Hmm, I haven't seen that one, but it looks like there are > many hits on google for that error message. Apparently it comes > up in different contexts. > > My guess is it has to do with someone being logged in over > the window manager or remote desktop. > > I think you'll have to ask Adobe what that means, as it says it > failed "at line 847", but doesn't say which file. > >> What is strange is that when I launch the line in a ssh session the >> images renders well: >> >> [loganM014:/Applications/Adobe After Effects CS3] dak% ./aerender >> -project >> "/Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep" >> -comp "test_ShaveAndHairCut" -s 3 -e 3 >> PROGRESS: Launching After Effects... >> Mon Sep 29 11:30:55 loganM014 aerender[2265] : 3891612: >> (connectAndCheck) Untrusted apps are not allowed to connect to or launch >> Window Server before login. >> _RegisterApplication(), FAILED TO establish the default connection to >> the WindowServer, _CGSDefaultConnection() is NULL. >> PROGRESS: ...After Effects successfully launched >> PROGRESS: 9/29/08 11:31:00 AM: Starting composition test_ShaveAndHairCut. > > That is strange, I'd say try logging into the window manager > on that machine, and see if that changes things. > >> - What will be the best way to add "chown" and a "chmod" command at the >> current image completion in order to make to rendered image belongs to >> the user who launched the job? > > It sounds like you're asking if you can chown the rendered image > so that it isn't owned by root, due to the aerender setuid hack. > (you should not need "chmod", the umask can control that) > > From the perl script, I'd say 'I don't recommend you try', > as there are too many things to make this hard: > > 1) You'd have to parse the image pathname from the log > which is in OS9 format, eg: > dakperso:dakperso_BASE:IMAGES:USERS:dak:TRACKING_TEST:work:test_AFX_rush.[#####].tga > ..so you'd have to translate that into unix format, > and convert the [#####] to the frame number. > > 2) You have to be sure the pathname you parse is valid; > you don't want to be chowning random files from the log > that might 'look like' a pathname. It would be more reliable > if AE printed some unique leader on the pathname line, like: > > Writing output image: dakperso:dakperso_BASE:IMAGES:USERS:dak:TRACKING_TEST:work:test_AFX_rush.[#####].tga > ^^^^^^^^^^^^^^^^^^^^^ > > 3) If the perl script is running as a normal user, trying to use > chown on a root-owned file will likely fail unless the chown > command is run as root itself (or sudo) > > All the above together is fairly complicated, and a little too scary > if you ask me. > > However, if you know how to hack AfterEffect's javascript, then you > might be able to hack the chown() command in there, since aerender > will be running as root, and within AE, the pathname will likely > be more easy to access. But I have no information on this. Possibly > Adobe support, or one of their newsgroups can advise on how to do this. > *** ***************************************************** *** Message #1788 ***************************************************** *** ***************************************************** Subject: Re: [OsX 10.5.5] After Effect CS3 render issue. From: Greg Ercolano Date: Mon, 29 Sep 2008 16:30:49 -0400 Greg Ercolano wrote: >> _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. BTW, citing some specifics, using Apple's own docs as a source: http://developer.apple.com/technotes/tn2005/tn2083.html#SECPERMISSIONTOCONNECT Since links at Microsoft and Apple have a way of going stale constantly, I'm quoting the relevant passage from that text here: *** Technical Note TN2083 *** [..] Permission To Connect [..] A process can only use the global window server service if its EUID is 0 (it's running as root) or matches the UID of the console user. All other users are barred from using it. For a demonstration of this, you can SSH to your own machine and try to run Activity Monitor from your shell. Listing 6 shows an example of doing this from Terminal. The first attempt to run Activity Monitor command works because it's running as the same user as Terminal. The second attempt fails because the test user (mrgumby) does not match the console user, and thus cannot access the global window server service. Listing 6: Accessing the window server from console and non-console users --------------------------------------------------------------------------- $ ssh ${USER}@localhost Password:******** Last login: Wed Jun 20 11:49:23 2007 $ id uid=502(quinn) gid=20(staff) groups=20(staff),81(_appserveradm), 104(com.apple.sharepoint.group.1),79(_appserverusr),80(admin), 101(com.apple.access_remote_ae),103(com.apple.access_ssh-disabled) $ ls -l /dev/console crw------- 1 quinn staff 0, 0 Jun 20 11:50 /dev/console $ # Launch Activity Monitor and then quit it. $ /Applications/Utilities/Activity\ Monitor.app/Contents/MacOS/Activity\ Monitor $ logout Connection to localhost closed. $ ssh mrgumby@localhost [..] $ id uid=503(mrgumby) gid=20(staff) groups=20(staff),105(com.apple.sharepoint.group.2), 104(com.apple.sharepoint.group.1) $ ls -l /dev/console crw------- 1 quinn quinn 0, 0 Oct 3 21:31 /dev/console $ # Activity Monitor fails to launch at all. $ /Applications/Utilities/ctivity\ Monitor.app/Contents/MacOS/Act_RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. 2007-06-20 11:54:31.798 Activity Monitor[863:10b] An uncaught exception was raised [...] This limitation makes it very hard to reliably use the global window service because: * Standard security practice is that daemons should not run as root; rather, they should be run by a dedicated user (that is, the wombatd daemon is run by a dedicated _wombat user). Also, standard security practice dictates that programs running as root should try to reduce their attack surface by limiting the list of frameworks that they use. Thus, in general, programs running as root should not use high-level frameworks that rely on the window server, like AppKit and HIToolbox. So, solving the problem by running as root is a security no-no. * There is no easy way to solve the problem by running your daemon as the console user because, with fast user switching, the console user can change at any time. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1787 ***************************************************** *** ***************************************************** Subject: Re: [OsX 10.5.5] After Effect CS3 render issue. From: Greg Ercolano Date: Mon, 29 Sep 2008 15:35:50 -0400 Stephan Kosinski wrote: > Hello Greg, hello rush folks! > > Thank you for the tip. Actaully it now renders the images > properly 2 of the 3 machines I'm testing AErender on. > > On the two rendering machines I still get the message: > > Mon Sep 29 10:52:51 loganM005 aerender[5254] : 3891612: (connectAndCheck) Untrusted apps are not allowed to connect to or launch Window Server before login. > _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. > > .... But it renders the current image properly. Sounds like no one is logged into the window manager on that machine. I'd ignore it as just a warning. Definitely lobby for Adobe to fix aerender so that it does not need to interact with the window manager. They really need to fix that so that aerender can be a true command-line-only app. All this is because the app is really still interacting with the window manager. > As mentionned in the link you sent the rendered images belong to root. > On the one of the rendering machine (loganM014)I get this message: > > Executing: aerender -project > "/Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep" > -comp "test_ShaveAndHairCut" -s 3 -e 3 > _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. > PROGRESS: Launching After Effects... > aerender ERROR -600: GetProcessForPID failed at line 847 > > .... and the image does not render. Hmm, I haven't seen that one, but it looks like there are many hits on google for that error message. Apparently it comes up in different contexts. My guess is it has to do with someone being logged in over the window manager or remote desktop. I think you'll have to ask Adobe what that means, as it says it failed "at line 847", but doesn't say which file. > What is strange is that when I launch the line in a ssh session the > images renders well: > > [loganM014:/Applications/Adobe After Effects CS3] dak% ./aerender > -project > "/Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep" > -comp "test_ShaveAndHairCut" -s 3 -e 3 > PROGRESS: Launching After Effects... > Mon Sep 29 11:30:55 loganM014 aerender[2265] : 3891612: > (connectAndCheck) Untrusted apps are not allowed to connect to or launch > Window Server before login. > _RegisterApplication(), FAILED TO establish the default connection to > the WindowServer, _CGSDefaultConnection() is NULL. > PROGRESS: ...After Effects successfully launched > PROGRESS: 9/29/08 11:31:00 AM: Starting composition test_ShaveAndHairCut. That is strange, I'd say try logging into the window manager on that machine, and see if that changes things. > - What will be the best way to add "chown" and a "chmod" command at the > current image completion in order to make to rendered image belongs to > the user who launched the job? It sounds like you're asking if you can chown the rendered image so that it isn't owned by root, due to the aerender setuid hack. (you should not need "chmod", the umask can control that) From the perl script, I'd say 'I don't recommend you try', as there are too many things to make this hard: 1) You'd have to parse the image pathname from the log which is in OS9 format, eg: dakperso:dakperso_BASE:IMAGES:USERS:dak:TRACKING_TEST:work:test_AFX_rush.[#####].tga ..so you'd have to translate that into unix format, and convert the [#####] to the frame number. 2) You have to be sure the pathname you parse is valid; you don't want to be chowning random files from the log that might 'look like' a pathname. It would be more reliable if AE printed some unique leader on the pathname line, like: Writing output image: dakperso:dakperso_BASE:IMAGES:USERS:dak:TRACKING_TEST:work:test_AFX_rush.[#####].tga ^^^^^^^^^^^^^^^^^^^^^ 3) If the perl script is running as a normal user, trying to use chown on a root-owned file will likely fail unless the chown command is run as root itself (or sudo) All the above together is fairly complicated, and a little too scary if you ask me. However, if you know how to hack AfterEffect's javascript, then you might be able to hack the chown() command in there, since aerender will be running as root, and within AE, the pathname will likely be more easy to access. But I have no information on this. Possibly Adobe support, or one of their newsgroups can advise on how to do this. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1786 ***************************************************** *** ***************************************************** Subject: Re: [OsX 10.5.5] After Effect CS3 render issue. From: Stephan Kosinski Date: Mon, 29 Sep 2008 14:49:50 -0400 Hello Greg, hello rush folks! Thank you for the tip. Actaully it now renders the images properly 2 of the 3 machines I'm testing AErender on. On the two rendering machines I still get the message: Executing: aerender -project "/Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep" -comp "test_ShaveAndHairCut" -s 4 -e 4 Mon Sep 29 10:52:51 loganM005 aerender[5254] : 3891612: (connectAndCheck) Untrusted apps are not allowed to connect to or launch Window Server before login. _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. .... But it renders the current image properly. As mentionned in the link you sent the rendered images belong to root. On the one of the rendering machine (loganM014)I get this message: Executing: aerender -project "/Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep" -comp "test_ShaveAndHairCut" -s 3 -e 3 _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. PROGRESS: Launching After Effects... aerender ERROR -600: GetProcessForPID failed at line 847 .... and the image does not render. What is strange is that when I launch the line in a ssh session the images renders well: [loganM014:/Applications/Adobe After Effects CS3] dak% ./aerender -project "/Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep" -comp "test_ShaveAndHairCut" -s 3 -e 3 PROGRESS: Launching After Effects... Mon Sep 29 11:30:55 loganM014 aerender[2265] : 3891612: (connectAndCheck) Untrusted apps are not allowed to connect to or launch Window Server before login. _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. PROGRESS: ...After Effects successfully launched PROGRESS: 9/29/08 11:31:00 AM: Starting composition test_ShaveAndHairCut. PROGRESS: Render Settings: Best Settings PROGRESS: Quality: Best PROGRESS: Resolution: Full PROGRESS: Size: 640 x 480 PROGRESS: Proxy Use: Use No Proxies PROGRESS: Effects: Current Settings PROGRESS: Disk Cache: Read Only PROGRESS: Color Depth: Current Settings PROGRESS: Frame Blending: On For Checked Layers PROGRESS: Field Render: Off PROGRESS: Pulldown: Off PROGRESS: Motion Blur: On For Checked Layers PROGRESS: Use OpenGL: Off PROGRESS: Solos: Current Settings PROGRESS: Time Span: Custom PROGRESS: Start: 00003 PROGRESS: End: 00003 PROGRESS: Duration: 00001 PROGRESS: Frame Rate: 30.00 (comp) PROGRESS: Guide Layers: All Off PROGRESS: Storage Overflow: On PROGRESS: Skip Existing Files: Off PROGRESS: PROGRESS: Output Module: Lossless PROGRESS: Output To: PROJECTS:dakperso:dakperso_BASE:IMAGES:USERS:dak:TRACKING_TEST:work:test_AFX_rush.[#####].tga PROGRESS: Format: Targa Sequence PROGRESS: Output Info: - PROGRESS: Start Frame: 3 PROGRESS: Output Audio: - PROGRESS: Channels: RGB + Alpha PROGRESS: Depth: Millions of Colors+ PROGRESS: Color: Straight PROGRESS: Stretch: - PROGRESS: Crop: - PROGRESS: Final Size: 640 x 480 PROGRESS: Profile: - PROGRESS: Embed Profile: PROGRESS: PROGRESS: Post-Render Action: None PROGRESS: PROGRESS: PROGRESS: PROGRESS: PROGRESS: 00003 (1): 0 Seconds PROGRESS: 9/29/08 11:31:01 AM: Finished composition test_ShaveAndHairCut. PROGRESS: Total Time Elapsed: 1 Seconds So here are my two questions: - What could I have omitted when I did install rush on this machine? - What will be the best way to add "chown" and a "chmod" command at the current image completion in order to make to rendered image belongs to the user who launched the job? Is it possible to it in the command line of the AfterEffect submit script? command perl $G::self -render $in{AEVersion} $in{ScenePath} $in{CompName} $in{OutputPath} $batchframes $batchend $in{Retries} $in{RetryBehavior} $in{MaxLogSize} $in{PrintEnvironment} $in{RenderFlags} It will be a dirty workaround but it would be totally invisible for users and allow them to delete their images without troubles. Thank you for your help. Stephan. Greg Ercolano wrote: > Stephan Kosinski wrote: >> While I have to problem in rendering some maya scenes I have an >> issue with AfterEffect I don't know how to workaround. (well, I'm pretty >> lame on OsX...) >> >> When I submit a job to one of the Mac station, here is what I get >> this error: >> >> _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. > > Pretty sure that's a variation of this problem: > http://www.seriss.com/rush-current/issues-afterfx-6.5/index.html#OSX > > AE is odd in that it needs access to the window manager > during rendering, on both Windows and OSX, so it's basically > a permission issue with AE not able to access the window manager. > > The fix is most likely just a matter of: > > cd '/Applications/Adobe After Effects 6.5/' > chown 0:0 aerender > chmod 4755 aerender > > ..just change the "Adobe After Effects 6.5" path to the > appropriate CS3 pathname. > *** ***************************************************** *** Message #1785 ***************************************************** *** ***************************************************** Subject: Re: [OsX 10.5.5] After Effect CS3 render issue. From: Greg Ercolano Date: Sat, 27 Sep 2008 03:04:02 -0400 Stephan Kosinski wrote: > While I have to problem in rendering some maya scenes I have an > issue with AfterEffect I don't know how to workaround. (well, I'm pretty > lame on OsX...) > > When I submit a job to one of the Mac station, here is what I get > this error: > > _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. Pretty sure that's a variation of this problem: http://www.seriss.com/rush-current/issues-afterfx-6.5/index.html#OSX AE is odd in that it needs access to the window manager during rendering, on both Windows and OSX, so it's basically a permission issue with AE not able to access the window manager. The fix is most likely just a matter of: cd '/Applications/Adobe After Effects 6.5/' chown 0:0 aerender chmod 4755 aerender ..just change the "Adobe After Effects 6.5" path to the appropriate CS3 pathname. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1784 ***************************************************** *** ***************************************************** Subject: [OsX 10.5.5] After Effect CS3 render issue. From: Stephan Kosinski Date: Fri, 26 Sep 2008 22:13:14 -0400 Hello Greg, Hello all I'm working on deploying rush for OsX in order to be able submit some Maya and some AfterEffect renders. While I have to problem in rendering some maya scenes I have an issue with AfterEffect I don't know how to workaround. (well, I'm pretty lame on OsX...) When I submit a job to one of the Mac station, here is what I get this error: _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. Here is the full log: ########################################################################################################################## ### ### loganm014.3: 0002 ### --------------- Rush 102.42a9 -------------- -- Host: loganm005 -- Pid: 2296 -- Title: test_AFX2 -- Jobid: loganm014.3 -- Frame: 0002 -- Tries: 7 -- Owner: dak (1113/20) -- RunningAs: dak (1113/20) -- Priority: 1 -- Nice: 10 -- Tmpdir: /var/tmp/.RUSH_TMP.32 -- LogFile: /Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep.log/0002 -- Command: perl /Volumes/logan_homes/logan/etc/RUSH_COMMON/OSX/examples/submit-afterfx.pl -render - /Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep test_ShaveAndHairCut - 1 0 3 Fail 0 off -- Started: Fri Sep 26 18:51:47 2008 -------------------------------------------- SCENEPATH: /Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep COMP: test_ShaveAndHairCut OUTPUTDIR: - AEVERSION: - RENDERFLAGS: BATCHFRAMES: 1 (2-2) RETRIES: 3 (Fail after 3 retries) PATH: /Applications/Adobe After Effects 6.5:/Applications/Adobe After Effects 7.0:/Applications/Adobe After Effects CS3:/usr/local/rush/bin:/usr/local/rush/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/libexec:/usr/local/rush/bin Executing: aerender -project "/Volumes/PROJECTS/dakperso/dakperso_BASE/2d/AFTER_EFFECTS/dak/TRACKING_TEST/project_files/rush_test.aep" -comp "test_ShaveAndHairCut" -s 2 -e 2 _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. PROGRESS: Launching After Effects... PROGRESS: ...After Effects successfully launched --- AERENDER SUCCEEDS ########################################################################################################################## ...............But actually it did not, succeed. What is really strange is that as soon as I physically log onto the the machine the render is going through perfectly and when I logout it refuses to launch again. I have to mention that we are using Open Directory to authenticate. I have allowed "all incoming connections" in the security tab as well as the "remote login" from any user. Anyone got this kind of trouble already? Thank you very much for your help. Stephan. *** ***************************************************** *** Message #1783 ***************************************************** *** ***************************************************** Subject: Re: [OSX/Admin] How to create a custom boot script to mount a fileserver From: Greg Ercolano Date: Thu, 25 Sep 2008 19:44:46 -0400 Greg Ercolano wrote: > [steps to create a boot script that mounts drives] Eric Jordan writes in to mention that on Leopard you can also use the 'Directory Utility' to mount drives NFS drives, eg. "/Applications/Utilities/Directory Utility.app" I have not tried this myself, but it sounds like a GUI way to do the equivalent. Thing is, I'm not sure at what point in the boot procedure this tool implements mounts to come up, and/or what the boot dependency is called, so that the Rush boot script can be modified to be dependent on this, to ensure rush isn't started until after the mount successfully comes up. *** ***************************************************** *** Message #1782 ***************************************************** *** ***************************************************** Subject: Re: [SUBMIT WINDOW] How to change the default size. From: Stephan Kosinski Date: Tue, 23 Sep 2008 21:40:54 -0400 Hello Greg, Thank you very much! Greg Ercolano wrote: > Stephan Kosinski wrote: >> I'm using the submit-maya.pl script a lot on Linux and the ScenePath, >> Project, ImageDirectory and MayaFlags field where a little bit short for >> long paths to fit in them. >> >> I have succeeded making the fields longer by adding some "__" in the >> perl submit script but unfortunately, when I launch the script in need >> to resize the window by hand to access the "browse" button. > > Yes; the 'ascii art' style input forms you see in the submit-xxx scripts > (which use that "___" syntax to define the submit form) indeed have a > "preset size" which all the submit scripts try to follow. > > That preset size of 700 is defined in the CreateInputForm() function > in the .common.pl file, which you'll find in the same directory as > your submit-maya.pl script. > > So to change it, just edit the .common.pl file, in the same dir as > your submit-maya.pl script, and find the line that reads: > > my $win_w = 700; # width of window > > ..and change the 700 to a larger value to make the window wider. > > This will then affect *all* the submit-xxx scripts that are 'ascii art' > oriented, which is most of them. > >> I've checked the method in "input-example.pl" but it does not seem to >> be the same in the maya one. > > Yes, that example does not use the 'ascii art' technique which > most of the submit scripts now use (for brevity). > > The ascii art is translated by the above 'CreateInputForm()' function > into the command language that the input-example.pl program uses. > This 'command language' is what all the submit scripts /used/ to use, > before I realized it made the scripts much easier to read and modify > by using the newer ascii art syntax, which is what they all use now. > > That's why it's a bit different. > *** ***************************************************** *** Message #1781 ***************************************************** *** ***************************************************** Subject: Re: [SUBMIT WINDOW] How to change the default size. From: Greg Ercolano Date: Tue, 23 Sep 2008 19:04:00 -0400 Stephan Kosinski wrote: > I'm using the submit-maya.pl script a lot on Linux and the ScenePath, > Project, ImageDirectory and MayaFlags field where a little bit short for > long paths to fit in them. > > I have succeeded making the fields longer by adding some "__" in the > perl submit script but unfortunately, when I launch the script in need > to resize the window by hand to access the "browse" button. Yes; the 'ascii art' style input forms you see in the submit-xxx scripts (which use that "___" syntax to define the submit form) indeed have a "preset size" which all the submit scripts try to follow. That preset size of 700 is defined in the CreateInputForm() function in the .common.pl file, which you'll find in the same directory as your submit-maya.pl script. So to change it, just edit the .common.pl file, in the same dir as your submit-maya.pl script, and find the line that reads: my $win_w = 700; # width of window ..and change the 700 to a larger value to make the window wider. This will then affect *all* the submit-xxx scripts that are 'ascii art' oriented, which is most of them. > I've checked the method in "input-example.pl" but it does not seem to > be the same in the maya one. Yes, that example does not use the 'ascii art' technique which most of the submit scripts now use (for brevity). The ascii art is translated by the above 'CreateInputForm()' function into the command language that the input-example.p