*** ***************************************************** *** Message #1956 ***************************************************** *** ***************************************************** Subject: Re: Server hostname change From: Daniel Browne Date: Sun, 08 Aug 2010 20:22:17 -0400 Hi Greg, Sounds then like we'll have to wait until we make the transition to a new l= icense server to change the names around. -Dan On Aug 7, 2010, at 12:54 AM, Greg Ercolano wrote: > [posted to rush.general] >=20 > Hi Daniel, >=20 > Rush's permanent floating licenses are locked to the > hostname/hostid pair, so changing either will disable > the license. >=20 > I can set you up with a temp license for the new hostname/hostid > combo, but an actual license migration would be needed to make > the change stick which may incur an administration fee, depending > on how many migrations you've had so far. >=20 > I'd suggest keeping the official hostname for the license server > the same (it is a license server after all ;) and making hostname > aliases or CNAMEs for the new names so that both resolve. >=20 > If you need the new name for samba to resolve, you can > likely hardcode 'vegas' into the smb.conf file for the > new machine. >=20 > Daniel Browne wrote: >> Sorry to bother you on a weekend, but I'm shifting my servers around, sp= lit=3D >> ting our database and intranet off onto a different server from licenses= an=3D >> d Rush. As a result I've had to rename the license/Rush server so that e= ver=3D >> yone's web browser points instead to the new one. This seems to now be b= rea=3D >> king Rush. The server and/or client daemons cannot resolve the host name= in=3D >> ternally, even though it's set correctly everywhere including in the old= se=3D >> rver's /etc/hostconfig. The original rush server was named "vegas", now = it'=3D >> s renamed "oldvegas". Here's what appears in the server log: >=20 > I'm surprised you're getting these errors with the hostname hard coded > in /etc/hostconfig: >=20 >> 08/06,23:37:32 INIT.D Local hostname oldvegas won't resolve (ping: c= annot resolve oldvegas: Unknown host): 10 sec retries >> 08/06,23:37:42 INIT.D Local hostname oldvegas won't resolve (ping: c= annot resolve oldvegas: Unknown host): 10 sec retries >=20 > ..but they're benign, since the hostname eventually resolves > after about 20 seconds. >=20 > But I find it interesting that the OSX boot procedure has the > OS up for that long without it having set its own hostname. > Likely the machine's official hostname is 'localhost' during this time, > from what I've seen from boot consoles on OSX. >=20 > This is the real problem, though: >=20 >> 08/06,23:38:04 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 = not in oldvegas:/usr/local/rush/etc/hosts >> 08/06,23:38:04 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com= '[vegas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) >> 08/06,23:38:04 LICENSE no servers could validate license (30 sec retr= ies ) >=20 > ..those errors are the license trying to resolve 'vegas'. >=20 > Contact me off list with the output of: >=20 > rushd -hostid >=20 > ..from the new machine, and I can set you up with a temp license > and migration instructions. >=20 > --=20 > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed)ext.23 > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) *** ***************************************************** *** Message #1955 ***************************************************** *** ***************************************************** Subject: Re: Server hostname change From: Daniel Browne Date: Sat, 07 Aug 2010 04:18:09 -0400 Ok, I'll see if switching back the hostname will work. Thanks greg. On Aug 7, 2010, at 12:54 AM, Greg Ercolano wrote: > [posted to rush.general] >=20 > Hi Daniel, >=20 > Rush's permanent floating licenses are locked to the > hostname/hostid pair, so changing either will disable > the license. >=20 > I can set you up with a temp license for the new hostname/hostid > combo, but an actual license migration would be needed to make > the change stick which may incur an administration fee, depending > on how many migrations you've had so far. >=20 > I'd suggest keeping the official hostname for the license server > the same (it is a license server after all ;) and making hostname > aliases or CNAMEs for the new names so that both resolve. >=20 > If you need the new name for samba to resolve, you can > likely hardcode 'vegas' into the smb.conf file for the > new machine. >=20 > Daniel Browne wrote: >> Sorry to bother you on a weekend, but I'm shifting my servers around, sp= lit=3D >> ting our database and intranet off onto a different server from licenses= an=3D >> d Rush. As a result I've had to rename the license/Rush server so that e= ver=3D >> yone's web browser points instead to the new one. This seems to now be b= rea=3D >> king Rush. The server and/or client daemons cannot resolve the host name= in=3D >> ternally, even though it's set correctly everywhere including in the old= se=3D >> rver's /etc/hostconfig. The original rush server was named "vegas", now = it'=3D >> s renamed "oldvegas". Here's what appears in the server log: >=20 > I'm surprised you're getting these errors with the hostname hard coded > in /etc/hostconfig: >=20 >> 08/06,23:37:32 INIT.D Local hostname oldvegas won't resolve (ping: c= annot resolve oldvegas: Unknown host): 10 sec retries >> 08/06,23:37:42 INIT.D Local hostname oldvegas won't resolve (ping: c= annot resolve oldvegas: Unknown host): 10 sec retries >=20 > ..but they're benign, since the hostname eventually resolves > after about 20 seconds. >=20 > But I find it interesting that the OSX boot procedure has the > OS up for that long without it having set its own hostname. > Likely the machine's official hostname is 'localhost' during this time, > from what I've seen from boot consoles on OSX. >=20 > This is the real problem, though: >=20 >> 08/06,23:38:04 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 = not in oldvegas:/usr/local/rush/etc/hosts >> 08/06,23:38:04 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com= '[vegas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) >> 08/06,23:38:04 LICENSE no servers could validate license (30 sec retr= ies ) >=20 > ..those errors are the license trying to resolve 'vegas'. >=20 > Contact me off list with the output of: >=20 > rushd -hostid >=20 > ..from the new machine, and I can set you up with a temp license > and migration instructions. >=20 > --=20 > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed)ext.23 > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) *** ***************************************************** *** Message #1954 ***************************************************** *** ***************************************************** Subject: Re: Server hostname change From: Greg Ercolano Date: Sat, 07 Aug 2010 03:54:46 -0400 Hi Daniel, Rush's permanent floating licenses are locked to the hostname/hostid pair, so changing either will disable the license. I can set you up with a temp license for the new hostname/hostid combo, but an actual license migration would be needed to make the change stick which may incur an administration fee, depending on how many migrations you've had so far. I'd suggest keeping the official hostname for the license server the same (it is a license server after all ;) and making hostname aliases or CNAMEs for the new names so that both resolve. If you need the new name for samba to resolve, you can likely hardcode 'vegas' into the smb.conf file for the new machine. Daniel Browne wrote: > Sorry to bother you on a weekend, but I'm shifting my servers around, split= > ting our database and intranet off onto a different server from licenses an= > d Rush. As a result I've had to rename the license/Rush server so that ever= > yone's web browser points instead to the new one. This seems to now be brea= > king Rush. The server and/or client daemons cannot resolve the host name in= > ternally, even though it's set correctly everywhere including in the old se= > rver's /etc/hostconfig. The original rush server was named "vegas", now it'= > s renamed "oldvegas". Here's what appears in the server log: I'm surprised you're getting these errors with the hostname hard coded in /etc/hostconfig: > 08/06,23:37:32 INIT.D Local hostname oldvegas won't resolve (ping: cannot resolve oldvegas: Unknown host): 10 sec retries > 08/06,23:37:42 INIT.D Local hostname oldvegas won't resolve (ping: cannot resolve oldvegas: Unknown host): 10 sec retries ..but they're benign, since the hostname eventually resolves after about 20 seconds. But I find it interesting that the OSX boot procedure has the OS up for that long without it having set its own hostname. Likely the machine's official hostname is 'localhost' during this time, from what I've seen from boot consoles on OSX. This is the real problem, though: > 08/06,23:38:04 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 not in oldvegas:/usr/local/rush/etc/hosts > 08/06,23:38:04 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com'[vegas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) > 08/06,23:38:04 LICENSE no servers could validate license (30 sec retries ) ..those errors are the license trying to resolve 'vegas'. Contact me off list with the output of: rushd -hostid ..from the new machine, and I can set you up with a temp license and migration instructions. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed)ext.23 Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1953 ***************************************************** *** ***************************************************** Subject: Server hostname change From: Daniel Browne Date: Sat, 07 Aug 2010 03:02:56 -0400 Hi Greg, Sorry to bother you on a weekend, but I'm shifting my servers around, split= ting our database and intranet off onto a different server from licenses an= d Rush. As a result I've had to rename the license/Rush server so that ever= yone's web browser points instead to the new one. This seems to now be brea= king Rush. The server and/or client daemons cannot resolve the host name in= ternally, even though it's set correctly everywhere including in the old se= rver's /etc/hostconfig. The original rush server was named "vegas", now it'= s renamed "oldvegas". Here's what appears in the server log: 08/06,23:37:31 INIT.D Startup script '/usr/local/rush/etc/S99rush start= ' 08/06,23:37:31 INIT.D Executing: ( /usr/local/rush/etc/S99rush waitloca= l && cd /usr/local/rush/var && /usr/libexec/StartupItemContext /usr/local/r= ush/bin/rushd ) & 08/06,23:37:32 INIT.D Local hostname oldvegas won't resolve (ping: cann= ot resolve oldvegas: Unknown host): 10 sec retries 08/06,23:37:42 INIT.D Local hostname oldvegas won't resolve (ping: cann= ot resolve oldvegas: Unknown host): 10 sec retries 08/06,23:37:52 INIT.D Local hostname oldvegas won't resolve (ping: cann= ot resolve oldvegas: Unknown host): 10 sec retries 08/06,23:38:04 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 not= in oldvegas:/usr/local/rush/etc/hosts 08/06,23:38:04 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com'[v= egas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) 08/06,23:38:04 LICENSE no servers could validate license (30 sec retries= ) 08/06,23:38:34 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 not= in oldvegas:/usr/local/rush/etc/hosts 08/06,23:38:34 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com'[v= egas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) 08/06,23:38:34 LICENSE no servers could validate license (30 sec retries= ) 08/06,23:39:04 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 not= in oldvegas:/usr/local/rush/etc/hosts 08/06,23:39:04 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com'[v= egas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) 08/06,23:39:04 LICENSE no servers could validate license (30 sec retries= ) 08/06,23:39:34 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 not= in oldvegas:/usr/local/rush/etc/hosts 08/06,23:39:34 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com'[v= egas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) 08/06,23:39:34 LICENSE no servers could validate license (30 sec retries= ) 08/06,23:40:04 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 not= in oldvegas:/usr/local/rush/etc/hosts 08/06,23:40:04 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com'[v= egas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) 08/06,23:40:04 LICENSE no servers could validate license (30 sec retries= ) 08/06,23:40:34 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 not= in oldvegas:/usr/local/rush/etc/hosts 08/06,23:40:34 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com'[v= egas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) 08/06,23:40:34 LICENSE no servers could validate license (30 sec retries= ) 08/06,23:41:04 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 not= in oldvegas:/usr/local/rush/etc/hosts 08/06,23:41:04 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com'[v= egas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) 08/06,23:41:04 LICENSE no servers could validate license (30 sec retries= ) 08/06,23:41:34 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 not= in oldvegas:/usr/local/rush/etc/hosts 08/06,23:41:34 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com'[v= egas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) 08/06,23:41:34 LICENSE no servers could validate license (30 sec retries= ) 08/06,23:42:04 ALERT Tcp::GetRemoteHostname(0.0.0.0): host 0.0.0.0 not= in oldvegas:/usr/local/rush/etc/hosts 08/06,23:42:04 LICENSE Connect to ???[0.0.0.0] failed: 'vegas.eep.com'[v= egas]: valid host is NOT in rush hostlist (/usr/local/rush/etc/hosts) 08/06,23:42:04 LICENSE no servers could validate license (30 sec retries= ) 08/06,23:42:24 INIT.D Shutdown script '/usr/local/rush/etc/S99rush stop= ' 08/06,23:42:24 EXIT Shutdown script '/usr/local/rush/etc/S99rush stop= '= *** ***************************************************** *** Message #1952 ***************************************************** *** ***************************************************** Subject: [Q+A] Mental Ray Standalone 2011: "ProductInformation.pit file not From: Greg Ercolano Date: Fri, 30 Jul 2010 12:49:21 -0400 Quite a few people using Mental Ray Standalone have run into this issue after upgrading to Maya 2011, so I'm posting it here. * * * > We just upgraded to Mental Ray Standalone 2011, and are getting > these errors during rendering from 'ray': > >--------------------------------------------------------------- > *MSG .0 error: ProductInformation.pit file not found* > *MSG .0 fatal: Sorry, no license available* >--------------------------------------------------------------- > > We can run 'ray' from the command line OK without getting that error, > but through rush, or even invoking the render script *manually* > from the command line, we get this error. > > So we think it's something in the render script that's causing > the problem, but are not sure what. SOLUTION The short answer: *don't* set the MI_ROOT environment variable in your render script when using Mental Ray Standalone 2011. This may sound odd, given historically things *wouldn't* work unless that variable is set, but it's now the case in 2011 that actually *setting* that variable *causes* trouble in 2011. This might change in future releases of Maya, but currently that's the case with the 2011 release on Autodesk's site as of this writing (July 2010). DETAILS The "ProductInformation.pit" error is a common one, and comes up prominently if you google for that error, this autodesk article being the #1 hit as of this writing: http://usa.autodesk.com/adsk/servlet/ps/dl/item?siteID=123112&id=15131292&linkID=12544120 Quoting the above page (in case the link goes stale): >> -------------------------------------------------------------------------- >> ISSUE >> You have successfully set up an Autodesk License Server with mental ray >> Standalone 2011. >> >> When you start rendering with mental ray Standalone, you will receive the >> following error: >> >> Render_node1: ~ assist% ./ray –v 5 test.mi >> MSG .0 info : mental ray, version 3.8.1.28 >> MSG .0 info : use -copyright option to view copyright and terms of use. >> MSG .0 error: ProductInformation.pit file not found >> MSG .0 fatal: Sorry, no license available >> Exit 1 >> >> SOLUTION >> Ensure that ProductInformation.pit file exists and MI_ROOT env variable value >> is *not set* on system level or in a shell. The location of ProductInformation.pit is: >> >> Mac OS X: >> /Applications/Autodesk/mrstand3.8.1-adsk2011/bin/AdLM/ProductInformation.pit >> >> Linux: >> /usr/autodesk/mrstand3.8.1-adsk2011-x64/bin/AdLM/ProductInformation.pit >> >> Windows: >> C:\Program Files\Autodesk\mrstand3.8.1-adsk2011\bin\AdLM\ProductInformation.pit >> >> If the ProductInformation.pit file is not found, please reinstall mental ray >> Standalone 2011. >> >> Note: You need to use the supplied setup script when installing mental ray >> Standalone 2011 on a Linux platform. >> -------------------------------------------------------------------------- SIMULATING THE ERROR FROM THE COMMAND LINE You can probably replicate the above "pit error" from a working shell if you first set the MI_ROOT/MI_BIN_DIR variables just before running the 'ray' command. For instance, from a BASH shell: ------------------------------------------------------------------------ export MI_ROOT="/usr/autodesk/mrstand3.8.1-adsk2011-x64" export MI_BIN_DIR="${MI_ROOT}/bin" export PATH="${MI_BIN_DIR}:${PATH}" ray -render 1 1 -verbose 5 //yourserver/mifiles/your.mi ------------------------------------------------------------------------ ..you should likely now get the "ProductInformation.pit" error, which would prove that is the cause. MODIFICATION EXAMPLE So in your submit-maya and/or submit-mray scripts, make sure MI_ROOT and related environment variable settings are commented out/unset, and that your PATH is set with an absolute path, instead of one based on the MI_ROOT/MI_BIN_DIR variables. So for instance, in the LINUX section: -------------------------------------------- snip elsif ( $G::islinux ) { ### LINUX # SPECIAL CASE FOR MR 3.8.1 $ENV{PATH} = "/usr/autodesk/mrstand3.8.1-adsk2011-x64/bin:" . # ADD THIS "/usr/autodesk/mrstand3.8.1-adsk2011-x64/bin/AdLM:" . # ADD THIS $ENV{PATH}; ## $ENV{MI_ROOT} = "/usr/autodesk/mrstand3.8.1-adsk2011-x64"; # COMMENT THIS OUT ## $ENV{MI_BIN_DIR} = "$ENV{MI_ROOT}/bin"; # COMMENT THIS OUT ## $ENV{PATH} = "$ENV{MI_BIN_DIR}:$ENV{PATH}"; # COMMENT THIS OUT } -------------------------------------------- snip *** ***************************************************** *** Message #1951 ***************************************************** *** ***************************************************** Subject: Re: [Q+A] How to add a 'Rush' button into Maya's menu From: Greg Ercolano Date: Mon, 12 Jul 2010 14:25:53 -0400 Greg Ercolano wrote: > The below mel script works on all platforms; OSX, Linux, Windows. Here's a python version of the userSetup.mel script, in case you're trying to move away from mel. Tested on Windows with maya 2010. If you have any trouble, feel free to follow up here. --- snip # START SUBMIT FORM def RUSH_Start(cmd): p = os.popen(cmd+" 2>&1","r") errout = "".join(p.readlines()) err = p.close() print errout sys.stdout.flush() # OPEN RUSH SUBMIT FORM FOR THE CURRENT SCENE def RUSH_Submit(): import maya.cmds as M submit_maya = "//meade/net/rushscripts/submit-maya.pl" # submit script sfrm = M.getAttr("defaultRenderGlobals.startFrame") # get start frame efrm = M.getAttr("defaultRenderGlobals.endFrame") # get end frame scene = M.file(q=1, sceneName=1) # get scene name defren = M.getAttr("defaultRenderGlobals.currentRenderer") # get renderer if ( scene == "" ): M.confirmDialog(message="No scene file", title="Error", button="Dismiss") return renderer = "maya(sw)" if ( defren == "mentalRay" ): renderer = "mentalray(mr)" if ( defren == "renderMan" ): renderer = "renderman(rman)" RUSH_Start("perl " + submit_maya + " -field ScenePath \"" + scene + "\"" + " -field Frames " + sfrm + " -field Renderer " + renderer) # ADD RUSH SUBMIT BUTTON TO MAYA'S MENUBAR # These should be run only once. # def RUSH_CreateMenu(): import maya.cmds as M M.setParent("MayaWindow") M.menu(label="Rush") M.menuItem(label="Submit", command="RUSH_Submit()") M.menuItem(label="Irush", command="RUSH_Start('irush')") M.menuItem(label="Rushtop", command="RUSH_Start('rushtop')") # CREATE THE RUSH MENU WHEN THIS SCRIPT GETS LOADED RUSH_CreateMenu() --- snip *** ***************************************************** *** Message #1950 ***************************************************** *** ***************************************************** Subject: [Q+A] WINDOWS: "Unexpected network error" in 'Frames' report From: Greg Ercolano Date: Wed, 07 Jul 2010 14:21:57 -0400 > We've recently been getting these errors intermittently > in the 'Frames' report: > > STATE FRAME TRY HOSTNAME PID JOBID [..] NOTES > ----- ----- --- -------- ---- ------- --------------------------------------------------------------------------------------------------------------- > Fail 0009 1 blade10 3268 svr.457 [..] ERROR: can't create log as rush@blade10:\\jobserver\share1\jobs\BUILDLOG: An unexpected network error occurred. > > This occurs randomly on different hosts. Our server \\jobserver is a CIFS share > on our SAN. Is this a Rush problem, or one with the Windows stack? That's a Microsoft error; it seems the OS gave an error when rush tried to check to see if the log directory exists. Microsoft has a few links for this error; maybe one of them will help you discover the problem: http://support.microsoft.com/kb/121913 http://support.microsoft.com/kb/897089 One non-obvious cause could simply be a DNS problem; when the OS sees the UNC path \\jobserver\share1, it has to resolve "jobserver" into an IP address, and possibly that hostname resolution is being intermittent. How long has this been happening? If it's recent, does its appearance coincide with any administrative changes on the server (\\jobserver\share1) or network infrastructure? When the problem is happening, try walking up to that client (eg. "blade10", based on the above error message), and try accessing the "backslash" version of that pathname with the DIR command from a DOS window, eg: dir \\jobserver\share1\jobs\BUILDLOG ..if you get the same error, you can probably at least debug it that way. If so, I'd probably try 'ping jobserver' first, just to see if the hostname lookup is the issue (ie. see if you get a 'jobserver: Hostname lookup failure' error), and if not that, probe around with NET USE and friends to see what's up. *** ***************************************************** *** Message #1949 ***************************************************** *** ***************************************************** Subject: Python Maya Submit Script From: Greg Ercolano Date: Fri, 02 Jul 2010 14:57:48 -0400 I'm about to release a python version of the submit-maya script. It will work precisely the same way as the perl version, just written in python instead. Looking for testers who are existing customers and know python well and prefer it to perl. Contact me by direct email, and I'll set you up. When the script is ready, I'll follow up here with the download info for general use. This script will be included in the next Rush release. *** ***************************************************** *** Message #1948 ***************************************************** *** ***************************************************** Subject: Re: [Q+A] Nuke cross platform pathnames From: Greg Ercolano Date: Thu, 01 Jul 2010 14:08:00 -0400 Greg Ercolano wrote: > Another way to protect the backslashes would be to use: > > r"p:\" # same as "p:\\" Meh, that 'r' technique only works *if* the backslash is not at the end of the string. Since that *is* the case here, r"p:\" won't work.. you'd *have* to use "p:\\". Of course, using front slashes bypasses all this. Front slashes work fine in this context, as the Windows kernel internally understands fronts (/) and backs (\) equally well in pathnames. It's only DOS commands, and some of Microsoft's browsers that have trouble with fronts, but that's due to the design of just those Microsoft applications, and not the OS itself. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1947 ***************************************************** *** ***************************************************** Subject: Re: [Q+A] Nuke cross platform pathnames From: Greg Ercolano Date: Wed, 30 Jun 2010 18:45:30 -0400 Robert Minsk wrote: > [posted to rush.general] > > Greg Ercolano wrote: >> Yes; short answer is to see Nuke's own 'customizing' documentation >> on handling file paths across platforms. > > More specifically you will need to define a python function that takes a > filename as an argument a returns a new filename. You then need to install it > as a FilenameFilter callback (nuke.addFilenameFilter) in your sites init.py file. Right; I believe the "Nuke-5.1v6-customizing.pdf" documentation cites this exact example [sic]: ---- def filenameFix(filename): if platform.system() in ("Windows", "Microsoft"): return filename.replace("/SharedDisk/", "p:\") else: return filename.replace("p:\", "/SharedDisk/") return filename ---- ..which appears to show how to replace e.g. "/SharedDisk/foo" <-> "p:\foo" depending on the platform it's running on. However, there's a few things wrong with that example I think; that backslash after the "p:" needs to be protected, otherwise python will interpret it as an escape character. Also, that final 'return filename' appears to be extraneous, as there's no way that line will ever execute. So I would suggest the following instead: ---- def filenameFix(filename): if platform.system() in ("Windows", "Microsoft"): return filename.replace("/SharedDisk/", "p:\\") else: return filename.replace("p:\\", "/SharedDisk/") ---- Another way to protect the backslashes would be to use: r"p:\" # same as "p:\\" This is pythons little way of protecting all characters between the double quotes, the leading 'r' in front of the leading double quote means "raw string", protecting the backslash from being mistaken for an escape sequence. An example that worked for a recent client who wanted to make sure UNC style pathnames were used on windows, (eg. "/bart/foo" would become "//bart/foo" on windows machines), I suggested they use: ---- import re import platform def filenameFix(filename): if platform.system() in ("Windows", "Microsoft"): return re.sub("^/*bart","//bart",filename) else: return re.sub("^/*bart","/bart",filename) ---- ..the special regex ensuring that e.g. //bart doesn't become ///bart -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1946 ***************************************************** *** ***************************************************** Subject: Re: [Q+A] Nuke cross platform pathnames From: Robert Minsk Date: Wed, 30 Jun 2010 18:01:55 -0400 Greg Ercolano wrote: > Yes; short answer is to see Nuke's own 'customizing' documentation > on handling file paths across platforms. More specifically you will need to define a python function that takes a filename as an argument a returns a new filename. You then need to install it as a FilenameFilter callback (nuke.addFilenameFilter) in your sites init.py file. *** ***************************************************** *** Message #1945 ***************************************************** *** ***************************************************** Subject: Re: Render nodes failing to pick up jobs after new license push From: Greg Ercolano Date: Tue, 29 Jun 2010 02:51:50 -0400 Daniel Browne wrote: > After we upgraded our license a week ago, we occasionally found that some render > machines would not pick up new jobs. In particular we encountered this > issue with machines that had been recently onlined, either manually during > the day or by our idle time script in the evenings. Restarting the rushd > daemon on the individual machines seems to resolve the issue, however we > puzzled as to why this occurred in the first place. Your instructions for > license installation indicate that we only needed to restart the Rush daemon > on the Rush server, but not on any of the client machines. Did I misread > your instructions or could this be a separate issue? Hi Daniel, It sounds like you read the instructions correctly; you would only restart the license server daemon, not the others. And even that isn't entirely necessary; you can either restart the license server daemon, or just wait after installing the license.dat file until the rushd.log shows the daemon has reloaded the license before pushing out the new license to the rest of the machines. (This usually takes up to 30 seconds before the 'Reloading' message appears in the rushd.log) What you're describing here sounds similar to the problem you reported back in December 2009: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1907+1908+1909+1910+1911+1912+1913+1914+1915 The symptoms being: after a license or hosts file is changed and pushed out, the machines continue to respond OK to 'rush -ping' but stop showing up in rushtop(1), and won't respond in 'rush -lac' (All Cpus) or 'rush -laj' (All Jobs) reports. If the problem you're encountering is on Macs running Leopard or Snow Leopard, make sure the version of rushd (ie. 'rushd -version') reports "102.42a9d" on those machines. If not, please upgrade those Macs to the latest 102.42a9d release: http://www.seriss.com/rush/releases/Upgrade-102.42a9c.html (This page is password protected; notify me by email directly if you don't know your Rush download password.) Note that even though that page is the 102.42a9c release (emphasis on 'c'), the Mac release had a special fix that bumped the version to 102.42a9d (emphasis on 'd'). If you've currently got 'd' on those machines, then the problem might be something else; maybe you can give me more details then, such as if there are any errors in the rushd.log, and if the machines respond at all to 'rush -ping', or if the daemons (rushd) are even running. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1944 ***************************************************** *** ***************************************************** Subject: Render nodes failing to pick up jobs after new license push From: Daniel Browne Date: Mon, 28 Jun 2010 22:06:19 -0400 Hi Greg, After we upgraded our license a week ago, we occasionally found that some r= ender machines would not pick up new jobs. In particular we encountered thi= s issue with machines that had been recently onlined, either manually durin= g the day or by our idle time script in the evenings. Restarting the rushd = daemon on the individual machines seems to resolve the issue, however we pu= zzled as to why this occurred in the first place. Your instructions for lic= ense installation indicate that we only needed to restart the Rush daemon o= n the Rush server, but not on any of the client machines. Did I misread you= r instructions or could this be a separate issue? -Dan= *** ***************************************************** *** Message #1943 ***************************************************** *** ***************************************************** Subject: [Q+A] Nuke cross platform pathnames From: Greg Ercolano Date: Tue, 22 Jun 2010 19:39:33 -0400 > When I submit a Nuke scene file created on a Mac, the artist > has to change the path within nuke from /myserver/share to //myserver/share > in order for the render to be successful on both Mac and Windows machines. > We are using the FixPath() function in the rushscripts/.common.pl > but that doesn't seem to affect pathnames within the Nuke scripts. > > Is there a way to configure Nuke to make these changes automatically? Yes; short answer is to see Nuke's own 'customizing' documentation on handling file paths across platforms. I'm almost sure the nuke documenation is wrong with respect to those backslashes, ie: "p:\" probably needs to be e.g. "p:\\". But you can probably use front slashes and avoid the whole 'backslash as an escape character' problem, which is typical in just about all languages including python, perl, C, Java, and all the shells. *** ***************************************************** *** Message #1942 ***************************************************** *** ***************************************************** Subject: Re: Rendering with CS5 AE From: Greg Ercolano Date: Fri, 18 Jun 2010 01:29:25 -0400 Daniel Browne wrote: > Unfortunately I've already overwritten the bad frame logs from today's test= > s. On the off chance that it will be of use you'll find a successful frame = > log at the end of this message. I have plenty of those ;) Need the text with the actual error to offer you a good regex to search for. You can see the usage of LogCheck() by doing a grep on all the submit scripts for LogCheck, ie. 'grep LogCheck /path/to/your/rushscripts/*.pl' The code for that function is defined in .common.pl, and can take one or more regex's as an array, so that you can search for several strings in one shot. It also has a separate string that resets the search, useful for it to skip over retries. > To me it looks as if AfterEffects is bound to the static location of the Ma= > c's /Library as well as the intended install locations of some if not all o= > f the other pieces of the CS5 suite itself, so it may not be possible at al= > l to have a central library. I don't suppose symbolic links would help? ie. make a symlink(s) in the /Library/xxx directory where the extra fonts should go. Might be worth a shot. > Using killall on aerendercore or aerender so far has not solved the zombie = > problem, so this issue may have gotten worse in CS5 from what you've descri= > bed. Since zombies aren't really processes, they're just the 'exit code' of the process, you don't really have to worry about them too much unless there's a lot of them. Killing them is useless, because they're already dead, just like in the movies ;) Zombies are created when the parent doesn't reap the exit code, which probably means the parent is still around somewhere. > One aspect of MacOS X that frustrates people who've used other unix pl= > atforms is that there are situations in which even kill -9 as root will not= > terminate a stuck process. This actually can happen on all unix platforms.. I used to run into this constantly on SGI and linux in the 90's (and to date) whenever a process is reading or writing to a hard mounted NFS share where the server has gone down or is not responding. The hanging process part is OK, you want it to do that, but if root comes along with kill -9, it SHOULD die, but it often never does. This is because the process is stuck in kernel mode, where signals aren't processed immediately the way they are in user mode. It can be done, but it's tricky/messy to do in kernel code at all the points where a hang could happen, so often processes stuck in kernel mode are unkillable. That would go for any kernel or driver operation that is 'stuck' for some reason, like tape operations or floppy drives... other situations where I've seen this occur. > This is particularly true if you have an OpenGL= > or graphics driver error. You'd think having created OpenCL that Apple wou= > ld finally want to do something about that... It's all to do with the driver code and how well it's written to handle pending signals. It's very tricky to properly detect and unravel a kernel mode call that's got pending signals and have it react like a user mode operation. > In the mean time I'll give the LogCheck function a try. Yes; see the usage in other submit scripts (like submit-maya, submit-nuke and submit-lightwave), and the definition of the function in .common.pl -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1941 ***************************************************** *** ***************************************************** Subject: Re: Rendering with CS5 AE From: Daniel Browne Date: Thu, 17 Jun 2010 22:53:25 -0400 Unfortunately I've already overwritten the bad frame logs from today's test= s. On the off chance that it will be of use you'll find a successful frame = log at the end of this message. To me it looks as if AfterEffects is bound to the static location of the Ma= c's /Library as well as the intended install locations of some if not all o= f the other pieces of the CS5 suite itself, so it may not be possible at al= l to have a central library. Using killall on aerendercore or aerender so far has not solved the zombie = problem, so this issue may have gotten worse in CS5 from what you've descri= bed. One aspect of MacOS X that frustrates people who've used other unix pl= atforms is that there are situations in which even kill -9 as root will not= terminate a stuck process. This is particularly true if you have an OpenGL= or graphics driver error. You'd think having created OpenCL that Apple wou= ld finally want to do something about that... In the mean time I'll give the LogCheck function a try. -Dan ### ### neildiamond.333: 0006 ### --------------- Rush 102.42a9d -------------- -- Host: r32 -- Pid: 275 -- Title: render_test5 -- Jobid: neildiamond.339 -- Frame: 0006 -- Tries: 0 -- Owner: dbrowne (1089/20) -- RunningAs: dbrowne (1089/20) -- Priority: 1 -- Nice: 10 -- Tmpdir: /var/tmp/.RUSH_TMP.6 -- LogFile: /Casino/Poker/Shows/frd/frdCOMMON/frdDEV/gfx/scripts/dennisw= /orDev_typeExploration_037-test.aep.log/0006 -- Command: perl /EEP/Tools/Settings/rush/rushscripts_v102.42a9/submit-a= fterfx.pl -render CS5 /Casino/Poker/Shows/frd/frdCOMMON/frdDEV/gfx/scripts/= dennisw/orDev_typeExploration_037-test.aep origami=7FMAIN=7FAssemble=7Fv3= =7FEDIT /Casino/Poker/Shows/frd/frdCOMMON/frdDEV/gfx/images/Renders/rush_te= sts/origami_MAIN_Assemble_v3_EDIT_[####].tif 5 100 3 Fail 0 off -- Started: Thu Jun 17 18:02:06 2010 -------------------------------------------- Possible unintended interpolation of @10 in string at /EEP/Tools/Settings/r= ush/rushscripts_v102.42a9/submit-afterfx.pl line 83. SCENEPATH: /Casino/Poker/Shows/frd/frdCOMMON/frdDEV/gfx/scripts/dennisw/= orDev_typeExploration_037-test.aep COMP: origami MAIN Assemble v3 EDIT OUTPUTDIR: /Casino/Poker/Shows/frd/frdCOMMON/frdDEV/gfx/images/Renders/r= ush_tests/origami_MAIN_Assemble_v3_EDIT_[####].tif AEVERSION: CS5 RENDERFLAGS:=20 BATCHFRAMES: 5 (6-10) RETRIES: 3 (Fail after 3 retries) PATH: /Network/Library/Fonts:/Applications/Adobe After Effects CS5:= /Applications/Adobe After Effects CS5:/Applications/Adobe After Effects CS5= :/Applications/Adobe After Effects CS5:/Applications/Adobe After Effects CS= 5:/usr/local/rush/bin:/usr/local/rush/bin:/sbin:/bin:/usr/sbin:/usr/bin:/us= r/local/rush/bin Executing: aerender -mem_usage 60 120 -mp -project "/Casino/Poker/Shows/frd= /frdCOMMON/frdDEV/gfx/scripts/dennisw/orDev_typeExploration_037-test.aep" -= output "/Casino/Poker/Shows/frd/frdCOMMON/frdDEV/gfx/images/Renders/rush_te= sts/origami_MAIN_Assemble_v3_EDIT_[####].tif" -comp "origami MAIN Assemble = v3 EDIT" -s 6 -e 10=20 Thu Jun 17 18:02:12 r32.eep.com aerender[279] : 3891612: (connectA= ndCheck) Untrusted apps are not allowed to connect to or launch Window Serv= er before login. Thu Jun 17 18:02:13 r32.eep.com aerender[279] : kCGErrorFailure: Set= a breakpoint @ CGErrorBreakpoint() to catch errors as they are logged. _RegisterApplication(), FAILED TO establish the default connection to the W= indowServer, _CGSDefaultConnection() is NULL. PROGRESS: 6/17/10 6:02:19 PM PDT: Starting composition =1Corigami MAIN Ass= emble v3 EDIT=1D. PROGRESS: Render Settings: Best Settings PROGRESS: Quality: Best PROGRESS: Resolution: Full PROGRESS: Size: 1920 x 1080 PROGRESS: Proxy Use: Use No Proxies PROGRESS: Effects: Current Settings PROGRESS: Disk Cache: Read Only PROGRESS: Color Depth: Current Settings PROGRESS: Frame Blending: On for Checked Layers PROGRESS: Field Render: Off PROGRESS: Pulldown: Off PROGRESS: Motion Blur: On for Checked Layers PROGRESS: Use OpenGL: Off PROGRESS: Solos: Current Settings PROGRESS: Time Span: Custom PROGRESS: Start: 00006 PROGRESS: End: 00010 PROGRESS: Duration: 00005 PROGRESS: Frame Rate: 29.97 (comp) PROGRESS: Guide Layers: All Off PROGRESS: Skip Existing Files: On PROGRESS: =20 PROGRESS: Output Module: TIFF Sequence with Alpha PROGRESS: Output To: /Casino/Poker/Shows/frd/frdCOMMON/frdDEV/gfx/images/R= enders/rush_tests/origami_MAIN_Assemble_v3_EDIT_[####].tif PROGRESS: Format: TIFF Sequence PROGRESS: Output Info: - PROGRESS: Start Frame: 6 PROGRESS: Output Audio: - PROGRESS: Channels: RGB + Alpha PROGRESS: Depth: Millions of Colors+ PROGRESS: Color: Premultiplied PROGRESS: Resize: - PROGRESS: Crop: - PROGRESS: Final Size: 1920 x 1080 PROGRESS: Profile: - PROGRESS: Embed Profile: =20 PROGRESS: =20 PROGRESS: Post-Render Action: None PROGRESS: =20 PROGRESS: =20 PROGRESS: =20 PROGRESS: =20 PROGRESS: 00006 (1): 1 Seconds PROGRESS: 00007 (2): 1 Seconds PROGRESS: 00008 (3): 1 Seconds PROGRESS: 00009 (4): 1 Seconds PROGRESS: 00010 (5): 0 Seconds PROGRESS: 6/17/10 6:02:25 PM PDT: Finished composition =1Corigami MAIN Ass= emble v3 EDIT=1D. PROGRESS: Total Time Elapsed: 6 Seconds aerender version 10.0x458 PROGRESS: Launching After Effects... PROGRESS: ...After Effects successfully launched --- AERENDER SUCCEEDS *** ***************************************************** *** Message #1940 ***************************************************** *** ***************************************************** Subject: Re: Rendering with CS5 AE From: Greg Ercolano Date: Thu, 17 Jun 2010 21:49:08 -0400 Daniel Browne wrote: > - Our NFS based /Network/Library/Fonts folder is not recognized by AE (I > believe this has always been the case), so the fonts needed for the scene had > to be copied locally. So far I've had no success trying to make it recognize > the central library from environment parameters. This happens even with > a render user logged in on the console, so it's not an automount issue. I'd suggest contacting Adobe about that one. Assuming you can replicate the problem from the command line (ie. ssh into the remote machine as the same user rush is rendering the job as when the problem occurs), and then report the problem to them in that context. Leave "rush" or "we're using a render queue" out of the support call, so that it doesn't confuse them. > - In that same vein I cannot centralize plug-ins either. Nor for that matter > could I centralize the entire AE package itself (even if I didn't have the > mounts set with the nosuid flag) Can't answer to this, as I'm not sure what limitations are built into AE that prevent this. I have a feeling that AE might still be internally using OS9 techniques for accessing files (eg. the old "some:path:dir" instead of "/some/path/dir" pathnames) One can sometimes see those old OS9 style pathnames in AE's output. I understand they're trying to get away from that old stuff, but it's been taking them a while because internally Adobe shares the same libraries among all their products. Long ago AE had problems accessing files on more than one network volume.. not sure if that's still the case, but I believe that was an OS9 limitation as well. Might be related. > - The aerender binary does not seem to return its error number consistently > on exit; I've had batches register as Done even when they've failed in the > frame logs, despite what is in my submit script. Can you include a copy of the complete log from one of these frames? Then I can help show you what commands to add to the submit script logic so you can catch those errors and handle them properly. > Do you know of another way > or do I simply have to capture the stdout and search for an error message? stdout/err capture is the only other way, and there's a function built into the scripts called LogCheck() which lets you specify one or more regex strings that searches the logs, correctly skipping over retry attempts. > - When I fail or dump a batch to re-que it, it doesn't kill the aerendercore > process and I have to kill it on the individual machine. This is probably > the most serious problem as often even killing aerendercore isn't enough. Yes, this is really annoying, details on that were posted here on the group a few months ago here: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1906+1906 This cropped up because Adobe made a change in CS3 that causes the render process to disconnect itself from the process hierarchy, effectively orphaning the process. Adobe is aware of the problem as of November 2009, and is trying to fix it. I worked directly with the AE devs on that issue. The AE engineers came up with a javascript mod as a temporary workaround to solve the problem; I can try to follow up with those engineers through myself on your behalf; contact me off list. This can be replicated from the command line as well by just running aerender interactively, and then killing it. You don't have to reboot to fix the problem, you can do something like a 'killall' command as root to kill the aerendercore and associated processes. Until Adobe fixes this, all render queue systems and distributed scripting of AE is affected by this. > The kill results in a zombie process that makes it impossible to run another > render on the machine until you do a "reboot -q" on it. If you look carefully at the process hierarchy, you can determine which processes need to be killed.. there's more than one. Mac's ps(1) command unfortunately doesn't seem to have an equivalent of the linux 'ps -fax' command, which shows the process hierarchy as a 'tree' so you can see what's parented to what. I do, however, have a script on my homepage that does the equivalent on the mac (and in fact, on most all unix platforms): http://seriss.com/people/erco/unixtools/piss ..which if you run it, will show you the process tree on a Mac so you can see which processes are parented to what, and will help you determine what needs to be killed to free up the box. Example report: http://seriss.com/people/erco/unixtools/sample-piss.html -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1939 ***************************************************** *** ***************************************************** Subject: Re: Rendering with CS5 AE From: Daniel Browne Date: Thu, 17 Jun 2010 20:59:59 -0400 Thanks Greg, I've gotten it to render, with a number of caveats (perhaps you can shed so= me light on some): - Our NFS based /Network/Library/Fonts folder is not recognized by AE (I be= lieve this has always been the case), so the fonts needed for the scene had= to be copied locally. So far I've had no success trying to make it recogni= ze the central library from environment parameters. This happens even with = a render user logged in on the console, so it's not an automount issue. - In that same vein I cannot centralize plug-ins either. Nor for that matte= r could I centralize the entire AE package itself (even if I didn't have th= e mounts set with the nosuid flag) - The aerender binary does not seem to return its error number consistently= on exit; I've had batches register as Done even when they've failed in the= frame logs, despite what is in my submit script. Do you know of another wa= y, or do I simply have to capture the stdout and search for an error messag= e? - When I fail or dump a batch to re-que it, it doesn't kill the aerendercor= e process and I have to kill it on the individual machine. This is probably= the most serious problem as often even killing aerendercore isn't enough. = The kill results in a zombie process that makes it impossible to run anothe= r render on the machine until you do a "reboot -q" on it. -Dan On Jun 2, 2010, at 10:24 PM, Greg Ercolano wrote: [posted to rush.general] Daniel Browne wrote: > Hi Greg, >=20 > I'm working out a setup for rendering with CS5 After Effects and it appea= rs Adobe still > expects you to be logged into the desktop as the user executing the job.= =20 Yes, after effects is still one of those renderers that has not yet separated their command line renderer from the GUI. AE's command line renderer, "aerender", still fires off the GUI and communicates with it to make the render happen. So it is still tied to the window manager, and this causes lots of weirdness. AE has always been this way, and it seems they have a very hard time changing it. > Is there a way > to force only AE scripts to execute as a specific user? You can use normal unix techniques to make the 'aerender' executable run as any user you want, just by setting the perms on the executable, eg: chown 0:0 /path/to/aerender chmod 4755 /path/to/aerender ..which will make aerender run as root (and all that implies) no matter what user invoked the command. Root is never denied access to the window manager, so that would force it to not have permission issues with the window manager. However, if you prefer to have aerender run as a normal user, you can use the same commands in a similar way. For instance if the username is 'render' and the group is "users", then you can use: chown render:users /path/to/aerender chmod 4755 /path/to/aerender ..to make "aerender" run as 'render' instead. Usually just making the aerender executable setuid as the user you want would do the trick, but in the recent versions of AE (CS3 and CS4 and possibly 5), you may need to go an extra step and make the AE GUI binary setuid. (It used to be the case that 'aerender' started the AE GUI with fork()/exe= c(), such that the GUI was part of the same process hierarchy, but they made a = mistake in CS3 that causes the GUI to disconnect from the process hierarchy.. meh) Someday we really all should hope Adobe will rewrite aerender so that it does the rendering itself, and does not need the GUI to do any rendering. This way distributed rendering with AE would be solid like the rest of the compositors (nuke, shake..) --=20 Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1938 ***************************************************** *** ***************************************************** Subject: Re: Rendering with CS5 AE From: Greg Ercolano Date: Thu, 03 Jun 2010 01:24:02 -0400 Daniel Browne wrote: > Hi Greg, > > I'm working out a setup for rendering with CS5 After Effects and it appears Adobe still > expects you to be logged into the desktop as the user executing the job. Yes, after effects is still one of those renderers that has not yet separated their command line renderer from the GUI. AE's command line renderer, "aerender", still fires off the GUI and communicates with it to make the render happen. So it is still tied to the window manager, and this causes lots of weirdness. AE has always been this way, and it seems they have a very hard time changing it. > Is there a way > to force only AE scripts to execute as a specific user? You can use normal unix techniques to make the 'aerender' executable run as any user you want, just by setting the perms on the executable, eg: chown 0:0 /path/to/aerender chmod 4755 /path/to/aerender ..which will make aerender run as root (and all that implies) no matter what user invoked the command. Root is never denied access to the window manager, so that would force it to not have permission issues with the window manager. However, if you prefer to have aerender run as a normal user, you can use the same commands in a similar way. For instance if the username is 'render' and the group is "users", then you can use: chown render:users /path/to/aerender chmod 4755 /path/to/aerender ..to make "aerender" run as 'render' instead. Usually just making the aerender executable setuid as the user you want would do the trick, but in the recent versions of AE (CS3 and CS4 and possibly 5), you may need to go an extra step and make the AE GUI binary setuid. (It used to be the case that 'aerender' started the AE GUI with fork()/exec(), such that the GUI was part of the same process hierarchy, but they made a mistake in CS3 that causes the GUI to disconnect from the process hierarchy.. meh) Someday we really all should hope Adobe will rewrite aerender so that it does the rendering itself, and does not need the GUI to do any rendering. This way distributed rendering with AE would be solid like the rest of the compositors (nuke, shake..) -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1937 ***************************************************** *** ***************************************************** Subject: Rendering with CS5 AE From: Daniel Browne Date: Wed, 02 Jun 2010 21:46:29 -0400 --_003_3D94E9A296F84F93AAB209CB87418945evileyepicturescom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Greg, I'm working out a setup for rendering with CS5 After Effects and it appears= Adobe still expects you to be logged into the desktop as the user executin= g the job. Is there a way to force only AE scripts to execute as a specific= user? I know there is a parameter in the rush config file to execute all j= obs as a specific user, but I'd rather it just be exclusive to AE jobs. Is = that possible? Thanks, -Dan ---------- Dan "Doc" Browne System Administrator --_003_3D94E9A296F84F93AAB209CB87418945evileyepicturescom_ Content-Type: image/tiff; name="pastedGraphic.tiff" Content-Description: pastedGraphic.tiff Content-Disposition: attachment; filename="pastedGraphic.tiff"; size=7876; creation-date="Wed, 02 Jun 2010 20:46:02 GMT"; modification-date="Wed, 02 Jun 2010 20:46:02 GMT" Content-Transfer-Encoding: base64 TU0AKgAACDqAP+BACCQWDQeEQmFQuGQR7w92xF1RN7RWCgmMBWNBSOAqPQ2QSGRQmBP+RyeRPGVM mWLiXNCYOGZOuaPObQsFzmNBUPT0Xz8j0Ec0MJUWUUeGSWkUtmU1RU9X1FyVOl0gM1cm1kwVuhjm q0elV+GTBoIqzKq0P21QgA20LhcLBoNBuihEGXcDXmCvu+PS/O/AOVzOZ04V/P1/Qsn4s8Y0cY+x UmB5GCPjLIPMJDNRV7QgP58aDQZiQSCMLaeMAkBauC2EAW2CgF/bPLPiaOtvN5v01mt3fQgC8E18 NA8UHcfKQTXVVr80x89j9GECsViohdfSCMD9t+d3vcuUW0AgTyeV9edvt9wL5fL9o++EC75J76DP 7ZHwSdd/suf2JnUgrTgsJwnCaGIYhg8Tzn0/LkoS8TggK8RqGoapYliWRyw0goHw6T8PijEKqwah sLliLMUIee6ChqGoaCoKgpQ6B7LHzEkHKQBEdHrHhYFgWRiSCgryw+T7+i4sDJpCX8mCZJzOO2Ay siaJIkiQfMsSzHEtoQAcvQiXheF6VxXFekrylZNLFiekcboKbk4B5OTCnSgqigkG4bhsCE+BQFAT wFBbZsTLktvE1JgmCYRTlOVCSuOBxhUkn4XpBG6+H2INNGLTiRruBgdh2HQjCMIrtgPTFC1UjwFF yXJdTIV6ChbWlOGKBtcIXG5C14PlfIWCdg0weViIQEVjjCMIwLrBdVRw8VTlGUZSGMYxjoKOdskZ bddSVN84BlcKbHm8Uji9c8Dhi2pdXYRd3Oaa6CrkDThjTGdU2c5MvAHLB9W2Rs6QiZeB0ohD8ueM b6E8gpNYaMuHpAiJ24REyCtIEl6tWAVB3y5MdAQ95pEwTBMoLAgnYqg7XG3lj5Bc2oe5jSRhKPBc 1pcXCCh+H4fRQLEVY6yjYOCAxLkuTMKGo8pkaY+wZtbJQ/6kQWqIKWmriXrKqnZrlQh2bWwPEM4z jLSja6CyNT3gShKEskrhjXthKagf8s1oFuWG2E+9mnvsIsjWzriFBYKgsCo6joOePrUfu0KW8WNE aRpHpkcKrgya3Mz4CCSmdzzQhokpAdHqQ/y4PPUER1SCiKIoiCmKYpaBxyqtSW5blxC5ZILVxc1I IyS9URHUDygsgmJr0uR4et07zCI6jqOd53x2iTwjDRyrMRnGV8PjMEGkopfEqJXwEbHzgj9NnRMK H2oKn4XbGM0teqk+NUx1RE64dudh+YD/3GOgGfAMIcBUwi8dolUJLODxBpDSGgFgLAVv0fqSIvIB hKiVEs5ka5nwPrwXGCmERUxyBkhMJuFDtHPDOK6gsEIIQQBwDgG8kqboKkENSKsVYrBei9F8rgBo 2YhDoiI3sE5Kh4mNDwIeJkFQxRPSKQVc4XShg4bPDchhGAEKuF2VEWEFz3jRN8N2CILEVPfe7BUc 8a10p0J2HcO4dUIsciwQhj6TBgFoFWxqAYzzcjeOoCtFTVBBB9kNGqNcBQhrwZMgUJQSgkuzjqQZ j57Bfw6FZHyAZ/wTSdHhJ8O0oREyjdpEIbKTgmJwG4QhT8oQ6PpAid6ScOCMO4dyj+C7fRpvLBVL 1ypWwwRRY7EQdARJjDVmQSCKsU4ryTNSKgVAqX/jBRnKZQaeQbsDGWnIHgw5vNBYUwgkZ4g2BsDX L0FCNY6oREkJISjYBtAjnlMgapJUUBZmgKhzcpkBLOXGFigAtqBEjXmHMOgcgCJejo2hjSWYmCIM APB37vSSztEkG+jBBYei9cE0Ed1HwgUhl0SM64QQrBWCrJJoMFyZDiX+oMQ1MXiElcyNZlzjA5U5 ckI2UsQgfU/P+SA8Rzwwg0RcPceyK1VL7QiU8Uh0VrkEZmzEHpJXGNOjCB2rUpjUmRUGvBAROyCk sGS60Iq4yQKsDWGwNQIwRAhIePhB0F0Fx6OiMggs8gRjSr4qw1wjrALZDmQUUlhQt2Hq8bMGFi41 jnl6CqkIQKfg+f+MCUYiXqELfSBBsYZ4XgfIrUp+xq2PwkUYKhOA3SEPEpiIZuhBTBjmkDJ8eDLi mjMb+WJry1RjELc2uNxhI4gM+BkggxA/jvkCcgatCJnFODGZwjwzpB1WR9k6Ca15B5Qh2XcIsgrc m4GRYQwp2hoQZ1UAxek4IBC1D+WIPKmtd7YkgWSGG8jKlvEETopROid5tLHBEVVq4tEpx1c2qxBc SLMkEU+8skqn6+DSheCFg1+SDCcwww8MpBbIrsF1bkk6WV0yMlmQuD0JgyM4VsQUswinoB1W6SYh r4gpPkILZ1kYmCqizx4yeCtiwYIzJyAtlwOsjOCosHvJRJXBM4xBhXGRDDb1UlMQXF+LSq5KD3a1 oKmggi1zBV0hTwniEFA5mdWxPQPKWwsQtpNZpiY3bHRaC5Rw/Z3e+lyVBTxRSwIXdu7qHEOs4MeD hNubSGzaZPnEgjgmjCXuuUd24t3RiAGVpco8Hg46bnKGxfZCBxahbhgMgrm0yCuy8knKJR5dUACx TVOxRXShm1ox8kbjFbY8FnKa97HwQa/mNsEIjmyEHeFDseQwfbGkFwmKbZ2hUR6IJOf/TsOhVkIy Bp0Ku21IO0M5qeDIlWmV4IPI8JWOatAdPxtIqond3OlewQjCZQQjohCipRAR4ijm3aSj4WDOJ4EI J290NvBd9HJhsV9OlgBHCg4dUEhUsNf8TBBhPYhBIkDg41xvjbEiFp3sOFvF+6VncJQdbFRgp9T1 kuCqo8ToMaT3g87TkzHZdTaVtLpOi43qHliAgKQLXkWg1UoxqOvNZJvLM5M1j5qVP4lJAQEAAA4B AAADAAAAAQAwAAABAQADAAAAAQAwAAABAgADAAAAAwAACOgBAwADAAAAAQAFAAABBgADAAAAAQAC AAABEQAEAAAAAQAAAAgBEgADAAAAAQABAAABFQADAAAAAQADAAABFgADAAAAAQOOAAABFwAEAAAA AQAACDEBHAADAAAAAQABAAABPQADAAAAAQACAAABUwADAAAAAwAACO6HcwAHAAAV0AAACPQAAAAA AAgACAAIAAEAAQABAAAV0EFQUEwCQAAAbW50clJHQiBYWVogB9gABAAXAAwANQA7YWNzcEFQUEwA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPbWAAEAAAAA0y1MT0dPttwQargZ8/io3599iUGwagAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQY3BydAAAAUQAAAAqRGV2RAAAAXAAAAUIQ0lF RAAABngAAAiLUG10cgAADwQAAAGqY2hhZAAAELAAAAAsclhZWgAAENwAAAAUZ1hZWgAAEPAAAAAU YlhZWgAAEQQAAAAUd3RwdAAAERgAAAAUbHVtaQAAESwAAAAUclRSQwAAEUAAAAAOZ1RSQwAAEVAA AAAOYlRSQwAAEWAAAAAOdmNndAAAEXAAAAMvZ21wcwAAFKAAAADEZGVzYwAAFWQAAABrdGV4dAAA AABDb3B5cmlnaHQgYnkgTE9HTyBHbWJILCBTdGVpbmZ1cnQAAAB0ZXh0AAAAAExHT1JPV0xFTkdU SAk2DQpDUkVBVEVECSI0LzIzLzIwMDgiICAjIFRpbWU6IDEyOjUzDQpLRVlXT1JECSJTYW1wbGVJ RCINCktFWVdPUkQJIlNBTVBMRV9OQU1FIg0KTlVNQkVSX09GX0ZJRUxEUwk1DQpCRUdJTl9EQVRB X0ZPUk1BVA0KU2FtcGxlSUQJU0FNUExFX05BTUUJUkdCX1IJUkdCX0cJUkdCX0INCkVORF9EQVRB X0ZPUk1BVA0KTlVNQkVSX09GX1NFVFMJNDINCkJFR0lOX0RBVEENCjEJQTEJMC4wMAkwLjAwCTAu MDANCjIJQTIJMC4wMAkwLjAwCTEyOC4wMA0KMwlBMwkwLjAwCTAuMDAJMjU1LjAwDQo0CUE0CTAu MDAJMTI4LjAwCTAuMDANCjUJQTUJMC4wMAkxMjguMDAJMTI4LjAwDQo2CUE2CTAuMDAJMTI4LjAw CTI1NS4wMA0KNwlCMQkwLjAwCTI1NS4wMAkwLjAwDQo4CUIyCTAuMDAJMjU1LjAwCTEyOC4wMA0K OQlCMwkwLjAwCTI1NS4wMAkyNTUuMDANCjEwCUI0CTEyOC4wMAkwLjAwCTAuMDANCjExCUI1CTEy OC4wMAkwLjAwCTEyOC4wMA0KMTIJQjYJMTI4LjAwCTAuMDAJMjU1LjAwDQoxMwlDMQkxMjguMDAJ MTI4LjAwCTAuMDANCjE0CUMyCTEyOC4wMAkxMjguMDAJMTI4LjAwDQoxNQlDMwkxMjguMDAJMTI4 LjAwCTI1NS4wMA0KMTYJQzQJMTI4LjAwCTI1NS4wMAkwLjAwDQoxNwlDNQkxMjguMDAJMjU1LjAw CTEyOC4wMA0KMTgJQzYJMTI4LjAwCTI1NS4wMAkyNTUuMDANCjE5CUQxCTI1NS4wMAkwLjAwCTAu MDANCjIwCUQyCTI1NS4wMAkwLjAwCTEyOC4wMA0KMjEJRDMJMjU1LjAwCTAuMDAJMjU1LjAwDQoy MglENAkyNTUuMDAJMTI4LjAwCTAuMDANCjIzCUQ1CTI1NS4wMAkxMjguMDAJMTI4LjAwDQoyNAlE NgkyNTUuMDAJMTI4LjAwCTI1NS4wMA0KMjUJRTEJMjU1LjAwCTI1NS4wMAkwLjAwDQoyNglFMgky NTUuMDAJMjU1LjAwCTEyOC4wMA0KMjcJRTMJMjU1LjAwCTI1NS4wMAkyNTUuMDANCjI4CUU0CTAu MDAJMC4wMAkzMi4wMA0KMjkJRTUJMC4wMAkwLjAwCTY0LjAwDQozMAlFNgkwLjAwCTAuMDAJMTYw LjAwDQozMQlGMQkwLjAwCTAuMDAJMTkyLjAwDQozMglGMgkwLjAwCTAuMDAJMjI0LjAwDQozMwlG MwkwLjAwCTMyLjAwCTAuMDANCjM0CUY0CTAuMDAJNjQuMDAJMC4wMA0KMzUJRjUJMC4wMAkxNjAu MDAJMC4wMA0KMzYJRjYJMC4wMAkxOTIuMDAJMC4wMA0KMzcJRzEJMC4wMAkyMjQuMDAJMC4wMA0K MzgJRzIJMzIuMDAJMC4wMAkwLjAwDQozOQlHMwk2NC4wMAkwLjAwCTAuMDANCjQwCUc0CTE2MC4w MAkwLjAwCTAuMDANCjQxCUc1CTE5Mi4wMAkwLjAwCTAuMDANCjQyCUc2CTIyNC4wMAkwLjAwCTAu MDANCkVORF9EQVRBDQoAdGV4dAAAAABMR09ST1dMRU5HVEgJNg0KQ1JFQVRFRAkiNC8yMy8yMDA4 IiAgIyBUaW1lOiAxMjo1Mw0KSU5TVFJVTUVOVEFUSU9OCSJleWUtb25lIGRpc3BsYXkiDQpNRUFT VVJFTUVOVF9TT1VSQ0UJIklsbHVtaW5hdGlvbj1Vbmtub3duCU9ic2VydmVyQW5nbGU9VW5rbm93 bglXaGl0ZUJhc2U9QWJzCUZpbHRlcj1Vbmtub3duIg0KS0VZV09SRAkiU2FtcGxlSUQiDQpLRVlX T1JECSJTQU1QTEVfTkFNRSINCk5VTUJFUl9PRl9GSUVMRFMJOA0KQkVHSU5fREFUQV9GT1JNQVQN ClNhbXBsZUlECVNBTVBMRV9OQU1FCVhZWl9YCVhZWl9ZCVhZWl9aCUxBQl9MCUxBQl9BCUxBQl9C DQpFTkRfREFUQV9GT1JNQVQNCk5VTUJFUl9PRl9TRVRTCTQyDQpCRUdJTl9EQVRBDQoxCUExCTAu MTAJMC4xNAkwLjM0CTEuMjYJLTEuNDMJLTQuMTkNCjIJQTIJNC41MgkyLjQ3CTI0LjQyCTE3Ljgw CTM0LjU3CS03NS4wMQ0KMwlBMwkyMC4yNQkxMC45MQkxMDkuNzkJMzkuNDMJNTguMjcJLTEyNC40 Mw0KNAlBNAk3LjgxCTE2LjIwCTIuNzgJNDcuMjQJLTU2LjIzCTQ0LjQ2DQo1CUE1CTEyLjIzCTE4 LjU0CTI2Ljg3CTUwLjE0CS0zMy44NgktMjMuNTgNCjYJQTYJMjcuOTcJMjYuOTcJMTEyLjE4CTU4 Ljk1CTcuOTIJLTkyLjM2DQo3CUIxCTM1LjA1CTcyLjc3CTExLjIwCTg4LjM0CS05Mi44OQk3Ny4w OQ0KOAlCMgkzOS40Ngk3NS4wNgkzNS4zNQk4OS40MgktODMuMTgJMzAuOTgNCjkJQjMJNTUuMDMJ ODMuMzEJMTIwLjE3CTkzLjE1CS01NS43MwktMzguNTMNCjEwCUI0CTExLjMwCTYuMzQJMC41OAkz MC4yNQk0NS4zNgk0MS4yMQ0KMTEJQjUJMTUuNzEJOC42NwkyNC42MAkzNS4zNAk1MS44MQktNDUu MTENCjEyCUI2CTMxLjQxCTE3LjA4CTEwOS42NQk0OC4zNgk2Ni42MQktMTA4Ljk0DQoxMwlDMQkx OS4wMAkyMi4zOAkzLjAxCTU0LjQzCS0xMi42NAk1NS4xMg0KMTQJQzIJMjMuNDEJMjQuNzEJMjcu MDQJNTYuNzkJLTEuODMJLTEyLjQwDQoxNQlDMwkzOC45OQkzMy4wMAkxMTEuNzkJNjQuMTYJMjQu MjIJLTgzLjEyDQoxNglDNAk0Ni4xNgk3OC43NQkxMS40Mgk5MS4xMgktNzAuNTgJODEuMjMNCjE3 CUM1CTUwLjQ4CTgwLjkzCTM1LjM0CTkyLjEwCS02Mi45NgkzNS42MQ0KMTgJQzYJNjYuMDUJODku MjQJMTIwLjEwCTk1LjY4CS00MC42MgktMzQuMTMNCjE5CUQxCTUwLjM5CTI3Ljc1CTEuMzIJNTku NjYJNzYuNjMJODAuMDYNCjIwCUQyCTU0Ljc2CTMwLjA1CTI1LjMxCTYxLjcwCTc5LjE2CS0wLjky DQoyMQlEMwk3MC4zNQkzOC40MgkxMTAuMTAJNjguMzMJODYuNjQJLTc0LjgxDQoyMglENAk1OC4w NQk0My43NgkzLjc1CTcyLjA3CTQyLjU4CTgwLjQ2DQoyMwlENQk2Mi4zOQk0Ni4wMgkyNy43NAk3 My41Ngk0Ni40NAkxNS4zMw0KMjQJRDYJNzcuOTUJNTQuMzkJMTEyLjMwCTc4LjY5CTU3LjYzCS01 OC40MA0KMjUJRTEJODUuMDEJOTkuODcJMTIuMTIJOTkuOTUJLTIwLjM0CTk0LjM3DQoyNglFMgk4 OS4zNAkxMDIuMTEJMzYuMDQJMTAwLjgxCS0xNi4wNQk0OS42Mw0KMjcJRTMJMTA0LjkyCTExMC40 MAkxMjAuNjUJMTAzLjg5CS0yLjQ4CS0yMC4zMg0KMjgJRTQJMC4zMAkwLjI0CTEuNDIJMi4xOQky LjU0CS0yMC4zMg0KMjkJRTUJMS4wNQkwLjY0CTUuNDkJNS43NwkxNi44MgktNDMuNTENCjMwCUU2 CTcuMTYJMy44NgkzOC44MAkyMy4yMQk0MS4xMQktODcuOTMNCjMxCUYxCTEwLjgyCTUuODAJNTgu NzIJMjguODkJNDcuNjUJLTEwMS4xOA0KMzIJRjIJMTUuMzEJOC4xOQk4My4wOQkzNC4zOAk1My42 MAktMTEzLjYyDQozMwlGMwkwLjQzCTAuODMJMC40NAk3LjUwCS0xNC44Nwk0LjU3DQozNAlGNAkx LjcwCTMuNDYJMC44MwkyMS44MQktMzIuODYJMjEuOTQNCjM1CUY1CTEyLjU1CTI2LjExCTQuMjgJ NTguMTQJLTY2LjE3CTUzLjI2DQozNglGNgkxOC42MgkzOC43OAk2LjE5CTY4LjU5CS03NS41OQk2 MS40OA0KMzcJRzEJMjYuNTMJNTUuMjIJOC42NQk3OS4xNwktODUuMDAJNjkuNzgNCjM4CUcyCTAu NjIJMC40MgkwLjM2CTMuODIJOC42OQktMC4xMg0KMzkJRzMJMi40NwkxLjQ1CTAuMzkJMTIuMjkJ MjUuNDUJMTMuOTANCjQwCUc0CTE4LjM3CTEwLjI1CTAuNzMJMzguMjkJNTMuNjkJNTIuMjINCjQx CUc1CTI3LjIyCTE1LjE0CTAuOTEJNDUuODMJNjEuNDkJNjIuMDQNCjQyCUc2CTM4LjgwCTIxLjQ4 CTEuMTUJNTMuNDcJNjkuNjkJNzEuNjgNCkVORF9EQVRBDQoAAHRleHQAAAAAVmVyc2lvbiA9ICJQ cm9maWxpbmdMaWIgNS4wLjYgMS43NS4xNi4yIE1heSAzMSAyMDA2Ig1Qcm9maWxlclR5cGUgPSAz DUdvcE1vZGUgPSBmYWxzZQ1Qcm9maWxlU2l6ZSA9IDANVXNlTGluZWFyaXphdGlvbiA9IDENUGVy Y2VwdHVhbCA9IDENT3B0aW9ucyA9IHsNCUNvcHlyaWdodCA9ICJDb3B5cmlnaHQgYnkgTE9HTyBH bWJILCBTdGVpbmZ1cnQiDQlCbGFja0NvbXBlbnNhdGlvbiA9ICIwIg0JLk1udHJUYXJnZXRHYW1t YSA9ICIyLjIiDQkuTW50clRhcmdldFRlbXBlcmF0dXJlID0gIjY1MDAiDQkuTW50clRhcmdldEx1 bWluYW5jZSA9ICIxMjAiDQkuTW50ck1pbkx1bWluYW5jZSA9ICIwLjEiDQkuTW50ckx1bWluYW5j ZSA9ICIxMTAiDQkuTW50clRlbXBlcmF0dXIgPSAiNjUwMCINCU1vbml0b3JHYW1tYU1vZGVsID0g IjEiDX0NAAAAc2YzMgAAAAAAAQsLAAAH/v//8lIAAAW+AAEAgv//+or///+z////HQAAwk1YWVog AAAAAAAAez0AAEJ1AAABblhZWiAAAAAAAABX6AAAqQkAABKPWFlaIAAAAAAAACOxAAAUggAAvy9Y WVogAAAAAAAA9tYAAQAAAADTLVhZWiAAAAAAAGjshABuZsgAeKeIY3VydgAAAAAAAAABAisAAGN1 cnYAAAAAAAAAAQIsAABjdXJ2AAAAAAAAAAECLgAAdmNndAAAAAAAAAAAAAMBAAABAAACAwQFBggJ CgsMDQ4PEBESExQVFhcYGhobHB0eHyAhIiMkJSYnKCkqKywsLS4vMDEyMzQ1Njc4ODk6Ozw9Pj9A QUJDQ0RFRkdISUpLTE1NTk9QUVJTU1RVVldYWVpaW1xdXl9gYWFiY2RlZmdoaWprbGxub3BxcnN0 dXZ4eXt8fX5/gIGCg4SFhoeIiYqLjI2Oj5CRkpOUlZaWl5iZmpqbnJ2en6ChoqOkpaWlpqeoqaqs ra6vsLGys7S1tre4ubq7vL2+v8DBwsPExsfIycrLzM3Oz9DR0tPV1tfY2drb3N7g4eLj5OXm6Onq 6+3v8PHy8/X29/j5/P39/f7+/gAAAgMEBQYICQoLDA0ODxAREhMUFRYXGBkaGxwdHR4fICEiIyQl JicnKCkqKywtLi8vMDEyMzQ1NjY3ODk6Ozw8PT4/QEFCQ0NERUZHSElJSktMTU5PT1BRUlNTVFVW V1hYWVpbXF1dXl9gYWFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHp7fH1+f4GCg4OEhYaHiImKiouM jY6PkJGSkpOUlZWWl5iZmZqbnJ2en5+goaGjpKWmp6ipqqusrK2ur7CxsrO0tba3uLm6u7y9vr/A wcLDxMXGxsfIycrMzc7P0NHS09TV1tfY2drb3N3e4OHi4+Tl5ufo6err6+zs7e0AAAIDBAUGBwgJ CgsMDQ4PEBESExQVFhYXGBkZGxwdHh8gISEiIyQlJicnKCkqKywsLS4vMDExMjM0NTY2Nzg5Ojs7 PD0+P0BAQUJDREVFRkdISUlKS0xNTk5PUFFRUlNUVVVWV1hYWVpbW1xdXl9fYGFiYmNkZWZmZ2hp amtsbW5vcHFyc3R1dnh5ent8fX5/gIGCgoOEhYaGh4iJioqLi4yNjo6PkJGSk5OUlZaWl5eYmZuc nZ2en6ChoqKjpKWmp6eoqaqrrK2urq+wsrO0tba3t7i5uru8vr/AwMHCw8TFxsfIycrLzM3Oz9DR 0tPU1dbX2Nna3N7f4ODh4eLigICBgYKCg4OEhIWFhoaHh4iIiYmKiouLjIyNjY4AZGF0YQAAAAAA AAAB4xEO7ZSBoKBf+/wg7Sx2L/jQNxRl/AieWTyWmNsw1XNd5w7G1b5MCdhQ1ll37xQtSvESKjE2 gj38wdfXid1uZmohm87Sf6phqZPYhjsqG6xm+RWvqcMiaW1w2EB93RzRAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAGRlc2MAAAAAAAAAETIzTWFpbl80LTIzLTA4XzEAAAAAAAAAAAAAABEy M01haW5fNC0yMy0wOF8xAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAA== --_003_3D94E9A296F84F93AAB209CB87418945evileyepicturescom_ Content-Type: text/plain; name="ATT00001.c" Content-Description: ATT00001.c Content-Disposition: attachment; filename="ATT00001.c"; size=80; creation-date="Wed, 02 Jun 2010 20:46:02 GMT"; modification-date="Wed, 02 Jun 2010 20:46:02 GMT" Content-Transfer-Encoding: base64 DQoNCg0KRXZpbCBFeWUgUGljdHVyZXMNCmRicm93bmVAZXZpbGV5ZXBpY3R1cmVzLmNvbQ0KT2Zm aWNlOiAoNDE1KSA3NzctMDY2Ng0KDQo= --_003_3D94E9A296F84F93AAB209CB87418945evileyepicturescom_-- *** ***************************************************** *** Message #1936 ***************************************************** *** ***************************************************** Subject: Re: can't render passes from maya in MR standalone via rush From: Greg Ercolano Date: Tue, 18 May 2010 16:04:16 -0400 Christopher Janney wrote: > [posted to rush.general] > > Using the current 'submit-maya.pl' script with 'mr standalone' selected, > we get the following Render command string: > > Render -r mi -proj > //sanD/proj1/CGI/anotherFancyJob/shots/shot81/work.artist/maya -s 55 -e > 55 -b 1 -file /tmp/.RUSH_TMP.1595/foo.mi -exportPathNames aaaaaaaaaa > //sanD/proj1/CGI/anotherFancyJob/shots/shot81/work.artist/maya/scenes/sht81_c01_a05_l07ks.mb > > That's fine for everything but render passes. Black frames, every one > of them. > > Any suggestions? First: are there any error messages from either the maya or ray passes that give an idea of what the problem is? eg. warnings or error messages? Look carefully at the output for both ray and maya. The problem could be caused during either the mi generation or ray. If not, I'd suggest opening a shell window and experimenting with the command line flags for both the Maya "Render" MI generation pass, and the Mental Ray "ray" render pass to see what makes it work. You should be able to copy/paste the "Render" and "ray" commands right out of the log into a shell window window to check them.. I'd just suggest you change the "/tmp/.RUSH_TMP.1595/foo.mi" in those commands to something else, like /var/tmp/foo.mi I don't think it's a problem in the submit script, but might just be some needed flags for either the Maya Flags: prompt or the Ray Flags: prompt. Or it might even be something that just needs to be changed in the scene file itself. When testing flags, it's probably best to go into the command line and play around with them so you can quickly make changes and see results. I'd suggest checking the help for both 'Render' and 'ray', and look for flags relating to passes/levels: Render -r mi -help ray -help Maya's online docs will cover the details of these flags; check into that to see if there's something that stands out. If you're still stuck, but can replicate from the shell, then I'd take it to the Maya related forums on rendering, pasting the Render and Ray commands you're using, and give some details about the scene's settings, and see what they say. If you can replicate purely from the command line, then it surely is a Maya/Mray specific issue that they can help with. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1935 ***************************************************** *** ***************************************************** Subject: can't render passes from maya in MR standalone via rush From: Christopher Janney Date: Tue, 18 May 2010 15:17:20 -0400 Using the current 'submit-maya.pl' script with 'mr standalone' selected, we get the following Render command string: Render -r mi -proj //sanD/proj1/CGI/anotherFancyJob/shots/shot81/work.artist/maya -s 55 -e 55 -b 1 -file /tmp/.RUSH_TMP.1595/foo.mi -exportPathNames aaaaaaaaaa //sanD/proj1/CGI/anotherFancyJob/shots/shot81/work.artist/maya/scenes/sht81_c01_a05_l07ks.mb That's fine for everything but render passes. Black frames, every one of them. Any suggestions? Thank you, -ctj *** ***************************************************** *** Message #1934 ***************************************************** *** ***************************************************** Subject: [Q+A] Newsgroup FAQ -- Summary of common questions/answers From: Greg Ercolano Date: Thu, 13 May 2010 14:01:27 -0400 Here's a summary of newsgroup items that I find myself referring people to often via email: * ADDING 'RUSH' TO THE MAYA MAIN MENU You can set up Maya so that users can access the 'submit-maya' form from within Maya's main menu: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1631 * USING MAYA + VRAY WITH RUSH There are a few techniques to get old submit-maya scripts to work with Vray; see this thread for details: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1922+1933 * DETAILS ON RUSH'S "FIFO" SCHEDULING FIFO scheduling was introduced in 102.42a9, and a maintenance fix in 102.42a9c made it stable for multiple job servers. There are some details on how Rush's FIFO scheduling option works here: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1871+1872+1873+1874 * [OSX] SETTING UP NFS MOUNTS ON BOOT Apple has many GUI ways to configure file server mounts, but it's hard to automate, and has invisible behaviors that are sometimes not appropriate for 24/7 operation. A better, more 'unixy' approach is to make a boot script that runs regular mount(8) commands. Such a script can then be easily distributed to all workstations/rendernodes (with e.g. rsync(1) or scp(1)) to ensure consistent mount behaviors. More info here: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1847+1847 * [OSX] SETTING UP A SMALL NFS NETWORK If you are new to OSX, and need to get a small network of OSX talking to each other, see this posting for details on setting up a small OSX network using regular unix commands: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1295+1297+1298+1839 * [WINDOWS] DEALING WITH DRIVE MAPS If you're using drive letters (instead of UNC paths), then you need some way to make them work reliably, as they are erratic by design: http://www.seriss.com/rush-current/rush/rush-td-faq.html#TDFAQ-UNC If you *must* use drive letters, here's ways to force them to work: a) With Maya, you can configure the MEL 'dirmap' command in the Maya startup scripts to replace drive letters with UNC equivalents: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1312 -- OR -- b) Configure the submit script to force drives to be mapped: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1222 I've had several confirmations the (a) approach works well, and is likely the best solution when maya is involved, as it ensures UNCs are actually used. The (b) approach tries to force Windows to make drive maps work, and will probably work 'most of the time', but using UNCs is best. Note that in recent releases of Windows (Vista, Windows7), actual working 'symbolic links' are now an option. For details on making symlinks under windows, see this thread: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1553+1554+1555+1556+1557+1566 * CUSTOM 'DONEMAIL' MESSAGES FROM RUSH Some folks like to configure custom email messages when a job finishes so that they can email their pagers/cellphones with short, specific contents: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1589 * USING EYEON'S "FUSION" WITH RUSH Fusion 5.1 (and higher) can now be used with Rush: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1530 * HOW TO GET RUSH TO SEE "csh" STYLE ENVIRONMENT VARIABLE SETTINGS Unless you are submitting CSH scripts, CSH settings in .cshrc files will not be 'sourced' by the perl submit scripts. Here's some ways to manage csh environment variables from within perl scritps: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1502 There are some other articles regarding user environment variables with respect to network rendering; see: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1501+1502+1503+1506+1505+1504 Also, some details on how to pass environment variable info from the user's environment during job submission to your render scripts: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+761+761 * [OSX] A COMPARISON OF AFP/SMB/NFS Often people ask about whether they should use AFP or SMB or NFS on OSX. The short answer is NFS. For details, see the following link that compares the different network file systems: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1062+1062 Also related: In Tiger, Apple broke AFP and SMB for multiuser access; see: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1013+1014+1015+1019+1020+1021+1024+1595 * HOW TO PARSE RUSH COMMAND LINE OUTPUT Here are some examples and insights into parsing rush's output into other scripting languages (including parsing error messages) with examples in perl: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1371+1371 [[ These items were culled from emails between me and customers via: ]] [[ egrep seriss.com.cgi-bin.rush.newsgroup | sed 's/^[ \t]*//' | sort | uniq ]] *** ***************************************************** *** Message #1933 ***************************************************** *** ***************************************************** Subject: Re: [Q+A] Maya + Vray From: Greg Ercolano Date: Tue, 11 May 2010 16:23:53 -0400 Greg Ercolano wrote: >> We'd like to render with Vray for Maya. >> >> But our submit-maya script doesn't have a 'vray' option >> in the submit form for "Renderer".. what should we do to render with Vray? > > [..]it's easy to add it; just find this (long) line in the submit-maya script: > > Renderer: "default,maya(sw),mentalray(mr),mentalray-standalone(mi),mi-gen(mi),hardware(hw),vector(vr),renderman(rman),ribgen(rib),3Delight(3delight),maxwell(maxwell)" ? > > ..and add ",vray(vray)" to the end of this line before the closing quote. > For example: > > BEFORE: Renderer: "default,maya(sw),mentalray(mr)......,3Delight(3delight),maxwell(maxwell)" ? > AFTER: Renderer: "default,maya(sw),mentalray(mr)......,3Delight(3delight),maxwell(maxwell),vray(vray)" ? > ^^^^^^^^^^^ > ..that way 'vray' will be in the Renderer: pulldown so you can select it > to force the render to run with vray. Just a followup -- I should add that this has also been brought up on the Chaos Group's vray forum list here: http://www.chaosgroup.com/forums/vbulletin/showthread.php?49195-Rush-render-queue-Vray-duplicate-frames-jobs-loops&p=425308#post425308 I think you need to be a member to read that article, but it's basically the same as the above; some users included a few details on the issues they were having. Be sure, of course, to enable the vray plugin in the maya plugin manager, and you may need to set the 'vray' plugin to 'autostart' (this from someone who was evaling the vray plugin, and was initially having trouble with the vray plugin using vray standalone licenses) *** ***************************************************** *** Message #1932 ***************************************************** *** ***************************************************** Subject: Re: How to treat limited render software licenses? From: Date: Tue, 27 Apr 2010 16:52:29 -0400 On 2010-03-16 08:20:17 -0700, "Abraham Schneider" said: > Sorry, maybe this is a stupid question, but until now we only had Shake > and unlimited render licenses, so I haven't had this problem: Everyone moving from Shake to Nuke has run into this same issue. > > how do I manage a limited number of render licenses of my production > software (in my case: Nuke) without restricting myself too much in the > hosts file? Buy more Nuke render licenses, render on fewer machines, or define fewer CPUs per workstation in your host file. > > What I'd like to do: install Nuke on as many machines (renderfarm and > user workstations) as possible. Add a +nuke group in the hosts file to > all these machines that have nuke on it. Then I'd like to submit my nuke > scripts with '+nuke=x@100' where x is the number of render licenses > that I bought. We used to define an 8-core Xserve as having 4 CPUs in the host file, but when switching to Nuke from Shake we ran into the problem of not having enough render licenses. To solve the problem temporarily we redefined the workstations as having 1 CPU instead of 4 and bumped up the threads and memory count in Nuke to maximise its use. It works well for certain Nuke jobs (with a lot of 3D rendering) but not all Nuke jobs. In the end we were tired of rush "licpause" errors and bought a lot more Nuke render licenses and redefined the render nodes to have 4 CPUs again. Works best, for the most part. :) Mat X The Embassy VFX *** ***************************************************** *** Message #1931 ***************************************************** *** ***************************************************** Subject: Re: SUID scripts in Perl From: Daniel Browne Date: Mon, 05 Apr 2010 21:52:14 -0400 That's kind of what I thought. Thanks Greg. On Apr 5, 2010, at 5:10 PM, Greg Ercolano wrote: [posted to rush.general] Daniel Browne wrote: > [posted to rush.general] >=20 > This is a bit off topic, but I thought if anyone knew it would be you, Gr= eg=3D > . Is there a way to do SUID perl scripts in MacOS X 10.6? I know some tim= e =3D > ago the direct method of setting the SUID permissions bit on a script was= b=3D > locked by Apple because of the security issue. Do you know of an alternat= iv=3D > e mechanism, other than resorting to C binary wrappers or sudo commands? = I =3D > can't find perlsec or suidperl executables in the standard Mac install. It's disabled by default on ALL platforms. It's the kernel that handles the #! stuff, so to turn it on would be a kernel tweak, so I'd expect sysctl would let you control it; see 'sysctl -a | grep script' But sudo and binary C wrappers are the approach I usually use, and in the case of C wrappers, carefully perm their execution bits so that only the appropriate users can run them. --=20 Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1930 ***************************************************** *** ***************************************************** Subject: Re: SUID scripts in Perl From: Greg Ercolano Date: Mon, 05 Apr 2010 20:10:50 -0400 Daniel Browne wrote: > [posted to rush.general] > > This is a bit off topic, but I thought if anyone knew it would be you, Greg= > . Is there a way to do SUID perl scripts in MacOS X 10.6? I know some time = > ago the direct method of setting the SUID permissions bit on a script was b= > locked by Apple because of the security issue. Do you know of an alternativ= > e mechanism, other than resorting to C binary wrappers or sudo commands? I = > can't find perlsec or suidperl executables in the standard Mac install. It's disabled by default on ALL platforms. It's the kernel that handles the #! stuff, so to turn it on would be a kernel tweak, so I'd expect sysctl would let you control it; see 'sysctl -a | grep script' But sudo and binary C wrappers are the approach I usually use, and in the case of C wrappers, carefully perm their execution bits so that only the appropriate users can run them. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1929 ***************************************************** *** ***************************************************** Subject: SUID scripts in Perl From: Daniel Browne Date: Mon, 05 Apr 2010 18:56:01 -0400 This is a bit off topic, but I thought if anyone knew it would be you, Greg= . Is there a way to do SUID perl scripts in MacOS X 10.6? I know some time = ago the direct method of setting the SUID permissions bit on a script was b= locked by Apple because of the security issue. Do you know of an alternativ= e mechanism, other than resorting to C binary wrappers or sudo commands? I = can't find perlsec or suidperl executables in the standard Mac install. Thanks, -Dan= *** ***************************************************** *** Message #1928 ***************************************************** *** ***************************************************** Subject: How to treat limited render software licenses? From: "Abraham Schneider" Date: Tue, 16 Mar 2010 11:20:17 -0400 Hi! Sorry, maybe this is a stupid question, but until now we only had Shake=20 and unlimited render licenses, so I haven't had this problem: how do I manage a limited number of render licenses of my production=20 software (in my case: Nuke) without restricting myself too much in the=20 hosts file? What I'd like to do: install Nuke on as many machines (renderfarm and=20 user workstations) as possible. Add a +nuke group in the hosts file to=20 all these machines that have nuke on it. Then I'd like to submit my nuke = scripts with '+nuke=3Dx@100' where x is the number of render licenses = that=20 I bought. All this works fine as long as there is only one render job on the farm. = But after submitting a second job, this one will start on another x=20 machines, which obviously will fail because I don't have enough render=20 licenses. One solution would be to only add x machines to the +nuke group in my=20 hosts file so I do a hard limit on the machines. That's what I'm doing=20 at the moment with our plugins like Furnace, Sapphire, etc. There I have = only 2 render licenses or so, so it is easy to maintain the hosts file=20 if a machine dies or something like this. But to manage this for 10, 20=20 or more render licenses (but less render licenses than available render=20 machines!) would be really problematic, especially because I have many=20 workstations that are only part time online in rush if a user is away=20 from his machine. So is there a better way to deal with this whole license problem and I=20 just haven't found it in the manual, or do I have to do this with the=20 number of machines I'd assign to a group? Thanks in advance, Abraham Abraham Schneider Senior VFX Compositor =20 ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv Sitz: M=FCnchen Registergericht: Amtsgericht M=FCnchen Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Dr. Martin Prillmann; Thomas Till *** ***************************************************** *** Message #1927 ***************************************************** *** ***************************************************** Subject: Re: [SUBMIT] auto confirm/submit Form From: Stephan Kosinski Date: Fri, 05 Mar 2010 14:41:54 -0500 Thank you so much Greg. I'll try that. Stephan. Greg Ercolano wrote: > Stephan Kosinski wrote: >> [posted to rush.general] >> >> Thank you very much Greg, it works perfectly. >> >> That will even allows me to generate one file per layer to render. >> >> One last thing, the "Be very Happy" window is still popping-up, which is >> really nice :) , but annoying when I have several to close after I >> launched several forms at a time. Any way to prevent this? > > Ah, that. > > Yes, I recently added the ability to turn that off, which will > be in the next release. (The new code takes a flexible argument > list that lets one disable that dialog) > > The code responsible for opening that dialog is in the RushSubmit() > function in .common.pl, specifically: > > ---- > open(STARTIRUSH, "|$inputcmd"); > print STARTIRUSH "-----------------\n" . > "--- SUBMIT OK ---\n" . > "-----------------\n\n" . > $msg; > unless ( close(STARTIRUSH) ) { return(0); } # picked "No Irush" > ---- > > What I'd suggest as a workaround is putting a conditional around that code > that checks an environment variable (which you'd set in your application > to turn that off), ie: > > ---- > if ( ! defined($ENV{RUSH_NO_SUCCESS_DIALOG}) ) ### ADD THIS LINE > { ### ADD THIS LINE > open(STARTIRUSH, "|$inputcmd"); > print STARTIRUSH "-----------------\n" . > "--- SUBMIT OK ---\n" . > "-----------------\n\n" . > $msg; > unless ( close(STARTIRUSH) ) { return(0); } # picked "No Irush" > } ### ADD THIS LINE > ---- > > ..then in your script, just before you invoke the submit script to > submit the job, set that environment variable RUSH_NO_SUCCESS_DIALOG to 1. > That will prevent the dialog from opening. > > You can set environment variables from all scripting languages, > including 'mel', which I think would be with: > > putenv("RUSH_NO_SUCCESS_DIALOG", "1"); > > Meanwhile other users invoking the submit script interactively > will continue to have the normal behavior, because that variable > would not be set. > > Let me know if that doesn't work.. I'd think it should. > *** ***************************************************** *** Message #1926 ***************************************************** *** ***************************************************** Subject: Re: [SUBMIT] auto confirm/submit Form From: Greg Ercolano Date: Thu, 04 Mar 2010 23:45:20 -0500 Stephan Kosinski wrote: > [posted to rush.general] > > Thank you very much Greg, it works perfectly. > > That will even allows me to generate one file per layer to render. > > One last thing, the "Be very Happy" window is still popping-up, which is > really nice :) , but annoying when I have several to close after I > launched several forms at a time. Any way to prevent this? Ah, that. Yes, I recently added the ability to turn that off, which will be in the next release. (The new code takes a flexible argument list that lets one disable that dialog) The code responsible for opening that dialog is in the RushSubmit() function in .common.pl, specifically: ---- open(STARTIRUSH, "|$inputcmd"); print STARTIRUSH "-----------------\n" . "--- SUBMIT OK ---\n" . "-----------------\n\n" . $msg; unless ( close(STARTIRUSH) ) { return(0); } # picked "No Irush" ---- What I'd suggest as a workaround is putting a conditional around that code that checks an environment variable (which you'd set in your application to turn that off), ie: ---- if ( ! defined($ENV{RUSH_NO_SUCCESS_DIALOG}) ) ### ADD THIS LINE { ### ADD THIS LINE open(STARTIRUSH, "|$inputcmd"); print STARTIRUSH "-----------------\n" . "--- SUBMIT OK ---\n" . "-----------------\n\n" . $msg; unless ( close(STARTIRUSH) ) { return(0); } # picked "No Irush" } ### ADD THIS LINE ---- ..then in your script, just before you invoke the submit script to submit the job, set that environment variable RUSH_NO_SUCCESS_DIALOG to 1. That will prevent the dialog from opening. You can set environment variables from all scripting languages, including 'mel', which I think would be with: putenv("RUSH_NO_SUCCESS_DIALOG", "1"); Meanwhile other users invoking the submit script interactively will continue to have the normal behavior, because that variable would not be set. Let me know if that doesn't work.. I'd think it should. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1925 ***************************************************** *** ***************************************************** Subject: Re: [SUBMIT] auto confirm/submit Form From: Stephan Kosinski Date: Thu, 04 Mar 2010 21:41:44 -0500 Thank you very much Greg, it works perfectly. That will even allows me to generate one file per layer to render. One last thing, the "Be very Happy" window is still popping-up, which is really nice :) , but annoying when I have several to close after I launched several forms at a time. Any way to prevent this? Thank you again for your precious help. Stephan. Greg Ercolano wrote: > Stephan Kosinski wrote: >> I am using this kind of command to generate and submit maya render form: >> >> submit-maya.pl -field JobTitle jobname -field ScenePath myScenePath >> etc..... >> >> It is really convenient since I can launch renders within maya but >> would be even more convenient I didn't have to click on the "Submit" >> button to launch the render on the farm. >> >> Is there a way to auto-confirm the form so it goes to the farm as soon >> as hit "enter" in my shell? > > Yes. It should be as easy as: > > perl //path/to/submit-maya.pl -submit /var/tmp/myfile.txt > > ..where the /var/tmp/myfile.txt contains the key/value pairs > of the submit form, which you can generate from within maya, > or from another script. > > When you run that command, the job will go straight to the farm. > > To do that, you can simulate what the submit script does when the > user hits the 'Submit' button, basically these two steps: > > 1) Save out a simple text file that contains the key/value pairs > for all the submit form's fields (see below) > > 2) Invoke the submit script with the arguments "-submit file" > where "file" is the pathname of the above text file. eg: > > perl //path/to/submit-maya.pl -submit /var/tmp/myfile.txt > > ..where /var/tmp/myfile.txt is a file that contains key/value pairs > of all the submit fields. > > When you submit a maya job, a copy of this key/value file > is saved in the user's home directory as: > > UNIX: ~/.rush/.submit-maya-last > WINDOWS: c:\temp\.rush-\.submit-maya-last > > So for instance, to resubmit the last maya job you submitted, > under unix you might do: > > perl //path/to/submit-maya.pl -submit ~/.rush/.submit-maya-last > > To generate the key/value file yourself, you can nab a copy of the > one the rush submit form generates, and use that as a template in your > own script, changing the fields as you want (Frames:, Cpus:, etc). > > It's important that the key/value file have the same fields the submit > script expects. Be aware that newer versions of the submit-maya script > sometimes have new fields, so your app would need to keep in sync with these. > > What follows is a unix bash script that does this. > If you need an example in some other language, let me know. > > Note you may need to modify the key/value data with one of your own > submit-maya-last files, and you'll need to change the pathname of > to the submit script for the location on your file server. > > * * * > > #!/bin/sh > > ### CREATE A KEY VALUE PAIR FILE > cat << EOF > /var/tmp/myfile.txt > AutoDump: off > BatchClip: yes > BatchFrames: 1 > Cpus: +any=5 > Debug: off > DoneMail: > DumpMail: > Frames: 1-10 > ImageCommand: > ImageDirectory: > JobDoneCommand: > JobDumpCommand: > JobStartCommand: > JobTitle: > LicenseBehavior: Licpause+Retry > LogDirectory: > LogFlags: Overwrite > MaxLogSize: 0 > MaxTime: 00:00:00 > MaxTimeState: Que > MayaFlags: > MrayCommand: ray > MrayFlags: > MrayVerbosity: 5 > NeverUseCpus: > PrintEnvironment: off > Project: > Ram: 1 > Renderer: maya(sw) > Retries: 3 > RetryBehavior: Fail > SaveMIDirectory: > ScenePath: //meade/net/tmp/scenes/foo.ma > StartIrush: yes > SubmitHosts: > SubmitOptions: > Threads: all > WaitFor: > WaitForState: Dump > _SAVE_ID: Maya-Submit > EOF > > ### NOW RUN THE SUBMIT-MAYA SCRIPT TO SUBMIT IT AS A JOB > perl //yourserver/rushscripts/submit-maya.pl -submit /var/tmp/myfile.txt > *** ***************************************************** *** Message #1924 ***************************************************** *** ***************************************************** Subject: Re: [SUBMIT] auto confirm/submit Form From: Greg Ercolano Date: Thu, 04 Mar 2010 21:14:34 -0500 Stephan Kosinski wrote: > I am using this kind of command to generate and submit maya render form: > > submit-maya.pl -field JobTitle jobname -field ScenePath myScenePath > etc..... > > It is really convenient since I can launch renders within maya but > would be even more convenient I didn't have to click on the "Submit" > button to launch the render on the farm. > > Is there a way to auto-confirm the form so it goes to the farm as soon > as hit "enter" in my shell? Yes. It should be as easy as: perl //path/to/submit-maya.pl -submit /var/tmp/myfile.txt ..where the /var/tmp/myfile.txt contains the key/value pairs of the submit form, which you can generate from within maya, or from another script. When you run that command, the job will go straight to the farm. To do that, you can simulate what the submit script does when the user hits the 'Submit' button, basically these two steps: 1) Save out a simple text file that contains the key/value pairs for all the submit form's fields (see below) 2) Invoke the submit script with the arguments "-submit file" where "file" is the pathname of the above text file. eg: perl //path/to/submit-maya.pl -submit /var/tmp/myfile.txt ..where /var/tmp/myfile.txt is a file that contains key/value pairs of all the submit fields. When you submit a maya job, a copy of this key/value file is saved in the user's home directory as: UNIX: ~/.rush/.submit-maya-last WINDOWS: c:\temp\.rush-\.submit-maya-last So for instance, to resubmit the last maya job you submitted, under unix you might do: perl //path/to/submit-maya.pl -submit ~/.rush/.submit-maya-last To generate the key/value file yourself, you can nab a copy of the one the rush submit form generates, and use that as a template in your own script, changing the fields as you want (Frames:, Cpus:, etc). It's important that the key/value file have the same fields the submit script expects. Be aware that newer versions of the submit-maya script sometimes have new fields, so your app would need to keep in sync with these. What follows is a unix bash script that does this. If you need an example in some other language, let me know. Note you may need to modify the key/value data with one of your own submit-maya-last files, and you'll need to change the pathname of to the submit script for the location on your file server. * * * #!/bin/sh ### CREATE A KEY VALUE PAIR FILE cat << EOF > /var/tmp/myfile.txt AutoDump: off BatchClip: yes BatchFrames: 1 Cpus: +any=5 Debug: off DoneMail: DumpMail: Frames: 1-10 ImageCommand: ImageDirectory: JobDoneCommand: JobDumpCommand: JobStartCommand: JobTitle: LicenseBehavior: Licpause+Retry LogDirectory: LogFlags: Overwrite MaxLogSize: 0 MaxTime: 00:00:00 MaxTimeState: Que MayaFlags: MrayCommand: ray MrayFlags: MrayVerbosity: 5 NeverUseCpus: PrintEnvironment: off Project: Ram: 1 Renderer: maya(sw) Retries: 3 RetryBehavior: Fail SaveMIDirectory: ScenePath: //meade/net/tmp/scenes/foo.ma StartIrush: yes SubmitHosts: SubmitOptions: Threads: all WaitFor: WaitForState: Dump _SAVE_ID: Maya-Submit EOF ### NOW RUN THE SUBMIT-MAYA SCRIPT TO SUBMIT IT AS A JOB perl //yourserver/rushscripts/submit-maya.pl -submit /var/tmp/myfile.txt -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1923 ***************************************************** *** ***************************************************** Subject: [SUBMIT] auto confirm/submit Form From: Stephan Kosinski Date: Thu, 04 Mar 2010 20:49:21 -0500 Hello, I am using this kind of command to generate and submit maya render form: submit-maya.pl -field JobTitle jobname -field ScenePath myScenePath etc..... It is really convenient since I can launch renders within maya but would be even more convenient I didn't have to click on the "Submit" button to launch the render on the farm. Is there a way to auto-confirm the form so it goes to the farm as soon as hit "enter" in my shell? Thank you very much for your help. Stephan. *** ***************************************************** *** Message #1922 ***************************************************** *** ***************************************************** Subject: [Q+A] Maya + Vray From: Greg Ercolano Date: Tue, 02 Mar 2010 04:49:38 -0500 > We'd like to render with Vray for Maya. > > But our submit-maya script doesn't have a 'vray' option > in the submit form for "Renderer".. what should we do to render with Vray? Just set the "Renderer:" field to "default" in the submit form, and it will use the renderer the scene file was set to use. So if the scene was set to use vray, that's what Maya will use to render the scene if the job was submitted with "Renderer: default". That goes for just about all the Maya plugin renderers. That's all you should need. [NOTE: Some releases of the maya vray plugin don't support this, in which case use the below] If, however, you want to be able to /force/ Maya renders to use vray even if the scene isn't set up to use it, and you have a submit-maya script that doesn't have 'vray' in the Renderer: pulldown menu, it's easy to add it; just find this (long) line in the submit-maya script: Renderer: "default,maya(sw),mentalray(mr),mentalray-standalone(mi),mi-gen(mi),hardware(hw),vector(vr),renderman(rman),ribgen(rib),3Delight(3delight),maxwell(maxwell)" ? ..and add ",vray(vray)" to the end of this line before the closing quote. For example: BEFORE: Renderer: "default,maya(sw),mentalray(mr)......,3Delight(3delight),maxwell(maxwell)" ? AFTER: Renderer: "default,maya(sw),mentalray(mr)......,3Delight(3delight),maxwell(maxwell),vray(vray)" ? ^^^^^^^^^^^ ..that way 'vray' will be in the Renderer: pulldown so you can select it to force the render to run with vray. But it's much easier to just leave the submit form set to "default". *** ***************************************************** *** Message #1921 ***************************************************** *** ***************************************************** Subject: Re: nuke path From: Greg Ercolano Date: Tue, 26 Jan 2010 15:16:52 -0500 I believe the problem here was one of permissions on the license file or directory, where the user the Rush render was running as didn't have read permission to the license file, so nuke couldn't load it. The way to debug this sort of thing is to login to the machine in question as the user the rush render is running as, which you can see from the "Host:" and "RunningAs:" lines in the rush log. To determine the problem, you would login as that user on that machine, and then try to 'more' the license file to see if you have permission to read it, eg: more "/Applications/Nuke5.0v2/Nuke5.0v2.app/Contents/MacOS/*.lic" more "/usr/local/foundry/FLEXlm/foundry.lic" more "/Library/Application Support/TheFoundry/FLEXlm/*.lic" Note that quotes are needed if the pathnames contain spaces. The above are unix commands, though similar commands should work in DOS as well, only you'd need to avoid the '*' wildcards, as the DOS shell doesn't handle wildcard expansions the same way as unix shells. (In DOS, wild cards are handled by the command, not the shell, and only certain DOS commands, like "DIR", can handle wildcards) > [..] now I get: > > [..] > Nuke 5.0v2, built May 28 2008, revision 6380. > Copyright (C) 2008 The Foundry Visionmongers Ltd. All Rights Reserved. > License failure: License server machine is down or not responding. > See the system adminstrator about starting the license server system, or > make sure you're referring to the right host (see LM_LICENSE_FILE). > Feature: nuke_i > Hostname: lserver > License path: /Applications/Nuke5.0v2/Nuke5.0v2.app/Contents/MacOS/*.lic: [..] > /Library/Application Support/TheFoundry/FLEXlm/*.lic:/usr/local/foundry/FLEXlm/foundry.lic > FLEXnet Licensing error:-96,7. System Error: 2 "No such file or directory" > For further information, refer to the FLEXnet Licensing End User Guide, > available at "www.macrovision.com". > --- NUKE LICENSE ERROR: EXITCODE=100 *** ***************************************************** *** Message #1920 ***************************************************** *** ***************************************************** Subject: Re: Batch Copy Script From: Greg Ercolano Date: Wed, 16 Dec 2009 14:27:09 -0500 Mr. Daniel Browne wrote: >>> The alternative of course is to do a cp operation for each frame, >>> but I was hoping to avoid the additional overhead. >> >> I wouldn't think the cp command would be much overhead, >> but I might be missing something. >> >> If you're trying to distribute the cp commands to a bunch >> of machines to parallelize the I/O, I'd think you'd want >> to do them either as single frames or in batches. >> >> Or better yet, batch several commands on the file server >> itself, eg: >> >> cp foo.0*.tif /some/other/dir & # copies 1000 = frames (0000 - 0999) in bg >> cp foo.1*.tif /some/other/dir & # copies 1000 = frames (1000 - 1999) in bg >> cp foo.2*.tif /some/other/dir & # etc.. >> >> ..which if the box has multiple procs and the network is idle, >> should go really fast. > Yes, this is what I wanted to go for. The problem obviously is that you > can't wildcard a frame range as a whole; you have to wildcard each digit > with its potential range of values. Tedious, but possible. I was just > hoping you might have a library routine already in place to do this. If you're using perl, you can use perl's built in copy command and make a loop, eg: use File::Copy; [..] for ( $t=$sfrm; $t<$efrm; $t++ ) { my $src = sprintf("/some/src/path/foo.%04d.tif", $t); my $dst = sprintf("/some/dst/path/foo.%04d.tif", $t); copy($src,$dst); } ..however you might find the operating system's own copy command to be faster (these might be optimized to take advantage of threading) in which case: for ( $t=$sfrm; $t<$efrm; $t++ ) { my $src = sprintf("/some/src/path/foo.%04d.tif", $t); my $dst = sprintf("/some/dst/path/foo.%04d.tif", $t); system("cp $src $dst"); } ..and you could probably get 3 or 4 of those going at a time with use of fork()/wait() or threads. (I'd suggest throttling this to limit starting too many at a time, and using wait(), you can start the next one as soon as one gets done, so that there are always 3 or 4 going at a time, until you work your way through the entire loop) For instance, I have a script that I wrote a while ago: http://seriss.com/people/erco/unixtools/prun ..the blurb description: ********************************************************************************* Runs unix commands in parallel Supply a list of commands to run, one per line, and it runs them all in parallel. You can limit the number of parallel processes with the -M flag. Even keeps track of all the stdout/stderr logs, serializing them for easy reading. Hit ^C to stop all processes. Great for multiple rsh, rcp or rdists to run on a whole network quickly. Very small, simple program no one has time to write, but everyone always needs when managing multiple hosts. ********************************************************************************* So it takes a file of one liner unix commands, and runs them in parallel with a throttle, backgrounding several at a time, each time one command gets done it moves further onto the next in the list (like rush's frame queue) keeping the throttle maxed. It was handy for running commands many copy commands in parallel and keeping the output syncrhonized. Might be helpful with or without Rush..! I hope it still works.. it looks like I wrote it last century, as it still uses pre-perl5 syntax. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1919 ***************************************************** *** ***************************************************** Subject: Re: Batch Copy Script From: "Mr. Daniel Browne" Date: Wed, 16 Dec 2009 13:24:03 -0500 >=20 > On Dec 15, 2009, at 9:22 PM, Greg Ercolano wrote: >=20 > [posted to rush.general] >=20 > Mr. Daniel Browne wrote: >> I'm writing a basic submit script to perform a unix cp copy =3D >> operation on the farm. >=20 > It sounds like you're submitting a job that copies frames, > but want it to be done with one command, as opposed to one > copy command per frame..? Correct >=20 > Since copying is pure I/O, I would think the best place to > run a large frame copy command is only on the file server, > to avoid network in/network out, so that the copy operation > is entirely local. Unfortunately the BlueArc architecture currently doesn't allow you do to = operations like that within the NAS head. In fact, most of the head's = shell commands do not support recursive operations. >=20 > For this reason I'm not sure I understand the question; > normally one wants to run things through the render queue > to harness the power of parallel cpu use, where each machine > works on a frame or batch of frames. Either that, or you want > to throw a single operation at some available machine, and have > it do the entire operation as a 'single frame rush job'. >=20 We had thought since we perform other operations on frames to and from = volumes on the NAS head that we could do the same with copy operations. = I admit, it sounds a bit zany. At the very least I do have a script that = would allow me to hand off the copy operation to a single farm machine = as opposed to splitting it into pieces. Theoretically this would be = faster as the head is executing these operations in parallel with the = help of its cache memory. >=20 > If you're trying to do shell substitution, I'm not sure, do you > mean wild card expansions, like: >=20 > cp /some/directory/foo.[0-9]*.tif /some/other/directory/ >=20 >> The alternative of course is to do a cp =3D >> operation for each frame, but I was hoping to avoid the additional =3D >> overhead. >=20 > I wouldn't think the cp command would be much overhead, > but I might be missing something. >=20 > If you're trying to distribute the cp commands to a bunch > of machines to parallelize the I/O, I'd think you'd want > to do them either as single frames or in batches. >=20 > Or better yet, batch several commands on the file server > itself, eg: >=20 > cp foo.0*.tif /some/other/dir & # copies 1000 = frames (0000 - 0999) in bg > cp foo.1*.tif /some/other/dir & # copies 1000 = frames (1000 - 1999) in bg > cp foo.2*.tif /some/other/dir & # etc.. >=20 > ..which if the box has multiple procs and the network is idle, > should go really fast. >=20 > If you can follow up with more specifics, I can probably > help you narrow down a specific technique. Yes, this is what I wanted to go for. The problem obviously is that you = can't wildcard a frame range as a whole; you have to wildcard each digit = with its potential range of values. Tedious, but possible. I was just = hoping you might have a library routine already in place to do this. I think I know what to do now; thanks Greg. Happy holidays. -Dan >=20 > --=20 > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) >=20 >=20 > ---------- > Dan "Doc" Browne > System Administrator >=20 > Evil Eye Pictures > d.list at evileyepictures.com > Office: (415) 777-0666 >=20 *** ***************************************************** *** Message #1918 ***************************************************** *** ***************************************************** Subject: Re: Batch Copy Script From: Greg Ercolano Date: Wed, 16 Dec 2009 00:22:30 -0500 Mr. Daniel Browne wrote: > I'm writing a basic submit script to perform a unix cp copy = > operation on the farm. It sounds like you're submitting a job that copies frames, but want it to be done with one command, as opposed to one copy command per frame..? Since copying is pure I/O, I would think the best place to run a large frame copy command is only on the file server, to avoid network in/network out, so that the copy operation is entirely local. For this reason I'm not sure I understand the question; normally one wants to run things through the render queue to harness the power of parallel cpu use, where each machine works on a frame or batch of frames. Either that, or you want to throw a single operation at some available machine, and have it do the entire operation as a 'single frame rush job'. > Is there a routine already available in Rush to = > perform shell substitution of the digits in a frame range, or would I = > have to do that myself? Hmm, I might need a few more details. If you're making a submit script to do a copy, the scripting language you're using should work well for doing shell substitution. Do not try to submit to rush a simple 'cp' command, but rather, have the script submit itself, so that the script runs the cp command on each machine, so that you'll have all the facilities of the scripting language you're working with at your finger tips. (ie. shell wildcard expansion, sed/perl/awk access, etc) For some simple examples of writing scripts that "submit themselves", see: http://www.seriss.com/rush-current/rush/rush-submit.html For digits in a frame range, there are two variables rush supplies at execution time on all the machines; RUSH_FRAME and RUSH_PADFRAME, the latter being 0000 format, and the former having no padding at all. If you need some different padding, you can use printf() to change RUSH_FRAME into whatever padding you want, eg: perl -e 'printf("%06d", $ENV{RUSH_FRAME});' If you're trying to do shell substitution, I'm not sure, do you mean wild card expansions, like: cp /some/directory/foo.[0-9]*.tif /some/other/directory/ ..or do you really mean substitution, like sed/perl regex to turn e.g. foo.0001.tif into foo.0002.tif? > The alternative of course is to do a cp = > operation for each frame, but I was hoping to avoid the additional = > overhead. I wouldn't think the cp command would be much overhead, but I might be missing something. If you're trying to distribute the cp commands to a bunch of machines to parallelize the I/O, I'd think you'd want to do them either as single frames or in batches. Or better yet, batch several commands on the file server itself, eg: cp foo.0*.tif /some/other/dir & # copies 1000 frames (0000 - 0999) in bg cp foo.1*.tif /some/other/dir & # copies 1000 frames (1000 - 1999) in bg cp foo.2*.tif /some/other/dir & # etc.. ..which if the box has multiple procs and the network is idle, should go really fast. If you can follow up with more specifics, I can probably help you narrow down a specific technique. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1917 ***************************************************** *** ***************************************************** Subject: Batch Copy Script From: "Mr. Daniel Browne" Date: Tue, 15 Dec 2009 21:07:31 -0500 Hi Greg, I'm writing a basic submit script to perform a unix cp copy = operation on the farm. Is there a routine already available in Rush to = perform shell substitution of the digits in a frame range, or would I = have to do that myself? The alternative of course is to do a cp = operation for each frame, but I was hoping to avoid the additional = overhead. Thanks, -Dan ---------- Dan "Doc" Browne System Administrator Evil Eye Pictures d.list at evileyepictures.com Office: (415) 777-0666 *** ***************************************************** *** Message #1916 ***************************************************** *** ***************************************************** Subject: Seriss Corp. offices moved From: Greg Ercolano Date: Tue, 15 Dec 2009 13:41:44 -0500 Hi All, Just FYI, our offices have moved to a new location; the new address and phone info (as an image to avoid ascii harvesting): http://seriss.com/images/seriss-corp-address.png Old numbers automatically forward to the new office numbers. *** ***************************************************** *** Message #1915 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Fri, 11 Dec 2009 23:57:14 -0500 Looks like Daniel has confirmed the fix, so I bumped the OSX version number to 102.42a9d (emphasis on "d") which has that fix, and modified the 102.42a9c release page so that downloading the mac version gets you the 'd' release binary with the fix. So if you want the 'd' fix release, go to the public upgrade page and download the Mac version. (ie. go to http://seriss.com/rush/ and click on the red upgrade monkeys) Greg Ercolano wrote: >> Greg Ercolano wrote: >>> Mr. Daniel Browne wrote: >>>> The machine stops responding after the last line, which corresponds to = >>>> the knew hosts file being acknowledged: >> OK, after a phone conversation, it seems on Snow Leopard machines >> when the rush/etc/hosts file reloads, the UDP listener is getting closed, >> and replaced with a unix domain socket. >> >> Since rush doesn't use unix domain sockets, this seems a bit odd. >> >> Also, I can't seem to replicate this on other platforms, only snow leopard. > > > Indeed, the same code running on Tiger and Leopard is fine, > so this seems to be operating system specific only to Snow Leopard. > > I think I've found the solution now -- running some tests. > > Dan, I'll have a new binary for you Tuesday to try. > > Seems to have to do with Snow Leopard's changes to hostname resolution > that use using unix domain sockets. *** ***************************************************** *** Message #1914 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Tue, 08 Dec 2009 01:39:08 -0500 > Greg Ercolano wrote: >> Mr. Daniel Browne wrote: >>> The machine stops responding after the last line, which corresponds to = >>> the knew hosts file being acknowledged: > > OK, after a phone conversation, it seems on Snow Leopard machines > when the rush/etc/hosts file reloads, the UDP listener is getting closed, > and replaced with a unix domain socket. > > Since rush doesn't use unix domain sockets, this seems a bit odd. > > Also, I can't seem to replicate this on other platforms, only snow leopard. Indeed, the same code running on Tiger and Leopard is fine, so this seems to be operating system specific only to Snow Leopard. I think I've found the solution now -- running some tests. Dan, I'll have a new binary for you Tuesday to try. Seems to have to do with Snow Leopard's changes to hostname resolution that use using unix domain sockets. *** ***************************************************** *** Message #1913 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Mon, 07 Dec 2009 19:31:21 -0500 Greg Ercolano wrote: > Mr. Daniel Browne wrote: >> The machine stops responding after the last line, which corresponds to = >> the knew hosts file being acknowledged: >> >>> 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts > > Hmm, we'll have to work offline on this; I can't seem to replicate here. > Will contact each other by phone. OK, after a phone conversation, it seems on Snow Leopard machines when the rush/etc/hosts file reloads, the UDP listener is getting closed, and replaced with a unix domain socket. Since rush doesn't use unix domain sockets, this seems a bit odd. Also, I can't seem to replicate this on other platforms, only snow leopard. Whatever the case, this is causing the daemon to become unresponsive to UDP transactions, but still responds to TCP (eg. 'rush -ping' still works). This can be seen pretty clearly with 'lsof' before and after: BEFORE: $ lsof | grep rushd rushd 1447 root cwd DIR 14,5 374 600231 /usr/local/rush/var rushd 1447 root txt REG 14,5 3086040 599807 /usr/local/rush/bin/rushd rushd 1447 root txt REG 14,5 1054960 24705 /usr/lib/dyld rushd 1447 root txt REG 14,5 199118848 526079 /private/var/db/dyld/dyld_shared_cache_i386 rushd 1447 root 0r CHR 3,2 0t0 303 /dev/null rushd 1447 root 1w REG 14,5 3169 600232 /usr/local/rush/var/rushd.log rushd 1447 root 2w REG 14,5 3169 600232 /usr/local/rush/var/rushd.log rushd 1447 root 3w REG 14,5 5 600323 /usr/local/rush/var/.rushd.LCK rushd 1447 root 4u IPv4 0x040f1ef8 0t0 TCP *:rushd (LISTEN) rushd 1447 root 5u IPv4 0x047cc9c8 0t0 UDP snow.erco.x:rushd ^^ ^^^^^^^^^^^^^^^^^^^^^ AFTER: lsof | grep rushd tail 1100 root 3r REG 14,5 3231 600232 /usr/local/rush/var/rushd.log rushd 1447 root cwd DIR 14,5 374 600231 /usr/local/rush/var rushd 1447 root txt REG 14,5 3086040 599807 /usr/local/rush/bin/rushd rushd 1447 root txt REG 14,5 1054960 24705 /usr/lib/dyld rushd 1447 root txt REG 14,5 199118848 526079 /private/var/db/dyld/dyld_shared_cache_i386 rushd 1447 root 0r CHR 3,2 0t0 303 /dev/null rushd 1447 root 1w REG 14,5 3231 600232 /usr/local/rush/var/rushd.log rushd 1447 root 2w REG 14,5 3231 600232 /usr/local/rush/var/rushd.log rushd 1447 root 3w REG 14,5 5 600323 /usr/local/rush/var/.rushd.LCK rushd 1447 root 4u IPv4 0x040f1ef8 0t0 TCP *:rushd (LISTEN) rushd 1447 root 5u unix 0x037c8910 0t0 ->0x03fbf1c0 ^^ ^^^^ ^^^^^^^^^^^^ Unix socket What the heck? I'll follow up when I've done more tracing to see what's causing that. *** ***************************************************** *** Message #1912 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Mon, 07 Dec 2009 18:45:25 -0500 Mr. Daniel Browne wrote: > The machine stops responding after the last line, which corresponds to = > the knew hosts file being acknowledged: > >> 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts Hmm, we'll have to work offline on this; I can't seem to replicate here. Will contact each other by phone. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1911 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: "Mr. Daniel Browne" Date: Mon, 07 Dec 2009 17:32:23 -0500 The machine stops responding after the last line, which corresponds to = the knew hosts file being acknowledged: > 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts On Dec 7, 2009, at 2:30 PM, Greg Ercolano wrote: [posted to rush.general] Hmm, at what point in the rushd.log did the daemon stop? Every START entry seems to have a matching EXIT entry. Is it possible the problem happened on a different day from today? (eg. if yesterday, paste the Orushd.log (O=3Dold)) It would appear there was a temporary problem with the machine's own = hostname (wayne) not being resolvable during startup, where it says: Local hostname wayne won't resolve (ping: cannot resolve wayne: = Unknown host): 10 sec retries The daemon won't start the rush daemon until 'ping' can resolve the = local machine's own hostname, to make sure hostname resolution is working before rush is = started. In pseudocode, basically: while ( ping `hostname` doesn't work ) { sleep 10 try again } Mr. Daniel Browne wrote: > Here you go (IP addresses removed): >=20 > 12/07,00:00:04 ROTATE rushd.log rotated. pid=3D3D203, 0/1 busy, = ONLINE > 12/07,00:00:04 ROTATE wayne RUSHD 102.42a9c PID=3D3D203 = Boot=3D3D12/04/09,14:07:03 > 12/07,09:47:26 SECURITY Daemon changed to Offline by root@wayne[], = Remark:offline by dbrowne via login script state (offline) by = root@wayne[] > 12/07,12:20:35 SECURITY Daemon changed to Online by root@wayne[], = Remark:online by dbrowne via logout script state (online) by = root@wayne[] > 12/07,12:20:36 SECURITY Daemon changed to Getoff by root@wayne[] = state (getoff) by root@wayne[] > 12/07,12:20:40 INIT.D Shutdown script '/usr/local/rush/etc/S99rush = stop' > 12/07,12:20:40 EXIT Shutdown script '/usr/local/rush/etc/S99rush = stop' > --------- > 12/07,12:21:34 INIT.D Startup script '/usr/local/rush/etc/S99rush = start' > 12/07,12:21:34 INIT.D Executing: ( /usr/local/rush/etc/S99rush = waitlocal && cd /usr/local/rush/var && /usr/libexec/StartupItemContext = /usr/local/rush/bin/rushd ) & > 12/07,12:21:34 INIT.D Local hostname wayne won't resolve (ping: = cannot resolve wayne: Unknown host): 10 sec retries > 12/07,12:21:45 LICENSE validated with server vegas > 12/07,12:21:45 LICENSE expires 03/15/2036 > 12/07,12:21:45 START wayne RUSHD 102.42a9c PID=3D3D189 = Boot=3D3D12/07/09,12:21:45 Online > 12/07,12:21:45 INFO TCP listening on port 696, service 'rushd', = sockfd=3D3D4 > 12/07,12:21:45 INFO UDP listening on port 696, service 'rushd', = sockfd=3D3D5 > 12/07,12:21:45 CHECKPOINT START: Loading = /usr/local/rush/var/jobs-checkpoint > 12/07,12:21:45 CHECKPOINT DONE > 12/07,12:21:51 SECURITY Daemon changed to Offline by root@wayne[], = Remark:offline by dbrowne via login script state (offline) by = root@wayne[172.21.7.9] > 12/07,13:57:54 PUSH /usr/local/rush/etc/hosts pushed from = ?@vegas:52291[] > 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts --=20 Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) ---------- Dan "Doc" Browne System Administrator Evil Eye Pictures d.list at evileyepictures.com Office: (415) 777-0666 *** ***************************************************** *** Message #1910 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Mon, 07 Dec 2009 17:30:20 -0500 Hmm, at what point in the rushd.log did the daemon stop? Every START entry seems to have a matching EXIT entry. Is it possible the problem happened on a different day from today? (eg. if yesterday, paste the Orushd.log (O=old)) It would appear there was a temporary problem with the machine's own hostname (wayne) not being resolvable during startup, where it says: Local hostname wayne won't resolve (ping: cannot resolve wayne: Unknown host): 10 sec retries The daemon won't start the rush daemon until 'ping' can resolve the local machine's own hostname, to make sure hostname resolution is working before rush is started. In pseudocode, basically: while ( ping `hostname` doesn't work ) { sleep 10 try again } Mr. Daniel Browne wrote: > Here you go (IP addresses removed): > > 12/07,00:00:04 ROTATE rushd.log rotated. pid=3D203, 0/1 busy, ONLINE > 12/07,00:00:04 ROTATE wayne RUSHD 102.42a9c PID=3D203 Boot=3D12/04/09,14:07:03 > 12/07,09:47:26 SECURITY Daemon changed to Offline by root@wayne[], Remark:offline by dbrowne via login script state (offline) by root@wayne[] > 12/07,12:20:35 SECURITY Daemon changed to Online by root@wayne[], Remark:online by dbrowne via logout script state (online) by root@wayne[] > 12/07,12:20:36 SECURITY Daemon changed to Getoff by root@wayne[] state (getoff) by root@wayne[] > 12/07,12:20:40 INIT.D Shutdown script '/usr/local/rush/etc/S99rush stop' > 12/07,12:20:40 EXIT Shutdown script '/usr/local/rush/etc/S99rush stop' > --------- > 12/07,12:21:34 INIT.D Startup script '/usr/local/rush/etc/S99rush start' > 12/07,12:21:34 INIT.D Executing: ( /usr/local/rush/etc/S99rush waitlocal && cd /usr/local/rush/var && /usr/libexec/StartupItemContext /usr/local/rush/bin/rushd ) & > 12/07,12:21:34 INIT.D Local hostname wayne won't resolve (ping: cannot resolve wayne: Unknown host): 10 sec retries > 12/07,12:21:45 LICENSE validated with server vegas > 12/07,12:21:45 LICENSE expires 03/15/2036 > 12/07,12:21:45 START wayne RUSHD 102.42a9c PID=3D189 Boot=3D12/07/09,12:21:45 Online > 12/07,12:21:45 INFO TCP listening on port 696, service 'rushd', sockfd=3D4 > 12/07,12:21:45 INFO UDP listening on port 696, service 'rushd', sockfd=3D5 > 12/07,12:21:45 CHECKPOINT START: Loading /usr/local/rush/var/jobs-checkpoint > 12/07,12:21:45 CHECKPOINT DONE > 12/07,12:21:51 SECURITY Daemon changed to Offline by root@wayne[], Remark:offline by dbrowne via login script state (offline) by root@wayne[172.21.7.9] > 12/07,13:57:54 PUSH /usr/local/rush/etc/hosts pushed from ?@vegas:52291[] > 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1909 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: "Mr. Daniel Browne" Date: Mon, 07 Dec 2009 17:01:26 -0500 Here you go (IP addresses removed): 12/07,00:00:04 ROTATE rushd.log rotated. pid=3D203, 0/1 busy, ONLINE 12/07,00:00:04 ROTATE wayne RUSHD 102.42a9c PID=3D203 = Boot=3D12/04/09,14:07:03 12/07,09:47:26 SECURITY Daemon changed to Offline by root@wayne[], = Remark:offline by dbrowne via login script state (offline) by = root@wayne[] 12/07,12:20:35 SECURITY Daemon changed to Online by root@wayne[], = Remark:online by dbrowne via logout script state (online) by = root@wayne[] 12/07,12:20:36 SECURITY Daemon changed to Getoff by root@wayne[] state = (getoff) by root@wayne[] 12/07,12:20:40 INIT.D Shutdown script '/usr/local/rush/etc/S99rush = stop' 12/07,12:20:40 EXIT Shutdown script '/usr/local/rush/etc/S99rush = stop' --------- 12/07,12:21:34 INIT.D Startup script '/usr/local/rush/etc/S99rush = start' 12/07,12:21:34 INIT.D Executing: ( /usr/local/rush/etc/S99rush = waitlocal && cd /usr/local/rush/var && /usr/libexec/StartupItemContext = /usr/local/rush/bin/rushd ) & 12/07,12:21:34 INIT.D Local hostname wayne won't resolve (ping: = cannot resolve wayne: Unknown host): 10 sec retries 12/07,12:21:45 LICENSE validated with server vegas 12/07,12:21:45 LICENSE expires 03/15/2036 12/07,12:21:45 START wayne RUSHD 102.42a9c PID=3D189 = Boot=3D12/07/09,12:21:45 Online 12/07,12:21:45 INFO TCP listening on port 696, service 'rushd', = sockfd=3D4 12/07,12:21:45 INFO UDP listening on port 696, service 'rushd', = sockfd=3D5 12/07,12:21:45 CHECKPOINT START: Loading = /usr/local/rush/var/jobs-checkpoint 12/07,12:21:45 CHECKPOINT DONE 12/07,12:21:51 SECURITY Daemon changed to Offline by root@wayne[], = Remark:offline by dbrowne via login script state (offline) by = root@wayne[172.21.7.9] 12/07,13:57:54 PUSH /usr/local/rush/etc/hosts pushed from = ?@vegas:52291[] 12/07,13:58:45 INFO/HOST Reloading /usr/local/rush/etc/hosts On Dec 7, 2009, at 12:52 PM, Greg Ercolano wrote: [posted to rush.general] Mr. Daniel Browne wrote: > [posted to rush.general] >=20 > Hi Greg, >=20 > The 102.42a9c update seems to have fixed most of our operation =3D= > problems with Rush on SL (we love the new GUI look too, by the way), = but =3D > there is one thing we've noticed; rush on SL machines seems to stop =3D > responding after pushing a hosts file. The failure happens even if the = =3D > file hasn't been changed. This requires you to restart the rush = daemon, =3D > much like how before this release we had to do whenever rush on an SL = =3D > machine stopped responding. Hmm, can you send me the contents of the rushd.log from that = machine where the daemon stopped responding? --=20 Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) ---------- Dan "Doc" Browne System Administrator Evil Eye Pictures d.list at evileyepictures.com Office: (415) 777-0666 *** ***************************************************** *** Message #1908 ***************************************************** *** ***************************************************** Subject: Re: Rush 102.42a9c and Snow Leopard From: Greg Ercolano Date: Mon, 07 Dec 2009 15:52:47 -0500 Mr. Daniel Browne wrote: > [posted to rush.general] > > Hi Greg, > > The 102.42a9c update seems to have fixed most of our operation = > problems with Rush on SL (we love the new GUI look too, by the way), but = > there is one thing we've noticed; rush on SL machines seems to stop = > responding after pushing a hosts file. The failure happens even if the = > file hasn't been changed. This requires you to restart the rush daemon, = > much like how before this release we had to do whenever rush on an SL = > machine stopped responding. Hmm, can you send me the contents of the rushd.log from that machine where the daemon stopped responding? -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1907 ***************************************************** *** ***************************************************** Subject: Rush 102.42a9c and Snow Leopard From: "Mr. Daniel Browne" Date: Mon, 07 Dec 2009 15:03:23 -0500 Hi Greg, The 102.42a9c update seems to have fixed most of our operation = problems with Rush on SL (we love the new GUI look too, by the way), but = there is one thing we've noticed; rush on SL machines seems to stop = responding after pushing a hosts file. The failure happens even if the = file hasn't been changed. This requires you to restart the rush daemon, = much like how before this release we had to do whenever rush on an SL = machine stopped responding. Thanks, -Dan ---------- Dan "Doc" Browne System Administrator Evil Eye Pictures d.list at evileyepictures.com Office: (415) 777-0666 *** ***************************************************** *** Message #1906 ***************************************************** *** ***************************************************** Subject: OSX + After Effects CS4: orphaned processes From: Greg Ercolano Date: Wed, 14 Oct 2009 05:21:38 -0400 This problem is Mac OSX specific, and is happening with CS4. It might be happening with CS3 as well, I haven't checked. It has come to my attention that After Effects CS4's 'aerender' program is now invoking AfterFx in a way where AfterFx no longer remains part of the process hierarchy, making it immune to receiving kill signals from normal process group management. So when you requeue busy frames, or dump a running After Effects job, the After Effects process remains running. In the case of rush, the frames show in the Die state, and don't immediately transition to Que until the AfterFx process finishes rendering the frame it's working on. Other render queues have trouble with this as well. Even scripts that are interrupted with ^C can leave AfterFx running in the background, taking up CPU. Apparently when aerender starts AfterFx, the AfterFx process becomes parented to launchd, and is no longer parented to the aerender process. I've notified Adobe, and they're working on a solution. *** ***************************************************** *** Message #1905 ***************************************************** *** ***************************************************** Subject: Re: Multicore workstations and clever farm use From: Greg Ercolano Date: Thu, 01 Oct 2009 17:15:27 -0400 Greg Ercolano wrote: > [..] you can go into the Task Manager, right click on the > 'rushd.exe' process, and assign it an affinity for cpus #0 and #1. > > Then all renders rush starts will be forced to /only/ use those > two processors; the other two processors will be idle, and > available for interactive desktop use. I played around with this a bit yesterday, and it seems to work quite well for restricting processors, and this is something you all can do with the current release of rush you have. In fact, the above should be able to work for all render managing software, or any processes, including interactive renders. I decided to add an option into the current alpha release of Rush (that will go into beta very soon) where you can include a "processor affinity" setting in the Rush hosts file, eg: #Host Cpus Ram MinPri Criteria/Hostgroups Affinity #------------- ---- ----- ------ ------------------- -------- ta 4 100 0 +any,+erco,+linux,linux,linux6.0,intel,dante,+farm - geneva 1 100 0 +any,+w2k,+work affinity=1 meade 8 100 0 +any,+erco,+linux,linux - superior 2 100 0 +any,+erco,+winxp,winxp affinity=3 [..] ..the new 'Affinity' field at the far right being the new option. Here it shows host 'geneva' that has 2 physical processors, but rush is set up to start only one instance of a render, and will restrict rendering only to cpu#0 (due to affinity=1), leaving cpu#1 available for interactive use. Also, 'superior' which has 4 physical processors, but will only start two instances of renders, and those renders will be restricted to only using physical cpus #0 and #1 (due to affinity=3). This way the new release of rush will be able to let the admin force it to only use certain physical processors. In my tests with the above alpha release, the task manager showed multi-threaded renders started by rushd were indeed restricted precisely to the cpus the 'affinity=' setting defined. However, practically, I'm not sure how useful this is. Despite the restriction leaving cpus completely idle, while renders were running, I still could "feel" slowness from the machine during interactive use, even though the 'render' was only doing a sin() computation in a tight loop, not using any I/O, ram, or networking. So even when you get the control you want, it may not get you the results you want. So forcing cpus to be idle won't necessarily get you responsiveness. It probably depends on the mother board I/O config and physical cpus (cores vs cpus) configurations. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1904 ***************************************************** *** ***************************************************** Subject: Re: unable to grab license over site-site vpn From: Greg Ercolano Date: Wed, 30 Sep 2009 19:58:40 -0400 Followup: We hooked up offline; think we found the problem was a hostname lookup issue. Apparently a notebook was used in the test which lived at site #1 and had a site#1 hostname/IP. When moved to site #2, the site#2 IP was assigned, but the hostname stayed the same. This caused the LA office license server to fail the license checkout because the hostname in the license checkout no longer matched the IP address. Michael's going to try a follow up making sure the hostname/IP match up at site #2. *** ***************************************************** *** Message #1903 ***************************************************** *** ***************************************************** Subject: Re: unable to grab license over site-site vpn From: Greg Ercolano Date: Wed, 30 Sep 2009 16:54:23 -0400 Hi Michael, Just tried calling you in NY, but I guess you already left. Was hoping I could help you out by phone.. were you able to get it to work, or would you like me to talk to one of the guys still at the NYC facility? BTW, I know the names of good freelance guys in NYC who are familiar with Rush and film/video systems administration. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1902 ***************************************************** *** ***************************************************** Subject: Re: unable to grab license over site-site vpn From: Greg Ercolano Date: Wed, 30 Sep 2009 14:36:18 -0400 Michael Oliver wrote: > I have two networks connected via a site to site vpn. The two sites are > on different private subnets but can both see each other. Other > services work perfectly (ie. DNS (forward and reverse), http, nfs, ldap, > etc...) However although computers on Site #2 can rush -ping computers > on Site#1, computers on site #2 cannot retrieve a license. Here is the > error I'm getting: > > 09/30,13:02:00 LICENSE Read from ns2[10.1.1.1] failed: read error: > BinaryRead(8192): recv() 0 bytes from ns2[10.1.1.1] > > 09/30,13:02:00 LICENSE no servers could validate license (30 sec retries) Is there a NAT'ing firewall in the mix? If so, that could be the problem; Rush looks at the IP packets to determine the return address in order to make replies, so if there's a NAT'ing firewall rewriting the return address to be the firewall's own, that could be the issue. Is 10.1.1.1 the correct IP address for ns2? Also, what are the hostnames and IP addresses of the machines at either end (license server and client), and can you show the 'rush -lah' report from both of these machines. If you prefer, you may want to reply to me via private email with this info if you don't want it to be public. Definitely you should be able to make this work, as there are several customers doing this (VPN WAN setups sharing licenses). -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1901 ***************************************************** *** ***************************************************** Subject: unable to grab license over site-site vpn From: Michael Oliver Date: Wed, 30 Sep 2009 13:03:50 -0400 I have two networks connected via a site to site vpn. The two sites are on different private subnets but can both see each other. Other services work perfectly (ie. DNS (forward and reverse), http, nfs, ldap, etc...) However although computers on Site #2 can rush -ping computers on Site#1, computers on site #2 cannot retrieve a license. Here is the error I'm getting: 09/30,13:02:00 LICENSE Read from ns2[10.1.1.1] failed: read error: BinaryRead(8192): recv() 0 bytes from ns2[10.1.1.1] 09/30,13:02:00 LICENSE no servers could validate license (30 sec retries) Any ideas? *** ***************************************************** *** Message #1900 ***************************************************** *** ***************************************************** Subject: Re: Multicore workstations and clever farm use From: Greg Ercolano Date: Wed, 30 Sep 2009 10:23:25 -0400 BTW, I added some screenshots below that show how assigning affinity really does force renders to use specific processors. See "SCREENSHOTS" below. > PROCESSOR AFFINITY > ------------------ > [..] > > Under Windows there's "Processor Affinity" which lets one ensure > a program only uses eg. 1 processor. You can also force a program/render to use only 2 processors, or even 4. And, you can specify precisely /which/ processors to use. "Processor Affinity" is an attribute of the process, and is also passed down to child processes. So an affinity assigned to a process, it will affect all child processes it creates. This means if you tell rushd to have an affinity, then all renders rushd starts will also have that same affinity. So say you have a 4 processor machine, and you want to force rush jobs on that machine to only use physical processors 0 and 1. This would ensure processors 2 and 3 would be completely available for interactive use. To do this, you can go into the Task Manager, right click on the 'rushd.exe' process, and assign it an affinity for cpus #0 and #1. Then all renders rush starts will be forced to /only/ use those two processors; the other two processors will be idle, and available for interactive desktop use. CAVEAT: Keep in mind that if the render makes heavy use of the hard disk, ram, or network, your "interactive use" may suffer at the limitations of those resources. So in other words, you may still 'feel' slowness if your interactive use involves any of those resources. Remember that a program's responsiveness is not /only/ due to cpu availability. I/O bandwidth is also at play. > One way to do this is with > the Windows 'start' command, eg. "start /AFFINITY ", > where indicates which processors to lock a command to.. The DOS 'START' command is nice if you want the render script to control processor use. Note that the START command's 'AFFINITY' flag is only available on machines with multiple processors. SCREENSHOTS ----------- Here are some screenshots that show how affinity really forces a process to use the processors you specify, no matter how many threads that process starts. In this case I'm running a program called 'createthread' which expects two parameters; a number of threads to start, and a number of seconds to run. In the following three screenshots, I'm running 'createthread 2 60' which starts two threads running for 60 seconds. In these 3 examples, I'm changing /only/ the affinity. Each screenshot includes the Task Manager's cpu graph, so you can see how the program makes use of the processors. Here are the three examples: 1) No affinity (use any cpus): http://seriss.com/rush/misc-docs/affinity-windows/affinity-none-threads-2.png 2) Affinity of 1 (use cpu#1): http://seriss.com/rush/misc-docs/affinity-windows/affinity-1-threads-2.png 3) Affinity of 2 (use cpu#2) http://seriss.com/rush/misc-docs/affinity-windows/affinity-2-threads-2.png Note that for "no affinity" both cpus are pegged 100%, total cpu usage is 100%. For "affinity 1", only cpu #1 is pegged, total cpu usage is only 50%. For "affinity 2", only cpu #2 is pegged, total cpu usage is only 50%. This makes it clear we can control precisely which processors get assigned to the process, and that regardless of the number of threads the process starts, only the processors with and affinity assigned are actually used. > for instance: > > START /AFFINITY 3 shake -exec /path/to/your.shk -t 1-69 > > ..which will 'lock' the shake process to cpus #0 and #1. And I should add to prevent START from opening another window, you can use: START /AFFINITY 3 /B /WAIT shake -exec /path/to/your.shk -t 1-69 -------- CAVEATS: one general problem with 'START'; it has a bug where it does not pass the exit code of the process back to the shell. The exit code returned is always 0, even if the process (in this case, shake) returns a non-zero exit code. So if you tried to use this in a script, it would not be able to tell if shake failed or succeeded. Kinda annoying. > The /AFFINITY value is a "bit field" which breaks down this way: > > Affinity > Value Cpu# Binary Value > -------- ---- ------------ > 1 0 0001 > 2 1 0010 > 3 0,1 0011 > 4 2 0100 > 5 0,2 0101 > 6 1,2 0110 > 7 0,1,2 0111 > : > 16 3 1000 > 17 3,0 1001 > : > etc. Correction -- a few mistakes in the above table the entries for 16 and 17 are wrong in two ways: First, my binary digits are wrong, and second, apparently the /AFFINITY parameter's value is *hex*. So a corrected table would be: Affinity Value Cpu# Binary Value -------- ---- ------------ 1 0 00001 2 1 00010 3 0,1 00011 4 2 00100 5 0,2 00101 6 1,2 00110 7 0,1,2 00111 8 3 01000 : : : f 0,1,2,3 01111 10 4 10000 11 4,0 10001 : etc. So if you have an 8 processor machine, and want a render to only use the first 4 processors (0,1,2,3), then use an affinity of 'f'. (Hex counts 0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f,10,11,12..) *** ***************************************************** *** Message #1899 ***************************************************** *** ***************************************************** Subject: Re: Multicore workstations and clever farm use From: Greg Ercolano Date: Tue, 29 Sep 2009 22:21:07 -0400 Abraham Schneider wrote: > Sometimes it would be really nice to use 4 of the cores to work in the > GUI and the other 4 cores to render in with rush. 'rush -reserve' is made for this purpose: http://www.seriss.com/rush-current/rush/rush-command-line-options.html#-reserve USING RESERVE JOBS ------------------ The CPUS number in the rush/etc/hosts file is really to be thought of as the number of instances of the renderer you want to run, so if you have '4' in the rush/etc/hosts file, and want to reserve 2 for GUI use, you could say: rush -reserve localhost=2@200 ..which says reserve 2 of rush's 'cpus' at 200 priority on the local machine. This means no job under 200 priority can use those two procs, even if its a 'killer' job. If you want to make it so no job can run, use 999. 'rush -reserve' creates a 'dummy sleep job' that holds the processors to prevent other renders from using them. Just dump the job to get rid of the reservation. PROCESSOR AFFINITY ------------------ However, if you've got multithreaded jobs and are specifying fixed numbers on the command line (eg. "shake -cpus 4") then threads might still leak into using processors you want "reserved". In such cases it can get tricky if you're trying to split hairs. The desire here is to really lock physical processors to a particular user. eg. if 'fred' is logged in, he wants to make sure that of the 8 processors on his machine, 4 can ONLY be used by him, so that even if a multithreaded shake job jumps on the machine, there'd be no chance that his 4 procs get used by the renders, no matter how many threads are running. Under Windows there's "Processor Affinity" which lets one ensure a program only uses eg. 1 processor. One way to do this is with the Windows 'start' command, eg. "start /AFFINITY ", where indicates which processors to lock a command to, for instance: START /AFFINITY 3 shake -exec /path/to/your.shk -t 1-69 ..which will 'lock' the shake process to cpus #0 and #1. The /AFFINITY value is a "bit field" which breaks down this way: Affinity Value Cpu# Binary Value -------- ---- ------------ 1 0 0001 2 1 0010 3 0,1 0011 4 2 0100 5 0,2 0101 6 1,2 0110 7 0,1,2 0111 : 16 3 1000 17 3,0 1001 : etc. There's a chart here: http://technet.microsoft.com/en-us/library/cc778499%28WS.10%29.aspx There are some other techniques you can do too. To simplify things, usually all people want to do is use their own machine for rendering, and not allow others to use it. WORKSTATION SPECIFIC RENDERING ------------------------------ To do this, a common technique is to use 'rush -reserve' to reserve the entire machine at some high priority, say @899. Then when you want to render on your machine, submit a job that asks for your machine (eg. 'tahoe') at a priority of 900k, eg: Cpus: tahoe=2@900k ..and setting e.g. the -cpus for shake to the number of threads you want, so that you don't use the entire machine. This will let your render bump the reservation job so that it can run, and when the render's done, the reserve job will maintain a hold on the processor again. If you're trying to set things up so that jobs can run on the farm AND on your workstation, but on the workstation, jobs never use more threads than you want, you'd have to modify the logic of the submit scripts to clamp eg. shake's "-cpus #" so that if the render is running on that machine, the -cpus number gets limited down to the value you want. Also, you can use MINPRI column of the hosts file is something you can use as well to put a 'wall' on the jobs the workstations accept. The 'MINPRI' value sets a minimum priority a job needs to render on that machine. This way the machine's processors will remain idle unless a job asks for this machine with a high enough priority. You can establish policies that allow users to only submit to their own machines with a priority sufficient to render on their own workstation, while keeping other renders away. If you need more specifics on any of these, let me know. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1898 ***************************************************** *** ***************************************************** Subject: Multicore workstations and clever farm use From: "Abraham Schneider" Date: Tue, 29 Sep 2009 10:54:03 -0400 Hi there! With all the 'new' multicore workstations we have now with 4, 8 or 16=20 cores, I'm thinking about using them in the renderfarm as well. But I=20 can't find a good solution to do this at the moment. Problem is: I defined all the machines in the hosts file to be a 'cluster' of 4=20 cores. So a machine with 4 real cores is listed as 1 Cpu in the hosts=20 file, a machine with 8 cores is listed as 2 Cpus. And in Shake I start=20 my jobs with '-cpus 4' to use 4 cores to render on one frame. That way of working is fine when I don't use the workstation at all and=20 want to use it completely in the farm. But for machines with 8 or more=20 cores there would be another nice scenario: Sometimes it would be really nice to use 4 of the cores to work in the=20 GUI and the other 4 cores to render in with rush. Is there any clever way to handle this in rush? At the moment the only=20 way I see is to edit the hosts file and change the 'cpus' value. But I=20 have to do this every time an operator wants to change from 'all cores=20 to the farm' to 'only some cores to the farm' and back. Really perfect would be an advanced 'onrush' where I have more than two=20 options: not online, partially online, fully online. Or something = similar. How do you all handle this? Thanks, Abraham -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Dr. Martin Prillmann; Thomas Till *** ***************************************************** *** Message #1897 ***************************************************** *** ***************************************************** Subject: Re: Snow Leopard -- Fixes for Rush 102.42a9b and older From: Greg Ercolano Date: Fri, 25 Sep 2009 05:03:45 -0400 > Once we removed .local from the search suffix all is well again. Sounds like you fixed it. Yes, I recommend avoiding the .local domain name suffix because that enables all that ZeroConf/Bonjours broadcast-based discovery stuff, including multicast DNS, which creates unreliable behavior that messes up network stability. ie: http://en.wikipedia.org/wiki/Zeroconf If you need zeroconf based stuff on your machines also running rush, you may be able to specify it without causing trouble IF you specify your internal domain first, and then .local after it. This way (maybe) regular DNS will be interrogated first, and then falls back to Bonjours only if that fails. Dylan Penhale wrote: > Yes I thought DNS at first but we thought we had it sorted as we where > using the same settings as other non snow leopard machines. I assume > DNS is used to reverse lookup the IP addresses of the hostnames but > that this isn't cached anywhere. Both OSX and Windows traditionally have DNS caching turned on as a default. "lookupd' used to be responsible for hostname caching in OSX, but now I think it's DirectoryServices(8) since lookupd was dropped. According to the docs, dscacheutil can be used to display the DNS cache (supposedly), ie. as root: dscacheutil -cachedump -entries Host However, when I do this on my snow leopard box, I get zippo. Even after I do things that /should/ trigger DNS cache hits, like pinging hosts by name, or running 'rush -lah'. Because of this empty output, I'm not entirely sure it IS caching namelookups by default in Snow Leopard. Just a casual observation though.. I haven't dug into SL deeply yet. > It turns out that snow leopard treats .local in a different way to > previous apple operating systems, as far as I can tell. The bad behavior with .local has always been in there, it just creeps in whenever you start with a fresh machine, and forget to configure all the DNS stuff statically. When Bonjours creeps in, you can get flakey name lookups due to the broadcast based discovery stuff. I refer to this in the prerequisite docs as something to avoid: http://www.seriss.com/rush-current/rush/rush-admin-faq.html#ADMINFAQ-PREREQ (Search for hostname.local in that text) > Usually this > wouldn't be a problem but as some of our domain uses .local this > appears to clash with the apple .local. which I believe is used for > zeroconf type broadcast traffic. Yes, correct. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1896 ***************************************************** *** ***************************************************** Subject: Re: Snow Leopard -- Fixes for Rush 102.42a9b and older From: Dylan Penhale Date: Fri, 25 Sep 2009 03:34:17 -0400 Yes I thought DNS at first but we thought we had it sorted as we where using the same settings as other non snow leopard machines. I assume DNS is used to reverse lookup the IP addresses of the hostnames but that this isn't cached anywhere. It turns out that snow leopard treats .local in a different way to previous apple operating systems, as far as I can tell. Usually this wouldn't be a problem but as some of our domain uses .local this appears to clash with the apple .local. which I believe is used for zeroconf type broadcast traffic. Once we removed .local from the search suffix all is well again. Thanks On 25/09/2009, at 5:10 PM, Greg Ercolano wrote: > [posted to rush.general] > > Dylan Penhale wrote: >> We are experiencing slow performance issuing a rush -lah on snow >> leopard. > >> Will the 102.42a9c maintenance release address this? > > I haven't seen a slowness problem on Snow Leopard > with 'rush -lah'. > > Slowness with 'rush -lah' usually means hostname lookups > aren't working correctly because it does a hostname<->IP > lookup for each hostname in the rush/etc/hosts file, in > order to show the IP addresses for each host. > > A very slow output of 'rush -lah' usually means the primary > DNS server is being unresponsive. > > Check your DNS settings, to make sure the primary DNS server > is responsive, and the search domain is set to your internal > domain. > > Also; are there any hosts coming up with ???'s for IP's? > If so, that's surely the cause, as it means DNS is failing > the lookup for those hosts, and that can slow that report > down quite a bit. > *** ***************************************************** *** Message #1895 ***************************************************** *** ***************************************************** Subject: Re: Snow Leopard -- Fixes for Rush 102.42a9b and older From: Greg Ercolano Date: Fri, 25 Sep 2009 03:10:55 -0400 Dylan Penhale wrote: > We are experiencing slow performance issuing a rush -lah on snow > leopard. > Will the 102.42a9c maintenance release address this? I haven't seen a slowness problem on Snow Leopard with 'rush -lah'. Slowness with 'rush -lah' usually means hostname lookups aren't working correctly because it does a hostname<->IP lookup for each hostname in the rush/etc/hosts file, in order to show the IP addresses for each host. A very slow output of 'rush -lah' usually means the primary DNS server is being unresponsive. Check your DNS settings, to make sure the primary DNS server is responsive, and the search domain is set to your internal domain. Also; are there any hosts coming up with ???'s for IP's? If so, that's surely the cause, as it means DNS is failing the lookup for those hosts, and that can slow that report down quite a bit. *** ***************************************************** *** Message #1894 ***************************************************** *** ***************************************************** Subject: Re: Snow Leopard -- Fixes for Rush 102.42a9b and older From: Dylan Penhale Date: Fri, 25 Sep 2009 01:19:31 -0400 Hi Greg We are experiencing slow performance issuing a rush -lah on snow leopard. Will the 102.42a9c maintenance release address this? Dylan Penhale Fuel VFX On 10/09/2009, at 8:45 AM, Greg Ercolano wrote: > [posted to rush.general] > > RUSH + SNOW LEOPARD > > There will be a new maintenance release of Rush that addresses > Snow Leopard: Rush 102.42a9c, which will beta in about a week. > (Still doing internal testing) > > It will have other things as well; watch this space for details. > > * * * > > If you have any questions, let me know. *** ***************************************************** *** Message #1892 ***************************************************** *** ***************************************************** Subject: Next release: Rush 102.42a9c beta testing From: Greg Ercolano Date: Mon, 21 Sep 2009 20:25:15 -0400 Hi All, I'm collecting names for those folks wanting to beta test the next release. I'm looking for no more than 5 companies to beta test. We're still doing alpha testing internally, but we're closing in on a beta release. Alpha revealed some things I'm fixing, and unearthed some good feature requests which I want to fit into the release as soon as those all go back through alpha. You'll find irush has several features in this release, including easy ways to view the frame logs in a text editor, popup contextual menus for the Jobs and Frames reports, easy ways to view job start/done/dump command logs interactively, as well as some other features. [..] Please contact me via email directly if you're interested in testing: erco@(email surpressed) *** ***************************************************** *** Message #1891 ***************************************************** *** ***************************************************** Subject: Re: OSX hidden process using 50% sys. From: Dylan Penhale Date: Fri, 18 Sep 2009 02:54:39 -0400 > From: Greg Ercolano > Reply-To: > Date: 17 Sep 2009 07:56:27 -0000 > To: > Subject: Re: OSX hidden process using 50% sys. > > [posted to rush.general] > > Dylan Penhale wrote: >>>> We are unable to read the log file from the problem render node, >>>> but the file is readable from all other hosts. >>> When you try to read the log, what happens, and with what >>> command/technique are you trying to look at the file? >>> (rush -log, more, type, cat, text editor, etc) >> >> Cat, more, tail - all return no output. >> >> As I say, the log file is ok because we can view it from other machines. I >> am curios why this box is even trying to write these files as this node >> isn't even supposed to be rendering this frame. I think this might be the >> clue to the issue. > > This sounds messed up at the file system level; > I'd suspect the network file system as the problem, > probably at the client side. > > Hopefully other folks can chime in here if they've seen > this too. > > What kind of machine is the file server at the other end > of the NFS mount? Possibly there's an NFS incompatibility > there.. only thing I can think of other than the client OS. > > I haven't heard of such a problem in OSX's nfs; in my > experience NFS has been pretty much problem free mounting > the linux NFS server here. > > BTW, you mention you're unable to umount the file server, > is this true even after you kill the Render process, and > lsof verifies there are no other mounts? Or is the Render > process unkillable because it's 'hung accessing the mount', > which would all the more point to an NFS client fault. > > Did you try 'umount -f'? Yes, this doesn't work though, it just hangs the shell. Certainly looks like a kernel error. Once in this state you can't cancel the umount command, you have to create a new ssh session or power cycle the machine. The mount options are "intr,bg,resvport,vers=3" which I would think would be pretty safe. The log server in this case is an OSX server (it's pretty much only used for rush logs). The server doesn't give anything away in it's logs. > >>>> We think that the kernel of this render node is stuck trying to access >>>> this file, which in turn is causing the high cpu sys load. > >>> Yes, most likely, if the cpu is not attributed to a process. > >> The render log looks the same as the others, nothing to suggest an error. >> Log size is 46K. > > Try yanking the network cable to see if it affects the 50% cpu use. > My guess is it won't, which means the kernel is spinning doing > 'something bad'.. hard to say what. > > Is the whole mount in a bad state (ie can you view other files OK?), > or is it just this file and all else works fine? Yes we can view other files, it's just this one file we have problems with. > > All the behavior sounds quite odd from an OS point of view. > I'd say the client OS is at fault here, with only a slim > possibility the NFS server might be triggering it. > > Once rush starts the Render process, its up to the OS and Render > to do its thing; rush has no part of the rendering process once > it starts until the process exits; then rush steps in and returns > the exit code. > > I think if you want to move forward, get out the nfs debugging > tools; tcpdump, possibly even ktrace (or whatever it's called > on Leopard). If you suspect maybe the server is at fault, > contact the vendor if you're on support. > >> RUSHD 102.42a9 > > That sounds fine. > > That release is a little over a year old (June 08), > and there's only been 2 minor maintenance releases since then.. > but nothing you'd need. > > -- > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) *** ***************************************************** *** Message #1890 ***************************************************** *** ***************************************************** Subject: Re: OSX hidden process using 50% sys. From: Greg Ercolano Date: Thu, 17 Sep 2009 03:56:27 -0400 Dylan Penhale wrote: >>> We are unable to read the log file from the problem render node, >>> but the file is readable from all other hosts. >> When you try to read the log, what happens, and with what >> command/technique are you trying to look at the file? >> (rush -log, more, type, cat, text editor, etc) > > Cat, more, tail - all return no output. > > As I say, the log file is ok because we can view it from other machines. I > am curios why this box is even trying to write these files as this node > isn't even supposed to be rendering this frame. I think this might be the > clue to the issue. This sounds messed up at the file system level; I'd suspect the network file system as the problem, probably at the client side. Hopefully other folks can chime in here if they've seen this too. What kind of machine is the file server at the other end of the NFS mount? Possibly there's an NFS incompatibility there.. only thing I can think of other than the client OS. I haven't heard of such a problem in OSX's nfs; in my experience NFS has been pretty much problem free mounting the linux NFS server here. BTW, you mention you're unable to umount the file server, is this true even after you kill the Render process, and lsof verifies there are no other mounts? Or is the Render process unkillable because it's 'hung accessing the mount', which would all the more point to an NFS client fault. Did you try 'umount -f'? >>> We think that the kernel of this render node is stuck trying to access >>> this file, which in turn is causing the high cpu sys load. >> Yes, most likely, if the cpu is not attributed to a process. > The render log looks the same as the others, nothing to suggest an error. > Log size is 46K. Try yanking the network cable to see if it affects the 50% cpu use. My guess is it won't, which means the kernel is spinning doing 'something bad'.. hard to say what. Is the whole mount in a bad state (ie can you view other files OK?), or is it just this file and all else works fine? All the behavior sounds quite odd from an OS point of view. I'd say the client OS is at fault here, with only a slim possibility the NFS server might be triggering it. Once rush starts the Render process, its up to the OS and Render to do its thing; rush has no part of the rendering process once it starts until the process exits; then rush steps in and returns the exit code. I think if you want to move forward, get out the nfs debugging tools; tcpdump, possibly even ktrace (or whatever it's called on Leopard). If you suspect maybe the server is at fault, contact the vendor if you're on support. > RUSHD 102.42a9 That sounds fine. That release is a little over a year old (June 08), and there's only been 2 minor maintenance releases since then.. but nothing you'd need. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1889 ***************************************************** *** ***************************************************** Subject: Re: OSX hidden process using 50% sys. From: Dylan Penhale Date: Thu, 17 Sep 2009 02:10:51 -0400 > From: Greg Ercolano > Reply-To: > Date: 17 Sep 2009 04:31:30 -0000 > To: > Subject: Re: OSX hidden process using 50% sys. > > [posted to rush.general] > > Dylan Penhale wrote: >> We are submitting a Maya mentalray render from a PC to a bunch of PC=92= > s >> and Macs. The job server is running XP. On one Mac render node, >> sometimes more than one, we seemingly randomly see the CPU pegged at 50= > % >> system. Top does not show what process is using that 50%, it appears to >> be hidden. > > The kernel can take CPU of its own to process IO, and that > won't show up associated with any process. For instance, in > rushtop this would show up in red. Yes, rushtop shows lots of red :) > >> A little digging reveals that there are two processes that >> appear to be =93holding=94 log files open on the rush log server. > >> Interestingly these commands are both trying to access the same log fil= > e >> and they are NOT the log file that the machine has rendered, nor been >> assigned to render (from what we can tell). > >> We are unable to read the log file from the problem render node, but th= > e >> file is readable from all other hosts. > > When you try to read the log, what happens, and with what > command/technique are you trying to look at the file? > (rush -log, more, type, cat, text editor, etc) Cat, more, tail - all return no output. As I say, the log file is ok because we can view it from other machines. I am curios why this box is even trying to write these files as this node isn't even supposed to be rendering this frame. I think this might be the clue to the issue. > >> We think that the kernel of this render node is stuck trying to access >> this file, which in turn is causing the high cpu sys load. > > Yes, most likely, if the cpu is not attributed to a process. > >> When we lsof we can see that the command that is accessing the log file >> is Render. >> =20 >> Render 19841 netrender 1w REG 26,10 47549 >> 37261653 >> /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_mat= > teSealA.log/0008 >> Render 19841 netrender 2w REG 26,10 47549 >> 37261653 >> /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_mat= > teSealA.log/0008 > > The process ids are the same (19841), so it's one process. > > There are two entries in lsof, one for the stdout file descriptor (1w) > and one for the stderr file descriptor (2w) which are both 'write only' = > (w). > > This is normal behavior during rendering, because the processes > are generating output that is being written to the file server. > > You might check to see what's going on in the log, as maybe it's generat= > ing > a large amount of output due to a high verbosity setting. > The render log looks the same as the others, nothing to suggest an error. Log size is 46K. > Often you can mitigate the cpu/network overhead of render's verbose outp= > ut > by setting the 'maxlogsize' to some very high number (like 10000000, ie.= > 10M) > so that the render output is piped through 'logtrim', which will line bu= > ffer > the render's output so that verbose output is buffered. > > BTW, what version of rush are you running? > I don't think it matters in this case, as the io has to do with the rend= > er > and likely its verbosity setting. But it may help to know on this end. RUSHD 102.42a9 > > > --=20 > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) *** ***************************************************** *** Message #1888 ***************************************************** *** ***************************************************** Subject: Re: OSX hidden process using 50% sys. From: Greg Ercolano Date: Thu, 17 Sep 2009 00:31:30 -0400 Dylan Penhale wrote: > We are submitting a Maya mentalray render from a PC to a bunch of PC=92= s > and Macs. The job server is running XP. On one Mac render node, > sometimes more than one, we seemingly randomly see the CPU pegged at 50= % > system. Top does not show what process is using that 50%, it appears to > be hidden. The kernel can take CPU of its own to process IO, and that won't show up associated with any process. For instance, in rushtop this would show up in red. > A little digging reveals that there are two processes that > appear to be =93holding=94 log files open on the rush log server. > Interestingly these commands are both trying to access the same log fil= e > and they are NOT the log file that the machine has rendered, nor been > assigned to render (from what we can tell). > We are unable to read the log file from the problem render node, but th= e > file is readable from all other hosts. When you try to read the log, what happens, and with what command/technique are you trying to look at the file? (rush -log, more, type, cat, text editor, etc) > We think that the kernel of this render node is stuck trying to access > this file, which in turn is causing the high cpu sys load. Yes, most likely, if the cpu is not attributed to a process. > When we lsof we can see that the command that is accessing the log file > is Render. >=20 > Render 19841 netrender 1w REG 26,10 47549 > 37261653 > /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_mat= teSealA.log/0008 > Render 19841 netrender 2w REG 26,10 47549 > 37261653 > /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_mat= teSealA.log/0008 The process ids are the same (19841), so it's one process. There are two entries in lsof, one for the stdout file descriptor (1w) and one for the stderr file descriptor (2w) which are both 'write only' = (w). This is normal behavior during rendering, because the processes are generating output that is being written to the file server. You might check to see what's going on in the log, as maybe it's generat= ing a large amount of output due to a high verbosity setting. Often you can mitigate the cpu/network overhead of render's verbose outp= ut by setting the 'maxlogsize' to some very high number (like 10000000, ie.= 10M) so that the render output is piped through 'logtrim', which will line bu= ffer the render's output so that verbose output is buffered. BTW, what version of rush are you running? I don't think it matters in this case, as the io has to do with the rend= er and likely its verbosity setting. But it may help to know on this end. --=20 Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1887 ***************************************************** *** ***************************************************** Subject: OSX hidden process using 50% sys. From: Dylan Penhale Date: Wed, 16 Sep 2009 23:04:54 -0400 --_000_C6D7E47110986dylanfuelvfxcom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi We have a tricky little problem here, I wonder if you have seen anything li= ke it. We are submitting a Maya mentalray render from a PC to a bunch of PC's and = Macs. The job server is running XP. On one Mac render node, sometimes more = than one, we seemingly randomly see the CPU pegged at 50% system. Top does = not show what process is using that 50%, it appears to be hidden. A little = digging reveals that there are two processes that appear to be "holding" lo= g files open on the rush log server. Interestingly these commands are both = trying to access the same log file and they are NOT the log file that the m= achine has rendered, nor been assigned to render (from what we can tell). We are unable to read the log file from the problem render node, but the fi= le is readable from all other hosts. We think that the kernel of this render node is stuck trying to access this= file, which in turn is causing the high cpu sys load. When we lsof we can see that the command that is accessing the log file is = Render. Render 19841 netrender 1w REG 26,10 47549 37261653 = /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_matteSe= alA.log/0008 Render 19841 netrender 2w REG 26,10 47549 37261653 = /Volumes/atlantic/rushlogs/3d/tjn_se_0210_c004rs_a033as_l014hh_seal_matteSe= alA.log/0008 Note: This not did not try to render frame 0008 from what we can tell. Should Render be writing the log file directly, or should the STDERR from R= ender be passed to Rush to do the writing? The PID's listed do not show up in the process list (top or ps) so we assum= e the kernel is experiencing some sort of bug. We also thought that perhaps this render node was assigned to render this f= rame, and at the last minute informed that another node was already doing i= t so told to abort it's render. It's hard to tell from the cpu.acct file wh= at is happening as I think this only gets updated at the end of each succes= sful frame. We are not able to umount the NFS share that the logs are written to, which= again suggests the logs are somehow the cause. In fact the only way to get= the CPU to behave normally is to restart the box, which can only be done b= y physically holding the power button in. In this case the render node is running OSX 10.5.4, and is an Intel Core 2 = Duo. Any ideas? DYLAN PENHALE IT MANAGER FUEL 65 KING STREET NEWTOWN SYDNEY NSW 2042 AUSTRALIA T. 61 2 9557 7799 F. 61 2 9557 7882 M. 0424 655 320 WWW.FUELVFX.COM --_000_C6D7E47110986dylanfuelvfxcom_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable OSX hidden process using 50% sys. Hi

We have a tricky little problem here, I wonder if you have seen anything li= ke it.  

We are submitting a Maya mentalray render from a PC to a bunch of PC’= s and Macs. The job server is running XP. On one Mac render node, sometimes= more than one, we seemingly randomly see the CPU pegged at 50% system. Top= does not show what process is using that 50%, it appears to be hidden. A l= ittle digging reveals that there are two processes that appear to be “= ;holding” log files open on the rush log server. Interestingly these = commands are both trying to access the same log file and they are NOT the l= og file that the machine has rendered, nor been assigned to render (from wh= at we can tell).

We are unable to read the log file from the problem render node, but the fi= le is readable from all other hosts.
We think that the kernel of this render node is stuck trying to access this= file, which in turn is causing the high cpu sys load.

When we lsof we can see that the command that is accessing the log file is = Render.

Render    19841      netrender &nbs= p;  1w      REG     = 26,10     47549 37261653 /Volumes/atlantic/rushlogs/3d/= tjn_se_0210_c004rs_a033as_l014hh_seal_matteSealA.log/0008
Render    19841      netrender &nbs= p;  2w      REG     = 26,10     47549 37261653 /Volumes/atlantic/rushlogs/3d/= tjn_se_0210_c004rs_a033as_l014hh_seal_matteSealA.log/0008

Note: This not did not try to render frame 0008 from what we can tell.

Should Render be writing the log file directly, or should the STDERR from R= ender be passed to Rush to do the writing?

The PID’s listed do not show up in the process list (top or ps) so we= assume the kernel is experiencing some sort of bug.

We also thought that perhaps this render node was assigned to render this f= rame, and at the last minute informed that another node was already doing i= t so told to abort it’s render. It’s hard to tell from the cpu.= acct file what is happening as I think this only gets updated at the end of= each successful frame.

We are not able to umount the NFS share that the logs are written to, which= again suggests the logs are somehow the cause. In fact the only way to get= the CPU to behave normally is to restart the box, which can only be done b= y physically holding the power button in.

In this case the render node is running OSX 10.5.4, and is an Intel Core 2 = Duo.

Any ideas?

DYLAN PENHALE
IT MANAGER

FUEL
65 KING STREET
NEWTOWN  SYDNEY
NSW  2042  AUSTRALIA

T. 61 2 9557 7799
F. 61 2 9557 = 7882
M. 0424 655 3= 20

WWW.FUELVFX.COM
 --_000_C6D7E47110986dylanfuelvfxcom_-- *** ***************************************************** *** Message #1886 ***************************************************** *** ***************************************************** Subject: Re: NFS Permissions & Snow Leopard From: Greg Ercolano Date: Thu, 10 Sep 2009 15:21:55 -0400 Greg Ercolano wrote: > Craig Allison wrote: >> Just to let everyone know I've just done an install of Rush version >> 102.42a9 on Snow Leopard and all works well as long as you install Rosetta. > > Actually, no, you /shouldn't/ need Rosetta > if you comment out this section of the install script: > [snip] > > The rush binaries will all run fine (including the GUIs) > without the above 'Rez' code. This is only needed for > older versions of OSX that depended on resource/data forks. Actually, I stand corrected on that. During testing last week, I noticed that all the Mac Universal releases of Rush to date (102.42a9b and older) had one binary that was *still* PPC that I didn't know about; the little executable that is in all the submit-*.app dirs. To address that, I've posted a patch in this newsgroup article: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1885 So that will fix the problem with the existing versions of Rush for OSX. There will also be a maintenance release of Rush (102.42a9c) that will beta next week that has a fix for that, as well as fixed for other obscure issues that came up over the last month or two. *** ***************************************************** *** Message #1885 ***************************************************** *** ***************************************************** Subject: Snow Leopard -- Fixes for Rush 102.42a9b and older From: Greg Ercolano Date: Wed, 09 Sep 2009 18:45:07 -0400 RUSH + SNOW LEOPARD There will be a new maintenance release of Rush that addresses Snow Leopard: Rush 102.42a9c, which will beta in about a week. (Still doing internal testing) It will have other things as well; watch this space for details. * * * Meanwhile, those of you running Rush 102.42a9b or older, I've put together a 'fix script' for you which fixes your existing Rush OSX installations to run on Snow Leopard: http://www.seriss.com/rush/releases/patches/rush-snow-leopard-fix-scripts--for-102-42a9b-and-older.tar.gz (Use your downloads login/password to install it. If you forgot your downloads password, please contact me.) See the README file for details. It's very simple to use; just run the appropriate script as root, and it will apply the fixes automagically. Can be run on older OSX releases as well, so that you can then just copy the mac installs to snow leopard machines. You do /not/ need to stop the queue to run these scripts; they can be run while renders are running and guis are open. The fix script repairs these things: o The rush/etc/bin/install.sh script o The rush/etc/S99rush boot script (small cosmetic fixes.. nothing critical) o The wrapper binaries in examples/Applications/submit-*.app/Contents/MacOS which handle bringing up the submit scripts from the Finder. This is somewhat important; it turns out these binaries were PPC and I didn't know it. The fix installs universal bins for these, since Snow Leopard no longer runs PPC binaries by default. So if you're running any release of Rush 102.42a8 thru 102.42a9b, this fix script should help you for running Rush on Snow Leopard. These fixes will be in all Rush releases 102.42a9c and up, so if you upgrade to that release, you won't need to run the fix script. If you have any questions, let me know. *** ***************************************************** *** Message #1884 ***************************************************** *** ***************************************************** Subject: Re: [SYSADMIN/OSX] Changing the global umask (eg. for the Finder,etc) From: Greg Ercolano Date: Thu, 03 Sep 2009 14:59:13 -0400 The 'defaults write' method is probably a good one. If you like editing files by hand or by script, here's a macosxhints article that covers a difference in Leopard: http://www.macosxhints.com/article.php?story=20071207091554360 Quoting the relevant bits of the article, in case the link goes stale: "Setting NSUmask in Leopard is done differently than in previous OS X releases (older hints on NSUmask). In 10.5, NSUmask is gone. To set a default umask (for both shell and GUI apps), edit /etc/launchd.conf and add this line: umask 077 ..where 077 is the new default umask. If nothing is there, the default is 022. Note, the /etc/launchd.conf file umask "trick" should work in Tiger too, but I didn't test it." Most of you will probably want 'umask 0' for wide open perms (rw-rw-rw-) or maybe 'umask 2' for wide open perms to just the user + group, and read only for 'other' (rw-rw--r) Mathieu Xavier Mauser wrote: > On 2006-04-18 15:23:48 -0700, Greg Ercolano said: > >> Greg Ercolano wrote: >>> OSX: HOW TO CHANGE THE GLOBAL UMASK FOR ALL USERS TO 002 >>> -------------------------------------------------------- > > Hi > > This is what works for my set up. > > 1. For group read-write perms, in the Finder. Use Terminal to set > globally (all users/same box): > > sudo defaults write /Library/Preferences/.GlobalPreferences NSUmask 2 > > 2. And, in Terminal, to set this for apps launched from the Bash shell: > > Put "umask 002" in /etc/profile (with no quotation marks) *** ***************************************************** *** Message #1883 ***************************************************** *** ***************************************************** Subject: Re: NFS Permissions & Snow Leopard From: Greg Ercolano Date: Wed, 02 Sep 2009 14:10:35 -0400 Craig Allison wrote: > Just to let everyone know I've just done an install of Rush version > 102.42a9 on Snow Leopard and all works well as long as you install Rosetta. Actually, no, you /shouldn't/ need Rosetta if you comment out this section of the install script: # rez=$RUSH_DIR/etc/bin/Rez # for i in $RUSH_DIR/examples/bin/input.app/Contents/MacOS/input \ # $RUSH_DIR/examples/bin/input; do # echo -n "-- Creating resource forks for 'input': " # i_dir=`dirname $i` # if ( cd $i_dir && $rez -t APPL $RUSH_DIR/etc/bin/mac.r -o input ); then # echo OK # else # echo "FAILED while working on ${i}" # let errors++ # fi # done The rush binaries will all run fine (including the GUIs) without the above 'Rez' code. This is only needed for older versions of OSX that depended on resource/data forks. Next release will have a fix for that, but you can just comment out the section by hand. If you're installing rush Universal build (which has the intel and ppc binaries) you won't need Rosetta. Rosetta would only be needed if you had an old version of Rush that was PPC only, which would be about 3 years old or more. > Slightly off topic as far as rush is concerned, but I wondered if anyone > out there is using Leopard or Snow Leopard over NFS to an XSan share and > is getting the same issue as me with permissions. > > If I open a tif file from a Leopard box client, for example, edit it > then try to save over the file I get a "file locked" permission error, > this is using both Preview and Photoshop. What are your NFS mount flags? Also, while the file is open, does ls(1) show any interesting flags on the file? (eg. 'ls -la@" (extended attributes), 'ls -lae' (ACLS) ) I often disable file locking over NFS because I've never trusted it; see 'man mount_nfs' the 'nolocks' option. Not sure though if this will affect your situation or not. It sounds like somehow the lock is remaining active. I'll try your test with Preview in just a bit. Could be an actual problem with how > And just another gripe with Snow/Leopard, why does the Finder often show > a bunch of identical 2k dpx files as different file sizes, even though > Get Info shows the correct, and identical, byte count???? i.e. A byte > count of 12,754,744 displays in the Finder as 15.4Mb, with the next file > in the sequence showing as 17.3Mb even though it has the same byte count!! Wow, sounds like a Finder bug. IIRC, in Snow Leopard the Finder is a complete rewrite, so there could easily be hiccups in this new code. For a long time back in the 10.0/10.1 days, there were bad bugs in the Finder's computation of disk size. Snow Leopard is only 10.5.0 at the moment (".0" releases are always the most unstable) -- there will probably be updates that fix it. But you might want to report it, giving details of your Xsan/NFS mount flags and file system details (HFS, HFS+). I'll try a few experiments later today with some large files just to check this, as I'm curious. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1882 ***************************************************** *** ***************************************************** Subject: NFS Permissions & Snow Leopard From: Craig Allison Date: Wed, 02 Sep 2009 12:17:28 -0400 --Apple-Mail-5-870611914 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Just to let everyone know I've just done an install of Rush version 102.42a9 on Snow Leopard and all works well as long as you install Rosetta. Slightly off topic as far as rush is concerned, but I wondered if anyone out there is using Leopard or Snow Leopard over NFS to an XSan share and is getting the same issue as me with permissions. If I open a tif file from a Leopard box client, for example, edit it then try to save over the file I get a "file locked" permission error, this is using both Preview and Photoshop. This is to a share coming from a Leopard server box, if I do the same to a share from a Tiger box it works fine. If I use a Tiger client I can overwrite files from both the Leopard share and the Tiger share. I have checked that the correct user with rwx is being exported from the servers and all is well. And just another gripe with Snow/Leopard, why does the Finder often show a bunch of identical 2k dpx files as different file sizes, even though Get Info shows the correct, and identical, byte count???? i.e. A byte count of 12,754,744 displays in the Finder as 15.4Mb, with the next file in the sequence showing as 17.3Mb even though it has the same byte count!! Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com --Apple-Mail-5-870611914 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Just to let everyone know I've just done an install of Rush version = 102.42a9 on Snow Leopard and all works well as long as you install = Rosetta.

Slightly = off topic as far as rush is concerned, but I wondered if anyone out = there is using Leopard or Snow Leopard over NFS to an XSan share and is = getting the same issue as me with permissions.

If I open a tif file from = a Leopard box client, for example, edit it then try to save over the = file I get a "file locked" permission error, this is using both Preview = and Photoshop.

This is to a share coming = from a Leopard server box, if I do the same to a share from a Tiger box = it works fine.

If I use a Tiger client I = can overwrite files from both the Leopard share and the Tiger = share.

I = have checked that the correct user with rwx is being exported from the = servers and all is well.

And just another gripe = with Snow/Leopard, why does the Finder often show a bunch of identical = 2k dpx files as different file sizes, even though Get Info shows the = correct, and identical, byte count???? i.e. A byte count of 12,754,744 = displays in the Finder as 15.4Mb, with the next file in the sequence = showing as 17.3Mb even though it has the same byte count!!

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


= --Apple-Mail-5-870611914-- *** ***************************************************** *** Message #1881 ***************************************************** *** ***************************************************** Subject: Re: Adding extra variables to logs From: Craig Allison Date: Thu, 13 Aug 2009 04:40:44 -0400 --Apple-Mail-49--884753689 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Thanks Greg Some great tips there if I have to change the way this works or add any extra reporting. Much appreciated once again! Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 12 Aug 2009, at 19:46, Greg Ercolano wrote: > [posted to rush.general] > > Hi Craig, > > Glad you found a solution. > > Here's what I was going to reply yesterday, but forgot > to relply, which may give you some other ideas as well: > >>> I have added custom fields to the rush submit and would like to be >>> able to include the slate notes on the jobdone email to give >>> production a better idea of what's been done. > > Your jobdonecommand can invoke a script that gathers the > info from the files left behind in the log directory, or > from rush itself. > > When the jobdonecommand runs, environment variables such > as RUSH_JOBID will be set, which you can use to gather > info about the job with regular rush commands (like 'rush -ljf', > 'rush -lf', etc). > > Any info you add to the submit form you can save into a > data file in the log directory at submit time, eg: > > # SAVE '$in{MySlateInfo}' TO A FILE IN THE RUSH LOG DIRECTORY > # This will be picked up by the jobdonecommand when the job is > done. > # > if ( $in{LogDirectory} ne "-" ) { > my $slatefile = "$in{LogDirectory}/slate-info.dat"; > open(SLATEFILE, ">$slatefile"); > print SLATEFILE "$in{MySlateInfo}\n"; > close(SLATEFILE); > } > > ..then, when the job finishes and your jobdonecommand is running, > it can use the RUSH_JOBID or RUSH_LOGFILE variables to determine > the location of the log directory, load the 'slate-info.dat' file > (if it exists) and use that for the emails. > >>> Does anyone know a good way of adding new variables to the >>> jobdonecommand log or the job log? > > To add info to the 'jobdonecommand log', just have your > script print the messages you want to either stdout or stderr > and they'll appear in the jobdonecommand log. > >>> I have used the jobdonecommand.log to pull in the artist's name >>> to the >>> email from the "Owner" field. > > To send mail from the jobdonecommand script, you can use > the various scripting techniques for sending email, such as > perl's Net::SMTP module, eg: > > ---- snip > #!/usr/bin/perl -w > # > # Example of how to send emails from perl on all platforms (win/ > unix/osx) > # > use strict; > use Net::SMTP; > my $mailto = "foo\@bar.com"; > my $relay = "relay.bar.com"; > my $smtp = Net::SMTP->new($relay); > if ( ! defined($smtp) ) { > # ERROR OCCURRED -- TRY AGAIN W/DEBUG ENABLED > $smtp = Net::SMTP->new($relay, Debug => 1); > if ( !defined($smtp) ) { > print "ERROR: Could not connect to $relay (see above)\n"; > exit(1); > } > } > my ($errs, $expect) = (0,0); > $expect++; $errs += $smtp->mail($mailto); > $expect++; $errs += $smtp->to($mailto); > $expect++; $errs += $smtp->data(); > $expect++; $errs += $smtp->datasend("From: Foo <$mailto>\n"); > $expect++; $errs += $smtp->datasend("To: $mailto\n"); > $expect++; $errs += $smtp->datasend("Subject: testing from SMTP/perl > \n"); > $expect++; $errs += $smtp->datasend("\n"); > $expect++; $errs += $smtp->datasend("This is a test\n"); > $expect++; $errs += $smtp->datasend("\n"); > $expect++; $errs += $smtp->dataend(); > $expect++; $errs += $smtp->quit; > print "EXPECT=$expect, ERRORS=$errs\n"; > if ( $expect != $errs ) { > print "SMTP_ERROR=".$smtp->cgi_error()."\n"; > } > exit(0); > ---- snip > > Craig Allison wrote: >> I've solved the issue by getting the submit script to send an >> email to >> the production team containing Shot/Artist/Frames/Notes and then the >> jobdonecommand lets them know when it's complete and ready for >> review. >> >> Cheers >> Craig >> >> On 10 Aug 2009, at 15:01, Craig Allison wrote: >> >>> Hello there >>> >>> Does anyone know a good way of adding new variables to the >>> jobdonecommand log or the job log? >>> >>> I have added custom fields to the rush submit and would like to be >>> able to include the slate notes on the jobdone email to give >>> production a better idea of what's been done. >>> >>> I have used the jobdonecommand.log to pull in the artist's name >>> to the >>> email from the "Owner" field. > > > -- > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) > --Apple-Mail-49--884753689 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Thanks Greg

Some = great tips there if I have to change the way this works or add any extra = reporting.

Much appreciated once = again!

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


On 12 Aug 2009, at 19:46, Greg Ercolano = wrote:

[posted to = rush.general]

Hi Craig,

Glad you found a = solution.

Here's what I was going to reply = yesterday, but forgot
to = relply, which may give you some other ideas as well:

I have added = custom fields to the rush submit and would like to be
able to include the slate notes on the jobdone email = to give
production a better idea of = what's been done.

Your = jobdonecommand can invoke a script that gathers the
info from the files left behind = in the log directory, or
from rush = itself.

When the jobdonecommand runs, = environment variables such
as = RUSH_JOBID will be set, which you can use to gather
info about the job with regular = rush commands (like 'rush -ljf',
'rush = -lf', etc).

Any info you add to the submit = form you can save into a
data file = in the log directory at submit time, eg:

# SAVE = '$in{MySlateInfo}' TO A FILE IN THE RUSH LOG DIRECTORY
# =A0 =A0 = This will be picked up by the jobdonecommand when the job is = done.
#
if ( = $in{LogDirectory} ne "-" ) {
=A0 =A0 my $slatefile =3D = "$in{LogDirectory}/slate-info.dat";
=A0 =A0 open(SLATEFILE, = ">$slatefile");
=A0 =A0 print SLATEFILE = "$in{MySlateInfo}\n";
=A0 =A0 = close(SLATEFILE);
}

..then, = when the job finishes and your jobdonecommand is running,
it can use the RUSH_JOBID or = RUSH_LOGFILE variables to determine
the = location of the log directory, load the 'slate-info.dat' file
(if it exists) and use that for = the emails.

=
Does anyone know a good way of adding new variables = to the
jobdonecommand log or the job = log?

To add info to the = 'jobdonecommand log', just have your
script = print the messages you want to either stdout or stderr
and they'll appear in the = jobdonecommand log.
I have used the jobdonecommand.log to pull in the = artist's name to the
email from the "Owner" = field.

To send mail from the = jobdonecommand script, you can use
the = various scripting techniques for sending email, such as
perl's Net::SMTP module, = eg:

---- snip
#
# Example of how to send emails from perl on all = platforms (win/unix/osx)
#
use strict;
use = Net::SMTP;
my $mailto =3D = "foo\@bar.com";
my $relay=A0 =3D = "relay.bar.com";
my $smtp =3D = Net::SMTP->new($relay);
if ( ! = defined($smtp) ) {
=A0 =A0 # ERROR OCCURRED -- TRY = AGAIN W/DEBUG ENABLED
=A0 =A0 $smtp =3D = Net::SMTP->new($relay, Debug =3D> 1);
=A0 =A0 if ( !defined($smtp) ) = {
=A0 =A0 =A0= =A0 print "ERROR: Could not connect to $relay (see = above)\n";
=A0 =A0 =A0 =A0 = exit(1);
=A0 =A0 }
}
my ($errs, $expect) =3D = (0,0);
$expect++; $errs +=3D = $smtp->mail($mailto);
$expect++; = $errs +=3D $smtp->to($mailto);
$expect++; = $errs +=3D $smtp->data();
$expect++; = $errs +=3D $smtp->datasend("From: Foo <$mailto>\n");
$expect++; $errs +=3D $smtp->datasend("To: = $mailto\n");
$expect++; $errs +=3D = $smtp->datasend("Subject: testing from SMTP/perl\n");
$expect++; $errs +=3D = $smtp->datasend("\n");
$expect++; = $errs +=3D $smtp->datasend("This is a test\n");
$expect++; $errs +=3D = $smtp->datasend("\n");
$expect++; = $errs +=3D $smtp->dataend();
$expect++; = $errs +=3D $smtp->quit;
print = "EXPECT=3D$expect, ERRORS=3D$errs\n";
if ( = $expect !=3D $errs ) {
=A0 =A0 print = "SMTP_ERROR=3D".$smtp->cgi_error()."\n";
exit(0);
---- snip

Craig Allison wrote:
=
I've solved the issue by = getting the submit script to send an email to
the production team containing = Shot/Artist/Frames/Notes and then the

Cheers

On 10 Aug 2009, at 15:01, Craig Allison = wrote:

=
Hello there

Does = anyone know a good way of adding new variables to the
jobdonecommand log or the job log?

I have = added custom fields to the rush submit and would like to be
able to include the slate notes on the jobdone email = to give
production a better idea of = what's been done.

I have used the jobdonecommand.log to pull in the = artist's name to the
email from the "Owner" = field.


--=A0
Greg = Ercolano, erco@(email surpressed)
Seriss Corporation
Rush = Render Queue, http://seriss.com/rush/
Tel: 626-795-5922x23
Fax: = 626-795-5947
Cel: 310-266-8906

=

= --Apple-Mail-49--884753689-- *** ***************************************************** *** Message #1880 ***************************************************** *** ***************************************************** Subject: Re: Adding extra variables to logs From: Greg Ercolano Date: Wed, 12 Aug 2009 14:46:33 -0400 Hi Craig, Glad you found a solution. Here's what I was going to reply yesterday, but forgot to relply, which may give you some other ideas as well: >> I have added custom fields to the rush submit and would like to be >> able to include the slate notes on the jobdone email to give >> production a better idea of what's been done. Your jobdonecommand can invoke a script that gathers the info from the files left behind in the log directory, or from rush itself. When the jobdonecommand runs, environment variables such as RUSH_JOBID will be set, which you can use to gather info about the job with regular rush commands (like 'rush -ljf', 'rush -lf', etc). Any info you add to the submit form you can save into a data file in the log directory at submit time, eg: # SAVE '$in{MySlateInfo}' TO A FILE IN THE RUSH LOG DIRECTORY # This will be picked up by the jobdonecommand when the job is done. # if ( $in{LogDirectory} ne "-" ) { my $slatefile = "$in{LogDirectory}/slate-info.dat"; open(SLATEFILE, ">$slatefile"); print SLATEFILE "$in{MySlateInfo}\n"; close(SLATEFILE); } ..then, when the job finishes and your jobdonecommand is running, it can use the RUSH_JOBID or RUSH_LOGFILE variables to determine the location of the log directory, load the 'slate-info.dat' file (if it exists) and use that for the emails. >> Does anyone know a good way of adding new variables to the >> jobdonecommand log or the job log? To add info to the 'jobdonecommand log', just have your script print the messages you want to either stdout or stderr and they'll appear in the jobdonecommand log. >> I have used the jobdonecommand.log to pull in the artist's name to the >> email from the "Owner" field. To send mail from the jobdonecommand script, you can use the various scripting techniques for sending email, such as perl's Net::SMTP module, eg: ---- snip #!/usr/bin/perl -w # # Example of how to send emails from perl on all platforms (win/unix/osx) # use strict; use Net::SMTP; my $mailto = "foo\@bar.com"; my $relay = "relay.bar.com"; my $smtp = Net::SMTP->new($relay); if ( ! defined($smtp) ) { # ERROR OCCURRED -- TRY AGAIN W/DEBUG ENABLED $smtp = Net::SMTP->new($relay, Debug => 1); if ( !defined($smtp) ) { print "ERROR: Could not connect to $relay (see above)\n"; exit(1); } } my ($errs, $expect) = (0,0); $expect++; $errs += $smtp->mail($mailto); $expect++; $errs += $smtp->to($mailto); $expect++; $errs += $smtp->data(); $expect++; $errs += $smtp->datasend("From: Foo <$mailto>\n"); $expect++; $errs += $smtp->datasend("To: $mailto\n"); $expect++; $errs += $smtp->datasend("Subject: testing from SMTP/perl\n"); $expect++; $errs += $smtp->datasend("\n"); $expect++; $errs += $smtp->datasend("This is a test\n"); $expect++; $errs += $smtp->datasend("\n"); $expect++; $errs += $smtp->dataend(); $expect++; $errs += $smtp->quit; print "EXPECT=$expect, ERRORS=$errs\n"; if ( $expect != $errs ) { print "SMTP_ERROR=".$smtp->cgi_error()."\n"; } exit(0); ---- snip Craig Allison wrote: > I've solved the issue by getting the submit script to send an email to > the production team containing Shot/Artist/Frames/Notes and then the > jobdonecommand lets them know when it's complete and ready for review. > > Cheers > Craig > > On 10 Aug 2009, at 15:01, Craig Allison wrote: > >> Hello there >> >> Does anyone know a good way of adding new variables to the >> jobdonecommand log or the job log? >> >> I have added custom fields to the rush submit and would like to be >> able to include the slate notes on the jobdone email to give >> production a better idea of what's been done. >> >> I have used the jobdonecommand.log to pull in the artist's name to the >> email from the "Owner" field. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1879 ***************************************************** *** ***************************************************** Subject: Re: Adding extra variables to logs From: Craig Allison Date: Wed, 12 Aug 2009 07:46:36 -0400 --Apple-Mail-37--960002535 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed I've solved the issue by getting the submit script to send an email to the production team containing Shot/Artist/Frames/Notes and then the jobdonecommand lets them know when it's complete and ready for review. Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 10 Aug 2009, at 15:01, Craig Allison wrote: > Hello there > > Does anyone know a good way of adding new variables to the > jobdonecommand log or the job log? > > I have added custom fields to the rush submit and would like to be > able to include the slate notes on the jobdone email to give > production a better idea of what's been done. > > I have used the jobdonecommand.log to pull in the artist's name to > the email from the "Owner" field. > > Cheers > > Craig > > Craig Allison > Digital Systems & Data Manager > The Senate Visual Effects > Twickenham Film Studios > St.Margarets > Twickenham > TW1 2AW > > t: 0208 607 8866 > skype: craig_9000 > www.senatevfx.com > > > --Apple-Mail-37--960002535 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 I've solved the issue by getting the submit script to send an email to = the production team containing Shot/Artist/Frames/Notes and then the = jobdonecommand lets them know when it's complete and ready for = review.

Cheers

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


On 10 Aug 2009, at 15:01, Craig Allison = wrote:

Hello there

Does anyone know a good = way of adding new variables to the jobdonecommand log or the job = log?

I have = added custom fields to the rush submit and would like to be able to = include the slate notes on the jobdone email to give production a better = idea of what's been done.

I have used the = jobdonecommand.log to pull in the artist's name to the email from the = "Owner" field.

Cheers

Craig

The Senate Visual = Effects
St.Margarets

t: 0208 607 8866=A0
skype: craig_9000

=


= --Apple-Mail-37--960002535-- *** ***************************************************** *** Message #1878 ***************************************************** *** ***************************************************** Subject: Adding extra variables to logs From: Craig Allison Date: Mon, 10 Aug 2009 10:01:36 -0400 --Apple-Mail-19-1022777502 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello there Does anyone know a good way of adding new variables to the jobdonecommand log or the job log? I have added custom fields to the rush submit and would like to be able to include the slate notes on the jobdone email to give production a better idea of what's been done. I have used the jobdonecommand.log to pull in the artist's name to the email from the "Owner" field. Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com --Apple-Mail-19-1022777502 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hello there

Does = anyone know a good way of adding new variables to the jobdonecommand log = or the job log?

I have added custom fields = to the rush submit and would like to be able to include the slate notes = on the jobdone email to give production a better idea of what's been = done.

I have = used the jobdonecommand.log to pull in the artist's name to the email = from the "Owner" field.

Cheers

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


= --Apple-Mail-19-1022777502-- *** ***************************************************** *** Message #1877 ***************************************************** *** ***************************************************** Subject: Re: Rush FIFO fix From: Greg Ercolano Date: Thu, 30 Jul 2009 01:54:06 -0400 Greg Ercolano wrote: > (If you don't use FIFO scheduling, you can ignore this) > > The folks at Deluxe just helped me identify a problem with FIFO scheduling > when multiple job servers are in use. > [..] > Watch this space for release info. I've uploaded zip/tar files for Deluxe to try out; basically a new rushd daemon solves the problem. Once they've confirmed things are working well, I'll make the new fixes generally available. *** ***************************************************** *** Message #1876 ***************************************************** *** ***************************************************** Subject: Rush FIFO fix From: Greg Ercolano Date: Wed, 29 Jul 2009 20:50:07 -0400 (If you don't use FIFO scheduling, you can ignore this) The folks at Deluxe just helped me identify a problem with FIFO scheduling when multiple job servers are in use. We just nailed down the problem minutes ago, and I've found the problem and I think I've fixed it.. will continue to do some testing, and should have a fix release by tonight. The problem shows up in Rush 102.42a9 thru 102.42a9b, and the problem shows up where jobs simply don't always schedule in FIFO order when multiple job servers are involved. 102.42a9c will have the fix. You won't likely see the problem if all your jobs are being submitted to a single job server (I know some of you prefer to run this way, so you may not have noticed it). I suppose that is something of a "workaround" if you don't have the ability to take the update. Watch this space for release info. *** ***************************************************** *** Message #1875 ***************************************************** *** ***************************************************** Subject: Rush 102.42a9 runs OK on Windows 7 RC From: Greg Ercolano Date: Fri, 24 Jul 2009 15:18:22 -0400 Rush 102.42a9 (and up) tested to run fine on Windows 7 64bit RC. Rush and Perl both installed normally with no caveats. Submitted and rendered network jobs with UNC paths to a linux/samba file server, bouncing cpu bars in rushtop. As always, be sure to run the install as the real Administrator. The UAC (introduced in Vista) causes there to be a distinction between running a DOS window as an administrative user vs. "running as Administator". An easy way toopen a DOS window as Administrator: 1) Start->All Programs->Accessories 2) Right click on 'Command Prompt' and choose "Run as administrator". This ensures the install script has sufficient privileges to make changes to the system environment variables (eg. for making changes to the PATH) * * * Some general comments about Windows 7 that are not Rush related: o 'edit' is gone, apparently. It was a nice DOS based text editor. o 'wordpad' is still not in the PATH, causing a 'command not found' in the DOS shell. o 'notepad' is still as dumb as a sack of hammers o Start -> Run, where have you gone? o To enable the real administrator user so that you can actually login as administrator from the login screen: 1) Start->All Programs->Accessories, right click "Command Prompt" and choose "Run as administrator". 2) In the DOS shell, run: net user administrator /active:yes 3) Restart or log off *** ***************************************************** *** Message #1874 ***************************************************** *** ***************************************************** Subject: Re: First In-First Out Tip From: Marco Recuay Date: Mon, 20 Jul 2009 19:37:49 -0400 Thanks for the correction Greg.. I guess I'll have to do some more testing on this. The clocks all appear synchronized on all the machines, and they all have the same rush version and updated rush.conf file. Using the same job server is a workaround for now, but when we have a free moment I'll see how they react to the tests you outlined to pinpoint where the problem lies. On 2009-07-20 08:40:37 -0700, Greg Ercolano said: > BTW, Marco, if you're having strange behavior with different > job servers, I'll be happy to help. > > Make sure you've got 'sched fifo' in ALL the rush.conf files, > and are running the same version of rush that supports FIFO > (102.42a9 or higher) on all the machines, since the older > releases won't understand FIFO scheduling. > > There are some easy tests you can do from the command line > to submit a couple of jobs and watch them compete. > > For instance, from a Unix machine (linux or OSX), you can > submit two 100 frame jobs as a test: > > (echo title AAA; echo frames 1-100; echo cpus +any=500@10; echo command > rush -sleep 10) | rush -submit > > (echo title BBB; echo frames 1-100; echo cpus +any=500@10; echo command > rush -sleep 10) | rush -submit > > Each line will submit a job that does nothing but sleep 10 seconds > per frame. > > Those should be two separate lines; make sure your newsreader > is wide enough to show those as complete lines before copy/pasting. > > Run the lines one at a time, with at least 1 or 2 seconds between > each submit, so the FIFO system can tell which job was first. > > When running, the AAA job should get all the cpus, and the BBB > job should wait around until the AAA job starts finishing. > > You should also be able to run similar command lines on other job > servers, or tack a hostname on the end, after the 'rush -submit', > to tell it to use some other machine as the job server, eg. to use > 'tahoe' as the job server: > > (echo title CCC; echo frames 1-100; echo cpus +any=500@10; echo command > rush -sleep 10) | rush -submit tahoe *** ***************************************************** *** Message #1873 ***************************************************** *** ***************************************************** Subject: Re: First In-First Out Tip From: Greg Ercolano Date: Mon, 20 Jul 2009 11:40:37 -0400 BTW, Marco, if you're having strange behavior with different job servers, I'll be happy to help. Make sure you've got 'sched fifo' in ALL the rush.conf files, and are running the same version of rush that supports FIFO (102.42a9 or higher) on all the machines, since the older releases won't understand FIFO scheduling. There are some easy tests you can do from the command line to submit a couple of jobs and watch them compete. For instance, from a Unix machine (linux or OSX), you can submit two 100 frame jobs as a test: (echo title AAA; echo frames 1-100; echo cpus +any=500@10; echo command rush -sleep 10) | rush -submit (echo title BBB; echo frames 1-100; echo cpus +any=500@10; echo command rush -sleep 10) | rush -submit Each line will submit a job that does nothing but sleep 10 seconds per frame. Those should be two separate lines; make sure your newsreader is wide enough to show those as complete lines before copy/pasting. Run the lines one at a time, with at least 1 or 2 seconds between each submit, so the FIFO system can tell which job was first. When running, the AAA job should get all the cpus, and the BBB job should wait around until the AAA job starts finishing. You should also be able to run similar command lines on other job servers, or tack a hostname on the end, after the 'rush -submit', to tell it to use some other machine as the job server, eg. to use 'tahoe' as the job server: (echo title CCC; echo frames 1-100; echo cpus +any=500@10; echo command rush -sleep 10) | rush -submit tahoe -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1872 ***************************************************** *** ***************************************************** Subject: Re: First In-First Out Tip From: Greg Ercolano Date: Mon, 20 Jul 2009 11:18:56 -0400 Marco Recuay wrote: > Just wanted to pass on my experience with the new FIFO scheduling, > since it wasn't working as I expected when first enabled. > > It appears that FIFO only schedules the jobs on the same job server. If > you want all renders across the network to schedule FIFO, you need to > make sure that jobs are sent to one submit host only, which then orders > them based on the job age. Hmm, an interesting theory, but no -- FIFO works off time stamps from when the job was submitted, regardless which machine is the job server. Assuming the machines acting as job servers are time synched, it shouldn't matter which machine is the job server. You can see the raw FIFO numbers used for scheduling in the "Jobs Full" report: "FifoSchedOrder" -- the raw unix time() value (eg. 3542120794) "FifoSchedOrderDate" -- the same value as a date/time (eg. 06/26/09,09:28:23.41) The 'Elapsed' time in the normal "Jobs" all "All Jobs" report can usually be used as the indicator of the FIFO scheduling value, assuming the clocks are synched well. The way FIFO works: when a proc becomes available, and the priority of the jobs asking to use it are equal, the unix time value is used to figure out which job gets to run; the oldest job wins the proc. So if you have 100 machines and everyone's submitting with +any=100@10, then no matter which machine is the job server, the oldest job gets the next available proc. If you have 100 machines and people are asking for less cpus than are on the farm, eg. +any=5@10, then the oldest job will get the first 5 procs, and the next oldest job(s) will get the rest. Note that if people are submitting with different priorities, then FIFO breaks up into tiers of priority, ie. priority takes precedence over FIFO. For instance if everyone is in the FIFO queue @10 priority (examples above), and NEW GUY submits @11 priority, his job will be in front of all the @10's. And if other people submit @11 (same as him), then FIFO rules will determine which of the @11 jobs get the next cpu. Anything "leftover" goes to the other jobs in the @10 priority. So if these jobs are submitted, the next cpu to come available will try to give itself to one of the jobs in this order, starting at the top: OWNER PRIORITY SUBMIT TIME ----- ---------- ----------- _ Fred +any=100@11 14:00 Today | 11 priority "tier" Gina +any=100@11 15:00 Today _| _ Jane +any=100@10 9:00 Today | Tim +any=100@10 13:00 Today | 10 priority "tier" Sandy +any=100@10 14:40 Today _| So the next available cpu will go to Fred first, and if his job is maxed out on cpus or has no more frames to render, Gina's job is next, then Jane, Tim, and Sandy last. Note that even though Jane submitted before everyone (9am), she was still @10 priority, so she still comes "after" all the higher @11 priority jobs. > One trick would also be to have more than one designated submit host, > so you could have separate FIFO queues for different tasks. You can get different FIFO queues via priority, as shown above where there are two FIFO queues; @11 and @10. You can also get different FIFO queues by requesting different hostgroups, ie. splitting the farm into two or more groups. So if there's a job that needs to be in front of everything in the FIFO queue, but only needs 5 procs, then that job should submit at a higher priority (eg. @11 in the examples above) but should just ask for =5, eg. +any=5@11 Let me know if anyone has questions about the above, or if you'd like more details. Pretty much the easiest way to make use of FIFO is to just have everyone submit at the SAME PRIORITY, and FIFO will be very predictable. It's only when you introduce priority differences that you get the multiple FIFO queues happening. *** ***************************************************** *** Message #1871 ***************************************************** *** ***************************************************** Subject: First In-First Out Tip From: Marco Recuay Date: Sun, 19 Jul 2009 21:08:19 -0400 Just wanted to pass on my experience with the new FIFO scheduling, since it wasn't working as I expected when first enabled. It appears that FIFO only schedules the jobs on the same job server. If you want all renders across the network to schedule FIFO, you need to make sure that jobs are sent to one submit host only, which then orders them based on the job age. One trick would also be to have more than one designated submit host, so you could have separate FIFO queues for different tasks. That's it... something that might have been obvious to other admins, but I think it's worth documenting here. -Marco *** ***************************************************** *** Message #1870 ***************************************************** *** ***************************************************** Subject: Re: Job Done Command Not Working On Some Boxes From: Greg Ercolano Date: Fri, 17 Jul 2009 23:20:37 -0400 Newsgroup follow up. OK, after working with Craig earlier this week, I was able to determine the exact combo of events it took to replicate this. Turned out these specifics were needed to make this problem show up: o Rush version 102.42a9 on both machines o Submit from a Tiger workstation with "Submit Host:" pointing at a leopard jobserver o Submitting user has no account on the jobserver, just their workstation o Rush configured to force all renders to run as "render" user o "render" user has a valid account on all machines (workstation and jobserver) o Job submitted with a jobdonecommand With this combo, the renders run ok, but the /jobdonecommand/ fails to run on the leopard machine with this series of errors in the log: >07/17,20:05:03 SECURITY RUSH: uid '0' outside configured range 100-65000 >07/17,20:05:03 INFO jobdonecommand 'ls' failed for jobid=leopard.2: RUSH: uid '0' outside configured range 100-65000 Now that I've got the problem canned, and replicated here at my office, I should have it solved soon -- pretty sure now it's a bug in Rush. *** ***************************************************** *** Message #1869 ***************************************************** *** ***************************************************** Subject: [Q+A] Submit script GUI forms From: Greg Ercolano Date: Thu, 16 Jul 2009 13:09:08 -0400 > We're playing around with customizing the input forms in the > submit scripts. It looks like the submit scripts use a compact > shorthand for defining the forms that is not the same as the format > described in the docs for the 'input.exe' program: > http://www.seriss.com/rush-current/input/ I'm guessing you mean the shorthand 'ascii art' representation of the submit forms that you see in all the modern Rush submit scripts (last 5 years) e.g.: -------------------------------------------------------------------------- snip <> "Rush" = Log Directory: ___________________________________________ ? Browse"" = Log Flags: "Keep Last,Keep All,Overwrite" ? = Max Log Size: ______________ ? AutoDump: "off,fail,done,donefail" ? Done Mail: ______________ ? Dump Mail: ______________ ? -------------------------------------------------------------------------- snip Right.. this format you see in the submit scripts is not actually passed directly to input.exe.. there's a function in .common.pl called CreateInputForm($$$) which converts the "ascii art" into the "long format" that input.exe understands. The reason for doing this: In ye olde days (2003 or so), each submit script had the "long form" input.exe commands, which made the scripts huge and hard to read. Also, everyone edits those scripts to customize, and all that redundant code seemed unnecessary bulk. So I thought, "what's the easiest way to represent these forms?" to reduce the size of the scripts and make them easier to read. 'ascii art' seemed like the best solution; it visually looks just like what you see in the submit forms; easy to read, easy to change. It avoids redundant blocks of code, and I took a few shortcuts to represent common things to all the submit forms (submit/help/default buttons, etc) So I inserted the CreateInputForm() function in .common.pl, and slowly modified all the submit scripts to use the ascii art representation, and pass it to CreateInputForm() to open the GUI. input.exe still understands the "long form", so old scripts with the long form content still work normally. So anyway, the intermediate "long form" GUI commands are generated on the fly by that CreateInputForm() function, and these files are left behind (for info purposes) in the usual place: UNIX: ~/.rush/submit-xxx.in WINDOWS: c:/temp/.rush-/submit-xxx.in What's interesting is that in the next release, input.exe will be able to parse the ascii art format directly. This was done so that submit scripts can be written in other languages (eg. python) without having to port all the logic in CreateInputForm() to each scripting language. *** ***************************************************** *** Message #1868 ***************************************************** *** ***************************************************** Subject: Re: Job Done Command Not Working On Some Boxes From: Greg Ercolano Date: Mon, 13 Jul 2009 16:31:44 -0400 Craig Allison wrote: > Hey Greg > > I'm working late tonight so we're kind of on the same time for now : ) > > Checked the log on the host machine and for the fail jobs I'm getting > the following: > > jobdonecommand 'perl /mnt/vfxxserve3/Scripts/Senate_DoneMail.pl' failed > for jobid=senatemac89.326: RUSH: uid '0' outside configured range 100-65000 > > I've got forceuid/gid set to 501 on the user's machine in rush.conf, > where is it picking this one up from? Hi Craig, Hmm.. apparently it thinks it should be trying to run the command as root on that machine (senatemac89). From that machine, can you send me (via private email): a) The contents of: /usr/local/rush/etc/rush.conf b) The output of: rush -ljf senatemac89.326 It sounds like maybe there's a forceuid of zero on that machine, or the job's owner is root, or there's a bug I need to look into.. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1867 ***************************************************** *** ***************************************************** Subject: Re: Job Done Command Not Working On Some Boxes From: Craig Allison Date: Mon, 13 Jul 2009 16:22:52 -0400 --Apple-Mail-1-773902012 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hey Greg I'm working late tonight so we're kind of on the same time for now : ) Checked the log on the host machine and for the fail jobs I'm getting the following: jobdonecommand 'perl /mnt/vfxxserve3/Scripts/Senate_DoneMail.pl' failed for jobid=senatemac89.326: RUSH: uid '0' outside configured range 100-65000 I've got forceuid/gid set to 501 on the user's machine in rush.conf, where is it picking this one up from? Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 13 Jul 2009, at 20:19, Greg Ercolano wrote: > [posted to rush.general] > > Craig Allison wrote: >> I'm having a problem with certain boxes not running the JobDone >> Command >> on complete of a render. I'm not getting any logs from the faulty >> boxes >> to even suggest the command is being initiated. > > Hi Craig, > > I'd suggest looking in the rushd.log first for the machine acting > as the job server for the job in question. Ie. login to the machine > whose hostname is in the jobid, and look at the rush/var/rushd.log > file for errors about the jobdonecommand. > >> Running Rush 102.42a9 on Leopard and Tiger, running a perl script on >> JobDone. >> >> perl /mnt/vfxxserve3/Scripts/Senate_Donemail.pl >> >> Any ideas? > > Try sending me via private email: > > 1) the 'rush -ljf' info for this job > 2) the contents of your rushd.log > > ..Be sure the job got 'done' since midnight, so that any errors > would likely be in "today's" rushd.log. > > I'll follow up to the group on what we determine is the problem. > > Certainly you should be seeing a 'jobdonecommand.log' in the job's > log directory. Only reason you wouldn't is if a) logs are disabled, > b) the -nolog flag was specified for the jobdonecommand, c) the log > directory was unwritable for some reason by the job server, d) a bug > I don't know about, as 102.42a9 is very recent. > > -- > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) > --Apple-Mail-1-773902012 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hey Greg

I'm = working late tonight so we're kind of on the same time for now : = )

Checked = the log on the host machine and for the fail jobs I'm getting the = following:

jobdonecommand 'perl = /mnt/vfxxserve3/Scripts/Senate_DoneMail.pl' failed for = jobid=3Dsenatemac89.326: RUSH: uid '0' outside configured range = 100-65000

I've got forceuid/gid set to 501 on the user's machine=A0in = rush.conf, where is it picking this one up from?


Cheers


Craig



Craig Allison

Digital Systems & Data=A0Manager

The Senate Visual = Effects

St.Margarets

Twickenham

TW1 2AW


t: 0208 607 8866=A0

skype: craig_9000


=


On 13 Jul 2009, at 20:19, Greg Ercolano = wrote:

[posted to = rush.general]

Craig Allison wrote:
I'm having a problem with = certain boxes not running the JobDone Command
on complete of a render.=A0 I'm not getting any logs from = the faulty boxes
to even suggest the command is = being initiated.

Hi Craig,

I'd = suggest looking in the rushd.log first for the machine acting
as the job server for the job in = question. Ie. login to the machine
whose = hostname is in the jobid, and look at the rush/var/rushd.log
file for errors about the = jobdonecommand.

=
Running Rush 102.42a9 on = Leopard and Tiger, running a perl script on

perl = /mnt/vfxxserve3/Scripts/Senate_Donemail.pl

Any = ideas?
Try sending me via private = email:

1) the 'rush -ljf' info for this = job
2) the contents of your = rushd.log

..Be sure the job got 'done' = since midnight, so that any errors
would = likely be in "today's" rushd.log.

I'll follow up to the group on = what we determine is the problem.

Certainly you should be seeing a = 'jobdonecommand.log' in the job's
log = directory. Only reason you wouldn't is if a) logs are = disabled,
b) the -nolog flag was specified = for the jobdonecommand, c) the log
directory = was unwritable for some reason by the job server, d) a bug
I don't know about, as 102.42a9 = is very recent.

--=A0
Greg = Ercolano, erco@(email surpressed)
Seriss Corporation
Rush = Render Queue, http://seriss.com/rush/
Tel: 626-795-5922x23
Fax: = 626-795-5947
Cel: 310-266-8906

=

= --Apple-Mail-1-773902012-- *** ***************************************************** *** Message #1866 ***************************************************** *** ***************************************************** Subject: Re: Job Done Command Not Working On Some Boxes From: Greg Ercolano Date: Mon, 13 Jul 2009 15:19:35 -0400 Craig Allison wrote: > I'm having a problem with certain boxes not running the JobDone Command > on complete of a render. I'm not getting any logs from the faulty boxes > to even suggest the command is being initiated. Hi Craig, I'd suggest looking in the rushd.log first for the machine acting as the job server for the job in question. Ie. login to the machine whose hostname is in the jobid, and look at the rush/var/rushd.log file for errors about the jobdonecommand. > Running Rush 102.42a9 on Leopard and Tiger, running a perl script on > JobDone. > > perl /mnt/vfxxserve3/Scripts/Senate_Donemail.pl > > Any ideas? Try sending me via private email: 1) the 'rush -ljf' info for this job 2) the contents of your rushd.log ..Be sure the job got 'done' since midnight, so that any errors would likely be in "today's" rushd.log. I'll follow up to the group on what we determine is the problem. Certainly you should be seeing a 'jobdonecommand.log' in the job's log directory. Only reason you wouldn't is if a) logs are disabled, b) the -nolog flag was specified for the jobdonecommand, c) the log directory was unwritable for some reason by the job server, d) a bug I don't know about, as 102.42a9 is very recent. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1865 ***************************************************** *** ***************************************************** Subject: Job Done Command Not Working On Some Boxes From: Craig Allison Date: Mon, 13 Jul 2009 12:17:23 -0400 --Apple-Mail-19-759179278 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello there I'm having a problem with certain boxes not running the JobDone Command on complete of a render. I'm not getting any logs from the faulty boxes to even suggest the command is being initiated. Most of my boxes execute the perl script no problem but a few don't and I can't see anything different between the ones that do and the ones that don't. Running Rush 102.42a9 on Leopard and Tiger, running a perl script on JobDone. perl /mnt/vfxxserve3/Scripts/Senate_Donemail.pl Any ideas? Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com --Apple-Mail-19-759179278 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hello there

I'm = having a problem with certain boxes not running the JobDone Command on = complete of a render. =A0I'm not getting any logs from the faulty boxes = to even suggest the command is being initiated.

Most of my boxes execute = the perl script no problem but a few don't and I can't see anything = different between the ones that do and the ones that = don't.

Running= Rush 102.42a9 on Leopard and Tiger, running a perl script on = JobDone.

perl = /mnt/vfxxserve3/Scripts/Senate_Donemail.pl

Any ideas?

Cheers

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


= --Apple-Mail-19-759179278-- *** ***************************************************** *** Message #1864 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: autostarting in KDE From: Greg Ercolano Date: Fri, 10 Jul 2009 14:42:44 -0400 Ahh, you got it then -- great. Stephan Kosinski wrote: > Greg, > > I found out. hanks for the Autostart clue. > > I actually placed there a script a long time ago that declare the > station offline as soon as a user log in KDE. > > A line was lauching rushtop but it wasn't working and with time I > forgot this line was existing. > > The new version of rushtop seems to be have became launchable by KDE > autostart script. > > Case closed. > > Thank you. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1863 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: autostarting in KDE From: Stephan Kosinski Date: Fri, 10 Jul 2009 14:24:33 -0400 Greg, I found out. hanks for the Autostart clue. I actually placed there a script a long time ago that declare the station offline as soon as a user log in KDE. A line was lauching rushtop but it wasn't working and with time I forgot this line was existing. The new version of rushtop seems to be have became launchable by KDE autostart script. Case closed. Thank you. Stephan. Stephan Kosinski wrote: > Greg Ercolano wrote: >> Starting a new thread for this.. >> >> Stephan Kosinski wrote: >>> A quick question though since I propagated the new binary on my >>> linux >>> stations rushtop is auto-starting at user login. >> >> Hmm, nothing in rushtop or the rush installers does any autostart >> type stuff.. at least not in my code. >> >> Possibly the GUI toolkit rushtop is using has changed some internals >> that makes it more compatible with window manager autostart stuff >> that it didn't have before. >> >> Do other tools autostart if you logout while they're running? >> ie. firefox, konqueror, clocks, etc? > > Yes but only when the session has been saved with them running. That > means at logout or when the user save is session manually > >> >> Sounds like a feature of the window manager remembering what >> was running when you logout, and starting them again when you >> login. >> >> This stuff is likely saved in an ascii file in your ~/.kde directory, >> but by what mechanism I don't know. >> > > I'll check there > > >>> What is reaaly weird is that I was never able to do that with >>> previous >>> binary even by saving the KDE session with rushtop running. >> >> Hmm, I don't know what could have changed in rushtop to do that. >> Nothing in the code to rushtop itself.. I don't know of any way >> to control it from the GUI toolkit's API. >> >>> Do you have any idea how to prevent rushtop to auto-start? Is it >>> related to my system or to the new binary? >> >> Try closing rushtop *then* logging out, then back in. >> I would assume it won't start then. > > > First thing I tried. Close rushtop, save my session, logout and > login again... Rushtop is stubbornly coming back . :) > >> >> I stopped using KDE a while ago; too many memory hungry daemons and >> menus kept moving around, and same to be said for gnome, so I >> standardized on a custom 'tiny' window manager that has no daemons >> and one menu that I defined. This way I can upgrade my workstation, >> but have the interface stay the same. >> >> However, I'll try bringing up KDE on one of my other systems to see >> if I can at least replicate. >> >> But I figure KDE's own docs on autostart stuff will give reason >> to how to control autostarting, and possibly even specifying >> exclusion terms. >> > > Thank you for your help. I will keep on searching. It is not big > deal to have rushtop autostarts, just I'd like to understand and know > how to control it. > > Have a nice weekend. > > Stephan. *** ***************************************************** *** Message #1862 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: autostarting in KDE From: Stephan Kosinski Date: Fri, 10 Jul 2009 14:15:35 -0400 Greg Ercolano wrote: > Starting a new thread for this.. > > Stephan Kosinski wrote: >> A quick question though since I propagated the new binary on my linux >> stations rushtop is auto-starting at user login. > > Hmm, nothing in rushtop or the rush installers does any autostart > type stuff.. at least not in my code. > > Possibly the GUI toolkit rushtop is using has changed some internals > that makes it more compatible with window manager autostart stuff > that it didn't have before. > > Do other tools autostart if you logout while they're running? > ie. firefox, konqueror, clocks, etc? Yes but only when the session has been saved with them running. That means at logout or when the user save is session manually > > Sounds like a feature of the window manager remembering what > was running when you logout, and starting them again when you > login. > > This stuff is likely saved in an ascii file in your ~/.kde directory, > but by what mechanism I don't know. > I'll check there >> What is reaaly weird is that I was never able to do that with previous >> binary even by saving the KDE session with rushtop running. > > Hmm, I don't know what could have changed in rushtop to do that. > Nothing in the code to rushtop itself.. I don't know of any way > to control it from the GUI toolkit's API. > >> Do you have any idea how to prevent rushtop to auto-start? Is it >> related to my system or to the new binary? > > Try closing rushtop *then* logging out, then back in. > I would assume it won't start then. First thing I tried. Close rushtop, save my session, logout and login again... Rushtop is stubbornly coming back . :) > > I stopped using KDE a while ago; too many memory hungry daemons and > menus kept moving around, and same to be said for gnome, so I > standardized on a custom 'tiny' window manager that has no daemons > and one menu that I defined. This way I can upgrade my workstation, > but have the interface stay the same. > > However, I'll try bringing up KDE on one of my other systems to see > if I can at least replicate. > > But I figure KDE's own docs on autostart stuff will give reason > to how to control autostarting, and possibly even specifying > exclusion terms. > Thank you for your help. I will keep on searching. It is not big deal to have rushtop autostarts, just I'd like to understand and know how to control it. Have a nice weekend. Stephan. *** ***************************************************** *** Message #1861 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Fri, 10 Jul 2009 14:04:45 -0400 Abraham Schneider wrote: > Works fine for me, thanks alot! > Abraham Great, thanks for the feedback + debugging Abraham + Stephan. If anyone has anything else to report with the new 07/08/2009 build date release of rushtop, let me know! -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1860 ***************************************************** *** ***************************************************** Subject: Rushtop: autostarting in KDE From: Greg Ercolano Date: Fri, 10 Jul 2009 13:41:30 -0400 Starting a new thread for this.. Stephan Kosinski wrote: > A quick question though since I propagated the new binary on my linux > stations rushtop is auto-starting at user login. Hmm, nothing in rushtop or the rush installers does any autostart type stuff.. at least not in my code. Possibly the GUI toolkit rushtop is using has changed some internals that makes it more compatible with window manager autostart stuff that it didn't have before. Do other tools autostart if you logout while they're running? ie. firefox, konqueror, clocks, etc? Sounds like a feature of the window manager remembering what was running when you logout, and starting them again when you login. This stuff is likely saved in an ascii file in your ~/.kde directory, but by what mechanism I don't know. > What is reaaly weird is that I was never able to do that with previous > binary even by saving the KDE session with rushtop running. Hmm, I don't know what could have changed in rushtop to do that. Nothing in the code to rushtop itself.. I don't know of any way to control it from the GUI toolkit's API. > Do you have any idea how to prevent rushtop to auto-start? Is it > related to my system or to the new binary? Try closing rushtop *then* logging out, then back in. I would assume it won't start then. I stopped using KDE a while ago; too many memory hungry daemons and menus kept moving around, and same to be said for gnome, so I standardized on a custom 'tiny' window manager that has no daemons and one menu that I defined. This way I can upgrade my workstation, but have the interface stay the same. However, I'll try bringing up KDE on one of my other systems to see if I can at least replicate. But I figure KDE's own docs on autostart stuff will give reason to how to control autostarting, and possibly even specifying exclusion terms. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1859 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Stephan Kosinski Date: Fri, 10 Jul 2009 13:10:39 -0400 Hello Greg, Thank you very much for the new version. It really is convenient!.. A quick question though since I propagated the new binary on my linux stations rushtop is auto-starting at user login. What is reaaly weird is that I was never able to do that with previous binary even by saving the KDE session with rushtop running. Do you have any idea how to prevent rushtop to auto-start? Is it related to my system or to the new binary? Thank you for your help and all your work. Stephan. Greg Ercolano wrote: > Abraham Schneider wrote: >> [..describes 'gray gap' problem and parenthesis disappearing..] > > OK, I've believe I've fixed both of those issues, and have posted > the new binaries here: > > http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort-07-08-2009.zip > > (Same download password as before) > > This version advertises itself with a build date of 07/08/2009 > in the 'rushtop -help' screen. > > Let me know if you encounter any further trouble. *** ***************************************************** *** Message #1858 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: "Abraham Schneider" Date: Fri, 10 Jul 2009 11:26:11 -0400 Works fine for me, thanks alot! Abraham Greg Ercolano schrieb: > [posted to rush.general] >=20 > Abraham Schneider wrote: >> [..describes 'gray gap' problem and parenthesis disappearing..] >=20 > OK, I've believe I've fixed both of those issues, and have posted > the new binaries here: >=20 > = http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort-07-= 08-2009.zip >=20 > (Same download password as before) >=20 > This version advertises itself with a build date of 07/08/2009 > in the 'rushtop -help' screen. >=20 > Let me know if you encounter any further trouble. -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Dr. Martin Prillmann; Thomas Till *** ***************************************************** *** Message #1857 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Wed, 08 Jul 2009 15:32:42 -0400 Abraham Schneider wrote: > [..describes 'gray gap' problem and parenthesis disappearing..] OK, I've believe I've fixed both of those issues, and have posted the new binaries here: http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort-07-08-2009.zip (Same download password as before) This version advertises itself with a build date of 07/08/2009 in the 'rushtop -help' screen. Let me know if you encounter any further trouble. *** ***************************************************** *** Message #1856 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Tue, 07 Jul 2009 12:55:24 -0400 Abraham Schneider wrote: > Sorry for not giving feedback sooner, it was a hard week here. Thanks > for doing this so fast and nice! Seems to work quite well. > > Only thing that isn't working for me with the new build is that after > restarting the app the number in parentheses which signals the core > count isn't showing anymore. With the old build it worked just fine. > (Talking about rushtop on a Mac Pro with 10.4). Strange thing is: if I > switch on the option the paranthesis appear, but the layout of the lines > is strange because I have a big grey gap after the first 5 lines. If I > quit and restart the app the gap is gone, but the parantheses as well. > Tell me if I should send you screenshots or test something. Thanks Abraham -- I'll look into those and will follow up here with fix binaries. > BTW: seems you have read my mind there, because after sending you the > original post I thought it would be nice to still see the amount of > cores somewhere. Then you have send the first build and it was already > in there :) Yes; I felt you had to see that info somehow.. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1855 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: "Abraham Schneider" Date: Tue, 07 Jul 2009 07:10:24 -0400 Hi Greg! Sorry for not giving feedback sooner, it was a hard week here. Thanks=20 for doing this so fast and nice! Seems to work quite well. Only thing that isn't working for me with the new build is that after=20 restarting the app the number in parentheses which signals the core=20 count isn't showing anymore. With the old build it worked just fine.=20 (Talking about rushtop on a Mac Pro with 10.4). Strange thing is: if I=20 switch on the option the paranthesis appear, but the layout of the lines = is strange because I have a big grey gap after the first 5 lines. If I=20 quit and restart the app the gap is gone, but the parantheses as well.=20 Tell me if I should send you screenshots or test something. BTW: seems you have read my mind there, because after sending you the=20 original post I thought it would be nice to still see the amount of=20 cores somewhere. Then you have send the first build and it was already=20 in there :) Thanks again and best regards, Abraham Greg Ercolano schrieb: > [posted to rush.general] >=20 > I have an update for the rushtop with the single cpu bar mode; >=20 > Dylan noticed that if "Use Decay" mode is on, all the bars = 'pulsate' > instead of bouncing smoothly. >=20 > I hadn't noticed this, because I had been testing with "Use Decay" > mode turned off (right click, choose Properties -> Use Decay to = toggle) > When disabled, the bars act normally. >=20 > Anyway, I just uploaded a new zip file with a fix for this: > = http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort-07-= 01-2009.zip >=20 > This new version advertises its build date as being '07/01/09' > when rushtop is invoked with the '-help' flag, eg: >=20 > rushtop -help > [..] > COPYRIGHT > Rush(R) Network Render Queue, V102.42b-prerelease, Build Date: = 07/01/09 > = ^^^^^^^^ >=20 >> Greg Ercolano wrote: >> Done -- give this a try: >> = http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort.zip= >> (Use your downloads password. If you don't know what it is, = contact me.) -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Dr. Martin Prillmann; Thomas Till *** ***************************************************** *** Message #1854 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Wed, 01 Jul 2009 04:23:23 -0400 I have an update for the rushtop with the single cpu bar mode; Dylan noticed that if "Use Decay" mode is on, all the bars 'pulsate' instead of bouncing smoothly. I hadn't noticed this, because I had been testing with "Use Decay" mode turned off (right click, choose Properties -> Use Decay to toggle) When disabled, the bars act normally. Anyway, I just uploaded a new zip file with a fix for this: http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort-07-01-2009.zip This new version advertises its build date as being '07/01/09' when rushtop is invoked with the '-help' flag, eg: rushtop -help [..] COPYRIGHT Rush(R) Network Render Queue, V102.42b-prerelease, Build Date: 07/01/09 ^^^^^^^^ > Greg Ercolano wrote: > Done -- give this a try: > http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort.zip > (Use your downloads password. If you don't know what it is, contact me.) *** ***************************************************** *** Message #1853 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Fri, 26 Jun 2009 20:44:43 -0400 Oh, and I should add how it works ;) Just right click on the new rushtop interface, and you should see: Preferences/Single Cpu Bar Per Host Also, if you run 'rushtop -help', you'll see the new "-single" and "-multi" command line flags to enable/disable this feature. eg: rushtop -single rushtop -multi Note that once turned on, rushtop remembers the setting for the next time rushtop is invoked (the setting is saved in the ~/.rushtoprc file). So if no flags are specified, the last setting is used. Note that when single mode is enabled, multiproc machines show the actual number of cpus in parens next to the hostname. There's also a new -nosort flag (which you didn't ask for, but another customer did) that disables sorting the hostnames in alphabetical order, so the hosts appear in rushtop in the order they appear in the rush/etc/hosts file, or in the order they appear on the command line, eg: rushtop -nosort zenith sony toshiba rca +farm If you have any trouble with the above, let me know. This version of rushtop SHOULD be compatible with recent versions of rush. If you have trouble, revert to the old rushtop (you did make a copy of the original rushtop, right? ;) > Greg Ercolano wrote: > Done -- give this a try: > http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort.zip > > 2009/6/26 Abraham Schneider : >>>> Is there any way to have Rushtop to only show one line per machine with >>>> a summary value for the CPU load instead of showing a bar for each individual core? -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1852 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Fri, 26 Jun 2009 20:36:33 -0400 Greg Ercolano wrote: > Sure, I can add that. Done -- give this a try: http://www.seriss.com/rush/releases/patches/rushtop-single-and-nosort.zip (Use your downloads password. If you don't know what it is, contact me.) That zip contains new rushtop executables for mac/linux/windows: mac/rushtop -- for all versions of OSX (intel/ppc) windows/rushtop.exe -- for all versions of windows fedora3/rushtop -- for most 32bit and 64bit versions of Linux ubuntu7-64/rushtop -- Use this if you've got a 64bit linux the fedora3 binary doesn't work with You can either use the install scripts in the zip file to install, or just copy the appropriate binary for your platform into the appropriate rush directory (rename out the old one first): WIN: c:\rush\bin\rushtop.exe MAC: /usr/local/rush/Applications/rushtop.app/Contents/MacOS/rushtop LINUX: /usr/local/rush/bin/rushtop Note that on the Mac the rushtop binary is NOT in rush/bin, it's in the .app directory shown above. The 'rushtop' you'll find in the rush/bin directory is a *script*, so don't replace that. Greg Ercolano wrote: > Sure, I can add that. > > I'll see if I can pull together some new 'rushtop' executables > that do this for you, and make it available on the website. 2009/6/26 Abraham Schneider : > >> Hi there! > >> > >> Is there any way to have Rushtop to only show one line per machine with > >> a summary value for the CPU load instead of showing a bar for each > >> individual core? -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1851 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Greg Ercolano Date: Fri, 26 Jun 2009 10:28:22 -0400 Sure, I can add that. I'll see if I can pull together some new 'rushtop' executables that do this for you, and make it available on the website. Dylan Penhale wrote: > [posted to rush.general] > I would second this request. > > Dylan Penhale > > 2009/6/26 Abraham Schneider : >> Hi there! >> >> Is there any way to have Rushtop to only show one line per machine with >> a summary value for the CPU load instead of showing a bar for each >> individual core? -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1850 ***************************************************** *** ***************************************************** Subject: Re: Rushtop: view with cpu summary instead of each single core? From: Dylan Penhale Date: Fri, 26 Jun 2009 07:17:50 -0400 I would second this request. Dylan Penhale 2009/6/26 Abraham Schneider : > [posted to rush.general] > > Hi there! > > Is there any way to have Rushtop to only show one line per machine with > a summary value for the CPU load instead of showing a bar for each > individual core? > > I'd really like to see my whole renderfarm on a screen without scrolling > like hell. > > In earlier times with only one or two cores per machine it was fine. But > now with Nehalem-based Mac Pros with 16 virtual cores, it's hard to keep > track of the status of the whole farm. > > Thanks for any help and best regards, > > Abraham > > -- > Abraham Schneider Dylan Penhale 6 Bennabra Place Frenchs Forest NSW 2086 Australia Mobile: 0424655320 Email: dylan@(email surpressed) Web: www.his-best.com *** ***************************************************** *** Message #1849 ***************************************************** *** ***************************************************** Subject: Rushtop: view with cpu summary instead of each single core? From: "Abraham Schneider" Date: Fri, 26 Jun 2009 04:37:53 -0400 Hi there! Is there any way to have Rushtop to only show one line per machine with a summary value for the CPU load instead of showing a bar for each individual core? I'd really like to see my whole renderfarm on a screen without scrolling like hell. In earlier times with only one or two cores per machine it was fine. But now with Nehalem-based Mac Pros with 16 virtual cores, it's hard to keep track of the status of the whole farm. Thanks for any help and best regards, Abraham -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Dr. Martin Prillmann; Thomas Till *** ***************************************************** *** Message #1848 ***************************************************** *** ***************************************************** Subject: Re: -b maya flag [MAYA 2009 UPDATE: FLOATING POINT RENDERING WITH From: Greg Ercolano Date: Tue, 09 Jun 2009 03:38:01 -0400 Replying to this old thread to update regarding rendering floating point frame ranges with Maya 2009 and up. In Maya 2009 they dropped the -se/-be flags that the submit-maya script used to handle floating point frame numbers. So if you try to render floating point frame numbers with Maya 2009, you'll get an error about the '-se' flag being uknown. So if you're using Maya 2009 with floating point frames and getting that error, find this line in the submit-maya.pl script: $opt{MayaFlags} .= "-se $se -be $be "; ..and change it to read: $opt{MayaFlags} .= "-rfs $se -rfb $be "; ..and that should solve it. [THIS FIX IS IN RUSH 102.42a9b AND UP] Greg Ercolano wrote: > Gary Jaeger wrote: >> Is there any way using rush to use a step rate with maya renders? I >> commonly render out slow versions of animations as tests using -b .5, >> etc. That doesn't work with rush. > > The new submit-maya6 script in 102.42a6 supports both > floating point frame ranges, and stepped frame ranges > when batching. > > Did you upgrade to 102.42a6 already? > If not, contact me directly and I'll supply you the upgrade info, > which is free to all existing customers. > > If so, open the submit-maya6 interface, and click the "Help" > button at the bottom, and scroll to the section on > "Maya Floating Point Rendering", eg: > > Maya Floating Point Rendering > ----------------------------- > Here's an example of rendering with floating point frames. > Maya has the capability of rendering the frames 'between frames', > sometimes useful for motion blurring, or slowing the animation > of a sequence down by rendering images between frames: > > Job Title: -- (Blank uses scene name) > Scene Path: //path/scenes/sc17a.ma -- Scene file to render > Renderer: maya(sw) -- Use Maya's software renderer > Threads: 1 -- One thread for each frame > Frames: 10.0-50.0,0.2 -- Render 10.0 - 50.0 stepping at 0.2 > Batch Frames: 1 -- Renders 1 frame per machine > Cpus: +any=5 -- Use any 5 available cpus > > > This will create a job that generates a floating point frame list, > where Maya will write out 'integerized' frame numbers (with the > decimal points removed) that looks like eg: > > Rush Frame Output Image > ---------- ------------ > 0010.0 foo.0100.iff > 0010.2 foo.0102.iff > 0010.4 foo.0104.iff > : > 0049.6 foo.0496.iff > 0049.8 foo.0498.iff > 0050.0 foo.0500.iff > > > When 'batching' is used, several floating point frames will be > rendered at a time, eg: > > Frames: 10.0-50.0,0.2 -- Render 10.0 - 50.0 stepping at 0.2 > Batch Frames: 5 -- Renders 5 frame per machine > > > .this will create a "batched frames" job that generates a frame list > for every 5th floating point frame, each of which will render 5 frames > at a time on each machine: > > Rush Scene > Frame Frame Output Image > -------- _ ------- ------------- > | 10.0 foo.0100.iff > | 10.2 foo.0102.iff > 0010.0 --| 10.4 foo.0104.iff > | 10.6 foo.0106.iff > |_ 10.8 foo.0108.iff > _ > | 11.0 foo.0110.iff > | 11.2 foo.0112.iff > 0011.0 --| 11.4 foo.0114.iff > | 11.6 foo.0116.iff > |_ 11.8 foo.0118.iff > : > : _ > | 49.0 foo.0490.iff > | 49.2 foo.0492.iff > 0049.0 --| 49.4 foo.0494.iff > | 49.6 foo.0496.iff > |_ 49.8 foo.0498.iff > _ > 0050.0 --|_ 50.0 foo.0500.iff > > *** ***************************************************** *** Message #1847 ***************************************************** *** ***************************************************** Subject: [OSX/ADMIN] Boot script to mount multiple file servers From: Greg Ercolano Date: Wed, 03 Jun 2009 00:53:06 -0400 Sometimes companies have multiple file servers/shares they want to have mounted up on boot of their OSX machines. The following shows a boot script for OSX that uses subroutines to handle the details of mounting, to make it easier to manage multiple mounts, so that there's just one mount command per line, eg: Mount 192.168.1.2:/jobs /net/jobs "intr,bg,resvport,vers=3" Mount 192.168.1.3:/Volumes/Prod1 /net/prod1 "intr,bg,tcp,vers=3" Mount 192.168.1.3:/Volumes/Prod2 /net/prod2 "intr,bg,tcp,vers=3" Mount 192.168.1.3:/Volumes/Prod3 /net/prod3 "intr,bg,tcp,vers=3" This is better for multiple mounts than the previous examples I posted last year, where to mount multiple servers you needed to have a paragraph of code for each mount. (Use of IP addresses is recommended in place of hostnames, as DNS is often unstable under OSX during boot.) The following describes how to create and install this boot script which, once working, can be copied to all your OSX machines to ensure each boots up with the correct mounts. * * * 1) Login as root, create a boot script directory called "MountFileServers": mkdir -m 755 /Library/StartupItems/MountFileServers chown 0:0 /Library/StartupItems/MountFileServers It's important the perms are 755 and owned by root/wheel, as shown. 2) 'cd' into that directory and create a "StartupParameters.plist" file: with the perms carefully set to 644 and owned by root/wheel: cd /Library/StartupItems/MountFileServers touch StartupParameters.plist chown 0:0 StartupParameters.plist chmod 644 StartupParameters.plist vi StartupParameters.plist Here's what to paste into that file (leave out the 'snip' lines!). --------------------------------------------------------- snip { Description = "Mounts the File Servers"; Provides = ("MountFileServers"); Requires = ("Network", "System Log", "Resolver"); OrderPreference = "Last"; Messages = { start = "Mounting local file server"; stop = "Unmounting local file server"; }; } --------------------------------------------------------- snip 3) While you're in that same directory, create an executable script called "MountFileServers", with the perms 755, root/wheel: cd /Library/StartupItems/MountFileServers touch MountFileServers chown 0:0 MountFileServers chmod 755 MountFileServers vi MountFileServers What follows is the contents of the "MountFileServers" script. Be sure to customize the "Mount" and "Umount" lines to suit your environment. When specifying mounts, use IP addresses, not hostnames. eg: Mount helium:/jobs /net/jobs "intr,bg" # BAD ^^^^^^ Mount 192.168.0.2:/jobs /net/jobs "intr,bg" # GOOD ^^^^^^^^^^^ Here's the script: --------------------------------------------------------- snip #!/bin/sh ### ### Mount File Servers -- Set up local mounts for our file server ### ### Save this file as /Library/StartupItems/MountFileServers/MountFileServers ### and make sure the files/dirs are owned by root/wheel and modes are 755 ### (rwx-r-xr-x). ### ### 05/27/2009 erco@(email surpressed) ### # SUBROUTINE: MOUNT A REMOTE DRIVE # # NOTE: Use ip addresses in the remote device name, do NOT use hostnames. # This avoids assuming DNS fully operational during boot. # Often DNS is unstable during boot, even with boot dependencies. # Mount() { remdrive="$1" # $1 -- the remote drive, eg. "192.168.1.10:/jobs" localdir="$2" # $2 -- local mount point, eg. "/net/jobs" nfsflags="$3" # $3 -- nfs flags, eg. "rw,intr,bg,vers=3" # ENSURE MOUNT POINT EXISTS if [ ! -d "$localdir" ]; then $LOGGER "Creating mount point $localdir" mkdir -p -m 755 "$localdir" fi # ALREADY MOUNTED? DONT MOUNT TWICE if mount | grep -q "$localdir"; then $LOGGER "$localdir already mounted" else $LOGGER "Mounting '$remdrive' on '$localdir'" ( mount -t nfs -o "$nfsflags" "$remdrive" "$localdir" ) 2>&1 | $LOGGER fi } # SUBROUTINE: UNMOUNT THE SPECIFIED DRIVE Umount() { localdir="$1" # $1 -- local mount point, eg. "/net/jobs" $LOGGER "Un-mounting '$localdir'" umount "$localdir" 2>&1 | $LOGGER } ### ### MAIN ### if [ -x /etc/rc.common ]; then . /etc/rc.common; fi export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/libexec if [ "`tty`" = "not a tty" ]; then LOGGER="logger -t MountFileServers" else LOGGER="logger -s -t MountFileServers"; fi # Handle stop/start/restart ARG1=${1:-start} # assume 'start' if unset case "$ARG1" in start) ### MOUNT ALL SERVERS ON BOOT Mount 192.168.1.2:/jobs /net/jobs "intr,bg,resvport,vers=3" # LINUX Mount 192.168.1.3:/Volumes/Prod1 /net/prod1 "intr,bg,tcp,vers=3" # OSX/TCP Mount 192.168.1.3:/Volumes/Prod2 /net/prod2 "intr,bg,tcp,vers=3" # OSX/TCP Mount 192.168.1.3:/Volumes/Prod3 /net/prod3 "intr,bg,tcp,vers=3" # OSX/TCP Mount 192.168.1.3:/Volumes/STOCK /net/stock "intr,bg,tcp,vers=3" # OSX/TCP Mount 192.168.1.4:/Volumes/admin /net/admin "intr,bg,vers=3" # OSX Mount 192.168.1.4:/Volumes/personal /net/personal "intr,bg,vers=3" # OSX ;; stop) ### UN-MOUNT ALL SERVERS ON SHUTDOWN Umount /net/jobs Umount /net/prod1 Umount /net/prod2 Umount /net/prod3 Umount /net/stock Umount /net/admin Umount /net/personal ;; restart) $0 stop $0 start ;; *) echo "usage: $0 {start|stop|restart}" exit 1 ;; esac exit 0 --------------------------------------------------------- snip 4) That's it. You can test if the script works by running it as root from your terminal with the start/stop options. To test the script to see that it mounts properly: /Library/StartupItems/MountFileServers/MountFileServers start ..and to test that it unmounts properly, run: /Library/StartupItems/MountFileServers/MountFileServers stop After each command, run 'mount' to verify the mounts appear and disappear correctly. Once working, test by actually rebooting the machine, and when you get a login, check the /var/log/system.log to make sure there are no errors, eg: grep MountFileServers /var/log/system.log This is where any errors will be from your mount commands would appear, in case there's errors. Then you can install the boot script directory on the other OSX machines with a simple scp command, eg: scp -rp /Library/StartupItems/MountFileServers HOST1:/Library/StartupItems/MountFileServers scp -rp /Library/StartupItems/MountFileServers HOST2:/Library/StartupItems/MountFileServers ...etc.. Replace "HOST1" and "HOST2" with the remote hostnames. The above has been tested to work equally well on both Tiger and Leopard. *** ***************************************************** *** Message #1846 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Tue, 02 Jun 2009 14:47:29 -0400 Craig Allison wrote: > Ok, the retiming of the slate didn't work, Shake still insisted on > rendering the slate/grey over and over again as it rendered the quicktime. > > Have used to following to submit the two scripts from one submit: > > my $submit = <<"EOF"; > title $in{JobTitle} > frames 1 > command perl $G::self -render $QT_script_to_render > $in{Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} > $in{PrintEnvironment} $in{Frames} $in{Production} $in{OutputQuicktime} > $in{ShakeFlags} > $submitoptions > EOF > > my $submit2 = <<"EOF"; > title $in{JobTitle} > frames 1 > command perl $G::self -render $Slate_script_to_render > $in{Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} > $in{PrintEnvironment} $in{ShakeFlags} > $submitoptions > EOF > > # ErrorWindow("$submit\n"); # DEBUGGING > RushSubmit($submithost, $submit2, $in{JobTitle}, $in{StartIrush}); > exit(RushSubmit($submithost, $submit, $in{JobTitle}, $in{StartIrush})); > > This works fine, only that when the second job starts, the log for > $submit2 is overwritten as both jobs share the same directory, not such > a problem, but may try to iron this out over the coming days. This is because both submits are using the same $submitoptions: my $submit = <<"EOF"; .. $submitoptions .. EOF my $submit2 = <<"EOF"; .. $submitoptions .. EOF If you look at the contents of the $submitoptions variable, I think you'll understand why you're getting so many settings common between the Shake and QT jobs. In this case, the $submitoptions contains the 'logdir' and 'frames' command from the shake job, and passing that to both jobs is what's causing the overlap. ie. thats why the frame range and logdir are coming up the same for both. What you'll want to do is /not/ include the $submitoptions variable in the second job, and just set the frames/logdir commands you want, eg: ### CREATE LOGDIR FOR QT JOB my $qtlogdir = "${Slate_script_to_render}.log"; if ( ! -d $qtlogdir ) { mkdir($qtlogdir, 0777); if ( $G::iswindows ) { my $dir = $qtlogdir; $dir =~ s%/%\\%g; system("cacls $dirname /e /c /g everyone:f"); } } ### SUBMIT THE QT JOB my $submit2 = <<"EOF"; title $in{JobTitle}_QT frames 1 command perl $G::self -render $Slate_script_to_render .. logdir $qtlogdir EOF BTW, if you want to see what the first job's $submitoptions is set to, you can stick an 'ErrorWindow();' command in there just above the first submit, eg: ErrorWindow("SUBMITOPTIONS IS SET TO:\n$submitoptions\n"); my $submit = <<"EOF"; .. This way when you submit a test job, you'll see what the variable is set to, so you can then determine what you'll need to pass in the second job, in case there's more than just 'frames' and 'logdir'. > As an aside does anyone know a quick way of always resetting to defaults > for the QT render instead of Rush picking up the last settings every > time? I assume it's a deletion of a certain local file, but maybe I > could just reset certain fields which would work much better? I'm thinking my answers above should help with that. Basically its that same "$submitoptions" variable being passed to both jobs. Let us know if you have any other questions. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1845 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Craig Allison Date: Tue, 02 Jun 2009 06:24:41 -0400 --Apple-Mail-23--656878284 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Ok, the retiming of the slate didn't work, Shake still insisted on rendering the slate/grey over and over again as it rendered the quicktime. Have used to following to submit the two scripts from one submit: my $submit = <<"EOF"; title $in{JobTitle} frames 1 command perl $G::self -render $QT_script_to_render $in {Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} $in {PrintEnvironment} $in{Frames} $in{Production} $in{OutputQuicktime} $in{ShakeFlags} $submitoptions EOF my $submit2 = <<"EOF"; title $in{JobTitle} frames 1 command perl $G::self -render $Slate_script_to_render $in {Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} $in {PrintEnvironment} $in{ShakeFlags} $submitoptions EOF # ErrorWindow("$submit\n"); # DEBUGGING RushSubmit($submithost, $submit2, $in{JobTitle}, $in{StartIrush}); exit(RushSubmit($submithost, $submit, $in{JobTitle}, $in {StartIrush})); This works fine, only that when the second job starts, the log for $submit2 is overwritten as both jobs share the same directory, not such a problem, but may try to iron this out over the coming days. As an aside does anyone know a quick way of always resetting to defaults for the QT render instead of Rush picking up the last settings every time? I assume it's a deletion of a certain local file, but maybe I could just reset certain fields which would work much better? I can see that I'm going to run in to trouble with artists not choosing the correct production and frame range etc so would like to force them into a choice each time. I will get them to start using "forms" too, I hadn't noticed that nice little feature until I was developing this, will come in useful for artists who are always working on the same shots. C Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 27 May 2009, at 18:11, Victor DiMichina wrote: > You can try something like this, suited to your needs of course. > Note the eval command (thanks go to Greg again and again, of course) > > ################################ > #!/bin/csh -f > if ( ! $?RUSH_DIR ) setenv RUSH_DIR /usr/local/rush > source $RUSH_DIR/etc/.submit > eval `/usr/local/rush/bin/rush -submit` << EOF > title (your title) > ram 1 > frames (your frame range) > logdir (your log dir) > command perl //server/raid/2D_Apps/RushSubmits/Applications/ > submit-shake.app/Contents/MacOS/submit-shake-autolog.pl -render 1 1 > Fail /server/raid/path/to/someshakeproject.shk 0 -v -proxyscale 1 -v > cpus +farm=10@10 > notes (optional) > EOF > if ( $status ) exit 1 > # (eval eats the setenv command, so we duplicate it here) > echo "setenv RUSH_JOBID $RUSH_JOBID" > echo $RUSH_JOBID > /tmp/curr_jobid.txt > # Job #2 - > /usr/local/rush/bin/rush -submit << EOF > title (your 2nd title) > ram 1 > frames (frame range) > logdir (your log dir) > command (another command) > cpus +farm=11@980 > waitfor $RUSH_JOBID > waitforstate done > notes (optional) > EOF > #################################################### > > > HTH > Vic > > > On Wed, May 27, 2009 at 7:54 AM, Craig Allison > wrote: > Ok, so I've finally got around to looking into this and have > decided to change my original way of thinking and give the artists > the opportunity to render a quicktime on the farm once they are > happy with the results previewed in Framecycler. > > I have taken the submit-shake-quicktime script and amended it to > allow users to select which film they are working on, select the > first frame in the sequence to render and dialogue for slate notes > etc - the combination of all this information chooses the correct > shake setup script on the network, replaces the variables, saves a > copy and submits it to the farm. > > The script renders a 2K slate & greyscale (named and numbered > correctly) and also a QT with the appropriate settings for the > production. I have it working fine but now want to speed it up by > splitting the job into two, i.e. one script to render the two 2k > frames and one script to render the QT. > > I've got rush creating the 2 distinct shake scripts to render, but > don't know how to get 2 scripts to render with a single rush > submit, is there an easy way? > > Cheers > > Craig > > Craig Allison > Digital Systems & Data Manager > The Senate Visual Effects > Twickenham Film Studios > St.Margarets > Twickenham > TW1 2AW > > t: 0208 607 8866 > skype: craig_9000 > www.senatevfx.com > > > > On 19 Dec 2008, at 09:20, Abraham Schneider wrote: > >> [posted to rush.general] >> >> Greg Ercolano schrieb: >>> Sounds interesting, but unless they have a non-gui interface, >>> it's not something I think Rush can control properly. >> >> yes, that's right. I'm doing it in two steps at the moment: rush=20 >> executes a jobdonecommand script which creates an entry with the >> path to = >> >> the image sequence in a queue folder. A small python script on my >> QT=20 >> creation Mac is watching this folder and does the actual 'feeding' >> of=20 >> the Silverstack rendering app (called Cinemator). >> >> That's not a perfect solution but it works quite well for us at >> the=20 >> moment. And with the template files it is easy to do different >> burnins,=20 >> codecs, framerates, etc. depending on project or user choose. >> >> Abraham >> >> -- >> Abraham Schneider >> Senior VFX Compositor, 2 D Artist >> ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = >> Germany >> Phone (Tel# suppressed) EMail aschneider@arri.de >> www.arri.de/filmtv >> ARRI Film & TV Services GmbH >> Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = >> Handelsregisternummer: HRB 69396 >> Gesch=E4ftsf=FChrer: Franz Kraus; Thomas Till >> > > > > > -- > Victor DiMichina > Pixel Magic > 818.760.0862 --Apple-Mail-23--656878284 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Ok, the retiming of the slate didn't work, Shake still insisted on = rendering the slate/grey over and over again as it rendered the = quicktime.

Have = used to following to submit the two scripts from one = submit:

my $submit =3D = <<"EOF";
title =A0 =A0 =A0 = =A0$in{JobTitle}
frames =A0 =A0 =A0 =A0 =A01
command = =A0 =A0 =A0 =A0 perl $G::self -render $QT_script_to_render = $in{Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} = $in{PrintEnvironment} $in{Frames} $in{Production} $in{OutputQuicktime} = $in{ShakeFlags}=A0
$submitoptions
EOF

my = $submit2 =3D <<"EOF";
title =A0 =A0 =A0 = =A0$in{JobTitle}
frames =A0 =A0 =A0 =A0 =A01
command = =A0 =A0 =A0 =A0 perl $G::self -render $Slate_script_to_render = $in{Verbosity} $in{Retries} $in{RetryBehavior} $in{MaxLogSize} = $in{PrintEnvironment} = $in{ShakeFlags}=A0
$submitoptions
EOF
=
=A0# = ErrorWindow("$submit\n"); # DEBUGGING
=A0=A0 = =A0RushSubmit($submithost, $submit2, $in{JobTitle}, = $in{StartIrush});
=A0=A0 =A0exit(RushSubmit($submithost, = $submit, $in{JobTitle}, $in{StartIrush}));



This works fine, only that when = the second job starts, the log for $submit2 is overwritten as both jobs = share the same directory, not such a problem, but may try to iron this = out over the coming days.


As an aside does anyone know a quick way of always resetting to = defaults for the QT render instead of Rush picking up the last settings = every time? I assume it's a deletion of a certain local file, but maybe = I could just reset certain fields which would work much better? =A0I can = see that I'm going to run in to trouble with artists not choosing the = correct production and frame range etc so would like to force them into = a choice each time. =A0I will get them to start using "forms" too, I = hadn't noticed that nice little feature until I was developing this, = will come in useful for artists who are always working on the same = shots.


C


Craig Allison

Digital Systems & Data=A0Manager

The Senate Visual = Effects

St.Margarets

Twickenham

TW1 2AW


t: 0208 607 8866=A0

skype: craig_9000


=


On 27 May 2009, at 18:11, Victor DiMichina = wrote:

You can try something like this,=A0 suited to your needs = of course.=A0=A0 Note the eval command (thanks go=A0 to Greg again and = again,=A0 of = course)

################################
#!/bin/csh -f
if ( = ! $?RUSH_DIR ) setenv RUSH_DIR /usr/local/rush
source = $RUSH_DIR/etc/.submit
eval `/usr/local/rush/bin/rush -submit` = << EOF
title=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your = title)
ram=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1
frames=A0=A0=A0=A0=A0= =A0=A0=A0=A0 (your frame range)
logdir=A0=A0=A0=A0=A0=A0=A0=A0=A0 = (your log dir)
command=A0=A0=A0=A0=A0=A0=A0 perl = //server/raid/2D_Apps/RushSubmits/Applications/submit-shake.app/Contents/M= acOS/submit-shake-autolog.pl -render 1 1 Fail=A0 = /server/raid/path/to/someshakeproject.shk=A0 0 -v -proxyscale 1 -v
= cpus=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 +farm=3D10@10
notes=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 (optional)
EOF
if ( $status ) exit 1
# (eval = eats the setenv command, so we duplicate it here)
echo "setenv = RUSH_JOBID $RUSH_JOBID"
echo $RUSH_JOBID > /tmp/curr_jobid.txt
= # Job #2 -
/usr/local/rush/bin/rush -submit << = EOF
title=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your 2nd = title)
ram=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1
frames=A0=A0=A0=A0=A0= =A0=A0=A0=A0 (frame range)
logdir=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your = log dir)
command=A0=A0=A0=A0=A0=A0=A0=A0 (another command)
= cpus=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 +farm=3D11@980
waitfor=A0=A0=A0=A0= =A0=A0=A0=A0 $RUSH_JOBID
waitforstate=A0=A0=A0 = done
notes=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = (optional)
EOF
####################################################<= br>

HTH
Vic


On Wed, May = 27, 2009 at 7:54 AM, Craig Allison <craig@(email surpressed)> = wrote:
Ok, so I've finally got around to looking into = this and have decided to change my original way of thinking and give the = artists the opportunity to render a quicktime on the farm once they are = happy with the results previewed in Framecycler.

I = have taken the submit-shake-quicktime script and amended it to allow = users to select which film they are working on, select the first frame = in the sequence to render and dialogue for slate notes etc - the = combination of all this information chooses the correct shake setup = script on the network, replaces the variables, saves a copy and submits = it to the farm.

The script renders a 2K slate = & greyscale (named and numbered correctly) and also a QT with the = appropriate settings for the production. =A0I have it working fine but = now want to speed it up by splitting the job into two, i.e. one script = to render the two 2k frames and one script to render the QT.
=

I've got rush creating the 2 distinct shake scripts = to render, but don't know how to get 2 scripts to render with a single = rush submit, is there an easy way?

Cheers
=

Craig

Craig Allison
Digital Systems = & Data=A0Manager
The Senate Visual Effects
Twickenham Film = Studios
St.Margarets
Twickenham
TW1 = 2AW

t: 0208 607 = 8866=A0
skype: = craig_9000


=

On 19 Dec 2008, at 09:20, Abraham Schneider = wrote:

[posted to rush.general]

Greg Ercolano = schrieb:
Sounds = interesting, but unless they have a non-gui interface,
= it's not something I think Rush can control properly.
=

yes, that's right. I'm = doing it in two steps at the moment: rush=3D20
executes a jobdonecommand script which creates an entry with the = path to =3D

the image sequence in a = queue folder. A small python script on my QT=3D20
creation Mac is watching this folder and does the = actual 'feeding' of=3D20
the = Silverstack rendering app (called Cinemator).

That's not = a perfect solution but it works quite well for us at the=3D20
moment. And with the template files it is easy to = do different burnins,=3D20
codecs, = framerates, etc. depending on project or user choose.

Abraham

--
Abraham Schneider
Senior VFX = Compositor, 2 D Artist
ARRI Film & = TV Services GmbH =A0 Tuerkenstr. 89 =A0 = D-80799 Muenchen / =3D
Germany
Phone +49 89 3809-1269 = =A0 =A0 EMail aschneider@arri.de
ARRI Film & TV Services GmbH
Sitz: M=3DFCnchen=A0 -=A0 = Registergericht: Amtsgericht M=3DFCnchen=A0 -=A0= =3D
Handelsregisternummer: HRB = 69396
Gesch=3DE4ftsf=3DFChrer: Franz = Kraus; Thomas Till


=



-- =
Victor DiMichina
Pixel = Magic
818.760.0862

= --Apple-Mail-23--656878284-- *** ***************************************************** *** Message #1844 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Craig Allison Date: Thu, 28 May 2009 04:53:53 -0400 --Apple-Mail-13-1053132520 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hey Greg The way my Shake script works is that it renders the QT without the need for the slate and grey to be generated as 2k, it files them in from a set location and adds the relevant text and overlays a mask with counter and shot details, plus it files in the same "empty" slate, adds the same text and has another fileout to render the single 2k frame. The greyscale that is added to the QT has a branched fileout to also render a single 2k grey at the end. The problem I have observed is that the 2k slate renders itself over and over again until the script reaches the end of it's frame count, i.e. for a 200 frame QT, the slate will render 200 times, hence the need for splitting up the job so that I can give the slate and grey globals of 1 and the QT 1-200 for example. I'm going to look into using an offset for the slate/grey to see if this addresses this issue, otherwise I'll look at Victor's suggestion. Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 27 May 2009, at 22:21, Greg Ercolano wrote: > [posted to rush.general] > > Craig Allison wrote: >> Ok, so I've finally got around to looking into this and have >> decided to >> change my original way of thinking and give the artists the >> opportunity >> to render a quicktime on the farm once they are happy with the >> results >> previewed in Framecycler. >> >> I have taken the submit-shake-quicktime script and amended it to >> allow >> users to select which film they are working on, select the first >> frame >> in the sequence to render and dialogue for slate notes etc - the >> combination of all this information chooses the correct shake setup >> script on the network, replaces the variables, saves a copy and >> submits >> it to the farm. >> >> The script renders a 2K slate & greyscale (named and numbered >> correctly) >> and also a QT with the appropriate settings for the production. I >> have >> it working fine but now want to speed it up by splitting the job into >> two, i.e. one script to render the two 2k frames and one script to >> render the QT. > > Sounds cool > > But does it really take that long to generate the 2K slate and > greyscale to make it worth while to have those happen in separate > executions? > > I would assume that the quicktime gen can't happen until the > 2K slate/grayscale are done anyway, so even if another box > were working on the slate, the qt part would still have to > wait for it to get done. > > But if there's something like 50 frames of slate, then I guess > it pays to have each frame render on a separate machine, > and then have the qt gen run as a 'jobdonecommand', so that > it doesn't run until all the slate frames are finished. > > Thing is, if what you're doing is generating just one slate > image and one grayscale, but need those to hold for eg. 25x each, > then just have the render create one image each, then make symbolic > links for all the others; that'll save on disk space and render time, > so that the qt gen can refer to the links. > > For instance, the script could render these two images: > > slate-myshow-myshot.dpx > grayscale-myshow-myshot.dbx > > and then create symlinks for the frame range, eg: > > # GRAYSCALE > ln grayscale-myshow-myshot.dpx myshow-shot-frame-0001.dpx > ln grayscale-myshow-myshot.dpx myshow-shot-frame-0002.dpx > ln grayscale-myshow-myshot.dpx myshow-shot-frame-0003.dpx > : > ln grayscale-myshow-myshot.dpx myshow-shot-frame-0025.dpx > # SLATE > ln slate-myshow-myshot.dpx myshow-shot-frame-0026.dpx > ln slate-myshow-myshot.dpx myshow-shot-frame-0027.dpx > ln slate-myshow-myshot.dpx myshow-shot-frame-0028.dpx > : > ln slate-myshow-myshot.dpx myshow-shot-frame-0050.dpx > > That way you can 'instantly' make all the frames from just > the single 2k image, so that the quicktime can insert them > into the shot as needed. > > HTH. > > -- > Greg Ercolano, erco@(email surpressed) > Seriss Corporation > Rush Render Queue, http://seriss.com/rush/ > Tel: (Tel# suppressed) > Fax: (Tel# suppressed) > Cel: (Tel# suppressed) > --Apple-Mail-13-1053132520 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hey Greg

The way = my Shake script works is that it renders the QT without the need for the = slate and grey to be generated as 2k, it files them in from a set = location and adds the relevant text and overlays a mask with counter and = shot details, plus it files in the same "empty" slate, adds the same = text and has another fileout to render the single 2k frame. =A0The = greyscale that is added to the QT has a branched fileout to also render = a single 2k grey at the end.

The problem I have = observed is that the 2k slate renders itself over and over again until = the script reaches the end of it's frame count, i.e. for a 200 frame QT, = the slate will render 200 times, hence the need for splitting up the job = so that I can give the slate and grey globals of 1 and the QT 1-200 for = example.

I'm = going to look into using an offset for the slate/grey to see if this = addresses this issue, otherwise I'll look at Victor's = suggestion.

Cheers

Craig



Craig Allison

Digital Systems & = Data=A0Manager

Twickenham Film Studios

St.Margarets

Twickenham

TW1 2AW


=A0

skype: craig_9000


=


On 27 May 2009, at 22:21, Greg Ercolano = wrote:

[posted to = rush.general]

Craig Allison wrote:
Ok, so I've finally got around = to looking into this and have decided to
change = my original way of thinking and give the artists the = opportunity
to render a quicktime on the = farm once they are happy with the results

I have taken the = submit-shake-quicktime script and amended it to allow
users to select which film they are working on, = select the first frame
in the sequence to render = and dialogue for slate notes etc - the
script on the network, replaces = the variables, saves a copy and submits
it to = the farm.

The script renders a 2K slate & greyscale (named = and numbered correctly)
and also a QT = with the appropriate settings for the production.=A0 I have
it working fine but now want to speed it up by = splitting the job into
two, i.e. one script to = render the two 2k frames and one script to
render = the QT.
Sounds cool

But does = it really take that long to generate the 2K slate and
greyscale to make it worth while = to have those happen in separate
= executions?
I would assume that the quicktime = gen can't happen until the
2K = slate/grayscale are done anyway, so even if another box
were working on the slate, the qt = part would still have to
wait for = it to get done.

But if there's something like 50 = frames of slate, then I guess
it pays = to have each frame render on a separate machine,
and then have the qt gen run as a = 'jobdonecommand', so that
it = doesn't run until all the slate frames are finished.

Thing is, = if what you're doing is generating just one slate
image and one grayscale, but need = those to hold for eg. 25x each,
then just = have the render create one image each, then make symbolic
links for all the others; that'll = save on disk space and render time,
so that = the qt gen can refer to the links.

For instance, the script could = render these two images:


and then create symlinks for the = frame range, eg:

# GRAYSCALE
ln = grayscale-myshow-myshot.dpx myshow-shot-frame-0001.dpx
ln grayscale-myshow-myshot.dpx = myshow-shot-frame-0002.dpx
ln = grayscale-myshow-myshot.dpx myshow-shot-frame-0003.dpx
:
ln = grayscale-myshow-myshot.dpx myshow-shot-frame-0025.dpx
# SLATE
ln = slate-myshow-myshot.dpx =A0 =A0 = myshow-shot-frame-0026.dpx
ln = slate-myshow-myshot.dpx =A0 =A0 = myshow-shot-frame-0027.dpx
ln = slate-myshow-myshot.dpx =A0 =A0 = myshow-shot-frame-0028.dpx
:
ln slate-myshow-myshot.dpx =A0 =A0 = myshow-shot-frame-0050.dpx

That way you can 'instantly' make = all the frames from just
the = single 2k image, so that the quicktime can insert them
into the shot as = needed.

HTH.

--=A0
Greg = Ercolano, erco@(email surpressed)
Seriss Corporation
Rush = Render Queue, http://seriss.com/rush/
Tel: 626-795-5922x23
Fax: = 626-795-5947
Cel: 310-266-8906

=

= --Apple-Mail-13-1053132520-- *** ***************************************************** *** Message #1843 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Wed, 27 May 2009 17:20:59 -0400 Craig Allison wrote: > Ok, so I've finally got around to looking into this and have decided to > change my original way of thinking and give the artists the opportunity > to render a quicktime on the farm once they are happy with the results > previewed in Framecycler. > > I have taken the submit-shake-quicktime script and amended it to allow > users to select which film they are working on, select the first frame > in the sequence to render and dialogue for slate notes etc - the > combination of all this information chooses the correct shake setup > script on the network, replaces the variables, saves a copy and submits > it to the farm. > > The script renders a 2K slate & greyscale (named and numbered correctly) > and also a QT with the appropriate settings for the production. I have > it working fine but now want to speed it up by splitting the job into > two, i.e. one script to render the two 2k frames and one script to > render the QT. Sounds cool But does it really take that long to generate the 2K slate and greyscale to make it worth while to have those happen in separate executions? I would assume that the quicktime gen can't happen until the 2K slate/grayscale are done anyway, so even if another box were working on the slate, the qt part would still have to wait for it to get done. But if there's something like 50 frames of slate, then I guess it pays to have each frame render on a separate machine, and then have the qt gen run as a 'jobdonecommand', so that it doesn't run until all the slate frames are finished. Thing is, if what you're doing is generating just one slate image and one grayscale, but need those to hold for eg. 25x each, then just have the render create one image each, then make symbolic links for all the others; that'll save on disk space and render time, so that the qt gen can refer to the links. For instance, the script could render these two images: slate-myshow-myshot.dpx grayscale-myshow-myshot.dbx and then create symlinks for the frame range, eg: # GRAYSCALE ln grayscale-myshow-myshot.dpx myshow-shot-frame-0001.dpx ln grayscale-myshow-myshot.dpx myshow-shot-frame-0002.dpx ln grayscale-myshow-myshot.dpx myshow-shot-frame-0003.dpx : ln grayscale-myshow-myshot.dpx myshow-shot-frame-0025.dpx # SLATE ln slate-myshow-myshot.dpx myshow-shot-frame-0026.dpx ln slate-myshow-myshot.dpx myshow-shot-frame-0027.dpx ln slate-myshow-myshot.dpx myshow-shot-frame-0028.dpx : ln slate-myshow-myshot.dpx myshow-shot-frame-0050.dpx That way you can 'instantly' make all the frames from just the single 2k image, so that the quicktime can insert them into the shot as needed. HTH. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1842 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Victor DiMichina Date: Wed, 27 May 2009 13:11:57 -0400 --001636e0aedbbae7e5046ae7f049 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit You can try something like this, suited to your needs of course. Note the eval command (thanks go to Greg again and again, of course) ################################ #!/bin/csh -f if ( ! $?RUSH_DIR ) setenv RUSH_DIR /usr/local/rush source $RUSH_DIR/etc/.submit eval `/usr/local/rush/bin/rush -submit` << EOF title (your title) ram 1 frames (your frame range) logdir (your log dir) command perl //server/raid/2D_Apps/RushSubmits/Applications/submit-shake.app/Contents/MacOS/submit-shake-autolog.pl -render 1 1 Fail /server/raid/path/to/someshakeproject.shk 0 -v -proxyscale 1 -v cpus +farm=10@10 notes (optional) EOF if ( $status ) exit 1 # (eval eats the setenv command, so we duplicate it here) echo "setenv RUSH_JOBID $RUSH_JOBID" echo $RUSH_JOBID > /tmp/curr_jobid.txt # Job #2 - /usr/local/rush/bin/rush -submit << EOF title (your 2nd title) ram 1 frames (frame range) logdir (your log dir) command (another command) cpus +farm=11@980 waitfor $RUSH_JOBID waitforstate done notes (optional) EOF #################################################### HTH Vic On Wed, May 27, 2009 at 7:54 AM, Craig Allison wrote: > Ok, so I've finally got around to looking into this and have decided to > change my original way of thinking and give the artists the opportunity to > render a quicktime on the farm once they are happy with the results > previewed in Framecycler. > I have taken the submit-shake-quicktime script and amended it to allow > users to select which film they are working on, select the first frame in > the sequence to render and dialogue for slate notes etc - the combination of > all this information chooses the correct shake setup script on the network, > replaces the variables, saves a copy and submits it to the farm. > > The script renders a 2K slate & greyscale (named and numbered correctly) > and also a QT with the appropriate settings for the production. I have it > working fine but now want to speed it up by splitting the job into two, i.e. > one script to render the two 2k frames and one script to render the QT. > > I've got rush creating the 2 distinct shake scripts to render, but don't > know how to get 2 scripts to render with a single rush submit, is there an > easy way? > > Cheers > > Craig > > > Craig Allison > > Digital Systems & Data Manager > > The Senate Visual Effects > > Twickenham Film Studios > > St.Margarets > > Twickenham > > TW1 2AW > > > t: 0208 607 8866 > > skype: craig_9000 > > www.senatevfx.com > > > > On 19 Dec 2008, at 09:20, Abraham Schneider wrote: > > [posted to rush.general] > > Greg Ercolano schrieb: > > Sounds interesting, but unless they have a non-gui interface, > it's not something I think Rush can control properly. > > > yes, that's right. I'm doing it in two steps at the moment: rush=20 > executes a jobdonecommand script which creates an entry with the path to = > > the image sequence in a queue folder. A small python script on my QT=20 > creation Mac is watching this folder and does the actual 'feeding' of=20 > the Silverstack rendering app (called Cinemator). > > That's not a perfect solution but it works quite well for us at the=20 > moment. And with the template files it is easy to do different burnins,=20 > codecs, framerates, etc. depending on project or user choose. > > Abraham > > -- > Abraham Schneider > Senior VFX Compositor, 2 D Artist > ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = > Germany > Phone (Tel# suppressed) EMail aschneider@arri.de > www.arri.de/filmtv > ARRI Film & TV Services GmbH > Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = > Handelsregisternummer: HRB 69396 > Gesch=E4ftsf=FChrer: Franz Kraus; Thomas Till > > > -- Victor DiMichina Pixel Magic (Tel# suppressed) --001636e0aedbbae7e5046ae7f049 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable You can try something like this,=A0 suited to your needs of course.=A0=A0 N= ote the eval command (thanks go=A0 to Greg again and again,=A0 of course)
################################
#!/bin/csh -f
if ( ! $?RUSH_DI= R ) setenv RUSH_DIR /usr/local/rush
source $RUSH_DIR/etc/.submit
eval `/usr/local/rush/bin/rush -submit` <= ;< EOF
title=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your title)
ram=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1
frames=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your= frame range)
logdir=A0=A0=A0=A0=A0=A0=A0=A0=A0 (your log dir)
comman= d=A0=A0=A0=A0=A0=A0=A0 perl //server/raid/2D_Apps/RushSubmits/Applications/= submit-shake.app/Contents/MacOS/submit-shake-autolog.pl -render 1 1 Fail=A0= /server/raid/path/to/someshakeproject.shk=A0 0 -v -proxyscale 1 -v
cpus=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 +farm=3D10@10
notes=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 (optional)
EOF
if ( $status ) exit 1
# (eval eats = the setenv command, so we duplicate it here)
echo "setenv RUSH_JOBI= D $RUSH_JOBID"
echo $RUSH_JOBID > /tmp/curr_jobid.txt
# Job #2 -
/usr/local/rush/bin/rush -submit << EOF
title=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 (your 2nd title)
ram=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 1
frames=A0=A0=A0=A0=A0=A0=A0=A0=A0 (frame range)
logdir=A0= =A0=A0=A0=A0=A0=A0=A0=A0 (your log dir)
command=A0=A0=A0=A0=A0=A0=A0=A0 = (another command)
cpus=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 +farm=3D11@980
waitfor=A0=A0=A0=A0= =A0=A0=A0=A0 $RUSH_JOBID
waitforstate=A0=A0=A0 done
notes=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 (optional)
EOF
###################################= #################


HTH
Vic


On Wed, May 27, 2009 at 7:54 AM, Craig Allison <craig@(email surpressed)> wrote:<= br>
Ok, so I've finally got around to looking into this and have decided to= change my original way of thinking and give the artists the opportunity to= render a quicktime on the farm once they are happy with the results previe= wed in Framecycler.

I have taken the submit-shake-quicktime script and amended i= t to allow users to select which film they are working on, select the first= frame in the sequence to render and dialogue for slate notes etc - the com= bination of all this information chooses the correct shake setup script on = the network, replaces the variables, saves a copy and submits it to the far= m.

The script renders a 2K slate & greyscale (named an= d numbered correctly) and also a QT with the appropriate settings for the p= roduction. =A0I have it working fine but now want to speed it up by splitti= ng the job into two, i.e. one script to render the two 2k frames and one sc= ript to render the QT.

I've got rush creating the 2 distinct shake scripts= to render, but don't know how to get 2 scripts to render with a single= rush submit, is there an easy way?

Cheers

Craig


Craig= Allison

Digital Systems & Data=A0Manager

The Senate Visual Effects

Twickenham Fi= lm Studios

St.Margarets

Twickenham

TW1 2AW


t: 0208 607 8866=A0

skype: craig_9000

www.senatevfx.com<= /p>


On 19 Dec 2008, at 09:20, Abraham Schneider wro= te:

[posted = to rush.general]

Greg Ercolano schrieb:
Sounds interesting, but u= nless they have a non-gui interface,
it's not something I think Rush ca= n control properly.

yes, that's right. I'm doing it in two steps= at the moment: rush=3D20
executes a jobdo= necommand script which creates an entry with the path to =3D

the image sequence in a queue folder. A small python script on my = QT=3D20
creation Mac is watching this fold= er and does the actual 'feeding' of=3D20
the Silverstack rendering app (called Cinemator= ).

That's not a perfect solution but it works quite well= for us at the=3D20
moment. And with the template files it is easy = to do different burnins,=3D20
codecs, fram= erates, etc. depending on project or user choose.

Abraham

--
Abraham Schneider
Sen= ior VFX Compositor, 2 D Artist
ARRI Film & TV Services GmbH =A0 Tuerkenstr. 89 =A0 D-80799 Muenchen / =3D
Germany
Phone +49 89 380= 9-1269 =A0 =A0 EMail aschneider@arri.de
ARRI Film = & TV Services GmbH
Sitz: M=3DFCnchen=A0 -=A0 Registergericht: Amtsgericht M=3DFCnchen=A0 -=A0 =3D
Handelsregisternummer: HRB 69396
Gesch=3DE4ftsf=3DFChrer: Franz Kraus; Thomas Till
<= div style=3D"margin: 0px; min-height: 14px;">
=



--
Victo= r DiMichina
Pixel Magic
818.760.0862
--001636e0aedbbae7e5046ae7f049-- *** ***************************************************** *** Message #1841 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Craig Allison Date: Wed, 27 May 2009 10:54:49 -0400 --Apple-Mail-3-988392269 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Ok, so I've finally got around to looking into this and have decided to change my original way of thinking and give the artists the opportunity to render a quicktime on the farm once they are happy with the results previewed in Framecycler. I have taken the submit-shake-quicktime script and amended it to allow users to select which film they are working on, select the first frame in the sequence to render and dialogue for slate notes etc - the combination of all this information chooses the correct shake setup script on the network, replaces the variables, saves a copy and submits it to the farm. The script renders a 2K slate & greyscale (named and numbered correctly) and also a QT with the appropriate settings for the production. I have it working fine but now want to speed it up by splitting the job into two, i.e. one script to render the two 2k frames and one script to render the QT. I've got rush creating the 2 distinct shake scripts to render, but don't know how to get 2 scripts to render with a single rush submit, is there an easy way? Cheers Craig Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 19 Dec 2008, at 09:20, Abraham Schneider wrote: > [posted to rush.general] > > Greg Ercolano schrieb: >> Sounds interesting, but unless they have a non-gui interface, >> it's not something I think Rush can control properly. > > yes, that's right. I'm doing it in two steps at the moment: rush=20 > executes a jobdonecommand script which creates an entry with the > path to = > > the image sequence in a queue folder. A small python script on my > QT=20 > creation Mac is watching this folder and does the actual 'feeding' > of=20 > the Silverstack rendering app (called Cinemator). > > That's not a perfect solution but it works quite well for us at the=20 > moment. And with the template files it is easy to do different > burnins,=20 > codecs, framerates, etc. depending on project or user choose. > > Abraham > > -- > Abraham Schneider > Senior VFX Compositor, 2 D Artist > ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = > Germany > Phone (Tel# suppressed) EMail aschneider@arri.de > www.arri.de/filmtv > ARRI Film & TV Services GmbH > Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = > Handelsregisternummer: HRB 69396 > Gesch=E4ftsf=FChrer: Franz Kraus; Thomas Till > --Apple-Mail-3-988392269 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Ok, so I've finally got around to looking into this and have decided to = change my original way of thinking and give the artists the opportunity = to render a quicktime on the farm once they are happy with the results = previewed in Framecycler.

I have taken the = submit-shake-quicktime script and amended it to allow users to select = which film they are working on, select the first frame in the sequence = to render and dialogue for slate notes etc - the combination of all this = information chooses the correct shake setup script on the network, = replaces the variables, saves a copy and submits it to the = farm.

The = script renders a 2K slate & greyscale (named and numbered correctly) = and also a QT with the appropriate settings for the production. =A0I = have it working fine but now want to speed it up by splitting the job = into two, i.e. one script to render the two 2k frames and one script to = render the QT.

I've got rush creating the = 2 distinct shake scripts to render, but don't know how to get 2 scripts = to render with a single rush submit, is there an easy way?

Cheers

Craig


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


On 19 Dec 2008, at 09:20, Abraham Schneider = wrote:

[posted to = rush.general]

Greg Ercolano schrieb:
Sounds interesting, but unless = they have a non-gui interface,
it's not = something I think Rush can control properly.

yes, = that's right. I'm doing it in two steps at the moment: rush=3D20
executes a jobdonecommand script which creates an = entry with the path to =3D

the image sequence in a queue = folder. A small python script on my QT=3D20
the Silverstack rendering app = (called Cinemator).
That's not a perfect solution = but it works quite well for us at the=3D20
moment. = And with the template files it is easy to do different = burnins,=3D20
codecs, framerates, etc. = depending on project or user choose.

Abraham

Abraham Schneider
Senior VFX Compositor, 2 D Artist
ARRI Film & TV Services GmbH =A0 Tuerkenstr. 89 =A0 D-80799 Muenchen / = =3D
Germany
Phone +49 89 3809-1269 =A0 =A0 EMail aschneider@arri.de
www.arri.de/filmtv
ARRI = Film & TV Services GmbH
Sitz: = M=3DFCnchen=A0 -=A0 Registergericht: Amtsgericht = M=3DFCnchen=A0 -=A0 =3D
Handelsregisternummer: HRB 69396
Gesch=3DE4ftsf=3DFChrer: Franz Kraus; Thomas = Till

=

= --Apple-Mail-3-988392269-- *** ***************************************************** *** Message #1840 ***************************************************** *** ***************************************************** Subject: Freelance Systems Admins and Render pipeline scripting, plug-ins From: Greg Ercolano Date: Wed, 13 May 2009 20:56:38 -0400 If you're looking for an available render pipeline scripting programmer, talk to Jorg Mohnen: 310-437-3614, jorg(at)mohnen(dot)de I've added him to my 'list' of freelance sysadmins/pipeline programmers (which I can send Rush customers via private mail) He's based here in LA, has worked for a lot of shops, and knows Rush quite well, and wants to help other shops looking to stream lining their Rush render pipelines. In the past he's been working fulltime for different companies, but has wanted to put more attention to doing freelance scripting and admin. His resume is here: http://www.wavgen.com/mohnen-TD.pdf -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1839 ***************************************************** *** ***************************************************** Subject: Re: [OSX/SYSADMIN] Example NFS setup for Rush on a small 'pure OSX' From: Greg Ercolano Date: Wed, 13 May 2009 20:37:16 -0400 Just a followup to this thread. When creating the 'render' user from the command line, Apple changed everything around in Leopard (10.5), so that now the 'niload' command is no longer available. They now recommend using their new 'dscl' command instead. So the Leopard version of the script to create the new 'render' user would be: #!/bin/sh # # make-render-user - Run this script to create the 'render' user on a Leopard # 1.0 05/10/06 - erco@(email surpressed) (original Tiger version) # 1.1 05/13/09 - erco@(email surpressed) (changed for Leopard) # username=render # user's name uid=555 # must be an unused uid gid=20 # 20=staff passwd=render # the actual *cleartext* password for the user # CREATE THE USER dscl . -create /Users/$username dscl . -create /Users/$username UserShell /bin/bash dscl . -create /Users/$username RealName $username dscl . -create /Users/$username UniqueID $uid dscl . -create /Users/$username PrimaryGroupID $gid dscl . -create /Users/$username NFSHomeDirectory /Users/$username dscl . -passwd /Users/$username $passwd # CREATE THE USER'S HOME DIRECTORY cp -R '/System/Library/User Template/English.lproj' /Users/$username # ENSURE OWNERSHIPS ASSIGNED TO USER'S ENTIRE HOME DIRECTORY chown -R ${uid}:${gid} /Users/$username So that should be it. The older Tiger script is below for reference. BTW, if you want the new user to be an "administrator" user, you can add this line to the list of dscl commands: dscl . -append /Groups/admin GroupMembership $username > STEPS TO SETUP > -------------- > > 1) On all machines, create the "render" user with a fixed UID/GID that > doesn't conflict with any uid/gid on the network. > > This can be done easily by running the following script on each machine > as root to create the 'render' account on each machine with the uid=555, > gid=20, and the password set to 'render'. (Tested under OSX 10.4.x) > > --- snip > #!/bin/csh -f > # > # make-render-user - Run this script to create the 'render' user on a machine > # 1.0 05/10/06 - erco@(email surpressed) - Supports Tiger (10.4), Panther (10.3) > # > set username = render # user's name > set uid = 555 # must be an unused uid > set gid = 20 # 20=staff > set passwd = 'oFxbR2cAnG902' # passwd is 'render' > > # CREATE PASSWD ENTRY > echo ${username}:${passwd}:${uid}:${gid}::0:0:${username}:/Users/${username}:/bin/bash | niload passwd / > > # CREATE USER'S HOME DIRECTORY > if ( ! -d /Users/${username} ) then > cp -R '/System/Library/User Template/English.lproj' /Users/${username} > endif > > # ENSURE OWNERSHIPS ASSIGNED TO USER'S ENTIRE HOME DIRECTORY > /usr/sbin/chown -R ${uid}:${gid} /Users/${username} > --- snip *** ***************************************************** *** Message #1838 ***************************************************** *** ***************************************************** Subject: Re: Log scrolling From: Greg Ercolano Date: Fri, 17 Apr 2009 17:39:51 -0400 Marco Recuay wrote: > [posted to rush.general] > > We're using version 102.42a8. > > I had already turned off log tailing like you'd mentioned, but reading > in the logs still happens progressively. The larger the log, the more > it slows down it seems. Granted, this was a 10mb log file (!), but the > way it was loading the log line by line seemed odd to me. It's trying to display it in a browser that's line oriented, but it should be reading the data in blocks of 4096. If its not, might be a bug in that version.. I'll check. You might try installing the more recent 102.42a9 (July 2008 release) on a single machine to see if it makes a difference. Which OS platform is this on? (Some platforms behave differently than others) -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1837 ***************************************************** *** ***************************************************** Subject: Re: Log scrolling From: Marco Recuay Date: Fri, 17 Apr 2009 16:53:28 -0400 We're using version 102.42a8. I had already turned off log tailing like you'd mentioned, but reading in the logs still happens progressively. The larger the log, the more it slows down it seems. Granted, this was a 10mb log file (!), but the way it was loading the log line by line seemed odd to me. Marco On 2009-04-13 21:00:37 -0700, Greg Ercolano said: > Hmm, which version of Rush are you running, and on which platform? > > You can find out inside irush via "Help -> About Irush", and include > the version number here, which will be something like 102.42aXX > > There were a few versions where the log tailing feature was slower > than other releases, so upgrading may help solve that. > > Given the version info, I can probably tell you if you can upgrade > just the irush tool, without having to upgrade the rest. *** ***************************************************** *** Message #1836 ***************************************************** *** ***************************************************** Subject: Re: Log scrolling From: Greg Ercolano Date: Tue, 14 Apr 2009 00:00:37 -0400 Marco Recuay wrote: > I was wondering if it is possible to disable, or speed up, IRush log > playback. Our Maya renders spit out an ungodly amount of info, which is > good for problem-solving, but it takes a long time to reach the end of > the file. Yes; if by playback you mean the auto-tailing feature, you can disable that via "Edit -> Preferences -> Tail Logs" > If we just navigate to the log location we can open it in a text editor > immediately and read it there, but that is a cumbersome process to > repeat often. Next release of irush will actually have a popup menu for alternately viewing the log in a separate editor (that you can set, eg. gvim/wordpad/TextEditor, etc) > Disabling log tailing will stop the automatic scroll, but the log file > is not loaded immediately, you still have to wait for the log to chug > along to the bottom. > > Any ideas? Hmm, which version of Rush are you running, and on which platform? You can find out inside irush via "Help -> About Irush", and include the version number here, which will be something like 102.42aXX There were a few versions where the log tailing feature was slower than other releases, so upgrading may help solve that. Given the version info, I can probably tell you if you can upgrade just the irush tool, without having to upgrade the rest. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1835 ***************************************************** *** ***************************************************** Subject: Log scrolling From: Marco Recuay Date: Mon, 13 Apr 2009 22:42:49 -0400 I was wondering if it is possible to disable, or speed up, IRush log playback. Our Maya renders spit out an ungodly amount of info, which is good for problem-solving, but it takes a long time to reach the end of the file. If we just navigate to the log location we can open it in a text editor immediately and read it there, but that is a cumbersome process to repeat often. Disabling log tailing will stop the automatic scroll, but the log file is not loaded immediately, you still have to wait for the log to chug along to the bottom. Any ideas? Marco *** ***************************************************** *** Message #1834 ***************************************************** *** ***************************************************** Subject: Maya 2009 not using all processors From: Greg Ercolano Date: Wed, 08 Apr 2009 17:27:21 -0400 If you're having trouble with the new Maya 2009 not using all processors on eg. quad core machines, check out this article: http://mattgadient.com/2009/02/22/make-maya-2009-use-a-multi-core-cpu/ I'm quoting the highlights of the article here, just in case the article "disappears" for some reason: Make Maya 2009 use a multi core cpu ----------------------------------- [February 22nd, 2009] Article by: Matthew Gadien "A small shocker using Maya on a Quad-Core Intel processor was that CPU usage by default was about 50%. Yes that’s right FIFTY PERCENT. It looked similar to this: [greg's ascii-art of their Task Manager showing 50% usage] ____________________________________________________________ | Windows Task Manager | |____________________________________________________________| | | | Cpu Usage _________ _________ _________ _________ | | .... .... | || || || || | || | | | .... .... | || || ||__|| | || | | | .... .... | ||_||__| ||___|__|__|| | | | XXXX XXXX |___ ___|| || ||_____ _| | | XXXX XXXX | |_| || || || |_| | | | XXXX XXXX |_________||_________||_________||_________| | | 50 % Cpu 1 Cpu 2 Cpu 3 Cpu 4 | |____________________________________________________________| "This was a little disappointing - Maya 2009 basically caused another dual-core machine to grind to a halt, and was only using about half the potential of the Quad-Core." He goes on to say (skipping some content): "If you’re using the default Maya Software renderer, there’s an option in the Render Settings menu, but I wont go into that here. The individual I was helping needed MENTAL RAY to be used (instead of Maya Software Renderer), and there wasn’t an equivilent option in that menu. "If you’re using Mental Ray, this is where you make the change [inside maya]: [Greg's summary of the article's screenshots] -------------------------------------------------------------- 1) Switch to the 'Rendering' submenu 2) Under the 'Render' menu, click on "Batch Render" 3) In the "Mental Ray Batch Render Option" dialog, in the "Parallelism" section: 3a) Set Render Threads: 8 3b) Disable the 'Auto Render Threads' checkbox -------------------------------------------------------------- [For a Quad-Core I needed to use 8 to max out the processors.] "This is the trick! "By default, Maya chooses the threads on it’s own. It actually appears to select 4 on a quad-core by default, but it doesn’t max out the cpu usage on each. Changing that number to 8 queues up threads, which pegs the processor to full usage, and speeds up the rendering considerably. "Of course, with all 4 cores maxed, everything else on the computer will be VERY SLOW - even typing out an email, I was able to type an entire sentance out before it appeared on the screen. You will not even want to try doing other stuff while it renders in the background - trust me. On the plus side, more renders happen. The stuff being rendered in my case was doing about 3 renders per minute for the section it was in (a simple section), and it jumped to between 5-6 per minute - not *quite* double, but still quite the increase. "Just for completeness, this is what Task Manager looked like in the end: [greg's ascii-art of their Task Manager showing 100% usage] ____________________________________________________________ | Windows Task Manager | |____________________________________________________________| | | | Cpu Usage _________ _________ _________ _________ | | xxxx xxxx |---------||---------||---------||---------| | | xxxx xxxx | | | || | | | || | || | | | | xxxx xxxx | | || | || || | | | | xxxx xxxx | || || || | | | xxxx xxxx |_________||_________||_________||_________| | | 100 % Cpu 1 Cpu 2 Cpu 3 Cpu 4 | |____________________________________________________________| "Finally, with 8 threads selected, the CPU usage is pegged at 100%." The article goes on to note memory usage stays low, at about 1/4 use in his case, and said he manually set the memory but it made little difference in usage. Then he talks about THREADS vs CORES, which you may find useful; quoting again: "This change talks about THREADS vs CORES. There is a difference, although for our purposes it doesn’t really matter. THREADS are basically like a process, and the operating system sends out threads to cores based on what core is the least busy at the time. We set up 8 threads in the above, so assuming they distribute evenly, each core should be getting fed 2 threads consistantly. We can’t specify CORES with Mental Ray (as far as I can tell), but it doesn’t matter - there wouldn’t be an advantage anyway, since enough threads will fill all the cores up regardless. The only real benefit to actually being able to specify cores would be that if you wanted to you could choose 3 cores, filling them up and leaving the 4th core alone (the system would probably remain snappy with 1 core untouched by Maya, although you’d be rendering at around 3/4 of the potential)." And he then goes on to speculate /why/ 4 threads doesn't max out the cpu cores, thinking it's one of two things [quoting again]: * "Each thread opens, does something, then closes, and the processor isn’t used much between the time a thread closes and a new one opens. * "Each thread ends up waiting on other things (memory reads/writes) which leave it idle." My guess (erco here) is more the latter; rendering can be IO intensive at times, so while IO is happening, the cpu is free to render, so having that extra thread keeps the proc slammed while IO buffers are waiting to clear. *** ***************************************************** *** Message #1833 ***************************************************** *** ***************************************************** Subject: Re: Rush submit menu in Shake From: Greg Ercolano Date: Thu, 12 Mar 2009 15:47:35 -0400 johan aberg wrote: > The following Shake code will put a menu in Shake that will call Rush's > submitscript for Shake. It will pass the name of the current Shake > script as a argument to submit-shake.pl. If you modify submit-shake so > that it picks up the new argument, all you have to do is to press the > update button in submit-shake.pl to set the framerange and fire off the > render. Since this original newsgroup posting was made (2004), all the rush submit scripts now support setting any of the fields in the submit form by using the -field command option to the submit script. For example, to preset the 'Shake Path:' field in the submit form to "/some/path/foo.shk", one can invoke the script with: perl /path/to/RUSH/submit-shake.pl -field ShakePath /some/path/foo.shk So to make the example Johan posted work, you don't need to change these new submit scripts at all, and would only need to change this one line in Johan's shake code from BEFORE to AFTER: BEFORE: const char* command = stringf("/path/to/RUSH/submit-shake.pl %s &", NRiMainWin1.scriptName); AFTER: const char* command = stringf("/path/to/RUSH/submit-shake.pl -field ShakePath %s &", NRiMainWin1.scriptName); ^^^^^^^^^^^^^^^^^^^ [Thanks to Vincent at Eclair Laboratories for verifying this] You can initialize other fields in the submit form this way as well. Multiple '-field' options can be specified to initialize several fields all at once, eg: perl /path/to/RUSH/submit-shake.pl \ -field ShakePath /some/path/foo.shk \ -field Frames 1-10 \ -field JobTitle TESTING_ONE_TWO Note that when specifying fields this way, leave out any spaces in the field's name. Example: "Shake Path" in the submit form becomes "ShakePath" on the command line. > Put the code in a file.h in Shake's include/startup/ui directory. > > ps. Mind the linewraps > > johan > > //////////// > > extern "C" { > int system(const char*); > } > > void CallSystemMenu() > { > const char* command = stringf("/path/to/RUSH/submit-shake.pl %s &", > NRiMainWin1.scriptName); > system(command); > } > > nuiPushMenu("RushRender"); > nuiMenuItem("Submit Rush Job", CallSystemMenu()); > nuiPopMenu(); > > //////////// *** ***************************************************** *** Message #1832 ***************************************************** *** ***************************************************** Subject: [MAC/ADMIN] FAILED TO GET ASN FROM CORESERVICES so aborting From: Greg Ercolano Date: Fri, 06 Feb 2009 18:39:32 -0500 If you're seeing either of these two errors on Leopard systems when running either REDline or After Effects renders: 1) FAILED TO GET ASN FROM CORESERVICES so aborting 2) CFMessagePort: bootstrap_register(): failed 1100 (0x44c) ...let me know via direct email, as I have a workaround that involves a small one line modification to the S99rush boot script. The above errors you'll see on applications that try to access the window manager (whether they actually open windows or not), and run into trouble with Apple's technote TN2083, which is new in Leopard. *** ***************************************************** *** Message #1831 ***************************************************** *** ***************************************************** Subject: [WINDOWS/ADMIN] How to force drive mapping within submit scripts, From: Greg Ercolano Date: Fri, 23 Jan 2009 16:23:03 -0500 This subject has been covered before, but just to tie it all up into a single article.. * * * You can have the submit scripts try to detect and fix broken drive maps on the fly by adding logic to the 'render' section of the submit script just before the render is invoked. Drives can be mapped with the DOS 'net use' command, eg: NET USE Z: \\YOURSERVER\YOURSHARE3 /PERSISTENT:YES You can add commands like this to the MAIN_Render() section of the script. Since in some cases the maps can time out, you may have to first delete the broken map before re-creating it. Some example perl code to do all this: if ( ! -d "z:/" ) { my $cmd = "net use z: /delete < nul"; print "Z: NOT MAPPED -- FORCE UNMAP FIRST: $cmd\n"; system($cmd); $cmd = "net use z: \\\\yourserver\\yourshare1 /PERSISTENT:YES < nul"; print "Z: MAPPING WITH: $cmd\n"; system($cmd); } Note that *backslashes* have to be used for the pathname of this DOS command, so be sure to 'double up' your slashes, since backslashes are an escape character in perl (and most all scripting languages). One problem with the above is a catch-22 situation, where the script itself may be specified with a drive letter; so if the drive isn't mapped, the machine can't even run the script to fix the problem, because the path to the script can't be reached. In this case, you have to ensure the pathname to the script *and* the log directory are both UNC paths. Fortunately, you can do this by modifying the FixPath() function in .common.pl on your fileserver, so that the pathname is fixed just before the job is submitted to Rush; this will handle translating the drive letters into a UNC equivalent. See the lines commented with "# ADD THIS LINE" in the following to handle changing several different drive mappings: sub FixPath($) { my ($path) = @_; if ( $G::iswindows ) { .. } else { .. } $path =~ s%/\./%/%g; # /foo/./bar -> /foo/bar $path =~ s%^z:%//yourserver/share1%gi; # ADD THIS LINE: z: -> //yourserver/share1 $path =~ s%^y:%//yourserver/share2%gi; # ADD THIS LINE: y: -> //yourserver/share2 $path =~ s%^x:%//yourserver/share3%gi; # ADD THIS LINE: x: -> //yourserver/share3 return($path); } *** ***************************************************** *** Message #1830 ***************************************************** *** ***************************************************** Subject: [ADMIN/OT] Environment variables in Windows' native file browsers.. From: Greg Ercolano Date: Thu, 22 Jan 2009 18:20:17 -0500 Just an FYI. I didn't know this, but apparently one can specify environment variables in Microsoft Window's file browsers? For example, if you have the environment variable called SHOW that is set to an existing path like \\server\jobs\honda, you can type at the Filename: prompt: %SHOW%\ ...and see a directory listing of \\server\jobs\honda. Since eg. Maya uses Microsoft's native file browsers, this means companies can come up with shortcut ways for users to access long pathnames easily, saving a bit of typing and manual surfing. Similarly, under Linux, with konqueror you can use $SHOW in the filename bar. Unfortunately, I don't think the Mac's file browsers work similarly. Pretty interesting, though. *** ***************************************************** *** Message #1829 ***************************************************** *** ***************************************************** Subject: Re: Maxwell Standalone Renders on OSX From: Luke Cole Date: Thu, 22 Jan 2009 18:12:01 -0500 Thanks for the extra info Robert, I'll try contacting Next Limit too and see what kind of response we can get. On 23/01/2009, at 9:49 AM, Robert Minsk wrote: > I contacted NextLimit over year ago about this issue and nothing > has changed. > Maybe if a few more users add some weight behind the issue they may do > something. > > Be careful with making the executable setuid. Maxwell will write > things all > over the drive given the chance. --- Luke Cole Research & Development FUEL International 65 King St., Newtown, Sydney NSW, Australia 2042 *** ***************************************************** *** Message #1828 ***************************************************** *** ***************************************************** Subject: Re: Maxwell Standalone Renders on OSX From: Robert Minsk Date: Thu, 22 Jan 2009 17:49:36 -0500 On Wednesday 21 January 2009 22:59, Greg Ercolano wrote: > I'd suggest opening a dialog with the Maxwell guys right away. > GUIs make unattended/distributed rendering almost impossible. > > Best thing I can advise trying is making the executable > setuid root, eg: > > chown 0.0 /path/to/maxwell/executable > chmod 4755 /path/to/maxwell/executable > > ..and see how that flies. I contacted NextLimit over year ago about this issue and nothing has changed. Maybe if a few more users add some weight behind the issue they may do something. Be careful with making the executable setuid. Maxwell will write things all over the drive given the chance. *** ***************************************************** *** Message #1827 ***************************************************** *** ***************************************************** Subject: Re: Maxwell Standalone Renders on OSX From: Luke Cole Date: Thu, 22 Jan 2009 02:04:55 -0500 Thanks Greg, We'll give it a try and see how it goes! On 22/01/2009, at 5:59 PM, Greg Ercolano wrote: > > I'd suggest opening a dialog with the Maxwell guys right away. > GUIs make unattended/distributed rendering almost impossible. > > Best thing I can advise trying is making the executable > setuid root, eg: > > chown 0.0 /path/to/maxwell/executable > chmod 4755 /path/to/maxwell/executable > > ..and see how that flies. --- Luke Cole Research & Development FUEL International 65 King St., Newtown, Sydney NSW, Australia 2042 *** ***************************************************** *** Message #1826 ***************************************************** *** ***************************************************** Subject: Re: Maxwell Standalone Renders on OSX From: Greg Ercolano Date: Thu, 22 Jan 2009 01:59:34 -0500 Luke Cole wrote: > We are experiencing a problem getting maxwell standalone renders > working successfully on osx. The problem appears to be due to the > maxwell standalone renderer needing to open a graphical console > window to output logging information rather than writing to stdout. > > Rush is configured to run apps on the osx machines under a dedicated > render user account, and if this user is not logged in to the osx > window server, the maxwell standalone "command line" client (mxcl) > fails with the following error, because the dedicated render user > does not have permission to access the window server: > > _RegisterApplication(), FAILED TO establish the default connection to > the WindowServer, _CGSDefaultConnection() is NULL. > > Annoyingly, there appears to be no means of forcing mxcl to run in a > silent mode - it will always attempt to open a graphical console > window. On our dedicated osx render nodes, this is not really an > issue, as we are able to leave these machines logged in under the > dedicated render user, and all is well. > > We would like to be able to render on some of our high-end osx > workstations in the evenings however, and as these are usually left > logged in under artist user accounts, the jobs end up failing due to > the above error. > > I was wondering if anyone might have any suggestions on how we could > work around this issue? I'd suggest opening a dialog with the Maxwell guys right away. GUIs make unattended/distributed rendering almost impossible. Best thing I can advise trying is making the executable setuid root, eg: chown 0.0 /path/to/maxwell/executable chmod 4755 /path/to/maxwell/executable ..and see how that flies. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1825 ***************************************************** *** ***************************************************** Subject: Maxwell Standalone Renders on OSX From: Luke Cole Date: Thu, 22 Jan 2009 01:41:14 -0500 Hi, We are experiencing a problem getting maxwell standalone renders working successfully on osx. The problem appears to be due to the maxwell standalone renderer needing to open a graphical console window to output logging information rather than writing to stdout. Rush is configured to run apps on the osx machines under a dedicated render user account, and if this user is not logged in to the osx window server, the maxwell standalone "command line" client (mxcl) fails with the following error, because the dedicated render user does not have permission to access the window server: _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. Annoyingly, there appears to be no means of forcing mxcl to run in a silent mode - it will always attempt to open a graphical console window. On our dedicated osx render nodes, this is not really an issue, as we are able to leave these machines logged in under the dedicated render user, and all is well. We would like to be able to render on some of our high-end osx workstations in the evenings however, and as these are usually left logged in under artist user accounts, the jobs end up failing due to the above error. I was wondering if anyone might have any suggestions on how we could work around this issue? Thanks in advance, --- Luke Cole Research & Development FUEL International 65 King St., Newtown, Sydney NSW, Australia 2042 *** ***************************************************** *** Message #1824 ***************************************************** *** ***************************************************** Subject: Re: Toxic 2009 From: Greg Ercolano Date: Wed, 14 Jan 2009 20:21:36 -0500 matx wrote: > [posted to rush.general] > > On 2009-01-14 14:39:08 -0800, Greg Ercolano said: >> Shouldn't be a problem to have a submit script for it, though. >> At very least, you can run it through submit-generic, eg: > > I haven't tried rendering with it in Terminal, but it does indeed > appear to be possible. I'll try the submit generic script. What I would do first is try to get it to work from the command line. If you can get it to work in the terminal, then toss the command into submit-generic, replacing the 'startframe' and 'endframe' values with the following, depending on your OS: UNIX ---- If you're using unix/osx, then in submit-generic, set "Shell: sh", and in place of the startframe and endframe, use: $BATCH_SFRM <-- use this in place of the start frame $BATCH_EFRM <-- use this in place of the end frame eg: source /opt/Autodesk/Autodesk_Toxik-2009/bin/toxik-env.sh txrender .. -s $BATCH_SFRM -e $BATCH_EFRM .. (Note the 'source' command is recommended by Toxik's docs) WINDOWS ------- If you using Windows, then in submit-generic set the "Shell: DOS-Batch", and in place of the startframe and endframe, use: %BATCH_SFRM% <-- use this in place of the start frame %BATCH_EFRM% <-- use this in place of the end frame eg: txrender .. -s %BATCH_SFRM% -e %BATCH_EFRM% .. If you have any trouble, email me privately, and when we iron out the specifics, I'll follow up here. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1823 ***************************************************** *** ***************************************************** Subject: Re: Toxic 2009 From: matx Date: Wed, 14 Jan 2009 19:30:09 -0500 On 2009-01-14 14:39:08 -0800, Greg Ercolano said: > > Shouldn't be a problem to have a submit script for it, though. > At very least, you can run it through submit-generic, eg: I haven't tried rendering with it in Terminal, but it does indeed appear to be possible. I'll try the submit generic script. $ txrender A project path must be given Usage: -userfilepath : User file path. (default: /Users/admin/Library/Preferences/Autodesk/Autodesk Toxik 2009b1//.txuser) -p,-project : Project path. -c,-composition : Composition path. -v,-version : Version name. -o,-outputs : List of output names ('/' separated). -s,-startframe : Start index of the interval that constrains the frames to render for every output. Can be omitted meaning there's no start limit. -e,-endframe : End index (exclusive) of the interval that constrains the frames to render for every output. Can be omitted meaning there's no end limit. -f,-frameindex : Renders only the frame at the given index in each output where possible. -frameoffset : Renders only the frame at the given offset from the start index (--startframe parameter required) in each output where possible (offset is 1-based => start frame is at offset 1). > Is this a serious contender to Nuke/AfterEffects/etc? Some people are looking at it seriously, as it integrates well with Maya. :) Mat X *** ***************************************************** *** Message #1822 ***************************************************** *** ***************************************************** Subject: Re: Toxic 2009 From: Greg Ercolano Date: Wed, 14 Jan 2009 17:59:21 -0500 Greg Ercolano wrote: > matx wrote: >> Any support currently for Toxic 2009? > > Autodesk "Toxik", the compositor? BTW, if you're seriously starting to use it, I'll make a submit script for it, and I'll work with you to test it out. But if it's just an inquiry, I won't write a script for it. Someone has to be willing to test it. -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1821 ***************************************************** *** ***************************************************** Subject: Re: Toxic 2009 From: Greg Ercolano Date: Wed, 14 Jan 2009 17:39:08 -0500 matx wrote: > Any support currently for Toxic 2009? Autodesk "Toxik", the compositor? I've never tried it. No one's really asked much about it. I think I've had one previous inquiry just seeing if people were using it. Shouldn't be a problem to have a submit script for it, though. At very least, you can run it through submit-generic, eg: Shell: DOS-Batch Job Title: TOXIK_TEST Frames: 1-100 Batch: 5 Log Directory: //yourserver/some/empty/dir Command: txrender –d DB_MYDATABASE –u User1 –p Project1 –c CompA –r “CompA_06-06-29_174526” –f “CompA-” –D “\\yourserver\renders” -s %BATCH_START% –e %BATCH_END% –i EXR Cpus: +any=5 Their docs on the command line are pretty thin (their PDF docs refer to a screenshot of someone running the command line help) Is this a serious contender to Nuke/AfterEffects/etc? -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1820 ***************************************************** *** ***************************************************** Subject: Toxic 2009 From: matx Date: Wed, 14 Jan 2009 16:49:35 -0500 Any support currently for Toxic 2009? -- :) Mat X *** ***************************************************** *** Message #1819 ***************************************************** *** ***************************************************** Subject: [Q+A/WINDOWS ADMIN] No more connections can be made to this remote From: Greg Ercolano Date: Thu, 08 Jan 2009 20:01:36 -0500 > We have a problem where we're getting this error > from some of the machines on our farm: > ******************************************************* > ERROR: can't create log: //server001/XSI/Scenes/bw12.scn.log: > No more connections can be made to this remote computer at > this time because there are already as many connections as the > computer can accept. > ******************************************************* > > All the systems used are Windows2000 Professional. > And the number of machines used for a rendering is 10. This is a standard Microsoft error when a machine tries to open a file on a file server that has maxed out its CAL licenses. In this case, there are too many file sharing connections open to file server //server001/XSI/. Microsoft devotes several "Knowledge Base" (KB) articles to this: 1) http://support.microsoft.com/kb/314882 2) http://support.microsoft.com/kb/328459 3) http://support.microsoft.com/kb/122920 The error is also covered pretty well on Usenet, eg: http://groups.google.com/group/microsoft.public.windowsxp.network_web/browse_frm/thread/1a5c4a7ed0f4d954?hl=en&lr=&ie=UTF-8&oe=UTF-8&rnum=7&prev=/groups%3Fhl%3Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26q%3DNo%2Bmore%2Bconnections%2Bcan%2Bbe%2Bmade%2Bto%2Bthis%2Bremote%2Bcomputer%2Bat%2B%26btnG%3DGoogle%2BSearch&fwc=1 If "server001" is running Windows 2000 Professional, there can only be 10 NTFS connections open at a time. Any more than that you would need to get Windows 2000 Server. On host 'server001', go into 'My Computer' and right click on the 'XSI' share, and check what the 'User Limit' is set to. You will find that, for Win2K professional, there's a 10 user/remote host connection limit. Microsoft says this cannot be changed for their "Professional" series of products; you /have/ to upgrade to one of the Microsoft "Server" products which support more than 10 connections, and you can purchase more "CAL" licenses to increase the connection count. (CAL=Client Access Licenses). Microsoft is unusual in that the license their file server's connections. A connection is "per-user", "per-share". This means even if there's one share, a single workstation might still use up more than one connection for that share, if more than one user account is actively accessing the share. This problem will appear to be intermittent, as connections will timeout after 15 minutes of inactivity, freeing them up. Then different machines will have the problem. Even users may encounter the error while opening files interactively. Administrators of Microsoft file servers have to make sure file servers have enough Microsoft's CAL licenses to support their entire render farm and workstation users all working concurrently. This is why many companies do not use Microsoft machines for file servers, to avoid the CAL licensing issues. The solution is to either: 1) Add more Microsoft CAL licenses to your "Server" 2) If your server is running a workstation version of Microsoft's OS, (XP Pro, 2K Pro, etc), you will need to install Microsoft "Server" instead. The workstation OS's are limited to 10 connections, period. 3) Use a non-Microsoft file server, eg. Mac, Linux, or some third party file server, like NetApp, BlueArc, Isilon, etc.. *** ***************************************************** *** Message #1818 ***************************************************** *** ***************************************************** Subject: [FYI] Freelance Perl/Python programmer available From: Greg Ercolano Date: Fri, 19 Dec 2008 15:50:23 -0500 CGI pipeline scripting programmers are about as rare as hens teeth, so thought you all would be interested to know.. Ammon Riley is apparently available for doing freelance perl/python/pyQT pipeline scripting via remote access from Portland. His resume: http://www.linkedin.com/in/ammonriley He used to work at R&H and Laika, and I guess he got caught up in the massive Laika layoffs due to the closing of the feature. Caught wind of him from one of the CGI mailing lists.. FYI. *** ***************************************************** *** Message #1817 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: "Abraham Schneider" Date: Fri, 19 Dec 2008 04:20:37 -0500 Greg Ercolano schrieb: > Sounds interesting, but unless they have a non-gui interface, > it's not something I think Rush can control properly. yes, that's right. I'm doing it in two steps at the moment: rush=20 executes a jobdonecommand script which creates an entry with the path to = the image sequence in a queue folder. A small python script on my QT=20 creation Mac is watching this folder and does the actual 'feeding' of=20 the Silverstack rendering app (called Cinemator). That's not a perfect solution but it works quite well for us at the=20 moment. And with the template files it is easy to do different burnins,=20 codecs, framerates, etc. depending on project or user choose. Abraham -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Thomas Till *** ***************************************************** *** Message #1816 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Thu, 18 Dec 2008 13:30:57 -0500 Craig Allison wrote: > I'm currently working on the Applescript suggestion to find/replace the > contents of a Shake "setup" script and then submitting to command line > render. Note that shake files are simple ascii, so you can use tools like sed or perl to take a template shake script and convert it on the fly to what you need. IIRC, you can also just edit the .shk file, and replace the strings you want to use with environment variables.. then all you have to do is be sure to set those env vars before running shake, and the replacing will be handled by shake. I think there may also be a way to pass variables from the shake command line. > Any QT additions to the Rush submit would be helpful for the artists, as > they could make use of the farm for quick preview QT's for long shots > instead of Framecycler/DJView/FlipBook. Right.. See also the submit-shake-quicktime submit script, which artists can use to do QT conversions. Use the one from 102.42a9, which has many codecs, and several fixes from previous versions. These tweaks are mentioned in the release notes for A9: http://www.seriss.com/rush-current/rush/release-notes.html#102.42a9 ..specifically: o submit-shake-quicktime -- numerous fixes to support codecs > I'll let you know how I get on with the Applescript as I'm just > unravelling how it all works and am seriously under the cosh with > deadlines!!! Yes, feel free to send pastes. I've done a few scripts with AppleScript to descan the slitscan sequences from 2001; I used applescript to hit the frame advance button in iDVD so that I could grab scanlines from the film to unwrap them. Fun pictures here: http://seriss.com/people/erco/2001/wip/ ..with the scripts at the bottom. See 'shoot.pl' for a perl script that sends applescript text strings to osascript on a remote machine. eg: system("( echo 'tell app \"DVD Player\"'; echo step dvd; echo end tell ) | rsh tow osascript"); Must admit, applescript is quite neat. But the fact it works through GUIs is a problem for automation in contexts where there is no window manager, or no one logged in. In the above case, I had to make sure the GUI was running before I could run the script. > Aye, aye, aye bring on Christmas! Amen to that..! -- Greg Ercolano, erco@(email surpressed) Seriss Corporation Rush Render Queue, http://seriss.com/rush/ Tel: (Tel# suppressed) Fax: (Tel# suppressed) Cel: (Tel# suppressed) *** ***************************************************** *** Message #1815 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Craig Allison Date: Thu, 18 Dec 2008 13:12:19 -0500 --Apple-Mail-30-61134651 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed I was looking at Silverstack recently and DJV-convert but they don't quite meet my requirements as we have movie specific masking/crops applied to the full 2k frames when they are converted to QT... I'm currently working on the Applescript suggestion to find/replace the contents of a Shake "setup" script and then submitting to command line render. Any QT additions to the Rush submit would be helpful for the artists, as they could make use of the farm for quick preview QT's for long shots instead of Framecycler/DJView/FlipBook. I'll let you know how I get on with the Applescript as I'm just unravelling how it all works and am seriously under the cosh with deadlines!!! Aye, aye, aye bring on Christmas! Cheers everyone - Have a good one! Craig Allison Digital Systems & Data Manager The Senate Visual Effects Twickenham Film Studios St.Margarets Twickenham TW1 2AW t: 0208 607 8866 skype: craig_9000 www.senatevfx.com On 18 Dec 2008, at 17:27, Greg Ercolano wrote: > [posted to rush.general] > >> Our pipeline at the moment is: we render downsized, LUT-converted >> JPEGs >> in the same setup as the main DPX files as a second output. After the >> job is done I submit a job to the renderqueue of Silverstack (called >> Cinemator) on a small Mac mini to generate quicktimes from these >> JPEGs. >> >> Silverstack can create templates for burnin, LUT conversion, >> codec, etc. >> which you can use while submitting the job to Cinemator on the >> commandline. >> >> At the moment, Cinemator is a GUI app which can be fed by a >> commandline >> command, but what I heard is that they are working on a more >> sophisticated solution. >> >> Maybe you want to have a look on it. > > Sounds interesting, but unless they have a non-gui interface, > it's not something I think Rush can control properly. > > What I'd like to do is have a way to make it easy to add any > kind of command line tool to handle QT conversions to the > submit scripts, and have a pulldown that lets one select which > converter to use, eg. ffmpeg, MetaRender, SequencePublisher, > qt-tools, etc. > > Probably what I'd do is add a "QT" tab to all the submit scripts > that has prompts for the input images and output movie, as well > as being able to select which codecs to use. All the rush submit > scripts would then be able to call a single, central script > to handle the actual QT conversion as a 'jobdonecommand'. > --Apple-Mail-30-61134651 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 I was looking at Silverstack recently and DJV-convert but they don't = quite meet my requirements as we have movie specific masking/crops = applied to the full 2k frames when they are converted to QT...

I'm currently working on = the Applescript suggestion to find/replace the contents of a Shake = "setup" script and then submitting to command line render.

Any QT additions to the = Rush submit would be helpful for the artists, as they could make use of = the farm for quick preview QT's for long shots instead of = Framecycler/DJView/FlipBook.

I'll let you know how I = get on with the Applescript as I'm just unravelling how it all works and = am seriously under the cosh with deadlines!!!

Aye, aye, aye bring on = Christmas!

Cheers everyone - Have a = good one!


The Senate Visual Effects

Twickenham Film = Studios


t: 0208 607 8866=A0

skype: craig_9000


=


On 18 Dec 2008, at 17:27, Greg Ercolano = wrote:

[posted to = rush.general]

=
Our pipeline at the moment = is: we render downsized, LUT-converted JPEGs=A0
in the = same setup as the main DPX files as a second output. After the=A0
job is = done I submit a job to the renderqueue of Silverstack (called=A0

Silverstack can create templates for burnin, LUT = conversion, codec, etc.=A0
which = you can use while submitting the job to Cinemator on the = commandline.

At the moment, Cinemator is a GUI app which can be = fed by a commandline=A0
command, = but what I heard is that they are working on a more=A0

Maybe you want to have a look on = it.

Sounds interesting, but unless = they have a non-gui interface,
it's not = something I think Rush can control properly.

What I'd = like to do is have a way to make it easy to add any
kind of command line tool to = handle QT conversions to the
submit = scripts, and have a pulldown that lets one select which
converter to use, eg. ffmpeg, = MetaRender, SequencePublisher,
qt-tools, = etc.

Probably what I'd do is add a = "QT" tab to all the submit scripts
that has = prompts for the input images and output movie, as well
as being able to select which = codecs to use. All the rush submit
scripts = would then be able to call a single, central script
to handle the actual QT = conversion as a 'jobdonecommand'.


= --Apple-Mail-30-61134651-- *** ***************************************************** *** Message #1814 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Thu, 18 Dec 2008 12:27:26 -0500 > Our pipeline at the moment is: we render downsized, LUT-converted JPEGs > in the same setup as the main DPX files as a second output. After the > job is done I submit a job to the renderqueue of Silverstack (called > Cinemator) on a small Mac mini to generate quicktimes from these JPEGs. > > Silverstack can create templates for burnin, LUT conversion, codec, etc. > which you can use while submitting the job to Cinemator on the commandline. > > At the moment, Cinemator is a GUI app which can be fed by a commandline > command, but what I heard is that they are working on a more > sophisticated solution. > > Maybe you want to have a look on it. Sounds interesting, but unless they have a non-gui interface, it's not something I think Rush can control properly. What I'd like to do is have a way to make it easy to add any kind of command line tool to handle QT conversions to the submit scripts, and have a pulldown that lets one select which converter to use, eg. ffmpeg, MetaRender, SequencePublisher, qt-tools, etc. Probably what I'd do is add a "QT" tab to all the submit scripts that has prompts for the input images and output movie, as well as being able to select which codecs to use. All the rush submit scripts would then be able to call a single, central script to handle the actual QT conversion as a 'jobdonecommand'. *** ***************************************************** *** Message #1813 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: "Abraham Schneider" Date: Wed, 17 Dec 2008 03:43:39 -0500 Greg Ercolano schrieb: > I've been trying to look for a nice cross-platform QT converter > solution. There are definitely a few commercial apps out there > (MetaRender looks like one, part of the iridas tools), > and some freeware apps as well (qt-tools, libquicktime, ffmpeg), > but many of the freeware apps are not "cross platform". >=20 > Some suggestions for command line quicktime converters I've seen: >=20 > Framecycler (SequencePublisher/MetaRender) > Nuke > Shake > Fusion > http://avidemux.org/admWiki/index.php?title=3DCommand_line_usage > djv_convert > ImageMagick+OpenQuicktime (alternative to ffmpeg) > MEncoder (http://www.mplayerhq.hu) > libquicktime + custom code > qt-tools (qt_export tool) >=20 One you haven't mentioned: Silverstack from Pomfort: www.pomfort.com Our pipeline at the moment is: we render downsized, LUT-converted JPEGs=20 in the same setup as the main DPX files as a second output. After the=20 job is done I submit a job to the renderqueue of Silverstack (called=20 Cinemator) on a small Mac mini to generate quicktimes from these JPEGs. Silverstack can create templates for burnin, LUT conversion, codec, etc. = which you can use while submitting the job to Cinemator on the = commandline. At the moment, Cinemator is a GUI app which can be fed by a commandline=20 command, but what I heard is that they are working on a more=20 sophisticated solution. Maybe you want to have a look on it. Abraham -- Abraham Schneider Senior VFX Compositor, 2 D Artist ARRI Film & TV Services GmbH Tuerkenstr. 89 D-80799 Muenchen / = Germany Phone (Tel# suppressed) EMail aschneider@arri.de www.arri.de/filmtv ARRI Film & TV Services GmbH Sitz: M=FCnchen - Registergericht: Amtsgericht M=FCnchen - = Handelsregisternummer: HRB 69396 Gesch=E4ftsf=FChrer: Franz Kraus; Thomas Till *** ***************************************************** *** Message #1812 ***************************************************** *** ***************************************************** Subject: Re: Auto Rendering of QT's on jobcomplete From: Greg Ercolano Date: Tue, 16 Dec 2008 17:34:10 -0500 matx wrote: > [posted to rush.general] > > On 2008-12-16 11:16:35 -0800, Craig Allison said: >> I'll have a look at AppleScript as a possible interim solution, > > We use an AppleScript to interface between a template Shake script and > the image sequence selected (drag drop folder onto script). After the > images are rendered by an artist they make a QT using this tools which > basically substitutes new fileins and fileouts, etc. If this is something you can call as a command, then you can make it the jobdonecommand for the job. > I'd love to see how you could incorporate a Shake script in a > shake-submit for Rush as a followup job using the variables of the > previous job. Can RUSH pass variables to another job? You can pass anything you want, but you'd need to add the operation to do it. The manual way is: submit your shake job, and lets say its jobid i