Rush - After Effects Issues

RUSH RENDER QUEUE - AFTER EFFECTS ISSUES
(C) Copyright 1995,2003 Greg Ercolano. All rights reserved.
V 102.42 10/26/04


	Using Rush with AfterFX 6.0

Problems with running After Effects on multiple processors under Rush.
Only use one entry per comp in the AE6 Render Queue
'Use storage overflow' and 'Skip existing frames' should be un-checked in the AE6 Render Settings
'Use Comp Frame Number' should be checked in the Output Module Settings.
You can leave the 'Output Path' blank in the Rush Submit-AfterFX form.
Steps for submitting a job in Rush
Getting AfterEffects rendering on two or more procs under Windows
OSX: Getting the INIT_Processeses [sic] error message


	Prevent more than one aerender from running on Windows machines.

Currently there's a bug in AE6 (under Windows only) where running more than one AERENDER process per machine under the Rush service either hangs or crashes aerender, causing either or both processes to fail with this 'Application Error' from AfterFX.EXE:

Adobe is trying to fix this in a future release or update. This shows up ONLY on multi-processor /Windows/ machines using After Effects for rendering.

You can work around the problem in either of these ways:

Reconfigure the Rushd service to run on the desktop using these instructions, then you can use all processors.

Limit your After Effects renders to only use one processor (see below)

#1 is probably the most desireable for farm machines, since you can get use of both processors for rendering. However, it does mean those machines need to be left logged in at all times with the 'Rushd' running in a stowed DOS window, instead of as a service.

To prevent Rush from assigning both processors to an AfterFX render, you can use the 'Ram' value in the submit-afterfx 'Advanced Options' to prevent more than one aerender from running on the machines.

To do this, assuming your render farm is configured in Rush to have 2048 of ram on each machine, then submit your job to the farm with the 'Ram' value in the submit form set to 2047.

This will prevent rush from starting more than 1 AERENDER process per host, because Rush will think the AfterFX job needs all of that ram to run each AERENDER process, preventing it from starting another AERENDER on the same box.

You can then submit other kinds of renders (eg. maya) to these machines using a Ram value of 1, and they'll be able to get on because of the 1MB the afterfx job leaves behind (2048-2047=1MB available).

Another way is to change the number of cpus in the rush hosts file to 1 for the dual proc machines, but that will affect non-afterfx renders too (eg. maya), so this is not recommended unless you're using Rush exclusively for afterfx rendering. The more flexible solution is the RAM solution above.

Only use one entry per comp in the AE6 Render Queue

If you have two or more instances of the same comp in the AE6 Render Queue, AERENDER will only use the settings from the first one. To prevent confusion, it is advised you only have one instance per comp in the AE6 Render Queue.
BUG NOTE: Even *un-checked* AE6 Render Queue items are recognized by the command line version of After Effects, so be sure to clean out your project's render queue before submitting it to Rush. (thanks to Kevin and Mark at The Orphanage)

'Use storage overflow' & 'Skip existing frames'
should be un-checked in the AE6 Render Settings

These two "Render Settings" should be un-checked if you're rendering specific comp sequences: 'Use storage overflow' and 'Skip existing frames':

If 'Use Storage Overflow' is checked, it can cause confusion if there's a permission problem with the output directory; AE will indicate it rendered OK, but there will be nothing in the output directory because it's failing over to the alternate storage dir.
If 'Skip existing frames' is checked, it will not render frames that already exist on disk, meaning it won't re-render frames. It is only useful to have 'Skip existing frames' checked *if* you submit the job with the 'Comp Name:' field blank, so AE6 can render only missing frames when running on multiple processors.
(Thanks to Mark at The Orphanage for helping figure this one out)

'Use Comp Frame Number'

You should make sure 'Use Comp Frame Number' is checked in the output module settings, eg:

If you don't, every frame in the rush job will try to write to frames numbered frame.0001.ext.
Since this is an AE6 specific feature, any 5.5 project submitted to the render farm needs to be opened in AE6, have the box checked, and then be re-saved named proj_AE6rndr.aep. Warning: this file will not be backwards compatible with AE5.5, so appending the render suffix is crucial.
(thanks to Kevin and Mark at The Orphanage)

You can leave the 'Output Path' blank in the Rush Submit-AfterFX form.

If you leave it blank, the output directory specified in your AEP scene file will be used, i.e. 'Render Queue | Output Module'
The 'Output Path' field is only needed if you want to override the path you have specified in your AE6 Render Queue's 'Output Module' for the comp.

Steps for submitting a job in Rush

Double-click on the shortcut to bring up the submit-afterfx interface.

Optionally specify a Job Title.

For the Scene Path, navigate to your AE project using the Browse button. If it is your first time submitting, you may have to manually type in //server/volume name in order to get a directory listing. Avoid using a mapped drive letter (z:/yada/yada) to get to your project, to avoid problems with UNC paths, eg. when people log out, and the drive maps disappear.

Leave the 'Output Path' blank, unless you want to override the path specified in your AE6 Render Queue entry.

Enter the exact 'Comp Name' that is queued to render. If you have more than one comp to render, you can submit separate rush submissions for each one.
Or, you can leave the 'Comp Name' field blank to render everything configured in the AfterFX Render Queue. In which case, be sure to enable the 'Skip existing frames' flag in the Render Settings. For more info on leaving 'Comp Name' blank, click on the '?' for this field in the submit form.

Set your batching to 5 or 10. This will render the comp in 5 frame increments and helps speed up the job by limiting the number of times AE has to start up.

For "Cpus", enter the list of machines you intend to render on. Click on the "?" next to that field for a description of what can be entered here.
To find out which hostgroups are available on your network, (+any=10, +farm=10, etc.), go into Irush, and in the menubar, click on Network | All Hostgroups.

(Thanks to Kevin at The Orphanage for helping write this list)


	Getting Both Procs Rendering With AE.

This workaround is necessary to work around a bug in After Effects where it can't run more than one instance of itself from a service without hanging.

To get AE rendering on both processors under Rush on your farm without having AE crash or hang, you can reconfigure the Rushd service to run from a DOS window instead of as a Service, and AE will render on both processors correctly.

To do this:

Disable the Rushd service from coming up automatically.
Go into the control panel 'Services', and change the Rushd service from 'Automatic' to 'Manual', and 'Stop' the service.
Configure 'farm' machines to automatically login as the rush user.
Using 'TweakUI', a Microsoft tool, you can configure the machine to, on reboot, automatically login as the user you had the Rushd service configured to run as.
You can download and install TweakUI from:
http://www.systemsmedic.com/TweakUI.htm
Once installed, login as the user you had the Rushd service configured to run as, go into Control Panel | TweakUI, and click the 'Logon' tab, and enable it to automatically login as the user you had the Rushd service configured to run as.
Use a StartUp script to automatically start Rushd in a DOS window
You can use this script to automatically start Rushd when the machine automatically logs in.
One way to do this: Under Windows 2000, go into:
..and click on the user you configured the Rushd service to run as, then click on "Profile", and set the "Logon Script" to the path to the above 'startup.bat' script.
That's it. Reboot to test.

Just leave the window manager logged in as the Rush user, and renders will run normally. You will be able to render AE on both processors through rush without AE crashing.

NOTES:

When rendering afterfx jobs, the AE interface will pop open during rendering. This is "normal".
If you logout the desktop, the rushd will stop running. This is because you've now configured rushd to only run when someone is logged in to a desktop.
It is advised if you plan to log out a farm machine to do 'rush -getoff' before logging out, so that renders are requeued cleanly, and
Rush will act completely normally for all rendering (Maya, Shake, etc) when set up this way. In those cases, no GUIs will appear on the screen, because Maya/Shake/etc are true command line applications.


	OSX: Getting the INIT_Processeses [sic] error message.

aerender will fail on an OSX machine if the 'aerender' process is running as a user that is not the same as the user logged in on that machine's window manager. The error message is usually:

INIT_Processeses(), could not establish the default connection 
to the WindowServer. Abort

This can be demonstrated with 'telnet' or 'rsh' by logging in as a non-root user that is different than the user logged into the window manager, and then trying to invoke 'aerender' to render a comp; you get the same error.

The problem is After Effects command line rendering still tries to open its GUI on the window manager, and if the permissions don't match, AE fails with an error.

The only known solutions:

Make 'aerender' run as root.
Root will have access to the window manager, no matter who is logged in. This can be done in various ways, including simply making 'aerender' setuid root:
Note: doing this will cause the rendered images to be owned by root.
Modify the render script to use 'sudo' to run the aerender command as root. Note: doing this will cause the rendered images to be owned by root.
Make a script that determines who is logged in, then force the the aerender command to run as that user using sudo, or similar. Note: doing this will cause the rendered images to be owned by the random users that were logged in at the time.

Another possible way to attack this would be to figure out how to override OSX's window manager authorization system. The behavior you're looking for would be similar to the X windows command 'xhost +', but for OSX's window manager. So far, there is no known way to do that in OSX.