From: Greg Ercolano <erco@(email surpressed)>
Subject: Re: line of rush.submit too long!
   Date: Mon, 09 Jul 2012 19:00:53 -0400

On 07/09/12 15:04, Kevin Sallee wrote:
> I'm using python. I guess i can get it running pretty fast (even if i 
> never used the pickle module, i've heard of it already and know more or 
> less how it works). If you have a small example it would help get things 
> done faster, but if not i'll do it

Hi Kevin,

	Here's one way without pickling, which simply writes the data
	to a file as valid python code, and then at render time it
	'sources' the datafile with the python execfile() command.

	However, the 'pythonic' way is to use the pickle module.
	I haven't used it either; I prefer to save the data as
	ascii in Key/Value pairs, so the data is readable to humans,
	and supply my own load/save routine for that.

	But the following makes use of python's own interpreter
	to do the work for us, which keeps the code simple, and
	the data file is still quite readable.

----------------------------------------------------------------- snip

#!/usr/bin/python

# Python example showing how to submit a rush job with data
# (too large to pass as command line arguments) via a datafile
# in the job's log directory.
#

import os,sys

### SUBMIT SECTION
if len(sys.argv) == 1:
    print "Submitting job.."
    logdir   = "/net/tmp/logdir"
    datafile = logdir + "/datafile.py"
    if not os.path.isdir(logdir):
        os.mkdir(logdir, 0777)

    # Data
    fp = open(datafile, "w")
    fp.write("""
scene    = "/some/veeeeeeeeeeeeeeeeeery/loooooooooooooooooooooooooooooooooong/path/to/foo.ma"
imagedir = "/some/veeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeery/loooooooooooooOOOOOOOOOOOOOOOOooooooooooong/path/to/imagedir"
etc      = "..a lot more data here in the form of a single looong line of text.."
etc      = etc + "that continues on and on and on.."
""")
    fp.close()

    # Submit job
    fp = os.popen("rush -submit", "w")
    fp.write("""title           LONG_LINE_TEST
                ram             1
                frames          1-10
                logdir          """ + logdir + """
                command         python """ + sys.argv[0] + " -render " + datafile + """
                cpus            +any=10@1
              """)
    ret = fp.close()
    if ret != None: sys.exit( ret >> 8)
    sys.exit(0)

### RENDER SECTION
if sys.argv[1] == "-render":
    import time
    execfile(sys.argv[2])       # load contents of datafile
    print """--- Working on frame """ + os.environ["RUSH_FRAME"] + """
           scene=""" + scene + """
        imagedir=""" + imagedir + """
             etc=""" + etc
    sys.stdout.flush()
    time.sleep(10)
    sys.exit(0)
----------------------------------------------------------------- snip