Hi Greg;
So we've finally had some time to put our thoughts together on what we'd
like to discuss with you re: Rush.
Our main focus is on filtering/sorting and affecting change regarding
priority post-submit.
With filtering and sorting, we'd like to be able to quickly and easily
extract specific attributes on specific groups of jobs. For example, we
may want to see all the jobs in the queue for user 'bob', or all the
jobs for project 'toronto'. To take that a step further, we may want
all the jobs for project 'toronto' that have been submitted by 'bob',
etc. I think that what you've already outlined in terms of the '-fmt'
option you're working on and the implementation of some sort of string
hash to the 'Notes[]' is going to be a good step forward. It would be
really nice if we could also pass sorting/filtering options. Of course
this type of filtering can be done from the Rush output but it seems to
make sense to us that it could be done internally to Rush, since that
information is already available to you (in a database?) for a simple
select. I think an internal sort/filter is likely faster than a Perl
script also and it would enable us to avoid data we're not interested
in. Since we're dealing with fairly large volumes of jobs/data, any
speed efficiencies we can get are going to be welcome.
Regarding priority, we're finding adjusting priority after the fact a
bit cumbersome. Ideally, we'd like the ability to apply changes (on
priority and/or cpu spec) at the job level, rather than having to
identify and take action at the task id level. Typically, we really
don't care about the task level as we just want to move the entire job
up or down in the priority list. Ultimately, we want to build something
where we can affect priority at the project/shot/scene/user levels.
We're not expecting you to handle this but we need a clean way to
identify these jobs with our own tools and then be able to apply blanket
changes. So for example we may want to identify all narnia/dbx100 shots
and move them to the highest priority because we need that shot out
asap. Any number of screening possibilities could be used to identify
the job list we're interested in affecting. Maybe you have some ideas
on how we might best accomplish this. It would also be nice if we could
generate 'priority'-centric reports. Perhaps we want to see all jobs at
'high' priority. (We've implemented a low/med/high priority enumeration
that the users can select from, so 'high' has a specific value dependent
upon the host group the job is running under). In effect, this would be
more of a priority status report than a per-job report.
A couple of quick questions:
i) if a job is submitted with a RAM spec of 2G and there are two
machines available where the first has 2G of RAM and the second has 4G,
which machine will get selected under your current queueing algorithm?
ii) when priority is adjusted on a job with stair stepping defined, how
does this affect the stair steps?
I got your mail regarding the new upgrade, so if you could send the
download details, we'll likely do an upgrade. Do you have a timeline on
when the -fmt and Notes[hash] functionality will be available?
We've got a couple of weeks now where we'll have some breathing room but
then we're back into the thick of it again with additional Narnia work.
It would be great if we could get a version with the new features we've
discussed. We can take a few machine from the farm to do some beta
testing before an official release if that helps.
Berj is away for a couple of days but will be back on Thursday. He and
I are tied up for the morning and possibly early afternoon but would you
be free for a conference call say around 4:00pm our time (1:00pm your time)?
Cheers,
Dan
|