All 102.42d beta testers: here's a list of items that will be fixed
in beta3, which will be going out later today.
(Below that, a list of the 102.42dbeta2 modifications that you
probably have already.)
BETA3 FIXES
o Fixed problem with -fmt parsing that caused the
first argument past
the format string to be ignored. eg:
rush -lj -fmt "Jobid Status" tahoe spirit erie
-----
|
'tahoe' argument
was being ignored
o Sped up jobid lookups for 'rush -lj `cat jobids`' took ~30
seconds
to list 4000 jobs (whereas 'rush -lj hostname' was instant).
Replaced linear lookups using IsJobid() with JOBID_GetID()
and doing simple
integer lookups.
o Fixed problem with 'rush -lj' not clearing away hostname on
the tty if report is small, eg:
rush -lj -fmt Remarks
..would print the hostname and a CRLF, then the report was
supposed to clear
away the hostname. But if the report is small (such as in
the above, a '-' response would leave
most of the hostname behind). Small changes to handle doing
a cleol.
o IMPORTANT: Added code to prevent checkpoint writes WHILE
daemon is loaded checkpoints.
It was possible to end up loosing jobs if one restarts the
daemon while
the daemon is loading a checkpoint. (BRENT/LUMA)
o Added 'WaitForDependency:' to -ljf report (FUELVFX/PAWEL)
o Fixed duplicate entries for 'Dependency' in -ljf reports
Jobids were showing up twice because submits now assign
jobid first,
causing two registrations to happen.(FUELVFX/PAWEL)
o Mac: switched irush's menubar to use the Mac's system menu
bar.
Internals: Switched MyMenuBar from using Fl_Menu_Bar ->
Fl_Sys_Menu_Bar
Internals:
o Updated FLTK 1.3.x-svn to r9562
> Fl_Tree mods for irush2 wip
> Enabled FLTK_ABI_FEATURES
BETA2 FIXES
> 'rush -cp' can now increase or
decrease the priority of all cpus
rush -cp '*' @+5 -- increase priority by +5
rush -cp '*' @-5 -- decrease priority by -5
rush -cp +any=50@100 @+10 -- increase priority from 100 to 110
Irush makes use of this feature in the 'Jobs' and 'All Jobs'
report,
so that one can easily raise/lower the priority of several jobs
at once:
> 'rush -cp' now tries to keep the
CPUSPEC column of 'rush -lc' synchronized with priority mods
If all the tasks of a cpu specification are changed to the same
priority value,
then the cpuspec is updated. Example:
$ rush -lc
CPUSPEC[HOST] STATE FRM PID JOBTID PRI ELAPSED JOBID NOTES
tahoe=2@100 Idle/Nak - - 2497 100 00:00:00 tahoe.8 -
tahoe=2@100 Idle/Nak - - 2498 100 00:00:00 tahoe.8 -
$ rush -cp tahoe=2@100 @200 *** CHANGE PRIORITY FROM @100 TO @200 ***
tahoe.8 tahoe=2@200: priority was tahoe=2@100, now tahoe=2@200
$ rush -lc *** CPUSPEC FIELD UPDATES ***
CPUSPEC[HOST] STATE FRM PID JOBTID PRI ELAPSED JOBID NOTES
tahoe=2@200 Idle/Nak - - 2497 200 00:00:00 tahoe.8 -
tahoe=2@200 Idle/Nak - - 2498 200 00:00:00 tahoe.8 -
$ rush -cp .2497 @300k *** CHANGE PRIORITY OF ONLY ONE OF THE TWO TASKS **
tahoe.8 tahoe=2@200: priority was @200, now @300k
$ rush -lc *** CPUSPEC FIELD WILL *NOT* UPDATE ***
CPUSPEC[HOST] STATE FRM PID JOBTID PRI ELAPSED JOBID NOTES
tahoe=2@200 Idle/Nak - - 2497 300k 00:00:00 tahoe.8 -
tahoe=2@200 Idle/Nak - - 2498 200 00:00:00 tahoe.8 -
$ rush -cp .2498 @300k *** CHANGE PRIORITY SO THAT *BOTH* TASKS NOW AGREE **
tahoe.8 tahoe=2@300k: priority was @200, now @300k
$ rush -lc *** CPUSPEC FIELD NOW UPDATES ***
CPUSPEC[HOST] STATE FRM PID JOBTID PRI ELAPSED JOBID NOTES
tahoe=2@300k Idle/Nak - - 2497 300k 00:00:00 tahoe.8 -
tahoe=2@300k Idle/Nak - - 2498 300k 00:00:00 tahoe.8 -
> Added new taskcleanup algorithm
Purges stale jobs out of the tasklists more frequently, lighter load.
Rush.conf now configured to run the new cleanup every 3 hours:
taskcleanuphours 0 3 6 9 12 15 18 21
New behavior can be disabled via rush.conf with:
use_taskcleanup_opt 0
taskcleanuphours 5
> 'rush -laj' now supports large reports (solves "Message too long" errors)
The following flags were added to the rush.conf file to manage the internals for this change:
use_udptransactors 0 # disables use of new udp transactors (reverts to old behavior)
udptransactor_fragsize 512 # max size of udp transactor fragmenting (-lac,-laj,-status,etc)
udptransactor_ttl 5 # #secs udp transactors remain on server before expiring
udptransactor_choke 0 # #usecs to usleep between sending /each/ frag sent by server
> Added 'processor affinity' support for linux.
So now both windows and linux can have 'affinity=x' in the hosts file.
(Still no affinity support for OSX: Apple's operating system does not support it)
> 'rush -push' now uses file compression to allow larger files to be pushed quickly
The following rush.conf option specifies the file size at which point
compression is used. (9000 bytes is the default):
udpcompress 9000
Compression can be disabled with:
udpcompress 0
> Added ()'s to cpu specifications, eg. +any(1-4)=5@100
..which lets one specify only hosts 1 through 4 in the +any group.
Forward ported from 102.42a8b (ROMEY/ZOIC)
This breaks the network protocol, so bump from 102.42c -> 102.42d
> Added 'rush -ljc' (List Job Counters).
Every rush job now maintains 'change counters' which are incremented
whenever an aspect of the job has changed:
'frame counter' -- incremented whenever the job's frames
are added, removed, or change state
This feature allows 'watcher scripts' to /easily/ see if some aspect
of the job has changed, so that it can know if the job's info needs
to be reloaded. This helps prevent a watcher script from constantly
having to reload all info on every job to detect if anything changed.
> Root can submit jobs IF forceuid/forcegid configured AND uidrange/gidrange includes roots uid/gid
> 'rush -af' allows multiple frame ranges to be specified. eg:
rush -af 1-10 20-45
> 'rush -catlog': added 'mail.log' and 'jobs-checkpoint' as
optional filenames to view.
> rush.conf:
Added the following optional commands:
> Added 'cpuacct.dbasedir':
Usage: cpuacct.dbasedir [path] [dirmask] [filemask]
Example: cpuacct.dbasedir "/server/somewhere" 0755 0644
Example would cause cpu.acct logs to be written as:
/server/somewhere/[YYYY]/[MM]/[DD]/[HOSTNAME]-cpu.acct
---------------------------
|
Automatically generated dirs
and filename
This feature will be used by web scripts that provide
network wide utilization reports (WARNER), eg:
> Added 'net.resvauth', which allows disabling of forcing
client connections to use port numbers below 1024. (affects
unix only). Prevents problems with numerous client connections
causing 'no more reserved ports' errors.
Usage: net.resvauth [0|1]
Example: net.resvauth 0 # disables use of reserved ports (disables default)
> Added 'version=###' prefix, to allow conditional code execution
based on the rush version#.
> Added 'usercmd' to 'permit' section, to allow per-user control
of the "rush -submit" 'user [uid] [gid] [username]' submit command.
> Added the following submit commands:
commandflags <flags> -- adjusts attributes of job's command
jobstartcommandflags <flags> -- adjusts attributes of job's jobstartcommand
jobrestartcommandflags <flags> -- adjusts attributes of job's jobrestartcommand
jobdonecommandflags <flags> -- adjusts attributes of job's jobdonecommand
jobdumpcommandflags <flags> -- adjusts attributes of job's jobdumpcommand
..and equivalent command line flags:
-commandflags <flags>
-jobstartcommandflags <flags>
-jobrestartcommandflags <flags>
-jobdonecommandflags <flags>
-jobdumpcommandflags <flags>
These flags control how commands execute:
quote -- Handle quoting
expandvars -- Expand environment variables of the form ${VARNAME} at runtime
escape -- Handle escape character (\)
useshell -- Use a native shell to run the command (unix:use bash, windows: use DOS)
nolog -- (jobdonecommand only) disables creating jobdonecommand.log
done -- (jobdonecommand only) runs command only if all frames are "Done"
fail -- (jobdonecommand only) runs command only if any frames are "Fail" and rest are "Done"
donefail -- (jobdonecommand only) runs command only if all frames are "Done" or "Fail"
always -- (jobdonecommand only) always runs command when job is done (default)
off -- turns off all the above flags
So for instance:
rush -submit <<EOF
..
command python //server/path/myscript.py -scene //server/path/${RUSH_JOBID}
commandflags useshell,expandvars
..
EOF
..tells rush to use a bash or DOS shell to run the above command,
and to expand the "${RUSH_JOBID}" at render time.
o Added -fmt flag to 'rush -lj' and 'rush -laj'
This allows jobs reports to be customized from the command line, eg:
See 'rush -lj -fmt help' for the large list of all the possible field names.
These fields can also be used in the rush.conf file for these new options:
# "rush -lj"
rush.lj.format "-Status<STATUS>:-8 Jobid<JOBID>:-16 Title<TITLE>:-25 Owner<OWNER>:-14 %Done<%DONE>:5 %Fail<%FAIL>:5 FramesBusy<BUSY>:4 Elapsed<ELAPSED>:10 Remarks<REMARKS>"
# "rush -laj"
rush.laj.format "-Status<STATUS>:-8 Jobid<JOBID>:-16 Title<TITLE>:-25 Owner<OWNER>:-14 %Done<%DONE>:5 %Fail<%FAIL>:5 FramesBusy<BUSY>:4 Elapsed<ELAPSED>:10 Remarks<REMARKS>"
# "rush -lj -x" + "rush -laj -x"
rush.lj_x.format "-Status<Status>:-8 Jobid<Jobid>:-16 Title<Title>:-25 Owner<Owner>:-14 %Done<%Done>:5 %Fail<%Fail>:5 %Hold<%Hold>:5 Frames<#Frms>:5 FramesBusy<Busy>:5 Priority[Vals]<Priority>:-8 Ram:5 Progress<Progress>:-20 RenderElapsed<Rendertime>:12 Elapsed<Age>:12 Remarks<Remarks>"
# "rush -lac"
rush.lac.format "Hostname:-15 Owner:-14 Jobid:-16 Title:-15 Frame:-7 Priority<Pri>:-5 Pid:-7 Elapsed:10 Remarks"
# "rush -lc"
rush.lc.format "CpuSpec[Host]:-22 State:-11 Frame:-7 Pid:-7 JobTid:-7 Priority:-8 Elapsed:-8 Jobid:-14 Notes"
# "rush -lf"
rush.lf.format "Status:-6 Frame:-7 Try:-3 Hostname:-15 Pid:-7 Jobid:-16 Start:-14 Elapsed:-8 Notes"
o Added 'rush -jobdata' and 'jobdata' submit command
This allows generic info to be added to a job, similar but separate from 'job notes'.
o Removed unused (and undocumented) -checkpoint command
("-jobs-checkpoint" is the only ever advertised command line flag)
o Removed unused (and undocumented) jobchkcommand
o input:
> Added Edit -> Preferences -> Help Messages
to allow disabling of new submit form tooltips.
[IRUSH FEATURES: WIP]
> Cleaner headings in reports
> Column resizing is now done in the header instead of the middle of the report
> Added more right-click menu features
> IRUSH_FRAMES and IRUSH_JOBS environment variables help "Hotkey" scripts
determine which frames have been selected in Frames reports,
and whhich jobs have been selected in Jobs and All Jobs reports.
[INPUT FEATURES: WIP]
o SUBMIT SCRIPTS
o > Added FixDriveMaps() to .common.pl
All the new submit scripts run this function in their MAIN_Render()
section to remap drives.
o Added $G::rushlogdir option to .common.pl so that admin can force
all logs to be written to a single directory. (default is off)
*** TODO: DO THIS WITH ALL BIG SUBMIT SCRIPTS
*** See the logdir section of submit-afterfx.pl
o submit-afterfx: brought over ability to submit paths with spaces
from mocean mods (03/22/2012)
*** TODO: DO THIS WITH ALL BIG SUBMIT SCRIPTS
*** See use of SpaceEscape() and SpaceUnescape().
Internals:
o Updated FLTK 1.3.x-svn to r9545
> Has fixes for SGI build
|
|