From: Greg Ercolano <erco@(email surpressed)>
Subject: Rush 103 major upgrade released 11/12/13
   Date: Wed, 13 Nov 2013 16:11:33 -0500
Msg# 2364
View Complete Thread (1 article) | All Threads
Last Next
Hi All,

    Rush 103 has been officially released as of 11/12/13.

    This is a major upgrade, and will be the new release stream for Rush.
    Bug support for the older 102 releases will continue for another year or so,
    but new features will only be added to 103.

    Companies with 102 can eval the new version, and can upgrade to 103
    for $20/host. Inquire for details and quotes: erco@(email surpressed)
    Upgrading also ups your support for +3 years.

    New customers who made their first purchase after January 2013
    already have 103 and do not need to upgrade.



	Also, the main docs for 103.xx are here:

103 Future Goals

    In addition to the features and fixes listed above, future goals will be
    for the 103 release to include a rewrite of the Irush interface to use a
    'tree' design, and to have it be more customizable, so as to disable
    certain unwanted features some companies consider dangerous or impractical,
    and hook in new features and pipeline processes.

    Also, some web administration scripts will be included to take advantage
    of the new centralized cpu accounting logs so that one can more easily
    collect rush accounting data, and generate host utilization and per-user
    job utilization reports via dynamic cgi-bin scripts that use a python API
    to manipulate the accounting data.

103 Feature Highlights

      The current 103 release includes several python submit scripts, in addition
      to all the perl submits. The perl scripts will remain in future releases,
      but all new/active development will be focused on the python submits.

      Render wranglers can more easily increase/decrease the overall priority
      of jobs, even multi-priority and 'staircased priority' jobs.

      There's a new "+/- Priority" setting in the Irush 'Jobs' and 'All Jobs' reports:
      By just highlighting the job(s) you want to change, you can then use the +/-
      button to increase/decrease the priority of those jobs. Details:

      Due to new features added to the backend (see the new -fmt flag below),
      Irush can now let users add/remove columns to the Jobs and All Jobs report
      by right clicking on the column headings to add/remove fields.

      Irush now has more context menu features for the various reports.
      You can highlight jobs, frames, hosts, cpus and then right click
      to make changes to your selection.

      When you double-click on jobs, it makes the job the current jobid.

      But if you hold down Alt while double-clicking, it will also automatically
      show the Frames report for that job.

      Or, if you want Irush to /always/ show the Frames report when double-clicking a job,
      you can turn on the "Auto Frames Report" preference under Edit | Preferences | Auto Frames Report.
      When this flag is on, double-clicking a job always shows the frames report automatically,
      and you can /prevent/ this automatic report by holding down Alt.

      Many features have been added to the backend 'rush' command:

          'rush -laj' and 'rush -lajf' modified to use a combo of UDP (to find job servers)
          and TCP (to pull potentially huge reports) to solve issues with large reports.
          (Will probably backport this fix to 102.xx)

           This lets the systems administrator configure a script that is run BEFORE each render executes
           to test if file system mounts are in place, and if not, the script can either try to fix them,
           or disable the machine from rendering.. or whatever failover logic you can come up with.
           This can be configured in the rush.conf file with the new mountcheck_cmd. Details:

           This is a script you can configure in the rush.conf file to run on boot to check
           if the machine is ready for Rush to use. This can be used to check that mounts and
           networking are in place, and if not, hold back rush from starting until they are.
           You can put any logic into this script that you want, including commands that try
           to fix known problems, or send warning emails, etc.

         * RUSH_FIRST_FRAME and RUSH_LAST_FRAME environment variables are passed to
           all commands, including the job's render command, jobstart/done/dump commands.

         * 'rush -args' -- Run several rush commands in one 'rush' instance:

         * 'rush -commandflags' -- Control how rush runs the render command (in a shell, quote handling, etc)

         * 'rush -drivemaps' -- List DOS drive mappings

         * 'rush -pathconvert' -- scripts can use this to use the rush pathname conversions in the path_convert file

         * 'rush -tail' -- Useful esp. under windows to do a live tail on a log (eg. the rushd.log)

         * 'rush -ljc' (-ljc == "List Job Changes")
           This new command lets one easily determine which aspect of jobs have changed
           in a low latency way. This is useful for 'watcher' scripts and GUIs that want to
           present realtime data from the queue without reloading /everything/ from the queue,
           so that only those jobs that have changed can be loaded.
           This report shows counters that change when some aspect of the job has changed:

               o 'frame' counter: changes when frames in a job have changed, either
                  when frames are added, removed, or have changed state.

               o 'cpus' counter that changes when cpus are added/removed

               o 'job info' counter that changes when some other aspect of the job changes,
                 such as the job's title, ram limit, the job's run state, etc.

    * Added new 'setenv' and 'unsetenv' commands to rush.conf,
      so that the admin can preset certain environment variables
      for renders. Examples:

	  os=windows setenv   PYTHONPATH "/some/special/dir"  # Windows only
	  os=linux   unsetenv DISPLAY                         # Linux only
	  os=all     setenv   SPM_HOST "foo@bar"              # all platforms

      * Added rush.conf 'cpuacct.dbasedir' command that lets one write cpu logs
        "real time" to a file server.

         Usage: cpuacct.dbasedir [path] [dirmask] [filemask]
       Example: cpuacct.dbasedir "/server/somewhere" 0755 0644

       Example would cause cpu.acct logs to be written as:

			     Automatically generated dirs
			     and filename

	This feature will be used by web scripts that provide
	network wide utilization reports (WARNER), eg:

  * 'rush -submit': Added the following submit commands:

     commandflags           <flags> -- adjusts attributes of job's command
     jobstartcommandflags   <flags> -- adjusts attributes of job's jobstartcommand
     jobrestartcommandflags <flags> -- adjusts attributes of job's jobrestartcommand
     jobdonecommandflags    <flags> -- adjusts attributes of job's jobdonecommand
     jobdumpcommandflags    <flags> -- adjusts attributes of job's jobdumpcommand

   * rush -commandflags            <flags>
   * rush -jobstartcommandflags    <flags>
   * rush -jobrestartcommandflags  <flags>
   * rush -jobdonecommandflags     <flags>
   * rush -jobdumpcommandflags     <flags>

     where <flags> can be:

     quote      -- Handle argument quoting in the command string
     expandvars -- Expand environment variables of the form ${VARNAME} at runtime
     escape     -- Handle escape characters (\)
     useshell   -- Use a native shell to run the command (unix:use bash, windows: use DOS)
     nolog      -- (jobdonecommand only) disables creating jobdonecommand.log
     done       -- (jobdonecommand only) runs command only if all frames are "Done"
     fail       -- (jobdonecommand only) runs command only if any frames are "Fail" and rest are "Done"
     donefail   -- (jobdonecommand only) runs command only if all frames are "Done" or "Fail"
     always     -- (jobdonecommand only) always runs command when job is done (default)
     off        -- turns off all the above flags

  * Added -fmt flag to 'rush -lj' and 'rush -laj' commands.
    This allows jobs reports to be completely customized from the command line.

   * And much, much more. See the release notes and features links above for full details.

Last Next