CHEAT(5) Erco's UNIX Cheat Sheet CHEAT(5)
NAME
cheat - cheat sheet for unix commands
SECTIONS
Section Description
------------------ ----------------------------------------
FUNCTION KEYS How to program your keyboard's Function keys
VI TEXT EDITOR Tricks in 'vi'
EX Tricks with 'ex'
NICE Running processes 'nice'ly
GENERAL TRICKS General tricks with rlogin, mail, printing etc.
TAR Creating/extracting tar backups
DD Using 'dd': with tape devices
SED Using 'sed': the file stream editor
GREP Using 'grep': wild card search for lines of text
CUT Using 'cut': cut character/field columns
TR Using 'tr': translate characters in files
FIND Using 'find': to search for files
SORT Using 'sort': sorting files alpha/numerically
AWK Using 'awk': tricks,techniques
PERL Using 'perl': tricks,techniques
PYTHON Using 'python': tricks,techniques
CSH (C Shell) Using 'csh': tricks,techniques,compare w/sh
SH (Bourne Shell) Using 'sh': tricks,techniques,compare w/csh
WINDOW MANAGER Recovering from frozen window managers (SGI)
DBX How to use dbx (SGI)
GDB How to use gdb (gnu debugger)
LLDB How to use lldb (OSX debugger)
SYSTEMD How to use systemd
DESCRIPTION
I don't know about you, but I forget this shit all the time, and like
to have a simple reference around.
This document has sections which have ALL CAPS section headings
in the left margin (key words shown above) to make it easy to quickly
seek to a section via the '/' search option in 'More'.
For example, to zip to the AWK section, at the 'More' prompt just type
the following and hit 'return', and you'll be there:
/^AWK
Working examples are used as much as possible to let the user just 'grab'
the examples and execute them verbatim to see how they work.
If you find bugs or inconsistencies, please send me mail.
-greg
(erco@3dsite.com)
-- -- -- -- -- -- -- -- -- -- --
FUNCTION KEY PROGRAMMING (TCSH)
-------------------------------
bindkey -r f1,"ls -la\n"
-- ----------
| |
| The command programmed to the key
The name of the function key
...How to get some of those nasty, hard to get characters:
\n # LINEFEED (LF)
\r # RETURN (CR)
\t # tab
\b # back space
\\ # backslash
\xxx # 3 digit octal
VI TEXT EDITOR
--------------
Here's some 'vi' text editor magic. These examples can also be used in
shell scripts with 'ex':
Search................................. /abc
Search multiple words.................. /\(abc\|def\)
Global replace......................... :g/old text/s//new text/g
Global replace btwn lines.............. :10,20g/old text/s//new text/g
Global uppercase....................... :g/./s//\U&/g
Global lowercase....................... :g/./s//\L&/g
Remove ^M's from IBM PC files.......... :g/<CONTROL-V><CONTROL-M>/s///g
Insert '>' to every line in file....... :g/^/s//>/g
Insert '>' to the end of the file...... :.,$s/^/>/
Reverse all lines in file.............. :g/^/m0
Delete all blank lines................. :g/^$/d
Delete blank or all-white lines........ :g/^[ TAB]*$/d
| |
| TAB: a 'tab' character
|
A single space
Alphabetize lines 10 - 20.............. :10,20!sort
Numerical sort lines 10 - 20........... :10,20!sort -n
SETUP FILE
VI's setup file is called ~/.exrc, and you can stick in all sorts
of custom keystrokes. Note that to get control characters, such as
^[ and ^E, you must type Control-V, followed by the control character.
My favorite ~/.exrc settings:
set autoindent -- auto indenting
set shiftwidth=4 -- indent 4 spaces with '>>'
set report=1 -- report changes to >=1 lines
set shell=/bin/csh -- use the CSH to run ! cmds
map q ^Y -- 'q' is scroll line up
map z ^E -- 'z' is scroll line down
map ; ^B -- ';' page up
map , ^F -- ',' page down
map = :vi#^[ -- switch to last edited file
map s :!spell -l < % \| more -- spell check current file
map ^N :n^[ -- edit next file
map F :'a,.!fmt^[ -- 'F' will reformat range
I hate hitting the goddamn control key to move around. I edit large
text files often, so I like 'q' and 'z' to scroll up and down. I like
';' and ',' to move up and down full pages.
To make it easier for me to do spell checking, I setup 's' to invoke
the spell checker. Since 's' is easy to accidentally hit, I don't have
it automatically execute; you have to hit 's' and return.
I like '=' to toggle between the last file I was editing, and ^N to
advance editing the next file in a chain, when 'vi file1 file2 file3'
was used on the command line.
Since VI has no automated paragraph formating, I programmed 'F' to
reformat text into ragged right paragraphs using the fmt(1) program.
To use it, go to the top of the text you want to reformat, and hit
'ma' to mark the start. Move to the bottom of the text to reformat,
then hit 'F'. This will reformat the text between the mark you set,
and the cursor positon. See 'man fmt' for more info on 'fmt'.
WILD CARDING
Search for a string using wildcards..In this case, search
for any string that starts with 'Fre', and is followed by any
number of characters, followed by 'Flin' (which could find
'Fred Flintstone' among other things):
/Fre.*Flin
EX
--
All of the above VI commands CAN ALSO BE USED IN SCRIPTS by piping the
commands to 'ex', the line editor that lays underneath vi. This allows
you to use all the ':' commands you are familiar with in vi. The following
example shows how to reverse all the lines of the file 'foo':
echo 'g/^/mo0\012wq' | ex foo
-------======
| |
| \012=Linefeed, wq=write and quit
|
The 'reverse all lines in a file' command from above
Note that '\012wq' needed to be appended to the command; the \012 is an
octal linefeed so ex can run the 'wq' that follows, which tells ex to
'write' out the file and 'quit'.
NICE
----
Nice will run your program at a lower priority. Often, this makes little
difference in actual execution time, and makes the machine much more
friendly to use for humans (quick response to key presses). If the machine
is doing nothing else, it will run low priority jobs as fast as possible
anyway. Tars are particularly good to run at low priority, because they
are usually waiting for the tape drive anyway.
/bin/nice -19 omni -fov 108 -reso 2048 1536 infile.rla outfile.rla
------------- ----------------------------------------------------
| |
| Your program as you would normally run it, with args
|
'nice' runs your program at lowest priority with -19
One thing confusing about 'nice' is that there are too many versions of
'nice'. So as not to confuse you, just trust me, and stick with /bin/nice.
GENERAL TRICKS
--------------
OK, a lot of this stuff is really old. Mail? cu? finger?
You must have gray hair by now if you still know how to use those.
RLOGIN
Suspending an rlogin. The tilde must be the first character on a new
line, so hit ENTER first if you are unsure, then type the following:
~^Z (type '~' then ctrl-Z)
Killing an rlogin. Again, tilde must be first on the line. This closes
the connection:
~.
NOTE: These '~' commands are the same in rlogin(1), rsh(1), and cu(1).
MAIL
To 'mail' someone a file, you can say:
cat file | Mail fred,jack # send 'file'
cat file | Mail fred,jack -s "Here's a file" # send with a 'subject'
To see EVERYONE's login names and full names, try:
ypcat passwd | awk -F: '{ printf("%-10s %s\n",$1,$5); }' # NIS
awk -F: '{ printf("%-10s %s\n",$1,$5); }' /etc/passwd # non-NIS
To see someone's login name based on their real name (or vice-versa):
finger flintstone # all matches for 'flintstone' shown
# gives phone numbers, etc.
ENVIRONMENT
To see your path in an easy-to-read fashion, try:
echo $PATH | tr -s ':' '\012'
PROCESS MANAGEMENT
To see an easier-to-read process listing than 'ps', try:
piss # Shows all processes and how they're parented
piss fred # Show all processes owned by fred
kill -STOP 5196 # suspend a process (like ^Z)
kill -CONT 5196 # continue a process from suspend
Note: 'piss' is available from:
http://seriss.com/people/erco/unixtools/piss
PRINTING (HARD COPY)
Usually, most sysadmins configure lp(1) for users to print files.
On SGI's, lp(1) can be used as a simple tool for printing text files
and/or images:
lp myfile # print a file
lp file1 file2 # print several files
cat file1 | lp # print thru a pipe
lp image.rgb # print an RGB image
TAR
---
A note about tar some people don't know: you can always ^Z (suspend)
a tar, and again be able to restart it without worry. You can ^Z and
'fg' a tar all day, and nothing will go wrong because of it. This goes
for tape reading and tape creation.
Note all examples show that you should use 'nice' when running tar
to avoid unnecessary system load:
/bin/nice -19 tar tvf /dev/xtape # Listing of a tar tape
/bin/nice -19 tar cvf /dev/xtape . # Backup the current dir
/bin/nice -19 tar xvf /dev/xtape # Extract entire tar tape
/bin/nice -19 tar xvf /dev/xtape ./tmp/myfile # extract 'myfile'
/bin/nice -19 tar xvf /dev/xtape `cat mine.lst` # Extract files in 'mine.list'
cat myfile.tarlist | tarsize # Total up file sizes from tar listing
How to run a tar in the background, redirecting the tar's output to
a file, and also to the screen (so you can see the output without
tailing the file):
csh -c '/bin/nice -19 tar cvf /dev/xtape .' |& tee ../junk.tarlist &
Note: 'csh -c' is necessary to allow seeing the listing's
progress in real time, so tar's stdout isn't buffered.
Without the 'csh -c', the listing will be buffered, and
will thus not show in real time.
How to copy entire directory trees to another machine, preserving
file creation times, modes, owner info, etc. using tar:
cd fromdir; tar cBf - . | rsh remote "(cd todir; tar xBfpv -)"
How to tar to a remote machine through dd (works with suns too)
pebble: tar cvfb - 20 . | rsh tahoe dd of=/dev/xtape obs=20b
DD
--
To make copies from one tape drive to another WITHOUT using a disk
for intermediate storage:
dd ibs=20 if=/dev/xtape | rsh remote obs=20 of=/dev/xtape
SED
---
How to print all lines from file up to a line that ends with 'STOP':
cat file | sed '/STOP$/q'
How to print lines 4 thru 10 from a file:
cat file | sed -n '4,10p'
How to globally replace a string in a file:
cat file.old sed '/oldstring/s//newstring/g' > file.new
How to delete line #12 from a file (this modifies the file):
sed -i 12d somefile.txt
How to delete lines 12 thru 20 from a file (this modifies the file):
sed -i 12,20d somefile.txt
How to remove blank lines from a file:
cat file | sed '/^$/d' > file.new
Search for a string using an embedded wildcard..print any line that
contains 'Thom' followed by any number of characters, followed by 'Ros':
cat /etc/passwd | sed -n '/Thom.*Ros/p'
To do the OPPOSITE of the above (note the ! infront of the 'p', and
how it must be escaped with '\'):
cat /etc/passwd | sed -n '/Thom.*Ros/\!p'
GREP
----
Print any lines that contain the string 'fred':
ps -elf | grep fred
Print any lines that DO NOT contain the string 'root':
ps -elf | grep -v root
Print any lines that contain 'jimbo' OR 'rman':
ps -elf | egrep 'jimbo|rman'
Print any lines that contain 'Jimbo', 'jimbo', 'JIMBO', etc:
cat file | grep -i jimbo
CUT
---
The 'cut' tool can cut character columns, or field delimited columns
sometimes easier in syntax than awk.
Cut out character columns 42 thru 53 from an 'ls -la' listing:
ls -la | cut -c42-53
Cut out the fifth field (-f5) delimited by the ':' character (-d:) from
the password file (basically just show full name field):
cat /etc/passwd | cut -f5 -d:
Improved version of above; show only the field delimited by ',' so that
only the full names are shown:
cat /etc/passwd | cut -f5 -d: | cut -f1 -d,
TR
--
'tr' translates one range of characters to another. It's also good for
converting one character to another throughout a file, which makes it
good for converting horizontal lists to vertical ones.
The following converts a DOS file to unix text file:
tr -d '\015' < text.dos > text.unix
The following converts a mac file to unix:
tr '\015' '\012' < mac.txt > text.unix
The following converts all ':' characters in your path to line feeds,
making the path easier to read:
echo $PATH | tr -s ':' '\012'
The following converts all 'words' consisting of the letters A-Z, a-z
into a vertical column of words, removing punctuation and spaces:
echo This is a test, you know. | tr -cs "[A-Z][a-z]/" "[\012*]"
The following forces all text on input to lower case on output:
cat upper_and_lower_case | tr "[A-Z]" "[a-z]" > all_lower_case
To remove all carriage returns from a file, effectively turning a file
into one long, single line:
cat file | tr -d '\012'
FIND
----
'find' is actually a really useful program to find files in directory
trees, although it's syntax is a little odd. It has quite a bit of
flexibility, and is worth learning. Here's some examples:
Show full pathname to all files in the current directory, and all
subdirs (Try this, it's like 'ls -R', but easier to parse w/pgms):
find . -print
Search several directories recursively for all occurances of
the file 'foo' in several directories:
find /usr /prod1 /prod2 -name foo -print
Search several directories recursively for all files that end in '.c':
find /usr /prod1 /prod2 -name '*.c' -print
Find also has arguments that control the output of filenames based on
their modification times, etc. The following shows all files that
were created today in the current directory (and all subdirs):
find . -mtime 0 -print
Show all files that are symbolic links:
find . -type l -print
UHOH! Watch out, this one is destructive. RECURSIVELY REMOVES
ALL FILES THAT ARE SYMBOLIC LINKS starting with the current directory:
find . -type l -exec rm {} \;
-- -- --
| | |
| | Terminates the -exec arguments.
| These are replaced by each filename found.
Execute the rm command.
Here's an alias I like to have in my .cshrc; 'findtext string' will
recursively invoke an egrep on all files from the current directory on
down for 'string', printing the filenames of all files that match:
alias findtext "find . -exec egrep '\!*' {} /dev/null \;"
...so if you type 'findtext fred', it invokes the command:
find . -exec egrep 'fred' {} /dev/null \;
SORT
----
To alphabetically sort a file:
cat file | sort
To alphabetically sort a file IN REVERSE:
cat file | sort -r
To sort a file numerically (the file contains numbers instead of words):
cat file | sort -n
To sort a file by a certain column of data, use sort's '+' option:
ls -la | sort -n +4 # sort an 'ls' listing by file size
Assuming you have a file containing three columns of ascii numeric data
(X Y Z) you can sort the file by any column you want. In this case, sort
by 'Z':
cat unsorted_xyz_data | sort -n +2 > xyz_sorted_by_z
I sometimes like to see 'ls -la' listings sorted by file size. I use
the following alias in my .cshrc on SGI systems.. you may need to change
the +4 to make it work on other platforms, since ls output differs:
alias lss '/bin/ls -la \!* | sort -n +4'
AWK
NAWK
----
(Update: Since perl has pretty much become a standard on all unix machines
now, and it kicks so much ass on awk/nawk it isn't funny, try to learn
perl for writing new programs instead.)
I love Nawk. I've written entire software systems using nawk.
It always comes with Unix bare bones, so it's great for sysadmin tools,
as well as user apps. It's a lot like C, really!
How to pad a frame number with four zeroes (Wavefront format):
echo $frame | awk '{printf("%.4d", $1)}'
How to extract the third and ninth column from an 'ls -la' listing:
ls -la | awk '{ print $3 " " $9 }'
How to sum all the values in a column of numbers (this example
sums all the numbers in column 5 in an SGI's ls listing):
/bin/ls -l *rla | awk 'BEGIN {sum=0} {sum+=$5} END {print sum " bytes"}'
How to do a for/next loop in awk (faster than the csh):
echo | awk 'BEGIN { for(t=1; t<1000; t++){ printf("Loop# %.5d\n",t); }}'
How to print columns of text in nicely formatted columns
cat /etc/passwd | awk -F: '{ printf("%-10s %s\n",$1,$5); }'
How to reverse all lines in a (short) file:
cat /etc/passwd | \
awk '{ foo[i++]=$0 } END { for (t=i-1;t>=0;t--) { print foo[t] } }'
How to print out your path in an easy to read fashion:
echo $PATH | nawk '{split($0,arr,":"); for(i in arr) print arr[i]}'
How to loop thru an associative array:
arr[one] = 1;
arr[two] = 2;
for ( i in arr ) { print arr[$i] }
How to walk the lines of a file:
while ( (getline s <filename) > 0 ) # how to open & read a file
{ print s }
How to load an array with the output of a command:
for ( tlines=0; "ls -la" | getline; tlines++ )
arr[tlines] = $0;
How to do a file existance check:
# DOES FILE EXIST?
function Exist(filename)
{
# DO A TEST OPEN
_err = ( getline _junk < filename );
close(filename);
if (_err == -1) return(0); else return(1);
}
Here are some functions that awk [NAWK] supports:
index(string, search); # Returns char pos'n of search
# 0 means search not in string
length(string); # Returns length of string
split(string, array [, sepchars]); # Splits string into an array
# sepchars defaults to FS
str = sprintf("fmt", expr); # Formatted string
str = substr(str, start[, length] ); # Take a sub string
yn = sub(regex, replace, str); # Single replace [NAWK]
nsub = gsub(regex, replace, str); # Global replace [NAWK]
err = system(cmd); # run command, get errcode
position = match(str, regex); # 0 if no matches [NAWK]
"/bin/id" | getline id_string; # run 'id', capture output
# (returns 0 when no more input)
Here are AWK's variable built-ins:
ARGC / ARGV - like C's args (ARGV[0..ARGC-1])
FILENAME - current input filename
FNR - input record number in current file
FS - input field separator (default blank)
NF - number of fields in input record
NR - input record number since beginning (total so far)
OFMT - output format for numbers (default %.6g)
OFS - output field separator (default blank)
ORS - output record separator (default newline)
RLENGTH - length of string matched by regex in match
RS - input record separator (default is newline)
RSTART - beginning position of string matched by match
SUBSEP - array subscripts sep of form [i,j,..] (default "\034")
AWK limits:
100 fields
3000 chars per input record
3000 chars per output record
1024 chars per field
3000 chars per printf() string
400 chars max literal string
400 chars in character class
15 open files
1 pipe
double-precision floating point
Things to watch out for:
o 'for (var in arr) { }' does not walk the list in the order created!
o '#!/bin/nawk' scripts invoked as 'foo -help', the -help may actually
get parsed by nawk (IRIX 4.0.5, 5.3)
If you have GAWK, you have built-in case conversion. But in AWK/NAWK you
have to do it yourself, and it's pretty non-obvious. This is the best way
I've seen.. right off alt.lang.awk, with some mods:
function upper(s,l,o,i)
{
l = "abcdefghijklmnopqrstuvwxyz"
for(i=1; i<=length(s); i++)
{
c = substr(s,i,1);
if (index(l,c) != 0) { o = sprintf("%s%c",o,index(l,c)+64); }
else { o = o c; }
}
return(o);
}
function lower(s,l,o,i)
{
l = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
for(i=1; i<=length(s); i++)
{
c = substr(s,i,1);
if (index(l,c) != 0) { o = sprintf("%s%c",o,index(l,c)+96); }
else { o = o c; }
}
return(o);
}
Here's how to get rid of leading and trailing white space:
gsub("^[ \t]*", "", s); # nuke leading white
gsub("[ \t]*$", "", s); # nuke trailing white
Here's how to fold white space spans to single spaces:
gsub("[ \t][ \t]*", " ", s);
How to parse out the text between <>:
gsub(".*<", "", s);
gsub(">.*", "", s);
How to parse out the text between "":
# PARSE OUT FILENAME FROM #include "filename"
sub("[^\"]*\"", "", s);
sub("\".*", "", s);
How to type out a large ascii file with delays between each screen:
nawk 'BEGIN { for (t=0; getline < "foo"; t++) { print $0; if ((t % 40)==39) system("sleep 3"); } } '
PERL
----
Perl kicks ass. These days you should write everything in perl
instead of awk. Back in the day, awk was all we had.
How to do the 'Wavefront' style 4 digit padding for frame numbers:
perl -e 'printf("%04d\n",$ARGV[0]);' $frame
IBM PC files have ^M/^J at the end of each line, rather than just
the unix ^J's. To remove the unwanted ^M's, you can use:
perl -e 'while(<>){s/\015//;print $_;}' < ibmpc_file > unix_file
Common gotchyas:
> All commands must end with a ;
> '==' only tests NUMERIC equality,
> 'eq' only tests STRING equality.
> =~ can be used to do regex matches, ie. if ( $s =~ /^root.*System/ )
> Subroutine calls are prefixed with & in perl4
> Variable use must be preceded with '$', EVEN in assignments.
> Use chop() to get rid of trailing CRLFs.
> $ means a string or numeric value
> @ means an array or list, eg. @arr = ( 1, 2, 3);
> % means a hash, eg. %foo = ( "One" => 1, "Two" => 2 );
> . can be used to concatenate strings ("this and" . "this")
> {} is mandatory around commands in if/while/for/foreach/etc blocks
> `` (backticks) will retain crlfs (unlike csh) for command output
> To print to files, use 'print FD "text";' and 'printf(FD "text");'
Variables similar in the CSH and PERL:
CSH PERL Description
----- ----- ----------------------------------------------
$var $var Expands variables
$#var $#var Number of elements in array (-1=empty in perl)
$status $? Status of last command
$$ $$ Current process pid
How to loop thru command line arguments (even if they contain spaces):
#!/bin/perl
printf("You supplied %d arguments: %s\n", $#ARGV, join(", ",@ARGV));
for ($i=0; $i<=$#ARGV; $i++)
{ print "\t$i) Working on: $ARGV[$i]\n"; }
You can also walk the arguments this way:
#!/bin/perl
foreach $i ( @ARGV )
{ print "\tWorking on: $i\n"; }
Closest thing to a 'switch' statement. This is considered optimal:
# Read line of input, including CRLF. Note == tests ignore the \n(!)
print "Type something: "; $s = <STDIN>;
if ($s == 1) { print "You entered one\n"; }
elsif ($s == 2) { print "You entered two\n"; }
else { print "No grok <$s>\n"; }
How to 'include' (source) another file, ie. to bring in subroutines,
variable settings, etc:
require "foo.pl"
How to do a 'foreach' loop:
# USES DEFAULT VARIABLE '$_'
foreach ( 0 .. 5 )
{ print "$_) $_ of 5 lines\n"; }
# USES $i
foreach $i ( "one" "two" "three" )
{ print "Value=$i\n"; }
How to do a 'while' loop:
$count = 0;
while ( $count < 10 )
{
print "$count\n";
$count++;
}
How to do an 'if' statement:
if ( $a == $b )
{ print "Print this if true\n"; }
else
{ print "Print this if false\n"; }
How to test for:
if ( -e "~/.cshrc" ) # ..the existence of a file
if ( -z "~/.cshrc" ) # ..the file is zero size
if ( -d "~/rman" ) # ..the path is a directory
How to read a line of input into a variable:
$VAR = <STDIN>; # read a line, including \n.
chop($VAR); # (how to remove trailing \n)
How to make a wrapper script (without worrying about wildcard args):
#!/bin/perl
exec('/bin/echo', 'Your arguments are:', @ARGV);
Open and read a file:
$file = "/etc/fstab";
open(F, $file) || die "$file: $!";
for ($t=0; $buf = <F>; $t++)
{ printf("%03d: %s", $t, $buf); }
close(F);
Open and write a file. In this case, print the current date 20 times:
require "ctime.pl";
$file = "foobar";
open(F, ">$file") || die "$file: $!";
for ($t=0; $t<20; $t++)
{ printf(F "%03d: %.24s\n", $t, &ctime(time())); }
close(F);
Open a file for append. Just change the open() in the above to:
open(F, ">>$file") || ..
--
How to declare (and call) a function:
# PROMPT FOR INPUT
# usage: &Prompt("Type something: ");
sub Prompt
{
local($s);
print @_;
$s = <STDIN>;
chop($s);
return($s);
}
# USE THE FUNCTION
$stuff = &Prompt("Type something: ");
print "You typed: $stuff\n";
How to read key/value pairs from a file, using ':' as field delimiter:
while (<STDIN>)
{
chop;
s/^[ \t]*//; # nuke leading white
@fields = split(/:/,$_,2); # first two fields
$fields[1] =~ s/^[ \t]//; # nuke leading white
$fields[1] =~ s/[ \t]*$//; # nuke trailing white
printf("%-12s <%s>\n", "<".$fields[0].">", $fields[1]);
}
How to declare and print an array:
local(@array) = ( "one", "two", "three" );
foreach ( @array )
{ print "Value=$_\n"; }
How to print large amounts of text (similar to the CSH):
print <<"EOF";
Lots and lots of text.
Embedded variables are expanded between the EOF's,
ie: the current process id is $$.
EOF
NOTE: As in the CSH, the EOF must be flush left.
The above technique can be used to set variables and in function calls:
$var = <<"EOF";
one two three.
Four five six.
EOF
system(<<"EOF");
ls -la;
echo That was fun;
cat /etc/passwd
EOF
Hashes can be initialized inline as:
%foo = (
"Name" => "Fred Flintstone",
"Wife" => "Wilma",
"Kid" => "Pebbles",
"Dog" => "Dino",
"Address" => "301 Cobblestone Way",
"Email" => "fred@yahoo.com"
);
# PRINT HASH SORTED ALPHABETICALLY
foreach $i ( sort ( keys ( %foo ) ) )
{ printf("KEY='%s', VALUE='%s'\n", $i, $foo{$i}); }
PYTHON
------
How to do 'Wavefront' style 4 digit padding for frame numbers:
set frame = 12
python -c 'import sys; print "%04d" % int(sys.argv[1])' $frame
-------------------------------------------
IBM PC files have ^M/^J at the end of each line, rather than just
the unix ^J's. To remove the unwanted ^M's, you can use:
#!/usr/bin/env python
import sys
for line in sys.stdin.readlines():
sys.stdout.write(line.replace('\r',''))
Common gotchyas:
o Code blocking is done with indenting, not braces
o No trailing semicolons needed
o All if/loops end in ':', eg. "while (something):"
o for loops aren't really for loops, they're foreach loops (see below)
o No printf.. use: print "Twelve is %d" % 12
o No sprintf.. use: var = "Twelve is %d" % 12
o No switch.. use: if (..): .. elsif (..) ..
o range(start:end) -- 'end' is NOT inclusive, eg. range(1:5) yields [1,2,3,4]
o import's arguments do not need quotes
o No pass-by-reference (eg. functions can't change parameters that are numbers or strings)
o When calling instances of classes, 'self' is secretly passed as the first argument
o 'import'ing .py files can try to create pyc files automatically. (potential for races)
o There are many implementiations of popen().. be sure to use the 'right' one
o Old 'import string' no longer needed; built in strings have same methods
[..WORK IN PROGRESS..]
Variables similar in the CSH and PERL:
Csh Perl Python Description
----- ------- ---------- ----------------------------------------------
$argv[] $ARGV[] sys.argv[] Command line arguments
$var $var var Expands variables
$#var $#var len(var) Number of elements in array (-1=empty in perl)
How to loop thru command line arguments (even if they contain spaces):
#!/usr/bin/env python
import sys
print "You supplied %d arguments:" % (len(sys.argv)-1)
for i in range(1, len(sys.argv)):
print "ARGV[%d]='%s'" % ( i, sys.argv[i] )
Simple functions:
def Add(a,b,c):
return(a+b+c)
print "ADD(1,2,3)=%d" % Add(1,2,3)
Functions returning multiple values:
def AddAndMultiply(a,b,c):
return(a+b+c, a*b*c)
(add, mult) = AddAndMultiply(1,2,3)
print " ADD=%d" % add
print "MULT=%d" % mult
Printing to stderr:
print >>> sys.stderr, "This is stderr" # This is changing in Python 3.x
sys.stderr.write("This is stderr\n")
A note about for loops. 'for' loops are really 'foreach' loops; it loops
through lists, and does not do conditional tests. This means you can't do
common things like changing the loop variable within the loop, a common thing
to do when doing argv[] parsing, eg. in C:
import sys
for i in range(1, len(sys.argv)):
if ( sys.argv[i] == "-file" ): # e.g. "-file filename"
i += 1 # NO! CHANGE WONT AFFECT NEXT ITERATION
filename = sys.argv[i]
else:
print "%s: Unknown argument" % sys.argv[i]
..you'd have to use a while loop instead:
i = 1;
while i < len(sys.argv):
if ( sys.argv[i] == "-file" ): # e.g. "-file filename"
i += 1 # OK
filename = sys.argv[i]
else:
print "%s: Unknown argument" % sys.argv[i]
i += 1
[..WORK IN PROGRESS..]
CSH (The C Shell)
-----------------
How to loop thru command line arguments (even if they contain spaces):
#!/bin/csh -f
echo You supplied $#argv arguments: $*:q
foreach i ( $*:q )
echo Working on: $i
end
How to redirect:
myprogram > file # ...stdout to a file
( myprogram > /dev/tty ) >& file # ...stderr to a file
myprogram >& file # ...stdout AND stderr to a file
To run a process correctly in the background:
prog >& err.log < /dev/null & # stdout/err to log
( prog > stdout.log ) >& stderr.log & # separate stdout/err logs
How to do a 'case' (or 'switch') statement:
switch ( $var )
case 'a':
case 'A':
echo "var is A"
breaksw
case 'b':
case 'B':
echo "var is B"
breaksw
default:
echo "var isn't A or B"
breaksw
endsw
How to 'include' (source) another file (to set environment variables):
source filename
How to do a 'foreach' loop:
foreach i ( *.rla )
echo $i
end
How to do a 'while' loop:
set count = 0
while ( $count < 10 )
echo $count
@ count ++
end
How to do an 'if' statement:
if ( "$variable" == "test" ) then
echo Print this if true
else
echo Print this is false
endif
How to test for:
if ( -e ~/.cshrc ) # ..the existence of a file
if ( -z ~/.cshrc ) # ..the file is zero size
if ( -d ~/rman ) # ..the path is a directory
How to extract parts of filenames:
set vari = "/usr/dd/bin/foo.12.c"
$vari:h = "/usr/dd/bin" # get 'Head' of pathname (path)
$vari:t = "foo.12.c" # get the 'Tail' of pathname (filename)
$vari:r = "/usr/dd/bin/foo.12" # get the Root of pathname (no extension)
$vari:e = "c" # get Extension of pathname only
How to read a line of input into a variable:
setenv VARNAME "$<" # read a line into VARNAME
setenv VARNAME "`line`" # this works too, but slower
How to disable '*', '?' and '[]' expansion (globbing):
set noglob # disable '*' expansion
unset noglob # enable '*' expansion
How to make a wrapper script (without worrying about wildcard args):
#!/bin/csh -f
real_program $*:q
How to parse arguments:
#!/bin/csh -f
set sfrm = -1
set efrm = -1
set file = ""
set argz = ""
@ i = 1
while ( $i <= $#argv )
switch ( $argv[$i] )
case '-s':
@ i ++
set sfrm = $argv[$i]
breaksw
case '-e':
@ i ++
set efrm = $argv[$i]
breaksw
case '-file':
@ i ++
set file = "$argv[$i]"
breaksw
default:
set argz = ( $argz "$argv[$i]" )
breaksw
endsw
@ i ++
end
echo sfrm is $sfrm
echo efrm is $efrm
echo file is $file
echo argz is $argz
SH (The Bourne Shell)
---------------------
How to loop through command line arguments (even if args contain spaces):
#!/bin/sh
echo You supplied $# args: $@
for i in "$@"; do
echo Working on: $i
done
How to redirect:
myprogram 1> file # ...stdout to a file
myprogram 2> file # ...stderr to a file
myprogram 1> file 2>&1 # ...stdout AND stderr to a file
myprogram 1> stdout.log 2> stderr.log # ...stdout AND stderr to separate files
How to do a 'case' (or 'switch') statement:
case "$var" in
a|A)
echo "var is A"
;;
b|B)
echo "var is B"
;;
*)
echo "var isn't A or B"
;;
esac
How to 'include' (source) another file (to set environment variables):
. filename
How to do a 'for' loop:
for i in *.rla; do
echo "<$i>"
done
How to do a 'while' loop:
count=0
while [ "$count" -lt 10 ]; do
echo $count
count=`expr $count + 1`
done
How to do an 'if' statement:
if [ "$variable" = "test" ]; then
echo true
else
echo false
fi
How to run a command, and act conditionally on its exit code:
if some_command; then
echo Command exited with zero
else
echo Command exited with non-zero
fi
How to test for:
if [ -f ~/.cshrc ] # ..the existence of a file
if [ -z ~/.cshrc ] # ..the file is zero size
if [ -d ~/rman ] # ..the path is a directory
if [ "$val" -lt 10 ] # ..if val is less than 10
if [ "$val" -gt 10 ] # ..if val greater than
if [ "$val" -eq 10 ] # ..if val equal
if [ "$val" -ne 10 ] # ..if val not equal
if [ "$val" -le 10 ] # ..if val less or equal
if [ "$val" -ge 10 ] # ..if val greater or equal
How to do variable math (old style sh):
val=`expr $val + 1` # val = val + 1
How to do variable math (new bash style):
let val=${val}+1 # val = val + 1
let val=${total}%5 # val = total % 1
How to set a variable to the output of a command:
me=`logname`
How to set a variable to the output of a complex command with a 'here' document:
output=$(write fred << EOF
This is a test.
So is this.
EOF
)
How to extract parts of filenames:
vari="/usr/greg/src/foo.12.c"
dirname $vari -> "/usr/greg/src" # get directory name
basename $vari -> "foo.12.c" # get filename
basename $vari '.c' -> "foo.12" # get filename without extension
How to read a line of input into a variable:
VARNAME=`line` # invokes /bin/line
How to disable '*', '?' and '[]' expansion (globbing):
set -f # disable '*' expansion
set +f # enable expansion
How to make a wrapper script (without worrying about wildcard args):
#!/bin/sh
real_program "$@"
My favorite prompt setup in ~/.profile:
# ERCO PROMPT
export PROMPT_COMMAND='echo; echo "`date \"+%a %D %T\"` `pwd`"'
export PS1='[\u@\h] \! \$ '
..which gives a shell prompt that looks like this:
Sat 05/13/17 11:52:40 /root
[root@zoar] 136 #
WINDOW MANAGER
--------------
If your window manager is 'hung' (you can't close or open windows
with the mouse, or logout, etc) you can try the following steps:
1) RECOVER. The following key sequence will attempt to recover the
window manager.
CTRL ALT SHIFT !
2) The following keyboard sequence will restart the window manager:
CTRL ALT F12 / # hold keys down in this order
3) If none of the above works, try to rlogin to the machine from another
host, and restart the window manager.
If that doesn't work, reboot the son of a bitch. That'll show it. Try
to get a sysadmin to reboot it for you. Don't use the reset button
unless you have to. Many of the newer machines (Indy's, O2's) will
do a clean shutdown if you hit the power button. Use reset only if
the whole system is locked.
DBX
---
[OK, dbx is really old -- the SGI debugger.
Probably nobody cares anymore, but leaving this here anyway, because
"history, man". In case someone is (still) using an SGI to debug
natively compiled C and C++ programs.]
Assumes you compiled with debugging enabled and the executable
is not stripped:
you@host: foo
core dumped ; bad way to start the day
you@host: dbx `which foo` ; let's find the problem
(dbx) file foo.C
(dbx) where ; shows calling hierarchy (stack trace)
(dbx) w ; view source (if compiled with '-g'!)
(dbx) up ; move up 1 in calling hierarchy
(dbx) w ; view source
(dbx) stop at 37 ; setup a stop at line 37 of foo.C
(dbx) run ; run the program
Stopped at line 37 ; dbx stops this far
(dbx) step ; now step thru program a line at a time
37: a = b;
(dbx) step ; another line
38: *c = 12; ; here's the line of code that killed it
Segmentation fault - core dumped ; dbx senses coredump
(dbx) print c ; what was 'c' at the time of assign?
(nil) ; that figures.. an assignment thru NULL
(dbx) quit
you@host:
GDB
---
It's assumed you compiled your program with debugging enabled (-g)
and your app is not strip(1)'ed. Here's how to debug a program that
already dumped core:
you@host: ./foo -arg1 -arg2
Segmentation fault (core dumped)
you@host: gdb ./foo ./core # Debug your app './foo' using the core file './core'
GNU gdb 4.17.0.11 with Linux support
[bla bla]
(gdb) where
#0 0x8048780 in main (argc=1, argv=0xbffffc44) at foo.C:5
#1 0x4008ccb3 in __libc_start_main (main=0x8048750 <main>, argc=1, argv=0xbffffc44,
init=0x8048604 <_init>, fini=0x80487ec <_fini>, rtld_fini=0x4000a350 <_dl_fini>,
stack_end=0xbffffc3c) at ../sysdeps/generic/libc-start.c:78
(gdb) list
1 #include <stdio.h>
2 int main(int argc, char **argv)
3 {
4 for ( int t=0; t<200; t++ )
5 argv[t][0] = 1;
6 printf("%d %s\n", argc, argv[1]);
7 return(0);
8 }
(gdb) print t
$1 = 1
(gdb) printf "%d\n",t
1
(gdb) quit
Or, you can run your program within the debugger, so that when it
crashes, gdb is already attached to it:
you@host: gdb foo # Load your program in the debugger
GNU gdb 4.17.0.11 with Linux support
[bla bla]
(gdb) run -arg1 -arg2 # Runs your app with arguments -arg1 and -arg2
[app runs until it crashes or until you hit ^C..]
(gdb) where
[..see above example..]
###
### Using GDB with C++ and breakpoints
###
You can set up GDB so that it stops at a certain function, so that you
can inspect variables, then continue execution. Even set a conditional
test so that the app stops at your breakpoint when the condition is matched:
you@host % gdb keyboard
(gdb) break MyWindow::handle(int) <-- set breakpoint
Breakpoint 1 at 0x804d8a5: file keyboard.cxx, line 54.
(gdb) run <-- run the program
Starting program: ..
..
Breakpoint 1, MyWindow::handle(int) (this=0x8080598, msg=16) at keyboard.cxx:54
54 if (msg==FL_MOUSEWHEEL)
(gdb) condition 1 msg==8 <-- set a condition
(gdb) cont <-- continue
Continuing.
<-- hit any key on your keyboard
<-- and the app will break
Breakpoint 1, MyWindow::handle(int) (this=0x8080598, msg=8) at keyboard.cxx:54
54 if (msg==FL_MOUSEWHEEL)
(gdb) print msg <-- condition is true
$1 = 8
###
### Using gdb to debug a threaded application
###
You only need to know two extra commands to debug a threaded app:
info threads -- shows the active threads
thread <thread#> -- selects a different thread to be the 'current thread'
With the threaded program suspended (or at a breakpoint), use the
'info threads' command to show all active threads, and use the
'thread <thread#>' command to choose which thread to be current:
(gdb) info threads
2 process 19339 thread 0x3fab 0x9002b888 in semaphore_wait_signal_trap ()
* 1 process 19339 thread 0xd03 0x9002b978 in semaphore_wait_trap ()
|__________
|
\|/
(gdb) thread 2
[Switching to thread 2 (process 19339 thread 0x3fab)]
Each thread will have it's own execution stack that 'where' will display,
depending on which thread is currently selected with the 'thread' command.
The 'up' and 'down' commands to move through the selected thread's stack
as usual.
You can either have the threaded app running under the debugger,
or attach to an already running process with 'gdb -p <pid>' as usaul.
###
### Using gdb to inspect memory
###
info proc mappings -- shows the heap start/end points
info variables -- lists all global variables
info files -- shows address ranges of program and loaded lib's data/heap, text, etc
x/1000bc 0x41e78000 -- hex dump of 1000 bytes starting at 0x41e78000, including ascii
x/1000s 0x41e78000 -- hex dump of 1000 bytes as strings starting at 0x41e78000
info symbol 0x14b62830 -- shows the variable/symbol whose address is 0x14b62830
LLDB
---
It's assumed you compiled your program with debugging enabled (-g)
and your app is not strip(1)'ed. Here's how to run a program in the
debugger, so when it crashes you can analyze it:
you@host: lldb unittests.app/Contents/MacOS/unittests # Debug the program
[bla bla]
(lldb) run
[bla bla]
Process 67973 stopped <-- crashed
[bla bla about crash]
(lldb) bt <-- generates a backtrace (similar to gdb 'where')
[shows stack trace]
(lldb) list <-- list source code of stopping point
[shows source code, just like gdb list]
(lldb) help <-- reasonable help info
[shows commands]
(lldb) ^D
SYSTEMD
-------
A small cheat sheet of systemd commands I find myself using.
SysV init vs. systemd:
SysV Init Systemd equivalent
=========================== ===============================
/etc/init.d/foo start systemctl start foo.service (".service" suffix is often optional)
/etc/init.d/foo stop systemctl stop foo
/etc/init.d/foo restart systemctl restart foo
/etc/init.d/foo status systemctl status foo
chkconfig foo off systemctl disable foo
chkconfig foo on systemctl enable foo
killall foo systemctl kill foo.service
ls /etc/init.d ls /usr/lib/systemd/system /etc/systemd/system
cat /etc/init.d/foo cat /usr/lib/systemd/system/foo.service -- vendor file
cat /etc/systemd/system/foo.service -- sysadmin's optional override of vendor file
Managing daemons started on boot:
systemctl start sshd.service -- Start it running now (doesn't affect next reboot)
systemctl stop sshd.service -- Stop it running now (doesn't affect next reboot)
systemctl enable sshd.service -- Enable service on next boot (does not affect if service is running now)
systemctl disable sshd.service -- Disable service on next boot (does not affect if service is running now)
systemctl status sshd.service -- See status of service; process hierarchy, last 10 lines in log
systemctl status -- See status of 'everything' (tree view)
Note: the '.service' suffix is optional *IF* there's no
other unit with that same name (e.g. foo.service vs foo.socket)
To see all the services and their status:
systemctl -t service -- Lists all of type service (similar to 'chkconfig --list')
To see a graphic horizontal time bar chart of how long it took
all the services to start during the last boot:
systemd-analyze plot > /tmp/foo.html -- open /tmp/foo.html in a browser.
Use browser's ^F (search) to find services of interest
Other interesting stuff:
echo Hello there | systemd-cat -- similar to logger(1)
systemctl is-enabled sshd.service -- see if service enabled for running
systemd-cgls -- systemd cgroups report -- shows process groupings as a tree,
similar to 'ps fax'
systemd-cgtop -- like top(1): shows cpu/mem use of all services (includes children!)
Note: cpu/mem/io accounting info *only* tallied for services
that have these values enabled in their .service file, e.g.
$ cd /usr/lib/systemd/system
$ egrep 'CPUAccounting|MemoryAccounting|BlockIOAccounting' sshd.service
CPUAccounting=1
MemoryAccounting=1
BlockIOAccounting=1
For details, see 'man 5 systemd.resource-control'
To enable the web GUI interface to systemd called 'cockpit':
1) yum install cockpit
2) yum install cockpit-ws
..then, whenever you want to use it:
1) systemctl start cockpit.socket -- turns the service on. (see below)
2) open a browser and go to: http://localhost:9090/
You'll be prompted to login.
Note on #1: The service automatically turns itself OFF
if no one logs into it within 10 minutes (for security reasons)
Some useful links for systemd:
Video of Lennart Poettering describing systemd at a RedHat summit (45 mins):
https://www.youtube.com/watch?v=S9YmaNuvw5U
Poettering's written description of why systemd was created:
http://0pointer.de/blog/projects/systemd.html
Someone's article on systemd daemon management:
https://www.usenix.org/system/files/login/articles/login_june_06_jedrzejewski-szmek.pdf
Top level document on systemd for all linux distros:
https://www.freedesktop.org/wiki/Software/systemd/
Wikipedia page overview:
https://en.wikipedia.org/wiki/Systemd
The systemd source code:
https://github.com/systemd
AUTHOR
------
erco@netcom.com erco@3dsite.com erco@seriss.com