书摘: Linux Shell Scripting Cookbook
Refs:
1. Linux Shell Scripting Cookbook(2nd Edition)
Table of Contents:
Preface
Chapter 1: Shell Something Out
Chapter 2: Have a Good Command
Chapter 3: File In, File Out
Chapter 4: Texting and Driving
Chapter 5: Tangled Web? Not At All!
Chapter 6: The Backup Plan
Chapter 7: The Old-boy Network
Chapter 8: Put on the Monitor's Cap
Chapter 9: Administration Calls
Index
说明：本文为书摘，例子多为在原书基础上的修改，为了让篇幅不太庞大，例子尽量缩为一行，使书摘结构更清晰
刚又花半个多小时学习了Markdown，感觉实在太好用，再也不用为从Geany里拷贝过来shi一样的缩进而忧愁了，
这里有chap 1-2, 其余部分还在读，将以markdown发布。
正文：
Preface$
Chapter 1: Shell Something Out
1. Bash (Bourne Again Shell); #(root) and $(non root); #!/bin/bash (Shebang), `bash script.sh`
2. ";" is like a newline in bash
3. Printing in the terminal (`echo` & `printf`)
1. echo: "some $text"(some ***),(!!! thing) , 'some $text'(some $text)
2. echo: "-n" no newline; "-e" convert "\t\n" etc
3. `printf "%-10s %4.2f\n" price 4.24; "-" for [left align]
4. Colored output:
Text: reset = 0, black = 30, red = 31, green = 32, yellow = 33, blue = 34, magenta = 35, cyan = 36, and white = 37
Back: reset = 0, black = 40, red = 41, green = 42, yellow = 43, blue = 44, magenta = 45, cyan = 46, and white=47
Usage: echo -e "\e[1;42m \e[1;31m Green Background, red Text \e[0m" 
4. Variables and Environment Variables
1. cat /proc/$PID/environ | tr '\0' '\n' ('\0' null character); `echo $PATH | tr [=:=] '\n'`
2. to get $PID: pgrep geany ===> 1420
3. length=${#var}
4. `if [ $UID -ne 0 ]; then echo "you are not root"; else echo "you are root"; fi`
5. Modify the Bash Prompt String
1. cat ~/.bashrc | grep "PS1"
2. \u ==> expands to username
3. \w ==> expands to the current working directory
4. \h ==> expands hostname
6. Math with the shell
1. let
let sum=a+b && echo $sum (a=4, b=6) ==> 10
2. [ math expr ]
sum=$[ a+ b ]; sum=$[ a + 10 ]
3. (( math expr ))
sum=$(( a + b))
4. expr
sum=`expr 24 + 40`
sum=`expr 42 + $a`
5. bc (decimal places, base conversion...)
echo "420 * 04 / 42" | bc
echo "scale=2; 3/8" | bc
7. File descriptors and Redirection
1. [0: stdin], [1: stdout], [2: stderr]
2. ">" redirection, ">>" redirection and append (stdout)
3. "2>", "2>>" (stderr)
4. "&>", "2>&1" (stdout & stderr)
5. Trash Can(Black Hole): /dev/null
6. cat file.txt | tee out.txt | cat -n
7. echo "hello" | tee - - - ("-" stdin), so we got 4 "hello"s
8. Read a file as stdin
cat <<DONE > output.txt
9. Customize file descriptor
1. exec (read mode, write with truncate mode, write with append mode)
2. e.g.
exec 4>>log.txt; echo "hello" >&4; echo "world" >&4; cat log.txt
3. cat<log.txt ("cat<log.txt" or"cat < log.txt", both fine)
8. Arrays and Associative Arrays
1. Arrays
1. array_var=(1 2 3 4 5)
2. echo $array_var <==> echo ${array_var[0]} (echo $array_var[0] works wrongly!!!)
3. echo ${array_var[*]}, echo ${array_var[@]}
4. echo ${array_var[$index]}
5. echo ${#array_var[*]}, echo ${#array_var[@]}
2. Associate Arrays
1. declare -A ass_array
2. ass_array=([gnat]=124 [tang]="name") && echo ${ass_array[tang]} && echo ${ass_array[*]}
3. echo ${!ass_array[*]} ==> output all indexes
9. Visiting Alias
e.g. alias install="sudo apt-get install"
10. Grabbing info about the terminal
1. tput: initialize a terminal or query terminfo database
1. tput cols/lines/longname
2. tput cup 100 100 # set cursor position
3. tput setb/setf NUM # set background/font color, NUM: 0~7 (I think it's better than "/e[1;32")
e.g. tput setb 2 setf 7 && echo "green background && white text" # you should "setb" first then "setf"
4. tput bold; tput smul; tput rmul # bold, set underline, remove underline
5. tput ed # I dont understand this...
2. stty: change and print terminal line settings
1. stty -echo # Fantastic way to joke others, 整人的好方法！！
2. stty echo # resume
11. Getting and Setting dates and delays
[你大爷的Fork Bomb!!! 不多抱怨，继续]
1. In Unix-like System: 1970-01-01 00:00:00 UTC( Epoch or Unix Time)
2. date; date +%s; date "+%s"; date --date "Sun Feb 2 18:48:51 CST 2014" +%s (specify date/time)
3. date "+%d %B %Y"
4. Set date/time: date -s "2 Feb 2014 18:48:51" (i prefer not try it)
6. Count time: start=$(date +%s); end=$(date +%s); diff=$((end - start)); echo "passed $diff seconds"
5. Formats:
Weekday: %a(Sun) or %A(Sunday)
Month: %b(Feb) or %B(February)
Day: %d(02)
Date in Format(mm/dd/yy): %D(02/02/14)
Year: %y(14) or %Y(2014)
Hour: %I(06) %H(18), I prefer 24 hour than 12 hour
Minute: %M
Second: %S
Nano Second: %N
Epoch Unix Time in seconds: %s
6. A good Example:
1 #!/bin/bash 2 #Filename: sleep.sh 3 echo -n Count: 4 tput sc 5 count=0; 6 while true; 7 do 8 if [ $count -lt 40 ]; 9 then 10 let count++; 11 sleep 1; 12 tput rc 13 tput ed 14 echo -n $count; 15 else exit 0; 16 fi 17 done
12. Debugging the Script
1. bash -x script.sh
2. Shebang hack: #!/bin/bash -xv # enable debugging
13. Functions and Arguments
1. fname() { echo -n Arg1: $1 " "; echo Arg2: $2; echo Args: $@; return 0;} && fname how are you
2. $@ v.s. $*, we prefer the former, $*(expands as "$1c$2c$3", where c is the first character of IFS), 百度到的IFS说明，by the way, 转载应注明地址
3. The Recusive Fucs:
1. F() { echo $1; sleep 1; F hello;} && F hello # better than the one on the book
2. Fork Bomb(不要在自己电脑上试试！！！完全没看懂，在自己电脑上试试看，结果回到了解放前！即上文的“你大爷。。”处）
ForkBomb: `:(){ :|:& };:` # 我是没看懂，如果尝试，请先保存在编辑文档。。
We can write a recursive function, which is basically a function that calls itself, It infinitely spawns processes and ends up in a denial-of-service attack.
4. Exporting Functions: export -f fname
5. Reading the return status of last command: cmd; echo $? (exit status)
6. Passing arguments to commands
14. Reading the output of a sequence of commands in a variable
1. Filters: cmd1 | cmd2 | cmd3 e.g. `ls | grep "os" | cat -n > output.txt`
2. Subshell Method: e.g. `cmd_output=$(ls | cat -n)
3. Back tick/ Back quotes: e.g. cmd_output=`ls | cat -n` # I use this a lot, as you can see in this paragraph
4. Subshell bonus:
1. Spawning a separate process with subshell: e.g. `pwd; (cd /usr; ls); pwd`
2. Subshell quoting to preserve spacing and the newline character: out=$(cat file.txt)
15. Read
1. read -n num_of_chars variable: `read -n 4 year`\
2. read -s password # nonecho mode
3. read -p "enter your username: " username # read with a message/prompt
4. read -t seconds var # read with a timeout
5. read -d delim_char var
16. Repeat func:
1. repeat() { while true; do $@ && return; done;}; repeat what_you_want_to_do
2. revised edition: repeat() { while :; do $@ && return; done;}; repeat what_you_want_to_do
3. repeat() { while :; do $@ && return; sleep 30; done;};
4. e.g. repeat wget -c www.sourceforge.com/somefile.tar.gz
17. File Separators and Iterators
1. IFS => Internal Field Iterator
2. CSV => Comma Separated Values
3. e.g. `data="how,are,you"; oldIFS=$IFS; IFS=,; for item in $data; do echo item: $item; done; IFS=$oldIFS
4. echo {a..z}; echo {1..420};
5. for((i=0; i<24; i++)){ echo $i;}
6. while condition; do something; done;
7. until condition; do somthing; done;
18. Comparisons and tests
1. if condition; then do_something; else if condition; then do_something; else do_something; fi;
2. [ condition ] && something_else; [ condition ] || something
1. -gt, -lt, -ge, -le # greater/less than/equal
2. -a, -o # and/or
3. -f, -d # file/dir
4. -x, -e # executable/exists
5. -c, -b # char/block devices
6. -w, -r # writeable/readable
7. -L # symlink
3. e.g.
`if [ -e /bin/bash ]; then echo exists; else echo not exists; fi`
`if [ -e "/bin/bash" ]; then echo exists; else echo not exists; fi` ==> these two are the same
4. String comparason
1. [[ $str1 == $str2 ]], [[ $str1 = $str2 ]] # same
2. [[ $str1 != $str2 ]]
3. [[ $str1 < $str2 ]]; [[ $str1 >$str2 ]]
4. [[ -z $str ]]; [[ -n $str ]] # empty/non-empty
19. My Notice(Gnat's notice)
1. echo something } #bad ==> echo soemthing; }
2. do;# bad ==> do :;
[ to be continued ]
就不花时间在这里手工缩进了。
Chap 2: Have a Good Command
1. Concatenating with Cat
1. `cat somefile.list`; `someoutput | cat`
2. Getting rid of blank lines: `cat -s file.txt`
3. -T(Display Tab as ^T); -n(line numbers); -b(line numbers, exculding blank lines)
2. Recording and Playing back of terminal session
1. script -t 2> timing.log -a output.session
2. scriptreplay timing.log output.session
3. Finding files and file listing
1. `find path`; equals to `find path -print`
2. `find path -print0`, delimed by '\0'
3. RegExpr:
1. `find path -name "*.txt"`; or we can use "-iname"(ignore case)
2. `find path \( -name "*.txt" -o -name "*.pdf" \)`
3. `find path -path "*/gnat/*"`
4. `find path -regex ".*\(\.py\|\.sh\)"`;
5. or ignore case when applying regexpr, "-iregex";Dont memorize it, check it when you use them
6. `find path ! -name "*.txt"`; excluding
7. -maxdepth, -mindepth 2
8. -type f/l/d/c/b/s/p (Regulay file, Symlink, Dir, CharDev, BlockDev, Socket, Pipe)
9. -atime/-ctime/-mtime(access, metadata change, modify), -7, 7, +7(less than, exactly, more than 7 days)
10. -amin/-cmin/-mmin; (like -atime.., but the time metric is in Minutes)
11. -newer another_file
12. -size 2k/+2k/-2k; k-> (b, c, w, k, M, G)==(Blocks, Bytes, Words, KiloBytes, MegaBytes, GigaBytes)
13. -delete; delete the selected(out-sorted)
14. -perm 644; (permissions)
15. -exec cat {} \; > all_c_files.txt
4. Playing with Xargs
1. Converting multi lines of input to a single-line output: cat files.txt | xargs
2. Converting single-line of input to multi lines of output: cat file.txt | xargs -n 3 (default delim: $IFS)
3. Specify delim: -d X (now the char "X" is the delim)
4. `cat args.txt | xargs -n 2 ./process_args.sh`; (each time pass 2 args to "process_args.sh")
5. `cat args.txt | xargs -I {} ./process_args.sh -p {} -l`; ( replacement )
6. `find some_filters -print | xargs rm -f` # bad
7. `find some_filters -print0 | xargs -0 rm -f ` # better (-0 specify the delim is '\0')
8. LOC(Lines of Code): `find path -type f -name "*.c" -print0 | xargs -0 wc -l`
9. `cat files.txt | ( while read arg; do cat $arg; done )` ==> `cat files.txt | xargs -I {} cat {}`
5. Translating with tr
1. tr [option] set1 set2; mapping from set1 to set2 (with given options), pay attention to len(set1) && len(set2) issue
2. `echo "WHO IS THIS" | tr 'A-Z' 'a-z'; (Other Valid Sets: 'a-fg-z', 'a-n0-9', 'aA,.',
3. tr '0-9' '987654321'; tr '987654321' '0-9' # encryption, decryption
4. tr 'a-zA-Z' 'n-za-mN-ZA-M' # the famous ROT13 Encryption (symetric, use it twice to decrypt)
5. Deleting chars: `tr -d '[set1]'`
6. Complementing character set: -c
7. Squeezing Characters with tr: tr -s ' '
8. Character Classes:
1. alnum, alpha, digit
2. cntrl: control(nonprinting) chars
3. graph(graphic), print(printable)
4. lower, upper, space(whitespace chas)
5. punct, xdigit(hexadecimal chars)
6. ============> Usage: e.g. [:alnum:]
6. Just for fun from Gnat:
`for x in {30..37}; do for y in {40..47}; do echo -e -n "\e[1;${x}m \e[1;${y}m Hello World "; done; echo -e "\e[1;30m"; done`
7. Checksum and Verification
1. md5sum filename ==> 32-Character Hexadecimal String
2. md5sum -c filename.md5sum ==> Ok/Filed, dont have to specify filename, it's in *.md5sum file
3. sha1sum, is like md5sum, but with 40-character Hex code
4. Checksum for Directories: md5deep / sha1deep -rl dir # -r(recursive), -l(use relative path)
8. Crptographic tools and Hashes
1. #I dont need these now, leaved for later reading
9. Sorting Unique and Duplicates
1. `sort files -o output.txt`; or `sort files > output.txt`
2. -n: numeric sort, -M: sort Months, -r: Reverse
3. -m: merge two sorted file: `sort -m sorted1.txt sorted2.txt`
4. sort file1.txt file2.txt | unique
5. -k NUM: sort by colomun NUM(1..INF), # Not finished, page 103 passed
6. unique: -u(dups will not print out), -c(count), -d(only dups), -s, -w(#pass, #compare)
10. Temporary file naming and random numbers
1. filename=`mktemp`; dirname=`mktemp -d`
2. generate a name but not create it now: `mktemp -u`
3. mktemp test.XXX # with file name test.???, randomly generated
11. Spliting files and data
1. split -b 10k data.file -d -a 4 (-d: numeric suffix, -a: 4 suffix)
2. split [args] [prefix]; e.g. `split -b 10k data.file -d -a 2 prat` ==> part01, part02, ...
3. split -l 10 Python; (split to 10 lines of text)
4. csplit --> split a file into sections determined by context lines
e.g. csplit server.log /SERVER/ -n 2 -s {*} -f server -b "%2d.log"
12. Slicing filenames based on extension
1. file="sample.jpg"; name=${file%.*}; echo $name ==> "sample" (% is nongreedy, % is greedy)
2. file="sample.jpg"; name=${file#*.}; echo $name ==> "jpg" (from left, ##: greedy)
13. Renaming and Moving files in Bulk
1. mv before after
2. rename *.jpg *.JPG
3. <space> to "_": rename 's/ /_/g' * # Rename all the file in `pwd`
4. Upper to Lower: rename 'y/A-Z/a-z/' *
5. Lower to Upper: rename 'y/a-z/A-Z/' *
14. Spell Checking and dictionary manipulation
1. dir /usr/share/dict (On DVP Keyboard, I prefer to type "dir" insteal of "ls")
15. I(Gnat) found something
1. In a word-processing app like geany, the line num is not the real one
2. Because: (offset by 1, for an empty file has 1 line but actually it's not)
3.
16. Automating interactive Input
1. Automating with expect
Y.
17. Making commands quicker by running parallel processes
Chap 3: File in, File out
1. Generating files of any size
1. dd if=/dev/zero of=./junk.data count=50 bs=1M # no <space> allowed
2. bs=1M: M->(c, w, b, k, M, G)
2. The Intersection and set difference (A-B) on text files
1. comm - compare two sorted files line by line
2. # Leave for later reading. page. 127
3. Finding and deleting duplicate files
4. Working with file permissions, ownership, and sticky bit,
1. extfs (extended FS)
5. Making files Immutable
6. Generating blank in bulk
7. Finding symbolic links and their target
1.
8. Enumeration file types statics
