Linux bash sed command All In One

Linux bash sed command All In One

# man sed
$ cat ./man-docs/man-sed.md

$ man sed
SED(1)                                 User Commands                                 SED(1)

NAME
       sed - stream editor for filtering and transforming text

SYNOPSIS
       sed [OPTION]... {script-only-if-no-other-script} [input-file]...

DESCRIPTION
       Sed  is  a stream editor.  A stream editor is used to perform basic text transforma‐
       tions on an input stream (a file or input from a pipeline).  While in some ways sim‐
       ilar  to  an  editor  which permits scripted edits (such as ed), sed works by making
       only one pass over the input(s), and is consequently  more  efficient.   But  it  is
       sed's  ability to filter text in a pipeline which particularly distinguishes it from
       other types of editors.

       -n, --quiet, --silent

              suppress automatic printing of pattern space

       --debug

              annotate program execution

       -e script, --expression=script

              add the script to the commands to be executed

       -f script-file, --file=script-file

              add the contents of script-file to the commands to be executed

       --follow-symlinks

              follow symlinks when processing in place

       -i[SUFFIX], --in-place[=SUFFIX]

              edit files in place (makes backup if SUFFIX supplied)

       -l N, --line-length=N

              specify the desired line-wrap length for the `l' command

       --posix

              disable all GNU extensions.

       -E, -r, --regexp-extended

              use extended regular expressions in the script  (for  portability  use  POSIX
              -E).

       -s, --separate

              consider files as separate rather than as a single, continuous long stream.

       --sandbox

              operate in sandbox mode (disable e/r/w commands).

       -u, --unbuffered

              load  minimal  amounts of data from the input files and flush the output buf‐
              fers more often

       -z, --null-data

              separate lines by NUL characters

       --help
              display this help and exit

       --version
              output version information and exit

       If no -e, --expression, -f, or --file option is given, then the first non-option ar‐
       gument  is  taken as the sed script to interpret.  All remaining arguments are names
       of input files; if no input files are specified, then the standard input is read.

       GNU sed home page:  <https://www.gnu.org/software/sed/>.   General  help  using  GNU
       software: <https://www.gnu.org/gethelp/>.  E-mail bug reports to: <bug-sed@gnu.org>.

COMMAND SYNOPSIS
       This  is  just  a brief synopsis of sed commands to serve as a reminder to those who
       already know sed; other documentation (such as the texinfo document)  must  be  con‐
       sulted for fuller descriptions.

   Zero-address ``commands''
       : label
              Label for b and t commands.

       #comment
              The  comment  extends until the next newline (or the end of a -e script frag‐
              ment).

       }      The closing bracket of a { } block.

   Zero- or One- address commands
       =      Print the current line number.

       a \

       text   Append text, which has each embedded newline preceded by a backslash.

       i \

       text   Insert text, which has each embedded newline preceded by a backslash.

       q [exit-code]
              Immediately quit the sed script without processing  any  more  input,  except
              that if auto-print is not disabled the current pattern space will be printed.
              The exit code argument is a GNU extension.

       Q [exit-code]
              Immediately quit the sed script without processing any more input.  This is a
              GNU extension.

       r filename
              Append text read from filename.

       R filename
              Append  a  line  read  from filename.  Each invocation of the command reads a
              line from the file.  This is a GNU extension.

   Commands which accept address ranges
       {      Begin a block of commands (end with a }).

       b label
              Branch to label; if label is omitted, branch to end of script.

       c \

       text   Replace the selected lines with text, which has each  embedded  newline  pre‐
              ceded by a backslash.

       d      Delete pattern space.  Start next cycle.

       D      If  pattern  space  contains no newline, start a normal new cycle as if the d
              command was issued.  Otherwise, delete text in the pattern space  up  to  the
              first  newline,  and  restart cycle with the resultant pattern space, without
              reading a new line of input.

       h H    Copy/append pattern space to hold space.

       g G    Copy/append hold space to pattern space.

       l      List out the current line in a ``visually unambiguous'' form.

       l width
              List out the current line in a ``visually unambiguous'' form, breaking it  at
              width characters.  This is a GNU extension.

       n N    Read/append the next line of input into the pattern space.

       p      Print the current pattern space.

       P      Print up to the first embedded newline of the current pattern space.

       s/regexp/replacement/
              Attempt  to  match  regexp against the pattern space.  If successful, replace
              that portion matched with replacement.  The replacement may contain the  spe‐
              cial character & to refer to that portion of the pattern space which matched,
              and the special escapes \1 through \9 to refer to the corresponding  matching
              sub-expressions in the regexp.

       t label
              If  a  s///  has done a successful substitution since the last input line was
              read and since the last t or T command, then branch to  label;  if  label  is
              omitted, branch to end of script.

       T label
              If  no  s/// has done a successful substitution since the last input line was
              read and since the last t or T command, then branch to  label;  if  label  is
              omitted, branch to end of script.  This is a GNU extension.

       w filename
              Write the current pattern space to filename.

       W filename
              Write the first line of the current pattern space to filename.  This is a GNU
              extension.

       x      Exchange the contents of the hold and pattern spaces.

       y/source/dest/
              Transliterate the characters in the pattern space which appear in  source  to
              the corresponding character in dest.

Addresses
       Sed  commands can be given with no addresses, in which case the command will be exe‐
       cuted for all input lines; with one address, in which case the command will only  be
       executed  for  input lines which match that address; or with two addresses, in which
       case the command will be executed for all input  lines  which  match  the  inclusive
       range of lines starting from the first address and continuing to the second address.
       Three things to note about address ranges: the syntax is addr1,addr2 (i.e., the  ad‐
       dresses  are  separated by a comma); the line which addr1 matched will always be ac‐
       cepted, even if addr2 selects an earlier line; and if addr2 is a regexp, it will not
       be tested against the line that addr1 matched.

       After  the address (or address-range), and before the command, a !  may be inserted,
       which specifies that the command shall only be executed if the address (or  address-
       range) does not match.

       The following address types are supported:

       number Match  only  the  specified line number (which increments cumulatively across
              files, unless the -s option is specified on the command line).

       first~step
              Match every step'th line starting with line first.   For  example,  ``sed  -n
              1~2p'' will print all the odd-numbered lines in the input stream, and the ad‐
              dress 2~5 will match every fifth line, starting with the second.   first  can
              be zero; in this case, sed operates as if it were equal to step.  (This is an
              extension.)

       $      Match the last line.

       /regexp/
              Match lines matching the regular expression regexp.  Matching is performed on
              the  current  pattern  space,  which  can  be  modified with commands such as
              ``s///''.

       \cregexpc
              Match lines matching the regular expression regexp.  The c may be any charac‐
              ter.

       GNU sed also supports some special 2-address forms:

       0,addr2
              Start  out  in  "matched first address" state, until addr2 is found.  This is
              similar to 1,addr2, except that if addr2 matches the very first line of input
              the  0,addr2  form  will be at the end of its range, whereas the 1,addr2 form
              will still be at the beginning of its range.  This works only when addr2 is a
              regular expression.

       addr1,+N
              Will match addr1 and the N lines following addr1.

       addr1,~N
              Will  match addr1 and the lines following addr1 until the next line whose in‐
              put line number is a multiple of N.

REGULAR EXPRESSIONS
       POSIX.2 BREs should be supported, but they aren't completely because of  performance
       problems.   The  \n  sequence in a regular expression matches the newline character,
       and similarly for \a, \t, and other sequences.  The -E option switches to using  ex‐
       tended  regular expressions instead; it has been supported for years by GNU sed, and
       is now included in POSIX.

BUGS
       E-mail bug reports to bug-sed@gnu.org.  Also, please include  the  output  of  ``sed
       --version'' in the body of your report if at all possible.

AUTHOR
       Written by Jay Fenlason, Tom Lord, Ken Pizzini, Paolo Bonzini, Jim Meyering, and As‐
       saf Gordon.  GNU sed home page: <https://www.gnu.org/software/sed/>.   General  help
       using  GNU  software:  <https://www.gnu.org/gethelp/>.  E-mail bug reports to: <bug-
       sed@gnu.org>.

COPYRIGHT
       Copyright © 2018 Free Software Foundation, Inc.  License GPLv3+: GNU GPL  version  3
       or later <https://gnu.org/licenses/gpl.html>.
       This is free software: you are free to change and redistribute it.  There is NO WAR‐
       RANTY, to the extent permitted by law.

SEE ALSO
       awk(1), ed(1), grep(1), tr(1), perlre(1), sed.info, any of various books on sed, the
       sed FAQ (http://sed.sf.net/grabbag/tutorials/sedfaq.txt), http://sed.sf.net/grab‐
       bag/.

       The full documentation for sed is maintained as a Texinfo manual.  If the info and
       sed programs are properly installed at your site, the command

              info sed

       should give you access to the complete manual.

sed 4.7                                December 2018                                 SED(1)

sed

sed, a stream editor

sed，流编辑器

awk, Perl, gawk

https://www.gnu.org/software/sed/manual/sed.html

# sed SCRIPT INPUTFILE...

# to replace all occurrences of ‘hello’ to ‘world’ in the file input.txt:

$ sed 's/hello/world/' input.txt > output.txt

# The following commands are equivalent:

$ sed 's/hello/world/' input.txt > output.txt

$ sed 's/hello/world/' < input.txt > output.txt

$ cat input.txt | sed 's/hello/world/' - > output.txt

Linux 命令行编辑器

sed & gawk

???

vi
vim
nano

emacs

https://www.gnu.org/software/emacs/

http://www.hypexr.org/bash_tutorial.php#emacs

Linux sed 命令

Linux sed 命令是利用脚本来处理文本文件。

sed 可依照脚本的指令来处理、编辑文本文件。

Sed 主要用来自动编辑一个或多个文件、简化对文件的反复操作、编写转换程序等。

# 语法
$ sed [-hnV][-e<script>][-f<script文件>][文本文件]

https://www.runoob.com/linux/linux-comm-sed.html

https://www.computerhope.com/unix/used.htm

macOS bug

$ man sed

$ sed --help
sed: illegal option -- -
usage: sed script [-Ealnru] [-i extension] [file ...]
        sed [-Ealnu] [-i extension] [-e script] ... [-f script_file] ... [file ...]

$ sed --version
sed: illegal option -- -
usage: sed script [-Ealnru] [-i extension] [file ...]
        sed [-Ealnu] [-i extension] [-e script] ... [-f script_file] ... [file ...]

sed API

参数名称	参数描述
-e	指定 sed 脚本命令, 多个 sed 命令使用 ; 分隔
-f	指定 sed 脚本文件 .sed 扩展名
-n	sed 禁用 STDOUT 输出

# 接受 `STDIN` 作为输入
$ echo "this is abc" | sed -e 's/abc/xyz/'

$ echo "this is abc" | sed -e 's/abc/xyz/' -n

# 读取文件，返回 `STDOUT` 作为输出
$ sed -e 's/abc/xyz/' ./test.txt

# ✅ -n filename
$ sed -e 's/abc/xyz/' -n ./test.txt 

# ❌ filename -n 
$ sed -e 's/abc/xyz/' ./test.txt -n 

# 多个 sed 命令使用 `;` 分隔, 分号与前面的斜线之间不能有空格 ❓
$ echo "this is CBA and China" | sed -e 's/CBA/NBA/; s/China/USA/'

# 分号与前面的斜线之间有空格 ✅
$ echo "this is CBA and China" | sed -e 's/CBA/NBA/ ; s/China/USA/'

# 执行 sed 脚本，sed 后面指定文件名写在命令行里
$ sed -f test.sed ./test.txt

$ sed -f test.sed ./test.txt >> ./test.out.md

# ✅ -n 
$ sed -f test.sed  -n ./test.txt
# ❌ -n
$ sed -f -n test.sed ./test.txt

# 注释: `.sed`扩展名的文件，是 sed 命令集合专用的，本质上还是一种 shell scirpt
# 无需写 sed 和 文件名了，因为`.sed`扩展名的文件中是纯 sed 命令
s/abc/🎉/
s/xyz/👻/
s/ufo/🛸/


# 执行 sed 脚本，sed 后面指定文件名写在命令行里
# $ sed -f test.sed ./test.txt
# $ sed -f test.sed -n ./test.txt
# $ sed -f test.sed ./test.txt >> ./test.out.md

# sed: 12: test.sed: invalid command code <
# <<EOF

# s/abc/xyz/ test.txt
# sed: 4: test.sed: bad flag in substitute command: 't'

# s/abc/xyz/ ./test.txt
# sed: 4: test.sed: bad flag in substitute command: '.'

# 's/abc/xyz/' ./test.txt
# sed: 4: test.sed: invalid command code ' ❌

# sed -e 's/abc/xyz/' ./test.txt
# sed: 2: test.sed: bad flag in substitute command: 's' ❌

# sed -e 's/abc/xyz/' -n ./test.txt

# EOF

1. 替换 s

substitute

2. 插入i 和追加 a

insert / append

3. 删除 d

delete

4. 读取 r

read

5. 写入 w

write

6. 打印 p

print

正则表达式

demos

git hooks

commit-msg

#!/bin/sh
#
# An example hook script to check the commit log message.
# Called by "git commit" with one argument, the name of the file
# that has the commit message.  The hook should exit with non-zero
# status after issuing an appropriate message if it wants to stop the
# commit.  The hook is allowed to edit the commit message file.
#
# To enable this hook, rename this file to "commit-msg".

# Uncomment the below to add a Signed-off-by line to the message.
# Doing this in a hook is a bad idea in general, but the prepare-commit-msg
# hook is more suited to it.
#
# SOB=$(git var GIT_AUTHOR_IDENT | sed -n 's/^\(.*>\).*$/Signed-off-by: \1/p')
# grep -qs "^$SOB" "$1" || echo "$SOB" >> "$1"

# This example catches duplicate Signed-off-by lines.

test "" = "$(grep '^Signed-off-by: ' "$1" |
	 sort | uniq -c | sed -e '/^[ 	]*1[ 	]/d')" || {
	echo >&2 Duplicate Signed-off-by lines.
	exit 1
}

XSS

#!/bin/bash

sed -i "s/xssor.io/$1/g" xssor/payload/probe.js

python3 manage.py runserver 0.0.0.0:8000

https://github.com/evilcos/xssor2/blob/master/run.sh

余弦

https://evilcos.me/about.html

(🐞 反爬虫测试！打击盗版⚠️）如果你看到这个信息, 说明这是一篇剽窃的文章，请访问 https://www.cnblogs.com/xgqfrms/ 查看原创文章！

Raspberry Pi & IP Address filter & sed

pi@raspberrypi:~/Desktop $ cat ip.md
192.168.18.168
pi@raspberrypi:~/Desktop $ ifconfig | perl -nle'/(\d+\.\d+\.\d+\.\d+)/ && print $1'
127.0.0.1
192.168.18.168
pi@raspberrypi:~/Desktop $ ifconfig | perl -nle'/(\d+\.\d+\.\d+\.\d+)/ && print $1'  | sed '1 d'
192.168.18.168
pi@raspberrypi:~/Desktop $ ifconfig | perl -nle'/(\d+\.\d+\.\d+\.\d+)/ && print $1'  | sed '2 d'
127.0.0.1

refs

https://www.cnblogs.com/xgqfrms/tag/sed

https://www.cnblogs.com/xgqfrms/p/16243419.html

---

---

©xgqfrms 2012-2021

www.cnblogs.com/xgqfrms 发布文章使用：只允许注册用户才可以访问！

原创文章，版权所有©️xgqfrms, 禁止转载 🈲️，侵权必究⚠️！

---