文本处理的一些技巧
jenkins 构建成功后，Java服务并没有启动
BUILD_ID=dontKillMe nohup java -jar /root/cdrcy/cdrcy.jar -Xms246m -Xmx500m --spring.profiles.active=dev --server.port=8090 >>newcdrcy.log 2>&1 &
winwos java设置
tomcat/bin/setclasspath.bat
set JAVA_HOME=C:\Program Files (x86)\Java\jdk1.6.0_10
set JRE_HOME=C:\Program Files (x86)\Java\jdk1.6.0_10
###mysql5.7.17以后group by
use mysql;
set sql_mode ='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION';
flush privileges;
###npm安装yarn，yarn安装vue
npm cache clean --force
npm cache verify
npm install -g yarn --registry=https://registry.npm.taobao.org
yarn global add @vue/cli
yarn global add @vue/cli-service
yarn install
yarn build
yarn add --dev jest #安装jest #需要安装至项目的根目录
yarn test #运行项目所有测试
yarn test xxx #运行项目指定文件夹测试
User-Agent : Mozilla/5.0 (Linux; Android 6.0.1; MuMu Build/V417IR; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/52.0.2743.100 Mobile Safari/537.36 MMWEBID/6886 MicroMessenger/7.0.10.1580(0x27000A59) Process/toolsmp NetType/WIFI Language/zh_CN ABI/arm32
在Windows机器上Python通常安装在C:\Python27位置，当然你可以在运行安装向导时修改此值。 要想把此目录添加到你的PATH环境变量中，你可以在DOS 窗口中输入以下命令
set path=%path%;C:\python27
#jq 的提示
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 127975, column 127
在每行的头添加字符，比如"HEAD"，命令如下： sed 's/^/HEAD&/g' test.file
在每行的行尾添加字符，比如“TAIL”，命令如下： sed 's/$/&TAIL/g' test.file
# 伪随机数
cat /dev/urandom | od -x | tr -d ' ' | head -n 1
#gb2312 -t utf-8
iconv -f gb2312 -t utf-8 or piconv -f "gbk" -t "utf-8"
echo '\u6df1\u5733' | ascii2uni -a U -q > xx
#url编码
URLBM=`echo "${i}" | tr -d '\n' | xxd -plain | sed 's/\(..\)/%\1/g'`
#计算数字向上取整,bc计算得到小数，{print int($0)+1}得到整数+1；
num3=`echo "sclae=4; $num1/$num2" | bc`
num=`echo ${num3} |awk '{print int($0)+1}' `
文件中每行都以\n结尾，如果要去掉换行符，使用sed命令 # sed -i 's/\n//g' FileName
或者使用tr命令 # cat fileName | tr '\n' ''
去除字符串前后空格， # cat xxx| awk '{gsub(/^\s+|\s+$/, "");print}' or sed s/[[:space:]]//g
[root@newzdaw01 opt]# echo "1 2 3 4 5" | tr -s " "
1 2 3 4 5
[root@newzdaw01 opt]# echo "1 2 3 4 5" | sed 's/ */ /g'
1 2 3 4 5
[root@newzdaw01 opt]# echo "1 2 3 4 5" | sed s/[[:space:]]//g
12345
多行文字合为一行
1. awk '{printf (/#/&&NR>1)?"\n"$0:$0}'
2. cat xxx|xargs -0
判断字符串是否为空 if [ -n "${ida}" ]; if [ -z "$str" ] （-n 为非空）
sort（-n 按数字排序 -r倒序 -t 按什么切割， -k 行数）
cat allnumber.txt|sort -n -r -t "," -k2
k1=$1
if [[ $k1 = 0 ]];then
echo "number=0"
elif [[ $k1 -gt 0 ]] && [[ $k1 -le 45 ]];then
echo "bigger than 0 ,little 45"
elif [[ $k1 -gt 45 ]] && [[ $k1 -le 450 ]];then
echo "dayu 45 ,xiaoyu 450"
elif [[ $k1 -gt 450 ]];then
echo "dayu450"
fi
pc
Opera
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36 OPR/26.0.1656.60
Opera/8.0 (Windows NT 5.1; U; en)
Mozilla/5.0 (Windows NT 5.1; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.50
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 9.50
Firefox
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0
Mozilla/5.0 (X11; U; Linux x86_64; zh-CN; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10
Safari
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2
chrome
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.133 Safari/534.16
360
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
淘宝浏览器
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.11 TaoBrowser/2.0 Safari/536.11
猎豹浏览器
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.71 Safari/537.1 LBBROWSER
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; LBBROWSER)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 732; .NET4.0C; .NET4.0E; LBBROWSER)"
QQ浏览器
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; QQBrowser/7.0.3698.400)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 732; .NET4.0C; .NET4.0E)
sogou浏览器
Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.84 Safari/535.11 SE 2.X MetaSr 1.0
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SV1; QQDownload 732; .NET4.0C; .NET4.0E; SE 2.X MetaSr 1.0)
maxthon浏览器
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.4.3.4000 Chrome/30.0.1599.101 Safari/537.36
UC浏览器
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 UBrowser/4.0.3214.0 Safari/537.36
phone
IPhone
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5
IPod
Mozilla/5.0 (iPod; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5
IPAD
Mozilla/5.0 (iPad; U; CPU OS 4_2_1 like Mac OS X; zh-cn) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5
Mozilla/5.0 (iPad; U; CPU OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5
Android
Mozilla/5.0 (Linux; U; Android 2.2.1; zh-cn; HTC_Wildfire_A3333 Build/FRG83D) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
Mozilla/5.0 (Linux; U; Android 2.3.7; en-us; Nexus One Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
QQ浏览器 Android版本
MQQBrowser/26 Mozilla/5.0 (Linux; U; Android 2.3.7; zh-cn; MB200 Build/GRJ22; CyanogenMod-7) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
Android Opera Mobile
Opera/9.80 (Android 2.3.4; Linux; Opera Mobi/build-1107180945; U; en-GB) Presto/2.8.149 Version/11.10
Android Pad Moto Xoom
Mozilla/5.0 (Linux; U; Android 3.0; en-us; Xoom Build/HRI39) AppleWebKit/534.13 (KHTML, like Gecko) Version/4.0 Safari/534.13
BlackBerry
Mozilla/5.0 (BlackBerry; U; BlackBerry 9800; en) AppleWebKit/534.1+ (KHTML, like Gecko) Version/6.0.0.337 Mobile Safari/534.1+
WebOS HP Touchpad
Mozilla/5.0 (hp-tablet; Linux; hpwOS/3.0.0; U; en-US) AppleWebKit/534.6 (KHTML, like Gecko) wOSBrowser/233.70 Safari/534.6 TouchPad/1.0
Nokia N97
Mozilla/5.0 (SymbianOS/9.4; Series60/5.0 NokiaN97-1/20.0.019; Profile/MIDP-2.1 Configuration/CLDC-1.1) AppleWebKit/525 (KHTML, like Gecko) BrowserNG/7.1.18124
Windows Phone Mango
Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0; HTC; Titan)
UC浏览器
UCWEB7.0.2.37/28/999
NOKIA5700/ UCWEB7.0.2.37/28/999
UCOpenwave
Openwave/ UCWEB7.0.2.37/28/999
UC Opera
Mozilla/4.0 (compatible; MSIE 6.0; ) Opera/UCWEB7.0.2.37/28/999
一键sshkey ssh-keygen -t rsa -P '' -f /root/.ssh/id_rsa
ssh-copy-id -i /root/.ssh/id_rsa.pub 192.168.0.9
查看本地外网ip地址 curl cip.cc
cat >/etc/resolv.conf <<eof
nameserver 192.168.0.1
nameserver 114.114.114.114
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 223.5.5.5
nameserver 223.6.6.6
eof
swap文件监理
#4096000是4G；；1024000=1G
dd if=/dev/zero of=/opt/kwxgd bs=1024 count=4096000
mkswap /opt/kwxgd
swapon /opt/kwxgd
误删除/dev/zero 解决方法：
mknod -m 666 /dev/zero c 1 5
1、curl请求GB2312网页乱码解决
Linux环境是UTF8的，请求GB2312内容是会乱码显示。需要转换格式，命令参考如下：
curl xxx | iconv -f gb2312 -t utf-8
这样有个小问题，如果文件下载不完全，会导致更多不可识别的乱码，先下载，再转码2、linux shell下16进制 “\uxxxx” unicode to UTF-8中文
1、安装一个uni2ascii-4.15-1.el6.rf.x86_64.rpm
下载地址:wget http://billposer.org/Software/Downloads/uni2ascii-4.18.tar.gz ,然后编译./config ,后make install，默认安装到./usr/bin目录下
2、然后可以使用ascii2uni命名转换
# echo '\u6df1\u5733' |ascii2uni -a U -q
