[terry笔记]GoldenGate_迁移同步_主库零停机
ogg根据scn同步数据，源库零停机时间
本次实验与上次的区别：更加注重细节，几乎包含所有步骤，把我越到的坑都作出了说明。并且同步是由10g向11g进行同步，更加符合升级迁移需求。
如下是主要步骤：
1. 配置好ogg源端的mgr、抓取和传送进程，并启动。
2. 配置好ogg目标端的mgr、复制进程，仅启动mgr。
3. 源端可自由进行交易，此时观察源与目标的trail文件是否都正常。
4. 查询源端此时的scn，并按照参数flashbask_scn进行expdp。
5. 目标端impdp导入。
6. 目标端start replicat xxx,aftercsn xxxxx
GoldenGate重要进程介绍：
1、Manager管理进程在两端开启，监控和重启其他进程；分配数据存储和报告错误及事件；
2、Extract进程从日志中抓取并传输到target端事务数据；
3、Server Collector进程在target(接受)端接受数据并写入trail文件；
4、Replicat进程读取trail文件，并应用到traget数据库；
5、trail文件时gg自己抓捕信息的文件，是一个OS文件，存放在./dirdat/下，以X00000命名，N顺序1，2,3…此文件用完可配置参数自动删除。
一、环境准备并安装GoldenGate
1. 数据库准备情况
1) 源服务器
hostname:ogg1
IP地址：192.168.100.133
数据库：10.2.0.5.0 64 bit
SID: super
操作系统版本：redhat5.5 64 bit
ogg版本：ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip
2) 目标服务器
hostname:ogg2
IP地址：192.168.100.130
数据库：11.2.0.4.0 64 bit
SID: test
操作系统版本：redhat5.5 64 bit
ogg版本：ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip
2. 创建用户、下载OGG软件并解压安装
ogg1：
[root@ogg1 ~]# mkdir /ogg [root@ogg1 ~]# useradd -g oinstall -G dba ggs [root@ogg1 ~]# passwd ggs [root@ogg1 ~]# su - ggs [ggs@ogg1 ~]$ id uid=1001(ggs) gid=1000(oinstall) groups=1000(oinstall),1001(dba) [ggs@ogg1 ~]$ exit logout [root@ogg1 ~]# chown -R ggs:oinstall /ogg [root@ogg1 ogg]# unzip ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip [root@ogg1 ogg]# tar xvf fbo_ggs_Linux_x64_ora11g_64bit.tar -C /ogg [root@ogg1 ogg]# chown -R ggs:oinstall /ogg
ogg2：
[root@ogg2 ~]# useradd -g oinstall -G dba ggs [root@ogg2 ~]# passwd ggs [root@ogg2 ~]# mkdir /ogg [root@ogg2 ~]# chown -R ggs:oinstall /ogg [root@ogg2 ogg]# unzip ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip [root@ogg2 ogg]# tar xvf fbo_ggs_Linux_x64_ora11g_64bit.tar -C /ogg [root@ogg2 ogg]# chown -R ggs:oinstall /ogg
3. 准备OGG环境变量
#源服务器、与目标服务器都要配置
$ vi /home/ggs/.bash_profile
#源端ggs增加如下行：
export PATH export ORACLE_BASE=/u01 export ORACLE_HOME=$ORACLE_BASE/product/10.2.0 export ORACLE_SID=super export PATH=$PATH:$HOME/bin:$ORACLE_HOME/bin export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/ogg:/usr/local/lib:/usr/X11R6/lib export LD_LIBRARY_PATH_64=$ORACLE_HOME/lib
$ vi /home/ggs/.bash_profile
#目标端ggs增加如下行：
export PATH export ORACLE_BASE=/u01 export ORACLE_HOME=$ORACLE_BASE/product/11.2.0 export ORACLE_SID=test export PATH=$PATH:$HOME/bin:$ORACLE_HOME/bin export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/ogg
4. 配置日志模式
#目标服务器不用配置(非双向)
1) 查看规档与日志模式
sqlplus / as sysdba select log_mode,supplemental_log_data_min,force_logging from v$database; LOG_MODE SUPPLEME FOR ------------ -------- --- NOARCHIVELOG NO NO
2) 配置为规档模式
shutdown immediate; startup mount; alter database archivelog; alter database open; alter system set log_archive_dest_1='location=/u01/archive' scope=both;
3) 配置日志模式(打开强制规档与补充日志模式)
alter database add supplemental log data; alter database force logging;
4）查看配置结果：
SQL> select log_mode,supplemental_log_data_min,force_logging from v$database; LOG_MODE SUPPLEME FOR ------------ -------- --- ARCHIVELOG YES YES
5. 创建GoldenGate用户帐号
1) 源服务器
su - oracle sqlplus /nolog conn / as sysdba; create tablespace ogg datafile '/u01/oradata/super/ogg.dbf' size 10m autoextend on next 10m; create user ggs identified by ggs default tablespace ogg temporary tablespace TEMP quota unlimited on ogg; grant connect,resource to ggs; grant create session,alter session to ggs; grant select any dictionary,select any table to ggs; grant alter any table to ggs; grant flashback any table to ggs; grant execute on dbms_flashback to ggs; 或直接grant dba to ggs;
2) 目标服务器
su - oracle sqlplus /nolog conn / as sysdba; create tablespace ogg datafile '/u01/oradata/test/ogg.dbf' size 10m autoextend on next 10m; create user ggs identified by ggs default tablespace ogg temporary tablespace TEMP quota unlimited on ogg; grant connect,resource to ggs; grant create session,alter session to ggs; grant select any dictionary,select any table to ggs; grant alter any table to ggs; grant flashback any table to ggs; grant execute on dbms_flashback to ggs; grant insert any table to ggs; grant delete any table to ggs; grant update any table to ggs; 或直接grant dba to ggs;
6. 安装GoldenGate软件
(1) 源服务器、与目标服务器都要配置
源服务器因为是10g，ogg存在依赖包的问题：
[root@ogg1 ogg]$ su - ggs [ggs@ogg1 ogg]$ cd /ogg [ggs@ogg1 ogg]$ ./ggsci #发现ggsci进不去 ./ggsci: error while loading shared libraries: libnnz11.so: cannot open shared object file: No such file or directory [ggs@ogg1 ogg]$ ldd ggsci #检查依赖包，发现缺少两个11g的包 libdl.so.2 => /lib64/libdl.so.2 (0x0000003d9b600000) libgglog.so => /ogg/libgglog.so (0x00002b8bd05ca000) libggrepo.so => /ogg/libggrepo.so (0x00002b8bd07ff000) libdb-5.2.so => /ogg/libdb-5.2.so (0x00002b8bd0954000) libicui18n.so.38 => /ogg/libicui18n.so.38 (0x00002b8bd0bf5000) libicuuc.so.38 => /ogg/libicuuc.so.38 (0x00002b8bd0f55000) libicudata.so.38 => /ogg/libicudata.so.38 (0x00002b8bd128f000) libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003d9ba00000) libxerces-c.so.28 => /ogg/libxerces-c.so.28 (0x00002b8bd226b000) libantlr3c.so => /ogg/libantlr3c.so (0x00002b8bd2783000) libnnz11.so => not found libclntsh.so.11.1 => not found libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003dad400000) libm.so.6 => /lib64/libm.so.6 (0x0000003d9b200000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003daa400000) libc.so.6 => /lib64/libc.so.6 (0x0000003d9ae00000) /lib64/ld-linux-x86-64.so.2 (0x0000003d9aa00000) 在服务器中找不到11g的包： [root@ogg1 ogg]# find / -name libnnz11.so [root@ogg1 ogg]# find / -name libnnz10.so /u01/product/10.2.0/lib/libnnz10.so /u01/product/10.2.0/lib32/libnnz10.so [root@ogg1 ogg]# find / -name libclntsh.so.11.1 [root@ogg1 ogg]# find / -name libclntsh.so.10.1 /u01/product/10.2.0/lib/libclntsh.so.10.1 /u01/product/10.2.0/lib32/libclntsh.so.10.1 此时需要做两个软连接： [oracle@ogg1 lib]$ ln -s /u01/product/10.2.0/lib/libnnz10.so libnnz11.so [oracle@ogg1 lib]$ ln -s /u01/product/10.2.0/lib/libclntsh.so.10.1 libclntsh.so.11.1
此时可以源端进入ggsci：
[root@ogg1 ogg]# su - ggs [ggs@ogg1 ~]$ cd /ogg [ggs@ogg1 ogg]$ ./ggsci Oracle GoldenGate Command Interpreter for Oracle Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230_FBO Linux, x64, 64bit (optimized), Oracle 11g on Apr 23 2012 08:32:14 Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved. GGSCI (ogg1) 1> create subdirs Creating subdirectories under current directory /ogg Parameter files /ogg/dirprm: already exists Report files /ogg/dirrpt: created Checkpoint files /ogg/dirchk: created Process status files /ogg/dirpcs: created SQL script files /ogg/dirsql: created Database definitions files /ogg/dirdef: created Extract data files /ogg/dirdat: created Temporary files /ogg/dirtmp: created Stdout files /ogg/dirout: created
(2) 目标服务器都要配置
[root@ogg2 ogg]# su - ggs [ggs@ogg2 ~]$ cd /ogg [ggs@ogg2 ogg]$ ./ggsci Oracle GoldenGate Command Interpreter for Oracle Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230_FBO Linux, x64, 64bit (optimized), Oracle 11g on Apr 23 2012 08:32:14 Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved. GGSCI (ogg2) 1> create subdirs Creating subdirectories under current directory /ogg Parameter files /ogg/dirprm: already exists Report files /ogg/dirrpt: created Checkpoint files /ogg/dirchk: created Process status files /ogg/dirpcs: created SQL script files /ogg/dirsql: created Database definitions files /ogg/dirdef: created Extract data files /ogg/dirdat: created Temporary files /ogg/dirtmp: created Stdout files /ogg/dirout: created
(3)此时在源端scott下创建了两张测试表test1与test2（必须有主键）：
SQL> conn scott/tiger Connected. SQL> create table test1 (id number); Table created. SQL> create table test2 (name varchar2(20)); Table created. SQL> insert into test1 values (1); 1 row created. SQL> insert into test1 values (2); 1 row created. SQL> insert into test1 values (3); 1 row created. SQL> commit; Commit complete. SQL> alter table test1 add constraints pk_test1 primary key(id); Table altered. SQL> alter table test2 add constraints pk_test2 primary key(name); Table altered. SQL> insert into test2 values('***'); 1 row created. SQL> insert into test2 values('likeqiang'); 1 row created. SQL> insert into test2 values('batman'); 1 row created. SQL> commit; Commit complete.
二、 GoldenGate DML同步源端配置
1. 源MGR进程
1) 编辑mgr程组
edit params mgr
port 7809 dynamicportlist 7800-7899 autorestart extract *,retries 5,waitminutes 2 purgeoldextracts ./dirdat/*,usecheckpoints,minkeepdays 7 lagreporthours 1 laginfominutes 30 lagcriticalminutes 45
说明:
port 指定mgr进程通信端口
dynamicportlist 表示mgr进程可以为源与目的端动态通信指定端口
autorestart extract 表示自动重启extract进程组，每2分钟尝试重启所有进程，重试5次。
配置参数后，重启mgr进程生效
purgeoldextracts xxx，usercheckpoints,minkeepdays x表示trail文件会保留7天，ogg会根据checkpoint删除使用过的trail文件。
lagreporthours 1 每隔1小时检查一次extract和replicat的lag。
laginfominutes 30 如果lag超过规定的值（lagcritical），会报错critical。如果没问题，会每30分钟报告信息。
lagcriticalminutes 45 会每隔45分钟报告critical信息。
2) 启动主管理进程
GGSCI (ogg1) 3> start mgr
Manager started.
GGSCI (ogg1) 4> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
2. 配置Extract进程组
1)GGSCI (ogg1) 5> edit params test_ext
GGSCI (ogg1) 6> view param test_ext
extract test_ext dynamicresolution setenv (NLS_LANG=AMERICAN_AMERICA.ZHS16GBK) userid ggs,password ggs exttrail /ogg/dirdat/te table scott.*;
2) 添加抽取进程
GGSCI (ogg1) 7> add extract test_ext,tranlog,begin now
EXTRACT added.
3) 添加本地trail文件,源extract进程负责写这部分文件，pump负责把这部分文件传到目标服务器端。
GGSCI (ogg1) 8> add exttrail /ogg/dirdat/te,extract test_ext
EXTTRAIL added.
GGSCI (ogg1) 9> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER STOPPED
EXTRACT STOPPED TEST_EXT 00:00:00 00:00:39
说明:
extract eora定义extract进程名字
dynamicresolution
setenv设置环境变量
userid 登录数据库
exttrail指定本地trail文件地址
table 定义同步的表
4) 启动服务
GGSCI (ogg1) 16> start extract test_ext
Sending START request to MANAGER ...
EXTRACT TEST_EXT starting
GGSCI (ogg1) 17> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING TEST_EXT 00:03:07 00:00:00
此时观察dirdat目录，可以发现抽取的文件：
[root@ogg1 dirdat]# pwd
/ogg/dirdat
[root@ogg1 dirdat]# ll
总计 4
-rw-rw-rw- 1 ggs oinstall 988 07-08 21:09 te000000
3. 配置Pump进程组
1) 编辑配置文件
edit params test_p
extract test_p dynamicresolution passthru rmthost 192.168.100.130,mgrport 7809,compress rmttrail /ogg/dirdat/tp table scott.*;
2) 添加pump进程
GGSCI (ogg1) 21> add extract test_p,exttrailsource /ogg/dirdat/te
EXTRACT added.
3) 添加远程trail文件
GGSCI (ogg1) 22> add rmttrail /ogg/dirdat/tp,extract test_p
RMTTRAIL added.
4) 启动pump进程
GGSCI (ogg1) 23> start extract test_p
Sending START request to MANAGER ...
EXTRACT TEST_P starting
GGSCI (ogg1) 24> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING TEST_EXT 00:00:00 00:00:02
EXTRACT RUNNING TEST_P 00:00:00 00:00:58
三、 GoldenGate DML同步目标端配置
1. 目标端MGR进程
1) 编辑配置文件
GGSCI (ogg2) 1> view param mgr
port 7809 dynamicportlist 7800-7899 purgeoldextracts /ogg/dirdat/*, usecheckpoints, minkeepdays 3
2) 启动
GGSCI (ogg2) 2> start mgr
Manager started.
GGSCI (ogg2) 3> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
GGSCI (ogg2) 4> exit
此时在目标端的dirdat中可以看到由源端传送过来的trail文件(如果没出现，可以在源端做几个事务即可，做完事务还没出现，那么就是配置有问题，好好检查源端pump和目标端mgr)
[root@ogg2 dirdat]# ll
总计 4
-rw-rw-rw- 1 ggs oinstall 1634 07-08 21:45 tp000000
2. 添加检查表
说明: 当我们在GLOBALS 文件里指定了默认的checkpoint 之后，新的Replicat groups 在创建时会自动使用这个参数，不需要其他指令
1) 目标端编辑全局配置文件
edit params ./GLOBALS
CHECKPOINTTABLE ggs.checkpoint
2）exit #这里需要退出ggsci终端
3) 添加checkpoint表
[ggs@ogg2 ogg]$ ./ggsci
GGSCI (ogg2) 1> dblogin userid ggs,password ggs
Successfully logged into database.
GGSCI (ogg2) 2> add checkpointtable ggs.checkpoint
Successfully created checkpoint table ggs.checkpoint.
切换到数据库下，可以看到checkpoint表
SQL> conn ggs/ggs Connected. SQL> select * from tab; TNAME TABTYPE CLUSTERID ------------------------------ ------- ---------- CHECKPOINT TABLE CHECKPOINT_LOX TABLE
3. 配置目标端replicat进程组
1) 编辑配置文件
edit params test_r
replicat test_r userid ggs,password ggs assumetargetdefs reperror default,discard discardfile /ogg/dirrpt/test_r.desc dynamicresolution map scott.*;target scott.*;
2) 添加复制进程，此时不要启动进程
GGSCI (ogg2) 4> add replicat test_r,exttrail /ogg/dirdat/tp,checkpointtable ggs.checkpoint
REPLICAT added.
GGSCI (ogg2) 5> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT STOPPED TEST_R 00:00:00 00:00:14
四、查询源scn，将源数据导入目标库
1.源库按照scn导入到目标库
此时源端的数据：
SQL> conn scott/tiger Connected. SQL> select * from test1; ID ---------- 1 2 3 SQL> select * from test2; NAME -------------------- batman likeqiang superman *** 此时源端的scn SQL> select current_scn from v$database; CURRENT_SCN ----------- 402721
按照scn导出schema：
[oracle@ogg1 dump]$ expdp system/oracle directory=dump dumpfile=scott.dmp schemas=scott flashback_scn=402721
Export: Release 10.2.0.5.0 - 64bit Production on Tuesday, 08 July, 2014 21:36:31 Copyright (c) 2003, 2007, Oracle. All rights reserved. Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options FLASHBACK automatically enabled to preserve database integrity. Starting "SYSTEM"."SYS_EXPORT_SCHEMA_01": system/******** directory=dump dumpfile=scott.dmp schemas=scott flashback_scn=402721 Estimate in progress using BLOCKS method... Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA Total estimation using BLOCKS method: 320 KB Processing object type SCHEMA_EXPORT/USER Processing object type SCHEMA_EXPORT/SYSTEM_GRANT Processing object type SCHEMA_EXPORT/ROLE_GRANT Processing object type SCHEMA_EXPORT/DEFAULT_ROLE Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA Processing object type SCHEMA_EXPORT/TABLE/TABLE Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT . . exported "SCOTT"."DEPT" 5.656 KB 4 rows . . exported "SCOTT"."EMP" 7.820 KB 14 rows . . exported "SCOTT"."SALGRADE" 5.585 KB 5 rows . . exported "SCOTT"."TEST1" 4.929 KB 3 rows . . exported "SCOTT"."TEST2" 4.960 KB 4 rows . . exported "SCOTT"."BONUS" 0 KB 0 rows Master table "SYSTEM"."SYS_EXPORT_SCHEMA_01" successfully loaded/unloaded ****************************************************************************** Dump file set for SYSTEM.SYS_EXPORT_SCHEMA_01 is: /dump/scott.dmp Job "SYSTEM"."SYS_EXPORT_SCHEMA_01" successfully completed at 21:36:59
源端模拟生产环境做出交易：
SQL> conn scott/tiger Connected. SQL> insert into test1 values(4); 1 row created. SQL> commit; Commit complete. SQL> insert into test2 values('michael'); 1 row created. SQL> commit; Commit complete.
传送至目标端：
[root@ogg1 dump]# scp scott.dmp 192.168.100.130:/dump
root@192.168.100.130's password:
scott.dmp 100% 256KB 256.0KB/s 00:00
目标端倒入：
[root@ogg2 dump]# su - oracle
[oracle@ogg2 ~]$ cd /dump
[oracle@ogg2 dump]$ impdp system/oracle directory=dump dumpfile=scott.dmp
Import: Release 11.2.0.4.0 - Production on 星期二 7月 8 21:43:08 2014 Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved. 连接到: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options 已成功加载/卸载了主表 "SYSTEM"."SYS_IMPORT_FULL_01" 启动 "SYSTEM"."SYS_IMPORT_FULL_01": system/******** directory=dump dumpfile=scott.dmp 处理对象类型 SCHEMA_EXPORT/USER 处理对象类型 SCHEMA_EXPORT/SYSTEM_GRANT 处理对象类型 SCHEMA_EXPORT/ROLE_GRANT 处理对象类型 SCHEMA_EXPORT/DEFAULT_ROLE 处理对象类型 SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA 处理对象类型 SCHEMA_EXPORT/TABLE/TABLE 处理对象类型 SCHEMA_EXPORT/TABLE/TABLE_DATA . . 导入了 "SCOTT"."DEPT" 5.656 KB 4 行 . . 导入了 "SCOTT"."EMP" 7.820 KB 14 行 . . 导入了 "SCOTT"."SALGRADE" 5.585 KB 5 行 . . 导入了 "SCOTT"."TEST1" 4.929 KB 3 行 . . 导入了 "SCOTT"."TEST2" 4.960 KB 4 行 . . 导入了 "SCOTT"."BONUS" 0 KB 0 行 处理对象类型 SCHEMA_EXPORT/TABLE/INDEX/INDEX 处理对象类型 SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT 处理对象类型 SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS 处理对象类型 SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT 作业 "SYSTEM"."SYS_IMPORT_FULL_01" 已于 星期二 7月 8 21:43:26 2014 elapsed 0 00:00:17 成功完成
2.验证目标库数据
此时的倒入的数据是源库scn=402721时刻
SQL> conn scott/tiger 已连接。 SQL> select * from test1; ID ---------- 1 2 3 SQL> select * from test2; NAME ------------------------------------------------------------ batman likeqiang superman ***
3.源库再次做出交易
SQL> insert into test1 values(5); 1 row created. SQL> commit; Commit complete. SQL> insert into test2 values('jackson'); 1 row created. SQL> commit; Commit complete.
此时原库的trail文件te和目标库的trail文件tp应当都出现了赠长：
[root@ogg1 dirdat]# ll
总计 4
-rw-rw-rw- 1 ggs oinstall 1611 07-08 21:44 te000000
[root@ogg2 dirdat]# ll
总计 4
-rw-rw-rw- 1 ggs oinstall 1634 07-08 21:45 tp000000
4.启动replicat（aftercsn）
GGSCI (ogg2) 9> start replicat test_r,aftercsn 402721
Sending START request to MANAGER ...
REPLICAT TEST_R starting
GGSCI (ogg2) 10> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT RUNNING TEST_R 00:00:00 00:00:00
5.验证目标库数据
SQL> conn scott/tiger 已连接。 SQL> select * from test1; ID ---------- 1 2 3 4 5 SQL> select * from test2; NAME ------------------------------------------------------------ batman jackson likeqiang michael superman *** 已选择6行。
blog：http://www.cnblogs.com/kkterry/
Weibo：http://weibo.com/kkterry
E-mail：doubleginger@163.com
欢迎转载，还请标明出处！多谢多谢！
