GP维护工具-gprecoverseg

[WARNING]:-Mirror Active, Primary Failed   Not In Sync

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

[gpadmin@datacle-gp01 ~]$ gpstate -c 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[INFO]:-Starting gpstate with args: -c 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.0.0-beta.1 build dev' 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.0.0-beta.1 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit compiled on Sep 24 2021 14:13:01' 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[INFO]:-Obtaining Segment details from master... 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[INFO]:-------------------------------------------------------------- 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[INFO]:--Current GPDB mirror list and status 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[INFO]:--Type = Spread 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[INFO]:-------------------------------------------------------------- 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[INFO]:-   Status                          Data State    Primary                   Datadir                         Port    Mirror                    Datadir                         Port 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[WARNING]:-Mirror Active, Primary Failed   Not In Sync   datacle-gp03   /gpdb/gp_base/gpdatap1/gpseg0   33000   datacle-gp04   /gpdb/gp_base/gpdatam1/gpseg0   43000   <<<<<<<< 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[WARNING]:-Mirror Active, Primary Failed   Not In Sync   datacle-gp04   /gpdb/gp_base/gpdatap1/gpseg1   33000   datacle-gp05   /gpdb/gp_base/gpdatam1/gpseg1   43000   <<<<<<<< 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[WARNING]:-Mirror Active, Primary Failed   Not In Sync   datacle-gp05   /gpdb/gp_base/gpdatap1/gpseg2   33000   datacle-gp06   /gpdb/gp_base/gpdatam1/gpseg2   43000   <<<<<<<< 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[WARNING]:-Mirror Active, Primary Failed   Not In Sync   datacle-gp06   /gpdb/gp_base/gpdatap1/gpseg3   33000   datacle-gp03   /gpdb/gp_base/gpdatam1/gpseg3   43000   <<<<<<<< 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[INFO]:-------------------------------------------------------------- 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[WARNING]:-4 segment(s) configured as mirror(s) are acting as primaries 20220513:16:22:24:017955 gpstate:datacle-gp01:gpadmin-[WARNING]:-4 primary segment(s) are not synchronized

　　处理过程

[gpadmin@datacle-gp01 tmp]$ gprecoverseg -o ./recov
20220513:16:24:18:018449 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Starting gprecoverseg with args: -o ./recov
20220513:16:24:18:018449 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.0.0-beta.1 build dev'
20220513:16:24:18:018449 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.0.0-beta.1 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit compiled on Sep 24 2021 14:13:01'
20220513:16:24:18:018449 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Obtaining Segment details from master...
20220513:16:24:18:018449 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Configuration file output to ./recov successfully.

[gpadmin@datacle-gp01 tmp]$ gprecoverseg -i ./recov

+ View Code?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94

[gpadmin@datacle-gp01 tmp]$ gprecoverseg -i ./recov 20220513:16:24:28:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Starting gprecoverseg with args: -i ./recov 20220513:16:24:28:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.0.0-beta.1 build dev' 20220513:16:24:28:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.0.0-beta.1 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit compiled on Sep 24 2021 14:13:01' 20220513:16:24:28:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Obtaining Segment details from master... 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Heap checksum setting is consistent between master and the segments that are candidates for recoverseg 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Greenplum instance recovery parameters 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery from configuration -i option supplied 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery 1 of 4 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Synchronization mode                 = Incremental 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance host                 = datacle-gp03 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance address              = datacle-gp03 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance directory            = /gpdb/gp_base/gpdatap1/gpseg0 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance port                 = 33000 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance host        = datacle-gp04 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance address     = datacle-gp04 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance directory   = /gpdb/gp_base/gpdatam1/gpseg0 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance port        = 43000 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Target                      = in-place 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery 2 of 4 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Synchronization mode                 = Incremental 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance host                 = datacle-gp04 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance address              = datacle-gp04 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance directory            = /gpdb/gp_base/gpdatap1/gpseg1 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance port                 = 33000 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance host        = datacle-gp05 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance address     = datacle-gp05 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance directory   = /gpdb/gp_base/gpdatam1/gpseg1 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance port        = 43000 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Target                      = in-place 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery 3 of 4 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Synchronization mode                 = Incremental 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance host                 = datacle-gp05 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance address              = datacle-gp05 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance directory            = /gpdb/gp_base/gpdatap1/gpseg2 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance port                 = 33000 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance host        = datacle-gp06 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance address     = datacle-gp06 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance directory   = /gpdb/gp_base/gpdatam1/gpseg2 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance port        = 43000 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Target                      = in-place 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery 4 of 4 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Synchronization mode                 = Incremental 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance host                 = datacle-gp06 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance address              = datacle-gp06 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance directory            = /gpdb/gp_base/gpdatap1/gpseg3 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance port                 = 33000 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance host        = datacle-gp03 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance address     = datacle-gp03 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance directory   = /gpdb/gp_base/gpdatam1/gpseg3 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance port        = 43000 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Target                      = in-place 20220513:16:24:29:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:----------------------------------------------------------   Continue with segment recovery procedure Yy|Nn (default=N): > y 20220513:16:24:30:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Starting to create new pg_hba.conf on primary segments Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! 20220513:16:24:31:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Successfully modified pg_hba.conf on primary segments to allow replication connections 20220513:16:24:31:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-4 segment(s) to recover 20220513:16:24:31:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Ensuring 4 failed segment(s) are stopped 20220513:16:24:33:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Ensuring that shared memory is cleaned up for stopped segments 20220513:16:24:33:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Updating configuration with new mirrors 20220513:16:24:33:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Updating mirrors 20220513:16:24:33:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Running pg_rewind on failed segments datacle-gp03 (dbid 2): Done! datacle-gp04 (dbid 3): Done! datacle-gp05 (dbid 4): Done! datacle-gp06 (dbid 5): Done! 20220513:16:24:52:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Starting mirrors 20220513:16:24:52:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-era is 9bd09bcc3f1ca874_220513162204 20220513:16:24:52:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait... . 20220513:16:24:53:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Process results... 20220513:16:24:53:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Triggering FTS probe 20220513:16:24:53:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-******************************** 20220513:16:24:53:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Segments successfully recovered. 20220513:16:24:53:018630 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-********************************

[gpadmin@datacle-gp01 tmp]$ gpstate -a

提示警告信息：[WARNING]:-Total number mirror segments acting as primary segments

+ View Code?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

[gpadmin@datacle-gp01 tmp]$ gpstate -a 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-Starting gpstate with args: -a 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.0.0-beta.1 build dev' 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.0.0-beta.1 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit compiled on Sep 24 2021 14:13:01' 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-Obtaining Segment details from master... 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-Gathering data from segments... 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-Greenplum instance status summary 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:----------------------------------------------------- 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Master instance                                           = Active 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Master standby                                            = datacle-gp02 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Standby master state                                      = Standby host passive 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total segment instance count from metadata                = 8 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:----------------------------------------------------- 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Primary Segment Status 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:----------------------------------------------------- 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total primary segments                                    = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total primary segment valid (at master)                   = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total primary segment failures (at master)                = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of postmaster.pid files missing              = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of postmaster.pid files found                = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of postmaster.pid PIDs missing               = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of postmaster.pid PIDs found                 = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of /tmp lock files missing                   = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of /tmp lock files found                     = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number postmaster processes missing                 = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number postmaster processes found                   = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:----------------------------------------------------- 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Mirror Segment Status 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:----------------------------------------------------- 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total mirror segments                                     = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total mirror segment valid (at master)                    = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total mirror segment failures (at master)                 = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of postmaster.pid files missing              = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of postmaster.pid files found                = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of postmaster.pid PIDs missing               = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of postmaster.pid PIDs found                 = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of /tmp lock files missing                   = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number of /tmp lock files found                     = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number postmaster processes missing                 = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number postmaster processes found                   = 4 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[WARNING]:-Total number mirror segments acting as primary segments   = 4                         <<<<<<<< 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:-   Total number mirror segments acting as mirror segments    = 0 20220513:16:34:01:022214 gpstate:datacle-gp01:gpadmin-[INFO]:----------------------------------------------------- [gpadmin@datacle-gp01 tmp]$

$ gprecoverseg
20220513:16:35:22:022832 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Starting gprecoverseg with args:
20220513:16:35:22:022832 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.0.0-beta.1 build dev'
20220513:16:35:22:022832 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.0.0-beta.1 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit compiled on Sep 24 2021 14:13:01'
20220513:16:35:22:022832 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Obtaining Segment details from master...
20220513:16:35:22:022832 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-No segments to recover

[gpadmin@datacle-gp01 ~]$ gprecoverseg -r

+ View Code?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184

[gpadmin@datacle-gp01 ~]$ gprecoverseg -r 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Starting gprecoverseg with args: -r 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.0.0-beta.1 build dev' 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.0.0-beta.1 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit compiled on Sep 24 2021 14:13:01' 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Obtaining Segment details from master... 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Greenplum instance recovery parameters 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery type              = Rebalance 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Unbalanced segment 1 of 8 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance host        = datacle-gp04 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance address     = datacle-gp04 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance directory   = /gpdb/gp_base/gpdatam1/gpseg0 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance port        = 43000 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Balanced role                   = Mirror 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Current role                    = Primary 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Unbalanced segment 2 of 8 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance host        = datacle-gp03 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance address     = datacle-gp03 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance directory   = /gpdb/gp_base/gpdatap1/gpseg0 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance port        = 33000 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Balanced role                   = Primary 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Current role                    = Mirror 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Unbalanced segment 3 of 8 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance host        = datacle-gp05 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance address     = datacle-gp05 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance directory   = /gpdb/gp_base/gpdatam1/gpseg1 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance port        = 43000 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Balanced role                   = Mirror 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Current role                    = Primary 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Unbalanced segment 4 of 8 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance host        = datacle-gp04 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance address     = datacle-gp04 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance directory   = /gpdb/gp_base/gpdatap1/gpseg1 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance port        = 33000 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Balanced role                   = Primary 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Current role                    = Mirror 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Unbalanced segment 5 of 8 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance host        = datacle-gp06 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance address     = datacle-gp06 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance directory   = /gpdb/gp_base/gpdatam1/gpseg2 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance port        = 43000 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Balanced role                   = Mirror 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Current role                    = Primary 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Unbalanced segment 6 of 8 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance host        = datacle-gp05 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance address     = datacle-gp05 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance directory   = /gpdb/gp_base/gpdatap1/gpseg2 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance port        = 33000 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Balanced role                   = Primary 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Current role                    = Mirror 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Unbalanced segment 7 of 8 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance host        = datacle-gp03 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance address     = datacle-gp03 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance directory   = /gpdb/gp_base/gpdatam1/gpseg3 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance port        = 43000 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Balanced role                   = Mirror 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Current role                    = Primary 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Unbalanced segment 8 of 8 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance host        = datacle-gp06 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance address     = datacle-gp06 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance directory   = /gpdb/gp_base/gpdatap1/gpseg3 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Unbalanced instance port        = 33000 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Balanced role                   = Primary 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Current role                    = Mirror 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[WARNING]:-This operation will cancel queries that are currently executing. 20220513:16:35:27:022898 gprecoverseg:datacle-gp01:gpadmin-[WARNING]:-Connections to the database however will not be interrupted.   Continue with segment rebalance procedure Yy|Nn (default=N): > y 20220513:16:35:28:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Determining primary and mirror segment pairs to rebalance 20220513:16:35:28:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Stopping unbalanced primary segments... 20220513:16:35:29:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Triggering segment reconfiguration 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Starting segment synchronization 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-=============================START ANOTHER RECOVER========================================= 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.0.0-beta.1 build dev' 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.0.0-beta.1 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit compiled on Sep 24 2021 14:13:01' 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Obtaining Segment details from master... 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Heap checksum setting is consistent between master and the segments that are candidates for recoverseg 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Greenplum instance recovery parameters 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery type              = Standard 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery 1 of 4 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Synchronization mode                 = Incremental 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance host                 = datacle-gp04 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance address              = datacle-gp04 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance directory            = /gpdb/gp_base/gpdatam1/gpseg0 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance port                 = 43000 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance host        = datacle-gp03 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance address     = datacle-gp03 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance directory   = /gpdb/gp_base/gpdatap1/gpseg0 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance port        = 33000 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Target                      = in-place 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery 2 of 4 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Synchronization mode                 = Incremental 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance host                 = datacle-gp05 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance address              = datacle-gp05 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance directory            = /gpdb/gp_base/gpdatam1/gpseg1 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance port                 = 43000 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance host        = datacle-gp04 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance address     = datacle-gp04 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance directory   = /gpdb/gp_base/gpdatap1/gpseg1 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance port        = 33000 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Target                      = in-place 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery 3 of 4 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Synchronization mode                 = Incremental 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance host                 = datacle-gp06 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance address              = datacle-gp06 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance directory            = /gpdb/gp_base/gpdatam1/gpseg2 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance port                 = 43000 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance host        = datacle-gp05 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance address     = datacle-gp05 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance directory   = /gpdb/gp_base/gpdatap1/gpseg2 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance port        = 33000 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Target                      = in-place 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Recovery 4 of 4 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Synchronization mode                 = Incremental 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance host                 = datacle-gp03 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance address              = datacle-gp03 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance directory            = /gpdb/gp_base/gpdatam1/gpseg3 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Failed instance port                 = 43000 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance host        = datacle-gp06 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance address     = datacle-gp06 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance directory   = /gpdb/gp_base/gpdatap1/gpseg3 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Source instance port        = 33000 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-   Recovery Target                      = in-place 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:---------------------------------------------------------- 20220513:16:35:37:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Starting to create new pg_hba.conf on primary segments Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! Hello , Non maintenance personnel are not allowed to login ! 20220513:16:35:38:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Successfully modified pg_hba.conf on primary segments to allow replication connections 20220513:16:35:38:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-4 segment(s) to recover 20220513:16:35:38:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Ensuring 4 failed segment(s) are stopped 20220513:16:35:39:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Ensuring that shared memory is cleaned up for stopped segments 20220513:16:35:40:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Updating configuration with new mirrors 20220513:16:35:40:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Updating mirrors 20220513:16:35:40:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Running pg_rewind on failed segments datacle-gp06 (dbid 8): no rewind required datacle-gp03 (dbid 9): no rewind required datacle-gp04 (dbid 6): no rewind required datacle-gp05 (dbid 7): no rewind required 20220513:16:35:42:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Starting mirrors 20220513:16:35:42:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-era is 9bd09bcc3f1ca874_220513162204 20220513:16:35:42:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait... . 20220513:16:35:43:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Process results... 20220513:16:35:43:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Triggering FTS probe 20220513:16:35:43:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-******************************** 20220513:16:35:43:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-Segments successfully recovered. 20220513:16:35:43:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-******************************** 20220513:16:35:43:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-==============================END ANOTHER RECOVER========================================== 20220513:16:35:43:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-****************************************************************** 20220513:16:35:43:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-The rebalance operation has completed successfully. 20220513:16:35:43:022898 gprecoverseg:datacle-gp01:gpadmin-[INFO]:-******************************************************************

[gpadmin@datacle ~]$ gpstate -c
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:-Starting gpstate with args: -c
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.0.0-beta.1 build dev'
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.0.0-beta.1 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit compiled on Sep 24 2021 14:13:01'
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:-Obtaining Segment details from master...
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:--------------------------------------------------------------
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:--Current GPDB mirror list and status
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:--Type = Spread
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:--------------------------------------------------------------
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:- Status Data State Primary Datadir Port Mirror Datadir Port
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized p5-thhidam-datacle-gp03 /gpdb/gp_base/gpdatap1/gpseg0 33000 p5-thhidam-datacle-gp04 /gpdb/gp_base/gpdatam1/gpseg0 43000
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized p5-thhidam-datacle-gp04 /gpdb/gp_base/gpdatap1/gpseg1 33000 p5-thhidam-datacle-gp05 /gpdb/gp_base/gpdatam1/gpseg1 43000
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized p5-thhidam-datacle-gp05 /gpdb/gp_base/gpdatap1/gpseg2 33000 p5-thhidam-datacle-gp06 /gpdb/gp_base/gpdatam1/gpseg2 43000
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized p5-thhidam-datacle-gp06 /gpdb/gp_base/gpdatap1/gpseg3 33000 p5-thhidam-datacle-gp03 /gpdb/gp_base/gpdatam1/gpseg3 43000
20220513:19:41:56:016154 gpstate:datacle:gpadmin-[INFO]:--------------------------------------------------------------
[gpadmin@datacle ~]$