AWS s3存储桶迁移

架构图

在目标 AWS 账户中创建 IAM 用户和角色

创建基于 IAM 身份的策略

s3_smartholder_sync

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:GetObjectTagging",
                "s3:GetObjectVersion",
                "s3:GetObjectVersionTagging"
            ],
            "Resource": [
                "arn:aws:s3:::awsexamplesourcebucket",  # 源数据桶
                "arn:aws:s3:::awsexamplesourcebucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:PutObjectTagging",
                "s3:GetObjectTagging",
                "s3:GetObjectVersion",
                "s3:GetObjectVersionTagging"
            ],
            "Resource": [
                "arn:aws:s3:::awsexampledestinationbucket",   
                "arn:aws:s3:::awsexampledestinationbucket/*"
            ]
        }
    ]
}

AmazonS3Smartdao-imgAccess

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::xxxxxxx"  # 替换为自己的桶
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "s3-object-lambda:*",
            "Resource": "arn:aws:s3:::xxxxxxx"   # 替换为自己的桶
        }
    ]
}

创建一个 IAM 用户

 

设置安全凭证

访问密钥

控制台登录

查看权限策略

创建角色

信任策略

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<destination_account>:user/<user_name>"
            },
            "Action": "sts:AssumeRole",
            "Condition": {}
        }
    ]
}

在源账户中创建并附加 S3 存储桶策略

存储桶策略位置

存储桶策略

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DelegateS3Access",
            "Effect": "Allow",
            "Principal": {"AWS": "arn:aws:iam::<destination_account>:user/<RoleName>"},   # destination role
            "Action": ["s3:ListBucket",
                       "s3:GetObject",
                       "s3:GetObjectTagging",
                       "s3:GetObjectVersion",
                       "s3:GetObjectVersionTagging"
                       ],
            "Resource": [
                "arn:aws:s3:::awsexamplesourcebucket/*",
                "arn:aws:s3:::awsexamplesourcebucket"
            ]
        }
    ]
}

awscli

配置 awscli iam

# aws configure

AWS Access Key ID [None]: xxxxxxx                 # iam 用户的 访问密钥 key
AWS Secret Access Key [None]: xxxxxx              # iam 用户的 访问秘钥 value
Default region name [None]: ap-northeast-1
Default output format [None]: json

awscli 加载 role

# aws sts assume-role --role-arn "arn:aws:iam::<destination_account>:role/<RoleName>" --role-session-name AWSCLI-Session

设置环境变量

export AWS_ACCESS_KEY_ID=RoleAccessKeyID
export AWS_SECRET_ACCESS_KEY=RoleSecretKey
export AWS_SESSION_TOKEN=RoleSessionToken

查看角色

# aws sts get-caller-identity

验证权限

# aws s3 ls s3://xxxxx --region ap-northeast-1

同步数据

# aws s3 sync s3://DOC-EXAMPLE-BUCKET-SOURCE s3://DOC-EXAMPLE-BUCKET-TARGET --source-region ap-northeast-1 --region ap-northeast-1

参考文档

https://docs.aws.amazon.com/zh_cn/prescriptive-guidance/latest/patterns/copy-data-from-an-s3-bucket-to-another-account-and-region-by-using-the-aws-cli.html