CEPH RGW集群和bucket的zone group 不一致导致的404异常解决 及 使用radosgw-admin metadata 命令设置bucket metadata 的方法
问题现象: 最近在研究zonegroup的配置操作,发现在配置zonegroup后修改了default zone,导致访问对象报404错误。
问题原因:
rgw 日志 报异常’request for data in a different zonegroup ’
rgw源码:
https://github.com/ceph/ceph/blob/d346b1b83597644b4ce58bae4bc14629f69ef667/src/rgw/rgw_op.cc
结论: bucket的zonegroup字段和集群新的id不一致。
解决方案: 使集群和bucket的zone group 一致。
1. 旧的zonegroup还存在的情况下,设置default zone group 为旧的 default。
2. 旧的zonegroup 删除的情况, 设置 dubbucket的zonegroup字段为新的zonegroup id。 可以使用如下工具批量设置bucket metadata ;注意要重启rgw 服务进程
主要配置命令:
radosgw-admin metadata get bucket:qtest3
radosgw-admin metadata get bucket.instance:qtest3:3d240698-79c4-4af0-abab-6b83b340a538.33547209.1 > conf.json
radosgw-admin metadata put bucket.instance:qtest3:3d240698-79c4-4af0-abab-6b83b340a538.33547209.1 < conf.json
cat set_bucket_meta.py #!/usr/bin/env python import rados import os import json import copy import subprocess import sys ceph_rgw_pool = ".rgw" def change_bucket_zonegroup(bucket, zgp): print("radosgw-admin metadata get bucket:" + bucket) me = os.popen("radosgw-admin metadata get bucket:" + bucket) meta = json.loads(me.read()) id = meta['data']['bucket']['bucket_id'] print("radosgw-admin metadata get bucket.instance:" + bucket + ":" + id) mei = os.popen("radosgw-admin metadata get bucket.instance:" + bucket + ":" + id) imeta = json.loads(mei.read()) newmeta = copy.copy(imeta) newmeta['data']['bucket_info']['zonegroup'] = zgp stdin = json.dumps(newmeta) print("radosgw-admin metadata put bucket.instance:" + bucket + ":" + id) process = subprocess.Popen(['radosgw-admin', 'metadata', 'put', "bucket.instance:" + bucket + ":" + id], stdin=subprocess.PIPE, stdout=subprocess.PIPE) process.stdin.write(stdin) process.stdin.close() process.wait() def get_all_buckets(): me = os.popen("radosgw-admin bucket list"); return json.loads(me.read()) if __name__ == '__main__': if sys.argv[1] == 'all': for bk in get_all_buckets(): #print(bk) change_bucket_zonegroup(bk, sys.argv[2]) #change_bucket_zonegroup('ivanwa','9affcd3e-c66c-48e4-b961-3a43a289268f') else: change_bucket_zonegroup(sys.argv[1],sys.argv[2]) #设置所有 bucket zone_group id sudo python set_bucket_meta.py all 833a4781-0da0-4810-8f0c-4e6e00b15a1e #设置名为test 的bucket zone_group id sudo python set_bucket_meta.py test 833a4781-0da0-4810-8f0c-4e6e00b15a1e
参考:https://blog.widodh.nl/2013/11/changing-the-region-of-a-rgw-bucket/