QEMU命令练习
Openstack nova/glance涉及很多instance、image、snapshot的概念和操作，一直搞不清楚。其中很多跟qemu有关系，底层都是通过libvirt调用kvm/qemu接口实现的。
所以希望从qemu命令入手，对此有所了解。
官网：
http://wiki.qemu.org/Main_Page
http://wiki.qemu.org/Manual
源码安装
https://en.wikibooks.org/wiki/QEMU/Linux
http://my.oschina.net/kelvinxupt/blog/265108
[felix@centos65 ~]$ wget http://wiki.qemu-project.org/download/qemu-2.3.1.tar.bz2
[felix@centos65 ~]$ tar xjvf qemu-2.3.1.tar.bz2
[felix@centos65 ~]$ cd qemu-2.3.1
[felix@centos65 qemu-2.3.1]$ sudo yum install zlib-devel
[felix@centos65 qemu-2.3.1]$ sudo yum install glib2-devel -y
[felix@centos65 qemu-2.3.1]$ sudo yum install install autoconf automake libtool
[felix@centos65 qemu-2.3.1]$ ./configure --enable-kvm --enable-debug --enable-vnc --target-list="x86_64-softmmu"
[felix@centos65 qemu-2.3.1]$ sudo make install
.....
install -d -m 0755 "/usr/local/bin"
install -c -m 0755 qemu-system-x86_64 "/usr/local/bin"
[felix@centos65 qemu-2.3.1]$ qemu-
qemu-ga qemu-img qemu-io qemu-nbd qemu-system-x86_64
[felix@centos65 qemu-2.3.1]$ which qemu-img
/usr/local/bin/qemu-img
[felix@centos65 qemu-2.3.1]$ which qemu-system-x86_64
/usr/local/bin/qemu-system-x86_64几个概念
snapshot
Snapshots in QEMU are images that refer to an original image using Redirect-on-Write to avoid changing the original image
参考2
Copy-on-Write（写时拷贝）
- A snapshot of a storage volume is created using the pre-designated space for the snapshot. （空间预留）
- When the snapshot is first created, only the meta-data about where original data is stored is copied. No physical copy of the data is done at the time the snapshot is created.（刚创建时只记录原始数据的位置）
- The snapshot copy then tracks the changing blocks on the original volume as writes to the original volume are performed.（记录原始数据的变化）
- The original data that is being written to is copied into the designated storage pool that is set aside for the snapshot before original data is overwritten, hence the name “copy-on-write”.（当原始数据被覆盖之前，原始数据会被copy到snapshot预留的空间中）
因此：
- This keeps the snapshot data consistent with the exact time the snapshot was taken.
- Read requests to the snapshot volume of the unchanged data blocks are redirected to the “copied” blocks in the snapshot, while read requests to active data blocks that have been changed are directed to the original volume. （访问snapshot时：读取未改变的数据的请求会被重定向到原来的volume block中，读取改变了的数据的请求会被发送到snapshot所在的block中 - 原文有点不对啊）
- Snapshot contains the meta-data that describes the data blocks that have changed since the snapshot was first created.（snapshot中的meta-data记录了数据的变化）
优点：
- 节省空间
缺点：
- 写数据时，必须先copy原来的数据，影响性能
- snapshot数据完备性依赖于原始数据
Redirect-on-write（写时重定向）
和copy-on-write相似，但没有两次写的动作：
- New writes to the original volume are redirected to another location set aside for snapshot.
（对原始数据执行写操作时，新的数据被写入到了snapshot所在的block中）
优点：
- 没有两次写的动作
缺点：
- 当删除snapshot时，其中的数据必须要reconciled back into the original volume
- 做多次snapshot时，情况更复杂
- snapshot数据完备性依赖于原始数据
（原文还有其他方式，这里不做记录）
思考：
nova boot instance用了qemu create with backing_file（？）即redirect-on-write方式，instance/_base/xxxxx（backing_file）就是original data，instance/yyyy/disk就相当于snapshot data。当nova存储后端为ceph时，没有backing_file和disk文件，那是怎么回事？
Update：
- 当nova image_type不为ceph时（默认qcow2），instance基本都有backing_file，除了openstack自带的cirros镜像
- 当nova image_type=ceph时，instance/yyy/目录下没有disk，disk存储在ceph中，例如：
# libvirt.xml <disk type="network" device="disk"> <driver type="raw" cache="writeback"/> <source protocol="rbd" name="vms/eb511866-3b8f-40e0-9650-ea040b439a7f_disk"> <host name="10.254.3.111" port="6789"/> </source> <auth username="cinder"> <secret type="ceph" uuid="ea60f8ea-18d1-4dba-afa9-6528b1685100"/> </auth> <target bus="virtio" dev="vda"/> </disk> op@ubuntu-op:/home/openstack/workspace/data/nova/instances$ qemu-img info rbd:vms/eb511866-3b8f-40e0-9650-ea040b439a7f_disk:id=cinder:conf=/etc/ceph/ceph.conf image: rbd:vms/eb511866-3b8f-40e0-9650-ea040b439a7f_disk:id=cinder:conf=/etc/ceph/ceph.conf file format: raw virtual size: 40G (42949672960 bytes) disk size: unavailable cluster_size: 4194304
对于raw格式的image启动instance，也不是用的redirect-on-write方式吧（因为没有backing_file）？
Update：用Openstack自带的cirros创建的虚机没有backing_file（因为它用kernel/ramdisk?）[admin@maqi-openstack instances]$ ls d8e20388-2ff0-4667-a9b3-29921bf415be/ console.log disk.config disk.info kernel libvirt.xml ramdisk但是自建的raw格式glance image，启动的虚机还是有backing_file的
nova image-create（Create a new image by taking a snapshot of a running server）是怎么回事？
Update：这个就是nova snapshot功能，它和qemu-img snapshot不是一回事。
lihao：(nova/virt/libvirt/driver.py _live_snapshot)
- 创建临时镜像，使用和instance相同的backing_file
qemu-img create yyy.delta -b _base/xxxx -f qcow2 - blockRebase
- 去掉临时镜像的backing_file
qemu-img convert -f qcow2 -O qcow2 yyy.delta - 上传snapshot到glance
- 删除临时镜像
- 创建临时镜像，使用和instance相同的backing_file
- 如果image有backing_file，对这个image做snapshot，会怎样？（尝试nova snapshot）
Update：见上。
base image/backing file
创建镜像的时候，可以使用-b指定backing_file，也叫base image。这样做的好处：
- 创建的镜像文件可以很小
- 用这个镜像启动虚拟机之后，对虚拟机所做的改动只写入镜像文件，不会（也不能）修改base image
- base image可以重用
qcow2，raw
https://en.wikibooks.org/wiki/QEMU/Images
raw:
(default) the raw format is a plain binary image of the disc image, and is very portable. On filesystems that support sparse files, images in this format only use the space actually used by the data recorded in them.
qcow2:
QEMU Copy-On-Write format with a range of special features, including the ability to take multiple snapshots, smaller images on filesystems that don’t support sparse files, optional AES encryption, and optional zlib compression
https://people.gnome.org/~markmc/qcow-image-format.html
http://www.ibm.com/developerworks/cn/linux/1409_qiaoly_qemuimgages/
命令
qemu-ga
QEMU Guest Agent
以后再看。
qemu-io
http://manned.org/qemu-io/8ccaf7b2
qemu-io is a command line utility to exercise the QEMU I/O path.
以后再看。
qemu-nbd
[admin@maqi-centos7 ~]$ qemu-nbd -h
Usage: qemu-nbd [OPTIONS] FILE
QEMU Disk Network Block Device ServerNBD：Network Block Device
简单来说，qemu-nbd可以把一个qemu disk image通过nbd方式挂载到系统上：
[admin@maqi-centos7 ~]$ sudo qemu-nbd --connect=/dev/nbd0 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
Failed to open /dev/nbd0: No such file or directory[admin@maqi-centos7 ~]$ sudo touch /dev/nbd0
[admin@maqi-centos7 ~]$ sudo qemu-nbd --connect=/dev/nbd0 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
nbd.c:nbd_init():L571: Failed to set NBD socket需要kernel支持nbd：http://www.aixchina.net/Question/123287?p=1
找另一个环境试验：
felix@ubuntu14-home:~|⇒ sudo qemu-nbd -c /dev/nbd0 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2
felix@ubuntu14-home:~|⇒ ll /dev/nbd0
brw-rw---- 1 root disk 43, 0 9月 21 03:05 /dev/nbd0
felix@ubuntu14-home:~|⇒ sudo fdisk /dev/nbd0
Command (m for help): p
Disk /dev/nbd0: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00010f6d
Device Boot Start End Blocks Id System
/dev/nbd0p1 * 2048 20971519 10484736 83 Linux
Command (m for help): q
felix@ubuntu14-home:~|⇒ sudo mount /dev/nbd0p1 /mnt
felix@ubuntu14-home:~|⇒ ll /mnt/home
total 4.0K
drwx------ 4 500 500 4.0K 2月 10 2015 admin
felix@ubuntu14-home:~|⇒ mount | grep nbd
/dev/nbd0p1 on /mnt type ext4 (rw)
felix@ubuntu14-home:~|⇒ sudo umount /mnt
felix@ubuntu14-home:~|⇒ sudo qemu-nbd -d /dev/nbd0
/dev/nbd0 disconnected这种是本地的镜像，当然也可以通过IP地址挂载网络上的远程镜像。
另外可以参考：
https://www.kumari.net/index.php/system-adminstration/49-mounting-a-qemu-image
http://edoceo.com/cli/qemu
https://en.wikibooks.org/wiki/QEMU/Images#Mounting_an_image_on_the_host
http://smilejay.com/2012/11/how-to-mount-a-qcow2-image/ (qcow2格式和raw格式方法不同)
Openstack中有qemu-nbd相关的code：
https://github.com/openstack/nova/blob/master/nova/virt/disk/mount/nbd.py
qemu-img
qemu-img allows you to create, convert and modify images offline. It can handle all image formats supported by QEMU.
使用格式：qemu-img command [command options]
支持以下命令
check [-q] [-f fmt] [–output=ofmt] [-r [leaks | all]] [-T src_cache] filename
对镜像做一致性检查(consistency check)，只支持”qcow2”, “qed” and “vdi”[admin@maqi-centos7 ~]$ qemu-img check bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 -- output=json { "image-end-offset": 616562688, "compressed-clusters": 22160, "total-clusters": 163840, "check-errors": 0, "allocated-clusters": 24398, "filename": "bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2", "format": "qcow2", "fragmented-clusters": 22453 }create [-q] [-f fmt] [-o options] filename [size]
创建镜像。options表示打开additional feature，例如backing_file:If the option backing_file is specified, then the image will record only the differences from backing_file. No size needs to be specified in this case.
backing_file will never be modified unless you use the “commit” monitor command (or qemu-img commit).nova中会用到这个参数：
# nova/virt/libvirt/utils.py def create_cow_image(backing_file, path, size=None): """Create COW image Creates a COW image with the given backing file :param backing_file: Existing image on which to base the COW image :param path: Desired location of the COW image """ base_cmd = ['qemu-img', 'create', '-f', 'qcow2'] cow_opts = [] if backing_file: cow_opts += ['backing_file=%s' % backing_file] base_details = images.qemu_img_info(backing_file) else: base_details = None if base_details and base_details.cluster_size is not None: cow_opts += ['cluster_size=%s' % base_details.cluster_size] if size is not None: cow_opts += ['size=%s' % size] if cow_opts: # Format as a comma separated list csv_opts = ",".join(cow_opts) cow_opts = ['-o', csv_opts] cmd = base_cmd + cow_opts + [path] execute(*cmd)这个方法最终会被nova/virt/libvirt/driver.py中的snapshot方法和nova/virt/libvirt/imagebackend.py中的Qcow2::create_image用到。
不使用backing_file:
[admin@maqi-centos7 qemu]$ qemu-img create -f qcow2 test.qcow 10G [admin@maqi-centos7 qemu]$ qemu-img info test.qcow image: test.qcow file format: qcow2 virtual size: 10G (10737418240 bytes) disk size: 196K cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false使用backing_file：
[admin@maqi-centos7 qemu]$ qemu-img create -f qcow2 -b bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 saved.qcow2 Formatting 'saved.qcow2', fmt=qcow2 size=10737418240 backing_file='bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2' encryption=off cluster_size=65536 lazy_refcounts=off [admin@maqi-centos7 qemu]$ ls -ltrh total 589M -rw-r-----. 1 admin admin 589M Sep 27 03:27 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 -rw-r--r--. 1 admin admin 193K Sep 27 06:42 saved.qcow2 [admin@maqi-centos7 qemu]$ qemu-img info saved.qcow2 image: saved.qcow2 file format: qcow2 virtual size: 10G (10737418240 bytes) disk size: 196K cluster_size: 65536 backing file: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 Format specific information: compat: 1.1 lazy refcounts: falsecommit [-q] [-f fmt] [-t cache] filename
Commit the changes recorded in filename in its base image or backing file.
If the backing file is smaller than the snapshot, then the backing file will be resized to be the same size as the snapshot.compare [-f fmt] [-F fmt] [-T src_cache] [-p] [-q] [-s] filename1 filename2
Check if two images have the same content. You can compare images with different format or settings.
convert [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-o options] [-s snapshot_name] [-S sparse_size] filename [filename2 […]] output_filename
Convert the disk image filename or a snapshot snapshot_name to disk image output_filename using format output_fmt. It can be optionally compressed (“-c” option) or use any format specific options like encryption (“-o” option).
You can use the backing_file option to force the output image to be created as a copy on write image of the specified base image; the backing_file should have the same content as the input’s base image, however the path, image format, etc may differ.
If the “-n” option is specified, the target volume creation will be skipped. This is useful for formats such as “rbd” if the target volume has already been created with site specific options that cannot be supplied through qemu-img.
# qcow2 -> raw [admin@maqi-centos7 ~]$ qemu-img convert -O raw bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw # raw -> qcow2 [admin@maqi-centos7 ~]$ qemu-img convert -O qcow2 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.2 # raw -> qcow2 (compress) [admin@maqi-centos7 ~]$ qemu-img convert -O qcow2 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.3 -c [admin@maqi-centos7 ~]$ ls -ltrh bcec* -rw-r-----. 1 admin admin 588M Sep 26 03:59 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 -rw-r--r--. 1 admin admin 10G Sep 26 11:12 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw -rw-r--r--. 1 admin admin 1.5G Sep 26 11:15 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.2 -rw-r--r--. 1 admin admin 586M Sep 26 11:18 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.3 [admin@maqi-centos7 ~]$ qemu-img info bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 image: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 file format: qcow2 virtual size: 10G (10737418240 bytes) disk size: 588M cluster_size: 65536 Format specific information: compat: 0.10 [admin@maqi-centos7 ~]$ qemu-img info bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.2 image: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.2 file format: qcow2 virtual size: 10G (10737418240 bytes) disk size: 1.5G cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false [admin@maqi-centos7 ~]$ qemu-img info bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.3 image: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2.3 file format: qcow2 virtual size: 10G (10737418240 bytes) disk size: 585M cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false [admin@maqi-centos7 ~]$ qemu-img info bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw image: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw file format: raw virtual size: 10G (10737418240 bytes) disk size: 1.5G1.
virtual_size都是一样的，表示用该image装的系统的disk大小
2.disk size表示image实际的大小info [-f fmt] [–output=ofmt] [–backing-chain] filename
查看镜像信息[root@node-7 instances]# pwd /var/lib/nova/instances [root@node-7 instances]# ll total 8 drwxr-xr-x 2 nova nova 69 Sep 25 04:22 60a7a481-baea-47b4-9e95-f28c2fb2e2e1 drwxr-xr-x 2 nova nova 69 Sep 22 03:18 6839e37f-d3b6-49f5-bc50-8393daf5f918 drwxr-xr-x 2 nova nova 53 Sep 26 08:43 _base -rw-r--r-- 1 nova nova 40 Sep 26 11:23 compute_nodes drwxr-xr-x 2 nova nova 69 Sep 25 03:08 e8c36933-fd3f-415d-8b50-bb365afb9a12_resize drwxr-xr-x 2 nova nova 69 Sep 25 08:36 f7345e0d-0f02-4ad8-9b3d-9082a433ecb6 drwxr-xr-x 2 nova nova 4096 Sep 25 02:16 locks drwxr-xr-x 2 nova nova 6 Aug 4 02:19 snapshots [root@node-7 instances]# qemu-img info f7345e0d-0f02-4ad8-9b3d-9082a433ecb6/disk image: f7345e0d-0f02-4ad8-9b3d-9082a433ecb6/disk file format: qcow2 virtual size: 40G (42949672960 bytes) disk size: 8.4M cluster_size: 65536 backing file: /var/lib/nova/instances/_base/da9c151216e24ec6136b139737006287fa476882 Format specific information: compat: 1.1 lazy refcounts: false [root@node-7 instances]# [root@node-7 instances]# qemu-img info /var/lib/nova/instances/_base/ da9c151216e24ec6136b139737006287fa476882 image: /var/lib/nova/instances/_base/da9c151216e24ec6136b139737006287fa476882 file format: raw virtual size: 10G (10737418240 bytes) disk size: 1.5G [root@node-7 instances]# qemu-img info 60a7a481-baea-47b4-9e95-f28c2fb2e2e1/disk image: 60a7a481-baea-47b4-9e95-f28c2fb2e2e1/disk file format: qcow2 virtual size: 20G (21474836480 bytes) disk size: 83M cluster_size: 65536 backing file: /var/lib/nova/instances/_base/da9c151216e24ec6136b139737006287fa476882 Format specific information: compat: 1.1 lazy refcounts: false是不是这样：
1）在Openstack中，用qcow2镜像启动的VM，都会有backing_file，并且这个backing_file是可以共享的。（nova用ceph做后端时，看不到backing_file。见“其他”）
2）nova会把这个qcow2镜像convert成raw格式，来作为backing_file（Yes）map [-f fmt] [–output=ofmt] filename
Dump the metadata of image filename and its backing file chain.
snapshot [-q] [-l | -a snapshot | -c snapshot | -d snapshot] filename
List, apply, create or delete snapshots in image filename
[admin@maqi-centos7 ~]$ qemu-img snapshot -l bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 [admin@maqi-centos7 ~]$ qemu-img snapshot -c snapshot-1 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress. qcow2 [admin@maqi-centos7 ~]$ qemu-img snapshot -l bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 Snapshot list: ID TAG VM SIZE DATE VM CLOCK 1 snapshot-1 0 2015-09-27 03:27:26 00:00:00.000 [admin@maqi-centos7 ~]$ qemu-img snapshot -c snapshot-2 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress. qcow2 [admin@maqi-centos7 ~]$ qemu-img snapshot -l bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 Snapshot list: ID TAG VM SIZE DATE VM CLOCK 1 snapshot-1 0 2015-09-27 03:27:26 00:00:00.000 2 snapshot-2 0 2015-09-27 03:27:46 00:00:00.000 [admin@maqi-centos7 ~]$ ll total 4318928 -rw-r-----. 1 admin admin 616759808 Sep 27 03:27 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 -rw-r--r--. 1 admin admin 1599733760 Sep 26 11:15 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress. qcow2.2 -rw-r--r--. 1 admin admin 613959168 Sep 26 11:18 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress. qcow2.3 -rw-r--r--. 1 admin admin 10737418240 Sep 26 11:12 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.raw drwxrwxr-x. 15 admin admin 4096 Sep 8 00:25 devstack -rw-rw-r--. 1 admin admin 2268 Aug 18 13:50 localrc -rw-rw-r--. 1 admin admin 825 Sep 13 08:06 send.py [admin@maqi-centos7 ~]$ qemu-img info bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 image: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 file format: qcow2 virtual size: 10G (10737418240 bytes) disk size: 588M cluster_size: 65536 Snapshot list: ID TAG VM SIZE DATE VM CLOCK 1 snapshot-1 0 2015-09-27 03:27:26 00:00:00.000 2 snapshot-2 0 2015-09-27 03:27:46 00:00:00.000 Format specific information: compat: 0.10Q：snapshot-1，snapshot-2存储在哪儿？？
rebase [-q] [-f fmt] [-t cache] [-T src_cache] [-p] [-u] -b backing_file [-F backing_fmt] filename
改变image的backing_fileChanges the backing file of an image. Only the formats “qcow2” and “qed” support changing the backing file.
The backing file is changed to backing_file and (if the image format of filename supports this) the backing file format is changed to backing_fmt. If backing_file is specified as “” (the empty string), then the image is rebased onto no backing file (i.e. it will exist independently of any backing file).
[admin@maqi-centos7 qemu]$ qemu-img info saved.qcow2 image: saved.qcow2 file format: qcow2 virtual size: 10G (10737418240 bytes) disk size: 196K cluster_size: 65536 backing file: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 Format specific information: compat: 1.1 lazy refcounts: false [admin@maqi-centos7 qemu]$ qemu-img rebase -b bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 saved .qcow2 [admin@maqi-centos7 qemu]$ ls -ltrh total 589M -rw-r-----. 1 admin admin 589M Sep 27 03:27 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 -rw-r--r--. 1 admin admin 193K Sep 27 06:43 saved.qcow2 [admin@maqi-centos7 qemu]$ qemu-img info saved.qcow2 image: saved.qcow2 file format: qcow2 virtual size: 10G (10737418240 bytes) disk size: 196K cluster_size: 65536 backing file: bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 Format specific information: compat: 1.1 lazy refcounts: false [admin@maqi-centos7 qemu]$ qemu-img rebase -b "" saved.qcow2 [admin@maqi-centos7 qemu]$ ls -ltrh total 2.6G -rw-r-----. 1 admin admin 589M Sep 27 03:27 bcec-centos-6.5-amd64-server-10G-v4-ovirt.compress.qcow2 -rw-r--r--. 1 admin admin 1.5G Sep 27 06:47 saved.qcow2 [admin@maqi-centos7 qemu]$ qemu-img info saved.qcow2 image: saved.qcow2 file format: qcow2 virtual size: 10G (10737418240 bytes) disk size: 2.0G cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: falseresize [-q] filename [+ | -]size
Change the disk image as if it had been created with size.
Before using this command to shrink a disk image, you MUST use file system and partitioning tools inside the VM to reduce allocated file systems and partition sizes accordingly. Failure to do so will result in data loss!
After using this command to grow a disk image, you must use file system and partitioning tools inside the VM to actually begin using the new space on the device.
amend [-q] [-f fmt] [-t cache] -o options filename
Amends the image format specific options for the image file filename. Not all file formats support this operation.
支持格式：
vvfat vpc vmdk vhdx vdi sheepdog rbd raw host_cdrom host_floppy host_device file qed qcow2 qcow parallels nbd iscsi gluster dmg cloop bochs blkverify blkdebug
qemu-system-i386
qemu-system-x86_64
这两个命令都用来启动虚拟机，参数比较多，可以google。
以下是nova起的一个虚拟机：
qemu 21604 1 0 Sep25 ? 00:05:04 /usr/bin/qemu-system-x86_64 -name instance-00000005 -S -machine pc-i440fx-2.0,accel=tcg,usb=off -m 512 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid d8e20388-2ff0-4667-a9b3-29921bf415be -smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=12.0.0,serial=dae72fe0-cc06-4eb0-b779-7f25bfaf69df,uuid=d8e20388-2ff0-4667-a9b3-29921bf415be,family=Virtual Machine -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000005.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -kernel /home/openstack/workspace/data/nova/instances/d8e20388-2ff0-4667-a9b3-29921bf415be/kernel -initrd /home/openstack/workspace/data/nova/instances/d8e20388-2ff0-4667-a9b3-29921bf415be/ramdisk -append root=/dev/vdb console=tty0 console=ttyS0 no_timer_check -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/openstack/workspace/data/nova/instances/d8e20388-2ff0-4667-a9b3-29921bf415be/disk.config,if=none,id=drive-ide0-1-1,readonly=on,format=raw,cache=none -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1 -drive file=/dev/disk/by-path/ip-10.133.6.83:3260-iscsi-iqn.2010-10.org.openstack:volume-ede0b75e-75c8-4c56-a372-052ebcb5e9db-lun-1,if=none,id=drive-virtio-disk1,format=raw,serial=ede0b75e-75c8-4c56-a372-052ebcb5e9db,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk1,id=virtio-disk1,bootindex=1 -netdev tap,fd=23,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:a0:1e:72,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/home/openstack/workspace/data/nova/instances/d8e20388-2ff0-4667-a9b3-29921bf415be/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -vnc 127.0.0.1:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on其他
nova都是调用的libvirt接口，跟上面命令执行的方式会有不同，还需要看看libvirt的接口。
Openstack中，nova/glance的后端都使用ceph时
1）glance的image目录为空：op@ubuntu-op:/home/openstack/workspace/data$ ll glance/images/ total 8 drwxr-xr-x 2 op op 4096 9月 16 00:22 ./ drwxr-xr-x 4 op op 4096 9月 16 00:22 ../ op@ubuntu-op:/home/openstack/workspace/data$ ll glance/cache/ total 635236 drwxr-xr-x 5 op op 4096 9月 24 19:18 ./ drwxr-xr-x 4 op op 4096 9月 16 00:22 ../ -rw-r----- 1 op op 4979632 9月 16 18:03 373b5d20-c82a-4f90-b8e4-66bc21e37c4b -rw-r----- 1 op op 25165824 9月 16 00:29 4d076e72-110d-4446-a20d-3716ad5e1d4a -rw-r----- 1 op op 3740163 9月 16 18:03 62540238-1fee-46da-a288-17ab31b1985e -rw-r----- 1 op op 616562688 9月 24 19:12 989d150b-ddb7-4079-83e4-2bbc2f01cd2c -rw-r----- 1 op op 3072 9月 24 19:18 cache.db drwxr-x--- 2 op op 4096 9月 24 19:12 incomplete/ drwxr-x--- 2 op op 4096 9月 16 00:23 invalid/ drwxr-x--- 2 op op 4096 9月 16 00:23 queue/2）nova下也有_base目录
op@ubuntu-op:/home/openstack/workspace/data/nova/instances$ ls 4d896e1c-30dc-4f76-97cc-b181953e622c ad0f2d70-4773-40b8-a628-1ca2f88b8531 eb511866-3b8f-40e0-9650- ea040b439a7f snapshots 5b14e497-09dc-4364-9862-adb5a2dc9241 _base f9b84ae7-0075-4e75-a873- e83e66d60683 7c303787-d2d8-43b1-a049-2cfc79507b2b compute_nodes locks op@ubuntu-op:/home/openstack/workspace/data/nova/instances$ ll _base total 1563504 drwxr-xr-x 2 op libvirtd 4096 9月 25 22:54 ./ drwxr-xr-x 11 op root 4096 9月 28 18:17 ../ -rw-r--r-- 1 op libvirtd 4979632 9月 16 18:03 373b5d20-c82a-4f90-b8e4-66bc21e37c4b -rw-r--r-- 1 op libvirtd 3740163 9月 16 18:03 62540238-1fee-46da-a288-17ab31b1985e -rw-r--r-- 1 op libvirtd 10737418240 9月 28 19:05 8eaa8bae3127538b02375cf3b2d000951de1b64e op@ubuntu-op:/home/openstack/workspace/data$ ll nova/instances/5b14e497-09dc-4364-9862-adb5a2dc9241/ total 436 drwxr-xr-x 2 op libvirtd 4096 9月 27 04:37 ./ drwxr-xr-x 10 op root 4096 9月 25 17:54 ../ -rw-rw---- 1 libvirt-qemu kvm 6367 9月 25 17:56 console.log -rw-r--r-- 1 libvirt-qemu kvm 419840 9月 25 17:54 disk.config -rw-r--r-- 1 op libvirtd 106 9月 25 17:54 disk.info -rw-r--r-- 1 op libvirtd 3008 9月 25 17:54 libvirt.xml op@ubuntu-op:/home/openstack/workspace/data$ qemu-img info nova/instances/5b14e497-09dc-4364-9862- adb5a2dc9241/disk.config image: nova/instances/5b14e497-09dc-4364-9862-adb5a2dc9241/disk.config file format: raw virtual size: 410K (419840 bytes) disk size: 412K op@ubuntu-op:/home/openstack/workspace/data$ qemu-img info nova/instances/5b14e497-09dc-4364-9862- adb5a2dc9241/disk.info image: nova/instances/5b14e497-09dc-4364-9862-adb5a2dc9241/disk.info file format: raw virtual size: 512 (512 bytes) disk size: 4.0Klibvirt.xml中关于disk的部分：
<disk type="network" device="disk"> <driver type="raw" cache="writeback"/> <source protocol="rbd" name="vms/5b14e497-09dc-4364-9862-adb5a2dc9241_disk"> <host name="10.254.3.111" port="6789"/> </source> <auth username="cinder"> <secret type="ceph" uuid="ea60f8ea-18d1-4dba-afa9-6528b1685100"/> </auth> <target bus="virtio" dev="vda"/> </disk> <disk type="file" device="cdrom"> <driver name="qemu" type="raw" cache="none"/> <source file="/home/openstack/workspace/data/nova/instances/5b14e497-09dc-4364-9862-adb5a2dc9241/disk. config"/> <target bus="ide" dev="hdd"/> </disk>ceph中的image是raw格式，没有backing_file：
op@ubuntu-op:/home/openstack/workspace/data/nova/instances$ qemu-img info rbd:vms/5b14e497-09dc-4364-9862- adb5a2dc9241_disk:id=cinder:conf=/etc/ceph/ceph.conf image: rbd:vms/5b14e497-09dc-4364-9862-adb5a2dc9241_disk:id=cinder:conf=/etc/ceph/ceph.conf file format: raw virtual size: 40G (42949672960 bytes) disk size: unavailable cluster_size: 4194304- KVM 介绍（7）：使用 libvirt 做 QEMU/KVM 快照和 Nova 实例的快照 （Nova Instances Snapshot Libvirt）
这篇文章总结得很全，里面的一些链接文章也很好 -
Internal Snapshots – A single qcow2 image file holds both the saved state
& the delta since that saved point. This can be further classified as :-Internal disk snapshot: The state of the virtual disk at a given point in time. Both the snapshot & delta since the snapshot are stored in the same qcow2 file. Can be taken when the guest is ‘live’ or ‘offline’.
- Libvirt uses QEMU’s ‘qemu-img’ command when the guest is ‘offline’.
- Libvirt uses QEMU’s ‘savevm’ command when the guest is ‘live’.
Internal system checkpoint: RAM state, device state & the disk-state of a running guest, are all stored in the same originial qcow2 file. Can be taken when the guest is running ‘live’.
- Libvirt uses QEMU’s ‘savevm’ command when the guest is ‘live’
External Snapshots – Here, when a snapshot is taken, the saved state will be stored in one file(from that point, it becomes a read-only backing file) & a new file(overlay) will track the deltas from that saved state. This can be further classified as :-
External disk snapshot: The snapshot of the disk is saved in one file, and the delta since the snapshot is tracked in a new qcow2 file. Can be taken when the guest is ‘live’ or ‘offline’.
- Libvirt uses QEMU’s ‘transaction’ cmd under the hood, when the guest is ‘live’.
- Libvirt uses QEMU’s ‘qemu-img’ cmd under the hood when the guest is ‘offline’(this implementation is in progress, as of writing this).
External system checkpoint: Here, the guest’s disk-state will be saved in one file, its RAM & device-state will be saved in another new file (This implementation is in progress upstream libvirt, as of writing this).
VM State: Saves the RAM & device state of a running guest(not ‘disk-state’) to a file, so that it can be restored later. This simliar to doing hibernate of the system. (NOTE: The disk-state should be unmodified at the time of restoration.)
- Libvirt uses QEMU’s ‘migrate’ (to file) cmd under the hood.
- TBD
