【dlib基础】dlib和opencv图片转换以及提取ROI

前言

  最近使用dlib进行红外图像的人脸检测，发现效果还不错，想要extract检测到的人脸区域作为数据集，其中需要从dlib图像类型数据中提取dlib::rectangle类型的人脸区域。

code

1. 将dlib image转换为opencv cv::Mat类型，同时提取图像区域的数据类型也是cv::Mat类型；

// Wrap dlib-image in cv::Mat
// dlib::toMat() is available through #include <dlib/opencv.h>
cv::Mat sample_mat = dlib::toMat(img); 
cv::Mat rect_sub = sample_mat(cv::Rect(rect_dlib.left(), rect_dlib.top(), rect_dlib.width(), rect_dlib.height()));

注意，需要用到opencv API，以及opencv开源库；

2. 使用dlib中的extract_image_chips / extract_image_chip 功能提取ROI区域，保存图像；

array2d<unsigned char> img;
load_image(img, imgpath);
array2d<unsigned char> face;
const dlib::chip_details chip_details(rect);
dlib::extract_image_chip(img, chip_details, face, dlib::interpolate_bilinear());
snprintf(imgpath, 64, "../face/face_%06d.jpg", k);
save_jpeg(face, imgpath, 100);

 extract_image_chip

    template <
        typename image_type1,
        typename image_type2,
        typename interpolation_type
        >
    void extract_image_chip (
        const image_type1& img,
        const chip_details& chip_location,
        image_type2& chip,
        const interpolation_type& interp
    );
    /*!
        ensures
            - This function simply calls extract_image_chips() with a single chip location
              and stores the single output chip into #chip.  It uses the provided
              interpolation method.
    !*/

    template <
        typename image_type1,
        typename image_type2
        >
    void extract_image_chip (
        const image_type1& img,
        const chip_details& chip_location,
        image_type2& chip
    );
    /*!
        ensures
            - This function is a simple convenience / compatibility wrapper that calls the
              above-defined extract_image_chip() function using bilinear interpolation.
    !*/

chip_details

    struct chip_details
    {
        /*!
            WHAT THIS OBJECT REPRESENTS
                This object describes where an image chip is to be extracted from within
                another image.  In particular, it specifies that the image chip is
                contained within the rectangle this->rect and that prior to extraction the
                image should be rotated counter-clockwise by this->angle radians.  Finally,
                the extracted chip should have this->rows rows and this->cols columns in it
                regardless of the shape of this->rect.  This means that the extracted chip
                will be stretched to fit via bilinear interpolation when necessary.
        !*/

        chip_details(
        ); 
        /*!
            ensures
                - #rect.is_empty() == true
                - #size() == 0
                - #angle == 0
                - #rows == 0
                - #cols == 0
        !*/

        chip_details(
            const drectangle& rect_
        );
        /*!
            ensures
                - #rect == rect_
                - #size() == rect_.area()
                - #angle == 0
                - #rows == rect_.height()
                - #cols == rect_.width()
        !*/

        chip_details(
            const rectangle& rect_
        );
        /*!
            ensures
                - #rect == rect_
                - #size() == rect_.area()
                - #angle == 0
                - #rows == rect_.height()
                - #cols == rect_.width()
        !*/

        chip_details(
            const drectangle& rect_, 
            unsigned long size_
        );
        /*!
            ensures
                - #rect == rect_
                - #size() == size_
                - #angle == 0
                - #rows and #cols is set such that the total size of the chip is as close
                  to size_ as possible but still matches the aspect ratio of rect_.
                - As long as size_ and the aspect ratio of rect_ stays constant then
                  #rows and #cols will always have the same values.  This means that, for
                  example, if you want all your chips to have the same dimensions then
                  ensure that size_ is always the same and also that rect_ always has the
                  same aspect ratio.  Otherwise the calculated values of #rows and #cols
                  may be different for different chips.  Alternatively, you can use the
                  chip_details constructor below that lets you specify the exact values for
                  rows and cols.
        !*/

        chip_details(
            const drectangle& rect_, 
            unsigned long size_,
            double angle_
        );
        /*!
            ensures
                - #rect == rect_
                - #size() == size_
                - #angle == angle_
                - #rows and #cols is set such that the total size of the chip is as close
                  to size_ as possible but still matches the aspect ratio of rect_.
                - As long as size_ and the aspect ratio of rect_ stays constant then
                  #rows and #cols will always have the same values.  This means that, for
                  example, if you want all your chips to have the same dimensions then
                  ensure that size_ is always the same and also that rect_ always has the
                  same aspect ratio.  Otherwise the calculated values of #rows and #cols
                  may be different for different chips.  Alternatively, you can use the
                  chip_details constructor below that lets you specify the exact values for
                  rows and cols.
        !*/

        chip_details(
            const drectangle& rect_, 
            const chip_dims& dims
        ); 
        /*!
            ensures
                - #rect == rect_
                - #size() == dims.rows*dims.cols 
                - #angle == 0
                - #rows == dims.rows
                - #cols == dims.cols
        !*/

        chip_details(
            const drectangle& rect_, 
            const chip_dims& dims,
            double angle_
        ); 
        /*!
            ensures
                - #rect == rect_
                - #size() == dims.rows*dims.cols 
                - #angle == angle_
                - #rows == dims.rows
                - #cols == dims.cols
        !*/

        template <typename T>
        chip_details(
            const std::vector<dlib::vector<T,2> >& chip_points,
            const std::vector<dlib::vector<T,2> >& img_points,
            const chip_dims& dims
        );
        /*!
            requires
                - chip_points.size() == img_points.size()
                - chip_points.size() >= 2 
            ensures
                - The chip will be extracted such that the pixel locations chip_points[i]
                  in the chip are mapped to img_points[i] in the original image by a
                  similarity transform.  That is, if you know the pixelwize mapping you
                  want between the chip and the original image then you use this function
                  of chip_details constructor to define the mapping.
                - #rows == dims.rows
                - #cols == dims.cols
                - #size() == dims.rows*dims.cols 
                - #rect and #angle are computed based on the given size of the output chip
                  (specified by dims) and the similarity transform between the chip and
                  image (specified by chip_points and img_points).
        !*/

        inline unsigned long size() const { return rows*cols; }
        /*!
            ensures
                - returns the number of pixels in this chip.  This is just rows*cols.
        !*/

        drectangle rect;
        double angle;
        unsigned long rows; 
        unsigned long cols;
    };

注意，一定要会阅读技术文档！！！

3. 初始化dlib中的rectangle类型；

std::vector<rectangle> dets = detector(img);
dlib::rectangle rect_sub(dets[0].left(), dets[0].top(), dets[0].right(), dets[0].bottom());

 4. 图像的尺寸和随机选择数据；

int rows = img.nr();  // array2d<unsigned char> img;
int cols = img.nc();
dlib::rectangle rect1(right+std::rand()%50, std::rand()%top, cols-std::rand()%50, rows-std::rand()%(rows-bottom));
dlib::rectangle rect2(left-std::rand()%20, bottom+std::rand()%20, right+std::rand()%(cols-right), rows-std::rand()%20);
std::vector<dlib::rectangle> rects;
rects.push_back(rect1);
rects.push_back(rect2);
const dlib::chip_details chip_details(rects.at(std::rand()%2));

 

 

参考

1. How to create cv::Mat from dlib::rectangle on dlib's array2d<rgb_pixel> image?

2. Region of Interest Extraction using DLib

3. How to crop the image on opencv and dlib on ubuntu；

4. Dlib与OpenCV图片转换；

5. Convert RGB Image to Grayscale Image in DLIB；

6. dlib C++ Library - interpolation_abstract.h；

7. Question : face_chip from frontal_face_detector · Issue #1057 · davisking/dlib；

完