[FIXED] How to feed multiple input into opencv dnn

Issue

I have a .pb model file. And I loaded the model using opencv’s readnetfromtensorflow(). Now I want to use the model to generate predictions. There are 4 types of model input.

Input data

  1. 256×256 image
  2. 64×64 image
  3. 64×64 image
  4. array (size is 4)

output data

  1. array (size is 2)

To generate the model’s predictions, I first had to transform the input into blobFromImages. However, I couldn’t use it because each image that needs to be converted has a different size. I also tried inserting each image into a vector in setInput() but it failed. What should I do when there are multiple inputs in this situation?

Here is the code I’ve tried.

cv::dnn::Net net = cv::dnn::readNetFromTensorflow("model/mfg.pb”);
cv::Mat input_face = cv::dnn::blobFromImage(face, 1, cv::Size(256,256), cv::Scalar(104,177,123), true, false);
cv::Mat input_leye = cv::dnn::blobFromImage(leye, 1, cv::Size(64,64), cv::Scalar(104,177,123), true, false);
cv::Mat input_reye = cv::dnn::blobFromImage(reye, 1, cv::Size(64,64), cv::Scalar(104,177,123), true, false);
cv::Mat input_bbox = (cv::Mat1d(1,4) << face_bbox[0]., face_bbox[1]., face_bbox[2]., face_bbox[3].);

std::vector<cv::Mat> input_image = {input_face, input_leye, input_reye, input_bbox};
net.setInput(input_image);
net.forward();

However, it failed and an error message appeared.

Error
libc++abi: terminating with uncaught exception of type cv::Exception: OpenCV(4.5.5) /tmp/opencv-20220714-27380-1eyun69/opencv-4.5.5/modules/core/src/matrix_wrap.cpp:81: error: (-215:Assertion failed) 0 <= i && i < (int)v.size() in function ‘getMat_’

Solution

You can’t set multiple inputs on a cv::dnn::Net.(It’s not supported.)

I suggest two alternatives.

  1. You do that DNN calculations with Tensorflow and other image/video tasks with OpenCV.
    You can wrap the data of a tensorflow::Tensor as a cv::Mat. (See Guillaume’s answer in this question.)

  2. Redesign the functional model into a sequential model.

Answered By – relent95

Answer Checked By – David Marino (Easybugfix Volunteer)

Leave a Reply

(*) Required, Your email will not be published