AssertionError: input size torch.Size([1, 3, 640, 480]) not equal to max model size (1, 3, 640, 640)

[KnightInsight]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

Hi, I trained a yolov5s model and exported to tensorrt engine file by typing "python export.py --weights /yolov5s.pt --include engine --device 0". The engine model is 640x640. I tried to load multiple images with different image size(not 640x640) using detect.py script. It's worked even though input size is not 640x640. However, I tested on "model = torch.hub.load('.', 'custom', 'yolov5s.engine', source='local')" then AssertionError: input size torch.Size([1, 3, 640, 480]) not equal to max model size (1, 3, 640, 640).

Additional

No response

[github-actions]

👋 Hello @KnightInsight, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[glenn-jocher]

@KnightInsight this issue seems to occur when loading the exported TensorRT engine model using torch.hub.load(). The error message states that the input size torch.Size([1, 3, 640, 480]) is not equal to the max model size (1, 3, 640, 640).

To address this issue, you can try resizing your input images to have a shape of (640, 640) before passing them to the torch.hub.load() function. This will ensure that the image dimensions match the expected input size of the model.

If you still encounter any problems or have further questions, please feel free to ask. The YOLOv5 community and the Ultralytics team are here to assist you.

[KnightInsight]

When resizing an image, it can change the original size, possibly distorting the aspect ratio and making it challenging to detect objects accurately. However, in the detect.py script, the image is temporarily resized for processing, but the output image maintains the original size. Can torch.hub.load() be used in a same way in detect.py to achieve this behavior?

[glenn-jocher]

@KnightInsight yes, you can achieve a similar behavior as the detect.py script when using torch.hub.load() by resizing the input image while keeping the aspect ratio intact. This can be done using the torchvision transforms.Resize() function.

Here's an example of how you can modify your code to achieve this:

import torch
import torchvision.transforms as transforms

# Load the model
model = torch.hub.load('.', 'custom', 'yolov5s.engine', source='local')

# Resize the input image while maintaining aspect ratio
image_path = 'path/to/your/image.jpg'
image = Image.open(image_path)
image_transform = transforms.Resize((640, 640))
resized_image = image_transform(image)

# Perform object detection on the resized image
results = model([resized_image])

# Resize the output image back to the original size
output_image = transforms.Resize(image.size)(results.imgs[0])

# Continue with the rest of your code

By following this approach, you can resize the input image while preserving the aspect ratio, perform object detection on the resized image, and finally resize the output image back to the original size.

Hope this helps! If you have any additional questions or concerns, please let me know.

[KnightInsight]

However, the bounding box coordinates are different since it used the resized_image. How can I get the bounding box coordinates of original_image? I would like to print(results.xyxy[0]).

[glenn-jocher]

@KnightInsight,

To get the bounding box coordinates of the original image, you can use the tensor2img function provided by YOLOv5. This function converts the tensor image back to a PIL image. Here's an example:

from PIL import Image
from utils.plots import colors
from IPython.display import display

# Convert the output tensor image back to a PIL image
output_pil = Image.fromarray(output_image.mul(255).byte().numpy())

# Plot the bounding box coordinates on the original image
boxes, labels, scores = results.xyxy[0][:, :4], results.xyxy[0][:, 5], results.xyxy[0][:, 4]
for box, label, score in zip(boxes, labels, scores):
    box = box.tolist()
    color = colors(int(label))
    output_pil.rectangle(xy=box, outline=color)
    text = f"{model.names[int(label)]}: {score:.2f}"
    output_pil.text(xy=(box[0], box[1]), text=text, fill=color)

# Display the image
display(output_pil)

This code snippet will plot the bounding box coordinates on the original image and display it. Please note that the utils.plots module provides the colors array, which assigns different colors to different object classes for visualization purposes. Make sure to import it in your code.

Let me know if you have any further questions or need any additional assistance.

Glenn Jocher

[KnightInsight]

@glenn-jocher ,

Hi, I have tested on the below code that you provided. However, I realized the accuracy of the detection getting lower during torch.hub.load. I have tested on detect.py but the accuracy maintained good. May I know what is the difference? Is detect.py not using transforms.Resize method? Besides, the inference time on detect.py(5ms) but torch.hub.load takes more than 50ms (not consistent too, can be 50ms,70ms,or 120ms), the inference range is large, hardware should not be the main issue to cause the spike, why?

image_transform = transforms.Resize((640, 640))
resized_image = image_transform(image)

Perform object detection on the resized image

results = model([resized_image])

Thank you!!!

[glenn-jocher]

@KnightInsight,

When using the torch.hub.load() method, there are a few factors that can lead to differences in model accuracy and inference time compared to running the detect.py script.

1. Data Preprocessing: It's possible that the detect.py script applies additional preprocessing or augmentation techniques to the input images before passing them through the model. These techniques can have an impact on accuracy, especially if the model was trained with similar preprocessing steps.

2. Batch Size: The detect.py script might perform inference on multiple images in a batch, which can lead to more efficient processing and possibly lower inference times compared to processing each image individually with torch.hub.load().

3. Hardware Acceleration: When using torch.hub.load(), the initial loading and optimization of the model for inference might introduce variability in inference time, especially if it is not consistently cached in memory across runs.

To get consistent performance and accuracy with torch.hub.load(), you might want to consider pre-processing your input images to match the expected model input size and experimenting with different batch sizes during inference.

I hope this helps in addressing the performance differences you are observing. If you have further questions or concerns, feel free to ask.

Glenn Jocher

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

[KXHH2021]

May I ask, does YOLOv8n-seg.engine use the above method as well?

[glenn-jocher]

@KXHH2021 no, YOLOv8 uses the ultralytics Python package for inference. You can see examples at https://github.com/ultralytics/ultralytics

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐