Batch Inference with Fine-tuned YOLOv5x6 Model on Custom Data

[Bycqg]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

How can I perform batch inference to speed up image processing with a YOLOv5x6 model that has been fine-tuned on custom data?

Additional

No response

[github-actions]

👋 Hello @Bycqg, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[Bycqg]

I noticed that the logic in detect.py performs inference on images one by one?

If there are millions of images to process, would batch processing be faster? How exactly should I do that? The corresponding model is YOLOv5x6 fine-tuned on custom data.

[Bycqg]

I will refer to this：
#12419
#1806

[glenn-jocher]

@Bycqg indeed, batch processing can significantly speed up inference when dealing with a large number of images. The detect.py script you've mentioned actually supports batch inference through the --batch-size argument. You can adjust this parameter to control the number of images processed simultaneously.

To execute batch inference with your custom-trained YOLOv5x6 model, you can modify the command you use to run detect.py by adding the --batch-size argument followed by your desired batch size. Here's a quick example:

python detect.py --weights your_trained_model.pt --source your_data_directory/ --batch-size 32

In this example, --batch-size 32 tells the script to process 32 images simultaneously. Just ensure that your hardware can support the batch size you select. Larger batch sizes can speed up the processing but require more memory.

I hope this helps! Keep pushing the limits and happy detecting! 😊

[Bycqg]

@glenn-jocher I didn't see --batch-size in the command-line arguments in detect.py. So, in reality, it does support the --batch-size parameter, it's just not explicitly written, right? How is this parameter handled in the code?

[glenn-jocher]

@Bycqg hi there! 😊 Yes, the --batch-size parameter is indeed supported in detect.py, even if it might not be immediately obvious from the command-line arguments documentation. This parameter allows you to specify the number of images processed simultaneously, which can significantly speed up inference times when processing a large volume of images.

The handling of the --batch-size parameter in the script adjusts the DataLoader's batch size, allowing multiple images to be fed through the model in a single pass. This is particularly useful for optimizing the inference process on capable hardware.

For example, to use the batch processing feature, you would run:

python detect.py --weights your_model.pt --source your_data_path/ --batch-size 16

This example sets the batch size to 16, meaning 16 images will be processed at once. Make sure to adjust the batch size based on your system's capabilities to maintain optimal performance. Happy coding!

[Bycqg]

@glenn-jocher Here is another question: I want to call it not through python detect.py --batch-size 16, but specify it in the code. How can I do that?

[glenn-jocher]

@Bycqg Absolutely, you can specify the batch size directly in the code too! 🚀

If you're diving into the detect.py script, you'll find the batch size is set when creating the DataLoader. You can adjust the batch_size parameter in the DataLoader instantiation. For example:

dataloader = DataLoader(dataset, batch_size=16, ... )

Just replace 16 with your desired batch size. This line directly controls how many images are processed together in each step of the inference.

Feel free to tweak this based on your requirements and hardware capabilities. Happy detecting! 😊