No Images in train dataset, please ensure

[bound123]

🔎 Search before asking

• I have searched the PaddleOCR Docs and found no similar bug report.
• I have searched the PaddleOCR Issues and found no similar bug report.
• I have searched the PaddleOCR Discussions and found no similar bug report.

🐛 Bug (问题描述)

ppocr ERROR: No Images in train dataset, please ensure
1. The images num in the train label_file_list should be larger than or equal with batch size.
2. The annotation file and path in the configuration file are provided normally.

🏃‍♂️ Environment (运行环境)

PYTHON 3.10

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

配置文件如图

Global:
  use_gpu: false
  epoch_num: 500
  log_smooth_window: 20
  print_batch_step: 10
  save_model_dir: ./output/rec_chinese_common_v2.0
  save_epoch_step: 3
  # evaluation is run every 5000 iterations after the 4000th iteration
  eval_batch_step: [0, 2000]
  cal_metric_during_train: True
  pretrained_model: D:\ancoda\anocada3\envs\paddle\Lib\site-packages\Preliminary_training\ch_ppocr_server_v2.0_rec_train\best_accuracy.pdparams
  checkpoints:
  save_inference_dir:
  use_visualdl: False
  infer_img: doc/imgs_words/ch/word_1.jpg
  # for data or label process
  character_dict_path: ppocr/utils/ppocr_keys_v1.txt
  max_text_length: 25
  infer_mode: False
  use_space_char: True
  save_res_path: ./output/rec/predicts_chinese_common_v2.0.txt


Optimizer:
  name: Adam
  beta1: 0.9
  beta2: 0.999
  lr:
    name: Cosine
    learning_rate: 0.001
    warmup_epoch: 5
  regularizer:
    name: 'L2'
    factor: 0.00004

Architecture:
  model_type: rec
  algorithm: CRNN
  Transform:
  Backbone:
    name: ResNet
    layers: 34
  Neck:
    name: SequenceEncoder
    encoder_type: rnn
    hidden_size: 256
  Head:
    name: CTCHead
    fc_decay: 0.00004

Loss:
  name: CTCLoss

PostProcess:
  name: CTCLabelDecode

Metric:
  name: RecMetric
  main_indicator: acc

Train:
  dataset:
    name: SimpleDataSet
    data_dir: D:\ancoda\anocada3\envs\paddle\Lib\site-packages\train_data\rec\train
    label_file_list: D:\ancoda\anocada3\envs\paddle\Lib\site-packages\train_data\rec\train.txt
    transforms:
      - DecodeImage: # load image
          img_mode: BGR
          channel_first: False
      - RecAug: 
      - CTCLabelEncode: # Class handling label
      - RecResizeImg:
          image_shape: [3, 32, 320]
      - KeepKeys:
          keep_keys: ['image', 'label', 'length'] # dataloader will return list in this order
  loader:
    shuffle: True
    batch_size_per_card: 256
    drop_last: True
    num_workers: 8

Eval:
  dataset:
    name: SimpleDataSet
    data_dir: D:\ancoda\anocada3\envs\paddle\Lib\site-packages\train_data\rec\val
    label_file_list: D:\ancoda\anocada3\envs\paddle\Lib\site-packages\train_data\rec\val.txt
    transforms:
      - DecodeImage: # load image
          img_mode: BGR
          channel_first: False
      - CTCLabelEncode: # Class handling label
      - RecResizeImg:
          image_shape: [3, 32, 320]
      - KeepKeys:
          keep_keys: ['image', 'label', 'length'] # dataloader will return list in this order
  loader:
    shuffle: False
    drop_last: False
    batch_size_per_card: 256
    num_workers: 8

[GreatV]

Duplicate of #14391

[GreatV]

The error message you are encountering, "No Images in train dataset, please ensure", typically indicates that the training dataset is not being loaded correctly. Below are the possible reasons and steps to resolve the issue:

---

🛠️ Possible Causes and Fixes:

1. Mismatch Between label_file_list and data_dir:

   • Ensure that the paths provided in Train.dataset.data_dir and Train.dataset.label_file_list are correct and correspond to the actual files.
   • Specifically:
     • Train.dataset.data_dir should point to the directory containing your training images.
     • Train.dataset.label_file_list should point to a .txt file with annotations.
   • Example of a valid annotation file (train.txt):

     image1.jpg label1
     image2.jpg label2
     

     Make sure the image filenames in train.txt exist in the directory specified by data_dir.

   Fix:

   • Verify that the paths provided exist and are accessible.
   • Check for typos or missing files.

   Example Update:

   Train:
     dataset:
       data_dir: /path/to/images
       label_file_list: /path/to/annotations/train.txt

---

2. Empty or Incorrect Annotation File (train.txt):

   • If the annotation file referenced in label_file_list is empty or malformed, the dataset loader will not find any valid entries.
   • Ensure that:
     • The file is not empty.
     • Each line corresponds to an image and its label in the correct format.

   Fix:

   • Open the annotation file and confirm it contains valid entries.
   • Example of a valid entry:

     D:/ancoda/anocada3/envs/paddle/Lib/site-packages/train_data/rec/train/image1.jpg label1
     

---

3. Batch Size Larger Than the Dataset:

   • The error mentions: "The images num in the train label_file_list should be larger than or equal with batch size."
   • If your dataset contains fewer images than the specified batch size (batch_size_per_card), this error will appear.

   Fix:

   • Reduce the batch size:

     Train:
       loader:
         batch_size_per_card: 16  # Adjust to match your dataset size

---

4. Incorrect Dataset Structure or File Permissions:

   • Confirm that the dataset directory structure is correct and accessible.
   • Ensure the images are in the format expected by PaddleOCR (e.g., .jpg, .png).

   Fix:

   • Run the following checklist:
     • Verify the dataset directory contains images.
     • Ensure the images have proper read permissions.

---

5. Transformations or Preprocessing Issues:

   • The transformations specified in the transforms section could cause issues if the images or labels are not compatible.

   Fix:

   • Ensure the transformations in your config file are appropriate for your dataset (e.g., image size, label encoding).
   • Example:

     transforms:
       - DecodeImage: # load image
           img_mode: BGR
           channel_first: False
       - RecResizeImg:
           image_shape: [3, 32, 320]

---

6. Debugging the Dataset Loader:
   • Add a debugging step to check if the dataset loader works correctly.
   • Modify your script to print the dataset size:

     from paddleocr.ppocr.data import build_dataloader
     
     dataloader = build_dataloader(configs['Train'])
     print(f"Number of training samples: {len(dataloader.dataset)}")

---

✅ Summary of Steps to Fix:

1. Verify that:
   • Train.dataset.data_dir points to the correct image directory.
   • Train.dataset.label_file_list points to a valid annotation file.
2. Ensure the annotation file is populated and correctly formatted.
3. Confirm that the number of images in the dataset is greater than or equal to the batch size.
4. Check for file permissions, dataset structure, and transformations.
5. Reduce batch_size_per_card if your dataset is small.

---

If the above steps do not resolve the issue, please provide:

• The contents of your annotation file (train.txt).
• A sample directory structure of your training dataset.

This will help diagnose the problem further.

---

Response generated by feifei-bot | chatgpt-4o-latest

[SWHL]

Dulicate with #14393