RuntimeError: shape '[51, 784]' is invalid for input of size 626688

[DannyFGitHub]

What's the likely cause for a "RuntimeError: shape '[51, 784]' is invalid for input of size 626688"
Seems to be happening on the forward pass of the model.

from avalanche.benchmarks.generators import ni_benchmark
from torchvision.transforms import Compose, ToTensor, Normalize, RandomCrop, RandomHorizontalFlip, Resize
from torchvision.datasets import ImageFolder
from torch.optim import SGD
from torch.nn import CrossEntropyLoss
from avalanche.models import SimpleMLP
from avalanche.training.strategies import Naive, CWRStar, Replay, GDumb, Cumulative, LwF, GEM, AGEM, EWC

import plot

def main():
    IMAGE_SIZE = 64
    BATCH_SIZE = 3
    EPOCHS = 1
    TRAIN_DIR = "../MyDataset-Minimal/train"
    TEST_DIR = "../MyDataset-Minimal/test"
    # TRAIN_DIR = "../MyDataset/train"
    # TEST_DIR = "../MyDataset/test"

    # Define transformations for training
    train_transform = Compose([
        # transforms.RandomCrop(32, padding=4),
        Resize([IMAGE_SIZE, IMAGE_SIZE]),
        RandomHorizontalFlip(),
        ToTensor(),  # Turn PIL Image to torch.Tensor
        Normalize(mean=(0.5, 0.5, 0.5), std=(0.5, 0.5, 0.5))
    ])

    # Define transformations for evaluation
    test_transform = Compose([
        Resize([IMAGE_SIZE, IMAGE_SIZE]),
        ToTensor(),
        Normalize(mean=(0.5, 0.5, 0.5), std=(0.5, 0.5, 0.5))
    ])

    train_dataset = ImageFolder(TRAIN_DIR, transform=train_transform)
    test_dataset = ImageFolder(TEST_DIR, transform=test_transform)

    num_of_classes = len(train_dataset.classes)

    scenario = ni_benchmark(
        train_dataset, test_dataset, n_experiences=1, shuffle=True, seed=1234,
        balance_experiences=True
    )

    model = SimpleMLP(num_classes=num_of_classes)
    optimizer = SGD(model.parameters(), lr=0.001, momentum=0.9)
    criterion = CrossEntropyLoss()
    cl_strategy = Naive(
        model, optimizer, criterion,
        train_mb_size=100, train_epochs=EPOCHS, eval_mb_size=100
    )

    # TRAINING LOOP
    print('Starting experiment...')
    results = []
    for experience in scenario.train_stream:
        print("Start of experience: ", experience.current_experience)
        print("Current Classes: ", experience.classes_in_this_experience)

        cl_strategy.train(experience)
        print('Training completed')

        print('Computing accuracy on the whole test set')
        results.append(cl_strategy.eval(scenario.test_stream))
    pass


if __name__ == '__main__':
    main()

[DannyFGitHub]

[AndreaCossu]

Note that the default input size of SimpleMLP is 784, you may want to change it to match the input size of your images.

[DannyFGitHub]

Changed input IMAGE_SIZE to 784 and Error was: "RuntimeError: shape '[51, 784]' is invalid for input of size 94042368"
Changed input IMAGE_SIZE to 28 (28 x 28 = 784) and Error was: "RuntimeError: shape '[51, 784]' is invalid for input of size 119952"

[DannyFGitHub]

Using SimpleCNN does not give me the same error and goes onto training as normal.

[vlomonaco]

HI @DannyFGitHub! I think this is related to the specific data formats of your data. You may try to isolate the problem testing only the forward pass and debugging it step by step!

[vlomonaco]

Hi @DannyFGitHub! I'll mark this issue as solved. Feel free to re-open the thread if want to discuss this further!

[SinHanYang]

I got the same issue. I tried to change the input size of SimpleMLP(input_size=my_input_size). But it still showed the same error. I am using from avalanche.benchmarks.datasets import CIFAR10 dataset.