QAT, latency stays the same

[maiiabocharova]

Trained a model according to documentation.
Added those configs lines for QAT

    cfg.QUANTIZATION.QAT.BATCH_SIZE_FACTOR = 1.0
    cfg.QUANTIZATION.BACKEND = "fbgemm"
    cfg.QUANTIZATION.QAT.FAKE_QUANT_METHOD = "default"
    cfg.QUANTIZATION.QAT.START_ITER = 1200
    cfg.QUANTIZATION.QAT.ENABLE_OBSERVER_ITER = 1200
    cfg.QUANTIZATION.QAT.ENABLE_LEARNABLE_OBSERVER_ITER = 1300
    cfg.QUANTIZATION.QAT.DISABLE_OBSERVER_ITER = 1200 + 300
    cfg.QUANTIZATION.QAT.FREEZE_BN_ITER = 12000 + 200

Trained for 1500 iterations.

The model is Ok and predictions are correct. But the model size and time of execution stayed exactly the same. Can you please help and advice on how to fix it?

[wat3rBro]

Are you running the exported torchscript moddel? the speed up and model size reduction is for running torchscript model on device.