How to let the script only save the best checkpoint?

[JiyangZhang]

Hi,

Thanks for developing such a nice tool for all researchers. I am new to this tool, so correct me if there is something wrong.

I assume that if keep_checkpoint is set to 1, the script will only save one checkpoint which is the best one during validation. But it seems that it only saves the latest checkpoint. Is there option to let the script only save the best checkpoint? Or I misunderstand something here?

Thank you.

[francoishernandez]

-keep_checkpoint indeed allows to keep the last N checkpoints, hence from a 'chronological' standpoint:

OpenNMT-py/onmt/opts.py

Lines 393 to 394 in 60125c8

	group.add('--keep_checkpoint', '-keep_checkpoint', type=int, default=-1,
	help="Keep X checkpoints (negative: keep all)")

If you'd like to implement something smarter, metric-based, PR welcome.
Most of the model saving logic is here:
https://github.com/OpenNMT/OpenNMT-py/blob/master/onmt/models/model_saver.py

[JiyangZhang]

Thanks for reply.
I would suggest changing from "Keep X checkpoints" to "Keep last X checkpoints" to be more clear.
Actually I have no idea why we can early stop the training but can not save the checkpoint at the "best" time.
I am happy to submit PR when I have the chance to implement something new.

[francoishernandez]

Actually I have no idea why we can early stop the training but can not save the checkpoint at the "best" time.

Good point. The early stopping mechanism should probably override this keep_checkpoint parameter.

[vince62s]

follow up on this in #1946