New CUDA kv cache quantization support merged in llama.cpp yesterday

[rosemash]

ggerganov#7527
Apologies if you are already aware of this (you most likely are) but this PR allows the KV cache to be quantized down to as small as q4_0 when loading a model, which dramatically reduces the amount of vram needed to offload layers in large models. I don't think there is a flag in koboldcpp to use this feature. It would really help to load large models on small GPUs.