Skip to content

koboldcpp-1.50.1
Compare
Choose a tag to compare
@LostRuins LostRuins released this 18 Nov 08:45
· 4086 commits to concedo since this release
v1.50.1
56a5fa7

koboldcpp-1.50.1

  • Improved automatic GPU layer selection: In the GUI launcher with CuBLAS, it will now automatically select all layers to do a full GPU offload if it thinks you have enough VRAM to support it.
  • Added a short delay to the Abort function in Lite, hopefully fixes the glitches with retry and abort.
  • Fixed automatic RoPE values for Yi and Deepseek. If no --ropeconfig is set, the preconfigured rope values in the model now take priority over the automatic context rope scale.
  • The above fix should also allow YaRN RoPE scaled models to work correctly by default, assuming the model has been correctly converted. Note: Customized YaRN configurations flags are not yet available.
  • The OpenAI compatible /v1/completions has been enhanced, adding extra unofficial parameters that Aphrodite uses, such as Min-P, Top-A and Mirostat. However, OpenAI does not support separate memory fields or sampler order, so the Kobold API will still give better results there.
  • SSE streaming support has been added for OpenAI /v1/completions endpoint (tested working in SillyTavern)
  • Custom DALL-E endpoints are now supported, for use with OAI proxies.
  • Pulled fixed and improvements from upstream, updated Kobold Lite

To use, download and run the koboldcpp.exe, which is a one-file pyinstaller.
If you don't need CUDA, you can use koboldcpp_nocuda.exe which is much smaller.
If you're using AMD, you can try koboldcpp_rocm at YellowRoseCx's fork here

Hotfix 1.50.1:

  • Fixed a regression with older RWKV/GPT-2/GPT-J/GPT-NeoX models that caused a segfault.
  • If ropeconfig is not set, apply auto linear rope scaling multiplier for rope-tuned models such as Yi when used outside their original context limit.
  • Fixed another bug in Lite with the retry/abort button.

Run it from the command line with the desired launch parameters (see --help), or manually select the model in the GUI.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001

For more information, be sure to run the program from command line with the --help flag.

Assets 5
Loading