How to compile with BLAS on Apple Silicon

[halbtuerke]

Hi,

I'm trying to compile koboldcpp with BLAS on my M2 Max, I tried OPENBLAS and CLBLAST, both installed through homebrew, but I always get the following messages:

ld: library not found for -lopenblas
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [koboldcpp_openblas] Error 1
make LLAMA_METAL=1 LLAMA_OPENBLAS=1

ld: library not found for -lclblast
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [koboldcpp_clblast] Error 1
make LLAMA_METAL=1 LLAMA_CLBLAST=1

How can I specify where the respective libraries can be found?

Thanks in advance and sorry for the noob question. 😅

[LostRuins]

You will have much better performance on a mac if you just use the already-supported Accelerate module, which is faster than OpenBLAS on mac.

So you don't need OpenBLAS at all. If you wanna use Metal, just try with make LLAMA_METAL=1 first. Otherwise, a simple make would use Accelerate.

[halbtuerke]

Thank you very much for the clarification. I thought it might help with the initial prompt ingestion, as it takes a significant amount of time for the model to generate tokens when handling large prompts, such as text summarization.

Do you happen to know a way to speed up the prompt ingestion phase?

[LostRuins]

You can try using --smartcontext which should reduce the number of times the prompt needs to be re-processed

[halbtuerke]

Thank you, I will try that.