Pinned Loading

1. hands-on-kernels hands-on-kernels Public

   High Performance Kernels on GPU/CPU

   Python 3

2. xLLM xLLM Public

   A lightweight llama2 inference framework. It can inference llama2-7b with 166+ tokens/s on signle 4090.

   C++ 3 1

3. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

4. sgl-project/sglang sgl-project/sglang Public

   SGLang is a fast serving framework for large language models and vision language models.

   Python 7k 651