GPU Inference

Open-source projects tagged GPU Inference, ranked by transparent GitHub momentum.

3 approved projects

Projects

Project	Category	Stars	7-day	Momentum	Language	Active
llama.cpp ggml-org/llama.cpp LLM inference in C/C++	Model InferenceIndependent	128.1K	+842	83.8	C++	3 hours ago
vLLM vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs	Model ServingInstitutional	91.7K	+551	72.5	Python	3 hours ago
Transformers huggingface/transformers 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.	Model InferenceStartup	165.6K	+686	49.6	Python	12 hours ago

llama.cpp
ggml-org/llama.cpp
83.8
LLM inference in C/C++
Model InferenceIndependent★ 128.1K+842 / 7dC++3 hours ago
vLLM
vllm-project/vllm
72.5
A high-throughput and memory-efficient inference and serving engine for LLMs
Model ServingInstitutional★ 91.7K+551 / 7dPython3 hours ago
Transformers
huggingface/transformers
49.6
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Model InferenceStartup★ 165.6K+686 / 7dPython12 hours ago