Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 50.2k 8.2k

2. vllm-project/llm-compressor vllm-project/llm-compressor Public

   Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

   Python 1.5k 151

3. neuralmagic/deepsparse neuralmagic/deepsparse Public archive

   Sparsity-aware deep learning inference runtime for CPUs

   Python 3.2k 188

4. advos advos Public

   RISC-V OS in Rust with hardware support for SiFive's HiFive1 board

   Rust

5. torch_bitmask torch_bitmask Public

   Implementations for fast bitmask compression for weight sparsity in PyTorch

   Python 4 1