llm-inference

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Find secrets with Gitleaks 🔑

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Open-source implementation of AlphaEvolve

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs