onnxruntime

ONNX Runtime is a cross-platform inference and training machine-learning accelerator.

ONNX Runtime inference can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries such as scikit-learn, LightGBM, XGBoost, etc. ONNX Runtime is compatible with different hardware, drivers, and operating systems, and provides optimal performance by leveraging hardware accelerators where applicable alongside graph optimizations and transforms. Learn more →

ONNX Runtime training can accelerate the model training time on multi-node NVIDIA GPUs for transformer models with a one-line addition for existing PyTorch training scripts. Learn more →

Get Started & Resources

• General Information: onnxruntime.ai

• Usage documentation and tutorials: onnxruntime.ai/docs

• YouTube video tutorials: youtube.com/@ONNXRuntime

• Upcoming Release Roadmap

• Companion sample repositories:

  • ONNX Runtime Inferencing: microsoft/onnxruntime-inference-examples
  • ONNX Runtime Training: microsoft/onnxruntime-training-examples

• Plugin EP repositories:

  • ONNX Runtime QNN Plugin EP: onnxruntime/onnxruntime-qnn

Releases

The current release and past releases can be found here: https://github.com/microsoft/onnxruntime/releases.

For details on the upcoming release, including release dates, announcements, features, and guidance on submitting feature requests, please visit the release roadmap: https://onnxruntime.ai/roadmap.