PageSourceSearch

https://onnxruntime.ai/docs/

html onnxruntime.ai collected 2026-10-01 08:35:53 UTC 41,930 bytes, 1 lines download raw bytes

1<!DOCTYPE html> <html lang="en-US"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=Edge"> <link rel="stylesheet" href="/assets/css/just-the-docs-default.css"> <link rel="stylesheet" href="/assets/css/just-the-docs-head-nav.css" id="jtd-head-nav-stylesheet"> <style id="jtd-nav-activation"> .site-nav > ul.nav-list:first-child > li:not(:nth-child(1)) > a, .site-nav > ul.nav-list:first-child > li > ul > li a { background-image: none; } .site-nav > ul.nav-list:not(:first-child) a, .site-nav li.external a { background-image: none; } .site-nav > ul.nav-list:first-child > li:nth-child(1) > a { font-weight: 600; text-decoration: none; }.site-nav > ul.nav-list:first-child > li:nth-child(1) > button svg { transform: rotate(-90deg); }.site-nav > ul.nav-list:first-child > li.nav-list-item:nth-child(1) > ul.nav-list { display: block; } </style>
vendor: 64 bytes, line 1
1 <script async src="https://www.googletagmanager.com/gtag/js?id=
1UA-156955408-1
vendor: 12 bytes, line 1
1"></script> 
1<script> 
vendor: 127 bytes, line 1
1window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', '
1UA-156955408-1
vendor: 4 bytes, line 1
1'); 
1</script>
1 
1<script src="/assets/js/vendor/lunr.min.js"></script>
1 
1<script src="/assets/js/just-the-docs.js"></script>
1 <meta name="viewport" content="width=device-width, initial-scale=1"> <link rel="icon" href="/favicon.ico" type="image/x-icon"> <!-- Begin Jekyll SEO tag v2.8.0 --> <title>ONNX Runtime | onnxruntime</title> <meta name="generator" content="Jekyll v3.10.0" /> <meta property="og:title" content="ONNX Runtime" /> <meta property="og:locale" content="en_US" /> <meta name="description" content="ONNX Runtime is a cross-platform machine-learning model accelerator" /> <meta property="og:description" content="ONNX Runtime is a cross-platform machine-learning model accelerator" /> <link rel="canonical" href="https://onnxruntime.ai/docs/" /> <meta property="og:url" content="https://onnxruntime.ai/docs/" /> <meta property="og:site_name" content="onnxruntime" /> <meta property="og:type" content="website" /> <meta name="twitter:card" content="summary" /> <meta property="twitter:title" content="ONNX Runtime" /> 
1<script type="application/ld+json"> {"@context":"https://schema.org","@type":"WebPage","description":"ONNX Runtime is a cross-platform machine-learning model accelerator","headline":"ONNX Runtime","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://onnxruntime.ai/images/ONNX-Runtime-logo.svg"}},"url":"https://onnxruntime.ai/docs/"}</script>
1 <!-- End Jekyll SEO tag --> <meta property="og:image" content="/images/logos/onnxruntime/ORT_icon_for_light_bg.png" /> 
1<script type="text/javascript" src="https://js.monitor.azure.com/scripts/c/ms.analytics-web-3.min.js" ></script>
1 
1<script type="text/javascript"> const config = { instrumentationKey: '360b0e675e0044398fd28c8bdf711b8e-1fe5434d-ee99-4837-99cc-a3a16462d82d-7262', channelConfiguration: { eventsLimitInMem: 50 }, propertyConfiguration: { env: 'PROD' }, webAnalyticsConfiguration: { autoCapture: { scroll: true, pageView: true, onLoad: true, onUnload: true, click: true, resize: true, jsError: true } } }; const analytics = new oneDS.ApplicationInsights(); analytics.initialize(config, []); </script>
1 </head> <body> <a class="skip-to-main" href="#main-content">Skip to main content</a> <svg xmlns="http://www.w3.org/2000/svg" class="d-none"> <symbol id="svg-link" viewBox="0 0 24 24"> <title>Link</title> <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-link"> <path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path> </svg> </symbol> <symbol id="svg-menu" viewBox="0 0 24 24"> <title>Menu</title> <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-menu"> <line x1="3" y1="12" x2="21" y2="12"></line><line x1="3" y1="6" x2="21" y2="6"></line><line x1="3" y1="18" x2="21" y2="18"></line> </svg> </symbol> <symbol id="svg-arrow-right" viewBox="0 0 24 24"> <title>Expand</title> <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-chevron-right"> <polyline points="9 18 15 12 9 6"></polyline> </svg> </symbol> <!-- Feather. MIT License: https://github.com/feathericons/feather/blob/master/LICENSE --> <symbol id="svg-external-link" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-external-link"> <title id="svg-external-link-title">(external link)</title> <path d="M18 13v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h6"></path><polyline points="15 3 21 3 21 9"></polyline><line x1="10" y1="14" x2="21" y2="3"></line> </symbol> <symbol id="svg-doc" viewBox="0 0 24 24"> <title>Document</title> <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-file"> <path d="M13 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V9z"></path><polyline points="13 2 13 9 20 9"></polyline> </svg> </symbol> <symbol id="svg-search" viewBox="0 0 24 24"> <title>Search</title> <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-search"> <circle cx="11" cy="11" r="8"></circle><line x1="21" y1="21" x2="16.65" y2="16.65"></line> </svg> </symbol> <!-- Bootstrap Icons. MIT License: https://github.com/twbs/icons/blob/main/LICENSE.md --> <symbol id="svg-copy" viewBox="0 0 16 16"> <title>Copy</title> <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-clipboard" viewBox="0 0 16 16"> <path d="M4 1.5H3a2 2 0 0 0-2 2V14a2 2 0 0 0 2 2h10a2 2 0 0 0 2-2V3.5a2 2 0 0 0-2-2h-1v1h1a1 1 0 0 1 1 1V14a1 1 0 0 1-1 1H3a1 1 0 0 1-1-1V3.5a1 1 0 0 1 1-1h1v-1z"/> <path d="M9.5 1a.5.5 0 0 1 .5.5v1a.5.5 0 0 1-.5.5h-3a.5.5 0 0 1-.5-.5v-1a.5.5 0 0 1 .5-.5h3zm-3-1A1.5 1.5 0 0 0 5 1.5v1A1.5 1.5 0 0 0 6.5 4h3A1.5 1.5 0 0 0 11 2.5v-1A1.5 1.5 0 0 0 9.5 0h-3z"/> </svg> </symbol> <symbol id="svg-copied" viewBox="0 0 16 16"> <title>Copied</title> <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-clipboard-check-fill" viewBox="0 0 16 16"> <path d="M6.5 0A1.5 1.5 0 0 0 5 1.5v1A1.5 1.5 0 0 0 6.5 4h3A1.5 1.5 0 0 0 11 2.5v-1A1.5 1.5 0 0 0 9.5 0h-3Zm3 1a.5.5 0 0 1 .5.5v1a.5.5 0 0 1-.5.5h-3a.5.5 0 0 1-.5-.5v-1a.5.5 0 0 1 .5-.5h3Z"/> <path d="M4 1.5H3a2 2 0 0 0-2 2V14a2 2 0 0 0 2 2h10a2 2 0 0 0 2-2V3.5a2 2 0 0 0-2-2h-1v1A2.5 2.5 0 0 1 9.5 5h-3A2.5 2.5 0 0 1 4 2.5v-1Zm6.854 7.354-3 3a.5.5 0 0 1-.708 0l-1.5-1.5a.5.5 0 0 1 .708-.708L7.5 10.793l2.646-2.647a.5.5 0 0 1 .708.708Z"/> </svg> </symbol> </svg> <header class="side-bar"> <div class="site-header"> <a href="/" class="site-title lh-tight"> <div class="site-logo" role="img" aria-label="onnxruntime"></div> </a> <button id="menu-button" class="site-button btn-reset" aria-label="Menu" aria-expanded="false"> <svg viewBox="0 0 24 24" class="icon" aria-hidden="true"><use xlink:href="#svg-menu"></use></svg> </button> </div> <nav aria-label="Main" id="site-nav" class="site-nav"> <ul class="nav-list"><li class="nav-list-item"><a href="/docs/" class="nav-list-link">
1ONNX Runtime</a></li><li class="nav-list-item"><a href="/docs/install/" class="nav-list-link">Install ONNX Runtime</a></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Get Started submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/get-started/" class="nav-list-link">Get Started</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/get-started/with-python.html" class="nav-list-link">Python</a></li><li class="nav-list-item"><a href="/docs/get-started/with-cpp.html" class="nav-list-link">C++</a></li><li class="nav-list-item"><a href="/docs/get-started/with-c.html" class="nav-list-link">C</a></li><li class="nav-list-item"><a href="/docs/get-started/with-csharp.html" class="nav-list-link">C#</a></li><li class="nav-list-item"><a href="/docs/get-started/with-java.html" class="nav-list-link">Java</a></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="JavaScript submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/get-started/with-javascript/" class="nav-list-link">JavaScript</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/get-started/with-javascript/web.html" class="nav-list-link">Web</a></li><li class="nav-list-item"><a href="/docs/get-started/with-javascript/node.html" class="nav-list-link">Node.js binding</a></li><li class="nav-list-item"><a href="/docs/get-started/with-javascript/react-native.html" class="nav-list-link">React Native</a></li></ul></li><li class="nav-list-item"><a href="/docs/get-started/with-obj-c.html" class="nav-list-link">Objective-C</a></li><li class="nav-list-item"><a href="/docs/get-started/community-projects.html" class="nav-list-link">Julia, Ruby and Rust APIs</a></li><li class="nav-list-item"><a href="/docs/get-started/with-windows.html" class="nav-list-link">Windows</a></li><li class="nav-list-item"><a href="/docs/get-started/with-mobile.html" class="nav-list-link">Mobile</a></li><li class="nav-list-item"><a href="/docs/get-started/training-on-device.html" class="nav-list-link">On-Device Training</a></li><li class="nav-list-item"><a href="/docs/get-started/training-pytorch.html" class="nav-list-link">Large Model Training</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Tutorials submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/tutorials/" class="nav-list-link">Tutorials</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/tutorials/api-basics.html" class="nav-list-link">API Basics</a></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Accelerate PyTorch submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/tutorials/accelerate-pytorch/" class="nav-list-link">Accelerate PyTorch</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/tutorials/accelerate-pytorch/pytorch.html" class="nav-list-link">PyTorch Inference</a></li><li class="nav-list-item"><a href="/docs/tutorials/accelerate-pytorch/resnet-inferencing.html" class="nav-list-link">Inference on multiple targets</a></li><li class="nav-list-item"><a href="/docs/tutorials/accelerate-pytorch/ort-training.html" class="nav-list-link">Accelerate PyTorch Training</a></li></ul></li><li class="nav-list-item"><a href="/docs/tutorials/tensorflow.html" class="nav-list-link">Accelerate TensorFlow</a></li><li class="nav-list-item"><a href="/docs/tutorials/huggingface.html" class="nav-list-link">Accelerate Hugging Face</a></li><li class="nav-list-item"><a href="/docs/tutorials/azureml.html" class="nav-list-link">Deploy on AzureML</a></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Deploy on mobile submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/tutorials/mobile/" class="nav-list-link">Deploy on mobile</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/tutorials/mobile/pose-detection.html" class="nav-list-link">Object detection and pose estimation with YOLOv8</a></li><li class="nav-list-item"><a href="/docs/tutorials/mobile/deploy-android.html" class="nav-list-link">Mobile image recognition on Android</a></li><li class="nav-list-item"><a href="/docs/tutorials/mobile/superres.html" class="nav-list-link">Improve image resolution on mobile</a></li><li class="nav-list-item"><a href="/docs/tutorials/mobile/deploy-ios.html" class="nav-list-link">Mobile objection detection on iOS</a></li><li class="nav-list-item"><a href="/docs/tutorials/mobile/helpers/" class="nav-list-link">ORT Mobile Model Export Helpers</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Web submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/tutorials/web/" class="nav-list-link">Web</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/tutorials/web/build-web-app.html" class="nav-list-link">
1Build a web app with ONNX Runtime</a></li><li class="nav-list-item"><a href="/docs/tutorials/web/env-flags-and-session-options.html" class="nav-list-link">The 'env' Flags and Session Options</a></li><li class="nav-list-item"><a href="/docs/tutorials/web/ep-webgpu.html" class="nav-list-link">Using WebGPU</a></li><li class="nav-list-item"><a href="/docs/tutorials/web/ep-webnn.html" class="nav-list-link">Using WebNN</a></li><li class="nav-list-item"><a href="/docs/tutorials/web/large-models.html" class="nav-list-link">Working with Large Models</a></li><li class="nav-list-item"><a href="/docs/tutorials/web/performance-diagnosis.html" class="nav-list-link">Performance Diagnosis</a></li><li class="nav-list-item"><a href="/docs/tutorials/web/deploy.html" class="nav-list-link">Deploying ONNX Runtime Web</a></li><li class="nav-list-item"><a href="/docs/tutorials/web/trouble-shooting.html" class="nav-list-link">Troubleshooting</a></li><li class="nav-list-item"><a href="/docs/tutorials/web/classify-images-nextjs-github-template.html" class="nav-list-link">Classify images with ONNX Runtime and Next.js</a></li><li class="nav-list-item"><a href="/docs/tutorials/web/excel-addin-bert-js.html" class="nav-list-link">Custom Excel Functions for BERT Tasks in JavaScript</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Deploy on IoT and edge submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/tutorials/iot-edge/" class="nav-list-link">Deploy on IoT and edge</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/tutorials/iot-edge/rasp-pi-cv.html" class="nav-list-link">IoT Deployment on Raspberry Pi</a></li></ul></li><li class="nav-list-item"><a href="/docs/tutorials/traditional-ml.html" class="nav-list-link">Deploy traditional ML</a></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Inference with C# submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/tutorials/csharp/" class="nav-list-link">Inference with C#</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/tutorials/csharp/basic_csharp.html" class="nav-list-link">Basic C# Tutorial</a></li><li class="nav-list-item"><a href="/docs/tutorials/csharp/bert-nlp-csharp-console-app.html" class="nav-list-link">Inference BERT NLP with C#</a></li><li class="nav-list-item"><a href="/docs/tutorials/csharp/csharp-gpu.html" class="nav-list-link">Configure CUDA for GPU with C#</a></li><li class="nav-list-item"><a href="/docs/tutorials/csharp/resnet50_csharp.html" class="nav-list-link">Image recognition with ResNet50v2 in C#</a></li><li class="nav-list-item"><a href="/docs/tutorials/csharp/stable-diffusion-csharp.html" class="nav-list-link">Stable Diffusion with C#</a></li><li class="nav-list-item"><a href="/docs/tutorials/csharp/yolov3_object_detection_csharp.html" class="nav-list-link">Object detection in C# using OpenVINO</a></li><li class="nav-list-item"><a href="/docs/tutorials/csharp/fasterrcnn_csharp.html" class="nav-list-link">Object detection with Faster RCNN in C#</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="On-Device Training submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/tutorials/on-device-training/" class="nav-list-link">On-Device Training</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/tutorials/on-device-training/android-app.html" class="nav-list-link">Building an Android Application</a></li><li class="nav-list-item"><a href="/docs/tutorials/on-device-training/ios-app.html" class="nav-list-link">Building an iOS Application</a></li></ul></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Build ONNX Runtime submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/build/" class="nav-list-link">
1Build ONNX Runtime</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/build/inferencing.html" class="nav-list-link">Build for inferencing</a></li><li class="nav-list-item"><a href="/docs/build/training.html" class="nav-list-link">Build for training</a></li><li class="nav-list-item"><a href="/docs/build/eps.html" class="nav-list-link">Build with different EPs</a></li><li class="nav-list-item"><a href="/docs/build/web.html" class="nav-list-link">Build for web</a></li><li class="nav-list-item"><a href="/docs/build/android.html" class="nav-list-link">Build for Android</a></li><li class="nav-list-item"><a href="/docs/build/ios.html" class="nav-list-link">Build for iOS</a></li><li class="nav-list-item"><a href="/docs/build/custom.html" class="nav-list-link">Custom build</a></li></ul></li><li class="nav-list-item"><a href="/docs/api/" class="nav-list-link">API Docs</a></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Execution Providers submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/execution-providers/" class="nav-list-link">Execution Providers</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/execution-providers/CUDA-ExecutionProvider.html" class="nav-list-link">NVIDIA - CUDA</a></li><li class="nav-list-item"><a href="/docs/execution-providers/TensorRT-ExecutionProvider.html" class="nav-list-link">NVIDIA - TensorRT</a></li><li class="nav-list-item"><a href="/docs/execution-providers/TensorRTRTX-ExecutionProvider.html" class="nav-list-link">NVIDIA - TensorRT RTX</a></li><li class="nav-list-item"><a href="/docs/execution-providers/OpenVINO-ExecutionProvider.html" class="nav-list-link">Intel - OpenVINO™</a></li><li class="nav-list-item"><a href="/docs/execution-providers/oneDNN-ExecutionProvider.html" class="nav-list-link">Intel - oneDNN</a></li><li class="nav-list-item"><a href="/docs/execution-providers/DirectML-ExecutionProvider.html" class="nav-list-link">Windows - DirectML</a></li><li class="nav-list-item"><a href="/docs/execution-providers/QNN-ExecutionProvider.html" class="nav-list-link">Qualcomm - QNN</a></li><li class="nav-list-item"><a href="/docs/execution-providers/NNAPI-ExecutionProvider.html" class="nav-list-link">Android - NNAPI</a></li><li class="nav-list-item"><a href="/docs/execution-providers/CoreML-ExecutionProvider.html" class="nav-list-link">Apple - CoreML</a></li><li class="nav-list-item"><a href="/docs/execution-providers/Xnnpack-ExecutionProvider.html" class="nav-list-link">XNNPACK</a></li><li class="nav-list-item"><a href="/docs/execution-providers/ROCm-ExecutionProvider.html" class="nav-list-link">AMD - ROCm</a></li><li class="nav-list-item"><a href="/docs/execution-providers/MIGraphX-ExecutionProvider.html" class="nav-list-link">AMD - MIGraphX</a></li><li class="nav-list-item"><a href="/docs/execution-providers/Vitis-AI-ExecutionProvider.html" class="nav-list-link">AMD - Vitis AI</a></li><li class="nav-list-item"><a href="/docs/execution-providers/Azure-ExecutionProvider.html" class="nav-list-link">Cloud - Azure</a></li><li class="nav-list-item"><a href="/docs/execution-providers/WebGPU-ExecutionProvider.html" class="nav-list-link">WebGPU</a></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Community-maintained submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/execution-providers/community-maintained/" class="nav-list-link">Community-maintained</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/execution-providers/community-maintained/ACL-ExecutionProvider.html" class="nav-list-link">Arm - ACL</a></li><li class="nav-list-item"><a href="/docs/execution-providers/community-maintained/ArmNN-ExecutionProvider.html" class="nav-list-link">Arm - Arm NN</a></li><li class="nav-list-item"><a href="/docs/execution-providers/community-maintained/TVM-ExecutionProvider.html" class="nav-list-link">Apache - TVM</a></li><li class="nav-list-item"><a href="/docs/execution-providers/community-maintained/RKNPU-ExecutionProvider.html" class="nav-list-link">Rockchip - RKNPU</a></li><li class="nav-list-item"><a href="/docs/execution-providers/community-maintained/CANN-ExecutionProvider.html" class="nav-list-link">Huawei - CANN</a></li></ul></li><li class="nav-list-item"><a href="/docs/execution-providers/add-execution-provider.html" class="nav-list-link">Add a new provider</a></li><li class="nav-list-item"><a href="/docs/execution-providers/EP-Context-Design.html" class="nav-list-link">EP Context Design</a></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Plugin Execution Provider Libraries submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/execution-providers/plugin-ep-libraries/" class="nav-list-link">Plugin Execution Provider Libraries</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/execution-providers/plugin-ep-libraries/usage.html" class="nav-list-link">Usage</a></li><li class="nav-list-item"><a href="/docs/execution-providers/plugin-ep-libraries/development.html" class="nav-list-link">Development</a></li><li class="nav-list-item"><a href="/docs/execution-providers/plugin-ep-libraries/testing.html" class="nav-list-link">Testing</a></li><li class="nav-list-item"><a href="/docs/execution-providers/plugin-ep-libraries/packaging.html" class="nav-list-link">Packaging</a></li></ul></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Generate API (Preview) submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/genai/" class="nav-list-link">Generate API (Preview)</a><ul class="nav-list"><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Tutorials submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/genai/tutorials/" class="nav-list-link">Tutorials</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/genai/tutorials/phi3-v.html" class="nav-list-link">
1Phi-3.5 vision tutorial</a></li><li class="nav-list-item"><a href="/docs/genai/tutorials/phi3-python.html" class="nav-list-link">Phi-3 tutorial</a></li><li class="nav-list-item"><a href="/docs/genai/tutorials/phi2-python.html" class="nav-list-link">Phi-2 tutorial</a></li><li class="nav-list-item"><a href="/docs/genai/tutorials/finetune.html" class="nav-list-link">Run with LoRA adapters</a></li><li class="nav-list-item"><a href="/docs/genai/tutorials/deepseek-python.html" class="nav-list-link">DeepSeek-R1-Distill tutorial</a></li><li class="nav-list-item"><a href="/docs/genai/tutorials/snapdragon.html" class="nav-list-link">Run on Snapdragon devices</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="API docs submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/genai/api/" class="nav-list-link">API docs</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/genai/api/python.html" class="nav-list-link">Python API</a></li><li class="nav-list-item"><a href="/docs/genai/api/csharp.html" class="nav-list-link">C# API</a></li><li class="nav-list-item"><a href="/docs/genai/api/c.html" class="nav-list-link">C API</a></li><li class="nav-list-item"><a href="/docs/genai/api/cpp.html" class="nav-list-link">C++ API</a></li><li class="nav-list-item"><a href="/docs/genai/api/java.html" class="nav-list-link">Java API</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="How to submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/genai/howto/" class="nav-list-link">How to</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/genai/howto/install.html" class="nav-list-link">Install</a></li><li class="nav-list-item"><a href="/docs/genai/howto/build-from-source.html" class="nav-list-link">Build from source</a></li><li class="nav-list-item"><a href="/docs/genai/howto/build-model.html" class="nav-list-link">Build models</a></li><li class="nav-list-item"><a href="/docs/genai/howto/build-models-for-snapdragon.html" class="nav-list-link">Build models for Snapdragon</a></li><li class="nav-list-item"><a href="/docs/genai/howto/troubleshoot.html" class="nav-list-link">Troubleshoot</a></li><li class="nav-list-item"><a href="/docs/genai/howto/migrate.html" class="nav-list-link">Migrate</a></li><li class="nav-list-item"><a href="/docs/genai/howto/past-present-share-buffer.html" class="nav-list-link">Past present share buffer</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Reference submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/genai/reference/" class="nav-list-link">Reference</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/genai/reference/config.html" class="nav-list-link">Config reference</a></li><li class="nav-list-item"><a href="/docs/genai/reference/adapter.html" class="nav-list-link">Adapter file spec</a></li></ul></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Extensions submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/extensions/" class="nav-list-link">Extensions</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/extensions/add-op.html" class="nav-list-link">Add Operators</a></li><li class="nav-list-item"><a href="/docs/extensions/build.html" class="nav-list-link">Build</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Performance submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/performance/" class="nav-list-link">Performance</a><ul class="nav-list"><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Tune performance submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/performance/tune-performance/" class="nav-list-link">Tune performance</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/performance/tune-performance/profiling-tools.html" class="nav-list-link">Profiling tools</a></li><li class="nav-list-item"><a href="/docs/performance/tune-performance/logging_tracing.html" class="nav-list-link">Logging & Tracing</a></li><li class="nav-list-item"><a href="/docs/performance/tune-performance/memory.html" class="nav-list-link">Memory consumption</a></li><li class="nav-list-item"><a href="/docs/performance/tune-performance/threading.html" class="nav-list-link">Thread management</a></li><li class="nav-list-item"><a href="/docs/performance/tune-performance/iobinding.html" class="nav-list-link">I/O Binding</a></li><li class="nav-list-item"><a href="/docs/performance/tune-performance/troubleshooting.html" class="nav-list-link">Troubleshooting</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Model optimizations submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/performance/model-optimizations/" class="nav-list-link">Model optimizations</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/performance/model-optimizations/quantization.html" class="nav-list-link">Quantize ONNX models</a></li><li class="nav-list-item"><a href="/docs/performance/model-optimizations/float16.html" class="nav-list-link">Float16 and mixed precision models</a></li><li class="nav-list-item"><a href="/docs/performance/model-optimizations/graph-optimizations.html" class="nav-list-link">Graph optimizations</a></li><li class="nav-list-item">
1<a href="/docs/performance/model-optimizations/ort-format-models.html" class="nav-list-link">ORT model format</a></li><li class="nav-list-item"><a href="/docs/performance/model-optimizations/ort-format-model-runtime-optimization.html" class="nav-list-link">ORT model format runtime optimization</a></li></ul></li><li class="nav-list-item"><a href="/docs/performance/transformers-optimization.html" class="nav-list-link">Transformers optimizer</a></li><li class="nav-list-item"><a href="/docs/performance/olive.html" class="nav-list-link">End to end optimization with Olive</a></li><li class="nav-list-item"><a href="/docs/performance/device-tensor.html" class="nav-list-link">Device tensors</a></li><li class="nav-list-item"><a href="/docs/performance/kleidiai/" class="nav-list-link">Arm® KleidiAI™ software</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Ecosystem submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/ecosystem/" class="nav-list-link">Ecosystem</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/ecosystem/acpt.html" class="nav-list-link">Azure Container for PyTorch (ACPT)</a></li></ul></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Reference submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/reference/" class="nav-list-link">Reference</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/reference/releases-servicing.html" class="nav-list-link">Releases</a></li><li class="nav-list-item"><a href="/docs/reference/compatibility.html" class="nav-list-link">Compatibility</a></li><li class="nav-list-item"><button class="nav-list-expander btn-reset" aria-label="Operators submenu" aria-expanded="false"> <svg viewBox="0 0 24 24" aria-hidden="true"><use xlink:href="#svg-arrow-right"></use></svg> </button><a href="/docs/reference/operators/" class="nav-list-link">Operators</a><ul class="nav-list"><li class="nav-list-item"><a href="/docs/reference/operators/OperatorKernels.html" class="nav-list-link">Operator kernels</a></li><li class="nav-list-item"><a href="/docs/reference/operators/ContribOperators.html" class="nav-list-link">Contrib operators</a></li><li class="nav-list-item"><a href="/docs/reference/operators/add-custom-op.html" class="nav-list-link">Custom operators</a></li><li class="nav-list-item"><a href="/docs/reference/operators/reduced-operator-config-file.html" class="nav-list-link">Reduced operator config file</a></li></ul></li><li class="nav-list-item"><a href="/docs/reference/high-level-design.html" class="nav-list-link">Architecture</a></li><li class="nav-list-item"><a href="/docs/reference/citing.html" class="nav-list-link">Citing ONNX Runtime</a></li></ul></li><li class="nav-list-item"><a href="/docs/build/dependencies.html" class="nav-list-link">Dependency Management in ONNX Runtime</a></li></ul> <ul class="nav-list"><li class="nav-list-item external"> <a href="https://github.com/microsoft/onnxruntime/tree/gh-pages" class="nav-list-link external" target="_blank" rel="noopener noreferrer" > ONNX Runtime Docs on GitHub </a> </li></ul> </nav> <div class="d-md-block d-none site-footer"> This site uses <a href="https://github.com/just-the-docs/just-the-docs">Just the Docs</a>, a documentation theme for Jekyll. </div> </header> <div class="main" id="top"> <div id="main-header" class="main-header"> <div class="search" role="search"> <div class="search-input-wrap"> <input type="text" id="search-input" class="search-input" tabindex="0" placeholder="Search onnxruntime" autocomplete="off"> <label for="search-input" class="search-label"> <span class="sr-only">Search onnxruntime</span> <svg viewBox="0 0 24 24" class="search-icon" aria-hidden="true"><use xlink:href="#svg-search"></use></svg> </label> </div> <div id="search-results" class="search-results"></div> </div> <nav aria-label="Auxiliary" class="aux-nav"> <ul class="aux-nav-list"> <li class="aux-nav-list-item"> <a href="/" class="site-button" > ONNX Runtime </a> </li> <li class="aux-nav-list-item"> <a href="/docs/install/" class="site-button" > Install </a> </li> <li class="aux-nav-list-item"> <a href="/docs/get-started/" class="site-button" > Get Started </a> </li> <li class="aux-nav-list-item"> <a href="/docs/tutorials/" class="site-button" > Tutorials </a> </li> <li class="aux-nav-list-item"> <a href="/docs/api/" class="site-button" > API Docs </a> </li> <li class="aux-nav-list-item"> <a href="https://www.youtube.com/onnxruntime" class="site-button" > YouTube </a> </li> <li class="aux-nav-list-item"> <a href="https://github.com/microsoft/onnxruntime" class="site-button" > GitHub </a> </li> </ul> </nav> </div> <div class="main-content-wrap">
1 <div id="main-content" class="main-content"> <main> <h1 class="no_toc" id="welcome-to-onnx-runtime"> <a href="#welcome-to-onnx-runtime" class="anchor-heading" aria-labelledby="welcome-to-onnx-runtime"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> Welcome to ONNX Runtime </h1> <p>ONNX Runtime is a cross-platform machine-learning model accelerator, with a flexible interface to integrate hardware-specific libraries. ONNX Runtime can be used with models from PyTorch, Tensorflow/Keras, TFLite, scikit-learn, and other frameworks.</p> <iframe height="315" class="table-wrapper py px" src="https://www.youtube.com/embed/waIeC3OIn70" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe> <h2 id="how-to-use-onnx-runtime"> <a href="#how-to-use-onnx-runtime" class="anchor-heading" aria-labelledby="how-to-use-onnx-runtime"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> How to use ONNX Runtime </h2> <div class="table-wrapper"><table> <tbody> <tr> <td><span class="fs-5"> <a href="./get-started" class="btn mr-4" target="_blank">Get started with ORT</a> </span></td> <td><span class="fs-5"> <a href="./api" class="btn" target="_blank">API Docs</a> </span></td> </tr> <tr> <td><span class="fs-5"> <a href="./tutorials" class="btn" target="_blank">Tutorials</a> </span></td> <td><span class="fs-5"> <a href="./ecosystem" class="btn" target="_blank">Ecosystem</a> </span></td> </tr> <tr> <td><span class="fs-5"><a href="https://www.youtube.com/channel/UC_SJk17KdRvDulXz-nc1uFg/featured" class="btn mr-4" target="_blank">ONNX Runtime YouTube</a> </span></td> <td> </td> </tr> </tbody> </table></div> <h2 id="contribute-and-customize"> <a href="#contribute-and-customize" class="anchor-heading" aria-labelledby="contribute-and-customize"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> Contribute and Customize </h2> <div class="table-wrapper"><table> <tbody> <tr> <td><span class="fs-5"> <a href="./build" class="btn mr-4" target="_blank">Build ORT Packages</a> </span></td> <td><span class="fs-5"><a href="https://github.com/microsoft/onnxruntime" class="btn" target="_blank">ONNX Runtime GitHub</a> </span></td> </tr> </tbody> </table></div><hr /> <h2 id="quickstart-template"> <a href="#quickstart-template" class="anchor-heading" aria-labelledby="quickstart-template"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> QuickStart Template </h2> <div class="table-wrapper"><table> <tbody> <tr> <td><span class="fs-5"> <a href="https://github.com/microsoft/onnxruntime-nextjs-template" class="btn mr-4" target="_blank">ORT Web JavaScript Site Template</a> </span></td> <td><span class="fs-5"> <a href="https://github.com/microsoft/onnxruntime-csharp-cv-template" class="btn mr-4" target="_blank">ORT C# Console App Template</a> </span></td> </tr> </tbody> </table></div><hr /> <h2 id="onnx-runtime-for-inferencing"> <a href="#onnx-runtime-for-inferencing" class="anchor-heading" aria-labelledby="onnx-runtime-for-inferencing"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> ONNX Runtime for Inferencing </h2> <p>ONNX Runtime Inference powers machine learning models in key Microsoft products and services across Office, Azure, Bing, as well as dozens of community projects.</p> <p>Examples use cases for ONNX Runtime Inferencing include:</p> <ul> <li>Improve inference performance for a wide variety of ML models</li> <li>Run on different hardware and operating systems</li> <li>Train in Python but deploy into a C#/C++/Java app</li> <li>Train and perform inference with models created in different frameworks</li> </ul> <h3 class="no_toc" id="how-it-works"> <a href="#how-it-works" class="anchor-heading" aria-labelledby="how-it-works"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> How it works </h3> <p>The premise is simple.</p> <ol> <li><strong>Get a model.</strong> This can be trained from any framework that supports export/conversion to ONNX format. See the <a href="./tutorials">tutorials</a> for some of the popular frameworks/libraries.</li> <li><strong>Load and run the model with ONNX Runtime.</strong> See the <a href="./tutorials/api-basics">basic tutorials</a> for running models in different languages.</li> <li><strong><em>(Optional)</em> Tune performance using various runtime configurations or hardware accelerators.</strong> There are lots of options here - see the <a href="./performance">Performance section</a> as a starting point.</li> </ol> <p>Even without step 3, ONNX Runtime will often provide performance improvements compared to the original framework.</p> <p>ONNX Runtime applies a number of graph optimizations on the model graph then partitions it into subgraphs based on available hardware-specific accelerators. Optimized computation kernels in core ONNX Runtime provide performance improvements and assigned subgraphs benefit from further acceleration from each <a href="./execution-providers">Execution Provider</a>.</p> <h3 id="model-validation"> <a href="#model-validation" class="anchor-heading" aria-labelledby="model-validation"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a>
1 Model Validation </h3> <p>You are responsible for testing and validating any model you use with ONNX Runtime, including its accuracy, performance, and suitability for your intended use case. ONNX Runtime will validate that the model conforms to the <a href="https://onnx.ai/onnx/index.html">ONNX</a> specification. It is, however, possible to construct a malicious model that, for example, consumes large amounts of memory or compute resources unnecessarily. If you are using a model from an untrusted source, we recommend inspecting the model and testing it in a safe environment before using it in production.</p><hr /> <h2 id="onnx-runtime-for-training"> <a href="#onnx-runtime-for-training" class="anchor-heading" aria-labelledby="onnx-runtime-for-training"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> ONNX Runtime for Training </h2> <ul> <li><a href="/docs/get-started/training-pytorch.html">Large Model Training</a></li> <li><a href="/docs/get-started/training-on-device.html">On-Device Training</a></li> </ul> </main> <hr> <footer> <p class="text-small text-grey-dk-100 mb-0">For documentation questions, please <a href="https://github.com/microsoft/onnxruntime/issues/new?assignees=&labels=documentation&projects=&template=02-documentation.yml&title=%5BDocumentation%5D+" target="_blank">file an issue</a>.</p> <div class="d-flex mt-2"> <p class="text-small mb-0"> <a href="https://github.com/microsoft/onnxruntime/tree/gh-pages/docs/index.md" id="edit-this-page">Edit this page on GitHub</a> </p> </div> <div class="d-md-none mt-4 fs-2"> This site uses <a href="https://github.com/just-the-docs/just-the-docs">Just the Docs</a>, a documentation theme for Jekyll. </div> </footer> </div> </div> <div class="search-overlay"></div> </div> </body> </html>

Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.