1<!DOCTYPE html> 2<html lang="en-US"> 3 <head> 4 5 <!-- Global site tag (gtag.js) - Google Analytics -->
vendor: 64 bytes, lines 5-6
5 6<script async src="https://www.googletagmanager.com/gtag/js?id=
6G-D1L0JVQB04
vendor: 12 bytes, line 6
6"></script>
7<script> 8
vendor: 134 bytes, lines 8-12
8window.dataLayer = window.dataLayer || []; 9 function gtag(){dataLayer.push(arguments);} 10 gtag('js', new Date()); 11 12 gtag('config', '
12G-D1L0JVQB04
vendor: 4 bytes, line 12
12');
13</script>
13 14 15 16 <meta charset="utf-8"> 17 <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> 18 <title>Benchmarks - SmartCore</title> 19 20 <link rel="apple-touch-icon-precomposed" sizes="57x57" href="/assets/favicon/apple-touch-icon-57x57.png" /> 21<link rel="apple-touch-icon-precomposed" sizes="114x114" href="/assets/favicon/apple-touch-icon-114x114.png" /> 22<link rel="apple-touch-icon-precomposed" sizes="72x72" href="/assets/favicon/apple-touch-icon-72x72.png" /> 23<link rel="apple-touch-icon-precomposed" sizes="144x144" href="/assets/favicon/apple-touch-icon-144x144.png" /> 24<link rel="apple-touch-icon-precomposed" sizes="60x60" href="/assets/favicon/apple-touch-icon-60x60.png" /> 25<link rel="apple-touch-icon-precomposed" sizes="120x120" href="/assets/favicon/apple-touch-icon-120x120.png" /> 26<link rel="apple-touch-icon-precomposed" sizes="76x76" href="/assets/favicon/apple-touch-icon-76x76.png" /> 27<link rel="apple-touch-icon-precomposed" sizes="152x152" href="/assets/favicon/apple-touch-icon-152x152.png" /> 28<link rel="icon" type="image/png" href="/assets/favicon/favicon-196x196.png" sizes="196x196" /> 29<link rel="icon" type="image/png" href="/assets/favicon/favicon-96x96.png" sizes="96x96" /> 30<link rel="icon" type="image/png" href="/assets/favicon/favicon-32x32.png" sizes="32x32" /> 31<link rel="icon" type="image/png" href="/assets/favicon/favicon-16x16.png" sizes="16x16" /> 32<link rel="icon" type="image/png" href="/assets/favicon/favicon-128.png" sizes="128x128" /> 33<meta name="application-name" content=" "/> 34<meta name="msapplication-TileColor" content="#FFFFFF" /> 35<meta name="msapplication-TileImage" content="/assets/favicon/mstile-144x144.png" /> 36<meta name="msapplication-square70x70logo" content="/assets/favicon/mstile-70x70.png" /> 37<meta name="msapplication-square150x150logo" content="/assets/favicon/mstile-150x150.png" /> 38<meta name="msapplication-wide310x150logo" content="/assets/favicon/mstile-310x150.png" /> 39<meta name="msapplication-square310x310logo" content="/assets/favicon/mstile-310x310.png" /> 40 41 42 <link rel="stylesheet" href="/assets/css/main.css"> 43 <link rel="stylesheet" href="https://cdn.rawgit.com/afeld/bootstrap-toc/v1.0.1/dist/bootstrap-toc.min.css"/> 44 <link rel="shortcut icon" type="image/png" href="/favicon.png"> 45 <!-- Begin Jekyll SEO tag v2.8.0 --> 46<title>Benchmarks | SmartCore</title> 47<meta name="generator" content="Jekyll v3.10.0" /> 48<meta property="og:title" content="Benchmarks" /> 49<meta name="author" content="SmartCore Developers" /> 50<meta property="og:locale" content="en_US" /> 51<meta name="description" content="Live benchmark trend charts for SmartCore â wall-clock time and deterministic instruction-count regression tracking on every development push." /> 52<meta property="og:description" content="Live benchmark trend charts for SmartCore â wall-clock time and deterministic instruction-count regression tracking on every development push." /> 53<link rel="canonical" href="https://smartcorelib.github.io/user_guide/benchmarks.html" /> 54<meta property="og:url" content="https://smartcorelib.github.io/user_guide/benchmarks.html" /> 55<meta property="og:site_name" content="SmartCore" /> 56<meta property="og:type" content="website" /> 57<meta name="twitter:card" content="summary" /> 58<meta property="twitter:title" content="Benchmarks" />
59<script type="application/ld+json"> 60{"@context":"https://schema.org","@type":"WebPage","author":{"@type":"Person","name":"SmartCore Developers"},"description":"Live benchmark trend charts for SmartCore â wall-clock time and deterministic instruction-count regression tracking on every development push.","headline":"Benchmarks","url":"https://smartcorelib.github.io/user_guide/benchmarks.html"}</script>
60 61<!-- End Jekyll SEO tag --> 62 63 </head> 64 <body> 65 <nav class="navbar navbar-expand-md navbar-dark bg-dark"> 66 <div class="container"> 67 <a class="navbar-brand text-primary font-weight-bold mt-1" href="/"> 68 <img src="/assets/logo/smartcore.png" width="50" height="50" class="d-inline-block align-middle" alt=""> 69 SmartCore 70 </a> 71 <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarNav"> 72 <span class="navbar-toggler-icon"></span> 73 </button> 74 <div class="collapse navbar-collapse" id="navbarNav"> 75 <ul class="navbar-nav mr-auto"> 76 77 <li class="nav-item "> 78 79 <a class="nav-link font-weight-bold align-middle" href="/user_guide/quick_start.html">Quick Start</a> 80 81 </li> 82 83 <li class="nav-item "> 84 85 <a class="nav-link font-weight-bold align-middle" href="https://docs.rs/smartcore/0.6/smartcore/">API</a> 86 87 </li> 88 89 <li class="nav-item "> 90 91 <a class="nav-link font-weight-bold align-middle" href="https://github.com/smartcorelib/smartcore-jupyter">Examples</a> 92 93 </li> 94 95 <li class="nav-item active"> 96 97 <a class="nav-link font-weight-bold align-middle" href="/user_guide/benchmarks.html">Benchmarks</a> 98 99 </li> 100 101 <li class="nav-item dropdown"> 102 <a class="nav-link dropdown-toggle font-weight-bold align-middle" href="#" id="navbarDropdown" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false"> 103 User Guide 104 </a> 105 <div class="dropdown-menu" aria-labelledby="navbarDropdown"> 106 107 108 <a class="dropdown-item" href="/user_guide/quick_start.html">Getting Started</a> 109 110 111 112 <a class="dropdown-item" href="/user_guide/supervised.html">Supervised Learning</a> 113 114 115 116 <a class="dropdown-item" href="/user_guide/unsupervised.html">Unsupervised Learning</a> 117 118 119 120 <a class="dropdown-item" href="/user_guide/preprocessing.html">Preprocessing</a> 121 122 123 124 <a class="dropdown-item" href="/user_guide/model_selection.html">Model Selection</a> 125 126 127 128 <a class="dropdown-item" href="/user_guide/benchmarks.html">Benchmarks</a> 129 130 131 132 <a class="dropdown-item" href="/user_guide/developer.html">Developer's Guide</a> 133 134 135 </div> 136 </li> 137 </ul> 138 <ul class="navbar-nav ml-auto"> 139 140 <li class="nav-item "> 141 142 <a class="nav-link font-weight-bold align-middle" href="/about.html">About Us</a> 143 144 </li> 145 146 <li class="nav-item"> 147 <a class="nav-link" href="https://github.com/smartcorelab/smartcore"> 148 <span><i class="fab fa-github" aria-hidden="true"></i></span> 149 </a> 150 </li> 151 </ul> 152 </div> 153 </div> 154</nav> 155 156 <div class="container"> 157 158 <div class="row flex-xl-nowrap"> 159 160 <div class="col-md-3"> 161 <div class=" bg-light rounded mt-3 p-3 text-center font-weight-bold text-muted"> 162 SmartCore 0.6 163 <br/> 164 <a class="align-bottom font-weight-light" href="/user_guide/versions.html">other version</a> 165 </div> 166 <nav id="toc" class="sticky-top bg-light rounded mt-3 p-3"></nav> 167 </div> 168 169 <div manual="true" class="col-md-9 mt-3 text-justify" markdown="1"> 170 171 <h1 id="benchmarks">Benchmarks</h1> 172 173<p><em>SmartCore</em> runs a continuous benchmark harness on every push to the <code class="language-plaintext highlighter-rouge">development</code> branch. The harness lives in the <a href="https://github.com/smartcorelib/smartcore-benches">smartcore-benches</a> repository, which covers the five hot paths flagged in <a href="https://github.com/smartcorelib/smartcore/issues/407">smartcore#407</a> as well as the legacy algorithm benches.</p> 174 175<h2 id="live-trend-charts">Live trend charts</h2> 176 177<p>Trend charts and JSON history are published to GitHub Pages by <a href="https://github.com/benchmark-action/github-action-benchmark">benchmark-action/github-action-benchmark</a>:</p> 178 179<table> 180 <thead> 181 <tr> 182 <th style="text-align: center">Tool</th> 183 <th style="text-align: center">Chart</th> 184 <th>What it plots</th> 185 <th>Role</th> 186 </tr> 187 </thead> 188 <tbody> 189 <tr> 190 <td style="text-align: center">â±ï¸ criterion</td> 191 <td style="text-align: center"><a href="https://smartcorelib.github.io/smartcore-benches/dev/">smartcorelib.github.io/smartcore-benches/dev/</a></td> 192 <td>Wall-clock time per bench (ns/iter) over time. Lower is better.</td> 193 <td>Advisory â 200% alert threshold. Wall-clock numbers on shared G
193itHub runners are noisy (20â60% variance is normal), so this chart is informational, not a merge gate.</td> 194 </tr> 195 <tr> 196 <td style="text-align: center">ð iai-callgrind</td> 197 <td style="text-align: center"><a href="https://smartcorelib.github.io/smartcore-benches/iai-dev/">smartcorelib.github.io/smartcore-benches/iai-dev/</a></td> 198 <td>Instructions retired (<code class="language-plaintext highlighter-rouge">Ir</code>) per bench â deterministic and machine-independent. Lower is better.</td> 199 <td>Deterministic gate â 120% alert threshold, fails the job on a >20% regression. Posts a <code class="language-plaintext highlighter-rouge">success</code>/<code class="language-plaintext highlighter-rouge">failure</code> status check back to the triggering smartcore commit.</td> 200 </tr> 201 </tbody> 202</table> 203 204<p>Open either chart in a browser. Each page shows a searchable line chart with one series per benchmark name (e.g. <code class="language-plaintext highlighter-rouge">matmul/1024</code>, <code class="language-plaintext highlighter-rouge">iai_matmul::matmul::bench_matmul_256</code>). Hover for the value and the commit that produced each point.</p> 205 206<h2 id="what-the-iai-callgrind-gate-means">What the iai-callgrind gate means</h2> 207 208<p>Instruction counts from <a href="https://valgrind.org/docs/manual/cl-manual.html">Valgrindâs Callgrind</a> are <strong>machine-independent</strong> â the same binary reports the same instruction count on every Linux x86_64 host, including GitHub-hosted runners. This makes a tight 120% threshold reliable: a >20% jump is a real code change, not measurement noise. When the <code class="language-plaintext highlighter-rouge">iai</code> job fails, it means a smartcore commit regressed the instruction count of one of the hot paths beyond the threshold â the smartcore commit receives a <code class="language-plaintext highlighter-rouge">smartcore-benches / iai-callgrind</code> failed status check, which branch protection can require.</p> 209 210<h2 id="per-version-snapshots">Per-version snapshots</h2> 211 212<p>In addition to the rolling trend, the <code class="language-plaintext highlighter-rouge">criterion</code> job records one frozen <code class="language-plaintext highlighter-rouge">criterion.json</code> per published smartcore version under <code class="language-plaintext highlighter-rouge">benches-results/v<version>/</code> in the benches repo â useful for diffing two smartcore releases against each other rather than tracking the rolling trend.</p> 213 214<h2 id="raw-data-and-artifacts">Raw data and artifacts</h2> 215 216<p>Each CI run uploads its raw bench output as a downloadable artifact:</p> 217 218<ul> 219 <li><code class="language-plaintext highlighter-rouge">criterion-json</code> â <code class="language-plaintext highlighter-rouge">bench-output/criterion.json</code> (bencher format)</li> 220 <li><code class="language-plaintext highlighter-rouge">iai-json</code> â <code class="language-plaintext highlighter-rouge">bench-output/iai.ndjson</code> (raw iai-callgrind NDJSON) + <code class="language-plaintext highlighter-rouge">bench-output/iai.json</code> (the <code class="language-plaintext highlighter-rouge">customSmallerIsBetter</code> array the adapter produces)</li> 221</ul> 222 223<p>Download from a run page, or via the <code class="language-plaintext highlighter-rouge">gh</code> CLI:</p> 224 225<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gh run download <run-id> <span class="nt">--repo</span> smartcorelib/smartcore-benches <span class="nt">-n</span> iai-json <span class="nt">-D</span> ./iai-out 226gh run download <run-id> <span class="nt">--repo</span> smartcorelib/smartcore-benches <span class="nt">-n</span> criterion-json <span class="nt">-D</span> ./crit-out 227</code></pre></div></div> 228 229<h2 id="where-to-learn-more">Where to learn more</h2> 230 231<p>The full guide on how to read the results, the dispatch contract that fires from smartcoreâs CI, and the required secrets are documented in the <a href="https://github.com/smartcorelib/smartcore-benches#results">smartcore-benches README</a>.</p> 232 233 234 <footer class="my-md-5 pt-md-5 border-top"> 235 <div class="row justify-content-md-center"> 236 <div class="col-md-auto"> 237 <div class="row"> 238 <img class="mr-1" alt="License Apache 2.0" src="https://img.shields.io/github/license/smartcorelib/smartcore"> 239 <img class="mr-1" alt="Build status" src="https://img.shields.io/circleci/build/github/smartcorelib/smartcore"> 240 <img class="mr-1" alt="Code coverage" src="https://img.shields.io/codecov/c/github/smartcorelib/smartcore"> 241 <img class="mr-1" alt="Latest version" src="https://img.shields.io/crates/v/smartcore"> 242 </div> 243 </div> 244 <div class="col"> 245 <div class="d-block mb-3 text-muted text-right"> 246 MIT/APACHE-2.0 © 2020-2026 SmartCore developers 247 </div> 248 </div> 249 </div> 250</footer> 251 252 </div> 253 </div> 254 </div> 255
255<script src="https://code.jquery.com/jquery-3.5.1.slim.min.js" integrity="sha384-DfXdz2htPH0lsSSs5nCTpuj/zy4C+OGpamoFVy38MVBnE+IbbVYUew+OrCXaRkfj" crossorigin="anonymous"></script>
255 256
256<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/umd/popper.min.js" integrity="sha384-9/reFTGAW83EW2RDu2S0VKaIzap3H66lZH81PoYlFhbGU+6BZp6G7niu735Sk7lN" crossorigin="anonymous"></script>
256 257
257<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js" integrity="sha384-B4gt1jrGC7Jh4AgTPSdUtOBvfO8shuf57BaghqFfPlYxofvL8/KUEfYiJOMMV+rV" crossorigin="anonymous"></script>
257 258
258<script src="https://cdn.rawgit.com/afeld/bootstrap-toc/v1.0.1/dist/bootstrap-toc.min.js"></script>
258 259
259<script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
259 260
260<script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
260 261
261<script src="/assets/js/main.js"></script>
261 262 </body> 263</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.