https://juliahpc.github.io/user_hpcprofiling/

1<!doctype html> <html lang=en > <meta charset=UTF-8 > <meta name=viewport  content="width=device-width, initial-scale=1"> 

1<script src="/libs/lunr/lunr.min.js"></script>

1 

1<script src="/libs/lunr/lunr_index.js"></script>

1 

1<script src="/libs/lunr/lunrclient.min.js"></script>

1 <link rel=stylesheet  href="/libs/highlight/github.min.css"> <link rel=stylesheet  href="/css/franklin.css"> <link rel=stylesheet  href="/css/poole_hyde.css"> <link rel=stylesheet  href="/css/custom.css"> <style> html {font-size: 17px;} .franklin-content {position: relative; padding-left: 1%; padding-right: 5%; line-height: 1.35em;} @media (min-width: 940px) { .franklin-content {width: 100%; margin-left: auto; margin-right: auto;} } @media (max-width: 768px) { .franklin-content {padding-left: 6%; padding-right: 6%;} } </style> <link rel=icon  href="/assets/favicon.png"> <title>HPC Profiling Tools</title> <style> .content {max-width: 60rem} </style> <div class=sidebar > <div class="container sidebar-sticky"> <div class=sidebar-about > <br> <h1><a href="/">Julia On HPC Clusters</a></h1> <img src="/assets/juliahpc_logo.png" width=100px> </div> <br> <style> </style> <nav class=sidebar-nav  style="opacity: 0.9; margin-bottom: 1.2cm;"> <a class="sidebar-nav-item " href="/"><b>Welcome</b></a> <br> <div class=course-section >For Users</div> <a class="sidebar-nav-item " href="/user_gettingstarted/">Getting started</a> <a class="sidebar-nav-item " href="/user_vscode/">Visual Studio Code</a> <a class="sidebar-nav-item active" href="/user_hpcprofiling/">Profiling tools</a> <a class="sidebar-nav-item " href="/user_hpcsystems/">Systems with Julia support</a> <a class="sidebar-nav-item " href="/user_faq/">FAQ</a> <div class=course-section >For System Admins</div> <a class="sidebar-nav-item " href="/sysadmin_julia/">How to provide Julia to users?</a> </nav> <!-- <form id=lunrSearchForm  name=lunrSearchForm > <input class=search-input  name=q  placeholder="Enter search term" type=text > <input type=submit  value=Search  formaction="/search/index.html"> </form> --> <div style="line-height:18px; font-size: 18px; opacity: 0.85"><a href="https://github.com/JuliaHPC/juliahpc.github.io/blob/main/website/LICENSE.md">© Carsten Bauer</a></div> <br> <div class=github-link > <a href="https://github.com/JuliaHPC/juliahpc.github.io"> <svg xmlns="http://www.w3.org/2000/svg" height=28px  fill=currentColor  viewBox="0 0 496 512"> <!--! Font Awesome Free 6.4.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license (Commercial License) Copyright 2023 Fonticons, Inc. --> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z" /> </svg> </a> </div> <br> </div> </div> <div class="content container"> <div class=franklin-content > <h1 id=hpc_profiling_tools ><a href="#hpc_profiling_tools" class=header-anchor >HPC Profiling Tools</a></h1> <p>There are plenty of HPC tools for profiling out there. Figuring out which one is good in what scenario and, more importantly, which one actually works with Julia is non-trivial. This page will try to provide helpful orientation.</p> <hr /> <p><a id=content  class=anchor ></a> <strong>Content</strong></p> <div class=franklin-toc ><ol><li><a href="#nvidia_nsight_systems">NVIDIA Nsight Systems</a><li><a href="#extrae">Extrae</a><li><a href="#score-p">Score-P</a><li><a href="#likwid">LIKWID</a><li><a href="#intel_vtune_profiler">Intel VTune Profiler</a><li><a href="#other_tools">Other tools</a></ol></div> <hr /> <h2 id=nvidia_nsight_systems ><a href="#nvidia_nsight_systems" class=header-anchor >NVIDIA Nsight Systems</a></h2> <p>Good for: GPU, MPI</p> <p><a href="https://developer.nvidia.com/nsight-systems">NVIDIA Nsight Systems</a> is a powerful profiling tool for analyzing &#40;multi-&#41;GPU and/or MPI-parallel applications &#40;the latter might be somewhat surprising&#41;. Especially useful when combined with <a href="https://github.com/JuliaGPU/NVTX.jl">NVTX.jl</a> for manual instrumentation &#40;

1you can name and even color sections of your code&#41;.</p> <p>Examples:</p> <ul> <li><p><a href="https://cuda.juliagpu.org/stable/development/profiling/#NVIDIA-Nsight-Systems">Relevant section</a> of the <a href="https://github.com/JuliaGPU/CUDA.jl">CUDA.jl</a> documentation</p> </ul> <p>Impressions:</p> <p><img src="/user_hpcprofiling/nsys.png" alt="" /></p> <h2 id=extrae ><a href="#extrae" class=header-anchor >Extrae</a></h2> <p>Good for: MPI, GPU, Threads</p> <p>The Julia package <a href="https://github.com/bsc-quantic/Extrae.jl">Extrae.jl</a> allows you to use <a href="https://tools.bsc.es/extrae">Extrae</a> for analyzing parallel Julia applications. It will produce trace files that can be visualized and analyzed with <a href="https://tools.bsc.es/paraver">Paraver</a>.</p> <p>Noteworthy limitations:</p> <ul> <li><p>The package isn&#39;t battle-tested.</p> <li><p>The Paraver GUI might be overwhelming and takes some getting used to.</p> <li><p>Only works on Linux.</p> </ul> <p>Impressions &#40;of Paraver&#41;:</p> <p><img src="/user_hpcprofiling/paraver_1.jpg" alt="" /> <img src="/user_hpcprofiling/paraver_2.gif" alt="" /></p> <h2 id=score-p ><a href="#score-p" class=header-anchor >Score-P</a></h2> <p>Good for: MPI</p> <p>The Julia package <a href="https://github.com/JuliaPerf/ScoreP.jl">ScoreP.jl</a> allows you to use <a href="https://www.vi-hps.org/projects/score-p/">Score-P</a> for analyzing MPI-parallel Julia applications. Output files are of type <code>.cubex</code> &#40;profiling&#41;, which can be opened with, e.g., <a href="https://www.scalasca.org/scalasca/software/cube-4.x/download.html">Cube</a> or <a href="https://www.cs.uoregon.edu/research/tau/home.php">ParaProf</a>, and <code>.otf2</code> &#40;tracing&#41;, which can be opened with, e.g., <a href="https://vampir.eu/">Vampir</a> or <a href="https://www.intel.com/content/www/us/en/developer/tools/oneapi/trace-analyzer.html#gs.oc8bgr">Intel Trace Analyzer</a>.</p> <p>Noteworthy limitations:</p> <ul> <li><p>The package isn&#39;t battle-tested and currently experimental.</p> <li><p>While manual instrumentation works, automatic tracing of Julia functions isn&#39;t &#40;yet&#41; supported.</p> </ul> <p>Examples:</p> <ul> <li><p>You&#39;re best chance is to check out the <a href="https://github.com/JuliaPerf/ScoreP.jl/blob/main/README.md">README.md</a>.</p> <li><p>Only works on Linux.</p> </ul> <p>Impressions &#40;of Cube and Vampir&#41;:</p> <p><img src="/user_hpcprofiling/scorep_cube.png" alt="" /> <img src="/user_hpcprofiling/scorep_vampir.png" alt="" /></p> <h2 id=likwid ><a href="#likwid" class=header-anchor >LIKWID</a></h2> <p>Good for: intra-node hardware-level profiling</p> <p><a href="https://github.com/JuliaPerf/LIKWID.jl">LIKWID.jl</a>, named after the underlying eponymous benchmarking suite <a href="https://github.com/RRZE-HPC/likwid">LIKWID</a>, enables &#40;interactive&#41; monitoring of the performance of arbitrary Julia functions on a hardware level by examining hardware performance counters inside of CPUs &#40;and NVIDIA GPUs&#41;.</p> <p>Noteworthy limitations:</p> <ul> <li><p>Manual installation of LIKWID necessary &#40;no JLL&#41;.</p> <li><p>Some features marked as experimental &#40;but basic core is solid&#41;.</p> <li><p>Only works on Linux.</p> </ul> <p>Examples:</p> <ul> <li><p><a href="https://juliaperf.github.io/LIKWID.jl/stable/tutorials/counting_flops/">Counting floating point operations of arbitrary Julia functions.</a></p> <li><p><a href="https://www.youtube.com/watch?v&#61;l2fTNfEDPC0">Monitoring Performance on a Hardware Level With LIKWID.jl | Carsten Bauer | JuliaCon 2022</a></p> </ul> <p>Impressions:</p> <p><img src="/user_hpcprofiling/likwid.png" alt="" /></p> <h2 id=intel_vtune_profiler ><a href="#intel_vtune_profiler" class=header-anchor >Intel VTune Profiler</a></h2> <p>Good for: serial, multithreading, GC</p> <p>The <a href="https://www.intel.com/content/www/us/en/developer/tools/oneapi/vtune-profiler.html">Intel VTune Profiler</a> is a nice tool, e.g., for finding hot spots in your code. It supports local and remote performance profiling. To make it work with Julia check out <a href="https://github.com/JuliaPerf/IntelITT.jl">IntelITT.jl</a> and our dedicated <a href="/user_hpcprofiling/intel_vtune/">Intel VTune &#43; Julia</a> page.</p> <p>Noteworthy limitations:</p> <ul> <li><p>Works best &#40;only?&#41; on systems with Intel CPUs.</p> <li><p>Can&#39;t profile on macOS &#40;only remotely on Linux machine&#41;.</p> <li><p>May require compiling Julia from source &#40;if you want more details, e.g., about GC&#41;.</p> </ul> <p>Examples:</p> <ul> <li><p><a href="/user_hpcprofiling/intel_vtune/">Intel VTune &#43; Julia</a> &#40;e.g. remote usage via GUI&#41;</p> <li><p><a href="https://github.com/carstenbauer/julia-intelvtune">Basic CLI Example</a></p> </ul> <p>Impressions:</p> <p><img src="/user_hpcprofiling/vtunes_saxpy_details.png" alt="" /></p> <h2 id=other_tools ><a href="#other_tools" class=header-anchor >Other tools</a></h2> <p>If you know something about the following tools, in particular if and how it supports Julia, please make a PR&#33;</p> <ul> <li><p><a href="https://www.cs.uoregon.edu/research/tau/home.php">TAU</a></p> <li><p><a href="https://www.i12.rwth-aachen.de/go/id/nrbe">MUST</a> &#40;MPI runtime correctness analysis&#41;</p> <li><p><a href="https://icl.utk.edu/papi/">PAPI</a> / <a href="https://github.com/JuliaPerf/PAPI.jl">PAPI.jl</a> &#40;to be compared to LIKWID&#41;</p> <li><p><a href="http://hpctoolkit.org/index.html">HPCToolkit</a></p> </ul> <p><a href="#content">⤴ <em><strong>back to Content</strong></em></a></p> <div class=page-foot > <div class=copyright > <a href="https://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a> C. Bauer. Last modified: November 07, 2025. <br>Website built with <a href="https://github.com/tlienart/Franklin.jl">Franklin.jl</a> and the <a href="https://julialang.org">Julia programming language</a>. </div> </div> </div> </div>