https://tridao.me/blog/

1<!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title> Blog | Tri Dao </title> <meta name="author" content="Tri Dao"> <meta name="description" content="Homepage of Tri Dao. "> <meta name="keywords" content="jekyll, jekyll-theme, academic-website, portfolio-website"> <meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self' 'unsafe-inline' https:; style-src 'self' 'unsafe-inline' https:; img-src 'self' data: https:; font-src 'self' data: https:; media-src 'self' https:; frame-src 'self' https:; connect-src 'self' https:;"> <link rel="stylesheet" href="/assets/css/bootstrap.min.css?v=a4b3f509e79c54a512b890d73235ef04"> <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/css/mdb.min.css" integrity="sha256-jpjYvU3G3N6nrrBwXJoVEYI/0zw8htfFnhT9ljN3JJw=" crossorigin="anonymous"> <link defer rel="stylesheet" href="/assets/css/academicons.min.css?v=f0b7046b84e425c55f3463ac249818f5"> <link defer rel="stylesheet" href="/assets/css/scholar-icons.css?v=62b2ac103a88034e6882a5be5f3e2772"> <link defer rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700|Roboto+Slab:100,300,400,500,700|Material+Icons&amp;display=swap"> <link defer rel="stylesheet" href="/assets/css/jekyll-pygments-themes-github.css?v=591dab5a4e56573bf4ef7fd332894c99" media="" id="highlight_theme_light"> <link rel="shortcut icon" href="data:image/svg+xml,&lt;svg%20xmlns=%22http://www.w3.org/2000/svg%22%20viewBox=%220%200%20100%20100%22&gt;&lt;text%20y=%22.9em%22%20font-size=%2290%22&gt;%E2%9A%9B%EF%B8%8F&lt;/text&gt;&lt;/svg&gt;"> <link rel="stylesheet" href="/assets/css/main.css?v=d41d8cd98f00b204e9800998ecf8427e"> <link rel="canonical" href="https://tridao.me/blog/"> 

1<script src="/assets/js/theme.js?v=5fea5159b787642c1bbc1f334d60f883"></script>

1 <link defer rel="stylesheet" href="/assets/css/jekyll-pygments-themes-native.css?v=5847e5ed4a4568527aa6cfab446049ca" media="none" id="highlight_theme_dark"> 

1<script>
2    initTheme();
3  </script>

3 </head> <body class="fixed-top-nav "> <header> <nav id="navbar" class="navbar navbar-light navbar-expand-sm fixed-top" role="navigation"> <div class="container"> <a class="navbar-brand title font-weight-lighter" href="/"> Tri Dao </a> <button class="navbar-toggler collapsed ml-auto" type="button" data-toggle="collapse" data-target="#navbarNav" aria-controls="navbarNav" aria-expanded="false" aria-label="Toggle navigation"> <span class="sr-only">Toggle navigation</span> <span class="icon-bar top-bar"></span> <span class="icon-bar middle-bar"></span> <span class="icon-bar bottom-bar"></span> </button> <div class="collapse navbar-collapse text-right" id="navbarNav"> <ul class="navbar-nav ml-auto flex-nowrap"> <li class="nav-item "> <a class="nav-link" href="/">About </a> </li> <li class="nav-item active"> <a class="nav-link" href="/blog/">Blog <span class="sr-only">(current)</span> </a> </li> <li class="nav-item "> <a class="nav-link" href="/publications/">Publications </a> </li> <li class="nav-item "> <a class="nav-link" href="/awards/">Awards </a> </li> <li class="nav-item "> <a class="nav-link" href="/teaching/">Teaching </a> </li> <li class="nav-item "> <a class="nav-link" href="/repositories/">Repositories </a> </li> <li class="nav-item"> <button id="search-toggle" title="Search" onclick="openSearchModal()"> <span class="nav-link">ctrl k <i class="fa-solid fa-magnifying-glass"></i></span> </button> </li> <li class="toggle-container"> <button id="light-toggle" title="Change theme"> <i class="fa-half-sun-moon" id="light-toggle-system"></i> <i class="fa-solid fa-moon" id="light-toggle-dark"></i> <i class="fa-solid fa-sun" id="light-toggle-light"></i> </button> </li> </ul> </div> </div> </nav> <progress id="progress" value="0"> <div class="progress-container"> <span class="progress-bar"></span> </div> </progress> </header> <div class="container mt-5" role="main"> <div class="post"> <div class="header-bar"> <h1>Blog</h1> <h2></h2> </div> <br> <div class="container featured-posts"> <div class="row row-cols-3"> <div class="col mb-4"> <a href="/blog/2026/replayssm/"> <div class="card hoverable"> <div class="row g-0"> <div class="col-md-12"> <div class="card-body"> <div class="float-right"> <i class="fa-solid fa-thumbtack fa-xs"></i> </div> <h3 class="card-title text-lowercase">ReplaySSM: Cache SSM Inputs, Not State</h3> <p class="card-text"></p> <p class="post-meta"> 41 min read   ·   <a href="/blog/2026"> <i class="fa-solid fa-calendar fa-sm"></i> 2026 </a> </p> </div> </div> </div> </div> </a> </div> <div class="col mb-4"> <a href="/blog/2026/sonicmoe-blackwell/"> <div class="card hoverable"> <div class="row g-0"> <div class="col-md-12"> <div class="card-body"> <div class="float-right"> <i class="fa-solid fa-thumbtack fa-xs"></i> </div> <h3 class="card-title text-lowercase">SonicMoE: A Hardware-Efficient and Software-Extensible Blueprint for Fine-Grained MoEs</h3> <p class="card-text"></p> <p class="post-meta"> 46 min read   ·   <a href="/blog/2026"> <i class="fa-solid fa-calendar fa-sm"></i> 2026 </a> </p> </div> </div> </div> </div> </a> </div> <div class="col mb-4"> <a href="/blog/2026/gram-newton-schulz/"> <div class="card hoverable"> <div class="row g-0"> <div class="col-md-12"> <div class="card-body"> <div class="float-right"> <i class="fa-solid fa-thumbtack fa-xs"></i> </div> <h3 class="card-title text-lowercase">Gram Newton-Schulz: A Fast, Hardware-Aware Newton-Schulz Algorithm for Muon</h3> <p class="card-text"></p> <p class="post-meta"> 78 min read   ·   <a href="/blog/2026"> <i class="fa-solid fa-calendar fa-sm"></i> 2026 </a> </p> </div> </div> </div> </div> </a> </div> </div> </div> <hr> <ul class="post-list"> <li> <div class="row"> <div class="col-sm-9"> <h3> <a class="post-title" href="/blog/2026/replayssm/">ReplaySSM: Cache SSM Inputs, Not State</a> </h3> <p></p> <p class="post-meta"> 41 min read   ·   June 15, 2026 </p> <p class="post-tags"> <a href="/blog/2026"> <i class="fa-solid fa-calendar fa-sm"></i> 2026 </a> </p> </div> <div class="col-sm-3"> <img class="card-img" src="/assets/img/2026-06-15-replayssm/headline_approach.png" style="object-fit: cover; height: 90%" alt="image"> </div> </div> </li> <li> <div class="row"> <div class="col-sm-9"> <h3> <a class="post-title" href="/blog/2026/sonicmoe-blackwell/">SonicMoE: A Hardware-Efficient and Software-Extensible Blueprint for Fine-Grained MoEs</a> </h3> <p></p> <p class="post-meta"> 46 min read   ·   April 22, 2026 </p> <p class="post-tags"> <a href="/blog/2026"> <i class="fa-solid fa-calendar fa-sm"></i> 2026 </a> </p> </div> <div class="col-sm-3"> <img class="card-img" src="/assets/img/2026-04-22-sonicmoe/blogpost_thumbnail.png" style="object-fit: cover; height: 90%" alt="image"> </div> </div> </li> <li> <h3> <a class="post-title" href="/blog/2026/gram-newton-schulz/">Gram Newton-Schulz: A Fast, Hardware-Aware Newton-Schulz Algorithm for Muon</a> </h3> <p></p> <p class="post-meta"> 78 min read   ·   March 30, 2026 </p> <p class="post-tags"> <a href="/blog/2026"> <i class="fa-solid fa-calendar fa-sm"></i> 2026 </a> </p> </li> <li> <div class="row"> <div class="col-sm-9"> <h3> <a class="post-title" href="/blog/2026/mamba3-part2/">Mamba-3 Part 2 - Methodological Deep Dive</a> </h3> <p></p> <p class="post-meta"> 27 min read   ·   March 16, 2026 </p> <p class="post-tags"> <a href="/blog/2026"> <i class="fa-solid fa-calendar fa-sm"></i> 2026 </a> </p> </div> <div class="col-sm-3"> <img class="card-img" src="/assets/img/2026-03-16-mamba-3/mamba3-method-thumbn

3ail.png" style="object-fit: cover; height: 90%" alt="image"> </div> </div> </li> <li> <div class="row"> <div class="col-sm-9"> <h3> <a class="post-title" href="/blog/2026/mamba3-part1/">Mamba-3 Part 1</a> </h3> <p></p> <p class="post-meta"> 15 min read   ·   March 16, 2026 </p> <p class="post-tags"> <a href="/blog/2026"> <i class="fa-solid fa-calendar fa-sm"></i> 2026 </a> </p> </div> <div class="col-sm-3"> <img class="card-img" src="/assets/img/2026-03-16-mamba-3/mamba3-fig.png" style="object-fit: cover; height: 90%" alt="image"> </div> </div> </li> </ul> <nav aria-label="Blog page navigation"> <ul class="pagination pagination-lg justify-content-center"> <li class="page-item disabled"> <a class="page-link" href="" tabindex="-1" aria-disabled="">&lt;</a> </li> <li class="page-item active"> <a class="page-link" href="/blog/" title="Blog">1</a> </li> <li class="page-item "> <a class="page-link" href="/blog/page/2/" title="Blog - page 2">2</a> </li> <li class="page-item "> <a class="page-link" href="/blog/page/3/" title="Blog - page 3">3</a> </li> <li class="page-item "> <a class="page-link" href="/blog/page/2/">&gt;</a> </li> </ul> </nav> </div> </div> <footer class="fixed-bottom" role="contentinfo"> <div class="container mt-0"> © Copyright 2026 Tri Dao. </div> </footer> 

3<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/jquery.min.js" integrity="sha256-/xUj+3OJU5yExlq6GSYGSHk7tPXikynS7ogEvDej/m4=" crossorigin="anonymous"></script>

3 

3<script src="/assets/js/bootstrap.bundle.min.js"></script>

3 

3<script src="https://cdn.jsdelivr.net/npm/[email protected]/js/mdb.min.js" integrity="sha256-NdbiivsvWt7VYCt6hYNT3h/th9vSTL4EDWeGs5SN3DA=" crossorigin="anonymous"></script>

3 

3<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/masonry.pkgd.min.js" integrity="sha256-Nn1q/fx0H7SNLZMQ5Hw5JLaTRZp0yILA/FRexe19VdI=" crossorigin="anonymous"></script>

3 

3<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/imagesloaded.pkgd.min.js" integrity="sha256-htrLFfZJ6v5udOG+3kNLINIKh2gvoKqwEhHYfTTMICc=" crossorigin="anonymous"></script>

3 

3<script defer src="/assets/js/masonry.js?v=a0db7e5d5c70cc3252b3138b0c91dcaf" type="text/javascript"></script>

3 

3<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/medium-zoom.min.js" integrity="sha256-ZgMyDAIYDYGxbcpJcfUnYwNevG/xi9OHKaR/8GK+jWc=" crossorigin="anonymous"></script>

3 

3<script defer src="/assets/js/zoom.js?v=85ddb88934d28b74e78031fd54cf8308"></script>

3 

3<script src="/assets/js/no_defer.js?v=2781658a0a2b13ed609542042a859126"></script>

3 

3<script defer src="/assets/js/common.js?v=c15de51d4bb57887caa2c21988d97279"></script>

3 

3<script defer src="/assets/js/copy_code.js?v=c8a01c11a92744d44b093fc3bda915df" type="text/javascript"></script>

3 

3<script defer src="/assets/js/jupyter_new_tab.js?v=d9f17b6adc2311cbabd747f4538bb15f"></script>

3 

3<script async src="https://d1bxh8uas1mnw7.cloudfront.net/assets/embed.js"></script>

3 

3<script async src="https://badge.dimensions.ai/badge.js"></script>

3 

3<script defer type="text/javascript" id="MathJax-script" src="https://cdn.jsdelivr.net/npm/[email protected]/es5/tex-mml-chtml.js" integrity="sha256-MASABpB4tYktI2Oitl4t+78w/lyA+D7b/s9GEP0JOGI=" crossorigin="anonymous"></script>

3 

3<script src="/assets/js/mathjax-setup.js?v=a5bb4e6a542c546dd929b24b8b236dfd"></script>

3 

3<script defer src="https://cdnjs.cloudflare.com/polyfill/v3/polyfill.min.js?features=es6" crossorigin="anonymous"></script>

3 

3<script defer src="/assets/js/progress-bar.js?v=2f30e0e6801ea8f5036fa66e1ab0a71a" type="text/javascript"></script>

3 

3<script src="/assets/js/vanilla-back-to-top.min.js?v=f40d453793ff4f64e238e420181a1d17"></script>

3 

3<script>
4    addBackToTop();
5  </script>

5 

5<script type="module" src="/assets/js/search/ninja-keys.min.js?v=a3446f084dcaecc5f75aa1757d087dcf"></script>

5 <ninja-keys hidebreadcrumbs noautoloadmdicons placeholder="Type to start searching"></ninja-keys> 

5<script src="/assets/js/search-setup.js?v=6c304f7b1992d4b60f7a07956e52f04a"></script>

5 

5<script src="/assets/js/search-data.js"></script>

5 

5<script src="/assets/js/shortcut-key.js?v=ccc841c459bfc0e64c1c2b5acd10df02"></script>

5 </body> </html>