1<script type="text/javascript">google.load("jquery", "1.3.2");</script>
vendor: 1 bytes, line 1
1
2<script 3 type="module" 4 src="https://gradio.s3-us-west-2.amazonaws.com/3.33.1/gradio.js" 5></script>
5 6 7<!doctype html> 8<html> 9<head> 10 <title>Nested Diffusion</title> 11 <!-- Required meta tags --> 12 <meta charset="utf-8"> 13 <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> 14 <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css"> 15 <link href="style.css" rel="stylesheet" type="text/css"> 16 <meta property="og:image" content="resources/teaser.png"/> 17 <meta property="og:title" content="Academic Webpage Template" /> 18 <meta property="og:description" content="a webpage template for academic projects" /> 19
19<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.1/jquery.min.js"></script>
19 20
20<script src="https://kit.fontawesome.com/ad96f96272.js" crossorigin="anonymous"></script>
20 21</head> 22 23<body> 24
24<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/js/bootstrap.bundle.min.js" integrity="sha384-ka7Sk0Gln4gmtz2MlQnikT1wXgYsOg+OMhuP+IlRH9sENBO0LRn5q+8nbTov4+1p" crossorigin="anonymous"></script>
24 25 <link href="https://cdn.jsdelivr.net/npm/[email protected]/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-1BmE4kWBq78iYhFldvKuhfTAU6auU8tT94WrHftjDbrCEXSU1oBoqyl2QvZ6jIW3" crossorigin="anonymous"> 26 27 <!-- navigation bar on the top, nav bar stick to the top --> 28 <nav id="navbar_top" class="navbar sticky-top navbar-expand-lg navbar-dark bg-primary" style="margin-bottom: 15px;"> 29 <div class="container"> 30 <a class="navbar-brand" href="">Nested Diffusion</a> 31 <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#main_nav"> 32 <span class="navbar-toggler-icon"></span> 33 </button> 34 <div class="collapse navbar-collapse" id="main_nav"> 35 <ul class="navbar-nav ms-auto"> 36 <li class="nav-item"><a class="nav-link" href="#generation">Anytime Generation</a></li> 37 <li class="nav-item"><a class="nav-link" href="#try">Try it!</a></li> 38 <li class="nav-item"><a class="nav-link" href="#interactive">Interactive Generation</a></li> 39 <li class="nav-item"><a class="nav-link" href="#paper">Paper</a></li> 40 </ul> 41 </div> 42 </div> 43 </nav> 44 <!-- end nav bar --> 45 46 <div class="container-xl"> 47 <!-- top button --> 48 <button type="button" class="btn btn-danger btn-floating btn-lg" id="btn-back-to-top"> 49 <i class="fas fa-arrow-up"></i> 50 </button> 51 <!-- end top button --> 52 53 <!-- top - author names, affiliation and links --> 54 <center> 55 <h1>Nested Diffusion Processes for Anytime Image Generation</h1> 56 <h4>WACV 2024</h4> 57 <div class="container"> 58 <div class="row justify-content-md-center"> 59 <div class="col-md-auto"> 60 <span style="font-size:24px"><a href="https://noamelata.github.io/" target="_blank" rel="noopener noreferrer">Noam Elata</a></span> 61 </div> 62 <div class="col-md-auto"> 63 <span style="font-size:24px"><a href="https://bahjat-kawar.github.io/" target="_blank" rel="noopener noreferrer">Bahjat Kawar</a></span> 64 </div> 65 <div class="col-md-auto"> 66 <span style="font-size:24px"><a href="https://tomer.net.technion.ac.il/" target="_blank" rel="noopener noreferrer">Tomer Michaeli</a></span> 67 </div> 68 <div class="col-md-auto"> 69 <span style="font-size:24px"><a href="https://elad.cs.technion.ac.il/author/mikielad/b" target="_blank" rel="noopener noreferrer">Michael Elad</a></span> 70 </div> 71 </div> 72 </div> 73 74 <p class="text-center affilation" style="font-size: 20pt;">Technion - Israel Institute of Technology<p> 75 76 <div class="container"> 77 <div class="row justify-content-md-center"> 78 <div class="col-md-auto"> 79 <span style="font-size:24px"><a href='resources/WACV_2024_Nested_Diffusion.pdf' target="_blank" rel="noopener noreferrer">[<i class="far fa-file-pdf"></i> Paper]</a></span> 80 </div> 81 <div class="col-md-auto"> 82 <span style="font-size:24px"><a href='https://arxiv.org/abs/2305.19066' target="_blank" rel="noopener noreferrer">[<i class="ai ai-arxiv"></i> ArXiv]</a></span> 83 </div> 84 <div class="col-md-auto"> 85 <span style="font-size:24px"><a href="https://github.com/noamelata/NestedDiffusion" target="_blank" rel="noopener noreferrer">[<i class="fab fa-github"></i> Code]</a></span> 86 </div> 87 </div> 88 </div> 89 </center> 90 <!-- end top --> 91 92 <!-- abstract and teaser image--> 93 <center> 94 <div class="container" style="margin-top: 20px;"> 95 <div class="row justify-content-md-center"> 96 <img alt="" id="teaser" src="./resources/diagram.png" style="max-width: 500px;"/> 97 </div> 98 </div> 99 <h1 style="margin-top: 10px;">Abstract</h1> 100 </center> 101 <p align="justify"> 102 Diffusion models are the current state-of-the-art in image generation, synthesizing high-quality images by breaking down the generation process into many fine-grained denoising steps. Despite their good performance, diffusion models are computationally expensive, requiring many neural function evaluations (NFEs). In this work, we propose an anytime diffusion-based method that can generate viable images when stopped at arbitrary times before completion. Using existing pretrained diffusion models, we show that the generation scheme can be recomposed as two nested diffusion processes, enabling fast iterative refinement of a generated image. In experiments on ImageNet and Stable Diffusion-based text-to-image generation, we show, both qualitatively and quantitatively, that our method's intermediate generation quality greatly exceeds that of the original diffusion model, while the final generation result remains comparable. We illustrate the applicability of Nested Diffusion in several settings, including for solving inverse problems, and for rapid text-based content creation by allowing user intervention throughout the sampling process. 103 </p> 104 <!-- end abstract and teaser image--> 105 106 <!-- video of a talk about your paper (youtube embedding) --> 107 <!-- <center><h1>Talk</h1></center> 108 <p align="center"> 109 <iframe width="560" height="315" src="https://www.youtube.com/embed/ly36kn0ug4k" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe> </p> --> 110 111 <!-- Generation --> 112 <hr id="generation"> 113 <center><h1 style="margin-top: 65px;">Anytime Image Generation</h1></center> 114 <p>Nested Diffusion is an anytime algorithm for using pretrained Diffusion Models 115 <br> 116 During generation, Nested Diffusion will yield sample images which converge to the final result.
117 </p> 118 <center> 119 <table class="table smallpad" width="1" style="max-width: 1000px;"> 120 <tbody> 121 <tr> 122 <td ><img id="generation_gif" class="round" width="100%" src="./resources/generation_hd.gif" style="max-width: 2000px;"/></td> 123 </tr> 124 </table> 125 </center> 126 <p> 127 Images created with Nested Diffusion will improve in quality untill the generation process terminates, with the final samples comparable in quality to a vanilla diffusion model with the same computational budget. The user also has the choice to stop the process at any point if the current image is satisfactory. 128 </br> 129 Nested Diffusion can be used with conditional and unconditional diffusion models, extending to text-to-image applications. Additionally, Nested Diffusion also works well in diffusion-based inverse-problem solving methods, such as <a href="https://ddrm-ml.github.io/" target="_blank" rel="noopener noreferrer">DDRM</a>. 130 </br> 131 Quantitative comparisons can be found in the <a href='resources/WACV_2024_Nested_Diffusion.pdf' target="_blank" rel="noopener noreferrer">paper</a>. 132 </p> 133 134 135 136 <!-- end Generation --> 137 138 <!-- Generation --> 139 <hr id="try"> 140 <center><h1 style="margin-top: 65px;">Try It Yourself!</h1></center> 141 <p>Nested Diffusion is available with <a href="https://stability.ai/blog/stable-diffusion-public-release">Stable Diffusion</a> online thanks to the generous help of <a href="https://huggingface.co/">HuggingFace</a>. 142 <br> 143 If space requires restarting, please wait a few minutes while the model is downloaded. 144 </p> 145 <center> 146 <!-- <iframe 147 src="https://noamelata-nested-diffusion.hf.space" 148 frameborder="0" 149 width="1100" 150 height="900" 151 ></iframe> --> 152 153 <gradio-app src="https://noamelata-nested-diffusion.hf.space"></gradio-app> 154 </center> 155 156 <!-- end Generation --> 157 158 159 <!-- Interactive --> 160 <hr id="interactive"> 161 <center><h1 style="margin-top: 65px;">Interactive generation</h1></center> 162 <p> 163 Nested Diffusion's back-and-forth algorithm inherently enables interacting with the diffusion process. 164 </br> 165 In our paper, we propose as an example pruning of unwanted generation trajectories, as well as intervation in the desired guidance during generation. 166 </p> 167 <center> 168 <table class="table smallpad" width="1" style="max-width: 1000px;"> 169 <tbody> 170 <tr> 171 <td ><img id="interative_gif" class="round" width="100%" src="./resources/interactive.gif" style="max-width: 2000px;"/></td> 172 </tr> 173 </table> 174 </center> 175 <!-- Interactive --> 176 177 178 <!-- Paper --> 179 <hr id="paper"> 180 <center><h1 style="margin-top: 65px;">Paper</h1></center> 181 <table align=center class="table" style="max-width: 750px"> 182 <tr> 183 <td> 184 <a href='resources/WACV_2024_Nested_Diffusion.pdf' target="_blank" rel="noopener noreferrer"> 185 <img alt="" class="layered-paper-big" style="height:100%; max-height: 175px;" src="./resources/paper.png"/> 186 </a></td> 187 <td><span style="font-size:14pt"><b>Nested Diffusion Processes for Anytime Image Generation</b><br> 188 Noam Elata, Bahjat Kawar, Tomer Michaeli, Michael Elad</span> 189 <br> 190 <span style="font-size:14pt"><a href='https://arxiv.org/abs/2305.19066' target="_blank" rel="noopener noreferrer">[<i class="ai ai-arxiv"></i> Arxiv]</a></span> 191 <span style="font-size:14pt"><a href='resources/WACV_2024_Nested_Diffusion.pdf' target="_blank" rel="noopener noreferrer">[<i class="far fa-file-pdf"></i> Paper]</a></span> 192 </span> 193 </td> 194 </tr> 195 </table> 196 197 <!-- bibtex citation --> 198 <p class="section" id="bibtex"><b>Bibtex</b></p> 199 <table border="0"> 200 <tbody> 201 <pre class="command-copy" 202 style=" display: block; 203 background: #eee; 204 white-space: pre; 205 -webkit-overflow-scrolling: touch; 206 max-width: 100%; 207 min-width: 100px; 208 border-radius: 20px; 209 padding: 0;;"> 210 211 @article{elata2023nested, 212 title={Nested Diffusion Processes for Anytime Image Generation}, 213 author={Elata, Noam and Kawar, Bahjat and Michaeli, Tomer and Elad, Michael}, 214 journal={arXiv preprint arXiv:2305.19066}, 215 year={2023} 216 } 217 </pre> 218 </tbody> 219 </table> 220 <!-- end bibtex citation --> 221 222 <div class="container"> 223 <p class="text-left"> 224 Our official code implementation can be found in the official <a href="https://github.com/noamelata/NestedDiffusion" target="_blank" rel="noopener noreferrer">github repository</a>. 225 </br> Nested Diffusion may be used online on the official <a href="https://huggingface.co/spaces
225/noamelata/Nested-Diffusion" target="_blank" rel="noopener noreferrer">HuggingFace space</a>. 226 </p> 227 </div> 228 229 <center> 230 <table class="table" width="1" style="max-width: 400px;"> 231 <tbody> 232 <tr> 233 <td width="50%" style="text-align: center;"> 234 <a href="https://huggingface.co/spaces/noamelata/Nested-Diffusion" target="_blank" rel="noopener noreferrer"> 235 <img alt="" id="github_logo" class="round" width="100%" style="max-width:150px" src="./resources/huggingface_logo.png"/> 236 </a> 237 </td> 238 <td width="50%" style="text-align: center;"> 239 <a href="https://github.com/noamelata/NestedDiffusion" target="_blank" rel="noopener noreferrer"> 240 <img alt="" id="github_logo" class="round" width="100%" style="max-width:150px" src="./resources/github_logo.png"/> 241 </a> 242 </td> 243 </tr> 244 <tr> 245 <td style="font-size:14pt; text-align: center;"> 246 <a target="_blank" rel="noopener noreferrer" href='https://huggingface.co/spaces/noamelata/Nested-Diffusion'>[Space]</a> 247 </td> 248 <td style="font-size:14pt; text-align: center;"> 249 <a target="_blank" rel="noopener noreferrer" href='https://github.com/noamelata/NestedDiffusion'>[Code]</a> 250 </td> 251 </tr> 252 </tbody> 253 </table> 254 </center> 255 256 <hr> 257 <br> 258 259 <!-- acknowledgement --> 260 <div class="container"> 261 <center><h3>Acknowledgements</h3></center> 262 <p style="font-size: 11pt;"> 263 This webpage is based on a <a target="_blank" rel="noopener noreferrer" href="https://github.com/matankleiner/academic-webpage-template/tree/main">template</a> made by <a href="https://www.linkedin.com/in/matan-kleiner/" target="_blank" rel="noopener noreferrer">Matan Kleiner</a> with the 264 help of <a target="_blank" rel="noopener noreferrer" href="https://www.linkedin.com/in/hilamanor/">Hila Manor</a>. 265 </p> 266 </div> 267 <br> 268 269</div> 270
271<script src="functionality.js"></script>
vendor: 1 bytes, line 271
271
272<script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script>
vendor: 1 bytes, line 272
272
273<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script>
vendor: 1 bytes, line 273
273
274<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script>
274 275</body> 276</html> 277
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.