1<!DOCTYPE html> 2<html> 3<head> 4 <meta charset="utf-8"> 5 <meta name="description" 6 content="N-Dimensional Gaussians for Fitting of High Dimensional 7 Functions"> 8 <meta name="keywords" content="Gaussian, Splatting, N-D Gaussian, NDG, N-D Gaussian Fitting"> 9 <meta name="viewport" content="width=device-width, initial-scale=1"> 10 <title>N-Dimensional Gaussians for Fitting of High Dimensional 11 Functions</title> 12 13 <!-- Global site tag (gtag.js) - Google Analytics -->
vendor: 66 bytes, lines 13-14
13 14 <script async src="https://www.googletagmanager.com/gtag/js?id=
14G-PYVRSFMDRL
vendor: 14 bytes, lines 14-15
14"></script> 15
15<script> 16
vendor: 155 bytes, lines 16-24
16window.dataLayer = window.dataLayer || []; 17 18 function gtag() { 19 dataLayer.push(arguments); 20 } 21 22 gtag('js', new Date()); 23 24 gtag('config', '
24G-PYVRSFMDRL
vendor: 6 bytes, lines 24-25
24'); 25
25</script>
25 26 27 <link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro" 28 rel="stylesheet"> 29 30 <link rel="stylesheet" href="./static/css/bulma.min.css"> 31 <link rel="stylesheet" href="./static/css/bulma-carousel.min.css"> 32 <link rel="stylesheet" href="./static/css/bulma-slider.min.css"> 33 <link rel="stylesheet" href="./static/css/fontawesome.all.min.css"> 34 <link rel="stylesheet" 35 href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css"> 36 <link rel="stylesheet" href="./static/css/index.css"> 37 <link rel="icon" href=""> <!-- TODO: change icon --> 38 39
39<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
39 40
40<script defer src="./static/js/fontawesome.all.min.js"></script>
40 41
41<script src="./static/js/bulma-carousel.min.js"></script>
41 42
42<script src="./static/js/bulma-slider.min.js"></script>
42 43
43<script src="./static/js/index.js"></script>
43 44</head> 45<body> 46 47 48<section class="hero"> 49 <div class="hero-body"> 50 <div class="container is-max-desktop"> 51 <div class="columns is-centered"> 52 <div class="column has-text-centered"> 53 <h1 class="title is-1 publication-title">N-Dimensional Gaussians for Fitting of High Dimensional 54 Functions</h1> 55 <h1 class="is-size-5 publication-authors"> SIGGRAPH 2024 (Conference Track)</h1> 56 <div class="is-size-5 publication-authors"> 57 <span class="author-block"> 58 <a href="https://sdiolatz.info">Stavros Diolatzis</a><sup>1</sup> </span> 59 <span class="author-block"> 60 <a href="https://alphanew.net/">Tobias Zirr</a><sup>1</sup> </span> 61 <span class="author-block"> 62 <a href="https://www.alexku.me/">Alexandr Kuznetsov</a><sup>1</sup> </span> 63 <span class="author-block"> 64 <a href="https://grgkopanas.github.io/">Georgios Kopanas</a><sup>2, 3</sup> </span> 65 <span class="author-block"> 66 <a href="http://kaplanyan.com/">Anton Kaplanyan</a><sup>1</sup></span> 67 </div> 68 69 <div class="is-size-5 publication-authors"> 70 <sup>1</sup> <a href="https://www.intel.com/content/www/us/en/developer/topic-technology/graphics-research/overview.html"><img style="width:15%; padding-right: 15px; padding-left:0px;" src="static/images/intel_logo.png"> </a> 71 <sup>2</sup> <a href="https://www.inria.fr/"><img style="width:20%; padding-right: 15px; padding-left:0px;" src="static/images/inria_logo.png"> </a> 72 <sup>3</sup> <a href="https://univ-cotedazur.eu/"><img style="width:22%; padding-right: 15px; padding-left:10px;" src="static/images/uca_logo.png"> </a> 73 </div> 74 75 <div class="column has-text-centered"> 76 <div class="publication-links"> 77 <!-- PDF Link. --> 78 <span class="link-block"> 79 <a href="./static/files/ndg-paper.pdf" 80 class="external-link button is-normal is-rounded is-dark"> 81 <span class="icon"> 82 <i class="fas fa-file-pdf"></i> 83 </span> 84 <span>Paper 84MB</span> 85 </a> 86 </span> 87 <span class="link-block"> 88 <a href="./static/files/ndg-paper-compressed.pdf" 89 class="external-link button is-normal is-rounded is-dark"> 90 <span class="icon"> 91 <i class="fas fa-file-pdf"></i> 92 </span> 93 <span>Paper 4MB</span> 94 </a> 95 </span> 96 <span class="link-block"> 97 <a href="./static/files/ndg-supp.pdf" 98 class="external-link button is-normal is-rounded is-dark"> 99 <span class="icon"> 100 <i class="fas fa-file-pdf"></i> 101 </span> 102 <span>Supplemental</span> 103 </a> 104 </span> 105 <!-- <span class="link-block"> 106 <a href="https://arxiv.org/abs/2011.12948" 107 class="external-link button is-normal is-rounded is-dark"> 108 <span class="icon"> 109 <i class="ai ai-arxiv"></i> 110 </span> 111 <span>arXiv</span> 112 </a> 113 </span> --> 114 <!-- Video Link. --> 115 <!-- Code Link. --> 116 <span class="link-block"> 117 <a href="https://github.com/intel/ngd-fitting" 118 class="external-link button is-normal is-rounded is-dark"> 119 <span class="icon"> 120 <i class="fab fa-github"></i> 121 </span> 122 <span>Code</span> 123 </a> 124 </span> 125 </div> 126 </div> 127 </div> 128 </div> 129 </div> 130 </div> 131</section> 132 133<section class="hero teaser"> 134 <div class="container is-max-desktop"> 135 <div class="hero-body"> 136 <img src="./static/images/teaser.png" 137 class="interpolation-image" 138 alt=""/> 139 140 <div class="content has-text-justified"> 141 Our method optimizes N-Dimensional Gaussians to approximate high dimensional anisotropic functions in a 142 few minutes. Our parameterization, culling and optimization-controlled refinement allows us to quickly estimate Gaussian 143 parameters to represent various complex functions. We show two applications: 144 <ul> 145 <li>10D+ Application (Top): Synthetic scenes for which we can render G-Buffers such as world position, albedo, roughness etc. can be shaded with global illumination through our 10D+ Gaussian mixture. Even though the Gaussians are evaluated on the surfaces their representation power can efficie
145ntly estimate the appearance of reflections and transmittance with correct parallax effects. Apart from the G-Buffers we support variability of moving objects and light sources as extra dimensions.</li> 146 <li> 6D Application (Bottom): Real world scenes with complex view dependent effects can be modeled efficiently through our 6 dimensional Gaussian mixture. The 6 dimensions of world position and view direction give the parameterization the same representation power for both diffuse and view dependent effects, reconstructing complex effects like the one through the magnifying glass.</li> 147 </ul> 148 </div> 149 </div> 150 </div> 151</section> 152 153 154<section class="hero is-light is-small"> 155 <div class="hero-body"> 156 <div class="container"> 157 <div id="results-carousel" class="carousel results-carousel"> 158 <div class="item item-cd"> 159 <video poster="" id="cd" autoplay controls muted loop height="100%"> 160 <source src="./static/videos/cd.mp4" 161 type="video/mp4"> 162 </video> 163 </div> 164 <div class="item item-bathroom"> 165 <video poster="" id="bathroom" autoplay controls muted loop height="100%"> 166 <source src="./static/videos/bathroom.mp4" 167 type="video/mp4"> 168 </video> 169 </div> 170 <div class="item item-aquarium"> 171 <video poster="" id="aquarium" autoplay controls muted loop height="100%"> 172 <source src="./static/videos/aquarium.mp4" 173 type="video/mp4"> 174 </video> 175 </div> 176 <div class="item item-salon"> 177 <video poster="" id="salon" autoplay controls muted loop height="100%"> 178 <source src="./static/videos/salon.mp4" 179 type="video/mp4"> 180 </video> 181 </div> 182 <div class="item item-tools"> 183 <video poster="" id="tools" autoplay controls muted loop height="100%"> 184 <source src="./static/videos/tools.mp4" 185 type="video/mp4"> 186 </video> 187 </div> 188 </div> 189 </div> 190 </div> 191</section> 192 193 194<section class="section"> 195 <div class="container is-max-desktop"> 196 <!-- Abstract. --> 197 <div class="columns is-centered has-text-centered"> 198 <div class="column is-four-fifths"> 199 <h2 class="title is-3">Abstract</h2> 200 <div class="content has-text-justified"> 201 <p> 202 In the wake of many new ML-inspired approaches for reconstructing and representing 203 high-quality 3D content, recent hybrid and 204 explicitly learned representations exhibit promising performance and quality characteristics. 205 However, their scaling to higher dimensions 206 is challenging, e.g. when accounting for dynamic content with respect 207 to additional parameters such as material properties, illumination, or time. 208 In this paper, we tackle these challenges for an explicit representations based on 209 Gaussian mixture models. With our solutions, we arrive at efficient fitting of 210 compact N-dimensional Gaussian mixtures and enable efficient evaluation at render time: 211 For fast fitting and evaluation, 212 we introduce a high-dimensional culling scheme 213 that efficiently bounds N-D Gaussians, inspired by Locality Sensitive Hashing. 214 For adaptive refinement yet compact representation, we introduce a 215 loss-adaptive density control scheme that incrementally guides the use of 216 additional capacity towards missing details. 217 With these tools we can for the first time represent complex appearance that 218 depends on many input dimensions beyond position or viewing angle within a compact, 219 explicit representation optimized in minutes and rendered in milliseconds. 220 </p> 221 </div> 222 </div> 223 </div> 224 <!--/ Abstract. --> 225 226 <!-- Paper video. --> 227 <!-- <div class="columns is-centered has-text-centered"> 228 <div class="column is-four-fifths"> 229 <h2 class="title is-3">Video</h2>
230 <div class="publication-video"> 231 <iframe src="" 232 frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe> 233 </div> 234 </div> 235 </div> --> 236 <!--/ Paper video. --> 237 </div> 238</section> 239 240<section class="section"> 241 <div class="container is-max-desktop"> 242 <h2 class="title is-3">Method</h2> 243 244 <img src="./static/images/overview.png" 245 class="overview" 246 alt=""/> 247 248 <div class="content has-text-justified"> 249 <p> 250 Our optimization receives a number of query points q of N dimensionality as input. For these given points we 251 estimate which Gaussians can be discarded safely through our N-Dimensional culling inspired by Locality Sensitive Hashing. With the remaining ones we evaluate 252 for each q our Gaussian mixture either in N dimensions for surface radiance fields or by first projecting the Gaussians to 3D. 253 Our optimization converges to high quality while it also controlling the introduction of new Gaussians via our Optimization Controlled Refinement. 254 </p> 255 </div> 256 257 <h3 class="title is-4">N-Dimensional Gaussians Culling</h3> 258 259 260 <div class="columns is-centered has-text-centered"> 261 <img src="./static/images/culling.png" 262 class="overview" 263 alt=""/> 264 </div> 265 266 <div class="content has-text-justified"> 267 <p> 268 Culling irrelevant Gaussians is challenging in higher dimensions. Inspired by Locality Sensitive Hashing where locality is estimated by projection to random vectors we project both our Gaussians and the query points on random vectors and discard Gaussians safely if they fall far away from the query points. This is a tunable culling process that doesn't require any data structure and it works as the Gaussians are changing during optimization. 269 </p> 270 </div> 271 272 273 <h3 class="title is-4">Optimization Controlled Refinement</h3> 274 275 <img src="./static/images/refinement.png" 276 class="overview" 277 alt=""/> 278 279 <div class="content has-text-justified"> 280 <p> 281 In higher dimensions the refinement process of the Gaussian mixture becomes much more challenging. With many dimensions to choose from which one should we use as a criterion for refinement? Which one should we split along? To avoid this we propose a refinement process controlled by the optimizer instead. Each main Gaussian (Blue) receives two low opacity/brightness child Gaussians (Green) that are linked with the parent through a hierarchical relationship (see paper). In this way the optimizer can choose to utilize them to add details. Once these Gaussians are used enough they become main Gaussians and have child Gaussians of their own. We can see in the training video how Green Gaussians are introduced to add details. 282 </p> 283 </div> 284 285 <video poster="" id="cd_comp" autoplay controls muted loop height="100%"> 286 <source src="./static/videos/refinement.mp4" 287 type="video/mp4"> 288 </video> 289 290 <div class="content has-text-justified"> 291 <p> 292 In this training every 300 iterations we introduce new child Gaussians (Green) to the main Gaussians (Blue). As the training progress the child Gaussians add more and more fine details to the 6D Gaussian mixture without a splitting or merging mechanism. 293 </p> 294 </div> 295 296 </div> 297</section> 298 299<section class="section"> 300 <div class="container is-max-desktop"> 301 <h2 class="title is-3">Comparison</h2> 302 303 <h3 class="title is-4">6D Gaussians</h3> 304 305 <video poster="" id="cd_comp" autoplay controls muted loop height="100%"> 306 <source src="./static/videos/cd_comp.mp4" 307 type="video/mp4"> 308 </video> 309 310 <img src="./static/images/splatting_comp.png" 311 class="splatting-comp" 312 alt=""/> 313 314 <h3 class="title is-4">10D+ Gaussians</h3> 315 316 <img src="./static/images/mitsuba.png" 317 class="mitsuba-comp" 318 alt=""/> 319 </div> 320</section> 321 322<!-- <section class="section" id="BibTeX"> 323 <div class="container is-max-desktop content"> 324 <h2 class="title">BibTeX</h2> 325 <pre><code>@article{park2021nerfies, 326 author = {Park, Keunhong and Sinha, Utkarsh and Barron, Jonathan T. and Bouaziz, Sofien and Goldman, Dan B and
326Seitz, Steven M. and Martin-Brualla, Ricardo}, 327 title = {Nerfies: Deformable Neural Radiance Fields}, 328 journal = {ICCV}, 329 year = {2021}, 330}</code></pre> 331 </div> 332</section> --> 333 334<section class="section"> 335 <div class="container is-max-desktop"> 336 <!-- Abstract. --> 337 <div class="columns is-centered has-text-centered"> 338 <div class="column is-four-fifths"> 339 <h2 class="title is-3">Acknowledgements</h2> 340 <div class="content has-text-justified"> 341 <p> 342 We would like to thank the anonymous referees for their valuable 343 comments and helpful suggestions. We also thank Laurent Belcour 344 and Sebastian Herholz for their valuable input and suggestions. 345 </p> 346 <p> 347 We thank the authors of <a href="https://github.com/nerfies/nerfies.github.io">Nerfies</a> for open sourcing the webpage templated which we used for this site. 348 </p> 349 </div> 350 </div> 351 </div> 352</section> 353 354</body> 355</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.