1<!DOCTYPE html> 2<html> 3 <head> 4 <meta charset="utf-8"> 5 <meta name="description" 6 content="UFORecon: Generalizable Sparse-View Surface Reconstruction from Arbitrary and UnFavOrable Sets"> 7 <meta name="keywords" 8 content="UFORecon, multi-view reconstruction"> 9 <meta name="viewport" content="width=device-width, initial-scale=1"> 10 <title>UFORecon: Generalizable Sparse-View Surface Reconstruction from Arbitrary and UnFavOrable Sets</title> 11 12 <!-- Global site tag (gtag.js) - Google Analytics -->
vendor: 68 bytes, lines 12-13
12 13 <script async src="https://www.googletagmanager.com/gtag/js?id=
13G-PYVRSFMDRL
vendor: 16 bytes, lines 13-14
13"></script> 14
14<script> 15
vendor: 155 bytes, lines 15-23
15window.dataLayer = window.dataLayer || []; 16 17 function gtag() { 18 dataLayer.push(arguments); 19 } 20 21 gtag('js', new Date()); 22 23 gtag('config', '
23G-PYVRSFMDRL
vendor: 6 bytes, lines 23-24
23'); 24
24</script>
24 25 26 <link 27 href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro" 28 rel="stylesheet"> 29 30 <link rel="stylesheet" href="./static/css/bulma.min.css"> 31 <link rel="stylesheet" href="./static/css/bulma-carousel.min.css"> 32 <link rel="stylesheet" href="./static/css/bulma-slider.min.css"> 33 <link rel="stylesheet" href="./static/css/fontawesome.all.min.css"> 34 <link rel="stylesheet" 35 href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css"> 36 <link rel="stylesheet" href="./static/css/index.css"> 37 38
38<script 39 src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
39 40
40<script defer src="./static/js/fontawesome.all.min.js"></script>
40 41
41<script src="./static/js/bulma-carousel.min.js"></script>
41 42
42<script src="./static/js/bulma-slider.min.js"></script>
42 43
43<script src="./static/js/index.js"></script>
43 44
44<script type="module" 45 src="https://ajax.googleapis.com/ajax/libs/model-viewer/3.1.1/model-viewer.min.js"></script>
45 46 </head> 47 48 <body> 49 <section class="hero"> 50 <div class="hero-body"> 51 <div class="container is-max-desktop"> 52 <div class="columns is-centered"> 53 <div class="column has-text-centered"> 54 <h1 class="title is-1 publication-title"><img src="static/images/free-icon-ufo-620689.png" width=60>UFORecon: Generalizable Sparse-View Surface Reconstruction from Arbitrary and UnFavOrable Sets</h1> 55 <h1 class="venue is-size-2">CVPR 2024</h1> 56 <div class="is-size-4 publication-authors"> 57 <!-- Paper authors --> 58 <span class="author-block"> 59 <a href="https://youngju-na.github.io/" target="_blank">Youngju Na</a><sup></sup>,</span> 60 <span class="author-block"> 61 <a href="https://wkim97.github.io/" target="_blank">Woo Jae Kim</a><sup></sup>,</span> 62 <span class="author-block"> 63 <a href="https://qbhan.oopy.io/" target="_blank">Kyu Beom Han</a><sup></sup>, 64 </span> 65 <span class="author-block"> 66 <a href="THIRD AUTHOR PERSONAL LINK" target="_blank">Suhyeon Ha</a><sup></sup>, 67 </span> 68 <span class="author-block"> 69 <a href="https://sgvr.kaist.ac.kr/~sungeui/" target="_blank">Sung-eui Yoon</a><sup></sup> 70 </span> 71 </div> 72 73 <div class="is-size-3 publication-authors"> 74 <span class="author-block"><sup></sup>KAIST<br></span> 75 <!-- <span class="corresponding"><small><br><sup>*</sup>Corresponding Author</small></span> --> 76 </div> 77 78 <div class="column has-text-centered"> 79 <div class="publication-links"> 80 <!-- Arxiv PDF link --> 81 <span class="link-block"> 82 <a href="https://arxiv.org/abs/2403.05086" target="_blank" 83 class="external-link button is-normal is-rounded is-dark"> 84 <span class="icon"> 85 <i class="fas fa-file-pdf"></i> 86 </span> 87 <span>arXiv</span> 88 </a> 89 </span> 90 91 92 <!-- Github link --> 93 <span class="link-block"> 94 <a href="https://github.com/Youngju-Na/UFORecon/" target="_blank" 95 class="external-link button is-normal is-rounded is-dark"> 96 <span class="icon"> 97 <i class="fab fa-github"></i> 98 </span> 99 <span>Code</span> 100 </a> 101 </span> 102 </a> 103 </span> 104 </div> 105 </div> 106 </div> 107 </div> 108 </div> 109 </div> 110</section> 111<section class="section"> 112 <div class="container is-max-desktop"> 113 <!-- Abstract. --> 114 <div class="columns is-centered has-text-centered"> 115 <div class="column is-full_width"> 116 117 <!-- <h2 class="title is-2">Introduction</h2> 118 <p align="center"> 119 <table> 120 <tr> 121 <th align="center">VolRecon</th> 122 <th align="center">Ours</th> 123 </tr> 124 <tr> 125 <td align="center"><img src="static/videos/scan65_volrecon.gif" alt="VolRecon" width="800"/></td> 126 <td align="center"><img src="static/videos/scan65_random.gif" alt="Ours(Random)" width="800"/></td> 127 </tr> 128 </table> 129 </p> --> 130 131 132 <h2 class="title is-2">Overview</h2> 133 <img src="static/images/pipeline.png" class="center" /> 134 <br> 135 <h2 class="title is-2">Abstract</h2> 136 <div class="content has-text-justified"> 137 <p> 138 Generalizable neural implicit surface reconstruction aims to obtain an accurate underlying geometry given a limited number of multi-view images from unseen scenes. 139 However, existing methods select only informative and relevant views using predefined scores for training and testing phases. 140 This constraint renders the model impractical in real-world scenarios, where the availability of favorable combinations cannot al
140ways be ensured. 141 We introduce and validate a view-combination score to indicate the effectiveness of the input view combination. 142 We observe that previous methods output degenerate solutions under arbitrary and unfavorable sets. 143 Building upon this finding, we propose UFORecon, a robust view-combination generalizable surface reconstruction framework. 144 To achieve this, we apply cross-view matching transformers to model interactions between source images and build correlation frustums to capture global correlations. 145 Additionally, we explicitly encode pairwise feature similarities as view-consistent priors. Our proposed framework significantly outperforms previous methods in terms of view-combination generalizability and also in the conventional generalizable protocol trained with favorable view-combinations. 146 The code is available at this <a href="https://github.com/Youngju-Na/UFORecon/" target="_blank">link</a>. 147 </p> 148 149 </div> 150 </div> 151 </div> 152 <!--/ Abstract. --> 153 154 <!-- Paper video. --> 155 <!-- <div class="columns is-centered has-text-centered"> 156 <div class="column is-full_width"> 157 <h3 class="title is-3">Reconstruction Results</h2> 158 <video id="dollyzoom", autoplay muted loop controls 159 height="100%"> 160 <source src="static/videos/uforecon_best/favorable_40.mp4" 161 type="video/mp4"> 162 </video> 163 </div> 164 </div> --> 165 166 <!-- <table width="110%"> 167 <tr> 168 <td align="left" valign="top" width="50%"> 169 <div class="video-compare-container" display:block> 170 <video class="video" id="toaster_mesh" loop playsinline autoPlay muted src="test.mp4" onplay="resizeAndPlay(this)"></video> 171 <canvas height=0 class="videoMerge" id="toaster_meshMerge"></canvas> 172 </div> 173 </td> 174 <td align="left" valign="top" width="50%"> 175 <div class="video-compare-container" display:block> 176 <video class="video" id="toaster_normals" loop playsinline autoPlay muted src="test.mp4" onplay="resizeAndPlay(this)"></video> 177 <canvas height=0 class="videoMerge" id="toaster_normalsMerge"></canvas> 178 </div> 179 </td> 180 </tr> 181 </table> --> 182 183 <!--/ Paper video. --> 184 <br> 185 <div class="columns is-centered has-text-centered"> 186 <h3 class="title is-2">Results</h2> 187 </div> 188 <div class="tabs is-toggle is-fullwidth"> 189 <ul id ="scan_selection"> 190 </ul> 191 </div> 192 193 <div id = "scan_content" class="has-text-centered"> 194 <!--/ Scan 24 --> 195 <div class="tabcontent" style="display: block;" scan="40"> 196 <!-- source views. --> 197 <h2 class="title is-5">Source views</h2> 198 <div class="columns is-centered has-text-centered" width="50%" > 199 <div class="column" width="20%"></div> 200 <div class="column"> 201 <img src="static/images/source_views/40/1.png" /> 202 </div> 203 <div class="column"> 204 <img src="static/images/source_views/40/2.png" /> 205 </div> 206 <div class="column"> 207 <img src="static/images/source_views/40/3.png" /> 208 </div> 209 </div> 210 211 <!-- Paper video. --> 212 <div class="columns is-centered has-text-centered"> 213 <div class="column is-full_width"> 214 <h3 class="title is-3">Reconstruction Results</h2> 215 <video id="uforecon", autoplay muted loop controls 216 height="100%"> 217 <source src="static/videos/uforecon_best/favorable_40.mp4" 218 type="video/mp4"> 219 </video> 220 <video id="volrecon", autoplay muted loop controls 221 height="100%"> 222 <source src="static/videos/volrecon/favorable_40.mp4" 223 type="video/mp4"> 224 </video> 225 </div> 226 </div> 227 228 <!-- source views. --> 229 <!-- 3D model. --> 230 <div class="columns is-centered has-text-centered"> 231 <div class="column is-full_width"> 232 <h2 class="title is-3">Reconstructed 3D model</h2> 233 <model-viewer id="transform" orientation="90deg 90deg 90deg" shadow-intensity="1" camera-controls touch-action="pan-y" ar 234 src="https://github.com/Youngju-Na/UFORecon-project-page/blob/master/24.glb" alt="A 3D model of an astronaut"> 235 <div class="controls"> 236 <div>Roll: <input id="roll" value="180" size="3" class="number"> degrees</div> 237 <div>Pitch: <input id="pitch" value="190" size="3" class="number"> degrees</div> 238 <div>Yaw: <input id="yaw" value="190" size="3" class="number"> degrees</div> 239 <div> 240 Scale: X: <input id="x" value="1" size="3" class="number">, 241 Y: <input id="y" value="1" size="3" class="number">, 242 Z: <input id="z" value="1" size="3" class="number"> 243 </div> 244 <button id="frame">Update Framing</button> 245 </div> 246 </model-viewer> 247
247<script> 248 const modelViewerTransform = document.querySelector("model-viewer#transform"); 249 const roll = document.querySelector('#roll'); 250 const pitch = document.querySelector('#pitch'); 251 const yaw = document.querySelector('#yaw'); 252 const x = document.querySelector('#x'); 253 const y = document.querySelector('#y'); 254 const z = document.querySelector('#z'); 255 const frame = document.querySelector('#frame'); 256 257 frame.addEventListener('click', () => { 258 modelViewerTransform.updateFraming(); 259 }); 260 261 const updateOrientation = () => { 262 modelViewerTransform.orientation = `${roll.value}deg ${pitch.value}deg ${yaw.value}deg`; 263 }; 264 265 const updateScale = () => { 266 modelViewerTransform.scale = `${x.value} ${y.value} ${z.value}`; 267 }; 268 269 roll.addEventListener('input', () => { 270 updateOrientation(); 271 }); 272 pitch.addEventListener('input', () => { 273 updateOrientation(); 274 }); 275 yaw.addEventListener('input', () => { 276 updateOrientation(); 277 }); 278 x.addEventListener('input', () => { 279 updateScale(); 280 }); 281 y.addEventListener('input', () => { 282 updateScale(); 283 }); 284 z.addEventListener('input', () => { 285 updateScale(); 286 }); 287 </script>
287 288
288<script> 289 const modelViewerParameters = document.querySelector("model-viewer#sphere"); 290 291 modelViewerParameters.addEventListener("load", (ev) => { 292 293 let material = modelViewerParameters.model.materials[0]; 294 console.log(material); 295 296 let metalnessDisplay = document.querySelector("#metalness-value"); 297 let roughnessDisplay = document.querySelector("#roughness-value"); 298 299 metalnessDisplay.textContent = material.pbrMetallicRoughness.metallicFactor; 300 roughnessDisplay.textContent = material.pbrMetallicRoughness.roughnessFactor; 301 302 // Defaults to gold 303 //material.pbrMetallicRoughness.setBaseColorFactor([0.7294, 0.5333, 0.0392]); 304 305 document.querySelector('#metalness').addEventListener('input', (event) => { 306 material.pbrMetallicRoughness.setMetallicFactor(event.target.value); 307 metalnessDisplay.textContent = event.target.value; 308 console.log(event.target.value); 309 }); 310 311 document.querySelector('#roughness').addEventListener('input', (event) => { 312 material.pbrMetallicRoughness.setRoughnessFactor(event.target.value); 313 roughnessDisplay.textContent = event.target.value; 314 }); 315 }); 316 </script>
316 317 </div> 318 </div> 319 320 321 322 <!--/ 3D model. --> 323 <!--/ Renderings --> 324 <h2 class="title is-3"></h2> 325 <table> 326 <tbody> 327 <tr> 328 <td align="left" valign="top" width="50%"> 329 <video class="video" preload="auto" id="water5" loop="" playsinline="" autoplay="" muted="" 330 src="static/videos/uforecon_best/favorable_40.mp4" onplay="resizeAndPlay(this)" style="height: 100px;"> 331 </video> 332 <canvas class="videoMerge" id="40Merge"></canvas> 333 </td> 334 </tr> 335 </tbody> 336 </table> 337 <!--/ Renderings --> 338 </div> 339 <!--/ Scan 24 --> 340 </div> 341 </div> 342 343 344 345</section> 346 347<section class="section" id="BibTeX"> 348<div class="container is-max-desktop content has-text-centered"> 349 <!-- <h2 class="title">Poster</h2> --> 350 <h2 class="title is-3" >Poster</h2> 351 <!-- <img src="static/images/concept.png" class="center" /> --> 352 </iframe> 353</div> 354 355</section> 356<section class="section" id="BibTeX"> 357 <div class="container is-max-desktop content"> 358 <h2 class="title">BibTeX</h2> 359 <pre><code>@article{na2024uforecon, 360author = {Youngju Na, Woo Jae Kim, Kyu Beom Han, Suhyeon Ha, Sung-eui Yoon}, 361title = {UFORecon: Generalizable Sparse-View Surface Reconstruction from Arbitrary and UnFavOrable Sets}, 362journal = {arXiv}, 363year = {2024}, 364}</code></pre> 365 </div> 366</section> 367 368 </body> 369 </html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.