1<!DOCTYPE html> 2<html> 3<head> 4 <meta charset="utf-8"> 5 <meta name="description" 6 content="3D Equivariant Graph Implicit Functions"> 7 <meta name="keywords" content="Equivariance, Implicit Functions, Reconstruction"> 8 <meta name="viewport" content="width=device-width, initial-scale=1"> 9 <title>3D Equivariant Graph Implicit Functions</title> 10 11 12 13 <link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro" 14 rel="stylesheet"> 15 16 <link rel="stylesheet" href="./static/css/bulma.min.css"> 17 <link rel="stylesheet" href="./static/css/bulma-carousel.min.css"> 18 <link rel="stylesheet" href="./static/css/bulma-slider.min.css"> 19 <link rel="stylesheet" href="./static/css/fontawesome.all.min.css"> 20 <link rel="stylesheet" 21 href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css"> 22 <link rel="stylesheet" href="./static/css/index.css"> 23 <link rel="icon" href="./static/images/favicon.svg"> 24 25
25<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
25 26
26<script defer src="./static/js/fontawesome.all.min.js"></script>
26 27
27<script src="./static/js/bulma-carousel.min.js"></script>
27 28
28<script src="./static/js/bulma-slider.min.js"></script>
28 29
29<script src="./static/js/index.js"></script>
29 30</head> 31<body> 32 33 34<section class="hero"> 35 <div class="hero-body"> 36 <div class="container is-max-desktop"> 37 <div class="columns is-centered"> 38 <div class="column has-text-centered"> 39 <h1 class="title is-1 publication-title">3D Equivariant Graph Implicit Functions</h1> 40 <div class="is-size-5 publication-authors"> 41 <span class="author-block"> 42 <a href="https://yunlu-chen.github.io/">Yunlu Chen</a><sup>1</sup>,</span> 43 <span class="author-block"> 44 <a href="https://basurafernando.github.io/">Basura Fernando</a><sup>2</sup>,</span> 45 <span class="author-block"> 46 <a href="https://homepages.inf.ed.ac.uk/hbilen/">Hakan Bilen</a><sup>3</sup>, 47 </span> 48 <span class="author-block"> 49 <a href="https://niessnerlab.org/">Matthias NieÃner</a><sup>4</sup>, 50 </span> 51 <span class="author-block"> 52 <a href="https://www.egavves.com/">Efstratios Gavves</a><sup>1</sup>, 53 </span> 54 </div> 55 56 <div class="is-size-5 publication-authors"> 57 <span class="author-block"><sup>1</sup>University of Amsterdam,</span> 58 <span class="author-block"><sup>2</sup>A*STAR</span> 59 <span class="author-block"><sup>3</sup>University of Edinburgh</span> 60 <span class="author-block"><sup>4</sup>Technical University of Munich</span> 61 </div> 62 63 <div class="is-size-5 has-text-weight-semibold mt-3 publication-title"> 64 <span class="author-block">ECCV 2022</span> 65 </div> 66 67 <div class="column has-text-centered"> 68 <div class="publication-links"> 69 <!-- PDF Link. --> 70 <span class="link-block"> 71 <a href="https://arxiv.org/abs/2203.17178" 72 class="external-link button is-normal is-rounded is-dark"> 73 <span class="icon"> 74 <i class="fas fa-file-pdf"></i> 75 </span> 76 <span>Paper</span> 77 </a> 78 </span> 79 80 <!-- Video Link. --> 81 <span class="link-block"> 82 <a href="https://www.youtube.com/watch?v=MrKrnHhk8IA" 83 class="external-link button is-normal is-rounded is-dark"> 84 <span class="icon"> 85 <i class="fab fa-youtube"></i> 86 </span> 87 <span>Video</span> 88 </a> 89 </span> 90 <!-- Code Link. --> 91 <span class="link-block"> 92 <a href="https://github.com/yunlu-chen/equivariant-graph-implicit" 93 class="external-link button is-normal is-rounded is-dark"> 94 <span class="icon"> 95 <i class="fab fa-github"></i> 96 </span> 97 <span>Code</span> 98 </a> 99 </span> 100 101 </div> 102 103 </div> 104 </div> 105 </div> 106 </div> 107 </div> 108</section> 109 110<section class="hero teaser"> 111 <div class="container is-max-desktop"> 112 <div class="hero-body"> 113 <div class="publication-video"> 114 <iframe src="https://www.youtube.com/embed/41UPVdlXg94?rel=0&showinfo=0" 115 frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe> 116 </div> 117 <br><br> 118 <h2 class="subtitle has-text-centered"> 119 We embed 3D implicit neural representations in graphs to achieve high-fidelity equivariant 3D reconstruction. 120 </h2> 121 </div> 122 </div> 123</section> 124 125<section class="section"> 126 <div class="container is-max-desktop"> 127 128 <!-- Abstract. --> 129 <div class="columns is-centered has-text-centered"> 130 <div class="column is-four-fifths"> 131 <h2 class="title is-3">Abstract</h2> 132 <div class="content has-text-justified"> 133 <p> 134 In recent years, neural implicit representations have made remarkable progress in modeling of 3D shapes with arbitrary topology. In this work, we address two key limitations of such representations, in failing to capture local 3D geometric fine details, and to learn from and generalize to shapes with unseen 3D transformations. To this end, we introduce a novel family of graph implicit functions with equivariant layers that facilitates modeling fine local details and guaranteed robustness to various groups of geometric transformations, through local k-NN graph embeddings with sparse point set observations at multiple resolutions. Our method improves over the existing rotation-equivariant implicit function from 0.69 to 0.89 (IoU) on the ShapeNet reconstruction task. We also show that our equivariant implicit function can be extended to other types of similarity transformations and generalizes to unseen translations and scaling. 135 </p> 136 </div> 137 </div> 138 </div> 139 <!--/ Abstract. --> 140 141 142 </div> 143</section> 144 145 146 147<section class="section"> 148 <div class="container is-max-desktop"> 149 150 <div class="columns is-centered"> 151 <div class="column is-four-fifths"> 152 153 <div class="content"> 154 <center> 155 <h2 class="title is-3"> Method</h2> 156 </center> 157 158 <center><img width="75%" src="./resources/fig1.jpg"></center> 159 <p>Our equivariant graph implicit function infers the implicit field for a 3D shape, given a sparse point cloud observation. When a transformation (rotation, translation, or/and scaling) is applied to the observation, the resulting implicit field is guaranteed to be the same as applying a corresponding transformation to the inferred implicit field from the untransformed input (middle). The property of equivariance enables generalization to unseen transformations, under which existing models often struggle (right).</p> 160 161 <h2 class="title is-4">
161Graph-structured local implicit feature embedding</h2> 162 <center><img width="120%" src="./resources/fig2.jpg"></center> 163 <p>To achieve high fidelity 3D reconstruction in local details, we embed implicit function in local k-NN graphs. The architecture is robust to similarity geometric transformations, while existing local implicit embedding methods based on convolutional grid structure are sensitive to these transformations.</p> 164 165 <h2 class="title is-4">Equivariant graph convolution layers</h2> 166 <center><img width="70%" src="./resources/fig4_hybrid.jpg"></center> 167 <p>We incorporate equivariant layer design with hybrid scalar and vector features for graph convolution layers, which facilitates numeric robustness against geometric transformations. The equivariant mechanism was adapted from Vector Neurons [Deng et al. ICCV 2021] and EGNN [Satorras et al. ICML 2021]. </p> 168 </div> 169 170 </div> 171 </div> 172 173 </div> 174</section> 175 176 177 178<section class="section"> 179 <div class="container is-max-desktop"> 180 <div class="columns is-centered"> 181 <div class="column is-four-fifths"> 182 183 <div class="content"> 184 <center> 185 <h2 class="title is-3"> Results</h2> 186 </center> 187 <p>Our models are denoted <b>GraphONet</b>, the graph implicit function without equivariant layers, and <b>E-GraphONet</b>, with equivariant layers for graph convolutions.</p> 188 189 190 <h2 class="title is-5">Object reconstruction</h2> 191 <p>Aside from the merit of transform-robustness, our method benefits high-fidelity 3D reconstruction in general even for canonically oriented shapes. Our graph mechanism allows focusing on more critical areas with surface points.</p> 192 193 <video id="res1" autoplay="" muted="" loop="" playsinline="" height="100%" width="100%"> 194 <source src="./resources/res1.mp4" type="video/mp4"> 195 </video> 196 197 <h2 class="title is-5">Scene reconstruction</h2> 198 <p>The graph feature embedding mechanism is translation-equivariant, so our methods scale to scene-level reconstruction.</p> 199 <p>We show results on Synthetic Rooms and ScanNet.</p> 200 201 <video id="res2" autoplay="" muted="" loop="" playsinline="" height="100%" width="100%"> 202 <source src="./resources/res2.mp4" type="video/mp4"> 203 </video> 204 <br> 205 <video id="res3" autoplay="" muted="" loop="" playsinline="" height="100%" width="100%"> 206 <source src="./resources/res3.mp4" type="video/mp4"> 207 </video> 208 209 210 <h2 class="title is-5">Reconstruction of shapes under unseen transformations</h2> 211 <p>We evaluate 3D implicit surface reconstruction with under unseen similarity transformations, including rotation, translation and scaling.</p> 212 213 <p>Our GraphONet is more robust to transformations than other baseline non-equivariant methods. Standard implicit representation method with global latent embedding ONet fails to reconstruct local details and it cannot handle transformations. Compared to ConvONets the grid method with local feature embeddings, GraphONet generates significantly less artifacts, 214especially under rotation. 215</p> 216 <p>Our E-GraphONet with equivariant layers further guarantees generalization to all similarity transformations, while the baseline GraphONet fails under extreme scaling. Previous rotation-equivariant method VN-ONet considers only rotation and the global latent embedding limits detailed reconstruction.</p> 217 218 <video id="res4" autoplay="" muted="" loop="" playsinline="" height="100%" width="130%"> 219 <source src="./resources/res4.mp4" type="video/mp4"> 220 </video> 221 </div> 222 223 </div> 224 </div> 225 226 </div> 227</section> 228 229 230<section class="section" id="BibTeX"> 231 <div class="container is-max-desktop content"> 232 <h2 class="title">BibTeX</h2> 233 <pre><code>@article{chen2022equivariant, 234 author = {Chen, Yunlu and Fernando, Basura and Bilen, Hakan and Nie{\ss}ner, Matthias and Gavves, Efstratios},
235 title = {3D Equivariant Graph Implicit Functions}, 236 journal = {ECCV}, 237 year = {2022}, 238}</code></pre> 239 </div> 240</section> 241 242 243<footer class="footer"> 244 <div class="container"> 245 246 <div class="columns is-centered"> 247 <div class="column is-8"> 248 <div class="content"> 249 <p> 250 The website template was borrowed from <a 251 href="https://github.com/nerfies/nerfies.github.io">Nerfies</a>. 252 </p> 253 </div> 254 </div> 255 </div> 256 </div> 257</footer> 258 259</body> 260</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.