1<!DOCTYPE html> 2<html> 3 4<head> 5 <meta charset="utf-8"> 6 <meta name="description" content="A Large-scale Knowledge Repository for Understanding Hand-Object Interaction"> 7 <meta name="keywords" content="Hand-Object Interaction, Dataset"> 8 <meta name="viewport" content="width=device-width, initial-scale=1"> 9 <title>OakInk Dataset CVPR2022 | æç©äº¤äºæ°æ®åº</title> 10 11 <!-- Global site tag (gtag.js) - Google Analytics -->
vendor: 66 bytes, lines 11-12
11 12 <script async src="https://www.googletagmanager.com/gtag/js?id=
12G-PYVRSFMDRL
vendor: 14 bytes, lines 12-13
12"></script> 13
13<script> 14
vendor: 155 bytes, lines 14-22
14window.dataLayer = window.dataLayer || []; 15 16 function gtag() { 17 dataLayer.push(arguments); 18 } 19 20 gtag('js', new Date()); 21 22 gtag('config', '
22G-PYVRSFMDRL
vendor: 6 bytes, lines 22-23
22'); 23
23</script>
23 24 25 <link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro" rel="stylesheet"> 26 <link rel="icon" type="image/png" href="img/oakink_icon.png"> 27 <link rel="stylesheet" href="./static/css/bulma.min.css"> 28 <link rel="stylesheet" href="./static/css/bulma-carousel.min.css"> 29 <link rel="stylesheet" href="./static/css/bulma-slider.min.css"> 30 <link rel="stylesheet" href="./static/css/fontawesome.all.min.css"> 31 <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css"> 32 <link rel="stylesheet" href="./static/css/index.css"> 33 34
34<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
34 35
35<script defer src="./static/js/fontawesome.all.min.js"></script>
35 36
36<script src="./static/js/bulma-carousel.min.js"></script>
36 37
37<script src="./static/js/bulma-slider.min.js"></script>
37 38
38<script src="./static/js/index.js"></script>
38 39</head> 40 41<body> 42 43 <nav class="navbar" role="navigation" aria-label="main navigation"> 44 <div class="navbar-brand"> 45 <a role="button" class="navbar-burger" aria-label="menu" aria-expanded="false"> 46 <span aria-hidden="true"></span> 47 <span aria-hidden="true"></span> 48 <span aria-hidden="true"></span> 49 </a> 50 </div> 51 <div class="navbar-menu"> 52 <div class="navbar-start" style="flex-grow: 1; justify-content: center;"> 53 <a class="navbar-item" href="https://github.com/oakink/OakInk"> 54 <span class="icon"> 55 <i class="fab fa-github"></i> 56 </span> 57 </a> 58 59 <div class="navbar-item has-dropdown is-hoverable"> 60 <a class="navbar-link"> 61 Related Projects 62 </a> 63 <div class="navbar-dropdown"> 64 <a class="navbar-item" href="https://kailinli.github.io/SemGrasp"> 65 SemGrasp: Language guided grasp generation (ECCV24) 66 </a> 67 <a class="navbar-item" href="https://oakink.net/v2"> 68 OakInk2 (CVPR2024) 69 </a> 70 <a class="navbar-item" href="#"> 71 CHORD: Category-level object recovery (ICCV23) 72 </a> 73 <a class="navbar-item" href="https://github.com/lixiny/POEM"> 74 POEM: Multiview mesh recovery (CVPR23) 75 </a> 76 <a class="navbar-item" href="https://github.com/oakink/OakInk-HMR"> 77 OakInk-HMR 78 </a> 79 <a class="navbar-item" href="https://github.com/oakink/OakInk-Grasp-Generation"> 80 OakInk-Grasp-Generation 81 </a> 82 <a class="navbar-item" href="https://github.com/oakink/Tink"> 83 Tink: Interaction Transfer 84 </a> 85 86 </div> 87 </div> 88 </div> 89 90 </div> 91 </nav> 92 93 94 <section class="hero"> 95 <div class="hero-body"> 96 <div class="container is-max-desktop"> 97 <div class="columns is-centered"> 98 <div class="column has-text-centered"> 99 <h1 class="post-title"> 100 <img class="left" src="img/oakinklogo.png" width="280px" height="auto" alt="OakInk_logo.png"> 101 </h1><br /> 102 <h1 class="title is-1 publication-title">OakInk: A Large-scale Knowledge Repository for Understanding 103 Hand-Object Interaction</h1> 104 <div class="is-size-5 publication-authors"> 105 <span class="author-block"> 106 <a href="https://lixiny.github.io">Lixin Yang</a><sup>1,2,*</sup>,</span> 107 <span class="author-block"> 108 <a href="https://kailinli.top/">Kailin Li</a><sup>1,*</sup>,</span> 109 <span class="author-block"> 110 <a href="https://scholar.google.com/citations?user=WurpqEMAAAAJ&hl=en">Xinyu 111 Zhan</a><sup>1,*</sup>, 112 </span> 113 <span class="author-block"> 114 <a href="#">Fei Wu</a><sup>1</sup>, 115 </span> 116 <span class="author-block"> 117 <a href="https://anran-xu.github.io">Anran Xu</a><sup>1</sup>, 118 </span> 119 <span class="author-block"> 120 <a href="https://scholar.google.com/citations?user=-_aPWUIAAAAJ&hl=en">Liu Liu</a><sup>1</sup>, 121 </span> 122 <span class="author-block"> 123 <a href="http://mvig.org">Cewu Lu</a><sup>1,2,✉</sup> 124 </span> 125 </div> 126 127 <div class="is-size-5 publication-authors"> 128 <span class="author-block"><sup>1</sup>Shanghai Jiao Tong University,</span> 129 <span class="author-block"><sup>2</sup>Shanghai Qi Zhi Institute</span> 130 </div> 131 132 * Equal contribution ✉ Corresponding author 133 134 <div style="font-size:30px; font-weight: bold;"> 135 CVPR 2022 136 </div> 137 138 <div class="column has-text-centered"> 139 <div class="publication-links"> 140 <!-- PDF Link. --> 141 <span class="link-block"> 142 <a href="https://openaccess.thecvf.com/content/CVPR2022/html/Yang_OakInk_A_Large-Scale_Knowledge_Repository_for_Understanding_Hand-Object_Interaction_CVPR_2022_paper.html" 143 class="external-link button is-normal is-rounded is-dark">
144 <span class="icon"> 145 <i class="fas fa-file-pdf"></i> 146 </span> 147 <span>Paper</span> 148 </a> 149 </span> 150 <!-- Code Link. --> 151 <span class="link-block"> 152 <a href="https://github.com/oakink/OakInk" class="external-link button is-normal is-rounded is-dark"> 153 <span class="icon"> 154 <i class="fab fa-github"></i> 155 </span> 156 <span>Code</span> 157 </a> 158 </span> 159 <span class="link-block"> 160 <a href="https://arxiv.org/abs/2203.15709" class="external-link button is-normal is-rounded is-dark"> 161 <span class="icon"> 162 <i class="ai ai-arxiv"></i> 163 </span> 164 <span>arXiv</span> 165 </a> 166 </span> 167 </div> 168 169 </div> 170 </div> 171 </div> 172 </div> 173 </div> 174 </section> 175 176 <section class="hero teaser"> 177 <div class="container is-max-desktop"> 178 <div class="hero-body"> 179 <img id="teaser" src="img/teaser.png" class="img-responsive" alt="overview" width="100%" /> 180 <h2 class="subtitle has-text-centered"> 181 OakInk consists of 1) <strong>OakBase</strong> of object affordance and 2) <strong>InkBase</strong> of hand's 182 interaction. 183 </h2> 184 </div> 185 </div> 186 </section> 187 188 <section class="section"> 189 <div class="container is-max-desktop"> 190 <div class="column is-full-width"> 191 <h3 class="title is-3">Update</h3> 192 <ul> 193 <li> 194 <b><tt>Jun,19,2024</tt></b>:   <a href="https://oakink.net/v2">OakInk2 @ CVPR2024</a> is released ! 195 </li> 196 <li> 197 <b><tt>Apr,04,2024</tt></b>:   OakInk could be easily cloned from Huggingface/Dataset at <a 198 href="https://huggingface.co/datasets/oakink/OakInk-v1">OakInk-v1</a> 199 </li> 200 <li> 201 <b><tt>Dec,11,2023</tt></b>:   <a href="https://github.com/oakink/OakInk-Grasp-Generation"> 202 <strong>Grasp Generation 203 </strong></a> models on OakInk-Shape are released! 204 </li> 205 <li> 206 <b><tt>Feb,11,2023</tt></b>:   <b>OakBase</b> is released! 207 </li> 208 209 <li> 210 <b><tt>Jan,03,2023</tt></b>:   <a href="https://github.com/oakink/OakInk-HMR"> <strong>Hand 211 Mesh Recovery 212 </strong></a> models on OakInk-Image are released! 213 </li> 214 215 <li> <b><tt>Oct,18,2022</tt></b>:   OakInk <b>public v2.1</b> is released! <br /> 216
216<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
216 217 <details> 218 <summary>expand to see details</summary> 219 Within this update, several artifacts have been fixed, including: wrong poses, time delay, and contact 220 surface mismatches; 221 <b>NOTE</b>: 222 If you downloaded the OakInk dataset before <u>11:00 AM October 18, 2022, UTC</u>, 223 You <b>only</b> need to replace the previous anno.zip by this newly released: <a 224 href="https://forms.gle/g6QEmmCeZYLGaVe29" target="_blank"><span class="icon"> <i 225 class="fas fa-download"></i> 226 </span>anno_v2.1.zip</a> <i style="color:gray;">(access via Google Forms)</i>, 227 unzip it and keep the same file structures as before, and install the latest 228 <a href="https://github.com/lixiny/OakInk"> <b>OakInk Toolkit</b></a>. 229 </details> 230 </li> 231 <li> 232 <b><tt>Jul,26,2022</tt></b>:   <a href="https://github.com/KailinLi/Tink"> <strong>Tink 233 </strong></a> has been made public. 234 </li> 235 <li> 236 <b><tt>Jun,28,2022</tt></b>:   OakInk <b>public v2</b> & <a href="https://github.com/lixiny/OakInk"> 237 <b>OakInk Toolkit</b></a> -- a Python dataloader, are released! 238 </li> 239 <li> 240 <b><tt>Mar,03,2022</tt></b>:   OakInk 241 got accepted by <a href="https://cvpr2022.thecvf.com"><strong>CVPR 2022</strong></a>. 242 </li> 243 </ul> 244 245 <!-- åå§å Foundation JS --> 246
246<script> 247 $(document).ready(function () { 248 $(document).foundation(); 249 }) 250 </script>
250 251 </div> 252 </div> 253 </section> 254 <section class="section"> 255 <div class=" container is-max-desktop"> 256 <h3 class="title is-3">About</h3> 257 <div class="content has-text-justified"> 258 OakInk contains three datasets:<br /> 259 <ul> 260 <li> 261 <b>OakBase</b>: Object Affordance Knowledge (Oak) base, including objects' part-level segmentation and 262 attributes. 263 </li> 264 <li> 265 <b>OakInk-Image</b>: a video dataset with 3D hand-object pose and shape annotations. 266 </li> 267 <li> 268 <b>OakInk-Shape</b>: a 3D grasping pose dataset with hand and object mesh models. 269 </li> 270 </ul> 271 272 The OakInk-Image contains 230K image frames that capture a total of 12 subjects performing 273 up to 5 intent-oriented interactions with 100 objects from 32 categories. 274 The object poses were captured from a MoCap system, while the MANO hand poses are fitted from 2D keypoint 275 annotation. 276 Based on the hand-object poses from real-world human demonstrations, 277 we transfer the hand pose on the real-world object to the virtual objects with similar affordances through a 278 interaction transfer module: Tink. 279 280 All the real-world and transferred interactions constitute the geometry-based dataset: OakInk-Shape. 281 The OakInk-Shape contains 50K different hand-object poses and models. 282 </p> 283 </div> 284 </div> 285 </section> 286 287 288 <section class="section"> 289 <div class="container is-max-desktop"> 290 <h3 class="title is-3">Download</h3> 291 292 <div id="oakbase-div" class="content has-text-justified"> 293 <h4 class="title is-4">OakInk</h4> 294 Download at: <a href="https://huggingface.co/datasets/oakink/OakInk-v1" target="_blank"><span class="icon"> 295 <i class="fas fa-download"></i> </span>Hugging Face</a>. 296 <br> 297 For researchers in China, you can download OakInk from the alternative mirror: 298 <a href="https://pan.baidu.com/s/1H2hW1ZXEbNccOUzYX9cPkA" target="_blank"><span class="icon"> <i 299 class="fas fa-download"></i> 300 </span>ç¾åº¦äºç</a> (hrt9) 301 <br> 302 </div> 303 After download all the files, you need to complete the <a href="https://forms.gle/g6QEmmCeZYLGaVe29" 304 target="_blank"><span class="icon"> <i class="fas fa-file-alt"></i> 305 </span>Google Form</a> to get the annotation file. 306 <br> 307 Arrange all zip files into the directory, eg.<code>$OAKINK_DIR/zipped</code> as follow 308 <br> 309 <pre><code>$OAKINK_DIR/zipped 310 âââ OakBase.zip 311 âââ image 312 â âââ anno_v2.1.zip # access via Google Forms 313 â âââ obj.zip 314 â âââ stream_zipped 315 â âââ oakink_image_v2.z01 316 â âââ ... 317 â âââ oakink_image_v2.z10 318 â âââ oakink_image_v2.zip 319 âââ shape 320 âââ metaV2.zip 321 âââ OakInkObjectsV2.zip 322 âââ oakink_shape_v2.zip 323 âââ OakInkVirtualObjectsV2.zip</code></pre> 324 and follow the <a href="https://github.com/oakink/OakInk/blob/main/docs/datasets.md#download-full-oakink" 325 target="_blank"><span class="icon"> <i class="fab fa-github"></i> 326 </span>instruction</a> to verify checksums and unzip the files.
327 </section> 328 329 <section class="section"> 330 <div class="container is-max-desktop content"> 331 <h3 class="title is-3">Resources</h3> 332 Details of dataset annotations, please refer to <a 333 href="https://github.com/oakink/OakInk/blob/main/docs/datasets.md#data-documentation" target="_blank"><span 334 class="icon"> <i class="fab fa-github"></i></span>Data documentation</a>. 335 <br> 336 Details of dataset splits for various tasks, please refer to <a 337 href="https://github.com/oakink/OakInk/blob/main/docs/datasets.md#data-splitting" target="_blank"><span 338 class="icon"> <i class="fab fa-github"></i></span>Data splitting</a>. 339 <br> 340 Load OakBase and visualize object parts and attributes, please refer to <a 341 href="https://github.com/oakink/OakInk/blob/main/scripts/demo_oak_base.py" target="_blank"><span class="icon"> 342 <i class="fab fa-github"></i></span>demo_oak_base.py</a>. 343 <br> 344 Load OakInk-Image and OakInk-Shape for visualization, please refer to <a 345 href="https://github.com/oakink/OakInk#load-and-visualize" target="_blank"><span class="icon"> <i 346 class="fab fa-github"></i></span>Load and visualize</a>. 347 <br> 348 Train hand mesh recovery models on OakInk-Image, please refer to <a href="https://github.com/oakink/OakInk-HMR" 349 target="_blank"><span class="icon"> <i class="fab fa-github"></i></span>OakInk-HMR</a>. 350 <br> 351 Train grasp generation models on OakInk-Shape, please refer to <a 352 href="https://github.com/oakink/OakInk-Grasp-Generation" target="_blank"><span class="icon"> <i 353 class="fab fa-github"></i></span>OakInk-Grasp-Generation</a>. 354 </div> 355 </section> 356 357 <section class="section" id="BibTeX"> 358 <div class="container is-max-desktop content"> 359 <h3 class="title is-3">BibTeX</h3> 360 <pre><code>@InProceedings{YangCVPR2022OakInk, 361 author = {Yang, Lixin and Li, Kailin and Zhan, Xinyu and Wu, Fei and Xu, Anran and Liu, Liu and Lu, Cewu}, 362 title = {{OakInk}: A Large-Scale Knowledge Repository for Understanding Hand-Object Interaction}, 363 booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, 364 year = {2022}, 365}</code></pre> 366 </div> 367 </section> 368 369 <footer class="footer"> 370 <div class="container"> 371 <div class="content has-text-centered"> 372 <a class="icon-link" href="https://arxiv.org/abs/2203.15709"> 373 <i class="fas fa-file-pdf"></i> 374 </a> 375 <a class="icon-link" href="https://github.com/oakink/OakInk" class="external-link" disabled> 376 <i class="fab fa-github"></i> 377 </a> 378 <br> 379 website template from <a href="https://github.com/nerfies/nerfies.github.io">nerfies</a> 380 381 </div> 382 </div> 383 </footer> 384 385</body> 386 387</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.