1<!DOCTYPE html> 2<html class=" w-mod-ix"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><style>.wf-force-outline-none[tabindex="-1"]:focus{outline:none;}</style> 3 4 <style> 5 .center { 6 display:block; 7 margin-left: auto; 8 margin-right: auto; 9 } 10 ol { 11 list-style-type: upper-roman; 12 list-style-position: inside; 13 } 14 </style> 15 16 <!-- Google tag (gtag.js) --> 17
17<script async="" src="./page_files/js"></script>
17 18
18<script> 19
vendor: 140 bytes, lines 19-23
19window.dataLayer = window.dataLayer || []; 20 function gtag(){dataLayer.push(arguments);} 21 gtag('js', new Date()); 22 23 gtag('config', '
23G-XB3PR2Y1TQ
vendor: 8 bytes, lines 23-24
23'); 24
24</script>
24 25 26 27 <meta name="viewport" content="width=device-width, initial-scale=1.0, shrink-to-fit=no"> 28 <title>Pseudo Dataset Generation for Out-of-domain Multi-Camera View Recommendation</title> 29 <link rel="stylesheet" href="./page_files/bootstrap.min.css"> 30 <link href="./page_files/css" rel="stylesheet" type="text/css"> 31 <link rel="stylesheet" href="./page_files/Highlight-Clean.css"> 32 <link rel="stylesheet" href="./page_files/styles.css"> 33 34 <!-- <link rel="apple-touch-icon" sizes="180x180" href="https://dreamfusion3d.github.io/apple-touch-icon.png"> --> 35 <!-- <link rel="icon" type="image/png" sizes="32x32" href="https://dreamfusion3d.github.io/favicon-32x32.png"> --> 36 <!-- <link rel="icon" type="image/png" sizes="16x16" href="https://dreamfusion3d.github.io/favicon-16x16.png"> --> 37 <!-- <link rel="manifest" href="https://dreamfusion3d.github.io/site.webmanifest"> --> 38 39 <meta property="og:site_name" content="Do Pre-trained Models Benefit Equally in Continual Learning?"> 40 <meta property="og:type" content="video.other"> 41 <meta property="og:title" content="Do Pre-trained Models Benefit Equally in Continual Learning?"> 42 <meta property="og:description" content="Do Pre-trained Models Benefit Equally in Continual Learning?, 2023."> 43 <!-- <meta property="og:url" content="https://dreamfusion3d.github.io/"> --> 44 <!-- <meta property="og:image" content="https://dreamfusion3d.github.io/assets/images/dreamfusion_samples.png"> --> 45 46 <!-- <meta property="article:publisher" content="https://dreamfusion3d.github.io/"> --> 47 <!-- <meta name="twitter:card" content="summary_large_image"> --> 48 <!-- <meta name="twitter:title" content="Do Pre-trained Models Benefit Equally in Continual Learning?"> --> 49 <!-- <meta name="twitter:description" content="We combine neural rendering with a multi-modal text-to-2D image diffusion generative model to synthesize diverse 3D objects from text."> --> 50 <!-- <meta name="twitter:url" content="https://dreamfusion3d.github.io/"> --> 51 <!-- <meta name="twitter:image" content="https://dreamfusion3d.github.io/assets/images/dreamfusion_samples.png"> --> 52 <!-- <meta name="twitter:site" content="" /> --> 53 54
54<script src="./page_files/video_comparison.js"></script>
54 55
55<script type="module" src="./page_files/model-viewer.min.js"></script>
55 56</head> 57 58<body data-new-gr-c-s-check-loaded="14.1086.0" data-gr-ext-installed=""> 59 <div class="highlight-clean" style="padding-bottom: 10px;"> 60 <div class="container" style="max-width: 768px;"> 61 <h2 class="text-center">Pseudo Dataset Generation for Out-of-domain Multi-Camera View Recommendation</h2> 62 </div> 63 <div class="container" style="max-width: 768px;"> 64 <div class="row authors"> 65 <div class="col-sm-4"> 66 <h5 class="text-center"><a class="text-center" href="https://www.linkedin.com/in/eric-lee-7070a67b/">Kuan-Ying Lee</a></h5> 67 <h6 class="text-center"><a class="text-center" href="https://illinois.edu/">UIUC</a></h6> 68 </div> 69 <div class="col-sm-4"> 70 <h5 class="text-center"><a class="text-center" href="https://scholar.google.com/citations?user=jrGRNJUAAAAJ&hl=en">Qian Zhou</a></h5> 71 <h6 class="text-center"><a class="text-center" href="https://www.cityu.edu.hk/">City University of Hong Kong</a></h6> 72 </div> 73 <div class="col-sm-4"> 74 <h5 class="text-center"><a class="text-center" href="https://siebelschool.illinois.edu/about/people/faculty/klara">Klara Nahrstedt</a></h5> 75 <h6 class="text-center"><a class="text-center" href="https://illinois.edu/">UIUC</a></h6> 76 </div> 77 </div> 78 </div> 79 <div class="buttons" style="margin-bottom: 8px;"> 80 <a class="btn btn-light" role="button" href="https://arxiv.org/abs/2410.13585"> 81 <svg style="width:24px;height:24px;margin-left:-12px;margin-right:12px" viewBox="0 0 24 24"> 82 <path fill="currentColor" d="M16 0H8C6.9 0 6 .9 6 2V18C6 19.1 6.9 20 8 20H20C21.1 20 22 19.1 22 18V6L16 0M20 18H8V2H15V7H20V18M4 4V22H20V24H4C2.9 24 2 23.1 2 22V4H4M10 10V12H18V10H10M10 14V16H15V14H10Z"></path> 83 </svg>Paper 84 </a> 85 <a class="btn btn-light" role="button" href="https://eric11220.github.io/publication/VCIP24/"> 86 <svg style="visibility:hidden;width:0px;height:24px;margin-left:-12px;margin-right:12px" width="0px" height="24px" viewBox="0 0 375 531"> 87 <polygon stroke="#000000" points="0.5,0.866 459.5,265.87 0.5,530.874 "></polygon> 88 </svg>Project 89 </a> 90 <a class="btn btn-light" role="button" href="https://github.com/eric11220/Pseudo-Dataset-for-OOD-Multi-Camera-Editing"> 91 <svg style="visibility:hidden;width:0px;height:24px;margin-left:-12px;margin-right:12px" width="0px" height="24px" viewBox="0 0 375 531"> 92 <polygon stroke="#000000" points="0.5,0.866 459.5,265.87 0.5,530.874 "></polygon> 93 </svg>GitHub 94 </a> 95 </div> 96 </div> 97 <!-- <hr class="divider"> 98 <div class="container" style="max-width: 768px;"> 99 <h2><b>Teaser</b></h2> 100 <div> 101 <img style="display:inline-block;" width="100%%" src="images/teaser.png"> 102 </div> 103 </div> 104 --> 105 <hr class="divider"> 106 <div class="container" style="max-width: 768px;"> 107 <div class="row"> 108 <div class="col-md-12"> 109 <h2>Abstract</h2> 110 <p> 111 <!-- <strong> --> 112 Multi-camera systems are indispensable in movies, TV shows, and other media. Selecting the appropriate camera at every timestamp has a decisive impact on production quality and audience preferences. Learning-based view recommendation frameworks can assist professionals in decision-making. However, they often struggle outside of their training domains. The scarcity of labeled multi-camera view recommendation datasets exacerbates the issue. Based on the insight that many videos are edited from the original multi-camera videos, we propose transforming regular videos into pseudo-labeled multi-camera view recommendation datasets. Promisingly, by training the model on pseudo-labeled datasets stemming from videos in the target domain, we achieve a 68% relative improvement in the model's accuracy in the target domain and bridge the accuracy gap between in-domain and never-before-seen domains. 113 <!-- </strong> --> 114 </p> 115 </div> 116 </div> 117 </div> 118 <hr class="divider"> 119 <div class="container" style="max-width: 768px;"> 120 <div class="row" style="padding: 0px 15px 0px 15px;"> 121 <div class="col-md-12"> 122 <h4 class="text-center">Multi-camera recommendation model suffers out-of-domain.</h4> 123 </div> 124 <br><br> 125 <div> 126 <img src="images/teaser.png" alt="Multi-camera editing model performs poorly out-of-domain." style="max-width: 100%;"> 127 </div> 128 <h6 class="caption"> 129 a) A multi-camera recommendation model trained on a labeled multi-camera editing dataset of a <span style="color:rgb(79,173,91);"> particular domain </span> generalizes poorly to a <span style="color:rgb(234,51,35);"> never-before-seen domain </span> and the accuracy drops significantly. (b) Our proposed method leverages regular videos to generate <span style="color:rgb(125,193,182);">
129 pseudo-labeled datasets </span> for the target domain and improve themodelâs accuracy. 130 </h6> 131 </div> 132 </div> 133 <hr class="divider"> 134 <div class="container" style="max-width: 768px;"> 135 <div class="row"> 136 <div class="col-md-12"> 137 <h4 class="text-center">Lack of labeled multi-camera editing dataset in different domains.</h4> 138 </div> 139 <div> 140 <img class="center" src="images/lack_of_data.png" alt="Few labeled multi-camera editing data" style="max-width: 80%;"> 141 </div> 142 </div> 143 </div> 144 <hr class="divider"> 145 <div class="container" style="max-width: 768px;"> 146 <div class="row"> 147 <div class="col-md-12"> 148 <h4 class="text-center"><strong>Intuition</strong> -- Videos are edited from original multi-camera videos.</h4> 149 </div><br><br> 150 <div> 151 <img class="center" src="images/intuition.png" alt="Pseudo dataset generation intuition" style="max-width: 100%;"> 152 </div> 153 </div> 154 </div> 155 156 <hr class="divider"> 157 <div class="container" style="max-width: 768px;"> 158 <div class="row"> 159 <div class="col-md-12"> 160 <h4 class="text-center">Generate pseudo-labeled data from edited videos!</h4> 161 </div><br><br> 162 <div> 163 <img class="center" src="images/pseudo_dataset.png" alt="Pseudo dataset generation pipeline" style="max-width: 80%;"> 164 </div> 165 </div> 166 </div> 167 168 <hr class="divider"> 169 <div class="container" style="max-width: 768px;"> 170 <div class="row"> 171 <div class="col-md-12"> 172 <h4 class="text-center">Pseudo datasets bridge the domain gap.</h4> 173 </div><br><br> 174 <div> 175 <img class="center" src="images/comp_w_baseline.png" alt="Comparison without pseudo dataset" style="max-width: 100%;"> 176 <br> 177 </div> 178 <h6 class="caption"> 179 Multiple factors could compound and intensify the domain gap. 180 <span style="font-weight:bold;">(Left)</span> Without any labels and simply leveraging regular videos, 181 the proposed pseudo-labeled dataset outperforms the professional labeled dataset that is on a different domain. 182 <span style="font-weight:bold;">(Right)</span> Training the model on the pseudo dataset created from videos of the same domain (video scene and type) achieves the best performance. 183 </h6> 184 </div> 185 </div> 186 187 <hr class="divider"> 188 189 <!-- Citation --> 190 <div class="container" style="max-width: 768px;"> 191 <div class="row"> 192 <div class="col-md-12"> 193 <h2>Citation</h2> 194 <code style="font-size: 13px"> 195 @inproceedings{lee2024_multicam_recom,<br> 196 author = {Lee, Kuan-Ying and Zhou, Qian and Nahrstedt, Klara},<br> 197 title = {Pseudo Dataset Generation for Out-of-domain Multi-Camera View Recommendation},<br> 198 booktitle = {IEEE visual communications and image processing (VCIP)},<br> 199 year = {2024},<br> 200 } 201 </code> 202 </div> 203 </div> 204 </div> 205 206 <hr class="divider"> 207 <div class="container" style="max-width: 768px;"> 208 <h6 class="text-center">Acknowledgement -- website template adopted from <a class="text-center" href="https://jonbarron.info/">Jon Barron</a></h6> 209 </div> 210 211
211<script src="./page_files/polyfill.js"></script>
211 212
212<script src="./page_files/yall.js"></script>
212 213
213<script> 214 yall( 215 { 216 observeChanges: true 217 } 218 ); 219 </script>
219 220
220<script src="./page_files/scripts.js"></script>
220 221
221<script src="./page_files/jquery.min.js"></script>
221 222
222<script src="./page_files/bootstrap.bundle.min.js"></script>
222 223
223<script src="./page_files/webflow.fd002feec.js"></script>
223 224 <!-- Import the component --> 225 226 227</body><grammarly-desktop-integration data-grammarly-shadow-root="true"></grammarly-desktop-integration></html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.