PageSourceSearch

https://koi953215.github.io/NaRCan_page/

html koi953215.github.io collected 2026-10-03 09:28:16 UTC 40,875 bytes, 681 lines download raw bytes

1<!DOCTYPE html>
2<html>
3
4<head lang="en">
5    <meta charset="UTF-8">
6    <meta http-equiv="x-ua-compatible" content="ie=edge">
7
8    <title>NaRCan</title>
9
10    <meta name="description" content="">
11    <meta name="viewport" content="width=device-width, initial-scale=1">
12
13    <meta property="og:image" content="https://koi953215.github.io/NaRCan_page/img/overview_combined.jpg">
14    <meta property="og:image:type" content="image/png">
15    <meta property="og:image:width" content="1296">
16    <meta property="og:image:height" content="840">
17    <meta property="og:type" content="website" />
18    <meta property="og:url" content="https://koi953215.github.io/NaRCan_page/"/>
19    <meta property="og:title" content="NaRCan: Natural Refined Canonical Image with Integration of Diffusion Prior for Video Editing" />
20    <meta property="og:description" content="We propose a video editing framework, NaRCan, which integrates a hybrid deformation field and a diffusion prior to generating high-quality natural canonical images to represent the input video. Our approach utilizes homography to model global motion and employs multi-layer perceptrons (MLPs) to capture local residual deformations, enhancing the model's ability to handle complex video dynamics. By introducing a diffusion prior from the early stages of training, our model ensures that the generated images retain a high-quality natural appearance, making the produced canonical images suitable for various downstream tasks in video editing, a capability not achieved by current canonical-based methods. Furthermore, we incorporate low-rank adaptation (LoRA) fine-tuning and introduce a noise and diffusion prior update scheduling technique that accelerates the training process by 14 times. Extensive experimental results show that our method outperforms existing approaches in various video editing tasks and produces coherent and high-quality edited video sequences." />
21
22    <meta name="twitter:card" content="summary_large_image" />
23    <meta name="twitter:title" content="NaRCan: Natural Refined Canonical Image with Integration of Diffusion Prior for Video Editing" />
24    <meta name="twitter:description" content="We propose a video editing framework, NaRCan, which integrates a hybrid deformation field and a diffusion prior to generating high-quality natural canonical images to represent the input video. Our approach utilizes homography to model global motion and employs multi-layer perceptrons (MLPs) to capture local residual deformations, enhancing the model's ability to handle complex video dynamics. By introducing a diffusion prior from the early stages of training, our model ensures that the generated images retain a high-quality natural appearance, making the produced canonical images suitable for various downstream tasks in video editing, a capability not achieved by current canonical-based methods. Furthermore, we incorporate low-rank adaptation (LoRA) fine-tuning and introduce a noise and diffusion prior update scheduling technique that accelerates the training process by 14 times. Extensive experimental results show that our method outperforms existing approaches in various video editing tasks and produces coherent and high-quality edited video sequences." />
25    <meta name="twitter:image" content="https://koi953215.github.io/NaRCan_page/img/overview_combined.jpg" />
26
27
28<link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>💊</text></svg>">
29
30    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
31    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.4.0/css/font-awesome.min.css">
32    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.8.0/codemirror.min.css">
33    <link rel="stylesheet" href="css/app.css">
34    <link rel="stylesheet" href="css/fontawesome.all.min.css">
35    <link rel="stylesheet"
36        href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
37
38    <link rel="stylesheet" href="css/bootstrap.min.css">
39
40    <!-- Google tag (gtag.js) -->
vendor: 66 bytes, lines 40-41
40
41  <script async src="https://www.googletagmanager.com/gtag/js?id=
41G-8ZERS5BVPS
vendor: 14 bytes, lines 41-42
41"></script>
42  
42<script>
43    
vendor: 140 bytes, lines 43-47
43window.dataLayer = window.dataLayer || [];
44    function gtag(){dataLayer.push(arguments);}
45    gtag('js', new Date());
46
47    gtag('config', '
47G-8ZERS5BVPS
vendor: 6 bytes, lines 47-48
47');
48  
48</script>
48
49
50    
50<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
50
51    
51<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
51
52    
52<script src="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.8.0/codemirror.min.js"></script>
52
53    
53<script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/1.5.3/clipboard.min.js"></script>
53
54    
54<script type="text/javascript" async
55        src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/MathJax.js?config=TeX-MML-AM_CHTML">
56    </script>
56
57    
57<script defer src="js/fontawesome.all.min.js"></script>
57
58    
58<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.5.0/Chart.min.js"></script>
58
59
60    
60<script src="js/app.js"></script>
60
61    
61<script src="js/video_comparison.js"></script>
61
62    
62<script src="js/synced_video_selector.js"></script>
62
63    
64    <style>
65        /* 影片容器的響應式設計 */
66        .video-container2 {
67            position: relative;
68            width: 100%;
69            padding-bottom: 56.25%; /* 16:9 寬高比 */
70            height: 0;
71            overflow: hidden;
72            max-width: 1200px; /* 最大寬度限制 */
73            margin: 0 auto;
74        }
75
76        .video-container2 iframe {
77            position: absolute;
78            top: 0;
79            left: 0;
80            width: 100%;
81            height: 100%;
82        }
83
84        /* Bootstrap 欄位調整 */
85        .col-md-8.col-md-offset-2 {
86            width: 100%;
87            margin-left: 0;
88            padding: 0 15px;
89        }
90
91        .publication-video {
92            margin: 20px 0;
93        }
94
95        /* 一般螢幕尺寸的響應式設計 */
96        @media (min-width: 768px) {
97            .col-md-8.col-md-offset-2 {
98                width: 66.66667%;
99                margin-left: 16.66667%;
100            }
101        }
102    </style>
103
104
105</head>
106
107<body style="padding: 1%; width: 100%">
108    <div class="container" style="max-width: 1500px; margin: auto;" id="main">
109        <div class="row">
110            <h2 class="col-md-12 text-center">
111                <b>NaRCan</b>:<br>Natural Refined Canonical Image with Integration of Diffusion Prior for Video Editing</br> 
112                <small>
113                NeurIPS 2024
114                </small>
115            </h2>
116        </div>
117        <div class="row">
118            <div class="col-md-12 text-center">
119                <ul class="list-inline">
120                    <li>
121                        <a href="https://koi953215.github.io/">
122                            Ting-Hsuan Chen
123                        </a>
124                    </li>
125                    <li>
126                        <a href="">
127                            Jiewen Chan
128                        </a>
129                    </li>
130                    <li>
131                        <a href="">
132                            Hau-Shiang Shiu
133                        </a>
134                    </li>
135                    <li>
136                        <a href="">
137                            Shih-Han Yen
138                        </a>
139                    </li>
140                    <li>
141                        <a href="https://jimmycv07.github.io/">
142                            Chang-Han Yeh
143                        </a>
144                    </li>
145                    <li>
146                        <a href="https://yulunalexliu.github.io/">
147                            Yu-Lun Liu
148                        </a>
149                    </li>
150                    </br>National Yang Ming Chiao Tung University
151                </ul>
152            </div>
153        </div>
154
155
156        <div class="row">
157                <div class="col-md-6 col-md-offset-3 text-center">
158                    <ul class="nav nav-pills nav-justified">
159                        <li>
160                            <a href="http://arxiv.org/abs/2406.06523">
161                            <image src="img/narcan_paper_image.jpg" height="60px">
162                                <h4><strong>Paper</strong></h4>
163                            </a>
164                        </li>
165                        <li>
166                            <a href="https://drive.google.com/file/d/1Ur6HEP2MizFe3z7iSXoaMQmC5_qHUn1S/view?usp=sharing">
167                            <image src="img/database_icon.png" height="60px">
168                                <h4><strong>Evaluation results</strong></h4>
169                            </a>                            
170                        </li>
171                        <!-- <li>
172                            <a href="https://www.youtube.com/watch?v=xrrhynRzC8k">
173                            <image src="img/youtube_icon.png" height="60px">
174                                <h4><strong>Video</strong></h4>
175                            </a>
176                        </li> -->
177                        <li>
178                            <a href="https://github.com/koi953215/NaRCan">
179                            <image src="img/github.png" height="60px">
180                                <h4><strong>Code</strong></h4>
181                            </a>
182                        </li>
183                        <li>
184                            <a href="https://huggingface.co/spaces
184/Koi953215/NaRCan_demo">
185                            <image src="img/logos_huggingface.png" height="60px">
186                                <h4><strong>Hugging Face demo</strong></h4>
187                            </a>
188                        </li>
189                    </ul>
190                </div>
191        </div>
192
193
194
195        <div class="row">
196            <div class="col-md-8 col-md-offset-2">
197                <video id="v0" width="100%" autoplay loop muted controls>
198                  <source src="img/teaser.mp4" type="video/mp4" />
199                </video>
200                        </div>
201        </div>
202
203
204        <div class="row">
205            <div class="col-md-8 col-md-offset-2">
206                <h3>
207                    Abstract
208                </h3>
209                <p class="text-justify">
210                    We propose a video editing framework, NaRCan, which integrates a hybrid deformation field and a diffusion prior to generating high-quality natural canonical images to represent the input video. Our approach utilizes homography to model global motion and employs multi-layer perceptrons (MLPs) to capture local residual deformations, enhancing the model's ability to handle complex video dynamics. By introducing a diffusion prior from the early stages of training, our model ensures that the generated images retain a high-quality natural appearance, making the produced canonical images suitable for various downstream tasks in video editing, a capability not achieved by current canonical-based methods. Furthermore, we incorporate low-rank adaptation (LoRA) fine-tuning and introduce a noise and diffusion prior update scheduling technique that accelerates the training process by 14 times. Extensive experimental results show that our method outperforms existing approaches in various video editing tasks and produces coherent and high-quality edited video sequences.
211                </p>
212            </div>
213        </div>
214
215
216        <div class="row">
217        <div class="col-md-8 col-md-offset-2">
218            <h3>Video</h3>
219            <div class="publication-video">
220                <div class="video-container2">
221                    <iframe src="https://www.youtube.com/embed/ufYRGldCV78?rel=0&amp;showinfo=0"
222                            frameborder="0" 
223                            allow="autoplay; encrypted-media" 
224                            allowfullscreen></iframe>
225                </div>
226            </div>
227        </div>
228    </div>
229
230        <br>
231        <div class="row">
232            <div class="col-md-8 col-md-offset-2">
233                <h3>
234                  <b><font color="#4472c4">NaR</font><font color="#ff0000">Can</font></b> = <font color="#4472c4"><b>Na</b>tural <b>R</b>efined</font> <font color="#ff0000"><b>Can</b>onical Image</font> with <font color="#ed7d31"><b>Diffusion Prior</b></font>
235                </h3>
236                <video id="v0" width="100%" autoplay loop muted>
237                    <source src="img/method_teaser.mp4" type="video/mp4" />
238                </video>
239                <!-- <image src="img/teaser.jpg" width=100% style="display: block; margin: auto;"></image> -->
240                <br>
241                <p class="text-justify">
242                    <b>Video representation with diffusion prior.</b> Given an RGB video, we can represent the video using a canonical image. However, the canonical image and reconstruction training process focuses only on reconstruction quality and could produce an unnatural canonical image. This could cause problems with downstream tasks such as prompt-based video editing. In the bottom example, if the bear is distorted in the canonical image, the image editor, such as ControlNet, may not recognize it and could introduce an irrelevant object instead. In this paper, we propose introducing the <i>diffusion prior</i> from a LoRA fine-tuned diffusion model to the training pipeline and constraining the canonical image to be natural. Our method facilitates several downstream tasks, such as (a) video editing, (b) dynamic segmentation, and (c) video style transfer.
243                </p>
244            </div>
245        </div>
246
247        <br>
248        <div class="row">
249            <div class="col-md-8 col-md-offset-2">
250                <h3>
251                    Our proposed framework
252                </h3>
253                <image src="img/overview_combined.jpg" width=100% style="display: block; margin: auto;"></image>
254                <p class="text-justify">
255                    Given an input video sequence, our method aims to represent the video with a <i>natural</i> canonical image, which is a crucial representation for versatile downstream applications. (a) First, we fine-tune the LoRA weights of a pre-trained latent diffusion model on the input frames. (b) Second, we represent the video using a canonical MLP and a deformation field, which consists of homography estimation and residual deformation MLP for non-rigid residual deformations. By relying entirely on the reconstruction loss, the canonical MLP often fails to represent a natural canonical image, causing problems for downstream applications. <i>E.g.</i>, image-to-image translation methods such as ControlNet may not be able to recognize that there is a train in the canonical image. (c) Therefore, we leverage the fine-tuned latent diffusion model to regularize and correct the unnatural canonical image into a natural one. Specifically, we sophistically design a noise scheduling corresponding to the frame reconstruction process. (d) The natural and artifacts-free canonical image can then be facilitated to various downstream tasks such as video style transfer, dynamic segmentation, and editing, such as adding handwritten characters.
256                </p>
257            </div>
258        </div>
259        
260        <br>
261            
262        <div class="row">
263            <div class="col-md-8 col-md-offset-2">
264                <h3>
265                    Diffusion prior for canonical image refinement
266                </h3>
267                <div class="text-center">
268                    <video id="v0" width="80%" autoplay loop muted controls>
269                        <source src="img/dm_prior.mp4" type="video/mp4" />
270                    </video>
271                </div>
272                <p class="text-justify">
273                    (<em>Left</em>) Without diffusion prior to regularizing the canonical image, the training process relies only on the frame reconstruction and could sacrifice the faithfulness of the canonical image. (<em>Right</em>) Our fine-tuned diffusion prior effectively corrects the canonical image to faithfully represent the input frames and results in natural canonical images.
274                </p>
275            </div>
276        </div>
277
278        <br>
279        <div class="row">
280            <div class="col-md-8 col-md-offset-2">
281                <h3>
282                    Noise and diffusion prior update scheduling
283                </h3>
284                <image src="img/scheduling.jpg" width=100% style="display: block; margin: auto;"></image>
285                <!-- <br> -->
286                <hr style="border-top: dotted 3px;" />
287                <div class="text-center">
288                    <video id="v0" width="80%" autoplay loop muted controls>
289                        <source src="img/schedule.mp4" type="video/mp4" />
290                    </video>
291                </div>
292                <p class="text-justify">
293                    Initially, our model fits object outlines before the fields converge and without the diffusion prior, resulting in unnatural elements in the canonical image due to complex non-rigid objects. Upon introducing the diffusion prior with increased noise and update frequency, the model learns to generate natural, high-quality images, leading to convergence. Thus, the strength of noise and the update frequency will also decrease. Moreover, it's worth mentioning that update scheduling cuts training time from 4.8 hours to 20 minutes.
294                </p>
295            </div>
296        </div>
297
298
299
300
301
302
303
304
305
306        <br>
307        <div class="row">
308            <div class="col-md-8 col-md-offset-2">
309                <h3>
310                    Text-guided video-to-video translation
311                </h3>
312
313                <div class="text-center ">
314                    <ul class="nav nav-pills center-pills">
315                        <li class="method-pill" data-value="input"
316                            onclick="selectCompVideo('styletransfer', this, activeScenePill)"><a>Input</a></li>
317                        <li class="method-pill" data-value="hash"
318                            onclick="selectCompVideo('styletransfer', this, activeScenePill)"><a>Hashing-nvd</a></li>
319                        <li class="method-pill active" data-value="codef"
320                            onclick="selectCompVideo('styletransfer', this, activeScenePill)"><a>CoDeF</a></li>
321                        <li class="method-pill" data-value="medm"
322                            onclick="selectCompVideo('styletransfer', this, activeScenePill)"><a>MeDM</a></li>
323                    </ul>
324                </div>
325
326                
326<script>
327                    activeMethodPill = document.querySelector('.method-pill.active-pill');
328                    activeScenePill = document.querySelector('.scene-pill.active-pill');
329                    activeModePill = document.querySelector('.mode-pill.active-pill');
330                </script>
330
331                
332                <div class="text-center">
333                    <div class="video-container">
334                        <video class="video" style="max-width: 100%;" m id="compVideo0" loop playsinline autoplay muted>
335                            <source src="videos/styletransfer_bear_codef_vs_ours_video.mp4" />
336                        </video>
337                        <video class="video" style="max-width: 100%;" id="compVideo1" loop playsinline autoplay muted hidden>
338                            <source src="videos/styletransfer_bear_codef_vs_ours_canonical.mp4" />
339                        </video>
340                    </div>
341                    <div class="text-center" style="color: black;" id="mode-pills">
342                        <div class="btn-group btn-group-sm">
343                            <span class="btn btn-primary mode-pill active" data-value="video"
344                                onclick="selectCompVideo('styletransfer', activeMethodPill, activeScenePill, null, this)">
345                                Edited video
346                            </span>
347                            <span class="btn btn-primary mode-pill" data-value="canonical"
348                                onclick="selectCompVideo('styletransfer', activeMethodPill, activeScenePill, null, this)">
349                                Canonical image
350                            </span>
351                            <span class="btn btn-primary mode-pill" data-value="canonical_edit"
352                                onclick="selectCompVideo('styletransfer', activeMethodPill, activeScenePill, null, this)">
353                                Edited canonical image
354                            </span>
355                        </div>
356                    </div>
357
358
359                    <p class="text-justify" style="text-align: center;">
360                        Text prompt: <span id="textPrompt"><strong>A Van Gogh-style cartoon bear walking in the forest.</strong></span><br>
361                        Baseline method (left) vs NaRCan (right). Try selecting different methods and scenes!
362                    </p>
363                    
363<script>
364                        video0 = document.getElementById("compVideo0");
365                        video1 = document.getElementById("compVideo1");
366                        video0.addEventListener('loadedmetadata', function() {
367                            if (activeVidID == 0 && select){
368                                video0.play();
369                                // print video size
370                                console.log(video0.videoWidth, video0.videoHeight);
371                                video0.hidden = false;
372                                video1.hidden = true;
373                            }
374                        });
375                        video1.addEventListener('loadedmetadata', function() {
376                            if (activeVidID == 1 && select){
377                                video1.play();
378                                // print video size
379                                console.log(video1.videoWidth, video1.videoHeight);
380                                video0.hidden = true;
381                                video1.hidden = false;
382                            }
383                        });
384                    </script>
384
385
386                    <div class="pill-row scene-pills" id="scene-pills">
387                        <span class="pill scene-pill active" data-value="bear" onclick="selectCompVideo('styletransfer', activeMethodPill, this, 3)">
388                            <img class="thumbnail-img" src="thumbnails/styletransfer_bear_thumbnail.jpg" alt="DTU/scan31" width="64">
389                        </span>
390                        <span class="pill scene-pill" data-value="boat" onclick="selectCompVideo('styletransfer', activeMethodPill, this, 3)">
391                            <img class="thumbnail-img" src="thumbnails/styletransfer_boat_thumbnail.jpg" alt="DTU/scan45" width="64">
392                        </span>
393                        <span class="pill scene-pill" data-value="hot-air-ballon" onclick="selectCompVideo('styletransfer', activeMethodPill, this, 3)">
394                            <img class="thumbnail-img" src="thumbnails/styletransfer_hot-air-ballon_thumbnail.jpg" alt="LLFF/fern" width="64">
395                        </span>
396                        <span class="pill scene-pill" data-value="overlook-the-ocean" onclick="selectCompVideo('styletransfer', activeMethodPill, this, 3)">
397                            <img class="thumbnail-img" src="thumbnails/styletransfer_overlook-the-ocean_thumbnail.jpg" alt="LLFF/horns" width="64">
398                        </span>
399                        <span class="pill scene-pill" data-value="shark-ocean" onclick="selectCompVideo('styletransfer', activeMethodPill, this, 3)">
400                            <img class="thumbnail-img" src="thumbnails/styletransfer_shark-ocean_thumbnail.jpg" alt="Re10K/sofa" width="64">
401                        </span>
402                    </div>
403
404                    
404<script>
405                        activeMethodPill = document.querySelector('.method-pill.active-pill');
406                        activeScenePill = document.querySelector('.scene-pill.active-pill');
407                        activeModePill = document.querySelector('.mode-pill.active-pill');
408                    </script>
408
409                </div>
410            </div>
411        </div>
412        <br>
413        
414
415
416
417        <div class="row">
418            <div class="col-md-8 col-md-offset-2">
419                <h3>
420                    Adding handwritten characters
421                </h3>
422
423                <div class="text-center ">
424                    <ul class="nav nav-pills center-pills">
425                        <li class="method-pill-handwrite" data-value="input"
426                            onclick="selectCompVideoHandwrite('handwrite', this, activeScenePillHandwrite)"><a>Input</a></li>
427                        <li class="method-pill-handwrite" data-value="hash"
428                            onclick="selectCompVideoHandwrite('handwrite', this, activeScenePillHandwrite)"><a>Hashing-nvd</a></li>
429                        <li class="method-pill-handwrite active" data-value="codef"
430                            onclick="selectCompVideoHandwrite('handwrite', this, activeScenePillHandwrite)"><a>CoDeF</a></li>
431                        <li class="method-pill-handwrite" data-value="medm"
432                            onclick="selectCompVideoHandwrite('handwrite', this, activeScenePillHandwrite)"><a>MeDM</a></li>
433                    </ul>
434                </div>
435
436                
436<script>
437                    activeMethodPillHandwrite = document.querySelector('.method-pill-handwrite.active-pill');
438                    activeScenePillHandwrite = document.querySelector('.scene-pill-handwrite.active-pill');
439                    activeModePillHandwrite = document.querySelector('.mode-pill-handwrite.active-pill');
440                </script>
440
441                
442                <div class="text-center">
443                    <div class="video-container">
444                        <video class="video" style="max-width: 100%;" m id="compVideo0_handwrite" loop playsinline autoplay muted>
445                            <source src="videos/handwrite_train_codef_vs_ours_video.mp4" />
446                        </video>
447                        <video class="video" style="max-width: 100%;" id="compVideo1_handwrite" loop playsinline autoplay muted hidden>
448                            <source src="videos/handwrite_train_codef_vs_ours_canonical.mp4" />
449                        </video>
450                    </div>
451                    <div class="text-center" style="color: black;" id="mode-pills">
452                        <div class="btn-group btn-group-sm">
453                            <span class="btn btn-primary mode-pill-handwrite active" data-value="video"
454                                onclick="selectCompVideoHandwrite('handwrite', activeMethodPillHandwrite, activeScenePillHandwrite, null, this)">
455                                Edited video
456                            </span>
457                            <span class="btn btn-primary mode-pill-handwrite" data-value="canonical"
458                                onclick="selectCompVideoHandwrite('handwrite', activeMethodPillHandwrite, activeScenePillHandwrite, null, this)">
459                                Canonical image
460                            </span>
461                            <span class="btn btn-primary mode-pill-handwrite" data-value="canonical_edit"
462                                onclick="selectCompVideoHandwrite('handwrite', activeMethodPillHandwrite, activeScenePillHandwrite, null, this)">
463                                Edited canonical image
464                            </span>
465                        </div>
466                    </div>
467
468
469                    <p class="text-justify" style="text-align: center;">
470                        Baseline method (left) vs NaRCan (right). Try selecting different methods and scenes!
471                    </p>
472                    
472<script>
473                        video0_handwrite = document.getElementById("compVideo0_handwrite");
474                        video1_handwrite = document.getElementById("compVideo1_handwrite");
475                        video0_handwrite.addEventListener('loadedmetadata', function() {
476                            if (activeVidIDHandwrite == 0 && select){
477                                video0_handwrite.play();
478                                // print video size
479                                console.log(video0_handwrite.videoWidth, video0_handwrite.videoHeight);
480                                video0_handwrite.hidden = false;
481                                video1_handwrite.hidden = true;
482                            }
483                        });
484                        video1_handwrite.addEventListener('loadedmetadata', function() {
485                            if (activeVidIDHandwrite == 1 && select){
486                                video1_handwrite.play();
487                                // print video size
488                                console.log(video1_handwrite.videoWidth, video1_handwrite.videoHeight);
489                                video0_handwrite.hidden = true;
490                                video1_handwrite.hidden = false;
491                            }
492                        });
493                    </script>
493
494                    
495                    <div class="pill-row scene-pills-handwrite" id="scene-pills-handwrite">
496                        <span class="pill scene-pill-handwrite active" data-value="train" onclick="selectCompVideoHandwrite('handwrite', activeMethodPillHandwrite, this, 3)">
497                            <img class="thumbnail-img" src="thumbnails/handwrite_train_thumbnail.jpg" alt="DTU/scan31" width="64">
498                        </span>
499                        <span class="pill scene-pill-handwrite" data-value="camel" onclick="selectCompVideoHandwrite('handwrite', activeMethodPillHandwrite, this, 3)">
500                            <img class="thumbnail-img" src="thumbnails/handwrite_camel_thumbnail.jpg" alt="DTU/scan45" width="64">
501                        </span>
502                        <span class="pill scene-pill-handwrite" data-value="cat" onclick="selectCompVideoHandwrite('handwrite', activeMethodPillHandwrite, this, 3)">
503                            <img class="thumbnail-img" src="thumbnails/handwrite_cat_thumbnail.jpg" alt="LLFF/fern" width="64">
504                        </span>
505                        <span class="pill scene-pill-handwrite" data-value="car-turn" onclick="selectCompVideoHandwrite('handwrite', activeMethodPillHandwrite, this, 3)">
506                            <img class="thumbnail-img" src="thumbnails/handwrite_car-turn_thumbnail.jpg" alt="LLFF/horns" width="64">
507                        </span>
508                        <span class="pill scene-pill-handwrite" data-value="tiger" onclick="selectCompVideoHandwrite('handwrite', activeMethodPillHandwrite, this, 3)">
509                            <img class="thumbnail-img" src="thumbnails/handwrite_tiger_thumbnail.jpg" alt="Re10K/sofa" width="64">
510                        </span>
511                    </div>
512
513                    
513<script>
514                        activeMethodPillHandwrite = document.querySelector('.method-pill-handwrite.active-pill');
515                        activeScenePillHandwrite = document.querySelector('.scene-pill-handwrite.active-pill');
516                        activeModePillHandwrite = document.querySelector('.mode-pill-handwrite.active-pill');
517                    </script>
517
518                </div>
519            </div>
520        </div>
521        <br>
522
523
524        <div class="row">
525            <div class="col-md-8 col-md-offset-2">
526                <h3>
527                    Segmentation-based tracking
528                </h3>
529
530                <div class="text-center ">
531                    <ul class="nav nav-pills center-pills">
532                        <li class="method-pill-dynamic" data-value="input"
533                            onclick="selectCompVideoDynamic('dynamic', this, activeScenePillDynamic)"><a>Input</a></li>
534                        <li class="method-pill-dynamic" data-value="hash"
535                            onclick="selectCompVideoDynamic('dynamic', this, activeScenePillDynamic)"><a>Hashing-nvd</a></li>
536                        <li class="method-pill-dynamic active" data-value="codef"
537                            onclick="selectCompVideoDynamic('dynamic', this, activeScenePillDynamic)"><a>CoDeF</a></li>
538                        <li class="method-pill-dynamic" data-value="medm"
539                            onclick="selectCompVideoDynamic('dynamic', this, activeScenePillDynamic)"><a>MeDM</a></li>
540                    </ul>
541                </div>
542
543                
543<script>
544                    activeMethodPillDynamic = document.querySelector('.method-pill-dynamic.active-pill');
545                    activeScenePillDynamic = document.querySelector('.scene-pill-dynamic.active-pill');
546                    activeModePillDynamic = document.querySelector('.mode-pill-dynamic.active-pill');
547                </script>
547
548                
549                <div class="text-center">
550                    <div class="video-container">
551                        <video class="video" style="max-width: 100%;" m id="compVideo0_dynamic" loop playsinline autoplay muted>
552                            <source src="videos/dynamic_coral-reef_codef_vs_ours_video.mp4" />
553                        </video>
554                        <video class="video" style="max-width: 100%;" id="compVideo1_dynamic" loop playsinline autoplay muted hidden>
555                            <source src="videos/dynamic_coral-reef_codef_vs_ours_canonical.mp4" />
556                        </video>
557                    </div>
558                    <div class="text-center" style="color: black;" id="mode-pills">
559                        <div class="btn-group btn-group-sm">
560                            <span class="btn btn-primary mode-pill-dynamic active" data-value="video"
561                                onclick="selectCompVideoDynamic('dynamic', activeMethodPillDynamic, activeScenePillDynamic, null, this)">
562                                Edited video
563                            </span>
564                            <span class="btn btn-primary mode-pill-dynamic" data-value="canonical"
565                                onclick="selectCompVideoDynamic('dynamic', activeMethodPillDynamic, activeScenePillDynamic, null, this)">
566                                Canonical image
567                            </span>
568                            <span class="btn btn-primary mode-pill-dynamic" data-value="canonical_edit"
569                                onclick="selectCompVideoDynamic('dynamic', activeMethodPillDynamic, activeScenePillDynamic, null, this)">
570                                Segmentation mask
571                            </span>
572                        </div>
573                    </div>
574
575
576                    <p class="text-justify" style="text-align: center;">
577                        Baseline method (left) vs NaRCan (right). The segmentation mask is acquired using the <a href="https://segment-anything.com/">Segment Anything Model (SAM)</a> based on the learned canonical image and propagated to the sequence. Try selecting different methods and scenes!
578                    </p>
579                    
579<script>
580                        video0_dynamic = document.getElementById("compVideo0_dynamic");
581                        video1_dynamic = document.getElementById("compVideo1_dynamic");
582                        video0_dynamic.addEventListener('loadedmetadata', function() {
583                            if (activeVidIDDynamic == 0 && select){
584                                video0_dynamic.play();
585                                // print video size
586                                console.log(video0_dynamic.videoWidth, video0_dynamic.videoHeight);
587                                video0_dynamic.hidden = false;
588                                video1_dynamic.hidden = true;
589                            }
590                        });
591                        video1_dynamic.addEventListener('loadedmetadata', function() {
592                            if (activeVidIDDynamic == 1 && select){
593                                video1_dynamic.play();
594                                // print video size
595                                console.log(video1_dynamic.videoWidth, video1_dynamic.videoHeight);
596                                video0_dynamic.hidden = true;
597                                video1_dynamic.hidden = false;
598                            }
599                        });
600                    </script>
600
601                    
602                    <div class="pill-row scene-pills-dynamic" id="scene-pills-dynamic">
603                        <span class="pill scene-pill-dynamic active" data-value="coral-reef" onclick="selectCompVideoDynamic('dynamic', activeMethodPillDynamic, this, 3)">
604                            <img class="thumbnail-img" src="thumbnails/dynamic_coral-reef_thumbnail.jpg" alt="DTU/scan31" width="64">
605                        </span>
606                        <span class="pill scene-pill-dynamic" data-value="butterfly" onclick="selectCompVideoDynamic('dynamic', activeMethodPillDynamic, this, 3)">
607                            <img class="thumbnail-img" src="thumbnails/dynamic_butterfly_thumbnail.jpg" alt="DTU/scan45" width="64">
608                        </span>
609                        <span class="pill scene-pill-dynamic" data-value="two-swan" onclick="selectCompVideoDynamic('dynamic', activeMethodPillDynamic, this, 3)">
610                            <img class="thumbnail-img" src="thumbnails/dynamic_two-swan_thumbnail.jpg" alt="LLFF/fern" width="64">
611                        </span>
612                        <span class="pill scene-pill-dynamic" data-value="woman-drink" onclick="selectCompVideoDynamic('dynamic', activeMethodPillDynamic, this, 3)">
613                            <img class="thumbnail-img" src="thumbnails/dynamic_woman-drink_thumbnail.jpg" alt="LLFF/horns" width="64">
614                        </span>
615                        <span class="pill scene-pill-dynamic" data-value="surf" onclick="selectCompVideoDynamic('dynamic', activeMethodPillDynamic, this, 3)">
616                            <img class="thumbnail-img" src="thumbnails/dynamic_surf_thumbnail.jpg" alt="Re10K/sofa" width="64">
617                        </span>
618                    </div>
619
620                    
620<script>
621                        activeMethodPillDynamic = document.querySelector('.method-pill-dynamic.active-pill');
622                        activeScenePillDynamic = document.querySelector('.scene-pill-dynamic.active-pill');
623                        activeModePillDynamic = document.querySelector('.mode-pill-dynamic.active-pill');
624                    </script>
624
625                </div>
626            </div>
627        </div>
628        <br>
629
630        <div class="row">
631            <div class="col-md-8 col-md-offset-2">
632                <h3>
633                    User study
634                </h3>
635                <image src="img/user_study.jpg" width=100% style="display: block; margin: auto;"></image>
636                <p class="text-justify">
637                    Our method achieves the highest user preference ratios across all three aspects, compared with MeDM, CoDeF, Hashing-nvd.
638                </p>
639            </div>
640        </div><br>
641
642
643
644
645
646
647
648
649
650        <div class="row">
651            <div class="col-md-8 col-md-offset-2">
652                <h3>
653                    Citation
654                </h3>
655                <div class="form-group col-md-10 col-md-offset-1">
656                    <textarea id="bibtex" class="form-control" readonly>
657@article{chen2024narcan,
658  title={NaRCan: Natural Refined Canonical Image with Integration of Diffusion Prior for Video Editing},
659  author={Chen, Ting-Hsuan and Chan, Jiewen and Shiu, Hau-Shiang and Yen, Shih-Han and Yeh, Chang-Han and Liu, Yu-Lun},
660  journal={Advances in Neural Information Processing Systems},
661  year={2024}
662}</textarea>
663                </div>
664            </div>
665        </div>
666
667        <div class="row">
668            <div class="col-md-8 col-md-offset-2">
669                <h3>
670                    Acknowledgements
671                </h3>
672                <p class="text-justify">
673                This research was funded by the National Science and Technology Council, Taiwan, under Grants NSTC 112-2222-E-A49-004-MY2. The authors are grateful to Google, NVIDIA, and MediaTek Inc. for generous donations. Yu-Lun Liu acknowledges the Yushan Young Fellow Program by the MOE in Taiwan.
674                    <br><br>
675                The website template was borrowed from <a href="http://mgharbi.com/">Michaël Gharbi</a> and <a href="https://dorverbin.github.io/refnerf">Ref-NeRF</a>.
676                </p>
677            </div>
678        </div>
679    </div>
680</body>
681</html>

Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.