PageSourceSearch

https://zhongdao.github.io/UniTrack/

html zhongdao.github.io collected 2026-10-03 10:14:44 UTC 25,162 bytes, 637 lines download raw bytes

1<!DOCTYPE html>
2<html>
3<head>
4  <meta charset="utf-8">
5  <meta name="description"
6        content="">
7  <meta name="keywords" content="UniTrack, object tracking">
8  <meta name="viewport" content="width=device-width, initial-scale=1">
9  <title>Do Different Tracking Tasks Require Different Appearance Models?</title>
10
11
12  <link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
13        rel="stylesheet">
14
15  <link rel="stylesheet" href="./static/css/bulma.min.css">
16  <link rel="stylesheet" href="./static/css/bulma-carousel.min.css">
17  <link rel="stylesheet" href="./static/css/bulma-slider.min.css">
18  <link rel="stylesheet" href="./static/css/fontawesome.all.min.css">
19  <link rel="stylesheet"
20        href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
21  <link rel="stylesheet" href="./static/css/index.css">
22
23  
23<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
23
24  
24<script defer src="./static/js/fontawesome.all.min.js"></script>
24
25  
25<script src="./static/js/bulma-carousel.min.js"></script>
25
26  
26<script src="./static/js/bulma-slider.min.js"></script>
26
27  
27<script src="./static/js/index.js"></script>
27
28</head>
29<body>
30
31
32
33<section class="hero">
34  <div class="hero-body">
35    <div class="container is-max-desktop">
36      <div class="columns is-centered">
37        <div class="column has-text-centered">
38          <h1 class="title is-1 publication-title">Do Different Tracking Tasks Require Different Appearance Models?</h1>
39          <div class="is-size-5 publication-authors">
40            <span class="author-block">
41              <a href="https://zhongdao.github.io">Zhongdao Wang</a><sup>1</sup>,
42
43              <a href="https://hszhao.github.io">Hengshuang Zhao</a><sup>2</sup>,
44
45              <a href="https://scholar.google.com/citations?user=aYLh7fsAAAAJ&hl=en&oi=ao">Ya-Li Li</a><sup>1</sup>,
46
47              <a href="https://scholar.google.com/citations?user=RgzLZZsAAAAJ&hl=en&oi=ao">Shengjin Wang</a><sup>1</sup>,
48 
49              <a href="https://www.robots.ox.ac.uk/~phst/">Philip H.S. Torr</a><sup>2</sup>,
50
51              <a href="https://www.robots.ox.ac.uk/~luca">Luca Bertinetto</a><sup>3</sup>
52            <br /><sup>1</sup>Tsinghua University&emsp;<sup>2</sup>University of Oxford&emsp;<sup>3</sup>FIVE AI
53            <p>
54            <div class="brmod"></div>NeurIPS 2021</span>
55          </div>
56
57
58
59
60          <div class="column has-text-centered">
61            <div class="publication-links">
62              <!-- PDF Link. -->
63              <span class="link-block">
64                <a href="https://arxiv.org/abs/2107.02156"
65                   class="external-link button is-normal is-rounded is-dark">
66                  <span class="icon">
67                      <i class="ai ai-arxiv"></i>
68                  </span>
69                  <span>arXiv</span>
70                </a>
71              </span>
72              <!-- Video Link. -->
73              <!--
74              <span class="link-block">
75                <a href="https://www.youtube.com/"
76                   class="external-link button is-normal is-rounded is-dark">
77                  <span class="icon">
78                      <i class="fab fa-youtube"></i>
79                  </span>
80                  <span>Video</span>
81                </a>
82              </span>
83              -->
84              <!-- Code Link. -->
85              <span class="link-block">
86                <a href="https://github.com/Zhongdao/UniTrack"
87                   class="external-link button is-normal is-rounded is-dark">
88                  <span class="icon">
89                      <i class="fab fa-github"></i>
90                  </span>
91                  <span>Code</span>
92                  </a>
93              </span>
94                  </a>
95            </div>
96
97          </div>
98        </div>
99      </div>
100    </div>
101  </div>
102</section>
103
104<section class="hero teaser">
105  <div class="container is-max-desktop">
106    <div class="hero-body">
107      <img src="static/images/radar.png"
108                 class="interpolation-image"
109                 alt="Interpolate start reference image."/>
110      </img>
111      <br>
112
113      <h2 class="subtitle has-text-centered">
114        <span class="dnerf"> We experiment with a large amount of self-supervised representations on five different tracking tasks and present a comprehensive comparison. A polygon with a larger area means better performance across the board. Dashed gray line indicates ImageNet pre-trained representation.
115      </h2>
116    </div>
117  </div>
118</section>
119
120
121<!-- <section class="hero is-light is-small">
122  <div class="hero-body">
123    <div class="container">
124      <div id="results-carousel" class="carousel results-carousel">
125        <div class="item item-steve">
126          <video poster="" id="steve" autoplay controls muted loop height="100%">
127            <source src="https://homes.cs.washington.edu/~kpar/nerfies/videos/steve.mp4"
128                    type="video/mp4">
129          </video>
130        </div>
131        <div class="item item-chair-tp">
132          <video poster="" id="chair-tp" autoplay controls muted loop height="100%">
133            <source src="https://homes.cs.washington.edu/~kpar/nerfies/videos/chair-tp.mp4"
134                    type="video/mp4">
135          </video>
136        </div>
137        <div class="item item-shiba">
138          <video poster="" id="shiba" autoplay controls muted loop height="100%">
139            <source src="https://homes.cs.washington.edu/~kpar/nerfies/videos/shiba.mp4"
140                    type="video/mp4">
141          </video>
142        </div>
143        <div class="item item-fullbody">
144          <video poster="" id="fullbody" autoplay controls muted loop height="100%">
145            <source src="https://homes.cs.washington.edu/~kpar/nerfies/videos/fullbody.mp4"
146                    type="video/mp4">
147          </video>
148        </div>
149        <div class="item item-blueshirt">
150          <video poster="" id="blueshirt" autoplay controls muted loop height="100%">
151            <source src="https://homes.cs.washington.edu/~kpar/nerfies/videos/blueshirt.mp4"
152                    type="video/mp4">
153          </video>
154        </div>
155        <div class="item item-mask">
156          <video poster="" id="mask" autoplay controls muted loop height="100%">
157            <source src="https://homes.cs.washington.edu/~kpar/nerfies/videos/mask.mp4"
158                    type="video/mp4">
159          </video>
160        </div>
161        <div class="item item-coffee">
162          <video poster="" id="coffee" autoplay controls muted loop height="100%">
163            <source src="https://homes.cs.washington.edu/~kpar/nerfies/videos/coffee.mp4"
164                    type="video/mp4">
165          </video>
166        </div>
167        <div class="item item-toby">
168          <video poster="" id="toby" autoplay controls muted loop height="100%">
169            <source src="https://homes.cs.washington.edu/~kpar/nerfies/videos/toby2.mp4"
170                    type="video/mp4">
171          </video>
172        </div>
173      </div>
174    </div>
175  </div>
176</section> -->
177
178
179<section class="section">
180  <div class="container is-max-desktop">
181    <!-- Abstract. -->
182    <div class="columns is-centered has-text-centered">
183      <div class="column is-four-fifths">
184        <h2 class="title is-3">Abstract</h2>
185        <div class="content has-text-justified">
186          <p>
187          Tracking objects of interest in a video is one of the most popular and widely applicable problems
188          in computer vision. However, with the years, a Cambrian explosion of use cases and benchmarks
189          has fragmented the problem in a multitude of different experimental setups. As a consequence, the
190          literature has fragmented too, and now the novel approaches proposed by the community are usually
191          specialised to fit only one specific setup. 
192          </p>
193
194          <p>
195          To understand to what extent this specialisation is actually
196          necessary, in this work we present <b>UniTrack</b>, a solution to address five different tasks within the same
197          framework. UniTrack consists of a single and task-agnostic appearance model, which can be learned
198          in a supervised or self-supervised fashion, and multiple “heads” to address individual tasks and that do
199          not require training. We show how most tracking tasks can be solved within this framework, and that
200          the same appearance model can be used to obtain performance that is competitive against specialised
201          methods for all the five tasks considered. The framework also allows us to analyse appearance models
202          obtained with the most recent self-supervised methods, thus significantly extending their evaluation
203          and comparison to a larger variety of important problems. 
204        </p>
205        </div>
206      </div>
207    </div>
208    <!--/ Abstract. -->
209
210    <!-- Paper video. -->
211    <!--
212    <div class="columns is-centered has-text-centered">
213      <div class="column is-four-fifths">
214        <h2 class="title is-3">Video</h2>
215        <div class="publication-video">
216          <iframe src=""
217                  frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
218        </div>
219      </div>
220    </div>
221    -->
222    <!--/ Paper video. -->
223  </div>
224</section>
225
226
227<section class="section">
228
229
230  <div class="container is-max-desktop">
231
232
233    <div class="columns is-centered">
234      <div class="column is-full-width">
235        <h2 class="title is-3">Proliferation of tracking setups</h2>
236
237        <div class="content has-text-justified">
238          <p>
239            Various use cases and benchmarks has fragmented the tracking problem in multiple setups.
240          </p>
241        </div>
242        <div class="columns is-vcentered interpolation-panel">
243
244          <img src="static/images/tasks.png"
245                 class="interpolation-image"
246                 alt="Interpolate start reference image."/>
247          </img>
248
249        </div>
250        <br/>
251
252
253
254      </div>
255    </div>
256
257
258
259    <div class="columns is-centered">
260
261
262      <div class="column">
263        <div class="content">
264          <h2 class="title is-3">The UniTrack framework</h2>
265          <p>
266            To investigate what consititutes a good representation of objects in videos,  we propose UniTrack, a unified tracking framework with a single shared appearance model. 
267            UniTrack is divided into three "levels":
268          </p>
269            <ul>
270              <li> <b>Level-1</b>: The shared, base appearance model. We are particularly interested in <i>self-supervised</i> representations like MoCo and SimCLR.</li> 
271              <li> <b>Level-2</b>: Two parameter-free algorithmic primitives: <i>propagation</i> and <i>assciation</i>, that operate on deep features extracted by the appearance model.</li> 
272              <li> <b>Level-3</b>: Task-specific solutions. Used to adapt the two level-2 primitives to task-specific outputs.
273 </li> 
274            </ul>
275        </div>
276      </div>
277
278
279      <div class="column">
280        <h2 class="title is-3"></h2>
281        <div class="columns is-centered">
282          <div class="column content">
283             <img src="static/images/framework.png"
284                 class="interpolation-image"
285                 alt="Interpolate start reference image."/>
286          </img>
287          </div>
288
289        </div>
290      </div>
291
292    </div>
293</section>
294
295  
296
297
298  
299<section class="section">
300  <div class="container is-max-desktop">
301    <h2 class="title is-3">Results visulization</h2>
302        Below we show qualitative results of UniTrack on five different tracking tasks. The appearance model used is an <b> ImageNet pre-trained ResNet-18</b>. <b>No</b> task-specific training is performed.  Note that we use this appearance model only for simplicity and consistency of comaprison. Many self-supervised representations are better on all task. 
303        <div class="brmod"></div>
304      <div class="hero is-light is-small">
305
306        <div class="hero-body">
307          <div class="container">
308            <a class="title is-5"> Single Object Tracking (SOT) @ OTB-2015</a>
309            <div id="results-carousel" class="carousel results-carousel">
310              <div class="item item-steve">
311                  <img src="static/images/vis/sot1.gif"
312                       class="interpolation-image"
313                       alt="Interpolate start reference image."/>
314              </div>
315              <div class="item item-steve">
316                  <img src="static/images/vis/sot2.gif"
317                       class="interpolation-image"
318                       alt="Interpolate start reference image."/>
319              </div>
320              <div class="item item-steve">
321                  <img src="static/images/vis/sot3.gif"
322                       class="interpolation-image"
323                       alt="Interpolate start reference image."/>
324              </div>
325              <div class="item item-steve">
326                  <img src="static/images/vis/sot4.gif"
327                       class="interpolation-image"
328                       alt="Interpolate start reference image."/>
329              </div>
330              <div class="item item-steve">
331                  <img src="static/images/vis/sot5.gif"
332                       class="interpolation-image"
333                       alt="Interpolate start reference image."/>
334              </div>
335              <div class="item item-steve">
336                  <img src="static/images/vis/sot6.gif"
337                       class="interpolation-image"
338                       alt="Interpolate start reference image."/>
339              </div>
340              <div class="item item-steve">
341                  <img src="static/images/vis/sot7.gif"
342                       class="interpolation-image"
343                       alt="Interpolate start reference image."/>
344              </div>
345              <div class="item item-steve">
346                  <img src="static/images/vis/sot8.gif"
347                       class="interpolation-image"
348                       alt="Interpolate start reference image."/>
349              </div>
350            </div>
351          </div>
352      </div>
353
354        <div class="hero-body">
355          <div class="container">
356            <a class="title is-5"> Video Object Segmentation (VOS) @ DAVIS-2017</a>
357            <div id="results-carousel" class="carousel results-carousel">
358              <div class="item item-steve">
359                  <img src="static/images/vis/vos1.gif"
360                       class="interpolation-image"
361                       alt="Interpolate start reference image."/>
362              </div>
363              <div class="item item-steve">
364                  <img src="static/images/vis/vos2.gif"
365                       class="interpolation-image"
366                       alt="Interpolate start reference image."/>
367              </div>
368              <div class="item item-steve">
369                  <img src="static/images/vis/vos3.gif"
370                       class="interpolation-image"
371                       alt="Interpolate start reference image."/>
372              </div>
373              <div class="item item-steve">
374                  <img src="static/images/vis/vos4.gif"
375                       class="interpolation-image"
376                       alt="Interpolate start reference image."/>
377              </div>
378              <div class="item item-steve">
379                  <img src="static/images/vis/vos5.gif"
380                       class="interpolation-image"
381                       alt="Interpolate start reference image."/>
382              </div>
383              <div class="item item-steve">
384                  <img src="static/images/vis/vos6.gif"
385                       class="interpolation-image"
386                       alt="Interpolate start reference image."/>
387              </div>
388              <div class="item item-steve">
389                  <img src="static/images/vis/vos7.gif"
390                       class="interpolation-image"
391                       alt="Interpolate start reference image."/>
392              </div>
393              <div class="item item-steve">
394                  <img src="static/images/vis/vos8.gif"
395                       class="interpolation-image"
396                       alt="Interpolate start reference image."/>
397              </div>
398            </div>
399          </div>
400      </div>
401
402      <div class="hero-body">
403          <div class="container">
404            <a class="title is-5"> Multi-Object Tracking (MOT) @ MOT-16</a>
405            <div id="results-carousel" class="carousel results-carousel">
406              <div class="item item-steve">
407                  <img src="static/images/vis/mot1.gif"
408                       class="interpolation-image"
409                       alt="Interpolate start reference image."/>
410              </div>
411              <div class="item item-steve">
412                  <img src="static/images/vis/mot2.gif"
413                       class="interpolation-image"
414                       alt="Interpolate start reference image."/>
415              </div>
416              <div class="item item-steve">
417                  <img src="static/images/vis/mot3.gif"
418                       class="interpolation-image"
419                       alt="Interpolate start reference image."/>
420              </div>
421              <div class="item item-steve">
422                  <img src="static/images/vis/mot4.gif"
423                       class="interpolation-image"
424                       alt="Interpolate start reference image."/>
425              </div>
426              <div class="item item-steve">
427                  <img src="static/images/vis/mot5.gif"
428                       class="interpolation-image"
429                       alt="Interpolate start reference image."/>
430              </div>
431              <div class="item item-steve">
432                  <img src="static/images/vis/mot6.gif"
433                       class="interpolation-image"
434                       alt="Interpolate start reference image."/>
435              </div>
436            </div>
437          </div>
438      </div>
439
440      <div class="hero-body">
441          <div class="container">
442            <a class="title is-5"> Multi-Object Tracking and Segmentation (MOTS) @ MOTS</a>
443            <div id="results-carousel" class="carousel results-carousel">
444              <div class="item item-steve">
445                  <img src="static/images/vis/mots1.gif"
446                       class="interpolation-image"
447                       alt="Interpolate start reference image."/>
448              </div>
449              <div class="item item-steve">
450                  <img src="static/images/vis/mots2.gif"
451                       class="interpolation-image"
452                       alt="Interpolate start reference image."/>
453              </div>
454              <div class="item item-steve">
455                  <img src="static/images/vis/mots3.gif"
456                       class="interpolation-image"
457                       alt="Interpolate start reference image."/>
458              </div>
459              <div class="item item-steve">
460                  <img src="static/images/vis/mots4.gif"
461                       class="interpolation-image"
462                       alt="Interpolate start reference image."/>
463              </div>
464            </div>
465          </div>
466      </div>
467
468
469        <div class="hero-body">
470          <div class="container">
471            <a class="title is-5"> Pose Tracking @ PoseTrack-2018 </a>
472            <div id="results-carousel" class="carousel results-carousel">
473              <div class="item item-steve">
474                  <img src="static/images/vis/posetrack1.gif"
475                       class="interpolation-image"
476                       alt="Interpolate start reference image."/>
477              </div>
478              <div class="item item-steve">
479                  <img src="static/images/vis/posetrack2.gif"
480                       class="interpolation-image"
481                       alt="Interpolate start reference image."/>
482              </div>
483              <div class="item item-steve">
484                  <img src="static/images/vis/posetrack3.gif"
485                       class="interpolation-image"
486                       alt="Interpolate start reference image."/>
487              </div>
488              <div class="item item-steve">
489                  <img src="static/images/vis/posetrack4.gif"
490                       class="interpolation-image"
491                       alt="Interpolate start reference image."/>
492              </div>
493              <div class="item item-steve">
494                  <img src="static/images/vis/posetrack5.gif"
495                       class="interpolation-image"
496                       alt="Interpolate start reference image."/>
497              </div>
498              <div class="item item-steve">
499                  <img src="static/images/vis/posetrack6.gif"
500                       class="interpolation-image"
501                       alt="Interpolate start reference image."/>
502              </div>
503            </div>
504          </div>
505      </div>
506
507
508    </div>
509  </div>
510</section>
511
512
513
514
515<section class="section">
516
517
518  <div class="container is-max-desktop">
519
520
521  
522
523
524    <div class="columns is-centered">
525
526
527      <div class="column is-two-thirds">
528        <div class="content">
529          <h2 class="title is-3">An evaluation platform for SSL models</h2>
530          <p>
531            The "decoupled" characterazition of appearance models in UniTrack allows us to use this framework as an evaluation platform for benchmarking self
531-supervised learning (SSL) models on multiple tracking tasks. And the evaluation does not requrire any training or finetuning!
532          </p>
533
534          <p>
535            In the right we show an example: the SSL model <a href="https://github.com/facebookresearch/moco">MoCo-v1</a> is tested on five tracking tasks.  Each vertex shows the ranking against the other models for a specific tracking task. The larger the area of a polygon is, the better that learned representation performs, across the board.
536          </p>
537
538          <p>
539            Resuls are shown at the <a href="#">top of the page</a>: the recent proposed <a href="https://jerryxu.net/VFS/">VFS</a> (bottom-right of the panel) dominates on all tasks except for SOT. We expect that newly-proposed representations will further improve performance across tasks.
540
541          </p>
542        </div>
543      </div>
544
545
546      <div class="column">
547        <h2 class="title is-3"></h2>
548        <div class="columns is-centered">
549          <div class="column content">
550             <img src="static/images/radar_example.png"
551                 class="interpolation-image"
552                 alt="Interpolate start reference image."/>
553          </img>
554          </div>
555
556        </div>
557      </div>
558
559    </div>
560</section>
561
562
563<section class="section">
564  <div class="container is-max-desktop">
565
566    <h2 class="title is-3">Open-world applications</h2>
567    <div class="columns is-centered">
568      
569        <div class='column is-centered'>
570        <div class="content">
571          <p>
572          In UniTrack, the appearance models are general-purpose representations. This means we can perform class-agnostic tracking in an open-world context. Below we show a demo by combining <a href="https://github.com/Megvii-BaseDetection/YOLOX"> YOLOX </a> with UniTrack. YOLOX is trained on the COCO dataset, so in this demo we can perform Multi-Object Tracking for 80 COCO classes. 
573          </p>
574          <p>
575          The YOLOX detector can be replaced with other detectors if you want to track objects of other classes. 
576          </p>
577        </div>
578      </div>
579    </div>
580    <!--/ Abstract. -->
581
582    <!-- Paper video. -->
583    <div class="columns is-centered has-text-centered">
584      <div class="column">
585        <div class="publication-video">
586          <iframe src="static/images/unitrack_demo.mp4"
587                  frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
588        </div>
589      </div>
590    </div>
591    <!--/ Paper video. -->
592  </div>
593</section>
594
595
596
597
598
599<section class="section" id="BibTeX">
600  <div class="container is-max-desktop content">
601    <h2 class="title">BibTeX</h2>
602    <pre><code>@article{wang2021different,
603  author    = {Wang, Zhongdao and Zhao, Hengshuang and Li, Ya-Li and Wang, Shengjin and Torr, Philip and Bertinetto, Luca},
604  title     = {Do different tracking tasks require different appearance models?},
605  journal   = {Thirty-Fifth Conference on Neural Infromation Processing Systems},
606  year      = {2021},
607}</code></pre>
608  </div>
609</section>
610
611
612<footer class="footer">
613  <div class="container">
614    <div class="content has-text-centered">
615      <a class="icon-link"
616         href="https://arxiv.org/pdf/2107.02156.pdf">
617        <i class="fas fa-file-pdf"></i>
618      </a>
619      <a class="icon-link" href="https://github.com/Zhongdao/UniTrack" class="external-link" disabled>
620        <i class="fab fa-github"></i>
621      </a>
622    </div>
623    <div class="columns is-centered">
624      <div class="column is-8">
625        <div class="content has-text-centered">
626          <p>
627             Website template borrowed from <a
628              href="https://github.com/nerfies/nerfies.github.io">Nerfies</a>.
629          </p>
630        </div>
631      </div>
632    </div>
633  </div>
634</footer>
635
636</body>
637</html>

Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.