PageSourceSearch

https://boyazeng.github.io/understand_bias/

html boyazeng.github.io collected 2026-10-03 08:43:47 UTC 21,932 bytes, 452 lines download raw bytes

1<!DOCTYPE html>
2<html>
3<head>
4  <meta charset="utf-8">
5  <meta name="description"
6        content="Understanding Bias in Large-Scale Visual Datasets.">
7  <meta name="keywords" content="dataset bias, visual datasets">
8  <meta name="viewport" content="width=device-width, initial-scale=1">
9  <title>Understanding Bias in Large-Scale Visual Datasets</title>
10
11  <link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
12        rel="stylesheet">
13
14  <link rel="stylesheet" href="./static/css/bulma.min.css">
15  <link rel="stylesheet" href="./static/css/bulma-carousel.min.css">
16  <link rel="stylesheet" href="./static/css/bulma-slider.min.css">
17  <link rel="stylesheet" href="./static/css/fontawesome.all.min.css">
18  <link rel="stylesheet"
19        href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
20  <link rel="stylesheet" href="./static/css/index.css">
21  <link rel="icon" href="data:,">
22
23  
23<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
23
24  
24<script defer src="./static/js/fontawesome.all.min.js"></script>
24
25  
25<script src="./static/js/bulma-carousel.min.js"></script>
25
26  
26<script src="./static/js/bulma-slider.min.js"></script>
26
27  
27<script src="./static/js/index.js"></script>
27
28</head>
29<body>
30
31<section class="hero">
32  <div class="hero-body">
33    <div class="container is-max-desktop">
34      <div class="columns is-centered">
35        <div class="column has-text-centered">
36          <h1 class="title is-1 publication-title">Understanding Bias in Large-Scale Visual Datasets</h1>
37          <div class="is-size-5 publication-authors">
38            <span class="author-block">
39              <a href="https://boyazeng.github.io">Boya Zeng</a><sup>1</sup>*,</span>
40            <span class="author-block">
41              <a href="https://davidyyd.github.io/">Yida Yin</a><sup>2</sup>*,</span>
42            <span class="author-block">
43              <a href="https://liuzhuang13.github.io/">Zhuang Liu</a><sup>3</sup>
44            </span>
45          </div>
46
47          <div class="is-size-5 publication-authors">
48            <span class="author-block"><sup>1</sup>University of Pennsylvania,</span>
49            <span class="author-block"><sup>2</sup>UC Berkeley</span>,
50            <span class="author-block"><sup>3</sup>Meta FAIR</span><br>
51            <span style="font-size: smaller;">*equal contribution</span>
52          </div>
53
54          <p>
55            <span class="is-size-5"><b>NeurIPS 2024</b><span>
56          </p>
57
58          <div class="column has-text-centered">
59            <div class="publication-links">
60              <!-- PDF Link. -->
61              <span class="link-block">
62                <a href="https://arxiv.org/pdf/2412.01876"
63                   class="external-link button is-normal is-rounded is-dark">
64                  <span class="icon">
65                      <i class="fas fa-file-pdf"></i>
66                  </span>
67                  <span>Paper</span>
68                </a>
69              </span>
70              <span class="link-block">
71                <a href="https://arxiv.org/abs/2412.01876"
72                   class="external-link button is-normal is-rounded is-dark">
73                  <span class="icon">
74                      <i class="ai ai-arxiv"></i>
75                  </span>
76                  <span>arXiv</span>
77                </a>
78              </span>
79              <!-- Video Link. -->
80              <span class="link-block">
81                <a href="https://www.youtube.com/watch?v=7cIaZmMhmZY"
82                   class="external-link button is-normal is-rounded is-dark">
83                  <span class="icon">
84                      <i class="fab fa-youtube"></i>
85                  </span>
86                  <span>Video</span>
87                </a>
88              </span>
89              <!-- Code Link. -->
90              <span class="link-block">
91                <a href="https://github.com/boyazeng/understand_bias"
92                   class="external-link button is-normal is-rounded is-dark">
93                  <span class="icon">
94                      <i class="fab fa-github"></i>
95                  </span>
96                  <span>Code</span>
97                  </a>
98              </span>
99            </div>
100
101          </div>
102          <div class="columns is-centered">
103            <div class="column is-four-fifths">
104              <img src="./static/images/teaser.png" style="max-width: 90%; height: auto;" />
105            </div>
106          </div>
107        </div>
108      </div>
109    </div>
110  </div>
111</section>
112
113
114<section class="section">
115  <div class="container is-max-desktop">
116    <!-- Abstract. -->
117    <div class="columns is-centered has-text-centered">
118      <div class="column is-four-fifths">
119        <h2 class="title is-3">Abstract</h2>
120        <div class="content has-text-justified">
121          <p>
122            A recent study (<a href="https://arxiv.org/abs/2403.08632">Liu and He, 2024</a>) has shown that large-scale visual datasets are very biased: they can be easily classified by modern neural networks. However, the concrete forms of bias among these datasets remain unclear. In this study, we propose a framework to identify the unique visual attributes distinguishing these datasets. Our approach applies various transformations to extract semantic, structural, boundary, color, and frequency information from datasets, and assess how much each type of information reflects their bias. We further decompose their semantic bias with object-level analysis, and leverage natural language methods to generate detailed, open-ended descriptions of each dataset's characteristics. Our work aims to help researchers understand the bias in existing large-scale pre-training datasets, and build more diverse and representative ones in the future.
123          </p>
124        </div>
125      </div>
126    </div>
127    <!--/ Abstract. -->
128  </div>
129</section>
130
131
132<section class="section">
133  <div class="container is-max-desktop">
134
135    
136    <div class="columns is-centered">
137      <div class="column is-full">
138        <h2 class="title is-3 has-text-centered">
138Background: Dataset Classification, Revisited</h2>
139      </div>
140    </div>
141    <p>
142      Dataset classification is a classification task where each dataset forms a class and models are trained to predict the dataset origin of each image.
143      Serving as an indicator of dataset bias, it was proposed by <a href="https://ieeexplore.ieee.org/document/5995347">Torralba and Efros</a> in 2011 on smaller-scale datasets, and recently revisited by <a href="https://arxiv.org/abs/2403.08632">Liu and He</a> on large-scale pre-training datasets.
144    </p>
145    <br>
146    <div class="columns is-centered">
147      <!-- Torralba and Efros. -->
148      <div class="column">
149        <div class="content">
150          <h3 class="title is-4">2011</h3>
151          <p>
152            Back in 2011, the <i>"Name That Dataset"</i> experiment proposed by <a href="https://ieeexplore.ieee.org/document/5995347">Torralba and Efros</a> revealed the built-in bias of visual datasets at that time (Caltech-101, COIL-100, LabelMe, etc.)- the datasets could be classified very well by SVM classifiers.
153          </p>
154          <div style="text-align: center;">
155            <img src="./static/images/bg/torralba_efros.jpg" style="height: 320px; width: auto;">
156          </div>
157        </div>
158      </div>
159      <!--/ Torralba and Efros. -->
160
161      <!-- Liu and He. -->
162      <div class="column">
163        <h3 class="title is-4">2024</h3>
164        <div class="columns is-centered">
165          <div class="column content">
166            <p>
167              Surprisingly, after a decade's effort in creating more diverse and comprehensive visual datasets, the current largest and uncurated datasets (e.g., <a href="https://arxiv.org/abs/1503.01817">YFCC100M</a>, <a href="https://arxiv.org/abs/2102.08981">CC12M</a>, <a href="https://arxiv.org/abs/2304.14108">DataComp-1B</a>) can still be classified with remarkably high accuracy.
168            </p>
169            <div style="text-align: center;">
170              <img src="./static/images/bg/ycd15.png" style="height: 320px; width: auto;">
171            </div>
172          </div>
173        </div>
174      </div>
175    </div>
176    <!--/ Liu and He. -->
177
178    <div class="columns is-centered">
179      <div class="column is-full has-text-centered">
180        <p style="font-size: larger;">
181          Although these large-scale datasets are very biased, a lingering question remains:<br>
182          <span style="color: red; font-size: larger;"><i>what are the concrete forms of bias among them?</i></span>
183        </p>
184      </div>
185    </div>
186
187    <hr>
188
189    <!-- Transformations. -->
190    <div class="columns is-centered">
191      <div class="column is-full-width">
192        <h2 class="title is-3 has-text-centered">Isolating Bias with Transformations</h2>
193
194        <div class="column is-full">
195          <p>
196            To better understand this, we apply various transformations to the datasets, selectively preserving or suppressing specific types of information.
197            We then perform dataset classification on the transformed datasets and analyze its performance.
198          </p>
199        </div>
200        
201        <!-- Reference. -->
202        <h3 class="title is-4">Reference</h3>
203        <div class="content has-text-justified">
204          <div style="text-align: center;">
205            <img src="./static/images/trans/baseline.jpg" style="max-width: 70%; height: auto;">
206          </div>
207          <p>
208            The dataset classification validation accuracy is 82.0% on the original datasets: YFCC, CC, and DataComp (abbreviated as YCD).
209          </p>
210        </div>
211        <!--/ Reference. -->
212        
213        <!-- Semantics. -->
214        <h3 class="title is-4">Semantics</h3>
215        <div class="content has-text-justified">
216          <div style="text-align: center;">
217            <img src="./static/images/trans/trans_semantic.png" style="max-width: 70%; height: auto;">
218          </div>
219          <p>
220            We extract semantic components from the images with decreasing levels of spatial details using semantic segmentation, object detection, and LLaVA captioning.
221            The dataset classification accuracy is consistently high on the resulting datasets of these transformations.
222            Also, passing the images through a VAE to potentially reduce low-level signature only marginally decreases the accuracy from the 82% reference.
223            These highlight that <b><i>semantic bias is an important component of dataset bias in YCD</i></b>.
224          </p>
225        </div>
226        <!--/ Semantics. -->
227
228        <!-- Structures. -->
229        <h3 class="title is-4">
229Structures</h3>
230        <div class="content has-text-justified">
231          <div style="text-align: center;">
232            <img src="./static/images/trans/trans_structure.jpg" style="max-width: 70%; height: auto;">
233          </div>
234          <p>
235            Next, we analyze the semantic bias in its most rudimentary forms, object shape and spatial geometry, using Canny edge, Segment Anything Model (SAM) contour, and depth estimation.
236            The close-to-reference dataset classification accuracies on these transformed datasets show that <b><i>object shape and spatial geometry variations are significant among the YCD datasets</i></b>.
237          </p>
238        </div>
239        <!--/ Structures. -->
240
241        <!-- Spatial Permutations. -->
242        <h3 class="title is-4">Spatial Permutations</h3>
243        <div class="content has-text-justified">
244          <div style="text-align: center;">
245            <img src="./static/images/trans/trans_spatial.jpg" style="max-width: 70%; height: auto;">
246          </div>
247          <div style="display: flex; align-items: center; justify-content: space-around;">
248            <!-- Image on the left -->
249            <div style="flex: 1; text-align: center;">
250              <img src="./static/images/trans/patch_shuf.png" style="max-width: 85%; height: auto;">
251            </div>
252            <!-- Text on the right -->
253            <div style="flex: 1; padding-left: 20px;">
254              <p>
255                We shuffle an image on the pixel and the patch level, following a fixed order and a random order for all images.
256                The significant performance drop with pixel shuffling shows <b><i>completely destructing the local structure in YCD can reduce its dataset bias to a large extent</i></b>. However, the minimal accuracy decrease after shuffling patches of size 16 indicates <b><i>patch-level local structures in spatial information is sufficient for identifying visual signatures of the YCD datasets</i></b>.
257              </p>
258            </div>
259          </div>
260        </div>
261        <!--/ Spatial Permutations. -->
262
263        <!-- RGB. -->
264        <h3 class="title is-4">RGB</h3>
265        <div class="content has-text-justified">
266          <div style="text-align: center;">
267            <img src="./static/images/trans/trans_color.jpg" style="max-width: 70%; height: auto;">
268            <img src="./static/images/trans/ori_dist.png" style="max-width: 85%; height: auto;">
269          </div>
270          <p>
271            Even if we take only the mean RGB value for each image, the datasets can still be classified with higher-than-chance-level accuracy.
272            The distribution of RGB channels for each dataset shows that <b><i>YFCC is much darker than CC and DataComp</i></b>.
273          </p>
274        </div>
275        <!--/ RGB. -->
276
277        <!-- Frequency. -->
278        <h3 class="title is-4">Frequency</h3>
279        <div class="content has-text-justified">
280          <div style="text-align: center;">
281            <img src="./static/images/trans/trans_freq.png" style="max-width: 70%; height: auto;">
282          </div>
283          <p>We perform low-pass and high-pass filters to isolate low-frequency (general structure and smooth variations) and high-frequency (textures and sharp transitions) information.
284            The high accuracy of models trained on either frequency component indicates that <b><i>visual bias in the YCD datasets exists in both low-frequency and high-frequency components</i></b>.
285          </p>
286        </div>
287        <!--/ Frequency. -->
288
289        <!-- Synthetic Image. -->
290        <h3 class="title is-4">Synthetic Image</h3>
291        <div class="content has-text-justified">
292          <div style="text-align: center;">
293            <img src="./static/images/trans/trans_synth.jpg" style="max-width: 70%; height: auto;">
294          </div>
295          <p>
296            We train an unconditional diffusion model on each dataset.
297            Dataset classification on the synthetic data generated from each model still reaches very high accuracy.
298            This shows that the <b><i>synthetic images sampled from a diffusion model can inherit the bias in the model's training images</i></b>.
299            Also, we revert the LLaVA-generated image captions to the image domain using text-to-image diffusion, resulting in a 58% classification accuracy.
300            This further confirms that <b><i>semantic discrepancy is a major contributor to dataset bias</i></b>.
301          </p>
302        </div>
303        <!--/ Synthetic Image. -->
304      </div>
305    </div>
306    <!--/ Transformations. -->
307
308    <div class="columns is-centered">
309      <div class="column is-full has-text-centered">
310        <p style="font-size: larger;">
311          Semantic bias is pronounced in YCD datasets…<br>
312          <span style="color: red; font-size: larger;"><i>how can we explain the semantic patterns?</i></span>
313        </p>
314      </div>
315    </div>
316
317    <hr>
318
319    <!-- Explaining. -->
320    <div class="columns is-centered">
321      <div class="column is-full-width">
322        <h2 class="title is-3 has-text-centered">Explaining Semantic Bias among Datasets</h2>
323
324        <!-- Object-level Queries. -->
325        <h3 class="title is-4">Object-level Queries</h3>
326        <div class="content has-text-justified">
327          <div style="text-align: center;">
328            <div style="text-align: center;">
329              <p>Object classes with highest proportions of YFCC, CC, or DataComp images</p>
330            </div>
331            <img src="./static/images/explain/dist.png" style="max-width: 85%; height: auto;">
332            <div style="text-align: center;">
333              <p>Object class ranking from logistic regression coefficients</p>
334            </div>
335            <img src="./static/images/explain/concept_rank.png" style="max-width: 85%; height: auto;">
336          </div>
337          <br>
338          <p>
339            Based on the image-level object annotations from ImageNet, LVIS, and ADE20k, we can identify the object classes representative of a certain dataset by either looking directly at the object distribution across datasets or their object class rankings according to coefficients of logistic regression model trained on a binary vector representing object presence.
340            YFCC emphasizes outdoor scenes, while CC and DataComp focus on household items and products.
341          </p>
342          <div style="text-align: center;">
343            <img src="./static/images/explain/unique_objects.png" style="max-width: 40%; height: auto;">
344          </div>
345          <p>
346            On average, YFCC contains the highest number of unique objects in each image, while DataComp exhibits the lowest.
347          </p>
348        </div>
349        <!--/ Object-level Queries. -->
350
351        <!-- Open-ended Language-based Analysis. -->
352        <h3 class="title is-4">Open-ended Language-based Analysis</h3>
353        <div class="content has-text-justified">
354          <p>
355            We use LLaVA-generated captions as a proxy to analyze the semantic themes in each dataset. Specifically, we use LDA for unsupervised topic discovery and procedurally prompt an LLM for summarization of the dataset characteristics.
356          </p>
357          <div style="text-align: center;">
358            <div style="text-align: center;">
359              <p>Unsupervised Topic Discovery (Latent Dirichlet Allocation)</p>
360            </div>
361            <img src="./static/images/explain/LDA.png" style="max-width: 85%; height: auto;">
362            <div style="text-align: center;">
363              <p>LLM Summarization</p>
364            </div>
365            <img src="./static/images/explain/llm_summarize_complete.png" style="max-width: 85%; height: auto;">
366          </div>
367          <p>
368            In summary, YFCC is characterized by abundant outdoor, natural, and human-related scenes, while DataComp concentrates on static objects and synthetic images with clean backgrounds and minimal human presence. In contrast, CC blends elements of both YFCC's dynamic scenes and DataComp's static imagery.
369          </p>
370        </div>
371        <!--/ Open-ended Language-based Analysis. -->
372      </div>
373    </div>
374    <!--/ Explaining. -->
375
376    <hr>
377
378    <!-- Discussion. -->
379    <div class="columns is-centered">
380      <div class="column is-full-width">
381        <h2 class="title is-4 has-text-centered">Lessons from the Curation of YFCC, CC, and DataComp</h2>
382      </div>
383    </div>
384    <div class="columns is-centered">
385      <div class="column is-one-third">
386        <!--
387        <div style="text-align: center;">
388          <img src="./static/images/discuss/datacomp.png" style="height: 70px; width: auto;">
389        </div>
390        -->
391        <p><strong>Filtering based on a reference dataset may inherit its bias.</strong></p>
392        <p>DataComp has the fewest unique objects per image. This is possibly because DataComp filters for images with visual content close to ImageNet data in the embedding space. Therefore, the remaining images tend to be object-centric.</p>
393      </div>
394      <div class="column is-one-third">
395        <!--
396        <div style="text-align: center;">
397          <img src="./static/images/discuss/flickr.png" style="height: 70px; width: auto;">
398        </div>
399        -->
400        <p><strong>The source website's image collection mechanism can introduce bias.</strong></p>
401        <p>YFCC is heavily skewed towards outdoor scenes and human interactions. This bias likely stems from its reliance on a single data source, Flickr.com, where user-uploaded content often focuses on personal photos, landscapes, and social interactions.</p>
402      </div>
403      <div class="column is-one-third">
404        <!--
405        <div style="text-align: center;">
406          <img src="./static/images/discuss/commoncrawl.png" style="height: 70px; width: auto;">
407        </div>
408        -->
409        <p><strong>Web-scraped images would naturally contain more digital graphics.</strong></p>
410        <p>Since CC and DataComp images are from Internet webpages, professionally created content like advertisements, infographics, and digital media are prioritized. Dataset users should evaluate if this composition aligns with the downstream goals.</p>
411      </div>
412    </div>
413    <!--/ Discussion. -->
414  </div>
415</section>
416
417
418<section class="section" id="BibTeX">
419  <div class="container is-max-desktop content">
420    <h2 class="title">BibTeX</h2>
421    <pre><code>@inproceedings{zengyin2024bias,
422  title={Understanding Bias in Large-Scale Visual Datasets},
423  author={Boya Zeng and Yida Yin and Zhuang Liu},
424  booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
425  year={2024},
426}</code></pre>
427  </div>
428</section>
429
430
431<footer class="footer">
432  <div class="container">
433    <div class="columns is-centered">
434      <div class="column is-8">
435        <div class="content">
436          <p>
437            This website is licensed under a <a rel="license"
438                                                href="http://creativecommons.org/licenses/by-sa/4.0/">Creative
439            Commons Attribution-ShareAlike 4.0 International License</a>.
440          </p>
441          <p>
442            Website adapted from the following <a
443              href="https://github.com/nerfies/nerfies.github.io">source code</a>.
444          </p>
445        </div>
446      </div>
447    </div>
448  </div>
449</footer>
450
451</body>
452</html>

Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.