1<!DOCTYPE html> 2<html> 3 4<head> 5 <!-- Google tag (gtag.js) -->
vendor: 66 bytes, lines 5-6
5 6 <script async src="https://www.googletagmanager.com/gtag/js?id=
6G-DERZX1PWZ4
vendor: 14 bytes, lines 6-7
6"></script> 7
7<script> 8
vendor: 143 bytes, lines 8-12
8window.dataLayer = window.dataLayer || []; 9 function gtag() { dataLayer.push(arguments); } 10 gtag('js', new Date()); 11 12 gtag('config', '
12G-DERZX1PWZ4
vendor: 6 bytes, lines 12-13
12'); 13
13</script>
13 14 15 <meta name="google-site-verification" content="Bi88m3QClAiFVPoDVcFRkhgAEYFU4M1M4khtU37bUlg" /> 16 <meta charset="utf-8"> 17 <meta name="description" content="Min-K%++"> 18 <meta name="keywords" 19 content="Membership Inference Attack, Copyrighted Information Detection, Dataset Contamination, Data Extraction, LLM, LLaMA, Mamba, GPT, WikiMIA, Min-K, min-k"> 20 <meta name="viewport" content="width=device-width, initial-scale=1"> 21 <title>Min-K%++</title> 22 23 <!-- Google Tag Manager --> 24
24<script>
vendor: 426 bytes, lines 24-31
24(function (w, d, s, l, i) { 25 w[l] = w[l] || []; w[l].push({ 26 'gtm.start': 27 new Date().getTime(), event: 'gtm.js' 28 }); var f = d.getElementsByTagName(s)[0], 29 j = d.createElement(s), dl = l != 'dataLayer' ? '&l=' + l : ''; j.async = true; j.src = 30 'https://www.googletagmanager.com/gtm.js?id=' + i + dl; f.parentNode.insertBefore(j, f); 31 })(window, document, 'script', 'dataLayer', '
31GTM-MFCT45H
vendor: 3 bytes, line 31
31');
31</script>
31 32 <!-- End Google Tag Manager --> 33 34 <link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro" rel="stylesheet"> 35 36 <link rel="stylesheet" href="./static/css/bulma.min.css"> 37 <link rel="stylesheet" href="./static/css/bulma-carousel.min.css"> 38 <link rel="stylesheet" href="./static/css/bulma-slider.min.css"> 39 <link rel="stylesheet" href="./static/css/fontawesome.all.min.css"> 40 <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css"> 41 <link rel="stylesheet" href="./static/css/index.css"> 42 <link rel="icon" href="./static/images/tifa.png"> 43 44
44<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
44 45
45<script defer src="./static/js/fontawesome.all.min.js"></script>
45 46
46<script src="./static/js/bulma-carousel.min.js"></script>
46 47
47<script src="./static/js/bulma-slider.min.js"></script>
47 48
48<script src="./static/js/index.js"></script>
48 49</head> 50 51<body> 52 <!-- Google Tag Manager (noscript) -->
vendor: 70 bytes, lines 52-53
52 53 <noscript><iframe src="https://www.googletagmanager.com/ns.html?id=
53GTM-MFCT45H
vendor: 91 bytes, lines 53-55
53" height="0" width="0" 54 style="display:none;visibility:hidden"></iframe></noscript> 55
55<!-- End Google Tag Manager (noscript) --> 56 57 58 <!-- <nav class="navbar" role="navigation" aria-label="main navigation"> 59 <div class="navbar-brand"> 60 <a role="button" class="navbar-burger" aria-label="menu" aria-expanded="false"> 61 <span aria-hidden="true"></span> 62 <span aria-hidden="true"></span> 63 <span aria-hidden="true"></span> 64 </a> 65 </div> 66 <div class="navbar-menu"> 67 <div class="navbar-start" style="flex-grow: 1; justify-content: center;"> 68 <a class="navbar-item" href="https://yushi-hu.github.io"> 69 <span class="icon"> 70 <i class="fas fa-home"></i> 71 </span> 72 </a> 73 74 <div class="navbar-item has-dropdown is-hoverable"> 75 <a class="navbar-link"> 76 More Research 77 </a> 78 <div class="navbar-dropdown"> 79 <a class="navbar-item" href="https://finegrainedrlhf.github.io/"> 80 Fine-Grained RLHF 81 </a> 82 <a class="navbar-item" href="https://yushi-hu.github.io/promptcap_demo/"> 83 PromptCap 84 </a> 85 </div> 86 </div> 87 </div> 88 89 </div> 90</nav> --> 91 92 93 <section class="hero"> 94 <div class="hero-body"> 95 <div class="container is-max-desktop"> 96 <div class="columns is-centered"> 97 <div class="column has-text-centered"> 98 <h1 class="title is-1 publication-title"> 99 <!--<span class="big-emoji">🕵️</span>--> 100 Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models 101 </h1> 102 <h1 style="font-size: 2em; color: orange;"> [ICLR'25 Spotlight] </h1> 103 <div class="is-size-5"> 104 <span class="author-block"> 105 <a href="http://zjysteven.github.io">Jingyang Zhang</a><sup>1</sup><sup>*</sup>, 106 </span> 107 <span class="author-block"> 108 <a href="https://jingwei-sun.com/">Jingwei Sun</a><sup>1</sup><sup>*</sup>, 109 </span> 110 <span class="author-block"> 111 <a href="https://eric-yeats.com/">Eric Yeats</a><sup>1</sup>, 112 </span> 113 <span class="author-block"> 114 <a href="https://oyy2000.github.io">Yang Ouyang</a><sup>1</sup>, 115 </span><br> 116 <span class="author-block"> 117 <a href="https://www.linkedin.com/in/martinkuo0427/">Martin Kuo</a><sup>1</sup>, 118 </span> 119 <span class="author-block"> 120 <a href="https://cei.pratt.duke.edu/people/jianyi-zhang">Jianyi Zhang</a><sup>1</sup>, 121 </span> 122 <span class="author-block"> 123 <a href="https://www.haofrankyang.net/">Hao Frank Yang</a><sup>1,2</sup>, 124 </span> 125 <span class="author-block"> 126 <a href="https://cei.pratt.duke.edu/people/hai-helen-li">Hai Li</a><sup>1</sup> 127 </span> 128 </div> 129 130 <br> 131 <div class="is-size-5 publication-authors"> 132 <span class="author-block"> <sup>*</sup>Equal Contribution</span><br> 133 <span class="author-block"><sup>1</sup>Duke University</span> 134 <span class="author-block"><sup>2</sup>Johns Hopkins University</span> 135 </div> 136 137 138 <div class="column has-text-centered"> 139 <div class="publication-links"> 140 <span class="link-block"> 141 <a href="https://openreview.net/forum?id=ZGkfoufDaU" class="external-link button is-normal is-rounded is-dark"> 142 <span class="icon"> 143 <i class="fas fa-file-pdf"></i> 144 </span> 145 <span>OpenReview</span> 146 </a> 147 </span> 148 149 <!-- PDF Link. --> 150 <span class="link-block"> 151 <a href="https://arxiv.org/abs/2404.02936" class="external-link button is-normal is-rounded is-dark"> 152 <span class="icon"> 153 <i class="fas fa-file-pdf"></i> 154 </span>
155 <span>ArXiv</span> 156 </a> 157 </span> 158 159 <!-- code Link. --> 160 <span class="link-block"> 161 <a href="https://github.com/zjysteven/mink-plus-plus" class="external-link button is-normal is-rounded is-dark"> 162 <span class="icon"> 163 <i class="fab fa-github"></i> 164 </span> 165 <span>Code</span> 166 </a> 167 </span> 168 169 <!-- Dataset Link. --> 170 <!--<span class="link-block"> 171 <a href="https://huggingface.co/datasets/swj0419/WikiMIA" 172 class="external-link button is-normal is-rounded is-dark"> 173 <span class="icon"> 174 <i class="far fa-images"></i> 175 </span> 176 <span>WikiMIA datasets</span> 177 </a> 178 </span>--> 179 180 <!-- Dataset Link. --> 181 <!--<span class="link-block"> 182 <a href="https://huggingface.co/datasets/swj0419/BookMIA" 183 class="external-link button is-normal is-rounded is-dark"> 184 <span class="icon"> 185 <i class="far fa-images"></i> 186 </span> 187 <span>BookMIA datasets</span> 188 </a> 189 </span>--> 190 191 </div> 192 193 </div> 194 </div> 195 </div> 196 </div> 197 </div> 198 </section> 199 200<section class="hero teaser"> 201 <div class="container is-max-desktop"> 202 <div class="hero-body"> 203 <h2 class="subtitle has-text-centered"> 204 <img src="./static/images/teaser_w_results_v1.png" alt="teaser"> 205 206 207 208 209<h3 class="subtitle"> 210 <style> 211 .subtitle a { 212 color: blue; 213 } 214 </style> 215 We propose a novel method for the detection of pre-training data of LLMs. This problem (see above figure left panel 216 for an illustration) has been receiving growing attention recently, due to its profound implications to copyrighted content detection, 217 privacy auditing, and evaluation data contamination.<br><br> 218 219 Our method, named <b>Min-K%++</b>, is theoretically motivated by revisiting LLM's 220 training objective (maximum likelihood estimation, MLE) through the lens of score matching. We show that the general MLE training under continuous distribution 221 will tend to make training samples be (or locate near) the local maxima along each input dimension. Then, translating this insight into a practical method for LLMs, 222 we propose to detect training data by inspecting whether the tokens in the input text form modes (or have relatively high probability) under the conditional categorical distribution modeled by LLMs.<br><br> 223 224 Empirically, Min-K%++ achieves state-of-the-art performance on the <a href="#wikimia">WikiMIA</a> benchmark, outperforming existing approaches by large margin (showcased by the above figure right panel). 225 On the more challenging <a href="#mimir">MIMIR</a> benchmark, Min-K%++ is also the best among reference-free methods and performs on par with reference-based methods. 226 227</h3> 228 229 230 </div> 231 </div> 232</section> 233 234<!-- 235<section class="section"> 236 <div class="container is-max-desktop"> 237 <div class="columns is-centered has-text-centered"> 238 <div class="column is-four-fifths"> 239 <h2 class="title is-3">Abstract</h2> 240 <div class="content has-text-justified"> 241 <p> 242 Although large language models (LLMs) are widely deployed, the data used to train them is rarely disclosed. 243 Given the 244 incredible scale of this data, up to trillions of tokens, it's nearly certain it includes potentially 245 problematic 246 text such as copyrighted materials, personally identifiable information, and test data for widely reported 247 reference 248 benchmarks. However, we currently lack knowledge on which data of these types is included or in what 249 proportions. 250 </p> 251 252 <li><b>Pretraining Ddata detection problem</b>. In this paper, we explore the pretraining data detection problem<span>🕵️
252</span>: given a piece of 253 text and 254 black-box access to an LLM without knowing the pretraining data, can we determine if the model was trained 255 on the 256 provided text? 257 258 <li><b>Dynamic benchmark WikiMIA</b>. To aid this study, we present <b><a href="https://huggingface.co/datasets/swj0419/WikiMIA" 259 target="_blank"> 260 a dynamic benchmark WikiMIA</a></b> <span>📖</span> that uses data created both before and 261 after model training 262 to support gold truth detection. 263 264 <li><b>Detection method Min-K% Prob</b>. We also design a new <b><a href="https://github.com/swj0419/detect-pretrain-code" target="_blank">detection 265 method Min-K% 266 Prob</a></b> <img src="static/images/min-k_icon.png" alt="Min-K Icon" style="width:25px; height:25px;">. This is built 267 on a straightforward 268 hypothesis: an unobserved example is more likely to have a few outlier words with low probabilities under 269 the LLM, 270 while a recognized example is less inclined to contain words with such reduced probabilities. Min-K% Prob 271 operates 272 without any insight into the pretraining corpus or any extra training, distinguishing it from past detection 273 strategies 274 that necessitate educating a reference model on data analogous to the pretraining data. Furthermore, our 275 tests indicate 276 that Min-K% Prob delivers a 7.4% enhancement on WikiMIA relative to these preceding techniques. 277 278 <li><b>Real-life use cases</b>. We employ 279 Min-K% Prob in 280 three real-life contexts: <strong>benchmark example contamination detection</strong>, 281 <a href="#privacy-auditing"><strong>privacy auditing of machine unlearning</strong></a>, 282 and <a href="#copyrighted-detection"><strong>copyrighted text detection</strong></a> in language models' pretraining 283 data. 284 </div> 285 </div> 286 </div> 287 </div> 288</section> 289--> 290 291 292<!-- 293<section class="section"> 294 <div class="container is-max-desktop"> 295 <div class="columns is-centered has-text-centered"> 296 <div class="column is-four-fifths"> 297 <h2 class="title is-3">Detection Method Min-K% Prob</h2> 298 <div class="content has-text-justified"> 299 <p> 300 <strong>What is Min-K% Prob?</strong><br> 301 We propose a pretraining data detection method named Min-K% Prob. Our method is based on a simple 302 hypothesis: an unseen example tends to contain a few outlier words with low probabilities, whereas a seen 303 example is less likely to contain words with such low probabilities. MIN-K% Prob computes the average 304 probabilities of outlier tokens. 305 </p> 306 307 <p> 308 <strong>How to use Min-K% Prob?</strong><br> 309 To check if a text was in LLM's pretraining: 310 <ol> 311 <li>Evaluate token probabilities in the text.</li> 312 <li>Pick the k% tokens with minimum probabilities.</li> 313 <li>Compute their average log likelihood.</li> 314 </ol> 315 If the average log likelihood is high, the text is likely in the pretraining data. â 316 </p> 317 318 <a href="https://arxiv.org/pdf/2310.16789.pdf" target="_blank">See more results in our paper</a> 319 </div> 320 </div> 321 </div> 322 </div> 323</section> 324--> 325 326 327<section class="section"> 328 <div class="container is-max-desktop"> 329 <div class="columns is-centered has-text-centered"> 330 <div class="column is-four-fifths"> 331 <!-- <h2 class="title is-3" id="copyrighted-detection">Copyrighted Book Detection</h2> --> 332 <h2 class="title is-3" id="wikimia"><a href="https://github.com/swj0419/detect-pretrain-code/">WikiMIA</a> results</h2> 333 <div class="content has-text-justified"> 334 335 <style> 336 table { 337 width: 100%; 338 border-collapse: collapse; 339 } 340 341 table, 342 th, 343 td { 344 border: 1px solid black; 345 } 346 347 th, 348 td { 349 padding: 8px; 350 text-align: left; 351 } 352 353 th { 354 background-color: #f2f2f2; 355 } 356 </style> 357 </head> 358 359 <body> 360 361 <table> 362 <caption>Detection AUROC (%) on WikiMIA_length32. <b>Min-K%++ achieves significantly improved results over Min-K% and other existing methods.</b> 363 For more results, don't hesitate to check our <a href="https://arxiv.org/abs/2404.02936">paper</a>. 364 </caption> 365 <thead> 366 <tr> 367 <th>Method</th> 368 <th>Mamba-1.4B</th> 369 <th>Pythia-6.9B</th> 370 <th>LLaMA-13B</th> 371 <th>LLaMA-30B</th> 372 <th>LLaMA-65B</th> 373 <th>Average</th> 374 </tr> 375 </thead> 376 <tbody> 377 <tr> 378 <td>Loss</td> 379 <td>61.0</td> 380 <td>63.8</td> 381 <td>67.5</td> 382 <td>69.4</td> 383 <td>70.7</td> 384 <td>66.5</td> 385 </tr> 386 387 <tr> 388 <td>Ref</td> 389 <td>62.2</td> 390 <td>63.6</td> 391 <td>57.9</td> 392 <td>63.5</td> 393 <td>68.8</td> 394 <td>63.2</td> 395 </tr> 396 397 <tr> 398 <td>Lowercase</td> 399 <td>60.9</td> 400 <td>62.2</td> 401 <td>64.0</td> 402 <td>64.1</td> 403 <td>66.5</td> 404 <td>63.5</td> 405 </tr> 406 407 <tr> 408 <td>Zlib</td> 409 <td>61.9</td> 410 <td>64.3</td> 411 <td>67.8</td> 412 <td>69.8</td> 413 <td>71.1</td> 414 <td>67.0</td> 415 </tr> 416 417 <tr> 418 <td>Neighbor</td> 419 <td>64.1</td> 420 <td>65.8</td> 421 <td>65.8</td> 422 <td>67.6</td> 423 <td>69.6</td> 424 <td>66.6</td> 425 </tr> 426 427 <tr> 428 <td>Min-K%</td> 429 <td>63.2</td> 430 <td>66.3</td> 431 <td>68.0</td> 432 <td>70.1</td> 433 <td>71.3</td> 434 <td>67.8</td> 435 </tr> 436 437 <tr> 438 <td>Min-K%++</td> 439 <td><b>66.8</b></td> 440 <td><b>70.3</b></td> 441 <td><b>84.8</b></td> 442 <td><b>84.3</b></td> 443 <td><b>85.1</b></td> 444 <td><b>78.3</b></td> 445 </tr> 446 447 </tbody> 448 </table> 449 450 </body> 451 452 </html> 453 454 </div> 455 </div> 456</section> 457 458 459<section class="section"> 460 <div class="container is-max-desktop"> 461 <div class="columns is-centered has-text-centered"> 462 <div class="column is-four-fifths"> 463 <h2 class="title is-3" id="mimir"><a href="https://github.com/iamgroot42/mimir">MIMIR</a> results</h2> 464 <div class="content has-text-justified"> 465 466 <style> 467 table { 468 width: 100%; 469 border-collapse: collapse; 470 } 471 472 table, 473 th, 474 td { 475 border: 1px solid black; 476 } 477 478 th, 479 td { 480 padding: 8px; 481 text-align: left; 482 } 483 484 th { 485 background-color: #f2f2f2; 486 } 487 </style> 488 </head> 489 490 <body> 491 492 <table> 493 <caption> 494 Detection AUROC (%) on MIMIR averaged over 7 subdomains. 495 The best result is <b>bolded</b>, with the runner-up <u>underlined</u>. 496 <b>Min-K%++ achieves SOTA among reference-free methods and performs on par with the Ref method which requires an extra reference LLM.</b> 497 </caption> 498 <thead> 499 <tr> 500 <th>Method</th> 501 <th>Pythia-160M</th> 502 <th>Pythia-1.4B</th> 503 <th>Pythia-2.8B</th> 504 <th>Pythia-6.9B</th> 505 <th>Pythia-12B</th> 506 </tr> 507 </thead> 508 <tbody> 509 <tr> 510 <td>Loss</td> 511 <td>52.1</td> 512 <td>53.1</td> 513 <td>53.5</td> 514 <td>54.4</td> 515 <td>54.9</td> 516 </tr> 517 518 <tr> 519 <td>Ref</td> 520 <td>52.2</td> 521 <td><b>54.6</b></td> 522 <td><b>55.6</b></td> 523 <td><b>57.4</b></td> 524 <td><u>58.7</u></td> 525 </tr> 526 527 <tr> 528 <td>Zlib</td> 529 <td>52.3</td> 530 <td>53.2</td> 531 <td>53.6</td> 532 <td>54.3</td> 533 <td>
53354.8</td> 534 </tr> 535 536 <tr> 537 <td>Neighbor</td> 538 <td>52.0</td> 539 <td>52.9</td> 540 <td>53.2</td> 541 <td>53.8</td> 542 <td>/</td> 543 </tr> 544 545 <tr> 546 <td>Min-K%</td> 547 <td><b>52.6</b></td> 548 <td>53.6</td> 549 <td>54.2</td> 550 <td>55.2</td> 551 <td>55.9</td> 552 </tr> 553 554 <tr> 555 <td>Min-K%++</td> 556 <td><u>52.4</u></td> 557 <td><u>54.1</u></td> 558 <td><u>55.3</u></td> 559 <td><u>57.0</u></td> 560 <td><b>58.7</b></td> 561 </tr> 562 563 </tbody> 564 </table> 565 566 567 </div> 568 </div> 569 </div> 570 </div> 571 </section> 572 573 574<section class="section is-light" id="BibTeX"> 575 <div class="container is-max-desktop content"> 576 <h2 class="title">BibTeX</h2> 577 <pre><code> 578@inproceedings{ 579 zhang2025mink, 580 title={Min-K\%++: Improved Baseline for Pre-Training Data Detection from Large Language Models}, 581 author={Jingyang Zhang and Jingwei Sun and Eric Yeats and Yang Ouyang and Martin Kuo and Jianyi Zhang and Hao Frank Yang and Hai Li}, 582 booktitle={The Thirteenth International Conference on Learning Representations}, 583 year={2025}, 584 url={https://openreview.net/forum?id=ZGkfoufDaU} 585} 586</code></pre> 587 </div> 588 </section> 589 590 591 592 <footer class="footer"> 593 <div class="container"> 594 <div class="columns is-centered"> 595 <div class="column is-8"> 596 <div class="content"> 597 <p> 598 This website is licensed under a <a rel="license" 599 href="http://creativecommons.org/licenses/by-sa/4.0/">Creative 600 Commons Attribution-ShareAlike 4.0 International License</a>. This means you are free to borrow the <a href="https://github.com/nerfies/nerfies.github.io">source 601 code</a> of this website, we just ask that you link back to this page in the footer. 602 </p> 603 </div> 604 </div> 605 </div> 606 </div> 607 </footer> 608 609</body> 610 611</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.