1<!DOCTYPE html> 2<html lang="en"> 3<head> 4 <meta charset="UTF-8"> 5 <meta name="viewport" content="width=device-width, initial-scale=1.0"> 6 <title>Publications - Mikio L. Braun</title> 7 8 <!-- OpenGraph / Social Meta Tags --> 9 10 <meta property="og:title" content="Mikio L. Braun"> 11 <meta property="og:description" content="ML and AI expert"> 12 <meta property="og:type" content="website"> 13 <meta name="twitter:card" content="summary"> 14 <meta name="twitter:title" content="Mikio L. Braun"> 15 <meta name="twitter:description" content="ML and AI expert"> 16 17 18
18<script src="https://cdn.tailwindcss.com?plugins=typography"></script>
18 19
19<script> 20 tailwind.config = { 21 theme: { 22 extend: { 23 fontFamily: { 24 sans: ['Inter', 'system-ui', 'sans-serif'], 25 }, 26 } 27 } 28 } 29 </script>
29 30 <link rel="preconnect" href="https://fonts.googleapis.com"> 31 <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> 32 <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet"> 33</head> 34<body class="bg-white text-gray-900 min-h-screen flex flex-col"> 35 <!-- Debug Site Switcher --> 36 37 38 <!-- Header --> 39 <header class="bg-white border-b border-gray-200"> 40 <div class="max-w-4xl mx-auto px-4 sm:px-6 py-6"> 41 <div class="flex flex-col sm:flex-row sm:items-center sm:justify-between gap-4"> 42 <div> 43 <a href="/" class="text-2xl font-bold text-gray-900 hover:text-gray-700 transition-colors"> 44 Mikio L. Braun 45 </a> 46 <p class="text-sm text-gray-500 mt-1">ML and AI expert</p> 47 </div> 48 <nav class="flex flex-wrap gap-4 text-sm"> 49 <a href="/" class="text-gray-600 hover:text-gray-900 transition-colors">Home</a> 50 <a href="/about" class="text-gray-600 hover:text-gray-900 transition-colors">About</a> 51 <a href="/projects" class="text-gray-600 hover:text-gray-900 transition-colors">Projects</a> 52 <a href="/software" class="text-gray-600 hover:text-gray-900 transition-colors">Software</a> 53 <a href="/publications" class="text-gray-600 hover:text-gray-900 transition-colors">Publications</a> 54 <a href="/research" class="text-gray-600 hover:text-gray-900 transition-colors">Research</a> 55 <a href="/teaching" class="text-gray-600 hover:text-gray-900 transition-colors">Teaching</a> 56 </nav> 57 </div> 58 </div> 59 </header> 60 61 <!-- Main Content --> 62 <main class="flex-1 max-w-4xl mx-auto px-4 sm:px-6 py-8 w-full"> 63 64<div class="max-w-3xl"> 65 <h1 class="text-3xl font-bold mb-6">Publications</h1> 66 67 68 69 70 <section class="mb-10"> 71 <h2 class="text-xl font-bold mb-2 text-gray-800">General Machine Learning</h2> 72 73 74 <div class="space-y-4"> 75 76 <div class="border-l-2 border-gray-200 pl-4 py-2"> 77 <p class="font-medium text-gray-900">Fast Cross-Validation via Sequential Analysis</p> 78 <p class="text-sm text-gray-600">T. Krüger, D. Panknin, M. Braun</p> 79 <p class="text-sm text-gray-500 mt-1"> 80 <p>NIPS BigLearning Workshop, 2011</p> 81 </p> 82 83 <a href="https://www.scribd.com/doc/76134303" 84 class="text-blue-600 hover:text-blue-800 text-sm" 85 target="_blank"> 86 View paper 87 </a> 88 89 90 <details class="mt-2"> 91 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 92 Show abstract 93 </summary> 94 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 95 With the increasing size of todayâs data sets, ï¬nding the right 96parameter conï¬guration via cross-validation can be an extremely 97time-consuming task. In this paperwe propose an improved 98cross-validation procedure which uses non-parametrictesting 99coupled with sequential analysis to determine the best parameter 100set on linearly increasing subsets of the data. By eliminating 101underperforming candidatesquickly and keeping promising candidates 102as long as possible the method speedsup the computation while 103preserving the capability of the full cross-validation.The 104experimental evaluation shows that our method reduces the 105computation timeby a factor of up to 70 compared to a full 106cross-validation with a negligible impacton the accuracy. 107 108 </p> 109 </details> 110 111 </div> 112 113 </div> 114 </section> 115 116 117 118 119 <section class="mb-10"> 120 <h2 class="text-xl font-bold mb-2 text-gray-800">Social Media Analysis</h2> 121 122 <p class="text-gray-600 text-sm mb-4">Analysis of Twitter data; Impact Analysis; Big Data, Real-Time; 123Stream Mining. 124</p> 125 126 127 <div class="space-y-4"> 128 129 <div class="border-l-2 border-gray-200 pl-4 py-2"> 130 <p class="font-medium text-gray-900">Real-time social media analysis with TWIMPACT</p> 131 <p class="text-sm text-gray-600">M. Braun, M. Jugel, K.-R. Müller</p> 132 <p class="text-sm text-gray-500 mt-1"> 133 <p>Demonstration at NIPS 2011.</p> 134 </p> 135 136 <a href="https://twimpact.tumblr.com/post/14164936717/preview-of-our-nips-poster-come-to-our-demo-on" 137 class="text-blue-600 hover:text-blue-800 text-sm" 138 target="_blank"> 139 View paper 140 </a> 141 142 143 <details class="mt-2"> 144 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 145 Show abstract 146 </summary> 147 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 148 Social media analysis has attracted quite some interest 149recently. Typical questions are identifying current trends, rating 150the impact or influence of users, summarizing what people are 151saying on some topic or brand, and so on. One challenge is the 152enormous amount of data you need to process. Current approaches 153often need to filter the data first and then do the actual 154analysis after the event. However, ideally you would want to be 155able to look into the stream of social media updates in real-time 156as the event unfolds, and not a few days or weeks later when 157you've processed your data. We will showcase the real-time social 158media analysis engine developed by TWIMPACT in cooperation with 159the TU Berlin. You will be able to look at the current trends on 160Twitter in real-time, in addition to being able to have a look at 161the last year which we will bring in a pre-analyzed form, such 162that you can zoom in on the historic events of this year like the 163revolutions in Egypt, or Libya, and so on. 164 165 </p> 166 </details> 167 168 </div> 169 170 </div> 171 </section> 172 173 174 175 176 <section class="mb-10"> 177 <h2 class="text-xl font-bold mb-2 text-gray-800">(Deep) Neural Networks</h2> 178 179 <p class="text-gray-600 text-sm mb-4">Studying how representations are built in (deep) neural networks 180using the techniques developed for kernel methods. 181</p> 182 183 184 <div class="space-y-4"> 185 186 <div class="border-l-2 border-gray-200 pl-4 py-2"> 187 <p class="font-medium text-gray-900">Deep Boltzmann Machines as Feed-Forward Hierarchies</p> 188 <p class="text-sm text-gray-600">G. Montavon, M. Braun, K.-R. Müller</p> 189 <p class="text-sm text-gray-500 mt-1"> 190 <p>International Conference on Artificial Intelligence and Statistics 191(AISTATS), 2012</p> 192 </p> 193 194 <a href="https://gregoire.montavon.name/publications/montavon-aistats12.pdf" 195 class="text-blue-600 hover:text-blue-800 text-sm" 196 target="_blank"> 197 View paper 198 </a> 199 200 201 <details class="mt-2"> 202 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 203 Show abstract 204 </summary> 205 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 206 The deep Boltzmann machine is a powerful model that extracts the 207hierarchical structure of observed data. While inference is 208typically slow due to its undirected nature, we argue that the 209emerging feature hierarchy is still explicit enough to be 210traversed in a feed-forward fashion. The claim is corroborated by 211training a set of deep neural networks on real data and measuring 212the evolution of the representation layer after layer. The 213analysis reveals that the deep Boltzmann machine produces a 214feed-forward hierarchy of increasingly invariant representations 215that clearly surpasses the layer-wise approach. 216 217 </p> 218 </details> 219 220 </div> 221 222 <div class="border-l-2 border-gray-200 pl-4 py-2"> 223 <p class="font-medium text-gray-900">Importance of Cross-Layer Cooperation for Learning Deep Feature Hierarchies</p> 224 <p class="text-sm text-gray-600">G. Montavon, M. Braun, K.-R. Müller</p> 225 <p class="text-sm text-gray-500 mt-1"> 226 <p>NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011</p> 227 </p> 228 229 <a href="Importance of Cross-Layer Cooperation for Learning Deep Feature Hierarchies" 230 class="text-blue-600 hover:text-blue-800 text-sm" 231 target="_blank"> 232 View paper 233 </a> 234 235 236 <details class="mt-2"> 237 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 238 Show abstract 239 </summary> 240 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 241 A common property of hierarchical models of the brain is their 242capacity to integrate bottom-up and top-down information in order 243to distill the task-relevant information from the sensory 244noise. In this paper, we argue that such cooperation between upper 245and lower layers is not only useful at prediction time but also at 246learning time in order to build a successful feature 247hierarchy. The claim is corroborated by training a set of deep 248networks on real data and measuring the evolution of the 249representation layer after layer. The analysis reveals that 250crosslayer cooperation enables the emergence of a sequence of 251increasingly invariant representations. 252 253 </p> 254 </details> 255 256 </div> 257 258 <div class="border-l-2 border-gray-200 pl-4 py-2"> 259 <p class="font-medium text-gray-900">Kernel Analysis of Deep Networks</p> 260 <p class="text-sm text-gray-600">G. Montavon, M. Braun, K.-R. Müller</p> 261 <p class="text-sm text-gray-500 mt-1"> 262 <p>Journal of Machine Learning Research (JMLR), 12, pp. 2563-2581, 2011</p> 263 </p> 264 265 <a href="https://jmlr.csail.mit.edu/papers/v12/montavon11a.html" 266 class="text-blue-600 hover:text-blue-800 text-sm" 267 target="_blank"> 268 View paper 269 </a> 270 271 272 <details class="mt-2"> 273 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 274 Show abstract 275 </summary> 276 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 277 When training deep networks it is common knowledge that an 278efficient and well generalizing representation of the problem is 279formed. In this paper we aim to elucidate what makes the emerging 280representation successful. We analyze the layer-wise evolution of 281the representation in a deep network by building a sequence of 282deeper and deeper kernels that subsume the mapping performed by 283more and more layers of the deep network and measuring how these 284increasingly complex kernels fit the learning problem. We observe 285that deep networks create increasingly better representations of 286the learning problem and that the structure of the deep network 287controls how fast the representation of the task is formed layer 288after layer. 289 290 </p> 291 </details> 292 293 </div> 294 295 <div class="border-l-2 border-gray-200 pl-4 py-2"> 296 <p class="font-medium text-gray-900">Layer-wise analysis of deep networks with Gaussian kernels</p> 297 <p class="text-sm text-gray-600">G. Montavon, M. Braun, K.-R. Müller</p> 298 <p class="text-sm text-gray-500 mt-1"> 299 <p>Advances in Neural Information Processing Systems 23 (NIPS 3002010), pp. 1678-1686, 2010</p> 301 </p> 302 303 <a href="https://books.nips.cc/papers/files/nips23/NIPS2010_0206.pdf" 304 class="text-blue-600 hover:text-blue-800 text-sm" 305 target="_blank"> 306 View paper 307 </a> 308 309 310 <details class="mt-2"> 311 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 312 Show abstract 313 </summary> 314 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 315 Deep networks can potentially express a learning problem more 316efficiently than local learning machines. While deep networks 317outperform local learning machines on some problems, it is still 318unclear how their nice representation emerges from their complex 319structure. We present an analysis based on Gaussian kernels that 320measures how the representation of the learning problem evolves 321layer after layer as the deep network builds higher-level abstract 322representations of the input. We use this analysis to show 323empirically that deep networks build progressively bet- ter 324representations of the learning problem and that the best 325representations are obtained when the deep network discriminates 326only in the last layers. 327 328 </p> 329 </details> 330 331 </div> 332 333 </div> 334 </section> 335 336 337 338 339 <section class="mb-10"> 340 <h2 class="text-xl font-bold mb-2 text-gray-800">Machine Learning Open Source Software</h2> 341 342 343 <div class="space-y-4"> 344 345 <div class="border-l-2 border-gray-200 pl-4 py-2"> 346 <p class="font-medium text-gray-900">The Need for Open Source Software in Machine Learning</p> 347 <p class="text-sm text-gray-600">S. Sonnenburg, M. L. Braun, C. S. Ong, S. Bengio, L. Bottou, 348G. Holmes, Y. LeCun, K.-R. M&uuml;ller, F. Pereira, C. E. Rasmussen, 349G. R&auml;tsch, B. Sch&ouml;lkopf, A. Smola, P. Vincent, J. Weston, 350R. Williamson 351</p> 352 <p class="text-sm text-gray-500 mt-1"> 353 <p>Journal of Machine Learning Research, 8(Oct):2443-2466, 2007</p> 354 </p> 355 356 <a href="https://jmlr.csail.mit.edu/papers/v8/sonnenburg07a.html" 357 class="text-blue-600 hover:text-blue-800 text-sm" 358 target="_blank"> 359 View paper 360 </a> 361 362 363 <details class="mt-2"> 364 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 365 Show abstract 366 </summary> 367 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 368 Open source tools have recently reached a level of maturity 369which makes them suitable for building large-scale real-world 370systems. At the same time, the field of machine learning has 371developed a large body of powerful learning algorithms for diverse 372applications. However, the true potential of these methods is not 373used, since existing implementations are not openly shared, 374resulting in software with low usability, and weak 375interoperability. We argue that this situation can be 376significantly improved by increasing incentives for researchers to 377publish their software under an open source model. Additionally, 378we outline the problems authors are faced with when trying to 379publish algorithmic implementati
379ons of machine learning 380methods. We believe that a resource of peer reviewed software 381accompanied by short articles would be highly valuable to both the 382machine learning and the general scientific community. 383 384 </p> 385 </details> 386 387 </div> 388 389 </div> 390 </section> 391 392 393 394 395 <section class="mb-10"> 396 <h2 class="text-xl font-bold mb-2 text-gray-800">Kernel Methods</h2> 397 398 <p class="text-gray-600 text-sm mb-4">Eigenstructure of the Kernel Matrix; Kernel PLS; Kernel Methods</p> 399 400 401 <div class="space-y-4"> 402 403 <div class="border-l-2 border-gray-200 pl-4 py-2"> 404 <p class="font-medium text-gray-900">Lanczos Approximations for the Speedup of Kernel Partial 405Least Squares Regression 406</p> 407 <p class="text-sm text-gray-600">N. Kr&auml;mer, M. Sugiyama, M.L. Braun</p> 408 <p class="text-sm text-gray-500 mt-1"> 409 <p>JMLR Workshop and Conference Proceedings, Volume 5: AISTATS 2009</p> 410 </p> 411 412 <a href="https://jmlr.csail.mit.edu/proceedings/papers/v5/kramer09a/kramer09a.pdf" 413 class="text-blue-600 hover:text-blue-800 text-sm" 414 target="_blank"> 415 View paper 416 </a> 417 418 419 <details class="mt-2"> 420 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 421 Show abstract 422 </summary> 423 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 424 The runtime for Kernel Partial Least Squares (KPLS) to compute the 425fit is quadratic in the number of examples. However, the necessity 426of obtaining sensitivity measures as degrees of freedom for model 427selection or confidence intervals for more detailed analysis 428requires cubic runtime, and thus constitutes a computational 429bottleneck in real-world data analysis. We propose a novel 430algorithm for KPLS which not only computes (a) the fit, but also 431(b) its approximate degrees of freedom and (c) error bars in 432quadratic runtime. The algorithm exploits a close connection 433between Kernel PLS and the Lanczos algorithm for approximating the 434eigenvalues of symmetric matrices, and uses this approximation to 435compute the trace of powers of the kernel matrix in quadratic runtime. 436 437 </p> 438 </details> 439 440 </div> 441 442 <div class="border-l-2 border-gray-200 pl-4 py-2"> 443 <p class="font-medium text-gray-900">On Relevant Dimensions in Kernel Feature Spaces</p> 444 <p class="text-sm text-gray-600">M.L. Braun, J. Buhmann, K.-R. M&uuml;ller</p> 445 <p class="text-sm text-gray-500 mt-1"> 446 <p>JMLR 9(Aug):1875--1908, 2008. (<a href="software.html#rdetools">software</a>) 447(<a href="research.html#rde">more information</a>)</p> 448 </p> 449 450 <a href="https://www.jmlr.org/papers/volume9/braun08a/braun08a.pdf" 451 class="text-blue-600 hover:text-blue-800 text-sm" 452 target="_blank"> 453 View paper 454 </a> 455 456 457 <details class="mt-2"> 458 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 459 Show abstract 460 </summary> 461 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 462 We show that the relevant information of a supervised learning 463problem is contained up to negligible error in a finite number of 464leading kernel PCA components if the kernel matches the underlying 465learning problem in the sense that it can asymptotically represent 466the function to be learned and is sufficiently smooth. Thus, 467kernels do not only transform data sets such that good 468generalization can be achieved using only linear discriminant 469functions, but this transformation is also performed in a manner 470which makes economical use of feature space dimensions. In the 471best case, kernels provide efficient implicit representations of 472the data for supervised learning problems. Practically, we propose 473an algorithm which enables us to recover the number of leading 474kernel PCA components relevant for good classification. Our 475algorithm can therefore be applied (1) to analyze the interplay of
476data set and kernel in a geometric fashion, (2) to aid in model 477selection, and (3) to denoise in feature space in order to yield 478better classification results. 479 480 </p> 481 </details> 482 483 </div> 484 485 <div class="border-l-2 border-gray-200 pl-4 py-2"> 486 <p class="font-medium text-gray-900">Kernelizing PLS, Degress of Freedom, and Efficient Model Selection</p> 487 <p class="text-sm text-gray-600">N. Kr&auml;mer, M.L. Braun</p> 488 <p class="text-sm text-gray-500 mt-1"> 489 <p>ICML 2007</p> 490 </p> 491 492 <a href="https://ml.cs.tu-berlin.de/~nkraemer/papers/icml_kernelpls.pdf" 493 class="text-blue-600 hover:text-blue-800 text-sm" 494 target="_blank"> 495 View paper 496 </a> 497 498 499 <details class="mt-2"> 500 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 501 Show abstract 502 </summary> 503 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 504 Kernelizing partial least squares (PLS), an algorithm which has 505been particularly popular in chemometrics, leads to kernel PLS 506which has several interesting properties, including a sub-cubic 507runtime for learning, and an iterative construction of directions 508which are relevant for predicting the outputs. We show that the 509kernelization of PLS introduces interesting properties not found 510in ordinary PLS, giving novel insights into the workings of kernel 511PLS and the connections to kernel ridge regression and conjugate 512gradient descent methods. Furthermore, we show how to correctly 513define the degrees of freedom for kernel PLS and how to 514efficiently compute an unbiased estimate. Finally, we address the 515practical problem of model selection. We demonstrate how to use 516the degrees of freedom estimate to perform effective model 517selection, and discuss how to implement cross-validation schemes 518efficiently. 519 520 </p> 521 </details> 522 523 </div> 524 525 <div class="border-l-2 border-gray-200 pl-4 py-2"> 526 <p class="font-medium text-gray-900">Accurate bounds for the eigenvalues of the kernel matrix</p> 527 <p class="text-sm text-gray-600">M. L. Braun</p> 528 <p class="text-sm text-gray-500 mt-1"> 529 <p>Journal of Machine Learning Research, Vol. 7(Nov) 2303-2328, 2006</p> 530 </p> 531 532 <a href="https://jmlr.csail.mit.edu/papers/v7/braun06a.html" 533 class="text-blue-600 hover:text-blue-800 text-sm" 534 target="_blank"> 535 View paper 536 </a> 537 538 539 <details class="mt-2"> 540 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 541 Show abstract 542 </summary> 543 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 544 The eigenvalues of the kernel matrix play an important role in a 545number of kernel methods, in particular, in kernel principal 546component analysis. It is well known that the eigenvalues of the 547kernel matrix converge as the number of samples tends to 548infinity. We derive probabilistic finite sample size bounds on the 549approximation error of individual eigenvalues which have the 550important property that the bounds scale with the eigenvalue under 551consideration, reflecting the actual behavior of the approximation 552errors as predicted by asymptotic results and observed in 553numerical simulations. Such scaling bounds have so far only been 554known for tail sums of eigenvalues. Asymptotically, the bounds 555presented here have a slower than stochastic rate, but the number 556of sample points necessary to make this disadvantage noticeable is 557often unrealistically large. Therefore, under practical 558conditions, and for all but the largest few eigenvalues, the 559bounds presented here form a significant improvement over existing 560non-scaling bounds. 561 562 </p> 563 </details> 564 565 </div> 566 567 <div class="border-l-2 border-gray-200 pl-4 py-2"> 568 <p class="font-medium text-gray-900">Denoising and Dimension Reduction in Feature Space</p> 569 <p class="text-sm text-gray-600">M.L. Braun, J. Buhmann, K.-R. M&uuml;ller</p> 570 <p class="text-sm text-gray-500 mt-1"> 571 <p>Advances in Neural Information Processing Systems 19 (NIPS 2006), 572B. Schölkopf, J. Platt, and T. Hoffman (eds.), MIT Press,
573Cambridge, MA, 185-192, 2007</p> 574 </p> 575 576 <a href="https://books.nips.cc/papers/files/nips19/NIPS2006_0407.pdf" 577 class="text-blue-600 hover:text-blue-800 text-sm" 578 target="_blank"> 579 View paper 580 </a> 581 582 583 <details class="mt-2"> 584 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 585 Show abstract 586 </summary> 587 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 588 We show that the relevant information about a classification 589problem in feature space is contained up to negligible error in a 590finite number of leading kernel PCA components if the kernel 591matches the underlying learning problem. Thus, ker- nels not only 592transform data sets such that good generalization can be achieved 593even by linear discriminant functions, but this transformation is 594also performed in a manner which makes economic use of feature 595space dimensions. In the best case, kernels provide efficient 596implicit representations of the data to perform 597classification. Practically, we propose an algorithm which enables 598us to recover the subspace and dimensionality relevant for good 599classification. Our algorithm can therefore be applied (1) to 600analyze the interplay of data set and kernel in a geo- metric 601fashion, (2) to help in model selection, and to (3) de-noise in 602feature space in order to yield better classification 603results. 604 605 </p> 606 </details> 607 608 </div> 609 610 <div class="border-l-2 border-gray-200 pl-4 py-2"> 611 <p class="font-medium text-gray-900">Model Selection in Kernel Methods based on a Spectral Analysis of 612Label Information 613</p> 614 <p class="text-sm text-gray-600">M.L. Braun, T. Lange, J. Buhmann</p> 615 <p class="text-sm text-gray-500 mt-1"> 616 <p>DAGM 2006, LNCS 4174, pp. 344-353</p> 617 </p> 618 619 <a href="https://www.springerlink.com/content/48481751l288w5q0/" 620 class="text-blue-600 hover:text-blue-800 text-sm" 621 target="_blank"> 622 View paper 623 </a> 624 625 626 <details class="mt-2"> 627 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 628 Show abstract 629 </summary> 630 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 631 We propose a novel method for addressing the model selection 632problem in the context of kernel methods. In contrast to existing 633methods which rely on hold-out testing or try to compensate for 634the optimism of the generalization error, our method is based on a 635structural analysis of the label information using the 636eigenstructure of the kernel matrix. In this setting, the label 637vector can be transformed into a representation in which the 638smooth information is easily discernible from the noise. This 639permits to estimate a cut-off dimension such that the leading 640coefficients in that representation contains the learnable 641information, discarding the noise. Based on this cut-off 642dimension, the regularization parameter is estimated for kernel 643ridge regression. 644 645 </p> 646 </details> 647 648 </div> 649 650 <div class="border-l-2 border-gray-200 pl-4 py-2"> 651 <p class="font-medium text-gray-900">Spectral Properties of the Kernel Matrix and their Relation 652to Kernel Methods in Machine Learning 653</p> 654 <p class="text-sm text-gray-600">M. L. Braun</p> 655 <p class="text-sm text-gray-500 mt-1"> 656 <p>Ph.D. thesis, University of Bonn, 2005. Published electronically.</p> 657 </p> 658 659 <a href="https://hss.ulb.uni-bonn.de/diss_online/math_nat_fak/2005/braun_mikio" 660 class="text-blue-600 hover:text-blue-800 text-sm" 661 target="_blank"> 662 View paper 663 </a> 664 665 666 <details class="mt-2"> 667 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 668 Show abstract 669 </summary> 670 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 671 Machine learning is an area of research concerned with the 672construction of algorithms which are able to learn from 673examples. Among such algorithms, so-called kernel methods form an 674important family of algorithms which have proven to be powerful 675and versatile for a large number of problem areas. Central to 676these approaches is the kernel matrix which summarizes the 677information contained in the training examples. The goal of this 678thesis is to analyze machine learning kernel methods based on 679properties of the kernel matrix. The algorithms considered are
680kernel principal component analysis and kernel ridge 681regression. This thesis is divided into two parts: a theoretical 682part devoted to studying the spectral properties of the kernel 683matrix, and an application part which analyzes the kernel 684principal component analysis method and kernel based regression 685based on these theoretical results. 686 687In the theoretical part, convergence properties of the 688eigenvalues and eigenvectors of the kernel matrix are studied. We 689derive accurate bounds on the approximation error which have the 690important property that the error bounds scale with the magnitude 691of the eigenvalue, predicting correctly that the approximation 692error of small eigenvalues is much smaller than that of large 693eigenvalues. In this respect, the results improve significantly on 694existing results. A similar result is proven for scalar products 695with eigenvectors of the kernel matrix. It is shown that the 696scalar products with eigenvectors corresponding to small 697eigenvalues are small a priori independently of the degree of 698approximation. 699 700In the application part, we discuss the following topics. For 701kernel principal component analysis, we show that the estimated 702eigenvalues approximate the true principal values with high 703precision. Next, we discuss the general setting of kernel based 704regression and show that the relevant information of the labels is 705contained in the first few coefficients of the label vector in the 706basis of eigenvectors of the kernel matrix, such that the 707information and the noise can be divided much more easily in this 708representation. Finally, we show that kernel ridge regression 709works by suppressing all but the leading coefficients, thereby 710extracting the relevant information of the label vectors. This 711interpretation suggests an estimate of the number of relevant 712coefficients in order to perform model selection. In an 713experimental evaluation, this approach proves to perform 714competitively to state-of-the-art methods. 715 716 </p> 717 </details> 718 719 </div> 720 721 </div> 722 </section> 723 724 725 726 727 <section class="mb-10"> 728 <h2 class="text-xl font-bold mb-2 text-gray-800">Brain Computer Interface</h2> 729 730 <p class="text-gray-600 text-sm mb-4">Detecting conditions of high mental and auditory workload based on 731EEG recordings under real-world conditions. 732</p> 733 734 735 <div class="space-y-4"> 736 737 <div class="border-l-2 border-gray-200 pl-4 py-2"> 738 <p class="font-medium text-gray-900">Improving BCI performance by task-related trial pruning</p> 739 <p class="text-sm text-gray-600">C. Sannelli, M. Braun, and K.-R. Müller</p> 740 <p class="text-sm text-gray-500 mt-1"> 741 <p>Neural Networks, Volume 22, Issue 9, November 2009, Pages 7421295-1304, Special Issue on Brain-Machine Interface</p> 743 </p> 744 745 <a href="https://dx.doi.org/10.1016/j.neunet.2009.08.006" 746 class="text-blue-600 hover:text-blue-800 text-sm" 747 target="_blank"> 748 View paper 749 </a> 750 751 752 <details class="mt-2"> 753 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 754 Show abstract 755 </summary> 756 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 757 Noise in electroencephalography data (EEG) is an ubiquitous 758problem that limits the performance of brain computer interfaces 759(BCI). While typical EEG artifacts are usually removed by trial 760rejection or by filtering, noise induced in the data by the subject's 761failure to produce the required mental state is very harmful. Such 762"noise" effects are rather common, especially for naive subjects 763in their training phase and, thus, standard artifact removal methods 764would inevitably fail. In this paper, we present a novel method 765which aims to detect such defected trials taking into account the 766intended task by use of Relevant Dimensionality Estimation (RDE), a 767new machine learning method for denoising in feature space. In this 768manner, our method effectively "cleans" the training data and thus 769allows better BCI classification. Preliminary results conducted on a 770data set of 43 naive subjects show a significant improvement for 74% 771of the subjects. 772 773 </p> 774 </details> 775 776 </div> 777 778 <div class="border-l-2 border-gray-200 pl-4 py-2"> 779 <p class="font-medium text-gray-900">Estimating Noise and Dimensionality in BCI Data Sets: Towards BCI 780Illiteracy Comprehension 781</p> 782 <p class="text-sm text-gray-600">C. Sannelli, M. Braun, M. Tangermann, and K.-R. Müller</p> 783 <p class="text-sm text-gray-500 mt-1"> 784 <p>In Proceedings of the <a href="https://bci.tugraz.at/BCI08/index.html">4th International Brain-Computer Interface 785Workshop and Training Course 2008</a>. Verlag der Technischen 786Universität Graz, 2008 (won the young scientists best talk award)</p> 787 </p> 788 789 790 </div> 791 792 <div class="border-l-2 border-gray-200 pl-4 py-2"> 793 <p class="font-medium text-gray-900">Improving Human Performance in a Real Operating Environment 794through Real-Time Mental Workload Detection 795</p> 796 <p class="text-sm text-gray-600">J. Kohlmorgen, G. Dornhege, M.L. Braun, B. Blankertz, 797K.-R. M&uuml;ller, G. Curio, K. Hagemann, A. Bruns, M. Schrauf, 798W.E. Kincses 799</p> 800 <p class="text-sm text-gray-500 mt-1"> 801 <p>in <em>Towards Brain-Computer Interfacing</em>, G. Dornhege, 802J. del R. Millián, T. Hinterberger, D. McFarland, 803K.-R. Müller (Eds), MIT Press, 2007</p> 804 </p> 805 806 <a href="https://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=11237" 807 class="text-blue-600 hover:text-blue-800 text-sm" 808 target="_blank"> 809 View paper 810 </a> 811 812 813 </div> 814 815 </div> 816 </section> 817 818 819 820 821 <section class="mb-10"> 822 <h2 class="text-xl font-bold mb-2 text-gray-800">Stability Based Model Selection</h2> 823 824 <p class="text-gray-600 text-sm mb-4">
824A general framework for estimating the right number of clusters 825based on the stability of the clustering solutions. 826</p> 827 828 829 <div class="space-y-4"> 830 831 <div class="border-l-2 border-gray-200 pl-4 py-2"> 832 <p class="font-medium text-gray-900">Stability-Based Validation of Clustering Solutions</p> 833 <p class="text-sm text-gray-600">T. Lange, V. Roth, M. Braun and J. Buhmann</p> 834 <p class="text-sm text-gray-500 mt-1"> 835 <p>Neural Computation 16:6, June 2004.</p> 836 </p> 837 838 <a href="https://www.mitpressjournals.org/doi/abs/10.1162/089976604773717621" 839 class="text-blue-600 hover:text-blue-800 text-sm" 840 target="_blank"> 841 View paper 842 </a> 843 844 845 <details class="mt-2"> 846 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 847 Show abstract 848 </summary> 849 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 850 Data clustering describes a set of frequently employed techniques 851in exploratory data analysis to extract "natural" group structure 852in data. Such groupings need to be validated to separate the 853signal in the data from spurious structure. In this context, 854finding an appropriate number of clusters is a particularly 855important model selection question. We introduce a measure of 856cluster stability to assess the validity of a cluster model. This 857stability measure quantifies the reproducibility of clustering 858solutions on a second sample, and it can be interpreted as a 859classification risk with regard to class labels produced by a 860clustering algorithm. The preferred number of clusters is 861determined by minimizing this classification risk as a function of 862the number of clusters. Convincing results are achieved on 863simulated as well as gene expression data sets. Comparisons to 864other methods demonstrate the competitive performance of our 865method and its suitability as a general validation tool for 866clustering solutions in real-world problems. 867 868 </p> 869 </details> 870 871 </div> 872 873 <div class="border-l-2 border-gray-200 pl-4 py-2"> 874 <p class="font-medium text-gray-900">Stability-Based Model Selection</p> 875 <p class="text-sm text-gray-600">T. Lange, M. Braun, V. Roth, and J. Buhmann</p> 876 <p class="text-sm text-gray-500 mt-1"> 877 <p>NIPS 2002</p> 878 </p> 879 880 <a href="https://books.nips.cc/papers/files/nips15/AA17.pdf" 881 class="text-blue-600 hover:text-blue-800 text-sm" 882 target="_blank"> 883 View paper 884 </a> 885 886 887 </div> 888 889 <div class="border-l-2 border-gray-200 pl-4 py-2"> 890 <p class="font-medium text-gray-900">Stability-Based Model Order Selection in Clustering with 891Applications to Gene Expression Data 892</p> 893 <p class="text-sm text-gray-600">V. Roth, M. Braun, T. Lange, and J. Buhmann</p> 894 <p class="text-sm text-gray-500 mt-1"> 895 <p>ICANN 2002, pp. 607-612, Lecture Notes in Computer Science 8962415, José R. Dorronsoro (Ed.)</p> 897 </p> 898 899 <a href="https://link.springer.de/link/service/series/0558/papers/2415/24150607.pdf" 900 class="text-blue-600 hover:text-blue-800 text-sm" 901 target="_blank"> 902 View paper 903 </a> 904 905 906 </div> 907 908 <div class="border-l-2 border-gray-200 pl-4 py-2"> 909 <p class="font-medium text-gray-900">A Resampling Approach to Cluster Validation</p> 910 <p class="text-sm text-gray-600">V. Roth, T. Lange, M. Braun, J. Buhmann</p> 911 <p class="text-sm text-gray-500 mt-1"> 912 <p>COMPSTAT 2002</p> 913 </p> 914 915 <a href="compstat02.pdf" 916 class="text-blue-600 hover:text-blue-800 text-sm" 917 target="_blank"> 918 View paper 919 </a> 920 921 922 </div> 923 924 </div> 925 </section> 926 927 928 929 930 <section class="mb-10"> 931 <h2 class="text-xl font-bold mb-2 text-gray-800">Noisy TSP</h2> 932 933 <p class="text-gray-600 text-sm mb-4">Applying methods from statistical physics to robustly sample good
934solutions for combinatorial optimization problems like the 935travelling salesman problem. 936</p> 937 938 939 <div class="space-y-4"> 940 941 <div class="border-l-2 border-gray-200 pl-4 py-2"> 942 <p class="font-medium text-gray-900">The Noisy Euclidian Traveling Salesman Problem and Learning</p> 943 <p class="text-sm text-gray-600">M. Braun, J. Buhmann</p> 944 <p class="text-sm text-gray-500 mt-1"> 945 <p>in: T. G. Diettrich, S. Becker, Z. Ghahramani (Eds.), Advances in 946Information Processing Systems 14, pp. 251-258, MIT Press</p> 947 </p> 948 949 <a href="https://books.nips.cc/papers/files/nips14/LT11.pdf" 950 class="text-blue-600 hover:text-blue-800 text-sm" 951 target="_blank"> 952 View paper 953 </a> 954 955 956 <details class="mt-2"> 957 <summary class="text-sm text-gray-500 cursor-pointer hover:text-gray-700"> 958 Show abstract 959 </summary> 960 <p class="text-sm text-gray-600 mt-2 pl-2 border-l border-gray-200"> 961 We consider noisy Euclidean traveling salesman problems in the 962plane, whcih are random combinatorial problems with underlying 963structure. Gibbs sampling is sused to compute average 964trajectories, which estimate the underlying structure common to 965all instances. This procedure requires identifying the exact 966relationship between permutations and tours. In a learning 967setting, the average trajectory is used as a model to construct 968solutions to new instances sampled from the same 969source. Experimental results show that the average trajectory can 970in fact estimate the underlying structure and that overfitting 971effects occur if the trajectory adapts too closely to a single 972instance. 973 974 </p> 975 </details> 976 977 </div> 978 979 </div> 980 </section> 981 982 983 984 985 <section class="mb-10"> 986 <h2 class="text-xl font-bold mb-2 text-gray-800">Retina Implant</h2> 987 988 <p class="text-gray-600 text-sm mb-4">From Mar. 1997 to Sept. 1999, I worked as a lab assitant in the <a 989href="https://www.nero.uni-bonn.de/projekte/ri/ri-index-en.htm">Retina 990Implant project</a> (which was a registered project of the 991EXPO2000) with Michael Becker. The goal was to construct a retinal 992prosthesis for people with <a 993href="https://en.wikipedia.org/wiki/Macular_degeneration">macula 994degeneration</a>. 995</p> 996 997 998 <div class="space-y-4"> 999 1000 <div class="border-l-2 border-gray-200 pl-4 py-2"> 1001 <p class="font-medium text-gray-900">Retina Implant Adjustment with Reinforcement Learning</p> 1002 <p class="text-sm text-gray-600">M. Becker, M. Braun, R. Eckmiller</p> 1003 <p class="text-sm text-gray-500 mt-1"> 1004 <p>Proc. ICASSP98, IEEE Press, 1998, Vol. II, pp. 1181-1184</p> 1005 </p> 1006 1007 <a href="https://citeseer.ist.psu.edu/becker98retina.html" 1008 class="text-blue-600 hover:text-blue-800 text-sm" 1009 target="_blank"> 1010 View paper 1011 </a> 1012 1013 1014 </div> 1015 1016 <div class="border-l-2 border-gray-200 pl-4 py-2"> 1017 <p class="font-medium text-gray-900">Retina Encoder Inversion for Retina Implant Simulation</p> 1018 <p class="text-sm text-gray-600">M. Becker, M. Braun, R. Eckmiller</p> 1019 <p class="text-sm text-gray-500 mt-1"> 1020 <p>Niklasson L. et al (eds), Proc. ICANN98, Springer 1998, pp. 791-796</p> 1021 </p> 1022 1023 <a href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.71.4726" 1024 class="text-blue-600 hover:text-blue-800 text-sm" 1025 target="_blank"> 1026 View paper 1027 </a> 1028 1029 1030 </div> 1031 1032 </div> 1033 </section> 1034 1035 1036</div> 1037 1038 </main> 1039 1040 <!-- Footer --> 1041 <footer class="bg-white border-t border-gray-200 mt-auto"> 1042 <div class="max-w-4xl mx-auto px-4 sm:px-6 py-6"> 1043 <p class="text-sm text-gray-500 text-center"> 1044 © Mikio L. Braun · 1045 <a href="/impressum" class="hover:text-gray-700">Impressum</a> 1046 </p> 1047 </div> 1048 </footer> 1049</body> 1050</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.