1<!DOCTYPE html> 2<html lang="en"> 3 <head> 4 <meta charset="utf-8" /> 5 <link rel="icon" type="image/svg+xml" sizes="any"/> 6 <meta name="viewport" content="width=device-width, initial-scale=1" /> 7 8 <link href="./_app/immutable/assets/0.c0ace33e.css" rel="stylesheet"> 9 <link rel="modulepreload" href="./_app/immutable/entry/start.6fb108d8.js"> 10 <link rel="modulepreload" href="./_app/immutable/chunks/index.72972d85.js"> 11 <link rel="modulepreload" href="./_app/immutable/chunks/singletons.60685853.js"> 12 <link rel="modulepreload" href="./_app/immutable/entry/app.d09ffb14.js"> 13 <link rel="modulepreload" href="./_app/immutable/nodes/0.3a038e35.js"> 14 <link rel="modulepreload" href="./_app/immutable/chunks/stores.582c9d99.js"> 15 <link rel="modulepreload" href="./_app/immutable/nodes/2.24a9aa80.js"> 16 <link rel="modulepreload" href="./_app/immutable/chunks/Seo.1d28f369.js"><title>Sham M. Kakade</title><!-- HEAD_svelte-1xw0bo2_START -->
vendor: 63 bytes, line 16
16<script async src="https://www.googletagmanager.com/gtag/js?id=
16UA-156644599-1
vendor: 16 bytes, lines 16-17
16"></script> 17
17<script>
vendor: 162 bytes, lines 17-22
17window.dataLayer = window.dataLayer || []; 18 function gtag() { 19 dataLayer.push(arguments); 20 } 21 gtag("js", new Date()); 22 gtag("config", "
22UA-156644599-1
vendor: 8 bytes, lines 22-23
22"); 23
23</script>
23<!-- HEAD_svelte-1xw0bo2_END --><!-- HEAD_svelte-1alr8gw_START --><meta name="description" content=""><meta property="og:title" content="Sham M. Kakade"><meta property="og:description" content=""><meta name="twitter:card" content="summary_large_image"><!-- HEAD_svelte-1alr8gw_END --> 24 </head> 25 <body> 26 <div> 27 28 29 30 31 32<header class="" data-sveltekit-noscroll data-sveltekit-preload-code="eager"> 33 <div class="layout-md content grid grid-cols-2 gap-5 pt-2 pb-2 object-scale-down"><div class=""><img class="object-scale-down rounded-t-lg" src="/assets/images/sham.jpg" alt=""></div> 34 35 <div class="grid-rows-4 md:space-y-0 place-content-center"><div class=""><h1 class="font-bold text-black text-2xl"><a href="/">Sham M. Kakade</a> 36 </h1></div> 37 <div class="anormal-font">(He/Him/His) </div> 38 <div class="anormal-font"><br> <span class="anormal-font-bold text-black">Co-director of the <a href="https://kempnerinstitute.harvard.edu" class="alink">Kempner Institute</a> 39 <br> 40 <br> Rampell Family Professor of <a href="https://seas.harvard.edu" class="alink">Computer Science</a> and <a href="https://statistics.fas.harvard.edu" class="alink">Statistics</a></span> 41 <br> 42 <br> <a href="https://www.harvard.edu" class="alink">Harvard University</a> 43 </div></div></div> 44 45 <div class="layout-md"><div class="flex-grow border-b-2 border-black"></div></div> 46 47 <nav class="svelte-1ggp7n1"><a href="/" class="hover:text-black transition-colors svelte-1ggp7n1 text-black">home 48 </a><a href="/publications" class="hover:text-black transition-colors svelte-1ggp7n1">publications 49 </a><a href="/teaching" class="hover:text-black transition-colors svelte-1ggp7n1">teaching 50 </a><a href="/group" class="hover:text-black transition-colors svelte-1ggp7n1">group 51 </a><a href="/bio" class="hover:text-black transition-colors svelte-1ggp7n1">bio 52 </a></nav> 53 54</header> 55 56<main> 57 58<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> 59 60<section class="layout-md mb-5"><h2 class="heading2">Research:</h2> 61 <div class="ml-4"><p class="mb-4">I work on advancing the fundamental capabilities needed to develop artificial general intelligence and create systems that can effectively interact with and add value to the real world. My current interests include:</p> 62 63 <ul class="list-none"><li class="mb-3">(i) developing full-stack training pipelines for foundation models, with particular focus on distributed systems architecture, scalable optimization algorithms, and principled approaches to data curation and composition.</li> 64 65 <li class="mb-3">(ii) investigating the mathematical and scientific principles that govern large-scale learning systems, with emphasis on understanding emergent capabilities, scaling laws, and fundamental limits of neural architectures.</li> 66 67 <li class="mb-3">(iii) advancing autonomous agent architectures that can reason, plan, and learn from interaction, with focus on bridging the gap between language models and embodied intelligence in complex environments.</li></ul></div></section> 68 69<section class="layout-md mb-5"><h2 class="heading2">Prospective Students:</h2> 70 <div class="ml-4">I am seeking students with diverse backgrounds, including those experienced in applied deep learning, as well as those with strong foundations in optimization, mathematics, and theoretical computer science. As co-director of the newly-established <a href="https://kempnerinstitute.harvard.edu" class="alink">[Kempner Institute]</a>, we offer substantial computational resources for cutting-edge research. If you're interested, I encourage you to apply to Harvard! 71</div></section> 72 73<section class="layout-md mb-5"><h2 class="heading2">Recent Blog Posts:</h2> 74 <div class="ml-4"><p class="mb-4">Selected blog posts (also see <a href="https://shamulent.github.io/publications" class="alink">[publications]</a> and <a href="https://kempnerinstitute.harvard.edu/research/deeper-learning/" class="alink">[Deeper Learning]</a>) exploring fundamental questions in AI and advancing technical innovations: 75</p> 76 77 <div class="space-y-2"><p><a href="https://kempnerinstitute.harvard.edu/research/deeper-learning/how-does-critical-batch-size-scale-in-pre-training-decoupling-data-and-model-size/" class="alink">[How Does Critical Batch Size Scale in Pre-training?]</a></p> 78 <p><a href="https://kempnerinstitute.harvard.edu/research/deeper-learning/mixture-of-parrots-experts-improve-memorization-more-than-reasoning/" class="alink">[Mixture of Parrots: Experts Improve Memorization More Than Reasoning]</a></p> 79 80 <p><a href="https://kempnerinstitute.harvard.edu/research/deeper-learning/transcendence-generative-models-can-outperform-the-experts-that-train-them/" class="alink">[Transcendence: Generative Models Can Outperform the Experts That Train Them]</a></p> 81 82 <p><a href="https://kempnerinstitute.harvard.edu/research/deeper-learning/repeat-after-me-transformers-are-better-than-state-space-models-at-copying/" class="alink">[Repeat After Me: Transformers are Better than State Space Models at Copying]</a></p></div></div></section> 83 84<section class="layout-md mb-5"><h2 class="heading2">Selected Service:</h2>
85 <div class="ml-4"><ul class="list-none space-y-2"><li><a href="https://awards.acm.org/acm-prize/committee" class="alink">[Committee for the ACM Prize in Computing]</a> (active)</li> 86 87 <li><a href="https://sloan.org/fellowships#tab-selection-committees" class="alink">[Committee for the Sloan Research Fellowships]</a> in Computer Science</li> 88 89 <li>Co-organizer for the Simons Symposium on <a href="https://www.simonsfoundation.org/event/new-directions-in-theoretical-machine-learning-2019/" class="alink">[New Directions in Theoretical Machine Learning]</a>, May 2019</li> 90 91 <li>Program chair for the 24th Annual Conference on Learning Theory (COLT 2011)</li></ul></div></section> 92 93<section class="layout-md mb-5"><h2 class="heading2">Other Good Stuff:</h2> 94 <div class="ml-4"><ul class="list-none space-y-2"><li><a href="https://dhruvmadeka.com/post/llms_education/" class="alink">[What is the Value of Human-Level AI to Education?]</a></li></ul></div></section></main> 95 96<footer class="layout-md mt-16 text-lg flex flex-col"><div class="row anormal-font svelte-1t3hv3r"><span class="anormal-font-bold svelte-1t3hv3r">eMail</span> 97 <hr class="svelte-1t3hv3r"> 98 <a href="mailto:[email protected]" class="flex-none alink svelte-1t3hv3r">[email protected] </a></div> 99 100 <div class="row anormal-font svelte-1t3hv3r"><span class="anormal-font-bold svelte-1t3hv3r">Twitter</span> 101 <hr class="svelte-1t3hv3r"> 102 <a class="link flex-none svelte-1t3hv3r" href="https://x.com/ShamKakade6">@ShamKakade6</a></div> 103 104 105 <div class="row anormal-font svelte-1t3hv3r"><span class="flex-none anormal-font-bold svelte-1t3hv3r">Office</span> 106 <hr class="svelte-1t3hv3r"> 107 <span class="flex-none svelte-1t3hv3r" href="">150 Western Av., Room 6.146, Allston MA 02134</span></div> 108</footer> 109 110 111 112
112<script> 113 { 114 __sveltekit_1mnrdv7 = { 115 base: new URL(".", location).pathname.slice(0, -1), 116 env: {} 117 }; 118 119 const element = document.currentScript.parentElement; 120 121 const data = [null,null]; 122 123 Promise.all([ 124 import("./_app/immutable/entry/start.6fb108d8.js"), 125 import("./_app/immutable/entry/app.d09ffb14.js") 126 ]).then(([kit, app]) => { 127 kit.start(app, element, { 128 node_ids: [0, 2], 129 data, 130 form: null, 131 error: null 132 }); 133 }); 134 } 135 </script>
135 136 </div> 137 </body> 138</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.