PageSourceSearch

https://nodeology.ai/tutorials/foundation-models-xrd

html nodeology.ai collected 2026-10-05 15:09:11 UTC 66,841 bytes, 98 lines download raw bytes

1<!doctype html>
2<html lang="en">
3  <head>
4    <meta charset="UTF-8" />
5    <link rel="icon" type="image/png" href="/favicon.png" />
6    <meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
7    <meta name="theme-color" content="#ffffff" />
8    <!-- route-meta:start -->
9    <title>Foundation Models Adaptation I: prompting, features, and fine-tuning — nodeology.ai</title>
10    <meta name="description" content="Part I of an interactive tutorial on adapting foundation models for scientific imaging: zero-shot and few-shot prompting, frozen features, and LoRA fine-tuning, with what each strategy changes and costs." />
11    <link rel="canonical" href="https://nodeology.ai/tutorials/foundation-models-xrd" />
12    <meta property="og:title" content="Foundation Models Adaptation I: prompting, features, and fine-tuning — nodeology.ai" />
13    <meta property="og:description" content="Part I of an interactive tutorial on adapting foundation models for scientific imaging: zero-shot and few-shot prompting, frozen features, and LoRA fine-tuning, with what each strategy changes and costs." />
14    <meta property="og:url" content="https://nodeology.ai/tutorials/foundation-models-xrd" />
15    <meta name="twitter:title" content="Foundation Models Adaptation I: prompting, features, and fine-tuning — nodeology.ai" />
16    <meta name="twitter:description" content="Part I of an interactive tutorial on adapting foundation models for scientific imaging: zero-shot and few-shot prompting, frozen features, and LoRA fine-tuning, with what each strategy changes and costs." />
17    
17<script type="application/ld+json">{"@context":"https://schema.org","@type":"LearningResource","@id":"https://nodeology.ai/tutorials/foundation-models-xrd#tutorial","name":"Foundation Models Adaptation Tutorial","description":"Adapting foundation models for scientific imaging, applied to XRD anomaly detection: few-shot prompting, frozen features, and LoRA fine-tuning.","url":"https://nodeology.ai/tutorials/foundation-models-xrd","learningResourceType":"Tutorial","author":{"@type":"Person","name":"Xiangyu Yin","url":"https://xiangyu-yin.com"},"isAccessibleForFree":true}</script>
17
18    <!-- route-meta:end -->
19    <meta property="og:type" content="website" />
20    <meta property="og:site_name" content="nodeology.ai" />
21    <meta property="og:image" content="https://nodeology.ai/og.png?v=2" />
22    <meta property="og:image:width" content="1200" />
23    <meta property="og:image:height" content="630" />
24    <meta
25      property="og:image:alt"
26      content="nodeology.ai — interactive scientific work"
27    />
28    <meta name="twitter:card" content="summary_large_image" />
29    <meta name="twitter:image" content="https://nodeology.ai/og.png?v=2" />
30    
30<script type="application/ld+json">
31      {
32        "@context": "https://schema.org",
33        "@type": "WebSite",
34        "name": "nodeology.ai",
35        "url": "https://nodeology.ai/",
36        "description": "Interactive scientific work: atlases, courses, resource boards, and simulators spanning AI and X-ray science.",
37        "author": {
38          "@type": "Person",
39          "name": "Xiangyu Yin",
40          "url": "https://xiangyu-yin.com",
41          "sameAs": [
42            "https://scholar.google.com/citations?user=YqRWi_kAAAAJ",
43            "https://github.com/xyin-anl"
44          ]
45        }
46      }
47    </script>
vendor: 80 bytes, lines 47-50
47
48    <script
49      async
50      src="https://www.googletagmanager.com/gtag/js?id=
50G-FS8W3PQFFJ
vendor: 21 bytes, lines 50-52
50"
51    ></script>
52    
52<script>
53      
vendor: 162 bytes, lines 53-58
53window.dataLayer = window.dataLayer || [];
54      function gtag() {
55        dataLayer.push(arguments);
56      }
57      gtag("js", new Date());
58      gtag("config", "
58G-FS8W3PQFFJ
vendor: 32 bytes, lines 58-59
58", { anonymize_ip: true });
59    
59</script>
59
60    
60<script
61      async
62      src="https://scripts.simpleanalyticscdn.com/latest.js"
63    ></script>
63
64    <link
65      rel="preload"
66      href="/fonts/literata-latin.woff2"
67      as="font"
68      type="font/woff2"
69      crossorigin
70    />
71    <link
72      rel="preload"
73      href="/fonts/public-sans-latin.woff2"
74      as="font"
75      type="font/woff2"
76      crossorigin
77    />
78    
78<script type="module" crossorigin src="/assets/index-CZU25NZ5.js"></script>
78
79    <link rel="modulepreload" crossorigin href="/assets/usePageMeta-CdaG6uYr.js">
80    <link rel="stylesheet" crossorigin href="/assets/index-CU1ON7Yq.css">
81  </head>
82  <body class="text-ink-900 font-sans antialiased">
83    <div id="root"><!--$--><div class="min-h-screen bg-white text-ink-900"><header class="z-40 sticky top-0 border-b border-hairline-soft bg-white"><div class="mx-auto flex h-14 max-w-[1020px] items-center justify-between gap-6 px-6 sm:px-12"><a aria-label="Nodeology home" class="shrink-0" href="/"><span class="font-mono font-medium tracking-tight text-xl"><span class="text-accent-emerald">n</span><span class="text-accent-mint-deep">o</span><span class="text-accent-emerald">d</span><span class="text-accent-coral">e</span><span class="text-accent-mint">o</span><span class="text-accent-orange">l</span><span class="text-accent-olive">o</span><span class="text-accent-mint-deep">g</span><span class="text-accent-mint">y</span><span class="text-ink-400">.</span><span class="text-accent-emerald">a</span><span class="text-accent-coral">i</span></span></a><div class="relative hidden w-[260px] sm:block lg:w-[340px]"><form class="relative w-full" role="search"><input placeholder="Search sections and topics" class="h-8 w-full border-b border-hairline bg-transparent pb-1 font-sans text-[0.85rem] text-ink-800 outline-none placeholder:text-ink-400 focus:border-ink-500" value=""/></form></div></div></header><main><div class="mx-auto max-w-[1020px] px-6 pb-[4.4rem] pt-[2.6rem] sm:px-12"><header><div class="font-sans text-[0.85rem] text-ink-500">MRS Spring 2026, Part I of III</div><h1 class="mt-2 max-w-[620px] text-balance font-display text-[1.6rem] font-semibold leading-[1.25] tracking-[-0.01em] text-ink-900">Foundation Models Adaptation I: prompting, features, and fine-tuning</h1><p class="mt-4 max-w-[560px] text-pretty font-sans text-[0.95rem] leading-[1.6] text-ink-500">From zero-shot prompting to few-shot examples, frozen features, and LoRA fine-tuning: a map of the adaptation strategies, what they change, and what they cost. Adapted from MRS Spring 26 MT01 tutorial materials.</p></header><div class="mt-16 grid gap-x-14 lg:grid-cols-[minmax(0,1fr)_220px]"><main class="min-w-0"><article class="space-y-20"><section id="fm-landscape" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">01</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">Where foundation models fit in</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">Foundation models are deep-learning models trained broadly once, then adapted many times for specific tasks.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="p-7"><div class="space-y-3"><div class="rounded-[3px] border px-5 py-4 border-ink-700/10 bg-white/70" style="margin-left:0px"><div class="font-sans text-[0.78rem] text-ink-500">Artificial intelligence</div><div class="mt-1 text-[14px] text-ink-600">systems that perform useful tasks</div></div><div class="rounded-[3px] border px-5 py-4 border-ink-700/10 bg-white/70" style="margin-left:28px"><div class="font-sans text-[0.78rem] text-ink-500">Machine learning</div><div class="mt-1 text-[14px] text-ink-600">models that learn from data</div></div><div class="rounded-[3px] border px-5 py-4 border-ink-700/10 bg-white/70" style="margin-left:56px"><div class="font-sans text-[0.78rem] text-ink-500">Deep learning</div><div class="mt-1 text-[14px] text-ink-600">models that learn representations</div></div><div class="rounded-[3px] border px-5 py-4 border-accent-emerald/35 bg-ink-100" style="margin-left:84px"><div class="font-sans text-[0.78rem] text-ink-500">Foundation models</div><div class="mt-1 text-[14px] text-ink-600">broad pretraining reused for many tasks</div></div></div><div class="mt-6 grid gap-2 sm:grid-cols-3"><div class="border-t border-ink-700/10 pt-2 font-sans text-[0.78rem] text-ink-500">language</div><div class="border-t border-ink-700/10 pt-2 font-sans text-[0.78rem] text-ink-500">vision</div><div class="border-t border-ink-700/10 pt-2 font-sans text-[0.78rem] text-ink-500">multimodal</div><div class="border-t border-ink-700/10 pt-2 font-sans text-[0.78rem] text-ink-500">reasoning</div><div class="border-t border-ink-700/10 pt-2 font-sans text-[0.78rem] text-ink-500">generation</div><div class="border-t border-ink-700/10 pt-2 font-sans text-[0.78rem] text-ink-500">science</div></div></div></div></div></section><section id="fm-capabilities" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">02</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">What foundation models can do</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">A pretrained model can help with routine analysis, natural-language interfaces, and discovery workflows when the task is well defined.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="grid gap-0 md:grid-cols-3"><div class="border-ink-700/10 p-6 md:border-r md:last:border-r-0"><div class="relative h-28 rounded-[3px] border border-ink-700/10 bg-white">
83<span class="absolute h-4 w-4 rounded-full bg-accent-emerald" style="left:18%;top:28%"></span><span class="absolute h-4 w-4 rounded-full bg-accent-emerald" style="left:42%;top:62%"></span><span class="absolute h-4 w-4 rounded-full bg-accent-emerald" style="left:66%;top:28%"></span><span class="absolute h-4 w-4 rounded-full bg-accent-emerald" style="left:88%;top:62%"></span><span class="absolute left-[22%] top-[37%] h-px w-[58%] rotate-12 bg-accent-forest/35"></span><span class="absolute left-[43%] top-[48%] h-px w-[34%] -rotate-12 bg-accent-forest/35"></span></div><h3 class="mt-5 text-xl font-semibold leading-tight text-ink-900">Automate routine tasks</h3><p class="mt-3 text-[14px] leading-relaxed text-ink-600">Summarize, classify, extract, write code, and check repeated analysis steps.</p></div><div class="border-ink-700/10 p-6 md:border-r md:last:border-r-0"><div class="grid h-28 place-items-center rounded-[3px] border border-ink-700/10 bg-white"><div class="w-28 rounded-[3px] border border-ink-100 bg-white p-3"><div class="h-2 w-16 rounded bg-accent-coral/70"></div><div class="mt-3 space-y-2"><div class="h-2 rounded bg-ink-100"></div><div class="h-2 w-3/4 rounded bg-ink-100"></div>
83<div class="h-2 w-1/2 rounded bg-accent-emerald/60"></div></div></div></div><h3 class="mt-5 text-xl font-semibold leading-tight text-ink-900">Create natural interfaces</h3><p class="mt-3 text-[14px] leading-relaxed text-ink-600">Let scientists ask for analysis, metadata, and visual explanations in normal language.</p></div><div class="border-ink-700/10 p-6 md:border-r md:last:border-r-0"><div class="relative h-28 rounded-[3px] border border-ink-700/10 bg-white"><div class="absolute left-6 top-7 h-16 w-16 rounded-full border-[10px] border-accent-coral/55"></div><div class="absolute right-7 top-8 h-14 w-14 rounded-full border-[10px] border-accent-emerald/65"></div><div class="absolute bottom-6 left-1/2 h-12 w-12 -translate-x-1/2 rounded-full border-[10px] border-accent-olive/70"></div></div><h3 class="mt-5 text-xl font-semibold leading-tight text-ink-900">Accelerate discovery</h3><p class="mt-3 text-[14px] leading-relaxed text-ink-600">Combine search, simulation, and experiment loops around measured outputs.</p></div></div></div></div></section><section id="pretrain-adapt-deploy" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">03</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">Pretraining is done once. Adaptation happens in the lab.</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">Model providers usually handle broad pretraining. Scientific users decide how the model is adapted, tested, and used.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="p-7"><div class="grid gap-4 md:grid-cols-[1fr_auto_1fr_auto_1fr] md:items-center"><div class="rounded-[3px] border p-5 border-ink-700/10 bg-white/70"><div class="font-sans text-[0.78rem] text-ink-500">Provider</div><div class="mt-2 text-xl font-semibold text-ink-900">Pretrain</div><p class="mt-2 text-[13px] leading-relaxed text-ink-600">model learns broad patterns from large datasets</p></div><div class="hidden text-center font-mono text-xl text-accent-forest/55 md:block">→</div><div class="rounded-[3px] border p-5 border-accent-emerald/35 bg-white"><div class="font-sans text-[0.78rem] text-ink-500">Lab</div><div class="mt-2 text-xl font-semibold text-ink-900">Adapt</div><p class="mt-2 text-[13px] leading-relaxed text-ink-600">choose prompts, examples, features, or adapters</p></div><div class="hidden text-center font-mono text-xl text-accent-forest/55 md:block">→</div><div class="rounded-[3px] border p-5 border-ink-700/10 bg-white/70"><div class="font-sans text-[0.78rem] text-ink-500">Workflow</div><div class="mt-2 text-xl font-semibold text-ink-900">Deploy</div><p class="mt-2 text-[13px] leading-relaxed text-ink-600">use only after behavior has been measured</p></div></div><div class="mt-6 grid gap-4 md:grid-cols-2"><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">Risks</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Domain mismatch</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Hallucinations</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Hidden failure modes</span></li></ul></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">Strategy</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Benchmark first</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Adapt if needed</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Scaffold with workflows</span></li></ul></div></div></div></div></div></section><section id="adaptation-landscape" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">04</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">The adaptation landscape</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">This map compares strategies by how much they change the input, how much they change the model, and how much work they require.</p></header><div class="min-w-0"><figure><div class="relative overflow-hidden border border-hairline bg-white "><div class="p-6"><div class="relative min-h-[430px] rounded-[3px] border border-ink-700/10 bg-white p-6"><div class="absolute font-sans text-[0.78rem] text-ink-500 left-5 top-4">less task data</div><div class="absolute font-sans text-[0.78rem] text-ink-500 right-5 top-4">more task data</div><div class="absolute font-sans text-[0.78rem] text-ink-500 bottom-4 left-5">less trainable compute</div><div class="absolute font-sans text-[0.78rem] text-ink-500 bottom-4 right-5">more trainable compute</div><svg class="pointer-events-none absolute inset-0 h-full w-full" aria-hidden="true" fill="none" preserveAspectRatio="none" viewBox="0 0 100 100"><path d="M 16 74 L 33 62 L 50 50 L 67 38 L 84 26" stroke="#6dd8bc" stroke-width="0.5" stroke-dasharray="1.6 1.8"></path></svg><button type="button" class="absolute w-[150px] rounded-[3px] px-4 py-3 text-left transition-colors duration-200 ease-out border border-accent-forest bg-white" style="left:5%;top:66%"><div class="text-[15px] font-semibold leading-tight text-ink-900">Zero-shot</div><div class="mt-1 text-[12px] leading-snug text-ink-500">lightweight baseline</div></button><button type="button" class="absolute w-[150px] rounded-[3px] px-4 py-3 text-left transition-colors duration-200 ease-out border border-hairline bg-white hover:border-ink-400" style="left:22%;top:54%"><div class="text-[15px] font-semibold leading-tight text-ink-900">Few-shot</div><div class="mt-1 text-[12px] leading-snug text-ink-500">in-context examples</div></button><button type="button" class="absolute w-[150px] rounded-[3px] px-4 py-3 text-left transition-colors duration-200 ease-out border border-hairline bg-white hover:border-ink-400" style="left:39%;top:42%"><div class="text-[15px] font-semibold leading-tight text-ink-900">Frozen features</div><div class="mt-1 text-[12px] leading-snug text-ink-500">kNN or linear head</div></button><button type="button" class="absolute w-[150px] rounded-[3px] px-4 py-3 text-left transition-colors duration-200 ease-out border border-hairline bg-white hover:border-ink-400" style="left:56%;top:30%"><div class="text-[15px] font-semibold leading-tight text-ink-900">LoRA / QLoRA</div><div class="mt-1 text-[12px] leading-snug text-ink-500">small trainable adapters</div></button><button type="button" class="absolute w-[150px] rounded-[3px] px-4 py-3 text-left transition-colors duration-200 ease-out border border-hairline bg-white hover:border-ink-400" style="left:73%;top:18%"><div class="text-[15px] font-semibold leading-tight text-ink-900">Full fine-tune</div><div class="mt-1 text-[12px] leading-snug text-ink-500">most expensive path</div></button></div><div class="border border-hairline bg-white mt-4 rounded-[3px]"><div class="grid gap-0 lg:grid-cols-[minmax(0,1.3fr)_minmax(0,1fr)_minmax(0,1fr)]"><div class="border-ink-700/10 p-5 lg:border-r"><div class="flex flex-wrap items-baseline justify-between gap-2"><h3 class="display-serif text-xl font-semibold text-ink-900">Zero-shot</h3><a href="#zero-shot" class="font-sans text-[0.8rem] font-medium text-accent-forest transition-colors hover:text-ink-900">jump to section</a></div><p class="mt-2 text-[13px] leading-relaxed text-ink-600">Describe the task and ask the model. Model weights do not change. Useful as a baseline even when you expect it to fail.</p><dl class="mt-4 grid grid-cols-3 gap-3"><div><dt class="font-sans text-[0.78rem] text-ink-500">labels</dt><dd class="mt-1 text-[12px] leading-snug text-ink-700">0</dd></div><div><dt class="font-sans text-[0.78rem] text-ink-500">compute</dt><dd class="mt-1 text-[12px] leading-snug text-ink-700">API or local CPU</dd></div><div><dt class="font-sans text-[0.78rem] text-ink-500">iteration</dt><dd class="mt-1 text-[12px] leading-snug text-ink-700">seconds</dd></div></dl></div><div class="border-ink-700/10 p-5 lg:border-r"><div class="font-sans text-[0.78rem] text-ink-500">Good for</div><ul class="mt-2 space-y-1.5 text-[13px] leading-snug text-ink-700"><li class="flex gap-2"><span aria-hidden="true" class="text-accent-emerald">+</span>fast triage</li><li class="flex gap-2"><span aria-hidden="true" class="text-accent-emerald">+</span>format testing</li><li class="flex gap-2"><span aria-hidden="true" class="text-accent-emerald">+</span>early prototyping</li></ul></div><div class="p-5"><div class="font-sans text-[0.78rem] text-accent-coral-deep">Weak points</div><ul class="mt-2 space-y-1.5 text-[13px] leading-snug text-ink-700"><li class="flex gap-2"><span aria-hidden="true" class="text-accent-coral">-</span>unstable formats</li><li class="flex gap-2"><span aria-hidden="true" class="text-accent-coral">-</span>hallucinated labels</li><li class="flex gap-2"><span aria-hidden="true" class="text-accent-coral">-</span>weaker domain grounding</li></ul></div></div></div><p class="mt-3 text-[13px] leading-relaxed text-ink-500">Click a strategy to compare its costs. More expensive is not automatically better: move to heavier methods only when simpler ones stop making real progress.</p></div></div><figcaption class="mt-3 max-w-[560px] font-sans text-[0.78rem] leading-[1.5] text-ink-500"><span class="font-semibold text-ink-800">Figure <!-- -->1</span> <!-- -->The adaptation landscape. Click a strategy; the detail panel reports what changes (input, weights, or both) and roughly what it costs.</figcaption></figure></div></section><section id="tradeoffs" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">05</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">Compute and memory trade-offs</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">Labels, compute, and iteration time change quickly as you move from prompting to adapters and full fine-tuning.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="divide-y divide-ink-700/10"><div class="grid gap-4 px-5 py-4 md:grid-cols-[38px_1.2fr_0.65fr_1fr_0.7fr] md:items-center"><div class="font-mono text-[11px] text-ink-400">01</div><div class="font-semibold text-ink-900">Zero-shot prompt (API)</div><div><div class="font-sans text-[0.78rem] text-ink-500">labels</div><div class="mt-1 text-[13px] leading-snug text-ink-600">0</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">compute</div><div class="mt-1 text-[13px] leading-snug text-ink-600">remote API, no local compute</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">iteration</div><div class="mt-1 text-[13px] leading-snug text-ink-600">seconds</div></div></div><div class="grid gap-4 px-5 py-4 md:grid-cols-[38px_1.2fr_0.65fr_1fr_0.7fr] md:items-center"><div class="font-mono text-[11px] text-ink-400">02</div><div class="font-semibold text-ink-900">Zero-shot prompt (local)</div><div><div class="font-sans text-[0.78rem] text-ink-500">labels</div><div class="mt-1 text-[13px] leading-snug text-ink-600">0</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">compute</div><div class="mt-1 text-[13px] leading-snug text-ink-600">GPU not necessary, CPU works</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">iteration</div><div class="mt-1 text-[13px] leading-snug text-ink-600">seconds</div></div></div><div class="grid gap-4 px-5 py-4 md:grid-cols-[38px_1.2fr_0.65fr_1fr_0.7fr] md:items-center"><div class="font-mono text-[11px] text-ink-400">03</div><div class="font-semibold text-ink-900">Few-shot prompt (API)</div><div><div class="font-sans text-[0.78rem] text-ink-500">labels</div><div class="mt-1 text-[13px] leading-snug text-ink-600">5–50</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">compute</div><div class="mt-1 text-[13px] leading-snug text-ink-600">remote API, no local compute</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">iteration</div><div class="mt-1 text-[13px] leading-snug text-ink-600">seconds / minutes</div></div></div><div class="grid gap-4 px-5 py-4 md:grid-cols-[38px_1.2fr_0.65fr_1fr_0.7fr] md:items-center"><div class="font-mono text-[11px] text-ink-400">04</div><div class="font-semibold text-ink-900">Few-shot prompt (local)</div><div><div class="font-sans text-[0.78rem] text-ink-500">labels</div><div class="mt-1 text-[13px] leading-snug text-ink-600">5–50</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">compute</div><div class="mt-1 text-[13px] leading-snug text-ink-600">GPU not necessary, CPU works</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">iteration</div><div class="mt-1 text-[13px] leading-snug text-ink-600">seconds / minutes</div></div></div><div class="grid gap-4 px-5 py-4 md:grid-cols-[38px_1.2fr_0.65fr_1fr_0.7fr] md:items-center"><div class="font-mono text-[11px] text-ink-400">05</div><div class="font-semibold text-ink-900">Frozen features</div><div><div class="font-sans text-[0.78rem] text-ink-500">labels</div><div class="mt-1 text-[13px] leading-snug text-ink-600">100 – 10k</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">compute</div><div class="mt-1 text-[13px] leading-snug text-ink-600">encoder needs GPU for forward pass</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">iteration</div><div class="mt-1 text-[13px] leading-snug text-ink-600">minutes</div></div></div><div class="grid gap-4 px-5 py-4 md:grid-cols-[38px_1.2fr_0.65fr_1fr_0.7fr] md:items-center"><div class="font-mono text-[11px] text-ink-400">06</div><div class="font-semibold text-ink-900">LoRA fine-tune (bf16)</div><div><div class="font-sans text-[0.78rem] text-ink-500">labels</div><div class="mt-1 text-[13px] leading-snug text-ink-600">1k – 100k</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">compute</div><div class="mt-1 text-[13px] leading-snug text-ink-600">multi-GPU for 7B+ in bf16</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">iteration</div><div class="mt-1 text-[13px] leading-snug text-ink-600">hours</div></div></div><div class="grid gap-4 px-5 py-4 md:grid-cols-[38px_1.2fr_0.65fr_1fr_0.7fr] md:items-center"><div class="font-mono text-[11px] text-ink-400">07</div><div class="font-semibold text-ink-900">QLoRA fine-tune (int4)</div><div><div class="font-sans text-[0.78rem] text-ink-500">labels</div><div class="mt-1 text-[13px] leading-snug text-ink-600">1k – 100k</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">compute</div><div class="mt-1 text-[13px] leading-snug text-ink-600">single GPU (int4 quantized base)</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">iteration</div><div class="mt-1 text-[13px] leading-snug text-ink-600">hours</div></div></div><div class="grid gap-4 px-5 py-4 md:grid-cols-[38px_1.2fr_0.65fr_1fr_0.7fr] md:items-center"><div class="font-mono text-[11px] text-ink-400">08</div><div class="font-semibold text-ink-900">Full fine-tune</div><div><div class="font-sans text-[0.78rem] text-ink-500">labels</div><div class="mt-1 text-[13px] leading-snug text-ink-600">10k – millions</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">compute</div><div class="mt-1 text-[13px] leading-snug text-ink-600">multi-GPU cluster (all weights)</div></div><div><div class="font-sans text-[0.78rem] text-ink-500">iteration</div><div class="mt-1 text-[13px] leading-
83snug text-ink-600">days</div></div></div></div></div></div></section><section id="zero-shot" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">06</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">Strategy 0: zero-shot prompting</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">A good prompt states the task, goal, context, and constraints clearly. Start simple before adding fragile prompt tricks.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="p-7"><div class="border border-hairline bg-white p-5 text-ink-800"><div class="font-sans text-[0.78rem] text-ink-500">Simple baseline</div><p class="mt-3 text-[17px] leading-relaxed">Given this diffraction image, identify every anomaly that applies.</p></div><div class="mt-5 grid gap-3 md:grid-cols-4"><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">01</div><div class="mt-2 font-semibold text-ink-900">Task</div><p class="mt-2 text-[13px] leading-relaxed text-ink-600">What should the model do?</p></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">02</div><div class="mt-2 font-semibold text-ink-900">Goal</div><p class="mt-2 text-[13px] leading-relaxed text-ink-600">What counts as a useful answer?</p></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">03</div><div class="mt-2 font-semibold text-ink-900">Context</div><p class="mt-2 text-[13px] leading-relaxed text-ink-600">What domain facts or label definitions matter?</p></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">04</div><div class="mt-2 font-semibold text-ink-900">Specs</div><p class="mt-2 text-[13px] leading-relaxed text-ink-600">What constraints, units, or format should be respected?</p></div></div></div><div class="grid gap-0 border-t border-ink-700/10 md:grid-cols-3"><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">Good for</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Fast triage</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Format testing</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Early prototyping</span></li></ul></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">weak points</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Unstable formats</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Hallucinated labels</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Weaker domain grounding</span></li></ul></div><div class="p-5 text-[13px] leading-relaxed text-ink-600">Prompt engineering is useful when it clarifies the interface. Newer reasoning models may be less sensitive to highly engineered prompts, so extra prompt details should earn their place.</div></div></div></div></section><section id="structured-outputs" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">07</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">Ask for structured outputs</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">For automation, the answer needs to be easy to parse and check. A schema helps downstream code trust what it receives.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="p-7"><div class="grid gap-5 lg:grid-cols-2"><div><div class="font-sans text-[0.78rem] text-ink-500">JSON mode</div><pre class="mt-3 overflow-x-auto border border-hairline bg-white p-5 font-mono text-[12px] leading-relaxed text-ink-700">response_format={&quot;type&quot;: &quot;json_object&quot;}
84
85{
86  &quot;wavelength_A&quot;: 0.9763,
87  &quot;detector_distance_mm&quot;: 250.0,
88  &quot;beam_center_px&quot;: [1024, 1024],
89  &quot;exposure_s&quot;: 0.5
90}</pre><p class="mt-3 text-[13px] leading-relaxed text-ink-600">Simple and low-boilerplate, but you still need to validate the fields.
90</p></div><div><div class="font-sans text-[0.78rem] text-ink-500">Structured outputs</div><pre class="mt-3 overflow-x-auto border border-hairline bg-white p-5 font-mono text-[12px] leading-relaxed text-ink-700">class XRDMetadata(BaseModel):
91    wavelength_A: float
92    detector_distance_mm: float
93    beam_center_px: tuple[int, int]
94    exposure_s: float | None = None
95
96response_format=XRDMetadata</pre><p class="mt-3 text-[13px] leading-relaxed text-ink-600">More setup, but the response is easier to check and use in Python.</p></div></div><div class="mt-5 grid gap-3 md:grid-cols-3"><div class="border-t border-accent-emerald/35 pt-3"><div class="font-sans text-[0.78rem] text-ink-500">Gate <!-- -->1</div><div class="mt-1 font-semibold text-ink-900">schema</div></div><div class="border-t border-accent-emerald/35 pt-3"><div class="font-sans text-[0.78rem] text-ink-500">Gate <!-- -->2</div><div class="mt-1 font-semibold text-ink-900">parser</div></div><div class="border-t border-accent-emerald/35 pt-3"><div class="font-sans text-[0.78rem] text-ink-500">Gate <!-- -->3</div><div class="mt-1 font-semibold text-ink-900">validator</div></div></div></div></div></div></section><section id="few-shot" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">08</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">Few-shot prompting</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">Few-shot prompting puts worked examples in the prompt. The model weights do not change.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="p-6"><div class="grid gap-3 lg:grid-cols-[1fr_auto_1fr_auto_1fr_auto_1fr] lg:items-center"><div class="contents"><figure class="mx-auto w-full max-w-[220px] overflow-hidden rounded-[3px] border border-ink-700/10 bg-white lg:max-w-none"><img src="/resources/fm-xrd/fewshot-ex-1.png" alt="support image for multimodal few-shot prompting" width="175" height="175" class="mx-auto aspect-square w-full max-w-[175px]" loading="lazy"/><figcaption class="border-t border-ink-700/10 p-3"><div class="font-sans text-[0.78rem] text-ink-500">support image</div><div class="mt-1 text-[13px] leading-snug text-ink-700">blurry; losing features</div></figcaption></figure><div class="hidden text-center font-mono text-xl text-accent-forest/55 md:block">→</div></div><div class="contents"><figure class="mx-auto w-full max-w-[220px] overflow-hidden rounded-[3px] border border-ink-700/10 bg-white lg:max-w-none"><img src="/resources/fm-xrd/fewshot-ex-2.png" alt="support image for multimodal few-shot prompting" width="175" height="175" class="mx-auto aspect-square w-full max-w-[175px]" loading="lazy"/><figcaption class="border-t border-ink-700/10 p-3"><div class="font-sans text-[0.78rem] text-ink-500">support image</div><div class="mt-1 text-[13px] leading-snug text-ink-700">major features; low contrast</div></figcaption></figure><div class="hidden text-center font-mono text-xl text-accent-forest/55 md:block">→</div></div><div class="contents"><figure class="mx-auto w-full max-w-[220px] overflow-hidden rounded-[3px] border border-ink-700/10 bg-white lg:max-w-none"><img src="/resources/fm-xrd/fewshot-ex-3.png" alt="support image for multimodal few-shot prompting" width="175" height="175" class="mx-auto aspect-square w-full max-w-[175px]" loading="lazy"/><figcaption class="border-t border-ink-700/10 p-3"><div class="font-sans text-[0.78rem] text-ink-500">support image</div><div class="mt-1 text-[13px] leading-snug text-ink-700">dark halo artifacts</div></figcaption></figure><div class="hidden text-center font-mono text-xl text-accent-forest/55 md:block">→</div></div><div class="contents"><figure class="mx-auto w-full max-w-[220px] overflow-hidden rounded-[3px] border border-ink-700/10 bg-white lg:max-w-none"><img src="/resources/fm-xrd/fewshot-ex-4.png" alt="query image for multimodal few-shot prompting" width="175" height="175" class="mx-auto aspect-square w-full max-w-[175px]" loading="lazy"/><figcaption class="border-t border-ink-700/10 p-3"><div class="font-sans text-[0.78rem] text-ink-500">query image</div><div class="mt-1 text-[13px] leading-snug text-ink-700">model predicts labels</div></figcaption></figure></div></div><div class="mt-5 rounded-[3px] border border-accent-emerald/20 bg-white p-4"><div class="font-sans text-[0.78rem] text-ink-500">Multimodal few-shot pattern</div><div class="mt-3 text-[14px] leading-relaxed text-ink-700">Each support example pairs an image with the answer format you want. The query image comes after the examples, and the weights do not change.</div></div><div class="mt-6 grid gap-4 lg:grid-cols-2"><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">teaches</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Allowed label set</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Expected answer format</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Multi-label edge cases</span></li></ul></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">cannot fix</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>A model that cannot read the pattern</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Data leakage</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>An evaluation split that does not match the task</span></li></ul></div></div></div></div></div></section><section id="frozen-features" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">09</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">Frozen features</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">A scientific image becomes an embedding vector. A small classifier then works on those saved vectors.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="p-7"><div class="grid gap-4 md:grid-cols-[180px_auto_1fr_auto_180px] md:items-center"><figure class="rounded-[3px] border border-ink-700/10 bg-white p-4 text-center"><img src="/resources/fm-xrd/diffraction-example.png" alt="image" width="175" height="175" class="mx-auto aspect-square max-h-40 object-contain" loading="lazy"/><figcaption class="mt-3 font-sans text-[0.78rem] text-ink-500">image</figcaption></figure><div class="hidden text-center font-mono text-xl text-accent-forest/55 md:block">→</div><div class="rounded-[3px] border border-accent-emerald/25 bg-white p-5 text-center"><div class="font-sans text-[0.78rem] text-ink-500">Frozen encoder</div></div><div class="hidden text-center font-mono text-xl text-accent-forest/55 md:block">→</div><div class="border border-hairline bg-white p-4 text-ink-800"><div class="font-sans text-[0.78rem] text-ink-500">Embedding</div><div class="mt-3 grid grid-cols-2 gap-2"><span class="border border-hairline px-2 py-2 text-center font-mono text-[11px] text-ink-600">0.12</span><span class="border border-hairline px-2 py-2 text-center font-mono text-[11px] text-ink-600">0.68</span><span class="border border-hairline px-2 py-2 text-center font-mono text-[11px] text-ink-600">-0.31</span><span class="border border-hairline px-2 py-2 text-center font-mono text-[11px] text-ink-600">0.44</span><span class="border border-hairline px-2 py-2 text-center font-mono text-[11px] text-ink-600">
960.03</span><span class="border border-hairline px-2 py-2 text-center font-mono text-[11px] text-ink-600">-0.77</span><span class="border border-hairline px-2 py-2 text-center font-mono text-[11px] text-ink-600">0.52</span><span class="border border-hairline px-2 py-2 text-center font-mono text-[11px] text-ink-600">0.19</span></div></div></div><div class="mt-5 border-t border-ink-700/10 pt-4 font-sans text-[0.78rem] text-ink-500">Small head: kNN / linear classifier / shallow MLP</div><div class="mt-4 grid gap-4 lg:grid-cols-2"><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">embedding / featurizer</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Numeric summary of an image</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Similar images should be nearby</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>The classifier learns from vectors instead of raw pixels</span></li></ul></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">Workflow</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Run the pretrained encoder once</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Save one vector per image</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Train a small head on vectors + labels</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Evaluate with the same split</span></li></ul></div></div></div></div></div></section><section id="clustering" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">10</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">Example clustering of embeddings</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">If the encoder is useful, visually similar diffraction patterns should sit near one another in embedding space.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="grid gap-0 lg:grid-cols-[1.05fr_1fr]"><div class="space-y-3 p-5"><div class="grid grid-cols-[82px_1fr] gap-3 rounded-[3px] border border-ink-700/10 bg-white/70 p-3"><div class="self-center"><div class="font-sans text-[0.78rem] text-ink-500">Cluster 1</div></div><div aria-label="Cluster 1 image row" class="max-w-[700px] rounded-[2px] border border-ink-700/10 bg-left bg-no-repeat" style="aspect-ratio:700 / 167.5;background-image:url(/resources/fm-xrd/embedding-examples-grid.png);background-size:100% 400%;background-position-y:0%"></div></div><div class="grid grid-cols-[82px_1fr] gap-3 rounded-[3px] border border-ink-700/10 bg-white/70 p-3"><div class="self-center"><div class="font-sans text-[0.78rem] text-ink-500">Cluster 2</div></div><div aria-label="Cluster 2 image row" class="max-w-[700px] rounded-[2px] border border-ink-700/10 bg-left bg-no-repeat" style="aspect-ratio:700 / 167.5;background-image:url(/resources/fm-xrd/embedding-examples-grid.png);background-size:100% 400%;background-position-y:33.33%"></div></div><div class="grid grid-cols-[82px_1fr] gap-3 rounded-[3px] border border-ink-700/10 bg-white/70 p-3"><div class="self-center"><div class="font-sans text-[0.78rem] text-ink-500">Cluster 3</div></div><div aria-label="Cluster 3 image row" class="max-w-[700px] rounded-[2px] border border-ink-700/10 bg-left bg-no-repeat" style="aspect-ratio:700 / 167.5;background-image:url(/resources/fm-xrd/embedding-examples-grid.png);background-size:100% 400%;background-position-y:66.66%"></div></div><div class="grid grid-cols-[82px_1fr] gap-3 rounded-[3px] border border-ink-700/10 bg-white/70 p-3">
96<div class="self-center"><div class="font-sans text-[0.78rem] text-ink-500">Cluster 4</div></div><div aria-label="Cluster 4 image row" class="max-w-[700px] rounded-[2px] border border-ink-700/10 bg-left bg-no-repeat" style="aspect-ratio:700 / 167.5;background-image:url(/resources/fm-xrd/embedding-examples-grid.png);background-size:100% 400%;background-position-y:99.99%"></div></div></div><div class="border-t border-ink-700/10 p-6 lg:border-l lg:border-t-0"><div class="font-sans text-[0.78rem] text-ink-500">Projected latent space</div><img src="/resources/fm-xrd/embedding-clusters.png" alt="Projected embedding space with visible clusters." width="438" height="269" class="mt-5 w-full max-w-[438px] rounded-[3px] bg-white" loading="lazy"/></div></div></div></div></section><section id="knn-vs-linear" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">11</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">kNN vs. linear probe</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">Both methods use the same saved embeddings. kNN predicts from nearby examples; a linear probe learns a simple boundary.</p></header><div class="min-w-0"><figure><div class="relative overflow-hidden border border-hairline bg-white "><div class="grid gap-0 lg:grid-cols-[minmax(0,1.35fr)_minmax(0,1fr)]"><div class="border-ink-700/10 p-5 lg:border-r"><div class="flex flex-wrap items-baseline justify-between gap-2"><div class="font-sans text-[0.78rem] text-ink-500">Embedding space, drag the query point</div><div class="flex items-center gap-3 font-sans text-[0.78rem] text-ink-500"><span class="flex items-center gap-1.5"><span class="h-2 w-2 rounded-full bg-accent-emerald" aria-hidden="true"></span>clean</span><span class="flex items-center gap-1.5"><span class="h-2 w-2 rounded-full bg-accent-coral" aria-hidden="true"></span>anomalous</span></div></div><svg viewBox="0 0 300 190" class="mt-3 h-auto w-full cursor-grab select-none active:cursor-grabbing" style="touch-action:none" aria-label="Embedding space with a draggable query point" role="application" fill="none"><rect x="1" y="1" width="298" height="188" rx="2" stroke="#d3d5d0" stroke-width="1"></rect><line x1="160" y1="5" x2="140" y2="185" stroke="#6dd8bc" stroke-width="1.5" stroke-dasharray="5 4"></line><text x="164.77777777777777" y="18" fill="#666a61" font-size="8" font-family="&#x27;Public Sans&#x27;, sans-serif">linear boundary</text><line x1="148" y1="108" x2="120" y2="96" stroke="#78cc7a" stroke-width="1" stroke-opacity="0.55"></line><line x1="148" y1="108" x2="118" y2="128" stroke="#78cc7a" stroke-width="1" stroke-opacity="0.55"></line><line x1="148" y1="108" x2="188" y2="104" stroke="#f1a16d" stroke-width="1" stroke-opacity="0.55"></line><line x1="148" y1="108" x2="98" y2="120" stroke="#78cc7a" stroke-width="1" stroke-opacity="0.55"></line><line x1="148" y1="108" x2="110" y2="70" stroke="#78cc7a" stroke-width="1" stroke-opacity="0.55"></line><circle cx="52" cy="40" r="3.6" fill="#37764a" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="75" cy="58" r="3.6" fill="#37764a" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="96" cy="44" r="3.6" fill="#37764a" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="64" cy="84" r="3.6" fill="#37764a" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="88" cy="92" r="3.6" fill="#37764a" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="110" cy="70" r="5" fill="#37764a" fill-opacity="1" stroke="#2f352f" stroke-width="1" style="transition:r 120ms ease"></circle><circle cx="56" cy="118" r="3.6" fill="#37764a" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle>
96<circle cx="98" cy="120" r="5" fill="#37764a" fill-opacity="1" stroke="#2f352f" stroke-width="1" style="transition:r 120ms ease"></circle><circle cx="120" cy="96" r="5" fill="#37764a" fill-opacity="1" stroke="#2f352f" stroke-width="1" style="transition:r 120ms ease"></circle><circle cx="76" cy="30" r="3.6" fill="#37764a" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="118" cy="128" r="5" fill="#37764a" fill-opacity="1" stroke="#2f352f" stroke-width="1" style="transition:r 120ms ease"></circle><circle cx="196" cy="70" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="214" cy="96" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="232" cy="72" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="206" cy="128" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="228" cy="116" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="246" cy="96" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="188" cy="104" r="5" fill="#f1a16d" fill-opacity="1" stroke="#2f352f" stroke-width="1" style="transition:r 120ms ease"></circle><circle cx="216" cy="148" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="240" cy="140" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="72" cy="152" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="94" cy="164" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="60" cy="170" r="3.6" fill="#f1a16d" fill-opacity="0.75" stroke="none" stroke-width="0" style="transition:r 120ms ease"></circle><circle cx="148" cy="108" r="7.5" fill="#fcfdf9" stroke="#2f352f" stroke-width="1.8"></circle><circle cx="148" cy="108" r="2.4" fill="#2f352f"></circle></svg><p class="mt-3 text-[13px] leading-relaxed text-ink-500">Both methods read the same embedding. kNN votes with the 5 nearest labeled points; the linear probe only checks which side of its boundary you are on. Try the pocket of anomalous points at the bottom left: kNN follows the local labels, the linear probe c
96annot.</p></div><div class="flex flex-col"><div class="border-b border-ink-700/10 p-5 bg-[#e9f7eb]"><div class="font-sans text-[0.78rem] text-ink-500">kNN (k = 5)</div><div class="display-serif mt-2 text-2xl font-semibold text-ink-900">clean</div><div class="mt-1 font-mono text-[12px] tabular-nums text-ink-500">1<!-- --> of 5 neighbors anomalous</div></div><div class="border-b border-ink-700/10 p-5 bg-[#e9f7eb]"><div class="font-sans text-[0.78rem] text-ink-500">Linear probe</div><div class="display-serif mt-2 text-2xl font-semibold text-ink-900">clean</div><div class="mt-1 font-mono text-[12px] text-ink-500">side of the fixed boundary</div></div><div class="flex-1 p-5"><div class="font-sans text-[0.78rem] text-accent-coral-deep">They agree here</div><p class="mt-2 text-[13px] leading-relaxed text-ink-600">Move toward the boundary, or into the bottom-left pocket, to find where the two rules split.</p></div></div></div><div class="divide-y divide-ink-700/10 border-t border-ink-700/10"><div class="grid gap-4 p-6 md:grid-cols-[140px_1fr_180px] md:items-center"><div class="font-sans text-[0.78rem] text-ink-500">kNN</div><div><div class="text-lg font-semibold text-ink-900">Store labeled embeddings</div><div class="mt-1 text-[14px] text-ink-600">Prediction comes from nearby labeled examples.</div></div><div class="font-sans text-[0.78rem] text-ink-500">useful when labels change</div></div><div class="grid gap-4 p-6 md:grid-cols-[140px_1fr_180px] md:items-center"><div class="font-sans text-[0.78rem] text-ink-500">linear probe</div><div><div class="text-lg font-semibold text-ink-900">Learn a boundary</div><div class="mt-1 text-[14px] text-ink-600">A small model learns a boundary in embedding space.</div></div><div class="font-sans text-[0.78rem] text-ink-500">useful with enough labels</div></div></div><div class="grid gap-0 border-t border-ink-700/10 lg:grid-cols-2"><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">kNN watch-outs</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Sensitive to noisy labels</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Needs similarity search at inference</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Does not give calibrated probabilities by default</span></li></ul></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">linear probe watch-outs</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Needs enough examples per class</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Must be re-fit when labels change</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Does not show similar examples by itself</span></li></ul></div></div></div><figcaption class="mt-3 max-w-[560px] font-sans text-[0.78rem] leading-[1.5] text-ink-500"><span class="font-semibold text-ink-800">Figure <!-- -->2</span> <!-- -->Same embeddings, two classifiers. Drag the query point; kNN votes with its neighbours while the linear probe answers by which side of its boundary the point falls on, and the anomalous pocket shows where they disagree.</figcaption></figure></div></section><section id="lora" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">12</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">LoRA fine-tuning</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">LoRA keeps most model weights fixed and learns a small low-rank update for the task.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="p-7"><div class="grid gap-4 text-center md:grid-cols-[1fr_auto_1fr_auto_1fr] md:items-center"><div class="rounded-[3px] border p-6 border-ink-700/10 bg-white/75"><div class="display-serif text-3xl font-semibold text-ink-900">W</div><div class="mt-2 text-[13px] text-ink-500">frozen base</div></div><div class="font-mono text-xl text-accent-forest/55">+</div><div class="rounded-[3px] border p-6 border-accent-emerald/35 bg-white"><div class="display-serif text-3xl font-semibold text-ink-900">B A</div><div class="mt-2 text-[13px] text-ink-500">small trainable update</div></div><div class="font-mono text-xl text-accent-forest/55">=</div><div class="rounded-[3px] border p-6 border-ink-700/10 bg-white/75"><div class="display-serif text-3xl font-semibold text-ink-900">W + BA</div><div class="mt-2 text-[13px] text-ink-500">adapted behavior</div></div></div><div class="mt-6 grid gap-4 lg:grid-cols-3"><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">cost spectrum</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Full fine-tune: update all weights</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>LoRA: train adapter matrices</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>QLoRA: quantized base plus LoRA adapters</span></li></ul></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">rank r</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Rank controls adapter size</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Typical r = 8-32</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Larger r adds flexibility and overfit risk</span></li></ul></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">adapter files</div><ul class="mt-3 space-y-2 text-[13px] leading-relaxed text-ink-600"><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Swap adapters without replacing the base model</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>Each task can have its own adapter</span></li><li class="flex gap-2"><span class="mt-[0.65em] h-1.5 w-1.5 shrink-0 rounded-full bg-accent-forest/55"></span><span>QLoRA stores the base in 4-bit and trains adapters in float16</span></li></ul></div></div></div></div></div></section><section id="fine-tune-tools" class="scroll-mt-24"><header class="mb-6 max-w-[620px]"><div class="flex items-baseline gap-3"><span class="font-sans text-[0.8rem] text-ink-400">13</span><h2 class="font-display text-[1.28rem] font-semibold leading-snug tracking-[-0.008em] text-ink-900">Fine-tuning tools</h2></div><p class="mt-3 max-w-[560px] text-pretty font-display text-[1.05rem] leading-[1.65] text-ink-700">There are many post-training methods beyond this tutorial. Open-source tools make them practical enough to try.</p></header><div class="min-w-0"><div class="relative overflow-hidden border border-hairline bg-white "><div class="p-6"><div class="grid gap-4 lg:grid-cols-2"><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">supervised tuning</div><div class="mt-3 flex flex-wrap gap-2"><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">SFT</span><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">instruction tuning</span><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">adapter tuning</span></div></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">preference tuning</div><div class="mt-3 flex flex-wrap gap-2"><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">RLHF</span><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">RLAIF</span><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">DPO</span><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">PPO / GRPO</span></div></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">efficient tuning</div><div class="mt-3 flex flex-wrap gap-2"><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">LoRA</span><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">QLoRA</span><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">prompt tuning</span><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">prefix tuning</span></div></div><div class="rounded-[3px] border border-ink-700/10 bg-white/70 p-4"><div class="font-sans text-[0.78rem] text-ink-500">model editing</div><div class="mt-3 flex flex-wrap gap-2"><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">distillation</span><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">reward modeling</span><span class="rounded-sm bg-ink-100 px-2 py-1 text-[13px] text-ink-700">tool-use tuning</span></div></div></div><div class="mt-6 rounded-[3px] border border-accent-emerald/25 bg-white p-5"><div class="font-sans text-[0.78rem] text-ink-500">Useful open-source tools</div><div class="mt-3 flex flex-wrap gap-2"><span class="rounded-[2px] border border-ink-700/10 bg-white px-3 py-2 font-semibold text-ink-900">PEFT</span><span class="rounded-[2px] border border-ink-700/10 bg-white px-3 py-2 font-semibold text-ink-900">TRL</span><span class="rounded-[2px] border border-ink-700/10 bg-white px-3 py-2 font-semibold text-ink-900">Unsloth</span><span class="rounded-[2px] border border-ink-700/10 bg-white px-3 py-2 font-semibold text-ink-900">Axolotl</span><span class="rounded-[2px] border border-ink-700/10 bg-white px-3 py-2 font-semibold text-ink-900">VERL</span></div><p class="mt-4 text-[13px] leading-relaxed text-ink-600">These tools make experiments easier to run, but they do not replace clean splits, metrics, and ablations.</p></div></div></div></div></section></article><nav aria-label="Part navigation" class="mt-20 grid gap-x-10 gap-y-8 border-t border-hairline pt-8 sm:grid-cols-2"><span></span><a class="group block text-right" href="/tutorials/foundation-models-xrd/part-2"><div class="font-sans text-[0.8rem] text-ink-400">Next</div><div class="mt-1 font-display text-[1.05rem] font-semibold leading-snug text-ink-900 transition-colors group-hover:text-accent-forest">Part II: evaluation is part of the method<span aria-hidden="true" class="ml-1 inline-block transition-transform duration-200 ease-out group-hover:translate-x-0.5">→</span></div><div class="mt-1 font-sans text-[0.85rem] text-ink-500">splits, confusion counts, and the right F1</div></a></nav></main><aside class="hidden lg:block"><div class="sticky top-20"><nav aria-label="Contents" class="custom-scroll max-h-[calc(100vh-6rem)] overflow-y-auto border border-hairline bg-white p-5"><div class="mb-3 font-sans text-[0.8rem] text-ink-500">Contents</div><ul class="space-y-1.5"><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">Where foundation models fit in</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">What foundation models can do</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">Pretraining is done once. Adaptation happens in the lab.</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">The adaptation landscape</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">Compute and memory trade-offs</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">Baseline: zero-shot prompting</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">Ask for structured outputs</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">Strategy 1: few-shot prompting</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">Strategy 2: frozen features</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">Example clustering of embeddings</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">kNN vs. linear probe</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">Strategy 3: LoRA fine-tuning</button></li><li class=""><button type="button" class="block w-full text-left font-sans text-[0.8rem] leading-snug transition-colors  text-ink-500 hover:text-ink-900">Fine-tuning tools</button></li></ul></nav></div></aside></div></div></main><footer><div class="mx-auto max-w-[1020px] px-6 py-7 font-sans text-[0.78rem] text-ink-400 sm:px-12"><span>nodeology.ai is maintained by<!-- --> <a href="https://xiangyu-yin.com" target="_blank" rel="noopener noreferrer" class="transition-colors hover:text-accent-forest">Xiangyu Yin</a>.</span></div></footer></div><!--/$--></div>
97  </body>
98</html>

Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.