1<!DOCTYPE html> 2<html lang="en-US"> 3 <head> 4 <title>Xiaoyi Zhang | A\</title> 5 6 <meta charset="UTF-8"> 7 <meta http-equiv="X-UA-Compatible" content="IE=edge"> 8 <meta name="viewport" content="width=device-width, initial-scale=1"> 9 <meta name="description" content="The Minimal Light is a simple and elegant jekyll theme for academic personal homepage."> 10 11 <meta name="keywords" content="minimal light"> 12 13 14 <link rel="canonical" href="https://minimal-light-theme.yliu.me/"/> 15 16 17 <link rel="icon" media="(prefers-color-scheme:dark)" href="./assets/img/favicon-dark.png" type="image/png" /> 18 <link rel="icon" media="(prefers-color-scheme:light)" href="./assets/img/favicon.png" type="image/png" /> 19
19<script src="./assets/js/favicon-switcher.js" type="application/javascript"></script>
19 20 21 <link rel=stylesheet href=https://cdnjs.cloudflare.com/ajax/libs/academicons/1.8.6/css/academicons.min.css integrity="sha256-uFVgMKfistnJAfoCUQigIl+JfUaP47GrRKjf6CTPVmw=" crossorigin=anonymous> 22 <link rel=stylesheet href=https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/all.min.css integrity="sha256-+N4/V/SbAFiW1MPBCXnfnP9QSN3+Keu+NlB+0ev/YKQ=" crossorigin=anonymous> 23 24 25 <link rel="stylesheet" href="./assets/css/font_sans_serif.css"> 26 27 28 <link rel="stylesheet" href="./assets/css/style.css"> 29 <link rel="stylesheet" href="./assets/css/publications.css"> 30 31 32 </head> 33 <body> 34 <div class="wrapper"> 35 <header> 36 37 38 <a class="image avatar"><img src="./assets/img/avatar.jpg" alt="avatar" /></a> 39 40 41 42 <a class="image avatar"><img src="./assets/img/nvshen.jpg" alt="avatar" /></a> 43 44 45 <h1>Xiaoyi Zhang</h1> 46 47 48 <position style="font-size:1.10rem;">Member of Technical Staff</position> 49 <br> 50 51 52 <autocolor>A\</autocolor> 53 <br> 54 <br> 55 56 57 <email>xiaoyizuw at gmail.com</email> 58 59 60 <br> 61 <br> 62 <div class="social-icons"> 63 64 <a style="margin: 0 5px 0 0" href="https://scholar.google.com/citations?user=8qtpdWAAAAAJ"> 65 <i class="ai ai-google-scholar" style="font-size:1.2rem"></i> 66 </a> 67 68 69 70 <a style="margin: 0 5px 0 0" href="assets/files/CV_20240617.pdf"> 71 <i class="ai ai-cv" style="font-size:1.3rem;"></i> 72 </a> 73 74 75 76 77 78 <a style="margin: 0 5px 0 0" href="https://www.linkedin.com/in/xiaoyiz/"> 79 <i class="fab fa-linkedin"></i> 80 </a> 81 82 83 84 <a style="margin: 0 0 0 0" href="https://x.com/xiaoyiz_uw"> 85 <i class="fab fa-twitter"></i> 86 </a> 87 88 </div> 89 <br> 90 91 </header> 92 <section> 93 94 <h2 id="about-me">About Me</h2> 95 96<p>I am currently at A\, focusing on HCI+AI research and product.</p> 97 98<p>Before this, I was a <strong>Staff Researcher & Manager</strong> at Apple (2018-2024), passionate about building <strong>userâfriendly developer tools for ML engineers</strong> to boost their productivity. My team built ML infra and tools for model analysis, optimization, and testing, which is mission-critical for the launch of <strong>Apple Intelligence</strong> (<a href="https://machinelearning.apple.com/research/introducing-apple-foundation-models">Research post, Optimization section</a>) and <strong>Vision Pro</strong> (<a href="https://machinelearning.apple.com/research/talaria">Paper</a>).</p> 99 100<p>I received CS (HCI focus) PhD from UW, and interned at Google Research (with Kayur Patel on Colab) and MSR (with Meredith Ringel Morris). My first project at Apple was extending my PhD research to an ML-powered <a href="https://techcrunch.com/2020/12/03/iphones-can-now-automatically-recognize-and-label-buttons-and-ui-features-for-blind-users/">iOS accessibility feature</a> that supports 20M+ blind usersâ daily life.</p> 101 102<p>Research Interests: <strong>Human-Computer Interaction (HCI)</strong>, ML Developer Tools, ML Model Optimization, and UI Accessibility.</p> 103 104<p><br /></p> 105 106<h2 id="achievements">Achievements:</h2> 107<ul> 108 <li><strong>Auto-OPT:</strong> ML infra to optimize all Apple Intelligence LLMs (on-device & server).</li> 109 <li><strong>Talaria:</strong> Developer tool to enable Vision Pro ML models to run in low latency.</li> 110 <li><strong>Screen Recognition:</strong> The first on-device ML model to recognize and announce iOS App UI, benefiting 20M+ blind users.</li> 111 <li><strong>2 Best Papers, 3 Best Paper nominees, 5 Patents.</strong></li> 112</ul> 113 114<p><br /></p> 115<h2 id="publications" style="margin: 2px 0px -15px;">Selected Publications</h2> 116 117<div class="publications"> 118Please see <b>full publication list (25+)</b> in <a href="https://scholar.google.com/citations?user=8qtpdWAAAAAJ">Google Scholar</a> 119<br /><br /> 120 121<ol class="bibliography"> 122 123 124 125<li> 126<div class="pub-row"> 127 <div class="col-sm-9" style="position: relative;"> 128 <div class="title"><a href="https://arxiv.org/pdf/2404.03085">Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference</a></div> 129 <div class="author">Fred Hohman, Chaoqun Wang, Jinmook Lee, Jochen Görtler, Dominik Moritz, Jeffrey P. Bigham, Zhile Ren, Cecile Foret, Qi Shan, <strong>Xiaoyi Zhang</strong></div> 130 <div class="periodical"><em>CHI 2024</em> 131 </div> 132 <div class="links"> 133 134 <a href="https://arxiv.org/pdf/2404.03085" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">PDF</a> 135 136 137 138 139 <a href="https://machinelearning.apple.com/research/talaria" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">Video</a> 140 141 142 143 <strong style="color:#2ea143; font-size: small">ð Best Paper Nominee; Critical for Apple Intelligence and Vision Pro models</strong> 144 145 146 </div> 147 </div> 148</div> 149</li> 150 151<br /> 152 153 154 155<li> 156<div class="pub-row"> 157 <div class="col-sm-9" style="position: relative;"> 158 <div class="title"><a href="https://docs-assets.developer.apple.com/ml-research/papers/icon-labelling-mobile-apps-chi-22.pdf">Towards Complete Icon Labeling in Mobile Applications</a></div> 159 <div class="author">Jieshan Chen, Amanda Swearngin, Jason Wu, Titus Barik, Jeffrey Nichols, <strong>Xiaoyi Zhang</strong></div> 160 <div class="periodical"><em>CHI 2022</em> 161 </div> 162 <div class="links"> 163 164 <a href="https://docs-assets.developer.apple.com/ml-research/papers/icon-labelling-mobile-apps-chi-22.pdf" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">PDF</a> 165 166 167 168 <a href="https://machinelearning.apple.com/research/icon-labelling" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">Project Page</a> 169 170 171 172 173 <strong style="color:#2ea143; font-size: small">Available in iOS</strong> 174 175 176 </div> 177 </div> 178</div> 179</li> 180 181<br /> 182 183 184 185<li> 186<div class="pub-row"> 187 <div class="col-sm-9" style="position: relative;"> 188 <div class="title"><a href="https://docs-assets.developer.apple.com/ml-research/papers/screen-recognition-chi-2021.pdf">Screen Recognition: Creating Accessibility Metadata for Mobile Apps from Pixels</a></div> 189 <div class="author"><strong>Xiaoyi Zhang</strong>, Lilian de Greef, Amanda Swearngin, Samuel White, Kyle Murray, Lisa Yu, Qi Shan, Jeffrey Nichols, Jason Wu, Chris Fleizach, Aaron Everitt, Jeffrey P Bigham</div> 190 <div class="periodical"><em>CHI 2021</em> 191 </div> 192 <div class="links"> 193 194 <a href="https://docs-assets.developer.apple.com/ml-research/papers/screen-recognition-chi-2021.pdf" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">PDF</a> 195 196 197 198 <a href="https://machinelearning.apple.com/research/creating-accessibility-metadata" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">Project Page</a> 199 200 201 <a href="https://www.youtube.com/watch?v=Z0pv0ZqFnC0" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">Video</a> 202 203 204 205 <strong style="color:#2ea143; font-size: small">ð Best Paper; Available in iOS</strong> 206 207 208 </div> 209 </div> 210</div> 211</li> 212 213<br /> 214 215 216 217<li> 218<div class="pub-row"> 219 <div class="col-sm-9" style="position: relative;"> 220 <div class="title"><a href="https://arxiv.org/pdf/2105.01734">When Can Accessibility Help? Exploration of Accessibility Feature Recommendation on Mobile Devices</a></div> 221 <div class="author">Jason Wu, Gabriel Reyes, Sam C White, <strong>Xiaoyi Zhang</strong>, Jeffrey P Bigham</div> 222 <div class="periodical"><em>W4A 2021</em> 223 </div> 224 <div class="links"> 225 226 <a href="https://arxiv.org/pdf/2105.01734" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">PDF</a> 227 228 229 230 <a href="https://machinelearning.apple.com/research/recommending-accessibility" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">Project Page</a> 231 232 233 234 235 <strong style="color:#2ea143; font-size: small">ð Best Paper</strong> 236 237 238 </div> 239 </div> 240</div> 241</li> 242 243<br /> 244 245 246 247<li> 248<div class="pub-row"> 249 <div class="col-sm-9" style="position: relative;"> 250 <div class="title"><a href="https://faculty.washington.edu/wobbrock/pubs/assets-18.01.pdf">Examining Image-Based Button Labeling for Accessibility in Apps through Large-Scale Analysis</a></div> 251 <div class="author">Anne Spencer Ross, <strong>Xiaoyi Zhang</strong>, James Fogarty, Jacob O Wobbrock</div> 252 <div class="periodical"><em>ASSETS 2018</em> 253 </div> 254 <div class="links"> 255 256 <a href="https://faculty.washington.edu/wobbrock/pubs/assets-18.01.pdf" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">PDF</a> 257 258 259 260 261 262 263 <strong style="color:#2ea143; font-size: small">ð Best Paper Nominee</strong> 264 265 266 </div> 267 </div> 268</div> 269</li> 270 271<br /> 272 273 274 275<li> 276<div class="pub-row"> 277 <div class="col-sm-9" style="position: relative;"> 278 <div class="title"><a href="https://faculty.washington.edu/wobbrock/pubs/assets-17.pdf">Epidemiology as a Framework for Large-scale Mobile Application Accessibility Assessment</a></div> 279 <div class="author">Anne Spencer Ross, <strong>Xiaoyi Zhang</strong>, James Fogarty, Jacob O Wobbrock</div> 280 <div class="periodical"><em>ASSETS 2017</em> 281 </div> 282 <div class="links"> 283 284 <a href="https://faculty.washington.edu/wobbrock/pubs/assets-17.pdf" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">PDF</a> 285 286 287 288 289 290 291 <strong style="color:#2ea143; font-size: small">ð Best Paper Nominee</strong> 292 293 294 </div> 295 </div> 296</div> 297</li> 298 299<br /> 300 301 302 303<li> 304<div class="pub-row"> 305 <div class="col-sm-9" style="position: relative;"> 306 <div class="title"><a href="https://faculty.washington.edu/wobbrock/pubs/chi-17.03.pdf">Interaction Proxies for Runtime Repair and Enhancement of Mobile Application Accessibility</a></div> 307 <div class="author"><strong>Xiaoyi Zhang</strong>, Anne Spencer Ross, Anat Caspi, James Fogarty, Jacob O Wobbrock</div> 308 <div class="periodical"><em>CHI 2017</em> 309 </div> 310 <div class="links"> 311 312 <a href="https://faculty.washington.edu/wobbrock/pubs/chi-17.03.pdf" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">PDF</a> 313 314 315 316
317 <a href="https://www.youtube.com/watch?v=v15FfDAsf04" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">Video</a> 318 319 320 321 <strong style="color:#2ea143; font-size: small">Foundation Work for Screen Recognition</strong> 322 323 324 </div> 325 </div> 326</div> 327</li> 328 329<br /> 330 331 332 333<li> 334<div class="pub-row"> 335 <div class="col-sm-9" style="position: relative;"> 336 <div class="title"><a href="https://www.microsoft.com/en-us/research/uploads/prod/2017/01/gazespeakUpdated.pdf">Smartphone-Based Eye Gesture Communication for People with Motor Disabilities</a></div> 337 <div class="author"><strong>Xiaoyi Zhang</strong>, Harish Kulkarni, Meredith Ringel Morris</div> 338 <div class="periodical"><em>CHI 2017</em> 339 </div> 340 <div class="links"> 341 342 <a href="https://www.microsoft.com/en-us/research/uploads/prod/2017/01/gazespeakUpdated.pdf" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">PDF</a> 343 344 345 346 347 <a href="https://www.youtube.com/watch?v=Nffiaaw9PvQ" class="btn btn-sm z-depth-0" role="button" target="_blank" style="font-size:12px;">Video</a> 348 349 350 351 <strong style="color:#2ea143; font-size: small">Patent <a href="https://patents.google.com/patent/US10353475B2/en" target="_blank">(Link)</a></strong> 352 353 354 </div> 355 </div> 356</div> 357</li> 358 359<br /> 360 361 362 363</ol> 364</div> 365 366 367 368 <br> 369 370 371 372 </section> 373 <footer> 374 375 </footer> 376 </div> 377
377<script src="/assets/js/scale.fix.js"></script>
377 378 379
379<script>
vendor: 364 bytes, lines 379-384
379 380 (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ 381 (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), 382 m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) 383 })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); 384 ga('create', '
384UA-111540567-4
vendor: 46 bytes, lines 384-386
384', 'auto'); 385 ga('send', 'pageview'); 386
386</script>
386 387 388 </body> 389</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.