https://www.mengweiren.com/research/

1<!DOCTYPE html><html lang="en"> <head><meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta name="description" content="Selected publications in computer vision, generative models, and image editing."><title>Research - Mengwei Ren</title><link rel="icon" href="/images/icon.png"><link rel="preconnect" href="https://fonts.googleapis.com"><link rel="preconnect" href="https://fonts.gstatic.com" crossorigin><link rel="stylesheet" href="/_assets/about.D2_wbF-k.css">
2<style>.research-intro[data-astro-cid-ynj567il]{max-width:1100px;padding:var(--space-xl) clamp(2rem,5vw,6rem) 0;margin:0 auto;text-align:left}.research-intro__title[data-astro-cid-ynj567il]{font-family:var(--font-display);font-size:clamp(2rem,4vw,3rem);font-weight:300;letter-spacing:-.02em;margin-bottom:1.2rem;color:var(--color-text)}.research-intro__text[data-astro-cid-ynj567il]{max-width:900px;margin:0 auto 1.25rem;font-size:.95rem;font-weight:300;line-height:1.9;color:var(--color-text-secondary)}.research-intro__text[data-astro-cid-ynj567il] a[data-astro-cid-ynj567il]{text-decoration:underline;text-underline-offset:4px;color:var(--color-accent)}.pub-row[data-astro-cid-ynj567il]{display:grid;grid-template-columns:220px 1fr;gap:var(--space-md);padding:1.5rem;border-radius:6px;border-left:2px solid transparent;align-items:start;transition:transform .3s ease,box-shadow .3s ease,border-color .3s ease}.pub-row[data-astro-cid-ynj567il]:hover{transform:translateY(-3px);box-shadow:0 8px 30px #0000000f;border-left-color:var(--color-accent)}.pub-row__image[data-astro-cid-ynj567il]{overflow:hidden;border-radius:4px;background-color:#f3f1ee;line-height:0;aspect-ratio:16 / 10;display:flex;align-items:center;justify-content:center}.pub-row__image[data-astro-cid-ynj567il] img[data-astro-cid-ynj567il]{width:100%;height:100%;object-fit:contain;display:block;padding:10px;transition:transform .6s ease}.pub-row[data-astro-cid-ynj567il]:hover .pub-row__image[data-astro-cid-ynj567il] img[data-astro-cid-ynj567il]{transform:scale(1.03)}.pub-row__content[data-astro-cid-ynj567il] .tag[data-astro-cid-ynj567il]{margin-bottom:1rem}.pub-row__title[data-astro-cid-ynj567il]{font-size:1.2rem;line-height:1.3;margin-bottom:.75rem;color:var(--color-text)}.pub-row__authors[data-astro-cid-ynj567il]{font-size:.85rem;line-height:1.7;margin-bottom:.75rem}.pub-row__authors[data-astro-cid-ynj567il] strong{color:var(--color-text);font-weight:600}.pub-row__desc[data-astro-cid-ynj567il]{font-size:.9rem;line-height:1.75;margin-bottom:var(--space-sm)}.pub-row__links[data-astro-cid-ynj567il]{display:flex;gap:1.25rem;flex-wrap:wrap}.pub-row__links[data-astro-cid-ynj567il] a[data-astro-cid-ynj567il]{font-size:.9rem;font-weight:500;letter-spacing:.06em;text-transform:uppercase;color:var(--color-accent);padding:.3em 0;border-bottom:1px solid transparent;transition:border-color var(--transition-fast)}.pub-row__links[data-astro-cid-ynj567il] a[data-astro-cid-ynj567il]:hover{border-bottom-color:var(--color-accent)}@keyframes fadeUp{0%{opacity:0;transform:translateY(30px)}to{opacity:1;transform:translateY(0)}}@media(max-width:768px){.pub-row[data-astro-cid-ynj567il]{grid-template-columns:1fr;gap:var(--space-md)}.pub-row__image[data-astro-cid-ynj567il]{max-width:400px}}
3</style></head> <body> <nav class="nav" id="main-nav" data-astro-cid-dmqpwcec> <div class="nav__inner" data-astro-cid-dmqpwcec> <a href="/" class="nav__logo" data-astro-cid-dmqpwcec>Mengwei Ren</a> <button class="nav__toggle" id="nav-toggle" aria-label="Toggle menu" data-astro-cid-dmqpwcec> <span data-astro-cid-dmqpwcec></span> <span data-astro-cid-dmqpwcec></span> </button> <ul class="nav__links" id="nav-links" data-astro-cid-dmqpwcec> <li data-astro-cid-dmqpwcec> <a href="/research" class="nav__link is-active" data-astro-cid-dmqpwcec> Research </a> </li><li data-astro-cid-dmqpwcec> <a href="/photography" class="nav__link" data-astro-cid-dmqpwcec> Photography </a> </li> </ul> </div> </nav> 

3<script type="module">const l=document.getElementById("main-nav"),s=document.getElementById("nav-toggle"),t=document.getElementById("nav-links");let n=0;window.addEventListener("scroll",()=>{const e=window.scrollY;l.classList.toggle("nav--scrolled",e>80),l.classList.toggle("nav--hidden",e>n&&e>300),n=e});s.addEventListener("click",()=>{const e=t.classList.toggle("is-open");s.classList.toggle("is-open",e),document.body.style.overflow=e?"hidden":""});t.querySelectorAll("a").forEach(e=>{e.addEventListener("click",()=>{t.classList.remove("is-open"),s.classList.remove("is-open"),document.body.style.overflow=""})});</script>

3  <main>   <section class="research-intro" data-astro-cid-ynj567il> <div class="container" data-astro-cid-ynj567il> <p class="research-intro__text reveal" data-astro-cid-ynj567il>
4My research focuses on building controllable, identity-preserving generative models for image editing,
5        composition, and restoration. I am particularly interested in problems where generation operates on
6        real images, requiring outputs to remain consistent, recognizable, and faithful under diverse,
7        real-world conditions.
8</p> <p class="research-intro__text reveal" data-astro-cid-ynj567il>
9My work spans early research exploration to tech transfer into real products, including large-scale
10        data curation, pipeline design, and deployment in production settings. At Adobe, I led the development
11        of generative harmonization technology from prototype to production, shipped as Harmonize in Photoshop,
12        enabling photorealistic compositing at scale.
13</p> <p class="research-intro__text reveal" data-astro-cid-ynj567il>
14Currently, my research explores personalized generative models for photography, preserving identity
15        across editing and generation.
16</p> <p class="research-intro__text reveal" data-astro-cid-ynj567il>
17See full publication list at <a href="https://scholar.google.com/citations?user=hVHCAAYAAAAJ&hl=en&authuser=1" target="_blank" rel="noopener" data-astro-cid-ynj567il>Google Scholar</a>.
18</p> </div> </section>  <section class="section publications" data-astro-cid-ynj567il> <div class="container" data-astro-cid-ynj567il> <span class="section-label reveal" data-astro-cid-ynj567il>Selected Publications</span> <article class="pub-row reveal reveal--delay-1" data-astro-cid-ynj567il> <div class="pub-row__image" data-astro-cid-ynj567il> <img src="/images/synthlight_thumb_new.jpg" alt="SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces" loading="lazy" data-astro-cid-ynj567il> </div> <div class="pub-row__content" data-astro-cid-ynj567il> <span class="tag" data-astro-cid-ynj567il>CVPR 2025</span> <h3 class="pub-row__title" data-astro-cid-ynj567il>SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces</h3> <p class="pub-row__authors" data-astro-cid-ynj567il>Sumit Chaturvedi, <strong>Mengwei Ren</strong>, Yannick Hold-Geoffroy, Jingyuan Liu, Julie Dorsey, Zhixin Shu</p> <p class="pub-row__desc" data-astro-cid-ynj567il>We explore portrait relighting by training a diffusion model to re-render synthetic faces.</p> <div class="pub-row__links" data-astro-cid-ynj567il> <a href="https://vrroom.github.io/synthlight/" target="_blank" rel="noopener" data-astro-cid-ynj567il>Project</a> <a href="https://arxiv.org/abs/2501.09756" target="_blank" rel="noopener" data-astro-cid-ynj567il>arXiv</a>   </div> </div> </article><article class="pub-row reveal reveal--delay-2" data-astro-cid-ynj567il> <div class="pub-row__image" data-astro-cid-ynj567il> <img src="/images/shadow_thumb_v4.jpg" alt="Generative Portrait Shadow Removal" loading="lazy" data-astro-cid-ynj567il> </div> <div class="pub-row__content" data-astro-cid-ynj567il> <span class="tag" data-astro-cid-ynj567il>SIGGRAPH Asia 2024</span> <h3 class="pub-row__title" data-astro-cid-ynj567il>Generative Portrait Shadow Removal</h3> <p class="pub-row__authors" data-astro-cid-ynj567il>Jae Shin Yoon, Zhixin Shu, <strong>Mengwei Ren</strong>, Xuaner Zhang, Yannick Hold-Geoffroy, Krishna Kumar Singh, He Zhang</p> <p class="pub-row__desc" data-astro-cid-ynj567il>A high-fidelity portrait shadow removal model that effectively enhances portraits by predicting appearance under disturbing shadows and highlights.</p> <div class="pub-row__links" data-astro-cid-ynj567il>  <a href="https://arxiv.org/abs/2410.05525" target="_blank" rel="noopener" data-astro-cid-ynj567il>arXiv</a>   </div> </div> </article><article class="pub-row reveal reveal--delay-1" data-astro-cid-ynj567il> <div class="pub-row__image" data-astro-cid-ynj567il> <img src="/images/relightful_thumb_wide.gif" alt="Relightful Harmonization: Lighting-aware Portrait Backgroun

18d Replacement" loading="lazy" data-astro-cid-ynj567il> </div> <div class="pub-row__content" data-astro-cid-ynj567il> <span class="tag" data-astro-cid-ynj567il>CVPR 2024</span> <h3 class="pub-row__title" data-astro-cid-ynj567il>Relightful Harmonization: Lighting-aware Portrait Background Replacement</h3> <p class="pub-row__authors" data-astro-cid-ynj567il><strong>Mengwei Ren</strong>, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, He Zhang</p> <p class="pub-row__desc" data-astro-cid-ynj567il>A lighting-aware diffusion model designed to seamlessly harmonize sophisticated lighting effects for foreground portraits using any background image.</p> <div class="pub-row__links" data-astro-cid-ynj567il> <a href="/research/relightful-harmonization/" rel="noopener" data-astro-cid-ynj567il>Project</a> <a href="https://arxiv.org/abs/2312.06886" target="_blank" rel="noopener" data-astro-cid-ynj567il>arXiv</a>  <a href="/data/cvpr24.bib" data-astro-cid-ynj567il>BibTeX</a> </div> </div> </article><article class="pub-row reveal reveal--delay-2" data-astro-cid-ynj567il> <div class="pub-row__image" data-astro-cid-ynj567il> <img src="/images/deblurring1.png" alt="Multiscale Structure Guided Diffusion for Image Deblurring" loading="lazy" data-astro-cid-ynj567il> </div> <div class="pub-row__content" data-astro-cid-ynj567il> <span class="tag" data-astro-cid-ynj567il>ICCV 2023</span> <h3 class="pub-row__title" data-astro-cid-ynj567il>Multiscale Structure Guided Diffusion for Image Deblurring</h3> <p class="pub-row__authors" data-astro-cid-ynj567il><strong>Mengwei Ren</strong>, Mauricio Delbracio, Hossein Talebi, Guido Gerig, Peyman Milanfar</p> <p class="pub-row__desc" data-astro-cid-ynj567il>A simple yet effective structure guidance for image-conditioned diffusion models that leads to significantly better visual quality on unseen images.</p> <div class="pub-row__links" data-astro-cid-ynj567il>  <a href="https://arxiv.org/abs/2212.01789" target="_blank" rel="noopener" data-astro-cid-ynj567il>arXiv</a>  <a href="/data/deblurring23.bib" data-astro-cid-ynj567il>BibTeX</a> </div> </div> </article><article class="pub-row reveal reveal--delay-1" data-astro-cid-ynj567il> <div class="pub-row__image" data-astro-cid-ynj567il> <img src="/images/kaf_thumb.jpg" alt="Keypoint-Augmented Self-Supervised Learning for Medical Image Segmentation with Limited Annotation" loading="lazy" data-astro-cid-ynj567il> </div> <div class="pub-row__content" data-astro-cid-ynj567il> <span class="tag" data-astro-cid-ynj567il>NeurIPS 2023</span> <h3 class="pub-row__title" data-astro-cid-ynj567il>Keypoint-Augmented Self-Supervised Learning for Medical Image Segmentation with Limited Annotation</h3> <p class="pub-row__authors" data-astro-cid-ynj567il>Zhangsihao Yang*, <strong>Mengwei Ren*</strong>, Kaize Ding, Guido Gerig, Yalin Wang</p> <p class="pub-row__desc" data-astro-cid-ynj567il>A keypoint-augmented fusion layer that extracts representations preserving both short- and long-range self-attention in a self-supervised manner.</p> <div class="pub-row__links" data-astro-cid-ynj567il>  <a href="https://arxiv.org/abs/2310.01680" target="_blank" rel="noopener" data-astro-cid-ynj567il>arXiv</a>   </div> </div> </article><article class="pub-row reveal reveal--delay-2" data-astro-cid-ynj567il> <div class="pub-row__image" data-astro-cid-ynj567il> <img src="/images/spatiotemporal_thumb.jpg" alt="Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis" loading="lazy" data-astro-cid-ynj567il> </div> <div class="pub-row__content" data-astro-cid-ynj567il> <span class="tag" data-astro-cid-ynj567il>NeurIPS (oral) 2022</span> <h3 class="pub-row__title" data-astro-cid-ynj567il>Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis</h3> <p class="pub-row__authors" data-astro-cid-ynj567il><strong>Mengwei Ren</strong>, Neel Dey, Martin A. Styner, Kelly N. Botteron, Guido Gerig</p> <p class="pub-row__desc" data-astro-cid-ynj567il>A local and multi-scale spatiotemporal representation learning method for image-to-image architectures trained on longitudinal images.</p> <div class="pub-row__links" data-astro-cid-ynj567il> <a href="/research/spatiotemporal-learning/" rel="noopener" data-astro-cid-ynj567il>Project</a> <a href="https://arxiv.org/abs/2206.04281" target="_blank" rel="noopener" data-astro-cid-ynj567il>arXiv</a> <a href="https://github.com/mengweiren/longitudinal-representation-learning" target="_blank" rel="noopener" data-astro-cid-ynj567il>GitHub</a> <a href="/data/spatiotemporal22.bib" data-astro-cid-ynj567il>BibTeX</a> </div> </div> </article><article class="pub-row reveal reveal--delay-1" data-astro-cid-ynj567il> <div class="pub-row__image" data-astro-cid-ynj567il> <img src="/images/snorm_thumb.jpg" alt="Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization" loading="lazy" data-astro-cid-ynj567il> </div> <div class="pub-row__content" data-astro-cid-ynj567il> <span class="tag" data-astro-cid-ynj567il>IEEE Trans. Medical Imaging 2021</span> <h3 class="pub-row__title" data-astro-cid-ynj567il>Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization</h3> <p class="pub-row__authors" data-astro-cid-ynj567il><strong>Mengwei Ren</strong>, Neel Dey, James Fishbaugh, Guido Gerig</p> <p class="pub-row__desc" data-astro-cid-ynj567il>A segmentation-renormalized image translation framework to reduce inter-scanner heterogeneity while preserving anatomical layout.</p> <div class="pub-row__links" data-astro-cid-ynj567il>  <a href="https://arxiv.org/abs/2102.06315" target="_blank" rel="noopener" data-astro-cid-ynj567il>arXiv</a> <a href="https://github.com/mengweiren/segmentation-renormalized-harmonization" target="_blank" rel="noopener" data-astro-cid-ynj567il>GitHub</a> <a href="/data/tmi21.bib" data-astro-cid-ynj567il>BibTeX</a> </div> </div> </article><article class="pub-row reveal reveal--delay-2" data-astro-cid-ynj567il> <div class="pub-row__image" data-astro-cid-ynj567il> <img src="/images/qspace_thumb.jpg" alt="Q-space Conditioned Translation Networks for Directional Synthesis of Diffusion Weighted Images from Multi-modal Structural MRI" loading="lazy" data-astro-cid-ynj567il> </div> <div class="pub-row__content" data-astro-cid-ynj567il> <span class="tag" data-astro-cid-ynj567il>MICCAI (oral) 2021</span> <h3 class="pub-row__title" data-astro-cid-ynj567il>

18Q-space Conditioned Translation Networks for Directional Synthesis of Diffusion Weighted Images from Multi-modal Structural MRI</h3> <p class="pub-row__authors" data-astro-cid-ynj567il><strong>Mengwei Ren</strong>, Heejong Kim, Neel Dey, Guido Gerig</p> <p class="pub-row__desc" data-astro-cid-ynj567il>A generative adversarial translation framework for high-quality DW image estimation with arbitrary Q-space sampling given commonly acquired structural images.</p> <div class="pub-row__links" data-astro-cid-ynj567il>  <a href="https://arxiv.org/abs/2106.13188" target="_blank" rel="noopener" data-astro-cid-ynj567il>arXiv</a> <a href="https://github.com/mengweiren/q-space-conditioned-dwi-synthesis" target="_blank" rel="noopener" data-astro-cid-ynj567il>GitHub</a> <a href="/data/miccai21.bib" data-astro-cid-ynj567il>BibTeX</a> </div> </div> </article><article class="pub-row reveal reveal--delay-1" data-astro-cid-ynj567il> <div class="pub-row__image" data-astro-cid-ynj567il> <img src="/images/atlas_thumb.jpg" alt="Generative Adversarial Registration for Improved Conditional Deformable Templates" loading="lazy" data-astro-cid-ynj567il> </div> <div class="pub-row__content" data-astro-cid-ynj567il> <span class="tag" data-astro-cid-ynj567il>ICCV 2021</span> <h3 class="pub-row__title" data-astro-cid-ynj567il>Generative Adversarial Registration for Improved Conditional Deformable Templates</h3> <p class="pub-row__authors" data-astro-cid-ynj567il>Neel Dey, <strong>Mengwei Ren</strong>, Adrian Dalca, Guido Gerig</p> <p class="pub-row__desc" data-astro-cid-ynj567il>A generative adversarial registration framework conditioned on flexible image covariates for improved deformable template estimation.</p> <div class="pub-row__links" data-astro-cid-ynj567il>  <a href="https://arxiv.org/abs/2105.04349" target="_blank" rel="noopener" data-astro-cid-ynj567il>arXiv</a> <a href="https://github.com/neel-dey/Atlas-GAN" target="_blank" rel="noopener" data-astro-cid-ynj567il>GitHub</a> <a href="/data/iccv21.bib" data-astro-cid-ynj567il>BibTeX</a> </div> </div> </article> </div> </section>  </main> <footer class="footer" data-astro-cid-sz7xmlte> <div class="footer__inner" data-astro-cid-sz7xmlte> <div class="footer__top" data-astro-cid-sz7xmlte> <a href="/" class="footer__logo" data-astro-cid-sz7xmlte>Mengwei Ren</a> <ul class="footer__links" data-astro-cid-sz7xmlte> <li data-astro-cid-sz7xmlte> <a href="mailto:[email protected]" rel="noopener" data-astro-cid-sz7xmlte> Email </a> </li><li data-astro-cid-sz7xmlte> <a href="https://scholar.google.com/citations?user=hVHCAAYAAAAJ&hl=en&authuser=1" target="_blank" rel="noopener" data-astro-cid-sz7xmlte> Scholar </a> </li><li data-astro-cid-sz7xmlte> <a href="https://github.com/mengweiren/" target="_blank" rel="noopener" data-astro-cid-sz7xmlte> GitHub </a> </li><li data-astro-cid-sz7xmlte> <a href="https://www.linkedin.com/in/mengwei-ren/" target="_blank" rel="noopener" data-astro-cid-sz7xmlte> LinkedIn </a> </li><li data-astro-cid-sz7xmlte> <a href="https://www.instagram.com/mengweir" target="_blank" rel="noopener" data-astro-cid-sz7xmlte> Instagram </a> </li> </ul> </div> <div class="footer__bottom" data-astro-cid-sz7xmlte> <p class="footer__copyright" id="footer-copyright" data-astro-cid-sz7xmlte>&copy; 2026 Mengwei Ren. All rights reserved.</p> <div class="footer__tracker" id="footer-tracker" data-astro-cid-sz7xmlte> <a href="https://clustrmaps.com/site/1c4hy" title="Visit tracker" data-astro-cid-sz7xmlte> <img src="//www.clustrmaps.com/map_v2.png?cl=e8e6e3&w=300&t=n&d=P8vcCvj_7_Z3QyyGD__x-J1xcoFUrmiib9cPIwnHKZ4&co=fafaf9&ct=9a9590" alt="Visitor map" data-astro-cid-sz7xmlte> </a> </div> </div> </div> </footer>  

18<script type="module">document.getElementById("footer-copyright")?.addEventListener("click",()=>{document.getElementById("footer-tracker")?.classList.toggle("is-visible")});</script>

18 

18<script type="module" src="/_assets/BaseLayout.astro_astro_type_script_index_0_lang.RwIyeObw.js"></script>

18 </body> </html>