1<!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title>Projects | Haw-Shiuan Chang</title> <meta name="author" content="Haw-Shiuan Chang"> <meta name="description" content="A simple, whitespace theme for academics. Based on [*folio](https://github.com/bogoli/-folio) design. "> <link rel="stylesheet" href="/assets/css/bootstrap.min.css?a4b3f509e79c54a512b890d73235ef04"> <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/css/mdb.min.css" integrity="sha256-jpjYvU3G3N6nrrBwXJoVEYI/0zw8htfFnhT9ljN3JJw=" crossorigin="anonymous"> <link defer rel="stylesheet" href="https://unpkg.com/[email protected]/dist/bootstrap-table.min.css"> <link rel="stylesheet" href="/assets/css/academicons.min.css?f0b7046b84e425c55f3463ac249818f5"> <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700|Roboto+Slab:100,300,400,500,700|Material+Icons"> <link rel="stylesheet" href="/assets/css/jekyll-pygments-themes-github.css?19f3075a2d19613090fe9e16b564e1fe" media="" id="highlight_theme_light"> <link rel="shortcut icon" href="data:image/svg+xml,<svg%20xmlns=%22http://www.w3.org/2000/svg%22%20viewBox=%220%200%20100%20100%22><text%20y=%22.9em%22%20font-size=%2290%22>%F0%9F%8C%8E</text></svg>"> <link rel="stylesheet" href="/assets/css/main.css?d41d8cd98f00b204e9800998ecf8427e"> <link rel="canonical" href="https://ken77921.github.io/projects/"> <link rel="stylesheet" href="/assets/css/jekyll-pygments-themes-native.css?e74e74bf055e5729d44a7d031a5ca6a5" media="none" id="highlight_theme_dark">
1<script src="/assets/js/theme.js?6185d15ea1982787ad7f435576553d64"></script>
1 </head> <body class="fixed-top-nav "> <header> <nav id="navbar" class="navbar navbar-light navbar-expand-sm fixed-top"> <div class="container"> <a class="navbar-brand title font-weight-lighter" href="/">Haw-Shiuan Chang</a> <button class="navbar-toggler collapsed ml-auto" type="button" data-toggle="collapse" data-target="#navbarNav" aria-controls="navbarNav" aria-expanded="false" aria-label="Toggle navigation"> <span class="sr-only">Toggle navigation</span> <span class="icon-bar top-bar"></span> <span class="icon-bar middle-bar"></span> <span class="icon-bar bottom-bar"></span> </button> <div class="collapse navbar-collapse text-right" id="navbarNav"> <ul class="navbar-nav ml-auto flex-nowrap"> <li class="nav-item "> <a class="nav-link" href="/">About</a> </li> <li class="nav-item dropdown active"> <a class="nav-link dropdown-toggle" href="#" id="navbarDropdown" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Projects<span class="sr-only">(current)</span></a> <div class="dropdown-menu dropdown-menu-right" aria-labelledby="navbarDropdown"> <a class="dropdown-item " href="/projects/#header-entailment_hierarchy">Entailment Hierarchy</a> <a class="dropdown-item " href="/projects/#header-curve-extrapolation-LM">Extrapolating the Distributions of an Infinitely Large Language Model</a> <a class="dropdown-item " href="/projects/#header-multifacet-embedding-LM">Multi-facet Embeddings for Language Modeling</a> <a class="dropdown-item " href="/projects/#header-active-learning">Active Learning and Crowdsourcing</a> <a class="dropdown-item " href="/projects/#header-NLP">Natural Language Processing</a> <a class="dropdown-item " href="/projects/#header-CV">Computer Vision (Clustering and Matching)</a> </div> </li> <li class="nav-item "> <a class="nav-link" href="/publications/">Publications</a> </li> <li class="nav-item "> <a class="nav-link" href="/teaching/">Teaching</a> </li> <li class="nav-item "> <a class="nav-link" href="/software/">Software</a> </li> <li class="toggle-container"> <button id="light-toggle" title="Change theme"> <i class="fa-solid fa-moon"></i> <i class="fa-solid fa-sun"></i> </button> </li> </ul> </div> </div> </nav> <progress id="progress" value="0"> <div class="progress-container"> <span class="progress-bar"></span> </div> </progress> <div class="container mt-5"> <div class="post"> <header class="post-header"> <h1 class="post-title">Projects</h1> <p class="post-description"></p> </header> <article> <hr> <div class="div-cat-header"> <h2 id="header-entailment_hierarchy">Entailment Hierarchy</h2> </div> <p><img src="../assets/img/entailment_hierarchy.png" class="img-responsive" width="100%"></p> <p>What is a good way to organize knowledge or the text in a large corpus? In this project, we propose a data structure called entailment hierarchy, where each node entails their parents in a tree structure. We found that the structure has many applications. For example, it can provide more fine-grained description than taxonomy for organizing LLM prompts, synthesize more natural and challenging constraints than back-translation, and measure the specificity of LLM prompts better than the prompt length. </p> <p><br></p> <h4>Organizing LLM Prompts using Entailment Hierarchy</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/comprehend.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> Understanding what kind of prompts users typically ask and how well an LLM performs on these prompt categories is crucial for LLM practitioners. The existing methods satisfy the demand by hierarchically clustering the user prompts and synthesizing the cluster names using an LLM to build a taxonomy. However, the cluster names from these methods are usually too general for humans to understand the detailed prompt distributions and identify the exact strengths and weaknesses of LLMs. </p> <p> In this study, we propose COMPREHEND (<b>COM</b>mon <b>PR</b>ompt <b>E</b>ntailment <b>H</b>ierarchy for <b>E</b>valuation a<b>N</b>d <b>D</b>evelopment), which automatically constructs an ultra-fine-grained entailment hierarchy for the prompts. The entailment hierarchy replaces the cluster names in the taxonomy with sentences to provide more precise cluster descriptions for more fine-grained categorization. </p> <p> Compared with EvalTree, a state-of-the-art taxonomy construction method, COMPREHEND improves an entailment/faithfulness metric from 20.2 to 40.4 and an informative metric from 13.3 to 27.1 on average in WildBench. Our experiments also show that the weakness report generated based on our entailment hierarchy is much more informative than EvalTree in terms of predicting the response scores in the benchmark (<a href="">Paper</a>). </p> </div> </div> <h4>More Natural and Challenging Back-Translation for LLM Instruction Following</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/unspecific.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> Large language models (LLMs) are increasingly expected to follow long lists of constraints in complex instructions, and synthesizing instructions from a reference document (i.e., back-translation) is a widely used method to measure/enhance LLMsâ ability to follow complex instructions. However, this method introduces a critical loophole: the constraint synthesis model copies text from the reference as a very specific constraint and the evaluated LLM trivially satisfies the constraint by copying its text in the response. </p> <p> To address these issues, we propose UNSPECIFIC (<b>UN</b>covering <b>S</b>ummary-resistant, <b>P</b>aste-reducing, and <b>E</b>valuation-revised <b>C</b>onstraints to <b>I</b>mprove <b>F</b>ollowing <b>I</b>
1nstruction <b>C</b>hallenge), a novel framework that synthesizes constraints common to two similar reference articles to reduce copy-pasting, selectively hardens only trivially satisfied constraints to balance difficulty and naturalness, and evaluates satisfaction on both the generated article and its summary to penalize superficial instruction following. </p> <p> Consequently, we built the UNSPECIFIC benchmark on news, story, and blog domains to analyze the copy-pasting behavior of LLMs. Our results show that our synthesized constraints are not only more challenging (e.g., the satisfaction rate of GPT-5 Mini drops from 90% to 78%) and natural (LLM win-rate gap improves by 30%) from a human perspective but also mitigate the copy-pasting. We also find that a large portion of constraints are satisfied superficially (i.e., not satisfied in the core narrative of the article) (<a href="https://openreview.net/pdf/fcf46f966b7374a7fae666dc69c55c9b92e72889.pdf" rel="external nofollow noopener" target="_blank">Paper</a>). </p> </div> </div> <h4>Inserting the Prompts into an Entailment Hierarchy</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/prompt2box.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> To discover the weaknesses of LLMs, researchers often embed prompts into a vector space and cluster them to extract insightful patterns. However, vector embeddings primarily capture topical similarity. As a result, prompts that share a topic but differ in specificity, and consequently in difficulty, are often represented similarly, making fine-grained weakness analysis difficult. </p> <p> To address this limitation, we propose Prompt2Box:, which embeds prompts into a box embedding space using a trained encoder. The encoder, trained on existing and synthesized datasets, outputs box embeddings that capture not only semantic similarity but also specificity relations between prompts (e.g., "writing an adventure story" is more specific than "writing a story". We further develop a novel dimension reduction technique for box embeddings to facilitate dataset visualization and comparison. specificity. </p> <p> Our experiments demonstrate that box embeddings consistently capture prompt specificity better than vector baselines and achieve up to 45% error reduction compared to the prompt length baseline. On the downstream task of creating hierarchical clustering trees for 17 LLMs from the UltraFeedback dataset, Prompt2Box: can identify 13.5% more LLM weaknesses than vector baselines and achieves an approximately 33% stronger correlation between hierarchical depth and instruction (<a href="https://openreview.net/pdf/fcf46f966b7374a7fae666dc69c55c9b92e72889.pdf" rel="external nofollow noopener" target="_blank">Paper</a>). </p> </div> </div> <h4>Automatically Measuring the Creativity of Large Language Models</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/CS4_horizontal.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> Evaluating the creativity of large language models (LLMs) in story writing is difficult because LLM-generated stories could seemingly look creative but be very similar to some existing stories in their huge and proprietary training corpus. To overcome this challenge, we introduce a novel benchmark dataset, CS4, with varying levels of prompt specificity. By increasing the number of requirements/constraints in the prompt, we can increase the prompt specificity and hinder LLMs from retelling high-quality narratives in their training data. Consequently, CS4 empowers us to indirectly measure the LLMs' creativity without human annotations. </p> <p> Our experiments on LLaMA, Gemma, and Mistral not only highlight the creativity challenges LLMs face when dealing with highly specific prompts but also reveal that different LLMs perform very differently under different numbers of constraints and achieve different balances between the model's instruction-following ability and narrative coherence. Additionally, our experiments on OLMo suggest that Learning from Human Feedback (LHF) can help LLMs select better stories from their training data but has limited influence in boosting LLMs' ability to produce creative stories that are unseen in the training corpora. (<a href="https://arxiv.org/abs/2410.04197" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="../assets/pdf/WNU_CS4_poster.pdf">Poster</a>). </p> </div> </div> <h4>Coarse-to-Fine Story Generation by Constructing Entailment Hierarchy</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/EH.png" class="img-responsive" width="100%"> </div> <div class="col"> When users want to write a story with a language model (LM) assistant such as ChatGPT, it is often very difficult to provide a prompt that clearly specifies all their interests. For the providers of LM assistants, it is also difficult to ensure their output stories come from a dataset without copyright concerns. Motivated by these limitations, we propose a coarse-to-fine (C2F) tree-based story generation framework, which is called C2F-StoryTree, where the LM iteratively generates more and more specific story prompts based on a userâs input prompt and the desired plot selected by the user. To realize our C2F-StoryTree framework, we propose an entailment hierarchy (EH) text structure, in which a more specific response entails more general prompt (e.g., a story entails a summary). We also propose novel annotation tasks, decoding methods, and a human-and-machine-in-the-loop procedure to minimize the annotation cost of building the text structure. We build an entailment hierarchy dataset on top of the story datasets with desired licenses and styles, on which the service providers can fine-tune or evaluate their LMs (<a href="https://www.amazon.science/publications/fine-to-coarse-entailment-hierarchy-construction-for-coarse-to-fine-story-generation" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="../assets/pdf/HCI+NLP_entailment_hierarchy_talk.key">Slides</a>, <a href="../assets/pdf/NAACL_HCI_NLP_24_poster.pdf">Poster</a>). </div> </div> <p><br></p> <div class="div-cat-header"> <h2 id="header-curve-extrapolation-LM">Extrapolating the Distributions of an Infinitely-Large Language Model</h2> </div> <p><br></p> <h4>
1Why Does Contrastive Decoding Work Well and How we could make it Better?</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/APD.png" class="img-responsive" width="100%"> </div> <div class="col"> To deepen our understanding of Contrastive Decoding (CD), we first theoretically prove that CD could be viewed as linearly extrapolating the next-token logits from a huge and hypothetical LM. We also highlight that the linear extrapolation could make CD unable to output the most obvious answers that have already been assigned high probabilities by the amateur LM. To overcome CDâs limitation, we propose a new unsupervised decoding method called <b>A</b>symptotic <b>P</b>robability <b>D</b>ecoding (APD). APD explicitly extrapolates the probability curves from the LMs of different sizes to infer the asymptotic probabilities (i.e., probabilities of an LLM with an infinite size) without inducing more inference costs than CD. (<a href="https://www.amazon.science/publications/explaining-and-improving-contrastive-decoding-by-extrapolating-the-probabilities-of-a-huge-and-hypothetical-lm" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://go.screenpal.com/watch/cZ6tFEnc2Ta" rel="external nofollow noopener" target="_blank">Talk</a>, <a href="../assets/pdf/EMNLP_2024_APD_poster.pdf">Poster</a>, <a href="../assets/pdf/EMNLP_2024_APD_release.key">Slides</a>, <a href="https://github.com/amazon-science/llm-asymptotic-decoding/" rel="external nofollow noopener" target="_blank">Code</a>) </div> </div> <p><br></p> <h4>Be Careful when LLM is more Uncertain than it Should Be.</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/REAL_sampling.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We propose REAL (<b>R</b>esidual <b>E</b>ntropy from <b>A</b>symptotic <b>L</b>ine) sampling, a decoding method that achieves improved factuality and diversity over nucleus sampling by predicting an adaptive threshold of p. Specifically, REAL sampling predicts the step-wise likelihood of an LLM to hallucinate, and lowers the p threshold when an LLM is likely to hallucinate. Otherwise, REAL sampling increases the p threshold to boost the diversity. </p> <p> To predict the step-wise hallucination likelihood without supervision, we construct a Token-level Hallucination Forecasting (THF) model, which predicts the asymptotic entropy (i.e., inherent uncertainty) of the next token by extrapolating the next-token entropies of an infinitely large language model from a series of LLMs with different sizes. If a LLM's entropy
1is higher than the asymptotic entropy (i.e., the LLM is more uncertain than it should be), the THF model predicts a high hallucination hazard, which leads to a lower p threshold in REAL sampling. </p> <p> After combined with contrastive decoding, REAL sampling outperforms 9 sampling methods, and generates texts that are more factual than the greedy sampling and more diverse than the nucleus sampling with p=0.5 in the FactualityPrompts benchmark. (<a href="https://arxiv.org/abs/2406.07735" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://github.com/amazon-science/llm-asymptotic-decoding/" rel="external nofollow noopener" target="_blank">Code</a>) </p> </div> </div> <p><br></p> <hr> <div class="div-cat-header"> <h2 id="header-multifacet-embedding-LM">Multi-facet Embeddings for Language Modeling</h2> </div> <p><img src="../assets/img/contrastive_learning.png" class="img-responsive" width="100%"></p> <p>In this project, we study the theoretical limitations of the single context embedding in LMs and how the theoretical analyses suggest new alternative softmax layers that encode a context as multiple embeddings. The proposed alternatives achieve better perplexity than the mixture of softmax (MoS), especially given an ambiguous context, without adding significant computational cost to LMs. Our approaches also let GPT-2 learn to properly copy the entities from the context, which increases the coherence of the generated text without requiring any labels. In addition to predicting the next word, we also use multiple CLS embeddings to improve state-of-the-art pretraining methods for BERT on natural language understanding (NLU) benchmarks without introducing significant extra parameters or computations, especially when the training datasets are small. Furthermore, we show that our multi-facet embeddings improve the sequential recommendation, scientific paper embeddings, distantly supervised relation extraction, and cold-start citation recommendation. Finally, we use the multiple vector embeddings to predict the future topics of a context, and build on the basis, we propose a novel interactive language generation framework. (<a href="https://web.archive.org/web/20221031171926id_/https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=3783&context=dissertations_2" rel="external nofollow noopener" target="_blank">PhD Thesis</a>, <a href="../assets/pdf/Haw-Shiuan_thesis_defense_slides.key">Slides</a>) </p> <p><br></p> <h4>Simply Replacing the Ouput Softmax Layer Improves Neural Sequential Recommenders by around 20% in 12 Widely-Used Datasets!</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/softmax_limits.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> The similarity structure of the global item embeddings in the softmax layer sometimes forces the single hidden state embedding to be close to new items when copying is a better choice, while sometimes forcing the hidden state to be close to the items from the input inappropriately. To alleviate the problem, we adapt the recently-proposed softmax alternatives such as <a href="https://arxiv.org/abs/2305.12289" rel="external nofollow noopener" target="_blank">softmax-CPR</a> to sequential recommendation tasks and demonstrate that the new softmax architectures unleash the capability of the neural encoder on learning when to copy and when to exclude the items from the input sequence. By only making some simple modifications on the output softmax layer for SASRec and GRU4Rec, softmax-CPR achieves consistent improvement in 12 datasets. With almost the same model size, our best method not only improves the average NDCG@10 of GRU4Rec in 5 datasets with duplicated items by 10% (4%-17% individually) but also improves 7 datasets without duplicated items by 24% (8%-39%)! (<a href="https://arxiv.org/abs/2310.14079" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://github.com/iesl/softmax_CPR_recommend" rel="external nofollow noopener" target="_blank">Code</a>) </p> </div> </div> <h4>Why and When Pointer Networks Improve LMs and How to Do Even Better (aka It does not Make Sense to use Attention in All Transformer Layers Except the Last Softmax Layer!)</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/dynamic_partitions.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> The softmax bottleneck (<a href="https://aclanthology.org/2022.acl-long.554.pdf" rel="external nofollow noopener" target="_blank">
1Chang and McCallum (2022)</a>) sometimes prevents the language models from predicting the desired distribution and the pointer networks can be used to break the bottleneck efficiently. Based on the finding, we propose the context/encoder partition by simplifying the pointer networks and the reranker partition to accelerate the word-by-word rerankers. By combining these softmax alternatives, softmax-CPR is significantly better and more efficient than mixture of softmax (MoS) in GPT-2, a state-of-the-art softmax alternative. In summarization experiments, without significantly decreasing its training/testing speed, softmax-CEPR based on T5-Small improves factCC score by 2 points in CNN/DM and XSUM dataset, and improves MAUVE scores by around 30% in BookSum paragraph-level dataset. (<a href="https://arxiv.org/abs/2305.12289" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://github.com/iesl/Softmax-CPR" rel="external nofollow noopener" target="_blank">Code</a>) </p> </div> </div> <h4>Ensembling BERT almost without Additional Cost!</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/multi-BERT_first_page.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We propose Multi-CLS BERT, a novel ensembling method for CLS-based prediction tasks that is almost as efficient as a single BERT model. Multi-CLS BERT uses multiple CLS tokens with a parameterization and objective that encourages their diversity. Thus instead of fine-tuning each BERT model in an ensemble (and running them all at test time), we need only fine-tune our single Multi-CLS BERT model (and run the one model at test time, ensembling just the multiple final CLS embeddings). To test its effectiveness, we build Multi-CLS BERT on top of a state-of-the-art pretraining method for BERT (Aroca-Ouellette and Rudzicz, 2020). In experiments on GLUE and
1SuperGLUE we show that our Multi-CLS BERT reliably improves both overall accuracy and confidence estimation. When only 100 training samples are available in GLUE, the Multi-CLS BERT_Base model can even outperform the corresponding BERT_Large model. We analyze the behavior of our Multi-CLS BERT, showing that it has many of the same characteristics and behavior as a typical BERT 5-way ensemble, but with nearly 4-times less computation and memory. (<a href="https://arxiv.org/abs/2210.05043" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://github.com/iesl/multicls/" rel="external nofollow noopener" target="_blank">Code</a>) We also show that Multi-CLS BERT significantly improves multi-domain scientific paper encoders. (<a href="https://arxiv.org/abs/2309.04333" rel="external nofollow noopener" target="_blank">Paper</a>) </p> </div> </div> <h4>Why Multiple Embeddings are Better in LM's Output Softmax Layer</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/Multi-facet_Softmax.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We theoretically show that this single hidden state cannot produce all probability distributions regardless of the language model (LM) size or training data size because the single hidden state embedding cannot be close to the embeddings of all the possible next words simultaneously when there are other interfering word embeddings between them. Our work not only deepens our understanding of softmax bottleneck and mixture of softmax (MoS) but also inspires us to propose multi-facet softmax (MFS) to address the limitations of MoS (<a href="https://aclanthology.org/2022.acl-long.554.pdf" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://aclanthology.org/attachments/2022.acl-long.554.software.zip" rel="external nofollow noopener" target="_blank">Code</a>, <a href="https://screencast-o-matic.com/watch/c3fresVYo4M" rel="external nofollow noopener" target="_blank">Talk</a>, <a href="../assets/pdf/ACL2022_slides.pdf">Slides</a>, <a href="../assets/pdf/ACL2022_poster.pdf">Poster</a>). </p> </div> </div> <h4>Predicting the Future Topics for Interactive Language Generation</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/proj1-nlp-interactive_LM_first_figure.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We design a framework that displays multiple candidate upcoming topics, of which a user can select a subset to guide the generation. Our framework consists of two components: (1) a method that produces a set of candidate topics by predicting the centers of word clusters in the possible continuations, and (2) a text generation model whose output adheres to the chosen topics. The training of both components is self-supervised, using only unlabeled text. Our experiments demonstrate that our topic options are better than those of standard clustering approaches, and our framework often generates fluent sentences related to the chosen topics, as judged by automated metrics and crowdsourced workers (<a href="http://arxiv.org/abs/2103.15335" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://github.com/iesl/interactive_LM" rel="external nofollow noopener" target="_blank">Code</a>, <a href="https://slideslive.com/38954487/changing-the-mind-of-transformers-for-topicallycontrollable-language-generation" rel="external nofollow noopener" target="_blank">Talk</a>, <a href="../assets/pdf/proj1-nlp-interactive_LM-EACL_interactive_LM-slides.key" download="slides-eacl2021-Changing the Mind of Transformers for Topically-Controllable Language Generation">Slides</a>, <a href="../assets/pdf/proj1-nlp-interactive_LM-EACL_interactive_LM-poster.pdf" download="poster-eacl2021-Changing the Mind of Transformers for Topically-Controllable Language Generation">Poster</a>). </p> </div> </div> <h4>Multi-facet Embeddings for Distantly Supervised Relation Extraction</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/proj2-nlp-Multi-facet%20Embeddings%20for%20Relation%20Extraction.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We propose multi-facet universal schema that uses a neural model to represent each sentence pattern as multiple facet embeddings and encourage one of these facet embeddings to be close to that of another sentence pattern if they co
1occur with the same entity pair. In our experiments, we demonstrate that multi-facet embeddings significantly outperform their single facet embedding counterpart, compositional universal schema (Verga et al., 2016), in distantly supervised relation extraction tasks. Moreover, we can also use multiple embeddings to detect the entailment relation between two sentence patterns when no manual label is available (<a href="http://arxiv.org/abs/2103.15339" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://github.com/rohanpaul11/multifacet-re" rel="external nofollow noopener" target="_blank">Code</a>, <a href="https://slideslive.com/38954382/multifacet-universal-schema" rel="external nofollow noopener" target="_blank">Talk</a>, <a href="../assets/pdf/proj2-nlp-Multi-facet%20RE-EACL_multi-facet_RE_small-slides.key" download="slides-eacl2021-Multi-facet Universal Schema">Slides</a>, <a href="../assets/pdf/proj2-nlp-Multi-facet%20RE-EACL_multi-facet_RE_small-poster.pdf" download="poster-eacl2021-Multi-facet Universal Schema">Poster</a>). </p> </div> </div> <h4>Predicting Cluster Centers for Sentence Representation</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/proj3-nlp-Multi-facet%20Embeddings%20for%20Sentence%20Representation.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We propose a novel embedding method for a text sequence (e.g., a sentence) where each sequence is represented by a distinct set of multi-mode codebook embeddings to capture different semantic facets of its meaning. The codebook embeddings can be viewed as the cluster centers which summarize the distribution of possibly co-occurring words in a pre-trained word embedding space. Our experiments show that the per-sentence codebook embeddings significantly improve the performances in unsupervised sentence similarity and extractive summarization benchmarks (<a href="http://arxiv.org/abs/2103.15330" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://docs.google.com/presentation/d/1k-OBWdBYsGmXUuvNc1_J_JrEppB_Aqh82bJPCgjFL-s/edit?usp=sharing" rel="external nofollow noopener" target="_blank">Slides</a>, <a href="../assets/pdf/proj3-nlp-Multi-facet%20Embeddings%20for%20Sentence%20Representation-poster.pdf" download="poster-aaai2021-Extending Multi-sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applications">Poster</a>). </p> </div> </div> <p><br></p> <hr> <div class="div-cat-header"> <h2 id="header-active-learning">Active Learning and Crowdsourcing</h2> </div> <p><br></p> <h4>Overcoming Practical Issues of Deep Active Learning</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/proj4-nlp-Overcoming%20Practical%20Issues%20
1of%20Deep%20Active%20Learning.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> Existing deep active learning algorithms achieve impressive sampling efficiency on natural language processing tasks. However, they exhibit several weaknesses in practice, including (a) inability to use uncertainty sampling with black-box models, (b) lack of robustness to noise in labeling, (c) lack of transparency. In response, we propose a transparent batch active sampling framework by estimating the error decay curves of multiple feature-defined subsets of the data. We perform extensive experiments on four named entity recognition (NER) tasks and results show that our methods greatly alleviate these limitations without sacrificing too much sampling efficiency (<a href="https://arxiv.org/abs/1911.07335" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://slideslive.com/38933012/using-error-decay-prediction-to-overcome-practical-issues-of-deep-active-learning-for-named-entity-recognition" rel="external nofollow noopener" target="_blank">Slides</a>, <a href="https://slideslive.com/38933012/using-error-decay-prediction-to-overcome-practical-issues-of-deep-active-learning-for-named-entity-recognition" rel="external nofollow noopener" target="_blank">Talk</a>). </p> </div> </div> <h4>Active Sampling for Estimating Quality of Experience (QoE) Model</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/active_QoE.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We use Bayesian learning to model the non-linear relationships between quality of experience (QoE) and multiple factors. </p> <p> Our experiment shows that active sampling can be used to reduce the number of samples collected from crowdsourcing for building the model, but the users' perception of the video quality would be affected by the active learning methods (<a href="../assets/pdf/proj9-cv-Active%20Sampling%20for%20estimating%20QoE%20model-paper.pdf" download="paper-ieee trans. multimed. 2018-Active learning for crowdsourced QoE modeling">Paper</a>). </p> </div> </div> <h4>Student Modeling and Prerequisite Verification in Knowledge Tree</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/proj8-education-student%20modeling.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We extract answering logs of the exercises from <a href="https://www.junyiacademy.org/" rel="external nofollow noopener" target="_blank">Junyi Academy</a>, an E-learning website similar to Khan Academy. </p> <p> We use crowdsourcing and machine learning to discover prerequisite relationships between exercises. Based on that, we design a mechanism of adaptive test to improve the learning experiences of Junyi academy (<a href="../assets/pdf/proj8-education-student%20modeling-paper.pdf" download="paper-edm2015-Modeling Exercise Relationships in E-Learning: A Unified Approach">Paper</a>, <a href="../assets/pdf/proj8-education-student%20modeling-presentation.pptx" download="slides-edm2015-Modeling Exercise Relationships in E-Learning: A Unified Approach">Presentation</a>, <a href="https://drive.google.com/file/d/1V5nEwmVkPclj_ZR-bKt9xzW7ApT6ZJo-/view?usp=sharing" rel="external nofollow noopener" target="_blank">Demo</a>, <a href="https://ken77921.github.io/assets/pdf/junyi_exer_relationship_released.zip">Code</a>, <a href="https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=1198" rel="external nofollow noopener" target="_blank">Dataset</a>). </p> </div> </div> <h4>Emphasize Uncertain Examples in Supervised Learning</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/proj7-nn.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> Inspired by active learning, we propose two alternatives to re-weight training samples based on lightweight estimates of sample uncertainty in stochastic gradient descent (SGD). Experimental results on six datasets show that our methods reliably improve accuracy in various network architectures, including additional gains on top of other popular training techniques (<a href="https://arxiv.org/abs/1704.07433" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://arxiv.org/abs/1704.07433" rel="external nofollow noopener" target="_blank">Poster</a>). </p> </div> </div> <p><br></p> <hr> <div class="div-cat-header"> <h2 id="header-NLP">Natural Language Processing</h2> </div> <p><br></p> <h4>Distributional Inclusion Vector Embedding for Unsupevised Hypernym Detection</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/proj5-nlp-DIVE.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We propose a novel word embedding method that preserves the distributional inclusion property in the sparse-bag-of-word (SBOW) feature. The embedding can be used to predict the generality of words, detect the hypernym relation, and discover the topics from the raw text simultaneously. The extensive experiments show that the embedding effectively compresses the SBOW, and achieves new state-of-the-art performances on the unsupervised hypernym detection tasks (<a href="https://arxiv.org/abs/1710.00880" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="https://github.com/iesl/Distributional-Inclusion-Vector-Embedding" rel="external nofollow noopener" target="_blank">Code</a>, <a href="../assets/pdf/proj5-nlp-DIVE-poster.pdf" download="poster-naacl2018-Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection">Poster</a>). We also show that DIVE could help us to do word sense induction more efficiently (<a href="https://arxiv.org/abs/1804.03257" rel="external nofollow noopener" target="_blank">Paper</a>, <a href="../assets/pdf/proj5-nlp-DIVE-slides.pdf" download="slides-TextGraphs2018-Efficient Graph-based Word Sense Induction by Distributional Inclusion Vector Embeddings">Slides</a>). </p> </div> </div> <h4>UMASS TAC-KBP 2016 System for Relation Extraction</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/proj6-nlp-UMASS%20TAC%202016.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> TAC-KBP is one of the most challenging text-based information retrieval tasks. We integrate research that is done in UMASS IESL in the past year, including embedding linker, multilingual Universal Schema, and LSTM sentence embedding. We perform extensive error analysis and develop some novel techniques (such as using a search engine to reduce noise in training data) to tackle the problems (<a href="../assets/pdf/proj6-nlp-UMASS%20TAC%202016-paper.pdf" download="paper-tac2016-Extracting Multilingual Relations under Limited Resources">Paper</a>). </p> </div> </div> <p><br></p> <hr> <div class="div-cat-header"> <h2 id="header-CV">Computer Vision (Unsupervised Clustering and Matching)</h2> </div> <p><br></p> <h4>Decomposition of Multiple Foreground Co-segmentation</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/MFC.png" class="img-responsive" width="100%"> </div> <div class="col"> <p>
1 We proposed an efficient algorithm that decomposes the unsupervised Multiple Foreground Co-segmentation problem into three sub-problems: segmentation, matching, and figure-ground classification. </p> <p> Our method improves the accuracy of the state-of-the-art method by 13% in a standard benchmark (<a href="../assets/pdf/proj11-cv-Decomposition%20of%20Multiple%20Foreground%20Co-segmentation-paper.pdf" download="paper-cviu2015-Optimizingthedecompositionformultipleforegroundcosegmentation">Paper</a>). </p> </div> </div> <h4>Hierarchical Image Segmentation without Training</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/proj10-cv-Hierarchical%20Image%20Segmentation%20without%20Training.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We proposed a general framework that applies classifiers with different complexity to discriminate segments in an image. </p> <p> Our unsupervised hierarchical segmentation results achieve similar or better performance in several standard benchmarks compared with the current state-of-the-art methods based on supervised learning (<a href="../assets/pdf/proj10-cv-Hierarchical%20Image%20Segmentation%20without%20Training-paper.pdf" download="paper-accv2014-Simple-to-Complex Discriminative Clustering for Hierarchical Image Segmentation">Paper</a>, <a href="../assets/pdf/proj10-cv-Hierarchical%20Image%20Segmentation%20without%20Training-poster.pdf" download="poster-accv2014-Simple-to-Complex Discriminative Clustering for Hierarchical Image Segmentation">Poster</a>). </p> </div> </div> <h4>Superpixel-Based Large Displacement Optical Flow</h4> <div class="row row-grid"> <div class="col-6"> <img src="../assets/img/proj12-cv-Superpixel-Based%20Large%20Displacement%20Optical%20Flow.png" class="img-responsive" width="100%"> </div> <div class="col"> <p> We formulated our objective function at the superpixel level rather than the pixel level as the traditional optical flow method did.</p> <p> Our method achieves a better large displacement matching capability than LDOF in videos with lower quality (<a href="../assets/pdf/proj12-cv-Superpixel-Based%20Large%20Displacement%20Optical%20Flow-paper.pdf" download="paper-icip2013-SUPERPIXEL-BASED LARGE DISPLACEMENT OPTICAL FLOW">Paper</a>, <a href="../assets/pdf/proj12-cv-Superpixel-Based%20Large%20Displacement%20Optical%20Flow-poster.pdf" download="poster-icip2013-SUPERPIXEL-BASED LARGE DISPLACEMENT OPTICAL FLOW">Poster</a>). </p> </div> </div> <div class="projects"> </div> </article> </div> </div> <footer class="fixed-bottom"> <div class="container mt-0"> © Copyright 2026 Haw-Shiuan Chang. Powered by <a href="https://jekyllrb.com/" target="_blank" rel="external nofollow noopener">Jekyll</a> with <a href="https://github.com/alshedivat/al-folio" rel="external nofollow noopener" target="_blank">al-folio</a> theme. Hosted by <a href="https://pages.github.com/" target="_blank" rel="external nofollow noopener">GitHub Pages</a>. </div> </footer>
1<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/jquery.min.js" integrity="sha256-/xUj+3OJU5yExlq6GSYGSHk7tPXikynS7ogEvDej/m4=" crossorigin="anonymous"></script>
1
1<script src="/assets/js/bootstrap.bundle.min.js"></script>
1
1<script src="https://cdn.jsdelivr.net/npm/[email protected]/js/mdb.min.js" integrity="sha256-NdbiivsvWt7VYCt6hYNT3h/th9vSTL4EDWeGs5SN3DA=" crossorigin="anonymous"></script>
1
1<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/masonry.pkgd.min.js" integrity="sha256-Nn1q/fx0H7SNLZMQ5Hw5JLaTRZp0yILA/FRexe19VdI=" crossorigin="anonymous"></script>
1
1<script defer src="https://cdn.jsdelivr.net/npm/imagesloaded@4/imagesloaded.pkgd.min.js"></script>
1
1<script defer src="/assets/js/masonry.js" type="text/javascript"></script>
1
1<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/medium-zoom.min.js" integrity="sha256-7PhEpEWEW0XXQ0k6kQrPKwuoIomz8R8IYyuU1Qew4P8=" crossorigin="anonymous"></script>
1
1<script defer src="/assets/js/zoom.js?7b30caa5023af4af8408a472dc4e1ebb"></script>
1
1<script defer src="https://unpkg.com/[email protected]/dist/bootstrap-table.min.js"></script>
1
1<script src="/assets/js/no_defer.js?d633890033921b33e0ceb13d22340a9c"></script>
1
1<script defer src="/assets/js/common.js?acdb9690d7641b2f8d40529018c71a01"></script>
1
1<script defer src="/assets/js/copy_code.js?9b43d6e67ddc7c0855b1478ee4c48c2d" type="text/javascript"></script>
1
1<script async src="https://d1bxh8uas1mnw7.cloudfront.net/assets/embed.js"></script>
1
1<script async src="https://badge.dimensions.ai/badge.js"></script>
1
1<script type="text/javascript">window.MathJax={tex:{tags:"ams"}};</script>
1
1<script defer type="text/javascript" id="MathJax-script" src="https://cdn.jsdelivr.net/npm/[email protected]/es5/tex-mml-chtml.js"></script>
1
1<script type="text/javascript">function progressBarSetup(){"max"in document.createElement("progress")?(initializeProgressElement(),$(document).on("scroll",function(){progressBar.attr({value:getCurrentScrollPosition()})}),$(window).on("resize",initializeProgressElement)):(resizeProgressBar(),$(document).on("scroll",resizeProgressBar),$(window).on("resize",resizeProgressBar))}function getCurrentScrollPosition(){return $(window).scrollTop()}function initializeProgressElement(){let e=$("#navbar").outerHeight(!0);$("body").css({"padding-top":e}),$("progress-container").css({"padding-top":e}),progressBar.css({top:e}),progressBar.attr({max:getDistanceToScroll(),value:getCurrentScrollPosition()})}function getDistanceToScroll(){return $(document).height()-$(window).height()}function resizeProgressBar(){progressBar.css({width:getWidthPercentage()+"%"})}function getWidthPercentage(){return getCurrentScrollPosition()/getDistanceToScroll()*100}const progressBar=$("#progress");window.onload=function(){setTimeout(progressBarSetup,50)};</script>
1 </header> </body> </html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.