1<!DOCTYPE html> 2<html> 3 4 <head> 5 <meta charset="utf-8"> 6 <meta http-equiv="X-UA-Compatible" content="IE=edge"> 7 <meta name="viewport" content="width=device-width, initial-scale=1"> 8 9 <title>ê²ìì ìí ë°ì´í° ë¤ë£¨ê¸° - ì°ìííì ë¤ ê¸°ì ë¸ë¡ê·¸</title> 10 <meta name="description" content=" ìë íì¸ì. ì°ìííì ë¤ ê²ìê°ë°í ì ì² ì ëë¤."> 11 <meta http-equiv="refresh" content="0;url=https://techblog.woowahan.com/change/?key=2021-03-03-search-system.md" /> 12 <link rel="canonical" href="https://techblog.woowahan.com/change/?key=2021-03-03-search-system.md" /> 13 14 <meta property="fb:pages" content="1012405502198937" /> 15 <meta property="fb:app_id" content="596588250503455"/> 16 <meta property="og:title" content="ê²ìì ìí ë°ì´í° ë¤ë£¨ê¸° - ì°ìííì ë¤ ê¸°ì ë¸ë¡ê·¸"/> 17 <meta property="og:site_name" content="ì°ìííì ë¤ ê¸°ì ë¸ë¡ê·¸"/> 18 <meta property="og:type" content="website"/> 19 <meta property="og:url" content="https://woowabros.github.io/woowabros/2021/03/02/search-system.html"/> 20 <meta property="og:image" content=""/> 21 <meta property="og:description" content=" ìë íì¸ì. ì°ìííì ë¤ ê²ìê°ë°í ì ì² ì ëë¤."/> 22 <meta name="naver-site-verification" content="9b6a9f890dc78f921d5c4b87fdb6ceaaf5549a17" /> 23 24 <link rel="stylesheet" href="/css/main.css"> 25 <link rel="canonical" href="https://woowabros.github.io/woowabros/2021/03/02/search-system.html"> 26 <link rel="alternate" type="application/rss+xml" title="ì°ìííì ë¤ ê¸°ì ë¸ë¡ê·¸" href="https://woowabros.github.io/feed.xml"> 27 <link rel="shortcut icon" href="https://www.woowahan.com/favicon/favicon.ico"> 28
28<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
28 29
29<script type="text/javascript" src="//cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
29 30</head> 31 32 33 <body> 34 35 36<div id="fb-root"></div>
37<script>(function(d, s, id) { 38 var js, fjs = d.getElementsByTagName(s)[0]; 39 if (d.getElementById(id)) return; 40 js = d.createElement(s); js.id = id; 41 js.src = "//connect.facebook.net/ko_KR/sdk.js#xfbml=1&version=v2.6&appId=596588250503455"; 42 fjs.parentNode.insertBefore(js, fjs); 43}(document, 'script', 'facebook-jssdk'));</script>
43 44 45 46 <header class="site-header"> 47 48 <div class="wrapper"> 49 50 <a class="site-title" href="/"> 51 <span class="site-logo">ì°ìííì ë¤</span> 52 <span class="site-title">기ì ë¸ë¡ê·¸</span> 53 </a> 54 55 <nav class="site-nav"> 56 <a href="#" class="menu-icon"> 57 <svg viewBox="0 0 18 15"> 58 <path fill="#424242" d="M18,1.484c0,0.82-0.665,1.484-1.484,1.484H1.484C0.665,2.969,0,2.304,0,1.484l0,0C0,0.665,0.665,0,1.484,0 h15.031C17.335,0,18,0.665,18,1.484L18,1.484z"/> 59 <path fill="#424242" d="M18,7.516C18,8.335,17.335,9,16.516,9H1.484C0.665,9,0,8.335,0,7.516l0,0c0-0.82,0.665-1.484,1.484-1.484 h15.031C17.335,6.031,18,6.696,18,7.516L18,7.516z"/> 60 <path fill="#424242" d="M18,13.516C18,14.335,17.335,15,16.516,15H1.484C0.665,15,0,14.335,0,13.516l0,0 c0-0.82,0.665-1.484,1.484-1.484h15.031C17.335,12.031,18,12.696,18,13.516L18,13.516z"/> 61 </svg> 62 </a> 63 64 <div class="trigger"> 65 66 67 68 69 70 <li> 71 <a class="page-link" href="/category/interview"> 72 interview (3) 73 </a> 74 </li> 75 76 77 78 79 80 <li> 81 <a class="page-link" href="/category/tools"> 82 tools (23) 83 </a> 84 </li> 85 86 87 88 89 90 <li> 91 <a class="page-link" href="/category/nerd"> 92 nerd (1) 93 </a> 94 </li> 95 96 97 98 99 100 <li> 101 <a class="page-link" href="/category/woowabros"> 102 woowabros (38) 103 </a> 104 </li> 105 106 107 108 109 110 <li> 111 <a class="page-link" href="/category/study"> 112 study (15) 113 </a> 114 </li> 115 116 117 118 119 120 <li> 121 <a class="page-link" href="/category/experience"> 122 experience (108) 123 </a> 124 </li> 125 126 127 128 129 130 131 132 133 <li> 134 <a class="page-link" href="/category/security"> 135 security (6) 136 </a> 137 </li> 138 139 140 141 142 143 <li> 144 <a class="page-link" href="/category/swift"> 145 swift (4) 146 </a> 147 </li> 148 149 150 151 152 153 <li> 154 <a class="page-link" href="/category/rnd"> 155 rnd (2) 156 </a> 157 </li> 158 159 160 161 162 163 <li> 164 <a class="page-link" href="/category/techcourse"> 165 techcourse (18) 166 </a> 167 </li> 168 169 170 171 172 173 <li> 174 <a class="page-link" href="/category/devrel"> 175 devrel (9) 176 </a> 177 </li> 178 179 180 181 182 183 <li> 184 <a class="page-link" href="/category/sre"> 185 sre (2) 186 </a> 187 </li> 188 189 190 191 192 193 <li> 194 <a class="page-link" href="/category/growth"> 195 growth (1) 196 </a> 197 </li> 198 199 200 201 202 203 <li> 204 <a class="page-link" href="/category/culture"> 205 culture (1) 206 </a> 207 </li> 208 209 <a class="page-link" href="http://woowabros.github.com">Github</a> 210 </div> 211 </nav> 212 213 </div> 214 215</header> 216 217 218 <div class="page-content"> 219 <div class="wrapper"> 220 <section class="content-layout"> 221 <article class="post" itemscope itemtype="https://schema.org/BlogPosting"> 222 <meta itemscope itemprop="mainEntityOfPage" itemType="https://schema.org/WebPage" itemid="https://woowabros.github.io"/> 223 224 <div itemprop="image" itemscope itemtype="https://schema.org/ImageObject"> 225 <meta itemprop="url" content=""> 226 <meta itemprop="width" content="800"> 227 <meta itemprop="height" content="800"> 228 </div> 229 230 <div itemprop="publisher" itemscope itemtype="https://schema.org/Organization"> 231 <meta itemprop="name" content="ì°ìííì ë¤"> 232 <div itemprop="logo" itemscope itemtype="https://schema.org/ImageObject"> 233 <meta itemprop="url" content="https://woowabros.github.io/logo_220x57.png"> 234 <meta itemprop="width" content="220"> 235 <meta itemprop="height" content="57"> 236 </div> 237 </div> 238 239 <header class="post-header"> 240 <h1 class="post-title" itemprop="name headline">ê²ìì ìí ë°ì´í° ë¤ë£¨ê¸°</h1> 241 242 <p class="post-meta"><time datetime="2021-03-02T09:30:00+09:00" itemprop="datePublished dateModified">Mar 2, 2021</time> ⢠<span itemprop="author" itemscope itemtype="https://schema.org/Person"><span itemprop="name">ì ì² </span></span><img src="https://www.gravatar.com/avatar/400ebaf765e049eb24b119f389b85c68?s=50" class="gravatar"></p> 243 </header> 244 245 <div class="post-content" itemprop="articleBody"> 246 <blockquote> 247 <p>ìë íì¸ì. ì°ìííì ë¤ ê²ìê°ë°í ì ì² ì ëë¤.</p> 248 249</blockquote> 250 251<p>ë°°ë¬ì민족 ê²ììì¤í ìì ê²ìì ì¬ì©ëë ë°ì´í°ë¥¼ ì ì¬íë©´ì ê²½ííë ì´ë ¤ìê³¼ í´ê²°íë ë°©ë²ì ê³µì íê³ ì í©ëë¤.</p> 252 253<h2 id="ê²ììì¤í ì´ë">ê²ììì¤í ì´ë?</h2> 254<p><code class="language-plaintext highlighter-rouge">
254ë¨¹ê³ ì¶ì ììì ì°¾ë 첫 ë¨ê³</code> <br /> 255ë°°ë¬ì민족ìì ê²ììì¤í ì 주문íê³ ì¶ì ê°ê² ëë ë©ë´ë¥¼ ë¹ ë¥´ê² ì°¾ì ì ìëë¡ ëì주ë ì¬ë¬ 기ë¥ì ê°ë°íê³ ììµëë¤.</p> 256<ul> 257 <li>ê²ì 기ë¥</li> 258 <li>ì¸ê¸°ê²ìì´</li> 259 <li>1ì¸ë¶, ì±ì ë±ì í ë§ ì§ë©´ 리ì¤í </li> 260 <li><code class="language-plaintext highlighter-rouge">ë°°ë¬ ë¹ ë¥¸ ì, ë°°ë¬í ë®ì ì</code>ê³¼ ê°ì ì ë ¬ê³¼ <code class="language-plaintext highlighter-rouge">ìµì주문ê¸ì¡, ì¿ í°, ìì½</code>ë±ì íí° ê¸°ë¥</li> 261</ul> 262 263<div style="text-align: center;"> 264 <img src="/img/2021-03-03/s1.jpg" /> <span style="display: block; color:#808080;">ê²ìê°ë°íì´ ì ê³µíë ìë¹ì¤</span> 265</div> 266<p><br /></p> 267 268<p>ë¨ìíë¤ê³ ìê°ë ì ìë ìì¤í ì´ì§ë§ ì¤ìê°ì¼ë¡ ê°ê²ì ëí ì 보를 ë기íí기 ìí´ì ìì¬ê°ì ì¸ë¶ìì¤í ì¼ë¡ë¶í° ì¤ìê° ì 보를 ì ì¥í´ì¼ íë©°, ìê°, ìì¹ë¥¼ 기ë°ì¼ë¡ ë³µì¡í ììì¼ë¡ ê°ê²ì ì¬ë¬ì 보를 ê³ì°í´ì¼íë ë³µì¡í ìì¤í ì ëë¤.<br /><br /></p> 269 270<h2 id="ì¶ê°ëë-ìë¹ì¤-ëì´ëë-ê³¼ë¶í">ì¶ê°ëë ìë¹ì¤, ëì´ëë ê³¼ë¶í</h2> 271<p>ê²ììì¤í ì ìì ìê°íë ê²ê³¼ ê°ì ìë¹ì¤ë¥¼ ì ê³µí기 ìí´ì ê´ê³ , ë©ë´, 리뷰, íµê³â¦ ë± ìì¬ ê°ì ë°ì´í°ë¥¼ awsì SQS를 ì¬ì©íì¬ ìì íê³ ì´ë¥¼ ê²ììì§ <code class="language-plaintext highlighter-rouge">Elasticseach</code>ì ì ì¥íê³ ììµëë¤.<br /></p> 272<div style="text-align: center;"> 273 <img src="/img/2021-03-03/s2.jpg" /><span style="display: block; color:#808080;">ë°ì´í° ìì 구조</span> 274</div> 275 276<p><br /> 277ì´ê¸° ìì¤í ì¤ê³ ì ë¤ì´ì¤ë ë°ì´í°ì ìì´ ê·¸ë ê² ë§ì§ ììê³ ì측 ê°ë¥í기 ë문ì ê°ë³ì ì¼ë¡ ë¤ì´ì¤ë ì´ë²¤í¸ë¥¼ ê±´ë°ì´ê±´ì¼ë¡ update request를 ë§ë¤ì´ì ë°ìíììµëë¤.<br /></p> 278<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// ê°ê² ì´ë²¤í¸ ìì </span> 279<span class="nd">@SqsListener</span><span class="o">(</span><span class="n">value</span> <span class="o">=</span> <span class="s">"shopSqs"</span><span class="o">,</span> <span class="n">deletionPolicy</span> <span class="o">=</span> <span class="no">NEVER</span><span class="o">)</span> 280<span class="kd">public</span> <span class="kt">void</span> <span class="nf">tagsUpdateListen</span><span class="o">(</span><span class="nd">@Payload</span> <span class="nc">ShopChangeEvent</span> <span class="n">shopChangeEvent</span><span class="o">,</span> <span class="nc">Acknowledgment</span> <span class="n">acknowledgment</span><span class="o">)</span> <span class="o">{</span> 281 <span class="n">shopElasticsearch</span><span class="o">.</span><span class="na">update</span><span class="o">(</span><span class="n">createUpdateRequest</span><span class="o">(</span><span class="n">shopChangeEvent</span><span class="o">));</span> 282<span class="o">}</span> 283<span class="o">...</span> 284<span class="c1">// 리뷰 ì´ë²¤í¸ ìì </span> 285<span class="nd">@SqsListener</span><span class="o">(</span><span class="n">value</span> <span class="o">=</span> <span class="s">"reviewSqs"</span><span class="o">,</span> <span class="n">deletionPolicy</span> <span class="o">=</span> <span class="no">NEVER</span><span class="o">)</span> 286<span class="kd">public</span> <span class="kt">void</span> <span class="nf">tagsUpdateListen</span><span class="o">(</span><span class="nd">@Payload</span> <span class="nc">ReviewChangeEvent</span> <span class="n">reviewChangeEvent</span><span class="o">,</span> <span class="nc">Acknowledgment</span> <span class="n">acknowledgment</span><span class="o">)</span> <span class="o">{</span> 287 <span class="n">shopElasticsearch</span><span class="o">.</span><span class="na">update</span><span class="o">(</span><span class="n">createUpdateRequest</span><span class="o">(</span><span class="n">reviewChangeEvent</span><span class="o">));</span> 288<span class="o">}</span> 289</code></pre></div></div> 290<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">POST</span><span class="w"> </span><span class="err">baemin-search/_update/</span><span class="mi">1234567</span><span class="w"> 291</span><span class="p">{</span><span class="w"> 292 </span><span class="nl">"doc"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> 293 </span><span class="nl">"shopName"</span><span class="p">:</span><span class="w"> </span><span class="s2">"배민ì ë§ì§ê°ê²"</span><span class="p">,</span><span class="w"> 294 </span><span class="nl">"address"</span><span class="p">:</span><span class="w"> </span><span class="s2">"ì¡í구 ë°©ì´ë 44-2"</span><span class="w"> 295 </span><span class="p">}</span><span class="w"> 296</span><span class="p">}</span><span class="w"> 297</span></code></pre></div></div> 298<p>íì§ë§ ê°ê²ê° ì¦ê°í¨ì ë°ë¼ ê°ê²ê° ì¤ííë ìê°ëë ì£¼ë¬¸ì´ ë§ìì ë©ë´ íì ë±ì ìí©ì´ ë§ì´ ë°ìíë ê²½ì°ìë ì´ë²¤í¸ë¥¼ ë§ì´ ìì íê² ëìê³ ê·¸ì ë°ë¼ 모ë ë¶ë´ì´ Elasticsearchì ì§ì¤ëë©´ì Elasticsearch Writer Thread Poolì´ ê°ë ì°¨ì requestê° rejected ëë 문ì ì Disk I/O 문ì ë¡ cpu ìì¹ê° 기íê¸ìì ì¼ë¡ ìì¹íë 문ì ê° ë°ìíììµëë¤.<br /></p> 299 300<p>ì´ë° ìí© ìì íì¬ ìí를 ì ì§í기ë ì´ë ¤ìì¡ê³ ëë¶ì´ <code class="language-plaintext highlighter-rouge">ë°°ë¬ ë¹ ë¥¸ ì, ë°°ë¬í ë®ì ì</code>ì ë ¬/íí°ì ê°ì´ ì¬ì©ììê² í¸ì를 ì¤ ì ìë ì¬ë¬ feature를 ì í¬ ìì¤í 문ì ë¡ ê°ë°ì´ ì§ì°ëë 문ì ê° ë°ìíììµëë¤.<br /><br /></p> 301 302<h2 id="ìì¸-ë¶ì">ìì¸ ë¶ì</h2> 303<h3 id="문ì -1-ì¦ì-ë³ê²½ì-ìí-disk-io-ì´ì">문ì 1. ì¦ì ë³ê²½ì ìí Disk I/O ì´ì</h3> 304<p>Elasticsearchì Indexë Lucene 기ë°ì ìì segment íì¼ë¡ 구ì±ëì´ ììµëë¤. ì´ segmentë ê²ì ìì²ì ë¹ ë¥´ê² ì²ë¦¬í기 ìí´ì ì¬ë¬ ê°ë¡ ìì±ì´ ëì´ììµëë¤. íì§ë§ ì´ segmentë ìë, ì±ë¥, ëìì± ì´ì, ìì¶ ë±ì ì´ì ë¡ immutable í ìì±ì ê°ì§ê³ ì기 ë문ì Indexì ëí ì ë°ì´í¸ ìì²ì´ ì¤ê² ëë©´ 기존 segmentì markë§ íê³ ìë¡ì´ segment를 ìì±íì¬ ë°ì´í°ë¥¼ 기ë¡íë ëìì ì§íí©ëë¤.</p> 305<div style="text-align: center;"> 306 <img src="/img/2021-03-03/s3.jpg" /><span style="display: block; color:#808080;">ì´ë¯¸ì§ ì¶ì² 구ê¸</span> 307</div> 308<p><br /></p> 309 310<p>ì í¬ì 문ì ë ë°ë¡ ì¬ê¸°ì ìììµëë¤. ì¦ì ì ë°ì´í¸ë¡ ì¸í´ ë¤ëì segmentê° ìì±ëë©´ì Disk I/Oê° ì¦ê°íê² ëìê³ ê·¸ ê²°ê³¼ cpuê° ëë° ìì¹íì¼ë©°, ìì ëì segment를 ì§ì°ê³ ë¨í¸í í´ì를 ìí´ ëìíë force merge ììë ë¶íê° ë°ìíë 문ì ê° íì¸ëììµëë¤. <br /></p> 311<div style="text-align: center;"> 312 <img src="/img/2021-03-03/s4.jpg" /><span style="display: block; color:#808080;">문ì ì cpu ìí</span> 313</div> 314<p><br /></p> 315 316<h3 id="문ì -2-ê±´ë°ì´ê±´-ìì²ì¼ë¡-ì¸í-thread-pool-ì´ê³¼">문ì 2. ê±´ë°ì´ê±´ ìì²ì¼ë¡ ì¸í thread pool ì´ê³¼</h3> 317<p>ê°ê²ì í´ë¹íë ìì± ë³ê²½ ì´ë²¤í¸ê° 몰리ë ìê°ëìë ìµë ììë§ ê±´ ì´ìì ì´ë²¤í¸ê° ë°ìí©ëë¤. ì´ ì´ë²¤í¸ë¥¼ ê±´ë°ì´ê±´ì¼ë¡ ì ë°ì´í¸ë¥¼ íë©´ì Elasticsearchì write thread poolì 모ë ì ì íê³ queueê¹ì§ ê°ë ì°¨ê² ëë©´ì ì¼ë¶ ì´ë²¤í¸ê° ì ìì ì¼ë¡ ì ë°ì´í¸ ëì§ ëª»íê³ rejected ëë ê²½ì°ê° ë¹
317ë²íê² ë°ìíììµëë¤.</p> 318<div style="text-align: center;"> 319 <img src="/img/2021-03-03/s5.jpg" /><span style="display: block; color:#808080;">rejected request ê°ì</span> 320</div> 321<p><br /><br /></p> 322 323<h2 id="ì ëµì-ì¸ì ë-기본ì-ìë¤">ì ëµì ì¸ì ë 기본ì ìë¤</h2> 324<blockquote> 325 <p>To optimize insert speed, combine many small operations into a single large operation. Ideally, you make a single connection, send the data for many new rows at once, and delay all index updates and consistency checking until the very end.<br /> 326ì¶ì² : https://dev.mysql.com/doc/refman/8.0/en/insert-optimization.html</p> 327 328 <p><code class="language-plaintext highlighter-rouge">insert ì±ë¥ì ëì´ê¸° ìí´ìë ìì ì¬ë¬ ì°ì°ì í° íëì ì°ì°ì¼ë¡ ë§ë¤ì´ë¼.</code> ë¼ë ë§ì ìì mysql docìì ë³¼ ì ìë¯ì´ ê·¸ë ê² ë°°ì ê³ ê·¸ë ê² ê°ë°ì ì§ííììµëë¤.<br /></p> 329</blockquote> 330 331<p>Elasticsearch ëí ê°ì ê°ë ì ê°ì§ê³ ìì¼ë©° ë¨ê±´ request ìí ë³´ë¤ Bulk requestì í¨ë¥ì´ í° ê²ì ì¬ë¬ 벤ì¹ë§í¹ì íµí´ìë ì ì¦ì´ ëììµëë¤.<br /></p> 332 333<p>ê·¸ë¼ ì°ë¦¬ë ì´ ë¬¸ì 를 í´ê²°í기 ìí´ bulkë¡ ëìì ìí¤ë ë°©ë²ì ì ì©í´ë³´ê¸°ë¡ íìµëë¤. íì§ë§ ì¸ì ë 문ì ë ì½ê² í´ê²°ëì§ ìë¯ì´ ìì 문ì ê° ìììµëë¤. 334ê²ìì íì¬ êµ¬ì¡°ë 모ë ì´ë²¤í¸ë¥¼ ë³ëì sqs를 listen íì¬ ë°ê³ ì기 ë문ì ìë¡ ë¤ë¥¸ sqsìì ë°ì´í°ë¥¼ ë°ìì ì²ë¦¬í기ìë ë¬´ë¦¬ê° ìììµëë¤.</p> 335 336<p>ê·¸ë¬ë¤ Elasticsearchì resthighlevelclient java apiìì <a href="https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-docs-bulk-processor.html">Bulk processor</a> ë¼ë 기ë¥ì ì¬ì©íì¬ request를 모ìì ì ë°ì´í¸íë 기ë¥ì ë°ê²¬íê² ëììµëë¤. 337Bulk Processorë íëì ë 립ì ì¸ íë¡ì¸ì¤ë¥¼ ì¤íìí¤ê³ request를 bufferì 모ìì ì í´ì§ ìê°, í¬ê¸° ë±ì ë§ê²
337 Elasticsearchì ë°ì´í°ë¥¼ Bulk request를 ë§ë¤ì´ì flushíëë¡ ëìíê³ ììµëë¤.</p> 338<div style="text-align: center;"> 339 <img src="/img/2021-03-03/s6.jpg" /><span style="display: block; color:#808080;">Bulk processor 구조</span> 340</div> 341<p><br /></p> 342 343<p>ì´ê±¸ ì¬ì©í´ì ì í¬ì 문ì 를 íì´ë³´ê¸°ë¡ íì¼ë©° ìë ê·¸ëë¡ ì¬ì©íì§ ìê³ ì¡°ê¸ ë ëì ì±ë¥ì ë§ë¤ê¸° ìí´ íëë í¨ê» ì§ííìµëë¤.<br /></p> 344 345<h3 id="íë-1">íë 1.</h3> 346<p>기존 Bulk processorë DocWriteRequest를 ë´ë List를 ê°ì§ê³ ìë BulkRequestë¼ë Bufferì© ê°ì²´ë¥¼ ì¬ì©íì¬ ë°ì´í°ë¥¼ ë´ê³ ìê°, ì¬ì´ì¦ ë±ì flush íì´ë°ì´ ëë©´ í´ë¹ ë²í¼ë¥¼ BulkProcessorHandler를 íµí´ ì ë¬íê³ Elasticsearchì Bulk request를 ìì²íë êµ¬ì¡°ë¡ ëì´ììµëë¤.</p> 347<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// BulkRequest (Buffer ìí ì íë ê°ì²´)</span> 348<span class="kd">public</span> <span class="kd">class</span> <span class="nc">BulkRequest</span> <span class="kd">extends</span> <span class="nc">ActionRequest</span> <span class="kd">implements</span> <span class="nc">CompositeIndicesRequest</span><span class="o">,</span> <span class="nc">WriteRequest</span><span class="o"><</span><span class="nc">BulkRequest</span><span class="o">></span> <span class="o">{</span> 349 350 <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="kt">int</span> <span class="no">REQUEST_OVERHEAD</span> <span class="o">=</span> <span class="mi">50</span><span class="o">;</span> 351 352 <span class="cm">/** 353 * Requests that are part of this request. It is only possible to add things that are both {@link ActionRequest}s and 354 * {@link WriteRequest}s to this but java doesn't support syntax to declare that everything in the array has both types so we declare 355 * the one with the least casts. 356 */</span> 357 <span class="kd">final</span> <span class="nc">List</span><span class="o"><</span><span class="nc">DocWriteRequest</span><span class="o"><?>></span> <span class="n">requests</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><>();</span> 358 <span class="o">..</span> 359 <span class="o">..</span> 360 <span class="o">..</span> 361 362 <span class="cm">/** 363 * Adds a list of requests to be executed. Either index or delete requests. 364 */</span> 365 <span class="kd">public</span> <span class="nc">BulkRequest</span> <span class="nf">add</span><span class="o">(</span><span class="nc">DocWriteRequest</span><span class="o"><?>...</span> <span class="n">requests</span><span class="o">)</span> <span class="o">{</span> 366 <span class="k">for</span> <span class="o">(</span><span class="nc">DocWriteRequest</span><span class="o"><?></span> <span class="n">request</span> <span class="o">:</span> <span class="n">requests</span><span class="o">)</span> <span class="o">{</span> 367 <span class="n">add</span><span class="o">(</span><span class="n">request</span><span class="o">,</span> <span class="kc">null</span><span class="o">);</span> 368 <span class="o">}</span> 369 <span class="k">return</span> <span class="k">this</span><span class="o">;</span> 370 <span class="o">}</span> 371<span class="o">}</span> 372 373<span class="c1">// BufferProcessor addë©ìë를 íµí´ BulkRequestì request를 ë§ë¤ì´ì ë´ë ë¶ë¶</span> 374<span class="n">bulkProcessor</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="nc">IndexRequest</span><span class="o">(</span><span class="s">"twitter"</span><span class="o">,</span> <span class="s">"_doc"</span><span class="o">,</span> <span class="s">"1"</span><span class="o">).</span><span class="na">source</span><span class="o">(</span><span class="cm">/* your doc here */</span><span class="o">));</span> 375<span class="n">bulkProcessor</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="nc">DeleteRequest</span><span class="o">(</span><span class="s">"twitter"</span><span class="o">,</span> <span class="s">"_doc"</span><span class="o">,</span> <span class="s">"2"</span><span class="o">));</span> 376 377<span class="c1">// BulkProcessor Handler</span> 378<span class="kd">public</span> <span class="kt">void</span> <span class="nf">execute</span><span class="o">(</span><span class="nc">BulkRequest</span> <span class="n">bulkRequest</span><span class="o">,</span> <span class="kt">long</span> <span class="n">executionId</span><span class="o">)</span> <span class="o">{</span> 379 <span class="nc">Runnable</span> <span class="n">toRelease</span> <span class="o">=</span> <span class="o">()</span> <span class="o">-></span> <span class="o">{};</span> 380 <span class="kt">boolean</span> <span class="n">bulkRequestSetupSuccessful</span> <span class="o">=</span> <span class="kc">
380false</span><span class="o">;</span> 381 <span class="k">try</span> <span class="o">{</span> 382 <span class="n">listener</span><span class="o">.</span><span class="na">beforeBulk</span><span class="o">(</span><span class="n">executionId</span><span class="o">,</span> <span class="n">bulkRequest</span><span class="o">);</span> 383 <span class="n">semaphore</span><span class="o">.</span><span class="na">acquire</span><span class="o">();</span> 384 <span class="n">toRelease</span> <span class="o">=</span> <span class="nl">semaphore:</span><span class="o">:</span><span class="n">release</span><span class="o">;</span> 385 <span class="nc">CountDownLatch</span> <span class="n">latch</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">CountDownLatch</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span> 386 <span class="n">retry</span><span class="o">.</span><span class="na">withBackoff</span><span class="o">(</span><span class="n">consumer</span><span class="o">,</span> <span class="n">bulkRequest</span><span class="o">,</span> <span class="k">new</span> <span class="nc">ActionListener</span><span class="o"><</span><span class="nc">BulkResponse</span><span class="o">>()</span> <span class="o">{</span> 387 <span class="nd">@Override</span> 388 <span class="kd">public</span> <span class="kt">void</span> <span class="nf">onResponse</span><span class="o">(</span><span class="nc">BulkResponse</span> <span class="n">response</span><span class="o">)</span> <span class="o">{</span> 389 <span class="k">try</span> <span class="o">{</span> 390 <span class="n">listener</span><span class="o">.</span><span class="na">afterBulk</span><span class="o">(</span><span class="n">executionId</span><span class="o">,</span> <span class="n">bulkRequest</span><span class="o">,</span> <span class="n">response</span><span class="o">);</span> 391 <span class="o">}</span> <span class="k">finally</span> <span class="o">{</span> 392 <span class="n">semaphore</span><span class="o">.</span><span class="na">release</span><span class="o">();</span> 393 <span class="n">latch</span><span class="o">.</span><span class="na">countDown</span><span class="o">();</span> 394 <span class="o">}</span> 395 <span class="o">}</span> 396 397 <span class="nd">@Override</span> 398 <span class="kd">public</span> <span class="kt">void</span> <span class="nf">onFailure</span><span class="o">(</span><span class="nc">Exception</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span> 399 <span class="k">try</span> <span class="o">{</span> 400 <span class="n">listener</span><span class="o">.</span><span class="na">afterBulk</span><span class="o">(</span><span class="n">executionId</span><span class="o">,</span> <span class="n">bulkRequest</span><span class="o">,</span> <span class="n">e</span><span class="o">);</span> 401 <span class="o">}</span> <span class="k">finally</span> <span class="o">{</span> 402 <span class="n">semaphore</span><span class="o">.</span><span class="na">release</span><span class="o">();</span> 403 <span class="n">latch</span><span class="o">.</span><span class="na">countDown</span><span class="o">();</span> 404 <span class="o">}</span> 405 <span class="o">}</span> 406 <span class="o">});</span> 407 <span class="n">bulkRequestSetupSuccessful</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span> 408 <span class="k">if</span> <span class="o">(</span><span class="n">concurrentRequests</span> <span class="o">==</span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span> 409 <span class="n">latch</span><span class="o">.</span><span class="na">await</span><span class="o">();</span> 410 <span class="o">}</span> 411 <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">InterruptedException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
412 <span class="nc">Thread</span><span class="o">.</span><span class="na">currentThread</span><span class="o">().</span><span class="na">interrupt</span><span class="o">();</span> 413 <span class="n">logger</span><span class="o">.</span><span class="na">info</span><span class="o">(()</span> <span class="o">-></span> <span class="k">new</span> <span class="nc">ParameterizedMessage</span><span class="o">(</span><span class="s">"Bulk request {} has been cancelled."</span><span class="o">,</span> <span class="n">executionId</span><span class="o">),</span> <span class="n">e</span><span class="o">);</span> 414 <span class="n">listener</span><span class="o">.</span><span class="na">afterBulk</span><span class="o">(</span><span class="n">executionId</span><span class="o">,</span> <span class="n">bulkRequest</span><span class="o">,</span> <span class="n">e</span><span class="o">);</span> 415 <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">Exception</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span> 416 <span class="n">logger</span><span class="o">.</span><span class="na">warn</span><span class="o">(()</span> <span class="o">-></span> <span class="k">new</span> <span class="nc">ParameterizedMessage</span><span class="o">(</span><span class="s">"Failed to execute bulk request {}."</span><span class="o">,</span> <span class="n">executionId</span><span class="o">),</span> <span class="n">e</span><span class="o">);</span> 417 <span class="n">listener</span><span class="o">.</span><span class="na">afterBulk</span><span class="o">(</span><span class="n">executionId</span><span class="o">,</span> <span class="n">bulkRequest</span><span class="o">,</span> <span class="n">e</span><span class="o">);</span> 418 <span class="o">}</span> <span class="k">finally</span> <span class="o">{</span> 419 <span class="k">if</span> <span class="o">(</span><span class="n">bulkRequestSetupSuccessful</span> <span class="o">==</span> <span class="kc">false</span><span class="o">)</span> <span class="o">{</span> <span class="c1">// if we fail on client.bulk() release the semaphore</span> 420 <span class="n">toRelease</span><span class="o">.</span><span class="na">run</span><span class="o">();</span> 421 <span class="o">}</span> 422 <span class="o">}</span> 423<span class="o">}</span> 424</code></pre></div></div> 425 426<p>ì í¬ë ì´ ë¶ë¶ìì ìì í ì´ë²¤í¸ë¥¼ Bulk processorì add í ë Update request를 ë§ë¤ì´ì bufferì ë°ì´ ë£ë ë¶ë¶ì´ ë²ê±°ë¡ì ê³ ê°ì indexì ëì¼í Update Request ê·ì¹ì ë§ë¤ëë 모ë ì´ë²¤í¸ ìì ë¶ë¶ì ì¼ê´ ì ì©í기 ì´ë ¤ì ìµëë¤. ê·¸ëì buffer ìí ì íë BulkRequest를 ê±·ì´ë´ê³ Update Request를 ë§ë¤ê¸° ìí Shopì´ë¼ë ê°ì²´ë¥¼ ë§ë¤ê³ ì´ ê°ì²´ë¤ì ë´ì ì ìë Mapì¼ë¡ ëì²´íììµëë¤<br /><br /> 427ê° ì´ë²¤í¸ ìì ìì¹ììë Bulk Processorì add í ë ê° ì´ë²¤í¸ë³ ë³ê²½ ë¶ë¶ë§ ì ë¬í ì ìëë¡ ë³ëì ê°ì²´(ShopMenu, ShopReview..)ë¤ì 구ì±íìê³ , ì´ ê°ë¤ì ì ë¬ë°ì Bulk processorë Shopê°ì²´ë¡ ì ííì¬ bufferì ì ì¥ì íëë¡ êµ¬ì±íììµëë¤.(<span style="color: red; weight: bold;">ì´ ë¶ë¶ ì½ë ë° íì©ì íë2ìì ë¤ë£° ìì </span>) ê·¸ë¦¬ê³ ë³ëì serializer ì ì± ì ì ì©í´ì flush 주기ì BulkProcessorHandlerì ë°ì´í°ë¥¼ ì ë¬íì¬ íìí ë¶ë¶ë§ ì ë°ì´í¸ ë ì ìëë¡ BulkRequest를 ìì±íë ë±ì ìì ì ì§ííììµëë¤.<br /><br /></p> 428 429<p>Bulk processor ëì ì¼ë¡ ì¸í í¨ê³¼ë¥¼ ì´ë¡ ì ì¼ë¡ ìê°íë ë¶ë¶ì´ìê³ í ì¤í¸ë¥¼ íµí´ ê²ì¦ë í´ë´¤ì§ë§ ì¤ì ì´ìììë ì´ë ì ë í¨ê³¼ê° ë°ìë ì§ ê¶ê¸íììµëë¤. 기ëê°ì ê°ì§ê³ ì´ììì íì¸í´ë´¤ìµëë¤.<br /></p> 430 431<p>ì´ì ë°ì´í° ê¸°ì¤ ì´ë²¤í¸ ìì ê±´ìë ì´ <em>9,261,187ê°</em> ììµëë¤.<br /></p> 432 433<p>Bulk processor ëì ì ìë 모ë ì´ë²¤í¸ë¥¼ ë¨ê±´ì¼ë¡ ì²ë¦¬íì기 ë문ì ì´ update requestë ì´ë²¤í¸ ìì ê±´ìì ëì¼íê² 9,261,185ê° ììµë
433ë¤.<br /> 434Bulk processor ëì íìë update request를 모ì Bulk request를 ë§ë¤ë©´ì requestê±´ìë 233,949ê±´ ë°ìíììµëë¤.</p> 435 436<blockquote> 437 <p>ëì ì : 9,261,185ê±´<br /> 438ëì í : 233,949ê±´</p> 439 440</blockquote> 441 442<p>Elasticsearchì ìì²íë request ê±´ìê° <span style="color:#d23c3c; font-weight:700"> ì´ 97.5% </span> ê°ìíììµëë¤. 443ì´ë¡ ì¸í´ write thread pool ê°ìë¡ ë§ì ì´ë²¤í¸ ìì ì ë°ìíë rejectedê° í´ìëìê³ , bulkë¡ ë¨ì¼ ìì² ëë¹ ìëµ ë기 ìê° ë° ë¤í¸ìí¬ ë¶í ê°ì í¨ê³¼ë¥¼ ë°ììµëë¤.</p> 444 445<h3 id="íë-2">íë 2.</h3> 446<p>íë 1ìì ì»ì í¨ê³¼ë ì¢ìì§ë§ ì¡°ê¸ ë ì±ë¥ì ì¬ë ¤ë³´ê³ ì¶ììµëë¤. Bulk requestê° single requestë³´ë¤ í¨ë¥ì ìì§ë§ Bulkì ì¬ì´ì¦ê° 커ì§ìë¡ ì¤íë ¤ Elasticsearchì ë¶ë´ì ì¤ ì ì기 ë문ì Bulkì ë´ê¸°ë request를 ë ì¤ì¬ë³´ê³ ì¶ììµëë¤.</p> 447 448<p>ë°ì´í°ë¥¼ ê°ë§í ë³´ë ê²ììì ì¬ì©íë 모ë ë°ì´í°ë shopId를 기ì¤ì¼ë¡ ì ë°ì´í¸ ì¿¼ë¦¬ê° ë§ë¤ì´ì¡ìµëë¤. ê·¸ë¼ request를 add íë ìê°ì ëì¼í ê°ê²ì ëí requestê° ì¡´ì¬íë©´ ë³ê²½ëë ë¶ë¶ë§ 기존 requestì ë£ì´ì£¼ë©´ ëê² ë¤ë ìê°ì ê°ì§ê² ëììµëë¤.<br /></p> 449 450<div style="text-align: center;"> 451 <img src="/img/2021-03-03/s11.jpg" /> <span style="display: block; color:#808080;">ê°ê² ì¤ì¬ì¼ë¡ request ë¨ì¼í ê³¼ì </span> 452</div> 453<p><br /><br /></p> 454 455<p>ê·¸ëì ë¤ìê³¼ ê°ì´ Bulk processorì bufferìí ì íë mapì í¤ë¥¼ shopIdë¡ ë¶ì¬íê³ value를 Shop ê°ì²´ë¡ ë¶ì¬ë¥¼ íìê³ ê° ì´ë²¤í¸ë¥¼ íµí´ ë¤ì´ì¤ë ìì²ì íëì Shopì¼ë¡ 묶ì´ì Bulk request를 ë§ë¤ëë¡ ì§ííììµëë¤.</p> 456<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span> <span class="n">ì¤ì </span> <span class="n">ì½ëë</span> <span class="n">ë</span> <span class="n">ë³µì¡íì¬</span> <span class="n">ìì</span> <span class="n">ì½ë를</span> <span class="n">기ì¬í©ëë¤</span><span class="o">.</span> 457 458<span class="cm">/** 459 * dataë ì ì°íê² ë°ì ì ìëë¡ ì ë¤ë¦ì ì¬ì©íìê³ , ì¤ì ê²ìììë Shopê°ì²´ë¥¼ ë°ê³ ìë¤. 460 * acknowledgmentsë ìì ë ì´ë²¤í¸ë¤ì ëí ì ì ì²ë¦¬ ì¬ë¶ì ë°ë¼ sqsì ack ì²ë¦¬ë¥¼ í기 ìí´ ë³´ê´ 461*/</span> 462<span class="nd">@Getter</span> 463<span class="nd">@Setter</span> 464<span class="nd">@NoArgsConstructor</span> 465<span class="kd">public</span> <span class="kd">class</span> <span class="nc">BulkData</span><span class="o"><</span><span class="no">T</span> <span class="kd">extends</span> <span class="nc">IndexDomain</span><span class="o">></span> <span class="o">{</span> 466 <span class="kd">private</span> <span class="no">T</span> <span class="n">data</span><span class="o">;</span> 467 <span class="kd">private</span> <span class="nc">Set</span><span class="o"><</span><span class="nc">Acknowledgment</span><span class="o">></span> <span class="n">acknowledgments</span><span class="o">;</span> 468 469 <span class="nd">@Builder</span> 470 <span class="kd">public</span> <span class="nf">BulkData</span><span class="o">(</span><span class="no">T</span> <span class="n">data</span><span class="o">,</span> <span class="nc">Set</span><span class="o"><</span><span class="nc">Acknowledgment</span><span class="o">></span> <span class="n">acknowledgments</span><span class="o">)</span> <span class="o">{</span> 471 <span class="k">this</span><span class="o">.</span><span class="na">data</span> <span class="o">=</span> <span class="n">data</span><span class="o">;</span> 472 <span class="k">this</span><span class="o">.</span><span class="na">acknowledgments</span> <span class="o">=</span> <span class="n">acknowledgments</span><span class="o">;</span> 473 <span class="o">}</span> 474<span class="o">}</span> 475 476<span class="nd">@Getter</span> 477<span class="nd">@Setter</span> 478<span class="kd">public</span> <span class="kd">class</span> <span class="nc">Shop</span> <span class="kd">extends</span> <span class="nc">IndexDomain</span> <span class="o">{</span> 479 <span class="kd">private</span> <span class="nc">Long</span> <span class="n">shopNumber</span><span class="o">;</span> 480 <span class="kd">private</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Menu</span><span class="o">></span> <span class="n">menus</span><span class="o">;</span> 481 <span class="kd">private</span> <span class="nc">String</span> <span class="n">content</span><span class="o">;</span> 482 <span class="kd">private</span> <span class="nc">String</span> <span class="n">subject</span><span class="o">;</span> 483 484 <span class="nd">@Builder</span> 485 <span class="kd">public</span> <span class="nf">Shop</span><span class="o">(</span><span class="nc">Long</span> <span class="n">shopNumber</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Menu</span><span class="o">></span> <span class="n">menus</span><span class="o">,</span> <span class="nc">String</span> <span class="n">content</span><span class="o">,</span> <span class="nc">String</span> <span class="n">subject</span><span class="o">)</span> <span class="o">{</span>
486 <span class="k">this</span><span class="o">.</span><span class="na">shopNumber</span> <span class="o">=</span> <span class="n">shopNumber</span><span class="o">;</span> 487 <span class="k">this</span><span class="o">.</span><span class="na">menus</span> <span class="o">=</span> <span class="n">menus</span><span class="o">;</span> 488 <span class="k">this</span><span class="o">.</span><span class="na">content</span> <span class="o">=</span> <span class="n">content</span><span class="o">;</span> 489 <span class="k">this</span><span class="o">.</span><span class="na">subject</span> <span class="o">=</span> <span class="n">subject</span><span class="o">;</span> 490 <span class="o">}</span> 491<span class="o">}</span> 492 493<span class="c1">// ì´ë²¤í¸ ë³ê²½ì 보를 ê°ì§ë ê°ì²´ì ê³µíµ ì¶ì í´ëì¤</span> 494<span class="kd">public</span> <span class="kd">abstract</span> <span class="kd">class</span> <span class="nc">ShopUpdatableDomain</span> <span class="kd">extends</span> <span class="nc">UpdatableDomain</span><span class="o"><</span><span class="nc">Shop</span><span class="o">></span> <span class="o">{</span> 495 496 <span class="nd">@Override</span> 497 <span class="kd">public</span> <span class="nc">Shop</span> <span class="nf">convert</span><span class="o">()</span> <span class="o">{</span> 498 <span class="nc">Shop</span> <span class="n">shop</span> <span class="o">=</span> <span class="nc">Shop</span><span class="o">.</span><span class="na">builder</span><span class="o">().</span><span class="na">build</span><span class="o">();</span> 499 <span class="n">convert</span><span class="o">(</span><span class="n">shop</span><span class="o">);</span> 500 <span class="k">return</span> <span class="n">shop</span><span class="o">;</span> 501 <span class="o">}</span> 502<span class="o">}</span> 503 504<span class="c1">// ë©ë´ ë³ê²½ì 보를 ê°ì§ê³ ìë ShopMenu</span> 505<span class="nd">@Getter</span> 506<span class="nd">@Setter</span> 507<span class="nd">@NoArguments</span> 508<span class="kd">public</span> <span class="kd">class</span> <span class="nc">ShopMenu</span> <span class="kd">extends</span> <span class="nc">ShopUpdatableDomain</span> <span class="o">{</span> 509 <span class="kd">private</span> <span class="nc">Long</span> <span class="n">shopNumber</span><span class="o">;</span> 510 <span class="kd">private</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Menu</span><span class="o">></span> <span class="n">menus</span><span class="o">;</span> 511 512 <span class="nd">@Builder</span> 513 <span class="kd">public</span> <span class="nf">ShopMenu</span><span class="o">(</span><span class="nc">Long</span> <span class="n">shopNumber</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Menu</span><span class="o">></span> <span class="n">menus</span><span class="o">)</span> <span class="o">{</span> 514 <span class="k">this</span><span class="o">.</span><span class="na">shopNumber</span> <span class="o">=</span> <span class="n">shopNumber</span><span class="o">;</span> 515 <span class="k">this</span><span class="o">.</span><span class="na">menus</span> <span class="o">=</span> <span class="n">menus</span><span class="o">;</span> 516 <span class="o">}</span> 517 518 <span class="nd">@Override</span> 519 <span class="kd">public</span> <span class="kt">void</span> <span class="nf">convert</span><span class="o">(</span><span class="nc">Shop</span> <span class="n">shop</span><span class="o">)</span> <span class="o">{</span> 520 <span class="n">shop</span><span class="o">.</span><span class="na">setShopNumber</span><span class="o">(</span><span class="n">shopNumber</span><span class="o">);</span> 521 <span class="n">shop</span><span class="o">.</span><span class="na">setMenus</span><span class="o">(</span><span class="n">menus</span><span class="o">);</span> 522 <span class="o">}</span> 523<span class="o">}</span> 524 525<span class="c1">// 리뷰 ë³ê²½ì 보를 ê°ì§ê³ ìë ShopReview</span> 526<span class="nd">@Getter</span> 527<span class="nd">@Setter</span> 528<span class="nd">@NoArguments</span> 529<span class="kd">public</span> <span class="kd">class</span> <span class="nc">ShopReview</span> <span class="kd">extends</span> <span class="nc">ShopUpdatableDomain</span> <span class="o">{</span> 530 <span class="kd">private</span> <span class="nc">Long</span> <span class="n">shopNumber</span><span class="o">;</span> 531 <span class="kd">private</span> <span class="nc">String</span> <span class="n">content</span><span class="o">;</span> 532 <span class="kd">private</span> <span class="nc">String</span> <span class="n">subject</span><span class="o">;</span> 533 534 <span class="nd">@Builder</span> 535 <span class="kd">public</span> <span class="nf">ShopReview</span><span class="o">(</span><span class="nc">Long</span> <span class="n">shopNumber</span><span class="o">,</span> <span class="nc">String</span> <span class="n">content</span><span class="o">,</span> <span class="nc">String</span> <span class="n">subject</span><span class="o">)</span> <span class="o">{</span>
536 <span class="k">this</span><span class="o">.</span><span class="na">shopNumber</span> <span class="o">=</span> <span class="n">shopNumber</span><span class="o">;</span> 537 <span class="k">this</span><span class="o">.</span><span class="na">content</span> <span class="o">=</span> <span class="n">content</span><span class="o">;</span> 538 <span class="k">this</span><span class="o">.</span><span class="na">subject</span> <span class="o">=</span> <span class="n">subject</span><span class="o">;</span> 539 <span class="o">}</span> 540 541 <span class="nd">@Override</span> 542 <span class="kd">public</span> <span class="kt">void</span> <span class="nf">convert</span><span class="o">(</span><span class="nc">Shop</span> <span class="n">shop</span><span class="o">)</span> <span class="o">{</span> 543 <span class="n">shop</span><span class="o">.</span><span class="na">setShopNumber</span><span class="o">(</span><span class="n">shopNumber</span><span class="o">);</span> 544 <span class="n">shop</span><span class="o">.</span><span class="na">setContent</span><span class="o">(</span><span class="n">content</span><span class="o">);</span> 545 <span class="n">shop</span><span class="o">.</span><span class="na">setSubject</span><span class="o">(</span><span class="n">subject</span><span class="o">);</span> 546 <span class="o">}</span> 547<span class="o">}</span> 548 549<span class="c1">// ì´ë²¤í¸ ìì ê³¼ Bulk processorì ì´ë²¤í¸ ë´ì© ì ë¬</span> 550<span class="nd">@SqsListener</span><span class="o">(</span><span class="n">value</span> <span class="o">=</span> <span class="s">"menu-sqs"</span><span class="o">,</span> <span class="n">deletionPolicy</span> <span class="o">=</span> <span class="no">NEVER</span><span class="o">)</span> 551<span class="kd">public</span> <span class="kt">void</span> <span class="nf">shopMenuV3UpdateListen</span><span class="o">(</span><span class="nd">@Payload</span> <span class="nc">ShopMenuEvent</span> <span class="n">eventDto</span><span class="o">,</span> <span class="nc">Acknowledgment</span> <span class="n">acknowledgment</span><span class="o">)</span> <span class="o">{</span> 552 <span class="n">bulkProcessor</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="nc">BulkProcessorBucket</span><span class="o">.<</span><span class="nc">ShopMenu</span><span class="o">></span><span class="n">builder</span><span class="o">()</span> 553 <span class="o">.</span><span class="na">data</span><span class="o">(</span><span class="nc">ShopMenu</span><span class="o">.</span><span class="na">builder</span><span class="o">()</span> 554 <span class="o">.</span><span class="na">shopNumber</span><span class="o">(</span><span class="n">eventDto</span><span class="o">.</span><span class="na">getShopId</span><span class="o">())</span> 555 <span class="o">.</span><span class="na">menus</span><span class="o">(</span><span class="n">eventDto</span><span class="o">.</span><span class="na">getMenus</span><span class="o">())</span> 556 <span class="o">.</span><span class="na">build</span><span class="o">())</span> 557 <span class="o">.</span><span class="na">build</span><span class="o">());</span> 558<span class="o">}</span> 559<span class="o">...</span> 560<span class="nd">@SqsListener</span><span class="o">(</span><span class="n">value</span> <span class="o">=</span> <span class="s">"reviewSqs"</span><span class="o">,</span> <span class="n">deletionPolicy</span> <span class="o">=</span> <span class="no">NEVER</span><span class="o">)</span> 561<span class="kd">public</span> <span class="kt">void</span> <span class="nf">tagsUpdateListen</span><span class="o">(</span><span class="nd">@Payload</span> <span class="nc">ReviewChangeEvent</span> <span class="n">reviewChangeEvent</span><span class="o">,</span> <span class="nc">Acknowledgment</span> <span class="n">acknowledgment</span><span class="o">)</span> <span class="o">{</span> 562 <span class="n">bulkProcessor</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="nc">BulkProcessorBucket</span><span class="o">.<</span><span class="nc">ShopReview</span><span class="o">></span><span class="n">builder</span><span class="o">()</span>
563 <span class="o">.</span><span class="na">data</span><span class="o">(</span><span class="nc">ShopReview</span><span class="o">.</span><span class="na">builder</span><span class="o">()</span> 564 <span class="o">.</span><span class="na">shopNumber</span><span class="o">(</span><span class="n">eventDto</span><span class="o">.</span><span class="na">getShopId</span><span class="o">())</span> 565 <span class="o">.</span><span class="na">content</span><span class="o">(</span><span class="n">eventDto</span><span class="o">.</span><span class="na">getContent</span><span class="o">())</span> 566 <span class="o">.</span><span class="na">subject</span><span class="o">(</span><span class="n">eventDto</span><span class="o">.</span><span class="na">getSubject</span><span class="o">())</span> 567 <span class="o">.</span><span class="na">build</span><span class="o">())</span> 568 <span class="o">.</span><span class="na">build</span><span class="o">());</span> 569<span class="o">}</span> 570 571 572<span class="c1">// bulk processor add ì½ë</span> 573<span class="nd">@Slf4j</span> 574<span class="kd">public</span> <span class="kd">class</span> <span class="nc">BulkProcessor</span><span class="o"><</span><span class="no">S</span> <span class="kd">extends</span> <span class="nc">IndexDomain</span><span class="o">></span> <span class="kd">implements</span> <span class="nc">Closeable</span> <span class="o">{</span> 575 576 <span class="kd">private</span> <span class="nc">Map</span><span class="o"><</span><span class="nc">String</span><span class="o">,</span> <span class="nc">BulkData</span><span class="o"><</span><span class="no">S</span><span class="o">>></span> <span class="n">datas</span><span class="o">;</span> 577 <span class="o">...</span> 578 <span class="o">..</span> 579 <span class="o">...</span> 580 <span class="kd">private</span> <span class="kd">synchronized</span> <span class="kt">void</span> <span class="nf">addData</span><span class="o">(</span><span class="nc">BulkProcessorBucket</span> <span class="n">request</span><span class="o">)</span> <span class="o">{</span> 581 <span class="nc">String</span> <span class="n">id</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="na">getData</span><span class="o">().</span><span class="na">idValue</span><span class="o">();</span> 582 583 <span class="c1">// bulk bufferìì ëì¼í ê°ê²ê° ì¡´ì¬íëì§ ì¡°í</span> 584 <span class="k">if</span> <span class="o">(</span><span class="k">this</span><span class="o">.</span><span class="na">datas</span><span class="o">.</span><span class="na">containsKey</span><span class="o">(</span><span class="n">id</span><span class="o">))</span> <span class="o">{</span> 585 <span class="nc">BulkData</span><span class="o"><</span><span class="no">S</span><span class="o">></span> <span class="n">bulkData</span> <span class="o">=</span> <span class="k">this</span><span class="o">.</span><span class="na">datas</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">id</span><span class="o">);</span> 586 587 <span class="c1">// ë¤ì´ì¨ ì´ë²¤í¸ ë°ì´í°ë¥¼ 기존 shop ë³ê²½ë¶ ì ë³´ì 매í</span> 588 <span class="n">request</span><span class="o">.</span><span class="na">getData</span><span class="o">().</span><span class="na">convert</span><span class="o">(</span><span class="n">bulkData</span><span class="o">.</span><span class="na">getData</span><span class="o">());</span> 589 <span class="k">return</span><span class="o">;</span> 590 <span class="o">}</span> 591 <span class="k">this</span><span class="o">.</span><span class="na">datas</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="n">id</span><span class="o">,</span> <span class="nc">BulkData</span><span class="o">.<</span><span class="no">S</span><span class="o">></span><span class="n">builder</span><span class="o">()</span> 592 <span class="o">.</span><span class="na">data</span><span class="o">((</span><span class="no">S</span><span class="o">)</span> <span class="n">request</span><span class="o">.</span><span class="na">getData</span><span class="o">().</span><span class="na">convert</span><span class="o">())</span>
593 <span class="o">.</span><span class="na">build</span><span class="o">());</span> 594 <span class="o">}</span> 595 <span class="o">...</span> 596 <span class="o">..</span> 597 <span class="o">.</span> 598<span class="o">}</span> 599</code></pre></div></div> 600<p><br /> 601í´ë¹ íë í¬ì¸í¸ë í¨ê³¼ê° ìììµëë¤. Bulk request를 ë§ë¤ì´ì flushíë 주기ì ê°ì ê°ê² ìì´ëë¡ ë¬¶ì´ë ì´ë²¤í¸ ì¢ ë¥ë³ ê±´ìì íìì ëí ì ë³´ë ìëì ê°ì´ ì ì§ ìì ìì¹ë¥¼ ë³¼ ì ììì¼ë©° ì¤ì ë¡ ì´ì ëí í¨ê³¼ë ë³¼ ì ìììµëë¤.</p> 602<div style="text-align: center;"> 603 <img src="/img/2021-03-03/s7.jpg" /><span style="display: block; color:#c2b4b4;"> request êµ°ì§ ì§í</span> 604</div> 605<p><br /><br /></p> 606 607<h2 id="ì±ë¥-í ì¤í¸-ë°-ì¤ì -ì´ì-리ìì¤-ë³í">ì±ë¥ í ì¤í¸ ë° ì¤ì ì´ì 리ìì¤ ë³í</h2> 608<h3 id="ì±ë¥í ì¤í¸-ê²°ê³¼">ì±ë¥í ì¤í¸ ê²°ê³¼</h3> 609<div style="text-align: center;"> 610 <img src="/img/2021-03-03/s8.jpg" /><span style="display: block; color:#c2b4b4;"> 27ë§ê±´ 무ìì ì´ë²¤í¸ í ì¤í¸ ê²°ê³¼</span> 611</div> 612<p><br /></p> 613 614<p>27ë§ ê±´ì ë°ì´í°ë¥¼ 무ììë¡ ì ì¡íê³ ë¦¬ìì¤ ë³íëì ì²´í¬í´ë´¤ìµëë¤. 615CPU ì¬ì©ëì 15% ì´ì ê°ìëìê³ request ë¨ìê° ììì§ë©´ì write complete ìë 30ë°° ì ë ê°ìíë í¨ê³¼ë¥¼ ì»ììµëë¤.<br /></p> 616 617<h3 id="ì´ì-ê²°ê³¼">ì´ì ê²°ê³¼</h3> 618<p>ì¤ì ì´ìì ì ì© ê²°ê³¼ ìì ì²ì 문ì ê° ë°ìíë ì기ì í¼í¬ìê° ê¸°ì¤ 61%를 ì¡ë°íë cpuê° ë¹êµ ëì ê¸°ì¤ ì´ë²¤í¸ê° ììë§ê±´ì´ ë ì¶ê°ëìììë ë¶êµ¬íê³ ìµë 29%ì ëì ìì ì ì¸ ìí를 ì ì§íê³ ììµëë¤.</p> 619<div style="text-align: center;"> 620 <img src="/img/2021-03-03/s10.jpg" /> <span style="display: block; color:#c2b4b4;">ìì ë ìì¤í </span> 621</div> 622<p><br /><br /></p> 623 624<p>ì´ ë¬¸ì í´ê²°ë¡ ë°°ë¬ ë¹ ë¥¸ ì, ë°°ë¬í ë®ì ìê³¼ ê°ì´ ì¤ìê°ì± ë°ì´í°ê° ë¤ì´ê°ì¼íë ìë¹ì¤ê° ì ì©ë ì ììê³ ê³ ê°ìê² ë ì¢ì ê°ì¹ë¥¼ ë리ëê² ê°ë¥í´ì¡ìµëë¤.</p> 625 626<p>2021ë ìë ê²ìì°½ì´ íë¨ì¼ë¡ ì®ê²¨ì§ê³ ì¸ê¸° ê²ìì´ ì§ë©´ ë± ë§ì 기ë¥ë¤ì´ ì¶ê°ëê³ ììµëë¤. ì´ ê¸ì ì½ê³ 모ë ì½ë를 ë³´ê³ ì¶ê±°ë, ê²ìê°ë°íì ê´ì¬ì´ ìê²¼ë¤ë©´ <a href="https://resume.woowahan.com/jobs/view.aspx?cc=244001&jidx=105376&pidx=10893&bidx=166014">[배민ìë¹ì¤ì¤] ê²ììë¹ì¤ ê°ë°ì 모ì§</a> ì ë§ì ì§ì ë¶íë립ëë¤.</p> 627 628<p>긴 ê¸ ì½ì´ì£¼ì ì ê°ì¬ë립ëë¤.</p> 629 630 </div> 631 632 <div class="fb-comments" 633 data-href="http://woowabros.github.io/woowabros/2021/03/02/search-system.html" data-width="100%" data-numposts="10"></div> 634 635 <a href="/" class="btn btn-block" id="back">목ë¡ì¼ë¡ ëìê°ê¸°</a> 636 637 </article> 638 <a href="https://www.woowahan.com/#/recruit/tech" class="sub-banner"></a> 639</section> 640
641<script> 642 var goback = document.getElementById("back"); 643 goback.addEventListener('click', function(event) { 644 window.history.back(); 645 event.stopPropagation(); 646 }); 647</script>
647 648 </div> 649 </div> 650 651 <footer class="site-footer"> 652 653 <div class="wrapper"> 654 © Woowabros. All rights reserved. Powered by GitHub Pages. 655 </div> 656 657</footer> 658
659<script> 660 (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ 661 (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), 662 m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) 663 })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); 664 //ga('create', 'UA-77115139-1', 'auto'); 665 ga('create', 'UA-167267783-1', 'auto'); 666 ga('send', 'pageview'); 667</script>
667 668 669 670 671 </body> 672 673</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.