1<html> 2 <head> 3 <meta charset="utf-8" /> 4 <meta name="generator" content="Hugo 0.88.1" /> 5 <meta name="viewport" content="width=device-width, initial-scale=1" /> 6 <link rel="icon" type="image/x-icon" href="asset/ditto.ico" /> 7 <link 8 href="https://fonts.googleapis.com/css?family=Roboto:300,400,700" 9 rel="stylesheet" 10 type="text/css" 11 /> 12 <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css"> 13 <link rel="stylesheet" href="css/custom.css" /> 14 <link rel="stylesheet" href="css/normalize.css" /> 15 16 <title>DiTTo-TTS</title> 17 <link href="css/bootstrap.min.css" rel="stylesheet" /> 18 <link 19 rel="stylesheet" 20 href="https://cdn.jsdelivr.net/npm/[email protected]/font/bootstrap-icons.css" 21 /> 22
22<script 23 src="https://cdn.jsdelivr.net/npm/[email protected]/dist/js/bootstrap.bundle.min.js" 24 integrity="sha384-C6RzsynM9kWDrMNeT87bh95OGNyZPhcTNXj1NW7RuBCsyN/o0jlpcV8Qyq46cDfL" 25 crossorigin="anonymous" 26 ></script>
26 27 28 <link rel="stylesheet" href="css/main.css" /> 29
29<script src="js/main.js" defer></script>
29 30 </head> 31 32 <body data-new-gr-c-s-check-loaded="14.1091.0" data-gr-ext-installed=""> 33 <div class="container"> 34 <header role="banner"></header> 35 <main role="main"> 36 <article itemscope="" itemtype="https://schema.org/BlogPosting"> 37 <div class="container pt-5 mt-5 shadow p-5 mb-5 bg-white rounded"> 38 <div class="text-center"> 39 <h1>DiTTo-TTS:</h1> 40 <h3> 41 42 <span style="font-weight: 760">Di</span>ffusion <span 43 style="font-weight: 760">T</span>ransformer 44 for Scalable <span 45 style="font-weight: 760">T</span>ext-<span 46 style="font-weight: 760">t</span>o-<span 47 style="font-weight: 760">S</span>peech 48 without Domain-Specific Factors 49 </h3> 50 <p class="lead fw-bold"> 51 |<span 52 style="padding: .375rem .75rem; font-size: 1rem;">ICLR 2025</span>| 53 </p> 54 <div class="affiliation"> 55 <span><a target="_blank" href="https://sites.google.com/view/keonlee9420">Keon Lee</a><sup>1</sup></span>, 56 <span><a target="_blank" href="https://github.com/ddwkim">Dong Won Kim</a><sup>1</sup></span>, 57 <span><a target="_blank" href="https://jaywalnut310.github.io/">Jaehyeon Kim</a><sup>2*</sup></span>, 58 <span><a target="_blank" href="https://sites.google.com/view/sjchung">Seungjun Chung</a><sup>1</sup></span>, 59 <span><a target="_blank" href="https://sites.google.com/view/jaewoongcho">Jaewoong Cho</a><sup>1</sup></span> 60 <p style="margin-bottom: 0;"><sup>1</sup><a target="_blank" href="https://www.krafton.ai/en/">KRAFTON</a>, <sup>2</sup><a target="_blank" href="https://www.nvidia.com/en-us/research/">NVIDIA</a></p> 61 </div> 62 <span><small><sup>*</sup>This work was done when Jaehyeon Kim was at KRAFTON.</small></span> 63 <p style="margin-top: 10;" class="links"> 64 <a target="_blank" href="https://openreview.net/forum?id=hQvX9MBowC">OpenReview</a> 65 <a target="_blank" href="https://arxiv.org/abs/2406.11427"><span class="icon"><i class="ai ai-arxiv"></i></span> <span>arXiv</span></a> 66 </p> 67 </div> 68 <p> 69 <b>Abstract.</b> 70 Large-scale latent diffusion models (LDMs) excel in content generation 71 across various modalities, but their reliance on phonemes and durations 72 in text-to-speech (TTS) limits scalability and access from other fields. 73 While recent studies show potential in removing these domain-specific 74 factors, performance remains suboptimal. In this work, we introduce DiTTo-TTS, 75 a Diffusion Transformer (DiT)-based TTS model, to investigate whether 76 LDM-based TTS can achieve state-of-the-art performance without domain-specific 77 factors. Through rigorous analysis and empirical exploration, we find that 78 (1) DiT with minimal modifications outperforms U-Net, (2) variable-length 79 modeling with a speech length predictor significantly improves results over 80 fixed-length approaches, and (3) conditions like semantic alignment in speech 81 latent representations are key to further enhancement. By scaling our training 82 data to 82K hours and the model size to 790M parameters, we achieve superior 83 or comparable zero-shot performance to state-of-the-art TTS models in 84 naturalness, intelligibility, and speaker similarity, all without relying 85 on domain-specific factors. Speech samples are available at 86 <a href="https://ditto-tts.github.io">https://ditto-tts.github.io</a>. 87 </p> 88 </div> 89 90 <div class="container pt-5 mt-5 shadow p-5 mb-5 bg-white rounded"> 91 <h2 id="model-overview" style="text-align: center"> 92 Model Overview 93 </h2> 94 95 <p style="text-align: center"> 96 <img src="asset/thisis9.png" height="200" width="800" 97 class="img-fluid" /> 98 </p> 99 100 <p> 101 <b>An overview of DiTTo-TTS</b>. (<i>middle</i>) The LDM backbone is trained to 102 denoise a span-masked noisy segment given its contextual counterpart, without utilizing 103 phoneme and phoneme-level duration (see <b>Section 3.1</b>). 104 (<i>left</i>) The inner structure of the DiT block incorporates multi-head cross-attention 105 with global AdaLN (see <b>Section 3.3</b>). 106 (<i>right</i>) The speech length predictor is based on causal transformer blocks (see <b>Section 3.2</b>). 107 Both DiT blocks and the speech length predictor employ cross-attention to condition on text representation. 108 Additionally, DiT blocks utilize AdaLN with the mean-pooled text embedding. 109 '<b>+</b>' denotes addition, and '<b>c</b>' represents concatenation. 110 '<b>text</b>' in sky blue oval represents the embedding from text encoder. 111 The red line indicates a long skip connection between the states before and after the DiT blocks. 112 </p> 113 </div> 114 115 <ul class="nav nav-tabs" id="myTab" role="tablist"> 116 <li class="nav-item" role="presentation"> 117 <button 118 class="nav-link" 119 data-bs-toggle="tab" 120 data-bs-target="#librispeech-box" 121 type="button" 122 role="tab" 123 > 124 LibriSpeech 125 </button> 126 </li> 127 <li class="nav-item" role="presentation"> 128 <button 129 class="nav-link" 130 data-bs-toggle="tab" 131 data-bs-target="#mls-box" 132 type="button" 133 role="tab" 134 > 135 Multilingual 136 </button> 137 </li> 138 <li class="nav-item" role="presentation"> 139 <button 140 class="nav-link active" 141 data-bs-toggle="tab" 142 data-bs-target="#celeb-box" 143 type="button" 144 role="tab" 145 > 146 Celebrities 147 </button> 148 </li> 149 <li class="nav-item" role="presentation"> 150 <button 151 class="nav-link" 152 data-bs-toggle="tab" 153 data-bs-target="#anime-box" 154 type="button" 155 role="tab" 156 > 157 Anime Characters 158 </button> 159 </li> 160 <li class="nav-item" role="presentation"> 161 <button 162 class="nav-link" 163 data-bs-toggle="tab" 164 data-bs-target="#robustness-box" 165 type="button" 166 role="tab" 167 > 168 Robustness 169 </button> 170 </li> 171 <li class="nav-item" role="presentation"> 172 <button 173 class="nav-link" 174 data-bs-toggle="tab" 175 data-bs-target="#speech-diversity-box" 176 type="button" 177 role="tab" 178 > 179 Speech Diversity 180 </button> 181 </li> 182 <li class="nav-item" role="presentation"> 183 <button 184 class="nav-link" 185 data-bs-toggle="tab" 186 data-bs-target="#speech-rate-contrl-box" 187 type="button" 188 role="tab" 189 > 190 Speech Rate Controllability 191 </button> 192 </li> 193 <li class="nav-item" role="presentation"> 194 <button 195 class="nav-link" 196 data-bs-toggle="tab" 197 data-bs-target="#speech-editing-box" 198 type="button" 199 role="tab" 200 > 201 Speech Editing 202 </button> 203 </li> 204 <li class="nav-item" role="presentation"> 205 <button 206 class="nav-link" 207 data-bs-toggle="tab" 208 data-bs-target="#compare-box" 209 type="button" 210 role="tab" 211 > 212 Comparison 213 </button> 214 </li> 215 </ul> 216 <div class="tab-content" id="myTabContent"> 217 <div 218 id="librispeech-box" 219 role="tabpanel" 220 class="tab-pane fade container pt-5 shadow p-5 mb-5 bg-white rounded" 221 > 222 <h2 id="librispeech" style="text-align: center"> 223 Zero-Shot TTS (LibriSpeech) 224 </h2> 225 <p> 226 Through in-context learning, DiTTo-TTS produces speech using a 227 reference audio and its corresponding text. We represent
228 <span style="color: #8fbddf; font-weight: bold" 229 >the prompt in blue</span 230 >, and 231 <span style="color: #9fde83; font-weight: bold" 232 >the generated audio in green</span 233 >. The resulting speech closely resembles the reference in 234 voice, background noise, and speaking style. When dividing 235 prompts into 3-second segments, the audio boundaries may not 236 always end in silence and could sometimes cut off words midway. 237 For further details, please see <b>English-only Evaluation</b> part in <b>Section 5.1</b> of our paper. 238 When you click on "Show baselines", you can see the ground truth 239 and baseline audio, etc. YourTTS, Vall-E, and CLaM-TTS audios are brought 240 from YourTTS demo<sup><a href="#footnote1-1">1</a></sup>, 241 Vall-E demo<sup><a href="#footnote1-2">2</a></sup 242 >, and CLaM-TTS demo<sup><a href="#footnote1-3">3</a></sup 243 > respectively. 244 </p> 245 </div> 246 <div 247 id="mls-box" 248 role="tabpanel" 249 class="tab-pane fade container pt-5 shadow p-5 mb-5 bg-white rounded" 250 > 251 <h2 id="mls" style="text-align: center"> 252 Zero-Shot TTS (Multilingual LibriSpeech) 253 </h2> 254 <p> 255 We represent 256 <span style="color: #8fbddf; font-weight: bold" 257 >the prompt in blue</span 258 >, and 259 <span style="color: #9fde83; font-weight: bold" 260 >the generated audio in green</span 261 >. The resulting speech closely resembles the reference in 262 voice, background noise, and speaking style. When dividing 263 prompts into 3-second segments, the audio boundaries may not 264 always end in silence and could sometimes cut off words midway. 265 For further details, please see <b>Multilingual Evaluation</b> part in <b>Section 5.1</b> of our paper. 266 When you click on "Show baselines", you can see the ground truth 267 and baseline audio, etc. CLaM-TTS audios are brought from 268 CLaM-TTS demo<sup><a href="#footnote2-1">1</a></sup>. 269 </p> 270 </div> 271 <div 272 id="celeb-box" 273 role="tabpanel" 274 class="tab-pane fade show active container pt-5 shadow p-5 mb-5 bg-white rounded" 275 > 276 <h2 id="celeb" style="text-align: center">Celebrities</h2> 277 <p> 278 DiTTo-TTS is capable of replicating the voices and speech styles of well-known personalities. 279 We represent 280 <span style="color: #8fbddf; font-weight: bold" 281 >the prompt in blue</span 282 >, and 283 <span style="color: #9fde83; font-weight: bold" 284 >the generated audio in green</span 285 >. The resulting speech closely resembles the reference in 286 voice, background noise, and speaking style. When you click on 287 "Show baselines", you can see the ground truth and baseline 288 audio, etc. Texts corresponding to the generated audio and CLaM-TTS audios are brought from 289 CLaM-TTS demo<sup><a href="#footnote3-1">1</a></sup>. 290 </p> 291 </div> 292 <div 293 id="anime-box" 294 role="tabpanel" 295 class="tab-pane fade container pt-5 shadow p-5 mb-5 bg-white rounded" 296 > 297 <h2 id="anime" style="text-align: center">Anime Characters</h2> 298 <p> 299 DiTTo-TTS is capable of replicating the voices and speech styles of well-known anime characters. 300 We represent 301 <span style="color: #8fbddf; font-weight: bold" 302 >the prompt in blue</span 303 >, and
304 <span style="color: #9fde83; font-weight: bold" 305 >the generated audio in green</span 306 >. The resulting speech closely resembles the reference in 307 voice, background noise, and speaking style. When you click on 308 "Show baselines", you can see the ground truth and baseline 309 audio, etc. Texts corresponding to the generated audio and Mega-TTS audios are brought from 310 Mega-TTS demo<sup><a href="#footnote4-1">1</a></sup>. 311 </p> 312 </div> 313 <div 314 id="robustness-box" 315 role="tabpanel" 316 class="tab-pane fade container pt-5 shadow p-5 mb-5 bg-white rounded" 317 > 318 <h2 id="robustness" style="text-align: center">Robustness</h2> 319 <p> 320 DiTTo-TTS can generate robust speech, as demonstrated by its low WER. 321 Additionally, our model is capable of consistently producing a 322 '<b>whispering</b>' effect (please listen to the first sample). 323 Baseline samples are taken from Mega-TTS demo<sup 324 ><a href="#footnote5-1">1</a></sup 325 > and CLaM-TTS demo<sup 326 ><a href="#footnote5-2">2</a></sup 327 >. 328 </p> 329 <div class="card card-body ditto-card"> 330 <div class="ditto-sample-box"> 331 <div class="table-responsive pt-3"> 332 <table class="table table-hover pt-2"> 333 <thead> 334 <tr> 335 <th 336 style=" 337 width: 500px; 338 max-width: 100%; 339 text-align: center; 340 " 341 > 342 Text 343 </th> 344 <th style="text-align: center">Mega-TTS</th> 345 <th style="text-align: center">CLaM-TTS</th> 346 <th style="text-align: center">DiTTo-TTS</th> 347 </tr> 348 </thead> 349 <tbody> 350 <tr> 351 <td 352 data-label="Text" 353 rowspan="1" 354 style=" 355 text-align: left; 356 vertical-align: middle; 357 max-width: 100%; 358 " 359 > 360 Thursday, via a joint press release and Microsoft 361 speech Blog, we will announce Microsoftâs continued 362 partnership with Shell leveraging cloud, speech, and 363 collaboration technology to drive industry 364 innovation and transformation. 365 </td> 366 <td data-label="Mega-TTS" style="text-align: center"> 367 <audio 368 controls="controls" 369 preload="none" 370 style="width: 140px" 371 > 372 <source src="audios/robust/1_mega.wav" autoplay /> 373 Your browser does not support the audio element. 374 </audio> 375 </td> 376 <td data-label="CLaM-TTS" style="text-align: center"> 377 <audio 378 controls="controls" 379 preload="none" 380 style="width: 140px" 381 > 382 <source src="audios/robust/1_clam.wav" autoplay /> 383 Your browser does not support the audio element. 384 </audio> 385 </td> 386 <td data-label="DiTTo-TTS" style="text-align: center"> 387 <audio 388 controls="controls" 389 preload="none" 390 style="width: 140px" 391 > 392 <source src="audios/robust/1_ours.wav" autoplay /> 393 Your browser does not support the audio element. 394 </audio> 395 </td> 396 </tr> 397 <tr> 398 <td 399 data-label="Text" 400 rowspan="1" 401 style=" 402 text-align: left; 403 vertical-align: middle; 404 max-width: 100%; 405 " 406 > 407 The great Greek grape growers grow great Greek 408 grapes one one one. 409 </td> 410 <td data-label="Mega-TTS" style="text-align: center"> 411 <audio 412 controls="controls" 413 preload="none" 414 style="width: 140px" 415 > 416 <source src="audios/robust/0_mega.wav" autoplay /> 417 Your browser does not support the audio element. 418 </audio> 419 </td> 420 <td data-label="CLaM-TTS" style="text-align: center"> 421 <audio 422 controls="controls" 423 preload="none" 424 style="width: 140px" 425 > 426 <source src="audios/robust/0_clam.wav" autoplay /> 427 Your browser does not support the audio element. 428 </audio> 429 </td> 430 <td data-label="DiTTo-TTS" style="text-align: center"> 431 <audio 432 controls="controls" 433 preload="none" 434 style="width: 140px" 435 > 436 <source src="audios/robust/0_ours.wav" autoplay /> 437 Your browser does not support the audio element. 438 </audio> 439 </td> 440 </tr> 441 </tbody> 442 </table> 443 </div> 444 </div> 445 </div> 446 <br> 447 <br> 448 <h2 id="robustness" style="text-align: center">Comparision Results on RAVDESS Benchmark</h2> 449 <p> 450 The RAVDESS dataset includes two specific textual prompts: âDogs are sitting by the doorâ used for the prompt text, and âKids are talking by the doorâ utilized for the synthesis text. 451 Baseline samples are taken from the NaturalSpeech 3 demo<sup 452 ><a href="#footnote5-3">3</a></sup 453 >. For more details, refer to <b>Section 5.1</b> of our paper. 454 </p> 455 <div class="card card-body ditto-card"> 456 <div class="ditto-sample-box"> 457 <div class="table-responsive pt-3"> 458 <table class="table table-hover pt-2"> 459 <thead> 460 <tr> 461 <th style="text-align: center">Prompt Emotion</th> 462 <th style="text-align: center">Prompt</th> 463 <th style="text-align: center">Ground Truth</th> 464 <th style="text-align: center">DiTTo-TTS</th> 465 <th style="text-align: center">NaturalSpeech 3</th> 466 <th style="text-align: center">NaturalSpeech 2</th> 467 <th style="text-align: center">Voicebox (R)</th> 468 <th style="text-align: center">VALL-E (R)</th> 469 <th style="text-align: center">Mega-TTS 2</th> 470 <th style="text-align: center">StyleTTS 2</th> 471 <th style="text-align: center">HierSpeech++</th> 472 </tr> 473 </thead> 474 <tbody> 475 <tr> 476 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">neutral</td> 477 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_prompt_1.wav">Your browser does not support the audio element.</audio></td> 478 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_gt_1.wav">Your browser does not support the audio element.</audio></td> 479 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/gt/rvd_bm_0_neutral_ours.wav">Your browser does not support the audio element.</audio></td> 480 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns3/rvd_bm_0_neutral.wav">Your browser does not support the audio element.</audio></td> 481 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns2/rvd_bm_0_neutral.wav">Your browser does not support the audio element.</audio></td> 482 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/voiceboxr/rvd_bm_0_neutral.wav">Your browser does not support the audio element.</audio></td> 483 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/valler/rvd_bm_0_neutral.wav">Your browser does not support the audio element.</audio></td> 484 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/megatts2/rvd_bm_0_neutral.wav">Your browser does not support the audio element.</audio></td> 485 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/styletts2/rvd_bm_0_neutral.wav">Your browser does not support the audio element.</audio></td> 486 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/hierspeechpp/rvd_bm_0_neutral.wav">Your browser does not support the audio element.</audio></td> 487 </tr> 488 489 <tr> 490 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">happy</td> 491 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_prompt_2.wav">Your browser does not support the audio element.</audio></td> 492 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_gt_2.wav">Your browser does not support the audio element.</audio></td> 493 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/gt/rvd_bm_1_happy_ours.wav">Your browser does not support the audio element.</audio></td> 494 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns3/rvd_bm_1_happy.wav">Your browser does not support the audio element.</audio></td> 495 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns2/rvd_bm_1_happy.wav">Your browser does not support the audio element.</audio></td> 496 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/voiceboxr/rvd_bm_1_happy.wav">Your browser does not support the audio element.</audio></td> 497 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/valler/rvd_bm_1_happy.wav">Your browser does not support the audio element.</audio></td> 498 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/megatts2/rvd_bm_1_happy.wav">Your browser does not support the audio element.</audio></td> 499 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/styletts2/rvd_bm_1_happy.wav">Your browser does not support the audio element.</audio></td> 500 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/hierspeechpp/rvd_bm_1_happy.wav">Your browser does not support the audio element.</audio></td> 501 </tr> 502 503 <tr> 504 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">calm</td> 505 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_prompt_3.wav">Your browser does not support the audio element.</audio></td> 506 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_gt_3.wav">Your browser does not support the audio element.</audio></td> 507 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/gt/rvd_bm_2_calm_ours.wav">Your browser does not support the audio element.</audio></td> 508 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns3/rvd_bm_2_calm.wav">Your browser does not support the audio element.</audio></td> 509 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns2/rvd_bm_2_calm.wav">Your browser does not support the audio element.</audio></td> 510 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/voiceboxr/rvd_bm_2_calm.wav">Your browser does not support the audio element.</audio></td> 511 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/valler/rvd_bm_2_calm.wav">Your browser does not support the audio element.</audio></td> 512 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/megatts2/rvd_bm_2_calm.wav">Your browser does not support the audio element.</audio></td> 513 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/styletts2/rvd_bm_2_calm.wav">Your browser does not support the audio element.</audio></td> 514 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/hierspeechpp/rvd_bm_2_calm.wav">Your browser does not support the audio element.</audio></td> 515 </tr> 516 517 518 <tr> 519 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">sad</td> 520 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_prompt_4.wav">Your browser does not support the audio element.</audio></td> 521 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_gt_4.wav">Your browser does not support the audio element.</audio></td> 522 <td style="text-align: center">
522<audio controls="controls" style="width: 160px;"><source src="audios/ravdess/gt/rvd_bm_3_sad_ours.wav">Your browser does not support the audio element.</audio></td> 523 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns3/rvd_bm_3_sad.wav">Your browser does not support the audio element.</audio></td> 524 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns2/rvd_bm_3_sad.wav">Your browser does not support the audio element.</audio></td> 525 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/voiceboxr/rvd_bm_3_sad.wav">Your browser does not support the audio element.</audio></td> 526 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/valler/rvd_bm_3_sad.wav">Your browser does not support the audio element.</audio></td> 527 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/megatts2/rvd_bm_3_sad.wav">Your browser does not support the audio element.</audio></td> 528 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/styletts2/rvd_bm_3_sad.wav">Your browser does not support the audio element.</audio></td> 529 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/hierspeechpp/rvd_bm_3_sad.wav">Your browser does not support the audio element.</audio></td> 530 </tr> 531 532 <tr> 533 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">angry</td> 534 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_prompt_5.wav">Your browser does not support the audio element.</audio></td> 535 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_gt_5.wav">Your browser does not support the audio element.</audio></td> 536 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/gt/rvd_bm_4_angry_ours.wav">Your browser does not support the audio element.</audio></td> 537 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns3/rvd_bm_4_angry.wav">Your browser does not support the audio element.</audio></td> 538 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns2/rvd_bm_4_angry.wav">Your browser does not support the audio element.</audio></td> 539 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/voiceboxr/rvd_bm_4_angry.wav">Your browser does not support the audio element.</audio></td> 540 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/valler/rvd_bm_4_angry.wav">Your browser does not support the audio element.</audio></td> 541 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/megatts2/rvd_bm_4_angry.wav">Your browser does not support the audio element.</audio></td> 542 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/styletts2/rvd_bm_4_angry.wav">Your browser does not support the audio element.</audio></td> 543 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/hierspeechpp/rvd_bm_4_angry.wav">Your browser does not support the audio element.</audio></td> 544 </tr> 545 546 <tr> 547 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">fearful</td> 548 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_prompt_6.wav">Your browser does not support the audio element.</audio></td> 549 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_gt_6.wav">Your browser does not support the audio element.</audio></td> 550 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/gt/rvd_bm_5_fearful_ours.wav">Your browser does not support the audio element.</audio></td> 551 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns3/rvd_bm_5_fearful.wav">Your browser does not support the audio element.</audio></td> 552 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns2/rvd_bm_5_fearful.wav">Your browser does not support the audio element.</audio></td> 553 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/voiceboxr/rvd_bm_5_fearful.wav">Your browser does not support the audio element.</audio></td> 554 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/valler/rvd_bm_5_fearful.wav">Your browser does not support the audio element.</audio></td> 555 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/megatts2/rvd_bm_5_fearful.wav">Your browser does not support the audio element.</audio></td> 556 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/styletts2/rvd_bm_5_fearful.wav">Your browser does not support the audio element.</audio></td> 557 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/hierspeechpp/rvd_bm_5_fearful.wav">Your browser does not support the audio element.</audio></td> 558 </tr> 559 560 <tr> 561 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">disgust</td> 562 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_prompt_7.wav">Your browser does not support the audio element.</audio></td> 563 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_gt_7.wav">Your browser does not support the audio element.</audio></td> 564 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/gt/rvd_bm_6_disgust_ours.wav">Your browser does not support the audio element.</audio></td> 565 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns3/rvd_bm_6_disgust.wav">Your browser does not support the audio element.</audio></td> 566 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns2/rvd_bm_6_disgust.wav">Your browser does not support the audio element.</audio></td> 567 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/voiceboxr/rvd_bm_6_disgust.wav">Your browser does not support the audio element.</audio></td> 568 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/valler/rvd_bm_6_disgust.wav">Your browser does not support the audio element.</audio></td> 569 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/megatts2/rvd_bm_6_disgust.wav">Your browser does not support the audio element.</audio></td> 570 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/styletts2/rvd_bm_6_disgust.wav">Your browser does not support the audio element.</audio></td> 571 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/hierspeechpp/rvd_bm_6_disgust.wav">Your browser does not support the audio element.</audio></td> 572 </tr> 573 574 <tr> 575 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">surprised</td> 576 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_prompt_8.wav">Your browser does not support the audio element.</audio></td> 577 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/2_gt_8.wav">Your browser does not support the audio element.</audio></td> 578 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/gt/rvd_bm_7_surprised_ours.wav">Your browser does not support the audio element.</audio></td> 579 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns3/rvd_bm_7_surprised.wav">Your browser does not support the audio element.</audio></td> 580 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/ns2/rvd_bm_7_surprised.wav">Your browser does not support the audio element.</audio></td> 581 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/voiceboxr/rvd_bm_7_surprised.wav">Your browser does not support the audio element.</audio></td> 582 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/valler/rvd_bm_7_surprised.wav">Your browser does not support the audio element.</audio></td> 583 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/megatts2/rvd_bm_7_surprised.wav">Your browser does not support the audio element.</audio></td> 584 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/styletts2/rvd_bm_7_surprised.wav">Your browser does not support the audio element.</audio></td> 585 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/ravdess/hierspeechpp/rvd_bm_7_surprised.wav">Your browser does not support the audio element.</audio></td> 586 </tr> 587 588 </tbody> 589 </table> 590 </div> 591 </div> 592 </div> 593 <br> 594 <br> 595 <h2 id="robustness" style="text-align: center">Zero-Shot TTS Samples (Emotion)</h2> 596 <p> 597 The text âWhy fades the lotus of the waterâ is extracted from the LibriSpeech dataset. 598 Baseline samples are taken from the NaturalSpeech 3 demo<sup 599 ><a href="#footnote5-3">3</a></sup 600 > 601 </p> 602 <div class="card card-body ditto-card"> 603 <div class="ditto-sample-box"> 604 <div class="table-responsive pt-3"> 605 <table class="table table-hover pt-2"> 606 <thead> 607 <tr> 608 <th style="text-align: center">Prompt Emotion</th> 609 <th style="text-align: center">Prompt</th> 610 <th style="text-align: center">DiTTo-TTS</th> 611 <th style="text-align: center">NaturalSpeech 3</th> 612 </tr> 613 </thead> 614 <tbody> 615 <tr> 616 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">neutral</td> 617 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/2_prompt_9.wav">Your browser does not support the audio element.</audio></td> 618 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/zs_emo/zs_emo_0_neutral_ours.wav">Your browser does not support the audio element.</audio></td> 619 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/ns3_zs_emo/zs_emo_0_neutral.wav">Your browser does not support the audio element.</audio></td> 620 </tr> 621 622 <tr> 623 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">happy</td> 624 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/2_prompt_10.wav">Your browser does not support the audio element.</audio></td> 625 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/zs_emo/zs_emo_1_happy_ours.wav">Your browser does not support the audio element.</audio></td> 626 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/ns3_zs_emo/zs_emo_1_happy.wav">Your browser does not support the audio element.</audio></td> 627 </tr> 628 629 <tr> 630 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">calm</td> 631 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/2_prompt_11.wav">Your browser does not support the audio element.</audio></td> 632 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/zs_emo/zs_emo_2_calm_ours.wav">Your browser does not support the audio element.</audio></td> 633 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/ns3_zs_emo/zs_emo_2_calm.wav">Your browser does not support the audio element.</audio></td> 634 </tr> 635 636 <tr> 637 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">sad</td> 638 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/2_prompt_12.wav">Your browser does not support the audio element.</audio></td> 639 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/zs_emo/zs_emo_3_sad_ours.wav">Your browser does not support the audio element.</audio></td> 640 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/ns3_zs_emo/zs_emo_3_sad.wav">Your browser does not support the audio element.</audio></td> 641 </tr> 642 643 <tr> 644 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">angry</td> 645 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/2_prompt_13.wav">Your browser does not support the audio element.</audio></td> 646 <td style="text-align: center"><audio controls="controls" styl
646e="width: 200px;"><source src="audios/ravdess/zs_emo/zs_emo_4_angry_ours.wav">Your browser does not support the audio element.</audio></td> 647 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/ns3_zs_emo/zs_emo_4_angry.wav">Your browser does not support the audio element.</audio></td> 648 </tr> 649 650 <tr> 651 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">fearful</td> 652 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/2_prompt_14.wav">Your browser does not support the audio element.</audio></td> 653 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/zs_emo/zs_emo_5_fearful_ours.wav">Your browser does not support the audio element.</audio></td> 654 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/ns3_zs_emo/zs_emo_5_fearful.wav">Your browser does not support the audio element.</audio></td> 655 </tr> 656 657 <tr> 658 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">disgust</td> 659 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/2_prompt_15.wav">Your browser does not support the audio element.</audio></td> 660 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/zs_emo/zs_emo_6_disgust_ours.wav">Your browser does not support the audio element.</audio></td> 661 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/ns3_zs_emo/zs_emo_6_disgust.wav">Your browser does not support the audio element.</audio></td> 662 </tr> 663 664 <tr> 665 <td style="text-align: center;vertical-align:middle;width: 200px;min-width: 200px">surprised</td> 666 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/2_prompt_16.wav">Your browser does not support the audio element.</audio></td> 667 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/zs_emo/zs_emo_7_surprised_ours.wav">Your browser does not support the audio element.</audio></td> 668 <td style="text-align: center"><audio controls="controls" style="width: 200px;"><source src="audios/ravdess/ns3_zs_emo/zs_emo_7_surprised.wav">Your browser does not support the audio element.</audio></td> 669 </tr> 670 </tbody> 671 </table> 672 </div> 673 </div> 674 </div> 675 676 <sup id="footnote5-1">1</sup 677 ><a href="https://mega-tts.github.io/demo-page/" 678 >https://mega-tts.github.io/demo-page/</a 679 ><br> 680 <sup id="footnote5-2">2</sup 681 ><a href="https://clam-tts.github.io/" 682 >https://clam-tts.github.io/</a 683 ><br> 684 <sup id="footnote5-3">3</sup 685 ><a href="https://speechresearch.github.io/naturalspeech3/" 686 >https://speechresearch.github.io/naturalspeech3/</a 687 > 688 </div> 689 690 <div 691 id="speech-diversity-box" 692 role="tabpanel" 693 class="tab-pane fade container pt-5 shadow p-5 mb-5 bg-white rounded" 694 > 695 <h2 id="speech-diversity" style="text-align: center"> 696 Speech Diversity 697 </h2> 698 <p> 699 DiTTo-TTS can synthesize speech with diverse <b>prosody and timbre</b>, 700 and even <b>environmental background sounds</b>, from <b>various speakers</b>
701 when generated without a prompt. First three text scripts are from Gigaspeech 702 test set, and the last four are brought from 703 Vall-E demo<sup><a href="#footnote6-1">1</a></sup>. 704 </p> 705 <div class="card card-body ditto-card"> 706 <div class="ditto-sample-box"> 707 <div class="table-responsive pt-3"> 708 <table class="table table-hover pt-2"> 709 <thead> 710 <tr> 711 <th>Text</th> 712 <th>Sample #1</th> 713 <th>Sample #2</th> 714 <th>Sample #3</th> 715 <th>Sample #4</th> 716 <th>Sample #5</th> 717 </tr> 718 </thead> 719 <tbody> 720 <tr> 721 <td data-label="Text"> 722 so itâs really for us. itâs kind of like the ah 723 political, social headquarters or center for our 724 community. 725 </td> 726 <td data-label="Sample #1"> 727 <audio 728 controls="controls" 729 preload="none" 730 src="audios/speaker_diversity/so_it_000.wav" 731 ></audio> 732 </td> 733 <td data-label="Sample #2"> 734 <audio 735 controls="controls" 736 preload="none" 737 src="audios/speaker_diversity/so_it_001.wav" 738 ></audio> 739 </td> 740 <td data-label="Sample #3"> 741 <audio 742 controls="controls" 743 preload="none" 744 src="audios/speaker_diversity/so_it_002.wav" 745 ></audio> 746 </td> 747 <td data-label="Sample #4"> 748 <audio 749 controls="controls" 750 preload="none" 751 src="audios/speaker_diversity/so_it_003.wav" 752 ></audio> 753 </td> 754 <td data-label="Sample #5"> 755 <audio 756 controls="controls" 757 preload="none" 758 src="audios/speaker_diversity/so_it_004.wav" 759 ></audio> 760 </td> 761 </tr> 762 <tr> 763 <td data-label="Text"> 764 provided itâs evidence evidence-based and rigorous 765 but, also, i believe, fair. 766 </td> 767 <td data-label="Sample #1"> 768 <audio 769 controls="controls" 770 preload="none" 771 src="audios/speaker_diversity/provi_000.wav" 772 ></audio> 773 </td> 774 <td data-label="Sample #2"> 775 <audio 776 controls="controls" 777 preload="none" 778 src="audios/speaker_diversity/provi_001.wav" 779 ></audio> 780 </td> 781 <td data-label="Sample #3"> 782 <audio 783 controls="controls" 784 preload="none" 785 src="audios/speaker_diversity/provi_002.wav" 786 ></audio> 787 </td> 788 <td data-label="Sample #4"> 789 <audio 790 controls="controls" 791 preload="none" 792 src="audios/speaker_diversity/provi_003.wav" 793 ></audio> 794 </td> 795 <td data-label="Sample #5"> 796 <audio 797 controls="controls" 798 preload="none" 799 src="audios/speaker_diversity/provi_004.wav" 800 ></audio> 801 </td> 802 </tr> 803 <tr> 804 <td data-label="Text"> 805 now you you've been with the office in Northern 806 Ireland for ten years. 807 </td> 808 <td data-label="Sample #1"> 809 <audio 810 controls="controls" 811 preload="none" 812 src="audios/speaker_diversity/now_000.wav" 813 ></audio> 814 </td> 815 <td data-label="Sample #2"> 816 <audio 817 controls="controls" 818 preload="none" 819 src="audios/speaker_diversity/now_001.wav" 820 ></audio> 821 </td> 822 <td data-label="Sample #3"> 823 <audio 824 controls="controls" 825 preload="none" 826 src="audios/speaker_diversity/now_002.wav" 827 ></audio> 828 </td> 829 <td data-label="Sample #4"> 830 <audio 831 controls="controls" 832 preload="none" 833 src="audios/speaker_diversity/now_003.wav" 834 ></audio> 835 </td> 836 <td data-label="Sample #5"> 837 <audio 838 controls="controls" 839 preload="none" 840 src="audios/speaker_diversity/now_004.wav" 841 ></audio> 842 </td> 843 </tr> 844 <tr> 845 <td data-label="Text"> 846 Number ten, fresh nelly is waiting on you, good 847 night husband. 848 </td> 849 <td data-label="Sample #1"> 850 <audio 851 controls="controls" 852 preload="none" 853 src="audios/speaker_diversity/Numbe_000.wav" 854 ></audio> 855 </td> 856 <td data-label="Sample #2"> 857 <audio 858 controls="controls" 859 preload="none" 860 src="audios/speaker_diversity/Numbe_001.wav" 861 ></audio> 862 </td> 863 <td data-label="Sample #3"> 864 <audio 865 controls="controls" 866 preload="none" 867 src="audios/speaker_diversity/Numbe_002.wav" 868 ></audio> 869 </td> 870 <td data-label="Sample #4"> 871 <audio 872 controls="controls" 873 preload="none" 874 src="audios/speaker_diversity/Numbe_003.wav" 875 ></audio> 876 </td> 877 <td data-label="Sample #5"> 878 <audio 879 controls="controls" 880 preload="none" 881 src="audios/speaker_diversity/Numbe_004.wav" 882 ></audio> 883 </td> 884 </tr> 885 <tr> 886 <td data-label="Text">Because we do not need it.</td> 887 <td data-label="Sample #1"> 888 <audio 889 controls="controls" 890 preload="none" 891 src="audios/speaker_diversity/Becau_000.wav" 892 ></audio> 893 </td> 894 <td data-label="Sample #2"> 895 <audio 896 controls="controls" 897 preload="none" 898 src="audios/speaker_diversity/Becau_001.wav" 899 ></audio> 900 </td> 901 <td data-label="Sample #3"> 902 <audio 903 controls="controls" 904 preload="none" 905 src="audios/speaker_diversity/Becau_002.wav" 906 ></audio> 907 </td> 908 <td data-label="Sample #4"> 909 <audio 910 controls="controls" 911 preload="none" 912 src="audios/speaker_diversity/Becau_003.wav" 913 ></audio> 914 </td> 915 <td data-label="Sample #5"> 916 <audio 917 controls="controls" 918 preload="none" 919 src="audios/speaker_diversity/Becau_004.wav" 920 ></audio> 921 </td> 922 </tr> 923 <tr> 924 <td data-label="Text"> 925 I must do something about it. 926 </td> 927 <td data-label="Sample #1"> 928 <audio 929 controls="controls" 930 preload="none" 931 src="audios/speaker_diversity/I_mus_000.wav" 932 ></audio> 933 </td> 934 <td data-label="Sample #2"> 935 <audio 936 controls="controls" 937 preload="none" 938 src="audios/speaker_diversity/I_mus_001.wav" 939 ></audio> 940 </td> 941 <td data-label="Sample #3"> 942 <audio 943 controls="controls" 944 preload="none" 945 src="audios/speaker_diversity/I_mus_002.wav" 946 ></audio> 947 </td> 948 <td data-label="Sample #4"> 949 <audio 950 controls="controls" 951 preload="none" 952 src="audios/speaker_diversity/I_mus_003.wav" 953 ></audio> 954 </td> 955 <td data-label="Sample #5"> 956 <audio 957 controls="controls" 958 preload="none" 959 src="audios/speaker_diversity/I_mus_004.wav" 960 ></audio> 961 </td> 962 </tr> 963 <tr> 964 <td data-label="Text">He has not been named.</td> 965 <td data-label="Sample #1"> 966 <audio 967 controls="controls" 968 preload="none" 969 src="audios/speaker_diversity/He_ha_000.wav" 970 ></audio> 971 </td> 972 <td data-label="Sample #2"> 973 <audio 974 controls="controls" 975 preload="none" 976 src="audios/speaker_diversity/He_ha_001.wav" 977 ></audio> 978 </td> 979 <td data-label="Sample #3"> 980 <audio 981 controls="controls" 982 preload="none" 983 src="audios/speaker_diversity/He_ha_002.wav" 984 ></audio> 985 </td> 986 <td data-label="Sample #4"> 987 <audio 988 controls="controls" 989 preload="none" 990 src="audios/speaker_diversity/He_ha_003.wav" 991 ></audio> 992 </td> 993 <td data-label="Sample #5"> 994 <audio 995 controls="controls" 996 preload="none" 997 src="audios/speaker_diversity/He_ha_004.wav" 998 ></audio> 999 </td> 1000 </tr> 1001 </tbody> 1002 </table> 1003 </div> 1004 </div> 1005 </div> 1006 <sup id="footnote6-1">1</sup 1007 ><a
1008 href="https://www.microsoft.com/en-us/research/project/vall-e-x/vall-e/" 1009 >https://www.microsoft.com/en-us/research/project/vall-e-x/vall-e/</a 1010 > 1011 </div> 1012 <div 1013 id="speech-rate-contrl-box" 1014 role="tabpanel" 1015 class="tab-pane fade container pt-5 shadow p-5 mb-5 bg-white rounded" 1016 > 1017 <h2 id="speech-rate-contrl" style="text-align: center">Speech Rate Controllability</h2> 1018 <p> 1019 DiTTo-TTS can synthesize the same text into audio of varying lengths. 1020 This variable-length modeling allows for speech rate control by adjusting 1021 the total duration of the generated speech, as shown in the following 1022 mel-spectrogram. The speech rate decreases from top to bottom. 1023 For more details, refer to <b>Section 5.2</b> of our paper. 1024 </p> 1025 <p style="text-align: center"> 1026 <img src="asset/speech_rate_control.png" height="400" width="600" 1027 class="img-fluid" /> 1028 </p> 1029 <div class="card card-body ditto-card"> 1030 <div class="ditto-sample-box"> 1031 <div class="table-responsive pt-3"> 1032 <table class="table table-hover pt-2"> 1033 <thead> 1034 <tr> 1035 <th>Text</th> 1036 <th>Length of Latent Audio</th> 1037 <th>Sample</th> 1038 </tr> 1039 </thead> 1040 <tbody> 1041 <tr> 1042 <td data-label="Text" rowspan="4">But to those who knew her well, it was a symbol of <br> her unwavering determination and spirit.</td> 1043 <td data-label="Length">38</td> 1044 <td data-label="Sample"> 1045 <audio controls="controls" preload="none" src="audios/speech_rate/speech_rate_control_0.wav"></audio> 1046 </td> 1047 </tr> 1048 <tr> 1049 <td data-label="Length">43</td> 1050 <td data-label="Sample"> 1051 <audio controls="controls" preload="none" src="audios/speech_rate/speech_rate_control_1.wav"></audio> 1052 </td> 1053 </tr> 1054 <tr> 1055 <td data-label="Length">53</td> 1056 <td data-label="Sample"> 1057 <audio controls="controls" preload="none" src="audios/speech_rate/speech_rate_control_2.wav"></audio> 1058 </td> 1059 </tr> 1060 <tr> 1061 <td data-label="Length">63</td> 1062 <td data-label="Sample"> 1063 <audio controls="controls" preload="none" src="audios/speech_rate/speech_rate_control_3.wav"></audio> 1064 </td> 1065 </tr> 1066 </tbody> 1067 </table> 1068 </div> 1069 </div> 1070 </div> 1071 </div> 1072 <div 1073 id="speech-editing-box" 1074 role="tabpanel" 1075 class="tab-pane fade container pt-5 shadow p-5 mb-5 bg-white rounded" 1076 > 1077 <h2 id="speech-editing" style="text-align: center"> 1078 Speech Editing 1079 </h2> 1080 <p> 1081 DiTTo-TTS is capable of correcting mispronounced words by editing generated speech contents, 1082 without the need for the speaker to re-record the audio. The examples are brought from 1083 Voicebox demo<sup><a href="#footnote7-1">1</a></sup> and 1084 VoiceCraft demo<sup><a href="#footnote7-2">2</a></sup>. 1085 </p> 1086 <div class="card card-body ditto-card"> 1087 <div class="ditto-sample-box"> 1088 <div class="table-responsive pt-3"> 1089 <table class="table table-hover pt-2"> 1090 <thead> 1091 <tr> 1092 <th>Original Text</th> 1093 <th>Original Audio</th> 1094 <th>Edited Text</th> 1095 <th>Voicebox<br>(16 kHz)</th> 1096 <th>VoiceCraft<br>(16 kHz)</th> 1097 <th>DiTTo-TTS<br>(22.05 kHz)</th> 1098 </tr> 1099 </thead> 1100 <tbody> 1101 <tr> 1102 <td data-label="Original Text"> 1103 in zero weather in mid-winter when <b>the earth is frozen to a great depth below the surface</b> when in driving over the unpaved country roads they give forth a hard metallic road 1104 </td> 1105 <td data-label="Original Audio"> 1106 <audio 1107 controls="controls" 1108 preload="none" 1109 src="audios/speech_editing/0.wav" 1110 ></audio> 1111 </td> 1112 <td data-label="Edited Text"> 1113 in zero weather in mid-winter when <b>jack frost has cast his icy spell upon the land</b> when in driving over the unpaved country roads they give forth a hard metallic road 1114 </td> 1115 <td data-label="Voicebox (16 kHz)"> 1116 <audio 1117 controls="controls" 1118 preload="none" 1119 src="audios/speech_editing/0_0.wav" 1120 ></audio> 1121 </td> 1122 <td data-label="VoiceCraft (16 kHz)"> 1123 <audio 1124 controls="controls" 1125 preload="none" 1126 src="audios/speech_editing/0_0_voicecraft.wav" 1127 ></audio> 1128 </td> 1129 <td data-label="DiTTo-TTS (22.05 kHz)"> 1130 <audio 1131 controls="controls" 1132 preload="none" 1133 src="audios/speech_editing/0_ditto_all.wav" 1134 ></audio> 1135 </td> 1136 </tr> 1137 <tr> 1138 <td data-label="Original Text"> 1139 and especially as i am not very much up in latin myself he said the suit was on <b>an insurance policy</b> that he was defending on the ground of misinterpretations 1140 </td> 1141 <td data-label="Original Audio"> 1142 <audio 1143 controls="controls" 1144 preload="none" 1145 src="audios/speech_editing/1.wav" 1146 ></audio> 1147 </td> 1148 <td data-label="Edited Text"> 1149 and especially as i am not very much up in latin myself he said the suit was on <b>a classified treasure map</b> that he was defending on the ground of misinterpretations 1150 </td> 1151 <td data-label="Voicebox (16 kHz)"> 1152 <audio 1153 controls="controls" 1154 preload="none" 1155 src="audios/speech_editing/1_0.wav" 1156 ></audio> 1157 </td> 1158 <td data-label="VoiceCraft (16 kHz)"> 1159 <audio 1160 controls="controls" 1161 preload="none" 1162 src="audios/speech_editing/1_0_voicecraft.wav" 1163 ></audio> 1164 </td> 1165 <td data-label="DiTTo-TTS (22.05 kHz)"> 1166 <audio 1167 controls="controls" 1168 preload="none" 1169 src="audios/speech_editing/1_ditto_all.wav" 1170 ></audio> 1171 </td> 1172 </tr> 1173 <tr> 1174 <td data-label="Original Text"> 1175 yet these petty operations incessantly continued in time surmount the greatest difficulties <b>mountains are elevated and oceans bounded</b> by the slender force of human beings 1176 </td> 1177 <td data-label="Original Audio"> 1178 <audio 1179 controls="controls" 1180 preload="none" 1181 src="audios/speech_editing/2.wav" 1182 ></audio> 1183 </td> 1184 <td data-label="Edited Text"> 1185 yet these petty operations incessantly continued in time surmount the greatest difficulties <b>vast challenges emerge and unexplored frontiers beckon</b> by the slender force of human beings 1186 </td> 1187 <td data-label="Voicebox (16 kHz)"> 1188 <audio 1189 controls="controls" 1190 preload="none" 1191 src="audios/speech_editing/2_0.wav" 1192 ></audio> 1193 </td> 1194 <td data-label="VoiceCraft (16 kHz)"> 1195 <audio 1196 controls="controls" 1197 preload="none" 1198 src="audios/speech_editing/2_0_voicecraft.wav" 1199 ></audio> 1200 </td> 1201 <td data-label="DiTTo-TTS (22.05 kHz)"> 1202 <audio 1203 controls="controls" 1204 preload="none" 1205 src="audios/speech_editing/2_ditto_all.wav" 1206 ></audio> 1207 </td> 1208 </tr> 1209 <tr> 1210 <td data-label="Original Text"> 1211 will find himself completely at a loss on <b>occasions of common and constant recurrence</b> speculative ability is one thing and practical ability is another 1212 </td> 1213 <td data-label="Original Audio"> 1214 <audio 1215 controls="controls" 1216 preload="none" 1217 src="audios/speech_editing/3.wav" 1218 ></audio> 1219 </td> 1220 <td data-label="Edited Text"> 1221 will find himself completely at a loss on <b>rare and unpredictable circumstances</b> speculative ability is one thing and practical ability is another 1222 </td> 1223 <td data-label="Voicebox (16 kHz)"> 1224 <audio 1225 controls="controls" 1226 preload="none" 1227 src="audios/speech_editing/3_0.wav" 1228 ></audio> 1229 </td> 1230 <td data-label="VoiceCraft (16 kHz)"> 1231 <audio 1232 controls="controls" 1233 preload="none" 1234 src="audios/speech_editing/3_0_voicecraft.wav" 1235 ></audio> 1236 </td> 1237 <td data-label="DiTTo-TTS (22.05 kHz)"> 1238 <audio 1239 controls="controls" 1240 preload="none" 1241 src="audios/speech_editing/3_ditto_all.wav" 1242 ></audio> 1243 </td> 1244 </tr> 1245 <tr> 1246 <td data-label="Original Text"> 1247 and <b>the carlsruhe</b> professor had to devise an ingenious apparatus which enabled him to bring the preparation at the required temperature on to the very plate of the microscope 1248 </td> 1249 <td data-label="Original Audio"> 1250 <audio 1251 controls="controls" 1252 preload="none" 1253 src="audios/speech_editing/4.wav" 1254 ></audio> 1255 </td> 1256 <td data-label="Edited Text"> 1257 and <b>the inventive</b> professor had to devise an ingenious apparatus which enabled him to bring the preparation at the required temperature on to the very plate of the microscope 1258 </td> 1259 <td data-label="Voicebox (16 kHz)"> 1260 <audio 1261 controls="controls" 1262 preload="none" 1263 src="audios/speech_editing/4_0.wav" 1264 ></audio> 1265 </td> 1266 <td data-label="VoiceCraft (16 kHz)"> 1267 <audio 1268 controls="controls" 1269 preload="none" 1270 src="audios/speech_editing/4_0_voicecraft.wav" 1271 ></audio> 1272 </td> 1273 <td data-label="DiTTo-TTS (22.05 kHz)"> 1274 <audio 1275 controls="controls" 1276 preload="none" 1277 src="audios/speech_editing/4_ditto_all.wav" 1278 ></audio> 1279 </td> 1280 </tr> 1281 <tr> 1282 <td data-label="Original Text"> 1283 this was george steers the son of a british naval captain and ship modeler who had become an american naval officer and was <b>the first man to take charge of the washington navy yard</b> 1284 </td> 1285 <td data-label="Original Audio"> 1286 <audio 1287 controls="controls" 1288 preload="none" 1289 src="audios/speech_editing/5.wav" 1290 ></audio> 1291 </td> 1292 <td data-label="Edited Text"> 1293 this was george steers the son of a british naval captain and ship modeler who had become an american naval officer and was <b>entrusted with the prestigious role of overseeing the operations at the renowned naval headquarters</b> 1294 </td> 1295 <td data-label="Voicebox (16 kHz)"> 1296 <audio 1297 controls="controls" 1298 preload="none" 1299 src="audios/speech_editing/5_0.wav" 1300 ></audio> 1301 </td> 1302 <td data-label="VoiceCraft (16 kHz)"> 1303 <audio 1304 controls="controls" 1305 preload="none" 1306 src="audios/speech_editing/5_0_voicecraft.wav" 1307 ></audio> 1308 </td> 1309 <td data-label="DiTTo-TTS (22.05 kHz)"> 1310 <audio 1311 controls="controls" 1312 preload="none" 1313 src="audio
1313s/speech_editing/5_ditto_all.wav" 1314 ></audio> 1315 </td> 1316 </tr> 1317 </tbody> 1318 </table> 1319 </div> 1320 </div> 1321 </div> 1322 <sup id="footnote7-1">1</sup 1323 ><a 1324 href="https://voicebox.metademolab.com/edit.html" 1325 >https://voicebox.metademolab.com/edit.html</a 1326 ><br> 1327 <sup id="footnote7-2">2</sup 1328 ><a 1329 href="https://jasonppy.github.io/VoiceCraft_web/" 1330 >https://jasonppy.github.io/VoiceCraft_web/</a 1331 > 1332 </div> 1333 <div 1334 id="compare-box" 1335 role="tabpanel" 1336 class="tab-pane fade container pt-5 shadow p-5 mb-5 bg-white rounded" 1337 > 1338 <h2 id="compare" style="text-align: center">Comparison with Zero-Shot Samples and Demo Cases</h2> 1339 <p> 1340 We provide some comparison samples to compare DiTTo-TTS with four baseline models. 1341 To generate samples from Simple-TTS, we use 1342 <a href="https://github.com/asappresearch/simple-tts">the official checkpoints</a> 1343 and the text scripts from LibriSpeech test-clean subset. For the other three, 1344 E3-TTS, NaturalSpeech 3, and Voicebox, we download the audio samples from the 1345 demo pages of each baseline and then compare them with the audio generated by 1346 our model without any resampling. Demos for each model can be found at their respective links: 1347 <a href="https://e3tts.github.io/">E3-TTS</a>, 1348 <a href="https://speechresearch.github.io/naturalspeech3/">NaturalSpeech 3</a>, and 1349 <a href="https://voicebox.metademolab.com/zs_tts.html">Voicebox</a>. 1350 Please refer to <b>Section 5.1</b> and <b>Appendix A.3</b> of our 1351 paper for more details about subjective evaluation. 1352 The details of each baseline can be found in <b>Appendix A.4</b> of our paper. 1353 </p> 1354 <div class="table-responsive pt-3"> 1355 <!-- <h3 style="text-align: center">Simple-TTS</h3> --> 1356 <table class="table table-hover pt-2"> 1357 <thead> 1358 <tr> 1359 <th style="text-align: center">Text</th> 1360 <th style="text-align: center">Prompt</th> 1361 <th style="text-align: center">Simple-TTS (24 kHz)</th> 1362 <th style="text-align: center">DiTTo-TTS (22.05 kHz)</th> 1363 </tr> 1364 </thead> 1365 <tbody> 1366 <tr> 1367 <td 1368 style="text-align: left;vertical-align:middle;width: 300px"> 1369 do not therefore think that the gothic school is an easy one</td> 1370 <td style="text-align: center"><audio controls="controls" 1371 style="width: 300px;"> 1372 <source src="audios/comparison/simpletts/1188-133604-0014_wav_p_0_wav_p.wav" autoplay="">Your 1373 browser 1374 does not support the audio element. 1375 </audio></td> 1376 <td style="text-align: center"><audio controls="controls" 1377 style="width: 300px;"> 1378 <source src="audios/comparison/simpletts/1188-133604-0014_wav_p_0.wav" 1379 autoplay="">Your 1380 browser does not support the audio element. 1381 </audio></td> 1382 <td style="text-align: center"><audio controls="controls" 1383 style="width: 300px;"> 1384 <source 1385 src="audios/comparison/simpletts/1188-133604-0014_wav_p_0_ours.wav" 1386 autoplay="">Your 1387 browser does not support the audio element. 1388 </audio></td> 1389 </tr> 1390 <tr> 1391 <td 1392 style="text-align: left;vertical-align:middle;width: 300px"> 1393 did ever anybody see the like screamed missus poyser running towards the table when her eye had fallen on the blue stream</td> 1394 <td style="text-align: center"><audio controls="controls" 1395 style="width: 300px;"> 1396 <source src="audios/comparison/simpletts
1396/2094-142345-0029_wav_p_2_wav_p.wav" autoplay="">Your 1397 browser 1398 does not support the audio element. 1399 </audio></td> 1400 <td style="text-align: center"><audio controls="controls" 1401 style="width: 300px;"> 1402 <source src="audios/comparison/simpletts/2094-142345-0029_wav_p_2.wav" 1403 autoplay="">Your 1404 browser does not support the audio element. 1405 </audio></td> 1406 <td style="text-align: center"><audio controls="controls" 1407 style="width: 300px;"> 1408 <source 1409 src="audios/comparison/simpletts/2094-142345-0029_wav_p_2_ours.wav" 1410 autoplay="">Your 1411 browser does not support the audio element. 1412 </audio></td> 1413 </tr> 1414 <tr> 1415 <td 1416 style="text-align: left;vertical-align:middle;width: 300px"> 1417 it is sold everywhere but for the last three weeks nobody will use any snuff but that sold at the civet cat</td> 1418 <td style="text-align: center"><audio controls="controls" 1419 style="width: 300px;"> 1420 <source src="audios/comparison/simpletts/3729-6852-0029_wav_p_1_wav_p.wav" autoplay="">Your 1421 browser 1422 does not support the audio element. 1423 </audio></td> 1424 <td style="text-align: center"><audio controls="controls" 1425 style="width: 300px;"> 1426 <source src="audios/comparison/simpletts/3729-6852-0029_wav_p_1.wav" 1427 autoplay="">Your 1428 browser does not support the audio element. 1429 </audio></td> 1430 <td style="text-align: center"><audio controls="controls" 1431 style="width: 300px;"> 1432 <source 1433 src="audios/comparison/simpletts/3729-6852-0029_wav_p_1_ours.wav" 1434 autoplay="">Your 1435 browser does not support the audio element. 1436 </audio></td> 1437 </tr> 1438 <tr> 1439 <td 1440 style="text-align: left;vertical-align:middle;width: 300px"> 1441 and this was why kenneth and beth discovered him conversing with the young woman in the buggy</td> 1442 <td style="text-align: center"><audio controls="controls" 1443 style="width: 300px;"> 1444 <source src="audios/comparison/simpletts/6829-68771-0006_wav_p_1_wav_p.wav" autoplay="">Your 1445 browser 1446 does not support the audio element. 1447 </audio></td> 1448 <td style="text-align: center"><audio controls="controls" 1449 style="width: 300px;"> 1450 <source src="audios/comparison/simpletts/6829-68771-0006_wav_p_1.wav" 1451 autoplay="">Your 1452 browser does not support the audio element. 1453 </audio></td> 1454 <td style="text-align: center"><audio controls="controls" 1455 style="width: 300px;"> 1456 <source 1457 src="audios/comparison/simpletts/6829-68771-0006_wav_p_1_ours.wav" 1458 autoplay="">Your 1459 browser does not support the audio element. 1460 </audio></td> 1461 </tr> 1462 </tbody> 1463 </table> 1464 </div> 1465 <div class="table-responsive pt-3"> 1466 <!-- <h3 style="text-align: center">E3-TTS</h3> --> 1467 <table class="table table-hover pt-2"> 1468 <thead> 1469 <tr> 1470 <th style="text-align: center">Text</th> 1471 <th style="text-align: center">Prompt</th> 1472 <th style="text-align: center">E3-TTS (24 kHz)</th> 1473 <th style="text-align: center">DiTTo-TTS (22.05 kHz)</th> 1474 </tr> 1475 </thead> 1476 <tbody> 1477 <tr> 1478 <td 1479 style="text-align: left;vertical-align:middle;width: 300px"> 1480 She felt the force of the objections.</td> 1481 <td style="text-align: center"><audio controls="controls" 1482 style="width: 300px;"> 1483 <source src="audios/comparison/e3tts/prompt_02_wav_p.wav" autoplay="">Your 1484 browser 1485 does not support the audio element. 1486 </audio></td> 1487 <td style="text-align: center"><audio controls="controls" 1488 style="width: 300px;"> 1489 <source src="audios/comparison/e3tts/prompt_02.wav" 1490 autoplay="">Your 1491 browser does not support the audio element. 1492 </audio></td> 1493 <td style="text-align: center"><audio controls="controls" 1494 style="width: 300px;"> 1495 <source 1496 src="audios/comparison/e3tts/prompt_02_ours.wav" 1497 autoplay="">Your 1498 browser does not support the audio element. 1499 </audio></td> 1500 </tr> 1501 <tr> 1502 <td 1503 style="text-align: left;vertical-align:middle;width: 300px"> 1504 She can't get it out of her head, even after fifty years.</td> 1505 <td style="text-align: center"><audio controls="controls" 1506 style="width: 300px;"> 1507 <source src="audios/comparison/e3tts/prompt_03_wav_p.wav" autoplay="">Your 1508 browser 1509 does not support the audio element. 1510 </audio></td> 1511 <td style="text-align: center"><audio controls="controls" 1512 style="width: 300px;"> 1513 <source src="audios/comparison/e3tts/prompt_03.wav" 1514 autoplay="">Your 1515 browser does not support the audio element. 1516 </audio></td> 1517 <td style="text-align: center"><audio controls="controls" 1518 style="width: 300px;"> 1519 <source 1520 src="audios/comparison/e3tts/prompt_03_ours.wav" 1521 autoplay="">Your 1522 browser does not support the audio element. 1523 </audio></td> 1524 </tr> 1525 <tr> 1526 <td 1527 style="text-align: left;vertical-align:middle;width: 300px"> 1528 "I don't think so," replied Tom.</td> 1529 <td style="text-align: center"><audio controls="controls" 1530 style="width: 300px;"> 1531 <source src="audios/comparison/e3tts/prompt_13_wav_p.wav" autoplay="">Your 1532 browser 1533 does not support the audio element. 1534 </audio></td> 1535 <td style="text-align: center"><audio controls="controls" 1536 style="width: 300px;"> 1537 <source src="audios/comparison/e3tts/prompt_13.wav" 1538 autoplay="">Your 1539 browser does not support the audio element. 1540 </audio></td> 1541 <td style="text-align: center"><audio controls="controls" 1542 style="width: 300px;"> 1543 <source 1544 src="audios/comparison/e3tts/prompt_13_ours.wav" 1545 autoplay="">Your 1546 browser does not support the audio element. 1547 </audio></td> 1548 </tr> 1549 <tr> 1550 <td 1551 style="text-align: left;vertical-align:middle;width: 300px"> 1552 But at that moment the voice of the stranger was heard from the window.</td> 1553 <td style="text-align: center"><audio controls="controls" 1554 style="width: 300px;"> 1555 <source src="audios/comparison/e3tts/prompt_17_wav_p.wav" autoplay="">Your 1556 browser 1557 does not support the audio element. 1558 </audio></td> 1559 <td style="text-align: center"><audio controls="controls" 1560 style="width: 300px;"> 1561 <source src="audios/comparison/e3tts/prompt_17.wav" 1562 autoplay="">Your 1563 browser does not support the audio element. 1564 </audio></td> 1565 <td style="text-align: center"><audio controls="controls" 1566 style="width: 300px;"> 1567 <source 1568 src="audios/comparison/e3tts/prompt_17_ours.wav" 1569 autoplay="">Your 1570 browser does not support the audio element. 1571 </audio></td> 1572 </tr> 1573 </tbody> 1574 </table> 1575 </div> 1576 <div class="table-responsive pt-3"> 1577 <!-- <h3 style="text-align: center">NaturalSpeech 3</h3> --> 1578 <table class="table table-hover pt-2"> 1579 <thead> 1580 <tr> 1581 <th style="text-align: center">Text</th> 1582 <th style="text-align: center">Prompt</th> 1583 <th style="text-align: center">NaturalSpeech 3 (16 kHz)</th> 1584 <th style="text-align: center">DiTTo-TTS (22.05 kHz)</th> 1585 </tr> 1586 </thead> 1587 <tbody> 1588 <tr> 1589 <td 1590 style="text-align: left;vertical-align:middle;width: 300px"> 1591 They think you're proud because you've been away to school or something.</td> 1592 <td style="text-align: center"><audio controls="controls" 1593 style="width: 300px;">
1594 <source src="audios/comparison/ns3/1_wav_p.wav" autoplay="">Your 1595 browser 1596 does not support the audio element. 1597 </audio></td> 1598 <td style="text-align: center"><audio controls="controls" 1599 style="width: 300px;"> 1600 <source 1601 src="audios/comparison/ns3/1.wav" 1602 autoplay="">Your browser does not support the audio 1603 element. 1604 </audio></td> 1605 <td style="text-align: center"><audio controls="controls" 1606 style="width: 300px;"> 1607 <source 1608 src="audios/comparison/ns3/1_ours.wav" 1609 autoplay="">Your 1610 browser does not support the audio 1611 element. 1612 </audio></td> 1613 </tr> 1614 <tr> 1615 <td 1616 style="text-align: left;vertical-align:middle;width: 300px"> 1617 There were only four stationers of any consequences in the town, and at each Holmes produced his pencil chips, and bid high for a duplicate.</td> 1618 <td style="text-align: center"><audio controls="controls" 1619 style="width: 300px;"> 1620 <source src="audios/comparison/ns3/4_wav_p.wav" autoplay="">Your 1621 browser 1622 does not support the audio element. 1623 </audio></td> 1624 <td style="text-align: center"><audio controls="controls" 1625 style="width: 300px;"> 1626 <source 1627 src="audios/comparison/ns3/4.wav" 1628 autoplay="">Your browser does not support the audio 1629 element. 1630 </audio></td> 1631 <td style="text-align: center"><audio controls="controls" 1632 style="width: 300px;"> 1633 <source 1634 src="audios/comparison/ns3/4_ours.wav" 1635 autoplay="">Your 1636 browser does not support the audio 1637 element. 1638 </audio></td> 1639 </tr> 1640 <tr> 1641 <td 1642 style="text-align: left;vertical-align:middle;width: 300px"> 1643 For, like as not, they must have thought him a prince when they saw his fine cap.</td> 1644 <td style="text-align: center"><audio controls="controls" 1645 style="width: 300px;"> 1646 <source src="audios/comparison/ns3/a_2_wav_p.wav" autoplay="">Your 1647 browser 1648 does not support the audio element. 1649 </audio></td> 1650 <td style="text-align: center"><audio controls="controls" 1651 style="width: 300px;"> 1652 <source 1653 src="audios/comparison/ns3/a_2.wav" 1654 autoplay="">Your browser does not support the audio 1655 element. 1656 </audio></td> 1657 <td style="text-align: center"><audio controls="controls" 1658 style="width: 300px;"> 1659 <source 1660 src="audios/comparison/ns3/a_2_ours.wav" 1661 autoplay="">Your 1662 browser does not support the audio 1663 element. 1664 </audio></td> 1665 </tr> 1666 <tr> 1667 <td 1668 style="text-align: left;vertical-align:middle;width: 300px"> 1669 The strong position held by the Edison system under the strenuous competition that was already springing up was enormously improved by the introduction of the three wire system and it gave an immediate impetus to incandescent lighting.</td> 1670 <td style="text-align: center"><audio controls="controls" 1671 style="width: 300px;"> 1672 <source src="audios/comparison/ns3/a_4_wav_p.wav" autoplay="">Your 1673 browser 1674 does not support the audio element. 1675 </audio></td> 1676 <td style="text-align: center"><audio controls="controls" 1677 style="width: 300px;"> 1678 <source 1679 src="audios/comparison/ns3/a_4.wav" 1680 autoplay="">Your browser does not support the audio 1681 element. 1682 </audio></td> 1683 <td style="text-align: center"><audio controls="controls" 1684 style="width: 300px;"> 1685 <source 1686 src="audios/comparison/ns3/a_4_ours.wav" 1687 autoplay="">Your 1688 browser does not support the audio 1689 element. 1690 </audio></td> 1691 </tr> 1692 </tbody> 1693 </table> 1694 </div> 1695 <div class="table-responsive pt-3"> 1696 <!-- <h3 style="text-align: center">Voicebox</h3> --> 1697 <table class="table table-hover pt-2"> 1698 <thead> 1699 <tr> 1700 <th style="text-align: center">Text</th> 1701 <th style="text-align: center">Prompt</th> 1702 <th style="text-align: center">Voicebox (16 kHz)</th> 1703 <th style="text-align: center">DiTTo-TTS (22.05 kHz)</th> 1704 </tr> 1705 </thead> 1706 <tbody> 1707 <tr> 1708 <td 1709 style="text-align: left;vertical-align:middle;width: 300px"> 1710 He was in deep converse with the clerk and entered the hall holding him by the arm</td> 1711 <td style="text-align: center"><audio controls="controls" 1712 style="width: 300px;"> 1713 <source src="audios/comparison/voicebox/61-70970-0007_wav_p.wav" autoplay="">Your 1714 browser 1715 does not support the audio element. 1716 </audio></td> 1717 <td style="text-align: center"><audio controls="controls" 1718 style="width: 300px;"> 1719 <source 1720 src="audios/comparison/voicebox/61-70970-0007.wav" 1721 autoplay="">Your 1722 browser does not support the audio 1723 element. 1724 </audio></td> 1725 <td style="text-align: center"><audio controls="controls" 1726 style="width: 300px;"> 1727 <source 1728 src="audios/comparison/voicebox/61-70970-0007_ours.wav" 1729 autoplay="">Your 1730 browser does not support the audio 1731 element. 1732 </audio></td> 1733 </tr> 1734 <tr> 1735 <td 1736 style="text-align: left;vertical-align:middle;width: 300px"> 1737 Number ten fresh nelly is waiting on you good night husband</td> 1738 <td style="text-align: center"><audio controls="controls" 1739 style="width: 300px;"> 1740 <source src="audios/comparison/voicebox/1089-134686-0004_wav_p.wav" autoplay="">Your 1741 browser 1742 does not support the audio element. 1743 </audio></td> 1744 <td style="text-align: center"><audio controls="controls" 1745 style="width: 300px;"> 1746 <source 1747 src="audios/comparison/voicebox/1089-134686-0004.wav" 1748 autoplay="">Your 1749 browser does not support the audio 1750 element. 1751 </audio></td> 1752 <td style="text-align: center"><audio controls="controls" 1753 style="width: 300px;"> 1754 <source 1755 src="audios/comparison/voicebox/1089-134686-0004_ours.wav" 1756 autoplay="">Your 1757 browser does not support the audio 1758 element. 1759 </audio></td> 1760 </tr> 1761 <tr> 1762 <td 1763 style="text-align: left;vertical-align:middle;width: 300px"> 1764 Yea his honourable worship is within but he hath a godly minister or tw
1764o with him and likewise a leech</td> 1765 <td style="text-align: center"><audio controls="controls" 1766 style="width: 300px;"> 1767 <source src="audios/comparison/voicebox/1221-135767-0014_wav_p.wav" autoplay="">Your 1768 browser 1769 does not support the audio element. 1770 </audio></td> 1771 <td style="text-align: center"><audio controls="controls" 1772 style="width: 300px;"> 1773 <source 1774 src="audios/comparison/voicebox/1221-135767-0014.wav" 1775 autoplay="">Your 1776 browser does not support the audio 1777 element. 1778 </audio></td> 1779 <td style="text-align: center"><audio controls="controls" 1780 style="width: 300px;"> 1781 <source 1782 src="audios/comparison/voicebox/1221-135767-0014_ours.wav" 1783 autoplay="">Your 1784 browser does not support the audio 1785 element. 1786 </audio></td> 1787 </tr> 1788 <tr> 1789 <td 1790 style="text-align: left;vertical-align:middle;width: 300px"> 1791 Thus did this humane and right minded father comfort his unhappy daughter and her mother embracing her again did all she could to soothe her feelings</td> 1792 <td style="text-align: center"><audio controls="controls" 1793 style="width: 300px;"> 1794 <source src="audios/comparison/voicebox/5639-40744-0020_wav_p.wav" autoplay="">Your 1795 browser 1796 does not support the audio element. 1797 </audio></td> 1798 <td style="text-align: center"><audio controls="controls" 1799 style="width: 300px;"> 1800 <source 1801 src="audios/comparison/voicebox/5639-40744-0020.wav" 1802 autoplay="">Your 1803 browser does not support the audio 1804 element. 1805 </audio></td> 1806 <td style="text-align: center"><audio controls="controls" 1807 style="width: 300px;"> 1808 <source 1809 src="audios/comparison/voicebox/5639-40744-0020_ours.wav" 1810 autoplay="">Your 1811 browser does not support the audio 1812 element. 1813 </audio></td> 1814 </tr> 1815 </tbody> 1816 </table> 1817 </div> 1818 <br> 1819 <br> 1820 <h2 id="compare" style="text-align: center">Comparison with LibriSpeech Zero-Shot Demo Cases</h2> 1821 <p> 1822 Here are some comparison samples to evaluate DiTTo-TTS against four other baseline models: 1823 Mega-TTS, NaturalSpeech 2, VALL-E, and CLaM-TTS. Our samples are downsampled to 16kHz for this section. 1824 Demos for each model can be found at their respective links: 1825 <a href="https://mega-tts.github.io/demo-page/">Mega-TTS</a>, 1826 <a href="https://speechresearch.github.io/naturalspeech2/">NaturalSpeech 2</a>, 1827 <a href="https://www.microsoft.com/en-us/research/project/vall-e-x/vall-e/">VALL-E</a>, and 1828 <a href="https://clam-tts.github.io/">CLaM-TTS</a>. 1829 The details of each baseline can be found in <b>Appendix A.4</b> of our paper. 1830 </p> 1831 <div class="table-responsive pt-3"> 1832 <table class="table table-hover pt-2"> 1833 <thead> 1834 <tr> 1835 <th style="text-align: center">Text</th> 1836 <th style="text-align: center">Prompt</th> 1837 <th style="text-align: center">Ground Truth</th> 1838 <th style="text-align: center">Mega-TTS</th> 1839 <th style="text-align: center">NaturalSpeech 2</th> 1840 <th style="text-align: center">VALL-E</th> 1841 <th style="text-align: center">CLaM-TTS</th> 1842 <th style="text-align: center">DiTTo-TTS</th> 1843 </tr> 1844 </thead> 1845 <tbody> 1846 <tr> 1847 <td style="text-align: left;vertical-align:middle;width: 500px">They moved thereafter cautiously about the hut groping before and about them to find something to show that Warrenton had fulfilled his mission.</td> 1848 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/0.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1849 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/0.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1850 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1851 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1852 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/valle/0.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1853 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/0.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1854 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/0.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1855 </tr> 1856 <tr> 1857 <td style="text-align: left;vertical-align:middle;width: 500px">And lay me down in thy cold bed and leave my shining lot.</td> 1858 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/1.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1859 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/1.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1860 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1861 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/1.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1862 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/valle/1.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1863 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/1.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1864 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/1.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1865 </tr> 1866 <tr> 1867 <td style="text-align: left;vertical-align:middle;width: 500px">Number ten, fresh nelly is waiting on you, good night husband.</td> 1868 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/2.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1869 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/2.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1870 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1871 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1872 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/valle/2.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1873 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/2.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1874 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/2.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1875 </tr> 1876 <tr> 1877 <td style="text-align: left;vertical-align:middle;width: 500px">Yea, his honourable worship is within, but he hath a godly minister or tw
1877o with him, and likewise a leech.</td> 1878 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/3.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1879 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/3.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1880 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/megatts/3.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1881 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/3.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1882 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/valle/3.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1883 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/3.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1884 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/3.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1885 </tr> 1886 <tr> 1887 <td style="text-align: left;vertical-align:middle;width: 500px">Instead of shoes, the old man wore boots with turnover tops, and his blue coat had wide cuffs of gold braid.</td> 1888 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/4.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1889 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/4.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1890 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/megatts/4.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1891 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1892 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/valle/4.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1893 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/4.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1894 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/4.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1895 </tr> 1896 <tr> 1897 <td style="text-align: left;vertical-align:middle;width: 500px">The army found the people in poverty and left them in comparative wealth.</td> 1898 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/5.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1899 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/5.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1900 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/megatts/5.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1901 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/5.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1902 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/valle/5.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1903 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/5.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1904 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/5.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1905 </tr> 1906 <tr> 1907 <td style="text-align: left;vertical-align:middle;width: 500px">Thus did this humane and right minded father comfort his unhappy daughter, and her mother embracing her again, did all she could to soothe her feelings.</td> 1908 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/6.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1909 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/6.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1910 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1911 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/6.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1912 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/valle/6.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1913 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/6.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1914 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/6.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1915 </tr> 1916 <tr> 1917 <td style="text-align: left;vertical-align:middle;width: 500px">He was in deep converse with the clerk and entered the hall holding him by the arm.</td> 1918 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/7.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1919 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/7.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1920 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/megatts/7.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1921 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1922 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/valle/7.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1923 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/7.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1924 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/7.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1925 </tr> 1926 <tr> 1927 <td style="text-align: left;vertical-align:middle;width: 500px">
1927Indeed, there were only one or two strangers who could be admitted among the sisters without producing the same result.</td> 1928 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/8.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1929 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/8.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1930 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1931 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/8.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1932 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1933 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/8.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1934 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/8.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1935 </tr> 1936 <tr> 1937 <td style="text-align: left;vertical-align:middle;width: 500px">For if he's anywhere on the farm, we can send for him in a minute.</td> 1938 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/9.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1939 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/9.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1940 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1941 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/9.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1942 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1943 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/9.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1944 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/9.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1945 </tr> 1946 <tr> 1947 <td style="text-align: left;vertical-align:middle;width: 500px">Their piety would be like their names, like their faces, like their clothes, and it was idle for him to tell himself that their humble and contrite hearts it might be paid a far-richer tribute of devotion than his had ever been. A gift tenfold more acceptable than his elaborate adoration.</td> 1948 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/10.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1949 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/10.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1950 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1951 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/10.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1952 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1953 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/10.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1954 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/10.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1955 </tr> 1956 <tr> 1957 <td style="text-align: left;vertical-align:middle;width: 500px">The air and the earth are curiously mated and intermingled as if the one were the breath of the other.</td> 1958 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/11.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1959 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/11.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1960 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1961 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/11.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1962 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1963 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/11.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1964 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/11.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1965 </tr> 1966 <tr> 1967 <td style="text-align: left;vertical-align:middle;width: 500px">I had always known him to be restless in his manner, but on this particular occasion he was in such a state of uncontrollable agitation that it was clear something very unusual had occurred.</td> 1968 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/12.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1969 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/12.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1970 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1971 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/12.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1972 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1973 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/12.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1974 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/12.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1975 </tr> 1976 <tr> 1977 <td style="text-align: left;vertical-align:middle;width: 500px">His death in this conjuncture was a public misfortune.</td> 1978 <td style="text-align: center"><audio controls="controls" style="width: 160px;">
1978<source src="audios/librispeech_16kHz/prompt/13.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1979 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/13.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1980 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1981 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/13.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1982 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1983 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/13.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1984 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/13.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1985 </tr> 1986 <tr> 1987 <td style="text-align: left;vertical-align:middle;width: 500px">It is this that is of interest to theory of knowledge.</td> 1988 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/14.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1989 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/14.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1990 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1991 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/14.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1992 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 1993 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/14.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1994 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/14.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1995 </tr> 1996 <tr> 1997 <td style="text-align: left;vertical-align:middle;width: 500px">For a few miles, she followed the line hitherto presumably occupied by the coast of Algeria, but no land appeared to the south.</td> 1998 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/prompt/15.wav" autoplay="">Your browser does not support the audio element.</audio></td> 1999 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/gt/15.wav" autoplay="">Your browser does not support the audio element.</audio></td> 2000 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 2001 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ns2/15.wav" autoplay="">Your browser does not support the audio element.</audio></td> 2002 <td style="text-align: center; font-size: 18px; line-height: 2.5; padding-top: 10px; padding-bottom: 10px;">—</td> 2003 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/clam/15.wav" autoplay="">Your browser does not support the audio element.</audio></td> 2004 <td style="text-align: center"><audio controls="controls" style="width: 160px;"><source src="audios/librispeech_16kHz/ours/15.wav" autoplay="">Your browser does not support the audio element.</audio></td> 2005 </tr> 2006 </tbody> 2007 </table> 2008 </div> 2009 </div> 2010 2011 </div> 2012 </div> 2013 </article> 2014 </main> 2015 </div> 2016 <template id="prompt-template"> 2017 <div class="card card-body ditto-card"> 2018 <div> 2019 <p class="ditto-prompt-text"> 2020 <strong>Prompt Text</strong>: <span></span> 2021 </p> 2022 <p class="ditto-text"><strong>Text</strong>: <span></span></p> 2023 </div> 2024 <div class="ditto-audioviz"> 2025 <canvas></canvas> 2026 <audio controls=""></audio> 2027 </div> 2028 <div class="ditto-toggle"> 2029 <div class="show"> 2030 <i class="bi bi-arrow-down-square"></i> 2031 Show baseline(s) 2032 </div> 2033 <div> 2034 <i class="bi bi-arrow-up-square" style="margin-right: 6.34px"></i> 2035 Hide baseline(s) 2036 </div> 2037 </div> 2038 <div class="collapse multi-collapse ditto-sample-box"> 2039 <div class="table-responsive pt-3"> 2040 <table class="table"> 2041 <thead> 2042 <tr> 2043 <th class="border-right">Prompt (Unmodified)</th> 2044 <th>Ground Truth</th> 2045 <th>YourTTS</th> 2046 <th>Vall-E</th> 2047 <th>CLaM-TTS</th> 2048 <th>Mega-TTS</th> 2049 </tr> 2050 </thead> 2051 <tbody> 2052 <tr> 2053 <td class="border-right" data-label="Prompt (Unmodified)"> 2054 <audio controls="controls" preload="none"></audio> 2055 </td> 2056 <td data-label="Ground Truth"> 2057 <audio controls="controls" preload="none"></audio> 2058 </td> 2059 <td data-label="YourTTS"> 2060 <audio controls="controls" preload="none"></audio> 2061 </td> 2062 <td data-label="Vall-E"> 2063 <audio controls="controls" preload="none"></audio> 2064 </td> 2065 <td data-label="CLaM-TTS"> 2066 <audio controls="controls" preload="none"></audio> 2067 </td> 2068 <td data-label="Mega-TTS"> 2069 <audio controls="controls" preload="none"></audio> 2070 </td> 2071 </tr> 2072 </tbody> 2073 </table> 2074 </div> 2075 </div> 2076 </div> 2077 </template> 2078 <div id="fab" class="fab"> 2079 <a class="mainop" href="#myTab"> 2080 <i class="bi bi-arrow-bar-up"></i> 2081 </a> 2082 </div> 2083 </body> 2084</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.