The MLLP has obtained competitive results in the IWSLT 2015 evaluation campaign on speech recognition, which were presented on 3–4 December in Da Nang, Vietnam. The MLLP’s ASR results were the only ones based completely on internally developed technology (the TLK toolkit for speech recognition). See here the full results of the IWSLT 2015 EN ASR evaluation campaign.
The International Workshop on Spoken Language Translation (IWSLT) is a scientific workshop, associated with an open evaluation campaign on spoken language translation, where both scientific papers and system descriptions are presented. Its 2015 edition took place in Da Nang, Vietnam, on 3–4 December 2015.
The Universitat Politècnica de València’s MLLP group has participated in IWSLT 2015′s evaluation campaign on speech recognition, with the paper “The MLLP ASR Systems for IWSLT 2015″, by MLLP members Miguel Ángel del Agua, Adrià Martínez-Villaronga, Santiago Piqueras, Adrià Giménez, Alberto Sanchis, Jorge Civera and Alfons Juan. In this paper, the authors present ASR results obtained with the MLLP’s internally developed transLectures-UPV toolkit for speech recognition (TLK).
The first MLLP submission to an IWSLT evaluation campaign, the MLLP’s IWSLT 2015 system for English speech recognition, has obtained a Word Error Rate (WER)1 of 13.3 on the IWSLT 2015 test set. These results sparked congratulations from other participants. From a total of six participating research groups worldwide, the MLLP was the only one to take part in the EN ASR evaluation campaign with results based completely on the use of internally developed ASR software (TLK).
Results of the IWSLT 2015 evaluation campaign on English speech recognition
| Participant | ASR software | Training corpus size (hours) | WER1 (test 2015) |
|---|---|---|---|
| MITLL–AFRL (USA) | KALDI + HTK | 336 | 6.6 |
| HLT-I2R (Singapore) | KALDI | 486 | 8.9 |
| KIT (Germany) | KALDI + JANUS | 579 | 9.2 |
| NAIST (Japan) | KALDI | 439 | 12.0 |
| MLLP (Spain) | TLK | 245 | 13.3 |
| IOIT (Vietnam) | KALDI | 520 | 13.8 |
Source: Proc. of 12th Intl. Workshop on Spoken Language Translation (IWSLT 2015)
You can read the details on the MLLP’s IWSLT 2015 ASR systems for the evaluation campaign in the IWSLT 2015 proceedings (pp. 39–44); you will also find there the details on the rest of the participating systems. For a quick summary of the MLLP’s ASR systems, have a look at the poster below.
For an overview of this year’s IWSLT 2015 (in the context of past editions), we recommend having a look at the slides that were used to present the “Report on the 12th IWSLT Evaluation Campaign, IWSLT 2015”.
We at the MLLP would like to thank the IWSLT 2015 organizers for the chance of participating in this year’s workshop. We look forward to future IWSLT editions, and will keep you updated here on further participation by the MLLP.
“The MLLP ASR Systems for IWSLT 2015″ poster (Full paper in IWSLT 2015 proceedings, pp. 39–44)
Adrià Martínez-Villaronga presented the MLLP’s paper “The MLLP ASR Systems for IWSLT 2015″ at IWSLT 2015 (Picture source: MLLP)
Note 1. ^ Word Error Rate (WER) is the ratio, expressed as a percentage, of the number of basic word editing operations required to convert an automatic transcription into the reference (correct) transcription, and the total number of words in the reference transcription. The lower the WER, the higher the ASR accuracy.