Accessibility Automatic Speech Recognition Computer-assisted transcription Confidence measures Deep Neural Networks Docencia en Red Education language model adaptation Language Modeling Language Technologies Length modelling Log-linear models Machine Translation Massive Adaptation Models basats en seqüències de paraules Multilingualism Neural Machine Translation Opencast Matterhorn Polimedia Simultaneous Speech Translation Sliding window Speaker adaptation Speech Recognition Speech Translation Statistical machine translation streaming text-to-speech transcripciones video lecture repositories Video Lectures
2026 |
Iranzo-Sánchez, Jorge ; Mas-Mollà, Gerard ; Gimenez, Adrià ; Civera Saiz, Jorge ; Sanchis, Albert ; Juan, Alfons MLLP-VRAIN UPV System for the IWSLT 2026 Simultaneous Speech Translation Task Inproceedings Proc. 23rd Intl. Conf. on Spoken Language Translation (IWSLT 2026), pp. 212–226, San Diego (USA), 2026. Abstract | Links | BibTeX | Tags: Cascade System, Context-Augmented SimulST, Simultaneous Speech Translation @inproceedings{Iranzo-Sánchez2026, title = {MLLP-VRAIN UPV System for the IWSLT 2026 Simultaneous Speech Translation Task}, author = {Iranzo-Sánchez, Jorge AND Mas-Mollà, Gerard AND Gimenez, Adrià AND Civera Saiz, Jorge AND Sanchis, Albert AND Juan, Alfons}, doi = {10.18653/v1/2026.iwslt-1.24}, year = {2026}, date = {2026-01-01}, booktitle = {Proc. 23rd Intl. Conf. on Spoken Language Translation (IWSLT 2026)}, pages = {212--226}, address = {San Diego (USA)}, abstract = {This work describes the participation of the MLLP-VRAIN research group in the shared task of the IWSLT 2026 Simultaneous Speech Translation track. Our submission utilizes the recently released Parakeet and Qwen 3.5 models to create a robust, cascaded solution for long-form SimulST through the use of adaptive black-box policies. We explore relaxations of these policies to achieve better quality-latency trade-offs. Compared to last year, we participate on all language directions. In addition to this, for the En→De, It, Zh directions we also participate in this year’s new context track employing a combination of ASR word-boosting and a RAG mechanism of offline pre-translated exemplars to guide generation and enrich our system with domain-specific context. Finally, we provide a detailed latency analysis of our system. Compared to last year, results on the MCIF En→De test set shows a substantial quality improvement of +5.82 XCOMET-XL. Our context track processing further improves performance by +1.03.}, keywords = {Cascade System, Context-Augmented SimulST, Simultaneous Speech Translation}, pubstate = {published}, tppubtype = {inproceedings} } This work describes the participation of the MLLP-VRAIN research group in the shared task of the IWSLT 2026 Simultaneous Speech Translation track. Our submission utilizes the recently released Parakeet and Qwen 3.5 models to create a robust, cascaded solution for long-form SimulST through the use of adaptive black-box policies. We explore relaxations of these policies to achieve better quality-latency trade-offs. Compared to last year, we participate on all language directions. In addition to this, for the En→De, It, Zh directions we also participate in this year’s new context track employing a combination of ASR word-boosting and a RAG mechanism of offline pre-translated exemplars to guide generation and enrich our system with domain-specific context. Finally, we provide a detailed latency analysis of our system. Compared to last year, results on the MCIF En→De test set shows a substantial quality improvement of +5.82 XCOMET-XL. Our context track processing further improves performance by +1.03. |
Mas Mollà, Gerard ; Sanchis, Albert ; Juan, Alfons Improving streaming ASR with foundation models using emission policies Inproceedings Proc. Interspeech 2026, pp. 4094–4098, Sidney (Australia), 2026. Abstract | Links | BibTeX | Tags: speech foundation models, streaming decoding @inproceedings{Mas2026, title = {Improving streaming ASR with foundation models using emission policies}, author = {Mas Mollà, Gerard AND Sanchis, Albert AND Juan, Alfons}, doi = {10.21437/Interspeech.2026-3358}, year = {2026}, date = {2026-01-01}, booktitle = {Proc. Interspeech 2026}, pages = {4094--4098}, address = {Sidney (Australia)}, abstract = {Speech Foundation Models (SFMs) have achieved state-of-the-art results in offline ASR, yet their application in streaming environments remains limited due to the performance degradation caused by restricted context. In this work, we propose a model-agnostic, training-free streaming pipeline designed to wrap around ASR models in a "black box" manner. Our approach integrates three key components: a sliding window audio buffer for incremental data ingestion, token-level timestamp management to eliminate redundant output, and text-only emission policies to govern output timing. We systematically evaluate several emission policies, assessing their impact on the latency-accuracy trade-off across three English datasets. Results show that our pipeline enables timestamp-producing SFMs like Parakeet and Canary to achieve near-offline transcription quality in real-time environments without requiring access to internal model tensors.}, keywords = {speech foundation models, streaming decoding}, pubstate = {published}, tppubtype = {inproceedings} } Speech Foundation Models (SFMs) have achieved state-of-the-art results in offline ASR, yet their application in streaming environments remains limited due to the performance degradation caused by restricted context. In this work, we propose a model-agnostic, training-free streaming pipeline designed to wrap around ASR models in a "black box" manner. Our approach integrates three key components: a sliding window audio buffer for incremental data ingestion, token-level timestamp management to eliminate redundant output, and text-only emission policies to govern output timing. We systematically evaluate several emission policies, assessing their impact on the latency-accuracy trade-off across three English datasets. Results show that our pipeline enables timestamp-producing SFMs like Parakeet and Canary to achieve near-offline transcription quality in real-time environments without requiring access to internal model tensors. |
2025 |
Iranzo-Sánchez, Jorge; Santamaría-Jordà, Jaume; Mas-Mollà, Gerard; Garcés Díaz-Munío, Gonçal V; Iranzo-Sánchez, Javier; Jorge, Javier; Silvestre-Cerdà, Joan Albert; Giménez, Adrià; Civera, Jorge; Sanchis, Albert; Juan, Alfons Speech Translation for Multilingual Medical Education Leveraged by Large Language Models Journal Article Artificial Intelligence In Medicine, 166 , pp. 103147, 2025. Abstract | Links | BibTeX | Tags: Automatic Speech Recognition, domain adaptation, large language models, Machine Translation, oncology, Speech Translation @article{Iranzo-Sánchez2025, title = {Speech Translation for Multilingual Medical Education Leveraged by Large Language Models}, author = {Jorge Iranzo-Sánchez AND Jaume Santamaría-Jordà AND Gerard Mas-Mollà AND Garcés Díaz-Munío, Gonçal V. AND Javier Iranzo-Sánchez AND Javier Jorge AND Joan Albert Silvestre-Cerdà AND Adrià Giménez AND Jorge Civera AND Albert Sanchis AND Alfons Juan}, doi = {10.1016/j.artmed.2025.103147}, year = {2025}, date = {2025-01-01}, journal = {Artificial Intelligence In Medicine}, volume = {166}, pages = {103147}, abstract = {The application of large language models (LLMs) to speech translation (ST), or in general, to machine translation (MT), has recently provided excellent results superseding conventional encoder-decoder MT systems in the general domain. However, this is not clearly the case when LLMs as MT systems are translating medical-related materials. In this respect, the provision of multilingual training materials for oncology professionals is a goal of the EU project Interact-Europe in which this work was framed. To this end, cross-language technology adapted to the oncology domain was developed, evaluated and deployed for multilingual interspeciality medical education. More precisely, automatic speech recognition (ASR) and MT models were adapted to the oncology domain to translate English pre-recorded training videos, kindly provided by the European School of Oncology (ESO), into French, Spanish, German and Slovene. In this work, three categories of MT models adapted to the medical domain were assessed: bilingual encoder-decoder MT models trained from scratch, pre-trained large multilingual encoder-decoder MT models and multilingual decoder-only LLMs. The experimental results underline the competitiveness in translation quality of LLMs compared to encoder-decoder MT models. Finally, the ESO speech dataset, comprising roughly 1,000 videos and 745 hours for the training and evaluation of ASR and MT models, was publicly released for the scientific community.}, keywords = {Automatic Speech Recognition, domain adaptation, large language models, Machine Translation, oncology, Speech Translation}, pubstate = {published}, tppubtype = {article} } The application of large language models (LLMs) to speech translation (ST), or in general, to machine translation (MT), has recently provided excellent results superseding conventional encoder-decoder MT systems in the general domain. However, this is not clearly the case when LLMs as MT systems are translating medical-related materials. In this respect, the provision of multilingual training materials for oncology professionals is a goal of the EU project Interact-Europe in which this work was framed. To this end, cross-language technology adapted to the oncology domain was developed, evaluated and deployed for multilingual interspeciality medical education. More precisely, automatic speech recognition (ASR) and MT models were adapted to the oncology domain to translate English pre-recorded training videos, kindly provided by the European School of Oncology (ESO), into French, Spanish, German and Slovene. In this work, three categories of MT models adapted to the medical domain were assessed: bilingual encoder-decoder MT models trained from scratch, pre-trained large multilingual encoder-decoder MT models and multilingual decoder-only LLMs. The experimental results underline the competitiveness in translation quality of LLMs compared to encoder-decoder MT models. Finally, the ESO speech dataset, comprising roughly 1,000 videos and 745 hours for the training and evaluation of ASR and MT models, was publicly released for the scientific community. |