Accessibility Automatic Speech Recognition Computer-assisted transcription Confidence measures Deep Neural Networks Docencia en Red Education language model adaptation Language Modeling Language Technologies Length modelling Log-linear models Machine Translation Massive Adaptation Models basats en seqüències de paraules Multilingualism Neural Machine Translation Opencast Matterhorn Polimedia Simultaneous Speech Translation Sliding window Speaker adaptation Speech Recognition Speech Translation Statistical machine translation streaming text-to-speech transcripciones video lecture repositories Video Lectures
2025 |
Iranzo-Sánchez, Jorge; Iranzo-Sánchez, Javier; Giménez, Adrià; Civera, Jorge Going Beyond Your Expectations in Latency Metrics for Simultaneous Speech Translation Inproceedings ACL (Findings) 2025, pp. 18205–18228, Vienna (Austria), 2025. Abstract | Links | BibTeX | Tags: latency metrics, Simultaneous Speech Translation @inproceedings{Iranzo-SánchezACL2025, title = {Going Beyond Your Expectations in Latency Metrics for Simultaneous Speech Translation}, author = {Jorge Iranzo-Sánchez AND Javier Iranzo-Sánchez AND Adrià Giménez AND Jorge Civera}, url = {https://www.mllp.upv.es/wp-content/uploads/2025/08/poster.pdf https://openreview.net/forum?id=mbNv6ne53X}, doi = {10.18653/v1/2025.findings-acl.937}, year = {2025}, date = {2025-01-01}, booktitle = {ACL (Findings) 2025}, pages = {18205--18228}, address = {Vienna (Austria)}, abstract = {Current evaluation practices in Simultaneous Speech Translation (SimulST) systems typically involve segmenting the input audio and corresponding translations, calculating quality and latency metrics for each segment, and averaging the results. Although this approach may provide a reliable estimation of translation quality, it can lead to misleading values of latency metrics due to an inherent assumption that average latency values are good enough estimators of SimulST systems' response time. However, our detailed analysis of latency evaluations for state-of-the-art SimulST systems demonstrates that latency distributions are often skewed and subject to extreme variations. As a result, the mean in latency metrics fails to capture these anomalies, potentially masking the lack of robustness in some systems and metrics. In this paper, a thorough analysis of the results of systems submitted to recent editions of the IWSLT simultaneous track is provided to support our hypothesis and alternative ways to report latency metrics are proposed in order to provide a better understanding of SimulST systems' latency.}, keywords = {latency metrics, Simultaneous Speech Translation}, pubstate = {published}, tppubtype = {inproceedings} } Current evaluation practices in Simultaneous Speech Translation (SimulST) systems typically involve segmenting the input audio and corresponding translations, calculating quality and latency metrics for each segment, and averaging the results. Although this approach may provide a reliable estimation of translation quality, it can lead to misleading values of latency metrics due to an inherent assumption that average latency values are good enough estimators of SimulST systems' response time. However, our detailed analysis of latency evaluations for state-of-the-art SimulST systems demonstrates that latency distributions are often skewed and subject to extreme variations. As a result, the mean in latency metrics fails to capture these anomalies, potentially masking the lack of robustness in some systems and metrics. In this paper, a thorough analysis of the results of systems submitted to recent editions of the IWSLT simultaneous track is provided to support our hypothesis and alternative ways to report latency metrics are proposed in order to provide a better understanding of SimulST systems' latency. |
Iranzo-Sánchez, Jorge ; Iranzo-Sánchez, Javier ; Giménez, Adrià ; Civera, Jorge ; Juan, Alfons MLLP-VRAIN UPV system for the IWSLT 2025 Simultaneous Speech Translation Translation Task Inproceedings IWSLT 2025, pp. 340–346, Vienna (Austria), 2025. Abstract | Links | BibTeX | Tags: Simultaneous Speech Translation @inproceedings{Iranzo-Sánchez2025b, title = {MLLP-VRAIN UPV system for the IWSLT 2025 Simultaneous Speech Translation Translation Task}, author = {Iranzo-Sánchez, Jorge AND Iranzo-Sánchez, Javier AND Giménez, Adrià AND Civera, Jorge AND Juan, Alfons}, url = {https://arxiv.org/pdf/2506.18828}, doi = {10.18653/v1/2025.iwslt-1.35}, year = {2025}, date = {2025-01-01}, booktitle = {IWSLT 2025}, pages = {340--346}, address = {Vienna (Austria)}, abstract = {This work describes the participation of the MLLP-VRAIN research group in the shared task of the IWSLT 2025 Simultaneous Speech Translation track. Our submission addresses the unique challenges of real-time translation of long-form speech by developing a modular cascade system that adapts strong pre-trained models to streaming scenarios. We combine Whisper Large-V3-Turbo for ASR with the multilingual NLLB-3.3B model for MT, implementing lightweight adaptation techniques rather than training new end-to-end models from scratch. Our approach employs document-level adaptation with prefix training to enhance the MT model's ability to handle incomplete inputs, while incorporating adaptive emission policies including a wait-k strategy and RALCP for managing the translation stream. Specialized buffer management techniques and segmentation strategies ensure coherent translations across long audio sequences. Experimental results on the ACL60/60 dataset demonstrate that our system achieves a favorable balance between translation quality and latency, with a BLEU score of 31.96 and non-computational-aware StreamLAAL latency of 2.94 seconds. Our final model achieves a preliminary score on the official test set (IWSLT25Instruct) of 29.8 BLEU. Our work demonstrates that carefully adapted pre-trained components can create effective simultaneous translation systems for long-form content without requiring extensive in-domain parallel data or specialized end-to-end training.}, keywords = {Simultaneous Speech Translation}, pubstate = {published}, tppubtype = {inproceedings} } This work describes the participation of the MLLP-VRAIN research group in the shared task of the IWSLT 2025 Simultaneous Speech Translation track. Our submission addresses the unique challenges of real-time translation of long-form speech by developing a modular cascade system that adapts strong pre-trained models to streaming scenarios. We combine Whisper Large-V3-Turbo for ASR with the multilingual NLLB-3.3B model for MT, implementing lightweight adaptation techniques rather than training new end-to-end models from scratch. Our approach employs document-level adaptation with prefix training to enhance the MT model's ability to handle incomplete inputs, while incorporating adaptive emission policies including a wait-k strategy and RALCP for managing the translation stream. Specialized buffer management techniques and segmentation strategies ensure coherent translations across long audio sequences. Experimental results on the ACL60/60 dataset demonstrate that our system achieves a favorable balance between translation quality and latency, with a BLEU score of 31.96 and non-computational-aware StreamLAAL latency of 2.94 seconds. Our final model achieves a preliminary score on the official test set (IWSLT25Instruct) of 29.8 BLEU. Our work demonstrates that carefully adapted pre-trained components can create effective simultaneous translation systems for long-form content without requiring extensive in-domain parallel data or specialized end-to-end training. |