Speech Technology

Apparently, 2x speedup for the cuda decoder. Surprising, how fast the algorithm is when correctly implemented. https://github.com/kaldi-asr/kaldi/pull/4811

This is nice!

https://github.com/alibaba-damo-academy/FunASR Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition Zhifu Gao, Shiliang Zhang, Ian McLoughlin, Zhijie Yan Transformers have recently dominated the ASR field. Although able to yield good performance, they involve an autoregressive (AR) decoder to generate tokens one by one, which is computationally inefficient. To speed up inference, non-autoregressive (NAR) methods, e.g. single-step NAR, were designed, to enable parallel generation. However, due to an independence assumption within the output tokens, performance of single-step NAR is inferior to that of AR models, especially with a large-scale corpus. There are two challenges to improving single-step NAR: Firstly to accurately predict the number of output tokens and extract hidden variables; secondly, to enhance modeling of interdependence between output tokens. To tackle both challenges, we propose a fast and accurate parallel transformer, termed Paraformer. This utilizes a continuous integrate-and-fire based predictor to predict the number of tokens and generate hidden variables. A glancing language model (GLM) sampler then generates semantic embeddings to enhance the NAR decoder's ability to model context interdependence. Finally, we design a strategy to generate negative samples for minimum word error rate training to further improve performance. Experiments using the public AISHELL-1, AISHELL-2 benchmark, and an industrial-level 20,000 hour task demonstrate that the proposed Paraformer can attain comparable performance to the state-of-the-art AR transformer, with more than 10x speedup. https://arxiv.org/abs/2206.08317

Some more results on Whisper.TFLITE and Whisper.CPP and Whisper Large-v2 https://alphacephei.com/nsh/2022/12/11/whisper-other.html

The 13th International Symposium on Chinese Spoken Language Processing, Singapore, 11-14 Dec 2022 http://www.iscslp2022.org/ Program here:

Demucs v4 is now on PyPI with HTDemucs now the default model. Use -n mdx_extra_q if you need he old one back, as for some tracks it might still be best. Also added an experimental 6 sources model htdemucs_6s with piano and guitar, although I observe some bleeding + artifacts. https://twitter.com/honualx/status/1600551855972663316

Unfortunately we missed this one: Towards Semi-Supervised and Reinforced Task-Oriented Dialog Systems Co-located with EMNLP 2022 but the topics are pretty interesting http://seretod.org/

What's new in the @OpenAI Whisper's world? Less than 24 hrs ago OpenAI team released the new Whisper large-v2 model 1. The "large-v2" model is trained for more epochs with regularization and shows improved performance compared to the previous large 2. Same architecture as V1 https://twitter.com/ramsri_goutham/status/1600134565074735105

https://github.com/fgnt/meeteval https://arxiv.org/abs/2211.16112 On Word Error Rate Definitions and their Efficient Computation for Multi-Speaker Speech Recognition Systems Thilo von Neumann, Christoph Boeddeker, Keisuke Kinoshita, Marc Delcroix, Reinhold Haeb-Umbach We present a general framework to compute the word error rate (WER) of ASR systems that process recordings containing multiple speakers at their input and that produce multiple output word sequences (MIMO). Such ASR systems are typically required, e.g., for meeting transcription. We provide an efficient implementation based on a dynamic programming search in a multi-dimensional Levenshtein distance tensor under the constraint that a reference utterance must be matched consistently with one hypothesis output. This also results in an efficient implementation of the ORC WER which previously suffered from exponential complexity. We give an overview of commonly used WER definitions for multi-speaker scenarios and show that they are specializations of the above MIMO WER tuned to particular application scenarios. We conclude with a discussion of the pros and cons of the various WER definitions and a recommendation when to use which.

Spiking networks are very energy-efficient and can demonstrate good WER now: https://arxiv.org/abs/2212.01187 Surrogate Gradient Spiking Neural Networks as Encoders for Large Vocabulary Continuous Speech Recognition Alexandre Bittar, Philip N. Garner Compared to conventional artificial neurons that produce dense and real-valued responses, biologically-inspired spiking neurons transmit sparse and binary information, which can also lead to energy-efficient implementations. Recent research has shown that spiking neural networks can be trained like standard recurrent neural networks using the surrogate gradient method. They have shown promising results on speech command recognition tasks. Using the same technique, we show that they are scalable to large vocabulary continuous speech recognition, where they are capable of replacing LSTMs in the encoder with only minor loss of performance. This suggests that they may be applicable to more involved sequence-to-sequence tasks. Moreover, in contrast to their recurrent non-spiking counterparts, they show robustness to exploding gradient problems without the need to use gates.

Another interesting one is ICASSP2023 General Meeting Understanding and Generation Challenge (MUG) https://modelscope.cn/headlines/article/52 Hosted on a new Chinese platform for models and datasets - Modelscope Wenet recently landed on Modelscope too https://modelscope.cn/models/wenet/u2pp_conformer-asr-cn-16k-online/summary

Overall, ICASSP challenges recently announced. The core value of the challenge is usually a dataset https://2023.ieeeicassp.org/signal-processing-grand-challenges/