Speech Technology

🚨 🔔: We've just released our GitHub repository for #ASR and #NLP tools for air traffic control communications, based on ATCO2 dataset @Atco2P ! We made public 5000+ hours of audio --> research on ASR for ATC. GitHub: https://github.com/idiap/atco2-corpus https://twitter.com/Pablogomez3/status/1640331512389279744

The amount of models this guy trained is quite outstanding https://malaya-speech.readthedocs.io/en/latest/index.html

Streaming punctuation model is interesting https://github.com/alibaba-damo-academy/FunASR/releases/tag/v0.3.0

Kincaid46 WER from Ursa announcement: AssemblyAI: 8.6 Speechmatics: 7.88 Microsoft: 9.70 Whisper Large-v2: 8.7

New model from Assembly AI. Definitely improved from before, but not as great as Speechmatics. On a toy test WER 10.89, previous assemblyAI (version 9) was at 11.04, version before 11.89. Speechmatics 6.88. Whisper large 8.94 https://twitter.com/AssemblyAI/status/1636050346240884744 Introducing Conformer-1: our latest state-of-the-art speech recognition model. Built on top of the Conformer architecture and trained on 650K hours of audio data, it achieves near-human-level performance, making up to 43% fewer errors on noisy data than other ASR models. We use a modified version of the conformer neural net published by Google Brain. It's built on top of an Efficient Conformer (Orange Labs, 2021), that introduces the following technical modifications: - Progressive Downsampling to reduce the length of the encoded sequence - Grouped Attention: A modified version of the attention mechanism that makes it agnostic to sequence-length These changes yield speedups of 29% at inference time and 36% at training time. To further improve our model’s accuracy on noisy audio, we implemented a modified version of Sparse Attention, a pruning method for achieving sparsity of the model’s weights in order to achieve regularization. We took inspiration from the data scaling laws described in DeepMind's Chinchilla paper and adapted them to the ASR domain. Our team curated a dataset of 650K hours of English audio - making our model the largest-trained supervised model for English available today. Based on our results, Conformer-1 is more robust on real-world data than popular commercial and open-source ASR models, making up to 43% fewer errors on average on noisy data: The biggest improvement with this new release is in our robustness to a wide variety of data domains and noisy audio.

Paraformer released models for other languages too: We release several new UniASR model: Southern Fujian Dialect model, French model, German model, Vietnamese model, Persian model. https://github.com/alibaba-damo-academy/FunASR

BUT 3rd System description https://www.fit.vutbr.cz/research/groups/speech/publi/2022/NIST_LRE_2022_System_Description.pdf

https://twitter.com/TanelAlumae/status/1635221485060227072 https://arxiv.org/abs/2302.14624

Tried a popular https://github.com/Kyubyong/g2p. As usual, networks are very bad for unseen cases. Missing letters, extra letters, etc. Watch outputs carefully. Example: bio-sand B AY1 OW0 S T AE2 N D

https://t.me/speechtech/1449 Repeated this test with new Speechmatics. Async WER improved to from 6.88. Indeed new Ursa model improved significantly!

faster whisper much faster than whisper.cpp https://github.com/ggerganov/whisper.cpp/discussions/589

12m hours of speech data https://arxiv.org/abs/2303.01037 Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages Yu Zhang, Wei Han, James Qin, Yongqiang Wang, Ankur Bapna, Zhehuai Chen, Nanxin Chen, Bo Li, Vera Axelrod, Gary Wang, Zhong Meng, Ke Hu, Andrew Rosenberg, Rohit Prabhavalkar, Daniel S. Park, Parisa Haghani, Jason Riesa, Ginger Perng, Hagen Soltau, Trevor Strohman, Bhuvana Ramabhadran, Tara Sainath, Pedro Moreno, Chung-Cheng Chiu, Johan Schalkwyk, Françoise Beaufays, Yonghui Wu We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages. This is achieved by pre-training the encoder of the model on a large unlabeled multilingual dataset of 12 million (M) hours spanning over 300 languages, and fine-tuning on a smaller labeled dataset. We use multilingual pre-training with random-projection quantization and speech-text modality matching to achieve state-of-the-art performance on downstream multilingual ASR and speech-to-text translation tasks. We also demonstrate that despite using a labeled training set 1/7-th the size of that used for the Whisper model, our model exhibits comparable or better performance on both in-domain and out-of-domain speech recognition tasks across many languages.

Nice Chinese chip with analog NPU (very power-efficient) and RISC core https://en.witmem.com/wtm2101.html