Speech Technology

https://twitter.com/reach_vb/status/1599718632551956481 It's the 5th & the Whisper fine-tuning event (powered by @LambdaAPI) is officially a go!! 🚀 As part of the launch, we'll be hosting talks today at 17:30 CET ⚡️ 1. OpenAI's @_jongwook_kim on the Whisper architecture. 2. MetaAI's @ChanghanWang on the VoxPopuli dataset. 🤗 You can join in on the action on our YouTube channel here: https://www.youtube.com/watch?v=fZMiD8sDzzg

https://twitter.com/hbredin/status/1598679504783872003 "How I reached 1st place at Ego4D 2022, 1st place at Albayzin 2022, and 6th place at VoxSRC 2022 speaker diarization challenges" 😎 Short answer: with 🎹#pyannote.audi

While authors are greatly respected, the unsupervised task statement is not very meaningful. It is not a problem to point to few phone samples or even provide some transcripts. And pure unsupervised learning is not accurate at all (expected). But there is the second problem - authors use wav2vec or wavlm which alreary learned on phoneme samples so you can not claim they are unsupervised anymore. No justification to compare wav2vec results to pure unsupervised methods.

There are papers like https://arxiv.org/abs/2211.17196 EURO: ESPnet Unsupervised ASR Open-source Toolkit Dongji Gao, Jiatong Shi, Shun-Po Chuang, Leibny Paola Garcia, Hung-yi Lee, Shinji Watanabe, Sanjeev Khudanpur This paper describes the ESPnet Unsupervised ASR Open-source Toolkit (EURO), an end-to-end open-source toolkit for unsupervised automatic speech recognition (UASR). EURO adopts the state-of-the-art UASR learning method introduced by the Wav2vec-U, originally implemented at FAIRSEQ, which leverages self-supervised speech representations and adversarial training. In addition to wav2vec2, EURO extends the functionality and promotes reproducibility for UASR tasks by integrating S3PRL and k2, resulting in flexible frontends from 27 self-supervised models and various graph-based decoding strategies. EURO is implemented in ESPnet and follows its unified pipeline to provide UASR recipes with a complete setup. This improves the pipeline's efficiency and allows EURO to be easily applied to existing datasets in ESPnet. Extensive experiments on three mainstream self-supervised models demonstrate the toolkit's effectiveness and achieve state-of-the-art UASR performance on TIMIT and LibriSpeech datasets. EURO will be publicly available at this https URL, aiming to promote this exciting and emerging research area based on UASR through open-source activity.

Puffin: pitch-synchronous neural waveform generation for fullband speech on modest devices Twenty times faster than HifiGAN https://arxiv.org/abs/2211.14130

New data on openslr https://www.openslr.org/132/ Summary: Arabic speech to text Quran data (24Gb)

Multimodal Information Based Speech Processing (MISP) Challenge 2022 November 15th, 2022: The download link of training set and development set has been sent to the registered participants via email. November 21st, 2022: The MISP 2022 Challenge has been accepted by ICASSP 2023 Signal Processing Grand Challenge. Please refer to https://2023.ieeeicassp.org/call-for-sp-grand-challenges for more details of ICASSP 2023 SPGC. https://mispchallenge.github.io/mispchallenge2022/

Repo https://github.com/prophesier/diff-svc

Today I'm showing off Naoki for Diff-SVC singing in Chinese! How is it? I'll continue to show off Naoki singing in different languages, including languages he doesn't have data for. https://twitter.com/uchuuzentai/status/1595771721113559041

Progress in next-gen-Kaldi team (in mixed Chinese/English) The following is the content of Daniel Povey's report at the 2nd SH Speech Technology Symposium that we compiled, because it is a translation. If there is any error, please correct me. https://mp.weixin.qq.com/s/b7qy4RnociKL1L91l-x6JQ

Ycombinator discussion on the same subject: https://news.ycombinator.com/item?id=33700792

In April 2022 Google released Conformer models in Cloud API https://cloud.google.com/speech-to-text/docs/latest-models. On Rev dataset https://rev.com/blog/transcription-blog/automatic-speech-benchmarking doesn't show much improvement, video model from 2019 is at 17.863% WER, new model 17.58% WER. Speech with music still dropped. Whisper on this set at 11%.

Details of the best system http://catedrartve.unizar.es/reto2022/IberSPEECH_2022_paper_80.pdf overall results http://catedrartve.unizar.es/albayzin2022results.html

https://twitter.com/ButSpeech/status/1594713688438706176 We are happy to announce the best system in 2022 Albayzín Evaluations organized by Spanish thematic network on speech technologies (RTTH). Our system was a massive "ASR hammer" - a fusion of several systems trained either from scratch or based on pre-trained models. Interesing name - wav2vecrnnt_kaldi_whisper. Pure Whisper is not that far though at 14.87

Google's paper demonstrates importance of merging ASR aspects into unified model (speaker ids would be nice here too and punctation (whisper-like)) https://arxiv.org/abs/2211.00786 Unified End-to-End Speech Recognition and Endpointing for Fast and Efficient Speech Systems Shaan Bijwadia, Shuo-yiin Chang, Bo Li, Tara Sainath, Chao Zhang, Yanzhang He Automatic speech recognition (ASR) systems typically rely on an external endpointer (EP) model to identify speech boundaries. In this work, we propose a method to jointly train the ASR and EP tasks in a single end-to-end (E2E) multitask model, improving EP quality by optionally leveraging information from the ASR audio encoder. We introduce a "switch" connection, which trains the EP to consume either the audio frames directly or low-level latent representations from the ASR model. This results in a single E2E model that can be used during inference to perform frame filtering at low cost, and also make high quality end-of-query (EOQ) predictions based on ongoing ASR computation. We present results on a voice search test set showing that, compared to separate single-task models, this approach reduces median endpoint latency by 120 ms (30.8% reduction), and 90th percentile latency by 170 ms (23.0% reduction), without regressing word error rate. For continuous recognition, WER improves by 10.6% (relative).

Microsoft take on Whisper https://arxiv.org/abs/2211.02499 A Weakly-Supervised Streaming Multilingual Speech Model with Truly Zero-Shot Capability Jian Xue, Peidong Wang, Jinyu Li, Eric Sun In this paper, we introduce our work of building a Streaming Multilingual Speech Model (SM2), which can transcribe or translate multiple spoken languages into texts of the target language. The backbone of SM2 is Transformer Transducer, which has high streaming capability. Instead of human labeled speech translation (ST) data, SM2 models are trained using weakly supervised data generated by converting the transcriptions in speech recognition corpora with a machine translation service. With 351 thousand hours of anonymized speech training data from 25 languages, SM2 models achieve comparable or even better ST quality than some recent popular large-scale non-streaming speech models. More importantly, we show that SM2 has the truly zero-shot capability when expanding to new target languages, yielding high quality ST results for {source-speech, target-text} pairs that are not seen during training.