Speech Technology

Learned about https://uberduck.ai/ from https://news.ycombinator.com/item?id=34736745 TTS is really popular these days

It is interesting how quickly people implement ideas. Like the one of podcast transcript with Whisper. Here is a selection https://podscript.ai/ https://podtext.ai/ https://podscription.app/ https://podsearch.page/ Discussion https://news.ycombinator.com/item?id=34727695

CMU pubs are nice. High quality TTS trained from Youtub https://github.com/b04901014/MQTTS https://arxiv.org/abs/2302.04215 A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speech Li-Wei Chen, Shinji Watanabe, Alexander Rudnicky Recent Text-to-Speech (TTS) systems trained on reading or acted corpora have achieved near human-level naturalness. The diversity of human speech, however, often goes beyond the coverage of these corpora. We believe the ability to handle such diversity is crucial for AI systems to achieve human-level communication. Our work explores the use of more abundant real-world data for building speech synthesizers. We train TTS systems using real-world speech from YouTube and podcasts. We observe the mismatch between training and inference alignments in mel-spectrogram based autoregressive models, leading to unintelligible synthesis, and demonstrate that learned discrete codes within multiple code groups effectively resolves this issue. We introduce our MQTTS system whose architecture is designed for multiple code generation and monotonic alignment, along with the use of a clean silence prompt to improve synthesis quality. We conduct ablation analyses to identify the efficacy of our methods. We show that MQTTS outperforms existing TTS systems in several objective and subjective measures.

https://twitter.com/DrJimFan/status/1622276293776793600 Looks like many of you are ready to embrace the Year of Sound Waves! Here’s a big and OPEN dataset for you to get your hands dirty on AI audio modeling: EPIC-SOUNDS, 78k segments of annotated, audible events and actions. Downloadable here: https://epic-kitchens.github.io/epic-sounds/

Similar recent thing from DeepMind https://github.com/deepmind/transformer_grammars

It is interesting that for things like NER for latest research Google returned to structured prediction instead of pure transformers https://github.com/lyutyuh/ASP https://arxiv.org/abs/2210.14698 Autoregressive Structured Prediction with Language Models Tianyu Liu, Yuchen Jiang, Nicholas Monath, Ryan Cotterell, Mrinmaya Sachan Recent years have seen a paradigm shift in NLP towards using pretrained language models ({PLM}) for a wide range of tasks. However, there are many difficult design decisions to represent structures (e.g. tagged text, coreference chains) in a way such that they can be captured by PLMs. Prior work on structured prediction with PLMs typically flattens the structured output into a sequence, which limits the quality of structural information being learned and leads to inferior performance compared to classic discriminative models. In this work, we describe an approach to model structures as sequences of actions in an autoregressive manner with PLMs, allowing in-structure dependencies to be learned without any loss. Our approach achieves the new state-of-the-art on all the structured prediction tasks we looked at, namely, named entity recognition, end-to-end relation extraction, and coreference resolution.

Some Zipformer ideas (multistream is nice): https://github.com/k2-fsa/icefall/issues/837#issuecomment-1412312846

Respected guys https://arxiv.org/abs/2301.13341 Neural Target Speech Extraction: An Overview Katerina Zmolikova, Marc Delcroix, Tsubasa Ochiai, Keisuke Kinoshita, Jan Černocký, Dong Yu Humans can listen to a target speaker even in challenging acoustic conditions that have noise, reverberation, and interfering speakers. This phenomenon is known as the cocktail-party effect. For decades, researchers have focused on approaching the listening ability of humans. One critical issue is handling interfering speakers because the target and non-target speech signals share similar characteristics, complicating their discrimination. Target speech/speaker extraction (TSE) isolates the speech signal of a target speaker from a mixture of several speakers with or without noises and reverberations using clues that identify the speaker in the mixture. Such clues might be a spatial clue indicating the direction of the target speaker, a video of the speaker's lips, or a pre-recorded enrollment utterance from which their voice characteristics can be derived. TSE is an emerging field of research that has received increased attention in recent years because it offers a practical approach to the cocktail-party problem and involves such aspects of signal processing as audio, visual, array processing, and deep learning. This paper focuses on recent neural-based approaches and presents an in-depth overview of TSE. We guide readers through the different major approaches, emphasizing the similarities among frameworks and discussing potential future directions.

For example https://www.youtube.com/watch?v=xKNAa8ihE7g

IWSLT has nice lecture channel too https://www.youtube.com/@sigslt

IWSLT also has many speech translation tracks https://iwslt.org/2023/#shared-tasks

https://sites.google.com/view/merlion-ccs-challenge/ About The inaugural MERLIon CCS Challenge focuses on developing robust language identification and language diarization systems that are reliable for non-standard, accented, spontaneous code-switched, child-directed speech collected via Zoom

https://news.ycombinator.com/item?id=34525936 alice.tar.bz2

speechbrain funded by OVH https://twitter.com/mirco_ravanelli/status/1618345249675542528