Speech Technology

http://www.asru2023.org/ Taiwan December 16-20, 2023

Whisper can actually do speaker diarization with a prompt. Magis is: or do a crude form of speaker turn tracking (e.g. " - Hey how are you doing? - I'm doing good. How are you?", note that the token for " -" is suppressed by default and will need to be enabled manually.) https://github.com/openai/whisper/discussions/117#discussioncomment-3727051

JaX is faster than Pytorch https://twitter.com/sanchitgandhi99/status/1649046661816287236

NaturalSpeech 2, a new powerful zero-shot TTS model in NaturaSpeech series🔥 1. Latent diffusion model + continuous codec, avoiding the dilemma in language model + discrete codec; 2. Strong zero-shot speech synthesis with a 3s prompt, singing synthesis with only a speech prompt! abs: https://arxiv.org/abs/2304.09116 project page: https://speechresearch.github.io/naturalspeech2/

AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models Yuancheng Wang, Zeqian Ju, Xu Tan, Lei He, Zhizheng Wu, Jiang Bian, Sheng Zhao Abstract. Audio editing is applicable for various purposes, such as adding background sound effects, replacing a musical instrument, and repairing damaged audio. Recently, some diffusion-based methods achieved zero-shot audio editing by using a diffusion and denoising process conditioned on the text description of the output audio. However, these methods still have some problems: 1) they have not been trained on editing tasks and cannot ensure good editing effects; 2) they can erroneously modify audio segments that do not require editing; 3) they need a complete description of the output audio, which is not always available or necessary in practical scenarios. In this work, we propose AUDIT, an instruction-guided audio editing model based on latent diffusion models. Specifically, AUDIT has three main design features: 1) we construct triplet training data (instruction, input audio, output audio) for different audio editing tasks and train a diffusion model using instruction and input (to be edited) audio as conditions and generating output (edited) audio; 2) it can automatically learn to only modify segments that need to be edited by comparing the difference between the input and output audio; 3) it only needs edit instructions instead of full target audio descriptions as text input. AUDIT achieves state-of-the-art results in both objective and subjective metrics for several audio editing tasks (e.g., adding, dropping, replacement, inpainting, super-resolution). This research is done in alignment with Microsoft's responsible AI principles. https://audit-demo.github.io/

Not sure about claimed accuracy but numbers are interestinghttps://blog.deepgram.com/nova-speech-to-text-whisper-api/

Space is closer than you think. Happy Cosmonautics day my friends.

GPU beam search in pytorch https://github.com/pytorch/audio/pull/3096

NeMo 1.17 is now released and and includes a lot of improvements that users have long requested. This includes a high level Diarization API, PyCTCDecode support for beam search, InterCTC Loss support, AWS Sagemaker tutorial and more ! https://twitter.com/alphacep/status/1644685634404073472

Learning model from Whisper https://github.com/speechcatcher-asr/speechcatcher

https://groups.inf.ed.ac.uk/edacc/ The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR. Ramon Sanabria, Bogoychev, Markl, Carmantini, Klejch, and Bell. ICASSP 2023. Presentation of the EdAcc.

https://www.openslr.org/136/ EMNS Identifier: SLR136 Summary: An emotive single-speaker dataset for narrative storytelling. EMNS is dataset containing transcriptions, emotion, emotion intensity, and description of acted speech. Category: Speech, text-to-speech, automatic speech recognition License: Apache 2.0 Downloads (use a mirror closer to you): raw_webm.tar.xz [192M] (Unprocessed raw recording ) Mirrors: [US] [EU] [CN] raw_alignment.tar.xz [470K] ( Alignment for raw audio recordings ) Mirrors: [US] [EU] [CN] cleaned_alignment.tar.xz [440K] ( Trimed silance from start and end of recording. ) Mirrors: [US] [EU] [CN] cleaned_webm.tar.xz [42M] ( Alignment for processed audio recordings ) Mirrors: [US] [EU] [CN] metadata.csv [311K] ( Pipe seporated csv, containing transcription, description, emotion, emotion intensity and path to audio recording.) Mirrors: [US] [EU] [CN] About this resource: Emotive Narrative Storytelling (EMNS) corpus introduces a dataset consisting of a single speaker, British English speech with high-quality labelled utterances tailored to drive interactive experiences with dynamic and expressive language. Each audio-text pairs are reviewed for artefacts and quality. Furthermore, we extract critical features using natural language descriptions, including word emphasis, level of expressiveness and emotion. EMNS data collection tool: https://github.com/knoriy/EMNS-DCT EMNS cleaner: https://github.com/knoriy/EMNS-cleaner You can cite the data using the following BibTeX entry: @Unpublished{EMNS_corpus, title={{EMNS /Imz/ Corpus: An emotive single-speaker dataset for narrative storytelling in games, television and graphic novels}}, author={Kari, Noriy and Xiaosong, Yang and Jian, Zhang}, month={march}, year={2023}, }

📢The largest 2,000 hours multi-layer annotated corpus QASR is available @ https://arabicspeech.org/qasr/ QASR is suitable for ASR, dialect ID, punctuation, speaker ID-linking, and potentially other NLP modules for spoken data. #nlproc #speechproc #Arabic #AI @QatarComputing @qcrialt https://twitter.com/ArabicSpeech/status/1641402805951815681

This is interesting, all open source conformer implementations have bugs: 📢 We have just released open source a bug-free 🚫🪲implementation of the Conformer model. 📌Check it at: https://github.com/hlt-mt/FBK-fairseq/blob/master/fbk_works/BUGFREE_CONFORMER.md Want to discover what "bug-free" means? ➡ Take a look at our paper: https://arxiv.org/pdf/2303.16166.pdf #opensource #conformer #speech #bug #bugfree #NLProc https://twitter.com/sarapapi/status/1641750885524029440

WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research Propose a three-stage processing pipeline for filtering noisy data and generating high-quality captions, where ChatGPT. Конвейер обработки для фильтрации зашумленных данных и создания высококачественных титров. 🖥 Github: https://github.com/xinhaomei/wavcaps ⏩ Paper: https://arxiv.org/abs/2303.17395v1 💨 Dataset: https://paperswithcode.com/dataset/sounddescs ai_machinelearning_big_data

12th ISCA Speech Synthesis Workshop (SSW) is now open for submissions! Final submission deadline: May, 3 2023 Late breaking reports submission deadline : June, 28 2023 The Speech Synthesis Workshop will be held in Grenoble, France and is organized as a satellite event of the Interspeech conference in Dublin, Ireland Come and join the SSW community and the people who creates machines that talk! Visit the official site for more information https://ssw2023.org/