Speech Technology

https://www.reddit.com/r/homeassistant/comments/1vhofep/hacked_and_debloated_an_echo_dot_2_local_llm/

There is a big interest in full duplex as I see, here is a nice collection of papers https://github.com/Ruiqi-Yan/Awesome-Full-Duplex-SDM

https://huggingface.co/nvidia/NVIDIA-NemotronLabs-VoiceChat-11B NVIDIA NemotronLabs VoiceChat is a 11B end-to-end, real-time speech full duplex (FD) model for conversational AI that jointly performs streaming speech understanding and speech generation [1, 2]. Unlike traditional cascaded stacks (ASR → LLM → TTS), this model achieves full duplex, real-time, seamless voice interaction in one unified architecture, elimi…

Things move on in OpenAI as well. Interesting that voice model is separate. And no turn detector anymore. https://x.com/OpenAI/status/2084378415818579975 Lots of interesting technical details, from realtime inference, to dynamic compaction, to WebRTC optimization. https://openai.com/index/continuous-voice-interaction-with-gpt-live/

One more https://github.com/AnXMuy/AgenticASR

https://huckiyang.github.io/voice-memory/ from NVIDIA https://arxiv.org/abs/2607.26410 Voice Memory for Agentic Speech Recognition Chao-Han Huck Yang, Zih-Ching Chen, Piotr Zelasko, Zhehuai Chen, Jagadeesh Balam, Boris Ginsburg We present Voice Memory, a inference-only scheme for agentic speech recognition: at stream time, a frozen corrector reads a single per-domain this http URL and decides per utterance whether…

https://huggingface.co/nyralabs/CrisperWhisper2.0_large Most speech-to-text systems never actually decide whether to write down what was said or what was meant. They inherit that choice from their training data and apply it inconsistently. CrisperWhisper 2.0 makes it an explicit, controllable choice. One recording, two transcripts: Verbatim, exactly what was said, in one consistent format: [um] so we we need to, to …

Some recent Uzbek things https://huggingface.co/datasets/k2speech/FeruzaSpeech - single speaker 40 hours TTS dataset https://huggingface.co/collections/navai-uz/navai-whisper-collection - recently trained Whisper models from Navai https://navai.pro https://huggingface.co/instinct-org/collections - some loosely organized data https://huggingface.co/datasets/OvozifyLabs/asr_evaluate_set - evaluation dataset with Te…

Everyone builds self-improvement loops in LLMs, I wonder how they could look like in ASR/TTS. Not many publications on that yet.

We compared three LALM judges against a calibrated human panel across 15 dimensions of speech quality. The LALMs tracked humans closely on relevance, answer quality, and instruction following—what was said—but were much less reliable on naturalness, emotion, pronunciation, and overall feel—how it was said. https://research.withdavid.ai/blog/lalm-as-judge-vs-hitl

Interesting math on speech LLM https://arxiv.org/abs/2604.08003v1 Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs Yuan Xie, Jiaqi Song, Guang Qiu, Xianliang Wang, Ming Lei, Jie Gao, Jie Wu Integrating large language models (LLMs) into automatic speech recognition (ASR) has become a dominant paradigm. Although recent LLM-based ASR models have shown promisin…

Interesting project https://github.com/Xiaobin-Rong/unipase UniPASE: A Generative Model for Universal Speech Enhancement with High Fidelity and Low Hallucinations Xiaobin Rong, Zheng Wang, Yushi Wang, Jun Gao, Jing Lu Universal speech enhancement (USE) aims to restore speech signals from diverse distortions across multiple sampling rates. We propose UniPASE, an extension of the low-hallucination PASE framework tail…