r/speechtech

I fine-tuned Fun-CosyVoice3-0.5B, but after training, during inference I observe significant distortion, noise, and vibration in the generated audio.

To isolate the issue, I performed the following tests:

1. HiFiGAN-only test

• Regenerated audio directly from an input audio chunk using HiFiGAN (no tokenizer or Flow)
• Regenerated Output is exactly like the original clean audio
• Suggests HiFiGAN is not the source of the issue

2. Full pipeline test (tokenizer → Flow → HiFiGAN)

• Passed clean audio samples from my dataset through the full pipeline
• Regenerated Output synthesis contains noticeable vibration and distortion, despite clean input

3. Base vs fine-tuned Flow

Tested with both:

• Base Flow model
• Fine-tuned Flow model
• Both produce similar vibration artifacts

Additional observation:

• A clicking/mouse-like sound appears at the start and end of generated audio

What I’ve tried:

• Multiple audio normalization techniques (LUFS) before feeding data to the tokenizer
• Also tried de-clipping
• No improvement

I have been stuck with this for weeks now and i cannot figure out a way out. would be really helpful if someone with past experience working with cosyvoice could help out.

Questions:

• Has anyone encountered similar vibration/distortion artifacts in the tokenizer → Flow → HiFiGAN pipeline?
• Could this be related to tokenizer encoding/decoding mismatch or preprocessing?
• Any suggestions on debugging?

I have been trying to capture voices in a noisy atmosphere with a Seeed Studio eSpeaker XVF3800 and a rasberry pi. But I can't get the audio clear enough to do the speaker diarization in a high enough level to accomplish what I need. Looking for someone to help me solve this problem. I think I need a sound engineer and someone who also knows how to leverage AI to help enhance the captured audio to do this at scale. Anyone interested or know someone who might be able to help?

hello im buildign a app via vibe coding and it really needs audio in and audio out for the ai questions and answers.what is peoples experiances of the best way of achieving a ultra clear audio in and audio out answer #audioai #vibecoding #ai #helpneeded

Hi colleagues, I have a SaaS that transcribes 10 million minutes of audio per month, and I've tried many different processing methods. Currently, I'm using orchardrun.com because it offers the best performance and price (0.025 per hour) and allows me to handle fairly large audio files. But do you know of any other, more economical options?

Marketing pages claim 90%+ accuracy on Hinglish. Reality from the teams I've talked to looks very different.

If you're using or have evaluated Indian-language STT for any use-case - voicebots, call analytics, video KYC, transcription, voice search, etc. would love to hear what you picked, why, and where it falls short.

Happy to share my learnings. Drop a comment or DM for a 30 min chat.

Using Large V3 model but facing issue in transcribing Srilankan language Sinhala. Did anyone try to transcribe this language and get a good result?