Training a custom TTS model for the Caribbean

Over the weekend I decided to take the plunge and train my own TTS model, starting with the open-source Pocket TTS from Kyutai Labs.

Since their training data is also open source, I was able to train the model on an H200, then fine-tune it further and experiment with adding a Caribbean accent 🇹🇹.

Here’s the result after about a day and a half of training and iterating. It’s definitely not perfect, and with cleaner, better-annotated, and more isolated training data, I think the result could be 10x better. But for a first attempt, I’m really happy with how it turned out

This entire project would not be possible without the team at Kyutai publishing and sharing their work, making it possible to build fun things like this in a weekend, excited to experiment and learn more :)

Check out the project: https://github.com/kyutai-labs/pocket-tts/tree/main/

If you're building AI in the Caribbean, let's collaborate :)