ElevenLabs offer state of the art Text to Speech generation. All you have to do is get an API key from their website, and you are ready to use it as it is built right into the Wolfram Language:
In[]:=
el=ServiceConnect["ElevenLabs"]
Out[]=
ServiceObject
You have a bunch of voices available:
In[]:=
ServiceExecute[el,"ListVoices"]//Dataset
Out[]=
The voices are of a very high quality.
In[]:=
ServiceExecute["ElevenLabs","TextToSpeech",{"Text"->"hello there!","Voice"->"Nicole"}]
Out[]=
In[]:=
ServiceExecute["ElevenLabs","TextToSpeech",{"Text"->"hello there!","Voice"->"Patrick"}]
Out[]=
One of the issues is that it appears that via the Wolfram Language we only have access to English languages in spite of ElevenLabs offering a host of other languages such as:
Chinese, Spanish, Hindi, Portuguese, French, German, Japanese, Arabic, Russian, Korean, Indonesian, Italian, Dutch, Turkish, Polish, Swedish, Filipino, Malay, Romanian, Ukrainian, Greek, Czech, Danish, Finnish, Bulgarian, Croatian, Slovak, and Tamil.
Chinese, Spanish, Hindi, Portuguese, French, German, Japanese, Arabic, Russian, Korean, Indonesian, Italian, Dutch, Turkish, Polish, Swedish, Filipino, Malay, Romanian, Ukrainian, Greek, Czech, Danish, Finnish, Bulgarian, Croatian, Slovak, and Tamil.
Let’s develop our own API function to access all of these languages - and additional features.
API function for ElevenLabs
API function for ElevenLabs
For the following I will assume that you added your API key to your system credentials like so:
SystemCredential["my_EvlevenLabs_API"]="Your API key goes here"
As in many other posts I am using GPT-4 to produce a Wolfram Language function to interact with the API. I usually show it an API function that works in the Wolfram Language (from one of my other posts), and then copy one or two examples from the API documentation in. The code then literally “writes itself”.
In[]:=
callElevenLabsTTS[text_String,modelID_String,apiKey_String]:=Module[{url,headers,data,response},url="https://api.elevenlabs.io/v1/text-to-speech/"<>"ZYHJMj9SBt49EJkaXP9r";(*Removedthe<voice-id>*)headers={"Accept"->"audio/mpeg","Content-Type"->"application/json","xi-api-key"->apiKey};data=ExportString[{"text"->text,"model_id"->modelID,(*Assumingthisspecifiesthevoicemodelcorrectly*)"voice_settings"->{"stability"->0.5,"similarity_boost"->0.5}},"JSON"];response=URLExecute[HTTPRequest[url,<|"Method"->"POST","Headers"->headers,"Body"->data|>,CharacterEncoding->"UTF8"]];response];
Note that “ZYHJMj9SBt49EJkaXP9r” is a voice ID. Voice IDs are given on the ElevenLabs website.
Let’s try this out:
In[]:=
callElevenLabsTTS["I hope you are having a great day and are looking forward to using ElevenLabs Text to Speech.","eleven_monolingual_v1",SystemCredential["my_EvlevenLabs_API"]]
Out[]=
Other Languages
Other Languages
So up to now that works. We now need to find a voice in another language such as Spanish. I have chosen a Spanish speaker/voice from Spain, called Inés. The voice id is “”UAGGJEcGVn4XzCmO4GBV””.
Ok. Let’s first generate a text in Spanish:
In[]:=
paella=LLMSynthesize["Explain in detail how to cook the Spanish national dish paella. Give the full recipe. Use Spanish for the response."]
Out[]=
La paella es un plato tradicional español originario de la Comunidad Valenciana, que se ha ganado el corazón de muchos alrededor del mundo. Aquí te doy una receta clásica de paella valenciana, en español:**Ingredientes**:- 400 gramos de arroz de grano corto (tipo bomba)- 800 ml de caldo de pollo o verduras- 200 gramos de conejo (opcional, si deseas la versión tradicional)- 200 gramos de pollo- 100 gramos de judías verdes planas- 100 gramos de garrofó (alubias blancas grandes típicas de Valencia)- 1 tomate maduro rallado- 1 pimiento rojo en tiras (opcional)- 1 ramita de romero fresco (opcional)- Hebras de azafrán o colorante alimentario- Aceite de oliva- Sal al gusto- Ramitas de romero para decorar (opcional)**Preparación**:1. Precalienta una paellera con aceite de oliva a fuego medio-alto.2. Salpimenta el pollo (y el conejo si lo usas) y sofríelos en la paellera hasta que estén bien dorados por todos lados.3. Añade las judías verdes y el garrofó. Sofríe todo junto durante unos 5 minutos.4. Agrega el tomate rallado y cocina por unos minutos hasta que el tomate pierda su agua y se concentre el sabor.5. Añade el arroz y distribúyelo bien por todo el fondo de la paellera. Sofríe durante un par de minutos para que el arroz se impregne bien de todos los sabores.6. Incorpora el caldo de pollo o verduras, que debe estar caliente. Asegúrate de que el caldo cubra bien el arroz. Si es necesario, puedes añadir más. Añade azafrán o el colorante y ajusta de sal.7. Baja el fuego a medio-bajo y cocina durante unos 18 a 20 minutos. Es importante no remover el arroz después de este punto.8. Cuando queden unos 10 minutos de cocción, coloca las tiras de pimiento rojo encima del arroz y las ramitas de romero.9. Vigila que el caldo no se consuma antes de que el arroz esté listo. Si es necesario, añade un poco más de caldo o agua caliente.10. Cuando el arroz esté al punto y todo el líquido se haya absorbido, retira la paellera del fuego y deja reposar unos minutos antes de servir.11. Decorar con ramitas de romero y servir caliente.Recuerda que existen muchas variantes de la paella, como la paella de marisco o la mixta. Sin embargo, la receta que te he dado es una versión tradicional valenciana. ¡Disfruta cocinando y buen provecho!
I now need to change the voice id:
In[]:=
callElevenLabsTTS[text_String,modelID_String,apiKey_String]:=Module[{url,headers,data,response},url="https://api.elevenlabs.io/v1/text-to-speech/"<>"UAGGJEcGVn4XzCmO4GBV";(*Removedthe<voice-id>*)headers={"Accept"->"audio/mpeg","Content-Type"->"application/json","xi-api-key"->apiKey};data=ExportString[{"text"->text,"model_id"->modelID,(*Assumingthisspecifiesthevoicemodelcorrectly*)"voice_settings"->{"stability"->0.5,"similarity_boost"->0.5}},"JSON"];response=URLExecute[HTTPRequest[url,<|"Method"->"POST","Headers"->headers,"Body"->data|>,CharacterEncoding->"UTF8"]];response];
Finally, I call ElevenLabs:
In[]:=
callElevenLabsTTS[paella,"eleven_multilingual_v2",SystemCredential["my_EvlevenLabs_API"]]
Let’s try a speaker from Madrid, called “Javier Madrid” with the voice id “Rfj8YxsU5Gg9QdQE7F9O”
This is certainly more Spanish than with an English voice; there are some issues with special characters. Let’s try something in German, meet Sven (voice ID “xHefI2Hkg2xJampDzR0h”):
We now need a text in German. This is the first sentence (!!!!) of the UN Human Rights Charter in German, according to the ExampleData[]:
It is quite obvious that special characters such as ü, ö etc are not read quire correctly. Let’s see whether we can fix that.
No, that did not work.... Well, let’s see how the German voice speaks English. Let’s take the first sentence of the Human Rights Declaration in English:
I am not an expert, but that sounds a bit like a German accent.
Conclusion
Conclusion
The Wolfram Language already offers access to ElevenLabs, but I could not get the non-English languages to work. The direct API functionality offers some additional flexibility. This is particularly interesting if you want to use the dubbing functionality via the API. Sound effects are coming soon and direct API access might immediately provide this feature.
CITE THIS NOTEBOOK
CITE THIS NOTEBOOK
All languages text-to-Speech with ElevenLabs
by Marco Thiel
Wolfram Community, STAFF PICKS, March 17, 2024
https://community.wolfram.com/groups/-/m/t/3142293
by Marco Thiel
Wolfram Community, STAFF PICKS, March 17, 2024
https://community.wolfram.com/groups/-/m/t/3142293