Semantics is the study of meanings in a language. It explains how words, phrases and sentences convey meaning to a listener or reader. In turn, a semantic search takes into account the meaning and context of a search query, rather than just matching keywords. The Wolfram Language has introduced semantic search capabilities in Version 14.1. As a food writer, I knew the first way I wanted to use it — for recipes.
Semantic search is a key step in the Wolfram Language’s support for retrieval-augmented generation (RAG). RAG begins with basic data, such as a set of documents, and transforms that data into vectors, known as embeddings. The similarity of any two embeddings is calculated using a distance function. The closer two vectors are, the more semantically similar they are. My data is a set of recipes, which the Wolfram Language transformed into a searchable vector database.
Once a semantic search identifies the relevant recipe data, the next step in RAG is to have the large language model (LLM) generate a meaningful response to the query, based on the context derived from the semantic search. I used simple, custom LLM tools to retrieve complete recipes, identify cooking actions, and answer questions about the recipes in the database.
Semantic search is a key step in the Wolfram Language’s support for retrieval-augmented generation (RAG). RAG begins with basic data, such as a set of documents, and transforms that data into vectors, known as embeddings. The similarity of any two embeddings is calculated using a distance function. The closer two vectors are, the more semantically similar they are. My data is a set of recipes, which the Wolfram Language transformed into a searchable vector database.
Once a semantic search identifies the relevant recipe data, the next step in RAG is to have the large language model (LLM) generate a meaningful response to the query, based on the context derived from the semantic search. I used simple, custom LLM tools to retrieve complete recipes, identify cooking actions, and answer questions about the recipes in the database.
Create a recipes list
Create a recipes list
To begin, I created a list of sample recipes:
In[]:=
sampleRecipes={{"Scrambled eggs->Ingredients: 2-3 eggs, 1 tablespoon butter. Instructions: Crack eggs into a bowl and whisk until combined. Heat a non-stick pan over medium heat and add butter. Pour in the eggs and let them sit for a few seconds. Gently stir with a spatula, folding the eggs as they cook. Remove from heat while still slightly runny and serve."},{"Oatmeal->Ingredients: 1 cup water or milk, 1/2 cup oats, optional toppings such as fruit, nuts or honey. Instructions: Bring water or milk to a boil in a small saucepan. Stir in oats and reduce to medium heat. Stirring occasionally, cook for 5-7 minutes until thickened. Remove from heat. Add toppings such as fruit, nuts or honey before serving."},{"Vegetable spring rolls->Ingredients: 8 rice paper wrappers, 1 carrot, 1 cucumber, 1 bell pepper, 1/2 cup red cabbage, 1/4 cup fresh cilantro, 1/4 cup fresh mint, 1/4 cup fresh basil. Instructions: Peel and julienne the carrot, cucumber, and bell pepper. Shred the red cabbage. Fill a large bowl with warm water. Dip one rice paper wrapper into the water for about 10-15 seconds to soften. Place the softened wrapper on a clean, flat surface. Arrange a small handful of each vegetable and herbs in the center of the wrapper. Fold the sides of the wrapper over the filling, then roll tightly from the bottom to secure. Repeat with remaining wrappers and vegetables. Serve with your favorite dipping sauce."},{"Tomato and basil soup->Ingredients: 2 cans diced tomatoes, 1 onion, 2 cloves garlic, 2 cups vegetable broth, 1/2 cup fresh basil leaves, 1 tablespoon olive oil, Salt and pepper to taste. Instructions: Begin by chopping the onion and mincing the garlic. In a large pot, heat olive oil over medium heat. Add the chopped onion and minced garlic, cooking until softened. Add the diced tomatoes and vegetable broth. Bring to a boil, then reduce heat and simmer for 20 minutes. Add the basil leaves and blend the soup until smooth using an immersion blender. Season with salt and pepper to taste and serve."},{"Chicken Caesar salad->Ingredients: 1 head romaine lettuce, 1 cup croutons, 1/2 cup Caesar dressing, 1/4 cup grated Parmesan cheese, 2 chicken breasts. Instructions: Chop the romaine lettuce and place it in a large bowl. Cook and slice the chicken breasts. Combine the romaine lettuce, croutons, and Parmesan cheese in the bowl. Add Caesar dressing and toss until the salad is evenly coated. Top with the sliced chicken breasts and serve."},{"Lentil stew->Ingredients: 1 cup dried lentils, 1 onion, 2 cloves garlic, 2 carrots, 2 celery stalks, 1 can diced tomatoes, 4 cups vegetable broth, 1 teaspoon ground cumin, 1 teaspoon ground coriander, 1/2 teaspoon smoked paprika, 1 tablespoon olive oil, Salt and pepper to taste. Instructions: Sort and rinse the lentils. Chop the onion, carrots, and celery. Mince the garlic. In a large pot, heat olive oil over medium heat. Add the chopped onion and minced garlic, cooking until softened. Add the chopped carrots and celery, cooking for an additional 5 minutes. Stir in the cumin, coriander, and smoked paprika, cooking for 1 minute until fragrant. Add the diced tomatoes, vegetable broth, and lentils. Bring to a boil, then reduce heat and simmer for 30-40 minutes, or until lentils are tender. Season with salt and pepper to taste and serve."},{"Baked salmon->Ingredients: 4 salmon fillets, 1 tablespoon olive oil, 1 lemon (sliced), 1 teaspoon dried dill, 1 teaspoon garlic powder, Salt and pepper to taste. Instructions: Preheat oven to 375°F (190°C). Place salmon fillets on a baking sheet lined with parchment paper. Drizzle with olive oil and season with dill, garlic powder, salt, and pepper. Top each fillet with lemon slices. Bake for 15-20 minutes, or until the salmon is cooked through and flakes easily with a fork. Serve hot."},{"Bulgogi->Ingredients: 1 lb thinly sliced beef ribeye, 1/4 cup soy sauce, 2 tablespoons sugar, 1 tablespoon sesame oil, 2 tablespoons mirin, 1/4 cup grated onion, 3 cloves garlic, minced, 2 tablespoons grated pear or apple, 2 green onions, chopped, 1 tablespoon sesame seeds, 1/4 teaspoon black pepper. Instructions: In a large bowl, combine soy sauce, sugar, sesame oil, mirin, grated onion, minced garlic, grated pear or apple, chopped green onions, sesame seeds, and black pepper. Add the thinly sliced beef to the marinade and mix well to ensure all pieces are coated. Cover the bowl and let it marinate in the refrigerator for at least 30 minutes. Heat a grill or a large skillet over medium-high heat. Cook the marinated beef in batches, ensuring not to overcrowd the pan, for about 2-3 minutes on each side or until fully cooked and slightly caramelized. Serve hot with rice and garnish with additional sesame seeds and green onions if desired."},{"No-bake cheesecake->Ingredients: 1 1/2 cups graham cracker crumbs, 1/3 cup butter (melted), 2 packages (8 oz each) cream cheese (softened), 1 cup powdered sugar, 1 teaspoon vanilla extract, 2 cups heavy whipping cream. Instructions: In a bowl, combine graham cracker crumbs and melted butter. Press mixture into the bottom of a 9-inch springform pan to form the crust. In a separate bowl, beat the softened cream cheese, powdered sugar, and vanilla extract until smooth. In another bowl, whip the heavy cream until stiff peaks form. Gently fold the whipped cream into the cream cheese mixture until well combined. Pour the filling over the crust and spread evenly. Refrigerate for at least 4 hours or until set. Serve chilled."},{"Key lime pie->Ingredients: 1 1/2 cups graham cracker crumbs, 1/3 cup butter (melted), 3 egg yolks, 1 can (14 oz) sweetened condensed milk, 1/2 cup key lime juice, 1 tablespoon lime zest, whipped cream for garnish. Instructions: Preheat oven to 350°F (175°C). In a bowl, combine graham cracker crumbs and melted butter. Press mixture into the bottom of a 9-inch pie pan to form the crust. In a separate bowl, whisk together the egg yolks, sweetened condensed milk, key lime juice, and lime zest until smooth. Pour the filling over the crust and spread evenly. Bake for 15 minutes, then let cool to room temperature. Refrigerate for at least 2 hours or until set. Garnish with whipped cream before serving."}};
Create the recipes index
Create the recipes index
I used this custom function to process the sample recipes list, by extracting the recipe names, ingredients and instruction sentences, and then created a semantic search index from the sentences:
In[]:=
makeSemanticDatabase[recipes_List,indexName_]:=Module[{},namesAndIngredientsAndInstructions=Map[StringTrim,StringSplit[#,"->"|"Instructions: "]&/@Flatten[sampleRecipes],{2}];namesAndIngredientsAndInstructionSentences={#1,#2,TextSentences[#3]}&@@@namesAndIngredientsAndInstructions;ruleList=Flatten[Function[{recipeName,ingredients,instructionSentences},(#->(#<>" (from recipe for \""<>recipeName<>"\")"))&/@instructionSentences]@@@namesAndIngredientsAndInstructionSentences];CreateSemanticSearchIndex[ruleList,indexName]]
In[]:=
index=makeSemanticDatabase[sampleRecipes,"SampleRecipesInstructions"]
Out[]=
SemanticSearchIndex
Make an LLM tool to retrieve a recipe
Make an LLM tool to retrieve a recipe
LLMTool created a tool to retrieve a full recipe from the index. I used it to retrieve the complete recipe for baked salmon:
In[]:=
(completeRecipe[ToLowerCase[#[[1]]]]:=(Echo[#[[1]]];#[[2]]))&/@StringSplit[#,"->"]&/@sampleRecipes;completeRecipeTool=LLMTool[{"Recipe","Get the complete recipe for a dish."},"recipe"->"String",completeRecipe[ToLowerCase[#recipe]]&]
Out[]=
LLMTool
In[]:=
completeRecipe["baked salmon"]
»
Baked salmon
Out[]=
Ingredients: 4 salmon fillets, 1 tablespoon olive oil, 1 lemon (sliced), 1 teaspoon dried dill, 1 teaspoon garlic powder, Salt and pepper to taste. Instructions: Preheat oven to 375°F (190°C). Place salmon fillets on a baking sheet lined with parchment paper. Drizzle with olive oil and season with dill, garlic powder, salt, and pepper. Top each fillet with lemon slices. Bake for 15-20 minutes, or until the salmon is cooked through and flakes easily with a fork. Serve hot.
Search for a recipe
Search for a recipe
The LLM can be wordy, so I often advise it not to add extra text in its response:
In[]:=
LLMSynthesize["Tomato and basil soup. No extra text please.",LLMEvaluator->LLMConfiguration[<|"Tools"->completeRecipeTool|>]]
»
Tomato and basil soup
Out[]=
Ingredients:- 2 cans diced tomatoes- 1 onion- 2 cloves garlic- 2 cups vegetable broth- 1/2 cup fresh basil leaves- 1 tablespoon olive oil- Salt and pepper to tasteInstructions:1. Begin by chopping the onion and mincing the garlic.2. In a large pot, heat olive oil over medium heat.3. Add the chopped onion and minced garlic, cooking until softened.4. Add the diced tomatoes and vegetable broth.5. Bring to a boil, then reduce heat and simmer for 20 minutes.6. Add the basil leaves and blend the soup until smooth using an immersion blender.7. Season with salt and pepper to taste and serve.
Make an LLM tool to retrieve a cooking action
Make an LLM tool to retrieve a cooking action
I also used LLMTool to find recipe steps that contain specific cooking actions, such as boiling, chopping and dicing. I was able to set the maximum number of steps I want returned with MaxItems:
In[]:=
findCookingActionTool=LLMTool[{"FindCookingAction","Find a step of a recipe that contains the given action."},"cookingAction"->"String",SemanticSearch[index,#cookingAction,MaxItems->3]&]
Out[]=
LLMTool
Search for a cooking action
Search for a cooking action
I enjoyed searching for preparation and cooking methods in the recipes:
In[]:=
LLMSynthesize["preheat",LLMEvaluator->LLMConfiguration[<|"Tools"->{findCookingActionTool}|>]]
Out[]=
Sure, I'll find a recipe step that involves preheating.Here are the recipe steps involving preheating:1. **Key Lime Pie**: Preheat oven to 350°F (175°C).2. **Baked Salmon**: Preheat oven to 375°F (190°C).
In[]:=
LLMSynthesize["Names of 3 recipes that include chopped vegetables. No extra text please.",LLMEvaluator->LLMConfiguration[<|"Tools"->{findCookingActionTool}|>]]
Out[]=
- Lentil stew- Bulgogi- Chicken Caesar salad
Search for a recipe by cooking action
Search for a recipe by cooking action
I used both LLM tools in this query:
Add to the recipe index
Add to the recipe index
An important feature of a vector database is that you can add to it incrementally. I used UpdateSemanticSearchIndex in a new function to quickly add more recipes to my index:
Check for new recipes in index
Check for new recipes in index
The new recipes added 10 embeddings to my original index, from 69 to 79. I wrote a query to check the updated index for one of the new recipes:
The LLM correctly retrieved the emulsify step from the new vinaigrette recipe. Next, I checked that the LLM still had access to the original recipes:
The LLM successfully retrieved the rice paper step from the vegetable spring rolls recipe in the original dataset.
Update the completeRecipe function
Update the completeRecipe function
To retrieve a complete recipe, I updated the function to apply to the new recipes:
The completeRecipe function and LLM tool now work for both the original and new recipes in the index:
Conclusion
Conclusion
With this successful implementation of semantic search and RAG using a small set of recipes, I’m excited to expand my recipes index for larger-scale recipe management and analysis with the Wolfram Language. Using a large set of recipes, I plan to explore clusters and patterns in food preparation and cooking methods. Please stay tuned!
CITE THIS NOTEBOOK
CITE THIS NOTEBOOK
Recipe management with SemanticSearchIndex and retrieval-augmented generation (RAG)
by Gay Wilson
Wolfram Community, STAFF PICKS, August 29, 2024
https://community.wolfram.com/groups/-/m/t/3260355
by Gay Wilson
Wolfram Community, STAFF PICKS, August 29, 2024
https://community.wolfram.com/groups/-/m/t/3260355