LLM has the problem of not being able to deal with content outside of the study, but the use of RAG (Retrieval Augmented Generation) technology allows it to deal with unknown information.
RAG works as follows:
0. Preparation: documents are segmented and paired with embedding to be registered in the database
1. Retrieval: Search the database for information related to the question
2. Augmented: query LLM by augmenting the question with the retrieved information
3. Generation: Generate an answer to the query
0. Preparation: documents are segmented and paired with embedding to be registered in the database
1. Retrieval: Search the database for information related to the question
2. Augmented: query LLM by augmenting the question with the retrieved information
3. Generation: Generate an answer to the query
I explain how to implement RAG with Wolfram Language using the Japanese Constitution as a subject .
A few months ago I wrote a post (in Japanese) on how to implement a RAG with Wolfram Language using the Japanese Constitution as the example.
Now version 14.1 has added functions(LLMPromptGenerator, SemanticSearch, VectorDatabaseSearch, ...) to support RAG, I will revise it, using these functions.
Preparation
Preparation
First, we need to prepare chunks of the Japanese Constitution, which consists of 103 articles, divided by article.
This preparation is an important step in improving RAG performance. This allows RAG to search more accurately.
In[]:=
xmlEN=Import["https://www.shugiin.go.jp/internet/itdb_english.nsf/html/statics/english/constitution_e.htm","XMLObject"];itemsEN=Cases[xmlEN,XMLElement["p",{},{x_}]StringTrim[x],∞];chunksEN=Select[itemsEN,StringTake[#,8]==="Article "&];
In[]:=
Length@chunksEN
Out[]=
103
Show the five chunks.
In[]:=
chunksEN[[;;5]]//TableForm
Out[]//TableForm=
Article 1. The Emperor shall be the symbol of the State and of the unity of the People, deriving his position from the will of the people with whom resides sovereign power. |
Article 2. The Imperial Throne shall be dynastic and succeeded to in accordance with the Imperial House Law passed by the Diet. |
Article 3. The advice and approval of the Cabinet shall be required for all acts of the Emperor in matters of state, and the Cabinet shall be responsible therefor. |
Article 4. The Emperor shall perform only such acts in matters of state as are provided for in this Constitution and he shall not have powers related to government. |
Article 5. When, in accordance with the Imperial House Law, a Regency is established, the Regent shall perform his acts in matters of state in the Emperor's name. In this case, paragraph one of the preceding article will be applicable. |
Question and chunks needed for answer
Question and chunks needed for answer
The questions shall be as follows
In[]:=
question="What is the term of office for members of both houses of Congress as defined by the Constitution of Japan? Also, what article does it say?";
The chunks needed for answer are Articles 45, 46, and 102.
In[]:=
chunksEN[[{45,46,102}]]//TableForm
Out[]//TableForm=
Article 45. The term of office of members of the House of Representatives shall be four years. However, the term shall be terminated before the full term is up in case the House of Representatives is dissolved. |
Article 46. The term of office of members of the House of Councillors shall be six years, and election for half the members shall take place every three years. |
Article 102. The term of office for half the members of the House of Councillors serving in the first term under this Constitution shall be three years. Members falling under this category shall be determined in accordance with law. |
RAG by SemanticSearch
RAG by SemanticSearch
Let’s try if we can retrieve the chunks needed to answer using SemanticSearch, which is supported in version 14.1.
In[]:=
index=CreateSemanticSearchIndex[chunksEN]
Out[]=
SemanticSearchIndex
In[]:=
SemanticSearch[index,question,"Item"]//TableForm
Out[]//TableForm=
Article 43. Both Houses shall consist of elected members, representative of all the people. |
Article 97. The fundamental human rights by this Constitution guaranteed to the people of Japan are fruits of the age-old struggle of man to be free; they have survived the many exacting tests for durability and are conferred upon this and future generations in trust, to be held for all time inviolate. |
Article 45. The term of office of members of the House of Representatives shall be four years. However, the term shall be terminated before the full term is up in case the House of Representatives is dissolved. |
Article 102. The term of office for half the members of the House of Councillors serving in the first term under this Constitution shall be three years. Members falling under this category shall be determined in accordance with law. |
Article 103. The Ministers of State, members of the House of Representatives and judges in office on the effective date of this Constitution, and all other public officials who occupy positions corresponding to such positions as are recognized by this Constitution shall not forfeit their positions automatically on account of the enforcement of this Constitution unless otherwise specified by law. When, however, successors are elected or appointed under the provisions of this Constitution, they shall forfeit their positions as a matter of course. |
Article 55. Each House shall judge disputes related to qualifications of its members. However, in order to deny a seat to any member, it is necessary to pass a resolution by a majority of two-thirds or more of the members present. |
Article 96. Amendments to this Constitution shall be initiated by the Diet, through a concurring vote of two-thirds or more of all the members of each House and shall thereupon be submitted to the people for ratification, which shall require the affirmative vote of a majority of all votes cast thereon, at a special referendum or at such election as the Diet shall specify. |
Article 57. Deliberation in each House shall be public. However, a secret meeting may be held where a majority of two-thirds or more of those members present passes a resolution therefor. |
Article 98. This Constitution shall be the supreme law of the nation and no law, ordinance, imperial rescript or other act of government, or part thereof, contrary to the provisions hereof, shall have legal force or validity. |
Article 9. Aspiring sincerely to an international peace based on justice and order, the Japanese people forever renounce war as a sovereign right of the nation and the threat or use of force as means of settling international disputes. |
Article 45 and 102 are retrieved among the Top 10, but Article 46 are not. (Article 46 is in the 18th.)
So, the Top 10 chunks does not provide the correct answer. The answer does not mention Article 46.
In[]:=
LLMSynthesize[question,LLMEvaluator->"Prompts"->LLMPromptGenerator@index]
Out[]=
The term of office for members of both houses of the Japanese parliament, known as the National Diet, is defined by the Constitution of Japan as follows:- For members of the House of Representatives (lower house): The term of office is four years. However, the term may end earlier if the House of Representatives is dissolved. (Article 45)- For members of the House of Councillors (upper house): The term of office for half of the members serving in the first term under the Constitution is three years. The members falling under this category are determined by law. (Article 102)So, in summary:- House of Representatives: Four years, with potential early dissolution. (Article 45)- House of Councillors: Three years for half of the members elected in the first term. (Article 102)
This will be due to the embed method “SentenceBERT” used in the FeatureExtractor option of CreateSemanticSearchIndex.
RAG by VectorDatabaseSearch
RAG by VectorDatabaseSearch
I usually use Japanese documents, so I use Hugging Face multilingual-e5-large and cohere embed-multilingual-v3.0 for embed.
Create a vector database by using cohere embed-multilingual-v3.0, since VectorDatabaseSearch is also supported in version 14.1.
In[]:=
Clear[cohereEmbed];cohereEmbed[str_String,type_String]:=Module[{url,jsonBody,myRequest,res},url="https://api.cohere.ai/v1/embed";jsonBody=ExportString[{"model"->"embed-multilingual-v3.0","texts"->{str},"input_type"->type},"JSON"];myRequest=HTTPRequest[url,<|"Method"->"POST","Headers"->{"Authorization"->"bearer "<>SystemCredential["Cohere_API_KEY"]},"Body"->jsonBody,"ContentType"->"application/json","accept"->"application/json"|>];res=URLExecute[myRequest];Association[res]["embeddings"][[1]]];
Embed 103 chunks.
embcohereEN=cohereEmbed[#,"search_document"]&/@chunksEN;
In[]:=
embcohereEN=;
Each chunk is embeded into a 1024-dimensional vector.
In[]:=
Dimensions@embcohereEN
Out[]=
{103,1024}
Create a vector database.
In[]:=
vdb=CreateVectorDatabase[Thread[embcohereEN->chunksEN]]
Out[]=
VectorDatabaseObject
Embed the question into a 1024-dimensional vector.
In[]:=
embQ=cohereEmbed[question,"search_query"];
In[]:=
embQ=;
The vector search results show that Articles 45, 46, and 102 are in the top 3.
In[]:=
VectorDatabaseSearch[vdb,embQ,"Metadata",5]//TableForm
Out[]//TableForm=
Article 45. The term of office of members of the House of Representatives shall be four years. However, the term shall be terminated before the full term is up in case the House of Representatives is dissolved. |
Article 102. The term of office for half the members of the House of Councillors serving in the first term under this Constitution shall be three years. Members falling under this category shall be determined in accordance with law. |
Article 46. The term of office of members of the House of Councillors shall be six years, and election for half the members shall take place every three years. |
Article 43. Both Houses shall consist of elected members, representative of all the people. |
Article 48. No person shall be permitted to be a member of both Houses simultaneously. |
Prepare a function to be specified for LLMPromptGenerator. The function retrieves five chunks from the vector database that are close to the specified question.
In[]:=
f[q_String]:="Answer using this information:\n\nBEGIN CONTEXT\n\n"<>StringRiffle[VectorDatabaseSearch[vdb,cohereEmbed[q,"search_query"],"Metadata",5],"\n"]<>"\n\nEND CONTEXT";
Provide the correct answer.
Conclusion
Conclusion
I explained RAG with Wolfram Language, which solve problems with LLMs, such as the inability to respond to content outside of the study and insufficient responses due to information overload.
RAG also enable LLMs to generate answers based on private information.
However, CreateVectorDatabase and LLMSynthesize use external services, so private information cannot be used.
To solve this problem, LLM/embed models that can be built in a private environment are released daily. Although there are some issues such as hardware environment and non-Latin languages support, with the evolution of AI warp speed, the day will soon come when LLMs can be run on private PCs without stress.
RAG also enable LLMs to generate answers based on private information.
However, CreateVectorDatabase and LLMSynthesize use external services, so private information cannot be used.
To solve this problem, LLM/embed models that can be built in a private environment are released daily. Although there are some issues such as hardware environment and non-Latin languages support, with the evolution of AI warp speed, the day will soon come when LLMs can be run on private PCs without stress.
CITE THIS NOTEBOOK
CITE THIS NOTEBOOK
Retrieval augmented generation with Wolfram Language
by Kotaro Okazaki
Wolfram Community, STAFF PICKS, August 6, 2024
https://community.wolfram.com/groups/-/m/t/3241848
by Kotaro Okazaki
Wolfram Community, STAFF PICKS, August 6, 2024
https://community.wolfram.com/groups/-/m/t/3241848