OllamaSynthesize

Interact with local AI/LLM models via an Ollama server

DefinitionResourceCreateNotebook

Clear[OllamaSynthesize]

ollamaServerBinary:=Switch[
$SystemID,
"Windows-x86-64",FileNameJoin[{$HomeDirectory,"AppData","Local","Programs","Ollama","ollama.exe"}],
"MacOSX-x86-64","/usr/local/bin/ollama",
"MacOSX-ARM64","/usr/local/bin/ollama",
"Linux-x86-64","/usr/local/bin/ollama"
]

Options[OllamaSynthesize]={
"OllamaModel"->"llava"(* default model, with vision *)
}

ollamaRunningQ[]:=Quiet[
URLRead["http://localhost:11434/","Body"]==="Ollama is running"
];

ollamaProcesses[]:=Dataset[
ImportByteArray[
URLRead["http://localhost:11434/api/ps","BodyByteArray"],
"RawJSON"]];

ollamaModels[]:=Dataset[
ImportByteArray[
URLRead["http://localhost:11434/api/tags","BodyByteArray"],
"RawJSON"]];

ollamaModelInformation[model_String]:=Module[{request},
request=HTTPRequest[
"http://localhost:11434/api/show",
<|"Body"->ExportString[
<|"model"->model|>,
"RawJSON"]
|>];
Dataset[ImportByteArray[URLRead[request,"BodyByteArray"],"RawJSON"]]]

ollamaStart[model_String,OptionsPattern[OllamaSynthesize]]:=Module[{},
StartProcess[{ollamaServerBinary,"serve"}];
While[Not[ollamaRunningQ[]],Pause[.1]];(* wait for server to become available *)
StartProcess[{ollamaServerBinary,"run",model}];
];

ollamaRequest[model_String,prompt_String,images_List]:=Module[{base64images,json},
base64images=Map[Function[{image},
BaseEncode[ExportByteArray[image,"PNG"]]
],images];
json=ExportByteArray[
<|"model"->model,"prompt"->prompt,"images"->base64images,"stream"->False|>,
"RawJSON"];
HTTPRequest[
"http://localhost:11434/api/generate",
<|"Body"->json
|>]
]

OllamaSynthesize[prompt_String,options___]:=OllamaSynthesize[prompt,{},options];
OllamaSynthesize[list_List,options___]:=OllamaSynthesize[StringRiffle[Select[list,StringQ],"\n"],Select[list,ImageQ],options];
OllamaSynthesize[prompt_String,images_List,OptionsPattern[OllamaSynthesize]]:=Module[{model,request,response,result},
model=OptionValue["OllamaModel"];
ollamaStart[model];
request=ollamaRequest[model,prompt,images];
response=URLRead[request];
result=ImportByteArray[response["BodyByteArray"],"RawJSON"];
Enclose[ConfirmMatch[result["response"],_String],Function[{failure},Failure["NoResult",<|
"MessageParameters"-><|"model"->model|>,"MessageTemplate"->
"Did not receive a result from the Ollama server...\nPossible causes:\n· Ollama is not installed\n· The model `model` is too large to run on your machine\n· The model `model` is not installed"
|>]]]
]

Documentation

UsageResourceCreateNotebook

Details & OptionsResourceCreateNotebook

OllamaSynthesize requires the user to pre-install the Ollama software and the desired models. This function does not manage the installation of Ollama and its models and will not work without it.

The default model used by OllamaSynthesize is "Llava" and must be pre-installed by the user. The "Llava" model supports images.

Models that do support images, like "LLama3.2" will not work with images and a failure will be returned.

You can specify a desired model with the "OllamaModel" option. Examples of how to do this are under the "Options" example section below.

OllamaSynthesize launches a server process and uses API requests to get its responses. The server binary is chosen automatically for default installation locations, but for non-default installation locations you can give the "OllamaServerBinary" option to specify that non-default location on your machine.

ExamplesResourceCreateNotebook

Basic Examples

Try a basic question:

OllamaSynthesize["Hello, who was Albert Einstein?"]

Ask a question about an image:

OllamaSynthesize["Who is the person in the picture?",{}]

Ask another question about a different image:

OllamaSynthesize["What animal is this?",{}]

A similar question with a different vision-enabled LLM model:

OllamaSynthesize["What is in this image?",{},"OllamaModel"->"moondream"]

Mix the question and image(s) in a single list:

OllamaSynthesize[{,"What is in this image?"}]

Scope

Solve basic math problems with step by step instructions:

OllamaSynthesize["I have three boxes of six pencils each. How many pencils do I have?"]//Style[#,"Text"]&

Options

The default model is "Llava". Use the "OllamaModel" option to specify another one:

OllamaSynthesize["Hello, who are what are you?","OllamaModel"->"llama3.2"]

Larger models work too, but are slower. Also your machine will need to have sufficient GPU memory to run the model:

OllamaSynthesize["Hello, who are what are you?","OllamaModel"->"gemma2:27b"]

If you specify a model that does not exist or a model that you did not download locally, then an error is raised:

OllamaSynthesize["Hello, who are what are you?","OllamaModel"->"xxx"]

Applications

Properties and Relations

Possible Issues

Repeated calls to OllamaSynthesize will give slightly randomized results and sometimes these results can be wildly incorrect:

OllamaSynthesize["Two train stations called A and B are 50 miles apart. A train leaves from A towoards B at 20mph. Another train leaves from B to A at 30 mph. How long after they leave do they pass each other?"]//Style[#,"Text"]&

You can use Solve to do this correctly. The result is given in hours:

Solve[Quantity[20,"Miles"/"Hours"]Quantity[x,"Hours"]==Quantity[50,"Miles"]-Quantity[30,"Miles"/"Hours"]Quantity[x,"Hours"],x]

You are responsible for installing Ollama on your machine and for the downloading of the models you wish to use. To list the available models, you can run the following (undocumented) command:

ollamaModels[]

Neat Examples

Source & Additional Information

Contributed ByResourceCreateNotebook

Arnoud Buzing

KeywordsResourceCreateNotebook

keyword 1

CategoriesResourceCreateNotebook

Related SymbolsResourceCreateNotebook

Related Resource ObjectsResourceCreateNotebook

Resource Name (resources from any Wolfram repository)

Source/Reference CitationResourceCreateNotebook

Source, reference or citation information

LinksResourceCreateNotebook

Link to other related material

TestsResourceCreateNotebook

MyFunction[x,y]

CompatibilityResourceCreateNotebook

Wolfram Language VersionResourceCreateNotebook

14.0+

Operating SystemResourceCreateNotebook

EnvironmentsResourceCreateNotebook

Cloud SupportResourceCreateNotebook

Required FeaturesResourceCreateNotebook

Author NotesResourceCreateNotebook

Additional information about limitations, issues, etc.

Submission NotesResourceCreateNotebook

Ignore previous submission, it contained some bugs.