This tool takes any tabular dataset and generates descriptive statistics to summarize the data. The summary is then passed to a large language model (LLM), which suggests appropriate types of visualizations based on the dataset’s characteristics. After preparing the data accordingly, the system produces both the summary statistics and the LLM-recommended visualizations. This process provides an automated pipeline for understanding and exploring tabular data through statistical summaries and tailored visual representations.
Standardize Dataset
Standardize Dataset
standardizeDataset ensures that the dataset is consistently represented as a list of associations, making it easier to process in downstream code.
This code defines a function standardizeDataset that takes a dataset data and returns it in a standardized format (as a list of associations), regardless of its original structure.
It checks the structure of data and:
If data is already a list of associations, it returns it as-is.
If data is a list of lists with a “Columns” key, it reconstructs each row as an association using the column names.
If data is already a list of associations, it returns it.
Otherwise, it returns data unchanged.
This code defines a function standardizeDataset that takes a dataset data and returns it in a standardized format (as a list of associations), regardless of its original structure.
It checks the structure of data and:
If data is already a list of associations, it returns it as-is.
If data is a list of lists with a “Columns” key, it reconstructs each row as an association using the column names.
If data is already a list of associations, it returns it.
Otherwise, it returns data unchanged.
standardizeDataset converts any supported dataset structure into a consistent list of associations, ensuring uniformity for downstream data processing regardless of the original format.
In[]:=
standardizeDataset[ds_] := Module[{data, columns}, Which[ MatchQ[Normal[ds][[1]], _Association], Normal[ds], ListQ[Normal[ds][[1]]] && ListQ[ds["Columns"]], columns = ds["Columns"]; data = Normal[ds]; AssociationThread[columns, #] & /@ data, ListQ[ds] && MatchQ[ds[[1]], _Association], ds, True, ds ]]
Strip Units
Strip Units
This function, stripUnits, takes a value and attempts to extract its numeric magnitude, regardless of whether the input is a number, a string containing a number (possibly with units or symbols), a Quantity object, or a symbol. If the value cannot be converted to a number, it returns Missing[]. This is useful for cleaning and standardizing data columns that may contain numbers with units or as strings.
This function, stripUnits, takes a value and attempts to extract its numeric component. It handles numbers, strings with embedded numbers, Quantity objects, and symbols, returning Missing[] if a numeric value cannot be extracted.
In[]:=
stripUnits[val_] := Which[ NumberQ[val], val, StringQ[val], Quiet@Check[ ToExpression[ StringReplace[val, RegularExpression["[^0-9\\.\\-]"] -> ""] ], Missing[] ], Head[val] === Quantity, QuantityMagnitude[val], MatchQ[val, _Symbol], Quiet@Check[ ToExpression[ StringReplace[ToString[val], RegularExpression["[^0-9\\.\\-]"] -> ""] ], Missing[] ], True, Missing[]]
Process Numeric Columns With Units
Process Numeric Columns With Units
The function processNumericColumnsWithUnits iterates over each column in the dataset ds, attempts to convert each value in the column to a plain number using the stripUnits function, and replaces the column with the cleaned values if at least 70% of the non-missing entries are numeric. This helps ensure that columns containing numbers with units, or numbers as strings, are converted to pure numeric form for further analysis. The function returns the updated dataset with standardized numeric columns.
Cleans and standardizes numeric columns in a dataset by stripping units and converting values to numbers when possible.
In[]:=
processNumericColumnsWithUnits[ds_] := Module[ {columns, newds, vals, numericCount, totalCount, cleanedVals}, columns = Keys[ds[[1]]]; newds = ds; Do[ vals = ds[[All, col]]; cleanedVals = stripUnits /@ vals; numericCount = Count[cleanedVals, _?NumberQ]; totalCount = Length[cleanedVals] - Count[cleanedVals, _Missing]; If[totalCount > 0 && numericCount/totalCount > 0.7, newds[[All, col]] = cleanedVals; ], {col, columns} ]; newds]
Data Input
Data Input
This code retrieves the dataset, converts it into a consistent list-of-associations format using standardizeDataset, and then processes each column to ensure numeric values are stripped of units and standardized using processNumericColumnsWithUnits. It prints the first five rows for a quick preview .
Loads a tabular dataset, standardizes its structure, cleans numeric columns, and extracts sample rows for inspection
In[]:=
rawds=ResourceData["Sample Tabular Data: Palmer Penguins"];ds=standardizeDataset[rawds];ds=processNumericColumnsWithUnits[ds];Print[ds[[1;;5]]];
{index1,speciesAdelie,islandTorgersen,bill_length39.1,bill_depth18.7,flipper_length181,body_mass3750,sexmale,year2007,index2,speciesAdelie,islandTorgersen,bill_length39.5,bill_depth17.4,flipper_length186,body_mass3800,sexfemale,year2007,index3,speciesAdelie,islandTorgersen,bill_length40.3,bill_depth18.,flipper_length195,body_mass3250,sexfemale,year2007,index4,speciesAdelie,islandTorgersen,bill_length36.7,bill_depth19.3,flipper_length193,body_mass3450,sexfemale,year2007,index5,speciesAdelie,islandTorgersen,bill_length39.3,bill_depth20.6,flipper_length190,body_mass3650,sexmale,year2007}
Visualize Data
Visualize Data
Suggester LLM
Suggester LLM
The code constructs a summary of the dataset, defines a comprehensive list of possible plot types, and creates a detailed system prompt instructing a large language model (LLM) to suggest a diverse set of visualizations tailored to the dataset. It then calls the LLM with sample data and the list of plot types, and prints the LLM’s output for review or debugging. This workflow automates the process of generating insightful and interpretable visualization suggestions for any given dataset
This code prepares a dataset summary and prompt, then uses an LLM to automatically suggest relevant Wolfram Language visualizations for the data.
In[]:=
summary = <| "Columns" -> Keys[ds[[1]]], "Types" -> ds[[1]] // Map[Head], "SampleRows" -> Normal[ds[[1 ;; 3]]]|>;allPlotTypes = { "Histogram", "BoxWhiskerChart", "PieChart", "BarChart", "WordCloud", "ListLinePlot", "ListPlot", "MatrixPlot", "ArrayPlot", "ListContourPlot", "ListDensityPlot", "BubbleChart", "DateListPlot", "SmoothHistogram", "ListPointPlot3D", "ListPlot3D", "BarChart3D", "Histogram3D", "GeoListPlot", "GeoRegionValuePlot", "ParallelAxisPlot", "PairwiseListPlot", "GraphPlot", "TreePlot"};systemPrompt = "You are an expert in data analysis and visualization using the Wolfram Language. Given the following dataset summary: <insert your dataset summary here>\n\n" <> "Suggest a diverse set of visualizations (both basic and advanced) that would be informative for this dataset. For each suggested visualization, provide:\n" <> "- \"PlotType\": The name of the Wolfram Language plot function (e.g., \"Histogram\", \"BarChart\", \"ListPlot\", etc.)\n" <> "- \"Columns\": A list of column names from the dataset to use for this plot (e.g., {\"bill_length\"}, {\"species\", \"body_mass\"}, etc.)\n" <> "- \"Description\": A short, clear explanation for the user describing what this plot shows and how to interpret it.\n\n" <> "Return your answer as a list of Wolfram Associations, one for each visualization, in the following format:\n" <> "{<|\"PlotType\" -> \"Histogram\", \"Columns\" -> {\"bill_length\"}, \"Description\" -> \"This histogram shows the distribution of bill lengths among all penguins in the dataset.\"|>, <|\"PlotType\" -> \"BarChart\", \"Columns\" -> {\"species\"}, \"Description\" -> \"This bar chart compares the number of penguins of each species.\"|>, ...}\n\n" <> "Only include visualizations that are relevant and interpretable for the dataset.";llm = LLMFunction[{systemPrompt, "SampleData: `SampleData`", "AllPlotTypes: `AllPlotTypes`"}];llmOut = llm[<|"SampleData" -> sampleRows, "AllPlotTypes" -> allPlotTypes|>];Print["LLM output: ", llmOut] (* For debugging *)
Prepare Plot Data
Prepare Plot Data
The preparePlotData function takes a dataset, a plot type, and a list of columns, and processes the data into the correct format for visualization. It handles a wide range of plot types—including histograms, box plots, bar charts, word clouds, bubble charts, 3D plots, and more—by grouping, counting, or restructuring the data as needed. This ensures that each plotting function receives data in the form it expects, enabling automated and robust visualization workflows
preparePlotData transforms dataset columns into the specific data structures required by each Wolfram Language plot type.
Show LLM Visualization
Show LLM Visualization
The showLLMVisualization function takes a visualization association (typically suggested by a large language model) and a dataset, extracts the plot type, columns, and description, prepares the data accordingly, and creates the appropriate plot using visualizeData. It then displays the plot together with its description in a neatly formatted column, making it easy to present and interpret LLM-suggested visualizations.
showLLMVisualization generates and displays a plot with a styled description based on a visualization specification from an LLM.
Main Conversational Loop
Main Conversational Loop
The mainConversationalLoop function first summarizes the input dataset, then queries a large language model (LLM) for visualization suggestions tailored to the data. It parses the LLM’s output, attempts to generate each suggested plot using helper functions, and displays the resulting visualizations in a neatly arranged grid. The function also provides feedback on the number of successful plots, enabling an interactive and automated data exploration workflow.
mainConversationalLoop orchestrates the process of summarizing a dataset, obtaining LLM-suggested visualizations, and displaying a grid of generated plots.
Runs The App
Runs The App
CITE THIS NOTEBOOK
CITE THIS NOTEBOOK
InsightCanvas - Automated Data Summary and Visualization
by Arif Jahangir
Wolfram Community, STAFF PICKS, July 9, 2025
https://community.wolfram.com/groups/-/m/t/3496059
by Arif Jahangir
Wolfram Community, STAFF PICKS, July 9, 2025
https://community.wolfram.com/groups/-/m/t/3496059